Prepared by Team Leader: Malay Das Team Member: Animesh Sharma, Shivangi Rai, Jagriti Singh February 2020 Agreement Number: LRP-439-2016/020-00 WFP Country Office Lao PDR Evaluation Manager: Sengarun Budcharern End line Evaluation of USDA Local Regional Procurement project in Nalae District, Luang Namtha Province in Lao PDR [FY 16-19] Report of End line Evaluation Decentralized Evaluation Decentralized evaluation for evidence-based decision making Acknowledgements The NRMC evaluation team wishes to acknowledge the guidance, support, and cooperation received from all the participants in the evaluation. NRMC takes this opportunity to extend sincere thanks to the distinguished Government officials at the Ministries and departments at national, provincial and district levels for their time and precious inputs. The NRMC Evaluation Team expresses its gratitude to the members of RBB and Yumiko Kanemitsu, for sharing their useful insights. Their suggestions have immensely helped in enhancing the design of the evaluation. We would also like to thank the staff of international development agencies which kindly took the time to meet us and give us their views on the Local & Regional Procurement program in Lao PDR. NRMC wishes to sincerely acknowledge the support and guidance received from the staff of the WFP Lao PDR Country Office, especially from Jan Dalbaere, Hakan Tongul, Sengarun Budcharern, Fumitsugu Tosu, Air Sensomphone, Yangxia Lee, Outhai SihaLath as-well-as the WFP staff from Luang Namtha sub-office for assisting with the planning of and facilitating the evaluation mission, and for supplying documentation. We are grateful to the team from Geo-Sys for their partnership with the NRMC evaluation team throughout evaluation, particularly for their untiring efforts for data collection. Last but not the least, the evaluation team wishes to acknowledge the cooperation received from all informants, including school children, school head, teachers, parents, cooks, farmers and VEDC members, during the primary data collection. Disclaimer The opinions expressed in this report are those of the Evaluation Team and do not necessarily reflect those of the World Food Program. Responsibility for the opinions expressed in this report rests solely with the authors. Publication of this document does not imply endorsement by WFP of the opinions expressed. The designation employed and the presentation of material in maps do no imply the expression of any opinion whatsoever on the part of WFP concerning the legal or constitutional status of any country, territory or sea area, or concerning the delimitation of frontiers. USDA Local Regional Procurement Activity Evaluation in Lao PDR Final Report i Table of Contents Acknowledgements List of Acronyms Executive Summary 1. Introduction............................................................................................................................................. 1 1.1. The Subject of the Evaluation .................................................................................................. 1 1.2. Context............................................................................................................................................. 5 1.3. Evaluation Methodology............................................................................................................ 8 2. Evaluation Findings ........................................................................................................................... 12 2.1. Relevance of LRP ........................................................................................................................12 Alignment and Contribution to Government Strategies...................................................................... 12 Coherence with WFP Country Program (2017-2021) ......................................................................... 13 Capacitating Smallholder Farmers and DAFO Officials........................................................................ 13 Providing Diverse and Nutritious Food to SMP ...................................................................................... 15 Gender Equality and Empowerment of Smallholder Farmers.......................................................... 15 2.2. Effectiveness of LRP...................................................................................................................16 Achievement of Outputs and Outcomes of the Intervention ............................................................. 17 Enhanced Access to Food Supply and Voluntary Contributions to SMP....................................... 24 Gender Equality and Empowerment........................................................................................................... 26 Changes in Dietary Diversity Score .............................................................................................................. 27 Replication in Other Districts ......................................................................................................................... 27 2.3. Efficiency of LRP .........................................................................................................................28 Partnership with Government Agencies for Implementation........................................................... 28 Efficiency of Farmer Groups............................................................................................................................ 30 Flexibility and Adaptability of Program..................................................................................................... 30 2.4. Impact of LRP...............................................................................................................................31 School Meal Program ......................................................................................................................................... 31 Unintended Effects of the Program.............................................................................................................. 32 Use of Improved Agriculture Techniques.................................................................................................. 32 Gender and Human Rights Impact................................................................................................................ 33 2.5. Sustainability of LRP .................................................................................................................33 Capacity Building of Farmers, MAF Officials and Other Partners.................................................... 34 USDA Local Regional Procurement Activity Evaluation in Lao PDR Final Report ii Ownership of Community-Driven School Lunch .................................................................................... 34 Sustaining LRP program................................................................................................................................... 35 Replicating the LRP Program.......................................................................................................................... 35 3. Conclusions and Recommendations ............................................................................................ 35 3.1. Overall Assessment/ Conclusions ........................................................................................35 3.2. Good practices and Lessons Learned..................................................................................36 3.3. Recommendations .....................................................................................................................38 Annexures ...................................................................................................................................................... 41 Annex A Map of LRP Intervention Area.............................................................................................................. 41 Annex B Evaluation Mission Schedule................................................................................................................ 42 Annex C Scope of Work for Activity Evaluation.............................................................................................. 46 Annex D Primary Users of Evaluation Report and Stakeholders Interviewed................................... 47 Annex E Broad activities planned under LRP program............................................................................... 48 Annex F Planned Outputs and Beneficiaries.................................................................................................... 50 Annex G Planned Outcomes of USDA LRP-Lao PDR...................................................................................... 51 Annex H Results Framework of WFP-Lao PDR: LRP (FY16)..................................................................... 60 Annex I Budget for WFP-LRP Program............................................................................................................. 61 Annex J Logical Framework................................................................................................................................... 63 Annex K Roles of Key Partners............................................................................................................................... 66 Annex L Brief Report from Scoping Visit........................................................................................................... 67 Annex M Evaluation Matrix...................................................................................................................................... 70 Annex N Approach and Methodology.................................................................................................................. 88 Annex O List of Stakeholders Interviewed........................................................................................................ 98 Annex P Documents gathered................................................................................................................................ 99 Annex Q Mapping of National Priorities and Logical Framework of LRP ..........................................102 Annex R List of Tables for Effectiveness Indicators ....................................................................................109 Annex S Data Collection Tools .............................................................................................................................114 Annex T Map for Province Oudumxay..............................................................................................................180 Annex U Case Studies ...............................................................................................................................................181 Annex V Glimpses from Validation Workshop..............................................................................................193 List of Tables Table 1: Snapshot of Program Subject..................................................................................................................... 3 Table 2: Value of Sales Production under LRP .................................................................................................. 21 Table 3: Comparison of Dietary Diversity Score............................................................................................... 27 Table 4: Recommendations, rationale and proposed actions..................................................................... 38 USDA Local Regional Procurement Activity Evaluation in Lao PDR Final Report iii List of Figures Figure 1: The SMP-LRP linkage................................................................................................................................... 2 Figure 2: Visual thinking for validating results................................................................................................. 10 Figure 3: Vegetable farming using Greenhouse technique .......................................................................... 14 Figure 4: Students having lunch at school........................................................................................................... 15 Figure 5: Number of individuals benefitting directly through LRP.......................................................... 17 Figure 6: Individuals who received short-term agricultural training ..................................................... 20 Figure 7: Percentage of LRP farmers growing different vegetables......................................................... 22 Figure 8: Variety of vegetables grown by LRP supported farmers........................................................... 23 Figure 9: Percentage of LRP farmers cultivating in one or two seasons ................................................ 23 Figure 10: Percentage of LRP farmers implementing best practices....................................................... 24 Figure 11: School lunch provided under LRP.................................................................................................... 25 Figure 12: Greenhouse supported by LRP .......................................................................................................... 33 Figure 13: Technical Approach................................................................................................................................ 89 USDA Local Regional Procurement Activity Evaluation in Lao PDR Final Report iv List of Acronyms ATSC Agriculture Technical Service Centre CBTs Cash-based transfers CO Country Office CP Country Program CPE Country Program Evaluation CSP Country Strategic Plan CRF WFP Corporate Results Framework CRS Catholic Relief Services DAC Development Assistance Committee (of the OECD) DAFO District Agriculture and Forestry Office DDS Dietary Diversity Score DEQAS Decentralized Evaluation Quality Assurance System (of WFP) DESB District Education and Sports Bureau DFAT Department of Foreign Affairs and Trade (Australia) DHO District Health Office DP Development Partner DTEAP Department of Technical Extension and Agro-Processing EB Executive Board (of WFP) ECE Early Childhood Education EM Evaluation Manager EQ Evaluation Question EQAS Evaluation quality assurance system (of WFP) ER Evaluation Report ESDF Education Sector Development Framework ESDP Education Sector Development Plan ET Evaluation Team FAO Food and Agriculture Organization FFA Food assistance For Assets FFE Food for Education FGD Focus-group discussions FLAs Field level agreements GDI Gender Development Index EEW Gender Equality and Empowerment of Women GoL Government of Lao PDR HDI Human Development Index USDA Local Regional Procurement Activity Evaluation in Lao PDR Final Report v HDR Human Development Report HGSF Home Grown School Feeding HHs Households HQ Headquarters ICT Information and Communication Technology IDI In-depth interview IEC Inclusive Education Centre IFPRI International Food Policy Research Institute IGDs Individual and groups discussions INGO International non-governmental organization IR Inception Report JICA The Japan International Cooperation Agency KIIs Key informant interviews LDC Least developed Country LRP Local and Regional Procurement LWU Lao Women’s Union LWF Lutheran World Federation LYU Lao Youth Union MA Monitoring Assistants MAF Ministry of Agriculture and Forestry MCQ Multiple Choice Questions MDG Millennium Development Goal MoES Ministry of Education and Sports MoH Ministry of Health MOUs Memorandum of Understanding MSC Most Significant Change MT Metric Ton NFS Nutrition and Food Security NFR Note for the record NNS National Nutrition Strategy NNSPA National Nutrition Strategy and Plan of Action NRMC NR Management Consultants India NSEDP National Socio-Economic Development Plan NSMP National School Meals Program NTFPs Non-timber forest products OECD Organization for Economic Cooperation and Development OEV WFP Office of Evaluation USDA Local Regional Procurement Activity Evaluation in Lao PDR Final Report vi PAFO Provincial Agriculture and Forestry Office PDR People’s Democratic Republic – Lao PHO Provincial Health Office PESS Provincial Education and Sports Services RBB Regional Bureau Bangkok RRB Regional Rural Banks SDG Sustainable Development Goal SITREP Country Situation Report SMAP School Meals Action Plan SMP School Meals Program SPR Standard Project Report TOC Theory of Change TOR Terms of Reference UNDP United Nations Development Program UNEG United Nations Evaluation Group UNESCO United Nations Educational, Scientific and Cultural Organization UNICEF United Nations Children's Fund UNFPA United Nations Population Fund UNCT’s United Nations Country Team UNHCR United Nations High Commissioner for Refugees USDA United States Department of Agriculture VEDC Village Education Development Committee WASH Water Sanitation and Hygiene WB World Bank WFP World Food Program USDA Local Regional Procurement Activity Evaluation in Lao PDR Final Report i Executive Summary Introduction 1. The activity evaluation for the United States Department of Agriculture (USDA) Local Regional Procurement (LRP) program in Nalae district of Lao People’s Democratic Republic (Lao PDR), commissioned by the WFP country office of Lao PDR (WFP CO), occurred over June-November 2019. The evaluation covered the LRP program period from January 2017 till June 2019. 2. The primary stakeholders and users of this evaluation include (1) WFP CO, (2) USDA, (3) the Regional Bureau Bangkok, (4) WFP Headquarter, (5) Office of Evaluation, and (6) Ministry of Agriculture and Forestry (MAF) and Ministry of Education and Sports (MoES) of Lao PDR. 3. Lao PDR has prioritised meeting the Sustainable Development Goal (SDG) 2 to ‘end hunger, achieve food security and improved nutrition and promote sustainable agriculture’. The USDA McGovern-Dole School Meals Program (SMP) is a step towards this, by way of serving meals in schools. Under SMP 2017-21, the community is expected to contribute vegetables for the meals. The LRP program was conceptualised to ensure the sustainability of SMP; it aimed at supporting smallholder farmers to produce vegetables and sell them to schools. The surplus was to be consumed at home and sold in the open market, thus helping augmentation of the household income. The LRP program was piloted across 47 villages of Nalae district. The objectives of LRP included: (1) sustained supply of fresh food for school lunches by providing cash1 support to schools; (2) increased intake of vegetables by students; (3) continuous application of improved agricultural techniques; (4) increased ownership of the school lunch by the communities; and (5) promotion of equal access to agricultural extension for male and female farmers. 4. Purpose of the Evaluation: The LRP program design, as per the USDA requirement, required an activity evaluation to critically evaluate its implementation and performance to generate recommendations that are replicable in other geographic areas. 5. Objectives of the evaluation a. Accountability: This evaluation assessed the USDA LRP performance and results of the implementation. b. Learning: The evaluation determined the reasons why certain results occurred, or not, to derive good practices and lessons learnt, providing evidence-based findings to inform future operational and strategic decision-making. Methodology 6. The evaluation used the OECD-DAC criteria to assess the relevance, effectiveness, efficiency, sustainability and impact of the LRP program through the lens of equality and inclusion of both genders and vulnerable groups. It provided an evidence-based assessment of the activities and outcomes using a Logic model. 7. The evaluation adopted a quasi-experimental evaluation design, which included the selection of LRP-supported (intervention) and non-supported (control) villages. A mixed-method approach was deployed to answer every evaluation criterion using key informant interviews (KIIs) and focus group discussions (FGDs). The evaluation design also included Most Significant Change (MSC), which involved identification and documentation of seven case studies in intervention villages, highlighting personal accounts of change of farmers who participated in LRP. Gender Equality and Empowerment of Women (GEEW) was mainstreamed in the evaluation by ensuring a gender￾balanced team, collecting information for boys, girls, men and women, and undertaking a gender￾disaggregated analysis. 8. The baseline study involved systematic random sampling for a selection of villages across lowland, upland and mountain regions.2 The same set of 15 intervention (6 model villages3; 9 non-model villages) and five control villages were covered during the end line evaluation. At the village level, FGDs were conducted with parents, farmers and Village Education and Development Committee (VEDC) members, while In-depth Interviews (IDIs) were carried out with schoolchildren, teachers 1 800 kips per student per day 2 Baseline Study – WFP Local Regional Procurement Program 3 10 LRP supported villages had expressed interest in cultivating vegetables with a commercial outlook. These villages were supported with intense interventions in year 2 and were termed as ‘model’ villages USDA Local Regional Procurement Activity Evaluation in Lao PDR Final Report ii and cooks. IDIs were also conducted with officials of the Government of Lao PDR (GoL) and other stakeholders. 9. Validation Exercise: To validate the evaluation findings and to aid cross-learning among different stakeholders, validation workshops were conducted in Nalae district and Vientiane. The workshops triggered discussions and enabled participants to generate key recommendations for sustaining the current program and designing and implementing a similar program in future. 10. The limitations of this study include: (1) the inability of the evaluation design to allow attribution of any changes to the program, (2) the inability of children in standards I-II to comprehend and respond to the questions, and (3) unavailability of program farmers in certain villages. Key Findings Relevance 11. The LRP program was designed to provide the means to the community to move towards self￾sufficiency in supplying vegetables for school meals, improving the dietary diversity of the community, and augmenting the income of smallholder farmers. With piloting of LRP in a disadvantaged region, the inclusive nature of the program was demonstrated. 12. The program partnered with the government for its implementation. District Agriculture and Forest Office (DAFO) officials attended training workshops, undertook exposure visits in Oudomxay province in September 2018, and in turn conducted training sessions for smallholder farmers from the intervention villages. 13. The LRP program was in line with the priorities stated in the country’s Agriculture Development Strategy to 2025 document, such as increasing multiple crop agricultural practices and diversification of food products to achieve food security. The program was also aligned with the National Nutrition Strategy to 2025, underpinned by inter-sectoral coordination. 14. LRP’s logical framework was in complete sync with three of the four strategic outcomes (SO1, SO3 and SO4) of WFP Country Strategic Plan: children in remote areas have sustained access to food (SO1), building sustainable livelihood opportunities for higher resilience to climatic shocks (SO3), and capacity building to strengthen institutions of local governance for improved service delivery (SO4). 15. In terms of gender equality and human rights, the universal coverage of the program ensured no girl or boy child was left out of the scheme of school meals, and both women and men smallholder farmers from intervention villages were trained on technical aspects related to soil improvement, multi-cycle cropping, etc. and provided with seeds and manual tools. Effectiveness 16. Overall, the program aimed at benefitting 5000 individuals (4500 students and 500 farmers) directly. The actual achievement increased from a little less than 80 per cent in year 1 (3936 individuals) to almost 100 per cent in year 2 (4973 individuals). About 48 per cent of the beneficiaries were women. The program had indirectly benefitted 22,913 persons including 11,227 female beneficiaries. 17. On average, a total of 10 farmers4 per village were trained on nutrition-sensitive agriculture, reaching a total of 460 (265 males; 195 females) and 474 farmers (200 males; 274 females) in semesters I and II respectively. Interactions with these farmers revealed that the training sessions have increased their knowledge levels around agriculture. 18. In terms of inputs, the program provided 11 types of seeds and manual agricultural tools such as sickles, manual water sprinklers and water buckets in year 1 for carrying out cultivation. Year 2 saw the provision of greenhouse plastic sheets, water pumps and piped water connections for farmers cultivating vegetables across 10 model villages. 19. Trained farmer groups in 19 villages, formed partnerships with schools in year 1 and began selling vegetables for school meals. In the second year, the program changed its strategy and worked with only 10 model villages. The program created and strengthened farmer groups, enabling them to focus on a diversified set of vegetables all-year-round using greenhouse techniques. 4 As per the WFP program team, a total of 10 farmer households were selected in every village. One man and one-woman farmer were selected from each such household. However, this approach is not reflected in the targets for the number of farmers trained. The program had a target of training 500 farmers, effectively meaning 10 farmers per village. USDA Local Regional Procurement Activity Evaluation in Lao PDR Final Report iii 20. The LRP program helped increase the variety of vegetables cultivated by farmers in the intervention villages from four varieties before the start of LRP to 20 varieties by year 2. 21. About 39 per cent of all trained farmers (184 in number) in year 1 managed to sell their vegetables, achieving more than 70 per cent of the total sales (in value) target. As the number of farmers in the market increased, the demand for vegetables reduced, resulting in lower prices, which affected the value of sales. 22. Availability of a variety of vegetables also impacted dietary diversity, with the Dietary Diversity Score (DDS) increasing from 4 to 8 in the lowland region, 7 to 8 in the mountainous region, and dropping marginally from 8 to 7 in the upland region. Efficiency 23. Leveraging farmer groups, which were formed with help from VEDCs, was an efficient strategy as it enabled an exchange of knowledge, seeds and tools, as well as planning around the production of different types of vegetables. 24. MAF officials have been trained, and along with WFP Monitoring Assistants, are currently providing technical support. Post exit of WFP, the officials will continue to help farmers practise improved agricultural techniques. 25. WFP designed a specific monitoring tool in KOBO (mobile/tablet-based monitoring data collection application) to track the project implementation process and the planned outputs. However, it was not regularly used during the two years of intervention. 26. After year 1, the LRP farmers did not experience a substantial increase in incomes as a result of cultivating vegetables and hence showed lukewarm interest. The geographical scope of the program, therefore, was limited to 10 villages in year 2, resulting in unutilised funds (about 37 per cent), which was utilised in additional 29 LRP villages by entering into a partnership with the Lutheran World Federation. Sustainability 27. Linking farmers with the school resulted in ownership among community members towards school meals. 28. Lack of market access and no substantial increase in income might affect sustained program participation in the future. Impact 29. Given that the activities for the two-year program only ended in October 2019, it was too early to capture and assess the true impact of the program. 30. The LRP program successfully built capacities of small landholder farmers for growing nutritious vegetables. In many cases, it was observed that the farmers contributed vegetables to the schools free of cost. In such instances, the 800 kips was used to procure meat for school meals. 31. In 14 out of 15 sampled intervention schools, the school meals continued uninterrupted despite the absence of food supply under USDA-SMP for the Sep’19-Mar’20 semester. 32. Discussions with schoolchildren indicated that on an average they consumed non-vegetarian meals three times per week. Improvement in the ability to concentrate in class and learning outcomes post SMP and LRP was reported by officials, teachers and parents. 33. As an unintended impact of the program, transfer of technical knowledge from the beneficiary farmers to non-beneficiary ones was reported, which resulted in the cultivation of similar vegetables by most farmers. Conclusions and Recommendations 34. Overall, the WFP LRP program has been able to achieve its intended outcomes for year 1 and has been flexible enough to adopt changes as per the community needs for year 2. Conclusions are drawn in terms of good practices, lessons learnt and recommendations are presented below. 35. The program design enabled the community to move towards self-sufficiency in supplying vegetables and ensuring access to nutritious food for children. The program identified and tackled both demand- and supply-side issues. 36. The collectivisation of farmers at the village level resulted in the transmission of technical knowledge and the sharing of seeds and tools. 37. Lack of access to markets made it difficult for the farmers to sell their produce, resulting in only a nominal increase in income levels. USDA Local Regional Procurement Activity Evaluation in Lao PDR Final Report iv 38. The program lacked provisions to ensure women’s participation in leadership and decision￾making roles. Good practice 39. The program adopted the approach of collaborating with multiple stakeholders. Its success can be primarily attributed to the fact that the demand (community) and supply (government) sides were brought together under the program. 40. Working with the farmer groups helped in building a sense of camaraderie among all farmers, enabling them to share knowledge and resources, as well as plan the farming of vegetables. Capacity building of LRP farmers resulted in increased technical knowledge, which was also transmitted among farmers from the control group. Lessons Learned 41. A needs assessment study is essential during the design stage as it helps understand the needs and aspirations of each region and accordingly customise the intervention. 42. Any such program in the future must consider (1) educating farmers about the demand and supply aspects and (2) bringing all of them on one platform to plan the potential vegetable production, keeping in mind the demand and supply constraints. While it is understood that sometimes it is imperative to make alterations to the original program design, the changes must be in sync with the initial idea of the program. A strong monitoring system provides a ready reference to the monitoring data and enables (i) quick checks to assess the direction of program movement and (ii) quick turnarounds by the program as a response to issues identified. 43. The key recommendations are presented in the table below: Sl. No. Recommendations Proposed actions MAF & DAFO 1. Providing technical support for small land farming There is a need to organise training on aspects such as regenerating seeds or building resilience to climate change. Creating a yearly calendar for such training and follow-up sessions would ensure high participation from farmers. 2. Providing farmer groups with technology for self￾monitoring MAF should create a self-monitoring system for farmer groups, encouraging them to record and share details about the types and quantities of vegetables cultivated with DAFO. 3. Formalisation of farmer groups To ensure the sustainability of farmer groups, MAF must formalise them by creating formal structures, ensuring regular meetings, selecting position holders, and delineating their roles and responsibilities. 4. Dashboard for DAFO to analyse monitoring data and take corrective actions There is a need to create a strong monitoring system, with a dashboard for DAFO officials, enabling them to identify issues and make timely corrections. Farmers 5. Monitoring of the vegetables grown and quantity produced MAF should create a self-monitoring system for farmer groups, encouraging them to record details about the types and quantities of vegetables cultivated. Access to real-time data would enable DAFO to carry out immediate corrective actions. WFP 6. Technological support for program monitoring Given WFP’s experience of the LRP program, it can provide technical support to MAF and DAFO in creating a monitoring system and linking it with the dashboard to capture critical information on a real-time basis. 7. Need for a feasibility study for market accessibility and community needs WFP should plan a needs assessment study before designing a similar program. The needs assessment study would capture first-hand information on variations that exist across regions, social groups, gender, livelihoods, skills, etc. 8. Ensuring more meaningful engagement with women Both women and men should be encouraged to volunteer for SMP activities, which would help in reducing women’s workload. At the same time, it is essential to ensure that women farmers are necessarily included in exposure visits and provided with opportunities to lead farmer groups. USDA Local Regional Procurement Activity Evaluation in Lao PDR Final Report 1 1. Introduction Overview of the Evaluation 1 The activity evaluation for the United States Department of Agriculture (USDA) supported Local Regional Procurement (LRP) program5 at Nalae district, of Luang Namtha province (details in Annex A), in Lao People’s Democratic Republic (Lao PDR), commissioned by the WFP country office (WFP CO) of Lao PDR, was carried out during June-November 2019 (mission schedule presented in Annex B). The evaluation covers the period from April 2017 till June 2019. As per the USDA requirement, the LRP program design included an activity evaluation to critically evaluate its implementation and performance to generate recommendations that are replicable in other geographic areas. 2 Specific objectives: Underpinned by the dual and mutually reinforcing objectives of accountability and learning, this evaluation had the following specific objectives: assess and report on (1) the performance of the implementation, (2) reasons for success and failure of activities, (3) relevance and effectiveness of capacity strengthening and linking to the School Meals Program (SMP), and (4) contribution towards meeting the food security and nutrition needs of women, men, girls and boys. 3 Scope of the evaluation (details in Annex C): The evaluation of the LRP program involved three key activities: (1) review of relevant documents including project documents, internal/external administrative records and primary data, (2) visiting LRP project sites as￾well-as non-project (control) sites within Nalae district to conduct primary data collection, and (3) interacting with representatives and staff members of governmental implementing partners. While the geographic scope for LRP program included 47 villages of Nalae district within Luang Namtha province, the end line evaluation covered 15 out of these intervention villages, along with five control villages to facilitate comparison. 4 Stakeholders in the evaluation: Several stakeholders have an interest in the results of the evaluation. They include (1) WFP CO, (2) USDA, (3) Regional Bureau Bangkok (RBB), (4) WFP Headquarters Office of Evaluation (OEV), and (5) the Ministry of Agriculture and Forestry (MAF) and Ministry of Education and Sports (MoES), Government of Lao PDR (GoL), and their respective departments at provincial and district levels. 5 Primary users of the report (details in Annex D): The primary users of this evaluation will be: (1) WFP CO for decision-making, notably related to program design and implementation; (2) USDA as the funder of the project and the evaluation; (3) RBB to provide strategic guidance, program support and oversight; (4) WFP HQ for wider organisational learning and accountability; (5) OEV for evaluation syntheses; and (6) MAF and MoES, which will utilise the evaluation findings as inputs for its strategy post handing over of the schools. 1.1. The Subject of the Evaluation 6 Under the SMP 2017-21, supported by USDA McGovern-Dole, rice, lentils and fortified oil were provided to intervention schools, and the communities were encouraged to voluntarily contribute vegetables and fuelwood for school meals. According to the end-line evaluation of SMP 2014-16, while the first component – the provision of food items for school meals – worked well, voluntary contributions from the communities were rare and irregular. As a result, there was a felt need to accentuate the importance of vegetables in school meals by encouraging and facilitating communities to produce different kinds of vegetables through agricultural extension, and ensuring a sustained supply of vegetables to schools. 7 This led to the conceptualisation of the Local Regional Procurement (LRP) program, which was implemented across 47 villages of Nalae district in Luang Namtha province as a pilot program (results framework presented in Annex H). The activities under the LRP program were envisioned to supplement SMP and hence were implemented only in schools receiving 5 USDA Local Regional Project Grant LRP-439-2016/02000 USDA Local Regional Procurement Activity Evaluation in Lao PDR Final Report 2 benefits under SMP. LRP’s key strategic objective (SO1) was to ensure improved effectiveness of food assistance through local and regional procurement for school meals as well as for the community, including parents and farmers (logical framework presented in Annex J). The LRP program provided cash support of 800 kips per student per day to schools for purchasing vegetables from the LRP supported farmers. 8 The key activities supported by the program included: (1) training for VEDC members; (2) training for farmers; (3) training on cooking and management with support from the Lao Women Union (LWU); and (4) partner monitoring and exchange visits for farmers (details of activities presented in Annex E). The program also envisaged close coordination between MAF and MoES with MAF providing support for the preservation of seeds for future crop cycles and plantings, and MoES incorporating the crops planted within the community into the Nutrition and School Agriculture curriculum. The relation between SMP and LRP program is presented in Figure 1. 9 Broadly speaking, the LRP program was based on two pillars: (1) supporting school meals for children by way of sustained supply of vegetables and (2) increasing household income by strengthening sustainable farming and establishing requisite commercial linkages. While the first pillar was largely concerned with encouraging farmers to cultivate vegetables and supply a portion of the farm produce to schools for meals, the second pillar aimed at increasing household income by linking farmers with the market for enhancing commercial activities. 10 Inputs for the program included technical training for cultivating vegetables and provision of vegetable seeds and essential manual tools such as water buckets and sprinklers. The program strategy, however, saw a major shift in the second year. While all 47 villages continued to grow vegetables and contribute to the school meals, only 106 of them expressed interest in cultivating vegetables with a commercial outlook. As a result, these 10 villages – termed ‘model’ villages – experienced intense interventions in year 2, directed towards enhancing the commercial aspects of the cultivation of vegetables. Interventions for the model villages included training farmers to process raw vegetables,7 and provision of greenhouse plastic sheets, water pumps and piped water connections to increase productivity and crop cycles. The remaining 37 (non-model) villages received seeds and manual agricultural tools in year 1, apart from technical training related to the cultivation of vegetables. No additional support was provided to these villages from the second year onwards. As a result, farmers in these villages were not able to sell their produce in markets, though they continued to cultivate vegetables and contribute a portion of these towards school meals. 11 The savings in the program budget as a result of the reduced scope of work was subsequently used to carry out an additional set of activities with support from the Lutheran World Federation (LWF). The component involved provision of (1) cash support to weavers for 6 6 List of ‘model’ villages: Hatlom, Lao, Namhaeng, Omh, Phavy, Longhaen, Hatnalaneg, Phoupad and Sakaen 7 Increasing the shelf life of vegetables by way of boiling and drying them. Figure 1: The SMP-LRP linkage USDA Local Regional Procurement Activity Evaluation in Lao PDR Final Report 3 purchasing weaving tools, in 12 villages, (2) manual tractors for tilling the land, in 10 villages, (3) domesticated animals like goat, sheep, pigs, cow and buffalo, in 15 villages, and (4) big cement stoves for schools, in 29 schools. The intervention with LWF was carried out for four months between July and October 2019. It is important to note that support provided under this component was not aligned to the original LRP program activities even though it intended to enhance the income of the beneficiaries. The component with LWF was largely carried out with farmers having enough resources and the ability to generate incremental income from the provision of assets. Given that the primary data collection for the evaluation was carried out in September-October 2019, it was not possible to observe and measure the effects generated as a result of involving LWF in the program during July–October 2019. Table 1: Snapshot of Program Subject Sl. No. Subjects USDA LRP 1 WFP contribution  Cash-based transfer for primary and pre-primary students in 47 schools  Seeds and agricultural tools for local farmers  Cooking utensils for schools  Agriculture education and training  Community exchange visits to best-performing communities 2 Main activities WFP assistance from April 2017 up to February 2019 consisted of:  Training A, a 1-day training for VEDC members  Training B, a 1-day training for farmers  Training C, a 2-day training on agriculture for farmers  Training D, 2/3-day training on expert assistance  Training E, 1-day training on cooking in community and cash management  A cash transfer to schools for the purchase of fresh food for the school lunch  Partner monitoring and exchange visits 3 Number of villages 49 villages8 4 Type of beneficiaries in Nalae  Women and men smallholder farmers in 49 villages who contribute towards 47 schools.  Primary and pre-primary students receiving school lunch through cash transfer (for buying fresh food) to their respective schools 4 Number of beneficiaries  500 smallholder farmers trained (equivalent to 12 per cent of the total 4507 smallholder farmers in Nalae)  3753 students or 100% (of which 1895 were girls and 1858 boys) of primary and pre-primary schools in 47 schools 12 Program geography: WFP CO, together with MAF and MoES, implemented the USDA LRP program in Nalae district of Luang Namtha across 47 targeted villages, covering 47 schools, between January 2017 and June 2019 (map of intervention area presented in Annex F). The program was planned for 49 villages; however, two villages were excluded from the second semester since they were relocated due to construction of a dam. 13 Program timeline: GoL has been receiving USDA support for SMP since 2008. The current SMP (2017-21) is being implemented in 31 select districts across eight provinces9 characterised by poverty, malnutrition and low literacy rates. The agreement between WFP CO of Lao PDR and USDA for the LRP program was signed in January 2017. Project implementation started in April 2017 and closed on 30 June 2019. 14 Planned outputs and beneficiaries: The LRP program was implemented across 47 villages and 47 schools of Nalae district, covering almost 500 women and men smallholder farmers and more than 3500 schoolchildren. Essentially, the program targeted 12 per cent of the smallholder farmers and 100 per cent of the children in these 47 villages/schools. A snapshot of all the targeted beneficiaries, as detailed in the ToR, has been provided in Annex F. 8 Annex 7 lists the villages in Nalae district covered by the LRP project; 49 villages were initially included, but eventually only 47 schools were covered under this project. 9 Attapeu, Khammuane, Saravane and Sekong provinces located in the south, and Luang Namtha, Luang Prabang, Oudomxay and Phongsaly provinces in the north. USDA Local Regional Procurement Activity Evaluation in Lao PDR Final Report 4 15 Planned outcomes: The strategic outcome of the LRP program was to achieve ‘Improved Effectiveness of Food Assistance through Local & Regional Procurement’. The three outcomes targeted by the program were: (i) improved cost-effectiveness of food assistance, (ii) improved timeliness of food assistance, and (iii) improved utilisation of nutritious and culturally acceptable food that meets quality standards. Details of the planned outcomes have been provided in Annex G. The performance indicators and the results framework provided in the evaluation ToR document (Annex H). For each of the outcomes, output indicators and activities have been listed in the planned outcomes matrix. The targeted and achieved results as per the semi-annual report have also been mapped for each outcome. 16 Program financing: The program was initially envisaged to cover two districts of Luang Namtha province (Nalae and Vieng Phoukha) with a proposed budget of about USD 1.9 million. However, USDA allocated a little below USD 1 million as financial assistance through LRP 439-2016/020-00 for FY2017/2018. As a result, LRP was implemented only in Nalae district. A break-up of the activity-wise budget (for both districts) is provided in Annex I. 17 Logical framework: The USDA LRP project’s strategic objective was aligned to and drawn from WFP Lao PDR’s SMP, with LRP SO1 focused on improved effectiveness of food assistance. The activities under the LRP program were directed towards achieving the outcomes stated in the logical framework. A table highlighting the outcomes, outputs and activities is presented in Annex J. The logical framework was comprised of outcomes and foundational results. The foundational results focused on building a conducive environment for the sustainability of the program, including capacity building of the government and other stakeholders. The three outcomes took care of the supply and demand aspects. From the supply side, they ensured improved cost-effectiveness and availability as well as the quality of food. As for the demand aspect, the program focused on improved utilisation of nutritious food by establishing market linkages and building knowledge among stakeholders about the consumption of nutritious food. 18 Partners: LRP in Nalae district was carried out in partnership with different government departments and local partners. Details of the roles of key partners mentioned below are presented in Annex K. a. Government partners: Department of Technical Extension and Agro-Processing (DTEAP) under Ministry of Agriculture and Forestry (MAF), Provincial Agriculture and Forestry Office (PAFO), District Agriculture and Forestry Office (DAFO), Ministry of Education and Sports (MoES), Provincial Education and Sports Services (PESS), and District Education and Sports Bureau (DESB) b. Others: Village Education and Development Committees (VEDCs) 19 Gender dimensions of the intervention: Gender Equality and Empowerment of Women (GEEW) and accountability to affected populations are part of the guiding principles of WFP’s action to achieve zero hunger and empowerment of women and other vulnerable groups. The evaluation is guided by WFP’s latest Gender Policy 2015-20. GEEW formed a key aspect of the LRP program and had been mainstreamed in the program design through its focus on one of the most disadvantaged regions of the country. While the program at the broader level targeted smallholder farmers in the region, the very nature of the community and the secondary status of women therein ensured that women formed a significant proportion of program beneficiaries. This can be seen in the evaluation questions, presented in Annex M, that address the influence of the program in the gender context as also the gender-specific impacts of the program. 20 WFP is committed to the 2030 Agenda’s global call to action and ensuring the underlying principle of ‘no-one left behind’. The LRP program is underpinned by the same principle and targets the smallholder farmers, including female farmers, in a remote area of the country. Through its support on improved farming techniques, it sought to help the farmers to build farming resilience against climate change and enable to continue supporting the National School Meals Program (NSMP) through the sale of vegetables. The inclusion of female USDA Local Regional Procurement Activity Evaluation in Lao PDR Final Report 5 farmers in LRP was directed towards empowering them to decide how to use their land through opportunities for higher earnings and a reliable source of income. 1.2. Context 21 Poverty, food and nutrition security: Lao PDR is one of the fastest-growing economies in East Asia and the Pacific with a Gross National Income (GNI) per capita of USD 6317 in 2018. However, the Human Development Index (HDI) 201910 ranks the country at 140 out of 189 countries. The Human Development Report 2019 designated 23.1 per cent of the population as multi-dimensionally poor; an additional 21.2 per cent live near multidimensional poverty. Nalae is a remote district inhabited by the ethnic Khmu community in the highland areas of Luang Namtha province, where around a quarter (28 per cent) of the population lives below the poverty line, which is higher than the national average. 22 According to the 2015 report of the International Food Policy Research Institute (IFPRI), the Global Hunger Index rates hunger levels for the country as ‘serious’ with Laos ranked 76 out of 104 nations.11 As regards its nutritional status, the country faces a huge challenge of stunting, malnutrition, anaemia and Vitamin A deficiency, as almost one-fifth of the population consumes less than the minimum dietary energy requirements.12 Currently, 21 per cent of children are underweight, while 33 per cent of children are stunted and wasting stands at 9 per cent. Stunting rates in Nalae were higher (39.5 per cent) than the national average. The global nutrition report for Laos13 indicates a difference in stunting and wasting among the under-5 boys and girls.14 While wasting was prevalent among 5.8 per cent girls and 6.9 per cent boys, stunting was prevalent among 42.6 per cent girls and 45.7 per cent boys. Also, with 48.6 per cent under-5 children stunted in the rural areas; the situation is quite grim in comparison to urban areas (27.4 per cent). 23 Micronutrient deficiencies also affect large parts of the population with IFPRI 2014 reporting the prevalence of anaemia in school-aged children as ‘severe’ and anaemia in pregnant and lactating women (PLW) at 45.3 per cent.15 According to the global nutrition report for Laos, while 29.2 per cent girls in the 5-19 age group were underweight in 2016, the corresponding figure for boys was 35.7 per cent. 24 Trends related to SDG 2 and SDG 17: Poverty is one of the root causes of malnutrition and hunger in the country. Therefore, Laos has been focusing on meeting the Sustainable Development Goal (SDG) 2 – ‘End hunger, achieve food security and improved nutrition and promote sustainable agriculture’ – through concentrated efforts and changes in policies. 25 The key outcome areas identified in the context of Lao PDR to meet SDG 2 include: (1) sustainable food production, improved agricultural productivity and resilient agricultural practices; (2) access for all to safe, nutritious and sufficient food all year round; (3) improved nutrition of vulnerable groups; and (4) improved management of genetic diversity.16 WFP is supporting the government in achieving SDG 2 through its multiple programs across the country. LRP is one such program that focuses on the nutrition and food security of vulnerable populations residing in remote locations. 26 WFP is also working towards achieving SDG 17 – ‘Strengthen the means of implementation and revitalize the Global Partnership for Sustainable Development’17 – by adopting the approach of building partnerships to work towards common goals. WFP partners with different departments of GoL including their offices at the central, provincial and district levels and other multilateral organisations for the implementation of its programs. Under the 10 http://hdr.undp.org/sites/all/themes/hdr_theme/country-notes/LAO.pdf 11 Ministry of Health 2013 12 https://docs.wfp.org/api/documents/WFP-0000023967/download/ 13 Data available for 2011 14 https://globalnutritionreport.org/media/profiles/v1.9.7/pdfs/lao-peoples-democratic-republic.pdf 15 Ministry of Health, Lao Statistics Bureau, UNICEF and WFP, 2015 16 https://laos.opendevelopmentmekong.net/topics/sdg-2-zero-hunger/ 17 https://sustainabledevelopment.un.org/sdg17 USDA Local Regional Procurement Activity Evaluation in Lao PDR Final Report 6 LRP program, the focus has been on capacity building of implementing partners, including the government and community organisations, and coordination among them for successful implementation of the program, which is in line with SDG 17. 27 Health: The under-5 mortality rate (U5MR) in Lao PDR was 67 in 2015. Although there has been a 59 per cent decline in U5MR from 1990, it has fallen short of the Millennium Development Goal (MDG) target for child mortality of 54. According to HDR 2016, poor nutrition causes 45 per cent of the deaths among children under the age of 5 and also leads to stunting and delays in physical development. 28 Education: While there has been a significant improvement in the status of children’s education in Lao PDR in recent years, females continue to lag behind males. The youth literacy rate amongst females was around 87 per cent, compared to 93 per cent among males (HDR 2016). The difference in literacy is starker among women from ethnic groups; close to 70 per cent of such women were illiterate and suffered further isolation given that few of them spoke the national language.18 The girl/boy ratio in schools in Nalae district which was at 0.98 for primary education, fell to 0.83 in secondary education and subsequently to 0.69 in upper secondary, indicating higher dropouts among girls.19 29 Agriculture: Lao PDR largely depends on agriculture and farming. However, smaller landholdings, absence of secure land tenures, and limited area under irrigation have led to low domestic food production and availability. Almost 90 per cent of the country’s farmers cultivate rice. This has resulted in a rice-dominated diet that is deficient in proteins, fats and micronutrients, relative to WHO-recommended levels, giving rise to stunting, wasting and other related problems. Due to its topography, Nalae district has been at high risk of natural disasters, such as heavy rainfall and landslides. The households most vulnerable to food insecurity and climatic shocks were those in remote areas with little access to basic infrastructure, those with low engagement in fishing and hunting or unskilled labourers, those practising upland farming on small slopes, women and men with small farmlands, and those without kitchen gardens.20 30 Government strategy, policies and programs: GoL aims to move from LDC status to that of a middle-income country by 2020. Through the 8th National Socio-Economic Development Plan (NSEDP) 2016-2020 and other policy instruments, the government is striving for sustainable economic growth and equitable social development. NSEDP includes sectoral plans of various departments including the School Meals Action Plan (SMAP) 2016-2020. Complementing this plan is the Agriculture Development Strategy 2025, through which GoL intends to combat malnutrition by promoting dietary diversity. This was drafted to achieve national food security, providing seed and technical assistance to increase production and quality of products, and ending shift cultivation practices.21 The National Nutrition Strategy to 2025 and Plan of Action 2016-2020 (NNSPA) aims at promoting equality in gender roles, emphasising women’s access to health services, nutrition and food security information, and food 31 Towards achieving universal access to primary education, GoL has made it free and compulsory. The Education Sector Development Plan (ESDP) 2016-2020 stresses the need to maintain and expand school feeding programs to encourage disadvantaged children (ethnic communities, children with disabilities, those in remote and impoverished circumstances) in lower primary grades to remain in school. In May 2014, GoL adopted the Policy on Promoting School Lunch, which laid the foundation of a nation-wide approach of offering school lunches as an incentive for children in primary school to attend school. 32 Gender dimensions: Despite playing a significant role in agricultural activities and contributing to economic earnings, women’s contribution still remains undervalued and 18 Investing in Rural People in the Lao People’s Democratic Republic, IFAD, 2014 19 Ibid. 20 http://www.la.one.un.org/sdgs/sdg-2-zero-hunger 21 https://theredddesk.org/sites/default/files/strategy_for_agricultural_development_2011_to_2020_1.pdf USDA Local Regional Procurement Activity Evaluation in Lao PDR Final Report 7 vulnerable to climatic and social shocks. With a Gender Development Index (GDI)22 value of 0.896, Lao PDR ranked 141 out of 188 countries in 2015.23 In 2016, however, Lao PDR demonstrated advancements concerning GDI, with the GDI value rising to 0.924.24 33 About GEEW, Lao’s Gender Inequality Index ranked 106 out of 159 countries in 2015. In 2016, the United Nations confirmed that Laos has one of the highest rates of Child, Early, and Forced Marriages (CEFM) in the region. One-third of women marry before age 18, while one￾tenth marry before age 15. Lao PDR is more rural in character than any other country in South East Asia. More than three-quarters of the total population live in rural areas and depend on agriculture and natural resources for survival. Geographical isolation fosters a persistent cultural environment effectively contributing to the continuation of CEFM. A UNPFA report noted that young girls growing up in isolated minority communities that were not integrated into a wider society saw marriage as their only option, partly because they were not aware of other options, and could not speak Lao-Thai, the national language, to effectively communicate with people outside of their isolated community. This shows the important linkages between SDGs 2, 425 and 526. 34 The grim situation of women and girls is aggravated by cultural beliefs that the role of a woman is to be a wife and a mother, and as a result, parents lacked the motivation to invest in educating their daughters and preparing them for paid work (HDR 2016). Further, formal educational attainment and informally obtained knowledge held particularly by mothers have both been shown to be significantly linked to improved nutrition among their children.27 Cross-country time series, as also studies using natural experiments, have confirmed that maternal education is a key determinant of birth weight, neonatal survival and children’s attained height. 35 Development assistance: WFP is one of the three main providers of school meals in Laos, along with GoL and Catholic Relief Services (CRS). WFP and FAO are piloting education material in three WFP-assisted schools in Luang Namtha; WFP and World Bank are piloting the use of clean cookstoves that reduce smoke exposure and the risk of lung disease. UNICEF’s WASH program was implemented in almost 100 WFP supported schools. 36 WFP’s portfolio in Laos is aligned to the development agenda laid out in the 8th NSEDP and United Nations Partnership Framework 2017-2021. WFP’s Country Strategic Plan 2017- 2021 supports GoL in its National Nutritional Strategy and Agriculture Development Strategy through the provision of sustainable access to food for schoolchildren by 2021, reducing stunting rates among children to meet national targets by 2025, increasing the climate resilience of vulnerable households against seasonal and long-term stresses, and strengthening national and local governance institutions to improve service delivery. 37 WFP, together with MAF and MoES and other partners, has implemented the USDA LRP program in Nalae district of Luang Namtha since 2017. Technical assistance was provided to the farming communities of 47 villages for practising improved agricultural techniques and supporting SMP that was being implemented in the village schools. While MAF was expected to provide guidance on the diversity and quantity of seeds or cuttings required and on the procurement of such items, MoES was entrusted the role of incorporating the crops planted in the communities into the Nutrition and School Agriculture curriculum. 22 GDI reflects gender inequalities in the achievement in the three dimensions of HDI: health, education and command over economic resources. 23 UNDP (2015). Human Development Report 2015: Work for human development. New York, USA: UNDP 24 UNDP (2016). Human Development Report 2016: Human development for everyone. New York, USA: UNDP 25 SDG 2 calls for ensuring the completion of primary and secondary education by all boys and girls, and guaranteeing equal access to opportunities for access to quality technical and vocational education for everyone. 26 SDG 5 calls for gender equality and aims at ending all discrimination against women and girls. 27 Smith and Haddad, 1999 USDA Local Regional Procurement Activity Evaluation in Lao PDR Final Report 8 1.3. Evaluation Methodology 38 The activity evaluation of the pilot LRP program was conducted between July and November 2019. The evaluation team undertook a five-day scoping mission from 29 July to 2 August 2019 to obtain a better understanding of the project and finalise the evaluation approach and methodology, in consultation with the WFP CO of Lao PDR (scoping report presented in Annex L). The data collection phase took place in the National Capital Vientiane, province headquarters Luang Namtha and Nalae district, between 16 September and 2 October 2019. The period aligned with the reopening of schools after the semester break. (Mission Schedule presented in Annex B). The scope of the evaluation for the LRP program was the period from April 2017 till June 2019. Evaluation Questions and Criteria 39 The evaluation was in concurrence with the ToR and used the OECD-DAC criteria to assess the relevance, effectiveness, efficiency, impact and sustainability of the LRP program. Overall, 20 evaluation questions (EQs) across these five criteria were framed to assess the program. 40 Relevance: Alignment with and contribution of the program to government strategies (EQ1); the extent to which the program design and implementation contribute to capacitating the smallholder farmers and linking them to markets (EQ2); the program’s contribution to enhancing farmers’ ability to provide diverse and nutritious food to SMP (EQ3); and the program’s contribution to gender equality and empowerment of the vulnerable farmers (EQ4). 41 Effectiveness: Assessing the reasons for achievement or non-achievement of program targets (EQ5); measuring the extent to which the program enhanced smallholder farmers’ contribution to school meals (EQ6); judging the contribution of the program towards gender equality and empowerment (EQ7), and assessing its contribution to improving dietary diversity (EQ8). Efficiency: Adequacy, sufficiency and timeliness of support provided by DTEAP, PAFO and DAFO for solving implementation issues (EQ9&11); efficiency of farmer groups in utilising the technical support for agriculture (EQ10); and flexibility and adaptability of the program to respond to the need for course corrections (EQ12). 42 Impact: The effects of LRP activities on SMP (EQ13); the intended and unintended effects on direct and indirect beneficiaries (EQ14); and the use of new agricultural techniques and knowledge (EQ15). Sustainability: Capacity building of farmers, MAF officials and other partners (EQ17); increased ownership of community-driven school lunches (EQ18); additional aspects for sustaining the LRP program (EQ19); and necessary factors for replicating the program (EQ20). 43 Further, the design and implementation of the program were also assessed using the lens of equality and inclusivity. Each of the five evaluation criteria has been analysed in detail, and the prerequisite factors vital for the LRP program to succeed were identified, along with the learnings to scale up the program in other geographies. For detailed information on evaluation questions and criteria, the Evaluation Matrix is attached as Annex M. Approach and Methodology 44 The evaluation provided an evidence-based performance assessment of the activities and outcomes under the program’s results framework. For this purpose, the Logic model, which provided logical linkages across program resources, activities, outputs and outcomes, was used to measure the effectiveness of the program. The technical approach to the end-line evaluation study has been illustrated in the form of a figure in Annex N. 45 The activity evaluation followed a quasi-experimental evaluation design that covered the study of LRP-supported (intervention) as also non-supported (control) villages and schools. The methodology entailed secondary research as well as primary data collection. A mixed￾method approach was deployed to answer the questions using key informant interviews (KIIs) and focus group discussions (FGDs) for both qualitative and quantitative data. The USDA Local Regional Procurement Activity Evaluation in Lao PDR Final Report 9 evaluation design also included Most Significant Change (MSC), which involved the identification and documentation of seven case studies (Annex U) in the intervention villages, highlighting personal accounts of change of farmers who participated in the LRP program. During the field visits, the evaluation team identified individuals or households which had experienced substantial changes as a result of participating in the program and documented the process of such change in detail. Sampling frame 46 Aligned to the evaluation requirements and to facilitate comparison across the two rounds, 20 sample villages (15 intervention villages and five control villages) for end-line evaluation were purposively drawn from the set of villages covered during the baseline study. The baseline study involved the application of systematic random sampling for selection of villages, disaggregated by lowland (0-500 metres above sea level), upland (500-100 metres above sea level), and mountainous regions (more than 1000 metres above sea level),28 in the proportion of the actual number of intervention villages within each of the three strata. However, two of the intervention villages had to be replaced with villages not covered during baseline study as the required number of respondents were not available. Of the 15 intervention villages covered for evaluation, six were model villages and the remaining nine were non-model villages. 47 The respondents were randomly selected to capture the overall effect of the program and also ensure adequate representation of the existing diversity among intervention villages. The quantitative sample size was calculated at the program level, using the ‘differences method’ formula with a finite population.29 A total of 156 farmers, 201 parents and 380 children across intervention and control villages were selected. A table providing the distribution of samples across different target groups for the quantitative and qualitative components is included in Annex N. Data Collection Methods 48 Other than the secondary literature review, the evaluation used semi-structured questionnaires containing a mix of quantitative and qualitative questions for interviews and group discussions with parents and smallholder farmers for primary data collection. A tool containing multiple-choice questions was administered for children of 5-10 years from classes III-V of the schools in the villages. Discussion guides were used for carrying out FDGs (with VEDC members) and KIIs (with school heads, teachers, cooks, traders, WFP staff, NGO partners and government staff). The list of stakeholders met is presented in Annex O and data collection tools are presented in Annex S. 49 Data from secondary research (documents gathered are presented in Annex P) and different respondent categories within the primary data collection component was triangulated. The evaluation matrix in Annex M presents different sources from where the data for evaluation questions were collected, along with the corresponding methods employed for carrying out data analysis. Data Analysis Methods 50 Given that the evaluation was primarily qualitative, in addition to the comparison between intervention and control villages, the focus was essentially on explaining the reason(s) behind the achievement or non-achievement of key performance indicators. 51 The evaluation study included the use of qualitative research tools such as the H-form tool and Most Significant Change. Qualitative data was translated into English, checked by the evaluation team for consistency based on the field visits, and subsequently analysed using content analysis. Quantitative data was cleaned for ensuring basic consistency and, subsequently, tabulated. 28 Baseline Study – WFP Local Regional Procurement Program 29 Cochran 1977. USDA Local Regional Procurement Activity Evaluation in Lao PDR Final Report 10 Integration of Gender into the Methodology 52 The evaluation integrated gender dimensions into its design. It examined the role and nature of the participation of men and women in the program, specifically through VEDCs and farmer groups. The evaluation matrix presented in Annex M highlights that gender was an integral theme for several evaluation questions, along with a focus on other vulnerable groups. Question 4, under the relevance criterion, captures the extent to which the program was in line with the needs of women and men smallholder farmers, and whether the program was based on sound gender analysis. Under the effectiveness criterion, question 7 captures the extent to which women and men smallholder farmers benefitted from the program activities. Under the impact criterion, question 14 focuses on the intended and unintended effects of the program on men and women smallholder farmers. 53 The data collection team was adequately trained to ensure that views of all diverse groups were considered, reflected upon and triangulated, with specific attention to issues revolving around gender. The data collection team was gender-balanced, with three male and three female enumerators, all of whom were fluent in the Lao language. The core evaluation team also had an equal number (two each) of male and female members. To the extent possible, participants for group discussions included both men and women in equal numbers; questions to assess their views on gender issues were included in the checklist. Validation Exercise 54 To validate findings of the LRP evaluation and aid cross-learning among stakeholders (WFP, MAF and MoES, VEDC members and farmers), validation workshops were conducted in Nalae district and Vientiane.30 The workshops were aimed at triggering discussions, particularly around feedback on the program and key recommendations for designing and implementing a similar program in future. 55 The workshop in Nalae district was attended by MAF and DAFO officials, WFP representatives, VEDC members and farmers. Post the presentation on the evaluation findings, five groups were created for further discussion. Each group discussed (1) what worked well with LRP; (2) what needed improvement; (3) how each group can ensure these improvements, and (4) what support each group would require from others to carry out these improvements. 56 In Vientiane, the workshop was attended by officials from WFP and USDA. In addition to the discussion on the findings of LRP, the participants specifically discussed: (1) key inferences they drew from the evaluation and the visual thinking exercise, and (2) recommendations for designing and implementing a similar program in future. Ethical Considerations and Quality Assurance 57 With its rich experience of working with UN agencies including WFP, NRMC has a deep understanding of the United Nations Evaluation Group (UNEG) norms, standards and ethical guidelines. Further, NRMC’s internal quality protocols were integrated with the process for information collection, collation, analysis and delivery. 58 The evaluation was particularly conscious of maintaining ethical norms for data collection and its reporting. In addition to providing the option to the respondent to participate in the study, proper informed consent was taken before initiating any discussion. Prior consent was taken from the school heads/teachers before interacting with children in schools. Extreme 30 The workshop in Nalae was conducted on 18 December 2019. In Vientiane, it was conducted on 20 December 2019. Figure 2: Visual thinking for validating results USDA Local Regional Procurement Activity Evaluation in Lao PDR Final Report 11 care was taken while interacting with children, ensuring no mental or physical harm or loss to them during or after the interaction. Similarly, at the time of reporting, the evaluation team ensured that the names of the respondents were not disclosed in the evaluation report, which could potentially lead to their recognition. 59 The data collection team consisted of Lao-based personnel who were well versed with the local language and had prior experience of collecting and collating field-level information. A gender-balanced team was deployed to gather the perspectives of boys, girls, men and women. Separate teams were deployed for quantitative and qualitative surveys. NRMC core team conducted discussions with government officials, WFP field teams and, partners. 60 A two-day training session on field ethics and data collection tools was conducted for enumerators by the NRMC core evaluation team. The team was provided with translated tools to overcome language barriers. 61 As part of quality control as also to ensure timeliness of data collection, NRMC developed detailed field movement plans before the survey. A daily team movement plan was shared well in advance with the team. At least two of the core evaluation team from NRMC were present in the field during the entire period of data collection, accompanying qualitative and quantitative interviewers. 62 An internal team within NRMC reviewed the draft evaluation report before it was shared with WFP. The exercise ensured that the report covered all the evaluation objectives and answered all evaluation questions, following the prescribed research methodology. The final report has been edited by an external editor before it has been shared with WFP. Limitations and Risks 63 While the evaluation made comparisons between case and control groups, it did not capture information about other interventions carried out in evaluation villages, and hence cannot attribute any changes to the program. The activity evaluation was quasi-experimental and hence can only comment on the contributions made, without attributing any changes to the program. However, primarily using qualitative data, the evaluation sought to understand and explain how the program influenced the observed results as highlighted in the evaluation questions. 64 The two key objectives of the baseline study included understanding the agricultural practices adopted by farmers and the impact of the location of a village on their agricultural practices. While analysing key components of the first aspect, it emerged that the baseline study analysed data at the geographical strata (lowland, upland and mountains)31 and individual village levels. The end-line study, however, presents findings at program and strata levels, and not for each village individually. 65 The children’s tool was administered to children from standards III-V as children from standards I-II would have found it difficult to comprehend and respond to the questions. The total number of children in a few schools was much lower than the minimum sample required for per school (27), which affected the total sample size achieved. 66 It is noteworthy that the program intervened with only 10 farmers within each intervention village. As a result, the end-line data collection adopted a census approach, involving all intervention farmers for the FGDs. However, it was observed that in many cases, certain farmers (and parents) would either shift to the uplands for paddy cultivation or move to the fields early in the morning, and hence could not be contacted. As a result, while the number of FGDs remained as planned, there was a slight shortfall in the number of individuals covered in such discussions. As mentioned earlier, the evaluation was primarily qualitative, putting a major focus on explaining the reason(s) behind the achievement or non￾achievement of key performance indicators. Since the number of qualitative activities and 31 Definitions of the geographical regions have been provided in a previous section of the study. USDA Local Regional Procurement Activity Evaluation in Lao PDR Final Report 12 discussions remains unchanged, we believe the shortfall in individuals would not have any implication on the findings. 67 The performance matrix shows that the number of indirect beneficiaries was computed by multiplying the number of direct beneficiaries by five. This approach was based on the assumption that every beneficiary reached would also have transferred benefits of the program to his/her family members. However, the approach failed to identify overlaps in the form of children and farmers belonging to the same households, or two siblings belonging to the same household. As a result, this approach of estimating indirect beneficiaries may have amounted to multiple counting of certain indirect beneficiaries, and therefore would have inflated the total figure. Also, it was not possible to determine the male-female ratio among the indirect beneficiaries. 68 While WFP had designed a specific monitoring tool in KOBO to track the project implementation process and its planned outputs, it was not regularly used during the two years of intervention. The absence of robust monitoring by WFP CO/implementing partner as also of financial data has impeded comprehensive and detailed analysis affecting the evaluation outcomes specifically for measuring effectiveness and efficiency. 69 The time frame for partnership with the Lutheran World Federation (LWF) was beyond the scope of this evaluation, and hence the evaluation could not evaluate the outcomes achieved as a result of this partnership. The evaluation can, therefore, only comment at a conceptual level on the idea of this intervention, but cannot assess how the intervention has been received by the community and the nature of impact thus created. 70 Following the ToR, evaluation design included an assessment of the impact of the program. However, it was realised that it was too early to capture the true impact of the program, as the two-year program had recently ended, in June 2019. Hence, the evaluation results indicate more short-term changes. 2. Evaluation Findings 2.1. Relevance of LRP 71 The key evaluation questions (EQs) presented in Annex M were the foundation for assessing the LRP program. This section focuses on questions about the relevance of the program and includes (i) alignment with and contribution of the program to government strategies (EQ1); (ii) the extent to which the program design and implementation contribute to capacitating the smallholder farmers and linking them to markets (EQ2); (iii) the program’s contribution to enhancing farmers’ ability to contribute diverse and nutritious food to SMP (EQ3); and (iv) the program’s contribution to gender equality and empowerment of the vulnerable farmers (EQ4). Alignment and Contribution to Government Strategies 72 The LRP program 2017-2019 cohered with the national priorities around agriculture, nutrition and education. GoL has been combating malnutrition by promoting dietary diversity at household, school and community levels through the implementation of the School Meals Action Plan (SMAP) 2016-2020 and Agriculture Development Strategy to 2025 and Vision to the Year 2030. Annex Q highlights components within the national Agriculture Development Strategy and Nutrition Development Strategy as also the National Education Promotion Policy that was in synchronisation with the logical framework of the program. 73 MoES has identified Home Grown School Feeding (HGSF) as the future strategy for supporting SMP. HGSF aims to provide students with food produced and purchased within the country to the maximum extent possible. It is increasingly being endorsed by governments and organisations for its potential benefits to education, nutrition and agricultural production through the generation of consistent local market demand. 74 Aligned with National School meals Program (NSMP), the LRP program also provisioned for 800 kips per student per day as cash transfer to schools for purchasing vegetables. It was USDA Local Regional Procurement Activity Evaluation in Lao PDR Final Report 13 expected that the availability of cash would enable schools to overcome any financial barriers that may have prevented the purchase of vegetables resulting in non-cooking of school lunches. Also, by supporting farmers with technical knowledge and supporting the schools financially, the program identified and tackled both the demand and supply issues as prescribed under NSMP. The piloting of LRP at Nalae has supported schools in the district for a smooth transition to the national program. By establishing the partnership mechanism between schools and farmer groups, it has ensured the continuity of school meals under NSMP. 75 The LRP program was in line with the priorities stated in the Agriculture Development Strategy to 2025 and Vision to the Year 2030 strategy document by way of investments to increase multiple crop agricultural practices and diversify food products to achieve food security. The program design included building public-private partnerships between government departments, program staff and farmers, involving the capacity building of all relevant stakeholders, including provincial- and district-level government officials to achieve sustained outcomes from the program. Detailed analysis showcasing the linkage between the LRP program and the strategy document is elucidated in Annex Q. 76 The LRP program was aligned with the National Nutrition Strategy (NNS) to 2025 and Plan of Action 2016-2020 and was underpinned by inter-sectoral coordination involving WFP CO, MoES and MAF for ensuring capacity building of government officials and farmers to promote improved nutrition in school meals. NNS defines nutrition through the prism of gender, highlighting access to health and nutrition equally by girls, boys, women and men, and ensuring the participation of women and men in decision-making across levels.32 77 The Education Sector Development Plan 2016-2020 states that the provision of school meals can help in reducing dropouts and improving retention in schools in lower grades.33 SMP is aligned with the government objectives of reducing dropouts as also improving learning outcomes by way of provision of school meals. The LRP program supported the sustainability of SMP by ensuring regular supply of locally grown fresh and nutritious vegetables for school meals and hence was in alignment with the government plan. 78 MoES is the nodal agency for the implementation of nation-wide school meals in Laos. It has set up an Inclusive Education Centre (IEC) unit for oversight and scaling up of the NSMP. WFP CO closely coordinated with MoES to implement the LRP program. The LRP program design required close coordination between MAF and MoES. While it was envisaged that MAF would extend support through training on the cultivation of vegetables required for meeting the nutritional needs of the schoolchildren, MoES was responsible for incorporating the crops planted in the community into the Nutrition and School Agriculture curriculum. Coherence with WFP Country Program (2017-2021) 79 The WFP Country Strategic Plan for Lao PDR 2017-2021,34 drafted in consultation with GoL, envisions the full handover of school meal activities to local communities by 2021. For the successful implementation of the plan, building institutional capacity at the central and sub-national levels together with the government ensured strengthened capabilities to assume ownership at the community level. The LRP program’s cash support enables the school management to decide on the spending mechanism and thus are in a position to receive cash support under NSMP. Simultaneously, it also supported building the capacities of the farmers for selling a variety of vegetables throughout the year to schools. Capacitating Smallholder Farmers and DAFO Officials 80 The USDA-supported SMP FY14-16 end-line evaluation had highlighted non-availability of vegetables as a critical barrier to the smooth implementation of SMP. Students attending 32http://www.nationalplanningcycles.org/sites/default/files/planning_cycle_repository/lao_peoples_democ ratic_republic/final_lao_version_nnspa_2016_matrix_updated_21_dec_2015_-_part_1_rta_-.pdf 33 http://www.dvv-international.la/fileadmin/files/south-and-southeast-asia/documents/ESDP_2016-2020- EN.pdf 34 https://docs.wfp.org/api/documents/WFP-0000021032/download/ USDA Local Regional Procurement Activity Evaluation in Lao PDR Final Report 14 school were deprived of the school lunch due to non-availability of vegetables. The problem is aggravated in challenging topographies such as in Nalae where farming is constrained by the terrain and lack of water for irrigation. Also, over 80 per cent of households in this area are subsistence farmers, growing mainly rice. The tradition of single-crop subsistence farming where households grow rice (and have perhaps planted a small kitchen garden) has led to limited availability of the varied commodities required for a nutritious meal to be prepared through SMP. 81 Under these circumstances the LRP program aimed at training the smallholder farmers to practise multi-cropping of diverse vegetables and supporting them with seeds and tools. The goal was to enable these households, through the use of improved agricultural techniques, to grow sufficient vegetables in their small land parcels. The diversified products would be sufficient to meet the school requirements and would also facilitate the consumption of nutritious meals at home. The excess produce was to be sold in the open market, which would contribute to the household income. 82 The climatic uncertainties caused by long dry spells followed by heavy rains constraint farmers from farming throughout the year. To further support the farmers, they were provided with greenhouse materials to ensure round-the-year vegetable production. A couple of farmers from each group were taken for exposure visits where they learnt from practising farmers on setting up greenhouses and doing greenhouse-based farming. These visits have helped the program farmers and they have been able to grow vegetables throughout the year. The deputy DAFO stated that due to these exposure visits, farmers have started demanding for support. This increase in demand is underpinned by the success achieved by the LRP supported farmers. 83 The importance of building capacities in the government for making it responsive to the needs of the community is critical for the success of any social safety program. DAFO officials attended training workshops and undertook exposure visits, and in-turn conducted training sessions for smallholder farmers from intervention villages. VEDC, LWU members, teachers, etc. were trained in various components of program implementation, while farmers were trained in modern agricultural practices. The trained officials can now be utilised as trainers for during replication and scaling up of the program. 84 The program involved government partners to impart onsite training to farmers on modern farming methods, including preparing the land for cultivation, preparing and using compost, growing vegetables, crop rotation etc. and providing seeds. Discussion with farmers indicated that easy access to trained officials had helped them in overcoming farming issues. Also, the officials had supported them to overcome the traditional wrong practices of farming. E.g. reducing the number of seeds along with proper spacing has helped in increasing production with reduced input cost. 85 MAF has also trained VEDCs to oversee implementation of the LRP. Key tasks performed by VEDC under LRP included handling of cash and managing accounts, developing a menu for school lunch meals and monitoring of activities at the village level. 86 The program by design had ensured support for both the demand and supply side. Along with supporting the farmers for producing a variety of vegetables throughout the year and supplying them to school, the program also provided financial support to the schools for purchasing vegetables. The amount of 800 kips per student per day provided to schools for purchasing food materials was equal to the allocation under NSMP. The schools had agreed Figure 3: Vegetable farming using Greenhouse technique USDA Local Regional Procurement Activity Evaluation in Lao PDR Final Report 15 that they would purchase vegetables for the school meals from the LRP supported farmers. This helped the farmers is securing a stable market where they could sell their produce. 87 The program had also envisaged that post supplying of vegetables to school and self￾consumption, the farmers would be left with surplus vegetables which can be sold in the open market. Hence, DAFO was expected to support the farmers in selling their produce. From field discussions, it emerged that DAFO had provided a separate space at the Nalae market for the LRP farmers to sell their vegetables. The farmer groups on a rotation basis went to the market to sell their produce. This helped in generating additional income for the farmers and make vegetable farming economically viable. Providing Diverse and Nutritious Food to SMP 88 The previous SMP evaluation reports suggest that parents found it difficult to provide vegetables for school lunch resulting in absenteeism and irregularity in the cooking of school lunch. In the absence of vegetables, rice and lentils were being served which resulted in dissatisfaction among children. In such a scenario, the LRP program tried to support farmers wherein they would sell a part of their produce to schools at a discounted price. This provided the farmers with a sustained market and simultaneously schools with a sustained flow of vegetables. The availability of vegetables has helped in regularising the school lunch for 90-95 per cent of school days. Further, this has also removed the burden from the families whose children skipped schools as they were unable to contribute to the school lunch. 89 The relevance of the activity is further established by the fact that farmers who were earlier farming four crops which included rice, rubber, cardamom and Inca are now growing minimum of six varieties of vegetables going up to 20 varieties. This excludes the time spent on rubber and cardamom farming. Farming on their small land has allowed in the production of vegetables which are now being donated to schools, used for self￾consumption and selling in the open market. The availability of vegetables from their land ensures that they don’t have to purchase vegetables at a higher price from the market thus ensuring that the families are also consuming a variety of vegetables which is helping them to meet their nutritional needs. Gender Equality and Empowerment of Smallholder Farmers 90 The program was piloted in Nalae district of Luang Namtha, which is one of the most disadvantaged regions in the country. The area is at risk of natural disasters, specifically landslides, for which the population has a demonstrated need for resilience-building strategies such as crop diversification. With a challenging terrain, the supply of vegetables for school meals was a major concern in the area. Further, the supply of materials under SMP was hampered due to the absence of metallic roads which contributed to the irregularity in the cooking of school lunches. 91 By piloting LRP in Nalae, the program targeted communities residing in remote areas and demonstrated success in reaching out to them. The program focused on building capacities of smallholder farmers who, due to their small land parcels, are at a disadvantage in terms of farming for commercial purposes. Further, any success in such a difficult terrain also helped in demonstrating the program’s potential for scaling up and replication. 92 Selection of farmers for the program was done by VEDCs in consultation with communities and with support from DAFO. The program was open for all farmers with small land parcels who agreed to cultivate and supply a portion of the produce for school meals at discounted prices. All farmers satisfying the condition of land ownership were allowed to participate in the program. This demonstrates the program’s unbiased targeting of smallholder farmers Figure 4: Students having lunch at school USDA Local Regional Procurement Activity Evaluation in Lao PDR Final Report 16 who otherwise have limited options in terms of agricultural support in enhancing their farm production. 93 The design of the program ensured that both women and men smallholder farmers within intervention areas were covered for capacity building and other benefits. In each village, a total of 10 households were selected for farming interventions. From each household, one man and one woman were trained on modern farm techniques and provided seeds and manual agricultural tools to enhance the quantity and quality of produce. Field observations highlighted that women undoubtedly benefitted from capacity-building measures. At the same time, their involvement in farming was largely restricted to carrying out manual labour; strategic decisions about agriculture continued to be dominated by men. 94 Aligned to the National Strategy for Gender Equality 2016–2025, the LRP program aimed at providing fresh and nutritious lunch meals in schools, on a sustained basis, for all schoolchildren in primary schools. The universal coverage of the program ensured no child, irrespective of his/her gender, was to be left out of the program. 95 Following the recommendations of the end-line FY14-16, WFP CO reported integrating gender components into its implementation of SMP. This included sharing of tasks in school gardens between boys and girls without defining tasks by gender, strengthening of nutrition education, and mainstreaming of gender in field-level activities by using new literacy materials to challenge unsuitable gender roles and improve sensitivity. 96 Field observations indicated that cooking of school lunches has been entrusted mainly to women, thus restricting the role to women. Moreover, there was no provision to ensure women’s participation in leadership and decision-making roles except for their participation in VEDCs as LWU representatives and school teachers. The review of program documents and interactions with the community and government officials revealed that while preference was accorded to women volunteers, the program did not lay specific focus on ensuring women’s involvement in strategic decision-making, especially at the village level. 2.2. Effectiveness of LRP 97 As discussed earlier, the LRP program was based on two pillars: (1) supporting school meals for children by way of sustained supply of vegetables, and (2) increasing household income by strengthening sustainable farming and establishing relevant commercial linkages. This section evaluates the effectiveness of the program by (i) assessing the reasons for achievement or non-achievement of program targets (EQ5); (ii) measuring the extent to which the program enhanced smallholder farmers’ contribution to school meals (EQ6); (iii) judging the contribution of the program towards gender equality and empowerment (EQ7); and (iv) assessing its contribution to improving dietary diversity (EQ8). 98 Before embarking on assessing the effectiveness of the program, it is essential to highlight the change in program outreach during the implementation. The two-year program had envisaged that in the first year emphasise will be made on the formation of farmer groups and skilling them on improved farming techniques. A positive result in the first year would have motivated the farmers in taking up farming commercially. However, discussions with farmers indicate that the commercialisation aspect was not articulated to them. Moreover, the absence of markets led to huge wastage of surplus vegetables. This demotivated a large number of farmers especially those from upland and mountain regions where the markets were far and few leading to non-participation in the second year. Thus, in the second year, farmers from only 10 out of 47 villages participated in the program. A pre-program situation analysis along with providing a clear picture of the program expectations and support would have helped in the participation from all the villages. 99 The program regarded all students and farmers who received benefits from the program activities as direct beneficiaries. The household members of these students and farmers, on the other hand, were considered indirect beneficiaries of the program. Overall, 17,854 people were directly benefitted, of which 8229 were female beneficiaries and 22,913 people. were indirectly benefitted including 11,227 female beneficiaries. USDA Local Regional Procurement Activity Evaluation in Lao PDR Final Report 17 100 The LRP program focussed on supporting the smallholder farmers. Field assessment indicates that the average land size of the farmers involved with the LRP program was about 0.65 acres. In comparison, the average land size of the farmers from the control villages was about 0.34 acres. Aligned to expectations, the average land size of farmers from the lowland area was 0.85 acres which were larger than those from the upland area and mountain regions (both 0.22 acres) demonstrating the effectiveness in targeting smallholder farmers. 101 The support to farmers under the LRP program was done through the formation of groups. With guidance from DAFO, 10 interested and eligible farmers (meeting the selection criteria as laid out under the program) were organised into farmer groups in each of the 47 villages. DAFO and WFP Monitoring Assistants (MA) provided technical knowledge and program support. Multiple rounds of training were conducted with these farmer groups to enhance their knowledge about different types of vegetables and their nutritional value. Achievement of Outputs and Outcomes of the Intervention Indicator: Number of individuals benefitting directly through local and regional procurement 102 At an aggregate level, the program aimed to reach out to and benefit 5000 individuals (4500 students and 500 farmers) for each of the five semesters. The program benefitted more than 15,000 students including 6884 girls; 2665 farmers including 1337 female farmers; and 32 government officials. While the actual achievement hovered around 75 per cent students for the first three semesters, it surpassed the target in the fourth semester. This is due to more students attending the schools than estimated under the LRP program. Similarly, the program reached out to 95 per cent of the farmers during semester I. From the second semester onwards the capacities of both the male and female members of the targeted households were built, leading to a higher number of beneficiaries. Training including exposure visits was also provided to 32 DAFO officials, among which eight were female officials. 103 By linking schools with local farmers, the program assured a ready market for selling vegetables. The program also focused on providing an enabling environment to the farmers for selling their surplus vegetables. After supplying to the schools and retaining for their self￾consumption, the farmers would still be left with surplus vegetables. Assuming it would be difficult for the farmers to sell their vegetables in the open markets due to opposition from the regular sellers, the program had engaged with DAFO for earmarking a space in the main district market for these farmers to sell their produce. This enabled the beneficiary farmers to sell their vegetables without getting into conflicts with other vegetable sellers. 104 The participation of farmer groups in the Nalae district market prompted a trader to come to a village to collect the vegetables instead of the farmers going to the market. This has increased the desire among non-group members also to be part of the vegetable farming activity and earn from it. Figure 5: Number of individuals benefitting directly through LRP USDA Local Regional Procurement Activity Evaluation in Lao PDR Final Report 18 105 While the program was not explicit in motivating the farmers towards organic cultivation, the techniques imparted as part of the training would ensure, in the long run, that the products are organically produced. The promotion of the generation and use of compost will help in eliminating the need for chemical fertilisers, thus protecting soil health as well as contributing to organic production. 106 The organic food market is growing larger not only within the country but internationally as well. Thus, there is a huge market potential to be tapped wherein these farmers can sell their organic products at a higher price in comparison to the products from other sellers. Indicator: Number of individuals benefitting indirectly through local and regional procurement 107 The program has benefited 22,913 individuals (11,227 female beneficiaries) who were not directly associated with the LRP program. Exposure visits for government officials and program supported farmers helped in disseminating the knowledge to other farmers and motivated them to adopt the improved agricultural practices. The regularity in school lunch has allowed nearly 30 per cent of the parents to increase their productive time and augment their income. Indicator: Number of public-private partnerships formed as a result of USDA assistance 108 Trained farmer groups in 19 out of 47 villages entered into formal partnerships with schools in semester II and began selling vegetables for school meals, thereby establishing local public-private partnerships. From the third semester onwards, WFP CO changed its approach and shortlisted 10 model villages. In these villages, the program created and strengthened farmer groups, enabling them to focus on a diversified set of vegetables all year round with the help of greenhouse techniques. 109 The gathering of 10 farmers into farmer groups within model villages resulted in technical knowledge exchange, and sharing of seeds and tools even among non-beneficiary farmers; in general, it improved social cohesion. Farmer groups have proven to be extremely useful, particularly in terms of providing a platform to farmers to come together and plan their sowing strategy for the next season to ensure a wide diversity of vegetables, keeping in consideration prevalent demand and supply conditions and thus avoiding the overproduction of particular vegetables. 110 Currently, these groups are functioning informally without having a meeting schedule, division of roles and responsibilities and recording the minutes of the meeting. The formalisation of these groups will further enhance the planning activities and contribute to making vegetable farming a sustainable economic activity 111 The idea of introducing a curriculum on nutrition in schools has been on the agenda for a while. Discussions with government officials at MoES revealed that the curriculum is ready and will be rolled out in schools from next semester onwards. Indicator: Total increase in installed storage capacity (dry or cold storage) as a result of USDA assistance 112 The program initiation involved new installation or refurbishment of storage space of 24 cubic metres (3x4x2) within each of the program schools. The schools were already supported with storage space under the SMP program. No further additions were made in storage capacity, as it was realised that the farmers did not need storage space since they only provided fresh vegetables to the schools or sold them in the market as per their need. However, the evaluation pointed towards the need for investment in a cold storage facility “We can work for longer hours without worrying about children’s meals. School meals has made children interested in attending school, and concentrate better.” – Parents in Village Saloy “The knowledge we received from trainings is used for preparing the soil and using manure; training matched our needs. We do not use fertilizers now.” – Farmers in Village Konechan USDA Local Regional Procurement Activity Evaluation in Lao PDR Final Report 19 or any other mechanism to ensure increased shelf life for vegetables, particularly during periods witnessing overproduction or low demand. Indicator: Cost of commodities procured as a result of USDA assistance 113 Under the LRP program, commodities worth USD 48,045 were procured during the period from October 2017 to March 2018. The procured commodities can be segregated into two groups: (1) commodities procured for growing vegetables, that is, seeds, agricultural tools and cooking utensils amounting to USD 21,425, and (2) procurement of vegetables and other eatables by schools, amounting to USD 26,620. Apart from this, chicken, fish fingerlings and fruit trees were procured under the LRP program during April 2018 to September 2018 period; however, the amount spent on these commodities is not available in monitoring data. 114 Qualitative discussions with farmers revealed receipt of 11 types of seeds and certain manual tools for carrying out agriculture in the first year across all program villages. While the farmers received the commodities on time, they were not satisfied with the quality of the tools provided. Farmers opined that the manual tools, particularly the water buckets and sprinklers if made of good quality plastic, would have lasted longer. Year 2 saw the provision of greenhouse plastic sheets, water pumps and piped water connections for farmers cultivating vegetables across 10 model villages. 115 Discussions with farmers highlighted that except for the 10 model villages which received water pumps and piped connections, availability of water continues to be a major issue, affecting farmers’ decision to invest in the cultivation of different crops. Farmers in non￾model villages highlighted the pressing need for investment in laying water pipelines and ensuring sustained access to water for cultivating vegetables. Despite the market challenges, more farmers may have continued with the program had they been aware of the irrigation support being provided under LRP. This underscores the point that the farmers were not fully aware of the overall LRP support that was planned to augment the farm-based income of smallholder farmers. Indicator: Cost of transport, storage and handling of commodities procured as a result of USDA assistance (by commodity) 116 Transportation costs include the amount spent under the program on transporting, storing and handling of procured commodities such as seeds, agricultural tools and cooking utensils. It is noteworthy that usually, the schools do not need to spend on transportation for procuring vegetables and other eatables, mainly because all procurement was carried out at the local level. The amount spent on transportation varied between USD 5200 and USD 7000 per semester, borne only up to the third semester. Indicator: Number of social assistance beneficiaries participating in productive safety nets as a result of USDA assistance 117 The indicator covers the total number of beneficiaries who participated in the USDA￾supported social assistance programming, which involved training aimed at increasing household assets or strengthening human capital. The program intended to reach out to the same 2500 people each semester. Semester II saw the transfer of cash resources for 3454 students (1787 boys and 1667 girls) across 47 schools, and technical assistance to 474 farmers (200 males and 274 females). The number was largely consistent across the next two semesters as well. In the third semester, 36 farmers from the model villages were taken for an exchange visit to learn greenhouse farming techniques. The program thus succeeded in reaching the stated target of ‘participating in productive safety nets’, and maintaining a near equal ratio among men and women, as also boys and girls Indicator: Number of individuals who have received short-term agricultural sector productivity or food security training as a result of USDA assistance “We received seeds and a number of tools under the program, mostly on-time. However, some of the tools, such as the big blue plastic watering can were of poor quality” – Farmers in Village Saloy USDA Local Regional Procurement Activity Evaluation in Lao PDR Final Report 20 118 The LRP program targeted reaching 500 farmers across 50 villages for two years which subsequently became 470 farmers in 47 villages. 10 farmers from each village were identified by VEDCs. Once they expressed interest in being part of the LRP program, these farmers were formed into groups. 119 On an average, 10 farmers per village were trained on nutrition-sensitive agriculture, reaching a total of 460 (265 males; 195 females) and 474 farmers (200 males; 274 females) in semesters I and II respectively. Since the program only catered to 10 model villages from year 2 onwards, we see a drastic fall in the number of farmers who received short-term agricultural sector training in semesters III-V. It emerged from the interactions with farmers from both model and non-model villages that the training sessions were extremely informative and have resulted in a permanent gain in their knowledge levels relating to agriculture. 120 During discussions, farmers from non-model villages expressed their desire to participate in refresher training and also to be included as part of farmer exchange visits. They have continued practising agriculture with the knowledge acquired under LRP. More support towards creating new markets fuelled with refresher training will help these farmers in terms of make farming profitable and sustainable. The program, therefore, succeeded in reaching out to the targeted number of farmers and building their capacity around short￾term agricultural sector productivity and food security training. However, the activity on seed replacement was still not being practised by all the farmers. The knowledge that seeds had to be replaced after a certain number of production cycles was found to be poor. While all farmers from model villages were aware of this, none from the non-model villages were aware of seed replacement. 121 Training modules of agriculture were prepared and designed by MAF. Feedback from farmers immediately after the rollout of initial training sessions revealed that the modules were dry and too theoretical. Based on the feedback, MAF redesigned the modules to make them simpler, interesting and easy to comprehend. Interactions with farmers during the end￾line evaluation showcased a general sense of satisfaction with the training material and the quality of trainers arranged under the program. 122 Nalae district is particularly prone to natural calamities like landslides and flash floods. As a result, disaster risk assessment and preparedness as also training for resilience building and resource sharing are extremely necessary for community members. With climatic uncertainties impacting farming in the area with volatile production of vegetables, the greenhouse technique has been able to address the climatic challenges to some extent. However, the technique has been limited to the model villages. This technique could have been provided to farmers in all the 47 villages thereby contributing to building resilience among the farmers. Further, the program could have potentially invested in establishing cold storages or small-scale food processing to increase the shelf life of vegetables, which could have been sold during lean seasons or in markets outside Nalae. 123 Nalae has three prominent regions based on its topography – lowlands, uplands and mountainous land. The program rolled out a standardised intervention, providing the same set of seeds, tools and training to farmers from all three regions. Qualitative discussions with community members highlighted that cultivation of vegetables was not very conducive in 460 474 36 38 54 0 100 200 300 400 500 Semester 1 Semester 2 Semester 3 Semester 4 Semester 5 Figure 6: Individuals who received short-term agricultural training USDA Local Regional Procurement Activity Evaluation in Lao PDR Final Report 21 some of the villages in the upland and mountain regions, largely because of the prevalent soil type and shortage of water. Also, such villages did not have ready access to markets for selling vegetables, resulting in wastages [Case study no. 4 in Annexure U]. 124 Communities in these regions spoke of greater dependence on shifting cultivation, livestock rearing and growing rubber and cardamom as livelihoods. Ideally, the program should have carried out formative research in the beginning, studying the needs of the community, prevalent physical conditions and availability of resources across each of the three regions, which could have informed the conceptualisation of a non-standardised program. Indicator: Value of public and private sector investments leveraged as a result of USDA assistance 125 As per the performance matrix, the value of investments leveraged has been estimated by assuming a total production of 50 kg of vegetables per week, which could either be consumed by the household or sold in the school or market. Leveraging investment is defined as any use of public and private resources intended to increase future production output or income. While the farmers started selling their produce from semester II onwards, it was too early to monetise the investments made. From semester IV onwards, the total value of leveraged investments was estimated at USD 82,569 for a total of 100 farmers across 10 model villages. 126 Field observations and qualitative discussions pointed out that the production of vegetables dipped particularly during the dry season (October-April), which should have been considered while estimating the total value of leveraged investments. Similarly, the sale of produce varied across villages, primarily depending upon (1) access to market and (2) demand for vegetables. It is believed that the above-mentioned factors need to be factored into consideration while carrying out such estimations. Indicator: Value of sales by project beneficiaries and volume of commodities (MT) sold by project beneficiaries 127 Data from the performance matrix highlights that only 39 per cent of all trained farmers in year 1 managed to sell their produce in the market. However, these farmers were able to achieve more than 70 per cent of the total program sales target (in value). Overall, against a target of USD 0.27 million worth of sales during the program period, the farmers have been able to sell about USD 0.16 million worth of vegetables. Similarly, against a target of about 1000 metric tons, the farmers could sell about 198 metric tons of vegetables. A long dry spell followed by heavy precipitation in a short span resulted in limited production of vegetables from October 2018 till March 2019. The limited quantity of vegetables produced was mainly used for supplying to school and self-consumption. There was no surplus available for sale during this period. 128 The increase in sales from April to September 2018 was a result of the participation by non￾intervention farmers. Observing the success of farmers supported by LRP program during the previous period, the farmers who were earlier reluctant to join or could not be included as part of the 10-member farmer group became inspired and started farming using the improved techniques. With support from DAFO and peer members from the farmer groups, these individual farmers also indirectly benefitted from the LRP program. Table 2: Value of Sales Production under LRP Period Value of Sales (USD) Volume of Commodities (MT) Oct 17-Mar 18 53,240 9.4 Apr 18-Sep 18 71,937 122.3 Oct 18-Mar 19 - - Apr 19-Sep 19 37000 66 “At first, only ten farmers from each village benefited from the program inputs. Later, technical inputs and seeds were shared by program farmers with non-program farmers, resulting in increase in production of vegetables.” – Farmers in Village Aome “Last year, we were forced to sell off vegetables at an extremely low price due to over production. Market is too far; no traders come here to purchase vegetables.” – Farmers in Village Longkhaen USDA Local Regional Procurement Activity Evaluation in Lao PDR Final Report 22 129 Field discussions highlighted the inability of the farmers in remote villages to ‘access the market’ as one of the main factors affecting the sales of vegetables. Given that Nalae district has a primarily mountainous terrain, accessing the local market in the district headquarters is extremely difficult, particularly for the villages located in upland and mountain regions. Lack of adequate transportation facilities at the village level accentuated the issue of transporting perishable food items to the market, resulting in wastage of vegetables. 130 Physical space was provided at the district market for farmers to sell their excess produce. However, the market is extremely small with limited demand, and hence product absorption through this market is limited [case study no. 4 in Annexure U]. As a result, while on the one hand villages in remote locations found it difficult to access the market, on the other hand, villages in closer proximity to the district market could not sell all their vegetables because of lack of demand. Hence, to manage the demand-supply balance, it will be essential to move beyond the wet market and leverage the dry market. 131 Issues such as this along with the weather resulted in a lower sale of vegetables during the period of April 2019 to September 2019. Farmers supported with greenhouses were able to grow vegetables during this period while other farmers lost out due to the dry spell followed by intense rains. It is interesting that due to greenhouse support, the farmers who earlier could not sell anything during the second period were able to produce, consume and sell. 132 Farmers in program villages were trained on creating and using compost, in place of chemical-based pesticides and fertilisers. However, discussions with DAFO, PAFO and MAF pointed to the absence of any provisions for issuing organic certificates to farmers. As a result, while the farmers invested enormous efforts in cultivating organic vegetables, the produce did not get its due in the market. The LRP program may have missed an opportunity to position these as premium products and access new markets. 133 While work around creating a policy around organic certification is in progress, this would take a considerable amount of time because of the bureaucratic procedures involved in the government set-up at national, provincial and district levels. However, to overcome the limited market availability, it is essential that new markets are identified and quality products taken beyond the limits of the local boundaries. To that extent, WFP with other donors can support GoL in formulating and implementing the organic certification process so that the farmers can reap the benefits of their efforts by getting premium prices for their organic vegetables from the urban markets. 134 Field observations indicated that in one of the intervention villages, a trader visited the place and bought surplus vegetables directly from farmers. The approach not only provided an assured market to the farmers but also helped them save time and the cost of carrying vegetables from their village to the market. While the phenomenon of a trader coming to a village to collect vegetables from farmers was observed in only one of the program villages, its replication in other intervention villages would have been extremely beneficial in easing the pressure off farmers in selling vegetables. Indicator: Number of crop types grown 135 The traditional agricultural practices required higher quantities of seeds but the productivity was not commensurate with the number of seeds sowed. MAF provided training on vegetable diversity and the number of seeds to be sown. The farmers were trained on seed selection according to nutritional needs, cultural acceptance and potential to meet the year-round nutritional requirements. The program also invested in encouraging farmers to adopt poly￾culture field practices and crop rotation. This not only allowed farmers to sell excess produce in the market but also helped them build resilience for future lean periods, climatic shocks 99 98 97 88 80 0 20 40 60 80 100 Figure 7: Percentage of LRP farmers growing different vegetables USDA Local Regional Procurement Activity Evaluation in Lao PDR Final Report 23 and disasters by way of building their capacities in modern farm techniques, seed regeneration and processing. 136 Interactions with farmers in control villages revealed that they cultivated relatively fewer numbers of crops. Sang-Ek village in the lowlands reported growing field rice along with cabbage, pumpkin and eggplant during the rains, and green cabbage, spring onion, morning glory, lettuce and coriander during the dry season. Control villages in upland and mountain regions saw cultivation only during the rainy season, growing upland rice and a few vegetables such as green cabbage, watermelon, pumpkin and eggplant. 137 At the time of introducing the LRP program, the farmers were largely growing rice, cardamom and rubber. The program provided a total of 11 types of seeds to farmers across 47 villages the first year. Monitoring data from the field reflects that this resulted in the cultivation of six crops on an average in year 1. As discussed earlier, the program strategy changed from the second year onwards, to focus only on 10 model villages. The formation of farmer groups in 10 model villages coincided with a sudden increase in the number of crops cultivated to 14 and thereafter to 20. The program also provided 9070 fruit trees to 907 households across 34 program villages, assisted with the expansion of greenhouses in five villages, and provided fish fingerlings in 30 villages. 138 Traditional agriculture practices in Nalae involved rainfed cultivation only. To overcome this constraint, the program supported in new agricultural practices through the distribution of greenhouse plastic sheets, water pumps and piped water connections that enabled continuous supply of vegetables to schools throughout the year. Primary data collected from the farmers in intervention and control areas revealed wide variations in terms of the number of crops grown throughout the year. While all farmers in control areas cultivated only one crop per year, almost 70 per cent of the farmers in intervention areas had moved to two crops a year. 139 Quantitative data highlights that even after completion of the program, a little over 80 per cent of farmers in intervention areas continue to cultivate vegetables. In contrast, only 10.7 per cent of the farmers from control areas reportedly cultivate vegetables. 140 Farmer groups formed in model villages from year 2 onwards have been particularly useful in helping farmers collectivise and plan future sets of inputs, particularly in terms of growing a wide variety of vegetables to avoid the overproduction of particular vegetables and the area to be cultivated [case study no. 1 in Annexure U]. 141 Qualitative findings suggest that the program provided a standardised set of 11 seed types across 47 villages in year 1. After witnessing the nature of benefits that accrued to program farmers, many non-program farmers also began to grow vegetables for supply to schools and selling in the local market. This phenomenon resulted in increased production of certain types of vegetables, culminating in excess supply, fall in market prices and wastages. 31 69 100 0 0 20 40 60 80 100 Only 1 Season 2 Seasons Intervention Control Figure 9: Percentage of LRP farmers cultivating in one or two seasons (n: intervention=128; control=22) Figure 8: Variety of vegetables grown by LRP supported farmers USDA Local Regional Procurement Activity Evaluation in Lao PDR Final Report 24 Indicator: Percentage of farmers implementing best practices from their farmer training 142 Monitoring data suggests that after the end of semester II, only 19 out of 47 farmers groups (39 per cent) started implementing best practices, in line with the training provided. As the number of focus villages for the LRP program dropped to only 10 from the second semester onwards, the compliance by farmer groups improved to 50 per cent in semester III and 80 per cent in semester IV. Qualitative discussions with farmers highlighted the need for a more stringent and regular on-ground follow-up of program activities. 143 It is noteworthy that while there are many factors which affect farmers’ intrinsic motivation to carry out best practices, some of the most prominent ones highlighted during qualitative discussions included: (1) market linkages or the ease with which farmers could sell their produce in the market, (2) prices fetched for the vegetables in the market, and (3) the assurance for water availability on a sustained basis. The district market was accessible only to a few villages, located in the vicinity of the district headquarters. As a result, farmers from other villages found merit in supplying vegetables to schools either free of cost or at a discounted price to reduce wastage. The contributions also aided the schools in utilising the money for procurement of animal protein for school meals. The farmers viewed such acts as their contribution towards ensuring nutrition and sound health among school children, and hence, experienced a sense of pride and accomplishment [case study no. 2 in Annexure U]. As a result, alteration of farm practices did not result in a substantial increase in farmers’ income, especially in ‘non-model’ villages. 144 The program ensured that success stories of farmers were demonstrated to the extent that other interested farmers could draw learnings from these stories. The program included exposure visits for the farmers. With exposure visits for one or two persons, the entire group benefitted in terms of the knowledge. Farmers who did not have the desired success after the initial intervention were taken to villages where the farmer groups had exceeded the targets. These exposure visits helped the farmers in understanding the nuances and also to identify areas that needed rectifications. The kind of knowledge farmers gathered from these exposure visits included group planning for vegetable cultivation, the quantum of different types of vegetables to be grown, etc. for avoiding overproduction. Further, the availability of peers as solution providers also helped the farmers in adopting the improved techniques and practising appropriate farming. 145 The program focused only on wet markets. If the program boundaries could have been extended and dry markets included as part of LRP, the number of model villages would have been higher than 10. Support to the food supply chain with the introduction of processing and packaging units could have opened up opportunities for landless households as well, along with smallholder farmers. Enhanced Access to Food Supply and Voluntary Contributions to SMP 146 Given that SMP in Nalae was handed over to the government in July 2019, and there were no in-kind food distributions planned for 2019-20, the schools were more or less completely dependent upon farmers and the village community for the supply of vegetables, chicken and meat on a sustained basis for the meals. The 70 70 80 80 39 50 80 80 0 20 40 60 80 100 Semester 2 Semester 3 Semester 4 Semester 5 Target Achieved Figure 10: Percentage of LRP farmers implementing best practices (n=47 villages in semester2 and 10 villages from semester 3 onwards) “The amount of 800 kips is too less and should be increased to at least 1000 kips so that the school could purchase meat and chicken.” – Parents in Village Saloy USDA Local Regional Procurement Activity Evaluation in Lao PDR Final Report 25 schools were scheduled to receive 800 kips per student per day as part of NSMP, but this was delayed and the cash was not sent to the schools at the beginning of the semester. Having said that, parents and VEDC members considered the cash contribution under the program to be grossly insufficient in terms of procuring anything valuable and nutritious. 147 As highlighted above, the farmers contributed vegetables to schools either free of cost or at a discounted price enabling the schools to use the cash for providing additional animal protein like meat, eggs or chicken. In addition to this, parents of children studying in schools and other villagers also provided cash and in-kind contributions every month. The quantum of contributions varied from village to village, ranging between 5000 and 30,000 kips per month, depending upon (i) the requirements of the corresponding VEDC and (ii) capacity of villagers to make such contributions. Indicator: Number of nutritional meals prepared by schools 148 The indicator records the total number of feeding days within LRP schools. While the monitoring records did not capture the exact number of school meals prepared during the reporting period, a broad estimation was carried out based on the number of schools reached and the number of working/feeding days. On average, the feeding days hovered around 90 for a semester, with the program rolled out in a total of 47 schools and reaching 3374 students (1720 boys; 1654 girls). 149 While lack of monitoring data prevents the presentation of the exact number of meals and ingredients used for cooking lunches, discussions with parents, teachers and VEDC members indicated that lunch was provided almost every day in the schools. There have been no instances of a shortage of raw materials due to which the school meal was not provided. Vegetables were provided as part of the meal daily, while fish, chicken and other animal protein were provided as part of the school lunch about two or three times a week. 150 Out of the sampled 15 intervention schools, lunch meals were provided to all children regularly in 14 schools. As we have noted above, the school meals are being sustained without food supplies under the USDA-SMP for the Sep’19-Mar’20 semester, resting largely on the contributions of farmers, parents and the larger village community. It was only in Phouchalae (a non-model village in the upland region) that no school meals were provided since the beginning of the school term in 2019 (April), as there was a lack of clarity regarding the implementation of the program, and the school ran out of budget to procure food. Also, no vegetables were contributed by the farmers to the school for meals in 2019. No school meals were provided in any of the five schools in control villages. 151 Cooking meals for children in schools was considered to be the community’s responsibility. As a result, VEDCs prepared a quarterly roster of cooking schedule, fixing responsibilities of all households on a running basis. In addition to this, VEDCs also recorded voluntary contributions in terms of vegetables and meat, by households, and displayed these on the school notice board. On average, one woman was supposed to ensure the cooking of school meals at least once every one or two months. It is noteworthy that the woman responsible for cooking on a particular day was also supposed to arrange for fuelwood and contribute chicken and meat for the meal. 152 Interactions with parents, however, revealed that each woman had a distinct cooking style which potentially affected the nutrition levels in cooked meals. It was also pointed out that having different women cook on different days could potentially affect the consistency of taste. As a result, parents advocated for hiring a full-time cook, trained in cooking nutritious meals. Figure 11: School lunch provided under LRP “Cooking is carried out on a rotational basis, resulting in no consistency in cooking style, taste and nutrition” – Parents in Village Phahou USDA Local Regional Procurement Activity Evaluation in Lao PDR Final Report 26 153 The program has helped ensure nutritious school meals for children regularly and provided additional livelihood options for villagers, and it has also helped parents save time and efforts in arranging lunch meals for their children [case study no. 2 in Annexure U]. Assurance of a nutritious and diverse lunch for children effectively meant that the parents were free to use this time on their livelihood, increasing household income. This was one of the major reasons for the communities to realise the importance of school meals, and come together to assume ownership of these meals. 154 According to VEDC members, their role under the program included deciding and coordinating with farmers about the types and quantities of vegetables needed for school meals, and convincing farmers to contribute or sell vegetables at discounted rates.35 In the majority of the villages, LWU played a central role in planning and deciding the food to be cooked as part of school meals and ensuring that the procurement from farmers was carried out per this plan. VEDC members, in other words, have played a pivotal role in ensuring the provision of school meals to children regularly. 155 According to the monitoring data, one training on financial procedures and cash-based transfers was conducted for the Department of Education and Sports Bureau (DESB) officials and VEDC members in April 2017. No other training on financial procedures was carried out after this. Discussions with VEDC members revealed that some members have been imparted training under the program relating to (i) storage of food materials, (ii) processing of vegetables to increase their shelf life,36 (iii) cash management and (iv) cooking food. VEDC members in Longkhaean village (mountain region) also reported attending an exposure visit to Oudomxay province on agriculture and livestock-based livelihoods. Nongkha (lowland region), Homchaleun and Salaeung (mountain region) villages, on the other hand, reported that there was no training for VEDC members under the LRP program. 156 Interactions with parents of children in school and VEDC members pointed towards a sense of community responsibility and ownership, in terms of organising and managing the school meals for children in school. Discussions revealed that the community realised the importance of providing regular and nutritious lunch meals to children, and viewed it as their responsibility [case study no. 2 in Annexure U]. 157 By supporting the farmers to cultivate different types of vegetables using improved techniques, the LRP program has ensured nutrition security by using farms from subsistence agriculture (rice) to a more resilient multiple-crop agricultural practice. The program was designed to enable communities to move towards self-sufficiency in supplying vegetables for school meals. The intervention supports families by promoting the supply of vegetables for school lunches. This has facilitated the regularising of attendance of children from poor families. It has also helped in ensuring sustained access to food for children in remote rural areas. Gender Equality and Empowerment 158 The program ensured nutritious lunch meals for both boys and girls on a sustained basis. The program also involved training of women and men smallholder farmers which resulted in increased technical skills related to the cultivation of vegetables. However, the involvement of women farmers was largely manual and devoid of strategic decision-making. While the program encouraged equal participation of women and men, it was primarily the women’s responsibility to cook school meals and procure vegetables and meat, which added 35 2000-3000 kips per kg 36 Making pickles from bamboo shoots, boiling and drying vegetables “Villagers contribute vegetables, bamboo, chicken and meat. Cooks bring their own fuelwood. We announce publically and appreciate families which contribute for school meals” – VEDC members in Village Aome USDA Local Regional Procurement Activity Evaluation in Lao PDR Final Report 27 to their existing workload. The program also lacked provisions to ensure women’s participation in leadership and decision-making roles. Changes in Dietary Diversity Score 159 The indicator of diet diversity measures different food groups consumed within households, providing an estimation of the quality of diet. The indicator divides different food items into a total of 11 food groups and assesses presence/absence of each of the food groups in the household diet. The maximum Dietary Diversity Score (DDS) is 11. 160 DDS among households from intervention villages (7) was found to be higher than for those in control villages (6). Further, scores for model villages across lowland, upland and mountain regions demonstrate that the impact of interventions among model villages was more pronounced in the lowland region than in the other two regions, so far as dietary diversity is concerned. Comparison with end￾line figures indicates that DDS of households in lowland and mountain regions increased to 8, whereas it dropped marginally to 7 in the upland region (Table 3). Despite the drop, DDS within intervention villages (7) of the upland region was still marginally higher than that among control villages (6.6). 161 There has been an increase in dietary diversity in the intervention villages. The primary contributor to this change is the availability of different types of vegetables throughout the year. However, the consumption of fruits, nuts and milk-based products is currently low and needs work to meet DDS of 11. Replication in Other Districts 162 Looking at the LRP program through the lens of cost-effective replication, it is essential to identify critical factors that help create an enabling environment for the program to function effectively. 163 Presence of farmer groups: As we have seen, the program focused on only 10 ‘model’ villages in year 2, which involved forming and working with 10 farmer groups, each comprising 10 farmers. It was observed that working with farmer groups was relatively more effective than working with individual farmers as collectivisation resulted in better transmission of technical knowledge and sharing of seeds and tools. The farmer group as a platform is extremely effective in helping farmers collectivise and plan future sets of inputs, such as deciding on the types of vegetables to be grown to avoid the overproduction of particular vegetables and the area to be cultivated. In terms of replication, therefore, areas that already have farmer groups would be better placed for an intervention like LRP. 164 Access to markets: One of the indicators to measure the success of the LRP program was its contribution to augmenting farmers’ income through the sale of vegetables to schools and open markets around the year. Field observations juxtaposed with stakeholder discussions indicate that farmers have been able to supply vegetables to schools regularly and in both the semesters. The surplus production enabled the farmers to donate vegetables instead of selling them to schools. However, the added intent of augmenting farmers’ income through the sale of vegetables in the open market was only partially successful. Villages which did not have access to markets, either due to distance or lack of transportation facilities, were not able to sell their produce, resulting in a negligible increase in their income levels. Drawing knowledge from LRP-supported farmers and with support from DAFO, non-program farmers also produced vegetables which became an additional challenge in an over-supplied local market. 165 Availability of water: Observations during data collection highlighted the importance of ready availability of water for cultivating vegetables. Villages which had access to water or a Table 3: Comparison of Dietary Diversity Score Type Baseline End-line Program-Lowland villages 4 8 Program-Upland villages 8 7 Program-Mountain villages 7 8 Control villages 6 6 USDA Local Regional Procurement Activity Evaluation in Lao PDR Final Report 28 mechanism to draw water from natural resources were better placed in terms of sustainable vegetable cultivation. 166 The Most Significant Change (MSC) analysis found that while five out of the seven farmers witnessed a positive shift in their incomes, which they attributed to their involvement in LRP, the remaining two farmers did not experience any tangible benefit from participating in the program. The analysis under MSC attempted to ascertain if common patterns were emerging after assessing the two groups of farmers – (a) who witnessed a positive tangible change as a result of the program [five farmers], and (b) those who did not observe any positive tangible change after the program [two farmers]. The analysis showed that [annexure U for all case studies] the five households which experienced tangible benefits as a result of program participation continued with vegetable cultivation and (i) were ready to take risks and be engaged in multiple livelihoods without fear of failure, (ii) had at least one member earning fixed income or involved in procuring and selling cash crops, thereby reducing dependence on agriculture, (iii) were well-educated, (iv) either lived in large villages (which served as a market for vegetables cultivated) or in villages close to the district market, (v) or cultivated a wide variety of vegetables to counter overproduction of a particular vegetable, or (vi) adopted practices that were usually not followed by the rest of the farmers in the village. 167 On the other hand, the farmers who, despite gaining from the technical training and provision of tools and seeds, did not experience any major shift in their income levels and hence discontinued vegetable cultivation (i) lived in villages far away from the district market, (ii) or lived in villages that already had sufficient supplies of vegetables from the uplands or the forests, or (3) were unable to tackle oversupply of a particular vegetable as they continue performing the same set of activities that all other farmers were practising. These factors need to be considered for replicating the program. 168 The first pillar of the LRP program, which aimed at supporting school meals for children by way of sustained supply of vegetables, involved provisioning of seeds, manual tools such as sickles and buckets, and necessary technical training for cultivation. This appears to be a cost-effective yet extremely potent factor as it ensured that (1) enough farmers were attracted towards the program because of the provision of free seeds and tools, and (2) there was a permanent investment in the farmers’ technical skills. 169 From discussions with farmers and DAFO officials, it emerged that the farmers had initially joined the program because of the tangible benefits being offered, in the form of seeds and manual tools but without awareness of the full import of the envisaged benefits. However, with training and continued support from DAFO officials, the farmers were able to reap the benefits of the program through a huge production of a variety of vegetables. The production and availability of a variety of vegetables also triggered interest among the non-program farmers to use these improved farming techniques for growing vegetables for self￾consumption and selling in the market. This shows the potential for replicating the program. 2.3. Efficiency of LRP 170 This section evaluates the efficiency of partnerships formed with the government and platforms used at the community level for implementation of the program. It assesses (i) the adequacy, sufficiency and timeliness of support provided by DTEAP, PAFO and DAFO for solving implementation issues (EQ9&11); (ii) efficiency of farmer groups in utilising the technical support for agriculture (EQ10); and (iii) flexibility and adaptability of the program to respond to the need for course corrections (EQ12). Partnership with Government Agencies for Implementation 171 The program strategised to partner with MAF and MoES for implementation of different components. With LRP focusing on the promotion of agriculture for nutrition, DTEAP, PAFO and DAFO were identified as the key implementing partners. 172 DTEAP, PAFO and DAFO officials were provided with capacity building training on program implementation and management as well as on various components of the program. The USDA Local Regional Procurement Activity Evaluation in Lao PDR Final Report 29 purpose of capacity building was to enable them to support the community in resolving their issues. It was envisaged that the trained officials would also act as master trainers to further train their counterparts, colleagues and the community. 173 DAFO works in close collaboration with the community and is more aware of the regional disparities and needs of communities. As a result, the selection of farmer group beneficiaries was carried out by DAFO in consultation with VEDCs. The selection of beneficiaries was based on land availability as also willing to work and contribute to SMP, to ensure successful implementation and sustainability of the program. This resulted in the participation of both men and women smallholder farmers, one of the prime concerns of the program. 174 DAFO officials provided agricultural training to all 47 farmer groups and LWU members. The training covered soil improvement, farmer group management, marketing and crop cycles. The training was followed up with constant monitoring and hand-holding of the farmer groups. 175 DAFO was the first point of contact for farmers seeking technical support for agriculture and for resolving issues related to farming activities. Discussions with farmer groups confirm that DAFO officials along with Monitoring Assistants (MAs) were readily available to resolve the issues faced by them. The farmer groups also indicated that due to the support provided by officials, they were able to grow different types of vegetables. 176 From the discussions with MAF, it emerged that the intervention support provided by the department officials for supporting the LRP program has also resulted in enhancing the knowledge and skills of the officials. MAF’s perception is that agriculture department officials at Nalae are not only well equipped to support farmers, they can now also impart training to their counterparts from other districts. 177 This highlights the importance of the capacity building of officials around program activities as well as the efficiency of government platforms in transferring agriculture technical knowledge to farmers. Both provincial and district level officials believed that the training provided to them under the program was very useful. The Deputy Director at Nalae Agriculture Office observed: ‘… exposure visits conducted under LRP helped us in understanding the mistakes we were making and correcting them…’ Further, DAFO Nalae stated: ‘… regular support from the MAs helped the DAFO team also as they were able to ask questions around improved agriculture techniques.’ 178 Further, it was found that the government departments were quick to respond to the changes made in the program strategy and activities from the second year onwards. This reflects the degree of their dedication towards achieving the intended outcomes and further justifies the decision of partnering with the government for program implementation. 179 As for the challenges faced by implementing partners, it was revealed during the discussions with DTEAP officials that there was a lack of coordination between WFP and DTEAP. DTEAP officials observed that, after the initial engagement between WFP and DTEAP, WFP started working directly with DAFO. This might impact the long-term sustainability of the program as DTEAP might not give the LRP program the desired importance and direct PAFO and DAFO to focus on other departmental activities. It must be kept in mind that DAFO reports to DTEAP through PAFO; as a result, if DTEAP is not kept in the loop, implementation of the program will suffer. It could not be ascertained whether this operational arrangement influenced the reduction of LRP villages from 47 to 10. 180 The WFP program team consisted of two MAs who worked as facilitators, helping DAFO to carry out its activities. The monitoring of the program activities was left to the agriculture department and MAs used the data from DAFO to fill in their semi-annual monitoring reports. It was found that there were no separate monitoring templates designed by WFP for collecting data related to planned activities, their outcomes and deliverables on the field. This might have resulted in the loss of the opportunity to gain first-hand feedback from target beneficiaries on the implementation of program activities. Also, timely course correction of activities would have been much easier with proper monitoring templates. Further, it was USDA Local Regional Procurement Activity Evaluation in Lao PDR Final Report 30 found that the data presented in the performance indicator matrix of semi-annual monitoring reports lacked clarity on various indicators. This posed difficulties in interpreting the performance of the program on certain aspects. A rigorous monitoring mechanism would have helped in avoiding such a situation. WFP had designed a technology-based monitoring system which was eventually not used for monitoring. 181 One of the critical constraints of implementation was the location of the villages. Lack of proper roads limited visits of officials. The problem was aggravated during the rainy season when the roads became muddy. Government officials observed that reaching the intervention villages in a remote area during the rainy season was difficult and posed a huge challenge in carrying out implementation activities. Efficiency of Farmer Groups 182 Aligned with MAF’s strategy of forming farmer groups, the LRP program also created farmer groups for transferring knowledge and support for improved and efficient farming. It was evident from discussions with farmer groups that working in groups resulted in smooth implementation of the program and also benefitted the farmers in several ways. Working in groups has developed cohesiveness among the farmers and this has enabled them to support each other better through the provision of seeds, knowledge sharing, labour and other aid as and when required. It has also given them better bargaining power for selling vegetables. 183 A farmer group is an efficient platform for working towards the common goal of ensuring nutrition and food sufficiency through locally grown vegetables and income enhancement. With time, the groups also gained experience and matured to undertake collective planning around types of vegetables to be grown on a rotational basis. This helped them in avoiding wastages because of overproduction that results in decreased demand and lower prices. Moreover, these groups were efficient in transferring knowledge to non-beneficiary farmers in the village. It was reported during field visits that beneficiary farmers had transferred their knowledge to other keen farmers in the village. Furthermore, from the implementation point of view, it was more efficient to work collectively in groups for passing off the information, developing a sense of ownership, cross-learning, etc. 184 As an alternative approach, working with individual farmers would have been less efficient as it would have increased the cost of reaching out to the intended beneficiaries; also, the opportunity to build social capital and a knowledge bank in terms of trained farmer groups within the village would have been lost. Flexibility and Adaptability of Program 185 The efficiency of the program was evaluated from the flexibility and adaptability point of view. The nature of course corrections in program design and implementation activities, the reasons thereof, challenges in implementing them, and implications of the changes were analysed. 186 The program had planned for interventions across 49 villages but due to construction of a dam in two villages, the number was reduced to 47 in the initial phase itself. From the second year onwards, the focus of the program shifted to commercial aspects, working towards ensuring income enhancement and livelihood strengthening activities in 10 villages. 187 Based on the willingness of farmers and other community members in intervention villages to work towards income enhancement, 10 model villages were identified by DAFO. Additional interventions around livelihoods were planned in these villages, keeping in consideration village size, access to markets and availability of land and labour. 188 As for the remaining 37 villages, they did not receive any additional support from the second year onwards. However, support of 800 kips per student continued in all 47 villages; exposure visits were planned for cross-learning and motivating farmers to work towards income enhancement. This ensured that the program focused only on interested farmer groups looking for an improved return on investment. USDA Local Regional Procurement Activity Evaluation in Lao PDR Final Report 31 189 It was found that due to this change in the program activities from year 2 onwards, a large component of the budget could not be utilised by the end of the program. Therefore, a partnership with LWF was formed to ensure that the budget was utilised efficiently towards targeted implementation activities and objectives. 190 Unspent funds of about USD 0.37 million (about 37 per cent of the program fund) indicate that farmer groups in new villages could have been supported under the program. But necessary action was not taken at an appropriate time. As a result, WFP had to engage LWU to utilise the LRP funds through their livelihood programs in the LRP program areas. 191 While it is understood that the current procedure of getting government clearance for supporting communities in the newer geographies is quite tedious and time-consuming, WFP could have utilised the fund for supporting communities in Vieng Phoukha province. WFP had initially taken government approval to work in Vieng Phoukha and Nalae but then had to limit the activities only to Nalae due to limited availability of funds. Thus, with the availability of funds after the first year, a few villages in Vieng Phoukha could also have been supported under the LRP program in the second year. 2.4. Impact of LRP 192 Following the ToR, the evaluation design included an assessment of the impact of the program. Since the program concluded in June 2019, efforts to capture and analyse the information sought to focus on the short-term changes rather than the actual impact. In this context, the impact of the program has been analysed primarily through three key evaluation questions: (i) the effects of LRP activities on SMP (EQ13); (ii) the intended and unintended effects on direct and indirect beneficiaries (EQ14); and (iii) the use of new agricultural techniques and knowledge (EQ15). All the three evaluation questions have also been examined through the lens of gender and human rights, to assess the impact of the program on reducing discrimination and ensuring equality and inclusion, as well as on the quality of life, income and drudgery. School Meal Program 193 The LRP program was envisioned to supplement SMP and particularly ensure the sustained availability of locally grown nutritious vegetables. The program successfully built the capacity of women and men smallholder farmers for growing vegetables using modern farming methods and provided them with seeds and manual tools. As a result, farmers grew vegetables, which were procured by the schools for meals. In many cases, it was observed that the farmers contributed vegetables to the schools free of cost [case study no. 2 in Annexure U]. 194 One of the major impacts of the program has been that school meals are continuing uninterrupted in 14 out of the sampled 15 intervention schools. This is even though these schools have been handed over to the government but are yet to receive the SMP money of 800 kips per student per day for the current semester. 195 In villages where farmers contributed vegetables to schools free of cost, the unutilised amount was usually used to buy meat and chicken from the local market. Discussions with schoolchildren from standards III-V indicated that on average, the children were consuming non-vegetarian meals three times a week. The findings were also validated during discussions with parents who stressed that the meals provided in schools are extremely nutritious, having different types of vegetables and meat. 196 The program has also been successful in terms of teaching the communities the importance of nutrition and the idea of including vegetables in meals. As a result, the community members have demonstrated an increased sense of ownership regarding managing and arranging for nutritious school meals for children. Field observations suggest that parents, “We are satisfied by the quantity and quality of food. Menu is decided by the teachers. Meat is also included regularly in the meals, ensuring proper nutrition.” – Parents in Village Hatnaleng USDA Local Regional Procurement Activity Evaluation in Lao PDR Final Report 32 and community members in general, were open to contributing vegetables, meat, chicken and firewood and offering services to cook school meals. Quantitative data highlights that high proportions of children in standards III-V recalled (84%) and reported consuming (72%) at least five vegetables as part of school meals in the week before the survey. 197 Discussions with district and provincial level officials, teachers, parents and VEDCs revealed an improvement in children’s ability to concentrate and their learning outcomes post SMP and LRP. VEDCs and parents also drew a link between the provision of nutritious school meals and children’s interest in attending school, which was subsequently validated by PESS officials. The PESS official stated that “..Both kids and parents have benefited from LRP. The number of children not coming to school has reduced over the last year from 4 per cent to three per cent, and now we are targeting to reach two per cent by next year. Parents are content that their children get good food at school and they get more time to work without worrying about the children…” [case study no. 2 in Annexure U] 198 The other advantage of the LRP program has been the increased awareness among the parents, farmers and other members of the village on fulfilling the nutritional needs of children both at home and school. Due to this desire, they are also growing vegetables like spring onion in big flower pots. Discussions with parents and VEDC members also revealed a reduction in the household expense on cooking because of the provision of nutritious lunches in schools. This also meant that both mothers and fathers could work for longer hours without worrying about their children’s lunch. 199 Discussions with parents and VEDC members reflected a certain sense of ownership and a feeling of responsibility among the community members towards managing and arranging for school meals. VEDC members have played a central role in making the community members aware of the importance of school meals and the need for making voluntary contributions, in terms of vegetables, meat and chicken, fuelwood, cash or cooking food for children. Unintended Effects of the Program 200 It was found that as a spillover effect, as well as due to the transfer of knowledge from the beneficiary to non-beneficiary farmers within and across villages, farmers started growing similar vegetables. This resulted in overproduction and oversupply of vegetables in the market, reduction in prices and increased wastages. Therefore, coordinated planning around types of vegetables to be grown is critical [case study no. 1 in Annexure U]. 201 Another challenge that has the potential to distort the NSMP implementation is the farmers’ choice of growing different types of vegetables. Because of their increased knowledge of improved agriculture practices, some farmers could switch to growing vegetables that have higher cash value in the open market rather than growing vegetables required for school lunches. VEDC and MAF need to support farmers in keeping a balance between cash vegetables and nutritious vegetables. 202 The results of farmer training in improving the farming intensity and productivity have seen an increase in demand for such training from the farmers. Responding to the demands, Nalae DAFO has established an Agriculture Technical Service Centre for providing training and seeds to farmers. Use of Improved Agriculture Techniques 203 The second year of the LRP program saw the formation of farmer groups in 10 model villages. These groups resulted in increased levels of camaraderie among the intervention farmers, reflected in the sharing of seeds and tools, and transmission of knowledge to non-program farmers. Farmer groups also started to hold meetings to discuss the choice of vegetables for cultivation, keeping in mind the demand and supply mechanisms, to avoid cases of overproduction. “We are no longer worried about children’s meals. Children get nutritious food, whereas we get more time to work. Also, children are now eager to attend school” – Parents in Village Namhaeng USDA Local Regional Procurement Activity Evaluation in Lao PDR Final Report 33 204 Farmers in 13 out of 15 sampled intervention villages affirmed changes in their agricultural practices after undergoing training under the LRP program. Such changes have increased the quantity and quality of the produce. The value of sales of vegetables as recorded in the monitoring data amounted to USD 71,937 during October 2018-March 2019. Provision of greenhouse plastic sheets and piped water connections in model villages has also ensured that the farmers can grow vegetables during both wet and dry seasons, garnering multiple cycles of each vegetable. 205 Discussions with farmers from model, non-model and control villages indicate the impact of the LRP in improving the vegetable production. While 11 per cent of the farmers from control villages, about 74 per cent from the non-model villages and 94 per cent from the model villages were selling their surplus vegetables in the open market. Nearly three-fourths of the farmers from non-model villages continued involvement in vegetable selling is an indication of the success of the support provided to farmers under the LRP program. 206 As regards the increase in income, only one-third of the farmer groups (five of 15) reported an increase in income in the past two years. In almost all the sampled villages of the mountainous region, particularly Homechaleun and Sainamthip, farmers reported inaccessibility of markets. Despite being unable to sell vegetables in the markets, farmers continued to grow vegetables in their capacities to support school meals and self￾consumption, indicating a change in dietary practices to include nutritious food in their diets. 207 VEDC members from the majority of the villages (10 out of 15 intervention villages) reported a change in the overall quality of lives of community members as a result of improved farming methods adopted by them and the change in their attitude towards nutrition. For instance, in Sainamthip village (mountain region), farmers reported that their prime reason for growing vegetables was to supply to schools and self-consumption, and not for sale as the market was far off. 208 Parents from the majority of the program villages reported that they were more willing to send their children to school without worrying about their nutrition and lunch meal (in nine out of 15 villages). This helps parents to take up more work during the daytime (in 12 out of 15 villages). Interactions with parents also pointed towards greater savings as a result of the reduction in household expenses on food (in 14 out of 15 villages). Gender and Human Rights Impact 209 The program design was gender-neutral across all its components. As per the performance indicator matrix, an almost equal number of men, women, boys and girls were to benefit from the program. However, no targeting was done to specifically include the women farmers or provide them with additional support depending on their requirements. The school meals were equally distributed between boys and girls. While the program encouraged the participation of women farmers, it did not display any additional activities to ensure their participation. 210 Both SMP and LRP promoted the sharing of cooking responsibilities between men and women. However, it was observed that the onus of cooking school meals fell only on women, which added to their existing workload. Similarly, procurement of vegetables and meat was primarily the teachers’ responsibility, shared with representatives from LWU, highlighting high levels of involvement for women. Going forward, it will be essential for VEDCs to encourage men to participate in the cooking of school meals. To further enhance the gender focus, it is imperative to ensure women participate in strategic as also leadership roles. 2.5. Sustainability of LRP Figure 12: Greenhouse supported by LRP USDA Local Regional Procurement Activity Evaluation in Lao PDR Final Report 34 211 This section seeks to assess the extent to which the outcomes achieved under the program can be sustained after the withdrawal of the LRP program, and the critical conditions required and available for ensuring sustainability. In this context, the sustainability of the program has been assessed to answer the following evaluation questions: (i) capacity building of farmers, MAF officials and other partners (EQ17); (ii) increased ownership of community-driven school lunches (EQ18); (iii) additional aspects for sustaining the LRP program (EQ19); and (iv) factors necessary for replicating the program (EQ20). 212 Sustainability is driven by a combination of stakeholder capacities, role clarity, resource availability, ownership and the intent of stakeholders. These elements play a critical role in ensuring the continuity of a program after the exit of the external agency. In the long run, communities and the government have to work in close coordination and shared understanding to sustain and also improve the outcomes. Capacity Building of Farmers, MAF Officials and Other Partners 213 The LRP program has been successful in bringing together all critical stakeholders at both the demand and supply ends through various program activities. MAF and MoES, and their respective departments at the provincial and district levels, played a crucial role in planning and implementing the program activities. The program also involved key participation from VEDCs, school authorities, parents and farmers. This is likely to help in sustaining the program after the withdrawal of the USDA-WFP support. 214 The LRP program invested in the training of selected women and men smallholder farmers on modern agricultural practices and providing them with seeds and manual tools to support them in growing vegetables. Peer-to-peer visits helped them to acquire knowledge on differentiating between vegetables to be grown during the dry and wet seasons. While the program worked only with 10 households per village, field observations suggest that knowledge about modern agriculture techniques were subsequently diffused to non￾program farmers as well. A permanent increase in the capacity levels of the program as also non-program farmers is expected to result in improved farm practices on a sustained basis. As a consequence of this, the farmers are expected to have higher productivity of vegetables, which is expected to improve self-consumption and supply for school meals. At the same time, this increase in productivity may or may not result in a substantial increase in income for all farmers. 215 The program has focused on the capacity building of government officials, which is an essential component for ensuring the sustainability of farmers’ technical capacity relating to modern agricultural methods. One Agriculture Technical Service Centre (ATSC) was supported under the program on the request of DAFO. This centre is supposed to not only provide seedlings to farmers but is also positioned as a learning centre. Having said that, no effects of the initiative were observable during the primary data collection, indicating it is still at a nascent stage. Despite the withdrawal of the LRP program, the farmers still have access to the officials who are providing technical support to them. This holds promise for effective implementation of the NSMP program in the district. 216 DTEAP has expressed interest in working with the farmer groups to provide them with market linkages. The program has, therefore, provided a set platform that can be leveraged by future interventions around livelihoods, agriculture in particular. Ownership of Community-Driven School Lunch 217 The program design ensured that the farmers grew a wide range of nutritious vegetables, a portion of which was either sold or contributed towards the school meals. Linking farmers with schools resulted in a certain sense of ownership among the community members relating to the provision of school meals, with high instances of contribution in the form of vegetables, fuelwood, chicken, meat and cash. It appears that the village community has taken the initial steps towards ownership of school meals on a sustained basis. If continued for a longer period, it is envisioned that the village community would assume complete charge of planning for and delivering school lunches for children. However, GoL should USDA Local Regional Procurement Activity Evaluation in Lao PDR Final Report 35 continue supporting NSMP so that in case of constrained supply of vegetables locally, the schools can purchase from other markets. Discussions with DESB officials revealed that the budget for NSMP has been approved and the funds were being transferred to the schools. Sustaining LRP program 218 While the LRP program is underpinned by farm-based activities, off-farm and non-farm￾based activities are also critical. This is because not all villages are situated in terrains conducive for agriculture. Hence, schools in such villages will be dependent on vegetables from the market. So, if the smallholder farmers and landless households are supported with other livelihood activities, they would be able to support the schools with cash equivalent to the vegetables they were expected to provide to the school. 219 LRP has been able to demonstrate the timely release of cash to schools for procuring vegetables. The schools have utilised this fund for purchasing vegetables, eggs and meat depending upon their requirements. This augurs well for the future of these villages, which will eventually be supported under NSMP. The exposure of VEDC members in terms of interacting with government officials will aid in the timely release of funds in future. Further, the training of VEDC members on the maintenance of accounts will help in continuing the process under NSMP. 220 As we have seen, 800 kips as provisioned under NSMP is insufficient to meet the desired nutritional requirements of children. While the perception of parents and VEDC members was that at least 1600 kips per student per day should be provisioned as an egg costs about 1000 kips, this proposition is unlikely to find acceptance as it would put a big financial burden on GoL. 221 WFP’s association with LWF provided livelihood support to select households in program villages by way of provision of cattle for animal husbandry, walking tractors to improve agriculture and cash credit for weavers. It is envisioned that the livelihood support would result in improved livelihoods and increased income. Replicating the LRP Program 222 The evaluation findings identified certain critical factors within the ecosystem which are essential for the program to be scaled up or replicated, and hence must be taken into account while planning for such actions. 223 Given that strengthening farm-based livelihoods formed a crucial component of the program, program participation needed to lead to a demonstrable increase in income as a natural effect. Farmers who lacked access to markets were not able to witness a substantial change in their income, which may affect sustained program participation in the future. 224 Case studies showcased that households which had a diversified livelihood mix, and relatively lower dependence on any one particular livelihood option, were better placed to undertake risks and hence open to new initiatives. On the other hand, excessive dependence on one livelihood option generally reduces risk appetite and hence may affect the household’s ability to try out new initiatives. 225 In the current program, women within the village cooked food in the school on a rotational basis. In the absence of proper training for the cooks, the cooked meals lacked consistency, in taste. While all students ate the meals regularly, and nobody reported skipping or wasting meals, however, there is a potential risk of these students skipping them in future. As a result, it would be essential for the program to invest in training of cooks within the communities so that the food is palatable to the children. To this extent, a customised menu book will also be helpful. 3. Conclusions and Recommendations 3.1. Overall Assessment/ Conclusions USDA Local Regional Procurement Activity Evaluation in Lao PDR Final Report 36 226 LRP, along with the McGovern-Dole support for school meals, supported MoES in promoting school lunches as well as contributed towards the Agriculture Development Strategy to 2025 through investments to increase multiple crop agricultural practices. The program design enabled communities to move towards self-sufficiency in supplying vegetables for school meals and ensured sustained access to nutritious food. It also ensured that both the supply side and demand side issues about cultivation and sale of vegetables by smallholder farmers were addressed. 227 The LRP program was well aligned with GoL policies on agriculture and education supporting dual objectives of improving nutritional and educational outcomes. It offered a standardised set of inputs to farmers, in terms of technical assistance, seeds and agricultural tools, without taking into consideration the variations that exist across mountainous, upland and lowland regions. 228 The program established farmer groups as platforms enabling farmers to share technical knowledge and plan inputs such as types of vegetables to be grown to avoid the overproduction of particular vegetables. Lack of access to markets for farmers turned out to be a major weakness of the program, resulting in only a nominal increase in income levels. 229 The program lacked specific provisions to ensure women’s participation in leadership and decision-making roles. 230 Lack of regular usage of the monitoring tool for collecting program monitoring data prevented timely identification of key issues and execution of course corrections. 231 Linking farmers with schools resulted in a certain sense of ownership among the community members, even as they made contributions to school meals in the form of vegetables, fuelwood, chicken, meat and cash. 232 There was a lack of coordination between WFP and DTEAP, which might negatively impact the long-term sustainability of the program. 233 Farmers’ decisions concerning the selection of vegetables for cultivation may subsequently be determined by the prevailing rates and market demand instead of the nutrition values. 234 Lack of farmers’ engagement in the regeneration of the mother seeds provided in the first year can be viewed as a risk of such farmers subsequently opting out of the cultivation of vegetables. 235 The program by design is gender-inclusive. However, there was no emphasis on ensuring female farmers get the benefit of exposure visits and exchange programs. As a result, male members took advantage of the exposure visits while female members were involved with the manual aspects of farming. 236 Cooking of the school lunch by default has become the responsibility of women without any additional benefits. 3.2. Good practices and Lessons Learned Good practices 237 The program adopted the approach of collaborating with multiple partners and stakeholders to ensure successful implementation. The success of the program can also be attributed to the fact that it brought together both demand (community) and supply (government) sides on one platform. Partnership with the government not only facilitated the implementation of the program but also ensured sustainability through strengthening capacities of the district and provincial officials. Collaboration with community organisations such as VEDCs and LWU strengthened the support system at the community level, ensuring greater ownership of the program activities. 238 Working with the farmer groups appeared to be a good move to get farmers at the village level under one platform. The strategy helped in building a sense of camaraderie among farmers, enabling them to share knowledge, tools and seeds among themselves. Farmers USDA Local Regional Procurement Activity Evaluation in Lao PDR Final Report 37 meetings also helped them choose vegetables for cultivation, keeping in mind the prevailing demand and supply in local markets. 239 Investing in the capacity building of 10 farmers from every intervention village resulted in increased technical knowledge concerning cultivating vegetables. The move also subsequently resulted in the transmission of this technical knowledge to non-program farmers. It is noteworthy that such transmission of knowledge is purely organic, using informal channels of communication and hence received higher acceptance. Lessons Learned 240 Reduction of the program geography during implementation highlights the need for carrying out a need assessment study before program design for a better understanding of the needs and aspirations for each region and customising the activities accordingly. 241 As observed earlier, lack of commercial gains in the first year was primarily due to the absence of markets and that led to non-participation of 37 out of the 47 groups. Further, the program’s primary objective was to support farmers in ensuring a continuous supply of vegetables to school meals. Consumption of diverse food items and the sale of surplus vegetables were additional benefits possible to be accrued under the program. This was not communicated to the farmer groups effectively leading to despair among farmer groups. 242 At the same time, it was also observed that the program did not communicate about potential commercial opportunities for farmers which could have increased their income. Overlooking commercial aspects, particularly those targeted at increasing farmers’ income could be detrimental to maintaining their interest and subsequently, sustaining the program. 243 Further, lack of coordination and planning among farmer groups of different villages led to the production of similar types of vegetables in large quantities, which resulted in oversupply and reduction in prices. Any such program in the future, therefore, must consider (i) educating farmers about the demand and supply aspects, and (ii) bringing all farmers from a village on one platform to plan the potential vegetable production, keeping in mind the demand and supply constraints. To minimise post-production wastage, opportunities for increasing the shelf life of the vegetables need to be identified and supported. 244 WFP partnered with LWF in June 2019 to meet the program deadlines in terms of using the unutilised budget and completing program activities. In this partnership, the community was supported with assets such as walking tractors, livestock and cooking stoves, or were provided cash credit for weavers. While the effect of this component could not be assessed in this evaluation as the intervention was still underway, provision of big assets such as walking tractor does not seem to be in line with the original spirit of the program. While it is understood that sometimes alterations to the original program design become imperative, the changes must be in sync with the initial idea of the program. 245 The program had two Monitoring Assistants who interacted with the community regularly and collated monitoring data collected by DESB and DAFO. WFP designed a specific monitoring tool in KOBO (mobile/tablet-based monitoring data collection application) to track project implementation process and its planned outputs. However, it was not regularly used during the two years of intervention. A strong monitoring system would have provided quick checks to assess the direction of the program and improve the response time to issues. 246 Going forward, support should be provided to forge and formalise the trader-farmer group partnership. The intent of forming the farmer groups was to leverage the economies of scale and have bargaining power for the smallholder farmers. A formal trader-farmer group partnership will also help in balancing the variety of vegetables to be produced for meeting the nutritional needs of the schoolchildren and community vis-à-vis the market demands. USDA Local Regional Procurement Activity Evaluation in Lao PDR Final Report 38 3.3. Recommendations 247 Presented below are the recommendations, rationale and proposed actions validated during validation workshops at Nalae and Vientiane. Table 4: Recommendations, rationale and proposed actions No. Recommendation Rationale Proposed actions Type Validated Ministry of Agriculture and Forestry (MAF) 1. Providing technical support for small land farming (Para 120 & 122): LRP focused on the training of intervention farmers on technical aspects related to vegetable cultivation. However, there were still certain aspects, such as seed replacement or building resilience to climate change, which could further be emphasised while imparting technical training to the farmers. a. Organise training on aspects such as seed replacement or building resilience to climate change. b. Create a yearly calendar for such training and follow-up sessions to ensure maximum participation from farmers. c. Develop the technical capacities of individuals at the village level who could further train fellow farmers on these aspects. Strategic Yes 2. Providing farmer groups with technology for self￾monitoring Dashboard for DAFO to analyse monitoring data and take corrective actions (Para 179): Due to lack of monitoring, majority farmers ended up cultivating the same set of vegetables, resulting in overproduction. (Para 179): Due to lack of monitoring tool, DAFO could not guide the farmers which resulted in most of the farmers ended up cultivating the same set of vegetables, resulting in overproduction. a. Create a self-monitoring system for farmer groups for recording details about types and quantities of vegetables cultivated. Access to real-time data would enable DAFO to carry out immediate corrective actions. b. Create a dashboard for DAFO officials connected to self￾monitoring tool for farmers, that will present information regarding the volume of vegetables sowed and produced by farmers across villages, enabling them to identify issues and execute timely course corrections. Operational Yes 3. Formalisation of farmer groups (Para 110): Farmer groups, formed in model villages under the LRP program, provide platforms to farmers to come together, plan their sowing strategy for the next season, and ensure a wide diversity of vegetables. However, these farmer groups are largely informal structures, without clearly laid-out roles and responsibilities for its members. a. Formalisation of farmer groups by supporting them in conducting regular meetings, selecting position holders, and delineating roles and responsibilities for different members. b. Enabling selection of women in decision making positions. Operational Yes 4. Providing irrigation support for farmers (Para 110): Several villages in Nalae district lack irrigation facilities. Absence of water sources, especially during the lean season, prevents farmers in such villages from cultivating vegetables. a. Provide water connections to households ensuring access to water on a sustained basis. Irrigation facilities would result in the cultivation of vegetables throughout the year, ensuring their regular supply for school meals. Operational No 5. Promoting off-farm and non-farm activities (Para 123): Villages, particularly in the upland and mountainous regions, depend on other livelihood options such as animal husbandry, cassava or rubber plantation, instead of vegetable cultivation. a. Identify livelihood options best suited for mountainous, upland and lowland regions, and promote them, instead of offering a standardised solution of cultivating vegetables. Strategic No 6. Easing the process of organic certification (Para 131): The LRP program has enabled the farmers to shift from chemical-based fertilisers to compost. As a result of this, they are now growing organic vegetables, which can a. Ease the process of issuing organic certificates to farmers. Individuals who receive the organic certificate could be treated as trainers, providing necessary support to other Strategic No USDA Local Regional Procurement Activity Evaluation in Lao PDR Final Report 39 Table 4: Recommendations, rationale and proposed actions No. Recommendation Rationale Proposed actions Type Validated fetch a relatively higher price, especially in towns. However, the current process of issuing an organic certificate is extremely tedious and cumbersome. farmers in the group to take up organic farming and obtain the necessary certification. 7. Engaging traders for purchasing vegetables from far￾off villages (Para 129): Majority of the villages do not have access to markets and hence are farmers unable to sell their produce, resulting in wastages and no significant increase in incomes. a. Tie-up with traders to ensure that they visit far-off villages to procure vegetables directly from farmers, charging a relatively higher commission in return. Operational No Ministry of Planning and Investment (MoPI) 8. Ensuring coordination between MoPI and MAF for improving the value chain, and reaching out to newer markets (Para 240): Lack of coordination and planning among farmer groups of different villages resulted in the production of similar types of vegetables in large quantities, which led to overproduction and, subsequently, reduction in prices. To reduce wastages due to overproduction, it is essential that; (i) farmers produce a variety of vegetables, (ii) farmers are linked to new markets, and (iii) farmers are trained on vegetable processing for increasing the shelf life of vegetables. a. Training women and men from villages on processing and storing vegetables for consumption during the lean season. b. Public investment for setting up processing units at the cluster level. c. With increased shelf life, newer markets such as Luang Namtha and other towns would become available and would also contribute to fetching higher prices. Strategic No Village Education Development Committees (VEDCs) 9. Formalising the role of cooks (Para 150 & 151): Different women from within the community cook school meals in turn. Discussions with parents revealed that each woman had a distinct cooking style which potentially affected the nutrition levels of cooked meals. a. A specialist cook from within the community should be provisioned for each school. The cook must be trained to cook meals according to the menu book to ensure retention of nutrition and consistency of taste. Operational No 10. Promoting dietary diversity among all the community members (Para 160): After vegetables are contributed for school meals, the remaining produce is either consumed by the farmers’ families or sold in the market. It is essential that the villagers are aware of the importance of nutrition and including vegetables in their meals. a. VECD members must be trained to ensure the dissemination of knowledge among fellow villagers concerning the importance of dietary diversity including nutritious properties of different vegetables. Operational No Farmers 11. Monitoring of vegetables grown and quantity produced (Para 179): Due to lack of monitoring, most of the farmers ended up cultivating the same set of vegetables, resulting in overproduction. a. Create a self-monitoring system for farmer groups, encouraging them to record details about the types and quantities of vegetables cultivated. Access to real-time data would enable DAFO to carry out immediate corrective actions. Strategic Yes 12. Ensuring that the groups meet regularly to decide (Para 110): Despite all the positives, farmer groups are largely informal structures, without clearly laid-out roles and responsibilities. As a result, there is no mechanism to a. Inclusion of non-intervention farmers as part of farmer groups. Operational No USDA Local Regional Procurement Activity Evaluation in Lao PDR Final Report 40 Table 4: Recommendations, rationale and proposed actions No. Recommendation Rationale Proposed actions Type Validated who is to grow what for each season bring all farmers of the village on one platform for planning sowing of vegetables at the beginning of every season. b. Conducting all-farmers’ meet before every season which would ensure better planning and help avoid the overproduction of certain vegetables. 13. Using traditional knowledge of processing food for future use till new methods are introduced (Para 122): Due to the limited demand for vegetables, farmers are unable to sell their produce, resulting in wastages. It is therefore important to increase the shelf life of the vegetables through processing and target newer markets. a. Invest in the processing of vegetables at the cluster level for increasing shelf-life of vegetables. a. Establishing cold storage for storing vegetables for later use. b. Farmers should use traditional knowledge to prepare local pickles to increase the shelf life and value of vegetables. Operational No World Food Program (WFP) 14. Technological support for program monitoring (Para 179): WFP designed a specific monitoring tool in KOBO which was not regularly used during the two years of intervention. a. WFP can provide technical support to MAF in creating a monitoring system and linking with the dashboard to capture information for undertaking corrective actions. Strategic Yes 15. Need for a feasibility study for market accessibility and community needs (Para 115): It was realised after the first year of LRP that a standardised intervention was not completely beneficial for all the farmer groups. As a result, the program had to be limited to 10 villages in the second year. a. WFP should plan a need assessment study before designing a similar program or replicating it in other geographies. The scope of different livelihood activities in the targeted areas, market potential, forward and backward linkages, etc. need to be ascertained before rolling out a similar program in another area. Strategic Yes 16. Ensuring more meaningful engagement with women (Para 96): The design promotes inclusiveness of gender and vulnerable groups in all its activities. However, socially defined gender roles still prevail, with cooking school meals being the women’s responsibility. Also, no mechanism was found in the place to ensure that women undertook strategic and decision-making roles in program implementation. a. Both women and men should be encouraged to volunteer for SMP activities such as cooking, gardening, fetching water and collecting wood. This would help in breaking down the stereotyping of gender roles at the household level and, subsequently, reduce women’s workload. b. Ensure inclusion of women farmers in exposure visits, and provided with opportunities to lead farmer groups. Strategic Yes 17. Supporting MAF in easing the process of organic certification (Para 129): The LRP program has enabled the farmers to shift from chemical-based fertilisers to compost, which can draw higher prices, especially in towns. However, the current process of issuing organic certificates is tedious and cumbersome. a. WFP’s can play a critical role in advocating for the issuance of organic certificates to farmers using compost, helping them draw higher prices and exploring newer markets. Strategic No 18. Organising joint workshop of ministries across levels to help finalise responsibilities of each stakeholder (Para 240): Lack of coordination between WFP and DTEAP was observed during the discussions with officials. This adversely affects the momentum and sustainability of any program aimed at collective efforts to achieve goals of common interest. a. Joint workshop/s of the ministries with their provincial and district-level officials could be considered wherein roles, procedures, and communication channels for each stakeholder could be delineated. Internal review meetings could also be carried out regularly to assess the performance of each stakeholder. Strategic No USDA Local Regional Procurement Activity Evaluation in Lao PDR Final Report 41 Annexures Annex A Map of LRP Intervention Area USDA Local Regional Procurement Activity Evaluation in Lao PDR Final Report 42 Annex B Evaluation Mission Schedule 1. The data collection phase of evaluation took place between 16th September and 2nd October 2019. A five-day scoping mission from 29th July to 2nd August 2019 was also undertaken by the evaluation team. The days have accounted for Saturday and Sunday being school holiday on which days the survey did not take place. Days Dates Activity Team member Locations/si tes Stakeholders Briefing by WFP team Day 1 18 July 2019 Briefing of NRMC team by WFP-CO WFP-CO and NRMC core team Vientiane and New Delhi WFP-CO and NRMC core team Literature Review and Planning The NRMC team spent 2 weeks to review some of the documents shared by WFP-CO to understand the program. In this phase, the team also planned for the scoping mission. Scoping Mission Day 1 30 July 2019 First meeting with WFP-CO to discuss the program details and understand perspectives of the WFP-CO in implementation and performance of the program. Team Leader and Evaluation Manager Vientiane WFP-Co including Country Head, Program Head, Program Manager and Evaluation Manager Day 2 and 3 31 July and 1 August 2019 Visited field sites in Nalae District Team Leader and Evaluation Manager 2 program villages in Nalae District Farmer groups, VEDC, school head, school teachers, cooks and WFP Program Manager & Monitoring Assistants Meeting at DAFO (Nalae) Nalae District Head and Deputy head, Department of Agriculture and Forestry Office, LRP Co￾ordinator (The Lutheran World Federation), WFP Program Manager & Monitoring Assistants Meeting at DESB (Nalae) Head, Department of Education and Sports Bureau, WFP Program Manager & Monitoring Assistants USDA Local Regional Procurement Activity Evaluation in Lao PDR Final Report 43 Days Dates Activity Team member Locations/si tes Stakeholders Day 4 2 August 2019 Debriefing meeting with WFP￾CO presenting the observations from the scoping visit and discussion of changes in proposed evaluation methodology Team Leader and Evaluation Manager Vientiane WFP-CO Meeting with Laos-partners (Geo-Sys) Team Leader and Evaluation Manager Vientiane NA Preparation Phase 4 August to 13 August 2019 Development of draft inception report Evaluation Manager, Gender Specialist and Researcher inputs provided by Team Leader and Advisor New Delhi NA 14 August 2019 Submission of the draft inception report Team Leader New Delhi NA 1 September to 7 September 2019 Preparation of fieldwork, translation of evaluation tools, hiring of data collectors, development of field movement plan indicating movement of each team member including the data collection teams and the core teams Laos-partners (Geo-Sys), Evaluation manager Vientiane and New Delhi NA 9 September to 14 September Addressed comments received by WFP’s Regional Evaluation Officer and DEQAS on draft inception report. Final Submission of revised Inception Report. Team Members New Delhi WFP-co Lao PDR and NRMC Team Data Collection Phase Day 1 and 2 16 and 17 September 2019 Classroom training of teams on data collection tools and ethical guidelines. Evaluation Manager and Researcher and Geo-Sys Vientiane Data collection team USDA Local Regional Procurement Activity Evaluation in Lao PDR Final Report 44 Days Dates Activity Team member Locations/si tes Stakeholders Day 3 18 September 2019 Travel to Nalae NRMC and Geo￾Sys teams Vientiane Day 4 to Day 17 19 September to 02 October 2019 Data collection in two villages by two teams per day. Debriefing of the team at the end of the field day. NRMC and Geo￾Sys teams Nalae Children, parents, school head, school teachers, cooks, storekeeper, VEDC, farmer groups, the village head Day 6-7 and 13- 14 21 September and 22 September 2019 28 and 29 September Schools closed. Documentation of qualitative data. Data management and upload of quantitative data, preliminary checks on data quality and debriefing of field teams. Data collection team, Geo-sys and NRMC team NA NA Day 9 24 September 2019 Discussion with Government official from the education department at the district level and DAFO and The Lutheran World Federation Evaluation Manager Nalae Head of DESB, Nalae Head of DAFO, Nalae Head of The Lutheran World Federation, Nalae Day 15 30 September 2019 Discussion with Government official from the agriculture department at the provincial level Evaluation Manager Namtha Head of PAFO, Luang Namtha Day 15 30 September 2019 Discussion with Government official from the education department at the provincial level Evaluation Manager Namtha Head of PESS, Luang Namtha Day 16 01 October 2019 Discussion with Government official from MoES and DTEAP at national level Discussion with Australian DFAT official Discussion with WFP Official Team Leader, Evaluation Manager Vientiane Head of MoES, Vientiane Head of DTEAP, Vientiane DFAT officials, Vientiane WFP official, Vientiane Day 17 02 October 2019 Debriefing presentation to WFP-CO NRMC team Vientiane WFP-CO and NRMC Data analysis and report writing stage USDA Local Regional Procurement Activity Evaluation in Lao PDR Final Report 45 Days Dates Activity Team member Locations/si tes Stakeholders 4 October to 4 November 2019 Field Notes finalization, Data management and upload of quantitative data; Quantitative and qualitative data cleaning and analysis; Draft report writing and editing Geo-Sys Team; NRMC Team - Evaluation Manager and researcher supported by technical inputs from the team leader and Advisor Vientiane and New Delhi NA 18 November to 25 November 2019 Draft final report writing (incorporating suggestions from RRB) Evaluation Manager and researcher supported by technical inputs from the team leader and Advisor New Delhi NA 10 December 2019 Final report writing (incorporating suggestions from DEQAS) Evaluation Manager and researcher supported by technical inputs from the team leader and Advisor New Delhi NA USDA Local Regional Procurement Activity Evaluation in Lao PDR Final Report 46 Annex C Scope of Work for Activity Evaluation S. No. Scope Evaluation LRP 1 Project Components and timeframe coverage The activity evaluation will cover the LRP operation in all five phases, by focusing on the four key activities (capacity building, plant crops and cash-based transfer, partners monitoring and exchange visits, community feedback and lessons learned) starting from April 2017 to February 2019 to answer the evaluation questions. 2 Expectation Expected to provide an evidence-based, independent assessment of the performance of the operation so that WFP and program partners inform any future program design. 3 Areas to be covered during Evaluation All 47 villages in Nalae District, Luang Namtha Province 4 Sample size The evaluation team is expected to collect field data from the same villages that were selected under the Baseline 5 Focus of evaluation The activity evaluation will focus primarily on the following activities, throughout which Gender Equality and the Empowerment of Women (GEEW) will be fully mainstreamed: Activity 1: Review of relevant documents including project documents, internal/external administrative records, collected data, monitoring reports and Project-Level Results Framework; Activity 2: Field visits to LRP project sites in Nalae district to conduct field data collection and interviews with focus groups (smallholder farmers, school teachers and students, and VEDC members) and observation at the village and school levels; Activity 3: Interviews with representatives and staff members of governmental implementing partners (central MAF, PAFO and DAFO, MoES – PESS and DESB), as well as interviews with community participants impacted by the project. 7 Partnership to achieve LRP results Ministry of Agriculture and Forestry, Department of Technical Extension and Agro-Processing (DTEAP)37 of Ministry of Agriculture and Forestry, and Ministry of Education and Sports. 8 Baseline Baseline conducted in February 2017 37 Used to be known as Department of Agriculture Extension and Cooperatives (DAEC) USDA Local Regional Procurement Activity Evaluation in Lao PDR Final Report 47 Annex D Primary Users of Evaluation Report and Stakeholders Interviewed a) Country Office (CO), Lao PDR: Responsible for the country-level planning and operations implementation, it has a direct stake in the evaluation and an interest in learning from experience to inform decision-making and future project design. It is also responsible to account internally as well as to its beneficiaries and partners for performance and results of its operation. b) Donors (USDA/Australian DFAT, JICA): WFP operations are voluntarily funded by a number of donors. They have an interest in knowing whether their funds have been spent efficiently and if WFP’s work has been effective and contributed to their own strategies and programs. USDA has specific interest in ensuring that operational performance reflects USDA standards and accountability requirements, as well as an interest in learnings to inform future changes in project strategy, result framework, and critical assumptions. c) Regional Bureau (RB) for Asia and the Pacific based in Bangkok: Responsible for both overseeing of Cos and technical guidance and support, the RB management has an interest in an independent/impartial account of the operational performance as well as in learning from the evaluation findings to apply this learning to other country offices. The Regional Evaluation Officer supports CO/RB management to ensure quality, credible and useful decentralized evaluations. d) WFP HQ: WFP HQ technical units are responsible for issuing and overseeing the rollout of normative guidance on corporate program themes, activities and modalities, as well as of overarching corporate policies and strategies. They also have an interest in the lessons that emerge from evaluations, as many may have relevance beyond the geographical area of focus. e) Office of Evaluation (OEV): OEV has a stake in ensuring that decentralized evaluations deliver quality, credible and useful evaluations respecting provisions for impartiality as well as roles and accountabilities of various decentralized evaluation stakeholders as identified in the evaluation policy. OEV may use the evaluation findings, as appropriate, to feed into evaluation syntheses as well as for annual reporting to the WFP Executive Board f) Government of Lao PDR: The Government has a direct interest in knowing whether WFP activities in the country are aligned with national priorities, harmonized with the action of other partners and meet the expected results. Issues related to capacity development, handover of program and sustainability will be of particular interest. Ministry of Agriculture and Forestry (MAF), and Ministry of Education and Sports (MOES) are partners in the design and implementation of WFP-CO Lao PDR Local Regional Procurement and School Meals activities. At sub-national level, Provincial Education and Sports Services (PESS), District Education and Sport Bureau (DESB), Provincial Agriculture and Forestry Office (PAFO), District Agriculture and Forestry Office (DAFO), Provincial Health Office (PHO), and District Health Office (DHO), all of these sub-national government institutions play key roles at implementation level. g) UN Country Team/ UNICEF/UNFPA/ The World Bank: The UNCT’s harmonized action should contribute to the realization of the government developmental objectives. It has, therefore, an interest in ensuring that WFP operation is effective in contributing to the UN concerted efforts. Various agencies are also direct partners of WFP at policy and activity level. Other implementing partners such as DFAT and UN agencies such as UNICEF and UNFPA as well as The World Bank will be interested in the results of the evaluation. USDA Local Regional Procurement Activity Evaluation in Lao PDR Final Report 48 Annex E Broad activities planned under LRP program WFP assistance from April 2017 up to February 2019 consists of:  Trainings A, 1-day training for VEDC members: the training on program information and modality, menu development, inventory, cash management and budgeting, as well as nutrition. In the meantime, a province-wide introductory training was delivered to MoES staff at the provincial level and to representatives of each school.  Training B, 1-day training for farmers: the training led by LWU and MAF partners provided the introduction of various crops and their preparation and taste. As many farmers plant a limited variety of crops, with reliance on rice; it was key to provide an understanding of the final product, its uses, and its nutritional value. MAF provided guidance on the diversity and quantity of seeds or cuttings required and on procurement of such. The selection of commodities to be planted in each area which relied heavily on 1) the nutritional needs, micro and macro nutrients and proteins, 2) culturally acceptable foods, and as well as 3) supplying equally nutritional meals year-round. Each community grew a large variety of complementary foods to meet these particular needs. MAF also provided clear instruction on the preservation of seeds for future crop cycles and plantings. MoES incorporated the crops planted within the community into the Nutrition and School Agriculture curriculum. For full integration into the schools, seeds for the crops were provided.  Training C, 2-days training on agriculture education for farmers: MAF staff met with small groups of farmers to illustrate and trained on the particular life-cycle of the crops that they were specifically growing; the training included providing seed or cuttings, identifying the best environment for various crops, preparing soil, daily care, and harvesting. A focus was made on the plant life cycle and propagation of the crop, emphasizing the necessary steps and activities to obtain seeds or necessary material for future crop planting. The training also discussed natural pest control and fertilizing in addition to introducing storage basics for the produce. These trainings continued on a cyclical basis, according to the proper planting season of crops and the corresponding harvest cycle. Seed management continued to be a strong focus for continuity of the program; as it is also a component of the SMP supported curriculum development of Nutrition and School Agriculture for primary education, the crops planted in the community was also integrated into the teaching.  Training D, 2-to-3-days on expert assistance: trainers from the partner organizations provided direct support to farmers for the following period to offer maximum support in the successful preparation and planting of the commodities.  Training E, 1-day on cooking in community and cash management: WFP, in collaboration with LWU, conducted cook training in communities. The provision of the SMP project was for cooking to be assigned on a rotating basis within the community, so all members of the community took turns producing the meals; kitchen and cooking utensils were provided. This training integrated only the newly available crop production into meals. There was also dedicated time for assistance in the development of a variety of menus. Inventory control, budgeting, and cash management were also addressed. At this time, WFP assisted in opening and understanding a bank account.  Cash Transfer to schools for purchase of fresh foods for school lunch: WFP have begun the Cash Based Transfers of 800kip per student per day to the participating schools since January 2017. A VEDC identified members withdrew necessary funds for the weekly preparation of the USDA Local Regional Procurement Activity Evaluation in Lao PDR Final Report 49 school meals and purchased the necessary commodities from within the local community according to the previously prepared menu. Farmers sold the commodities to the local school.  Partner monitoring and exchange visits: In the spring of 2018, exchange visits between the LRP-funded schools and nearby MoES or WFP CBT schools began. The region of Oudomxay, a neighbouring province, where there was a NSMP which was receiving the cash transfers; the region was also closer to reaching self-sustaining school meals and could offer suggestions in the management of the meals and menu development. LRP-funded schools could offer suggestions on the growth of nutritious crops and storage. The lessons learned through the program was shared and exchanged, with the opportunity for integrating alternatives into the village systems with the support of WFP, MoES, and MAF staff while the program was on-going. VEDC members and farmers were given opportunities to share the experience and expectations for the future. Representatives from each village were invited to participate in a district-wide session of feedback and close-out of the program. Schools were invited to participate in the production of a calendar with particular note to seasonal crops, including recipes and notes on the preparation of nutritious school meals. The calendar was published and distributed to schools nationwide. USDA Local Regional Procurement Activity Evaluation in Lao PDR Final Report 50 Annex F Planned Outputs and Beneficiaries Type of beneficiaries in Nalae • Women and men small farmers (targeted 500) at 49 villages that contributes toward 47 schools. • Primary and Pre-primary students receiving School Lunch through Cash Transfer to their respective schools for buying fresh food for school lunch Number of direct beneficiaries (Targeted) • 500 small farmers trained (Equivalent to 12% of total 4,507 small-holder farmers in Nalae) • 3,753 students or 100% (of which 1,895 girls and 1,858 boys) of primary and pre-primary schools in 47 schools Number of indirect beneficiaries (Targeted) • 25000 persons Number of direct beneficiaries (Achieved till March 2019) • Students boys= 1,720 • Students girls= 1,654 • Total students= 3,374 • Male farmer= 522 • Female farmer= 502 • Male government counterpart= 6 • Female government counterpart= 2 • Total individuals=4,406 Number of indirect beneficiaries (Achieved till March 2019) • Farmers and non-farmers family members: Male: 11,669; • Female: 11,212; Total: 22,881 • Government counterparts: • Male 17, Female 15, total 32 staff. • Total indirect individuals=22,913 USDA Local Regional Procurement Activity Evaluation in Lao PDR Final Report 51 Annex G Planned Outcomes of USDA LRP-Lao PDR Result Performanc e Indicator Unit of Measurem ent Data Source Target Year 1 Progress per Aug￾17 Target Year 2 Reporting period Oct 2017-Mar 2018 Reporting period April- Sept 2018 Target Year 3 Reporting Period October 2018- March 2019 Reporting period April￾Sept 2019 Remarks October 1, 2016 - September 30, 2017 October 1, 2017 - September 30, 2018 October 1, 2018 - September 30, 2019 LRP SO 1 Number of individuals benefiting directly from USDA￾funded intervention Individuals Training attendance records 5000 people Male: 265 Female: 195 Total: 460 Student: 3,680 (1823 female, 1857 male) 5000 people continuing from Year 1 Male: 200 Female: 274 Total: 474 Student: 3,454 (1,667 girl students, 1,787 boy students) Male government counterpart= 6 and Female government counterpart= 2 Total individual= 3,462 Students boys= 1,768 Students girls= 1,683 Total students= 3,451 Male farmer= 20 Female farmer= 5. Male government counterpart= 6 and Female government counterpart= 2 Total individual= 3,484 5000 persons Students boys= 1,720 Students girls= 1,654 Total students= 3,374 Male farmer= 522 Female farmer= 502 Male government counterpart= 6 and Female government counterpart= 2 Total individual=4,4 06 6,026 people Students: 4,888 (1,880 girls) Farmers: 1,132 (female: 556) Government counterparts: 6 (2 female) LRP SO 1 Number of individuals benefiting indirectly from USDA￾funded intervention Individuals Training attendance records, Beneficiary HH multiplier 25,000 persons Male: 1325 Female: 975 Total: 2300 Student: 18,400 (9,115 females, 9,285 male) 25,000 persons continuing from Year 1 Male: 1,000 Female: 1,370 Total: 2,370 Student: 17,270 (8,335 girls and female beneficiaries, and 8,935 boy students and male beneficiaries) Students and their family members: Male= 7,040 Female= 6,764 Total= 13,804 Farmer group members and family: Male= 51 and female= 49 Total 100 persons. Government counterparts Male 17, 25000 persons Farmers and non-farmers family members: Male: 11,669 Female: 11,212 Total: 22,881 Government counterparts: Male 17, Female 15, total 32 staff. Total indirect individuals= 22,913 Female= 11,227 25,564 persons (13,032 females) Based on the most recent statistics for Nalae District USDA Local Regional Procurement Activity Evaluation in Lao PDR Final Report 52 Result Performanc e Indicator Unit of Measurem ent Data Source Target Year 1 Progress per Aug￾17 Target Year 2 Reporting period Oct 2017-Mar 2018 Reporting period April- Sept 2018 Target Year 3 Reporting Period October 2018- March 2019 Reporting period April￾Sept 2019 Remarks October 1, 2016 - September 30, 2017 October 1, 2017 - September 30, 2018 October 1, 2018 - September 30, 2019 Female 15, total 32 staff. Total indirect individuals= 13,936 LRP 1.3.2.2 Value of sales by project beneficiaries U.S. Dollar Form 5, 6 and 7 of CBT reporting, and another form to randomly record sales from farmers groups to market USD25,000 Data not yet available, the data will be collected through SM-BCM USD75,000 $ 53,240 $25500 USD 100,000 $ 71,937 USD37,000 Farmer groups could only plant in the greenhouses and were not able to plant vegetables in open spaces as it was out of season, too hot and too wet. This is the reason for the decreased income compared to the previous reporting period. LRP 1.1 Volume of commodities (MT) sold by project beneficiaries MT Beneficiary contact monitoring on School Meals 120 Data not yet available, the data will be collected through SM-BCM 300 9.4 42.3 420 122.3 66 As mentioned above, farmers could not plant as much as the previous reporting period due to seasonality. LRP 1.4.3/1.4 .4 Number of public￾private partnerships formed as a result of USDA assistance Number: Partnershi ps District Agriculture and Forestry Officer, District Industry and Trade Officer 0 Data not yet available, the data will be collected through SM-BCM 10 19 10 villages 0 10 villages 0 USDA Local Regional Procurement Activity Evaluation in Lao PDR Final Report 53 Result Performanc e Indicator Unit of Measurem ent Data Source Target Year 1 Progress per Aug￾17 Target Year 2 Reporting period Oct 2017-Mar 2018 Reporting period April- Sept 2018 Target Year 3 Reporting Period October 2018- March 2019 Reporting period April￾Sept 2019 Remarks October 1, 2016 - September 30, 2017 October 1, 2017 - September 30, 2018 October 1, 2018 - September 30, 2019 Value of public and private sector investments leveraged as a result of USDA assistance U.S. Dollar District Agriculture and Forestry Officer, District Industry and Trade Officer 0 Data not yet available, the data will be collected through SM-BCM $14,400 No data yet 0 $ 14,400.00 Household consumption per week 20Kg A farmer contributes to school lunch program per week 10.5kg Sell to market per farmer per week 28kg Price of vegetable per kg=5,000LAK A farmer benefits from its plantation per week is: Self consumption= 100,000LAK/w eek Sell to school=52,000 LAK/week Sell to market=141,00 0LAK/week Average income per farmer per month is 1,171,000LAK Within this 6 month farmer could generate income in total 1,171,000LAK* 100*6=702,60 0,000/8500=$ 82,659 $0 USDA Local Regional Procurement Activity Evaluation in Lao PDR Final Report 54 Result Performanc e Indicator Unit of Measurem ent Data Source Target Year 1 Progress per Aug￾17 Target Year 2 Reporting period Oct 2017-Mar 2018 Reporting period April- Sept 2018 Target Year 3 Reporting Period October 2018- March 2019 Reporting period April￾Sept 2019 Remarks October 1, 2016 - September 30, 2017 October 1, 2017 - September 30, 2018 October 1, 2018 - September 30, 2019 LRP 1.1 Total increase in installed storage capacity (dry or cold storage) as a result of USDA assistance meter cubic Beneficiary contact monitoring on School Meals 337.5 Data not yet available, the data will be collected through SM-BCM 675 1,152 N/A 1012.5 N/A 0 No storage facilities were installed as famers sell their products directly from the farm or at a local market. LRP 1.4.1 Number of policies, regulations and/or administrati ve procedures in each of the following stages of development as a result of USDA assistance number of policies School Meals working group Notes of Meeting 1 1 0 0 0 1 1 0 LRP 1.1 Quantity of commodity procured (MT) as a result of USDA assistance MT Form 5, 6 and 7 of CBT reporting, and Beneficiary Contact Monitoring 120 Don't have data yet. The data have to collect through SM-BCM 300 40.3 N/A 420 N/A 0 LRP 1.1 Cost of commodity procured as a result of USDA assistance (by commodity and source country) U.S. Dollar WFP Procureme nt, to buy Vegetable Seeds and NFI USD 75,000 USD 25,000 $ 48,045 N/A USD 100,000 N/A 0 USDA Local Regional Procurement Activity Evaluation in Lao PDR Final Report 55 Result Performanc e Indicator Unit of Measurem ent Data Source Target Year 1 Progress per Aug￾17 Target Year 2 Reporting period Oct 2017-Mar 2018 Reporting period April- Sept 2018 Target Year 3 Reporting Period October 2018- March 2019 Reporting period April￾Sept 2019 Remarks October 1, 2016 - September 30, 2017 October 1, 2017 - September 30, 2018 October 1, 2018 - September 30, 2019 LRP 1.1 Cost of transport, storage, and handling of commodity procured as a result of USDA assistance (by commodity) U.S. Dollar WFP Supply Chain, and Beneficiary Contact Monitoring $ 5,750 $ 5,750 $ 5,900 $ 5,220 N/A $ 11,650 N/A - LRP SO 1 Number of social assistance beneficiaries participating in productive safety nets as a result of USDA assistance Individuals Training attendance records 2,500 persons 2500 2,500 same persons as Year 1 Total of student: 3,454 Male: 1,787 Female: 1,667 Trained male farmer: 200 Trained female farmer: 274 Total of student: 3,451 Male: 1,788 Female: 1,663 2,500 persons 3,374 students 36 farmers (from 6 villages) conducted peer to peer exchange visit of greenhouse gardening Two government counterparts, one female and one male; Total: 3,412 1,132 farmers (556 female) LRP 1.3.2 Number of individuals who have received short-term agricultural sector productivity or food security training as a result of USDA assistance Individuals Training attendance records 500 farmers Male: 265 Female: 195 Total: 460 500 same farmers trained Male: 200 Female: 274 Total: 474 Participated in study visit: Male￾30; Female 6 500 farmers 36 farmers and two government counterparts participated in peer to peer exchange visit in neighbouring village for greenhouse intervention; Total: 38 53 53 participants (30 females) were trained on basic nutrition and food processing for farmer groups and school principals/teach ers. USDA Local Regional Procurement Activity Evaluation in Lao PDR Final Report 56 Result Performanc e Indicator Unit of Measurem ent Data Source Target Year 1 Progress per Aug￾17 Target Year 2 Reporting period Oct 2017-Mar 2018 Reporting period April- Sept 2018 Target Year 3 Reporting Period October 2018- March 2019 Reporting period April￾Sept 2019 Remarks October 1, 2016 - September 30, 2017 October 1, 2017 - September 30, 2018 October 1, 2018 - September 30, 2019 Number of Agricultural Education Trainings delivered (by topic and participant type) number of training per topic and type of participant Training attendance records 5 training 2 training 2 training 0 7 7 training 6 on-site support trainings 0 Number of Financial Procedure Trainings delivered Number of training Training attendance records 2 training 2 training 0 0 not yet 2 training No financial procedure training delivered in the reporting period; 0 Number of nutrition trainings delivered (by participant type) Number of training Training attendance records 40 cluster training 40 clusters training 40 cluster training 0 1 80 cluster training In September 2018, 25 farmers (20 male and 5 females) of 10 villages and 8 government counterparts (6 males and 2 females) went to B. Viengsa, Xay district Oudomxay province for an exchange visit to learn about setting up and managing low cost greenhouses. 85 farmers from 12 villages had engaged in peer to peer visits within the district to learn how set up a greenhouse and plant 0 USDA Local Regional Procurement Activity Evaluation in Lao PDR Final Report 57 Result Performanc e Indicator Unit of Measurem ent Data Source Target Year 1 Progress per Aug￾17 Target Year 2 Reporting period Oct 2017-Mar 2018 Reporting period April- Sept 2018 Target Year 3 Reporting Period October 2018- March 2019 Reporting period April￾Sept 2019 Remarks October 1, 2016 - September 30, 2017 October 1, 2017 - September 30, 2018 October 1, 2018 - September 30, 2019 different vegetables during the dry and lean season. Number of other trainings delivered (by topic and participant type) number of training per topic and type of participant Training attendance records 8 cluster training 8 cluster training 0 Farmer group management TOT training for 7 DAFO and 2 PAFO Mushroom TOT training for 7 DAFO & 2 PAFO Marketing management TOT training for 7 DAFO & 2 PAFO 2 8 cluster training N/A 1 20 farmers from 10 different villages (including 5 females) participated in study visits to organic farmer cooperatives in Vientiane capital and Vientiane province. Number of nutritional meals prepared by schools Number of School Lunch for the LRP period WFP monitoring 5,000 4550 8,750 4,944 2,400 323,904 meals prepared by schools N/A Most of the 47 schools prepared nutritious meals on a daily basis, but there is no data on the exact number of meals prepared during the reporting period. Number of crop types grown Crop types District Agriculture and Forestry Officer TBD 1: Rice 2: Rubber 3: Cardamom 4: Inca TBD 1: Morning glory 2: Garlic 3: Beans 4: Tomatoes 5: Rice 6: Pumpkin 14 20 crop types N/A WFP also provided 9,070 fruit trees to 907 households in 34 villages, assisted with the expansion of greenhouses for 54 families in 5 villages and providing fish fingerlings to 208 households in 30 villages. USDA Local Regional Procurement Activity Evaluation in Lao PDR Final Report 58 Result Performanc e Indicator Unit of Measurem ent Data Source Target Year 1 Progress per Aug￾17 Target Year 2 Reporting period Oct 2017-Mar 2018 Reporting period April- Sept 2018 Target Year 3 Reporting Period October 2018- March 2019 Reporting period April￾Sept 2019 Remarks October 1, 2016 - September 30, 2017 October 1, 2017 - September 30, 2018 October 1, 2018 - September 30, 2019 Number crop life cycles completed, by type Crop life cycles District Agriculture and Forestry Officer TBD Rice- 1 Rubber-1 Cardamon￾1 Inca-1 Rice- 1 Rubber-1 Cardamon-1 Inca-1 1: Morning glory, 4 cycles 2: Garlic, 2 cycles 3: Beans, 3 cycles 4: Tomatoes, 2 cycles 5: Rice, 1 cycle 6: Pumpkin, 2 cycles Morning glory: 4 cycles. Beans: 3 cycles. Garlic: 2 cycles. Tomato: 2 cycles. Pumpkin: 2 cycles. Spring onions, Coriander, green leave vegetable, cabbage, spinach and mins are in all year round. Also, there is some kind of fruit tree that recently yield harvest. Morning glory: 4 cycles. Beans: 3 cycles. Garlic: 2 cycles. Tomato: 2 cycles. Pumpkin: 2 cycles. Spring onions, Coriander, green leave vegetable, cabbage, spinach and mins are in all year round. Also, there is some kind of fruit tree that recently yield harvest. Up to 10 cycles per year for crops produced in greenhous es Diet Diversity of Households score Score Baseline and End￾line evaluations TBD 11 11 11 11 11 Diet Diversity of School Meals score Score Baseline and End￾line evaluations TBD 10 10 10 10 10 Change in agricultural practice by farmers (need to be re-phrased into % of farmers who implement best practices % Monitoring and End￾line evaluations TBD Don’t have data yet. The data have to collect through SM-BCM 70% 39% 50% 80% Approxima tely 80% The latest figures are based on informal findings through monitoring visits. USDA Local Regional Procurement Activity Evaluation in Lao PDR Final Report 59 Result Performanc e Indicator Unit of Measurem ent Data Source Target Year 1 Progress per Aug￾17 Target Year 2 Reporting period Oct 2017-Mar 2018 Reporting period April- Sept 2018 Target Year 3 Reporting Period October 2018- March 2019 Reporting period April￾Sept 2019 Remarks October 1, 2016 - September 30, 2017 October 1, 2017 - September 30, 2018 October 1, 2018 - September 30, 2019 from their farmer trainings) USDA Local Regional Procurement Activity Evaluation in Lao PDR Final Report 60 Annex H Results Framework of WFP-Lao PDR: LRP (FY16) USDA Local Regional Procurement Activity Evaluation in Lao PDR Final Report 61 Annex I Budget for WFP-LRP Program Items Total (USD) Food commodities cost (seed, tools and cooking utensils) 152,500 Other Direct Operational Costs (ODOC) 2,500 Cash-Based Transfers 94,600 Capacity Development & Augmentation (CD&A) 421,374 Monitoring and Evaluation 164,004 Direct Support Costs (DSC) 165,020 TOTAL DIRECTED COSTS: DSC+DOC+CD&A 1,000,000 Activity Description TOTAL (USD) Phase I Training A – VEDC members $ 20,803 Training B – Farmer training $ 47,677 Training C – Agricultural education $ 49,500 Training D – Expert Assistance $ 19,933 Training E –Financial Training $ 37,989 Seed Purchase $ 50,000 Agriculture tool purchase $ 75,000 Cooking utensil purchase $ 27,500 Phase II Cash Based Transfers to schools $ 41,525 Partner monitoring visits $ 8,100 Agricultural education training $ 46,653 Phase III Debrief A&B $ 75,385 Partner Monitoring Visits $ 12,375 Phase IV CBT $ 53,075 Partner Monitoring Visits $ 16,500 Agricultural education training $ 48,399 Exchange Visits $ 17,360 Phase V District Debriefing $ 12,200 Calendar Compilation, Publication and Distribution $ 8,500 Monitoring and Evaluation $ 164,004 Administration $ 65,420 Other Direct Operational Costs (ODOC) $ 2,500 Direct Support Costs (DSC) $ 99,600 Grand Total $ 1,000,000 Budget Narrative Other Direct Operational Costs: WFP defines Other Direct Operating Costs (ODOC) as the total cost of all activity inputs provided to beneficiaries in conjunction with food-related activities or utilized by host governments or cooperating partners to implement food-based activities. ODOC does not include costs for transport, storage, handling, or delivery of food. Direct Support Costs: WFP defines Direct Support Costs (DSC) as those costs that can be directly attributed to supporting an activity and that would not be incurred should that activity cease. These costs are calculated as a percentage of all Direct Operating Costs (the sum of commodity costs, transportation costs, ODOC, and CD&A). WFP-Lao PDR requires $146,120 of DSC for the two years project life. USDA Local Regional Procurement Activity Evaluation in Lao PDR Final Report 62 Indirect Support Costs: WFP defines Indirect Support Costs (ISC) as costs that support the execution of projects and activities but cannot be directly linked to their implementation. These costs are predominately costs incurred by WFP’s global operation, rather than at the country-level. ISC is somewhat similar to the Indirect Cost Rate charged by international NGOs through a Negotiated Indirect Cost Rate Agreement (NICRA). ISC is always charged as a percentage of the total project budget, and WFP’s current rate for all donors – including USDA and USAID – is 7 percent of direct project costs. Monitoring and Evaluation (M&E): The Monitoring & Evaluation Unit within the Country Office will be responsible for managing the monitoring system for program activities in the proposed project. An M&E team of international and national staff will work with WFP program unit, field offices and partners to coordinate and conduct routine monitoring of USDA‐supported activities, share and disseminate the findings and take action where needed throughout the project period Both WFP and USDA’s Monitoring and Evaluation Policies will guide the management and implementation of all monitoring and evaluation activities under the proposed project. In accordance with those policies, implementation of a baseline study, and final evaluation will all be managed by the Monitoring and Evaluation Officer at WFP’s regional bureau in Bangkok. The two studies will be carried out by a team of independent, third party consultants who are experienced in impact evaluation, research, and survey design. USDA Local Regional Procurement Activity Evaluation in Lao PDR Final Report 63 Annex J Logical Framework Outcomes Outputs Indicators Activities SO1: Improved Effectiveness of Food Assistance through Local and Regional procurement  Number of individuals benefiting directly from USDA-funded intervention  Number of individuals benefiting indirectly from USDA-funded intervention  Number of social assistance beneficiaries participating in productive safety nets as a result of USDA assistance LRP 1.1: Improved Cost Effectiveness of Food Assistance  LRP 1.1.1: Improved cost￾effectiveness of procurement • Cost of commodity procured as a result of USDA assistance (by commodity and source country) • Quantity of commodity procured (MT) as a result of USDA assistance • The volume of commodities (MT) sold by project beneficiaries Capacity Building (Phase I)  LRP 1.1.2: Improved Cost Effectiveness of Delivery • Cost of transport, storage, and handling of commodity procured as a result of USDA assistance (by commodity) Capacity Building (Phase I) Plant Crops and Start Cash Based Transfer (Phase II)  LRP 1.1.3: Improved cost￾effectiveness of distribution • The total increase in installed storage capacity (dry or cold storage) as a result of USDA assistance Plant Crops and Start Cash Based Transfer (Phase II) LRP 1.2: Improved Timeliness of Food Assistance  LRP 1.2.1: Improved Timeliness of Procurement • Number of crop types grown • Number crop life cycles completed, by type • Number of nutritional meals prepared by schools Plant Crops and Start Cash Based Transfer (Phases II)  LRP 1.2.3: Improved Timeliness of Distribution • Number of training delivered (by topic and participant type) • Number of crop types grown • Number crop life cycles completed, by type • Number of nutritional meals prepared by schools Plant Crops and Start Cash Based Transfer (Phases II) Capacity Building (Phase I) USDA Local Regional Procurement Activity Evaluation in Lao PDR Final Report 64 Outcomes Outputs Indicators Activities  LRP 1.2.2: Improved Timeliness of Delivery • Number of training delivered (by topic and participant type) Capacity Building (Phase I) LRP 1.3: Improved Utilization of Nutritious and Culturally acceptable food that meet quality standards  LRP 1.3.1: Increased Access to Culturally Acceptable Foods • Number of Agricultural Education Training delivered (by topic and participant type) • Number of Financial Procedure Trainings delivered • Number of nutrition training delivered (by participant type Capacity Building (Phase I)  LRP 1.3.2: Strengthened Local and Regional Food Market Systems • Number of individuals who have received short-term agricultural sector productivity or food security training as a result of USDA assistance • Change in agricultural practice by farmers LRP 1.3.2.1: Increased Agricultural Productivity • Number of crop types grown • Number crop life cycles completed, by type • Value of sales by project beneficiaries Plant Crops and Start Cash Based Transfer (Phase II) Community Feedback and Lessons Learned (Phase III) LRP 1.3.2.2: Increased Value added to Post Production Agricultural Products • Value of public and private sector investments leveraged as a result of USDA assistance Community Feedback and Lessons Learned (Phase III) Gradual Handover: Improve harvest technique, plant crops and cash-based transfer (Phase IV) Complete Handover (Phase V)  LRP 1.3.3: Improved Access to Nutritional Foods • Number of nutritional meals prepared by schools • Diet Diversity of Households score • Diet Diversity of School Meals score Complete Handover (Phase V) LRP 1.4: Conducive Foundational Environment Created  LRP 1.4.1: Increased Capacity of Government Institutions • Number of policies, regulations and/or administrative procedures in each of the stages of development as a result of USDA assistance • Number of capacity building training provided (topic and participants wise) to government functionaries Capacity Building of MoES and MAF (Phase I) Cash-Based transfer (Phase II) Gradual Handover: Improve harvest technique, plant crops and cash-based transfer (Phase IV) USDA Local Regional Procurement Activity Evaluation in Lao PDR Final Report 65 Outcomes Outputs Indicators Activities • Number of Partnerships formed  LRP 1.4.3: Improved Capacity of Relevant Organizations • Number of capacity building training provided (topic and participants wise) to VEDC and farmer groups • Number of public-private partnerships formed as a result of USDA assistance Capacity Building (Phase I) Community Feedback and Lessons Learned (Phase III) Complete Handover (Phase V)  LRP 1.4.4: Increased Leverage of Private Sector Resources • Number of public-private partnerships formed as a result of USDA assistance • Number of capacity building training provided (topic and participants wise) Capacity Building of MoES and MAF (Phase I) Cash-Based transfer (Phase II) Gradual Handover: Improve harvest technique, plant crops and cash-based transfer (Phase IV) USDA Local Regional Procurement Activity Evaluation in Lao PDR Final Report 66 Annex K Roles of Key Partners Partners Roles and Responsibility Department of Technical Extension and Agro￾processing (DTEAP) Coordinate with PAFO and DAFO Share training materials with DAFO and PAFO Assist WFP to coordinate with MAF Provide trainers to support DAFO and PAFO Ministry of Agriculture and Forestry (MAF) Provide guidelines on the diversity, quantity and procurement of seeds Provide instructions on the preservation of seeds for future crop cycle and plantings. Monitoring farming results and provide training accordingly Provincial Agriculture and Forestry Office (PAFO) Provide technical assistance to Nalae DAFO, Assist WFP and DAFO for agricultural training, Provide guidance about seeds Assist WFP, Project and district team to coordinate with other government departments District Agriculture and Forestry Office (DAFO) Take Lead in implementing LRP Provide resources to support project implementation Assist project team to mobilize resources Follow up and provide advice to farmers Ministry of Education and Sports (MOES) Coordinate with WFP and DESB Incorporating the crops information in the school curriculum Provide training on menu development, inventory, cash management and budgeting. District Education and Sports Bureau (DESB) Arranging for community exchange visits to best-performing communities Providing cooking utensils to schools Monitoring the activities under LRP Village Education and Development Committee (VEDC) Assisting WFP, DAFO and DESB for project implementation Connecting local farmers with schools Using the CBT for increasing diversity of school meal The Lutheran World Federation Implementing Partner of Local & Regional Procurement Program USDA Local Regional Procurement Activity Evaluation in Lao PDR Final Report 67 Annex L Brief Report from Scoping Visit Observations from Scoping Visit and Proposed Changes in Sample 1. NR Management Consultants India Pvt. Ltd. (NRMC) has been engaged by World Food Program country office (WFP-CO) Lao PDR to undertake the activity evaluation (FY16–FY19) of United States Department of Agriculture (USDA) supported Local Regional Procurement program in Nalae district of Luang Namtha Province of Lao PDR. Introduction 2. WFP is currently implementing a Local Regional Procurement Program across 47 schools and 49 villages. These schools are beneficiary of USDA McGovern-Dole School Meal Program and the intended benefits and interventions under LRP are in addition to that of SMP. LRP program is also aligned with the objectives of the Government’s Nutrition Strategy and Agriculture Development Strategy (2020). 3. Under the LRP program, WFP has a dual objective of supporting women and men small farm holders in improving their agricultural practices, as well as ensuring continuous nutritious food intake by school children, their parents and families of farmers. The program aims at improving the awareness and knowledge levels of children, farmers and community in general on dietary requirements and importance of nutritious food intake. 4. The program works by building capacities of local women and men small farm holders in adopting better farming practices (multiple cropping, using natural manure and pesticides, crop rotation, etc.) to improve the productivity as well as the nutritional output from the farms. Providing market linkage to these farmers after one cycle of production is completed and creating a linkage between these farmer groups and schools for procurement of vegetables for school meals, thus ensuring continuous nutritious food for children. 5. The aim of the activity evaluation (grant FY16-19) is to critically and objectively evaluate the implementation and performance with an eye to generating recommendations that will strengthen replication of the program. The evaluation will review and take stock of the program implementing experience and the implementing environment, assess whether targeted beneficiaries are receiving services as expected, review the results frameworks and assumptions, document initial lessons learned, and discuss necessary modifications or mid-course corrections that may be necessary to effectively and efficiently meet the stated goals and objectives. Purpose of the Scoping Visit 6. The NRMC evaluation team undertook a five-day scoping visit to Laos between 29th July and 2nd August 2019. The objectives of the scoping visit were: • Meet the WFP (WFP-LRP) team for a better understanding of the project and field layout; • Visit some of the project sites that will inform the evaluation design and tools to be developed for the evaluation; and • Finalise the evaluation methodology and design in consultation with the WFP Country Office. 7. The following stakeholders were met during the scoping visit: a) WFP program team and evaluation team b) Farmer groups from two intervention villages c) VEDC Members Observations from the Scoping Visit 8. Review of the program documents shared by WFP and interactions with the stakeholders during the scoping visit aided the NRMC evaluation team to better understand the current ecosystem of the LRP program. 9. As presented in the figure below, the school going children and the small farm holders (women & men) in select villages are the focus of the LRP program. There are four key players, a) the farmer groups who are strengthened and encouraged to produce nutritious vegetables of good quality for their own sustenance as well as for supplying to market and schools, b) the school management or VEDC who are responsible for procurement of vegetables from farmers, effective utilisation of cash benefit received and ensuring nutritious food to children and are also critical to the implementation and sustenance of the program, c) WFP which is facilitating the LRP program, and d) the Government of Lao PDR which is implementing partner for the LRP program and is expected to eventually scale up the program. USDA Local Regional Procurement Activity Evaluation in Lao PDR Final Report 68 10. Further, during the scoping visits, our interaction with farmer groups in the model village revealed that as a result of the program interventions (training and seeds/tools provided to farmer groups), farmers have been able to increase the frequency of farming as well as the types of vegetables grown has also increased. 11. Under the program, the market has been provided to the beneficiaries for selling of vegetables by the DAFO. After the initial few visits that were made to the market by the farmers, the traders offered to come to the village itself for procuring vegetables. This saves the farmers from additional time and costs in travelling to the market. 12. There seems a possibility that the traders might be selling these vegetables at double the price in retail, however, the farmer groups are content in whatever they are earning as their income has reportedly been improved from earlier times. Role of traders is to be further explored during the evaluation visit. 13. As regards, the procurement of vegetables by schools from these farmer groups, majority of the farmers in model villages were found to be voluntarily contributing to the school meals by providing the vegetables free of cost. The 800 kips received by school per student per day under the LRP program was utilised for purchasing chicken, fish etc., which further increased the nutritious value of the meals served at schools. 14. During the discussions with officials, it was revealed that 10 of 49 villages have been identified as model villages and were provided support after the first year. Further, 3 out of these 10 villages have been awarded the certificate for organic production by DAFO. 15. It can be inferred that the reason of the varying performance of farmer groups in different villages might be because some of the farmer groups are more progressive or have had more exposure through interventions as compared to other groups. 16. Other possible reasons for the observed differences in uptake could be food habits of communities and farming practices. The communities in the habit of eating vegetables might have associated easily with the program activities that resulted in faster uptake of activities. Similarly, farmers already acquainted with vegetable farming might have adopted the activities under the program much easily as compared to those cultivating paddy only. Other than these, another reason influencing the uptake of program activities could be the terrain of the farming area. Further reasons behind high and low uptake will be analysed during the evaluation visit. 17. It was also found that 30 of the 49 intervention villages under the LRP program are also being supported by the Lutheran World Federation. 18. Additionally, clarity on criteria of selection of specific intervention villages is needed from the WFP program team, which will help in identifying the factors responsible for interventions to show better uptake. Revision in the Sample Distribution • Focus on reducing short-term hunger and increasing attendance • Building ecosystem for local procurement with involvement of local Farmers • Ensuring sustainability • Political willingness to scale up • Implementing Partner • Convergence to be explored • Responsible for facilitation and implementation at school and community level CURRENT ECOSYSTEM LOCAL REGIONAL PROCUREMENT Government VEDC/ SMC Farmer Groups WFP • Improved agricultural practices and production • Improved dietary diversity and income USDA Local Regional Procurement Activity Evaluation in Lao PDR Final Report 69 19. It was found during the scoping visit that of the 49 villages covered under the intervention, there were 10 model villages who were outperforming others and therefore, they were provided continued support after the first year. Of these 10 villages, 3 villages have farmer groups who have been awarded organic produce certification by DAFO. From the list of baseline villages, Hatlom village in lowland region is one of the model villages which will be covered in the activity evaluation as well. Further, during the evaluation, if any model villages could be traced in mid￾land and high-land regions, efforts will be made to replace the sampled villages with model villages in consultation with WFP. 20. As for the study respondents & sample size at the village level, it majorly remains unchanged with a slight change in stakeholders. The table below gives a snapshot of the proposed changes. Respondent Original SS Change, if any Farmers 13 Two groups of 6-8 participants; one from intervened farmers and second from the non-intervened group Parents 13 Two groups of 6-8 participants Children 27 One group, covering 5-6 children each from I-V classes VEDC 13 One group, covering all members present Supply Managers 1 Replace with Traders (one per village/school – the case for overlaps possible) Teacher 1 Unchanged School Observation 1 Unchanged Cook 1 Carry out IDI if not part of VEDC discussion Principal 1 To be covered as part of VEDC Stories for MSC 3 Unchanged Local Partner for Data Collection and Management of Field Work 21. As part of its proposal, NRMC had proposed to collaborate with Geo-sys (Laos) Co. Ltd for the data collection process who have the personnel and experience for completing the tasks. USDA Local Regional Procurement Activity Evaluation in Lao PDR Final Report 70 Annex M Evaluation Matrix Evaluation Questions Sub-questions Source of Information/ Method of Data Collection Key Performance Indicators Data Analysis Methods and Triangulation Evidence Availability/Reliability RELEVANCE 1.To what extent is the intervention aligned and contributing to the government agricultural strategies, and plans? To what extent is the program aligned with the National agricultural, nutritional and educational priorities (national nutritional strategy 2016-20, agriculture development strategy 2020) and WFP's country strategy 2017-21? Secondary Review of program documents including ToC KIIs with DTEAP (Q12) and MoES (Q11) KIIs with WFP Program Staff (QA) • Qualitative indicators commenting on the design of the program, compared with the priorities set (1) at the national level for agriculture, nutrition and education, and (2) by WFP Laos • Qualitative indicators with respect to the nature of changes/course corrections made during the intervention period to factor in changes in the external environment, particularly in the social, economic and political environment 1. Narrative/thematic analysis of secondary data 2. Analysis of primary data (interviews/focus groups) 3. Triangulation of same/similar areas of information from different stakeholders using different methods (quantitative & qualitative) and sources (primary & secondary) to check for consistency of responses and create a logical narrative for the information collected (Blue=strong; green = fair; orange = weak) Strength of evidence reliability and availability for this research question is strong as the information is based on reliable secondary documents and interviews with stakeholders involved in the program from WFP and government side To what extent is the program contributing to addressing issues pertaining to education, health & nutrition and gender? KIIs with DAFO (Q14) and DESB (Q12) KIIs with WFP Program staff (QB) KIIs with NGO partners (QA) Secondary Review of program documents (including Monitoring, semi-annual reports and country strategy documents) 1. Narrative/thematic analysis of secondary data 2. Analysis of primary data (interviews/focus groups) 3. Triangulation of same/similar areas of information from different stakeholders using different methods (quantitative & qualitative) and sources (primary & secondary) to check for consistency of responses and create a logical narrative for the information collected (Blue=strong; green = fair; orange = weak) Strength of evidence reliability and availability for this research question is strong as the information is based on reliable secondary documents and interviews with stakeholders involved in the program from WFP and government side To what extent is the program design sensitive to the changing external social, economic and political environment? KIIs with WFP program staff (QC) KIIs with DAFO (Q15), DESB (Q13) KIIs with NGO partners (QB) Secondary Review of program documents 1. Narrative/thematic analysis of secondary data 2. Analysis of primary data (interviews/focus groups) 3. Triangulation of same/similar areas of information from different stakeholders using different methods (quantitative & qualitative) and sources (primary & secondary) to check for consistency of (Blue=strong; green = fair; orange = weak) Strength of evidence reliability and availability for this research question is strong as the information is based on reliable secondary documents USDA Local Regional Procurement Activity Evaluation in Lao PDR Final Report 71 Evaluation Questions Sub-questions Source of Information/ Method of Data Collection Key Performance Indicators Data Analysis Methods and Triangulation Evidence Availability/Reliability (including Monitoring, semi-annual reports) responses and create a logical narrative for the information collected and interviews with stakeholders involved in the program from WFP and government side 2. To what extent did the design and implementation of the intervention contribute to capacity strengthening on new agricultural techniques for local women and men smallholder farmers enabling them to support school lunch program and link to the local market in Nalae District? To what extent was the program design and implementation in line with the needs and demands for capacity strengthening for women and men small farmers? KIIs with NGO partners (QC) KIIs with DAFO (Q5) KIIs with WFP Program staff (QD) Discussions with Farmers Group (Q7) • Number of capacity building training provided (topic and participants wise) 1. Analysis of primary data (interviews/focus groups) 2. Triangulation of same/similar areas of information from different stakeholders using different methods (quantitative & qualitative) to check for consistency of responses and create a logical narrative for the information collected 3. In-depth Analysis: data disaggregated by gender (to capture gender equality & inclusiveness or discrimination, if any) and socio-economic variations, if any (Blue=strong; green = fair; orange = weak) Strength of evidence reliability and availability for this research question is strong as the information is unbiased and can be revalidated and triangulated from both demand (Farmer groups) and supply-side (WFP & Govt.) To what extent has the design and implementation of the intervention addressed existing gaps in linking to the local markets and school meal program and ensuing issues and challenges? KIIs with NGO partners (QD) KIIs with DAFO (Q6) KIIs with WFP Program staff (QE) Discussions with Farmers Group (Q9) • Number of nutritional meals prepared by schools • The volume of commodities (MT) sold by project beneficiaries • Value of sales by project beneficiaries 1. Analysis of primary data (interviews/focus groups) 2. Triangulation of same/similar areas of information from different stakeholders using different methods (quantitative & qualitative) to check for consistency of responses and create a logical narrative for the information collected 3. In-depth Analysis: data disaggregated by gender (to capture gender equality & inclusiveness or discrimination, if any) and socio-economic variations, if any (Blue=strong; green = fair; orange = weak) Strength of evidence reliability and availability for this research question is strong as the information is unbiased and can be revalidated and triangulated from both demand (Farmer groups) and supply-side (WFP & Govt.) USDA Local Regional Procurement Activity Evaluation in Lao PDR Final Report 72 Evaluation Questions Sub-questions Source of Information/ Method of Data Collection Key Performance Indicators Data Analysis Methods and Triangulation Evidence Availability/Reliability 3. To what extent was the program designed towards addressing malnourishment in poor￾performing communities and geographies? To what extent did the program design provide solutions to smallholder farmers towards adopting nutrition focussed agricultural practices? KIIs with NGO partners (QE) KIIs with DAFO (Q13) KIIs with WFP Program staff (QF) Discussions with Farmers Group (Q9) • Number of crop types grown • Change in agricultural practice by farmers • Number crop life cycles completed, by type 1. Analysis of primary data (interviews/focus groups) 2. Triangulation of same/similar areas of information from different stakeholders using different methods (quantitative & qualitative) to check for consistency of responses and create a logical narrative for the information collected 3. In-depth Analysis: data disaggregated by gender (to capture gender equality & inclusiveness or discrimination, if any) and socio-economic variations, if any (Blue=strong; green = fair; orange = weak) Strength of evidence reliability and availability for this research question is strong as the information is unbiased and can be revalidated and triangulated from both demand (Farmer groups) and supply-side (WFP & Govt.) 4. To what extent the program design and implementation integrate principals of inclusiveness and equality with respect to gender and vulnerable groups? To what extent was the intervention based on a sound gender analysis? KIIs with WFP program staff (QG) Secondary Review of program documents • Number of individuals benefiting directly from USDA-funded intervention • Number of individuals benefiting indirectly from USDA-funded intervention • Number of social assistance beneficiaries participating in productive safety 1. Narrative/thematic analysis of secondary data 2. Analysis of primary data (interviews/focus groups) 3. Triangulation of same/similar areas of information from different stakeholders using different methods (quantitative & qualitative) and sources (primary & secondary) to check for consistency of responses and create a logical narrative for the information collected (Blue=strong; green = fair; orange = weak) Strength of evidence reliability and availability for this research question is strong as the information is based on reliable secondary documents and interviews with stakeholders involved in the implementation of the program. Triangulation of information makes it more reliable. USDA Local Regional Procurement Activity Evaluation in Lao PDR Final Report 73 Evaluation Questions Sub-questions Source of Information/ Method of Data Collection Key Performance Indicators Data Analysis Methods and Triangulation Evidence Availability/Reliability To what extent has the program design and implementation ensured reaching out to socially and economically vulnerable groups (access to infra and technology, addressing their key issues and concerns and capacity￾strengthening)? KIIs with WFP program staff (QH) Secondary Review of program documents KIIs with DAFO (Q5) KIIs with NGO partners (QF) nets as a result of USDA assistance 1. Narrative/thematic analysis of secondary data 2. Analysis of primary data (interviews/focus groups) 3. Triangulation of same/similar areas of information from different stakeholders using different methods (quantitative & qualitative) and sources (primary & secondary) to check for consistency of responses and create a logical narrative for the information collected (Blue=strong; green = fair; orange = weak) Strength of evidence reliability and availability for this research question is strong as the information is based on reliable secondary documents and interviews with stakeholders involved in the implementation of the program. Triangulation of information makes it more reliable. EFFECTIVENESS 5. To what extent were the outputs and outcomes of the intervention achieved and what were the major factors influencing the achievement or non￾achievement? Examine the technical and implementation effectiveness (as outlined under Performance Indicators in the ToR) mapping the changes and reasons thereof, and the difference between project and control areas in terms of: 1. access to food (school feeding) and dietary diversity 2. access to school Discussions with VEDC (including teachers and school head) (Q9/11/12/13/15/16) Discussions with Farmers Groups (Q27/7/9/13) Discussions with Parents (Q7/8/11/12/16) and Children (Q1/4/9/10) KIIs with NGO partners (QG) KIIs with WFP program staff (QI) • Number of nutritional meals prepared by schools • Diet Diversity of Households score • Diet Diversity of School Meals score (vegetables eaten in school last week) • Number of Agricultural Education Training delivered (by topic and participant type) • Number of 1. Narrative/thematic analysis of secondary data 2. Analysis of primary data (interviews/focus groups) 3. Triangulation of same/similar areas of information from different stakeholders using different methods (quantitative & qualitative) and sources (primary & secondary) to check for consistency of responses and create a logical narrative for the information collected 4. In-depth Analysis: data disaggregated by gender (to capture gender equality & inclusiveness or discrimination, if any) and socio-economic variations, if any 5. Statistical analysis to ascertain the (Blue=strong; green = fair; orange = weak) Strength of evidence reliability and availability for this research question is strong as the information can be revalidated and triangulated from both demand (Farmer groups, children and parents, VEDC, school staff) and supply-side including the implementing partners (WFP & Govt.). Also, information from USDA Local Regional Procurement Activity Evaluation in Lao PDR Final Report 74 Evaluation Questions Sub-questions Source of Information/ Method of Data Collection Key Performance Indicators Data Analysis Methods and Triangulation Evidence Availability/Reliability supplies, tools and materials 3. knowledge levels and skills of farmers and VEDC members 4. engagement of parents in the school feeding model 5. student attendance and enrolment 6. farmers’, parents’ and children’s knowledge about nutrition Secondary review of program documents Financial Procedure Trainings delivered • Number of nutrition training delivered (by participant type • Number of individuals who have received short￾term agricultural sector productivity or food security training as a result of USDA assistance significant difference between means and proportions secondary sources helps to further revalidating the information. 6. To what extent did the intervention contribute to (a) enhancing smallholder farmers to provide appropriately diverse, nutritious food within the school lunch and community, and (b) enhancing access to nutritious foods supply and voluntary contributions? To what extent have the school children (boys and girls) been able to receive and consume fresh and nutritious food as part of school meals on a continuous basis, using vegetables from farmer groups? Discussions with VEDC (including teachers and school head) (Q11) Discussions with Parents (Q7) and children (Q10) Secondary review of program documents KIIs with WFP Program Staff (QJ) • Number of nutritional meals prepared by schools • Diet Diversity of School Meals score (vegetables eaten in school last week) 1. Narrative/thematic analysis of secondary data 2. Analysis of primary data (interviews/focus groups) 3. Triangulation of same/similar areas of information from different stakeholders using different methods (quantitative & qualitative) and sources (primary & secondary) to check for consistency of responses and create a logical narrative for the information collected 4. In-depth Analysis: data disaggregated by gender (to capture gender equality & inclusiveness or discrimination, if any) and socio-economic variations, if any 5. Statistical analysis to ascertain the significant difference between means and proportions (Blue=strong; green = fair; orange = weak) Strength of evidence reliability and availability for this research question is fair as the information can be revalidated and triangulated from both demands (Children and parents, VEDC, school staff) and supply-side primarily WFP, but depends to a large extent on responses from children. To what extent have the farmers been able to supply diverse, culturally acceptable and nutritious food to Discussions with VEDC (including teachers and school head) (Q12/13) Discussions with • Number of crop types grown • Number crop life cycles completed, by type 1. Narrative/thematic analysis of secondary data 2. Analysis of primary data (interviews/focus groups) 3. Triangulation of same/similar areas of (Blue=strong; green = fair; orange = weak) Strength of evidence reliability and availability for this USDA Local Regional Procurement Activity Evaluation in Lao PDR Final Report 75 Evaluation Questions Sub-questions Source of Information/ Method of Data Collection Key Performance Indicators Data Analysis Methods and Triangulation Evidence Availability/Reliability schools for daily meals on a continuous basis during the program period? Farmers Groups (Q13) KIIs with WFP program staff (QK) Secondary review of program documents information from different stakeholders using different methods (quantitative & qualitative) and sources (primary & secondary) to check for consistency of responses and create a logical narrative for the information collected 4. In-depth Analysis: data disaggregated by gender (to capture gender equality & inclusiveness or discrimination, if any) and socio-economic variations, if any 5. Statistical analysis to ascertain the significant difference between means and proportions research question is strong as the information can be triangulated from demand (Farmer groups, VEDC, school staff) and supply-side primarily WFP. Also, information from secondary sources helps to further revalidating the information. 7. To what extent did the intervention contribute to gender equality and empowerment to the targeted and most vulnerable women and men smallholder farmers in Nalae? To what extent have the targeted and most vulnerable women and men farmers been benefitted by way of receipt of program inputs? Discussions with VEDC (Q13) Discussions with Farmers Groups (Q24/27/7) KIIs with WFP program staff (QL) KIIs with DAFO (Q17) Secondary review of program documents • Cost of commodity procured as a result of USDA assistance (by commodity and source country) • Quantity of commodity procured (MT) as a result of USDA assistance • The volume of commodities (MT) sold by project beneficiaries • Cost of transport, storage, and handling of commodity procured as a result of USDA assistance (by commodity) • The total increase in installed storage capacity (dry or cold storage) as a result 1. Narrative/thematic analysis of secondary data 2. Analysis of primary data (interviews/focus groups) 3. Triangulation of same/similar areas of information from different stakeholders using different methods (quantitative & qualitative) and sources (primary & secondary) to check for consistency of responses and create a logical narrative for the information collected 4. In-depth Analysis: data disaggregated by gender (to capture gender equality & inclusiveness or discrimination, if any) and socio-economic variations, if any 5. Statistical analysis to ascertain the significant difference between means and proportions (Blue=strong; green = fair; orange = weak) Strength of evidence reliability and availability for this research question is strong as the information can be triangulated from demand (Farmer groups, VEDC) and supply-side (WFP& Govt.). Also, information from secondary sources helps to further revalidating the information. To what extent have the targeted and most vulnerable women and men farmers who participated in the program been Discussions with VEDC (Q18) Discussions with Farmers Groups (Q18) KIIs with DAFO (Q17) Secondary review of program documents 1. Narrative/thematic analysis of secondary data 2. Analysis of primary data (interviews/focus groups) 3. Triangulation of same/similar areas of information from different stakeholders using different methods (quantitative & (Blue=strong; green = fair; orange = weak) Strength of evidence reliability and availability for this research question is strong as the information USDA Local Regional Procurement Activity Evaluation in Lao PDR Final Report 76 Evaluation Questions Sub-questions Source of Information/ Method of Data Collection Key Performance Indicators Data Analysis Methods and Triangulation Evidence Availability/Reliability benefitted by way of increase in income? of USDA assistance • Value of sales by project beneficiaries qualitative) and sources (primary & secondary) to check for consistency of responses and create a logical narrative for the information collected 4. In-depth Analysis: data disaggregated by gender (to capture gender equality & inclusiveness or discrimination, if any) and socio-economic variations, if any 5. Statistical analysis to ascertain the significant difference between means and proportions can be triangulated from demand (Farmer groups, VEDC) and supply side (WFP& Govt.). Also, information from secondary sources helps to further revalidating the information. To what extent have the targeted and most vulnerable women and men farmers who participated in the program been benefitted by way of improvement in the quality of life? Discussions with VEDC (Q18) Discussions with Farmers Groups (Q18) KIIs with WFP program staff (QM) Secondary review of program documents 1. Narrative/thematic analysis of secondary data 2. Analysis of primary data (interviews/focus groups) 3. Triangulation of same/similar areas of information from different stakeholders using different methods (quantitative & qualitative) and sources (primary & secondary) to check for consistency of responses and create a logical narrative for the information collected 4. In-depth Analysis: data disaggregated by gender (to capture gender equality & inclusiveness or discrimination, if any) and socio-economic variations, if any 5. Statistical analysis to ascertain the significant difference between means and proportions (Blue=strong; green = fair; orange = weak) Strength of evidence reliability and availability for this research question is strong as the information can be triangulated from demand (Farmer groups, VEDC) and supply-side (WFP& Govt.). Also, information from secondary sources helps to further revalidating the information. USDA Local Regional Procurement Activity Evaluation in Lao PDR Final Report 77 Evaluation Questions Sub-questions Source of Information/ Method of Data Collection Key Performance Indicators Data Analysis Methods and Triangulation Evidence Availability/Reliability 8. To what extent has the intervention results contributed to improving the diet diversity of students (girls, and boys) in school and at home? To what extent has the program resulted in an increase in knowledge towards dietary diversity and nutrition among girls and boys in school? Discussions with Parents (Q7) and Children (Q9) • Recall of nutrition￾related information taught in the school curriculum 1. Analysis of primary data (interviews/focus groups) 2. Triangulation of same/similar areas of information from different stakeholders using different methods (quantitative & qualitative) to check for consistency of responses and create a logical narrative for the information collected 3. In-depth Analysis: data disaggregated by gender (to capture gender equality & inclusiveness or discrimination, if any) and socio-economic variations, if any 4. Statistical analysis to ascertain the significant difference between means and proportions (Blue=strong; green = fair; orange = weak) Strength of evidence reliability and availability for this research question is fair as the information can be triangulated from demand (Children and parents, VEDC, school staff) and supply-side primarily WFP, but depends to a large extent on responses from children. To what extent has the program resulted in improvement in dietary diversity and nutrition among girls and boys in school? Discussions with VEDC (including teachers and school heads) (Q15) Discussions with Parents (Q16) and Children (Q10) Secondary review of program documents • Number of nutritional meals prepared by schools • Diet Diversity of School Meals score (vegetables eaten in school last week) 1. Narrative/thematic analysis of secondary data 2. Analysis of primary data (interviews/focus groups) 3. Triangulation of same/similar areas of information from different stakeholders using different methods (quantitative & qualitative) and sources (primary & secondary) to check for consistency of responses and create a logical narrative for the information collected 4. In-depth Analysis: data disaggregated by gender (to capture gender equality & inclusiveness or discrimination, if any) and socio-economic variations, if any 5. Statistical analysis to ascertain the significant difference between means and proportions (Blue=strong; green = fair; orange = weak) Strength of evidence reliability and availability for this research question is fair as the information can be triangulated from demand (Children and parents, VEDC, school staff) and supply-side primarily WFP, but depends to a large extent on responses from children. EFFICIENCY USDA Local Regional Procurement Activity Evaluation in Lao PDR Final Report 78 Evaluation Questions Sub-questions Source of Information/ Method of Data Collection Key Performance Indicators Data Analysis Methods and Triangulation Evidence Availability/Reliability 9. Was the agricultural extension support provided by DTEAP, PAFO, DAFO sufficient and in a timely manner for solving problems in the implementation of the intervention? To what extent was the support provided by DTEAP, PAFO, DAFO with the help of WFP for agricultural extension was adequate and sufficient in solving problems in the implementation of the intervention? KIIs with DAFO (Q12), PAFO (10) and DTEAP (9) KIIs with WFP Program staff (QN) Discussions with Farmers groups (Q7) KIIs with NGO partners (QG) Secondary Review of program documents • Number of individuals who have received short￾term agricultural sector productivity or food security training as a result of USDA assistance • Number of Agricultural Education training delivered (by topic and participant type) • Number of Financial Procedure Trainings delivered • Number of nutrition training delivered (by participant type) 1. Narrative/thematic analysis of secondary data 2. Analysis of primary data (interviews/focus groups) 3. Triangulation of same/similar areas of information from different stakeholders using different methods (quantitative & qualitative) and sources (primary & secondary) to check for consistency of responses and create a logical narrative for the information collected 4. In-depth Analysis: data disaggregated by gender (to capture gender equality & inclusiveness or discrimination, if any) and socio-economic variations, if any (Blue=strong; green = fair; orange = weak) Strength of evidence reliability and availability for this research question is strong as the information can be triangulated from demand (Farmer groups) and supply-side including implementing partners (WFP& Govt. & NGOs). Also, information from secondary sources helps to further revalidating the information. To what extent was the support provided by DTEAM, PAFO, DAFO was timely in nature? KIIs with DAFO (Q12), PAFO (10) and DTEAP (9) KIIs with WFP Program staff (QO) Discussions with Farmers Groups (Q7) KIIs with NGO partners (QH) Secondary Review of program documents 1. Narrative/thematic analysis of secondary data 2. Analysis of primary data (interviews/focus groups) 3. Triangulation of same/similar areas of information from different stakeholders using different methods (quantitative & qualitative) and sources (primary & secondary) to check for consistency of responses and create a logical narrative for the information collected 4. In-depth Analysis: data disaggregated by gender (to capture gender equality & inclusiveness or discrimination, if any) and socio-economic variations, if any (Blue=strong; green = fair; orange = weak) Strength of evidence reliability and availability for this research question is strong as the information can be triangulated from demand (Farmer groups) and supply-side including implementing partners (WFP& Govt. & NGOs). Also, information from secondary sources helps to further revalidating the information. USDA Local Regional Procurement Activity Evaluation in Lao PDR Final Report 79 Evaluation Questions Sub-questions Source of Information/ Method of Data Collection Key Performance Indicators Data Analysis Methods and Triangulation Evidence Availability/Reliability Nature of gaps identified or problems faced by the DTEAP, PAFO, DAFO while supporting the implementation? KIIs with DAFO (Q12), PAFO (10) and DTEAP (9) KIIs with WFP Program staff (QP) KIIs with NGO partners (QI) Secondary Review of program documents 1. Narrative/thematic analysis of secondary data 2. Analysis of primary data (interviews/focus groups) 3. Triangulation of same/similar areas of information from different stakeholders using different methods (quantitative & qualitative) and sources (primary & secondary) to check for consistency of responses and create a logical narrative for the information collected 4. In-depth Analysis: data disaggregated by gender (to capture gender equality & inclusiveness or discrimination, if any) and socio-economic variations, if any (Blue=strong; green = fair; orange = weak) Strength of evidence reliability and availability for this research question is strong as the information can be triangulated from multiple stakeholders, including implementing partners (WFP& Govt. & NGOs). Also, information from secondary sources helps to further revalidating the information. 10. Was the establishment of “farmer groups” an efficient platform for the farmers to receive agriculture technical support? To what extent has the channel leveraging farmers groups been successful in making agriculture technical support available to farmers, keeping into consideration the inputs involved? KIIs with DAFO (Q18), PAFO (13) and DTEAP (13) KIIs with WFP Program staff (QQ) Discussions with Farmers Groups (Q6) KIIs with NGO partners (QJ) • Number of public￾private partnerships formed as a result of USDA assistance • Qualitative indicators with respect to leveraging farmers groups as a platform - support received, efforts in mobilization and collectivisation, spill over of technical information to the community etc. 1. Analysis of primary data (interviews/focus groups) 2. Triangulation of same/similar areas of information from different stakeholders using different methods (quantitative & qualitative) to check for consistency of responses and create a logical narrative for the information collected 3. In-depth Analysis: data disaggregated by gender (to capture gender equality & inclusiveness or discrimination, if any) and socio-economic variations, if any (Blue=strong; green = fair; orange = weak) Strength of evidence reliability and availability for this research question is strong as the information can be triangulated from demand (Farmer groups) and supply-side including implementing partners (WFP& Govt. & NGOs). To what extent were other alternative efficient platforms (apart from farmer groups) for the farmers to receive agriculture technical support available? To what extent would KIIs with DAFO (Q18), PAFO (13) and DTEAP (13) KIIs with WFP Program staff (QR) KIIs with NGO partners (QK) 1. Analysis of primary data (interviews/focus groups) 2. Triangulation of same/similar areas of information from different stakeholders using different methods (quantitative & qualitative) to check for consistency of responses and create a logical narrative for the information collected 3. In-depth Analysis: data disaggregated (Blue=strong; green = fair; orange = weak) Strength of evidence reliability and availability for this research question is strong as the information can be triangulated from relevant stakeholders, USDA Local Regional Procurement Activity Evaluation in Lao PDR Final Report 80 Evaluation Questions Sub-questions Source of Information/ Method of Data Collection Key Performance Indicators Data Analysis Methods and Triangulation Evidence Availability/Reliability such alternatives have been efficient? by gender (to capture gender equality & inclusiveness or discrimination, if any) and socio-economic variations, if any including implementing partners (WFP& Govt. & NGOs). 11. Was the use of DTEAP, PAFO, DAFO an efficient platform for delivery knowledge transfers to the small-holder farmers? To what extent has the channel leveraging DTEAP, DAFO and PAFO been successful in delivering knowledge transfers to the smallholder farmers, keeping into consideration the inputs involved? KIIs with DAFO (18), PAFO (13) and DTEAP (13) KIIs with WFP Program staff (QS) KIIs with NGO partners (QL) • Number of individuals who have received short￾term agricultural sector productivity or food security training as a result of USDA assistance • Number of Agricultural Education training delivered (by topic and participant type) • Number of public￾private partnerships formed 1. Analysis of primary data (interviews/focus groups) 2. Triangulation of same/similar areas of information from different stakeholders using different methods (quantitative & qualitative) to check for consistency of responses and create a logical narrative for the information collected 3. In-depth Analysis: data disaggregated by gender (to capture gender equality & inclusiveness or discrimination, if any) and socio-economic variations, if any (Blue=strong; green = fair; orange = weak) Strength of evidence reliability and availability for this research question is strong as the information can be triangulated from relevant stakeholders, including implementing partners (WFP& Govt. & NGOs). To what extent were other alternative efficient platforms (apart from DTEAP, PAFO, DAFO) for delivery of knowledge transfers to small farmers available? To what extent would such alternatives have been efficient? KIIs with WFP Program staff (QT) KIIs with NGO partners (QM) 1. Analysis of primary data (interviews/focus groups) 2. Triangulation of same/similar areas of information from different stakeholders using different methods (quantitative & qualitative) to check for consistency of responses and create a logical narrative for the information collected 3. In-depth Analysis: data disaggregated by gender and socio-economic variations, if any (Blue=strong; green = fair; orange = weak) Strength of evidence reliability and availability for this research question is strong as the information is to be sought from WFP and NGO partners only. 12. How flexible and adaptable was the intervention (how quickly could the invention be To understand the frequency and nature of course corrections carried out in the program design and implementation? KIIs with DAFO (Q19), PAFO (14) and DTEAP (14) KIIs with WFP Program staff (QU) • Qualitative indicators with respect to the course corrections made during the intervention, 1. Analysis of primary data (interviews/focus groups) 2. Triangulation of same/similar areas of information from different stakeholders using different methods (quantitative & qualitative) (Blue=strong; green = fair; orange = weak) Strength of evidence reliability and availability for this research question is USDA Local Regional Procurement Activity Evaluation in Lao PDR Final Report 81 Evaluation Questions Sub-questions Source of Information/ Method of Data Collection Key Performance Indicators Data Analysis Methods and Triangulation Evidence Availability/Reliability changed to correct the approach?) KIIs with NGO partners (QN) reasons for making such changes, challenges faced in incorporating such changes and the nature of implications of course corrections 3. In-depth Analysis: data disaggregated by gender (to capture gender equality & inclusiveness or discrimination, if any) and socio-economic variations, if any strong as the information can be triangulated from relevant stakeholders, including implementing partners (WFP& Govt. & NGOs). To understand (1) the reasons for carrying out such course corrections, (2) challenges faced in implementing them and (3) their implications. KIIs with DAFO (Q19), PAFO (14) and DTEAP (14) KIIs with WFP Program staff (QV) KIIs with NGO partners (QO) 1. Analysis of primary data (interviews/focus groups) 2. Triangulation of same/similar areas of information from different stakeholders using different methods (quantitative & qualitative) to check for consistency of responses and create a logical narrative for the information collected 3. In-depth Analysis: data disaggregated by gender (to capture gender equality & inclusiveness or discrimination, if any) and socio-economic variations, if any (Blue=strong; green = fair; orange = weak) Strength of evidence reliability and availability for this research question is strong as the information can be triangulated from relevant stakeholders, including implementing partners (WFP& Govt. & NGOs). IMPACT USDA Local Regional Procurement Activity Evaluation in Lao PDR Final Report 82 Evaluation Questions Sub-questions Source of Information/ Method of Data Collection Key Performance Indicators Data Analysis Methods and Triangulation Evidence Availability/Reliability 13. What were the effects of the LRP on the School Lunch Program in Nalae district? How have the activities of LRP impacted the School Lunch Program in terms of availability of nutritious food for school meals (dietary diversity at schools and involvement of community for self￾sustenance of school meals), improving school attendance and improving learning outcomes? KIIs with PESS (7) & DESB (8); Discussions with VEDC (including with teachers and School Head) (Q15) • Number of nutritional meals prepared by schools • Diet Diversity of School Meals score (vegetables consumed in school last week) • Qualitative indicators on changes in school attendance and learning outcomes 1. Analysis of primary data (interviews/focus groups) 2. Triangulation of same/similar areas of information from different stakeholders using different methods (quantitative & qualitative) to check for consistency of responses and create a logical narrative for the information collected 3. In-depth Analysis: data disaggregated by gender (to capture gender equality & inclusiveness or discrimination, if any) and socio-economic variations, if any 4. Statistical analysis to ascertain significance difference between means and proportions (Blue=strong; green = fair; orange = weak) Strength of evidence reliability and availability for this research question is strong. Information is being sought from all possible stakeholders pertaining to the question - that is, relevant government partners, VEDC and school staff. 14.Were there intended and unintended (positive or negative) effects of the intervention for men and women smallholder farmers and respective stakeholders in Nalae District? What have been the intended and unintended impact of LRP on beneficiaries direct and indirect? KIIs with DAFO (Q20) Discussions with VEDC (Q15/18/20) Discussions with Farmer Groups (Q18) Discussions with Parents (Q8/10/13) KIIs with WFP program staff (QW) Secondary review of program documents • Value of sales by project beneficiaries • Change in agricultural practice by farmers • Diet Diversity of School Meals and Household score • Qualitative indicators with respect to changes in earnings, income, effort levels and overall quality of life 1. Narrative/thematic analysis of secondary data 2. Analysis of primary data (interviews/focus groups) 3. Triangulation of same/similar areas of information from different stakeholders using different methods (quantitative & qualitative) and sources (primary & secondary) to check for consistency of responses and create a logical narrative for the information collected 4. In-depth Analysis: data disaggregated by gender (to capture gender equality & inclusiveness or discrimination, if any) and socio-economic variations, if any 5. Statistical analysis to ascertain the significant difference between means and proportions (Blue=strong; green = fair; orange = weak) Strength of evidence reliability and availability for this research question is strong as the information can be triangulated from demand (Farmer groups, parents, VEDC) and supply side including implementing partners (WFP& Govt.). Also, information from secondary sources helps to further revalidating the information. USDA Local Regional Procurement Activity Evaluation in Lao PDR Final Report 83 Evaluation Questions Sub-questions Source of Information/ Method of Data Collection Key Performance Indicators Data Analysis Methods and Triangulation Evidence Availability/Reliability How has the program impacted sensitization, participation, decision making and reducing discrimination among different social & ethnic groups and individuals (gender equality)? Discussions with VEDC (Q18) Discussions with Farmers Groups (Q18) KIIs with WFP Program staff (QX) • Qualitative indicators with respect to changes in knowledge levels of individuals, overall levels of participation and decision making within village level institutions, and impact on levels of discrimination (especially on the basis of gender and socio-economic vulnerability) 1. Analysis of primary data (interviews/focus groups) 2. Triangulation of same/similar areas of information from different stakeholders using different methods (quantitative & qualitative) to check for consistency of responses and create a logical narrative for the information collected 3. In-depth Analysis: data disaggregated by gender (to capture gender equality & inclusiveness or discrimination, if any) and socio-economic variations, if any (Blue=strong; green = fair; orange = weak) Strength of evidence reliability and availability for this research question is strong as the information can be triangulated from both demand (Farmer groups, parents, VEDC) and supply-side (primarily WFP). 3. How has the program impacted the beneficiaries in terms of quality of life, income, drudgery, time & efforts invested? Discussions with VEDC (Q18) Discussions with Farmers Groups (Q18) • Diet Diversity of School Meals and Household score • Qualitative indicators with respect to changes in earnings, income, effort levels and overall quality of life 1. Analysis of primary data (interviews/focus groups) 2. Triangulation of same/similar areas of information from different stakeholders using different methods (quantitative & qualitative) to check for consistency of responses and create a logical narrative for the information collected 3. In-depth Analysis: data disaggregated by gender (to capture gender equality & inclusiveness or discrimination, if any) and socio-economic variations, if any 4. Statistical analysis to ascertain the significant difference between means and proportions (Blue=strong; green = fair; orange = weak) Strength of evidence reliability and availability for this research question is strong as the information can be triangulated from multiple stakeholders (Farmer groups, parents, VEDC) SUSTAINABILITY USDA Local Regional Procurement Activity Evaluation in Lao PDR Final Report 84 Evaluation Questions Sub-questions Source of Information/ Method of Data Collection Key Performance Indicators Data Analysis Methods and Triangulation Evidence Availability/Reliability 15. To what extent did the implementation arrangements include considerations for sustainability, such as the capacity building of Nalae District of agriculture and forestry office, the small-holder farmer groups in the target villages and other respective partners? (Merged Question 18th)* How did the program design integrate aspects of sustainability within its implementation model towards making the program self-sufficient with its benefit continuing even after the work of WFP has ceased? (capacity strengthening of stakeholders, development of disseminable knowledge-based content, the formation of social groups, partnerships, institutional acceptance and adoption of program elements) KIIs with DAFO (Q11/12), DESB (9 & 10), DTEAP (7 & 9), MoES (6 & 8), PAFO (9 & 10), PESS (9 & 10) KIIs with WFP program staff (QY) Secondary review of program documents • Number of policies, regulations and/or administrative procedures in each of the following stages of development as a result of USDA assistance • Number of capacity building training provided (topic and participants wise) to government functionaries, VEDC and farmer groups • Number of public￾private partnerships formed as a result of USDA assistance • Number of Partnerships formed • Value of public and private sector investments leveraged as a result of USDA assistance 1. Narrative/thematic analysis of secondary data 2. Analysis of primary data (interviews/focus groups) 3. Triangulation of same/similar areas of information from different stakeholders using different methods (quantitative & qualitative) and sources (primary & secondary) to check for consistency of responses and create a logical narrative for the information collected 4. In-depth Analysis: data disaggregated by gender (to capture gender equality & inclusiveness or discrimination, if any) and socio-economic variations, if any (Blue=strong; green = fair; orange = weak) Strength of evidence reliability and availability for this research question is strong as the information is sought from multiple stakeholders from supply-side (WFP& Govt.). Also, information from secondary sources helps to further revalidating the information. USDA Local Regional Procurement Activity Evaluation in Lao PDR Final Report 85 Evaluation Questions Sub-questions Source of Information/ Method of Data Collection Key Performance Indicators Data Analysis Methods and Triangulation Evidence Availability/Reliability 16. To which extent is the investment in agricultural extension facilitating increased ownership and strengthening the community-driven school lunch implementation? To what extent has the LRP program - enabled community stakeholders towards greater participation and commitment in the school lunch program - established a sustainable support system with respect to consistent, community-driven and reliable supply for school lunch Discussions with VEDC (including teachers and school head) (Q18) Discussions with Farmers Groups (Q19) Discussions with Parents KII with DAFO (Q21) • Number of individuals benefiting directly from USDA-funded intervention • Number of individuals benefiting indirectly from USDA-funded intervention • Qualitative indicators specifying changes in community participation and ownership levels for school lunch implementation 1. Analysis of primary data (interviews/focus groups) 2. Triangulation of same/similar areas of information from different stakeholders using different methods (quantitative & qualitative) to check for consistency of responses and create a logical narrative for the information collected 3. In-depth Analysis: data disaggregated by gender (to capture gender equality & inclusiveness or discrimination, if any) and socio-economic variations, if any (Blue=strong; green = fair; orange = weak) Strength of evidence reliability and availability for this research question is strong as the information can be triangulated from demand (Farmer groups, parents, VEDC) and supply-side (primarily DAFO). 17. What elements of the program implementation and design have the potential to be replicated or scaled up for similar programs? (Merged Question 19th and 20th)* What are the ecosystem factors necessary for the program to be scaled up or replicated for similar programs? KIIs with DAFO (Q12), DESB (10), DTEAP (9), MoES (8), PAFO (10), PESS (10) Secondary review of program documents KIIs with WFP program staff (QZ) • Number of policies, regulations and/or administrative procedures in each of the following stages of development as a result of USDA assistance • Value of public and private sector investments leveraged as a result of USDA assistance • Number of Partnerships formed Other potential factors necessary to 1. Narrative/thematic analysis of secondary data 2. Analysis of primary data (interviews/focus groups) 3. Triangulation of same/similar areas of information from different stakeholders using different methods (quantitative & qualitative) and sources (primary & secondary) to check for consistency of responses and create a logical narrative for the information collected 4. In-depth Analysis: data disaggregated by gender (to capture gender equality & inclusiveness or discrimination, if any) and socio-economic variations, if any (Blue=strong; green = fair; orange = weak) Strength of evidence reliability and availability for this research question is strong as the information is sought from multiple stakeholders from supply-side (WFP& Govt.). Also, information from secondary sources helps to further revalidating the information. USDA Local Regional Procurement Activity Evaluation in Lao PDR Final Report 86 Evaluation Questions Sub-questions Source of Information/ Method of Data Collection Key Performance Indicators Data Analysis Methods and Triangulation Evidence Availability/Reliability ensure scalability and replicability to be identified as part of the study What are some of the elements of the program that have been absorbed, adopted or integrated within institutional processes? KIIs with DAFO (Q11), DESB (9) DTEAP (7), MoES (6), PAFO (9), PESS (9) Secondary review of program documents WFP program staff (QAA) • Number of policies, regulations and/or administrative procedures in each of the following stages of development as a result of USDA assistance 1. Narrative/thematic analysis of secondary data 2. Analysis of primary data (interviews/focus groups) 3. Triangulation of same/similar areas of information from different stakeholders using different methods (quantitative & qualitative) and sources (primary & secondary) to check for consistency of responses and create a logical narrative for the information collected 4. In-depth Analysis: data disaggregated by gender (to capture gender equality & inclusiveness or discrimination, if any) and socio-economic variations, if any (Blue=strong; green = fair; orange = weak) Strength of evidence reliability and availability for this research question is strong as the information is sought from multiple stakeholders from supply-side (WFP& Govt.). Also, information from secondary sources helps to further revalidating the information. To what extent has gender and human rights of vulnerable groups been incorporated as an integral aspect of the program design and implementation model? KIIs with DAFO (Q11/Q15), DESB (9, 12), DTEAP (7, 15), MoES (6, 12), PAFO (9, 15), PESS (9, 12) Secondary review of program documents KIIs with WFP program staff (QAB) • Parameters set for program participation, ensuring selection of the most vulnerable, marginalised women and men smallholder farmers • Parameters for selection of villages for intervention 1. Narrative/thematic analysis of secondary data 2. Analysis of primary data (interviews/focus groups) 3. Triangulation of same/similar areas of information from different stakeholders using different methods (quantitative & qualitative) and sources (primary & secondary) to check for consistency of responses and create a logical narrative for the information collected 4. In-depth Analysis: data disaggregated (Blue=strong; green = fair; orange = weak) Strength of evidence reliability and availability for this research question is strong as the information can be triangulated from demand (Farmer groups) and supply-side (WFP & Govt.). Also, information from secondary sources USDA Local Regional Procurement Activity Evaluation in Lao PDR Final Report 87 Evaluation Questions Sub-questions Source of Information/ Method of Data Collection Key Performance Indicators Data Analysis Methods and Triangulation Evidence Availability/Reliability by gender (to capture gender equality & inclusiveness or discrimination, if any) and socio-economic variations, if any helps to further revalidating the information. Record key challenges that emerged during the course of program implementation, learnings and best practices. KIIs with DAFO (Q11/12), DESB (9, 10) Discussions with VEDC (Q19) Discussions with Farmers Groups (Q21) Secondary review of program documents KIIs with WFP program staff (QAC) • Qualitative indicator specifying key challenges faced during the course of implementation, lessons learnt, key actions taken to address the challenges and best practices identified. 1. Narrative/thematic analysis of secondary data 2. Analysis of primary data (interviews/focus groups) 3. Triangulation of same/similar areas of information from different stakeholders using different methods (quantitative & qualitative) and sources (primary & secondary) to check for consistency of responses and create a logical narrative for the information collected 4. In-depth Analysis: data disaggregated by gender (to capture gender equality & inclusiveness or discrimination, if any) and socio-economic variations, if any (Blue=strong; green = fair; orange = weak) Strength of evidence reliability and availability for this research question is strong as the information can be triangulated from demand (Farmer groups, VEDC) and supply-side (WFP & Govt.). Also, information from secondary sources helps in further revalidating the information. USDA Local Regional Procurement Activity Evaluation in Lao PDR Final Report 88 Annex N Approach and Methodology 1. The activity evaluation was planned between July 2019 and January 2020. The data collection phase of evaluation took place in Vientiane and Nalae (Luang Namtha) districts of Lao PDR, between 16th September and 2nd October 2019, the period immediately after the beginning of the semester in schools. The evaluation team also undertook a five-day scoping mission from 29th July to 2nd August 2019 for a better understanding of the project and finalising the evaluation approach and methodology in consultation with the WFP-CO Lao PDR. The scope of the evaluation was from April 2017 to the point of activity evaluation in September 2019. 2. The evaluation was in concurrence with the ToR as it used the international evaluation criteria (OECD￾DAC) to assess the relevance, effectiveness, efficiency, impact, and sustainability of the LRP program with the lens of equality and inclusion of vulnerable groups and all genders. Each of the five criteria has been analysed in detail, and the pre-requisite factors that are necessary for LRP to succeed have been identified along with the learnings to scale up and replicate in other geographies. Evaluation Matrix with detailed information on evaluation questions and criteria is attached. 3. Wherever appropriate, human rights and gender dimensions were factored into the sub-questions and indicators for each evaluation question. Selection of a diverse set of stakeholders was ensured to understand and include their perspectives with respect to participation in the program, access to benefits (nutrition, enhanced income, training, etc.) and decision making in the program-related activities. 4. Under the criteria of relevance, the evaluation assessed the alignment of the WFP-LRP with the national priorities related to agriculture development and ensuring zero hunger and nutrition to most vulnerable groups of society. The Agriculture Development Strategy (2020) of Lao PDR, National Nutrition Strategy (2025), Lao PDR’s Policy on Promoting School Lunch (2014), the WFP’s Country Strategic Plan (CSP 2017-2021) and LRP’s results framework were reviewed to assess the alignment and relevance of the program. In addition, the evaluation also examined whether the design and implementation of the program were relevant to the local needs of the most vulnerable groups and if it is aligned with the gender policy of the government of Laos and that of WFP. 5. To assess the effectiveness of the program, the evaluation establishes the end-line values of dietary diversity; student attendance and enrolment; number of women & men farmers benefitting from the LRP; increase in farmers’ income; access to market; children and vulnerable groups having access to regular nutritious food; changes in knowledge levels, skills and agricultural practices; changes in dietary intake; and other indicators, and compares them with the baseline values to identify the change over the program period. Comparison between control & intervention area is also drawn wherever possible for attributing the change to the program activities. The effectiveness part also looks into the performance of the program across the three strata – that is, low lands region (0-500 meters above sea level) upland region (500-100 meters above sea level) and mountainous region (> 1000 meters above sea level). 6. To measure the efficiency of the program the program evaluated the reasons of delay of program activities (if any), timely support and delivery of services and solutions to issues, roles and responsibilities of partners (DTEAP, PAFO, DAFO, PESB, DESB, etc.), involvement of community institutions and groups, involvement of The Lutheran World Federation, etc. Additionally, the evaluation also looked into the utilisation of funds, need assessments conducted (if any), reasons and methods adopted in coarse corrections (if any), flexibility and adaptability of the program, etc. 7. The evaluation also examined the overall impact of the program by examining the change in availability of nutritious food at school, change in dietary diversity of children, farmers and their families, the involvement of communities, change in attendance, enrolment and learning outcomes, the response of non-beneficiaries, changes in overall quality of life, etc. Moreover, the changes in discrimination against women and other vulnerable groups were also analysed. 8. The evaluation also analysed how the program design integrated the aspects of Sustainability to make it self-sufficient after WFP exits. The evaluation assessed the extent of capacity building of community USDA Local Regional Procurement Activity Evaluation in Lao PDR Final Report 89 members, community institutions, government partners and other stakeholders to enable them for self￾sustenance of school meals and continue other activities with minimal support. Improving the involvement of community in ensuring school meals, forming partnerships for the adoption of program elements, etc. were also assessed. In addition, incorporation of gender equality and human rights-driven approach as an integral part of the program was also assessed. Moreover, best practices were recorded for replication and upscaling. Also, variations across the three regions (low land, upland, mountainous) were recorded for suggesting changes in future programs to ensure sustainability. The sustainability aspect was also evaluated from the supply side wherein the Government’s willingness and capacity to take over the activities under WFP-LRP was evaluated. 9. The evaluation provides an evidence-based performance assessment of the activities and outcomes under the program’s results framework. For the purpose, the Logic model was used to measure the effectiveness of the program through changes in the outcomes. It provides logical linkages among program resources, activities, outputs, different stakeholders, and outcomes related to the identified issues. The evaluation takes into account the risk factors in the results framework and provides recommendations accordingly. The figure presenting our technical approach for the end line evaluation study is presented below: 10. The evaluation analyses how the program has addressed issues of equity and inclusion and hence the analysis views the outcomes from the perspectives of gender, vulnerable groups and those residing in hard to reach geographical areas. This helps in providing evidence of the extent of gender inclusion through the program and activity evaluation. 11. With learning as an essential component, three key stages were a part of the approach to fulfilling the objectives of the end line evaluation. The first stage, the contextual analysis involved examining the relevance and appropriateness of the program; identifying key issues that affect the cross-cutting  Food Availability  Nutritional requirements  Access to school supplies  Change in agricultural practices • Cash based transfer in primary and pre-primary schools • Seeds and agriculture tools for small farmers • Cooking utensils for schools • Agriculture education and training • Community exchange visits to best performing communities RESULTS Immediate Intermediate Long-Term Improved Service Delivery Service Utilization Improved health and livelihood - Increased access to nutritional food - Awareness on health and nutrition - Awareness among farmers about growing nutritious foods - Reduced short￾term hunger - Improved nutritional intake - Increased participation of community in education - - Improved health and dietary practices - Dietary diversity at school and household level - Improvement in agricultural practices Relevance | Effectiveness | Efficiency | Impact | Sustainability Figure 13: Technical Approach USDA Local Regional Procurement Activity Evaluation in Lao PDR Final Report 90 factors across the domains of the framework; examining policy environment; understanding the physical, institutional and social structures that determine education, and nutrition of school-going children; 12. The second stage, comprehending design and implementation of the program involved examining what was designed to address the contextual issues and how were they implemented in consideration of locally specific priorities of the stakeholders; and 13. The third stage, analysing data, findings and dissemination involved examining the contribution of the program, what worked, what did not work, reasons for success or impeding factors, unexpected outcomes of the initiative; recommend the way forward highlighting resource requirement; key factors to be considered for enabling replication of LRP approach, that is, case-based transfer and capacity strengthening of female and male smallholder farmers in other school lunch districts; generating evidence-based success stories and outlining the risk factors associated. 14. The current end line evaluation is a quasi-experimental, pre-post cross-section study design. A mixed￾method approach was deployed to answer the questions under the criteria using key informant interviews (KIIs), and focus-group discussions for qualitative data as-well-as quantitative data. This quasi-experiment study was based on the principle of counterfactual analysis. A beneficiary’s outcome in the absence of the program intervention was it's counterfactual. The baseline formed a benchmark for the indicators. 15. Given that the evaluation study was primarily qualitative in nature, in addition to comparison of intervention and control villages (the difference between baseline and end-line indicators across intervention and control villages), the focus was essentially laid on explaining the reason(s) behind the manner in which the intervention villages have responded to the program. The evaluation study included the use of qualitative research tools like H-form tool and Most Significant Change. The H-form tool was primarily used to understand the views and opinions of relevant stakeholders in a structured format, identifying factors and conditions for the program’s success. The Most Significant Change tool, on the other hand, delineated the process of changes that occurred within the community and around during the project period. 16. The baseline involved systematic random sampling for selection of villages, further broken down into lowland, upland and mountain region38. While the sample for schools and villages at the end line have been mirrored as per the baseline, the total sample size for the current study was substantially increased and re-established. This was done in order to effectively capture the overall impact of the program as well as for the adequate representation of the diversity that exists among program villages. In addition to the sample at the community level, the evaluation team also interacted with WFP staff, government officials from MoES, DTEAP, PAFO, PESS, DAFO, DESB, The Lutheran World Federation and DFAT. Integration of Gender into the Methodology 17. The evaluation has integrated gender dimensions into its design. In addition, the evaluation also examined the role and nature of participation of men and women in the program specifically through the VEDC and how has the program addressed the issues and needs of women farmers and other marginalised groups. Evaluation attempted to understand whether women in their different roles as farmers, group members and at the household level have been able to take part in various decision￾making processes. Quantitative data were disaggregated by gender to look for variations if any in the dietary intake of men and women (both parents and farmers). Also, qualitative interviews compared dietary intake of men, women, boys and girls and explored the reasons for differences, if any. 18. The evaluation adopted a mixed-methods approach to capture all voices from the field (men, women, boys, and girls, other vulnerable groups) and take them into consideration during the evaluation. The data collection team was adequately trained to ensure that the views of all key groups are considered, reflected and triangulated, with due attention to issues focussing on gender. It was ensured that the data collection team is gender-balanced. 38 Source: Baseline Study – WFP Local Regional Procurement Program USDA Local Regional Procurement Activity Evaluation in Lao PDR Final Report 91 19. The evaluation ensured GEEW is integrated and mainstreamed throughout the evaluation. The evaluation matrix presented in annexure highlights that gender was an integral theme across all the evaluation criteria along with the focus on other vulnerable groups. Questions 2 & 4 under relevance criteria has a sub-component on the extent to which the program is in line with the needs of women & men smallholder farmers and to what extent is the program based on sound gender analysis. Under effectiveness criteria, questions 7 & 8 capture extent to which the women and men smallholder farmers have benefitted from the program activities in terms of received inputs, increased income and overall quality of life. It also looks into the dietary diversity of girls and boys in schools. Under impact criteria, question 14 focuses on how the program impacted on reducing discrimination based on gender and marginalised sections. In the end, the sustainability aspect looks into whether the gender component was incorporated as an integral part of the program design or not. Site Mapping 20. It is noteworthy that in accordance with the requirement of the ToR, sampling approach adopted for the end line evaluation was same as the one for the baseline. Selection of the villages was purposive in nature, and largely same villages were covered which were also covered during the baseline study. For the purpose of comparison, a total of five control villages, which were covered during the baseline, were selected for the end line study. Similarly, the same set of fifteen intervention villages which were covered during the baseline were selected for the end line study. Control and intervention villages for the baseline were selected from the same district, hence are consistent as-far-as the district level context is concerned. 21. However, there were changes in two villages selected at the time of baseline namely, Nalae and Tonglahang. As informed by the field monitoring assistants of WFP, Nalae had too little population hence it was replaced with Kanha village. As for the Tonglahang village, it was replaced with Aome village. Also, it was found in the field that the village Salaeung was a part of group of resettlement villages and is now known by the name of Sainamthip. 22. Total sample within the fifteen intervention and five control villages covered during the baseline were distributed across the three strata – that is, (1) low lands region: 0-500 meters above sea level, (2) upland region: 500-100 meters above sea level, and (3) mountain region: > 1000 meters above sea level, in the proportion of villages falling in each of the three strata. 23. The total sample size for the current study was substantially increased over the baseline and re￾established. This was done in order to effectively capture the overall impact of the program as well as for the adequate representation of the diversity that exists among program villages. Sample size covered under end line evaluation was calculated at the program level, using the ‘differences method’ formula with a finite population (Cochran’s 1977). The sample size calculation was based on certain considerations— the sample size would enable comparison of baseline and end-line groups at the project level; same set of villages were covered in intervention and control areas during the end line that was also visited during the baseline; and since there were no further levels of sampling other than selection of villages (Primary Sampling Unit), we had considered the design effect (multiplier for levels of sampling) for the study as ‘1’. 24. The sample size calculated included 5 per cent of buffer for the number of children, parents of children 5-10 years and smallholder farmers. The sample size was calculated to be 27 children and 13 parents and 13 farmers in each village, making it a total of 403 for children and 201 for parents of children 5-10 years and smallholder farmers each. 25. However, the number of children in schools were mostly low (less than or equal to 27 in classes 3-5). Therefore, in most of the cases, all the children present in classes 3-5 were selected. The sample for the end line across intervention and control villages were distributed equally across all sampled villages. 26. The evaluation covered a total of 156 farmers, 201 parents and 380 children across intervention and control villages. At the beginning of the field visit, it was decided in discussions with WFP team, to select children from classes 3-5 instead of 1-5 because children of classes 1 and 2 were too young to comprehend and respond to the information sought. As for the farmers, not all beneficiary farmers from the group could participate in the discussion as many of them had reportedly gone to upland fields to USDA Local Regional Procurement Activity Evaluation in Lao PDR Final Report 92 keep a check on their crops and treat them to protect from pests if needed, as this was the crucial period just before harvesting in October. To ensure gender representation, measures were taken for equal participation of men and women in the group discussions with parents and farmers. The table providing the distribution of sample covered across target groups for the quantitative and qualitative study is presented below: Table 5: Sample Villages Intervention Villages Control Villages Village Name Children Parents Farmers Village Name Children Parents Farmers Low Land Region Low Land Region Namhaeng 20 7 9 Kunha 9 8 7 Hatnalang 28 11 8 Sang-AK 15 11 6 Hatto 15 11 10 Upland Region Saloy 23 7 10 Kiewhinxang 15 10 5 Hatlom 23 14 11 Mountain Region Konechan 27 11 8 Phouhong 10 6 5 Nongkha 7 11 7 Phouthon 21 7 5 Aom 27 9 8 Upland Region Longmoun 27 7 5 Phahou 17 7 8 Phouchalae 20 13 7 Sakaen 13 13 8 Mountain Region Homchaleun 24 14 9 Longkhaean 28 11 10 Salaeung (Donethip/Saina mthip) 13 13 10 Table 6: Quantitative and Qualitative Sample Quantitative Details Intervention Control Children - 312 70 Parents (50:50 Mothers and Fathers) - 159 42 Farmers – those part of farmer groups (50:50 Women and Men) - 128 28 Total 599 140 Qualitative Details Intervention Control VEDC Members 1 FGD per village 15 5 School Head (cover during discussion with VEDC) 1 IDI per village 15 5 School Teacher 1 IDI per village 15 5 Traders 2-3 in district - - Cook (only if not covered during discussion with VEDC) 1 IDI per village 15 5 Total 66 20 Other stakeholders at district and province level a. Ministry of agriculture and forestry (MAF)- Department of Technical Extension and Agro￾Processing (DTEAP) b. Provincial Agriculture and Forestry Office (PAFO) c. District Agriculture and Forestry Office (DAFO) d. Ministry of Education and Sports (MOES) e. Provincial Education and Sports Services f. District Education and Sports Service g. The Lutheran World Federation h. Australian DFAT i. WFP Officials - 9 - USDA Local Regional Procurement Activity Evaluation in Lao PDR Final Report 93 27. In addition to the sample at community level, the evaluation team also interacted with WFP staff, government officials from MoES, DTEAP, PAFO, PESS, DAFO, DESB, The Lutheran World Federation and DFAT. Data Collection and Analysis Methods 28. As already stated, the evaluation used a mixed-method approach for data collection. Other than the secondary literature review, for primary data collection semi-structured questionnaires containing a mix of quantitative and qualitative questions, for individual and group discussions with parents and smallholder farmers, were used. 29. It was understood that the importance of qualitative responses would particularly be immense while explaining the manner in which the LRP program impacted the community, and drawing lessons for scaling up and replicating the program in other geographies. Therefore, semi-structured questionnaires were developed, consisting of a mix of quantitative (objective type) as-well-as qualitative (descriptive) questions. Individual and groups discussions (IGDs) were carried out, wherein, a group of respondents assembled at one location. Objective-type quantitative questions were administered with each respondent individually while maintaining confidentiality. This was followed by a group discussion with all the respondents together, with the descriptive questions. 30. The quantitative tools for the end line survey had semi-structured questionnaires for: a. Children: Since children were from the primary age-group (classes 3-5), reliable data could not have been obtained through their direct interviews. Hence, Multiple Choice Questions (MCQ) tool was developed to collect information on their attendance, frequency of receipt and consumption of school meals, the regularity of such meals, its composition, dietary diversity at home (recall of last three meals) etc. b. Parents: The semi-structured questionnaires administered to the parents of children of 5-10 years included data on the social and demographic profile of the household, questions pertaining to the school going behaviour of the child, pattern of food consumption in school and at home, composition of meals in school and home, and practices of the parents at home towards nutrition and their contribution/engagement in school meals program. c. Men and women small Farmers: Semi-structured questionnaires administered with small farmers was aimed to understand the nature of their engagement in capacity building and training activities in terms of modern tools and techniques, provision of seeds, knowledge about nutritious food crops to be sown, experience of supplying food crops directly to schools and contributing to the school meals program. d. School Facility Observation Sheet was filled in for all the schools visited. This was an observation sheet to understand softer aspects of school capacities, infrastructure and maintenance, such as status and cleanliness of cooking and store areas, buildings and classroom etc. 31. The qualitative tools aimed to guide the discussions to get deeper insights on achievements, challenges, processes that worked and those which did not work. a. Discussion Guides for VEDC members (Focus Group Discussions), School head (included in VEDC FGD), school teacher, the school cook and traders (Key Informant Interviews) were administered. All VEDC members within the village were included in the discussion, subject to their availability. At least one school teacher, who has primarily been involved in the LRP and ensuring school lunch for children was also interviewed. b. Discussion guides for Key informant interviews (KII) based on the finalised indicators. These guides were administered with implementation partners, The Lutheran World Federation, WFP-CO Lao PDR staff and Government officials from MoES, DTEAP, PAFO, PESS, DAFO and DESB. c. Most-Significant Change39: This technique uses stories or personal account to explain the changes that have happened with individuals and groups since the program activities have started. Through the 39 Rick Davies and Jess Dart (2005): “The ‘Most Significant Change’ (MSC) Technique - A Guide to Its Use” USDA Local Regional Procurement Activity Evaluation in Lao PDR Final Report 94 collection of these stories, the program can obtain an understanding of the outcome and impact of the intervention – and what stakeholder considers to be issues that need to be addressed. The national consultant from Geo-Sys (hired field agency for data collection in Laos) was present during the actual fieldwork in villages and followed the conversations with different stakeholders, ranging from parents, farmers to VEDC members. Using these conversations as cues, he explored these stories in greater detail by conversing with the relevant actors and catalysts. Special focus was given on capturing stories of change from the standpoint of men and women as farmers, group representatives or at the household level. A gendered approach to highlighting stories of brought a more nuanced component to the study of nutrition, agriculture, the formation of social capital and its differing significance for men and women. d. MSC was used to get stories of change explicitly from male and female farmers or male and female representatives within VEDCs in order to enhance insights on the interrelatedness of nutrition, education and agriculture. c. H-form Tool: The H-form40 tool can be used to record the views and ideas of individuals and/or groups in a focused and structured way. It assists in learning the positive and negative aspects of a specific factor in a community. Factors that were identified and assessed included access to school meals, quality of such meals, its composition, changes in dietary diversity in the school and at home. It aimed to highlight the reasons for success and steps yet to be taken by the implementation partners, farmers, schools and the parents in program areas. 32. Thereafter, the data from the secondary review and the primary survey was triangulated to assess the reliability and validity of the data. The evaluation matrix presents the sources from where the data for the evaluation questions were collected and analysis methods. It also highlights whether the data on different evaluation questions was available and reliable. Triangulation was mainly undertaken by comparison of perspectives of different stakeholders on the same issue. 33. The quantitative data analysis was done using statistical software – SPSS. The quantitative information collected was in line with the baseline information available, particularly related to dietary diversity. The analysis of quantitative data also included a descriptive analysis of sex-disaggregated data showing the comparison of baseline and end-line. This highlighted the change in outcome indicators over time. A pre-and-post analysis with control and intervention data provided the difference in the two scenarios with the contribution to the program. 34. Qualitative data were analysed using content analysis. The analysis focused on providing the reasons for the change observed. The best practices, challenges, and learnings were captured through qualitative data. Qualitative data was translated into English and checked by the evaluation team for consistency based on the field visits by the core evaluation team and the quantitative data. The analysis also recorded the variations in the performance of the program in villages across the three regions (low land, upland and mountainous regions). Limitations and Risks 35. As part of our risk management protocols, we ensure the safety of our teams during evaluations, especially in complex situations. When visiting an extremism affected area, we ensure that the community has been informed in time and support from the local influential leaders sought. We made sure that the monitoring assistants under the LRP program in District Nalae guide our team members during all field visits and the schools and villages were prior informed by them about the date and time of visit. The monitoring assistants helped us connect with the relevant respondent groups within the intervention villages, and facilitate interactions. Our administrative team coordinates our travel ensuring that the accommodations are safe and the team members follow the safety guidelines. Some of the limitation and risks that we envisaged and faced during the period of activity evaluation study have been listed below: 40 Tips for trainers: Introducing the ‘H-form’ - a method for monitoring and evaluation, PLA Notes (1999), Issue 34, pp.84–87, IIED London USDA Local Regional Procurement Activity Evaluation in Lao PDR Final Report 95 a) We felt that the fieldwork might get affected, because of high rainfall and flash floods. In order to mitigate this risk, we had a team of eight supervisors, so that we could complete the data collection in the least possible time, giving us buffer in case of bad weather. b) The activity evaluation was a quasi-experimental, cross-section study which followed a pre-and￾post study design. Given the limited information on key indicators in the baseline conducted in 2017. The evaluation, therefore, was only able to attribute limited changes to the program. However, qualitative data helped in understanding how the program has influenced the observed results and what worked what didn’t work. c) The two key objectives of the baseline study included agricultural practices adopted by farmers in the targeted villages and the impact of the village relocation on agricultural practices in the villages. While analysing key components from the first aspect, it emerged that the baseline analysed data at the level of geographical strata (lowland, upland and mountainous)41 as well as at individual village level. While the end line assessment has ensured that relevant baseline indicators would be covered, the end line presents findings at the three strata levels but not for each village individually. It was not feasible to comment on the potential changes as a consequence of the program at the micro-level of the village as a unit. d) The methodology adopted at the baseline study was also studied from the lens of data reliability. It was noted that the methodology included extensive interactions with the community as well as village and school-level stakeholders. However, findings from these interactions were not further triangulated with that of other relevant stakeholders. Another key gap that emerged in the baseline was that it did not highlight and elaborate on the gender aspect within the findings to the extent it was expected. Hence, comparison in these aspects was not possible. e) The children of classes 1st and 2nd were not able to comprehend and respond to the information areas, hence the sample selection for children was modified to select children only from classes 3rd to 5th. The total number of children in these three classes was less than 27 in many schools, therefore the sample size of 27 children per school could not be achieved. However, it can be safely stated that the entire universe was selected for children present in the school in classes 3rd to 5th on the day of the visit. f) Despite prior information to the school and village, not all beneficiary farmers from the group were available for discussion as many of them had gone to upland farm field for work. However, all the beneficiary farmers present were included in the discussion without any exclusion. Ethical Considerations and Quality Assurance 36. WFP’s decentralised evaluations conform to WFP and UNEG ethical standards and norms. Accordingly, NRMC is responsible for safeguarding and ensuring ethics at all stages of the evaluation cycle. This includes, but is not limited to, ensuring informed consent, protecting privacy, confidentiality and anonymity of participants, ensuring cultural sensitivity, respecting the autonomy of participants, ensuring fair recruitment of participants (including children, women and socially excluded groups) and ensuring that the evaluation causes no harm to participants or their communities. 37. The evaluation study was particularly conscious in terms of maintaining ethical norms with respect to data collection and its reporting. In addition to providing the option to the respondent to participate in the study, proper informed consent was taken before initiating any discussion. Prior consent was taken from school head/teachers before interacting with children in schools. Extreme care was taken while interacting with children, ensuring there is no mental or physical harm or loss to them during or after the interaction. Similarly, at the time of reporting, the evaluation team ensured not disclosing names of any respondents within the evaluation report, which could potentially lead to their recognition. 38. WFP’s Decentralized Evaluation Quality Assurance System (DEQAS) defines the quality standards expected from this evaluation and sets out processes with in-built steps for Quality Assurance, Templates for evaluation products and Checklists for their review. DEQAS is closely aligned to the WFP’s evaluation quality assurance system (DEQAS) and is based on the United Nations Evaluation Group (UNEG) Norms and Standards and good practice of the international evaluation community and aims to 41 Definitions of the geographical regions provided in a previous section of the study USDA Local Regional Procurement Activity Evaluation in Lao PDR Final Report 96 ensure that the evaluation process and products conform to best practice. DEQAS will be systematically applied to this evaluation guiding the NRMC team. 39. Measures adopted to ensure quality and safeguard ethical issues throughout the evaluation included: a) The data collection team comprised of personnel, well experienced in collecting and collating both quantitative and qualitative information. Qualitative discussions with the Government officials, WFP field offices and partners were conducted by the NRMC core team with the help of a translator wherever required. b) As a part of quality control and to ensure the timeliness of data collection, NRMC developed detailed field movement plan in advance of the survey. This detailed out the daily team movement plan including outlining the number of quantitative and qualitative interviews to be conducted. It also included the plan for the core team reflecting their division of time on monitoring of field data and discussions with secondary stakeholders of the program. Towards this, NRMC sought WFP’s support in contacting the secondary stakeholders (government officials, other partners etc.) and scheduling the meetings with them in advance. The selection of stakeholders was made in consultation with the WFP team. Further, the field plan for the data collection team was crystallised and modified on witnessing the field realities based on two to three days of data collection exercise. c) Laos-based data collection team was trained by NRMC on data collection. Specific sessions were conducted on ethical issues faced during data collection, data integrity and interacting with children. d) The evaluation was led by a core team from a different country (India) which is the first step towards ensuring impartiality in the evaluation. Further, it was ensured that the data collection team was gender-balanced such that the perspectives of both men and women are captured adequately and impartially. e) A robust monitoring process was followed for quality assurance during data collection. A field plan was created and shared with WFP-CO Lao PDR highlighting the estimated time to complete the primary survey. At least two members of the core evaluation team of NRMC were present in the field during the entire period of data collection accompanying the moderators. WFP-CO Lao PDR was provided weekly updates on the status of the progress of data collection. 40. NRMC’s internal quality protocols were integrated with the process to obtain reliable data and ensure the quality of deliverables. Our internal quality control measures are reflected at each step, from designing of tools, hiring & training of field teams, data collection and analysis to the monitoring of the processes involved and reporting of findings. Some of the key steps to ensure data quality are enumerated below— a) A three-stage internal review process of key deliverables: Our evaluation teams are structured in a way that all deliverables including data collection tools, inception report, data analysis plan and final report are reviewed by the project coordinator, by the team leader and by an external technical backstopping expert designated for the project. b) Strong monitoring measures during data collection: As mentioned earlier, we propose to carry out paper-based data collection for the evaluation study, with robust measures for data quality assurance. Our evaluation team travels extensively to the field to ensure that the data quality is never compromised. c) Assuring data quality during analysis: We ensure that the transcripts of qualitative interviews along with field notes are made available to the evaluation team for better analysis. d) Unique support system: It is a two-pronged approach ensuring that—(i) Training of the field teams is conducted by the core team members of the evaluation team; and (ii) Quick resolution of errors is achieved by the internal data processing team. 41. In order to obtain clean and reliable data for research studies, NRMC follows a comprehensive approach for quality control and assurance. USDA Local Regional Procurement Activity Evaluation in Lao PDR Final Report 97 a) Robust Field Plans: Field plans for the data collection to be undertaken will be prepared prior to the data collection starts. These field plans will be developed taking into account the spatial spread of the villages/schools and the estimated time needed for data collection to finish. b) Fieldwork progress and reporting: A fieldwork tracking sheet will be created for the purpose of maintaining the progress of the fieldwork and report weekly to WFP-CO Lao PDR about the status of this progress. c) Scrutiny Plan: At the time of the field manual preparation, a scrutiny plan for the data will also be made for the team to follow during the field checks. The quality control plan envisages quality check mechanisms at the data collection level. USDA Local Regional Procurement Activity Evaluation in Lao PDR Final Report 98 Annex O List of Stakeholders Interviewed S. No. Stakeholders Interviewed Designation & Office 1. Mr. Sengarun Budcharern Monitoring & Evaluation Officer M&E Section, World Food Program 2. Mr. Air Sensomphone Program Manager, World Food Program 3. Ms. Kaikhoun Official, Australian DFAT 4. Mr. Vassana Department of Technical Extension and Agro￾Processing (DTEAP), Ministry of Agriculture and Forestry (MAF) 5. Ms. Dara Khiemthammakhone Acting Director, IEC, MoES 6. Mr. Vilason Khanhunsa Head, Division of Rural Development, PAFO, Luang Namtha 7. Mr. Somhak Soukaphone Coordinator for LRP, PESS 8. Mr. Khamluoan Keovanxay Vice District Agriculture and Forestry officer, Division of Agriculture Promotion, DAFO, Nalae 9. Mr. Inthong Duoangphaserd Head, DESB Other stakeholders (across 20 villages) Stakeholders Children Parents Farmer group members VEDC members School heads School teachers Cooks 99 Annex P Documents gathered Document Type Comment / Titles & dates of documents received Received – Y/N (N/A) Link to Evaluation matrix Project-related documents [if applicable] Appraisal mission report Project document (including Logical Framework in ANNEXURE) Project Proposal including the results framework and performance indicator matrix Agreement document between WFP and DAEC Agreement between WFP and Lutheran World Federation Y Relevance, Effectiveness, Efficiency, Sustainability Standard Project Reports Baseline report Semi-annual and monthly monitoring reports Y Effectiveness, Impact Budget Revisions Project Proposal including budget details Y Efficiency, Effectiveness Note for the record (NFR) from Program Review Committee meeting (for original intervention and budget revisions if any) N/A Approved Excel budget (for original intervention and budget revisions if any) N/A Intervention/Project Plan (the breakdown of beneficiary figures and food requirements by region/activity/month and partners) Project Proposal including the results framework and performance indicator matrix Y Relevance, Effectiveness Other Country Office Strategic Documents (if applicable) Country Strategy Document (if any) Annual Report for Country Strategic Plan (2017 – 2021) FY17 & FY18 SMP Evaluation Reports: Baseline FY17-21; End line FY14-16 Y Relevance, Sustainability Other Meeting Minutes of SM TWG meeting for Jan and Feb 2018 Assessment Reports [if applicable] Comprehensive Food Security and Vulnerability Assessments Mid Term Review of National Nutrition Strategy Plan of Action (2016- 2020) Community Strength Assessment Mid Term Review – National Nutrition Plan of Action- Education Sector (2016-2020) Education and Sports Sector Development Plan (2016-2020) Y Relevance, Effectiveness, Sustainability Crop and Food Security Assessments (FAO/WFP) Agriculture Development Strategy to the year 2025 and Vision to 2030 Y Relevance, Sustainability Emergency Food Security Assessments Food Security Monitoring System Bulletins 100 Market Assessments and Bulletins Joint Assessment Missions (UNHCR/WFP) Inter-Agency Assessments Rapid needs assessments Cash and voucher feasibility studies Other Monitoring & Reporting (if applicable) M&E Plan Performance Indicator Matrix Y Relevance, Effectiveness, Impact Country Situation Report (SITREP) Country Executive Brief WFP Lao PDR Country Brief Y Relevance Food Distribution and Post-distribution Monitoring Reports • End line Evaluation of McGovern-Dole School Feeding in Laos 2015– 2016 Evaluation Report • Baseline Evaluation of McGovern-Dole School Feeding in Laos 2017– 2021 Evaluation Report Y Relevance, Effectiveness, Impact Monthly Monitoring Reports Semi-annual and monthly project reports Y Effectiveness, Impact, Efficiency Beneficiary Verification Reports Donor specific reports Output monitoring reports (if applicable) Actual and Planned beneficiaries by activity and district/ location by year Performance Indicator Matrix (actual and planned outcomes and targeted number of beneficiaries only) Y Effectiveness, Impact, Efficiency Male vs. Female beneficiaries by activity and district/ location by year Beneficiaries by age group Actual and Planned tonnage distributed by activity by year Commodity type by activity Actual and Planned cash/voucher requirements (US$) by activity by year Operational documents (if applicable) Organogram for main office and sub-offices Provided in ToR Y Activity Guidelines Provided in ToR Y Relevance, Effectiveness, Efficiency Mission Reports Pipeline overview for the period covered by the evaluation 101 Logistics capacity assessment Partners (if applicable) Annual reports from cooperating partners List of partners (Government, NGOs, UN agencies) by location/ activity/ role/ tonnage handled List of partners has been provided in ToR. Relevant documents have been shared for their roles. Y Effectiveness, Sustainability Field level agreements (FLAs), Memorandum of Understanding (MOUs) MoU with DAEC Y Effectiveness, Efficiency, Sustainability Cluster/ Coordination meetings (if applicable) Logistics/Food Security/nutrition cluster documents NFRs of coordination meetings Evaluations/ Reviews Evaluations/ reviews of past or on-going operations/ interventions • End line Evaluation of McGovern-Dole School Feeding in Laos 2015– 2016 Evaluation Report • Baseline Evaluation of McGovern-Dole School Feeding in Laos 2017– 2021 Evaluation Report • Baseline Report of WFP-LRP FY17 Y Effectiveness, Impact, Sustainability Resource mobilisation (if applicable) Resource Situation N/A Contribution statistics by month Resource mobilization strategy Maps (if applicable) Map of the intervention Yes Y Logistics Map Food/Cash/voucher Distribution Location Map Food Security Map 102 Annex Q Mapping of National Priorities and Logical Framework of LRP Table 7: Mapping of National Agriculture Development Strategy to 2025 and Logical Framework of LRP Agriculture Development Strategy to 2025 Logical Framework of LRP To fulfil Millennium Development Goals (MDG1)42; Target 1.C43 to be successful, cooperation across relevant sectors is required to ensure food security and improvement of nutrition (Refer to section 3.1.2, pg. 23 of MAF: Agriculture Development Strategy to the year 2025 and Vision to 2030 Document) Human resource development activities for private and government personnel both, through training and capacity building in leadership and technical aspects of agriculture at the province, district, and village level (3-built policy) as well as a technical extension centres to enable them to guide farmers to apply modern and sustainable agriculture production techniques. (Refer to section 3.1.2, pg. 22 and section 3.1.4, pg. 26 of MAF: Agriculture Development Strategy to the year 2025 and Vision to 2030 Document) LRP 1.4: Conducive Foundational Environment Created LRP 1.4.1: Increased Capacity of Government Institutions LRP 1.4.3: Improved Capacity of Relevant Organizations LRP 1.4.4: Increased Leverage of Private Sector Resources Building a partnership with private and government departments as part of the program design. The program included capacity building of all the relevant stakeholders (including provincial and district level government departments) for sustained and efficient management of the program. To achieve the objectives and goals in developing of the Agriculture and Forestry Sector by 2020 and 2025, especially the industrialization and modernization in Agriculture Sector, ensuring food security, support and promote the production of agriculture goods in line with sustainable agriculture development direction contributing to poverty reduction of people of all ethnic groups  Develop appropriate training curriculum to organize on-site training, occupational training locations, schools, agriculture technique extension. In addition, organize study visits both inside and outside the country.  Expand the use of storage methods in order to store food for consumption in the shortage season. There should be training to increase knowledge of people on food storage techniques. (Refer section 3.1.4, pg. 25 of MAF: Agriculture Development Strategy to the year 2025 and Vision to 2030 Document) LRP 1.1: Improved Cost Effectiveness of Food Assistance LRP 1.1.1: Improved cost-effectiveness of procurement LRP 1.1.2: Improved cost-effectiveness of delivery LRP 1.1.3: Improved cost-effectiveness of distribution LRP 1.2: Improved Timeliness of Food Assistance LRP 1.2.1: Improved timeliness of procurement LRP 1.2.3: Improved timeliness of distribution LRP 1.2.2: Improved timeliness of delivery LRP 1.3: Improved Utilization of Nutritious and Culturally acceptable food that meet quality standards LRP 1.3.1: Increased access to culturally acceptable foods LRP 1.3.2: Strengthened local and regional food market systems  LRP 1.3.2.1: Increased Agricultural Productivity  LRP 1.3.2.2: Increased Value added to Post Production Agricultural Products LRP 1.3.3: Improved access to nutritional foods LRP 1.4: Conducive Foundational Environment Created 42 MDG 1: Eradicate Extreme Poverty and Hunger 43 Halve the proportion of people who suffer from hunger. Hunger and malnutrition are measured by the percentage of children under 5 years of age who are underweight (malnourished) and by the proportion of population consuming less than the daily minimum energy requirement (undernourished), respectively. (https://www.adb.org/sites/default/files/publication/175162/ki2015-mdg1.pdf) 103 LRP 1.4.1: Increased Capacity of Government Institutions LRP 1.4.3: Improved Capacity of Relevant Organizations LRP 1.4.4: Increased Leverage of Private Sector Resources The program provided tools and seeds and imparted training to women and men smallholder farmers on modern agriculture methods and vegetable production. They were also provided opportunities for exposure visits for learnings exchange. The program had a component on imparting training related to storage of farm inputs and produce. Gender Focus For women advancing development or gender activities is regarded as one of the important priorities in implementing Agriculture and Forestry Development Programs by supporting and creating favourable opportunities for female personnel and farmers to participate in the decision-making process at each level. Female personnel shall involve in implementing the strategic plan and participate in capacity building, upgrading of knowledge, exchange of lessons on new techniques and the female farmers, poor families of ethnic groups in remote rural areas to attend new production techniques learning, access to information, appropriate mechanisms of fund/credit/finance/ marketing and others. (Refer to section 3.1.4, pg. 28 of MAF: Agriculture Development Strategy to the year 2025 and Vision to 2030 Document) SO1: Improved Effectiveness of Food Assistance through Local and Regional procurement The program has included gender equality aspects and has been able to achieve equal women participation in the program. Certain observations from the field in this regard showed that:  Majority of teachers across schools were women.  Program ensured participation of one man and one woman from selected farmer households, thus covering women participation under the program. However, at the same time, it was also observed that no specific provisions were made under the program to lend special focus or prioritize women participation. 104 Table 8: Mapping of National Nutrition Strategy to 2025 and Logical Framework of LRP National Nutrition Strategy to 2025 Logical Framework of LRP SO, 1: Improve the nutrient intake SO8: Improve institutions and coordination SO9: Develop human resources SO10: Increase the quantity and quality of information  Promote capacity building in institutions in order to ensure that NFS is provided efficiently and effectively. (Refer to figure 2 and table 2, pg. 11-12 of National Nutrition Strategy to 2025 and Plan of Action 2016-2020 Document) LRP 1.4: Conducive Foundational Environment Created LRP 1.4.1: Increased capacity of government institutions LRP 1.4.3: Improved capacity of relevant organizations LRP 1.4.4: Increased leverage of private sector resources Building a partnership with private and government departments as part of the program design. The program included capacity building of all the relevant stakeholders (including provincial and district level government departments) for sustained and efficient management of the program. SO3: Produce food for consumption SO4: Improve access to nutritious food SO11: Increase investments in nutrition interventions  Increase the cultivation of crops which have high nutritional value.  Produce and promote meat which has protein for household consumption: poultry, fish, and other aquatic life. Provide materials and equipment for production along with the necessary infrastructure: small-scale irrigation, agricultural service units, and so forth.  Build post-harvest facilities (including food dehydrators, and food storage facilities) and apply technology to food processing, preservation, and storage so that it remains safe and nutritious as a means of ensuring food availability all year round  Promote income-generating activities, such as the cultivation of crops, NTFPs, and traditional medicines and foods, to build household incomes.  Provide food in schools  Promote vegetable gardens in schools  Integrate nutrition into curricula (Refer to figure 2 and table 2, pg. 11-13 of National Nutrition Strategy to 2025 and Plan of Action 2016-2020 Document) LRP 1.1: Improved Cost Effectiveness of Food Assistance LRP 1.1.1: Improved cost-effectiveness of procurement LRP 1.1.2: Improved cost-effectiveness of delivery LRP 1.1.3: Improved cost-effectiveness of distribution LRP 1.2: Improved Timeliness of Food Assistance LRP 1.2.1: Improved timeliness of procurement LRP 1.2.3: Improved timeliness of distribution LRP 1.2.2: Improved timeliness of delivery LRP 1.3: Improved Utilization of Nutritious and Culturally acceptable food that meet quality standards LRP 1.3.1: Increased access to culturally acceptable foods LRP 1.3.2: Strengthened local and regional food market systems  LRP 1.3.2.1: Increased agricultural productivity  LRP 1.3.2.2: Increased value added to post production agricultural products LRP 1.3.3: Improved Access to Nutritional Foods The program provided tools and seeds and imparted training to women and men smallholder farmers on modern agriculture methods and vegetable production. They were also taken on exposure visits for learnings exchange. The program established a link between schools and farmers, enabling the farmers to sell their vegetables directly to the school. Additionally, 800 KIP per student per day was transferred to the schools to buy nutritious vegetables for school meals. Also, some other livelihood activities were undertaken like imparting training on animal husbandry, provision of livestock to a few households, provision of a fund to weavers. All the income-generating support provided to such families was linked to school meals; those getting livestock or funds for weaving were required to contribute a portion of their earnings from these 105 livelihoods towards school meals. Training on nutrition and cooking to community representatives including, teacher, VEDC and parents were focused upon to improve their knowledge and attitude around the consumption of nutritious food. The school garden is already a part of the SMP of WFP and Integrating nutrition into the curriculum is also being promoted. Gender Focus The NNS shall promote gender roles, placing emphasis on women’s access to health services, to Nutrition and Food Security (NFS) information, and food. Women and girls shall receive an education and training and be enabled to earn an income and participate in household and community decision making on an equal basis with men. (Refer section 3.4.5, pg. 10 of National Nutrition Strategy to 2025 and Plan of Action 2016-2020 Document) SO1: Improved Effectiveness of Food Assistance through Local and Regional procurement The program has included gender equality aspects and has been able to achieve equal women participation in the program. Certain observations from the field in this regard showed that:  Majority of teachers across schools were women.  Program ensured participation of one man and one woman from selected farmer households, thus covering women participation under the program. However, at the same time, it was also observed that no specific provisions were made under the program to lend special focus or prioritize women participation. 106 Table 9: Mapping of Education Sector Development Plan 2020 and Logical Framework of LRP Education Sector Development Plan 2020 Logical Framework of LRP From the lessons learnt from past ESDP (2011-2015), the challenges to work upon were identified under the ESDP (2016-2020) document. The major challenge for the sector is foreseen as reducing dropout and repetition rates at Grade 1 and the need for school meals have been identified as one of the interventions needed to improve this scenario. Expected Outcome 1: Number of learners from ECE to lower secondary grade 9 increases with special focus on the disadvantaged and ensuring gender equity  Implementation of school meals, clean water, sanitation and safe facilities for disadvantaged school children, especially young girls, and those living in food-insecure districts in order to increase enrolments and retention. (Refer to section 2.3 pg.5 and section 4.4.1, pg. 10-11 of National Education and Sports Sector Development Plan 2016-2020 Document) LRP 1.3: Improved Utilization of Nutritious and Culturally acceptable food that meet quality standards LRP 1.3.1: Increased access to culturally acceptable foods LRP 1.3.2: Strengthened local and regional food market systems LRP 1.3.3: Improved access to nutritional foods The SMP is being implemented by WFP under and supplemented by SMP in District Nalae by ensuring availability of nutritious vegetables. The program provided 800 kips per student per day to help the school buy nutritious vegetables for the meals. Also, some other livelihood activities were undertaken, such as imparting training on animal husbandry, provision of livestock to a few families, provision of funds to weavers. All the support provided to these families was linked with the school meals; those getting livestock or fund for weaving were required to contribute a portion of their earnings from these livelihoods towards school meals. Training on nutrition and cooking to community representatives including, teacher, VEDC and parents were focused upon to improve their knowledge and attitude around the consumption of nutritious food. Gender Focus Objective 3: Human Asset Index target for exit from Least Developed Country status is achieved with reduced disparity. For example, gender, disability, poverty, ethnicity etc. 1. Secondary GER increases from 60% in 2015 to 75%. 2. Literacy rate of 15-24-year olds reaches 99%. 3. Literacy rate of 15-year olds and above reaches 95%. (Refer section 4.3.3, pg. 10 of National Education and Sports Sector Development Plan 2016-2020 Document) SO1: Improved Effectiveness of Food Assistance through Local and Regional procurement The program has included gender equality aspects and has been able to achieve equal or more women participation in the program. Though the program focused on ensuring nutritious school meals for all children irrespective of their gender, no special focus was laid on improving girl child enrolment or attendance in schools. Table 10: Mapping of WFP’s Country Strategic Plan (2017-2021) and Logical Framework of LRP WFP’s Country Strategic Plan (2017-2021) Logical Framework of LRP 107 The CSP highlights the need for WFP to continue providing food assistance in the short-term while also working to strengthen national and local capacities and investing in sustainable food and nutrition security programs to support the country’s progress towards MIC status and achievement of SDG 2 SO1: School Children in remote rural areas have Sustainable Access to Food by 2021  Provide school meals SO3: Vulnerable households in climate-sensitive districts are more resilient to seasonal and long-term shocks and stresses  Create productive assets and sustainable livelihood opportunities to build community resilience SO4: National and local governance institutions are Strengthened to improve service delivery, especially in hard-to-reach areas, by 2025  Develop the capacity of rural communities in designing and implementing nutrition-sensitive development plans (Refer to section 3.2 pg.8-16 of WFP’s Country Strategic Plan 2017-2021 Document) SO1: Improved Effectiveness of Food Assistance through Local and Regional procurement LRP 1.1: Improved Cost Effectiveness of Food Assistance LRP 1.1.1: Improved cost-effectiveness of procurement LRP 1.1.2: Improved Cost-effectiveness of delivery LRP 1.1.3: Improved cost-effectiveness of distribution LRP 1.2: Improved Timeliness of Food Assistance LRP 1.2.1: Improved timeliness of procurement LRP 1.2.3: Improved timeliness of distribution LRP 1.2.2: Improved timeliness of delivery LRP 1.3: Improved Utilization of Nutritious and Culturally acceptable food that meet quality standards LRP 1.3.1: Increased access to culturally acceptable foods LRP 1.3.2: Strengthened local and regional food market systems  LRP 1.3.2.1: Increased agricultural productivity  LRP 1.3.2.2: Increased value added to post production agricultural products LRP 1.3.3: Improved access to nutritional foods LRP 1.4: Conducive Foundational Environment Created LRP 1.4.1: Increased capacity of government institutions LRP 1.4.3: Improved capacity of relevant organizations LRP 1.4.4: Increased leverage of private sector resources LRP program is supplementing School Meals Program of WFP USDA McGovern-Dole by ensuring the availability of locally grown nutritious vegetables for school meals. The program provided tools, seeds and training to local smallholder farmers on modern agricultural techniques and vegetable production. The program provided 800 kips per student per day to help the school buy nutritious vegetables for the meals. Also, some other livelihood activities were undertaken, such as imparting training on animal husbandry, provision of livestock to a few families, provision of funds to weavers. All the support provided to these families was linked with the school meals; those getting livestock or fund for weaving were required to contribute a portion of their earnings from these livelihoods towards school meals. Gender Focus All strategic outcomes will contribute to Sustainable Development Goal (SDG) 2 and be in synergy with the other SDGs, particularly SDG 5 on gender, SDG 4 on SO1: Improved Effectiveness of Food Assistance through Local and Regional procurement 108 education (strategic outcome 1), and SDG 13 on climate action (strategic outcome 3) and sustainable development. (Refer to section 1.2 pg.5 of WFP’s Country Strategic Plan 2017-2021 Document) Objective 3: Human Asset Index target for exit from Least Developed Country status is achieved with reduced disparity. For example, gender, disability, poverty, ethnicity etc. 1. Secondary GER increases from 60% in 2015 to 75%. 2. Literacy rate of 15-24-year olds reaches 99%. 3. Literacy rate of 15-year olds and above reaches 95%. (Refer to section 4.3.3, pg. 10 of National Education and Sports Sector Development Plan 2016-2020 Document) The program has included gender equality aspects and has been able to achieve equal women participation in the program. Certain observations from the field in this regard showed that:  Majority of teachers across schools were women.  Program ensured participation of one man and one woman from selected farmer households, thus covering women participation under the program. However, at the same time, it was also observed that no specific provisions were made under the program to lend special focus or prioritize women participation. USDA Local Regional Procurement Activity Evaluation in Lao PDR Final Report 109 Annex R List of Tables for Effectiveness Indicators Indicator: Number of individuals benefitting directly through Local & Regional Procurement Time Period Semester No. Target (in nos.) Achieved (in nos.) % Achieved Apr’17-Sept’17 I 5000 4140 82.8 Oct’17-Mar’18 II 5000 3936 78.7 Apr’18-Sept’18 III 5000 3484 69.7 Oct’18-Mar’19 IV 5000 4406 88.1 Apr’19-Sept’19 V 5000 4973 99.4 Indicator: Number of individuals benefitting indirectly through Local & Regional Procurement Time Period Semester No. Target (in nos.) Achieved (in nos.) % Achieved Apr’17-Sept’17 I 25000 20700 82.8 Oct’17-Mar’18 II 19640 78.6 Apr’18-Sept’18 III 13936 55.7 Oct’18-Mar’19 IV 22913 91.7 Overall 22913 91.7 Indicator: Value of sales by project beneficiaries Time Period Semester No. Target (in USD) Achieved (in USD) % Achieved Apr’17-Sept’17 I $25000 Data not available n/a Oct’17-Mar’18 II $75000 $53240 71.0 Apr’18-Sept’18 III $75000 $25500 n/a Oct’18-Mar’19 IV $75000 $71937 95.9 Apr’19-Sept’19 V $100000 $37000 37.0 Indicator: Volume of commodities (MT) sold by project beneficiaries Time Period Semester No. Target (in metric tonnes) Achieved (in metric tonnes) % Achieved Apr’17-Sept’17 I 120 mt Data not available n/a Oct’17-Mar’18 II 150 mt 9.4 mt 6.3 Apr’18-Sept’18 III 150 mt 42.3 n/a Oct’18-Mar’19 IV 130 mt 122.3 mt 94.0 Apr’19-Sept’19 V 420 mt 66 mt 15.7 USDA Local Regional Procurement Activity Evaluation in Lao PDR Final Report 110 Indicator: Number of public-private partnerships formed as a result of USDA assistance Time Period Semester No. Target (in nos.) Achieved Apr’17-Sept’17 I 0 Data not available Oct’17-Mar’18 II 10 19 Apr’18-Sept’18 III 10 10 Oct’18-Mar’19 IV 0 10 Apr’19-Sept’19 V 0 0 Indicator: Value of public and private sector investments leveraged as a result of USDA assistance Time Period Semester No. Target (in USD) Achieved (in USD) Apr’17-Sept’17 I 0 Data not available Oct’17-Mar’18 II $14400 Data not available Apr’18-Sept’18 III $14400 Data not available Oct’18-Mar’19 IV $14400 $82659 Apr’19-Sept’19 V $14400 0 Indicator: Total increase in installed storage capacity (dry or cold storage) as a result of USDA assistance Time Period Semester No. Target (meter cubic) Achieved (meter cubic) Apr’17-Sept’17 I 338 Data not available Oct’17-Mar’18 II 338 1152 Apr’18-Sept’18 III 338 0 Oct’18-Mar’19 IV 0 0 Apr’19-Sept’19 V 1013 0 Indicator: Number of policies, regulations and/or administrative procedures in each of the following stages of development as a result of USDA assistance Time Period Semester No. Target (in nos.) Achieved (in nos.) Apr’17-Sept’17 I 1 1 Oct’17-Mar’18 II 0 0 Apr’18-Sept’18 III 0 0 Oct’18-Mar’19 IV 1 1 Apr’19-Sept’19 V 0 0 USDA Local Regional Procurement Activity Evaluation in Lao PDR Final Report 111 Indicator: Quantity of commodity procured (MT) as a result of USDA assistance Time Period Semester No. Target (in metric tonnes) Achieved (in metric tonnes) Apr’17-Sept’17 I 120 Data not available Oct’17-Mar’18 II 150 40.3 Apr’18-Sept’18 III 150 Data not available Oct’18-Mar’19 IV 300 Data not available Apr’19-Sept’19 V 420 0 Indicator: Cost of commodity procured as a result of USDA assistance (by commodity and source country) Time Period Semester No. Target (in USD) Achieved (in USD) Apr’17-Sept’17 I 75000 Data not available Oct’17-Mar’18 II 25000 48045 Apr’18-Sept’18 III 25000 Data not available Oct’18-Mar’19 IV 50000 0 Apr’19-Sept’19 V 100000 0 Indicator: Cost of transport, storage, and handling of commodity procured as a result of USDA assistance (by commodity) Time Period Semester No. Target (in USD) Achieved (in USD) Apr’17-Sept’17 I 5750 5750 Oct’17-Mar’18 II 5900 5220 Apr’18-Sept’18 III 0 7000 Oct’18-Mar’19 IV 0 Data not available Apr’19-Sept’19 V 11650 Data not available Indicator: Number of social assistance beneficiaries participating in productive safety nets as a result of USDA assistance Time Period Semester No. Target (in nos.) Achieved (in nos.) Apr’17-Sept’17 I 2500 2500 USDA Local Regional Procurement Activity Evaluation in Lao PDR Final Report 112 Oct’17-Mar’18 II 2500 3928 Apr’18-Sept’18 III 2500 3451 Oct’18-Mar’19 IV 2500 3414 Apr’19-Sept’19 V 2500 1132 Indicator: Number of individuals who have received short-term agricultural sector productivity or food security training as a result of USDA assistance Time Period Semester No. Target (in nos.) Achieved (in nos.) Apr’17-Sept’17 I 500 460 Oct’17-Mar’18 II 500 474 Apr’18-Sept’18 III 500 36 Oct’18-Mar’19 IV 500 38 Apr’19-Sept’19 V 500 53 Indicator: Number of nutritional meals (feeding days) prepared by schools Time Period Semester No. Target (in no. of days) Achieved (in no. of days) Apr’17-Sept’17 I 5000 4550 Oct’17-Mar’18 II 4375 4944 Apr’18-Sept’18 III 4375 2400 Oct’18-Mar’19 IV 4375 4512 Apr’19-Sept’19 V 4375 Data no available Indicator: Number of crop types grown Time Period Semester No. Target (in no. of crops) Achieved (in no. of crops) Apr’17-Sept’17 I Data no available 4 Oct’17-Mar’18 II Data no available 6 Apr’18-Sept’18 III Data no available 14 Oct’18-Mar’19 IV Data no available 20 Apr’19-Sept’19 V Data no available 20 Indicator: % farmers who implement best practices from their farmer trainings Time Period Semester No. Target (in %) Achieved (in %) USDA Local Regional Procurement Activity Evaluation in Lao PDR Final Report 113 Apr’17-Sept’17 I Data no available Data not available Oct’17-Mar’18 II 70 39 Apr’18-Sept’18 III 70 50 Oct’18-Mar’19 IV 80 80 Apr’19-Sept’19 V Data no available 80 USDA Local Regional Procurement Activity Evaluation in Lao PDR Final Report 114 Annex S Data Collection Tools Focus Group Discussions: Village Education Development Committee (VEDC) Name of the School/VEDC: Name of the Village: Name of the District: No. of members: S. No Name Age Gender Position Appointment, Role and Responsibilities 1. In which year was the VEDC formed? With what objectives was the VEDC formed? How has the expected role of VEDC evolved over the years? Who all are the members of VEDC - do you have members from the Lao Women’s Union, Lao Youth Union and parent’s representatives? 2. How often does the VEDC meet? In the last one year, how many times has the VEDC met? Who all attend the VEDC meeting? Do you maintain a meetings register? (Ask for it if available and check if there are notes of the meetings.) What are usual agenda discussed during VEDC meetings? About the Village 3. Please tell us about your village - What are the different ethnic groups here? What languages do they speak? How many households are there in this village? What are the main occupations practiced? USDA Local Regional Procurement Activity Evaluation in Lao PDR Final Report 115 4. Can you please tell us a bit about the climatic conditions in and around your village throughout the year? What is the situation with respect to availability of water resources in your village? What all other natural resources are available around your village? 5. How many schools are there in your village? Till which grade? How do children commute to school? Are these some children in your village who do not go to school? (Probe especially for children 5-10 years in pre-primary and primary school). What are the reasons for not going to school? Local Regional Procurement (LRP) Program 6. How many schools from your village are being provided assistance under Local Regional Procurement Program of WFP? What type of assistance is being provided? Are the same schools being provided benefits under Schools Meals Program? What type of benefits are provided under SMP? (Difference between the two programs, do they complement each other? Can LRP function independently without SMP in place?) 7. What are the different departments involved in implementation of the LRP program? What are the different activities being undertaken in the LRP program (probe for training of farmers, learning and exposure visits, provision of seeds and tools, CBT, etc.)? 8. What are the current mechanisms in place to monitor the progress under the program at village level? Who all are involved in the process, and what all activities are monitored? What actions are usually taken based on the findings from monitoring exercise? What issues and challenges are usually experienced in monitoring of progress? USDA Local Regional Procurement Activity Evaluation in Lao PDR Final Report 116 9. Have there been any trainings provided to you under the program? What are the responsibilities assigned to VEDC members under the program (what help is provided by VEDC to the school, does it mobilise the community in any way for the school lunch - to volunteer as cooks, seek support from farmer groups, awareness generation among community, monitoring etc.)? Have you been part of any learning and exposure visits during the course of the program? 10. Does the school receive monthly cash transfer under LRP? How much and for what purposes? From whom? Who is responsible for handling the cash and how is it decided? How is this budget usually spent? Who all are involved in taking financial decisions and procuring food or non-food materials from the budget? 11. Is there a functional kitchen for cooking of meals under the program? Who is responsible for procurement of fruits and vegetables, and cooking? Who decides the menu for meals to be provided at schools? Have there been trainings provided regarding cooking – please explain the nature of such trainings? Is there a mechanism to match the agreed menu with what farmers grow and supply to schools? 12. Does the school source vegetables from the local farmer groups? How does the VEDC support the school in this? Is this supply free of cost, or are farmers paid? How are the farmers paid? On an average, how many days in a week does the VEDC procure vegetables from local farmers? What type of vegetables are procured – kindly specify for all seasons? What mechanism are in place to ensure safe storage of food materials? How has the program contributed towards improving mechanisms for storing food materials? 13. Who decides on selection of farmers for training and procurement of vegetables? Who decides on type of vegetables to be grown by farmers? What inputs have been provided to the selected farmers under the program – probe specifically for women and men farmers separately and the most (socially and economically) vulnerable farmers? USDA Local Regional Procurement Activity Evaluation in Lao PDR Final Report 117 14. Do farmers and other community members also contribute in cash or other materials (like chicken, meat, fuelwood) for school meals? How frequently? How is this decided – consistent for all or individually decided – same for parents and non-parents? What is the role of VEDC in this process? Effect of LRP on School Children and their Parents 15. What impact have you observed among pre-primary and primary students in school – particularly (1) before SMP, (2) after SMP but without LRP, and (3) after LRP with respect to nutrition, attendance, and learning outcomes? What could be the potential reasons for this? Have you observed any other changes among students and within the school across the three time frames? Kindly explain. 16. In this school, on an average, how many days in a week is the school lunch provided to the students? If less than 5 days, what are the reasons for not providing the school lunch? Is it quantity of the food supplied, or lack of vegetables or lack of water or fuel or absence of the cook? 17. Does the school have a school garden? Is the produce from the garden used in the school lunch? How many times in a month does the school use produce from school garden for lunch? What are the benefits of having a school garden? What are the challenges in maintaining the garden? (probe for lack of water, lack of labour to tend to the garden) 18. Have you observed any change in community’s attitude in perceiving the importance of nutrition of their children? How do the parents interact with the VEDC? What are the issues that the parents bring to the VEDC when it comes to the school lunch? How does the VEDC resolve them? a. What changes have you observed among the farmers participating in this program? (Probe for change in productivity, income, dietary diversity at home, efforts and overall quality of life). USDA Local Regional Procurement Activity Evaluation in Lao PDR Final Report 118 b. Do you see any changes with respect to improvement in sensitization, participation and decision making, and reduction in discrimination among women and men small landholder farmers? c. What changes have you observed with respect to community-level participation and commitment towards delivery of school lunch implementation on a continuous basis? 19. What are the challenges in the LRP program? (Probe for conflicts in procurement and allocation of responsibilities, equal contribution to work, etc.) How do you think the LRP program can be improved? 20. In your opinion, how effective is the Local Regional Procurement program in improving the well-being of our children, in terms of nutrition, education, and health and local farmers, in terms of agricultural output, nutritional intake and improved resilience? Thank you for your valuable time, your feedback is much appreciated. USDA Local Regional Procurement Activity Evaluation in Lao PDR Final Report 119 Individual and Group Discussions: Farmers INTERVIEW WITH FARMERS First four pages of the tool is to be filled for each of the respondents separately. Enumerators to carry 8-10 copies for the first four pages and administer one-by-one for each farmer. Participant Information Name: Village: District: Demographic and Socio-Economic Information Particulars Options Code Remarks 1. Gender of Respondent (farmer) Male 1 Female 2 2. Gender of Head of HH Male 1 Female 2 3. Number of HH members Total: Female: Male: 4. Education Status of the respondent (farmer) Illiterate 1 Completed Primary (1st-5th) 1 Completed Secondary (6th -8th) 2 Completed High School (9th-12th) 3 Above 12th 4 5. Marital Status of the respondent (farmer) Single 1 Married 2 Widowed 3 Divorced 4 6. How many members are involved in economic activities? 7. What are the different sources of household income? (List) 8. Total HH Income Earned per month In Kip …………… 9. How much of the monthly household income comes from agriculture? In Kip …………… 10. How much land size do you practice agriculture on? In Square meters …………… 11. How many seasons and months in a year, on an average do you practice agriculture? Seasons …………… Months …………… USDA Local Regional Procurement Activity Evaluation in Lao PDR Final Report 120 Child Specific Information Particulars Options Code Remarks 12.How many children do you have? How many of them are in the age group 3-10 years? Total No. of Children No. of Children (3-10 years) Select the youngest child in the age group 3-10 years and ask the questions in the section with respect to that child 13.How many days on an average in a month does your child attend school? < 7 days 1 7-15 Days 2 15-20 Days 3 >20 Days 4 14.How many days in a week no meals are provided to children in schools? No. of days _______________ Skip to 14, if coded 0 15.What are the reasons for not providing meals in school during certain days of the week? Cook absent 1 Any event in school 2 No supply for vegetables or meat 3 No supply of rice, lentils or oil 4 No reason 5 Don’t know 9 Others (specify) Information related to food intake at home Particulars Options Code Remarks 16.On an average how many meals did your family eat per day in last week? 1 meal 1 2 meals 2 3 meals 3 17.How many meals did your family eat yesterday? ………………………… 18.(a) How many days in the past week did your family eat the following food item? (b) How many times did your family eat these food items per day? S. No. Food Item No. of Days Eaten in last week No. of Days Eaten per day a) Rice b) Maize / Corn c) Cassava d) Other roots and tubers (potatoes, yam) e) Pulses/Lentils/Tofu/Curd f) Vegetables Green Leafy Carrots Onions Tomatoes g) Bamboo shoots h) Mushrooms USDA Local Regional Procurement Activity Evaluation in Lao PDR Final Report 121 i) Fruits j) Fish/ fish paste k) Other aquatic animals (crabs, snails, shrimps, etc.) l) Meat (beef, pork) m) Chicken n) Wild animals o) Eggs p) Milk/ milk products q) Sugar r) Oil/Butter Information related to program inputs Particulars Options Code Rema rks 19.Have you received seeds under LRP program? Yes 1 No 2 20.If yes, pls. specify number of seeds types you received under LRP program? No. of types of seeds received …………………. 21. Kindly specify the types of seeds/plants received, in each of the three seasons? (Tick under appropriate seasons) Seeds/Plants Seaso n 1 Seaso n 2 Seaso n 3 Beans Garlic Tomatoes Pumpkins Morning Glory Cucumber Long Bean Chilli Eggplant Bottle Guard 22.In addition to these, did the program also provide you with chickens, fish fingerlings or fruit trees? (Tick for relevant items) Chickens Fish Fingerlings Fruit Trees Piglets 23.Have you received tools under LRP program? Yes 1 No 2 24.Kindly specify the types of tools received? (Tick for relevant items) Water Buckets Manual Water Sprinklers Sickles USDA Local Regional Procurement Activity Evaluation in Lao PDR Final Report 122 Particulars Options Code Rema rks Plastic Basin Nails PVC Others specify 25. Were all the tools received in one￾go, or in a staggered manner? One-go 1 Stagerred Manner 2 Don’t Remember 9 26.Have you received trainings under LRP program? Yes 1 No 2 27.If yes, what types of trainings have you received under LRP program? How many times have you attended the trainings in last two years? Trainings Code No. of Times Procurement of Seeds or cuttings 1 Quantity of seeds to select 2 Quality of seeds to select 3 Preservation of seeds for future crop cycles and plantation 4 Life cycle of crops 5 Reparation of soil & daily care 6 Harvesting 7 Propagation of crops 8 Natural pest control & fertilizers 9 Basics on Storage of crops 10 Knowledge of greenhouse contruction 11 Making compost 12 Other (Specify) 28.Have you been sent for a learning and exposure visit under the program? Yes 1 No 2 Qualitative Tool- WFP LRP End-line Evaluation in Lao PDR 123 FOCUS GROUP DISCUSSIONS: FARMERS’ GROUP Introduction 1. Please tell us about your village – What are the different ethnic groups here? What languages do they speak? How many households are there in this village? What are the main occupations practiced? 2. Can you please tell us a bit about the climatic conditions in and around your village throughout the year? What is the situation with respect to availability of water resources in your village? What all other natural resources are available around your village? Local Regional Procurement (LRP) Program 3. How many months in a year do you practice farming and in which months? What crops are usually grown in different seasons? Fill the table provided below – and probe for reasons for change in crops grown for each of the three seasons. Details Season 1 (Specify crops grown) Season 2 (Specify crops grown) Season 3 (Specify crops grown) Pre-LRP Post-LRP 4. What are the status of land ownership in your village – what variations do we see in terms of land ownership and use of such land for agriculture in your village? To what extent are farmers practicing single or multiple cropping? And what are the reasons for this? What apprehensions do farmers have for practicing multiple cropping? What is the status of use of bio-pesticides and fertilizers in farms? 5. What are the livelihood activities, other than agriculture that you all are involved in? How does the involvement change during different months of the year? Qualitative Tool- WFP LRP End-line Evaluation in Lao PDR 124 6. Are there any farmer groups in and around your village? Is anybody part from this group part of the farmer groups? What advantages and disadvantages do you see as a result of having a farmer group? 7. What does LRP program mean to farmers like you? What kind of benefits did you receive under the program? Probe for trainings, provision of tools, seeds and plants, access to market etc. for women and men farmers separately, and the most (socially and economically) vulnerable farmers). Were these inputs/benefits sufficient and timely in nature? What kind of changes have you experienced as a result of the program? If yes, to what extent were the capacity building measures under the program were in line your needs (women and men small holder farmers)? 8. How were the farmers from this village selected to be covered under the LRP program? How did the farmers who were left out of the program respond? Were there any conflicts in the process? Are there instances where new farmers intend to be included in the program – what benefits do they visualise in this program? Have they been included? 9. Did you apply the knowledge imparted by way of trainings in your own farming? Please tell us a bit about the trainings you received and the manner in which you applied this knowledge in your farming? Have you experienced any change in quantity and quality of output of farming thereafter? Has your income increased over last two years? If yes, what reasons do you think contributed towards the increase in income (improved quality of produce, improved knowledge on crop rotation, diversifying crops, etc.)? To what extent has the program successfully linked you with the local markets and the schools for meal program - How? What contribution do you see of this program in providing solutions to farmers like you to practice nutrition focussed agriculture? Qualitative Tool- WFP LRP End-line Evaluation in Lao PDR 125 10. Where do you procure seeds, tools, fertilizers, pesticides, etc. from? How far is the market for procurement? What are the challenges you face in procuring farm inputs? Did you receive any benefit under LRP program on inputs procurement for farming? How frequently? Do you have any feedback on the nature of support provided under the program with respect to inputs? 11. To what extent do you use regenerated seeds – and for what crops? What difference in the quantity of produce do you observe between mother seeds and regenerated seeds? 12. What are the major challenges that farmer like you face while growing different types of vegetables? Probe for availability of water and extreme weather conditions (landslides and flash floods). In what manner did the program support farmers like you? 13. Do you supply vegetables to schools? How did you decide which schools would you supply to and when? What vegetables do you supply to schools and how frequently? How does this change by every season? What crops are grown during the lean season (May-August), when no supply is made to schools? For what purpose? How are these decisions made – collectively or individually? Do you experience wastage during lean season – are there any plans to mitigate this risk? 14. Do you foresee a risk of overproduction of vegetables, as a result of all farmers growing same set of vegetables? What potential consequences do you visualise as a result of overproduction? How do you all plan to mitigate this risk in future? Has the program ever talked or trained you all to handle such risk? Qualitative Tool- WFP LRP End-line Evaluation in Lao PDR 126 15. Who decides on types of vegetables to be grown and supplied to school? Probe for role of VEDC and cooks. Do farmers like you sell this produce or merely supply free of cost/at discounted rates? How much quantity – consistent for all farmers or individually decided? What challenges do you and farmers like you face with respect to supplying vegetables to schools? What is the role of VEDC in this process? 16. Do you and farmers like you also contribute in cash or other materials (like chicken, meat, fuelwood) for school meals? How frequently? How is this decided – consistent for all farmers or individually decided – same for parents and non-parents? What is the role of VEDC in this process? 17. Are you able to produce enough to be able to sell in markets other than schools? Where else do you sell the produce from your farming? What is the mechanism for selling the produce to the traders/in the market? What challenges do you and other farmers like you face in accessing market and selling produce? Has the mechanism for accessing traders and market changed in the last two-three years under the program – how? Did you receive any benefit under LRP program with respect to accessing market for selling farm produce? 18. What changes have you observed in the overall farm productivity, income, effort level and overall quality of life of farmers? a. Do you see any changes with respect to improvement in sensitization, participation and decision making, and reduction in discrimination among women and men small landholder farmers? 19. What changes have you observed with respect to community-level participation and commitment towards delivery of school lunch implementation on a continuous basis? 20. Where do you store your farm inputs and final produce for sustaining through the lean season? Do you have enough storage space? Did you receive any support or guidance regarding storage under the program – kindly specify? Is the storage area safe from natural threats? (Challenges and support needed in this direction). Qualitative Tool- WFP LRP End-line Evaluation in Lao PDR 127 21. What are the challenges in the LRP program (Probe for conflicts and challenges)? How do you think the LRP program can be improved? 22. In your opinion, how effective is the Local Regional Procurement program in improving the well-being of our children, in terms of nutrition, education, and health, and local farmers, in terms of improved agricultural output, nutritional intake and resilience? Use the H-form tool for this question. Not at all well= 0 (when the program started); extremely well = 10. Ask them to rate the effectiveness of the program on a scale of 1-10. Based on their score, ask the positive reasons for their achievement. Then ask why have they given the score? Why isn’t the score 10? Then ask for how this score can be improved in the future? Thank you for your valuable time, your feedback is much appreciated. Ways this score could be improved in the future Extremely well Not at all well How well have the Program interventions (CBT, dietary diversity, farmers training, etc.) improved the well-being of children and farmers in the project period? Negative reasons Positive reasons 6.5 Qualitative Tool- WFP LRP End-line Evaluation in Lao PDR 128 INDIVIDUAL AND GROUP DISCUSSIONS: PARENTS INTERVIEW WITH PARENTS First four pages of the tool is to be filled for each of the respondents separately. Enumerators to carry 8-10 copies for the first four pages and administer one-by-one for each farmer. Participant Information Name: Village: District: Demographic and Socio-Economic Information Particulars Options Code Remarks 1. Gender of Respondent (farmer) Male 1 Female 2 2. Gender of Head of HH Male 1 Female 2 3. Number of HH members Total: Female: Male: 4. Education Status of the respondent (farmer) Illiterate 1 Completed Primary (1st-5th) 1 Completed Secondary (6th -8th) 2 Completed High School (9th-12th) 3 Above 12th 4 5. Marital Status of the respondent (farmer) Single 1 Married 2 Widowed 3 Divorced 4 6. How many members are involved in economic activities? 7. What are the different sources of household income? (List) 8. Total HH Income Earned per month In Kip …………… 9. How much of the monthly household income comes from agriculture? In Kip …………… Child Specific Information Particulars Options Code Remarks 10.How many children do you have? How many of them are in the age group 3-10 years? Total No. of Children No. of Children (3-10 years) Qualitative Tool- WFP LRP End-line Evaluation in Lao PDR 129 Select the youngest child in the age group 3-10 years and ask the questions in the section with respect to that child 11.How many days on an average in a month does your child attend school? < 7 days 1 7-15 Days 2 15-20 Days 3 >20 Days 4 12.How many days in a week no meals are provided to children in schools? No. of days _______________ Skip to 14, if coded 0 13.What are the reasons for not providing meals in school during certain days of the week? Cook absent 1 Any event in school 2 No supply for vegetables or meat 3 No supply of rice, lentils or oil 4 No reason 5 Don’t know 9 Others (specify) Information related to food intake at home Particulars Options Code Remarks 14.On an average how many meals did your family eat per day in last week? 1 meal 1 2 meals 2 3 meals 3 15.How many meals did your family eat yesterday? ………………………… 16.(a) How many days in the past week did your family eat the following food item? (b) How many times did your family eat these food items per day? S. No. Food Item No. of Days Eaten in last week No. of Days Eaten per day s) Rice t) Maize / Corn u) Cassava v) Other roots and tubers (potatoes, yam) w) Pulses/Lentils/Tofu/Curd x) Vegetables Green Leafy Carrots Onions Qualitative Tool- WFP LRP End-line Evaluation in Lao PDR 130 Tomatoes y) Bamboo shoots z) Mushrooms aa) Fruits bb) Fish/ fish paste cc) Other aquatic animals (crabs, snails, shrimps, etc.) dd) Meat (beef, pork) ee) Chicken ff) Wild animals gg) Eggs hh) Milk/ milk products ii) Sugar jj) Oil/Butter USDA Local Regional Procurement Activity Evaluation in Lao PDR Final Report 131 FOCUS GROUP DISCUSSIONS: PARENTS’ GROUP Introduction 1. Please tell us about your village – What are the different ethnic groups here? What languages do they speak? How many households are there in this village? What are the main occupations practiced? Education and Schooling of Children 2. How many schools are there in your village? Till which grade? If no school within the village, then what is the distance to the nearest pre-primary and primary schools? How do children commute to school? 3. Are these some children in your village who do not go to school (all ethnic groups)? (Probe especially for children 5-10 years in pre-primary and primary school). What are the reasons for not going to school? (Probe to understand if there are different things that prevent boys and girls from going to school). Local Regional Procurement (LRP) Program 4. Have you heard of Local Regional Program? Did you or your child receive any benefit under the program? What type of benefits (school meals, training, etc.) have you or your child received, if any? 5. How many days does the school provide lunch in a week? If not every day, why does the school not provide lunch on a regular basis? What do children eat when the lunch is not provided in the school? USDA Local Regional Procurement Activity Evaluation in Lao PDR Final Report 132 6. Please let us know about the quantity of food served during lunch in schools? Do you think the food served as lunch is sufficient for your children? On days when the lunch is provided, does your child still have lunch at home? 7. Are you satisfied with the quality of school lunch? Do you think it is nutritious for your children? Please tell us your views regarding the diversity in school lunch provided to children in school? Who decides about the menu? From where does the school procure fruits and vegetables? Do you think it is cooked properly and in hygienic manner? Please elaborate, why do you think so? 8. What, according to you are benefits of the LRP program? a. How has it affected the attendance of children in school and their interest in going to schools? b. How has it helped improve the nutrition of your children? Has it changed the way you cook food at home? c. Has it addressed children’s short term hunger in school in any manner? Do you think it helps children concentrate or be more attentive in class? d. Has the school lunch reduced your household expenditure in any way? How? 9. Does the school have a school garden? If yes, what all is grown there? What is the purpose of having a school garden? Kindly inform us about the mechanism of parents and VEDC members volunteering in the school garden? 10. Prior to current LRP program, did the school have school meal program (SMP) for children? How do you compare that with the current LRP program? What changes have you observed USDA Local Regional Procurement Activity Evaluation in Lao PDR Final Report 133 among your child and other children in the vicinity (1) before SMP, (2) after SMP but without LRP, and (3) after LRP? What could be the potential reasons for this? Kindly explain. 11. If you have questions or issues regarding LRP program, whom do you approach first? What is the role of VEDC members with respect to LRP program? Do you have a representative in the VEDC? What is the role of school authorities? 12. What are the challenges in the LRP program (Probe for conflicts and challenges)? How do you think the LRP program can be improved? 13. In your opinion, how effective is the Local Regional Procurement program in improving the well-being of our children in terms of nutrition, education, and health, and farmers, in terms of agricultural produce, income, nutrition intake, etc.? Use the H-form tool for this question. Not at all well= 0 (when the program started); extremely well = 10. Ask them to rate the effectiveness of the program on a scale of 1-10. Based on their score, ask the positive reasons for their achievement. Then ask why have they given the score? Why isn’t the score 10? Then ask for how this score can be improved in the future? USDA Local Regional Procurement Activity Evaluation in Lao PDR Final Report 134 Thank you for your valuable time, your feedback is much appreciated. Ways this score could be improved in the future Extremely well Not at all well How well have the Program interventions (say, school meals or trainings) improved the well￾being of children and farmers in the project period? Negative reasons Positive reasons 6.5 USDA Local Regional Procurement Activity Evaluation in Lao PDR Final Report 135 Interview with Children Participant Information Name: Class: School: Village: Age: Date: Enumerator: Program Related Information Please tick mark (√) the most appropriate response according to you for each question. Please note that there are no right or wrong responses to the questions asked. Particulars Options Code Remarks 1. How many days in last week did you attend school? < 3 days 1 3-5 Days 2 Every day when school is open 3 2. Do you like coming to school? Yes 1 No 2 3. Are you provided lunch in school? Yes 1 No 2 4. How many days in the last week were you provided lunch in the school? < 2 days 1 2-3 Days 2 3-5 Days 3 Every day when school is open 4 5. Do you like the taste of the food that is provided to you? Yes 1 No 2 6. How many school meals on an average per week have chicken, meat, pork, fish or beef in it? ………………………………………. 7. Do you go back home immediately after eating school meal? Yes 1 No 2 8. Are you taught about vegetables and crops in your curriculum? Yes 1 No 2 9. Can you name any six vegetables that you have read about in your curriculum? 1. ………………… 2. …………………. 3. …………………. 4. ………………… 5. ………………… 6. …………………. 10.Of the vegetables you are taught about, which all did you get to eat in school meals last week? 7. ………………… 10. ………………… 8. …………………. 11. ………………… 9. …………………. 12. ………………… 11.How many meals did you eat yesterday? ………………………………………. Thank you for your time! USDA Local Regional Procurement Activity Evaluation in Lao PDR Final Report 136 KEY INFORMANT INTERVIEW: SCHOOL HEADMASTER Name of Headmaster:_____________ Name of School:____________ Name of Village:______________ 1. When was this school established? Does each grade have a separate class or do they share space? How many students are enrolled in your school? What is the usual attendance percentage in the school? Are there sufficient number of teachers in school (separate teachers for each class and subject)? 2. Is this school supported with WFP’s Local Regional Procurement Program? Since When? What are the kinds of support provided by WFP under the LRP (Probe on Cash transfer benefit; procurement of vegetables; training or capacity building support; etc.)? How is it different from School Meals Program? 3. What all government departments are involved in LRP? What is the role of government departments/agencies in implementation of LRP? 4. Apart from you, are there any other people associated with the LRP (for instance, Teachers, Cook, Store Managers, VEDC members, Parents, Farmers Groups etc.)? Please tell us briefly the roles of each of these stakeholders in the LRP? USDA Local Regional Procurement Activity Evaluation in Lao PDR Final Report 137 5. Does the school receive monthly cash transfer under LRP? How much and for what purposes? From whom? Who is responsible for handling the cash and how is it decided? How this budget is usually spent? Who all are involved in taking financial decisions and procuring food or non-food materials from the budget? 6. What are the contributions of different stakeholder including community/parents (cash or kind) in assuring school meals to children? Please provide details on average Monthly Contribution (including monetized value of contributions from community in kind) and Expenditure on School Lunch. Contribution by Contribution in Kind Contribution in Cash Type Quantity CBT-LRP CBT-SMP Parents Farmers Other Community Members Kitchen Garden Other 7. Is there a functional kitchen in school for cooking of meals? What types of vegetables are generally procured for school meal? Who decides the menu of the school meals (variety in the meals prepared at school)? Who is responsible for procurement of vegetables? 8. Does the school source vegetables from the local farmer groups? How does the program support the school in this? Is this supply free of cost, or are farmers paid? How are the farmers paid? On an average, how many days in a week does the school procure vegetables from local farmers and in what quantities? What type of vegetables are procured – kindly specify for all seasons? Who decides on selection of farmers for procurement of vegetables? USDA Local Regional Procurement Activity Evaluation in Lao PDR Final Report 138 9. In case of voluntary contribution made by the farmers in the form of vegetables, how is the regularity of supply ensured? How is the food menu matched with the vegetables grown and supplied by the farmers? 10. What mechanism are in place to ensure safe storage of food materials? How has the program contributed towards improving mechanisms for storing food materials? 11. Who is responsible for cooking the school lunch? Is the person permanent or people from community take responsibility for cooking on a rotational basis? Is anything paid in cash or kind to the cook? What kind of training has the cook been provided under LRP? 12. How many days in the last week did the school provide lunch? If less than 5 days, what was the constraint in providing lunch on a regular basis? What do the children eat when the lunch is not provided in the school? What are the measures that can be taken up to increase the provision of school lunch? Do children like the taste of the school meal? 13. What vegetables are usually grown in the school garden? Who decides what needs to be grown? How many times in a month are you able to harvest produce from school garden that meets that particular day’s vegetable needs? What are the benefits of the school garden? Who is responsible USDA Local Regional Procurement Activity Evaluation in Lao PDR Final Report 139 for maintenance of the school garden? Are there any challenges in maintaining the school garden? 14. Does the school have a nutrition curriculum for students? Have there been changes in the curriculum post implementation of LRP (inclusion of vegetables grown by local farmers in the curriculum)? What vegetables and crops are included in the curriculum of children? 15. Have you, your teachers, your students or VEDC members received any training from WFP? On what issues were you trained? Was the training helpful (Probe for both yes and no)? 16. According to you, what are the three significant positive changes that LRP has been able to make? (Probe for increase in enrolment, attendance, children’s improved attentiveness, increase in school teacher’s interest, improved infrastructure, parent’s increase in interest in education, consumption of nutritious food in school and at home, benefit to local farmers, etc.) 17. What impact have you observed in the attendance among pre-primary and primary students in school – particularly (1) before SMP, (2) after SMP but without LRP, and (3) after LRP? What could be the potential reasons for this? Have you observed any other changes among students and within the school across the three time frames? Kindly explain. 18. According to you, what are the three constraints that is impeding in deriving the full benefit of LRP? (Probe for supply issues, quality issues, interest of the neighbourhood in sending children especially girls to school, transport issue) USDA Local Regional Procurement Activity Evaluation in Lao PDR Final Report 140 19. If the WFP support stops, will you still continue with the activities under LRP (local procurement) in your school? If yes, do you have the resources to continue the program? Do you think the parents would support continuity of LRP by increasing their contribution through cash or kind? If no, how will that affect in enrolment and retention of students in school? On a long term, do you think it will increase health related issues among the children caused due to consumption of non-nutritious food? 20. Do you feel that boys and girls have equal access to education in your village? Have you come across any instances of gender discrimination when it comes to educating children? Do girls or boys have more responsibilities in the HH that would prevent them from going to school? Or any other reasons that could be different levels of opportunity for boys and girls in attending school. 21. Have you observed any change in community’s attitude in perceiving the importance of education and nutrition of their children? Thank you for your valuable time, your feedback is much appreciated. USDA Local Regional Procurement Activity Evaluation in Lao PDR Final Report 141 Key Informant Interviews (KIIS): Teachers Name of the School: Name of the Village: Name of the Teacher: Introduction 1. How long have you been teaching in this school? Are you a permanent teacher in this school? Are you an appointed teacher or a volunteered teacher? What is your highest educational qualification? 2. Do you have child/children who attend pre-primary or primary classes in this school? Do you also cook the school lunch in your school? If yes, then how many days in a school week do you cook the school lunch? Do you have to skip teaching any class for cooking the school lunch? Do you contribute vegetables/eggs/ meat/cash for school lunch? 3. How many days in a week does the school provide lunch? If less than 5 days, what is the constraint in providing lunch on a regular basis? What do the children eat when the lunch is not provided in the school? What are the measures that can be taken up to increase the provision of school lunch? 4. How, according to you has LRP benefitted in addition to School Meals Program? Do you contribute for the school lunch in any manner? How? 5. Who is responsible for cooking the school lunch? Is the person permanent or people from community take responsibility for cooking on a rotational basis? Is anything paid in cash or kind to the cook? What kind of training has the cook been provided under LRP? USDA Local Regional Procurement Activity Evaluation in Lao PDR Final Report 142 6. Is there a functional kitchen in school for cooking of meals? What types of vegetables are generally procured for school meal? Who decides the menu of the school meals (variety in the meals prepared at school)? Who is responsible for procurement of vegetables? 7. Does the school source vegetables from the local farmer groups? How does the program support the school in this? Is this supply free of cost, or are farmers paid? How are the farmers paid? On an average, how many days in a week does the school procure vegetables from local farmers and in what quantities? What type of vegetables are procured – kindly specify for all seasons? Who decides on selection of farmers for procurement of vegetables? 8. What vegetables are usually grown in the school garden? Who decides what needs to be grown? How many times in a month are you able to harvest produce from school garden that meets that particular day’s vegetable needs? What are the benefits of the school garden? Who is responsible for maintenance of the school garden? Are there any challenges in maintaining the school garden? 9. Does the school have a nutrition curriculum for students? Have there been changes in the curriculum post implementation of LRP (inclusion of vegetables grown by local farmers in the curriculum)? USDA Local Regional Procurement Activity Evaluation in Lao PDR Final Report 143 10. Does the school use school agriculture teaching material to teach the students and discuss about nutrition? Have the school gardens helped in increasing children’s and parents’ awareness and preference for vegetables. Training and Capacity Building 11. Have you received any training from WFP during last two years? If yes, on what and how beneficial was it? In what ways do you apply the trainings you received? Did you face any difficulty in application of the training? Do you think you would need particular type of training to enhance your teaching skill? Perspective and Observation 12. According to you, what are the three significant positive changes that LRP has been able to make? (Probe for increase in enrolment, attendance, children’s improved attentiveness, increase in school teacher’s interest, improved infrastructure, parent’s increase in interest in education, consumption of nutritious food in school and at home, benefit to local farmers, etc.) 13. What impact have you observed in the attendance among pre-primary and primary students in school – particularly (1) before SMP, (2) after SMP but without LRP, and (3) after LRP? What could be the potential reasons for this? Have you observed any other changes among students and within the school across the three time frames? Kindly explain. USDA Local Regional Procurement Activity Evaluation in Lao PDR Final Report 144 14. Are there families in this village who face challenges in sending their children to school? If yes, what are the challenges they face? Can you give us some examples? Do you think that the provision of school lunch significantly impacts the decision of children to come to school? 15. Do you feel that boys and girls have equal access to education in your village? Have you come across any instances of gender discrimination when it comes to educating children? Do girls or boys have more responsibilities in the HH that would prevent them from going to school? Or any other reasons that could be different levels of opportunity for boys and girls in attending school. 16. Have you observed any change in community’s attitude in perceiving the importance of education and nutrition of their children? Sustainability 17. According to you, what are the three constraints that is impeding in deriving the full benefit of LRP? (Probe for supply issues, quality issues, interest of the neighbourhood in sending children especially girls to school, transport issue) 18. What are the main issues faced in carrying out the Local regional procurement program? If the WFP support stops, will the activities under LRP (local procurement) still continue in your school? If yes, does the school have the resources to continue the program? Do you think the parents would support continuity of LRP by increasing their contribution through cash or kind? USDA Local Regional Procurement Activity Evaluation in Lao PDR Final Report 145 If no, how will that affect in enrolment and retention of students in school and farmers group in the village? Thank you for your valuable time, your feedback is much appreciated. USDA Local Regional Procurement Activity Evaluation in Lao PDR Final Report 146 Key Informant Interviews (KIIS): Cooks Name of the School: Name of the Village: Name of the Cook(s): Introduction 1. Are you the appointed cook for the school lunch or do you volunteer to cook? How long have you been appointed as the cook/volunteered as a cook? How many days in a school week do you cook the school lunch? What are your responsibilities as a cook? What motivated you to become a cook? 2. Is there a roster for the cooking responsibilities? Who prepares the roster? Have you been consulted during the preparation of the roster? In case you have problems with the roster, who do you inform? How is your problem accommodated? 3. Are you also a teacher in the school? Do you have a child who also attends pre-primary or primary classes in this school? 4. Who cooks the lunch when you are on leave or unable to come to school? Responsibilities in implementing the Program 5. How many days in a week does the school provide lunch? If less than 5 days, what is the constraint in providing lunch on a regular basis? What do the children eat when the lunch is not provided in the school? What are the measures that can be taken up to increase the provision of school lunch? USDA Local Regional Procurement Activity Evaluation in Lao PDR Final Report 147 6. Is the menu of school meal pre-decided? Who decides the menu of school lunch? What types of vegetables are served usually in school meals? Is same meal cooked every day or is it different on all days of the week? Do children like the food? 7. Does the school have a dedicated room as a kitchen? When do you clean your kitchen? How often do you clean the kitchen? Is there a separate storage space for keeping raw material (in kitchen or a separate store)? 8. Who is in-charge of the store where vegetables and other raw materials are stored? Does the store keeper lock the store room when he/she is on leave? If yes, do you get access to the store room? If no, what do you cook when the store room is locked for the lunch? Where do you get the food from to cook when the store keeper is on leave for several days? Or does the school skip the lunch when the store keeper is on leave? 9. What is the source of water that is used for cleaning and cooking food? Is the water available all the time? Do you wash the pots/utensils in which the food is cooked and stored, before and after cooking? Do you wash the vegetables and rice before cooking? How do you store cooked food prior to serving the students? 10. In the last week, how many days did you use vegetables from the school garden? Who is in charge of taking care of the school garden? What are the constraints faced in maintaining the school garden? USDA Local Regional Procurement Activity Evaluation in Lao PDR Final Report 148 11. Does the school source food from the local farmer groups? Who is responsible for procurement of vegetables from farmers? Who decides on what is to be purchased form farmers and in what frequency? What are the different types of vegetables that are purchased from the farmer groups? What is the ideal quantity that is purchased from farmer groups? Where does the transaction take place (in school or market place)? How are the farmers paid? 12. What according to you are the issues that you face in preparing the school lunch? (Probe issues like amount of food, water, space or lack of vegetables, etc.) Whom do you approach on issues regarding preparing lunch? Is VEDC helpful and approachable? Are the school authorities helpful? 13. What other support do you think is required in preparing the school lunch? Probe for support like infrastructure, capacity building, food items, lack of utensils, etc. Contribution to the school lunch program 14. Do you contribute to the local procurement apart from cooking lunch? If yes, in what manner? Training and Capacity Building 15. Have you received any training from WFP during last two years (menu, safe food preparation, storage practices, nutritional value of food, etc.)? If yes, on what and how beneficial was it? In what ways do you apply the trainings you received? Did you face any difficulty in application of the training? Do you think you would need particular type of training to improve your cooking skill? Sustainability USDA Local Regional Procurement Activity Evaluation in Lao PDR Final Report 149 16. Do you receive salary for cooking the school lunch or do you receive in kind? If yes, who pays your salary? Do you find it satisfactory? Why or why not? Does it supplement your income from this work? 17. What is it that you find to be the most challenging as a cook? Perspective and Observation 18. In your opinion how has the LRP program contributed towards the well-being of farmers, their families, nutritional intake of farmer’s families and children in school and educational status in schools (enrolment and attendance)? 19. Do you feel that boys and girls have equal access to education in your village? Have you come across any instances of gender discrimination when it comes to educating children? Do girls or boys have more responsibilities in the HH that would prevent them from going to school? Or any other reasons that could be different levels of opportunity for boys and girls in attending school. Thank you for your valuable time, your feedback is much appreciated. USDA Local Regional Procurement Activity Evaluation in Lao PDR Final Report 150 Key Informant Interviews (KIIS): Traders Name of the Village: Name of the Trader: Introduction 1. What all commodities do you trade in? Which all market(s) do you usually operate in? Farmers from which all villages usually sell their produce to you? Whom do you usually sell this produce? Do farmers usually add any value (in terms of grading/sorting or packaging) before selling? Do you add value in the produce before selling? 2. Are you aware of LRP program implemented in Nalae district? What all do you know about the program? Have you been approached by the program team – was this followed by any form of training under the program? What all have you been trained on? How many traders were trained? What is the nature of your involvement envisaged under the program? To what extent has this been realised in the last two years? 3. Do you see any increase in the quantum of business from village ______ in the last two years? What all commodities have the farmers from the village usually sell? Do you meet these farmers in the market or come to the village to buy their produce? Is there any other benefit provided to the farmers from this village? Are there any more traders who usually buy from farmers of this village? How many other LRP villages do you cover? 4. To what extent do you think the farmers sell you vegetables which are in demand in the market? Do you provide any inputs to these farmers with respect to the nature of vegetables to be grown, based on prevalent prices and demand in the market? USDA Local Regional Procurement Activity Evaluation in Lao PDR Final Report 151 5. Have there been instances where all farmers from this village sold same type(s) of vegetables to you? Have there been instances of wastages on account of low demand/high supply? What steps can the farmers take to mitigate risks of low demand/high supply? 6. Have there been instances when farmers from this village did not produce and sell vegetables that were in extremely high demand/low supply? Why weren’t the farmers from this village unable to match the demand? What factors, according to you, resulted in low production despite high demand and prices in the market? What steps can the farmers take in such situations to make the most of high demand? 7. Who, do you think, influences farmers to grow the type of vegetables they grow? Can this be changed to keep the produce in accordance with prevalent demand and prices? What role can you play in influencing farmers? What other considerations need to be taken into account by farmers in order to increase their income and reduce wastage? Thank you for your valuable time, your feedback is much appreciated. USDA Local Regional Procurement Activity Evaluation in Lao PDR Final Report 152 Key Informant Interviews (KIIS) With Provincial Agriculture and Forestry Office (PAFO) Name of the Official: Designation: Date of Joining: Date of Interview: 1. Since when are you associated with LRP program? What are the roles and responsibilities of the department under the program (indicative list: partnering and liaising for implementation of program activities, support and mobilization of local communities, organizing training program, developing training modules, monitoring of program activities, planning and implementation of project activities, procurement of agricultural tools and seeds)? What are your duties related to the program? 2. How are the program objectives aligned with the objectives of the department priorities? Did government make (or plans to make) any policy changes related to nutrition or agriculture which might be directly or indirectly linked to the LRP program? 3. How is Nalae district different from other areas of the country in terms of nutrition and agriculture status? (Status, challenges, terrain, ethnicity etc.). 4. What all departments and organizations have your partnered with for the implementation of the program activities? Please elaborate on type of partnership and responsibilities of the partnering organization. 5. Have you or somebody from your department been provided any training under the program? If yes, on what, what frequency and topic? Has the training been beneficial in any way? If, yes how? USDA Local Regional Procurement Activity Evaluation in Lao PDR Final Report 153 6. What activities were carried out under the program (trainings, CBT, exposure visits, etc.) and what type of capacity building programs were organized for the beneficiaries? What role did your department play in that? (training of farmers, cook, teacher, VEDC) 7. What is the monitoring mechanism in place for monitoring of implementation of program interventions? Was there any mechanism to take feedback from community on the program interventions? 8. According to you how is the LRP program different from SMP and how has it fared in its intended objectives (local procurement of vegetables for school meals, increased production quantity and quality by farmers, improved dietary diversity of children and families of farmers, providing market linkage to farmers, involvement of women farmers, improving school attendance and improving learning outcomes etc.)? 9. What according to you has worked best and what did not work in the operational modality of the program? What should be replicated as it is and what needs to be changed and how? Do you think certain aspects within the program design resulted in sustainability within the implementation model? What in your opinion can be the best approach and strategy to ensure sustainability of impact generated under the LRP program? How do you plan to carry forward the activities and outcomes of the LRP program (probe for program elements that will be retained – check if gender and human rights have been retained as an integral part of program design)? How do you plan to engage with farmer groups in future (both existing groups from this program and new farmer groups)? 10. What are the problems faced by the department in implementing and supporting the intervention? Was there any delay in providing extension support (If yes, why it happened)? Was the department able to provide adequate extension support (If no, what would have done better?) What challenges do you foresee in independent execution of similar programs in future? What are the necessary conditions required for scaling and replication of intervention? 11. How has your overall experience been with WFP and what are your future expectations from them? How do you plan to engage with farmer groups in future (both existing groups from this program and new farmer groups)? USDA Local Regional Procurement Activity Evaluation in Lao PDR Final Report 154 12. What other government owned or externally funded programs and activities are being carried out currently or are in pipeline that are related to nutrition or agriculture? 13. How the formation of farmers group helped the department in providing necessary technical support and required information to farmers? Do you think there could have been a better alternative to leveraging farmers group? Why or why not? 14. Were there any instances of changes made in the program design or planned activities during the course of intervention? What were the reasons for making such changes? How difficult it was for the department to adapt to the changes made? What were the implications of making such changes? 15. To what extent is the program contributing towards addressing issues pertaining to health, nutrition, and gender and human rights? What kind of solutions does the program offer to smallholder farmers towards adopting nutrition focussed agricultural practices? Thank you for your responses USDA Local Regional Procurement Activity Evaluation in Lao PDR Final Report 155 Key Informant Interviews (KIIS) with Provincial Education and Sports Services Name of the Official: Designation: Date of Joining: Date of Interview: 1. Since when are you associated with LRP program? What are the roles and responsibilities of the department under the program (indicative list: partnering and liaising for implementation of program activities, support and mobilization of local communities, organizing training program, developing training modules, monitoring of program activities, planning and implementation of project activities, procurement of agricultural tools and seeds)? What are your duties related to the program? 2. How are the program objectives aligned with the objectives of the PESS? Did government make (or plans to make) any policy changes related to nutrition or education which might be directly or indirectly linked to the LRP program? 3. How is Nalae district different from other districts in the Province in terms of nutrition and education status? (Status, challenges, etc.) 4. Have you or somebody from your department been provided any training under the program? If yes, on what, what frequency and topic? Has the training been beneficial in any way? If, yes how? 5. What is the monitoring mechanism in place for monitoring of implementation of program activities in general and the components you are responsible for? Was there any mechanism to take feedback from community on the program interventions? USDA Local Regional Procurement Activity Evaluation in Lao PDR Final Report 156 6. Did you play any role in mobilising communities for the program? If yes, what was your approach in mobilising communities? Were there incidences of conflict at village level while selection of beneficiaries? If yes, how were they resolved? 7. According to you how is the LRP program different from SMP and how has it fared in its intended objectives (local procurement of vegetables for school meals, increased production quantity and quality by farmers, improved dietary diversity of children and families of farmers, providing market linkage to farmers, involvement of women farmers, improving school attendance and learning outcomes etc.)? 8. How has the program impacted children in schools and their parents in general (the dietary intake of children in schools, their attendance and regularity, health, knowledge on nutritious food, improving school attendance and improving learning outcomes etc.)? 9. What according to you has worked best and what did not work in the operational modality of the program? What should be replicated as it is and what needs to be changed and how? Do you think certain aspects within the program design resulted in sustainability within the implementation model? What in your opinion can be the best approach and strategy to ensure sustainability of impact generated under the LRP program? How do you plan to carry forward the activities and outcomes of the LRP program (probe for program elements that will be retained – check if gender and human rights have been retained as an integral part of program design)? How do you plan to engage with farmer groups in future (both existing groups from this program and new farmer groups)? 10. What are the problems faced by the department in implementing and supporting the intervention? Was there any delay in providing extension support (If yes, why it happened)? Was the department able to provide adequate extension support (If no, what would have done better?) What challenges do you foresee in independent execution of similar programs in future? What are the necessary conditions required for scaling and replication of intervention? 11. To what extent is the program contributing towards addressing issues pertaining to health, nutrition, and gender and human rights? What kind of solutions does the program offer to smallholder farmers towards adopting nutrition focussed agricultural practices? Key Informant Interviews (KIIS) with District Agriculture and Forestry Office (DAFO) USDA Local Regional Procurement Activity Evaluation in Lao PDR Final Report 157 Name of the Official: Designation: Date of Joining: Date of Interview: 1. Since when are you associated with LRP program? What are the roles and responsibilities of the department under the program (indicative list: partnering and liaising for implementation of program activities, support and mobilization of local communities, organizing training program, developing training modules, monitoring of program activities, planning and implementation of project activities, procurement of agricultural tools and seeds)? What were your duties and responsibilities related to the program? 2. How is Nalae district different from other areas of the country in terms of nutrition and agriculture status? (Status, challenges, terrain, ethnicity etc.). How are LRP program villages different from other villages of the district – non-model, model certified and model non-certified? How are program objectives aligned with objectives of the department priorities? 3. What all departments and organizations have your partnered with for the implementation of the program activities? Please elaborate on type of partnership and responsibilities of the partnering organization. 4. Have you or somebody from your department been provided any training under the program? If yes, on what, what frequency and topic? Has the training been beneficial in any way? If, yes how? 5. What was the approach for mobilising communities, especially farmer groups to participate in the program interventions? What was the selection criteria of beneficiary farmers? Who was responsible for selecting beneficiaries? Were there incidences of conflict at village level while selection of beneficiaries? If yes, how were they resolved? Were there any specific mechanisms to reach out to socially and economically vulnerable groups? USDA Local Regional Procurement Activity Evaluation in Lao PDR Final Report 158 6. What activities were carried out under the program (trainings, CBT, exposure visits, etc.) and what type of capacity building programs were organized for the beneficiaries? What role did your department play in that? (training of farmers, cook, teacher, VEDC)? To what extent was the program design and implementation in-line with the needs and demands for capacity strengthening for women and men small landholder farmers? 7. What other types of capacity building programs are organized by your department and for whom? What is the criteria of awarding organic producer certificate to a farmer or farmer groups? To what extent has the intervention linked farmers with the local markets and the schools? 8. What is the monitoring mechanism in place for monitoring of implementation of program interventions? Was there any mechanism to take feedback from community on the program interventions? 9. According to you how is the LRP program different from SMP and how has it fared in its intended objectives (local procurement of vegetables for school meals, increased production quantity and quality by farmers, improved dietary diversity of children and families of farmers, providing market linkage to farmers, involvement of women farmers, improving school attendance and improving learning outcomes etc.)? 10. What according to you has worked best and what did not work in the operational modality of the program? What should be replicated as it is and what needs to be changed and how? Do you think certain aspects within the program design resulted in sustainability within the implementation model? What in your opinion can be the best approach and strategy to ensure sustainability of impact generated under the LRP program? How do you plan to carry forward the activities and outcomes of the LRP program (probe for program elements that will be retained – check if gender and human rights have been retained as an integral part of program design)? How do you plan to engage with farmer groups in future (both existing groups from this program and new farmer groups)? USDA Local Regional Procurement Activity Evaluation in Lao PDR Final Report 159 11. What are the problems faced by the department in implementing and supporting the intervention? Was there any delay in providing extension support (If yes, why it happened)? Was the department able to provide adequate extension support (If no, what would have done better?) What challenges do you foresee in independent execution of similar programs in future? What are the necessary conditions required for scaling and replication of intervention? 12. How has your overall experience been with WFP and what are your future expectations from them? 13. What other government owned or externally funded programs and activities are being carried out currently or are in pipeline that are related to nutrition or agriculture? 14. To what extent is the program contributing towards addressing issues pertaining to health, nutrition, and gender and human rights? What kind of solutions does the program offer to smallholder farmers towards adopting nutrition focussed agricultural practices? 15. Can you provide instances where the program readily responded to the changing external social, economic and political environment? 16. To what extent have women and men farmers been benefitted by receiving program inputs? Was there any change in income If, yes how and to what extent? 17. How the formation of farmers group helped the department in providing necessary technical support and required information to farmers? Do you think there could have been a better alternative to leveraging farmers group? Why or why not? 18. Were there any instances of changes made in the program design or planned activities during the course of intervention? What were the reasons for making such changes? How difficult it was for the department to adapt to the changes made? What were the implications of making such changes? USDA Local Regional Procurement Activity Evaluation in Lao PDR Final Report 160 19. What potential impact you were expecting before the implementation of the intervention? What are the changes you can see now? Was there any unintended effect of intervention on stakeholders? 20. How participative are the community members in the implementation of LRP? How the agricultural extension service facilitating increased community ownership and strengthening of school lunch implementation? Thank you for your responses USDA Local Regional Procurement Activity Evaluation in Lao PDR Final Report 161 Key Informant Interviews (KIIS) with District Education and Sports Bureau Name of the Official: Designation: Date of Joining: Date of Interview: 1. Since when are you associated with LRP program? What are the roles and responsibilities of the department under the program (indicative list: partnering and liaising for implementation of program activities, support and mobilization of local communities, organizing training program, developing training modules, monitoring of program activities, planning and implementation of project activities, procurement of agricultural tools and seeds)? What are your duties related to the program? How are the program objectives aligned with the objectives of the DESS? 2. How is Nalae district different from other districts in the Province in terms of nutrition and education status? (Status, challenges, etc.) 3. What type of capacity building programs were organized for the beneficiaries under the program and what role did you play in that? (training of cook, teacher, farmer, VEDC) 4. Have you or somebody from your department been provided any training under the program? If yes, on what, what frequency and topic? Has the training been beneficial in any way? If, yes how? 5. What is the monitoring mechanism in place for monitoring of implementation of program activities in general and the components you are responsible for? Was there any mechanism to take feedback from community on the program interventions? 6. Did you play any role in mobilising communities for the program? If yes, what was your approach in mobilising communities? How were beneficiaries (farmers) selected and who was responsible USDA Local Regional Procurement Activity Evaluation in Lao PDR Final Report 162 for selection? Were there incidences of conflict at village level while selection of beneficiaries? If yes, how were they resolved? 7. According to you how is the LRP program different from SMP and how has it fared in its intended objectives (local procurement of vegetables for school meals, increased production quantity and quality by farmers, improved dietary diversity of children and families of farmers, providing market linkage to farmers, involvement of women farmers, improving school attendance and improving learning outcomes etc.)? 8. How has the program impacted children in schools and their parents in general (the dietary intake of children in schools, their attendance and regularity, health, knowledge on nutritious food, improving school attendance and learning outcomes etc.)? 9. What according to you has worked best and what did not work in the operational modality of the program? What should be replicated as it is and what needs to be changed and how? Do you think certain aspects within the program design resulted in sustainability within the implementation model? What in your opinion can be the best approach and strategy to ensure sustainability of impact generated under the LRP program? How do you plan to carry forward the activities and outcomes of the LRP program (probe for program elements that will be retained – check if gender and human rights have been retained as an integral part of program design)? How do you plan to engage with farmer groups in future (both existing groups from this program and new farmer groups)? 10. What are the problems faced by the department in implementing and supporting the intervention? Was there any delay in providing extension support (If yes, why it happened)? Was the department able to provide adequate extension support (If no, what would have done better?) What challenges do you foresee in independent execution of similar programs in future? What are the necessary conditions required for scaling and replication of intervention? 11. What other government owned or externally funded programs are being carried out currently or are in pipeline that are related to nutrition or education? USDA Local Regional Procurement Activity Evaluation in Lao PDR Final Report 163 12. To what extent is the program contributing towards addressing issues pertaining to health, nutrition, and gender and human rights? What kind of solutions does the program offer to smallholder farmers towards adopting nutrition focussed agricultural practices? 13. Can you provide instances where the program readily responded to the changing external social, economic and political environment? Thank you for your responses USDA Local Regional Procurement Activity Evaluation in Lao PDR Final Report 164 Key Informant Interviews (KIIS) with Department of Technical Extension and Agro￾Processing (previously known as DAEC) Name of the Official: Designation: Date of Joining: Date of Interview: 1. Since when are you associated with LRP program? What are the roles and responsibilities of the department under the program (indicative list: partnering and liaising for implementation of program activities, support and mobilization of local communities, organizing training program, developing training modules, monitoring of program activities, planning and implementation of project activities, procurement of agricultural tools and seeds)? What are your duties related to the program? 2. How are the program objectives aligned with national interest and priorities? Did government make (or plans to make) any policy changes related to nutrition or agriculture which might be directly or indirectly linked to the LRP program? 3. How is Luang Namtha province and Nalae district in particular different from other areas of the country in terms of nutrition and agriculture status? (Status, challenges, terrain, ethnicity etc.) 4. What is the monitoring mechanism in place for monitoring of implementation of program interventions? Was there any mechanism to take feedback from community on the program interventions? 5. What all departments and organizations have your partnered with for the implementation of the program activities? Please elaborate on type of partnership and responsibilities of the partnering organization. USDA Local Regional Procurement Activity Evaluation in Lao PDR Final Report 165 6. According to you how is the LRP program different from SMP and how has it fared in its intended objectives (local procurement of vegetables for school meals, increased production quantity and quality by farmers, improved dietary diversity of children and families of farmers, providing market linkage to farmers, involvement of women farmers, improving school attendance and improving learning outcomes etc.)? 7. What according to you has worked best and what did not work in the operational modality of the program? What should be replicated as it is and what needs to be changed and how? Do you think certain aspects within the program design resulted in sustainability within the implementation model? What in your opinion can be the best approach and strategy to ensure sustainability of impact generated under the LRP program? How do you plan to carry forward the activities and outcomes of the LRP program (probe for program elements that will be retained – check if gender and human rights have been retained as an integral part of program design)? How do you plan to engage with farmer groups in future (both existing groups from this program and new farmer groups)? 8. What is your approach and strategy for sustaining the impact of LRP program? How do you plan to carry forward the activities of the LRP program? 9. What are the problems faced by the department in implementing and supporting the intervention? Was there any delay in providing extension support (If yes, why it happened)? Was the department able to provide adequate extension support (If no, what would have done better?) What challenges do you foresee in independent execution of similar programs in future? What are the necessary conditions required for scaling and replication of intervention? 10. How has your overall experience been with WFP and what are your future expectations from them? USDA Local Regional Procurement Activity Evaluation in Lao PDR Final Report 166 11. What other government owned or externally funded programs are being carried out currently or are in pipeline that are related to nutrition or agriculture? 12. To what extent is the program aligned with the national agricultural, nutritional and educational priorities? 13. How the formation of farmers group helped the department in providing necessary technical support and required information to farmers? Do you think there could have been a better alternative to leveraging farmers group? Why or why not? 14. Were there any instances of changes made in the program design or planned activities during the course of intervention? What were the reasons for making such changes? How difficult it was for the department to adapt to the changes made? What were the implications of making such changes? 15. To what extent is the program contributing towards addressing issues pertaining to health, nutrition, and gender and human rights? What kind of solutions does the program offer to smallholder farmers towards adopting nutrition focussed agricultural practices? Thank you for your responses USDA Local Regional Procurement Activity Evaluation in Lao PDR Final Report 167 Key Informant Interviews (KIIS) with Ministry of Education and Sports Services Name of the Official: Designation: Date of Joining: Date of Interview: 1. Since when are you associated with LRP program? What are the roles and responsibilities of the department under the program (indicative list: partnering and liaising for implementation of program activities, support and mobilization of local communities, organizing training program, developing training modules, monitoring of program activities, planning and implementation of project activities, procurement of agricultural tools and seeds)? What are your duties related to the program? 2. How are the program objectives aligned with the objectives of the MoES and national government? Did government make (or plans to make) any policy changes related to nutrition or education which might be directly or indirectly linked to the LRP program? 3. How is Luang Namtha province and Nalae district in particular different from other areas of the country in terms of nutrition and education status? (Status, challenges, etc.) 4. What is the monitoring mechanism in place for monitoring of implementation of program activities in general and the components you are responsible for? Was there any mechanism to take feedback from community on the program interventions? 5. How has the program impacted children in schools and their parents in general (the dietary intake of children in schools, their attendance and regularity, health, knowledge on nutritious food, improving school attendance and improving learning outcomes etc.)? USDA Local Regional Procurement Activity Evaluation in Lao PDR Final Report 168 6. What according to you has worked best and what did not work in the operational modality of the program? What should be replicated as it is and what needs to be changed and how? Do you think certain aspects within the program design resulted in sustainability within the implementation model? What in your opinion can be the best approach and strategy to ensure sustainability of impact generated under the LRP program? How do you plan to carry forward the activities and outcomes of the LRP program (probe for program elements that will be retained – check if gender and human rights have been retained as an integral part of program design)? How do you plan to engage with farmer groups in future (both existing groups from this program and new farmer groups)? 7. What in your opinion can be the best approach and strategy of sustaining the impact created under the LRP? How do you plan to carry forward the activities of the LRP program? 8. What are the problems faced by the department in implementing and supporting the intervention? Was there any delay in providing extension support (If yes, why it happened)? Was the department able to provide adequate extension support (If no, what would have done better?) What challenges do you foresee in independent execution of similar programs in future? What are the necessary conditions required for scaling and replication of intervention? 9. How has your overall experience been with WFP and what are your future expectations from them? 10. What other government owned or externally funded programs are being carried out currently or are in pipeline that are related to nutrition or education? 11. To what extent is the program aligned with the national agricultural nutritional and educational priorities? 12. To what extent is the program contributing towards addressing issues pertaining to health, nutrition, and gender and human rights? What kind of solutions does the program offer to smallholder farmers towards adopting nutrition focussed agricultural practices? Thank you for your responses USDA Local Regional Procurement Activity Evaluation in Lao PDR Final Report 169 Key Informant Interviews (KIIs) with Ngo Partners, Lao PDR Name of the Official: Designation: Date of Joining: Date of Interview: 1. Since when are you associated with LRP program? What are your roles and responsibilities? Were you also part of the designing process of the program? Who are other partners in implementation of the program (government and non-government)? What are the roles and responsibilities of each of the partner organizations (please provide details for each partner)? QA. To what extent is the program contributing towards addressing issues pertaining to education, health & nutrition and gender? QB. To what extent is the program design sensitive to the changing external social, economic and political environment? QC. To what extent was the program design and implementation in-line with the needs and demands for capacity strengthening for women and men small farmers? QE. To what extent did the program design provide solutions to small holder farmers towards adopting nutrition focussed agricultural practices? 2. How is LRP linked to USDA-MGD SMP and how is it contributing to the objectives of SMP? Have there been any changes in the program interventions or modalities than initially planned? What were the changes and reasons for bringing those changes? USDA Local Regional Procurement Activity Evaluation in Lao PDR Final Report 170 QD. To what extent has the design and implementation of the intervention addressed existing gaps in linking to the local markets and school meal program and ensuing issues and challenges? QJ. To what extent has the channel leveraging farmers groups been successful in making agriculture technical support available to farmers, keeping into consideration the inputs involved? QN. Understand the frequency and nature of course corrections carried out in the program design and implementation? QO. Understand (1) the reasons for carrying out such course corrections, (2) challenges faced in implementing them and (3) their implications. 3. What were the challenges and issues associated with the implementation of the program? How were they overcome? Have there been any delays in the implementation of planned activities (phase wise) and what have been the reasons of delay? (Please provide phase wise details)? QG. To what extent was the support provided by DTEAP, PAFO, DAFO with the help of WFP for agricultural extension was adequate and sufficient in solving problems in implementation of the intervention? QH. To what extent was the support provided by DTEAM, PAFO, DAFO was timely in nature? QI. Nature of gaps identified or problems faced by the DTEAP, PAFO, DAFO while supporting the implementation? USDA Local Regional Procurement Activity Evaluation in Lao PDR Final Report 171 4. The program started with 49 villages then why were two villages dropped? Was support to farmer groups continued in the dropped villages? What was the criteria for selection of 10 model villages? What extra interventions were undertaken with these 10 model villages? (Location of model villages) What is constraining the remaining 37 villages in becoming the model villages? 5. How were the beneficiaries of program identified and decided? Were there any incidences of conflict at any level (between partners, between other stakeholders and implementing partners, between beneficiaries, between beneficiaries and other community members, etc.)? If yes, how were they resolved? QF. To what extent has the program design and implementation ensured reaching out to socially and economically vulnerable groups (access to infra and technology, addressing their key issues and concerns and capacity strengthening)? 6. Who developed the training modules for training provided to different stakeholders on agriculture development and nutrition? QK. To what extent were other alternative efficient platform (apart from farmer groups) for the farmers to receive agriculture technical support available? To what extent would such alternatives have been efficient? QL. To what extent has the channel leveraging DTEAP, DAFO and PAFO been successful in delivering knowledge transfers to the small holder farmers, keeping into consideration the inputs involved? USDA Local Regional Procurement Activity Evaluation in Lao PDR Final Report 172 QM. To what extent were other alternative efficient platform (apart from DTEAP, PAFO, DAFO) \ for delivery of knowledge transfers to small farmers available? To what extent would such alternatives have been efficient? 7. What is the monitoring system being followed currently for the program activities? What is the frequency of monitoring and who is responsible for it (at all levels, frequency)? 8. Were trainings on group management and marketing conducted (semi-annual monitoring reports mention of these planned trainings)? When? By whom? Could its targeted impact on production quantities be achieved (20kg sale per month in 1st year and 50 kg sale in 2nd year)? Who were the beneficiaries of the training? How were they identified? 9. How the average income of farmers and combined sales calculated as were stated in the semi￾annual monitoring reports? 10. Were the trainings provided by DAFO on animal husbandry part of LRP? How were the beneficiaries identified? 11. When was livestock related support included in the program? How were the beneficiaries for distribution of different livestock identified? Which component of budget was used for this activity as it was not a planned activity at the time of the inception of the program? 12. How were the beneficiaries of exchange visits of farmer groups identified? How many groups from how many villages have been sent on exposure visits? What were the reasons for less number of women being part of those visits while the program has nearly 50 per cent of farmer beneficiaries? USDA Local Regional Procurement Activity Evaluation in Lao PDR Final Report 173 13. Was Agriculture Technical Service Center established in Nalae? What was its purpose? What is production capacity of this center and how were the beneficiaries for distribution of mushroom seeds identified? What was the reason for focussing on plantation of mushrooms? Do people follow 'shift cultivation' or 'slash and burn' cultivation? 14. What was the purpose of pellets (semi-annual reports mention about development of pallets) and why were they needed? 15. According to you, did the program meet the intended and expected objectives (what and what not)? How can the program be replicated in other areas (what worked, what didn’t work) QG. To what extent were the outputs and outcomes of the intervention achieved and what were the major factors influencing the achievement or non-achievement? Thank you for your time! USDA Local Regional Procurement Activity Evaluation in Lao PDR Final Report 174 Key Informant Interviews (KIIS) with WFP Officials, Lao PDR Name of the Official: Designation: Date of Joining: Date of Interview: 1. Since when are you associated with LRP program? What are your roles and responsibilities? Who are the partners in implementation of the program (government and non-government)? What are the roles and responsibilities of each of the partner organizations (please provide details for each partner)? Was any nutritionist involved in the program? QA. To what extent is the program aligned with the National agricultural, nutritional and educational priorities (national nutritional strategy 2016-20, agriculture development strategy 2020) and WFP's country strategy 2017-21? QB. To what extent is the program contributing towards addressing issues pertaining to education, health & nutrition and gender? QC. To what extent is the program design sensitive to the changing external social, economic and political environment? QD. To what extent was the program design and implementation in-line with the needs and demands for capacity strengthening for women and men small farmers? QF. To what extent did the program design provide solutions to small holder farmers towards adopting nutrition focussed agricultural practices? 2. When and how the LRP program was conceived (what triggered the inception of the idea)? How is it linked to USDA-MGD SMP and how is it contributing to the objectives of SMP? Please USDA Local Regional Procurement Activity Evaluation in Lao PDR Final Report 175 explain in detail different phases, components, activities in each phase, and stakeholders involved in the LRP program. Have there been any changes in the program interventions or modalities than initially planned? What were the changes and reasons for bringing those changes? QE. To what extent has the design and implementation of the intervention addressed existing gaps in linking to the local markets and school meal program and ensuing issues and challenges? QQ. To what extent has the channel leveraging farmers groups been successful in making agriculture technical support available to farmers, keeping into consideration the inputs involved? QU. Understand the frequency and nature of course corrections carried out in the program design and implementation? QV. Understand (1) the reasons for carrying out such course corrections, (2) challenges faced in implementing them and (3) their implications. QAB. To what extent has gender and human rights of vulnerable groups been incorporated as an integral aspect of the program design and implementation model? 3. What were the challenges and issues associated with the implementation of the program? How were they overcome? Have there been any delays in the implementation of planned activities (phase wise) and what have been the reasons of delay? (Please provide phase wise details)? QN. To what extent was the support provided by DTEAP, PAFO, DAFO with the help of WFP for agricultural extension was adequate and sufficient in solving problems in implementation of the intervention? QO. To what extent was the support provided by DTEAM, PAFO, DAFO was timely in nature? QP. Nature of gaps identified or problems faced by the DTEAP, PAFO, DAFO while supporting the implementation? USDA Local Regional Procurement Activity Evaluation in Lao PDR Final Report 176 QAC. Record key challenges that emerged during the course of program implementation, learnings and best practices. 4. The program started with 49 villages then why were two villages dropped? Was support to farmer groups continued in the dropped villages? What was the criteria for selection of 10 model villages? What extra interventions were undertaken with these 10 model villages? (Location of model villages) What is constraining the remaining 37 villages in becoming the model villages? 5. How were the beneficiaries of program identified and decided? Were there any incidences of conflict at any level (between partners, between other stakeholders and implementing partners, between beneficiaries, between beneficiaries and other community members, etc.)? If yes, how were they resolved? QG. To what extent was the intervention based on a sound gender analysis? QH. To what extent has the program design and implementation ensured reaching out to socially and economically vulnerable groups (access to infra and technology, addressing their key issues and concerns and capacity strengthening)? 6. Who developed the training modules for training provided to different stakeholders on agriculture development and nutrition? Were the modules developed by Inclusive Education Center used for any of the trainings? QR. To what extent were other alternative efficient platform (apart from farmer groups) for the farmers to receive agriculture technical support available? To what extent would such alternatives have been efficient? QS. To what extent has the channel leveraging DTEAP, DAFO and PAFO been successful in delivering knowledge transfers to the small holder farmers, keeping into consideration the inputs involved? USDA Local Regional Procurement Activity Evaluation in Lao PDR Final Report 177 QT. To what extent were other alternative efficient platform (apart from DTEAP, PAFO, DAFO) \ for delivery of knowledge transfers to small farmers available? To what extent would such alternatives have been efficient? 7. What is the monitoring system being followed currently for the program activities? What is the frequency of monitoring and who is responsible for it? 8. Were trainings on group management and marketing conducted (semi-annual monitoring reports mention of these planned trainings)? When? By whom? Could its targeted impact on production quantities be achieved (20kg sale per month in 1st year and 50 kg sale in 2nd year)? Who were the beneficiaries of the training? How were they identified? 9. How the average income of farmers and combined sales calculated as were stated in the semi￾annual monitoring reports? 10. Were the trainings provided by DAFO on animal husbandry part of LRP? How were the beneficiaries identified? 11. When was livestock related support included in the program? How were the beneficiaries for distribution of different livestock identified? Which component of budget was used for this activity as it was not a planned activity at the time of the inception of the program? 12. How were the beneficiaries of exchange visits of farmer groups identified? How many groups from how many villages have been sent on exposure visits? What were the reasons for less number of women being part of those visits while the program has nearly 50 per cent of farmer beneficiaries? 13. Was Agriculture Technical Service Center established in Nalae? What was its purpose? Which component of budget provided for the cost incurred? What is production capacity of this center and how were the beneficiaries for distribution of mushroom seeds identified? What was the reason for focussing on plantation of mushrooms? Do people follow 'shift cultivation' or 'slash and burn' cultivation? USDA Local Regional Procurement Activity Evaluation in Lao PDR Final Report 178 14. What was the purpose of pellets (semi-annual reports mention about development of pallets) and why were they needed? 15. Does the program account for drop-outs in case of Cash based transfers (CBTs) made to schools every semester? 16. What do you mean by value chain study (Is it the supply chain being referred as 'value chain')? Report of value chain study is requested. How was it important for the program? Did you adopt any of the recommendations made in value chain study? If yes, which ones? 17. According to you, did the program meet the intended and expected objectives (what and what not)? What is your future plan of action in the context of LRP (handing over, replication to other areas, etc.)? QI. To what extent were the outputs and outcomes of the intervention achieved and what were the major factors influencing the achievement or non-achievement? QK. To what extent have the farmers been able to supply diverse, culturally acceptable and nutritious food to schools for daily meals on a continuous basis during the program period? QL. To what extent have the targeted and most vulnerable women and men farmers been benefitted by way of receipt of program inputs? QM. To what extent have the targeted and most vulnerable women and men farmers who participated in the program been benefitted by way of improvement in quality of life? QW. What have been the intended and unintended impact of LRP on beneficiaries direct and indirect? QX. How has the program impacted sensitization, participation, decision making and reducing discrimination among different social & ethnic groups and individuals (gender equality)? USDA Local Regional Procurement Activity Evaluation in Lao PDR Final Report 179 QJ. To what extent have the school children (boys and girls) been able to receive and consume fresh and nutritious food as part of school meals on a continuous basis, using vegetables from farmer groups? QZ. What are the ecosystem factors necessary for the program to be scaled up or replicated for similar programs? QY. How did the program design integrate aspects of sustainability within its implementation model towards making the program self-sufficient with its benefit continuing even after the work of WFP has ceased? (Capacity strengthening of stakeholders, development of disseminable knowledge based content, formation of social groups, partnerships, institutional acceptance and adoption of program elements) QAA. What are some of the elements of the program that have been absorbed, adopted or integrated within institutional processes? Thank you for your time! USDA Local Regional Procurement Activity Evaluation in Lao PDR Final Report 180 Annex T Map for Province Oudumxay USDA Local Regional Procurement Activity Evaluation in Lao PDR Final Report 181 Annex U Case Studies Case Study 1: Aome Village, Nalae District, Luang Namtha Province Context: Aome village is located near the Nalae city, having only 10 minutes of walking distance to the main district market, where all farmers could sell their products. The village is also located near the Namtha river. More than 80% of the village is mountainous with a small area available for paddy cultivation. In the major part of the village, farmers used to practice shifting cultivation (upland rice), which has now been substantially reduced, with a number of households gravitating towards cultivating rubber and cardamom. As a result, a major portion of the income comes from cultivation and sale of cash crops. In addition to this, a large number of people in this village also work in Laos PDR government offices. - Mrs Nali Sengmaly (key respondent) has a total of 5 members in her family, including 3 children. Her husband works in a school. Two of her children study in the same school, whereas the third child has become a monk at a temple in Aome. The family stays in a 2-story house which is half-concrete and half-timbered. - She has a weaving instrument, which she uses for making traditional Lao skirts. She later sells these skirts to the traders in the market. - Majority land around the village was earlier being used for upland paddy. Under the GIZ project, this land was cleared and provided 10-10 square meters of it to ten families interested in cultivation for vegetables. The program enabled the household to create vegetable gardens. GIZ project worked with certain households, helping them grow and sell vegetables, apart from providing some support in maintaining livestock. After the GIZ project, the same set of farmers, including Mrs. Nali, joined the LRP program and received support, in terms of tools, seed and training. Problem before LRP As mentioned above, Mrs. Nali received good support from the GIZ project in terms of clearing the land from paddy field and helping households for practice vegetable garden. While the project results, as-far-as implementation is concerned were good, the quality of the produce wasn’t up to the mark. The program beneficiaries did not receive any form of technical training in terms of improving the quality of vegetable. As a result, it was difficult for her to farm with limited technical know-how, produce and sell low-quality vegetables, and sustain three children. The GIZ project did not provide any training on making compost and bio-fertilizers to produce organic green vegetables. The farmers were used to purchasing chemical-fertilizer from the market near villages, adding to their expenses. Without much focus on the nutritional value of vegetables, farmers earlier used to only limit themselves to the traditional vegetables, and hence used to find it difficult to generate demand for their produce. After LRP: After the announcement from the head of the village about the LRP program, Mrs. Nali expressed her interest in joining it. As per the criteria set for joining the LRP project, it provided first priority to farmers who had already worked with GIZ and had land as-well-as prior experience of growing vegetables. Other farmers, on the other hand, needed an additional budget to clear paddy land, which was not provided under the LRP. In terms of support, LRP only provided tools and seeds to the selected farmers. Some of the major changes that happened as a result of LRP include: • Farmers now have the technical know-how to grow vegetables, create compost and half￾process these vegetables to preserve them for a long period of time. USDA Local Regional Procurement Activity Evaluation in Lao PDR Final Report 182 • The program created farmer groups, and supported them with basic tools for farming and seedlings, in addition to technical training on different aspects of agriculture such as: making compost and bio anti-insect, bio-fertilizer with local resources which could be adapted to use for vegetables. As a result of this training, all farmers within the group have started not only each other but also other farmers within the village. All program farmers are technically equipped to support each other, in case of any agriculture-related problem. • All the training they attended were found to be useful. They have been able to witness the change in the productivity and quality of vegetables after implementing the knowledge provided to them during these training. Mrs. Nali reported that all farmers have observed in improvement as-far-as quantity (growing vegetables round the year) and quality of vegetables is concerned, resulting in more income. During monsoons, the area witnesses heavy rainfall and flooding. As a result, farmers do grow corn instead of vegetables for those three months. • From her perspective, she is happy that LRP was implemented in their village. As a result of participation in the program, they could improve both quantity as-well-as quality of vegetables. This helps her to get a better price for vegetables in the market. She is also extremely happy that LRP has been able to link them with the school for the lunch meals. All farmers provide support for the school meals, by way of contributing vegetables. She is content that all program farmers are now able to provide good nutrition vegetables round the year to her family as-well-as school children. • Mrs. Nali also received fund under the program to purchase weaving equipment, which she now uses for making Lao skirts. Currently, she is able to sell about 3-4 pieces per month, earning around 1 million kips per month. She earns, on an average 300,000 kip/month by selling vegetables in the markers. Apart from this, the family also collects and sells rubber latex for approximately 20,000,000 kips per year. How does the future look like for her: • Mrs. Nali would like to get support from LRP for the greenhouse (plastic roof cover only). She has seen the same greenhouse material provided by the program to farmers in other villages. Greenhouse helps the farmers grow vegetables during the rainy as-well-as dry season. • Farmers in the village face major issue with respect to a continuous supply of water for farming. Mrs. Nali would like LRP to resolve the issue of water supply by way of building a water tank and providing piped water supply and pump to individual farmers. USDA Local Regional Procurement Activity Evaluation in Lao PDR Final Report 183 Case Study 2: Khonchanh Village, Nalae District, Luang Namtha Province Context: Mrs Bouaphanh stays with her family, comprising of five people, including three children. Her house is made up of wood, with a total size of approximately 45 square meters. She has installed within her house a weaving instrument for making traditional Lao skirts, which she sells in the market. Mrs Bouaphanh used to live in Phongsaly Province, which is primarily a mountainous area, with hard living conditions. She used to cultivate upland rice under shifting cultivation, which included intense physical effort. She used to walk up and down in the uphill area, carrying tools and material for agriculture. During harvests, she used to carry rice from upland shifting area down to villages. However, after her marriage in 2009, she, along with her husband, decided to move to Nalae district in Luang Namtha. The couple, after moving to Nalae, began staying in a rented room. Initially, she started planting some vegetables, whereas her husband joined a construction contractor. After saving some money for a year, the couple bought a small piece of land near the village school, which they used for creating a vegetable garden. Apart from self-consumption, the couple started selling the excess vegetable in the village and local market. This was their first step forward towards improving their livelihood. Within a short span of time, they bought some more land to construct their own house. After a while, they bought another plot of land for rubber plantation and some cardamom. Problem and before LRP: After constructing a house, the couple bought some more land for cultivation near the river, which was approximately 7-8 km minutes away. While the couple was ready to experiment and try new livelihood options, they lacked knowledge of farming and had limited or no access to tools. Apart from the lack of technical knowledge, Mrs Bouaphanh also didn’t know how to link her vegetables to market. During that time, the couple had to prepare meals for their children early in the morning before leaving for work. They had to leave their work mid-way to come back and prepare lunch for the children, resulting in loss of time and money. Sometimes, when they needed to work extra hours in the field, they would leave their children at home, missing school. Farming with limited technical skills resulted in low productivity as-well-as low quality, which affected their selling price. After LRP: As-soon as the head of the village announced about the LRP program, Mrs. Bouaphanh decided to join it. • She became a part of the farmers' group, who got support from LRP for basic tools of farming, seedlings, along with consistent training around technical issues of farming, including methods of preparing compost and bio anti-insect with local resources. All the training that she attended was found to be useful. • Using the program fund, she bought weaving equipment for preparing Lao skirts, which are now sold for about 70,000 Lao Kip - 80,000 Lao Kip per piece. Mrs. Bouaphanh is able to sell 3-4 pieces of such skirts on a monthly basis, earning 300,000-400,000 Lao Kips. LRP provided a loan of a million Kips for each family which was interested in setting up the weaving unit. The program selected a total of 16 such families in the first year. The loan will be charged with an interest of 5%, amounting to 50,000 Lao Kips. Out of the interest amount, 5,000 Lao Kips are contributed towards the village weaving community, whereas the remaining 45,000 Lao Kip is reserved for school meals. • Mrs. Bouaphanh now also has 2 hectares of land where she grows cardamom. Last year, she earned a total of 8,000,000 Kip by selling the cash crop, USDA Local Regional Procurement Activity Evaluation in Lao PDR Final Report 184 • She also has 500 trees of rubber. The couple also sell rubber to the trader, adding to the family income by approximately 3,200,000 Lao Kip. She has been planning to increase her rubber plantation area in the coming year, adding another 500 rubber trees. • After LRP, her children now have nutritious school lunches on a daily basis, which means that she has more time to work. This has resulted in increased income, particularly through the vegetable garden and the waving activity. How does the future look for her: She would like to get support from LRP for the greenhouse (plastic roof cover only). She has seen the same greenhouse material provided by the program to farmers in other villages. Greenhouse helps the farmers grow vegetables during the rainy as-well-as dry season. Mrs Bouaphanh sees herself as a successful person. She is extremely happy with the LRP project as it has not only helped her family earn more money, it has also shown her the path to try out different livelihood options without fearing about failures. USDA Local Regional Procurement Activity Evaluation in Lao PDR Final Report 185 Case Study 3: HomChalaurn Village, Nalae District, Luang Namtha Province Context: Mr. Somnuek stays with his family with ten people, out of which five are women and five, men. He has a total of three children, two sons and one daughter, staying the same house. He moved from Mokjok village, which is 15 kms away from village HomChalaurn. The village was in the mountainous region, devoid of any proper access to roads, electricity, health centre, school and other infrastructure. Thereafter, the then Lao government made a policy to ”merge small villages together” wherein smaller villages were moved to locations where the government could provide and support with the necessary infrastructure. Mr. Somnuek moved to village HomChalaurn in 2005 when there were only 25 households, they belong to the Khmue tribe. After 2005, he found that village HomChalaurn had better livelihood opportunities than their old village. The new village is well connected by the road network, and people have access to a market and basic infrastructure including road, electric, telephone, machine. More importantly, the new village has a healthcare centre and school for children. Majority of the households in the new village are engaged in growing upland rice. Some of the households have forayed into construction work, whereas another group has taken up cultivation of fresh vegetable for consumption. Majority of the farmers mix upland rice with vegetable gardens, where they grow fresh vegetables for own consumption. After LRP: The LRP programs come to villages and he was interesting to join with the LRP project. After implementation of vegetable garden and in the first year they can support school and also consumptions in his old family, but they are some problem because 10 farmers they got the same type vegetable and they grow same time then when harvest time every farmer need to sell to the school, and second-year not many farmers grow much because the seed cannot grow after they open seed can in the first year. So, they just grow some vegetable only and add some traditional one that they can preserve the seed every year. • Mr. Somnuek and his family are not much successful from creating the vegetable garner for supporting school meals. In his owned land near the river, he cannot practice agriculture during monsoon season. Due to limited land and a lean season, he can only support the school meals for a particular time period and not throughout the year. • He got support under the program for basic farming tools, seedlings and technical know-how. He learnt techniques for tilling his land, growing different varieties of vegetables, methods of irrigation, ways of creating compost and bio anti-insect using local resources. • He attended all the training provided under the program, which he feels have resulted in a massive improvement in terms of quantity and quality of vegetable produce. However, given that he only has a small patch of land, his income from the vegetable garden is not beyond 10,000- 20,000 Lao kip per month. His main income comes from cultivating cardamom. How does the future look for him: • His first priority would be to increase the land size of his vegetable garden. It is only at a scale that he would be able to contribute more to the school means and earn a substantial income from selling vegetables. As a result, he seeks support from the program in terms of buying an additional piece of land. • He needs support from LRP in terms of establishing linkages with the market for selling his produce. At this moment, it does not make it feasible for him to transport a small volume of vegetables to the market for sale. Mass procurement of vegetables through traders would help farmers like him to cut down on their logistics-related costs, and increase income. USDA Local Regional Procurement Activity Evaluation in Lao PDR Final Report 186 • Cultivation and sale of cash crops form a major chunk of earnings for most households in the village. As a result, Mr. Somnuek intends to initiate Rubber plantation on a small piece of the plot. It is expected that earnings from rubber would help him sustain his family, whereas other livelihoods, such as vegetable garden and cardamom cultivation enable him to save some money for the future. USDA Local Regional Procurement Activity Evaluation in Lao PDR Final Report 187 Case Study 4: Pha Hou Village, Nalae District, Luang Namtha Province Context: Mr. Sunset Khonsamay, is 66 years old, lives in village Pha Hou, with a family of seven, comprising of three women and four men. His house is made from timber wood, which has a weaving instrument underneath it, used for making traditional Lao skirt. He also has a small rice mill machine which is often used by other villagers as-well, in lieu of rice bran for pigs and chicken. Mr. Sunset remained the head of his village for 28 years and was instrumental in convincing his community to relocate to the new village. While the earlier village was surrounded by forests from three directions, the new village has better road access, electricity, health centre and school. Most villagers continue practising upland rice as their main occupation. Some of the villagers have also started mixing vegetable gardening, along with upland rice. Other major livelihood related activities for the villagers include livestock rearing. Including cow, pig and goat. Before LRP: The main issue of the village is the terrain with the majority of the land being mountainous in nature, leaving the only option of upland rice cultivation. Some of the villagers have initiated cultivating cash crops like rubber and cardamom in this area. A few households within the village continue to depend upon the forests for collecting non-timber forest produce (NTFPs), which has good demand in China. Traders usually come to the village to collect NTFPs from villagers. After LRP: When the LRP program was announced in the village, a number of farmers showcased their interest in joining it. After the implementation of the vegetable gardens in the first year, the farmers supported the school meals. However, one of the major issues faced by farmers was that all of them grew the same type of vegetables, around the same time, resulting in an oversupply of these vegetables. As a result, these farmers faced difficulties in selling the surplus in markets, resulting in no major increase in their income. As a result, in the second year, a number of farmers reduced the volume of vegetables cultivated, in order to reduce wastages. All the farmers, including Sunset Khonsamay got support in terms of basic farming tools, seedlings and technical training during the last one year. The farmers were trained on a number of technical aspects around farming, including regeneration of seeds, creation of compost and bio anti-insect using local resources. Another issue faced by this village included that the majority of the farmers continued spending majority of their time cultivating upland rice. During harvest season, farmers need to stay in the fields, resulting in a lack of time to take care of vegetables. Given that cultivating vegetables did not result in any significant increase in income, farmers have moved back to spending more than ten months a year cultivating rice. Farmers in this village now only spend two-three months during monsoon season to grow vegetables. The farmers continue to contribute vegetables for school meals, majorly from their vegetable gardens, as-well-as from upland forests, which has resulted in nutritious school meals. However, farmers do not see much hope of increasing their income from vegetable gardens. USDA Local Regional Procurement Activity Evaluation in Lao PDR Final Report 188 Case Study 5: NongKham Village, Nalae District, Luang Namtha Province Context: Mrs. Bounphet Munbuapha, along with a family of five people, stays in Nongkham village. Her house is made from wood, which was built by a dam construction company after all the villagers were relocated to this village. Her house has a weaving instrument for making traditional Lao skirt, which she sells in the market. Villagers from this village used to stay near Namtha river and were affected by the dam construction. As a result, the dam construction company constructed new houses for all the villagers at the new location. Most people in this village carry out upland rice and collect NTFPs collected from nearby villages. In the last six-eight years, villagers have initiated new cash crops like rubber, cardamom and Galanga, which have significantly increased their income. After LRP: • She became a part of the farmers' group, who got support from LRP for basic tools of farming, seedlings, along with consistent training around technical issues of farming, including methods of preparing compost and bio anti-insect with local resources. All training that she attended was found to be useful. She has started contributing vegetables for school meals and selling the surplus in the market. • After a year of implementation, her family has been able to extend the size of the garden and subsequently moved to a new area near the river. • She also spends a considerable amount of time in weaving Laos skirts, earning more than 400,000 Lao Kip/ month. She has connected with traders who visit her house to procure skirts. • Because of the increase in income, her family has stopped upland cultivation of rice for the past two years. They see more benefit in growing vegetables under LRP and selling skirts to traders. With the increased income, they can easily purchase sticky rice from the market, instead of cultivating in the upland regions. • She also plants the Galanga as the fruit is in huge demand from Chinses businessman. She also has three other plots, where she grows cardamom and rubber, which have a considerable share in the total household income. • In addition to this, now she has also forayed into the trade of cardamom. She has started procuring cardamom from all villagers, dries them and thereafter sells them to the businessmen at 50 per cent profit. This year, she has also started procuring Sacha Inchi from all villagers at the rate of 5000 kips, which is subsequently sold to businessmen at the rate of 10,000 kip/kg. • According to her, cultivating vegetables is a bonus which has been possible because of the support she received under LRP. Otherwise, there is not much effort that is required to manage a vegetable garden. She can continue cultivating vegetables, along with carrying out multiple other livelihood activities. How does the future look for her: She will continue to grow vegetables even if the LRP project ends this year. She feels that managing vegetable gardens is a good way of not only earning extra money but also ensuring nutritious food for her family and school children. The only major task that she needs to carry out while managing vegetable gardens is to fence the plot to protect from animals. USDA Local Regional Procurement Activity Evaluation in Lao PDR Final Report 189 • She intends to become a textile trader, procuring skirts from other weavers as well, and thereafter, selling them at a much higher rate in the town. She has developed a health issue because of which she cannot sit for long hours and carry out weaving herself. She believes that becoming a trader would further increase her earnings. She also intends to buy a four￾wheeler soon so that she can easily travel to and from the market to sell these skirts. • She intends to hire labourers for performing other activities, such as collecting cardamom and rubber tapping. She believes that this will considerably free up her time, which she can invest in something more productive. USDA Local Regional Procurement Activity Evaluation in Lao PDR Final Report 190 Case Study 6: SaiyNamthip Village, Nalae District, Luang Namtha Province Context: Mr. Somsack and Mrs. Taiw were affected by dam construction and as a result, moved from the low area near river Nam Tha to the top of the mountain three years back. SaiyNamthip is the new name of the village formed after merging more than eight villages (some people call this as Donthip). Now after relocation, the population of the merged village is almost equal to that of a small town. As a result, the new village has an independent market of its own, where the locals can buy and sell their produce. Mr. Somsack and Mrs. Taiw have a family of five people, consisting of three children, all of whom have completed secondary school. In the past before relocation to the new village, their main occupation included upland rice. Local people in this area usually consume rice, along with meat, pork and chicken, and vegetables collected from nearby forests. As a result, a number of households are still engaged in collecting NTFPs, a portion of which is consumed, and the remaining sold in the market. Some of the families also carry out rubber plantation which is a well-paying cash crop. In addition to this, some families have recently started cultivating cardamom and fruit Galanga. For quite some time, the water supply in the village is not enough for self-consumption and agriculture. So while LRP selected ten farmers for the project, not all were able to carry out the implementation in the desired manner, due to lack of water. Before LRP: Mr. Somsack and Mrs. Taiw still practice upland rice every year. They cultivate rice on a plot of three hectares and have recently added an additional small plot for cultivating vegetable. The household produces a good quantity of upland rice, a portion of which is sold but is not enough in terms of earnings. Therefore, Mr. Somsack decided to work in Thailand for four years to earn additional money. After four years, with the money he earned after working as a labourer in Thailand, he bought some material for the house and some tools for agriculture activities. As soon as the head of the village announced about the LRP programs, Mr. Somsack and Mrs. Taiw are immediately convinced and joined the program. They became a part of the farmers' group, who got support from LRP for basic tools of farming, seedlings, along with consistent training around technical issues of farming, including methods of preparing compost and bio anti-insect with local resources. All the training that they attended were found to be useful. However, according to them, LRP was not as successful as in other villages, due to many reasons: • At the time the project was implemented, not every villager for ready since they were still engaged in relocation-related aspects • The village does not have enough water supply for agriculture • Majority of the families selected had small landholdings. Implementation of LRP activities also could not ensure an increase in income • Most villagers continue spending maximum time and effort in cultivating rice, resulting in very time and resources for other livelihood options However, Mr. Somsack and Mrs. Taiw have decided to stop cultivating upland rice as they want to change the way people look at livelihood in this region. They firmly believe that nothing will change if people continue practising upland rice. USDA Local Regional Procurement Activity Evaluation in Lao PDR Final Report 191 After the second year, Mr. Somsack and Mrs. Taiw extended their farmland for short time vegetables and long-time harvest of trees. The vegetable garden also has a greenhouse roof though not from the support from LRP. They sell vegetables and support two schools at the same time throughout the year. They have developed a garden in which they grow a wide variety of vegetables, including Chinese cabbage, lettuce, Morning glory, coriander and spring onion. Given that the new village has a huge population, there is an immense demand for these vegetables in the market, resulting in a good income. Not only vegetable, but they have also recently created a small area for feeding frogs, which are eventually sold within the village. The family also recently forayed into producing Lao Kao, which is the traditional white whisky for commercial sales. USDA Local Regional Procurement Activity Evaluation in Lao PDR Final Report 192 Case Study 7: LongMoun Village, Nalae District, Luang Namtha Province Context: Villagers moved to this new village in 1998 after the Laos government initiated a policy to move smaller, isolated villages in the mountainous region to lower area with better access to road, electricity, healthcare centre, schools and market. The new village consists only of Khmu ethnic group, who speak Khmu. There are a total of 222 households in the village, whose main occupation is upland farming of rice. The village has a long history of conflict among villagers. As a result, not many people were ready to participate in our interviews, except for teachers and a few parents. Village organization has been recently formed but has not yet made an announcement about the selected head of the village. After discussions with villagers, it is safe to assess that the conflict within the two factions of this village is still alive, which has also negatively affected the implementation of LRP. Mr. Cher Saen, also moved from an old village to this area, has a total of seven members in the family, along with a child. His house is made from wood and bamboo, with a metal roof. The main livelihood activities in his family include upland rice cultivation, which he practices with his elder brother on a plot of about 2 hectares, and managing a vegetable garden of around 150 sq. meter. This main occupation of the village is upland rice. Some families have converted some of the fallow lands for rubber plantation and cultivating cardamom. These two cash crops are in high demand in this region because of a big market in China. Usually, traders come to the village to procure the vegetables providing good rates. Another major cash crop Sacha Inchi is also usually in high demand, though its production has been relatively lower this year. Due to climate change and insufficient rains, this year has also seen a fall in production of upland rice. As a result, households which had multiple sources of income were relatively better off, as they had money to buy rice from the market. Before LRP: Mr. Cher Saen used to grow the vegetable in his vegetable garden. The main focus at that time was only to grow enough for self-consumption, and exchange among the community. The main problems that farmers like him faced were (1) lack of technical knowhow around taking care of vegetables and (2) growing organic vegetables. In addition, to this, the farmers did not have any information about the nutrition value of the vegetables grown. The farmers used to focus major efforts in growing upland rice in the uphill area, usually mixed with other crops such as corn, cucumber and eggplant for self-consumption. After LRP: After the announcement about the LRP program, Mr. Cher Saen grew extremely interested in it and inquired about it. After clarifications, he decided to join the program as one of the ten farmers from this village. He became a part of the farmers' group, who got support from LRP for basic tools of farming, seedlings, along with consistent training around technical issues of farming, including methods of preparing compost and bio anti-insect with local resources. All training that she attended was found to be useful. He has started contributing vegetables for school meals and selling the surplus in the market. His family have also engaged in maintaining livestock (3 pigs, 4 goats.) for selling in each year. Recently, he has also purchased 2 hectares of land of rubber plantation. He hopes that rubber plantation will be the main source of income for their family, and subsequently he can also include cardamom and Sacha Inchi plantation. USDA Local Regional Procurement Activity Evaluation in Lao PDR Final Report 193 Annex V Glimpses from Validation Workshop At Nalae USDA Local Regional Procurement Activity Evaluation in Lao PDR Final Report 194 At Vientianne