Prepared Decentralized Evaluation Final evaluation of the USDA-supported Local and Regional Procurement (LRP) project in Kenya FY 2017-2020 Evaluation Report May 2020 WFP Kenya Evaluation Managers: Beatrice Mwongela and Ruth Musili Agreement Number: LRP-615-2017/035-00 Prepared by Sophia Dunn, Team Leader John Otsola, Research Specialist DECENTRALIZED EVALUATION FOR EVIDENCE-BASED DECISION MAKING ii Acknowledgements The evaluation team wish to acknowledge the contributions that the WFP Kenya team have put into this evaluation. In particular, the team wishes to thank Daniel Ndungu for helping with the enumerator training, Gabriel Ekaale for organizing interviews in Turkana County, Regina Kyalo for supervising enumerators in Baringo, and the Monitoring and Evaluation Team, especially Beatrice Mwongela, Ruth Musili and Polly Akwanyi for organizing the fieldwork, supervising the enumeration teams and for their ongoing support to the field mission. Thank you also to all the WFP personnel who provided their input through key informant interviews. The team would also like to thank all the national and county level Government of Kenya stakeholders, especially the personnel from the Ministry of Education, Ministry of Agriculture and Irrigation, Ministry of Health and Ministry of Industry, Trade and Cooperatives who participated in the evaluation as key informants. Special thanks also to the government personnel, particularly the Ministry of Agriculture teams in Baringo and West Pokot, who helped organize interviews and/or accompanied the Team Leader for field visits. The evaluation team also extends their thanks to all the traders, Farmer Organization members and school personnel who participated in the qualitative and quantitative interviews. Special thanks are also due to the team of enumerators, survey supervisors and drivers without whom we could not have completed the quantitative data collection: Enumerators: Amos Apalekem Pkiach, Amos Yeko, Betty Judy Mukami Mutwiri, Caroline Ikai, Dennis Kiplagat, Dennis Marias, Edward Kipchirchir Cheruiyot, Elizabeth Ndole, Emily Akuto, Ezekiel Chebii, Geoffrey Nalima, Joyce Chebet, Magdalene Lokaale, Melvin Kirwa, Millicent Malago, Miriam Lotukoi Ikimat, Murket Chepkite Rose, Nancy Ajega Ongere, Norah Chemsto Kariwo, Peter Nzamba, Philip Ojowi Akhonya, Pius Rotich, Ryan Wambua Makau, Shirley Layo Loumo, Stanley Elain and Zakayo Muindi. Drivers: Jackson Ngige, Kennedy Omwenga, Simon Njaramba and Wilson Ereng Disclaimer The opinions expressed in this report are those of the Evaluation Team, and do not necessarily reflect those of the World Food Programme. Responsibility for the opinions expressed in this report rests solely with the authors. Publication of this document does not imply endorsement by WFP of the opinions expressed. The designation employed and the presentation of material in maps do no imply the expression of any opinion whatsoever on the part of WFP concerning the legal or constitutional status of any country, territory or sea area, or concerning the delimitation of frontiers. iii Table of Contents 1 Introduction............................................................................................................................1 1.1 Overview of the evaluation subject......................................................................................2 1.2 Context..................................................................................................................................4 1.3 Evaluation methodology and limitations..............................................................................8 2 Evaluation Findings...............................................................................................................13 2.1 Overview of surveyed beneficiaries....................................................................................13 2.2 Evaluation Theme 1: Relevance ..........................................................................................15 2.3 Evaluation Theme 2: Effectiveness .....................................................................................18 2.4 Evaluation Theme 3: Efficiency ...........................................................................................36 2.5 Evaluation Theme 4: Impact ...............................................................................................38 2.6 Evaluation Theme 5: Sustainability .....................................................................................41 3 Conclusions and Recommendations......................................................................................42 3.1 Overall Assessment/Conclusions........................................................................................42 3.2 Recommendations..............................................................................................................44 4 Annexes................................................................................................................................46 5 List of Acronyms.................................................................................................................. 123 iv List of Annexes Annex 1: Endline evaluation terms of reference.................................................................................46 Annex 2: Key LRP stakeholders and their interest in the endline evaluation.......................................59 Annex 3: Graphic representation of the relationship between project stakeholders..........................60 Annex 4: Results Framework – LRP Kenya...........................................................................................61 Annex 5: Completed LRP Performance Monitoring Plan .....................................................................62 Annex 6: List of LRP baseline recommendations (2018) ......................................................................64 Annex 7: Maps of Kenya showing WFP and government supported school feeding activities during LRP baseline (March 2018) and at endline (October 2019). ................................................................65 Annex 8: Evaluation Matrix ..................................................................................................................66 Annex 9: List of data and documents provided to the evaluation team by WFP Kenya ......................70 Annex 10: Quantitative survey tools....................................................................................................71 Annex 11: Description of the calculation of the evaluation indices.....................................................94 Annex 12: Qualitative data collection tools .........................................................................................96 Annex 13: Silhouette measure of cohesion and separation...............................................................100 Annex 14: List of surveyed LRP and non-LRP comparison schools.....................................................101 Annex 15: Detailed description of the original (baseline) school sampling methodology .................104 Annex 16: Evaluation field mission schedule .....................................................................................106 Annex 17: Stakeholders Interviewed .................................................................................................108 Annex 18: Additional findings from the Farmer Organization survey................................................110 Annex 19: Additional findings from the trader survey .......................................................................115 Annex 20: Additional findings from the school survey.......................................................................117 Annex 21: Bibliography ......................................................................................................................121 List of Tables Table 1: Planned LRP activities...............................................................................................................3 Table 2: LRP locations.............................................................................................................................4 Table 3: Number of Farmer Organizations and traders participating in the LRP, by county..................4 Table 4: Timeline of emergencies in LRP counties – 2017-2020 ............................................................5 Table 5: Preliminary Key Evaluation Questions......................................................................................9 Table 6: Number of LRP and non-LRP schools surveyed at endline evaluation, by county..................12 Table 7: LRP Targeted traders included in the evaluation survey ........................................................12 Table 8: Characteristics of surveyed Farmer Organizations, by county. ..............................................13 Table 9: Gender of surveyed traders supported by the LRP ................................................................14 Table 10: Business information of surveyed traders, by county ..........................................................14 Table 11: Characteristics of surveyed schools, by county ....................................................................15 Table 12: Planned vs actual delivery of food commodities acquired by WFP though forward delivery contracts (Turkana only) ......................................................................................................................19 Table 13: Number of participants attending training on HGSMP management (2017-2018)..............20 Table 14: Number of policies/strategies supported as a result of USDA assistance............................22 Table 15: Percentage of LRP traders who received training on procurement processes.....................22 Table 16: Number of participants in workshops to develop new school meal menus ........................24 Table 17: Number of individuals benefiting from this USDA funded intervention ..............................24 v Table 18: Commodities grown by FOs..................................................................................................25 Table 19: Ensuring quality products for sale ........................................................................................25 Table 20: FO sales of members’ commodities .....................................................................................26 Table 21: FO sale of commodities in last one year...............................................................................26 Table 22: Number of FOs with sale contracts and value addition enterprises.....................................27 Table 23: Volume and value of sales by LRP FOs..................................................................................27 Table 24: Percentage of traders selling commodities to schools through competitive tenders..........28 Table 25: Total estimated volume of commodities sold per year........................................................29 Table 26: Percentage of traders reporting barriers to purchasing from local farmers........................29 Table 27: Main source of food for school lunches (Term 3, 2019).......................................................30 Table 28: Percentage of schools reporting providing a school meal every school day........................31 Table 29: Reasons for not providing school meals...............................................................................31 Table 30: Change in Nutrition Score: Baseline vs. Endline ...................................................................32 Table 31: Mean volume of food per child received..............................................................................33 Table 32: Change in Cost-effectiveness Score......................................................................................34 Table 33: Average number of days after start of Term 3, 2019 that food was delivered ....................37 Table 34: Change in Timeliness Score ..................................................................................................37 Table 35: Change in Impact Score ........................................................................................................38 List of Figures Figure 1: Average time the FOs have been established .......................................................................14 Figure 2: FO members’ average use of own production ......................................................................26 Figure 3: Percentage of traders reporting their main source of commodities.....................................28 Figure 4: Percentage of traders reporting selling to the following buyers...........................................28 Figure 5: Percentage of traders reporting sale of staple crops............................................................29 Figure 6: Percentage of traders reporting sale of pulses .....................................................................29 Figure 7: Percentage of surveyed schools reporting developing new school menus since start of LRP ..............................................................................................................................................................31 Figure 8: Percentage of schools reporting using food commodities for their school lunches - Term 3, 2019......................................................................................................................................................32 Figure 9: Percentage of schools reporting food procurement process completed before start of Term 3, 2019..................................................................................................................................................36 Figure 10: Percentage of schools reporting that school lunch commodities from non-MoE sources were delivered before the start of Term 3, 2019.................................................................................36 WFP Kenya Endline LRP Evaluation Report – May 2020 |Page vi Executive Summary 1. This report is the endline activity evaluation of the World Food Programme’s (WFP) Local and Regional Food Aid Procurement Programme (LRP) in Kenya. The programme is funded by the United States Department of Agriculture (USDA) and is implemented in three arid counties in north-west Kenya: Baringo, Turkana and West Pokot. The evaluation is commissioned by the WFP Kenya Country Office and follows a baseline assessment conducted in April 2018 by the same evaluation team.1 The main objective of the final evaluation is to assess the performance and results achieved through the LRP in the three targeted counties over the project period from September 2017-March 2020. The evaluation serves the dual and mutually reinforcing objectives of accountability and learning. 2. The LRP was implemented from October 2017 to March 2020 using a grant of USD 1 million. The LRP was specifically designed to support the implementation of the Government’s Home-Grown School Meals Programme (HGSMP) by supporting farmers, farmer organizations (FOs) and local traders to produce and provide sufficient food for the ongoing school feeding activities in the three targeted counties. USDA also gave approval for WFP to locally purchase sorghum and cowpeas from FOs for use in the school feeding programme. There have not been any budget revisions or changes to the programme design since it was approved in September 2017, and there are no sub-recipients under this agreement. No other donors have contributed funds to the LRP per se, however some of the FOs supported under the LRP have also benefitted from support from other donors. The LRP is implemented directly by WFP Kenya, in close collaboration with the Ministries of Education (MoE), Agriculture and Irrigation (MoA) and Health (MoH), at county and sub-county levels. 3. The key objectives of the LRP are as follows: • Improve effectiveness of food assistance by improving cost-effectiveness and improving timeliness. • Increase the capacity of suppliers and school meals procurement committees to effectively and efficiently procure local commodities for school’s meals, promoting sustainability of school feeding; • Strengthen local and regional food market systems, improving access to culturally acceptable commodities and connecting them to Government of Kenya HGSMP; and • Improve nutrition of students by increasing access to and use of various high quality, nutritious and culturally appropriate foods in school’s meals. 4. To achieve these objectives, the LRP has five main activities: 1. Assessment of local food systems, 2. Capacity building for national and county institutions, 3. Capacity strengthening for local traders and FOs, 4. Develop school meal menus using local and nutritious produce, and 5. Procure locally produced, drought-tolerant crops. The main users of this evaluation report are WFP Kenya and USDA, both of whom have a responsibility to ensure that the evaluation proceeds as per the programme agreement between the two parties. There are also several other internal and external stakeholders with an interest in the evaluation findings including the MoE, MoA and MoH and other WFP offices. 5. Context: WFP and the MoE have jointly implemented a school meals programme in Kenya since 1980, targeting the most food-insecure counties with the lowest enrolment and completion rates and high gender disparities. Initially, school meals used an in-kind modality with food commodities provided directly by international donors including the USDA/McGovern-Dole. In 2009, the Government of Kenya started the national HGSMP to provide meals to children at school, using a cash transfer modality, providing cash to schools to enable them to buy their food directly from local traders and farmers. The HGSMP stimulated local agricultural production through purchase of food from smallholder farmers and local traders. The arid counties in Kenya’s north were the last to transition to cash-transfers, to give adequate time to assess the local markets and ensure they would be able to cope with the demand of school feeding. All the targeted LRP schools in Baringo and West Pokot were handed over to the HGSMP in September 2017. WFP officially handed over all remaining WFP-supported schools, including the LRP schools in Turkana to the 1Dunn, S & Otsola, J (2018) Baseline of the Local and Regional Food Aid Procurement Project in Kenya, 2017-2020. June 2018. WFP Kenya Endline LRP Evaluation Report – May 2020 |Page vii government’s HGSMP in June 2018. This was on the understanding that the government would provide cash to schools to procure their own food for school meals. 6. Methodology: This evaluation uses the OECD-DAC2 international evaluation criteria of relevance, effectiveness, efficiency, sustainability and impact to evaluate the implementation of the LRP project. The evaluation team has also ensured that gender equality and the empowerment of women (GEEW) was mainstreamed through the evaluation approach and methodology by ensuring that whenever possible, men and women, boys and girls from different stakeholder groups, participated in the evaluation, and data has been gender disaggregated when possible. Where possible, the evaluation has also tried to assess whether the project had different impacts on male and female farmers and traders. The evaluation used mixed methods and collected both quantitative and qualitative primary data to answer each of the evaluation questionsincluding - 1. Was the project designed to reach the right people with the right type of assistance? 2. Is the project aligned with national government’s relevant policies and strategies? 3. To what degree have the interventions resulted in the expected results? 4. Did assistance reach the right beneficiaries at the right time? 5. Were there any unintended outcomes, either positive or negative? 6. What internal and external factors affected the project’s results? 7. To what extent is it likely that the benefits of the project will continue after the end of the project? And 8. What are the key factors that affect the likelihood of sustainability of the results of the project? 7. The evaluation also utilized WFP Kenya’s own project monitoring data, to complete the Performance Monitoring Plan (PMP) required by USDA. The evaluation incudes three quantitative surveys: FOs, local traders and schools, as well as qualitative field interviews and focus group discussions, to assess the effect of the program against the LRP objectives. The evaluation methodology has three identified limitations: Firstly, that two schools from the original baseline sample were not included at endline.3 These two schools were replaced with schools with 99% matching characteristics4 to the original schools, so there should not be any effect on the group comparison of results from baseline. Secondly, that the school survey was partially conducted during the Term 1, 2020 half term holidays. This meant the survey teams had to organize times with each school to conduct their visit, to ensure that appropriate school personnel would be available to answer the survey questions. The mitigation measures were successful, and personnel from each school were located to participate in the survey. Lastly, although there is some gender disaggregated data available from WFP, the LRP Results framework does not include gender indicators. Furthermore, the beneficiary units are largely genderless (schools and farmer organizations). This has limited the gender￾related findings of the evaluation. 8. Key findings: Evaluation question 1: Was the LRP relevant? The LRP was designed in collaboration with MoE, MoA and MoH and the evaluation found the design to be relevant to support HGSMP transition in the three targeted counties. The planned modality of cash-transfers for providing school meals was the preferred option of all evaluation key informants, as it not only supports local farmers and trader, but aligns well with the government’s existing HGSMP. Overall, the endline evaluation found that the LRP complements the HGSMP, and aligns well with key Government of Kenya policies, strategies and framework, as well as WFP’s own polices and with the direction of other United Nations actors in Kenya. 9. Evaluation question 2: Was the LRP effective? WFP and partners implemented most of the LRP project activities as planned. However, because of the 2017/18 drought, the MoE decided not to transition the targeted LRP schools onto cash-transfers as planned, but to keep them on in-kind assistance until food prices reduced. This has had a significant effect on the LRP results and some of the intended school-related objectives have not been achieved. Out of the 191 schools surveyed, only 18 non-LRP schools in West Pokot reported receiving cash-transfers for HGSMP as planned. None of the other schools received any resources (food or cash) from the MoE for school meals for Term 3, 20195 As a result, only around a quarter of schools 2 Organisation for Economic Co-operation and Development’s Development Assistance Committee 3 One because a bridge had been destroyed by floods and the school was no longer accessible, and the second due to insecurity. 4 The replacement schools were matched using the following attributes: Grouping variable (LRP or non-LRP); County; enrolment levels; the original selection anomaly index, peer ID, and peer size using the silhouette measure of cohesion and separation algorithm. 5 Term 3, 2019 was the last school term of the LRP and therefore designated as the endline term for evaluation purposes. WFP Kenya Endline LRP Evaluation Report – May 2020 |Page viii were able to provide lunch every day that term. Although many schools tried to obtain food from other sources to supplement any left-over food from Term 2, many schools (n=75) were unable to provide any school meals during Term 3, 2019. 10. Through the LRP, WFP conducted several formal training exercises and workshops, in collaboration with the relevant government ministries. While the evaluation received positive feedback on all the training work, most participants did not have a chance to put the learning into practice due to schools not requiring local procurement as planned. The exception has been the procurement training for FOs and traders. As the HGSMP procurement process follows the government’s procurement process, traders have been able to use their learning to bid for other government tenders, particularly for food for secondary school and boarding school meals. Similarly, the training for FOs has contributed to more FOs monitoring product quality requirements. The market forums and other support to FOs has also effectively resulted in changes to FO production including a greater percentage of FOs aggregating and marketing members food commodities, and FOs in West Pokot reported selling a greater percentage of their production. This despite the in-kind provision by MoE reducing the potential market opportunities for FOs and acting as a disincentive to produce the promoted crops. The collaborative implementation approach taken by WFP with MoE, with MoA, MoH and MoTIC has also been an effective capacity strengthening approach, providing government personnel with added confidence to provide training on their own in future. 11. Evaluation question 3: Was the LRP efficient? One of the objectives of the LRP was to improve the timeliness of procurement of school meals commodities. This was done by providing schools with cash transfers, enabling them to do local procurement of commodities, hopefully resulting in more timely delivery of commodities to the schools. However, since the MoE did not provide any cash resources to LRP schools for Term 3, 2019, none of the LRP schools undertook a procurement process. The target of 90 percent of LRP schools completing procurement before the start of term has therefore not been achieved. Further, none of the LRP schools received any in-kind resources from the MoE for Term 3, 2019, so the target of 90 percent of LRP schools having their food delivered before the start of term has also not been achieved. 12. Evaluation question 4: What was the impact of the LRP? The LRP was specifically designed to support the transition to cash-based transfers in the LRP schools. The impact of the LRP was therefore considerably reduced by the MoE’s decision to provide in-kind commodities to schools instead of providing cash-transfers to school as planned. As a result, the intended objective of improving the effectiveness of assistance through improved timeliness of procurement and delivery, improved cost-effectiveness has not been achieved. The objective to increase access to, and use of various high quality, nutritious and culturally appropriate foods in school’s meals of the school meals by using locally produced crops has been achieved only in Turkana Country and not in Baringo or West Pokot. This is because in Turkana, WFP directly procured and delivered the commodities to LRP schools. 13. The objective to increase the capacity of suppliers and school meals procurement committees to effectively and efficiently procure local commodities for school’s meals, has been partially achieved. FOs, traders and school personnel all reported attending training and gaining knowledge on HGSMP. School personnel also gained increased awareness of the importance of dietary diversity and how the school meal menus can be altered to increase the diversity of commodities. However, most of these groups were unable to put their learning into practice as no school-based procurement took place. 14. The final LRP objective was to strengthen local and regional food market systems, improving access to culturally acceptable commodities and connecting them to Government of Kenya HGSMP. Although the FOs and traders were unable to link into the HGSMP as planned, the LRP activities have resulted in several positive outcomesfor FOs and traders. These include increased awareness of FOs and traders of the HGSMP in general, and increased knowledge for FOs and traders on the required HGSMP procurement process. As a result of the LRP market forums, some traders were also able to supply commodities to other schools – secondary and boarding schools – that also implement school meals. Qualitative interviews also identified some negative outcomes for FOs and traders, as both groups were left with larger than usual volumes of WFP Kenya Endline LRP Evaluation Report – May 2020 |Page ix commodities after planning to sell stock to the HGSMP schools. Some traders also allowed schools to buy commodities on credit, pending future payment of the cash-transfers, however with uncertain plans for reverting to cash, it is unclear when schools will be able to pay off those debts. 15. Finally, the evaluation commends WFP for the ongoing effort to include women in the programme and impact the lives of women. Women have been actively encouraged to participate in all aspects of the programme, and this active recruitment has succeeded, with women making up half the traders sampled (46.3%) and more than half the FO membership. 16. Evaluation question 5: Are the results of the LRP sustainable? As a programme designed to support the HGSMP, the LRP was never intended to be a sustainable programme in its own right. However, several of the LRP activities have contributed to sustainable results. The decision by the MoE to implement an in-kind modality reduced the effectiveness of the LRP, but it in no way reduced the sustainability of the HGSMP itself. It is, however, difficult to clearly state whether some of the LRP results are likely to be sustained or not as the sustainability of some results depend on the MoE’s future plans regarding providing cash transfers. Regardless of the HGSMP modality, the LRP has succeeded in strengthening relationships between traders and FOs and contributed to improving the policy landscape promoting smallholder procurement. The least sustainable aspect of the LRP is likely to be the use of drought tolerant crops in the school menus. Currently, those commodities are more expensive than maize and beans, so unless schools are provided with a cash transfer rate than enables procurement of these commodities, they are unlikely to purchase those commodities regardless of the nutritional or economic benefits to the community 17. Overall conclusions: The evaluation found that while all the planned LRP activities and outputs have been completed, and the project has resulted in several positive outcomes, the LRP has not been able to achieve the school-based objectives of improving the timeliness or cost-effectiveness of food assistance. This is a result of MoE providing LRP schools with in-kind commodities instead of the cash-transfers that were planned, and which the LRP was specifically designed to support. The LRP has however succeeded in increasing the capacity of traders, FOs and school meals procurement committees on the procurement processes of HGSMP, although none of these groups have been able to put their learning into practice. Similarly, although capacity of school personnel on nutrition and diversity of the school menus has been strengthened though training and the menu development workshops, this training and the resulting menus have not been put to use in Baringo and West Pokot. In Turkana County, schools used the new menus to incorporate sorghum and cowpeas into the school meals. The expected boost to the local and regional food market systems that the HGSMP would have provided, has not resulted, although other market activities have helped FOs and traders sell greater volumes. The evaluation found it was not possible for WFP to have foreseen the sudden change in position on cash-transfers by the MoE especially after successful transition to cash transfers in other arid areas prior to handover to government. Based on the findings and conclusions of this evaluation, the evaluation team made the following recommendations: • Recommendation 1: WFP and the MoE school feeding technical officers should continue to advocate to the MoE senior leadership on the importance of school feeding, timeliness of delivery, timeliness of cash disbursement and importance of providing daily school meals. This advocacy role can also be done at county level in support of the draft ECDE framework. • Recommendation 2: WFP and MoE should develop a position paper outlining the key decisions over the last decade that have led to the largely cash-based modality of HGSMP that is seen today. The paper should also include the evidence that underpins those decisions including recent market assessments in the arid areas. The paper should then be used for ongoing advocacy with the MoE as described in Recommendation 1. • Recommendation 3: WFP and the MoE school feeding technical officers should consider ways to support improvements in the MoE’s in-kind procurement and delivery system. This is especially important if the MoE intends to continue providing in-kind support to the arid and semi-arid areas which are among the locations most in need of the food and nutrition security improvements that the HGSMP can provide. WFP Kenya Endline LRP Evaluation Report – May 2020 |Page x • Recommendation 4: WFP should continue to work with county/sub-county governments to support farmers, FOs and market system improvement in general. This is especially important for work involving the MoA, as their authority has been devolved to the counties. • Recommendation 5: WFP should continue to work together with MoH at national and county level, on school health and nutrition related work, especially on activities that improve the nutritional status of school-aged children such as the provision of nutritious, locally produced food commodities. • Recommendation 6: WFP should look for funding to continue to the work started through the LRP supporting farmers and FOs in Baringo and West Pokot as the LRP was the first opportunity for support, and without the opportunity to put their learning into practice during the LRP, any gains made are likely to regress without some reinforcement. • Recommendation 7: WFP should continue to work with the MoA and other relevant ministries on the promotion of small holder procurement. This should include continuing the development of county and national level strategies and other means to increase opportunities for small holder engagement in public procurement. • Recommendation 8: WFP should continue to support MoE’s efforts to conduct regular programme monitoring to ensure that HGSMP processes and procedures are being adequately followed. • Recommendation 9: WFP and partners should continue to support the active inclusion of women in all WFP projects and ensure that gender related indicators are present in future WFP projects, as appropriate. WFP Kenya Endline LRP Evaluation Report – May 2020 |Page 1 1 Introduction 1. This report is the endline activity evaluation of the World Food Programme’s (WFP) Local and Regional Food Aid Procurement Programme (LRP) in Kenya. The programme is funded by the United States Department of Agriculture (USDA) and is implemented in three counties in north-west Kenya: Baringo, Turkana and West Pokot. The evaluation is commissioned by the WFP Kenya Country Office (CO) who provided the Terms of Reference (Annex 1). This endline evaluation follows a baseline assessment conducted in April 2018 by the same evaluation team.6 2. The main objective of the final evaluation is to assess the performance and results achieved of the LRP in the three targeted counties over the project period from October 2017 to March 2020. The evaluation serves the dual and mutually reinforcing objectives of accountability and learning. • Accountability: The evaluation will assess and report on the performance and results of the USDA LRP support to WFP School Feeding Programme in Kenya. • Learning: As this is the first LRP support to WFP Kenya, the evaluation will generate lessons that WFP Kenya, USDA and the Government of Kenya can use to inform future programmes. Findings will be actively disseminated, and lessons learned will be incorporated into relevant lesson sharing systems. 3. The purpose of this report is to present the main findings, conclusions and recommendations of the evaluation. It also provides information on the context, the subject of the evaluation, its stakeholders, the approach and methodology that was used. 4. The LRP aims to support the Government of Kenya’s effort to expand the national Home-Grown School Meals Programme (HGSMP) into the arid counties and stimulate local economic development through the procurement of food from local markets, increasing farmers’ incomes and creating additional jobs in the community. The LRP is implemented directly by WFP Kenya, in close collaboration with the Ministries of Education (MoE), Agriculture and Irrigation (MoA) and Health (MoH), as well as county and sub-county governments. There are no sub-recipients under this agreement. 5. In the Evaluation Plan agreed with USDA in 2017, WFP committed to conduct a final evaluation to provide an evidence-based, independent assessment of performance of the programme in order to evaluate the project’s success, ensure accountability, and generate lessons learned. 6. The main users of this evaluation report are WFP Kenya and USDA, both of whom have a responsibility to ensure that the evaluation proceeds as per the programme agreement between the two parties. Along with WFP Kenya and USDA, several other internal and external stakeholders have an interest in the findings of this evaluation including the MoE, MoA and MoH that implement the programme and other WFP offices. Annex 2 of this report provides a more detailed preliminary stakeholder analysis and lists the stakeholders that contributed to evaluation, and Annex 3 provides a graphic representation of the relationship between stakeholders. The primary users of this evaluation are: • The WFP Kenya Country Office is interested in findings notably around programme implementation and/or design, Country Strategy and partnerships. • The Government of Kenya, particularly the MoE, the MoA and the MoH and local authorities in the three LPR counties, is interested in findings as they took over the implementation and management of the school feeding activities in 2018. These three ministries all have an interest in learning how best to engage local farmers and suppliers, so they continue to be involved in the provision of locally produced, nutritious food commodities to school feeding activities in arid areas. 6 Dunn, S & Otsola, J (2018) Baseline of the Local and Regional Food Aid Procurement (LRP) Project in Kenya, 2017-2020. Final Report, June 2018. WFP Kenya Endline LRP Evaluation Report – May 2020 |Page 2 • USDA is interested in the evaluation findings, both from an accountability perspective, and for the purpose of expanding their body of knowledge on LRP programmes. USDA may also use the findings and lessons learnt from this final evaluation to inform future program funding, design, and implementation decisions. • The WFP Regional Bureau in Nairobi (RBN) is expected to use the evaluation findings to provide strategic guidance, programme support, and oversight. • WFP Headquarters (HQ) may use evaluation findings for wider organizational learning and accountability. 7. As accountability to affected populations is tied to WFP’s commitments, LRP beneficiaries also have a right to contribute to the evaluation, and an interest in the overall finding of this evaluation. LRP beneficiaries were therefore included in key informant interviews and/or focus group discussions. For the LRP there are several groups of beneficiaries including government officers, farmers, Farmer Organizations (FOs), local traders, school personnel, School Meal Committee (SMC) members, parents and school children. Each of these groups participated in the evaluation. To ensure gender equality and women’s empowerment in the evaluation process, the evaluation ensured participation and consultation by women, men, boys and girls as appropriate. WFP Kenya will also ensure that the evaluation findings are disseminated to stakeholders. 1.1 Overview of the evaluation subject 8. The LRP was specifically designed to support the implementation of the Government’s HGSMP. It was designed to support farmers, FOs and local traders to produce and provide sufficient food for the ongoing school feeding activities by the Government of Kenya in Baringo, Turkana and West Pokot counties. The LRP was implemented from October 2017 to March 2020 using a grant of USD 1 million. USDA also gave approval for WFP to locally purchase sorghum and cowpeas from local FOs for use in the school feeding programme. There have not been any budget revisions or changes to the programme design since it was approved in September 2017, and there are no sub-recipients under this agreement. No other donors have contributed funds to the LRP per se, however as described ahead, some of the FOs supported under the LRP have also benefitted from support from other donors. 9. Before the LRP, the targeted schools were implementing the school meals programme (SMP) through in￾kind food distribution by WFP. Primary school children were provided with a hot lunch at school which comprised 150g of cereals, 40g of pulses, 5g of oil and 2g of iodized salt per child per day. The LRP was designed to transition the LRP schools onto cash transfers under the HGSMP to enable schools to procure their own food commodities, potentially enabling faster delivery (timeliness), reducing overall costs (cost￾effectiveness) and increasing the diversity of the foods provided (nutrition). 10. Objectives and activities: The key objectives of the LRP are to: • Improve effectiveness of food assistance by improving cost-effectiveness and improving timeliness. • Increase the capacity of suppliers and school meals procurement committees to effectively and efficiently procure local commodities for school’s meals, promoting sustainability of school feeding; • Strengthen local and regional food market systems, improving access to culturally acceptable commodities and connecting them to Government of Kenya HGSMP; and • Improve nutrition of students by increasing access to and use of various high quality, nutritious and culturally appropriate foods in school’s meals. 11. To accomplish these objectives the project has five main activities asshown in Table 1. All the LRP activities were implemented in close collaboration with the MoE, MoA and/or MoH and county governments. Government officers at the national and county levels ensured that the required government approvals WFP Kenya Endline LRP Evaluation Report – May 2020 |Page 3 were received on time, participants identified, mobilized and invitations issued to the capacity development events. The government officers also participated in facilitating capacity building sessions including training sessions. The majority of activities were completed by October 2019 with the official end date being 31st March 2020 when the few remaining activities, including the reporting of this evaluation, will have been completed. The evaluation scope covers all the activities implemented as part of the LRP in each of the three targeted counties. The evaluation has mainstreamed gender equality across the objectives, as far as possible within the limitations of the results framework. Table 1: Planned LRP activities Key LRP activities Planned Assessment of local food systems Two assessments: Baringo and West Pokot Capacity building for national and county institutions Train schoolteachers, parents and Education Officers in Turkana, Baringo and West Pokot Establish monitoring and oversight plan Support the development of the National Implementation Strategy for Local Procurement from Smallholders Coordinate first implementation of the above strategy Capacity strengthening for local traders and FOs Train local FOs, suppliers and traders in Baringo and West Pokot Conduct market linkage forums in Baringo and West Pokot Develop School Meals Menus Using Local and Nutritious Produce Nutrient profile developed for selected crops Support schools to develop, use and promote diversified school meal menus, including a training guide Assess inclusion of local commodities in school meals Procure Locally Produced, Drought￾Tolerant Crops Purchase of locally grown, drought-tolerant food commodities (sorghum and cow peas) from participating FOs 12. Although the HGSMP has been ongoing since 2009 using a cash transfer modality, the government maintained in-kind support to the arid counties including the three LRP counties. This was due to concerns that the market system in the arid areas would not be able to support sufficient local procurement of commodities, at a competitive price, to support school feeding activities. Since 2015, the arid counties have been successfully transitioned to HGSMP, with the three LRP counties among the last to transition. 13. The LRP was based on the USDA-based LRP Results Framework (RF) (Annex 4), which was modified by WFP Kenya to suit the Kenyan context. The RF provides a picture of the relationship between the activities, outputs and outcomes and how the project intends to reach its objectives. USDA’s definition of “local and regional procurement” simply means procurement within the country or within the region.7 This means that the LRP is intended to supply schools from food within Kenya, or from the East Africa Region instead of importing food from overseas. The Performance Monitoring Plan (PMP) prepared during the baseline assessment (Annex 5) shows the indicators that are being used to measure performance of the project. As per the grant agreement, there are 24 specific performance and results indicators against which performance of the programme will be measured. The PMP has been updated and completed during this endline evaluation. 14. Project beneficiaries: The LRP supports 382 primary schools in Baringo, Turkana and West Pokot Counties, covering four sub-counties as shown in Table 2. 7 Hanrahan, C.E. (2010) Local and regional procurement for U.S. International Emergency Food Aid. Congressional Research Service. WFP Kenya Endline LRP Evaluation Report – May 2020 |Page 4 Table 2: LRP locations 15. In addition to the personnel and children attending the 382 primary schools, the direct beneficiaries of the LRP are local farmers and FOs and local traders. Participation in the LRP by traders and farmers/FOs is voluntary and WFP used an ongoing recruitment process. By the end of the programme, WFP was working with 78 FOs and 29 traders (Table 3), each of whom are working in the vicinity of the 382 targeted primary schools. WFP had previously worked with six of the FOs in Turkana as part of their now complete Purchase for Progress (P4P) programme, while all the other FOs are new WFP partners/beneficiaries specifically for the LRP. Table 3: Number of Farmer Organizations and traders participating in the LRP, by county 16. Gender dimensions of the intervention: The LRP has implemented several strategies to ensure that women are included in the programme as much as possible. For example, although FO and supplier participation in the project is voluntary, the LRP actively tried to support women farmers and women￾managed FOs and actively encouraged them to register with the programme. Further, when conducting training for traders, WFP encouraged women traders to register and participate. The WFP LRP team also conducted multiple trainings for FOs and traders, including Gender in Agribusiness in an effort to support women in business. Despite these efforts to improve gender equity and equality, the LRP does not have any stated gender-related results that it is trying to achieve. The PMP does however include several indicators that require gender disaggregation so that gender differences can be further analysed. 17. Baseline findings: The 2018 LRP baseline assessment8 made several recommendations, the implementation of which were evaluated as part of this endline evaluation. In general, the baseline found that the activities and results framework were appropriate and would result in the planned objectives being achieved. The full list of those recommendations can be found in Annex 6. 1.2 Context 18. The latest Human Development Index (HDI) value for Kenya (2018) is 0.579, which puts it in the medium human development category - positioning it at 147 out of 189 countries and territories.9 Kenya’s population is currently 49.36 million10 the majority of whom live in rural areas. It also has the largest, most diversified economy in East Africa. As of 2019, Kenya had an estimated Gross Domestic Product (GDP) of USD109 billion and per capita GDP of USD2,015 making it the 62nd largest economy in the world.11 8 Dunn, S & Otsola, J (2018) Baseline of the Local and Regional Food Aid Procurement Project in Kenya, 2017-2020. Final Report, June 2018. 9 UNDP (2019) Inequalities in Human Development in the 21st Century. Briefing note for countries for the 2019 Human Development Report. Kenya. 10 https://www.imf.org/en/Countries/KEN 11 https://www.imf.org/external/datamapper/profile/KEN/WEO. Accessed 9 January 2020 County Sub-county No. LRP targeted schools Baringo East Pokot (Tiaty) 125 Turkana Turkana South & East 130 West Pokot North Pokot 127 Total 382 Baringo Turkana West Pokot Total Farmer organizations 23 35 20 78 Traders 3 -- 26 29 WFP Kenya Endline LRP Evaluation Report – May 2020 |Page 5 19. Poverty: A recent World Bank report12 indicates that the proportion of Kenyans living on less than the international poverty line13 declined from 43.6 percent in 2005/06 to 35.6 percent in 2015/16 with no more recent figures available. According to the same report, poverty incidence in Kenya is amongst the lowest in East Africa and is lower than the Sub-Saharan African regional average. However, the report notes that poverty rates in Kenya remain relatively high compared to other lower middle-income countries. 20. Characteristics of the LRP counties: The three LRP counties are all located in the north-western part of Kenya within the Rift Valley Region (Annex 7) and are classified as arid. People in Turkana County are mainly pastoralist with some small-scale crop production for household consumption. Livelihoods fall within the North-western pastoral zone and the North-western agropastoral zone around the irrigation schemes. Baringo and West Pokot Counties have a more agropastoral profile falling within the Western agropastoral zone while the north of Baringo falls into the North-western pastoral zone. Local crop farmers have limited opportunities to participate in the local markets due to production challenges including high input costs, high transport costs and the presence of disease/pests. This means it is difficult for farmers within the arid areas to compete with products coming from more productive areas. 21. Food security: Food and nutrition security is one of the major challenges affecting development in Kenya and is closely linked to the high level of poverty in the country. Most of the hunger prone areas are located in arid and semi-arid lands in the north. Turkana stands out as being more food insecure than any other county, with almost one in five households (19%) having poor levels of food consumption and a further 24 percent of households having borderline food consumption. Baringo and West Pokot are also in the five most food insecure counties (by food consumption score).14 22. Addressing national food security is a key objective of Kenya’s agricultural sector. However, in the last decade the country has faced severe food insecurity. At the end of 2017, the Government of Kenya declared a national emergency as a result of drought. The drought was severe in all three LRP targeted counties, with Turkana and Baringo being among the worst-affected areas in the country. Over the next two years Kenya experienced floods then rain failure (Table 4), with the food security situation deteriorating over 2019. As of October 2019, it is estimated that 3.1 million people in Kenya were experiencing food security Crisis (IPC Phase 315) or worse outcomes, including some households that are likely in Emergency (IPC Phase 4) including in Turkana and Baringo where the LRP is implemented.16 The Kenya Zero Hunger Strategic Review17 indicates that Kenya is unlikely to meet Sustainable Development Goal (SDG) 2 of achieving zero hunger by 2030. Table 4: Timeline of emergencies in LRP counties – 2017-2020 Year Emergency context 2017 Government of Kenya declares drought a national emergency 2018 Long rains assessments (March) indicate heavier than usual rains, resulting in flooding Short rains failed 2019 Long rains failed Short rains provided more than 100x the expected rainfall, but resulted in good crop production due to the timing of the rains 12 Awiti, C et al (2018) Kenya Economic Update: Policy options to advance the Big 4 - unleashing Kenya’s private sector to drive inclusive growth and accelerate poverty reduction (English). April 2018, Edition No. 17, World Bank Group 13 USD1.90 per day in 2011 PPP 14 Republic of Kenya (2018) Towards zero hunger: Strategic Review. May 2018 15 The Integrated Food Security Phase Classification System (IPC) categorizes acute food insecurity into five categories: IPC 1: Minimal/none, IPC 2: stressed, IPC: crisis, IPC 4: emergency and IPC 5: catastrophe/famine 16 FEWSNET (2019) Kenya Food Security Outlook Update: 3.1 million expected to face crisis (IPC3) or worse by October. August 2019. 17 Republic of Kenya (2018) Towards zero hunger: Strategic Review. May 2018 WFP Kenya Endline LRP Evaluation Report – May 2020 |Page 6 2020 Long rains assessment indicates timely onset of rains with average to above average rainfall forecast Locust plague affecting Ethiopia, Kenya and other parts of East Africa. The timing has not affected the current crop bit has potential to affect the next harvest (October) if not well controlled. 23. Nutrition: Recent nutrition surveys indicate that Turkana County has one of the highest burdens of malnutrition in the country, with acute malnutrition levels above emergency level in all four zones. 18,19 At the end of 2019, the nutrition status wasserious in Baringo North and South sub-counties where the Global Acute Malnutrition (GAM) rate was 9.3 percent, and critical in Tiaty sub-county (20.9%)20where the LRP is implemented. During 2018, West Pokot recorded an improvement in nutrition status of children aged 6 to 59 months, recording a GAM prevalence of 11% (serious) compared to 20.4% in 2017. 21 The evaluation was unable to find any recent nutrition information on school aged children. 24. National agriculture sector: The agriculture sector plays a vital role in the rural economy, contributing 26 percent of the GDP and another 27 percent of GDP indirectly through linkages with other sectors. The sector employs more than 40 percent of the total population and more than 70 percent of Kenya's rural people. 22 The sector also accounts for 65 percent of the export earnings and provides a livelihood for more than 80 percent of the Kenyan population. However, only about 20 percent of Kenyan land is suitable for farming, and even in arable areas, maximum yields have not been achieved, leaving considerable potential for increases in productivity. The sector is also the main driver of the non-agricultural economy including manufacturing, providing inputs and markets for non-agricultural operations such as building/construction, transportation, tourism, education and other social services. Agricultural policy reform is one the seven flagship projects to be implemented under Vision 2030,23 which outlines the Government of Kenya’s development direction. 25. Gender: In numerous aspects of life, gender inequalities are present in Kenyan society. The Gender Development Index (GDI) measures gender inequalities in achievement in three basic dimensions of human development: health, education and command over economic resources. The 2018 female GDI value for Kenya is 0.553 in contrast with 0.593 for males, resulting in a GDI value of 0.933, placing it into Group 3 (out of 5). The Gender Inequality Index (GII) reflects gender-based inequalities in three different dimensions: reproductive health, empowerment, and economic activity. Kenya has a GII value of 0.545, ranking it 134 out of 162 countries in the 2018 index. These rankings indicate that men still have greater access to education, have better health, and have more power over resources than women. In the last decade or so, the Government of Kenya has approved several gender-related policies24 to try and rectify gender imbalances in a range of sectors. In 2018, the United Nations Conference for Trade and Development (UNCTAD) noted that East African countries need to put better policies in place to address gender inequalities and bring women further into the workforce.25 The evaluation was not able to locate any specific contextual information on gender for the three LRP counties. 26. Education: The Government of Kenya re-introduced the policy of primary education free of all fees in 200326 with a view to achieving universal primary education and the goal of “Education for All” to correct the regional disparities, and social economic and gender imbalances in formal education. The Kenya 18 Republic of Kenya (2018) Turkana SMART Nutrition Surveys. June 2018 19 Turkana Central 17.2%, T. North 15.9%, T. South 16.2% and T. West 15.3%. 20 Republic of Kenya (2019) Baringo County: 2019 Long rains food and nutrition security assessment report. July 2019. 21 Republic of Kenya (2018) West Pokot County Integrated SMART Survey Report. June 2018. 22 http://www.fao.org/kenya/fao-in-kenya/kenya-at-a-glance/en 23 http://www.vision2030.go.ke 24 Including the National Human Rights Policy and Action Plan; The Marriage Act 2014; The Matrimonial Properties Act 2013; The Land Act 2016; and Draft National Policy on Gender and Development among others. 25 UNCTAD (2018) East African Community Regional Integration: Trade and Gender Implications. United Nations. 26 Free primary education was first introduced in Kenya in 1974 when the government at the time abolished the school fees for Standards 1 to 4. The elimination of school fees was extended to Standards 5 to 7 in 1978. Subsequently, it was reintroduced in 1979 and most recently in 2003. WFP Kenya Endline LRP Evaluation Report – May 2020 |Page 7 National Education Sector Strategic Plan (NESSP) 2018-202227 highlights the increasing number of primary schools and increased enrolment across the country as a result. On gender parity, the plan notes that government investment in primary education has resulted to improved parity index from 0.96 in 2013 to 0.97 in 2018. The completion rate of primary education has also increased considerably from 80 percent in 2013 to 84.2 percent in 2018, while the retention rate increased from 77 percent to 86 percent during the same period. 27. School meals programming: WFP and the MoE have jointly implemented a school meals programme in Kenya since 1980, targeting the most food-insecure counties with the lowest enrolment and completion rates and high gender disparities. Initially, school meals used an in-kind modality with food commodities provided directly by international donors including the USDA/McGovern-Dole. USDA/McGovern-Dole supported the school feeding programme in Kenya from 2004-2018. This long-term support enabled WFP to engage fully with the Government of Kenya on the benefits of school feeding, whilst developing the capacity of the government to take over ownership of the programme in a number of areas including; procuring food locally to stimulate local economies; raising awareness on the importance of education; building and rehabilitating school kitchens, storage and sanitation facilities; raising awareness on nutrition; and raising awareness on hygiene and sanitation. This capacity development support contributed significantly to the Government of Kenya being ready to take on the full responsibility for a nationally implemented school meal programme. After the 2018 handover to the Government of Kenya, USDA/McGovern-Dole ceased the provision of in-kind commodities but continues to support capacity strengthening interventions. 28. In 2009, the Government of Kenya started a national HGSMP to provide meals to children at school, using a cash transfer modality, providing cash to schools to enable them to buy their food directly from local traders and farmers. The HGSMP stimulated local agricultural production through purchase of food from smallholder farmers and local traders. The arid counties in Kenya’s north were the last to transition to cash-transfers, to give adequate time to assess the local markets and ensure they would be able to cope with the demand of school feeding. WFP and the MoE developed a transition strategy for the handover of the School Meals Programme to the Government of Kenya in 2012 with the schools in the arid counties joining the HGSMP from 2015. All the targeted LRP schools in Baringo and West Pokot were handed over to the HGSMP in September 2017 (see maps in Annex 7). WFP officially handed over all remaining WFP￾supported schools, including the LRP schools in Turkana to the government’s HGSMP in June 2018. This was on the understanding that the government would provide cash to schools to procure their own food for school meals. 29. School meals are provided as a hot lunch to schoolchildren comprising 150g of cereals, 40g of pulses, 5g of oil and 2g of iodized salt per child per day. According to the HGSMP Implementation Guidelines28, a variety of food items should be selected for the food basket and a combination of the different food groups should be consumed daily. Nutrition education should also be integrated. The government is committed to strengthening the school meals programme, aiming to provide at least one nutritious meal per school day. However, because of the 2017 drought, the MoE decided not to transition the targeted LRP schools onto cash-transfers (HGSMP) as planned, but to keep them on in-kind assistance until food prices reduced. This has had a significant effect on the LRP results as will be presented ahead. WFP continues to support the government in the nationwide roll out of HGSMP. Part of this support includes: • Facilitating the government’s mapping of locally grown, nutritious foods in target counties • Building the capacity of producers and traders to supply produce to schools under HGSMP 27 Republic of Kenya (2018) The National Education Sector Strategic Plan for the period 2018-2022. Ministry of Education. 28 Republic of Kenya (2016) Home Grown School Meals Programme Implementation Guidelines. World Food Programme. WFP Kenya Endline LRP Evaluation Report – May 2020 |Page 8 • Training the Ministry of Education and HGSMP procurement committees on enhancing food procurement systems and • Supporting schools to develop menus that utilize locally available nutrient-dense foods 30. Other WFP work and work of other development actors: WFP Kenya is currently implementing a Country Strategic Plan (CSP) (2018-2023), 29 with an overarching aim to shift WFP from the direct provision of transfers and services to strengthening of national systems and capacities to deliver food and nutrition security. The CSP supports the government’s ‘Big Four’ priorities30 including achieving 100 percent food and nutrition security and contributes to SDGs 2 and 17 (working in partnership). Over the LRP implementation period, WFP has also implemented humanitarian programmes in other locations to support refugee and host communities improve their food and nutrition security. WFP supports multiple refugee communities within Kenya, responding to displacement from Ethiopia, Somalia and South Sudan. Although Turkana County includes Kakuma refugee camp, that area is not a targeted LRP location. No displacement of refugee populations are found in either Baringo or West Pokot counties. WFP is the main international agency providing support in the three targeted LRP counties, although they collaborate with several other United Nations agencies to implement their programmes. These partnerships include FAO for supporting agriculture related programmes, UNHCR for refugee programmes and UNICEF to education, nutrition and WASH technical support. In Turkana County, the LRP follows on from WFP’s previous work with the MoA to support farmers and traders, including the now completed Purchase for Progress (P4P) and the Agriculture, Market Access and Linkage (AMAL) project that is ongoing in Turkana County. 1.3 Evaluation methodology and limitations 1.3.1 Approach and methodology 31. This final activity evaluation used the OECD-DAC31 international evaluation criteria of relevance, effectiveness, efficiency, sustainability and impact to evaluate the implementation of the LRP project. Given this is a final evaluation, greater emphasis has been put on effectiveness, efficiency, sustainability and impact of the project. The evaluation team also ensured that gender equality and the empowerment of women (GEEW) was mainstreamed through the evaluation approach and methodology by ensuring that whenever possible, men and women, boys and girls from different stakeholder groups, participated in the evaluation, and data has been gender disaggregated when possible. Where possible, the evaluation has also tried to assess whether the project had different impacts on male and female farmers and traders. 32. As per the USDA-WFP agreed Evaluation Plan, this evaluation provides an evidence-based, independent assessment of the performance of the programme. Specifically, the final evaluation will: • Review the project’s relevance, effectiveness, efficiency, impact, and sustainability. • Collect additional data as appropriate for evaluation of the strategic objective and higher-level results. • Assess whether or not the project has succeeded in achieving its objective. • Investigate the project’s overall impact, and • Identify meaningful lessons learned that WFP, USDA, and other stakeholders can apply to future programming. 33. The evaluation scope covers all the activities implemented as part of the LRP in each of the three targeted counties. The endline evaluation builds on the 2018 baseline assessment so that endline and baseline findings can be compared, as well as results from LRP and non-LRP schools. Together these comparisons 29 WFP Kenya (2018) Kenya Country Strategic Plan 2018-2023. WFP/EB.A/2018/8-A/3 30 Increasing manufacturing, achieving universal healthcare, expanding affordable housing and achieving 100 percent food and nutrition security 31 Organisation for Economic Co-operation and Development’s Development Assistance Committee WFP Kenya Endline LRP Evaluation Report – May 2020 |Page 9 enable evaluation of the effectiveness and impact of the programme. The evaluation has used mixed methods and collected both quantitative and qualitative data to answer each of the evaluation questions. All information collection has been directed towards providing situational analysis and endline figures for the LRPs PMP and providing information on each element of the LRP Results Framework. Overall, the evaluation has tried to assess the project’s contribution to the LRP strategic objective: ‘Improved effectiveness of Food Assistance through Local and Regional Procurement’. The evaluation has also utilized WFP Kenya’s own project monitoring data, to complete the PMP required by USDA. The evaluation has assessed the effect of the program against the following objectives: • Improved cost effectiveness of food assistance • Improved timeliness of food assistance • Improved utilization of nutritious and culturally acceptable foods that meet quality standards 34. The OECD-DAC criteria have been used as the basis of the key questions for the evaluation. Table 5 outlines some of the key questions under each of these criteria as per the ToR. The table has been expanded into a full evaluation matrix, which can be found in Annex 8. The evaluation matrix was developed during the baseline’s inception phase, with a view to utilizing the same matrix for this endline evaluation. The evaluation team has therefore made no changes to the original matrix. Overall, the evaluation matrix systematically sets out how each evaluation question will be addressed, ensuring the evaluation design is transparent and that findings can be linked to the evaluation questions through appropriate methods, analysis and data. The evaluation matrix also forms the analytical framework of the evaluation. Table 5: Preliminary Key Evaluation Questions Focus Area Final Evaluation Key Questions Relevance • Was the project designed to reach the right people with the right type of assistance? • Did the project’s implementation lead to meeting the intended beneficiaries’ needs considering gender differences with the right mix of assistance? • Is the project aligned with national government’s relevant policies and strategies? • Did the project complement other donor-funded and government initiatives? Effectiveness • To what degree have the interventions resulted (or not) in the expected results (outputs and outcomes as per the PMP)? Efficiency • Did assistance reach the right beneficiaries at the right time? Impact • Were there any unintended outcomes, either positive or negative? • What internal and external factors affected the project’s results? Sustainability • To what extent is it likely that the benefits of the project will continue after the end of the project? • Did the outcomes benefit boys, girls, women and men equally? • What are the key factors that affect the likelihood of sustainability of the results of the project? • What gender aspects should be considered to ensure gender equality and women empowerment in LRP related interventions are sustained? General • What are lessons learned from the project? • How can WFP improve future programming, in the context of these lessons learned? • How can the lessons be applied to benefit boys, girls, men and women equally whilst considering their gender differences? 1.3.2 Data collection methods and tools 35. The evaluation used three main data collection methods: Secondary data review, quantitative data collection, and qualitative data collection. Both quantitative and qualitative data collection were carried out in all three LRP counties. Detailed descriptions of the data collection methods can be found below. WFP Kenya Endline LRP Evaluation Report – May 2020 |Page 10 36. Secondary data review: WFP Kenya has made a number of secondary documents available to the evaluation team including project documents, LRP monitoring data, and relevant background documents. These documents have been used as secondary data and additional documents have also collected during the fieldwork. The full list of documents and data provided to the evaluation team by WFP Kenya can be found in Annex 9. 37. Quantitative data collection: The quantitative data collection was done between Feb 10-28, 2020 using three assessment tools: a survey of traders, a survey of FOs, and a survey of schools (SMC Member, Head Teacher or teacher responsible for the school meals). Although the targeted schools are the primary unit of analysis, inclusion of the FO and trader surveys will help provide a basis of understanding of the state of the local market and the linkages between small holder farmers, traders and schools. Each of the three assessment tools are the same as the ones used at baseline, to enable direct comparison of results over time. This comparison of results has helped provide an understanding as to why the intended results have been achieved in the schools. Each of the quantitative tools can be found in Annex 10. The evaluation team and a team of independent enumerators were responsible for collecting all the required endline evaluation data. A brief summary of the areas covered by each tool is provided below: • Trader/supplier survey: Ability of traders to participate in tenders, issues with food procurement and linkage to small holder farmers [Related to A5, LRP 1.3.2.3, LRP 1.3.2 and LRP 1.3.3 of the RF] • FO survey: Ability to produce drought tolerant crops and the value and volume of sales to schools [Related to A5, LRP 1.3.2.3, LRP 1.3.2 and LRP 1.3.3 of the RF] • School survey: Timeliness and cost-effectiveness procurement of food commodities for school meals, utilization of nutritious and culturally acceptable food. [Related to LRP 1.1, LRP 1.2 and LRP 1.3 of the RF]. 38. Trader and FO surveys: The questionnaire that was used for these two surveys was originally designed by the WFP team for programme monitoring purposes and was used to collect data at baseline on all FOs and traders registered at the time. Since the questionnaire is larger than required for evaluation purposes, the evaluation team collected that whole data set but have presented only the same sections of the questionnaire that were analysed at baseline as they are the most relevant to answering the evaluation questions: Section B: Production, Section D: Marketing, Section F: Training. The remaining data will be used by the WFP team for further analysis and reporting. 39. School survey: At baseline, the school survey included questions specifically for the measurement of the LRP PMP, using Term I, 2018 as the reference point. It also included observation of the presence of food for the start of Term 2, 2018. The same schools were surveyed again at endline but with a focus on Term 3, 2019 and observation on Term 1, 2020 to see the difference that two full school years of LRP support has made. 40. Quantitative data cleaning and checking: A code book of the key variables was produced at baseline, and used again at endline, to check on data distribution and flag any outliers for follow up. Data entry was standardized using the ODK platform to minimize errors that occur when using post-data collection entry approach as well as reduce the risk of duplication. Data errors were mitigated by building consistency checks within the ODK platform thereby enhancing data accuracy. Data cleaning was also automated within the ODK platform. The data was therefore collected, entered and stored real time using tablets. 41. Quantitative data analysis methods: Data analysis follow the same methodologies used at baseline. That included analysis using STATA version 14, SPSS version 24 and MS Excel. The analytical path used a funnel approach through the following progressive steps: descriptive statistics; association analyses as well as significance testing; and computation of indices. Specific analyses include the following: Levene's test to measure the equality of variances between LRP and non-LRP schools as well as between the different WFP Kenya Endline LRP Evaluation Report – May 2020 |Page 11 counties; Chi square test of independence to evaluate the statistical significance of the relationship between categorical variables based on the corresponding p-values. For instance, Chi-Square tests of independence were carried out to test the statistical difference between LRP and non-LRP schools as well as differences within counties in the provision of lunch. As the number of traders sampled is small (n=29), non-parametric tests have been used for data analysis that are free from homoscedasticity assumptions.32 42. The endline results of the project have been computed using multivariate statistical analyses. This approach helped in computing the performance of the intervention while controlling for both direct and indirect factors that may explain the variance in the intervention at the endline relative to baseline. At baseline, the evaluation team constructed three indices/scores using multiple variables around the three constructs that correspond to the three main outcomes of the RF: improved cost-effectiveness, timeliness and nutrition. These indices were then summed to give the unit for overall measurement at school level: Impact Score. These same indices have been calculated at endline, to make direct comparison. Details on the calculation of these scores can be found in Annex 11. 43. Qualitative data collection: Qualitative data collection was done from February 17 – March 5, with follow up interviews done remotely by Skype. As per baseline, the Team Leader wasresponsible for the collection of qualitative data using the semi-structured interview guides found in Annex 12. These are the same ones used at baseline and were used as a starting point for interviews and as a guide to ensure all topics are covered. The full evaluation mission schedule can be found in Annex 13. 1.3.3 Ensuring evaluation quality 44. Ethics: This evaluation followed the UNEG ethical norms and standards33 to ensure appropriate safeguarding and ethical processes were followed throughout the evaluation. This included obtaining informed consent from key informants by explaining the purpose of the interview and that any information they provided would be kept confidential, and their name would not be attributed to statements. The evaluation team also ensured that interviewees were comfortable with the time and location of interviews, including conducting them outside as appropriate. No children were interviewed for the evaluation, so it was not necessary to include child safety measures. 45. Data availability and reliability: The reliability34 of research findings is particularly important in a study with multiple windows of observation35 like the LRP evaluation. The evaluation team therefore tested the reliability of data collection tools during the baseline assessment36 to assess the internal consistency of the research tools. Since the same data collection instruments were used at endline, the evaluation team is confident that the results are still reliable. Confirmation analysis wasrepeated to validate data reliability during the endline analysis phase. Regarding data availability, WFP Kenya provided the evaluation team with the full set of LRP programme monitoring data. Where possible, the evaluation team verified the results during qualitative interviews with WFP personnel, government representatives and school personnel. This helped to ensure that this secondary data is reliable. The evaluation report also includes gender disaggregated data where possible. The evaluation team is aware that WFP has actively tried to engage women’s FOs in the project, so the evaluation team has tried to assess whether women’s groups received different benefits from the project than men’s or mixed groups. However, the same disaggregation was not possible for the trader data, as Baringo only has three traders, all of whom are women. West Pokot trader data has been gender disaggregated where relevant. The team also made all 32 Kendall's tau_b 33 United Nations Evaluation Group (2016). Norms and Standards for Evaluation. New York: UNEG. 34 Reliability relates to the degree to which research can be repeated while obtaining consistent results. It is therefore essential that the validity and the reliability of the data collection tools is ascertained to guarantee consistence in research results at different windows of observation. 35 Baseline and endline 36 By using the Cronbach alpha. The instruments attained an alpha score of 0.8 which was above the required threshold of an alpha of 0.7. WFP Kenya Endline LRP Evaluation Report – May 2020 |Page 12 possible efforts to ensure both male and female government representatives and school personnel were included in the evaluation. 46. Data quality: The evaluation team made every effort to ensure the quality of data (validity, consistency and accuracy) was high throughout the analytical and reporting phases. This was done by conducting rigorous checks on the quantitative data and triangulating all findings, including methodological and data triangulation. As per the ToR, the evaluation team was provided with all relevant documentation within the provisions of the directive on disclosure of information. This ensured that findings are based on as many data sources as possible, to give a rounded conclusion. In addition, no WFP staff were present in evaluation interviews so that independence of findings, and open communication was had between the evaluation team and the key informants. 1.3.4 Site mapping 47. Surveyed schools: The endline evaluation used the same sample of schools as the baseline assessment. This enabled direct comparison of results to see the change over time. At baseline the original sample of 192 schools was selected by clustering schools with similar characteristics.37 This technique was used to match LRP and non-LRP schools with similar spatial and demographic attributes. More information on the matching method used can be found in Annex 14 and the full list of LRP and non-LRP schools surveyed can be found in Annex 15. A more detailed description on how the sampled schools were originally selected and matched can be found in Annex 16. The endline sample was 191 schools after one school in West Pokot could not complete the school questionnaire because there was no teacher available to answer the questions (Table 6). Table 6: Number of LRP and non-LRP schools surveyed at endline evaluation, by county 48. Surveyed Farmer Organizations: At baseline, the WFP Kenya LRP team completed an extensive survey of all the FOs and local suppliers participating in the project at the time (n= 98). The evaluation team utilized the data as a secondary source and conducted its own analysis of the data. For the endline evaluation, there were 78 FOs still participating in the project and all 78 were surveyed. 49. Surveyed local traders: The evaluation team completed data collection on all traders supported by the project in Baringo and West Pokot (Table 7). No traders were supported in Turkana. Table 7: LRP Targeted traders included in the evaluation survey 50. Qualitative data collection: Key informant interviews and focus groups discussions were done in each of the three target counties as well as in Nairobi and remote interviews carried out over telephone. Interviews focused on personnel from WFP, USDA, MoE, MoA and MoH representatives at national and county level, FO representatives, traders, school personnel and others as appropriate. The results of the 37 Using the Silhouette measure of cohesion and separation County LRP Schools Non -LRP Comparison Schools Total Baringo 29 29 58 (30.2%) Turkana 35 33 68 (35.4%) West Pokot 32 33 65 (34.4%) TOTAL 96 95 191 (100%) County Male Female Total Baringo 0 3 3 West Pokot 12 13 25 TOTAL 12 16 28 WFP Kenya Endline LRP Evaluation Report – May 2020 |Page 13 qualitative data collection have been used both as primary data and used to triangulate and elaborate on the quantitative data collection. In total, 118 key informants and focus group discussion participants have provided their input. The full list of key informants is in Annex 17. 1.3.5 Limitations and risks • Two schools from the original baseline sample were not included at endline.38 These two schools were replaced with schools with 99% matching characteristics39 to the original schools to limit potential differences between the baseline and endline samples. • The school survey was partially conducted during the Term 1, 2020 half term holidays. This meant the survey teams had to organize times with each school to conduct their visit, to ensure that appropriate school personnel would be available to answer the survey questions. The mitigation measures were successful, and personnel from each school were found to participate in the survey. • Although there is some gender disaggregated data available from WFP, the LRP Results framework does not include gender indicators. Furthermore, the beneficiary units are largely genderless (schools and farmer organizations). This has limited the gender-related findings of the evaluation. 2 Evaluation Findings 2.1 Overview of surveyed beneficiaries 51. Before reporting the findings, this section presents a short overview of the three beneficiary groups that were surveyed as part of the evaluation: traders, farmers (FOs), and schools. The information is from the endline quantitative survey. • Farmer Organizations 52. At endline, there were 79 FOs registered with the LRP. These 79 FOs represent 39,682 members, with the FOs in Turkana being the largest with an average membership of 776 members (Table 8). Women constitute more than half the members in all three counties, with Baringo FOs reporting the largest percentage of female members (72%) and the largest percentage of active members (92.6%). In Baringo and Turkana, most FOs had been established for more 5 years, with more than a third of FOs in Turkana (37.1%) being established for more than 20 years. The FOs in West Pokot were formed more recently, with most (75%) being established for less than 10 years (Figure 1). Table 8: Characteristics of surveyed Farmer Organizations, by county. Baseline Endline Baringo Turkana West Pokot Baringo Turkana West Pokot Number of surveyed FOs 46 36 16 23 35 21 Membership Total members 978 31,142 424 11,895 27,165 622 Average number per FO 21 865 26.5 517 776 30 Average % female 74% 56.8% 66.3% 72.0% 61.5% 65.3% 38 One because a bridge had been destroyed by floods and the school was no longer accessible, and the second due to insecurity. 39 The replacement schools were matched using the following attributes: Grouping variable (LRP or non-LRP); County; enrolment levels; the original selection anomaly index, peer ID, and peer size using the silhouette measure of cohesion and separation algorithm. WFP Kenya Endline LRP Evaluation Report – May 2020 |Page 14 Figure 1: Average time the FOs have been established • Local traders 53. At endline, WFP identified 28 local traders from Baringo and West Pokot participating in the LRP. Most of these were located in West Pokot, with only three traders identified in Baringo; all female. Women constitute more than half the LRP traders (57.1%) (Table 9). A small percentage of traders (14.3%) reported having tertiary education,40while the majority had only primary (46.4%) or secondary (39.3%). Table 9: Gender of surveyed traders supported by the LRP Baseline Endline Baringo West Pokot Total Baringo West Pokot Total Number of traders Total 13 41 54 3 25 28 Gender Male 7 22 29 (53.7%) 0 12 12 (42.9%) Female 6 19 25 (46.3%) 3 13 16 (57.1%) 54. Table 10 shows that the majority of surveyed traders have been in business for around 10 years, with an average of three employees, one more than at baseline (p= 0.0480). All three Baringo traders had a valid business license by endline, compared with 76 percent of traders in West Pokot. In both counties, these percentages had increased from baseline, but the difference was not statistically significant (p= 0.3456). The proportion of traders with business registration certificates in both counties was still low. Table 10: Business information of surveyed traders, by county 55. The evaluation found no significant difference between male and female traders in West Pokot in terms of average time in business, 41 the number of employees,42 or in the percentage having a valid business licence.43 The main difference was the percentage with a business registration certification, with more female traders (62%) having one, compared with the male traders (33%). However, this difference is not statistically significant (p = 0.1670) 44. 56. The majority of traders reported using lorries or motorcycles to transport their commodities for sale, and most (66.7 % in Baringo, and 80% in West Pokot) use permanently rented warehouses for storing their commodities. Only 1 trader in Baringo and 4 (16%) traders in West Pokot owned their warehouse. 40 Diploma or degree level education 41 West Pokot male traders = 11 years; female = 12 years 42 West Pokot male traders = 2 employees; female = 3 43 West Pokot male traders =75% with valid business licence; female = 77% 44 Determined using a non-parametric test which is consistent with t-test p- value of 0.1719. 0% 50% 100% Baringo Baseline Baringo Endline Turkana Baseline Turkana Endline West Pokot Baseline West Pokot Endline <= 5 years 6-10 years 11-15 years 16 – 20 years 21+ years Baseline Endline Baringo West Pokot Baringo West Pokot Average time in business 6 years 7 years 9 years 11 years Average number of employees 2 2 3 3 Percentage of traders with a valid business license 92.3% 58.5% 100% 76% Percentage of traders with a business certificate 53.8% 26.5% 33.3% 48% WFP Kenya Endline LRP Evaluation Report – May 2020 |Page 15 • Schools 57. Table 11 shows that the majority of surveyed schools were day schools, with several being both day and boarding. Most schools had an average enrolment of 300-400 students. Table 11: Characteristics of surveyed schools, by county 2.2 Evaluation Theme 1: Relevance 2.2.1 Was the project designed to reach the right people with the right type of assistance? 58. WFP has been working with the MoE on implementing and improving the design of the school meals programme in Kenya for decades. When the LRP was designed in 2017, the MoE played a significant role in the design, including agreeing on the selection of the target counties, and the modality of cash transfers. The three LRP counties were targeted as they are the last three counties planned to transition to a cash￾transfer modality. These decisions aligned with the existing HGSMP, as the government took over implementation in the LRP sub-counties in September 2017. The intention of LRP design was to support the HGSMP implementation by ensuring that schools were able to purchase the commodities for school mealsfrom the local market once they had received cash-transfers from the MoE. This would help support the local economy and align with an underlying intention of the HGSMP and align with the government preference of using a cash-transfer modality; a preference expressed since before 2009 when school meals started transitioning to local ownership and local procurement. 59. The evaluation team (ET) conducted qualitative interviews with several government personnel and beneficiary groups including FOs and traders to establish the relevance of the LRP project. All interviewed stakeholders without exception, expressed support for the HGSMP and providing cash to schoolsto enable local procurement. Stakeholders believed this was the best modality as it supported the local economy by providing a stable market for local farmers and traders. Further, stakeholders felt that that the government did not have the capacity to provide commodities to schools on a timely basis, citing long procurement processes, lack of logistic capabilities, and poor warehousing and transport facilities at county level. 60. Stakeholder also agreed that the activities included in the LRP were highly relevant, including value chain assessments (VCA) to help establish the minimum cost required to provide school meals to students in each county, and nutrition-related activities to promote a wider range of food commodities into the school meal menu while promoting locally grown, drought-tolerant crops. Providing training to FOs and traders on the procurement process of the HGSMP was also much appreciated by stakeholders, to enable the local market players to have a better chance at winning the tenders for school meals. The only activity that received mixed reviews was the forward delivery contracts (FDCs) in Turkana. It was acknowledged by stakeholders that the WFP FDCs supported the FOs to provide food in schools, however this does not align well with the idea of schools doing their own procurement. The evaluation found the LRP activities to be well linked, working together to reach the overall objective of the project. Key informants also agreed that Type of school Baringo Turkana West Pokot Total LRP Non- LRP LRP Non- LRP LRP Non- LRP LRP Non- LRP Day school 19 23 31 28 7 16 57 67 Boarding school 0 0 1 0 2 0 3 0 Day + Boarding 10 6 3 5 23 17 36 28 Average enrolment 313 224 404 320 337 383 354 312 Total number of schools 29 29 35 33 32 33 96 95 15.2% 15.2% 18.3% 17.3% 16.7% 17.3% 100% WFP Kenya Endline LRP Evaluation Report – May 2020 |Page 16 the targeting of FOs and traders was also appropriate – through voluntary inclusion after WFP and the MoITC providing information sessions in each county about the project. 61. Stakeholders appreciated the effort made by WFP to include women in the LRP, bringing GEEW elements into the LRP design. Women traders and FOs with women-only membership have been actively identified and encouraged to participate in all aspects of the programme, with women making up almost half the traders sampled (46.3%) and more than half the FO membership. Overall, 15 percent of the surveyed FOs were women-only organizations, with FO membership in all three counties dominated by women.45 This is particularly the case in Turkana County, where WFP has been supporting farmers and FOs for several years. In Turkana 59 percent of FO members and 48 percent of FO leaders are women. Participating traders are also mainly women – all three of the traders in Baringo and 52 percent of the West Pokot traders. This active inclusion of women was intended to provide women traders and FO members with increased economic opportunities and to support women to participate in the traditionally male￾dominated trade sector. 2.2.2 Is the project aligned with national government’s relevant policies and strategies? 62. WFP has been a key partner of the Government of Kenya for many years, supporting policy and strategy development in several sectorsrelevant to school feeding. In recent years this has included supporting the development of the National School Meals and Nutrition Strategy (NSMNS) (2017-2022)46 that outlines the design and implementation of nutrition-sensitive school meals in Kenya. Among other objectives, the NSMNS highlights the need to “increase awareness and intake of locally available and nutritious foods among school children and their communities,” and aims to “strengthen governance and accountability in implementing school meals and nutrition programmes.” The LRP aligns well with both these objectives. The NSMNS also stipulates that the school meal should increase nutritional intake and meet healthy food consumption habits, and that the procurement of commodities should also be procured directly from local supplies where possible. Like the LRP, the NSMNS is a collaboration between the MoE, the MoA and the MoH, signifying the start of greater collaboration between these ministries on the HGSMP. The LRP is also an opportunity for increased involvement of the county level nutrition teams into the HGSMP. This is appropriate and aligns with the handover to the Government of Kenya. 63. The nutrition aspect of the LRP, developing more diverse school meals menus aligns with several MoH guidelines and strategies including the National Guidelines for Health Diets and Physical Activity. 47 Interviews with MoA personnel at national and county levels indicate that the LRP also strongly links to existing and upcoming MoA strategies, polices and frameworks including the Agricultural Sector Transformation and Growth Strategy,48 National Agribusiness Strategy49, the Agricultural Sector Development Strategy, 50 and the National Food Security and Nutrition Policy.51 The LRP also aligns with the MoA’s ongoing work to promote local and drought tolerant crops, and with work supporting local farmers. In turn this links to the planned national implementation framework supported by WFP under the LRP to guide and direct procurement by public institutions from smallholder farmers. The active 45 Percentage of female members in mixed membership FOs = Baringo 72%, Turkana 61% and West Pokot 65%. 46 Republic of Kenya (2018) National school meals and nutrition strategy 2017-2022. Ministry of Education, Ministry of Health and the Ministry of Agriculture and Irrigation. 47 Republic of Kenya (2018) Kenya food composition tables. The Food and Agriculture Organization of the United Nations, The Ministry of Health, Republic of Kenya and The Ministry of Agriculture and Irrigation. 48 Republic of Kenya (2019) Agricultural Sector Transformation and Growth Strategy: Towards sustainable agricultural transformation and food security in Kenya (2019-2029). Abridged Version. Ministry of Agriculture and Irrigation. 49 Republic of Kenya (2012) National Agribusiness Strategy. Agricultural Sector Coordination Unit (ASCU). 50 Republic of Kenya (2009) National Agricultural Sector Development Strategy 2009-2020. 51 Republic of Kenya (2011) National Food Security and Nutrition Policy. Agricultural Sector Coordination Unit (ASCU). WFP Kenya Endline LRP Evaluation Report – May 2020 |Page 17 inclusion of women in the LRP also aligns with the new National Policy on Gender and Development52 which aims to achieve gender equality with equal access to opportunities in all spheres of life. 64. Kenya’s Public Procurement and Asset Disposal Act 2015 Part XII, Preference and Reservations Regulations 2011 provide a framework for the implementation of preferential procurements in Kenya’s public procurement. The regulations currently allow preferential procurement from special groups: youth, women and persons with disabilities. Under these regulations, public entities should reserve at least 30 percent of procurement budgets to special groups. The MoA has been exploring the possibility of including smallholder farmers into the ‘special groups’ included in the preference procurement scheme. 2.2.3 Did the project complement other donor-funded and government initiatives? 65. The LRP aligns with the new WFP Kenya CSP, based on the National Zero Hunger Strategic Review53 and aligned with the Government of Kenya’s Vision 203054 and Third Medium-Term Plan55. The CSP focuses on food systems through the development and modelling of solutions along the food production, transformation and consumption chain that can be scaled up by the government and the private sector. The CSP also aligns with the United Nations Development Assistance Framework for Kenya56 which cites three strategic priority areas, all aligned to government direction: transformational governance, human capital development and sustainable and inclusive growth. 66. The LRP is also coherent with previous work that WFP has done in collaboration with the MoA to support farmers and traders, including the now completed Purchase for Progress (P4P) and the Agriculture, Market Access and Linkage (AMAL) project that is ongoing in Turkana County. LRP also links to WFP’s Retail Engagement Initiative, 57 which works to strengthen markets around refugee camps in Turkana County, promoting smallholder procurement. 67. Although the LRP marks the start of WFP support for farmers in Baringo and West Pokot, it links closely with asset creation projects implemented in recent years in those counties. Some of the FOs in Turkana have also been supported with farming equipment from complementary funds from the European Union Trust Fund (EUTF) and/orfrom the Federal Ministry of Economic Cooperation and Development (Germany) (BMZ). Other donors have also supported various training activities or the production of training materials. These include the Canadian Government, Unilever, and government funds from the MoE, MoA and MoH. Summary of key findings: Relevance • The LRP was designed in collaboration with MoE, MoA and MoH and the evaluation found the design to be relevant to support HGSMP transition in the three targeted counties. • The planned modality of cash-transfers for providing school meals was the preferred option of all evaluation key informants, as it not only supportslocal farmers and trader, but aligns well with the government’s existing HGSMP. • The LRP is well aligned with MoE, MoA and MoH policies and their current activities and directions. • Overall, the endline evaluation found that the LRP complements the HGSMP, and aligns well with key Government of Kenya policies, strategies and framework, as well as WFP’s own polices and with the direction of other United Nations actors in Kenya. 52 Republic of Kenya (2019) Sessional Paper No.2 of 2019 on National Policy on Gender and Development. Towards creating a just, fair and transformed society free from gender-based discrimination in all spheres of life practices. Ministry of Public Service, Youth and Gender. 53 Republic of Kenya (2011) National Zero Hunger Strategic Review. May 2018 54 Government of Kenya’s Vision 2030: https://vision2030.go.ke 55 Republic of Kenya (2018) Third Medium-Term Plan (2018-2022): Transforming Lives: Advancing socio-economic development through the “Big Four”. National Treasury and Planning. Nairobi http://extwprlegs1.fao.org/docs/pdf/ken189643.pdf 56 United Nations Kenya (2018) United Nations Development Assistance Framework (UNDAF) for Kenya, 2018-2022. Nairobi 57 WFP Kenya (2018) The Kenya Retail Engagement Initiative: Getting more value from Bamba Chakula cash transfers. Infobrief No. 17. Kenya WFP Kenya Endline LRP Evaluation Report – May 2020 |Page 18 2.3 Evaluation Theme 2: Effectiveness 68. This section details the effectiveness of the LRP project starting with a review of the planned activities, then the outcomes that have resulted. Overall, WFP has implemented the LRP project activities as planned in the original agreement with USDA. However, it is important to remember that the LRP project was designed on the agreement from the MoE that they would be providing cash transfers to schools to try and improve the cost-effectiveness and timeliness of procurement and delivery. The effectiveness of the LRP has therefore been compromised by the MoE providing in-kind support to all schools in the three LRP counties, throughout the whole project period. Although this decision is within the HGSMP guidelines, it does not align with the implementation purpose of the LRP as it has prevented local farmers and traders from having any opportunity to supply food to the primary schools as intended. The unintended outcomes that have resulted from the government’s decision to implement that modality are discussed ahead in Section 2.5: Impact. 69. The evaluation found that it was not possible for WFP to have foreseen the sudden change in position on cash-transfers by the MoE. All the technical specialists from the MoE, MoA, MoH and MoITC that participated in the evaluation believed that the HGSMP modality of cash-transfers was a better model to pursue than in-kind provision and had been expecting the cash-transfers to be implemented as planned. At baseline, it was understood that the central government decided to provide in-kind commodities for Term 1, 2018 due to the 2017/18 drought which resulted in poor availability of food commodities and high food prices in arid areas. However, the evaluation was unable to establish why in-kind provision has continued throughout the LRP period despite the findings of market assessments described ahead. The most likely reason seems to be a change of senior leadership in the MoE with a preference for in-kind support. 2.3.1 To what degree have the interventions resulted (or not) in the expected results (outputs and outcomes as per the PMP)? 70. The following key has been used to highlight the survey results and/or the achievement of LRP targets: Improvement since baseline and/or LRP target reached Similar result to baseline Deterioration since baseline and/or LRP target not reached Activity and output level results 71. As noted earlier in Table 1, the LRP includes five main activities, the results of each are described below. • Assessment of local food systems 72. WFP and partners have conducted several market-related assessments in the target counties over the LRP period. This includes food supply chain assessments in Baringo58 and West Pokot59 in September 2018 to establish the extent of functioning of the local markets. The aim being to determine the capability of local traders to supply the schools under the HGSMP. These food supply assessments also provided estimates of the required cash transfer rates for the HGSMP in each county based on the prevailing market prices. The assessments found that values between KES 10-13 per child per day would be appropriate in both counties depending on the distance from main markets. It was intended that the MoE would use these values to ensure that schools received adequate cash resources to procure sufficient food for the HGSMP. In July 2019, WFP also conducted value chain assessments of traditional nutritional foods in Baringo60 and 58 Kisingu, J. & Mutuku, J. (2018) Food Supply Chain Assessment Report: Baringo County 59 Kisingu, J., Mutuku, J. & Kithale, D (2018) Food Supply Chain Assessment Report: West Pokot County 60 WFP (2019) Traditional Nutritional Foods in Baringo County: Value Chain Study Report WFP Kenya Endline LRP Evaluation Report – May 2020 |Page 19 West Pokot61 to assess the viability of promoting the production and marketing of locally produced nutritious foods. 73. Together, these assessments found that the local markets in each for the three LRP counties would be able to meet the demand from the school meals programme. Although the climatic conditions for producing the four promoted drought-tolerant crops (sorghum, cowpeas, green grams and millets) was good, their consumption was constrained due to low volumes produced (lack of availability) and high prices when compared to maize and beans. Conducting these assessments provided an opportunity for collaboration and capacity building for WFP and MoA and the findings were particularly appreciated by WFP project personnel and by county-level authorities, for providing evidence that the markets in each county was able to meet the added demands of school meals, despite the drought and floods that occurred during the project period. The assessments also enabled WFP and technical county-level representatives, particularly from MoA, to continue to promote market-based approaches, despite growing resistance from MoE senior leadership. • Procure locally produced, drought-tolerant crops 74. In an effort to develop the local market, the WFP-USDA agreement included provision for WFP to procure 80 mt of cowpeas and 288 mt of sorghum from local farmers in Turkana County to support the school meals programme. The intention was to strengthen local and regional food market systems and improve access to culturally acceptable foods while connecting local farmers and FOs to the HGSMP. The market assessments described above had identified that drought-tolerant varieties were produced in low volumes, making them too costly at point of sale. It was therefore hoped that contracting farmers and FOs though forward delivery contracts (FDCs) would help create a demand that would help to increase production, ultimately making the commodity more affordable. 75. Five FOs were selected for FDCs based on their production capacity, their agreement to WFP’s price, and the presence of the required documentation to comply with WFP’s procurement process. The original FDCs were to supply food for school meals for 20 days per term during Terms 1 and 2, 2018 (two days per week). This aligned with the start of the planned transition for the LRP schools to HGSMP. Unfortunately, as a result of the 2017/18 drought, sorghum production was low, and the FOs were unable to supply sufficient quantities of commodities. WFP therefore provided 1,275 farmers with early maturing cowpea seeds (using complementary funding from BMZ) to support the next planting season. FOs were then able to supply almost their full contract volume by October 2018 with additional FDCs then drawn up to provide additional sorghum for 2019. Table 12 shows that the volume of commodities acquired through FDC in 2018 was 133.8mt of sorghum and 79mt of cowpeas, with a further 136.8mt being delivered in 2019. In total, FOs have succeeded in delivering the quantities they were contracted for, 95.5% of the original planned quantity at a value of USD 147,655. Table 12: Planned vs actual delivery of food commodities acquired by WFP though forward delivery contracts (Turkana only) 61 WFP (2019) Traditional Nutritional Foods in West Pokot County: Value Chain Study Report Total planned volume (mt) Actual (mt) Achievement Delivered 2018 Delivered 2019 TOTAL Cowpeas 80 79 - 79 98.8% Sorghum 288 133.8 136.8 270.6 93.4% Total 366 212.8 136.8 349.6 95.5% Total value of commodities (USD) 103,725.26 43,930 147,655.26 WFP Kenya Endline LRP Evaluation Report – May 2020 |Page 20 76. The FDCs provided local FOs with a known market for their commodities. These commodities were then directly provided to LRP schools by WFP to provide school meals. As a starting point to introduce FOs to the idea of growing sorghum and other drought-tolerant crops for commercial purposes, the FDCs were successful. When coupled with the training on the HGSMP described ahead, FDC winning FOs members should now have a clearer picture of opportunities, demands and processes that a cash based HGSMP can bring. • Capacity building for national and county institutions 77. The LRP has provided several opportunities for capacity building for national and county institutions. Aside from working with the MoA on market assessments as described earlier, the WFP-USDA agreement includes two main areas of capacity building: 1. Working with the MoE to train schoolteachers, parents, and education officers in Baringo and West Pokot on the HGSMP management and the school-based food procurement process, along with developing a monitoring and oversight plan for the HGSMP in Baringo and West Pokot Counties; and 2.Support the development of the Government of Kenya’s national implementation strategy to guide direct local procurement from smallholder farmers by government institutions.62 78. Capacity building on HGSMP: WFP supported the MoE to undertake HGSMP management training in Baringo and West Pokot Counties for government representatives and for school personnel including cooks, teachers, and school directors. Table 13 shows that 721 people (19% women) participated in the training designed to equip school level implementers with the necessary skills to ensure effective management of the HGSMP. The training covered several modules including overall HGSMP management, food procurement procedures, roles and responsibilities of various stakeholders, food quality, and nutrition and hygiene promotion among others. As per the baseline recommendations, the training included food quality and safety strategies for school feeding programmes (including aflatoxin testing). The evaluation found that the training, included relevant topics and was set at an appropriate level for the participants and that all the topics were relevant. Table 13: Number of participants attending training on HGSMP management (2017-2018) 79. In addition to the HGSMP training, WFP and MoE conducted regular monitoring visits to schools. During those visits, data was collected on record and stock management and distribution. Although there has been some progress made on the frequency of school level monitoring, qualitative interviews indicate that the government does not employ sufficient resources to make project monitoring a priority, with government personnel hampered by lack of transportation to visit project sites. 80. Capacity building on smallholder procurement: For several years, WFP has working with the MoA to promote procurement from small holder farmers. WFP and MoA have been advocating that the national government revise their procurement guidelines to explicitly require some level of public procurement from smallholder farmers. To this end, WFP, together with the MoE, MoA, MoH and FAO,63 jointly 62 Including schools, the National Youth Service, hospitals and the Kenya Police Service 63 Other stakeholders included the Kenya Dairy Board, and the National Cereals and Produce Board (NCPB). County Date of training Number trained Male Female Total Baringo 20-26 November 2017 280 60 340 West Pokot 27 November – 3 December 2017 301 80 381 TOTAL 721 WFP Kenya Endline LRP Evaluation Report – May 2020 |Page 21 developed a Cabinet Memo64 in 2016 asking that every public procuring entity nationwide allocate at least thirty percent of its procurement value to smallholder farmers. The intention during the LRP was to have a national level procurement strategy endorsed, and then follow it with two county level strategies: Baringo and West Pokot, as had been done for Turkana County in 2016.65 Unfortunately, the national strategy work stalled due to changes in senior leadership in the MoA and the complex politics of public procurement, especially when advocating for legislative changes that would affect multiple ministries and sectors. 81. To gather advocacy evidence, in 2017 the MoA led an assessment on public institutional food procurement from smaller famers66 and identified several opportunities and challenges for providing direct access for smaller holder famers. The assessment found there is a viable food market for small holder farmers, especially given the large numbers of public institutions that procure food. However, the government’s procurement process brings some challenges that would need to be discussed and negotiated. In July 2019, WFP supported an assessment specifically on the potential local economy impacts of Kenya’s HGSMP. 67 This was done using a local economy wide impact evaluation (LEWIE) approach. The LEWIE study found that encouraging schools to purchase more food directly from local farmers would increase the programme’s total benefits while providing income gains to HGSMP households. Further, making farmers near HGSMP schools more productive, would substantially increase the benefits that the HGSMP createslocally. The promotion of small holder procurement is therefore a worthy cause, which would have multiple benefits. 82. While waiting for discussions on the National Implementation Strategy to restart, WFP found another opportunity to promote smallholder procurement, shifting focus to the county Early Childhood Development and Education service (ECDEs) and working with Turkana County Department of Education on an ECDE Framework. As ECDE personnel had been involved in the HGSMP management training described above, it was an opportune time, while interest was high, to discuss the possibility of utilizing smallholder procurement to provide meals for the ECDEs. This work has resulted in a draft framework (September 2019) that includes supporting a home-grown feeding programme as part of the ECDE services. Although smallholder procurement is not specifically mentioned, this is an important starting point for future work. 83. This work was well received by Turkana County representatives, with particular praise from MoA and MoE for WFP demonstrating flexibility to find other opportunities once the National Implementation Strategy talks stalled. Qualitative interviews indicate that the draft framework aligns well with the Turkana County Integrated Development Plan (CIDP) (2018-2012)68 and The County Early Childhood Education Bill (2018)69 which states the importance of improving ECDE centres in Turkana County. 84. In recent months, discussions with County Officers have restarted with a view to continuing the development of a county level strategy in Baringo County before LRP funds expire. If endorsed, the process and final strategy would then serve as a guide for other counties interested in pursuing the same. The stalling of the national Implementation Strategy meant that the target set under the LRP of developing three policies, has not been achieved (Table 14) although there are now two strategies in draft form. 64 Republic of Kenya (2016) Cabinet memo on strengthening smallholder farmers’ production and marketing by facilitating direct access to public food procurement opportunities. Ministry of Agriculture, Livestock and Fisheries and National Treasury. November 2016. 65 WFP (2016) Turkana County Strategy to improve access to markets for smallholder farmers and small-scale processors (2016-2019). 66 Republic of Kenya (2017) Assessment report on public institutional food procurement form small holder farmers in Kenya. December 2017. 67 J.E. Taylor et al (2019) Potential local economy impacts of Kenya’s Home-Grown School Meals Program. UC Davis. 68 Turkana County (2018) Turkana County Integrated Development Plan, CIDP II: 2018-2022. Popular version. 69 The County Early Childhood Education Bill (2018). http://kenyalaw.org/kl/fileadmin/pdfdownloads/bills/2018/CountyEarlyChildhoodEducationBill_2018.pdf WFP Kenya Endline LRP Evaluation Report – May 2020 |Page 22 Table 14: Number of policies/strategies supported as a result of USDA assistance Indicator Target Baseline Endline Number of policies, regulations and/or administrative procurements in each of the following stages of development, as a result of USDA assistance 3 0 0 • National Implementation Strategy in progress • ECDE Framework drafted: Sept 2019 • Capacity strengthening for local traders and FOs 85. This activity aimed to increase the capacity of suppliers (traders and FOs) and school meals committees to effectively and efficiently procure local commodities for school’s meals. To identify appropriate traders to participate, mapping and profiling of local traders and FOs was conducted on a rolling basis. From this exercise, 44 FOs and 35 traders were identified for capacity building and support through the LRP. Training on the HGSMP was then provided to create awareness on the government’s procurement process and requirements, to enable them to participate effectively and potentially qualify to supply food to schools. The capacity building package covered specific areas that are key for market readiness, such as post￾harvest handling and storage, collective marketing, food safety and quality, and bidding and contracting to supply the HGSMP. In total, 3,099 FO members and traders have benefitted from short-term agricultural productivity or food security training as a result of USDA assistance. 86. Table 15 shows that the majority of traders surveyed for the endline evaluation confirmed participating in HGSMP training by WFP. The percentage of trained traders in West Pokot reduced over the LRP period because the rolling registration of traders meant that even though all registered traders were trained by baseline, some traders registering later have missed training. A few tradersreported participating in other procurement training including some short information sessions from Ministry of Industry, Trade and Cooperatives (MoITC), but WFP and MoE are the main trainers in this sector. The evaluation found a significant increase in the percentage of FOs reporting that their committee members had been trained, increasing from 36.7 percent at baseline to 74.7 percent at endline (p<0.0001). 70 The range of topics that FOs reported being trained on can be found in Annex 18. Generally, FOs reported that three committee members had been trained with the expectation that they would pass the information to other members. Table 15: Percentage of LRP traders who received training on procurement processes Baseline Endline Baringo West Pokot Baringo West Pokot % of traders trained on HGSMP procurement (by WFP) 100% 100% 100% 72% % of traders trained on procurement (by other agencies) 7.7% 9.8% 0% 4% 87. The evaluation found these trainings were well implemented, with qualitative interviews with FO members indicating the all the topics were useful. One issue, however, was that some of the materials they were provided with were too difficult for people with low levels of education. Although the information was written in English, and members preferred that to Kiswahili as many people only read English, the level of English was often too difficult, with members requesting more pictorial content. 88. In addition to the formal training, WFP also organized four market linkage forums covering four sub￾counties in Baringo and four sub-counties in West Pokot. The forums brought together FO members, traders, HGSMP schoolteachers, public secondary school teachers and county ECDE officers for two-way supply and demand matching discussions. This is in line with a recommendation from the baseline 70 t-test results - Pr(|T| > |t|) = 0.0000 WFP Kenya Endline LRP Evaluation Report – May 2020 |Page 23 assessment: that WFP and the MoA should ensure that the planned LRP market linkage forums provide opportunities for FOs to meet with local traders.71 89. Stakeholders reported that these forums were a very practical way to improve relationships between farmers and traders, with many believing that they have contributed to traders increasingly buying from local FOs to supplement other sources and reduce transport costs. As it became clearer that the HGSMP primary schools would continue to be supported by in-kind commodities from the government instead of cash transfers, and that local FOs and traders would not be able to sell to them as planned, the market forums have provided FOs with other selling opportunities. Notably, the inclusion of ECDEs and public secondary schools in the market linkage forums connected FOs with other potential buyers. 90. In Turkana, WFP has also used complementary funding from EUTF and BMZ to further strengthen the capacity of some FOs. WFP is actively supporting five of the LRP FOs to start flour milling operations, enabling them to mill maize and sorghum, and provide either single grain or blended flours to different markets as required. WFP has also provided FOs with equipment such as maize shellers, tractors, hermetic bags72 to improve the product quality. Some FOs have also been provided with support for business modelling, poultry, or value adding using the EUTF and BMZ funds. • Develop School Meals Menus Using Local and Nutritious Produce 91. According to the HGSMP implementation guidelines, 73 a variety of food items should be selected for the school meal food basket and a combination of the different food groups should be consumed daily to ensure a healthy diet and prevent malnutrition. Nutrition education should also be integrated. To ensure the HGSMP guidelines are adhered to, the LRP included a nutrition-related activity: working with MoE, MoA and MoH and other stakeholders74 to develop a variety of school menus using the four drought￾tolerant commodities identified in the VCAs, to align with other programme activities. This is in line with two of the recommendations of the baseline assessment. 75 92. The original plan for this activity was for WFP to develop nutrient profiles for each of the key drought￾tolerant crops that the project would promote, and those would be used as the basis of nutrition education. However, during early discussions with the MoH it became clear that the agreed nutrient profiles would not be required, as the Government of Kenya was about to release of Kenyan National Food Composition Tables (FCT)76 which include comprehensive nutrient analysis of 509 raw and cooked foods that are either commonly consumed in Kenya, are rich in nutrients of public health interest or are commonly consumed commercial packaged foods. 93. Once the FCTs were released, WFP supported three workshops, one in each LRP county from September 2018 to July 201977 to bring stakeholders together to provide training on the importance of dietary diversity and promotion of locally produced crops. The workshops also acted as a community sensitization forum and a forum for gaining consensus on whether the four identified commodities would be available and culturally acceptable in each county. The main issues identified were the price of the drought-tolerant 71 Dunn, S & Otsola, J (2018) Baseline of the Local and Regional Food Aid Procurement Project in Kenya, 2017-2020. Final Report, June 2018. 72 Hermetic bags are grain storage bags that cut off oxygen to the commodity inside. This kills weevils and other insects that might otherwise damage or destroy the commodity. 73 Republic of Kenya (2016) Home Grown School Meals Programme Implementation Guidelines. World Food Programme. 74 County level MoE, MoA, MoH. MoITC, NGOs and FO members also participated in some counties. 75 Recommendation 5: That WFP and MoH should collaborate to develop practical guidance for LRP schools on ways to introduce local produced drought-tolerant crops into their school menus; and Recommendation 6: WFP and MoH should collaborate to ensure that schools receive regular nutrition support since this is a new project area. 76 FAO/Government of Kenya (2018) Kenya Food Composition Tables. Nairobi. 77 Safe Food Handling and Preparation, sorghum and cow peas menu and recipes development report – Turkana County – September 2018; Sorghum, finger millet, green grams and cowpeas menu and training package development workshop report for West Pokot County – July 2019; Sorghum, finger millet, green grams and cowpeas menu and training package development workshop report for Baringo County – July 2019. WFP Kenya Endline LRP Evaluation Report – May 2020 |Page 24 commodities, and the potential lack of familiarity for people for eating cow peas as a pulse. People mainly consume cow pea leaves as a vegetable and not as a pulse. Once that was agreed, stakeholders collaborated on the development of new school meal menus and recipes. In total, 456 people (37% female) participated in the workshops (Table 16). Table 16: Number of participants in workshops to develop new school meal menus 94. The evaluation found that the workshops were very well received by stakeholders, with qualitative interviews citing good collaboration, capacity strengthening and partnership building. The menu development process was also considered by stakeholders to have provided a key advocacy opportunity to inform the county governments about the importance of dietary diversity and school meals. This was especially helpful for promoting a home-grown approach to school meals for ECDE. 95. Overall, the LRP activities have resulted in WFP supporting more than 216,000 individuals, either directly or indirectly (Table 17). This has exceeding both project targets, mainly because of the number of children benefiting from the food provided to LRP schools by WFP through the FDCs in Turkana. Table 17: Number of individuals benefiting from this USDA funded intervention Indicator Target Baseline Endline Achievement Number of individuals benefiting directly from USDA funded intervention 31,150 721 54,895 176% Number of individuals benefiting indirectly from USDA funded intervention 61,000 0 162,082 266% Outcome results • Strengthening of local markets 96. The project was intended to increase access to markets for the sale of agricultural products, ultimately resulting in a stronger local and regional food market system. Increased sales of local food commodities were also expected to increase access to the locally produced drought-tolerant crops being promoted through the project: sorghum, millet, cowpeas and green grams. Procurement by schools, directly with local traders and FOs was also expected to improve the timeliness, cost-effectiveness and nutrition of the school meals. All these outcomes were assessed through the FO, trader and school surveys as well as qualitative interviews with stakeholders. 2.3.2 Key results from the FO survey 97. One aspect of the LRP was that FOs were informed about the commodities required for the HGSMP and encouraged to grow larger volumes to prepare for potential sales, while ensuring that the commodities would pass the required food safety standards. However, Table 18 shows that in all three counties the proportion of FO members growing the major school meals crops has decreased since baseline, with the exception of beans. Qualitative interviews indicated that this is linked to the drought conditions but also to the lack of market demand once FOs became aware that HGSMP schools would not be asking for commodities. All FOs reported growing some form of fruit and vegetables, which was a positive change since baseline. Male Female Total Baringo 8 12 20 Turkana 270 147 417 West Pokot 11 8 19 Total 289 167 456 WFP Kenya Endline LRP Evaluation Report – May 2020 |Page 25 Table 18: Commodities grown by FOs 98. Since food quality and safety are important aspects of the procurement and handling of food, FOs were asked to report how they ensure the quality of commodities. At baseline, less than a third of the surveyed FOs reported making any effort to ensure quality. By endline, there had been an improvement, with more than half in the FOs in each county monitoring product quantity. More FOs in all three counties reported sorting produce after harvest at endline, while more farmers in Turkana and West Pokot also reported at endline that they cleaned their product to ensure it was free of foreign matter. Interviews indicate that these changes were largely due to the training received on HGSMP as it included a food safety module (Table 19). Table 19: Ensuring quality products for sale Baseline Endline Baringo Turkana West Pokot Baringo Turkana West Pokot FOs reporting monitoring market trends/ requirements related to product quality and market demand 22.4% (n=10) 27.8% (n=10) 25% (n=4) 73.9% (n=17) 62.8% (n=22) 57.1% (n=12) Out of those who try and ensure quality: Methods to ensure quality standard Use certified seeds 10% (n=1) 100% (n=10) 100% (n=4) 23.5% (n=4) 37.1% (n=8) 42.9% (n=5) Testing moisture content 0 10% (n=1) 75% (n=3) 0% 9.1% (n=2) 0% Clean product, free of foreign matter 90% (n=9) 33% (n=3) 50% (n=2) 29.4% (n=5) 31.4% (n=7) 23.8% (n=3) Sorting produce after harvest 0 20% (n=2) 75% (n=3) 30.4% (n=5) 22.9% (n=5) 41.6% (n=5) 99. Another positive result for the project is that FOs in West Pokot reported selling a greater percentage of their production at endline (57.1%) compared to baseline (26.5%). Sale of commodities through FOs in Baringo and Turkana remained similar to baseline. (Figure 2). Baringo Turkana West Pokot Baseline Endline Baseline Endline Baseline Endline Maize 89.1% 56.5% 86.1% 88.6% 93.8% 71.4% Sorghum 43.5% 17.4% 88.9% 88.6% 68.8% 9.5% Beans 69.6% 57.2% 0.0% 11.4% 93.8% 52.4% Cowpeas 17.4% 8.7% 91.7% 77.1% 56.3% 4.8% Green grams 19.6% 0.0% 72.2% 71.4% 62.5% 14.3% Vegetables 54.3% 100.0% 83.3% 100.0% 87.5% 100.0% Fruits 47.8% 100.0% 47.2% 100.0% 56.3% 100.0% WFP Kenya Endline LRP Evaluation Report – May 2020 |Page 26 Figure 2: FO members’ average use of own production NB. No data was collected at baseline in Turkana on the percentage of production sold through FOs. 100. Table 20 shows that over the course of the LRP, there has been a significant increase (p<0.001)78 in the percentage of FOs aggregating and marketing members’ food commodities, from around 25 percent to around 50 percent in all three LRP counties. The lack of FO aggregation and sales of members commodities were areas specifically mentioned in the baseline assessment as an area needing strengthening, so this is a noteworthy positive result that qualitative interviews attributed to the support from WFP and MoA. The main barriers to selling members produce identified by FO representatives, included poor transport infrastructure, unpredictable prices, and limited consumer demand. Despite these barriers, the percentage of FOs reporting selling commodities of members in the last year increased significantly (p<0.05)79 from baseline (Table 21). Table 20: FO sales of members’ commodities Baseline Endline Baringo Turkana West Pokot Baringo Turkana West Pokot Percentage of FOs aggregating and marketing members food commodities 26.1% 27.8% 25% 47.8% 42.9% 57.1% Table 21: FO sale of commodities in last one year Baseline Endline Baringo Turkana West Pokot Baringo Turkana West Pokot Percentage of FOs reporting selling commodities in the last one year 25% 25% 25% 43.5% 28.6% 57.1% 101.Other positive results for FOs identified through both qualitative and quantitative methods were that FOs are increasingly looking towards value-adding to increase their revenue, and that FOs are more actively looking for regular sales contracts. Table 22 shows that four FOs in Baringo were successful in gaining contracts to sell commodities, mainly to traders, while FOs in all three LRP counties have started value addition enterprises including the ones mentioned earlier supported by EUTF and BMZ. 78 t-test results Pr(|T| > |t|) = 0.0002 79 t-test results Pr(|T| > |t|) = 0. 0066 0% 50% 100% Baringo Baseline Baringo Endline Turkana Baseline Turkana Endline West Pokot Baseline West Pokot Endline Percentage of production consumed by HH members Percentage of production sold through FO WFP Kenya Endline LRP Evaluation Report – May 2020 |Page 27 Table 22: Number of FOs with sale contracts and value addition enterprises 102. The percentage of FOs selling to different buyers reduced since baseline, with households, retail traders, other traders (wholesalers) and schools continuing to be the main markets. In Turkana, with support from WFP and the county governments, three specific market days have been established each month in Lodwar, Kakuma and Lockichar, providing an additional opportunity for FO members to sell their commodities. Overall, as the majority of surveyed FOs did not have the opportunity to sell commodities to the LRP schools, two of the key project indicators, the volume and value of sales by project beneficairies (to HGSMP schools) only includes data from the five FOs in Turkana that had FDCs (Table 23). Table 23: Volume and value of sales by LRP FOs 103. The FO survey collected data on the total sale volumes of FOs to all buyers and that indicates that sales of beans and other commodities except for maize have generally increased in all three LRP counties over the LRP period. These findings and others from the FO survey can be found in Annex 18. 2.3.3 Key results from the trader survey 104. Traders were asked to report the main source of the commodities they sell. By endline, the buying profiles had changed in both counties, with Baringo traders reporting increased buying from large-scale farmers or FOs (33%) and other traders (33%). West Pokot traders reported buying almost solely (92%) from small￾scale farmers, with the rest from other traders (Figure 3). Baseline Endline Baringo Turkana West Pokot Baringo Turkana West Pokot Number of FOs with signed contracts to sell commodities 1 3 1 4 0 0 Number of FOs that have value addition enterprises No data 5 No data 7 14 6 Indicator Target Baseline Endline Value of sales by project beneficiaries USD 1,179,612 Baringo: KES 16,674,366 (USD169,744) Turkana: No value given West Pokot: KES 244,433,000 (USD244,433) Turkana: USD147,655 Volume of commodities (mt) sold by project beneficiaries 565 Baringo: 2020.4 mt Turkana: 16.4mt West Pokot: 9334.3 mt Turkana: 349.6mt Number of public-private partnerships formed as a result of USDA assistance 220 5 FOs (9 contracts) 5 FOs (12 contracts) Value of public and private sector investments leverages as a result of USDA assistance USD 1,179,612 0 Turkana: USD147,655 WFP Kenya Endline LRP Evaluation Report – May 2020 |Page 28 Figure 3: Percentage of traders reporting their main source of commodities 105. At both baseline and endline, traders reported selling their commodities mainly to households, retail stores and schools. In Baringo, 77 percent of traders sold to schools at baseline, compared with 67 percent at endline. In West Pokot, the percentage of traders selling to schools increased from 32 to 72 percent. Schools includes secondary schools, primary schools and boarding schools (Figure 4). Table 24 shows that since baseline there has been a slight increase in the percentage of traders supplying schools through competitive tenders. Given that the government provided the food commodities to government primary schools, these tenders relate solely to boarding and secondary schools. Figure 4: Percentage of traders reporting selling to the following buyers Table 24: Percentage of traders selling commodities to schools through competitive tenders Baseline Endline Baringo West Pokot Baringo West Pokot Percentage supplying schools through competitive tender 61.5% 31.7% 66.7% 44% 106. Traders were asked about the variety of staple crops and pulses they sell, to assess whether the four promoted crops were being more widely used in the community. In Baringo, all three traders reported a change in their sales of staple crops over the project (Figure 5), with maize, sorghum and rice being sold by all. Baringo traders also reported a wider variety of pulses being sold at baseline (Figure 6), including cowpeas, pigeon peas, green gram and lentils. West Pokot traders reported a very similar sale profile of both staples and pulses as the baseline profile. 0 20 40 60 80 100 Baseline Endline Baseline Endline Baringo West Pokot Small scale farmers Large scale farmers or FOs Other traders Other 0 50 100 Baringo Baseline Baringo Endline West Pokot Baseline West Pokot Endline Households Retail store Millers/brewers Other traders Hospitals Government food agencies Development agencies Schools NGOs Others WFP Kenya Endline LRP Evaluation Report – May 2020 |Page 29 Figure 5: Percentage of traders reporting sale of staple crops Figure 6: Percentage of traders reporting sale of pulses 107.Overall, the volume of commodities sold by the three Baringo LRP traders increased since baseline, but difference was not statistically significant (Table 25). On the other hand, the volume sold by West Pokot traders had decreased significantly (p<0.05). Table 25: Total estimated volume of commodities sold per year 108. Finally, traders were asked to rank the key barriers to grain training in general, as well as the barriers to purchasing from local farmers. Table 26 shows the traders in both counties said that transport costs and purchase price were the two main barriers to purchasing from local farmers. Other barriers included that the local farmers did not produce sufficient volumes to supply throughout the year. Additional findings from the trader survey can be found in Annex 19. Table 26: Percentage of traders reporting barriers to purchasing from local farmers 109.Overall, these evaluation findings show that the LRP has contributed to strengthening the local market by increasing the production, variety and quality of the crops grown by supported FOs. FOs have also sold a greater percentage of their production, with a significant increase (p<0.001)80 in the percentage of FOs aggregating and marketing members’ food commodities to achieve larger sale volumes. FOs are also now increasingly looking towards value-adding to increase their income, and actively looking for regular sales contracts. The LRP has also supported new markets within Turkana County, providing additional opportunities for FOs to sell their commodities. Overall, as the majority of surveyed FOs did not have the 80 t-test results Pr(|T| > |t|) = 0.0002 0 50 100 Baringo Baseline Baringo Endline West Pokot Baseline West Pokot Endline Maize Sorghum Rice Millet Other 0 50 100 Baringo Baseline Baringo Endline West Pokot Baseline West Pokot Endline Beans Cowpeas Pea Pigeon pea Green gram Groudnuts Lentils Baseline Endline Baringo West Pokot Baringo West Pokot Mean volume sold per year (MT) 150.3 515 226.1 258.1 Range 12-600 1.3 – 3,240 100 - 312 0.8 - 2,700 Barriers Volume produced by farmers Purchase price Transport costs Other barriers Baringo 66.7% 66.7% 66.7% 33.3% West Pokot 20% 52% 64% 16% WFP Kenya Endline LRP Evaluation Report – May 2020 |Page 30 opportunity to sell commodities to the LRP schools, two of the key project indicators, the volume and value of sales by project beneficiaries (to HGSMP schools) only includes data from the five FOs in Turkana that had FDCs. This small group of FOs have benefitted from the project by having a contract for sale ahead of harvest. This has contributed to the increased sale volumes of FOs since baseline. The LRP has also resulted in increased linkages between farmers, traders and buyers which should support local farmers. 2.3.4 Key results from the school survey • Provision of school meals 110. At endline, all the surveyed schools were part of the MoE’s HGSMP. Although the LRP schools had been informed in 2017 that they would be receiving cash-transfers from Term 1, 2018, this did not happen. Only 18 surveyed schools, all of which were non-LRP schools in West Pokot, reported receiving cash transfers during Term 3, 2019. Qualitative evaluation interviews also confirmed the findings of a recent WFP assessment81 that found that the MoE did not provide any in-kind commodities for Term 3, 2019. Schools received commodities for Terms 1 and 2, 2019 after which they were expected to utilize left-over commodities during Term 3. Given that the endline evaluation focused on Term 3, 2019, which was the last term during LRP implementation, the lack of food provision during this term has greatly affected the evaluation results. Table 27: Main source of food for school lunches (Term 3, 2019) 111. During the school survey, the enumerators looked at school records82 to establish how many days during Term 3, 2019, the school had provided a school meal. Table 28 shows that the percentage of schools providing a school meal every school day during Term 3 has significantly reduced (p<0.001) from baseline with many schools (n=75) not providing a meal at all during the whole term. Less than 10 percent (8.5%) of the surveyed LRP schools reported providing a meal every school day, with an average of 23 days (out of 45) missed (Table 29) compared to an average of 6 days missed at baseline. Similarly, only 12.3 percent of non-LRP schools provided a meal every day during Term 3, with an average of 10 days missed, up from 4.5 at baseline. Table 29 also shows that at both baseline and endline, the main/only reason for missing a day’s meal is because there was no food available. 81 WFP (2020) School meals supply chain compliance assessment report 82 Together with school personnel such as School Director or the teacher responsible for managing the school meals Main source of food Baringo Turkana West Pokot Total LRP Non￾LRP LRP Non￾LRP LRP Non￾LRP LRP Non￾LRP Through WFP funding (in-kind) -- -- -- -- -- -- -- -- Through WFP funding (cash transfers) -- -- -- -- -- -- -- -- Through MoE funding (HGSMP in-kind) 29 29 35 33 32 15 96 77 Through MoE funding (HGSMP cash transfers) -- -- -- -- -- 18 -- 18 Total 29 29 35 33 32 33 96 95 15.2% 15.2% 18.3% 17.3% 16.8% 17.3% 50.3% 49.7% WFP Kenya Endline LRP Evaluation Report – May 2020 |Page 31 Table 28: Percentage of schools reporting providing a school meal every school day Baringo Turkana West Pokot Total LRP Non￾LRP LRP Non￾LRP LRP Non￾LRP LRP Non￾LRP Baseline: Schools providing lunch every school day in Term 1, 2018 48.3% 75.9% 61.8% 88.2% 63.6% 45.5% 58.3% 69.8% Endline: Schools providing lunch every school day in Term 3, 2019 4.2% 16.8% 6.3% 5.3% 15.6% 9.5% 8.5% 12.3% Table 29: Reasons for not providing school meals Average days missed due to… Baringo Turkana West Pokot Total LRP Non￾LRP LRP Non￾LRP LRP Non￾LRP LRP Non￾LRP Baseline No food available 10 2 3 0 6 10 6 5 No one to prepare meals 0 1 0 0 0 0 0 0 Endline No food available 37 14 19.5 24 14 20 23 20 112.Qualitative interviews indicated that aside from the three LRP counties, the MoE also reverted other arid and semi-arid counties to in-kind commodities from Term 1, 2018 and maintained in-kind provision up to the time of the evaluation. This included Samburu, Tana River and Isiolo that have been implementing the HGSMP through cash-transfers since 2015. • Provision of more diversified school meals 113. Aside from the actual provision of a daily school meal, the LRP aimed to provide children with a more diversified school meal menu, which included locally grown drought-tolerant crops. This has been measured through a proxy indicator of the diversity of foods in the school meals menus. No consumption data was collected from children. The school survey confirmed that almost half the LRP schools (46%) reported working on diversifying their school menus, and eight percent of schools had already completed the development of new menus (Figure 7). Unfortunately, the MoE’s in-kind provision food of rice/maize, beans, oil and salt meant that the new menus could not be utilized. The exception was in Turkana, where the sorghum and cowpeas procured by WFP through FDCs were distributed to schools in Turkana South. A field monitoring report by MoE indicated that most of the schools in that sub-county had adopted the new menus developed with support of USDA. However, those schools received sorghum and cowpeas for Term 1, 2019 so the endline survey, which focused on Term 3, 2019, did not capture that information. Figure 7: Percentage of surveyed schools reporting developing new school menus since start of LRP 114. During Term 3, 2019, none of the schools reported serving the four promoted drought-tolerant crops in their school menus (Figure 8). Monitoring visits also confirmed that school cooks had improved their safe 0 50 100 LRP Non-LRP LRP Non-LRP LRP Non-LRP LRP Non-LRP Baringo Turkana West Pokot Total No Yes, in discussion Yes, changed menus WFP Kenya Endline LRP Evaluation Report – May 2020 |Page 32 food preparation as a result of the training they had received from WFP and MoE. WFP and MoH are planning an After-Action Review of this work during Q2 2020, to follow up on the inclusion of local commodities in school meals. Figure 8: Percentage of schools reporting using food commodities for their school lunches - Term 3, 2019 • 115. At baseline, the ET developed and calculated a Nutrition Score for the surveyed schools to show the level of diversification in the school meals. The Nutrition score is based on the volume of food items provided each day, the diversity in the overall meal provided and the number of days that lunch was provided. The score uses the actual volumes of food delivered to schools, so the calculation works regardless of whether the schools received in-kind commodities or commodities purchased through cash-transfers. 116. Table 30 shows that at baseline, the average Nutrition Score for LRP schools was 67.4, reducing to 64.5 at endline. At baseline there was no statistical difference found between LRP and non-LRP schools (p > 0.05), while at endline there was a statistical difference between LRP and non-LRP schools, with non-LRP schools scoring higher (better) (71.5) than LRP schools.83 The difference between the LRP and non-LRP schools is mainly due to the higher volumes of food that non-LRP schools were able to obtain from other sources, as described ahead. Nutrition Score = Av ((TVCD/150), (TVPD/40), (TVOD/5), (TVCS/2)*W1/3, (DLS/70)* W1/3, (MEN/3)* W1/3 Where: Av Average TVCD Total volume of cereals fed per child per day TVPD Total volume of pulses fed per child per day TVOD Total volume of oil fed per child per day TVSD Total volume of salt fed per child per day DLS Actual days lunch was provided MEN Menu items diversity W1/3 1/3 weighting Table 30: Change in Nutrition Score: Baseline vs. Endline 83 p < 0.001 - Pr(|Z| > |z|) = 0.0000) following a statistical z-test. 0% 50% 100% Rice Wheat Beans Yellow split peas Oil Salt Vegetables Onions MSB Beans Maize Oil Salt Onions LRP School Non LRP School Food Item Baseline Food Item Endline Baringo Turkana West Pokot Total LRP Non- LRP LRP Non- LRP LRP Non- LRP LRP Non- LRP Baseline – Term 1, 2018 66.6 69.7 70.5 70.6 64.8 60.5 67.4 66.9 68.2 70.6 62.7 67.1 Endline – Term 3, 2019 61.0 78.5 42.3 54 70.7 70.5 64.5 71.5 76.3 49.3 70.6 68.8 WFP Kenya Endline LRP Evaluation Report – May 2020 |Page 33 • Cost effectiveness of provision of school meals 117. Another objective of the LRP was to improve the cost-effectiveness of school meal delivery. Cost effectiveness is based on the volume of food received and the cost to buy it. As mentioned earlier, although schools did not receive the commodities (or cash) they were expecting, some schools have still been able to provide some meals during Term 3, 2019 (refer back to Table 28). This was largely due to using left over food from Term 2, and/or school personnel looking for alternative sources of food, including seeking donations from churches, PTAs, parents, or NGOs and asking traders to sell food on credit. However, even though 33.5 percent of surveyed schools reported obtaining alternative food sources, the volumes have not been sufficient to provide food every school day. 118.Over the 70-day school term period of Term 1, 2018 (baseline) each child should have received 13.8kg of food based on the SMP ration84 and Table 31 shows that the average volume received at baseline was 13kg. For the 45-days of Term 3, 2019, children should have received 9kg of food. Table 31 shows that the volumes obtained is considerably less that. Since no food was provided by the MoE during Term 3, 2019 the mean volume for LRP schools is zero. For the 18 non-LRP schools that received cash transfers from MoE to buy their own food, the average volume was 4.1kg, well below the required volume. For schools finding alternative sources to supplement the left-over commodities from Term 2, the average volumes were 4.8kg for LRP schools and 7.9kg for non-LRP schools. Non-LRP schools in Baringo managed to obtain the required volume of 9.2kg per child. Overall, there is a significant difference between LRP and non-LRP schools (p<0.001),85 with non-LRP schools receiving more commodities. Table 31: Mean volume of food per child received Baringo Turkana West Pokot Total Baseline: Mean volume of food per child, Term 1, 2018 LRP schools 12.8 14 11.6 12.8 Non-LRP schools 16.1 12.9 9.6 12.9 Endline: Mean volume of food per child provided by MoE, Term 3, 2019 LRP schools 0 0 0 0 Non-LRP schools 0 0 4.1* 4.1* Endline: Mean volume of food per child provided from alternative sources, Term 3, 2019 LRP schools 7.8 2.4 6.9 4.8 Non-LRP schools 9.2 6.3 6.4 7.9 * Only includes data from the 18 non-LRP schools that reported receiving cash transfers, as they were the only schools that purchased any food using MoE resources. 119. Discussions with school personnel during the quantitative interviews indicate that the volume of food received by schools during Term 2, 2019 was insufficient in most schools to last into Term 3. Interviewees reported that there are often differences in the school’s enrolment figures and the figures in the MoE HQ planning data. This is the most common reason cited for schools not being provided with sufficient food to last the year. Qualitative evaluation interviews also identified insufficient resources at central level after sweeping budget cuts across several ministries. On top of that, during Term 3, 2019, due to a technical difficulty, the MoE was unable to provide cash-transfers to other HGSMP counties if they had bank accounts with banks other than Kenya Commercial Bank (KCB). 84 The recommended SMP ration provides a total of 197g per child per day = 150g cereals, 40g pulses, 5g oil and 2g salt. 85 The results of a t-test are: diff = mean (Non LRP) – mean (LRP); Ho: diff = 0; Ha: diff > 0; Pr (T > t) = 0.0002 WFP Kenya Endline LRP Evaluation Report – May 2020 |Page 34 120. The Cost-Effectiveness Score developed by the ET at baseline provides an estimate of the funding provided to feed one child per day (in KES). The Score is calculated using the volume of food delivered, and the cost of the that food, so at endline, it was only possible to calculate the Cost-Effectiveness Score for the 18 non￾LRP schools in that received cash-transfers from the MoE. Cost-effectiveness Score = GMC * VCPD Where: GMC = ATA/ATV Actual grouped median cost = actual total cost of commodities/ actual total volume of commodities VCPD = VCPT/DLS Volume per child per day = Volume per child per term/ number of days lunch is served 121. Table 32 shows that at baseline, the cost of providing food to schools in the three targeted LRP counties was between 10-13 KES per child per day, with Baringo being the most expensive at 12.1 KES. At endline however, the monies spent by the 18 non-LRP schools in West Pokot for their school meals, equates to only 8.7 KES per child per day. The N/A for all the remaining schools indicates that either no food was delivered in Term 3 from the MoE and/or no food was provided from any other source that had to be paid for (i.e. it was donated). Table 32: Change in Cost-effectiveness Score Baringo Turkana West Pokot Total LRP Non- LRP LRP Non- LRP LRP Non- LRP LRP Non- LRP Baseline: Term 1, 2018 11.5 12.7 10.9 9.8 9.6 8.9 10.7 10.4 12.1 10.4 9.3 10.5 Endline: Term 3, 2019 N/A N/A N/A N/A N/A 8.7* N/A 8.7* N/A N/A N/A N/A * Only includes data from the 18 non-LRP schools that reported receiving cash transfers, as they were the only schools that purchased any food for the school meals. 122. The baseline assessment recommended that WFP support the MoE to review the HGSMP transfer rate once updated market assessment findings were available for Baringo and West Pokot. With those assessments indicating transfer rates of 10-13 KES the baseline values align with the market assessments. At endline, the only schools that received cash-transfers have spent less than 9 KES per child, which is insufficient to procure the required value of commodities for the term. This accounts for the low volume of commodities reported by the 18 non-LRP schools earlier in Table 31. Additional findings from the school survey can be found in Annex 20. Summary of key findings: Effectiveness: • WFP has implemented most of the LRP project activities as planned, resulting in several positive outcomes. However, the MoE’s provision of in-kind commodities to schools throughout the project period, instead of the cash-transfers that were planned, means that many of the intended school-related objectives have not been achieved. • It was not possible for WFP to have foreseen the sudden change in position on cash-transfers by the MoE especially after successful transitional cash transfers in arid areas prior to handover of arid counties to government. The evaluation found that the most likely reason was a change of senior leadership in the MoE. • Out of the 191 schools surveyed, only 18 non-LRP schools in West Pokot reported receiving cash-transfers for HGSMP as planned. None of the other schools received any resources (food or cash) from the MoE for school meals for Term 3, 2019. As a result, only around a quarter of schools were able to provide lunch every day. Although many schools tried to obtain food from other sources to supplement any left-over food from Term 2, many schools (n=75) were unable to provide any school meals during Term 3, 2019. • For the 45-days of Term 3, 2019, children should have received 9 kgs of food. However, since no food was provided, the mean volume for LRP schools is zero. For the 18 non-LRP schools that received cash transfers to WFP Kenya Endline LRP Evaluation Report – May 2020 |Page 35 buy food, the average volume purchased was 4.1kg, well below the required volume to feed children for the whole term. • The in-kind provision of commodities also made it impossible for most schools to include the locally procured, drought-tolerant crops as part of the school meals in Baringo and West Pokot Counties. The only LRP sub-county from which there is any evidence of consumption of a more diversified school meals basket is Turkana South, where schools received in-kind provision of sorghum and cowpeas by WFP through the FDCs with FOs. As a result, there was no significant change in the Nutrition Score from baseline to endline. • The market assessments have been effective at confirming that cash is a viable option in each of the three LRP counties, and that the market would be able to supply the HGSMP schools. However, the information collected by the assessments, including estimates of appropriate cash-transfer values were not utilized effectively by the MoE. • The development of national and county level strategies on small holder procurement have not occurred as planned. However, WFP have found other ways to promote small holder procurement, including shifting focus to the county ECDEs and working with Turkana County Department of Education on an ECDE Framework. Although smallholder procurement is not specifically mentioned within the document, this is an important starting point for future work. • The Cost effectiveness Score could only be calculated for the 18 non-LRP schools that reported receiving cash transfers, as they were the only schools that purchased any food. Their score indicates spending of 8.9 KES per child per day, which is insufficient to procure enough food for the full term. • WFP have conducted several formal training exercises and workshops as part of the LRP. All of these were conducted in collaboration with the relevant government ministries. While the evaluation received positive feedback on all the training work, most participants did not have a chance to put the learning into practice due to schools not requiring local procurement. The exception has been the procurement training for FOs and traders. As the HGSMP procurement process follows the government’s procurement process, traders have been able to use their learning to bid for other government tenders, particularly for secondary school and boarding school meals. Similarly, the training for FOs has contributed to more FOs monitoring product quality requirements. • The market forums and other support to FOs has effectively resulted in changes to FO production including a greater percentage of FOs aggregating and marketing members food commodities, and FOs in West Pokot reported selling a greater percentage of their production. This despite the in-kind provision by MoE reduced the potential market opportunities for FOs, acting as a disincentive to produce the promoted crops. • The collaborative approach that WFP has taken on implementation with MoE, with MoA, MoH and MoTIC has been an effective capacity strengthening approach, providing government personnel with added confidence to provide training on their own in future. • The Cost effectiveness Score could only be calculated for the 18 non-LRP schools that reported receiving cash transfers, as they were the only schools that purchased any food. Their score indicates spending of 8.9 KES per child per day, which is insufficient to procure enough food for the full term. • WFP have conducted several formal training exercises and workshops as part of the LRP. All of these were conducted in collaboration with the relevant government ministries. While the evaluation received positive feedback on all the training work, most participants did not have a chance to put the learning into practice due to schools not requiring local procurement. The exception has been the procurement training for FOs and traders. As the HGSMP procurement process follows the government’s procurement process, traders have been able to use their learning to bid for other government tenders, particularly for secondary school and boarding school meals. Similarly, the training for FOs has contributed to more FOs monitoring product quality requirements. • The market forums and other support to FOs has effectively resulted in changes to FO production including a greater percentage of FOs aggregating and marketing members food commodities, and FOs in West Pokot reported selling a greater percentage of their production. This despite the in-kind provision by MoE reduced the potential market opportunities for FOs, acting as a disincentive to produce the promoted crops. WFP Kenya Endline LRP Evaluation Report – May 2020 |Page 36 • The collaborative approach that WFP has taken on implementation with MoE, with MoA, MoH and MoTIC has been an effective capacity strengthening approach, providing government personnel with added confidence to provide training on their own in future. 2.4 Evaluation Theme 3: Efficiency 2.4.1 Did assistance reach the right beneficiaries at the right time? 123. Timeliness of procurement: One of the objectives of the LRP was to improve the timeliness of school meal procurement and delivery. This was based on the assumption that schools would be receiving cash transfers and doing their own procurement. As it was, none of the LRP schools and only 18 (18.9%) of the non-LRP comparison schools went through any procurement process. Out of those 18 schools, only 7 (39.4%) completed the procurement process before the start of Term 3, 2019. The LRP has therefore not been able to achieve the target of 90 percent of schools completing the procurement process before the start of term (Figure 9). Figure 9: Percentage of schools reporting food procurement process completed before start of Term 3, 2019 124. Timeliness of delivery: For the 18 non-LRP schools in West Pokot that reported receiving cash from the MoE, 13 (70.2%) received their locally procured food before the start of the term. For the schools that received food from alternative sources, only six percent of LRP schools and 25 percent of non-LRP schools received their food before the start of the Term 3, 2019. This is understandable, because schools only started looking for alternative food sources once Term 3 had started. These figures indicate that the LRP has not been able to achieve the target of 90 percent of schools having their food delivered before the start of term (Figure 10). Figure 10: Percentage of schools reporting that school lunch commodities from non-MoE sources were delivered before the start of Term 3, 2019 125. For the schools receiving food from alternative sources, the average delivery time was 35 calendar days after the start of Term 3, 2019 for LRP schools and 26 days for non-LRP schools, both longer delays than at baseline (Table 33). The high number of schools not being able to locate alternative sources, therefore having no delivery during Term 3 is why the average is so high. 0 50 100 LRP Schools Non-LRP Schools LRP Schools Non-LRP Schools Baseline Endline Baringo Turkana West Pokot 0% 50% 100% Baringo Turkana West Pokot LRP Target = 90% LRP Target = 90% WFP Kenya Endline LRP Evaluation Report – May 2020 |Page 37 Table 33: Average number of days after start of Term 3, 2019 that food was delivered Baringo Turkana West Pokot Total LRP Non￾LRP LRP Non￾LRP LRP Non￾LRP LRP Non￾LRP Percentage of schools receiving food deliveries from the MoE before the start of Term 3, 2019 0 0 0 0 0 70.2* 0 70.2* Percentage of schools receiving food delivery from any source before the start of Term 3, 2019 3% 14% 1% 0% 2% 12% 6% 25% Baseline: Average number of days delayed – Term 1, 2018 21 9 3 1 24 9 16 6 Endline: Average number of days delayed – Term 3, 2019 39 13 32 41 33 21 35 26 * Only includes data from the 18 non-LRP schools that reported receiving cash transfers, as they were the only schools that received any resources from MoE for Term 3, 2019. 126. Aside from no delivery in Term 3, 2019, qualitative interviews indicate that even in other terms when food was provided, the MoE’s procurement process has been challenging. One term, the bean delivery was delayed, and although the rice was ready to be sent to schools, the MoE waited to be able to transport rice and beans together, resulting in a very long delay before schools received any commodities. Delays are also common once food is received at county level, as additional transport needs to be organized to send the food to schools. Some stakeholders also voiced concern that county warehouses are not in appropriate condition to store food, resulting in a risk to food safety and quality. 127. Like the other evaluation scores developed by the ET specifically for the LRP, the Timeliness Score was designed to provide a way to compare the timeliness of the MoE’s/schools food delivery over time. As with the Cost-Effectiveness Score, the Timeliness Score can only be calculated for schools that actually received resources from the MoE, either as in-kind or cash for Term 3, 2019. Again, this is only the 18 non￾LRP schools in West Pokot that received cash transfers. Timeliness Score = (FD – DD – ML)/ (FD) * 100 Where: FD = Full days (Term days + maximum number of delayed delivery days + maximum number of days of lunch missed)86 DD = Actual number of delayed delivery days ML = Actual number of missed school meal (lunch) days 128. The result was a Timeliness Score of 70.2 percent, which is significantly lower (worse) than at baseline (Table 34). This is due to delays in cash-disbursement, followed by procurement and delivery delays. Table 34: Change in Timeliness Score Baringo Turkana West Pokot Total LRP Non- LRP LRP Non- LRP LRP Non- LRP LRP Non- LRP Baseline – Term 1, 2018 77.9% 91.8% 95.9% 99.1% 79% 86.2% 84.7% 92.5% 84.9% 97.5% 82.6% 88.6% Endline – Term 3, 2019 0 0 0 0 0 70.2%* 0 70.2%* 0 0 70.2% 70.2% * Only includes data from the 18 non-LRP schools that reported receiving cash transfers 86 For Term 1, 2018: Term days = 70, maximum delayed delivery days = 70, maximum number of days of lunch missed = 70 WFP Kenya Endline LRP Evaluation Report – May 2020 |Page 38 Summary of key findings: Efficiency • One of the objectives of the LRP was to improve the timeliness of procurement of school meals commodities by providing schools with cash transfers, enabling them to do local procurement of commodities, which would hopefully result in more timely delivery of commodities to the schools. However, since the MoE did not provide any cash resources to LRP schools for Term 3, 2019, none of the LRP schools were required to undertake a procurement process. • The target of 90 percent of LRP schools completing procurement before the start of term has therefore not been achieved. • Further, none of the LRP schools received any in-kind resources from the MoE for Term 3, 2019, so the target of 90 percent of LRP schools having their food delivered before the start of term has not been achieved. • The Overall Timeliness Score for the LRP schools is therefore zero. 2.5 Evaluation Theme 4: Impact 129. At baseline, stakeholders were overwhelmingly positive about the LRP schools’ transition to the HGSMP citing the potential improved timeliness of food delivery and economic benefits to local communities. Unfortunately, despite the strong implementation of all the project activities by WFP and partner ministries, the MoE’s provision of in-kind commodities to the LRP schools has not enabled local procurement to take place as planned, which has significantly reduced the intended impact. 130. To measure the overall impact of the LRP, the ET had planned to use the Impact Score developed at baseline, which takes into account the intended outcomes as measured by the timeliness, cost￾effectiveness and nutrition scores described earlier. Impact Score = ((TS + NS - CS)/187) *100 Where: TS = Timeliness Score NS = Nutrition Score CS = Cost Effectiveness Score 131. Table 35 shows that at baseline, non-LRP schools had a better overall score than the LRP schools, indicating more timely, cost-effective and nutritious school meals were provided through the HGSMP (cash transfers from the MoE) than though in-kind provision by WFP or MoE (LRP schools). The difference was statistically different (p<0.01). There was also statistically significant difference between counties (p<0.001), with schools in Turkana ranked as the best performers in the three outcome areas, and West Pokot the poorest. Unfortunately, however, at endline the Impact Score cannot be calculated for the LRP schools since their Timeliness and Cost-Effectiveness Scores could not be calculated due to no resources being provided for Term 3, 2019. Like the other scores, the Impact Score can therefore only be calculated for the 18 non-LRP schools in West Pokot that received cash (Table 37). Those 18 schools showed a positive change from the baseline result for West Pokot non-LRP schools. Table 35: Change in Impact Score * Only includes data from the 18 non-LRP schools that reported receiving cash transfers. Baringo Turkana West Pokot Total LRP Non- LRP LRP Non- LRP LRP Non- LRP LRP Non- LRP Baseline – Term 1, 2018 71.1% 79.6% 83.2% 85.5% 71.8% 73.7% 75.6% 79.7% 75.4% 84.3% 72.7% 77.6% Endline – Term 3, 2019 N/A N/A N/A N/A N/A 84.2%* N/A 87.7%* N/A N/A N/A N/A WFP Kenya Endline LRP Evaluation Report – May 2020 |Page 39 2.5.1 Were there any unintended outcomes, either positive or negative? 132. In the earlier sections on the effectiveness and efficiency of the project, it is clear that the intended objectives of improving the effectiveness of school meals through improved timeliness of procurement and delivery, improved cost-effectiveness and improved nutrition, have not resulted from the project. However, the LRP activities have resulted in several market improvements, including increased opportunities for FOs and traders, increased awareness of FOs and traders of the HGSMP in general, increased knowledge for FOs and traders on the required procurement process, and increased awareness among LRP school personnel on the importance of dietary diversity and how the school meal menus can be altered to increase the diversity of commodities. The evaluation identified some gender-related results from the FO data that indicate that women FO members are building their capacity. • Less (42%) women’s FOs collectively own land compared with 54% of FOs with mixed membership • One third (31%) of the FOs with women-only membership reported being able to access loan facilities compared to only 9% of FOs with mixed membership. • 83% of women only FOs have had their committee members trained in the past year compared to 73% of FOs with mixed membership. • 50% of women only FOs have had their members trained in the past year compared to 40% of FOs with mixed membership. 133.Qualitative interviews were also able to identify some negative outcomes for all beneficiary groups, farmers (FOs), traders and schools. All these negative outcomes were not as a result of the LRP per se, but as a result of the provision of in-kind commodities when cash-transfers had been expected. The main concern was that after receiving the HGSMP training both FOs and traders were preparing for the potential opportunity that school meals would bring. For FOs that involved growing or aggregating increased volumes of beans and maize and/or the drought tolerant crops promoted by the LRP. For traders, it meant organizing larger stock volumes to allow for increased potential sales. Ultimately, neither group had the opportunity to bid to supply food to the LRP or non-LRP HGSMP primary schools, leaving both groups with larger volumes of food to sell to other buyers. 134. While each of the traders that participated in the qualitative interviews indicated they were eventually able to sell the larger stock volumes, in some cases to buyers that they met through the WFP market forums, and in some cases they arranged to provide to HGSMP schools on credit, while schools waited for the planned government HGSMP cash-transfers. Given that these cash-transfers never eventuated, it is not clear how long it will take for schools to be able to repay their debts. The accumulation of larger than normal volumes of stock was therefore only a temporary negative outcome, but the debts from schools has the potential to be a long-term negative outcome for traders. 135. The FOs with larger than usual stock volumes either sold them or kept the excess for household consumption, so again, the negative outcome was temporary. The FOs that were not as affected by the government’s in-kind provision to schools were the FOs in Turkana that were given FDCs to supply sorghum or cowpeas to WFP. Although some of those FOs had also planned to bid for school tenders, they were still able to supply at least a portion of their commodities to WFP. 136. For many of the surveyed schools, both LRP and non-LRP, the main negative impact was not related to the decision to provide in-kind commodities per se, but the lack of delivery in Term 3, which meant that most schools were unable to provide school meals every school day during that term. WFP Kenya Endline LRP Evaluation Report – May 2020 |Page 40 2.5.2 What internal and external factors affect the project’s achievement of intended results? Internal factors 137. Strong programme management and technical expertise: The design and implementation of the LRP activities have benefitted from strong technical expertise and experience of the WFP LRP team in agriculture, agri-business and market support. The LRP has also benefitted from the technical expertise of several other WFP teams including the Vulnerability Assessment and Mapping (VAM) (market assessments), nutrition (implementation) and of course decades of experience within the School Meals team (implementation). In addition, the flexibility demonstrated by the programme managers when it was clear that in-kind provision would be extended beyond Term 1, 2018 enabled WFP to expand the focus of LRP to include the ECDEs. This enabled the LRP capacity building activities to reach a wider audience, ultimately resulting in the development of the ECDC draft framework. 138. Collaborative implementation approach: Since the handover of the HGSMP to the MoE in July 2018, WFP now plays an ‘enabling’ role rather than an ‘implementing’ role in the school meals programme. This recent shift in WFP’s modus operandi has been well demonstrated during the LRP implementation, with WFP and central and county level authorities working together on most of the LRP activities. This collaborative approach was appreciated by the government key informants, as WFP was able to provide several capacity strengthening opportunities for government representatives, both formally and informally, building the government’s technical knowledge, skills and confidence to implement the HGSMP in future without support. 139. Previous support to farmers/FOs in Turkana County: Some of the results of the LRP are due in part to the support provided by WFP and the MoA to farmers and FOs in Turkana County over the last decade. It was through other WFP projects that some farmers were encouraged to form the FOs that the LRP has supported. In addition, WFP has provided technical and financial support to some FOs to enable them to reach the standard that enabled them to win the FDCs with WFP for this project. Complementary funding from EUTF and BMZ has also enabled some of the LRP FOs to expand their businesses, including value adding to products, and/or establishing additional enterprises. External factors 140. Strong relationship with Government of Kenya: The relationship between WFP and the Government of Kenya, particularly with the MoE, goes back decades. During this period, WFP has played a major role in building the national school meals policy landscape and providing ongoing technical support to the MoE. WFP has seconded staff to several government ministries to build their capacity, including the MoE’s School Feeding Unit. As a result, the technical expertise on school feeding within the MoE is now well established. Throughout the LRP, WFP and MoE have worked in partnership to ensure that the activities were completed as planned. 141.National ownership of the HGSMP: Once the MoE formally took over the implementation and management of the HGSMP in July 2018, all the decision-making responsibilities for the HGSMP rest with them. While the initial decision to provide in-kind commodities during the baseline term (Term 1, 2019) was founded on understandable concerns about the capacity of the market during drought, the reason/s behind the subsequent decision to continue provision of in-kind commodities after the drought ended, were more difficult to establish. It was the ongoing provision of in-kind commodities that isthe main factor behind the poor outcome results of the LRP. That is because the LRP was specifically designed to support a cash based HGSMP. The additional decision to not provide any resources, either cash or in-kind, to schools during Term 3, 2019, has also had significant impacts on the provision of school meals in Term 3, and on the endline evaluation results. These decisions all appear to be the result of a change in senior leadership within the MoE, with the new incumbent not previously engaged with school feeding activities. WFP Kenya Endline LRP Evaluation Report – May 2020 |Page 41 Since the provision of in-kind commodities is a permitted modality according to the HGSMP Implementation Guidelines87 and NSMS, the MoE was within its rights to make those decisions. Summary of key findings: Impact • The LRP was specifically designed to support the transition to cash-based transfers in the LRP schools. The impact of the LRP was therefore considerably reduced by the MoE’s decision to provide in-kind commodities to schools instead of providing cash-transfers to school as planned. • The intended outcomes of improved timeliness of procurement and delivery, improved cost-effectiveness and improved nutrition, have not been realized, and the overall impact of the LRP could not be quantitatively measured as per the evaluation design. • The LRP activities have resulted in some improvement of market opportunities for FOs and traders, increased awareness of FOs and traders of the HGSMP in general, increased knowledge for FOs and traders on the required procurement process, and increased awareness among LRP school personnel on the importance of dietary diversity and how the school meal menus can be altered to increase the diversity of commodities. • The LRP activities have also resulted in increased awareness of government’s intent to use cash-transfers for local procurement for school meals, increased knowledge for FOs and traders on the required procurement process, and increased awareness among LRP schools on the importance of dietary diversity and how the school meal menus can be altered to increase the diversity of commodities. The evaluation also identified some gender-related results from the FO data that indicate that women FO members are building their capacity. • Qualitative interviews also identified some negative outcomes for FOs and traders, as both groups were left with larger than usual volumes of commodities after planning to sell stock to the HGSMP schools. Some traders also allowed schools to buy commodities on credit, pending future payment of the cash-transfers, however with uncertain plans for reverting to cash, it is unclear when schools will be able to pay off those debts. • Aside from the MoE’s decision to provide in-kind commodities, there were several other internal and external factors that have contributed to the achievement of the intended results. These include strong programme management and technical support from WFP, the collaborative approach taken to implementation and the support WFP had provided to farmers and FOs in Turkana County in previous years. 2.6 Evaluation Theme 5: Sustainability 142. As a programme designed to support the HGSMP, the LRP was never intended to be a sustainable programme in its own right. However, several of the LRP activities have contributed to sustainable results. The LRP was specifically designed to support schools in the three targeted counties to transition to the cash-based modality of the HGSMP instead of being provided with in-kind commodities. Although the decision to implement an in-kind modality throughout the two-year implementation period reduced the effectiveness of the LRP, it in no way reduces the sustainability of the HGSMP itself. The HGSMP Implementation Guidelines88 allow for provision of commodities using either cash-transfers or in-kind, and although the MoE has previously stated its long-held preference for cash-transfers due to the difficulties of logistics and transport and the need for the delivery of commodities to schools to be more timely, it is their prerogative to use either cash or in-kind depending on the situation. Since the handover of the HGSMP from WFP to the government in July 2018, the MoE continues to be committed to the programme, currently implementing the NSMNS until 2022 with an annual budget of USD 24 million (2018/19) and continual lobbying for additional funding. 87 Republic of Kenya (2016) Home Grown School Meals Programme Implementation Guidelines. World Food Programme. 88 Republic of Kenya (2016) Home Grown School Meals Programme Implementation Guidelines. World Food Programme. WFP Kenya Endline LRP Evaluation Report – May 2020 |Page 42 143. The qualitative evaluation interviews received varied information about the MoE’s future HGSMP modality plans for the arid and semi-arid counties, therefore it is difficult to clearly state whether some of the LRP results are likely to be sustained or not. Some stakeholders within the MoE thought it might take several years before the LRP counties transition to cash as planned. If that is the case, then any knowledge obtained through the LRP’s training components will be lost. However, other MoE representatives reported that cash transfers are already being planned for school year (SY) 2020/21. If that is the case, it is likely that some of the learning will be retained. Regardless of the modality of the HGSMP, there are several results that are likely to be sustained, including the strengthened relationships between traders and FOs as a result of the market forums. If the strategy development work can be finalized through other funding, and the strategies put into place, the capacity strengthening work will also have some sustainability. 144. The aspects of the LRP that the possibly the least sustainable are the procurement of drought-tolerant commodities from FOs for use in the HGSMP, and the use of drought tolerant crops in the school menus. Currently, those commodities are more expensive than maize and beans, so unless schools are provided with a cash transfer rate than enables procurement of these commodities, they are unlikely to do so, regardless of any nutritional benefits, or benefits to the local economy. If FOs can be supported to continue to produce drought-tolerant crops and the market demand increases, there is the potential for the price to come down. However, with the low cash-transfer rate compared to market prices and/or the late disbursement of cash transfers to schools, it is unlikely that schools will choose more expensive products unless the price comes down considerably. Summary of key findings: Sustainability • As a programme designed to support the HGSMP, the LRP was never intended to be a sustainable programme in its own right. However, several of the LRP activities have contributed to sustainable results. • The decision to implement an in-kind modality throughout the two-year LRP implementation period reduced the effectiveness of the LRP, but it in no way reduced the sustainability of the HGSMP itself. • It is difficult to clearly state whether some of the LRP results are likely to be sustained or not as the sustainability of some results depend on the MoE’s future plans regarding providing cash transfers. • Regardless of the HGSMP modality, the LRP has succeeded in strengthening relationships between traders and FOs and contributed to improving the policy landscape promoting smallholder procurement. • The least sustainable aspect of the LRP is likely to be the use of drought tolerant crops in the school menus. Currently, those commodities are more expensive than maize and beans, so unless schools are provided with a cash transfer rate than enables procurement of these commodities, they are unlikely to purchase those commodities regardless of the nutritional or economic benefits to the community. 3 Conclusions and Recommendations 3.1 Overall Assessment/Conclusions 145. The LRP offered an opportunity to ensure that the targeted schools in Baringo, Turkana and West Pokot transition smoothly to the HGSMP by ensuring that school personnel, FOs and local traders were aware of the required procurement process for acquiring commodities for the school meals. It also provided a starting point for WFP to work with farmers and FOs in Baringo and West Pokot. 146. The LRP was designed and implemented by WFP in collaboration with MoE, MoA and MoH with the activities aligning well with and with MoE, MoA and MoH policies, and with WFP’s other work in Kenya. The evaluation found that the planned cash-transfer HGSMP modality for providing school meals was the preferred option of all evaluation key informants, because of the potential benefits to the local economy WFP Kenya Endline LRP Evaluation Report – May 2020 |Page 43 and the potential to improve the timeliness and cost-effectiveness the procurement and delivery of commodities. The design of the LRP was therefore highly relevant. 147. The LRP has been well implemented, with the baseline recommendations being appropriately actioned. Unfortunately, the intended school-related outcomes of improved timeliness of procurement and delivery, improved cost-effectiveness and improved nutrition, have not been realized, and the overall impact of the LRP could not be quantitatively measured as per the evaluation design. This is largely due to the MoE providing in-kind commodities to the LRP schools instead of the cash-transfers that the LRP was designed to support. While the initial decision to provide in-kind commodities was understandable given the 2017/18 drought, the decision to continue providing in-kind commodities came as a surprise to WFP and to all the other programme partners, including the MoE technical officers. The evaluation concluded that WFP could not have foreseen the decision, especially since the HGSMP has moving towards national implementation of cash-transfers for more than a decade. 148. Despite this setback, the LRP has succeeded in strengthening the local market although not to the extent it was originally envisaged. Improvements were noted in crop production of some FOs and in the percentage of FOs aggregating their commodities to enable larger sales. The LRP also resulted in improved relationships and linkages between FOs and traders and created awareness about the HGSMP and the procurement processes that are required. When implemented through a cash-transfer modality, the HGSMP has the potential to provide a stable, long term market for local traders and farmers as evidenced by the findings of the LEWIE study. 89 However, without a cash-based modality, the involvement of the local market is severely limited, with no incentive for local farmers to produce the specific crops for school meals, particularly the drought tolerant crops. 149. Finally, WFP should be commended for the ongoing effort they have made during the LRP to include women in the programme. Women have been actively encouraged to participate in all aspects of the programme, and this active recruitment has succeeded, with women making up half the traders sampled (46.3%) and more than half the FO membership. 150.Overall, these findings indicate that while all the planned LRP activities and outputs have been completed by WFP, they have not been effective at improving the timeliness or cost-effectiveness of food assistance. This is largely as a result of MoE providing LRP schools with in-kind commodities instead of the cash￾transfers that were planned, and which the LRP was specifically designed to support. The LRP has however partially succeeded in increasing the capacity of suppliers and school meals procurement committees to effectively and efficiently procure local commodities for school’s meals, however, although none of these groups have been able to put their learning into practice. Similarly, although capacity of school personnel on nutrition and diversity of the school menus has been strengthened though training and the menu development workshops, this training and the resulting menus have not been put to use. The expected boost to the local and regional food market systems that the HGSMP would have provided, as not resulted, although other market activities have helped FOs and traders sell greater volumes. 151. As a programme designed to support the HGSMP, the LRP was never intended to be a sustainable programme in its own right. The HGSMP continues to be implemented by the Government of Kenya, with an annual budget of USD 24 million (2018/19) and with continual lobbying for additional funding. Since the handover to the government in July 2018 WFP now plays an enabling role rather than an implementing role. As a result, there was little that WFP could do to address the MoE’s continued provision of in-kind commodities in the LRP schools except provide technical support to the programme, gather evidence of the viability and benefits of cash transfer modality, and help the MoE improve their in-kind delivery, while 89 Taylor, J.E et al (2019) Potential local economy impacts of Kenya’s Home-Grown School Meals Program. UC Davis. WFP Kenya Endline LRP Evaluation Report – May 2020 |Page 44 continuing to advocate for cash-transfers with the MoE. WFP has managed that role well and have been flexible enough to find other opportunities to expand the reach of the LRP, including bringing in the ECDEs. This decision resulted in the development of a draft ECDE framework that includes supporting a home￾grown feeding programme as part of the ECDE services, with the potential to ultimately result in new opportunities to promote smallholder procurement. 3.1.1 What are lessons learned from the project? 152. The LRP has twice been affected by changes in senior leadership in the LRP partner ministries. 1. A new senior leader in the MoE decided to provide ongoing in-kind commodities throughout the LRP period, despite previous agreement with the MoE that cash-transfers would be provided. This decision has resulted in poor outcomes for the LRP; and 2. The retirement of one of the key MoA champions of the process to develop the National Smallholder Implementation Strategy guiding the direct procurement by public institutions from smallholder farmers, resulted in the approval of the National Implementation Strategy stalling and the expected development of county level strategies being put on hold. While neither of the situations could have been foreseen, the two scenarios are potentially indicative of challenges that WFP will continue to face now that they not responsible for programme implementation. It would therefore be useful for WFP and government partners to discuss what lessons, if any, can be learned from these experiences and determine whether there were any contingency plans that could be put in place to avoid these situations repeating again. 153.Once it was clear that the intended LRP schools would not be used a cash-based methodology as expected, WFP looked for other opportunities to continue to promote smallholder procurement. This flexibility resulted in opening new areas of work, supporting home-grown feeding in the ECDE services, which has strengthened their relationship with county level authorities, and potentially resulted in new opportunities to promote smallholder procurement. WFP programme management should be commended for this flexibility in programming, responding well to the situation at hand. 3.2 Recommendations 154. Based on the findings and conclusions of this evaluation, the recommendations of the evaluation team are outlined below. Priority recommendations (to be implemented before the start of the next school year) • Recommendation 1: WFP and the MoE school feeding technical officers should continue to advocate to the MoE senior leadership on the importance of school feeding, timeliness of delivery, timeliness of cash disbursement and importance of providing daily school meals. This advocacy role can also be done at county level in support of the draft ECDE framework. • Recommendation 2: WFP and MoE should develop a position paper outlining the key decisions over the last decade that have led to the largely cash-based modality of HGSMP that is seen today. The paper should also include the evidence that underpins those decisions including recent market assessments in the arid areas. The paper should then be used for ongoing advocacy with the MoE as described in Recommendation 1. • Recommendation 3: WFP and the MoE school feeding technical officers should consider ways to support improvements in the MoE’s in-kind procurement and delivery system. This is especially important if the MoE intends to continue providing in-kind support to the arid and semi-arid areas which are among the locations most in need of the food and nutrition security improvements that the HGSMP can provide. WFP Kenya Endline LRP Evaluation Report – May 2020 |Page 45 • Key areas of LRP work to continue • Recommendation 4: WFP should continue to work with county/sub-county governments to support farmers, FOs and market system improvement in general. This is especially important for work involving the MoA, as their authority has been devolved to the counties. • Recommendation 5: WFP should continue to work together with MoH at national and county level, on school health and nutrition related work, especially on activities that improve the nutritional status of school-aged children such as the provision of nutritious, locally produced food commodities. • Recommendation 6: WFP should look for funding to continue to the work started through the LRP supporting farmers and FOs in Baringo and West Pokot as the LRP was the first opportunity for support, and without the opportunity to put their learning into practice during the LRP, any gains made are likely to regress without some reinforcement. • Recommendation 7: WFP should continue to work with the MoA and other relevant ministries on the promotion of small holder procurement. This should include continuing the development of county and national level strategies and other means to increase opportunities for small holder engagement in public procurement. • Recommendation 8: WFP should continue to support MoE’s efforts to conduct regular programme monitoring to ensure that HGSMP processes and procedures are being adequately followed. • Recommendation 9: WFP and partners should continue to support the active inclusion of women in all WFP projects and ensure that gender related indicators are present in future WFP projects, as appropriate. WFP Kenya Endline LRP Evaluation Report – May 2020 |Page 46 4 Annexes Annex 1: Endline evaluation terms of reference Introduction These terms of reference (TOR) are for the final activity evaluation of the World Food Programme (WFP) Local and Regional Food Aid Procurement Programme (LRP). The United States Department of Agriculture’s (USDA) funded programme was implemented in Baringo, Turkana and West Pokot Counties. This evaluation is commissioned by WFP Kenya Country office and will cover the period from 2017 to 2020. These TOR were prepared by the WFP Kenya Country Office M&E unit based upon an initial document review consultation with stakeholders and following a standard template. The TOR are aligned to an evaluation plan developed and agreed by both WFP and USDA prior to the commencement of the programme. The purpose of the TOR is twofold: (i) provide key information to the evaluation team and help guide them throughout the evaluation process; and (ii) provide key information to stakeholders about the proposed evaluation. Reasons for the Evaluation The reasons for the evaluation being commissioned are presented below: Rationale This evaluation follows on the baseline survey conducted in 2018 and is being commissioned for the following reasons: The USDA LRP aimed at supporting the Government of Kenya’s effort to expand the national home-grown school feeding programme (HGSMP) into the arid lands while stimulating local economic development through the procurement of food from local structured markets, increasing farmers’ incomes and creating additional jobs in the community. USDA is one of the longest -standing, important donors to WFP School feeding in Kenya. However, WFP Kenya received LRP funds for the first time in 2017. The activities under this programme were implemented for two years starting October 2017 at a total cost of USD 2,825,509. As per grant agreement signed by both USDA and WFP in 2017, there are 24 specific performance and results indicators against which performance of the programme will to be measured. In the evaluation plan agreed between with USDA, WFP committed to conduct a final evaluation to provide an evidence-based, independent assessment of performance of the programme in order to evaluate the project’s success, ensure accountability, and generate lessons learned. Thus, WFP is commissioning an evaluation at the final point of programme implementation. Objectives The main objective of this evaluation is to assess and report on the performance and results achieved (intended or unintended, positive or negative) of LRP programme in the three targeted counties (Baringo, Turkana and West Pokot). The evaluation will serve the dual and mutually reinforcing objectives of accountability and learning. • Accountability – The evaluation will assess and report on the performance and results of the USDA LRP support to WFP School Feeding Programme in Kenya from 2017 to 2020. • Learning – As this is the first LRP support to WFP Kenya, this evaluation will generate useful lessons that WFP Kenya, USDA and the Government of Kenya could use to inform future programmes. Findings will be actively disseminated, and lessons learned will be incorporated into relevant lesson sharing systems. WFP Kenya Endline LRP Evaluation Report – May 2020 |Page 47 Stakeholders and Users Numerous stakeholders both inside and outside WFP Kenya have interests in the results of the evaluation and some of these will play a role in the evaluation process. Table 1 below provides a preliminary stakeholder analysis which may amended by the evaluation team at the inception phase. Accountability to affected populations is tied to WFP’s commitments to include beneficiaries as key stakeholders in WFP’s work. As such, WFP is committed to ensuring gender equality and women’s empowerment in the evaluation process, with participation and consultation in the evaluation by women, men, boys and girls from different groups. Table 1: Preliminary Stakeholders’ analysis Stakeholders Interest in the evaluation and likely uses of evaluation report to this stakeholder INTERNAL STAKEHOLDERS Country Office (CO) Kenya Responsible for the country level planning and operations implementation, it has a direct stake in the evaluation and an interest in learning from experience to inform decision-making. It is also called upon to account internally as well as to its beneficiaries and partners for performance and results of its operation. Regional Bureau (RB) Nairobi Responsible for both oversight of country offices and technical guidance and support, the RB management has an interest in an independent account of the operational performance as well as in learning from the evaluation findings to apply this learning to other country offices. Office of Evaluation (OEV) OEV has a stake in ensuring that decentralized evaluations deliver quality, useful and credible evaluations. OEV management has an interest in providing decision-makers and stakeholders with independent accountability for results and with learning to inform policy, strategic and programmatic decisions. WFP Executive Board (EB) The WFP governing body has an interest in being informed about the effectiveness of WFP operations. The evaluation results will not be presented to the EB, but its findings may feed into corporate learning processes. EXTERNAL STAKEHOLDERS Beneficiaries As the ultimate recipients of assistance, beneficiaries have a stake in WFP determining whether its assistance is appropriate and effective. As such, the level of participation in the baseline of women, men, boys and girls will be determined and their respective perspectives will be sought. More specifically, teachers, school meals procurement committees, students, smallholder farmers, local traders and traders should be considered as key stakeholders. Government, National and County Levels Both county and national governments have a direct interest in knowing whether WFP activities in the country are aligned with its priorities, harmonised with the action of other partners and meet the expected results. The Government has the overall ownership of the school feeding programme and shares the interest in learning lessons for design of future programmes, including transition to cash model. The key line Ministries are: Ministry of Education, Ministry of Agriculture, Livestock and Fisheries, Ministry of Health including relevant Ministries at county level. County and Sub-county Education Officers, School Management Committees are also key as they are involved in school feeding programme implementation and policy support. WFP Kenya Endline LRP Evaluation Report – May 2020 |Page 48 United Nations and Development Partners The Kenya United Nations Development Assistance Framework (UNDAF) should contribute to the realisation of the government developmental objectives. Kenya United Nations Country Team (UNCT) has therefore an interest in ensuring that WFP operation is effective in contributing to the United Nations concerted efforts. WFP implements the programme within a wider UN system of support to government priorities. The partner agencies are interested in learning to what extent WFP interventions are contributing to the overall outcomes committed to the UNDAF particularly UNICEF, UNESCO, FAO, UNDAF thematic working groups, the Education Sector Donors Groups. Donors [USDA, Australia, Germany, Japan, Russia, Private donors] Whereas LRP is funded by USDA, WFP operations are voluntarily funded by a number of donors. This project builds on the existing school feeding programme which is a multi-donor initiative. As such, USDA as well as other donors will have an interest in knowing how findings from this evaluation fit in the larger school meals programme implementation context. The primary users of this evaluation will be: • The Kenya country office and its partners in decision-making, notably related to programme implementation and/or design, Country Strategy and partnerships. • This final evaluation will contribute to the body of knowledge on the LRP projects. USDA may use the findings and lessons learnt from this final evaluation to inform future program funding, design, and implementation decisions. • The RB is expected to use the evaluation findings to provide strategic guidance, programme support, and oversight • WFP HQ may use evaluation findings for wider organizational learning and accountability. Context and subject of the Evaluation Context The Government of Kenya through Ministry of Education (MOE) in collaboration WFP have implemented school meals programme since 1980. The programme targets schools in semi-arid and arid areas where cases of food insecurity, malnutrition and low enrolment rates are prevalent. In 2009, the Government of Kenya started a national home-grown school meal programme (HGSMP) to provide a meal to children at school; to support education achievements while also stimulating local agricultural production through purchase of food from smallholder farmers and local food suppliers. Unlike other school feeding programmes, the HGSMP seeks to deliver simultaneously on ‘local’ economic growth and social protection or poverty education objectives. After close to four decades of joint WFP-Government programming, the transition of school feeding activities to Government ownership was completed in June 2018. Until this time (June 2018), both WFP and MOE provided midday meals to 1.5 million children. WFP support is currently focused on strengthening the capacity of the Government of Kenya towards ensuring a universal and sustainable school feeding programme as envisaged in the National School Meals and Nutrition Strategy (2017-2022). The HGSMP supports access to education while also stimulating local economic development (including agricultural production) through procurement of food from local structured markets, increasing farms’ income and creating additional jobs in the community. HGSMP is an investment in the local economy, linking small-scale farmers and suppliers to an ongoing school market. The programme promotes smallholder farmer productivity and capacity by offering a widespread market that is accessible to farmers due its decentralized procurement approach. Structured Demand markets are markets created by public or non-profit entities that have a predictable and reliable demand for food products. The HGSMP provides a structured demand market. On the demand side, the procuring entity can offer a market and an additional source of income for smallholder farmers through inclusive public procurement processes. On the supply side, structured demand markets like schools can inspire farmer organizations to increase their levels of production and organization WFP Kenya Endline LRP Evaluation Report – May 2020 |Page 49 in order to meet the demands of structured demand markets and other high-value markets. The project aimed to use the structured demand markets of the HGSMP to encourage local agricultural production and strengthen local food supply systems. The HGSMP provides market opportunities for local suppliers (traders, farmer’s organizations and smallholder farmers). However, a 2014 external evaluation of the HGSMP concluded that it has been difficult for farmers’ groups to supply schools without sustained capacity development support. Moreover, it has been established that existing procurement regulations, processes and tools do not fully support the inclusion of smallholder farmers, promote transparency, or ensure schools get value for the funds spent. Food quality and safety is an important aspect in the procurement and handling of locally procured food, there is therefore a need to support County Governments to develop and implement food quality and safety strategies for school feeding programmes (including aflatoxin testing). This is especially pertinent for sub counties that have been implementing HGMSP. Schools have the potential to become powerful platforms for mainstreaming nutrition while promoting lifelong healthy eating habits. Daily school meal plays a considerable impact on the overall daily dietary intake of a student. Currently, for schools in the arid and semi-arid lands, the school basket consists of three food groups, including cereals, pulses and oil, which provides 30 percent of the recommended daily energy requirements but is lacking in dietary diversity and micronutrient intake. The 2014 Kenya Demographic Health Survey (KDHS) revealed that only 21 percent of children in the lowest wealth quintile consumed four or more food groups. Identifying local and culturally appropriate foods, including locally available drought tolerant crops, to add to the school meals menus can contribute to the improvement of the student’s nutritional status. Subject of the evaluation The LRP project was implemented over a period of two years running from U.S fiscal year (FY) 2017 -2020. The project is in support of WFP and the government of Kenya’s ongoing school feeding project in Baringo, Turkana and West Pokot counties. It was envisaged that the project would build on WFP’s accomplishments in supporting the expansion of the government-led school meal programme. Key objectives of the project are to: • Improve effectiveness of food assistance by improving cost-effectiveness and improving timeliness • Increase the capacity of suppliers and school meals procurement committees to effectively and efficiently procure local commodities for school’s meals, promoting sustainability of school feeding; • Strengthen local and regional food market systems, improving access to culturally acceptable commodities and connecting them to Government of Kenya home-grown school meals programme; and • Improve nutrition of students by increasing access to and use of various, high quality nutritious and culturally appropriate foods in school’s meals. • The objectives were to be achieved through a broad set of activities and inputs including: Assessment and Mapping of Local Food Systems and Value Chains This entails WFP support to the government to conduct market assessments of local value chains in Baringo and West Pokot sub￾counties, where the Government of Kenya started providing cash transfers to schools under the Home-Grown School Meals Program (HGSMP) in January 2018. These market assessments aimed to map the production and supply capacity of local traders and farmers to schools. They were expected to be used to estimate the cash transfer rates that the government should use when transitioning schools to the HGSMP. Further, WFP was to support the government to conduct value chain analyses to identify and map locally produced commodities and local agricultural production that are well positioned to participate in the structured demand markets created by the HGSMP. Building upon the Cash Transfers to Schools (CTS) market assessments, the value chain analyses were to focus on the availability, costs and future potential of local, nutritious, and culturally acceptable food to become part of the HGSMP food basket. The assessments were also to consider the local availability of drought-tolerant crops in order to inform the development of diverse menus under the HGSMP. WFP Kenya Endline LRP Evaluation Report – May 2020 |Page 50 Capacity Building for National and County Institutions WFP planned to work in partnership with the Ministry of Education (MOE) to train schoolteachers, parents, and education officers in Baringo and West Pokot on the HGSMP’s Cash Transfer to Schools (CTS) model in order to enhance accountability and transparency in school-based food procurement process. WFP and MOE were to establish a monitoring and oversight plan in Baringo and West Pokot, building upon the WFP Beneficiary Complaints and Feedback mechanism (helpline) and reporting structure developed by WFP and MOE under McGovern-Dole Program Agreement No. FFE-615-2016-014-00. WFP was to support the development of the Government of Kenya’s national implementation strategy to guide direct local procurement from smallholder farmers by government institutions, such as schools, the National Youth Service, hospitals and the Kenya Police Service. WFP in collaboration with the Ministry of Agriculture, Livestock and Fisheries (MOALF) and the United Nations Food and Agriculture Organization (FAO), had obtained consensus from ministries of education, health among others to support procurement from smallholder farmers and agricultural processors by developing an implementation strategy. The strategy was to be first implemented in Turkana, Baringo and West Pokot, before a national rollout. Capacity Strengthening for Local Traders and FOs WFP planned to train local farmer organizations, and traders in Baringo and West Pokot on the key requirements for becoming suppliers to the HGSMP, including HGSMP eligibility criteria for traders, procurement procedures for schools, food quality assurance, and post-harvest handling. WFP was to collaborate with ministries of education, health and agriculture to conduct market linkage forums in Baringo and West Pokot Counties. The purpose of the market of the linkage forums was to provide an opportunity for potential suppliers to interact with school meals procurement committees. Market linkage forums increase awareness, promote transparency during the procurement process, and provide school meals procurement committees with the opportunity to coordinate with suppliers to adjust the HGSMP food basket based on locally available products. Develop School Meals Menus Using Local and Nutritious Produce WFP was to support schools to develop, use, and promote diverse school meals menus based on locally produced, nutritious, and drought-tolerant crops, including sorghum, millet, and cowpeas, in Baringo, West Pokot, and Turkana. WFP planned to analyse the nutrient profile of the selected crops and incorporate them into the school meal basket for select schools, ensuring that the newly diversified school meal basket is culturally acceptable and still meets the nutrient requirements for calories, protein, fats, and micronutrients for primary school students. WFP was to assess the inclusion of locally produced foods and nutrition education on the dietary diversity of the school meals and students’ diet. WFP planned to use this assessment to provide evidence-based recommendations on the minimum standards for school meal composition, working in collaboration with MOE and MOH. WFP planned to advocate for investments in the integration of health and nutrition education into the school curriculum to support the national School Nutrition and Meals Strategy and HGSMP guidelines. Procure Locally Produced, Drought-Tolerant Crops WFP planned to pilot the local procurement of sorghum and cowpeas for schools in Turkana supported under McGovern-Dole Program Agreement No. FFE-615-2016-014-00. WFP will procure these locally produced, drought-tolerant crops using forward delivery contracts (FDCs) and direct contracts issued to six farmer organizations in Turkana County. The farmer organizations have received technical support on cultivating sorghum and cowpeas and using FDC and direct contract mechanisms through WFP activities supported by non-United States Government funds. The locally procured sorghum and cowpeas will diversify the existing food basket, by replacing a portion of the bulgur wheat and green split peas provided under McGovern-Dole Program Agreement No. FFE-615-2016-014-00 for two days per week (20 days per term) for two school-terms. WFP Kenya Endline LRP Evaluation Report – May 2020 |Page 51 The LRP project used USDA cash funding to contribute towards USDA LRP highest strategic objective (LRP SO1); improved effectiveness of food assistance through local and regional procurement. The following activities will contribute to the achievement of LRP SO 1: • Improved cost-effectiveness of Food assistance: Capacity building for national and county institutions • Improved timeliness of Food Assistance: Capacity building for national and county institutions and Capacity strengthening for local suppliers • Improved Utilization of Nutritious and Culturally Acceptable Food that Meet Quality Standards: Assessment and mapping of local food systems and value chains, develop school meals menus using local and nutritious produce and Procure locally produced, drought-tolerant crops • Capacity building for national and county institutions will contribute to the foundational results of increased capacity of government institutions and improved policy and regulatory framework. Capacity strengthening for local suppliers will also contribute to the foundational result of improved capacity of relevant organizations and increased leverage of private sector resources. For a graphical representation of the project’s theory of change, including the linkages among key activities and results, see the results framework in Annex 2 of this document. Further, the performance indicators table gives details of the indicators that will be used to measure performance of the project. Evaluation Approach Scope The evaluation will be for LRP activities implemented from 2017 to 2020 in Baringo, Turkana and West Pokot Counties. The final evaluation will use the internationally agreed criteria of relevance, effectiveness, efficiency, sustainability and impact. As per the agreed-on evaluation plan, this evaluation is to provide an evidence-based, independent assessment of performance of the programme in order to evaluate the project’s success, ensure accountability, and generate lessons learned. Specifically, the final evaluation should: (i) review the project’s relevance, effectiveness, efficiency, impact, and sustainability. (ii) Collect additional data as appropriate for evaluation of the strategic objective and higher-level results. (iii) Assess whether or not the project has succeeded in achieving its objective. (iv) Investigate the project’s overall impact, and (v) identify meaningful lessons learned that WFP, USDA, and other stakeholders can apply to future programming. Worth noting, this evaluation will build upon the baseline study and process monitoring to assess the project’s contribution to LRP strategic objective (Improved effectiveness of Food Assistance through Local and Regional Procurement). It will assess the effect of the program against the following objectives: • Improved cost effectiveness of food assistance • Improved timeliness of food assistance • Improved utilization of nutritious and culturally acceptable foods that meet quality standards Evaluation Criteria and Questions Evaluation criteria: The evaluation will apply the international evaluation criteria of Relevance, Effectiveness, Efficiency, Impact and Sustainability. Gender Equality and Women Empowerment (GEWE) should be mainstreamed throughout the processes. Evaluation Questions: Allied to the evaluation criteria, the evaluation will address the following a set of key questions outlined in Table 2 below. Noteworthy, detailed evaluation design and evaluation questions will be finalized by evaluation team and confirmed by key stakeholders during the inception period. The final evaluation will cover all areas of intervention with specific focus on Baringo, Turkana and West Pokot Counties. Below are the key criteria and broad questions to be evaluated: WFP Kenya Endline LRP Evaluation Report – May 2020 |Page 52 Table 2: Preliminary Key Evaluation Questions Focus Area Key Questions – Final Evaluation Data Sources Relevance Was the project designed to reach the right people with the right type of assistance? Did the project’s implementation lead to meeting the intended beneficiaries’ needs with the right mix of assistance? Is the project aligned with national government’s relevant policies and strategies? Did the project complement other donor-funded and government initiatives? Document review, key stakeholder focus groups; Baseline report and datasets; Effectiveness To what degree have the interventions resulted (or not) in the expected results (outputs and outcomes as per the PMP)? What internal and external factors affect the project’s achievement of intended results? Key stakeholder focus groups; key informant interviews monitoring reports and other programme reports Efficiency Did assistance reach the right beneficiaries at the right time? Impact Were there any unintended outcomes, either positive or negative? What internal and external factors affected the project’s results? Key stakeholder focus groups Sustainability To what extent is it likely that the benefits of the project will continue after the end of the project? What are the key factors that affect the likelihood of sustainability of the results of the project? General What are lessons learned from the project? How can WFP improve future programming, in the context of these lessons learned? Data Availability The evaluation will collect both qualitative and quantitative primary data in line with the PMP. Secondary data will also be used as applicable. Any quantitative data collection will be done with support of WFP Kenya M&E team. The following is a list of background data and information available to the evaluation team. The list is however not exhaustive, and it is expected that the team will expand it at inception phase. • Final evaluation report for the WFP USDA McGovern-Dole International Food for Education and Child Nutrition Program’s Support in Kenya from 2014 to 2016 • Baseline report for the WFP USDA McGovern-Dole International Food for Education and Child Nutrition Program’s Support in Kenya from 2016 to 2020 • Midterm evaluation report for the WFP USDA McGovern-Dole International Food for Education and Child Nutrition Program’s Support in Kenya from 2016 to 2020 • Kenya Country Strategic Plan 2018- 2023 • Standard Project Reports (SPRs) • Annual Country Report • Strategy to Strengthen & Expand the Home-Grown School Meals (HGSM) Programme into the Arid Lands of Kenya (Validated version 2013) • USDA commitment letter for Agreement • Evaluation Plan • National School Meals and Nutrition Strategy (2017-2022) • Baseline of the Local and Regional Food Aid Procurement (LRP) Project in Kenya 2017-2020 To enhance the quality of data and information provided, the baseline team should: WFP Kenya Endline LRP Evaluation Report – May 2020 |Page 53 • Assess data availability and reliability as part of the inception phase expanding on the information provided. This assessment will inform the data collection • Systematically check accuracy, consistency and validity of collected data and information and acknowledge any limitations/caveats in drawing conclusions using the data. Methodology As indicated in the previous sections, the evaluation methodology will be designed by the evaluation team during the inception phase and should: • Employ the relevant evaluation criteria above [relevance, efficiency, effectiveness, impact and sustainability] • Demonstrates impartiality and lack of biases by relying on a cross-section of information sources (variety of documents, interview of a variety of stakeholder groups, including men and women; national and county level) and a transparent sampling process; • Ensures that women, girls, men and boys from different stakeholder’s groups participate and that their different voices are heard and reflected in the final report; • Mainstreams gender equality and women’s empowerment in the way the evaluation is designed, data is collected and analysed; findings are reported, conclusions and recommendations are made. This will enable the team to reflect on lessons and recommendations for the conduct of a gender responsive evaluation that may be of use to future evaluations; • Uses an evaluation matrix as the organizing tool to ensure all key evaluation questions are addressed and the conclusions are based on credible evidence • Ensure the methodology and baseline implementation are ethical and conform to the UNEG Ethical Guidelines for Evaluation. The evaluation team is expected to elaborate appropriate sampling methods for collecting primary quantitative and qualitative data. The evaluation team will draw a statistically representative sample from the sample frame consisting of the total number of 390 schools, 3,000 smallholder farmers in multiple farmer organizations and around 200 local traders spread across the three targeted counties of Baringo, Turkana and West Pokot. The final evaluation will take a programme theory approach based on the results framework. In its execution, the evaluation will draw on the existing body of documented data as far as possible. The evaluation will use mixed methods and triangulate information from different methods and sources to enhance the reliability of findings. In particular, the evaluation will combine qualitative and quantitative approaches to collect field-level data and information from the three arid counties. Separate questionnaires will be applied to the different primary sources of information, focusing on infrastructure, staff, enrolment and attendance, exam scores, completion rates and community involvement in the programme. The qualitative component of the evaluation will use participatory methods where relevant to highlight lessons learned and case studies representative of the interventions. In particular, the methodology will involve focus group discussions with head teachers, school management committee members, government officers, farmer organization representatives and traders, pupils and key informants drawn from education stakeholders. This component will employ relevant interview schedules as a key data collection method which will be collated to provide general impressions of the programme. Fieldwork will be based on a follow-up to the baseline conducted in 2018. Where possible and relevant, before/after comparison will be done through design of comparable sampling strategy. To ensure independence and impartiality, Evaluation Committee and an Evaluation Reference Group will be established, and reference made to Technical Note on Independence and Impartiality for guidance. WFP Kenya Endline LRP Evaluation Report – May 2020 |Page 54 Quality Assurance WFP’s Decentralized Evaluation Quality Assurance System (DEQAS) defines the quality standards expected from this evaluation and sets out processes with in-built steps for Quality Assurance, Templates for evaluation products and Checklists for their review. DEQAS is closely aligned to the WFP’s evaluation quality assurance system (EQAS) and is based on the UNEG norms and standards and good practice of the international evaluation community and aims to ensure that the evaluation process and products conform to best practice. DEQAS will be systematically applied to this evaluation. The WFP Evaluation Manager will be responsible for ensuring that the evaluation progresses as per the DEQAS Step by Step Process Guide and for conducting a rigorous quality control of the evaluation products ahead of their finalization. WFP has developed a set of Quality Assurance Checklists for its decentralized evaluations. This includes Checklists for feedback on quality for each of the evaluation products. The relevant checklists will be applied at each stage, to ensure the quality of the evaluation process and outputs. In addition, to enhance the quality and credibility of this evaluation, an external reviewer directly managed by WFP’s Office of Evaluation in Headquarter will provide: • Systematic feedback on the quality of the draft inception and evaluation reports; and • Recommendations on how to improve the quality of the evaluation. This quality assurance process does not interfere with the views and independence of the evaluation team, but ensures the report provides the necessary evidence in a clear and convincing way and draws its conclusions on that basis. The evaluation team will be required to ensure the quality of data (validity, consistency and accuracy) throughout the analytical and reporting phases. The evaluation team should be assured of the accessibility of all relevant documentation within the provisions of the directive on disclosure of information. This is available in WFP’s Directive (#CP2010/001) on Information Disclosure. Phases and Deliverables This final evaluation will proceed through the five phases presented in Figure 1 below. The evaluation schedule annex provides a detailed breakdown of the proposed timeline for each phase over the full timeframe. A summary of the deliverables and deadlines for each phase are as follows: Figure 1: Summary process map Preparation phase (September–October 2019): The evaluation manager will conduct background research and consultation to frame the evaluation; prepare the TOR; select the evaluation team and contract the company for the management and conduct of the evaluation. The TOR will be shared with USDA for comments and or inputs. Inception phase (November 2019): This phase aims to prepare the evaluation team for the evaluation phase by ensuring that it has a good grasp of the expectations for the evaluation and a clear plan for conducting it. The inception phase will include a desk review of secondary data and initial interaction with the main stakeholders. The inception report will be shared with USDA for comments and or inputs. Preparation Inception Inception Report Evaluation Reporting Evaluation Report Dissemination and follow-up WFP Kenya Endline LRP Evaluation Report – May 2020 |Page 55 Evaluation phase (January – February 2020): The fieldwork will be completed in one month owing to the number of project sites and primary and secondary data collection from local stakeholders. A debriefing session will be held upon completion of the field work. Reporting phase (March 2020): The evaluation team will analyse the data collected during the desk review and the field work, conduct additional consultations with stakeholders, as required, and draft the evaluation report. The draft evaluation report will be submitted to the evaluation manager for quality assurance. Stakeholders will be invited to provide comments, which will be recorded in a matrix by the evaluation manager and provided to the evaluation team for their consideration before report finalisation. Follow-up and dissemination phase: The final evaluation report will be shared with the relevant stakeholders. The management responsible will respond to the evaluation recommendations by providing actions that will be taken to address each recommendation and estimated timelines for taking those actions. The evaluation report will also be subject to external post-hoc quality review to report independently on the quality, credibility and utility of the evaluation in line with evaluation norms and standards. The final evaluation report will be published on the WFP public website. Findings will be disseminated, and lessons will be incorporated into other relevant lesson sharing systems. The expected deliverables from the evaluation are as follows: • Inception report written following WFP recommended template. The report should include but not limited to: • Detailed baseline design • Quality Assurance Plan • Detailed work plan, including, timeline and activities • Bibliography of documents/secondary data sources utilised; • Final data collection tools, data bases, analysis plan • Power-point on methodology, overall survey plan, timeline and activities • Final report, including a first draft, and a final report using WFP recommended template. The final report should include an executive summary and recommendations for optimizing both project implementation and monitoring. Annexes to the final report include but not limited to a copy of the final ToR, bibliography, list of sampled schools, detailed sampling methodology, Maps, A list of all meetings and participants, final survey instruments etc. • Clean data set • Transcripts from key informant interviews, focus group discussions, etc. • Table of all standard and custom indicator baseline values • List of sites visited • Power-point presentation of main findings and conclusions for de-briefing and dissemination purposes Organization of the Evaluation Evaluation Conduct The evaluation team will conduct the evaluation under the direction of its team leader and in close communication with the independent evaluation manager appointed by WFP to manage the evaluation in line with WFP decentralized evaluation guidelines. The team will be hired through an HR process following agreement with WFP on its composition and in line with the baseline schedule. The team members will not have been involved in the design or implementation of the subject of evaluation or have any other conflicts of interest. Further, they will act impartially and respect the code of conduct of the evaluation profession. The evaluation will be supported by WFP M&E team specifically in planning and implementation of quantitative data collection where applicable. WFP Kenya Endline LRP Evaluation Report – May 2020 |Page 56 Team composition and competencies The final evaluation will be led by a senior evaluator and will be supported by a statistician. The duo will be external evaluators with no prior association with the subject of evaluation. The team will conduct this baseline in adherence to DEQAS standards (where applicable) and requirements of the UNEG Norms and Standards and Code of Conduct for Evaluators. The Team Leader will be a senior evaluator with at least 20 years of experience in evaluation of complex interventions, with demonstrated expertise in managing multidisciplinary teams. The team leader will preferably have experience evaluating school meals and or education/capacity related interventions, and experience working in Kenya. She/he will be confident in using mixed quantitative and qualitative methods and will have good understanding of home grown school meals programmes, nutrition/ procurement of local commodities to support school meals programs and additional significant experience in other development and management positions. The Team Leader will also have expertise in designing methodology and data collection tools and demonstrated experience in leading similar evaluations. She/he will also have leadership and communication skills, including a track record of excellent writing and presentation skills. Her/his primary responsibilities will be: (i) defining the baseline approach and methodology; (ii) guiding and managing the team; (iii) leading the baseline mission and representing the baseline team; (iv) drafting and revising, as required, the inception report, the end of field work i.e. (exit) debriefing presentation and baseline report. The statistician will have at least five years of experience in data analysis for development and humanitarian organizations. He/she must have a strong demonstrated knowledge of quantitative methods. The statistician primary responsibilities will be: (i) provide statistical expertise in development of the evaluation design; (ii) develop data collection and analysis plans; (iii) lead data collection preparation and actual collection with support from WFP M&E team; (iv) perform all required analysis and support drafting and revision of the final evaluation report. The team will abide by the Code of Conduct for evaluators (attached to individual contracts), ensuring they maintain impartiality and professionalism. Security Considerations Security clearance will be obtained prior to travel as applicable and in line with applicable standard operating procedures. The United Nations Department of Safety & Security (UNDSS) system covers consultants hired through HR process for United Nations including consultants contracted directly by WFP. The consultants will be required therefore obtain UNDSS security clearance for travelling from designated duty station. In addition, the consultants are expected to complete BSAFE, the online security awareness training course in advance and print out their certificates. However, to avoid any security incidents, the Evaluation Manager will ensure that: The WFP CO registers the team members with the Security Officer on arrival in country and arranges a security briefing for them to gain an understanding of the security situation on the ground. The team members observe applicable UN security rules and regulations. The WFP CO registers the team members with the Security Officer on arrival in country and arranges a security briefing for them to gain an understanding of the security situation on the ground. The team members observe applicable UN security rules and regulations – e.g. curfews etc. Roles and Responsibilities of Stakeholders The Kenya Country Office: The Kenya country Office management (Deputy Country Director will take responsibility to: • Ensure an independent Evaluation Manager for the evaluation: • Compose the internal evaluation committee and the external evaluation reference group • Approve the final TOR, inception and evaluation reports. • Ensure the independence and impartiality of the evaluation at all stages, including establishment of an Evaluation Committee and of a Reference Group WFP Kenya Endline LRP Evaluation Report – May 2020 |Page 57 • Participate in discussions with the evaluation team on the evaluation design and the evaluation subject, its performance and results with the Evaluation Manager and the evaluation team • Organise and participate in two separate debriefings, one internal and one with external stakeholders • Oversee dissemination and follow-up processes, including the preparation of a Management Response to the evaluation recommendations Evaluation Manager/s: • Manages the evaluation process through all phases including drafting this TOR • Ensure quality assurance mechanisms are operational • Consolidate and share comments on draft TOR, inception and evaluation reports with the evaluation team • Ensures expected use of quality assurance mechanisms (checklists, quality support etc.) • Ensure that the team has access to all documentation and information necessary to the evaluation; facilitate the team’s contacts with local stakeholders; set up meetings, field visits; provide logistic support during the fieldwork; and arrange for interpretation, if required. • Organise security briefings for the evaluation team and provide any materials as required • Chairs the External Reference Group meetings An Internal Evaluation Committee will be formed as part of ensuring the independence and impartiality of the evaluation. The membership includes M&E officer, evaluation manager, technical unit in charge of school feeding programme, Deputy (country director programmes), One staff each from finance and logistics unit. The key roles and responsibilities of this team includes providing input to evaluation process and commenting on evaluation products. An External Evaluation Reference Group will be formed, with representation from USDA/FAS, Ministry of Education, Feed the Children, WFP Country office and Regional Bureau and will review the evaluation products as further safeguard against bias and influence (See annex 5; External reference Group TOR). The Regional Bureau in Nairobi: The RB management will be expected to: • Assign focal point for the evaluation. • Participate in discussions with the evaluation team on the evaluation design and on the operation, its performance and results. In particular, the RB should participate in the evaluation debriefing and discussions with the evaluation manager and team, as required. • Provide comments on the TORs, inception report and the evaluation report. Headquarters: Some HQ divisions might, as relevant, be asked to discuss WFP strategies, policies or systems in their area of responsibility and to comment on the evaluation TOR and report. United States Department of Agriculture (USDA) will review, comment and approve the baseline TOR; serve as a member of baseline reference group; participate in a key informant interview phone call with the selected evaluator prior to fieldwork; and participate in stakeholder meetings and presentation of the evaluation findings. The partnerships officer (Washington Office) will work closely with the WFP CO, RB, OEV and the USDA to ensure smooth communication and submission of key baseline deliverables. The partnerships officer will review baseline deliverables for adherence to USDA policy and facilitate communication with USDA; Provide feedback on draft TORs and draft baseline report; coordinate with the donor (USDA) to seek feedback of TORs, inception and baseline reports. The Office of Evaluation (OEV). OEV will advise the Evaluation Manager and provide support to the evaluation process where appropriate. It is responsible to provide access to independent quality support mechanisms reviewing draft inception and evaluation reports from an evaluation perspective. It also ensure a help desk function upon request from the Regional Bureaux. WFP Kenya Endline LRP Evaluation Report – May 2020 |Page 58 Other Stakeholders (Government, NGOs, and United Nations agencies) will be identified for interviews by the evaluation team in addition to the list provided by WFP which will be based on the preliminary stakeholder analysis detailed in table 1. Communication To enhance the learning from this evaluation, the evaluation team should place emphasis on transparent and open communication with key stakeholders. These may for example take place by ensuring a clear agreement on channels and frequency of communication with and between key stakeholders. Communication with evaluation team and stakeholders should go through the Evaluation manager. As part of the international standards for evaluation, WFP requires that all evaluations are made publicly available. Following the approval of the final evaluation report, dissemination will be broad, and workshops will be conducted both internally and with partners, looking at the recommendations and the way forward. Specifically; WFP Kenya Country Office will organize an internal workshop to discuss evaluation findings and recommendations, where the consultant will present the key findings; WFP in collaboration with the Ministry of Education, Science and Technology will organize a workshop targeting relevant external audiences, where the consultant will present the key findings. WFP will discuss the report with USDA and disseminate the findings and recommendations in various ways, including through discussions with WFP senior management and staff as well as with the key partners including ministries of education, health, agriculture as well as non-governmental partners and United Nations agencies. WFP Kenya Endline LRP Evaluation Report – May 2020 |Page 59 Annex 2: Key LRP stakeholders and their interest in the endline evaluation Source: Endline Evaluation ToR provided by WFP Kenya Stakeholders Interest in the evaluation and likely uses of evaluation report to this stakeholder INTERNAL STAKEHOLDERS Country Office (CO) Kenya Responsible for the country level planning and operations implementation, it has a direct stake in the evaluation and an interest in learning from experience to inform decision-making. It is also called upon to account internally as well as to its beneficiaries, partners and donors for the programme’s performance and results. Regional Bureau (RB) Nairobi Responsible for both oversight of country offices and technical guidance and support, the RB management has an interest in an independent account of the operational performance as well as in learning from the evaluation findings to apply this learning to other country offices. Office of Evaluation (OEV) OEV has a stake in ensuring that decentralized evaluations deliver quality, useful and credible evaluations. OEV management has an interest in providing decision-makers and stakeholders with independent accountability for results and with learning to inform policy, strategic and programmatic decisions. WFP Executive Board (EB) The WFP governing body has an interest in being informed about the effectiveness of WFP operations. The evaluation results will not be presented to the EB but its findings may feed into corporate learning processes. EXTERNAL STAKEHOLDERS Beneficiaries As the ultimate recipients of assistance, beneficiaries have a stake in WFP determining whether its assistance is appropriate and effective. As such, the level of participation in the baseline of women, men, boys and girls will be determined and their respective perspectives will be sought. More specifically, teachers, school meals procurement committees, students, smallholder farmers, local traders and traders should be considered as key stakeholders. Government, National and County Levels Both county and national governments have a direct interest in knowing whether WFP activities in the country are aligned with its priorities, harmonised with the action of other partners and meet the expected results. The Government has the overall ownership of the school feeding programme and shares the interest in learning lessons for design of future programmes, including transition to cash model. The key line Ministries are: Ministry of Education, Ministry of Agriculture, Livestock and Fisheries, Ministry of Health including relevant Ministries at county level. County and Sub-county Education Officers, School Management Committees are also key as they are involved in school feeding programme implementation and policy support. United Nations and Development Partners The Kenya United Nations Development Assistance Framework (UNDAF) should contribute to the realisation of the government developmental objectives. Kenya United Nations Country Team (UNCT) has therefore an interest in ensuring that WFP operation is effective in contributing to the United Nations concerted efforts. WFP implements the programme within a wider UN system of support to government priorities. The partner agencies are interested in learning to what extent WFP interventions are contributing to the overall outcomes committed to the UNDAF particularly UNICEF, UNESCO, FAO, UNDAF thematic working groups, the Education Sector Donors Groups. Donors [USDA, Australia, Germany, Japan, Russia, Private donors] Whereas LRP is funded by USDA, WFP operations are voluntarily funded by a number of donors. This project builds on the existing school feeding programme which is a multi-donor initiative. As such, USDA as well as other donors will have an interest in knowing how findings from this evaluation fit in the larger school meals programme implementation context. WFP Kenya Endline LRP Evaluation Report – May 2020 |Page 60 Annex 3: Graphic representation of the relationship between project stakeholders Source: Amended from a diagram originally developed by the LRP Evaluation team during the baseline inception phase WFP Kenya Endline LRP Evaluation Report – May 2020 |Page 61 Annex 4: Results Framework – LRP Kenya WFP Kenya Endline LRP Evaluation Report – May 2020 |Page 62 Annex 5: Completed LRP Performance Monitoring Plan LRP – World Food Programme - Kenya Targets Baseline values October 1, 2017 - March 31, 2018 Endline: Life of project values October 2017- September 2019 # Standard/ Custom # Result Performance Indicator Year 1 Year 2 Life of project 1 Oct 2017 – 30 Sept 2018 1 Oct 2018 – 30 Sept 2019 1 Standard #1 LRP SO 1 Number of individuals benefiting directly from USDA-funded intervention 31,150 31,150 31,150 721 54,895 2 Standard #2 LRP SO 1 Number of individuals benefiting indirectly from USDA-funded intervention 61,000 61,000 61,000 0 162,082 3 Standard #3 LRP 1.3.2.2 Value of sales by project beneficiaries 582,524 597,087 1,179,612 West Pokot: KES 24,433,00/ USD 244,433 Baringo: KES 16,974,366/ USD 169,744 Turkana: No value given Turkana: USD 147,655 4 Standard #4 LRP 1.3.2.1 Volume of commodities (metric tons) sold by project beneficiaries 415 150 565 West Pokot: 9,334.3MT Baringo: 2020.4MT Turkana: 16.4 MT Turkana: 349.6 mt 5 Standard #5 LRP 1.4.3/ 1.4.4 Number of public-private partnerships formed as a result of USDA assistance 220 220 220 5 FOs (9 contracts) 5 FOs (12 contracts) 6 Standard #6 LRP 1.4.3/ 1.4.4 Value of public and private sector investments leveraged as a result of USDA assistance 582,524 597,087 1,179,612 0 Turkana: USD 147,655 8 Standard #8 LRP 1.4.1/ 1.4.2 Number of policies, regulations and/or administrative procedures in each of the following stages of development as a result of USDA assistance 1 2 3 0 0 • National implementation Strategy in progress • ECDE Framework drafted WFP Kenya Endline LRP Evaluation Report – May 2020 |Page 63 9 Standard #9 LRP 1.3 Quantity of commodity procured (MT) as a result of USDA assistance (by commodity and source country) 365 0 365 0 Turkana: 349.6 mt 10 Standard #10 LRP 1.3 Cost of commodity procured as a result of USDA assistance (by commodity and source country) 165,120 0 165,120 0 Turkana: USD 147,655 12 Standard #12 LRP 1.1/ 1.2 Cost of transport, storage, and handling of commodity procured as a result of USDA assistance (by commodity) 100,301 0 100,301 0 18,242 13 Standard #13 LRP SO 1 Number of social assistance beneficiaries participating in productive safety nets as a result of USDA assistance 30,000 30,000 30,000 0 50,564 (All the children in the schools that received FDC commodities from WFP) 14 Standard #14 LRP 1.3.2 Number of individuals who have received short￾term agricultural sector productivity or food security training as a result of USDA assistance 200 200 400 2,535 3,099 #1 Custom N/A Cost of distribution in schools in LRP areas compared to non LRP areas TBD TBD TBD __ N/A #2 Custom N/A Percentage of LRP schools procuring food before beginning of the term 80% 90% 90% N/A 0 #3 Custom N/A Percentage of schools where food is delivered to schools before school term begins 80% 90% 90% 19.8% 0 #4 Custom N/A Percentage of schools using diversified school meals menus 50% 100% 100% 0% 0 WFP Kenya Endline LRP Evaluation Report – May 2020 |Page 64 Annex 6: List of LRP baseline recommendations (2018) • Recommendation 1: WFP and partners should continue to implement the LRP project as per the current agreement (2017-2020) including WFP providing technical support to the MoE and LRP schools as they transition to the HGSMP. • Recommendation 2: WFP should undertake additional data collection in LRP schools once they have transitioned to the HGSMP. This should include collection of timeliness and cost data and recalculation of the timeliness and cost-effectiveness scores. • Recommendation 3: WFP and the MoA should ensure that the planned LRP market linkage forums provide opportunities for FOs to meet with local traders. • Recommendation 4: WFP should continue to support the development of a framework that supports and prioritizes procurement from local farmers. This may include uncoupling commodities and allowing FOs to provide only the items they grow and not the whole school meals basket. • Recommendation 5: WFP and the MoH should collaborate to develop practical guidance for LRP schools on ways to introduce locally produced drought tolerant crops into their school menus within their limited resources. • Recommendation 6: WFP and the MoH should collaborate to ensure that schools receive regular nutrition support since this is a new project area. • Recommendation 7: WFP should support the MoE to review the HGSMP transfer rate once updated market assessment findings are available for Baringo and West Pokot at end of 2018/early 2019. • Recommendation 8: WFP and the MoA should continue to support farmers to improve their production capacity. • Recommendation 9: WFP should continue to support County Governments to develop and implement food quality and safety strategies for school feeding programmes (including aflatoxin testing). WFP Kenya Endline LRP Evaluation Report – May 2020 |Page 65 Annex 7: Maps of Kenya showing WFP and government supported school feeding activities during LRP baseline (March 2018) and at endline (October 2019). • Map 1 – March 2018 Source: WFP Kenya • Map 2 – October 2019 Source: WFP Kenya WFP Kenya Endline LRP Evaluation Report – May 2020 |Page 66 Annex 8: Evaluation Matrix Evaluation questions Baseline questions Indicator Main sources of information Data collection methods Main method of data analysis Data availability RELEVANCE Determine if the LRP project design is relevant and coherent with key policies and strategies and identify any shortcomings. Was the project designed to reach the right people with the right type of assistance? i.e. were women and girls appropriately targeted, and their relevant needs met? Does the project meet a specific need? How was targeting done? How were schools chosen for the LRP? How were Farmer Organizations chosen? How were suppliers chosen? Have any key groups been excluded? Do stakeholders have suggestions for improving the LRP design and/or targeting criteria? Clear project purpose. Clear targeting criteria and rationale for selection of all beneficiary groups (FOs, supplier and schools). LRP project related documents including market assessments if available. Background documents on the context of agriculture in the targeted counties. Interviews with key LRP stakeholders. Secondary document review. Qualitative interviews with WFP personnel, school personnel, FOs and traders. Descriptive analysis. Triangulation of documents and results from key informant interviews. Project documentation and policy documents are available. WFP to provide additional documentation when available. Information from stakeholders will be collected in the field. Information on interventions by other partners to be collected by the team with support from WFP and through KII interviews as appropriate. Is the project aligned with national government’s relevant policies and strategies, including gender policies? Does the current design of the LRP align with relevant policies and strategies of the Government of Kenya and WFP? Does the LRP design align with other WFP work in Kenya? Does the LRP design align with the work of other development partners working in the same sector? Do stakeholders have suggestions for improving the LRP design? Alignment of the LRP approach with government objectives and legislative direction. LRP project related planning and implementation documents. Government and WFP policy documents. Interviews with key LRP stakeholders. Secondary document review. Qualitative interviews with MoE representatives at national and county level. Does the project complement other donor￾funded and government initiatives? Alignment of the LRP approach with the direction of other development actors. Evidence of partnerships. Evidence of complementarity and no evidence of duplication. Documentation of projects by other development actors at national and district level. Interviews with other development partners. Secondary document review. Qualitative interviews with other development actors. WFP Kenya Endline LRP Evaluation Report – May 2020 |Page 67 Evaluation questions Baseline questions Indicator Main sources of information Data collection methods Main method of data analysis Data availability EFFECTIVENESS Determine if the activities outlined in the LRP are likely to result in an effective programme. Has the project resulted in increased access to markets for sale of agricultural products? E.g. were women farmers targeted and supported to increase their access to markets for sale (or provide gender disaggregated data) Is the produce from targeted FOs being sold to schools? What are the current volumes of food commodities (and type) being produced by the FOs? (compared to how much food schools need) i.e. Are producers able to meet demand? Is it likely that project activities will enable farmers to supply adequate volumes of food for the schools in their locations? If not, what are the current challenges faced by farmers? Current value of sales by project beneficiaries (FOs and traders) Current volume of commodities (metric tons) sold by project beneficiaries. Ability of FOs and traders to access the school procurement process. Identified support required by schools, FOs and traders. Quantitative survey of key LRP stakeholders. FO and trader records (if required). WFP monitoring data. Interviews with key LRP stakeholders. Secondary document review. Qualitative interviews with key LRP stakeholders. Quantitative survey of FOs and traders. Quantitative data analysis of survey findings. Comparison of case and control school findings. Triangulation of quantitative survey findings and results from key informant interviews. Information from stakeholders will be collected in the field. WFP to provide monitoring data as appropriate. Has the project contributed to strengthening local and regional food market systems? Has the presence of the HGSMP had any impact on the local markets? Changes in price? Volume? Etc. Ability of FOs and traders to meet demand. Number of traders and local farmers contributing to HGSMP Evidence of improved linkages between farmers and schools Quantitative survey of key LRP stakeholders WFP monitoring data and market surveys Interviews with key LRP stakeholders. Quantitative survey of FOs and traders. Has the project resulted in improved access to culturally acceptable foods? E.g. have the nutritional needs of boys and girls been met? Is there evidence of increased access to and use of various, high quality nutritious and culturally appropriate foods in school’s meals? What are the barriers to this? Evidence that schools use locally procured food to provide diversified school meals e.g. menus. Number of schools using diverse menus Interviews with key LRP stakeholders. Quantitative survey of school personnel. WFP Kenya Endline LRP Evaluation Report – May 2020 |Page 68 Evaluation questions Baseline questions Indicator Main sources of information Data collection methods Main method of data analysis Data availability Has the project resulted in improved cost-effectiveness of food assistance delivery? What are the costs of providing food assistance to schools – through HSGMP and through other means? Comparison of the price of school meals of the HGSMP and the in-kind modality being used in Turkana Cost for procurement, distribution and delivery of food under the two modalities Endoline to establish changes in the cost of food assistance provision over time. WFP financial data Secondary document review. Qualitative interviews with WFP Quantitative data analysis of WFP data WFP to provide monitoring data as appropriate. EFFICIENCY Determine if the activities outlined in the LRP are likely to result in an efficient programme. Has the project improved the timeliness of food assistance? Has the project improved the timeliness of food procurement for schools? Has the project improved the timeliness of food delivery to schools? Has the project improved the timeliness of food distribution schools? What is the current procurement process for selling food commodities to schools? What are the barriers to access? What is the current lead-time for procurement? For delivery? And for distribution? How often does LRP face pipeline breaks compared with other food procurement modalities (i.e. Turkana)? Timeliness of food delivery to LRP schools (Term 1, 2018): • Number of schools receiving food prior to start of term. • Dates when funding provided to schools. • Tendering process start dates. • Food delivery dates • Distribution dates Frequency and duration of pipeline breaks. Number of school feeding days Quantitative survey of key LRP stakeholders. School records (if required). Documentation from tender process of Term 1, 2018. Secondary document review. Qualitative interviews with key LRP stakeholders Quantitative data analysis of survey findings. Comparison of case and control school findings. Information from stakeholders will be collected in the field. WFP to provide monitoring data as appropriate How many schools are currently receiving their food commodities before the first day of school term? Do stakeholders have suggestions for improving the timeliness of procurement? Quantitative survey of FOs, traders and school personnel. Triangulation of quantitative survey findings and results from key informant interviews. WFP Kenya Endline LRP Evaluation Report – May 2020 |Page 69 Evaluation questions Baseline questions Indicator Main sources of information Data collection methods Main method of data analysis Data availability IMPACT Determine if the LRP is likely to have any unintended positive or negative outcomes or impact. What are the possible unintended outcomes, either positive or negative? Is the project taking into consideration an appropriate mitigation strategy? Can these unintended outcomes be gender disaggregated? Can stakeholders identify any potential unintended outcomes or impacts of the project? And identify mitigation measures? Stakeholder able to identify potential negative outcomes or impacts. Potential mitigation strategies identified and/or already actioned. LRP project documents re potential risks identified during design phase and mitigation measures undertaken. Qualitative interviews with key LRP stakeholders. Triangulation of results from key informant interviews and quantitative survey results. LRP project documents are available from WFP. Information from stakeholders will be collected in the field. Do the other assessment findings indicate that a change to project design is required in order to prevent negative impacts? Interviews with key LRP stakeholders. Qualitative interviews with key LRP stakeholders. Triangulation of results from key informant interviews and quantitative survey results. LRP project documents are available from WFP. Information from stakeholders will be What internal and collected in the field. external factors are likely to affect the project’s results? What are the key factors that are likely to contribute to the success or otherwise of the LRP? Interviews with key LRP stakeholders. SUSTAINABILITY Determine if the results of the LRP are likely to be sustainable. What are the challenges that could affect the sustainability of the programme? To what extent is it likely that the benefits of the project will continue after the end of the project? Are the LPR activities likely to produce sustainable results? Can stakeholders identify any specific barriers to sustainability? What is the involvement of the government? How much input is from WFP vs. Government? Stakeholders believe that project will be sustainable. Evidence of sufficient government involvement and funding. Evidence of partnerships and leveraging private funding. Interviews with key LRP stakeholders. Qualitative interviews with key LRP stakeholders. Triangulation of results from key informant interviews and quantitative survey results. LRP project documents are available from WFP. Information from stakeholders will be collected in the field. What are the key factors that are likely to affect the sustainability of the results of the project? Multiple stakeholders identify similar factors that are likely to affect to sustainability of the project. WFP Kenya Endline LRP Evaluation Report – May 2020 |Page 70 Annex 9: List of data and documents provided to the evaluation team by WFP Kenya • 190701 Final Value Chain Study Report Baringo County .pdf • 190701 Final Value Chain Study Report West Pokot County.pdf • 200680 SPR 2017- Published Version.pdf • ACR- Final Published Report.pdf • AMAL FO profiles and contacts -Turkana.xls • Approved CSP 2018.pdf • Baringo and West Pokot LRP Supplier Information-excel sheet.xlsx • Copy of WFP Kenya - Performance Monitoring Plan_ver2.xlsx • Final Baseline Report 13 November.pdf • Kenya LRP-615-2017-035-00- Signed Agreement.pdf • Kenya MGD SF Midline Report Final 31 October 2018 Revised 18 June 2019.pdf • Kenya- LRP biannual narrative report -April -September 2019 reviewed.docx • Kenya- LRP Semi-annual Narrative Report - Oct 2018 -March 2019.docx • Semi-annual LRP Report Narrative April- September 2018.docx • Semi-annual Report Narrative September 2017- March 2018- LRP.docx • SPR-200680-12.02.2019.pdf • West Pokot Food Supply Chain Assessment Report - Final- 28.09.2018.pdf WFP Kenya Endline LRP Evaluation Report – May 2020 |Page 71 Annex 10: Quantitative survey tools Local and Regional Procurement Project Endline Evaluation Survey Questionnaire for Traders/Suppliers Introduction INTERVIEW DATE ______/______/______ DD MM YY INTERVIEW START TIME: INTERVIEW END TIME: A. Personal Information Full Names of Business Owner Address Mobile/Phone No. Email address Gender of owner (Tick one) r Male r Female Age bracket r 18-35 r 36-55 r Over 55 County Sub￾County Trading Centre Educational Qualifications (Please tick the highest education level attained) rPrimary r Secondary rDiploma rDegree B. Business Information Full Names of Business Which year did the business begin? No. of Employees Gender of Employees r Male ……………. r Female …………. Does your business have the following licenses? Valid trade license r Yes r No A business registration certificate r Yes r No Any other related licenses ……………………………………………………………………… WFP Kenya Endline LRP Evaluation Report – May 2020 |Page 72 C. Grain Trading 1.What staple commodities do you trade in? Cereals r Maize r Rice r Sorghum r Bulgur wheat r Others Cereals …………………………………………………… …………………………………………………… …………………………………………………… …………………………………………………… …………………… Pulses r Beans r Peas r Cowpeas r Pigeon peas r Green grams r Others Pulses ……………………………………………………………… ……………………………………………………………… ……………………………………………………………… ……… 2. Please estimate total volume of commodities you sell each year ________________________ MT per year 3. From whom do you buy most of your commodities? r Small Scale farmers If yes, please estimate the number of smallholder farmers you purchase from each year r<50 r 50-100 r >150 r Large Scale farmers r Farmer Organizations Please estimate the volume of commodities purchased from FOs ……………. (MT) r Other traders/middlemen r Others (specify) 4. When you have to transport commodities, what means of transportation do you usually use? r Carried (head or back) r Bicycle/Motorcycle r Pick up r Lorry WFP Kenya Endline LRP Evaluation Report – May 2020 |Page 73 (multiple answers allowed) r Public Transport r None/ Don’t transport 5. What facilities do you use for long – term storage? rBasic earth granaries (traditional storage) rPermanent Owned Warehouse (Belongs to the trader) Capacity in MT ……………… rPermanent Rented Warehouse 6. Current volume of grains in KShs./Stock (MT/Bags) _______________ KShs _______________MT 7. To whom do you sell your commodities? (multiple answers allowed) rHouseholds/individuals rRetail store rMillers/brewers/processors rOther traders rHospitals rGovernment Food agencies rInternational Development Agencies rSchools rNGOs Others…………………………………………………………….………………………………………………… ……………......………………………………………………………………………….…………………………… …………… 8. Did you sell commodities to schools through a competitive tender between 2016-2017? r Yes If yes, were you able to provide all the required food items as ordered before the start of school term? r Yes r No If no, why not? r Order from school came too late r Unable to locate some required items r Delayed due to problems of transport r Other (specify) r No If no, have you bid but not won? r Yes r No 9. Have you ever received any training on how to prepare bids for tender, or any similar training? r Yes If yes, please provide details of which agency and year of training. r No WFP Kenya Endline LRP Evaluation Report – May 2020 |Page 74 10. What are the main barriers to you purchasing higher volumes of food commodities from smallholder farmers? r Volume of commodities produced by each farmer r Purchase price r Transportation of commodities￾r Other _________________________________________________________________ _________________________________________________________________ 11. What are the ______ major challenges encountered in grain trading? Please rate between 1(least)-5(major)s Lack of finances _____________________________ Lack of proper equipment _____________________________ Limited market/market information _____________________________ High transport costs _____________________________ Any other major challenges _____________________________ THANK YOU VERY MUCH FOR YOUR PARTICIPATION WFP Kenya Endline LRP Evaluation Report – May 2020 |Page 75 Agricultural Market Access and Linkages for Smallholder Farmers Baseline Survey Questionnaire for Farmer Organizations Introduction My name is and I am working for the World Food Programme /. Thank you [name of organization] for making time to meet with us. We are conducting a survey of farmers’ organizations and would like to get information about groups’ governance, production, assets, marketing of its members’ commodities, financial management, trainings and sustainability. The United Nations World Food Programme is the world's largest humanitarian agency fighting hunger worldwide. The mission of WFP is to help the world achieve Zero Hunger in our lifetimes. Every day, WFP works worldwide to ensure that no child goes to bed hungry and that the poorest and most vulnerable, particularly women and children, can access the nutritious food they need. We would like to talk with you to understand the group better so that we can plan and support you in a more informed way. The survey is voluntary and we shall not use the information for any other reason than aforementioned. Do you have any questions? Questionnaire number Name of the interviewer WFP Kenya Endline LRP Evaluation Report – May 2020 |Page 76 Background Information County: Sub-county: Village name: Urban/Rural: Name of farmers’ organization Contact Person Mobile Number Name of respondent Position in the farmers’ organization Date of interview |__|__| |__|__| 20|__|__| Day Month Year Section A. Group Governance A1. In what year was this farmer’s organization established? |__||__||__||__| A1.1 Is your farmer’s organization legally registered? If No skip to A2 1 = Yes 2 = No A 1.2 In what year was this farmer’s organization legally registered? |__||__||__||__| A2. How many registered members does the organization have? How many have paid their annual membership fee to date? How many elected leaders (committee members) does the organization have? A B C 1 Men |__|__|__|__|__| |__|__|__|__|__| |__|__|__|__|__| 2 Women |__|__|__|__|__| |__|__|__|__|__| |__|__|__|__|__| A3. How many members of the organization are transitioned from food assistance Men Women |__|__|__|__|__| |__|__|__|__|__| A 3.1 No Formal education Primary education Secondary education Post-Secondary education How many of the committee members have? (completed) A4. Is there clear division of responsibility for the committee members? (Each committee members role should be clearly defined and also executed) (Probe 1 = Yes 2 = No A5. Are executive committee members held accountable for management of the group? 1 = Yes 2 = No A5.I Please indicate how 1. Regular elections 2. Minimum mandatory committee meeting attendance 3. Performance of roles/activities assigned in the Committee A6. Is there an incentive to be a committee member? (Reward (payment or in-kind) to committee members 1 = Yes 2 = No WFP Kenya Endline LRP Evaluation Report – May 2020 |Page 77 depending on amount of responsibility and this should be documented) A7. How many group members are active? A8. Type of farmer organization/group? (Circle the answer) 1 = Self Help Group 2 = Community Based Organization (CBO) 3 = Cooperative/FO with trading company 4 = Other (specify) A8.I What governance system governs your group? 1. A constitution 2. Articles of association 3. Other A9. How often does your organization hold elections? (Circle the answer) 1 = Every year 2 = Every two years 3 = Every three years 4= Other (specify) A10. Were elections done as per the governance system if there is one? (Refer to A8.1) 1 = Elections done as per the governance system stipulations 2 = Elections not done as per the governance system stipulations A11. How often does the organization hold meetings for all members? (Circle the answer) 1 = Weekly 2 = Monthly 3 = Bimonthly 4 = Quarterly 5 = Annually 6= Never meet 7=Other A12. Is the frequency of meetings done as per the governance system? 1 = Done as per the governance system stipulations 2 = Not done as per the governance system stipulations A13. What kind of records does the organization keep? (Circle all answers applicable) Verify the records by seeing them 1 = Minutes of meetings 2 = Membership register 3 = List of assets 4 = Store inventory for food commodities and other assets 5 = Invoices, delivery notes and receipt books 6 = Bank Statements or Deposit and withdrawal slips 7 = Financial statements and Monthly, Annual Income /Expenditure Account, 8= Member contribution records 9 = Training records 10= Other (specify)____________________ A14. How many employees does the organization have Men |__|__|Women |__|__| If none move to section B A14.1 Type of employee (for the different types of employees indicate numbers under casual or permanent, or otherwise zero) Type of employee Contract type Casual Contract type Permanent 155. Manager 156. Accountant 157. Clerk 158. Watchman 159. Others (Specify) Section B. Production B1. Estimated cumulative total area actually planted by members in acres? WFP Kenya Endline LRP Evaluation Report – May 2020 |Page 78 B2. Type of crop grown (Grains)? 1. Sorghum 5. Green grams 2. Pigeon peas 6. Maize 3. Cow peas 7. Other (Specify) 4. Beans 9. B2.2 Type of crop grown (Vegetables)? 1. Kale 5. Irish Potatoes 2. Cabbages 6. Other (specify) 3. Tomatoes 4. Onions 160. B2.3 Type of crop grown (Fruits)? 1. Mangoes 2. Watermelons 3. Paw paws 4. Other (Specify) 161. B3. Seasons? Mm/yy –mm-yy (to capture all seasons) B4. How many members of the group have their soil quality tested before planting? B5. Besides crop production what other agricultural activities are undertaken by the group? 1. Poultry farming 2. Bee keeping 3. Cattle farming 4. Dairy Farming 5. Fish Farming 6. Goat farming 7. Other (specify) B6. Do members of the group use certified/recommended inputs? If no, skip to B15 1. Yes 2. No B6.1 Which are the certified/recommended inputs used? 1. Seeds 2. Fertilizer 3. Agrochemicals 4. Feeds 5. Other (specify) B7. Does this organization collectively buy inputs? If no, move to question B15 1 = Yes 2 = No B7.1 How do members of the groups access the inputs? If through donation, skip to B12 1. Direct Purchase 2. Donation B7.2. How many of your registered members access inputs through the organization? Men |__|__||__| : Women |__|__||__| B7.3 If not all members are accessing inputs through the organization what are the reasons? ? 1. Lack of adequate awareness/knowledge 2. Lack of access to credit 3. Poor FO governance/management 4. Purchase inputs individually 5. Any other reason (specify); WFP Kenya Endline LRP Evaluation Report – May 2020 |Page 79 B8. Where (which suppliers) has the organization bought inputs from in the past one year? (Tick/circle all that apply) 1= Seed companies 2 = Pest/Disease control companies 3= NCPB 4 = Input suppliers 5 = Retailers 6 = Other (specify) B 9. Did the organization sign any contract with the suppliers above? 1 = Yes 2 = No B110.Who determines the prices paid when the group purchases inputs? 1 = Supplier 2 = Farmer Organization 3 = Government 4 = Other (specify) B111. Where are most of your suppliers located? 1 = Within the ward 2 = Within the county but outside the ward 3 = Outside the county 4 = Outside the country B12.Did you receive the following types of assistance? (Prompt as necessary) Who provided the assistance? 1 = Yes 2 = No See codes below a b 1 Subsidized or free seeds |__| |__| 2 Subsidized or free fertilizer |__| |__| 3 Subsidized or free farming implements (tools/ agriculture machinery ) |__| |__| 4 Subsidized or free pesticides/herbicides |__| |__| 5 Providing or rehabilitating storage facilities |__| |__| 6 Loans of agricultural tools or work animals |__| |__| B12 b: Assistance provider codes 1 = Government 2 = International/national NGO 3 = International development agencies (UN, USAID, GTZ, etc.) 4 = Buyers 5 = National/Umbrella Farmers’ Association 6 = Other (specify) B13. What is the most common way the organization gets inputs from the suppliers? |__| B13: Delivery method codes 1 = Organization collects inputs from suppliers 2 = Suppliers deliver to the organization 3= Not Applicable B14. What is the most common way the organization gets inputs to its members? |__| B14. Delivery method codes 1 = Members collect inputs from farmer organization WFP Kenya Endline LRP Evaluation Report – May 2020 |Page 80 2 = Farmer organization delivers inputs to its members 3= Not Applicable B15. How do members handle their commodity during/after harvest? 1. Timely harvest 2. Conduct sorting for the produce after harvesting 3. Placing produce on open/bare ground during harvesting and drying 4. Use of tarpaulins during harvesting/drying B16. How does the organization support members on post-harvest management? 1. Provide drying facilities 2. Provision of handling equipment/technology – (hermetic bags, tarpaulins, moisture meters etc) 3. Provision of transport facilities to farmers 4. Provision of Storage facilities 5. Post-harvest information sharing/education to members 6. Other (specify) B17. If aggregating/ selling collectively, what is the estimated quantity of commodity produce expected from all members per season? (list per commodity) 1. Commodity one |__||__||__||__|mts 2. Commodity two |__||__||__||__|mts 3. Commodity three |__||__||__||__|mts B18. What is the estimated percentage of the main produce that is consumed by members at the household level? 1. Less than 25% 2. 25-50% 3. 50-65% 4. Over 75% B19. Does the organization have strategies in place which make the members less vulnerable to weather-related risks or natural disasters? The weather-related risks or natural disasters include floods, drought, high/low rainfall, high/low temperature, high/low humidity, earthquakes and volcanic activities. If none skip to B20 1 = Yes 2 = No B19.1 If yes, what type of strategies are in place? 1 = Insurance 2 = Irrigation 3 = Crop diversification and intercropping 4 = Mixed farming 5= Drought and pest resistant varieties 6= Other (specify) B 20. Does the organization have strategies in place which make the members less vulnerable to biological and environmental risks? The biological and environmental risks include crop/livestock contamination, diseases and pests, soil erosion If none skip to section C 1 = Yes 2 = No B20.1 If yes, what type of strategies are in place? 1 = Drought/pest resistant varieties 2 = Adequate production and harvesting techniques 3 = Adequate post-harvesting techniques and technology 4 = Food safety and Quality management 5 = Crop diversification and intercropping 6 = Good farm hygiene 7 = Other (specify) WFP Kenya Endline LRP Evaluation Report – May 2020 |Page 81 Comments Section C. Group Assets C1. Does this organization have access to storage facilities? 1 = Yes 2 = No (If no skip to C4) C2. If yes, what are the terms ownership? 1 = Leased/loaned temporarily 2 = Long term lease 3 = Own temporary store 4= Own permanent store 5 = Other (specify) C3. What is the capacity of the store? (Record in MT) C4. Does the organization have any of the following marketing equipment? 1 = Yes 2 = No move to C5 C4.1 If yes, for each equipment list the following? Type/Name Number Current Condition 1. In good condition 2. In need of service 3. Broken down 4. Never Used 1. Sieve 2. Drier 3. Tarpaulins 4. Sheller 5. Moisture meter 6. Weighing scale 7. Pallets 8. Other (specify) 162. C5. Does the organization own land collectively? If No skip to section D 1 = Yes 2 = No C5.1 If yes record the size in acres |__|__||__||__|__||__| C5.2 How is the land utilized? 1. Own farming activities 2. Own aggregation activities 3. Leasing to others for use 4. Building storage facility 5. Not utilized WFP Kenya Endline LRP Evaluation Report – May 2020 |Page 82 6. Other Comments Section D: Marketing D1. Does this organization have a business plan? If No skip to section D2 1 = Yes 2 = No D1.1 Who developed the business plan? 1. Committee members 2. External party (such as consultant, NGO, Financial institution, County Government officer) D1.2 Has the organization followed through on the business plan 1 = Yes 2 = No D1.3 What is the strength of the business plan (strength measured by: SMART objectives, market analysis, projection of volumes to be aggregated )- ensure you review the copy of the business plan 1= Considered weak 2= Considered average 3 = Considered strong D1.4 Is there clear system of monitoring business plan (are the respondents able to explain the system) list a few aspects of the system 1 = Yes 2 = No D2. Does this organization have a marketing plan? If No skip to section D3 1 = Yes 2 = No D2.1 Who developed the marketing plan ? 1. Committee members 2. External party (such as consultant, NGO, Financial institution, County Government officer) D2.2 Has the organization followed through on the marketing plan 1 = Yes 2 = No D3. Does the organization monitor market trends/requirements related to product quality and market demand? If No skip to section D4 1 = Yes 2 = No D3.1 What are the sources of these information? 1. Media (radio, TV, Newpapers) 2. County staff (extension, agribusiness officers) 3. Mobile application 4. Others D4. Does this organization aggregate and market members’ food commodities? If no, move to question D112.2 1 = Yes 2 = No D5. What percentage of total production volume do members sell through the organization? D6. Where (which markets) has the organization sold their commodities in the past one year? (Tick/circle all that apply) 1= Consumers (individuals) 2 = Retailer traders 3= Wholesaler traders 4 = Schools 5 = Millers 6 = Hospitals 7 = Brewers 8= WFP 9= NCPB 10 = Other (specify) WFP Kenya Endline LRP Evaluation Report – May 2020 |Page 83 D7. In the past one year, what is the total quantity (mt) that the organization has sold to different markets? (Record exact value in mt) D8. Does the organization ensure they aggregate good quality commodity? If no, move to question D9 1 = Yes 2 = No D8.1 How do they ensure the standards are adhered to? 1. By harvesting at the right time and using good practices at harvesting (harvesting dry, mature produce, shielding from rain) 2. By testing the moisture content 3. Ensuring produce is clean and free of any foreign matter 4. Grading and sorting of produce after harvest 5. Proper storage techniques 6.Any other (specify) D9. Did the organization sign any contract with the buyers above? If no, move to question D10 1 = Yes 2 = No D9.1 If yes, how many contracts did the organization sign |__|__||__| D9.2 What was the monetary value of the contract(s) D9.3 Of the total contracts signed, how many were executed successfully? |__|__||__| D9.4. Of the total contracts signed, how many were partially defaulted |__|__||__| D9.5. Of the total contracts signed, how many were totally defaulted |__|__||__| D10.Who determines the commodity prices? 1 = Buyer 2 = Farmer Organization 3 = Government 4 = Other (specify) D11. Where are most of your buyers located? WFP market is considered outside the county 1 = Within the ward 2 = Within the county but outside the ward 3 = Outside the county 4 = Outside the country D12. On average, how many of your registered members market their commodities through the organization? D12.1 How many of the members that market commodities through FO are women? |__|__||__| D12.2 If not all members are marketing their commodities through the organization , what are the reasons ? 1. Lack of adequate awareness/knowledge 2. Inadequate production 3. No (not enough) storage 4. Difficult to find markets to sell large volumes 5. Farmers unwilling to deliver to group 6. Any other reason (specify); D13. What is the most common way the organization gets produce from members to a collection point for sale or delivery? |__| D13. Delivery method codes 1 = Farmer organization collects produce from members WFP Kenya Endline LRP Evaluation Report – May 2020 |Page 84 2 = Members deliver their produce to the organization 3 = The buyer collects the produce from individual members (If Buyer collects then skip D14) D14. What is the most common way the organization gets members’ produce from the organization’s collection point to a market/buyer where you can sell? |__| D14: Delivery method codes 1 = Organization delivers products to buyers 2 = Buyers collect from organization D15. What are the five most critical problems your organization faces in selling staple commodities on behalf of your members? a b c |__| |__| |__| D15: Problems codes 1 = Limited consumer demand for products 2 = Limited access to pricing information 3 = High costs of collecting and preparing commodities for market 4 = Limited access to credit to pre-purchase commodities from members 5 = Low volume of staple commodities available from members (for example, because of late payment from buyers, lack of trust in organization, low production, etc.) 6 = Government trade restrictions 8 = Poor transportation infrastructure 9 = Not able to meet quality demands of buyers 10 = Unpredictable prices/price fluctuations 11 = Other (specify) D16. What source of price and market information have members relied on most often? |__| D16: Market information sources 1 = Radio/TV 2 = Information boards at local agricultural offices 3 = Newspapers 4 = SMS system/mobile phone 5 = Other (specify) D17. Does the organization have any value addition activity on current produce If No skip to D18 1 = Yes 2 = No D17.1 If yes specify 1. Sorting 2. Grading 3. Packaging 4. Milling 5. Polishing 6. Baking 7. Fortification 8. Drying 9. Other (specify) D18. How can your organization be assisted to market its produce? 1. Improve infrastructure 2. Avail market prices 3. Price determination 4. Training on marketing 5. Policy and advocacy 6. Market linkage WFP Kenya Endline LRP Evaluation Report – May 2020 |Page 85 7. Support on aggregation 8. Other (specify) Comments Section E: Group Financial Management E1. Does your group have a bank account 1 = Yes 2 = No E1.1 What type is the account 1. Savings 2. Current (transactional) 3. Fixed(investment) E2. Has this organization ever applied for cash loans? 1 = Yes 2 = No (If “Yes”, go to question E3) (If “No”, go to question E6) E3. Were any credit applications approved and the loans received? 1 = Yes 2 = No (If “Yes”, go to question E4) (If “No”, go to question E5) E4. Please tell me about the cash loans this organization has received in the last one year. Lender Purpose of the loan What amount did you receive? What is the current status of the loan? What is the interest rate per annum? (%) How many members benefitted? (Gender disaggregated) What is the repayment period (no. of months) Men Women a c d e f g h 1 |__|__| |__|__| |__|__|__|__|__|__|__| |__| |__|__|__|__| |__|__| |__|__| |__|__| 2 |__|__| |__|__| |__|__|__|__|__|__|__| |__| |__|__|__|__| |__|__| |__|__| |__|__| 3 |__|__| |__|__| |__|__|__|__|__|__|__| |__| |__|__|__|__| |__|__| |__|__| |__|__| E4 a: Lender types 5 = Agricultural supply companies E4d: Loan status codes 1 = Fully repaid 2 = Payments up to date but not fully paid off 3 = Payments not yet due 4 = In default 1 = Bank 2 = Buyers (forward payment) 3 = Microfinance institution, including SACCOS 4 = NGOs, International development agencies (UN, USAID, GTZ, etc.) 6 = Affiliated farmers’ organization 7 = Government fund 8 = Other (specify) E4b: Purpose of loan 1 = To purchase inputs and equipment 2 = Buy more products (aggregation)for sale 3= Build/improve infrastructure (store, water and sanitation, irrigation, electricity) 4= Buy land 5= Market product (including transport) 6 = Other (specify) E5. What was the main reason the farmers’ organization did not receive the loan? (Go to question E7) |__| 1 = Had other outstanding loans 2 = Could not provide business plan 3 = Did not meet the qualification criteria WFP Kenya Endline LRP Evaluation Report – May 2020 |Page 86 4 = Could not provide requested collateral 5 = Lack of credit history 6 = Other (specify) E7. Does this group offer credit services to its members? 1 = Yes 2 = No (If “Yes” go E8) E7.1 Why doesn’t the group offer credit to members 1. Lack of funds 2. Members do not require credit from the FO 3. Members defaulted in the past 4. Most members access credit individually 5. Other (specify) E8. If yes, please tell me about the cash loans this organization has given to its members in the last one year. How much was disbursed to members? What is the interest rate per annum? (%) How many members benefitted? (Gender disaggregated) Men Women a b c d 1 |__|__|__|__|__|__|__|__| |__|__|__|__| |__|__|__| |__|__|__|__| 2 |__|__|__|__|__|__|__|__| |__|__|__|__| |__|__|__| |__|__|__|__| 3 |__|__|__|__|__|__|__|__| |__|__|__|__| |__|__|__| |__|__|__|__| E9.( For groups aggregating) does the organization pay members for their commodity before the buyers make payment (If No skip to E10) 1 = Yes 2 = No E9.1Where do the funds come from? 1. Loan from a financial service provider; 2. Advances from buyers; 3. Farmer Organization’s financial reserves; 4. Other {Specify} E10. Does the organization set aside some funds to facilitate production and marketing activities 1 = Yes 2 = No E11. Does the organization ensure a financial strategy and planning? (makes budgets, financial projections, allocation of funds and monitoring use) 1 = Yes 2 = No E12. Does the organization retain reserves for use in times of financial distress? (savings, profits, membership fees and others) 1 = Yes 2 = No Section F: Training F1: Has any of your committee members been trained in the past year? 1 = Yes 2 = No (If “No”, go to question F2) F1.1 If yes, how many were trained on? E6. Why has the organization never applied for credit? |__| 1 = Did not need credit 2 = Lack of consensus in the organization 3 = No credit providers in our area 4 = Credit providers do not give credit to farmers 5 = Rates are too high 6 = Do not have the required collateral 7 = Lack of knowledge about access to loans 8 = Other WFP Kenya Endline LRP Evaluation Report – May 2020 |Page 87 TOPIC YES NO Number of committee Members trained Training provider e.g. Cereal Growers Association, Ministry of Agriculture Male Female a. Group dynamics(governance) b. Organizational behaviour c. Leadership Skills d. Other (specify) e. Other (specify) F2: Has any of your members been trained in the past year? 1 = Yes 2 = No (If “No”, go to question F3) F2.1 If yes, how many were trained on? TOPIC YES NO Number of Members trained Training provider e.g. Cereal Growers Association, Ministry of Agriculture Male Female a. Good agronomic practices b. Conservation agriculture b. Post-harvest handling c. Entrepreneurship d. Gender in agribusiness e. Setting and negotiating prices for produce f. Procurement processes e.g. filling in tenders g. Record keeping h. Aflatoxin awareness i. Financial Management/literacy k. Collective marketing (aggregation) l. Contract management m Price margin and profit calculation n. Other (specify) F3. Does the organization offer services to its members (If “No”, go to comments section) 1 = Yes 2 = No F4. If yes, what types of services do they provide 1 = Trainings and other technical assistance 2 = Quality control and inspection 3 = Market research WFP Kenya Endline LRP Evaluation Report – May 2020 |Page 88 4 = Other Comments: How has the organization applied the skills obtained during the trainings? Comments: What is the impact of the trainings on the activities of the organization? Comments: What other areas do your members need training in the next year? Comments: What other services are provided by NGOs and government that have not been covered during this session? Section G: Sustainability G1. Does the group have good relations with organizations who can assist them in building their group capacities? 1 = Yes 2 = No G2. Can the group operate and function independently from organizations that assist in building group capacities? 1 = Yes 2 = No G3. Does the group have a good relationship with the local communities it is operating in? 1 = Yes 2 = No G4. Does the group have adequate technical support from the line ministries? 1 = Yes 2 = No G5. What other support does the group feel it is required but not addressed by the current enablers? GPS coordinates (record in decimal degrees)………………………… Latitude (N/S) Longitude East WFP Kenya Endline LRP Evaluation Report – May 2020 |Page 89 Local and Regional Procurement Project Endline Evaluation Survey Questionnaire for Teacher’s responsible for school meals and/or School Meal Committee Members Introduction INTERVIEW DATE ______/______/______ DD MM YY INTERVIEW START TIME: INTERVIEW END TIME: ______/______ HH MM County: Sub-county: ID and name of school: |__||__||__| _____________________________________________________________ Type of school |__| Day school |__| Boarding school |__| Both day and boarding school Case or control school: |__| Case (LRP targeted): |__| Control Number of students as at Term 3, 2019 (Excluding ECD) Contact Person Mobile Number Provision of school lunch 1. How does your school currently provide school lunch? |__| Directly provided by WFP |__| Through WFP funding (cash) (Cash to Schools) |__| Through MoE funding (HGSMP) |__| Directly provided by the government 2. Other than the source above, does your school also get food for lunch from any of the following sources? YES NO |__| |__| The county government |__| |__| School’s farm |__| |__| The church |__| |__| The parents teachers association |__| |__| Direct contributions from parents |__| |__| Other NGOs WFP Kenya Endline LRP Evaluation Report – May 2020 |Page 90 3. For Term 3, 2019 was food for lunch delivered before the school term began? (tick the appropriate cell). |__| Yes |__| No 4. How many days after start of Term 3, 2019 did the lunch food delivery arrive? ____________ days (If delivered before start of term enter zero) 5. Did your school experience any of the following problems with the lunch food supplied during Term 3, 2019? YES NO |__| |__| Bags arrived in poor condition (e.g. open/torn) |__| |__| Food was infested |__| |__| Incorrect amount arrived (i.e. not as ordered) |__| |__| Supplier refused to bring food directly to school |__| |__| Other (specify) _____________________________________________________________ ______________________________________________________________ 6. During Term 3, 2019, how many days was the school NOT able to provide lunch due to the following reasons? Number of days |__|__| No food available |__|__| No one to prepare meals |__|__|Not enough water |__|__| Not enough firewood |__|__|Food cannot be prepared when it rains due to poor condition of kitchen |__|__|Other (please specify) ____________________________________________ ______________________________________________________________________ 7. In total how many days during Term 3, 2019 did you NOT provide school lunch? ____________ days 8. For Term 3, 2019 was the lunch food procurement process completed before the start of the term? |__| Yes |__| No |__| WFP procured the food |__| Central government procured the food 9. How many traders did your school utilize for school lunch provision during Term 3, 2019? |______| 10. What was the total volume for each commodity your school procured for lunch during Term 3, 2019 (in KG)? 11. What was the total value of commodities procured for school lunch during Term 3, 2019? Commodity Q10. Total Volume Q11. Total Value Sorghum Rice CSB MSB Bulgur Wheat Beans Yellow split peas Green gram Maize Oil Salt Cowpeas Vegetables Fruits Tomatoes Onions Other 1 (specify)……………………………… Other 2 (specify)……………………………… WFP Kenya Endline LRP Evaluation Report – May 2020 |Page 91 Other 3 (specify)……………………………… Other 4 (specify)……………………………… TOTAL 12. How much did the school/teachers’ have to pay during Term 3, 2019 for additional lunch food transportation or any other costs not included in the tender value? (e.g. storage, unloading etc.). |_______| Food transport |_______| Food storage |_______| Food unloading |_______| Other (specify) Please estimate the total additional monies spent during Term 2, 2019. |_______| School menus 13. What meals do you serve last week for lunch on the following days? (Use the codes in question 14) Monday: |__| |__| |__| |__| Tuesday: |__| |__| |__| |__| Wednesday: |__| |__| |__| |__| Thursday: |__| |__| |__| |__| Friday: |__| |__| |__| |__| 14. Does the school have a written nutrient profile for the meal? Meal Code Q14. Does the school have a written nutrient profile for the meal? YES NO 1. Sorghum |__| |__| 2. Rice |__| |__| 3. CSB |__| |__| 4. MSB |__| |__| 5. Wheat |__| |__| 6. Beans |__| |__| 7. Yellow split peas |__| |__| 8. Green gram |__| |__| 9. Maize |__| |__| 10. Oil |__| |__| 11. Salt |__| |__| 12. Cowpeas |__| |__| 13. Vegetables |__| |__| 14. Fruits |__| |__| 15. Tomatoes |__| |__| 16. Onions |__| |__| 17. Other 1 (specify)……………………………… |__| |__| WFP Kenya Endline LRP Evaluation Report – May 2020 |Page 92 18. Other 2 (specify)……………………………… |__| |__| 15. Has your school done any work with WFP during the last two years to revise your school lunch menus? |__| Yes, we are in discussion |__| Yes, we changed our menus |__| No 16. Does your school currently do any activities to promote consumption of a wide variety of foods (dietary diversity) for lunch? YES NO |__| |__| Use drought tolerant crops for school meals (including sorghum, millet or cowpeas) |__| |__| Provide more than one type of school meal for lunch |__| |__| Nutrition education for the children talking about the benefit of eating a wide variety of foods |__| |__| School personnel received training from WFP or MoE on improving dietary diversity of school meals |__| |__| Use food from a school garden in school meals |__| |__| Other (please specify) ____________________________________________ Capacity building 17. How many of the following staff members received training in the last two years on the Home￾Grown School Meal Programme (HGSMP) from WFP/MoE? Staff Cadre Total Number in School Total Number Trained Head Teacher Teachers (Deputy head teacher, senior teacher, any other teacher) School meals committee members School chairman Other (please specify) ……………………………………… 18. How many of the following staff members received training in the last two years on tendering processes for procuring food from local traders for school meals from WFP/MoE? Staff Cadre Total Number in School Total Number Trained Head Teacher Teachers (Deputy head teacher, senior teacher, any other teacher) School meals committee members School chairman Other (please specify) ……………………………………… 19. How many full-time cooks (cooks every school day) does your school have? |______| 20. Other than the full-time cooks, how many other teachers/ members of staff /other people help in preparing school lunch? |______| 21. How many cooks/members of staff/teachers that help prepare lunch were trained in the last two years on good nutrition and menus? |______| Cooks |______| School meal teacher |______| School meal committee member WFP Kenya Endline LRP Evaluation Report – May 2020 |Page 93 School observations: 1. Has the school received their food delivery for Term 1, 2020? |__| Yes |__| No If yes, when was it received? (specify date) ___ / ___/ If no, are you still able to serve school meals |__| Yes |__| No 2. Ask to see the nutrient profiles of any meals. How many meals have written profiles? |__| 3. Please take a photograph of any nutrient profiles that schools have. 4. Please ask if you can see the school’s food store. 5. Please take a photograph of the food store ensuring that we can see how the food is stored on the ground. INTERVIEW END TIME: INTERVIEW END TIME: ______/______ HH MM THANK YOU VERY MUCH FOR YOUR PARTICIPATION. GPS Coordinates __________________________________________________ WFP Kenya Endline LRP Evaluation Report – May 2020 |Page 94 Annex 11: Description of the calculation of the evaluation indices 1. Cost-effectiveness score Cost-effectiveness Score (CS) = GMC*VPCD GMC = ATA/ATV VPCT = ATV/#Children VPCD = VPCT/DLS Where: GMC = Actual grouped median cost ATA = Actual total cost of commodities ATV = Actual total volume of commodities VCPT = Volume per child per term VCPD = Volume per child per day DLS = Days lunch served Validation of the Cost-effectiveness Score There is no statistical difference between the ATA (Actual total cost of commodities) and the cost effectiveness score, p > 0.05. Assumptions • The cost effectiveness is computer using the direct cost of food items only. 2. Timeliness score Timeliness Score (TS) = (FD-DD-ML)/ (FD)*100 Where: FD = Full Days (Maximum term days + maximum delayed delivery days + maximum lunch missed days) DD = Actual delayed delivery days ML = Actual missed lunch days Assumptions and justification • The maximum term days are 70. • The maximum delayed delivery days are 70. • The maximum days for missing lunch is 70. • Some of the “delayed” delivery is voluntary given the surplus food in the school store. We control for ‘voluntary delay” by including actual days lunch was not served in the schools. • The outcome index is a percentage (maximum is 100 percent where delay days are 0 and actual days lunch was not served in the schools is 0. • The extraction method is mean score to mitigate exclusion of the decimal performing schools from contributing to the overall score. Paired Samples Test Paired Differences t df Sig. (2-tailed) Mean Std. Deviation Std. Error Mean 95% Confidence Interval of the Difference Lower Upper Pair 1 ATA - CS 20753.7 277762.9 28497.9 -35829.5 77336.9 .728 94 .468 WFP Kenya Endline LRP Evaluation Report – May 2020 |Page 95 3. Nutrition score Nutrition Score (NS) = Av ((TVCD/150), (TVPD/40), (TVOD/5), (TVCS/2))*W1/3, (DLS/70)* W1/3, (MEN/3)* W1/3 Where: Av = Average TVCD = Total volume of cereals per child per day fed TVPD = Total volume of pulses per child per day fed TVOD = Total volume of oil per child per day fed TVSD = Total volume of salt per child per day fed DLS = Actual days lunch was provided MEN = Menu items diversity W1/3 = 1/3 weighting Assumptions and justification • The maximum cereals consumed in a day is 150 grams. • The maximum pulses consumed in a day is 40 grams. • The maximum oil consumed in a day is 5 grams. • The maximum salt consumed in a day is 2 grams. • The maximum actual days lunch was provided is 70 • The maximum menu item diversity value is 5 representing unique combinations in a week of 5 days. 4. Overall Impact Score Overall Impact Score = (TS + NS - CS)/187*(100) Where: TS = Timeliness Score NS = Nutrition Score CS = Cost Effectiveness Score Assumptions • The maximum Timeliness Score is 100% • The maximum Nutrition Score is 100% • The observed Cost Effectiveness Score is 13 • The desired situation is to spend a maximum of 13 KES per child per day while providing the most nutritious food every school day and in a timely manner WFP Kenya Endline LRP Evaluation Report – May 2020 |Page 96 Annex 12: Qualitative data collection tools Semi-structured interview guides 1. WFP personnel (LRP and/or market access team) • How were counties/sub-counties selected? • How were traders and FOs selected? • How were schools selected? • To what extent are government authorities (national, county, sub-county) involved in the LRP? • How does the LRP relate to key government policies/strategies? • Have any WFP policies specifically been included in the design? • Does the LRP relate to any work by other agencies? • Explain transition from SMP to HGSMP in the three LRP counties: • Timing of transition • Specific challenges encountered per county • Explain previous work with targeted FOs/ traders if any. • Specific activities implemented • What are the key challenges you foresee for the LRP? • Has there been any specific consideration of gender issues? How have they been incorporated? Check monitoring of the following indicators: Number of individuals benefiting directly from USDA-funded intervention Number of individuals benefiting indirectly from USDA-funded intervention Number of public-private partnerships formed as a result of USDA assistance Value of public and private sector investments leveraged as a result of USDA assistance Number of policies, regulations and/or administrative procedures in each of the following stages of development as a result of USDA assistance Quantity of commodity procured as a result of USDA assistance Cost of commodity procured as a result of USDA assistance Cost of transport, storage and handling of commodity procured as a result of USDA assistance Number of social assistance beneficiaries participating in productive safety nets as a result of USDA assistance Number of individuals who have received short-term agricultural sector productivity or food security training as a result of USDA assistance 2. MoE personnel (government representatives) • What is/going to be the role of the MoE in the LRP? o Check any specific roles for national and county/sub-county MoE. • Please comment on co-operation and information sharing between the MoE and WFP? • What do you feel are going to be the main challenges for the MoE in implementing HGSMP? o Any challenges related to LRP? WFP Kenya Endline LRP Evaluation Report – May 2020 |Page 97 Check monitoring of the following indicators: Number of policies, regulations and/or administrative procedures in each of the following stages of development as a result of USDA assistance 3. School Meals Committee members Knowledge of LRP 3. What do you know about the new WFP LRP project? What are they hoping to achieve? 4. How many traders did your school utilize for SM provision during SY 2016/17? 5. Did you have any problems with any of the tenders? a. Elaborate Purchase of local food commodities for provision of school meals 6. Experience purchasing through tenders. a. What have been the best aspects of purchasing through tender instead of having food provided by WFP? b. What have been the most difficult aspects of local purchase? c. Comment on the current procurement process. d. Have you experienced any issues re timeliness with local procurement? e. How can procurement process be improved? 7. Approximately how many traders do you use to purchase the stock for the school meals programme? 8. Was your school able to provide school meals every school day during SY 2016/2017. If no, why not? Any reason related to insufficient food? 9. Discuss cost-effectiveness of procurement 10. Discuss timeliness of tender process and timeliness of delivery a. Check timing of completing tender b. Check timing of delivery of commodities (preferably before start of school term) 11. Volume of commodities procured locally during SY 2016/17? 12. Main commodities purchased. 13. Value of commodities procured for school meals during 2016/17 14. Value of any additional monies spent on transport or any other costs related to food (transport, storage, loading, unloading etc.). Capacity building • Did your school administrator receive training during 2016/17 on HGSMP? • Did any of your School Board of Management members receive training during 2016/17 on HGSMP? • Has your school administrator received training during 2017/18 on buying food from local traders? Tender process? • Did any of your School Board of Management members receive training during 2017/18 on buying food from local traders? Tender process? • Have any of your cooks been trained on menus, hygiene and food handling procedures? Yes No • If yes, how many were trained during SY 18/19? WFP Kenya Endline LRP Evaluation Report – May 2020 |Page 98 School menus 15. How many menus (per month) does your school currently follow for school meals? 16. Does your school have any nutrient profiles for the meals being provided? a. Yes No (If yes, how many meals have profiles? ____) 17. Has your school done any work with WFP during SY2017/18 to revise your school menus? 18. If yes, are you currently using revised menus? Suggestions and complaints 19. Do you have any suggestions on how to improve the project? Make it sustainable? 20. Have you received any complaints about food purchased locally during SY 2016/17 or about WFP? a. Any issues with working with WFP or their partner agencies? (Programmatic or financial) Check monitoring of the following indicators: Percentage of LRP schools procuring food before beginning of term Percentage of schools where food is delivered to schools before term begins Percentage of schools using diversified menus Turkana only: Cost of distribution in schools in LRP areas compared to non-LRP areas 4. Grain traders • How familiar are you with the tender process? • What is your relationship with local small holder farmers? • How can WFP better support the use of local food commodities into schools? • What do you find most difficult about the tender process? What is the easiest aspect? Do you have any suggestion for improving the tender process? • What foods have the school meal committees most regularly purchased from you? • Were you given any guidance about the type of food that can be purchased? Or the quality of food that is required? Elaborate • Where are your suppliers based? • Where are your buyers based? • Do you supply to clients outside of XX County? Elaborate how far they trade. • Have you made any changes to your business or business practices as a result of this project or due to the transition to CTS/HGSMP? • Have there been any positive impacts of this project on your business? Elaborate • Have there been any negative impacts of this project on your business? Elaborate Check monitoring of the following indicators: Volume of sales by project beneficiaries Volume of commodities sold by project beneficiaries WFP Kenya Endline LRP Evaluation Report – May 2020 |Page 99 5. Farmer Organizations • The LRP is a project that will help farmers be more prepared for tendering processes and access a new market opportunity (school meals). o Has this FO ever directly applied for school meals or other large tenders? o If yes, what have you found to be the main challenges in the tendering process? • What are the main challenges you face in participating in local tenders? • How familiar are you with the tender process? • What is your relationship with local traders? • How can WFP better support the use of local food commodities into schools? Check monitoring of the following indicators: Volume of sales by project beneficiaries Volume of commodities sold by project beneficiaries WFP Kenya Endline LRP Evaluation Report – May 2020 |Page 100 Annex 13: Silhouette measure of cohesion and separation Silhouette measure of cohesion and separation was used to define and validate the consistency of matching and clustering of LRP and non-LRP schools. This technique was used to match control and treatment schools with similar spatial and demographic attributes. The matching and clustering process are validated by the model strength. The model converged at a silhouette value of 0.7 as shown in the figure below. The silhouette value is a measure of cohesion of study cases which included control and treatment schools. The silhouette ranges from −1 to +1, where a high value indicates that the clustered study cases are well matched. The mean enrolment of 288.65 was validated by the grouped median of the initial unmatched population of 229.67. The grouped median is a trimmed median that mitigates all the outliers in both the LRP and non-LRP schools. WFP Kenya Endline LRP Evaluation Report – May 2020 |Page 101 Annex 14: List of surveyed LRP and non-LRP comparison schools Baringo County # LRP schools Non-LRP schools 1 Tangulbei Primary School Bartabwa Primary School 2 Lomuge Primary School Chemogoch Primary School 3 Chepkalacha Primary School Chemoinoi 4 Koipapich Primary Chepnyorgin Primary School 5 Koloa Primary Chesongo Primary School 6 Ngoron Emining 7 Tukomoi Primary Kabilany Primary School 8 Kaisakat Primary School Kaboron Primary 9 Chemayes Primary School Kalabata Primary School 10 Ptikii Primary School Kamweton Primary School 11 Loiwat Primary School Kapkelelwa Primary School 12 Dira Primary School Kaptara Primary School 13 Nalukumong'in Katunoi Primary School 14 Cheptunoyo Primary Kimose 15 Kositei Primary School Kipkaren 16 Chemolingot Boarding School Kipsoit Primary School 17 Tamkal Primary School Kures Primary School 18 Meuto Primary School Lomanira Primary 19 Barpello Primary Lombagishu Primary School 20 Maaron Primary School Molo Sirwa Primary 21 Nginyang Boarding Primary School Nato Primary 22 Aic Chemoril Primary School Ngurubeti Primary 23 Chesacam Primary School Noiwet Primary 24 Chepturu Primary School Radad Primary School 25 Tilingwa Primary School Rosoga Primary 26 Kapunyany Primary School Sosion Primary School 27 Chepelow Primary Tebei Primary School 28 Plesian Primary Terik Primary School 29 Churo Primary School Tiloi Primary School Turkana County # LRP schools Non-LRP schools 1 Namalteny Lotiira Primary 2 Nakuse Primary Maggies Akatuman Primary 3 Lomunyenakwan Primary Kawarnaparan Primary 4 Agape Primary Kotela Primary 5 Nakukulas Primary Kotaruk Primary 6 Lochwaa Angikamatak Primary Nagis Primary 7 Arumrum Kanukurudio Primary 8 Nakwasinyen Primary Namoruputh Primary 9 Kalodicha Nataparkakono Primary 10 Kangitit Primary St.Teresa Nakwamor Primary 11 Kanaodon Primary Lokangae Primary 12 Aic Nadoto Primary Katiko Primary 13 Naregaekamar Nakoriogora Primary 14 St.Emmaculate Ngimuriae Primary 15 Lorogon Primary Kodopa 16 Abururu Naremit Primary WFP Kenya Endline LRP Evaluation Report – May 2020 |Page 102 17 Naoyaregae Kabokorit Primary 18 Namorutunga Primary St. Cosmas Napopongoit 19 Lokorkor Lorengipi Primary 20 Kangimanyin Nameyana Primary 21 Nakatongwa Kalopiria Primary 22 St. Teresa Kimabur Kalokol Girls Primay 23 Loyapat Katula Primary 24 Kalapata Kabulokor Primary 25 Kakoel Lokamarinyang Primary 26 Kidewa St. Bakhita Teremkus 27 Kaaruko Longech Primary 28 Kangakipur Nanyangakipi 29 Kakong Loturerei 30 Kadam Lolupe Primary 31 Kaibole Napuu Primary 32 Lomonyang Primary School Kangagetei Primary 33 Namakat Kangirisae 34 Nayanaekaton Kaikir West Pokot County # LRP schools Non-LRP schools 1 Kapkewa Primary School Runo Primary School 2 Kasei Primary Chemaley Primary School 3 Timale Primary School Wakor Primary School 4 Kiwawa Primary Koposes Primary 5 Mbaru Primary Katimoril Primary 6 Kauriong Primary Sigor Girls Primary School 7 Cherangan Primary School Kapsimatia Primary School 8 Konyao Dorcas Primary School Chepserum Primary School 9 Konyao Arid Primary Ipeet Primary School 10 Nakwapuo Primary School Rukey Primary School 11 Kopulio Primary School Chepkukui Primary School 12 St Joseph Ack Nakwijit Primary School Tindar Primary School 13 Ngotut Primary School Lodupup 14 Kodera Primary School Kochar Primary School 15 Korpu Saya Primary School 16 Aic Asilong Primary School Ortum Boys Primary School 17 Kamketo Primary School Ortum Girls Primary School 18 Naruoro Primary School Kangisha Primary School 19 Kacheliba Primary School Ptulungwo Primary School 20 Tiyenei Primary School Pserum Primary School 21 Nakwoilal Primary School Sebit Primary School 22 Chepkinah Primary School Loklochoi/Nachecheyat 23 Natemeri Primary School Chepkobegh Primary School 24 Chelopoy Primary School Sokka Primary School 25 Aic Akiriamet Sobukwo Primary School 26 Lokichar Primary School Murpus Primary School 27 Sincholol Primary School Seretow Primary School 28 Aic Kameris Parek Primary School 29 Nauyapong Mixed Boarding Primary School March Pass Primary 30 Korkou Primaryschool Sangat Primary School 31 Kapterema Primaryschool Cheposekek WFP Kenya Endline LRP Evaluation Report – May 2020 |Page 103 32 Takar Primary School Miskwony Primary School 33 Kasaka Primary School Tunoyo Primary School WFP Kenya Endline LRP Evaluation Report – May 2020 |Page 104 Annex 15: Detailed description of the original (baseline) school sampling methodology A total sample of 192 schools were used (96 LRP schools and 96 non-LRP schools). a) There are 382 LRP schools and 613 non LRP schools totalling 995 schools. b) Schools with similar characteristics were clustered using the silhouette measure of cohesion and separation. The model converged at 0.7. The most cohesive cluster had skewness of 0.098 returning a universe population of 354 LRP and non LRP schools c) A population of 354 LRP and non-LRP schools were matched using the Silhouette measure of cohesion and separation that was utilized to define and validate the consistency of matching and clustering of LRP and non￾LRP schools. This technique was used to match LRP and non LRP schools with similar spatial and demographic attributes. The matching and clustering processes were validated by the model strength. The model converged at a silhouette value of 0.7. The silhouette value is a measure of cohesion of study cases which included LRP and non LRP schools. The silhouette ranges from −1 to +1, where a high value indicates that the clustered study cases are well matched. d) The actual sample was determined by the formula below: n = z² x (p) x (1-p) m² n = sample size z = confidence level at 95% (standard value of 1.96) p = percentage picking choice (0.5) m = margin of error at 5% (standard value of 0.05) Correcting for finite population (N = 354) n = n 1 + n - 1 N n = required sample; n = old sample size; N = population BASELINE SAMPLE SIZE School sample = 192 Sample size for treatment schools = 96 Sample size for control schools = 96 WFP Kenya Endline LRP Evaluation Report – May 2020 |Page 105 Sample distribution by county Replacement of Schools Schools that were impossible to access during data collection were replaced. The access barriers included insecurity, flood-affected roads and bridges, and distance (for instance, it was not feasible to travel for a whole day to track one school within limited resources available for the study). Such schools were replaced by schools in the same silhouette cluster or overlapping schools in the contiguous cluster. The evaluation model: Where: OS (Overall Score) = Score at T1 relative to T0; FTO = Farmer and Trader Organization Index/ranking; Schools = Selected Schools impact score; Group = Exposure Variable; RE = Covariates (Both direct outcomes and the confounding); E = Model Estimation/Error Terms; LT = Latent Variables At baseline, the study team computed score indices for the key outcomes, namely cost-effectiveness, timeliness, nutrition scores and the overall impact benchmarking score. These measures will be computed again at the endline phase and the variance computed while controlling for random effects using multivariate analyses. County Matched Weight LRP Schools Non LRP Schools Baringo 30.2 29 29 Turkana 35.4 34 34 West Pokot 34.4 33 33 Total 100 96 96 WFP Kenya Endline LRP Evaluation Report – May 2020 |Page 106 Annex 16: Evaluation field mission schedule LRP Endline Evaluation -Quantitative Data Collection: John OTSOLA – Feb 10-29 Date Activity Location Feb 10 Finalization of planning for enumerator training WFP Office, Nairobi Feb 11-14 Enumerator training: School survey, FO survey and Trader survey WFP Office, Nairobi Feb 15-16 Travel to field sites Nairobi to LRP Counties Feb 17-28 Quantitative data collection Turkana, West Pokot and Baringo counties LRP Endline Evaluation - Qualitative Interview Schedule: Sophie DUNN - Feb 17-21, 2020 Department Designation Name Date Time Venue WFP CCS Programme Policy Officer (SMP) Charles Njeru __ TBD B234 Resilience, Livelihood and Nutrition Programme Policy Officer (Agriculture Markets) Olive Wahome __ TBD O2 Office Programme Policy Officer (Market Linkage) Daniel Ndungu 17.02.20 9am Programme Policy Officer (Supply Chain) Rosemary Babu 17.02.20 9am Nutritionist Joyce Owigar/Josephine Mwema 18.02.20 10.30am Nutrition Office Programme Operations VAM Team Allan Kute 18.02.20 9.30am VAM office Julius Kisingu 19.02.20 3.30pm Skype Field offices Head of Nairobi satellite Office Kipsang Rotuno 20.02.20 2.30pm Skype Head of Nairobi Lodwar Field Office Ruth Amatalo/Gabriel Ekaale 24.02.20 GOK MOE Deputy Coordinator-School Meals Programme Boniface Ouko 20.02.20 10am Jogoo House MOA Head of Nutrition Mrs. Wambugu and Kithale David __ TBD WFP Kenya Endline LRP Evaluation Report – May 2020 |Page 107 LRP Endline Evaluation – Field Schedule: Sophie DUNN: Feb 23-29, 2020 Day Time Activity Contact person 23rd February 2020 14.00-16.00 Depart Nairobi for Lodwar by Flight WFP Lodwar Office 24th February 2020 8.00 - 9.00 Meet WFP Lodwar staff for Briefing/Planning 9.00 -10.00 onwards Commence Qualitative Interviews Afternoon - Commence Qualitative Interviews, Depart for Eldoret with evening flight, overnight at Eldoret 25th February 2020 7am to 9am Travel from Eldoret to Kapenguria WFP Nairobi Field Office/Polly 9am to 9:45am Meeting with County Director of Education and County School meals Programme Officer 10am to 10:45am Meeting with County Director of Agriculture 11:00 – 01:30pm Travel to and meeting with two Farmer organisations, one all male group and one all￾female groups. 1:30- 2:00pm Lunch 2:00- 3.00pm Meeting with two suppliers 3.00- 5:30pm Travel to and visit to one school in North Pokot (HGSMP) 26th February 2020 9:00am to 2:00pm Travel from Kapenguria to Kabarnet WFP Nairobi Field Office/Polly 2:30pm to 3:15pm Meeting with County Director of Education and County School meals Programme Officer 3:30pm to 4:15pm Meeting with County Director/Officer of Agriculture 6.00pm Overnight at Marigat 27th February 2020 10:00- 1:00pm Travel to and meeting with two Farmer organisations, one all male group and one all￾female group 1:00- 1:30pm Lunch 1:30- 2:30pm Visit to school 3:00pm Depart to Nakuru enroute to Nairobi 28th February 2020 6.00am Depart to Nairobi WFP Kenya Endline LRP Evaluation Report – May 2020 |Page 108 Annex 17: Stakeholders Interviewed • Number of qualitative interviews conducted KEY INFORMANT INTERVIEWS Name Position Organization Gender Nairobi+ 1 Ada Ihenachor International Programme Specialist (School Feeding and Humanitarian Branch) FAS/USDA F 2 Boniface Ouko Deputy Coordinator School Meals Programme Ministry of Education M 3 Immaculate Nyaugo Programme Manager – Healthy Diets Ministry of Health F 4 Allan Kute VAM Officer WFP M 5 Charles Njeru Programme Policy Officer (School Meals Programme) WFP M 6 Daniel Ndungu Programme Policy Officer (Market Linkages) WFP M 7 Fidelia Mwende Programme Associate, Nairobi Field Office WFP F 8 Josephine Mwema Programme Policy Officer (Nutrition) WFP F 9 Judy Ndungu Programme Policy Officer (School Meals) WFP F 10 Julius Kisingu Programme Policy Officer (Markets) WFP M 11 Kipsang Rotuno Head of Nairobi Satellite Office WFP M 12 Mari Hassinen Head, Country Capacity Strengthening WFP F 13 Rosemary Babu Programme Policy Officer (Supply Chain) WFP F 14 Zippy Mbati Programme Policy Officer (Refugee Self￾Reliance) WFP F Baringo 15 Vincent Abuje County Direction of Agriculture Ministry of Agriculture and Irrigation M 16 James Chepkech Representative of Country Director and County Government Ministry of Education M 17 Moses Karati Country Director of Education Ministry of Education M 18 Susan Mitie County School Meals Programme Officer Ministry of Education F 19 Anne Kimwa County Nutrition Coordinator Ministry of Health F 20 Jackson Langat Country Weights and Measures Officer Ministry of Industry, Trade & Cooperatives M Turkana 21 Samson Akichem Lokale Trader Emany Traders Ltd M Nairobi+ Turkana West Pokot Baringo Farmer Organizations __ 2 2 2 Traders __ 1 2 0 Ministry of Agriculture 0 1 3 2 Ministry of Education 1 3 2 3 Ministry of Health 1 3 1 1 Ministry of Industry, Trade & Cooperatives 0 0 1 0 WFP 10 1 2 USDA 1 __ __ __ TOTAL 13 11 13 8 45 interviews/FGDs WFP Kenya Endline LRP Evaluation Report – May 2020 |Page 109 22 Eli Karani County Agribusiness Officer Ministry of Agriculture and Irrigation M 23 Vitales Juma County M&E Officer Ministry of Agriculture and Irrigation M 24 Josephine Walela County Education Officer Ministry of Education F 25 Peter Magiri Country Director of Education Ministry of Education M 26 Anne Kimwa Country Nutrition Coordinator Ministry of Health F 27 Ann Kanume Wumboi Public Health Officer Ministry of Health, Turkana Central Sub-County F 28 Lelak Mohave Public Health Officer Ministry of Health, Turkana Central Sub-County F 29 Gabriel Ekaale Programme Policy Officer (Agribusiness) WFP M West Pokot 30 Susan Domoo Trader Kapchok Stores F 31 Christine Krop Trader Lelan Stores F 32 Stephen Adieme Agricultural Extension Officer Ministry of Agriculture and Irrigation M 33 Thomas Wasike Acting County Director of Agriculture Ministry of Agriculture and Irrigation M 34 Wekesa Agricultural Extension Officer Ministry of Agriculture and Irrigation M 35 Atina Ntabo Quality Assurance and Standards Officer (Monitoring) Ministry of Education M 36 Jacob Onyiego County Director of Education Ministry of Education M 37 Charles Manyara Sub-County Director of Education Ministry of Education, West Pokot Sub-County M 38 Jane Limangura County Nutrition Coordinator Ministry of Health F 39 Ibrahim Lomoi Chief Officer Ministry of Industry, Trade & Cooperatives M FOCUS GROUP DISCUSSIONS WITH FO MEMBERS Group name Female Male Baringo County 1 Karmoskoi Tingtinyon Farming System 8 9 2 Namunyak Women’s Group 11 0 Turkana County 3 Etic Women’s Group 8 0 4 Kolioro Irrigation Scheme 9 17 West Pokot County 5 Myakith Youth Group 1 12 6 Kameto Women’s Group 4 0 TOTAL 41 38 WFP Kenya Endline LRP Evaluation Report – May 2020 |Page 110 Annex 18: Additional findings from the Farmer Organization survey • Characteristics of surveyed Farmer Organizations, by county. Baseline Endline Baringo Turkana West Pokot Baringo Turkana West Pokot Number of surveyed FOs 46 36 16 23 35 21 Membership Total members 978 31,142 424 11,895 27,165 622 Average number per FO 21 865 26.5 517 776 30 Median number of members 17 394 17.5 22 417 23 Average % female 74% 56.8% 66.3% 72.0% 61.5% 65.3% Active members 83.7% 66.2% 91% 92.6% 61.2% 87.8% • Planting data by county Baseline Endline Baringo Turkana West Pokot Baringo Turkana West Pokot Total area planted by members (acres) Total area 1,200 18,701 634 868 15,947 481 Average 26 519.5 40 38 456 23 Median 17 253 27.5 10 210 10 % of members using certified/recommended inputs 81.2%% 61.7% 90.1% 91.3% 88.6% 52.4% % of members having their soil quality tested before planting No data 28.6% No data 5% 34% 1.3% • Percentage of FOs reporting use of various agricultural inputs • Equipment accessible to FO members 0 20 40 60 80 100 Baringo Baseline Baringo Endline Turkana Baseline Turkana Endline West Pokot Baseline West Pokot Endline Total Baseline Total Endline Seeds Fertilizer Agrochemicals Feeds Other 0 20 40 60 80 100 Baringo Baseline Baringo Endline Turkana Baseline Turkana Endline West Pokot Baseline West Pokot Endline Total Baseline Total Endline Sieve Drier Tarpaulin Sheller Moisture meter Scales Pallets WFP Kenya Endline LRP Evaluation Report – May 2020 |Page 111 • Percentage of FOs reporting different methods of handling their commodity during/after harvest • Assistance provided to FOs from the government or other agencies • Training provided to FOs in the last one year Baseline Endline Baringo Turkana West Pokot Total Baringo Turkana West Pokot Total FOs reporting their committee members have been trained 17.4% 72.2% 12.5% 36.7% 78.3% 71.4% 76.2% 74.7% • Percentage of FOs reporting receiving training for their members in the past one year 0 50 100 Baringo Baseline Baringo Endline Turkana Baseline Turkana Endline West Pokot Baseline West Pokot Endline Total Baseline Total Endline Timely harvest Conduct sorting for the produce after harvesting Placing produce on open/bare ground during harvesting and drying 0% 20% 40% 60% 80% 100% Baringo Baseline Baringo Endline Turkana Baseline Turkana Endline West Pokot Baseline West Pokot Endline Subsidized or free seeds Subsidized or free fertilizer Subsidized or free farming implements (tools) Subsidized or free pesticides/herbicides Providing or rehabilitating storage facilities Loans of agricultural tools or work animals Baseline Endline Baringo Turkana West Pokot Baringo Turkana West Pokot Good agronomic practices 21.7 66.7 68.8 39.1 25.7 33.3 Conservation agriculture 21.7 47.2 81.3 30.4 25.7 28.6 Post-harvest handling 13.0 52.8 68.8 17.4 22.9 33.3 Entrepreneurship 21.7 19.4 100.0 30.4 22.9 23.8 Gender in agribusiness 4.3 47.2 62.5 34.8 20.0 9.5 Setting prices for produce 6.5 38.9 50.0 26.1 20.0 9.5 Procurement processes 4.3 19.4 37.5 26.1 14.3 14.3 Record keeping 26.1 25.0 87.5 43.5 22.9 23.8 Aflatoxin awareness 4.3 44.4 25.0 21.7 20.0 14.3 Financial management 17.4 27.8 31.3 43.5 22.9 23.8 Other 13.0 0.0 0.0 8.7 0.0 0.0 WFP Kenya Endline LRP Evaluation Report – May 2020 |Page 112 Group assets • Land ownership by FOs Baseline Endline Baringo Turkana West Pokot Baringo Turkana West Pokot Percentage of FOs owning land 0% 100% 25% 34.8% 74.3% 33.3% Average size of land owned by FO __ 512 acres 0.9 acres 7.3 acres 845.6 acres 8.6 acres • • Use of most farming equipment90 by FOs did not change significantly since baseline. Most FOs reported using sieves at both baseline and endline, with few other equipment. The main change was in the use of pallets (34% at endline – zero at baseline) and scales (9-32%) increased. • Percentage of surveyed FOs having access to storage facilities Baseline Endline Baringo Turkana West Pokot Baringo Turkana West Pokot Access to storage facilities 60.8% 52.8% 87.5% 30.4% 51.4% 28.6% • FO member training Baseline Endline Baringo Turkana West Pokot Total Baringo Turkana West Pokot Total FOs reporting their committee members have been trained 17.4% 72.2% 12.5% 36.7% 78.3% 71.4% 76.2% 74.7% Average number of committee members trained 3 5 4.5 4 4 15 3 8 FOs reporting their members have been trained 37% 75% 93.8% 60.2% 60.9% 34.3% 33.3% 41.8% • Percentage of FOs reporting receiving training for their committee members in the past one year • Percentage of FOs reporting receiving training for their committee members in the past one year or nominating additional training topics Baringo Turkana West Pokot Total Good agronomic practices 39.1 25.7 33.3 31.7 Conservation agriculture 30.4 25.2 28.6 27.9 Post-harvest handling 17.4 22.9 33.3 24.1 Entrepreneurship 30.4 22.9 23.8 25.3 Gender in agribusiness 34.8 20 9.5 21.5 Price setting 26.1 20 9.5 19 Procurement process 26.1 14.3 14.3 17.7 Record keeping 43.5 22.9 23.8 29.1 Aflatoxin awareness 21.7 20 14.3 19 Financial management 43.5 22.9 32.8 29.1 Other 8.7 0 0 2.5 90 FO member were asked about their use of sieves, driers, tarpaulins, shellers, moisture meters, scales and pallets. Baringo Turkana West Pokot Total Group dynamics/governance 52.2 54.3 61.9 55.7 Organizational behaviour 52.2 40 47.6 45.6 Leadership skills 60.9 57.1 71.4 62 Other 4.4 20 19.1 15.2 WFP Kenya Endline LRP Evaluation Report – May 2020 |Page 113 Additional training requested • Baringo: Group dynamics; Marketing and market linkages; Post-harvest management; Agribusiness and financial management; Conservation agriculture; Increasing productivity and sound agricultural practice • Turkana: Cooperative management; Good agronomic practices; Marketing and market linkages; Post-harvest management; Agribusiness and financial management; Aflatoxin prevention • West Pokot: Group marketing; Financial management and record keeping, and how to develop school meals menus using local and nutritious produce • Percentage of FOs reporting members sell commodities to various buyers • Percentage of FOs reporting barriers to selling their members products Baringo Turkana West Pokot Baseline Endline Baseline Endline Baseline Endline Households 50.0 34.8 66.7 37.1 0.0 42.9 Retail traders 58.3 47.7 88.9 37.1 50.0 38.1 Millers/brewers 8.3 8.7 11.1 8.6 25.0 0.0 Other traders 50.0 21.7 11.1 22.9 25.0 33.3 Hospitals 0.0 0.0 0.0 2.9 25.0 0.0 Government food agencies/NCPB 0.0 4.3 22.2 0 0.0 4.8 International development agencies (WFP) 0.0 0.0 44.4 8.6 0.0 0.0 Schools 16.7 17.4 33.3 5.7 25.0 33.3 Others 0.0 8.7 0.0 8.6 0.0 0.0 Baseline Endline Percentage of FOs Baringo Turkana West Pokot Total Baringo Turkana West Pokot Total Limited consumer demand 57.25 8.33 4.17 28.57 13 25.7 28.6 22.8 Limited access to price information 4.35 22.22 25.00 8.50 21.7 17.1 23.8 20.3 High cost of collecting/ preparing commodities for market 7.97 2.78 10.42 7.14 17.4 17.1 4.8 13.9 Limited access to credit 0.00 22.22 6.25 1.36 13.0 14.3 14.3 13.9 Low volume of commodities available from members 9.42 16.67 2.08 6.80 4.4 25.7 23.5 19 Government trade restrictions 1.45 25.00 8.33 4.76 4.4 0 0 1.3 Poor transport infrastructure 2.90 19.44 33.33 9.52 39.1 34.3 47.6 39.2 Not able to meet quality demands 17.39 13.89 27.08 15.65 8.7 0 4.8 3.8 Unpredictable prices 0.00 5.56 0.00 0.00 34.8 20 47.6 31.65 Other 0.72 0.00 0.00 1.02 4.4 2.9 9.5 5.1 WFP Kenya Endline LRP Evaluation Report – May 2020 |Page 114 • Volume and value of total sales by FOs during last one year Baseline Endline Baringo Turkana West Pokot Baringo Turkana West Pokot Volume of maize sales per FO (MT) Average 30.7 (n=42) 9.6 (n=8) 392.1 (n=16) 8 (n=9) 32.4 (n=29) 5.4 (n=12) Median 19.8 9.6 12.5 8 15.2 4.1 Volume of bean sales per FO (MT) Average 11.9 (n=37) NIL 128.8 (n=14) 7.8 (n=7) 847.9 (n=1) 2.9 (n=9) Median 5.1 2.9 7.8 847.9 2.9 Volume of sales of other commodities per FO (MT) Average 10.1 19.0 12.7 8.0 15.2 2.5 Median 2.5 6.3 1.1 8.0 14.0 2.5 Total volume of sales Total all FOs 2,020.4 MT 176 MT 9,334.3 MT 359MT 9,757MT 66.6MT Median 38.9 MT 9.3MT 19.1 MT 8MT 14MT 2.5MT WFP Kenya Endline LRP Evaluation Report – May 2020 |Page 115 Annex 19: Additional findings from the trader survey • Personal characteristics of surveyed traders, by county (n) Baseline Endline Baringo West Pokot Total Baringo West Pokot Total Number of surveyed traders Total 13 41 54 3 25 28 Gender Male 7 22 29 (53.7%) 0 12 12 (42.9%) Female 6 19 25 (46.3%) 3 13 16 (57.1%) • Age breakdown of surveyed traders (n) Baseline Endline Baringo West Pokot Total Baringo West Pokot Total Male Female Male Male Female Male Male Female Male Female Male Female 18-35 years 2 4 11 11 13 15 0 0 4 2 4 2 36-55 years 5 2 11 8 16 10 0 3 8 11 8 14 Total 7 6 22 19 29 25 0 3 12 13 12 16 13 41 54 3 25 28 • Educational qualifications of surveyed traders (n) Baseline Endline Baringo West Pokot Total Baringo West Pokot Total Male Female Male Female Male Female Male Female Male Female Male Female Primary 1 0 12 7 13 7 0 0 5 8 5 8 Secondary 6 1 7 8 13 9 0 2 7 2 7 4 Diploma 0 3 0 3 0 6 0 1 0 2 0 3 Degree 0 2 3 1 3 3 0 0 0 1 0 1 Total 7 6 22 19 29 25 0 3 12 13 12 16 13 41 54 3 25 28 • Business information of surveyed traders, by gender (West Pokot only) West Pokot Male Female Average time in business 11 years 12 years Average number of employees 2 3 Percentage of targeted traders with a valid business license 75% 77% Percentage of targeted traders with a business registration certificate 33% 62% • Main mode of commodity transportation used by surveyed traders (%) • Main commodity storage facility used by surveyed traders (%) Baseline Endline Baringo West Pokot Baringo West Pokot Carried by hand/Head/Back 0.0 4.9 0.0 28.0 Bicycle or motorcycle 7.7 24.4 66.7 80.0 Pick up 69.2 17.1 33.3 28.0 Lorry 53.8 63.4 66.7 72.0 Public transport 7.7 0.0 33.3 40.0 None/Don’t transport 0.0 4.9 0.0 0.0 Baseline Endline Baringo West Pokot Baringo West Pokot Basic earth granaries 15.4 7.3 0.0 4.0 Permanently owned warehouse 7.7 39.0 33.3 16.0 Permanently rented warehouse 76.9 53.7 66.7 80.0 WFP Kenya Endline LRP Evaluation Report – May 2020 |Page 116 • Total estimated value and volume of grains in stock Baseline Endline Baringo West Pokot Baringo West Pokot KES Average 239,615 159,996 506,850 983,290 Median 117,500 10,800 560,000 297,500 Range 0-1,260,000 0-5 million 160,550 - 800,000 0 – 7,350,000 MT Average 12.6 1348.3 226.1 25.2 Median 5 200 266.4 5.9 Range 0-50 0-8,925 3-180 0-280 WFP Kenya Endline LRP Evaluation Report – May 2020 |Page 117 Annex 20: Additional findings from the school survey • Number of days schools reported NOT providing school meal Baringo Turkana West Pokot Total LRP Non- LRP LRP Non- LRP LRP Non- LRP LRP Non- LRP Baseline – Term 1, 2018 Mean 9.6 2.9 2.6 0.5 5.9 10.1 5.8 4.5 Maximum 32 21 24 10 45 42 45 42 Grouped median 2 1.5 1.8 0.1 2.6 3.5 1.4 0.4 Endline – Term 3, 2019 Mean 36.8 14.2 19.5 24.4 13.8 20.2 23 20 Maximum 45 45 45 45 45 45 45 45 Grouped median 42.9 2 17 18.3 6.3 14.7 21.8 10 • Number of suppliers utilized by the school Baseline – Term 1, 2018 Mean 1 1 1 1 2 1 Range 2 0 2 2 2 1 Minimum 1 1 1 1 1 0 Maximum 3 1 3 3 3 1 Grouped median 1 1 1 1 2 1 Endline – Term 3, 2019 Mean 1 1 1 1 1 1 Range 0 1 0 0 1 1 Minimum 1 1 1 1 1 1 Maximum 1 2 1 1 2 2 Grouped median 1 1 1 1 1 1 • Percentage of schools reporting consumption of foods for school meals during Term 3, 2019 School Type Food Item Baseline Endline N Percent of Cases N Percent of Cases LRP School Rice 93 96.9% 55 57.3% Wheat 45 46.9% 0 0% Beans 62 64.6% 69 71.9 Yellow split peas 34 35.4% 0 0% Oil 96 100.0% 66 68.8% Salt 94 97.9% 52 54.2% Vegetables 1 1.0% 1 1.0% Onions 1 1.0% 1 1.0% Non LRP School MSB 1 1.1% 0 0% Beans 95 100.0% 85 89.5% Maize 95 100.0% 50 52.6% Oil 95 100.0% 85 89.5% Salt 95 100.0% 78 82.1% Onions 1 1.1% 2 2.1% Baringo Turkana West Pokot LRP Non- LRP LRP Non- LRP LRP Non- LRP WFP Kenya Endline LRP Evaluation Report – May 2020 |Page 118 • Additional monies paid by teachers (KES) Baringo Turkana West Pokot Total LRP Non- LRP LRP Non- LRP LRP Non- LRP LRP Non- LRP Baseline - Term 1, 2018 Food transport 1,500 __ 10,000 __ 4,250 5,620 5,000 5,620 Food storage __ __ __ __ __ __ __ __ Food unloading 1,500 __ 2,400 __ __ __ 2,220 __ Other __ 3,000 5,000 __ __ 10,000 5,000 6,500 Total 3,000 3,000 17,400 0 4,250 15,620 12,220 12,120 Endline - Term 3, 2019 Food transport __ 2,750 __ __ 7,688 4,229 7,688 4,018 Food storage __ __ __ __ __ __ __ __ Food unloading __ 500 1,010 __ __ 1,000 1,010 750 Other 400 6,000 __ __ 2,400 2,125 1,400 3,417 Total 400 9,250 1010 10,088 7,354 10,098 8,185 • Percentage of schools completing procurement before Term 3, 2019 Completed procurement – Term 3, 2019 Baringo Turkana West Pokot Total LRP Non- LRP LRP Non- LRP LRP Non- LRP LRP Non- LRP Yes __ 55.2% 5.7% 3% 6.3% 39.4% 4.2% 31.6% No 75.9% 31.0% 62.9% 81.8% 43.8% 57.6% 60.4% 57.9% N/A (food was provided) 24.1% 13.8% 31.4% 15.2% 50% 3.0% 35.4% 10.5% • Percentage of schools reporting using these food commodities for their school lunches Baseline Endline School Type Food Item N Percent of Cases N Percent of Cases LRP School Rice 93 96.9% 55 57.3% Wheat 45 46.9% 0 0% Beans 62 64.6% 69 71.9 Yellow split peas 34 35.4% 0 0% Oil 96 100.0% 66 68.8% Salt 94 97.9% 52 54.2% Vegetables 1 1.0% 1 1.0% Onions 1 1.0% 1 1.0% Non LRP School MSB 1 1.1% 0 0% Beans 95 100.0% 85 89.5% Maize 95 100.0% 50 52.6% Oil 95 100.0% 85 89.5% Salt 95 100.0% 78 82.1% Onions 1 1.1% 2 2.1% • Summary of baseline findings on timeliness Target LRP schools Non-LRP comparison schools Percentage of schools procuring food before the beginning of Term 1, 2018 90% N/A 76% Percentage of schools with food delivered before the beginning of Term 1, 2018 90% 19.8% 57.3% Percentage of schools with food delivered before the beginning of Term 2, 2018 90% 34.4% 28.1% WFP Kenya Endline LRP Evaluation Report – May 2020 |Page 119 Percentage of schools procuring food before the beginning of Term 3, 2019 90% 56.9% 76.7% Percentage of schools with food delivered before the beginning of Term 3, 2019 90% 11.8% 55.8% Percentage of schools with food delivered before the beginning of Term 1, 2020 90% 75% 74.7% • Percentage of schools experiencing problems with food commodities received - ENDLINE Baringo Turkana West Pokot Total LRP Non- LRP LRP Non- LRP LRP Non- LRP LRP Non- LRP Bags arrived in poor condition 0% 0% 0% 0% 0% 3.0% 0% 1.1% Food was infested 0% 0% 2.9% 0% 0% 3.0% 1.0% 1.1% Incorrect amount arrived (i.e. not as ordered) 0% 3.4% 2.9% 0% 0% 6.1% 0% 3.2% Supplier refused to bring food directly to the school 0% 0% 0% 3.0% 6.3% 3.0% 2.1% 2.1% WFP Kenya Endline LRP Evaluation Report – May 2020 |Page 120 • Schools receiving food from alternative sources Baringo Turkana West Pokot Total Secondary sources of food LRP Non￾LRP LRP Non- LRP LRP Non- LRP LRP Non- LRP County Government 1 (Day School) __ __ __ __ 4 (Day School) 1 (Day School) 4 (Day School) School garden/farm 1 (Day School) 1 (Day & Boarding School) __ __ __ __ 1 (Day School) 1 (Day & Boarding School) 1 (Day School) 1 (Day & Boarding School) 1 (Day School) 1 (Day & Boarding School) Church 1 (Day School) 1 (Day School) __ 2 (Day School) 1 (Day & Boarding School) 2 (Day & Boarding School) 1 (Day & Boarding School) 1 (Day School) 2 (Day & Boarding School) 3 (Day School) 2 (Day & Boarding School) PTA __ 2 (Day School) __ 2 (Day School) 1 (Day & Boarding School) 1 (Day School) 1 (Boarding School) 10 (Day & Boarding School) 11 (Day & Boarding School) 1 (Day School) 1 (Boarding School) 10 (Day & Boarding School) 4 (Day School) 12 (Day & Boarding School) Direct contribution from parents 1 (Day & Boarding School) __ __ 1 (Day School) 1 (Day & Boarding School) 5 (Day & Boarding School) 2 (Day School) 1 (Day & Boarding School) 6 (Day & Boarding School) 3 (Day School) 2 (Day & Boarding School) NGOs 1 (Day School) 1 (Day & Boarding School) 1 (Day School) __ 2 (Day School) 2 (Day & Boarding School) __ 1 (Day School) 1 (Day School) 1 (Day & Boarding School) 4 (Day School) 2 (Day & Boarding School) Total 4 (Day School) 3 (Day & Boarding School) 4 (Day School) __ 7 (Day School) 5 (Day & Boarding School) 1 (Day School) 1 (Boarding School) 17 (Day & Boarding School) 8 (Day School) 14 (Day & Boarding School) 5 (Day School) 1 (Boarding School) 20 (Day & Boarding School) 19 (Day School) 19 (Day & Boarding School) 11 12 41 64 WFP Kenya Endline LRP Evaluation Report – May 2020 |Page 121 Annex 21: Bibliography Awiti, C et al (2018) Kenya Economic Update: Policy options to advance the Big 4 - unleashing Kenya’s private sector to drive inclusive growth and accelerate poverty reduction (English). April 2018, Edition No. 17, World Bank Group Dunn, S & Otsola, J (2018) Baseline of the Local and Regional Food Aid Procurement (LRP) Project in Kenya, 2017- 2020. Final Report, June 2018. FAO/Government of Kenya (2018) Kenya Food Composition Tables. Nairobi. FEWSNET (2019) Kenya Food Security Outlook Update: 3.1 million expected to face crisis (IPC3) or worse by October. August 2019. Government of Kenya’s Vision 2030: https://vision2030.go.ke http://www.fao.org/kenya/fao-in-kenya/kenya-at-a-glance/en http://www.vision2030.go.ke https://www.imf.org/external/datamapper/profile/KEN/WEO Hanrahan, C.E. (2010) Local and regional procurement for U.S. International Emergency Food Aid. Congressional Research Service. Kisingu, J. & Mutuku, J. (2018) Food Supply Chain Assessment Report: Baringo County Kisingu, J., Mutuku, J. & Kithale, D (2018) Food Supply Chain Assessment Report: West Pokot County Republic of Kenya (2009) National Agricultural Sector Development Strategy 2009-2020. Republic of Kenya (2011) National Food Security and Nutrition Policy. Agricultural Sector Coordination Unit (ASCU). Republic of Kenya (2011) National Zero Hunger Strategic Review. May 2018 Republic of Kenya (2012) National Agribusiness Strategy. Agricultural Sector Coordination Unit (ASCU). Republic of Kenya (2016) Cabinet memo on strengthening smallholder farmers’ production and marketing by facilitating direct access to public food procurement opportunities. Ministry of Agriculture, Livestock and Fisheries and National Treasury. November 2016. Republic of Kenya (2016) Home Grown School Meals Programme Implementation Guidelines. World Food Programme. Republic of Kenya (2017) Assessment report on public institutional food procurement form small holder farmers in Kenya. December 2017. Republic of Kenya (2018) National school meals and nutrition strategy 2017-2022. Ministry of Education, Ministry of Health and the Ministry of Agriculture and Irrigation. Republic of Kenya (2018) The National Education Sector Strategic Plan for the period 2018-2022. Ministry of Education. Republic of Kenya (2018) Third Medium-Term Plan (2018-2022): Transforming Lives: Advancing socio-economic development through the “Big Four”. National Treasury and Planning. Nairobi http://extwprlegs1.fao.org/docs/pdf/ken189643.pdf Republic of Kenya (2018) Towards zero hunger: Strategic Review. May 2018 Republic of Kenya (2018) Turkana SMART Nutrition Surveys. June 2018 Republic of Kenya (2018) West Pokot County Integrated SMART Survey Report. June 2018. Republic of Kenya (2019) Agricultural Sector Transformation and Growth Strategy: Towards sustainable agricultural transformation and food security in Kenya (2019-2029). Abridged Version. Ministry of Agriculture and Irrigation. WFP Kenya Endline LRP Evaluation Report – May 2020 |Page 122 Republic of Kenya (2019) Baringo County: 2019 Long rains food and nutrition security assessment report. July 2019. Republic of Kenya (2019) Sessional Paper No.2 of 2019 on National Policy on Gender and Development. Towards creating a just, fair and transformed society free from gender-based discrimination in all spheres of life practices. Ministry of Public Service, Youth and Gender. Taylor, J.E et al (2019) Potential local economy impacts of Kenya’s Home-Grown School Meals Program. UC Davis. The County Early Childhood Education Bill (2018). http://kenyalaw.org/kl/fileadmin/pdfdownloads/bills/2018/CountyEarlyChildhoodEducationBill_2018.pdf Turkana County (2018) Turkana County Integrated Development Plan, CIDP II: 2018-2022. Popular version. UNCTAD (2018) East African Community Regional Integration: Trade and Gender Implications. United Nations. UNDP (2019) Inequalities in Human Development in the 21st Century. Briefing note for countries for the 2019 Human Development Report. Kenya. United Nations Evaluation Group (2016). Norms and Standards for Evaluation. New York: UNEG. United Nations Kenya (2018) United Nations Development Assistance Framework (UNDAF) for Kenya, 2018- 2022. Nairobi WFP (2016) Turkana County Strategy to improve access to markets for smallholder farmers and small-scale processors (2016-2019). WFP (2018) Safe Food Handling and Preparation, sorghum and cow peas menu and recipes development report – Turkana County – September 2018; WFP (2019) Sorghum, finger millet, green grams and cowpeas menu and training package development workshop report for West Pokot County – July 2019. WFP (2019) Sorghum, finger millet, green grams and cowpeas menu and training package development workshop report for Baringo County – July 2019. WFP (2019) Traditional Nutritional Foods in Baringo County: Value Chain Study Report WFP (2019) Traditional Nutritional Foods in West Pokot County: Value Chain Study Report WFP (2020) School meals supply chain compliance assessment report WFP Kenya (2018) Kenya Country Strategic Plan 2018-2023. WFP/EB.A/2018/8-A/3 WFP Kenya (2018) The Kenya Retail Engagement Initiative: Getting more value from Bamba Chakula cash transfers. Infobrief No. 17. Kenya WFP Kenya Endline LRP Evaluation Report – May 2020 |Page 123 5 List of Acronyms AMAL Agriculture, Market Access and Linkage BMZ Federal Ministry of Economic Cooperation and Development (Germany) CIDP County Integrated Development Plan CO (WFP) Country Office CSP (WFP) Country Strategic Plan DoA Department of Agriculture DoE Department of Education ECDE Early Children Development Education EQAS (WFP Centralized) Evaluation Quality Assurance Service ET Evaluation Team EUTF European Union Trust Fund FAO (UN) Food and Agriculture Organization FCT Food Composition Tables FDC Forward Delivery Contract FO Farmer Organization GAM Global Acute Malnutrition GDI Gender Development Index GDP Gross Domestic Product GEEW Gender Equality and Empowerment of Women GII Gender Inequality Index HDI Human Development Index HGSMP Home Grown School Meals Programme HQ Headquarters IPC Integrated (Food Security) Phase Classification KES Kenyan Shilling (currency) LEWIE Local Economy Wide Impact Evaluation LRP Local and Regional Food Aid Procurement Programme MoA Ministry of Agriculture and Irrigation MoE Ministry of Education MoH Ministry of Health MoITC Ministry of Industry, Trade and Cooperatives NESSP National Education Sector Strategic Plan NGO Non-Governmental Organization NSMNS National School Meals and Nutrition Strategy OECD-DAC Organisation for Economic Co-operation and Development’s Development Assistance Committee P4P Purchase for Progress PMP Performance Monitoring Plan RBN (WFP) Regional Bureau Nairobi RF Results Framework SDG Sustainable Development Goal SMC School Meal Committee SY School Year TOR Terms of Reference UNDSS United Nations Department of Safety & Security UNEG United Nations Evaluation Group UNHCR United Nations High Commission for Refugees WFP Kenya Endline LRP Evaluation Report – May 2020 |Page 124 UNICEF United Nations Children’s Fund USD United States Dollar (currency) USDA United States Department of Agriculture VCA Value Chain Assessment WASH Water, sanitation and hygiene WFP World Food Programme [Name of commissioning Office ] [Link to the website] [Place, Month and Year, Report number]