aa Prepared Decentralized Evaluation Evaluation of USDA’s Local and Regional Food Aid Procurement Program (Rwanda 2017-2019) Endline – Final Report May, 2020 WFP Rwanda Evaluation Manager: Sameera Ashraf AAS Prepared by Muriel Visser, team leader Laure Steer, Agricultural Value Chain and Farmer Organization Specialist Ernest Midega, Statistician and data analyst Moses Mwangi, Lead statistician and quantiative quality assurance expert Raphael Rurangwa, Agriculture expert DECENTRALIZED EVALUATION FOR EVIDENCE-BASED DECISION MAKING Agreement: LRP-696-2016/019-00 Evaluation Report – LRP Rwanda Endline i |P a g e Acknowledgements The evaluation team gratefully acknowledges the contributions of cooperative leaders and members as well as other stakeholders in Rwanda for this report. The team also thanks the WFP Country Office in Rwanda for its assistance in planning and implementing the baseline and endline data collection. Disclaimer The opinions expressed in this report are those of the Evaluation Team, and do not necessarily reflect those of the World Food Programme. Responsibility for the opinions expressed in this report rest solely with the authors. Publication of this document does not imply endorsement by WFP of the opinions expressed. The designation employed and the presentation of material in maps do no imply the expression of any opinion whatsoever on the part of WFP concerning the legal or constitutional status of any country, territory or sea area, or concerning the delimitation of frontiers. DISCLAIMER: This publication was produced at the request of the United States Department of Agriculture. It was prepared by an independent third-party evaluation firm. The author’s views expressed in this publication do not necessarily reflect the views of the United States Department of Agriculture or the United States Government. Accessibility Note: An accessible version of this document can be made available by contacting fas.monitoring.evaluation@usda.gov Evaluation Report – LRP Rwanda Endline ii |P a g e Evaluation Report – LRP Rwanda Endline iii |P a g e Table of Contents Acknowledgements.............................................................................................................i Disclaimer ...........................................................................................................................i Figures................................................................................................................................ v Tables................................................................................................................................ vi Executive Summary ......................................................................................................... viii Methodology ix Key Findings ix Operational Recommendations xii Strategic Recommendations xii 1. Introduction .............................................................................................................. 14 1.1. Overview of the Evaluation Subject 14 1.2. Context 16 1.3. Evaluation Methodology and Limitations 19 1.4. Overview of the characteristics of survey respondents 22 1.5. Ethics and gender 23 1.6. Limitations 24 2. Evaluation Findings.................................................................................................... 25 2.1. Evaluation Question 1 - How relevant is the design of the intervention in terms of the context; needs of the most vulnerable groups; needs of male and female beneficiaries; and priorities of the government and WFP partners? 25 2.2. Evaluation Question 2 - How has cooperative ability ‘to be’, ‘to organize’, ‘to relate’ and ‘to do’ evolved from baseline to endline? 28 2.3. Evaluation Question 3 - Have the LRP project interventions affected male and female SHF knowledge, capacity and choices/behaviour from baseline to endline? 38 2.4. Evaluation Question 4 - Has the LRP contributed to creating new market opportunities for male and female SHF? 43 2.5. Evaluation Question 5 - To what extent do producers market food products that meet quality standards and that are nutritious and culturally accepted? 47 2.6. Evaluation Question 6 - Has the LRP programme affected male and female SHF income, and in what way? 48 2.7. Evaluation Question 7 - Is there evidence that the LRP programme affected male and female SHF differently? 51 Evaluation Report – LRP Rwanda Endline iv |P a g e 2.8. Evaluation Question 8 – What internal and external factors affected results? 55 3. Conclusions and Recommendations .......................................................................... 58 3.1. Overall Assessment/Conclusions 58 3.2. Lessons learnt 61 3.3. Recommendations 61 Annexes............................................................................................................................ 65 Annex 1. Terms of Reference ..................................................................................................65 Annex 2. LRP Project activities and targets .............................................................................81 Annex 3. Detailed Theory of Change for the LRP Project .........................................................82 Annex 4. Evaluation Matrix.....................................................................................................83 Annex 5. List of persons met at endline ..................................................................................93 Annex 6. Survey tool...............................................................................................................94 Annex 7. How has cooperatives ability ‘to be’, ‘to organize’, ‘to relate’ and ‘to do’ evolved from baseline to endline? .....................................................................................................................97 Annex 8. Have the LRP project interventions affected male and female SHF knowledge, capacity and choices/behaviour from baseline to endline? ............................................................................. 109 Annex 9. Additional information on the question: Has the LRP contributed to creating new market opportunities for male and female SHF? (including from schools)............................................... 116 Annex 10. Additional information on the question: To what extent do producers market food products that meet quality standards and are nutritious and culturally accepted? ..................... 119 Annex 11. Has the LRP programme affected male and female SHF income and in what way?121 Annex 12. Assessment of the project’s performance looking at the monitoring indicators.. 122 Annex 13. Bibliography....................................................................................................... 124 List of Acronyms......................................................................................................................... 136 Evaluation Report – LRP Rwanda Endline v |P a g e Figures – Main report Figure 1: Representation of cooperative abilities (scoring based on qualitative data collection during the endline)..29 Figure 2: Farmers reasons for being interested in cooperative activity (qualitative data collection based on 12 cooperatives, multiple option choice)..................................................................................................................31 Figure 3: Level of payment of shares in cooperatives (qualitative data collection)......................................................32 Figure 4: Cooperative resource mobilisation systems (qualitative data collection)……………………………………..……………31 Figure 5 - Percentage of SHF with excellent score on training on GAP and PHHS (quantitative survey)……36 Figure 6 - Percentage of SHFs that have heard of GAP and PHHS (quantitative survey)……………………………36 Figure 7: Evolution of the amount spent on fertilizers in RWF (quantitative data collection)…………………..38 Figure 8 (left): LRP cooperative contract achievement rate for (RWARRI) ..................................................................44 Figure 9 (right): Factors that influence cooperative choice by clients (multiple options possible, qualitative data collection, N=12 cooperatives).............................................................................................................................44 Figure 10: Quantity of maize kept for household consumption during the last planting season (survey results)........47 Figure 11: Percentage of farmers who purchased maize and beans for family consumption in seasons A and B .......49 Figure 12: Quantity of maize sold in kg by season from baseline to endline (quantitative survey)..............................49 Figure 13: Amount of money from the sales of maize from baseline to endline (quantitative survey) .......................49 Figure 14: Percentage of farmers who purchased maize and beans for family consumption in seasons A and B (survey results)..................................................................................................................................................................49 Figure 15 - Percentage of farmers who purchased maize and beans for family consumption in seasons A and B (survey results)………………………………………………………………………………………………………………………………50 Figures – Annexes Figure 14: location of FTMA cooperatives in the four targeted districts (WFP):...........................................................81 Figure 15: Basic information on LRP cooperatives (qualitative data collection and WFP documents).........................97 Figure 16: Representation of the abilities of high potential cooperatives (scoring based on qualitative data collection during baseline) ....................................................................................................................................................98 Figure 17: Representation of the abilities of cooperatives that are losing momentum (scoring based on qualitative data collection at baseline)...................................................................................................................................99 Figure 18: Representation of the abilities of nascent cooperatives (scoring based on qualitative data collection during baseline) ...............................................................................................................................................................99 Figure 19: Importance of support received from LRP (open choice question to 12 cooperatives, qualitative data collection) ...........................................................................................................................................................101 Figure 20: Cooperative ‘common vision (qualitative data collection, to 12 cooperatives, multiple option choice)..102 Figure 21: Cooperative financial resources (qualitative data collection)....................................................................103 Figure 22: Use of cooperatives resources (multiple option choice, qualitative data collection, 12 cooperatives).....103 Evaluation Report – LRP Rwanda Endline vi |P a g e Figure 23 : Risks of credit for cooperatives (multiple choice option, qualitative data collection based on 12 cooperatives)......................................................................................................................................................104 Figure 24 : Percentage of the marshland dedicated to maize (qualitative data collection and data from WFP) .......105 Figure 25: Crops grown by season (qualitative data collection).................................................................................105 Figure 26: Maize production estimation (qualitative data collection and data from WFP)........................................106 Figure 27: Evolution of cooperatives sale and buyers (qualitative data collection)....................................................106 Figure 28: Evolution of cooperative post-harvest handling and storage capacities (qualitative data collection) ......106 Figure 29 : Analysis of the evolution of services received by cooperative members (quantitative data collection)..108 Figure 30 : Percentage of farmers planting maize, comparing baseline and endline for season A and B (quantitative survey) ................................................................................................................................................................112 Figure 31- Percentage of farmers planting beans, comparing baseline and final for season A and season B (quantitative survey)……………………………………………………………………..……………………………………………………………………….110 Figure 32 : SHFs source of fertilizers (quantitative data collection)............................................................................114 Figure 33: Evolution of maize production in Rwanda (NISR annual surveys)..............................................................116 Figure 34 : Delivery of produce from LRP cooperatives to the formal market (RWARRI)...........................................117 Figure 35: Percentage of farmers by household preference on characteristics of the maize and beans (quantitative survey) ................................................................................................................................................................119 Figure 36 - Nutrition analysis of a daily meal consisting of 120g maize meal, 30g beans and 15g vegetable oil .......120 Figure 37: Net margin/ha for maize cultivation based on different cropping systems, price and buyers (qualitative data collection) ...........................................................................................................................................................121 Tables – main report Table 1 : Overview of evaluation questions................................................................................................................. viii Table 2: Actors working in agriculture and rural development in Rwanda (documentary review) ..............................19 Table 3: Agreed evaluation questions and their coverage at baseline and end-line ....................................................20 Table 4 : Background characteristics of the small holder farmers (SHF)......................................................................22 Table 5 : Relevance of LRP interventions to the priorities of the GoR ..........................................................................26 Table 6: Women’s participation in LRP's activities (WFP reports).................................................................................52 Tables – Annexes Table 7: Cooperative capacity to document their activities and resources (qualitative data collection) ...................102 Table 8: Evaluation team assessment of cooperative document and record keeping system (qualitative data collection) ............................................................................................................................................................................102 Table 9: Analysis of extension services received by SHFs (quantitative survey) .........................................................109 Evaluation Report – LRP Rwanda Endline vii |P a g e Table 10 : Number of SHFs trained on GAP and PHHS (WFP/RWARRI).......................................................................109 Table 11: Analysis of the training received by SHFs (quantitative survey)..................................................................110 Table 12 : Adoption of GAP practices (mVAM results)................................................................................................111 Table 13: Mean hectares for maize and beans planting (quantitative survey)...........................................................113 Table 14 : PHE received by cooperatives (WFP)..........................................................................................................114 Table 15 : how SHFs are using the profit generated my maize (FGDs)........................................................................121 Evaluation Report – LRP Rwanda Endline viii |P a g e Executive Summary This endline report was commissioned by the Rwanda World Food Programme (WFP) Country Office and has been prepared as part of the activity evaluation of the Local and Regional Food Aid Procurement (LRP) project funded by the United States Department of Agriculture (USDA). The baseline took place in October-November 2017, and the endline data was collected in June 2019. The purpose of the endline was to compare the situation of the LRP intervention at the time the project was ending with the starting point in September 2017. The evaluation combined accountability and learning objectives. At endline, the evaluation focused on providing a comprehensive assessment of the relevance, effectiveness, and lessons learned from the intervention. The expected users for this report are USDA and the WFP Rwanda County Office, as well as the Government of Rwanda and its partners. Agriculture is a key element of Rwanda’s development policy (Vision 2020 and 2050). It contributes 33 percent of the Gross Domestic Product and employs 80 percent of the population. However, farmers still face many challenges including poor capacity, access to inputs, and access to markets. These factors were taken into account when designing the LRP project. The objective of the LRP project has been to strengthen farmer cooperatives to promote increased use of locally purchased food. Expected outcomes include improving access to loans and markets and enhancing cooperative capacity to be reflected in increased sales, improved quality of produce, and reduced commodity losses. The LRP covered five key project activities: purchasing of maize and beans from small holder farmers (SHF), building capacities of SHFs, connecting farmers to the Patient Procurement Platform, connecting SHFs to new markets and collaborating with the Government of Rwanda. The LRP has been implemented by WFP in four districts (Huye, Gisagara, Nyaruguru and Nyamagabe), with a total budget of USD two million, of which USD 1.36 million was assigned to the procurement of food from SHF. The total number of beneficiaries of the LRP over two years was 5,617 farmers and respectively 43,855 and 41,521 primary school pupils in 2018 and 2019. The evaluation covered the LRP cooperative activities and outcomes. Data on the coverage of LRP in terms of primary pupils have been included in this evaluation report, but the purchasing of food for primary schools was not directly evaluated.1 Table 1 : Overview of evaluation questions Evaluation question To what extent was the design of the intervention relevant to wider context, aligned with needs of the most vulnerable groups, cognizant of the needs of male and female beneficiaries, and in line with priorities of the government and WFP partners? 1 The food purchased and delivered by WFP through LRP is part of the Home Grown School Feeding Programme and will be evaluated as such in future. Evaluation Report – LRP Rwanda Endline ix |P a g e Evaluation question What are the effects of the project on the cooperatives’ ability to be, to organize, to relate, and to do? How does the LRP programme affect male and female SHF capacity and behaviour? Has the LRP contributed to creating new opportunities for male and female SHF? What is the level of participation of men and women? Are women well represented, including in leadership positions? What are the disaggregated effects on women? Has the LRP programme affected male and female SHF differently? What internal and/or external factors affected the project’s achievement of intended results? How and to what extent does the LRP programme contribute to producers marketing food products that meets quality standards, are nutritious, and are culturally acceptable? How does the LRP programme affect male and female SHF income? Is there emerging evidence that the cooperatives have capacity to create linkages with schools? Methodology At inception for the baseline, the evaluation team reviewed the feasibility of doing an impact evaluation which had been suggested as part of the terms of reference, but it was concluded that such a design was not feasible. The agreed evaluation design therefore used a before and after approach to examining the effects of the intervention2 . At final evaluation after the intervention (T1), data was collected on the same variables as the baseline, and a comparison done to determine changes in the indicators. A survey was applied to the 16 targeted cooperatives and covered 828 SHF. In-depth interviews were conducted with 12 cooperatives and with members of six cooperatives. The WFP implementation team, value chain actors (producers, middlemen, buyers, agro-dealers), support actors (local Non￾Governmental Organizations, extension agents), as well as national and local authorities and decentralized units from the relevant ministries were also interviewed. The before and after design was combined with a contribution analysis approach at endline which examined WFP’s work in relation to the work by other partners. The main limitations at endline were the limited availability of cooperative executives and members, the limited specific reporting (as LRP reporting is integrated with the reporting of the Farmer to Market Alliance (FTMA) reports) and the lack of documentation existing at cooperative level. In addition, it was not possible to cross-check information on quantity and price from interviews with documentary sources. Key Findings Specific findings include: EQ 1 - Project relevance/coherence and alignment: The design of the project was relevant. The project has targeted some of the most important constraints that SHF’s face in developing 2 At baseline (T0), quantitative and qualitative data was collected using a combination of documentation/secondary data review, a survey, in-depth key informant interviews, and observation. Data collection for the endline used the same data collection methods Evaluation Report – LRP Rwanda Endline x |P a g e agricultural activities in a profitable way (access to inputs, credit, capacity building and development of marketing). The design aligned with the policies and strategies of the Government of Rwanda and with the interventions of development partners. The project was well aligned with the WFP strategic outcome on SHFs. EQ 2 – Cooperative capacity: The evaluation examined cooperative capacity from the perspective of ability ‘to be’, ‘to do’, ‘to relate’ and ‘to perform’: • Cooperative ability ‘to be’ was found to be relatively good, though ownership and existence of a common vision among cooperative members remained low. • Abilities of cooperatives ‘to do’ was found to be limited, especially as relates to management tools. • In terms of the ability ‘to relate’, cooperatives were already connected to a network of partners and have developed business partnerships with buyers. However, at endline cooperatives remained dependent on the LRP project to make the linkages with these partners. • Cooperatives ability ‘to perform’ improved over the project period but remains fragile. Aggregation of products increased, and some cooperatives now demonstrate a business mindset. However, cooperatives are still unable to demonstrate whether they are making profit. EQ 3 – Male and female capacity and behaviour: Male and female SHF’s behaviour is driven by the need to secure household food consumption, the lack of funds to invest in production, and a focus on minimizing production risks. Agricultural knowledge and practices improved over the project period. Farmers have been investing more in inputs, especially fertilizers. Exposure to market information has increased but remains limited. Access to credit was still very limited at endline (and the conditions are not adapted to farmers). EQ 4 – Opportunities for male and female SHF: Buyers now consider cooperatives as business partners. Quality of maize has improved, and cooperatives have managed to aggregate bigger volumes of products. Cooperatives were not selling directly to schools at endline and the model still does not allow for this in the context of the WFP school feeding programme. However, the up-coming pro-SHF school feeding strategy should create new opportunities for cooperatives in the future. A school feeding strategy paper was under discussion at the time of the endline with the active participation of key ministries and WFP. EQ 5 – Income of smallholder farmers: At endline maize was contributing to both farmers’ income and food security. Production of maize has increased, there are less post-harvest losses and farmers have been able to increase their sales of maize. Profitability of maize cultivation is found to have increased but is still limited by the lack capacity to invest in inputs and by climatic hazards that regularly hamper production. Evaluation Report – LRP Rwanda Endline xi |P a g e EQ 6 – Quality/nutrition of the food marketed by producers: Farmers at endline were aware of the quality standards. Consumption of maize is increasingly important for households. With the improved quality and with the post-harvest equipment that was supplied by the project, farmers have been able to store their maize for longer periods. At endline SHFs were providing good quality and nutritious food that can be included in a school feeding program. EQ 7 – Participation of women: The LRP has had benefits for women in several areas. Overall, maize planting improved significantly for females during season A and season B, but not for men. Thus, women had increased marketable surplus and access to markets through a more efficient supply chain. The purchase of maize and beans for family consumption significantly reduced at the end line compared to baseline for females, but not for males. And women’s access to extension services increased. Equal number of male and female SHF were found to be members of cooperatives (although with variation between cooperatives). However, decision-making structures remained male￾dominated. Factors limiting the participation of female SHFs include time and capacity. EQ 8 – Factors affecting results: Various characteristics of the implementation methodology have positively affected results, including a strong and well-selected local partner (Rwanda Rural Rehabilitation Initiative (RWARRI)), good coordination between the RWARRI team and WFP, and frequent field officers’ visits to cooperatives. The monitoring system in place has not been very strong, and operational reporting has been weak. Externally the project has been adversely affected by climatic hazards. There has been a positive effect on the demand for maize and therefore on the price, and this has created a favourable context for LRP cooperatives to market their maize. Farmers that managed to have good production, despite the difficult year benefited from the situation. Conclusions: Conclusion 1: Project performance, relevance and effectiveness have been good. Out of the 16 monitoring indicators for the project, ten had been achieved at endline (or were very close to being achieved), four were partially achieved and two had not been achieved. Conclusion 2: The capacity of cooperatives has increased, although cooperatives remain fragile and overly project-dependent at endline. Conclusion 3: Farmers’ knowledge has increased as a result of the intervention and changes in the practices were evident. However, at endline the change is not complete and still fragile. Conclusion 4: LRP demonstrated that with appropriate support cooperatives can have access to better market opportunities. Conclusion 5: LRP has not yet had an effect at farmer level, but it contributes to an increased interest and profitability of maize production for males and females, with specific benefits for women. Evaluation Report – LRP Rwanda Endline xii |P a g e Conclusion 6: Capacity building is a long-term process and cooperatives have only received support for two seasons. Hence the change seen is very fragile and sustainability of the intervention and outcome is low as cooperatives are still very project dependant. Operational Recommendations Recommendation 1: In the coming six months, WFP should prepare a short learning document on LRP’s implementation and results for dissemination and organize an FTMA global evaluation in Rwanda. This document should bring out the specific benefits and continued constraints for female SHF, including with respect to their participation in cooperative activities. Recommendation 2: In the remaining time of the project, and for the next phase, WFP should work with RWARRI to prioritize cooperative capacity building activities, with a focus on governance, work organization, business management, and financial accountability. Recommendation 3: For the remaining time of the project, WFP should continue working with ICCO Terrafina to strengthen the access to the finance component of the project. Recommendation 4: In the next six months, WFP Rwanda should work with the WFP Regional Bureau and the Government of Rwanda, to develop a project proposal and raise funding for a second phase of the LRP. As part of this, WFP should conduct a study to assess the place and roles of women in the maize value chain to ensure that women are not being excluded now that maize is becoming profitable. Strategic Recommendations Recommendation 5: In the coming year, WFP should liaise with relevant stakeholders (Ministry of Agriculture and Animal Resources and development partners) to integrate a climate smart approach in the cropping model and to review the training curriculum and materials. Recommendation 6: In the coming three months, WFP should conduct a precise assessment of the different marketing options for small holder farmers. Recommendation 7: In the coming three years, WFP should continue to liaise with value chain actors and relevant ministries to continue to strengthen and further formalize the maize value chain dialogue platform. Recommendation 8: In the coming year, WFP should develop a market-oriented approach for the supply of post-harvest equipment to farmers, in coordination with interested companies and relevant ministries (commerce and agriculture). Recommendation 9: In the coming year, WFP should conduct a study to assess the diversity of crops grown by SHFs on cooperative land and on their own individual plots to identify potential opportunities for SHFs to supply a diversity of products for a school feeding programme. Evaluation Report – LRP Rwanda Endline xiii |P a g e Recommendation 10: Capacity and time constitute major constraints on the participation of women in cooperative decision making. Moving forward with its work in Rwanda, WFP should tailor its intervention strategies with women and coopartives to consider these constraints for women and seek to proactively address them. Evaluation Report – LRP Rwanda Endline 14 |P a g e 1. Introduction 1. This Endline Report was commissioned by the World Food Programme (WFP) Country Office (CO). Terms of reference can be found in Annex 1. It is the final product of an activity evaluation of the Local and Regional Food Aid Procurement (LRP) project in Rwanda which is funded by the United States Department of Agriculture (USDA). The LRP has been implemented by WFP in four districts (Huye, Gisagara, Nyaruguru and Nyamagabe). Main fields of intervention and activities are presented in Annex 2. 2. The purpose of the endline was to compare the situation of the LRP intervention at the time the project was ending (June 2019) with the starting point in September 2017. The other products of the activity evaluation are an Inception Report (October 2017) and a baseline report (February 2018). 3. The evaluation combined accountability and learning objectives: • Accountability: the evaluation reports on the performance of the LRP project, seeking to help WFP present credible and high-quality evidence of the effects of the project to USDA and other relevant stakeholders. • Learning objective: the evaluation provides insight into the reasons why certain results occurred or not, and brings out lessons learned, good practices, and priorities for learning. 4. At inception, the evaluation team reviewed the feasibility of an impact evaluation as requested in the terms of reference. It was concluded that such a design was not appropriate because of the short project implementation timeframe (two years) and that it was not realistic to expect behavioural and nutrition status changes in such a short period of time. The impact evaluation was thus re-framed as an activity evaluation focused on outcomes’ appraisal and the evaluation questions were refocussed closer to what could realistically be achieved in a 2-year time-frame. 5. In line with WFP and USDA requirements, this report provides information on the context and subject of the evaluation, the evaluation methodology, and the findings of the evaluation against each of the key questions and sub-questions. The expected users for this report are USDA and the WFP Rwanda Country Office (CO) who will use the report to inform decision making. The report is also expected to be of interest to the Government of Rwanda and other stakeholders who are working on cooperative and small holder farmer strengthening in Rwanda or similar settings. 1.1. Overview of the Evaluation Subject 6. USDA’s LRP was awarded in 2016 and implemented for a duration of two years from April 2017 to September 2019. The LRP project had a total budget of USD two million of which USD 1.36 million were assigned to the procurement of food from Small Holder Farmers (SHF). The objective of the program was to improve the effectiveness of food assistance and the expected outcomes included increased value of sales by project beneficiaries. The design assumed that food procured by WFP from farmer cooperatives or through pro- Evaluation Report – LRP Rwanda Endline 15 |P a g e smallholder aggregators would be distributed to WFP schools after fortification as part of the school feeding programme in two districts in the south of Rwanda (Nyamagabe and Nyaruguru).3 Of the LRP budget, USD 127,000 was to be used for capacity augmentation (mainly of SHFs and cooperatives) and USD 370,000 for direct support costs (i.e. costs directly related to management of the LRP project). 7. The main direct beneficiaries of project activities (capacity building and support) were SHFs, targeted through 16 cooperatives. These cooperatives were selected by WFP in four districts (Nyamagabe, Nyaruguru, Huye and Gisagara). Locations of cooperatives in the districts are presented in Annex 2. The total number of direct beneficiaries of the LRP over two years were 5,617 farmers (approximately 10 percent of farmers in the targeted areas), that had the opportunity to sell part of their production to formal FTMA (Farm to Market Alliance) buyers through cooperatives. 8. On the other end, the LRP project contributed to food purchases from Minimex4 to support the HGSF (Home Grown School Feeding) program. Considering potential synergies, it is likely that some of the Minimex maize was sourced from LRP cooperatives, and other FTMA buyers. However, the share of LRP direct beneficiaries supply to Minimex is unknown and likely limited, and the contribution of LRP farmers beneficiares to supply of the HGSF program is not possible to establish. 9. The focus of the project was on strengthening farmer cooperatives. Expected outcomes included improving access to financial services and markets and enhancing cooperatives capacities to enhance yield, improve quality of produce, and reduce commodity losses. Products marketed through cooperatives were partially purchased by WFP5 , leading to an increased use of locally purchased food for school meals. Through these interventions the intention was to improve food security among targeted beneficiaries. 10. An overview of the main project activities is found in Annex 2. The link between WFP/USDA inputs and the outputs, outcomes and impact of the LRP is explained in more detail in the LRP Theory of Change (ToC) which was drawn up at inception phase and is presented in Annex 3. The project was implemented according to the design of the project. 11. The main partners of LRP were: USDA, 16 cooperatives benefiting from training and inputs, the Rwanda Rural Rehabilitation Initiative (RWARRI) - a Non-Governmental Organization (NGO) which supported WFP in implementing the project - and various government ministries and departments which also provided support to cooperatives and had an oversight and learning role in the project. The key ministries included the Ministry of Agriculture and Animal Resources (MINAGRI), the Ministry of Local Government 3 Where WFP is implementing the McGovern-Dole International Food for Education and Child Nutrition program. Two other districts under the McGovern-Dole Program Home Grown School Feeding (HGSF) are serving meals that are prepared using Corn Soya Blend Plus (CSB+) porridge which is donated by USDA. 4 the only local producer of fortified maize meal that is able to comply with WFP quality standards. 5 using for this purpose the funds of the LRP Evaluation Report – LRP Rwanda Endline 16 |P a g e (MINALOC), the Ministry of Trade and Industry, the Ministry of Finance and Economic Planning (MINECOFIN), and the Ministry of Gender Promotion. The Ministry of Education (MINEDUC) is a partner as far as school feeding is concerned and was involved in the supervision of school feeding components. The provincial government monitored and coordinated support to cooperatives. 12. The design and implementation of LRP specifically included gender considerations. Participation of women in the cooperatives was one of the selection criteria for LRP cooperatives and for all the activities, interventions aimed to have a balanced number of beneficiaries. However, no specific gender activities have been included. 13. Several other interventions by WFP Rwanda are of relevance to this operation. WFP has been supporting SHF and the private sector in Rwanda since 2009. Earlier work through WFPs Purchase for Progress (P4P) programme and a follow-up pilot - the Patient Procurement Platform (PPP) - focused on increasing market access for SHFs. No specific evaluation of LRP programme had been conducted at endline. WFP also implements the Farm to Market Alliance (FTMA) initiative.6 FTMA has been of direct relevance to the LRP as it has provided access to markets, finance and post-handling services for approximately 180 cooperatives. The cooperatives committed to supplying mainly maize and WFP has acted as a neutral broker and facilitator bringing together potential buyers and cooperatives and facilitating access to services. 14. The LRP has covered five key project activities. Details on the activities are in Annex 2: • Purchasing commodities from small holder farmers (with a target of 418 metric tons (MT) of maize and 225 MT of beans) • Building capacities of SHFs • Connecting farmers to the PPP • Connecting SHFs to new markets • Collaborating with the Government of Rwanda (GoR) 1.2. Context 15. Rwanda, a small landlocked country, has an estimated total population of 12.3 million people, one of the highest population densities in Africa (445 people per km2 ), and an annual growth rate of 2.4 percent.7 The population is young and mostly rural, with urban growth rates outstripping those of rural areas. Since the 1994 genocide, Rwanda has focused on national unity and on improving quality of life. The country has seen strong economic growth and decreasing income inequality. 16. Rwanda’s long-term development goals are defined in a strategy entitled “Vision 2020”. It aims at transforming the country from a low-income agriculture-based economy to a knowledge-based, service-oriented economy with a middle-income country status by 6 FTMA is the new name for PPP. 7 http://www.statistics.gov.rw/publication/size-resident-population, accessed 22 July 2019 Evaluation Report – LRP Rwanda Endline 17 |P a g e 2020. To achieve this, the GoR formulated the second Economic Development and Poverty Reduction Strategy (EDPRS-2) which ended in 2018. In 2017, Rwanda launched its Vision 2050. Vision 2050 aspires to take Rwanda to high living standards by the middle of the 21st century. The implementation instrument for the remainder of Vision 2020 and the first years of Vision 2050 is the National Strategy for Transformation (NST) 2017-2024. This is the new name for the EDPRS-3. It focuses on three thematic areas including economic transformation, social transformation and transformational governance. 17. Rwanda is among African countries that have made strong progress on the achievement of the Millennium Development Goals (MDG). It has made considerable progress in poverty reduction with a dramatic drop in households living below the poverty line from 56.7 percent in 2005/2006 to 38.2 percent in 2016/2017; and extreme poverty reduced to just 16 percent in 2016/2017 from 24.1 percent in 2010/11.8 There have also been substantial improvements in living standards, with a two-thirds drop in child mortality, the attainment of near-universal primary school enrolment, and progress on reducing HIV prevalence and increasing environmental sustainability. Rwanda has experienced a strong drop in levels of hunger, with the country's Global Hunger Index (GHI) score reducing from 58.5 in 2000 to 28.7 in 2018, although it is still categorized a country with ‘serious’ hunger.9 Rwanda is hosting refugees from Democratic Republic of Congo and Burundi. At the end of 2018, there were 150,448 people living in in six refugee camps in Rwanda. 18. Agriculture plays a key role in the Rwandan economy, contributing 33 percent of the Gross Domestic Product (GDP). Agriculture is key to export earnings and represents over 80 percent of the total value of exports. Coffee and tea are the two main export crops and the most widely cultivated cash crops. Small-scale subsistence farmers produce most of the agricultural output and more than 80 percent of the population are smallholder farmers. The Government of Rwanda has made efforts to diversify the country’s exports by investing heavily in horticulture geared towards exports. The country produces several staple foods: maize, sorghum, rice, wheat, beans, soya beans, Irish potato, sweet potato, cassava and bananas. Ninety percent of agricultural production is food crops and 66 percent is consumed by producers. 19. Despite improvements in the agricultural sector, smallholder farmers and agricultural markets continue to face many challenges, including underdeveloped input markets and the continued use of basic tools and indigenous seeds, resulting in low yields and low incomes. Links to markets are also a challenge (GoR, 2016a). In the past years, climate change is having a noticeable impact on rainfall and impacting agricultural production (Dutch Sustainability Unit, 2015). 8 Fifth Integrated Household Living Conditions Survey 2016/2017, EIC5, Rwanda Poverty Report, National Institute of Statistics of Rwanda (NISR) 2018. 9 http://ghi.ifpri.org, accessed 22 June 2019. This information shows that Rwanda experienced a 50 percent drop in hunger between 2000 and 2018, although the more recent period saw a slight increase from 2016. Evaluation Report – LRP Rwanda Endline 18 |P a g e 20. The Strategic Plan for the Transformation of Agriculture in Rwanda (PSTA IV) is part of the NST. PSTA IV focuses on four priority areas: innovation and extension; productivity and resilience; inclusive markets and value addition; and enabling environment and responsive markets. The strategic plan prioritizes value chain development, product quality, and obtaining premium prices, as well as arrangements for bulking up production in order to improve access to inputs, services and markets. 21. Cooperatives are considered important vehicles to achieve the GoR's strategic plans. The number of cooperatives in the country has expanded very rapidly over the past years (GoR, 2011; USAID, 2013). At national level, more than 8,000 cooperatives are officially registered. In addition, there are also numerous informal organisations of producers. About 1,500 cooperatives were registered in the Southern province where the LRP has been implemented. There are different types of cooperatives and the way they function varies. Cooperatives also differ in their degree of success in promoting intensification, increasing market orientation and stimulating agricultural growth. 22. Rwanda has made significant progress in promoting gender equality, largely driven by strong Government commitment, and has the sixth highest score in the world on the 2018 Global Gender Index Gap Report10. Gender equality is enshrined in the constitution and Rwanda was the first country in the world to have more than 50 percent female members of Parliament (64 percent in the lower chamber). Nonetheless, challenges remain in terms of female representation in some areas, in the education and health sectors. 79.1% of females are engaged in agriculture activities (54.4% for males) and women receive more income from agriculture compared to other source of income. 11 Women and men farmers in dual households are characterized by unequal power relations, which leave women with very limited decision-making power. This affects their control over agricultural assets, inputs, produce and capacity building opportunities, leading to low agriculture productivity. Thus, women’s’ plots are typically less productive than those operated by men. Gender inequalities are persistent in selling of agriculture produces where males remain the responsible persons. Women also have less access to credit than males. 23. In addition to WFP several other actors were supporting SHF and the cooperative sector. These interventions are shown in Table 2. 10 http://reports.weforum.org/global-gender-gap-report-2018/data-explorer/#economy=RWA, accessed 15 August 2019. 11 Gender and agriculture, Gender office monitoring, GoR, March 2017 Evaluation Report – LRP Rwanda Endline 19 |P a g e Table 2: Actors working in agriculture and rural development in Rwanda (documentary review) Actor/organization Area of intervention The United Nations (UN) Food and Agriculture Organization (FAO) Agriculture, food security and nutrition, support to rural communities and linking farmers to markets The International Fund for Agriculture Development (IFAD) Support to MINAGRI in the implementation of the “Climate resilient Post￾Harvest Agribusiness support” project with a focus on aggregation of products, transformation and value addition The Alliance for a Green Revolution in Africa (AGRA) Member of FTMA World vision Input provision, capacity building, support to processing Korean International Cooperation Agency (KOIKA) Marshland development, erosion control, terrace building, support to seed multipliers Trocaire Resource rights (including land and water), women’s empowerment and humanitarian programme, value chain development, access to micro￾finance, climate resilience through water harvesting technologies, citizens’ participation and capacity building Interchurch Organization for Development Cooperation (ICCO)Terrafina Agricultural microfinance and support to inclusive entrepreneurial initiatives Caritas Promoting economic and agricultural development, reinforcing coherence between agriculture, economic activities and social protection Duhamic-Adri Support to maize and bean production, saving and credit groups, entrepreneurship United States Agency for International Development (USAID) Funding for the Future Rwanda Hinga Weze Project aiming at sustainably increasing smallholder farmer income, improving the nutritional status of women and children, and increasing the resilience of Rwanda’s agricultural and food systems to climate change 1.3. Evaluation Methodology and Limitations 24. The agreed upon evaluation questions are outlined in the Table 3 below. The questions cover the dimensions of relevance, effectiveness, and efficiency. As agreed with WFP at inception, the evaluation covered the LRP cooperative activities and outcomes. Data on the coverage of LRP purchased for schools (number of pupils benefitted) have been included in this evaluation report but the purchasing of food for primary schools was not directly evaluated.12 12 The food purchased and delivered by WFP through LRP is part of the Home Grown School Feeding Programme and will be evaluated as such in future. Evaluation Report – LRP Rwanda Endline 20 |P a g e Table 3: Agreed evaluation questions and their coverage at baseline and end-line # Evaluation question Evaluation criterion Data collected at 1 To what extent was the design of the intervention relevant to wider context; aligned with needs of the most vulnerable groups; cognizant of the needs of male and female beneficiaries; and in line with priorities of the government and WFP partners? Relevance Baseline and end-line 2 What are the effects of the project on the cooperatives ability to be, to organize, to relate, and to do?13 Effectiveness Base & end￾line 3 How does the LRP programme affect male and female SHF capacity and behaviour? Effectiveness Base & end￾line 4 Has the LRP contributed to creating new opportunities for male and female Small Holder Farmers (SHF)? Effectiveness Baseline and end-line 5 What is the level of participation of men and women? Are women well represented, including in leadership positions? What are the disaggregated effects on women? Has the LRP programme affected male and female SHF differently? Effectiveness Baseline (partially) and end-line 6 What internal and/or external factors affected the project’s achievement of intended results? Effectiveness & efficiency End-line 7 How and to what extent does the LRP programme contribute to producers marketing food products that meets quality standards and is nutritious, and is culturally acceptable? Effectiveness End-line 8 How does the LRP programme affect male and female SHF income? Effectiveness Baseline and end-line 9 Is there emerging evidence that the cooperatives have capacity to create linkages with schools? Effectiveness Baseline and end-line 25. As agreed during the inception of the study the evaluation design used a mixed method￾before and after approach to examining the effects of the intervention. At baseline (T0), quantitative and qualitative data was collected using a combination of documentation/secondary data review, a survey, in-depth key informants’ interviews, and observations. At end-line (T1) the same information was collected for comparison. At each stage the survey covered all targeted cooperatives and covered a sample of SHFs. 14 SHFs who participated in the survey were randomly sampled within each of the participating cooperatives. The sample size per cooperative was calculated according to the membership size and considering gender proportions. The survey was developed at baseline and tested prior to use. The same survey was applied at endline with minor modifications to improve wording and remove questions that were found to be redundant. 13 We used and adapted different tools to measure cooperatives capacity: the ability methods (South Research), the governance matrix (IRAM) if relevant, the 3 circles models (SNV), the level of development of producers organization (IRAM), the PO trajectory tool (IRAM) as well as some scoring tools developed based on these tools (reference: “Sélection de références méthodologiques en analyse organisationnelle”, IRAM, 2009) 14 At baseline the data collection also included schools. This was deprioritized at the endline and so no survey was applied to schools at endline. Evaluation Report – LRP Rwanda Endline 21 |P a g e 26. Focus Group Discussions (FGD) were organised in six cooperatives. The choice of cooperatives was based on the initial findings from the quantitative data collection, with the objective of having a diversity of situations in terms of dynamics, geographical coverage and size of cooperatives. The choice of FGD participants was made by cooperative executives (with even participation of females and males). In-depth key informant interviews (KII) were conducted with representatives from cooperatives, value chain actors (producers, middlemen, buyers, agro-dealers), support actors (Microfinance Institutions (MFI), local NGOs, extension agents), schools and staff in schools, as well as national and local authorities and decentralized units from the relevant ministries (a list of person met is in annex 5). The KII were selected based on the stakeholder analysis and sought to coincide to the extent possible with persons interviewed at baseline. The interviews themselves were guided by a semi-structured topic list (annex 13). Documents and secondary data were analysed for relevant information (the bibliography is annex 14). All informants were guaranteed anonymity and provided with the option of terminating the interview at any time. 27. To ensure that the quantitative data was meaningfully collated with qualitative data collected for this study, a preliminary descriptive analysis of the quantitative data was carried out during the data collection phase, and the results were used to develop questions for qualitative data collection. This ensured that the qualitative data collection provided a deeper understanding and explanation of the trends observed from the quantitative data analysis. 28. The before and after design was combined with a contribution analysis approach which seeks to identify whether the changes can be attributed to LRP or other interventions. The ToC highlights the specific expected contribution by LRP. While the quantitative survey was used to document and measure changes, qualitative data and key informant interviews were conducted to seek and attribute reasons for such changes. During interviews beneficiaries and other stakeholders were asked about the specificity of LRP compared to other interventions and this was used to understand in what way the LRP specifically has contributed to the observed changes. The ToC assumptions were found to be plausible and agreed upon by the main stakeholders, activities have been implemented as set out in the ToC and key results from the ToC are confirmed. Other influencing factors have been explained, analysed and taken into consideration. 29. Table 3 above recalls the main evaluation questions and criteria used for the endline study. The data collection instruments drew on the evaluation ToC and the evaluation matrix (see annex 4). Triangulation of information was sought, both within sources (i.e. by comparing perspectives of stakeholders) and across different methods of data collection (between the survey and interview findings). In addition, it should be noted that the use of multiple qualitative and quantitative data sources, as well as contribution analysis, aids triangulation and therefore validates findings. Evaluation Report – LRP Rwanda Endline 22 |P a g e 30. All participants in the evaluation were provided with an explanation of the purpose of the study and with the option to withdraw at any time from the survey and interviews should they wish to do so. Individual responses were kept confidential and coded so that they could not be related to specific respondents. 1.4. Overview of the characteristics of survey respondents 31. The target sample size for the survey across the four districts was 782. The actual sample size during the baseline survey was 841 (108 percent), comparable to the final evaluation survey with 828 (106 percent) respondents. 32. Distribution of most of the background characteristics of the respondents was comparable between baseline and final (Table 4). This is a desired result demonstrating stability of the sampled population, implying reduced bias in measuring change in the specific indicators. 33. As can be seen from Table 4, nearly 68 percent of the respondents are heads of households. Twenty two percent of households were headed by females15 and in 16 percent of the households there are no males providing labour. Around 40 percent of the farmers interviewed did not receive any formal education and 54 percent only attended primary school (at national level, according to statistics, 35.7 percent of farmers do not have any education and 56.3 percent have primary education in rural areas). A total of 12 cooperatives were covered by the qualitative (interview) work, and all 16 participated in the quantitative survey. Table 4 : Background characteristics of the small holder farmers (SHF) Variables Female Male Total Baseline (n=387) Final (n=411) p value Baseline (n=454) Final (n=417) p value Baseline (n=841) Final (n=828) p value Respondent also the household head 36.4% 39.7% 0.348 92.1% 95.7% 0.027 66.5% 67.9% 0.541 Gender of the household head Female 43.4% 42.8% 0.867 2.4% 2.6% 0.840 21.3% 22.6% 0.521 Male 56.6% 57.2% 97.6% 97.4% 78.7% 77.4% Highest educational level of the household head No formal education 33.6% 40.9% 0.001 22.2% 26.4% <0.001 27.5% 33.6% <0.001 Primary 65.4% 54.3% 74.7% 65.5% 70.4% 59.9% Secondary 1.0% 3.9% 1.8% 7.9% 1.4% 5.9% Tertiary 0.0% 1.0% 1.3% 0.2% 0.7% 0.6% Marital status of the household head Married 53.7% 61.1% 0.036 82.6% 91.4% <0.001 69.3% 76.3% 0.001 Not married 46.3% 38.9% 17.4% 8.6% 30.7% 23.7% Total number of males in your household None 8.5% 7.3% 0.789 0.0% 0.7% 0.327 3.9% 4.0% 0.841 1 to 2 51.7% 53.8% 48.7% 49.2% 50.1% 51.4% 15 At national level, 29% of the households are headed by women, and 33% in the Southern province according to the National Statistics Institute or Rwanda Evaluation Report – LRP Rwanda Endline 23 |P a g e 3 to 4 32.6% 30.7% 41.2% 39.6% 37.2% 35.1% More than 4 7.2% 8.3% 10.1% 10.6% 8.8% 9.4% Total number of females in your household None 0.0% 1.5% 0.039 2.0% 1.7% 0.723 1.1% 1.6% 0.543 1 to 2 52.5% 46.0% 46.3% 47.0% 49.1% 46.5% 3 to 4 40.1% 44.5% 42.3% 39.8% 41.3% 42.1% More than 4 7.5% 8.0% 9.5% 11.5% 8.6% 9.8% Total number of males in your household who normally provide labour for household chores including farming None 29.2% 29.7% 0.768 2.0% 2.6% 0.934 14.5% 16.1% 0.718 1 to 2 66.4% 66.9% 93.4% 92.8% 81.0% 80.0% 3 to 4 4.1% 2.9% 4.2% 4.1% 4.2% 3.5% More than 4 0.3% 0.5% 0.4% 0.5% 0.4% 0.5% Total number of females in your household who normally provide labour for household chores including farming None 1.8% 2.7% 0.244 5.5% 5.0% 0.522 3.8% 3.9% 0.256 1 to 2 91.7% 93.7% 90.3% 92.1% 91.0% 92.9% 3 to 4 5.7% 3.4% 4.0% 2.4% 4.8% 2.9% More than 4 0.8% 0.2% 0.2% 0.5% 0.5% 0.4% 1.5. Ethics and gender 34. WFP's decentralised evaluations must conform to WFP and United Nations Evaluation Group (UNEG) ethical standards and norms. The contractors undertaking the evaluation are responsible for safeguarding and ensuring ethics at all stages of the evaluation cycle. This includes, but is not limited to, ensuring informed consent, protecting privacy, confidentiality and anonymity of participants, ensuring cultural sensitivity, respecting the autonomy of participants, ensuring fair recruitment of participants (including women and socially excluded groups) and ensuring that the evaluation results in no harm to participants or their communities. 35. Gender was considered in all the steps of data collection and in the following manner: • An equal number of male and female respondents were included in the quantitative survey. • All FGD included women. While no gender specific FGD were organized but a specific attention was given to the participation of women in the discussion. • Across the research questions the team sought to establish whether there were significant differences in the way women and men had participated and or benefitted from the interventions. • Some specific questions were included in the KII and FGDs about place and role of women. • The qualitative information collected was analysed to triangulate the ideas and words of the male and female actors with elements of context and emphasized the main issues affecting the actors. Where relevant, the different voices of male and female farmers have been reflected in the discussion of the findings. Evaluation Report – LRP Rwanda Endline 24 |P a g e • Data is disaggregated by gender in the report, whenever there is a difference between males and females. When not disaggregated, the results were similar for males and females. 1.6. Limitations 36. The evaluation faced the following limitations: • For operational reasons, LRP progress reporting has (mostly) been part of FTMA reports. This has limited the amount of detail on LRP specific information and reduced the extent to which reports could provide information of use to this evaluation. The evaluation team addressed this challenge by triangulating information from non-documentary sources (in particular through KII, FGD and the survey). • The evaluation team was unable to access all the relevant documents from the LRP project and from partners. This limited the analysis of project implementation and efficiency. Analysis has therefore focused primarily on results at farmer and cooperative level. • At endline, some cooperative leaders and members who had been interviewed at baseline were not available, and the data collection schedule did not allow to reschedule interviews. The evaluation team sought to fill this gap by interviewing alternative cooperative leaders and members. At data analysis stage the team checked whether the responses of these cooperatives were broadly aligned with those of the respondents who were interviewed both at base and endline. • Documentation at cooperative level was found to be very limited. This made it difficult to confirm the veracity of verbal information from cooperatives and SHF, for example in relation to quantity and pricing. To the extent feasible, the evaluation team triangulated information through additional interviews. • FGD participants were chosen by cooperative leaders which could bias representativeness of cooperative members. Yet, the research team sought to address this by asking that the SHF selection for the FGD should reflect the diversity of the full group of members (in terms of gender and age), by checking that there was a diversity of participants, and by making sure that all participants had a chance to express themselves. Women contributed freely to the discussions and specific questions on the roles and involvement of women were asked. • Finally, an overarching limiting factor is the short timeframe between the baseline and the endline, as processes that require changes in practice and behaviour by different intervening actors generally require at least three seasons to become established. The evaluation report therefore provides a sense of direction of changes, but a longer time period would be needed to achieve outcomes and impacts envisioned by this intervention and to be able to assess whether these have taken place. Evaluation Report – LRP Rwanda Endline 25 |P a g e 2. Evaluation Findings 37. The following section of the report presents the evaluation findings by evaluation question. 2.1. Evaluation Question 1 - How relevant is the design of the intervention in terms of the context; needs of the most vulnerable groups; needs of male and female beneficiaries; and priorities of the government and WFP partners? Relevance in terms of beneficiary needs and priorities 38. Agriculture is the backbone of the economy of Rwanda and it employs most of the population. According to the latest Comprehensive Food Security and Vulnerability Analysis (CFSVA) (2018), about 50 percent of all Rwandan households experience at least occasional difficulties with access to food and about 20 percent of the population is considered food insecure. Thirty-nine percent of the population is poor, and the southern province is the second poorest province. Food insecurity remains high in the targeted LRP districts. About 30 percent of the population is food secure in Huye, Nyaruguru and Gisagara districts. In Nyamagabe, only 11 percent of the population is food secure (CSFVA 2015). Improving food crop production is therefore extremely relevant to the needs of beneficiaries in the targeted districts. Agriculture in Rwanda faces low productivity and high post-harvest losses. Therefore, building capacities of farmers on Good Agricultural Practices (GAP) and Post-Harvest Handling and Storage (PHHS) were very relevant activities of the LRP project. 39. Working with cooperatives was also a very relevant entry point and strategic focus. In the Rwandese context, cooperatives are key actors in the development of marshlands that belong to the GoR. Cooperatives provide access to consolidated land, and therefore allow for work on one large area with the same crops, which facilitates access to inputs, organisation of the work in the fields, and access to extension services. Cooperatives also enable farmers to exchange on their practices, thus facilitating dissemination of good agricultural practices. Through cooperatives, it is possible to aggregate larger volumes of crops, which enables linkage with formal buyers.16 40. Supporting the development of contracts between farmers and buyers was equally very relevant to increase production, quality and access to markets (key objectives of the LRP). Without formal buyers, farmers are often forced to sell at a much lower price to middlemen. Lower prices dissuade them from investing in quality. With contracts, marketing of the product is guaranteed, and this encourages SHF to increase production and quality. 16 usually large buyers are not interested in purchasing quantities that are less than five to ten metric tons. Evaluation Report – LRP Rwanda Endline 26 |P a g e 41. Gender was not specifically taken into account in the design of the project. The project did not analyse and target specific constraints affecting women (such as lower literacy rate, lower self-confidence, heavy workload). The project also did not particularly encourage participation of women in cooperative decisions-making and governance. There is no evidence either that division of labour and profit between men and women in the maize value chain was analysed to make sure that there was no risk of eviction of women from some activities and/or control over some resources. Nonetheless, the LRP paid attention to the participation of women to the activities. 42. LRP’s design also did not include specific actions to target the most vulnerable households. Instead the choice of targeted cooperatives was made with district authorities based on characteristics that reflected good agricultural potential. Given that the focus of the project was on value chain development, this was a reasonable choice. Investing in production to increase the marketable surplus requires targeting farmers that can afford to take this risk. Relevance to the priorities of the government 43. As the table below show, areas of intervention of LRP have been well aligned with priorities of the government at national level. Focal points from MINAGRI and MINEDUC have been regularly consulted and participated in several LRP project activities. Table 5 : Relevance of LRP interventions to the priorities of the GoR Policy/strategy Key focus of the policies and strategies National Strategy for transformation (2017-2024) Modernize and increase productivity of agriculture Strengthen commercialisation of crops by increasing private sector engagement Promote market-oriented agri-business Increase surface of consolidated land Enhance farmers’ access to improved seeds Increase average productivity of crops Increase post-harvest and storage capacity Facilitate access to finance for farmers Economic Development and Poverty Reduction Strategy 2 (2013- 2018) Increasing economic growth by investing in and modernizing agriculture. Increasing agriculture productivity Connect rural communities to economic opportunity Vision 2020 Promotion of intensification in agriculture Production of high value crops Replace subsistence farming by a fully monetised, commercial agriculture sector Integration of gender into development policies and strategies Vision 2050 Sustained food security and nutrition for all households Technology intensive agriculture with a commercial focus Universal access to financial services PSTA IV Having a productive, green and market led agriculture sector 4 pillars: • Enabling environment and responsive institutions Evaluation Report – LRP Rwanda Endline 27 |P a g e Policy/strategy Key focus of the policies and strategies • Productive and inclusive markets and value addition • Increased productivity, diversity, sustainability and resilience of agricultural production • Research, innovation and empowerment Crop Intensification Program Increase agricultural productivity. Provide access to inputs (improved seeds and fertilizers), consolidation of land use, provision of extension services, and improvement of post-harvest handling and storage mechanisms six priority crops: maize, wheat, rice, Irish potato, beans and cassava. National policy on cooperatives (2018) Strengthening of capacity development and entrepreneurship promotion for cooperatives Improve policy dialogue to enhance cooperative contribution to national development Increase ownership of the cooperative movement by cooperatives members Promote self-financing Promote performance-based management National Post￾Harvest Staple Crop Strategy Strengthening the harvesting, post-harvest handling, trade, storage, and marketing of staple crop value chains Strengthening food security strengthening competitiveness of the staple crop value Improve efficiency and decrease marketing costs Enhance producers’ access to, and linkages with, markets. National Gender Policy Promote gender equality and equity Mainstream gender needs and concerns across all sectors of development The Rwanda Education Sector Strategic Plan Strengthen school nutrition Expansion of the HGSF Programme National School Health Policy Improve and expand food and nutrition, as well as school performance in schools (pre-primary, primary and secondary 44. At district level, LRP interventions were very relevant to district priorities because of the focus on land use consolidation, increased use of inputs, private sector involvement, capacity strengthening of farmers, increased post-harvest facilities and increasing productivity and production, all of which have been key areas of concern for these districts. Districts authorities have regularly been consulted by LRP project and reports were regularly submitted to the Joint Action Development Forum in order to ensure coherence with other interventions. Relevance for WFP 45. WFP has a long history of supporting cooperatives to increase production and capacity and has typically done so with the objective of increasing access to local produce for WFP’s programmes. The LRP project was also relevant to WFP other priorities as it targets school feeding. Coherence with other interventions Evaluation Report – LRP Rwanda Endline 28 |P a g e 46. For district authorities the coherence and alignment of the LRP project with other interventions has been very good. According to them, the LRP brought a specific focus on the inclusion of the private sector partnership in the project. Districts considered this a relevant approach which has been well aligned with their development priorities. 47. At national level, a dialogue mechanism is in place for development partners and the GoR. Sector Working Groups are a forum through which stakeholders meet to discuss sector planning and prioritization and WFP is part of the Working Group on Agriculture. Most of the development partners work on supporting agricultural production and target cooperatives. However, the specificity of the LRP is the linkage to buyers and the support to post-harvest and handling storage. Key findings and conclusions – Question 1 • The LRP focus on agriculture and promoting staple crops were relevant choices. Rwanda’s staple food sector is growing substantially, and the country still faces serious food insecurity issues. • Supporting cooperatives to improve their production and marketing capacities through forward delivery contracts has been particularly relevant. • The LRP has a specific added value in linking SHF to buyers/the private sector and is seen as complementary to the work of other partners in doing so. • The LRP did not specifically target the most vulnerable households. Inclusion of poorest farmers in cooperatives has not been a point of attention for cooperatives and local authorities. • The LRP project design was in line with the GoR strategies and policies and well aligned with the priorities of the districts that were covered by the LRP. • WFP has engaged in regular dialogue with MINAGRI and MINEDUC, through existing dialogue mechanisms at national level between development partners and the Government of Rwanda. 2.2. Evaluation Question 2 - How has cooperative ability ‘to be’, ‘to organize’, ‘to relate’ and ‘to do’ evolved from baseline to endline? 48. General information on LRP cooperatives is presented in annex 7.1. The endline used the same four criteria used for the baseline study (ability ‘to be’, ‘to organize’, ‘to relate’ and ‘to do’) to assess the evolution of cooperative capacity. A description of these criteria and of the scoring exercise can be found in Annex 7.2. 49. Capacity building of cooperatives has been a key element of LRP’s intervention. The project mainly focused on technical capacities (Good Agriculture Practices (GAP) and Post￾Harvest Handling and Storage (PHHS)) but some activities also targeted cooperatives governance, operational and financial management, as well as access to finance (more information on the different activities is in annex 7.3). All the LRP cooperatives received training on GAP, PHHS, and governance. Seven cooperatives have been trained on access to finance through saving groups. Evaluation Report – LRP Rwanda Endline 29 |P a g e 50. Overall, at endline, the evaluation found that capacities of cooperatives have increased, although most of them are still fragile. A visual representation of the scoring at baseline is presented in annex 7.2. The visual representation at endline is presented in Figure 1. The data suggest that at endline, cooperative capacity remains largely variant from one cooperative to another. For example, cooperative scoring on ‘capacity to perform’ varies from 32 to 82 percent. Figure 1: Representation of cooperative abilities (scoring based on qualitative endline data) 10 20 30 40 50 60 70 80 ability to be ability to do ability to relate ability to perform group 1 : high potential cooperatives Icyerekezo Coamanya Gishubi CCM Muganza 10 20 30 40 50 60 70 80 ability to be ability to do ability to relate ability to perform group2 : Not well functionning cooperatives Kopianya Duterimbere Abemeriihigo Abishyizhzehamwe Urwonhjya Abishyizhzehamwe Ngera Evaluation Report – LRP Rwanda Endline 30 |P a g e 51. Cooperatives identified at baseline as ‘high potential cooperatives’ were the best performing cooperatives at endline (group 1). They have fully benefited from the opportunities and training provided by the LRP project. Some of the selected cooperatives appear as particularly weak (group 3). Some of these were already losing momentum at baseline and others were rated as nascent but collapsed (due to governance and management problems). The last group of cooperatives are those not yet functioning well (group 2). The opportunities provided by the LRP project have contributed to boost these cooperatives, but they are not yet able to be sustainable. WFP is currently in the process of transitioning the LRP cooperatives to the McGovern Dole programme to ensure continuity of support moving forward. Ability to be 52. Cooperatives’ governance and common vision had not changed much at endline. In the Rwandese context, cooperatives are mainly farmers’ organisations promoted by the government to organize production and access to extension services and subsidized inputs. All cooperatives interviewed are organizing their statutory meetings in line with the Rwanda Cooperative Authority (RCA) guideline: holding a General Assembly twice a year and an executive committee meeting every month. 53. The evaluation found that the LRP interventions brought a new dynamic to cooperatives, through training and running activities of the project. This resulted in more operational meetings as members now have activities to organize and decisions to take. However, interviews at endline highlighted that LRP cooperatives still had no clearly defined common vision and no strategies (as figure 20 in annex 7.4 shows). 10 20 30 40 50 60 70 80 ability to be ability to do ability to relate ability to perform group 3 : weak cooperatives Abibumbye Kaiimu Kobabomango Kopiki Evaluation Report – LRP Rwanda Endline 31 |P a g e 54. The LRP contributed to increased farmer interest in cooperative membership and participation. At endline seven (out of twelve) cooperatives considered themselves as more active than two years earlier. Main changes mentioned included increased member participation (seven cooperatives), more trust between members (four cooperatives), and interest from other farmers in joining the cooperative (three cooperatives). From the data collected there appeared to be a stronger sense of ownership from farmers. Figure 2 below suggests that this interest has been sparked by LRP project, and in particular through the focus on market linkages and the organization of various training activities. Figure 2: Farmers reasons for being interested in cooperative activity (qualitative data collection based on 12 cooperatives, multiple option choice) 55. All cooperatives had leaders in place who have been regularly renewed. However, there is a distant relationship between leaders and members, which didn’t improve at endline. Leaders are seen as cooperatives’ authority but not necessarily prioritising the defences of cooperatives members’ interests. Moreover, transparency and accountability towards members is seen as low. Training on management and governance has targeted leaders but cooperatives’ members have not been involved and do not have the capacity to monitor and control the committee. This created problems in several cooperatives of which the committee was renewed, and the skills and knowledge acquired though the capacity building activities were lost with the turn-over. Several cooperatives also experienced problems of committee members using funds of the institution for their own benefit. In the case of Kopiki, this almost resulted in the collapse of the cooperative. Ability to organize 56. Interviews at endline highlighted that document archiving was still a challenge for most cooperatives. Cooperatives lack understanding on the importance and use of documents. As table 8 in annex 7.4 shows, two cooperatives had a good archiving system, five had a 0 1 2 3 4 5 6 7 8 9 access to training access to market access to input no interest increased production working together, organizing production access to equipment Evaluation Report – LRP Rwanda Endline 32 |P a g e system with considerable shortcomings and four had poor systems. None of the cooperatives mentioned an improvement in their account-keeping and management system in the last two years (see figure 21 in annex 7.3). Archiving and accounting was included in the training package but consisted in a one-off training. Nine cooperatives mentioned that they would need more training and regular coaching to be able to improve document archiving and account-keeping (see table 7in annex 7.4). 57. The evaluation found that LRP cooperatives have developed a stronger business mindset. Only two cooperatives had no business mindset and conducted limited planning of their activities (Abibumbye and Kopiki). The other nine cooperatives had a relatively strong business mindset at endline- they were producing for the market and were able to discuss with buyers and negotiate prices and conditions. Five cooperatives (Coamanya Gishubi, CCM Muganza, Icyerekezo, Duterimbere and Abemerhigo) were one step further, in other words, they considered their activity as a business and were looking for opportunities to develop their activity. 58. Yet, none of the cooperative had a real business plan or an expense forecast of expected benefits at the end of the season. Cooperatives do not analyse their financial results and therefore have only a very limited idea of the profitability of their activities. Cooperatives were still making opportunistic and externally-driven choices instead of balancing options and taking decisions based on potential profit and assessment of the opportunities they have. 59. Funding of cooperatives continued to be an area of fragility. Lack of funds continues to be a major obstacle for cooperatives to develop their activities. All cooperatives interviewed considered that their capital is currently not enough to cover expenses related to the running of their activities and to invest (for example in accounting staff). However, the situation has improved from baseline to endline. The quantitative survey showed that 100 percent of the SHFs were paying dues to their cooperative (compared to 64.4 percent at baseline). This improvement in resource mobilization was also evident from the qualitative data collected. Shares have been paid (at least partially) in all the cooperatives (see figure 3) and most cooperatives have a regular resource mobilisation system (see figure 4). At baseline, cooperatives did not have a clear view of the level of payment from their members and evidence showed that many SHFs did not meet their obligations. Aggregating production and organizing PHHS collectively contributed to better resource mobilisation as cooperatives usually retrieve the amount of dues/commission of sales before paying their members. Figure 3: Level of payment of shares in cooperatives (qualitative data collection) Evaluation Report – LRP Rwanda Endline 33 |P a g e 60. All cooperatives had a bank account. Cooperatives interviewed demonstrated better knowledge of the amount of funds in their accounts. By the time of the endline, amounts varied between 54,000 Rwanda Francs (RWF) and 2.1 million RWF depending on whether cooperatives had already sold crops in season B and/or invested in agricultural activities (see figure 22 in annex 7.4). The cooperative with the lowest amount of funds is a well￾performing cooperative which was still waiting to be paid for its season A production17 at the time of data collection. 61. Money collected by cooperatives has been mainly used to purchase inputs, pay for health insurance, and to invest (in infrastructure, equipment, or activities). It has also been used to pay for the running costs of the cooperatives (see figure 22 in annex 7.4). 62. Credit has not been seen by cooperatives as a means to increase resources for agriculture activities. Three cooperatives contracted credit (but only one during the LRP project and through a FTMA member). As figure 23 in annex 7.4 shows, cooperatives considered that credit is risky because of the climate hazards and because there have often been delays before receiving the funds18 . Other factors that have affected access to credit are the high 17 This cooperative (CCM Muganza) produced seeds for the Rwanda Agriculture Board (RAB) and was still awaiting payment six months after delivery. 18 Kaiimu received credit through FTM but the funds arrived too late for planting. SHFs had to use their own savings in order to reimburse the loan. Figure 4 Figure 4: Cooperative resource mobilisation systems (qualitative data collection) Evaluation Report – LRP Rwanda Endline 34 |P a g e interest rates, the costs of credit, and the requirement to provide collateral. However, eight cooperatives considered that they could access credit easily if they decided to do so and could usually choose between several banks, Micro Finance Institutions (MFIs) and Savings and Credit Cooperatives (SACCO). A change in cooperative mentality about credit was clearly noted from baseline to endline, and several cooperatives mentioned that they are considering contracting credit in the future, provided they can first strengthen their cooperative and have a clear and profitable project to finance. Ability to relate 63. At endline cooperatives mostly had the same partnerships as baseline (see annex 7.4). The LRP has clearly been instrumental in linking cooperatives to FTMA buyers (a brief presentation of FTMA buyers is in annex 9.2). This is a major achievement of the project. For cooperatives, the partnership with buyers represented a key change in their situation from baseline to endline. All cooperatives now knew FTMA buyers (at baseline cooperatives had very limited knowledge of big buyers and only one of them had sold maize through contract). Most cooperatives were satisfied with the partnerships, with the exception of three cooperatives that experienced some challenges with buyers.19 Cooperatives considered that working with FTMA buyers ensured more transparency in terms of conditions and fairer transactions (no cheating on weight). 64. However, the evaluation view is that the partnership with buyers remains very fragile as cooperatives only sold once or twice to FTMA buyers. Cooperatives were not yet strong enough to conduct negotiation on sales without the intervention of the LRP project. Every time that difficulties have arisen, intervention by the LRP project was needed to find solutions. Cooperatives also remained very dependent on LRP project staff to find buyers; in several cases RWARRI’s field officer had to look for marketing options for maize that was not sold to FTMA buyers. 65. Cooperatives’ knowledge and ownership of contracts was still low. Only one cooperative was able to present the contract to the evaluation team. Some of the cooperatives had not received a copy of the contract (even for contracts that relate to season A in 2018). Other cooperatives mentioned receiving the contract but not being able to locate it. Cooperatives were able to explain the conditions of the contract but did not understand the detailed contract conditions. In particular, they did not know what to do when the buyer does not respect its part of the contract. 66. Choice of buyers should be done freely by cooperatives depending on the conditions offered. To a certain extent this was found to be the case, but some cooperatives mentioned that they were only introduced to one buyer. Actually, all the buyers were invited but some of them are still reluctant to work with LRP cooperatives and did not 19 The challenges in each case were different and included: rejection of their produce; direct purchase of maize, and delays in payment Evaluation Report – LRP Rwanda Endline 35 |P a g e participate. Also, interviews with buyers highlighted collusion among buyers within catchment area. As a matter of fact, nearly all sales made by LRP cooperatives have been purchased by the same buyer. This represents a risk for the future, should this buyer decide to take advantage of the situation and offer non-competitive conditions. WFP is now looking for localized formal buyers willing to sign contracts with cooperatives and that should create more opportunities for farmers. Ability to perform 67. Agriculture is the main economic activity for cooperatives. Cooperatives that are also agro￾dealers do so to support production rather than to make a profit. The main crop that is produced is maize (all the cooperatives were growing maize and four of them had been growing it more than once a year). The second most important cooperative crop is beans (nine cooperatives are growing beans). Other crops grown on cooperative lands were Irish potatoes, vegetables and rice (see figure 25 in Annex 7.4). 68. The main cropping season is season A. For most of the cooperatives, season A is maize season to make sure they have a good production (as they consider it as their most profitable crop). However, their main strategy has been to minimize risks so if the season is delayed (by lack of rain or if seeds arrive late) cooperatives will often switch to beans. Choice of crops has also been driven by recommendations from the agronomists, characteristics of the marshland, market and price (price of maize is higher in season B as there is a smaller quantity of maize on the market). Production on cooperative land is still mainly used to feed the family even if cooperative lands are considered as commercial plots. Maize is seen as a cash crop although only about a third of the production is sold. At the time that the evaluation data collection took place beans had not been sold by cooperatives20 . 69. Cooperatives are not yet at their full potential for maize production. On average, cooperatives grow maize on 60 percent of their marshland, with significant variation from cooperative to cooperative (with cooperatives growing between 10 to 100 percent of their marshland with maize). Six cooperatives have fully dedicated their land to maize but five of them grow less than 30 percent of their marshland with maize. The main reason is soil fertility issues on some marshlands as maize is a demanding crop and parts of the marshland are not appropriate for maize. Some marshlands are not yet fully constructed and experience frequent flooding and/or droughts. Some farmers are members of the cooperative but not really interested in its activity and strategy. Therefore, they do not grow maize but grow their own choice of crop. 70. It has been difficult to assess how cooperatives’ maize production has evolved because cooperatives only keep records of sales and not of yields, and the LRP project did not 20 Data collection for the final evaluation took place in June 2019. WFP organized a campaign to purchase beans from LRP cooperatives in July 2019. The evidence related to this campaign could therefore not be included in this report. Evaluation Report – LRP Rwanda Endline 36 |P a g e monitor yield. However, all cooperatives interviewed reported that their production had increased. Information collected from cooperative levels suggests that yields have been between 1.5 MT/ha and 3.5MT/ha21 (with a majority of farmers mentioning between 2 and 3 MT/ha). As figure 26 in Annex 7.4 shows, the production can therefore be estimated at between 796 and 1,195.5 MT for 15 cooperatives (there was no information on the area planted for Abibumbye cooperative). At baseline, the potential production - based on the area planted and high yield expectation - was estimated at 841,3 MT. It appears, therefore, that most cooperatives have improved production. There is likely still room for improvement given that the production potential is around 2500 MT, would the marshland be fully utilized for maize and yields keep improving. 71. Ten (out of 12) cooperatives mentioned that their sales had improved with the support of the LRP project. At the time of the endline, sales data was not available for all cooperatives (not everything had been sold). However, the evolution of the registered sales, showed a clear increase (see figure 27 in Annex 7.4). For example, Coamanya Gishubi was selling 60 MT in 2017 and was planning to sell 200 MT in 2019. In 2018, RWARRI registered 452.1522 MT sold to FTMA buyers through the LRP project, as figure 27 in Annex 7.4 shows). This represents a clear increase from the baseline (when the quantity sold either by cooperatives or individually was estimated at around 307 MT). It is interesting to note that in 2019 the quantity of maize sold continued to increase even though it was a difficult year for agricultural activity (several climatic hazards). This highlights the interest of cooperatives to sell to big buyers and to increase their marketed share. Because record keeping by cooperatives is poor, it was not possible to estimate the net margin made by cooperatives. 72. All cooperatives have increased the adoption of PHHS practices. Cooperatives reported that their view on post-harvest treatment of maize has completely changed because they now have a market for quality maize. Most cooperatives are harvesting at the same time, aggregating production, and drying on cooperative ground (See figure 28 in Annex 7.4). However, cooperatives remain very reluctant to store maize before selling it because members are in need of cash at harvest time. Only two cooperatives reported being able to store more easily than before. Seven cooperatives considered that they need a building to dry their maize. However, given that the LRP project introduced some practices to dry maize on temporary ladders, this is probably more a question of social status for the cooperative than a real operational constraint. 73. Service provision by cooperatives to SHFs have seen a significant improvement from baseline to endline. As figure 29 in annex 7.4 shows, these services include an increase in 21 At baseline, it was between 0,6 MT and 3 MT 22 Data collected during qualitative data collections are different, but cooperatives do not use written records to indicate the amount sold so there can be mistakes. WFP data are more reliable. Evaluation Report – LRP Rwanda Endline 37 |P a g e the purchasing of maize and beans from members, improved supply of fertilisers and other chemicals, better market information and supplying other agricultural inputs (bags). Evaluation Report – LRP Rwanda Endline 38 |P a g e Key findings and conclusions – Question 2 • All LRP cooperatives have been trained on GAP, PHHS and governance. • Capacities have increased but are very variable from one cooperative to another. Cooperatives are still fragile. • Overall, governance is good, and the project contributed to a new dynamic and a change of mentality in the cooperatives. Interest and participation of members has increased. • Cooperatives have a clearer business orientation now as compared to baseline. Production and sales have increased. • Capacity of the cooperatives to organize and to manage their activity is still relatively low and they are struggling with accounting, record keeping, and business planning. • Cooperative capital is increasing but funding of agricultural activities is still a limiting factor. Cooperatives are reluctant to have credit, because of the need for collateral, risks, and the costs. • Linking cooperatives with FTMA buyers is a key achievement of the LRP project and has had a very positive effect on cooperative performance (production, post-harvest practices and sales). However, cooperatives remain very dependent on the LRP project to link with buyers. • Both males and female members benefited from cooperatives’ capacity building and market linkages. 2.3. Evaluation Question 3 - Have the LRP project interventions affected male and female SHF knowledge, capacity and choices/behaviour from baseline to endline? 74. Analyses on gender are presented in section 2.7. Extension services, training, and adoption of practices 75. There was no significant improvement in farmers receiving agricultural extension services at endline compared to baseline for both females (increased by 5.2 percent, p=0.102) and males (increased by 2.5 percent, p=0.380) (see table 9 in Annex 8.1).). 76. These results are coherent with findings from qualitative data collection - cooperatives only mentioned that they received extension services from RWARRI/WFP. While the district agronomist was informed about the LRP, local government staff were not involved in the design and implementation of training and monitoring of farmers and cooperatives. 77. Training was conducted by RWARRI field officers with a system of lead farmers (Annex 7.3). GAP training aimed to enhance farmer skills in maize production and focussed on selection of the right variety, timely planting, proper use of inputs, and spacing. Training on post-harvest handling and storage aimed at sensitizing farmers with appropriate use of post-harvest equipment to improve quality of the maize and avoid contamination. 4,230 SHFs (including 2,082 women) have been trained on GAP and 5,310 SHFs (including 2,620 women) have been trained on PHHS. Table 10 in annex 8.1 shows the details by cooperative. On average, 89.8 percent of cooperative members have been trained on GAP Evaluation Report – LRP Rwanda Endline 39 |P a g e and 96 percent of cooperative members have been trained on PHHS. The lead farmer system has been highly effective in reaching and mobilizing SHFs. 78. The score on good agricultural practices and post-harvest handling was calculated using 11 key items (see table 12a in Annex 8.1). At both baseline and final, as seen in the Figure 5 above, the proportion of farmers who achieved an excellent score on good agricultural practices and post-harvest handling was significantly high at final compared to baseline for both females (increased by 17.7 percent, p<0.001) and males (increased by 16.9 percent, p<0.001). 79. All the cooperatives mentioned that members’ knowledge and skills on GAP and PHHS has improved. FGDs highlighted the importance of particular skills such as planting in line, correct spacing, harvesting maize when mature, and using tarpaulin to avoid contamination from the ground. The LRP project did not do any specific monitoring of the adoption of practices. However, a mobile phone survey was conducted for FTMA (results are in table 12b Annex 8.1). These results are quite coherent with findings from the qualitative data collection.23 23 Results from the phone survey concerning skills and practices are subject to interpretation (farmers can think they do the right thing but do it wrong in the field for example) and less effective compared to monitoring fields practices directly. Figure 5 - Percentage of SHF with excellent score on training on GAP and PHHS (quantitative survey) Figure 6 - Percentage of SHFs that have heard of GAP and PHHS (quantitative survey) Evaluation Report – LRP Rwanda Endline 40 |P a g e 80. Main new practices adopted included planting in rows, systematic use of organic fertilisers combined with chemical ones, sorting of cobs after harvest, and avoiding of grains touching the soils after shelling. Farmers who were interviewed at endline reported that these practices have been largely adopted. Other practices24 have been only partially adopted and would require a specific monitoring study to obtain accurate adoption rates. During the FGDs, farmers mentioned that there were still some cooperative members who do not apply the practices which they had learned about. For some practices (correct dosage of fertilizers, use of protective equipment, use of hermetic bags for storage) the main limiting factor for the adoption is the cost of the inputs. For fertilisers, farmers usually know the quantity to apply (and this was already the case at baseline), but they apply the quantity they can afford. Access to market information and credit 81. At baseline, information on the market was limited, and SHF were only able to report on the price offered by local middlemen. At endline, the situation had evolved. SHFs were aware of the GoR floor price and of the price and quality specification of the main buyers. However, SHF were not yet able to explain the price mechanism (for example from cob price to grain price) and were not aware of the minimum price they need to practice gaining profit. Cooperatives (and SHFs) were still dependent on the project to access market information and none of them were using a market information system. 82. The quantitative survey showed that the percentage of SHFs that received credit for their agriculture activities had not changed from baseline to end line (20 percent). SHF use loans mainly for farm inputs and labour. However, the percentage of farmers that use credit to purchase inputs reduced from baseline to endline (from 64.6 percent at baseline to 53.5 percent at endline). Investments in productive assets for agriculture 83. The results of the quantitative survey (figure 30 in annex 8.2) shows that there has been a slight improvement in the percentage of SHFs that planted maize in season A (from 68 percent to 74.2 percent) and season B (from 44.5 percent to 55.2 percent). Overall, the percentage of farmers planting beans was higher than for maize (in both season A and B) and the survey did not show an evolution from baseline to endline. Interest of farmers in maize was already high at baseline and had improved over time. Nonetheless, beans were still a very important crop. Surprisingly, the survey showed a slight decrease in the area cultivated in maize and beans (see figure 31 in annex 8.2). This can probably be explained by the difficult year in terms of climatic conditions. Farmers mentioned that there were However, if we do not focus on the precise percentage, it can give valuable information on trends in terms of adoption of practices. 24 Correct distance between rows and between seeds, use of appropriate quantity of seeds and fertilizers, proper handling of crop protection equipment and products, use of bags, checks of moisture. Evaluation Report – LRP Rwanda Endline 41 |P a g e several challenges related to droughts and flooding and that probably affected the area they could cultivate. 84. A change from baseline to endline is that farmers were at endline prioritizing the purchasing of enough inputs to invest in their maize farm. There was a slight change in the amount spent on seeds (the survey indicates it evolved from 12.168 RWF on average at baseline to 12.345 RWF at endline). Both quantitative and qualitative data showed that the real change took place in terms of the purchasing of fertilizers. Several cooperatives mentioned that it was one of the investments made by their members with the extra income from maize. 85. The below figure confirms that farmers are now spending more in fertilizers than at baseline. As figure 32 in annex 8.3 also shows, more farmers were getting their fertilizers from cooperatives. At baseline, they mainly accessed fertilizers from agro-dealers (59.4 percent) and the cooperative was the second source (35.8 percent). At endline these percentages had reversed - 33.6 percent of SHFs were accessing fertilizers through agro￾dealers and 54.9 percent through cooperatives. 86. However, it remained very difficult for some farmers to purchase sufficient fertilisers to apply the recommended dosage. This is clearly one of the factors that limit the increase in production. It is also an issue that affects the sustainability of the cropping system: maize is a very demanding crop and without any fallow in the rotation (and sometimes no beans) there is a risk of dropping soil fertility. 87. The main limiting factor for agriculture production is climate. According to farmers, climate conditions have had a bigger impact on the production than the non-application of good practices. Farmers mentioned that production seasons are becoming increasingly difficult to plan. There have been several instances of droughts and flooding episodes and it is becoming increasingly difficult to predict when the rainy season will start. Farmers reported struggling to adapt their cropping pattern. Several cooperatives reported losing all their crops and having to replant. This situation was very evident in the field at endline, when there were simultaneously, in the field crops from a very late season A, crops from season B, and already crops from season C. Finding and implementing cropping systems Figure 7: Evolution of the amount spent on fertilizers in RWF (quantitative data collection) Evaluation Report – LRP Rwanda Endline 42 |P a g e that are more resilient to climatic hazards is becoming a key priority to guaranty production and food security. 88. Other limiting factors mentioned by cooperatives were pests and disease, delays in receiving seeds, low quality of some seeds (GoR now wants to supply farmers with locally produced seeds, but farmers consider that they are not productive enough), and costs of the inputs. Access to post-harvest equipment 89. By the end of April 2019, over 7,500 hermetic bags, 5,700 tarpaulins and 100 plastic silos had been distributed to 3,864 SHFs. On average, 73.6 percent of cooperative members had accessed some equipment, but the access to equipment had been very variable from one cooperative to another. From the qualitative data collection, it appeared that eight cooperatives (out of twelve) were very satisfied with the equipment and considered that it contributed to a reduction of post-harvest losses. The other four cooperatives either mentioned that the cost of equipment was expensive, or that not all members could get some equipment. A real change from baseline was evident, when farmers were not using any equipment (maize was mostly on the ground or on pieces of cloth). However, without support from WFP, SHFs cannot access equipment as it is not available locally. This hampers the sustainability of the intervention.25 More details on the post-harvest equipment is presented annex 8.3. Key findings and conclusions – Question 3 • 4,230 SHFs have been trained on GAP and 5,310 on PHHS. Knowledge has improved and some practices are being adopted. • Access to market information has improved but remains very project dependent. • Slightly more farmers are planting maize than at baseline. • SHFs spend more on inputs that at baseline. SHFs now consider that investing in inputs is a priority but high cost remains a limiting factor. • Climatic hazards hamper production and it is becoming increasingly difficult for farmers to plan their cropping season. They regularly lose their investment. • Through LRP, SHFs accessed post-harvest equipment (7,500 bags, 5,700 tarpaulins and 100 silos). Farmers are now using post-harvest equipment (this was not the case at baseline). Yet equipment is not locally available which hampers sustainability of the intervention. 25 However, that may change as WFP has contacted a Tanzanian company that is considering investing in the market and organizing distribution through agro-dealers. Evaluation Report – LRP Rwanda Endline 43 |P a g e 2.4. Evaluation Question 4 - Has the LRP contributed to creating new market opportunities for male and female SHF? 90. Specific analyses on gender are in part 2.7 At the level of the buyers 91. Rwanda’s maize sector is dynamic and presents considerable opportunities for farmers: demand is high and increasing. All the maize produced in Rwanda comes from SHFs (either through cooperatives or through middlemen). Production does not meet demand and Rwanda continues to be a net maize importer. A recent ban on maize imports from Uganda (which was the main maize importer in Rwanda) has boosted prices. Quality of the maize produced in Rwanda is overall considered good in comparison with neighbouring countries. More information on Rwanda’s maize sector can be found in Annex 9.1. 92. The maize market is dominated by a few relatively large local buyers and agro￾processors26. Minimex-Prodev, Africa Improved Food (AIF), East Africa Exchange (EAX), Sarura and Rwanda Grains and Cereals Corporation (RGCC) are the main stakeholders. All five companies are members of the FTMA. 93. The buyers considered the cooperatives had positively changed from baseline to endline. Buyers were already purchasing from cooperatives but perceived cooperatives as unreliable, only able to aggregate small quantities, and unable to produce good quality produce. At endline, all the three buyers met considered that FTMA cooperatives are their business partners and that LRP/FTMA had been instrumental in the change. They were purchasing more maize locally and the share of cooperative product purchased had increased27 . Specifically, buyers reported that: • Cooperative mentality has changed and has become much more business-oriented (for example, cooperatives now provide tax declarations). • Trust between farmers and buyers has improved and there is less conflict. • Cooperatives are better organised, and their capacity has increased. • Cooperatives aggregate more production. • Quality has improved and rejection is less frequent. 94. Except from RGCC which focused more on beans, the four FTMA buyers signed contracts with cooperatives. Most of the LRP cooperatives signed contracts with AIF. According to figures from RWARRI, 12 cooperatives signed contracts for season A 2018 and nine for season B 2019 (see figure 34 in annex 9.2). 452 MT were sold to FTMA buyers in 2018 and 252 MT in 2019 (but maize was not yet sold for four cooperatives). However, it is unlikely that the figure of 2018 will be reached because 2019 was not a very good agriculture year 26 More information on FTMA buyers is in Annex 7.4.2. 27 It was not possible to have precise data from buyers on quantity purchased from LRP/FTMA cooperatives. Evaluation Report – LRP Rwanda Endline 44 |P a g e (several climatic hazards). In 2018, on average cooperatives sold 117 percent of what they had committed to in their contracts. In 2019, the cooperatives “only” sold 89.8 percent of what was in their contracts (the figure below shows the result from the different cooperatives). Six cooperatives provided above their contract’s commitment in 2018 (and three in 2019). On average, LRP cooperatives signed contracts for 30.75 MT in 2018 (ranging from 3MT to 100 MT) and 28.9 MT in 2019 (ranging from 10 to 50 MT). Quantity delivered was on average 37.7 MT in 2018 (from 1.3 MT to 172.5 MT) and 24.9 MT in 2019 (from 3 to 74.3 MT). These results are very promising. However, these contracts are still fragile, which is highlighted by the fact that some cooperatives that had very good results in 2018 could not deliver in 2019. Figure 8 (left): LRP cooperative contract achievement rate for (RWARRI)28 Figure 9 (right): Factors that influence cooperative choice by clients (multiple options possible, qualitative data collection, N=12 cooperatives) At the Cooperatives’ level 95. Cooperatives reported satisfaction with the contracts they have signed with FTMA buyers. Only three cooperatives mentioned difficulties (some delays in payment and defaulting from the buyer). These cooperatives explained that RWARRI supported them to find a solution and they would consider signing a contract with the same buyer again. 96. Overall, cooperatives reported good contract conditions. FTMA/LRP has guided buyers in adapting their contracts to respond to cooperative needs and constraints. For most cooperatives the choice of buyers is determined by their willingness to purchase maize in cob. This is an evolution from baseline to endline, as no buyer was purchasing maize in cob in 2017. This change has been initiated by AIF which uses the services of a social enterprise (Kumwe) that is organizing collection of cobs just after harvest, transportation to a central processing facility, shelling and drying before supplying to AIF. This innovative model has contributed to reducing post-harvest operation time for farmers29, reducing 28 For some of the cooperatives that have not yet delivered anything it would be interesting to assess the situation after season B as they may not have sold yet. 29 When they harvest maize in season A, farmers are already very busy with preparing season B and planting beans Evaluation Report – LRP Rwanda Endline 45 |P a g e post-harvest losses, and reducing contamination with aflatoxins. More information on Kumwe can be found in annex 9.2. 97. Other important factors for cooperatives are payment delays, organization of transport, quality checks on site, and price offered. Farmers are in urgent need of cash at harvest time and it is therefore very difficult for them to wait (this is the main reason why in the past they would sell to middlemen at a lower price before full maturity). Organization of transport is difficult and costly for cooperatives, so having maize collected at the farm gate is a positive factor. Quality checking the maize before sales is a way for farmers to avoid rejection at the factory (in which case they have to bring the maize back to the cooperative if they cannot find another buyer). 98. LRP cooperatives reported having improved their skills for quality management and business orientation. This puts them in a much more favourable position to respond to the demand from big buyers than was the case at baseline. There is a clear opportunity here to develop value chain dialogue and find sustainable long-term solutions. Already, there has been some dialogue between buyers (buyers mentioned that they now have a common vision, exchange on their practices, and have fairer business practices) and some adaptation of contracts to cooperative constraints. Together with the very conducive context (high price, reduction of imports, among other factors), there is room to strengthen the whole value chain and to have all the stakeholders on board (producers, input providers, buyers, government, NGOs, etc.) to collectively discuss and find solutions for an inclusive value chain development and to make sure this is reflected in government policies and strategies on agri-business. At WFP level 99. Under the LRP programme, WFP has purchased beans to supply to the Home-Grown School Feeding (HGSF) programme. WFP purchased 190,178 MT of beans from RGCC and 30,800 MT of maize meal have been purchased from Minimex, the only local producer of fortified maize meal that is able to comply with WFP quality standards. It is very likely that some of the Minimex maize meal was from LRP cooperative maize, and other FTMA buyers, but the evaluation was not able to evaluate the quantity. At the time of the evaluation WFP had not directly purchased beans from LRP cooperatives. However, shortly afterwards WFP purchased 82 MT from 3 cooperatives, using non-USDA funds. There is no traceability system in place and thus no guarantee that the food supplied to the schools in the LRP districts comes from the cooperatives from the LRP districts.This is the consequence of a centralized system and high quality processing standards. With schools 30 However, WFP is planning a purchasing campaign in July 2019 in the southern province Evaluation Report – LRP Rwanda Endline 46 |P a g e 100.LRP contributed to establishment of a high level inter-ministerial working group (with representatives from MINEDUC, MINAGRI, Ministry of Trade and Industry (MINICOM) and WFP) to develop a pro-smallholder farmer procurement strategy for school feeding. A study was conducted to explore school meal options that would be nutritious and would promote procurement from local farmers. The objective of the strategy is to create sustainable demand at farmer level. The strategy does not promote a unique model but rather explores modalities, effects and potential trade-offs of different options and models. This has contributed to raising awareness of government stakeholders on the potential of a school feeding programme for both the agriculture and education sectors. The strategy is not practical enough to guide the drafting of an implementation plan without further data collection and analysis. Also, in order to translate into and implementation plan, the GoR will need to take decisions on the model to promote and its modalities in the field. 101.LRP cooperatives were not providing maize or beans to schools directly at the time of the evaluation, although a procurement process was ongoing for three LRP cooperatives to supply 82 tons of beans to WFP for use in the HGSF programme. One cooperative mentioned that they were selling maize to a miller that is supplying a secondary school. Cooperative awareness of the regulations and specifications for supplying to schools was still very low. However, with the improvement of the quality and a growing cooperative aggregation capacity, opportunities for selling to schools could be explored (some of the good cooperatives could probably apply for tenders with some support from RWARRI). Given the likely evolution in terms of school procurement regulation and strategy, supporting cooperatives to procure to schools may become an interesting perspective. Key findings and conclusions – Question 4 • Rwanda’s maize sector is dynamic and presents a strong opportunity for SHFs. • Buyers consider cooperatives as reliable business partners. 452 MT in 2018 and 252 MT in 2019 were sold to FTMA buyers by LRP cooperatives • Contract conditions have been well adapted to SHFs’ constraints, needs and capacity. Some cooperatives were still defaulting, but overall LRP cooperatives respected their commitments. Some delivered more than what is contracted. • Adoption of the cob model has been an innovation that has been very positive for cooperatives as they can save time for post-harvest operation and improve quality. • Cooperatives were not selling to schools, but this may evolve in the coming years in light of the new national school feeding strategy. • WFP had purchased 190,178 MT of beans and 500 MT of maize meal through the LRP project but not directly from LRP cooperatives. Evaluation Report – LRP Rwanda Endline 47 |P a g e 2.5. Evaluation Question 5 - To what extent do producers market food products that meet quality standards and that are nutritious and culturally accepted? 102.SHF consume mainly beans, maize and sweet potatoes. Most of SHF food consumption comes from their own production. Qualitative data collection confirmed that beans are essential for food security. However, several cooperatives mentioned that maize was also becoming increasingly important. After harvest, SHFs reported prioritizing keeping enough maize to feed their family. However, their strategy is not to store everything, but rather to plant maize and beans on several plots and in different seasons to ensure a regular supply. In this context, FGDs highlighted the importance of diversifying crops. 103.Five cooperatives reported increased food security through improved production. Several FGD participants mentioned that they were able to store more and that this contributed to reducing their need for maize purchase. Other SHF mentioned that they are often less compelled to sell beans.31 104.The survey results underlined that strategies differ for male and female SHF. As figure 10 below shows, women slightly increased the quantity of maize they keep for family consumption whilst it strongly decreased for men. Men used the extra income from sales to secure their households needs rather than storing the product. Figure 10: Quantity of maize kept for household consumption during the last planting season (survey results) 105.Qualitative data collection highlighted that SHFs have changed their perception of quality since baseline. In KII with cooperatives and FGDs with farmers, participants were able to explain at least some factors affecting quality of the maize. All cooperatives reported that maize quality and maize conservation had clearly improved in the last two years. There was an apparent contradiction with the survey findings on household preference, as explained below. 106.Household preference for maize and beans was assessed using five aspects, namely, insects, colour, dryness, stones, and variety (see figure 35, in annex 10.1). There was a 31 Beans are still considered as the major food security crop, and selling beans is a last resort action when in urgent needs of cash. Evaluation Report – LRP Rwanda Endline 48 |P a g e significant decrease (p<0.001) in the proportion of farmers having a specific preference for any of the five characteristics at final compared to baseline, for both females and males. This suggests that households place greater emphasis on quality. Improvement of the quality of maize was confirmed by interviews with buyers, highlighted by quickly dropping rejection rates (from 90 percent to 15 percent for AIF and from 65 percent to 5 percent for Minimex). 107.SHFs mentioned that they appreciate consumption of maize because it was possible to consume maize in different forms (grilled, flour, porridge etc.) and was very easy to prepare. An FGD participant called it “the perfect crop”. Consumption patterns had already changed before the project i.e. farmers were already eating maize regularly while it was not consumed 7-10 years ago. This evaluation confirms that this is a continuing trend. 108.Crops promoted by the LRP can be part of a nutritious meal for a school feeding program. Annex 10.2 gives more details about how the LRP project contributed to a stronger acknowledgment of nutritious and culturally acceptable food products for schools in Rwanda. Key findings and conclusions – Question 5 • Most of SHF food consumption has come from their own production. • Maize has become increasingly important for SHF food security and its importance in household food patterns is increasing. • Cooperatives and buyers acknowledge that quality of maize has increased considerably in the last two years. 2.6. Evaluation Question 6 - Has the LRP programme affected male and female SHF income, and in what way? 109.LRP’s implementation period has been very short making it difficult to discern any real effect on income. While SHF production and sales have shown an increase over one or two seasons, it is clearly too early to talk of impact. However, there are some signs suggesting that the project has affected SHF income. 110.SHF choices were still mainly driven by the household food needs at endline. The majority of what SHF produce is consumed by the family or sold to purchase other food products. This did not change from baseline to endline. FGD highlighted that crops that contribute the most to SHF income are maize, Irish potatoes, coffee and rice (for those cooperatives that are growing rice and coffee). Beans are still a crucial crop for SHF food security, but maize is gradually also becoming a key crop for food security. 111.As figure 11 below shows, the percentage of farmers who purchased maize and beans for family consumption reduced from baseline to endline. The reduction was significant for females (reduced by 14.5 percent, p<0.001), but not for males (reduced by 5.1 percent, p=0.117). This was coherent with findings from the qualitative data collection: farmers Evaluation Report – LRP Rwanda Endline 49 |P a g e have been able to increase their production and have better storage conditions, so they rely less on purchasing maize and beans. Figure 11: Percentage of farmers who purchased maize and beans for family consumption in seasons A and B 112.Production and yield increased for maize in the marshland areas. However, overall production was still low due to poor soil fertility, high erosion, climatic hazards, inadequate crop rotation and association, and the presence of pests and diseases. Avoiding risk was also still an important consideration for SHF. Therefore, mixed cropping remains the best strategy to mitigate risks. SHF also rely on tubers (mainly sweet potatoes) that can remain in the ground and be harvested gradually. 113.As the figure 12 below shows, quantity of maize harvest sold during season A was significantly higher at final than at baseline for females (increased by 55 kgs, p<0.001) but not males (increased by 37 kgs, p=0.102). Figure 12: Quantity of maize sold in kg by season from baseline to endline (quantitative survey) 114.The amount of money earned from the sale of maize during season A was significantly higher at final compared to baseline for females (p<0.001) but not males (p=0.747), as figure 13 shows. Similarly, the amount of money earned from the sale of maize during season B was significantly higher at final compared to baseline for females (p<0.001) but not males (p=0.745). Figure13: Amount of money from the sales of maize from baseline to endline (quantitative survey) Evaluation Report – LRP Rwanda Endline 50 |P a g e 115.In the focus group discussions, farmers confirmed that maize sales had improved compared the period preceding the LRP project and attributed this to increased production and reduced post-harvest losses. Nine cooperatives out of twelve (qualitative data collection) mentioned that member income had increased. The cooperatives that did not report a change were the ones that have not yet sold to big buyers. 116.With increased production, better prices, and a reduction of post-harvest losses, profitability of maize cultivation increased from baseline to endline even if the amount of work and inputs have increased. However, maize cultivation is not profitable (see figure 37 in annex 11) if a SHF was to give a financial value to his work in the farm. Profitability is low given the amount of work it requires to get production. Nonetheless, maize is a profitable crop from the perspective of the cash that SHF receive at the end of the season. 117.Selling to middlemen is clearly not profitable if a farmer invests in quality inputs, applies GAPS and PHHS. When selling to FTMA the profitability for buyers mainly depends on the price offered. Price of maize is very volatile (in 2019, the cooperative sale price to AIF ranged from 210 RWF/kg to 270 RWF/kg) and this can affect profitability of the operation (at 210 RWF/kg and taking into account family labour, growing maize was not profitable in the last season). 118.Economic calculations also show that selling in cobs is less profitable. It reduces post￾harvest work but cost of inputs and labour remain the same (and they are the biggest part of the expenses). Also, selling in cobs stops farmers from using the by-products of maize which they would normally apply on other plots as mulch (on coffee farms for example) or to prepare fire for domestic use. When selling in cobs farmers will probably have to purchase more firewood32. However, in most cases, farmers indicated they still prefer selling in cobs because they were able to work faster on their other plots and because it reduces the risks of post-harvest losses, contamination, and moulds. This underscores that reducing risks is more important than increasing profit. Increased profit was also 32 This can also have consequences on the environment and increase the pressure on natural resources. Evaluation Report – LRP Rwanda Endline 51 |P a g e mentioned by farmers in the FGD. They mentioned that from baseline to endline more farmers have been able to pay for the health insurance and to purchase inputs for the next agricultural season (see table 15 in annex 11). Other (less frequently mentioned) results include that famers being able to purchase animals, pay school fees, and invest in new clothes. Key findings and conclusions – Question 6 • The LRP implementation period was too short to see a real impact on SHFs but there are indications that the LRP project has had an effect on SHF income. • Maize is the crop that contributes most to SHFs income and it is also very important for food security. Purchase of maize and beans for household consumption has reduced from baseline to endline. • Maize cultivation is profitable if the financial family labour is not taken into consideration. • Selling to FTMA buyers is more profitable than selling to middlemen. However, price volatility can hamper profitability. • Selling in cobs is less profitable than selling in grains but it is more convenient for farmers. 2.7. Evaluation Question 7 - Is there evidence that the LRP programme affected male and female SHF differently? Gender in LRP’s activities and results 119.The LRP project did not implement activities that specifically targeted female SHFs’ conditions or concerns. Nonetheless, women’s participation in LRP activities has been strong, as the table below shows. LRP reporting disaggregates data by sex, though it is not systematic. For example, quantity of maize produced by cooperatives is not disaggregated by gender. It was therefore not possible to assess whether women benefitted equally from the new opportunities associated with the LRP project. Evaluation Report – LRP Rwanda Endline 52 |P a g e Table 6: Women’s participation in LRP's activities (WFP reports) Participants Training on GAP Training on PHHS Access to PHE Training on governance Percentage of female participants 49.2% 49.3% 48.6% 48.6% 120.In terms of effects of project activities on women and men, a key finding from the survey is that while the LRP programme targeted both females and males, effects on females was much more pronounced for a number of key indicators. This suggests that some of the activities were of particular relevance to women, even if they were not explicitly planned in this manner. 121.Thus, overall, maize planting improved significantly for females during season A (improved by 10.2 percent; p=0.002) and season B (improved by 16.8 percent; p<0.001), but not by similar scale for males during season A (improved by 2.8 percent; p=0.337) and season B (improved by 5.1 percent; p=0.137). Figure 14 - Percentage of farmers planting maize at baseline and final, for season A and season B 122.Women also purchased less maize and beans for family consumption. Thus, the survey clearly highlighted that the purchase of maize and beans for family consumption in the last season A and B significantly reduced at final compared to baseline (reduced by 14.5 percent, p<0.001) for females, but not for males (reduced by 5.1 percent, p=0.117). Figure 15 - Percentage of farmers who purchased maize and beans for family consumption in seasons A and B (survey results) 62.5% 72.7% 42.6% 59.4% 72.7% 75.5% 46.0% 51.1% 68.0% 74.2% 44.5% 55.2% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Baseline Final Baseline Final Baseline Final Baseline Final Baseline Final Baseline Final Season A Season B Season A Season B Season A Season B Female Male Total Evaluation Report – LRP Rwanda Endline 53 |P a g e 123.Women’s access to extension services has also increased, but not for men. Women were more likely to have access extension services through WFP, other UN agencies, Government services and CBOs than men, and were also more likely to have accessed extension services in the past six months (see table 9 in annex 8.1). Women’s participation in the cooperatives 124.Overall there was a balanced number of men and women in the cooperatives, though it has slightly decreased since baseline (from 53.2 percent to 51 percent, on average). Some cooperatives have a very low number of women and that was not the case at the baseline. For example, Icyerekezo has only 46 women now while there were 147 women at baseline. An identical situation was observed for Abemeherigo cooperative which reported 364 women at baseline against 180 women at endline. No evidence found can explain this, but some assumptions can be made. The figures may reflect poor record keeping but highlights a need to ensure that there is attention on whether some women are excluded from the cooperatives. It may also be, for example, that husbands are taking over now that maize cultivation is becoming an interesting opportunity. This is something that has been noticed several times in different contexts - when a crop becomes a source of income, males take up the crop that was formerly managed by women as a source of food for the family. 125.At baseline, it was noted that in some cases women’s participation in the cooperative reflected low male interest in cooperative activity (with husbands sending their wives so they can focus on other activities). Both women and men have been taking part in the discussions and General Assemblies of the cooperatives. In the FGDs, both women and men participated the discussions and no specific issues concerning gender emerged. 126.All cooperatives had women participating in the committee. However, women still play a limited role in the decision–making structures of the cooperatives. Qualitative data collection highlighted three main limiting factors for women: a lack of time to dedicate to cooperative as they are already very busy in the house; a lack of confidence to express 77.5% 63.0% 67.0% 61.9% 71.8% 62.4% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Baseline Final Baseline Final Baseline Final Female Male Total Evaluation Report – LRP Rwanda Endline 54 |P a g e themselves and take decisions; and literacy issues that prevent women from playing an active role in the cooperative committee. Key findings and conclusions – Question 7 • The LRP design did not include gender specific activities but participation of women has been good. • From baseline to endline, some project activities brought changes for women, but not for men, even though the project did not design interventions specifically to meet the needs of women. • Thus, at endline women were planting more maize, had improved access to extension services, and were purchasing less maize and beans for family consumption as compared to men. • There is a balanced number of men and women in cooperatives. However, in some cooperatives, the number of women members had decreased from baseline to endline. • All cooperatives had women in the cooperative committees but their participation in decision making is limited. Evaluation Report – LRP Rwanda Endline 55 |P a g e 2.8. Evaluation Question 8 – What internal and external factors affected results? Internal factors affecting results: Factors affecting implementation and monitoring of the project and the overall efficiency 127.The choice to implement the project through a local partner has been a key factor of success. RWARRI has very good skills and experience in working with SHFs and cooperatives and has dedicated appropriate staff and resources to the LRP project. Implementation has been carried out without facing major difficulties. Coordination among the team and with WFP has been very good, according to the different stakeholders interviewed. In particular, frequent visits by the field officer to cooperatives have been particularly useful to support the implementation of good practices and has improved cooperatives’ linkages with the market33. One field officer is in charge of about nine cooperatives which represents a reasonable ratio. However, the knowledge and skills of field officers appears to be lacking in some areas. This is particularly the case for governance capacity building, accounting and business management where skills are not strong enough to ensure proper support and monitoring of cooperatives. 128.Out of the 16 monitoring indicators selected for the project (see annex 12), ten have been achieved (or are very close to being achieved) at endline, six are partially achieved. The effectiveness of the LRP project is therefore good. However, the monitoring system in place has not been strong enough to capture changes at cooperative and farmer level. In addition, some of the USDA specific indicators have clearly not been relevant to this project, for example, “number of public-private partnerships formed as a result of USDA assistance”. Other indicators had to be re-interpreted to make them relevant to the project. This is the case, for example, with “number of social assistance beneficiaries participating in productive safety nets as a result of USDA assistance” which has been translated into “number of small holder farmers that have benefitted from post-harvest equipment”. Various indicators that would be very relevant to this project have not been monitored. This is the case for specific yield and production from farmers/cooperatives, as well as individual sales made by farmers. Adoption of practices has been measured through a mVAM survey, which does not seem very appropriate for this kind of indicators. Thus, the 98 percent adoption rate of GAP is a doubtful result after only two seasons of support. 129.Reporting of the project has been limited. Specific LRP reporting is limited to WFPs report to USDA. Operational reporting is included in the reports from FTMA. While this makes sense at an operational level it has not brought out the specific situation of the LRP cooperatives vis-à-vis other cooperatives that receive support. Reports also did not include analyses of results. 33 That was frequently quoted in the FGDs and KII Evaluation Report – LRP Rwanda Endline 56 |P a g e 130.It is not possible to assess the efficiency of the project because most of the project’s activities are capacity building related. However, the budget available for capacity building (127,000 USD) was very reasonable considering the number of farmers trained. There were no issues with the amount available for these activities according to RWARRI and no delays in receiving the funds from WFP. External factors affecting results 131.The results of the project have been adversely impacted by climatic hazards. As was previously mentioned, it is becoming increasingly difficult for farmers to plan their cropping seasons and there were several cases of production losses because of droughts or floods. These elements were confirmed both by the project team, farmers and by government officers. This negatively affected the result of the project and maize production in the country. 132.However, climate hazards have contributed to a positive impact on the demand for maize and therefore on the price, which created a favourable context for LRP cooperatives to market their maize. Farmers who managed to get good production in spite of the difficult year benefited from the situation. For other farmers, the price increase did not fully compensate the losses. In a similar way, the closing of the border with Uganda also increased the demand and price and favoured the sale of maize by the LRP cooperatives. 133.Farmers’ choices on cropping systems are more regulated in Rwanda compared to other countries. The strong push from the GoR towards maize cultivation definitely contributed to the results of the project. This is especially true in the marshland areas (which belong to the GoR) where farmers are strongly encouraged to grow maize (with the risk of losing access to marshland if they are growing something else). 134.Operational difficulties reduced the GoR capacity to provide inputs and affected the project. Several cooperatives mentioned that they received the seeds very late, as a result of which at least one cooperative chose an alternative crop. Also, farmers consider (though it is not possible for the evaluation team to know whether this is correct) that the seeds produced in Rwanda are giving lower yield than seeds imported from Kenya. Key findings and conclusions – Question 8 • Internally, project achievement was positively affected by the choice of local partner, by good coordination, and frequent field presence of field officers. • However, field officers’ knowledge and skills on capacity building for governance, accounting and business management were not strong enough to provide adequate support. • The monitoring processes for the project have been weak. • Externally, the project has been affected by adverse climatic conditions and challenges faced by the Government of Rwanda in providing agricultural inputs to farmers. The centrally regulated system of production contributed to a strong push for maize Evaluation Report – LRP Rwanda Endline 57 |P a g e production, and high maize prices have favoured farmers who managed to achieve good production rates. Evaluation Report – LRP Rwanda Endline 58 |P a g e 3. Conclusions and Recommendations 135.Based on the findings presented in the previous section, an overall assessment by evaluation question is provided below. This is followed by ten recommendations for future work by WFP in this area. 3.1. Overall Assessment/Conclusions Conclusion 1- Project performance, relevance and implementation processes have been good. 136.In a very short period of time, the LRP project has been able to produce some substantive results. Considering the very short implementation period, the evaluation team concludes that the results of LRP intervention are good. 137.Relevance of the project has been strong. The project has been fully aligned with the orientation and priorities of the GoR. The project has targeted the most important constraints faced by SHFs. The project design did not, however, take into account the specific needs and constraints faced by female SHF. 138.Implementation was carried out without facing major difficulties. The implementation mechanism in place was very good and well adapted to the activities implemented. 139.Out of the 16 monitoring indicators selected for the project (see annex 12), ten have been achieved (or are very close to being achieved) and six are partially achieved. The effectiveness of LRP project is therefore assessed as being good. 140.Reporting and monitoring of the project has been a weak point. There was no specific monitoring of the adoption of practices and of the production of SHFs and cooperatives (only of the sales at cooperative level) and this has limited the findings and conclusions at impact level. Monitoring and reporting on gender has been limited. Conclusion 2- LRP has created a dynamic at cooperatives level that contributed to building cooperatives’ capacity. However, cooperatives are still fragile and cannot fully take advantages of new opportunities without project support. 141.LRP cooperatives’ ability ‘to be’ at endline was relatively good mainly because of the strong drive from the government on organization of cooperatives. Ability ‘to do’ remained limited. Cooperative skills on governance, organization, management, accounting and business planning were still weak. LRP’s intervention did not focus enough on building cooperative capacity in these key areas. As a result, cooperatives remain very fragile. They develop new activities, but they do not have the skills to manage them independently of the project. This has limited, and will continue to limit, the capacity of cooperatives to aggregate more maize and to link with buyers and/or service providers. 142.‘Ability to relate’ has improved and cooperatives are now regularly signing contracts with big buyers which was not the case at baseline. Evaluation Report – LRP Rwanda Endline 59 |P a g e 143.‘Ability to perform’ has also considerably increased from baseline to endline. More than 600 MT of maize have been aggregated by LRP cooperatives and the target is about to be reached (there was still some sales going on at the time of the endline). This is a major achievement by the project taking into consideration that in 2019 production was low (mainly due to adverse climatic conditions). Conclusion 3- Farmers’ knowledge on GAP and PHHS has increased and there is evidence of changes in input use and adoption of practices. However, this change is not complete and is very fragile. Farmers’ access to productive assets remains difficult and production is very climate sensitive. 144.GAP and PHHS capacity building activities have been a success at farmer level. The capacity building scheme, methodology, and material were particularly good and adapted to the situation on the ground. The activities have been much appreciated by beneficiaries. The evaluation finds that farmers’ knowledge improved, and some practices are now largely adopted. 145.LRP has contributed to improved access to several production factors. Capacity building activities improved farmers’ utilization of inputs which, combined with application of GAP, resulted in a better production, as long there is no climatic hazards or pest attacks. Training sensitized farmers on issues related to quality and post-harvest handling and this has completely changed SHFs’ perceptions of the quality of maize. Supply of PHE has also been a key factor for the adoption of these practices. LRP had not yet reached its target when the endline survey was conducted, but it was expected that the target would be reached by the end of the project (September 2019). 146.SHF investment continued to be constrained by the limited financial means at their disposal. LRP’s results in terms of access to credit for farmers and cooperatives have been very limited. Conclusion 4 - LRP demonstrated that with appropriate support, cooperatives have access to better market opportunities through linkages with formal buyers 147.The success story of the LRP is to demonstrate what can happen when farmers are connected to the formal markets through linkages between cooperatives and buyers. LRP has clearly contributed in creating new opportunities for SHF. At endline buyers consider cooperatives as reliable business partners and they have increased their sourcing from cooperatives. Most of the LRP cooperatives have signed contracts with FTMA buyers although they continue to rely heavily on the project team to support them in making these connections. 148.LRP cooperatives have not linked directly with schools. However, the context should become more favourable in the coming years with the drafting (with support from WFP) of a pro-small holder farmers school feeding strategy by the GoR. Evaluation Report – LRP Rwanda Endline 60 |P a g e 149.Quality has increased and the rejection rates at buyer level have strongly decreased. This is linked with capacity building, supply of post-harvest equipment, and adaptation of the marketing model by some buyers. Some buyers now purchase maize in cobs which contributes to a reduction of aflatoxin contamination. It has also reduced the time that SHF invest in post-harvest operation. However, it is not the most profitable option in terms of income. Conclusion 5- The LRP does not yet have an effect at farmer level but it contributes to an increased interest and profitability of maize production for both males and females. 150.As a project, the LRP has been implemented for less than two years. This is too short to have a long-term impact at SHF level, or indeed to assess impact. However, at endline there are signs that the LRP has positively affected farmer income. Farmers’ cropping choices are still mainly driven by household food needs and a focus on avoiding risks. SHFs continue to rely on mixed cropping and on bean cultivation, but maize plays an increasingly important role in food security. Production and sales of maize have increased, and post-harvest losses have decreased. Therefore, profitability of maize has increased (but it is still very low if farmers were to include their time on the farm in the financial calculation). 151.The specificity of LRP has been to tackle issues affecting farmers from the market side, capacity side, and production assets side at the same time. It created a virtuous circle for farmers and cooperatives to get engaged in an improved production (quality and quantity) and created a synergy in the interventions. With better opportunities, farmers have an incentive to invest in production and good practices and cooperatives become the vehicle to get these opportunities, so farmers are getting interested in their cooperatives. This dynamic is the added value of LRP and contributed to the effect on the ground. 152.The project did not implement gender specific activities and there were no gender specific activities in the project design. However, women and men participated equally in the activities of the project. This has contributed to positive results for women. Thus, maize planting has increased more for women than for men, women SHF access to extension services has improved from baseline to end-line, and with the improved production, female SHF have purchased less maize and beans for consumption now than at baseline. The role and position of women in cooperatives has not changed. While cooperatives overall have the same number of male and female members, the involvement of women in the governance and decision-making of cooperatives remains limited. 153.Several intermediate outcomes in the ToC have already been achieved. The ToC clearly shows, however, that two major outcomes have not been reached and hamper results: cooperatives and SHFs still do not have a strong position in the value chain, and value chain dialogue is not developed enough. The challenge for LRP moving forward will be to tackle the factors limiting achievement of these two outcomes and to create conditions for the change achieved to continue and to make it sustainable without external support. Evaluation Report – LRP Rwanda Endline 61 |P a g e Conclusion 6 - The change that is seen is very fragile and sustainability of the intervention and outcomes is low. 154.At endline, the evaluation finds that farmers and cooperatives continue to be very project￾dependent. Some of the support received is not based on locally available inputs (this is the case for post-harvest equipment). And, the cropping model that has been promoted is not environmentally sustainable. It is based on intensive cultivation and input use, it is very sensitive to climate hazards, and may affect soil fertility. 3.2. Lessons learnt 155.LRP’s implementation process clearly highlights the importance of having a strong presence in the field to support cooperatives and farmers on a regular basis and not only through periodic training. This has clearly contributed to the good results of the project. 156.The main lesson learnt from LRP is the catalyst effect of working both on improving capacities and assets of farmers and on linkages with the market. Improved market access has clearly contributed to the interest of farmers for GAP and PHHS. And through PHHP and GAP farmers have been able to access market opportunities that were previously out of reach. Working on both elements created a virtuous circle. 157.The LRP highlights the importance and opportunity of working jointly with MINAGRI and MINEDUC on the school feeding strategy. The drafting of a pro-SHF strategy is a very interesting process and could produce promising result as actors work together on a strategy that is mutually beneficial and adapted to the local conditions. 3.3. Recommendations 158.The recommendations of the evaluation team are outlined below. These are presented here in order of priority. However, taken together the recommendations are complementary and should be considered collectively to ensure success moving forward. Operational recommendations – linked to the current LRP programme 159.Recommendation 1 (related to conclusion 1): In the coming six months, WFP should work with the GoR to prepare a short lesson learning document of the LRP implementation and its results for dissemination and learning. LRP and FTMA constitute an innovative way of working with cooperatives that deserves a study covering more cooperatives than the 16 LRPs ones. The evaluation has highlighted that there are lessons to be learned for agriculture and value chain development in Rwanda and on a broader scale from the LRP project. The FTMA program in Rwanda probably has very interesting results which are of importance not only for WFP but also for other development partners. In light of the gender specific findings presented above, a short reader-friendly learning document could be drafted which draws on this report. The Evaluation Report – LRP Rwanda Endline 62 |P a g e lesson learning document should bring out the specific benefits and continued constraints for female SHF, including with respect to their participation in cooperative activity. 160.Recommendation 2 (related to conclusion 2 and 4): For the remaining time of the project and for the next phase, WFP should work with RWARRI to prioritize cooperative capacity building activities in the areas of governance, work organization, business management, and financial accountability. This should be the main focus at cooperative level for at least three more years to ensure that cooperatives can handle the new activities and opportunities in a sustainable way. We recommend that the next phase of the project considers hiring a cooperative capacity building specialist as part of the implementation team, to design a curriculum and adapted material. Training sessions should be organized in all the cooperatives on a regular basis. Field officers should be trained and monitored so that they can provide day-to day coaching to the cooperatives. All the elements concerning ‘ability to be’ and ‘ability to do’ should be included and the following specific elements should be included: record keeping, contract management, capacity strengthening targeted at women, economic calculation to verify profitability of an activity, and negotiation skills. In addition, WFP’s hired cooperative capacity building specialist as recommended above can ensure supervision and support and coach the field officers. 161.Recommendation 3 (related to conclusion 3 and 4): For the remaining time of the project, WFP should continue working with ICCO Terrafina to strengthen the access to finance component of the project. In order to improve farmer and cooperative access to credit, the savings and credit groups that were established emerge as a good option to build up cooperative capital (whether or not it is to access formal loans). For a second phase of the project, this approach should be experimented in more cooperatives with regular monitoring and further training on financial management and loan management. At the same time, work should continue with banks and microfinance institutions to develop specific products for agriculture. This should include attention to a crop insurance system. Another potential option is to have built-in credit from the maize buyers. The situation is now more favourable for this kind of system as the trust in cooperatives has increased and buyers are looking for options to secure their supply. It could be tried on a pilot approach with only well-performing cooperatives and with a close monitoring to develop good practices before upscaling such a scheme. 162.Recommendation 4 (related to conclusion 2 and 3): In the next six months, WFP Rwanda should work with the WFP Regional Bureau and the GoR, to develop a project proposal and raise funding for a second phase of the LRP. As part of this, WFP should conduct a study to assess the place and roles of women in the maize value chain to ensure that women are not getting excluded now that maize is becoming profitable. Capacity building and behaviour changes are a long process. The evaluation has highlighted that the interventions have brought about important changes, but cooperatives remain fragile and adoption of good practices is not yet Evaluation Report – LRP Rwanda Endline 63 |P a g e anchored in farmers activities. At least one round of refresher training on GAP and PHHS should be organized for cooperatives and coaching should continue for three years minimum with the objective of progressively switching to demand-based advice and extension services (which should progressively support cooperatives for specific requests). Support should be more specific to the situation, capacities and needs of each cooperative. An analysis of the situation of the cooperatives should be done. If needed, cooperatives with very low ownership and strong governance problems, and those that are collapsing, should be excluded. The implementation design could remain very similar but with a better inclusion of district/sector level agriculture staff in the activities to build their capacity and improve appropriation of the results of the project by the local authorities. Strategic recommendations (beyond the current project period) and for future interventions 163.Recommendation 5 (related to conclusion 5 and 6): In the coming year, WFP should liaise with relevant stakeholders (MINAGRI and development partners working on these issues) to integrate a climate smart approach in the cropping model promoted and review the training curriculum and materials. So far, the LRP has focused only on maize and improving the yield. In order to increase sustainability of the intervention a more comprehensive approach of environmental issues is critically important. Improving resilience of cropping system to climatic hazards is key not only for farmer income and food security, but also to ensure that cooperatives can supply products on a regular basis. Specific expertise is needed to identify the main actions that should be integrated. This could cover issues such as: better integration of animals raised in cropping systems, crop￾diversification, introduction of trees in cropping systems, fertility management, among other areas. There is a need for sensitization at farmer level but also at government level in order to review GoR recommendations for agriculture in the marshlands. 164.Recommendation 6 (related to conclusion 4 and 5): In the coming three months, WFP should conduct a precise assessment of the different marketing options for SHFs. There is growing interest in the cob model, but the conditions are not in favour of farmers in terms of added value and access to the empty cob. At a minimum there should be sensitization on those aspects (including economic aspects) in all the cooperatives before the end of the project so that farmers can make a balanced choice. We also suggest conducting specific economic studies on the different models (for farmers but also for buyers) and if needed to lobby for an improvement of the conditions offered to farmers. 165.Recommendation 7 (related to conclusion 4 and 6): In the coming three years, WFP should continue to liaise with value chain actors and the relevant ministries to continue to strengthen and further formalize the maize value chain dialogue platform. The context is very favourable at the moment to strengthen value chain dialogue with the different stakeholders in order to find win-win solutions for an inclusive development of the value chain. Discussion on marketing models (cobs vs grains), access Evaluation Report – LRP Rwanda Endline 64 |P a g e to finance or post-harvest equipment are typical examples of issues that could be tackled jointly because finding sustainable solutions could have a positive impact across the whole chain. Discussions on those issues should continue. In order to do that, there is a link to be made between cooperatives (either through existing Unions like Unicoopagri or, if needed, through support to the emergence of an association of maize cooperatives) in order to ensure that farmers voices are heard and that they can collectively discuss with buyers. 166.Recommendation 8 (related to conclusions 3 and 4): In the coming year, WFP should develop a market-oriented approach for the supply of post-harvest equipment to farmers with interested companies and relevant ministries (commerce and agriculture). The system in place, where everything is imported and distributed by WFP raised farmers interest and engagement, but it is not sustainable. Contacts have been made with several companies and they should be supported to build a business plan either to manufacture in Rwanda or to import the required equipment and distribute this to farmers through agro-dealers. If that turns out not to be possible, then priority should be on working with what is already available locally. 167.Recommendation 9 (related to conclusion 4, 5 and 6) : In the coming year, WFP should conduct a study of the types of crops grown by SHFs on cooperative land and on individual plots to identify potential opportunities for small holder farmer to supply a diversity of products for a school feeding programme (including vegetables). For a second phase of LRP, WFP should define priority actions to improve cropping systems (not through mono-cropping) towards more productivity but also towards improved food security and risk management. This should focus first on crops grown on cooperative land because organizing aggregation and sales will probably be easier and there is already a diversity of products that can be taken into account (including Irish potatoes, beans, maize, vegetables). 168.Recommendation 10: Capacity and time constitute major constraints on the participation of women in cooperative decision making. Moving forward WFP should tailor its intervention strategies with women and cooperatives to take into account these constraints for women and seek to proactively address them. Evaluation Report – LRP Rwanda Endline 65 |P a g e Annexes Annex 1. Terms of Reference 1. Introduction 1. These Terms of Reference (TOR) are for an evaluation (including an impact evaluation) of a World Food Programme’s (WFP) Local and Regional Procurement (LRP) intervention, funded by USDA. The project works through local WFP purchase commitments from small holder farmers within the Homegrown School Feeding (HGSF) program supported by USDA’s McGovern - Dole Food for Education and International Child Nutrition Program (MGD). The program itself runs from April 2017 to March 2019, and it is intended that the evaluation would be undertaken between July 2017 and December 2019. 2. WFP Rwanda is seeking to appoint a contractor to conduct the evaluation of the Local and Regional Procurement program, including the design of the full evaluation and the impact evaluation component, collecting the baseline and end-line, where the endline will be conducted contingent on satisfactory completion of the baseline, as well as analysis and reporting. In order to guarantee the consistency of the approach and the quality and credibility of the data collection and the analysis, the contractor will need to document and record thoroughly the sampling strategy and data collection tools and instruments that will be employed at baseline and endline. 3. These TOR were prepared by the WFP Rwanda Country Office Monitoring and Evaluation team upon an initial document review and consultation with stakeholders and following a standard template. The purpose of the TOR is twofold. Firstly, it provides key information to the evaluation team and helps guide them throughout the evaluation process; and secondly, it provides key information to stakeholders about the proposed evaluation. 2. Reasons for the Exercise 2.1 Rationale 4. The evaluation (including an impact evaluation) is being commissioned for the following reasons: • To develop a program evaluation design that will assess the overall results and impacts of the LRP program against standard performance and results indicators (Annex 3), measuring the changes in outcomes for the target populations (disaggregating results for sub-groups) and the reasons why; • To develop and measure custom livelihood and well-being indicators for the evaluation specific to this program; • To design and implement a rigorous quasi-experimental impact evaluation study design (including well-defined comparison groups, and appropriate sampling strategy) for parts of the program activities. This should be documented and repeated at end-line. • To provide a baseline assessment of the situation on the ground, and to provide baseline values to be used in the final evaluation. 2.2 Objectives Evaluation Report – LRP Rwanda Endline 66 |P a g e 5. Evaluations in WFP serve the dual and mutually reinforcing objectives of accountability and learning. In the context of this evaluation, it is expected that both objectives are given equal importance. • Accountability – Overall,the evaluation will assess and report on the performance and results of the Local Regional Procurement (LRP) project to help WFP present high-quality and credible evidence of actual impact to its donors. • Learning - The evaluation will determine the reasons why certain results occurred or not to draw lessons, derive good practices and pointers for learning. It will provide evidence-based findings to inform operational and strategic decision-making. Findings will be actively disseminated and lessons will be incorporated into relevant lesson sharing systems. 2.3 Stakeholders and Users 6. A number of stakeholders both inside and outside of WFP have interests in the results of the baseline evaluation and some of them will be asked to play a role in the evaluation process. Table 1 below provides a preliminary stakeholder analysis, which should be deepened by the evaluation team in the inception report. 7. Accountability to affected populations, is tied to WFP’s commitments to include beneficiaries as key stakeholders in WFP’s work. As such, WFP is committed to ensuring gender equality and women’s empowerment in the evaluation process, with participation and consultation in the evaluation by women, men, boys and girls from different groups. Table 1: Preliminary Stakeholders’ analysis Stakeholders Interest in the evaluation and likely uses of evaluation report to this stakeholder INTERNAL STAKEHOLDERS Country Office (CO) Rwanda Responsible for the country level planning and operations implementation, it has a direct stake in the evaluation and an interest in learning from experience to inform decision-making. It is also called upon to account internally as well as to its beneficiaries and partners for performance and results of its operation. Regional Bureau (RB) Nairobi Responsible for both oversight of COs and technical guidance and support, the RB management has an interest in an independent/impartial account of the operational performance as well as in learning from the baseline findings to apply this learning to other country offices. The Regional Evaluation Officer supports CO/RB management to ensure quality, credible and useful decentralized evaluations. Office of Evaluation (OEV) OEV has a stake in ensuring that decentralized evaluations deliver quality, credible and useful evaluations respecting provisions for impartiality as well as roles and accountabilities of various decentralised evaluation stakeholders as identified in the evaluation policy. WFP Executive Board (EB) The WFP governing body has an interest in being informed about the effectiveness of WFP operations. This evaluation will not be presented to the EB but its findings may feed into annual syntheses and into corporate learning processes. EXTERNAL STAKEHOLDERS Beneficiaries As the ultimate recipients of food assistance, beneficiaries have a stake in WFP determining whether its assistance is appropriate and effective. As such, the level of participation in the evaluation of women, men, boys and girls from Evaluation Report – LRP Rwanda Endline 67 |P a g e different groups will be determined and their respective perspectives will be sought. More specifically, smallholder farmers, farmers’ organizations, teachers and parent-teacher associations should be considered as key stakeholders. Government The Government has a direct interest in knowing whether WFP activities in the country are aligned with its priorities, harmonised with the action of other partners and meet the expected results. Issues related to capacity development, handover and sustainability will be of particular interest. Key audiences of project partners within the Ministry of Education, the Ministry of Agriculture and Animal Resources, the Ministry of Local Administration, and district officials will be informed of the evaluation results. UN Country team (UNCT) The UNCT’s harmonized action should contribute to the realisation of the government developmental objectives. It has therefore an interest in ensuring that WFP operation is effective in contributing to the UN concerted efforts. Various agencies are also direct partners of WFP at policy and activity level, and particularly work together through the Rwanda OneUN family. NGOs (Adventist Development Relief Agency (ADRA) and World Vision (WVI)) NGOs are WFP’s partners for the implementation of some activities while at the same time having their own interventions. The results of the evaluation might affect programs/projects design, future implementation modalities, strategic orientations and partnerships within local and regional procurement. Donors (USDA) WFP operations are voluntarily funded by a number of donors. They have an interest in knowing whether their funds have been spent efficiently and if WFP’s work has been effective and contributed to their own strategies and programs. USDA is funding the LRP intervention. This evaluation will give USDA a better understanding of the results of their funding. 8. The primary users of this evaluation will be: • The WFP Rwanda country office and its partners in decision-making, notably related to program implementation and/or design, Country Strategy and partnerships as well as further fundraising. • This evaluation will contribute to the body of knowledge on the MGD program. USDA, as the funder of the evaluation, will use findings and lessons learned to inform program funding, design, and implementation decisions. • Given the core functions of the Regional Bureau (RB), the RB is expected to use the evaluation findings to provide strategic guidance, program support, and oversight to other COs in the region. • WFP HQ may use evaluations for wider organizational learning and accountability. • Office of Evaluations (OEV) may use the evaluation findings, as appropriate, to feed into evaluation syntheses as well as for annual reporting to the Executive Board. 3. Context and subject of the evaluation 3.1 Context 9. Rwanda is a low-income, developing country with a ranking of 161 out of 187 countries based on the 2016 United Nations Human Development Report. Rwanda has one of the highest population densities in Africa, with 416 people living per square kilometre. The total population of 11.5 million people is growing at an annual rate of 2.39 percent. Since the 1994 genocide, the country has been rebuilding itself and improving the population's quality of life. Under the Vision 2020 program, Rwanda plans to increase its per capita income from USD 644 to USD 1,240 by 2020, and increase life expectancy to 66 years from 49 years in 2000. Rwanda has seen an impressive annual Gross Domestic Product (GDP) growth rate of 7.2 percent since 2010 alongside decreasing income inequality. Agriculture continues to play a key role in the economy, contributing 33 percent of the national GDP, generating 80 percent of export revenue, and Evaluation Report – LRP Rwanda Endline 68 |P a g e accounting for more than 71.6 percent of the labour force. According to the World Bank, Rwanda has maintained steady economic growth recently, and is expected to continue doing so in the short term, with projections to exceed 2017 regional growth rates. 10. Household food insecurity remains a major challenge, affecting 20 percent of Rwandan households according to the 2015 Comprehensive Food Security and Vulnerability Analysis (CFSVA). Household access to food is constrained by poverty, topography and scarcity of land, low productivity, inadequate employment opportunities, high food prices, recurrent climate-related shocks, and conflicts in the neighbouring countries. Food insecurity is most prevalent in rural areas bordering Lake Kivu and along the Congo Nile Crest, where soils are less fertile and land is more susceptible to erosion. Almost half the population are vulnerable to food insecurity caused by drought, particularly in eastern areas, while other areas of the country are vulnerable to flooding and landslides. 11. The McGovern Dole (MGD) funded Home Grown School Feeding (HGSF) program, launched in July 2016, aims to provide 15 million meals per year to 83,000 students in 104 primary schools over five years. In line with USDA and Government of Rwanda’s (GoR) priorities, the most vulnerable and food insecure districts in Rwanda, namely Nyamagabe, Nyaraguru (southern province), and Rutsiro and Karongi (western province) are supported by the HGSF program. The project is designed to support the GoR’s nascent school feeding program, financed and managed by the government, which envisages school meals based on local purchase of commodities with a view to eventual nationwide implementation without external support. The HGSF program uses a combination of both MGD in‐kind resources and locally procured commodities. 12. To align with the MGD-funded HGSF program, USDA’s 2016 Local and Regional Procurement (LRP) project will support students, smallholder farmers and communities from the most economically vulnerable sectors for two years where 1.36 million USD will be used to purchase food, 127,000 USD will be used for capacity augmentation, and 370,000 USD for direct support costs34 This will undoubtedly be beneficial to the ongoing HGSF program, both in the provision of essential commodities, and in setting up a durable model to potentially supply schools with food from farmers based in the communities where the schools are located. 13. The main beneficiaries of the LRP project include vulnerable smallholder farmers (SHFs), most of whom will be located in the MGD-supported districts of Nyamagabe and Nyaruguru, which have some of the highest poverty levels in the country (64 percent and 77 percent).35 The other beneficiaries are the students in the HGSF schools, who will be recveiving meals made from ingredients bought from the smallholder farmers. By strengthening farmer’s cooperatives, facilitating access to loans and markets, and enhancing their capacities to increase yields and limit commodity losses, the proposed project will not only allow for the effective provision of locally sourced commodities to the school feeding program and further contribute to its sustainability, it will also increase economic resilience and improve food security among target beneficiaries through a holistic mix of innovative approaches. 14. WFP has been supporting smallholder farmers and private sector development since 2009, through its Purchase for Progress (P4P) program, integral to the government’s HGSF local purchase approach. Building upon successes of P4P, WFP Rwanda started another initiative to increase market access for smallholder farmers since late 2015 in closer collaboration with private-sector value chain stakeholders. Under the pilot project “Patient Procurement Platform (PPP),” WFP, in collaboration with the Alliance for Green Revolution for Africa (AGRA) and their local partners, facilitated smallholder farmer participation across 34 Food purchases will cost around $1,368,326.40, capacity augmentation will cost $127,000, and direct support costs $373,831.60. 35 MINAGRI, NISR and WFP. (2016). Comprehensive food security and vulnerability analysis. Kigali. Evaluation Report – LRP Rwanda Endline 69 |P a g e the entire value chain. Today, PPP has become the Farm to Market Alliance (FTMA) that engages smallholder farmers through multi-year commitments in the form of ‘patient’ buyer contracts. WFP Rwanda, through the FTMA, provides access to markets, and finance and post-harvest handling services among other services. FTMA has so far linked 81 cooperatives composed of approx.. 22,000 farmers (49% women) to‘off-takers who agreed to pre-planting forward delivery contracts. Cooperatives have committed to supplying a total of 5,287 MT of maize during the post-harvest period of March-May 2017, up from the 2,700 MT delivered in the first harvest season from 2016. Through the P4P, PPP and MGD￾funded HGSF programs, WFP already works closely with the Ministry of Agriculture and Animal Ressources (MINAGRI), Ministry of Education (MINEDUC), and Ministry of Local Government (MINALOC). Strong partnerships with the Ministries of Trade and Industry, Finance and Economic Planning, Gender and Family Promotion as well as the provincial governors, are also integrated into all aspects of programming. This ensures that the project is fully aligned with existing national development strategies. The collaboration between GoR and WFP will be essential to support the local production of the USDA‐requested commodities and the development of robust national frameworks, further strengthened through the complementary LRP grant, enabling Rwanda to sustain the benefits of USDA support beyond the life of the projects. 3.2 Subject of the evaluation and activities undertaken by the LRP program 15. Under the LRP project, WFP will procure commodities from smallholder farmers for use in the WFP Rwanda’s Homegrown School Feeding (HGSF) program. The funds will also be used to build capacity for smallholder farmers through trainings on how to reduce post-harvest losses, warehouse management, organizational structure and management, agricultural markets, business planning, microfinancing, and methods to increase the quality of production in order to meet food safety and quality standards. 16. Specifically, the USDA’s LRP funds will be used by WFP to carry out five key activities: 1) Purchase Commodities from Smallholder Farmers; 2) Connect Smallholder Farmers (SHF) to New Markets; 3) Connect Farmers to the Patient Procurement Platform; 4) Build Capacity for Smallholder Farmers, and 5) Collaborate with the Government of Rwanda. 17. The first activity will enable WFP to buy beans and maize meal directly from smallholder farmers (SHFs) or pro-smallholder millers; the second activity will promote procurement from SHFs since WFP will be able to buy only from suppliers that hold 50% of grains procured from SHFs – where SHF has documentation proof of this in order to be readily evaluated by the evaluation team; the third activity will facilitate forward delivery contracts between private-sector off-takers and market-ready farmers’ organizations; the fourth activity will offer training to four cooperatives on the procurement of raw materials, processing, business management and marketing and to other 16 cooperatives on post-harvest handling and storage, warehouse management and organizational strenghtening and the fifth activity will enable WFP to strongly advocate for the advancement of SHFs through their integration into the agricultural sector working group and other forums. 18. The LRP will support approximately 45,000 beneficiaries in 20 targeted cooperatives and 49 schools in the Southern and Eastern provinces. The grant will enable WFP to create synergies with other programming areas. As WFP uses the funds to purchase food commmodities that are used in WFP Rwanda’s Homegrown School Feeding (HGSF) program local school students will also benefit, through access to the HGSF. Table 2: Target Numbers per Year Student Beneficiaries Farmer Beneficiaries Total Evaluation Report – LRP Rwanda Endline 70 |P a g e Females Males Females Males Year One 20,855 19,251 1,568 1,632 43,306 Year Two 43,306 2,496 2,637 45,239 4. Evaluation Approach 4.1 Scope 19. The LRP evaluation will focus on the activities of the LRP program designed for purchasing food commodities used in the McGovern Dole-supported Homegrown School Feeding (HGSF) program, from smallholder farmers through WFP’s Purchase for Progress (P4P) program. 20. Twenty cooperatives will be targeted in Nyamagabe and Nyaruguru and other relevant districts. The selection of cooperatives considers to the extent possible the region where the schools that will be utilizing the food procured under this project are located, particularly in Nyamagabe and Nyaruguru Districts. However, farmer groups different from those under the McGovern-Dole will be supported to avoid concentration of assistance to same groups of beneficiaries for efficient use of USDA funds. Other cooperatives in the South and East are targeted according to their potential of production and commercialization of food commodities required by the project. Each farmer group will be assessed and additional assistance will take into account actual needs of the group in line with the expected results of the project. 21. The full evaluation will cover all five activities of the LRP program. It is expected that the quasi￾experimental design can only cover activity 1 (Purchase Commodities form Smallholder Farmers) and 2 (Connect Smallholder Farmers to New Markets) of the LRP program due to the nature of the activities and timing (see list of activities in Annex 2). However, the evaluation team is expected to propose a detailed design at the inception stage. 22. The evaluation should start with an inception phase of 1-2 months, which will also enable familiarity to USDA LRP standard indicators, and development of data collection instruments. The baseline data collection should be undertaken in August 2017. The baseline data and analysis and report writing is expected to be concluded in November 2017. The endline data collection should take place between Augusts and September 2019. 23. A qualified, independent, third-party agency will be contracted to develop the full evaluation design, including a quasi-experimental component for some of the activities, and undertake a data collection, analysis and write a comprehensive baseline and endline reports. It is expected that the following are done in the different phases of the evaluation: a. During Inception, the evaluation team should: i. confirm and define the evaluation questions and sub-questions. ii. develop and thoroughly document the evaluation design (including how methods are mixed or combined), a sampling strategy, data collection tools and instruments, and code the units. The evaluation design should include a quasi-experimental component, and therefore the evaluation team should define an appropriate counterfactual and comparison groups. iii. confirm which monitoring data is being collected by the WFP Rwanda office to avoid duplication. iv. submit a full evaluation matrix (that links methods and data collection strategy to each of the evaluation questions) to WFP as part of the inception report. Evaluation Report – LRP Rwanda Endline 71 |P a g e b. During Baseline phase, the evaluation team should establish indicator baseline information and to verify the targets established in the project as part of the baseline report. c. The final product of the evaluation is a comprehensive end-line report, which should analyse the end-line data against the baseline and respond to the specified evaluation questions, using the methods identified during inception. 4.2 Evaluation Questions and Criteria 24. The evaluation will address all five OECD-DAC criteria (relevance, effectiveness, efficiency, impact and sustainability), as per USDA Monitoring and Evaluation policy. The evaluation will address the following key questions, which will be further developed by the evaluation team during the inception phase. Collectively, the questions aim at highlighting the key lessons and performance of the Local Regional Procurement (LRP) through WFP Rwanda, which could inform future strategic and operational decisions: a. Activity 1 (Purchase Commodities form Smallholder Farmers) and 2 (Connect Smallholder Farmers to New Markets): “how does the LRP program affect smallholder farmers’ behaviours (for example, long-term investments in productive assets, education, etc.) and households’ well-being indicators (for example, assets, income, health and nutrition status)?” b. Activity 3 (Connect Farmers to the Farm to Market Alliance) and 4 (Build Capacity of Smallholder Farmers): “what are the effects of the project on the way the targeted cooperatives position themselves in their respective value-chain?” c. Activity 5 (Collaborate with the Government of Rwanda): “how did the changes in the institutional framework create a more favourable environment for smallholder farmers?” 25. Corollary questions are: a. How and to what extent does the LRP program affect the cost-effectiveness of food assistance, procurement, delivery, and distribution? b. How and to what extent does the LRP program affect the timeliness of food assistance procurement, distribution, and delivery? c. How and to what extent does the LRP program contribute to improved utilization of nutritious and culturally acceptable food that meet quality standards? d. What internal and/or external factors affected the project’s achievement of intended results? e. Is there emerging evidence that the linkages between smallholder farmers and schools will endure? f. What is the level of participation of men and women? Are women well represented, including in leadership positions? What are the disaggregated impacts on women? 26. Table 3 below presents key evaluation criteria and corresponding questions: Table 3 – evaluation criteria and questions Criteria Key question Relevance • To what extent was the design of the intervention relevant to wider context, and aligned with needs of the most vulnerable groups, the government and WFP partners? Evaluation Report – LRP Rwanda Endline 72 |P a g e Effectiveness • How and to what extent does the LRP program contribute to improved utilization of nutritious and culturally acceptable food that meet quality standards? • What internal and/or external factors affected the project’s achievement of intended results? • What is the level of participation of men and women? Are women well represented, including in leadership positions? What are the disaggregated impacts on women? • how did the changes in the institutional framework create a more favourable environment for smallholder farmers? Efficiency • How and to what extent does the LRP program affect the cost-effectiveness of food assistance, procurement, delivery, and distribution? • How and to what extent does the LRP program affect the timeliness of food assistance procurement, distribution, and delivery? Impact • how does the LRP program affect smallholder farmers’ behaviours (for example, long-term investments in productive assets, education, etc.) and households’ well￾being indicators (for example, assets, income, health and nutrition status)? • what are the effects of the project on the way the targeted cooperatives position themselves in their respective value-chain? Sustainability • Is there emerging evidence that the linkages between smallholder farmers and schools will endure? 4.3 Data Availability 27. There is limited secondary data available and it is expected that the evaluation team will develop comprehensive survey instruments to collect primary data for the baseline and end-line. 28. WFP Rwanda Country Office’s M&E staff will routinely collect data on LRP standard indicators throughout the duration of the program. The data will be made available to the evaluation team for the endline assessment, and the team should clarify during inception which indicators are measured by the WFP Rwanda Office to avoid duplication. 29. The evaluation team should gather data from cooperatives and government institutions. The availability and quality of such data cannot be assured by WFP. The team is expected to formulate a strategy to collect such information and check its reliability, as such, a data quality assurance plan will be included as a a deliverable. The strategy has to be documented for future reference. 30. Concerning the quality of data and information, the evaluation team should systematically check accuracy, consistency and validity of collected data and information and acknowledge any limitations/caveats in drawing conclusions using the data. 4.4 Methodology 31. The independent evaluation team is responsible for developing the full methodology during the inception phase. The team should identify potential risks of the approach and mitigation measures. The following should be considered and included by the evaluation team: Evaluation Report – LRP Rwanda Endline 73 |P a g e • Firstly, confirm and define specific evaluation questions that are answered, and record them in the WFP Evaluation Matrix. • Develop and agree an appropriate evaluation design, including appropriate counterfactual for some of the activities, so that there can be attribution of impacts to the LRP program. • Identify an appropriate sample for the treatment and comparison group (estimated sample size: approx. 300 households per group). However, the evaluation team will be responsible for conducting sample size calculations and determining a sample size large enough to answer the primary evaluation questions. • Design credible survey instruments to collect household survey data as part of the baseline and again repeated for the end-line. The survey design should take account of any seasonal variation (therefore needs to be collected during the same season), and incorporate a range of appropriate well-being and behavioral change indicators (including but are not limited to, income, asset accumulation/holding, education and health and nutrition status). • Use mixed methods in the evaluation design and data collection (including quantitative and qualitative) to ensure a comprehensive design, and the reasons for the changes in indicators can be explained. This can include triangulation of information through a variety of means, or different evaluation questions being answered through different methods and types of data. The use of mixed methods should be documented in the inception report. • WFP anticipates that the consultants will recommend a methodology and will likely include carrying out key informant interviews and focus group discussions. The qualitative data collection will gather information on gender equality, capacity of cooperatives and changes in the institutional context. However, bidding companies should also propose a wider variety of methods (including, but not limited to most significant change, outcome harvesting, etc.) whenever they feel these could be useful in enriching the evaluation products. • Ensure the evaluation design takes into account ways to ensure that the voices of women, girls, men and boys are heard and documented; • Ensure the methodology and evaluation implementation are ethical and conform to the UNEG Ethical Guidelines for Evaluation. 4.5 Quality Assurance and Quality Assessment 32. WFP’s Decentralized Evaluation Quality Assurance System (DEQAS) defines the quality standards expected from this evaluation and sets out processes with in-built steps for Quality Assurance, Templates for evaluation products and Checklists for their review. DEQAS is closely aligned to the WFP’s evaluation quality assurance system (EQAS) and is based on the UNEG norms and standards and good practice of the international evaluation community and aims to ensure that the evaluation process and products conform to best practice. 33. DEQAS will be systematically applied to this evaluation. The WFP Evaluation Manager will be responsible for ensuring that the evaluation progresses as per the DEQAS Process Guide and for conducting a rigorous quality control of the evaluation products ahead of their finalization. 34. WFP has developed a set of Quality Assurance Checklists for its decentralized evaluations. This includes Checklists for feedback on quality for each of the evaluation products. The relevant Checklist will be applied at each stage, to ensure the quality of the evaluation process and outputs. 35. To enhance the quality and credibility of this evaluation, an outsourced quality support (QS) service directly managed by WFP’s Office of Evaluation in Headquarter provides review of the draft inception and evaluation report (in addition to the same provided on draft TOR), and provide: Evaluation Report – LRP Rwanda Endline 74 |P a g e a. systematic feedback from an evaluation perspective, on the quality of the draft inception and evaluation report; b. recommendations on how to improve the quality of the final inception/evaluation report. 36. The evaluation manager will review the feedback and recommendations from QS and share with the team leader, who is expected to use them to finalise the inception/ evaluation report. To ensure transparency and credibility of the process in line with the UNEG norms and standards[1], a rationale should be provided for any recommendations that the team does not take into account when finalising the report. 37. This quality assurance process as outline above does not interfere with the views and independence of the evaluation team, but ensures the report provides the necessary evidence in a clear and convincing way and draws its conclusions on that basis. 38. The evaluation team will be required to ensure the quality of data (validity, consistency and accuracy) throughout the analytical and reporting phases. The evaluation team should be assured of the accessibility of all relevant documentation within the provisions of the directive on disclosure of information. This is available in WFP’s Directive CP2010/001 on Information Disclosure. 39. All final evaluation reports will be subjected to a post hoc quality assessment by an independent entity through a process that is managed by OEV. The overall rating category of the reports will be made public alongside the evaluation reports. 5 Structure of the evaluation 40. The evaluation will be conducted in two stages: a baseline assessment to be conducted between June and November 2017 and an endline evaluation that will take place between August and December 2019. Although the two stages are interconnected steps of the same evaluative exercise, their objectives are slightly different as outlined in the following sections. 5.1 Baseline Assessment 41. The objectives of the baseline assessment are to establish the methodological approach for the entire evaluation, measure baseline values and provide a situational analysis before the start of the intervention. 42. The evaluation firm selected for this assignment will develop the methodological approach following the indications provided in 4.2 Evaluation Questions and 4.4 Methodology. The evaluators should also validate or revise the assumptions and risk analysis underlying the project design. 43. The main deliverables of the baseline assessment are the following: • Inception report. It must be written following WFP recommended template. The evaluators must confirm the final evaluation questions, which approach and methods are chosen, and how they are going to be implemented in practice, and used to answer the IE questions. This means setting out a full study design including what data is being collected and for what purpose, how sampling is done (to be determined by the evaluation team), how the data is being analysed and triangulated. The inception report must also include how the data has been quality assured, and how the evaluators will manage and safeguard ethics during the life of the evaluation. Annexed to the inception report, the evaluation team should include a detailed work plan, including , timeline and activities. [1] UNEG Norm #7 states “that transparency is an essential element that establishes trust and builds confidence, enhances stakeholder ownership and increases public accountability” Evaluation Report – LRP Rwanda Endline 75 |P a g e • Baseline report, including a first draft, where the final approach, methodology and data collection tools are clearly recorded, including their limitations and mitigations measures. The report must record all standard and custom indicator baseline values. • Clean data sets, including quantitative data sets in Excel, statistical software code, and transcripts and/or notes from focus group discussions and key informant interviews. Table 4 - Baseline Report Outline: 1. Executive Summary i. Methodology ii. Baseline values of key indicators 2. Introduction iii. Overview of the Evaluation Subject iv. Context v. Evaluation Methodology and Limitations 3. Situational Anlysis 4. Baseline values vi. Relevance vii. Main Evaluation Question 1 viii. Main Evaluation Question 2 ix. Main Evaluation Question 3 x. Corollary Evaluation Questions 5. Additional findings 6. Conclusions and recommendations 7. Annexes Table 5: Baseline Deliverables Dates Deliverables July 2017 • Desk review of key project documents • Create a data quality assurance plan • Confirm and finalise evaluation questions and evaluation design and methodology (including sampling strategy), and draft an inception report for agreement • Finalize inception report with the inclusion of Evaluation Reference Group’s comments • Data collection instruments • Arrange field visits August – September 2017 • Conduct field visits • Conduct baseline survey • Conduct key stakeholder focus groups and key informant interviews and collect data with other suggested methods • Enter, clean, and analyze data Evaluation Report – LRP Rwanda Endline 76 |P a g e October 2017 • Draft baseline report • Seek Evaluation Reference Group’s comments on the draft baseline report • Finalize baseline report • Present baseline findings 5.2 Endline evaluation 44. The objective of the final evaluation is to provide an evidence-based, independent assessment of the performance of the project in order to evaluate the project’s success, ensure accountability, and generate lessons learned. Specifically, the final evaluation will: (1) use the same methodology developed for the baseline to measure key indicators, (2) analyse data to compare results between treatment and control groups where a quasi-experimental design is possible, (3) analyse data to compare results before and after the intervention for the activities where it is not possible to design a quasi-experimental design, and (4) identify meaningful lessons learned that WFP, USDA, and other relevant stakeholders can apply to future programming. 45. WFP anticipates carrying out the final evaluation during the final year of the USDA-LRP grant between August and November 2019. 46. The main deliverables of the endline are the following: • Endline report, including a first draft, using WFP recommended template. It must set out a detailed methodology section, study design, and any limitations or where the study design was compromised. Should detail how data was collected, validated and analysed, and how conclusions were drawn. How different types of methods were brought together in the analysis. Annexes to the final report include but are not limited to a copy of the final ToR, bibliography, list of sampled farmer organizations, detailed sampling methodology, maps, a list of all meetings and participants, final survey instruments, transcripts from key informant interviews, focus group discussions, table of all standard and custom indicator with baseline and endline values, list of supported schools. • Clean data sets, including quantitative data sets in Excel, statistical software code, and transcripts and/or notes from focus group discussions and key informant interviews. • Powerpoint presentation of main findings and conclusions for de-briefing and dissemination purposes. • 2-page brief containing min findings, conclusions and recommendations. Table 6: Final Evaluation Deliverables Dates Deliverables August – September 2019 • Updated methodology and sampling plan • Data collection instruments • Conduct field visits • Conduct endline survey • Conduct key stakeholder focus groups and key informant interviews and collect data with other suggested methods • Enter, clean, and analyze data October 2019 • Draft endline report • Seek Evaluation Reference Group’s comments on the draft endline report Evaluation Report – LRP Rwanda Endline 77 |P a g e • Finalize endline report November 2019 • Share a powerpoint presentation and a 2-page brief with key stakeholders • Conduct workshop to share evaluation findings with key stakeholders 6 Organization of the Evaluation 6.1 Evaluation Conduct 47. The evaluation team will conduct the evaluation under the direction of its team leader and in close communication with WFP Rwanda M&E Officers, the Head of Programmes, and the Country Director. The team will be hired following agreement with WFP on its composition. 48. The evaluation team will not have been involved in the design or implementation of the subject of evaluation or have any other conflicts of interest. Further, they will act impartially and respect the code of conduct of the evaluation profession. 6.2 Team composition and competencies 49. The Team Leader should be a senior evaluator with at least 10 years of experience in research, evaluation and or baselines with demonstrated expertise in managing multidisciplinary and mixed quantitative and qualitative method evaluations, complemented with experience in implementing evaluations with a quasi￾experimental designs and additional significant experience in other development and management positions. In addition, the team leader should also have prior experience evaluating school meals programs, or agricultural development programs. 50. The Team leader will also have expertise in designing methodology and data collection tools and demonstrated experience in leading similar baselines or evaluations. She/he will also have leadership and communication skills, including a track record of excellent writing and presentation skills. Her/his primary responsibilities will be: i) defining the baseline approach and methodology; ii) guiding and managing the team; iii) leading the baseline mission and representing the baseline team; iv) drafting and revising, as required, the inception report, the end of field work i.e. (exit) debriefing presentation and baseline report. 51. The team must include strong demonstrated knowledge of qualitative and quantitative data and statistical analysis will be required. It should include both women and men and at least one team member should be familiar with WFP’s operations (preferably P4P or FTMA). 52. The team will be multi-disciplinary and include members who together include an appropriate balance of expertise and practical knowledge in the following areas: • Local purchase/procurement. • Capacity development (focus on smallholder farmers). • Post harvest handling and agriculture supply chains. • Food security. • Gender expertise / good knowledge of gender issues. • All team members should have strong analytical and communication skills, evaluation experience and familiarity with Rwanda. Evaluation Report – LRP Rwanda Endline 78 |P a g e • The team should have knowledge of English, French and Kinyarwanda. The expected language of the evaluation report is English. 53. Team members will bring together a complementary combination of the technical expertise required and have a track record of written work on similar assignments. 54. Team members will: i) contribute to the methodology in their area of expertise based on a document review; ii) conduct field work; iii) participate in team meetings and meetings with stakeholders; iv) contribute to the drafting and revision of the evaluation products in their technical area(s). 6.3 Security Considerations 55. Security clearance where required is to be obtained from Rwanda’s United Nations Department of Safety and Security (UNDSS) office in Kigali. • As an ‘independent supplier’ of evaluation services to WFP, the evaluation company is responsible for ensuring the security of all persons contracted, including adequate arrangements for evacuation for medical or situational reasons. The consultants contracted by the evaluation company do not fall under the UN Department of Safety & Security (UNDSS) system for UN personnel. • Consultants hired independently are covered by the UN Department of Safety & Security (UNDSS) system for UN personnel which cover WFP staff and consultants contracted directly by WFP. Independent consultants must obtain UNDSS security clearance for travelling to be obtained from designated duty station and complete the UN system’s Basic and Advance Security in the Field courses in advance, print out their certificates and take them with them.36 56. However, to avoid any security incidents, the Evaluation Manager is requested to ensure that: • The WFP CO registers the team members with the Security Officer on arrival in country and arranges a security briefing for them to gain an understanding of the security situation on the ground. • The team members observe applicable UN security rules and regulations – e.g. curfews etc. 7 Roles and Responsibilities of Stakeholders The Rwanda Country office: 57. The WFP Rwanda Country Office management (Director or Deputy Director) will take responsibility to: o Assign an Evaluation Manager for the baseline. o Compose the internal evaluation committee and the evaluation reference group (see below). o Approve the final TOR, inception and evaluation reports. o Ensure the independence and impartiality of the evaluation at all stages, including establishment of an Evaluation Committee and of a Reference Group. o Participate in discussions with the evaluation team on the evaluation design and the evaluation subject, its performance and results with the Evaluation Manager and the evaluation team. o Organise and participate in two separate debriefings, one internal and one with external stakeholders. o Oversee dissemination and follow-up processes, including the preparation of a Management Response to the evaluation recommendations. 58. The Evaluation Manager: 36 Field Courses: Basic; Advanced Evaluation Report – LRP Rwanda Endline 79 |P a g e o Manages the evaluation process through all phases including drafting this TOR o Ensures quality assurance mechanisms are operational o Consolidates and shares comments on draft TOR, inception and evaluation reports with the evaluation team o Ensures expected use of quality assurance mechanisms (checklists, quality support) o Ensures that the team has access to all documentation and information necessary to the evaluation; facilitates the team’s contacts with local stakeholders; sets up meetings, field visits; provides logistic support during the fieldwork; and arranges for interpretation, if required o Organises security briefings for the evaluation team and provides any materials as required 59. An Internal Evaluation Committee is ensuring the independence and impartiality of the exercise. The membership includes baseline manager, technical unit in charge of school feeding program, head of Local Procurement unit and Head of Programme. The key roles and responsibilities of this team, includes providing input to the evaluation process and commenting on evaluation products. 60. A baseline reference group with representation from USDA, WFP Regional Bureau and WFP Country office will review the evaluation products as further safeguard against bias and influence. 61. Independent evaluation team: under the leadership of the evaluation team leader, the evaluation team will be responsible for undertaking the evaluation, as per this TOR, independently. The evaluation team will select and interview staff from the Country Office. The team will also have contact with CO staff who are members of the ERG during inception and dissemination. The CO staff who are members of the ERG will be required to provide comments on the evaluation products. The responsibilities of the evaluation manager are clearly stated above and will, in addition to other provisions for impartiality already put in place, ensure the evaluation is implemented as per the WFP decentralized evaluation quality assurance system. Any support e.g. logistical support, that will be required from by the evaluation team from the CO will be discussed with baseline manager who will in turn follow up and organize with CO. 62. United States Department of Agriculture (USDA): In addition to participating in the baseline reference group, USDA will provide comment on the TOR, inception report, draft evaluation report and participate in a stakeholder interviews with the selected evaluation team prior to the start of fieldwork 63. The Regional Bureau: the RB will take responsibility to: o Advise the Evaluation Manager and provide support to the evaluation process where appropriate. o Support in the formulation of the Terms of Reference o Participate in discussions with the evaluation team on the evaluation design and on the evaluation subject as relevant, as required. o Provide comments on the draft TOR, Inception and Evaluation reports o Support the Management Response to the evaluation and track the implementation of the recommendations. While the Regional Evaluation Officer will perform most of the above responsibilities, other RB relevant technical staff may participate in the evaluation reference group and/or comment on evaluation products as appropriate. 64. Other Stakeholders (Government, NGOs, and UN agencies) will be identified for interviews by the evaluation team, which will be based on the preliminary stakeholder analysis detailed as follows: the Ministry of Agriculture and Animal Resources, Ministry of Education, Ministry of Local Administration, district executive committees, and head teachers. The following stakeholders will be targeted for focus Evaluation Report – LRP Rwanda Endline 80 |P a g e group discussions: Parent teacher associations (PTAs), farmers’ groups, students, and community members. 65. The Office of Evaluation (OEV). OEV, through the Regional Evaluation Officer, will advise the Evaluation Manager and provide support to the evaluation process when required. It is responsible for providing access to the outsourced quality support service reviewing draft ToR, inception and evaluation reports from an evaluation perspective. It also ensures a help desk function upon request. 8 Communication and budget 8.1 Communication 66. To ensure a smooth and efficient process and enhance the learning from this evaluation, the evaluation team should place emphasis on transparent and open communication with key stakeholders. These will be achieved by ensuring a clear agreement on channels and frequency of communication with and between key stakeholders. 67. Communication with baseline team and stakeholders should go through the baseline manager. 68. WFP Rwanda Country Office will organize a workshop to discuss evaluation findings and recommendations, where the evaluation team will present the key findings. 69. As part of the international standards for evaluation, WFP requires that all evaluations are made publicly available. Following the approval of the final evaluation report, the report will be shared publically on WFP’s website, and all external stakeholders will be notified of its availability. 8.2 Budget 70. For the purpose of this evaluation, the budget will be disbursed against the high quality and timely delivery of key products inception report, baseline report and end-line report. 71. Procurement using Long-term Agreements (sometimes called “service level agreement”), based on pre￾agreed rates. Table 7: Proposed Evaluation Budget Evaluation Activity Estimated Date Approximate Cost Impact Evaluation Contract June 2017 – November 2019 USD 175,000 Evaluation Report – LRP Rwanda Endline 81 |P a g e Annex 2. LRP Project activities and targets Planned project activities Target Activity 1: Purchase commodities from SHF Purchase of beans and maize from SHF in Rwanda or other regional markets (e.g. Tanzania or Uganda) 418 metric tons of maize and 225 metric tons of beans Activity 2: Build capacity of SHF Train in procurement of raw materials, processing, business management and marketing 4 cooperatives Facilitate trading and marketing relationships between milling cooperatives, factories and 16 other cooperatives for raw materials 16 cooperatives Train cooperatives on post-harvest handling and storage, warehouse management, organizational governance, agricultural markets, business planning, microfinance, and techniques for increasing production, etc. 16 cooperatives Coach cooperatives on business plan implementation 16 cooperatives Supply medium sized storage facilities and provide training and technical field support 1,000 SHF Activity 3: Connect farmers to the patient procurement platform Promote market access for SHF through facilitation of forward delivery contracts between private sector off-takers and farmers’ organizations No target Activity 4: Connect SHF to new markets Promote pro-smallholder procurement and widen market for SHF by purchasing maize meal and beans through buyers such as MINIMEX (miller), EAX, SARURA and RGCC No target Collaborate with superintendent agencies for food inspection and quality testing No target Activity 5: Collaborate with the government of Rwanda Advocate through agricultural sector working group to advance SHF integration and to advance markets for SHF and institutional procurement No target Draft a strategy for sustainable market access for SHF and to increase procurement by national traders and institutional suppliers One strategy Figure 5: location of FTMA cooperatives in the four targeted districts (WFP) Evaluation Report – LRP Rwanda Endline 82 |P a g e Annex 3. Detailed Theory of Change for the LRP Project Assumptions, linked to each of the numbers in the diagram, are as follows: Assumption 1: Big buyers are interested in improving their access to sellers Assumption 2: There is a demand (other than at WFP level) for quality products Assumption 3: Producing quality maize and beans is profitable (both in term of land productivity and labour use) for SHF Assumption 4: Dialogue structures between partners in agriculture will remain strong Assumption 5: WFP is seen as a credible facilitator of linkages by all parties concerned Assumption 6: Other actors in the value chain (credit agencies etc.) continue to invest in agriculture Assumption 7: SHF constraints are linked to the kind of inputs that cooperatives and other value chain actors can provide Assumption 8: Other value chain actors (middlemen for example) will not implement strategies to guarantee their supply of maize and beans Assumption 9: There is no severe drought, pest attacks or disease affecting severely production. Assumption 10: The social, economic and politic context remains stable Endline Report - August 2019 (draft) | P a g e 83 Annex 4. Evaluation Matrix Evaluation question Sub-Question Indicator Main sources of information Method for data collection Method for data analysis Quality of available data Relevance 1-To what extent was the design of the intervention: relevant to wider context; aligned with the needs of the most vulnerable groups; cognizant of the needs of male and female beneficiaries; and in line with priorities of the government and WFP partners? 1.1-To what extent is LRP’s location, priorities and beneficiary selection in line with the needs of the most vulnerable groups? Integration of beneficiaries characteristics, activities and needs in the design and activities of LRP Integration of specificities of districts and agro-ecological regions in LRP’s design and activities Attention to the different needs and situation of male and female beneficiaries LRP project related documents (reports, studies…) SHF, cooperatives and other value chain actors Background documents on the context of agriculture in the concerned districts Document review (needs assessment reports, LRP documents and background project design documents, reports of beneficiaries engagement meetings) FGD with cooperatives and SHF Descriptive and inferential analysis of survey data. Triangulation of documents and results from FGD and key informant interview Very little project related documentation available Documentation from other projects and policy documents have been collected WFP to provide additional documentation when available. Information from stakeholders will be collected in the field Quality of the information l largely depends on whether WFP has document presenting their strategy in terms of project design 1.2-To what extent does the LRP project contribute to the government’s objectives? Alignment of the approach and expected results with government’s objectives and legislative framework Evidence that the LRP intervention has made a specific contribution to the implementation of the GoR objectives LRP project related planning and implementation documents (reports, studies…) Policy documents Documents of other partners implementing similar initiatives Institutional stakeholders (MINAGRI, MINEDUC, MINALOC, MINEACOM) at national and district level Document review (LRP documents, policies and strategies documents from the government, project design document, Ministry of Agriculture and Animal Resources strategic plan) KII with institutional stakeholders (national and district) Triangulation of documents and key informant interview Contribution analysis Documentation from other projects and policy documents have been collected Further documents on the LRP planning and implementation to be provided to the evaluation team by WFP Information from stakeholders will be collected in the field Quality of policy documents is expected to be good 1.3-What is the level of complementarity with other development Evidence of complementarity of LRP’s approach with other LRP project related documents (reports, studies…) Document review (LRP project design documents and documents from other projects Descriptive and inferential analysis of survey data. Information from stakeholders will be collected in the field Endline Report - August 2019 (draft) | P a g e 84 Evaluation question Sub-Question Indicator Main sources of information Method for data collection Method for data analysis Quality of available data interventions targeting the same regions, including with WFPs own interventions? development projects at the design stage Evidence that LRP’s approach does not duplicate the work by other actors Level of implication of partners in the design and implementation of LRP Documents from other projects, partners and donors Institutional stakeholders (MINAGRI, MINEDUC, MINALOC, MINEACOM) at national and district level, with other development actors, and with WFP and donors, coordination meetings reports of development projects) KII with institutional stakeholders (national and district) including with a view to establish what has changed in the overall context over time and to what extent this can be attributed to the LRPs interventions Triangulation of documents and results from key informant interview Mapping of interventions in the LRP area for complementarity Contribution analysis Information on interventions by other partners to be collected by the team with support from WFP Further documents on the LRP planning and implementation to provide to the evaluation team by WFP, in particular on foreseen complementarity with other actors Quality of the information l largely depends on whether WFP and other development interventions have document presenting their strategy in terms of project design 1.4-Does the logic of intervention promote sustainability? Existence of a clear strategy for sustainability and exit strategy Approach and activities of LRP based on market related solutions (better business linkages between chain actors and cooperatives in order to improve access to services and market opportunities for SHF) Awareness at government level (both national and district level) of SHF and pro-SHF buyers LRP project related documents (reports, studies…) SHF, cooperatives and other value chain actors, with other development actors, and with WFP Institutional stakeholders (MINAGRI, MINEDUC, MINALOC, MINEACOM) at national and district level Document review (LRP projects documents and progress reports, project design document, report on engagement meetings with government officials) KII with cooperative and value chain actors, other development actors, WFP and government FGD with SHF and cooperatives Triangulation of documents and results from FGD and key informant interview Currently there is no project document that provides a complete overview of the sustainability or exit strategy. Information from stakeholders will be collected in the field Further documents on the LRP planning and implementation to provide to the evaluation team by WFP, in particular on sustainability/exit strategy Quality of the information depends on whether WFP have document presenting their intervention strategy (for the market-based solutions) and on their exit strategy. For government, quality of the data collected will depend on institutional stakeholder knowledge of the different policies and strategies. Effectiveness Endline Report - August 2019 (draft) | P a g e 85 Evaluation question Sub-Question Indicator Main sources of information Method for data collection Method for data analysis Quality of available data 2-What are the effects of the project on the cooperative’s ability “to be”, “to organize”, “to relate”, and “to do” 2.1-Are cooperative better organized? Evidence that cooperative have a common vision. Evidence that number of meetings of cooperative members and of cooperative leaders has increased Evidence of increased knowledge of leaders’ roles by cooperative members Existence of up to date records at cooperative level and regular utilization of accounting systems Increased number of collective operations registered by cooperative (purchase of inputs or services, sales, access to credit) Increased number of members paying their dues/improved timeliness of payment by members Cooperative records and interviews WFP implementing partners records and interviews Reports from the relevant government ministries in charge cooperatives Progress Reports from WFP and implementing partners WFP baseline and monitoring data on the LRP cooperatives Cooperatives checklist/review of LRP cooperative baseline data FGD with cooperative members KII with cooperative management, implementing partners, and WFP Review of documentation on cooperatives and from implementing partners Triangulation of primary & secondary data sources. Thematic analysis of qualitative data Descriptive and inferential analysis of survey data. Information from stakeholders will be collected in the field WFP also collects information on the cooperatives which will provide a sense of progress against specific indicators. Documents from WFP and its implementing partners will be provided to the evaluation team Quality of the data collected will depend on whether cooperatives record are existing and updated. 2.2-Are cooperatives developing partnerships? Number of partnerships with banks, service providers, institutions and other cooperatives Cooperative records and interviews WFP baseline and monitoring data on the LRP cooperatives Cooperatives partners (MFI, service providers, institutions) Documentations and Progress reports from cooperatives Progress Reports from WFP and implementing partners Cooperatives checklist KII with cooperative management, implementing partners, and WFP Review of cooperative documentation Descriptive & inferential analysis of survey data. Triangulation of qualitative and quantitative data Triangulation of primary & secondary data sources. Thematic analysis of qualitative data It is assumed that cooperatives keep reasonably good records of membership, contracts, etc. This needs to be verified. Information from stakeholders will be collected in the field Documents from WFP and its implementing partners will be provided to the evaluation team Quality of the information from WFP and implementing partners is assumed to be good. Quality of data at cooperatives level depends on the quality of the records. KII should provide good quality data on partnerships. Endline Report - August 2019 (draft) | P a g e 86 Evaluation question Sub-Question Indicator Main sources of information Method for data collection Method for data analysis Quality of available data 2.3- Are the cooperatives becoming more economically dynamic? Cooperatives plan their business activities at the beginning of the season and make an assessment of their result at the end Cooperatives have better skills on credit and loans management Evidence that the cooperative net margins have improved Volumes of products sold through the cooperatives increased Cooperative records and interviews WFP baseline and monitoring data on the LRP cooperatives Other value chain actor interviews WFP implementing partners interviews and documentation Progress Reports from WFP and implementing partners Cooperatives checklist KII with cooperatives and value chain actors Cooperative records Descriptive & inferential analysis of survey data. Triangulation of qualitative and quantitative data Triangulation of primary & secondary data sources. Thematic analysis of qualitative data Information from stakeholders will be collected in the field Documents from WFP and its implementing partners will be provided to the evaluation team WFP is collecting information on loans at cooperatives levels. Data on business skills and planning at cooperative is expected to be of medium quality as it requires cooperatives to have already sufficient business awareness to talk about it. Quality of data on volumes and net margin of cooperatives depends on the quality of the records at cooperatives levels (it will be difficult to have accurate information during interviews) 2.4-Do activities of the cooperative benefit to their members? Evidence that cooperatives regularly distribute dividends to their members Evidence that cooperatives offer more effective and timely services in one or more of these fields: access to inputs, access to finance, marketing, access to information, extension, storage or processing There is a self-financing mechanism for cooperative activities Cooperative records WFP baseline and monitoring data on the LRP cooperatives Other value chain actors SHF who are members of the cooperative Cooperatives partners (MFI, service providers, institutions) WFP implementing partners FGD with SHF KII with cooperative leadership and partners, value chain actors and WFP implementing partners On site observation (cooperative documents) Triangulation of qualitative and quantitative data Triangulation of primary & secondary data sources. Descriptive & inferential analysis of survey data. Thematic analysis of qualitative data Information from stakeholders will be collected in the field It is assumed that cooperatives keep reasonably good records of their support to SHF. This needs to be verified. Documentation from WFP implementing partners to be made available by WFP to the team Quality of the data will depend on whether cooperatives have up to date records. It is expected that the quality of these data will be poor to medium. 3 - How does the LRP programme affect male and female 3.1 - Has SHF knowledge of options and alternatives improved? Are the Cooperatives are more regularly seeking information on markets (options, price, markets...) SHF survey Triangulation of information between sources Information from stakeholders will be collected on the field Endline Report - August 2019 (draft) | P a g e 87 Evaluation question Sub-Question Indicator Main sources of information Method for data collection Method for data analysis Quality of available data SHF capacity and behaviour? benefits equal for male and female farmers? SHF demonstrate improved knowledge on production practices Cooperatives demonstrate improved knowledge on post￾harvest handling SHF have regular extension services adapted to their needs from LRP implementing partners and from the government Cooperatives checklist/review of LRP cooperative baseline data FGD with SHF. Reports reviews from WFP and implementing partners of LRP Thematic analysis of quantitative data. Descriptive and inferential analysis of survey data. Documents and data from WFP will be provided to the evaluation team Quality of the data is expected to be medium (there is no records at farmers level or cooperative levels on how they seek information or on the adoption of good practices. Triangulation will be a key element to ensure quality of the analysis. 3.2-Have SHF choices around short-term productive assets, marketing, good practices changed? Are the benefits equal for male and female farmers? Increased/improved investment by SHF in short term productive assets for maize & beans (seeds, tools, inputs and casual labour) Increase in the area cultivated in maize and beans Evidence of improved/increase in access by SHF to credit Evidence that SHF adopt improved practices in production and post￾harvest handling SHF Cooperatives Implementation partners Cooperatives partners (MFI, service providers, institutions) Documentation and reports from WFP including WFP baseline and monitoring data on the LRP cooperatives SHF survey Cooperatives checklist KII with sample of SHF. KII with WFP staff, government representatives, cooperatives lead Reports reviews from WFP and implementing partners of LRP Descriptive and inferential analysis of survey data. Triangulation of information between sources Thematic analysis of quantitative data. Information from stakeholders will be collected on the field Documents and data from WFP and implementing partners will be provided to the evaluation team Information on access to credit should have a good quality as it is monitored by WFP. Quality of data on short term productive assets and on area cultivated, as well as adoption of good practices is expected to be medium. 4-Has the LRP contributed to creating new opportunities for male and female SHF? 4.1-Has LRP contributed to the design and adoption of a pro-SHF and buyers partnership strategy? Does the strategy benefit both male and female SHF? Existence of a strategy at government level to support development of business partnership between cooperatives and pro-SHF buyers Evidence of LRP contribution to the design and adoption of a pro-SHF and buyers partnership strategy Cooperative KII Institutional stakeholders (MINAGRI, MINEDUC, MINALOC, MINEACOM) at national and district level Pro-SHF buyers WFP KII with government officials, WFP staff, cooperatives and other stakeholders Review of Reports from WFP Reports from the relevant government ministries in charge of cooperatives Triangulation of primary & secondary data Information from stakeholders will be collected on the field Documents and data from WFP will be provided to the evaluation team (WFP and FTMA) Quality is expected to be medium. Endline Report - August 2019 (draft) | P a g e 88 Evaluation question Sub-Question Indicator Main sources of information Method for data collection Method for data analysis Quality of available data 4.2 - Are big buyers increasingly willing to purchase maize and beans from SHF?? Evidence of increased awareness by large buyers of cooperatives as business partners Evidence that the number of buyers purchasing maize and beans from cooperatives has increased Volume of food procured by WFP from pro-SHF buyers Percentage of food from the platform going to buyers other than WFP Cooperatives interviews and records WFP baseline and monitoring data on the LRP cooperatives Pro-SHF buyer interviews and records Progress Reports from WFP and implementing partners Cooperatives checklist KII with cooperatives leaders, buyers, WFP staff, other stakeholders Review of Reports from WFP and implementing partners Descriptive and inferential analysis of survey data. Triangulation of documents and results from key informant interview Thematic analysis of qualitative data Information from stakeholders will be collected on the field Documents and data from WFP will be provided to the evaluation team (WFP and implementing partners) Quality of the data is expected to good (it is assumed that large buyers have records) if buyers are ready to openly discuss these issues. Quality of data collected by WFP on food procurement on the volume of sales at platform level is expected to be very good. 4.3 - Are producers able respond to this increased demand? Cooperatives have an effective system to manage quality standards Number of contracts signed between cooperatives and buyers Volumes of products from cooperatives purchased by big buyers Evidence of reduction in the number and percentage of contract’s defaulting Contracts ‘conditions in terms of schedule, transport, time of payments quality and price are adapted to needs and constraints of buyers and SHF Cooperatives Documentations and Progress reports from cooperatives WFP Pro-SHF buyers Progress Reports from WFP and implementing partners KII with cooperatives leaders, pro-SHF buyers and WFP staff FGD with SHF Review of Reports from WFP and implementing partners Review of contracts Cooperative and buyers’ records Triangulation of documents and results from key informant interview Thematic analysis of qualitative data Information from stakeholders will be collected on the field Documents and data from WFP will be provided to the evaluation team (WFP and implementing partners) Quality of the data collected is expected to be medium to good depending on the records kept at cooperative levels. There may be a lack of transparency from cooperatives and buyers on contracts and contract’s defaulting which would affect quality of data. Efficiency Endline Report - August 2019 (draft) | P a g e 89 Evaluation question Sub-Question Indicator Main sources of information Method for data collection Method for data analysis Quality of available data 5- How does the procurement, delivery, and distribution of the modality for school feeding which is promoted through LRP compare - in terms of cost￾effectiveness and timeliness - to the alternative modalities of food procurement?37 5.1 Has the LRP brought about reduced prices of school meals? Comparison of the price of school meals in the two main HGSMP modalities (local procurement and imported food) National and local representatives of MINEDUC WFP records on food procurement, distribution, and delivery costs for both modalities WFP reports to USDA and to McGovern Dole MINEDUC reporting if available Documentation and WFP data base analysis for information about cost of procurement, handling and delivery. Triangulation of information between the different data collection methods and sources Thematic analysis of qualitative data Most of the data is judged to be available from WFP records Quality is expected to be very good. 5.2-Has the LRP resulted in improved procurement, distribution and delivery time, and enhanced efficiency compared with other food procurement modalities? Timeliness for procurement, distribution and delivery of food under the two modalities Cost for procurement, distribution and delivery of food under the two modalities National and local representatives of MINEDUC WFP records on food procurement and delivery timelines Data and information from schools in the two modalities WFP reports to USDA and to McGovern Dole MINEDUC reporting if available Documentation and data base analysis for information related to procurement, handling and delivery. KII with school heads, SMP officials from MINEDUC and other relevant stakeholders Interviews with WFP staff, in particular staff involved in procurement and management of food modalities Triangulation of information between the different data collection methods and sources Thematic analysis of qualitative data Most of the data is judged to be available from WFP records Quality is expected to be very good 5.3-How often does LRP face pipeline breaks compared with other food procurement modalities? Frequency and duration of pipeline breaks under the two modalities Number of school feeding days under the two modalities WFP records on pipeline management WFP records on number of days of school feeding per year WFP reports to USDA and to McGovern Dole Documentation review (monitoring reports, reports to the donor) WFP data base analysis Triangulation of information between the different data collection methods and sources Thematic analysis of qualitative data Most of the data is judged to be available from WFP records Quality is expected to be very good 37 This question was only examined at baseline. In agreement with WFP it was dropped for the endline. We have retained it in the matrix for the sake of completion but there is no end-line assessment against this question in the report. Endline Report - August 2019 (draft) | P a g e 90 Evaluation question Sub-Question Indicator Main sources of information Method for data collection Method for data analysis Quality of available data MINEDUC reporting Intermediate outcomes and impact 6-How does the LRP programme affect male and female SHF income? 6.1-Has male and female SHF income improved? Volumes and value of maize and beans sales by SHF, disaggregated by gender Volumes of sales of the main crops grown by smallholder farmers in the southern districts (wheat, rice, cassava and Irish potatoes…), disaggregated by gender Net margin/ha and net margin/man day from maize and bean production, disaggregated by gender Maize and beans significant source of income for SHF SHF Cooperative Implementation partners Cooperatives partners (MFI, service providers, institutions) Documentation and reports from WFP, including WFP baseline and monitoring data on the LRP cooperatives SHF survey Cooperatives checklist KII with a sample of SHF. KII with WFP staff, government representatives, cooperatives leads Reports reviews from WFP and implementing partners of LRP Triangulation of information between the different data collection methods and sources. Thematic analysis of quantitative data. Descriptive and inferential analysis of survey data. Information from stakeholders will be collected on the field Documents and data from WFP and implementing partners will be provided to the evaluation team Quality of the data is expected to be medium (SHF do not usually keep records and there is often different tools used to measure volumes) 7- How and to what extent does the LRP programme contribute to producers marketing food products that meets quality standards and is nutritious, and is culturally acceptable? (impact) 7.1-Has the overall quality of maize and bean products consumed by SHF improved? Evidence of improved quality of maize and beans products utilized by SHF supported by LRP SHF Cooperatives Big buyers Implementation partners WFP baseline and monitoring data on the LRP cooperatives SHF survey Cooperatives FGD with cooperatives, big buyers & partners Descriptive and inferential analysis of survey data. Triangulation of information Information from stakeholders will be collected on the field Quality of the information is expected to be medium Endline Report - August 2019 (draft) | P a g e 91 Evaluation question Sub-Question Indicator Main sources of information Method for data collection Method for data analysis Quality of available data 7.2-Is there evidence that SHF are consuming and marketing food products that meets quality standards and are nutritious and culturally acceptable? SHF keep a part of their quality production for consumption Evidence from schools confirms that SHF produce is considered nutritious and culturally acceptable SHF Cooperatives Big buyers Implementation partners WFP baseline and monitoring data on the LRP cooperatives Parents and children in schools SHF survey KII & FGD w/ cooperatives, buyers & implementing. Partners School interviews (end-line) Descriptive and inferential analysis of survey data. Triangulation of information between sources Information from stakeholders will be collected on the field Quality of the information is expected to be medium to good 8-What is the level of participation of men and women? Are women well represented, including in leadership positions? What are the disaggregated impacts on women Has the LRP programme affected male and female SHF differently? 8.1-How has the participation of men and women evolved in cooperative decision￾making structures? Evidence that the number of women in leadership position in cooperatives has increased SHF Cooperatives (leadership and documentation) WFP and their implementation partners WFP baseline and monitoring data on the LRP cooperatives Cooperatives checklist Review of cooperatives documentation FGD with SHF and cooperative leaders Review of reports (WFP & implementing partners of LRP) Triangulation of information between the different data collection methods and sources Thematic qualitative data analysis Information from stakeholders will be collected on the field Documents and data from WFP will be provided to the evaluation team Assuming that cooperatives keep records of their members, quality of data is expected to be very good. 8.2- Has the percentage of female SHF who are members of cooperatives increased over time? Evidence that the relative proportion of male and female beneficiaries has increased in favour of women SHF Cooperatives (leadership and documentation) WFP and their implementation partners WFP baseline and monitoring data on the LRP cooperatives Cooperatives checklist Review of cooperatives documentation Review of reports from WFP and implementing partners of LRP Triangulation of information between the different data collection methods and sources Thematic qualitative data analysis Information from stakeholders will be collected on the field Assuming that cooperatives keep records of their members, quality of data is expected to be very good. Sustainability Endline Report - August 2019 (draft) | P a g e 92 Evaluation question Sub-Question Indicator Main sources of information Method for data collection Method for data analysis Quality of available data 9-Is there emerging evidence that the cooperatives have capacity to create linkages with schools? 9.1- Has LRP contributed to develop business relationship between cooperatives and schools Evidence that mutual awareness of school and cooperatives as business partners has increased (i.e. school know they can purchase food at cooperatives level and cooperatives know they can sell to school) Cooperatives (leaders and documents) Schools (school head and documents) KII with school leads FGD with cooperative leaders Documents review of cooperatives records Triangulation of information between the different data collection methods and sources Descriptive analysis of cooperative checklist Information from stakeholders will be collected on the field Quality of the data is expected to be good 9.2- Are there cooperatives delivering food to schools (primary/secondary) in the targeted districts? Evidence of increased sales from cooperatives to schools Cooperatives (documents) WFP baseline and monitoring data on the LRP cooperatives Cooperatives checklist Documents review of cooperatives records Triangulation of information between the different data collection methods and sources. Descriptive analysis of cooperative checklist Information from stakeholders will be collected on the field Quality of the data depends on whether cooperatives have up to date records on their sales and clients 9.3- Do school (primary/secondary) purchase agriculture products from cooperatives? Existence of a budget for school to purchase foods Evidence of direct purchases to cooperatives from schools Cooperatives (leaders and documents) Schools (school head and documents) WFP baseline and monitoring data on the LRP cooperatives KII with school leads FGD with cooperative leaders Cooperatives checklist Documents review of cooperatives records Triangulation of information between the different data collection methods and sources Descriptive analysis of cooperative checklist Information from stakeholders will be collected on the field Quality of the data is expected to be good assuming schools have records of their purchase Endline Report - August 2019 (draft) | P a g e 93 Annex 5. List of persons met at endline NAMES INSTITUTION/FUNCTION DATE Location Belange Carine Bideri Joseph RWARRI LRP/Project Manager RWARRI LRP/Project M and E RWARRI/Executive Secretary 11/06/2019 11/06/2019 11/06/2019 Kigali Sam Gata Sylvie MINEDUC MINAGRI 11/06/2019 11/06/2019 Kigali Kabalira Gerard COAMANYA/PRESIDENT 12/06/2016 Gisagara Francois Nkurikiyimana RWARRI FIELD OFFICER 12/06/2019 Gisagara Nshimiyimana Jean Bosco Duterimbere Agatorove 13/06/2019 Nyaruguru Paulin RWARRI LRP/FIELD OFFICER 13/06/2019 Nyaruguru Thomas District/Agronomist 13/06/2019 Nyaruguru Kwizera Theogene Abishyizehamwe Urwonja 13/06/2019 Nyaruguru Maniraho Jean Baptiste Abibumbiyehamwe Cyanika 14/06/2019 Nyamagabe Mukanyiribanje Josephine Koperative Icyerekezo Isimbi 14/06/2019 Huye Immaculee Musabyimana COPEKI/Vice President 15/06/2019 Huye Landouald Nzabamwita Abemeramihigo ba Mwogo/President 15/06/2019 Huye Ndayisaba Pascal Abishyizehamwe Ngera: executives and lead farmers 16/06/2019 Nyaruguru Mwitirehe Augustin CCM Muganza=CCRM/Prezida 17/06/2019, 9:30am Gisagara Nshimiyimana Alphonse KAIMU Abadacogora 17/06/2019, 1:20am Gisagara Aime DJAF Gisagara 17/06/2019 Gisagara DJAF Huye 17/06/2019 Huye Edouard RIAS 18/06/2019 Huye Nsanzabaganwa Kopianya President and KIIs 18/06/2019 Nyaruguru Rurangwa Vincent KOBABOMANGO 18/06/2019 Nyaruguru Gatera Celestin RIAS 19/06/2019, 8:30am Kigali Amar Amy Wim and Sameera WFP LRP/FTMA Program Manager WFP Home grown School feeding program FTMA strategy and WFP/LRP Project M & E 19/06/2019, 2pm 19/06/2019, 4pm Kigali Eisha Africa Improved Food (AIF)/CHAI 20/06/2019, 8:20am Kigali Olivier EAX 20/06/2019 Kigali Carine RWARRI LRP Monitoring and Evaluation 21/06/2019 Kigali Moses Minimex 21/06/2019 Kigali Icco-terrafina 21/06/2019 Kigali Endline Report - August 2019 (draft) | P a g e 94 Annex 6. Survey tool Gender of the respondent Is the respondent also the HHH Gender of the HH head Age of the HHH Educational level of the HHH Marital status of the HHH Total males in the HH Total females in the HH Total number of males who normally provide labour Total number of females who normally provide labour Main source of income for the HH (main economic activity of the HHH) What is the average monthly household income of your household? Approximately what proportion of the household income was invested in agriculture? Were you in need of a loan of credit/loan in the past planting session for finance your agricultural purposes? Why did you require the credit/loan? Did you receive the credit/loan? If no why did you not receive the load/credit? What is the source of the loan? did you receive the loan • Bank • Cooperative • Soft loan from friends of family • Agricultural inputs on credit How easy was it to revive credit/loan for agricultural purposes? Please comment on the following on the aspect of the credit • Timeliness • The amount applied for • The repayment period • Collateral to receive the loan Have you defaulted in payment of the loan Main reason for defaulting What crops does your household usually plant? What crops did your household plant in the last planting season? how much land does the household currently own Of the land owned by the household, how much was allocated to crop planting in the last planting season? In the last planting session, did your household, rent or buy more land for crop faming purposes? What was the main reason for renting more land? What proportion of the HH farm is allocated to crop farming In the past planting session, which crop did you plant and in what acreage Maize Beans Millet Sorghum others How much of following crops did you harvest in the past planting session Maize Beans Millet Sorghum others How would you consider the quantity of crops (specifically maize and beans) harvested form the last planting session? Endline Report - August 2019 (draft) | P a g e 95 Was not enough even for household consumption only Was enough for HH consumption What proportion of the harvested maize/beans was allocated for household consumption? Maize Beans Millet Sorghum Others Where/to who was the surplus maize and beans harvested sold? After the season harvest, did the household need to purchase maize and beans to meet the household needs? How long after harvest did the household require to purchase the extra? In the past planting session, did you make any commitment of supplying your agricultural products to any person/group before harvesting? Who did you make the commitment to? What proportion of the planted maize and beans did you make commitment to? What is the current main source for agricultural labour for the household? In the past planting session, did you hire extra labour to meet your agricultural needs? In the past planting session, who much did your household spend on the following agricultural inputs Seeds Agricultural equipment’s Fertilizers Hired labour Extra farm land Transportation of agricultural materials From what source did you receive the information? What information did you receive? During the last planting session, did you practice any improved agricultural practices? What improved agricultural technology did you practice? During the last planting session, did you receive any agricultural extension services ? From who did you receive the extension services How frequent was the extension services? During the last harvest season, did you practice any post-harvest practices What post-harvest handling practices did you practice last session cooperatives 1. Are you a member of a cooperative? 2. How activate are you in the activities related to the cooperative 3. What are the services currently offered to you by the cooperatives? 4. Are the services offered by the cooperative cheap but of quality as compared to the same services offered by other providers? 5. Are the services offered by the cooperative profitable to you? 6. Do you feel that the activities of the cooperative are benefiting you? 7. If yes, what specific activities of the cooperative are benefiting you? Markets 1. Do you have information of existing markets for your produce? 2. What information do you know in particular about the markets? 3. What is the source of the information? 4. Do you feel the information you have about the markets is sufficient? 5. If No, what extra information would you like to have? Debt/loans 1. During the last planting sessions, did you require a loan/ financial support to finance your agricultural activities? 2. Why did you require the loan? Endline Report - August 2019 (draft) | P a g e 96 3. Did you receive the loan/financial support required? 4. From who did you receive the financial support? 5. Please comment of the following aspects of the financial support/loan received a. Timeliness b. Collateral for the financial support required c. The amount required against amount received d. Interest rates e. The repayment period 6. In the past planting session, did you get into debt with middlemen before the harvesting session? 7. What was the main reason for getting into debt with the middlemen against your harvest? 8. If yes, did you receive better ‘deal’ from the middlemen against your harvest Short term assets investments 1. In the last plating session, how much did you spend on the following short term productive assets a. Seeds b. Tools c. Agricultural inputs d. Casual labour e. Extension services To who and what quantities did you sell you produce of the last planting session? Maize Beans Millet Sorghum In after the last harvest, was there need for your household to purchase maize and beans for household consumptions If yes, what quantities were purchased Maize Beans Endline Report - August 2019 (draft) | P a g e 97 Annex 7. How has cooperatives ability ‘to be’, ‘to organize’, ‘to relate’ and ‘to do’ evolved from baseline to endline? 7.1. General information on LRP cooperatives Cooperatives selected for the LRP program are relatively young. The oldest one started its activities around 1987 but for most of them the formation is more recent (around 2010). Establishment of the LRP cooperatives occurred in a context of cooperative development all over the country. Several experiences in different regions of the world show that it takes cooperatives between 15 and 20 years to build their capacities in order to have a good command of economic and financial tools and to become real economic stakeholders. In that sense, we can still consider the LRP cooperatives as emerging cooperatives. A majority of the cooperatives are organized around marshlands (12 out of 16 cooperatives). Working on a marshland is easier for a cooperative as all the farmer plots are grouped in the same area. This facilitates input provision, extension services, control of practices, and production. Usually, exploitation of the land is individual (each farmer has been assigned a small plot to grow on the marshland). Also, land on the marshland belongs to the state and it has a stronger control over the production choices: farmers feel obliged to comply with government recommendations on these plots as they fear losing their land. As shown by figure below, LRP cooperatives present very diverse situations: membership varies from 39 to 862 members. Four cooperatives have less than 80 members and 5 have more than 500 members. The area available for cooperatives ranges from 5 ha to 200 ha. Figure 6: Basic information on LRP cooperatives (qualitative data collection and WFP documents) Cooperative’s name Number of members Female members Type of land exploited Total land available for the cooperative Area planted (season A 2019) Coamanya Gishubi 547 272 marshland 130 ha 60 ha Duterimbere 867 575 marshland 75 ha 20 ha Abishyizehamwe Urwonja 649 322 marshland 49.5 ha 49 ha Abibumbye 824 537 marshland 72 ha ? Icyerekezo 264 147 marshland 30 ha 30 ha Kopiki 80 ? marshland 5 ha 0.5 ha Abemerimihigo 540 180 marshland 82 ha 46 ha Abishyizehamwe Ngera 67 37 marshland 5 ha 5 ha CCM Muganza 224 90 marshland 120 ha 37 ha Kaiimu 430 118 terraces 75 ha 45 ha Kopianya 51 28 terraces 3.5 ha 3.5 ha Kobabomango 39 30 terraces 6 ha 6 ha Koabiwa 147 95 marshland 9 ha 4 ha Koaimu 150 65 terraces 100 ha 12 ha Kobarwo 317 148 marshland 40 ha 40 ha Coamagi 400 280 ? 200 ha 40 ha Endline Report - August 2019 (draft) | P a g e 98 7.2. Assessment of cooperative capacities The criteria used to assess cooperatives’ capacities are: - ‘Ability to be’: referring to the governance, ownership, leadership, existence of a common vision and a strategy to reach its objectives - ‘Ability to organize’: reflecting human, material and financial resources, management and organization of activities, planning and assessment of the results - ‘Ability to relate’: ability to develop relations with the public sector (local or national authorities, extension agents), development partners (national or international) and private sector partners (inputs providers, microfinance institutions, buyers and other value chain actors) - ‘Ability to do’: ability to produce, existence of a post-harvest and marketing system that enables both the cooperative and its members to generate profits To assess capacity, twelve cooperatives (qualitative data collection) were scored against these four abilities. Scoring covered all criteria. Each criterion was assessed by the evaluation team, and when possible, this was complemented by a self-assessment from cooperative leaders. Each criterion was ranked from zero (inexistent or very low) to five (very good). This score was then compiled by indicator in a percentage form. This provided a transparent and objective guide to assess capacity and to identify gaps that need to be addressed. The resulting cooperative scores at baseline are shown in the figures below. Figure 7: Representation of the abilities of high potential cooperatives (scoring based on qualitative data collection during baseline) Endline Report - August 2019 (draft) | P a g e 99 Figure 8: Representation of the abilities of cooperatives that are losing momentum (scoring based on qualitative data collection at baseline) Figure 9: Representation of the abilities of nascent cooperatives (scoring based on qualitative data collection during baseline) At endline, it was not possible to meet all the cooperatives that were visited during the baseline exercise (leaders were not always available at the time where the team planned the visit). However, 12 out of 16 cooperatives were visited and this is high enough to give a good view of the overall capacities and situation of the cooperatives. In addition, of 12 eight had been visited at baseline and endline which made it possible to examine specific changes at cooperative level over time. It was not possible to cross check the data collected with the Rabobank International Advisory Services (RIAS) scorecard on cooperative capacity. Doing so should also provide interesting information on how capacities have changed at cooperative level. 7.3. Description of LRP’s capacity building and project implementation scheme LRP cooperative capacity building is embedded in the FTMA capacity building activities. It involved several organizations providing different types of services/expertise to cooperatives under the coordination of the WFP/FTMA team. RWARRIis the implementing partner for LRP cooperatives. LRP cooperatives are included in the FTMA cooperatives (supported either by Rwanda Development Organisation or RWARRI). Implementation and capacity building are harmonized inside FTMA and the Endline Report - August 2019 (draft) | P a g e 100 tools and schemes of support are shared. In terms of capacity building, the support covered several topics, and involved different partners: GAP and PHHS RWARRI is training and monitoring 46 cooperatives (including the 16 LRP cooperatives) in Nyamagabe, Nyaruguru, Gisagara and Huye districts. RWARRI assigned one Field officer (FO) in each district. Field officers are responsible for conducting lead farmer training and for subsequently monitoring the lead farmer-to-farmer training. FOs also assist cooperatives through the contract process with buyers and in their interactions with other stakeholders based on cooperative demand and needs. So far (March 2019), for the 16 LRP cooperatives, 5,617 farmers have been trained on GAP and PHHS and training manuals have been designed and distributed to lead farmers. They cover all the steps of the production, from preparing the lands to post-harvest handling and storage. Training also emphasizes the appropriate use of inputs. Each lead farmer is responsible for training two groups of farmers (about 20 farmers per group). FOs supervise the training and they often participate to make sure that lead farmers can pass on the key messages to farmers. Governance and cooperative management Training of FTMA cooperatives on this topic has been assigned to RIAS, which is a member of FTMA. It operates as a subsidiary of Rabobank offering advisory services on management to different organizations including cooperatives. RIAS does not have staff in the field but employs consultants that travel to the field for the training sessions of cooperatives. An assessment of the cooperative governance, operational and financial management was done (with a score card system quite similar to the one used in the evaluation). Trainings were then organized. The training consists in several training sessions targeting the executives of the cooperatives. Access to finance To facilitate access to finance for cooperatives, LRP aimed at building up cooperative capital and capacity to negotiate with banks and MFI. Increasing cooperative capital will enable cooperatives to have stronger bargaining power with banks and better credit conditions. WFP contracted ICCO Terrafina to strengthen microfinance institutions and to support cooperatives. ICCO Terrafina is an organization that aims at boosting agriculture through microfinance to improve smallholder farmer livelihoods and increase food security. ICCO Terrafina trained FTMA cooperatives and supported the implementation of group solidarity lending. In group solidarity-based lending, the members of a small group co-guarantee the repayment of each other loan which constitutes a substitute for the usual requirement for collateral. For the LRP cooperatives, the groups have not reached this stage yet and they mainly use the capital either to provide small loans to group members or simply as a way to have some savings available in case of needs. For season A 2018, ICCO implemented 82 saving groups in 15 cooperatives, including five LRP cooperatives. In the five LRP cooperatives, 202 savings groups have been formed and the total amount saved is 3,073,850 RWF. In 2019, ICCO Terrafina continued supporting these cooperatives and started training two other LRP cooperatives. Endline Report - August 2019 (draft) | P a g e 101 Linkages between cooperatives and buyers LRP cooperatives were introduced to FTMA buyers through a meeting organized before the season which brought together representatives of FTMA cooperatives and buyers. Each buyer presented the proposed contract conditions. These pre-contracting meetings enabled cooperatives to approach their chosen buyer. Contracts are systematically read and explained to cooperatives. Buyers and cooperatives conduct negotiations and amend the contract if needed. The quantity to be provided is fixed in the contract but can be revised before harvest, to reflect actual production. The price is not fixed in the contract but the GoR minimum price is included as reference. The price is fixed at delivery time based on market price (and may sometimes reflect a premium). Buyers and cooperatives organize the delivery. The maize is paid after delivery (this may take a few days or weeks) on the cooperative account. Cooperatives then pay their members based on what they have individually delivered. RWARRI’s field officer supports cooperatives at all stages of the contract preparation, signing and delivery. The figure illustrates what type of support was received by LRP cooperatives in the context of the project. It shows that cooperatives particularly appreciated the support that was provided in the form of training on GAP and PHHS, access to market, and the access to equipment. Figure 10: Importance of support received from LRP (open choice question to 12 cooperatives, qualitative data collection) 7.4 Other elements on cooperative capacity LRP’s cooperatives ‘capacity to be’ Endline Report - August 2019 (draft) | P a g e 102 • Common vision Figure 11: Cooperative ‘common vision (qualitative data collection, to 12 cooperatives, multiple option choice) LRP’s cooperatives ‘capacity to organize’ • Document keeping Table 7: Cooperative capacity to document their activities and resources (qualitative data collection) Cooperatives that consider they have the capacity to document activities and resources Cooperatives that consider that they need more support to document activity and resources CCM Muganza Duterimbere Coamanya Gishubi Abishyizehamwe Urwonja Abibumbye Icyerekezo Kopiki Abemerimihigo Abishyizehamwe Ngera Kaiimu Kopianya Kobabomango Table 8: Evaluation team assessment of cooperative document and record keeping system (qualitative data collection) Good document keeping system Average document keeping system Poor document keeping system Coamanya Gishubi CCM Muganza Duterimbere Icyerekezo Abishyizehamwe Ngera Kaiimu Kopianya Abishyizehalwe Urwonja Kobabomango Abibumbye Abemerimihigo Kopiki Endline Report - August 2019 (draft) | P a g e 103 • Cooperative funds in bank accounts at the time of the interview Figure 12: Cooperative financial resources (qualitative data collection) Cooperative’s name Amounts on the account Coamanya Gishubi 640,000 RWF Duterimbere 500,000 RWF Abishyizehamwe Urwonja 2,000,000 RWF Abibumbye 250,000 RWF Icyerekezo 1,500,000 RWF Kopiki ? Abemerimihigo 1,400,000 RWF Abishyizehamwe Ngera 300,000 RWF CCM Muganza 54,000 RWF Kaiimu ? Kopianya 2,100,000 RWF Kobabomango 400,000 RWF • Cooperative use of the resources provided by members Figure 13: Use of cooperatives resources (multiple option choice, qualitative data collection, 12 cooperatives) Endline Report - August 2019 (draft) | P a g e 104 • Access to credit LRP cooperative ‘ability to relate’ Cooperatives are building partnership with several types of actors: - Public institutions: These are still key actors for cooperatives. District and sector agronomists support cooperatives to follow government rules and guidelines concerning organisation of cooperatives and agricultural activities. Visits from the agronomists are not frequent at cooperative level, and cooperatives do not see them as extension agents. - Financial partners: As mentioned before several banks and MFIs are available for cooperatives at sector and district levels. However, credit conditions still appear non-adapted to the specific needs and constraints of cooperatives, and agriculture is clearly not yet a priority for commercial institutions. This issue is taken into consideration in the FTMA programme and ICCO Terrafina works on building capacity and awareness of MFIs in order for them to provide better services to cooperatives. Currently, three commercial institutions are involved in FTMA. In 2018, they provided funds to 28 cooperatives in Rwanda (but only one LRP cooperative), which illustrates that there is still some work to be done to link cooperatives to financial services. - Service providers: Input providers (agro-dealers and Tubura38) are well identified by cooperatives. However, it seems that input provision is regularly delayed and as a result more and more cooperatives are becoming agro-dealers. In these cases, they directly submit their order to APTC (Agro-Processing Trust Cooperation LTD39). Cooperatives that started this activity are satisfied. - Partnership with cooperatives : Through LRP, cooperatives have had several opportunities to meet and exchange. However, cooperatives did not consider this as something particularly 38 Company implemented through the 1-acre fund NGO to supply inputs to farmers (sometimes at credit). 39 Parastatal organization in charge of inputs distribution in Rwanda. Figure 14 : Risks of credit for cooperatives (multiple choice option, qualitative data collection based on 12 cooperatives) Endline Report - August 2019 (draft) | P a g e 105 interesting or useful. Cooperatives have limited relations with other cooperatives, and they do not plan to work together or build a Union of cooperatives. - Development actors : RWARRI staff are well known to cooperatives and they appreciate the way RWARRI is working with them. Apart from LRP partners, cooperatives have very limited links with other development actors. No new partnership was built. LRP cooperative ‘ability to perform’ • Exploitation of the marshland for maize cultivation Figure 15 : Percentage of the marshland dedicated to maize (qualitative data collection and data from WFP) • Cooperative production choices Figure 16: Crops grown by season (qualitative data collection) Coamanya Gishubi Duterim bere Abishyiz ehamwe Urwonja Abibu mbye Iciyer ekezo Abem eheri go Abishyizehamw e Ngera CCM Muganz a Kaiim u Kopia nya Kobabo mango Kopiki Seaso n A Maize (beans) Maize Irish potatoe Maize Beans Maize Beans Maize Rice Maize Beans Maize Beans Irish potat oes Maize Maize Seaso n B Maize Beans Maize Irish potatoe Maize Veget ables Maize Vegetables Rice Maize beans Veget ables Beans Irish potat oes Beans Beans Seaso n C Maize Beans Beans Veget ables Vegetables Maize Endline Report - August 2019 (draft) | P a g e 106 • Estimation of the production/potential of production Figure 17: Maize production estimation (qualitative data collection and data from WFP) Figure 18: Evolution of cooperatives sale and buyers (qualitative data collection) • Post-harvest handling and storage practices of cooperatives Figure 19: Evolution of cooperative post-harvest handling and storage capacities (qualitative data collection) Easier to aggregate Improvement of post-harvest handling and storage by the cooperative More farmers are harvesting at the same time More farmers are drying maize at cooperative’s drying ground Increased storage of maize Need for bigger drying area Coamanya Gishubi yes yes yes yes no no Duterimbere yes yes yes yes no yes Abishyizehamwe Urwonja yes yes Yes but still difficult yes no yes Abibumbye no yes no yes no yes Icyerekezo yes yes yes yes yes no Kopiki no yes no no no no Abishyizehamwe Ngera yes yes yes yes no yes Abehemerigo yes yes yes yes no yes CCm Muganza yes yes yes yes yes no Kaiimu no yes no no no yes Kopianya yes yes yes yes no no Cooperative’s name estimated production high (MT) estimated production low (MT) potential maize area (ha) potential production (MT) Coamanya Gishubi 180 120 80 280 Duterimbere 60 40 40 140 Abishyizehamwe Urwonja 148,5 98 49 171,5 Abibumbye 72 252 Icyerekezo 90 60 30 105 Kopiki 1,5 1 5 17,5 Abemerimihigo 138 92 60 210 Abishyizehamwe Ngera 15 10 5 17,5 CCM Muganza 111 74 80 280 Kaiimu 135 90 60 210 Kopianya 10,5 7 3,5 12,25 Kobabomango 18 12 6 21 Koabiwa 12 8 8 28 Koaimu 36 24 70 245 Kobarwo 120 80 40 140 Coamagi 120 80 120 420 total 1195,5 796 728,5 2549,75 Cooperative’s name sales 2017 (MT) buyer 2017 sales 2018 (MT) buyer 2018 Sales 2019 (MT) buyer 2019 Coamanya Gishubi 60 AIF/LOCAL 128 AIF 200 AIF Duterimbere ? LOCAL 21 AIF/LOCAL 70 AIF Abishyizehamwe Urwonja 16 LOCAL 29 AIF 36,5 AIF Abibumbye ? LOCAL ? LOCAL 20 AIF Icyerekezo ? ? 37 AIF 40 EAX/MILLER Kopiki ? LOCAL ? LOCAL ? LOCAL Abemerimihigo ? LOCAL 28,8 AIF ? AIF Abishyizehamwe Ngera ? LOCAL 9 AIF 14 AIF CCM Muganza 29,3 LOCAL 34,12 AIF ? AIF Kaiimu ? LOCAL 17,4 AIF 0 Kopianya ? LOCAL 21,15 AIF 17,3 AIF/MILLER Kobabomango ? LOCAL 1,7 AIF ? AIF Endline Report - August 2019 (draft) | P a g e 107 Kobabomango yes yes yes yes no yes Endline Report - August 2019 (draft) | P a g e 108 • Services received by cooperatives ‘members Figure 20 : Analysis of the evolution of services received by cooperative members (quantitative data collection) Variables Female Male Total Baseline Final p value Baseline Final p value Baseline Final p value Training 71.3% 40.4% <0.001 73.1% 38.1% <0.001 72.3% 39.3% <0.001 Credit facilities/loans 7.0% 5.8% 0.342 5.9% 5.3% 0.37 6.4% 5.6% 0.46 Supplying tools 8.8% 8.0% 0.451 9.9% 9.6% 0.411 9.4% 8.8% 0.674 Linkages with banks to access credit 3.6% 2.9% 0.389 4.6% 5.3% 0.302 4.2% 4.1% 0.826 Supplying other agricultural inputs (bags) 5.4% 9.7% 0.025 5.7% 12.9% <0.001 5.6% 11.4% <0.001 Extension services to members 32.8% 26.8% 0.035 27.5% 31.2% 0.105 30.0% 29.0% 0.685 Purchasing maize and beans from members 5.4% 39.9% <0.001 7.9% 41.2% <0.001 6.8% 40.6% <0.001 Supplying seeds 63.0% 56.4% 0.034 58.6% 56.1% 0.513 60.6% 56.3% 0.083 Providing market information 16.8% 21.2% 0.135 19.2% 24.0% 0.034 18.1% 22.6% 0.018 Supplying fertilizer and other chemicals 53.7% 64.2% 0.005 54.0% 60.2% 0.026 53.9% 62.2% <0.001 Providing collateral for loans/credit facilities 4.1% 0.0% <0.001 2.2% 0.5% 0.022 3.1% 0.2% <0.001 Linkage with extension service providers 23.3% 13.9% <0.001 23.6% 11.8% <0.001 23.4% 12.8% <0.001 Endline Report - August 2019 (draft) | P a g e 109 Annex 8. Have the LRP project interventions affected male and female SHF knowledge, capacity and choices/behaviour from baseline to endline? 8.1. Capacity and knowledge • Extension Table 9: Analysis of extension services received by SHFs (quantitative survey) Variables Female Male Total Baseline Final p value Baseline Final p value Baseline Final p value Received agricultural extension services in the past planting season 69.3% 74.5% 0.102 74.7% 77.2% 0.380 72.2% 75.8% 0.088 From whom farmers receive extension services WFP/other UN agencies 13.1% 46.1% <0.001 9.7% 42.5% <0.001 11.2% 44.3% <0.001 Government extension officers 79.5% 64.1% <0.001 77.6% 68.9% 0.012 78.4% 66.6% <0.001 NGOs 32.8% 29.1% 0.332 34.8% 28.9% 0.102 33.9% 29.0% 0.061 CBOs 43.3% 32.0% 0.005 45.4% 38.2% 0.06 44.5% 35.2% 0.001 In the last 6 months, received visits from extension services agents 94.4% 97.7% 0.039 96.5% 98.1% 0.186 95.6% 97.9% 0.018 • GAP and PHHS training Table 10 : Number of SHFs trained on GAP and PHHS (WFP/RWARRI) Cooperatives total trained on GAP Women trained on GAP total trained on PHHS women trained on PHHS CCM MUGANZA 151 46 197 97 COAMANYA GISHUBI 354 116 347 186 COAMAGI 269 124 327 113 COAMANYA GISHUBI 354 116 0 0 KAIIMU 405 207 448 220 KOAIMU INGENZI 164 81 161 83 KOPIKI 207 125 154 113 ABEMERIMIHIGO 471 250 521 304 ICYEREKEZO SIMBI 270 138 270 125 ABISHYIZEHAMWE CYANIKA 0 0 662 365 ABISHYIZEHAMWE NGERA 68 48 68 37 ABISHYIZEHAMWE URWONJYA 531 261 660 342 DUTERIMBERE 425 251 944 396 KOABIWA 150 90 148 92 KOBABOMANGO 57 36 57 24 KOBARWO 302 164 295 95 KOPIANYA 52 29 51 28 Endline Report - August 2019 (draft) | P a g e 110 Table 11: Analysis of the training received by SHFs (quantitative survey) Variables Female Male Total Baseline (n=387) Final (n=411) p value Baseline (n=454) Final (n=417) p value Baseline (n=841) Final (n=828) p value Trained on the following good agricultural practices and post￾harvest handling Seed selection 86.3% 94.2% <0.001 88.1% 96.6% <0.001 87.3% 95.4% <0.001 Planting and spacing 94.8% 94.9% 0.970 96.5% 96.9% 0.738 95.7% 95.9% 0.859 Use of fertilizer 96.1% 94.9% 0.401 96.3% 96.9% 0.612 96.2% 95.9% 0.752 Factors affecting grain quality 83.7% 92.0% <0.001 87.9% 95.2% <0.001 86.0% 93.6% <0.001 When to harvest 88.1% 94.4% 0.002 90.5% 95.9% 0.002 89.4% 95.2% <0.001 Ways to minimize grain infestation 81.7% 93.2% <0.001 85.7% 95.7% <0.001 83.8% 94.4% <0.001 Grain grade 66.7% 89.1% <0.001 67.0% 90.9% <0.001 66.8% 90.0% <0.001 How to clean grains before storage 84.5% 94.6% <0.001 87.4% 96.6% <0.001 86.1% 95.7% <0.001 How to avoid damages to grain during threshing and shelling 83.7% 93.7% <0.001 87.7% 96.2% <0.001 85.9% 94.9% <0.001 Precaution for good drying 84.8% 94.4% <0.001 88.8% 96.6% <0.001 86.9% 95.5% <0.001 Good practice for storage 81.1% 94.4% <0.001 86.6% 96.6% <0.001 84.1% 95.5% <0.001 Overall training score on good agricultural practices and post￾harvest handling (%) <25% 4.7% 4.9% <0.001 2.4% 2.9% <0.001 3.4% 3.9% <0.001 25 to <50% 6.7% 0.2% 5.3% 0.5% 5.9% 0.4% 50 to <75% 12.1% 0.7% 12.8% 0.2% 12.5% 0.5% 75 to <100% 22.0% 10.0% 21.1% 9.1% 21.5% 9.5% 100% 54.5% 84.2% 58.4% 87.3% 56.6% 85.7% Endline Report - August 2019 (draft) | P a g e 111 • Adoption of practices (mVAM results) A survey on the adoption of GAP practices was done in 2019. 386 farmers have been interviewed by phone of which 85 were lead farmers. They mentioned that 78% of their group members were applying GAP. The main results from the individual interviews are presented in the table below. Table 12a : Adoption of GAP practices (mVAM results) Practice Adoption rate Use of improved seeds 100% Right dosage of seeds 59% Row planting 99% Correct spacing between rows 58.8% Correct spacing between seeds 21.4% Use of chemical fertilizers 100% Percentage of the correct dosage applied 64% Use of organic fertilizers 100% Use of crop protection products 87% Percentage of farmers that dress differently to apply crop protection products 83.2% Use of gloves to apply 29.3% A survey was done in 2018 on the adoption of post-harvest practices. 380 farmers were interviewed. 91 were lead farmers and mentioned that 70% of the members of their groups were applying the practices taught. The main results are presented in the table below. Table 12b : Adoption of GAP practices (2018 survey) Practice Adoption rate Sorting of cobs 89% Use of hermetic bags 53% Use of tarpaulin for harvest 45% Avoid grains touching the soil during shelling 99% Use of tarpaulins for shelling 93% Use of improved methodology to check moisture 49% 8.2 SHF choice for maize and bean cultivation Overall, maize planting improved significantly for females during season A (improved by 10.2%; p=0.002) and season B (improved by 16.8%; p<0.001); but not for males during season A (improved by 2.8%; p=0.337) and season B (improved by 5.1%; p=0.137). Generally, beans planting was most preferred by farmers than maize. However, there was no significant change in beans planting females Endline Report - August 2019 (draft) | P a g e 112 during season A (decrease by 2.6%; p=0.238) and season B (changed by 0.1%; p=0.984). The results were similar for males during season A (decrease by 1.4%; p=0.455) but significantly improved during season B (improved by 5.2%; p=0.026). Figure 30 : Percentage of farmers planting maize, comparing baseline and endline for season A and B (quantitative survey) Figure 31- Percentage of farmers planting beans, comparing baseline and final for season A and season B (quantitative survey) The table below presents the results of mean hectares used for planting maize and beans. Overall, there was reduction (though insignificant) in the size of land use for planting maize between baseline and final in both season A (p=0.115) and B (p=0.648). The size of land for beans planting was insignificantly reduced for females in both season A (p=0.491) and B (p=0.431) but significantly reduced for males in both season A (p=0.002) and B (p=0.002). 62.5% 72.7% 42.6% 59.4% 72.7% 75.5% 46.0% 51.1% 68.0% 74.2% 44.5% 55.2% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Baseline Final Baseline Final Baseline Final Baseline Final Baseline Final Baseline Final Season A Season B Season A Season B Season A Season B Female Male Total 90.4% 87.8% 82.4% 82.5% 92.3% 90.9% 83.5% 88.7% 91.4% 89.4% 83.0% 85.6% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Baseline Final Baseline Final Baseline Final Baseline Final Baseline Final Baseline Final Season A Season B Season A Season B Season A Season B Female Male Total Endline Report - August 2019 (draft) | P a g e 113 Table 13: Mean hectares for maize and beans planting (quantitative survey) Variables Female Male Total Baseline Final p value Baseline Final p value Baseline Final p value Mean hectares for maize planting Season A 0.0995 0.1270 0.297 0.1962 0.1314 0.002 0.1553 0.1293 0.115 Season B 0.1049 0.0925 0.418 0.1613 0.2191 0.357 0.1364 0.1515 0.648 Mean hectares for beans planting Season A 0.1248 0.1158 0.491 0.2329 0.1627 0.002 0.1837 0.1398 0.001 Season B 0.1268 0.1502 0.431 0.2063 0.1469 0.002 0.1700 0.1485 0.211 Endline Report - August 2019 (draft) | P a g e 114 8.3 SHFs investments in short term productive assets • Agriculture inputs Figure 32 : SHFs source of fertilizers (quantitative data collection) • Post-harvest equipment Table 14 : PHE received by cooperatives (WFP) Hermetic Bags Plastic Silos Tarpaulins Number of beneficiaries CCM MUGANZA 5 130 79 COAMANYA GISHUBI 10 4 290 272 COAMAGI 243 3 272 247 KAIIMU 275 8 213 169 KOAIMU INGENZI 170 7 66 84 KOPIKI 256 281 204 ABEMERIMIHIGO 840 2 440 572 ICYEREKEZO SIMBI 305 1 42 123 ABIIBUMBIYEHAMWE CYANIKA 1860 7 693 126 ABISHYIZEHAMWE NGERA 71 2 95 71 ABISHYIZEHAMWE URWONJYA 790 10 468 277 DUTERIMBERE 740 4 480 300 KOABIWA 120 2 241 237 KOBABOMANGO 65 4 36 35 KOBARWO (Jyambere Muhinzi Kibeho) 290 3 297 164 KOPIANYA 95 6 75 85 Endline Report - August 2019 (draft) | P a g e 115 Concerning bags and tarpaulins, the amount of equipment received by cooperatives is relatively high (on average 409 bags per cooperatives and 257 tarpaulins). But quantities vary significantly from one cooperative to another (from no bags received to 1860 bags and from no tarpaulinsto 693 tarpaulins). The number of silos distributed is limited (on average each cooperative received 4.5 silos). The most appreciated equipment is the tarpaulin: its use is now widespread amongst farmers that use it to continue drying the maize after cobs have been put to dry on ladders in drying grounds and after shelling. Using it during harvest is less frequent. Hermetic bags are used by farmers to store maize grains at home. They consider it is very useful to protect grains from infestation and to increase duration of storage. Silos are also considered useful, but a lot of farmers cannot afford them. The size is also very big (500L) in comparison of what farmers usually keep at home and silos sometimes do not fit in houses. Overall the price is considered good and farmers appreciated the promotional offers (free tarpaulins and bags included) but several farmers mentioned that the purchasing period was not adapted (they were sold at a time when they had no cash in hand) Six cooperatives mentioned that they could not get enough equipment to satisfy demand by their members. There is a contradiction with WFP’s experience in distribution (WFP had to ask for an extension of project duration in order to reach its target). This can probably be explained by the careful attitude of many SHFs. They were waiting to see results at farmer’s level before investing themselves. Endline Report - August 2019 (draft) | P a g e 116 Annex 9. Additional information on the question: Has the LRP contributed to creating new market opportunities for male and female SHF? (including from schools) 9.1. Information on the maize sector in Rwanda Figure 21: Evolution of maize production in Rwanda (NISR annual surveys) Season A 2019 Season A 2018 Season B 2018 Season A 2017 Season B 2017 Area cultivated (ha) 215,159 218,179 78,151 210,609 Production (MT) 331,090 332,670 91,534 324,368 85,912 The area cultivated with maize has multiplied by two since 2007 (Musabangi, 2017). Yield and production have increased with the implementation of the Crop intensification Program (CIP) and the land consolidation programme. The GoR has invested in import and subsidies of inputs. However, this has recently changed for the seeds as the government now wants to produce maize seeds in the country. The national maize market is located in Kigali. Exports are very limited (South Sudan, Democratic Republic of the Congo (DRC) and Burundi). Rwanda’s potential market for maize is large. In 2012, the demand was estimated around 550,000MT and demand is still increasing (it is increasing more in urban areas). In terms of the East Africa Region, the overall domestic consumption has increased from 7,278,000 MT in 2000 to 10,215,000 MT in 2012. The outletsfor maize production are the food market (maize flour), beer market (maize grits) and animal feeds (maize bran). The GoR has implemented a floor price for maize in order to protect producers from price collapses and to stimulate production. Since 2014, at the end of each season, MINICOM gathers representatives from cooperatives, millers and buyers to discuss and agree on a floor price. This price is fixed based on production costs. Until last year, Uganda was the main importer of maize from Rwanda (it represented up to 90 percent of the maize imported), followed by Tanzania. Since February 2019, the border between Rwanda and Uganda is closed which blocks import from Uganda. There is not yet clear information on the consequences of the closure for the maize sector. Interviews with buyers at endline suggested that purchasing has moved to more local maize but also to import from other countries (mainly Zambia). Endline Report - August 2019 (draft) | P a g e 117 9.2 Sales from LRP cooperatives to formal buyers Figure 22 : Delivery of produce from LRP cooperatives to the formal market (RWARRI) There are five buyers that are members of the FTMA and a social enterprise that supports collection, shelling and drying of maize: Africa Improved Foods Africa Improved Foods (AIF) is a public-private partnership involving DSM, Government of Rwanda, International Finance Cooperation (IFC), CDC Group and FMO. AIF provides a scalable and sustainable solution to malnutrition via local production of highly nutritious foods. 65 million USD has been invested in Rwanda already in best-in-class technology, which is operational since December 2016. AIF is a social enterprise and embedded in its business model is a comprehensive strategy to reduce poverty, create jobs and address stunting and malnutrition through partnerships with non-profit institutions, such as WFP and Governments, as well as making affordable commercial products for the mass market. AIF is employing over 300 staff. The plant has an annual production capacity of 45,000 tons, making it one of the largest nutrition factory sites on the African continent. AIF works with over 24,000 farmers across Rwanda and produces meals for 1.5 million people every day. The majority of AIF’s production capacity is allocated towards the production of Super Cereal Plus for WFP. In 2018, 51 percent of the sourcing was local (through cooperatives and buyers), with the rest coming from Zambia. AIF has implemented the cob model in Rwanda through services provided by Kumwe (see below). FO Signed tonnage 2018 A Quantity delivered 2018A % of achivement Signed tonnage 2019A Quantity delivere d 2019A % of achievement CCM MUGANZA 40 34,127 85% 50 10,9 22% COAMANYA GISHUBI 100 87,859 88% 30 34,7 116% COAMAGI 50 172,5 345% KAIIMU 10 14,18 142% 20 0% KOAIMU INGENZI 5 2,96 59% 10 0% KOPIKI 10 0% ABEMERIMIHIGO 30 27,885 93% ICYEREKEZO SIMBI 30 0% 20 9,769 49% ABIIBUMBIYEHAMWE CYANIKA 40 0% 20 ABISHYIZEHAMWE NGERA 5 9,054 181% 12 11,82 99% ABISHYIZEHAMWE URWONJYA 50 29,394 59% 50 36,132 72% DUTERIMBERE 15 19,007 127% 15 35,036 234% KOABIWA 20 15 KOBABOMANGO 3 1,3 43% 10 3 30% KOBARWO (Jyambere Muhinzi Kibeho) 40 32 80% 50 74,338 149% KOPIANYA 21 21,886 104% 23 9 39% Produce delivery to formal market Endline Report - August 2019 (draft) | P a g e 118 Minimex-Prodev Minimex started operating in 2007 and PRODEV in 2012. Prodex and Minimex are sister companies: PRODEV is in charge of sourcing and Minimex does processing. Minimex is producing maize flour, fortified maize flour, maize grit and maize bran. They process between 28,000 MT and 30,000 MT per year, but their total processing capacity is 45,000 MT and their storage capacity is 15,000 MT. In 2018, 45% of their sourcing was local. Minimex sells its products to WFP, NGOS, government institutions (prisons for example), brewing industries and final consumers. They are a member of FTMA. East Africa Exchange East Africa Exchange (EAX) is a regional commodity exchange offering commodity trade services in Rwanda and East Africa. The company was created in 2013 to further strengthen EAC regional integration by developing a common and coherent financial sector in agriculture, energy and mining. In Rwanda it mainly trades maize and beans. They are operating a warehouse electronic receipt system. Farmers from cooperatives working with EAX can get funding from several partner financial institutions. The farmers use electronic receipts issued by the commodity exchange as collateral. EAX works with about 20,000 farmers in Rwanda (187 cooperatives). In season A 2019, they only sourced maize from cooperatives (1,500 MT). Their main clients are Minimex, AIF and medium-scale millers. EAX is a member of FTMA. Kumwe Kumwe Harvest is a social enterprise based in Rwanda, founded to address long-standing quality issues in Rwanda’s maize value chain. To help increase the quantities of maize purchased by agro-processors in Rwanda, Kumwe Harvest is working in partnership with AIF to introduce a new way of processing maize post-harvest in Rwanda. Utilizing our expertise in trucking and logistics, Kumwe started in 2017 by establishing mobile collection centres at farms on behalf of AIF. Using mobile shelling machines, harvest maize is shelled and transported to AIF for industrial drying and storage in the same day. By eliminating on-farm post-harvest processes, the harvest to market timeline is reduced from two months to two days and 100% of maize is accepted by AIF. Undertaking a larger, revised pilot in Season A 2018, Kumwe Harvest is maintaining the quality, speed, and payment benefits of the mobile collection centres from 2017, buying unshelled maize from farmers and transporting it to a central facility in Kigali to be shelled. Kumwe Harvest has processed and sold 4,000 MT of maize to commercial buyers in 2018. The model maintains the quality, speed and payment benefits of the mobile collection centres introduced in season A 2018, Kumwe Harvest will undertake a larger-scale pilot of this model with support from AIF, the Clinton Health Access Initiative, and World Vision Endline Report - August 2019 (draft) | P a g e 119 Annex 10. Additional information on the question: To what extent do producers market food products that meet quality standards and are nutritious and culturally accepted? 10.1 Evolution of the quality of maize Figure 23: Percentage of farmers by household preference on characteristics of the maize and beans (quantitative survey) 10.2. Interest of the crops promoted in school feeding In support of the drive by MINEDUC to further develop a national school feeding programme40, WFP has made an assessment of various nutritious school feeding menus and contributed to the formulation of a pro-smallholder farmer procurement strategy for school feeding in Rwanda. One of the objectives was to provide nutrition sensitive information on various school meals options that could be taken into consideration in a school feeding program. In order to reach this objective, studies were organized to propose a range of viable, nutritious menu options for both pre-primary, primary and secondary aged students, at a maximum cost of 150 RWF per serving, and based on food prices and availability within all five provinces in Rwanda. The study analysed nutrition data of two food baskets representing the main school feedings programs operating in Rwanda (HGSF programme and the ‘one cup of milk program’). This study is interesting for the evaluation as it gives indications on how LRP’s intervention can contribute to a better nutrition for school children. Nutrition analysis of a daily meal consisting of 120g maize meal, 30g beans and 15g vegetable oil When offering students, a meal of 120g maize meal, 30g beans and 15g vegetable oil, the general nutrition trend is that while protein and fat content are acceptable, especially for pre-primary 40 There is a government's school feeding programme (a third programme in addition to HGSF and Once Cup of Milk) which consists of direct cash transfer to schools with no set menus. Endline Report - August 2019 (draft) | P a g e 120 students, the meal is low in energy, especially for the daily nutrient requirements of primary and secondary students. Micronutrients level is not optimal. Figure 24 - Nutrition analysis of a daily meal consisting of 120g maize meal, 30g beans and 15g vegetable oil The study then modelled several varieties of meals in order to find out the best viable option for school feeding programs in Rwanda. The study clearly shows that there are various well-balanced nutritious meal options based on fresh food items alone, without adding any fortified food items. The more diversified the menu is, the more diverse and well-balanced the nutrient intake tends to become. It also shows that there are locally available options that can contribute to the nutritional balance of a meal (dodo leaves, cassava leaves, pineapples…). In order to implement a well-adapted school feeding program, the study recommends creating a range of menu options based on local agricultural potentialities and prices in local markets. Maize and beans are included in most of the proposed options. Therefore, improving maize (and bean) production and aggregation does make sense in terms of developing a national school feeding system, and particularly so if the program aims at procuring food locally. The pro-small holder procurement school feeding program strategy paper is taking this direction. It is not yet adopted, but it clearly shows that there is a potential for FTMA (and LRP) cooperatives to become suppliers of maize (and beans) for schools. LRP’s intervention may also pave the way to involving cooperatives in supplying other food products, which will contribute to the implementation of diversified school meals options. The LRP project has sensitized cooperatives on quality, aggregation and marketing of their products and they have experienced the positive effect that sales can have on their income. There is an opportunity here to start working with the same cooperatives on other products because farmers cropping systems are already very diversified (which is a key element of the food security of Rwanda). This would require a careful approach and a thorough understanding of farmer cropping systems (both on marshland and hillsides) to improve production and efficiency of the supply of food products without hampering their food security (the latter is ensured to a large extend by mixed cropping which ensures production throughout the year and thus reduces risks). Endline Report - August 2019 (draft) | P a g e 121 Annex 11. Has the LRP programme affected male and female SHF income and in what way? Figure 25: Net margin/ha for maize cultivation based on different cropping systems, price and buyers (qualitative data collection) Cropping system Production Buyer/type of purchase Price Net margin/ha (including family labour) Net margin/ha (excluding family labour) GAP/PHHS Improved seeds Appropriate dosage of fertilisers/manure 3.5 MT/ha FTMA buyer Grains 210 RWF/kg -75,000 RWF 355,000RWF GAP/PHHS Improved seeds Appropriate dosage of fertilisers/manure 3.5 MT/ha FTMA buyer Grains 240 RWF/kg 30,000 RWF 460,000 RWF GAP/PHHS Improved seeds Appropriate dosage of fertilisers/manure 3.5 MT/ha FTMA buyer Grains 270 RWF/kg 135,000 RWF 565,000 RWF GAP/PHHS Improved seeds Appropriate dosage of fertilisers/manure 3.5 MT/ha middlemen Grains 190 RWF/kg -145,000 RWF 285,000 RWF GAP/PHHS Improved seeds Partial dose of fertilizers 2 MT/ha FTMA buyer Grains 210 RWF/kg -212,000 RWF/kg 96,000 RWF/kg GAP/PHHS Improved seeds Partial dose of fertilizers 2 MT/ha FTMA buyer Grains 240 RWF/kg -152,000 RWF 156,000RWF GAP/PHHS Improved seeds Partial dose of fertilizers 2 MT/ha FTMA buyer Grains 270 RWF/kg -92,000 RWF 216,000RWF GAP/PHHS Improved seeds Partial dose of fertilizers 2 MT/ha Middlemen grains 190 -252,000 RWF 56,000RWF GAP/PHHS Improved seeds Appropriate dosage of fertilizers 4,540 MT/ha FTMA buyer Cobs 190 -3,050 RWF/kg 13,710 RWF/kg Table 15 : how SHFs are using the profit generated my maize (FGDs) Improved for a majority of members Possible for some members Possible for a few members only Use of the profit made from the sales of maize - Purchasing inputs for the coming agriculture season - Paying for health insurance -paying school fees -purchasing new clothes -purchasing small animals -improving the housing -purchasing equipment -increasing savings Endline Report - August 2019 (draft) | P a g e 122 Annex 12. Assessment of the project’s performance looking at the monitoring indicators The table below has compiled WFP’s own monitoring data. As noted in paragraph 99, there is no direct link between LRP produce from cooperatives and the purchasing of food for schools i.e. it is not possible to demonstrate that food purchased by WFP from the centralized Minimax facility was supplied by LRP cooperatives. Nonetheless we have maintained the indicator “number of individuals benefitting directly from USDA funded intervention” in the table below (the first line) as this reflects what WFP has reported and reflects use of LRP funds for the purchasing of food for schools. Indicator target result Percentage of achievement Comments Number of individuals benefiting directly from USDA-funded intervention 46,780 school children 43,855 school children 93.7% In 2019, less school children benefited from HGSF (41,521 children). Several factors explain this change: larger outgoing primary class 6 children compared with in-coming primary grade 1 children, new schools established in the catchment area, and students transfer to other schools. In 2020, LRP did not financially contribute to the HGSF programme. 5,133 SHFs 5,617 SHFs 109% Target is achieved. However, the number of beneficiaries should be updated considering changes in membership of some cooperatives. Number of individuals benefiting indirectly from USDA-funded intervention 75,651 70,345 93% The number also decreased in 2019 due to the decrease of direct beneficiaries Value of sales by project beneficiaries 280,000 164,244 58.6% The figure will probably evolve as it was calculated before the end of the sales season in 2019 Volumes (MT) of commodities sold by project’s beneficiaries 643 686 106% Overall, cooperatives supplied more maize than what was contracted Number of public￾private partnership formed as a result of USDA assistance 16 16 100% This indicator does not seem very clear when mentioning cooperatives. LRP team measured it by number of cooperatives that received training on governance. Value of public-private sector investment leveraged as a result of USDA assistance NA NA NA Total increase in installed storage capacity as a result of USDA assistance 1000 796 79.6% LRP is extended in order to continue supply of PHHS equipment to farmers Number of policies, regulations and/or administrative procedures in each of the following stages of development as a result of USDA assistance 1 1 100% Elaboration of a pro-SHH school feeding strategy. Not yet adopted and operationalized Quantity of commodity procured (MT) as a result of USDA assistance (by 800 MT of maize 250 MT of beans 800 MT of beans 190 MT of beans 94.3 % Procurement was not made from LRP cooperatives but is from SHFs. Endline Report - August 2019 (draft) | P a g e 123 Indicator target result Percentage of achievement Comments commodity and source country) 25 MT of salt Cost of transport, storage, and handling of commodity procured as a result of USDA assistance (by commodity) Target reached (but inconsistency in the figures) Number of social assistance beneficiaries participating in productive safety net as a result of USDA assistance 4014 3864 96% The target is about to be reached as supplies of PHHE are still on going. Number of individuals who have received short-term agricultural sector productivity or food security training as a result of USDA assistance 2000 4765 238% Participation was excellent for GAP and PHHS training. Percentage of default rate of WFP pro-smallholder farmer procurement contracts, disaggregated by reason and aggregation system 5 0 0% Overall quantity of maize supplied exceeded the target, but the situation is not the same in all the cooperatives. Several cooperatives actually defaulted but they were not directly contracted by WFP. Number of USDA￾supported aggregation systems that have access to credit from formal financial institutions 5 2 40% Access to credit remain a limiting factor for cooperatives. However, savings groups have been implemented with success in several cooperatives Percentage of milling facility operators who demonstrate proper application of technologies and practices as a result of USDA assistance 100 NA NA Percent of farmers that apply improved post￾harvest practices as a result of USDA assistance 75% 98% 130% This figure comes from the mVAM survey done for FTMA (not LRP specific). Endline Report - August 2019 (draft) | P a g e 124 Annex 13. Interview and FGD topic guidelines Semi-structured Interview Guidelines Cooperative leaders KII Presentation of the cooperative Managing structures Membership and its evolution /rules to be a member Staff? Main changes in the last 2 years? Cooperative identity Activities of your cooperative What do you want to achieve as a cooperative/vision How do you see your coop in 5 years/10 years Strategy? Documents Governance Decision making process/Roles of the executives/meetings of the comitee Last elections? Renewal of some executives? Last general assembly? How was it organized? What decisions were taken? Can the cooperative respect the requirements from its status? Why? Roles and tasks of the district/sector agronome? Other support? Main changes in the last two years in the cooperative’s governance? Difficulties faced? Place and roles of women in the cooperatives/ in the executive committee Members participation and commitments to the cooperative’s activities Interest of members in the cooperative (for what? ) and how it has changed ? How easy is it to mobilize farmers? Why ? any constraints? How to improve it? Communication between executives/members Are there meetings between members? Members and executives? (Frequency/purpose/records…) Endline Report - August 2019 (draft) | P a g e 125 Cooperative activity planning and organization Planning of activities (how, who, constraints, changes, plot attribution…) Implementation of activities (how, when, who, controls, constraints, changes…) Roles/tasks of the agronome/other partner Organization of the work between members (conflicts?) Records? challenges in term of work planning, organizations and records? What has changed in the last 2 years? Do you think the cooperative is better organized? Why? What are the services you provide to your member? Changes? Plans? Resources of the cooperatives What are the resources of the cooperative? (shares/dues/other) + evolution Are they sufficient to conduct the cooperatives’ activities? Why? Do you plan to increase them (how?) number of members that paid their share? Evolution? Why? number of members that paid dues? Evolution? Why? Commission on sales? Evolutions? Amount collected? Sales of inputs? Internal loans? Material resource/building? Evolution (with which funds? ) What are the main changes in the cooperatives resources in the last 2 years? Why? Access to finance How easy is it for your cooperative to access credit? What is your experience with credit ? Is it something you want to develop? What credit facilities can your cooperative access? From who? Conditions? Have you received support to access credit? on what? How did it help the cooperative? What are the main constraints about credit/access to credit? Difficulties with repayment? Individual credit to members : do you support your member to have credits? Do you provide them loans (how it works? ) Self help groups in the cooperatives? What has changed in the last 2 years for the access to credit? Planning of the production/results Objectives? (planned how?) How do you organize the collection from members? (purchased or intermediation with buyers?) why? Endline Report - August 2019 (draft) | P a g e 126 Organization of purchase (conditions/time/information/price/quality…) Quantity of products collected/sold+ evolution? Number of members that sell to/through the cooperative? Constraints? Evolutions in the last 2 years? Why? Was the activity profitable? Why? How do you know? Post harvest handling Are you organizing post harvest services for your members? What are you doing? Why? For members that don’t do it at cooperative level, where do they do it? What does that change? What are the main constraints that your cooperative face for post harvest handling? What has changes over the last 2 years? Storage Does your cooperative offer storage services to producers? Conditions/costs? DO you have enough storage capacities? Why? How long do you store? Why? Why are some members not storing at the cooperative level? What are the main constraints and problems of storage? Post harvest losses? How to control? %? What has changes in the last 2 years? Quality Is quality important for maize/bean? What are the quality criteria? How easy or hard is it for producers to respect these criteria? What is feasible for producers ? Is there a market for quality products? Do you promote quality with your members? How? Is it interesting for them? Is it more difficult to produce quality maize? Is it interesting to produce quality (in terms of work, cost, added value…)? What is the interest of producing quality for producers? And for the cooperative? How do you monitor quality? Do you have a monitoring system? How does it work? What do you do if a producers maize/bean is not up to the standards? Are extension services looking at quality issues? (what issues? What extension services? Why? ) Relations with agrodealers Does the coop provide seeds/inputs to producers (members/non members) ? Conditions? Where do you get the seeds? How do you pay for the seeds? Are there any difficulties in the access to seeds/fertilizers/pesticides? Endline Report - August 2019 (draft) | P a g e 127 Any changes in the last 2 years? Extension service Who provides extension services to your members? How does it work? Adequate? Useful? Relation with government Do you have partnership with local authorities/agric service? Others Content of the partnership Adequate? How to improve? What is the role/ participation of the government in the cooperative activities and accountabilities? Taxes? Linkage with other cooperatives Knowledge of other coops/common interest? Associations of coop? discussions? Marketing Where are the main markets for maize/ beans? Do you purchase and sell the maize from members or organize for buyers to come? Why? Sales : Who do you sale the maize to? Group sales? Individual sales? Middlemen? Buyers? Schools? When did you sell maize from the coop? how did you decide to sell? Who decide? Are the sales conditions good (measures, price..) Are you negociating for group transport for your members? Cost/constraints of transport Who are the buyers? When are they coming? Are there times where there is no buyer? Objectives in term of sales? Last season? Next season? In 5 years? Do producers prefer selling individually or collectively? Why? Regular buyers/spot buyers? Marketing constraints Find buyers : What are you doing to find buyers? Who are your buyers? What is the role of the cooperative? Do you negotiate better conditions? Examples? What are the conditions of sales? How is organized? Are there time when you cannot find buyers? Experience with large buyers Endline Report - August 2019 (draft) | P a g e 128 Do you know any large buyers ? have you worked with them? Why? Where do these buyers purchase maize/beans? Experience with contract farming? Marketing contract? Interest/risks? Results? Contract conditions Constraints? Defaulting? price Price/price evolution/ how is the price fixed? Possible to negotiate? what makes the price go up or go down? Benefit Does your cooperative make profit? How do you know? What are the main expenses for your cooperative? How is your cooperative making profit? Where fdoes the profit comes from? Do you provided dividens to your members? Why? How are they calculated? Organization of payment to producers? + time? Market information How do you get info on market/price? How often? How do you have info? How do you use this information? Is there a market info system? Relevant? Reliable? Does the coop use it? How do you give the information to you members? How do they use it? What should be improved, what other information do they need? Partners Who are the partners of the coop? (Developing partners) How did they become partners? Since when/for how long? To do what Main achievement Quality of the partnership Trainings What training did your cooperative attended? Topics/when/ who? What capacity do you think you cooperative needs training on? Why? Knowledge on WFP/LRP What is it? What do you know about it? Semi-structured FGD Guidelines Endline Report - August 2019 (draft) | P a g e 129 Guidelines for producers (FGD ) Production Crops grown (Lowland/upland, cooperative land/own land, per season, mix cropping?) Contribution to income/feeding the family (change in the last 2 years?) Interest of maize and beans compared to other options Main constraints for maize/bean production Land Is it easy to access land (to buy/to rent/cooperative land)? Have you purchased/rent land in the last 2 years? Why? What are the conditions to grow on the land of the cooperative? Any change in the last 2 years? Are there cooperative members that cannot access the land? Why? Are there members of the cooperatives that use to grow on cooperative land and have stopped? Why? Labour/organization of work: Do you have enough workforce to work on your land/coop land? temporary labour? Own land/coop land (cost/availability..) Any change in the last 2 years? organization of the work on the cooperative land? How does it work, How is it controlled? Tools: What tools do you purchase? Any change in the last 2 years? Have you been able to purchase new type of tools in the last 2 years? Which one? Where do you get the tools from? Is it easy to find tools? Is your cooperative helping you to access tools? How? Access to tractors/animals for ploughing? Seeds (varieties, costs, supply…) (difference collective/own farm) Seeds availability? Cost/quality/fraud What are the main constraints in accessing seeds? Any changes in the last 2 years about access to seeds? Does your cooperative help you to access seeds? How? What seeds are supplied? How does it work? How and when do you pay them? Are they available on time? Fertilizers /chemical (collective farm/own farm) Availability? Quantity/quality/cost Can you always purchase the recommended doses? If not, what are you doing? Endline Report - August 2019 (draft) | P a g e 130 What are the main constraints for accessing/using fertilizers? Any change in the last two years? Same for Other chemicals (pesticide) Yield/production (collective/individual) Maize/bean/season A/B/C How do you see the yield compared with 2 years ago? Why? Good production practices/training Have you been trained on maize/bean prod? What have you learnt? Was is useful? Which practice do you think is the most useful? What about post harvest (grading?) From what you learn, what can you do in your farm? What can’t you do? What is difficult? Which one are you currently doing? Extension service What can you do when you are facing a constraint /questions related to agriculture What extension service do you receive? Modalities? From? What do you think about it? What do you do with the maize harvested ? (collective/individual)/ beans - Quantity sold (sold when?) - Quantity eaten? - Quantity kept for seeds? - Quantity given? Who decides quantity to keep/sell? Who sells? Who keeps the money? Quality of the maize/beans purchased/produced (criteria, price, varieties… ) Consumption Do you produce enough to feed your family? Purchase of maize and beans for the family consumption: When? Why? Why not keeping more of your production? If you compare the quantity of maize/bean you consume now and 2 years ago, are there any changes? Which one (same for maize/bean sold and purchased) How do you compare the quantity of maize/beans available on the market now and 2 years ago? Prices? How important is quality? What are the characteristic of the maize/beans you like to eat the more? Why? Are school purchasing maize/beans? (conditions/quality) Are you providing maize and beans to school? Why? Endline Report - August 2019 (draft) | P a g e 131 Post harvest (HH/cooperative) What/Why/organization/costs/time/who does it Can you store the maize/beans? Why Do you have facilities at home? At coop level? Does your cooperative offer a storage service, how it works? Who pays? How much? Storage constraints Losses? Any changes in the last 2 years in post harvest practices/losses/storage? Marketing strategy : How do you chose where you sale the maize/beans? Where? Why? Direct sales at the market Why do you sale at the market? Where is the market? How do you go there? How do you transport the maize? + costs Advantages/contraints of this marketing strategy Sales to middlemen/buyers What are the products you sell (fresh, dry…) Who are the buyers How do you find buyers? Are there times when you cannot find buyers? How does it work? Where/price/conditions Any changes in the last 2 years? Sales to/through cooperative Is your cooperative purchasing maize/beans? Is your cooperative helping you to sell maize/ bean? How? What is the interest of selling to/through the cooperative? Do you sell all your maize/bean to the cooperative? Why? Would you like to? Any change in the last 2 years? Why? How is It organized How do you bring the maize/bean to the cooperative? (+ cost of transport) How/when are you getting paid? Difference with other buyers? How Is the price negotiated by/offered by cooperative? How is it fixed? Is it a good price compared to market price? What are the conditions to sell to/through the cooperative? Can you sell when you are not a member? Is the price different? Endline Report - August 2019 (draft) | P a g e 132 Do you agree before on the quantity to provide? If you cannot provide this quantity what happens? And if you provide more what could the cooperative do better to help you deliver more maize and beans What is the quality of the maize that the cooperative wants? How is it controlled? Is it a quality that you can easily achieve? Why? What are the main constraints of selling the maize to/through the cooperative? Changes in the last 2 years Big buyers experience/contract Marketing strategy price of maize/beans + evolution What are the differences between selling to middlemen/cooperative/buyers/market Market info What information on markets/price do you have? Is it useful? Why? Any change in the last 2 years? Is the cooperative helping you having information on market? How? How to improve? Access to credit Do you know where to get credit? what do you think about credit? Have you been trained on credit management? Have you received credit? When? How? How much? LRP/WFP What do you know about it? Cooperative Are you a member of a cooperative? Which one? Since when? And your spouse? Why are you a member? What does It mean? Rules? What are the roles of women in the cooperative? Meetings? What for, why? when? Who are your leaders? How were there selected? What is their roles? Are there shares to pay to join the coop ? amount? Have you finished paying your shares? Do you pay dues? Amount. Are you up to date? Changes in the last 2 years What service do you receive from the cooperative? Any changes in the last 2 years? Is your cooperative profitable? What are the main changes in your cooperatives in the last 2 years? Impact: How has your income changed in the last 2 years? How? Why? Endline Report - August 2019 (draft) | P a g e 133 What other changes have you noticed? Gender Place and roles of womein in value chain Specific constraints? Support from the project? Endline Report - August 2019 (draft) | P a g e 134 Annex 14. Bibliography Agriculture Gender Strategy, Minagri, 2010. Alinda, Abbott, Agricultural Policy and Institutional framework for Transformation of Agriculture, Economic Development and Poverty Reduction in Rwanda, Institute of Policy Analysis and Research – Rwanda, 2012. Améliorer la position des agriculteurs familiaux sur les marchés en Afrique. Importance des organisations paysannes en Afrique de l’Ouest et de l’Est et recommandations politiques, Collectif sécurité alimentaire, 2014. Assessment of post-harvest opportunities in Rwanda, USAID, 2010. Climate Smart agriculture and sustainable intensification: assessment and priority setting for Rwanda, USAID, 2016 CoopAfrica working paper 12 : The hope for rural transformation: a rejuvenating cooperative movement in Rwanda, ILO, 2010 Huye district development plan, Huye district, 2012. Galtier F, David-Benz H, Subervie J, Egg J, 2014. Les systèmes d'information sur les marchés agricoles dans les pays en développement : nouveaux modèles, nouveaux impacts. Cah Agric 23: 245-58. doi: 0.1684/agr.2014.0715. Gisagara district développent plan, Gisagara district, 2012. ICCO-TERRAFINA project report (2018) Iram (2009). Sélection de références méthodologiques en analyse organisationnelle. Kiaya, Post-harvest losses and strategies to reduce them, ACF, 2014. LRP semi-annual reports to WFP (2017, 2018, 2019) Musabanganji, Contraintes et strategies d’amelioration de la filiere maïs au Rwanda, Dissertation originale présentée en vue de l’obtention du grade de Docteur, Université de Liège-Gembloux Agro-Bio Tech, 2017. Manuel de formation pour l’amélioration du traitement et du stockagedes grains après récolte, WFP, University of Greenwich National Gender Policy, Ministry of gender and family promotion, 2010. Nyamagabe district Development plan, Nyamagabe district, 2012. Nyaruguru district development plan, Nyaruguru district, 2012. Nutritious school meals options for a national school feeding programme, WFP, 2019 Orientations pour la construction de programmes de renforcement des capacités des organisations de producteurs, AVSF, 2014. Pedagogical materials on Farmers’ Organisations and Farmers’ Organisations’ support, CIRAD-CIEPAC, 2006. Endline Report - August 2019 (draft) | P a g e 135 Rwanda Comprehensive food security and vulnerability analysis, Minagri, NISR, WFP, 2015. Rwanda country strategic plan 2019-2013, WFP Rwanda country strategic review of food and nutrition strategy, WFP, 2018 Seasonal agriculture survey 2016, 2017,2018, season A 2019, National institute of statistic of Rwanda Strategy paper advancing small holder farmers procurement in school feeding in Rwanda, Interministerial task force, 2019 Verhofstadt Ellen, Maertens Miet, Can Agricultural Cooperatives Reduce Poverty? Heterogeneous Impact of Cooperative Membership on Farmers’ Welfare in Rwanda, Division of Bio economics, Department of Earth and Environmental Sciences, KU Leuven, 2014. Endline Report - August 2019 (draft) | P a g e 136 List of Acronyms AGRA Alliance for a Green Revolution in Africa AIF AFRICA Improved Food APTC Agro-processing Trust Corporation CFSVA Comprehensive Food Security and Vulnerability Analysis CIP Crop intensification Program CO Country Office CSB+ Corn Soya Blend Plus DRC Democratic Republic of Congo EAX East Africa Exchange EDPRS Economic Development and Poverty Reduction Strategy FAO Food and Agriculture Organization FGD Focus Group Discussion FTMA Farmer To Market Alliance FO Field officer GAP Good Agricultural Practices GDP Gross Domestic Product GHI Global Hunger Index GoR Government of Rwanda Ha Hectare HGSF Home Grown School Feeding ICCO Interchurch Organization for Development Cooperation IFAD International Fund for Agriculture Development IFC International Finance Corporation Kg Kilogram KII Key Informant Interview KOIKA Korean International Cooperation Agency LRP Local and Regional Procurement MDG Millennium Development Goal MFI Microfinance Institution MINAGRI Ministry of Agriculture and Animal Resources MINALOC Ministry of Local Government MINECOFIN Ministry of Finance and Economic Policy MINEDUC Ministry of Education Endline Report - August 2019 (draft) | P a g e 137 NGO Non-Governmental Organisation NISR National Institute for Statistic of Rwanda NST National Strategy for Transformation P4P Purchase for Progress PHHS Post-Harvest handling and Storage PHE Post-harvest equipment PPP Patient Procurement Platform PSTA Strategic Plan for the Transformation of Agriculture in Rwanda RAB Rwanda Agriculture Board RCA Rwanda Cooperative Agency RDO Rwanda Development Organisation RGCC Rwanda Grains and Cereals Corporation RIAS Rabobank International Advisory Services RWARRI Rwanda Rural Rehabilitation Initiative RWF Rwandese Franc MT Metric Ton SACCO Saving and Credit Cooperatives SHF Small Holder Farmer TOC Theory of Change TOR Terms of Reference UN United Nations UNEG United Nations Evaluation Group USAID United Nations Agency for International Development USD Dollar USDA United Stated Department of Agriculture WFP World Food Programme [Name of commissioning Office ] [Link to the website] b6/download/ https://docs.wfp.org/api/documents/cdcbe4be57914c65b0dbcbd201196a [Place, Month and Year, Report number]