MID TERM PERFORMANCE EVALUATION FEED THE FUTURE RWANDA HINGA WEZE ACTIVITY December 2020 This publication is made possible by the support of the American People through the United States Agency for International Development (USAID) and was prepared by Integra Government Services International LLC for the Learning, Evaluation, and Analysis Project (LEAP III). FINAL EVALUATION REPORT Mid Term Performance Evaluation of FEED THE FUTURE Rwanda Hinga Weze Activity Contract Title: LEAP III: Learning, Evaluation, and Analysis Project Contract Number: GS-10F-083CA / 7200AA18M0004 Activity Number: LEAP III 2020 – 1009.1042 Submitted: April 1, 2021 Contractor: Integra Government Services International LLC 1156 15th Street NW, Suite 800 Washington, D.C. 20005 USAID Office: USAID/Rwanda Prepared by: Deanna Gordon, Elin Cohen, Nathan Martinez, Sarah Carello, and Martin Hakizayezu DISCLAIMER This report is made possible by the support of the American people through USAID. The authors’ views expressed in this publication do not necessarily reflect the views of USAID or the United States Government. 3 TABLE OF CONTENTS LIST OF TABLES 4 LIST OF FIGURES 4 ACRONYMS 5 ACKNOWLEDGMENTS 7 EXECUTIVE SUMMARY 8 1. INTRODUCTION AND EVALUATION DESIGN 12 2. FINDINGS 19 3. CHALLENGES AND RECOMMENDATIONS 54 ANNEX A: SURVEY IMPLEMENTATION REPORT 59 ANNEX B: COMPLETE COST-BENEFIT ANALYSIS 72 ANNEX C: KEY INFORMANTS INTERVIEWED 97 ANNEX D: SUMMARY OF IRRIGATION SITES 103 REFERENCES 107 4 LIST OF TABLES TABLE 1 SURVEY RESPONDENTS BY TYPE AND DISTRICT 16 TABLE 2 MOST IMPORTANT SERVICES ACCESSED THROUGH HINGA WEZE 20 TABLE 3 INCREMENTAL PRODUCTION AND REVENUE FOR ONE HECTARE OF LAND 25 TABLE 4 COST OF INPUTS FOR TERRACING AMENDMENTS 36 TABLE 5 BENEFITS AND COSTS 40 TABLE 6 CHALLENGES AND RECOMMENDATIONS 58 LIST OF FIGURES FIGURE 1 MAIN BUYERS OF COMMODITIES BY HINGA WEZE VALUE CHAIN 21 FIGURE 2 FARM HOUSEHOLD INCREMENTAL NET PRESENT VALUE FOR 1.23 HECTARES OF MAIZE 41 5 ACRONYMS AAR - Activity Authorization Request CBA - Cost-benefit analysis CBV - Community Based Volunteer CDCS - Country Development Cooperation Strategy CEA - Cost-effectiveness analysis CFSVA - Comprehensive Food Security Vulnerability Analysis CHW - Community Health Worker CNFA - Cultivating New Frontiers in Agriculture COR - Contracting Officer’s Representative CRS - Catholic Relief Services DAP - Diammonium phosphate DEC - Development Experience Clearinghouse DHS - Demographic and Health Survey FGD - Focus Group Discussion FTF - Feed the Future GAP - Good agricultural practices GOR - Government of Rwanda IMTA - Irrigation Management Transfer Agreement ISOMACO - Irrigation Scheme Operation and Maintenance Committee KII - Key Informant Interview LEAP III - Learning, Evaluation, and Analysis Project III MINAGRI - Ministry of Agriculture and Animal Resources MTE - Midterm evaluation 6 MOU - Memorandum of Understanding NGO - Non-governmental organization NISR - National Institute of Statistics of Rwanda NPK - Nitrogen, phosphorus, and potassium NPV - Net present value NTSI - National Strategy for Transformation OFSP - Orange flesh sweet potato PPE - Personal protective equipment PSTA-4 - Strategic Plan for Agriculture Transformation 4 RAB - Rwanda Agricultural Board RDO - Rwanda Development Organization RWF - Rwandan Franc SACCO - Savings and Credit Cooperative SBC - Social and Behavioral Change SME - Small and Medium Enterprises SNS - Smart Nkuganire System STTA - Short-Term Technical Assistance USAID - United States Agency for International Development USG - United States Government 7 ACKNOWLEDGMENTS The evaluation team would like to thank the USAID/Rwanda Team and Feed the Future Rwanda Hinga Weze Team for their invaluable support throughout the course of this evaluation. The team would also like to thank Dalberg for developing and implementing the survey instrument during an extraordinarily difficult time for implementing field activities. Thanks also to Beate Uwera for her efforts on logistics for fieldwork and coordinating interviews. Thanks to Kaitlyn Turner for her technical review. Finally, thanks to David Quinn, Pin Ganyapak, and Isabella Cazier for their contributions to Quality Assurance, Operations, and Administrative activities respectively. 8 EXECUTIVE SUMMARY Growth in Rwanda’s agricultural sector, which is linked to a decline in overall poverty, has been robust in recent years. Between 2007 and 2014, food crop production growth was more than double the rate of population growth but despite these gains, agricultural productivity lags behind its full potential and malnutrition remains a persistent problem.1 The most recent Demographic and Health Survey for Rwanda, carried out in 2020, found the rate of persistent malnutrition (stunting) among children to be 33 percent, far more prevalent than acute malnutrition (wasting) which was found in one percent of children. A lack of market access, soil erosion, low fertilizer and improved seed use, minimal mechanization, and lack of knowledge of good agricultural practices (GAP) hinder productivity, while a lack of education about nutrition and proper child feeding practices constrain improvements to nutrition. Recurring extreme weather shocks and global climate change also pose serious challenges. Undernutrition remains pervasive. Despite a reduction in under-five stunting rates from 36.7 percent to 33 percent between 2015 and 2020 (DHS 2020, 2015), the rate remains unacceptably high. The Feed the Future (FTF)-funded Rwanda Hinga Weze Activity is addressing these challenges and advancing the Government of Rwanda's (GOR) Vision 2050 goal of "transforming agriculture into a sector that is market-driven, linked to urbanization and trade, and nearly 15 times more productive than today,” the U.S. Government’s Global Food Security Strategy’s goal of “sustainably reducing global hunger, malnutrition, and poverty,” the National Strategy for Transformation’s (NST1) goal of a “robust performance in the agriculture sector with 5.7 percent average growth per year,” and the Strategic Plan for Agriculture Transformation 4’s (PSTA-4) goal that “seeks the transformation of Rwandan agriculture for subsistence sector to a knowledge-based value creating sector, that contributes to the national economy and ensure food and nutrition security in a sustainable and resilient manner.” It is against this background that the United States Agency for International Development (USAID)/Rwanda requested that the LEAP III team undertake a mid-term performance evaluation and provide recommendations to USAID/Rwanda and the FTF Hinga Weze implementer, Cultivating New Frontiers in Agriculture (CNFA), on how to optimize the activity’s implementation to improve performance over the remaining life of the activity and also to help USAID determine what steps to take to sustain the activity’s positive outcomes. Hinga Weze is a $32.6 million USAID-funded activity focused on sustainably increasing smallholder farmers’ income, improving the nutritional status of women and children, and increasing the resilience of agriculture and food systems to the changing climate. The activity is being implemented in 10 districts over a five-year period from June 22, 2017 through June 22, 2022. Led by CNFA, the consortium is comprised of a diverse range of international and Rwandan partners including Plan International, Souktel, Rwanda Development Organization (RDO), and the Imbaraga Farmers’ Union. FTF’s Hinga Weze Activity works in ten districts throughout Rwanda including Gatsibo, Kayonza, Bugesera, and Ngoma in the Eastern Province; Nyabihu, Rutsiro, Ngororeo, Nyamasheke, and Karongi in 1Rwanda Demographic and Health Survey (2020) 9 the Western Province; and Nyamagabe in the Southern Province. The Activity has three objectives: 1) sustainably increase farmers’ incomes, 2) improve the nutritional status of women and children, and 3) increase the resilience of the agriculture and food systems in the face of a changing climate. The evaluation team used a mixed methods approach, combining a desk review, qualitative interviews, and quantitative data collection methods to answer the evaluation questions. A quantitative survey instrument was developed by the LEAP III team alongside Dalberg Research—the international consulting firm that administered the survey—targeting farmers in all ten districts where Hinga Weze is implemented (see Annex A for the complete implementation report.) After consulting with USAID/Rwanda and the available Hinga Weze reports and information, the team worked with Dalberg to select sites for the survey from across the ten districts in the Eastern, Western, and Southern Provinces. Key informant interviews (KIIs) and focus group discussions were also conducted in the field. It was clear from the sources consulted for this evaluation that Hinga Weze is highly regarded by all stakeholders and is providing well-targeted services. Some sustainability concerns were identified related to equipment ownership and maintenance for the irrigation schemes as well as access to inputs for the terracing schemes (addressed in the matrix of challenges and recommendations presented in Table 6 in Section 3). The evaluation addresses five questions, presented below, along with a brief summary of findings: 1. As a way of increasing farmers’ income, one approach used by Hinga Weze is to facilitate increased productivity and access to markets for smallholder farmers through building cooperatives’ market power in price negotiations with processors and aggregators. Hinga Weze also supports processors and aggregators to improve product quality and their capacity to procure from the farmers. What are the risks associated with these approaches and how can they be addressed during the rest of the activity lifetime? During the period of their involvement with Hinga Weze, how have farmers’ incomes changed? The analysis found no conflict between the approaches Hinga Weze has taken with regard to smallholder farmers and buyers. Survey data for this evaluation, along with KII and FGD responses, indicates that there are some ongoing challenges for cooperatives, particularly relating to management capacity, but also in understanding and accessing market information. While agro￾business clusters have proven to be a valuable mechanism for creating linkages between cooperatives, buyers, processors, and other key sectoral stakeholders, it is imperative for sustainability that the private sector take on a greater role in organizing and operating these clusters. While income, as reported in Hinga Weze monitoring data, has only increased in twenty percent of households, MTE survey data as well as KIIs and FGDs show an increase in food expenditures for the majority of households, which is a strong indicator of either reduced costs for other household expenses or increased income. 2. To what extent has Hinga Weze ensured that farmers or communities own terracing and irrigation schemes and also have plans and associated financial and human resources required for future sustainable maintenance of these productive assets? 10 While farmers indicate a clear understanding that they will eventually own irrigation and terracing equipment, the handover has not yet taken place and there is still some ambiguity about how those handovers will be structured. Maintenance and function of irrigation equipment is also an ongoing issue, with five out of seven sites reporting inadequate water. Many respondents cited solar pumps that only function during daylight hours and were not strong enough to pump sufficient water. Long waits for maintenance are another issue, indicating a need for capacity￾building within the irrigation cooperatives on performing maintenance. Because it appears that many terracing activities are in the initial stages, it is difficult to draw many conclusions about progress at this point. Only about 59 percent of farmers in these activities reported receiving training on maintenance, so there is a clear need to continue delivering that training. Inputs such as lime and fertilizer are critical to the sustainability of terraced land, but there does not seem to be much organization among farmers on how they will save money to pay for those inputs. These farmers will also need extension training that is specific to terraced land, as well as training on market linkages. It is the evaluation team’s understanding that Hinga Weze was in the process of planning and implementing these types of trainings at the time of this writing. 3. Hinga Weze has promoted a number of agricultural technologies aimed at increasing farmer-level productivity and climate resilience. Which are the top three technologies on a cost-benefit basis that would be recommended to continue to roll out at scale over the rest of the activity period? With the exception of planting/cultural practices, the average farmer is experiencing a net benefit, with those under the genetics intervention experiencing the greatest net benefit (NPV), ranging from 582,476 RWF ($611) to 718,682 RWF ($753). Farmers under some other intervention or combination of interventions (i.e. those not evaluated independently in this analysis), as well as those applying pesticides or soil conservation practices, are experiencing an NPV ranging from 105,608 ($112) to 276,965 ($290). Farmers participating in the planting/cultural practices experience an NPV of -16,888 RWF ($18). The primary drivers for this negative return are lower yields and higher hired labor costs. Hinga Weze farmers producing Irish potato have an incremental NPV of 1,236,477 ($512), while OFSP producers have an incremental NPV of 298,424 ($313). One of the key drivers for the large increase in returns for Irish potato producers is the estimated reduction in the hired labor needed for land preparation, weeding, and harvesting rather than large increases in yields. 4. To what extent do Hinga Weze’s agricultural productivity and market access interventions contribute to improved nutritional outcomes for women and children? In what ways do the current interventions address or not address the underlying constraints towards improved nutritional status of the target households? Activities in agricultural productivity and market access contribute to all four pillars of food security. Some income gains are used to purchase additional and more nutritious foods; at least some increased production is allocated to consumption; improved post-harvest handling is important to food safety (utilization); and access to finance and increased incomes are also likely to contribute to stability, but that would need to be measured over a longer period of time. Nutrition sensitive agriculture has been effective in achieving “saturation” of nutrition messages, as evidenced by the high number of households reporting learning from trainings and the high 11 number reporting positive changes to their diets. While farmers do receive training in the nutritional value of the commodities they grow and while this does include messages about food￾sharing at the household level and proper child nutrition and feeding practices, the results of CNFA’s 2019 Social and Behavioral Change assessment concerning equity of decision-making and allocation of food resources indicate that there is significant room for growth in the role of male champions in helping to spread nutrition messaging more broadly among participants in other Hinga Weze activities. 5. Hinga Weze’s approaches to increased productivity, improved market access, and enhanced nutrition are all driven in large part by grassroots-level community outreach organizations and volunteers. To what extent have the capacity-building approaches for Twigire Muhinzi Extension model, Community-Based Volunteers (CBVs), and Village Savings and Credit Mechanisms pursued by Hinga Weze led to institutionalized improvements that can be maintained after the activity ends? Hinga Weze’s capacity-building approaches are based on existing structures that pre-date Hinga Weze. Using these structures can greatly contribute to sustainability. In the case of the Twigire Muhinzi Extension model, Hinga Weze has used the farmers groups, however according to reporting in KIIs and FGDs, has not done enough to improve current management issues. Savings and credit or solidarity groups are a key mechanism to ensure all of Hinga Weze’s activities are sustainable, because they can provide income for farmers to reinvest in production and access to finance for small farmers and other small business owners with little or no collateral. They also create a culture of savings among households, and through their broad appeal, provide an access point for other important education and behavior change messaging. Hinga Weze has worked closely with the groups and their work has attracted new members, which is key to sustainability through growth and expansion. Hinga Weze has also built on the successes of Community Health Workers by providing additional education and training that has been passed on very effectively to community members. Investments in instilling a culture of savings and in the management and growth of savings groups, as well as in increased involvement of youth in the agriculture sector and in savings groups, both as participants and as peer trainers and as catalysts of behavior change around nutrition and women’s roles are key to sustainability. To date, Hinga Weze has been successful in engaging youth, thus ensuring a better educated, more capable population for the future, that can continue ensuring these activities are sustainable. 12 1. INTRODUCTION AND EVALUATION DESIGN The United States Agency for International Development (USAID)/Rwanda engaged Integra Government Services International through the Learning, Evaluation, and Analysis III Project (LEAP III) to conduct a mid-term evaluation of the Hinga Weze Activity managed by Cultivating New Frontiers in Agriculture (CNFA). 1.1 OVERVIEW OF HINGA WEZE Hinga Weze is a $32.6 million USAID-funded activity that aims to sustainably increase smallholder farmers’ income, improve the nutritional status of women and children, and increase the resilience of agriculture and food systems to the changing climate. The activity is implemented in ten districts over a five-year period from June 22, 2017 through June 22, 2022. Led by CNFA, the consortium comprises a diverse range of international and Rwandan partners including Plan International, Souktel, Rwanda Development Organization (RDO) and the Imbaraga Farmers’ Organization. The Hinga Weze Program works in the following districts: Gatsibo, Kayonza, Bugesera, and Ngoma in the Eastern Province; Nyabihu, Rutsiro, Ngororeo, Nyamasheke, and Karongi in the Western Province; and Nyamagabe in the Southern Province. The Activity has three objectives: 1) sustainably increase farmers’ incomes; 2) improve the nutritional status of women and children; and 3) increase the resilience of the agriculture and food systems in the face of a changing climate. The Activity has three intermediate results: 1) sustainably increase agricultural productivity; 2) improve farmers’ market access; and 3) improve the nutrition outcome of agriculture interventions. In congruence with USAID/Rwanda’s Country Development Cooperation Strategy (CDCS), Hinga Weze aims to accelerate Rwanda’s progress toward middle-income status and a better quality of life for its inhabitants. Hinga Weze will also contribute to the achievement of the 2015-2020 CDCS development objective one, “Economic Opportunities Increased and Sustained,” and its intermediate result 1.1, “Increased Agricultural Productivity and Nutrition Outcomes of Agriculture.” Under USAID/Rwanda’s Country Development Cooperation Strategy, the Hinga Weze activity aims to accelerate Rwanda’s progress toward middle income status and a better quality of life for its inhabitants through three mutually reinforcing components, seen in Figure 2. The evaluation team used a mixed methods approach, combining qualitative information from a desk review and interviews and quantitative data to answer the evaluation questions. The team assessed the achievements of Hinga Weze’s activities to date, through the lens of the five evaluation questions provided by USAID/Rwanda and identified successes and opportunities for adaptation or improvement for the remaining two years of the program. A quantitative survey instrument was developed by the evaluation team alongside Dalberg Research, the regional consulting firm that administered the survey and conducted three focus group discussions (FGDs). Sites for the survey were selected in consultation with USAID from across the ten districts where Hinga Weze operates in the Eastern, Western, and Southern Provinces. The team also conducted key informant interviews (KIIs) and focus group discussions (FGDs), as detailed in this section. A breakdown of survey respondents can be found in the Survey Implementation Report in Annex A. A total of 67 KIIs were carried out by the evaluation team and Dalberg and the evaluation team carried out three and four FGDs, respectively. A breakdown of KIIs and FGDs can be found in Annex D. 13 This approach is reflective of USAID’s parallel combinations approach where different methods are used to collect and analyze information, which is then synthesized to answer individual evaluation questions. Data sources include FTF reports and key activity documents, project monitoring records, staffing information, KIIs, FGDs, survey responses, and relevant secondary data and literature. Recognizing that a wide range of stakeholders needed to be consulted for this evaluation, the team conducted remote and face-to-face KIIs and FGDs (with appropriate precautions for COVID-19) and interviews with activity staff, sub-partner organizations, government officials, the private sector, beneficiaries, and other stakeholders. 1.2 EVALUATION OVERVIEW AND METHODOLOGY 1.2.1 EVALUATION QUESTIONS Based on the Scope of Work and consultations with USAID/Rwanda, the evaluation focused on five key questions: 1. As a way of increasing farmers’ income, one of the most common approaches is to facilitate smallholder farmers to increase productivity and access to markets through building cooperatives’ market power in price negotiations with processors and aggregators. Hinga Weze also supports processors and aggregators to improve product quality and their capacity to procure from the farmers. What are the risks associated with these approaches and how can they be addressed during the rest of the activity lifetime? During the period of their involvement with Hinga Weze, how have farmers’ incomes changed? To address this question, the survey included questions on farming practices, expenditures and sales by season, membership and involvement in cooperatives or other farmers groups, linkages with buyers, and pricing. FGDs were conducted with processors and aggregators as well as cooperatives, including cooperatives that were experiencing challenges along the axes relevant to Hinga Weze, such as market linkages and access to finance. This was done to ensure that a variety of perspectives were collected. The data provided a more comprehensive quantitative picture on changes experienced by participants and the FGD and KIIs were used to better understand the drivers of those changes and their relationship to Hinga Weze activities. Variation in cooperatives allowed the evaluators to observe differences in market power across different cooperatives, illuminating some of the drivers of price-negotiating power for farmers. KIIs were conducted to further understand these relationships. Risk assessment focused on the qualitative participatory input, but also included examination of institutional capacity to negotiate sales contracts. 2) To what extent has Hinga Weze ensured that farmers or communities own terracing and irrigation schemes and have plans and associated financial and human resources required for future maintenance of these productive assets? To address this question, the survey included questions on access to finance, participation in savings groups, involvement in and implementation of terracing or irrigation activities, and land ownership. The team also utilized FGDs made up of beneficiary farmers, cooperatives, and government extension agents and volunteer farmer promoters from Twigire Muhinzi supported by Hinga Weze. It is important to note that separate focus groups were conducted for beneficiary farmers and extension agents to ensure that 14 all felt comfortable speaking freely. KIIs were conducted with cooperative members and leaders, extension agents, and savings group members. 3) Hinga Weze has promoted a number of agricultural technologies aimed at increasing farmer-level productivity and climate resilience. Which are the top three technologies on a cost-benefit basis that would be recommended to continue to roll out at scale over the rest of the activity period? The survey included questions on farming practices, productivity, market access, and access to finance, among other measures. This information was compared to the baseline survey, as well as the 2018 and 2019 beneficiary surveys, to estimate the incremental effect of the intervention on farming costs and practices. After consultations with USAID/Rwanda, it was determined that the evaluation team would conduct cost benefit analyses (CBAs) of three crops supported by the Hinga Weze project - maize, orange flesh sweet potatoes (OFSP), and Irish potatoes. The CBA models reported on the net returns to farmers and the economy by the group of farming practices or technologies that farmers adopted due to Hinga Weze. For example, farmers benefiting from improved water harvesting or irrigation were assessed separately from those who adopted improved soil management practices. The mid-term evaluation survey, secondary research, and interviews were used to address the issue of climate change resilience. In total, the CBA time horizon estimated the costs and benefits to USAID beneficiaries over a 10-year time horizon, starting with the point when the USAID beneficiary receives assistance. The complete CBA can be found in Annex B. 4) To what extent do Hinga Weze’s agricultural productivity and market access interventions contribute towards improved nutritional outcomes for women and children? In what ways do the current interventions address/not address the underlying constraints towards improved nutritional status of the target households? To better address this question, in addition to questions on involvement in other Hinga Weze activities, the survey included questions on dietary diversity, nutrition education (including membership in care groups and participation in trainings), and changes in dietary and child-feeding practices resulting from those programs. These questions were asked for all households with children from 0-23 months old, women from 15-49 years old, and pregnant and lactating women. Data from the households’ participation in other components was used along with that collected from the nutrition questions to examine relationships between those components and nutrition improvements. FGDs were used to further explore these measures and included members of Care Groups, and groups of community-based volunteers, who are trained by Hinga Weze and provide the education and training to the Care Groups. KIIs were carried out with beneficiaries, CBVs, and nutrition, child development, and gender experts. Secondary data and both internal and external reports and assessments were also used in the response to this question. 5) Hinga Weze’s approaches to increased productivity, improved market access, and enhanced nutrition are all driven in large part by grassroots-level community outreach organizations and volunteers. To what extent have the capacity-building approaches for Twigire Muhinzi Extension, Community-Based Volunteers, and Village Savings and Credit Mechanisms pursued by Hinga Weze led to institutionalized improvements that can be maintained after the activity ends? 15 FGDs were the most important approach to answering these questions, along with interviews of community leaders (including leaders of farmer organizations) and extension agents. FGDs were conducted with Twigire Muhinzi extension agents, members of savings groups, beneficiary farmers, and cooperatives. In addition to the CBV focus group conducted for nutrition, CBVs who are farmer promoters were also included. KIIs were conducted with beneficiary farmers, cooperative leaders, extension agents, savings group members, CBVs, and high-level stakeholders in the government and NGO communities that are closely involved in the project. Information and data collected in response to all of the other questions was synthesized for this question, as each of the questions provided some insights into institutional changes and capacity-building needs at the grassroots level. 1.2.2 QUANTITATIVE SURVEY OF HINGA WEZE BENEFICIARY HOUSEHOLDS Dalberg Research, an international consulting firm working across Africa, administered the survey. Dalberg has extensive survey experience in Rwanda and has conducted evaluation surveys in other countries under LEAP III. The survey was conducted from September 14–29, 2020. Survey Instrument The evaluation team developed a questionnaire to survey farming households benefiting from the Hinga Weze project. They worked with Dalberg to review the questions, and Dalberg developed the survey instrument. A list of the survey questions can be found Annex C and the complete survey instrument has been shared with USAID in the form of an Excel spreadsheet named HingaWezeMidtermSurvey.xls. USAID provided helpful comments on the draft survey instrument. In addition to the topics discussed above under the evaluation questions, the survey instrument also asked more general questions on farmers’ and Care Group members’ experiences, the potential benefits from participating in Hinga Weze’s activities, and the accessibility and relevance of trainings and demonstrations. Sampling Strategy A total of 408 farming households that engaged in Hinga Weze activities were sampled to participate in the survey. This number was selected as a result of budgetary consideration for the cost of the survey, and the fact that the survey was carried out in 10 districts, and it was therefore necessary to have a sufficient number of respondents in each district. The sample drew from three groups of Hinga Weze farmers: 1) “Conventional” Hinga Weze farmers; 2) farmers that are benefitting from the Hinga Weze irrigation scheme; and 3) farmers that are benefitting from the Hinga Weze terracing activity. “Conventional” farmers refers to farming households that are engaged in Hinga Weze’s activities but are not located within the irrigation perimeters or terracing areas. Farmers benefiting from the irrigation scheme (180) or the terracing activity (5,609) make up a significantly smaller percentage of the total number of Hinga Weze participants (294,520). The remaining 288,731 farmers fall under the “conventional” category. However, considering that several of the evaluation questions relate to irrigation and terracing, beneficiaries from these groups were oversampled. 16 The largest proportion of the overall sample was allocated to the “conventional” farmer group, which is the largest population. For farmers from the irrigation scheme and the terracing activity, it was assumed that the farmers and their activities would be similar in nature, and therefore a smaller sample would still yield meaningful data. TABLE 1: SURVEY RESPONDENTS BY TYPE AND DISTRICT PROVINCE DISTRICT IRRIGATION TERRACING CONVENTIONAL Eastern Gatsibo 10 25 Kayonza 15 25 Bugesera 17 10 20 Ngoma 10 25 Western Nyabihu 33 20 Rutsiro 21 20 Ngororeo 20 20 Nyamasheke 20 25 Karongi 10 20 Southern Nyamagabe 12 30 Total 52 126 230 Dalberg used a cluster sampling approach, which sampled multiple villages from within a district, further stratifying the sample by activity to ensure that the survey captured adequate observations for all activities. After dividing the population by district and activity, respondents were randomly sampled. Dalberg obtained participant lists which formed the basis for the sampling strategy and breakdown of respondents presented in Table 3. The complete implementation report for the survey, with detail on sampling, is in Annex A. 1.2.3 FOCUS GROUP DISCUSSIONS Dalberg Research conducted three FGDs and Integra’s Kigali-based team conducted an additional four. FGDs were divided between Dalberg and the Integra team to maximize the number of communities and groups reached within time and budget constraints. All FGDs were carried out in person in a group setting, with proper precautions for COVID-19. Participants were drawn from nearby communities and offered masks, hand sanitizer, and fresh water at all meetings. Hinga Weze stakeholders that participated in FGDs were drawn from the following groups: 1) processors and aggregators to discuss marketing and linkages with farmers; 2) women in a care group to discuss nutrition and gender issues; 3) farmer’s cooperatives, including cooperatives that are well-managed and provide their members with services as well as cooperatives that are struggling; 4) cooperatives that are engaged in agri-business clusters and out￾grower schemes; 5) cooperatives/farmers groups that are located in irrigation scheme areas and those that are located in terracing areas; 6) savings group(s); 7) community-based volunteers; and 8) Twigire Muhinzi extension agents. Specifically, Dalberg carried out the following three FGDs: Farmers in irrigation Table 1 Source: Authors 17 schemes supported by Hinga Weze, agricultural commodity buyers supported by Hinga Weze, and Care Group members. 1.2.4 KEY INFORMANT INTERVIEWS (KIIs) The evaluation team conducted qualitative, in-depth individual interviews with key stakeholders, project partners, and other informants. Key informants included the following: 1) the USAID FTF team; 2) CNFA and its sub-partners; 3) government officials, including the Ministry of Agriculture and Animal Resources officials and extension agents in the Twigire Muhinzi extension program; 4) the private sector, including aggregators and processors; 5) activity beneficiaries; and 6) representatives of cooperatives. These interviews were also conducted with technical experts and others from donors or non-governmental organizations (NGOs) operating in the area. Some of the KIIs were conducted remotely, while others were conducted in-country. A list of stakeholders engaged by the team can be found in Annex C of this report. Additionally, the local team was able to conduct a number of in-person focus group discussions and interviews. A list of these contacts can also be found in Annex C. 1.2.5 COST-BENEFIT ANALYSIS (CBA) Details on the CBA conducted for this evaluation can be found above in the approach to question 3 as well as below in the findings for question 3 and Annex B, which contains the full CBA. 1.3 LIMITATIONS The principal limitation for this evaluation was the travel limits imposed due to the COVID-19 pandemic. This forced the team to adapt the evaluation approach to account for the fact that most of the team was not able to travel to Rwanda to participate in the fieldwork. To address this challenge, at the suggestion of the Mission, Integra added an additional local national expert to the team who is an evaluation expert. This expert provided critical input during the design of the survey and evaluation approach, but left the team prior to the beginning of fieldwork. The agriculture expert and the local facilitator worked closely with the team members outside Rwanda during fieldwork to ensure an optimal flow of information in both directions. Through close communication with the in-country team members, the evaluation team was able to respond in real time to the outputs from KIIs and FGDs and make adjustments to interview structures and reporting modalities to optimize the use of the team’s time and convey the necessary information. There may be some loss of acuity to the analysis and recommendations without the on-the￾ground experience, but an experienced team of local consultants worked to minimize that effect. In addition to the pandemic, this evaluation faced some more common limitations. KIIs, and FGDs were a major data source for this evaluation and the team depended in part on USAID/Rwanda contacts and the implementing partner to identify and communicate with key stakeholders. Therefore, there is some risk of selection bias due to the potential of selecting a large proportion of interviewees who have experienced success or otherwise have a positive view of the program. Although the evaluation team invited a variety of stakeholders to participate in the interview process, individual stakeholders are the ultimate decision-makers creating the risk of interviewing a significant proportion of stakeholders who are motivated by their strong opinions about the program. In addition, there is always a risk in interviews with more than one stakeholder and with focus groups that people may feel constrained from voicing their opinions for reasons the evaluators may not be aware. Finally, the quantitative survey relied on 18 recall data, and while timeframes and phrasing of questions were chosen to minimize potential biases, there is always a risk of inaccuracy and potential bias when using such data, as opposed to data observed by monitors or evaluators. Realizing these limitations, the evaluation team worked with USAID and the implementing partner to conduct interviews with stakeholder groups representative of the greater population. This included beneficiaries and cooperatives that are struggling, in addition to those that have shown significant results. Finally, the evaluation team worked diligently to identify and analyze secondary information that they triangulated with data from key informant interviews. 19 2. FINDINGS 2.1 EVALUATION QUESTION 1: COOPERATIVES’ AND BUYERS’ MARKET POWER As a way of increasing farmers’ income, one of the most common approaches is to facilitate smallholder farmers to increase productivity and access to markets through building cooperatives’ market power in price negotiations with processors and aggregators. Hinga Weze also supports processors and aggregators to improve product quality and their capacity to procure from the farmers. What are the risks associated with these approaches and how can they be addressed during the rest of the activity lifetime? During the period of their involvement with Hinga Weze, how have farmers’ incomes changed? This response divides the question up into components, as there are multiple actors involved with multiple forces acting on them through complex relationships. Findings are broken down by the different factors to provide a complete picture of the situation. Hinga Weze’s support to build farmer cooperatives’ capacity. Hinga Weze is supporting the formation of farmers’ cooperatives and is seeking to strengthen cooperative management capacity. For buyers, it is too cumbersome to deal with hundreds of small producers, which is why the cooperatives’ ability to aggregate production and represent their members vis-à-vis buyers is so important. By supporting the formation of cooperatives and strengthening their management, Hinga Weze enables farmers to connect to markets. To date, Hinga Weze has focused much of its work on increasing production, providing extensive training in good agricultural practices (GAP), training farmer promoters, and supporting farm field schools. Hinga Weze has also linked farmers with the input subsidy program Smart Nkunganire System (SNS), and conducted training and support to reduce post-harvest loss. To date, cooperatives that participated in qualitative interviews shared that they had received little training to understand the drivers of market prices including supply, demand, quality, market information, and price negotiations. The survey of 408 beneficiaries conducted for this mid-term evaluation found that the majority of farmer beneficiaries are members of a cooperative supported by Hinga Weze (65 percent), while about one-third of farmers (29 percent) are not members of a cooperative. Half of the farmers that are not members of a cooperative explained that there is no active cooperative in their area. Others cited high membership shares/fees and limited awareness of existing cooperatives. Many cooperatives are constrained by capacity, providing their members with few relevant services, and have limited market information awareness. A recent assessment of 141 farmer cooperatives supported by Hinga Weze showed that the majority (72 percent) have limited cooperative management procedures in place, and few (15 percent) had formal supply contracts with buyers (CNFA 2020). The assessment, carried out by the Cooperative Advisory Academy (CAA), found that only 18 percent provide their members with services such as access to inputs, markets, finance, or production skills. According to the CAA assessment, the majority of cooperatives have limited market information and their products are of poor quality; they therefore have limited bargaining power with potential buyers. However, the assessment found that approximately one quarter of the cooperatives, primarily in the maize and Irish potato value chains in the Northern Province, are relatively well-established and well-structured. Nevertheless, some of these cooperatives failed to meet contract obligations, due to limited volumes. 20 There were also some reports of side-selling in interviews and FGDs, but these could not be confirmed beyond anecdotal reports, although there is no reason to believe that the reports are not credible. The CAA assessment noted that of the cooperatives that have supply contract agreements, only 17 percent can consistently supply commodities that meet the contractual minimum requirements, which undermines possibilities for longer term business relationships. More than 50 percent of cooperatives sell their commodities, not sorted by grade, at low prices at the farm-gate to the first buyer that appears. Moreover, the assessment found that among maize producer cooperatives with limited postharvest handling skills and storage, it was not uncommon for their commodities to be rejected due to high moisture content. Hinga Weze commissioned the cooperative assessment in an effort to design a more targeted approach to building the capacity of farmer cooperatives. Hinga Weze is currently (late October 2020) in the process of concluding contracts with two cooperative development services providers, who will train cooperatives in seven different capacity-building modules. The cooperative development services providers will train the cooperatives in business planning; supplying input; commodity aggregation management; entrepreneurship and growth strategies; cooperative financial management; accounting and payment procedures; strategic planning, and marketing. Because the training has not yet begun, the evaluation team cannot comment on its effectiveness, but findings from the Cooperative Advisory Academy assessment indicate that most farmer cooperatives do need significant training and capacity￾building support to be able to aggregate and negotiate contract terms with buyers. Linking farmers and buyers. These linkages were identified by farmers as one of the most important services they have obtained from Hinga Weze. A quarter of farmers in the mid-term evaluation survey (24 percent) said that linkages to buyers was the most important service they had obtained from the project. When considering what services farmers had not had access to before Hinga Weze that they now have access to, 36 percent of farmers said that linkages to agricultural buyers was most important (see table 2). TABLE 2: MOST IMPORTANT SERVICE ACCESSED THROUGH HINGA WEZE (BY PERCENTAGE OF RESPONDENTS) MOST IMPORTANT SERVICE ACCESSED THROUGH HINGA WEZE MOST IMPORTANT SERVICE FARMER ACCESSED THROUGH HINGA WEZE THEY DID NOT HAVE ACCESS TO BEFORE Savings groups 28% (N=108) Linkages to agricultural buyers 36% (N=140) Linkages to agricultural buyers 24% (N=91) Savings groups 19% (N=72) Training on good agricultural practices 15% (N=56) Training on good agricultural practices 8% (N=29) Twenty-eight percent (N=116) of the farmers in the mid-term evaluation said that Hinga Weze has linked them to buyers. Those linked to buyers shared that the linkages have improved their ability to plan for and predict what quantity they will be able to sell (56 percent); that linkages have improved their ability Table 2 Source: Mid-Term Evaluation Survey 21 to predict what price they will sell the commodity for (59 percent), and that linkages to buyers have improved their ability to meet quality requirements (63 percent). While Hinga Weze supports both farmers that are part of a cooperative and those that are not, it is much easier to link cooperatives rather than individuals; 80 percent of the farmers that Hinga Weze linked to buyers were members of a cooperative. In the mid-term evaluation survey, 59 percent (N=239) of farmers said that they sold some commodities in season 2020B (February 2020–May 2020). The most commonly sold commodities were beans (part of “other” in figure 1), maize, Irish potatoes, horticulture, and iron fortified beans. As shown in figure 1, farmers sell most commonly to peer farmers and neighbors (N=74); followed by buyers identified through the agro-business cluster (N=58); the cooperative (N=41); village agents for aggregator (N=32); direct to aggregators (N=24); buyer through contract farming (N=10); while sales to village agents representing processors (N=2) are rare. Figure 1. Main buyers of commodities by Hinga Weze value chain crops Benefits of linkages for buyers. Buyers are looking for producers that can deliver the agreed upon quality and quantity of commodities. Relationship building, honesty, and trust between the buyer and the seller is important for business relations to last. The larger buyers Hinga Weze works with and links to producer cooperatives are benefitting from Hinga Weze’s intervention in a number of different ways. Aggregation is time-consuming and costly, and buyers therefore prefer to purchase from cooperatives that can aggregate their own production. The buyers point out that it is an advantage to work with cooperatives that Hinga Weze is supporting, as this support provides the cooperatives with managerial Figure 1 Source: Mid-term Evaluation Survey 22 capacity-building services. In addition, members of the cooperatives have also obtained training on good agricultural practices which may improve the quantity and quality of production, as well as post-harvest handling. Linkages to buyers through agro-business clusters. To connect producer cooperatives to buyers, Hinga Weze has created 30 agro-business clusters. These platforms bring together input suppliers, cooperatives, processors, aggregators, transporters, financial institutions, equipment suppliers, and the district office in charge of agriculture to create linkages and support business deals between input suppliers, producers, and buyers. Agro-business clusters are one of the primary ways producers connect to buyers. During the agro-business cluster meetings at the beginning of the season, producers share production projections and buyers signal what quantity and quality they are looking to buy. However, the sale, especially the price negotiation, does not take place until after the harvest. Because there are considerable price fluctuations, the stakeholders the assessment team talked to emphasized that prices for commodities destined for the domestic or regional market cannot be set in advance. Advanced price negotiations would leave farmers vulnerable to being locked into sales contracts, and would expose buyers to contract breach when farmers sell elsewhere at a higher price. Some stakeholders noted that in some agro-business clusters there were too few buyers of certain commodities, limiting competition. The stakeholders proposed that agro-business clusters can be connected across districts to expand the number of buyers and producers, therefore increasing competition. Several district officials the evaluation team interviewed highlighted the successes the agro￾business clusters have had in connecting producers and buyers, but noted that continued operation of the clusters depended on District authorities, as the private sector had so far shown limited interest in managing the clusters. Another stakeholder that works closely with Hinga Weze noted that Hinga Weze staff is deeply involved in organizing the agro-business cluster meetings and questioned whether the clusters will be sustained when Hinga Weze concludes. Hinga Weze is currently paying for agro-business cluster related expenses, which primarily involves meeting participants’ transportation fees and lunch during the meeting. Hinga Weze should facilitate a discussion with agro-business cluster members about how the members, especially larger buyers with more financial resources, can take ownership of the agro￾business clusters, and how members can contribute towards the operating costs of the clusters, in view to transition Hinga Weze off of financing the operating costs of the clusters during year four. Linkages through contract farming. In a contract farming arrangement, the buyer and the producer agree on the terms of the contract prior to the start of the season. The buyer typically provides some input on credit and may also provide extension services to ensure that the quality and quantity of the crop meet the buyer’s specifications. Hinga Weze has focused its efforts in contract farming on the irrigation sites, as they have the potential to produce high value crops for the export market, which is ideal for a contract farming arrangement. Farmers that are in a contract farming arrangement said that they make more money through contract farming. They shared that they plant and sell higher value crops, produce more with good input and extension services provided by the buyer, obtain better prices, and have less post-harvest loss. Hinga Weze has linked exporters with growers at the irrigation sites for contract farming arrangements. Regular access to water, paired with good quality input and extension services, enables the farmers to produce high value crops. The evaluation team spoke with several buyers-exporters that shared how 23 valuable it is for them to work closely with the Hinga Weze farmers at the irrigation sites because the sites have access to water (although as discussed further below, there are significant issues with access to sufficient water at multiple sites); the soil conditions are favorable; the farmers are clustered together; and Hinga Weze has organized them into farmer groups/cooperatives. The exporters shared that through contract farming arrangements with the farmers, they provide farmers with input and intense extension services, and in return they are expecting to get high quality commodities upon harvest. Hinga Weze supports the implementation of the contract farming arrangement (the contract is between the farmer group and the exporter) by educating the farmer group about their contractual obligations and how a contract farming arrangement is structured. The exporters have the market power, as one exporter noted: “Our contract is a standard contract, we do not negotiate prices [with the farmers]. It is helpful that Hinga Weze can explain the contract to the farmers, and what it means to be part of an out-grower scheme, and that they cannot “side sell.” Anecdotally, farmers on the irrigation sites with contract farming arrangements have increased their income multifold due to producing higher value crops, increasing production, increasing the number of growing seasons, and having a stable buyer (see table Q in annex F for details about the contract farming arrangements and anecdotal reports about profit gains). One exporter (Lotec) shared that they are no longer working with one Hinga Weze irrigation site where farmers were “side selling”. Buyers’ and producers’ price setting power. Hinga Weze is working with different types of buyers. Some of the buyers are larger, well established businesses. The larger buyers set the prices and are not necessarily engaging in price negotiations. In local markets with limited competition from other buyers, the buyers have little incentive to negotiate prices. One cooperative processing maize shared that they were the only buyer of maize in the area, and they could therefore purchase maize from other cooperatives at the government floor price without engaging in price negotiations. However, larger buyers shared also that they pay a premium for high quality commodities. For example, the government set a floor price for maize but buyers pay more for higher quality maize with lower moisture content. Buyers also noted though that many cooperatives have limited awareness about grades, how to harvest and store commodities to preserve higher grades, and how to sort the commodities according to grade. Farmers’ limited awareness about grades is both an issue of knowledge and access to technologies and storage infrastructure to reduce post-harvest loss and increase quality. To this end, Hinga Weze is implementing post-harvest reduction interventions to reduce loss and improve quality. For example, Hinga Weze’s grant to the processor Kumwe Harvest to purchase maize shelling and dryer machines has allowed Kumwe to purchase maize on the cob from farmers after the harvest for immediate shelling and drying. The “cob model” reduces post-harvest loss and controls the moisture content which enables Kumwe to sell the maize to premium buyers. Cooperatives have limited price and market information. Other production cooperatives the evaluation team interviewed used informal channels and had limited access to information about market prices. One cooperative shared that the government’s reference price is all the price information they have, and they get paid around the floor price. Another maize producing cooperative said that their buyer sets the price and typically adds 5-10 percent on the government’s floor price. However, members of several cooperatives shared in interviews with the evaluation team that they do some market research, calling relatives or other contacts in different markets, including markets in Kigali, to find out market 24 prices. This information helps when negotiating prices with buyers. One large buyer of maize noted that “the only thing that works in Rwanda in regard to price setting is to not set the price ahead of time. There are a lot of price fluctuations and farmers naturally want to sell at the highest price, so they sell to someone else. So buyers and sellers agree on the quantity and the grade, but the price is not set until the time of the transaction when you know what the market price is.” Hinga Weze is funding the development of improved digital agricultural market platforms to address the limited availability of price information, and in year one of the project conducted a digital market assessment. Based on the findings from the assessment, Hinga Weze funded two firms to improve existing digital agricultural market platforms to deliver high-quality and relevant input and output market information for small hold farmers. The output market information platform M-LIMA will enable small￾hold farmers and cooperatives to broadcast supply they are looking to sell (including volumes and expected prices), which buyers can respond to on the platform. Transactions that take place on the platform will be used to generate market information about production volumes, prices, buyers, and sellers which will help cooperatives and farmers to gain greater information about market prices. The digital platform for output markets has been tested with five farmer cooperatives and a broader roll-out is expected in 2021. As noted in the cooperative assessment (CNFA 2020), in addition to having limited market information, there are cooperatives that are not market ready, even if they have surplus production to sell. The evaluation team met with several cooperatives where the members were not working as a group, but rather individually. In one buyers’ cooperative, the members made their separate purchases of agricultural commodities and did not pool their resources. In a maize producing cooperative the members harvested and stored all production individually, and the cooperative heard that members had surplus to sell, but did not know how much the members had produced or how much they wanted to sell. The cooperative was new and members were “not yet convinced” of the cooperative idea of aggregating and working together. Hence, there are some cooperatives that need a lot of capacity-building before they are ready for market linkages. Hinga Weze’s approach to working with buyers is not in conflict with their activities to strengthen the capacity of producers of agricultural commodities. Hinga Weze’s work to improve farmer’s agricultural practices, post-harvest handling, and access to inputs, combined with the support to form and capacitate farmer cooperatives has strengthened farmers’ abilities to produce output for marketing. Many cooperatives have limited capacity to provide their members with valuable services including aggregation, price discovery, and contract negotiation. Hinga Weze should spend time and resources to develop these capacities in cooperatives over the next two years. Considering the investments Hinga Weze has made in irrigation and terracing, these cooperatives should be prioritized. Moreover, an honest assessment of which cooperatives could reach a level of maturity where they could play an important marketing role is needed, so that more intense coaching and resources can be focused on those cooperatives. The rollout of the market information system should be helpful in bridging the knowledge gap producers have about price information. Supporting the formation of longer-term business relations between producers and buyers, including, where appropriate, out-grower-schemes, should be a priority. Strategic grants that may enable such business relations to take root and flourish should be 25 evaluated. Thus, supporting buyers to form long-term business relationships with producers is beneficial for both producers and buyers. Changes to Beneficiary Income: According to Hinga Weze monitoring data from annual reporting in 2017 and 2018, 20 percent of respondents reported an increase in income. To look more closely at changes at the farm household level that contribute to income changes, the evaluation team examined data from the cost-benefit analysis (CBA) presented in question 3 and in Annex B. The best picture of how farmers’ income changed during their participation in the program can be seen in table 3, where incremental yield changes are translated into incremental revenue per hectare. It is important to note that these changes cannot necessarily be attributed to Hinga Weze activities, only that they reflect the experience of Hinga Weze farmers during the time they have been involved with the program. Figures in this table have been aggregated across all interventions to provide a clearer picture of how revenues changed. All interventions yielded positive incremental revenue, although it varied greatly across crops and was subject to both yield and price fluctuations, but it can be observed that incremental revenues from maize production are significantly higher per hectare. Decision-making around which crops to cultivate and which technologies to use however, should factor in the more detailed aspects of the CBA presented in this section and in Annex B. While only 20 percent of households reported income increases, there are measurable revenue increases associated with Hinga Weze interventions (see question 3 for an analysis of net benefits by technology). As discussed in more detail in question 4, about nutrition, there is a clear correlation between participation in Hinga Weze and increasing food purchases. This is a strong indicator that income constraints are being relaxed, as expenditures are the more common measure of income rather than direct questions about income. TABLE 3: INCREMENTAL PRODUCTION AND REVENUES FOR ONE HECTARE OF LAND (KILOGRAMS (KGS)) CROP INTERVENTION YEAR 2 YIELD (KGS INCREMENTAL) YEAR 2 YIELD (KGS TOTAL) INCREMENTAL REVENUES (RWF) Maize All Interventions 2792 8960 647, 813 Irish Potato All Interventions 739 8,771 84,384 OFSP All Interventions 455 5,609 5,757 In conclusion, the analysis finds no conflict between the approaches Hinga Weze has taken with regard to smallholder farmers and buyers. Survey data for this evaluation, along with KII and FGD responses, indicates that there are some ongoing challenges for cooperatives, particularly relating to management capacity, and also in understanding and accessing market information. While agro-business clusters have proven to be a valuable mechanism for creating linkages between cooperatives, buyers, processors, and other key sectoral stakeholders, district authorities report that they are still responsible for ensuring their continuity, as so far the private sector has not shown much interest, although private sector engagement Table 3 Source: Authors 26 will be necessary for sustainability in the long term. While income, as reported in Hinga Weze monitoring data, has only increased in twenty percent of households, MTE survey data as well as KIIs and FGDs (see response to question 4) show an increase in food expenditures for the majority of households, which is a strong indicator of either reduced costs for other household expenses or increased income. 2.2 EVALUATION QUESTION 2: SMALL SCALE IRRIGATION SCHEMES To what extent has Hinga Weze ensured that farmers or communities own terracing and irrigation schemes and have plans and associated financial and human resources required for future sustainable maintenance of these productive assets? There are seven operating irrigation sites, five of which are faced with sustained technical failures of the pumps, which has led to water shortages. As a result of the water shortage, for five of the sites only one￾fifth to one-third of the site is under irrigation. The irrigation sites at Abishyizehamwe (Kayonza District) and Koperative Twigire Muhinzi Rukumberi (Ngoma District) are fully operational, while the irrigation sites at Terimbere Muhinzi (Kayonza district), Icyerekezo Rugenge Cooperative (Gatsibo District), Abahuje Akabuga Cooperative (Gatsibo District), Terimberemuhinzi (Bugesera District), and Abakoranamurava Ba Mayange Cooperative (Bugesera District) do not have sufficient water and are operating at reduced capacity. Annex E provides a more detailed description for each of the seven irrigation sites. To investigate these challenges, Hinga Weze requested that the service providers conduct an assessment of the existing irrigation schemes. The report was not complete at the time of this assessment. The irrigation sites have a lot of potential and some fruitful contract farming arrangements have already been established between cooperatives and buyers. If the technical issues leading to water shortages can be addressed swiftly, with continued capacity-building support and market linkages, the irrigation sites should be sustainable. From interviewing multiple stakeholders, it is clear that Hinga Weze has actively engaged with the Government of Rwanda at a national and district level in planning for the establishment and operation of the irrigation schemes. The Rwanda Agricultural Board commends Hinga Weze for aligning their activities with the government’s policy goals and notes that Hinga Weze is “one of the best projects we have in terms of collaboration with the government.” CURRENT USE OF SMALL-SCALE IRRIGATION Hinga Weze to date has constructed seven irrigation sites (ten hectares/site) and an additional three sites are currently under construction. The evaluation team visited three of the active sites and consulted over the phone with the remaining four operational 27 sites. In addition, the mid-term evaluation survey sampled 52 farmers from the Hinga Weze irrigation sites (note that the survey data in this section is referring only to the farmers on the irrigation sites and excludes other Hinga Weze beneficiaries). The majority of farmers that participated in the survey (78 percent) irrigated parts of their fields in season 2020B (February–May 2020), while 22 percent did not irrigate their fields. Lack of water in the dam was the most common reason why farmers did not irrigate their fields at all during season 2020B. The qualitative interviews showed that five of the operating sites do not have enough water, but just two to three hectares per site are under irrigation, and that some farmers that do irrigate do not have enough water [see picture at right]. Representatives of five out of the seven operating irrigation sites shared that they do not have sufficient water to irrigate the ten hectares of land on the irrigation site. The chief complaint was that the solar water pump does not pump enough water into the dam. According to the farmers on the irrigation sites there are two main reasons for why the pumps are not filling up the dam. First, the solar water pumps do not have a battery and the pump is therefore only working when it is sunny, and not during the night or if it is rainy or overcast. Second, the pumps are pumping water from a water source with debris, but the pumps do not have a filtering system, and therefore the pumps break frequently when debris enters the pump. In addition, several cooperatives shared that their pipes are breaking either because they were not installed correctly, or because the farmers were not trained on how to properly use the pipes. The two irrigation sites that are in full production appear to have the same type of pump, which is fully charging the dam, and it detects if there is debris in the water source. As a result, these two sites report that they have sufficient water and have not had issues with the pump failing. A summary from each of the seven sites can be found in Annex E. Fully operational irrigation sites. A key component of sustainability of irrigation sites beyond the life of the Hinga Weze Project is that all the sites need to be fully operational. Currently, the majority of operating sites are irrigating just one quarter to one third of the site due to water shortages. There are a number of technical issues that need to be thoroughly investigated: Why are the pumps not able to fully charge the dams? Can batteries be added to allow the pumps to pump water when there is no sun? Can filters or sensors be added so that debris from the source of the water does not enter and break the water pump? Why are the pipes breaking? A systematic investigation of these recurring technical issues and a swift implementation of an action plan to address them are key to getting the sites up and running as soon as possible. Without fully functional equipment and sites, farmers will not want to risk investing their own resources in maintaining the operations or the equipment. To date, all sites are still under warranty, and the cooperatives have therefore not had to pay for maintenance or repair. However, it appears that the pipes are not under the warranty, and they are breaking frequently; four sites reported that they have already spent between 40,000 RWF and 250,000 RWF on new pipes, even though most of the sites have been in operation for less than a year. Capacity-building for cooperatives. Hinga Weze supported the creation of a cooperative at each irrigation site. The majority of the surveyed farmers on irrigation sites (85 percent) reported that they or a member of their household are members of a cooperative supported by Hinga Weze. The cooperatives on the irrigation sites are active; all of the members said they had participated in at least one meeting in the last six months; sixty-one percent said that they have participated in five or more meetings 28 in the last six months. Consequently, the cooperatives are active and meet regularly, which is a key ingredient for collaboration among the farmers on the irrigation scheme. However, the issue of lack of water is causing some friction and addressing the water issue will strengthen cohesion. As discussed above for evaluation question 1, a majority of cooperatives need capacity-building support. However, Hinga Weze recognizes that the cooperatives at the irrigation sites need additional support because the cooperatives are new, and the farmers are new to irrigation and are adapting new farming practices and technology skills. Hinga Weze has an internship program where interns with irrigation engineering experience spend six to twelve months on the site full-time to provide support on the use and maintenance of the irrigation equipment. All the irrigation sites have or have had an intern at the site. However, the interns are primarily training the cooperatives in good agricultural practices and water usage, but do not have the practical and technical knowledge to repair irrigation equipment (the intern alerts Hinga Weze and the service supplier that the equipment needs to be repaired). Hinga Weze also provides the cooperatives with more intensive extension support on good agricultural practices tailored to irrigated agriculture. Intensive capacity-building support in equipment maintenance, planning and saving for maintenance, as well as negotiating and marketing skills for the cooperatives at the irrigation sites is recommended. The irrigation cooperatives need to function well to maintain and utilize the irrigation equipment, and if they are well organized and acquire additional management and marketing skills, they will get greater bargaining power vis-à-vis their buyers. For larger irrigation schemes, there is typically a water user association for the site. Considering the small size of the Hinga Weze irrigation sites, rather than forming water user associations (which are separate legal entities from the cooperatives), Hinga Weze is working with the Rwanda Agricultural Board (RAB) to establish irrigation scheme operation and maintenance committees (ISOMACOs) within the cooperative. RAB is satisfied that Hinga Weze is establishing ISOMACOs as opposed to water user associations. According to RAB, ISOMACOs are more feasible for small irrigation sites as the financial, administrative, and human resource burden of registering and operating two organizations (a cooperative and a water user association), can be too burdensome for a small group of farmers. All irrigation sites have established water committees with the support of Hinga Weze. According to the cooperatives, RAB was not involved in establishing the water committees, and it is not clear if the water committees are official ISOMACOs or if RAB needs to get involved. The water committees, all established within the last year, typically have three members. As the water committees are newly established, they need continued support and training to clearly define their responsibilities, and how to meet them. Ownership of the irrigation infrastructure and equipment. In 2021, Hinga Weze is planning to start handing over established irrigation sites to the cooperatives. During the handing over ceremony, which will be attended by Hinga Weze, the farmer cooperative, and District representatives, Hinga Weze is planning for a Proper Exit Plan and a signed Irrigation Management Transfer Agreement. A Proper Exit Plan outlines the irrigation infrastructure established, the technologies used at the site, and a capacity￾building and maintenance plan. An Irrigation Management Transfer Agreement (IMTA) is typically signed between the irrigation water user association, the District, and RAB. Because the Hinga Weze irrigation cooperatives are forming irrigation scheme operation and maintenance committees (ISOMACO) rather than separate irrigation water user committees, the evaluation team note that it is important to clarify whether it will be the cooperative that signs the IMTA. IMTAs are anchored in Ministerial Order number 29 001/11.30 of November 23, 2011, establishing irrigation water users’ organizations, so Hinga Weze will need to clarify with MINAGRI/RAB that Cooperatives with ISOMACOs can enter into an IMTA. The IMTA outlines the responsibilities for managing and maintaining the irrigation infrastructure and equipment. According to a sample IMTA RAB shared with the evaluation team, the government transfers the responsibility of the management of the irrigation scheme to the irrigation water users’ organizations (in the case of Hinga Weze that would presumably be the Cooperative). However, the IMTA states that “Ownership of infrastructures and equipment remains with the Government of Rwanda.” According to the sample IMTA, MINAGRI/RAB has the responsibility of supporting capacity-building, covering the cost of heavy maintenance work, and monitoring the performance and operationalization of the irrigation system. The irrigation water user association (in the case of Hinga Weze that would presumably be the cooperative) is responsible for maintaining the irrigation infrastructure and equipment, protecting the irrigation system from floods, collecting water user fees, signing performance contracts with the District Irrigation Steering Committee, and developing annual work plans and budgets. The District is responsible for providing technical and managerial support, drafting performance contracts, monitoring the operation of the irrigation infrastructure, and collecting water user fees, as well as approving work plans and budgets. When the evaluation team talked to RAB, RAB noted that they have shared the IMTA with Hinga Weze, but have not yet discussed the content and the implications of the IMTA with Hinga Weze. It is important that Hinga Weze commence this work with RAB, the District, and the irrigation cooperatives to prepare for implementing the IMTAs. There are many responsibilities the cooperatives will have under the agreement, and the cooperatives will most likely need support and coaching to acquire the capacity to, for example, develop annual work plans, budgets, and water distribution plans. Finally, it is worth noting that in the mid-term evaluation survey, almost all surveyed farmers at the irrigation site shared that the cooperative would own the irrigation infrastructure and equipment (98 percent). Thus, there’s a common understanding among the farmers that they are or will be the owners of the infrastructure and the site. However, it is imperative that the ownership and the ownership transfer process be clarified promptly to allow sufficient time for a planned transfer. Planning for maintenance. According to cooperative leaders, none of the cooperatives have written irrigation equipment and infrastructure maintenance plans. The majority of surveyed farmers from the irrigations sites (80 percent) believed that there is a maintenance plan in place, however such plans do not yet exist. When asked about the type of future maintenance the cooperative members thought that they would be able to perform themselves and the type of maintenance they would need external support for, many did not know or had only vague ideas about what they would be able to do versus those that would require support. Thus Hinga Weze needs to bring together each cooperative with RAB and the respective District to develop a maintenance plan for each irrigation site. Operation and maintenance training for irrigation equipment. As detailed above, multiple irrigation sites are experiencing recurring failures of the irrigation equipment. Some of the breakdowns, notably the breaks and leakages of pipes, could be prevented with additional training on proper pipe usage. As part of the installation agreement, the irrigation service provider provides a one-year guarantee, and is responsible for repairing and replacing failing equipment. In addition, the irrigation service provider should provide a group of cooperative members with operation and maintenance training during the first year of operation. However, interviews with the cooperative leaders revealed that the service providers 30 have been reluctant to train cooperative members in maintenance, and that little training has taken place on any of the irrigation sites to date. Cooperative leaders shared with the evaluation team that service providers came to the site to perform repairs, but did not show the water committee or other designated cooperative members how to perform the repair. Cooperative leaders believed that the service providers wanted to preserve their market for business, and wanted to continue to perform repairs rather than teaching the cooperative how to do it themselves. While it is worth noting that some of the irrigation sites have been in operation for just three or four months, which might explain why little training has taken place, especially given the disruptions to in-person training plans brought about by COVID-19. However, Hinga Weze needs to ensure that maintenance training is taking place. Given the limited technical training that has taken place so far, it is clear that a small group of selected farmers on each site will need to get adequate, hands-on, technical training on how to operate and use the irrigation equipment as well as on how to maintain the equipment and infrastructure. Cooperative members that had received some operation and maintenance training shared that it was insufficient. For example, members of the irrigation cooperative in the Ndego Sector shared that they have received training on how to use the equipment and had made a study visit to another site. “They trained us on how to use pipes when we are irrigating… and how to open and to close, switch on and off the pump,” said one farmer. Another farmer added, “We were trained on how we can know that the pump is working or has stopped working by observing how the signals on the pump are showing When the signal … shows red, it means that the pump has stopped working, while when it is green it means that the pump is working properly. They came here and showed us all about that. They told us that when it shows the red color you should report that because that sign indicates that the pump is not functioning properly. Or if it shows the red signal, it is a difficult problem.” A third farmer concurred: “They also trained us on how to know whether the machine is properly working, or it has gotten a problem. But the training was not sufficient. They did not train us how to mend a pipe which has broken, or which has some holes. This requires the intervention of another person from elsewhere. Anyway, they have not given us sufficient training.” The group of farmers all agreed that “the training we have so far is not sufficient.” According to Hinga Weze’s maintenance and sustainability plan for the irrigation schemes, the service provider provides hands-on maintenance training to a selected group of farmers (“committee elected for maintenance”) for one year. According to Hinga Weze’s sustainability plan “After one year, beneficiaries should be able to maintain the irrigation system or hire required services for repairing damaged infrastructure, if any.” Based upon what cooperatives have shared with the evaluation team, the cooperatives will need longer and/or more intensive hands-on training. For example, the cooperative in the Ndego Sector has been in operation for a year and half, but members do not have the depth of knowledge to be able to maintain the infrastructure—and as discussed below, they do not yet have sufficient funds to hire outside help. In addition to training a smaller group of farmers at each irrigation site, it may also be beneficial to train local technicians with existing mechanical repair skills, such as auto mechanics, on how to maintain irrigation equipment. One stakeholder the evaluation team talked to shared that in 2016–2017 Rwanda Agricultural Board (RAB) partnered with the Rwanda Workforce Development Authority to train local people with relevant vocational skills, such as auto mechanics, and lead farmers on how to maintain irrigation equipment and mechanization (such as tractors). The stakeholder praised the initiative as “one 31 of the best models” he had seen. RAB’s Land Husbandry and Irrigation Unit noted that the program was good and built local capacity that was easy for irrigation scheme cooperatives to access. However, RAB was not able to budget for the program, and the program came to an end. RAB noted that “It would be a good program to start again, it was easier and cheaper for farmers to access skilled maintenance support.” Considering that several cooperatives shared that it would often take around two weeks before obtaining maintenance service, support for establishing locally available maintenance services is worth exploring. Linkages to RAB and suppliers for repair support. Interviews with cooperative leaders reveal that cooperatives experiencing infrastructure breakdowns have contacted Hinga Weze, the District, and the supplier of the irrigation equipment to get help to repair breakdowns—notably when debris is entering the water pumps. Hinga Weze staff are often visiting the sites together with District officials, and cooperative members are reporting that they are familiar with the District staff. Some cooperatives report that they contact the District directly when they have questions or need support. The close collaboration and involvement of the District during the course of the project is building the District’s institutional memory and knowledge, and the relationship between the cooperative members and the District staff can be sustained beyond the life of the project. However, multiple cooperatives report that it takes the suppliers two weeks to make it out to the site. Two weeks of a non-operating irrigation system can be detrimental to producing high value, and highly sensitive irrigated crops. Hinga Weze needs to address this issue and ensure that suppliers adhere to their obligations under the contract and respond more swiftly to calls for maintenance support. Linkages to markets. Linkages to markets is key for sustaining the investment in the irrigation system. A representative from RAB’s Irrigation and Land Husbandry Unit noted that “The main thing is access to markets. The farmers need to produce good quality crops, with access to markets, they will get a good price. With money in their pockets, that’s what will make it sustainable. They will see the value of irrigation and invest in and take care of the equipment. An out-grower scheme is a very good way to do that. But how will you make it sustainable after the project? So the production needs to work well. Quality and quantity need to be high so that it’s worthwhile for the buyer to invest time and resources in the out-grower scheme.” Hinga Weze has linked several irrigation cooperatives to buyers that have formed out-grower schemes for irrigated horticulture. Hinga Weze provides training in good agricultural practices, which the buyers build on with their own extension services to support the farmers so that they meet their high quality standards. Lotec, one of the horticulture exporters the evaluation team talked to, has out-grower schemes at two different Hinga Weze irrigation sites. The exporter provides seeds/seedlings, fertilizers and agricultural chemicals, as well as intensive extension services for the French beans and chilis grown on the sites. The exporter has an agronomist that visits each irrigation site two to three times per week to ensure that the crop meets the rigorous export requirements. The exporter noted that it is advantageous for them to work with the cooperative at the Hinga Weze site because the fields are prepared, the irrigation equipment is in place, and Hinga Weze organizes the farmers into a cooperative. However, the exporter noted that the farmers need continued support to view agriculture as a business, and view farming as a serious business endeavor. The exporter used to work with three Hinga Weze irrigation sites, but when farmers on one of the sites broke their contract by “side selling” the horticulture commodities, the exporter could not trust the farmers and pulled out of the arrangement. 32 Another exporter, Garden Fresh, is working with two Hinga Weze irrigation sites. The exporter provides the input for chili and French beans and also has a full-time agronomist on site. However, there are no sorting and grading facilities on site, which the exporter said is required for their Global Gap certification, and as a result, the exporter cannot purchase the crop. The exporter has submitted a grant application to Hinga Weze to construct the onsite sorting and grading facilities, but is still waiting to hear back. In the meantime, the farmers are getting frustrated that they are growing high quality crops that they cannot sell to the exporter because of the missing link in the value chain; the packaging house. During the remaining time of the project, it will be of vital importance that Hinga Weze continue to nurture market linkages for the irrigation sites, and to the extent possible, enable, through funding or linkages with other programs, that all the pieces of the value chain are in place with the high value horticultural markets. Moreover, continued agricultural, organizational, and business support to the farmer organizations will continue to be valuable to ensure that cooperatives manage the sites as a commercial business venture. Exporters engaged in contract farming on the irrigation sites that the evaluation team talked to stressed the importance of strengthening the cooperative management on the sites. Moreover, one exporter noted that the farmers on the irrigation sites need coaching on how to shift their mindset and consider their farming activities as a business. Targeted training on how to improve production quality to meet the exporter’s specific demands was also mentioned as an area where Hinga Weze can provide additional support. The upcoming cooperative development training will address some of these issues, which Hinga Weze can continue to reinforce. Saving for maintenance and repairs. All stakeholders the evaluations team consulted with stressed the need for the cooperatives to save for maintenance and repair needs. RAB noted that in their experience, a lot of cooperatives are not able to save the amount of money that they need. However, if the technical difficulties at the irrigation sites are addressed, and the farmers learn how to use the pipes correctly, the irrigation sites should, according to experts in the field, not need major repair for the first five years of operation, which will give the cooperative ample time to build up a reserve. Moreover, the operation cost of solar pumps is a fraction of the cost of operating diesel pumps, which should allow the cooperatives to save more money. To address misuse of water user funds, RAB has instituted a system where the cooperative and the District have a joint bank account for the collected water user funds, which provides more oversight of how the funds are used. The irrigation cooperatives collect water user fees that should be reserved for repair and maintenance. All of the irrigation sites, other than the site at Ndego, are collecting water fees. The cooperatives are charging 15,000 RWF–30,000 RWF/ha/season. Upon harvest, the cooperatives deduct the water user fees before distributing the payment from the buyers. The cooperatives decided the rate for the water user fee based on what they thought was reasonable, and some cooperatives checked with other cooperatives to see what they charged. However, the cooperatives have not obtained support to project the estimated maintenance and repair expenses and based upon the projected needs calculate how much they should save. Moreover, some cooperatives view the water fees as savings for the cooperative, and have used part of the water fee funds to pay for the cooperative’s administrative expenses. The cooperatives shared that they currently have 82,000 RWF–280,000 RWF in savings. However, the cooperative has come to realize that the savings might not be enough for the maintenance and repair they need to do. Due to the limited water supply they get from the irrigation installations however, they are 33 hard pressed to pay the existing fee. “It is not easy to find the money from our agricultural production because water is insufficient. … I fear that you will hear that the cooperative has stopped or the dam has no water due to lack of money to maintain the equipment, mainly the pump, because cooperative members have poor production and cannot pay that RFW 50,000.” The cooperatives’ ability to save money for the sites operation, maintenance, and repair needs is dependent on the irrigation systems working, the sites remain in production, and that cooperatives are linked to buyers. If all those pieces are in place, the cooperatives should be able to save money. However, if any one of those building blocks is not in place, the cooperative members ability to save will be compromised, and the sustainability of the site will be at risk. It is unclear if the amount of money collected through water user fees is sufficient to sustain future maintenance and repair needs. The cooperatives need support to work with RAB and the District to calculate how much money they will need to save to take care of their operation, maintenance, and repair expenses. TERRACING Hinga Weze is targeting to construct, rehabilitate, or valorize 2,000 hectares of terraces in six districts with high erosion risk: Nyabihu, Ngororero, Rutsiro, Karongi, Nyamasheke, and Nyamagabe. To reduce erosion and improve productivity, terracing is common in the six identified districts. However, previous terracing projects have had mixed results, where some terraces have low productivity or are abandoned. Low productivity or abandoned sites have been attributed to a number of factors including unsuitable site selections, poor terracing construction practices (topsoil and subsoil get mixed up), and limited buy-in from the local farmers. Moreover, there has been limited attention on long-term soil fertility management and applying sufficient and timely lime and organic manure. In addition, the destruction of existing crops and the construction of terracing during the planting season has led to food shortages. To avoid these mistakes, Hinga Weze conducted feasibility studies, and coordinated very closely with the Government of Rwanda at a national and local level, the development of detailed site selection criteria and selection of sites. Hinga Weze and the government partners recognized that the terracing sites should be established without mixing the topsoil (fertile) with the deep soil to preserve the soil fertility, and provided terracing packages including lime and organic manure for soil fertility, as well as trees, shrubs, and grasses to stabilize the sites. In addition, Hinga Weze involved the local farming communities in the planning and construction processes. To maintain the soil fertility, organic manure and lime need to be applied to the land every two years. The first year, Hinga Weze provided input packages, however during year three for some sites, the farmers need to purchase the organic manure and lime, and it has proved difficult for farmers to access the input. The farmers need to plan and save money for the organic manure and lime, and secondly, there is a shortage of biomass which is used in organic manure, making it hard to find organic manure, even if cooperatives have the funds. Participation of farmers in establishing and constructing the terracing sites. Hinga Weze worked closely with RAB, local authorities, and lead farmers to identify the terracing sites. The sites were selected using a comprehensive list of criteria to promote sustained use of the site. These criteria included clear ownership of the land, ongoing agricultural production, availability of access roads, feasibility of the land for terracing, and support from the Government of Rwanda to terrace the site. The farmers on the identified sites were consulted and a part of multiple dialogue meetings. The land-owners, the direct 34 beneficiaries, were also recruited as workers to construct the terraces. Due to the experience from previous terracing projects, some farmers were reluctant to provide their land for terracing. Hinga Weze conducted outreach and awareness raising campaigns to explain how the terracing techniques and soil fertility management used by Hinga Weze would be different from previous projects. Recruiting direct beneficiaries to be part of the work crew to construct the terraces had multiple advantages; they learned how to construct, rehabilitate, and repair terraces; the farmers earned money which they could save for purchasing input, and it cultivated a sense of ownership of the terraces. The thorough and participatory selection process promotes sustainable use of the terracing sites. Improved soil management to prevent soil erosion and crop losses. The terraces are built to prevent soil erosion and increase productivity in areas with high soil erosion. Soil erosion is a serious concern for farmers on the terracing sites. According to the mid-term evaluation survey, the main forms of soil erosion farmers on the Hinga Weze terracing sites are experiencing are loss of topsoil (45 percent), reduction of yield over time (26 percent), and change of soil color (8 percent). Hinga Weze has supported the farmers on the terracing sites to improve their soil management practices through a number of ways, most importantly through the construction of terraces (radical 53 percent and progressive 32 percent), planting trees (74 percent), planting cover plants (53 percent), digging trenches (45 percent), building water drainage (33 percent), and planting agroforestry (18 percent). Farmers on the Hinga Weze terracing sites said that they had experienced significant crop loss in the last two years due to extreme flooding (23 percent), heavy rainfall (9 percent) and landslides (2 percent). Seventy percent of farmers on the Hinga Weze terracing sites shared that the terracing has helped them handle risk and losses from extreme weather events. It should be noted that the survey covers a period before terracing was constructed for many sites, so some of these loss reports may be from before terracing. Active use of the terraced sites. For the terraces to continue to be used beyond the life of the project, the farmers need to start cultivating the land while Hinga Weze is operating and can support the farmers. The majority of farmers on the Hinga Weze terracing sites said that the terracing on their land was completed (85 percent) or the work was in progress (15 percent). In season 2020B, almost all farmers (99 percent) cultivated crops on their terraced land. During this same period, most farmers came to the terrace several times per week to work on the farm. About half of the farmers on the terracing sites live ten minutes or less from the site (52 percent), making it easy to access the site regularly. However, for one quarter of the farmers, it takes over 25 minutes to get to the site. Building terracing cooperatives’ capacity. Hinga Weze helped farmers on the terracing sites to form farmers’ groups/cooperatives early on in the project. The cooperatives are Hinga Weze’s entry point for engaging farmers in training, extension services, savings groups, and linkages to markets. According to Hinga Weze, there is a cooperative at every terracing site. However, not all farmers on the site consider themselves to be a member of the cooperative, as 65 percent of farmers said that they are members of the cooperative/farmer’s group on the site. Two thirds of cooperative members shared that they have attended one to four cooperative meetings in the last six months. For the terracing sites to be sustainable, the cooperatives need to be able to organize members into collective maintenance work and provide members with valuable services including collective purchasing of input, provisioning of savings groups, and aggregating marketing efforts. Moreover, the majority of farmers on the site need to be members of the cooperative for the cooperative to be effective in managing the site. Hinga Weze is 35 planning to provide the terracing and irrigation cooperatives with more intense, targeted training and support to promote the sustainability of the terracing and irrigation sites. In addition, Hinga Weze is planning to establish maintenance committees within every cooperative. The evaluation team agrees that the terracing and irrigation cooperatives need focused capacity-building attention. Accessing organic manure and lime. Experiences from other previous terracing projects has shown that the application of too little or no organic manure, lime, and mineral fertilizers reduces productivity on the site to the point where it is unsustainable to continue production. To avoid previous project’s limited attention to soil management, Hinga Weze provided lime and organic manure free of charge for the first year to farmers, to apply on the new terracing sites. Thus, in the first season, farmers applied organic manure and lime obtained from Hinga Weze, and purchased seeds and mineral fertilizers. The provision of organic manure/compost and lime, combined with the use of improved seeds and the application of mineral fertilizers, farmers saw increased production. An implementing partner staff member shared with the evaluation team how farmers that previously produced 80 kilograms (kgs) of Irish potatoes were now producing 350 kgs after receiving the input packages. Organic manure is applied every season and lime needs to be applied every two years to ensure soil fertility. Consequently, for the first terracing sites constructed with Hinga Weze support, season 2021A (September 2020–January 2021) was the first time the farmer cooperatives had to purchase organic manure and lime themselves. Lime is subsidized by the government for the first year after the construction/rehabilitation/valorization of a terrace, but for subsequent years it is not. The evaluation team met with two terracing cooperatives that shared that they collected funds from their members to purchase lime, but they were unable to purchase organic manure. One of the cooperatives did not have sufficient funds to purchase the organic manure, while the second cooperative had been very successful producing and selling Irish potatoes, and had the money, but could not find organic manure to purchase. Conversations with the implementing partners in the districts confirmed that it is 1) expensive for the terracing cooperatives to purchase organic manure and lime and 2) even if they have the money, there is limited supply of organic manure on the market. As a result, the two terracing cooperatives were planning to plant season 2021A without applying organic manure. Table 4 below details the cost of input for terracing amendments that practice crop rotation. Conversations with leaders in the terracing cooperatives as well as the implementing partners highlighted that there is a shortage of biomass, which is used for blending the organic manure. The shortage of biomass is a bottleneck for producing organic manure, and thus leads to a shortage. Several stakeholders the evaluation team talked to identified the provision of organic manure as the key challenge for sustaining the terraces beyond the life of the project. 36 TABLE 4: COST OF INPUTS FOR TERRACING AMENDMENTS TYPE OF INPUT APPLICATION RATE UNITY COST RWF/MT COST 1 HA (RWF) Travertine (radical terraces) 5 MT/Ha 40,000–45,000 200,000–225,000 Lime (progressive terraces) 3 MT/Ha 70,000–80,000 210,000–240,000 Organic manure/compost (radical and progressive terraces) 10 MT/Ha 25,000–35,000 250,000–350,000 Hinga Weze has trained farmers on the terracing sites on composting techniques and encourages them to use the compost as an alternative to organic manure. However, it was unclear to the evaluation team what the uptake of composting is and if sufficient quantities are produced for the application to improve soil fertility. Relatively few (16 percent) farmers on the terracing sites have obtained the Hinga Weze husbandry kits (see evaluation question 4 for further details on the husbandry kits), and just a few (six) are using the animals to produce manure for fertilization. However, almost one third of farmers on the Hinga Weze terracing sites have purchased small livestock using money they saved or borrowed from the savings groups, which they could use for producing some organic manure, if they also had access to biomass. The organic manure shortage is a serious issue that needs to be further studied, and is a threat to the sustainability of the terracing sites. Possible solutions, including the planting of agroforestry to use for biomass, encouraging and supporting members of terracing cooperatives to save for and invest in livestock (for manure), partnering with the private sector, the government, or development partners that can invest in organic manure production, need to be investigated. Moreover, Hinga Weze supported the production of organic manure for the first season of terracing, so it is possible that parts of that arrangement can be replicated for commercial production without financial support from Hinga Weze. The ability of cooperatives to purchase lime and organic manure is the greatest threat to the terracing sites’ sustainability and is therefore is an area needing immediate attention. Training and planning for maintenance. The majority of farmers in the survey said that they have been trained on how to maintain the terrace (59 percent), but a significant portion of farmers said they have not received this maintenance training (41 percent). As sites are new and some have not yet started maintenance work, it is possible that the training lead farmers, foremen, and cooperative leaders have received has not yet trickled down to all members. As Hinga Weze continues to strengthen the capacity of cooperatives, cooperatives are developing maintenance plans for the sites. Thirty-four percent of farmers on the Hinga Weze terracing sites said that the cooperative has a maintenance plan for the terraced areas; while the majority said that there is either no maintenance plan (12 percent) or they did not know if such a plan existed (54 percent). Table 4 Source: Key Informant Interview with Hinga Weze Agronomy Advisor 37 However, it’s important to note that this statistic is based upon the farmer’s perception, and that this mid￾term evaluation did not survey cooperatives to investigate how many cooperatives do in fact have maintenance plans. The terracing cooperatives the evaluation team visited did not have a maintenance plan. A maintenance plan allows the cooperatives to plan ahead and save money for maintenance, especially for lime and organic manure. Thus, it is beneficial if more cooperatives put maintenance plans in place, engaging their members in developing those plans. Although there may or may not be a formal maintenance plan in place, maintenance of the terracing sites is already ongoing and two-thirds of farmers on the terracing sites shared that they have already worked on maintaining the terraces. Together with local authorities, terracing sites are organizing community work (umuganda) to reshape risers, level bench slopes, replace damaged agroforestry trees, and maintain the drainage system. In the survey, farmers on the Hinga Weze terracing sites were asked what future maintenance they think they will have the knowledge, capacity, and resources to complete. The farmers shared that they could clean the ditches, dig new ditches for rainwater run-off, plant grass and trees to avoid soil erosion, and repair the terraces. For example, one farmer shared “There are trees which are dry on our farms. In a meeting we did, we concluded that group members will plant other trees together to replace the old ones.” However, there were also a fair number of farmers who said they did not know or had not yet thought about repairs. Others said that they needed external support for more involved maintenance like planting trees. With well-functioning cooperatives in place, the majority of maintenance work on the terracing sites can be carried out by the farmers on the site. To provide guidance and oversight over the maintenance of the terraces, Hinga Weze plans to establish district-level steering committees for terrace infrastructure management. The committees will be made up of district, sector, and village level representatives, farmer promoters and farmer field schools representatives, the terracing cooperatives, as well as NGOs engaged in agricultural development in the district. To ensure sustainability of the steering committees, it will be important that the district takes on ownership and organizes the work and coordination of the committee. Saving for organic manure, lime, and maintenance. Collecting funds for organic manure and lime application, as well as other maintenance and repairs costs, allows cooperatives to plan for good soil management and regular maintenance of the terracing sites. However, according to the mid-term evaluation survey, few cooperatives on the terracing sites collect funds from their members for future maintenance costs. Just six percent of farmers on the terracing sites said that they contribute financially towards the cooperative’s maintenance fund. Instead, some farmers suggested that they will collect money at the time when they need to do maintenance work or repairs, rather than creating a savings fund. One farmer shared “In case there is somewhere to repair [on the terrace], we can collect money from group members to use.” However, even though the level of organized collective savings is low, 22 percent of farmers on the terracing sites said that they are personally saving money from one season to the next for lime and other soil amendment materials. The main expense for maintaining good soil fertility and maintaining the site is the cost of lime and organic manure. As those costs are significant, the cooperative needs support to budget for how much funds to save to be able to purchase sufficient lime and organic manure. In addition, considering that the availability of organic manure is limited, the cooperatives are more likely to be able to secure the quantity they need if they have the funds and are able to make arrangements with providers early. 38 Sixty-one percent of the farmers on the terracing site shared that they or a member of their household is a member of a savings group started by Hinga Weze. The members of the savings groups use the money to cover a variety of household expenses, but a large proportion also used part of their funds from the savings groups for farming input (68 percent) and other farm related activities (61 percent). The savings groups can be used as a vehicle to save for the purchase of lime and organic manure, but the cooperative needs to know how much each member (based on the size of the plot) needs to contribute. Extension services for farming on terracing sites. Productivity on the terracing sites need to increase for the sites to be sustainable. Hinga Weze provides farmers on the terracing sites with extension services and demonstrations on specific farming techniques for terracing sites. Hinga Weze has established demonstration plots on Twigire Muhinzi farmers’ terraced land. The demonstration plots are used to demonstrate how to prepare the soil, planting techniques, cultivating new varieties of crops and using improved seeds, as well as applying organic manure, lime, and mineral fertilizers. Implementing partners are supporting farmers to register for the input subsidy program SNS. In addition, the farmers are also trained on post-harvest handling and storage. Three quarters of the farmers on the terracing site said that they had obtained extension services from Twigire Muhinzi or farmer promoters in the last year. Almost all felt that the visit(s) focused on issues that were very relevant (70 percent) or for the most part relevant (25 percent) to improving their farming practices. Many farmers on the terracing sites shared that following the extension services they adopted good agricultural practices for planting, input application, post-harvest handling, and some noted that they had adopted new practices for terracing such as lime application. Hinga Weze is planning to continue the extension services and hands on training in good agricultural practices on the terracing sites. Hinga Weze is also planning to focus resources on training on making compost and planting high biomass crops to provide biomass for organic manure/composting. The evaluation team agrees that training and support on making compost and planting biomass crops should be a priority for the next two years. Access to markets. As the farmers have improved the soil fertility and are using more improved seeds and fertilizers, their production is increasing and they have a surplus for marketing. One terracing cooperative the evaluation team met shared that farmers had increased their production from 80kg of Irish potatoes to 350kg after the terracing and input packages intervention. Another cooperative shared that they had increased their production of maize from 300kg to 740kg. Hinga Weze has connected the potato growing cooperative to a buyer, and after the sale, the cooperative had sufficient funds to purchase lime and organic manure (but as highlighted above, the cooperative was not able to locate a supplier of organic manure). Consequently, the market linkages are another key piece in ensuring sustainability of the terracing sites. The second cooperative producing maize however, had not been connected with a buyer, and even if they had been, they did not yet have the knowledge and awareness about how to aggregate, let alone negotiate with a buyer. The cooperative did not know what their yields were; they each harvested individually, did not differentiate between grades following the harvest, and did not discuss how much they had harvested or how much they wanted to sell as a group. Some members of the cooperative were interested in aggregating, while others did not yet understand the collective aspect of a cooperative, or were skeptical and resisting collective actions. Moreover, they had not considered if they had the infrastructure to 39 aggregate or if the access roads were conducive for a truck to pick up the commodities. Consequently, Hinga Weze needs to capacitate the cooperatives to get market ready, and then link them with buyers. Both irrigation and terracing schemes under Hinga Weze are fairly new, so there has not been much time to manifest observable results in all cases. While farmers do indicate a clear understanding that they will own the irrigation and terracing equipment in the long term, the handover has not taken place yet and as noted in detail above, there is still some ambiguity around how those handovers will be structured, as some documentation contains references to Government of Rwanda ownership. The maintenance and function of irrigation equipment is also an ongoing issue. Five out of seven sites reported inadequate water, in many cases citing solar pumps that only functioned during daylight hours and were not strong enough to pump sufficient water. If farmers are not satisfied with the equipment, it will be difficult to convince them to invest time and other resources in maintaining the equipment, which will impede progress towards ownership. Long waits for maintenance are another issue and there is a need for capacity-building within irrigation cooperatives around maintenance. Because it appears that many terracing activities are in their initial stages, it is difficult to draw many conclusions about their progress at this point. Only about 60 percent of farmers in terracing areas reported receiving training on maintenance and it will be difficult to ensure proper maintenance plans are in place without more access to training. It is clear that inputs such as lime and fertilizer will be critical to the sustainability of terraced land, but there does not seem to be much organization among farmers at this point around how they will save for and pay for those inputs. These farmers will also need extension training that is specific to terraced land, as well as training on market linkages. It is the evaluation team’s understanding that Hinga Weze is in the processing of planning and implementing these types of trainings at the time of this writing. 2.3 EVALUATION QUESTION 3: COST-BENEFIT ANALYSIS Hinga Weze has promoted a number of agricultural technologies aimed at increasing farmer-level productivity and climate resilience. Which are the top three technologies on a cost-benefit basis that would be recommended to continue to roll out at scale over the rest of the activity period? This section provides results and a brief description of the methodology of a (CBA) for select interventions under the Hinga Weze project. See Annex B for the full analysis. A CBA compares the costs and benefits of an investment with the costs and benefits of a status-quo situation where an individual or organization does not make the investment. The evaluation team developed a series of CBA models to estimate the net impact of select interventions under the Hinga Weze project. The models use monitoring data from the first two years of the project, as well as data from the mid-term evaluation survey, to evaluate progress to date and to forecast the returns over the next several years. In total, the CBA time horizon estimates the costs and benefits to USAID beneficiaries over a ten-year time horizon, starting with the point when the beneficiary first receives the assistance. Using the baseline beneficiary survey, the 2018 and 2019 progress surveys, and the mid-term evaluation survey, the team analyzed the changes in farmers yields and costs. It is important to note that the data collected had some limitations for our analysis. First, the baseline survey was not of Hinga Weze beneficiaries, but of a population believed to be a representative sample. Secondly, the subsequent data in the beneficiary survey and the survey for this evaluation only included beneficiaries and therefore our 40 findings are not based on analyzing both beneficiaries and a control group, but by comparing the beneficiaries to a constructed counterfactual. Because the data does not allow us to follow the same farmers from baseline to 2020 and we do not have a data-based counterfactual, some bias could remain in the results. Based on consultations with the Mission, the CBA focused on three key crops: maize, Irish potato, and orange flesh sweet potato (OFSP). For Irish potato and OFSP, the team developed general CBA models analyzing the combined impact of all interventions on these crops. This was due to the smaller sample of farmers that grew these crops. The result of this analysis shows the impact of Hinga Weze on beneficiaries; however, it does not determine which technologies work best for these crops. For Hinga Weze maize farmers, CBA models were developed for interventions where there was enough data to produce meaningful results. These interventions include the following: Crop Genetics: Including improved or certified seeds that could be higher-yielding and more resilient to climate impacts. Cultural Practices: Including agriculture management techniques such as seedling production and transplantation cultivation practices. Pest Management: Including Integrated Pest Management, improved insecticides and pesticides, and improved and environmentally sustainable use of insecticides and pesticides. Soil-related Fertility and Conservation: Including soil management practices that increase biotic activity and soil organic matter levels, such as soil amendments that increase fertilizer-use efficiency. Genetics, Pest Management, and Soil Conservation: Represents farmers receiving all three of these interventions. All Remaining Interventions: Includes all remaining interventions that were not assessed separately due to an absence of data or the fact that the intervention was provided in various combinations with other interventions. Table 5 presents the benefits and costs used in the models. Detailed information on how these were calculated is available in Annex B. TABLE 5: BENEFITS AND COSTS BENEFITS COSTS Increased productivity Physical inputs Reduced losses due to increased climate change resilience Hired labor Reduced Co2 emissions due to improved input use Loan interest Table 5 Source: Authors 41 SENSITIVITY ANALYSIS To build cost-benefit models, it is necessary to make assumptions and projections about key variables that could change in the future. The uncertainty inherent in those assumptions impacts the level of validity attributed to the result, which is why it is essential to analyze the sensitivity of the model to those assumptions. This is done using one-way and two-way tables that show how the result changes with modifications to the values of certain parameters, everything else being held constant. Sensitivity analysis was performed for each of the benefit and cost measures for crop price changes, input price changes, and wage rate changes. Detailed results can be seen in Annex B. RESULTS Figure 2: Farm Household Incremental Net Present Value (NPV) for 1.23 hectares Maize The main benefit for farmers is increased yields and the intervention showing the greatest benefits is genetics, at 3.4 million RWF/ha, followed by the combination of interventions to include soil conservation, pesticides, and genetics at 2.6 million RWF/ha (Figures and Tables for all results are available in Annex B). It is important to note that this analysis could not control for selection bias as the farmers were able to choose the interventions that were best suited for them. This could help explain why the intervention including soil conservation, pesticides, and genetics did not perform as well as genetics alone. The farmers who adopted pesticides were most likely facing more pest issues than those who adopted genetics alone. Due to the limitations of the data discussed above, it was not possible to isolate these impacts. Figure 2 Source: Authors 42 The interventions with the lowest NPV of benefits are the planting interventions at 413,376 RWF/ha and the soil conservation intervention at 2,600,412 RWF/ha. The average maize farmer under all Hinga Weze interventions has an average NPV of 1,052,322 RWF/ha benefits, compared to 215,826 PV of benefits RWF/ha for OFSP producers. Note that the team did not attempt to quantify nutrition-related benefits due to the complexity and inherent uncertainty of such calculations. To the extent that nutrition-sensitive agricultural commodities such as OFSP improve nutrition among beneficiaries (addressed more in findings for Question 4), the benefits can be assumed to be somewhat higher than what is quantified here. While soil conservation had lower increases in yields, it also had much lower increases in costs at a PV of 48,048 RWF/ha. Planting and cultural practices, which also did not demonstrate a very large increase in yields, had much higher costs at an PV of 340,171 RWF/ha. Similarly, all other interventions led to a high PV of costs at 463,168 RWF/ha. Adoption of genetics was associated with higher costs through physical inputs but reduced costs from hired labor. Overall, this resulted in a net increase in the NPV of costs by 356,689 RWF/ha. Costs of a similar pattern can be seen for adoption of improved pesticide practices (PV of costs of 296,600 RWF/ha), and the bundle including soil conservation, genetics, and pesticides (PV of costs of 133,937 RWF/ha). For a producer of Irish potatoes, a farmer has incremental savings of 106,491 RWF/ha and OFPS farmers also had incremental savings with an NPV of 277,550 RWF/ha. Figure 2 provides a summary of the incremental NPV of maize farmers under one of the Hinga Weze interventions. These results assume that farmers devote an average of 0.23 hectares of land to maize production and the CBA time-horizon is 10-years. Farmers participating in the genetics intervention experience an average NPV of 784,034 RWF. This compares to 598,095 RWF for the combination of soil conservation, pesticides, and genetics; 323,201 RWF for pesticides; 242,034 RWF for the combination of all other interventions; 95,076 RWF for soil conservation; and 74,807 RWF for planting/cultural practices. Irish potato farmers had an incremental net benefit with an average PV of 456,461 RWF and the net benefits of OFSP farmers had an average PV of 215,826 RWF. The CBA used an average cost of $30 per beneficiary and an overall beneficiary population of 533,000 to estimate the net project costs and benefits. The cost per beneficiary from the USAID perspective was calculated by dividing the budget for training farmers by the number of beneficiaries. The CBA also used survey data and performance documents to estimate the percentage of the population who are under each intervention and crop. For example, the model assumes that 70 percent of the beneficiary population are producing maize under one of the Hinga Weze interventions, compared to 20 percent who are producing Irish potato, and 4.4 percent producing OFSP. In addition, the CBA model assumes that 90 percent of trained farmers will adopt the technologies and farming practices, while there will be a 5 percent attribution rate after the Hinga Weze project has ended. Each of these assumptions was in the sensitivity analysis. With the exception of planting/cultural practices, the average farmer is experiencing a net benefit, with those under the genetics intervention experiencing a net benefit (NPV) ranging from 582,476 RWF ($611) to 718,682 RWF ($753). Farmers under some other intervention or combination of interventions (i.e. those not evaluated independently in this analysis), as well as those applying pesticides or soil conservation practices, are experiencing an NPV ranging from 105,608 ($112) to 276,965 ($290). Farmers participating in the planting/cultural practices experience an NPV of 16,888 RWF ($18). The primary drivers for this negative return are lower yields and higher hired labor costs. Hinga Weze farmers producing Irish potato 43 have an incremental NPV of 1,236,477 ($512), while OFSP producers have an incremental NPV of 298,424 ($313). One of the key drivers for the large increase in returns for Irish potato producers is the estimated reduction in the hired labor needed for land preparation, weeding, and harvesting rather than large increases in yields. To estimate the overall net benefits and costs of the project, the evaluation team has included the project costs that have been used to advance new technologies and train farmers. At this point in time, the CBA model does not incorporate a wider set of impacts beyond those that come with immediate financial implications for the farmers, including the impact of trade-based distortions (e.g. tariffs or subsidies). Based on this approach, maize interventions under the Hinga Weze project are expected to create an NPV of $40 million, based on an NPV of $53 million in benefits and $13 million in costs. Within maize, the intervention with the greatest productivity was genetics (improved seed), followed by the intervention including genetics, soil conservation practices, and improved pest control practices, and the third intervention with the best performance for maize was improved pest control practices. The project’s interventions under Irish potato create an estimated NPV of $29 million, based on an NPV of $33 million in benefits and $4 million in costs. Finally, the project creates an estimated NPV of $2.8 million under its OFSP interventions, based on an NPV of $34.6 million in benefits and $0.81 million in costs. There are several considerations when interpreting these results. First, while the team was able to build models for a limited set of interventions, some interventions were not analyzed due to data limitations or the absence of data. For example, there was not enough production data from irrigation farmers to develop a representative model that differed greatly from irrigation CBA models that had previously been developed by CNFA. In addition, while the cost data for this analysis comes primarily from the project’s 2018 progress survey, the method for collecting hired labor cost data in this survey differed from the approach used in the baseline survey. This means there could be discrepancies in the estimated labor costs reported in the CBAs. The overall project benefits are sensitive to an assumed project cost of $30 per beneficiary, which could vary over time and under changing conditions. Finally, the CBA model does not include higher-level economic impacts of these interventions. For example, the team has not incorporated fertilizer subsidies that the Government of Rwanda provides to farmers, meaning an intervention promoting higher fertilizer use could ultimately cause higher costs for the Government of Rwanda. These kinds of high-level, economy-wide impacts would be best assessed through an in-depth impact evaluation. 2.4 EVALUATION QUESTION 4: NUTRITION To what extent do Hinga Weze’s agricultural productivity and market access interventions contribute towards improved nutritional outcomes for women and children? In what ways do the current interventions address/not address the underlying constraints towards improved nutritional status of the target households? Hinga Weze set a goal of improving by 40 percent the number of women ages 15–49 and children ages 0–23 months consuming diverse, minimally acceptable diets. According to the Ministry of Health, national targets for stunting are 29.9 percent in 2020 and 19 percent in 2024. Stunting rates are currently quite high, estimated at 33 percent by the 2020 Rwanda DHS, but they vary dramatically across income quintiles. The DHS found a 49 percent stunting rate in the lowest income quintile and 11 percent in the highest 44 quintile. There are high levels of exclusive breastfeeding up to six months of age but proper complementary feeding after six months is less common (CNFA 2019). According to the 2020 Demographic and Health Survey (DHS), only 22 percent of children aged 6–23 months were fed in accordance with standards. Only 28 percent of women 15–49 meet the requirements for minimum dietary diversity (CFSVA 2018). Question 4 can be broken out into two interdependent questions: first, did the interventions under Hinga Weze address the main underlying constraints to nutrition, and second, did interventions carried out under Components 1 and 2 have an effect on those outcomes. To address these questions, the team examined results from monitoring surveys, the MTE quantitative survey, KIIs with experts in nutrition, gender, and child development, and FGDs with members of Care Groups and Community-Based Volunteers (CBVs)—also referred to in this section as Community Health Workers (CHWs). FOOD SECURITY AND CONSTRAINTS TO NUTRITION Pillars of Food Security: Food security comprises four pillars, each of which poses unique challenges to improving nutrition: Access, Availability, Utilization, and Stability. Constraints and outcomes are examined here through the lens of those pillars. Undernutrition in Rwanda arises from a multitude of complex and interacting factors, which vary by geography. As a result, Hinga Weze incorporated a broad range of approaches into its nutrition-focused activities, intended to address the full spectrum of constraints. In the East, where drought is more common, households more commonly report being constrained by availability of food and instability, whereas in the West, chronic malnutrition persists despite availability of food, due to lack of knowledge, undervaluing of nutrition, or disparities of decision￾making power within the household. According to a Hinga Weze partner, the USAID-supported Gikuriro program, implemented by Catholic Relief Services (CRS), farmers in the West often sell what they produce, only retaining lower value foods for home consumption and not supplementing with purchases (despite low levels of food purchases, households still spend approximately half their monthly incomes on food (CFSVA, 2018). Hygiene and food safety affect utilization in all regions because utilization reflects how people are able to process and maintain health through nutrient consumption. When hygiene and food safety are compromised, both long and short-term food-borne illnesses are more common, which can have a significant impact on nutritional status, particularly for small children. In the survey conducted for this evaluation, 89 percent of respondents reported having access to nutritional food to make changes in their diets, but only 68 percent report having the financial resources to change their diets. While caution is necessary in interpreting the results of a single survey, this result does suggest that production increases are playing an important role in improved diets. This is supported by other results from this survey: 58 percent of respondents report having increased diversity within the household diet and 17 percent report having added more food overall. FGD participants also noted the importance of the poultry they received through Hinga Weze in increasing their access to animal proteins through egg and chicken consumption. Gender Differences in Labor and Decision-Making: Gender-based divisions of labor pose a significant challenge to improving nutrition. While men are encouraged to participate and Hinga Weze works to identify “male champions,” interviewees reported that it is difficult to get them to attend activities like cooking demonstrations. CRS reports that as men see the success of Care Groups, 45 particularly in terms of dietary improvements and savings groups, they are showing more interest. This poses a potential new problem however in that it is important to maintain women’s access to these benefits, particularly savings groups, and prevent crowding out by men. Women having control of financial assets is key to increasing investment in nutrition. Savings groups within the Care Groups in particular are often cited as an important benefit that allows women to improve nutrition, invest in kitchen gardens and small livestock, and smooth consumption through shocks. Evidence suggests that a loss of control over this resource by women would significantly undermine delivery of those benefits to the household. For example, the most recent DHS survey shows animal protein deficiency as a significant concern, a result that was also reported in KIIs and in Hinga Weze’s SBC Gap Analysis (CNFA 2019). This is both an access and availability issue, as within the same household, it may be true that male adults and/or male children consume adequate animal protein, while females do not. There is no evidence to suggest that the reverse is common, where adult females and/or female children receive preferential access to food resources. This also connects to disparities in decision-making power. Hinga Weze performed a Social and Behavior Change (SBC) gap analysis in July 2019 which illuminated many of the constraints to improving nutrition. The analysis found that overall, most households derive most of their consumption from what they produce and do not purchase much. It also found that for most household members, the main source of animal protein is dried fish. Eggs and meat are rarely purchased for household consumption, but men often consume meat at bars, purchasing “beer and brochette (goat meat).” Men are also prioritized for food consumption within the household based on the assumption that non-household work has greater energy requirements. Women are not often in control of household monetary resources, so even households that report having financial resources to purchase nutritious foods may not be making those purchases. Respondents reported being aware of special dietary needs for pregnant or lactating mothers, but that they did not make changes based on that awareness. Limited knowledge of food safety and lack of access to refrigeration also impacts nutrition via utilization of nutrients. For example, households report consuming raw milk that has not been refrigerated for up to two days after purchase. This is especially dangerous for young children, highlighting the need to emphasize avoiding fresh cow milk consumption for children under 12 months old. Women reported limited income earning opportunities outside the home and they receive lower wages for that employment. Mothers working outside the home can also negatively impact child feeding if other household members or caregivers do not have access to information and resources for proper child feeding. In addition, nearly all domestic labor is done by women, limiting the time they have to work outside the home, although they report having greater influence on household decisions the more money they earn (more evidence of rigid gender-based divisions of labor being a major barrier to improved nutrition.) Members of a Care Group that participated in an FGD discussed the importance of women’s rights to control resources and that because they have access to money through participation in the Care Group, they are able to spend on what is most important for the household. HINGA WEZE RESPONSES Behavior Change and Education: Behavior change is of course the key to any nutrition intervention. As noted previously, 58 percent of respondents report having increased diversity within the household diet and 17 percent report having added more food overall. Care Group members that participated in a 46 focus group discussion for this evaluation reported observing a reduction in cases of Kwashiorkor, a condition afflicting children that arises from a lack of protein in the diet. This condition affects children that are receiving adequate calories, just not enough protein, which highlights the importance of having access to sources of protein and for adults in the household, including adult males, to have sufficient understanding of children’s dietary needs. Women in this Care Group also reported pooling food resources to produce more nutritious food, with members contributing what they had available and making use of what they had learned from cooking lessons to produce more nutritious food for the group than they would have been able to produce individually. They also report increasing cultivation of high￾iron beans after learning of the nutritional benefits as well as improving practices around hand washing and other sanitation and food handling measures at the household level (these practices are key to increasing utilization). Another example, Care Group members provided of nutrition-driven behavior change was that they previously preferred to cook foods that “multiply themselves” such as grains and potatoes, but now understand the value of including greens, groundnuts, and dried fish (dried fish are the most common source of animal protein at the household level, with other kinds of meat generally consumed by men outside the home). Related to this, FGD participants also discussed the importance of poultry and eggs to their diets, a direct result of Hinga Weze Care Group activities, so this suggests that Hinga Weze interventions have diversified the protein sources available to households and potentially the amount of protein available, although that is not possible to say definitely based on available data. Care Groups: In the mid-term evaluation survey, 185 out of 408 households (45 percent) report that a member of their household participates in a Care Group. Of households that have at least one pregnant or lactating mother, 54 percent report participating in a Care Group (43 out of 84). Of households that report participation in a Care Group, 44 percent indicate that someone in the household is a member of a cooperative. This is a surprising result given that outreach around Care Groups is generally linked to cooperative-related activities, but from interviews, many Care Groups members reported that they were drawn to join the group after observing the benefits (in particular, access to savings groups) that accrued to others in their community participating in the group. This suggests that there may be untapped potential for recruitment into Care Groups through activities under Components 1 and 2. There is also evidence from the survey of nutrition education among respondents who did not report participation by their households in a Care Group. Seventy-eight percent of respondents reported receiving some kind of nutrition education even though only 45 percent reported a Care Group member in the household. This could be the result of education from other sources, including community health centers, spillover of benefits from Care Groups being in the community, or some combination of the two. Nutrition messages within the Care Groups focused on value chains covered by Hinga Weze: maize, OFSP, Irish potato, high-iron beans, and horticulture. Survey respondents reported high levels of participation in all Care Group activities, including nutrition and cooking classes, cooking demonstrations, kitchen gardens, and hygiene and sanitation/food safety. Eighty-two percent reported learning something from the interventions with 59 percent reporting that they learned “a lot.” Reported levels of learning did not vary significantly across types of intervention. Community Based Health Workers: Households surveyed for this assessment did cite Community Health Workers (CHWs) as an important influence, indicating that Hinga Weze investments in training of these volunteers is providing positive returns. Person-to-person communication was cited as the main 47 source of information transfer, which is something to consider during COVID-19, when such communication may be subject to restrictions or unpredictability. Mass media, in particular radio, has been shown to be an effective way of communicating health messages, and is a relatively low cost way to reach large numbers of people. This may be an option to maintain messaging and information flow during COVID-19 restrictions. Some Care Group members also noted that messages around nutrition were received at health centers when they went for vaccinations and other services, which may be an important opportunity for further collaboration. Participation in other Hinga Weze Activities: Seventy-eight percent of households that participate in savings groups (81 percent when the respondent was female) report having the financial resources to make changes to their diets, another indicator that participation in savings groups/access to finance, particularly for women, is a crucial component of successful nutrition interventions. FGD participants also noted that increased availability of production for consumption freed up money to purchase vegetables, an important element of increased dietary diversity. A comparison of Hinga Weze monitoring surveys performed in 2018 and 2019 shows that a large proportion of households that experience income increases report investing that income in increased purchases of nutritious food (65 percent and 67 percent, respectively.) According to those same surveys however, less than 20 percent of households reported increased incomes. As discussed in the response to question 1, Hinga Weze participants are reporting positive changes in food expenditures, suggesting that they are at least allocating more income towards food. Allocation of any increased income towards improved nutrition is not a certainty, but other results from the MTE survey do indicate that participation in Hinga Weze interventions (which per question one appears to be correlated with increased incomes) is also correlated with improved nutrition. In looking at the two Hinga Weze interventions ranked as of highest importance by respondents, savings groups and linkages with buyers, we see that 73 percent of households that listed savings groups as among the top three most important reported purchasing more food versus 58 percent of those who did not. The sample size was too small to discern the reasons for purchasing more food so it is not possible to say for certain that nutrition was a driver, but given that Hinga Weze beneficiaries also broadly report increased production, it is reasonable to assume that it was not scarcity of food that drove purchasing decisions. Of these same households, 67 percent report that income was not a constraint to proper nutrition. An interesting result was that of households that either did not participate in savings groups or list them as important, 75 percent did not report income being a constraint to nutrition. It is not possible to discern, given available data at this point, whether this is because households experiencing income constraints are more likely to select into savings groups or view them as important or whether participation in savings groups may cause a (possibly temporary) increase in the constraint on food budgets, due to the allocation of additional income to savings. The overall positive view of savings groups across all groups both interviewed and surveyed suggests that even if the latter is true, households view the tradeoff as worthwhile. Survey results are similar for households that ranked linkages with buyers as among the top three most important services they received through Hinga Weze. Of those households, 76 percent reported purchasing more food versus 59 percent of those who did not rank these linkages in the top three. Eighty￾one percent of these households reported that income was not a constraint to nutrition for them. 48 In conclusion, activities under Component 1 and Component 2 contribute to all four pillars of food security. Some income gains are used to purchase additional and more nutritious foods; at least some increased production is allocated to consumption; improved post-harvest handling is important to food safety (utilization); and access to finance and increased incomes are also likely to contribute to stability, but that would need to be measured over a longer period of time. Nutrition sensitive agriculture has been effective in achieving “saturation” of nutrition messages, as evidenced by the high number of households reporting learning from trainings and the high number reporting positive changes to their diets (99 percent of households in the MTE survey reported at least one positive change in diversity, amount, or understanding of nutrition and feeding practices.) Education around nutrition has also been effective in identifying and strengthening the numbers of male champions. In interviews conducted with implementers by Kigali-based evaluation team members, they reported that they do work with farmers to understand the nutritional value of the commodities they grow and while this does include messages about food-sharing at the household level and proper child nutrition and feeding practices, the results of CNFA’s 2019 Social and Behavioral Change assessment concerning equity of decision-making and allocation of food resources indicated that there is significant room for growth in the role of male champions. One of the clearest contributions of market access and production activities can be seen in the form of majority female cooperatives. In an interview with the evaluation team, the KOAIM I processors’ cooperative reported receiving training in savings and credit groups, post-harvest handling, nutrition, and being linked with Savings and Credit Cooperatives (SACCOs) through Hinga Weze. As was true for other cooperatives (see questions 1 and 2) they had not received business management training. A focus group conducted with a women’s buyers’ cooperative reported similar types of training to those reported by other cooperatives and noted that one of the initial purposes of creating the cooperative was to fight malnutrition. They report that one of the important benefits of Hinga Weze has been mobilizing farmers around nutrition and the production of more fruits and vegetables. For their own work, they report that linkages with vegetable suppliers have been helpful to them. The constraints to nutritional improvements are numerous: food availability, which relates to production and markets, lack of knowledge, undervaluing of nutrition, and disparities of decision-making power within the household. Improved nutritional outcomes are therefore a result of several factors working together. Having access to more nutritious food does not guarantee improved nutrition, in the absence of financial resources and behavior change/education that promotes prioritization of nutrition, especially for young children and pregnant/lactating women when it comes to household expenditures. Hinga Weze’s market access and productivity activities are clearly correlated with increased availability of and access to nutritious foods, through both increased production and increased income. Training and education around nutrition-sensitive agriculture also appear to have contributed to widespread uptake of nutrition messages, although an in-depth impact study would be required to better quantify the impacts of different types of nutrition education messaging. To continue along this path of behavior change in support of nutrition outcomes however, there is much more that could be done in regard to gender disparities in decision-making and equitable access to household resources. 49 2.5 EVALUATION QUESTION 5: CAPACITY-BUILDING Hinga Weze’s approaches to increased productivity, improved market access, and enhanced nutrition are all driven in large part by grassroots-level community outreach organizations and volunteers. To what extent are the capacity￾building approaches for Twigire Muhinzi Extension model, Community-Based Volunteers (CBVs), and Village Savings and Credit Mechanisms pursued by Hinga Weze led to institutionalized improvements that can be maintained after the activity ends? Informants universally cited Hinga Weze’s coordination with and alignment with the Rwandan Government and other key stakeholders as one of its most impressive features, and when asked to comment on sustainability, indicated that these efforts would help to ensure continuity beyond the life of the project. One key aspect of Hinga Weze’s ability to align with and contribute to efforts to operationalize existing Rwandan strategies around agriculture and food security was the incorporation of existing models and systems into its activities. The use of Community-Based Volunteers (CBVs) as both Farmer Promoters and Community Health Workers (CHWs) is at the core of many of the project’s activities and CBVs in both agriculture and health predate the beginning of Hinga Weze by several years, meaning that Hinga Weze was effectively able to “buy in” to existing structures, avoiding creating parallel activities that would then have to be handed over to authorities during close-out of the project. To evaluate specific approaches to capacity development and correlated institutional change, it is first important to define what is being assessed. Sustainable capacity-building requires a focus not just on outputs or results, but on processes, as it is institutionalization of these processes that will ensure continuity of services beyond the life of assistance programs. USAID’s Capacity Development Recommendations (2017) cite four “Domains of Performance”: Effectiveness (achieving results, meeting standards), Relevance (engaging stakeholders, learning), Efficiency (delivering services, enhancing reach), and Sustainability (mobilizing resources, increasing social capital). While a deep dive into each of these elements for Hinga Weze’s capacity-building activities is beyond the scope of this evaluation, these areas were used to anchor the analysis. USAID’s recommendations note the importance of local ownership and linkages to performance measures, but also that capacity development is a non-linear process, a point which is important to take into account when attempting to evaluate capacity-building efforts at the mid￾point of an activity. This is because at this point in the activity, depending on the type of training and the target audience, it may not yet be possible to observe outcomes. As such, it is also important to consider outputs, such as number of people trained at different levels and in different geographic areas, as a key advantage of approaches such as the cascade “Training of Trainers” approach used in many activities by Hinga Weze is the wide dissemination of knowledge and knowledge sharing skills. Informants universally cited Hinga Weze’s coordination and alignment with the Rwandan Government and other key stakeholders as one of its most impressive features, and when asked to comment on sustainability, indicated that these efforts would help to ensure continuity beyond the life of the project. Examples of high-level efforts to create lasting linkages for knowledge sharing and dissemination between the private sector, farmers groups, NGOs, and the national government have included a networking event on extension services and study tours for GOR officials. 50 One key aspect of Hinga Weze’s ability to align with and contribute to efforts to operationalize existing Rwandan strategies on agriculture and food security, and therefore build internal capacity in the process, was the incorporation of its activities into existing models and systems. This includes the early involvement of government partners, local experts, community representatives, and other stakeholders during the conceptualization of the Hinga Weze program itself and the design and implementation of individual activities. The use of community based volunteers (CBVs) as both Farmer Promoters and CHWs is at the core of many of the project’s activities and CBVs in both agriculture and health predate the beginning of Hinga Weze by several years, meaning that Hinga Weze was effectively able to “buy in” to existing structures, avoiding creating parallel activities that would then have to be handed over to authorities during close-out of the project. The use of “cascading” or Training-of-Trainer approaches to capacity-building was also an effective means of ensuring better “saturation” of training messages in both depth and breadth of service provider and beneficiary populations. This method also established and strengthened linkages between several levels of service providers, community volunteers, and the populations they served. While not specifically a Training-of-Trainers effort, Hinga Weze staff in all components of the program received key trainings in areas such as nutrition sensitive agriculture, dietary diversity and child feeding practices, and women’s empowerment, with the explicit intention of integrating these messages in all activities and providing a source of knowledge and skills transfer to government partners, CBVs, and beneficiaries. Twigire Muhinzi: The Twigire Muhinzi (“Farmer Promoters” in Kinyarwanda) extension model is a Rwandan model built around decentralizing agricultural extension to increase access to inputs and extension services at the grassroots level. Launched in 2015 under MINAGRI, the model provides training and incentives to volunteer Farmer Promoters, who then facilitate Farmer Field schools and provide other kinds of technical assistance and market access linkages to farmers through demonstration plots and other channels. The model has the advantage of being demand driven, in that communities can form their own farming groups and elect their own promoters to become part of the official extension system, which allows for greater feedback by farmers about their specific extension needs. A Gap Assessment performed by CNFA found that despite the intentions and strengths of the model, there was a need for improvements in implementation; most importantly, addressing poor management and the lack of a clear mandate for extension agents under Twigire Muhinzi. While goals were aligned with Imihigo (the Rwandan process for setting goals for service delivery), they did not provide specific enough direction on how to achieve those goals and did not reflect farmer feedback on training needs. While both RAB and Hinga Weze have included training and extension strategies in their activities to address these gaps, this was cited as an ongoing issue in discussions that the evaluation team had on irrigation and terracing, where farmers reported a lack of training on operation and maintenance of equipment for irrigation and insufficient access to inputs for terraced land. It should be noted though that irrigation and terracing activities represent a relatively small number of beneficiary households (see response to question two for further detail) and that overall, in the survey conducted for this evaluation, households cited high levels of satisfaction with the training they received. Access to training on agricultural practices was listed as the third most important service received through Hinga Weze by respondents, after savings groups and linkages to buyers. While Farmer Promoters represent a key delivery channel for these trainings, as part of the Training-of￾Trainers model used, several interviewees noted the disparity in incentives between Farmer Promoters 51 and the CHW volunteers who provide nutrition education. This was noted as a constraint not only to motivation, but to the ability to perform extension services, often leading to under-resourcing of proper inputs for demonstration plots. Recognizing this challenge, CNFA undertook a Study of Options for a Farmer Promoter Incentive Fund to address this under-compensation. Such an activity is key to sustainability—without systems in place to support Farmer Promoters beyond the life of the program, the Twigire Muhinzi system will continue to suffer from poor coordination and under-resourcing. As previously discussed, an integral part of Hinga Weze’s design was the use of existing institutions and processes to avoid duplicating efforts, but more importantly, to strengthen those institutions and processes to ensure sustainability of Hinga Weze successes. The Twigire Muhinzi model is consistent with the use of cascading or Training-of-Trainer methodologies as it rests on the idea of training higher level experts that are employees of MINAGRI and supporting the dissemination of that knowledge to a broader range of providers of extension services, ultimately reaching CBV Farmer Promoters and Farmer Field School Facilitators. These networks of service providers form much of the backbone of Components 1 and 2 as they are critical to disseminating new technologies and improved inputs, and also play a key role in organizing farmers and creating linkages between producers, agro-input dealers, and buyers, as described in the responses to other questions. Strengthening the capacity of Twigire Muhinzi experts and extension agents at all levels, must take place alongside the strengthening of cooperative capacity, to optimize the contributions of Farmer Promoters. Closer linkages with the nutrition program could also provide a mutually reinforcing mechanism for sustaining involvement and perhaps some possibility of resource sharing and capacity-building across components, for example by involving CHWs in sharing information about training opportunities under other components, creating more channels to share information about nutrition sensitive agriculture and the advantages of organizing (as seen in the next section on Solidarity Groups.) Village Savings and Credit: Sustainability of savings and credit groups depends on several factors. Beneficiaries reported involvement in grassroots level savings and credit mechanisms through several sources: Care Groups, farmer cooperatives, and the more institutionalized SACCOs. The first two appear under different names in respondents’ description of their involvement but for both Care Groups and cooperatives, they are organized as “Solidarity Groups,” intended to introduce access to and a culture of savings to beneficiaries at the grassroots level. The importance of savings and access to finance is addressed in responses to the first four evaluation questions, and it is clear that sustainability of these groups and this culture is key to ensuring that the benefits of all activities under Hinga Weze continue beyond the end of the program. A Concept Note for Improving Solidarity Groups produced for CNFA notes several of the challenges in operationalizing these groups, from motivation of members to knowledge and management capacity. There are savings groups associated with Twigire Muhinzi farmers groups but they have not necessarily been active or well managed and previous experience with savings, and credit projects is a disincentive for some to becoming more involved. The proactive stance of the Hinga Weze project, under which assessments of needs for savings groups were recognized and assessed early in the program and adaptations built into future work plans, bodes well for sustainability. It is clear from Hinga Weze performance indicators, interviews, and focus groups, as well as the MTE survey, that Hinga Weze’s approach to instilling a culture of savings through self-selection of participants into savings groups and training in business and operational practices and lending has been successful to 52 date. Savings groups are listed by 32 percent of respondents in the survey as the most important service accessed through Hinga Weze, ranking it as the service most often identified as the most important, followed by linkages to buyers, at 28 percent. Savings group participation was also cited in interviews and focus groups with Care Group members as the primary driver behind their decision to join a Care Groups. Savings Groups also increase women’s access to and control over financial resources, which increases investments in nutrition. Finally, Care Group members reported using resources accessed through Solidarity Groups to purchase health coverage, Mutuelle de Santé, community-based health insurance provided by the Rwandan Government. This level of uptake of membership in groups and adoption of a culture of savings forms a strong basis for lasting institutional change. Sustainability of savings groups can be measured along multiple axes: financial, operational, and expansion and growth. A 2019 study of sustainability of microfinance institutions in Ghana found that expansion and growth were the key determinants of sustainability. This is a positive finding for Hinga Weze, as it is clear from project reporting, the MTE survey, and KIIs and FGDs that beneficiaries are highly motivated to join savings groups. As the groups expand, uptake of financial and operational best practices will be critical to supporting these groups as they grow. While it is likely too early in the activity to observe measurable impacts of this sort of training, it is clear from project documentation and reporting that it is a high priority and is an integral part of the training and outreach provided to savings groups. Based on the observed success of those efforts to date, as reflected in the high ranking of savings groups as a benefit and the high demand to join them, it is likely that these efforts are also on track to achieve the necessary institutional changes required to support sustainability. Community Health Workers: The health sector, particularly at the grassroots level, had a higher level of organization prior to the beginning of Hinga Weze and is therefore further along the institutional capacity curve and more likely to sustain activities beyond the life of the program. It is also clear from KIIs and FGDs that CHWs are absorbing and transferring the messages passed down through the cascading model used by Hinga Weze. While causality cannot be determined with certainty, the fact that beneficiaries are reporting better understanding of nutrition and child feeding in the MTE survey, and that Hinga Weze performance indicators are showing consistent, on target improvements in all nutritional status indicators used by the project, strongly suggests that this approach has been effective. Prior to Hinga Weze, CHWs were already engaged with communities but Hinga Weze has provided resources and ongoing education and training, and by linking nutrition-sensitive agriculture with these more direct nutrition interventions, has promoted a resilient and sustainable model. The Hinga Weze model is consistent with the principles set out in USAID’s 2014–2025 Multi-Sectoral Nutrition Strategy, focusing on the appropriate age range for beneficiaries and aligning with existing country plans and the evidence supporting these kinds of multisectoral approaches to nutrition. The most critical threat to sustainability of this aspect of the program is restrictions imposed due to COVID-19. This has interrupted trainings and monitoring of the food security situation and undermines the ongoing reinforcement necessary to promote behavior change. This is particularly critical when it comes to persistent cases of chronic malnutrition. It was noted by several respondents that despite a reduction in overall rates, they see many persistent cases of malnutrition, despite interventions, and that more intensive efforts, including home visits, were deemed important to solving these more challenging cases. CHWs also reported 53 wanting additional training in child feeding and in the operation of savings groups, on which they reported receiving advising but not training. Youth Participation in the Agriculture Sector: Sustainability of all components depends on the involvement of youth in the agriculture sector, in activities ranging from extension to agri-business. Hinga Weze has undertaken several promising efforts in this regard, including internship-type opportunities in cooperatives, such as the partnership with the Rwanda Youth Agriculture Forum, a platform supported by MINAGRI for farmers and agri-business people at all levels of the value chain. Programs like these create “youth champions” in the agriculture sector who then perform an outreach function, recruiting their peers into the sector. Without continual entry of new actors into the sector, savings and credit groups have more difficulty amassing capital, which is a significant threat to their sustainability. Entry of youth into the agriculture sector also facilitates behavior change, because habits and practices around child feeding, women’s decision-making power, and numerous other key factors are easier to instill when they start at earlier levels of education. In this regard, Hinga Weze capacity-building activities for youth have included training on women’s empowerment and equitable household decision-making. Youth are also being recruited as trainers. It is difficult to observe the impacts of this training at this point because there are so far a fairly small number of youth and Hinga Weze is still actively working to recruit more, but the literature cited earlier in this section regarding the effectiveness of Training-of-Trainer models also notes the effectiveness of using peer advocates to convey messages, which suggests that Hinga Weze’s efforts to include youth as both trainers and peer advocates as well as participants in its activities are on track to engender lasting change. Hinga Weze’s capacity-building approaches are based on existing structures that pre-date Hinga Weze. Using these structures can greatly contribute towards sustainability, but the use should be based on a model of improvement, not just the status quo. In the case of the Twigire Muhinzi Extension model, Hinga Weze has supported existing extension agents, farmers groups, and farmer promoters but, according to reporting in KIIs and FGDs, has not done enough to improve the management issues that currently exist. Savings and credit or solidarity groups are a key mechanism for ensuring all of Hinga Weze’s activities are sustainable, as they can provide income for farmers to reinvest in production and access to finance for small farmers and other small businesspeople with little or no collateral. They also create a culture of savings among households and through their broad appeal, provide an access point for other important education and behavior change messaging. Hinga Weze has been able to work closely with the groups and their work has attracted new members, which is key to sustainability through growth and expansion. Hinga Weze has also built on the successes of Community Health Workers by providing additional education and training that has been passed on very effectively to community members, as was seen in the response to question 4, where Care Group members reported behavior change driven by education, demonstrations, and capacity-building around poultry husbandry. Investments in instilling a culture of savings and in the management and growth of savings groups, as well as in increased involvement of youth in the agriculture sector and in savings groups, both as participants and as peer trainers and as catalysts of behavior change around nutrition and women’s roles are key to sustainability. Hinga Weze has been successful so far in engaging youth, thus ensuring a better educated, more capable population for the future, that can continue ensuring the sustainability of these activities. 54 3. CHALLENGES AND RECOMMENDATIONS Table 6 presents the challenges identified during this evaluation along with recommendations for addressing them. They are organized by the themes in column one rather than evaluation questions because there were several themes, such as sustainability, that were shared between questions. TABLE 6. CHALLENGES AND RECOMMENDATIONS QUESTION 1 CHALLENGE RECOMMENDATION Foster strong linkages with exporters to ensure the sustainability of the irrigation sites. Expand contract farming linkages between farmer cooperatives and buyers. Contract farming allows farmers to grow higher value crops, and access input and extension services, which enables them to produce more and higher standard commodities. Contract farming provides the buyer with access to high value commodities, which is why the buyer is investing time and resources in training and supporting the farmers. Hinga Weze should expand on the contract farming linkages between well-functioning cooperatives that has the potential to grow higher value crops and buyers of high value crops. Need for longer term, more predictable business relationships between farmers and buyers. Award strategic grants to processors or buyers that foster long-term business relations with farmer cooperatives. Strategic grants, such as the grant to Kumwe Harvest or a grant for packaging facilities for irrigated horticulture, enable buyers to expand their business and, through linkages with farmers, farmers are able to access markets that were previously not available to them. As Hinga Weze is entering the final years of the project, strategic grants that foster new market linkages are recommended. Cooperatives lack bargaining power and information about prices. Strengthen the cooperatives’ capacity to collaborate around price setting and price negotiations. Many cooperatives have limited management capacity and it may not be within their reach to be price makers. However, there are cooperatives in Nyabihu that are coordinating and collaborating to become price makers. A closer assessment of the experience among the potato growing cooperatives in Nyabihu would be valuable to see if their influence on price setting can be replicated in other cooperatives. Limited private sector ownership of the agro-business clusters. Empower members to take ownership of the agro-business clusters. During the start-up phase of the agro-business clusters, Hinga Weze has been involved in its management and paid for related expenses. While the operating costs are fairly modest (primarily transportation reimbursements to meetings and lunch during the meetings), the responsibility for the costs and the organization of the cluster meetings needs to be transitioned to the members. In planning for the transition, Hinga Weze should engage with the larger buyers in the agro-business cluster to reflect on the value of the cluster and encourage them to take on more leadership and ownership. During the next agro-business clusters’ meetings, Hinga Weze should facilitate discussions about Hinga Weze transitioning the ownership and the operating costs of the clusters to the members. Different cost 55 structures are possible, including a modest membership fee where cooperatives pay less and big buyers more, but this is something that the members need to discuss and agree upon. The goal should be for Hinga Weze to stop paying for the operating costs of the agro-business clusters during year four, so that Hinga Weze can still provide some capacity-building support as needed, but the agro-business clusters can demonstrate financial and operational independence in year five. Continuity of agro-business clusters without Hinga Weze support. Transition the ownership of the agro-business clusters to the districts or private sector actors. Hinga Weze is deeply involved in the organization of the agro-business clusters. It is time to scale back Hinga Weze’s direct involvement and transition the leadership and ownership of the agro-business clusters to the districts or private sector actors to ensure that the agro-business cluster activities continue beyond the life of the project. QUESTION 2 Linking irrigation scheme farmers to markets. Ensure that all irrigation sites are connected to a contract farming arrangement. Contract farming is particularly suitable for the irrigation sites, which can produce high value crops for the export market. Contract farming provides farmers a reliable buyer which offers access to quality input and extension services. Access to a reliable water source, input and intensive extension services allows the farmers to produce more at a higher standard. Contract farming provides the buyer with access to high value commodities, which is why the buyer is investing time and resources in training and supporting the farmers. Once Hinga Weze has addressed the equipment failures and access to water challenges at the site, Hinga Weze should ensure that all irrigation sites are connected to one or more exporters in a contract farming arrangement. In particular, once the technical challenges at Ndego irrigation site are resolved, the cooperative needs to be linked to an exporter. Malfunctioning and poorly maintained equipment. Conduct a thorough investigation of why the pumps are not fully charging the dams. At five of the seven currently operating irrigation sites, there is a shortage of water. Multiple technical issues need to be investigated promptly and systematically. The following issues need to be carefully investigated; Why are the pumps not able to fully charge the dams? Can batteries be added to allow the pumps to pump water when there is no sun? Can filters or sensors be added so that debris from the source of the water does not enter and break the water pump? Why are the pipes breaking? Following the assessment, a swift action plan to address the issues needs to be put into place. It is of great importance that the technical issues are resolved with speed, so that there is time for the irrigation cooperatives to start production and get linked to buyers before Hinga Weze is coming to an end. Additional training for irrigation cooperative members on how to use and maintain the irrigation equipment. Multiple irrigation sites are experiencing recurring breaks and failures of the irrigation equipment. Some of the breakdowns, notably the breaks and leakages of pipes, could be prevented with additional training on how to properly use the pipes. The service providers should, as part of the installation agreement, provide the cooperatives with training, but this training is clearly not adequate as the equipment keeps on breaking after just a few months of use. Consequently, a closer review of how the training is carried out and what specific training each of the cooperatives needs is needed. Moreover, the cooperatives need training on how to do some more basic maintenance of the irrigation system themselves. Currently, the farmers have very rudimentary understanding of how the equipment works and how to maintain it. They would also benefit from assistance in calculating how much money is needed for maintenance and repair expenses to better plan their saving strategies. 56 Train local auto mechanics how to repair irrigation equipment. Collaborate with RAB and build on previous experience of vocational skills training in irrigation equipment repair and maintenance. The availability of local expertise on how to repair irrigation equipment would create easy and affordable access to maintenance services in close proximity to the irrigation sites. Farmer ownership and management of irrigation schemes. Provide the irrigation cooperatives with targeted capacity-building support to manage the Irrigation Management Transfer Agreement. Irrigation Management Transfer Agreements (IMTA) will be signed between the MINAGRI/RAB, the District, and irrigation cooperatives. The IMTA outlines these three actors’ responsibilities for the operation, management, and maintenance of the irrigation system. The cooperative will have a host of responsibilities under the IMTA, such as developing annual work plans, budgets, water distribution plans and collecting water user fees that the cooperatives will need additional capacity-building and coaching to be able to execute and take ownership. The implementer should also clarify the status of post-activity infrastructure ownership. Resources spread across many activities. Consider reducing the number of activities and focusing on scaling up/sustainability. Identify activities that have been fruitful, several of which are noted in this evaluation. Consider whether there are activities that are still not showing returns or that have been slow to get started and carefully consider the costs and benefits of continuing them. Lack of inputs needed for effective cultivation and sustainability of terraced land. Develop a comprehensive strategy to ensure that farmers on the terracing sites can access organic manure and lime. There is a shortage of organic manure and it is expensive for farmers to purchase organic manure and lime. A rapid review of practical interventions that can be carried out in a short timeframe should be conducted. The review should consider a variety of interventions to address the issue from multiple angles, including composting, the rearing of household farm animals, planting of biomass, etc. The activities should include the active participation of the farmers on the terracing sites. Assess whether strategic grants can be used to entice the private sector in producing biomass for organic manure. There is a shortage of biomass, which has created a bottleneck for the production of organic manure. A grant to a private sector actor to invest in the planting and growing of biomass, which upon harvest can be blended for organic manure or compost, should be considered. QUESTION 3 Lack of detailed information about the costs and benefits of Hinga Weze agricultural productivity interventions. Factor in the information provided in the CBA as part of this evaluation into work planning. It is difficult to make broad programmatic recommendations based on CBA results because those results are sensitive to many factors. It is worth noting however that planting/cultural practices was the only category of technology to show negative returns. The recommendation for CBA results is that planners consider the factors presented in the analysis when estimating whether returns for an activity will approximate those presented in the analysis. The highest impact intervention analyzed for maize was genetics. The bundle of soil conservation, pesticides, and genetics also had a largely positive impact, followed by soil conservation, and pesticides. Technologies could only be disaggregated for maize due to sample size, but the bundle of technologies also showed positive impacts for Irish potato and OFSP. Selection of a particular value chain for a given area would need to consider factors such as relative costs of labor and other inputs. QUESTION 4 57 Persistent, chronic malnutrition. Provide additional training and support to CHWs to address persistent, chronic malnutrition that is the product of poor access to and utilization of food resources. Trainings would focus on the reasons that malnutrition can persist at the household level and how to address that—linking people to resources, providing nutrition education/child-feeding education; involvement of male champions or more intensive focus on male household members if it is a resource issue. Support intensive monitoring and home visits by CHWs to ensure no ground is lost as a result of COVID-19. Lack of investment of household resources in nutrition. Continue to identify and support “male champions” and encourage them to take a more proactive role in spreading information about nutrition and feeding practices. These could be drawn from male participants in Care Groups, prominent community members, or men who have been exposed to these ideas through the other components, like nutrition sensitive agriculture, and show an interest and aptitude. As with the youth engagement, they would play a role in outreach to men and in normalizing a male role in nutrition, child-feeding, and other household labor. Disruption to nutrition education and food security monitoring as a result of COVID-19. Provide PPE (if possible or encourage MOH to do so) and training on disease transmission to CHWs. Provide PPE and ongoing engagement on current COVID-19 protocols and information. This should be a two-way street, allowing CHWs the opportunity to discuss their needs and any challenges they are facing in maintaining their activities so that solutions can be found early. A major threat to nutrition and the sustainability of benefits from Hinga Weze is the disruption to nutrition activities during the pandemic. Provide training and equipment to safely maintain ongoing contact between health workers and their communities. Lack of women’s earning ability and control over resources. Continue to Prioritize women’s access to credit, inputs, and capacity-building. Increasing women’s involvement in the agriculture sector and control over financial resources contributes to overall economic growth and improvements in gender equality and household nutrition. Safeguard resources and benefits that have accrued to women under Hinga Weze activities. Women lack decision-making power to increase household investment in nutrition. Continue to mainstream women’s empowerment messages and nutrition education throughout the program. Make use of all points of contact through the program, under all components, to support these priorities. QUESTION 5 Management Gaps in Twigire Mihinzi. Continue to address gaps addressed by Twigire Mihinzi Gap assessment. In particular, in addition to building technical skills, include management training for extension agents and others engaged in management of Twigire Muhinzi. Farmer Promoters are resource constrained. Invest in addressing under-compensation of Farmer Promoters. Use the remaining years of the program to help establish a more organized system to support farmer promoters, including addressing the disparities in incentives between the agriculture and health sectors. 58 Lack of Youth involvement in the agriculture sector and savings and credit mechanisms. Increase investment in promoting youth involvement through existing partnerships. Interventions under this activity focusing on youth have been well-received and shown promise, particularly regarding sustainability of program benefits. Consider increasing investment in these ongoing activities to involve more youth. Lack of inputs needed for effective cultivation and sustainability of terraced land. Train the savings groups on how much they need to save for lime and organic manure. Few cooperatives on the terracing sites are planning ahead and saving for the purchase of lime and organic manure every other year. As those costs are significant, the cooperatives need support to budget for how much funds to save to purchase sufficient lime and organic manure. Table 6 Source: Authors 59 ANNEX A: SURVEY IMPLEMENTATION REPORT Mid-Term Performance Evaluation of Feed the F t re sR anda Hinga Weze Activity Field Report Prepared for Integra Government ServicesInternational LLC October 2020 60 61 Mid-Term Performance Eva 1 uation of Feed the Future s Rwanda Hinga Weze Activity | October 2020 LIST OF ACRONYMS COVID-19 Coronavirus Disease of 2019 DR Dalberg Research FGD Focus Group Discussion LLC Limited Liability Company NISR National Institute of Statistics of Rwanda USAID United States Agency for International Development 62 63 64 65 66 67 68 69 70 71 72 ANNEX B: COMPLETE COST-BENEFIT ANALYSIS The following section describes the methodology and results of a cost-benefit analysis (CBA) select interventions under the Hinga Weze project. The goal of this analysis is to identify the impact of various technologies and agriculture practices on farm income. CBA compares the costs and benefits of an investment with the costs and benefits of a status-quo situation where an individual or organization does not invest. The LEAP III evaluation team has developed a series of CBA models to estimate the net impact of select interventions under the Hinga Weze project. The models use performance data from the first two years of the project, as well as survey data from the midterm evaluation survey, to evaluate progress to date and to forecast the returns over the next several years. In total, the CBA time horizon estimates the costs and benefits to USAID beneficiaries over a 10-year time horizon, starting with the point when the USAID beneficiary receives assistance. Hinga Weze supports Rwandan farmers by increasing production and improving access through over 14 different interventions. After consulting with USAID/Rwanda and CNFA, the LEAP III evaluation team has narrowed down the scope of this evaluation to include a select number of key crops including maize, Irish potato, and orange flesh sweet potato (OFSP). These crops were selected because they cover a vast majority of Hinga Weze’s beneficiaries. Although there was not enough data to develop CBA models for select interventions for Irish potato and OFSP farmers, the team has developed general CBA models covering all interventions for these crops. For Hinga Weze maize farmers, CBA models have been developed for interventions where there was enough data to produce meaningful results. These interventions include the following: ● Crop Genetics: to include improved or certified seeds that could be higher-yielding and/or more resilient to climate impacts. ● Cultural Practices: To include agriculture management techniques such as seedling production and transplantation cultivation practices ● Pest Management: to include Integrated Pest Management, improved insecticides and pesticides, and improved and environmentally sustainable use of insecticides and pesticides. ● Soil-related Fertility and Conservation: To include soil management practices that increase biotic activity and soil organic matter levels, such as soil amendments that increase fertilizer-use efficiency ● Genetics, Pest Management, and Soil Conservation: Represents farmers receiving all three interventions. ● All Remaining Interventions: Includes all remaining interventions that were not assessed due to an absence of data or the fact that the intervention was provided in various combinations with other interventions. 73 METHODOLOGY The cost-benefit model reflects the core logic of project alternatives by comparing the incremental results of USAID Hinga Weze interventions with the “business-as-usual” or “without intervention” scenario, over a 10-year time horizon. For this model, the “business-as-usual” is defined as farmers in the targeted regions of Rwanda who would not receive U.S. Government or other technical assistance that would substantially alter current agricultural activities over the 10-year time horizon. As previously mentioned, this analysis looks at the impact of Hinga Weze on farmers growing three crops: Maize, Irish Potato, and OFSP. The table below identifies the benefits and costs considered and the beneficiaries. TABLE 1: Benefits, Costs, and Stakeholders Impacts Farmers USAID Rwanda B1 - Increased Productivity ✔ ✔ B2 - Reduced losses due to Increased Climate Change Resilience ✔ ✔ B3 - Reduced Carbon Emissions due to Improved Inputs ✔ C1 - Physical Inputs ✔ ✔ C2 - Hired Inputs ✔ ✔ C3 - Family Labor Inputs ✔ ✔ C4 - Loan Interest Inputs ✔ ✔ C5 - Cost of Project ✔ 74 BENEFITS B1 - INCREASED PRODUCTIVITY The main benefit from adopting new technologies and improved farming practices is increased productivity. Table 2 provides the formula used to make this calculation, including variables such as land size, yields and yield growth, and crop prices.2 The CBA study quantifies the increased productivity accruing to Hinga Weze farmers compared to the status quo of farmers receiving no intervention. According to the project’s Baseline Report, maize farmers had an average baseline yield of 1,028 kgs per ha, compared to 8,032 kg per ha and 5,154 kg per ha for Irish potato and OFSP, respectively (2018). Table 2: Calculation of Benefit 1 Inputs Unit 𝐻𝐶𝑟𝑜𝑝 Land Area (Hectares per farmer) ha 𝑌𝐶𝑟𝑜𝑝 𝐵𝑎𝑠𝑒𝑙𝑖𝑛𝑒 Baseline Yield per Hectare kg 𝐺𝐶𝑟𝑜𝑝 𝑊𝑖𝑡ℎ𝑜𝑢𝑡 Annual growth rate of crop yields without intervention % 𝐺𝐶𝑟𝑜𝑝,𝐺𝐴𝑃 𝑊𝑖𝑡ℎ Annual growth rate of crop yields with intervention % 𝐶𝐶𝑟𝑜𝑝 𝐶𝑜𝑛𝑠𝑢𝑚𝑒𝑑 Portion Consumed by Household % 𝑆𝐶𝑟𝑜𝑝 Crop Price Sensitivity factor % 𝑃𝐶𝑟𝑜𝑝 Farmgate Price per kg RWF Calculation Benefit: 𝐵1𝑡 𝐹𝑎𝑟𝑚𝑒𝑟𝑠 = 𝐻𝐶𝑟𝑜𝑝 × 𝑌𝐶𝑟𝑜𝑝 𝐵𝑎𝑠𝑒𝑙𝑖𝑛𝑒 × [(1 + 𝐺𝐶𝑟𝑜𝑝 𝑊𝑖𝑡ℎ ) 𝑡 − (1 + 𝐺𝐶𝑟𝑜𝑝 𝑊𝑖𝑡ℎ𝑜𝑢𝑡 ) 𝑡 ] × (1 − 𝐶𝐶𝑟𝑜𝑝 𝐶𝑜𝑛𝑠𝑢𝑚𝑒𝑑) × 𝑃𝐶𝑟𝑜𝑝 × 𝑆𝐶𝑟𝑜𝑝 Figure 1 shows the estimated yields and incremental revenues for maize farmers two years after the start of the project. Maize producers receiving the genetics intervention experienced the highest increase in yields at 2,449 kg per ha and an incremental value of 329,672 RWF per HA ($346 per HA). The estimated growth in yields and revenues for the other interventions 2 The CBA considers the yields to be inclusive of post-harvest losses. 75 includes the following: soil conservation, pesticide, and genetics intervention: 1,942 kg per ha and 212,048 RWF ($222) per ha; pesticides: 1,482 kg per ha and 105,256 ($110) per ha; average across all other interventions: 1,390 kg per ha and 83,984 RWF ($88); soil conservation: 1,197 kg per ha and 39,276 RWF ($41); and planting: 1,167 kg per ha and 32,248 RWF ($34).3 Irish Potato farmers receiving assistance from Hinga Weze were estimated to increase their yields from 8037 to 8,771 kg per hectare from the baseline to the second year of the project. Producers of OFSP increased their yields from 5,154 kg per ha from baseline to an estimated value of 5,756 kg per ha by year 3. Table 3: Incremental Production and Revenues for 1 Hectare of Land Crop Intervention Baseline Year 2 Yield (incremental) Year 2 Yield (total) Incremental Revenues (RWF) Maize Planting / Cultural Practices 1,028 143 1,171 33,176 Soil Conservation 1,028 169 1,197 39,276 All Other Interventions 1,028 392 1,420 90,944 Pesticides 1,028 502 1,530 116,464 Soil Conservation / Pesticides / Genetics 1,028 176 1,204 40,832 Genetics 1,028 1,410 2,438 327,121 Irish Potato All Interventions 8,032 739 8,771 84,384 OFSP All Interventions 5,154 455 5,609 5,757 Most farmers consume a portion of their own crop production. This is treated as a cash inflow valued at the price the farmer would have received had they sold the crop along with the rest of their product. The values used to estimate the portion of crops consumed by the household are based on the National Institute of Statistics 2017 Agricultural Household Survey to include 34 3 The CBA uses an average farmgate price of 232 Rwandan Franc (RWF) per kg of maize and an exchange rate of 955 RWF/USD. 76 percent of maize production and 15.3 percent and 44 percent of Irish potato and OFSP production, respectively. With the exception of the maize interventions associated with introducing crop genetics, the CBA model for maize assumes that yield growth resulting from the Hinga Weze intervention will increase for an average of 4 years after the farmer starts to receive the intervention and then will revert back to an average empirical growth of between 1.5 and 2 percent per year. Figure 1 provides a visual representation of the yield growth assumptions for the maize interventions. These yield growth assumptions are based on actual yields reported in Hinga Weze’s baseline and progress surveys. Similarly, the Irish potato model uses an average annual yield growth rate of 4.5 percent compared to a growth rate of 4.3 percent per year for OFSP. 77 Figure 1: Incremental Production and Revenues for 1 Hectare The price point for maize, Irish potato, and OFSP are based on average prices reported in Hinga Weze’s 2018 and 2019 progress surveys. For maize, the average price is 232 RWF per kg, compared to 170 RWF per kg for Irish potato and 192 RWF per kg for OFSP. Similarly, farm land size is based on the weighted average land use across each of the three seasons. Specifically, the CBA model uses an average land size of 0.23 hectares per year for maize. The estimated average farm size for a household producing Irish potato is 0.25 hectares per year. Lastly, producers dedicate, on average, 0.07 hectares per year to OFSP. 78 B2 - REDUCED LOSSES DUE TO INCREASED CLIMATE CHANGE RESILIENCE Another potential benefit that may accrue to farmers adopting new technologies and improved farming practices that are intended to combat on-farm losses associated with environmental factors and farming techniques. The variables used to calculate this benefit are similar to the first benefit (B1). The additional factors that are needed to make this calculation are the annual estimated losses due to climate change, pests, and disease. In addition, there are the additional project loss reductions that are needed to estimate the potential reduction in on-farm losses due to the Hinga Weze project. Table 4: Calculation of Benefit 2 Inputs Unit 𝐻𝐶𝑟𝑜𝑝 Land Area (Hectares per farmer) ha 𝑌𝐶𝑟𝑜𝑝 𝐵𝑎𝑠𝑒𝑙𝑖𝑛𝑒 Baseline Yield per Hectare kg 𝐺𝐶𝑟𝑜𝑝 𝑊𝑖𝑡ℎ𝑜𝑢𝑡 Annual growth rate of crop yields without intervention % 𝐺𝐶𝑟𝑜𝑝,𝐺𝐴𝑃 𝑊𝑖𝑡ℎ Annual growth rate of crop yields with intervention % 𝐿𝐶𝑟𝑜𝑝 Annual losses due to Climate Change and Pests % 𝑅𝐶𝑟𝑜𝑝 Reduced Losses from Natural Disasters and Climate Change % 𝑆𝐶𝑟𝑜𝑝 Crop Price Sensitivity Factor % 𝑃𝐶𝑟𝑜𝑝 Farmgate Price per kg RWF Calculation Benefit: 𝐵2𝑡 𝐹𝑎𝑟𝑚𝑒𝑟𝑠 = 𝑌𝐶𝑟𝑜𝑝 𝐵𝑎𝑠𝑒𝑙𝑖𝑛𝑒 1 − 𝐿𝐶𝑟𝑜𝑝 × 𝐻𝐶𝑟𝑜𝑝 × (1 + 𝐺𝐶𝑟𝑜𝑝 𝑊𝑖𝑡ℎ ) 𝑡 × 𝑅𝐶𝑟𝑜𝑝 × (1 + 𝑆𝐶𝑟𝑜𝑝) × 𝑃𝐶𝑟𝑜𝑝 According to the 2020 midterm evaluation survey, over 70 percent of Hinga Weze farmers experience on-farm losses, which are mainly attributed to pests, droughts, floods, and crop diseases. It is estimated these losses amount to approximately 35 percent of the true yield potential (CNFA 2018). To address these constraints, Hinga Weze has promoted the use of crop technologies and farming practices that can potentially reduce on-farm losses. There are, however, several specifications that farmers need to follow to reap the gains of adopting sustainable agriculture practices. Moreover, the regenerative benefits of improving soil structure and strengthening crop resilience often take several seasons to materialize. For this analysis, the 79 cba is assuming a two percent reduction in on-farm losses that will start after the second year of participating in the Hinga Weze project (figure 2). These savings are a percentage of producers’ overall production potential, which increases with time. The evaluation team considers this a conservative estimate, as 57 percent of Hinga Weze farmers said the program’s interventions were helping them adapt to climate change and extreme weather events. This assumption is tested in the sensitivity analysis section to show how movements in this variable impact the overall results. Figure 2: Reduced Climate Change, Disease, and Pest-related Losses B3 - REDUCED CARBON EMISSIONS DUE TO IMPROVED INPUT USE The use of non-organic fertilizers results in carbon emissions that can be reduced with improved farming practices, such as the use of organic manure and compost materials as is promoted. The Hinga Weze Program promotes practices that reduce carbon emissions including improved inputs and practices. These practices have great potential benefits for the environment in Rwanda, due to the data required to quantify these benefits our analysis focuses on the impacts from changed use of fertilizers. The inputs used to make this calculation are summarized in Table 6. Table 6: Calculation of Benefit 4 Inputs Unit 80 𝐻𝐶𝑟𝑜𝑝 Land Area (Hectares per farmer) ha 𝐶𝑂2 𝑈𝑟𝑒𝑎 CO2 per kg of Urea fertilizer kg 𝐶𝑂2 𝐷𝐴𝑃 CO2 per kg of DAP fertilizer kg 𝑈𝐶𝑟𝑜𝑝 Kg of Urea used at Baseline kg 𝐷𝐶𝑟𝑜𝑝 Kg of DAP used at Baseline kg 𝑈𝐶𝑟𝑜𝑝,𝐺𝐴𝑃 𝑊𝑖𝑡ℎ Kg of Urea used with intervention kg 𝐷𝐶𝑟𝑜𝑝,𝐺𝐴𝑃 𝑊𝑖𝑡ℎ Kg of DAP used with intervention kg 𝑆𝐶𝐶 𝑅𝑤𝑎𝑛𝑑𝑎 Country level Social Cost of Carbon USD 𝐸 Real Exchange Rate RWF Calculation Benefit: 𝐵4𝑡 𝑅𝑤𝑎𝑛𝑑𝑎 = 𝐻𝐶𝑟𝑜𝑝 [𝐶𝑂2 𝑈𝑟𝑒𝑎(𝑈𝐶𝑟𝑜𝑝,𝐺𝐴𝑃 𝑊𝑖𝑡ℎ − 𝑈𝐶𝑟𝑜𝑝 ) + 𝐶𝑂2 𝐷𝐴𝑃(𝐷𝐶𝑟𝑜𝑝,𝐺𝐴𝑃 𝑊𝑡𝑖ℎ − 𝐷𝐶𝑟𝑜𝑝)] × 𝑆𝐶𝐶 𝑅𝑤𝑎𝑛𝑑𝑎 × 𝐸 The impact of inputs such as DAP and urea were quantified by measuring the change in their use through the baseline and progress surveys. The amount of carbon from a kg of fertilizer was estimated from Lal (2004). After estimating the change in carbon emissions due to the change in input use, the team valued these emissions using the country level social cost of carbon from Ricke et al. 2019. Similar to the loan benefit, the benefits accruing from reduced emissions accounts for a small portion of the overall benefit. For this reason, this benefit flow can be referenced in the net income section of this report. COSTS C1 - PHYSICAL INPUTS This cost includes the cost to the farmers of adopting the new technologies and agriculture practices. It specifically refers to the use of physical inputs used to produce the focus crops under question. The key variables used to make these estimates include the additional quantities associated with production, such as improved seeds, fertilizers, and lime, and the prices for these inputs. The calculations used for the CBA study were estimated using the weighted average use of the input, in kg per hectare, for the 2018 progress survey relative to the baseline. The inputs were then valued using the average price from the 2018 progress survey. A summary of the inputs and calculations used for the CBA models is provided in Table 7. 81 Table 7: Calculation of Cost 1 Inputs Unit 𝐻𝐶𝑟𝑜𝑝 Land Area (Hectares per farmer) ha 𝐼𝑆 Input Price Sensitivity Factor kg 𝑆𝑒𝑒𝑑𝐶𝑟𝑜𝑝 𝐷𝑖𝑓𝑓 Change in Seed Use with Intervention kg 𝐿𝑖𝑚𝑒𝐶𝑟𝑜𝑝, 𝐶ℎ𝑎𝑛𝑔𝑒 Change in Lime Use with Intervention kg 𝑈𝐶𝑟𝑜𝑝 𝐷𝑖𝑓𝑓 Change in Urea Use with Intervention kg 𝑁𝑃𝐾𝐶𝑟𝑜𝑝 𝐷𝑖𝑓𝑓 Change in NPK Use with Intervention kg 𝐷𝐶𝑟𝑜𝑝 𝐷𝑖𝑓𝑓 Change in DAP Use with Intervention kg 𝑂𝑀𝐶𝑟𝑜𝑝 𝐷𝑖𝑓𝑓 Change in Organic Manure Use with Intervention kg 𝑆𝑒𝑒𝑑𝐶𝑟𝑜𝑝 𝑃 Price of Seeds RWF 𝐿𝑖𝑚𝑒𝐶𝑟𝑜𝑝 𝑃 Price of Lime RWF 𝑈𝐶𝑟𝑜𝑝 𝑃 Price of Urea RWF 𝑁𝑃𝐾𝐶𝑟𝑜𝑝 𝑃𝑓 Price of NPK RWF 𝐷𝐶𝑟𝑜𝑝 𝑃 Price of DAP RWF 𝑂𝑀𝐶𝑟𝑜𝑝 𝑃 Price of Organic Manure RWF Calculation Benefit: 𝐶1𝑡 𝐹𝑎𝑟𝑚𝑒𝑟𝑠 = 𝐻𝐶𝑟𝑜𝑝 × 𝐼𝑆 [𝑆𝑒𝑒𝑑𝐶𝑟𝑜𝑝 𝐷𝑖𝑓𝑓 × 𝑆𝑒𝑒𝑑𝐶𝑟𝑜𝑝 𝑃 + 𝐿𝑖𝑚𝑒𝐶𝑟𝑜𝑝, 𝐷𝑖𝑓𝑓 × 𝐿𝑖𝑚𝑒𝐶𝑟𝑜𝑝 𝑃 + 𝑈𝐶𝑟𝑜𝑝 𝐷𝑖𝑓𝑓 × 𝑈𝐶𝑟𝑜𝑝 𝑃 + 𝑁𝑃𝐾𝐶𝑟𝑜𝑝 𝐷𝑖𝑓𝑓 × 𝑁𝑃𝐾𝐶𝑟𝑜𝑝 𝑃𝑓 ] + 𝐻𝐶𝑟𝑜𝑝 × 𝐼𝑆 [𝐷𝐶𝑟𝑜𝑝 𝐷𝑖𝑓𝑓 × 𝐷𝐶𝑟𝑜𝑝 𝑃 + 𝑂𝑀𝐶𝑟𝑜𝑝 𝐷𝑖𝑓𝑓 × 𝑂𝑀𝐶𝑟𝑜𝑝 𝑃 ] Table 8 provides a summary of the incremental quantities for the different physical input types. With the exception of planting / cultural practices and OFSP, seed quantities increase by all intervention types and crops. There are some instances where the use of physical inputs has 82 declined relative to the baseline. There are some common reductions in NPK and organic fertilizer use, although these reductions vary by intervention and crop. Table 8: Physical Inputs (kg) per Hectare by Crop and Intervention Intervention → Physical Input ↓ Soil Conserva tion Genetics Planting / Cultural Practices Pesticide s Soil Conserva tion / Pesticide s / Genetics All Other Interventi ons Irish Potato OFSP Seeds 59 17 -1 33 4 8 28 -32 Lime 24 95 28 54 0 149 Urea -19 25 19 5 30 10 27 5 NPK -4 -4 2 -4 2 -1 4 18 DAP -20 40 -1 40 5 17 37 2 Organic Fertilizer -483 1,965 351 1,769 2,188 749 3,491 -4,683 The quantities listed above have been multiplied by the market prices reported in the baseline and progress surveys. A summary of the overall costs per hectare for the different maize interventions is provided in Figure 3. With the exception of the soil conservation intervention, most maize producers experience an increase in the overall use and costs for physical inputs. The main increases in physical inputs for OFSP producers are seeds (181,714 RWF per ha) and lime (25,151 RWF per ha). The primary cost drivers for Irish potatoes is lime (20,256 RWF per ha) and NPK (18,368 RWF per ha). 83 Figure 3: Physical Input Costs by Maize Intervention C2 - HIRED LABOR INPUTS Due to adopting various good agricultural practices, many farmers have adjusted labor practices. Some practices have increased the amount of hired labor required for each farmer. Increasing hired labor decreases the net income for each farmer. There was high variability in hired labor costs across crops and interventions increased. In fact, it was not uncommon in the data to see decreased cost of labor for some steps in the production process. The cost of increased hired labor was calculated using the baseline and 2018 progress data. One difference in the measurement of labor costs across years is that at baseline farmers were asked to report the number of workers, days worked, and the wage rate while in 2018 farmers were only asked for the total cost. The evaluation team compared the two calculations by comparing the total cost per hectare. However, the two different approaches for collecting the data could explain the large variability between two data sources. This could explain why labor costs, relative to the baseline, decline in some instances when we probably expect them to rise. To perform the hired labor calculations, the team used the inputs and formulas outlined in Table 9. Table 9: Calculation of Cost 2 Inputs Unit 𝐻𝐶𝑟𝑜𝑝 Land Area (Hectares per farmer) ha 84 𝑊𝑆 Wage Rate Sensitivity % 𝑊𝐺 Wage Rate Growth Index % 𝐺𝐿𝐶𝑟𝑜𝑝 Change in Grading Labor Costs per hectare RWF 𝐶𝐿𝐶𝑟𝑜𝑝 Change in Cleaning Labor Costs per hectare RWF 𝐻𝐿𝐶𝑟𝑜𝑝 Change in Harvest Labor Costs per hectare RWF 𝑊𝐿𝐶𝑟𝑜𝑝 Change in Weeding Labor Costs per hectare RWF 𝑃𝐿𝐶𝑟𝑜𝑝 Change in Pest Control Labor Costs per hectare RWF 𝐹𝐿𝐶𝑟𝑜𝑝 Change in Fertilizing Labor Costs per hectare RWF 𝐿𝑃𝐶𝑟𝑜𝑝 Change in Land Preparation Labor Costs per hectare RWF 𝑆𝐿𝐶𝑟𝑜𝑝 Change in Seeding Labor Costs per hectare RWF Calculation Benefit: 𝐶2𝑡 𝐹𝑎𝑟𝑚𝑒𝑟𝑠 = 𝐻𝐶𝑟𝑜𝑝 × 𝑊𝑆 × 𝑊𝐺 [ 𝐺𝐿 + 𝐶𝐿 + 𝐻𝐿 + 𝑊𝐿 + 𝑃𝐿 + 𝐹𝐿 + 𝐿𝑃 + 𝑆𝐿 ] Table 10 provides a summary of the hired labor costs for Irish potato and OFSP. Producers of Irish potatoes experience a net increase in hired labor costs of 49,605 RWF per ha. Almost half of this value is associated with increased hired labor costs tied to land preparation. On the other hand, OFSP producers experience a net reduction of 38,050 RWF per ha. These cost reductions are primarily due to reduced labor hired labor demand for land preparation, seeding, and weeding. Table 10: Hired Labor Costs per HA for Irish Potato and OFSP Production Cost Irish Potato OFSP Grading 809 0 Cleaning 96 0 Harvesting 5,251 -2,709 85 Weeding 2,703 -14,294 Pesticides 2,703 1,718 Fertilizers 5,191 6,921 Land preparation 24,294 -14,808 Seedling 8,558 -14,878 Total 49,605 -38,050 Figure 4 provides a summary of the hired labor costs by production activity for each of the interventions. Hired labor costs for weeding, land preparation, and seedling costs decline for half or over half of the interventions. As previously mentioned, one reason why these costs may be lower is the different methods that were used to capture this data. There are also substantial increases in hired labor costs for planting / cultural practices. These large cost increases have a significant impact on farm incomes described later in this section. Figure 4: Hired Labor Costs by Production Activity 86 C3 - FAMILY LABOR INPUTS Due to adopting various good agricultural practices, many farmers are expected to adjust labor practices. Some practices are expected to increase the amount of labor for each farmer including hired and family labor. This cost looks at the change in family labor days. The change in these days is valued at 70 percent of the expected increase in the number of hired labor days for maize, 58 percent of any additions to the number of hired labor days for Irish Potato, and 24 percent of any increase in hired labor days for OFSP. These assumptions are based on findings from the midterm evaluation survey showing that the average household uses an average of 90 family labor days in one season. Family labor costs account for a small portion of the overall costs. For this reason, this cost outflow can be referenced in the net income section of this report. Table 11: Calculation of Family Labor Inputs Inputs Unit 𝐻𝐶𝑟𝑜𝑝 Land Area (Hectares per farmer) ha 𝑊𝑆 Wage Rate Sensitivity % 𝑊𝐺 Wage Rate Growth Index % 𝐹𝑎𝑚𝐶𝑟𝑜𝑝 𝐿𝑎𝑏𝑜𝑟 Change in Family Labor with Intervention Days 𝑇 𝐷𝑎𝑦𝑠 Hinga Weze Training Days Days 𝐹𝑎𝑚𝑊𝑎𝑔𝑒 Family Wage Rate RWF Calculation Benefit: 𝐶3𝑡 𝐹𝑎𝑟𝑚𝑒𝑟𝑠 = 𝐻𝐶𝑟𝑜𝑝 × 𝑊𝑆 × 𝑊𝐺 × 𝐹𝑎𝑚𝑊𝑎𝑔𝑒 × (𝐹𝑎𝑚𝐶𝑟𝑜𝑝 𝐿𝑎𝑏𝑜𝑟 + 𝑇 𝐷𝑎𝑦𝑠) 87 C4 - LOAN INTEREST INPUTS Based on findings from the 2020 midterm evaluation survey, it is estimated that 73 percent of households access some form of credit and the average loan amount is $60 per year. The cba model uses the average national interest rate of 17 percent, and a loan time horizon of 6 months, to calculate the average interest rate paid by an Hinga Weze farm household. Based on these assumptions, the average Hinga Weze household pays a little over 400 rwf/ha. Table 10: Calculation of Cost 4 Inputs Unit 𝐴𝑚𝑛𝑡 Average Loan Amount RWF 𝐿𝐴𝐶𝑟𝑜𝑝 𝑊𝑖𝑡ℎ𝑜𝑢𝑡 Portion of Farmers with Loan Access without Intervention % 𝐿𝐴𝐶𝑟𝑜𝑝 𝑊𝑖𝑡ℎ Portion of Farmers with Loan Access with Intervention % 𝑟 Interest Rate % 𝑃𝑃 Payback Period Months Calculation Benefit: 𝐶4𝑡 𝐹𝑎𝑟𝑚𝑒𝑟𝑠 = 𝑟 12 × 𝑃𝑃 × 𝐴𝑚𝑛𝑡 × (𝐿𝐴𝐶𝑟𝑜𝑝 𝑊𝑖𝑡ℎ − 𝐿𝐴𝐶𝑟𝑜𝑝 𝑊𝑖𝑡ℎ𝑜𝑢𝑡 ) 88 SENSITIVITY ANALYSIS To build cost-benefit models, it is necessary to make assumptions and projections about key variables that could change in the future. The uncertainty inherent in those assumptions impacts the level of validity attributed to the result, which is why it is essential to analyze the sensitivity of the model to those assumptions. This is done using one-way and two-way tables that show how the result changes with modifications to the values of certain parameters, everything else being held constant. Table 11 provides a summary of the key variables tested in the sensitivity analysis. TABLE 11: Sensitivity Analysis Impacts Gain in Expected Crop Yield Investment Cost Over￾run Input Requirement s Labor Cost Sensitivity Range +/- 20% deviation 0 - 30% +/- 40% deviation +/- 40% deviation B1 - Increased Productivity ✔ B2 - Reduced losses due to Increased Climate Change Resilience ✔ B3 - Reduced Carbon Emissions due to Improved Inputs C1 - Physical Inputs ✔ C2 - Hired Inputs ✔ C3 - Family Labor Inputs ✔ C4 - Loan Interest Inputs C5 - Cost of Project ✔ 89 90 RESULTS Figure 5 shows the net present value (NPV) of benefits received by maize farmers due to the Hinga Weze project.4 The main benefit is increased yields and the intervention showing the greatest benefits is genetics 3.4 million RWF ($), followed by the combination of interventions to include soil conservation, pesticides, and genetics is 2,6 million RWF per ha. The interventions with the lowest NPV of benefits are the planting intervention (354,719 RWF per ha) and the soil conservation intervention (442,898 RWF per ha.) The average farmer under all Hinga Weze interventions has an average NPV of 1,129,986 RWF per ha in terms of benefits, compared to 780,371 RWF per ha for OFSP producers. Figure 5: Maize Benefits NPV Summary by Intervention (per ha) Figure 6 below shows the costs associated with each intervention. While we saw that soil conservation had lower increases in yields, it also had much lower increases in costs at an NPV of 61,937 RWF per ha. Planting and cultural practices, which also did not demonstrate a very large increase in yields, had much higher costs at an NPV of 522,630 RWF per ha. Similarly, all other interventions experience a high NPV of costs at 463,168 RWF per ha. Adoption of genetics was associated with higher costs through physical inputs but reduced costs from hired labor. Overall, this resulted in a net increase in the NPV of costs by 337,008 RWF per ha. costs of A similar pattern can be seen for adoption of improved pesticide practices (NPV of costs of 296,600 4 The CBA uses a discount rate of 12 percent to calculate the NPV of the various interventions and crops. This assumption will be tested in the sensitivity analysis. 91 RWF per ha), and the bundle including soil conservation, genetics, and pesticides (NPV of costs of 133,937). For a producer of Irish potatoes, a farmer has incremental savings of 106,491 RWF per ha and OFPS farmers also had incremental savings of 277,550 RWF per ha. Figure 6: Maize Cost Summary by Intervention (per ha) Figure 7 provides a summary of the incremental NPV of maize farmers under one of the Hinga Weze interventions. As previously mentioned, these results assume that farmers devote an average of 0.23 hectares of land to maize production and the CBA time-horizon is 10-years. Farmers participating in the genetics intervention experience an average NPV of 704,569 RWF ($739). This compares to 581,673 RWF ($610) for the combination of soil conservation, pesticides and genetics; 253,030 ($265) for pesticides; 158,042 ($166) for the combination of all other interventions; 82,072 RWF ($86) for soil conservation; and -5385 RWF (-$6) for planting / cultural practices. Irish potato farmers had an average NPV of 483,293 ($507) and OFSP farmers had an average NPV of 243,191 ($255). 92 Figure 7: Farm Household Incremental NPV for 0.23 ha of Maize Production The CBA uses an average cost of $30 per beneficiary and an overall beneficiary population of 533,000 to estimate the net project costs and benefits. The CBA also uses survey data and performance documents to estimate the percentage of the population who are under each intervention and/or crop. For example, the model assumes that 70 percent of the beneficiary population are producing maize5 under one of the Hinga Weze interventions, compared to 20 percent who are producing Irish potato and 4.4 percent producing OFSP. In addition, the CBA model assumes the 90 percent of trained farmers will adopt the technologies and farming practices, while there will be a 5 percent attribution rate after the Hinga Weze project has ended. Each of these assumptions will be tested in the sensitivity analysis. The sensitivity analysis showed how the results react to changes with deviations in estimations of yields, labor costs, inputs, and project costs. The results of this analysis showed that changes to the yield estimates and input use had large impacts on NPV however changes to labor costs including increasing and decreasing by 40 percent had smaller impacts on the farmer benefits. The table below shows the NPV from the farmers perspective with a 20 percent increase or decrease in yield estimates. Overall, all interventions remain largely positive from both the farmer and economic perspective. 5 For the maize interventions, the following distributions were used to evaluate the net benefits and costs of the project: soil conservation (11 percent), genetics (9 percent); planting (1 percent); pesticides (1 percent); soil conservation; pesticides; and genetics (5 percent); and all other interventions (72 percent). 93 Figure 8: Farm Household Incremental NPV with Sensitivity Analysis of Yields Throughout all sensitivity scenarios, genetics remains the best intervention followed by the combined soil conservation, pesticides, and genetics. The third best also remains consistently pesticides. All interventions always remain positive, excluding planting practices for maize. The sensitivity analyses of project costs showed that even with 30 percent overrun of project costs, the maize interventions alone would have an economic benefit over 34 million USD. In Table 12, the overall net benefits to farmers along with the costs from the perspective of USG. We can see that in terms of NPV for maize interventions. The “All Other Interventions” category has the highest NPV, this includes many combinations of different interventions, due to data limitations we could not separate their impacts. The highest impact intervention analyzed for maize was genetics. The bundle of soil conservation, pesticides, and genetics also had a largely positive impact, followed by soil conservation, and pesticides. Analysis of planting and cultural practices resulted in a negative NPV due to the limited increase in yields and the higher cost from inputs and labor. Irish Potato interventions led to an NPV of $29,640,696 USD and interventions for sweet potatoes led to an NPV of $2,864,702 USD. The large difference between these two was due to the larger population growing Irish Potato relative to OFSP. 94 TABLE 12: Comparison of interventions (USD) Maize Net Benefits to Farmers (all farm households) Cost from Perspective of USG NPV Soil Conservation 2,256,740 1,513,201 743,350 Genetics 15,083,515 1,160,990 13,922,729 Planting / Cultural Practices -33,473 182,628 -216,081 Pesticides 592,391 130,448 461,951 Soil Conservation, Pesticides, Genetics 7,313,100 678,331 6,634,886 All Other Interventions 26,786,814 9,574,909 17,212,568 Irish Potato 33,423,698 3,783,002 29,640,696 OFSP 3,696,963 832,261 2,864,702 CONCLUSION To estimate the net benefits and costs accruing to farmers under select interventions and crops under the Hinga Weze project, the team has calculated the financial net present value (NPV) using cost and benefit data from several surveys. The financial NPV reports the net financial gain or loss from the perspective of a typical farmer. With the exception of planting / cultural practices, the average farmer is experiencing a net benefit, with those under the genetics intervention experiencing a net benefit (NPV) of 704,569 RWF ($739). Farmers applying pesticides and the soil conservation practices, are experiencing an NPV of 253,030 RWF ($265) and 82,072 RWF ($86), respectively. Farmers participating in the planting / cultural practices experience an NPV of -5385 RWF ($-6). The primary drivers for this negative return are increased hired labor costs. Hinga Weze farmers producing Irish potato have an incremental NPV of 483,293 RWF ($507), while OFSP producers have an incremental NPV of 243,191 ($255). One of the key drivers for the high returns for Irish potato producers is the estimated reduction in the hired labor needed for land preparation, weeding, and harvesting. To estimate the overall net benefits and costs of the project, the evaluation team has included the project costs that have been used to advance new technologies and train farmers. At this point in time, the CBA model does not incorporate a wider set of impacts beyond those that come with immediate financial 95 implications for the farmers, including the impact of trade-based distortions (e.g. tariffs, subsidies, etc.). Based on this approach, maize interventions under the Hinga Weze project are expected to create an NPV of $38 million, including an PV of $51 million in benefits and $13 million in costs. The project’s interventions under Irish potato create an estimated NPV of $29 million, including an NPV of $33 million in benefits and $4 million in costs. Finally, the project creates an estimated NPV of $2.8 million under its OFSP interventions, including an NPV of $3.6 million in benefits and $1 million in costs. There are several considerations when interpreting these conclusions. First, while the team was able to build models for a limited set of interventions, there were several interventions that were not analyzed due to data limitations or the absence of data. For example, there was not enough production data from irrigation farmers to develop a representative model that differed greatly from irrigation CBA models that had previously been developed by CNFA. Second, while the cost data for this analysis comes primarily from the project’s 2018 progress survey, the method for collecting hired labor cost data in this survey differed from the approach used in the baseline survey. This means there could be discrepancies in the estimated labor costs reported in the CBAs. Fourth, the overall project benefits are sensitive to an assumed project cost of $30 per beneficiary. This is estimated by dividing the overall budget for training by the number of beneficiaries as reported in the program documents. Finally, the CBA model has not yet incorporated the broader economic impacts of these interventions. For example, the team has not incorporated fertilizer subsidies that the Government of Rwanda provides to farmers. This means an intervention promoting higher fertilizer use could cause higher costs for Rwanda. 96 SOURCES CNFA (2018). Feed the Future Rwanda Hinga Weze Activity: Quarter 2 Report For Fiscal Year 2018 Lal, R. (2004). Carbon emission from farm operations. Environment International, 30(7), 981-990. doi:10.1016/j.envint.2004.03.005 Ricke, K., Drouet, L., Caldeira, K. et al. Country-level social cost of carbon. Nature Climate Change 8, 895–900 (2018). https://doi.org/10.1038/s41558-018-0282-y 97 ANNEX C: KEY INFORMANTS INTERVIEWED KEY INFORMANT INTERVIEWS (Remote) NAME ORGANIZATIONAL ROLE COMPANY TYPE OF COMPANY Innocent Kabayiza Research Consultant Solution Seekers Ltd BDS Service Provider Olivier Muvandimwe Program Manager Rwanda Youth in Agribusiness Forum (RYAF) NGO Ammar Kawash Head, Smallholder Agricultural Market Support Unit World Food Programme International NGO Aimable Gakirage Agriculture Market Expert Garden Fresh Ltd Horticulture Company/Exporter Geoffrey Karemera Agriculture Engineer Africa Improved Foods (AIF) Aggregator- Maize and Soybeans Innocent Nzeyimana Agriculture Expert International Finance Corporation (IFC) Irrigation Finance Dr. Charles Murekezi Director General Ministry of Agriculture (MINAGRI) Government Agency Joseph Nzakunda Director of Cooperative Registration and Legal Affairs Rwanda Cooperatives Agency Government Agency Pacifique Mugwaneza Division Manager, Inspections Rwanda Cooperatives Agency Government Agency Telesphore Ndabamenye Embedded Advisor MINAGRI advisor to Cultivating New Frontiers in Agriculture (CNFA) Government Agency 98 Venuste Muhamyankama Executive Director Scaling Up Nutrition (SUN) NGO Partner Dr. Yvonne Umurungi Deputy Chief of Party Catholic Relief Services (CRS) Gikuriro Program NGO Partner/ Academic Dr. Didace Ndahimana Professor University of Rwanda College of Agriculture and Veterinary Medicine Academic Emmanuel Musabyimana Engineer Rwandan Agriculture Board (RAB) Water User Association (WUA) Government Agency Dr. Jules Rutebuka Land Husbandry Specialist RAB Terracing and Land Husbandry Department Government Agency Papias Mucyo Small Scale Irrigation Technologies Expert RAB Irrigation Department Government Agency Vestine Mukandayisenga Program Coordinator and Gender Expert DUHAMIC NGO Jean Claude Izamuhaye Head of Crop Research and Technology Transfers RAB/ Twigire Muhinzi Extension Government Agency Laurence Mukamana Deputy Chief of Party and Senior Agronomist CNFA/ Rwanda Implementing Partner Olivier Habimana Director, Agriculture Business Market and Finance Development CNFA/Rwanda Implementing Partner Source: Authors 1 99 KEY INFORMANT INTERVIEWS (In Person) Names Position Organization District/Sector Hyacinthe Tuyisenge Outreach C2 Imbaraga Rutsiro/ Mushubati Fiston Mutabazi BMFDO Imbaraga Rutsiro/ Mushubati Leonie Mukanshimyimana Outreach Officer C3 Imbaraga Rutsiro/ Mushubati Alice Mukasano Outreach Officer C1 Imbaraga Rutsiro/ Mushubati Immaculee Nabacu DAPO Imbaraga Rutsiro/ Mushubati Jeannette Nyirahabimana Chairperson Girubuzima Bwiza Kanjongo cooperative Nyamasheke/Kanjongo Speciose Manirafasha Member Girubuzima Bwiza Kanjongo cooperative Nyamasheke/Kanjongo Mediatrice Mukeshimana Secretary Girubuzima Bwiza Kanjongo cooperative Nyamasheke/Kanjongo Chantal Uwimana Vice Chairperson Girubuzima Bwiza Kanjongo cooperative Nyamasheke/Kanjongo Marcelline Nyabyenda Advisor Girubuzima Bwiza Kanjongo cooperative Nyamasheke/Kanjongo Agnes Uwizeye Accountant Girubuzima Bwiza Kanjongo cooperative Nyamasheke/Kanjongo Anne Marie Nyirahabineza District outreach Officer Imbaraga Karongi/ Rubengera Constatin Tuyishime District outreach Officer Imbaraga Karongi/ Rubengera J d’Arc Mutuyimana District outreach Officer Imbaraga Karongi/Rubengera Valens Mugabo District Agro Productivity Officer Imbaraga Karongi/ Rubengera Dominique Harindintwari BMFD Officer Imbaraga Karongi/ Rubengera Deogratias Kabera BMFDO Imbaraga Nyamasheke/ 100 Jean Minani DOO Imbaraga Nyamasheke/ Desire Nzeyimana HGO Imbaraga Nyamasheke/Kanjongo Liliane Akimana Niwenshuti DOO Imbaraga Nyamasheke/ Emmanuel Ndateba DAPO Imbaraga Nyamasheke/ Egidia Niyonsaba DM Imbaraga Nyamasheke/ Emmanuel Sibobugingo Chairman Urunana Mutuntu Karongi/Mutuntu Pierre Sinigenga Member Urunana Mutuntu Karongi/Mutuntu Hyacentha Uwamariya Secretary Urunana Mutuntu Karongi/Mutuntu Dismas Ngirishema Member Urunana Mutuntu Karongi/Mutuntu Speciose Nyirankuriza Member Urunana Mutuntu Karongi/Mutuntu Barthazar Niyingenera Member Urunana Mutuntu Karongi/Mutuntu Ange Ngabonziza President KADECO Karongi Vincent Bikorimana Manager KADECO Karongi Marie Jeanne Bankudiye Agro-dealer Imbaraga Karongi Emmanuel Karambizi Farmer Terimbere Muhinzi Nyarugenge Bugesera/ Nyarugenge Byukusenge Vianney Chairperson Terimbere Muhinzi Nyarugenge Bugesera/ Nyarugenge Laurien Ndikuryayo Farmer Terimbere Muhinzi Nyarugenge Bugesera/ Nyarugenge Marie Nyirashumbusho Farmer Terimbere Muhinzi Nyarugenge Bugesera/ Nyarugenge Olive Mukarushema Farmer Terimbere Muhinzi Nyarugenge Bugesera/ Nyarugenge Marthe Nyiragema Farmer Terimbere Muhinzi Nyarugenge Bugesera/ Nyarugenge Dismas Nsengiyumva Irrigation Intern Terimbere Muhinzi Nyarugenge Bugesera/ Nyarugenge Jean Bosco Kabanda Agronomist Umucyo Cooperative Bugesera/ Mareba 101 Sylvestre Twagirimana Signatory Umucyo Cooperative Bugesera/ Mareba Eliphaz Hakizimana Chairperson Umucyo Cooperative Bugesera/ Mareba Daniel Ndaberetse PR/ Auditing Abanyamurava ba Mayange Bugesera/ Mayange Protais Nkundirihamye Vc/Executive Abanyamurava ba Mayange Bugesera/ Mayange Ramadhan Nyundukozera Water User Association Abanyamurava ba Mayange Bugesera/ Mayange Jean Claude Ntambara Accountant Abanyamurava ba Mayange Bugesera/ Mayange Fidesius Ahishakiye Farmer Promoter Abanyamurava ba Mayange Bugesera/ Mayange Patricie Kamaraba Vc/Auditing Abanyamurava ba Mayange Bugesera/ Mayange Nyiramugwahashashe Advisor Abanyamurava ba Mayange Bugesera/ Mayange Alice Tuyishimire Sec/Executive Abanyamurava ba Mayange Bugesera/ Mayange Felicien Biziyaremye Pr/Executive Abanyamurava ba Mayange Bugesera/ Mayange Sylvie Umuhoza Intern Abanyamurava ba Mayange Bugesera/ Mayange Nicaise Ntirandeka M&E RDO Bugesera/ Nyamata B. Patrick Ngirabakunzi DOO RDO Bugesera/ Nyamata Fidele Uwizeyimana BMFDO RDO Bugesera/ Nyamata Consolee Bagwaneza DOO RDO Bugesera/ Nyamata Radys Nsengiyumva DAPO RDO Bugesera/ Nyamata Jacky Mutony DOO RDO Bugesera/ Nyamata Clarisse Mukampayana NGO RDO Bugesera/ Nyamata Jean de Dieu Niyonzima President Dusangire Amajyambere Bugesera/ Rilima Source: Authors 102 Province District Category Number Western Rutsiro IP 8 Rutsiro Coop 1 Nyamasheke Local Government 1 Nyamasheke SACCO 1 Nyamasheke IP 6 Nyamasheke Coop 1 Karongi IP 5 Eastern Bugesera Buyer 1 Bugesera Coop 14 Bugesera Farmer Promoter 1 Bugesera IP 7 Bugesera local Government 1 Bugesera CBV 1 Total 48 Interviews: Martin and Beate Category District Coop Nyamasheke Coop Karongi Coop Bugesera SACCO Bugesera Focus Groups 103 ANNEX D: SUMMARY OF IRRIGATION SITES ABISHYIZEHAMWE/BUSASAMANA (KAYONZA DISTRICT, RWANKWAVU SECTOR) The irrigation site in Rwankwavu sector has been in use since July 2020. According to the cooperative’s president, the irrigation system is working well, they have sufficient water in the dam and the cooperative is irrigating the full site (10 ha).6 Like other sites, the water pump is not pumping water unless it is sunny. However, it appears like the water pump has a greater capacity to pump water into the dam than pumps at some of the other sites. The water pump, which the cooperative said was from Germany, has sensors which detects debris in the water source. Thanks to the sensors, the water pump turns off automatically when there is debris in the water. This allows the cooperative to clear the debris so that it does not enter the pump. KOPERATIVE TWIGIRE MUHINZI RUKUMBERI (NGOMA DISTRICT, RUKUMBERI SECTOR) There are two adjacent irrigation sites of 10 Ha each in Rukumberi sector. One of the sites has been in operation since September 2019, while the second site is under construction. The completed site is in full operation and all the ten hectares are under irrigation. The cooperative shared that they do not have any issues with the irrigation equipment or infrastructure. The Rukumberi sector irrigation scheme has the same type of solar pump made in Germany as the irrigation scheme in Rwankwavu sector in Kayonza. Moreover, cooperative representative noted that the water is captured from a clean source of water. Between the clean source of water and the sensors on the equipment that detects debris in the water, the cooperative has not had issues with debris entering the pump. The cooperative is an out-grower for Lotec, which exports French beans and chili (discussed further below). TERIMBERE MUHINZI/NDEGO (KAYONZA DISTRICT, NDEGO SECTOR) The irrigation site in Ndego sector has been in use since January 2019. The site is 20 ha, but only 2 ha is currently under irrigation. The pump on the site is not pumping enough water to fill the dam. Like other cooperatives, the cooperative in Ndego sector attribute the low level of water in the dam to the solar pump and that there is no battery which would allow the pump to run during the night. Similarly, debris from the water source enters the pump and the pump is frequently broken. One farmer shared: “This technology system that we are using has no capacity to irrigate this land. /…/ we have three pumps but every week it gets closed and stop functioning until they come to repair them. We do not know whether the problem is caused by mud. But I think that is not the problem because even where the pump of our neighbors is there is 6 However, one member of the cooperative that the evaluation team spoke with just briefly, said that the site doesn’t have enough water. “I think you have seen this farmland. There is no water. It is drought, it is like where they have burnt charcoal” -farmer at Ndego irrigation site 104 mud”[the neighbor’s pump is not breaking]. 7 The cooperative shared that they often have to wait for two weeks before the equipment supplier come to repair the pump, which means that they cannot irrigate their field during that time. In addition, the farmers believed that the building material and the distance between the pump and the dam are problematic. One female farmer shared: “There are many challenges. I say this because when they were building the dam, they said that it is be well built, but by now, the dam is not well built. It seems that the dam does not maintain water due to the bad clay that they used when they were building it. The dam leaks, and it cannot be full, and when it gets full and two farmers irrigate, the dam becomes empty.” Another female farmer explained that Hinga Weze had organized a study visit of the irrigation site in Rukumberi sector, Ngoma District, and she shared: “When we were in Rukumberi, we saw that the distance between pumps and dam is between 50 and 70 meters which means that water reaches the dam easily and quickly. This differs from here where the distance between the pumps and the dam is like 700 meters. So, water does not reach the dam easily. It is difficult to have the dam full. Another thing is that the way that water passes to the dam is like climbing mountain because it is built on the mountain. I think the project of building dam was not good planned. My wish is that the pump should have two pipes so that the dam could be filled quickly and that farmers use water at time. … If we have enough water to irrigate our production we can gain money to solve other challenges. The problem is insufficient water.” Members of the cooperative in Ndego district are clearly frustrated by the lack of sufficient water. They were alternating between irrigating different fields, with the result that irrigated fields did not get enough water, and large track of land remained without irrigation. For example, one farmer grew watermelons during season 2020 C, but they were too dry due to limited irrigation. Others shared that they bought seeds, but “in the end the crop that we planted died due to lack of water”. It was particularly frustrating because those that did not irrigate planted earlier and were able to harvest, while those irrigating planted later and lost their harvest. One farmer shared: “[those who did not irrigate harvested because they] “cultivated earlier than us who believed in irrigation. [Hinga Weze] have recommended us to cultivate at the same time and promised us that there will not be a water problem. We waited for all the cooperative members so that we could cultivate at the same time while those who do not irrigate planted before us. In the middle of growing time, the water became a problem and the crops dried when they were flowering. Consequently, we did not harvest. Those who did not irrigate, their crops grew well, because they had planted on time.” In fact, the members of the cooperative shared that their family’s nutritional status had worsened since they started irrigating. “life in our families is getting worse. I say this because in last season (B) other farmers who are not in irrigation scheme harvested much production because they cultivated on time, but for us, we waited the announcement of cultivating believing that we will irrigate our crops. /…/They harvested while ours were dying from sun and lack of water. They are eating while we are suffering from hunger. They harvested maize while our maize dried when they were starting to produce. They grew sorghum while we did not. They grew cassava while 7 The neighboring irrigation scheme was installed by REMA (Rwanda Environment Management Authority). According to the farmer, the neighboring irrigation system has been in operation for three years “but their pump has not been broken or closed at all.” 105 our farms are like a ground. They grew sweet potatoes while we did not. Anyway, our family’s life is becoming worse.” ICYEREKEZO RUGENGE COOPERTIVE (GATSIBO DISTRICT, KIRAMURUZI SECTOR) With construction completed in June 2020, the irrigation site has been in operation for close to four months. Only two hectares out of the ten hectares are under irrigation as the solar pump is not pumping enough water into the dam to irrigate a greater area. The pump is only operating when there’s sun, which is not enough time for the pump to fill up the dam. In addition, debris from the water source gets into the pump and it breaks down. As part of the irrigation package, the cooperative obtained 16 pipes which a representative from the cooperative describes as “old” as they have been destroyed after just a couple of months of use and are now leaking. While the supplier has provided some training, according to the cooperative, the training was not sufficient to familiarize the farmers with the appropriate use of the pipes, but the pipes broke. The cooperative has tried to repair the pipes with tape and other local material to stop the leakage. Despite these challenges, the cooperative is growing French beans on the two hectares of irrigated land. Hinga Weze linked the cooperative to Excella, a horticultural exporter, in an out-grower scheme arrangement. Excella stationed a full-time extension support specialist on the site, and the cooperative harvested its first crop of 700 Kg of French beans during season 2020C (July –September 2020), which Excella purchased. The cooperative is linked to a buyer, which is investing technical assistance in the site, but because of the issues surrounding the pump’s capacity, the site is only utilized for one-fifth of its planned capacity. ABAHUJE AKABUGA COOPERTIVE (GATSIBO DISTRICT, KIRAMURUZI SECTOR) The Abahuje Ababuga irrigation site has been in use since September 2019. The irrigation site is ten hectares, and seven and a half of those hectares are prepared for irrigation, while the remaining two and half hectares of the land is not amended for irrigation. The cooperative is faced with a water shortage and is therefore only able to irrigate two and a half hectares of the prepared land. The pump is only pumping water when there is sun, and the pump does not have the capacity to pump enough water to fill up the dam. In addition, the water pump is often broken as debris from the water source get in the pump. The cooperative has detected that water source become filled with debris when it is raining, and they are therefore turning off the pump during rainy periods to not risk further damage to the pump. The cooperative noted that “we are no experts on why the pump breaks but we turn it off to not risk having the pump breaking”. The cooperative noticed that the pipes were not installed deep enough in the ground, and they broke pipes when digging or amending the soil. The cooperative alerted the District, who contacted the supplier and Hinga Weze. According to the cooperative, the supplier will come and reinstall the pipes. In addition, the cooperative only got eight irrigation pipes, which they feel is too little. The cooperative has discussed purchasing more pipes, but they are wondering if it is worthwhile to purchase pipes when there is not enough water for the site. The cooperative reported that the supplier has stationed a full-time technician at the irrigation site to assist the cooperative if anything breaks. However, the cooperative shared that the technician is repairing the equipment, which they appreciate, but does not teach the cooperative members how to properly use and repair the equipment themselves. TERIMBEREMUHINZI /NYARUGENGE (BUGESERA DISTRICT, NYARUGENGE SECTOR) 106 The irrigation scheme has been in operation since September 2019, but only one-third of the ten hectares of land is under irrigation. Like the other sites, the water pump is not pumping enough water to fill the dam, and the pump is only working when there is sun. The farmers complained also that the pressure was poor. The evaluation team visited this site in person and saw that the soil was dry. One farmer the team talked to was so unhappy with the irrigation site that she said she was ready to abandon the irrigation system all together. Similarly, to the other locations, debris from the water source gets caught in the pump and the pump is often broken. Hinga Weze has assigned an intern to the site to support the cooperative’s use of the irrigation system, but according to the cooperative, the intern has limited knowledge about repairs, so the cooperative needs to call the supplier for maintenance and repair support. However, the cooperative lamented that it takes on average two weeks before the supplier arrives to do the repairs. The cooperative had an out-grower scheme arrangement with Spicy Rwanda to grow chili, but the contract was not renewed.8 ABAKORANAMURAVA BA MAYANGE COOPERATIVE (BUGESERA DISTRICT, MAYANGE SECTOR) The irrigation site was completed at the end of 2019, and the cooperative started using the irrigation system in March 2020 (season 2020B). However, due to limited water in the dam, only three hectares is under irrigation. Members of the cooperative grow French beans, chili and eggplants under an out-grower scheme arrangement that Hinga Weze facilitated with the horticultural exporter Lotec. Like other cooperatives, the cooperative attributed the low level of water in the dam to the water pump and the fact that it is only pumping water when it is sunny. According to the cooperative, the dam has only been at full capacity three times since they started irrigating the site. The cooperative has also experienced the pump breaking from debris from the water source entering the pump. Lotec has an irrigation extension specialist on site to support the cooperative and is able to help trouble shoot when the irrigation equipment fails. However, like other cooperatives, the Mayange sector cooperative said that it can take up to two weeks for the supplier to come and repair the pump. In addition, five irrigation pipes have broken for the cooperative. The cooperative shared that were trained by the supplier on how to use the irrigation equipment, but despite the training, the farmers did not know how to use the pipes properly and they broke. The irrigation is under warranty for the first year and the cooperative is waiting for the pipes to be replaced by the supplier. The shortage of water delayed the cooperative to plant, and as seen in the picture, members of the cooperative are also watering the plants manually to supplement the irrigation. 8 The cooperative was not able to explain to the evaluation team why the contract was not renewed. 107 REFERENCES CNFA, Feed the Future Rwanda Hinga Weze Activity. National Extension Program “Twigire Muhinzi” Gap Assessment (September 2017). CNFA, Feed the Future Rwanda Hinga Weze Activity. Baseline Report (April 2018). CNFA, Feed the Future Rwanda Hinga Weze Activity. Irrigation Technologies Needs Assessment Report (July 2018). CNFA, Feed the Future Rwanda Hinga Weze Activity. Gender Gap Assessment (July 2018). CNFA, Feed the Future Rwanda Hinga Weze Activity, Youth Engagement Strategy and Engagement Plan (October 2018) CNFA, Feed the Future Rwanda Hinga Weze Activity. Annual Progress Report FY 2018 (October 2018). CNFA, Feed the Future Rwanda Hinga Weze Activity. Beneficiary- Based Survey (December 2018). CNFA, Feed the Future Rwanda Hinga Weze Activity. Beneficiary-Based Survey (December 2018). CNFA, Feed the Future Rwanda Hinga Weze Activity, Report on Implementation of Solidarity Groups Model Within Farmers’ Cooperatives. (February 2019). CNFA, Feed the Future Rwanda Hinga Weze Activity. Report on Selection of Nutrition- Sensitive Horticulture Value Chains and Rapid Market Analysis (March 2019). CNFA, Feed the Future Rwanda Hinga Weze Activity. Report on Options, Approach and Next Steps for the Establishment of the Farmer Promoter Incentive Fund (June 2019). CNFA, Feed the Future Rwanda Hinga Weze Activity. SBCC Gap Analysis (July 2019). CNFA, Feed the Future Rwanda Hinga Weze Activity. Report on Hinga Weze Care group Capacity (July 2019). CNFA, Feed the Future Rwanda Hinga Weze Activity. Revised Monitoring, Evaluation and Learning Plan: October 2018- June 2022 (September 2019). CNFA, Feed the Future Rwanda Hinga Weze Activity. Quarter 4 Progress Report and Year 2 Progress Report (October 2019). CNFA, Feed the Future Rwanda Hinga Weze Activity. Participant- Based Annual Survey Report FY 2019 (November 2019). 108 CNFA, Feed the Future Rwanda Hinga Weze Activity. Business Performances and Capacity Needs Assessment for Hinga Weze Beneficiaries; final version. [Study conducted by the Cooperative Advisory Academy] (December 2019). CNFA, Feed the Future Rwanda Hinga Weze Activity. Final Report Bench Terraces: CBA analysis, Agronomic and Climate Resilient Recommendations for Sustainable Use (2019). CNFA, Feed the Future Rwanda Hinga Weze Implementation Strategies, Maintenance and Sustainability Plan for Small Scale Irrigation Schemes, September 25, 2020, p.3. CNFA, Feed the Future Rwanda Hinga Weze Activity, 2020. How the SNS and MLIMA platform contribute to Hinga Weze MEL Indicator #11 (Number of Farmers Reached by New or Improved Market Information Systems) (October 11, 2020). CNFA, Feed the Future Rwanda Hinga Weze Activity. Concept Note: Savings Groups Quality Improvement Program (undated). CNFA, Feed the Future Rwanda Hinga Weze Activity. Concept Note: Nutrition Strategy (undated). CNFA, Feed the Future Rwanda Hinga Weze Activity. Concept Note: Hinga Weze Village Agents Business Model through Engaging Farmer Promoter and Youth as Professional Intermediaries (undated). CNFA, Feed the future Rwanda Hinga Weze Implementation Strategies. Title of the Intervention: Extension (undated). CNFA, Feed the Future Rwanda Hinga Weze Activity, Leadership Workshop: Hinga Weze Activity Presentation on PHH Practices Supporting SMEs and Farmers in Rwanda (undated). CNFA, Feed the Future Rwanda Hinga Weze Activity, Implementation Strategy: Gender and Female Empowerment Implementation Strategy (undated). CNFA, Feed the future Rwanda Hinga Weze Implementation Strategies. Title of the Intervention: Terraces for Soil Erosion Control and Increased Crop Yields (undated). Dalberg Research, Mid Term Performance Evaluation of Feed the Future’s Rwanda Hinga Weze Activity. Field Report (October 2020). Marfo, M and Peprah, J A. (2019, January) "Determinants Of Sustainability Of Microfinance Institutions In Ghana" International Journal of Innovative Research and Advanced Studies (IJIRAS) 1, no 6: 5- 15. http://www.ijiras.com/2019/Vol_6-Issue_1/paper_2.pdf Ministry of Agriculture and Natural Resources (MINAGRI). Nutrition Sensitive Agriculture Mainstreaming Guideline (May 2020). Ministry of Agriculture and Natural Resources (MINAGRI). Customized Agriculture Extension System in Rwanda (CAES) 2021- 2024 (June 2020). 109 Schreiner, M. (2000, August) "Ways Donors Can Help the Evolution of Sustainable Microfinance Organizations" Savings and Development 24, No. 4: 423- 37. http://www.microfinance.com/English/Papers/Donors_and_Evolution.pdf USAID/Rwanda. Engaging Males in Female-Dominated Family Duties (undated). USAID/Rwanda. New GALS Approach Builds Stable Homes for Farmer Communities (undated). USAID/Rwanda. Participatory Cooking Demonstration and Nutrition Education Empower and Improve Farmer Communities and Expand Nutrition Knowledge (undated). USAID/Rwanda. Rwanda Families Empowered to Curb Malnutrition and Stunting (undated). USAID/Rwanda. Contribution of Small Livestock Improve the Nutrition Status Among Youth and Young Adolescents (undated). USAID/Rwanda. USAID Support to Farmers in Bugasera Promotes Nutrition (undated). USAID. Multisectoral Nutrition Strategy: 2014-2025 (May 2014). USAID. Organizational Capacity Development Measures (May 2017).