MIDTERM PERFORMANCE EVALUATION OF THE ORORA WIHAZE ACTIVITY September 12, 2022 September 2022 This report was prepared for the United States Agency for International Development by ME&A, Inc. MIDTERM PERFORMANCE EVALUATION OF THE ORORA WIHAZE ACTIVITY September 12, 2022 Prepared for the United States Agency for International Development under USAID/Rwanda Collaborating, Learning, and Adaptation Activity (CLAA) Contract No.: Task Order 720-696-21-R-00005 Submitted to: Joshua Vetter jvetter@usaid.gov Submitted by: ME&A 1020 19th Street NW, Suite 875 Washington, DC 20036 Tel: (240) 762-6296 Evaluation Team: Dr. Gary Woller, Team Leader Pierre Célestin Habyarimana, Market Systems Expert Dr. Innocent Simpunga, Food Security Expert Kyung Endres, Monitoring and Evaluation Expert Fils Uwitonze, Monitoring and Evaluation Specialist Photo Credit: Small livestock market in Gakenke district, Janja sector (photo by Kyung Endres) DISCLAIMER The authors’ views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. ABSTRACT This midterm performance evaluation of the USAID/Rwanda Orora Wihaze (Raise Animals for Self￾Sufficiency) activity, conducted by ME&A, used a mixed methods approach to evaluate the activity’s outcomes, constraints, relevance, gender and social inclusion, and sustainability and provide recommendations for the remaining life of the activity and USAID follow-on programming. Implemented by a consortium led by Land O’ Lakes Venture37, Orora Wihaze aims to increase the availability of, access to, and consumption of animal-sourced foods (ASF) in eight districts within Rwanda by strengthening the ASF market system in four value chains: fish, goats, pigs, and poultry. To accomplish its objectives, Orora Wihaze uses a market system development approach characterized by multiple activities simultaneously addressing multiple identified market gaps/weaknesses related to ASF availability, accessibility, and consumption. To date, this approach has achieved considerable success contributing to improved animal raising and business practices of ASF market actors in the fish, pig, and poultry value chains and, subsequently, to improved nutritional outcomes, notwithstanding the numerous systemic constraints impeding the increased availability, accessibility, and consumption of ASF. In the process, Orora Wihaze has strengthened the targeted value chains, although evidence that it has contributed to systemic market strengthening is yet to emerge, while numerous gaps/weaknesses in the value chains remain, threatening the long-term sustainability of activity achievements. At the same time, Orora Wihaze has successfully integrated women and, to a lesser extent, youth into its activities but has struggled to integrate people with disabilities. TABLE OF CONTENTS EXECUTIVE SUMMARY....................................................................................................................................................i 1.0 INTRODUCTION................................................................................................................................................... 1 1.1 OVERVIEW OF THE ORORA WIHAZE ACTIVITY................................................................................ 1 2.0 EVALUATION PURPOSE, AUDIENCE, AND SCOPE .................................................................................. 2 2.1 EVALUATION PURPOSE, AUDIENCE, AND SCOPE............................................................................. 2 2.2 EVALUATION QUESTIONS........................................................................................................................... 3 3.0 EVALUATION METHODS.................................................................................................................................... 3 3.1 SAMPLING........................................................................................................................................................... 4 3.2 DATA ANALYSIS............................................................................................................................................... 5 3.3 METHODOLOGICAL LIMITATIONS.......................................................................................................... 5 4.0 FINDINGS & CONCLUSIONS ............................................................................................................................ 7 4.1 FINDINGS: EQ 1................................................................................................................................................ 7 4.2 CONCLUSIONS: EQ 1..................................................................................................................................... 9 4.3 FINDINGS: EQ 2................................................................................................................................................ 9 4.4 CONCLUSIONS: EQ 2................................................................................................................................... 12 4.5 FINDINGS: EQ 3.............................................................................................................................................. 12 4.6 CONCLUSIONS: EQ 3................................................................................................................................... 13 4.8 CONCLUSIONS: EQ 4................................................................................................................................... 17 4.9 FINDINGS: EQ 5.............................................................................................................................................. 17 4.10 CONCLUSIONS: EQ 5................................................................................................................................... 22 4.11 FINDINGS: EQ 6.............................................................................................................................................. 22 4.12 CONCLUSIONS: EQ 6................................................................................................................................... 27 4.13 FINDINGS: EQ 7.............................................................................................................................................. 27 4.14 CONCLUSIONS: EQ 7................................................................................................................................... 31 4.15 FINDINGS: EQ 8.............................................................................................................................................. 31 4.17 FINDINGS: EQ 9.............................................................................................................................................. 34 4.18 CONCLUSIONS: EQ 9................................................................................................................................... 36 4.19 FINDINGS: EQ 10............................................................................................................................................ 36 4.20 CONCLUSIONS: EQ 10 ................................................................................................................................ 37 5.0 RECOMMENDATIONS ....................................................................................................................................... 37 ANNEX 1. STATEMENT OF WORK ................................................................................................................... 40 ANNEX 2. BIBLIOGRAPHY .................................................................................................................................... 46 ANNEX 3. KII AND FGD GUIDES........................................................................................................................ 48 ANNEX 4. KIIS AND FGDS CONDUCTED...................................................................................................... 67 ACRONYMS Acronym Description ABIBM ASF-Based Inclusive Business Model AMIR Association of Microfinance Institutions in Rwanda ASF Animal-Sourced Foods CAHW Community Animal Health Worker CBSG Community-Based Savings Group CDCS Country Development Cooperation Strategy CLA Collaborating, Learning, and Adapting CLAA Collaborating, Learning, and Adapting Activity CRS Catholic Relief Services DRC Democratic Republic of Congo ECD Early Childhood Development EQ Evaluation Question FGD Focus Group Discussions GOR Government of Rwanda ICT Information and Communications Technologies IP Implementing Partner IR Intermediate Result KII Key Informant Interview LOA Life of Activity MEL Monitoring, Evaluation, and Learning MFI Microfinance Institutions MINAGRI Ministry of Agriculture and Animal Resources, Aquaculture MoU Memorandum of Understanding MSA MarketShare Associates MSD Market Systems Development MSMEs Micro, Small, and Medium Enterprises MTE Mid-Term Evaluation OAF One Acre Fund PWD Persons with Disabilities RCVD Rwanda Council of Veterinary Doctors RFA Request for Application RICA Rwanda Inspectorate, Competition, and Consumer Protection Agency Acronym Description RICH Rwanda Interfaith Council on Health RWF Rwandan Franc RYAF Rwanda Youth in Agribusiness Forum SACCO Savings and Credit Cooperative Organizations SBCC Social and Behavior Change Communication SOW Scope of Work TA Technical Assistance TMG The Manoff Group USAID United States Agency for International Development i EXECUTIVE SUMMARY EVALUATION OVERVIEW The United States Agency for International Development (USAID) commissioned this midterm performance evaluation (MTE) of the Orora Wihaze (Raise Animals for Self-Sufficiency) activity in Rwanda. The five-year activity, which began in October 2019, aims to increase the availability of, access to, and consumption of animal-sourced foods (ASF) by strengthening the ASF market system. The purpose of the MTE was to assess the continued relevance of the Orora Wihaze activity design and assumptions in addition to its achievements, progress, and challenges in reaching activity goals. The MTE answered the 10 evaluation questions (EQs) included in the Findings and Conclusions section below. ACTIVITY BACKGROUND Orora Wihaze uses a market systems development (MSD) approach to increase household income and sales of livestock and livestock-related goods and services over the life of the activity in the fish, pig, poultry, and goat value chains. The activity was rolled out in eight target districts of Rwanda representing the country’s Northern, Southern, Eastern, and Western Provinces: Burera, Gakenke, Kayonza, Ngoma, Ngororero, Nyamagabe, Nyamasheke, and Rutsiro. The activity works with small livestock producer households and non-producer households and market actors in the four value chains to increase the capacity within the value chains, strengthen relationships between value chain actors, and promote the consumption of ASF. Land O’ Lakes Venture37 leads the Orora Wihaze consortium of Catholic Relief Services, MarketShare Associates, The Manoff Group, and Urunana Development Communications. At the time of the MTE, Orora Wihaze was implementing 27 activities falling under the two objectives and six intervention categories aimed at increasing the availability, accessibility, and consumption of ASF. EVALUATION DESIGN AND LIMITATIONS The MTE used a mixed methods approach to answer the EQs consisting of a review of documents and secondary data, key informant interviews (KIIs), and focus group discussions (FGDs). Data collection focused on a sample of 27 Orora Wihaze activities in the fish, pig, and poultry value chains in four of the eight activity districts representing all four provinces in the county: Gakenke (Northern), Nyamagabe (Southern), Kayonza (Eastern), and Nyamasheke (Western), plus Kigali. (No interventions in the goat value chain were sufficiently advanced to include in the MTE.) Overall, the evaluation team completed 77 KIIs and 23 FGDs. The evaluation methods were subject to several limitations. One was the possible presence of sampling bias resulting from the use of purposive sampling to select the districts, stakeholder groups, and key informants included in the evaluation. Another limitation was the reliance on qualitative data and respondent subjective perceptions to measure activity outcomes in lieu of “hard data.” Related to this limitation was the limitation caused by the unavailability of updated activity performance data, while the most recent performance data (i.e., Fiscal Year 2021) was limited in scope due to COVID-19-related implementation delays. Another limitation was the challenge of assessing sustainability given that several interventions were in their early stages (in part due to delays caused by the Government of Rwanda (GOR)-mandated COVID lockdowns) and had yet to begin producing outcomes. A final limitation was that the predominantly qualitative methods, combined with the lack of a comparison group, did not allow observed outcomes to be attributed to activity interventions. ii FINDINGS AND CONCLUSIONS EQ 1: Did the partnerships established between Orora Wihaze and the market actors incentivize changes at the household, firm, or system level? Findings: Thirty of 53 (54 percent) key informants covering all stakeholder categories at all levels of the three value chains reported making changes to their livestock raising, business, and nutrition practices after receiving Orora Wihaze support. This includes Orora Wihaze’s private sector and government partners, small livestock producers, agro-dealers, agro-vets, cooperatives, Savings and Credit Cooperative Organizations (SACCOs,) and livestock producer and non-producer households. Participating in Orora Wihaze’s interventions is contributing to improved livestock raising and business practices among intervention participants. A smaller share of key informants— (18 of 44) or 41 percent— reported changes in nutrition practices but still enough to indicate an emerging pattern. If, however, we count only first-hand reports, the percentage increases to 53 percent spread across all four districts examined by the MTE, again suggestive of a more generalized phenomenon. Conclusions: Partnerships established between Orora Wihaze and market actors have generated numerous examples of improved livestock raising, business, and nutrition practices among small livestock producers, private market actors, and small livestock producer and non-producer households. While the evidence for improved livestock raising, business, and nutrition practices is found in all districts and all value chains examined by the MTE, it is still too early to conclude whether these emerging trends constitute evidence of broad-based system change. EQ 2: Is the activity demonstrating intended outcomes? Are households and businesses making more money, are markets transformed, are households consuming more ASFs? Findings: Key informants in all stakeholder categories and at multiple levels in the fish, pig, and poultry value chains reported improved business or nutrition outcomes due to changes they made to their livestock raising, business, or nutrition practices. This, again, includes Orora Wihaze private sector partners, small livestock producers, agro-dealers, agro-vets, cooperatives, SACCOs, and livestock producer and non-producer households. Forty-two percent of interviewed key informants reported improved livestock raising or business outcomes after receiving Orora Wihaze assistance. If only first￾hand accounts are counted, the percentage increases to 44 percent. Another one-third of key informants self-reported improved nutrition outcomes, including increased consumption of ASF, increased dietary diversity or consumption of a minimum diet, reduced malnutrition, reduced stunting, and reduced wasting. This figure increases to 40 percent if we count only first-hand accounts. Like the livestock raising and business outcomes, reported improvements in nutrition outcomes are found across all stakeholder groups and in each of the four MTE districts. Conclusions: Partnerships established between Orora Wihaze and market actors have generated numerous examples of improved livestock raising, business, and nutrition outcomes among small livestock producers, private market actors, and small livestock producer and non-producer households. These include, for example, increased livestock production, sales, and profits; increased input availability, lower input costs, and increased input sales; increased ASF availability in local markets; and increased ASF consumption. While the evidence for improved livestock raising, business, and nutrition outcomes is found in all districts and all value chains examined by the MTE, it is still too early to conclude whether these emerging trends constitute evidence of broad-based system change. iii EQ 3: Are there unintended outcomes generated by the activity interventions? Findings: The evaluation team found little evidence for unintended outcomes. Across the 100 KIIs and FGDs completed by the evaluation team, with few exceptions, key informants did not/could not cite examples of unintended outcomes, whether good or bad, generated by Orora Wihaze’s interventions. Two exceptions include the Afrisol poultry out-grower activity in Kayonza and the OX small livestock inputs distribution intervention in Nyamasheke and Rutsiro. In the first case, AfriSol withdrew from the outgrower scheme after its client restaurants, hotels, and bars were forced to close due to the COVID￾19 pandemic, leading to an oversupply of chickens. Notwithstanding, the other firm involved in the scheme, One Acre Fund, saw the potential of the outgrower model and is currently developing plans to replicate and scale up the poultry outgrower scheme drawing on the lessons learned from the Afrisol pilot, albeit potentially outside Orora Wihaze’s eight districts. In the latter case, the OX intervention exceeded expectations in terms of the degree and speed with which OX has scaled up the input distribution model. Conclusion: To date, the outcomes generated by Orora Wihaze’s interventions have been consistent with those predicted. In the two cases where key informants cited unexpected outcomes, the outcomes themselves were not unexpected, but the way they occurred. EQ 4: What are the drivers for differential performance of targeted districts? Findings: Districts differ in several features that are potentially relevant to explaining the differential performance across the eight Orora Wihaze districts, including their overall business climate, comparative advantages in targeted value chains, and the quality of potential partners. In the latter case, Orora Wihaze is looking to partner with “change agents” and/or “scale agents” with a vision for transforming the market and scaling up market reforms. Overall, the distribution of business grants across districts shows no clear pattern. Neither the strength of the overall business climate nor a district’s comparative advantage in a particular value chain helps explain partner selection or intervention outcomes. The findings point to the quality of partnerships as the primary determinant of partner selection and intervention performance, not differences in district characteristics. Conclusions: Differences in Orora Wihaze districts have not influenced either Orora Wihaze’s intervention strategy or the performance of those interventions. Rather, the primary factor driving Orora Wihaze’s intervention strategy is the quality of partners found in its targeted districts, where quality is defined by the soundness of the partners’ business ideas and their capacity and motivation to implement them and take them to scale. There is no evidence to date suggesting that the results of those interventions have been materially influenced by district differences. EQ 5: Is the Orora Wihaze MSD approach still sufficiently effective and adaptive to achieve nutrition outcomes and a profitable market in the current context? Findings: Orora Wihaze’s MSD approach incorporates several features designed to enhance its effectiveness and adaptability and ensure its relevance to the current context in Rwanda. ● Close alignment with GOR goals: Orora Wihaze’s emphasis on ASF consumption as a strategy to improve nutrition, especially among women and children, is featured in key GOR policy documents, including Rwanda Nutrition Policy and National Food and Nutrition Policy. ● In-depth market research: Orora Wihaze undertakes extensive market research to understand the factors affecting ASF markets and its targeted value chains. ● Active engagement with local market actors: Orora Wihaze field staff engage with local market system actors through 21 district-level and 138 sector-level producer platforms. ● Careful selection of partners: Orora Wihaze conducts extensive due diligence to select its grant partners, looking for partners willing and able to test new business models that unlock learning opportunities and new pathways for an inclusive business model (i.e., change agents) and/or partners that mobilize, support, align, or coordinate the efforts of individuals, firms, and institutions system wide (i.e., scale agents). iv ● Integration with established networks and institutions: Orora Wihaze strategically grafts its interventions into established and trusted volunteer, media, and community-based networks. ● Targeting of multiple constraints at multiple levels: Orora Wihaze’s interventions typically address multiple constraints within the target value chains and at multiple levels of the value chains. Conclusions: Orora Wihaze’s MSD approach remains an effective and adaptive approach to achieving nutrition outcomes and a profitable market in the current context. EQ 6: What are the constraints for achieving results? Findings: Primary market constraints identified by key informants include— in order of the number of mentions: lack of extension services, lack of access to inputs, lack of access to credit, high cost of inputs, lack of capacity, lack of transport, low production, high animal morbidity/mortality, lack of vet services, and lack of market linkages. Orora Wihaze also faces implementation constraints that affect its ability to achieve desired results, perhaps more so than the market constraints. In the Rwanda ASF market system, constraints exist at multiple points in the system involving multiple actors. The specific challenge is to identify what and where these constraints are and then to address them through strategic interventions working through capable and motivated partners given the activity’s budget and time constraints. Orora Wihaze’s interventions strategically address multiple ASF market constraints in its target value chains. Notwithstanding, despite Orora Wihaze’s successful efforts to strengthen different value chain functions, numerous gaps remain. For example, nutrition messaging interventions stress overcoming barriers to ASF consumption, but these are not necessarily matched by interventions strengthening ASF availability or accessibility in the same locations. A failure to coordinate availability and/or accessibility interventions to utilization interventions can result in unfulfilled expectations and de-motivation. This coordination challenge is heightened by Orora Wihaze’s rigorous and long grant application process. Conclusions: The ASF sector faces numerous market constraints related to ASF availability, accessibility, and consumption that adversely affect the functioning of market systems, and these constraints exist to varying degrees in each of Orora Wihaze’s targeted poultry, pig, fish, and goat value chains. Orora Wihaze’s MSD approach relies on extensive research and due diligence to identify and simultaneously address multiple market constraints with each intervention; however, resource and time constraints together with challenges in identifying capable and motivated partners limit Orora Wihaze’s ability to address all market constraints. EQ 7: Is the MSD approach as implemented by Orora Wihaze creating necessary conditions for sustainability of interventions and outcomes? What are the key barriers to sustainability? Orora Wihaze partners can be grouped into five partner profiles: 1) private market actors; 2) community￾based private market actors (i.e., community-based savings groups (CBSGs), cooperatives); 3) social enterprises (i.e., Urunana, Viamo); 4) community volunteers; and 5) GOR ministries. The prospects for sustainability for each partner profile are described below. ● Private market actors: Providing follow-up support; working to solve the access to affordable inputs, extension, and credit constraints; and linking them to other market actors up, down, and across the value chain will increase the prospects for sustainability. ● Community-based market actors: Providing follow-up support; working to solve the access to affordable inputs, extension, and credit constraints; and linking them to other market actors up, down, and across the value chain will increase the prospects for sustainability. ● Social enterprises: Social enterprise partners have demonstrated an ability to secure financial resources to operate, and this is not expected to change. Prospects for sustainability depend on their commitment to ASF programming and the extent they have internalized ASF programming. v ● Community volunteers: For community volunteers doing nutrition messaging, the likelihood that they continue depends on their commitment to ASF and the extent to which they have internalized ASF messaging. Orora Wihaze can increase sustainability prospects by providing follow-up support to ensure that ASF messaging, and messaging capacity are embedded within the volunteer networks. The sustainability of the community animal health worker (CAHW) model will depend on the creation of a fee-based service delivery model that will allow them to profit from their activities. ● GOR ministries: Providing follow-up technical and capacity-building support will increase the prospects for sustainability. Conclusion: Orora Wihaze’s interventions are putting in place the foundations for sustainability in supported value chains. However, sustainability is not assured as gaps in market support remain, thus more work is required to build on the existing foundation to increase the prospects for sustainability. Orora Wihaze can enhance the prospects for sustainability by working with partners to embed ASF as an institutional priority and building their capacity to play their roles within the market system. For private market actors and community-based market actors, Orora Wihaze can increase their prospects for sustainability by ensuring they are linked in mutually beneficial commercial relationships with other market actors, while for CAHWs, Orora Wihaze can increase their prospects for sustainability by working with its partners to develop a fee-based service delivery model. EQ 8: Are the initial assumptions made by USAID and the contractor still valid? Orora Wihaze’s MSD approach is informed by three critical assumptions: 1. Income Pathway Assumption: ASF availability and accessibility are increased through market interventions that increase the income earned by ASF market actors. 2. Nutrition Pathway Assumption: ASF consumption is increased through activities that address the constraints to ASF consumption. 3. Mutual Dependence Assumption: Income and nutrition pathways are mutually dependent. Achieving sustainable improvements in nutrition outcomes require strategic interventions in both. Orora Wihaze’s intervention strategy incorporates six strategic priorities reflecting a set of assumptions about how different combinations of the income and nutrition pathways will contribute to increased ASF availability, accessibility, and consumption. During KIIs and FGDs, Orora Wihaze partners and beneficiaries reported numerous outcomes that were consistent with each of the six strategic priorities and their underlying assumptions, lending credibility to their validity. Conclusions: The assumptions underlying Orora Wihaze’s MSD approach are based on sound conceptual reasoning and are supported by evidence in the form of outcomes reported by activity partners and beneficiaries. The assumptions, thus, remain valid. EQ 9: How did Orora Wihaze integrate women, youth, and persons with disabilities (PWDs) into its programs? Findings: Orora Wihaze’s integration of women exceeds targets while its integration of youth lags targets. Whereas key informants across all activity interventions consistently cited efforts to integrate women and, to a lesser extent, youth into their programs, few cited efforts to integrate PWDs. Women, youth, and PWD participation are heavily concentrated in CBSGs and cooperatives. CBSGs and cooperatives have a majority of female members and include youth and PWDs. Orora Wihaze’s interventions seek to integrate social inclusion through three inclusion strategies: 1) quota targeting; 2) strategic targeting; and 3) merit targeting. Quota targeting occurs when Orora Wihaze or its partners have specific social inclusion policies to ensure a certain level of female, youth, or PWD representation in their programs. Strategic targeting occurs when Orora Wihaze or its partners target women and youth as part of their business or organizational strategy. Merit targeting does not involve vi quotas/targets nor strategies for social inclusion but rather is a gender and age-neutral approach that opens participation to all parties based on merit. Conclusions: Orora Wihaze has effectively integrated women into its programs using a combination of quota targeting, strategic targeting, and merit targeting. It has had moderate success integrating youth but relatively little success integrating PWDs using the same approaches. Orora Wihaze’s work with CBSGs and cooperatives has been its most effective approach for integrating women, youth, and PWDs, in the latter case the sole successful approach. EQ 10: Does the Orora Wihaze social inclusion strategy ensure continuity beyond the life of the activity? Findings: Each of Orora Wihaze’s three social inclusion strategies (i.e., quota targeting, strategic targeting, and merit targeting) embed factors that will determine how sustainable they are after the activity ends. Partners using a quota targeting strategy to include women (most often), youth (less often), or PWDs (rarely) are motivated by a combination of internal priorities and outside pressures (e.g., donor priorities). Partners practicing strategic targeting were practicing strategic targeting before they entered a partnership with Orora Wihaze and are expected to continue doing so after the activity ends. Factors determining the sustainability of merit targeting are less evident. Within the scope of Orora Wihaze’s interventions and partnership network, a critical factor contributing to the sustainability of merit targeting is its success. To the extent women, youth, or PWDs succeed in the opportunities provided to them through merit targeting, it potentially creates a demonstration effect incentivizing additional merit-based opportunities for women, youth, and PWDs over time. Conclusions: Orora Wihaze’s social inclusion strategy is likely to be sustained beyond the life of the activity. This is due to the internal policies or characteristics of Orora Wihaze’s partners. Partners practicing quota targeting or strategic targeting are doing so independently of Orora Wihaze and are expected to continue this practice after the activity ends. Partners practicing merit targeting are also doing so largely independently of Orora Wihaze. Whether they continue this practice will be determined by a variety of factors outside Orora Wihaze’s control. It is, nonetheless, hoped that the success of merit targeting will create a demonstration effect within the targeted market systems leading to increased social inclusion beyond the life of the activity. RECOMMENDATIONS ● Simplify and shorten the grant application process. The current grant application process potentially screens out qualified applicants who lack the time, patience, or inclination to complete the application process in addition to qualified candidates who do not meet the approval thresholds, notwithstanding possessing the desired capacity and motivation. ● Align nutrition messaging with what is available in the local market. Nutrition messaging should be adapted to what is available and affordable in the local markets. Also target value chain support and social and behavior change communication (SBCC) activities so that they overlap, again to the extent feasible. ● Monitor more closely intervention outreach to and impact on target beneficiaries. The Orora Wihaze MEL plan tracks yearly changes in outcome indicators across activity catchment areas and activity beneficiaries. This survey is done annually and provide useful information about activity outreach and outcomes. At the same time, however, the survey is large and complex and takes considerable time to analyze, which limits its management and decision-making utility. More timely and targeted information could also be collected using a variety of approaches like in-person, mobile phone, or SMS surveys; rapid assessments; crowdsourcing, market observations; etc. This will provide additional useful information on the effectiveness and sustainability of activity interventions. vii ● Provide market information matching ASF buyers to ASF sellers. To help ensure that demand for ASF corresponds to supply, use media to broadcast information to consumers about what types of ASF products are available and where. The same information networks could be used to provide information on ASF demand to facilitate partner recruitment. ● Provide follow-up support to implementing partners. Providing follow-up training and reinforcement support to implementing partners will be important to ensure the sustainability of Orora Wihaze’s interventions. ● Invest in increasing Orora Wihaze field staff. Orora Wihaze’s District Portfolio Managers and Nutrition Specialists are understaffed and under-resourced. Each district should have at least one dedicated, full-time field staff. Investigate the feasibility of increasing the number of Rwanda Youth in Agribusiness Forum (RYAF) interns and/or extending their period of service from six to 12 months. ● Buy down the risk of SACCOs and microfinance institutions (MFIs) to lend to small livestock producers. SACCOs and MFIs offer the best opportunities for scaling loan access. USAID/Rwanda can help buy down the risk further either by linking SACCOs and MFIs with the Business Development Fund loan guarantee program or linking pig and poultry producers to the GOR’s subsidized livestock insurance scheme. ● Target CBSGs and cooperatives to extend outreach to youth and PWDs. Extending outreach to youth and PWDs will require an explicit targeting strategy with cooperatives and CBSGs offering the greatest potential for achieving significant outreach to these groups. Note that this recommendation does not rule out implementing intentional activities targeted specifically to youth and PWDs nor the continued layering of integration activities on existing scale partnerships. ● Develop, pilot, and scale up a fee based CAHW extension model. Developing, testing, and (if time allows) implementing a fee-based extension model for CAHWs is important to ensure the sustainability of the CAHW model and should be a priority over the remaining life of the activity. 1 1.0 INTRODUCTION This report presents the findings, conclusions, and recommendations from the mid-term evaluation (MTE) of the Orora Wihaze activity funded by the United States Agency for International Development (USAID) in Rwanda and implemented by a consortium of partners led by Land O’ Lakes Venture37. The evaluation was conducted by a team managed by the Collaborating, Learning, and Adapting Activity (CLAA), a $9.8 million, five-year contract USAID/Rwanda awarded to ME&A under the USAID/Kenya and East Africa Evaluation, Assessments, and Analyses Indefinite Delivery Indefinite Quantity Contract. 1.1 OVERVIEW OF THE ORORA WIHAZE ACTIVITY The Orora Wihaze (Raise Animals for Self-Sufficiency) activity started in October 2019. It works to increase the availability of, access to, and consumption of animal-sourced foods (ASF) by strengthening the ASF market system. Activity interventions fall under two broad objectives: 1) private sector led ASF value chains strengthened; and 2) increased demand for ASF consumption for women and children. Each objective, in turn, includes three intervention categories (see Table 1). The activity’s theory of change posits that the production (availability) of and access to ASF and the desirability and consumption of ASF work simultaneously and in a mutually supportive manner to increase both the supply of and demand for ASF. Orora Wihaze uses a market systems development (MSD) approach to increase household income and sales of livestock and livestock-related goods and services over the life of the activity (LOA) in the fish, pig, poultry, and goat value chains in eight target districts of Rwanda representing the country’s Northern, Southern, Eastern, and Western Provinces: Burera, Gakenke, Kayonza, Ngoma, Ngororero, Nyamagabe, Nyamasheke, and Rutsiro (see Figure 1). Figure 1. Map of Rwanda Additionally, Orora Wihaze seeks to increase the proportion of children 6-23 months of age and women of reproductive age receiving a minimum acceptable diet in the eight districts. Toward these ends, the activity works with small livestock producer households and non-producer households and a range of market actors in the four targeted value chains to increase the capacity within the value chains, strengthen 2 relationships between value chain actors, and promote the consumption of animals and animal products in producer and non-producer households. Land O’ Lakes Venture37 leads the Orora Wihaze consortium of Catholic Relief Services, MarketShare Associates, The Manoff Group, and Urunana Development Communications. As of the time of the MTE, Orora Wihaze had 27 individual activities active or in process. Table 1 shows how these 27 activities are distributed by objective and intervention category. Individual interventions cover anywhere from one to all eight activity districts. Table 1: Orora Wihaze Individual Activities by Objective and Intervention Category Objective/Intervention Number of Activities Objective 1: Strengthen Private Sector-Led ASF Value Chains 16 Intervention 1: Development of ASF Production 9 Intervention 2: Development of End Market Access 4 Intervention 3: Development of Financial Services Market 3 Objective 2: Increase Demand for ASF Consumption for Women and Children 11 Intervention 4: Development of Nutrition Extension 5 Intervention 5: Development of ASF Product Market 4 Intervention 6: Development of Women’s Empowerment 2 2.0 EVALUATION PURPOSE, AUDIENCE, AND SCOPE 2.1 EVALUATION PURPOSE, AUDIENCE, AND SCOPE After two and one-half years of implementation, USAID/Rwanda has commissioned this MTE of Orora Wihaze. The purpose of the evaluation is to assess the continued relevance of the Orora Wihaze activity design and assumptions. The overall objective of the MTE is to assess Orora Wihaze’s achievements, progress, and challenges in reaching activity goals. Specific objectives include the following: ● Identify lessons learned in its first two and one-half years of implementation, particularly from applying an MSD approach within the Rwandan context. ● Inform the work plans of the activity’s remaining two years to support the achievement of target results. ● Contribute to the learning of the contributions of the MSD approach on ASF, the integration of disadvantaged groups using the MSD approach, the sustainability of systems changes, and the extent to which Orora Wihaze has successfully used collaborating, learning, and adapting (CLA). The primary intended audience for the MTE is USAID/Rwanda, specifically its Economic Growth team. Orora Wihaze implementing partners (IPs) and MSD stakeholders are the secondary audiences. Audiences will use evaluation findings and recommendations to identify potential midcourse corrections for the activity and future USAID/Rwanda MSD programming. The evaluation will also provide an opportunity to share lessons learned with USAID/Rwanda and other key stakeholders. The scope for the MTE is all activities implemented by Orora Wihaze from its inception through May 2022 across each of its eight implementation districts and Kigali. These activities and implementation districts form the sampling frame from which the sample of interventions and districts is drawn for the MTE (see Section 3.1). 3 2.2 EVALUATION QUESTIONS The MTE will answer the 10 EQs found below. The 10 EQs were modified from the original 11 EQs found in the evaluation Statement of Work (SOW) (See Annex 1 for the evaluation SOW.) The 10 EQs will be used to address the four broader concepts, as follows: Performance 1. Did the partnerships established between Orora Wihaze and the market actors incentivize changes at the household, firm, or system level?1 2. Is the activity demonstrating intended outcomes? Are households and businesses making more money, are markets transformed, are households consuming more ASFs? 3. Are there unintended outcomes generated by the activity interventions? 4. What are the drivers for differential performance of targeted districts? 5. Is the Orora Wihaze MSD approach still sufficiently effective and adaptive to achieve nutrition outcomes and a profitable ASF market in the current context? 6. What are the constraints for achieving results? 2 Sustainability 7. Is the MSD approach as implemented by Orora Wihaze creating necessary conditions for sustainability of interventions and outcomes? What are the key barriers to sustainability among different partner profiles? CLA3 8. Are the initial assumptions made by USAID and the contractor still valid? Cross Cutting issues 9. How did Orora Wihaze integrate women, youth, and persons with disabilities (PWDs) into its programs? 10. Does the Orora Wihaze social inclusion strategy ensure continuity beyond the life of the activity? 3.0 EVALUATION METHODS The MTE used a mixed-methods approach that included both primary (qualitative) and secondary (qualitative and quantitative) data collection. Primary data collection methods included a document review, key informant interviews (KIIs) with activity stakeholders, and focus group discussions (FGDs) with activity beneficiaries. (See Annex 2 for a bibliography of documents reviewed for the MTE, Annex 3 for copies of the KII and FGD discussion guides, and Annex 4 for breakdown of KIIs and FGDs conducted.) Overall, the evaluation team completed 77 KIIs and 23 FGDs (Table 2) with diverse activity stakeholders and beneficiaries at the national, district, and community levels in four sample districts. Apart from a limited 1 The original EQ 6 in the SOW was moved to EQ 1 while the original EQs 1, 2, 3, 4, and 5 are now EQs 2, 3, 4, 5, and 6. This change was made because changes in behaviors/practices logically precede outcomes, so it made sense to discuss them first. The revised list of EQs also removed language in EQ 1 (formerly EQ 6) asking about constraints to achieving changes as these are discussed at length in the response to EQ 6 (formerly EQ 5). 2 EQ 6 (formerly EQ 5) initially read, “What are the constraints for achieving results, and what are the recommendations for improvement?” The last half of the question was removed because recommendations are provided in their own section of the report based on the findings and conclusions from all 10 EQs. 3 The original EQ 9 in the SOW under CLA—"What changes or adaptations does the contractor need to make in the remaining implementation period?”—was removed from the list of EQs, as recommended changes and adaptions are addressed in their own recommendations section. 4 number of virtual KIIs conducted with activity staff, KIIs and FGDs were done in person with stakeholders and beneficiaries located in Kigali and the sample districts. Table 2: KIIs and FGDs Completed by Location Location KIIs FGDs Total Kigali 24 0 24 Nyamagabe 15 5 20 Nyamasheke 14 8 22 Gakenke 13 4 17 Kayonza 11 6 17 Total 77 23 100 3.1 SAMPLING The evaluation team used a purposive sampling approach to select a reasonably representative sample of activities covering both activity objectives and all six activity categories in four of the eight activity districts.4 To select the sample districts and activities, the evaluation team used the following sampling criteria: ● Agroecological zones. The evaluation team identified four distinct agroecological zones among the eight activity districts and selected one district from each zone. Based on this criterion, the evaluation team selected Gakenke in the Northern Province, Nyamasheke in the Western Province, Nyamagabe in the Southern Province, and Kayonza in the Eastern Province. ● Value chain. The evaluation team sought to include activities representing each of the four activity value chains. Notwithstanding, there was no activity in the goat value chain that was sufficiently mature at the time of the MTE to warrant investigation. Thus, the final sample was limited to activities in the poultry, fish, and pig value chains.5 ● Intervention categories. The evaluation team sought to include activities representing each of the six intervention categories. The evaluation team worked closely with activity IPs to identify activities, stakeholders, and individual key informants to participate in the KIIs and FGDs using the above sampling criteria. However, to preserve its independence and objectivity, the evaluation team made the final decision on activities, districts, and key informants to include in the sample. The final list of activities selected for the MTE, and the districts where they are being implemented, is presented in Table 3. Table 3: Orora Wihaze Mid-Term Evaluation Sample by Activity and District Intervention Kayonza Gakenke Nyamgabe Nyamasheke 1.1.1: AfriSol poultry outgrower model x 1.1.3: OX small livestock inputs distribution x x x x 4 Purposive sampling is a non-probability sampling technique that depends on the evaluator’s informed discernment to select the units—such as people, organizations, cases, events, pieces of data, etc.—that are studied, which are judged to offer sufficient detail and variability to answer the EQs. The amount of the sample is smaller in comparison to probability sampling techniques. Unlike probability sampling techniques, purposive sampling does not take arbitrary units from a population to create a sample report to generalize information like statistical inferences. It is the general intent of a qualitative research design. 5 Although goat raising is found throughout Rwanda, goats are generally seen more as a store of value as opposed to a source of income. Thus, Orora Wihaze has had a more difficult time finding qualified partners in the goat value chain than in the other three value chains. 5 Intervention Kayonza Gakenke Nyamgabe Nyamasheke 1.2.1: Rwanda Council of Veterinary Doctors (RCVD) extension services x x x x 2.1.2: A-Z Pig Ltd. pork product business x 2.1.4: GACOPROCO egg collection center x 3.1.1: Memorandum of Understanding (MoU) with Vision Fund Rwanda x x x x 3.1.2: Tripartite MoU with Nguriza Nshore and Association of Microfinance Institutions in Rwanda (AMIR) x x x x 4.1.2: Rwanda Interfaith Council for Health (RICH) ASF consumption messaging x x x x 4.1.3: Targeted radio programming to enhance ASF messaging x x x x 4.2.2: Viamo information and communications technologies (ICT) for Development for ASF consumption messaging x x x x 4.2.3: Urunana dissemination of ASF messaging x x x x 5.1.2: Triumvirate Food Co. Ltd. fish products x 5.2.2: Hope and Fine Fish Products x 5.2.3: MoU with Rwanda Inspectorate, Competition and Consumer Protection Agency (RICA) x x x x 6.2.1: Community-based savings groups (CBSGs) and cooperatives’ ASF-based inclusive business model (ABIBM) x x x x 6.2.2: MoU with Rwanda Youth in Agribusiness Forum (RYAF) x x x x 1.1.1: AfriSol poultry out-grower model x 3.2 DATA ANALYSIS Qualitative data analysis involved coding the data using the Atlas.ti software to identify themes in the interview notes and program documents and assess their relative importance in answering the EQs. 3.3 METHODOLOGICAL LIMITATIONS The MTE evaluation methods were subject to the limitations described below. Sampling bias: The main disadvantage of purposive sampling is the possibility of sampling bias as the sample is based on the judgment of the evaluators, activity IPs, and other knowledgeable parties. To mitigate this limitation, the evaluation team sought to achieve a reasonably representative sample covering the full range of activity objectives, interventions, value chains, and locations using a set of explicit sampling criteria. Nonetheless, some unknown amount of selection bias may exist in the qualitative sample. Assessing outcomes: Another limitation involves challenges in assessing the business, household, and community-level outcomes, like increased sales, increased consumption of ASF, improved nutritional status, etc. The qualitative nature of the evaluation and the small number of businesses and households interviewed (out of potentially thousands affected by the activity) did not allow the evaluation team to 6 generalize findings and conclusions about how the activity has affected changes at these levels. Consequently, this report limits itself to reporting its findings on these topics and drawing conclusions based on the evidence available without attempting to generalize them to the relevant populations. Assessing sustainability: Assessing the sustainability of Orora Wihaze’s activities at its midterm is challenging given that many activities are still in their early stages. This challenge is especially pronounced given that Orora Wihaze does not deliver services directly to target populations but rather works through partners (in this case market actors/intermediaries) to deliver services. As discussed later in this report, the process of selecting qualified partners requires due diligence and negotiation and may also require building the partner’s capacity before it is ready to play its assigned role. Exacerbating this challenge was the Government of Rwanda (GOR) imposed shutdowns and other restrictions in response to the COVID￾19 pandemic (particularly during January-July 2021) that delayed activity implementation by up to several months. To address this challenge, the evaluation focuses on factors that, based on the nature of the activities and available evidence, contribute to, or detract from, the likely sustainability of the evaluated activities. The subjectivity of evidence: Qualitative responses provided during KIIs and FGDs often reflect the subjective perceptions of key informants. While the evaluation team was not able to verify all perceptions and anecdotes cited by key informants, it sought to compensate for this limitation through cross-team validation exercises and systemic data analysis that synthesized multiple data sources supported by using advanced qualitative data analysis software. Unavailability of updated Orora Wihaze performance monitoring data: Orora Wihaze’s primary method for collecting performance monitoring data on the outcomes of interest to the MTE (and included in the activity’s Monitoring, Evaluation, and Learning (MEL) Plan) is its annual household survey. Unfortunately, at the time of the MTE, Orora Wihaze had yet to begin data collection for its 2022 household survey, and the most recent performance data on the outcomes of interest are from its 2021 household survey. In addition, Orora Wihaze had conducted a baseline internal evaluation and, at the time of the MTE, had recently completed its midterm internal evaluation. The evaluation team had hoped to use the results of the MTE (using the baseline as a benchmark) to supplement its findings. Unfortunately, at the time of the MTE, MarketShare Associates (MSA) was still working on analyzing the midterm results, meaning they were not available to the evaluation team in time to prepare this report. Assessing attribution: A final limitation is that the qualitative and small sample nature of the qualitative evaluation methods do not allow the evaluation team to attribute any observed changes to Orora Wihaze activities. Observed outcomes may be the combined result of many factors, including (in addition to Orora Wihaze) other development programs, economic or social trends or events, actors entering or leaving the market, etc. For example, the evaluation team found several other donor-funded programs working in Orora Wihaze intervention sites. Given that this is a mid-term performance evaluation, and the absence of a comparison group, it was only possible to assess progress toward intended results as opposed to measuring the full achievement of those intended results. Thus, the evaluation results should be interpreted bearing this caveat in mind. 7 4.0 FINDINGS & CONCLUSIONS This section presents key findings and conclusions related to each of the MTE’s 10 EQs. It is followed in Section 5 by a presentation of recommendations based on the findings and conclusions. 4.1 FINDINGS: EQ 1 Did the partnerships established between Orora Wihaze and the market actors incentivize changes at the household, firm, or system level?6 Key informants in all stakeholder categories and at all levels of the three value chains covered by this MTE (fish, pigs, and poultry) who were interviewed for the MTE reported making changes to their livestock raising,7 business, and nutrition practices after receiving Orora Wihaze support.8 This includes Orora Wihaze’s private sector and GOR partners who, as a condition for receiving activity support, were required to change their business or operating model. It also includes other private sector actors (e.g., small livestock producers, agro-dealers, agro-vets, cooperatives, Savings and Credit Cooperative Organizations (SACCOs), etc.), livestock producer households, and non-producer households who also reported having changed their business and/or nutrition practices after receiving Orora Wihaze support. Given the qualitative nature of the MTE, evidence for changes in livestock raising, business, and nutrition practices is the number and share of key informants who are reporting changes. The higher the number of key informants that are reporting changes in practices, the more confident we can be that this is a generalized phenomenon rather than merely a collection of anecdotes. This is in fact what we see in Table 4, which reports the number and share of key informants who reported having changed their livestock raising, business, and nutrition practices after receiving Orora Wihaze assistance. Looking first at livestock raising and business practices, 30 of 53 (57 percent) of key informants interviewed reported changing their practices after receiving Orora Wihaze support. This figure includes district officials who were providing general observations of livestock and business practices in their district, not necessarily their own. Focusing on first-hand reporting produces a marginal increase in the share of key informants who changed their livestock raising or business practices to 57 percent (30 of 53). Table 4. Number and Share of Key Informants Reporting Changes in Livestock Raising, Business, and Nutrition Practice Stakeholder Group9 Livestock/Business Practices Nutrition Practices Agro-dealers/agro-vets 9/14 N/A Cooperatives 6/6 4/6 Farmer groups 4/6 3/6 6 The evaluation team elected not to include Orora Wihaze’s performance monitoring results in its response to EQs 1 and 2 because the results of its 2022 annual household survey were not available, while the 2021 results were limited in scope due to COVID-19-related implementation delays. 7 Livestock raising practices refer to small livestock producer households who are not technically businesses but who raise livestock for commercial purposes (in addition to other purposes). 8 Nutrition practices refer to ASF purchasing and consumption practices and intra-household dynamics that affect household ASF purchasing and consumption practices. 9 Private business grant recipients and GOR ministries are not included in Table 4 as changing their business or operational practices were a condition for receiving the grant. 8 Stakeholder Group9 Livestock/Business Practices Nutrition Practices SACCOs 3/6 N/A Banks/Microfinance institutions (MFIs) 1/3 N/A Community-Based Animal Health Workers (CAHWs) 2/2 1/2 Community volunteers 2/12 6/12 CBSGs 4/4 2/4 District officials 6/14 2/14 Total minus district officials 30/53 16/30 Total 36/67 18/4410 Participating in Orora Wihaze’s activities is contributing to improved livestock raising and business practices among intervention participants.11 The persistence of these findings across all four MTE districts indicates that they are more than a collection of anecdotes. To understand what kinds of changes in livestock raising and business practices Orora Wihaze participants are making, an illustrative sample of quotes from the KIIs and FGDs is presented below. “Before, I did not know the value of raising small livestock. Orora Wihaze trained us to rear small livestock. Before, we kept large animals, like cows. Now I have 15 hens. I have pigs.” - CBSG in Kayonza “We are doing farming professionally, and we have increased pig productivity. We do not mix pigs from the outside [i.e., do not mix breeds]. Now we are using meat, fish, and eggs. We are increasing our revenue and we have increased nutrition at home due to training from Orora Wihaze.” - Cooperative in Kayonza “We observed an improvement in terms of the number of women and youth who become members of the SACCO due to the Orora Wihaze training. The training received has changed our mindset, and our appetite for these value chains has increased and we are doing more mobilization. We have been convinced that poultry, pig, goats, and sheep are profitable when farmers do it professionally.” - SACCO in Gakenke “The capacity building we received from Orora Wihaze has changed how we are doing business. We have decided to sell quality feeds, which help farmers to increase production and productivity.” - Agro-dealer in Nyamasheke “After the training, our group members are saving and borrowing to buy small livestock, especially chicken and pigs. We are planning to put in place a common farm model and shelter where members could come to apply farming best practices.” - Small livestock producers in Nyamagabe A smaller share of key informants reported changes in nutrition practices than livestock raising or business practices (18 of 44) 41 percent but still enough to indicate an emerging pattern. If we remove the second-hand observations from the results and focus only on first-hand 10 The number of key informants reporting on nutrition practices is less than the number reporting on livestock raising and business practices as questions on nutrition practices were not relevant to several of the key informants, for example, agro-dealers, agro-vets, SACCOs, etc. 11 There could be other factors contributing to these outcomes, or they could be the result of random chance. Due to the limitations of the MTE methodology, these explanations cannot be ruled out. 9 reports, the share of key informants reporting changes in nutrition practices increases to 53 percent spread across all four districts examined by the MTE, again suggestive of a more generalized phenomenon. To understand what kinds of changes in nutrition practices Orora Wihaze participants are making, an illustrative sample of quotes from the KIIs and FGDs is presented below. “In the past, we did not cook animal-sourced foods. After the training from Orora Wihaze, when we are pregnant, we now eat [animal] products.” - Community members in Gakenke “We received training on animal consumption through Orora Wihaze support. We work with the village nutrition school to tell our neighbors what we learned. We got goat sheds, we have increased the insemination rate, and we know how to get medicine after training. Before the training, we used the same generations. Now we know breeders for insemination.” - Cooperative members in Nyamasheke “We are now using eggs to nourish our kids while we were selling it before. We now eat small fish as a nutrient source while we were not consuming it before. The training we received will help group members do small livestock farming professionally. We are now feeding our kids with a balanced diet.” - Community volunteers in Nyamagabe “I see the following general changes: 1) animal-sourced food consumption is increasing, for example, households are eating more eggs while at the same time income is not increasing; 2) households are asking vendors to sell them small portions; before they were shy to make such requests; 3) fathers used to eat meat while out (along with beer), but now they are discussing this with wives, who are being more assertive.” - National-level partner 4.2 CONCLUSIONS: EQ 1 Partnerships established between Orora Wihaze and market actors have generated numerous examples of improved livestock raising, business, and nutrition practices among small livestock producers, private market actors, and small livestock producer and non-producer households. While the evidence for improved livestock raising, business, and nutrition practices is found in all districts and all value chains examined by the MTE, it is still too early to conclude whether these emerging trends constitute evidence of broad-based system change. 4.3 FINDINGS: EQ 2 Is the activity demonstrating intended outcomes? Are households and businesses making more money, are markets transformed, are households consuming more ASFs? Key informants in all stakeholder categories and at multiple levels in the fish, pig, and poultry value chains who were interviewed for the MTE reported improved business or nutrition outcomes due to changes they made to their livestock raising, business, or nutrition practices. This includes Orora Wihaze private sector partners who changed their business models with Orora Wihaze grant support, other private sector actors (i.e., small livestock producers, agro-dealers, agro-vets, cooperatives, SACCOs), livestock producer households, and non-producer households. Among Orora Wihaze’s business grantees, the OX small livestock inputs distribution intervention, which began as a last-mile delivery service of animal feed to small livestock producers in Nyamasheke and Rutsiro, has significantly increased the volume of feed transported to underserved areas of Nyamasheke and Rutsiro and is now expanding to Nyamagabe and Ngororero. According to OX, the productivity of the company's services has considerably increased with the support of Orora Wihaze. The company started with one car, then two, and now it is operating 20 vehicles, including four 15-ton trucks and 16 small vehicles. Since April 2021, OX has delivered 382 metric tons of animal feed from Kigali to over 80 agro￾dealers in Nyamasheke and in the process increasing sales, driving down the costs of inputs to farmers (from 30 RWF to around 15 RWF according to one key informant), and increasing employment from five 10 to 15 full-time staff, around half of whom are women. In addition to expanding its area of operations, OX is diversifying into cold chain transport. While GACOPROCO has only recently received a grant, Orora Wihaze’s leveraged its field presence to connect the cooperative to a supply network and local government entities. Since Orora Wihaze helped facilitate GACOPROCO to set up an egg collection center in Gakenke, it has increased egg collections and sales from around 500 trays (15,000 eggs) a week to 6,000 trays a week (180,000 eggs) sourced from 130 small egg producers in the district. GACOPROCO customers include Early Childhood Development (ECD) Centers, primary schools, and secondary schools, thus reaching large numbers of food-insecure children in the district. At the district and community levels, Table 5 shows the number and share of key informants who self￾reported improved livestock raising, business, or nutrition outcomes after receiving Orora Wihaze support. (Self-reported improved nutrition outcomes include increased consumption of ASF, increased dietary diversity or consumption of a minimum diet, reduced malnutrition, reduced stunting, and reduced wasting.) Readers will note that the number and percentage of key informants reporting improved outcomes are smaller than the number and percentage reporting changes in practices. This is to be expected; changes in practices precede outcome improvements, improved outcomes lag practice changes, and not every practice change produces an improved outcome. With that said, 42 percent of key informants interviewed covering all stakeholder groups and all MTE districts reported improved livestock raising or business outcomes after receiving Orora Wihaze assistance. If the second-hand observations of district officials are removed, the share reporting improved livestock raising or business outcomes increases marginally to 44 percent. Table 5. Number and Share of Key Informants Reporting Improved Livestock Raising, Business, and Nutrition Outcomes Stakeholder Group Livestock/ Business Outcomes Nutrition Outcomes Agro-dealers/agro-vets 9/14 N/A Cooperatives 5/6 3/6 Farmer groups 4/6 3/6 SACCOs 1/6 N/A Banks/MFIs 1/3 N/A CAHWs 2/2 1/2 Community volunteers 1/12 4/12 CBSGs 1/4 1/4 District officials 5/14 2/14 Total minus district officials 23/53 12/30 Total 28/67 14/44 Below is a sample of quotes taken from the KIIs and FGDs that illustrate the types of livestock raising and business outcomes reported by key informants. “The mindset changed, especially in pig farming. In fact, the Eastern province has a reputation for cow farming. But currently, producers are raising pigs at the household level. Farmers know that pigs are more productive and generate more income.” - District official in Kayonza 11 “I have increased sales and revenues due to the training, material, and equipment received from the project. My business will be sustainable because we have gained knowledge, which helped to improve how we do business. Customers are increasing and shop revenues are also increasing.” -Agro-dealer in Kayonza “The material and equipment received have changed how the slaughter is done. We have increased the number of clients, and so we have increased revenues.” - Cooperative members in Gakenke “Egg collection is now operating; customers find easily where to buy eggs. Improved races are available for pigs and chicks. Modern equipment has been available like incubators to improve and disseminate improved poultry breeds.” - District official in Gakenke “Before the project, animal feed sold was less than a ton a week. Today we trade around 10 tons per week. Orora Wihaze supported the linkage with Ox, and the price transport of inputs was reduced.” - Agro-dealer in Nyamasheke “Agrodealers are well organized and receive quality inputs on time. They are increasing their sales, and farmers are increasing productivity due to the quality of feeds.” - District official in Nyamasheke “They linked industrial feed and farmers. Animal production and nutrition have increased. There has been an increase in meat from small livestock, especially chickens and pig.” - District official in Nyamagabe “We have trained five veterinarians, and they have trained 50 CAHWs, who will form two groups of farmers of 30 persons each. These farmers will be our clients. Our sales have already increased.” - Agro-vet in Nyamagabe “We were trained in goat, pig, fish, and poultry farming. Our group is linked to SACCO Dukire Kabarondo, and we got a loan of 500,000 RWF. We already paid it back and have applied for a second loan for 1,500,000 RWF at the same SACCO.” - CBSG in Kayonza “Progressively our number of clients has increased. We have started to lend. We have also increased the effort we put in this sector to mobilize more clients. The training received has changed our mindset and our appetite for these value chains has increased and we are doing more mobilization. We have been convinced that poultry, pig, goats, and sheep are profitable when farmers do it professionally.” - SACCO in Nyamasheke “Many of the clients were there before the training. However, the number of clients is progressively increasing due to the training and mobilization. Clients are also increasing the number of loans requested.” - Bank in Gakenke One-third of key informants reported improved nutrition outcomes. This figure increases to 40 percent if we eliminate the second-hand observations by district officials and only count first-hand reports. Like the livestock raising and business outcomes, reported improvements in nutrition outcomes are found across all stakeholder groups and in each of the four MTE districts. The following quotes provide an illustrative sample of the types of nutrition outcomes reported by key informants. “We have changed and improved our nutrition system; we buy and consume small fish regularly, and this was not the case before. We are now eating a balanced diet using eggs and milk.” - Goat farmers in Kayonza “Before the training, we had 25 under five years with malnutrition where now we have only five cases.” -RICH volunteers in Gakenke 12 “Orora Wihaze supported the beneficiaries to reduce malnutrition. Beneficiaries are aware that animal and milk consumption is important to reduce malnutrition; it is useful to have a balanced diet. Nutritional village kitchen demonstrations were a good model to teach people animal foods are available that include eggs and a small fish named Isambaza." - RICH volunteers in Nyamasheke “We have improved the consumption of animal-sourced food. We are now consuming eggs, chicken meats, and small fish thanks to revenues from our poultry farming.” - Poultry group members in Nyamagabe As a final example, RICH volunteers in Nyamasheke participating in an FGD told the evaluation team that cases of wasting among the children of the households they visited had diminished significantly since their community began working with Orora Wihaze on nutrition messaging. Whereas the number of wasting cases among their assigned households averaged 11 per volunteer when they started to work with Orora Wihaze (Table 6), the number had fallen to an average of only 0.5 per volunteer by the time of the MTE. Of course, several factors may help explain this outcome, but the general takeaway (i.e., improved household nutrition outcomes) is consistent with other evidence gathered for the MTE and cited above. Table 6. Changes in the Number of Wasting Cases in Nyamasheke as Reported by FGD Participants FGD Participant Number of Wasting Cases before Orora Wihaze Number of Wasting Cases at the MTE 1 23 2 2 27 0 3 0 0 4 27 0 5 3 1 6 7 0 7 3 1 8 7 0 9 7 0 10 6 1 4.4 CONCLUSIONS: EQ 2 Partnerships established between Orora Wihaze and market actors have generated numerous examples of improved livestock raising, business, and nutrition outcomes among small livestock producers, private market actors, and small livestock producer and non-producer households. These include, for example, increased livestock production, sales, and profits; increased input availability, lower input costs, and increased input sales; increased ASF availability in local markets; and increased ASF consumption. While the evidence for improved livestock raising, business, and nutrition outcomes is found in all districts and all value chains examined by the MTE, it is still too early to conclude whether these emerging trends constitute evidence of broad-based system change. 4.5 FINDINGS: EQ 3 Are there unintended outcomes generated by the activity interventions? The evaluation team found little evidence for unintended outcomes. Across the 100+ KIIs and FGDs completed by the evaluation team, with few exceptions, key informants did not/could not cite examples of unintended outcomes, whether good or bad, generated by Orora Wihaze’s interventions. One exception was the Afrisol poultry outgrower intervention in Kayonza. Afrisol is a Rwandan feed manufacturer operating a 10-ton per hour capacity feed mill working with poultry and pig farmers 13 throughout Rwanda. Orora Wihaze supported Afrisol to pilot a poultry outgrower model in Kayonza District with ten poultry producers in which Afrisol would sell inputs to the producers and purchase their poultry production at an agreed price, slaughter it, and sell it to private buyers. Joining Afrisol as a partner to provide poultry extension services was One Acre Fund (OAF), a global non-profit social enterprise. Ten farmers participated in the Phase 1 pilot, and the grant was later expanded to 84 during farmers during Phase II who produced 30,030 birds. During Phase II, however, market distortions created by the COVID-19 pandemic produced an oversupply of chickens on the market, which negatively impacted prices, left the market saturated for an extended period, and compelled AfriSol to sell its remaining stock at a lower price. At the end of Phase II, AfriSol elected to scale back the outgrower model to pursue more favorable ventures in the animal-sourced foods sector. This disappointing result, however, led to a surprising twist. Notwithstanding the unanticipated cessation of the poultry outgrower model, OAF saw the potential of the poultry outgrower model and is, as a result, currently developing its plans to replicate and scale up the poultry outgrower scheme drawing on the lessons learned from the Afrisol pilot, albeit potentially outside Orora Wihaze’s eight districts. Another exception was the OX small livestock inputs distribution intervention in Nyamasheke and Rutsiro. As described under the findings for EQ 1, the OX intervention has so far produced impressive results. It was the speed with which these outcomes occurred that key informants familiar with the OX intervention found surprising. 4.6 CONCLUSIONS: EQ 3 To date, the outcomes generated by Orora Wihaze’s interventions have been consistent with those predicted in the interventions’ theory of change. In the two cases where key informants cited unexpected outcomes, Afrisol and OX, the outcomes themselves were not unexpected, but the manner in which they occurred. In Afrisol’s case, the replication and scale-up of the poultry out-grower scheme were predicted by the intervention’s theory of change; what was not expected was that OAF is poised to replicate and scale up the model, not Afrisol. In OX’s case, the unexpected outcome was the speed by which the predicted outcomes occurred, not that they occurred. 4.7 FINDINGS: EQ 4 What are the drivers for differential performance of targeted districts?12 Districts differ in several features that are potentially relevant to explaining the differential performance across the eight Orora Wihaze districts. One feature is the overall business climate in the district, referring to the level and dynamism of the business sector in general, the entrepreneurial character of district residents, natural resource endowments, economic infrastructure, etc. For example, districts located on the border with the Democratic Republic of Congo (DRC) (i.e., Nyamasheke and Rutsiro) tend to have more dynamic business climates facilitated by extensive formal and informal trading networks with the DRC. A second feature is the prevalence and strength of the specific animal value chains targeted by Orora Wihaze. For example, districts that border on lakes (i.e., Nyamasheke, Rutsiro, and Burera) or benefit from abundant fishponds (i.e., Kayonza and Ngoma) tend to be stronger in the fish value chains (see Table 7). Other districts have stronger cultural traditions, and thus a comparative advantage, in raising shoats (i.e., Burera, Kayonza, and Ngoma) or pigs (Gakenke, Ngoma, Nyamagabe, and Nyamasheke). Finally, other districts tend to be stronger in poultry production (i.e., Gakenke, Burera, 12 The analysis in Section 4.7 focuses on Orora Wihaze grant agreements and MOUs targeted to a specific value chain (i.e., fish, pig, or poultry). Whereas Orora Wihaze also implements numerous activities that cut across all value chains and districts (as seen in Table 8), the fact that they cut across all value chains and districts implies that considerations such as the overall business climate, comparative advantage, or the availability of quality partners do not influence the district in which Orora Wihaze chooses to implement them. 14 Kayonza, Ngoma, and Rutsiro); however, poultry is the value chain with the lowest entry barriers and has the most prevalent production throughout Orora Wihaze’s eight districts. Table 7. Ranking the Strength of Districts in Orora Wihaze Value Chains District Pig Poultry Fish Shoats Burera 3 2 2 1 Gakenke 1 1 3 3 Kayonza 3 2 2 1 Ngoma 2 2 2 2 Ngororero 3 3 3 3 Nyamagabe 2 3 3 3 Nyamasheke 2 3 1 3 Rutsiro 3 2 1 3 Source: KIIs with Orora Wihaze 1=Best; 2=Medium; 3=Worst A third feature is the quality of potential partnerships within the districts. Orora Wihaze’s intervention strategy explicitly targets partners who are willing and able to test new business models that unlock learning opportunities and new pathways for an inclusive business model (i.e., change agents) and/or partners that mobilize, support, align, or coordinate the efforts of individuals, firms, and institutions system wide(i.e., scale agents).13 Within each district, there are a limited number of market actors working in the targeted value chains who both fit this description and who want to work with Orora Wihaze. Potential partners are identified through Orora Wihaze’s ongoing interactions with national, district, and community market actors or via a Request for Application (RFA) process. In either case, Orora Wihaze negotiates an agreement with the partner to provide it grant or other in-kind support (e.g., capacity building) with the agreement and its terms formalized in an Activity Design Document or Grant Negotiation Memo. To date, Orora Wihaze has awarded nine grants to business partners, two of which were to women-owned businesses. A district’s overall business climate does not explain Orora Wihaze’s business partnering strategy while a district’s animal production pattern does (to an extent.) Table 8 lists all the Orora Wihaze activities in its activity pipeline as of the time of the MTE. The activities can be loosely grouped into three categories: 1) business grants, which are implemented in a subset of districts in conjunction with Orora Wihaze business partners to increase the availability and accessibility of ASF within target value chains (indicated by ✓ Table 8); 2) enabling environment activities, which are implemented in all eight Orora Wihaze districts (except the International Livestock Research Institute) to create a more supportive environment facilitating the increased availability, accessibility, and consumption of ASF (indicated by • in Table 8); and 3) group-based activities, which are implemented in all eight Orora Wihaze districts and work through existing community-based group structures (i.e., CBSGs and cooperatives) to increase the production and consumption of ASF in producer and non-producer households (indicated by ◘ in Table 8). The first two categories of interventions cut across objectives and interventions whereas the third category is limited to intervention 6.2.1. 13 A partner may be a change agent, scale agent, or both. Orora Wihaze’s strategy stipulated that during first three years of the activity, it would target more change agents to see what potential models may be feasible to impact the market system. Then in Years 4 and 5, it would focus on scale agents to bring these ideas to reach more participants. 15 For interventions falling in Category 1, the evaluation team asked members of the Orora Wihaze management team to rank the general business climate in the eight districts on a scale of 1-3, with 1 being ‘best’ and 3 being ‘worst. The numbers in parentheses next to the districts in Table 7 shows the results: Gakenke and Kayonza have the best business climates; Burera, Nyamgabe, and Ngororero have the worst business climates; and Ngoma, Nyamasheke, and Rutsiro have average business climates. Considering this, Table 7 provides clues as to how district differences affected Orora Wihaze’s intervention strategy. Gakenke was consistently cited by key informants as the district with the most dynamic business sector. Not surprisingly, it also has the largest number of grant partners. Gakenke also has a comparative advantage in the pig and poultry value chains mainly due to its proximity to the Kigali city market and the DRC Goma city market. On the other hand, Kayonza only has a single grant partner, this one in the poultry value chain in which Kayonza has a comparative advantage. Among the districts ranked lowest in terms of business climate, Nyamagabe has two current business partners (the OX input distribution intervention is scheduled to be scaled up there during its Phase 2 implementation), one in the pig value chain, where it possesses a comparative advantage, and one in the fish value chain, despite the absence of lakes or fishponds. Burera has two grant partners, one in the fish value chain, where it has a comparative advantage, and one in the pig value chain where it does not. Finally, Ngororero has no current grant partners, although it is scheduled to participate in Phase 2 of the OX intervention. Among the districts with an average business climate, Ngoma and Nyamasheke each has a business partner in the fish value chain where each possesses a comparative advantage (Nyamasheke due to its proximity to Lake Kivu and Ngoma due to the presence of lakes and fishponds, whereas Nyamasheke and Rutsiro also participate in Phase 1 of the OX intervention. Finally, Rutsiro also has a business partner in the poultry value chain. Although poultry production is relatively low in the Western province, poultry is the one value chain that is most consistently present across the activity districts, poses the lowest barriers to entry, and is most capable of being scaled up. The distribution of business grants across districts shows no clear pattern. The strength of the overall business climate does not appear to be driving partner selection (except possibly Gakenke). A district’s comparative advantage in a particular value chain helps explain partner selection (i.e., partnerships generally align with comparative advantage in the districts), albeit with notable exceptions. Even then, possession of comparative advantage in a particular value chain does not imply a lack of demand for the relevant ASF products. Market opportunities can still exist if demand is not matched by availability and accessible supply (as is the case for Hope and Fine Fish in Nyamagabe, see below). District differences do not explain Orora Wihaze’s partnering strategy for Category 2 and 3 activities, nor do they appear to influence intervention performance. For activities in Categories 2 and 3, district differences did not figure in Orora Wihaze’s intervention strategy in that Orora Wihaze works through the same partner in each district implementing a largely standardized package of activities. When asked how differences across Orora Wihaze districts might influence activity implementation or results, Category 2 and 3 partners overwhelmingly replied that district differences had no material influence. Where key informants did cite differences that might have some influence, they inevitably cited different animal production patterns across districts, although even in this case, they mostly did not believe animal production differences affected either intervention implementation or its results. 16 Table 8. Orora Wihaze Activity Pipeline N S E W Activities Gakenke (1) Burera* (3) Nyamgabe (3) Kayonza (1) Ngoma* (2) Nyamasheke (2) Rutsiro* (2) Ngororero* (3) 1.1.1: AfriSol poultry out-grower model ✓ 1.1.2: COPIKI and Frefish Fingerlings and Feeds* ✓ 1.1.3: OX small livestock inputs distribution ✓2 ✓1 ✓1 ✓2 1.1.5: Last Mile Pig artificial insemination* ✓ ✓ 1.1.6: Fish hatcheries certification and professionalization • • • • • • • • 1.1.7: Establishment of value chain platforms • • • • • • • • 1.1.8: International Livestock Research Institute • • 1.2.1: RCVD extension services • • • • • • • • 1.2.2: Facilitate and implement livestock training at the district level • • • • • • • • 2.1.1: UVEPO Fish Aggregation* ✓ 2.1.2: A-Z Pig Ltd. pork product business ✓ ✓ 2.1.3: Cooperative Indyonziza pig aggregator and processing* ✓ 2.1.4: GACOPROCO egg collection center ✓ 3.1.1: MoU with Vision Fund Rwanda • • • • • • • • 3.1.2: Tripartite MoU with Nguriza Nshore and AMIR • • • • • • • • 3.1.3: Linkage meetings with banks, MFIs, and ASF market actors • • • • • • • • 4.1.1: National Child Development Agency Nutrition Policy • • • • • • • • 4.1.2: RICH ASF consumption messaging • • • • • • • • 4.1.3: Targeted radio programming to enhance ASF messaging • • • • • • • • 4.2.2: Viamo ICT for the development of ASF consumption messaging • • • • • • • • 4.2.3: Urunana dissemination of ASF messaging • • • • • • • • 5.1.1: JFILEWO poultry products* ✓ 5.1.2: Triumvirate Food Co. Ltd. fish products ✓ 5.2.2: Hope and Fine Fish Products ✓ 5.2.3: MoU wit RICA • • • • • • • • 6.2.1: CBSGs/ABIBM ◘ ◘ ◘ ◘ ◘ ◘ ◘ ◘ 6.2.2: MoU with RYAF • • • • • • • • *Districts and activities not covered by the MTE ✓1Ox Phase 1 ✓2Ox Phase 2 The primary factor influencing Orora Wihaze’s intervention strategy and results is the quality of the partnership in each district. Orora Wihaze is looking to partner with change agents and/or scale agents. Take for example the Hope and Fine Fish intervention in Nyamagabe. For five years the grant recipient has been purchasing fish in Nyamasheke and Karongi districts and transporting the fish in plastic jerry cans to Nyamagabe using public transport. The grant is allowing her to formalize and expand her operations in a market with a limited supply of fish products relative to demand. To cite another example, the A-Z Pig intervention in Gakenke and Burera is expanding from pig trading to pig processing to produce and sell 17 pork products targeted to low-income consumers. Thus, in the absence of any clear factor driving differential performance across districts, and to the extent performance does differ, the findings point to the quality of partnerships as the primary determinant, not differences in district characteristics. 4.8 CONCLUSIONS: EQ 4 Differences in Orora Wihaze districts have not influenced either Orora Wihaze’s intervention strategy or the performance of its activities. Rather, the primary factor driving Orora Wihaze’s intervention strategy is the quality of partners found in its targeted districts, where quality is defined by such factors as the soundness of the partners’ business ideas and their capacity and motivation to implement those ideas and take them to scale and the partners’ commitment to serve Orora Wihaze’s target beneficiaries, whether as producers, suppliers, buyers, service providers, sellers, etc. There is no evidence to date suggesting that the results of those activities have been materially influenced by district differences. 4.9 FINDINGS: EQ 5 Is the Orora Wihaze MSD approach still sufficiently effective and adaptive to achieve nutrition outcomes and a profitable market in the current context? Orora Wihaze’s MSD approach incorporates several features designed to enhance its effectiveness and adaptability and ensure its relevance to the current content in Rwanda. These features include 1) close alignment with GOR goals; 2) in-depth market research to understand the nature of the constraints facing the ASF sector generally and targeted value chains; 3) active engagement with local market actors; 4) careful selection of partners; 5) integration with established networks; and 6) targeting of multiple constraints at multiple levels. Each of these features is discussed below. Close Alignment with GOR Goals The emphasis on ASF consumption as a strategy to improve nutrition, especially among women and children, is featured in GOR policy documents. These documents include, for example, the Rwanda Nutrition Policy (Ministry of Gender and Family Promotion), National Food and Nutrition Policy (Ministry of Local Government, Ministry of Health, and Ministry of Agriculture and Animal Resources), and the National Nutrition Strategy (National Child Development Agency).14 Quoting from the Rwanda Nutrition Policy (p. 23), “The availability and consumption of ASF is a high priority of this policy as it has the potential to greatly decrease stunting levels. A focus will be put on strengthening the role of young people in ASF production and nurturing positive attitudes towards ASF consumption through behavioral change. Women’s empowerment interventions, including supporting them to gain greater control over household resources (such as deciding whether to consume their own production or prioritize ASF purchasing) and to access loans (to establish or expand economic and agricultural businesses) will be vital.” The themes mentioned in this quote evoke different features of the Orora Wihaze MSD approach: ASF availability and consumption, youth, women, women’s empowerment, behavioral change, and access to loans. The close alignment of Orora Wihaze’s purpose, objectives, and intervention strategy help ensure its relevance within the framework of GOR policy aims. 14 In 2020, Orora Wihaze provided inputs to the National Child Development Agency in drafting its National Nutrition Strategy wherein the agency adopted ASF as a key priority. 18 In-Depth Market Research Orora Wihaze undertakes extensive market research to understand the factors affecting ASF markets and its targeted value chains. This research falls into five categories. The first category is general market research, which is market research to understand ASF market characteristics more generally across and within targeted value chains, including constraints, supply and demand characteristics, attitudes, perceptions, etc. Examples include its Consumption Study and Gender and Social Inclusion Strategy and its Market Systems Analyses for the poultry, sheep and goats, pig, and fish value chains. The second category is intervention-related market research, which is research undertaken to support each of its activities. This research is summarized in the Activity Design Documents and Negotiation Memos associated with each intervention. The third category of market research is research conducted as part of the activity’s MEL Plan, which includes its annual household survey and internal evaluations. The fourth category is research done for learning purposes. We have already undertaken studies for Afrisol and CBSGs. Findings from this research are used to inform future interventions or generalized learning about the success of the business models/innovations to be disseminated to other market actors. The fifth category is market research to assist businesses to make decisions around product packaging, pricing, and positioning based on consumer/user feedback. Since its inception, Orora Wihaze has generated large amounts of data about the ASF market systems in Rwanda, much of which is reflected both in its overall MSD approach and in its individual activities. Active Engagement with Local Market Actors Orora Wihaze field staff actively engage with local market system actors both through their routine work and through producer platforms. Orora Wihaze has a District Portfolio Manager and Nutrition Advisor, each of whom covers two districts, who are responsible for engaging and supporting local market actors and communicating district-level intelligence back to the home office in Kigali. Activity field staff also support 24 district-level platforms (eight poultry, eight pig, two goats, and eight agro-dealer) and 138 sector-level platforms in the poultry, pig, and goat value chains across its eight districts (Table 9). Table 9. District and Sector-Level Producer Platforms Supported by Orora Wihaze District District Platforms Number Committee Members Number Sector Platforms Number Committee Members Gakenke Poultry 5 19 95 Pig 5 19 95 Agro-dealer 5 0 0 Kayonza Poultry 5 4 20 Pig 5 4 20 Goat 5 3 15 Agro-dealer 5 0 0 Ngoma Poultry 5 5 25 Pig 5 5 25 Goat 5 3 15 Agro-dealer 5 0 0 Burera Poultry 5 15 85 Pig 5 15 85 Agro-dealer 5 0 0 19 District District Platforms Number Committee Members Number Sector Platforms Number Committee Members Nyamagabe Poultry 0 6 18 Pig 0 6 18 Agro-dealer ?15 0 0 Nyamasheke Poultry 0 5 15 Pig 0 6 18 Agro-dealer 3 3 6 Ngororero Poultry 5 5 25 Pig 5 5 25 Agro-dealer 5 0 0 Rutsiro Poultry 5 5 20 Pig 5 5 20 Agro-dealer 5 0 0 The producer platforms create a formal structure at the district and sector levels whereby market actors working in different value chains come together to discuss issues affecting them and to advocate for reform. As such, they constitute a mechanism both for collective action among platform members and for Orora Wihaze to engage platform members to facilitate activity implementation. For example, the poultry and pig platforms in Gakenke facilitate information exchanges between 240 poultry and pig farmers and other market actors, such as agro-dealers who use the platforms to find customers. As a result of its active local engagement, Orora Wihaze is well-known among local market actors who uniformly spoke positively about their engagement with the activity, as illustrated by the following quotes: “Orora Wihaze is targeting farmers involved in small livestock production, including poultry, goats, sheep, and fish. Its support to farmers to create the pig and poultry platforms in the Gakenke District is well received. The Joint Action District Forum in Gakenke has had a good experience with Orora Wihaze.” - District Official in Gakenke “The way [Orora Wihaze] activities are implemented ensures the sustainability of the project. Activities are planned in collaboration with the district and these activities are in line with the district plan.” - District Official in Nyamasheke “Orora Wihaze is collaborating well with the district. We plan project activities together, and we follow up the implementation of project activities.” – District Official in Nyamgabe “We work with Orora Wihaze on malnutrition. It is on the [District] Sector Malnutrition Program Committee. - District Official in Kayonza Careful Section of Partners Orora Wihaze conducts extensive due diligence in selecting its business grant partners, looking for a change and/or scale agents with the potential to test and/or scale innovative business models. Partner selection is informed by the activity’s research on the relevant market system and the partner’s capacity as demonstrated through a rigorous application process. (The partner selection process is discussed in greater depth in response to EQ 6 in Section 4.10.) 15 Nyamgabe has established an agro-dealer platform but has yet to select its committee members. 20 Integration with Established Networks and Institutions Orora Wihaze strategically grafts its activities into established and trusted networks for delivering technical assistance (TA), extension services, and social and behavior change communication (SBCC) messaging. These include networks of CAHWs, RICH volunteers, Urunana listeners, mobile phone users, CBSG members, and cooperative members. Working through existing networks like the above offers several important advantages. First, it is cost-effective. For a reasonable cost, the activity can reach tens of thousands of households. Second, piggybacking on the existing network lends credibility to Orora Wihaze and its messages. Third, Orora Wihaze’s network partners share Orora Wihaze’s commitment to strengthening the ASF sector, whether to support small producers and other market actors or to improve nutrition outcomes. Their partnership with Orora Wihaze in turn offers them the opportunity to pursue their own objectives and do so more effectively and at scale, as the following quotes illustrate. Fourth, as these are preexisting networks with a history of delivering SBCC messaging, they can continue delivering messaging around ASF availability, accessibility, and consumption potentially long after Orora Wihaze ends, thus contributing to the sustainability of ASF messaging activities. “SBCC is the key activity for RICH. Around 96 percent of Rwandans identify as a member of a religious organization, so this is an effective way to reach out and use existing structures and existing volunteer networks that are already doing SBCC messaging on health and nutrition. This is cost-effective, easier to coordinate, and generates useful feedback due to the existing structure. RICH’s role is to provide the platform, using existing networks for ASF messaging. It provides trained volunteers who are experienced and provides them with key messages to communicate.” - RICH participant “Urunana is not doing anything particularly new under Orora Wihaze. What is different is the ASF focus and content. Urunana is interested in ASF as it is consistent with its focus on reproductive health and female empowerment. It recognizes that ASF is a key component contributing to the welfare of its target audience.” -Nutrition messaging participant Targeting of Multiple Constraints at Multiple Levels Orora Wihaze’s activities typically address multiple constraints within the target value chains and at multiple levels of the value chains. (This is discussed at length in Section 4.11 below.) To demonstrate, Table 10 shows how the Hope and Fine Fish intervention in Nyamagabe, A-Z Pig intervention in Gakenke and Burera, and GACOPROCO intervention in Gakenke are addressing several constraints at several levels within the fish, pig, and poultry value chains. Table 10. Details of the Hope and Fine Fish, A-Z Pig, and GACOPROCO Activities Constraint Hope and Fine Fish A-Z Pig Production Hope and Fine Fish has contracts with three cooperatives in Nyamasheke to supply fresh fish (Isambaza), which will also help the cooperatives reduce post￾harvest losses. Additionally, Hope and Fine Fish has a formal contract with Inter Alpha Choice to supply fresh tilapia fish from Kigali. A-Z Pig sources pigs from Indyo Nziza Cooperative, which operates a slaughterhouse in Gakenke. A￾Z Pig aggregates pigs from various pig producers in Gakenke. GACOPROCO is increasing egg purchases from 4,000 trays a week sourced from 95 small egg producers to 8,000 trays a week sourced from 180 small egg producers. 21 Constraint Hope and Fine Fish A-Z Pig Transport Hope and Fine Fish uses a motor van to distribute fresh tilapia and Isambaza to four dispersed selling points. This helps prevent spoilage and enables faster responses to supply demand at each location N/A GACOPROCO is investing in motorized tricycles to transport eggs thereby reducing the number of eggs damaged in transport. Processing Hope and Fine Fish uses a fish cutting machine to make small pieces of fish readily available and affordable on the market. Storage facilities have improved equipment, including an ice￾making machine, plastic crates, and additional refrigerators. A-Z has a new pig processing facility equipped with processing equipment purchased with the Orora Wihaze grant. GACOPROCO has established an egg collection center that eliminates the middlemen and allows egg producers to capture a higher return for their eggs. Retail Sales Hope and Fine Fish is expanding to two additional selling points in the Nyamagabe district making a total of four selling points where it sells Isambaza and smaller fish pieces targeted to low-income consumers. A-Z Pig sells pig products targeted through retail shops and kiosks, restaurants, and supermarkets in rural areas. It targets low-income consumers by offering affordable units or smaller packages ranging from 100 grams to 1 kilogram. GACOPROCO is establishing selling points targeted to households with children under two years old, pregnant, and breastfeeding mothers, and youth and is selling eggs to schools for school feeding programs and ECD centers. Retail Distribution Hope and Fine Fish is hiring four marketing agents to oversee the distribution of fish/Isambaza to customers using bicycles to reach different rural markets including schools and small centers. The number of women who will sell Isambaza and tilapia fish directly to households.is planned to increase to 30. A-Z Pig is engaging 20 agents to sell pig products in the rural markets of Musanze, Burera, and Gakenke. GACOPROCO is expanding egg sales by expanding distribution to three schools, nine ECDs, and two kiosks and by partnering with local market actors in five strategic centers of the Gakenke District to distribute eggs among the neighboring communities. Extension Services Hope and Fine Fish is linked to CBSGs and village nutrition education sessions on how to prepare meals with fish. Sessions are conducted twice a month, and Hope and Fine Fish is targeting different platforms for markets and health facilities during vaccination sessions, malnutrition screening sessions, and community gatherings. Pig producers in Gakenke and Burera benefit from improved extension services and improved access to production inputs and vet services from CAWHs, agro￾dealers, and agro-vets through Orora Wihaze’s partnership with RCVD. Small poultry farmers in Gakenke benefit from improved extension services and improved access to production inputs and vet services from CAWHs, agro-dealers, and agro-vets through Orora Wihaze’s partnership with RCVD. 22 Constraint Hope and Fine Fish A-Z Pig Enabling Environment Orora Wihaze is working with the Ministry of Agriculture and Animal Resources, Aquaculture (MINAGRI), the Technologies for Africa Agricultural Transformation project, and African Development Bank to develop SOPs for fish hatchery management that meet international standards. Orora Wihaze is working with RICA to increase the capacity of meat inspectors and the quality of meat products by encouraging/ enforcing compliance of meat processors with regulatory requirements and international quality standards. N/A SBCC The RICH, Urunana, and Viamo activities are communicating messages on fish preparation and consumption and intra￾household dietary decision￾making supported by local structures like ECD centers and village nutrition schools. The RICH, Urunana, and Viamo activities are communicating messages on pig consumption and intra￾household dietary decision￾making supported by local structures like ECD centers and village nutrition schools. The RICH, Urunana, and Viamo activities are communicating messages on egg consumption and intra-household dietary decision-making supported by local structures like ECD centers and village nutrition schools. 4.10 CONCLUSIONS: EQ 5 Orora Wihaze’s MSD approach remains an effective and adaptive approach to achieve nutrition outcomes and a profitable market in the current context. Several features of Orora Wihaze’s MSD approach contribute to this conclusion: its close alignment with GOR goals, its in-depth market research, its active engagement with local market actors, its careful selection of partners, its integration with established networks, and its targeting of multiple constraints at multiple levels. 4.11 FINDINGS: EQ 6 What are the constraints for achieving results? Market constraints for achieving activity results are numerous, exist at all levels in each of the four target value chains, and affect all aspects of food security/nutrition. A hallmark of the MSD approach is that it intervenes in a market system to address known constraints to improve system functioning so that it delivers improved benefits to target populations. in the form of higher returns to labor and capital for market actors and improved socio-economic welfare for small farmers (or another target) households manifested in the case of Orora Wihaze by a significant and sustainable improvement in the availability, accessibility, and consumption of animal-sourced foods and, ultimately, in improved nutritional outcomes in small livestock producer and non-producer households, especially among women and children. The evaluation team’s review of project documents and secondary information identified a long list of market constraints to ASF availability, accessibility, and consumption, numbering several dozens of constraints.16 Collapsing this long list into a smaller number of broad categories yields the set of constraints shown in Table 11. While each value chain has its share of unique market constraints, those listed in Table 11 are found in each of Orora Wihaze’s four value chains. The same documents do not 16 As part of each intervention, Orora Wihaze conducts market research to identify the market constraints relevant to the intervention and which the intervention is designed to address. For those interested in understanding the market constraints facing the ASF sector, Orora Wihaze Activity Design Documents and Negotiation Memos are useful sources. 23 provide a clear sense of which constraints are the most important, rather the importance of a constraint depends on the context and what the intervention is attempting to achieve. One or more Orora Wihaze activities address each of the primary market constraints. The second column of Table 11 shows which of the Orora Wihaze activities are addressing each of the constraints listed in Table 11. The asterisk (*) symbol in Table 11 (and Table 12) indicates a constraint that was also identified either as a core value chain constraint or value chain sub-constraint in Orora Wihaze’s 2021 strategy document.17 As indicated by a large number of asterisks, there exists a significant overlap between constraints identified by key informants and those identified by Orora Wihaze through its market research. Orora Wihaze’s activities address each of the constraints found in Table 11, and in most cases, each constraint is addressed by multiple activities and each intervention addresses multiple constraints. The challenge that an MSD activity like Orora Wihaze faces is extending further the breadth of constraints addressed and the depth to which it addresses them, both of which are a function of, among other things, resource and time constraints, the different market characteristics in different districts, and the availability of quality partners. This challenge is reflected in the following quote by a member of Orora Wihaze’s senior management team, “The first challenge is how deep you can go across districts because each district can be unique… and the types of investments are small, needing a lot of nurturing. he nature of the approach and the difference across districts and the depth to which you can go to attend to the specificity of each district, yeah, it’s quite a challenge.” Table 11. Summary of Commonly Mentioned ASF Market Constraints and Orora Wihaze Activities Addressing Them Constraints Activities18 Availability Constraints Low animal productivity* 1.1.1, 1.1.2, 1.1.5, 1.2.1, 1.2.2, 6.2.1, 6.2.2 Limited access to or use of quality inputs* 1.1.2, 1.1.3, 1.1.5, 1.1.6, 1.2.1 High cost of feeds and other inputs* 1.1.2, 1.1.1, 1.1.3, 1.1.5, 1.1.6 Limited access to vet services19* 1.1.5, 1.2.1 Limited access to extension services* 1.1.1, 1.1.2, 1.1.5, 1.2.1, 1.2.2 Low livestock/business capacity* 1.1.1, 1.1.2, 1.2.1, 1.2.2, 6.2.1, 6.2.2 Limited access to credit* 1.1.1, 1.1.2, 3.1.1, 3.1.2, 3.1.3, 6.2.1 High post-harvest losses 5.1.2, 5.2.2 Accessibility Constraints High cost/low affordability of ASF products* 2.1.1, 2.1.2, 2.1.3, 2.1.4, 5.1.1, 5.1.2, 5.2.2 Inappropriate product packaging20* 2.1.1, 2.1.2, 2.1.3, 5.1.2, 5.1.1, 5.2.2 17 Orora Wihaze. (2021.) “Feed the Future Orora Wihaze Market Systems and Behavior Change Strategy,” pp. 12- 15. Major constraints identified by key informants include all seven of the core constraints identified in the strategy document and 15 of the 22 unique sub-constraints. 18 This column lists the intervention numbers assigned by Orora Wihaze as shown in Table 11. Although each intervention is associated with a specific objective (e.g., increased production or increased end market access), in practice specific interventions can serve multiple objectives and intervention categories. 19 Related to the limited access to vet services are high animal morbidity and mortality rates and limited access to livestock insurance schemes. Orora Wihaze’s intervention with RCVD addresses both this constraint and the limited access to extension services. 20 ASF products are often sold in packages that are too large or too expensive for poor consumers and, given the lack of refrigeration, cannot be stored safely to prevent spoilage. 24 Constraints Activities18 Low product innovation21* 2.1.1, 2.1.2, 2.1.3, 5.1.2, 5.1.1, 5.2.2 Limited retail outlets in rural areas* 2.1.1, 2.1.2, 2.1.3, 2.1.4, 5.1.2, 5.2.2 Poor distribution infrastructure* 2.1.1, 2.1.2, 2.1.3, 2.1.4, 5.1.1, 5.2.1, 5.2.2 Poor product quality22 1.1.6, 2.1.1, 2.1.3, 2.1.4, 5.1.1, 5.1.2, 5.2.2, 5.2.3 Lack of standards/poor regulatory structure* 1.1.6, 5.2.3 Consumption Constraints No tradition of consuming ASF/Low demand*23 2.1.4, 4.1.1, 4.1.2, 4.1.3, 4.2.2, 4.2.3, 6.2.1 Social and cultural norms*24 2.1.4, 4.1.1, 4.1.2, 4.1.3, 4.2.2, 4.2.3, 6.2.1 Limited knowledge of ASF benefits* 1.1.2, 4.1.1, 4.1.2, 4.1.3, 4.2.2, 4.2.3, 6.2.1 Decision-making gender imbalance* 4.1.1, 4.1.2, 4.1.3, 4.2.2, 4.2.3, 6.2.1 Inequitable household consumption* 4.1.1, 4.1.2, 4.1.3, 4.2.2, 4.2.3, 6.2.1 Generic nutrition messaging* 2.1.4, 4.1.1, 4.1.2, 4.1.3, 4.2.2, 4.2.3, 6.2.1 Low message coordination* 4.1.1, 4.1.2, 4.1.3, 4.2.2, 4.2.3, 6.2.1 Market constraints identified by key informants focus on constraints to ASF availability, particularly in terms of the unavailability and high cost of inputs, low producer capacity, and lack of access to credit. In KIIs and FGDs, the evaluation team asked key informants to identify— unprompted— what they considered the most important constraints facing the ASF sector generally or the ASF value chain in which they worked. Their responses, as measured by the number of mentions occurring during KIIs and FGDs, are shown in Table 12. As seen there, primary constraints from the perspective of interviewed market actors are the lack of extension services, lack of access to inputs together with the high cost of inputs (particularly animal feed), lack of access to credit, and low capacity of animal producers. However, ranking the severity of constraints is not straightforward because constraints are often mutually dependent. For example, the unavailability or high cost of inputs is affected by the lack of transport; low animal productivity is affected by low producer capacity; limited access to extension services, inputs, credit, vet services, and animal insurance; and high animal morbidity/mortality is affected by the lack of access to inputs, extension services, and vet services and low producer capacity. With that said, two key takeaways emerge from Table 12. First, if we consider lack of extension services and lack of producer capacity to be different dimensions of the same underlying constraint (i.e., low producer capacity), key informants appear most concerned with three constraints: input unavailability/cost, low producer capacity, and lack of access to credit. Second, constraints identified by key informants focus almost exclusively on constraints to ASF availability; key informants’ unprompted responses to this question rarely identified constraints related to ASF accessibility or consumption. This last takeaway is not surprising given the history of government and donor market activities, which have traditionally focused on production (availability). One of the critical insights of the MSD approach is that without a corresponding focus on accessibility and consumption, any production gains achieved are less likely to achieve significant scale or to be sustainable. 21 ASF producers and sellers generally do not design products for sale or consumption by poor consumers. Examples of ASF innovations include grinding Isambaza into powder (e.g., Triumvirate) or packaging/selling smaller cuts of meat (e.g., Hope and Fine Fish and A-Z Pig) 22 The quality of ASF products (i.e., sight, smell, taste, freshness, etc.) affects product demand and consumer trust in ASF producers/sellers. 23 For example, ASF may be consumed as part of a traditional holiday or life event celebrations but not as part of the routine household diet. 24 Norms include, for example, limited female participation intra-household decision-making about food purchases and consumption and males eating meat outside the home without sharing with other household members. 25 Table 12. ASF Market Constraints Identified by Key Informants Constraint Number of Mentions Lack of extension services 57 Lack of access to inputs* 53 Lack of access to credit 51 High cost of inputs* 50 Lack of capacity* 42 Lack of or high cost of transport* 30 Low production* 25 High animal morbidity/mortality* 19 Lack of veterinarian services* 17 Lack of market linkages* 17 Lack of gender inclusion 11 Lack of youth inclusion 11 Lack of access to animal insurance 6 Poor animal quality 5 Lack of PWD inclusion 3 Orora Wihaze also faces implementation constraints that affect its ability to achieve desired results, perhaps more so than the market constraints. Market systems are complex with multiple actors working at multiple levels within the system. This is true as well for the ASF market systems in the fish, poultry, pig, and goat value chains. Market actors include, but are not limited to, producers, traders, processors, transporters, input suppliers, service providers (e.g., vet services), extension agents, distributors, wholesalers, retailers, lenders, and regulators/inspectors that are linked to each other in a variety of formal and informal business and personal networks all working within the context of an enabling environment consisting of regulations, laws, and policies that directly and indirectly affect system functioning. In an underdeveloped market system, which characterizes the ASF sector in Rwanda, constraints often exist at multiple points in the market system involving multiple market actors. For example, solving the access to feed constraint for pig or poultry producers does not necessarily solve the cost of feed constraint. Solving the cost of feed constraint does not necessarily solve the lack of buyers, or lack of extension, or lack of vet services constraints. Solving all these constraints does not necessarily solve the lack of access to end markets. Once a product reaches the market, it may be inappropriately packaged or cost too much for low-income or food-insecure consumers. Finally, even if the product is appropriately packaged and priced, there may be social norms preventing their purchase and consumption by targeted households or inequitably allocating their consumption to certain household members (i.e., males) at the expense of other household members (i.e., women and children). The challenge for an MSD activity like Orora Wihaze is to identify what and where these constraints are and then to address them through strategic interventions working through capable and motivated partners. Exacerbating this challenge is that any MSD activity inevitably possesses constrained resources, both financial and human, and has a relatively short window (e.g., five years) to identify the market weaknesses, recruit partners, intervene, and set up the conditions for sustainability once the activity ends. Given the complexity of the market systems, the multiple weaknesses found therein, the limited resources and time available, and (potentially) the limited number of capable and motivated partners, the MSD activity can at best hope to address a sub-set of the market constraints. This implementation constraint is perhaps the greatest constraint Orora Wihaze faces in achieving the activity’s desired results, even more so than the numerous market constraints elaborated above. 26 Orora Wihaze’s activities strategically address multiple ASF market constraints in its target value chains. Orora Wihaze’s activities reflect in turn an understanding of market and implementation constraints— described above— and a purposive strategy to address them. This is reflected in Table 10 where we see that each constraint is addressed by multiple activities and each intervention addresses multiple constraints, at times addressing constraints involving different combinations of ASF availability, accessibility, and consumption. In addition, activities mutually support each other, with the clear intention being to create a synergistic effect. For example, activity nutrition messaging activities (Intervention 4) support the overall activity by working to increase demand for ASF products to match the expected increase in ASF supply resulting from the activity’s other activities (Activities 1-3). Similarly, a purpose of activities to develop the ASF product market (Intervention 5) is to ensure that the increase in ASF supply from Activities 1-3 arrives at the market packaged and at a price point attractive to Orora Wihaze’s target beneficiaries (i.e., poor households, particularly women and children). The potential for synergistic effects of this type of complementary intervention strategy is reflected in the following two quotes: “SACCOs are not familiar with the ASF sector and see it as risky. But the fact that the project is working to increase the capacity of producers and others in the sector gives them more confidence, and they are now more willing to consider the sector as a potential market.” -Access to finance participant “Why ASF? If you are talking about health, you must deal with nutrition. ASF is key to the welfare of women of reproductive age and youth. Before Orora Wihaze, we were aware of ASF and its importance, but it did not promote it. What has changed is that Orora Wihaze is simultaneously working on availability and access. It didn’t make sense to promote the consumption of foods that were not available in the marketplace or that households had no means of purchasing. What we like most about Orora Wihaze is that it is also working on availability and access, both information and products, which are each necessary to increase consumption.” - Nutrition messaging participant Notwithstanding, gaps in addressing ASF market constraints remain. As alluded to above, gaps in addressing market system constraints will almost inevitably exist owing to resource and time constraints. Despite Orora Wihaze’s successful efforts to strengthen different value chain functions, numerous gaps remain. For example, nutrition messaging activities stress overcoming barriers to ASF consumption, but these are not necessarily matched by activities strengthening ASF availability or accessibility in the same locations. A failure to coordinate availability and/or accessibility activities to utilization activities can result in unfulfilled expectations and de-motivation This coordination challenge is heightened by the partner selection process in that there may not be partners available with the capacity and motivation to address every market gap. Indeed, of the 197 responses Orora Wihaze received to its request for business grant applications (RFA), it selected only 21 to submit a full proposal of which it determined only five were qualified for activity grant and TA support and an additional eight for TA support alone. Grantees interviewed by the evaluation team described the grant application process as long, complicated, and difficult requiring up to nine months or two years to complete. Orora Wihaze’s justified its approach by emphasizing that it consciously takes this approach to allow it to more effectively manage its grant funds and identify partners with feasible and sustainable business ideas. This requires, in turn, a rigorous screening process. If we define a Type 1 error in this case as erroneously rejecting a grant application from a qualified candidate and a Type II error as erroneously accepting a grant application from an unqualified candidate, Orora Wihaze’s approach appears to prioritize avoiding the latter. The advantage of this approach is that it increases the likelihood that Orora Wihaze does not waste time and resources on unproductive partnerships and that those partnerships it does pursue are more likely to have a catalytic effect. The disadvantage of this approach is that it also increases the likelihood that Orora Wihaze loses opportunities to form productive partnerships, either because potential partners are disincentivized from completing 27 the application process or from applying in the first place. The ability or patience to engage in a long, complicated application process are not necessarily the hallmarks of a successful entrepreneur. Arguably, the focus on avoiding Type II errors means fewer partnerships and fewer opportunities to address market constraints and fill identified market gaps. An illustrative example is A-Z Pig, which Orora Wihaze is supporting to move into pig processing and retailing targeted to poor households in Gakenke and Burera. Orora Wihaze initially screened A-Z Pig out for grant consideration; however, after the owner protested, Orora Wihaze, after further investigation, reconsidered and approved A-Z Pig’s grant application. Yet one more implementation constraint is Orora Wihaze’s human resource constraints. At present, Orora Wihaze has eight field staff, including a Portfolio Manager and Nutrition Advisor who each cover two districts. Field staff plays a critical role not only in supporting ongoing activities and coordinating with district officials and district and sector-level platforms but also in identifying partnership opportunities through their active engagement with market actors. Even with the help of the 16 RYAF interns (two per district) (Intervention 6.2.2), field staff are too few for the workload and responsibilities they carry, limiting their effectiveness. Finally, even if capable and motivated partners are available, they may not serve the same markets/communities served by other Orora Wihaze activities, a concern reflected in the following quote from a RICH nutrition messaging participant, “Social behavioral change is here, and the animal value chain is there. There is still a large gap linking the two. I don’t see Orora Wihaze linking producers to consumers. I might have changed my behavior, but if where I live isn’t targeted for promoting animal production, it’s discouraging. So, there is a need to ensure that those who receive messages are also targeted for promoting small animal production." 4.12 CONCLUSIONS: EQ 6 The ASF sector faces numerous market constraints related to ASF availability, accessibility, and consumption that adversely affect the functioning of market systems, and these constraints exist to varying degrees in each of Orora Wihaze’s targeted poultry, pig, fish, and goat value chains. According to key informants, the primary constraints are unavailability and high cost of inputs (particularly animal feed), the lack of access to credit, and low capacity among animal producers and other market actors coupled with the need for ongoing (i.e., initial plus follow-up) capacity building. Orora Wihaze’s MSD approach relies on extensive research and due diligence to identify and simultaneously address multiple market constraints with each intervention; however, resource and time constraints (the latter exacerbated by COVID-related delays) together with challenges identifying capable and motivated partners limit Orora Wihaze’s ability to address the market constraints related to ASF availability, accessibility, and consumption. Whereas Orora Wihaze’s extensive due diligence in selecting business partners increases the likelihood that it maximizes the return on its grant funds and identifies partners with feasible and sustainable business ideas, the extent of the due diligence and its focus on avoiding Type II errors, in addition to district-level staffing constraints, potentially contribute to missing partnership opportunities that would allow Orora Wihaze to further strengthen the targeted value chains and go deeper to plug other market gaps within its targeted districts. 4.13 FINDINGS: EQ 7 Is the MSD approach as implemented by Orora Wihaze creating necessary conditions for the sustainability of interventions and outcomes? What are the key barriers to sustainability among different partner profiles? Orora Wihaze’s interventions appear to be laying the foundations for sustainability in supported value chains, although remaining gaps in market support pose threats to sustainability. A primary rationale for adopting an MSD approach is that it looks for the root causes of market failure to identify constraints that disadvantage small producers and prevent them from improving their economic and social conditions by increasing returns to labor and investment. Because it works to 28 strengthen the ways in which the market system functions, to the benefit of all market actors, the MSD approach includes the seeds for its own sustainability. The findings reported above that Orora Wihaze’s interventions targeting multiple levels of its supported value chains are beginning to generate positive outcomes in the form of improved livestock raising, business, and nutrition behaviors and outcomes suggest that the activity is contributing to improved market functioning in the poultry, fish, and pig value chains, thus beginning to lay the foundations for sustainability. At the same time, however, the MTE findings also suggest that gaps remain within its supported value chains that, if not filled, threaten the sustainability of Orora Wihaze’s interventions and outcomes. Orora Wihaze partners can be grouped into five partner profiles: 1) private market actors; 2) community-based private market actors; 3) social enterprises; 4) community volunteers; and 5) GOR ministries. Below is a brief description of each of the partner profiles. Private market actors: These are for-profit firms, sole proprietors, or commercial small animal producers. Their primary objective is to earn a profit from their ASF-related market activities. Partners include animal producers, agro-dealers, agro-vets, transporters, processors, retailers, SACCOs, MFIs, banks, etc. Community-based private market actors: These are community-based organizations whose members are involved in animal production for a profit but who also exist to serve a social function for their members (e.g., social solidarity or group collective action). Partners include CBSGs and cooperatives. Social enterprises: These are profit-seeking or social mission entities whose objective is to achieve a social return while also generating sufficient profits or financial resources to ensure their sustained operations. Partners include Viamo and Urunana. Community volunteers: These are entities that utilize their networks of community-based volunteers to provide extension services and SBCC messaging to Orora Wihaze’s target beneficiaries. Partners include CAHWs [RCVD], RICH volunteers, RYAF interns, and other community-based volunteer organizations (e.g., Early Childhood Development Centers). GOR: These are GOR ministries responsible for oversight and reform of the ASF enabling environment. Partners include the Ministry of Agriculture and Animal Resources and the Rwanda Agriculture and Animal Resources Development Board. The barriers to the sustainability of Orora Wihaze’s interventions depend on the profile of the partners implementing those interventions. Barriers to sustainability differ for each of the five partner profiles depending on, among other things, the objectives and funding sources for each partner. These factors are summarized below. Private Market Actors Determinants of Sustainability: The principal determinant of sustainability for private market actors is the profitability of their business activities. A secondary determinant is the existence of a stable and facilitating business-enabling environment. Barriers to Sustainability: Primary barriers to sustainability include low business/management capacity and the need for follow-up technical support; lack of access to affordable input, vet services, and credit; unaddressed market gaps (i.e., lack of commercial linkages to other market actors); a weak or unstable business enabling environment; and unanticipated market shocks. Regarding credit access, notwithstanding progress in this area (i.e., the Bank of Kigali created an agri-business lending unit with support from the USAID Nguriza Nshore activity and with inputs from Orora Wihaze on its livestock lending strategy), 29 banks are unlikely to lend to small animal producers outside of, perhaps, cooperatives.25 SACCOs and MFIs offer the best opportunities for significantly scaling loan access, but they remain unfamiliar with the small livestock market, are risk-averse, and require collateral many (most) small animal producers do not have. Prospects for Sustainability: Orora Wihaze has worked to lay the foundation for sustainability for ASF private market actors. To the extent Orora Wihaze can utilize the remaining LOA to stabilize and build on this foundation by providing follow-up support; working to solve the access to affordable inputs, extension, and credit constraints; and linking them to other market actors up, down, and across the value chain the greater the prospects for their sustainability. Community-Based Private Market Actors Determinants of Sustainability: Like private market actors, the determinants of sustainability for community￾based private market actors are the profitability of their business activities and the existence of a stable and facilitating business enabling environment. Also contributing to their sustainability are group cohesion and the delivery of social benefits (e.g., mutual support, solidarity, empowerment, etc.) to group members. Barriers to Sustainability: Primary barriers to sustainability include low business/management/governance capacity and the need for follow-up technical support; lack of access to affordable inputs, vet services, and credit; unaddressed market gaps (i.e., lack of commercial linkages to other market actors); a weak or unstable business enabling environment; and unanticipated market shocks. Prospects for Sustainability: Orora Wihaze has worked to lay the foundation for sustainability for community-based market actors. To the extent Orora Wihaze can utilize the remaining LOA to stabilize and build on this foundation by providing follow-up support; working to solve the access to affordable inputs, extension, and credit constraints; linking them to other market actors up, down, and across the value chain; and contributing to ongoing group cohesion and the delivery of social benefits to group members the greater the prospects for their sustainability. Social Enterprises Determinants of Sustainability: The principal determinant of sustainability for social enterprises is their ability to generate financial resources over time to fund their operations, whether through profits or ongoing external funding support together with their ongoing commitment to ASF programming. Barriers to Sustainability: Primary barriers to sustainability include shifting organizational priorities that weaken their commitment to ASF-related programming and messaging over time. None of Orora Wihaze’s social enterprise partners are ASF-centric; their focus on ASF corresponds to their engagement with Orora Wihaze. They may or may not maintain this focus post-Orora Wihaze, depending on factors like social or economic trends, changing government priorities, windows created by external events, message fatigue among target populations, or changing donor/funding agency priorities. As Orora Wihaze’s social enterprise partners have adapted their programming in line with Orora Wihaze’s financial and technical support, they may adopt it again in line with support from another partner. Prospects for Sustainability: Orora Wihaze’s social enterprise partners have demonstrated an ability to secure financial resources to operate before their involvement with the activity; this is not expected to change once the activity ends. Prospects for the sustainability of ASF messaging depend on the social 25 The midterm performance evaluation of the USAID Nguriza Nshore activity found that the activity NN struggled to engage the interest of commercial banks in lending to micro, small, and medium enterprises (MSMEs); however, SACCOs and other non-bank FIs were more motivated to participate and showed more interest in lending to MSMEs (https://pdf.usaid.gov/pdf_docs/PA00Z2SP.pdf). 30 enterprises’ commitment to ASF programming as determined by the factors listed above. Also important is the extent to which they have internalized ASF programming due to their work with Orora Wihaze. Community Volunteers Determinants of Sustainability: The primary determinants of sustainability for community volunteers are the ongoing social commitment of volunteers to serve their communities; the technical and financial capacity of their parent/oversight organizations to recruit, mobilize, train, and replace their volunteer force; and, in the case of CAHWs, the ability to profit from their activities. Barriers to Sustainability: SBCC volunteers will continue nutrition messaging, as they did prior to Orora Wihaze, but this messaging may or may not include ASF-focused messages. A barrier to sustainability is shifting organizational priorities that weaken the community volunteers’ commitment to ASF-focused messaging over time. Like social enterprises, shifting organizational priorities may occur due to social or economic trends, changing government priorities, windows created by external events, message fatigue among target populations, or changing donor/funding agency priorities. CAHWs are likely to continue providing ASF services given the market segment they support (i.e., small animal producers). Their sustainability will depend on maintaining the technical capacity and financial support of the volunteer network given normal volunteer turnover and anticipated budget shortfalls. In the latter case, RCVD, which oversees a CAHW network, has a full-time staff of only four (director, accountant, logistics, and driver), itself a result of severe budget constraints. One manifestation of this staffing shortage is that RCVD has fallen months behind processing CAHW expense reimbursements, which is a source of growing tension within its CAHW network. Community volunteers working on SBCC messaging have no expectation of benefitting financially from their volunteer work; their return for labor is primarily social (e.g., enhanced social status). With that said, the evaluation team talked to several community volunteers who complained about the lack of financial remuneration, particularly when they saw other community volunteers like CAHWs being reimbursed for their expenses, and lack of transport. To date, however, this has not affected the sustainability of the community volunteer model. The same is not true for CAHWs who are providing value-added services to small producers, agro-dealers, and agro-vets for which private market actors would normally pay a fee. Stakeholders involved in the RCVD intervention recognize that the sustainability of the CAHW extension model ultimately depends on developing a payment system that would allow CAHWs to earn a profit, whether through service fees paid by small animal producers or through something akin to a sales commission or a commission acting as sales agent for agro-dealers and agro-vets. Prospects for Sustainability: For SBCC community volunteers, the prospect that they sustain their ASF messaging depends on their ongoing commitment to ASF messaging as determined by the factors listed above. Also important is the extent to which they have internalized ASF messaging due to their work with Orora Wihaze. Orora Wihaze can increase the likelihood of this outcome by providing follow-up support to ensure that the importance of ASF messages and the capacity to deliver them are embedded within the volunteer networks and their support organizations. Orora Wihaze has an opportunity to significantly improve the sustainability prospects of the CAHW network by working with RCVD and other stakeholders to develop a fee-based service delivery model that will allow CAHWs to profit from their activities. “The key to sustainability is for the CAHWs to act as agents between farmers and agro-dealers. CAHWs are not paid a salary; however, the hope is that by acting as an agent between farmers and agro-dealers, they can earn a fee to sustain them. This is the key to long-term sustainability, finding a way for CAHWs to monetize their service.” -Participant in the RCVD intervention 31 GOR Ministries Determinants of Sustainability: The principal determinant of sustainability of GOR ministries is their ability to secure budgets over time to fund their ASF activities, together with their ongoing commitment to improving the ASF-enabling environment. Barriers to Sustainability: Primary barriers to sustainability include shifting organizational priorities, budget shortfalls, and low capacity (in part due to staff turnover) that adversely affect their commitment and ability to improve the ASF-enabling environment. Government ministries are also subject to changing priorities, whether due to internal imperatives (e.g., change in administration) or the changing landscape and priorities of their external partners (e.g., bilateral or multilateral agencies.) Prospects for Sustainability: Orora Wihaze has worked to lay the foundation for the sustainability of ASF enabling environment support at GOR ministries and continues to make changes to policy and training to increase sustainability. To the extent, Orora Wihaze can utilize the remaining LOA to stabilize and build on this foundation by providing follow-up technical and capacity-building support, the greater the prospects for sustainability. 4.14 CONCLUSIONS: EQ 7 Orora Wihaze’s interventions are putting in place the foundations for sustainability in supported value chains. However, sustainability is not assured as gaps in market support remain, thus more work is required to build on the existing foundation to increase the prospects for sustainability. Factors affecting the sustainability prospects vary for the different Orora Wihaze partner profiles; however, opportunities exist for Orora Wihaze to enhance the prospects for sustainability in each case by working with them to further embed ASF as an institutional priority and building their capacity to play their roles with the market system. For private market actors and community-based market actors, Orora Wihaze can increase their prospects for sustainability by ensuring they are linked in mutually beneficial commercial relationships with other market actors, while for CAHWs Orora Wihaze can increase their prospects for sustainability by working with its partners to develop a fee-based service delivery model that will allow CAHWs to profit from value-added market services they provide. 4.15 FINDINGS: EQ 8 Are the initial assumptions made by USAID and the contractor still valid? Orora Wihaze’s MSD approach is informed by three critical assumptions: income pathway assumption, nutrition pathway assumption, and mutual dependence assumption.26 The rationale for Orora Wihaze, its strategy, and its design are each based on a set of assumptions, as are each of Orora Wihaze’s 27 activities. To answer this EQ, the evaluation team sought to identify through its document review and KIIs/FGDs those assumptions that the team considered most useful for understanding Orora Wihaze’s MSD approach. Based on this analysis, the evaluation team has identified the following three underlying assumptions: 1. Income Pathway Assumption: ASF availability and accessibility are increased through market activities that increase the income earned by ASF market actors. 2. Nutrition Pathway Assumption: ASF consumption is increased through activities that address the constraints to ASF consumption, both within targeted households and among women and children within those households. 26 The evaluation team’s understanding of the term “assumption” in the context of EQ 8 is more or less analogous to its understanding of the term “hypothesis.” 32 3. Mutual Dependence Assumption: Income and nutrition pathways are mutually dependent. Achieving sustainable improvements in nutrition outcomes, including among women and children, require strategic activities in both in different combinations. Orora Wihaze’s intervention strategy incorporates six strategic priorities reflecting a set of assumptions about how different combinations of income and nutrition pathways will contribute to increased ASF availability, accessibility, and consumption. In Year 2 of the activity, Orora Wihaze drew on the lessons learned from the Year 1 implementation and the completion of its baseline survey, consumption study, and gender and social inclusion study to develop a revised set of six strategic priorities, each reflecting different assumptions about the pathways that increase the income of small livestock producers and improve nutrition outcomes among small livestock producer and non￾producer households. Evidence for the validity of Orora Wihaze’s underlying assumptions is found in the emergence of outcomes consistent with the outcomes predicted by those assumptions. During KIIs and FGDs, Orora Wihaze partners and beneficiaries reported numerous outcomes that were consistent with each of the six strategic priorities and their underlying assumptions. While these outcomes were self-reported and, for the most part, unverifiable within the scope of the MTE, they were reported with sufficient frequency that they constituted evidence that the assumptions underlying Orora Wihaze’s MSD approach are valid.27 To demonstrate the congruity of the reported outcomes with assumptions, the report below juxtaposes an illustrative sample of quotes taken from the KIIs and FGDs and information from activity documents with each of Orora Wihaze’s six strategic priorities/assumptions. Priority 1/Assumption 4 (Income Pathway): Investing in the production of ASF will contribute to increased productivity and income of producer households and (potentially) the availability and accessibility of animal foods. Except for small fish, target households generally cannot afford ASF. Thus, a necessary (although not sufficient) condition for increasing ASF consumption is to increase the income of ASF producer households, absent an expectation that they will necessarily change their dietary practices. “The support from Orora Wihaze helps the company to cover some extra costs due to bad conditions of roads and long distances in remote areas. It helps also to fix the prices at the agro￾dealer level. The productivity of company services has increased considerably with the support of Orora Wihaze. The demand and sales have also increased.” - Participant in OX intervention Priority 2/Assumption 5 (Nutrition and Income Pathway): Prioritizing activities that improve the accessibility of specific ASF products that are perceived to be within the “woman’s domain” will contribute to increased ASF consumption and improved nutrition. ASF products are perceived to be in the woman’s domain including small, dried fish, fish powder, and organ meat. Activities should thus seek to improve the accessibility of these products. “We have improved the consumption of animal-sourced food. We are now consuming eggs, chicken meats, and small fish thanks to revenues from our poultry farming.” - Poultry group in Nyamagabe Priority 3/Assumption 6 (Nutrition Pathway): Targeting and changing social gender norms and practices around food procurement and consumption will contribute to increased ASF consumption. The 27 The emergence of these outcomes is not necessarily “proof” that the assumptions are valid, as the same outcomes might have emerged for other reasons (see the above discussion on the methodological limitations related to attribution). At the same time, the lack of such outcomes does not necessarily imply that the assumptions are invalid, as there are several reasons why they may not have emerged yet despite being valid, such as the delay in activity implementation caused by the COVID-19 lockdowns, unanticipated adverse events, etc. With that said, the congruence of reported outcomes with underlying assumptions lends credence to their validity. 33 deployment of SBCC messaging using multiple communications channels (e.g., radio, mobile phone, community-based activism), together with other community-based activities, can be an effective tool to change gender norms that inhibit ASF consumption, for example, by increasing women’s input in household dialogue/decisions about food purchase and consumption and increasing men’s responsibility for ASF procurement. “The project performed well in the reduction of stunting from 51 percent to 33 percent. Positive changes are observed in women’s involvement in nutrition, in collaboration and mobilization, and involvement and mindset change of vulnerable communities belonging to Ubedehe Categories 1 and 2. Women played a big role in the integration of animal-sourced foods in diets and making a balanced and complete food.” - District Official in Nyamagabe Priority 4/Assumption 7 (Nutrition Pathway): Targeting and changing social norms and practices around intra-family dialogue will contribute to increased ASF consumption. Orora Wihaze’s market research found that when husband and wife collaborate on nutrition-related decisions, decision-making improves with respect to the care and sale of animals and the acquisition, and consumption of ASF. The deployment of SBCC messaging combined with other community-based activities can also be an effective tool to change social norms that inhibit inclusive intra-family dialogue. “Before we were doing farming on one farm managed by one member, and we were using a local variety which was less productive. We decided to change the way we do farming. Now each member will have their farm and will use improved variety accommodated in a good shelter, so we will increase pig productivity and revenues. We have increased the consumption of animal￾sourced food like eggs, fish, and meat.” - CBSG member in Gakenke Priority 5/Assumption 8 (Nutrition Pathway): Addressing constraints to consuming own production among small producer households will contribute to increased ASF consumption. Orora Wihaze’s market research found that ASF-producing households consume similar levels of ASF as non-producer households. This requires activities that directly address the constraints to own-production consumption in addition to using the income from ASF production to increase purchases and consumption of ASF products. “Egg consumption increased especially for children and mothers. Men are buying small fish and eggs to feed the children and the whole family.” - District Official in Kayonza Priority 6/Assumption 9 (Income and Nutrition Pathway): Prioritizing the strengthening of local ASF markets will contribute to increased income of ASF market actors (including ASF producer households), increased local availability of ASF products, and increased ASF consumption among ASF market actors and non-producer households. Building local ASF markets will help ensure that more ASF is produced in local markets, that more of the ASF produced in local markets remain there, and that residents consume more ASF (presuming it is adapted to local market demand characteristics), while also opening more business opportunities within local ASF value chains for residents, including women and youth. “When GACOPROCO started, it was working with 65 egg producers who were supplying 1,200 trays of egg per week. As of now, GACOPROCO has increased the number of suppliers and is working with 80-95 producers (44 females and 51 males) and is selling 4,000 trays of egg in a week, indicating more trust, cooperation, collaboration, and strengthened linkages within the Gakenke poultry market system. Now they are aiming to reach 180 (63 females and 117 males) egg producers who will be supplying 8,000 trays of eggs per week in the next 6 months. These egg producers will be reached through linkage with poultry producer’s platform that was established at the sector and district level to strengthen the poultry value chain.” - GACOPROCO Negotiation Memo 34 4.16 CONCLUSIONS: EQ 8 The assumptions underlying Orora Wihaze’s MSD approach are based on sound conceptual reasoning and are supported by evidence in the form of outcomes reported by activity partners and beneficiaries. The assumptions, thus, remain valid. 4.17 FINDINGS: EQ 9 How did Orora Wihaze integrate women, youth, and PWDs into its programs? Orora Wihaze’s integration of women into its programs exceeds targets while its integration of youth lags targets, so far. According to Orora Wihaze’s performance monitoring results, by the end of the Fiscal Year 2021 (the last date for which figures are available), 59.9 percent of its participants were women, exceeding its target of 40 percent. In contrast, only 23.7 percent of its participants were youth compared to its target of 50 percent. Orora Wihaze does not report PWD participation. Whereas key informants across all activity activities consistently cited efforts to integrate women and, to a lesser extent youth, into their programs, few cited efforts to integrate PWDs. Those who did cite efforts to integrate PWDs (i.e., RICH and RYAF) were unsuccessful. Women, youth, and PWD participation are heavily concentrated in three Orora Wihaze activities: RCVD, RICH, and ABIBM. Table 13 shows the breakdown of the female, youth, and PWD outreach by 13 different Orora Wihaze activities as of the end of Quarter 3 in Fiscal Year 2022. As can be seen there, the heaviest concentrations of women, youth, and PWDs considering both percentages and numbers are the RCVD, RICH, and ABIBM activities. RCVD reaches the largest number of females and PWDs while RICH reaches the largest number of youths. Table 13. Female, Youth, and PWD Outreach of Orora Wihaze Activities ADD Total Outreach % Female %Youth %PWD # Female # Youth # PWD 0.0: Animal Husbandry Training 985 46.5% 34.8% 1.7% 458 343 17 0.0: Linkage 1,292 40.1% 16.5% 1.2% 518 213 16 0.0: Non-ADD 47 31.9% 12.8% 0.0% 15 6 0 0.0: Platform 555 40.4% 21.9% 2.9% 224 122 16 1.1.1: AfriSol 10 20.0% 0.0% 0.0% 2 0 0 1.2.1: RCVD 21,721 49.7% 16.0% 2.2% 10,795 3,475 478 2.1.1: UVEPO 170 69.4% 23.8% 4.8% 118 40 8 2.2.1: COVID-19 Egg 19 15.8% 0.0% 0.0% 3 0 0 3.1.2: AMIR-NN 122 45.1% 9.2% 0.0% 55 11 0 4.1.1: NCDA Nutrition Policy 2,204 49.4% 17.5% 0.9% 1,089 386 20 4.2.1: RICH 12,210 74.0% 41.7% 0.6% 9,035 5,092 73 4.2.2: Urunana DC 17 47.1% 17.6% 0.0% 8 3 0 6.2.1: ABIBM 8,478 70.9% 28.5% 2.4% 6,011 2,416 203 With few exceptions, the percentage share of females reached per activity exceeded 40 percent and averaged 46.2 percent overall. Aside from non-ADD, AfriSol, and COVID-19 egg activities, the remaining 10 activities reached a minimum of 40 percent females while three activities— 35 UVEPO, RICH, and ABIBM—achieved around 70 percent or above female outreach, although in the case of UVEPO the absolute number of females reached was low. The percentage share of youth and PWDs reached was much lower, averaging, respectively, 18.5 percent and 1.3 percent. The RICH activity reached the highest percentage share of youth, and the ABIBM activity reached the third highest percentage share, while the percentage share at the RCVD activity was below the overall average. In absolute terms, the RICH activity reached the most youth followed by the RCVD and ABIBM activities. The percentage share and absolute number of PWDs reached were trivial in all cases. Among the activities in Table 13, the ABIBM activity, or specifically working with CBSGs and cooperatives, is an intervention strategy that is commonly found across different types of MSD activities. From a gender and youth perspective, this makes sense. A majority of CBSG members are female, while cooperatives also typically have a large female membership. CBS’s and cooperatives can also have significant youth participation and in some cases can have a large PWD participation. For example, Orora Wihaze works with CBSGs and cooperatives that have a majority PWD membership or include PWDs as members. In Nyamagabe, Orora Wihaze is supporting the poultry cooperative COTUTAMU, which has 20 members (10 male and 10 female) all of whom are PWDs, including three who are deaf and 17 who have assorted physical disabilities. To ensure the participation of the cooperative’s deaf members, meetings are accompanied by sign language. Orora Wihaze’s activities seek to integrate social inclusion through three inclusion strategies: 1) quota targeting, 2) strategic targeting, and 3) merit targeting. Quota targeting occurs when Orora Wihaze or its partners have specific social inclusion policies to ensure a certain level of female, youth, or PWD representation in their programs. Strategic targeting occurs when Orora Wihaze or its partners target women and youth as part of their business or organizational strategy. Merit targeting does not involve quotas/targets nor strategies for social inclusion but rather is a gender and age￾neutral approach that opens participation to all parties based on merit. Quota Targeting The inclusion of women is an explicit or understood priority for most Orora Wihaze partners.28 Inclusion of youth is a lower priority among Orora Wihaze partners, and inclusion of PWD is generally not prioritized among partners. For example, gender is a key recruitment criterion for RYAF interns. Of the 18 interns originally recruited by RYAF, seven (38.9 percent) were female.29 This compares favorably to the overall share of females in the RYAF database in which only around 30 percent of the 4,000 youth in the database are female (reflecting in turn the relatively low participation of females in Rwanda’s agribusiness sector). RICH explicitly recruits both females and youth into its network of community-based religious volunteers to the point that women and youth comprise most of its volunteer network. RCVD has a target of 30 percent female participation in its network CAHWs tasked with providing extension services to small livestock farmers. Strategic Targeting Certain Orora Wihaze partners strategically target women or youth as part of their organizational mission. For example, Urunana’s primary target market for its nutrition messaging is women of childbearing age (i.e., young women). Orora Wihaze is working with CBSGs to support livestock production and as a mechanism for communicating messages on ASF consumption precisely because 28 This finding is based on statements by key informants at Orora Wihaze partner organizations. The evaluation team was not able to verify all statements claiming an organizational commitment to social inclusion. 29 Orora Wihaze/RYAF assigns two interns to each of the activity’s eight districts while the remaining two interns remain in Kigali, one supporting the Orora Wihaze MEL team and one supporting RYAF. 36 CBSGs are a majority female, i.e., it is not a fortunate coincidence that the CBSGs are majority female membership but the strategic reason why Orora Wihaze has elected to work with them. The CBSG/ABIBM intervention is the sole example found by the evaluation team in which Orora Wihaze or its partners successfully targeted and involved PWDs in their programs. A final example is RCVD, which strategically targets youth for its CAHW network because it sees youth as being more energetic and more open to innovation and new ideas than their older counterparts. Merit Targeting The best example of the merit targeting approach is Orora Wihaze’s partnering strategy. As discussed earlier, partner selection is driven by the potential of the partner to catalyze system change. This opens possibilities for women, youth, and PWDs to the extent they have good ideas and the capacity to realize them. Of the nine grants Orora Wihaze has awarded to private business partners, two are with women-owned/led firms: Hope & Fine Fish and JFILEWO. Another example of merit targeting is OX, which in the process of expanding its fleet of trucks has hired more Drivers Plus. One half of the Drivers Plus currently employed by OX are women. To accommodate the female drivers, OX has been engaging with local communities to provide latrine facilities at points along the driving routes. 4.18 CONCLUSIONS: EQ 9 Orora Wihaze has effectively integrated women into its programs—whether as partners, community￾based agents, or beneficiaries—using a combination of quota targeting, strategic targeting, and merit targeting. It has had moderate success integrating youth but relatively little success integrating PWDs into its programs using the same approaches. Orora Wihaze’s work with CBSGs has been its most effective approach for integrating women, youth, and PWDs, in the latter case the sole successful approach. More generally, working through community-based volunteers and group structures (e.g., religious volunteers, CAHWS, CBSGs, cooperatives) has proven to be a particularly effective approach for ensuring/increasing social integration. 4.19 FINDINGS: EQ 10 Does the Orora Wihaze social inclusion strategy ensure continuity beyond the life of the activity? Factors determining the sustainability of Orora Wihaze’s social inclusion strategy are embedded in each of its three social inclusion strategies. Each of Orora Wihaze’s three social inclusion strategies (i.e., quota targeting, strategic targeting, and merit targeting) embed factors that will determine how sustainable they are after the activity ends. Quota Targeting Partners using a quota targeting strategy to include women (most often), youth (less often), or PWDs (rarely) are motivated by a combination of internal priorities and outside pressures (e.g., donor priorities). The evaluation team found little to suggest that quota targeting, to the extent it is occurring, is occurring at the behest of Orora Wihaze. Nor did the evaluation team find evidence suggesting that these partners’ commitment to social inclusion will end with Orora Wihaze’s end. Thus, barring changes in other factors motivating these partners’ social inclusion policies (e.g., changes in donor priorities), they are expected to continue this practice beyond the LOA. Strategic Targeting Partners practicing strategic targeting were practicing strategic targeting before they entered a partnership with Orora Wihaze. Indeed, one factor that made them attractive partners for Orora Wihaze was that they practiced strategic targeting. Orora Wihaze leveraged these partners’ commitment to social inclusion to pursue its own market development, nutrition/food security, and social inclusion objectives. Again, it is expected that these partners will continue their practice of strategically targeting women or youth once Orora Wihaze ends. 37 CBSGs and cooperatives are well-established community-based institutions with a strong tradition of female and youth participation. They offer a mechanism for women, youth, and even PWDs to come together and leverage group social solidarity and economic resources to pursue economic and social objectives. It is this history and function that make CBSGs and cooperatives attractive partners for Orora Wihaze. They will continue playing these roles well beyond the LOA. Merit Targeting Factors determining the sustainability of merit targeting are less evident. Ultimately, Orora Wihaze can only influence this practice within the scope of its activity and partnership network. Social attitudes, social and business norms, government policies, etc. exercise greater influence over merit-based social inclusion practices than any activity or project like Orora Wihaze. However, within the scope of Orora Wihaze’s activities and partnership network, a critical factor contributing to the sustainability of merit targeting is its success. To the extent women, youth, or PWDs succeed in the opportunities provided to them through merit targeting, it potentially creates a demonstration effect incentivizing additional merit-based opportunities for women, youth, and PWDs over time, whether through intentionally seeking opportunities to include women, youth, or PWDs or through the reduction of practices that exclude them from consideration. 4.20 CONCLUSIONS: EQ 10 Orora Wihaze’s social inclusion strategy is likely to be sustained beyond the life of the activity. However, this is due less to Orora Wihaze than to the fact that to the internal policies or characteristics of Orora Wihaze’s partners. Partners practicing quota targeting or strategic targeting are doing so largely independently of Orora Wihaze and are, thus, expected to continue this practice after the activity ends. Orora Wihaze selected the latter precisely because of their pre-existing commitment to social inclusion. Partners practicing merit targeting are also doing so largely independently of Orora Wihaze. Whether they continue this practice will be determined by a variety of factors outside Orora Wihaze’s control. It is, nonetheless, hoped that the success of merit targeting will create a demonstration effect within the targeted market systems leading to increased social inclusion beyond the life of the activity. 5.0 RECOMMENDATIONS Drawing on the above findings and conclusions, this section presents a set of recommendations for Orora Wihaze to consider for adapting the Activity’s strategy and activities over the remaining life of the activity and for USAID/Rwanda to consider for its post-Orora Wihaze MSD programming. • Simplify and shorten the grant application process. Entrepreneurs' strengths do not necessarily lie in completing paperwork or managing complicated administrative processes. The current grant application process, which focuses on avoiding Type II errors, potentially screens out qualified applicants who lack the time, patience, or inclination to complete the application process. It also potentially screens out qualified candidates who do not meet the approval thresholds, notwithstanding possessing the desired capacity and motivation. For example, Orora Wihaze might create a 1–2-page application containing only the most essential pieces of information and streamline the management process to 2-3 months. Or, for grant applicants who fall below thresholds (within a certain band) but who fill an important market gap, conduct additional due diligence to determine whether potential exists that is not evident in the grant application. 38 • Align nutrition messaging with what is available in the local market. To the extent feasible, nutrition messaging should be adapted to what is available and affordable in the local markets (recognizing that full customization of messages may not be cost or administratively feasible). Also target value chain support and SBCC activities so that they overlap, again to the extent feasible. In revising the grant application process (see above), the overlap with SBCC or other activities could be adopted as a screening criterion. • Monitor more closely intervention outreach to and impact on target beneficiaries. Orora Wihaze’s final target beneficiaries are small, food-insecure, low-income producer and non-producer households. Activities ultimately aim to reach these target beneficiaries through, for example, increased access to affordable inputs (e.g., OX), vet and extension services (e.g., RCVD), credit (e.g., AMIR and Vision Fund Rwanda), affordable ASF products (e.g., A-Z Pig, GACOPROCO, Hope and Fine Fish), and nutrition messaging (e.g., RICH, Urunana, Viamo). The Orora Wihaze annual household, or catchment, survey tracks yearly changes in outcome indicators across activity catchment areas. For example, to what extent are the targeted small livestock producers benefitting from lower input prices in OX districts, as opposed to larger livestock producers? Or, to what extent are food insecure, low-income households purchasing A-Z Pig, GACOPROCO, or Hope and Fine Fish products? At the same time, however, this annual survey is large and complex and takes considerable time to analyze, which limits its management and decision-making utility. More timely and targeted information could be also collected using a variety of approaches like in-person, mobile phone, or SMS surveys; rapid assessments; crowdsourcing, market observations; etc. This will provide additional useful information on the effectiveness and sustainability of activity interventions. • Provide market information matching ASF buyers to ASF sellers. To help ensure that demand for ASF corresponds to supply, use media to broadcast information to consumers about what types of ASF products are available and where. For example, Orora Wihaze could work through, or modify, its existing agreements with Urunana, Viamo, and radio stations to provide information on ASF availability. The same information networks could be used to generate information on ASF demand to facilitate the recruitment of partners to increase the supply of ASF and its accessibility in specific markets. • Provide follow-up support to implementing partners. Implementing partners in all stakeholder groups and at all levels of the supported valued chains consistently cited the need for additional follow￾up training and reinforcement support to ensure the sustainability of Orora Wihaze interventions. It is acknowledged that providing follow-up support limits the activity’s ability to expand and scale up its activities, including activities that fill identified market gaps (see recommendation above). However, this does not rule out the need to allocate resources and time to ensure that key learnings are embedded at supported partners and that they have the capacity to continue implementing them after the activity ends. • Invest in increasing Orora Wihaze field staff. Orora Wihaze’s District Portfolio Managers and Nutrition Specialists are expected to coordinate with numerous private market actors, district, and sector-level officials, SACCOs and MFIs, and community volunteers, not to mention several district and sector-level producer platforms, each covering two activity districts. In addition to performing these essential functions, they are Orora Wihaze’s, eyes and ears for identifying market gaps and partnership opportunities. They are understaffed and under-resourced to play these roles to the optimal extent. Each district should have at least one dedicated, full-time field staff. To support the field staff, investigate the feasibility of increasing the number of RYAF interns and/or extending their period of service from six to 12 months.25 39 • Buy down the risk of SACCOs and MFIs to lend to small livestock producers. Banks are unlikely to lend to small livestock producers (outside of, perhaps, cooperatives). SACCOs and MFIs offer the best opportunities for scaling loan access; however, they are unfamiliar with the livestock sector and are risk averse, often requiring collateral that most small livestock producers do not have. USAID/Rwanda can help buy down the risk further either by linking SACCOs and MFIs with the Business Development Fund loan guarantee program or linking pig and poultry producers to the GOR’s subsidized livestock insurance scheme. Either approach may require an additional, temporary subsidy by USAID to buy down the risk to the point where the potential for significant demonstration effects exists. • Target CBSGs and cooperatives to extend outreach to youth and PWDs. While Orora Wihaze has surpassed its women’s inclusion targets, it is falling way short of its youth inclusion targets and has achieved only marginal outreach to PWDs. To the extent youth and PWD outreach is a priority for USAID/Rwanda, this will require an explicit targeting strategy with cooperatives and CBSGs offering the greatest potential for achieving significant outreach to these groups. Note that this recommendation does not rule out implementing intentional activities targeted specifically to youth and PWDs nor the continued layering of integration activities on existing scale partnerships, such as, in this case, RCVD and RICH. • Develop, pilot, and scale up a fee based CAHW extension model. Key informants familiar with the RCVD intervention expressed skepticism about its sustainability absent a mechanism for compensating CAHWs for the agricultural extension services they provide. It is not reasonable to expect CAHWs to continue providing services that in a functioning market would be fee-based services. Developing, testing and (if time allows) implementing this model should be a priority over the remaining life of the activity. 40 ANNEXES ANNEX 1. STATEMENT OF WORK STATEMENT OF WORK MID TERM EVALUATION FOR THE FEED THE FUTURE ORORA WIHAZE ACTIVITY Background Despite Rwanda's economic growth, levels of childhood stunting remain persistently high (33 percent, DHS 2020), and only a small proportion of women and children consume a minimum acceptable diet, with animal source foods (ASF) consisting of around 1% of the normative food basket. Two of the interconnected root causes of low animal source protein consumption levels are a poorly functioning ASF sector with high barriers to entry and accompanying low demand for animal food products by the Rwandan public. Like in all emerging economies, an increase in the middle-class population has triggered increased demand for ASF products and diverse diets, hence attracting more investors to the small livestock sector. It is in this market context that USAID/Rwanda decided to catalyze investment in ASF through Orora Wihaze, building on existing efforts to improve the enabling environment by the Government of Rwanda and other donor programs. Feed the Future Orora Wihaze (Raise Animal for Self-Sufficiency) Activity is a five-year (2019-2024) investment aiming to develop an inclusive, profitable market with the goal of sustainably increasing the availability of access to, and consumption of ASF in Rwanda. To achieve this goal, Orora Wihaze is intervening to strengthen private sector led ASF value chains, by facilitating the production of poultry, swine, fish, and goat, and strengthening relationships of market system actors. In addition, Orora Wihaze is seeking to increase demand for ASF consumption among women of reproductive age and children of 6–23 months by increasing the availability and accessibility of ASF for safe consumption in target districts (Rutsiro, Ngororero, Nyamasheke, Gakenke, and Burer,a and Nyamagabe, Kayonza and Ngoma) and fostering greater women’s empowerment. By the end of this five-year activity, the income of 125,000 households will have increased by 30%, farmer and firm sales will have increased by 45% and 35%, respectively, and the prevalence of children of 6-23 months of age and women of reproductive age receiving a minimum acceptable diet will have increased by 40%. Orora Wihaze primarily contributes to the Development Objective 3 (PROSPER) of the Country Development Cooperation Strategy (CDCS); intermediate result (IR) 2: (Agriculture modernized as a driver of growth). Animal-sourced foods are both high-value and nutritious. Orora Wihaze contributes to PROSPER sub-IR 2.1: (Production Diversified into High Value and Nutrition Value Chains). The Activity seeks to utilize new production technologies and approaches to increase yields, contributing to sub-IR 2.2: (Adoption of Productivity-enhancing Climate Smart Technologies) and to better connect producers to markets, contributing to sub-IR 2.3: (Expanded Access and Responsiveness to Markets). Orora Wihaze also contributes to policy dialogue and development for the ASF sector, thereby contributing to sub-IR 1.1: (Enterprise￾Driven Policies Adopted and Implemented). Additionally, the emphasis on ASF consumption improvements, particularly for women and children, contributes to the CDCS cross-cutting nutrition objective while the farm/firm income growth interventions contribute to the CDCS cross-cutting objective of financing self￾reliance. 41 Orora Wihaze Result Framework: Land O’Lakes Venture37 and its consortium, Catholic Relief Services (CRS), MarketShare Associates (MSA), The Manoff Group (TMG) and Urunana DC, in partnership with Rwanda industry leaders, is applying a market systems development (MSD) approach by working with existing market actors (partners) to improve the availability of, accessibility to, and demand for ASF among men, women, and children in Rwanda. The use of the MSD facilitative approach will not only change the practices and behaviors of actors or households, as described above, but aims to also shift how the market system works so that other market actors outside the Activity’s purview will also adapt and benefit. Orora Wihaze will support partners to implement innovations that support the achievement of Activity objectives and will disrupt business-as￾usual within the system and change interrelationships and perspectives of market actors. Other actors will take notice of the changes and modify their behavior accordingly, such that the market system functions shift in the direction of improved inclusivity and better outcomes for our target populations. Both the changes at the individual agent-level and market system-level taken together will reach the Orora Wihaze goal of sustainably increasing the availability and access to, and consumption of ASF in Rwanda. The consortium spent the first two years conducting analysis to understand the market and consumption barriers, facilitating linkages in the districts, rolling out the Activity’s first partnerships, and developing new partnerships to implement over the life of the Activity. Despite this progress, Orora Wihaze started implementation during the COVID-19 pandemic that delayed the roll-out of some activities, hence affecting its projected achievements to date. 42 Overall Objective: Assess Orora Wihaze achievements, progress, and challenges in reaching Activity goals. Specific Objectives USAID/Rwanda will conduct a mid-term evaluation of Orora Wihaze to identify lessons learned in its first two and a half years of implementation, particularly from applying a market systems development (MSD) approach within the Rwandan context. The Mission will use the recommendations to inform the work plans of the remaining two years to support the achievement of target results. The Mission expects that the findings from this mid-term evaluation will contribute to learning in the following areas: Activity Performance ● Understand the extent to which the MSD approach applied by Orora Wihaze, focusing on partnerships for innovation, is contributing to availability of, access to, and demand for animal source foods by the target population. Integration of cross cutting themes ● Determine the appropriateness and effectiveness of interventions focused on integrating gender, youth, and persons with disabilities as well as private sector and civil society towards achieving the objectives of Orora Wihaze. Sustainability of interventions ● Assess early evidence of sustainable change30 at the agent31 and the system levels produced by Orora Wihaze. Collaborating Learning and Adapting (CLA) ● Assess the extent to which CLA has contributed to Orora Wihaze’s implementation and achievements. Evaluation Questions: The evaluation will respond to the following questions and provide recommendations where appropriate. The evaluation will assess implementation from the activity start date to the present and draw conclusions based on its findings. It will identify opportunities and challenges and formulate recommendations for improvement during the remaining implementation period. If necessary, the evaluation team may submit additional evaluation questions and share them with USAID for consideration. Below are the key evaluation questions: Performance ● Is the activity demonstrating intended outcomes? Are households and businesses making more money, are markets transformed, are households consuming more ASFs? ● Are there unintended outcomes generated by the activity interventions? ● What are the drivers for differential performance of targeted districts? ● Is the Orora Wihaze MSD approach still sufficiently effective and adaptive to achieve nutrition outcomes and a profitable ASF market in the current context? 30 An example of systemic change is increased interest and businesses investing in geographic areas, where they were not present before, such as new slaughterhouses in a rural area. Or working with the private sector to create demand for improved reproductive systems for pig genetics. 31 Agent level change is increased interest for households or organizations to change behaviors because of a new skill or knowledge received from Orora Wihaze intervention. This new behavior may attract adaptation from other agents or support growth or increase participation in the market. 43 ● What are the constraints for achieving results and what are the recommendations for improvement? ● Did the partnerships established between Orora Wihaze and the market actors incentivize changes at the household, firms, or system level? What changes have been realized and what are the constraints? Sustainability: ● Is the MSD approach as implemented by Orora Wihaze creating necessary conditions for sustainability of interventions and outcomes? What are the key barriers to sustainability among different partner profiles? CLA ● Are the initial assumptions made by USAID and the contractor still valid? ● What changes or adaptations does the contractor need to make in the remaining implementation period? Cross cutting issues ● How did Orora Wihaze integrate women, youth, people with disabilities (PWD) into its programs? ● Does the Orora Wihaze social inclusion strategy ensure continuity beyond the life of the activity? Deliverables and Methodology Methodology: The mid-term evaluation team will propose the methodology to be used, which will be reviewed and approved by USAID/Rwanda. The evaluation must follow USAID guidance for high quality evaluations; consider a range of possible methods and approaches for collecting and analyzing data to answer the evaluation questions; and conduct a review of key documents produced by Orora Wihaze including the work plans, implementation strategy, MEL plans, indicator data and progress reports among others. The evaluation team, in collaboration with USAID, will finalize the evaluation methods before fieldwork begins. The contractor will submit the preliminary evaluation design in response to the Statement of Work for review by USAID/Rwanda. The Evaluation Contracting Officer’s Representative (COR) will approve the finalized evaluation design at least two weeks prior to the team’s arrival in the country at the recommendation of the USAID/Rwanda activity manager. Deliverables: The following deliverables are required: ● Work Plan. Within 5 business days from the award date, the team lead will complete and present a draft work plan for the evaluation to the COR. The work plan will provide a detailed schedule and logistical arrangements, a list of team members including their roles and responsibilities, evaluation milestones, proposed evaluation methodology, and the evaluation report outline. The USAID team must review the draft work plan and provide feedback within five business days. The evaluation team will consolidate USAID comments, update the work plan, and submit the final work plan within 3 business days. ● Evaluation design: Within 10 business days of work plan approval, the evaluation team must submit an evaluation design to the activity manager. The design will become an annex to the evaluation report. The evaluation design will include: ○ Detailed evaluation design matrix that links the Evaluation Questions from the SOW (in their finalized form) to data sources, methods, and the data analysis plan. ○ Draft questionnaires and other data collection instruments or their main features. 44 ○ List of potential interviewees and sites to be visited and proposed selection criteria and/or sampling plan (must include sampling methodology and methods, including a justification of sample size and any applicable calculations) by district. ○ Limitations to the evaluation design; and ○ Dissemination Plan (designed in collaboration with USAID) Unless exempted from doing so by the COR, the evaluation design will be shared with partner country stakeholders as well as with the implementing partners for comment before being finalized. ● In briefing: Within two business days from the award date, the evaluation team will meet with the USAID/Rwanda Economic Growth and Program offices for introductions and to discuss the team’s understanding of the assignment, initial assumptions, evaluation questions, methodology, and work plan, and/or to adjust the SOW, if necessary. ● Out briefing with the data collection team: Two days after the data collection is finalized, the team will present to USAID how the data collection process went and the preliminary findings and recommendations. ● Draft Evaluation Report: The draft evaluation report should be consistent with the guidance provided. The report will address each of the questions identified in the SOW and any other issues the team considers to have a bearing on the objectives of the evaluation. Any such issues can be included in the report only after consultation with USAID. The submission date for the draft evaluation report will be determined in the evaluation work plan. Once the initial draft evaluation report is submitted, USAID will have 10 business days in which to review and comment on the initial draft, after which point the activity manager will submit the consolidated comments to the evaluation team. The evaluation team will then be asked to submit a revised final draft report within 10 business days. This performance evaluation will comply with USAID evaluation policy requirements for evaluations, which is available in Annex A of the Evaluation Policy and at the following link: http://www.usaid.gov/sites/default/files/documents/2151/USAIDEvaluationPolicy.pdf ● Oral Preliminary findings presentation After the completion of the field work, the Evaluation Contractor will deliver a presentation (verbal and with PowerPoint) on preliminary results, key findings, and recommendations. The audience will include USAID/Rwanda and activity stakeholders. The presentation will: ● Review and assess Orora Wihaze performance based on the questions listed above. The core of the presentation should be aligned with the main topics of this evaluation. ● Provide the methodology and approach used to evaluate the activity. ● Provide preliminary analysis, findings, and conclusions; and ● Provide sources of information. ● Final evaluation report The evaluation team will be asked to take no more than 10 business days (or as agreed upon in the work plan) to respond to and incorporate final draft evaluation report comments from USAID. The evaluation team lead will then submit the final report to the activity manager. The final report must be ADS 508 compliant. The final report must not exceed 40 pages, excluding the cover page, table of contents, executive summary, and annexes. ● Submission of raw data set to Development Data Library 45 Per USAID’s Open Data policy (see ADS 579, USAID Development Data) the contractor must also submit to the COR and the Development Data Library (DDL), at www.usaid.gov/data, in a machine-readable, non-proprietary format, a copy of any dataset created or obtained in performance of this award, if applicable. The dataset should be organized and documented for use by those not fully familiar with the intervention or evaluation. Please review ADS 579.3.2.2 Types of Data to Be Submitted to the DDL to determine applicability. ● Submission of the Final Evaluation Report to the Development Experience Clearinghouse Per USAID policy (ADS 201.3.5.18) the contractor must submit the evaluation final report and its summary or summaries to the Development Experience Clearinghouse (DEC) within three months of final approval by USAID. Note: The draft evaluation, oral preliminary findings presentation and final report should include: ○ Recommendations for improvement of OW implementation, ○ Identify clear pathways for implementing policy change on small livestock ○ Recommendations on areas to focus on in their implementation Timeline USAID expects that the mid-term evaluation will be conducted within a period of three months starting from Early April - late June 2022. Team Composition To meet the requirements of team composition, and to ensure rigorous field work and analysis, the following is suggested for key team composition: ● Market systems evaluation specialist (with demonstrated extensive experience on enterprise and agribusiness development and conducting market systems development evaluation on previous USAID or other donors funded projects) ● Food Security specialist with technical experience in the livestock value chains development and nutrition - with preferable experience in the local context ● Monitoring, Evaluation and Learning Specialist with market systems or related experience, field research, and stakeholder engagement in dialogue via appropriate methods to collect feedback and data analysis. Annex: Suggested questions for stakeholder interviews ● Orora Wihaze uses both solicited and unsolicited approaches to identify potential partners. As they continuously engage with new and existing and potential partners, to what degree has the implementation team balanced their planning process with emerging opportunities, while still following through on ongoing work? How has OW balanced planned interventions with emerging opportunities towards achieving results? ● Assess the degree and benefits of efficacious coordination, collaboration, and convergence with locally based sub-partners. Possibly add Learning and adaptation. Have local partnerships and actions around them translated into results? ● Has the information and lessons learned generated by OW translated into achieving results? 46 ANNEX 2. BIBLIOGRAPHY Gakenke District. (2018). “District Development Strategy 2018-2024.” Kayonza District. (2018). “District Development Strategy 2018-2024.” Land O’Lakes Venture37. (2020). “Annual Performance Report, Fiscal Year 2020, Q4.” Land O’Lakes Venture37. (2020). “Orora Wihaze Annual Performance Report.” Land O’Lakes Venture37. (2021). “Annual Performance Report: Fiscal Year 2021, Q3-Q4.” Land O’Lakes Venture37. (2021). “Feed the Future Rwanda Orora Wihaze Market Systems and Behavior Change Strategy.” Land O’Lakes Venture37. (2021). “Orora Wihaze Annual Performance Report. Land O’Lakes Venture37. (2021). “Orora Wihaze Fiscal Year 2, Q3 Performance Report.” Land O’Lakes Venture37. (2021). ‘’Orora Wihaze Baseline Survey Results, Viamo’’ Land O’Lakes Venture37. (2022). ‘’Orora Wihaze Midline Scope of Work.’’ Land O’Lakes Venture37. (2021). ‘’Orora Wihaze Activity Design Document. ‘’ Land O’Lakes Venture37. (2022). ‘’Orora Wihaze Observation Report of Ecological Factors’’ Land O’Lakes Venture37. (2022). ‘’Orora Wihaze Animal Sourced Foods Market Baseline/Midline Study Assessment ‘’ Land O’Lakes Venture37. (2022). "Orora Wihaze CBSGs and Coops ASF-Based Inclusive Business Models.’’ Rwanda Poultry Industry Association (RPIA). (2022-2026). ‘’Five Year Strategic Plan ‘’ Rwanda Catfish Hatchery. (2022). ‘’Standard Operating Procedures (SOPs)’’ Market Share Associates. (2020). “Feed the Future Rwanda Orora Wihaze: Baseline Report.” National Institute of Statistics Rwanda (2018). “Agricultural Household Survey.” National Institute of Statistics Rwanda (2020). “Agricultural Household Survey.” Nyamagabe District. (2018). “District Development Strategy 2018-2024.” Nyamasheke District. (2014). “District Development Strategy 2018-2024.” USAID/Rwanda. (2020.) “Orora Wihaze Baseline Report.” USAID/Rwanda. (1999). “Food Security Indicators and Framework for Use in The Monitoring and Evaluation of Food Aid Programs.” USAID/Rwanda. (2015). “Gender Analysis for USAID/Rwanda: Feed the Future Project.” USAID/Rwanda. (2019). “Gender and Social Inclusion Analysis Report.” USAID/Rwanda. (2020). “Market Systems Development Design and Implementation Manual.” USAID/Rwanda. (2021). “Gender And Social Inclusion Integration Strategy.” USAID/Rwanda. (2021). “Orora Wihaze Goat Market Systems Analysis.” USAID Rwanda. (2021). “Orora Wihaze Market Systems and Behavior Change Strategy.” USAID/Rwanda. (2021). “Learn Project Gender and Social Inclusion Analysis February.” USAID/Rwanda. (2022). Orora Wihaze Poultry Market Systems Analysis.” 47 Weatherspoon, Dave, Miller, Steve, Ngabitzinze, Jean, Weatherspoon, Lorraine, and Oehmke, James. (2019). “Stunting, Food Security, Markets and Food Policy in Rwanda BMC Public Health.” USAID/LEO and DFID/BEAM. (2016) ‘’Market Share Associates. Testing Tools for Assessing Systemic Change: overview of the tool trials.’’ USAID/Rwanda. (2016). ‘’Market Share Associates. Testing Tools for Assessing Systematic Change: A Synthesis.’’ Dunn, Elizabeth et al. (2016). ‘’Guidelines for Monitoring, Evaluation, and Learning in Market Systems Development.’’ 48 ANNEX 3. KII AND FGD GUIDES Key Informant Interview Guide: Companies Receiving Direct Support ● Production (Intervention 1) ● End Market Access (Intervention 2) ● ASF Product Market (Intervention 5) Intervention #: Date: Interviewer: District: Sector: Participants: Gender: __M/F__ Introduction ● Introduce yourself and explain the purpose for the interview or discussion (if appropriate, present the letter of introduction from Orora Wihaze). ● Reassure respondents that their responses will be held strictly confidential. ● Ask if respondents have any questions. ● Secure respondents’ permission to proceed with the interview. Background 1. Please describe your business or organization. PROBES ● Type of product/service produced ● Value chain (i.e., poultry, pig, fish, goat/sheep) ● Customers 2. What is your position or role in your business or organization? 3. What type of support has your business/organization received from the Orora Wihaze project? Challenges/Constraints 4. What are the primary challenges or constraints facing your business? (EQ 5) PROBES ● In general (i.e., other businesses like this one) ● Specific to this business 5. How and to what extent does the support you received from the Orora Wihaze project address these challenges or constraints? (EQ 4) Adoption of New Practices or Technologies 6. What changes have you made to your business practices because of Orora Wihaze support? (EQ 6) PROBES • Developed new business strategy • Adopted new production methods, technologies, or equipment 49 • Developed products or services • Created distribution channels or sales outlets • Expanded to new markets/locations • Targeted new customers or market segments 7. What challenges or constraints do you face in making these or other changes to your business practices? (EQ 6) Access to Financing 8. Did the project help you to receive a loan or other financial support from a financial institution (bank, microfinance institution)? Please explain. 9. Please explain the loan process. ● Timing ● Level of difficulty ● What was assistance from Orora Wihaze? 10. Did the terms of the loan meet your needs? ● Amount ● Interest rate ● Relative to production cycle o Payment schedule (weekly, monthly, annually…) o Length of loan (1 year, 9 months, 5 years…) 11. Overall, how satisfied were you with the loan and the loan process? Why? 12. Do you plan to get a loan again? Why? Outcomes of Orora Wihaze Support 13. What have been the outcomes of the changes you made to your business practices due to Orora Wihaze support? (EQ 1) PROBES ● Increased production or productivity ● Increased sales ● Increased profits ● Increased employment 14. Were there any outcomes (either good or bad) that surprised you (unintended)? What were they? (EQ 2) Gender and Social Inclusion 15. What support did you receive from Orora Wihaze to increase the inclusion of women, youth, or people with disabilities in your business? (EQ 10) 16. What changes have you made to your business because of this support? (EQ 10) PROBES ● Hired or promoted women, youth, or people with disabilities ● Developed policies business/organizational policies covering women, youth, or people with disabilities ● Developed products/services targeted to women, youth, or people with disabilities 17. What are the primary challenges for increasing the inclusion of women, youth, or people with disabilities in your business? 50 Sustainability 18. Thinking about all the changes you have made to your business we have discussed today and their outcomes, how likely is it that these will be sustained over the long term, especially once Orora Wihaze support to your business/organization ends? (EQ 11) 19. Why? What factors make it more or less likely that they will be sustained over the long term? Recommendations 20. Overall, how satisfied are you with the support your business/organization has received from the Orora Wihaze project? PROBES ● Relevance or usefulness to specific challenges or constraints ● Type of support ● Amount of support ● Quality of support ● Follow-up or reinforcement 21. What would you recommend to improve the support you have received from the Orora Wihaze project? (EQ 5, EQ 9) PROBES ● Increase the relevance or usefulness to specific challenges or constraints ● Improved/different type of support ● Improve outcomes ● Increase the amount of support ● Increase the quality of support ● Provide more/better follow-up or reinforcement 51 Key Informant Interview Guide: Orora Wihaze District Staff Date: Interviewer: District: Interviewee: Position: Gender: __M/F__ Email: Phone: Introduction ● Introduce yourself and explain the purpose for the interview or discussion (if appropriate, present the letter of introduction from Orora Wihaze). ● Reassure respondents that their responses will be held strictly confidential. ● Ask if respondents have any questions. ● Secure respondents’ permission to proceed with the interview/discussion. Challenges/Constraints 1. What are the primary challenges or constraints to market system development and improved nutrition in your district? (EQ 5) 2. How is this different from the challenges or constraints that exist in other districts? (EQ 3) 3. How well do you think the Orora Wihaze project is addressing these challenges or constraints (EQ 4)? Adoption of New Practices or Technologies 4. What evidence is there that market actors are adopting new behaviors because of Orora Wihaze support in your district? (EQ 6) PROBES ● Developed new business strategy ● Adopted new production methods, technologies, or equipment ● Developed products or services ● Created distribution channels or sales outlets ● Expanded to new markets/locations ● Targeted new customers or market segments ● Purchasing and consuming more animal-sourced foods 5. What challenges or constraints do market actors face in trying to adopt these new behaviors? (EQ 6) Outcomes of Orora Wihaze Support 6. What evidence is there of improved market or nutrition outcomes because of Orora Wihaze support in your district? (EQ 1) 52 PROBES ● Increased production or productivity ● Increased sales ● Increased profits ● Increased employment ● Increased purchases/consumption of animal-sourced foods ● Improved nutrition 7. Were there any outcomes (either good or bad) that surprised you? What were they? Partnerships 8. How did Orora Wihaze select the partners in your district. How effective was this process? 9. What are the challenges in finding good partners? 10. How well are the partnerships working? What problems are challenges are you encountering working with them? 11. What do you recommend to improve the process for selecting and working with partners? Social Inclusion 12. What is Orora Wihaze doing to increase the inclusion of women, youth, or people with disabilities in your district? (EQ 10) 13. What evidence is there that market actors are increasing their inclusion of women, youth, or people with disabilities in your district? (EQ 10) PROBES ● Hired or promoted women, youth, or people with disabilities ● Developed policies business/organizational policies covering women, youth, or people with disabilities ● Developed products/services targeted to women, youth, or people with disabilities ● Targeted messaging for women, youth, or people with disabilities 14. What challenges or constraints do market actors face increasing the inclusion of women, youth, or people with disabilities in your district? Sustainability 15. Thinking about all the changes we have discussed today and their outcomes, how likely is it that these will be sustained over the long term, especially once Orora Wihaze support ends? (EQ 11) 16. Why? What factors make it more or less likely that they will be sustained over the long term? Recommendations 17. What would you recommend to improve Orora Wihaze interventions in your district? (EQ 5, EQ 9) PROBES ● Increase the relevance or usefulness to specific challenges or constraints ● Improved/different type of support ● Improve outcomes ● Increase the amount of support ● Increase the quality of support ● Provide more/better follow-up or reinforcement 53 Key Informant Interview Guide: Government Agencies, National Organizations, District Officials, or Other Organizations Supporting Orora Wihaze Interventions Intervention #: Date: Interviewer: Organization: District: Interviewee: Interviewee’s Position: Gender: __M/F__ Email: Phone: Introduction ● Introduce yourself and explain the purpose for the interview or discussion (if appropriate, present the letter of introduction from Orora Wihaze). ● Reassure respondents that their responses will be held strictly confidential. ● Ask if respondents have any questions. ● Secure respondents’ permission to proceed with the interview. Background 1. What does your organization do? 2. What is your position or role within your organization? 3. How familiar are you with the Orora Wihaze project? 4. How is your organization working with the Orora Wihaze project? Challenges/Constraints 5. Regarding the work your organization does, what problems or challenges are you seeking to address? (EQ 5) 6. How do these problems or challenges vary across districts? ● Focus on differences across the eight districts supported by Orora Wihaze. 7. How and to what extent does the Orora Wihaze project address these problems or challenges? (EQ 4) Outcomes of Orora Wihaze Support 8. What have been the outcomes of the Orora Wihaze project related to the work your organization does? PROBE ● Focus the discussion on those outcomes sought by the interviewer and his/her organization, particularly in terms of broader market or other changes/improvements. 9. How do these outcomes vary across districts? PROBE ● Focus on differences across the eight districts supported by Orora Wihaze. 10. Were there any outcomes (either good or bad) that surprised you? What were they? 54 Social Inclusion 11. Are you familiar with work Orora Wihaze is doing to increase the inclusion of women, youth, and people with disabilities? 12. If yes, please describe this support. 13. What has been the result of this support? 14. What are the primary challenges or constraints related to increasing the inclusion of women, youth, or people with disabilities? Sustainability 15. Thinking about all the outcomes of Orora Wihaze support we have discussed today, how likely is it that these will be sustained over the long term, especially once Orora Wihaze support to your business/organization ends? (EQ 11) 16. Why? What factors make it more or less likely that they will be sustained over the long term? Recommendations 17. Overall, how satisfied are you with your work with the Orora Wihaze project? PROBES ● Relevance or usefulness to problems or challenges ● Type of support ● Amount of support ● Quality of support ● Follow-up or reinforcement 18. What would you recommend to improve the Orora Wihaze project? (EQ 5, EQ 9) PROBES ● Increase the relevance or usefulness to problems or challenges ● Improve outcomes ● Different type of support ● Increase the amount of support ● Increase the quality of support ● Provide more/better follow-up or reinforcement 55 Key Informant Interview Guide: Financial Institutions ● Access to Finance (Intervention 3) Intervention #: Date: Interviewer: Financial Institution: Interviewee: Interviewee’s Position: Gender: __M/F__ Email: Phone: Introduction ● Introduce yourself and explain the purpose for the interview or discussion (if appropriate, present the letter of introduction from Orora Wihaze). ● Reassure respondents that their responses will be held strictly confidential. ● Ask if respondents have any questions. ● Secure respondents’ permission to proceed with the interview. Background 1. What is your position or role in your financial institution? 2. Please describe the type support you have received from the Orora Wihaze project. Challenges/Constraints 3. What are the primary challenges or constraints do you face in lending to businesses or small producers in livestock sectors like goats, pigs, poultry, and fish? (EQ 5) 4. How do these challenges vary across districts? PROBE ● Focus on differences across the eight districts supported by Orora Wihaze. 5. How and to what extent does the support you received from the Orora Wihaze project address these challenges or constraints? (EQ 4) Adoption of New Practices or Technologies 6. What changes have you made in your small livestock lending practices because of Orora Wihaze support? (EQ 6) PROBES ● Developed new lending policy or strategy ● Changed organizational structure (e.g., created new departments) ● Hired livestock lending specialists ● Developed new loan products ● Adjusted loan terms, such as interest rates, collateral requirements, repayment schedules, etc. 7. What challenges or constraints do you face in making these or other changes to your small livestock lending practices? (EQ 6) 56 Outcomes of Orora Wihaze Support 8. What have been the outcomes of the changes you made to your small livestock lending practices due to Orora Wihaze support? (EQ 1) PROBES ● Increased number or value of small livestock loans ● Increased profits ● Increased employment ● Increased management or organizational support for small livestock lending 9. Were there any outcomes (either good or bad) that surprised you? What were they? Social Inclusion 10. What support did you receive from Orora Wihaze to increase the inclusion of women, youth, or people with disabilities in your lending practices? (EQ 10) 11. What changes have you made to your lending practices because of this support? (EQ 10) PROBES ● Hired or promoted women, youth, or people with disabilities ● Developed policies business/organizational policies covering women, youth, or people with disabilities ● Developed loan products targeted to women, youth, or people with disabilities 12. What challenges or constraints do you face increasing the inclusion of women, youth, or people with disabilities in your lending practices? Sustainability 13. Thinking about all the changes you have made to your lending practices we have discussed today and their outcomes, how likely is it that these will be sustained over the long term, especially once Orora Wihaze support to your business/organization ends? (EQ 11) 14. Why? What factors make it more or less likely that they will be sustained over the long term? Recommendations 15. Overall, how satisfied are you with the support you have received from the Orora Wihaze project? PROBES ● Relevance or usefulness to specific challenges or constraints ● Type of support ● Amount of support ● Quality of support ● Follow-up or reinforcement 16. What would you recommend to improve the support you have received from the Orora Wihaze project? (EQ 5, EQ 9) PROBES ● Increase the relevance or usefulness to specific challenges or constraints ● Improved/different type of support ● Improve outcomes ● Increase the amount of support ● Increase the quality of support ● Provide more/better follow-up or reinforcement 57 Focus Group Discussion Guide Intervention #: Date: Interviewer: District: Sector: Participants: Gender: __M/F__ Introduction ● Introduce yourself and explain the purpose for the interview or discussion (if appropriate, present the letter of introduction from Orora Wihaze). ● Reassure respondents that their responses will be held strictly confidential. ● Ask if respondents have any questions. ● Secure respondents’ permission to proceed with the interview. Small Livestock Producers ● Production (Intervention 1) ● End Market Access (Intervention 2) ● ASF Product Market (Intervention 5) Background 1. Please describe your production activity. PROBES ● Type of product produced ● Value chain (i.e., poultry, pig, fish, goat/sheep) INSTRUCTION: Explain that Orora Wihaze project has been assisting [name of business or supported individual] to improve how they work with small livestock producers like yourselves. Confirm whether and how the FGD participants are involved with [name of business or supported individual] and explain that the remaining questions refer to the participants’ interactions with [name of business or supported individual]. Challenges/Constraints 2. What are the primary challenges or constraints you face in this production activity? (EQ 5) PROBE ● Lack of access to finance ● Lack of access to inputs ● Lack of production/business knowledge ● Lack of markets/buyers ● Poor quality Extension Services 3. What kind of extension services did you receive? 58 PROBE ● Access to information ● Access to inputs (e.g., increased access and increased quality) ● Use of inputs ● Access to markets/buyers ● Access to credit ● Access to business development or extension services ● Increased certainty about sales (e.g., use of contract sales arrangement where buyer guarantees to buy and/or at a specific price) Outcomes 4. What has recently changed, due to these extension services, in your production activity in the following areas and how? PROBE ● Increased access to information ● Increased access to inputs (e.g., increased access and increased quality) ● Increased use of inputs ● Increased access to markets/buyers ● Increased access to credit ● Increased access to business development or extension services ● Increased certainty about sales (e.g., use of contract sales arrangement where buyer guarantees to buy and/or at a specific price) 5. What has been the result of these changes to your production activities? PROBE ● Increased production/productivity ● Increased investment in business activity ● Improved quality ● Increased sales ● Increased income ● Increased employment 6. Were there any results (either good or bad) that surprised you? What were they? 7. If you production activity has improved, how has this affected the food purchases and consumption in your household? PROBE ● Focus on whether participants are using any additional sales/income from their small animal production activities to improve their purchases and consumption of nutritious foods. Sustainability 8. Thinking about these improvements in your production activity and their results, how likely is it that these will be sustained over the long term? 9. What factors make it more or less likely that they will be sustained over the long term? 10. What additional support do you need to improve your production activity and its performance? 59 Community Members ● Nutrition (Intervention 4) Nutrition Messaging 1. Have you heard any messages, received any training, or participated in any meetings on the importance of improving your household’s consumption of animal-sourced food? 2. If yes, from what sources? PROBES ● Radio/Urunana ● Community meetings ● Formal training/instruction ● Government ● Community animal health workers or other community volunteers ● Family or friends ● Mobile phone/SMS ● Cooking demonstration ● Church ● Other 3. What do you think is the most effective/trusted way to communicate messages about consuming animal-sourced food to people like you? PROBE ● See Q 12 4. What did you learn from these messages, trainings, or meetings? PROBE ● Important/key messages ● New/improved behaviors or practices Outcomes of Nutrition Messaging 5. How have these messages, trainings, or meetings changed what you think about the importance of consuming animal-sourced food? 6. How important do you think it is to increase your household’s consumption of animal-sourced food? Why? 7. How have these messages, trainings, or meetings changed your consumption of animal-sourced foods? PROBES ● Increased purchases of animal-sourced foods ● Increased dietary diversity (i.e., more different types of animal-sourced foods) ● Increased consumption by women and children (i.e., whether women and children are consuming in addition to male or adult household members) ● Increased women’s influence on household decisions on food purchases and consumption 8. Were there any results (either good or bad) that surprised you? What were they? 9. What are the primary challenges or constraints to increasing your household’s consumption of animal-sourced foods? 60 Sustainability of Nutrition Messaging 10. Thinking about any changes in your household’s consumption of animal-sourced foods, how likely is it that these will be sustained over the long term? 11. What factors make it more or less likely that they will be sustained over the long term? 12. What more can be done to increase your household’s consumption of animal-sourced foods? 61 Focus Group Discussion Guide: Community Volunteers Intervention #: Date: Interviewer: District: Sector: Participants: Gender: __M/F__ Introduction ● Introduce yourself and explain the purpose for the interview or discussion (if appropriate, present the letter of introduction from Orora Wihaze). ● Reassure respondents that their responses will be held strictly confidential. ● Ask if respondents have any questions. ● Secure respondents’ permission to proceed with the interview. Community Volunteers ● Nutrition (Intervention 4) Roles and Responsibilities 1. What type of community volunteer are you? PROBE ● Clarify that the following questions refer specifically to their community volunteer support to improve community nutritional practices, including increased consumption of animal-sourced foods. 2. In your role as a community volunteer, 1) what are your duties, 2) how do you implement them (i.e., what methods and venues do you use), and 3) what objectives are you seeking to achieve? 3. Who are the primary participants/beneficiaries of your volunteer work? Training Received 4. What training did you receive for your role as community volunteer and from whom did you receive the training? 5. To what extent and how did your training incorporate methods for increasing animal-sourced foods? 6. How satisfied are you with the training you received? Did it prepare you for your role as a community volunteer? 7. How could the training be improved? PROBES ● More time/longer ● More practical ● Better trainers ● More convenient ● More follow-up/reinforcement 62 Challenges and Constraints 8. What are the primary challenges or constraints to improving community nutritional practices, including increased consumption of animal-sourced foods? 9. To what extent does your volunteer nutrition work address these challenges or constraints? Outcomes 10. What have been the outcomes of your volunteer nutrition work? PROBES ● Increased knowledge of nutritional importance and topics ● Improved attitudes about nutrition ● Improved nutrition practices ● Increased consumption of animal-sourced foods, including by women and children 11. Were there any outcomes (either good or bad) that surprised you? What were they? Gender and Social Inclusion 12. To what extent and how does your nutrition volunteer work incorporate methods for increasing women’s empowerment and social inclusion of women, youth, or people with disabilities? 13. What have been the outcomes of your volunteer work related to women’s empowerment and social inclusion of women, youth, or people with disabilities? 14. What are the primary challenges for increasing women’s empowerment and the social inclusion of women, youth, or people with disabilities in volunteer nutrition support? Sustainability 15. Thinking about these outcomes of your volunteer nutrition work, how likely is it that these will be sustained over the long term? 16. What factors make it more or less likely that they will be sustained over the long term? 17. What more can be done to improve volunteer nutrition work and its outcomes related to improved nutrition practices, including increased consumption of nutritious foods? 63 Community Volunteers ● Women’s Empowerment and Social Inclusion (Intervention 6) Roles and Responsibilities 1. What type of community volunteer are you? PROBE ● Clarify that the following questions refer specifically to their community volunteer support to improve women’s empowerment and social inclusion of women, youth, or people with disabilities. 2. In your role as a community volunteer, 1) what are your duties, 2) how do you implement them (i.e., what methods and venues do you use), and 3) what objectives are you seeking to achieve? 3. Who are the primary participants/beneficiaries of your volunteer work? Training Received 4. What training did you receive for your role as community volunteer and from whom did you receive the training? 5. How satisfied are you with the training you received? Did it prepare you for your role as a community volunteer? 6. How could the training be improved? PROBES ● More time/longer ● More practical ● Better trainers ● More convenient ● More follow-up/reinforcement Challenges and Constraints 7. What are the primary challenges or constraints to improving women’s empowerment and social inclusion of women, youth, or people with disabilities? 8. To what extent does your volunteer work address these challenges or constraints? Outcomes 9. What have been the outcomes of your volunteer work? PROBES ● Increased knowledge about women’s empowerment and social inclusion ● Improved attitudes about women’s empowerment and social inclusion ● Improved practices related to women’s empowerment and social inclusion 10. Were there any outcomes (either good or bad) that surprised you? What were they? Sustainability 11. Thinking about these outcomes of your volunteer work in women’s empowerment and social inclusion, how likely is it that these will be sustained over the long term? 12. What factors make it more or less likely that they will be sustained over the long term? 13. What more can be done to improve volunteer work and its outcomes related to increased women’s empowerment and social inclusion of women, youth, or people with disabilities? 64 Focus Group Discussion Guide: Community Volunteers Intervention #: Date: Interviewer: District: Sector: Participants: Gender: __M/F__ Introduction ● Introduce yourself and explain the purpose for the interview or discussion (if appropriate, present the letter of introduction from Orora Wihaze). ● Reassure respondents that their responses will be held strictly confidential. ● Ask if respondents have any questions. ● Secure respondents’ permission to proceed with the interview. COMMUNITY VOLUNTEERS ● NUTRITION (INTERVENTION 4) Roles and Responsibilities 1. What type of community volunteer are you? PROBE ● Clarify that the following questions refer specifically to their community volunteer support to improve community nutritional practices, including increased consumption of animal-sourced foods. 2. In your role as a community volunteer, 1) what are your duties, 2) how do you implement them (i.e., what methods and venues do you use), and 3) what objectives are you seeking to achieve? 3. Who are the primary participants/beneficiaries of your volunteer work? Training Received 4. What training did you receive for your role as community volunteer and from whom did you receive the training? 5. To what extent and how did your training incorporate methods for increasing animal-sourced foods? 6. How satisfied are you with the training you received? Did it prepare you for your role as a community volunteer? 7. How could the training be improved? PROBES ● More time/longer ● More practical ● Better trainers ● More convenient ● More follow-up/reinforcement 65 Challenges and Constraints 8. What are the primary challenges or constraints to improving community nutritional practices, including increased consumption of animal-sourced foods? 9. To what extent does your volunteer nutrition work address these challenges or constraints? Outcomes 10. What have been the outcomes of your volunteer nutrition work? PROBES ● Increased knowledge of nutritional importance and topics ● Improved attitudes about nutrition ● Improved nutrition practices ● Increased consumption of animal-sourced foods, including by women and children 11. Were there any outcomes (either good or bad) that surprised you? What were they? Gender and Social Inclusion 12. To what extent and how does your nutrition volunteer work incorporate methods for increasing women’s empowerment and social inclusion of women, youth, or people with disabilities? 13. What have been the outcomes of your volunteer work related to women’s empowerment and social inclusion of women, youth, or people with disabilities? 14. What are the primary challenges for increasing women’s empowerment and the social inclusion of women, youth, or people with disabilities in volunteer nutrition support? Sustainability 15. Thinking about these outcomes of your volunteer nutrition work, how likely is it that these will be sustained over the long term? 16. What factors make it more or less likely that they will be sustained over the long term? 17. What more can be done to improve volunteer nutrition work and its outcomes related to improved nutrition practices, including increased consumption of nutritious foods? 66 Community Volunteers ● Women’s Empowerment and Social Inclusion (Intervention 6) Roles and Responsibilities 1. What type of community volunteer are you? PROBE ● Clarify that the following questions refer specifically to their community volunteer support to improve women’s empowerment and social inclusion of women, youth, or people with disabilities. 2. In your role as a community volunteer, 1) what are your duties, 2) how do you implement them (i.e., what methods and venues do you use), and 3) what objectives are you seeking to achieve? 3. Who are the primary participants/beneficiaries of your volunteer work? Training Received 4. What training did you receive for your role as community volunteer and from whom did you receive the training? 5. How satisfied are you with the training you received? Did it prepare you for your role as a community volunteer? 6. How could the training be improved? PROBES ● More time/longer ● More practical ● Better trainers ● More convenient ● More follow-up/reinforcement Challenges and Constraints 7. What are the primary challenges or constraints to improving women’s empowerment and social inclusion of women, youth, or people with disabilities? 8. To what extent does your volunteer work address these challenges or constraints? Outcomes 9. What have been the outcomes of your volunteer work? PROBES ● Increased knowledge about women’s empowerment and social inclusion ● Improved attitudes about women’s empowerment and social inclusion ● Improved practices related to women’s empowerment and social inclusion 10. Were there any outcomes (either good or bad) that surprised you? What were they? Sustainability 11. Thinking about these outcomes of your volunteer work in women’s empowerment and social inclusion, how likely is it that these will be sustained over the long term? 12. What factors make it more or less likely that they will be sustained over the long term? 13. What more can be done to improve volunteer work and its outcomes related to increased women’s empowerment and social inclusion of women, youth, or people with disabilities? 67 ANNEX 4. KIIS AND FGDS CONDUCTED KIIs Completed Names Position Organization Location 1 Jackson Kwikiriza Senior Program Manager AMIR Kigali 2 Alexis Bizimana Head of Agribusiness Unit Bank OF Kigali Kigali 3 Mathilde Mukasekuru Acting Director General of Animal Resources MINAGRI Kigali 4 Nadine Manzi DCOP, Investment Specialist & Investment Specialist Nguriza Nshore Kigali Michael Bangana 5 Dr. Claire D’ Andre Senior Researcher RAB Kigali 6 Dr. Alphonse Nshimiyimana Executive Secretary RCVD Kigali 7 Gaspard Simbarikure Registration and Licensing Specialist RICA Kigali 8 Erick Willy Uwimana Program Manager RICH Kigali 9 Silvia Muteteli Project Coordinator Urunana Kigali 10 Sarah Uwineza Program Manager Viamo Kigali 11 Theophile Nsabimana Head of Operations Vision Fund Rwanda Kigali 12 David Kubwimana Long Hall Driver Plus Manager OX Kigali 13 Jimmy Dushimimana Product Specialist One Acre Fund Kigali 14 Dr Fabrice Ndayisenga Head of Department of Animal Research &Technology RAB Kigali 15 Antoine Niyitegeka Lead of Intervention 6 Orora Wihaze Kigali BCC Advisor 16 Bosco Kwizera Lead Of Intervention 4 Orora Wihaze Kigali GSI Specialist 68 Names Position Organization Location 17 Ramadhan Bahati Lead of Intervention 5 Orora Wihaze Kigali Nutrition Advisor 18 Dennis Karamuzi Chief of Party Orora Wihaze Kigali 19 Seraphin Niyonsenga Lead of Intervention 1 Orora Wihaze Kigali 20 Lucia Zigiriza Lead of Intervention 2 & 3 And DCOP Orora Wihaze Kigali 21 Yves Bakunda Monitoring And Evaluation Specialist Orora Wihaze Kigali 22 Kennedy Manager AFRISOL Kigali 23 Joseph Gafaranga Managing Director Imbaraga Farmers Organization Kigali 24 Jean Claude Shirimpumu Managing Director Vision Agribusiness Farm Kigali 25 Letitia Mugiraneza Manager SACCO Gasaka Nyamagabe 26 Samuel Ntakirutimana EAR Church Representative RICH Nyamagabe 27 Esperance Nyirasafari Managing Director Hope and Fine Fihs Nyamagabe 28 Nzansabimana Athanaze CAHW RCVD Nyamagabe 29 Donatille Ayinkamiye Manager SACCO Musebeya Nyamagabe 30 Emile Mbaraga Head of Joint Action District Forum (JADF) District Nyamagabe 31 Thaddee Habimana Economic Vice Mayor District Nyamagabe 32 Jean Pierre Bizimana District Animal Resources Officer District Nyamagabe 33 Janvier Ndayambaje Director of Agriculture District Nyamagabe 34 Emmanuel Niyonsenga Inputs Supplier Private/RCVD Nyamagabe 35 Straton Fatahose District Portfolio Manager Orora Wihaze Nyamagabe 69 Names Position Organization Location 36 Jean Baptiste Nsengumurenyi Agro-Vet /Private Veterinary Private/RCVD Nyamagabe 37 Iradukunda Josiane Permanente Secretary of Private Sector Federation Private Nyamagabe 38 Clotilde Mukamusoni AGRO-VE Private/RCVD Nyamagabe 39 Amani Twgirayezu Managing Director Private/RCVD Nyamagabe 40 David Kubwimana Long Haul Driver Plus Manager OX Nyamasheke 41 Donat Tuyinganyiki Pastor Free Methodist Church Of Rwanda RICH Nyamasheke 42 Alexis Murindangw Farmer of Poultry RCVD Nyamasheke 43 Annonciata Mukarugir Manager SACCO Kajongo Nyamasheke 44 Albert Sengambi Director of Agriculture and Natural Resources District Nyamasheke 45 Peruth Bor Shop Jyambere Muhinzi /AGRODEALER Private Nyamasheke 46 Caleb Karangira District Nutrtion Advisor Orora Wihaze Nyamasheke 47 Chantal Niyonkuru Agro-Dealer /Retailer Private Nyamasheke 48 Dorcas Muhawenimana Owner of The Shop Agrovet- Kagano Private Nyamasheke 49 Iraguha Pascal Agent of Gorilla Feed Company GORILLA FEED COMPANY Nyamasheke 50 Paul Kanza Agro -Dealer Private Nyamasheke 51 Jacques Tuyizere Public Relation &Communication Officer District Nyamasheke 52 Felix Sindayiheba District Animal Resource Officer District Nyamasheke 53 Musafiri Ildephonse Agro - Dealer Private Nyamasheke 54 Felicien Mujyambere Agro - Dealer Private/RCVD Gakenke 70 Names Position Organization Location 55 Jean Marie Sindibona Managing Director A-Z Pig Gakenke 56 Dusengimana Anthere Shareholder/GACOPROC O GACOPROCO LTD Gakenke 57 Vivianne Cyuzuzo Manager SACCO Kungahara Gakenke Gakenke 58 Sylvester Rubayiza Community Volunteer RICH Gakenke 59 Augustin Murenzi District Director of Agriculture District Gakenke 60 Alfred Hanyurwabake Chairman of Cooperative Committee Cooperative Gakenke 61 Benjamin Nizeyimana District Portfolio Manager Orora Wihaze Gakenke 62 Murerwa Moreen Agro Dealer Tunga Feed Company Gakenke 63 Dr. Daniel Mbaga Agro-Vets /RCVD Representative at District and Province Level Private/RCVD Gakenke 64 Dushimiyimana Monique Director Agriculture and Animal Resource District Gakenke 65 Uwamahoro Thriphonie District JADF JADF Gakenke 66 Martin Munyakazi Business Bank Loan Officer Unguka Bank Gakenke Gakenke 67 Vumiriya Bonaventure Agro-Vet/Vet Private Kayonza 68 Martin Eliezel Hakundimana Loan Officer SACCO Dukire Kabarondo Kayonza 69 James Ngiruwonsanga Rich Coordinator/ Volunteers RICH Kayonza 70 Regine Mukakalisa Chairman of The Cooperative Committee Cooperative Kayonza 71 Uwamahoro Jaqueline Poultry Producers Private Kayonza 72 Gatanazi Longin Sector Executive Secretary Sector Kayonza 71 Names Position Organization Location 73 Aime Nshimiyimana Head of Joint Action District Forum District Kayonza 74 Margueritte Mukawera District Portfolio Manager Orora Wihaze Kayonza 75 Innocent Nyandwi Agro-Dealer / Input Supplier Private Kayonza 76 Anastazi Uzarama Agrovet Shop Private Kayonza 77 Murenzi Valens Head Business Development Unity/BDU District Kayonza FGDs Completed Names of Organization # of Participants Location Group Name Total Male Female 1 RICH 15 6 9 Nyamagabe Community Members 2 ABIBM 10 0 10 Nyamagabe Twuzuzanye Group 3 RCVD 11 6 5 Nyamagabe Community Members 4 COTUTAMU 12 4 8 Nyamagabe PWD Members 5 RICH 10 2 8 Nyamagabe Community Volunteers 6 RICH 10 5 5 Nyamasheke Community Members 7 Cooperative 11 5 6 Nyamasheke Tugire Ubukire T 8 RCVD 12 4 8 Nyamasheke Community Members 9 Group 10 5 5 Nyamasheke TUZAMURANE Group 10 Cooperative 9 2 7 Nyamasheke SACCO Members 11 RICH 10 4 6 Nyamasheke Community Volunteers 12 TB 9 3 6 Nyamasheke Community Members 13 RICH 10 5 5 Nyamasheke Community Volunteers 72 Names of Organization # of Participants Location Group Name Total Male Female 14 RICH 10 1 9 Gakenke Community Members 15 ABIBM 11 2 9 Gakenke Community Members 16 ABIBM 10 5 5 Gakenke Community Volunteers 17 RICH 10 5 5 Gakenke Community Volunteers 18 RICH 11 10 1 Kayonza Community Members 19 Cooperative 9 4 5 Kayonza KOPABARWI 20 ABADAHIGWA 10 4 6 Kayonza Community Members 21 RICH 9 4 5 Kayonza Savings Group 22 RICH 10 5 5 Kayonza Community Volunteers 23 RCVD/CAHW 9 8 1 Kayonza Community Volunteers Total 238 99 139