P i Prepared for MERCY CORPS 4 April 2014 RISE MID-TERM EVALUATION REPORT RISE Mid-Term Evaluation Report i Table of Contents Executive Summary ............................................................................................................................................................... vi I. Introduction ....................................................................................................................................................................... 1 a. (E)DRC context ........................................................................................................................................................... 1 b. RISE project summary ............................................................................................................................................. 1 II. MTE Scope ......................................................................................................................................................................... 5 a. Objectives ...................................................................................................................................................................... 5 b. Themes .......................................................................................................................................................................... 5 c. Evaluation team .......................................................................................................................................................... 5 d. Methodology ................................................................................................................................................................ 5 e. Constraints and limitations ................................................................................................................................... 7 III. MTE Findings and Recommendations ................................................................................................................. 8 a. Introduction and organization of findings ...................................................................................................... 8 b. Findings in relation to the Results Framework ............................................................................................ 8 i. Strategic Objective 1: Smallholder farming households in target areas have increased and diversified production and profit. .......................................................................................................................... 8 ii. Strategic Objective 2: Improved nutritional status among pregnant and lactating women and children under 5 in target areas ...................................................................................................................18 iii. Strategic Objective 3: Strengthened community governance of food security in target areas .................................................................................................................................................................................28 IV. General and Cross-cutting Observations and Recommendations ..........................................................37 a. General Observations .............................................................................................................................................37 b. General Recommendations ................................................................................................................................. 39 c. Cross-cutting (BCC and gender) ........................................................................................................................41 i. BCC ............................................................................................................................................................................41 ii. Gender ....................................................................................................................................................................43 d. Commodities .............................................................................................................................................................45 V. Evaluation Themes and Conclusions ....................................................................................................................49 RISE Mid-Term Evaluation Report i List of Tables Table 1: Summary of RISE Geographic Shift in 2013 ................................................................................................ 2 Table 2: Role of RISE Implementing Partners, Original and Updated ............................................................... 3 Table 3: Summary of RISE Key Beneficiary Groups and Targets ......................................................................... 4 Table 4: Summary table of the main implementation differences between MC and CRS/Caritas ......... 9 Table 5: Commodities Planned vs. Distributed, FY2012-FY2013 .....................................................................45 Table 6: Evaluation Theme Conclusions ......................................................................................................................49 RISE Mid-Term Evaluation Report i List of Annexes Annex A: Summary of Interventions by Geographic Area Annex B: Project achievements in the scaled down areas prior to the geographic shift Annex C: RISE Project Map Annex D: RISE MTE Methodology Annex E: RISE MTE Terms of Reference Annex F.1: List of Documents Consulted Annex F.2: Meetings and interviews with RISE Staff and Consortium - List of Participants Annex F.3: RISE MTE Fieldwork Detail Annex F.4: RISE MTE Participatory Workshop - Agenda Annex F.5: RISE MTE Participatory Workshop – List of Participants Annex F.6: RISE MTE Data Collection Tools Annex F.7: RISE MTE Field Work Calendar Annex G: RISE Results Framework Annex H1: MTE Summary Findings in Relation to the Results Framework Annex H2: Proposed Modified RISE Results Framework Annex I: MTE Summary Findings in Relation to the Evaluation Questions Annex J: SO1 Implementation Differences between MC and CRS Annex K: Summary Table of SIMAMA New Varieties Annex L: Sample exit and sustainability strategy from the CRS MYAP in Madagascar Annex M1: RISE Activity Protocol template Annex M2: Protocol Example 1 from the CRS MYAP in Madagascar Annex M3: Protocol Example 2 from the CRS MYAP in Madagascar Annex N: RISE – Commodity Issues RISE Mid-Term Evaluation Report ii Acknowledgements Oxu Solutions would like to acknowledge the many people who made this work possible. First and foremost, we would like to thank the Chief of Party for taking the exercise seriously and encouraging a critical exercise and frank reporting of our findings. Our deep appreciation also goes to Design, Monitoring, Evaluation and Learning Specialist for Mercy Corps and M&E Coordinator for RISE, who organized the evaluation and managed logistics for us in a challenging environment. We would also like to thank: 1. Our three interpreters 2. RISE participants who met with us in the field for the FGDs and the KIIs 3. The MoH staff in Birambizo Health Zone: the Medical Director, the Health Management Team in Katwe (Equipe Cadre de Zone de Santé); and the Health Center staff in the following Health Areas: Bwalanda, Kikuku, Bambu and Kabizo 4. RISE staff with a special thank you to staff in Nyanzale and Katwe who set up FGDs and interviews, and those bases which hosted us 5. The two drivers who took us to Birambizo and Rutshuru Finally, Oxu takes full responsibility for the content of this work. Any errors or misrepresentations in the document are our own. RISE Mid-Term Evaluation Report iii Abbreviations and Acronyms Abbreviations/ Acronyms Definition ARR Annual Results Report BCC Behavior Change Communication BXW Banana Xanthomonas Wilt CARG Conseil Agricole Rural de Gestion CARS Conseil Agricole Rural de Secteur (Secteur/Chefferie Rural Agriculture Council) CART Conseil Agricole Rural du Territoire (Territory Rural Agriculture Council) CRS Catholic Relief Services CSB Corn Soy Blend CU2 Children under 2 years of age CU5 Children under 5 years of age DIP Detailed Implementation Plan DRC Democratic Republic of Congo DRR Disaster Risk Reduction EDRC Eastern Democratic Republic of Congo ENA Essential Nutrition Actions ETD Entité Territoriale Décentralisée (Decentralized Territorial Entity) EWS Early Warning Systems FFP Food For Peace (USAID) FFS Farmer Field School FFT Food For Training (used in SO1 by MC) FFW Food for Work (used in SO3 by MC) FGD Focus Group Discussion FHH Female Heads of Household (also Female-headed Households) FL Farmer Leader FM Farmer Multiplier (MC) FS Food Security FY Fiscal Year ha Hectare, equivalent to 100 ares HA Health Area (Aire de Santé, a sub-division of Health Zone) HC Health Center HH Households HKI Helen Keller International HMT Health Management Team (for Birambizo Health Zone) HQ Headquarter HZ Health Zone (Zone de Santé) IGA Income Generating Activities IMCI Integrated Management of Childhood Illnesses INERA National Institution for Agricultural Research IP Implementing Partner (MC, CRS, Caritas and CEDERU) IPM Integrated Pest Management IPTT Indicator Performance Tracking Table RISE Mid-Term Evaluation Report iv IR Intermediate Result KAP Knowledge, Attitude and Practice KII Key Informant Interview KPC Knowledge, Practice and Coverage LDC Locality Development Committee LDP Locality Development Plan LQAS Lot Quality Assurance Sampling MC Mercy Corps M&E Monitoring and Evaluation Medical Director Médecin Chef de Zone ML Mother Leader (Maman Leader) MoA Ministry of Agriculture MoH Ministry of Health MT Metric Ton MTE Midterm evaluation MUAC Mid-Upper Arm Circumference MYAP Multi-Year Assistance Programme (USAID) OFAC Office of Foreign Assets Control OFSP Orange-Fleshed Sweet Potato PDM Post-Distribution Monitoring PG Producer Group (OP – Organisation de Producteurs) PH Post-Harvest PHH Post-Harvest Handling PICS Purdue Improved Cowpea Storage (hermetic plastic bag) PLW Pregnant and Lactating Women PM2A Preventing Malnutrition in Children under 2 Approach PNC Prenatal Consultation PRONANUT National Nutrition Program RECO Community Health Volunteer (Relais Communautaire) RISE Resources to Improve Food Security in Eastern DRC SBCC Social & Behavior Change Communication SENASEM Service National de Semences, DRC’s office for seed certification SGBV Sexual and Gender-based Violence SHF Smallholder Farmers SILC Savings and Internal Lending Communities SO Strategic Objective TA Technical Advisor (MC) TE Technical Evaluators (MC) TOR Terms of Reference U2 Children under 2 years of age U5 Children under 5 years of age UNDP United Nations Development Programme VA Village Agents (for VSLA support) VC Value Chain VCA Value Chain Analysis RISE Mid-Term Evaluation Report v VDC Village Development Committee VSLA Village Savings and Loans Association (SILC for CRS/Caritas) WASH Water, Sanitation and Hygiene RISE Mid-Term Evaluation Report vi Executive Summary The Resources to Improve Food Security in Eastern DRC (RISE) project, implemented by a consortium led by Mercy Corps, is a five year USAID/ Office of Food for Peace-funded multi-year assistance program (MYAP) with an overall program goal to reduce food insecurity among vulnerable populations in North Kivu. Mercy Corps’ principal international partner is Catholic Relief Services (CRS) with its implementing partners, Caritas and CEDERU. This Mid-Term Evaluation (MTE), conducted in January – February 2014, assesses the strengths and weaknesses of program implementation, identifies factors associated with activities’ greater or lesser performance, and identifies early evidence of whether the activities are leading to change (positive or negative) and factors that may be impeding or promoting positive change. Finally, the MTE offers a number of recommendations for the rest of the project. The period covered by this MTE report starts with the project’s initiation (September 2011) and goes through December 2013. The MTE was prepared and carried out by a team of evaluators using qualitative methods for primary data collection. This included: a desk review of RISE project and other relevant documents; meetings and interviews with key RISE staff and partners of the RISE consortium; Focus Group Discussions (FGD) with project participants and other community stakeholders; Key Informant Interviews (KII) with community and institutional stakeholders; direct observations of project activities; and a participatory workshop with RISE project staff to discuss initial findings and recommendations. In general, our field work revealed a dedicated project staff logging long hours and working diligently in a difficult and risky operating environment. It is worth noting that the food insecurity situation in Eastern DRC (EDRC) in general and North Kivu in particular is exacerbated by the nearly non-existent infrastructure; governance challenges at every level of government; little to no provision of services by the state; unpredictable markets and frequent insecurity. The RISE project has been particularly challenged by insecurity in many project areas, which was the basis for a significant geographic shift away from certain parts of Rutshuru Territory in Year 2. Key findings can be summarized as follows: Program design: From the outset, the program design for RISE was too ambitious given the context of working in Eastern DRC, and in many cases does not seem to reflect the realities of the target areas. Mercy Corps and its partners underestimated the time it would take to register beneficiaries and begin project activities, overestimated the near-term opportunities for smallholder farmer integration into market systems, and attempted to implement a shopping list of possible activities related to the three main project components or Strategic Objectives (SOs): Agriculture/Livelihoods, Nutrition, and Governance. The negative effects of the design flaws have in many ways not yet been overcome. Strategy and Implementation: The project suffers from a lack of a clear overall impact strategy for the three main components as well as for the cross-cutting themes of gender and behavior change communication (BCC). As a result, the same component and even the same activity may be implemented differently within the same geographic area, and approaches for some activities have changed significantly over time with staff changes. Unfortunately, implementation has not always been based on the particular context of individual communities or based on learning through project implementation. In addition, there are few linkages between SOs, there is limited integration of beneficiaries across project components, and there is little in the way of a concrete exit and sustainability strategy. Within the SOs and cross-cutting themes: RISE Mid-Term Evaluation Report vii 1) Agriculture: At the time of the MTE, two of the Intermediate Results (IRs) (agriculture marketing and financing activities) of the Results Framework had not had any activities undertaken, except for the Village Savings and Loan Associations (VSLAs). In summary, the project thus far does not seem effective in reaching the aims for this SO due to: a lack of systematic targeting; a lack of project integration (within the SO, across the three SOs, and through the cross-cutting themes); and a lack of appreciation for the smallholder farmer’s perspective. 2) Nutrition: The nutrition interventions are based on the Preventing Malnutrition in Children Under 2 (PM2A) strategy and seem relevant and appropriate for the context. RISE is making good progress on a number of the Outcomes for the three Intermediate Results, especially improved diet for pregnant and lactating women and children under 2 via the consumption of rations; increased use of health services, especially for prenatal care; and increased knowledge and adoption of nutrition and hygiene practices. One important issue that impedes progress is the quality of both the preventive and curative services is not up to Congolese government standards due to 1) chronic shortfalls in the Ministry of Health (MoH) provision of essential medicines and supplies to the Health Centers and 2) delays in the RISE-sponsored training program for certain topics for the health service providers. A second issue that will affect sustainability of impact in terms of improved nutritional status is the lack of integration between SO1 and SO2. The SO1 agricultural activities are having little effect on the SO2 households due to a lack of strategic targeting and the scale of SO1 activities. 3) Governance: The Governance component appears to have suffered from difficulties of high turnover in leadership, compounded by an unrealistic design as reflected in the Results Framework and associated activities. The Governance team has devoted considerable efforts to initiating activities from the long list in the Results Framework, but unfortunately the result is a series of unconnected activities that would appear unlikely to contribute meaningfully to strengthened community governance for food security. 4) BCC: Although BCC is not the integrating factor envisioned in the project design and there have been delays in implementation, both Mercy Corps and CRS are now making concerted efforts to put behavior change at the forefront of activities, using formative research, multiple channels, a variety of activities, and creativity to promote improved practices and behaviors across SOs. 5) Gender: Women’s participation in project activities is greater than 50 percent for most components; however, meaningful impact level change is less readily apparent. A strong gender assessment was completed in September 2012, but little progress has been made in implementing the recommendations. Management and Monitoring & Evaluation (M&E): The project has had disruptively high staff turnover, particularly at senior levels. Turnover includes several changes in the Chief of Party position and at least one turnover for each of the three SO lead positions. The result has been gaps in leadership coupled with inconsistent and delayed project implementation. In addition, there is a serious lack of documentation and systematic monitoring for many aspects of RISE, including management of quality. There is little systematic recording of decisions, data analysis reports, response to critical internal evaluations, or even brief reports on successful campaigns. High staff turnover – especially in management positions – coupled with minimal documentation and little institutional memory are recipes for failure. Finally, while M&E reporting shows that RISE has reached high percentages of its beneficiary numbers, the evaluators had difficulties understanding how RISE arrived at overall beneficiary figures as a whole and for each SO. In addition, the project monitoring system has no way of tracking beneficiaries that are targeted under multiple objectives, RISE Mid-Term Evaluation Report viii which severely limits the project’s ability to understand program integration across the three objectives. Accountability: While MC and partners use a complaint hotline that beneficiaries can access on distribution days, our findings indicate that there is no additional project accountability system in place (though at the time of the field work, a Community Accountability Coordinator had been hired and development of a community accountability system was underway). For an undertaking as complex as the RISE project, a much more comprehensive accountability system should have been established at the outset of the project. Even a systematic (i.e. regular and structured) monitoring system at the field level by managers and M&E staff combined with a standard feedback mechanism for project participants could have improved accountability significantly. Progress related to Evaluation Themes: For the MTE, the evaluation team used the following evaluation themes as lenses for analysis:  Relevance, appropriateness and effectiveness  Project targets, early outcomes, goal-setting  Community engagement and accountability  Sustainability and spillover effects  Adaptability to the dynamic security situation Based on these themes, we have drawn preliminary conclusions and used a color-coded system of assigning a green (seems on track to achieve this by project end), yellow (some possibility of achievement by project end, but serious weaknesses must be overcome), or red (highly unlikely to be achieved by project end without fairly dramatic and immediate change) color to each theme, presented in the table below. Table: Evaluation Theme Conclusions Theme and Conclusions Color Relevance, Appropriateness, Effectiveness: To date, activities have been scattered and not well￾coordinated or integrated across components. There is no strategic targeting of households (HHs), which would maximize impact if done well. Project design was overly ambitious (for example, many planned activities in SO1 have not even begun), and much of the project looks more like an emergency intervention rather than a longer-term development program. Adaptability and Security: The project has shown adaptability to insecurity but we found little evidence of contingency planning, either internally or as part of discussions with communities around project activities. Neither did we find much documentation of how decisions were made related to insecurity. Ongoing and proposed geographic expansion puts the project at risk of being overstretched geographically, with potential negative consequences for quality implementation. Project Targets, Early Outcomes, and Goals: From a numbers perspective, the project has been impressive in its ability to meet many of the beneficiary targets, especially in terms of total households and percentage of female beneficiaries. Yet the quality with which activities have been implemented remains a strong concern, especially the contribution to outcomes and the project objectives. Community Engagement and Accountability: Relationships with communities do not seem like partnerships and project accountability has been extremely limited. Sustainability and Spillover: Other than VSLAs, the local dissemination of new cassava and bean varieties, and some of the health and nutrition-related behaviors and practices being promoted under SO2, there is little indication of potential for sustainability for most of the results. Spillover effects seem incidental rather than planned. There are no well-defined sustainability and exit strategies in place. RISE Mid-Term Evaluation Report ix Moving forward, it will be vital that in the remaining time for RISE implementation, Mercy Corps and CRS focus on changes that will: improve systems (M&E, accountability, documentation, etc.); ensure sustainability/appropriate exit; scale back expectations and activity levels; and maximize geographic and community level impact. The most critical recommendations we would emphasize include: 1. Modify the RISE Results Framework, especially for SO1 and SO3 Based on the challenges to date, a redesign of the Results Framework needs to take place to have more modest, integrated and realistic objectives focused on long-lasting impact. While RISE has reached a significant number of its target beneficiary numbers, our findings have suggested that the quality of implementation and level of medium-to-long term impact on the beneficiaries reached to date is in question. The lack of strategic targeting means that each SO targets different individuals, households and communities, resulting in a lost opportunity to maximize impact by concentrating activities on the same households and communities where feasible. A proposed modified RISE Results Framework with a more narrow focus, especially for SO1 and SO3, is provided as Annex H2 for Mercy Corps’ and partners’ consideration.1 Once the Results Framework has been modified, the project should also review and revise as necessary project indicators to ensure that they are more meaningful and that there is a better balance of qualitative indicators to accompany the lengthy data monitoring sheets. A quality assurance plan also needs to be put in place which would include regular and structured visits from project managers and M&E staff to ensure the quality of achievements and consistency of data reported with realizations in the field. 2. Create a more integrated, holistic food security project During our field work we found very little evidence of project integration across SOs. For example, during SO2 FGDs with PM2A participants and with Mother Leaders, very few of the women indicated that their spouse or other household members participated in any RISE activities such as agriculture, VSLA or Food for Work. Similar findings were made with SO1 and SO3 participants. This compartmentalizing by sector runs counter to the purpose of a MYAP where the SOs should be tightly integrated and mutually reinforcing in order to maximize the effects of cross-sectoral programming. The lack of integration and strategic targeting constitutes a major weakness in project implementation. RISE staff must make a concerted effort to create an integrated project that adds up to more than the sum of its sectoral parts. 3. Implement a system for better project documentation As a development project of this size and scale, especially with the changes that have occurred, the project needs to document things more consistently and effectively. It was extremely challenging to find key project documents, and many were simply not available. This may require taking a pause in implementation for two weeks to develop these with the project teams, or seek external support from the respective organizations headquarters, or an external consultant, but we do believe this work of revising the Results Framework and the documents suggested below will help the work bet more focused and efficient. Specifically, the project should place emphasis on the following key documents:  SO and other key strategies (e.g. BCC): Each SO and cross-cutting theme should have an overall strategy document which links key project activities to the Results Framework, the Indicator Performance Tracking Table (IPTT), and the Detailed Implementation Plan (DIP), as well as the Activity Protocols and Exit and Sustainability Strategy mentioned below. 1 The SO2 part of the proposed modified Results Framework is largely the work of the SO2 team, assisted by CRS headquarters and with input from the SO2 evaluator. RISE Mid-Term Evaluation Report x These documents should provide a vision for the SO or the theme through the end of the project and beyond, including integration of SOs. Such documents should be dynamic and a process articulated for regular review and updating. This process should involve technical support from the consortium partner HQs to mitigate dramatic changes in implementation every time there is turnover in project staff.  Clear and concrete sustainability and exit strategies: For each SO and cross-cutting theme, the project should develop more detailed sustainability and exit strategy documents that identify the targeted community structures/actors, explaining 1) what their intended role would be by the end of the project and beyond (which should be linked to their role in other DRC policy documents) and 2) how the project will reinforce their capacity to play this role. These documents should be able to link the Results Framework Intermediate Results and outputs to the activities and indicators (see Annex L for a for a sample exit and sustainability strategy from the CRS MYAP in Madagascar, which CRS Madagascar agreed could be shared).  Protocols for key project activity packages: Protocols will help ensure consistency of implementation and should highlight the specific steps to be taken for activity implementation for key project activity packages and how they contribute to quality of outputs related to the Results Framework. They permit the review and agreement by Mercy Corps, CRS, Caritas, and CEDERU, and also can permit reviews by respective technical advisors of those organizations. The protocols can plan for differences based on context and give guidance as to when such variation would be appropriate or not, which can help in monitoring and follow-up of these activities. These protocols do not need to be extremely long initially, but even basic protocols would likely remove some of the issues of unexplained variation in implementation seen in and across areas, particularly for SO1 and SO3. Such protocols would also ensure better quality management in the scale-up of SO2 activities. A sample template is provided in Annex M1, along with two examples in Annexes M2 and M3 from the CRS MYAP in Madagascar. The latter annexes, while lacking a few details which are suggested in the template, show concrete examples of documenting project activities. RISE Mid-Term Evaluation Report 1 Oxu Solutions LLC I. Introduction a. (E)DRC context Ranked 186th out of 187 countries in the UNDP’s 2013 Human Development Index, the Democratic Republic of Congo (DRC) faces critical challenges that affect its capacity to ensure a peaceful and sustainable living for its inhabitants. While showing signs of political and economic recovery following the transitional process (2005) that saw insecurity declining and international donors supporting and funding programs anew, the country is still prone to violence and insecurity. This is especially the case in Eastern DRC (EDRC), as evidenced by the M23 rebel group taking over the city of Goma in November 2012. The rebels officially withdrew from Goma in December 2012 but continued to have a presence in Kiwanja and Rutshuru until October 2013. Sporadic fighting and security incidents are the norm in EDRC, including many project areas. Despite the potential stemming from its abundant natural resources, including 135 million hectares of fertile farmland, the country faces extremely high levels of chronic food insecurity. In North Kivu, households (HHs) can barely meet half of their minimum energy requirements in terms of food consumption. Food insecurity leaves 69% of the province’s population undernourished and 58% of children under 5 (CU5) stunted2. Underlying these statistics are all of the major causes of food insecurity including gaps in food availability, access and utilization. In RISE project areas, 51% of CU5 are stunted. The baseline assessment for the RISE project found that the major constraints to improving HH production and marketing include: lack of access to arable land; large pre- and post￾harvest losses; crop disease; nearly universal inability to access credit; long distances to market centers; and taxation (informal and formal) on products going to markets. Within health and nutrition, challenges cited include the following: lack of dietary diversity; poor hygiene and sanitation practices and lack of infrastructure; poor understanding of optimal nutrition actions; and underutilization of de-worming medicines and iron and vitamin A supplementation. Adding to these challenges, the baseline also found that communities are very weakly organized and lack the cohesion to collectively address development challenges. Popular trust in local government is extremely low. The food insecurity situation in EDRC in general and North Kivu in particular is exacerbated by the incredibly challenging operating environment, which is characterized by: nearly non-existent infrastructure; corruption at every level of government; little to no provision of services by the state; and frequent insecurity. In such an environment, the baseline assessment concluded that “the key to the success of this program lies in an objective and realistic assessment of the situation, the integrated implementation of activities across the three Strategic Objectives, a results-based management style, a judicious use of food resources, the reorganization of communities and the strengthening of relationships between local governments and populations with a focus on inclusion and accountability.” One interpretation of this comment is that development in EDRC must be undertaken with a thorough understanding of the local context and a healthy pragmatism about what can be accomplished. b. RISE project summary In September 2011, a consortium led by Mercy Corps (MC) and including sub-grantees Catholic Relief Services (CRS) and Helen Keller International (HKI) began a five year USAID FFP-funded multi-year assistance program (MYAP) – Resources to Improve Food Security in Eastern DRC 2 Ministry of Planning and UNICEF. “Enquête Par Grappes à Indicateurs Multiples (MICS) – 2010, Résultats Préliminaires,” September 2010, p. 23. RISE Mid-Term Evaluation Report 2 Oxu Solutions LLC (RISE)3 – with an overall program goal to reduce food insecurity among vulnerable populations in North Kivu.4 The RISE program strategy, which utilizes an integrated, gender-sensitive approach focused on behavior change, relies on three Strategic Objectives (SO) to achieve the goal:  SO1: Smallholder farming HHs in target areas have increased and diversified production and profit (the project’s “Agriculture” component)  SO2: Improved nutritional status among pregnant and lactating women and children under 5 in target areas (the project’s “Nutrition” component)  SO3: Strengthened community governance of food security in target areas (the project’s “Governance” component) RISE focuses on improving agricultural production, controlling and managing crop disease, increasing HH incomes, and preventing childhood malnutrition. Activities specifically address the challenges that vulnerable HHs face regarding food availability, access, and utilization. Activities aim to build communities’ resilience to shocks by introducing and encouraging adoption of sustainable HH coping mechanisms. The program includes a substantial focus on governance and disaster risk reduction (DRR). RISE originally targeted five Health Zones (HZ) in North Kivu, four in Rutshuru Territory (Rutshuru, Rwanguba, Binza and Birambizo) and one in Nyiragongo Territory (Karisimbi). (See Table 1 below for details). In 2013, there was a shift in geographic coverage areas due to insecurity as a result of M23 rebel activities in the Rutshuru area. The M23 took the city of Goma on November 20th, 2012 and then withdrew in December 2012 to allow for peace talks with the Congolese government. Parts of Rutshuru have since experienced sporadic fighting and insecurity, which led to numerous temporary suspensions of activities. In addition, HKI withdrew from the consortium in November 2012 because of insecurity. In 2013, citing ongoing insecurity in parts of Rutshuru Territory, RISE made the decision (with donor approval) to significantly scale down activities in Rwanguba, Binza and Rutshuru HZs in Rutshuru Territory and Karisimbi HZ in Nyiragongo Territory; all activities were concluded except for a final season of seed multiplication, support to women and youth associations, and follow-up with Village Savings and Loan Associations (VSLAs). The shift did not entail any changes in the overall numbers of beneficiaries but did include scaling up activities in Birambizo HZ as well as geographic expansion to two new territories (Lubero and Beni), referred to in this document as Butembo/Beni for the territorial capitals. Table 1 below provides a summary of the geographic shift. Table 1: Summary of RISE Geographic Shift in 2013 Territory Original areas at same scale as started Original areas scaled down Areas of expansion Nyiragongo Karisimbi Rutshuru Birambizo Health Zone Rutshuru, Binza and Rwanguba Health Zones Additional Health Areas of Birambizo HZ 3 Locally the program is called “SIMAMA,” which means “rise” in Swahili. 4 The overall goal for Title II non-emergency funding in DRC is to reduce food insecurity among chronically food insecure households in the DRC. The RISE program goal is meant to contribute to the overall Title II goal for DRC multi-year programming. RISE Mid-Term Evaluation Report 3 Oxu Solutions LLC (SO2) Lubero, Beni Lubero Territory (Musienene, Masereka HZs); Beni Territory (Kalunguta HZ) Annex A provides a summary of the Results Framework, and which outcomes are targeting which geographic locations (based on project achievements in Years 1-2 and current planning for Years 3- 5). Annex B provides project achievements in the scaled down areas prior to the geographic shift, which allows for recognition of the work that was done and then essentially stopped in numerous project areas approximately 18-22 months after the project began. A project map showing the geographic intervention areas is also provided as Annex C. Following HKI’s withdrawal from the consortium, the project also modified the role of each implementing partner (IP), as is shown in Table 2 below. Table 2: Role of RISE Implementing Partners, Original and Updated Component SECTOR LEAD – Original SECTOR LEAD – Updated Strategic Objective 1: Agriculture Value Chains and Market Development MC MC Staple Crops CRS CRS Crop Disease Management CRS CRS Microcredit MC MC, CRS Strategic Objective 2: Nutrition Nutrition CRS CRS Strategic Objective 3: Governance Governance MC MC Disaster Risk Reduction MC MC Cross-cutting Social and Behavior Change Communication HKI MC, CRS Gender MC MC Monitoring and Evaluation MC MC Commodity Logistics MC MC CRS is currently the current main project sub-grantee and works through local partners Caritas and CEDERU to implement the CRS-led activities. RISE also supports and collaborates with relevant government agencies, including the Ministry of Agriculture, Fish and Livestock (IPAPEL) and the national seed service (SENASEM); the Ministry of Health (MoH) and its institutions (specifically the local HZ structure; the National Nutrition Program (PRONANUT); and the provincial MoH office); and local governance structures, including CARGs (Conseil Agricole Rural de Gestion) and Community Development Committees. RISE aims to reach a total of 62,965 beneficiary HHs during the life of the project. As of the end of September 2013, RISE had benefited 50,564 HHs (See Table 3 below). While RISE has achieved a high percentage of target beneficiaries overall, we had difficulties understanding how RISE arrived at the overall beneficiary figures for the project as a whole and for each Strategic Objective. In particular, we noted the following:  Total beneficiaries are reported on as “benefiting households,” but we were not able to piece together how the project reached over 50,000 HHs in the first two years of implementation. RISE Mid-Term Evaluation Report 4 Oxu Solutions LLC  SO1 beneficiaries are reported on as “individual beneficiaries,” which are approximated by multiplying the number of direct beneficiaries of SO1 activities (i.e. benefiting SO1 households) by 5.4 (which is the average household size used by the project). In addition, a beneficiary in SO1 includes any person that has received any sort of assistance through RISE, including approximately 4,000 direct beneficiaries whose only participation in the project has been as seed distribution beneficiaries from seed produced on seed multiplication plots. Finally, while the figure in the IPTT for Year 2 cumulative individual beneficiaries is greater than 80,000, the evaluation team never received a full breakdown of SO1 beneficiaries reached, despite repeated requests for further information.  SO2 beneficiaries are also reported on as “individual beneficiaries” but our understanding is that they are being calculated by counting the number of individual pregnant and lactating women (PLW) and children under two years of age (CU2) served through SO2 activities.  SO3 beneficiaries are defined as “individual beneficiaries” participating in SO3 activities, although there is a discrepancy whereby the IPTT suggests 19,800 for the LOA target but the FY 2013 ARR suggests the target to be 12,000. The SO3 section has an additional brief discussion of SO3 beneficiary counting.  The project monitoring system has no way of tracking beneficiaries that are targeted under multiple objectives, therefore there is no accurate data regarding beneficiary overlap among the project’s three sectors. Table 3: Summary of RISE Key Beneficiary Groups and Targets5 Beneficiary Group # Reached (Sept 2013) LOA Target % of LOA Target Achieved Benefiting Households 50,564 62,965 80% SO1 Beneficiaries (individuals)6 82,250 61,742 133% SO2 Beneficiaries (individuals)7 18,648 65,160 29% SO3 Beneficiaries (individuals) 2,140 19,800 11% Females (as % of program beneficiaries, non-cumulative) 76% (Yr 1) 62% (Yr 2) 50% N/A 5 This information is taken from the IPTT from Year 2 of the project. 6 As noted above, for SO1 the individual beneficiaries are approximated by multiplying the number of direct beneficiaries (i.e. benefiting households) by 5.4 (which is the average household size used by the project). 7 As noted above, for SO2 the individual beneficiaries are calculated by adding the number of PLW and CU2 served by SO2 activities. RISE Mid-Term Evaluation Report 5 Oxu Solutions LLC II. MTE Scope a. Objectives The overall goal of the Mid-Term Evaluation (MTE) is to: assess the strengths and weaknesses of program implementation, identify factors associated with activities’ greater or lesser performance, and identify early evidence of whether the activities are leading to change (positive or negative) and factors that may be impeding or promoting positive change. The period covered by this MTE report starts with the project’s initiation (September 2011) and goes through December 2013. To achieve the goal of the MTE, the evaluation objectives as laid out in the Terms of Reference are as follows:  To evaluate the strengths and weaknesses of program implementation and the quality of outputs  To seek evidence of changes and evaluate how the changes relate to progress toward program objectives  To evaluate program strategies  Based on the findings from the first three points, to recommend adjustments to program implementation or design  To facilitate a workshop to engage various stakeholders, especially program staff and the donor, in an analysis and reflection on the evaluation results. A full description of the evaluation objectives, themes, and proposed methodology are provided in the evaluation methodology document in Annex D. b. Themes From the MTE Terms of Reference document prepared by Mercy Corps (Annex E), there are clearly a number of questions of interest which cut across the three SOs. For the purposes of this exercise, the evaluation team has considered the following themes as lenses of analysis (see Annex D for full descriptions of the themes):  Relevance, appropriateness and effectiveness  Project targets, early outcomes, goal-setting  Community engagement and accountability  Sustainability and spillover effects  Adaptability to the dynamic security situation c. Evaluation team The evaluation team conducting the field work consisted of the Team Leader, who also served as lead for SO3 (Governance); an SO1 (Agriculture/Livelihoods/Commodities) lead accompanied by an SO1 agronomist; and an SO2 (Nutrition) lead that also covered Behavior Change as a cross￾cutting issue. The field team was supported remotely by an Oxu team of an Associate Consultant and Senior Consultant. d. Methodology Given the MTE’s overall goal and objectives, the Results Framework served as the primary source for designing the evaluation materials. The evaluation design also drew on the extensive list of evaluation questions articulated by RISE (see Annex E); Oxu prioritized these questions, using the Results Framework and considerations of time and resource constraints to highlight certain questions and to accord a lower priority to others. RISE Mid-Term Evaluation Report 6 Oxu Solutions LLC The MTE was prepared and carried out using qualitative methods for primary data collection. Monitoring data that had been collected prior to the start of the MTE was also reviewed during the desk review. The methods included:  A desk review of RISE project documents (project proposal, documents related to SOs, etc.), annual and other progress reports submitted to the donor, and donor documentation related to Title II programming in DRC (such as the FY2010 Request for Applications and DRC Country Specific Guidance; the Bellmon Estimation for DRC; etc.). The review sought to be as comprehensive as possible, though a number of crucial project documents were received only a few days prior to our departure or after the team arrived in DRC. A list of documents consulted is included in Annex F1.  Meetings and interviews with key RISE staff and partners of the RISE consortium. A list of these meetings and interviews including the participants in each is included in Annex F2.  Focus Group Discussions (FGD) with project participants and other community stakeholders. See Annex F3 for detailed information for the FGDs conducted, including numbers of each FGD completed per SO and per geographic location, as well as the number of participants (broken down by gender) and participant description for each FGD.  Key Informant Interviews (KIIs) with community and institutional stakeholders. Annex F3 also includes detailed information for the KIIs conducted, including the number of KIIs completed per SO and per geographic location, as well as the title/position of the key informant for each interview.  Direct observations of project activities such as model farms, rabbit breeding centers, seed multiplication fields and commodity storage in community-level food depots  A participatory workshop with RISE project staff to discuss initial findings and recommendations. The agenda for this meeting is provided in Annex F4 and the list of workshop participants can be found in Annex F5. The methodology plan called for purposive sampling to select communities in which to conduct field work, with a view to cover: 1) diversity of geographic area; 2) focus on areas where activities have been ongoing for at least a year to see tangible results;8 and 3) a certain amount of variation of quality as perceived by project staff. Data collection instruments – including FGD guides and KII question guides – were drafted for each SO prior to arrival in-country and then field-tested and finalized upon arrival. The individual SO tools also included questions related to cross-cutting issues such as gender, behavior change, knowledge management, commodity preferences, etc. A separate commodities interview guide was also prepared for conducting interviews related to commodity management with RISE staff. All of the question guides for the data collection tools are included as Annex F6 At the end of each day’s data collection the team members met to discuss observations and identify areas of further questioning to staff members or in subsequent field work. The data from the field work was triangulated through comparing findings for a particular topic with different stakeholders; using project documents to check facts; direct observation of activities; and verifying information with stakeholders, especially to gain the historical perspective and the evolution of the project. 8 As noted above, due to insecurity the project shifted geographic target areas in 2013. As a result, the new geographic areas the project expanded into (in parts of Lubero and Beni territories) were not included in the evaluation team’s field work given activity implementation had been ongoing for less than a year. RISE Mid-Term Evaluation Report 7 Oxu Solutions LLC e. Constraints and limitations A list of sites visited per Strategic Objective can be found in Annex F3. Whereas the number of FGDs/ KIIs/ Interviews with project staff is fairly consistent in regards to what was planned (though with some discrepancies), the evaluation team experienced unforeseen situations that may have affected the quality of the information collected. Examples include: 1) the composition of Focus Groups was not always in line with what the team requested and 2) some individual interviews turned into small group discussions. These discrepancies occurred more for SO1 and SO3 work than for SO2. Once in country, the evaluation team further refined the geographic sampling with the help of RISE staff. However, following through on the geographic sampling as it was originally intended proved to be difficult due to delays in receiving project maps, a lack of harmonized lists of project sites, and a lack of documents/maps showing where there was overlap of SOs in the intervention areas. Overall, the calendar for field visits (see Annex F7) was very condensed; as a result, rapid market assessments/analyses that were initially proposed had to be cut and the team spent less time in the field than desired: a total of seven days per team member was spent conducting field work, which turned out to be only one day per site in most villages/sites. Although the team requested that more rural sites be included, larger village centers were primarily visited, making it difficult to see to what extent project activities are reaching more remote areas, especially since the project did not have any mapping of activities. In explaining the proposed sites for the field work, project staff did note that due to security concerns, there were heavy concentrations of population around towns and some villages are still abandoned. In addition, the evaluation team was informed that targeting at the beginning of the project focused on village centers due to the presence of health centers and accessibility issues, which were important to consider because of logistics and security constraints. These limitations could certainly lead to bias in the findings, but we believe that the nature of the findings suggest that the potential bias is unlikely to have been significant. For example, one might argue that the limited number of FGDs and KIIs due to the limited days in the field might not be adequate to get a sufficiently accurate picture for areas visited. Similarly, the fact that RISE project staff members helped to organize the FGDs and KIIs might have opened the field work up to other bias. However, in both cases, this might have been of greater concern had the results been very consistent and very positive. The fact that the MTE team saw significant variations even in a relatively small sample suggests that any such bias would seem unlikely to have had significant impact on the findings if there was even such a bias. Finally, in most cases, while the MTE team had question guides, they asked many questions in addition to those questions and also sought to triangulate findings with other sources (sources which validated and helped to confirm our findings). RISE Mid-Term Evaluation Report 8 Oxu Solutions LLC III. MTE Findings and Recommendations a. Introduction and organization of findings The Results Framework is the central piece of the RISE project, as it articulates the project’s theory of change, allows for tracking progress against the overall strategy (via the Indicator Performance Tracking Table (IPTT)), and forms the basis for project management and the detailed implementation plan (DIP). (See Annex G for the current version of the RISE Results Framework.) In addition, as mentioned earlier in the methodology section, the evaluation methodology, tools, and detailed questions were structured around the Results Framework. The evaluation was accordingly organized by SO: Agriculture (SO1), Nutrition (SO2) and Good Governance (SO3), with a dedicated evaluation expert being in charge of each SO. The evaluation team also addressed additional questions on cross-cutting themes, program quality issues, and implementation processes. Most of the MTE findings are organized according to the Results Framework structure, i.e. SO/Intermediate Result/Outcome. A summary of the findings in relation to the Results Framework for each Strategic Objective is provided in Annex H1 and which led to some proposed revisions to the results framework, shown in Annex H2. To the extent possible, the evaluation themes and additional questions are included in each SO discussion (see Annex I for a summary of the findings in relation to the evaluation questions, against SO-oriented and general questions). Each Strategic Objective section below is divided into four parts: 1) an overall summary of progress under the objective; 2) a discussion of progress on each of the IRs and the IR Outcomes, with recommendations for moving forward during the remaining two and a half years; 3) further general observations and recommendations; and 4) conclusions. The more general findings and recommendations related to cross-cutting themes, program quality, and implementation processes are addressed in section IV. b. Findings in relation to the Results Framework i. Strategic Objective 1: Smallholder farming households in target areas have increased and diversified production and profit. Overall summary of progress under the Objective At the time of the MTE, all activities related to IR1.1 were being implemented, but none for IR 1.2 and only VSLAs for IR 1.3. The only real progress we found for SO1 has been: in improving the production of five individual crop types (corn, beans, cassava, banana, vegetables) for, at most, 20% of the smallholder farming HHs of Bwito chefferie9; and the creation of nearly 200 VSLAs. Production diversification efforts have been limited to encouraging vegetable crops with some SO2 participants and SO1 groups. There has been no intentional progress on increasing smallholder farmers’ (SHF) profit, save for what is sold from increased individual production, and improving access to financial services has only happened through VSLAs. The project has implemented SO1 activities in Bwito (corresponds roughly to Birambizo HZ) and Bwisha chefferies. CRS/Caritas worked in Bwito from the beginning and were joined by MC in 9 The total number of CRS and MC SO1 Producer Group beneficiaries is at most 25,000 out of the 120,000 farming HHs in the chefferie as reported by its agronomist. RISE Mid-Term Evaluation Report 9 Oxu Solutions LLC March 2013 when MC stopped working in Bwisha after several months of insecurity. Currently RISE works in four out of seven groupements and in 17 out of 54 localités of Bwito chefferie. Discussion of progress on each IR and the IR outcomes CRS/Caritas have been working in Bwito Chefferie since the beginning of the project and are currently involved in four of its seven groupements, in 22 Health Areas and cumulatively with 132 associations. CRS/Caritas also implement all of the SO2 activities, the VSLAs (since November 2013), and in September 2013 subcontracted the SO1 activities in the groupement of Tongo to their other partner CEDERU, whose headquarters are in Rutshuru across the national park. Mercy Corps moved into Bwito Chefferie in early 2013 and currently implements activities in one groupement, in a few localités and with 24 PGs. MC’s other SO1 activities have been: 1) to contract associations for purchasing fruit tree seedlings (once for distribution to SO2 Mother Leaders and once for PG members) and agro-forestry seedlings (once to PG members and in 2014 to public interest groups - schools, churches, public lands, some Food For Work locations, etc.) and 2) to start the production and dissemination of rabbit and guinea pigs via livestock associations. Through MC, the majority of Mother Leaders (205 MLs) and 262 other participants also received vegetable seed packets; MC also implements all the Food For Work (FFW), Food For Training (FFT) and SO3 activities. Only MC distributes USAID food rations for SO1 activities and although the rations are identical, FFW is generally for large infrastructure rehabilitation activities and FFT is used as an incentive for agricultural activities. The project has had mixed impacts on associations as some associations have seen their membership grow due to the appeal of project activities (CRS providing corn or cash) while others complain of reductions in membership and reduced trust in the association due to the late distribution of food rations (MC implementing FFT and FFW) and no cash or food ration incentives (CRS/banana). The current method of implementation for both IPs has been to select already existing associations with agricultural activities and sufficient economic and technical capacity to implement multiplication activities for a single crop. If an association has more than 30-40 members interested in a particular crop, then it is divided into two or more PGs, each with a single crop specialty. There is at most one demonstration/multiplication field per crop variety in each localité and there have not yet been any BCC campaigns for SO1, though a barrier analysis was conducted in October 2013 and a campaign is planned for mid-2014. Beyond this basic mode of operation, however, MC and CRS work very independently and very differently in nearly all aspects of the implementation of SO1 activities (see full table of the 12 major differences in Annex J). Table 4: Summary table of the main implementation differences between MC and CRS/Caritas Item MC CRS/Caritas/CEDERU 1. Community agents More than 100 local technical agents (5 per Association), all paid in FFT 10 paid local agronomist supervisors 2. Payments/ compensation FFT for hundreds of PG members for adopting practices for 2 $500 for each PG for at least one season to cover for 1ha land rental and RISE Mid-Term Evaluation Report 10 Oxu Solutions LLC seasons. Authorities lend field to PGs. labor 3. Model farms 4, each run by an association. The farm manager and the 10-20 farm laborers are paid in FFT in rotations. 75% of the crops go to the Association running the model farm. None. Banana fields act as model farms but for bananas only – no payment effected. 4. Seed Multiplication By Farmer Multipliers (FM) on small private plots separate from the PG’s demonstration plots. The FM keeps 70% of the bean seed and 30% goes to the PG members. The PG rent 1 hectare (ha) multiplication plot. The PG keeps all the cassava tuber, 20% of the cassava stalk and 50% of the corn seed for dissemination to its members. 5. Distribution to vulnerable HH 25% of vegetable seed from the model farms. No bean seed distribution, now or planned. 80% of cassava stalk, 50% of corn seed from the PG’s plot. No banana distribution yet, though planned. 6. Crops (See Annex K for details) 3 bean varieties, 3 local vegetable varieties: amaranth, spinach, eggplant, 1 OFSP variety 3 corn varieties, a dozen cassava varieties, many kinds of banana varieties Conclusion: the net effects of these differences among IP implementation strategies are: 1) confusion of beneficiaries leading to demotivation and low satisfaction, 2) low impact and sustainability, 3) low project integration, and 4) low efficiency. Intermediate Result 1.1 - Smallholder farmers demonstrate increased productivity The four core activities in IR1.1 are still very relevant to farmer needs for improved food security:  Availability: 1) by overcoming the two major staple crop epidemics in the Great Lakes region through the introduction and dissemination of new mosaic disease resistant cassava varieties and the introduction and dissemination of BXW management and control techniques and healthy banana plants; and 2) by the introduction and dissemination of higher yield corn varieties  Availability (and utilization): by increasing the nutritive value of crops with more nutritious and higher yield bean varieties, three kinds of local vegetables and eventually Orange-Fleshed Sweet Potato (OFSP)  Availability: by increasing productivity through improvements in the technical knowledge of SHFs with demonstration and training in agricultural techniques  Availability and utilization: by encouraging the breeding of rabbits and guinea pigs to boost HH nutrition rather than the more theft-prone goats and chickens. Improved access to food is to be addressed through IR 1.2 and 1.3 and improved utilization was to be addressed through linkages with SO2 but this has not occurred (for example none of the farmers knew that they were growing bio-fortified beans, and there have been minimal efforts made to target SO2 beneficiary HHs with SO1 agricultural activities). However, the appropriateness and effectiveness in implementation of these four relevant components has been very uneven as explained under each Outcome below. RISE Mid-Term Evaluation Report 11 Oxu Solutions LLC Conclusion: PG members have demonstrated increased productivity but only in the limited circumstances offered by the program for individual crops but not as integrated farming systems. This seems to be the result of the project focusing on the seed multiplication of single crops but under-emphasizing the introducing a more robust FFS component (see details on FFS below under Outcome 1.1.2) to address land productivity and risk reduction, natural resource management, pre￾and post-harvest losses and other integrated farming issues that SHFs face. In addition, there is no information on whether SHFs generally (outside the PGs) have increased their productivity or whether PG farmers have increased their productivity for any of the other RISE crops they otherwise produce. Outcome 1.1.1 - Smallholder farmers adopt improved crop varieties that meet diversified nutritional and market needs. From FGDs, it is clear that PG farmers almost unanimously appreciate the new varieties offered, particularly beans and cassava, and especially for the increase in productivity/yield. The PG farmers who received vegetable seed very much appreciated them, as previously less than half of them grew vegetables. The only two reported cases of failures were with two varieties of corn, one prone to lodging (Bambu) and one due to the sourcing of an uncertified and diseased seed (ZM). SO1 activities are not related to SO2 nutrition activities, save for what seems to be an unsystematic distribution of vegetable seed to many Mother Leaders (MLs) and a few SO2 participants. Nutritional needs have only been addressed through increased production of some of the staple crops and the distribution of vegetable seed to some PGs. It is anticipated that nutritional needs will eventually be addressed further through the yet-to-be implemented OFSP dissemination and small animal pyramid distribution. Market needs in Bwito, both for supply and demand, have not yet been assessed and no current activity pertains to this aspect. However, MC has encouraged some Farmer Multipliers (FMs) to specialize in vegetable seed production but these farmers have not found markets for their seed. The dissemination of new seed stock and improved techniques is occurring primarily within the selected PGs and associations. For CRS/Caritas, 84% of its 24,421 beneficiaries in Bwito are from 153 Producer Groups (but 132 associations as 15% of associations have had repeat activities, usually for different crops). The other 16% (3,892 beneficiaries) are from vulnerable HHs outside the PGs. The proportion of PG crops is 16% for cassava, 55% for corn and 29% for banana. MC could not provide a similar analysis but nearly 100% of their beneficiaries are from the PG/associations, estimated at less than 1,000 farmers. It should be noted that corn, the majority of seed multiplication activities, is grown mostly for sale, unlike beans, cassava and banana. Conclusion: It has been very appropriate to use smallholder farmers in multiple locations to multiply the seed stock of new varieties and to demonstrate new techniques (particularly for the non-destructive management and control of BXW in banana) as this provided opportunities both for demonstration to all the surrounding farmers and for improving the skills of the farmers doing seed multiplication. Equally, it has been appropriate to ensure the quality of seed multiplication with the assistance of a national agency, the SENASEM (MC from the beginning and CRS/Caritas starting in 2014). However, dietary diversification efforts are likely to lack scale and targeting the more vulnerable/neediest HH is not done systematically. RISE Mid-Term Evaluation Report 12 Oxu Solutions LLC Outcome 1.1.2 - Smallholder farmers implement environmentally-responsible integrated crop, disease and pest management practices From FGDs, farmers almost unanimously disliked the project’s prime focus on monoculture for seed multiplication while neglecting a more integrated farming approach as this increased the risk that a crop failure would negatively impact the farmers’ ability to pay the land rent. Farmers expressed the desire for more options, both for crop types and techniques. The two lead IPs never work together, at the same time and with the same farmers in order to: 1) present the project as an integrated whole rather than two separate NGO activities or 2) provide complementary activities, such as inter- and relay cropping, soil conservation/environmental protection and nutrition information. Furthermore as detailed above the two lead IPs have near opposite ways of working with farmers, causing much confusion among the project participants and real operational difficulties for the implementing staff and community agents. While the project calls its seed demonstration and seed multiplication field activities “Farmer Field Schools”, they are not what is considered proper FFS where farmers experiment, compare and learn for themselves rather than apply what they are told. FFS traditionally bring together concepts and methods from agro-ecology, experiential education and community development. In practice, the four MC model farms incur more disadvantages than advantages. While they provide a small window on good practices, mostly for vegetable production, and a space for learning and opportunities for FFT, they tend to benefit mostly the association running the model farm, they are not sustainable, and they do not offer farmers comparative demonstrations, as in true FFS. Conclusion: PGs and other farmers do not implement environmentally-responsible integrated crop, disease and pest management practices as the project is not integrated, either at the IP level or at the farmer level. The techniques emphasized by the project are practical for the production of seed, a needed one-off activity on flat land but not well-adapted to farmers’ usual mixed cropping practices, on sloping land where there is also a need for soil conservation. Recommendations for IR 1.1 1) Strategic: Agree on a common strategy for SO1 for maximum impact and sustainability and define clear exit strategies for all activities and their impacts (see Annex L for a sample exit and sustainability strategy from the CRS MYAP in Madagascar). Focus on what an evaluator should find if she came two years after the project’s end. Set up operational plans starting with the new final objectives and work backwards from the post end-of-project to today. Use the final year of the project as an exit year, assessing and coaching existing groups toward their own sustainability. 2) Strategic: As detailed in Annex H2, the Proposed Modified Results Framework, IR1.1 must implement a greater focus on integrated crop and soil management as well as on compatibility with local needs; a continued focus on crop varieties that meet nutritional needs; less emphasis on market need and intentional linkages with SO2 beneficiary HHs. 3) FFS: Institute real FFS at the center of farming communities with: 1) true participatory varietal trials, 2) inter-cropping trials with soil conservation on sloped plots, and 3) an adequate response to farmers’ request for the use of compost, groundnuts, soya, Irish potato, sorghum, etc. Model farms should also include FFS activities and Oxu has also already provided a report from the SO1 field work with detailed recommendations for how to improve FFS. RISE Mid-Term Evaluation Report 13 Oxu Solutions LLC 4) FFS: Address post-harvest (PH) issues as part of FFS and all other SO1 activities (e.g.: seed is different from grain, cleanliness, grading, moisture and pest management, storage, etc.). Do not rely only on, as yet, non-local and costly technology such as CRS using the PICS hermetic bags, though they have a place. It seems that CEDERU was utilizing a successful approach to addressing PH losses in Rutshuru, which could serve as a model for the rest of the project areas. 5) Seed: Continue to conduct seed multiplication activities as scale has not yet been achieved. 6) Seed: Conduct seed assessments in target geographic areas to ensure the sustainable and widespread availability of quality seed. These assessments could be conducted using CRS’ global methodology for seed assessments. 7) Seed/ Chefferie: Based on the results of the seed assessments, organize, with the Chefferie seed fairs in locations with sufficient quality seed available locally. These seed fairs would have the triple function of: 1) providing markets for local producers (whether PGs or individuals); 2) providing access to new quality seed to all farmers in communities; and 3) providing venues to raise awareness about project activities as well as community and external resources: nutrition, FFS, VSLAs, Post-Harvest, marketing, where and how to get seed and advice, etc. Vouchers could be provided for more vulnerable HHs. CRS also has a global methodology for conducting seed fairs. If successful, RISE can work with the Chefferie to design a plan for continuing seed fairs beyond the life of the project (which would be organized by the Chefferie). 8) Livestock: Define immediately a clear strategy for each solidarity chain with transparent community participation (for MLs and most vulnerable HHs) and for what happens to the central breeding units when the project ends. Place new breeding units at HH level, not in centralized centers so as to increase sustainability and resistance to insecurity and decrease the MLs’ workloads. 9) Effectiveness: Use Farmer Leaders (FLs) to facilitate the FFS, to be community educators and mobilizers, in the same manner that SO2 uses MLs for community education and mobilization. They should be a key part of the linkages between SO1 and SO2 and between SO1 participants and external stakeholders (at chefferie and higher levels). 10)Effectiveness: Use local events where people already congregate to ensure that as many people as possible in project areas have access to both knowledge and resources. For example, the project could have a stall during market days and organize activities with schools (demonstration gardens, field days), with churches, during FFW activities and during food distributions. The information relayed should include: testimonials and examples supporting new varieties and new practices, where to observe and acquire new varieties and new techniques, who are the resource people, the links between agriculture and nutrition and even the sale of new seeds/cuttings. 11)Effectiveness: Test and use the more effective methods for cassava propagation of micro￾cuttings (5cm stems) and coppicing (at seven months) rather than the current practice of using 25cm stems after 12 months. 12)Effectiveness: Conduct a campaign of mass promotion of the simple management and control BXW techniques, with support from the Chefferie. 13)Chefferie: Track monthly the prices in local markets with chefferie and community assistance. This should be at least for: bananas, corn, beans, cassava and oil, going back to 2010. This would be useful at three levels: 1) offer some clues on the effect of SO1 activities on food prices, especially for cassava and bananas whose production was decimated by epidemics, 2) discern if free food distributions have had an impact on markets, and 3) provide some information as part of an Early Warning System (EWS). RISE Mid-Term Evaluation Report 14 Oxu Solutions LLC 14)Chefferie: Involve the Chefferie agronomists and veterinarians in planning, implementation and post-project support, including the exit strategy and sustainability plan. They could be helpful with sourcing inputs, training, dissemination, and monitoring. Intermediate Result 1.2 - Smallholder farmers and other value chain actors have improved processing, storage and market access Outcome 1.2.1 - Processors increase capacity and knowledge to transform raw commodities into processed goods. Outcome 1.2.2 - SHFs and traders enhance product quality and reduce losses. Outcome 1.2.3 - SHFs, input suppliers, and other Value Chain (VC) actors increase access to market information. There have been no activities related to this IR in Birambizo HZ but current production surplus seems to be marketed and sold outside Bwito Chefferie, especially corn and cassava, which may not improve food availability locally and is likely a constraint to SHFs to access local market opportunities. This needs to be assessed during the coming value chain analysis (VCA), scheduled for early 2014 During FGDs, farmers were very enthusiastic to receive support for marketing activities, in Birambizo HZ. For Beni/ Butembo, a VCA was conducted in September and October, 2013, and can be used to help prioritize and determine marketing activities in those areas during the remaining project period. Intermediate Result 1.3: Smallholder farmers and other value chain actors improve integration into markets Outcome 1.3.1 - Improved technical services system provides quality market information to smallholder farmers, input suppliers, processors, transporters and traders. Outcome 1.3.2 - Enhanced informal and formal financial systems provide improved financial services to smallholder farmers, input suppliers, processors, transporters, traders and support businesses. Outcome 1.3.3 - Improved business management skills increase the business efficiency of smallholder farmers, other value chain actors and support services. There has been no activity related to this IR save for the VSLAs. The VCA for Bwito, as noted above, is scheduled for early 2014. From the FGDs, the evaluators learned that farmers do not use any external finance for their agricultural activities and that there is none available in their communities. In Bwito, there will be 110 to 120 VSLAs in FY 2014 assisted by 22 Village Agents (VA), plus the 55 surviving out of the original 60 VSLAs in Rutshuru and Karisimbi, for a total of about 175 VSLAs serving approximately 4,400 members. RISE Mid-Term Evaluation Report 15 Oxu Solutions LLC Conclusion: It is clear that VSLAs have been very well-received and the survival of the great majority in Rutshuru in spite of severe insecurity speaks well for people’s ingenuity, their strong desire to make them work, and the fact that VSLAs are addressing a felt need. Recommendations for IR 1.2 and 1.3: VSLA 1) Ensure that all key community volunteers are mixed (e.g. MLs and FLs) as VSLA members as these have proven to be a good medium for information sharing and for creative activities. However, it should be noted that VSLAs are excellent for improving HH resiliency but usually have a small impact on food security. 2) Encourage VSLAs to provide finance for agriculture or marketing, individually or as a group. 3) Ensure that the self-replicability of VSLAs is assured (through competent and incentivized/motivated VAs and the sustainable production of VSLA kits) so that they can reach autonomously all those interested within a few years. 4) Increase the number of VSLAs to at least 500 during the life of the project as this is likely to be the only financial service offered at scale during the final two years. Other 5) Strategy: As recommended in the Proposed Modified Results Framework (Annex H2), it is critical for the project to adjust the current IR1.2 and IR1.3 to be much more modest and focused in scope. Specifically, we recommend a consolidation of the two IRs into one IR and to emphasize at the Outcome level: i) post-harvest handling, storage, and processing; ii) sustainably improving physical transport, infrastructure, and market information for market actors; and iii) improving informal financial services via expansion of VSLAs. 6) Strategy: Focus first on the basics for marketing activities (quality management, input procurement, basic rules for calculating yields and farm economics, etc.) as they take a long time to implement and there are only two years left for project implementation. Based on the Bwito VCA, the marketing activities must be holistic and realistically considered. For example there is the dilemma of whether to focus on a few better-off stakeholders in an already functioning value chain (the low-hanging fruit, as in Beni/Butembo) or to focus at a more basic level, lower down the chain and with more actors (potentially more impact for more people as in Bwito). 7) Value chain analysis: The Bwito VCA must take into account the possibility for the marketing of rabbits and corn flour or locally made CSB, all requested by participants. 8) Marketing capacity: Conduct a capacity assessment for PGs in terms of organizational and financial management as well as potential ability of the PG to aggregate and market production. 9) Marketing capacity: Based on the results of the PG capacity assessment, select PGs to train with clear intent and potential capacity for dedicated marketing on the more advanced production, management and post-harvest methods. Those SO1 association members and FFS participants not included in this group should receive only basic PH training. 10)Marketing: Group Associations/PGs into unions and federations to have a chance to become formal market players as there are already local groups of marketing intermediaries, such as ADEDEVI. 11)Marketing: Test the production and marketing of locally produced “CSB” and consider carefully the planned giving away of crop stores worth $1,000-$2,500 to a few high performing business groups/associations. This is an expensive and high risk activity given the short time and the low capacity level of such groups. RISE Mid-Term Evaluation Report 16 Oxu Solutions LLC 12)Exit strategy: Empower individuals and groups to provide the source seed and other inputs the project provided and transition toward cost recovery from the current practice of giving everything away. Other findings and recommendations From the gender perspective, in all FGDs women reported being well-integrated into SO1 activities (agriculture and VSLAs) and faced the same issues as men, particularly the high cost of renting land. However, widows were singled out as being less able to participate due to difficulties meeting land rental costs and farm labor availability. The adaptability to insecurity is mixed. It is best adapted for the VSLAs with volunteer VAs for support (movable and complete community responsibility) and for seed multiplication (multiple locations) though with some constraints due to the regular need for cash or FFT. It is not well adapted for the centralized small animal breeding units (not movable and single location), MC’s use of many field agents incentivized through FFT, and using a single field for OFSP multiplication. The BCC materials used are of uneven quality: MC: In French, uses photos, does not explain much, does not engage the reader CRS: In Kiswahili, with quality graphics, tells a story, engages the reader CRS: Several BXW management and control public billboards with good graphics but in French instead of local languages Recommendations  BCC: Harmonize all BCC materials across IPs (recommend to use the CRS/Caritas model) and French should not be used for participant/beneficiary use. Ensure that complete (i.e. all crops) “boîte à images” are available to key people in communities (Farmer Leaders, agronomists, one per association, etc.).  Sustainability: Motivate people beyond giving money and food as this is a development program. Instead, use social and technical recognition and provide small equipment (either personal: such as boots, rain ponchos, etc. or technical tools such as for making measurements: scales, tape measures, etc.).  Sustainability: Reward impact and results rather than activities and reward at the group level rather than at the individual level. For example, reward the group with project￾appropriate smart rewards for improving long-term food security, such as tools or RISE Mid-Term Evaluation Report 17 Oxu Solutions LLC equipment (a mill, a seed store) but not money or food. This approach also contributes to social cohesion.  Sustainability: Transfer responsibilities from project staff/agents to local entities (individuals and groups) as much and as early as possible.  Beneficiary counting: Create a documented definition bank of “beneficiaries” (Is someone who receives a one-time donation of a few kg of seed truly a beneficiary?). Perhaps use “participants” for those who actively participate and contribute over a period of time and “beneficiary” for those who only receive gifts. Also, carefully assess the multiple counting of participants and beneficiaries, as individuals and as HHs: multiple-counting is to be encouraged inside and across SOs, i.e.: individuals in a HH could be part of a VSLA and participate in several cropping activities, in FFW and in SO2 activities but be counted only once as a “participating HH.”  Targeting: Define who are the most/more vulnerable HHs using a food security filter (e.g.: cropland size ownership and access, HH labor availability) and use participatory and transparent methods. Then target first the most vulnerable HHs among the SO2 beneficiaries and second the non-SO2 beneficiaries. Currently, vulnerable HHs without access to land are not part of SO1 activities. The current vulnerable HHs reached through distributions are selected by committees with assistance from local authorities and are located only within a few kilometers (km) of the multiplication fields.  Targeting: Define specific and integrated approaches to reach the most/more vulnerable HHs. Approaches range from sensitization/information sharing, gifts of seed or small animals, inclusion in VSLAs, participation in trainings and FFS visits at crucial times, etc. For example, ensure that new bean varieties are also widely disseminated outside the PG/associations, that the landless are also beneficiaries of rabbits and guinea pigs, and that OFSP is made widely available. Conclusion Considering that the overall goal of RISE is that Vulnerable households and communities in North Kivu build and sustain food security and the purpose of SO1 is that Smallholder farming households in target areas have increased and diversified production and profit, the project thus far is not effective in reaching either the overall goal or SO1 due to: a lack of systematic targeting (vulnerable HHs and smallholder farmers); a lack of project integration (inside SO1, across the three SOs and through the cross-cutting themes); and a lack of appreciation for the smallholder farmer’s perspective. Furthermore, the sustainability of results and impacts is only evident for the small number of VSLA members (2.5% of chefferie HHs) and the small number of PG members who received new seed for individual crop varieties (currently around 20% of chefferie HHs). All the new seed varieties can be sustainably propagated except for corn, which has to be renewed from foundation seed every few years. From the FGDs and KIIs, it is evident that the project participants, as individuals or groups, and project staff are unclear about the future of activities and expected results, except for seed multiplication-distribution. Also from the FGDs, the spillover effects of SO1 are limited to the immediate vicinity of individual PG members. RISE Mid-Term Evaluation Report 18 Oxu Solutions LLC ii. Strategic Objective 2: Improved nutritional status among pregnant and lactating women and children under 5 in target areas Overall summary of progress under the objective The SO2 activities are designed around the Preventing Malnutrition in Children Under 2 Approach (PM2A), a strategy for preventing malnutrition that includes three pillars: 1) distribution of food rations starting with pregnancy and continuing until a child’s second birthday; 2) promoting behavior changes that will enhance optimal nutrition and improved health status; and 3) increased utilization of preventive and curative health services for women and children.10 Activities initially started in five Health Areas (HA) of the Birambizo HZ but when the geographic shift occurred, activities were scaled up dramatically with the goal of reaching 30 of the 35 HAs in the zone. The three IRs for SO2 include the three PM2A pillars and generally speaking, RISE is making progress on all three IRs, but the level of progress varies from one Outcome to another. For instance, for IR 2.1 (utilization of health and counseling services), utilization of preventive services is quite high, especially for prenatal consultations (PNC). However, the quality of the both the preventive and curative services is not up to GoDRC standards due to 1) chronic shortfalls in the MoH provision of essential medicines and supplies to the HCs and 2) delays in the RISE-sponsored training program for certain topics for the health service providers. For IR2.2 (adoption of improved nutrition and health behaviors), the MTE findings show high levels of knowledge and anecdotal evidence of adoption of better practices, especially for Essential Nutrition Actions (ENAs). The distribution of food rations (an Outcome for IR 2.3) is proceeding as scheduled and there is a high level of satisfaction among the participants receiving rations. During the MTE the SO2 team was in the process of revising its Results Framework and took into consideration suggestions from the SO2 evaluator. The CRS Senior Technical Nutrition Advisor followed up during her recent visit to the project and the proposed revised SO2 Results Framework was finalized. (See Annex H2). Because the revision was a work in progress, the MTE focused on the original Results Framework; the following discussion of IRs and Outcomes is based on the original Results Framework. Discussion of progress on each IR and the IR Outcomes Intermediate Result 2.1 - Pregnant and lactating women and caregivers of children under 5 have increased their utilization of counseling and health services in accordance with GoDRC standards Early outcomes: Anecdotal evidence from almost everyone interviewed (women participating in PM2A, MLs, community leaders, and HC staff) shows significantly higher attendance at PNC and at Under 5 (U5) consultations; more deliveries taking place at HCs; and higher birth weights for infants whose mothers participate in PM2A. The Head Nurse at one of the HCs in Bambu (Centre de Santé de Référence) for example, said that he has not recorded a birth weight under three kgs. (6.6 pounds) since RISE began distributing rations. (See Annex F2 for a list of project staff interviewees and Annex F3 for a list of KIIs.) 10 Preventing Malnutrition in Children Under 2: A Food-Assisted Approach (Title II Technical Reference Materials) RISE Mid-Term Evaluation Report 19 Oxu Solutions LLC Outcome 2.1.1 - Health facilities improve provision of pre- and post-natal, IMCI services to women and children and reinforce Essential Nutrition Actions (ENA) and improved hygiene behaviors. There is an ambitious program to train hundreds of health care providers (MoH staff, RECOs [Community Health Volunteers] and MLs) on a number of topics including ENA, Key Hygiene Practices, improved PNC, and Integrated Management of Childhood Illnesses (IMCI), both the clinical version for HC staff and the community-level version for the RECOs. Some of the training has already taken place and the SO2 team and their MoH colleagues are to be commended for this effort although much remains to be done if the original training program is to be completed. Another issue is that, to date, there seems to be little follow-up to the training in the form of supportive supervision at the HC and community levels. This is due partly to the fact that the MoH Health Management Team (HMT) for the Birambizo HZ has been waiting for an updated formal Memorandum of Understanding (Protocole d’Accord); this was signed in February 2014 and paves the way for joint project-MoH supervisions in Birambizo and in Butembo and Béni. Conclusion: Much has been done to improve the quality of services through training of RECOs and HC staff and the SO2 team has an ambitious training program planned for the remainder of the project. The benefits of this program could be jeopardized if there is no follow-up with the trainees, including field-based supervision. Outcome 2.1.2 - Pregnant and lactating women access improved pre- and post-natal care services All six Head Nurses interviewed stated that there is a significant increase in the number of pregnant women attending PNC sessions, an observation supported by the other groups interviewed during the qualitative field work. This is not surprising since enrollment in PNC is the first step in the process to be eligible for the PM2A program. Once pregnant women are registered, MLs and HC staff strongly encourage them to come for four PNCs before delivery and to deliver at a HC rather than giving birth at home. Unfortunately, there does not seem to be a corresponding rise in the number of women utilizing post-natal services. Both the Birambizo Medical Director and the CRS/Caritas project staff are interested in documenting the increased use of services by comparing the HC records before and after the project started. Although this would not be a rigorously scientific comparison since not all the women and children participating in RISE go regularly to the HCs, it would provide more concrete evidence of reported changes in utilization of services. Although this increased utilization of services is a positive outcome of the RISE project, the Head Nurses raised two issues with the evaluation team. First, their workload has increased significantly now that more women and children are coming for consultations yet they receive nothing from the project. This is in stark contrast to the policies of other organizations, which provide either a monthly bonus or some other type of financial compensation for HC staff who are involved in the programs they support. (It should be noted, however, that these other programs fall into the category of emergency programs; RISE is the first development program in the area and the approach is different). Second, the HCs lack basic supplies such as medicines, forms and registers for PNCs and for U5 consultations. This was a concern that the Birambizo Medical Director and the Birambizo HMT also raised. As the HMT noted, most HCs lack the basic elements to carry out “a prenatal consultation worthy of its name”. The project is making an effort to provide forms for PNC and U5 consultations but there are currently no plans to provide additional supplies such as preventive and curative medicines. With the withdrawal of Merlin’s support from all but six RISE Mid-Term Evaluation Report 20 Oxu Solutions LLC facilities in Birambizo, this has serious implications for the quality of care available to the women and children accessing services. CRS is very aware of these issues and informed the evaluation team that a health services system specialist will carry out a consultancy to see how RISE can best address some of the issues raised by the MoH partners and provide better support, especially to the HCs. Conclusion: There has been a significant increase in attendance at PNCs as a result of RISE activities. However, this increased utilization of prenatal services has not been matched by increased utilization of post-natal services. The increased utilization has also negatively affected the workload of HC staff, who may be trained by RISE in providing this care but often lack basic supplies and materials for these services. Outcome 2.1.3 - Health service providers increase referrals for acute malnutrition. At the time of the MTE, the main group of health service providers conducting screening for malnutrition in CU5 was the RECOs. RISE provides RECOs kits with materials such as the MUAC armbands; according to the Mid-Term Progress Summary report prepared by the SO2 team, the RECOs referred a total of 3,660 malnourished children to HCs for follow-up nutritional services during a nine-month period from January through September 2013. While this is commendable, there are several concerns with progress for this Outcome. First of all, many HCs lack basic supplies for nutrition services, including medicine for treating infections and supplemental food for rehabilitation, so that even if a caretaker brings a child to the HC, there may be very little that the HC staff can do. Second, it is not clear whether there is a system in place for follow-up. Such a system would need to provide information for at least the following questions: How many children who are referred by the RECOs actually go to a HC? How many receive recuperative nutrition services? How many improve and maintain their nutritional status? If the child does receive services, what is the post-service/ treatment follow-up at the HH level by the RECOs or other community volunteers such as the MLs? What measures have been put in place to prevent a recurrence (e.g., one-on-one counseling and participation in a nutrition demonstration)? Setting up and maintaining a follow-up system would require motivated RECOs and initially, RISE staff support, which would mean increasing the number of Nutritionists in the Birambizo HZ. Conclusion: Although the RECOs are conducting community screening for malnutrition, it is difficult to ascertain how well they are doing it and what the effects are as there is no standard system for monitoring the results. Another concern is the lack of nutrition services, supplies and materials at many facilities. Recommendations for IR 2.1 1) Outcome 2.1.1: Prepare a comprehensive training plan for the remainder of the project. For each session include how many people are to be trained in each category (MoH trainers of trainers, CRS/Caritas staff, HC personnel, RECOs, MLs, and community leaders), topics for each session, number of days, trainers needed, and the proposed time period (month and year). Review the plan to see if it is feasible given the time remaining and other SO2 priorities (such as recruiting and supporting an additional 800 MLs and ensuring quality activities in all the HAs.) Consider scaling back the training plan if necessary. 2) Outcome 2.1.1: Work with the Birambizo HMT and Goma-based MoH colleagues, including PRONANUT, the National Nutrition Program, to develop a realistic schedule for follow-up supervision of the HC staff and RECOs trained. Agree on tools to be used for supportive supervision. 3) Outcome 2.1.2: If the MoH concurs and resources permit, conduct a before/after survey of HC records to determine to what extent utilization of MCH services has changed since RISE started. RISE Mid-Term Evaluation Report 21 Oxu Solutions LLC 4) Outcome 2.1.2: Follow through with plans to bring in a CRS health systems specialist to address issues affecting the quality of MCH services. Determine to what extent RISE can help the HMT acquire the minimum supplies needed for optimal prenatal and U5 consultations. 5) Outcome 2.1.2: Increase the emphasis on a set of messages for pregnant women, their husbands and community leaders to encourage women to 1) attend at least one post-natal consultation and 2) complete all four prenatal consultations. This will include identifying barriers and potential incentives to increasing access to these services. 6) Outcome 2.1.2: After identifying barriers and incentives, reinforce the BCC activities to highlight the importance of bringing CU5 to community malnutrition screenings and to the U5 consultations at the HC. 7) Outcome 2.1.3: To fully benefit from the community-level malnutrition screening carried out by the RECOs, work with the MoH structures to ensure systematic record-keeping, follow up of cases referred, and to the extent possible, improved nutritional rehabilitation services. Intermediate Result 2.2 - Pregnant and lactating women and children under 5 have adopted improved nutrition hygiene behaviors It is clear from the results of the FGDs conducted during the MTE that knowledge levels for ENA messages and good hygiene practices, especially the importance of hand washing, are quite high among groups of women. Both the women themselves and the MLs state that they are now following these improved practices. These results are backed up at the individual level by the findings from the Lot Quality Assurance Sampling (LQAS) conducted in August 2013. What was somewhat surprising is how well men have assimilated these messages and their assertion that they support their wives in implementing the recommendations from HC staff and MLs. Men interviewed during the MTE were able to provide details on such diverse themes as the three advantages of immediate breastfeeding and the different food groups that comprise a balanced meal. The men said they learned all of this from radio spots sponsored by RISE, from what their wives tell them, and from the MLs’ visits to their homes. Outcome 2.2.1 - Households adopt prevention and management practices for maternal and childhood illnesses. Much of the credit for the high levels of knowledge – and reported adoption of improved nutrition and hygiene behaviors – goes to the MLs who have been effective communicators. They know the subject matter well and have been tireless in their efforts to carry out home visits and nutrition demonstrations. Other factors contributing to the success of the SO2 BCC activities include: the use of formative research to define the first set of messages and activities, especially for nutrition; using LQAS to monitor progress and refine messages; and reinforcing the messages through radio spots in four local languages. The SO2 team plans to continue to reinforce ENA and good hygiene behaviors during the second half of the project, but will focus more attention on recognition of danger signs for maternal and child illnesses, when and how to provide home care, and when to go to a health care facility. Although the home visits are effective and using one-on-one negotiation is a proven method for effecting behavior change, this may not be the most effective approach given the number of PM2A participants each ML is expected to mentor (the number varies between 20-150.) Even when an additional 800 MLs are recruited (planned for Year 3) it is simply not possible for these volunteers to conduct home visits on a regular basis to all participants and to cover all the BCC themes well. RISE Mid-Term Evaluation Report 22 Oxu Solutions LLC Given the ML to PM2A participant ratio, it would be more efficient if the MLs used group meetings for BCC and reserved the home visits for women and other caretakers who need additional support. Presently, the SO2 team relies primarily on its own staff, the MLs and radio spots for promoting behavior change. Other opportunities and audiences should be explored such as promoting ENA during SO1 and SO3 activities, experimenting with other channels of communication and communicators, and diversifying the approach to add variety and newness to the themes. Conclusion: The results of the MTE interviews and the LQAS showed that the BCC component has been effective. However, other themes still need to be incorporated and more variety in approaches and communication channels is needed. (See also the section on cross-cutting themes for a more in￾depth discussion of BCC.) Outcome 2.2.2 - Pregnant and lactating women and caregivers of children under 5 have access to improved water and sanitation facilities. This SO2 Outcome (“access to improved water and sanitation facilities”) is outdated and is being revised as SO2 activities do not include improving water and sanitation facilities. (The original RISE proposal had a significant WASH component, but it was taken out due to budget constraints.) At the time of the MTE, no organizations in the project area had any major activities to improve water and sanitation infrastructure in the communities targeted by RISE. Although members of the SO2 team have met with UNICEF to discuss the possibility of instituting the “Healthy Villages” program (Villages Assainis) in the project area, no definitive decision has been made. As for BCC activities related to WASH, the principal hygiene and sanitation messages now being promoted include hand washing at critical moments and the importance of good kitchen and HH hygiene practices. Plans to expand and reinforce the hygiene and sanitation messages are underway, but the lack of adequate water and sanitation facilities means that making significant reductions in diarrhea disease will remain a challenge. (Note: The SO2 evaluator suggested that the team might want to explore the Environmental Enteropathy Hypothesis since recent studies and pilot programs postulate that environmental enteropathy is a primary cause of stunting. However, given the priorities already established for the remaining project time and the effort needed to develop another BCC theme, the evaluator agrees with the SO2 team to not include this topic.) Conclusion: It is unlikely that the project will achieve this Outcome (as it is now worded) due to the lack of a WASH component within RISE or any significant water and sanitation infrastructure projects in the catchment area. Opportunities for disease prevention are present: a partnership with UNICEF’s “Healthy Villages” program to develop activities; the promotion of simple low to no costs measures for improving hygiene such as TippyTaps and better food and water handling; and a creative BCC campaign for the key WASH messages. Recommendations for IR 2.2 1) Outcome 2.2.1: Encourage MLs to conduct more BCC in groups rather than relying on home visits. Reserve home visits for HHs where women and children need more individualized attention (e.g., low birth weight, previous child died, faltering weight during nutrition screening and non-attendance at PNC consultations). 2) Outcome 2.2.1: Once Mercy Corps has conducted and evaluated its agricultural BCC campaign, determine how BCC activities for SO2, especially new communication channels, could be diversified further. 3) Outcome 2.2.1: To help MLs focus on barriers rather than simply repeating messages, have each ML work with a small group of pregnant and lactating women (PLW) to identify barriers RISE Mid-Term Evaluation Report 23 Oxu Solutions LLC and to propose ways to overcome those barriers. Convene a representative group of MLs to discuss their findings with the BCC team and revise messages and activities (and target groups) accordingly. 4) Outcome 2.2.2: Investigate the possibility of promoting simple WASH measures such as hand washing stations (Tippy Taps). Demonstration Tippy Taps could be set up at such places as markets, commodity distribution sites, and HCs. See www.tippytaps.org. 5) Outcome 2.2.2: Since “improved water and sanitation facilities” are not part of the RISE project, revise this Outcome to reflect this change in project design. Focus first on hand washing at the four critical moments and then work with caretakers to define other improved hygiene behaviors such as food hygiene, avoiding open-air defecation and proper disposal of children’s feces. 6) Outcome 2.2.2: Follow through on the initial contact with UNICEF and if feasible, select a limited number of communities to introduce activities for the “Healthy Villages” initiative. Intermediate Result 2.3 - Pregnant and lactating women and children under 5 consume more diverse foods The daily diet for PLW and children 6-23 months receiving PM2A rations has definitely improved according to those interviewed during the FGDs. What is not so evident is whether there has been any sustainable improvement in the quantity and quality of foods, including more foods rich in micronutrients, available for the average HH. Outcome 2.3.1 - Pregnant and lactating women and children under 2 consume preventative rations. In the Birambizo HZ the distribution of PM2A rations began in mid-2012 in five Health Areas. When SO2 activities in other HZs were suspended due to security concerns, a major effort was made to expand the PM2A activities, including the distribution of rations, in Birambizo. With the enthusiastic support of the Birambizo Medical Director, the goal is to cover at least 30 of the 35 HAs; RISE is currently distributing rations to over 24,000 women and children in 28 HAs and plans to reach its target of approximately 31,000 HHs by April 2014. Although the exact percentage of eligible women who participate in PM2A was not available, the level of participation appears to be quite high as the vast majority of PLW are registered in the program. This was confirmed by HC staff and by the men and women who participated in the FGDs, including the MLs. As one woman expressed it: “Every husband wants his wife to participate in the program.” Enthusiasm for the program was such that women living in HAs that did not yet participate crossed over into neighboring HAs to register. This overloading of HCs has been remedied to some extent by the rapid expansion of the project into the majority of the Health Areas in Birambizo. In general, the women receiving rations expressed satisfaction with the type of rations and with the distribution process although they did note that they may have to spend an entire day waiting their turn at the monthly distribution. Men whose families receive rations also expressed their appreciation for the rations and everyone knew that the commodities came from the U.S., often stating that they came from “USAID America”. Those interviewed during the MTE brought up two issues with the rations. First, although both men and women understand that the rations are for PLW and children 6-23 months, they noted that the quantity distributed should be larger since it is difficult for a parent not to share the rations with other small children in the HH. Not providing a family ration for the rest of the family could dilute the overall impact of a PM2A program such as RISE Mid-Term Evaluation Report 24 Oxu Solutions LLC RISE as it is highly likely that the rations are shared in some HHs. Second, women noted that the amount of oil is small compared to the amount of CSB distributed. Conclusion: The distribution of PM2A rations has been one of the things the participants appreciate most about the project and indications are that the distribution, for the most part, is generally done correctly and in a timely manner. (Commodity issues related to PM2A distributions are provided in detail in sub-section IV.d on Commodities below). Outcome 2.3.2 - Pregnant and lactating women and children under 5 consume adequate quantities of locally produced foods rich in micronutrients. Ideally, the SO1 activities would directly support increased availability of and access to a wider variety of locally-produced foods, either through increasing production and/or providing income opportunities, but there has been no systematic integration of SO1 activities with SO2 HHs. No sustained effort has been made to ensure that SO2 participants benefit from the agriculture and income-producing activities promoted by SO1 or from the FFW activities under SO3. During the FGDs with PM2A participants and with the MLs, very few of the women indicated that their spouse or another household member participated in any RISE activities such as agriculture or FFW. This runs counter to the purpose of a MYAP where the SOs should be tightly integrated and mutually reinforcing in order to maximize the effects. This lack of integration and strategic targeting constitutes a major weakness in project implementation. The project design calls for SO1 to promote and support increased production of fruits and vegetables through gardening and fruit trees in order to improve dietary diversity. SO1 is also supposed to manage the small animal raising activity so that HHs will have affordable access to more protein in their diets. At this time, the SO1 activities are too modest and scattered to have an impact on the nutritional status of women and children participating in PM2A, especially a sustainable impact. Only three groups of MLs are raising rabbits, one ML group received cuttings for orange-fleshed sweet potatoes (OFSP), and a handful of women received seeds and some training for gardening. Even making seeds more available to purchase on the market has not yet been achieved. In addition, men and women participating in SO1 agricultural activities receive little or no information on the nutritional value of the crops they are planting and why it is important to diversify the diet with foods rich in micronutrients. Unless RISE makes a concerted effort to bring SO1 food production activities to scale and to target communities where there are PM2A participants, it is unlikely that dietary diversity will improve for the majority of participants and this Outcome will not be achieved. One thing that is promising for this Outcome is that both women and men interviewed during the MTE expressed an interest in learning how to make their own version of CSB, using local crops. Although this porridge would not be fortified, it would be a definite improvement over what is traditionally fed to young children. The SO2 team is looking internally for resources to support the development of local porridges and other foods for infants and young children. Conclusion: Little progress has been made toward achieving this Outcome, due largely to the lack of integration among the three SOs, particularly between SO1 and SO2. Recommendations for IR 2.3 1) Outcome 2.3.2: (For all three SO teams) Work together to develop a strategy for better targeting of SO2 HHs or at least of communities where people are participating in SO2. While it will not be possible to reach all the HHs receiving PM2A rations, a concerted effort should be RISE Mid-Term Evaluation Report 25 Oxu Solutions LLC made to identify vulnerable HHs and to locate SO1 and SO3 activities in geographical areas where SO2 HHs can benefit from them. Activities that enhance food availability such as improving the yield of staple crops, diversifying protein sources through small animal raising, and increasing fruit and vegetable production should be a particular focus. 2) Outcome 2.3.2: (For the Mercy Corps and CRS/Caritas BCC teams) Develop a list of key nutrition messages that can be disseminated to SO1 participants during FFS and other activities and to FFW participants. Work with the SO2 team to promote activities that result in greater nutritional diversity and thus improved nutritional status. 3) Outcome 2.3.2: (For CRS, Caritas, CEDERU and Mercy Corps) Work with interested parties to promote an improved porridge and other dishes for young children using the staple crops available in the area. This would include working with the SO1 team to: - Perfect age-appropriate recipes using locally available ingredients. - Promote the production of soybeans and groundnuts where conditions allow. - Provide mills suitable for grinding corn and sorghum either to individuals or to groups such as well-established ML groups or PGs. General observations Overload: An important concern is the overload factor for the SO2 component: recruiting and supporting an additional 800 MLs; increasing the number of PM2A participants to more than 31,000 by May 2014; the rapid expansion of SO2 activities from five HAs to 30 or more HAs in the Birambizo HZ; the entry into Butembo and Beni, which will further tax human and financial resources; carrying out an ambitious training program; working with the SO1 and SO3 teams to reorient activities so that SO2 participants are targeted more directly – all of this will seriously stretch the SO2 team’s ability to deliver quality services. Mother Leaders: The MLs are a group of volunteers created specifically for the RISE project. They are an integral part of the SO2 activities and it is largely due to their efforts that many families participating in PM2A have high levels of knowledge on the key ENA and hygiene messages promoted by RISE. At the present time there are 205 MLs for approximately 24,000 women enrolled. Obviously the ratio of MLs to participants is not realistic since the MLs are only available part-time to conduct home visits and to hold group talks with PM2A participants. This means that many PM2A participants, perhaps the majority, are not fully benefitting from the BCC activities led by the MLs. In addition to an unrealistic number of PM2A participants to follow, the MLs in certain communities are also following up on other activities, some initiated by MC. These include rabbit raising at three centralized breeding sites, home gardens, OFSP fields (one so far), etc. With the number of participants enrolled in PM2A expected to reach 31,000+ by May 2014, RISE plans to recruit and train an additional 800 MLs before the end of Year 3. This will mean a ratio of approximately 40 participants per ML, still high but more manageable. At the present time there is no well-defined plan to ensure the sustainability of the ML groups after the project ends. The Birambizo Medical Director suggested that they, like the RECOs, eventually be managed by the HC staff. Given that the project plans to have over 1,000 MLs in place by the end of Year 3, it will be a real challenge to “institutionalize” these volunteers in the same way that the RECOs are part of the HC teams. Another challenge is providing incentives to motivate the MLs for the long term, incentives that could come initially from the project but should eventually be provided by the communities. RISE Mid-Term Evaluation Report 26 Oxu Solutions LLC SO2 Field Staff in Birambizo HZ: The rapid expansion of the SO2 activities into a projected total of 30 out of 35 HAs in Birambizo also significantly increases the workload of the SO2 field staff and the Goma-based team. There are currently 15 Nutritionists and two BCC Agents stationed in the Health Zone. Ideally, each Nutritionist would have only one HA to cover but some now have two or three. As a result, the ratio of the Nutritionists to MLs and to PM2A participants is quite low, potentially compromising the quality of the field activities. An additional constraint for the Nutritionists is mobility for them and for the Katwe-based nutrition coordination team. Not all of the Nutritionists have motorcycles and this hampers their ability to cover their expanding territory. As for vehicles at the Katwe office, two vehicles are shared among the Commodities, Ag and Nutrition teams. The competing demands on the vehicles make regular supervision problematic. Role of the RECOs: The initial project design called for the RECOs to be an integral part of the RISE project and ideally, they would work in tandem with the MLs. The training plan, which is already ambitious and will require significant time and resources, also includes training these volunteers. However, the RECOs are under the supervision of the HCs and have a number of roles and responsibilities, which can occupy them for several weeks at a time. In addition, they are in demand by other projects too. As a result, they are not always available when needed for RISE activities. Extension into Butembo/Béni: RISE is gearing up to have activities for all three SOs in three new Health Zones in two other Territories of North Kivu Province: Lubero and Béni (referred to as Butembo/Béni). These new catchment areas are not contiguous with Birambizo HZ or with any other HZ where there are still some SO1 and SO3 activities. Questions have been raised elsewhere in this report about the advisability of such an expansion when less than three years remain and the scale-up of activities in Birambizo, especially for SO2, has been very rapid. Although no PM2A ration distribution is planned for this geographic expansion, the concern is that this extension could negatively impact the quality of SO2 activities in Birambizo HZ due to current staff capacity. A promising development is that the SO2 team is considering adapting its strategy in Butembo/Béni. Rather than attempting to initiate activities for all three IRs, the proposal is to implement only Outcome 2.3.2 of IR 2.3 (Pregnant and lactating women and children under 5 consume adequate quantities of locally produced foods rich in micronutrients.) With this in mind, the SO2 team would work much more closely with SO1 than has been the case to date in Birambizo. Ideally, SO1 and SO2 would target the same participants to the extent possible, which would result in much greater synergy and theoretically in improved nutritional outcomes for PLW and young children. Gender: In terms of gender, SO2 focuses more on women (PLW and the MLs), but the MTE showed that men are equally knowledgeable about nutrition and hygiene, having assimilated the same messages as the women either through the radio spots or through visits from the MLs. There are also a handful of men who are informally designated as Papa Leaders and the SO2 team plans to incorporate more men as peer educators for other men. Since men ultimately control household decisions, involving them more directly will reinforce the SO2 activities and help to ensure sustainability. A promising pool for the Papas Leaders might be the male RECOs as they are already included as one of the groups benefitting from training through RISE. Additional recommendations for SO2 1) Program quality: Ensure that adequate human, financial and materials resources are made available and that a rigorous quality monitoring system is in place. RISE Mid-Term Evaluation Report 27 Oxu Solutions LLC 2) MLs: Complete the recruitment of the 800 additional Mother Leaders as soon as possible and provide adequate resources for their training and support. Include resources for all active MLs for such items as T-shirts, a waterproof bag or folder for their documents, the pagnes (cloth) that RISE promised them when the first Mother Leaders were recruited, visual aids for BCC sessions, and annual recognition ceremonies (carnivals). 3) MLs: Work with communities to help them find ways to express their appreciation of the MLs on a regular basis. 4) MLs: Ensure that each ML has a manageable number of other PM2A participants to mentor and that they live within a geographical area accessible on foot. In addition, inspired by a visit to the Burundi PM2A project, the SO2 team plans to select the new MLs from women participating in PM2A rather than leaving selection to community leaders and the local HC, an idea which we would support. 5) MLs: (CRS/Caritas and Mercy Corps) Work together to lay out a long-term plan for activity implementation so that the MLs are not overloaded. To avoid radically different approaches, the organizations must also agree on how the MLs are to be rewarded for their support of and participation in RISE activities. 6) MLs: If resources for training and supporting community volunteers are limited, develop the training plan and strike an appropriate balance between near-term effectiveness, whereby the Mother Leaders have shown to be quite effective, against the long-term sustainability of the model and integration with the MoH systems, via training of the RECOs. 7) MLs: Prioritize the Mother Leader groups for training in VSLA. This will provide another incentive for the groups to remain together after the project ends and will be an additional source of motivation for them. 8) SO2 field staff: Analyze the need for more SO2 staff (Nutritionists and BCC Agents) in Birambizo, taking into account geographical distances to be covered, the number of PM2A participants and the status of activities in each Health Area. Develop a plan for recruiting and training additional Nutritionists to ensure adequate coverage of each Health Area and quality support for the MLs. 9) SO2 field staff: Provide motorcycles for all Nutritionists who are capable of riding them and one motorcycle for the nutrition coordination team in Katwe. 10)Expansion into Butembo/Béni: For this expansion, focus solely on Outcome 2.3.2, working closely with SO1 to ensure that PLW and young children consume a more diverse diet with locally-produced micronutrient-rich foods. 11)Expansion into Butembo/Béni: Convene a small group of stakeholders to document lessons learned and mistakes to avoid as RISE begins to scale up in these new geographic areas, using as a point of departure the question: “What will RISE do differently to achieve impact during a relatively short timeframe?” Prepare a well-defined strategy with objectives and targets tailored to the context of Butembo/Béni. 12)Gender: Find creative ways to increase the involvement of men, especially fathers, in SO2 activities. This could include the more systematic recruitment and training of Papa Leaders as peer educators. Conclusion The SO2 activities, based as they are on the PM2A strategy, are relevant and appropriate for this food-insecure environment. A potential weak link in the approach is the fact that many HCs do not have the necessary materials to conduct quality PNC and U5 consultations; this is important because accessing quality curative and preventive health services is one of the three pillars of PM2A. The effectiveness of the approach is also compromised to some extent by the fact that currently there are not enough MLs and field staff to support the large number of PM2A participants. A third weak link that may prevent RISE from reaching targets is the lack of integration between SO1 activities and SO2 activities. As a result, improving availability of food, RISE Mid-Term Evaluation Report 28 Oxu Solutions LLC especially micronutrient-rich foods, to ensure a more diverse diet may not happen for the majority of participants unless a concerted effort is made for closer integration and better targeting. iii. Strategic Objective 3: Strengthened community governance of food security in target areas Overall summary of progress under the Objective Like many other components of the program, the governance component appears to have suffered from difficulties of high turnover in leadership, compounded by an unrealistic design as reflected in the Results Framework and associated activities. As a result, to recover from the delays the SO3 team has devoted considerable efforts to initiating activities from the long list of activities in the Results Framework, but unfortunately the result is a series of unconnected activities that would appear unlikely to contribute meaningfully to strengthened community governance of food security. There were a number of activities undertaken by the first Governance Manager (formerly titled Governance Advisor) and these were summarized in an April 2012 report documenting the progress during the reporting period. The new Governance Manager started in February 2013 after a gap of several months. He initiated a number of activities from the Results Framework, which the project had not yet started. Many of the core activities and achievements do not seem to have strong logical links to IRs or Outcomes from the Results Framework, and there is not a single document that outlines the remaining activities and how they build on previous achievements to realize stated Outcomes and IRs. Most SO3 activities appear to be targeted at the following groups: CARGs (Conseil Agricole Rurale de Gestion, or Rural Agriculture Management Councils): CARGs were established pursuant to a ministerial decree in 2006, but during field work, interviews with some of the Bwito Chefferie CARG members, including the Chefferie Agronomist, yielded a copy of the guidance dated from 2012 citing a 2011 law. This document suggests a system with committee composition significantly different to what the project put in place. The Conseil Consultatif, which is the parallel structure at the provincial level supported by a Coordinator, has been a very significant partner of the RISE project activities to date. Associations: The project selected associations in project areas to participate in trainings, some of which are also referred to the SO3 team by SO1, as they also selected associations for their activities. Generally for SO3 selection, associations had to be registered or have formal statutes to participate in most project activities, although some trainings on the organization and structure of non-profit associations were open to more associations in an area. SO3 activities to date appear to be the same for SO1 associations and for those selected by SO3 staff, and have been fairly minimal (1-2 trainings) but for a very large number of associations (over 125). Locality Development Committees (LDCs): These were put in place by the project when community consultations occurred for the development of Locality Development Plans (LDPs), facilitated by a RISE-contracted consulting firm. Although not presented directly during orientation briefings for the MTE team, the ongoing and planned program activities now seem to be broadly grouped in the following activity areas: RISE Mid-Term Evaluation Report 29 Oxu Solutions LLC  CARG support: Re-activating of Provincial, Territory, and Chefferie CARGs, including training in resolution of land conflicts and Congolese agricultural law, as well as the support to quarterly meetings of the CARGs and broad support to planning processes (although other than the provincial planning process that happened, it is not clear whether or what kind of planning happened and the link to specifically community governance of food security is questionable);  Locality Development Plans (and Local Development Committees): To which Food for Work activities are to be linked going forward;  Support to Associations: Training for associations in 1) structure and organization of non￾profit associations, 2) some training on gender, and 3) two future themes which have yet to be defined, suggesting the lack of a clear vision for what capacities these associations need to develop to contribute to SO3 ;  Food for Work: This has been implemented through December 2013 independently of the SO3 team, with projects selected based on requests from SO1 staff at Mercy Corps or CRS/Caritas. In late 2013, a decision was made to identify projects on the basis of action plans for each locality using Locality Development Plans as a reference, as a means of reinforcing the use of LDPs, but the actual mechanics of FFW implementation was to remain as previously implemented.  Disaster Risk Reduction (DRR): Initial mapping of natural disaster risks by project staff had begun, but was still in the early stages at the time of the MTE and had not yet been documented. Relatedly, there has been no work to date done on Early Warning Systems (EWS). These key activity areas seem to be implemented in ways that do not clearly demonstrate a contribution to stated Intermediate Results or to SO3 as a whole. As such it is difficult to organize observations and recommendations against the Results Framework, and many IRs and Outcomes do not have clear activities which contribute to the result. Discussion of progress on each IR and the IR Outcomes Intermediate Result 3.1 - Community leaders and members work together to address community food security concerns. At this point in the project, there do not seem to have been targeted efforts to encourage or facilitate meaningful interactions between community leaders and community members. Moreover, there were no activities targeted around food security, as the locality development planning exercises were general (although they did include priorities around agriculture, some of which would eventually contribute to food security.) The project, through a consulting firm, spent a significant amount of effort over the course of several months in 2013 to develop Locality Development Plans in 17 localities in Bwito Chefferie (Birambizo Health Zone). These LDPs are discussed relative to this IR, even though the contribution specifically to IR3.1 or to SO3 in general, is not obvious. While LDPs are a common foundational element to general governance programs, the connection to the focus specifically on food security is not obvious. In those instances where FGD participants (who mainly consisted of members of associations participating in the project) had heard of the LDPs, most participants appeared proud of them, referring to them as “the book”. In many cases those interviewed (local leaders) knew the document was in the office of the Locality Chief or with the Groupement Chief. However, many could not cite specific elements of the plan and could not name members of the LDC (which were RISE Mid-Term Evaluation Report 30 Oxu Solutions LLC apparently formed at the end of the community-level exercises to develop the LDPs and were not themselves involved in the actual documentation or review of the plans). There are a few fundamental challenges with the LDPs:  Neither Locality Chiefs nor LDCs would have the means or the capacity to update or review the 150-200 page printed, bound plans. Many components of the LDPs are nearly identical for each locality, which would seem to present significant obstacles to meaningful ownership, even if community members are proud of the nice quality of the document.  LDCs are not active: In many cases they do not seem to have met since the day they were formed and do not have an identified, specific role in the follow-up of the LDPs, raising further questions about ownership.  LDPs do not include sections on DRR or EWS or sections specifically on food security, but rather include more general sections on agriculture and nutrition. While these are not bad things necessarily for a long-term community development vision for a community, these LDPs as written will not necessarily contribute meaningfully to the stated goal of SO3 or to Intermediate Result 3.1. This also seems to be the most appropriate place to discuss the Food for Work (FFW) activities to date. In FGDs and KIIs, these activities were the most widely recognized of nearly all the SO3 activities, more so than the LDPs or the support to associations. Many of the FFW projects, particularly the roads, were described very positively by FGD participants and key informants; however, there were a number of complaints by associations and others about how associations were selected and how the workers were selected for the projects. There were numerous variations in how workers were selected. In some cases, the association selected to oversee the work identified each of the other associations in the community as forepersons for different sections of the road, and they then selected the workers, so that each of the associations in the community had members participating, which appeared to be well-respected by focus group participants. However, complaints arose in other communities where the selected association targeted largely its own members. In many cases, communities acknowledged that workers were often not selected because of vulnerability but were rather association members. One of the difficulties raised by FGD participants was the fact that one of the usual selection criteria for vulnerable HHs eligible to work in FFW projects is the ability for a HH member to work one’s fields (as a sign of physical strength). Based on the FGDs there do not seem to have been formal lists of eligibility criteria, which suggests some potential challenges in the identification of vulnerable HHs for participation, which might have been avoided if greater attention were focused on discussing eligibility criteria with communities in advance. A number of issues were troubling regarding the Food for Work:  Irregularities in the composition and process for tender committees to select the associations who would oversee the FFW project, including some committees with candidate associations being members and winning bidders  Maintenance committees were not in place prior to FFW projects being implemented. The project has recognized this issue, and at the time of the MTE field work, RISE staff were going back to completed projects to establish maintenance committees and distribute equipment. For road “rehabilitation,” for instance, association members or local leaders interviewed (or in focus groups) could not easily distinguish between rehabilitation and maintenance activities, suggesting some FFW activities of paying people food rations to do work might actually harm sustainability. Maintenance committees are planned to be put in RISE Mid-Term Evaluation Report 31 Oxu Solutions LLC place for all new projects and the project staff were reportedly in the process of putting these in place for previous projects, at least in Bwito chefferie. The FFW activities were not previously linked to SO3, and were actually run in conjunction with the commodities team under a separate expatriate manager, so the governance problems might be explainable, but some of the above-mentioned variations run counter to even basic procurement rules. Linking these activities to SO3 activities does make sense as they can be used to reinforce community decision-making, planning, and governance; however, these processes need to be clarified and adequate time and attention given to the processes to ensure that these activities, which are highly visible in the communities, model good governance over public resources. From a design perspective, many FFW activities (including road and market structure rehabilitation) will continue to contribute to the logic of SO1 in terms of improving physical access to markets. Outcome 3.1.1 - Community stakeholders effectively organized and advocated for improved local food security governance There appear to be no activities completed which meaningfully contribute to this Outcome. Outcome 3.1.2 - Local leaders are receptive to community-raised concerns There appear to be no efforts to improve mechanisms by which community concerns are raised to local leaders (see comments in IR 3.3 regarding the appropriateness of chefferie committees as “local” or “community” leaders). Outcome 3.1.3 – Small-holder farmers improve access to land There appear to be no activities related to this Outcome. Some trainings were done for CARG members on management of land conflict, but there does not have seem to be an analysis suggesting that land conflicts are a major obstacle to land access, as most conflicts mentioned during field work interviews and focus groups were linked to land boundary disputes and destruction of crop areas by livestock, which do not seem to be major contributors to food insecurity more broadly, although certainly potential contributors to local conflicts. Conclusion: Little progress has been made toward achieving to IR 3.1, due largely to the lack of progress toward the individual outcomes and the fact that some outcomes do not logically contribute clearly to the IR. Intermediate Result 3.2 - Communities have increased capacity to anticipate and mitigate food security risks This IR has not been significantly addressed, other than the initial mapping of natural disaster risks. Outcome 3.2.1 - Community disaster risk reduction plans have protected food security As noted above, there have been some efforts by project staff to begin to map disaster risks but the details of how DRR plans would be developed and the specific nature of community engagement were not yet clear at the time of the mid-term evaluation. Outcome 3.2.2 - Communities are making decisions informed by early warning systems There appear to have been no efforts to date to develop early warning systems in project areas. LDPs were put in place before the DRR analysis was done and the LDPs do not include aspects of EWS. An analysis of disaster risks in each locality has begun and then is expected to be incorporated into action plans for each locality linked to the LDP. RISE Mid-Term Evaluation Report 32 Oxu Solutions LLC Conclusion: Little progress has been made toward achieving to IR 3.2, due largely to the fact that this work was not integrated into an overall planning process which would ensure discussion by communities of disaster risk, and the initial mapping done so far seems to be focusing more on the project staff identifying the risks consulting some community stakeholders, for the sake of expediency. Intermediate Result 3.3 - Communities have improved linkages with territory and provincial food security structures To achieve this IR, one must implicitly ensure that there are actions taken by territory and provincial structures and by “communities” either directly or via community structures. There was some confusion in the support to the CARG system about what system is being developed. The project sought to reconstitute or redynamize the CARGs at the chefferie, territory and provincial levels, but there was some confusion during the MTE about the guidelines by which they should be formed. In most cases, it seemed that they were constituted in the way that authorities indicated they should be constituted and not according to the 2010 law referred to in the 2011 decree (the latter was obtained from the Bwito Chefferie agronomist. The CARG for Bwito Chefferie and for Rutshuru Territory were not set up according to this 2010 decree, and there are a number of concerns:  The President and Vice President positions were given to the head of the chefferie/territory and the Agronomist respectively, creating a dynamic of dominance by the local authorities, whereas the Chefferie Chief should be only an honorary president, and there should be an elected coordinator position;  The committee participation seems heavily weighted to those people who live near the center of the chefferie or the territory, and provisions requiring representatives from all the groupements were not respected, raising questions about the representativity of the committee. There has been some work with leaders in the CARG system at the province, territory, and chefferie levels, but it is difficult to consider chefferie representatives as a “community” structure given the size of a chefferie, and it suggests a major gap in the achievement of this IR. The project has undertaken some training of CARG members and supported the development of a provincial plan for agriculture, which would likely incorporate some aspects of food security, but this plan is not clearly linked to input coming directly from community members, via groupement and village structures, as is described in the 2011 decree. Understanding the latest texts and the fit with decentralization should be a critical part of supporting the implantation of such structures, and should be researched further by the team. Outcome 3.3.1 - CARGs have increased capacity to effectively play their role CARGs have been put in place where they did not previously exist and some interactions, via a provincial planning process have reintroduced mechanisms for interaction between chefferie-level CARGs and the province. These interactions do not appear yet to reinforce the roles specific to the level of the entity (e.g. chefferie vs. territory), and there seems to be significant confusion about the role of the CARG relative to the Ministerial decree of 2010. Outcome 3.3.2 - Two-way information flows are established between CARGs and community￾based food security structures CARGs were put in place for all chefferies and territories within North Kivu, including the four chefferies and three territories of the current and future project areas. However, in field work RISE Mid-Term Evaluation Report 33 Oxu Solutions LLC areas, the Rutshuru territory CARG representative interviewed suggested that the chefferie-level CARG was not active, as it is largely overshadowed by the Territory CARG. This might be partially explained by the inability of one of the members of the Territory CARG to articulate the CARG’s role in a way that refers to and builds on the work of the chefferie-level CARG. Indeed, during the interview, the Rutshuru territory CARG representative repeated similar messages about the role of the CARG to sensitize the population on improved agriculture techniques and use of improved seed varieties, rather than mentioning any sort of oversight of territory or chefferie service providers or to advocate for needs raised by chefferie CARGs. Moreover, there do not yet appear to be any structures in place intended to provide the two-way information flow from community-based structures. Outcome 3.3.3 - Two-way information flows are established between CARGs and Provincial MoA concerning community food security The project supported the provincial equivalent of the CARG to conduct a planning process for agriculture, which involved the representatives of chefferie and territory CARGs. It would appear that the provincial plan, which includes sections related to each of the chefferies, demonstrates a linkage between Provincial MoA and CARGs, but the linkage to community food security is not evident, given the lack of apparent interface between chefferie-level committees and groupement, locality, or village levels. Conclusion: Some progress has been achieved toward IR3.3, in that the Provincial CARG structures and systems reportedly have made progress and are more dynamic than previously; however, the connection to the community level still seems quite far away. Further general observations and SO3 recommendations Recommendations The design of Strategic Objective 3 seems to have been challenging from the outset, with a longer list of activities (42) than for SO1 (29) and for SO2 (11) combined. As a first step, similar to the recommendation for other SOs, the project should develop a more detailed strategy document which shows the targeted community structures/actors, explaining what would be their intended role at the end of the project (which should be linked to their role in other DRC policy documents) and how the project will support their capacity to play this role. This strategy document should link the Results Framework Intermediate Results and outputs to the activities and indicators. Given that half the project timeframe has already been completed, the project should realistically seek to use the remaining two and a half years to focus on consolidating some of the rather separate SO3 activities that have taken place to demonstrate interactions specifically related to food security between communities, via community structures (i.e. at the village or locality levels) with authorities and service providers at the locality, groupement, Health Area, or chefferie levels. In the remaining time period, we recommend: 1) Strategy: Revise the Results Framework to focus more on community level engagement and discussion of food security problems, planning, and oversight and accountability around resources for food security. 2) LDCs: “Complete” the LDC membership in accordance with the decentralization laws for Village Development Committees11. Representation of the more remote villages in this 11 While this should be researched and verified by the project staff thoroughly so that future activities do not run counter to actual or upcoming laws or guidance, the decentralization laws anticipate the formation of village development committees as well as groupement development committees. Under the restructuring RISE Mid-Term Evaluation Report 34 Oxu Solutions LLC committee is very important, as is including representatives of women, youth, and vulnerable or marginalized groups. 3) Food security plans: Develop Food security action plans that are linked to LDPs and ensure sustainability of RISE SO1 and SO2 outcomes. The LDPs should be adapted by the LDCs to take into consideration a fixed period of time and potentially available resources. A timeframe of 1.5 years is proposed, given that there are 2.5 years left in the project. It will take several months to go through this process, and a buffer to ensure completion before project end would be prudent. As the project likely intended initially, this planning process should be done in conjunction with the SO1 and SO2 teams, and ideally with the Ministry of Agriculture and Ministry of Health service providers to: focus on the sustainability of the SO1 and SO2 activities, engage with communities on how to maximize the effects of these interventions, and ensure that they contribute to the LDPs, being conscious of the resources which the project can bring to communities as part of those SOs. This LDP action plan can then become the central point of interaction between community members and service providers, particularly agriculture and health service providers, and local chiefs, to engage on food security issues, which is what SO3 sets out to do. These plans should also include an analysis of conflict risks in the area and what communities can do to minimize the consequences to food security (as well as natural disasters, if this analysis has already been done). 4) LDCs: Support the LDC in each locality (properly formed to be fully representative) to launch a call for projects specifically linked to the food security action plans (and ideally to the sustainability of SO1 or SO2 activities or to DRR): Money or support for anything ongoing should be limited, and the selection of the associations and projects should be led by a representative LDC to ensure that any resources provided to the association will benefit the entire community’s food security in the long term. The exception might be for a project for a particularly vulnerable group within the community, in which case agreement on the criteria should be transparent and clear. The LDC, local service providers (e.g. locality and groupement agronomists, health center staff), and local chiefs should feel ownership and responsibility to follow up with the association. The calls for projects should be very specific, and should involve very few “functioning” costs for the association to foster those who are already engaged in a particular sector, or to encourage certain activities that can be sustained after the project end (e.g. advocacy, or potentially mills which can then fund continued functioning of a demonstration plot, etc.). The process followed here (including the representativeness of the LDC) will be important to contributing to the community governance of food security. 5) CARSs and CARTs: Reduce engagement with CARS (“Conseil Agricole Rurale de Secteur”, or Secteur/Chefferie Rural Agriculture Committee) and CART (“Conseil Agricole Rurale de Territoire”, or Territory Rural Agriculture Committee) until local structures have been developed, and seek to develop linkages in the last year of the project. CARS and CART should be informed of the activities with the localities, and should be retrained to understand that these decentralized territory entity (“entités territoriales décentralisées”, or ETD) structures (at the groupement, in particular) are their conduits to gain understanding of issues in their community. This should reduce the demands for missions discussions underway as part of the decentralization, the VDC would actually be at the equivalent of the current “locality” as the government will seek to re-group smaller villages into more consistent politico￾administrative structures. These structures are mentioned in the texts found during the MTE from the Bwito Chefferie but do not seem to have factored into project planning or even the design of the activities linked to the CARG, which seems quite problematic. RISE Mid-Term Evaluation Report 35 Oxu Solutions LLC throughout the community and the unrealistic expectation that they are supposed to do direct sensibilization of communities in the chefferie. There does need to be significant work to re-orient CARS and CART members, which should also be “completed” to be in compliance with the ministerial order defining them, to ensure that they represent the entire chefferie (or territory) and not just those who happen to be close to the “cité” which is the center of the chefferie. Their role vis-à-vis chefferie, territory, and provincial authorities needs to be re-established to combat the dynamic that seems to exist that the political authorities “supervise” the CARS or CART and the CARS or CART is supposed to sensitize communities. On the contrary, the sensitization of communities on agricultural practices and uses of improved seeds ought to be the role of service providers and the CARS and CART are supposed to play an oversight role of the chefferie and territory service providers and resources, based on information gathered about community needs, to ensure that services are being provided that are responsive to community needs. 6) Groupement Development Committees: Consider the formation of the Groupement Development Committees, which are formed by representatives elected from each Village Development Committee. There are advantages and disadvantages to this, but this is a critical element to putting in place the two-way feedback structures to the CARS, as these are the ETD structures between the chefferie and the village (currently locality), and are the general development structures to which the CARG system is to refer at levels below chefferie. 7) DRR: Where natural disaster risk analysis has not already been completed, disaster risk reduction should focus initially on conflict disaster risk and those natural disasters which communities have encountered: Communities already have knowledge and experience to analyze conflict risks and how to mitigate the consequences, and in fact the project has had to pull out of areas or suspend activities for several days in nearly all the project areas due to the conflict. Arguably, had this component of the project been implemented effectively for conflict and natural disasters from the beginning, the project might not have needed to withdraw from Rutshuru. While it would certainly be ideal to also address natural disaster risks in the remaining project timeframe, given that community planning activities have not been done over a long period of time and in a deeply community-driven way, it is better to focus on addressing real and probable risks which communities are best equipped to understand and actually take actions. Where the natural disaster risk analysis has already been done, or where communities can readily identify them, like unusually heavy rains that destroy crops, etc. this should be incorporated into the food security action planning sessions along with risks for insecurity. 8) Targets: The SO3 team indicated significant pressure to reach these numbers through trainings, particularly to association members and government. This focus should be on community participants in community meetings on the specific subjects of food security where problems are debated and decisions are made through engagement between communities and services providers. This is much more consistent with the objective and IRs of SO3, and the target would need to be revised, but should be achievable given the emphasis on a few very participatory community planning meetings. The specific list of eligible “activities” to count as an SO3 beneficiary should be detailed explicitly. Conclusion As stated in the introduction to this section, SO3 has suffered from a problematic design from the outset as well as senior staff changes. The project has initiated a number of activities which are consistent with governance programs, but do not logically reflect clearly a governance component of a food security program. The support to the CARG system was initiated without a clear linkage to the community levels (village and locality) where the rest of the project works. The other aspects RISE Mid-Term Evaluation Report 36 Oxu Solutions LLC did not have a clear connection to food security. A number of the project achievements can and must be leveraged toward more focused food security outcomes in the remainder of the project. Ideally, the component would have been an entry point for joint planning with communities, ensuring coordination in the identification of SO1 and SO2 beneficiaries and activities, and addressing disaster risk reduction and early warning systems which are very appropriate for these areas. With only 2.5 years left in the project, this is not possible, but a number of the recommendations above, focusing on food security action plans is an effort to build on what has been accomplished in a way to initiate some concrete and focused activities related to food security. The SO3 team seems to recognize these issues after discussions during the analysis workshop held at the end of the MTE field work, and they did accomplish a significant amount of work in the last 9- 10 months of last year, so there is a good chance that with greater focus and precision and stronger monitoring of quality, the project can make a substantive contribution to food security governance and to the sustainability of project interventions. RISE Mid-Term Evaluation Report 37 Oxu Solutions LLC IV. General and Cross-cutting Observations and Recommendations a. General Observations The RISE project has faced a number of challenges in its first two and a half years, which have created delays or difficulties in implementation. These include:  High staff turnover, particularly at senior levels, including several changes in the chief of party position and at least one turnover for each of the SO coordinator positions: The result has been gaps in leadership coupled with inconsistent and delayed project implementation.  Insecurity in many project areas, leading to a significant geographic shift in Year 2: This has meant that many activities have been temporarily suspended on numerous occasions, and implemented for a total of less than 18 months, making it difficult for the evaluation team to draw definitive conclusions. In addition, the RISE team has been forced to rapidly expand activities in new areas, sometimes without adequate preparation or resources.  Inconsistent approaches and lack of clarity in communications within the project itself and between the project and the participating communities: The result is that the same program sector and even the same activity may be implemented differently from one geographic area to the next. In addition, there are few linkages among SOs and there is limited integration of beneficiaries across project sectors. There is also confusion – seemingly among communities, project staff, and other key stakeholders – about whether RISE is really a development program as some of its implementation characteristics more closely resemble an emergency project. In general, many of the project approaches and activities seem more oriented toward an emergency program as opposed to a development program. Many of the SO1 activities, for example, rely on directly giving inputs, money, and food to beneficiary HHs, without requiring any real contribution from the beneficiary. There are several project activities that may have lasting effects but not necessarily to the degree intended in the project design. Many of the activities are implemented in isolation of each other (e.g., no obvious attempt to have SOs reinforce each other and a lack of strategic targeting of HHs and communities), and there is often not enough attention paid to the way in which communities would continue to practice the behaviors and techniques being promoted by the project. Another element that suggests more of an emergency approach is that there are few observable commitments that are being clearly articulated to beneficiaries and communities by the project, other than the sensitization around the PM2A ration duration. During the FGDs and KIIs, project participants and even some RISE field agents were not able to articulate what the next steps were for a particular activity, as everyone seemed to be waiting for instructions for what would come next. Without a good sense of how the project is meant to evolve and what the mutual commitments are between participants and the RISE project, there is very little chance that communities can truly own the project outcomes. In addition, as the project is currently being implemented, it does not encourage community accountability: in many cases participants stated that they did not know the quantities of seeds they are expected to receive, what will happen to the rabbits they are raising, and what their “rights” and “responsibilities” are related to project activities and outcomes. Further, while MC and partners use a complaint hotline that beneficiaries can access on distribution days, our findings indicate that there is no additional project accountability system in place (though plans for an RISE Mid-Term Evaluation Report 38 Oxu Solutions LLC accountability system and hiring an Accountability Coordinator were mentioned in the Year 2 Annual Results Report). For an undertaking as complex as the MYAP a much more comprehensive accountability system should have been established at the outset of the project (and still should be put in place). Such a system could be based on the Humanitarian Accountability Principles; given the context of EDRC, an accountability system would also need to include a way for beneficiaries to report incidents of sexual and gender-based violence (SGBV), theft, and insecurity. Targeting is another general area that does not seem to follow a clearly defined strategy. The FY11-FY15 Food Security Country Framework for DRC recommends MYAP programming to focus in target regions at greatest risk of chronic food insecurity for the following groups: chronically food insecure smallholder farming HHs, women farmers and female-headed HHs, conflict-affected HHs, PLW and children under two (CU2), and adolescent girls and SGBV victims.12 While targeting for SO2 follows the standard PM2A targeting, there was no evidence to show that SO1 and SO3 targeting includes the vulnerable groups listed above, and especially the most food insecure HHs. In fact, activities under these objectives seem to target those individuals who are the most involved in their communities and not necessarily the most vulnerable. Moreover, there also seems to be a bias toward implementing activities in larger population centers such as near HCs and in locality centers. The evaluation team could not assess this directly, but there is a risk that the project may not be as effective in reaching the most food insecure and those in more rural areas. Another targeting concern mentioned in the SO2 section is that SO1 and SO2 activities are not systematically targeting the same communities and HHs. This raises serious concerns for the sustainability of any SO2 outcomes, other than knowledge change and some improvements in practices. If the availability of and access to food, especially more diverse food groups, are not improved for HHs participating in PM2A, the gains in nutritional status for PLW and young children may not be sustained post-project. Finally, from a management perspective, the evaluation team observed a number of issues: 1) Lack of structured monitoring: a. The Goma-based teams, especially managers, do not seem to have a well-defined plan for regular field visits with targeted monitoring objectives. b. A system for monitoring the quality of activities has not been put in place. The M&E team is preparing a system now, but it is not yet operational. 2) Lack of documentation: There is a serious lack of documentation for many aspects of RISE. One of the most striking gaps in implementation is the lack of strategy documents laying out a vision for each SO and for each cross-cutting theme such as BCC. In addition, the project seems to have made no real progress in detailing a sustainability or exit strategy (other than the sustainability framework that was provided as an annex to the original proposal). There is no systematic recording of decisions, data analysis reports, response to critical internal evaluations, or even brief reports on successful campaigns. Without a well-documented, systematic and regular review of activities, including lessons learned and mistakes to avoid, it is difficult to tell which approaches and even which activities are successful and should be continued. Course correction is hampered by the lack of documentation, and lack of documentation means minimal project institutional memory. High staff turnover – especially in management positions – coupled with minimal documentation and little institutional memory are recipes for failure. 12 USAID Office of Food for Peace Food Security Country Framework, the Democratic Republic of Congo, FY 2011 – 2015. RISE Mid-Term Evaluation Report 39 Oxu Solutions LLC 3) Efforts are scattered: a. RISE is implementing a laundry list of different activities happening in different areas, without a clear logic to why certain activities are in certain geographic areas (and not in others). The exception is SO2, which implements a fairly standard package of activities across geographic areas. b. Geographically, efforts have become more dispersed than before with the expansion in the Birambizo Health Zone (Bwito Chefferie) and in Beni and Butembo, while still continuing some activities in Rutshuru (Bwisha Chefferie). There is a greater risk now that the project is being stretched too thin. In particular, the MTE team had serious concerns about the decision to undertake this expansion, especially for: i. The expansion into Beni and Butembo: The MTE team noted a distinct lack of clarity combined with real concern among project staff about which interventions would be implemented in Beni and Butembo and how (in terms of staffing and budget for scale-up). Moreover, the reason for expansion into new geographic areas shift was not clear, particularly given that the decision to enter Beni and Butembo was made after security had improved in Bwisha Chefferie (Rutshuru area). Also, it is still very uncertain whether the necessary resources in terms of budget and staffing are available, particularly given the significant distance to those areas, relative to the initially budgeted project areas. ii. Status of Rutshuru (Bwisha Chefferie/Rwanguba Health Zone and Karisimbi): Some project documents suggested that the activities in these areas were “scaled back” due to insecurity. However, during interviews and in discussions with project staff, it was stated that USAID/FFP had made the decision for RISE that they needed to pull out of Rutshuru because of the M23 addition to the OFAC list and other security concerns. In some instances, “close-out” rather than “scaling back” was mentioned. What is clear is that RISE project staff are re￾starting support to women’s and youth associations and VSLAs; there is, however, no clear articulation of how these isolated activities would contribute to overall project objectives, in particular food security, in areas where no nutrition or agriculture activities are expected to continue. When members of the MTE visited these areas, community members continually expressed their expectation that the project had suspended activities due to insecurity and was coming back now that security had been restored. They had not received any other information despite the fact that security had returned and M23 had withdrawn from Rutshuru in late October 2013 (more than three months prior to the field work the MTE team conducted). A rough estimate would suggest that it would take at least 3-6 months to fulfill the commitments which community members say they are awaiting (if their assertions are correct, and even if not correct, significant sensitization would be required). b. General Recommendations Modify the RISE Results Framework, especially for SO1 and SO3 In our view, the project design from the beginning was overly ambitious given the myriad challenges of the EDRC context. In addition, a major geographic shift and high management staffing gaps and turnover mean that, from our perspective, a redesign of the Results Framework needs to take place to have more modest and realistic objectives. While RISE has reached a significant number of its target beneficiaries, our findings have suggested that the quality of implementation and level of medium-to-long term impact on the beneficiaries reached to date are in question. A RISE Mid-Term Evaluation Report 40 Oxu Solutions LLC proposed modified RISE Results Framework is provided as Annex H2; it provides a more narrow focus, especially for SO1 and SO3. Implement a system for better project documentation. As a development project of this size and scale, especially with the changes that have occurred, the project needs to document things more consistently and effectively. It was extremely challenging to find key project documents, and many were not available. This may require taking a pause in implementation for two weeks to develop these with the project teams, or seek external support from the respective organizations headquarters, or an external consultant, but we do believe this work of revising the Results Framework and the documents suggested below will help the work bet more focused and efficient. Specifically, the project should place emphasis on the following key documents:  SO and other key component strategies (e.g. BCC): Each SO and cross-cutting theme should have an overall strategy document which links key project activities to the Results Framework, the IPTT, and the DIP, as well as the Activity Protocols and Exit and Sustainability Strategy mentioned below. These documents should provide a vision for the SO or the theme through the end of the project and beyond. Such documents should be dynamic and a process should be articulated for regular review and updating. This should involve technical support from the consortium partner HQs to avoid dramatic changes in implementation every time there is turnover in project staff.  Clear and concrete exit strategy: The project should develop a more detailed strategy document which identifies the targeted community structures/actors, explaining what would be their intended role at the end of the project (which should be linked to their role in other DRC policy documents) and how the project will support their capacity to play this role. This document should be able to link the Results Framework Intermediate Results and outputs, to the activities, and indicators (see Annex L for a sample exit and sustainability strategy from the CRS MYAP in Madagascar).  Protocols for key project activity packages: These will help ensure consistency of implementation and should highlight the specific steps to be taken for activity implementation for key project activity packages and how they contribute to quality of outputs related to the Results Framework. They permit the review and agreement by Mercy Corps, CRS, Caritas, and CEDERU, and also can permit reviews by respective technical advisors of those organizations. The protocols can plan for differences based on context and give guidance as to when such variation would be appropriate or not, which can help in monitoring and follow-up of these activities. These protocols do not need to be extremely long, initially, but even basic protocols would likely remove some of the issues of variations in implementation seen across areas, particularly for SO1 and SO3. Such protocols would also ensure better quality management in the scale-up of SO2 activities. A sample template is provided in Annex M1, along with two examples provided in Annexes M2 and M3 from the CRS MYAP in Madagascar. The latter annexes, while lacking a few details which are recommended in the Annex M1 template, show concrete example of documenting certain project activities.  Key decisions (particularly in relation to changes in project scope or geographic coverage): A document presenting a justification for the shift in geographic areas was sent to USAID for approval. While it is useful in documenting some aspect of the decision, it does not provide enough detail for internal purposes to guide the action of project staff. This explains to some degree why it was very difficult for the MTE team to get a clear answer regarding: o Expansion into Beni and Butembo: Which activities will start when, and how will the planned staff be adequate for the require activities. RISE Mid-Term Evaluation Report 41 Oxu Solutions LLC o Status of Rutshuru (Bwisha Chefferie/Rwanguba Health Zone and Karisimbi): At various times, project staff, including managers, used different terms for the situation in Rutshuru – scaling back, closing out, and continuing. Finally, it seemed that the closest understanding we had was that VSLAs and support to youth and women’s associations were continuing, and that agriculture, specifically seed multiplication activities, were phasing out after the next harvest. This was not clearly documented and understood by all project staff, and most importantly to beneficiaries in those communities, many of whom had very specific outstanding expectations from the project which would presumably still take some time to close out. Conduct a refresher training on development practices for project staff, emphasizing techniques for interacting with communities, a “Do No Harm” approach, and what can be sustained post￾project. This refresher training should also emphasize the need for project staff to think about the holistic needs of the community and how all project components interact within a community. Staff should also understand that doing this well requires understanding the differences in the power dynamics within each community, as well as socio-cultural and ethnic differences within and between communities. Improve monitoring, data analysis, and planning: There should be improved monitoring of the quality of outputs according to regular schedules, particularly by Goma-based management (sectoral and general) and by the M&E team. This should then also feed into more quantitative data and indicator tracking, and these must actually be used to make decisions and incorporate feedback into subsequent planning.  Hold a meeting with all sectors to analyze the quarterly data before finalizing plans for the following quarter.  Develop a hierarchical planning system: multi-year (by quarters), annual (by month), quarterly (by week), monthly (day by day).  Initiate annual project reviews for the remaining years of the project. Use more consistent terminology throughout the project:  Refer to the project as SIMAMA: We would recommend referring to the project as SIMAMA, as this is more meaningful for community members. RISE could still be used for donor reporting, if USAID/FFP prefers this. However, even there, consistency in terminology would be ideal to avoid confusion and to create stronger identity.  Use “Project Participants” with communities and internally, even if reporting to the donor continues to use the term “beneficiaries” for IPTT requirements, etc. c. Cross-cutting (BCC and gender) i. BCC In the original project design, HKI was the consortium partner that had responsibility for BCC. The BCC strategy was intended to be a cross-cutting theme supporting all three RISE Strategic Objectives, promoting “…effective knowledge transfer and self-efficacy to create actual behavior change aimed at supporting sustainable and valuable increase in agriculture production, nutritional improvement and resilience to food insecurity in the targeted communities.”13 However, HKI withdrew from the consortium in November 2012 due to security concerns and this vision was not 13 RISE project proposal RISE Mid-Term Evaluation Report 42 Oxu Solutions LLC realized. Following a six-month period of limbo when little was done to further the BCC strategy, activities picked up again. Currently, both Mercy Corps and CRS/Caritas have BCC teams, with Mercy Corps directing their actions to SO1 and SO3 and CRS focusing primarily on SO2 with the possibility of some interventions for CRS’s agriculture component. The Mercy Corps BCC group team includes two teams. The mass media team is working with community radios, training “development journalists” and planning for radio listening groups on RISE-specific topics, including gender equity themes. The media team is also preparing learning videos, with the first three focusing on women entrepreneurs in the project area; these will be shown as success stories to promote discussion in RISE communities and to inspire others. The other Mercy Corps BCC team focuses on campaigns, with the first one highlighting SO1 messages; it is scheduled to start in March 2014. The CRS/Caritas BCC is composed of a BCC point person and three field agents, two assigned to Birambizo and one assigned to Butembo. Although BCC is not the unifying component envisioned for the project, it does have a number of strong points:  Both the CRS and the Mercy Corps team leaders have participated in Designing for Behavior Change workshops, which provide a solid foundation for developing BCC activities.  Both CRS and Mercy Corps are making good use of formative research including the baseline KPC survey, FGDs, and barrier analyses for key behaviors for all three SOs  LQAS is being used to monitor progress in BCC. One survey has already been conducted for SO2 and three more are planned, two for SO1 and one more for SO2. When the results of the August 2013 LQAS showed low levels of knowledge/adoption of hand washing, recognition of danger signs, and the importance of completing four PNCs, the SO2 organized FGDs to seek additional information. The results of the LQAS and the FGDs were widely shared with project staff and MLs so that BCC messages and counseling could be tailored to be more effective.  A variety of communication channels and activities are used, including: o Interpersonal communication (the MLs have proven very effective for SO2) o Demonstrations (cooking demonstrations for SO2, agriculture demonstrations for SO1) o Radio spots (mentioned especially by men during the SO2 midterm FGDs as one of the principal ways they learned about health and nutrition) o Visual aids (posters in HCs, portable flipcharts, booklets, and photos) o Videos as teaching tools o Listening groups  The SO2 team includes BCC training for its field agents, MLs, RECOs and MoH personnel. The major weakness of the BCC component is that there is no overall strategy for the project; neither Mercy Corps nor RISE has a finalized written plan detailing activities for the remaining two and a half years. Other important weaknesses include:  RISE does not take advantage of all the opportunities for BCC activities (missed opportunities).  There is no real effort for cross-messaging from one SO to another.  Many field agents and managers do not see themselves as change agents and may not have had even basic BCC training in how to promote better practices.  There has been a definite effort to provide BCC support materials, especially for SO1 and SO2 and some of the materials are nicely done. However, other materials could be improved. For example, some of the SO1 and SO2 materials are in French instead of the local RISE Mid-Term Evaluation Report 43 Oxu Solutions LLC language, severely limiting the audience. (See also the discussion of SO1 materials with examples in the SO1 section.) BCC Recommendations: 1) Strategy: Mercy Corps and CRS/Caritas need to prepare a joint BCC plan/strategy for the remaining 2.5 years, harmonizing as much as possible their respective elements so as to have a comprehensive, coherent plan. The strategy should include some of the “missed opportunities” (see following Recommendation) and plans for cross-learning between the Mercy Corps and the CRS/Caritas BCC teams. 2) Maximize messaging opportunities: Take advantage of all opportunities for BCC messaging. For example: a) Set up demonstrations and exhibits at markets and other public gatherings. This could include a model of a hand washing station, a display of improved varieties (SO1) with someone explaining the advantages, or a photo exhibit illustrating a success story. b) Work with religious leaders to incorporate some of the key messages into church gatherings. A number of NGOs such as World Vision have guides for working with religious leaders. This might be a particularly good venue for promoting responsible parenthood and gender equity. c) Consider working with schools, especially for WASH messages and demonstrations. d) Use FFW audiences for simple messages and demonstrations. e) Use FFS to provide project-related BCC messages in nutrition and hygiene. 3) BCC Curriculum: Develop a basic curriculum with five to ten key messages that could be used for any SO audience. For example: a) During SO1 activities, promote the nutritional value of certain crops. It was surprising to hear that farmers did not know that the bean variety they had planted was bio-fortified. b) In both SO1 and SO2, disseminate the message about keeping part of the harvest for seed, part for consumption and part for sale. c) Ensure that at least one standard message on gender equity is woven into BCC for all three SOs. 4) Staff training: Prepare a brief, 1-2 day training for all SOs, focusing on adult learning, communication techniques and practical exercises on how to be an effective BCC agent. ii. Gender In general, RISE seems to offer women relatively equal, and in some cases even greater, participation in project activities than men. Thus, gender does not seem to be an obstacle to participation at this point: - SO2: PM2A activities are by nature directed toward women, given the focus on PLW. - SO1: More than half of the participants in SO1 activities are women (e.g., VSLAs, vegetable seed distribution and small animal solidarity chains) and during MTE interviews, women stated that their challenges as well as their access to SO1 activities are similar to men’s. However, the project has not sought to address the issues of targeting widows or female heads of households (FHH). Commercialization activities, which have not yet started, might present a more significant gender bias in favor of men. - SO3: Women are represented on various committees (VDCs and CARG), but generally in relatively low numbers. Anecdotal evidence regarding the selection of associations for FFW suggest that the informal committees brought together for these activities might not include meaningful participation of women or may not even have women present. The more significant issue here would be active participation in the committees, but since these committees have not yet engaged in many activities there was little to verify during the field work. RISE Mid-Term Evaluation Report 44 Oxu Solutions LLC These aspects only address access to activities and do not necessarily address systemic imbalances to improve outcomes for women. Thanks in large part to an excellent gender assessment completed by a consultant in September 2012 (RISE Gender Assessment, Dana Benasuly, September 2012), the project staff is well aware of how to improve its positive impact on outcomes for women, but 16 months later has made little progress in implementing the recommendations, several of which are not technically that challenging (e.g. removing or modifying some eligibility criteria for the selection of associations in SO3 activities). The MTE team agrees with the content of the gender assessment report, which is very comprehensive with clear, actionable recommendations. The MTE team does recognize, however, that some of the recommendations may take time to implement and would require additional staff and financial resources. Nevertheless, at a minimum RISE staff should go through the recommendations to: 1) determine which ones actually do require outside technical assistance to implement and which ones are only slight modifications to current project practices; and 2) develop and implement a plan of action. The SO3 team has made efforts to address one of the recommendations in that they organized a gender training for CARG members and some association members although the training does not appear to have been linked to targeted follow-up activities by training participants, nor targeted follow-up support from project staff to ensure that actions were taken. The SO3 team and the Mercy Corps BCC team are also including gender concerns in BCC: two of the first three BCC messages for SO3 focus on women’s associations and women’s role in HH decision making. Another follow-on to the report is Mercy Corps’ decision to hire a Gender Advisor. We recommend that the SOW for this position be expanded slightly to be a social inclusion advisor as some of the issues around access and gender are also influenced by vulnerability and vulnerability criteria might help identify those who are the most food insecure. The project should prioritize the following key recommendations related to gender, several of which are taken from the gender assessment. Gender Recommendations: 1) Men’s participation: Engage men more in SO2 activities. The gender consultant recommended the development of strategies to include men throughout the SO2 activities in order to emphasize the importance of nutrition-related activities and to engage men more meaningfully in decisions related to reproductive health and child care. The discussions by the SO2 team to formalize the work with Papa Leaders might provide a useful entry point to initiate progress relative to this recommendation. 2) Women’s associations: Target greater support to women’s associations to play a more active role in community activities and decision-making, particularly related to project activities. Pursuant to one of the SO3 recommendations that the associations have documented statutes, (also a recommendation in the gender assessment), ensure that some associations (ideally one per locality) be women-focused and women-run; focus on their capacity to engage in community discussions. 3) Women’s decision-making: Encourage discussion of women’s involvement in decision￾making at the household and community-level in discussions with local leaders and community members regarding action plans (linked to the LDPs). 4) SO1 Marketing activities: Commercialization activities related to SO1 should not begin until specific strategies are developed to ensure long-term equitable access and outcomes for women, as the gender imbalance is likely to be more significant for these activities. RISE Mid-Term Evaluation Report 45 Oxu Solutions LLC d. Commodities14 RISE commodities are distributed in three ways: 1) Through PM2A/SO2/CRS to registered PLW and CU2 – the majority of the overall tonnage, including 100% of the CSB and 62% of the vegetable oil 2) Through FFW/SO1 and SO3/MC for public infrastructure rehabilitation – the second largest amount, including 85% of dry goods (i.e. yellow split peas and corn meal) and 32% of the oil 3) Through FFT/SO1/MC as incentives for community volunteers, model farm workers and rewards for PG members, including 15% of dry goods and 6% of the oil Table 5 below provides a summary of the planned vs. actual distribution of commodities during the first two years of the project. Table 5: Commodities Planned vs. Distributed, FY2012-FY2013 Year15 Quantities distributed (MT): Planned16 Quantities distributed (MT): Realized17 FY2012 884 63 FY2013 1650 128218 Total 2534 1345 % realized compared to planned in first two years of project: 53% The discrepancy in the table above is a result of a later than planned start for distributions in Year 1 as well as slower than planned registration of pregnant women in PM2A in Year 2. Mercy Corps did end up beginning FFW distributions ahead of schedule to attempt to compensate for the PM2A delays. According to interviews with MC staff, the project is approximately six months behind schedule with commodity distributions, which MC stated that it intends to make up by the end of the project. In the remaining three years of the project, MC plans to distribute 4718 MT in Year 3; 2462 MT in Year 4; and 678 MT in Year 5.19 The additional 841 MT difference from Years 1 and 2 will need to be allocated across Years 3-5 as well to reach the total of 10,392 MT allocated to the project in the Cooperative Agreement Modification #8. To ensure distribution of all planned commodities, a detailed distribution plan for the remaining months in the project should be put in place as soon as possible. This plan can be completed now, especially since MC has already set end of April 2014 as the point beyond which there will be no new PM2A beneficiaries entering the program (given that there will be approximately 31,000 PLW and CU2 enrolled at that point). Beneficiary selection for PM2A seems well-managed, reaching very nearly all eligible women (as reported in FGDs) through registration at the health centers. Anecdotal evidence suggests that the 14 Please note that the project is utilizing cash from monetizing commodities through participation in a monetization consortium led by Food for the Hungry (FH). Given FH’s lead role in performing monetization for all DRC MYAP awardees, our evaluation focuses on commodities distributed through the RISE project and does not include a review of monetization done to date. 15 The project’s Implementing Year and Fiscal Year calendars are one month off (IY is Sept-Aug; FY is Oct￾Sept). For this table, we have used results based on the FY calendar. 16 Data taken from the RISE Cooperative Agreement, Modification #8 17 Data taken from RISE Commodity Status Reports, submitted quarterly to FFP 18 This figure does not include 348 MT which MC reported on as distributed in its CSR and RSR for an emergency response in FY13 Q1, as the 348 MT were actually a loan to CRS and NRC for their emergency response operations. The 348 MT was reimbursed by FFP in June 2013. 19 Cooperative Agreement Modification #8 RISE Mid-Term Evaluation Report 46 Oxu Solutions LLC food ration has been the most important incentive for PLW (and their CU2) to participate in PM2A activities. For those eligible but not participating, it seems that personal choice is the reason some HHs do not register for the PM2A rations; there seem to be few cases of distance from distribution sites being a problem. Where a HC requires a fee for enrolling in PNC, this may be an obstacle for some HHs. For FFW, beneficiaries do 22 workday rotations and the first enlisted are the members of the associations selected to manage the FFW activities and then any person in their neighborhoods who is fit and willing to work. There are many cases of repeat rotations of association members and there seem to be few if any attempts to enlist members from more vulnerable HHs. In addition, post-distribution monitoring (PDM) reports have noted isolated incidents of registration lists not being properly completed and maintained as well as work norms not being systematized in certain areas. Finally, according to the IPTT submitted as part of the FY13 ARR, women comprised 52% of FFW participants during the first two years of RISE (the project target is 50% women participation in FFW activities). There have not been any reported major problems with the importing and receiving of the commodities in Goma. There have been a few issues with transporters delivering commodities to the sites (including minor delays, insecurity along transport routes, and a bribery attempt by a truck driver). To transport food to the sites, MC currently uses two local companies as well as its own 15 MT truck purchased by MC for the project. Given the significant increase in tonnage for Years 3 and 4, MC will need to ensure that there is adequate transport available to deliver the monthly quantities required for distribution. To smooth out the distribution process, there are 22 small community depots where a few tons of CSB can be kept and transferred between depots to make up any shortfalls. Thefts during transport have been minimal, but delivery delays for PM2A recipients of up to two weeks are not uncommon due to bad roads, rain and breakdowns. For FFW, on average, two-thirds of deliveries take place between one and two months after the end of the rotations and one-third between two and three months. There are additionally several cases of alleged non-delivery both in Rutshuru and Bwito due to unclear expectations and protocols. PM2A commodities recipients would like greater quantities to also feed the other children in their HHs, as there is anecdotal evidence that in some HHs the PM2A rations are also used to feed other children and the oil gets used up quickly. FFW recipients are also complaining about the distribution delays and the smallness of the rations, such that men are now supposedly keeping away from FFW. As the agriculture activities started late, FFT distributions have not yet started in earnest, though the model farm workers complain of the same delays as the FFW recipients. Storage of commodities in the sites has reportedly not been an issue. In most areas the communities provide security themselves and food is distributed within 2-4 days of arriving at the final distribution point. In a few places the distribution process takes several days and women must wait to receive their food. We were informed that this issue would be resolved in the future using color-coded vouchers for distinct distribution days. The most serious issues related to commodities that emerged during our time in the field include: 6) repeated rejection by PM2A beneficiaries of a total of approximately 120 MT of CSB for unacceptable taste and smell, between August 2013 and January 2014; and, 7) reports that some PM2A participants did not receive the full oil ration during two distributions as well as discrepancies in the documentation of the amounts of oil individuals received on beneficiary ration cards. RISE Mid-Term Evaluation Report 47 Oxu Solutions LLC Details related to each of these issues are explained in Annex N: RISE Commodity Issues. Because of the seeming seriousness of both of these issues, we recommend a more in-depth investigation in the form of a project commodity audit, which could be conducted jointly by commodity management staff from both Mercy Corps and from CRS to capitalize on the experiences of both organizations, or by an external consultant. In terms of accountability related to commodity distribution and use, MC and partners are using a complaints hotline that beneficiaries can access on distribution days. This distribution day hotline is the extent of the project’s accountability system. For the rejected CSB the Société Civile group has had to help intervene on beneficiaries’ behalf. Both the CSB and vegetable oil issues highlight the need for a more robust and comprehensive project accountability system In addition, post-distribution/ end use monitoring has been conducted but not in any sort of systematic way. Staff interviews seemed to indicate that PDM was being done by SO rather than as part of the project M&E system. We were told, however, that as of December 2013, new PDM forms have been put in place and field-level monitors are now supposed to be monitoring up to 40 households/ month / field monitor using the PDM form. This information is then supposed to be entered into an M&E database. For the MTE, we were given several reports from PDM visits, as well as the new PDM forms for FFW/FFT and PM2A. PDM needs to be systematized in terms of the numbers of FFW/FFT and PM2A households to visit each month/quarter in a given geographic location, the sampling strategy for the household surveys, and data entry, cleaning, and analysis. The most recent DRC Bellmon Estimation (Sept 2010) recommends on-going monitoring of local market conditions (prices for key commodities, volumes traded, and numbers of traders), especially given the risk of negative local market impact from PM2A distributions given that local markets are so poorly integrated in DRC (i.e., distribution impacts would be felt at a highly localized level as small, poorly integrated markets can be more easily saturated and disrupted).20 The project does not currently track food prices in local markets (or other local market characteristics), though from our interviews and field visits there is no evidence that commodity distribution has disincentivized local production, partly because the commodities do not compete with local products and its scale seems relatively small. There have been some reports of beneficiaries selling at least some of the food they received (yet without a full PDM system in place it is impossible to know how prevalent this issue might actually be). In the remaining years of the project, however, it will be important to put in place at the very least a basic market monitoring system at the local market level to track major market changes over time. Local market monitoring would also be useful for RISE production and marketing activities (SO1); supporting access to dietary diversity through understanding affordability of nutritious local foods on the market (SO2); and community￾level EWS (SO3). Market monitoring should be standard best practice for any MYAP, and, as such, at the global level MC or CRS should have standard market monitoring tools that can be used by the DRC offices. Finally, there are reports that the use of CSB is encouraging HHs to grow and eat more corn and creating a demand for corn flour as well as a desire for beneficiaries to learn to make a local version of CSB using corn, sorghum and soy flours. 20 USAID Office of Food For Peace, DR Congo, Bellmon Estimation. September 2010. See p. 9 and p. 112-116. RISE Mid-Term Evaluation Report 48 Oxu Solutions LLC Commodities Recommendations: 1) Conduct a project commodity audit, specifically to investigate more fully the CSB and vegetable oil issues noted above and detailed in Annex N on RISE Commodity Issues. The commodity audit could be conducted by Mercy Corps headquarters commodity management staff or an external party and should give highest priority to the following: a) CSB storage b) Systems for ensuring commodity quality and safety prior to distribution c) Systems of accountability for distributing accurate quantities d) Social cost to relationship with project beneficiaries related to the CSB and Veg Oil issues 2) Put in place a comprehensive accountability system for the project, which would include: a) Full complaints system b) PDM system c) Market monitoring system 3) Establish a detailed commodity distribution plan (by month for each location) for the remainder of the project. RISE Mid-Term Evaluation Report 49 Oxu Solutions LLC V. Evaluation Themes and Conclusions As mentioned in Section II, MTE Scope, the evaluation team has used five main themes as lenses of analysis for the RISE MTE. Based on these themes, we have drawn preliminary conclusions and used a color-coded system of assigning a green (seems on track to achieve this by project end), yellow (some possibility of achievement by project end, but serious weaknesses must be overcome), or red (highly unlikely to be achieved by project end without fairly dramatic and immediate change) color to each theme, presented in the table below. Table 6: Evaluation Theme Conclusions Theme and Conclusions Color Relevance, Appropriateness, Effectiveness: To date, activities have been scattered and not well￾coordinated or integrated across components. There is no strategic targeting of households (HHs), which would maximize impact if done well. Project design was overly ambitious (for example, many planned activities in SO1 have not even begun), and much of the project looks more like an emergency intervention rather than a longer-term development program. Adaptability and Security: The project has shown adaptability to insecurity but we found little evidence of contingency planning, either internally or as part of discussions with communities around project activities. Neither did we find much documentation of how decisions were made related to insecurity. Ongoing and proposed geographic expansion puts the project at risk of being overstretched geographically, with potential negative consequences for quality implementation. Project Targets, Early Outcomes, and Goals: From a numbers perspective, the project has been impressive in its ability to meet many of the beneficiary targets, especially in terms of total households and percentage of female beneficiaries. Yet the quality with which activities have been implemented remains a strong concern, especially the contribution to outcomes and the project objectives. Community Engagement and Accountability: Relationships with communities do not seem like partnerships and project accountability has been extremely limited. Sustainability and Spillover: Other than VSLAs, the local dissemination of new cassava and bean varieties, and some of the health and nutrition-related behaviors and practices being promoted under SO2, there is little indication of potential for sustainability for most of the results. Spillover effects seem incidental rather than planned. There are no well-defined sustainability and exit strategies in place. This type of analysis allows MC and its partners to take a broad view of the project and see where adjustment in the remaining years of the project should be focused. In addition, the recommendations provided in the sections above have been developed to help address the project weaknesses identified in Table 6. In conclusion, our field work revealed a dedicated project staff logging long hours and working diligently in an incredibly difficult and risky operating environment to try to improve food security for vulnerable populations in North Kivu. In addition, we found a certain number of project successes, especially in SO2 and in reaching women beneficiaries. At the same time, the project in general suffers from trying to do too much in too many places without adequate staffing or the systems in place to track progress and learn from experience. From the outset, the project design was too ambitious. While this report presents a lengthy list of recommendations, it is critical that in the remaining time for RISE implementation, MC and partners focus on changes that will: improve systems (M&E, accountability, documentation, etc.); ensure sustainability/appropriate exit; scale back expectations and activity levels; and maximize beneficiary impact.