Food for the Hungry DRC Development Food Assistance Program Mid￾Term Evaluation Report January 2014 i FHDRC Development Food Aid Program Mid-term Evaluation Report This publication was produced for review by the United States Agency for International Development. It was prepared by Lucas Shindeldecker, Mitzi Hanold, Tesfaye Tezera, and Sarah Bauler from Food for the Hungry. This document was submitted by Food for the Hungry to the United States Agency for International Development under: USAID Cooperative Agreement No. AID-FFP-11-00007 DISCLAIMER The authors’ views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States government. ii FHDRC Development Food Aid Program Mid-term Evaluation Report Table of Contents LIST OF ACRONYMS ....................................................................................................................... iii EXECUTIVE SUMMARY ................................................................................................................... 4 Summary of Findings and Recommendations ................................................................................... 4 INTRODUCTION ................................................................................................................................. 5 Program Overview ............................................................................................................................. 5 EVALUATION DESIGN AND METHODLOGY ............................................................................... 6 Research Methodology ...................................................................................................................... 6 Mid-Term Evaluation Team Composition ......................................................................................... 7 Evaluation Limitations ....................................................................................................................... 7 SUMMARY OF PROGRAM ................................................................................................................ 8 FINDINGS AND CONCLUSIONS ...................................................................................................... 9 Design and Implementation of the Program ...................................................................................... 9 Program Partnerships ....................................................................................................................... 13 Communities’ Contributions to Design, Implementation, and Monitoring ..................................... 14 Efficiency of Reaching Intended Beneficiaries ............................................................................... 15 Value of Food Aid and Activities .................................................................................................... 16 Sustained Outcomes and Impact ...................................................................................................... 17 Monitoring and Evaluation .............................................................................................................. 19 Technical Sectors ............................................................................................................................. 20 Effective behavior change communications .................................................................................... 23 Different perspectives and needs of men and women and gender equity in households and communities ..................................................................................................................................... 24 Changes in the community, households, and individuals ................................................................ 25 RECOMMENDATIONS ..................................................................................................................... 27 Annex 1 – Scope of Work .................................................................................................................... 29 Annex 2 – Individuals and Groups Contacted ..................................................................................... 36 Annex 3 – Individual and Group Discussion Guide ............................................................................ 37 iii FHDRC Development Food Aid Program Mid-term Evaluation Report LIST OF ACRONYMS BA Barrier Analysis CC Cross-cutting CDC Community Development Committee CG Care Groups CTS Commodity Tracking System DRC Democratic Republic of Congo FFP Food for Peace FFW Food for Work FH Food for the Hungry FL Farmer Leader GD Group Discussion GM/P Growth Monitoring Promotion Ha Hectare HAZ Height for Age Z-score HDDS Household Dietary Diversity Score HHS Household Hungry Scale HH Household HIV Human Immunodeficiency Virus H/N Health and Nutrition IITA International Institute for Tropical Agriculture IPAPEL Provincial Inspectorate of Agriculture, Fisheries and Livestock IPTT Indicator Performance Tracking Table ITN Insecticide Treated Net IYCF Infant and Young Child Feeding KPC Knowledge, Practice, Coverage LQAS Lot Quality Assurance Sampling ML Mother Leader MoH Ministry of Health MTE Mid-Term Evaluation MYAP Multi-Year Assistance Program NGO Non-Government Organization NRM Natural Resource Management SBC Social and Behavior Change SO Strategic Objective SoW Scope of Work USAID United States Agency for International Development WASH Water and Sanitation, Hygiene WAT/SAN Water/Sanitation 4 FH/DRC Development Food Aid Program Mid-term Evaluation Report EXECUTIVE SUMMARY Food for the Hungry DRC has been implementing a five year Development Food Assistance Program (DFAP) since August 1, 2011. The program targets 63,110 vulnerable households in the territories of Kalémie and Moba in Katanga Province and in the territory of Walungu in South Kivu Province. In an integrated approach, the program covers multiple sectors including Agriculture/NRM, Health & Nutrition, WATSAN and access to credit. In addition, developing community capacity and gender mainstreaming were taken up as cross cutting themes. In November 2013, at the mid-point of its five-year DFAP, FH conducted its Mid-Term Evaluation, primarily to examine the strengths and weaknesses of the various strategies and approaches and suggest recommendations for necessary changes to ensure effective program implementation in the second half of the program. Summary of Findings and Recommendations Key findings and recommendations are summarized as follows: General findings are: 1) The DFAP is building on gains from FH’s previous MYAP, and is better tailored to beneficiaries’ needs than the MYAP. 2) From the evidence of the early signs of positive changes in behaviors, the cascade training models (care groups and farmer leader groups) proved to be effective behavioral change communication vehicles. 3) VSLAs and gender component of the program were among the most successful interventions in terms of adoption and influence of social behaviors and relations. 4) Some significant implementation delays were observed especially in value chain development, drainage & irrigation and rolling out some agricultural and health modules. Assessment team recommendations are: 1) Capitalize on existing strengths within the program in order to enhance results and within the current scope of the program- • Utilize CDC structures and integrated nature of the program to further emphasize sustainability and community ownership from now to continue after the end of the program. • Expand and bolster VSLA approach to solidify household savings and economic gains and build community solidarity and resilience to shocks. • Build upon strengths of gender sensitization approach to expand reach and impact relationships and decision making at the household and community levels, and capitalize on gains made in female staff hiring to increase and empower more women project staff. 2) Address weaknesses within the project to better ensure program effectiveness and position the program for greater sustainability- • Given the volatile nature of the program implementation areas, ensure focus is given on development of emergency preparedness resources and mitigation plans in conjunction with recommendation above to improve sustainability and resiliency. • Continue improving response times to procurement and logistical difficulties to more quickly get BCC materials to fields to use to improve beneficiary learning and utilization. 5 FH/DRC Development Food Aid Program Mid-term Evaluation Report INTRODUCTION Per its agreement with USAID, FH conducted this Mid-Term Evaluation (MTE) to assess its current performance and to make programmatic and process recommendations for improving performance in the remaining years of the program. Specifically, the evaluation addresses general implementation, implementation of technical components, cross-cutting issues, and programmatic outcomes. A three person team from Food for the Hungry’s Washington, DC office conducted the MTE field work over a two-week period from November 1 and November 14, 2013. The evaluation relied primarily on collecting qualitative data through group discussions (GDs), observations, and semi-structured individual interviews with beneficiaries, partners, and program staff. In addition, relevant documents including the project proposal, ARRs, PREPs and annual LQAS results were reviewed. FH’s Regional Food Security Coordinator provided ongoing technical support and oversight to the design, field work, and analysis of this evaluation. Program Overview FH DRC’s Development Food Assistance Program (DFAP) is funded by USAID/FFP for the period of August 1, 2011 to July 31, 2016. The total budget is $47,880,500 and total beneficiaries over the life of the award are 668,953 people. The program’s goal is to ‘Reduce Food Insecurity of Vulnerable Households in South Kivu and Katanga Provinces’ and the SOs and IRs are as follows: SO 1: Improved Livelihood Capacities of Vulnerable Households IR 1.1: Increased Agricultural Production IR 1.2: Improved Land Management and Natural Resource Conservation IR 1.3: Increased Household Income from Agricultural Production SO 2: Improved Health and Nutrition of Individuals within Vulnerable Households IR 2.1: Improved Use of Essential Nutrition Behaviors IR 2.2: Increased Diversity and Consumption of Nutritious Food IR 2.3: Improved Household and Community Management of Conditions and Diseases that Exacerbate Malnutrition IR 2.4: Improved Use of Clean Water, Sanitation Facilities, and Hygiene Behaviors CC 1: Improved Gender Equity in Decision Making and Labor Sharing CC 2: Increased Access to Credit CC 3: Improved Community Capacity and Resilience to Shocks The program is in its third year of implementation. The program covers agriculture, health and nutrition, water and sanitation, gender, and community capacity interventions. This program works through a variety of interventions, including physical infrastructure, biological assets, technical trainings and capacity building, and cascade groups and other mass media for message dissemination. Figure 1, below, illustrates the territories where FH is implementing this project. 6 FH/DRC Development Food Aid Program Mid-term Evaluation Report EVALUATION DESIGN AND METHODLOGY Research Methodology The mid-term evaluation employed a participatory qualitative study methodology to assess the strength and weaknesses of the various program approaches and strategies as well as implementation processes. It was an iterative process of data collection from multiple sources and analysis throughout the evaluation period. Participatory data collection tools were used in the form of household interviews, key informant interviews (especially with partners), individual interviews, and group discussions. Observation and document reviews were also conducted during the field work. Routine monitoring reports were also reviewed, especially those related to the collection of quantitative data. Detailed checklists of open-ended questionnaires were also prepared in advance of the evaluation, which were reviewed and commented by FH, and Food for Peace. The following specific objectives helped guide the design, implementation and analysis of the mid-term evaluation: • To evaluate the strengths and weaknesses of program implementation and the quality of outputs so far, in terms of adherence to terms agreed by FFP and their acceptability and perceived value to target groups, identifying factors that appear to enhance and detract from the quality, acceptability and usefulness of implementation and outputs. • To evaluate the strengths and weaknesses of the program’s strategy to promote gender equity in targeted communities. • To seek evidence of changes (intended and unintended) associated with program activities and identify factors that appear to promote or hinder the program’s progress toward desired objectives. 7 FH/DRC Development Food Aid Program Mid-term Evaluation Report • To evaluate program strategies that seek to assure that positive results can be sustained after the program ends, identifying the strengths and weaknesses of these strategies. • Based on the findings, to recommend adjustments to program implementation or design and explain how these changes would improve program outcomes and sustained impact. Qualitative inquiries into the implementation of the various program activities were conducted at multiple sites per activity type, including sites from each of the three geographic areas of operation—Moba, Kalemie and Mubumbano. In order to explore the gender￾differentiated perceptions and opinions about the program and its benefits, part of the GDs were organized with a mix of men and women participants while others were separate men and women group discussions. The core evaluation team tried to allow as much time as possible to ensure the discussions were participatory and that women were able to voice their feelings and perceptions on the programs progress. Apart from the direct beneficiaries, the team also interviewed non-beneficiary households to get a broader view of the program, especially perceptions about the targeting processes. In depth interviews were also conducted with project managers and staff of FH and its partners as well as the key informant interviews with government staff, partners, and selected community members. Finally, physical observations of selected activities were also conducted in accessible sites, which followed discussions with men and women beneficiaries and program staff. In order to establish the context and review progress, various progress and field visit reports, ARRs, PREPs, and the proposal were also thoroughly reviewed in helping to inform the MTE. In total, the MTE Core Evaluation Team conducted 15 group discussions and 37 interviews. Please see Annex 2 for a list of these discussions and interviews. Mid-Term Evaluation Team Composition The core evaluation team was composed of staff from FHUS and the FH Africa Regional office, with the assistance of FH DRC and partners who were responsible for logistics and arranging meetings. In consideration of the technical components of the program, the MTE team included staff with experience and expertise in food security/livelihoods, MCN, agriculture systems, behavior change, gender, environment, and commodity management. The MTE team included FH’s Senior Food Security Program Officer, Health and Nutrition Officer and Senior Coordinator of Education Programs from the headquarters office in Washington DC. The Food Security Coordinator from the Africa Regional Office provided technical support and coordination. The core team members of the evaluation have extensive experience in monitoring and evaluation, food security programming, health and nutrition, food aid, and data collection and analysis. Evaluation Limitations The operating environment in Eastern DRC and time available for conducting the evaluation affected the type, quantity, and quality of data available for this evaluation. These include: • Logistical difficulties- Traveling to South Kivu and Katanga Provinces is always logistically challenging—even in months that are usually not considered the rainy season. As Mubumbano in South Kivu is still considered a high security risk, security protocols recommended by the UN and FH require visitors and staff not based there to leave Bukavu no earlier than 8am and to return to Bukavu no later than 5pm. This curfew limited the number of hours spent in Mubumbano considerably, as it is a two 8 FH/DRC Development Food Aid Program Mid-term Evaluation Report hour drive from Bukavu. Moba and Kalemie in Katanga Province do not share the same security risks as Mubumbano, but geographically they are much more difficult to access. Flights are limited and not always available and poor road conditions makes traveling a short distance a laborious and time-consuming endeavor. • Interpretation reduces data fidelity- The MTE Core Evaluation Team had to conduct interviews with beneficiaries in their local languages. In Mubumbano, for example, the local language is Meshi. Interviews were given in English, transmitted in Meshi and back-translated to English. In Kalemie and Moba, beneficiary interviews and discussions were conducted in French, Swahili and local languages. Interpretation impedes accurate communication and there is always the danger of the interpreter translating the information incorrectly. Although the MTE Core Evaluation Team probed continuously to clarify issues and improve understanding, it can be assumed that there was loss of data fidelity. SUMMARY OF PROGRAM In its proposal, FH planned to reduce food security at the household level through a series of activities related to increasing livelihood capacities of households (SO1) and improving the health and nutrition status of individuals within those households. A description of these activities follows. Capitalizing on significant infrastructure investment, including roads, bridges, and markets, in SO1 FH intended to improve agricultural production (IR 1.1) in sustainable ways (IR 1.2) to increase smallholder farmer incomes by reestablishing and enabling producers’ vertical integration into profitable value chains (IR 1.3). Based on learning from its previous MYAP, FH adopted a new strategy in the current program to scale up and reach wider farming population with the improved agricultural techniques through an expanded agricultural cascade extension (ACE) training model. Further, a new strategy of field-based seed testing and multiplication stations is being used to increase access to improved, diverse and disease￾resistant seeds and cuttings. Using FFW resources to rehabilitate large-scale irrigation canal systems, FH intended to revitalize over 12,000 ha of fertile land and enable these zones to reclaim their ‘breadbasket’ status though significant increases in agricultural production. FH intended also to ramp up its agro-forestry and reforestation activities to replant over 1,000 ha of denuded hillsides using FFW. A partnership with Pharmakina, a local agro-business company in S. Kivu, would integrate community reforestation with a well-established and profitable quinquina bark production value chain. Participative market analysis and business skill development would enable smallholders in all sectors to successfully sell their agricultural produce to both local and regional markets. Under the Health and Nutrition (H/N) SO 2, FH would expand its CG model, which builds the capacity and social standing of women in their communities to regularly reach over 50,000 mothers of children aged under two (IR 2.1). Men would also be targeted with H/N messages through a specific focus on influencing community leaders and creative radio programming aimed at both men and women. In an effort to improve dietary diversity and nutrition, FH would complement related H/N messages through the CGs with promotion of HH vegetable gardens, fruit trees and small livestock animal production for all households with pregnant women or children aged under two (IR 2.2). In response to potable water 9 FH/DRC Development Food Aid Program Mid-term Evaluation Report coverage rates of 9% in Katanga and 23% in S. Kivu, using FFW, FH would construct 330 water points. One of the lessons learned from the previous MYAP was that livelihood improvements are not enough to improve wellbeing of communities where there is widespread gender inequity and gender based violence. In a strategic shift of programmatic focus in the current program FH has been promoting a constructive, participative and culturally appropriate dialogue around issues of gender inequity through partnership with Search for Common Ground (SFCG). Using a diverse range of tools including participatory theatre, and radio production, FH/SFCG work to raise knowledge, shift behavior, and improve the status of women in the targeted communities (CC 1). The link between gender issues and achieving improved availability, access, and utilization of food has been incorporated into each SO and IR in the proposed program framework. A redesigned and expanded agricultural extension model is being used to ensure equal gender representation and inclusion at all levels under SO1. FH aims to improve access to credit to over 12,000 HHs through the establishment of VSLAs and connecting agricultural producers to appropriate credit mechanisms (CC 2). As the key point of entry into communities, FH builds the capacity of CDCs to monitor, coordinate, and create local synergies between different community development actions (CC 3). Over the past 10-15 years, FH staff have served eastern DRC communities through periods of intense conflict, massive displacement and subsequent return. Working alongside families to rebuild homes, livelihoods and infrastructure, FH has gained their trust and respect. Building on these years of physical and social investment, FH expects that the creative behavior and social norm changing, and innovative market-oriented programming described above will result in dramatic improvements in all aspects of food availability, access, and utilization for the targeted food insecure households in S. Kivu and Katanga. FINDINGS AND CONCLUSIONS This section includes findings and conclusions for each of the main evaluation questions (found within the MTE SoW in Annex 1). While the findings presented for each question provide evidence for conclusion for that question, conclusions were at times drawn upon findings from other questions and from quantitative program data. The findings are presented according to the main evaluation questions as opposed to the discrete technical sectors. The evaluators saw that the integrated nature of the program did not lend itself to a presentation by technical sector and instead chose to present the findings in a way they felt best reflected the answers and impressions received from the interviews and discussions held. Information on each sector is addressed throughout the main evaluation questions, and sector-specific findings are also included. Design and Implementation of the Program Strengths in Design and Implementation Overall, the programmatic methodologies employed in this program have been effective in reaching most of the Year 3 targets for performance indicators as seen in the FY13 ARR IPTT. Using cascading schemes to reach rural farmers in Farmer Groups and rural mothers in Care Groups continues to be an effective strategy in achieving the desired behavior change. This can also be attributed to FH’s behavior change teaching materials, which include formative research and have been revised from the MYAP program based on field experience. The agriculture and livelihoods curricula are relatively new for the Farmer 10 FH/DRC Development Food Aid Program Mid-term Evaluation Report Leaders groups, and it was clear during interviews with staff and beneficiaries that these modules were well utilized and helped to reinforce learning. For example, beneficiaries and staff cited that the model allows for better coverage and is a “very participative” because it encourages group discussion and mutual learning. The use of the cascade model instead of Farmer Field Schools significantly expanded the scope of the agriculture program as seen in the number of beneficiaries, higher frequency of lessons, and the rate of adoption of new practices such as improved storage techniques. The promotion and support of Village Savings and Loans Associations (VSLAs) has also been extremely successful. This finding cannot be overstated. The VSLAs were mentioned in every GD as one of the most important activities in improving access to essential services, such as health and education, but also as a mechanism to build equity and greater livelihood opportunities. Beneficiaries repeatedly cited that savings were used to pay for school fees, medical care, and food in times of shortage. In numerous cases, beneficiaries also expressed that they had started small income generation activities and opened new fields or expanded their existing fields. The VSLAs are growing organically at unanticipated rates. For instance, VSLA groups’ formation for each of the past two fiscal years has been about five times higher than planned. FH staff in Moba have even started two VSLAs amongst themselves after observing how effective they have been in the communities they serve. VSLA members have been approached by their neighbors to help form new groups in their community. Beneficiaries stated that the VSLAs have built greater solidarity in their communities as well, which is evidence of increased social capital. One benefit of this activity has been equal participation of men and women. This was cited numerous times during interviews and FG as a strength of this approach. Another strength in the program design and implementation has been the increased focus on gender mainstreaming. Food for the Hungry’s previous Multi-Year Assistance Program (MYAP) did not directly address gender issues, but this DFAP was designed to target these issues strategically in partnership with Search for Common Ground (SFCG). This change in design has also included an FH-developed gender sensitization training of all their staff, which has led to changes in staff hiring and increased acceptance of women in leadership. FH has recently promoted four female staff to the level of supervisor. Although the percentage of female staff compared to male staff is still low, more women have been given opportunities to grow professionally in a context that has historically discriminated against women in positions of professional responsibility. FH has hired women for a Program Manager position, as well as for senior finance, logistics, and IT positions in field sites and at the head office. For example within two years, FH has increased the percentage of female staff from 10% to 16%. The increased focus on gender is also changing the lives of female beneficiaries. According to interviews amongst beneficiaries, more women have been elected to serve as leaders in their communities, and the Community Development Committees (CDCs) all have female (almost equal ratio of men and women) participating in these committees. Male beneficiaries noted throughout the interviews and GDs that they have increased respect for their wives, and some even openly shared that they no longer beat their wives and are more willing to participate in daily household and child care activities. Compared to the MYAP, the DFAP has less emphasis on food distributions and greater focus on building the skills and capacities of the targeted beneficiaries for sustainability. The DFAP has fostered community ownership of the activities and strategies that have been promoted by FH, and there was a general respect and appreciation by the communities for the opportunity to learn new behaviors and techniques. For examples, CDCs have expressed that they have 11 FH/DRC Development Food Aid Program Mid-term Evaluation Report been actively involved in forming, training, and monitoring activities of groups within their intervention zones. They recognize their role as a bridge between the communities and other actors (NGOs, government entities). Also, Mother Leaders and Farmer Leaders have embodied the cascade model and understand that their role in teaching is effective in fostering positive changes in behavior. The approach of the DFAP is more developmental than the more emergency-transition mindset of the MYAP, leading to greater ownership of their own development and future by the community. The use of formative research is another strength in the design and implementation of the program. Food for the Hungry program staff have been trained in Barrier Analysis (BA) and health program staff have participated in the Local Determinants of Malnutrition (LDM) study. When asked, staff could accurately describe the purpose of formative research and results, which were integrated into the training materials. Using BA in the formulation of the agricultural curricula has enabled the ACE model to be more flexible and pertinent to beneficiaries’ needs. A recent FH reorganization and restructuring of staff and target area supervision has ensured that supervisors, promoters and senior level staff1 live in the communities they serve which has greatly reduced travel costs and also improved overall management of field site operations and implementation of program activities. The process involved a restructuring of the management of the target areas, moving to a zonal approach from one based on axes from town centers. There has also been an increased effort to hire staff locally, especially in hiring promoters for Care Groups and Farmer Groups. The hope behind this change is that the promoters will continue to live where they have been working, which will help ensure the sustainability of activities and retention of knowledge within the communities. Another strength of the program has been to allow all formed groups, including Farmer and Care Groups, to select their own leaders. Based on feedback from the communities, this appears to have led to greater acceptance of these leaders in the communities, which has helped them carry out their activities more effectively. One mother leader expressed “the community supports me because they have seen a change in their lives.” There has also been a greater integration of sectors in the DFAP than the previous MYAP. The Assistant Health Coordinators, Assistant Agriculture Coordinators and other site-level staff develop their work plans together, and staff are trained cross-sectorally. For example, when interviewing beneficiaries, the MTE Core Evaluation Team found that agriculture beneficiaries knew health and nutrition information, could explain the VSLAs, the work of the Community Development Committees and also the WASH committee work, which is evidence of sectoral integration and coverage. This also demonstrates the power of the social networks within the community – that even when only group of men (or women) receive a message, the information travels throughout the family and to others in the household and community, increasing the capacity throughout. Finally, there are no other NGOs working in Moba at similar scales or in the same sites other than FH and only a few other NGOs working in Kalemie and Mubumbano. This has made it easy to avoid duplication of activities from other NGOs, as the DFAP targeted communities are some of the poorest and least accessible communities in the DRC. 1 At each site in Moba, Kalemie and Mubumbano, there is a Site Coordinator, an Assistant Heath Site Coordinator, and an Assistant Agriculture Site Coordinator. 12 FH/DRC Development Food Aid Program Mid-term Evaluation Report Weaknesses in Design and Implementation One notable weakness in the design of the program is the drainage and irrigation activity original plan did not align with the amount and scale of implementation that a later Environmental Assessment revealed, leading to higher levels of volume of work and budget required than initially estimated. As per the plan, it was supposed to be implemented using community labor through FFW commodities. Nevertheless, the recently done technical design study revealed that successful completion of the activity requires a significantly higher budget in terms of cash and commodities that calls for a major revision of the program or sourcing additional funding. Although not a weakness in the design of the program, a challenge in the implementation of the program has been the ongoing conflict in Mubumbano (South Kivu). This conflict has posted considerable delays in reaching program activity time frames and schedules. From November 2012 to February 2013, all FH staff in Mubumbano were evacuated to Bukavu due to heightened rebel activity in the catchment area. FH should improve its contingency planning to ensure minimal program delay and impact should emergency or conflict occur. Moreover, Katanga Province is often the recipient of emergency food aid and non-food items. This has created an expectation by beneficiaries of handouts from FH, and at times beneficiaries have refused to participate in activities if no incentives or emergency inputs were provided. Flipcharts for cascade group messages have been designed, but printing has been delayed and often times incorrectly printed by the printing company. Not all of the flipcharts have reached the project sites as a result. 2 The distribution of seeds and seedlings has also been delayed, which indicates a weakness within FH’s logistics systems and on the side of implementing partners. There was a misperception noted in the group discussions about numbers of goats distributed. Beneficiaries did not understand the methodology used and consequently thought that there were insufficient goats when the planned amount were distributed. This indicates a weakness in FH’s communication about the goats to the beneficiaries. FH’s partnerships are weak in appropriate marking and branding. This is also a weakness that the MTE Core Evaluation Team observed during their field work. Although flipcharts and written materials are properly branded, a few communities and regions where FH is working lack appropriate signage. Of the signs that did exist, many were from the previous MYAP. Though adequate numbers of staff were placed to run the program, staff quality issues were cited as a weakness, especially at the start of the program. According to staff discussions, some of the initial recruitment of a small number of lower-level staff was inconsistent with FH policy and marked by personal favoritism. There were instances where staff were hired with sub-par skills and experience that didn’t fulfill minimum requirements; these staff were replaced as quickly as possible. FH’s leadership has taken corrective measures to rectify the malpractices including termination of contracts of implicated staff. A restructuring has been made through a thorough review of the job descriptions of all positions followed by re-hiring of staff with strict vetting process. FH collaborates with all other international organizations in the region, sharing information about former staff to jointly combat this type of corruption, which is common. 2 For example, in Mubumbano 115 out of 4069 flipcharts are still needed for Module 3 for the Mother Leaders and 101 out of 3731 Flipcharts are still needed for Module 1 for the Farmer Leaders. 13 FH/DRC Development Food Aid Program Mid-term Evaluation Report Program Partnerships Before the strengths and weaknesses are described, it is important to first provide a short summary of various partners and their roles in providing technical and implementation support for this DFAP: • International Institute of Tropical Agriculture – IITA (formally called CIALCA) is working with FH to provide monitoring and control of plant diseases, including BXW, BBTB and CMV. They have also helped FH in selecting disease-resistant seeds and have been contracted by FH to provide technical assistance in evaluating value chains for greater agro-development opportunities among DFAP farmers and beneficiaries. • INERA – INERA is a government-funded partner that has been providing training and technical support to FH since 2008 in small livestock multiplication and management activities. They work “hand in hand”3 with FH in also the multiplication of seeds and management of plant disease and research to improve agricultural productivity. • IPAPEL – IPAPEL, the government’s agricultural ministry, provides trainings and technical support to FH in improved animal husbandry practices and ongoing veterinary support. They are also a key partner in FH’s seed and cassava multiplication activities. • Pharmakina –Pharmakina conducted viability assessments of DFAP catchment area soil in order to grow quinquina, which is a plant used to make quinine for anti￾malarials. Pharmakina provides technical assistance to quinquina growers and is committed to purchasing additional quinquina produced. • Search For Common Ground – SFCG provides technical support and resources in the development and facilitation of constructive, participative and culturally appropriate dialogue around issues of gender inequity. To achieve this, they use tools such as participatory theater, radio and television production in order to improve community and household relations and women’s empowerment in DFAP targeted communities. • Ministry of Health –The MOH, although limited in resources, works with FH in conducting immunization and deworming campaigns. The MOH also provides nutritional screening of children less than five years of age and pregnant women. FH coordinates its H/N activities in conjunction with the MOH. Strengths of the Program’s Partnerships One of the greatest strengths of FH’s programming in Eastern DRC is their close collaboration and coordination with partners and local government institutions. The DFAP activities and strategies are well aligned with the objectives of the national, provincial governments. Below are some of the findings from the field work that are evidence of this strength: • Government staff (including IPAPEL and the MOH) were quick to say that they work “hand in hand” with FH and that FH activities and strategies are well integrated with government food security and health and nutrition strategies and policies.4 The 3 Direct Quote, Director, INERA, Mulungu, November 7, 2013, Individual Interview. 4 Nutritionist and Chief of Health Zone, Ministry of Health, Wulungu Zone, November 11, 2013, Group Discussion with MOH staff and IPAPEL, Wulungu Zone, November 13, 2013, Individual Interview 14 FH/DRC Development Food Aid Program Mid-term Evaluation Report evaluation team heard positive remarks by government institutions about FH’s work and respect for the expertise of FH staff and DFAP strategies and activities. • Requests were made by the MOH in both Mubumbano5 and Kalemie for FH to expand their health activities, especially the health and nutrition work to other territories within the provinces that are beyond the DFAP catchment area. According to IPAPEL from Katanga Province, malnutrition is decreasing in the areas where FH is working, but in regions in the province where FH is not working (namely Kongolo, Kabalo, Manono, Nyuuzu), malnutrition is reportedly increasing. • IPAPEL in Katanga requested for FH to expand their agriculture activities throughout Katanga Province. • Search for Common Ground staff sits at FH office in Mubumbano, Kalemie and Moba, enabling both organizations to plan and coordinate all activities together. • Agricultural activities are jointly planned with IPAPEL, Pharmakina, INERA, and IITA. • Partners (Pharmakina, INERA, SFCG, and IITA) and government institutions (IPAPEL and MOH) demonstrated deep knowledge of FH’s scope of work and sector activities and strategies as demonstrated in the interviews and discussions. These evidence the level of partnership and joint ownership and planning that FH endeavors to achieve in the program. Weaknesses of the Program’s Partnerships Challenges and weaknesses in FH’s partnerships are as follows: • Although generally satisfied, the MOH and IPAPEL government employees believed that FH could do a better job building the government’s physical capacity by providing resources, such as four-wheel drive vehicles. Investing in these kinds of resources, however, is beyond the scope of this project. • One evident risk is that while FH strives to work with and to build capacity of its partners, when one partner fails to deliver, it puts other program activities at risk, as seen in the delay of the value-chain studies and lower-than-expected output of seeds. • The Core Evaluation Team also identified the government’s low financial and human capacity to be a general challenge of the DFAP operating milieu. As stated by the MOH, the government only has the capacity to treat 10% of malnutrition cases. This is problematic as part of FH’s health and nutrition strategy is to identify children with SAM (severe acute malnutrition) and MAM (moderate acute malnutrition) and refer them to health clinics for therapeutic care. Communities’ Contributions to Design, Implementation, and Monitoring A number of interviews and group discussions pointed towards the CDCs as the best example of community involvement in the design, implementation and monitoring of the program. The CDCs collaborate with and engage in the program in various ways. CDCs participate in the planning of each activity within the community. For example, they exercise micro￾targeting of water scheme construction sites, identify beneficiaries, mobilize community participation, monitor activities, etc. Evidence confirmed to the evaluation team that FH’s strategic emphasis to work through CDCs has been successful. CDCs in all communities in which the evaluation was conducted were lauded as important entities which support, monitor, and strengthen the other groups in their community including CG, FLG, water committees, land use and management committees, and VSLAs. Through group discussions 5 Ministry of Health, Wulungu Zone, November 11, 2013, Group Discussion with MOH staff 15 FH/DRC Development Food Aid Program Mid-term Evaluation Report and personal interviews, community members highlighted the role of CDCs in bringing people together to discuss and find solutions to community problems. Since CDCs cover 3-4 villages, people gather from multiple communities for CDC-led meetings. These gatherings allow people from different communities meet and get to know each other. This greater opportunity to dialogue has improved social cohesion and strengthened social networks, according to community members. One CDC member highlighted that “Good governance has given me compassion and love [for my community].” Further, the program strategies, especially the Care Groups and the Agricultural Cascade Extension models were lauded by staff, government officials, and community leaders as a success in terms of encouraging target community members to join in the implementation and monitoring of activities. As opposed to the traditional methods of training that try to reach each beneficiary directly, these strategies followed a cascading approach where communities teach each other in a structured modular training. This model allowed community members to play part in the actual implementation and monitoring of the program. It also allows for a far greater scope than the previous MYAP approach where the ACE program component was not implemented. Farmer Leaders visit farmer beneficiary fields to monitor whether certain recommended techniques and behaviors were practiced on their personal fields. Likewise, Mother Leaders make home visits to Mother Beneficiary households to monitor whether promoted health behaviors are being practiced. These monitoring visits allow ML a platform to encourage the proper practice of health and nutrition behaviors highlighted during trainings using flipcharts. One challenge in community participation has been that beneficiaries in some (not all) group discussions requested more resources from FH, such as refreshments and materials for meetings, that were beyond the scope of this project and contrary to the mindset that FH is trying to instill. Some communities assessed by the MTE Core Evaluation Team seemed to understand much more clearly FH’s role as a partner than other communities. It seemed apparent that not all communities were “brought up in the same way,”6 and that some beneficiaries had received more training in the importance of taking ownership of FH￾supported activities and their critical role in the sustainability of these activities. This more emergency-oriented mindset is prevalent in various parts of the target areas. Efficiency of Reaching Intended Beneficiaries Of staff and community members interviewed, all were satisfied with the targeting of the program in their community. Much of this is reflected in the design of the DFAP which enables the beneficiaries themselves to work together in selecting those who receive services. For example, pregnant and breastfeeding women in the communities are clustered according to geographic location and then select a leader amongst themselves to receive training and teach them on a bimonthly basis. Farmers who were interested in being involved in the DFAP (as informed by CDCs and FH staff) attended meetings and formed Farmer Leader Groups in which they nominated their own group leaders. Similarly, WASH committee members, and CDC members were nominated by the community members that they serve. Even when team members requested to interview those “unaffected by the programs” in many cases they were involved in one aspect or another of the program. It was difficult to find someone living in the community who was not touched or involved in some manner unless they were recent or temporary arrivals. 6 Program Director, Bukavu, November 12, 2013, Individual Interview. 16 FH/DRC Development Food Aid Program Mid-term Evaluation Report Value of Food Aid and Activities In general, the team found that the food-based activities, primarily NRM activities, were appropriate and effective. Participants in those activities reported reasonable work schedules, relevant activities that were beneficial to the community, appropriate ration composition and size, and good timing for the lean season. Food for Work activities targeted the most vulnerable, including the elderly, disabled, and female-headed households. Each of these households designated a member of the household to participate in FFW activities to qualify for food for the household. Food was originally programmed for this project because it was determined that it was a more viable intervention during the hungry season than cash or vouchers given local market constraints. Food is less available and more expensive during the hungry season affecting vulnerable households. Food was seen to be most appropriate as it was most likely to result in improved food consumption within vulnerable households. In order to achieve the full scope of NRM activities, compensation through FFW was necessary to accomplish necessary activities. Since beneficiaries also receive training and messaging at the NRM sites, over time, community ownership of assets has been observed by FH staff to increase. In line with the approved plan, FH DRC has been receiving 1,230 metric tons of cornmeal, 180 metric tons of split peas and 90 metric tons of vegetable oil (a total of 1,500 metric tons) per year for direct distribution to beneficiaries. It was reported that FH received more than 99% of the amount for the period except for a low percentage (less than one percent) of marine and inland transport losses. The program appears to have never faced problems in reporting and/or accounting for food commodities because it has a very good commodity tracking system (CTS) in place. The evaluation team was only able to talk with few food aid beneficiaries, as they constitute a small portion of the total beneficiaries of the program. Interviewed beneficiaries were able to mention the type and amount of commodities they have been receiving in each distribution. The community members appear to be satisfied with the types and quantities of commodities being distributed. The project staff claimed that this is a result of communities’ sensitizations, pre-distribution meetings and banners exhibited at distribution points. According to a review of records, the overall performance of FH’s Inland Transportation, Storage and Handling has been very good, although there was a case of commodities misappropriation loss of 2.55 metric tons of cornmeal and 0.187 metric tons of vegetable oil which is a total of 2.737 metric tons, equivalent to 0.18% of the total annual allocation. This was reported according to regulations. The targeting process was done in collaboration with CDCs in a transparent manner and women’s inclusion was high as female-headed households were given priority during selection. Interviewed members of the targeted communities, who are food aid beneficiaries, are happy with the distribution processes and timing, as they receive information at all steps. FH staff and community members confirmed that members of communities have been encouraged to report through their leadership when they notice cases of fraud, protection violations, and under-receipt before, during, and after distributions. 17 FH/DRC Development Food Aid Program Mid-term Evaluation Report The program faced no major challenges with regard to food procurement and import but food deliveries from Dar es Salaam port to DRC primary warehouses in FY 2010-2011 were delayed because the contractor had insufficient trucks to move commodities quickly. The issue was reported to have been resolved amicably and no similar incidences were experienced. During the delay of the food commodities, FH had regular meetings with communities to encourage them to continue working until all commodities are delivered to the FH warehouses. It appears that program acceptability and effectiveness was not affected because when commodities arrived in the country beneficiaries received the full rations to which they are entitled. Considering the entire commodity pipeline, call forward, procurement, shipping, receipt in the country and final distributions, the evaluation team felt that these processes have worked well. Warehouses in general were well kept in accordance with FFP regulations. These are routinely monitored by both Bukavu-level and Regional commodity staff to ensure compliance and good handling of the food resources. Sustained Outcomes and Impact In the design of the program, FH based its exit strategy on five key outcomes: 1) community capacity to plan and manage development; 2) productive community assets; 3) community ability to continue learning; 4) integration into productive value chains; and 5) women actively participating in the community’s social and economic life. In addressing this question, therefore, the Mid-Term Evaluation Team evaluated how well the DFAP is in reaching these outcomes. Community Capacity to Plan and Manage Development The Community Development Committees are active and have taken the lead and coordination of development actions within their respective communities, such as protection of water resources.7 Members of the CDCs stated that they were “better united” and that they had created a network between other villages in order to exchange “lessons of development.”8 When there is a problem in the community, the members of the CDCs would take these issues to the chief of the village rather than to FH. This is also positive evidence that the CDCs have been instrumental in building and sustaining community ownership. However, some close-by communities share one CDC rather than each having one in their village. The MTE Core Evaluation Team feels that there could be increased emphasis on broadening the CDC coverage within the catchment area and also deepening the quality and quantity of resources and trainings these CDCs receive. Trainings on designing community development plans is planned for this upcoming year with the CDCs. Also, reinforcing trainings on leadership and facilitation are important for CDCs to strengthen the role that CDCs have in their communities. CDCs need to be equipped to know where to go for assistance and how to respond to needs once the program ends. Productive Community Assets Productive community assets include community land management and usage plans and also land revitalization (e.g. irrigation systems) and/or reforestation. Land revitalization and reforestation are working well as reported by communities. Communities appreciate previously-unarable land being reclaimed and express interest in expanding and continuing this activity. The variety of species was mentioned by communities due to their improved 7 Community Development Committee, Kurundi, Mubumbano, November 6, 2013, Focus Group Discussion. 8 Ibid. 18 FH/DRC Development Food Aid Program Mid-term Evaluation Report survival rates and increased utility, such as fruit, cash, and erosion-control. The issue of land ownership and management, however, is a sensitive one. FH must continue to keep this in consideration in planning activities for the remaining years. This program plans to organize community forums especially in South Kivu where land ownership remains a challenge, FH will invite key stakeholders including government representatives, other implementing organizations, and community leaders to discuss land conflicts and ownership issues to advocate for greater access to long-term use of land for small-holder farmers. Community Ability to Continue Learning Improving the communities’ ability to continue to learn is a strength of the DFAP and the peer-to-peer education model employed by the Care Groups and Farmer Groups. The MOH and IPAPEL both expressed a deep value for the cascade model used in both the Care Group structure and Farmer Group structure in efficiently reaching large numbers of rural households with useful and appropriate messages. The MOH and IPAPEL sign off on FL and ML identification cards so recognize the important roles that these volunteers have in furthering their own mandates. Both government institutions also expressed a strong desire to expand these models to territories and zones outside FH’s catchment area, as they believed FH’s model to be very effective in achieving lasting social and behavior change.9 Moreover, the beneficiaries also expressed in every site their appreciation for learning new things and how the learning had translated into changed behaviors and ultimately improved quality of life. CDCs interviewed knew the project end date and either knew or were interested to know more about outside resources for assistance in their communities. The MOH and IPAPEL also seem likely to continue to support these activities, but because of the government institutions’ low resources it is difficult to say with confidence that they will be able to receive an appropriate level of support Integration into Productive Value Chains This is perhaps the least developed strategy among the five outcomes in ensuring program outcomes and impact. IITA has been contracted by FH to provide technical support in assessing what kinds of value chains will be the most effective agro-development. The evaluation team noted that in March 2013 IITA conducted value chain studies on five major agricultural crops in eastern DRC including cassava, maize, groundnuts, banana and sweet potato. In total 275 producers and consumers, 21 transporters and 61 traders and wholesalers of these products were interviewed. However, as of the date of the interview (eight months after the data collection), this assessment report from IITA had yet to be provided to FH. Due to the delay of the report, FH was not able to accomplish any value chain development activities that depend on value chain recommendations. Overall, the evaluation team felt that the delay is significant and not well justified. FH should follow up on the report to quickly begin these activities in the remaining years.10 Women Actively Participating in the Community’s Social and Economic Life 9 Nutritionist and Chief of Health Zone, Ministry of Health, Wulungu Zone, November 11, 2013, Group Discussion with MOH staff and IPAPEL, Wulungu Zone, November 13, 2013, Individual Interview 10 A draft of the value chain report was submitted to FH from IITA on November 26, 2013, though the final is still outstanding. 19 FH/DRC Development Food Aid Program Mid-term Evaluation Report From the mixed-gender group discussions, the MTE Core Evaluation Team found that overall women were not hesitant to share their thoughts, experiences and opinions. Women were also active participants in every part of the DFAP program. This program has also allowed more women to become leaders in their communities, as they have been given the opportunity to be Leader Mothers in Care Groups, Farmer Leaders in Farmer Groups, and also active participants with leadership positions in the CDCs, Wat/San Committees and the VSLAs. Women comprise 35% of Farmer Leaders and 55% of VSLA members. The VSLAs seem to be particularly effective in improving the opportunity for women to increase their financial status, as it has given them the funds needed to purchase livestock and seeds. However, there is still much more that can be done in this area as the recent LQAS health assessment found that an alarming 75% of men still feel that it is okay for a man to hit a women in one or more situations. This contrasts with some interviewed communities reporting reduction in GBV and some communities’ self-reporting community enforcement of GBV through shaming and other methods. Both men and women reported increased sharing of household financial resources and decisions and more importance placed on children’s education. Additional anecdotal evidence from interviews and group discussions includes fathers helping out more around the house fetching water and helping with and bathing children even. Monitoring and Evaluation Strengths of the Program’s M&E System Food for the Hungry has a robust M&E system, which includes: 1) monthly quarterly, and annual monitoring of activities, outputs and inputs; 2) a baseline, this mid-term, and a final evaluation; and 3) periodic monitoring of beneficiaries through mini-KPC surveys, pre- and post-tests, quality improvement verification checklists, and random spot checks. Food for the Hungry M&E staff have been well-trained in Lot Quality Assurance Sampling (LQAS) and are able to conduct assessments efficiently and accurately in the collection and dissemination of information. Weaknesses of the Program’s M&E System Ongoing M&E training and technical support is still required as the MTE Core Evaluation Team found that there seemed to be some confusion and a lack of understanding among non￾M&E staff in regards to the role of monitoring and evaluation. Monitoring and Evaluation staff stated that at times they were regarded as the “police,” which created tension between them and other program staff. FH’s current efforts to ensure that staff understand the role of M&E better, that it is not a policing tool but rather a supportive tool, ought to be prioritized. Finally, in regards to M&E and the sustainability of program outcomes, there is also a need to train to the CDCs so that they are equipped with the knowledge and tools to better monitor development performance within their communities. FH is working on developing a more robust M&E database, the need for which was identified both by the Data Quality Assessment and this evaluation. This includes improving data organization and storage at the central and field offices. These changes would systematize data collection and retention, protecting against data loss and alterations. FH is working on implementing recommendations from the DQA. 20 FH/DRC Development Food Aid Program Mid-term Evaluation Report Technical Sectors This section of the report includes specific findings and conclusions for each technical sector and intervention. WatSan Infrastructure According to sample site visit findings, the overall technical quality of the program-supported infrastructure (specifically water wells/springs and latrines) appears to be good and in line with local standards. The beneficiary community members didn’t raise any issues with quality of work, rather responded with appreciation for the infrastructure. The assessment team did not find any adverse environmental effects of these installations. • Wells and pumps were nicely located and are well managed by the communities.11 o Communities reported that wells were placed with community input and have reduced the distances they have to travel to fetch water. It was reported that wells have reduced disease (such as diarrhea) incidence in communities. Wells were all managed by WatSan committees and sustainability practices included household fees for usage, regular opening and closing hours, and preventative maintenance and care. • Latrines built at schools and other public places were appreciated by the community. o There were an adequate number of latrines per male and female students and latrines were well-utilized by school children according to WatSan committee members. Health and Nutrition • Review of documents and staff interviews revealed that FH has conducted formative research (LDM and BA studies) as originally planned. The evaluation team noted that the research findings were used to adapt and fine-tune training modules and flip charts to better respond to existing barriers. For example, the barrier analysis identified the determinants that prevent pregnant mothers from going to at least four pre-natal visits and necessary adjustments were made on Module 8. Other findings that led to improved messages were the findings on snacks(peanuts and grilled maize) that healthy children were consuming and the introduction of semi-solid/mashed food at six months. • Hygiene practices have been promoted through the CG structure. Beneficiaries shared that IYCF behaviors have improved, and men are more involved with child care. Women and their family members expressed that they appreciate the value of the lessons and practices and are eager to continue to learn new things. • Some incentives such as T-shirts have caused divisions between beneficiaries – some mothers refused to participate until they too receive an incentive. Based on these and other similar findings in other FH countries, it is strongly advised to not use incentives with volunteer leaders unless all mothers are able to share and receive the incentive. • Health flipcharts have been delayed, slowing down the rollout of health teaching. The enterprise which was selected through a competitive bidding process was delayed in printing the modules. Also they incorrectly printed a health module which was identified when the modules were delivered to FH. Modifications were made to the printed material. 11At certain times of the day wells were closed to avoid community members using them inappropriately. 21 FH/DRC Development Food Aid Program Mid-term Evaluation Report • In Mubumbano, FH staff felt that the ratio of promoters to Care Groups was too low which meant that Promoters could not adequately reach their Care Groups and Leader Mothers. Ratios and work load should be reviewed and compared with workload in other regions to assess if changes are necessary or if it is merely a result of the relief mentality felt by many in the Mubumbano region. • Demand for family planning (FP) is high. The FP module has been developed and will be a very important module to promote through the CG structure. This module will be taught in the current implementation year. Natural Resource Management (NRM) • Beneficiaries reported that both the quality and quantity of the seeds/seedlings are good. Communities also appreciate that through this program previously unarable land will be reclaimed for the benefit of the community. Large tracts of land will be restored for use by small-holder farmers while other areas will be reforested to prevent erosion. • The work plan for the drainage and irrigation activities has been received but a decision on the budget has not yet been resolved. As mentioned, this activity requires more resources than had originally been allotted after receiving a technical evaluation by an NRM expert. • There is a need to increase awareness of the importance and benefits of reforestoration amongst the communities so that they take greater ownership of this activity. While some communities expressed understanding the benefits of decreased soil erosion, there needs to be a greater emphasis on making this well-understood by all. • Seedling production sites were found to be too far from planting sites in some cases making transportation difficult. FH adjusted to create more, closer production sites to ease logistics. • While FH’s partnership with existing local research organizations for seed supply is commendable, the fact that seedlings are not always available was mentioned as a weakness, which may make it difficult for FH to reach some of the indicators related to agriculture production. • Land tenure issues make land-related decisions difficult, creating a challenge for FH to support communities in capitalizing on the land available for agriculture and livelihood production and growth. Access to land is a challenge in South Kivu as the local chief holds rights to the land. Often land can be granted to an individual only under specific circumstances, or in exchange for money. Small holder farmers often lack the initial capital to pay in order to have long-term rights to the land. A consequence of this is that few farmers have an official legal right to land in South Kivu. These issues were understood during program design and are being addressed under program priorities under IR 1.2. FH will help to organize community forums with local and provincial authorities to advocate for land use rights for smallholder farmers. • Food for Work beneficiaries report satisfaction with the levels of work and ration size and composition. Agriculture • There is greater community participation and better coverage in the agriculture-related activities in the DFAP than in the MYAP due to the use of the cascade model. 22 FH/DRC Development Food Aid Program Mid-term Evaluation Report • Integrating teaching and trials is working very well, and the seed multiplication stations have been an excellent teaching opportunity. This can be seen in the postharvest technologies use and improved crop variety usage indicators’ targets (593% and 117% respectively.) • As a result of this program, many more women and men are now working together in agriculture than in previous years, and FH’s gender sensitization activities (in partnership with SFCG) seem to be effective in bringing this change. • A weakness of this sector is that agriculture modules for farmer groups have been significantly delayed due to logistical issues (same reasons as health modules being delayed) leading to delays in training of certain modules. In the meantime refresher trainings have been organized regularly in order to reinforce messages included in past trainings. • In Mubumbano, it has been difficult to move directly to the development phase from the emergency phase. There was no transitional phase from emergency and development, and there are still NGOs in Mubumbano who are conducting emergency distributions, undermining FH’s approach to building communities’ ownership of the development process. For example, community members receive free food rations given by other organizations in the same areas where FH has FFW activities. In this program, food is given in exchange for work reinforcing the idea that development is community-oriented and requires energy and time which counters an emergency distribution approach. Access to Credit • The VSLAs have been extremely successful and are a strength of the program. Community members have been using VSLAs for income generation (i.e. purchase of livestock and more farming opportunities) and to help pay for medicine, school fees, etc. The VSLAs have been replicating organically and are increasingly demanded from community members. o As evidence for demand both from men and women for VSLAs, the following is excerpted from the IPTT: FY12 FY13 Target Achieved % of Target Target Achieved % of Target Number of VSLA groups formed 13 59 454% 26 141 552.9% Number of participants in VSLA groups 260 1336 514% 510 3979 780.2% Men 150 727 485% 305 1779 583.3% Women 110 609 554% 320 2200 687.5% • While VSLAs are a successful form of promoting savings and credit access, it appears there is a need for a different type of economic development support in terms of training beneficiaries in non-farm income generating activities (e.g. masonry, sewing, baking) so that they can diversify their income sources above and beyond the traditional farming related ventures. • A recent training from FH’s Savings Group Technical Specialist was well received by staff and helped to build their capacity. During this meeting all key staff involved in VSLA activities shared lessons learned, best practices, and received trainings on identifying and training volunteers. Community Development Committees 23 FH/DRC Development Food Aid Program Mid-term Evaluation Report • Community Development Committee members demonstrated knowing when the program will end, and they are taking charge of development within their communities. The CDCs have fostered greater networking with other villages and provided a mechanism to exchange lessons of development. The CDCs have also fostered a greater acceptance of refugees into the community which has improved community solidarity and unity. Some CDCs have coordinated road rehabilitation efforts in their communities. Others have started to build a health center or pharmacy to serve the local population. • Many CDCs verbalized the belief that they were “a bridge between FH and the community.” However, not all of them were able to describe their role once FH left the community. Some CDCs had already taken steps to aid others in the community on their own initiative. Others expected to end once FH left the community. Greater sensitization and capacity building is needed to allow CDCs to execute good governance after the departure of FH, in response to the needs of their own community. According to the DIP, additional training will be given in the coming years to increase their awareness of disaster risk reduction and response to emergencies and shocks. Gender • Food for the Hungry has a greater number of female staff working than ever before, and male staff are becoming more willing to work alongside female staff as equal partners. • Program trainings are scheduled in and with communities to accommodate women and to make it easier for women to attend. • Men are reporting helping more with household activities (i.e. men are collecting water and assisting in child rearing). • There is more public shaming of men who beat women, which will increase the stigma associated with violence against women. • The delay in the startup of the gender component of the program has limited what the implementing partners could have achieved if it started from the beginning. However, the team appreciated that the delay in the first year was largely associated with logistical challenges due to bad road conditions and frequent flight cancellations, as SFCG staff were shuttling between Goma and Bukavu to recruit staff and set up offices. Currently the gender component of the program is on schedule. Effective behavior change communications Strengths of the behavior change programs include the development of flipchart materials which are easily understood by the beneficiaries, translated into the local language (for ease of use with promoters) as well as pictorial so even the non-literate participants are able to teach and receive the appropriate messages. When asked about favorite messages, or messages which had been most beneficial, all of those interviewed could easily recall something they had heard, implemented and changed in their own life because of the teaching. Beneficiaries reported that the messages were beneficial to them and easy to apply to their own lives. Several farmers reported that for years they had been raising goats, but the training module they received from FH contained simple information that they had never heard before which was helping them to better care, feed, and breed local goats. As with the MYAP, Barrier Analysis and LDM findings have been integrated into the flipchart and lesson plan materials making it easy to transfer the findings from studies to application in the teaching and promotion of new behaviors. 24 FH/DRC Development Food Aid Program Mid-term Evaluation Report Weaknesses of the behavior change materials/implementation relate to delays in the rollout of materials. Both health and agricultural modules have been delayed as troubles with printing companies, logistics and development have put FH behind schedule. To help overcome the anticipated delays, FH hired a consultant last year to speed the process of module development. As of the date of the MTE, one review workshop had been held in Bukavu to review the draft agriculture modules, but the artist’s drawings were still being completed. By early March FH expects to have four additional agriculture modules (seasonal calendar, staple crops and post-harvest management, fruit trees, and agroforestry) printed and ready to be used with the farmers. Both the lead health and agriculture technical specialists have recently resigned which has also slowed the process of module review. Nonetheless, FH is in the process of filling these positions, and field staff are now involved with material reviews to speed the revisions and final rollout. Different perspectives and needs of men and women and gender equity in households and communities Strengths The gender component of the program has been implemented in partnership with SFCG and, in spite of the startup delays, various gender activities were accomplished with encouraging results. It appears that the changes in the behavior of the village chiefs is a proof to the positive changes happening in the communities. For instance, SFCG staff in Mubumbano explained that there were lots of questions and resistance when they first gathered with the local chiefs to explain about the program. Various questions were asked including, “What are you bringing? Do you want our women to start a revolt? Are you trying to encourage them to be disobedient?”. But after sensitization meetings, theater and film shows, the chief of a neighboring district came back to ask if the project could mobilize men in his community. He said that “I have noticed from the start that the movies and teaching has broken some of the laziness of the men who want to stay at home while women work.” FH and SFCG staff and community interviews revealed that the results of the gender equity messages have been impressive. Traditionally, rural women in the target provinces are responsible for the majority of farming activities in the field, besides their reproductive roles. After attending the various gender sensitization meetings, dramas, etc. community members have now started to notice more men accompanying the women in the field work. They reported that assumptions about specific roles for men and women are now gradually changing. Men are helping with fetching water and caring for kids; HH expenditure management is changing and fighting/beatings are visibly down. Now, there is communal shaming if a man beats his wife. Furthermore, community members asserted that women attend and participate in community meetings more than before. According to staff interviews, most of program-initiated community groups and committees such as CDCs and VSLAs, as a matter of necessity, have considered women’s participation as a priority. The evaluation team found that FH requires VSLAs to have at least 50% women members. In addition, it was learned that more female supervisors have been hired recently in a move to balance gender among staff. For example, there are 18 female staff among a total of 68 in Kalemie, currently. 25 FH/DRC Development Food Aid Program Mid-term Evaluation Report Overall, the evaluation findings strongly suggest that the gender component of the program has been effective in delivering the messages and influencing behaviors towards more equitable gender relations and toward improved household food security. Weaknesses Although impressive results were achieved there is yet a lot to be accomplished to further improve gender equity among the communities. WATSAN committees and farmer leaders groups are still composed predominantly of men. An indirect beneficiary woman in Kalemie, for instance, reported that she wanted to join the WATSAN committee in her community, but didn’t feel comfortable as a woman to join an all-male group. This may have been related to the way elections were held in this particular community since 42% of all WATSAN committee members are women. In addition, it was noted in several interviews that the gender activities were quite small compared to other interventions. Women and men alike asked for these activities (gender groups) to be expanded so that other men and women in the communities could hear the teachings and be part of the discussion groups. Despite the limited activities planned by SFCG, gender attitudes seem to be changing rapidly within communities where SFCG operates. This impact reaches beyond the small number of direct beneficiaries and has shown promising signs of changing social norms and perspectives linked to gender in the communities where SFCG has targeted thus far in the program. Changes in the community, households, and individuals Community members and project and partner staff were asked to express views about their observation of any positive and negative changes as a result of the program. In spite of the evaluation team’s probing, however, no negative or unintended changes were mentioned in the interviews and group discussions. The positive findings are described below by sector. Health and Nutrition Apparently, significant signs of positive changes in the lives of the target communities have been observed in the area of health and nutrition. According to staff and community discussions, mothers were able to appreciate and take up various health and nutrition messages promoted by the program. For instance, a leader mother in Kalemie said, ‘before attending the Care Group messages, we used to wash our hands all in the same basin. Now we use soap and separate basins so our family won’t be sick’. Women in a group discussion, on the other hand, compared their practices before and after the start of the program. They said, in old days ladies did not eat enough during pregnancy and were usually weak during delivery. These practices have now significantly changed for the better, especially for most of those who participate in Care Groups. Interviewed mothers also reported significant improvement in breastfeeding and other child feeding practices. A breastfeeding mother in Kalemie said that she “used to breastfeed for only a short time and start giving porridge early before the child is ready. Also in the past people were not encouraged to send their children to the health clinic. See my baby – 5 ½ months – she is still exclusively breastfed.” Yet another mother stated that normally when coming back from the field, she brings cassava leaves to eat. Now that she has been trained about the benefits of diversified meals, she adds groundnuts and oil. When cooking vegetables, she tries to have fish so children can have a balanced diet. 26 FH/DRC Development Food Aid Program Mid-term Evaluation Report Other positive changes cited by care group participants and staff include: • Children are now well-fed compared to previous situation; • More families use mosquito nets; • More interest and support to leader mothers is being shown, which used to be a challenge at the start of the program; • More and more women who used to deliver at home now visit clinics; • Reduced maternal and child deaths during delivery. The quantitative result of the Health and Nutrition interventions from the LQAS study conducted for the ARR corroborate these achievements as evidenced in the table below. FY13 Results from the IPTT Indicator Target Achieved % of Target met % of children 0-5.9 months exclusively breastfed 55% 84% 153% Children 6-23 mo. receiving a minimally acceptable diet 40% 78% 194% % of children sleeping under a mosquito net the previous night 45% 74% 164% % of women 15-49 who had four or more antenatal visits 45% 72% 159% Agriculture/NRM Although many of the planned activities in the agriculture/NRM sector have been delayed, there are some activities that the community members mentioned as successful. As a result of the goat production training, for example, many farmers reported receiving new ideas and knowledge about goat rearing. A farmer leader in Kalemie reported that he has been raising goats traditionally since he was a child. He continued to apply the traditional methods without any knowledge of improved practices till he joined FH’s farmer leader groups. He credits the program for the better understanding about improved rearing practices including how to differentiate the age of the goats, prevent illnesses, staking animals and when to feed and keep them in the house. The seed stations and demonstration sites were also praised for their effectiveness to reach illiterate members of the community. As District Inspector of IPAPEL put it, FH’s demonstration-based trainings allowed the illiterate to participate even if they can’t read. The use of flipcharts and field-based demonstrations allowed all community members regardless of literacy rate to participate and learn together. Water and Sanitation Since the start of the program FH supported the building of multiple water points in the three provinces. It appears that the water schemes have brought positive impact on the lives of the beneficiaries around the water wells. A WATSAN committee member in Kalemie reported that people were often sick from dirty water before FH facilitated construction of water well in their community. Community members in Moba also reported great reductions in diarrhea incidence in the community. They are also relieved from walking as far as 10 km to find potable water, as they can now find it very close. Access to Credit 27 FH/DRC Development Food Aid Program Mid-term Evaluation Report The major intervention regarding access to credit was the Village Savings and Loan Association (VSLA), which was one of the most successful components of the program. Community interviews were extremely positive about the VSLAs and their impact. Women in groups discussions cited various advantages of participating in the VSLA that include supporting vulnerable members of the community, paying for a grinding mill, medicine and labor, as the need arises. Because people have saved, they can now look for opportunities that can benefit them and the entire village. Owing to the strong interest from the target communities to participate in VSLAs, FH was able to go beyond its planned target during the past two years as shown above. Civil Society Strengthening The strong evidence to the program’s civil society strengthening efforts that was repeatedly mentioned in the discussions was the commendable performance of the community development committees. The evaluation team is convinced that CDCs are effective grassroots-level structure to facilitate community mobilization, community development, ownership, and management of resources. For instance, a target community in Kalemie had very heavy rains that destroyed the roads – even bicycles could not pass. CDC sensitized the community and together they rebuilt the road, without any involvement of FH. In Moba also the team met with a CDC who had taken the initiative to repair their community road. Once they started working themselves, other community members saw their example and the benefit and contributed labor and material to repair the road of their own volition. RECOMMENDATIONS Based on the findings and conclusions, the evaluation team recommends the following points to enhance the program’s effectiveness during the second half of the program life. Some of the recommendations may have budgetary implications that would require consultation and approval. • Move the drainage and irrigation activities forward - It is recommended to expedite the finalization of the revised plan and/or the decision on the direction of the drainage and irrigation activities. These activities are delayed and getting them back on track is essential to meeting the program goals.12 • Encourage female participation in activities and in leadership prior to selection of community members for the differing activities - As noted from beneficiaries, many groups would have selected members differently had they first had the gender training. For example, the CDCs, although including an equal ratio of men and women, women in far greater numbers hold lower positions as compared to the men. This can be achieved through organizing participatory theater or projecting a film highlighting the benefits of men and women working together. • Foster greater community ownership- Community ownership of activities is key for program sustainability. It would be helpful to hold exchange visits between CDCs to encourage information sharing and best practices. Place greater focus on communities where CDCs are not involved in monitoring or program implementation activities. 12 FH has submitted a request to FFP to alter the scope and budget of these activities and is awaiting a reply. 28 FH/DRC Development Food Aid Program Mid-term Evaluation Report As this is not apparent in all communities, it would be best for staff to focus on those communities where ownership is low. • Continued technical support to increase staff capacity- This includes increasing supervision and providing feedback for savings groups, monitoring and evaluation, and gender sensitization with staff and communities. • Further context-specific market assessments should be done to decide on the types of technical skills to promote and also look for partners within the project area that can provide VSLA members with skills training. • Regular technical meetings to develop staff capacity should be conducted (at least yearly) to continue and troubleshoot problems within the VSLA system. • Monitor construction of tree nurseries closer to reforestation sites to determine whether this should be adopted on a wider scale. • Maintain and improve communications with government partners to define expectations - FH should hold conversations with the MOH and IPAPEL to reiterate FH’s commitment to its program objectives and actions within the scope of the project to dispel ideas that FH has the capacity to donate vehicles during the life of the program. • Improve flipchart availability - FH should look for ways to provide flipchart materials in greater quantities than currently, both agriculture and health and nutrition, to IPAPEL and the MOH. This would be a good opportunity to work with government institutions to better allocate resources to prioritize expenditures on effective methods. Additionally, FH should follow up and conduct regular spot checks in the future on selected printing companies to prevent incorrect printing of modules. • Improve internal communication regarding importance of M&E and clarify its role in reporting and program improvement. • Develop backup plans in case partners do not meet their commitments, so that the program overall does not suffer too much from delays brought on by partners. 29 FH/DRC Development Food Aid Program Mid-term Evaluation Report Annex 1 – Scope of Work FH DRC Development Food Aid Program Mid-term Evaluation Scope of Work 1. Introduction a. Description of the program FH DRC’s Development Food Aid Program (DFAP) is funded by USAID/FFP and operates from August 1, 2011 to July 31, 2016. The program’s goal is ‘Reduce Food Insecurity of Vulnerable Households in South Kivu and Katanga Provinces’ and the SOs and IRs are as follows. SO 1: Improved Livelihood Capacities of Vulnerable Households IR 1.1: Increased Agricultural Production IR 1.2: Improved Land Management and Natural Resource Conservation IR 1.3: Increased Household Income from Agricultural Production SO 2: Improved Health and Nutrition of Individuals within Vulnerable Households IR 2.1: Improved Use of Essential Nutrition Behaviors IR 2.2: Increased Diversity and Consumption of Nutritious Food IR 2.3: Improved Household and Community Management of Conditions and Diseases that Exacerbate Malnutrition IR 2.4: Improved Use of Clean Water, Sanitation Facilities, and Hygiene Behaviors CC 1: Improved Gender Equity in Decision Making and Labor Sharing CC 2: Increased Access to Credit CC 3: Improved Community Capacity and Resilience to Shocks Under SO1, capitalizing on significant infrastructure investment, including roads, bridges, and markets, FH will improve agricultural production (IR 1.1) in sustainable ways (IR 1.2) to increase smallholder farmer incomes by reestablishing and enabling producers’ vertical integration into profitable value chains (IR 1.3). Based on learning from the MYAP, FH adopted a new strategy in the current program to scale up and reach wider farming population with the improved agricultural techniques through an expanded agricultural cascade extension (ACE) training model. Further, a new strategy of field-based seed testing and multiplication stations will be used to dramatically increase access to improved, diverse and disease-resistant seeds and cuttings. Using FFW resources to rehabilitate large-scale irrigation canal systems, FH will revitalize over 12,000 ha of fertile land and enable these zones to reclaim their ‘breadbasket’ status though significant increases in agricultural production. FH will also ramp up its agro-forestry and reforestation activities to replant over 1,000 ha of denuded hillsides using FFW. An innovative partnership with Pharmakina, a local agro-business company in S. Kivu, will integrate community reforestation with a well-established and profitable quinquina bark production value chain. Participative market analysis and business skill development will enable smallholders in all sectors to successfully sell their agricultural produce to both local and regional markets. 30 FH/DRC Development Food Aid Program Mid-term Evaluation Report After years of emergency relief assistance, many doubted that the volunteer-based CG model would work in DRC. Proving them wrong, these volunteer ‘mother leaders’ have become some of the most dynamic and motivated development actors in their communities. Under the Health and Nutrition (H/N) SO 2, FH will expand its successful CG model, which builds the capacity and social standing of women in their communities to regularly reach over 50,000 mothers of children aged under two (IR 2.1). Men will also be targeted with H/N messages through a specific focus on influencing community leaders and creative radio programming aimed at both men and women. In an effort to improve dietary diversity and nutrition, FH will complement related H/N messages through the CGs with promotion of HH vegetable gardens, fruit trees and small livestock animal production for all households with pregnant women or children aged under two (IR 2.2). In response to potable water coverage rates of 9% in Katanga and 23% in S. Kivu, using FFW, FH will construct 330 water points. One of the lessons learned from the previous MYAP was that livelihood improvements are not enough to improve wellbeing of communities where there is widespread gender inequity and gender based violence. In a strategic shift of programmatic focus in the current program FH intends to create a constructive, participative and culturally appropriate dialogue around issues of gender inequity through partnership with Search for Common Ground (SFCG). Using a diverse range of tools including participatory theatre, and radio production, FH/SFCG will raise knowledge, shift behavior, and improve the status of women in the targeted communities (CC 1). The link between gender issues and achieving improved availability, access, and utilization of food has been carefully incorporated into each SO and IR in the proposed program framework. A redesigned and expanded agricultural extension model will ensure equal gender representation and inclusion at all levels under FH’s Livelihoods Strategic Objective 1 (SO 1). FH will also improve access to credit to over 12,000 HHs through the establishment of VSLAs and connecting agricultural producers to appropriate credit mechanisms (CC 2). As the key point of entry into communities, FH will build the capacity of CDCs to monitor, coordinate, and create local synergies between different community development actions (CC 3). Over the past 10-15 years, FH staff have sacrificially served EDRC communities through periods of intense conflict, massive displacement and subsequent return. Working alongside families to rebuild homes, livelihoods and infrastructure, FH has gained their trust and respect. Building on these years of physical and social investment, FH expects that the creative behavior and social norm changing, and innovative market-oriented programming described above will result in dramatic improvements in all aspects of food availability, access, and utilization for the targeted food insecure households in S. Kivu and Katanga. b. Evaluation objectives The objectives of the internal mid-term evaluation are: • To evaluate the strengths and weaknesses of program implementation and the quality of outputs so far, in terms of adherence to terms agreed by FFP and their acceptability and perceived value to target groups, identifying factors that appear to enhance and detract from the quality, acceptability and usefulness of implementation and outputs. 31 FH/DRC Development Food Aid Program Mid-term Evaluation Report • To evaluate the strengths and weaknesses of the program’s strategy to promote gender equity in targeted communities. • To seek evidence of changes (intended and unintended) associated with program activities and identify factors that appear to promote or hinder the program’s progress toward desired objectives. • To evaluate program strategies that seek to assure that positive results can be sustained after the program ends, identifying the strengths and weaknesses of these strategies. • Based on the findings from 1-4, to recommend adjustments to program implementation or design and explain how these changes would improve program outcomes and sustained impact. 2. Key Evaluation Questions The MTE will seek to answer the following key evaluation questions. General Implementation 2.1 What are the strengths and weaknesses in the design and implementation of the various activities, considering the terms of the agreement with USAID, appropriateness of methods, clarity/communication of vision, the adequacy and use of resources (human and material), and achievement of outputs? How well have the results of preliminary analyses been applied to program and activity designs? 2.2 What are the strengths and weaknesses of the program’s partnerships with local institutions and other actors (government, NGO, private)? 2.3 How successfully has the program strategies encouraged members of target communities to contribute to the design, implementation and monitoring of each activity? 2.4 How efficiently is the program reaching the intended beneficiaries? (accuracy of targeting) 2.5 How well does food aid distribution add value to or detract from the performance and effectiveness of the related activities? How well is the program implementation managing and distributing the food commodities? 2.6 How well has the program initiated measures to ensure sustained outcomes and impact? 2.7 What are the strengths and weaknesses of the program’s M&E system in terms of collecting and disseminating accurate information that is useful to the decision making and activity performance? Implementation of Technical Components (Infrastructure development, Health & Nutrition, Natural Resource Management, Agriculture, Access to Credit, Civil Society Development & Emergency Preparedness) 2.8 What is the technical quality of the implementation of and outputs from the various activities in the different sectors? What aspects of these activities are or are not appreciated and valued by members of the target communities? What are the 32 FH/DRC Development Food Aid Program Mid-term Evaluation Report strengths and weaknesses of FH’s various new approaches to implementation in this context? (See detailed questions to the individual sectors in annex) Cross-cutting Issues 2.9 Social Behavior Change Communication: What are the strengths and weaknesses of the ways the program reaches the various target groups with effective behavior change communications? 2.10 Gender equity: What are the strengths and weaknesses of the ways that the program implementation considers and responds to the different perspectives and needs of men and women and promotes gender equity in households and communities? Outcomes 2.11 What changes (positive and negative) in the community, households and individuals have community members and local staff observed so far that they attribute to the program activities, and what factors appear to have promoted these changes and hindered others? 3. Evaluation Methodology The mid-term evaluation will employ a participatory qualitative study methodology to assess the strength and weaknesses of the various program approaches and strategies as well as implementation processes. It will be an iterative process of data collection from multiple sources and analysis throughout the evaluation period. Participatory data collection tools such as focus group discussions, household interviews, key informant interviews, observation and document reviews including routine monitoring reports will be employed to collect data from multiple sources. Detailed checklist of open-ended questions shall be prepared in advance of the evaluation which will be reviewed and commented by FH and partner staff. Qualitative inquiry into the implementation of the various program activities will be conducted at multiple sites per activity type, including sites from each of the three geographic areas of operation. In order to be able to explore the gender differentiated perceptions and opinions about the program and its benefits, part of the FGDs will be organized with a mix of men and women participants while others will be separate men and women group discussions. The discussions will be as participatory as possible, where all participants especially women will be encouraged to voice their feelings and perceptions on the programs progress. Apart from the direct beneficiaries, the team will also interview non-beneficiary households to get a broader view of the program, especially perceptions about the targeting processes. In depth interviews will be held with project managers and staff of FH and its partners as well as the key informant interviews with government staff, CBO’s leaders and selected community members. Apart from these, physical observation of selected activities will be conducted in accessible sites, which will be followed by discussions with men and women beneficiaries and program staff. In order to establish the context and also review progress, various progress and field visit reports, PREPs, proposal document, etc, will be thoroughly reviewed. 4. Composition of MTE team 33 FH/DRC Development Food Aid Program Mid-term Evaluation Report The core evaluation team will be composed of staff from FHUS and Africa Regional office, with the assistance of FH DRC and partners, who will be responsible for logistics and arranging meetings and team up during various community assessments. In consideration of the technical components of the program, the MTE team shall include staff with experience and expertise in food security/livelihoods, MCN, agriculture systems, gender, environment, and commodity management. Accordingly, FH has assigned three staff (Senior Food Security Officer, Health and Nutrition Officer and Coordinator of Education Programs) from the HQ and Food Security Coordinator from the Africa regional office. Between them the core team members of the evaluation have extensive experience in monitoring and evaluation, food security programming, health and nutrition, food aid, etc that will be beneficial for the process of data collection and analysis. The Regional Food Security Coordinator will be the evaluation team leader coordinating the assistance of the rest of the team. 5. Deliverables The following deliverables will be submitted to FHUS and DRC leadership. o Full description of the evaluation plan, including preliminary qualitative study tools and checklist o Debriefings: The team will debrief FH DRC and partner staff on their findings, conclusions and recommendations and collect feedback o Draft Evaluation Report o Final Mid-term evaluation Report. The mid-term evaluation report will contain: i. Executive summary ii. Introduction iii. Objective of evaluation iv. Brief description of the program v. Detail analysis of findings by technical sector – Agriculture/NRM, Health and Nutrition, Water and Sanitation, Access to Credit and Civil Society strengthening • Accomplishments of outputs (compared to plans), factors that promote and constrain implementation, beneficiary participation and output • Strengths and weaknesses of strategies and implementation processes • Evidence of early outcomes and factors that appear to promote or constrain adoption of desired practices • Program quality (management, commodity management, program sustainability, etc) vi. Cross cutting issues – gender equity, M&E vii. Conclusions and Recommendations viii. Annexes 6. Timeframe 34 FH/DRC Development Food Aid Program Mid-term Evaluation Report FH DRC Development Food Aid Program – Midterm Evaluation Schedule Key Activities Responsible Timeframe Pre-Planning Contact possible TLs, MTE team member candidates and consulting firms. Food Security and livelihoods Director and Senior Coordinator for Health Programs Sept. 12 Discuss & finalize MTE timeline and list number of potential MTE team members from partners and identify FH staff that will participate in the evaluation process Sept. 25 Organize project information and make available on dropbox: • Develop a list of all implementation villages/communities and classify them by level of program performance (e.g., excellent, moderate, needs extensive support). • Prepare number of beneficiaries/ CGs/ FLGs/ CDCs/ … etc by territory/ community FH DRC team Oct 7 Organize pertinent documents and make them available to the MTE team in a ‘dropbox’ folder. FH DRC team and core evaluation team Oct 11 Meet with partners to brief about the MTE and request representatives who will participate as MTE team members. (Before the meeting determine the core competencies of the team members, their level of experience and numbers needed). FH DRC Planning Finalize development of qualitative survey tools Core Evaluation Team and FH Global M&E Advisor Oct 9 Select sample communities/villages and decide on number of FGDs and Key informant interviews “ Oct 10 Develop an evaluation work plan and logistical requirements ” Oct 14 Review existing reports, documents and data (quantitative survey, program records). “ Oct 14-18 Arrange all logistics. FH DRC logistics Oct 31 Inform selected communities and individuals of upcoming survey FH DRC team Oct 31-Nov 1 Implementation Arrive at Bukavu Core Evaluation Team Nov 3 Introductory meeting with MTE team, brief/ train the MTE team (objectives, organizations, methods, approaches, tools) and finalize the qualitative study tools. Core Evaluation team Nov 4-5 Travel to Kalemie, Mububano and Moba Regional evaluation team (three teams, each led by one Core Nov 6 35 FH/DRC Development Food Aid Program Mid-term Evaluation Report Evaluation Team member) Conduct the field work (i.e., interviews, focus group discussions, observations, analyses, triangulations). Field interview/data collection and discussion with community leaders, consortium members and local government authorities Regional evaluation team (three teams, each led by one Core Evaluation Team member) Nov 7-13 Travel back to Bukavu “ Nov 14 Key informant interviews with partners and staff at Bukavu Core evaluation team Nov 15 Sectoral data analysis and synthesis “ Nov 16-20 Present the preliminary observations and results to FH/SFCG and partners to validate findings and interpretations. Regional Evaluation Team/ Sectoral leads Nov 21 Travel out of Bukavu to Individual destinations Core evaluation team Nov 22 Reporting Prepare a draft report and share it with USAID and concerned FH/SFCG Depts Core Evaluation Team Dec 9 The MTE TL addresses the comments and incorporates inputs Core Evaluation Team Dec 16-19 Finalize and submit final MTE report. Core Evaluation Team Jan 10 36 FH/DRC Development Food Aid Program Mid-term Evaluation Report Annex 2 – Individuals and Groups Contacted Individuals and Groups Contacted Bukavu Mubumbano Kalemie Moba FH and Partners Staff FH Programs Director 1 FH Commodities Manager 1 FH Regional Commodity Coordinator 1 FH Commodity Tracking systems Coordinator 1 FH HR manager 1 FH Finance Manager 1 FH M&E officer 1 1 FH site coordinator 1 FH Assistant site coordinator - Health 1 1 FH Assistant site coordinator - Agriculture 1 1 SFCG – Chief of party 1 Pharmakina representative 1 IITA representative (in Kabare) 1 INERA representative (in Mulungu) 1 IPAPEL representative individual interviews 1 2 Ministry of Health Walungo zone 1 Community Care Group Leader Mothers Group discussion 1 1 Care group mother beneficiaries Group discussion 1 1 Care group promoter group discussion (1M, 1F) 1 Care group promoter individual interview 1 Farmer Leader Group Discussion 1 1 Farmer leader individual interview 1 Farmer beneficiaries Group discussion (all male) 1 VSLA members individual interviews (1M, 2F) 3 VSLA members group discussion 1 1 WatSan individual interviews (1 latrines, 1 water point) 1 2(1F, 1M) CDC interviews (1M, 1F) 2 CDC group discussion 1 1 Non-participant interview 1 3(F) Female-headed HH doing FFW interview 1 FLG Promoter interview (M) 1 Mixed beneficiaries group discussion (M&F) 1 1 All female beneficiary group discussion 1 37 FH/DRC Development Food Aid Program Mid-term Evaluation Report Annex 3 – Individual and Group Discussion Guide INDIVIDUAL INTERVIEWS/ QUALITATIVE INQUIRY FH – Bukavu and Site staff Location: Interviewer: Date: Description of Group: Number in Group/Sex: Thank you for agreeing to talk with us and for your participation in this mid-term evaluation. Food for the Hungry wanted to conduct this internal evaluation with the objective of assessing progress of FH DRC’s DFAP to date and identify areas for improvements. We believe that you are in good position to tell us about your impression about the DFAP program. We anticipate the interview will last about an hour to an hour and half and appreciate any information you may provide. Your answers to the questions are completely confidential. 1) What are your general impressions of the relevance of the program and its goals? 2) What are your general impressions of the progress made toward achieving those goals? 3) Did you notice any changes in the original assumptions of the program that required changes/amendments to the program targets and/or interventions? 4) What do you think about the effectiveness of current methods and strategies towards meeting those goals? (Probe with those not mentioned) Agriculture strategies/methods a. The Expanded Agriculture Cascade Extension (ACE) model b. Integrated Agro-pastoral strategy c. Introduction of improved seeds d. Integrated soil fertility management e. Prevention of post harvest losses f. Value chains development, etc Health and Nutrition Strategies • Care group model • Essential nutrition/hygiene behaviours • Linkages with agriculture component 5) Have there been any amendments or changes to the program’s strategies and focus areas since the inception of the program and what was the rationale for these changes? 6) Are the program activities on track to bring about the planned results? a. Are they delayed? If they are delayed or cancelled, could you give examples? What do you think are the reasons? 7) How is communication between different levels of the hierarchy in FH and with partners? Are there any improvement areas? 38 FH/DRC Development Food Aid Program Mid-term Evaluation Report 8) What is your impression about the partnership with SFCG? a. Was the coordination effective? b. What challenges did you face? 9) What is your assessment of the partnership with other partners including INERA, CIALCA (and Pharmakina)? 10) What were the major constraints to managing and implementing the program? 11) What lessons did you learn during the past two years of program implementation? 12) What were the results of the LDM and the BAs? Have these results been implemented or integrated into program activities? 13) How many women and how men are employed by the DFAP? What is the male/female ratio? a. How many women hold management positions? b. What are the barriers in hiring women for leadership positions? c. Have program staff received any gender training? 14) How does the community view men’s and women’s participation? 15) What measures have been taken to assure that both men and women can access and participate in activities without added risk? 39 FH/DRC Development Food Aid Program Mid-term Evaluation Report Search for Common Ground (SFCG) Program Director and Project Staff Location: Interviewer: Date: Description of Group: Number in Group/Sex: Thank you for agreeing to talk with us and for your participation in this mid-term evaluation. Food for the Hungry wanted to conduct this internal evaluation with the objective of assessing progress of FH DRC’s DFAP to date and identify areas for improvements. We believe that you are in good position to tell us about your impression about the DFAP program. We anticipate the interview will last about an hour to an hour and half and appreciate any information you may provide. Your answers to the questions are completely confidential. 1. Please describe your organization’s activities in DRC and its role in the DFAP program. 2. Did you use studies to better inform your planning of the gender component in the program? a. What kind of study? What were the major findings? b. Were the findings used to inform the program implementation? How? 3. How do you coordinate your activities with FH DRC at Bukavu and the project sites? a. Do you have regular coordination meetings? How frequent? b. What do you discuss in those meetings? 4. How do you assess the partnership with FH? a. Has it been effective? b. Are there any improvement areas? 5. Do you think FH/SFCG’s gender activities are on track to bring about the planned results? a. Could you give examples? b. Are there activities that have been delayed or cancelled? What do you think is the reason? 6. In your opinion, which activities were successful? Why? a. Which activities were not successful? Why? 7. What do you think are the major achievements of the program with respect to improving gender relations? 8. What were the challenges you faced implementing the gender component of the program? a. How did you approach the challenges? 9. What new activities and approaches do you want to see included in the future? Why? 10. Are there any activities and approaches that you want to see revised or dropped? Why? 11. What steps are you taking to ensure long-term improvements (sustainability) in gender relationships? 40 FH/DRC Development Food Aid Program Mid-term Evaluation Report Government Agricultural Office Personnel Location: Interviewer: Date: Description of Group: Number in Group/Sex: (Note: FH DRC should select a manager or staff of Ag office who knows the program well.) Thank you for agreeing to talk with us and for your participation in this mid-term evaluation. Food for the Hungry wanted to conduct this internal evaluation with the objective of assessing progress of FH DRC’s DFAP to date and identify areas for improvements. We believe that you are in good position to tell us about your impression about the DFAP program. We anticipate the interview will last about an hour to an hour and half and appreciate any information you may provide. Your answers to the questions are completely confidential. 1) Are you aware of the DFAP program? In what ways do you participate in the program? 2) What are your general impressions of the program and its goals? 3) Are the program goals and interventions in line with the government policies and strategies? How? 4) What are your general impressions of the progress made toward achieving those goals? 5) What do you think about the effectiveness of current methods and strategies toward those goals? (Probe with those not mentioned) a. The Expanded Agriculture Cascade Extension (ACE) model b. Integrated Agro-pastoral strategy c. Introduction of improved seeds d. Integrated soil fertility management e. Prevention of post harvest losses f. Value chains development, etc 6) Do you feel a part of the process? Are you consulted during planning and implementation of the program? 7) How well has FH DRC managed the coordination/ relationship with your offices at different levels? 8) Impressions of how well the program is managed? 9) In your opinion, what are some of the areas FH shall improve in the second half of the program life? a. What areas are going well? 41 FH/DRC Development Food Aid Program Mid-term Evaluation Report Government Health Office Personnel Location: Interviewer: Date: Description of Group: Number in Group/Sex: Thank you for agreeing to talk with us and for your participation in this mid-term evaluation. Food for the Hungry wanted to conduct this internal evaluation with the objective of assessing progress of FH DRC’s DFAP to date and identify areas for improvements. We believe that you are in good position to tell us about your impression about the DFAP program. We anticipate the interview will last about an hour to an hour and half and appreciate any information you may provide. Your answers to the questions are completely confidential. 1) What is your position? And for how long have you held this position? 2) What are the major challenges for the MoH in reaching their goals for health for all? 3) Can you please describe what you know of FH’s DFAP, and more specifically what you know of the Care Group program? In what ways do you participate in this program? 4) What are your general impressions of the program and its goals? 5) Are the CG program goals and interventions in line with the government policies and strategies? How? 6) What are your general impressions of the progress made toward achieving those goals? 7) What do you think about the effectiveness of current methods and strategies toward those goals? (Probe with those not mentioned) a. Improved use of essential nutrition behaviors b. Increased diversity and consumption of nutritious foods c. Improved household and community management of conditions and diseases that exacerbate malnutrition d. Improved use of clean water, sanitation facilities, and hygiene behaviors 8) Do you feel a part of the process (planning, implementation, etc. of the program)? 9) How well FH DRC has managed the coordination/ relationship with your offices at different levels? 10) Are there any advantages to the Care Group program? If so, please describe them. 11) Are there any disadvantages to the Care Group program? If so, please describe them. 12) How has the Care Group program been integrated into the existing MOH health structure? How is this program perceived by the MOH? 13) Can you describe how FH has supported GM/P activities? How have FH interventions affected utilization and availing of clinic services? 42 FH/DRC Development Food Aid Program Mid-term Evaluation Report 14) In your opinion, what are some of the areas FH should improve in the second half of the program life? a. What areas are going well? Logistics Coordinator and Regional Commodity Coordinator Location: Interviewer: Date: Description of Group: Number in Group/Sex: Thank you for agreeing to talk with us and for your participation in this mid-term evaluation. Food for the Hungry wanted to conduct this internal evaluation with the objective of assessing progress of FH DRC’s DFAP to date and identify areas for improvements. We believe that you are in good position to tell us about your impression about the DFAP program. We anticipate the interview will last about an hour to an hour and half and appreciate any information you may provide. Your answers to the questions are completely confidential. 1. Start with an introduction of the size and types of commodities planned for the program and that received during the past two and half years. 2. Did you receive the entire planned amount for the period? If not why not? 3. How do you assess the overall performance of FH DRC’s Inland Transport Storage and Handling of commodities? 4. Have you experienced significant losses? a. What were the causes? b. What measures did you take or procedures did you put in place to avoid instances of losses being repeated in the future? 5. What is the ration size and mix a beneficiary receives per month? 6. Do you think the beneficiaries know the type and amount of commodities they are entitled to receive? a. Do they know who the donor is? If yes, how did you know they know? 7. What procedures do you follow to identify/ target the beneficiaries of food commodities? 8. Are beneficiaries required to contribute labor to be eligible for food distribution? 9. How do you address the differential needs of men and women in the distribution of commodities? a. Is there any special consideration for children and women especially pregnant and lactating women? 10. (Log.coord) What is your assessment of the support you have been getting from the region and head quarters in the management of commodities? Was it adequate? What do you want to see improved? 11. (Reg.Comm.coord) How do you assess the communication and relationship with FHDRC’s DFAP staff in the management of commodities? 43 FH/DRC Development Food Aid Program Mid-term Evaluation Report M&E Coordinators Location: Interviewer: Date: Description of Group: Number in Group/Sex: Thank you for agreeing to talk with us and for your participation in this mid-term evaluation. Food for the Hungry wanted to conduct this internal evaluation with the objective of assessing progress of FH DRC’s DFAP to date and identify areas for improvements. We believe that you are in good position to tell us about your impression about the DFAP program. We anticipate the interview will last about an hour to an hour and half and appreciate any information you may provide. Your answers to the questions are completely confidential. 1. Please explain your monitoring and evaluation system for the DFAP program. a. Are there forms/ sheets in place that are used for routine data collection as part of the monitoring? b. If there are forms, then are these forms designed to track indicators in the IPTT to manage the program and meet reporting needs? Please explain. c. How frequently do you collect the information from project sites? d. How do you aggregate information to generate data for reporting and program management? 2. What tools do you use in your M&E system? Can you please show them to me? a. Monthly, quarterly and annual reports i. What are the due dates for these reports from project sites to Bukavu and DC level? Are these due dates met? ii. Who is responsible to compile the report? Who feeds the information? b. How many mini-KPC surveys did you conduct? Does it cover all components? If not why not? c. Did you do pre- and post- training tests? d. Are there any Quality Improvement and Verification Checklists (QIVC)? Please explain and share the copy of the checklist you use. e. Do you conduct random spot checks during your monitoring? i. If yes, please give examples. What were your findings? 3. Do you have an up-to-date performance management plan and IPTT? (Check the copies) a. Are the indicators gender disaggregated? 4. Have you faced any challenges implementing your M&E plan? a. What were your challenges and how did you resolve them? 5. From your monitoring findings, how do you rate the overall performance of the program so far? a. The degree to which inputs were appropriately used b. The degree to which planned activities were carried out c. The quantity and quality of outputs that resulted from the inputs and activities 44 FH/DRC Development Food Aid Program Mid-term Evaluation Report Farmer Leaders/Farmer Leader Supervisors Location: Interviewer: Date: Description of Group: Number in Group/Sex: Thank you for agreeing to talk with us and for your participation in this mid-term evaluation. Food for the Hungry wanted to conduct this internal evaluation with the objective of assessing progress of FH DRC’s DFAP to date and identify areas for improvements. We believe that you are in good position to tell us about your impression about the DFAP program. We anticipate the interview will last about an hour to an hour and half and appreciate any information you may provide. Your answers to the questions are completely confidential. 1) Please describe your participation and experience with the DFAP program thus far. 2) What lessons/ techniques have you learned? Please list. 3) What lessons/techniques have you implemented? a. Why did you not implement all lessons? 4) Were there lessons or techniques that were not easy to understand? 5) What lessons or techniques were most beneficial? In what way? 6) What more things might be useful to learn? 7) How have communications been between you and your supervisor? 8) Have you encountered any difficulties in implementing new techniques? (Probe with the following areas) Personal – availability of inputs Extrinsic – community, legal, political, tribal 9) Please describe your experiences cascading lessons in terms of teaching them and farmers learning them. a. What are the successes of your efforts in this regard? b. What challenges did you face? How did you tackle the challenges? 10) Can you describe the interaction you have with the other sectors of the DFAP program? (i.e. health/MOH, agriculture, savings, WASH, CDCs). a. How can these linkages be improved? 45 FH/DRC Development Food Aid Program Mid-term Evaluation Report Leader Mothers/ Promoters Location: Interviewer: Date: Description of Group: Number in Group/Sex: Thank you for agreeing to talk with us and for your participation in this mid-term evaluation. Food for the Hungry wanted to conduct this internal evaluation with the objective of assessing progress of FH DRC’s DFAP to date and identify areas for improvements. We believe that you are in good position to tell us about your impression about the DFAP program. We anticipate the interview will last about an hour to an hour and half and appreciate any information you may provide. Your answers to the questions are completely confidential. 1) How were you selected to be a Leader Mother or Promoter? 2) Please describe your participation in the Care Group program thus far? 3) How long does it take you to complete your Leader Mother (or Promoter) asks each week? a. How long on average does it take you to reach one of your assigned groups? 3) What lessons/ techniques have you learned? Can you please list them? (ex.for promoters—facilitation techniques) 5) Were any of the lessons difficult to understand or teach to your group? a) Which lessons were the most boring? b) Which lessons generated the most conversation? 6) What lessons did you find the most beneficial? In what way? 7) What other lessons or things might be useful to learn or do you hope to learn in the future? 8) How have communications been between you and your supervisor? 9) How does your family feel about you participating as a Leader Mother? How does your family feel about you being a Promoter? How does your community perceive you? 10) How do you feel about yourself and your role as a Leader Mother (or Promoter)? 11) Have you encountered any difficulties in implementing new lessons or health behaviors? (Probe with topics below as needed) a. Personal – availability of inputs b. Extrinsic – community, legal, political, tribal (i.e. sometimes health facilities lack essential medicines and supplies) 12) Please describe your experiences teaching your group from the flipcharts. a. What are the successes of your efforts in this regard? 46 FH/DRC Development Food Aid Program Mid-term Evaluation Report b. What challenges did you face? How did you tackle the challenges? 13) What is the goal of the Care Group program? 14) What is your dream or hope for your community? 15) Promoters only: Can you describe the interaction you have with the other sectors of the DFAP program? (i.e. health/MOH, agriculture, savings, WASH, CDCs). a. How can we improve these linkages? 47 FH/DRC Development Food Aid Program Mid-term Evaluation Report Indirect Beneficiary Interviews Location: Interviewer: Date: Description of Group: Number in Group/Sex: Thank you for agreeing to talk with us and for your participation in this mid-term evaluation. Food for the Hungry wanted to conduct this internal evaluation with the objective of assessing progress of FH DRC’s DFAP to date and identify areas for improvements. We believe that you are in good position to tell us about your impression about the DFAP program. We anticipate the group discussion will last about an hour and half and appreciate your active participation. 1) Have you or anyone else in your families interacted with the program in any way? What do you know about the program? 2) Did you have an opportunity to join part of the program? a. Why did you choose not to participate ? (i.e. CGs, VSLAs, Farmer Groups) 3) What are some of the major activities implemented in your community and the benefits you, your families, and your community seen from the program? o From the health program?  Do you or any member of your family participate in care groups?  What benefits did you or any member of your family get from participating in care groups? o From the ag program? o From the water and sanitation program? o From the VSLA program? o From the Gender program? o Other aspects? 4) In your opinion, which activities were successfully accomplished in your community and which ones were not as successful? a. What do you think is the reason? 5) What additional activities would you like to see the DFAP program take on in the future? 6) Have household or community relations changed since the program began? 7) Are there any negative effects of the program in your HH or community? 48 Partner Questions (INERA, CIALCA, Pharmakina) Location: Interviewer: Date: Description of Group: Number in Group/Sex: Thank you for agreeing to talk with us and for your participation in this mid-term evaluation. Food for the Hungry wanted to conduct this internal evaluation with the objective of assessing progress of FH DRC’s DFAP to date and identify areas for improvements. We believe that you are in good position to tell us about your impression about the DFAP program. We anticipate the interview will last about an hour to an hour and half and appreciate any information you may provide. Your answers to the questions are completely confidential. 1) Are you aware of the DFAP program? In what ways do you participate in the program? 2) What are your general impressions of the program and its goals? 3) Are the program goals and interventions in line with the government policies and strategies? How? 4) What are your general impressions of the progress made toward achieving those goals? 5) What do you think about the effectiveness of current methods and strategies toward those goals? 1) Do you feel a part of the process (planning, implementation, etc. of the program)? 6) How well FH DRC has managed the coordination/ relationship with your offices at different levels? 7) What are your impressions of how well the program is managed? 8) In your opinion, what are some of the areas FH should improve in the second half of the program life? a. What areas are going well? 49 FOCUS GROUP DISCUSSION GUIDE/ QUALITATIVE INQUIRY Beneficiary Households Location: Interviewer: Date: Description of Group: Number in Group/Sex: Note: these are groups of 6-10 household heads who benefit from any component of the program. Thank you for agreeing to talk with us and for your participation in this mid-term evaluation. Food for the Hungry wanted to conduct this internal evaluation with the objective of assessing progress of FH DRC’s DFAP to date and identify areas for improvements. We believe that you are in good position to tell us about your impression about the DFAP program. We anticipate the group discussion will last about an hour and half and appreciate your active participation. 1) Have you or anyone else in your families interacted with the program in any way? What do you know about the program? 2) What are some of the major activities implemented in your community and the benefits you, your families, and your community seen from the program? o From the health program?  Do you or any member of your family participate in care groups?  What benefits did you or any member of your family get from participating in care groups? o From the ag program?  Did you receive any improved seeds? Did you notice any improvement in yield as a result of using the seeds? o From the water and sanitation program? o From the VSLA program? o From the Gender program? o Other aspects? 3) In your opinion, which activities were successfully accomplished in your community and which ones were not as successful? What do you think is the reason? 4) What additional activities would you like to see the DFAP program take on in the future? 5) Have household or community relations changed since the program began? 6) Are there any negative effects of the program in your HH or community? 7) How have relations and communications been between you and FH? a. What could be better? 8) Do men and women have equal opportunities to participate in the program activities? a. If no, why? 9) Who is the donor of the project? 10) Have you participated in any activities in exchange for rations? a. How have you found the rations? How have you found their ease of cooking? b. Were you familiar with the ration commodity types before? 11) Have there been reforestation activities conducted in your communities? How do you feel about the choice of trees? 50 Neighbor Group – Location: Interviewer: Date: Description of Group: Number in Group/Sex: Thank you for agreeing to talk with us and for your participation in this mid-term evaluation. Food for the Hungry wanted to conduct this internal evaluation with the objective of assessing progress of FH DRC’s DFAP to date and identify areas for improvements. We believe that you are in good position to tell us about your impression about the DFAP program. We anticipate the group discussion will last about an hour and half and appreciate your active participation. 1) When and how was your Neighbor Group formed? How many mothers (or women) participate in your Neighbor Group? 2) How many times each month do you meet in your Neighbor Group? a. Are there any challenges to meeting? If so, what are those challenges? 3) How often do you meet with your Leader Mother? a. Do you always meet with her in your group or do you sometimes meet with her individually? 4) What do you do during your Neighbor Group meetings? 5) Are there benefits/advantages to having Neighbor Groups in your community? (Social Capital) If so, what are they? (Probe for more responses, and try not to have any leading questions.) 6) Are there disadvantages to having Neighbor Groups in your community? If so, what are they? (Probe for more responses) 7) How do you find the flip charts (leader mothers)? Are they easy to understand or difficult to understand? Should they be made easier or are they too simplistic? Or are they just right? 8) Do most people approve of you participating in Neighbor Groups? a. How do your husbands’ feel about you participating in Neighbor Groups? 9) Do you feel that your CG is well supported by your Leader Mother? 10) What topics or lesson plans did you find most interesting or generated the most discussion? a. What topics did you find most boring? b. What topics would you like to learn more about? 11) Do you feel that your voice is heard when you have a problem? Either a problem with your role or with your family? 51 12) Have you made any changes in how you care for your child based on the lessons you have learned? 13) Tell me about any food demonstrations that you have seen. How did you feel about them? 14) When you visit the clinic, are you happy with the services provided to you? 15) What is the goal of the program? 16) What is your dream or hope (collective dream or hope) for your community? 52 Farmer Leader Groups (Members) Location: Interviewer: Date: Description of Group: Number in Group/Sex: (Note: these are 6-10 members of an FLG) Thank you for agreeing to talk with us and for your participation in this mid-term evaluation. Food for the Hungry wanted to conduct this internal evaluation with the objective of assessing progress of FH DRC’s DFAP to date and identify areas for improvements. We believe that you are in good position to tell us about your impression about the DFAP program. We anticipate the group discussion will last about an hour and half and appreciate your active participation. 1) When and how was your group formed? How many male and female members do you have? 2) How often do you meet with FH’s Agricultural promoters? 3) What have you learned by participating in the program? 4) What lessons/techniques have you implemented? 5) What, if any, techniques were not easy to understand? 6) What techniques were beneficial to you? a. Which ones were the most beneficial in terms of improved or increased output or labor/time saved? 7) What additional topics might be useful? 8) Were communications from the trainers and FH clear? 9) Did you have any difficulties in implementing new techniques? Personal – availability of inputs Extrinsic – community, legal, political, tribal 10) Were there any natural resource management activities such as reforestation and soil conservation structures, conducted in your area? a. What is your impression about the success of these activities? 11) Did you or any member of your community receive small livestock for reproduction? a. How do you assess the effectiveness of this particular activity in improving the lives of the community? 12) How do your family members feel about you participating in the farmer groups? 53 13) Do women feel like they have equal opportunity to participate in the farmer groups? If no, why? 54 Community Development Committees Location: Interviewer: Date: Description of Group: Number in Group/Sex: (Note: these are members of one CDC in the selected community/village) Thank you for agreeing to talk with us and for your participation in this mid-term evaluation. Food for the Hungry wanted to conduct this internal evaluation with the objective of assessing progress of FH DRC’s DFAP to date and identify areas for improvements. We believe that you are in good position to tell us about your impression about the DFAP program. We anticipate the group discussion will last about an hour and half and appreciate your active participation. 1. Who initiated the formation of the CDC? Why? When? 2. Please explain the process and steps taken to form the CDC and its main objectives. 3. What do you know about the FH DRC’s DFAP program? 4. How do you describe your committee’s relationship with FH DRC in the process of the DFAP program planning and implementation? 5. In the past two years, how many times did FH staff visit your community? What did you discuss in your meetings with FH staff? 6. Did you participate in any training program organized by FH? a. What are the types of training? b. Was it helpful? How? 7. Can you describe your role as a development committee in your community? 8. What do you think are your major achievements so far? a. Please compare the situation before and after the CDC formation. 9. Do you have any relationship with other community groups such as FLGs, CGs and Water Committees? How is your impression of these relationships? 10. What challenges did you face serving your communities? 11. Do you evaluate the performance of the committee? a. How often? b. Can you cite some of your evaluation findings and recommended actions? 12. Does your CDC include an emergency preparedness plan? How do you measure how ready the community is for an emergency? 55 Village Saving and Loan Associations Location: Interviewer: Date: Description of Group: Number in Group/Sex: (Note: these are members of a selected VSLA) Thank you for agreeing to talk with us and for your participation in this mid-term evaluation. Food for the Hungry wanted to conduct this internal evaluation with the objective of assessing progress of FH DRC’s DFAP to date and identify areas for improvements. We believe that you are in good position to tell us about your impression about the DFAP program. We anticipate the group discussion will last about an hour and half and appreciate your active participation. 1. Tell us a little bit about your association. a. Why was it formed? b. When and how it was formed? c. How many members do you have (men, women, wealth status, etc)? d. How much do you save and how frequently? 2. How many cycles did your VSLA complete? Did you drop members in different cycles? Why? 3. What support did you get from FH DRC? 4. Are you satisfied with the level of support you have been getting? 5. How do you and other members of the VSLA use their savings? a. On what activities do you use the majority of the savings? Consumption, social activities, productive activities? b. What do you want to see changed in the way savings are used in the future? 6. Are you linked with any MFI as a VSLA? a. If yes, what did you benefit from the MFI? b. Are there any challenges working with MFIs? 7. Are there any success stories you can tell us about VSLA members? 8. What are the major constraints you have as an association? 9. What is your plan in the future as an association? 10. What do you like to see changed in the functioning of VSLAs in the future? 11. Have you obtained credit anywhere else? 12. Have other members of the community approached you about joining your group or starting their own? 56 WatSan Committee (members of one WatSan committee) Location: Interviewer: Date: Description of Group: Number in Group/Sex: Thank you for agreeing to talk with us and for your participation in this mid-term evaluation. Food for the Hungry wanted to conduct this internal evaluation with the objective of assessing progress of FH DRC’s DFAP to date and identify areas for improvements. We believe that you are in good position to tell us about your impression about the DFAP program. We anticipate the group discussion will last about an hour and half and appreciate your active participation. 1. Please describe the situation of the water scheme. a. When was the water scheme constructed? b. Describe the type and capacity of water scheme constructed. c. How many people (women and men) benefit? d. Is the water scheme currently operational? e. Have you done any maintenance since the construction was finalized? f. What was the source of water before the construction of the water point? 2. Please tell us a little about the WatSan committee. a. When was it formed? b. How many men and women members do you have? c. What is your role? d. What did you accomplish so far? 3. Did your community participate in the construction of the scheme? a. Contribute labor/ cash/ materials? b. Participated in site selection? If not, why not? 4. What is your overall impression of the construction of the water scheme? a. Do you think the water scheme is constructed in the right place in your village? b. Are you happy with the yield of the water scheme? 5. What mechanisms do you have to manage the water schemes? a. Do you collect user fees or any other contribution? b. If yes, what do you plan to do with the money? c. Whom do you contact when you need any support related to the water scheme? 6. To whom does the WatSan committee report? 7. What impact did the water scheme bring about in the lives of your (women and men) community members? 8. What do the members of your community do to sustainably use the water scheme? 1 | FOOD FOR THE HUNGRY - DRC MID - TERM EVALUATION REPORT Search for Common Ground (SFCG) Program Director and Project Staff and FH staff FGD Questions Location: Interviewer: Date: Description of Group: Number in Group/Sex: (Note: the PD and project staff interviews will be conducted separately using the same questions) Thank you for agreeing to talk with us and for your participation in this mid-term evaluation. Food for the Hungry wanted to conduct this internal evaluation with the objective of assessing progress of FH DRC’s DFAP to date and identify areas for improvements. We believe that you are in good position to tell us about your impression about the DFAP program. We anticipate the interview will last about an hour to an hour and half and appreciate any information you may provide. Your answers to the questions are completely confidential. 12. Please describe your organization’s activities in DRC and its role in the DFAP program. 13. Did you use studies to better inform your planning of the gender component in the program? a. What kind of study? What were the major findings? b. Was the findings used to inform the program implementation? How? 14. How do you coordinate your activities with FH DRC at Bukavu and the project sites? a. Do you have regular coordination meetings? How frequent? b. What do you discuss in those meetings? 15. How would you describe the partnership with FH? a. Has it been effective? b. Are there any improvement areas? 16. Do you think FH/SFCG’s gender activities are on track to bring about the planned results? a. Could you give examples? b. Are there activities that have delayed or cancelled? What do you think is the reason? 17. In your opinion, which activities were successful? Why? Which activities were not successful? Why? 18. What do you think are the major achievements of the program with respect to improving gender relations? 19. In your opinion, has the gender portion of the program impacted women’s or other community member’s food security? How so? Why or why not? 20. What new activities and approaches do you want to see included in the future? Why? 21. Are there any activities and approaches that you want to see revised or dropped? Why? 22. What were the challenges you faced implementing the gender component of the program? How did you approach the challenges? 23. To what extent do you think that the results of the gender interventions are sustainable?