1 RESPONSIVE ECONOMIC ASSISTANCE TO CONFLICT-AFFECTED HOUSEHOLDS (REACH) 4 FINAL EVALUATIONREPORT 1 Table of Contents LIST OF ACRONYMS..............................................................................................................................................................2 EXECUTIVE SUMMARY..........................................................................................................................................................3 SECTION 1 : INTRODUCTION............................................................................................................................................5 SECTION 2: METHODOLOGY.............................................................................................................................................6 1.1 Data Collection Approach............................................................................................................................................6 1.2 Sampling of Respondents...............................................................................................................................................6 1.3. Data Collection and Analysis.......................................................................................................................................9 1.3.1 Desk Review of Documents..................................................................................................................................9 1.3.2 Questionnaires.......................................................................................................................................................9 1.3.3 Key Informants’ Interviews (KIIs).........................................................................................................................9 1.3.4 Focus Group Discussions (FGDs).......................................................................................................................10 1.3.5 Data Analysis........................................................................................................................................................10 LIMITATIONS.........................................................................................................................................................................11 SECTION 3: RESULTS/FINDINGS.......................................................................................................................................12 Question 1 -Achievements...............................................................................................................................................12 Achievement of Indicator Targets....................................................................................................................................15 Question 2 -Diversity of Beneficiaries............................................................................................................................18 Question 3 - Intended and Unintended Outcomes........................................................................................................20 Question 4 – Household Food Security and Nutrition Outcomes...............................................................................22 Question 5 – Collaborations.............................................................................................................................................24 Question Six -Leveraging and Layering ............................................................................................................................24 SECTION 3: SYNTHESIS, RECOMMENDATIONS AND LESSONS LEARNED.............................................................27 3.1 Synthesis and Conclusions..........................................................................................................................................27 3.2 Recommendations.......................................................................................................................................................28 3.3 Lessons Learned...........................................................................................................................................................29 SECTION 4: CONFLICTS OF INTEREST............................................................................................................................29 ANNEXES...............................................................................................................................................................................30 ANNEX 1: TOR FOR THE EVALUATION.....................................................................................................................30 ANNEX 2: LIST OF RESPONDENTS..............................................................................................................................38 ANNEX 3: DATA COLLECTION TOOLS.................................................................................................................39 Annex 3.1: Key Informant Interview Guide (with Project Beneficiaries)...........................................................................39 Annex 3.2: Key Informant Interview Guide (with Project Implementers).........................................................................42 Annex 3.3: Key Informant Interview Guide (with Market Vendors)..................................................................................46 Annex 3.4: Key Informant Interview Guide (with Leaders)................................................................................................49 Annex 3.5: Focus Group Discussion Guide (with Project Beneficiaries)...........................................................................52 ANNEX 4: PHOTOS.........................................................................................................................................................56 2 LIST OF ACRONYMS BCC Behavior Change Communication BHA Bureau for Humanitarian Assistance CRC Community Relief Committee FCS Food Consumption Score FES Fuel-efficient stoves FGD Focus Group Discussion FSS Food Security Sector HDI Human Development Index HDMI Household Decision-Making Index HH Household IDPs Internally Displaced Persons IYCF Infant and Young Child Feeding KAP Knowledge, Attitudes, and Practices KII Key Informant Interview LGA Local Government Area MAD Minimum Acceptable Diet MAM Moderate Acute Malnutrition MDD-W Minimum Dietary Diversity for Women MUAC Mid Upper Arm Circumference MSG Mother Support Groups NGO Non-government Organization NE North East PDM Post Distribution Monitoring PLWD People Living with Disabilities rCSI Reduced Coping Strategy Index REACH 4 Responsive Economic Assistance to Conflict-affected Households 4 SAM Severe Acute Malnutrition TOR Terms of Reference UN United Nations USAID United States Agency for International Development USG United States Government WASH Water, Sanitation, and Hygiene 3 EXECUTIVESUMMARY Responsive Economic Assistance for Conflict Affected Households (REACH) 4 is a Food for Peace/Bureau for Humanitarian Assistance (BHA)-funded emergency food security program that began in February 2020 and continued through March 2021. The ultimate goal of REACH 4 was to enable vulnerable, conflict-affected households in Damboa have improved nutrition and food security during the program’s lifetime. As a result of the increasing need of humanitarian assistance and to meet these urgent needs, Mercy Corps proposed to build upon the previous iterations of BHA food security responses in North East (NE) Nigeria with a multi-pronged approach that was expected to increase the access to both staple and nutrient-dense food through local markets while also shaping behaviors to support optimal nutrition. The REACH 4 program was designed to reach the most vulnerable households in Damboa with food assistance to enable them meet their essential food needs by engaging in multiple market systems; and promoting sustainability and resilience by investing in nutrition and production capacities. Using this approach, REACH 4 addressed both the immediate food security needs, as well as commenced the work of taking Damboa along the path from food dependency to food self-reliance. The program targeted 13,512 households and 94,584 indirect program participants. The endline project was aimed at evaluating the extent to which the project goals have been met and to identify best practices and lessons learned while implementing the program KEY FINDINGS 1. REACH 4 Achievements: Adherence to planned schedules, intended goals, purposes and outcomes. Despite the impact of COVID-19 which delayed the program implementation for about 4-5 months, other interventions within the REACH 4 Program were implemented according to the schedule. There was also significant improvement in the nutrition and food security indicators as against the baseline. The percentage of households with acceptable Food Consumption Score (FCS) improved from 13.98% to 20.57%. Also, the number of households experiencing severe hunger reduced from the baseline value of 6.35% to 0.33% at endline. There was a significant change in the Reduced Coping Strategy Index (rCSI) from a baseline value of 16.63% to 9.43% at endline. Also, the percentage of children (6-59) months with MUAC score above 12.5cm also increased by 8% from the baseline values. The percentage of women of reproductive age (15-49 years) who consume Minimum Dietary Diversity (MDD￾W) increased by ~24% (31.11% - 59.96%) from the baseline. Although, the project did not meet the 94% target for children under 6 months who are breastfed, there was a noticeable increase in the percentage of breastfed children under 6 months by ~24% (63.5% - 87.1%). The percentage of people targeted by the nutrition and hygiene promotion program who know at least three (3) of the five (5) critical times to wash handssurpassed the 70% baseline target by 14%. The REACH 4 program contributed to an improvement in gender equality in the target communities, with a significant increment of 47.69% in the indicator ‘Household decision making index’ which shows the level of women’s involvement in decision-making on key issues in the households. Also, the beneficiaries of the Fuel-Efficient Stoves (FES) reported a slight reduction in their expenditure for energy from an average of 912 naira to about 894 Naira. 2. Effectiveness and Efficiency of Interventions: The REACH 4 program was effective and efficient in the way resources were distributed and utilized. The differential vulnerabilities were considered in designing and implementing the program. Women and child headed households were adequately included in the program as well as people living with disabilities (PLWD). However, in the implementation of the program, ease of access and convenience were not appropriately considered for the PLWD. 4 3. Intended and Unintended Food and Nutrition Security Outcome: The intended outcomes for REACH 4 were met and this is evident through the significant improvements in the indicators for nutrition and food security such as the FCS, HHS, and rCSI. The COVID-19 pandemic also provided opportunity to intensify campaign on outcomes such as the safe hygiene practices. The percentage of people targeted by the nutrition and hygiene promotion program who are now aware of at least three (3) of the five (5) critical times to wash hand increased significantly from 7.18% at baseline to about 84.38%. The program has also led to some unintended outcomes, such as reduction in crime rate and unemployment which were not the primary focus of the intervention. The use of local vendors from the community stimulated the market dynamics in the community and created some indirect employment opportunities. The beneficiaries of the poultry farm assistance; who received Noiler birds, have moved beyond providing nutritional supplements for their households and now supply egg and meat to the local market for their economic sustenance. 4. Contribution of specific interventions in improving household food security and nutrition outcome The nutrition promotion campaigns to sensitize beneficiaries on the sources and benefits of nutritious foods were effective in improving the choice of food items redeemed by the beneficiaries. This was evident in the percentage of women of reproductive age (15-49 years) who are consuming minimum dietary diversity (MDD), which increased from 31.11% at baseline to 59.96%. The Mother Support Groups were a critical component in reinforcing the messaging during the advocacies and has helped to further spread valuable information in hygiene and need for nutritious food among members of the community. 5. Collaborations The stakeholders, especially the government and beneficiaries showed satisfaction with the implementation of REACH 4; this was demonstrated by the government’s willingness to provide farmland to be used as demonstration plots for the backyard garden initiative. The government also provided consultants who trained beneficiaries on some livelihood initiatives. There is a clamor from stakeholders; government, direct and indirect beneficiaries for the implementation of similar activities in the community due to the perceived success of the REACH 4 program. 6. Leverage and Layering: There was considerable collaboration with USG and non-USG funded Implementing Partners(IPs) in the target communities especially in the design of the intervention to avoid overlapping of activities. The Program was able to leverage on government resourcessuch as officers who assisted beneficiaries in setting up their poultries and land for backyard garden for the beneficiaries. The Program also relied on the security provided by the military personnel from the Nigerian government to ensure the safety and security of commodities and personnel delivering and distributing them. 5 SECTION 1: INTRODUCTION The REACH 4 program was a built on the learning gathered from implementing previous REACH programs (1, 2 and 3). The aim of the program was to enable 13,512 conflict-affected households in Damboa have improved food and nutrition security during the program’s lifetime of 12 months. The program was targeted at meeting the essential food security needs of the most vulnerable households; IDPs, Returnees and the Host community while contributing to a foundation for resilience in Damboa LGA. To achieve this, Mercy Corps used a multi-pronged approach to increase access to both staple and nutrient-dense food through local markets while also shaping behavior change to support optimal nutrition. The program accomplished this objective through interventions in food security, nutrition, and household energy (fuel-efficient stoves (FES): • Food Security: REACH 4 ensured that 13,512 conflict-affected households had access to nutritionally adequate and diverse foods via redeemable vouchers for nutritious food. During the REACH 4, Mercy Corps explored expanding food options to potentially encourage households to purchase vegetables, fruits, animal-source foods and other fresh foods that are locally available and culturally appropriate — in addition to the staple food items that are already accessed through market vendors. Expansion of food options will have the added benefit of supporting local producers, while also ensuring these households consume essential micronutrients from dietary diversity. • Nutrition: REACH 4 conducted nutrition/hygiene promotion activities based on the outcome of the needs assessment. The program responded to negative coping mechanisms, promoted proper hygiene practices and Infant and young Child Feeding (IYCF), and trained women and men in child health and nutrition. This led to behavioral change among groups of women and adolescent girls who were within reproductive ages. The nutrition component of REACH 4 also created linkages that drove the demand for nutritious foods and nutrition and health services. • Fuel Efficient Stove (FES): REACH 4 worked to ensure that households can prepare staple and fresh foods safely and hygienically. The program targeted a small pilot of beneficiaries to support increased access to fuel efficient stoves (FES) for vulnerable households. This final evaluation presents the performance of the program targets by: • Assessing the achievements of the program in relation to the goal, objectives, results and targets; • Evaluating the program’s effects on the local markets, and how it affected certain groups of interest (women, men, youth, boys, and girls); • Evaluating the effectiveness and relevance of the modality, transfers, and complementary interventions to achieve program outcomes; • Assessing the sustainability of its results; and • Identifying best practices, lessons learned, strengths and challenges in program design, including log frame, and implementation of the program. The result of the endline provides an in-depth understanding of the program activities which can serve as a reference document for subsequent programming and funding. It also serves as final report to the program funder; the United States Agency for International Development Office for Bureau for Humanitarian Assistance (USAID BHA) and other stakeholders. 6 SECTION 2: METHODOLOGY 1.1 Data Collection Approach The endline used a mixed-method approach in which more than one method for data collection was used to answer the evaluation questions. The methodological approach was predominantly quantitative to measure the key indicators. The qualitative data collection tools (KII and FGD) employed open-ended questions to give room for respondents to express their views on the project without restrictions within the time frame. The data collectors carefully probed to ensure clarity and precision of information given. Data was collected from beneficiaries and stakeholders across 8 communities where REACH 4 was implemented; Abatchari, Aburi, Alkaleri, Damboa Central, Kachalaburari Gana, Kachalaburari Kura, Shuwari, and Wuyaram. The endline conducted a total of 79 KIIs among randomly selected beneficiaries and stakeholders and held 3 focus group discussions using a qualitative interview guide. Target Audience Method of Data Collection Vulnerable Households: Men, Women and Adolescent 40 KII (5 respondents from each community) Beneficiaries Living with Disability 3 KII Market Vendors 8 KII (1 from each community) Leaders (Community, Religious, Household Leaders, Local Government Officials, and CRC members) 22 KII Implementing Staff in Nutrition (e.g., program trainers/ nutrition promoters 2 KII Implementing Staff in Food Security 2 KII Implementing Staff in Fuel Efficient Stove 2 KII Focus Group Discussion 3 FGD sessions consisting of 6 - 10 individuals A: Head of Households B: Women of Reproductive Age (15-45); including pregnant women and lactating mothers of children under 5 years C: Individuals responsible for household Cooking and caregivers 1.2 Sampling of Respondents The survey used multi-stage random sampling to select respondents from vulnerable households which had benefitted from the Program. In the first stage, the 8 communities in Damboa LGA were each divided into clusters of streets where the REACH 4 program was implemented. The streets were selected using simple random sampling technique while preference was streets densely populated by IDPs. In the second stage, beneficiary households in the selected street were selected using circular systematic random sampling. The household were selected at a preset order of five (5) after the initial household selected randomly. The sample size for the quantitative data collected differ based on the program indicator measured and it was determined using the formula for calculating sample size. ninitial = Dest[ 𝑍𝑍1−𝛼𝛼�2𝑃𝑃(1−𝑃𝑃)+𝑍𝑍1−𝛽𝛽�𝑃𝑃1,𝑒𝑒𝑒𝑒𝑒𝑒(1−𝑃𝑃1,𝑒𝑒𝑒𝑒𝑒𝑒)+(𝑃𝑃2,𝑒𝑒𝑒𝑒𝑒𝑒(1−𝑃𝑃2,𝑒𝑒𝑒𝑒𝑒𝑒) 𝛿𝛿 ]2 Where ninitial = is the initial sample size required by the surveys for each of the two time points 𝛿𝛿= 𝑃𝑃1,𝑒𝑒𝑒𝑒𝑒𝑒− 𝑃𝑃2,𝑒𝑒𝑒𝑒𝑒𝑒 = minimum effect size to be achieved over the time frame specified by the two surveys 7 𝑃𝑃1,𝑒𝑒𝑒𝑒𝑒𝑒= represents a survey estimate of the true population proportion 𝑃𝑃1 at baseline [If such an estimate is not available from prior surveys, please use 0.5] 𝑃𝑃2,𝑒𝑒𝑒𝑒𝑒𝑒 = represents a survey estimate of the true population proportion 𝑃𝑃2, at endline 𝑃𝑃=𝑃𝑃1,𝑒𝑒𝑒𝑒𝑒𝑒 + 𝑃𝑃2,𝑒𝑒𝑒𝑒𝑒𝑒 2 𝑍𝑍1−𝛼𝛼= is the value from the normal probability distribution corresponding to a confidence level 1−𝛽𝛽. For 1−𝛽𝛽=0.95, the corresponding value is 𝑍𝑍0.95= 1.64. 𝑍𝑍1−𝛽𝛽 is the value from the normal probability distribution corresponding to a power level of 1−𝛽𝛽. For 1−𝛽𝛽 = 0.80, the corresponding value is 𝑍𝑍0.80= =0.84. 𝐷𝐷𝑒𝑒𝑒𝑒𝑒𝑒= is the estimated design effect (DEFF) of the survey. 8 Sample Size for Key Program Indicators Sector Emergency Food assistance Emergency Food assistance Emergency Food assistance Complementary Food Security Programming Complementar y Food Security Programming Complementar y Food Security Programming Complementary Food Security Programming Complementar y Food Security Programming Indicator Food Consumption Score - Percent of households with poor, borderline and acceptable food consumption score Reduced Coping Strategy Index (rCSI) - Mean Household Hunger Score (HHS) - Prevalence of households with moderate or severe hunger Minimum Dietary Diversity for Women (MDD-W) & Minimum Acceptable Diet (MAD) for 6- 23months Percent of people targeted by the hygiene promotion program who know at least three (3) of the five (5) critical times to wash hands Percentage of children under 5 with children with MUAC Score above 13.5cm (135mm) Prevalence of exclusive breastfeeding of children under six months of age Percentage of community members who report enhanced awareness and demand about product availability and benefits Indicator Type Proportion Mean Proportion Proportion Proportion Proportion Proportion Proportion P1,est 50% (0.5) 50% (0.5) 50% (0.5) 50% (0.5) 50% (0.5) 9% (0.09) 50% (0.5) 50% (0.5) P2,est 40% (0.4) 40% (0.4) 40% (0.4) 40% (0.4) 40% (0.4) 40% (0.04) 40% (0.4) 40% (0.4) Z1-α 95% (1.64) 95% (1.64) 95% (1.64) 95% (1.64) 95% (1.64) 95% (1.64) 95% (1.64) 95% (1.64) Z1-β 80% (0.84) 80% (0.84) 80% (0.84) 80% (0.84) 80% (0.84) 80% (0.84) 80% (0.84) 80% (0.84) Dest 2 2 2 2 2 2 2 2 ninitial 305 305 305 357 357 600 357 305 Non-response adjustment 10% 10% 10% 10% 10% 10% 10% 10% Average household Size 7 7 7 7 7 7 7 7 Population 13,512 13,512 13,512 13,512 13,512 13,512 13,512 13,512 Final sample size 678 678 678 397 397 667 397 678 9 1.3. Data Collection and Analysis 1.3.1 Desk Review of Documents The team started data collection by reviewing relevant documents and reports used for implementation to obtain routine Program information and better understand the context under which implemented occurred. The review of relevant literature guided the team to identify the key stakeholders to be interviewed, the design of the data collection tools and helped the team gain a better understanding of the Program which in-turn informed the approach to the analysis. The document review process also helped to identify existing knowledge gaps in the Program and contribute to the formulation of recommendations to be made to Mercy Corps Nigeria. Examples of relevant documents include REACH 4 Monitoring and Evaluation Plan, baseline assessment reports and the Program assessment of REACH 3. 1.3.2 Questionnaires Quantitative data was collected using the mobile data application (ODK) and it was hosted on the Mercy Corps ONA server. The data collection involved the administration of questionnaires to individual program participants across the eight (8) communities in Damboa LGA, where REACH 4 was implemented. Questionnaires were used as a primary data collection tools for quantitative data during the evaluation. The questionnaires were developed using a structured and semi-structured set of questions. Designed to get a sense of behavior, perception, perception, attitude, and practice of the program participants with concentrated precision. 1.3.3 Key Informants’ Interviews (KIIs) The key informants interviewed during the endline were the key stakeholders on the REACH 4 Program, which includes the Program team (identified by the Mercy Corps M&E Team) as well as the community and religious leaders identified in the IDP camps and community. The purpose of conducting KIIs with the key stakeholders was to allow the team to gather richer information from a wide range of ‘‘people who had relevant knowledge and insight’’1 on the Program. The key stakeholders, with their particular knowledge and understanding of the Program, were supposed to provide insight on constraints and opportunities as it relates to the program as well as provide relevant recommendation to Mercy Corps. The principles of ‘‘Do No Harm’’ and ‘‘Do Nothing About Them Without Them’’2 guided our engagement with all the participants and stakeholders. This enabled the team to take measures to ensure that its approach to data gathering did not put participants or stakeholders at increased risk of harm and consulted with them to understand their needs and priorities. The following were put in place to ensure that the KII was successful: i. The team sent earlier communication to each key informant to seek their consent for the interview as well as schedule a convenient time for the interview(s) at each of the IDP camp/ households in the community. ii. The data collection team visited each camp comprised four members: Two data collectors assigned (one conducting the interview while the other takes notes and audio-records) and an Assistant 1 https://www.betterevaluation.org/en/evaluation-options/key_informant_interviews 2 https://www.youthpower.org/youth-drg-toolkit-inclusive-development 10 Researcher to ensure quality assurance. The time used for each interview was between 30 minutes and 90 minutes. iii. The team sought signed informed consent from every respondent before the commencement of every interview. The actual data collection involved the use of a semi-structured questionnaire designed along the line of the key informant’s focus with respect to the evaluation questions. iv. The data collectors documented the responses from the KIIs via note taking and audio-recording, transcribed and analyzed the data collected to produce a summary of the transcript, highlighting the key information that would contribute in achieving the objectives of the analysis. These were sent daily to the data analyst for coding of responses into a structured data set for a comprehensive analysis and interpretation using NVIVO 12- a qualitative analysis tool used for coding interview transcripts. Findings that require further investigations and/or clarifications were also identified for follow-up interviews. 1.3.4 Focus Group Discussions (FGDs) FGDs with the various group representatives provided a good opportunity to get at the common/community/institutional views/opinions and perceptions,3 relating to the focus of the analysis. The planning and conducting of FGDsinvolved a careful selection of participants who were beneficiaries. Three FGDs were conducted which included 6-10 individuals each. The FGD participants were selected from these three categories; head of households, women of reproductive Age (15-45); including pregnant women and lactating mothers of children under 5 years and individualsresponsible for household cooking and caregivers. The facilitators for the sessions also adopted a consultative approach that ensured the participants freely contributed their views, needs and priorities. The following procedures were undertaken in conducting the FGDs: i. The team sent earlier communication to each prospective FGD participant to inform them of the purpose of the FGD, seek their consent and affirm the earlier discussed date, time and the focus of the discussion based on the objective of the analysis. To limit bias and improve confidentiality of the FGDs, the facilitators avoided delving into personal opinions and views in conducting the discussions. ii. Roles: Two enumerators facilitated the FGDs; a moderator, supported by a colleague who took charge of the recording and note taking. A Research Assistant helped to maintain meeting protocols and ensure adherence to time. The number of participants for each FGD was between six to ten. iii. Each FGD lasted between 90 and 120 minutes. The moderator used a semi-structured questionnaire to guide the group and ensure they do not go out of the focus of the discussion. The data from the FGDs were analyzed in using similar methods to those used for the analysis of data from the KIIs. 1.3.5 Data Analysis 1.3.5.1 QUANTITATIVE DATA ANALYSIS The quantitative data collected was analyzed using STATA 15. The analysis involved descriptive statistics; which reports data in percentages and mean values and the findings were presented with charts and tables. 3 www.evalued,bcu.ac.uk/tutorial/4b.htm 11 Secondary data from the baseline study were also analyzed to determine the extent of change that has occurred on the program indicators at endline. 1.3.5.2 Qualitative Data Analysis The KIIs and FDGs were recorded and transcribed. The interviews carried out in local language were translated into English for ease of analysis. The audio recordings were used to later clean and verify accuracy of transcripts.The Qualitative data from the KIIs and FGDs were coded thematically using NVivo 14 to address each evaluation question. Prior to analysis, the team created a codebook internally to ensure relevance of the coding structure to the data and consistent code application during analysis. LIMITATIONS A major limitation of the study was the language barrier in conducting the interviews. The data collection tools were scripted in English, but the questions had to be translated in the local languages of Kanuri, Marghi and Hausa to be administered to the respondents. Their responses were recorded in these languages before being translated into English for analysis. These issues present obvious challenges of possible losses in meaning during translation; however, the data collectors were properly trained through this process and the evaluation team included pairs of data collectors who could speak these languages for each data collection event (KIIs, FGDs). 12 SECTION 3: RESULTS/FINDINGS QUESTION 1 - ACHIEVEMENTS To what extent have the interventions adhered to planned implementation - schedules, participant targeting, resource transfer composition and quantities, inputs and service delivery, and outputs - and achieved intended goals, purposes, and outcomes? Generally, the interventions were delivered according to the implementation plan especially in terms of participant and beneficiary targeting, composition of resource transfer, quantities, inputs, and service delivery etc. The evaluation team was able to confirm from the beneficiaries that all the products and commodities expected from the intervention were all shared. However, the major drawback from the implementation plan resulted from the impact of the COVID-19 pandemic in 2020 and not having the full complement of staff on ground for the start of the Program, this delayed the speed of implementation and affected the reach since a higher number of staff at the onset of the program would have ensured better coverage at the minimum. So, almost a month passed before all the necessary staff were brought in. As you know, REACH 4 is a 12 months program, All the necessary staff that were supposed to be on board to implement the program, were not available at onset. Program Staff first of all, COVID-19, there was a plan for registration. But we're not able to do that house-to-house registration as planned due to COVID-19. There are restriction movement, restriction, and other preventive measure that we need to do with, to curtail the spread of COVID-19. Even though as at then it wasn't registered that anybody from Damboa got COVID-19 but still we had to like follow suit. So, to know, to curtail the spread. it took us extra one month, which is to say around February/ March, we have not done the registration. We had to discuss on the approach to do. We now did a verification of the existing participants that is for REACH 3, then we got a referral for new participants. So also registered them using that same tool and applied those criteria that is one. So COVID-19 caused a slight delay in it, aside from delaying the registration. There is also the policy that says you cannot gather more than 20 people in a gathering, because you have to apply for social distancing and the rest. Program Staff Appropriateness of beneficiary targeting: The beneficiary targeting for the intervention was both appropriate and included both men and women. From a total of 13,512 households captured in the REACH 4 intervention, 35% (4,779) were female headed households and 65% (8,734) were male headed households. Child headed households were also included as beneficiaries (0.23%, 31 HOH) and the aged were also captured (4.55%, 615 HOH). The beneficiaries selected were the vulnerable populations in the target communities including people living with disabilities. It is however important to point out that due to the COVID-19 pandemic, the implementation team did not have enough time to conduct a fresh mapping of target beneficiaries, and hence relied on the beneficiaries inherited from REACH 3 13 “We had plans of registering 13,512 households, but we could not do that. So, we just carried over our 13,512 households from the REACH-3 program to REACH 4. There was no new registration due to the COVID-19 pandemic. But what we did rather was verification, so that we could have the right participants or other components of the program. Appropriateness and effectiveness of intervention to the beneficiaries based on the nature of their vulnerabilities: The interventions under REACH-4 were appropriate as they were able to address critical challenges that the target communities were experiencing. Specifically, the intervention was able to address hunger and economic sustenance through the distribution of commodities (grains, vegetables and meat) and introduction of livelihood components, such as teaching participants on backyard gardens for growing food and vegetables (about 945 beneficiaries), poultry farming (about 240 beneficiaries). These initiatives from REACH-4 were not only timely but also helped to assuage the suffering of the communities by helping to improve the nutrition of the target population and plant seeds of economic sustenance as the beneficiaries are now equipped on how to grow their own food without relying entirely on aid from organizations and government. Beyond training the people on poultry farming and backyard gardens, the livelihood component also provided direct link between the beneficiaries and suppliers to ensure continued access to information, vaccines, and even external markets. For example, let me use the birds we do not only train them, but also, we link them to the suppliers of the Noilerbirds. Now, we have certified poultry farmers over here and they are trained by Amo farms. So, Amo farms come down to train these major poultry suppliers down here. So now our intervention has made participants have a link to sourcing the adequate vaccines and the right vaccines. And they having these birds come at the right condition. So, you see now through our intervention, we've provided that enabling environment whereby participants can reach out to Amo, or the mother unit over here to get these birds. Program Staff The distribution of the food and fuel-efficient stoves (FES) (about 5000 beneficiaries), which requires less firewood to cook, inherently meant that members of the community were less exposed to attacks from terrorists by limiting the number of times they would venture out to get supplies such as firewood, food and hunting for animals. Well, this local stove they gave us has really helped us because we usually suffer to secure firewood but the introduction of these local stoves have alleviated that suffering. We feel at ease using these stoves. It is economic friendly and easy to use. Then, we had to go to the bush to get firewood or have to buy but now we thank God. Example, one can go and buy firewood of N10,000 but with the local stove, one could just buy firewood of N5,000 and save the other N5,000 14 Effectiveness of targeting approach in achieving the program goal? The implementation team adopted an effective targeting approach in REACH-4 by using the Community Relief Committees (CRC), which comprised of respected and influential women, religious and youth opinion leaders in the target populations, to identify and select vulnerable beneficiaries. This approach was quite important in that it helped to identify the most appropriate beneficiaries for the program and prevented attacks and disaffection from community members who were not selected as part of the program. Factors that promoted or inhibited adherence to plans and targets and how they were managed The major factors that affected the implementation negatively includes the impact of COVID-19 and the delay in identifying suitable staff for the implementation of the intervention. To minimize this, the team used the period of the COVID-19 lockdown to identify, recruit and train the required implementation staff for the Program to ensure they were ready to be deployed to the field as soon as the lockdown was eased. On the other hand, familiarity of both the CRC members and some implementation staff from REACH-3 helped to improve acceptance by the communities and identify the most suitable beneficiaries for the initiatives. 15 ACHIEVEMENTOFINDICATOR TARGETS S/ N Indicator Name Baseline Value Target End line Value Remarks 1 Food consumption score (FCS) - % of Households with poor, borderline, and acceptable FCS Acceptable=13.98% borderline=46.00% poor=40.02% 50% - Acceptable 50% - Borderline 0% - Poor 20.57% - Acceptable 45.65% - Borderline 33.77% - Poor There was a noticeably improved score for the percentage of households with acceptable FCS during the intervention, from about 14% to 21%. Most of the gains here came from a reduction in the number of households with poor FCS at baseline; the score for borderline FCS remained unchanged. 2 Reduced Coping Strategy Index (rCSI) – Mean 16.63 (mean value, 0 to 56 as maximum scores) 14.967 Weighted Score (mean value) 9.43 (mean value, 0 to 40 as maximum scores) The project surpassed the target for this indicator significantly, achieving a reduction of more than 7 points from the baseline value. This indicates an increased ability of the beneficiaries to manage the realities of their environment in terms of both nutrition and sustenance. 3 Household Hunger Score (HHS) – prevalence of HHS with moderate or severe hunger Little to no hunger=23.83% moderate hunger=69.82% severe hunger=6.35% 56.17% - Moderate or Severe hunger 43.83% - Little to No hunger Little to no hunger=43.23% moderate hunger=56.44% severe hunger=0.33% The REACH 4 project mostly achieved the set targets for this indicator, reducing the population of beneficiaries experiencing severe hunger by about 6 percent (~6%) from the baseline. This is a reasonable demonstration of the effect that the Program had on hunger in the target communities. 4 Proportion of women of reproductive age (15-49 years) who are consuming Minimum Dietary Diversity (MDD-W) 31.11% 36.11% 59.96% The intervention significantly improved the nutrition component of women of reproductive age by about 28% from the baseline. The increment of the baseline values(31.11%) to 59.96% at endline indicate the percentage of beneficiaries who were able to consume the minimum dietary diversity. 5. Proportion of children 6-23 months receiving a minimum acceptable diet (MAD) 11.6% 31.6% 51.44% The intervention significantly increased the proportion of children between 6-23 months receiving a minimum acceptable diet by ~40% from the baseline. This indicates a positive impact of the nutrition activities on young children. 16 6 Prevalence of exclusive breastfeeding of children under six months of age 63.46% 93.46% 87.10% Though the interventions in REACH 4 significantly increased the percentage of breastfed children under 6 months, the project team did not quite meet the target for this indicator. However, the above result was achieved in spite of COVID-19 and disruption of a few months, showing that given time and resources, the Program may have significantly surpassed this target. 7 % of people targeted by the nutrition and hygiene Promotion program who know at least three (3) of the five (5) critical times to wash hands 7.18% 70% 84.38% The significant achievement in this indicator is a pointer to the effectiveness of the messaging and behavior change communication carried out by the project team despite the obvious challenges of time and COVID-19. This also shows the impact of the alternative BCC platforms like the Mother Support groups in propagating the messages disseminated through channelssuch as group meetings and person￾person conversation of practices. 8 Percent of men and women who have knowledge of appropriate child nutrition practices A=46.39% B=25.38% C=21.64% D=6.49% A = 60% B = 21.67% C = 15% D = 3.33% This measures the percentage of men and women who are knowledgeable of appropriate child nutrition especially exclusive breastfeeding. A represent percentage of respondents giving baby only breast milk and no other liquids (including water) or solids for up to 6months B represent percentage of respondents giving baby breast milk and water C represent percentage of respondents giving baby breast milk and some other liquids D represent percentage of respondents who are not knowledgeable of appropriate child nutrition practices. 9 % of children (6-59) months with MUAC score below 12.5cm SAM(<11.5): 1.83% MAM(11.5-12.5): 24.51% SAM- 21.83% MAM- 44.51% SAM(<11.5): 4% MAM(11.5- 12.5): 18% The SAM and MAM scores surpassed the set targets for this indicator with an increase of 8% for children with normal or above normal MUAC scores over the baseline figures. 17 *SAM (severe acute malnutrition), MAM (moderate acute malnutrition) NORMAL (>12.5): 73.66% NORMAL (12.6- 13.5): 30% ABOVE 13.5 (>13.5): 48% 10 Average score of household decision making index (HDMI) 0.307 (1 as maximum) 0.3377(1 as maximum score) 0.4534 (1 as maximum) The results for this indicator show the positive influence REACH 4 had on gender equality in the target communities, with a significant increment of about 48% indicating a significant increase in the level of women’s involvement in decision-making on key issues in the households surveyed. 11 Percent of participating households reporting reduced expenditure on fuel HH with FES = 912 Naira HH without FES = 919 Naira (Mean value spent on fuel per week) HH with FES = 638.4 naira HH without FES = 643.3 naira (mean value spent on fuel per week) HH with FES = 894.81 naira HH without FES =900.83 naira (mean value spent on fuel per week) 68% (3,400 households out of 5,000 households)reported reduced expenditure using fuel efficient stoves. Although the Program did not meet the targets set for this indicator, they achieved an increments on the baseline figures for both households with FES and households without FES. 12 Percentage of community members who report enhanced awareness and demand about product availability and benefits 84% The messaging and communication on the Program seems to have had a significantly positive effect as more than 80% of the community members reported that they were aware of the availability and benefits of the products. 13 % of beneficiaries (disaggregated by sex, age, and diversity) reporting that humanitarian assistance is delivered in a safe, accessible, and participatory manner - Maximum score Possible is 28 92.86% (Score result of 26) This indicator score of 26 out of a maximum of 28 score indicates 92.86 % of the programparticipants reported REACH 4 intervention was delivered in a safe, accessible, and participatory manner. 18 QUESTION 2 - DIVERSITY OF BENEFICIARIES To what extent did the interventions consider the differential vulnerability by gender equity, protection, age, physical and emotional challenges of the participants, and risks to participation in various interventions in program design and implementation? The implementation of the project considered a diverse range of beneficiaries; the project implementation not only put into consideration the gender of the program participants, age groups, disability status, it also ensured convenience and ease of access to elderly beneficiaries and people living with disabilities. According to one of the respondents: ‘‘’I will say the project has considered different categories of beneficiaries because some of the beneficiaries are female headed households, children, the elderly, the disabled, the youth, and if you check our data sets, during redemption, you will see all the pregnant, lactating, also inclusive.’’ Program Staff How management adapted the Program design or implementation based on monitoring information and feedback from the target population a) Communication and Feedback Channels Given the sensitive nature of the REACH 4 Program, the implementation team made efforts to set up various communication channels (post-distribution monitoring visits (PDMs), suggestion boxes, complaint desk (physical) and a hotline (virtual)) that would facilitate an efficient flow of information between the team and the program participants and also enable a feedback mechanism needed for course correction, where appropriate. REACH 4 made use of embedded community members as master trainers to facilitate feedback. The Program also ensured the availability of ‘hotlines’ to enable relaying of feedback and complaints. However, the program participants showed preference for in-person complaint mechanism where they speak directly to the Mercy Corps staff or use the help desk. 19 All the complaints received are card related and mostly had to do with fingerprints not being found, cards being out of use, or missing cards. There were limited reports of beneficiaries exchanging vouchersfor money or of deceased beneficiaries. These complaints were however resolved b) Effective Monitoring of Procedures The M&E team on REACH 4 conducted monthly post-distribution monitoring (PDM) to assess the success of the previous month’s distribution procedures and compliance to the distribution plan. Feedback from these visits helped the program team to effectively redesign the initiatives to fit the needs of the communities and beneficiaries. The comments below from some of the respondents buttresses these observations: They even brought a complain box to gather our problems so that they can solve it for us and also you can go straight to their office and lay your complaint. Like when they started, they were compelling us to take food that we don't want, when we laid our complaints, they started allowing us to choose what we know. Beneficiary The beneficiaries noted that the program implementation has been flexible to adapt to their reality and need. The monitoring data has been effectively used to learn and adapt the REACH-4 program Lessons learned regarding program design and implementation 1. There is a need to have the full complement of implementation staff in order to make the best use of time and resources, especially for short-term Activities such as REACH-4. 2. Use of community members for beneficiary identification helped to ease the tensions that normally surround sensitive Activities such as REACH. 20 Level of efficiency with regards to cost-per-program participant, timely delivery of the goods or services, and adjusting the transfer amount based on price and need changes ● Timely Redemption of REACH 4 support The implementation team did not experience any delay in both delivery and redemption of the commodities distributed to the beneficiaries as up to 99% of the target beneficiaries showed up to claim their food packages. They also did not have any disruptions to the planned schedules of food distribution throughout the period of the Program. ‘’from my perspective we have achieved 99% of redemption, why I say 99% is the plan targets with respect to dry food is 13,512 and the monthly redemption, the number of participants that actually made the redemption is 99 percent. So, based on this statistic, I can confidently say we have achieved 99% of redemption’’. Program Officer 3. QUESTION 3 - INTENDEDANDUNINTENDEDOUTCOMES Intended and unintended food and nutrition security outcome: what changes has the program brought in terms of the household food consumption score, household hunger scores and coping strategy index? ● Food Security The beneficiaries noted that they have had about thirteen (13) to eleven (11) rounds of fresh and dry food redemption respectively. Based on the planned distribution within the 12 months, the rounds of redemption imply 100% success rate for distribution. This frequency of food distribution resulted in the availability of different types of foods for the various households and has improved food security of the households. There was a noticeably improved score for the percentage of households with acceptable FCS during the intervention, from < 14% to 21% It seems most of the gains here came from a reduction in the number of households with poor FCS at baseline; the score for borderline FCS remained unchanged. From the baseline data, about 6.35% of the households had severe hunger which at the end of the program has drastically reduced to 0.33%, while the number of people with little to no hunger had increased by over 20%, from 23.83% at baseline to 43.23%. The negative coping strategy (rCSI) which was at a weighted score of 16.63 at baseline has also reduced to 9.43. This value is far lower than the target set (14.97) at the inception of the program. The increased awareness of the need for nutritious food and the livelihood support introduced to the REACH 4 is largely responsible for this progress. ● Improvement of Nutrition in Children The Program also focused its messaging on the importance of adequate nutrition in children to help the early years of growth and development and help to build immunity against common illnesses. Some of the behavior change communications were geared towards awareness of the importance of breast milk for newborns as a source of colostrum, exclusive breastfeeding for infants, and preparation of complementary foods for babies. The prevalence of exclusive breastfeeding of children under six months of age has also 21 increased from 63.46% to 87.10%. The beneficiaries noted that there is a visible decline of malnourished children in Damboa community. From the data collected on the children between 6 to 59 months, about 48% now have a MUAC score above 13.5 and 30% are normal with MUAC score ranging between 12.5 to 13.5. The target for the REACH 4 was to reduce children with Severe Acute Malnutrition (SAM) to about 21.83%, but this has drastically reduced to about 4%. ‘’And to a great extent, wherever we go, we see the evidence. We don’t really see malnourished children anymore, because aside from the messages on good nutrition for children, we are also given the food’’. Beneficiary To what extent has the program improved household coping and adaptive capacities in the course of recurring conflicts and displacements. ● Livelihood Support Beneficiaries of livelihood support now have sustenance that helps them eat better and improve their ability to cope with the uncertainties and inconvenience of displacement. The livelihood support activities include poultry assistance and backyard gardening, this has helped to improve the ability of the Households to feed more nutritiously and even earn some income from selling their produce. This poultry that they provided us with is very important and we really appreciate it because even if you don't have anything else you can sell it and use the money or even eat it with your family - Beneficiary ● Business The livelihood support provided have enabled the beneficiaries to engage in business activities. There is a ready market in the community for eggs, chicken, and vegetables which a good number of the beneficiaries are already taking advantage of the demand. For instance, the military and the NGOs source their food from the community and this has helped increase the demand. Some of the beneficiaries, have gone beyond poultry farm to rearing of sheep and cow. ‘’Our intended outcome was once they grow and develop the birds, our participants can maybe want to slaughter it and use it for meat for their nutritional diet or eat the egg or even sell it and form another source of livelihood with the eggs like they can use to supplement other needs, but to our surprise the outcome exceeded what we plan for, some of them actually cared for the poultry up to the maturity age and sell the birds and also acquire sheep and some cow. So, which is another higher level of income for them’’ ● Employment Index Vendors engaged through the REACH 4 program provided employment opportunities to indigenes of the target communities thereby reducing unemployment. This was a win-win situation for both the vendors and the community as the earlier had an easily available source of labor and the latter had an increased number of youths engaged in meaningful income earning activities. ‘’Even though we have the capacity to bring vendors from outside to supply our goods but we hired the local vendors, which indirectly is another source of livelihood for them because these vendors have grown to the stage that the now 22 need more hands. So, they hired local boys and men to work for them; helping the Program to indirectly create employment’’ Program Staff ● Reduction in Crime Rate The insurgency influenced peoples’ attitude towards crime and there have been numerous criminal incidents in Damboa, REACH 4 indirectly created income generating jobs, engaged youths, and this consequently helped to reduce the rate of conflict and criminal activities. Also, the hunger prevalence is an incentive for criminal activities such as robbery, the REACH 4 program has helped ameliorate hunger in the community. The impact which is felt in the reduction in the number of robbery cases in the community. ‘’In the context of emergency where we are right now, without better livelihood, crime rates will continue to increase, which is a known fact. But because of the food provided by Mercy Corps and also their other activities such as poultry farm, fuel efficient stoves, vegetable farm, people are now able to feed their households and even generate some income. This has ultimately reduced conflict and criminality in the community’’. Government Official (Director Planning, Research and Statistics) What factors facilitated or inhibited these changes? – include expected and unexpected, positive, and negative factors. 4. QUESTION 4 – HOUSEHOLD FOOD SECURITYANDNUTRITION OUTCOMES Assess the contribution of specific interventions in improving household food security and nutrition outcome: how have the mass sensitization rallies and the vendors shop nutrition promotion during voucher distributions with informational banners led to changes in participants’ food selection pattern? ● Improvement in the food selection The implementation team conducted health and nutrition promotion campaigns to sensitize beneficiaries on the sources and benefits of nutritious foods; Nutrition promoters conducted the advocacies in the communities and also monitored the redemption procedures for food items like eggs, soya milk, and beans distributed to the beneficiaries. There was a significant improvement in the choice of food item redeemed by the beneficiaries due to the nutritional advocacies and mass rallies. This is also evident in the percentage of women of reproductive ages (15-49 years) who are consuming minimum dietary diversity (MDD), which has increased from 31.11% at baseline to 59.96%. “Initially we used to collect carbohydrates such as maize and millet, later on mercy corps staffs changed our mind to pick a healthy and nutritious food to other food such as eggs, beans, milk, meat, and so on so that we will be healthy with us and our children” Beneficiary ● Improved Infant and Young Child Feeding (IYCF) The Mother Support Group set up during the intervention meets twice a month to discuss various self￾help practices including IYCF and good hygiene practices. These meetings were impactful as the mothers share with their friends, innovative practices they use at home to achieve better outcomes in health, nutrition, and hygiene, thereby creating a ripple effect on the community. Prior to the REACH 4 activities, 23 there are certain unhealthy nutritional practices such as not feeding a baby with the first milk at birth which is now fast eradicated due to the nutritional messages from the MSG. ‘’Many of the mothers do not know, because there are most of them who don’t know the importance of IYCF. But with the meeting that we normally have, like I said practices such as good hygiene practices as well baby feeding has really improved. Before now, mothers usually throw away the first milk that comes out of the breast after a baby is born, but with our house-to-house engagement, this practice is no longer popular’’ Improved Hygiene Practices The mass sensitization has also improved the hygiene practices in the community. The percentage of people targeted by the nutrition and hygiene Promotion program who are now aware of at least three (3) of the five (5) critical times to wash hands has increased significantly from 7.18% at baseline to about 84.38%. About 51% of the beneficiaries now use Soap to wash their hands. This is largely due to the sensitization and inclusion of soap and detergents in the collection basket. Impact of Voucher System on Food Security The voucher system has allowed the beneficiaries an unrestricted access to fresh food and diverse variety of foods. REACH 4 set up the operation in a decentralized fashion such that redemption could be made at any of the vendor stores across the communities in Damboa LGA. The redemption does not require cash transfer, thereby reducing incidents of financial theft. The voucher is secured and cannot be used by a non-beneficiary, hence reducing the possibility of theft. However, a lot of card related complaints were also received from beneficiaries and most of which had to do with fingerprints not being found or cards being out of use 24 There were also complaints from vendors on issues from the card readers and printer during redemption. The card readers are shared among vendors of dry food and fleshy food and often times they experience faults on the machine which they believed could be due to how their co-vendors use the device. There were suggestions that each vendor should be given his own card reader. This will enable proper accountability in usage. “it has made our work easier and improve transparency of the redemption process. The only hindrances we experience is the issue of our machine scanner and printer during distribution. Sometimes, the machine is allocated is not working and this may be because the vendor who used it before did not manage it properly. This usually brings delay during redemption. I wish Mercy Corp will give each vendor his own scanner and printer so that if it has issue, the vendor will be held responsible” Vendor 5. QUESTION 5 – COLLABORATIONS Level of satisfaction of the stakeholders including government departments, food security cluster, participants, and others ● Regular Stakeholders Meeting To ensure consistent commitment and buy-in from the Program stakeholders, the REACH 4 Program team held regular meetings with the CRC, government representatives and other stakeholders in the communities to carry them along with the successes and challenges experienced by the team and in the process improve transparency, accountability and build trust with the beneficiaries. ‘’Now for other stakeholders like the military, the military gave us very good cooperation, whereby they provided on escort for our vendors to bring in items from other locations. So, they have been cooperative in our implementation. Also, each time we call for meetings, or we have messages to give to the military, they are always available to get or so-so.’’ Program staff ● Gained Community Support The stakeholders were satisfied with the Mercy Corps REACH 4 program, especially the government and the beneficiaries. The government see the initiative as well-crafted and it has reduced the pressure the people would have had on the government resources which is limited. The goodwill built by REACH 4 through its high-level acceptance by the community throughout the implementation phase has laid a footprint for any subsequent humanitarian intervention in Damboa. “We have become a footprint other can learn from, because we have practically laid example on different approaches for them to follow while implementing our own program. I must say, adult and children in every household, know about mercy corps, we have gained 100 percent acceptability within the community.’’ Program Staff 6. QUESTION SIX -LEVERAGINGANDLAYERING Leverage and layering: to what extent did the program leverage existing other USG and non-USG investments in the same space to facilitate linkages with complementary services, 25 layering with earlier investments, and implementing and exit strategy to minimize the dependency on external assistance? ● Livelihood Investment The REACH 4 Program team was able to liaise with other USG IPs, Borno State Government and Bill and Melinda Gates Foundation and used their resources to facilitate the delivery of the livelihood activities, comprising supply of food commodities, assistance with establishment of small holder poultries, set up of backyard garden and distribution of fuel-efficient stoves. ‘’So yeah, we've been able to leverage not just on facilities, but also resources, be it human capital resources from other implementing agencies like during coordination, we get feedback from other people, which has helped us in our implementation. We try to balance our activities like okay, like what we provide, and what other people also provide not to have Overlapping. Okay, yeah. So yes, we've done greatly in that. And then we have like, talking about poultry. It was not really a US government funded USAID funded program, it was a Bill and Melinda Gates funded program, where poultry birds were given registered participants’’ ● Leveraging on Government Support The REACH 4 team also worked with consultants from the Borno State Agricultural Development Program, who trained participants on smart agriculture, good agricultural practices, and business. Also, with the help of government, REACH 4 received lands used as demo farm to educate the beneficiaries. There was also military support during the implementation. “Yeah, we had some consultants from the Borno State Agricultural Development Program that we invited or we partnered with to train our participants on smart agriculture, ‘farmaculuture’, good agricultural practice, farmers business school. With the help of local government, we were able to secure lands to do our demonstration farms, so they gave us and we used those things to teach and demonstrate to the participants of the backyard gardening. There has been that cooperation between us and other government agencies.’’ Extent to which the benefits of the interventions can be sustained after the completion of this Program ● Resource Management Training The Program incorporated a training component to prepare the beneficiaries in sustaining the outcomes from REACH 4. As part of the objectives of REACH 4, the participants were trained in poultry farming, backyard garden, infant nutrition to ensure a transfer of live saving and sustainability skills to enable them support themselves with or without intervention from Mercy Corps or similar organizations. The Program team complemented the training with provision of materials to enable them to put into practice some of the knowledge gained during the trainings. The Program team used interventions such as the backyard garden to enable the participants obtain some of the benefits that would normally accrue to them from their farmlands without exposing themselves to insurgents on the way to the farms located some distance from the communities. 26 “I will continue with the training I received on livestock farming and other micro gardening and endeavor to sustain myself and make it as a business purpose of business and for survival” – Program Beneficiary “Yes, if they will stay it will be good but even if they are not around, they have enabled us with many things like poultry and they showed us many skills that we can sustain ourselves with” – CRC Member 27 SECTION 3: SYNTHESIS, RECOMMENDATIONS AND LESSONS LEARNED 3.1 SYNTHESISAND CONCLUSIONS Q 1 The REACH-4 intervention achieved most of the intended targets, the interventions were innovative and relatable to the immediate needs of the beneficiaries. The interventions addressed issues like hunger, nutrition, households’ livelihoods, and unintended outcomes like ensuring security of beneficiaries by giving them alternatives to going to their farms and being exposed to security risks, providing employment to boys and youth in the communities. There is value in using CRC members who are well known within the community to identify participants. Q 2 In addition to a diverse range of beneficiaries (women, elderly, People Living with Disabilities (PLWD)) selected, the program also employed a wide range of benefitssuch as food and livelihood assistance that not only addressed the immediate needs of the communities but also included a resilience and sustainability element that the beneficiaries can continue using after the end of the Program. The program made use of effective behavior change messages and communication channels such as-, complaint desks, phone hotlines, mother support groups, and nutrition promoters to reach the intended targets. These open communication channels helped to give the communities a sense of belonging, seeing that their opinions and complaints were heard and responded to, helped to contribute to the good will REACH 4 experienced in the target population and also helped to reinforce the more tangible elements of the program. Q 3 The initiatives under REACH 4 were useful to the communities, both directly and indirectly. The program employed youths in the communities, engaged both male and female master trainers as well as Community Nutrition Promoters. The direct beneficiaries received food, trainings, and other forms of support from the program. Indirect benefits accrued to the communities through employment provided to other youths in the community by vendors engaged by the program. The behavior and change communication were not restricted to the target beneficiaries alone but also filtered out to other members of the community. These benefits further resulted in a greater number of engaged youths, community members with improved capacities and livelihoods, and ultimately in reduced crime rates in the communities. It is easy to imagine that innovative outputs of the program such as the fast-reproducing mixed breed chickens for Poultry farming will enable more members of the community rebuild their household economies and lead to sustenance of the outcomes of the program. Q 4 The Program was able to contribute in ensuring that vulnerable households have access to nutritionally diverse food through the supply of various food classes, information on nutritional alternatives during advocacies, and provision of other classes of food through the livelihood elements such as the backyard 28 garden for growing vegetables and poultry farming for rearing chickens for protein. The use of vouchers also afforded beneficiaries the ability to choose from a greater variety of foods. The IYCF and Mother Support Groups were critical components in reinforcing the messaging during the advocacies and helped to further spread valuable information in hygiene and survival between members of the community. Such sense of togetherness and community helped in eliminating issues such as attacks from disaffected community members in previous iterations of the REACH Program. Q 5 The implementation was able to achieve a reasonable level of collaboration and cooperation from both government, USG-funded and non-USG funded organizations in implementing REACH 4. Various government officials interviewed indicated satisfaction with the efforts of Mercy Corps on REACH-4; the Borno state government provided a plot of land to serve as demonstration plot for the backyard gardening and seconded officials from the ministry of Agriculture who provided guidance to beneficiaries on some of the livelihood activities. The Program succeeded in preventing overlaps and duplication of efforts because they were able to regularly check-in and discuss with other implementing mechanisms in the area; such efforts are commendable especially for Implementing partners working in conflict zones where resources are scarce and the target populations increase with every fresh security incident. Q 6 The REACH 4 Program effectively leveraged resources from other partners and used competences available in government counterparts to enhance the objectives of the Program. They were able to utilize trainers from other government funded programs to deliver some elements of the livelihood component of the Program. Many beneficiaries interviewed indicated their willingness and ability to carry on with the skills and knowledge they gained from the livelihood activities and trainings to ensure economic sustenance post￾REACH 4. 3.2RECOMMENDATIONS Q 1 ● Mercy Corp should consider early identification and recruitment of all needed implementation staff for such short-term assignment especially, given the time constraints. This will enable the optimization of resources (time, money and human) available for the Program; ● Having achieved reasonable levels in addressing hunger via the food consumption and hunger scores for the Program, subsequent interventions should focus more on the livelihood elements of the Program to ensure sustainability ● With the success of the backyard garden, other livelihood components such as dry season farming should be considered in subsequent interventions given the availability of water (stream in Damboa) within the confines of the communities. Q 2 ● In implementing the program, people living with disabilities were targeted but ease of access and convenience were not appropriately considered. There is need to make collection points more friendly for PLWD. 29 ● The targeting for subsequent related activitiesshould capture beneficiaries who have not benefitted from REACH 3 and 4 as there are complaints from those who felt marginalized while others have benefited more than ones. Q 3 ● Subsequent programsshould consider creating more internal and external linkages for markets, so that more beneficiaries can translate the livelihood activities from mere avenues of sustenance to income generating businesses. ● Other livelihood components such as skills acquisition maybe considered in furtherance of efforts to keep the youths engaged and reduce crime rate. Q 4 ● The Mother Support Group is an effective group that should be sustained beyond the REACH 4 program for sustainability of nutritional gains Q 5 ● The government expressed satisfaction with the impact of the REACH 4 Program in the target communities, there exists an opportunity to leverage on this to further liaise with the government for further collaboration in subsequent Activities. ● Through partnership with the government, safe lands could be obtained to encourage dry season farming. Q 6 ● Greater collaboration between donors should be further enhanced at the donor level; donors should require their implementing partners to liaise among themselves to ensure improvements in results. Information such as reliable vendors, community penetration strategies, cost saving opportunities, security arrangements etc. should be shared and exploited between implementing partners to optimize value for money, especially in conflict affected areas. ● The IP should intensify advocacy to the various governments (local and state) to provide amenities such as markets, secure access roads etc. that will enable beneficiaries to implement and improve on the livelihood activities. Learning some of these skills and willingness to implement them may be hampered by a lack of amenities that will enable them to turn the activities to businesses. 3.3 LESSONS LEARNED 1. The CRC members were highly involved during the program implementation. The involvement of the CRC in the program increased community participation, buy-in and ownership of the program. They also served a reliable channel for feedback between the community and the program team. SECTION 4: CONFLICTSOF INTEREST The evaluation team did not identify any conflict of interest, or appearance thereof from the programstaff. However, the evaluation team identified a potential conflict of interest in the selection of the target beneficiaries by the CRC; being members of the community there was a potential for the CRC members to choose/select relatives of theirs or friends to be included in the beneficiary list. To mitigate this, the REACH 4 Program staff were also hands-on in the selection process to ensure that all the beneficiaries identified met the selection criteria outlined and that only the most vulnerable were selected for the Program. 30 ANNEXES ANNEX1: TOR FOR THEEVALUATION Project/Consultancy Title: Project Location(s): Duration: Final Evaluation – REACH 4 – Food For Peace Damboa, Borno State. 20 Days (from 01 February 2021 TO 20 February 2021) Background: Entering its tenth year, the humanitarian crisis in Borno State, North East Nigeria continues to face challenges with no clear signs of abating. As of September 2019, 1.8 million Nigerians have fled from their homes and are internally displaced, the majority within Borno State – the epicenter of the crisis. 80% of internally displaced people are women and children, and one in four are under the age of five. Ongoing hostilities and military operations led to waves of mass displacement and continue to impact humanitarian operations. In the Damboa Local Government Area (LGA) of Borno State, more than 80,000 people are displaced, 58,000 of whom reside in Damboa town. These individuals are experiencing severe impacts upon their food security and nutritional status. Mercy Corps’ REACH programming in Damboa has contributed to substantial improvements in food security (the proportion of households with a poor or borderline Food Consumption Score (FCS) dropped from 60% in October 2017 to 47% in August 2018 among targeted households). Despite improvements, the underlying conditions that contributed to food insecurity in Damboa remain unchanged; food security gains observed in households participating in REACH’s programming are largely dependent on existing programming. While the World Food Programme (WFP) through their partner IMC, provides food rations for displaced households residing in the IDP camp in Damboa town, three-fifths of the IDP population living in Damboa town are integrated in the host community. Mercy Corps is the only other organization in Damboa providing food assistance to only about 23% of those. Responsive Economic Assistance for Conflict Affected Households (REACH) 4 is a Food For Peace-funded emergency food security program that began in February 2020 and runs through January 2021. The ultimate goal of REACH 4 is to enable conflict-affected households in Damboa to meet essential food needs during the program’s lifetime. As a result of the increasing need of humanitarian assistance and to meet these urgent needs, Mercy Corps proposes to build upon the previous iterations of FFP food security responses in NE Nigeria with a multi-pronged approach that increases access to both staple and nutrient￾dense food through local markets while also shaping behaviors to support optimal nutrition. The REACH 4 program is expected to reach the most vulnerable household who are food insure households in Damboa with food assistance to enable them meet their essential food needs by engaging in multiple market systems; and promote sustainability and resilience by investing in nutrition and production capacities. Using this approach, REACH 4 will address both the immediate food security needs, as well as commence the work of taking Damboa along the path from food dependency to food self-reliance. Purpose / Project Description: The ultimate goal of the REACH 4 is to enable 13,512 conflict-affected households in Damboa have improved food and nutrition security during the program’s lifetime of 12 months. Building on three years of experience successfully implementing the REACH program (1, 2 & 3), Mercy Corps aimed to meet the 31 essential food security needs of the most vulnerable households this include the IDPs, Returnees and the Host community while contributing to a foundation for resilience in Damboa LGA. To achieve this, Mercy Corps used a multi-pronged approach to increase access to both staple and nutrient-dense food through local markets while also shaping behavior change to support optimal nutrition. The program accomplishes this objective through work in food security, nutrition and household energy (fuel efficient stoves (FES): • Food Security: REACH 4 worked to ensure that 13,512 conflict-affected households access nutritionally adequate and diverse foods via vouchers for nutritious food items for vulnerable households. During the phase 4, Mercy Corps explored expanding food options to potentially encourage households to purchase vegetables, fruits, animal-source foods and other fresh foods that are locally available and culturally appropriate -- in addition to the staple food items that are already accessed through market vendors. Expansion of food options will have the added benefit of supporting local producers, while also ensuring these households consume essential micronutrients. • Nutrition: REACH 4 conducted nutrition/hygiene promotion activities as per the outcomes of the needs assessment responding to negative coping mechanisms and to promote proper hygiene practices and Infant and young Child Feeding (IYCF), and train women and men in child health and nutrition. This will emphasize behavior change communications and strategies with groups of women and adolescent girls, while also creating linkages to, and driving demand for, nutritious foods and nutrition and health services. • Fuel Efficient Stove (FES): REACH 4 worked to ensure that households are able to prepare staple and fresh foods safely and hygienically, a small pilot to support increased access to fuel efficient stoves (FES) for vulnerable households. The purpose of the Final evaluation of the program is to (a) assess the program’s achievements of the activity in relation to the goal, objectives, results and targets (b) Evaluate the activity’s effects on the local markets, and how it affected certain groups of interest (women, men, youth, boys and girls), Evaluate the effectiveness and relevance of the modality, transfers and complementary interventions to achieve activity outcomes (d) assess the sustainability of its results, (e) Identify best practices, lessons learned, strengths and challenges in activity design, including log frame, and implementation of achieving project achievements. The evaluation is to cover all sub-sectors of the intervention. The results of the Final evaluation will be used to inform the final reporting to the donor and will be shared with the food security Cluster in Nigeria and to help inform Mercy Corps’ current and future food security programming. The consultant is expected to work in-country (or remotely in case of international) and to deliver a comprehensive Final evaluation report no later than February 25, 2021 - with a first draft submitted for review by February 18, 2020. Evaluation Questions The evaluation questions that will be considered as cornerstones for this assessment are: • Achievements: To what extent have the interventions adhered to planned implementation - schedules, participant targeting, resource transfer composition and quantities, inputs and service delivery, and outputs - and achieved intended goals, purposes and outcomes? Did interventions reach the appropriate target groups and individuals within the target areas? Are interventions appropriate and effective for the target group based on the nature of their vulnerabilities? How effective was the targeting 32 approach in achieving the project goal? What factors promoted or inhibited adherence to plans and targets? How were problems and challenges managed? What lessons were learned? • Effectiveness and Efficiency of Interventions and Intervention Implementation: To what extent did the interventions consider the differential vulnerability by gender equity, protection, age, physical and emotional challenges of the participants, and risks to participation in various interventions in project design and implementation? How has management adapted the project design or implementation based on monitoring information and feedback from the target population? What lessons were learnt regarding program design and implementation? What was the level of efficiency with regards to cost-per-project participant, timely delivery of the goods or services, and adjusting the transfer amount based on price and need changes? How have the nutrition activities impacted the nutritional outcomes of project participating HHs, in general, and households with vulnerable participants like, pregnant women, people living with disabilities, and the elderly, in particular household with children screened with severe and moderate acute malnutrition? • Intended and Unintended Food and Nutrition Security Outcome: What changes has the program brought in terms of the household food consumption score, household hunger scores and coping strategy index? To what extent has the program improved household coping and adaptive capacities in the course of recurring conflicts and displacements. Include expected and unexpected, positive and negative – what factors facilitated or inhibited these changes? • Assess the contribution of specific interventions in improving household food security and nutrition outcome: How have the mass sensitization rallies and the vendors shop nutrition promotion during voucher distributions with informational banners led to changes in participants’ food selection pattern? How effective has, the Mother Support Groups been in passing nutritional massages? How impactful is the voucher system? How can the food intervention be improved in the future? • Collaborations: What is the level of satisfaction of the stakeholders including government departments, food security cluster, participants, and others? • Leverage and Layering: To what extent did the project leverage existing other USG and non-USG investments in the same space to facilitate linkages with complementary services, layering with earlier investments, and implementing and exit strategy to minimize the dependency on external assistance. To what extent can the benefits of the interventions be sustained after the completion of this project? Consultant Deliverables: The consultant is expected to provide a means of answering the evaluation questions using both qualitative and quantitative means of data-collection, providing the sampling strategy adopting the sample size from the feed the future sample size calculator and data analysis. This will include the key deliverables of: • Develop an Inception Report detailing the process and methodologies to be employed to answer the evaluation questions. • Undertake desk review of the relevant program documents and secondary analysis to further inform the results interpretation • Design qualitative and quantitative data framework and tools: The external consultant is expected to conduct a mixed methods evaluation using tools and a workplan approved by the Country MEL Manager prior to the start of the evaluation. Data collection shall involve visits to a sample of project locations, meetings with program partners, targeted participants and other key stakeholders. The consultant will lead the qualitative and quantitative data collection, including supervising data collection teams, and completing the analysis within the approved timeline. 33 o Qualitative: The qualitative evaluation must capture lessons learned and best practices through a variety of qualitative methods. The evaluation team will design the overall qualitative study approach and should consider a variety of primary data collection methods, including: semi-structured in-depth interviews, focus group discussions, and observations. The evaluation team leader and members will be responsible for collecting and analyzing qualitative data. Data will be collected from key stakeholders through interviews, discussions, consultative processes, and observations. o Quantitative: The Final evaluation will include primary data collection and analysis of quantitative survey data. The Final survey tool will be designed by the consultant and must utilize the same data collection instruments, level of statistical precision, and statistical power as the baseline survey. The field operations - from hiring and training of enumerators and testing of tools - will be led by the consultant with support from Mercy Corps. The evaluation shall be designed to detect statistically significant changes in estimates from baseline to Final for key indicators • Quantitative designs should allow to examine statistical significant changes in estimates from baseline to Final for key indicators (sampling frame and sample size to be calculated using the revised FFP MEL guideline). In addition to the three primary indicators, the evaluator will examine other performance indicators as per the log frame; Indicator Source Indicator Data Collection Method FFP-EFSP 2 Food Consumption Score (FCS) - Percent of households with poor, borderline and acceptable food consumption score Participants based survey FFP-EFSP 3 Reduced Coping Strategy Index (rCSI) - Mean Participants based survey FFP-EFSP 4 Household Hunger Score (HHS) - Prevalence of households with moderate or severe hunger Participants based survey FAO USAID WHO Fanta Minimum Dietary Diversity for Women (MDD-W) & Minimum Acceptable Diet for children 6-23 months Participants based survey Adapted from USAID/MCHIP Household Water and Sanitation Module Percent of people targeted by the hygiene promotion program who know at least three (3) of the five (5) critical times to wash hands Participants based survey Custom Percentage of children under 5 with children with MUAC Score of 13.5cm above. Participants based survey Feed the Future Indicator Handbook 2014 (also FFP indicator and MCHIP/USAI Prevalence of exclusive breastfeeding of children under six months of age. Participants based survey Custom Percentage of community members who report enhanced awareness and demand about product availability and benefits Participants based survey Custom % of participating households reporting reduced expenditure on fuel Participants based survey DHS/FFP/ CARE/WEA Average score on the program relevant household decision making index Participants based survey 34 • The consultant is expected to carry out the following tasks and to produce a comprehensive Final evaluation report. • Train, and oversee the work of Enumerators/Surveyors • Prepare draft evaluation report • Conduct in-country presentation of findings for Mercy Corps • Prepare Final evaluation report and presentation • Draft Report brief of 4-5 pages summarizing key findings and program achievements Below is an overview of the activities, their duration, and the stakeholders responsible. The duration/Level of Effort included is an estimate. Duration Activity Stakeholder Week 1- 5 Working days Review draft evaluation SOW with the external evaluator to clarify timeframe and available budget External evaluator, Program Manager, Country MEL Manager, Humanitarian Response Director Undertake desk review of the relevant program documents that include the Proposals, Implementation Plans, Revised Program designs and timelines, Program Implementation Reports, Mercy Corps strategy documents, Monthly PDM reports, Assessment reports and any other relevant documents. Develop an Inception Report detailing the process and methodologies to be employed to answer the evaluation questions. This should include all evaluation tools, and important time schedules for this exercise, and be presented to Mercy Corps for review and further inputs before going to the field. External evaluator Provide feedback to inception report and tools for external evaluator to incorporate Program Manager, Country MEL Manager, Humanitarian Response Director With input from MCN Programs team and MEL teams, refine data collection tools and translate them as appropriate External evaluator 35 Week 2 – 5 working days Train Enumerators/Surveyors; Pre-test data collection instruments External evaluator/MEL Officer Finalize data collection instruments External evaluator Oversee data collection External evaluator Week 3&4 – 10 working days Encode and Analyze data (using ONA and ODK) External evaluator Prepare draft evaluation report External evaluator Provide detailed feedback to draft report Program Manager, Country MEL Manager, Field Manager, Technical Sector Manager, Humanitarian Response Director Finalize report, produce presentation of findings, and share back with MC (Not more 25 pages – all other additions can be included as Annexes) External evaluator The consultant will be provided with transportation and accommodation in-country and location of Final evaluation – but Mercy Corps will provide guidance and recommendations. The evaluation consultant will include their airfare as part of the contract which will be covered by Mercy Corps– and Mercy Corps will support with access to field/implementation sites. Mercy Corps Monitoring, Evaluation and Learning (MEL) team will be available to work directly with the consultant throughout the duration of the consultancy and to answer any question as they emerge. Report Structure & Content: • Cover Page, List of Acronyms • Table of Contents • Executive Summary: This section should be a clear and concise stand-alone document that gives readers the essential contents of the evaluation report, including a summary of major findings, lessons learned, and recommendations. • Methodology: This section should be sufficiently detailed to help the reader judge the accuracy of the report and its findings. • Limitations: This section should address constraints and limitations of the methodology, and the implications of these limitations for the findings, including whether and why any of the evaluation findings are inconclusive. 36 • Results: This section should provide a clear assessment of progress with respect to indicators / targets / objectives and/or evaluation questions, production of indicator estimate with tables showing the indicators, BL/EL indicator value and statistically compare the two value. Reference baseline and midterm evaluation information as well as program logic, theory of change, etc. • Synthesis, Recommendations and Lessons Learned: This is space for the evaluation team to think about the data and results, and make concrete recommendations for current or future program improvements, pull out organization lessons learned, and generally comment on data and results. Everything presented in this section should be directly linked back to the information presented in the Results section of the report • Conflicts of Interest: Disclose any conflicts of interest or the appearance of conflicts of interest, including the interest of program staff in having a successful program. • Annexes: These should include a complete file of data collection instruments in English, list of stakeholder groups with number and type of interactions; SOW, qualitative protocols developed and used, any data sets (these can be provided in electronic format), any required photos, participant profiles or other special documentation needed. The following are the key deadlines for the report: • First draft report to be submitted by February 18, 2021. • Mercy Corps will review draft report and provide feedback no later than February 20, 2020. • Final report, incorporating feedback, will be due on February 25, 2021. Timeframe / Schedule: It is expected that the consultant is available to start on February 01, 2021 with an initial meeting with the cash team. The consultant will then work until February 20, 2021 where the Final evaluation report (adjusted according to Mercy Corps’ feedback) is to be shared. It is expected that the consultant will work for an estimated 20 working days between February 01, 2021 and February 20, 2021 (both days inclusive). The consultant should provide an estimate of the number of days/level of effort they expect is required. The figures above are estimates. The invoice for services should be provided to Mercy Corps immediately after the delivery of the Final evaluation report (due of February 25, 2020). The Consultant will report to: Mercy Corps‘ Country MEL Manager The Consultant will work closely with: Mercy Corps’ REACH FFP program manager, MEL Team, CASH Program Manager and Grants and Reporting Manager, Humanitarian Response Director and Field Manager. Required Experience & Skills: The following are the qualifications and experiences the consultant should possess: 37 • Strong experience conducting major research exercises in support of major development programmes – preferably in food and nutrition security – in challenging operational environments, previous experience in Nigeria is desirable. • A strong approach to assuring quality assurance of data collected. • A strong ethical approach to data collection – while still being able to meet the objectives of the consultancy. • Demonstrable experience in leading evaluations of humanitarian programs responding to major disasters, with specific emphasis on food security and protection in emergencies. • Knowledge of strategic and operational management of humanitarian operations and proven ability to provide strategic recommendations to key stakeholders; • Strong analytical skills and ability to clearly synthesize and present findings, draw practical conclusions, make recommendations and to prepare well-written reports in a timely manner; • Demonstrated experience in both quantitative and qualitative data collection and data analysis techniques, especially in emergency operations; • Data visualization skills highly desirable • Consultant is expected to have strong skills in survey form design for mobile data collection (using ONA and ODK) • Experience with evaluation USAID-funded projects • Experience, knowledge and clear understanding of Nigeria humanitarian context; • Good interpersonal skills and understanding cultural sensitivities; • Readiness to travel to North East (Damboa, Borno State) conduct direct standard assessment activities as well as field visits to program sites. Assessment and award of the assignment Mercy Corps will evaluate technical and financial proposals (include accommodation, flights and enumerators management costs) and award the assignment based on technical and financial feasibility. Mercy Corps reserves the right to accept, or reject one or all proposals received without assigning any reason, and is not bound to accept the lowest or the highest bidder. Only those shortlisted will be contacted. Submission deadline for the technical and financial proposal is before COB December 11, 2020 Any sub-contracting under this evaluation consultancy will not be accepted. 38 ANNEX2: LIST OF RESPONDENTS LIST OF RESPONDENTS (Physical Data Collection) TARGET AUDIENCE METHOD OF DATA COLLECTION Remark Vulnerable Households: - 5 KII (At least one respondent from each of these categories: Men, Women, Adolescent Livelihood Support beneficiary and Fuel￾efficient stove beneficiary) - 3 FGD sessions consisting of 5-7 individuals A: Head of Households B: Women of Reproductive Age (15-45); including pregnant women and lactating mothers of children under 5 years C: Individuals responsible for household Cooking and caregivers - 40(5 KII from each community) - 3 Focus Group Discussion (representatives from each community) Beneficiary living with Disability - 3KII - 3 KII (3 randomly selected from any of the communities) Market Vendors (REACH 4 and Non-REACH 4) - 2 KII - 16(1 KII from REACH 4 and non-REACH 4 respective in each community) - Leaders (Community or Household Leaders) Religious leaders (Christian and Muslim - 1 KII - 4 KII - 8 KII (1 KII from each community) - 4 KII (2KII randomly selected from each of the religion) 39 Local government official (Desk officer and Secretary) Community Relief Committee members - 2 KII - 1 KII - 2KII selected local government officials - 8 KII (1 KII from each community and should include youth and women representatives) Implementing Staff in Nutrition (eg program trainers/nutrition promoters - 1 KII - 1 KII Implementing Staff in Food Security - 2 KII - 2 KII FFP Officer (Food for peace): Stephen Livelihood- Backyard gardan and poultry. Implementing Staff in Fuel Efficient Stove - 1 KII - 1 KII Cash team: Meshach Total 85 KII and 3 FGDs ANNEX3: DATA COLLECTION TOOLS ANNEX 3.1: KEY INFORMANT INTERVIEW GUIDE (WITH PROJECT BENEFICIARIES) Please use this guide to facilitate interviews with project beneficiaries. This guide will elicit questions on perspectives and experiences of the Responsive Economic Assistance to Conflict affected Household (REACH) 4. Participants should be thoroughly probed for information. Enumerators should start by introducing themselves and a brief about the project. They should collect the information needed in the table below before proceeding with the questions. My name is {enumerator}, I work with Mercy Corps on the REACH 4 Project. I am here to conduct an interview on the support provided to you. Do you give me consent to proceed? BACKGROUND INFORMATION 40 Interview code: Interview date: Interviewer’s name: Respondent’s name: Interview start time Interview end time Category of participant: ( ) Fuel Efficient Stoves Beneficiary ( ) Livelihood Support Beneficiary (Backyard garden and Poultry) ( ) People with Disability (PLWD) ( ) Men ( ) Women ( ) Adolescents Age of participant: Local Government Area: Community: Language in which narrative interview was undertaken: Evaluation Objectives Questions Probes (a) Assess the program’s achievements of the program in - Tell me about the REACH 4 project? - How has the project helped you (and your child) in improving access to nutritious 41 relation to the goal, objectives, results and targets food? - How has REACH 4 ensured your household was able to prepare staple and fresh food safely and hygienically through provision of fuel-efficient stove? - - Do you think people in your community now have access to improved nutrition? (If yes, what makes you think so?) (b) Evaluate the program’s effects on the local markets, and how it affected certain groups of interest (women, men, youth, boys and girls) - What changes has the program brought with regards to affordability of food? - What change has the program brought with regards to availability of food in the market? - What support is being provided to ensure men/women/adolescents/people with disability groups participate in the project? (Kindly share experiences) - Can you now buy food regularly and when you need it? - Is your choice of food usually available in the market? (c) Evaluate the effectiveness and relevance of the modality,; transfers and complementary interventions to achieve program outcomes - What relevant changes has the program brought in terms of the household food consumption? Do you now eat quality meals more frequently? - Do you recall being asked for feedback to improve project effectiveness. - Can you share areas of improvement the project has made based on your feedback? - How has the nutrition activities (e.g., sensitization) impacted your nutritional outcomes? - What is your opinion about how effective the Mother Support Groups have been in passing nutritional massages? (d) Assess the sustainability of its results, - How has the project prepared you to sustain your access to nutritious food? 42 - How would you ensure the results of the intervention be sustained after the completion of this project? - What would you do to continue growing crops, manage poultry and maintain the use of the fuel￾efficient stoves given to you? (e) Identify best practices, lessons learned, strengths and challenges in program design, including log frame, and implementation of achieving project achievements. - How well do you think this project has done in meeting your nutritional needs? - In your opinion, what are some of the things that should be improved on this project? - How have the mass sensitization rallies and the vendors shop nutrition promotion during voucher distributions with informational banners led to changes in your food selection pattern? - Can you describe briefly how the voucher system has been impactful? What lessons can be drawn from the voucher system? - How has the items supplied through the voucher meet your demand for food? - What do you like most about the project? - What don’t you like about the project? ANNEX 3.2: KEY INFORMANT INTERVIEW GUIDE (WITH PROJECT IMPLEMENTERS) Please use this guide to facilitate interviews with project implementers. This guide will elicit questions on perspectives and experiences of the Responsive Economic Assistance to Conflict affected Household (REACH) 4. Participants should be thoroughly probed for information. Enumerators should start by introducing themselves and a brief about the project. They should collect the information needed in the table below before proceeding with the questions. My name is {enumerator}, I work with Mercy Corps on the REACH 4 Project. I am here to conduct an interview on the support provided to you. Do you give me consent to proceed? BACKGROUND INFORMATION Interview code: Interview date: Interviewer’s name: Respondent’s name: 43 Interview start time Interview end time Job designation: Unit of participant: [ ] Cash team [ ] Nutrition team [ ] FFP team [. ] Livelihood team Age of participant: Local Government Area: Community: Language in which narrative interview was undertaken: Evaluation Objectives Questions Probes (a) Assess the program’s achievements of the program in relation to the goal, objectives, results and targets - To what extent have the interventions adhered to planned implementation - schedules, participant targeting, resource transfer composition and quantities, inputs and service delivery, and outputs - and achieved intended goals, purposes and outcomes? - What changes has the program brought in terms of the household food consumption score, household hunger scores and coping strategy index? - Do you think people in Damboa now have access to improved nutrition? (If yes, what makes you think so?) 44 - How has the project improved the coping and adaptive capacities in the course of recurring conflicts? (b) Evaluate the program’s effects on the local markets, and how it affected certain groups of interest (women, men, youth, boys and girls) - How effective was the targeting approach in achieving the project goal? - To what extent did the interventions consider the differential vulnerability by gender equity, protection, age, physical and emotional challenges of the participants, and risks to participation in various interventions in project design and implementation? - How have the mass sensitization rallies and the vendors shop nutrition promotion during voucher distributions with informational banners led to changes in participants’ food selection pattern? - How effective has, the Mother Support Groups been in passing nutritional messages? - How impactful was the voucher system in meeting the food demand of beneficiaries? - Has it influence the demand and consumption of nutritious food? (c) Evaluate the effectiveness and relevance of the modality, transfers and complementary interventions to achieve program outcomes - Did interventions reach the appropriate target groups and individuals within the target areas? - How appropriate and effective are the interventions for the target group based on the nature of their vulnerabilities? 45 - What was the level of efficiency with regards to cost-per-project participants, timely delivery of the goods or services, and adjusting the transfer amount based on price and need changes? - Do you recall being asked for feedback to improve project effectiveness? - Can you share areas of improvement the project has made based on your feedback? (d) Assess the sustainability of its results, - What activities have been put in place to ensure the continuity of this project beyond the funding years? - How has management adapted the project design or implementation based on monitoring information and feedback from the target population? - How well has the project leverage on existing USG and non-USG investments in the same space to facilitate linkages with complementary services, layering with earlier investments, and implementing and exit strategy to minimize the dependency on external assistance. - What consistent sustainability practices have you observed among the beneficiaries? - To what extent can the benefits of the interventions be sustained after the completion of this project? (e) Identify best practices, lessons learned, strengths and challenges in program design, including log frame, and implementation of achieving project achievements. - What lessons were learnt regarding program design and implementation? - What are some of the challenges faced either in delivering or accessing these improved food items? - How were problems and challenges managed? What lessons were learned? (if any)? - What do you like most about the project? - What don’t you like about the project? - What factors promoted or inhibited adherence to plans and targets? 46 ANNEX 3.3: KEY INFORMANT INTERVIEW GUIDE (WITH MARKET VENDORS) Please use this guide to facilitate interviews with market vendors. This guide will elicit questions on perspectives and experiences of the Responsive Economic Assistance to Conflict affected Household (REACH) 4. Participants should be thoroughly probed for information. Enumerators should start by introducing themselves and a brief about the project. They should collect the information needed in the table below before proceeding with the questions. My name is {enumerator}, I work with Mercy Corps on the REACH 4 Project. I am here to conduct an interview on the support provided to you. Do you give me consent to proceed? BACKGROUND INFORMATION Interview date: Enumerators name: Respondent’s name: Religion of participant: Gender of the participant: Male [ ] Female [ ] Age of participant: Community Name: Role in the community Head of house [ ] Religious leader [ ] Women leader [ ] Youth leader [ ] CRC member [ ] None [ ] Status in the community IDP [ ] Returnee [ ] Host community [ ] Language in which narrative interview was undertaken: 47 Evaluation Objectives Questions Probes (a) Assess the program’s achievements of the program in relation to the goal, objectives, results and targets - Do you think REACH 4 has helped improve food security in this community? If yes, how? (Access to food to eat daily) - Do you think there is an improvement in the nutrition of the people in your community due to the REACH 4 project? (Access to nutritious food) - Is there a change in the attitude to nutrition in the community, especially pregnant and lactating mothers due to the REACH 4 project? - How has other complimentary programmes such as the backyard garden, poultry farm improved the livelihood and nutrition of the people? - How has the fuel efficient stoveshelped the livelihood, food security and nutrition of the people in your community? - Do people now have access to food to eat? - Do you think people in your community now have access to improved nutrition? (If yes, what makes you think so?) (b) Evaluate the program’s effects on the local markets, and how it affected certain groups of interest (women, men, youth, boys and girls) - In what way has REACH 4 had effect on the local market in your community - Has it increased or decreased demand of food and other complementary commodities in the market - Has it increased or decreased demand of food and other complementary commodities in the market - Are food and other commodities now readily available in the market unlike before - How about the price of commodity, are there 48 - How has the REACH 4 programme affected women, men, youth, boys and girls in your community? changes due to the Reach 4 programme (c) Evaluate the effectiveness and relevance of the modality, transfers and complementary interventions to achieve program outcomes - What relevant changes has the program brought in terms of improving household food consumption, and reducing household hunger? - What change has the program brought with regards to affordability of food, can beneficiaries buy food regularly? - How relevant are the activities of REACH 4 to the community? Does it meet the pressing need of the community? - How has your role as market vendor improve the effectiveness of the REACH 4 activities? (d) Assess the sustainability of its results, - In what ways can the result of the REACH 4 be sustained when the project is over? (Speak for yourself) - Have you had access to investments in the past? If yes, how has the project helped you leverage on the activities to achieve sustainability for your market? Access to food, safe hygiene practices, nutritious food, Fuel￾efficient stoves etc (e) Identify best practices, lessons learned, strengths and challenges in program design, including log frame, and implementation of achieving project achievements. - How well do you think this project has done in meeting the nutritional needs of the people? - How have the mass sensitization rallies and the vendors shop nutrition promotion during voucher distributions with informational banners led to changes in participants’ food selection pattern? - How impactful is the voucher system? - In your opinion, what are some of the - What do you like most about the project? - What don’t you like about the project? - How can the food intervention be improved in 49 challenges against the programme and how can the project be improved? the future? ANNEX 3.4: KEY INFORMANT INTERVIEW GUIDE (WITH LEADERS) Please use this guide to facilitate interviews with market vendors. This guide will elicit questions on perspectives and experiences of the Responsive Economic Assistance to Conflict affected Household (REACH) 4. Participants should be thoroughly probed for information. Enumerators should start by introducing themselves and a brief about the project. They should collect the information needed in the table below before proceeding with the questions. My name is {enumerator}, I work with Mercy Corps on the REACH 4 Project. I am here to conduct an interview on the support provided to you. Do you give me consent to proceed? BACKGROUND INFORMATION Interview date: Enumerators name: Respondent’s name: Religion of participant: Gender of the participant: Male [ ] Female [ ] Age of participant: Community Name: Role in the community Head of house [ ] Religious leader [ ] Women leader [ ] Youth leader [ ] CRC member [ ] Local government official [ ] None [ ] 50 Status in the community IDP [ ] Returnee [ ] Host community [ ] Language in which narrative interview was undertaken: Evaluation Objectives Questions Probes (a) Assess the program’s achievements of the program in relation to the goal, objectives, results and targets - Do you think REACH 4 has helped improve food security in this community? If yes, how? (Access to food to eat daily) - Do you think there is an improvement in the nutrition of the people in your community due to the REACH 4 project? (Access to nutritious food) - Is there a change in the attitude to nutrition in the community, especially pregnant and lactating mothers due to the REACH 4 project? - How has other complimentary programs such as the backyard garden, poultry farm improved the livelihood and nutrition of the people - How has the fuel-efficient stoves helped the livelihood, food safety and nutrition of the people in your community? - Do people now have access to food to eat? - Do you think people in your community now have access to improved nutrition? (If yes, what makes you think so?) 51 (b) Evaluate the program’s effects on the local markets, and how it affected certain groups of interest (women, men, youth, boys and girls) - In what way has REACH 4 had effect on the local market in your community - How has the REACH 4 programme affected women, men, youth, boys and girls in your community? ● Has it increased or decreased demand of food and other complementary commodities in the market ● Are food and other commodities now readily available in the market unlike before ● How about the price of commodity, are there changes due to the Reach 4 programme (c) Evaluate the effectiveness and relevance of the modality, transfers and complementary interventions to achieve program outcomes - What relevant changes has the program brought in terms of improving household food consumption, reducing household hunger scores and Improving coping strategies in reoccurring conflicts? - Are the REACH 4 activities relevant to the context of your community (food vouchers, food type, Fuel-efficient stoves, backyard garden) - Did the intervention get to the appropriate target groups and individuals within the target areas? Are the interventions appropriate and effective for the target group based on the nature of their vulnerabilities? - Does it meet the pressing need of the community? (d) Assess the sustainability of its results, - In what ways can the result of the REACH 4 be sustained when the project is over? (Speak for yourself) - What sustainability practices have you observed among the beneficiaries? - In what ways would you contribute Access to food, safe hygiene practices, nutritious food, Fuel efficient cooking etc 52 towards the sustainability of the REACH 4 project results? (e) Identify best practices, lessons learned, strengths and challenges in program design, including log frame, and implementation of achieving project achievements. - How well do you think this project has done in meeting the nutritional needs of the people? - How have the mass sensitization rallies and the vendors shop nutrition promotion during voucher distributions with informational banners led to changes in participants’ food selection pattern? - How effective has, the Mother Support Groups been in passing nutritional massages? - How impactful is the voucher system? - In your opinion, what are some of the challenges against the programme and how can the project be improved? - What do you like most about the project? - What don’t you like about the project? - How can the food intervention be improved in the future? ANNEX 3.5: FOCUS GROUP DISCUSSION GUIDE (WITH PROJECT BENEFICIARIES) Please use this guide to facilitate discussion with project beneficiaries. This guide will elicit questions on perspectives and experiences of the Responsive Economic Assistance to Conflict affected Household (REACH) 4. Participants should be thoroughly probed for information. Enumerators should start by introducing themselves and a brief about the project. They should collect the information needed in the table below before proceeding with the questions. My name is {enumerator}, I work with Mercy Corps on the REACH 4 Project. I am here to moderate a discussion on the support provided to you. Do you give me consent to proceed? BACKGROUND INFORMATION Interview code: Interview date: 53 Interviewer’s name: Interview start time Interview end time Category of participant: ( ) Head of Households ( ) Women of Reproductive Age (15-45); including pregnant women and lactating mothers of children under 5 years ( ) Individuals responsible for household Cooking and caregivers Local Government Area: Community: Language in which narrative interview was undertaken: Evaluation Objectives Questions Probes (c) Assess the program’s achievements of the program in relation to the goal, objectives, results and targets - Tell me about the REACH 4 project? - How has the project helped you (and your child) in improving access to nutritious food? - How has REACH 4 ensured your household was able to prepare staple and fresh food safely and hygienically through provision of fuel-efficient stove? - - Do you think people in your community now have access to improved nutrition? (If yes, what makes you think so?) (d) Evaluate the program’s effects - What changes has the program brought - Can you now buy food regularly 54 on the local markets, and how it affected certain groups of interest (women, men, youth, boys and girls) with regards to affordability of food? - What change has the program brought with regards to availability of food in the market? - What support is being provided to ensure men/women/adolescents/people with disability groups participate in the project? (Kindly share experiences) and when you need it? - Is your choice of food usually available in the market? (c) Evaluate the effectiveness and relevance of the modality, transfers and complementary interventions to achieve program outcomes - What relevant changes has the program brought in terms of the household food consumption? Do you now eat quality meals more frequently? - Do you recall being asked for feedback to improve project effectiveness. - Can you share areas of improvement the project has made based on your feedback? - How has the nutrition activities (e.g., sensitization) impacted your nutritional outcomes? - What is your opinion about how effective the Mother Support Groups have been in passing nutritional massages? (d) Assess the sustainability of its results, - How has the project prepared you to sustain your access to nutritious food? - How would you ensure the results of the intervention be sustained after the completion of this project? - What would you do to continue growing crops, manage poultry and maintain the use of the fuel￾efficient stoves given to you? (e) Identify best practices, lessons learned, strengths and challenges in program design, including log - How well do you think this project has done in meeting your nutritional needs? - In your opinion, what are some of the things that should be improved on this project? - How have the mass sensitization rallies - What do you like most about the project? - What don’t you like about the project? 55 frame, and implementation of achieving project achievements. and the vendors shop nutrition promotion during voucher distributions with informational banners led to changes in your food selection pattern? - Can you describe briefly how the voucher system has been impactful? What lessons can be drawn from the voucher system? - How has the items supplied through the voucher meet your demand for food? 56 ANNEX4:PHOTOS Photo 1: Household use of fuel-efficient stoves