DEMOCRATIC REPUBLIC OF CONGO Ituri Province: February to June 2022 16 June 2022 FINAL REPORT FINAL EVALUATION: PROVIDING HUMANITARIAN ASSISTANCE FOR SELF-RELIANCE IN EASTERN CONGO (PHASE) This evaluation is made possible by the generous support of the American people through the United States Agency for International Development (USAID). The contents are the responsibility of Mercy Corps and do not necessarily reflect the views of USAID or the United States Government. Acknowledgements: We would like to express our gratitude to the team of local enumerators, whose dedication and courage during the data collection phase made this evaluation possible. We also want to express gratitude to Mercy Corps staff for their assistance in this evaluation. Published by Mercy Corps 45 SW Ankeny St. Portland, OR 97204 USA Website: www.mercycorps.org June 2022 © Mercy Corps 2022. You may copy, distribute, display, download and otherwise freely deal with this work for any purpose, provided that you attribute Mercy Corps as the owner. Disclaimer This publication does not necessarily reflect the policy position of Mercy Corps, Socorro: Global Humanitarian Consultants or any affiliated organization. The information in this publication was based on available information at the time of preparation. No responsibility is accepted by Socorro: Global Humanitarian Consultants or any affiliated organization for any errors or omissions contained within this publication. Mercy Corps PHASE/Final Evaluation Internal Socorro i 16 June 2022 Global Humanitarian Consultants TABLE OF CONTENTS I. INTRODUCTION AND PROJECT BACKGROUND 2 II. EVALUATION PURPOSE AND SCOPE 3 III. METHODOLOGY & LIMITATIONS 3 A. Evaluation design 3 B. Data Sources 4 C. Data Analysis 5 D. Ethics & Accountability 6 E. Limitations 7 IV. FINDINGS 9 A. Food Security and Resilience 9 B. VSLAs, Food Security, Self-Sufficiency 24 C. Agricultural training, Provision of Seeds and Tools 25 D. Complementary Activities & Economic Opportunities for Women and Young Adults 26 E. Limiting Factors 27 F. To what extent are the net benefits of the PHASE programs likely to continue? 30 G. Mercy Corps' Perception of the PHASE program 32 H. Lessons Learned & Recommendations 34 V. CONCLUSIONS AND RECOMMENDATIONS FOR MANAGEMENT ACTION 36 VI. EVALUATION FRAMEWORK 37 VII. SOURCES OF INFORMATION 40 A. Desk review material 40 B. Qualitative Data Sources: KIIs and FGDs 42 VIII. SOW 43 IX. DATA COLLECTION INSTRUMENTS 48 A. KII and FGD Evaluation Questionnaires 48 B. Online Survey (MC Staff) 55 Mercy Corps PHASE/Final Evaluation Internal Socorro ii 16 June 2022 Global Humanitarian Consultants Acronyms and Abbreviations AAP Accountability to Affected Populations CBA Cash-Based Assistance C-HAT Congo Humanitarian Analysis Team CVA Cash and Voucher Assistance CW Concern Worldwide CFW Cash for Work CWG Cash Working Group DRC Democratic Republic of the Congo EWS Early Warning System FARDC Forces Armées de la RDC (DRC Armed Forces) FCS Food Consumption Score FEC Fédération des Entreprises du Congo FGD Focus Group Discussion FS Food Security HRP / HNO Humanitarian Response Plan / Humanitarian Needs Overview IDP Internally Displaced Person IPC Integrated Food Security Phase Classification KII Key Informant Interview KCRI Kivus Crisis Response for IDPs MC Mercy Corps MEB Minimum Expenditure Basket MEL Monitoring, Evaluation and Learning MONUSCO United Nations Organization Stabilization Mission in the DR Congo MIRA Multisectoral Initial Rapid Assessment NFIs Non-Food Items PDM Post Distribution Monitoring OECD/DAC Organisation for Economic Co-operation and Development/Development Assistance Committee rCSI Reduced Coping Strategy Index REACH Responding with Emergency Assistance to Communities and Displaced Households in North Kivu RRMP Rapid Response to the Movement of Population SAM Severe Acute Malnutrition SOPs Standard Operating Procedures TMB Trust Merchant Bank UCT Unrestricted Cash Transfer VSLA Village Savings and Loan Association WASH Water, Sanitation and Hygiene WG-RR Working Group Rapid Response Mercy Corps PHASE/Final Evaluation Internal Mercy Corps 1 PHASE/Final Evaluation Internal Executive Summary Overall, the PHASE program positively impacted the lives of program participants. Food fairs and cash distributions had a significant and immediate impact in reducing the food insecurity of PHASE participants. Village Savings and Loan Associations (VSLA) and agriculture activities enhanced the dignity of those who participated in them, as well as contributing to participants' longer-term resilience and well-being. The extension of these activities to a great number of program participants would have great benefit to the targeted communities. In terms of Cash, Voucher or Food assistance, Cash was the clearly preferred option; however, future programs may benefit from augmenting the amount distributed to each household and considering the positives and negatives of multiple rounds of Cash distributions. The success of the PHASE program was tempered by insecurity, which prevented beneficiaries in two key areas from fully benefiting from all distributions and activities. Future early recovery programs in conflict settings may benefit from a more critical evaluation of program deficiencies by program staff, inclusion of a long-term, all-of-family behavioural change component, and a more robust communication strategy with targeted communities. Conclusions 1. A focused and thoughtful design, the PHASE program was well aligned to the needs of the vulnerable people of Ituri. The FS, VSLA, and agricultural trainings were well conceived and of great and immediate benefit to all participants. To a broad degree, they met the initial intentions. 2. The activities of the PHASE program increased individual and household resilience in the short- and medium-terms. 3. The PHASE program revealed there to be significant opportunities to positively affect youth participation and leadership. While this was not fully realized in the PHASE program, future, youth-targeted activities could significantly enhance the resilience and food security of targeted communities in the medium- to long-term. 4. The PHASE Program achieved noticeable success in empowering women; however, this increase in women’s agency was not positively perceived by many men. 5. PHASE programs significantly delivered on their original goals in terms of accountability to affected communities. 6. The program had mixed impact on local markets; success of food fairs were affected by supply chain and quality control issues. PROJECT DETAILS PROJECT NAME: Providing Humanitarian Assistance for Self￾Reliance in Eastern Congo (PHASE) PROJECT LOCATIONS: Aungba, Mwanga, Boga, Nyakunde, Djaiba, Djalasiga START DATE: 28 September 2020 END DATE: 31 May 2022 DONOR: USAID's Bureau of Humanitarian Assistance TOTAL BUDGET: $10,000,000 USD THEMATIC AREAS: Food Security, Agriculture, Village Savings and Loan Association ESTIMATED BENEFICIARIES (INDIVIDUALS): 108,600 (51,042 women, 57,558 men) Mercy Corps PHASE/Final Evaluation Internal Mercy Corps 2 PHASE/Final Evaluation Internal 7. Despite the endemic insecurity and unclear exit and transition plans, ultimately the PHASE Program contributed to short-term positive impact for community members. Insecurity impacted the medium-term sustainability of the program. However, program beneficiaries will continue to apply the knowledge, skills, and confidence gained via participation in PHASE programs long after the closure of Mercy Corps’ PHASE activities. Recommendations 1. CONTINUE and EXPAND Food Security and Agricultural training programs, with an increased emphasis on cash (vs vouchers) and on high yield, small plot cultivation programs1 (permagardens), respectively. 2. MAINTAIN and EXTEND VLSA schemes: consider the inclusion of men/ separate schemes for men. 3. REVIEW and REDESIGN future early recovery programs' commitment to include: a. Pre- and post-intervention price monitoring, b. Social behaviour change as a cross-cutting theme (e.g., positive masculinity), c. Increased focus on targeting persons living with disabilities and those caring for them,2 d. Pest mitigation, e. All participants benefit from agriculture trainings, f. Provision of NFIs (including tools3 ), g. Shelter assistance. 4. SUSTAIN efforts to increase the prominence of the voice of women, including those with a disability, and to positively effect youth leadership. Brainstorm about more substantive ways these two activities can be implemented despite ongoing insecurity. 5. INCREASE awareness of the existence of program criteria among program participants. Actively ensure that all pertinent parties know of criteria and distribution dates. I. Introduction and Project Background The Democratic Republic of the Congo (DRC) continues to face an acute and complex humanitarian crisis. Currently, there are an estimated 5 million internally displaced persons (IDPs) in DRC, 1.5 million of whom are in Ituri Province. Over a quarter of the total population of DRC is food insecure—the second highest rate in the world. Mercy Corps' Providing Humanitarian Assistance for Self-Reliance in Eastern Congo (PHASE) program— financed by USAID's Bureau for Humanitarian Assistance (BHA) for $10,000,000 USD—was designed to support 102,300 individuals (IDPs, returnees and vulnerable host families) affected by the armed conflict 1 For those who are unable to access large areas of land to cultivate due to continuing insecurity. 2 As per the 7th criteria of the social vulnerability category of the Cash and Voucher Assistance (CVA) Program: "Mercy Corps Emergency Application for DRC – PHASE – Revised Submission. September 3, 2020", (p8). 3 Participants in Mercy Corps' PHASE program benefited from a $35 USD voucher to purchase "certified" seeds and tools "seed and agricultural tools fairs" in accordance to the approach recommended by the Food Security Cluster regarding agricultural activities in Eastern DRC. However, program participants consistently expressed to the consulting team that a dearth of agricultural tools was a factor that impinged upon their capabilities to reduce their food insecurity. Figure 1: Food Fair Mercy Corps PHASE/Final Evaluation Internal Mercy Corps 3 PHASE/Final Evaluation Internal in Ituri Province in Eastern Democratic Republic of the Congo. The objective of the PHASE program was to reduce the vulnerability of these households to conflict-caused shocks that negatively impacted their capacity to cover basic food needs, while progressively increasing options for self-reliance. The theory of change (ToC) buttressing the PHASE program is that if food insecure households are supported with food and early recovery assistance they will emerge safely and with dignity from crises. The principal activity to bring about this change was the distribution of Unrestricted Cash Assistance (UCT) and/or Restricted Food Vouchers; this program was complemented by two additional activities: Agricultural Training and Provision of Seeds to 1,050 food insecure households, and support for the creation of 14 community-level savings and lending groups (VSLA) with host-community members, with a focus on women's participation. II. Evaluation Purpose and Scope The purpose of the evaluation is to assess the relevance of PHASE's strategy and to evaluate to what extent? and for whom? and in what circumstances? and why? PHASE activities contributed to achieving the project's objectives. The evaluation will identify lessons learned and make recommendation to improve future early recovery programs in contexts defined by conflict and high population mobility. The evaluation covers the entire PHASE implementation period from September 2020 to May 2022 carried out by Mercy Corps' PHASE program in Ituri Province in eastern DRC. The audience of the evaluation is Mercy Corps staff and relevant USAID BHA program officers. III. Methodology & Limitations A. Evaluation design The Evaluation design followed the Scope of Work and was based on a mixed evaluation approach to measure the performance of the project against its selected indicators through ongoing quantitative analysis (“Component 1”) reinforced by a qualitative analysis of the contributions of the project to the observed results (“Component 2”). The quantitative component was conducted internally by Mercy Corps' emergency monitoring, evaluation and learning (MEL) team members through the use of rolling endline surveys (or enhanced post-distribution monitoring - PDMs) following the end of each intervention in order to provide statistical data relating to the project’s logical framework indicators. The qualitative component was conducted externally by a group of international and national consultants, and focused on pre-selected intervention zones only (see Section E. Limitations). This approach enabled the triangulation and synthesis of both primary and secondary data generated through multiple sources and via various methods. The final report provides a comparative analysis of the quantitative data and other sources provided by Mercy Corps (i.e., PDM reports, tools, etc.), and of the qualitative data collected during field trips to project areas in Ituri Province. The comparison was stated in the Evaluation Grid (see Appendix I) that categorized the evaluation questions according to the following OECD/DAC criteria: 1. RELEVANCE—is the intervention doing the right things? 2. EFFECTIVENESS—is the intervention achieving its objectives? What changes have been achieved? 3. IMPACT—what difference does the intervention make? What external and internal factors contributed to/ limited the results? Mercy Corps PHASE/Final Evaluation Internal Mercy Corps 4 PHASE/Final Evaluation Internal 4. COHERENCE—how well does the intervention fit? 5. SUSTAINABILITY—will the benefits last? The evaluation sub-questions were developed and workshopped with the consultancy team's FSL expert and PHASE Steering Committee members. Different questionnaires were developed according to the relevant stakeholder groups. 1. Sampling Methods & Sample Size The quantitative data collection followed Mercy Corps internal rules and guidelines. The results were provided in form of aggregated PDM reports and an updated Indicator tracking table. Qualitative data was gathered from key stakeholders (e.g., beneficiaries, non-beneficiaries, local partners, suppliers, and authorities) in Nyakunde, Djalasiga and Djaiba. 4 Qualitative data was gathered via Key Informant Interviews (KII), Focus Group Discussions (FGD) and direct observation from a sampling of all relevant parties in these areas. Given the time and budget constraints under which the evaluation was subject to, stratified, purposive sampling was used to identify the niche demographics required to obtain the specific data needed for this research. These sampling techniques allowed the evaluation team to gather qualitative responses from the best-fit participants who provided valuable insights and precise and relevant research results. A representative sample of participants from each profile (gender, age, diversity) and geographical location was selected by Mercy Corps staff to take part in this evaluation. The sample of participants represents the target population and the key actors involved in PHASE activities; therefore, the generalisability of our findings is deemed to be high. Details can be found in the section below. Similarly, the Mercy Corps' steering committee provided the evaluation team with a library of documents relevant to the operations; both qualitative and quantitative data were derived from a review of these documents. An analysis of these documents informs the findings: a list of these documents can be found in the Appendix II. B. Data Sources 1. Primary and Secondary Data Collection and Sources Data collection took place between February - April 2022. The evaluation was conducted using a mixed methods approach: data sources included a review of documents, a stakeholder mapping, semi-structured key informant interviews (KIIs) and focus group discussions (FGDs), an anonymous online survey, and a workshop/webinar where the evaluation team shared their preliminary findings. DATA COLLECTION & SOURCES Document analysis 18 Internal documents were reviewed (for details, see Appendix II. Sources of Information). Semi structured KIIs 105 semi-structured interviews were conducted with program beneficiaries, suppliers, partners, community leaders and community members in Nyakunde, Djaiba and Djalasiga (for details, see Appendix II), as well as Mercy Corps staff and partners in Bunia and Goma. 4 The endemic insecurity in Ituri province that played a significant role in making this area particularly food insecure also limited the ability of the evaluation team to collect qualitative data in all PHASE project areas Mercy Corps PHASE/Final Evaluation Internal Mercy Corps 5 PHASE/Final Evaluation Internal Focus Group Discussions 26 FGDs were conducted with 312 relevant persons in Nyakunde, Djaiba and Djalasiga. These included: program beneficiaries, community leaders and community members (for details, see Appendix II). Quantitative and qualitative data Quantitative and qualitative data was extracted from Mercy Corps' PHASE Program PDMs and an anonymous survey taken by 41 Mercy Corps staff (program and non-program staff identified by Mercy Corps). Webinar/Workshop sharing of findings Workshop conducted in April 2022 with key PHASE operation members. 2. Qualitative Data Collection Tools The quantitative data collection tools are standardized by Mercy Corps MEL Department and were provided to Socorro Global Humanitarian Consultants for review. The qualitative data collection tools (e.g., questionnaires, observation sheets) were developed based on the Evaluation Grid and approved by Mercy Corps MEL Department. These were translated to Swahili and Lingala by the DRC-based consultants during a translation workshop.5 The data collection team assigned interviews according to areas of expertise and followed four guidelines: a) focus group discussions were carried out by two people; b) female beneficiaries (whether in a group or individually) were to be interviewed exclusively by female members of the team; c) in terms of personal data, only gender, age and number of participants (for focus group discussions) was collected; d) obtaining informed consent was before conducting any interview. After receiving training in how to conduct KIIs and FGDS, the data collection team (four DRC-based consultants working under the guidance of an international consultant) collected qualitative data using a paper-based, stakeholder-specific set of evaluation questions (see Appendix IV). Upon receiving consent from the interviewee(s), an audio of each KII and FGD was recorded. When typing up their notes from each interview, the enumerating team reviewed the audio recording to ensure the accuracy of the qualitative data recorded. Data from each KII and FGD was then entered into a MS Word document and saved electronically. Additionally, an online anonymous survey for Mercy Corps staff, with closed and open-ended questions, was developed using MS Forms (see Appendix IV). The survey was shared by Mercy Corps internally6 with program and non-program7 staff who had worked or had experience with the PHASE program. C. Data Analysis The quantitative data provided contained in Mercy Corps' Post-Distribution Monitoring (PDM) reports was analysed using descriptive and comparative methods. This analysis was conducted via secondary sources (the primary data sets were not available to the consultants) and served as a basis for the more detailed analysis, responding to “What happened?” or “What is/ was the situation?” and "What were the results?" The transcripts developed from the audio recordings of the KIIs and FGDs, combined with the notes taken during the interview, were crosschecked, translated into French and English, and along with 5 The data collection was conducted in French, Swahili, Lingala and Alur. The data collection team collected information in the preferred language of the counterpart(s), and then provide transcription in French. 6 The evaluation team was not provided with the complete list of people who received the online survey. 7 Non-program staff refers to all Mercy Corps staff that did not directly work on the PHASE program , such as MEL staff, PAQ staff, logistics, finance, operations, security or staff of other programs (i.e., SAFER, etc.). Mercy Corps PHASE/Final Evaluation Internal Mercy Corps 6 PHASE/Final Evaluation Internal relevant program documents, were uploaded to Dedoose. The qualitative data analysis was conducted using Dedoose8 software in the following manner. Pertinent sections of transcripts and relevant documents were tagged with individual codes derived from the conceptual framework established in the Evaluation Grid. Data was filtered both according to sub-group of data source (e.g., gender, stakeholder position, document, etc.) and thematic and programmatic criteria. This enabled the identification and analysis of trends, commonalities, discrepancies and differences across the totality of all qualitative data collected according to the frequency patterns of coded content. D. Ethics & Accountability 1. Research Ethics The qualitative data collection team consisted of an equal number of men and women, all trained in the basic safeguards of conducting ethical research, as per Guidance on Safeguarding in International Development Research.9 The team leader monitored the entirety of the qualitative data collection process to ensure that any safeguarding, ethical, protection or gender-based violence issues that might arise during the qualitative data collection process would be immediately addressed/ reported to the appropriate personnel or via the appropriate structure. All key informants and participants in focus group discussions gave their informed consent for their responses to be recorded prior to the commencement of each KII, FGD; additionally, this extended to first hand observations conducted during visits to the field. This consent was asked for and given in the preferred language of the key informant; and, working with translators, all interviews and focus group discussions were conducted in the preferred tongue of the key informant (i.e., Lingala, French, Alur or Swahili) as appropriate. Transcripts that were developed from KIIs and FGDs were de-identified and each was assigned a Unique Identifier Code to preserve their anonymity. A secure system was used to safeguard all information gathered over the course of the evaluation and the evaluation adhered to Mercy Corps’ data protection and privacy policies. Mercy Corps staff contributed to the qualitative data gathered over the course of the evaluation both via Key Informant Interviews and through participation in an online survey. The identity of key informants in both cases was protected: they are only referred to as “program” or “non-program” staff. No minors were involved in the collection of data for this evaluation. 2. Community Perspectives & Accountability Community perspectives were gathered via multiple FGD and KIIs conducted with a representative sampling of key stakeholders from targeted communities. "Gender", "age" and "type of assistance received" were the basic selection criteria employed to select key informant groups for participation in KIIs and FGDs. KIIs were conducted with one or two people, while FGDs contained up to a maximum of 12 persons. 10 8 A cross-platform application for analysing qualitative and mixed methods: https://www.dedoose.com/. 9 https://safeguardingsupporthub.org/documents/guidance-safeguarding-international-development-research 10 The FGD were conducted in places chosen jointly by the Mercy Corps staff accompanying the evaluation teams and local leaders. On several occasions, it was not possible to keep the maximum number of 12, and in that case, the evaluation team enable more community members within the given gender and age group to participate, changing the overall methodology from FGD towards townhall meeting. Mercy Corps PHASE/Final Evaluation Internal Mercy Corps 7 PHASE/Final Evaluation Internal The evaluation findings have been shared with and validated by a range of key stakeholders at two distinct points during the course of the evaluation: firstly, during a workshop/webinar during which the preliminary findings were shared with many relevant stakeholders; and secondly, the first draft of the Final Report was shared with the steering committee and many relevant PHASE stakeholders who reviewed and provided feedback on the evaluation's findings. This feedback was addressed and incorporated into the approved final version of the Final Report. E. Limitations This evaluation faced multiple limitations: ▪ Insecurity prevented the evaluation team from collecting end-line qualitative data in all PHASE project areas (i.e., Boga and Mwanga). This limitation was mitigated by analysing information gathered via PDMs in project areas the evaluation team was unable to access. ▪ In the communities of Djalasiga and Aungba, the population had recently been affected by a new wave of insecurity and was largely displaced from their original areas. This limitation was mitigated by orienting the questions to the past (to discuss the assistance prior to displacement); however, their displacement meant that it was impossible for the evaluation team to triangulate the responses of interviewees from these areas with direct observation. ▪ Data gathered during the FGDs and KIIs conducted during the field's visits may have been subject to observation bias. Given the time-, security- and logistics-constraints the evaluation operated under, all participants were identified by Mercy Corps' Staff based on criteria provided in advance by the evaluation team; 11 and even though members of the consulting team indicated their independence from Mercy Corps and guaranteed that everything they said would be de￾identified and their anonymity guaranteed, it is not clear that the consultants were seen as independent by all key stakeholders. ▪ Due to the aforementioned time-, security- and logistics-constraints, and to the compressed period of time the evaluation team was able to spend at each site location, the evaluation team was unable to speak with target people with special needs (such as persons living with disabilities, vulnerable women, etc.). Among the beneficiaries and non-beneficiaries randomly selected by Mercy Corps staff, these groups were underrepresented. ▪ Certain questions in the online survey taken by Mercy Corps staff were not correctly understood and answered (e.g., triangulation between the Likert scale response and narrative response to the same question revealed an occasional lack of correspondence), and thus some answers had to be deleted during the data cleaning process. ▪ Quantitative data collection was not complete at the time of this evaluation; this hindered the ability of the evaluation team to make substantive comparisons between program sites. 12 As it is difficult to compare quantitative with qualitative findings with incomplete quantitative data: this 11 Mercy Corps provided a list of key informants/key stakeholders that was used by the consultants to identify randomly some of the KIIs, however, due to high mobility of people and limited time spent in the project sites, this was not always possible. 12 Specifically, at the time of evaluation, results of all three rounds of distribution were only available for the interventions in Aungba. The other sites (Nyakunde – displaced; Nyakunde – returned; Djaiba, Mwanga, Boga) where three rounds of distribution were planned, only two (Nyakunde – returned) or one round (Nyakunde – displaced, Djaiba, Mwanga) had been evaluated by time this evaluation was conducted. Mercy Corps PHASE/Final Evaluation Internal Mercy Corps 8 PHASE/Final Evaluation Internal permitted only a limited analysis in trying to gauge the extent of the impact of three rounds vs one round of food security interventions. Individually and cumulatively, these limitations may have impinged upon the depth and breadth of the evaluation, with potential corresponding gaps or oversights in analysis. This was especially true when analysing the complementary VSLA and agricultural programs: no supporting documents (i.e., reports, PDMs) were provided to the evaluation team that could have contextualized program achievements against their target indicators. It was possible for the evaluation team to visit only one site where VSLA and agricultural activities were implemented; therefore, the comparative analysis of the quantitative and qualitative findings in these two program areas remains limited. Mercy Corps PHASE/Final Evaluation Internal Socorro 9 16 June 2022 Global Humanitarian Consultants IV. Findings A. Food Security and Resilience The table Suivi des Indicateurs presented in this section was shared by Mercy Corps prior the field evaluation, the most updated version (presented in this report) was downloaded on May 16, 2022. An analysis of the quantitative data shows the PHASE program to have significantly and positively reduced the food insecurity of program participants. These positive results are especially clear when examining the results of the PHASE program via the Reduced Coping Strategies Index (rCSI): the target was achieved across all interventions. Overall, PHASE activities generated a perception of positive change in the households of female beneficiaries (with the exception of interventions in Nyakunde),13 and largely improved the Household Dietary Diversity Score (HDDS). The improvements in HDDS were not universal, however; dietary diversity decreased in Boga, and Mwanga, and in Djaiba the observed change did not meet the target. This table was provided by Mercy Corps MEL department, and represents the aggregated data per Indicator Table.14 Reduced Coping Strategies Index (rCSI) Percentage of households with poor, borderline, and adequate Food Consumption Scores (FCS) Average % change in Household Dietary Diversity Score (HDDS) % of female participants reporting positive HH Δ target:<20 Target: Poor: 20%; Borderline: 50%; Acceptable: 30% target: 30% target: 80% Baseline Endline Baseline Endline Baseline Endline Δ % Intervention Activity # # Poor Borderl ine Acceptable Poor Borderl ine Acceptable # # % Boga Distribut ion 24.38 11 86% 12% 2% 49% 3% 48% 3.16 1.34 -57% 95% Aungba 1 Cash/A gri/VSL A 17.6 10 72% 27% 1% 15% 48% 36% 3.7 6 62% 92% Mwanga 1 Cash/A gri/VSL A 20 15 67% 26% 8% 37.72% 49% 13% 5 3 -40% 96.41% Aungba 2 Cash/A gri/VSL A 17.6 7 72% 27% 1% 18% 72% 10% 3.7 7 89% 91% 13 In Nyakunde, two separate groups were targeted: returned population (marked as “R”), and displaced population (marked as “D”). 14 This table does not provide the disaggregation stated in the Indicator Table (see the Annex). It represents a snapshot of progress as recorded in the ITT at the outset of the evaluation, meaning that not all data are final. Mercy Corps PHASE/Final Evaluation Internal Socorro 10 16 June 2022 Global Humanitarian Consultants Nyakundé -R-1 Distribut ion 23.62 3 86% 14% 0% 46% 42% 11% 4.27 6 41% 29% Aungba 3 Cash/A gri/VSL A 17.6 2 72% 27% 1% 0% 18% 82% 3.7 8 116% 98% Nyakundé-R-2 Distribut ion 23.62 4 86% 14% 0% 38% 48% 14% 4.27 8 87% 53% Nyakundé-D-1 Distribut ion 24.00 1.18 86% 14% 0% 43% 47% 10% 4 7 75% 73% Djaiba-1 Distribut ion 14 7 92% 7% 0 34% 51% 16% 4 5 25% 82% In general, PHASE activities did not meet program targets in the frequency of different food groups consumed by participating households. FCS targets (decreasing poor and borderline FCS scores, increasing adequate FCS scores) were achieved by only two interventions: Aungba 1 and Aungba 3, where program participants benefited from the full complement of three rounds of assistance. However, it is important to note that while overall PHASE activities did not achieve FCS program targets, the PHASE program largely succeeded in increasing the frequency of consumption of different food groups in participant households: all communities that benefitted from PHASE activities registered substantive improvements in their capabilities to consume different food groups with greater frequency. The PHASE program showed demonstrable progress in positively advancing beneficiaries along the FCS scale, e.g., from poor to borderline, and from borderline to adequate), and therefore an additional indicator of change should be envisaged (as proposed in the table below for Boga and Nyakunde R2). baseline endline % change baseline endline % change baseline endline % change poor Borderline acceptable Boga 86% 49% 37% 12% 3% 9% 2% 48% 46% Nyakunde R-2 86% 38% 48% 14% 48% 34% 0% 14% 14% This table shows that while PHASE activities did not achieve the targeted percentage of FCS change in either Boga or Nyakunde R2, they significantly and positively impacted food security in these communities by moving people towards a more food secure category. While a collection of data according to program indicators is valuable, it is only the first step in understanding "how?" and "for whom?" and "in what ways?" the extent to which the PHASE program did or did not succeed. The PDMs demonstrate a dearth of cross-evaluation across indicators and secondary analysis. For example, in the case of Boga, not only were FCS targets not reached, even though positive changes occurred (see above), but HDDS decreased. Discussions with the key informants captured in the Boga PDM attributed the poor HDDS results with insecurity in the area Mercy Corps PHASE/Final Evaluation Internal Socorro 11 16 June 2022 Global Humanitarian Consultants that prevents certain food stuffs from reaching the local markets.15 This assertion is curious given the high correlation between FCS and HDDS: it is rare to see a positive change registered in one indicator while showing a negative change in the other. 16 Furthermore, given the nature of the intervention—food fair—it is striking that HDDS would be negatively impacted by an increase in the availability of food in local markets since the food fair should have provided sufficient food group variety. This table was provided by Mercy Corps MEL department and represents aggregated data per Indicator table.17 Output 1: Most vulnerable households affected by shocks receive emergency food assistance Output 2: Food insecure households receive agricultural trainings Output 3: Host-community members have access to livelihood opportunities supported through increased access to financial services Number of individuals participating in USG food security programs. Total value (USD) of assistance distributed (restricted/unre stricted) to participants % of female participants who report having felt safe during distribution activities Number of individuals who have received USG supported short term agricultural sector productivity or food security training Percent of participants reporting that techniques helped build their household self-reliance Percent of trained participants on permagarden techniques producing a yield three weeks after the training # of VSLAs created in host communi ties % decrease of average HHS of VSLA members % of participants reporting an expansion of their IGA as a result of their VSLA participation target: 102,300 target: $4,603, 500 USD target: 90% target: 1,050 target: 80% target: 70% target: 14 target: 40% target: 80% (level 1), 60% (level 2) # $ % # % % # Baseline Endline Δ % Intervention Activity # # % Boga Distribution 28,976 $453,559.00 100% n/a18 n/a n/a n/a n/a n/a n/a n/a Aungba 1 Cash/Agri/VSLA 16,669 $382,310.00 99% 378 100% R. IDP 6 3.01 2.95 2% 32% Mwanga 1 Cash/Agri/VSLA 8,498 $136,720.00 100% 0 R. IDP R. IDP 8 R. IDP R. IDP R. IDP R. IDP 15 Note: according to the PDM, Mercy Corps organized a fair in Boga; therefore the qualitative explanation in the PDM seems to contradict the activity narrative. 16 See: https://fic.tufts.edu/assets/Different-Indicators-of-HFS.pdf 17 This table represents a snapshot of progress as recorded in the ITT at the outset of the evaluation, meaning that not all data are final. 18 N/a was used for interventions where an indicator was not collected (e.g., areas where VSLA and agriculture activities were not implemented), R. IDP was used for areas where a given indicator hadn’t been collected at the time of the evaluation. Mercy Corps PHASE/Final Evaluation Internal Socorro 12 16 June 2022 Global Humanitarian Consultants Aungba 2 Cash/Agri/VSLA 16,649 $381,940.00 95% 2,772 100% R. IDP 0 3.01 2.95 2% 32% Nyakundé -R￾1 Distribution 28,538 $466,274.50 99% n/a n/a n/a n/a n/a n/a n/a n/a Aungba 3 Cash/Agri/VSLA 16,662 $381,830.00 99% 2,772 100% R. IDP 0 3.01 2.95 2% 32% Nyakundé-R-2 Distribution 28,522 $466,274.50 100% n/a n/a n/a n/a n/a n/a n/a n/a Nyakundé-D-1 Distribution 24,394 $395,790.50 100% n/a n/a n/a n/a n/a n/a n/a n/a Djaiba-1 Distribution 12,536 $197,981.50 100% n/a n/a n/a n/a n/a n/a n/a n/a Totals $3,262,680 2,772 14 The quantitative data provided by Mercy Corps shows an overall high level of achievement for Output 1: emergency food assistance. Significantly, the PHASE program exceeded program targets in terms of the number beneficiaries reached by at least 5,000. 19 At the time of the evaluation the PHASE program had not reached the program objective in terms of total value of assistance delivered: $1,340,820 remained undistributed. While this most likely is a result of insecurity preventing the full complement of distributions in Djaiba, Boga, and Nyakunde, this warrants further investigation into the reasons behind this significant underspend. Overwhelmingly, female participants in the PHASE program indicated that they felt safe during distributions.20 Although much data that would enable a complete analysis of the complementary Agricultural and VSLA activities (Outputs 2 and 3) is not available, a few observations may be made from that data that is available. The number of VSLAs formed as a result of the PHASE program (14) is equal to that of the program goal, while the number of participants reached with agricultural activities (2,772) exceeded program objectives (1,050). 19 This table does not take into consideration interventions in Djaiba 2 and Muhito which did not have finalized results at the time of this evaluation was finalized. 20 It is to be noted that the questionnaires only allow respondents to say “yes” or “no”, however, does not allowed them to abstain from answering. It would be better to update the possible answers to: "yes," "no," "I do not want to answer," "I do not know," and ask the same question to all beneficiaries and later disaggregate the responses by gender, or to ensure that a statistically significant number of female respondents is achieved. Furthermore, the PDM reports do not specify gender and age of enumerators who administer the questionnaire. They do attest to their “local” affinity, which can impact answers provided. Mercy Corps PHASE/Final Evaluation Internal Socorro 13 16 June 2022 Global Humanitarian Consultants Participants who benefited from the agricultural activities unanimously affirmed that participating in the Agricultural program increased their resilience: this is to be applauded. However, this result is derived solely from an indicator measuring self-reported improvement, which may be subject to bias. Subsequent Early Recovery programs may want to consider including additional indicators of change, such as Reported Agriculture Output,21 Acquired Knowledge/Skill of a specific agriculture technique,22 or Adoption of Promoted Practise23; similar to the indicators that had been planned (but not collected) to monitor progress with permagarden activity. Measuring the increase of knowledge due to training (via pre- and post-tests) may reveal the extent of a program's success to a greater degree than simply listing the number of participants who received trainings. Furthermore, as there is no data available on the permagardens, it is not possible to evaluate to what extent they achieved their desired objective. Given the innovatory design of the permagarden program and its particular suitability for highly insecure areas where access to fields is limited, it is unfortunate that no data is available on this PHASE program component. The VSLAs had not achieved the desired reduction in either the Household Hunger Score (HHS) or the desired increase in Income Generating Activities (IGAs), although since the evaluation was not conducted at the end of the VSLA cycle, it is possible that with more time the VSLAs may yield quantifiable change in the lives of those who participate in them. Given the positive feedback of the communities on the VSLA schemes: they clearly stated the positive impact of VSLAs, indicating that membership in VSLAS gave them the capabilities to start certain types of income generating activities; it may be useful to broaden the indicators24 from purely food security/livelihoods oriented indicators to some that would enable better understanding of realistic expectations of VSLAs over an extended period of time. This information would be valuable if Mercy Corps were to implement similar activities in future Early Recovery programs in insecure areas such as Ituri. In terms of the indicators used to evaluate the activities listed under Outputs 2 and 3, Mercy Corps may want to consider providing supplementary reports that clearly state the methodology of the data collection, main findings, analysis of the results and orientation for the program. Frequency of data collection should be included within this (the vagueness of "end of activity" does not allow for a crisp, time-sensitive snapshot of a program). A few concluding remarks: the PHASE program's quantitative indicators (which are in line with the Food Security Cluster Guidelines)25 are appropriate to assess the performance of PHASE activities. In general, relevant program data is being collected using appropriate tools at the appropriate moment; this is certainly true for emergency food assistance, and, as has been noted previously, is valid to a lesser extent for VSLA and Agriculture activities. However, while the data gathered enables the demonstration of positive achievements, for activities and/ or communities where target goals were not reached, the data gathered is not used to provide an analysis of the factors that impinged upon program achievement. In such instances, the evaluation team could find no evidence that Mercy Corps conducted mid-program investigations into under-performing program activities/ areas. Perhaps that is yet to be done and will result from this final evaluation. The evaluation team could find no evidence that the data gathered via PDMs enabled Mercy Corps to conduct a cross analysis between indicators (e.g., FCS and HDDS); nor do the PDMS contain in-depth analyses in areas/ activities where specific targets were not met (i.e., the relatively low score of women 21 https://www.indikit.net/indicator/4-agriculture-and-nrm/206-reported-agricultural-output 22 https://www.indikit.net/indicator/4-agriculture-and-nrm/213-acquired-knowledge-skills 23 https://www.indikit.net/indicator/4-agriculture-and-nrm/215-adoption-of-promoted-practices 24 Indicators such as: Total Value of Provided Loans, Average Savings of the VSLA members. 25 See: https://fscluster.org/democratic-republic-congo/document/lignes-directrices Mercy Corps PHASE/Final Evaluation Internal Socorro 14 16 June 2022 Global Humanitarian Consultants perceiving positive change in their household in Nyakunde). In certain cases, the explanations provided in PDMs appear to express the opinions of the enumerators or Mercy Corps MEL staff, rather than demonstrating the results of an in-depth investigation into contributing underlying factors. The narrative of the PDMs speak only to the data presented in the tables and graphs; this presents a missed opportunity to gather supplementary, explanatory qualitative data that could deepen the analysis of each PDM. 26 Finally, the Mercy Corps MEL team may want to explore alternate methods of presenting project results that better differentiate between beneficiaries who benefited from one round of assistance versus those that benefited from multiple rounds of assistance, and from those who benefited from multiple activities. This could be achieved by dedicating sheets to each intervention area feeding into one cumulative sheet that provides the program overview. Such an arrangement would allow for a disaggregation of results as stated in Tableau des indicateurs, which determines the disaggregation of results (e.g., sex, age, status or activity), but which is not reflected in the Suivi des indicateurs, thus significantly reducing the potential level of analysis.27 26 It may add value if the indicator scores were evaluated against the thresholds as indicated in Table 14 of the Cluster Guidelines. This may enable a quick analysis of the situation presented in each individual PDM. 27 The evaluation team did not have access to the raw data sets used for calculation of the indicators, however, the PDMs show to a certain degree that data enabling disaggregation of the final indicators as stated in Tableau des indicateurs are collected. Therefore, the MEL department could significantly improve the quality of analysis provided as feedback to the programs by ensuring different visual presentation of the results collected. Mercy Corps PHASE/Final Evaluation Internal Socorro 15 16 June 2022 Global Humanitarian Consultants 1. QUANTITATIVE INDICATORS FROM SITES VISITED BY THE EVALUATION TEAM Comparative analysis of the PHASE indicators collected in three sites visited by the evaluation team (Nyakunde, Djaiba and Aungba) demonstrate that the program succeeded in its overall goal to reduce the vulnerability of households affected by conflict-caused shocks that had negatively impacted their ability to meet their basic food needs, while gradually increasing their self-reliance. In all three targeted communities, PHASE activities significantly decreased both the frequency and severity of the negative coping strategies participating households had engaged in prior to the PHASE program.28 In Aungba, the program seemed to be especially successful in improving the food consumption score: in Aungba 3, the proportion of households with poor consumption score was reduced from 72% to 0%; in Aungba 1 and 2, they were reduced from 72% to 15% and 16% respectively. The PHASE program did not have a similar effect on the household consumption scores of households in Djaiba and Nyakunde, where almost double the desired number of households (20%) maintained a "poor" consumption score. 28 Reliance on less expensive food, borrowing/relying on help from friends and relatives, limitation of portion sizes at mealtime, restricted consumption of food adults in order for small children to eat, reduction of the number of meals eaten in a day. 38% 46% 43% 34% 18% 15% 48% 42% 47% 51% 18% 72% 48% 14% 11% 10% 16% 82% 10% 36% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Nyakundé-R-2 Nyakundé-R-1 Nyakundé-D-1 Djaiba-1 Aungba 3 Aungba 2 Aungba 1 Food Consumption Score (FCS) Poor Bordeline Acceptable Mercy Corps PHASE/Final Evaluation Internal Socorro 16 16 June 2022 Global Humanitarian Consultants The comparative analysis of the results of the PDMs between rounds of interventions indicate that PHASE activities improved the dietary diversity of participant households (HDDS) in Aungba, as well as in Nyakunde.29 However, despite these positive indications, there is not enough evidence to make a final conclusion, since there is not enough evidence and HDDS in non-observed areas proved to be a complicated indicator. As a rule, similar to the overall comments in the previous section, PDM conclusions fail to provide explanations for reported results: program achievements in a particular community are presented without substantive comment or analysis as to why targets were not achieved. For example, in Nyakunde R1, where none of the three interventions brought about the desired change in the "poor" consumption score, the PDM reports provide no information on the relatively high proportion of households with "poor consumption score after the interventions. At the same time, several conclusions provided in the PDM are anecdotal (e.g., improvement of HDDS from 4 to 6 are associated with the food fair and with the sensitization activities conducted by Mercy Corps staff; however, no evidence is included that shows the correlation between sensitization activities and HDDS). Furthermore, generally, the conclusions of the PDMs do not allow for any additional analysis of whether or not additional rounds of distribution might improve progress along food security indicators: The results prove that some of the indicators of the PHASE program have significantly progressed, that is the case for the [household dietary] diversity score that saw a 100% increase, from 4 to 8 from the Baseline to the SPI2. The survival strategy index [Reduced Copying Strategy Index] demonstrated a slight negative increase by 1 point on the index, moving from 3 at the SPI￾1 to 4 at the SPI-2. The average of the food consumption score went from 30 at the SPI-1 to 30.2 at the SPI-2, recalling that this figure was 19 at the beginning of the program in this zone.30 29 Note: HDDS assesses a household's economic access to food (i.e. its ability to produce, purchase or otherwise secure food for consumption by all household members). It does not provide data on the nutritional quality of a person's diet. 30 PDM Nyakunde R-2 : Original text: Les résultats prouvent que certains indicateurs du programme PHASE ont fortement progressé, c’est le cas pour le score de diversité qui a connu une augmentation de 100% soit de 4 à 8 de la Baseline au SPI 2. L’indice de stratégie de survie a quant à lui connu une légère augmentation négative de 1 point de l’indice en passant de 3 au SPI 1 à 4 au SPI 2. La moyenne de score de consommation alimentaire est passé de 30 au SPI-1 à 30.2 au SPI-2 rappelons que celle-ci était de 19 au démarrage du projet dans la zone. Mercy Corps PHASE/Final Evaluation Internal Socorro 17 16 June 2022 Global Humanitarian Consultants In the above example, not only does it employ incorrect terminology, no additional information related to the achieved change in HDDS is provided; nor does this result seem to correspond to the indicator table, where the achieved change is 87%. Given the change between first and second round (each “adding” two additional food groups), it would be useful to understand how the different groups are added. At the same time, the conclusion states that the rCSI score reduced; however, no additional information is given as to specific coping strategies. Importantly, the conclusion states an improvement in FCS and alludes to its relative success (showing an improvement of the mean FCS from 19 to 30); however, it provides no mention of the relatively high proportion of households who remain with a "poor" consumption score. Furthermore, the report does not address the fact that both rounds of distributions in Nyakunde significantly under-performed in terms of female participants' appreciation of the program’s positive change in their household. In Aungba, the quantitative data supports the hypothesis that multiple rounds of assistance significantly improve food security (with indicators getting progressively better in comparison with the initial baseline and with each other); while the results from Nyakunde (returnees) are inconclusive, and almost contradictory. In all cases, the PDMs provide little analysis of under-achieving activities. The conclusions focus principally on positive changes and contain little analysis. For subsequent programs that contain multi-round interventions, Mercy Corps may want to increase the analytical robustness of the PDM tool in order to: 1. Compare the results of different rounds of distributions, not only against the baselines scores, but also with each other. 2. Capitalize on the quantitative and qualitative data collection to provide a holistic understanding of the results of the activities, while abstain from presenting unsubstantiated conclusions. This will require a stronger qualitative data collection and data analysis methodology. 3. Identify information gaps and recommend additional data collection to better understand why certain activities do not achieve the desired results. 2. MERCY CORPS' STAFF PERCEPTIONS OF THE PHASE PROGRAM31 Mercy Corps' PHASE program staff felt that overall the PHASE program positively achieved the desired results. The mixed responses that were received (i.e., “not positive, not negative”) principally related to incomplete program activities in areas where insecurity (Mwanga, Boga, etc.) prevented Mercy Corps from completing all planned PHASE program activities. On the other hand, non-program staff (including staff from other programs, MEL staff and support staff in Goma and Bunia) were more reserved in their estimation as to what extent the PHASE program brought about the desired results: similarly, but to a greater extent, they highlighted the lack of completion of 31 Mercy Corps staff participated in an anonymous online survey. See Annex IV. 0% 20% 40% 60% 80% 100% 120% Reported positive change (women) Mercy Corps PHASE/Final Evaluation Internal Socorro 18 16 June 2022 Global Humanitarian Consultants certain activities. However, they also indicated that more stable areas need to be chosen for similar activities in order to achieve positive results. One non-program staff proposed moving the complementary activities (e.g., agriculture, VSLA) to another province; while another stated that the level of incompletion of certain activities made it impossible to evaluate the PHASE program.32 3. PARTICIPANTS PERCEPTIONS OF THE PHASE PROGRAM Overall, program beneficiaries indicated that the PHASE program positively impacted their lives: food fairs and cash distributions significantly and immediately reduced their food insecurity and enhanced their general resilience, while the complementary VSLA and agriculture activities enhanced their dignity and contributed to their longer-term resilience and well-being. However, PHASE program beneficiaries highlighted specific and substantive program deficiencies; these included criticism of insufficient quantity and quality of assistance provided; others had difficulty accessing assistance. The provision of food assistance had many positive benefits to program participants. Principally, PHASE reduced participants households' food insecurity and corresponding use of negative copings strategies (Reduced Coping Strategy Index/ rCSI); PHASE increased participants' dietary diversity and food frequency (Food Consumption Score/ SCA). PHASE reduced hunger and enabled beneficiaries to have the strength needed to work in the fields. In many cases, beneficiaries described the immediate impact of the PHASE distributions as “we felt strong and healthy”, or “my children were no longer sick” referring to improved dietary diversity and caloric intake. Others claimed that they “could eat as much as they wanted, even during the day”, that “children did not go away because they knew there will be food in the house”, or “we spent time together as a family”, clearly stating the impact on the overall well-being and improved food frequency. However, despite those positive impacts, the program had several areas that could be improved upon. Specifically, many communities felt that household size was incorrectly or inaccurately measured and that this led to some households receiving an insufficient amount of food. This perception that household size was incorrectly/ inaccurately measured was repeated across all areas and type of interviews. This seems to be linked to the way anti-fraud measures are implemented by Mercy Corps' field teams. While this may be a result of insufficient communication of both PHASE criteria and dates and times of distributions by Mercy Corps, with subsequent results in a misunderstanding of PHASE criteria and participants missing or arriving late to distributions, they may be linked to non-achievement of certain indicators. The beneficiaries, and most often women, declare that the registration/ targeting process did not account for people who were absent during the initial household count: this includes children who were at school at the time of the count, or people who were away for work or other reasons. Beneficiaries claimed that no appeal process existed; thus, there was no possibility to review incorrectly registered household size (e.g., by providing birth certificates or present missing household members at a later time). This created rifts within the community, since similarly-size household were reportedly not counted as such on the beneficiary lists. In Nyakunde, several returned families claimed that only mothers and very small children were present in the house during the registration, and despite their protests, they were registered as size 2, 3 or 4, while in reality at least 6 or 7 household members benefited from the final package. In the same area, the villager leaders declared that the package would have been sufficient, should the size of each household was correctly determined, but in reality, some households only had food for two weeks instead of four. 32 Answers of 4 staff were marked as “incorrect answer”, because their marking did not match their narrative descriptions. Mercy Corps PHASE/Final Evaluation Internal Socorro 19 16 June 2022 Global Humanitarian Consultants Several interviewees indicated that at least in one instance, excess food was provided to PHASE participants with several reporting surplus food laying rotting in storage. Several interviewees indicated that some of the food items available at local fairs were not of good quality (e.g., palm oil) and that the food provided did not always align with local preferences. In regard to food fairs and cash distributions, the most common objection among program beneficiaries concerned the registration process. This principally focused on the method by which household size was determined, followed by sporadic complaints as to the quality and availability of products during the fair and the quality of bills that were distributed as part of the provision of cash. Specific complaints revolved around: the terms of inclusion and exclusion criteria, the preference of female household members as primary PHASE focal points (as opposed to men) and a low level of understanding (among participants) of how participants were targeted in instances of multiple waves of displacement/ return, and within the host communities. In certain areas, Mercy Corps staff overly relied on local chiefs to relay program information, instead of communicating directly with PHASE participants. In several instances, delays in program implementation (resulting from security or logistics issues) was not properly communicated to program beneficiaries in a timely manner. While the intended program results were largely achieved, questions remained as to whether all beneficiaries had sufficient access to information about the assistance they were to be provided with. Remedies for this include pro-active communication about: selection criteria, household size criteria, the reason behind the prioritisation of women as household leads, ensuring the permanent presence of complaint response mechanisms before, during and after distributions, as well as ensuring there are clearly understood dispute mechanisms. Similar to the Cash and Voucher Assistance (CVA), VSLA and agriculture trainings and distributions were positively evaluated (and suggested as follow-up activities by beneficiaries in areas not benefiting from them). This demonstrates the relevance of these components to the overall success of the PHASE program and its contextual validity, but also the importance of self-sufficiency and dignity among PHASE program beneficiaries. On the other hand, as they comprised a relatively small component of the overall PHASE program, the complementary VSLA schemes and agriculture activities were perceived as selective with unclear criteria: these two activities were less developed in terms of both scope and supervision. Both Mercy Corp staff and PHASE program beneficiaries highlighted the need for more in￾depth trainings, supervision and coaching, as well as enhanced targeting of youth and other at-risk community groups. 4. UNANTICIPATED RESULTS POSITIVE: COMPLEMENTARITY OF PHASE PROGRAM WITH MERCY CORPS' SAFER PROGRAM Both beneficiaries and Mercy Corps staff positively evaluated the complementarity between PHASE (food security) and SAFER (NFI, multipurpose cash) programming for newly displaced and newly returned households. Overall, beneficiaries did not distinguish between the two programs; however, several beneficiaries recalled different funding sources (“the US” or “the UK”). Most understood the PHASE program to provide both food and NFIs; this led to confusion with regards to distribution schedules (one-off vs multiple rounds of assistance) and rumours that IDPs and members of the returned population were not receiving the same type of assistance. Clearly, the joint interventions of the two programs significantly improved the beneficiaries' living conditions. As many of their most important needs were covered, program participants were not obliged to sell NFI items to buy food, or to use CVA to purchase needed NFIs. However, in certain areas, misunderstandings concerning multiple-round of assistance (e.g., PHASE) versus one-off distributions (e.g., SAFER), led to rumours that IDPs received Mercy Corps PHASE/Final Evaluation Internal Socorro 20 16 June 2022 Global Humanitarian Consultants multiple rounds of both food and NFI assistance, while members of the returned population received only one round. 33 In instances where PHASE food distributions were coupled with SAFER (cash/NFI distributions: Djaiba and Nyakunde), the combined assistance enabled participants to address their essential NFI needs, such as medical supplies, toiletries, kitchen utensils, etc. This combination was very positively received. In Nyakunde, several leaders suggested to integrate PHASE and SAFER programming and to add additional components, such as: support to agriculture activities and cash-only distributions that would enable beneficiaries to rebuild livelihoods and/ or create new ones. Food assistance provided to host communities positively impacted the overall perception of the joint food and NFI assistance to IDPs in the same area. Some beneficiaries stated that while they mainly required food assistance because they were unable to access their fields, on the other hand “the [internally displaced people] came with nothing and need everything.” Additionally, participants who benefited from both programs reported that as a result their children had greater access to education and that members of their households were able to access needed health care. The following table provided by Mercy Corps summarizes areas of intervention that benefited from joint programming: PHASE and SAFER. LOCATION SAFER PHASE Aungba Return + IDP Complemented 1st round of PHASE with NFI Full PHASE programming: 3 rounds of food, agriculture, VSLA Mwanga 1 st round food + NFI 2 additional rounds of food, VSLA with PHASE Nyakunde Return + IDP 1 st round food + NFI 2 additional rounds of food + agriculture with PHASE; given population pressure Djaiba 1 st round food + NFI and WASH 2 additional rounds of food with PHASE It is important to note that while the complementarity was well appreciated by the community, there is no clear evidence from the collected indicators that the joint rounds have better impact on the food security indicators than the rounds without. This can, however, be somehow explained by the NFIs being used for prolonged periods. In this respect, in order to provide more evidence, it is necessary to compare interventions where only food/only NFIs/combination were provided. NEGATIVE: POTENTIAL FOR HOUSEHOLD AND INTER- AND INTRA-COMMUNITY CONFLICTS Mercy Corps' targeting methodology—based on long-term efforts to maximize efficiency and minimize diversions of humanitarian aid—is built around several assumptions: these include the gender-specific assumption that prioritizing female representatives of targeted households increases the access of all household members to assistance. However, in highly unequal patriarchal communities such as that 33 This was specifically mentioned by certain leaders of the returned population in Nyakunde. Mercy Corps PHASE/Final Evaluation Internal Socorro 21 16 June 2022 Global Humanitarian Consultants which exists in Ituri, without an accompanying communications campaign or efforts at long-term social behaviour change, this positive discrimination is perceived as actively disrespecting the communities targeted by the PHASE program, and the leaders of these communities.34 Indeed, the increased female independence and resilience that came about as a result of the PHASE program was perceived by many community members as "arrogance" and "stubbornness," with negative impacts on household stability. Additionally, despite lists of beneficiaries that clearly contain a number of polygamous households, interviewees (especially women) reported that members of polygamous families not living together were excluded from assistance, or received less assistance per household member as their households appeared too big. In designing subsequent interventions, Mercy Corps might consider how to evaluate the potential risks increased female participation poses both to the women who benefit from the program themselves and to social cohesion in general and implement appropriate measures to mitigate those risks. The impact of such activities might be measured through standardized indicators going beyond collection of SADD, such as Gender Equitable Men Scale, Household Decision Making Index or Women’s Economic Decision-Making Index.35 When rolling out similar programs, Mercy Corps may want to enhance its community engagement and community feedback mechanisms which might include a gender analysis or employ a gender and protection risk matrix. 36 Likewise, prior to commencing subsequent early recovery programs in Ituri province and similar areas, Mercy Corps may want to consider how to deepen their understanding of the complex dynamics and power structures of the region, for example through rapid actor mapping and conflict analysis. Deep￾rooted conflicts with recurrent outbreaks of violence have created a vicious circle of displacement, trauma and grievances in Ituri. Repeated displacement and the crucial role of the host communities as first responders has created an additional burden on communities that were already facing food insecurity, loss of livelihoods and lack of access to basic services. On several occasions, and especially in Nyakunde where two separate interventions were carried out for IDPs and returned populations, beneficiaries pointed out that one group was perceived as benefiting from “better” or “more significant support” than the other one. Returnees complained that more people came to their area after the first distribution to IDPs; some stated that IDPs received more money and better-quality items. While not directly observed in the intervention areas, beneficiaries and community leaders indicated that prior humanitarian assistance may have contributed to inter-community tensions. 37 5. MULTI-ROUND CASH ASSISTANCE 34 Some leaders were expressing their disagreement with statements like “Does Mercy Corps think I will be lying about the household size? I am the chief, I never lie.” 35 More Gender Equality indexes can be found here: https://www.indikit.net/sector/78-gender-equality 36 More about RGA: http://gender.careinternationalwikis.org/care_rapid_gender_analysis_toolkit 37 Those tensions might not result in open conflict; however, they might fuel pre-existing grievances and jealousies and thus impact both community acceptance of the humanitarian actors and the willingness of the communities to re-host IDPs. It is important to keep in mind that traumatized and retraumatized communities without sufficient mental health and psychosocial support (MHPSS) systems in place might demonstrate tendencies to use violence to solve even minor problems. The conflict-sensitive approach, entrenched in the “do no harm” approach and protection mainstreaming, needs to be reinforced throughout the project cycle management, especially in areas with high level of poverty and prevalence of the inter-community tensions, such as Irumu, Djugu or Mahagi territories. This also needs to be in line with the gender sensitive approach, as several studies link food insecurity and recent experience with displacement to increased incidence and acceptance of both general gender-based violence (GBV) and intimate partner violence (IPV). Mercy Corps PHASE/Final Evaluation Internal Socorro 22 16 June 2022 Global Humanitarian Consultants The provision of cash was important for multiple reasons. In many instances it was impossible for PHASE beneficiaries to cultivate their land due to insecurity, and money enabled beneficiaries to rebuild their houses, pay for school fees, rent fields, access medical care, or buy other needed resources (i.e., soap, oil, etc.). Some beneficiaries invested the money they received, but it was difficult to move forward with their businesses because of conflict. Spreading the distribution of cash or food vouchers and fairs over time/ three distribution phases (as compared to a one-off distribution) seemed to positively increase the socio-economic resilience of vulnerable families to the different shocks to their lives caused by the armed conflict in the region that occurred over the lifetime of PHASE activities: most beneficiaries either acknowledge that (Aungba), or suggested that as an area for improvement (Nyakunde, Djaiba). This benefit could not be observed in the zones (Mwanga and Boga) where Mercy Corps was not able to complete three rounds of distributions due to the degradation of the security situation, which also prevented the evaluation team from interviewing PHASE program beneficiaries. Despite its upside, the provision of cash was not unproblematic: in Upano, participants indicated that money exchangers took a fee for their services and that rate was not stable. Beneficiaries in Ukorodoko indicated that money was needed to help with the management of orphans. The effect of the PHASE cash and voucher program on local markets varied according to location and perspective.38 In areas where food was distributed, participants noted (positively) that this led to a decrease in prices and inflation. In certain areas, beneficiaries claimed that immediately after PHASE distributions some of the main staples that were distributed were not available on local markets for several weeks due to low demand. However, at the time of this evaluation, no basic items were missing from local markets, and the majority of informants declared the supply disruption was only temporary because “nobody wanted to buy it [rice, mattresses] anyway”. Some vendors reportedly lost money when selling their goods because of fixed pricing set by Mercy Corps (they had bought the goods at a higher price than Mercy Corps asked them to sell them at). In Jupachinvor and Fataki/Djaiba, some produce was not sold because of an overabundance. Most of the beneficiaries (in PHASE and SAFER intervention areas) claimed that the overall amount of assistance received was insufficient to cover all priority needs, so they opted for buying “a little bit of everything” or “the most important things” while “continuing to go to the market to buy the rest”. On the other hand, in areas that received cash, suppliers, PHASE beneficiaries and certain program staff interviewed indicated that prices went up in local markets as a result of the cash interventions. Price increases in local markets were also noted by beneficiaries in areas that had benefited from VSLA funding (Upano, Jupukum). According to interviewees, the price increase affected various items, such as mattresses (100,000 shillings to 150,000 shillings in Ukurokwodo) and corn (13,000 shillings to 18,000 shillings in Loto). To sum it up, given the life-saving and emergency nature of both PHASE CVA and SAFER interventions, and the relatively high flexibility of the market supply, a short-term impact on markets is not unexpected. It is important to note that overall, both the beneficiary communities and Mercy Corps PHASE program staff considered that the PHASE program did not negatively impact local markets, or that the impact was temporary. Several non-program staff (including MEL) reported that that cash and voucher distributions produced minor negative impacts on local markets in Nyakunde, Djaiba and Boga. Unfortunately, by and 38 Neither Mercy Corps, nor the evaluation team conducted specific post-intervention market assessments, therefore only observations from the field teams (both program and non-program) and information collected directly from the beneficiaries could be used to assess/evaluate the impact of the interventions on the local markets. Mercy Corps PHASE/Final Evaluation Internal Socorro 23 16 June 2022 Global Humanitarian Consultants large PDM reports did not evaluate the PHASE program's effect on local markets, and therefore the evidence to the PHASE program's effect on local markets remains purely qualitative. The two PDMs that evaluated the price of food staples made available at PHASE-sponsored fairs vs. the price of the same items at local markets indicated there to have been no impact on local markets; however, this conclusion was similarly based on qualitative rather than quantitative data such as might be gathered via an independent market survey. Given the lack of quantitative data, it is not possible to come to a definitive conclusion as to whether and/ or to what extent the PHASE program activities affected local markets. Mercy Corps may want to include market price monitoring among future MEL activities, including conducting specific surveys during the PDM data collection process. While most of the beneficiaries did not consider the market impact important enough to require specific mitigation measures, several participants suggested cash assistance be provided in secret to avoid seeing an impact in local market prices and exchange rates (which reportedly went up in Aungba). Also, some of the participants from the same area who received cash, but did not receive tools and training, expressed their preference to receive both (Ukurokwodo). 6. MERCY CORPS STAFF Overall, MC staff (both PHASE program and non-program staff) perceived the PHASE program to be "very positive" and "positive." Across all aspects of the program, PHASE program staff were more positive about the design and impact of the program than non-PHASE program staff. Similar to the majority of PHASE beneficiaries, Mercy Corps staff indicated their preference for distributing cash over tokens/ fairs; however, they felt that there was resistance to this from Mercy Corps itself: "Mercy Corps did not want to distribute money because of problems that happened in areas like Djalasiga where after receiving money some [program participants] took advantage of alcohol”: this left the family without food or stability (Aungba and Mwanga). In other areas, women used the money they received to buy alcohol and "behaved badly" (Djalasiga). Mercy Corps staff felt that even though food aid is an emergency response (as opposed to an early recovery response), the provision of food enabled people to use their savings on other necessary items, which increased their resilience and contributed to their short- to medium-term self-sufficiency. Staff members also indicated they felt the PHASE program to be insufficiently staffed. They also expressed the need to increase the number of households reached by future early recovery programs. Mercy Corps staff felt the PHASE program to be effective in increasing the resilience of those who participated in the program. PHASE activities in Djalasiga, Nyakunde, and Djaiba were deemed by Mercy Corps staff to be "effective"; however, they indicated that in comparison to the other areas, activities in Boga and Mwanga were not nearly as effective. Insecurity in the area prevented Mercy Corps accessing Boga in the latter part of the PHASE program, which prevented participants in Boga from receiving the full complement of distributions: therefore PHASE activities in Boga were deemed to not be nearly as effective as those in other areas. While integration of additional activities (e.g., VSLA, agriculture) correlated with an increased perception of efficiency among both PHASE program and non-PHASE program staff, there was no clear explanation why certain non-PHASE program staff considered certain distributions to be less efficient than others. Mercy Corps PHASE/Final Evaluation Internal Socorro 24 16 June 2022 Global Humanitarian Consultants B. VSLAs, Food Security, Self-Sufficiency Overall, the VSLA scheme seemed to have overachieved its objectives in terms of the number of VSLAs started and their operationalization. While they did achieve their targeted indicators, VSLAs enabled participants to buy livestock and engage in agricultural activities (Ukurokwodo). The increased access to money that resulted from participation in VSLAs (both loans and savings) helped participants buy tools that had been lost in the conflict, and it enabled them to send their children to school, to set up small businesses and to better take care of themselves (Loto). (It is important to note that certain VSLAs decided to start agricultural activities and in this respect, it can be interesting to envisage joint activities for VSLA and agriculture beneficiaries to maximize the impact of trainings.) Participation in VSLA schemes enhanced participants' self-sufficiency, they empowered women (Upano), enabled them to support their family, achieve greater social balance and gave them an increased sense of dignity. Membership in VSLA schemes not only enabled participants to strengthen their economic standing and increase capability to access to money, it allowed them to increase their level of self-care and provided a mutual help associations that members could turn to when in need. The success of the VSLA program was not universal, however; various participants indicated that vis-à￾vis the means they had at their disposal and their situation (some of these participants were looking after orphans and multiple children) the quantity of money they had access to via the VSLA scheme was insufficient to commence their desired activities. In some instances, this could be attributed to the fact that these participants were looking after orphans and multiple other children; while, in others, the amount of money was inadequate either to start the small business or raise the livestock they desired.39 Others noted that while the money they were able to access via the VSLA scheme helped them start a small business, insecurity prevented them from accessing their fields so they were unable to reap the full and desired benefits of the program. VSLA programming also directly responded to requests to support “self￾sufficiency” (auto-prise en charge) and to restore livelihoods. It is important to note that self-sufficiency is closely aligned to dignity, and is frequently used as an ultimate goal and aspiration of persons in both displaced and returned communities. Self-sufficiency was also regularly mentioned by women and men of different ages, with respect to programs that specifically targeted women and youth. There was a clear, positive correlation between self-sufficiency, respect and perceived positive contribution towards one's household and community. 1. MERCY CORPS STAFF PERCEPTION In contrast to the food aid program, which Mercy Corps staff considered to be an emergency response, Mercy Corps staff considered the VSLA scheme and the agricultural program to be more sustainable than the food security program. Both PHASE and non-PHASE program staff only considered the fully integrated interventions in Aungba and Djalasiga to be very efficient. In comparison, Mercy Corps staff 39 It must be noted that at the time of the evaluation, VSLAs had not been operating for long. Typically, it is only during the second cycle of activity that VSLA participants are able to build a sufficient amount of capital, both as a group and an individual, that enable them to distribute sizeable loans to their members. "THIS MONEY HELPED A LOT FOR INCOME GENERATING ACTIVITIES AND IT ALLOWED THEM TO PAY FOR HEALTH CARE. OVERALL THEY ARE ABLE TO TAKE RESPONSIBILITY FOR THEMSELVES KNOWING THAT THEY INVESTED IN AGRICULTURE." COMMUNITY LEADER, AUNGBA Mercy Corps PHASE/Final Evaluation Internal Socorro 25 16 June 2022 Global Humanitarian Consultants estimated interventions carried out in other areas to be incomplete, or to focus on food assistance only. This sentiment was echoed by communities that had benefited from such integrated interventions, and by those who had not benefited from such interventions, but wished for them. The PHASE program in no way solved all food needs, however, the VSLA schemes contributed to the self-sufficiency of program participants. Mercy Corps staff advocated for the need to increase the number of households that benefit from future early recovery programs. C. Agricultural training, Provision of Seeds and Tools The provision of agricultural training, seeds/ seedling and tools supported and enhanced agricultural activities and supported overall improvements in participants' quality of life and helped families to fight against hunger and malnutrition in their children. In particular, the use of how to work in and cultivate fields without using bushfires was cited as especially useful by program participants. The agricultural training program assisted many program participants to address their food security issues—both in terms of access to the quantity of food and an increase in the variety of food types—and improve their living conditions. The knock on benefits of this component of the PHASE program to those who had access to and could cultivate land both prior to and subsequent to Mercy Corps' activities were significant: produce that resulted from participation in the program improved the economic and social situation of participants significantly enough that it allowed them to send their children to school and to access health care. However, some of the food security gains brought about by this program were mitigated by pest infestations, which in some areas ravaged harvests. This suggests that future early recovery programs may want to incorporate context-specific anti-pest components. And, as is the case for many aspects of the PHASE program in Ituri, the state of endemic insecurity prevented many people from cultivating their fields and thus were unable to benefit from the agricultural program. In these instances, participants noted, the provision of cash was especially beneficial, as it enabled them to live. In Nyakunde, where no agriculture activities had been implemented at the time of this evaluation, key informants consistently suggested that agriculture distributions and trainings be implemented alongside the 3 rounds of food distribution. This was due to the fact that as insecurity prevented them from going to their fields, community members were looking for agriculture opportunities they could implement close to their houses (i.e., permagardens). 40 Indeed, this was a common sentiment expressed by many program participants: that people were only able to cultivate crops on the spot they were living in, and that it was not possible to cultivate large areas of land due to insecurity. This suggests that a particularly valuable aspect of the agricultural training component of the PHASE program is permagardens. While it is not possible to evaluate the extent to which permagardening techniques composed a portion of the overall agricultural trainings, the findings strongly indicate the value that trainings in how to cultivate high-yield, 40 Nb. Subsequent to the field trip conducted by the consultants, Mercy Corps has implemented agricultural activities in Nyakunde. "THAT MERCY CORPS COMES TO HELP US! MY POPULATION IS AT RISK OF SHUTTING DOWN, AT RISK OF DYING, WALKING NAKED BECAUSE OF LACK OF CLOTHES. WE ARE ALL VULNERABLE, ALL SLEEPING OUTSIDE. WE DON'T KNOW WHEN THIS SITUATION CAN CHANGE SO THAT MERCY CORPS COMES TO HELP US AGAIN. CURRENTLY, IN MY VILLAGE WE CANNOT MEET 2 PEOPLE: YOU ARE KILLED IN THE FIELD. THAT MERCY CORPS SEES HOW TO REDO THESE ACTIVITES BECAUSE WE ARE IN A BAD SITUATION." VILLAGE CHIEF, UKOROKODO Mercy Corps PHASE/Final Evaluation Internal Socorro 26 16 June 2022 Global Humanitarian Consultants small plot gardens in the spots of land that people living on and can access, along with the provision of the appropriate seeds/ seedlings, fertilizers, pesticides and tools, would be extremely well-suited to future early recovery programs aimed at reducing food insecurity and improving resilience in endemically insecure areas. D. Complementary Activities & Economic Opportunities for Women and Young Adults The VSLA scheme enabled women to buy livestock and contribute to agriculture activities: it increased their self-sufficiency and empowered them. The increased knowledge of agricultural techniques and training on how to cultivate fields increased women's capabilities. However, men did not appreciate the change in women's attitudes that this increased self-sufficiency and empowerment brought about: they said that women became more stubborn and arrogant. The increased agency that came with women returning home with money brought about marital problems and conflict within couples: women reportedly became "selfish" and "arrogant" (Molu, Djalasiga) and this led to couples splitting up. Some men saw inequity in the fact that only women were included in the VSLA scheme. 41 In terms of measuring the extent to which the "agricultural training and provision of seeds" activity specifically increased the capabilities of young adults to cover their basic food needs, while progressively increasing options for self-reliance, the qualitative data gathered by the evaluation team is inconclusive. The quantitative data provided by Mercy Corps via PDMs' does not speak to the extent to which young adults were successfully targeted by agricultural activities. PDMs were only conducted after cash distributions and food fairs—no PDMs were conducted after PHASE agricultural activities. However, the quantitative data provided by Mercy Corps via their PDMs after cash distributions and food fairs show the median age of program participants to be 36.5. If the median age of those who participated in PHASE's cash and voucher programs holds true for those who participated in the complementary agricultural activities, which is statistically probable given that all agricultural activity participants were a subset of those participating in the larger unrestricted cash and restricted voucher assistance program, it is clear that the complementary agricultural component of the PHASE program succeeded in targeting adults. It is impossible to determine to what extent PHASE activities succeeded in targeting young adults. However, it is possible to say that the PHASE program positively affected children in two important ways: the PHASE program supported young people to go to school, and it gave them the opportunity to play (to be children). Mercy Corps staff considered there to be significant gaps in the PHASE program in terms of both young adult and female empowerment. Some considered that sensitization activities at household level are not sufficient for sustainable empowerment, while others pointed out that perceived 41 It is important to note that perception of gender equality as a zero-sum game (e.g. men need to give up their rights, if women are to get more rights) is a crucial part of understanding of gender roles, responsibilities and gendered power dynamics in DRC. See: https://promundoglobal.org/wp-content/uploads/2014/12/Gender-Relations-Sexual-and-Gender-Based-Violence-and￾the-Effects-of-Conflict-on-Women-and-Men-in-North-Kivu-Eastern-DRC-Results-from-IMAGES.pdf "THE NEIGHBOURS BECAME RICH AND THE CHILDREN AND WOMEN BECAME VERY WELL AND BUY MANY THINGS AND PAY THEIR HOSPITAL BILLS AND STUDENTS ATTEND SCHOOL WITH SNACKS." COMMUNITY MEMBER, AUNGBA Mercy Corps PHASE/Final Evaluation Internal Socorro 27 16 June 2022 Global Humanitarian Consultants exclusion of male beneficiaries from certain activities (such as VSLA), can have negative impact on female beneficiaries in the long term. Among the most important comments that Mercy Corps staff proposed are that the number of VSLA and agriculture activities be expanded, which would broaden the beneficiary base; it was also suggested to focus on participants with multiple vulnerabilities (e.g., illiterate women), to provide more trainings and supervision, to include men in activities focusing on positive masculinity and to improve the participation of youth, with a focus on youth entrepreneurship. E. Limiting Factors Inability to access fields due to endemic chronic insecurity was the primary factor that impinged upon the PHASE program's capability to increase people's access to food in the medium- and long-term. This insecurity affected not only those who benefited from the seeds and agricultural training, but those who participated in the VSLA schemes: despite having VSLA money to invest in the cultivation of crops, it was too dangerous for many program participants to tend their fields (Loto). However, it is also important to note that chronically insecure and volatile areas, such as Irumu, Djugu and Mahai territories, need immediate, emergency and life-saving programming and also early recovery and resilience activities. It should be an imperative—and it was also repeatedly requested by the beneficiaries across all three evaluated areas—that life-saving support be provided alongside activities that enable communities and households to regain autonomy and self-sufficiency. The French phrase “auto-prise en charge” was mentioned in every single focus group discussion, regardless of gender, age or type of assistance received. Despite the disruption of the activities of VSLA and agriculture, the beneficiaries in Aungba continued to evaluate those programs positively because, unlike the food or CVA intervention, the self-sufficiency brought about by these programs is valued by the communities and often associated with well-being and dignity.42 In the areas where PHASE program participants could access their fields, the lack of/ or inconsistent/ or inadequate provision of tools (as a component of the agriculture training and provision of seeds/ seedlings program) prevented some participants from fully realizing the benefits of the program in terms of adequate food production. Inconsistent results in terms of the amount of money VSLA scheme participants could access could limit the success of this program. Some participants reported that the VSLA enabled them to become self￾sufficient in terms of food security and they were able to invest in small livestock, breeding and agriculture. However, others reported that the amount of money they were able to access was not enough to enable them to achieve their objectives, which was to raise livestock: they had to settle for raising poultry. It is important to note that these reported insufficiencies may have less to do with the value of VSLAs schemes themselves, but could be more directly related to the timing of the evaluation 42 In general, the displaced and returned communities were very grateful for the assistance provided by Mercy Corps, however, often stated that being dependent on external support for their immediate needs is not their preferred option. In this respect, it was not the assistance but the situation in which people needed assistance that felt undignifying for certain. Some women summed it up saying “it does not feel good when you can’t provide for your family and you are dependent on others”, “before, I could feed my children from our field, but now I don’t have enough”, while one of the leaders stated “we were very grateful to receive the assistance from Mercy Corps, my wife even calls rice at home “mercy corps” now, but I would like Mercy Corps to help us make living from farming again, so we can keep feeding our children”. "WHEN WE WERE GIVEN THIS ASSISTANCE THE CHANGE WAS HUGE. BUT WHEN THE MONEY RAN OUT, MISERY CAUGHT UP WITH US AGAIN." FEMALE PARTICIPANT, NYAKUNDE Mercy Corps PHASE/Final Evaluation Internal Socorro 28 16 June 2022 Global Humanitarian Consultants vis-à-vis the timeline of the VSLA cycle, with the time requirements necessary to realize significant impact from VSLA activities, i.e., the evaluation was conducted at a premature stage in the VSLA cycle where capital accumulation and IGAs were still in their incipient stages. Subsequent early recovery programs with VSLA components may want to consider potentially envisaging a longer time-frame for monitoring and reporting on these activities. Additionally, multiple participants in Aungba indicated that there were local merchants and/ or money exchangers who were aware of the existence and timing of the PHASE program raised prices and manipulated local exchange rates, tactics, which they said served effectively to "steal" the full benefits of the Cash and Voucher and VSLA programs from program participants. Despite the visible engagement of Mercy Corps vis-a-vis gender equality and the targeted participation of women and youth in the PHASE program—this was especially evident in the complementary VSLA and agriculture components of the PHASE program—a greater Gender Equality and Social Inclusion (GESI) integration reinforced across the project management cycle, may benefit communities targeted by future Mercy Corps programs. Such increased focus may make Mercy Corps staff more attentive to complaints from women and cause them to proactively investigate instances of female beneficiary dissatisfaction, such as was expressed in Nyakunde and remained unaddressed. The evaluation team witnessed a certain level of resistance from certain Mercy Corps team members in regards to ensuring gender sensitivity during the evaluation process, for example, questioning the logic behind ensuring gender parity in the evaluation team composition and evaluation design so as to ensure that female beneficiaries were interviewed exclusively by female evaluators. Similarly, in spite of many communities' dissatisfaction with PHASE's gendered approach to program registration,43 the evaluation team found no evidence that this dissatisfaction prompted review of the practice or increased community engagement on this topic. On the contrary, many Mercy Corps staff perceived this approach as empowering for women and enabling the participation of women as decision-makers. Yet, over the course of the KIIs and FGDs, it became obvious to the evaluation team that both female and male beneficiaries did not understand the logic behind this approach: men were critical and women were confused. Mercy Corps staff expressed that stand-alone food security expertise was of primary importance to the success of the program while downplaying the importance of gender and protection sensitive approaches. This approach is unfortunate, given the broad evidence gathered in both DRC and other sub-Saharan African countries that clearly demonstrate the close correlation between gender inequality, food insecurity and chronic conflicts. It is also important to note that gender-sensitive programming cannot be mistaken for women-oriented programming, but rather needs to address inequality in roles, responsibilities and expectations for both women and men, such as might be found in programs on positive masculinity or gender-transformative activities. Given the extended presence in each community (due to both multiple rounds and/or VSLA and agriculture components of the PHASE program), Mercy Corps might consider engaging with the underlying causes of gender inequality, power dynamics and youth involvement. This could start with greater engagement of Mercy Corps staff and an increased presence of female staff in the field. The evaluation team also concluded that entrenched male resistance to women's empowerment was the principle barrier that, if left unaddressed, would prevent the PHASE program's short-term successes in increasing women's capabilities from becoming sustainable in the medium- to long-term. Though well￾intended, the policy of specifically targeting only women for inclusion in the VSLA schemes—which was a definite success!—also aroused resentment in men in the communities who felt the inclusion criteria to 43 The evaluation team is not questioning the logic behind PHASE's gendered approach, but rather to the lack of community engagement on this important program aspect. Mercy Corps PHASE/Final Evaluation Internal Socorro 29 16 June 2022 Global Humanitarian Consultants be unfair and inequitable. This exclusion caused men, rather than appreciating the benefits this increased earning power would have for their entire household, to look unfavourably at the immediate changes in women's agency ("they became arrogant and caused conflict in the family"), that the PHASE VSLA program brought about. Male resistance is a palpable barrier that will prevent the long-term achievement of increasing women's capabilities and must be addressed in future iterations of similar early recovery programs. Options could include expanding the criteria of the VSLA schemes to include men and/ or subsequent VSLA programs could be coupled with a holistic, whole of family, behavioural￾change component that covers women's financial empowerment in terms of family financial management, budgeting, creating of small businesses, etc. 1. RISKS ASSOCIATED TO 3 ROUNDS OF DISTRIBUTIONS Through revisions of provided documents, as well as field observations and community consultations, the evaluation team finds Mercy Corps to demonstrate a high level of commitment in relation to fraud and aid-diversion, with specific policies related to: the division of tasks, multiple evaluation and intervention teams, the involvement of different stakeholders, the presentation of strict criteria during targeting and registration and advanced methods of identity verification (including photos). This commitment was acknowledged and often referred to by beneficiaries, non-beneficiaries and other stakeholders (such as local leaders) with various degree of agreement as to their success. 44 During the PHASE program, only one round of targeting/ registration of beneficiaries was organized and results were used across all three rounds of distributions, as well as for VSLA and agriculture activities. This measure was put in place to prevent additional people to be included at a later point in the program. However, discussions with community members revealed a few points of concern, which Mercy Corps may want to address when rolling out similar Early Recovery Programs: ▪ The fact that the possibility of receiving assistance might serve as "pull factor", and being a factor that displaced populations take into consideration when evaluating whether to stay in the locations to which they had relocated or whether to return to their points of origin, which had become relatively stable. ▪ Several interviews indicated that community members and leaders might be untruthfully increasing household size and requesting financial “motivation” for collaboration. These same discussions also revealed relative dissatisfaction of the beneficiary communities with some of the anti-fraud measures (e.g., one-time registration, direct observation of household size including visits to participants' houses, etc.). ▪ In mixed communities, the representatives of the returned population made allusion to three rounds of assistance becoming a source of conflict between displaced, returned and host communities. While this could not be verified, the evaluation team noted bitterness among the returned population who accused IDPs of waiting for assistance before returning (see the first bullet point), not paying back their debts despite having received assistance and a general feeling of injustice. In this respect, it seems advisable that PHASE continues ensuring that anti-fraud measures are fully implemented, reinforces communication about those measures, and ensures specific risk assessments are conducted for each project site, especially in mixed settings. In future Early Recovery Programs, 44 In 2019, the FCDO funded an Operational Review that identified local leaders as an important factor in elaborating fraud schemes. As a result, many humanitarian actors, including Mercy Corps, have developed additional anti-fraud/anti-aid￾diversion measures, which, naturally, are not always well appreciated by the local leaders. Mercy Corps PHASE/Final Evaluation Internal Socorro 30 16 June 2022 Global Humanitarian Consultants Mercy Corps may want to implement three rounds of distributions in areas with long-term displacement and where the longer-term presence of evaluation teams can be ensured, and consider the benefits of one-off distributions in areas subject to pendular displacement. In all cases, Mercy Corps should consider closely monitoring community perceptions. 2. COMMUNITY ENGAGEMENT AND PERCEPTION OF MERCY CORPS While the overall evaluation of Mercy Corps activities in the intervention areas remained largely positive and beneficiaries regularly expressed their gratitude for being included in the PHASE program, there are several areas in which Mercy Corps might consider how to improve its performance. In terms of the community engagement, the lack of clear communication about the nature of the intervention(s) and relevant program criteria created frustrations among targeted communities. Additionally, several of Mercy Corps internal standards/ rules were considered to be disrespectful to local leaders (i.e., the practice of excluding community leaders from receiving PHASE program assistance due to their lack of “vulnerability”).45 At the same time, it is important to note that some of the leaders were in disagreement with the rules, rather than not understanding them, and sometimes considered those rules as disrespectful: “I could not lie about the size of my household, I am the chief of the village.”). Other issues significantly impacting how the community perceived the PHASE program: many participants reported repeated lateness and a lack of respect as to agreed timing by Mercy Corps staff. This seemed to be both an individual and a structural issue (heavy procedures, delayed payments, etc.). In certain areas, beneficiaries claimed that chronic lateness was also the main cause of incorrect beneficiary counts as Mercy Corps staff came when adults left for work and children left to school. The chronic lateness seemed to be based on a lack of understanding on the part of Mercy Corps staff on the time needed to complete standard operating procedures (e.g., security check, fuelling of vehicles, paper￾work, administrative obligations of the new teams in the field, etc.), which then obliged to them to change the schedule at the last minute. In several instances, this led to the unavailability of certain stakeholders and to Mercy Corps' staff chronic lateness to agreed meetings. F. To what extent are the net benefits of the PHASE programs likely to continue? The provision of cash, food and vouchers helped food insecure families to redress their food insecurity in the short term. However, as useful and immediately impactful as this component of the PHASE program was in the short term, it is unclear how or if the benefits of this aspect of the PHASE program would be sustained in the medium- to long-term. In respect to CVA, no medium or long-term impact can be expected given the emergency and life-saving nature of the interventions. Despite three rounds of interventions, each distribution only provides assistance sufficient to cover needs of participating households for one month. Instead of looking at sustainability of the assistance itself, the evaluation team observed the medium-term results produced by 45 In Nyakunde, the local leaders claimed on several occasions that they were excluded from the assistance. However, while questioned more in-depth, it was revealed that the leaders’ households were benefiting from the assistance with their wives being registered as the household representatives. This was considered as offensive by some of the leaders, however, there was not an agreement among them whether the leaders should receive extra support for themselves (referred to as “motivation for collaboration”) or whether they should be stated as the household representative (stated as “my wife is not the head of household”). Mercy Corps PHASE/Final Evaluation Internal Socorro 31 16 June 2022 Global Humanitarian Consultants the assistance provided, such as feeling relieved and not sad, feeling healthy, feeling satisfied to be able to provide for one’s family, etc. Even interviewing program participants several months after a distribution, the positive feelings about the assistance they received was palpable; this, was sometimes accompanied by a request for an additional round of assistance or an additional type of assistance. Concerning the complementary VSLA and agriculture activities, medium-term benefits were clearly imperilled by new waves of displacement experienced by communities between the completion of the PHASE program and the time this evaluation was conducted. The knowledge gained via the agricultural trainings and permagarden techniques will continue to benefit PHASE program participants for the rest of their lives.46 The improvements in households' food insecurity situation brought about by the provision of seeds/seedlings could prove sustainable in the near future. However, the endemic insecurity in Ituri and the very real possibility that new waves of violence might displace program beneficiaries and thus prevent them from accessing the land and gardens they have cultivated, means that the sustainability of this aspect of the PHASE program is unknown and subject to various external factors. If PHASE beneficiaries are not subsequently displaced in the next months or years, the benefits from this aspect of the program will continue to reduce their food insecurity; but, if they are displaced the sustainability of this aspect of the program will be significantly lower. Overall, though the amount of seeds and tools provided to program beneficiaries (through a $35 USD voucher to be redeemed at a "seed and agricultural tools fair") was in line with Food Security Cluster guidelines regarding agriculture in emergencies in eastern DRC, program beneficiaries consistently noted a dearth of agricultural tools as a factor impinging upon their capabilities to reduce their food insecurity. A future focused expansion of the "agricultural training and provision of seeds" activity— combined with a more robust provision of tools and seeds—will better ensure medium-term food security at both the household and community level. That being said, beneficiaries in some locations noted that they had enough seeds and tools/ resources/ training to support future planting, but access to their fields remained uncertain (Jupachivor). VSLA schemes were positively perceived and often referred to as the most innovative and sustainable aspects of the PHASE program by participants. However, the established groups seem to require additional resources in terms of time and supervision in order to be fully self-sufficient. While some beneficiaries reported positive change within the lifespan of the current project, the measured change on the household level was relatively low. VSLA beneficiaries have significant ambitions (such as ownership of livestock), which in itself might require additional resources (such as investment in veterinary services). Another potential risk to the sustainability of VSLA schemes is the potential for future displacement of VSLA members. The PHASE program brought about an increase in agency and empowerment of the women who participated in the program. However, given the substantive male resistance to the increase in women's agency and the high degree to which their economic and livelihood gains could be disrupted by future displacement, the important question of "if" and "how" this empowerment will be sustained in the long run remains unknowable. In terms of the sustainability of the overall well-being of those who had benefitted from the PHASE program, while participants indicated the positive impact PHASE programs had in addressing their immediate food insecurity, they also expressed the need for more comprehensive and far-reaching programs, such as: 46 As stated before, the impact of the trainings was self-reported, and no indicators related to permagardens were collected by the time of the evaluation. Mercy Corps PHASE/Final Evaluation Internal Socorro 32 16 June 2022 Global Humanitarian Consultants ▪ Schooling; ▪ Medical, shelter and veterinary support; ▪ Youth livelihoods training; ▪ Increased access to potable water. G. Mercy Corps' Perception of the PHASE program Overall, Mercy Corps' staff felt that the PHASE program positively achieved the desired results. There were also mixed responses that (i.e., “not positive, not negative”) principally related to incomplete program activities in areas where insecurity (Mwanga, Boga, etc.) prevented Mercy Corps from completing all planned PHASE program activities. Non-program staff (who had a lower level of knowledge of the PHASE program) were more reserved in their estimation as to what extent the PHASE program brought about the desired results: similar, but to a greater extent, they highlighted the lack of completion of certain activities. However, they also indicated that more stable areas need to be chosen for similar activities in order to achieve positive results. One non-program staff proposed moving non￾emergency activities (e.g., agriculture, VSLA) to another province; while another stated that the level of incompletion of certain activities made it impossible to evaluate the PHASE program.47 Overall, program beneficiaries indicated that the PHASE program positively impacted their lives: food fairs and cash distributions had a significant and immediate impact on their lives, while VSLA and agriculture activities enhanced their dignity and contributed to their longer-term resilience and well-being. However, PHASE program beneficiaries highlighted specific and substantive program deficiencies; these included criticism of insufficient quantity and quality of assistance provided; others had difficulty accessing assistance. In regard to food fairs and cash distributions, the most common objection among program beneficiaries concerned the registration process. This principally focused on the method by which household size was determined, followed by sporadic complaints as to the quality and availability of products during the fair and the quality of bills that were distributed as part of the provision of cash. Specific complaints revolved around: the terms of inclusion and exclusion criteria, the preference of female household members as primary PHASE focal points (as opposed to men) and a low level of understanding (among participants) of how participants were targeted in instances of multiple waves of displacement/ return, and within the host communities. In certain areas, Mercy Corps staff overly relied on local chiefs to relay program information, instead of communicating directly with PHASE participants. In several instances, delays in program implementation (resulting from security or logistics issues) was not properly communicated to program beneficiaries in a timely manner. While the intended program results were largely achieved, questions remained as to whether all beneficiaries had sufficient access to information about the assistance they were to be provided with. Remedies for this include pro-active communication about: selection criteria, household size criteria, the reason behind the prioritisation of women as household leads, ensuring the permanent presence of complaint response mechanisms before, during and after distributions, as well as ensuring there are clearly understood dispute mechanisms. Similar to the principal food and CVA component of the PHASE program, VSLA and agriculture trainings and distributions were positively evaluated (and suggested as follow-up activities by beneficiaries in areas not benefiting from them). This demonstrates the relevance of these components to the overall 47 Answers of 4 staff were marked as “incorrect answer”, because their marking did not match their narrative descriptions. Mercy Corps PHASE/Final Evaluation Internal Socorro 33 16 June 2022 Global Humanitarian Consultants success of the PHASE program and its contextual validity, but also the importance of self-sufficiency and dignity among PHASE program beneficiaries. On the other hand, as they comprised a relatively small component of the overall PHASE program, the VSLA schemes and agriculture activities were perceived as selective with unclear criteria: these two activities were less developed in terms of both scope and supervision. Both Mercy Corp staff and PHASE program beneficiaries highlighted the need for more in￾depth trainings, supervision and coaching, as well as enhanced targeting of youth and other at-risk community groups. 1. Unanticipated results POSITIVE: COMPLEMENTARITY OF PHASE PROGRAM WITH MERCY CORPS' SAFER PROGRAM Both beneficiaries and Mercy Corps staff positively evaluated the complementarity between PHASE (food security) and SAFER (NFI) programming for newly displaced and newly returned households. Overall, beneficiaries did not distinguish between the two programs; however, several beneficiaries recalled different funding sources (“the US” or “the UK”). Clearly, the joint interventions of the two programs significantly improved the beneficiaries' living conditions. As many of their most important needs were covered, program participants were not obliged to sell NFI items to buy food, or to use CVA assistance to purchase needed NFIs. However, in certain areas, misunderstandings concerning multiple rounds of assistance (e.g., PHASE) versus one-off distributions (e.g., SAFER), lead to rumours that IDPs received multiple rounds of both food and NFI assistance, while members of the returned population received only one round. Food assistance provided to host communities positively impacted the overall perception of the joint food and NFI assistance to IDPs in the same area. Some beneficiaries stated that while they mainly required food assistance because they were unable to access their fields, on the other hand “the [internally displaced people] came with nothing and need everything.” Additionally, participants who benefited from both programs reported that as a result their children had greater access to education and that members of their households were able to access needed health care. NEGATIVE: POTENTIAL FOR HOUSEHOLD AND INTER- AND INTRA-COMMUNITY CONFLICTS Mercy Corps' targeting methodology—based on long-term efforts to maximize efficiency and minimize diversions of humanitarian aid diversion—is built around several assumptions: these include the gender￾specific assumption that prioritizing female representatives of targeted households increases the access of all household members to assistance. However, in highly unequal patriarchal communities such as that which exists in Ituri, without an accompanying communications campaign or efforts at long-term social behaviour change, this positive discrimination is perceived as actively disrespecting the communities targeted by the PHASE program, and the leaders of these communities. Indeed, the increased female independence and resilience that came about as a result of the PHASE program was perceived by many community members as arrogance and stubbornness, with negative impacts on household stability. Additionally, members of polygamous families were excluded from assistance. In subsequent interventions, Mercy Corps must evaluate the potential risks increased female participation poses both to the women who benefit from the program themselves and to social cohesion in general, and implement appropriate measures to mitigate those risks. The impact of such activities might be measured through standardized indicators going beyond collection of SADD, such as Gender Mercy Corps PHASE/Final Evaluation Internal Socorro 34 16 June 2022 Global Humanitarian Consultants Equitable Men Scale, Household Decision Making Index or Women’s Economic Decision-Making Index.48 Likewise, Mercy Corps must better understand the complex dynamics and power structures of the region. Deep-rooted conflicts with recurrent outbreaks of violence have created a vicious circle of displacement, trauma and grievances. Repeated displacement and the crucial role of the host communities as first responders has created an additional burden on communities that were already facing food insecurity, loss of livelihoods and lack of access to basic services. On several occasions, and especially in Nyakunde where two separate interventions were carried out for IDPs and returned populations, beneficiaries pointed out that one group was perceived as benefiting from “better” or “more significant support” than the other one. Returnees complained that more people came to their area after the first distribution to IDPs; some stated that IDPs received more money and better-quality items. While not directly observed in the intervention areas, beneficiaries and community leaders in Nyakunde indicated that prior humanitarian assistance caused inter-community tensions. Across both KIIs and the anonymous survey, Mercy Corps program (and to a certain extent non￾program) staff demonstrated an extremely positive perception of the design and impact of the PHASE program that is belied by the findings of this evaluation. This may be due to an insufficiently critical engagement with program design and/ insufficient consideration of potential negative impacts. While it is not possible to evaluate to what extent response bias was or was not controlled for in the Post￾distribution Monitoring (PDM) that accompanied many of the PHASE activities, certainly (with two minor exceptions49) the PDM reports present a universally positive view of the PHASE program. Despite visibly engaging with gender equality—this was especially evident in the VSLA and agriculture subcomponents of the PHASE program—a greater Gender Equality and Social Inclusion (GESI) focus, reinforced across the project management cycle, may benefit communities targeted by future Mercy Corps programs. For example, it was unclear to the evaluation team whether during the PDM process female beneficiaries were being interviewed by female enumerators. H. Lessons Learned & Recommendations Both qualitative (KIIs) and quantitative (PDM reports) data shows that in the short-term, the PHASE program largely achieved its objectives in reducing household food insecurity (rCSI), improving dietary diversity (SCA, SDAM) and bringing about a positive change in the households that received PHASE assistance. However, the benefits of the PHASE program exceeded the field of food insecurity. These impacts include: ▪ Increased health; ▪ Reduced death; ▪ Decreased theft; ▪ Balance within communities; ▪ Increased access to toilets; ▪ Capability to purchase necessary household items; ▪ Capability to send children to school; 48 More Gender Equality indexes can be found here: https://www.indikit.net/ 49 The PDM in Nyakunde November 2021 reported that 71% of program beneficiaries reported a negative change in their households as a result of their participation in the PHASE program; however, it is unclear that this negative change was due solely to the poor quality of palm oil, rice and beans PHASE participants received. The PDM in Aungba II conducted in June 2021 showed a reduction in the SCA score between SCA SPI-1 (36%) and SCA SPI-2 (10%); however, the claim that this regression was "fortement liée à une mauvaise utilisation du cash" is not explored or explained in any way. Mercy Corps PHASE/Final Evaluation Internal Socorro 35 16 June 2022 Global Humanitarian Consultants ▪ Capability to buy medical supplies; ▪ Capability to access trainings of other sorts. The evaluation found that multi-purpose cash assistance is not just about the distribution of cash: it plays an important role in restoring dignity to endemically disadvantaged populations. However, one important sector that the benefits of the PHASE program did not extend to was on increased access to adequate shelter and housing—a significant factor impinging upon participants’ overall well-being and long-term resilience. Multi-round distributions (as opposed to one-off distributions) reinforced the resilience of program participants who lived in communities affected by violence and increased the food security of those households. However, some key informants felt this approach also slightly weakened the participants in that it also served to keep them always waiting for more assistance. While the practice of not including men in all aspects of PHASE activities/ the focus on empowering and targeting women did bring about an increase in women's agency, it also became a source of conflict in beneficiary households. Multiple participants indicated that the inclusion of technical trainings and planning for small businesses (i.e., tailors/ sewers, carpentry, hair braiding, food production, etc.) in future VSLA programs would better enable participants to become more self-sufficient. Reported registration irregularities (such as the reduction in household size for some participants), and ruptures may have reduced the positive impact of these activities. Capacity building to train local partners who are familiar with the realities of each zone of operation on how to collect reliable data in the local language may produce better results than using program staff who are unfamiliar with the intervention zone and the language spoken there. An increased focus on training local data collectors how to use data collection tools is necessary. Another key lesson is the significant negative impact the ongoing war in Ituri had on Mercy Corps' ability to deliver the program as designed (i.e., insecurity prevented PHASE from completing the final rounds of distributions in Boga and Mwanga) and the impact this had on the sustainability of the benefits that the PHASE program was designed to bring about in the lives of program participants. On one hand, the strategy of three rounds of distributions to the same beneficiaries is to be encouraged—one single round of distributions, while helpful in the short-term, does little to increase the overall resilience of vulnerable households. However, Mercy Corps’ capability to carry out the planned third distributions was problematic for two reasons: in several instances insecurity prevented Mercy Corps from carrying out the third distribution; in other instances, when calm returned to areas from which IDPs had come, many returned to the villages where they had previously lived and so were unable to benefit from the third round of assistance. On the other hand, some returnees (Nyakunde) indicated certain IDPs stayed in order to receive all three rounds of distributions and/ or arrived for the distribution and then hired cars to go back to their place of origin. Mercy Corps may want to consider how the promise of multiple rounds of assistance over time might affect program beneficiaries' relocation decisions. Another result of doing three rounds of distributions was that it reportedly revealed the details of Mercy Corps' system to PHASE participants. Some beneficiaries told the evaluation team that in Nyakunde, this allowed individuals to devise ways to manipulate or work around program requirements in order to receive (more) assistance than they were entitled to, with one beneficiary stating s/he was on both IDP and returned lists. Nb. The evaluation team was unable to determine either the extent or the veracity of these recorded observations. Mercy Corps PHASE/Final Evaluation Internal Socorro 36 16 June 2022 Global Humanitarian Consultants War and conflict was the most significant and pervasive barrier to reducing endemic food insecurity and sustainably enhancing participants' self-reliance in the medium- to long-term. Future early recovery programs in conflict situations must do a better job of planning for the possibility of activity interruption and be prepared to adapt activity strategies so as to better meet program objectives; this may include considering whether to shorten the time between CVA distributions so that all CVA program beneficiaries can receive the complete package. Such a decision, if it were to be made, would have significant impacts on the ability of Mercy Corps to conduct the much-valued, longer program span of VSLA and agricultural activities. In terms of practical recommendations, Mercy Corps might consider how to more accurately estimate the tonnage of goods that need to be transported to a site well in advance of the distribution day so as to eliminate risks relating to moving excess goods into and then back out again along uncertain roads. This estimation should be relayed to suppliers so that they know the quantity of goods to bring. Extra sheets should be included in the estimation to help other traders display their products, as well. 1. MERCY CORPS STAFF Both program staff and non-program staff agreed on the importance of first and second phase response combined together in one program to be important. However, it is important to note that those observations were made based on personal perceptions, rather than on hard data. A few staff made observations related to gender aspect of the intervention, such as “if we do not involve men [better] in the response, it will create problems in the households of our beneficiaries”, or related to the final beneficiaries, such as “[PHASE] did better to respond to the needs of the returned populations, but there is still room for improvement”. Other staff also pointed out that internal Mercy Corps procedures and collaboration among different programs as both opportunities and lessons learned, such as: “complementarity between the PHASE and the SAFER programs”; or “difficulties with internal procedures, non-respect of job descriptions, delays in payments, etc.” In several cases, Mercy Corps staff contested the 3-round assistance as potentially hampering the return of displaced populations to their zones of origin (pull factor), and suggested the possibility to divide activities between displaced and returned populations. Certain staff also suggested a two-part instead of three-part approach, while increasing VSLA and permaculture activities. Furthermore, many Mercy Corps staff highlighted that the PHASE program’s specific design improved beneficiaries’ sense of dignity, something that many beneficiaries themselves talked about in their own terms, such as: “we were feeling better, we could provide for our family”; “there was a lot of happiness in my home”; as well as “I felt like I can provide again for my children”. The VSLA and agriculture programs (more specifically permagarden activities) had a high impact on self-sufficiency (auto-prise en charge), which was often lacking among beneficiaries that only benefited from food and CVA distributions. The conclusion is straightforward: more beneficiaries should participate in VSLA and agriculture programs; and where possible, VSLA participants should receive both structural and financial support, such as coaching and support to develop IGAs. V. Conclusions and Recommendations for Management Action See Executive Summary for key conclusions and recommendations. Mercy Corps PHASE/Final Evaluation Internal Socorro 37 16 June 2022 Global Humanitarian Consultants VI. Evaluation Framework A. Project Results What Changes Have Been Achieved? (Coherence, Sustainability) Specific Evaluation Question Relevant Indicator MC staff Direct beneficiaries Other community members External stakeholders Other (TBS) Do the quantitative data for logical framework indicators as collected through rolling endline surveys (enhanced PDMs) reflect qualitative observations of changes in participant outcomes? All indicators FG FG What (if any) unanticipated results has the program caused? O4, O1.3, O2.2, O3.2, O3.3 FG, AS KII, FG, IS, CRM KII KII In the face of ongoing security crises and population migration, to what extent are the effects and changes among participants sustainable? n/a FG, AS KII, FG, IS KII B. Project Logic What was the contribution of the PHASE project to the results obtained? (Effectiveness, Impact, Coherence) Specific Evaluation Question Relevant Indicator MC staff Direct Beneficiaries Other community members External stakeholders Other (TBS) How have project activities improved the food security of households exposed to conflict, including the autonomy of women and youth? O1, O2, O3, O4 per type of benef. FG, KII, IS KII KII (family members) To what extent were different transfer modalities (cash and/or vouchers) effective and relevant in achieving the project objectives? O1, O2, O3, O4 per intervention FG, AS Mercy Corps PHASE/Final Evaluation Internal Socorro 38 16 June 2022 Global Humanitarian Consultants How did different modality types affect local markets? Were there impacts in terms of empowerment of women and youth particularly? Gender sensitive O4, O1.3, O3.2, O3.3 KII, IS, CRM KII What are the contribution of the project in terms of well-being, particularly in terms of food, on participants, and especially on women and youth? Are these effects positive or negative? O4, O1.3, O3.2, O3.3 per gender of participant FG, KII, IS, CRM KII To what extent has the food assistance provided by the project, in cash or food, improved the food security of participants? O1, O2, O3 FG, KII, IS KII KII To what extent has the project’s implementation, training and support of VSLAs improved the food security and self-sufficiency of participants? O3.3 FG, KII, IS KII To what extent has access to training in agricultural techniques, inputs and farming tools provided by the project improved the living conditions of the participants, particularly in terms of strengthening their food security? O2.2 To what extent have VSLAs and agricultural support created economic opportunities for women and youth? O3.3 FG, KIIs, IS KII KII (local authorities, civil society) Mercy Corps PHASE/Final Evaluation Internal Socorro 39 16 June 2022 Global Humanitarian Consultants C. Lessons Learned What could Mercy Corps have done differently? (Efficiency, Effectiveness) Specific Evaluation Question Relevant Indicator MC staff Direct beneficiaries Other community members External stakeholders Other (TBS) What are the lessons learned and recommendations that can be drawn from the implementation of the different activities, the strategy and the context of the projects? n/a FG, AS KII, CRM KII KII What could have been done differently or better? n/a FG, AS KII, CRM KII KII Mercy Corps PHASE/Final Evaluation Internal Socorro 40 16 June 2022 Global Humanitarian Consultants VII.Sources of Information A. Desk review material Title Date Source Providing Humanitarian Assistance for Self-Reliance in Eastern Congo (PHASE): Performance Indicator Reference Sheets No date Downloaded on Feb 28 and May 16 Mercy Corps/ USAID Providing Humanitarian Assistance for Self-Reliance in Eastern Congo (PHASE): USAAID/FFP Emergency Food Security Project Application September 2020 Mercy Corps Providing Humanitarian Assistance for Self-Reliance in Eastern Congo (PHASE): Indicator Table January 2021 Mercy Corps/ USAID Providing Humanitarian Assistance for Self-Reliance in Eastern Congo (PHASE): Rapport d’enquête de satisfaction foire aux vivres à Boga, territoire d’lrumu, Ituri March 2021 Mercy Corps/ USAID Providing Humanitarian Assistance for Self-Reliance in Eastern Congo: Rapport de Suivi Post Intervention _ PHASE BOGA April 2021 Mercy Corps/ USAID Providing Humanitarian Assistance for Self-Reliance in Eastern Congo: Rapport de Suivi Post Intervention_ PHASE AUNGBA June 2021 Mercy Corps/ USAID Providing Humanitarian Assistance for Self-Reliance in Eastern Congo: Rapport de Suivi Post Intervention_ PHASE AUNGBA II June 2021 Mercy Corps/ USAID Providing Humanitarian Assistance for Self-Reliance in Eastern Congo: Rapport d’enquête de satisfaction _ PHASE Aungba July 2021 Mercy Corps/ USAID Providing Humanitarian Assistance for Self-Reliance in Eastern Congo: Rapport d’enquête de satisfaction _ PHASE Mwanga August 2021 Mercy Corps/ USAID Providing Humanitarian Assistance for Self-Reliance in Eastern Congo: Rapport de Suivi Post Intervention _ PHASE MWANGA II September 2021 Mercy Corps/ USAID Providing Humanitarian Assistance for Self-Reliance in Eastern Congo: Rapport d’enquête de satisfaction _ PHASE Aungba III October 2021 Mercy Corps/ USAID Providing Humanitarian Assistance for Self-Reliance in Eastern Congo (PHASE): FY22 Q1 Report Oct. 12021 – Dec. 31 2021 Mercy Corps Providing Humanitarian Assistance for Self-Reliance in Eastern Congo (PHASE): Rapport d’enquête de satisfaction foire aux vivres AME à Nyakundé I. territoire, d’Ituri, en province d’Ituri November 2021 Mercy Corps/ USAID Providing Humanitarian Assistance for Self-Reliance in Eastern Congo: Rapport de Suivi Post Intervention_ PHASE _ NYAKUNDE November 2021 Mercy Corps/ USAID Providing Humanitarian Assistance for Self-Reliance in Eastern Congo: Rapport de Suivi Post Intervention_ PHASE AUNGBA November 2021 Mercy Corps/ USAID Providing Humanitarian Assistance for Self-Reliance in Eastern Congo (PHASE): PHASE AR – Narrative Report November 2021 Mercy Corps/ USAID Mercy Corps PHASE/Final Evaluation Internal Socorro 41 16 June 2022 Global Humanitarian Consultants Providing Humanitarian Assistance for Self-Reliance in Eastern Congo (Phase): Rapport d’enquête de satisfaction _ PHASE 2/ NYAKUNDE December 2021 Mercy Corps/ USAID Providing Humanitarian Assistance for Self-Reliance in Eastern Congo (Phase): Rapport d’enquête de satisfaction foire aux vivres DJAIBA, territoire de DJUGU, en ITURI February 2022 Mercy Corps/ USAID Mercy Corps PHASE/Final Evaluation Internal Socorro 42 16 June 2022 Global Humanitarian Consultants B. Qualitative Data Sources: KIIs and FGDs F F Total H H Total M M Total Total Site FG KII FG KII FG KII Bunia 2 2 7 7 9 Partner 2 2 2 MC Staff 7 7 7 Djaiba 48 8 56 36 17 53 12 3 15 124 Partner 3 3 3 Leader 12 1 13 12 12 25 Beneficiary 48 6 54 24 7 31 85 Non-Beneficiary 2 2 6 6 8 MC Staff 3 3 3 Djalasiga 48 13 61 60 34 94 12 12 167 Leader 12 1 13 12 12 25 Beneficiary 48 7 55 36 25 61 116 Non-Beneficiary 6 6 12 8 20 26 Goma 2 2 1 1 3 Partner 2 2 2 MC Staff 1 1 1 Nyakunde 36 8 44 48 10 58 12 12 114 Partner 1 1 1 Leader 12 2 14 12 12 26 Beneficiary 36 6 42 36 4 40 82 Non-Beneficiary 2 2 3 3 5 Grand Total 132 29 161 144 65 209 36 11 47 417 F = femmes / female participant, H = homme / male participant, M = mixed / gender non-specific or gender identity is not stated to avoid re-identification Total KII participants – 105. Total FGD participants – 312 • 161 interviewed people were identified as women (43.5%), women constituted 31% of people interviewed through key informant interviews • 76 participants were identified in a community leadership role (18%), including leaders of youth • The majority of participants (283, or 68%) were direct beneficiaries, this number is excluding community leaders (76) who were also beneficiaries. • 8 partners (mostly suppliers) and 11 Mercy Corps staff, and 39 community members who did not benefited from the program were interviewed Mercy Corps PHASE/Final Evaluation Internal Socorro 43 16 June 2022 Global Humanitarian Consultants VIII. SOW Mercy Corps PHASE/Final Evaluation Internal Socorro XLIV 16 June 2022 Global Humanitarian Consultants Mercy Corps PHASE/Final Evaluation Internal Socorro XLV 16 June 2022 Global Humanitarian Consultants Mercy Corps PHASE/Final Evaluation Internal Socorro XLVI 16 June 2022 Global Humanitarian Consultants Mercy Corps PHASE/Final Evaluation Internal Socorro XLVII 16 June 2022 Global Humanitarian Consultants Mercy Corps PHASE/Final Evaluation Internal Socorro 48 16 June 2022 Global Humanitarian Consultants IX. Data Collection Instruments A. KII and FGD Evaluation Questionnaires Évaluation finale: MC PHASE: Bénéficiaires Méthodologie: Discussion de groupe Date: Lieu: Durée: Facilitateur/trice: Co-facilitateur/trice: Composant: o Distribution o VSLA o Agri Nombre de participants: 17-24 25-59 +60 Total # Femmes X # Hommes X Communauté Hote IDPs Autres Notes aux traducteurs/enquêteurs : N'OUBLIEZ PAS D'UTILISER DES MOTS SIMPLES : par exemple : lorsque vous posez des questions sur leur expérience avec les différents piliers de la réponse, assurez-vous d'expliquer en termes simples à quoi vous faites référence, "la communication des risques et l'engagement communautaire" ne leur parlera pas beaucoup. Il faudra dire quelque chose du genre : « les équipes qui se sont rendues dans les ménages pour parler de la maladie », etc. ▪ Merci – Je vous remercie pour l’entretien et votre temps aujourd’hui ▪ Votre nom – Je m’appelle __________ et je travaille comme consultante pour Mercy Corps. Nous menons une évaluation finale externe du programme PHASE dont les activités comprennent la fourniture d'une aide alimentaire, le renforcement de la production agricole et l'amélioration du revenu des ménages pour les plus vulnérables parmi les personnes déplacées à l'intérieur du pays (PDI), les rapatriés et les familles d'accueil touchées par le conflit. L’objectif de notre travail est d’identifier les succès de l’opération, ses lacunes (points faibles) ; les leçons apprises afin de donner des recommandations afin d’améliorer nos opérations futures. ▪ Confidentialité / autorisation d’enregistrer – Serait-ce acceptable que j’enregistre cette entretien, juste comme une sauvegarde de mes notes ? Nous supprimerons les enregistrements une fois que nous aurons fini de taper et de réviser nos notes. Cette entrevue est entièrement confidentielle, et votre nom sera retiré de toutes les transcriptions et enregistrements de l’entrevue. Les citations ne seront attribuées à aucune personne. Tout ce que vous dites ici n’affectera aucunement les soutiens futurs que vous pourriez recevoir. Pour nous il est vraiment important de savoir si les choses ont été mal parce que ça nous aide à nous améliorer dans le futur. ▪ Durée : La discussion devrait durer environ 60 à 90 minutes – est-ce que ça va ? Si vous n’avez pas beaucoup de temps, faites-le moi savoir et je me concentrerai sur les questions les plus importantes. De plus, c’est correct si vous ne connaissez pas la réponse à une question ; si vous préférez ne pas répondre ou si la question n’est pas pertinente pour votre rôle – faites￾le moi savoir et j’adapterai la discussion en conséquence. ▪ Possibilité de poser des questions – Avez-vous des questions avant de commencer ? Bien sûr, vous êtes également invités à poser des questions lors de l’entretien si quelque chose n’est pas clair. S’il y a des éléments que je n’ai pas évoqués lors de l’entretien, n’hésitez pas à les partager vers la fin. Participer dans cette discussion est complètement sur une base volontaire et vous pouvez mettre fin à la conversation quand vous le voulez. 000 Comment Mercy Corps a-t-elle travaillé dans votre communauté ? Mercy Corps PHASE/Final Evaluation Internal Socorro 49 16 June 2022 Global Humanitarian Consultants 001 À votre avis, quels ont été les résultats les plus importants de [la composante spécifique du programme] pour vous et votre communauté ? 002 [Pour la distribution] : Comment la distribution d'argent vous a-t-elle aidé à préparer l'avenir? [Pour VSLA et AGRI uniquement] : Que pensez-vous qu'il adviendra de [VSLA] / [outils agricoles] dans l'année ou les années à venir ? Pensez-vous que les VSLA / la formation agricole et les outils et semences agricoles seront encore utiles dans 2 ans ? Si oui, pourquoi ? Quels sont les aspects qui les feront durer ? Si non, pourquoi ? Quels sont les obstacles ou les défis à la durabilité de ces programmes ? 003 Comment accédez-vous à la nourriture dans cette communauté ? En parlant de votre bien-être et surtout de la nourriture, qu'est-ce qui s'est le plus amélioré dans votre famille et dans votre communauté depuis que Mercy Corps a commencé [la composante spécifique du programme]? [Pour l'agriculture seulement] : Aviez-vous accès à la terre avant le programme ? L'avez-vous maintenant ? 004 Pour la distribution] : Quel type d'assistance avez-vous reçu ? Vous a-t-elle aidé à surmonter les problèmes d'insécurité alimentaire ? Pouvez-vous donner un exemple ? 005 GROUPES D'HOMMES : Quel a été l'impact de [la composante spécifique du programme] sur la participation et l'autonomisation des femmes et des jeunes dans votre communauté ? GROUPES DE FEMMES : Vous sentez-vous investie d'un pouvoir d'action grâce à votre participation à [la composante du programme] ? Comment ? 006 Quelles activités auxquelles vous avez participé ont, selon vous, le plus contribué à votre bien-être, notamment sur le plan alimentaire ? 007 [Pour la distribution uniquement] : A votre avis, la quantité reçue était-elle suffisante pour permettre à un ménage de vivre pendant un mois ? Y a-t-il eu des obstacles à ce que les gens échangent les bons contre de la nourriture ? 008 [Pour VSLA uniquement] : Que pensez-vous des associations villageoises d'épargne et de prêt (VSLA) dans votre communauté ? Quels aspects ont été utiles ? Pourquoi ? Quels aspects du programme ont été les moins utiles ? Pourquoi ? 009 [Pour l'agriculture uniquement] : Que pensez-vous des formations et des outils agricoles qui ont été distribués dans votre communauté ? Les outils étaient-ils de bonne qualité ? Les semences étaient￾elles adaptées à la région dans laquelle vous vivez ? Quels aspects de la formation ont été utiles ? Quels aspects de la formation n'ont pas été très utiles ? 010 [Pour VSLA et agri uniquement] : Quel est le rôle et les responsabilités des femmes dans votre communauté ? Avez-vous constaté des changements depuis que Mercy Corps a commencé à travailler ici ? 012 Y a-t-il autre chose que vous aimeriez ajouter ? Évaluation finale: MC PHASE: Membres de la communauté/Non￾bénéficiaires Méthodologie: Entretien avec Informateur-clé (KII) Date: Lieu: Durée: Facilitateur/trice: Co-facilitateur/trice: Status: O chef de la communauté/autorité locale (non participant) O membre de la famille (non participant) O membre de la communauté (non participant) Bénéficiaire : O femme O homme Mercy Corps PHASE/Final Evaluation Internal Socorro 50 16 June 2022 Global Humanitarian Consultants O 18-24 ans O 25 + Notes aux traducteurs/enquêteurs : N'OUBLIEZ PAS D'UTILISER DES MOTS SIMPLES : par exemple : lorsque vous posez des questions sur leur expérience avec les différents piliers de la réponse, assurez-vous d'expliquer en termes simples à quoi vous faites référence, "la communication des risques et l'engagement communautaire" ne leur parlera pas beaucoup. Il faudra dire quelque chose du genre : « les équipes qui se sont rendues dans les ménages pour parler de la maladie », etc. ▪ Merci – Je vous remercie pour l’entretien et votre temps aujourd’hui ▪ Votre nom – Je m’appelle __________ et je travaille comme consultante pour Mercy Corps. Nous menons une évaluation finale externe du programme PHASE dont les activités comprennent la fourniture d'une aide alimentaire, le renforcement de la production agricole et l'amélioration du revenu des ménages pour les plus vulnérables parmi les personnes déplacées à l'intérieur du pays (PDI), les rapatriés et les familles d'accueil touchées par le conflit. L’objectif de notre travail est d’identifier les succès de l’opération, ses lacunes (points faibles) ; les leçons apprises afin de donner des recommandations afin d’améliorer nos opérations futures. ▪ Confidentialité / autorisation d’enregistrer – Serait-ce acceptable que j’enregistre cette entretien, juste comme une sauvegarde de mes notes ? Nous supprimerons les enregistrements une fois que nous aurons fini de taper et de réviser nos notes. Cette entrevue est entièrement confidentielle, et votre nom sera retiré de toutes les transcriptions et enregistrements de l’entrevue. Les citations ne seront attribuées à aucune personne. Tout ce que vous dites ici n’affectera aucunement les soutiens futurs que vous pourriez recevoir. Pour nous il est vraiment important de savoir si les choses ont été mal parce que ça nous aide à nous améliorer dans le futur. ▪ Durée : La discussion devrait durer environ 60 à 90 minutes – est-ce que ça va ? Si vous n’avez pas beaucoup de temps, faites-le moi savoir et je me concentrerai sur les questions les plus importantes. De plus, c’est correct si vous ne connaissez pas la réponse à une question ; si vous préférez ne pas répondre ou si la question n’est pas pertinente pour votre rôle – faites-le moi savoir et j’adapterai la discussion en conséquence. ▪ Possibilité de poser des questions – Avez-vous des questions avant de commencer ? Bien sûr, vous êtes également invités à poser des questions lors de l’entretien si quelque chose n’est pas clair. S’il y a des éléments que je n’ai pas évoqués lors de l’entretien, n’hésitez pas à les partager vers la fin. Participer dans cette discussion est complètement sur une base volontaire et vous pouvez mettre fin à la conversation quand vous le voulez. 000 Veuillez décrire votre rôle/relations vis-à-vis du projet PHASE : - chef de la communauté/autorité locale (non participant) - membre de la famille (non participant) - membre de la communauté (non participant) 005 A Quel type d'assistance a été fourni dans votre communauté ? Pensez-vous que cela les a aidés à surmonter les problèmes d'insécurité alimentaire ? - Beaucoup / un peu / pas vraiment / pas du tout 001 A votre avis, quels ont été les résultats les plus importants du programme dans votre communauté ? 002 Quels seront les changements à moyen et long terme de la programmation de Mercy Corps sur les bénéficiaires et votre communauté ? 003 Avez-vous des suggestions à faire à Mercy Corps sur la manière dont ils pourraient améliorer certains aspects des futurs programmes afin de les rendre meilleurs pour votre communauté ? 004 Pour les personnes qui ont participé au programme, qu'est-ce qui, selon vous, a le plus amélioré leur bien-être, notamment en ce qui concerne la sécurité alimentaire et les activités génératrices de revenus ? 005 B [Pour les distribution] : Selon vous, le montant reçu était-il suffisant pour permettre à un ménage de vivre pendant un mois ? Y a-t-il eu des obstacles à ce que les gens échangent les bons contre de la nourriture Mercy Corps PHASE/Final Evaluation Internal Socorro 51 16 June 2022 Global Humanitarian Consultants 006 Après que Mercy Corps ait fourni une assistance à la communauté, avez-vous remarqué des changements sur le marché local ? - Beaucoup de changements / une peu de changements / pas de changements, je ne sais pas, je ne veux pas répondre Pouvez-vous donner un exemple ? 007 Pensez-vous que la participation au programme a renforcé l'autonomie des femmes et des jeunes de votre communauté ? - Beaucoup / un peu / pas vraiment / pas du tout, je ne sais pas, je ne veux pas répondre Pouvez-vous donner un exemple ? 008 [Pour les distribution] : Pensez-vous que l'aide alimentaire a amélioré la sécurité alimentaire des participants en termes d'accessibilité à la nourriture, de disponibilité de la nourriture, de stabilité de l'approvisionnement alimentaire et d'utilisation responsable de la nourriture ? - Beaucoup / un peu / pas vraiment / pas du tout, je ne sais pas, je ne veux pas répondre Pouvez-vous donner un exemple ? 009 [Pour VSLA] : Pensez-vous que la participation à la VSLA a amélioré les opportunités d'activités génératrices de revenus pour les bénéficiaires ? - Beaucoup / un peu / pas vraiment / pas du tout, je ne sais pas, je ne veux pas répondre Pouvez-vous donner un exemple ?" 010 [Pour agri] : Pensez-vous que la participation à la VSLA a amélioré les opportunités d'activités génératrices de revenus pour les bénéficiaires ? - Beaucoup / un peu / pas vraiment / pas du tout, je ne sais pas, je ne veux pas répondre Pouvez-vous donner un exemple ? L'aide de PHASE a-t-elle amélioré un ou plusieurs de ces piliers de la sécurité alimentaire ? : Accessibilité, disponibilité, stabilité de l’approvisionnement, utilisation responsable - Beaucoup / un peu / pas vraiment / pas du tout, je ne sais pas, je ne veux pas répondre Comment ? 011 Pensez-vous qu'il y a plus de femmes leaders maintenant qu'avant le début du programme dans votre communauté ? - Beaucoup plus / un peu plus / même qu’avant / moins, je ne sais pas, je ne veux pas répondre Pouvez-vous donner un exemple ? 012 Pensez-vous que le programme a atteint les résultats que vous attendiez ? - Totalement / un peu / pas vraiment / pas du tout, je ne sais pas, je ne veux pas répondre. Comment Mercy Corps a-t-il travaillé dans votre communauté ? 013 Que pensez-vous des partenaires/fournisseurs avec lesquels Mercy Corps a coopéré pour apporter de l'aide à votre communauté ? 014 Y a-t-il autre chose que vous aimeriez ajouter ? Évaluation finale: MC PHASE: Partie prenantes (externe) Méthodologie: Entretien avec Informateur-clé (KII) Date: Lieu: Durée: Facilitateur/trice: Co-facilitateur/trice: Status: O chef de la communauté/autorité locale (non participant) O membre de la famille (non participant) O membre de la communauté (non participant) Bénéficiaire : O femme O homme O 18-24 ans O 25 + Notes aux traducteurs/enquêteurs : N'OUBLIEZ PAS D'UTILISER DES MOTS SIMPLES : par exemple : lorsque vous posez des questions sur leur expérience avec les différents piliers de la réponse, assurez-vous d'expliquer en termes simples à quoi vous faites référence, "la communication des risques et l'engagement communautaire" ne leur parlera pas beaucoup. Il faudra dire quelque chose du genre : « les équipes qui se sont rendues dans les ménages pour parler de la maladie », etc. Mercy Corps PHASE/Final Evaluation Internal Socorro 52 16 June 2022 Global Humanitarian Consultants ▪ Merci – Je vous remercie pour l’entretien et votre temps aujourd’hui ▪ Votre nom – Je m’appelle __________ et je travaille comme consultante pour Mercy Corps. Nous menons une évaluation finale externe du programme PHASE dont les activités comprennent la fourniture d'une aide alimentaire, le renforcement de la production agricole et l'amélioration du revenu des ménages pour les plus vulnérables parmi les personnes déplacées à l'intérieur du pays (PDI), les rapatriés et les familles d'accueil touchées par le conflit. L’objectif de notre travail est d’identifier les succès de l’opération, ses lacunes (points faibles) ; les leçons apprises afin de donner des recommandations afin d’améliorer nos opérations futures. ▪ Confidentialité / autorisation d’enregistrer – Serait-ce acceptable que j’enregistre cette entretien, juste comme une sauvegarde de mes notes ? Nous supprimerons les enregistrements une fois que nous aurons fini de taper et de réviser nos notes. Cette entrevue est entièrement confidentielle, et votre nom sera retiré de toutes les transcriptions et enregistrements de l’entrevue. Les citations ne seront attribuées à aucune personne. Tout ce que vous dites ici n’affectera aucunement les soutiens futurs que vous pourriez recevoir. Pour nous il est vraiment important de savoir si les choses ont été mal parce que ça nous aide à nous améliorer dans le futur. ▪ Durée : La discussion devrait durer environ 60 à 90 minutes – est-ce que ça va ? Si vous n’avez pas beaucoup de temps, faites-le moi savoir et je me concentrerai sur les questions les plus importantes. De plus, c’est correct si vous ne connaissez pas la réponse à une question ; si vous préférez ne pas répondre ou si la question n’est pas pertinente pour votre rôle – faites-le moi savoir et j’adapterai la discussion en conséquence. ▪ Possibilité de poser des questions – Avez-vous des questions avant de commencer ? Bien sûr, vous êtes également invités à poser des questions lors de l’entretien si quelque chose n’est pas clair. S’il y a des éléments que je n’ai pas évoqués lors de l’entretien, n’hésitez pas à les partager vers la fin. Participer dans cette discussion est complètement sur une base volontaire et vous pouvez mettre fin à la conversation quand vous le voulez. 000 Veuillez décrire votre rôle/relations vis-à-vis du projet PHASE : 001 Selon vous, quels ont été les résultats les plus importants de [la composante spécifique du programme en fonction du type de KII] sur les communautés affectées ? 002 Que pensez-vous de la durabilité des résultats de la composante spécifique du programme en fonction du type de KII ? Quels sont ceux qui sont durables et ceux qui ne le sont pas ? 003 Que proposeriez-vous à Mercy Corps pour que la composante spécifique du programme en fonction du type de KII soit encore meilleur pour les communautés affectées ? 004 Que pensez-vous qui aurait pu être fait différemment/meilleur dans la composante spécifique du programme en fonction du type de KII ? 005 Comment évaluez-vous l'efficacité et la pertinence de la composante sur laquelle vous travaillez avec Mercy Corps ? [question à adapter à la partie prenante]. 006 Quels ont été les résultats inattendus les plus surprenants (positifs ou négatifs) de la composante spécifique du programme en fonction du type de KII ? 007 Quels ont été les résultats inattendus les plus surprenants (positifs ou négatifs) de la composante spécifique du programme en fonction du type de KII ? 008 Quel a été votre plus grand défi et votre plus grande valeur ajoutée ? 009 Y a-t-il autre chose que vous souhaiteriez ajouter ? Évaluation finale: MC PHASE: Membres de la communauté bénéficiaires Méthodologie: Entretien avec Informateur-clé (KII) Date: Lieu: Durée: Facilitateur/trice: Mercy Corps PHASE/Final Evaluation Internal Socorro 53 16 June 2022 Global Humanitarian Consultants Co-facilitateur/trice: Composante: O aide alimentaire O VSLA O agriculture Bénéficiaire : O femme O homme O 18-24 ans O 25 + Notes aux traducteurs/enquêteurs : N'OUBLIEZ PAS D'UTILISER DES MOTS SIMPLES : par exemple : lorsque vous posez des questions sur leur expérience avec les différents piliers de la réponse, assurez-vous d'expliquer en termes simples à quoi vous faites référence, "la communication des risques et l'engagement communautaire" ne leur parlera pas beaucoup. Il faudra dire quelque chose du genre : « les équipes qui se sont rendues dans les ménages pour parler de la maladie », etc. ▪ Merci – Je vous remercie pour l’entretien et votre temps aujourd’hui ▪ Votre nom – Je m’appelle __________ et je travaille comme consultante pour Mercy Corps. Nous menons une évaluation finale externe du programme PHASE dont les activités comprennent la fourniture d'une aide alimentaire, le renforcement de la production agricole et l'amélioration du revenu des ménages pour les plus vulnérables parmi les personnes déplacées à l'intérieur du pays (PDI), les rapatriés et les familles d'accueil touchées par le conflit. L’objectif de notre travail est d’identifier les succès de l’opération, ses lacunes (points faibles) ; les leçons apprises afin de donner des recommandations afin d’améliorer nos opérations futures. ▪ Confidentialité / autorisation d’enregistrer – Serait-ce acceptable que j’enregistre cette entretien, juste comme une sauvegarde de mes notes ? Nous supprimerons les enregistrements une fois que nous aurons fini de taper et de réviser nos notes. Cette entrevue est entièrement confidentielle, et votre nom sera retiré de toutes les transcriptions et enregistrements de l’entrevue. Les citations ne seront attribuées à aucune personne. Tout ce que vous dites ici n’affectera aucunement les soutiens futurs que vous pourriez recevoir. Pour nous il est vraiment important de savoir si les choses ont été mal parce que ça nous aide à nous améliorer dans le futur. ▪ Durée : La discussion devrait durer environ 60 à 90 minutes – est-ce que ça va ? Si vous n’avez pas beaucoup de temps, faites-le moi savoir et je me concentrerai sur les questions les plus importantes. De plus, c’est correct si vous ne connaissez pas la réponse à une question ; si vous préférez ne pas répondre ou si la question n’est pas pertinente pour votre rôle – faites-le moi savoir et j’adapterai la discussion en conséquence. ▪ Possibilité de poser des questions – Avez-vous des questions avant de commencer ? Bien sûr, vous êtes également invités à poser des questions lors de l’entretien si quelque chose n’est pas clair. S’il y a des éléments que je n’ai pas évoqués lors de l’entretien, n’hésitez pas à les partager vers la fin. Participer dans cette discussion est complètement sur une base volontaire et vous pouvez mettre fin à la conversation quand vous le voulez. 000 Veuillez décrire les activités de la PHASE auxquelles vous avez participé : - aide alimentaire (cash, voucher, foire) - VSLA - soutien agricole 001 A votre avis, quels ont été les résultats les plus importants du programme pour vous et votre famille ? 002 De ce que vous avez appris/reçu/gagné, qu'est-ce qui vous restera en mémoire ou dont vous et votre famille bénéficierez le plus dans l'année à venir ? Pourquoi ? Quel a été l'aspect le moins efficace ou le moins utile du programme ? Pouvez-vous donne un exemple ? 003 Que proposeriez-vous à Mercy Corps pour que [la composante spécifique du programme] soit encore meilleure pour vous et votre famille ? 004 Comment accédez-vous à la nourriture dans cette communauté ? Dans votre foyer, y a-t-il eu une amélioration liée au bien-être de votre famille depuis que Mercy Corps a commencé sa [ composante spécifique du programme] ? Si oui, quelle a été la plus grande amélioration liée à votre bien-être et surtout à la nourriture depuis que Mercy Corps a commencé son programme ? Si Mercy Corps PHASE/Final Evaluation Internal Socorro 54 16 June 2022 Global Humanitarian Consultants non, pouvez-vous nous donner une raison pour laquelle les programmes de Mercy Corps n'ont pas amélioré le bien-être de votre famille ? [Pour l'agriculture uniquement] : Aviez-vous accès à la terre avant le programme ? L'avez-vous maintenant ? 005 [Pour les distribution] : Parlez-moi du type d'aide que vous avez reçu (cash/voucher/foire). Pensez-vous qu'elle était appropriée à vos besoins ? - Très appropriée / un peu appropriée / pas vraiment appropriée/ pas du tout appropriée, je ne sais pas, je ne veux pas répondre Pouvez-vous donner un exemple ? 006 [Pour les distribution] Après avoir reçu l'assistance dans la communauté, avez-vous remarqué des changements sur le marché local ? - Beaucoup de changements / un peu de changements / pas de changements, je ne sais pas, je ne veux pas répondre Pouvez-vous donner un exemple ? 007 RÉPONDANT HOMME: Pensez-vous que la participation au programme a renforcé l'autonomie des femmes et des jeunes de votre communauté ? - Beaucoup / un peu / pas vraiment / pas du tout, je ne sais pas, je ne veux pas répondre Comment ? Pouvez-vous donner un exemple ? RÉPONDANTE FEMME: Vous sentez-vous davantage capable grâce à votre participation au programme ? - Très fort / un peu / pas vraiment / pas du tout, je ne sais pas, je ne veux pas répondre Comment ? Pouvez-vous donner un exemple ? 008 Quelles activités auxquelles vous avez participé ont, selon vous, le plus contribué à votre bien-être, notamment sur le plan alimentaire ? 009 [Pour la distribution] : Pensez-vous que l'aide alimentaire que vous avez reçue a amélioré votre bien-être, notamment sur le plan alimentaire ? - Beaucoup / un peu / pas vraiment / pas du tout, je ne sais pas, je ne veux pas répondre Comment ? Pouvez-vous donner un exemple ? A votre avis, la quantité reçue était-elle suffisante pour permettre à un ménage de vivre pendant un mois ? Y a-t-il eu des obstacles à ce que les gens échangent les bons contre de la nourriture ? 010 [[Pour VSLA] : Pensez-vous que la participation à la VSLA a amélioré votre bien-être, votre autosuffisance et vos opportunités économiques ? - Beaucoup / un peu / pas vraiment / pas du tout, je ne sais pas, je ne veux pas répondre Comment ? Pouvez-vous donner un exemple ? Pouvez-vous donner un exemple d'une AGR résultant de votre participation à la VSLA ? 011 [Pour l'agriculture] : Pensez-vous que la participation aux formations et la réception d'outils agricoles et de semences ont amélioré votre bien-être et votre autosuffisance ? - Beaucoup / un peu / pas vraiment / pas du tout, je ne sais pas, je ne veux pas répondre Comment ? Pouvez-vous donner un exemple ? 012 [Pour VSLA et agri] : Pensez-vous qu'il y a plus de femmes leaders maintenant qu'avant le début du programme dans votre communauté ? - Beaucoup / un peu / pas vraiment / pas du tout, je ne sais pas, je ne veux pas répondre Comment ? Pouvez-vous donner un exemple ? 013 Pensez-vous que le programme a atteint les résultats que vous attendiez ? - Totalement / un peu / pas vraiment / pas du tout, je ne veux pas répondre Comment ? Pouvez-vous donner un exemple ? Qu'est-ce qui a permis à Mercy Corps de mettre en œuvre le programme dans votre communauté ? 014 Y a-t-il autre chose que vous aimeriez ajouter ? Mercy Corps PHASE/Final Evaluation Internal Socorro 55 16 June 2022 Global Humanitarian Consultants B. Online Survey (MC Staff) Mercy Corps PHASE/Final Evaluation Internal Socorro 56 16 June 2022 Global Humanitarian Consultants Mercy Corps PHASE/Final Evaluation Internal Socorro 57 16 June 2022 Global Humanitarian Consultants Mercy Corps PHASE/Final Evaluation Internal Socorro 58 16 June 2022 Global Humanitarian Consultants Mercy Corps PHASE/Final Evaluation Internal Socorro 59 16 June 2022 Global Humanitarian Consultants Mercy Corps PHASE/Final Evaluation Internal Socorro 60 16 June 2022 Global Humanitarian Consultants Mercy Corps PHASE/Final Evaluation Internal Socorro 61 16 June 2022 Global Humanitarian Consultants Mercy Corps PHASE/Final Evaluation Internal Socorro 62 16 June 2022 Global Humanitarian Consultants