USAID REPORT JEOP FINAL PERFORMANCE EVALUTION USAID.GOV | 1 JOINT EMERGENCY OPERATION (JEOP 1.0) FINAL PERFORMANCE EVALUATION REPORT May 2023 Cooperative Agreement No: AID-FFP-A-12-00009 This publication was produced at the request of CRS/ETH on behalf of the JEOP consortium. It was prepared by JaRco Consulting USAID REPORT JEOP FINAL PERFORMANCE EVALUTION USAID.GOV | 2 ABSTRACT The purpose of this final performance evaluation is to provide an independent examination of the overall progress and accomplishments of the Joint Emergency Operation (JEOP) decision-making and future programming. JEOP programmatic activities are centered around the provision of emergency food assistance. The evaluation covered the implementation period from August 2012 to April-2022 and aimed to assess two Strategic Objectives (SO): • SO1: Transitory food insecure populations have met their emergency food needs, and • SO2: Targeted households protect their assets. Additionally, the evaluation assessed how management, implementation and external factors affected results. A mixed research approach was employed combining qualitative and quantitative methods. JEOP implementation areas included Amhara, Oromia, Southern Nations Nationalities and People’s Region (SNNPR), Sidama, Tigray and Dire Dawa City Administration. Sampling involved grouping the target woredas (127) into 35 clusters based on geographic proximity, historical administrative structure, and livelihood systems. The target sample size was 12,600 participant households, with 11,727 achieved due to inaccessibility of some woredas in Oromia and Tigray. The evaluation found that JEOP was instrumental in supporting transitory food insecure households and had a limited impact in reducing stress sale of productive assets. Agriculture is the main means of livelihood in all regions, although reliance on agriculture decreased, while reliance on relief assistance and remittance/family support grew in Tigray following the conflict. Moderate levels of food insecurity were found in all regions. Causes of vulnerability varied across regions, with the most common factors being environmental shocks (primarily recurrent drought/ shortage of rain, flooding, low soil fertility), compounded by conflicts. Challenges that affected program implementation included partial family targeting, distribution delays, targeting errors and bias. An important lesson from the decade-long implementation period is the need for JEOP to form and/or strengthen linkages with development initiatives as an ongoing exit strategy, which has the potential to strengthen community resilience and possibly reduce the need for aid. This sentiment was repeatedly expressed by multiple participants, non-participants, and GoE representatives. USAID REPORT JEOP FINAL PERFORMANCE EVALUTION USAID.GOV | 3 ACKNOWLEDGEMENT JaRco would like to acknowledge the contributions of numerous stakeholders who supported the accomplishment of this evaluation conducted in the Regional States of Amhara, Tigray, Dire Dawa City Administration, Oromia, Sidama and SNNPR. Firstly, JaRco would like to express our gratitude to CRS for the opportunity to undertake the evaluation and for their proactive support throughout the evaluation. CRS is the consortium lead for JEOP funded by the United States Agency for International Development/Food For Peace (USAID/FFP) from 2012 to 2023. JaRco would also like to extend appreciation to CRS management and technical staff for their participation and assistance throughout all stages of the evaluation process. Secondly, JaRco would like to acknowledge JEOP implementing partners – CARE, Food for the Hungry Ethiopia (FHE), Hararghe Catholic Secretariat (HCS), Meki Catholic Secretariat (MCS), ORDA, Relief Society of Tigray (REST) and World Vision (WV) for their coordination and unreserved effort during data collection. We would also like to extend our thanks to the field staff of these organizations for their comprehensive support during our visit to their respective operational areas. Finally, JaRco would like to acknowledge project participants, non-participants, implementing partner staff and stakeholders at all levels, who gave their time to answer the interview questions and provide considerable insights. Without their cooperation, this report would not have been possible. DISCLAIMER: The author’s views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development of the United States Government. USAID REPORT JEOP FINAL PERFORMANCE EVALUTION USAID.GOV | 4 TABLE OF CONTENTS ABSTRACT..................................................................................................................................................... 2 ACKNOWLEDGEMENT .............................................................................................................................. 3 LIST OF TABLES........................................................................................................................................... 6 LIST OF FIGURES ......................................................................................................................................... 8 LIST OF ANNEXES....................................................................................................................................... 9 ACRONYMS ................................................................................................................................................. 10 EXECUTIVE SUMMARY ............................................................................................................................ 12 I. EVALUATION PURPOSE AND EVALUATION QUESTIONS ......................................................... 14 I. BACKGROUND................................................................................................................................... 15 CHANGE IN JEOP CASELOADS IN RESPONSE TO CHANGING CONTEXT........................ 15 EVALUATION METHODS AND LIMITATIONS ........................................................................... 16 SAMPLE DESIGN ................................................................................................................................ 16 RESPONSE RATES.............................................................................................................................. 17 LIMITATIONS AND MITIGATING MEASURES ............................................................................ 18 CHARACTERISTICS OF STUDY POPULATION .......................................................................... 18 II. FINDINGS ................................................................................................................................................ 22 1. RELEVANCE ........................................................................................................................................ 22 1.1. FOOD SECURITY ....................................................................................................................... 22 1.2. MODALITIES ................................................................................................................................ 28 1.3. FOOD COMMODITY QUALITY CONTROL........................................................................... 30 2. EFFECTIVENESS..................................................................................................................................... 31 2.1. FOOD SHORTAGE...................................................................................................................... 31 2.2. ASSET PROTECTION................................................................................................................. 33 2.3. UTILIZATION OF JEOP FOOD/CASH ASSISTANCE ........................................................... 34 2.4. MEETING THE NEEDS OF THE VULNERABLE..................................................................... 37 2.5. TARGETING ................................................................................................................................. 40 2.6. JEOP EARLY WARNING SYSTEM (EWS) ............................................................................... 43 2.7. JEOP ACCOUNTABILITY MECHANISM................................................................................. 45 2.8. PROGRAM AGILITY.................................................................................................................... 47 3. EFFICIENCY............................................................................................................................................. 48 3.1. EFFICIENCY OF STAFFING AND MANAGEMENT STRUCTURES ................................... 48 3.2. TIMELINESS.................................................................................................................................. 49 3.3. WAIT-TIME AT FINAL DISTRIBUTION POINTS.................................................................. 51 3.4. PROXIMITY OF FINAL DISTRIBUTION POINTS ................................................................. 52 3.5. COORDINATION WITH KEY STAKEHOLDERS................................................................... 53 3.6. EFFICIENCY OF STAKEHOLDERS ENGAGED IN ASSISTANCE DELIVERY................... 55 USAID REPORT JEOP FINAL PERFORMANCE EVALUTION USAID.GOV | 5 4. IMPACT .................................................................................................................................................... 56 4.1. JEOP COMPLIANCE AND RISK MONITORING SYSTEM .................................................... 56 4.2. CONFLICT WITHIN THE COMMUNITY................................................................................ 57 4.3. IMPACT ON THE ENVIRONMENT .......................................................................................... 59 4.4. LENGTH OF PARTICIPANT ENGAGEMENT WITH JEOP.................................................. 60 4.5. MAINSTREAMING OF PROTECTION AND SAFEGUARDING .......................................... 61 5. COHERENCE........................................................................................................................................... 64 5.1. OBSERVATION AT FEDERAL, REGIONAL, WOREDA AND KEBELE LEVELS............... 64 5.2. ADDING VALUE WHILE AVOIDING DUPLICATION OF EFFORT................................... 66 III. LESSONS LEARNED............................................................................................................................. 68 IV. CONCLUSIONS .................................................................................................................................... 69 ACCOMPLISHMENTS................................................................................................................................ 69 CHALLENGES ............................................................................................................................................. 69 V. RECOMMENDATIONS.......................................................................................................................... 70 ANNEX 1: EVALUATION TEAM INFORMATION................................................................................ 73 USAID REPORT JEOP FINAL PERFORMANCE EVALUTION USAID.GOV | 6 LIST OF TABLES Table 1: JEOP Indicators 15 Table 2: Qualitative sample response rate 16 Table 3: Number of data collectors by region and gender 17 Table 4: Quantitative survey response rate 17 Table 5: Gendered household composition 19 Table 6: Age of household heads 20 Table 7: Disability status of household heads 21 Table 8: Households’ means of livelihood across regions (except Tigray) 21 Table 9: Households’ source of cash income 22 Table 10: Households in poor, borderline and acceptable FCS by region 23 Table 11: Mean and median rCSI by region 24 Table 12: Reasons for insufficiency of JEOP food assistance 25 Table 13: Perception of respondents on the adequacy of cash received 26 Table 14: Reasons for food insecurity 27 Table 15: Whether JEOP food items met the food preference of respondents 28 Table 16: Type of relief assistance received by respondents 29 Table 17: Types of cash assistance 29 Table 18: Types of market related challenges faced by respondents. 29 Table 19: Type of productive assets sold by households 33 Table 20: Households’ utilization of food commodities 34 Table 21: Reasons for sale of JEOP food assistance 35 Table 22: Utilization of cash-in-lieu of food 36 Table 23: Ability to fulfill family food needs without seeking external support 37 Table 24: Types of delegates used by vulnerable respondents 40 Table 25: Reasons why respondents did not collect their own rations 40 Table 26: Perceptions of inclusion and exclusion errors 41 Table 27: Feedback channels used for submitting and responding to feedback 46 Table 28: Impact of COVID-19 on the lives of participants 47 Table 29: Households who received JEOP food assistance as per the communicated schedule 49 Table 30: Sources of notification of food distribution schedule 50 Table 31: Channel preference for notification of food distribution schedule 51 Table 32: Average wait-time at FDPs 51 Table 33: Reasons for long wait-time at FDPs 52 Table 36: Time taken by respondents to travel to FDPs 52 USAID REPORT JEOP FINAL PERFORMANCE EVALUTION USAID.GOV | 7 Table 35: Conflict between participant and non-participant households 58 Table 37: Duration and pattern of household engagement with JEOP 60 Table 38: Basic needs that households met without resorting to negative coping mechanisms 60 Table 42: Location of security/safety issues faced by respondents 63 Table 43: Source of additional assistance households received 65 Table 44: Type of additional assistance households received 66 Table 45: Effect of being involved in two or more programs 67 USAID REPORT JEOP FINAL PERFORMANCE EVALUTION USAID.GOV | 8 LIST OF FIGURES Figure 1: Average household size 19 Figure 2: Proportion of households in each hunger scale category 23 Figure 3: Food items preferred the least by respondents 28 Figure 4: Respondents whose food needs during food gap months were fulfilled by JEOP 31 Figure 5: Typical length of time JEOP food assistance lasted for those that lasted less than a month 31 Figure 6: Respondents who received amount and mix of food assistance as per entitlement 32 Figure 7: Respondents who reported the ration was sufficient for a full month 32 Figure 8: Sale of productive assets to buy staple food 33 Figure 9: Households that sold different types of livestock to buy food in the year 2021 34 Figure 10: Type of livestock sold by households to buy food in the year 2021 34 Figure 11: Food items sold by respondents 35 Figure 12: Respondents’ perception on the suitability of FDPs for vulnerable participants 37 Figure 13: Respondents’ perception on the prioritization of vulnerable participants 38 Figure 14: Fully and partially targeted participant households 42 Figure 15: Households who reported the early warning information was both timely and relevant 43 Figure 16: Proportion of households who took risk education and demonstrable actions 44 Figure 17: Respondents who know and have used JEOP accountability mechanism 45 Figure 18: Response from JEOP accountability mechanism and respondents’ satisfaction 46 Figure 19: Notice provided to respondents regarding delivery of assistance 50 Figure 20: Households who paid for transportation to and from FDPs 53 Figure 21: Causes of conflict between participant and non-participant households 58 Figure 22: Respondents’ perception on JEOP positive impact on the environment 59 Figure 23: Receipt of other assistance concurrently to JEOP over the last 12 months 65 USAID REPORT JEOP FINAL PERFORMANCE EVALUTION USAID.GOV | 9 LIST OF ANNEXES Annex 1: Evaluation team information Annex 2: List of surveyed woredas Annex 3: USAID Approved Evaluation Scope of Work Annex 4a-d: household characteristics- sex, age, HH size Annex 5: Indicator estimate table Annex 6: Data collection tools Annex 7: Coping Strategy Index by Household Composition Annex 8: Household Hunger Scale Index by household composition Annex 9: Inclusion and exclusion error by disability-who goes to collect rations Annex 10a-c: FCS disaggregated by different levels Annex 11: Stakeholder mapping Annex 12: Highest education level attained by household heads Annex 13a-c: Means of transport to and from FD Annex 14: All sources of information or data Annex 15a-d: Early warning Annex 16a-b: Households’ main means of livelihood Annex 17: Risk education Annex 18a-b: Reasons for long waiting and Disability status of household members Annex 19a-b: Disability status of household heads Annex 20: Signed disclosure of conflict of interest Annex 21: Waiting time at FDPs Annex 22: Household heads with disability who experienced a safety security issue during food distribution Annex 23: Basic needs met by households by disability status Annex 24: Location where theft happens by disability Annex 25: Utilization of cash-for-transport Annex26: Reason for not being satisfied with the response to feedback USAID REPORT JEOP FINAL PERFORMANCE EVALUTION USAID.GOV | 10 ACRONYMS ARR Annual Results Report ARS Annual Results Survey CARE Cooperative for Assistance and Relief Everywhere CHD Community Help Desk CoP Chief of Party CRS Catholic Relief Services CSB Corn-Soy Blend rCSI Reduced Coping Strategy Index DNH Do No Harm DRM Disaster Risk Management DRMO Disaster Risk Management Office EDRMC Ethiopia Disaster Risk Management Commission ETB Ethiopian Birr EWS Early Warning System FCS Food Consumption Score FDP Final Distribution Point FGD Focus Group Discussion FHE Food for the Hungry Ethiopia FSTF Food Security Task Force GoE Government of Ethiopia HCS Hararghe Catholic Secretariat HHS Household Hunger Scale HRP Humanitarian Response Plan ICCG Inter-Cluster Coordination Group IDP Internally Displaced Person ICT4D Information and communication technology for development IR Intermediate Result JEOP Joint Emergency Operation JEOP 1.0 Joint Emergency Operation 1.0 JEOP 2.0 Joint Emergency Operation 2.0 JEOP-NR Joint Emergency Operation – Northern Response KII Key Informant Interview MCS Meki Catholic Secretariat MoU Memorandum of Understanding NGO Non-Governmental Organization USAID REPORT JEOP FINAL PERFORMANCE EVALUTION USAID.GOV | 11 OECD Organization for Economic Co-operation and Development PDA Personal Digital Assistant PDM PDP Post Distribution Monitoring Primary Distribution Point PSNP Productive Safety Net Program PWD(s) REST Person with Disabilities Relief Society of Tigray SO Strategic Objective SNNPR TWG Southern Nations Nationalities and People’s Region Technical Working Group UN OCHA USAID/FFP United Nations Office for the Coordination of Humanitarian Affairs United States Agency for International Development/Food For Peace VBT Vulnerability Based Targeting WFP World Food Program WV World Vision WOWCA Woreda Women and Children Affairs YSP Yellow Split Peas USAID REPORT JEOP FINAL PERFORMANCE EVALUTION USAID.GOV | 12 EXECUTIVE SUMMARY BACKGROUND JEOP was awarded by the United States Agency for International Development (USAID) in August 2012 with an end date of July 29, 2023. CRS worked in Partnership with CARE, Food for the Hungry Ethiopia (FHE), ORDA, REST, World Vision International (WV), Hararghie Catholic Secretariat, (HCS), and Meki Catholic Secretariat, (MCS) to provide emergency food assistance to transitory food insecure households. The purpose of this evaluation was to assess the impact and efficiency of JEOP, and to facilitate learning from the challenges experienced during program delivery, which will further help to inform future food security interventions in Ethiopia and beyond. The evaluation aimed to assess six evaluation questions (relevance, effectiveness, efficiency, impact, coherence, and lessons learned) according to the Organization for Economic Co-operation and Development (OECD) criteria. JEOP has two Strategic Objectives: • SO1: Transitory food insecure populations have met their emergency food needs; • SO2: Targeted households protect their assets, and assess how the management, implementation and external factors affected results. EVALUATION QUESTIONS, DESIGN, METHODS AND LIMITATIONS A mix of qualitative and quantitative research methods was employed to obtain and triangulate valid data collected from 27th March–2nd May 2022 in Tigray and 26th March–30th April 2022 in all other regions. To obtain an appropriate quantitative sample, 127 JEOP-targeted woredas were grouped into 35 clusters based on geographic proximity, historical administrative structure, and livelihood systems. The number of woredas per cluster ranged between two and seven with an equal number of samples (360) in each cluster – resulting in a total of 12,600 participant households sampled. Due to the inaccessibility of some woredas in Oromia and Tigray, a total of 11,727 respondents were reached. In Tigray region, data had to be collected using paper-based surveys due to internet blockade induced communication problems leading to delays. The timing of the surveys had implications, especially in Amhara as it was conducted during ongoing distribution and belg (pre-harvest) season. Qualitative data collection consisted of 128 Focus Group Discussions (FGDs) and 185 Key Informant Interviews (KIIs). The evaluation followed numerous quality control steps including training of data collectors, translation of the quantitative survey tool into seven local languages, backchecking of results and triangulation using multiple data sources. MAJOR FINDINGS Across all regions, 31% households were female-headed, with Tigray having the highest proportion of female household heads (37%) and Dire Dawa having the lowest (19%). Most (86%) households had both male and female adult members. Amhara had the highest proportion of households with female adult no adult male (18%). Five households in Tigray and one in SNNPR had only children and no adults. The most common household size was four to six followed by those with one to three members. Very few households (less than 5%) had 10 or more members, with large household size more common in Oromia and SNNPR. Most household heads were aged 25 to 64 with an average of 47 years. While 12% of USAID REPORT JEOP FINAL PERFORMANCE EVALUTION USAID.GOV | 13 household heads reported living with disability, 7% reported that another household member was living with disability. SNNPR had the highest disability levels for both household heads and members. Across all regions, the main means of livelihood was agriculture, although the proportion of those relying on agriculture fell considerably in Tigray due to conflict, while reliance on relief assistance and remittance/family support grew. The main source of cash for households was the sale of agricultural products – mainly crops, followed by sale of livestock. Causes of vulnerability varied slightly across regions with conflicts and environmental factors, such as shortage of rain, flooding, and low soil fertility, being the most common. The evaluation found that JEOP was instrumental in supporting households faced with transitory food shortage. The program played a limited role in reducing stress sale of productive assets of targeted households. In terms of utilization of JEOP commodities, most respondents (93%) used all or some of their rations for household consumption. Of the 5% of respondents who reported selling rations, 86% indicated selling wheat and 16% indicated selling Yellow Split Peas (YSP). Cash-in-lieu of wheat was largely used for its intended purpose, whereas cash-for-transport was often used for purchasing other food to increase the overall amount and/or variety of food available for the household. All regions had moderate levels of food insecurity with pockets of severe food insecurity, particularly Tigray and SNNPR. About 4% of households in Tigray and SNNPR experienced severe food insecurity demonstrated by high levels of use of negative coping strategies, high levels of reported hunger and low food consumption scores. Food insecurity was more common in households headed by a person with a disability than those headed by a person without a disability. Results of the intermediate indicator “Transitory food insecure populations have met their emergency needs” were unsatisfactory, as only 17% of households were able to meet their emergency needs without resorting to negative coping mechanisms, indicating a plurality of complex needs among targeted households. Stress sales of productive assets such as livestock and seeds/land, which is taken as a negative coping strategy exercised during food shortage, increased. Other negative coping mechanisms included limiting portion sizes or reducing the number of meals per day. JEOP has a robust Early Warning System (EWS) which supports complementary efforts of the Government of Ethiopia (GoE) and ensures contributors to the system use it effectively; however, challenges that impede effective management of identified risks were timeliness of information provision, quality of information and response capacity. Provision of capacity building and resources may help increase efficacy of the EWS. Despite JEOP heavily promoting use of the GoE’s national targeting guidelines, complaints of exclusion of eligible households, inclusion of ineligible households or partial family targeting were common. Almost a third (29%) of respondents reported partial family targeting as a cause for insufficiency of food assistance at household level. Targeting related issues were also the most common form of complaint received by Community Help Desks (CHDs) – a grievance and complaints handling mechanism established in 2016. Participants’ awareness of JEOP accountability mechanisms was promising, 73% of respondents knew about the mechanism in place, of which, 30% had submitted feedback/complaint. USAID REPORT JEOP FINAL PERFORMANCE EVALUTION USAID.GOV | 14 Final Distribution Points (FDPs) were deemed to be safe, only 7% of respondents reported experiencing a safety/security issue during distribution – most commonly in Sidama and Tigray. The most common threat was theft/robbery of commodities/cash followed by crowding and toppling. JEOP has close ties with a GoE initiative that aims to address chronic food insecurity called Productive Safety Net Program (PSNP). The contextual shifting of caseloads between PSNP and JEOP was a best practice that helped JEOP avoid duplication of effort and therefore be efficient. Common feedback from participants was the need for development initiatives to be implemented alongside JEOP to help communities become more self-sufficient and reduce their need for assistance. Although JEOP is not a development initiative, linkages could be drawn more intentionally with initiatives that include agricultural development programs, water provision and irrigation (particularly in Amhara and Oromia), youth and women employment and entrepreneurship programs, financial literacy training, livestock breeding, infrastructure development and education. Additionally, many participants reported their desire for JEOP to re-introduce initiatives it ran that had elements of nutrition and access to finance. I. EVALUATION PURPOSE AND EVALUATION QUESTIONS The purpose of this final performance evaluation was to assess the impact and efficiency of JEOP and to facilitate learning from the challenges experienced to inform future food security interventions in Ethiopia. The evaluation responded to six evaluation questions to assess program achievements against Strategic Objectives (SOs) and Intermediate Results (IRs), and assessed how JEOP management, implementation and external factors affected results. Evaluation Questions are listed below: 1. Was JEOP relevant to its targeted project participants? 2. Was JEOP delivered effectively? 3. Was JEOP delivered efficiently? 4. What was the impact of JEOP? 5. How coherent was JEOP with other actors in adding value to existing initiatives? 6. What lessons were learned from the successes and challenges faced during implementation? This final performance evaluation reviewed achievements against the following SOs: - SO 1: Transitory food insecure populations have met their emergency food needs - SO 2: Targeted households protect their assets The IRs were: - Targeted communities and project stakeholders use functional participatory EWS - Targeted populations access a sufficient full basket of food in a timely manner Timely food transfers reduce stress sales of livestock in targeted household To help achieve JEOP IRs, a series of activities were delivered – primarily the provision of emergency food assistance. Alongside this, JEOP strengthened the EWS including training of staff, GoE partners and participants. These activities were intended to support program delivery by ensuring that participants achieve food security and receive timely information to trigger prompt action. USAID REPORT JEOP FINAL PERFORMANCE EVALUTION USAID.GOV | 15 I. BACKGROUND CRS entered into a cooperative agreement (AID-FFP-A-12-00009) with USAID/FFP in August 2012 to support GoE in providing emergency food assistance to transitory food insecure households in Ethiopia. Since then, JEOP has undergone multiple modifications and the award is scheduled to end in July 2023, with estimated annual caseloads ranging between 1.5 and 8 million people. CHANGE IN JEOP CASELOADS IN RESPONSE TO CHANGING CONTEXT JEOP was implemented by a CRS-led consortium across food insecure woredas in five regions and one city administration: Amhara, Oromia, Sidama, SNNPR, Tigray and Dire Dawa. The implementing partners in the consortium were CARE, FHE, ORDA, REST, WV, HCS, and MCS. The goal of JEOP was to save lives and protect livelihoods of Ethiopians. The food security contexts have varied over the course of the project, shaped by crises including the COVID-19 pandemic, desert locusts, political instability, and inter￾communal conflict and displacement. Amidst these challenges, JEOP provided support to up to one-third of Ethiopia’s food insecure population. JEOP provided support in fast-onset crises by expanding its caseloads in Amhara, Tigray, and East and West Hararghe areas of Oromia in 2019 and early 2020. Changes in security risks and the return of IDPs to their home communities resulted in further fluctuations in participant numbers. The scale of JEOP was adjusted on a semi-annual basis in consultation with USAID through a geographic and household targeting process led by the Ethiopia Disaster Risk Management Commission (EDRMC), as per the national Humanitarian Response Plan (HRP), which outlines emergency needs based on bi-annual assessments. The number of participants fluctuated between 1.5 and 8 million people. In 2021, following the identification of many eligible participants due to conflict-related challenges, and at the request of EDRMC to expand, the number of JEOP-assisted woredas increased from 90 to 127. Moreover, the instability in northern Ethiopia since late 2020 has led to an expansion of geographic scope and increasing of JEOP caseload. The number of woredas under JEOP implementation in Tigray alone was 55 following the start of the conflict, an increase from the 12 woredas served in 2020. JEOP evolved into two separate programs in mid-2022: JEOP-NR to cover Amhara and Tigray, and JEOP 2.0 to cover all other regions, with JEOP NR folding into JEOP 2.0 in January 2024. Given that the final evaluation and the two baseline assessments cover the same geographic areas, data for this final performance evaluation was collected with the two baselines for JEOP-NR and JEOP 2.0. The primary audience for this evaluation includes USAID, JEOP consortium members, GoE, and other governmental actors and Non-Governmental Organizations (NGOs) working on food security activities in Ethiopia. While the data is based on a representative sample, particular efforts were made to ensure that the perspectives of vulnerable groups were captured. The indicators used in the evaluation are shown in Table 1 below. Table 1: JEOP Indicators Indicator No. Indicator Title LOA targets Results NA % Of respondents whose food needs during food gap months were fulfilled by emergency food assistance 65% 17.2% USAID REPORT JEOP FINAL PERFORMANCE EVALUTION USAID.GOV | 16 NA % Of respondents who received amount and mix of food assistance as per entitlement 100% 54.6% NA % Of respondents who reported receiving JEOP food assistance as per communicated distribution schedule 75% 24.8% NA % Of respondents who sold productive assets to purchase staple food 17.5% 39.4% NA % Of respondents who sold livestock to purchase food 7% 37.8% EVALUATION METHODS AND LIMITATIONS This section describes the methodology used to obtain data for the final performance evaluation, including information on the sample design, data collection tools, response rates and limitations. SAMPLE DESIGN JEOP implementation area ranged between 80 and 127 woredas over the implementation period. To generate the sample frame, 127 target woredas were grouped into 35 clusters based on geographic proximity, historical administrative structure, and livelihood systems. The number of woredas per cluster ranged from two to seven (Annex 2: List of surveyed woredas). A total of 360 households for each group were set using a two-stage cluster probability-proportionate-to-size sampling strategy, where the cluster is the kebele within a group and the size is the participant household in a kebele (12 kebeles per cluster and 30 households per kebele). The survey used a comparative sample size calculation formula for indicators expressed as percentages and averages. This resulted in a total sample size of 12,600 JEOP participant households with 11,727 households achieved. The selection was self-weighted within a woreda group rather than within partner or JEOP aggregate levels. Hence, the indicator estimates at partner and JEOP aggregate levels were weighted using the weighting procedures suggested in the Feed the Future population-based survey sampling guide of 2018.1 Identification and selection of participants for the qualitative study was informed by stakeholder mapping (see Annex 11: Stakeholder mapping). A total of 185 KIIs and 128FGDs were conducted. The breakdown of planned and achieved KIIs and FGDs are detailed in Table 2 below. Table 2: Qualitative sample response rate Region KII FGD Planned Achieved Planned Achieved Dire Dawa 10 10 8 5 Oromia 35 30 32 32 SNNPR 18 17 16 16 Sidama 9 8 8 8 Amhara 30 30 20 20 Tigray 84 72 50 44 Federal 22 18 3 3 Total 208 185 137 128 1 Available at: https://www.fantaproject.org/monitoring-and-evaluation/sampling USAID REPORT JEOP FINAL PERFORMANCE EVALUTION USAID.GOV | 17 A survey questionnaire of 15 modules was developed jointly with CRS. The questionnaire used in Tigray was amended to better capture information on pre-post conflict without affecting data comparability.2 Questionnaires were translated into Tigrinya for Tigray, Afan Oromo for Oromia and Dire Dawa, Sidamu Afoo for Sidama, Hadiyissa, Kembatissa, and Gede’uffaa, for SNNPR, and Amharic for Amhara. In Tigray, a five-day training of enumerators took place, including one day of pre-testing whereas in other regions, a seven-day training and two-day pre-test took place. With the exception of Tigray, the survey was conducted using Personal Digital Assistant (PDA) devices. In Tigray, given the complex operating environment with limited infrastructure and unpredictable access at the time of data collection, use of electronic devices was impossible. Electronic devices were prohibited from being brought into the region, a PDA programmer was not able to travel to the region to troubleshoot IT issues, telecommunications were shut down and electricity supply was inconsistent, rendering electronic data capture extremely difficult. Therefore, paper surveys were used in Tigray. A total of 188 data collectors (75 female, 113 male) participated in the study, as presented in Table 3. Due to complexity of data collection in Tigray, the team was supported by eight CRS and REST staff who engaged as supervisors to oversee data collection, conduct data quality assurance, and manage field logistics. Table 3: Number of data collectors by region and gender Region Male Female Total Oromia and Dire Dawa 39 26 65 Tigray 38 25 63 Amhara 21 14 35 SNNPR and Sidama 15 10 25 Total 113 75 188 Qualitative data collection tools were developed to underpin KIIs and FGDs with stakeholders including GoE officials, participants, non-participants, and representatives from JEOP implementing partners. RESPONSE RATES The field work took place from 27th March–2nd May 2022 in Tigray and 26th March–30th April 2022 in all other regions. The response rate was high, as shown in Table 4 below. Table 4: Quantitative survey response rate Region Partner Target Actual % Achieved Amhara FHE 1,440 1,440 100 ORDA 1,080 1,080 100 Total 2,520 2,520 100 REST 2,340 2,159 92.3 CARE 330 1794 54.2 2 Examples include asking about source of livelihood before conflict and now (Q2.22 and Q2.23) and household characteristic including returnee (Q2.9). 4 The relatively low response rates in CARE and REST operational areas in Tigray are a result of omission of three woredas from CARE (Eastern and Central Zones) and four woredas from REST (Central, Southern and Eastern Zones) operational areas, all of which are remote woredas bordering with Eritrea, and which were inaccessible due to security reasons. USAID REPORT JEOP FINAL PERFORMANCE EVALUTION USAID.GOV | 18 Region Partner Target Actual % Achieved Tigray3 WV 510 509 99.8 FHE 420 420 100 Total 3,600 3,267 90.8 Dire Dawa HCS 360 360 100 Oromia WV5 900 360 40 CARE 1,440 1,440 100 HCS 1,080 1,080 100 MCS 900 900 100 Total 4,320 3,780 87.5 Sidama WV 360 360 100 SNNPR WV 1,440 1,440 100 Total 12,600 11,727 93.1 LIMITATIONS AND MITIGATING MEASURES Security affected data collection in all regions, leading to replacement of 10 kebeles in Amhara, 12 in Tigray, three in Oromia and four in SNNPR. These revisions were not deemed to have had an impact on validity of extrapolating from and making conclusions based on the results, as replaced kebeles were randomly selected and the data carries statistical significance. Moreover, 333 households in seven woredas of Tigray and 540 households in seven woredas of Oromia were omitted due to insecurity. Replacing woredas was not possible as all woredas under JEOP were sampled. The conflict in Northern Ethiopia hindered access in Tigray. Fuel shortages made data collection and supervision very costly and created delays. Without functioning telecommunications, teams were unable to communicate regularly and receive technical or analytical support which meant that paper-based surveys were necessary in Tigray. Paper-based surveys required increased time and effort as each survey had to be scanned and sent electronically, creating back-and-forth communication over corrections and errors, and delaying data entry. While paper-based surveys increased the likelihood of human error, risks were minimized by strict quality control measures – deploying data collection teams in two phases to allow for data quality and security checks halfway through the fieldwork period. The timing of the survey had implications on survey responses: in Amhara, data collection took place during ongoing distribution and Belg (pre-harvest) season. This resulted in Amhara showing relatively better scores in food security and dietary diversity measures indicating a better situation of the households during the data collection time. CHARACTERISTICS OF STUDY POPULATION The quantitative study covered 11,727 households across 391 kebeles. Seven JEOP partners were working across the six regions. Details on partner coverage are included in Annex 2: List of surveyed woredas. 3 Due to security restrictions, some HH were not reached in all partners operational areas in Tigray region except FHE 5 540 households were not reached in West Guji zone of WV operational areas due to security problems. USAID REPORT JEOP FINAL PERFORMANCE EVALUTION USAID.GOV | 19 Across all regions, 69% of households were male-headed and 31% were female-headed, with Tigray having the highest proportion of female household heads (37% compared to 29% in Amhara, 28% in Oromia, 32% in Sidama, 33% in SNNPR and 19% in Dire Dawa). When examining household composition by gender, most households had both male and female adult members (86%). Amhara had the highest proportion of households with female adult no adult male (18%). Only six households consisted of children and no adults, five in Tigray and one in SNNPR. Table 5 below provides more details. Table 5: Gendered household composition Region Partner Sex of household head Gendered Household Type No. of respondents Female Male Adult Female & Male Adult Female no adult Male Adult Male no adult Female Child, no Adult Amhara FHE 29.1 70.9 78.5 17.2 4.3 0 1,440 ORDA 28.5 71.5 75.4 18.3 6.3 0 1,080 Total 28.8 71.2 77.2 17.7 5.2 0 2,520 Tigray REST 38.1 61.9 88.9 8.6 2.4 0.1 2,159 CARE 38.5 61.5 88.3 10.6 1.1 0 179 WV 39.1 60.9 88.6 9 2.2 0.2 509 FHE 29.3 70.7 88.3 10 1.4 0.2 420 Total 37.1 62.9 88.7 8.9 2.2 0.2 3,267 Dire Dawa HCS 18.9 81.1 87.8 10 2.2 - 360 Oromia WV 46.7 53.3 87.2 12.5 0.3 - 360 CARE 24.2 75.8 86.5 12.2 1.3 - 1,440 HCS 24.8 75.2 91.7 8.2 0.1 - 1,080 MCS 28.7 71.3 91.1 7.7 1.2 - 900 Total 27.6 72.4 89.2 10 0.8 - 3,780 Sidama WV 32.2 67.8 93.6 5.8 0.6 - 360 SNNPR WV 32.7 67.3 87.1 11.6 1.3 0.1 1,440 Total 31.6 68.7 86.3 11.4 2.2 0.1 11,727 Average household size, age, and educational attainment The most common household size category was four to six members (46%) followed by one to three members (27%). Households with 10 or more members accounted for only 3% with the highest levels in Oromia and SNNPR (11% for both). The mean household size was 5.1. Figure 1: Average household size USAID REPORT JEOP FINAL PERFORMANCE EVALUTION USAID.GOV | 21 Table 7: Disability status of household heads Region Household heads Household members No. of respondents Person without disability Person with disability Person without disability Person with disability Amhara 87.2 12.8 92.4 7.3 2,520 Oromia 90.8 9.2 94.8 5.2 3,780 Tigray 88.4 11.6 93.4 6.0 3,267 SNNPR 75.3 24.6 82.7 17.3 1,440 Dire Dawa 93.3 6.7 96.1 3.9 360 Sidama 95.0 5.0 97.2 2.8 360 Total 88.1 11.9 93.0 6.5 11,727 Means of livelihood and sources of cash income Due to the prolonged conflict in Tigray, means of livelihood were analyzed differently by taking the conflict as a reference point. Multiple options could be selected by respondents therefore the responses add up to be over 100%. As shown in Annex 16b: Households’ main means of livelihood in Tigray, 69% of respondents stated agriculture was their main source of livelihood pre-conflict, which reduced to 63% post-conflict. Agriculture was followed by relief assistance, with 29% of respondents indicating this had been their main source of pre-conflict livelihood compared with 46% afterwards. Similarly, remittance/family support was the main means of livelihood for 7% of Tigray households pre-conflict, increasing to 16% post-conflict. Reliance on support from PSNP was non-existent during the time of data collection, which was 9% pre￾conflict. Absence of PSNP after the conflict increased the number of community members requiring assistance, increasing JEOP caseload. The trend showed a decrease in production and development work and increased reliance on support from family or relief assistance. As in Tigray, the main means of livelihood was agriculture in Dire Dawa (89%), Amhara (86%), Oromia (85%), Sidama (76%), and SNNPR (60%). The second most common means of livelihood was relief food assistance in Dire Dawa (74%), Oromia (67%) and Sidama (28%), and manual labor in SNNPR (50%). Table 8: Households’ means of livelihood across regions (except Tigray) Means of living Amhara Dire Dawa Oromia Sidama SNNPR Total Agriculture 86.3 89.4 85.4 75.8 59.8 83.3 Relief food assistance 32.5 74.4 66.5 27.8 49.9 44.9 Manual labor 8.4 21.1 18.2 22.5 50.2 15.8 Petty trade 6.0 5.6 6.7 15.0 14.7 7.4 PSNP 9.3 2.2 0.6 0.3 0.1 5.3 Skilled work 1.8 4.2 1.6 2.8 5.2 2.1 Remittance/family support 2.8 0.0 1.3 1.9 0.7 2.1 Regular employment 1.8 5.6 0.6 0.6 0.4 1.3 Other* 0.6 0.8 0.4 0.3 1.5 0.6 No. of respondents 2,520 360 3,780 360 1,440 8,460 USAID REPORT JEOP FINAL PERFORMANCE EVALUTION USAID.GOV | 22 Others* include: no response, pension, farmers veterinary scout, transport service (bajaj), support from others, sell of livestock, religious service, income from tourism, house rent, begging ** The total percentage can add up to more than 100% due to multiple response Agriculture was the main source of livelihood for the majority of households. Therefore, selling crops was also the main source of cash. This was reported by 60% of the households across all regions followed by sale of livestock and livestock products (16%) which was found to be higher in Dire Dawa (65%) than elsewhere. Table 9 below shows begging was only reported in Tigray (6%) indicating the severity of food shortage in the region during data collection due to the conflict. Table 9: Households’ source of cash income Source of cash Amhara Dire Dawa Oromia Sidama SNNPR Tigray Total Sale of crops 71.1 65.6 69.8 59.2 50.0 54.6 60.3 Sale of livestock and livestock products 18.9 64.7 17.6 19.7 4.4 15.3 16.4 Remittance/family support 5.1 0.3 2.7 3.1 6.5 14.4 10.2 Petty trading 6.9 5.6 11.2 20.8 15.0 9.9 9.7 Humanitarian assistance 21.0 46.7 8.4 6.7 8.3 4.5 9.2 Unskilled labor 9.5 13.9 21.4 25.8 47.9 1.8 8.3 Begging - - 0.1 - - 6.0 3.5 Pension payment 0.4 0.6 0.2 0.0 0.3 5.9 3.5 Skilled labor 2.1 2.5 1.2 3.1 10.1 2.6 2.6 Friends/neighbors - - - - - 4.1 2.4 Other* 3.5 5.6 1.1 3.6 3.5 0.9 1.7 No. of respondents** 2,520 360 3,780 360 1,440 3,267 11,727 Others* include: regular employment, transport service, traditional medicine, sale of honey, house rent, from own saving. ** The total percentage can add up to more than 100% due to multiple response II. FINDINGS 1. RELEVANCE This section describes the relevance of JEOP to meet participants’ needs. It also notes alignment with local and national GoE policies. To assess the relevance of the project, this section explores the rationale underpinning the overall goal of the intervention of saving lives and protection of assets. Its relevance in improving household food security, modality, and preference is also addressed. 1.1. FOOD SECURITY Food Consumption Score (FCS), a composite score based on dietary diversity, food frequency and the relative nutritional importance of different food groups, was used to assess food security. FCS was better in Dire Dawa and Amhara than in SNNPR, Sidama and Tigray, based on the proportion of households from each region in the “acceptable” category. The overall mean FCS value was 30.6, with considerable differences across regions ranging from 22.7 in SNNPR, 25.3 in Sidama and 27.7 in Tigray (regions with USAID REPORT JEOP FINAL PERFORMANCE EVALUTION USAID.GOV | 24 disability (66%). Both “moderate” and “severe” hunger were reported more by household heads with a disability (30% “moderate” and 5% “severe”) than those without a disability (26% “moderate” and 4% “severe”). Another dimension used to assess food security was reduced coping strategy index (rCSI). The rCSI explores household food security by assessing how far households use negative coping strategies when they do not have enough food or money to buy food. Table 11: Mean and median rCSI by region rCSI Amhara Dire Dawa Oromia Sidama SNNPR Tigray Total Mean 6.3 7.8 17.3 20.9 20.7 28.3 21.3 Median 0.0 5.0 15.0 20.0 19.0 33.0 20.0 Standard Deviation 9.4 7.6 14.0 11.2 10.8 14.9 16.3 The mean value of rCSI was 21.3 and the median was 20.0 (with a maximum score of 56), showing that households were moderately food insecure and exercising medium negative coping strategies. Dire Dawa and Amhara had relatively better rCSI values than other regions. In Dire Dawa, the mean rCSI value was 7.8 and the median was 5.0, while in Amhara these figures were 6.3 and 0.0, indicating that households were applying fewer negative coping mechanisms. As noted in the limitations section, in Amhara, results may be linked to the survey timing and most recent distribution. The rCSI values were 17.3 in Oromia, 20.7 in SNNPR, 20.9 in Sidama, and 28.3 in Tigray, indicating food security was moderately affected. When analyzing by household gender composition, the mean value for rCSI in Sidama was high for households with adult male no adult female (27.5), lower for adult female no adult male (21.6) and lower still for households with both male and female adults (20.8), indicating moderate use of negative coping mechanisms. In Tigray, the mean rCSI value was higher for households with adult male and female (28.5) and households with adult female no adult male (28.1), and slightly lower for households with adult male no adult female (21.1). See Annex 7 for more details. The above indicators illustrate the prevailing conditions across JEOP areas were of moderate food insecurity. The perception from participants regarding JEOP ability to meet participants’ food needs was mixed. However, the perception of respondents at the federal level was that JEOP adequately met food needs of its participants. “JEOP food assistance helped me and my family very much by meeting our food needs. As there was drought, there was severe food shortage in our kebele, but I did not suffer because I was getting food assistance. I am currently included in PSNP and not in JEOP. I get money monthly to fulfil food needs of my family.” - Non￾participant with disability, Sidama, WV area “JEOP did not fulfill our food needs. However, it supported us with our daily food needs to some extent. Although the quantity is too small, it will help in providing some meals.” - Participant with disability, Amhara, FHE area Adequacy USAID REPORT JEOP FINAL PERFORMANCE EVALUTION USAID.GOV | 25 JEOP is perceived to have been instrumental in providing life-saving support but did not fully meet the food needs of participants due to the customary quota system which resulted in partial family targeting. The quota system establishes caseload based on annual assessments at federal level, which is then allocated to regions, woredas and kebeles. This was external to the JEOP consortium, where they had limited involvement in participant selection. Additionally, over the 11 years of implementation there have been series of crises which have dramatically increased food insecurity due to persistent drought, reliance on traditional agricultural practices, locust invasions and conflict. The top three reasons mentioned for insufficiency of JEOP assistance were insufficiency of full allocated rations, partial family targeting and delay of assistance. Table 12: Reasons for insufficiency of JEOP food assistance Reasons for JEOP food assistance insufficiency Amhara Dire Dawa Oromia Sidama SNNPR Tigray Total Insufficiency of rations despite being the correct amount 75.6 87.9 34.4 27.4 45.6 63.1 39.8 Partial family targeting 25.6 59.5 65.5 44.0 73.1 46.6 29.4 Delay in food distribution 17.9 52.9 26.8 36.8 28.8 25.2 15.7 Quantity of food received was below entitlement 5.5 37.2 25.1 10.3 5.0 18.2 10.2 Portion of ration was sold/exchanged/given away 0.9 0.0 1.8 3.8 5.8 4.3 2.1 Others* 1.0 0.0 1.4 13.7 3.9 2.5 1.4 No. of respondents ** 2,101 331 3,232 234 1,162 2,701 9,761 Others* include: don’t know, there is no other source, large family size, sharing with others, improper use, no response **The total percentage can add up to more than 100% due to multiple response Sufficiency of assistance depended on the number of family members targeted. If some members of a household were targeted but the family did not have an alternative livelihood source to fill the food gap, food was shared across members. Therefore, vulnerable people with reduced ability to generate an income, such as the elderly, Person with Disability (PWDs) and female household heads had a reduced capacity to cope in the absence of JEOP. Some implementing partners at the federal (head office) level mentioned that JEOP did not fully address participants’ food needs because of the quota system. Participants noted that quotas underestimated actual food insecurity levels, resulting in GoE conducting partial family targeting to spread food across families. Partial family targeting resulted in insufficient food to address food insecurity problems. “The amount of assistance does not match the number of people that need it. The quota provided is not enough. This then leads to the Kebele FSTF (Food Security Task Force) using partial targeting which allows them to give some assistance to all that need it in the kebele.” - JEOP partner program manager Respondents from all regions (participants and GoE stakeholders) reflected similar perspectives that the quota system constrained JEOP ability to address food shortage of participants because it resulted in partial family targeting that rarely met households’ food needs. USAID REPORT JEOP FINAL PERFORMANCE EVALUTION USAID.GOV | 26 "JEOP is a means of survival for poor people. But it does not have any real positive impact on food shortage because the food is not enough to sustain the whole family." - Village leader, Tigray, REST area “JEOP activities were planned to meet the community’s food needs, but the amount is too small to sustain our needs for a month for the entire family. Sometimes, the distribution is delayed which exposes us to further food shortage.” – Female participant, Sidama, WV area This perspective is a recurrent finding with results echoed in the Post-Distribution Monitoring (PDM) and Annual Results Surveys (ARS) conducted throughout the project. However, the GoE-led quota system showed limited improvement. It will be critical under any follow on funding to focus on obtaining a clear consensus on why partial family targeting practice is conducted and put clear guidance about the necessity of full family targeting to increase clear communication between stakeholders. JEOP has delivered cash in lieu of wheat to participants in specific kebeles in Amhara, Oromia and Dire Dawa since 2018 with rates set according to localized price of wheat. Cash-for-transporting commodities home was introduced in 2019 and 2020 in Amhara. Although the decision to provide cash-in-lieu of food was informed by an area-specific assessment that identified better food availability in the market, 64% of respondents reported that the cash received was insufficient to meet food needs, mainly due to persistent inflation of grain prices. Cash-in-lieu of food was also reported to be insufficient in Oromia (74%). By contrast, 24% of cash-for-transport recipients confirmed that cash provided was sufficient to some extent, while 62% reported it was sufficient to a great extent. Table 13: Perception of respondents on the adequacy of cash received Region Type of cash assistance To a great extent To some extent Hardly Don’t know No. of respondents Amhara Food 0.5 48.4 36.1 15.0 155 Transport 61.9 23.1 8.5 6.5 162 Dire Dawa Food 5.6 66.7 27.8 0.0 18 Transport - - - - - Oromia Food 1.3 23.6 73.5 1.6 244 Transport - - - - - Total Food 1.3 30.5 63.9 4.3 417 Transport 61.9 23.1 8.5 6.5 162 These quantitative results are corroborated by qualitative findings from partners and participants, indicating the insufficiency of the cash-in-lieu of food. “The cash that is given to participants should match the current market price. If a solution is not provided for this problem soon, participants will fail to live with the money provided.” - Food distribution officer, Oromia, HCS area “The amount of money that they give us is not sufficient to buy the amount of food that we are expected to buy. As a result, we want the food assistance as the money that they give us is not sufficient.” Female participant, Kersa Oromia, HCS area USAID REPORT JEOP FINAL PERFORMANCE EVALUTION USAID.GOV | 27 An important reason why JEOP could not fully address the food needs of insecure households was growing food insecurity over the 11 years of implementation. For instance, participants from Lay Gayint FHE area of Amhara noted that erratic and low rainfall was increasingly delaying harvests – leading to more months of food shortages. Similarly, female participants from Wonago, WV area of SNNPR, noted that poorer households that were dependent on rain-fed agriculture were more vulnerable to climate change. Similarly, multiple respondents in MCS area of Oromia noted increasing food insecurity because of drought, locusts, and conflict. This feedback on increased food insecurity is corroborated by Annual Results Reports (ARRs), which described severe droughts in 2016 and 2017, desert locust invasions in 2021 and 2022, civil unrest in Oromia in 2018 and 2019, and conflict in Tigray, Amhara and Afar in 2021 and 2022. The multiple causes of food insecurity have led participants to rely more heavily on JEOP to address their food insecurity, which is not the aim of the program (provide assistance to transitory food insecure households). Table14 below details respondents’ perceptions on food insecurity reasons with poverty being the main cause (41%) followed by shocks (32%). As previously noted, the main means of livelihood for most participants was agriculture, which requires sufficient farmland and resources. However, many participants lacked resources including adequate farmland (30%) and as a result had low productivity (29%). Although the conflict in Tigray increased food insecurity, the major cause of food insecurity was the high level of poverty in the region, according to 71% of respondents. Poverty as a cause of food insecurity was reported only by participants in Tigray. Based on a Farm Africa study conducted in 2017, this was also the case prior to the conflict, with low agricultural productivity providing less than half of households’ annual basic food needs.7 The low agricultural productivity of Tigray before the conflict possibly stayed the same or worsened following the conflict. Table 14: Reasons for food insecurity Perceived reasons for food insecurity Amhara Dire Dawa Oromia Sidama SNNPR Tigray Total Poverty - - - - - 71.1 41.2 Unable to meet food needs due to a shock 79.1 4.4 61.4 94.4 46.3 4.1 32.0 No/little productive assets 18.9 93.9 52.2 10.6 33.2 28.8 29.8 No/little crop productivity 52.8 84.7 48.6 6.7 61.2 13.7 29.3 Displacement - - 0.3 3.9 3.8 40.8 23.9 Vulnerability factors (elderly, disability, etc.) 10.0 6.1 8.1 1.4 14.7 12.4 11.2 Conflict/War 10.6 0.0 0.0 0.3 0.6 4.4 5.0 Death of household head - - - - - 2.1 1.2 Frost/flood - - - - - 2.4 1.4 No. of respondents ** 2,520 360 3,780 360 1,440 3,267 11,727 *Others: Don’t know, no response, Low salary/Regular salary payment stopped, large family size, Boll worm. **The total percentage can add up to more than 100% due to multiple responses 7 Food Security in Tigray (farmafrica.org) USAID REPORT JEOP FINAL PERFORMANCE EVALUTION USAID.GOV | 28 1.2. MODALITIES Modalities for providing relief assistance have evolved over the award to better meet the needs of participant communities, in close consultation with GoE and in accordance with international standards for food assistance. While commodities remain the primary modality, cash in lieu of wheat as a complementary modality was introduced in 2019. Table 15 below shows whether JEOP food commodities met the preference of respondents with 25% of Tigray respondents indicating they did not. This was rarely the case in other regions; 100% of respondents in Dire Dawa, and more than 94% in other regions reported that JEOP commodities aligned with their preference. A possible reason for a higher proportion of Tigray respondents reporting they did not prefer the commodities provided may be the conflict leading to the inclusion of “non-traditional participants” (urban and peri-urban residents who are normally used to more variety). Table 15: Whether JEOP food items met the food preference of respondents Preference of food items provided Amhara Dire Dawa Oromia Sidama SNNPR Tigray Total Yes 97.8 100.0 99.7 95.6 93.6 74.9 84.6 No 2.2 - 0.3 4.4 6.4 25.1 15.4 No. of respondents 2,520 360 3,780 360 1,440 3,267 11,727 Among respondents who reported JEOP commodities did not align with their preference, YSP was the least preferred (63.2%) followed by wheat (58.5%) and oil (56.6 %). Figure 3: Food items preferred the least by respondents In 2019, JEOP introduced cash-for-transport in Amhara and in 2020, added flexible resource transfers in Amhara, Oromia and Dire Dawa. These allowed participants to receive two rounds of cash distributions (replacing wheat with cash). Cash Feasibility Assessments, which assess market and infrastructure availability, participant preferences and other factors, and Cash Standard Operating Procedures 0.0 20.0 40.0 60.0 80.0 100.0 120.0 140.0 160.0 180.0 200.0 Amhara Oromia Sidama SNNPR Tigray Total 24.7 35.0 53.3 58.1 56.6 48.8 61.3 55.1 59.2 58.5 31.5 24.4 100.0 85.5 63.9 63.2 Oil Wheat YSP (pulse) USAID REPORT JEOP FINAL PERFORMANCE EVALUTION USAID.GOV | 29 demonstrate the thorough processes undertaken to determine when and how to distribute cash in certain areas. Overall, only 1% of JEOP participants received cash, as indicated in Table 16 below. Table 16: Type of relief assistance received by respondents Type of cash assistance Amhara Dire Dawa Oromia Sidama SNNPR Tigray Total Food commodities only 98.7 95.0 95.5 100.0 100.0 100.0 99.1 Both cash and food commodities 1.3 5.0 4.5 - - - 0.9 No. of respondents 2520 360 3780 360 1440 3267 11727 Of the participants who received cash, 74% received cash-in-lieu for wheat, 15% received cash-for￾transport, and less than 1% received cash both for transportation and in lieu of food. No participant is eligible for both cash transfers at the same time. The amount of cash provided was reported to be insufficient to buy the equivalent of 15kg of wheat, due to price inflation, but the cash-for-transport was found to be sufficient. Table 17: Types of cash assistance Types Amhara Dire Dawa Oromia Total In-leu of wheat 49.4 100.0 100.0 83.3 For transportation to/from FDPs 46.9 - - 15.4 Both for transportation and in-lieu of wheat 1.8 - - 0.6 No response 1.8 - - 0.6 No. of respondents 317 18 244 579 Some respondents suggested revising the modality from cash back to commodities, such as woreda Food Security Task Force (FSTF) members and participants from the HCS area of Oromia, especially if the cash amount cannot be increased. However, as the amounts had been set following GoE guidelines, they could only be revised upon lengthy negotiations with GoE. Some respondents from non-cash areas requested cash in the future. In Dire Dawa, a female participant requested it to accommodate those who travelled long distances to FDPs and did not have access to means of transportation, and to provide people with the choice of purchasing their desired food. About 9% of respondents from Dire Dawa reported travelling more than two hours to reach FDPs. Of the households who received cash-in-lieu of wheat, 18% reported facing difficulties when purchasing wheat, particularly in Oromia (33%) as compared to Amhara (2.5%) and none in Dire Dawa. The most reported challenge was the price of commodities being unaffordable (98%), followed by lack of local markets (11%) – particularly in Amhara (37%).The follow-up question of types of market related challenges was not explored in Dire Dawa. Table 18: Types of market related challenges faced by respondents Types of challenges Amhara Dire Dawa Oromia Total Prices are too high/unaffordable 100.0 - 97.4 97.6 Market is too far 36.7 - 9.7 11.3 Insufficient supply of the food items 48.4 - 1.3 4.1 USAID REPORT JEOP FINAL PERFORMANCE EVALUTION USAID.GOV | 30 There are few suppliers 21.3 - 2.4 3.5 Quality is not good enough 0.0 - 2.6 2.4 Did not feel safe travelling to/from the market 6.1 - 0.0 0.4 No response 0.0 - 1.3 1.2 No. of respondents 13 - 87 100 Qualitative data showed that recipients of cash-in-lieu preferred either increased cash or wheat in-kind, as the cash amount did not match market prices. Some participants expressed interest in replacing some/all food with cash either to buy maize, (preferred it over wheat in some areas), or to help pay for transport. Cash-for-transport, provided to participants in Amhara that travel more than 2 hours to reach FDPs in areas with difficult terrain, was requested by some participants in Oromia and Tigray. However, participants from Amhara were seen to be using cash-for-transport to buy additional food items. This may be an indication of the severity of food shortage, linked to the northern conflict, climate change and soil infertility. Some participants indicated preference for cash because it gave people choice to purchase their desired food and was easier for those who had to pay for transport. Others mentioned that cash was preferred because it gave them opportunity to purchase commodities of choice. “When we compare the relative merits of direct cash assistance with food assistance, cash is very advantageous. This is because with cash, participants can buy things like exercise books for their children and other food items they want.” - Village leader, Dire Dawa HCS area 1.3. FOOD COMMODITY QUALITY CONTROL JEOP ensures that assistance is of a high standard by utilizing strict quality control measures8 aligned with USAID commodity handling minimum standards. Thorough analysis and clearance of commodities are carried out in the USA prior to dispatching to Ethiopia. The shelf life of the commodities is limited at two years for grains, quality assurance that the commodities are within date is carried out at Djibouti port using laboratory testing. Discharge and delivery surveyors then ensure the commodities are dispatched from Djibouti port to safely arrive at Primary Distribution Points (PDPs). Dispatchers and storekeepers inspect PDPs to ensure they meet minimum standards before commodities are stored. They also ensure commodities that arrive at warehouses are within date, are of standard quality and are free from infestation. Additionally, they implement an integrated pest management. "Firstly, we have a very strict follow-up protocol starting from dispatch from PDPs to FDPs. By ensuring timely dispatch, we make sure relief food is distributed on time. Our storekeepers verify the quality and quantity of items delivered on the spot to avoid irregularities." - Implementing partner commodity and logistics officer Commodity quality and quantity management systems are conducted by an independent supervisor who ensures participants receive the correct amount of their entitled rations. The quality of the commodity is checked by looking at the presence of damage and weight is checked to ensure the correct amount is being delivered. JEOP quality control measures were effective as indicated by participants in all regions in this evaluation. 8 PA00WZTK.pdf (usaid.gov) USAID REPORT JEOP FINAL PERFORMANCE EVALUTION USAID.GOV | 31 2. EFFECTIVENESS 2.1. FOOD SHORTAGE The extent to which JEOP addressed transitory food insecurity amongst participants is best assessed utilizing the indicator under Strategic Objective 1. The indicator looks at the percentage of participants whose food needs during food gap months were fulfilled by emergency food assistance. As per the Performance Monitoring Plan, this is calculated by dividing the number of participants whose JEOP rations fulfilled their family food needs for a full month by those who reported receiving the full quantity and mix of food (15kg cereals, 1.5kg YSP and 0.45 kg/half liter oil) during the last distribution round. Only 17% of respondents reported their emergency food needs during food gap months were met by JEOP assistance. Large differences were observed between regions, with the lowest in Dire Dawa (10%), Oromia (15%) and Amhara (15%), and the highest proportion found in Sidama (35%). Figure 4: Respondents whose food needs during food gap months were fulfilled by JEOP The typical length of time assistance lasted is depicted in Figure 5 below. Overall, 49% of respondents reported that assistance lasted two to four weeks, while 45% reported it lasted one to two weeks and 4% reported it lasted less than one week. The region with the highest level of respondents reporting JEOP assistance typically lasted between two to four weeks was Amhara (63%) followed by Tigray (51%). The region with the highest proportion of respondents reporting assistance lasted less than one week was SNNPR (14%) followed by Oromia (9%). Figure 5: Typical length of time JEOP food assistance lasted for those that lasted less than a month 0.0 5.0 10.0 15.0 20.0 25.0 30.0 35.0 40.0 Amhara Diredawa Oromia Sidama SNNPR Tigray Total 14.8 9.8 15.0 35.2 21.2 18.6 17.1 USAID REPORT JEOP FINAL PERFORMANCE EVALUTION USAID.GOV | 33 2.2. ASSET PROTECTION The second objective of JEOP was to protect participants from stress sale of productive assets (land, seed, farm tools/ production materials, livestock or other valuables) that can be utilized to meet basic needs. In this regard, the evaluation assessed the indicator - Percentage of JEOP assisted households that have sold productive assets to purchase food. Findings showed that 39% of households sold productive assets to purchase food. Analysis by region showed variation; 50% in Dire Dawa compared with 22% in Sidama. Figure 8: Sale of productive assets to buy staple food The type of assets sold by participants were also examined with results shown in Table 19 below. As seen, 80.7% of respondents reported selling livestock needed for farming, breeding, or transport ranging from the highest 98.9% in Dire Dawa to the west 59.3% in SNNPR. In SNNPR and Sidama a significant percentage sold seeds/planting materials and land as compared to the other regions. Table 19: Type of productive assets sold by households Assets Amhara Dire Dawa Oromia Sidama SNNPR Tigray Total Livestock 89.9 98.9 79.6 70.0 59.3 79.9 80.7 Seed/planting materials 10.8 0.0 8.4 22.5 40.6 2.9 5.8 Land (including leasing) 2.0 6.1 19.2 25.0 23.1 1.1 4.3 Farm tools/ production materials 1.5 0.0 5.8 15.0 11.3 2.9 3.4 Valuable assets e.g., jewelry 0.9 0.0 0.8 0.0 0.0 2.6 2.0 House 0.3 0.0 0.1 3.8 0.0 0.0 0.1 Don’t know 2.1 0.6 1.3 0.0 2.8 14.8 10.9 Total 717 181 1422 80 421 1613 4434 8.1 14.1 15.6 16.020.7 34.7 16.0 0.0 5.0 10.0 15.0 20.0 25.0 30.0 35.0 40.0 DIREDAWA OROMIA TIGRAY AMHARA SNNPR SIDAMA TOTAL 22.2 23.629.436.047.4 50.3 39.4 SIDAMA AMHARA SNNPR OROMIA TIGRAY DIREDAWA TOTAL USAID REPORT JEOP FINAL PERFORMANCE EVALUTION USAID.GOV | 35 Use of JEOP food rations Amhara Dire Dawa Oromia Sidama SNNPR Tigray Total Used all/some for own consumption 96.8 100.0 97.3 91.1 95.3 90.9 93.3 Sold part of it 2.6 - 1.4 6.4 2.0 7.5 5.3 Gave part of it 0.1 - 1.1 1.7 2.6 1.0 0.9 Sold all of it 0.6 - 0.1 0.8 - 0.6 0.5 Gave all of it - - 0.2 - - 0.1 0.1 No response - - - - 0.1 0.1 0.1 No. of respondents 2520 360 3780 360 1440 3267 11727 In the 2021 PDM, 11% of respondents from Amhara and 21% from Tigray reported selling their assistance commodities. In Tigray, the highest sale of commodities was among urban residents (43% Mekelle and 26% Shire). Additionally, the 2021 ARS showed 3.8% of respondents sold their assistance commodities with the highest being in Oromia and Amhara (5%). This final performance evaluation found 53% of respondents who sold all or part of their rations to buy other food items. Respondents also reported buying non-food items (46%) such as soap, cleaning products and clothing – as detailed in Table 21 below. Table 21: Reasons for sale of JEOP food assistance Reasons for sale of rations Amhara Oromia Sidama SNNPR Tigray Total To buy other household food items 60.7 46.4 53.8 35.0 52.4 53.0 To buy other non-food items 18.7 53.2 11.5 75.3 49.7 45.5 Pay for milling 0.0 - - - 12.3 10.0 Cover transportation to or from FDPs 6.9 15.6 3.8 10.1 7.3 7.5 Pay back loan 11.9 9.1 30.8 20.0 4.7 6.5 Pay for house rent 4.3 - - - 3.5 3.4 Pay for medical services or medicines 6.3 2.9 3.8 13.5 2.8 3.4 Pay government tax/contributions 5.0 3.9 7.7 0.0 1.7 2.3 Pay for education or school supplies 4.1 5.9 7.7 0.0 0.3 1.1 Difficulty in transporting rations to their home 1.7 6.4 3.8 0.0 0.6 1.0 Others * 3.8 - 11.5 0.0 0.4 1.0 No. of respondents** 64 54 26 23 242 409 * Others include: don’t know, to buy sheep and goat, the wheat is not good for health (gastric), due to the interest of other respondents who are assigned together to share the commodities – they preferred the money, the respondent always are in restaurants **percentage cannot be added up and might be greater than 100% due to multiple response Households that sold entitlements were asked which items they sold. Figure 11 below shows that wheat (86%) followed by YSP (16%) were most frequently sold. This trend was common across all regions except SNNPR, where 99% of those who sold reported selling oil due to the comparatively high market value. Figure 11: Food items sold by respondents USAID REPORT JEOP FINAL PERFORMANCE EVALUTION USAID.GOV | 37 Of those accessing cash-for-transport, 23.9% of respondents indicated they used it to purchase food. Only 30% of respondents used all and 27% used some of the cash for transport. The remaining respondents used cash for other purposes including paying loans (15%), paying for education materials or medical care (17%). 2.4. MEETING THE NEEDS OF THE VULNERABLE Prioritization of vulnerable groups JEOP has consistently prioritized the most vulnerable individuals (PWDs, the elderly, people with chronic illness, pregnant and lactating women, etc.), although the approach taken to meet their specific needs has evolved over time. Findings from successive ARS and a Disability and Inclusion Barrier Analysis (2019) indicated that challenges remain in prioritizing the most vulnerable, particularly related to identification, distribution and accountability. JEOP seeks to ensure the specific needs of the most vulnerable are met: the proposal indicates that vulnerable groups should be targeted, and FDPs should be located in accessible areas. Commodity Management Minimum Standards also reiterates that vulnerable populations should be prioritized. Respondents were asked to reflect on accessibility of FDPs in terms of physical barriers, distance from their home and existence of facilities such as shade/resting places, water, and sanitation. Figure 12 below shows 16% of vulnerable respondents believed there was improved access compared to previous years during/post distribution. High variability was seen across regions, with the lowest in Tigray (8%) and highest in Dire Dawa (47%) and Sidama (45%). The result from Tigray likely reflects the challenges of providing assistance to 4.5 million participants amidst the conflict in the region and the security threats that made some FDPs inaccessible. Figure 12: Respondents’ perception on the suitability of FDPs for vulnerable participants The reported level of household food needs being always met was different when analyzed by disability status of the household head – 11% of household heads with disability and 17% of those without disability said food needs were always met. However, those who reported their food needs were not met at all was higher among household heads with a disability (41%) than without (25%). This shows that households headed by person with disability were more likely to be food insecure. Table 23: Ability to fulfill family food needs without seeking external support 8.0 24.0 26.6 27.3 44.7 46.7 16.0 TIGRAY OROMIA AMHARA SNNPR SIDAMA DIREDAWA TOTAL USAID REPORT JEOP FINAL PERFORMANCE EVALUTION USAID.GOV | 38 Region Disability Status Yes, always Yes, sometimes Not at all No. of respondents Amhara Without disability 1.2 70.0 28.9 2,216 With disability 2.0 47.1 50.9 304 Total 1.3 67.0 31.7 2,520 Dire Dawa Without disability 1.5 92.6 6.0 336 With disability 0.0 75.0 25.0 24 Total 1.4 91.4 7.2 360 Oromia Without disability 6.8 50.5 42.5 3,428 With disability 4.8 38.5 56.7 351 Total 6.7 49.4 43.8 3,780 Sidama Without disability 3.8 28.7 67.5 342 With disability 5.6 11.1 83.3 18 Total 3.9 27.8 68.3 360 SNNPR Without disability 13.4 42.7 43.4 1,149 With disability 18.9 46.5 34.7 290 Total 14.7 43.6 41.3 1,440 Tigray Without disability 26.3 56.2 16.8 2,883 With disability 15.6 50.0 33.5 384 Total 25.1 55.5 18.7 3,267 Total Without disability 17.0 57.9 24.7 10,354 With disability 11.3 47.6 40.6 1,371 Total 16.3 56.7 26.6 11,727 Vulnerable individuals should be selected for assistance and prioritized first. However, most vulnerable9 (75%) and non-vulnerable (74%) respondents believed priority was not given to vulnerable people during distribution of assistance. Figure 13: Respondents’ perception on the prioritization of vulnerable participants 9Vulnerable people include: Lactating women, Pregnant women, Elderly (above 65 years), Disabled and Chronically ill. Other households were considered as non-vulnerable USAID REPORT JEOP FINAL PERFORMANCE EVALUTION USAID.GOV | 40 Table 24: Types of delegates used by vulnerable respondents Person collecting rations Amhara Dire Dawa Oromia Sidama SNNPR Tigray Total My self 67.1 51.9 68.9 81.9 74.5 69.8 69.4 Spouse 6.6 38.6 22.2 11.4 15.7 13.7 13.3 Delegate (family members/close relatives) 24.5 9.4 8.6 6.4 9.4 10.5 13.3 Delegate (non-family member) 1.7 0 0.3 0.3 0.4 4.8 3.3 Don't know 0 0 0 0 0 1.2 0.7 No. of respondents 2,520 360 3,780 360 1,440 3,267 11,727 Table 25 below shows that the most common reason respondents did not collect their own rations was old age (33%), followed by ill health (29%), lack of time (20%), disability (17%) and being away (14%). Old age was the most common reason in Amhara (52%) and the least common in Dire Dawa (14%). Table 25: Reasons why respondents did not collect their own rations Reasons for having delegates Amhara Dire Dawa Oromia Sidama SNNPR Tigray Total Old age 51.5 13.9 22.3 40 35.5 28.2 33.4 Ill health 36.8 13.9 23.9 32.3 45.4 25.2 28.5 Lack of time/busy 15.5 77.5 46.4 21.5 30.6 14.3 20.0 Disability 17.4 8.1 14.9 9.2 13.5 18.3 17.3 Distance to FDP 2.2 4 9.7 1.5 14.5 20.5 14.1 Pregnancy/lactating 4.6 0.6 1.9 12.3 2.6 4.8 4.4 Fear (feel insecure) 1.8 0 0.4 6.2 1.7 4.3 3.1 Underage 0 0 0 0 0 0.3 0.2 Deceased household head 0 0 0.1 0 1.3 0 0.1 To reduce temptation to sell rations 0 0 0.6 4.6 0.2 0 0.1 No response 0.3 1.2 2.2 0 1.8 5.4 3.5 No. of respondents 655 173 1228 65 358 902 3381 Respondents who used delegates to collect assistance were asked whether their delegates faced any challenges. However, the result shows that only 2% reported that their delegates had faced challenges. A CHD member from ORDA area in Amhara noted that PWDs, elderly people and women faced challenges or were unable to carry rations home without assistance. Suggestions raised to mitigate such challenges included opening FDPs closer to kebeles or hiring daily laborers to help take rations home. 2.5. TARGETING Due to the nature of the process, participant targeting is exposed to bias, partial family targeting as well as inclusion and exclusion errors. This is due to the subjectivity and use of intuition in the process of selecting participants compromising impartiality, especially excluding persons with disability and the most vulnerable. As a result, targeting is a key source of complaint across all JEOP operational areas. USAID REPORT JEOP FINAL PERFORMANCE EVALUTION USAID.GOV | 41 JEOP is addressing this as consortia alongside other food actors. A series of workshops, capacity building, and meetings were held to discuss underlying issues with stakeholders. JEOP introduced standard training guidelines in 2018 which helped to reach woreda and kebele targeting taskforces with intensive training on early warning, humanitarian principles and operationalize the national targeting guidelines. However, this alone was insufficient to prevent targeting problems. As a result, since 2021, JEOP piloted a new approach called Vulnerability Based Targeting (VBT) to improve targeting. The approach has proven to be effective in addressing targeting challenges. The VBT guideline was designed to support extended roll-out of implementation of the national targeting guideline. Scale-up of VBT has been underway and by November 2022 more than 30 woredas were reached. Full scale up to all JEOP woredas had not been completed yet at the time of the evaluation, it remains a priority for the follow-on award. The correct identification of appropriate participants allows JEOP to be efficient and effective. Due to lack of strict application of the targeting guidelines by GoE counterparts to date, there were multiple reports of the targeting having deviated from the guidelines, resulting in candidates qualifying for JEOP assistance not being targeted and those that do not qualify being targeted. However, as shown in Table 26 below, most respondents reported there were no/minimal errors (69%), while 16% reported a medium level error (16%). Only 12% of respondents reported high levels of error with the highest proportion of such reports coming from Amhara (13%), Tigray (12%) and Oromia (11%). Table 26: Perceptions of inclusion and exclusion errors Region No /minimum error Medium error High error Don’t know No. of respondents Amhara 75.0 7.8 13.4 3.8 2,520 Dire Dawa 76.9 10.8 10.8 1.4 360 Oromia 77.6 9.0 11.1 2.3 3,780 Sidama 92.5 3.9 2.8 0.9 360 SNNPR 66.0 12.0 9.0 12.9 1,440 Tigray 64.4 21.1 12 2.5 3,267 Total 69.2 15.8 12 3.0 11,727 Qualitative data indicated that targeting had complaints of errors where respondents reported of inclusion/exclusion errors and partial family targeting. Non-participant households, including PWDs, reported some kebeles had not involved the community in the targeting/ selection process and instead practiced nepotism, excluding seemingly rightful candidates. "The criteria for selecting people for JEOP assistance is that those who are poor and who don’t have any food to eat should be targeted, but participants are not always selected based on that; the selection process showed favoritism where participants were selected based on being friends, with use of nepotism and bribes.” - Village leader, Abaya Oromia, WV area USAID REPORT JEOP FINAL PERFORMANCE EVALUTION USAID.GOV | 43 A partner program manager in Amhara reported that the number of complaints received, and reports of ration-sharing dropped considerably when full family targeting was instated. Indeed, full family targeting was so successful in addressing the needs of the community that the implementing partner was awarded a prize with acknowledgment from donors and regional and zonal officials. Therefore, the practice of full family targeting needs to be enforced by all food operators with positive results shared across all implementing partners. 2.6. JEOP EARLY WARNING SYSTEM (EWS) JEOP supports GoE relief assistance strategy in multiple ways including by providing early warning information to the existing GoE EWS. This allows for more accurate estimation of semi-annual and annual relief needs, as released in the HRP. JEOP shares the EWS information with partners and helps to facilitate the use of the information by providing training. JEOP is also instrumental in the effective distribution and management of humanitarian operations by assisting in identification of target regions. Ideally, collated and analyzed early warning information should be sent back to communities from which it was gathered so it can be used to mitigate potential problems. However, an implementing partner program manager reported having experienced challenges in communicating this information back to communities, especially from woreda level Disaster Risk Management (DRM) actors. EWS at regional and local levels The process of early warning information collection is consistent across regions. When a potential disaster (natural/ manmade) in the community is identified by field agents, it is reported up the chain to the zonal DRMO for immediate action. The DRM early warning team then estimates the relief assistance required to mitigate the possible disaster.The zonal DRMO discusses the matter with the zonal administration and implementing partners to facilitate an intervention to mitigate the problem in the community. Kebeles keep an emergency risk profile based on long term and seasonal early warning indicators, such as climate, soil, farming outputs, market prices, disasters, and demographic data. Capacity building training to GoE stakeholders at various levels is provided on different topics including the EWS. The household survey attempted to assess the relevance and timeliness of early warning information collected and shared with support of JEOP. The result shows 54% of respondents believed that the early warning information provided was both timely and relevant – particularly in Sidama (77%). The region with the lowest report of timeliness and relevance of early warning information was SNNPR (25%). Figure 15: Households who reported the early warning information was both timely and relevant 25.430.337.5 56.7 57.2 76.7 53.6 SNNPR DIRE DAWA OROMIA AMHARA TIGRAY SIDAMA TOTAL USAID REPORT JEOP FINAL PERFORMANCE EVALUTION USAID.GOV | 45 administrative levels were observed by Oromia CARE Kebele FSTF, SNNPR Zonal DRM, and multiple Tigray Woreda and Kebele FSTFs. Moreover, the EWS information was described as incomplete or not adequately checked: for example, an Oromia CARE Kebele FSTF mentioned that the information passed on within the EWS is not cross-checked. Additionally, a Tigray Woreda FSTF mentioned that monitoring of early warning data and follow-up were irregular. Lastly, some respondents mentioned that response actions were often delayed or not taken; for example, an Oromia Zonal DRM mentioned that the Zonal DRM lacked capacity to provide timely intervention. In Tigray, one Kebele FSTF noted that community response was not possible as they lacked materials or equipment to implement FSTF recommendations. Kebele FSTF staff from WV area of SNNPR discussed challenges with the EWS. When meetings between woreda and Kebele FSTF staff were conducted, presence of Kebele FSTF was, at times, poor. Training was also lacking. Woreda FSTF staff from the WV area of Boricha, Sidama, noted the lack of a zonal office in their region, which meant they had to work with regional level officials – complicating the response. "Woreda FSTFs need training on assessment, monitoring and evaluation on early warning information based on meteorological forecasting regarding food security together with Kebele FSTF and JEOP staff. Currently, as we have no zone, we are interacting with regional level officials to react on early warning system.” – Woreda FSTF, Boricha Sidama, WV area An FGD with female participants from FHE area of Lay Gaint, Amhara, reported that although the EWS can be used to solve problems, the speed of information flows needs to be improved, dependent on those involved in the EWS. A DRM staff in Dire Dawa also suggested modernizing the EWS information collection system to improve the efficiency and effectiveness of responses. Modernization, such as using digital platforms, might also help to increase the data holding capacity of the EWS. Another zonal DRM staff from MCS area of West Arsi, listed several recommendations as in the following quote. “I believe these systems have been improving significantly in recent years. However, I think the capacity of the system needs to be strengthened, for example in digitization of data collection and archiving for emergency assessment, forecasting and M&E through staff training as well as infrastructure building.” - Zonal DRM, West Arsi Oromia, MCS area 2.7. JEOP ACCOUNTABILITY MECHANISM Accountability mechanisms were established in 2016 to allow participants to submit feedback and complaints, receive a response, and appeal if necessary. It also enables JEOP partners to learn from feedback for improvements. This evaluation assessed three key aspects of the accountability mechanism: familiarization of participants in using the system, extent of usage and mechanism response capacities. Participants can provide feedback or submit complaints by talking to CHD members, written feedback/complaints via suggestion boxes and by calling dedicated lines or directly speak to project staff or local government leaders. Figure 17 below shows the level of awareness about the Accountability Mechanism and actual usage. It shows 73% of respondents knew how to submit complaints, out of those, only 30% submitted a complaint. Figure 17: Respondents who know and have used JEOP accountability mechanism USAID REPORT JEOP FINAL PERFORMANCE EVALUTION USAID.GOV | 47 Region Channel CHD GoE staff Suggesti on box Telepho ne JEOP staff Commu nity No respons No. of respond ents Response 74.6 36.0 0.3 119 Sidama Submission 66.7 25.9 0.0 3.7 22.2 0.0 0.0 27 Response 62.5 62.5 0.0 16 SNNPR Submission 79.3 25.3 5.1 2.1 14.7 0.0 0.0 96 Response 81.7 32.1 0.0 35 Tigray Submission 55.3 51.2 0.0 0.1 3.4 3.2 1.0 964 Response 47.2 52.8 2.3 570 Total Submission 60.7 45.2 0.3 0.3 4.4 2.6 0.8 1796 Response 52.2 47.9 - - - - 1.9 900 Note*: Percent cannot added up as the question is multiple type Communication methods for feedback and monitoring include telephones, CHD, and suggestion boxes. The CHD feedback system was well-received by respondents. FGD with male participants from the FHE area of Sahala, Amhara reported having used the CHD to give complaints on targeting, mostly resolved by the CHD members. When the issue was beyond the capacity of the CHD members, it was reported to the kebele administration or Woreda FSTF. Response rate (55%) to feedback/complaints submitted by participants indicates the mechanism was not fully effective. Additionally, whether the mechanism has been especially tailored to handle sensitive issues such as gender-based violence and exploitation is unclear. A central independent feedback channel, such as a phone line free from the influence of local community members may be beneficial. "They [those who appeal] say, “You are complaint handling committee, but you are in the Office of Agriculture and Rural Development. Your office should have been separate. We come to appeal to you. But all the workers of kebele are there, the staff of Agriculture and Rural Development are there. The administration is there. Better if you are alone”.” CHD, Tigray, CARE area 2.8. PROGRAM AGILITY Covid-19 pandemic The COVID-19 pandemic presented an unprecedented challenge to health and food security systems globally. As such, the impact of COVID-19 on JEOP participants and how JEOP mitigated some of the negative impacts were assessed. As shown in Table 28 below, the commonest impact of COVID-19 was feeling scared/depressed (41%), followed by school closures (37%), reduced household income (29%), loss of labor job (18%) and reduction in food (13%). Table 2828: Impact of COVID-19 on the lives of participants COVID-19 impact Amhara Dire Dawa Oromia Sidama SNNPR Tigray Total Feeling scared/lonely/depressed 32.1 44.4 13.7 5.8 19.5 53.3 41.1 Children left school 23.6 59.2 38.4 3.6 47.1 42.4 37.1 USAID REPORT JEOP FINAL PERFORMANCE EVALUTION USAID.GOV | 48 Reduced household income 12.9 30.0 14.3 13.9 36.5 39.2 29.3 Nothing happened to me or my family 49.8 33.3 43.6 73.6 37.5 7.6 24.4 Loss of labor job 8.3 15.8 13.5 6.1 38.0 22.1 18.1 Less food to eat 10.1 31.4 13.0 13.3 27.3 12.8 12.8 Others* 0.8 1.1 0.7 0.0 0.5 2.3 1.6 No response 0.3 0.0 6.6 3.9 2.9 8.0 5.8 No. of respondents** 2,520 360 3,780 360 1,440 3,267 11,727 Others* include: Reduced pay from employer, No pay from employer, Unable to get medical treatment, Travel restriction and escalation of tariff, Social life limitation, Inflation **percentage cannot be added up and might be greater than 100% due to multiple response JEOP placed preventive mechanism to allow households to feel safe during distribution. Staff in SNNPR, Amhara, Oromia and Sidama described adaptations including double- distributions, longer distribution days, social distancing and grouping participants during distributions. “A JEOP partner had the only functional office during COVID-19. To curb food shortage due to drought, JEOP is always prepared to take action.” - Food distribution officer, Sidama, WV area “COVID-19 protection measures such as physical distancing and hand washing were strictly enforced at distribution points and on transport.” – Male participant, Oromia, HCS area Security Issues JEOP operated effectively in conflict areas including the Gedio-Guji and Tigray conflicts. JEOP coordinated well with OCHA and Zonal DRM to drive a revised joint targeting exercise during Gedeo-West Guji conflict in 2019. Additionally, JEOP increased implementing partners in Tigray to accommodate the massive increase in individuals requiring assistance. Only one partner (REST) was operating in Tigray before the conflict. Following the conflict, CARE, FHE and WV began operating in the area to help address the huge surge in the number of participants of 4.5 million. Strong and effective coordination between the partners helped to address this massive challenge which was critical in helping participants with food shortage thereby saving countless lives. 3. EFFICIENCY 3.1. EFFICIENCY OF STAFFING AND MANAGEMENT STRUCTURES There were multiple valuable lessons learned by JEOP which then led to adaptations of approach, such as providing cash-in-leu of food in areas with adequate food supply and providing cash-for-transport for those residing far from FDPs. A key structure that has made JEOP so efficient is the wide-reaching presence of partners. Regular meetings between JEOP partners on shared challenges and appropriate mitigation measures help to promote overall efficient programming. Meetings with food operators is one such regular meeting. Examples of lessons shared include individual scooping and success of full family targeting in specific areas. JEOP commodity handling minimum standards, followed by all partners, add to JEOP efficiency. One of the ways CRS ensures these standards are maintained is by conducting regular checks USAID REPORT JEOP FINAL PERFORMANCE EVALUTION USAID.GOV | 49 on partner engagement at FDP level. JEOP has efficient management structures which keep costs low. These include having only four international employees out of about four thousand staff and hiring private transporters to dispatch food, which is cheaper than maintaining a fleet of dedicated vehicles. However, when JEOP expands its coverage, a shortage of warehouses, transporters, and staff can occur. According to a JEOP program manager, working within a consortium makes it easier to introduce new technologies with input from wider and more varied implementers, which increases efficiency. A factor that affected efficiency at lighter levels was related to auxiliary staff such as laborers. One of the main causes for laborers’ inefficiency was the complaint of low wages, which are set by GoE, possibly leading to low levels of motivation at work. This was echoed by multiple respondents from all regions. “We are suffering doing this heavy labor work for very meager fees. So, I request consideration be made on the current situation and adjustment be made on the labor fee rate from current levels which won’t even buy soap, let alone feed our families” - Loader, Kombolcha Oromia, CARE area 3.2. TIMELINESS Delays in distribution can lead to food shortages. Adequate notice is required so that participants can travel to FDPs. Results showed that 23% of respondents reported they always received their rations as per communicated schedule, 50% rarely did. Timely deliveries was reported the most by respondents from Amhara (56%), followed by Sidama (54%) and SNNPR (44%). Table 29: Households who received JEOP food assistance as per the communicated schedule Region Always Most of the time Rarely Not at all Don't know No. of respondents Amhara 55.7 25.2 18.6 0.5 0.1 2,520 Sidama 54.2 19.4 14.7 1.7 10.0 360 SNNPR 44.4 30.9 20.6 1.1 3.0 1,440 Oromia 31.3 31.7 26.7 9.4 1.0 3,780 Dire Dawa 8.1 57.5 34.2 0.3 0.0 360 Tigray 6.1 11.1 70.2 10.4 2.2 3,266 Total 23.1 18.2 49.6 7.4 1.7 11,726 Many participants from Oromia, Amhara, SNNPR, and Dire Dawa reported that assistance was delivered irregularly, which at times led to food shortages. “Currently the food arrives late after people have faced hunger and have been exposed to informal loans” - Male participant, SNNPR, WV area Delays were also experienced during expansion due to access, connectivity and conflict-related challenges in coordinating actors involved. Other causes of delays included remote locations of FDPs, transporters failing to dispatch in a timely manner due to other priorities and security problems making transport of commodities unsafe. USAID REPORT JEOP FINAL PERFORMANCE EVALUTION USAID.GOV | 51 Notification sources Amhara Dire Dawa Oromia Sidama SNNPR Tigray Total Others* 1.0 0.0 0.6 1.4 6.0 1.4 1.4 No. of households 2,520 360 3,780 360 1,440 3,266 11,727 Others* include: Community radio, Don’t know, No response, Neighbors/Other community members Participants’ preferred communication channels for the announcement of distribution schedules are detailed in Table 31 below. Most (62%) respondents preferred kebele/village leaders followed by community contact person (52%) and community leaders (16%). Regionally, kebele/village leaders were preferred at 79% in Amhara, 52% in Dire Dawa, 67% in Oromia, 77% in Sidama, 89% in SNNPR and 52% in Tigray. Table 31: Channel preference for notification of food distribution schedule Region Amhara Dire Dawa Oromia Sidama SNNP Tigray Total Kebele/ village leaders 78.5 52.2 66.7 76.9 89.1 51.6 61.6 Community/village contact person 39.6 49.2 30.9 16.4 8.7 65.9 52.2 Community leaders 16.5 4.2 21 5.3 6.7 15 15.6 Food Security Task Force 3.7 5.3 8.6 22.5 10.9 6.7 6.7 Volunteer youth messengers 0.2 0 1.3 0.3 13.6 1.6 1.6 Community radio 0.1 0 0 0.8 3.5 2.2 1.4 CHD members 0.9 9.7 0.8 3.6 11.4 0.7 1.3 Public announcement 0 0 0 0 0.1 0.6 0.4 Neighbors 0.1 0 0.1 0 0 0.1 0.1 Phone 0 0 0.1 0 0.6 0 0 JEOP Staff 0 0 0 0 0.1 0 0 No. of respondents 2520 360 3780 360 1440 3266 11726 The process of informing participants of an upcoming distribution, which was seen as moderately effective, starts with kebele leaders being informed of distribution schedules by telephone. Kebele leaders pass the information in person to village leaders and CHDs. Village leaders make house to house visits or call a village gathering to disseminate the information. 3.3. WAIT-TIME AT FINAL DISTRIBUTION POINTS Wait time was evaluated to assess if it was in alignment with JEOP minimum standards of less than two hours. Table 32 shows only 26% of respondents across all regions waited for less than two hours. SNNPR (20%) and Tigray (19%) had the longest waiting time and Dire Dawa had the shortest, with 68% of respondents waiting less than two hours. Little difference was observed related to disability status. Household heads without disability reported they waited less than two hours at 27% compared to 25% for those with disabilities (see annex 21). The highest proportion of household heads with disability waiting less than two hours was seen in in Dire Dawa at 79% compared to 18% in SNNPR. Table 322: Average wait-time at FDPs Wait times Amhara Dire Dawa Oromia Sidama SNNPR Tigray Total 2 hours or longer 64.0 32.0 59.2 57.8 79.5 79.4 72.6 USAID REPORT JEOP FINAL PERFORMANCE EVALUTION USAID.GOV | 52 Less than 2 hours 34.5 68 40.6 42.2 19.9 19.3 26.3 Don’t know 1.5 0.0 0.2 0.0 0.6 1.3 1.1 No. of respondents 2,520 360 3,780 360 1,440 3,266 11,726 Table 33 below displays respondents’ perceived reasons for long wait-times. Many participants invited for distribution at the same time was the most widely indicated reason (81%), followed by late arrival of some participants10 (40%) and late arrival of distribution staff (16%). Table 333: Reasons for long wait-time at FDPs Reasons Amhara Dire Dawa Oromia Sidama SNNPR Tigray Total Many people called at once 87.6 96.5 89.3 91.3 95.3 76.1 80.9 Late arrival of some participants 31.7 7.8 23.7 11.1 16.1 47.0 39.5 Late arrival of distribution staff 15.9 7.0 11.3 1.4 15.9 17.2 16.0 Insufficient distribution staff 14.7 13.0 9.1 7.7 19.5 5.4 8.3 Late arrival of GoE officials overseeing/witnessing distributions 5.4 0.9 2.4 1.4 7.7 7.8 6.6 Long sharing time in the case of group distributions 0.1 0.0 0.7 0.5 0.0 2.9 2.0 Late arrival of food commodities 0.7 0.0 0.1 0.5 8.5 1.8 1.6 Bad weather conditions 0.2 0.0 0.4 0.5 0.0 1.5 1.1 Others 0.1 0.0 0.0 0.5 0.0 1.9 1.2 No response 0.2 0.0 1.5 0.0 0.1 2.9 2.0 No. of respondents** 1,326 115 2,183 208 1,085 2,622 7,539 **percentage cannot be added up and might be greater than 100% due to multiple response 3.4. PROXIMITY OF FINAL DISTRIBUTION POINTS JEOP Commodity Management Minimum Standards indicate that participants should not walk more than two hours or 10 kilometers to FDPs. As shown in Table 36 below, only a third of respondents traveled for more than two hours (34%). In Amhara, almost half of the respondents (48%) travelled for more than two hours, compared with only 5% in Sidama. Table 34: Time taken by respondents to travel to FDPs Region Within two hours More than two hours Don’t know No. of respondents Amhara 51 48.4 0.6 2,520 Dire Dawa 90.8 9.2 0 360 Oromia 81.1 18.3 0.6 3,780 10 JEOP not adopted first come first served approach. The first come usually wait until sufficient number (at least 80% of expected) gathered to start distribution. USAID REPORT JEOP FINAL PERFORMANCE EVALUTION USAID.GOV | 53 Region Within two hours More than two hours Don’t know No. of respondents Sidama 95.3 4.7 0 360 SNNPR 89.3 10.3 0.3 1,440 Tigray 65.5 33.6 0.9 3,267 Total 65.8 33.5 0.7 11,727 Table 34 below shows the methods of transport used to travel to and from FDPs. Almost all respondents walked to FDPs (90%) whereas only half walked home (56%). Common transport methods were animal drawn carts (25%), motor vehicles (10%) and motorcycles/bajaj (9%). Table 34: Methods of transportation used by respondents to travel to and from FDPs Type of transport Travel destination Amhara Dire Dawa Oromia Sidama SNNPR Tigray Total Walking Back Home 53.7 57.2 53.3 72.5 47.8 57.5 56 To FDP 84.6 83.9 78 82.5 82.2 96.2 90.3 Animal drawn cart Back Home 19.1 8.6 16.5 7.2 9.8 30.5 24.7 To FDP 0.6 0.3 3.7 0 0.4 0.5 0.9 Vehicle Back Home 10.4 28.6 13.1 0 1 9.5 9.8 To FDP 3.4 12.2 5.2 0 0.4 2.7 3.1 Motorcycle (bajaj) Back Home 16.7 5.6 17 20.3 41.3 2.1 9.2 To FDP 11.4 3.6 12.9 17.5 16.9 0.2 5.4 No response Back Home 0.1 0 0.1 0 0 0.3 0.2 To FDP 0 0 0.1 0 0 0.3 0.2 No. of respondents 2,520 360 3,780 360 1,440 3,267 11,727 The respondents that didn’t walk were asked whether they had to pay for transport, most (55%) stated they did not. Payment for transport was the most prevalent in Tigray (89%) and the least in Sidama (23%). Figure 20: Households who paid for transportation to and from FDPs 3.5. COORDINATION WITH KEY STAKEHOLDERS 23.334.2 35.8 40.4 57.1 89.4 45.1 SIDAMA AMHARA DIRE DAWA OROMIA SNNPR TIGRAY TOTAL USAID REPORT JEOP FINAL PERFORMANCE EVALUTION USAID.GOV | 54 The Chief of Party (CoP) chairs program coordination meeting for partner program managers. This forum collaborates with JEOP’s Technical Working Groups (TWGs), drives program activities by bringing together management and program issues, and addresses operational challenges and commodity management findings. It also acts as a forum to discuss regional issues and provides an overall picture for all partners on program activities. CRS JEOP technical leads (MEAL, EW, Compliance and Risk Monitoring, Cash, Operation, Finance) coordinate with each counterpart and respective external stakeholders. JEOP is a member of the EDRMC-led Food Prioritization Committee and submits a regular pipeline analysis to coordinate relief distribution. JEOP coordinates with World Food Programme (WFP), EDRMC to mitigate or avoid potential pipeline breaks and with United Nations Office for the Coordination of Humanitarian Affairs (UN OCHA) when required. This coordination helped JEOP to effectively reach communities in need of assistance in a timely manner. For instance, in 2014, JEOP covered additional hotspot priority 1 woredas outside of JEOP coverage area to fill in critical resource gaps, loaned Corn￾Soy Blend (CSB) to WFP to support refugees from South Sudan and transferred CSB+ to WFP’s targeted supplementary feeding programs. JEOP is also a member of the USAID Emergency Task Force, OCHA￾facilitated Inter-Cluster Coordination Group (ICCG), Inter-Cluster Information Working Group, Logistics Cluster, and Co-Lead for the Food Cluster. In 2018, JEOP joined the Cash Working Group Steering Committee. This enables JEOP to play an influential role in advocating hotspot classification, geographic prioritization, modality selection, cash transfer and stronger collaboration with WFP and other actors on market monitoring. Coordination with GOE is established through regional and woreda-level Memorandum of Understanding (MoUs) signed at the beginning of JEOP and renewed periodically. The MoU includes standards, modalities, emergency response operational calendar, conditionality issues, targeting criteria and clarity of respective roles of the GoE and JEOP in the distribution process. Coordination at woreda, regional and national levels occur regularly between JEOP staff and GoE counterparts, enabling communications, reflections, coordination with Resilience Food Security Activities and other actors, addressing emerging challenges. Coordination within JEOP CRS works closely with JEOP partners to address emerging humanitarian crises and develop guidance manuals that ensure quality standards are complied with during humanitarian responses. For example, WV has coordinated well during the Gedeo-Guji conflict to deliver assistance packages to communities that needed it. Monthly meetings were designed for exchanging challenges and updates whereas the TWG meetings were designed for deep dives into specific themes. Additionally, regional level training and experience sharing meetings within the consortium provide support in JEOP implementation. “Most interactions regarding JEOP with CRS and consortium members are at Head Office. At the regional level some of the interactions include experience sharing meetings and trainings within the consortium. These provide significant support in JEOP implementation. For example, World Vision shared the good experience on how it became the lead in transporting food in a timely manner." - JEOP partner program manager Each JEOP partner is assigned to specific geographic localities to avoid duplication of efforts and promote efficiency. Partners coordinate operations with one another – for example staggering collection of food assistance from PDP warehouses to prevent delays. Partners coordinate closely with transport companies USAID REPORT JEOP FINAL PERFORMANCE EVALUTION USAID.GOV | 55 to ensure assistance is distributed in a timely manner. Functional and effective partnerships between CRS and partners indicate the success of JEOP consortium, although there is always room for improvement. “Learning experience sharing can be stronger. We have a success story together. By the way full family targeting experience was presented at national level. Consortium experience sharing among partners should be strengthened. It might be strong at national level. We implementers should share our plans, learning experiences and challenges across the consortium." - Partner program manager, Amhara, ORDA 3.6. EFFICIENCY OF STAKEHOLDERS ENGAGED IN ASSISTANCE DELIVERY Efficiency of Transporters Transporters take commodities from centrally located PDPs to FDPs. Timely delivery is key to ensuring efficiency. One transport representative reported that when trucks faced difficulties, such as muddy roads in the rainy season, they discussed it with CRS, which sometimes led to FDP sites being changed to allow timely delivery of assistance. However, other representatives stated they had not encountered serious problems as they operated in relatively peaceful convenient areas. “If a vehicle falls over or breaks down, we are responsible for the repair or replacement of the vehicle within the time specified in the contract so that food assistance can be delivered to communities as per the set timetable.” - Umer transporters representative According to a CRS supply chain manager, the rising costs of transport and warehousing led to some transport companies and warehouse facilities being excluded. This limits the number of available transport trucks and warehouses, which can then cause delays in distribution, affecting efficiency. One of the main reasons transport costs are increasing is growing insecurity issues. "For instance, transport rates per metric tons for our current Mekelle operations have significantly increased. In light of security concerns and other factors, necessary adjustments are considered during the bidding process. Likewise, warehouse costs have been increasing continuously. " - CRS supply chain manager JEOP applies a food commodity quality assurance mechanism ensuring trucks are clean and equipped with required tarpaulin to protect commodities from rain or contaminants prior to loading at PDPs. JEOP has an agreement with transporters that they will pay for damages during transportation. Efficiency of Laborers Efficient delivery of assistance requires laborers to carry out duties effectively. Laborers from multiple woredas complained that their GoE set daily wages were inadequate. One laborer in Kombolcha, reported that they were paid three Ethiopian Birr (ETB) per 100kg, recently increased to six ETB per 100kg. However, this was inadequate. This perspective was echoed across all regions. "The payment we receive is very low. We have been paid six ETB per quintal for the last two years. It is not revised. We are paid 15 ETB per quintal when working for merchants. But we are paid six ETB per quintal here. " – Loader, Sahala Amhara, FHE area USAID REPORT JEOP FINAL PERFORMANCE EVALUTION USAID.GOV | 56 Loaders from the ORDA area of Wadla, FHE area of Sahala and FHE area of Lay Gayint, Amhara, reported the process of loading and unloading commodities from trucks to warehouses and vice versa was efficient, and loaders worked well with team members. A monitoring mechanism was also in place to ensure effectiveness. Cash incentives were provided by truck drivers to laborers to ensure trucks were unloaded in a timely manner. According to a laborer from the HCS area of Dire Dawa, loaders ensure food commodities remain undamaged during the offloading. "To prevent damage to commodities, we perform every one of our activities carefully. Hence, we take care not to tear sacks and not to be damaged by rain, etc. Of course, we have at times encountered sacks that had been torn and the food assistance had spilled over in the truck." - Laborer, Dire Dawa, HCS area However, there were reports of some laborers not performing their duties effectively, especially during the war. According to a loader from the REST area of Tigray, some laborers were careless when carrying out their duties resulting in sacks being dropped and tearing, which may have been related to the conflict impacting their salaries. Efficiency of staff involved in direct delivery of JEOP assistance The distribution process was perceived to be efficient. However, there were some issues raised by staff involved in direct delivery of commodities by monitors and distribution officers related to lack of shade and waiting areas, which may have led to delays in distribution during the rainy season. “Rainy days limit our ability to complete the distribution at the expected time, sometimes we have to extend the distribution onto the next day.” Food distribution officer, Sidama, WV area “Possible solutions to mitigate challenges include building conducive waiting areas for participants to have some protection”- Field Monitor, SNNPR, WV area 4. IMPACT 4.1. JEOP COMPLIANCE AND RISK MONITORING SYSTEM JEOP has a robust risk and compliance monitoring system with database, managed centrally. A unit assesses different aspects of JEOP in all target woredas monthly. The results are presented at TWG meetings, followed by discussions where challenges are addressed, and experiences and lessons shared. A partner project manager described aspects that get assessed, including targeting, commodity/warehouse management, and records and documentation. JEOP commodity and distribution practices have evolved, with introduction of commodity management training, shifting from paper-based to digital tools in 2015, restructuring its monitoring structure in 2016 and finalizing Commodity and Distribution Minimum Standards in 2017. The standards provide comprehensive guidance for appropriate warehousing, storage and stacking, vermin and insect control, warehouse materials and equipment, and documentation requirements. Additionally, the standards provide clear guidelines for distribution such that the process is explained to project participants; is as convenient as possible to the participants; is within reasonable distance of 10 km, prioritization of vulnerable groups, and nomination of a delegate; and is appropriately staffed, documented, and monitored. The standards also list the processes to ensure that distribution follows JEOP Accountability Mechanism minimum standards, providing participants with an opportunity USAID REPORT JEOP FINAL PERFORMANCE EVALUTION USAID.GOV | 57 to submit complaints, receive a response, and appeal if deemed necessary. Importantly, the accountability mechanism also enables JEOP partners to learn from feedback raised for adaptive improvements. JEOP risk model, a component of its compliance and risk monitoring system, enables frequent monitoring of higher-risk FDPs in efforts to reduce their risk level. The risk model classifies FDPs against fiscal and non-fiscal risk variables and scores FDPs on a scale of 0 (low) to 5 (high), with qualitative risk factors weighted based on importance, sensitivity, and caseload size. Each FDP receives a risk score quarterly, FDPs are selected for monitoring each month, based on their risk score (in the top 80% of riskiest FDPs) and are also selected if they have not been visited in the quarter or through a semi-random spot check. In 2017, JEOP finalized its information and communication technology for development (ICT4D) reporting dashboard, enabling all partners to view and analyze monitoring reports in real time, helping compliance officers to address and close issues as quickly and efficiently as possible. This risk model was used other than in 2021, during connectivity shutdown in Amhara and Tigray. The number of reports generated annually ranged from 1,500 to 2,728, and the number of FDPs visited monthly ranged from 93 to 110 (data from 2020 and 2021 is not available). JEOP utilized findings from the Commodity Risk Monitoring Unit reports to host Monitoring Unit TWG meetings to address commodity management issues and conduct training for partners on commodity compliance. Compliance monitoring has adapted over the life of JEOP regarding numbers and locations of officers. The number and location of monitors are adjusted annually to adapt to the changing context. For example, in 2019, JEOP deployed 28 monitors to account for the increased commodity distributions; in 2020, JEOP deployed 26 monitors and four compliance officers to account for the increased caseload; and in 2021, JEOP deployed 39 monitors to cover the caseload surge in Tigray and Amhara. Additionally, the standards were updated with DNH, conflict sensitivity, gender, and protection lenses in 2021. According to ARRs, this system has enabled CRS to support partners in reducing risk scores. Partner risk scores reduced significantly from 2017 to 2018, and no partners received high risk in 2018, 2019, 2020 and 2021. Risk findings remained relatively consistent from 2016 to 2021. In subsequent ARRs, risks identified in 2016 (targeting, standardized scooping materials, hiring scoopers, accountability mechanism) were not identified as major risks. JEOP compliance and risk monitoring system contributed to adherence with commodity and distribution minimum standards by providing a standardized way to rank FDPs as low to high risk and setting up systems to monitor FDPs and follow up on monitoring findings. The effectiveness of this system is evident in the reduction of partner risk scores and the annual resolution of findings. Strengths of the system lie in a personnel structure that allows for different teams to monitor and follow up, ability of monitoring results to be available in real-time, flexibility of the system to include more monitors and officers in different locations as required by contextual shifts, and monitoring of FDPs according to commodity management and distribution minimum standards. A suggestion for improvement would be forming a closer link between the risk monitoring system with the accountability mechanism and PDM, given that issues of distribution and participant end-use are raised in all three of these processes. This suggests collection of similar data, which may be either reduced for increased efficiency or triangulated to prioritize critical issues. 4.2. CONFLICT WITHIN THE COMMUNITY USAID REPORT JEOP FINAL PERFORMANCE EVALUTION USAID.GOV | 58 Conflict within Community and household members The extent of conflict between participants and non-participants related to the assistance was assessed. Results shown in Table 35 below shows that 69% of respondents reported there were no conflicts between participants and non-participants, while 21% reported that involvement with JEOP had created some level of conflict and 5% reported it had caused conflict. Table 35: Conflict between participant and non-participant households Responses Amhara Dire Dawa Oromia Sidama SNNPR Tigray Total No 64.7 98.3 87.1 87.8 87.1 64.8 69.1 Somehow 26.4 0.6 2.3 3.6 4.9 24 20.5 Certainly 7.9 0.3 0.7 7.2 3.2 5.7 5.4 Not sure 1.1 0.8 9.8 1.4 4.8 5.6 4.9 No. of respondents 2520 360 3780 360 1440 3267 11727 Respondents who reported they had experienced conflict due to involvement with JEOP reported on causes. Most conflicts were related to targeting (69%) and sharing of food items/cash (35%). Targeting issues arose when non-participants perceived participants had been targeted incorrectly. “Individuals who go to FDPs without ration card and are not targeted participants sometimes get into conflicts with participants and kebele administration. There were conflicts between participants and non-participants in this kebele.” - Male participant, Abaya Oromia, WV area Figure 21: Causes of conflict between participant and non-participant households Additionally, conflicts between members of the same household due to JEOP were also assessed, with the results shown in Table 36 below. Almost all respondents reported there were no conflicts (97%). Table 36: Internal conflict between members of same households Responses Amhara Dire Dawa Oromia Sidama SNNPR Tigray Total 0.0 10.0 20.0 30.0 40.0 50.0 60.0 70.0 80.0 90.0 100.0 Amhara Diredawa Oromia Sidama SNNPR Tigray Total Targeting-related 75.0 0.0 44.2 50.0 62.6 68.5 69.4 Sharing of food/cash items 44.4 99.5 65.1 37.5 59.9 27.5 35.4 Utilization of cash received 0.4 0.0 4.8 0.0 0.0 0.0 0.3 No response 0.1 0.0 0.0 12.5 2.4 13.0 7.9 Targeting-related Sharing of food/cash items Utilization of cash received No response USAID REPORT JEOP FINAL PERFORMANCE EVALUTION USAID.GOV | 60 4.4. LENGTH OF PARTICIPANT ENGAGEMENT WITH JEOP The duration and pattern of participant engagement with JEOP was assessed with results shown in Table 37 below. The proportion of households engaged with JEOP for over five years on a regular basis was only 3%, which was only from Tigray. Yet 10% households who were in JEOP over five years in an intermittent way. The highest proportion falls under one year on a regular basis at 30%. Table 3736: Duration and pattern of household engagement with JEOP Duration and pattern Amhara Diredawa Oromia Sidama SNNPR Tigray Total 1 year regularly 6.5 6.9 32.7 33.6 20.1 40.6 30.7 6 months or less 78.3 0.8 9.4 18.1 32.8 12.5 28.0 Intermittently for over 5 years 3.8 42.8 7.3 7.5 3.4 13.5 10.2 2 years regularly 3.3 12.2 17.2 21.9 13.5 10.5 10.1 Intermittently for 5 years or less 0.0 0.0 0.0 0.0 0.0 15.8 9.2 3-5 years regularly 2.9 34.2 28.3 15.3 10.4 0.0 5.2 5 years regularly 0.0 0.0 0.0 0.0 0.0 5.2 3.0 Intermittently for 5 years 5.1 3.1 4.5 3.6 18.8 0.0 2.5 Don’t know 0.0 0.0 0.6 0.0 0.9 1.8 1.2 No. of respondents 2520 360 3780 360 1440 3267 11727 Additionally, the ability of JEOP participants to meet their household basic needs without resorting to negative coping mechanisms was assessed. On average, only 12% of respondents met their basic needs which include essential goods, utilities, services and required resources to attain minimum living standards. As shown in Table 38 below, most households (66%) reported meeting some of their basic needs without resorting to negative coping mechanisms; 22% of respondents reported being unable to meet any of their basic needs. Table 3837: Basic needs that households met without resorting to negative coping mechanisms Region All Most Some None Don’t know No. of respondents Amhara 0.8 6.3 69.3 23.4 0.3 2,520 Dire Dawa 0.3 36.7 61.4 1.7 0.0 360 Oromia 2.1 15.1 64.7 17.8 0.1 3,780 Sidama 2.2 18.9 56.9 21.1 0.6 360 SNNPR 1.1 19.1 36.8 41.8 0.2 1,440 Tigray 1.5 9.7 66.4 22.1 0.0 3,267 Total 1.4 10.3 65.6 22.4 0.1 11,727 The results above show that households were unable to meet some of their basic needs. This was consistent with the findings of food security indicators including household hunger scale, food consumption score, and coping strategy (see sec 2.1). Sale of assets and livestock were also high (see sec 1.1). USAID REPORT JEOP FINAL PERFORMANCE EVALUTION USAID.GOV | 61 4.5. MAINSTREAMING OF PROTECTION AND SAFEGUARDING JEOP seeks to strengthen protection of vulnerable groups by inclusive participation during targeting and distribution; supporting local GoE to provide information on entitlements, targeting and selection criteria; establishing multiple FDPs per woreda along accessible routes to minimize travel times; prioritizing distributions to vulnerable households; and circulating community watch volunteers during distribution to ensure safety. In 2021, JEOP conducted a protection risk assessment that highlighted protection risks related to JEOP areas of operation and specific activities and then provided recommendations for mitigation. Critical risks included gender-based violence (GBV), particularly against women and girls; civilian safety and ethnic-based violence; child abuse and exploitation; and exclusion in access to services. The assessment found that women, children especially IDPs and unaccompanied and disabled children, PWDs, older adults, ethnic minorities and people living in rural areas faced higher protection risks than others. Tigray, Amhara, and Oromia were higher risk areas because of civil unrest, with Amhara’s risk exacerbated due to its terrain and long distances to FDPs. The assessment found that specific project components that may bring about protection risks included potential power imbalances within and between communities and households who have and have not been targeted. Such power imbalances can potentially be exacerbated in conflict￾affected areas with potential protection risks resulting from delayed or staggered distribution. Safeguarding and Protection in JEOP JEOP places safeguarding and protection as the priority during implementation at all stages and established CHDs to collect feedback and complaints from the community at each FDP. However, due to lack of confidentiality and fear of retaliation from negative complaints, CHDs receive only limited feedback, according to members. Hence, to safeguard community members, confidentiality of feedback system must be improved. " CHD members don’t take documentation home, its left at the warehouse. Anyone who wanted to see documents can access it. As we are farmers, we might not be available at the place, and this was the reason the complaint handling committee did not take the documents home." - CHD, Tigray, REST area According to a partner program manager, JEOP provides training on safeguarding and protection to staff and partners, including GoE officials. Among other benefits such as increasing awareness on safeguarding incident reporting, training allows stakeholders to protect vulnerable community members. Reports of significant safeguarding incidents are not common and minor incidents are dealt with locally. Lack of reporting may be due to a culture of silence as indicated during a FGD with CRS. Safety of Participants JEOP prioritizes the safety of partner staff and participants. Some examples of safety measures at warehouses employed by partners include display of informative posters on potential hazards; placement of fire extinguishers and first aid kits; hiring guards for the facility; training, orientation, and supervision of laborers by JEOP staff; and emergency first aid care provision when an incident does occur. Additionally, JEOP instated measures to protect participants including manning FDPs with guards. Table 39 below details the proportion of households who experienced safety/security threats. Overall, 7% of USAID REPORT JEOP FINAL PERFORMANCE EVALUTION USAID.GOV | 62 respondents reported experiencing safety/security issues; the highest report in Sidama (10%) and lowest level in SNNPR (3%). Table39: Respondents who experienced safety/security threats Region Safety issue experienced No of respondents Amhara 3.9 2,520 Dire Dawa 2.2 360 Oromia 4.8 3,780 Sidama 9.7 360 SNNPR 2.9 1,440 Tigray 8.2 3,266 Total 6.6 11,726 Respondents delineated whether the safety/security threat was experienced by themselves or family members, or an acquaintance. They delineated whether the issue was a lived experience or fear for safety. Half of respondents reported the security issue was fear of, not lived (52%). The remaining reported a security problem as a lived experience, only 16% reported it happened to them or their family and 39% reported it happened to someone they knew. Table 40: Security/safety threats experienced by respondents Region Lived, happened to me/my family Lived, happened to other people I know Fear, neither happened to me nor people I know Don’t know No. of respondents Amhara 31.9 63.6 4.7 0.0 80 Dire Dawa 87.5 12.5 0.0 0.0 8 Oromia 30.5 79.6 2.7 0.0 154 Sidama 28.6 77.1 0.0 0.0 35 SNNPR 57.8 41.9 4.7 8.8 62 Tigray 10.0 27.1 71.0 0.5 282 Total 16.4 38.6 52.3 0.5 621 The types of threats experienced are detailed in Table 41 below. The most common threat was theft/robbery of commodities (43%) followed by crowding and toppling at distribution points (23%) and family tension (21%). Theft was highest in Amhara (81%) and Sidama (80%), lowest in SNNPR (27%) and Dire Dawa (38%). Table 41: Type of safety/security incidents experienced by respondents Type of incidents Amhara Diredawa Oromia Sidama SNNPR Tigray Total Theft/robbery of food/money 81.4 37.5 65.6 80.0 26.7 31.3 42.6 Crowding/toppling 19.7 12.5 32.1 8.6 43.7 22.1 22.7 Verbal harassment 7.1 0.0 25.3 34.3 44.5 3.3 7.4 Physical harassment 0.0 0.0 7.8 17.1 3.0 6.3 5.8 Sexual threats 0.0 0.0 2.1 0.0 0.0 3.0 2.3 USAID REPORT JEOP FINAL PERFORMANCE EVALUTION USAID.GOV | 63 Fighting 5.1 0.0 14.8 25.7 8.6 5.8 7.1 Stigma from the community 4.7 12.5 5.5 2.9 16.2 0.6 1.9 Tension in the family 0.1 0.0 2.2 2.9 1.7 29.0 21.3 Others* 2.3 50.0 0.2 2.9 12.4 1.0 1.4 No response 7.7 12.5 6.0 0.0 7.1 13.2 11.3 No of respondents 80 8 154 35 62 282 621 Others* includes: Eviction from the house, Being forced to leave the community, Lost donkey while traveling, Car accident to/from FDP Locations where respondents faced security threats are detailed in Table 42. The most common location was FDPs (34%), followed by in transit (23%). Table 382: Location of security/safety issues faced by respondents Region Amhara Dire Dawa Oromia Sidama SNNPR Tigray Total At FDPs 75.3 - - 64.3 82.9 19.3 33.9 In transit to/from FDP 21.6 - - 21.4 39.9 29.2 22.6 In transit to the market 3.2 - - - - 14.8 8.7 In the village 4.6 - - 17.9 8.8 5.1 4.9 At home 2.9 - - 0 17.1 3.7 2.9 Local market 6.3 - - 7.1 - - 2 Don’t know - 100.0 100.0 - - 31.9 31.9 No of respondents 69 3 104 28 21 80 305 To protect commodities from theft, there were guards assigned at FDPs due to participants’ perception of the threat of theft. “There is no fear. This is because the way the staff provide food rations does not pose safety and security issues. They provide us the food on time and orient participants to take care of any theft risks. There are also some kebele security guards who oversee security matters.” - Male participant, Tanqua Milash Tigray, FHE area To minimize security risks, JEOP opened additional FDPs closer to villages. Closer FDPs allowed participants to travel to and from FDPs on the same day with less exposure to violence as they did not have to spend the night outside their own homes. Some participants reported travelling to and from FDPs in groups, which helped reduce security threats. “Participants ourselves take care of each other by travelling in groups when we go and come back from food distribution points. As there are different age groups, we travel with the elderly to keep their properties safe.” - Female participant, Wonago SNNPR, WV area JEOP implemented safety measures to help protect participants. These include starting distribution earlier in the day and some FDPs giving people rations on a first-come-first-served basis, though there was some room to wait until more participants arrived. Reports of safety issues have been minimal, although some might have been unreported. USAID REPORT JEOP FINAL PERFORMANCE EVALUTION USAID.GOV | 64 “There are no worries regarding our safety and security at the distribution point; no one threatens our rights; everyone receives assistance based on the rules and regulations of the project.” - Female participant with disability, Abaya Oromia, WV area 5. COHERENCE 5.1. OBSERVATION AT FEDERAL, REGIONAL, WOREDA AND KEBELE LEVELS JEOP aligns with GoE in amount and distribution cycle of food assistance. JEOP works closely with reciprocal support provision with GoE on transportation, targeting and caseloads. JEOP also ensures its responses to humanitarian emergencies always align with those of EDRMC and WFP. This is done by coordinating launch of rounds, maintaining clear communication, described by the head of the Food Cluster. However, according to the CRS CoP, there are times when GoE may prioritize forms of assistance not provided by JEOP, which directs resources away from JEOP and can be difficult to overcome. For example, there have been times when most transport trucks were mandated by GoE to transport fertilizers, leaving JEOP lacking transporters. Challenges such as these are communicated through the Food Cluster. The customary practice of quotas and the GoE running targeting undermined JEOP’s ability to reach transitory food insecure households. Respondents from kebele to federal level (including participants) stated that quotas limited JEOP’s ability to include all of those in need of support, resulting in partial family targeting. Additionally, targeting practices without independent monitoring can lead to strong perceptions of exclusion errors and bias. These policies were perceived to constrain JEOP ability to address the food needs of participants. Findings indicate that JEOP was perceived by respondents (participants, implementing partners and government officials) to support policy efforts at national level, woreda relief, and humanitarian priorities; including reaching the most vulnerable population. However, it is not intended to support longer-term and sustainable food insecurity priorities. Qualitative investigations further indicate that JEOP was perceived to support woreda relief and poverty￾alleviation priorities and women's empowerment objectives. However, the perception was that JEOP should support local priorities in addressing longer-term food insecurity, which is beyond JEOP’s focus. Respondents from Woreda Women and Children Affairs (WOWCA) in most regions agreed that JEOP supported woreda-level gender objectives because JEOP prioritized protection, safeguarding and distribution to women; the only dissenting opinion was from Oromia, which indicated that JEOP support for women was not focused on empowerment. Dissenting perspectives suggest a difference in expectations on what JEOP is intended to achieve. JEOP interlinkages with other interventions include operational synergies with PSNP, shifting of caseloads between PSNP and JEOP depending on the shifting context and complementary programmatic practices. The perception of synergies was present at management levels and less so at the community level. Additionally, JEOP food rations are consistent with international GOE and SPHERE standards. USAID REPORT JEOP FINAL PERFORMANCE EVALUTION USAID.GOV | 65 The most often cited example of synergies between JEOP and GoE-led interventions was that of PSNP. Diverse respondents, from kebele-level FSTF and JEOP participants to Zonal DRMs, from all regions, confirmed that JEOP and PSNP existed within the same woreda. Perception at GoE and NGO leadership levels, as indicated by a Woreda FSTF in Oromia and Tigray REST partner program manager, was that the interventions are programmatically complementary, PSNP focuses on chronic food insecurity and JEOP focuses on emergency support. Participants can receive support from one intervention or the other, depending on their cause of food insecurity. This finding is consistent with secondary data, which indicates that JEOP and PSNP caseloads can be altered depending on contextual shifts: for example, PSNP participants that experience sudden shocks can be incorporated into JEOP, while multi-year JEOP recipients could be shifted onto PSNP. For example, PSNP absorbed 3.5 million participants from the HRP (which determines JEOP caseload) in 2016, four million PSNP participants were added to the HRP in 2018, and 450,000 PSNP participants were added to the HRP in 2020. This interlinkage was further synergized in 2021 when JEOP aligned its distribution with the 45-day cycle of PSNP. Perceived synergies between the interventions were largely operational, JEOP and PSNP noted that the projects shared warehouses, staff, vehicles, and operational resources, while coordinating commodity management processes. Synergies allowed for substantial cost savings and supported alignment of approaches. Synergies between JEOP and other GoE-led programs were described in HCS area of Dire Dawa by a Woreda FSTF member, who noted that the combination of JEOP and the woreda's provision of seeds and fertilizers helped people grow out of assistance dependency. Quantitative results show JEOP participants had limited access to other forms of assistance. Figure 23 below shows only 12% of households received additional assistance to JEOP over the last 12 months, which was slightly higher in Tigray at 16%. Figure 23: Receipt of other assistance concurrently to JEOP over the last 12 months Table 43 below details the type of assistance received by households that reported receiving additional support to JEOP. Overall, 58% got it from other humanitarian organizations. Amhara had the highest level at 69% followed by Dire Dawa at 67% and Tigray at 56%. Sidama had the most households who received additional assistance from the same implementing partners as JEOP, at 57%, followed by SNNPR at 21% and Dire Dawa at 17%. Table 393: Source of additional assistance households received Sources of assistance Amhara Dire Dawa Oromia Sidama SNNPR Tigray Total Another humanitarian 69.0 66.7 49.9 21.4 15.5 55.8 58.2 1.9 1.93.33.9 11.8 15.7 12.2 SNNPR OROMIA DIREDAWA SIDAMA AMHARA TIGRAY TOTAL USAID REPORT JEOP FINAL PERFORMANCE EVALUTION USAID.GOV | 66 Sources of assistance Amhara Dire Dawa Oromia Sidama SNNPR Tigray Total organization Volunteers 0.8 0 0 0 0 5.6 9.6 Government 6 8.3 2.8 7.1 35.4 10.6 9.5 Cooperatives/unions 0.6 0 9.1 14.3 42.4 10.6 8.5 Safety Net (PSNP) 31.6 33.3 20.5 0 7 0 7.6 Religious institution 0.1 0 0 0 0 9.2 6.9 JEOP implementing partner that provides JEOP assistance 9.6 16.7 12.4 57.1 20.6 5.3 6.8 Financial service providers (banks and microfinance institutions) 0 0 0.4 0 0 2.9 2.1 Community-saving and lending groups 0 0 1.7 0 17.9 1.7 1.4 Don't know 0.5 0 0 0 6.4 14.7 No. of participants 316 12 93 14 24 445 904 Table 44 illustrates the type of additional assistance participants received during the last 12 months. Overall, 80% of the households accessed food or cash, with the highest proportions found in Amhara, Sidama and Dire Dawa. In SNNPR, 50% of those that received additional assistance were provided with food or cash, followed by financial service at 42%, and livestock provision (large or small animals) at 7%. Table 404: Type of additional assistance households received Response options Amhara Dire Dawa Oromia Sidama SNNPR Tigray Total Assistance (Food or Cash) 99.4 91.7 78.7 92.9 50.3 74.3 80.2 Agricultural inputs 0.1 - - 7.1 3.6 18.4 13.5 Healthcare or WASH or medical services 2.4 0.0 3.1 0.0 0.0 15.9 12.3 Shelter 1.4 0.0 0.0 7.1 0.0 7.8 6.0 Livestock provision 0.3 16.7 2.7 0.0 7.2 1.9 1.6 Financial service (Credit/loan services) 1.4 0.0 1.7 0.0 42.4 0.9 1.3 Household materials 1.0 0.0 1.5 0.0 7.0 1.1 1.1 Others* 1.1 0.0 2.1 0.0 0.0 0.0 0.3 No response 0.0 0.0 0.0 0.0 0.0 2.6 1.9 No. of respondents 316 12 93 14 24 420 879 Others* include clothes and don’t know. 5.2. ADDING VALUE WHILE AVOIDING DUPLICATION OF EFFORT USAID REPORT JEOP FINAL PERFORMANCE EVALUTION USAID.GOV | 67 Collaboration between JEOP with PSNP was positive and prevented duplication. Perspectives of JEOP ability to add value were largely in GoE-led efforts to ensure that participant lists for JEOP and PSNP were mutually exclusive. Additionally, JEOP aimed to programmatically avoid duplication of efforts by streamlining its access to finance component into CRS-led development activities. The main approach taken by JEOP partners to avoid duplication of effort is to ensure that PSNP and JEOP targets do not overlap. Respondents from all regions from kebele to zonal levels established that JEOP and PSNP participants are mutually exclusive, such that a single person cannot simultaneously benefit from both programs and that one of the targeting criteria for JEOP is non-participation in PSNP. In all regions, efforts to avoid duplication of targeting was led by GoE rather than by JEOP staff. In Oromia and SNNPR, verifying the exclusiveness of the JEOP and PSNP participant lists was managed by the woreda or kebele; in Amhara and Dire Dawa, this responsibility was led by the Zonal DRM. The exceptions noted were in cases of PSNP-led partial family targeting or an emergency affecting a PSNP household, which allowed PSNP participants to receive JEOP support, as mentioned by Amhara ORDA partner program manager. Similarly, SNNPR WV non-participant FGD observed that households that lost their homes or other assets could receive both PSNP and JEOP support after the conflict. Quantitative findings demonstrate that the additional assistance received by households had multiple effects. Table 45 below reveals that such support received over the last year increased access to food for 76% of multiple program recipients, increased access to cash for 20% and improved family health for 18%. Table 415: Effect of being involved in two or more programs Effect Amhara Dire Dawa Oromia Sidama SNNPR Tigray Total Accessed more food 72.2 83.3 68.8 85.7 44.3 77.1 75.7 Accessed more cash 61.1 66.7 45.2 0 55.6 6 19.5 Improved family health 4.9 16.7 10.8 0 21.3 22.6 18.3 Children get access to school 19.3 16.7 16.6 0 19.1 5.5 8.9 Improved health facility 10.8 8.3 3.5 0 32 7.7 8.4 Accessed more assets and livestock 0 0 3.5 7.1 17.7 9.7 7.4 Better prepared to withstand shocks 14.7 25 19.7 50 9.7 3.5 6.7 Improved knowledge and skill 0 0 1.4 0 7 0 0.1 Don't know 0.5 0 1.8 0 0 2.2 1.8 No. of respondents 316 12 93 14 24 445 904 JEOP has strong linkages with peer agencies. JEOP and WFP ensure complementary responses to emergencies by coordinating the launches rounds and ensuring coherent messaging. This allows for effective food response. They work together to tackle challenges by including the wider humanitarian donor community on, for example, how to reach remote areas. JEOP staff also work closely with GoE officials when facing challenges related to security and food distribution. GoE representatives have been helpful in tackling challenges as reported by a field monitoring officer from MCS area of Oromia. USAID REPORT JEOP FINAL PERFORMANCE EVALUTION USAID.GOV | 68 III. LESSONS LEARNED JEOP operated for over 10 years, with multiple lessons learned. The lessons learned include joint planning and coordination, regular meetings, and standardization of modalities across partners through the development of manuals and guidelines as well as experience sharing. These lessons allowed JEOP to grow in efficiency, of which working with implementing partners was key. The partnerships were efficiently maintained via regular meetings and common operating standards set by CRS. Additionally, joint planning and coordination allowed the partners to effectively address transitory food shortage in their implementation regions whilst avoiding duplication of efforts. However, an important lesson learned regarding design and implementation was regarding coordination with GoE counterparts and integration of JEOP with PSNP. Additional lessons that may be useful for design of subsequent programs were regarding the value-add of conditionality of assistance for certain participants and providing access to bank accounts for those receiving cash. Improvements in the targeting process were noted to result from closer working relationships between the woreda FSTF and kebele leaders, as mentioned by FSTF from Dire Dawa. Other methods that improved targeting include supervision of targeting by the Zonal DRM and inclusion of local communities to evaluate targeting, as reported by Oromia MCS Zonal DRM; provision of verbal and written warnings to Kebele FSTF to prioritize vulnerable groups as raised by Oromia MCS Food Monitor; and JEOP increasing logistic and financial support for targeting, assessment, verification, and monitoring activities as reported by a Field Monitor from WV area of Oromia. The need for improved coordination between JEOP and GoE staff was raised by kebele, woreda and zonal food security and disaster risk management counterparts, particularly in Oromia. Suggestions included strengthening linkages between DRMO and government and agriculture bodies and harmonizing distribution by GoE and JEOP. “The interaction between DRMO for relief interventions should be strengthened with support from JEOP. The support might be in terms of training local emergency management capabilities and response.”- Woreda FSTF, Oromia, HCS intervention area The most noted suggestion from all regions and stakeholder levels was that JEOP effectiveness would be improved by linkages with development programs to strengthen community resilience and reduce assistance dependency. When asked about preferred development activities, respondents listed activities including agricultural development, water provision and irrigation (particularly in Amhara and Oromia), youth and women employment and entrepreneurship, financial literacy for women, livestock breeding, infrastructure development, education and re-starting of JEOP interventions that had elements of nutrition and finance assistance. Linkages with other programs can allow implementation of development and emergency education, WASH, and health programs in the same areas as JEOP. All respondents expressed their wish that participants would engage in development initiatives, not be long-term assistance recipients and believed that addressing food insecurity required interventions beyond provision of food. USAID REPORT JEOP FINAL PERFORMANCE EVALUTION USAID.GOV | 69 “In order to address the poverty prevalent in the area and to make the community come out of need for assistance, we suggest government considers running developmental projects in the area. There is also a need to give more attention to income generating activities to engage the landless youth.” – Male participant, SNNPR, WV area Respondents from all regions and at all levels supported the notion of combining assistance with development activities, similar to Food for Work and PSNP models, which they believed would reduce assistance dependency and support overall community development. This was echoed multiple times by different stakeholders. “We wish to have developmental activities in the area such as factories”. - Non-participant, SNNPR, WV area “Emergency relief shouldn’t be the ultimate goal. JEOP should work not only to protect assets but also to enable participants to produce assets. Participants should do some work and contribute to society when taking the assistance from JEOP. In this case dependency will not become 100%.” – Woreda FSTF, Amhara, FHE area IV. CONCLUSIONS ACCOMPLISHMENTS JEOP supports the community to overcome transitory food shortages and reduce stress sale of assets and achieved this across all its operational areas despite targeting related and conflict driven challenges over the past decade. JEOP was able to reach people in need of food assistance by swiftly scaling up its caseload. JEOP being implemented by fully staffed consortium implementing partners who have presence across all implementation woredas has facilitated efficient program operations. There was some experience sharing between implementing partners which allowed the sharing of best practices and achieve an overall uniformity in the implementation of the program. JEOP interlinkages with other interventions such as PSNP, and the shifting of caseloads between PSNP and JEOP based on the changing context, is one of the best practices that helped JEOP avoid duplication of effort and therefore be efficient. Across all regions, the main means of livelihood was agriculture with the main source of cash also coming from the sale of agricultural products. This indicates the importance of agricultural productivity in ensuring food security in target areas. However, agricultural productivity has been decreasing due to multiple factors including conflicts and environmental problems such as drought and floods. The food security measures assessed in this evaluation also reflect the level of food insecurity in JEOP areas where participants had moderate levels of food insecurity with pockets of severe food insecurity, especially in Tigray and SNNPR. In keeping with the level of food insecurity, stress sale of assets by participants has also increased. Other negative coping mechanisms such as limiting portion sizes at mealtimes and reducing number of meals eaten in a day have also been observed. CHALLENGES USAID REPORT JEOP FINAL PERFORMANCE EVALUTION USAID.GOV | 70 The two greatest challenges perceived in activity design and implementation were related to targeting and distribution. Targeting and quota allocation were perceived to have resulted in partial family targeting. Additionally, bias and lack of transparency were perceived to have caused exclusion and inclusion errors. These factors were ultimately perceived to limit JEOP capacity to address food needs of participants. As the quota system is outside the control of JEOP, targeting challenges have persisted. In fact, JEOP has played its role in participant selection by providing capacity building training but has not been able to influence GoE decision regarding quotas. The challenge related to targeting might be reduced by implementing VBT in all operational woredas, an approach which has been implemented in limited woredas.This approach is believed to assure most vulnerable households in targeted communities are prioritized during participant selection. The household survey showed delays and distribution schedules not being adhered to were important challenges. Only 23% of respondents reported receiving their ration as per the communicated schedule. Similarly, FGD participants from Oromia, Amhara, SNNPR, and Dire Dawa reported that the food distribution cycle was not regular, which at times led to food shortages. In addition, only 26% of respondents reported receiving assistance within the set minimum standard of less than two hours of wait-time. Long wait-times to collect rations were raised as an issue in all regions and at all levels and perceived to be caused by external and internal factors. The factors included late arrival of GoE representatives to FDPs, late arrival of participants, challenges with securing laborers due to low GoE set wages, difficulties finding transporters willing to operate under difficult conditions such as conflicts, and remote locations of FDPs. Some of these challenges were partially addressed by increasing the number of FDPs. The accountability mechanism was set up to allow participants to submit their feedback and complaints, receive a response, and appeal if deemed necessary. It also enables JEOP partners to learn from feedback raised for adaptive improvements. However, only 73% of respondents knew how to submit complaints with only 30% of those ever having submitted a complaint. Additionally, of those that submitted a complaint, only about half received a response (55%) with 42% satisfaction rate. This indicates the accountability mechanism was still not functioning well in allowing participants to have their concerns addressed as only half had a response for their complaints; and more importantly, its value in improving implementation may not be fully utilized. Furthermore, the appropriateness of the available mechanism to handle sensitive issues such as gender-based violence and exploitation is not clearly defined. V. RECOMMENDATIONS Food Shortage and Sale of Productive Assets Though JEOP has been providing emergency food assistance for the last 10 years, stress sales of productive assets, such as livestock and seeds/land, still existed. This might likely be related to the compounding effect of prolonged conflict in the north, community unrest in other parts of the country, severe drought, and locust invasion in 2020, and the partial family targeting practices, all of which exacerbated food insecurity and drove households to sell assets to survive. There were two main contributing factors towards food shortage of JEOP participants – partial family targeting and delays in delivery of rations. Targeting and quota allocation were perceived to be biased and non-transparent. However, the main issues raised by participants was partial family targeting. It is therefore important for future programming to pay USAID REPORT JEOP FINAL PERFORMANCE EVALUTION USAID.GOV | 71 close attention to the issue of targeting as it affects overall efforts to address transitory food shortage – mainly from sharing of rations across an entire family. Stronger engagement of JEOP and GoE may influence improved targeting practices. However, increasing the quota that is provided to kebeles and full-scale implementation of VBT will likely be the key to curbing the practice of partial family targeting. Delays in distribution was another challenge. Reasons for delays included delayed GoE appeals or release of distribution times, prolonged transport from PDP to FDP, laborers’ unwillingness to work for GoE￾approved rates, difficulties securing agreements with transport companies, transport companies taking issue with rising fuel prices not matching payment for their services and remote location of FDPs. Possible solutions that can help address these issues include timely announcements of distribution schedules, increasing the wages of staff involved in direct delivery of JEOP assistance, increasing payment to transport companies in accordance with contemporary fuel prices and increasing the number of FDPs. There were notable limitations in terms of JEOP ability to align cash with the rising market prices. This is because the amount provided did not consider the rapidly rising market prices. It is recommended to expand cash-in-lieu after addressing initial challenges related to market availability and price monitoring, with care taken to ensure that cash amounts are based on contemporary market prices. The current amount falls short of buying the intended 15kg of wheat, which exacerbates food shortages. There were also concerns related to distance of FDPs from participant homes and availability of facilities at FDPs. Hence, increasing the number of FDPs so that they are more accessible to participants might improve distribution times, save participant transport costs, and mitigate potential security risks. Participant persistent food shortage and vulnerability may be minimized by the introduction of tailored JEOP exit strategies. At present, JEOP participants are removed from the program once deemed to come out of transitory food insecurity. However, unless they are linked with a development initiative, they may return to facing food shortages and therefore require JEOP assistance repeatedly. Therefore, linking participants with development initiatives, that address the specific needs of their locality, can allow chronically food insecure individuals to attain lasting food security without looking to JEOP for assistance. Early Warning JEOP EWS supports the early warning efforts of GoE. Contributors to the JEOP EWS generally use it effectively. However, challenges that impede effective management of risks are timeliness, reach and quality of information and response capacity. Thus, capacity building training to GoE staff and community awareness creation activity might help improve response capacity, quality and timeliness of the early warning information and make the EWS more effective. Disability Although JEOP prioritizes most vulnerable groups when it comes to targeting, challenges remain in terms of including more PWDs, as there were multiple allegations of biased targeting practices. Additionally, PWDs remain challenged regarding accessing FDPs and the accountability mechanism. Including the different vulnerability characteristics of participants on the list held at FDPs can help staff better identify and serve vulnerable participants in accordance with their needs. Additionally, providing training on cross￾cutting issues including disability may facilitate the prioritization of vulnerable groups, including PWDs during targeting and food distribution. Accountability Mechanism USAID REPORT JEOP FINAL PERFORMANCE EVALUTION USAID.GOV | 72 Though some participants have awareness of how to file complaints, the number of complaints submitted, and response rates were unsatisfactory. Additionally, the suitability of the mechanism for handling sensitive issues such as GBV and exploitation is not clearly defined. Therefore, accountability mechanisms and contextual feedback systems need to be improved or better designed by allowing private complaints to be submitted confidentially. Furthermore, raising the awareness of participants on the mechanism may help increase the submission of feedback/complaints which in turn can facilitate implementing partners tailor implementation in a need-responsive manner. Preference of JEOP Food Commodities Some of the food commodities provided by JEOP were reportedly less preferred by respondents. The most unpreferred food items were wheat and YSP. Hence, JEOP may consider awareness creation initiatives to mitigate misconceptions to increase utilization and decrease sale of rations by participants. Motivation With respect to staff motivation, findings show that frontline workers such as scoopers and loaders were dissatisfied with the incentives and wages. Hence, future similar programming might consider advocating for improved wages with payment scale based on recent inflation with consultation with GoE. USAID REPORT JEOP FINAL PERFORMANCE EVALUTION USAID.GOV | 73 ANNEX 1: EVALUATION TEAM INFORMATION • (MSC), Project Manager and Reviewer • Aweke Shishigu (PhD), Evaluator • Esther Lee (MSC), Quality Assurance and Reviewer • Lemlem Tewolde-Berhan (MBBS, MPH), Evaluator • Hanna Hailu (BA), Data Analyst