Page | 1 RISING II Recovery in Sudan for Improved Nutrition and Growth Final Evaluation Report April 2022 Page | 2 ACKNOWLEDGEMENTS We would like to give special thanks to Cecilia Adalla (Chief of Party, RISING II project, Sudan), Md. Abu Hanif (MEAL Manager, RISING II/ Taadoud II project), Alison Long (CRS Regional MEAL Advisor), Thekkiniath Jose (CRS Regional MEAL Advisor, Kenya) and Hani El Mahdi (Country Representative, CRS, Sudan) for their technical and administrative support and guidance. We would also like to acknowledge Reham Bashir (Senior MEAL Officer), Ahmed Adduma (Senior MEAL Officer) and Mohaned Ibrahim (Senior MIS Officer), CRS Sudan for their valuable contribution on the tools translation in Arabic, co-facilitation of the quantitative and qualitative survey online training. The success of the RISING II final evaluation has been due to the collaborative efforts of the Government of Sudan and Bureau for Humanitarian Assistance (BHA), formerly the Office of Food for Peace (FFP) and the Office of U.S. Foreign Disaster Assistance (OFDA) of United States Agency for International Development (USAID). The study would not have been possible without the support from the Government authorities including the Ministry of Agriculture (MoA), Ministry of Health (MoH) and the Humanitarian Aid Commission (HAC) for necessary approvals and survey tools review. The federal HAC serving as an ethical board was a particularly important counterpart and vital component to the approval of survey tools as well as for the overall study. We would like to express our sincere gratitude to the Government of Sudan. We would also like to thank USAID/BHA for supporting this study. Special thanks were due to the quantitative and qualitative survey teams of interviewers and national consultants who conducted the fieldwork during the COVID-19 pandemic crisis. The survey enumerators exhibited great dedication during every day of the survey to reach the required sample and ensure data quality; without their hard work this survey would not have been possible. Most importantly, heartfelt gratitude goes to the project participating households and communities, who welcomed survey teams into their homes and lives, took time from their day to sit, speak, and share their knowledge with the interviewers and provided invaluable information for the RISING II Partnership. It is our sincere hope that the RISING II project will benefit from this study for future programming. Mark W. Langworthy, Ph.D., Vice President/Economist Suzanne Nelson, Ph.D., Senior Research Scientist AKM Towfique Aziz, Senior Advisor for M&E and Research TANGO International Inc. April 2022 Disclaimer: The opinions expressed are those of the baseline study team and do not necessarily reflect those of the RISING II/ USAID/BHA. Responsibility for the opinions expressed in this report rests solely with the authors. Publication of this document does not imply endorsement by RISING II/USAID of the opinions expressed. Page | 3 Table of Contents ACKNOWLEDGEMENTS ...................................................................................................................................... 2 LIST OF TABLES................................................................................................................................................... 4 LIST OF FIGURES................................................................................................................................................. 4 LIST OF ACRONYMS............................................................................................................................................ 6 EXECUTIVE SUMMARY ....................................................................................................................................... 7 A. INTRODUCTION............................................................................................................................................ 13 B. RISING II FINAL EVALUATION....................................................................................................................... 14 1. Purpose and objectives............................................................................................................................ 14 2. Methodology ........................................................................................................................................... 14 3. Household Demographics........................................................................................................................ 18 C. RISING II FINAL EVALUATION FINDINGS ...................................................................................................... 19 4. Project Impact.......................................................................................................................................... 19 1.1. What have been the impacts of RISING II activities on project households?..............................19 1.2. Have different types of households experienced different types of impact? .............................23 1.3. Did the project have any unintended consequences – either positive or negative? ..................23 5. Livelihood Outcomes............................................................................................................................... 23 1.4. How have household livelihoods changed as a result of the project? ........................................23 1.5. How has access to markets, particularly for women, changed as a result of the project? .........30 1.6. How has access to loans and savings changed as a result of the project? ..................................31 6. Access to Natural Resources Outcomes.................................................................................................. 31 1.7. How has household/community access to natural resources changed as a result of the project? 32 1.8. How has management of community natural resources (including conflict mitigation) changed as a result of the project? ........................................................................................................................33 7. Health and Nutrition Outcomes .............................................................................................................. 35 1.9. How has community access to safe water and sanitation facilities changed as a result of the project? ....................................................................................................................................................35 1.10. How have household hand-washing behaviors changed as a result of the project? ..............35 1.11. How has the nutritional status of children under five (CU5) changed as a result of the project? 37 1.12. How have attitudes about or practices in exclusive breastfeeding of babies for the first six months changed as a result of the project? ............................................................................................38 8. Project Management Outcomes: OECD-DAC .......................................................................................... 39 1.13. Relevance .................................................................................................................................40 1.14. Effectiveness............................................................................................................................40 1.15. Efficiency ..................................................................................................................................43 1.16. Coherence ................................................................................................................................44 1.17. Sustainability............................................................................................................................45 D. CONCLUSIONS AND RECOMMENDATIONS ................................................................................................. 45 E. ANNEXES ...................................................................................................................................................... 47 ANNEX I Sample Size Estimation.................................................................................................................. 47 ANNEX II Data Analysis and Tabulation Plan................................................................................................ 50 ANNEX III Detailed results for RISING II Logframe Indicators...................................................................... 59 Page | 4 ANNEX IV Qualitative Research Questions .................................................................................................. 60 ANNEX V Proposed Field Schedule for Qualitative Data Collection............................................................. 61 ANNEX VI List of Qualitative Interviews....................................................................................................... 63 ANNEX VII Analysis Tables for Demographics and Background Characteristics.......................................... 67 ANNEX VIII Additional Results...................................................................................................................... 69 LIST OF TABLES Table 1: RISING II Project Logframe Indicators 8 Table 2: Sample size by five critical outcome indicators 48 Table 3 Survey Sample and Respondent's Profile 68 Table 4 Household Demographics 69 Table 5: RISING II Sample and Non-response, by Survey Rounds 70 Table 6: Reasons for non-response, by Survey Rounds 70 Table 7. Nutritional Weights 70 Table 8. Food-based Coping Strategies and Severity Weights 71 LIST OF FIGURES Figure 1: Sample Population Distribution by Age and Sex, by Survey Rounds 18 Figure 2: Households with Adequate Food Consumption (FCS>42), by Survey Rounds 20 Figure 3: Households with Moderate to Severe Hunger (HHS), by Survey Rounds 21 Figure 4: Average Reduced Coping Strategy Index (rCSI), by Survey Rounds 21 Figure 5: Households Reporting Net Income/Profit, by Survey Rounds 24 Figure 6: Number of Hectares of Land Under Improved Agricultural Methods, by Survey Rounds 26 Figure 7: Households with Access to Sufficient Seeds for Planting, by Survey Rounds 28 Figure 8: Number of Months of Household Food Self-Sufficiency, by Survey Rounds 29 Figure 9: Household Year-round Access to at Least Three Natural Resources, by Survey Rounds 32 Figure 10: Households Using at Least Two Improved NRM Practices, by Survey Rounds 33 Figure 11: Awareness of at Least Three of the Five Critical Times to Wash Hands, by Survey Rounds 35 Figure 12: Children 6-23 Months Consuming Four or More Food Groups During the Previous Day, by Survey Rounds 37 Figure 13: Children Aged 0-5 Months Exclusively Breastfed During the Previous Day (past 24 hours), by Survey Rounds 38 Figure 14: Primary Occupation of Household Head, by Survey Rounds 70 Figure 15: Consumption of 9 Food Groups in the Past 7 Days, by Survey Rounds 71 Figure 16: Household Livelihood Activities in the Past 12 months, by Survey Rounds 71 Figure 17: Household Coping Strategies in the Past 7 Days, by Survey Rounds 72 Figure 18: Household Income Generating Activities, by Survey Rounds 72 Figure 19: Households Reporting Net Profit, by Livelihood Activity and Survey Rounds 73 Figure 20: Change in Household Income Over the Last 12 Months, by Survey Rounds 73 Figure 21: Availability of Natural Resources, by Survey Rounds 73 Figure 22: Year-Round Household Access to Natural Resources, by Survey Rounds 73 Figure 23: Households Using Improved NRM Practices in the Past 12 months, by Survey Rounds 74 Figure 24: Household Access to Farmland, by Survey Rounds 74 Page | 5 Figure 25: Household Utilization of Farmland to Which They Have Access, by Survey Rounds 74 Figure 26: Household Use of Improved Practices in the Last Planting Season, by Survey Rounds 74 Figure 27: Households Using at Least Three Improved Production Practices, by Survey Rounds 75 Figure 28: Households Using at Least Six Improved Production Practices, by Survey Rounds 75 Figure 29: Households’ Main Source of Food in the Past 12 Months, by Survey Rounds 75 Figure 30: Access (easy/very easy) to Markets within 5km, by Survey Rounds 76 Figure 31: Households With Improved Drinking Water Sources, by Survey Rounds 76 Figure 32: Households Using Unsafe Drinking Water, by Survey Rounds 76 Figure 33: Household Actions to Make Unsafe Drinking Water Safe, by Survey Rounds 76 Figure 34: Households With Improved Toilet Facilities, by Survey Rounds 77 Figure 35: Household Toilet Facilities, by Survey Rounds 77 Figure 36: Respondents’ Perception of the Five Critical Times to Wash Hands, by Survey Rounds 77 Figure 37: Males Who Know at Least Three of Five Critical Times to Wash Hands, by Survey Rounds 77 Figure 38: Females Who Know at Least Three of Five Critical Times to Wash Hands, by Survey Rounds 77 Figure 39: Boys Aged 6-23 Months Consuming Four or More Food Groups During the Previous Day, by Survey Rounds 78 Figure 40: Girls Aged 6-23 Months Consuming Four or More Food Groups During the Previous Day, by Survey Rounds 78 Figure 41: Boys Aged 0-5 Months Exclusively Breastfed During the Previous Day (past 24 hours), by Survey Rounds 78 Figure 42: Girls Aged 0-5 Months Exclusively Breastfed During the Previous Day (past 24 hours), by Survey Rounds 78 Page | 6 LIST OF ACRONYMS BHA Bureau for Humanitarian Assistance CAFOD Catholic Agency for Overseas Development CAG Community Action Group CI Confidence Interval CRS Catholic Relief Services rCSI Reduced Coping Strategies Index CU5 Children under five years old DFID Department for International Development DRR Disaster Risk Reduction ENA Essential Nutrition Actions EFSP Emergency Food Security Program FANTA Food and Nutrition Technical Assistance FCS Food Consumption Score FGD Focus Group Discussion FTF Feed The Future HAC Humanitarian Aid Commission HHS Household Hunger Scale IGA Income-generating activity INGO International Non-Governmental Organization IPM Integrated Pest Management KG Kilogram MEAL Monitoring, Evaluation, Accountability and Learning NCA Norwegian Church Aid NGO Non-governmental organization OFDA Office of U.S. Foreign Disaster Assistance OLS Ordinary least squares PaBS Participant-based Surveys PIRS Performance Indicator Reference Sheet PNK Phosphorous, nitrogen, potassium (chemical fertilizer) PPS Probability Proportional to Size PSP Private Service Provider RISING Recovery in Sudan for Improved Nutrition and Growth SDG Sudanese Pound SILC Savings and Internal Lending Communities TANGO Technical Assistance to Non-governmental Organizations TO Topical Outline UN United Nations USAID U.S. Agency for International Development USG United States Government WASH Water, Sanitation, and Hygiene WVI World Vision International Page | 7 EXECUTIVE SUMMARY The Recovery in Sudan for Improved Nutrition and Growth II (RISING II) project is an 18-month project (February 1, 2020 to June 30, 2021) that aimed to improve livelihoods and nutrition among 28,060 targeted households in all five Darfur states. A No Cost Extension (NCE) was provided through January 31, 2022, extending the Life of the Activity (LoA) to a total of 25 months. Funded by the United States Agency for International Development (USAID) Bureau for Humanitarian Assistance (BHA),1 the RISING II project implemented complementary activities (i.e., with Taadoud II) to assist households to transition from the emergency to long-term recovery program with the goal that “Vulnerable communities in Darfur have enhanced food and nutrition security year-round”. RISING II focused on mitigating the impacts of shocks, preventing loss of household assets, and accelerating recovery for vulnerable returnee and resident (“host”) households in 92 targeted communities. Led by Catholic Relief Services (CRS), the RISING II project was implemented by a consortium that also included World Vision International (WVI), Norwegian Church Aid (NCA), and the Catholic Agency for Overseas Development (CAFOD), four private sector partners, and seven local implementing partners. The purpose of the final evaluation is to assess the outcome and overall impact of the RISING II project and to understand the degree to which it was effective at achieving its goals. Methodology. The RISING II endline/final evaluation applied a mixed-methods approach using quantitative methods to establish endline data points for key project indicators, and qualitative methods to generate narrative data to better understand and triangulate with the quantitative data, as well as answer certain evaluation questions on project success, constraints, challenges, and lessons. Per the Scope of Work, results are presented only for the 12 top-line impact and outcome logframe indicators presented in Table 1. Results for indicators used as part of calculating the 12 logframe indicators are also reported here but their data are presented in Annex VIII. Quantitative: The RISING II endline quantitative survey (i.e., the same instrument as used in the baseline study) was designed without control groups and followed an “adequacy study design” or “non￾experimental study design” for simple pre-post comparison. A two-stage cluster sampling design was followed to select 48 communities (52 clusters) in stage-one using Probability Proportional to Size (PPS). Total sample size was 1,040 project participant households and 20 households were selected randomly from each of the 52 sample clusters (48 communities) in stage-two. The five data collection teams (27 enumerators and 5 team leaders) attended a five-day virtual training. Quantitative data collection took approximately one month to complete and occurred from September 26, 2021 to October 22, 2021. Baseline quantitative data were collected in May 2020, after the winter harvesting and before the lean season started (May/June). In contrast, endline data were collected during the last two months of the lean season (Sept/Oct). As a result, some of the logframe indicator results, in particular, household hunger (Impact i) and reduced coping strategy index (Outcome 1.2.1), may be biased by seasonal and survey timing differences. Qualitative: The evaluation methodology also included focus group discussions (FGD) and key informant interviews (KII) with project participants, implementing partners, and government stakeholders in order to gather information that would help explain results from the quantitative indicator data and that would specifically speak to the effectiveness of project management, systems, and processes. A purposive sampling strategy was used to select respondents in a way that would ensure adequate representation of marginalized groups and a range of variation across beneficiary types (e.g., men, women, youth, mothers/caregivers of certain aged children, etc.), geographic locations, and programming strategies. Five national consultants were trained and assigned one of the five states in Darfur to serve as field supervisor for each state’s field enumerators. Data Quality Assurance & Analysis. Quantitative data was analyzed for two levels of disaggregation at the household level (RISING II participating household): i) household status (returnees vs hosts) and ii) 1 The award was originally jointly funded by USAID Food for Peace (FFP) and the Office of Foreign Disaster Assistance (OFDA). Page | 8 gendered household types (FNM, MNF, M&F and CNA2 ) to report disaggregated information for the logframe indicators. The sample size was estimated to be representative across the entire sample, as suggested in the Feed the Future Population Based Sampling (PBS) guidelines,3 but not at household levels of disaggregation (stratification). The sample households were selected across the gendered household types, a list of which was generated from the household roster of the sample households. Additionally, data was disaggregated by the sex of the household head and household livelihood status (farmers or pastoralist). The statistical software STATA 15.1 was used to analyze quantitative data. Statistical significance for the differences from baseline to endline was tested and level of significance is denoted as: *p<0.10; **p<0.05; and ***p<0.01. Analysis of qualitative data involved use of a data matrix to organize and condense, filter and aggregate data to identify patterns, trends, and outliers with respect to the research questions and topical outlines. The non-response rate was 10% at endline, almost twice the non￾response rate at the baseline (6%), primarily due to the absence of household members during data collection. Key Findings. Overall, project achievements for logframe indicators were positive, with statistically significant improvement between baseline and endline for six of the 12 indicators: IMPACT i, Outcome 1.1, Outcome 1.3.1, Outcome 2.1, Outcome 2.1.1, and Outcome 2.1.2 (Table 1). Among these six, the EOP targets were nearly achieved (over 80% achievement) for all but Outcome 2.1.1, which overachieved its EOP target (132% achievement). Logframe indicators related to household hunger scale (IMPACT ii) and reduced coping strategy index (Outcome 1.2.1) were slightly worse at endline but differences were not statistically significant. Changes between baseline and endline for remaining logframe indicators were mostly positive but the differences were not statistically significant nor were EOP targets met. In general, it appears that the EOP targets for the logframe indicators were heading in the right direction, but were too ambitious/unrealistic to achieve in a short-term (18-25 months) project. Table 1: RISING II Project Logframe Indicators LOGFRAME INDICATORS BASELINE (May 2020) ENDLINE (Oct 2021) ACHIEVEMENT ALL 95% CI DEF TARGET % Achieved 1. IMPACT i: % of HH with poor, borderline, and adequate Food Consumption Score (FCS) % Poor (FCS: 0-28) 23.5 13.4*** (9.4, 17.5) 1.801 0.0 95% % Acceptable (FCS >42) 47.1 59.0** (51.7, 66.3) 2.263 62.1 Partially 2. IMPACT ii: Prevalence of HH with moderate or severe hunger (Household Hunger Scale, HHS) Percentage (%) 20.8 21.9 (18.1, 25.8) 1.425 10.8 Not Achieved 3. OUTCOME 1.1: % of HHs with equitable and uninterrupted access to shared natural resources throughout the year. At least 3 (%) 45.9 61.3*** (55.6, 67.0) 1.775 65.9 93% At least 4 (%) 24.6 38.4*** (32.5, 44.4) 1.857 Partially All 5 (%) 7.5 12.3** (8.5, 16.0) 1.735 4. OUTCOME 1.2.1: Reduced Coping Strategies Index (rCSI) Mean 11.7 12.8 (11.4, 14.3) 2.045 8.8 Not STD 10.72 10.70 Achieved Min. 0.0 0.0 Max. 56.0 52.0 5. OUTCOME 1.3.1: % of HH using improved natural resources. Percentage (%) 33.8 52.5*** (44.8, 60.2) 2.226 63.8 82% Partially 6. OUTCOME 2.1: Number of months of HH food self-sufficiency as a result of improved agricultural production programming Mean (Month) 4.6 5.5*** (5.1, 5.9) 2.026 5.9 94% STD 3.0 2.9 Partially Min. 0.0 0.0 Max. 12.0 12.0 n (household) 924 938 7. OUTCOME 2.2: % of beneficiaries reporting net income from their livelihood. Percent (%) 72.9 67.1 (62.9, 71.2) 1.301 92.9 Not n (household) 755 868 Achieved 8. TOTAL (Hectares) 2,758 4,007* - 2.509 3,034 132% 2 FNM: Adult Female and No adult Male; MNF: Adult Male and No adult Female; M&F: Both Adult Male and adult Female; CNA: Child and No Adults; adult means 18 years or older. 3 Feed the Future Population-Based Survey Sampling Guide, Diana Maria Stukel, PhD, April 2018 (page 25-26). Page | 9 LOGFRAME INDICATORS BASELINE (May 2020) ENDLINE (Oct 2021) ACHIEVEMENT ALL 95% CI DEF TARGET % Achieved OUTCOME 2.1.1: Number of hectares under improved agricultural methods N (household)4 13,551 13,501 Over Achieved 9. OUTCOME 2.1.2: % of HH with access to sufficient seed to plant Percent (%) 33.1 55.0*** (47.5, 62.5) 2.187 58.1 95% n (household) 924 Partially 10. OUTCOME 3.1: Proportion of children 6-23 months of age who receive foods from 4 or more food groups All (%) 19.3 25.3 (17.9, 32.7) 0.979 39.3 Not Boy (%) 15.6 21.4 (11.8, 31.0) 0.968 Achieved Girl (%) 24.4 29.4 (19.5, 39.4) 0.879 n (children) 197 136 11. OUTCOME 3.1.1: Proportion of infants 0-5 months of age who are fed exclusively with breast milk All (%) 45.7 50.7 (36.3, 65.2) 1.320 65.7 Not Boy (%) 48.7 55.3 (36.7, 73.8) 1.293 Achieved Girl (%) 41.6 44.2 (26.9, 61.4) 0.999 n (children) 85 85 12. OUTCOME 3.1.2: % of people targeted by the hygiene promotion program who know at least three (3) of the five (5) critical times to wash hands All (%) 31.1 35.6 (29.1, 42.0) 2.058 51.1 Not Male (%) 20.8 31.4* (23.5, 39.3) 1.630 Achieved Female (%) 39.2 38.2 (31.3, 45.0) 1.678 n (person) 924 938 *p<0.10, **p<0.05 and ***p<0.01; Statistically significant from baseline to endline Household Food Consumption, Food Security, and Reduced Coping Strategies: The percentage of households with adequate food consumption increased significantly from 47% at baseline to 59% at endline (p<0.05). This pattern was similar for all categories of project participating households. The percentage of households with moderate or severe hunger showed a slight worsening (i.e., an increase from 20.8% to 21.9% of households between baseline and endline) but was not statistically significant. Similarly, the average reduced coping strategy index changed from 12 at baseline to 13 at endline (on a scale of 0 to 56), which reflects a slight but not statistically significant worsening. Results for household hunger and reduced coping strategies are in the opposite predicted direction (i.e., they are predicted to improve between baseline and endline as a result of project activities), and presumably reflect the difference in timing of data collection for the baseline and endline. The baseline data was collected in May 2020, which was after the winter harvest and before lean season starts throughout Darfur. In contrast, endline data were collected during the last two months of the lean season, from the end of September to mid-October 2021, i.e., at the time of most food security stress. The significant increase in households with adequate levels of food consumption may be due, at least in part, to distribution of food baskets by the RISING II project to project participant households during the lean season. Regardless, the end of project (EOP) target of 62.1% (Impact i) was not achieved (Table 1). Qualitative data show that people mostly felt their food security and nutrition situations had improved as a result of the RISING II project, although this varied across states. Some respondents suggested that the food baskets and supplementary feeding provided through RISING II contributed to better nutrition and less hunger. Overall, the majority of farmer and pastoralist respondents felt that their food security had improved largely as a result of learning new agricultural techniques along with better and more diversified crop production and use of improved seeds provided by the RISING II project. Livelihood and Income Generating Activities (IGAs): Participant households were mainly engaged in two income generating activities: i) farming/crop production and sales, and ii) livestock/livestock products production and sales. Household engagement in both increased significantly from baseline to endline. Households self-reporting they had net income (indicator Outcome 2.2) decreased from 73% at baseline to 67% at endline (not statistically significant). However, more households reported their income had increased over the last 12 months at endline (34%) than at baseline (19%), a positive and statistically 4 Extrapolated to a total of 13,351 farmers and pastoralists in the baseline and 13,501 at the endline (RISING II project participants). Page | 10 significant trend. Overall, households experienced decreased net income/profit, due in large part to higher input costs, higher fuel costs and associated commodity price inflation,5 and inaccessibility of markets due to COVID-19 restrictions. The EOP target of 93% for the logframe indicator Outcome 2.2 was not achieved (Table 1). Insights from qualitative data show that most respondents thought one of the project’s biggest impacts was on household income, primarily as a result of increased production. In particular, FGD participants across all five states felt that farmers had strengthened their income-generating capacity “because of crop production and livelihood skills gained in SILC groups.” Increased household income provided the funds for school fees and purchasing livestock, such as goats and donkeys, the donkeys being used for ploughing and transport purposes. SILC activities were also perceived to have improved people’s ability to save money for later investment in productive activities and improved their ability to engage in markets. Equitable Year-Round Access to Natural Resources: RISING II project activities improved availability of and year-round access to community-managed natural resources for all categories of project participant within the short duration of project implementation. Household access to grazing/pasture land, forest/forest products, and government inputs that support natural resources management increased significantly from baseline to endline. Year-round access to at least three community-managed natural resources increased significantly (p<0.01) from baseline (46%) to endline (61%) (Outcome 1.1) across all participant categories. Despite this achievement, the project only partially achieved (93% achievement) its EOP target (65.9%) for this indicator (Table 1). This pattern holds for the logframe indicator Outcome 1.3.1 as well: there was a statistically significant increase in the percentage of households using at least two improved natural resources practices (34% at baseline and 53% at endline) but despite this positive trend, the project did not achieve the EOP target of 63.8% (Table 1). The use of improved crop culture practices, permaculture, climate mitigation practices, and agroforestry all increased by endline, though the increases were only statistically significant for the first two practices. Insights from qualitative interviews with project participants, partners, and government stakeholders suggest that most communities experienced better management of - and access to - natural resources as a result of the RISING II project. There was widespread agreement among all participants in FGDs and from KIIs that the RISING II project provided communities with good training on natural resource management and disaster risk reduction (DRR), both of which helped them to manage their natural resources and deal with risks from hazards, such as drought, more effectively. Although respondents overwhelmingly indicated that everyone had equal access to the natural resources managed by the community, they also acknowledged that conflicts can arise, particularly over shared use of land and water. The CAG system and/or ajawids served as effective conflict-mitigating mechanisms in the communities. Improved Agricultural Practices: The RISING II project promoted 11 improved agricultural technologies and practices, seven of which are used to compute the “Number of hectares under improved agriculture practices” indicator (Outcome 2.1.1). The total number of hectares under improved practices increased from 2,758 hectares at baseline to 4,007 hectares at endline, a statistically significant increase. Thus, the project overachieved the EOP target of 3,034 hectares for the logframe indicator Outcome 2.1.1 (Table 1). The Outcome 2.1.2 indicator measures and tracks availability of an uninterrupted and sufficient supply of seeds for RISING II project participants. By linking to local seed suppliers and distributing vouchers for seeds through DiNER fairs, the project achieved 95% of its EOP target (58.1%) (Table 1). The percentage of households with access to sufficient seed increased significantly between baseline (33%) and endline (55%); this was true for all participant categories except households with adult males but no adult females (MNF), for whom there was no statistically significant change. The majority of project participant households used their own production as their main source of food at both baseline and endline. However, the average number of months of household food self-sufficiency 5 Sudan experienced an inflation rate of 300% in 2020 and 2021, which caused major distortions in market prices. Page | 11 (indicator Outcome 2.1) statistically increased by one month, from 4.6 months at baseline to 5.5 months at endline (p<0.01). This result represents 94% achievement of the EOP target of 5.9 months of food self￾sufficiency (Outcome 2.1) (Table 1). Focus group participants noted that many families have increased the number and/or size of their cultivated plots and have experienced an increase in crop production through implementation of various new farming techniques learned through trainings conducted by the project. For example, in Central Darfur, cultivating fodder (e.g., alfalfa) for livestock was an important new practice promoted through the project; seeds were provided by RISING II through seed fairs. Farmers increased production by using agricultural inputs provided by the RISING II project as well as by increasing their use of improved practices/technologies on land they cultivated. Production levels also increased as a result of farmers interacting with better trained agricultural extension workers, for example, trained in the use of good agricultural / climate resilient practices. Some farmers noted that they cultivated reclaimed land, which they long thought to be inappropriate for growing crops. Household training on the nutritional value of crops, combined with better agronomy through use of quality seeds, crop rotations, winter agriculture, and better postharvest storage benefitted agricultural production in many areas. Women, in particular, felt they had benefitted greatly from the project, in both production and increased engagement in markets. It was, however, noted that female-headed households and returnees did not “enjoy the same benefits” from agricultural services (and SILCs) as others in the community and were thought to often lack access to seeds, tools, and knowledge on good farming techniques. Hygiene Practices: Overall, hand washing practices improved though they were not statistically different than at baseline; knowledge of at least three of the five critical times to wash hands was 31% at baseline and 36% at endline (Table 1). Overall, the RISING II project did not achieve the EOP target of 51% for Outcome 3.1.2. According to both men and women in FGDs and KIIs across the five states, there is much more awareness and practice of washing hands at critical times, such as before eating, after defecation, etc. now than previously, including washing with soap when available and with ash if not. The project, along with the Ministry of Health, conducted large-scale awareness-raising campaigns through the radio and by mobile phone, especially as part of the COVID-19 response. According to some, state-promoted COVID-19 responses included establishing hand washing stations in communities, which were supplied with soap for hand washing and chlorine for disinfection and water treatment. Infant and Child Feeding Practices: Positive – but not statistically significant – changes from baseline to endline were observed for the two infant and child (0-23 months of age) feeding indicators: i) children 6-23 months who received at least four out of seven possible food groups in the past 24 hours (Outcome 3.1), and ii) children 0-5 months exclusively breastfed in the past 24 hours (Outcome 3.1.1). As a result, the project did not achieve its EOP target for Outcome 3.1 (39.3%) or Outcome 3.1.1 (65.7%) (Table 1). Across all states, FGDs and KIIs perceived that there was great improvement in the nutritional status of young children (CU5), due to eating more meals per day as well as eating more diverse foods (e.g., from vegetable production). Quantitative data show the percentage of boys and girls 6-23 months old consuming four or more food groups increased between baseline and endline, although the differences were not statistically significant. CHWs also reported that attitudes around breastfeeding have started to change, mainly among women and mothers who are now exclusively breastfeeding just after birth and for the first six months. While most mothers understand the importance of providing breastmilk, some still have not applied the practice. Although it was widely agreed that RISING II activities contributed to improved knowledge about hygiene, hand washing, sanitation practices (use of toilets rather than open defecation), and the importance of nutritious and diverse foods, challenges remain, including lack of any medical services or health centers in many communities, a lack of staff in existing centers, and limited success changing some behaviors. Qualitative information from community health workers (CHWs), mothers/caregivers, and others suggests the project may have placed more emphasis on identifying moderate and severe acute malnourished children and referring them to health care facilities where it Page | 12 should have focused more on IYCF messaging and awareness. COVID-19 restrictions also interrupted – or delayed – community awareness-raising activities. Conclusion and Recommendations. Quantitative and qualitative evidence from the final evaluation shows that the RISING II project had significant and positive impacts on beneficiary households across all five states in Darfur. Although not all end of project targets were achieved, tremendous progress was made. The positive outcomes from the work indicate there are numerous opportunities for future programs to learn from the RISING II final evaluation results and to use this knowledge for greater impact on food and nutrition security of vulnerable populations in Darfur. Several lessons learned have emerged, particularly from qualitative interviews, which are presented below along with some recommendations. Project Activities ❖ Livelihood activities: Shocks from climate change (e.g., drought) are likely to continue negatively impacting households’ livelihoods (primarily farming and livestock-rearing) and putting households and communities at recurring risk of food insecurity. Future programs should identify and promote alternative and innovative income-generating activities that are at less risk from the effects of climate change. ❖ Access to Financial services: Although the SILC activities are effective and should be continued/expanded, participants from the FGDs and KIIs recommend more support for financial service providers and links with financial institutions in future programming, including for women. ❖ Nutrition and Health: According to project stakeholder interviews, there is currently an over-emphasis on referring children with severe/moderate acute malnourishment to treatment centers. Future programs should better balance project activities that promote nutritional referrals with those that promote other key health interventions and messaging (e.g., exclusive breastfeeding, diet diversity, handwashing). Messaging strategies can be strengthened by introducing hardware components (e.g., toilet parts, materials/parts for water points) with hygiene promotion activities and training lead mothers at the community level for IYCF messaging. Project Management ❖ Data collection: The timing of data collection can complicate interpretation of project outcomes. Data collection activities should be timed and coordinated to occur during the same time of year (ideally during the lean season) so that food security and other indicators with potential seasonal effects (e.g., coping strategies) can be measured and interpreted accurately. ❖ End of Project targets: The End of Project (EOP) targets for the 12 logframe indicators reported here were overly ambitious given the short project timeframe. The baseline and endline surveys were conducted approximately one year apart. EOP targets should be based on findings from project baseline data combined with what project technical staff feel is realistic to achieve over the LOA rather than using targets from projects with longer timeframes (i.e., Taadoud). ❖ Activities appropriate to project duration: Even with a NCE, 25 months is too short a timeframe to facilitate some types of change, particularly for many nutritional outcomes, behavior change related to culturally sensitive issues (e.g., breastfeeding, women’s engagement outside the house), and people’s coping strategies in response to shocks and stressors. It also takes time to strengthen resilience capacity and for households to adopt – and adapt to – new ways of growing crops and rearing livestock or taking on new types of livelihoods. An 18- or 25-month project is more in line with humanitarian response programming whereas development programs more typically involve a minimum of four years for implementation. If donors are not able or willing to extend project timelines, then implementing partners need to prioritize activities to those for which improvements can be made and easily measured within a short timeframe. ❖ Internal processes: Based on feedback from state-level project staff and implementing partners, CRS should streamline and/or strengthen their procurement process(es) to improve efficiency, logistics, and Page | 13 timeliness of delivering goods and services. They should also review staffing levels and capacity to inform hiring additional staff at the state level. ❖ Project visibility: Insights from qualitative data suggest the need to increase visibility of the project to better engage beneficiaries, communities, and government partners (e.g., government ministries), particularly as part of its launch. Government, community, and other partners need to be engaged in all phases of the project, including the design phase and community leaders can help mobilize their communities. A. INTRODUCTION The USAID/BHA-funded Recovery in Sudan for Improved Nutrition and Growth II (RISING II) was implemented to improve the livelihood and nutrition outcomes of 28,060 households in 92 communities across all five states of Darfur using a combination of mixed modalities and complementary activities that helped households transition from emergency assistance to longer-term recovery programs. RISING II was originally an 18-month project (February 1, 2020 to June 30, 2021) but received a No Cost Extension (NCE) through January 31, 2022 (for a total of 25 months) and was designed to build on and leverage results from its predecessor project, RISING, while expanding from three states (West, Central and South Darfur) to include North and East Darfur. The expansion aligned RISING with Taadoud implementation areas, allowing RISING II to reach vulnerable households in two states that are experiencing food insecurity rates equal to or higher than the current project areas. RISING II program activities leverage the livelihoods, nutrition and conflict mitigation components of Taadoud. The goals and strategic objectives of both programs are aligned and approaches are designed to be complementary. The three strategic objectives of RISING II were: i) Communities have inclusive institutions and governance of natural resources, ii) Livelihoods of food insecure people are improved, and iii) Households have improved nutrition behaviors. RISING II focused on mitigating the impact of shocks, preventing erosion of household assets, and accelerating recovery for vulnerable returnee and resident households in the targeted communities. CRS led a consortium, including World Vision International (WVI), Norwegian Church Aid (NCA) and the Catholic Agency for Overseas Development (CAFOD). The consortium was complemented by private sector partners, including Haggar & SAY Groups, DAL Innovative Agriculture Company (DIAC), a learning partner, Tufts University, and seven local implementing partners. RISING II shared management, partners, and project structures with Taadoud. RISING II employed tailored interventions designed to help households move from crisis to recovery. Communities throughout Darfur have relied on humanitarian interventions for decades; services, practices and resources ebb and flow with project cycles and donor interests. Based on the combined experience of CRS and consortium partners in transformational programming, RISING II activities, including the most critical life-saving interventions, focused on engaging community members and promoting local ownership and greater sustainability of practices after the project ends. That is, RISING II engaged communities to lay the groundwork necessary to begin building ownership for eventual sustainability. This work was aligned with the Government of Sudan’s (GoS) Darfur Recovery Strategy, this holistic community-based approach promotes a transition from humanitarian response to recovery and sustainable practices. The RISING II project included a robust baseline, and an accountability, learning and evaluation component to improve understanding and analysis of resilience in the Darfur context. TANGO International was commissioned to conduct the final evaluation of the RISING II project, which includes design of quantitative and qualitative methodologies, preparation of evaluation instruments, training and technical support on quantitative and qualitative data collection and analysis, dissemination of evaluation findings and provision of a comprehensive report. The final evaluation report provides explanation and interpretation of the achievements of surveyed logframe indicators, project impacts and outcomes and lesson learned. Based on the final evaluation findings, TANGO had made some recommendations that might be useful for future programming. Page | 14 B. RISING II FINAL EVALUATION 1. Purpose and objectives The main purpose of the RISING II final evaluation is to gauge evidence on the outcome and impact of the RISING II project. The evaluation results provide useful information to understand the extent to which the project achieved its outcomes and goals, and the overall effectiveness of project intervention that was implemented for a total of 25 months, including a 7-month No Cost Extension (NCE). Following are the specific objectives of the RISING II final evaluation: Quantitative. To measure quantitative changes compared to baseline situation: ● Determine end-line values for the project outcome and impact indicators validate and compare the progress with baseline and identify the implementation gaps with justification and context-based recommendations. ● Review the proposed assumptions within the LogFrame Approach (LFA) and provide analysis and feasible recommendations for each. Qualitative. To understand and measure qualitative changes, effective strategies, and lessons learned from program implementation and inform recommendations for future programming: ● Evaluate the achievements of the activity in relation to the goal, objectives, results, and targets. ● Evaluate how the activity affected local markets and certain groups of interest (women and men; the youth population; boys and girls, etc.). ● Evaluate the effectiveness and relevance of the modalities, transfers, and complementary interventions to achieve activity outcomes. ● Identify best practices, lessons learned, strengths, and challenges in the activity design and implementation for achieving project achievements. 2. Methodology The RISING II endline/final evaluation applied a mix-methods approach using quantitative methods to establish endline data points for key project indicators and assess achievement of LOA targets for 12 logframe indicators. Qualitative methods were used to generate narrative data to provide context and better understanding of the quantitative data, as well as answer evaluation questions on project success, constraints, challenges, and lessons. The mixed-methods approach also involved a desk review of project documents, including from the baseline (e.g., quantitative and qualitative tools, inception and final reports, the qualitative plan and summary report, and training materials), Taadoud documents (e.g., baseline and endline reports, quantitative and qualitative tools, and training materials), Performance Indicator Reference Sheets (PIRS) for the 12 logframe indicators, among others. The study methodology was designed to allow for triangulation of quantitative and qualitative information in an integrated manner and for generating evidence-based lesson learning. Quantitative. The RISING II endline quantitative survey was designed to estimate the extent of changes in the RISING II impact and outcome level logframe indicators over the 25 months of project implementation. Although quasi-experimental design including control groups is appropriate for measuring the true effect of program interventions in a project such as RISING II, it was not possible to identify and work in control areas due to the lack of government concurrence/security concerns and similar existing projects (Taadoud II) in the potential counterfactual communities. It is also suggested in the USAID/BHA EFSP project sampling guideline6 to use cross-sectional study design for pre and post-test for detecting certain percentage changes from baseline to endline. Therefore, the RISING II endline quantitative survey was designed without control groups and followed an “adequacy study design” or “non-experimental study design” for simple pre-post comparison. A two-stage cluster sampling design was followed, with RISING II project 6 FFP Guidance for an Abbreviated Statement of Work (SoW) for a Baseline Study and Final Evaluation for Emergency Programs (Revised on December 2019) Page | 15 “communities” considered as “clusters”. The “Probability Proportional to Size (PPS)”procedure was applied to select the sample clusters. The sampling design did not involve any stratification and samples were selected across the five Darfur program states and localities. With approval from BHA and CRS, the final sample size was estimated to be representative at the project level but was not sufficient for disaggregation by returnee, host, or gendered household types at the state level. That is, the sample size allows for detecting differences at the overall sample level across rounds but not for different levels of stratification. The minimum required sample size was estimated first for the household level indicators considering 10- percentage point changes from baseline to endline with 15% non-response rate. The estimated sample size for household level indicators was inflated to get representative sample of children 0-5 months and 6-23 months to estimate infant and child feeding indicators. Therefore, the final estimated sample size was 1,040 households. The sample size calculation is detailed in ANNEX I. In two-stage cluster sampling design, 52 clusters (48 project communities7 ) were selected in stage-1 using PPS sampling procedure across all 5 Darfur program states. In stage-2, 20 households per sample cluster were selected randomly from the list of RISING II project participating household sampling frame to survey 1,040 households across all five Darfur program states and localities. Five quantitative data collection teams with 27 enumerators (all male8 ) and 5 team leaders (all male) were engaged in collecting quantitative data. The quantitative survey field work was conducted from September 26, 2021 to October 22, 2021. Data analysis was based on the guidelines and PIRS of the 12 logframe indicators. A data analysis and tabulation plan (ANNEX II) was prepared for the logframe indicators and was approved by RISING II during the inception period. Disaggregated results for the 12 logframe indicators are presented in ANNEX III. The data analysis plan includes indicator definitions with detailed computation procedures and reference question numbers in the quantitative survey questionnaire (Volume II). Mainly, the statistical software STATA 15.1 was used to analyze the dataset. The sampling method for the RISING II quantitative survey was designed without stratification and clusters were sampled across the 5 Darfur program states and localities. Therefore, sample-weights were not applied to estimate totals. However, non-response weights was applied to adjust non-response by cluster. The analyses in this report include descriptive statistics and composite indicator computation with statistical significance tests for the differences from baseline to endline. Confidence intervals (95%) and standard deviation9 with design effect were computed for the logframe indicators to test the achievement of the indicator targets. The indicator “Outcome 2.1.1: Number of hectares under improved agricultural methods” was computed for the total farmers (project participants only) applying weights suggested in the FTF/USAID PaBS guideline.10 Qualitative. The RISING II endline evaluation collected qualitative data from the community and beneficiary levels to provide contextual information for the quantitative findings across the RISING II implementation area regarding the effects of RISING II activities on i) people’s livelihoods and household income, including access to markets and financial services, ii) community governance of natural resources and conflict resolution, and iii) food security and nutrition outcomes and hygiene behaviors. See Volume II for topical outlines specific to different stakeholder groups. Additionally, qualitative data were collected to assess the effectiveness of project management, systems, and processes established by the project, including 7 More than one clusters were selected from the larger communities in PPS sampling procedure 8 No female enumerators were either available or qualified. 9 For the indicators with mean value. 10 Feed the Future “Participant-Based Survey Sampling Guide for Feed the Future Annual Monitoring Indicators”, Diana Maria Stukel, PhD, September 2018 Page | 16 relevance, effectiveness, efficiency, impacts,11 and sustainability. Key questions regarding project management and processes are presented in ANNEX IV. A Training of Trainers (TOT) occurred October 11-14 and 17 for the national consultants and RISING partner staff from each of the five states. Training was conducted virtually for 2.5 hours each day (p.m. in Sudan), with teams meeting during their mornings (i.e., 9 a.m. – noon Sudan time) to continue reviewing the topical outlines and practicing interviewing each other. Training included a review of the purpose of the study, methods, norms and guidance on qualitative data collection (e.g., gender sensitivity, research ethics, inclusion, confidentiality, neutrality, etc.), and detailed review of the topical outlines. The five national consultants were assigned one of the five states in Darfur, serving as the field supervisor for each state team of field enumerators and conducting KIIs with staff from implementing partners, government, and other stakeholders. Field enumerators from each state were selected through consultation between CRS and their implementing partners, and consisted of local project staff with knowledge of the project, area, and local language(s). A purposive sampling strategy was used to ensure adequate representation of marginalized groups and a range of variation across beneficiary types (e.g., men, women, youth, mothers/caregivers of certain aged children, etc.), geographic locations, and programming strategies. Project staff in each state worked with community leaders to select individuals for FGDs and KIIs. ANNEX V shows the proposed field schedule for qualitative data collection. While purposive sampling may introduce some selection bias, the mixed￾methods approach allows for the triangulation of qualitative findings with statistically representative data from the quantitative study. Data collection occurred from October 18, 2021 to November 11, 2021. All interviews are preceded by a brief introduction of the purpose of the study and an explanation of the voluntariness and confidentiality of interviews. Qualitative data was collected from focus group discussions (FGDs) to collect insights from communities and targeted households (e.g., CAG members, farmers/ pastoralist, mothers/caregivers with children 0-23 months). FGDs were primarily mixed-gender. Exceptions included several FGDs with mothers/caregivers of children 0-23 months. Other single-gender FGDs were due to the availability of respondents rather than by design. A list of the focus groups and key informant interviews that were conducted is provided in ANNEX VI. Key informant interviews (KIIs) or small FGDs were used with key stakeholders, including government counterparts, project staff, service providers and donors. For both beneficiaries and key stakeholders, data was collected with topical outlines designed to complement and contextualize findings from the quantitative survey and capture insights on the qualitative research questions presented in ANNEX IV. Topical outlines are presented in Volume II of this report. All FGDs and KIIs were conducted in Arabic or the local language when necessary. Analysis of qualitative data involved entering all interviews into a matrix format, which allowed the narrative data to be condensed, filtered and/or aggregated to identify patterns, trends, and outliers with respect to the research questions and topical outlines. All data was translated in-country from Arabic (or local languages where necessary) and submitted in English to TANGO. Qualitative results were triangulated with quantitative results from the household survey. Limitation and Challenges 1. Survey timing and season effects: A difference in the timing of the baseline (May 2020) and endline (October 2021) surveys might affect comparisons of some of the indicators. The baseline was conducted before the lean season and after the winter harvest, i.e., at a time of reasonable access to food; the endline was conducted during the lean season, when food is often scarce. This seasonal effect might have a direct influence on the food insecurity indicators (household hunger scale and reduced coping 11 Impacts is used here to refer to what difference, if any, the project has made for beneficiary households. It is not used to denote use of a counterfactual as part of an impact evaluation (IE). Page | 17 strategy index) at both times. That is, households are more likely to have acceptable food security levels at baseline, when it is reasonable to assume they have better access to food (e.g., harvest surplus) than they might at the endline, conducted during the lean season (e.g., low, if any, surpluses). Ideally, the baseline and endline surveys should be conducted at the same time of year in order to more accurately assess the effects of programming (versus seasonality) on any changes in food security that might have occurred over time. 2. Online/virtual training for the data collection teams: TANGO consultants could not travel to Sudan to conduct in-person data collection team training due to COVID-19 travel restrictions. Instead, the survey teams were trained through online sessions (5 days for quantitative teams and 4.5 days for qualitative teams). Many participants joined the training remotely from their respective Darfur States, where internet access was poor or not always reliable. Thus, it is more than likely that the efficacy of the online training was limited due to the periodic absence of individuals and particularly the loss of interactions among participants. This was likely also exacerbated by trainings being conducted in English (with simultaneous Arabic translation), as some state-level participants may not have been comfortable speaking the language. Thus, some enumerators may have learned only partial concepts for some questions. Unfortunately, the implication of this limitation is that the quality of data collected – either quantitative or qualitative – may not be of high quality. In this case, results may not be sufficiently representative of the true effect of the RISING II project on beneficiaries – in either a positive or negative sense. 3. Gender balance of data collection teams: Unfortunately, neither of the quantitative or qualitative teams were gender-balanced, as cultural norms of the operational context make it very difficult for women to work as enumerators. Quantitative enumerators and qualitative data collectors were hired by CRS and the evaluation team cannot speak to their hiring process nor what, if anything, was attempted to gender-balance the team. Qualitative data collectors consisted of RISING II state-level project/implementing partner staff and again, the evaluation team cannot speak to their selection process. The qualitative team included one female national consultant and at least one female field data collector. Including women on data collection teams can help female respondents feel more at ease answering some questions, helping to ensure their voices are represented in the data. Although gender￾balanced teams are preferred, the evaluation team has no reason to suspect that either the quantitative or qualitative data are not representative of female perspectives – or at least no less so than what would be the cultural norm in Sudan. 4. Unavailability of qualified national consultants/field data collection teams for qualitative data collection: Unfortunately, several very qualified national consultants who had been involved with previous evaluations for CRS (e.g., Taadoud) were unavailable for the RISING II endline and it proved difficult for CRS to find equally qualified national consultants who were available. Additionally, the TOT approach meant that there was no quality control by the evaluation team for training local partner staff, who ultimately conducted the focus groups and key informant interviews. Given the nature of qualitative data received from the qualitative teams, it does not appear that all qualitative team members clearly understood their roles and responsibilities or the questions on the topical outlines. While there is some very good data that was collected, there is also a general lack of depth to much of the qualitative data submitted (see No. 5 for more explanation). Overall, however, there are gaps in the qualitative data. 5. Incomplete qualitative information: Unfortunately, the lead national consultant for the qualitative team passed away unexpectedly only several days into the field work. This resulted in serious challenges, including bringing on a qualified replacement to conduct interviews in Central Darfur. Equally important, his passing left the qualitative team without a leader and local figure for quality control, which had a profoundly negative effect on the quality of data collected and/or submitted (see No. 4 above). In addition, the qualitative team barely arrived in the field when a coup occurred in Khartoum, resulting in the shutdown of government and NGO offices across the country, the restriction of movement for field enumerators in terms of conducting interviews, and a total loss of Page | 18 communications for days, followed by sporadic and unpredictable internet and mobile data access for weeks. Qualitative team members (e.g., national consultants, field data collectors) were unable to conduct interviews at the state level or return to Khartoum and their families. It was an unprecedented context in which to try and conduct interviews with key informants (including government officials) and focus groups. 6. Challenges in internet accessibility for online interview to the program management, government counterparts and donors: Internet access was difficult, or spotty, even before the coup. Afterwards, it was not possible to conduct any interviews via remote/virtual approaches. Taken together, this resulted in deficits of certain perspectives in some states (e.g., ministries) and may allow for potential biases in some of the qualitative data. 3. Household Demographics The total endline sample size was 1,040 project participating households, which was slightly higher than the baseline sample size 980 households. The endline sample size was estimated as being large enough to be able to interview adequate sample households with children of ages 0-5 months and 6-23 months for infant and child feeding logframe indicators. The overall non-response rate at endline was 10%, which is approximately two-fold higher than the baseline non-response rate of 6% (Table 5, ANNEX VIII). The main reason for non-response was the unavailability of sample households during the field data collection in both survey rounds (71% in baseline and 46% in endline) (Table 6; ANNEX VIII). Interviews with 21% of the non-response cases were not attempted by the data collection teams; i.e., they finished field work without interviewing some of the sample households. The non-response rate was significantly higher in West￾Darfur (29% at baseline and 24% at endline) than the overall response rate (Table 3; ANNEX VII). The population pyramid generated from the household roster of the sample population for both baseline and endline is shown in Figure 1. Both of the graphs show the distribution of the RISING II project population is consistent with the typical pattern found in least developed countries and remained the same from baseline to endline. The overall sex-ratio (i.e., males to females) was 100 at baseline, which was slightly higher at the endline (i.e., 101), but percentages of males (50.1 % at baseline and 50.3% at endline) and females (49.9% at baseline and 49.7% at endline) remained the same. The sex-ratio for the 15-49 years of age category was found to be 93.2% at baseline and 87.6% at endline, far below that of the overall sex￾ratio (100 or 101). Figure 1 shows the percentage of males was lower (48% at baseline and 47% at endline) than the percentage of females (52% at baseline and 53%) for the age group 15-49 years, which is not the same as the overall percentage for all age groups. This suggests there may be male out-migration (sex-ratio threshold 99.9)12 among this age group of project-participating households, which was slightly higher at endline. Figure 1: Sample Population Distribution by Age and Sex, by Survey Rounds BASELINE 2020 ENDLINE 2021 12 Demography learning module, The Harriet and Robert Heilbrunn Department of Population and Family Health, Columbia University Page | 19 Approximately one-half (47% at baseline and 48% at endline) of the population were children (ages 0-14 years), while only 3% were elderly (65 years or above), with a dependency ratio of 101 at baseline and 106 at endline. The child dependency ratio increased from 95% at baseline to 100% at endline and indicates that the population of RISING II participating households has a population distribution pattern that follows a similar pattern for least developed countries. A large average family size (6.6 at baseline and 6.5 at endline; Table 4, ANNEX VII) and a high population of children (ages 0-59 months) (12.7% at baseline and 13.7% at endline) suggests that family planning is not common among RISING II project participating households. The overall percentage of female-headed households increased from 37% at baseline to 44% at endline, which supports the statement in the previous section that the outmigration of male household members increased over time (Table 3; ANNEX VII). Approximately, three-fourths (71% at baseline and 70% at endline) of household heads were between 15 and 49 years old and there were no household heads younger than 15 years old for both the baseline and endline. Figure 14 (ANNEX VIII) shows crop farming is the main occupation for all household heads within the RISING II project, which decreased slightly from 94 % at baseline to 90% at endline. This percentage is relatively higher among female-headed households (95 % at each round) and decreased from 93% at baseline to 86% at endline among male-headed households. Table 4 (ANNEX VII) shows that close to one-half (43%) of household heads reported no education at endline, which was a slight increase from baseline (39%). Two-thirds of female household heads reported having no education at baseline (63%) as well as at the endline (68%). Education levels for most other types of household heads were typically either at the primary level and/or khalwa13 education. Overall, household heads with a primary level education increased from 16% at baseline to 25% at endline. C. RISING II FINAL EVALUATION FINDINGS The quantitative and qualitative data collected for the endline evaluation have been analyzed and are presented as integrated findings. Quantitative data are disaggregated by household status (returnee/host), household livelihood categories, household gender compositions (adult female with no male (FNM), adult male with no female (MNF), adult male and female (M&F) and child no adult14 (CNA) households), sex of the household head and sex of the respondents (for child health and nutrition related indicators only). Per agreement with the CRS Country Office and the Scope and Work, results for household status and gender compositions are presented in the main report; disaggregated results for all categories are provided in the detailed analysis tables in ANNEX III. Both quantitative and qualitative findings are based on – and organized by – key quantitative and qualitative research questions outlined in the Scope of Work. 4. Project Impact This section provides both quantitative and qualitative findings on what impact the project had on participating households, that is, what difference the project made in their lives and what types of positive or negative consequences were perceived as a result of project activities. The term “impact” is used throughout this report in a general (i.e., non-technical) way in that no counterfactual (i.e., control group) is used to establish attribution of outcomes to the RISING II project; results only show contribution. 1.1. What have been the impacts of RISING II activities on project households? As previously mentioned, the RISING II endline quantitative survey was designed to detect statistically significant changes in impact and outcome level logframe indicators over the 25 months of project implementation. Key quantitative measures of the overall impact of the RISING II project include the following: the Food Consumption Score (FCS), the Household Hunger Score (HHS), and the Reduced Coping Strategy Index (rCSI). 13 Khalwa is a traditional Quranic education school in Sudan that plays an important role in eradicating illiteracy and promoting religious education. This education is different from the primary level education. 14 CNA are households with only children under 18 years of age; there were no CNA households in the sample. Page | 20 Figure 2: Households with Adequate Food Consumption (FCS>42), by Survey Rounds Statistical significance for the difference between endline and baseline are denoted as: *p<0.10; **p<0.05; and ***p<0.01. No stars signify no statistically significant difference. Household Food Consumption Score (FCS). The FCS is computed based on household dietary diversity and food consumption frequency in the seven days prior to the survey. To measure this indicator, respondents were asked about the frequency of their households’ consumption of 16 food or food groups over the previous seven days. The 16 food/food groups were then regrouped into 9 food groups. To compute the FCS, the consumption frequencies were weighted using the individual food group’s nutritional weight (Table 7; ANNEX VIII) and then summed. The sample households were then classified into three groups based on their weighted scores – poor (FCS: 0-28), borderline (FCS: 28.5-42), or acceptable (FCS: >42) using the World Food Program’s recommended cut-off points.15 The cut-off points have been considered for the countries with daily consumption of sugar and oil. Overall, households with an adequate (acceptable) level of food consumption increased significantly from 47.2% at baseline to 59.0% at endline, which falls slightly short of achieving the EOP target (Figure 2). This change from baseline to endline is statistically significant. All categories of households showed statistically significant improvements except returnee and MNF households. Overall, consumption of vegetables, pulses and oil/fat increased significantly from baseline to endline, while consumption of milk/dairy food and eggs/meat/fish consumption also increased but was not statistically significant (Figure 15; ANNEX VIII). One of the reasons for better achievement of this indicator might be the distribution of food baskets from the RISING II project during the lean season. Household Hunger Scale (HHS). The HHS is a food insecurity scale that measures the percentage of households experiencing hunger. The household member responsible for preparing food on a daily basis was asked about the frequency with which the following three events were experienced by any household member in the past 30 days:16 a) There was no food to eat of any kind; b) Any household member went to sleep at night hungry; and c) Any household member went a whole day and night without eating anything at all. The HHS and the household hunger categories (no/little, moderate, and severe hunger) were calculated based on responses from the three questions using the FANTA guideline: 1) Rarely (1–2 times); 2) Sometimes (3–10 times) and 3) Often (more than 10 times). The logframe impact indicator “Prevalence of households with moderate or severe hunger Household Hunger Scale (HHS)” is reported as the combined percentage of moderate and severe household categories. 15 USAID Food for Peace. February 2019. Indicators for Emergency Program Performance Indicator Reference Sheets. P. 7. 16 1) USAID Food for Peace. February 2019. Indicators for Emergency Program Performance Indicator Reference Sheets. P. 12 2) FANTA Guideline: “Household Hunger Scale: Indicator Definition and Measurement Guide”, August 2011, Terri Ballard, Jennifer Coates, Anne Swindale, Megan Deitchler Page | 21 Overall, the percentage of project participating households with moderate to severe household hunger increased slightly from 20.8% at baseline to 21.9% at endline, but the difference was not statistically significant (Figure 3). Figure 3: Households with Moderate to Severe Hunger (HHS), by Survey Rounds Statistical significance for the difference between endline and baseline are denoted as: *p<0.10; **p<0.05; and ***p<0.01. No stars signify no statistically significant difference. The project did not achieve the EOP target of 10.8% for this outcome indicator. Although participant households from host communities, households with Male with no Adult Female (MNF), and male-headed households reported slightly less moderate to severe hunger at the endline compared to the baseline, the differences are not statistically significant. Reduced Coping Strategy Index (rCSI). Although not used as a logframe impact indicator, the rCSI index is a proxy indicator for household food insecurity that is measured from a list of five food-based coping strategies. The index is computed using both the frequency of each coping strategy (i.e., how many times the coping strategy was used by any member of the household in the past seven days) and severity weights (i.e., how serious the strategy). Figure 4: Average Reduced Coping Strategy Index (rCSI), by Survey Rounds Statistical significance for the difference between endline and baseline are denoted as: *p<0.10; **p<0.05; and ***p<0.01. No stars signify no statistically significant difference. Information was collected for the five coping strategies that the household used in the seven days prior to the survey. The rCSI raw scores were computed by multiplying the frequency with which a behavior was Page | 22 used (number of days the household applied this strategy in the past 7 days) by a universal severity weight (Table 8; ANNEX VIII), then summing the weighted scores for each coping strategy.17 The changes in average rCSI score indicates whether household food security status is declining (i.e., improving) or not. There is no universal threshold for rCSI, but the higher the rCSI score, the more severe the coping strategy applied by a household. The weighted rCSI score ranges from 0 to 56, i.e., a household that used all five strategies every day for the last seven days would have a raw score of 56. Higher rCSI indicates a worse food security situation and vice versa. Similar to the HHS, the average rCSI increased slightly from 11.7 at baseline to 12.8 at endline (i.e., worsened), but the difference is not statistically significant (Figure 4). This slight increase was due to the significant increase in use of two coping strategies: 1) reduce adult food consumption; and 2) rely on less expensive or less preferred food (Figure 17; ANNEX VIII). The RISING II project did not achieve the EOP target of 8.8 for the rCSI. A difference in the timing of the baseline (May 2020) and endline surveys (October 2021) may have effected results for the two food security indicators (HHS and rCSI). The baseline was conducted before the lean season, when food is typically not as scarce as during the lean season; the endline was conducted during/at the end of the lean season, when food reserves are typically severely limited. Overall, insights from focus group discussions (FGDs) with project beneficiaries about the impact of RISING II suggest that people mostly felt their food security and nutrition situations had improved as a result of the RISING II project. However, this varied widely across states. For example, key informants in North Darfur suggested there have been tangible improvements in feeding practices, especially for mothers, such as consumption of more diverse foods, while FGDs in West Darfur noted the existence of a hunger gap, which they perceived as contributing to malnutrition, especially among children less than five years old (CU5) and for pregnant and lactating women (PLW). Food baskets and supplementary feeding provided through RISING II for CU5 and PLWs were perceived by numerous key informants (e.g., government partners, project staff) as having contributed to better nutrition and less hunger. In South Darfur, the majority of farmer and pastoralist FGD participants felt that their food security had improved largely as a result of learning new agricultural techniques, coupled with better and more diversified crop production (e.g., tomato, okra, groundnuts, millet, sorghum), seeds of which were provided by the RISING II project. It should be noted that some of the qualitative data presented above (and elsewhere in this report) appear to contradict the quantitative results presented. For example, households with moderate to severe hunger did not change over time (Figure 3), yet FGD participants felt that their food security and nutrition situations had improved as a result of the RISING II project. 17 USAID Food for Peace. February 2019. Indicators for Emergency Program Performance Indicator Reference Sheets. P. 10. Readers should recall that qualitative insights are personal perceptions from a few individuals selected, in this case, by project staff and community leaders and may not represent the average household across the entire implementation area (i.e., results from quantitative analysis). Additionally, participants in FGDs or KIIs often feel they must provide very positive responses to the questions being asked, particularly if there is any chance for additional funding of activities from which they or their communities might benefit. The important point is that both the quantitative and qualitative results are valid even when seemingly contradictory; one must simply keep the context for each in mind when interpreting results. Quantitative results represent the average across all sampled households and qualitative results represent the opinions of the individuals who provided them and may not be generally applicable across their community or beyond. Page | 23 1.2. Have different types of households experienced different types of impact? Figure 2 also clearly shows that the impact of RISING II activities on food consumption differed for different types of households. Overall, fewer female-headed households and Female with No Adult Male (FNM) households tended to have adequate FCSs compared to their male counterpart households at both baseline and endline. However, both improved their food consumption status over time; significantly more female-headed and FNM households reported adequate FCS at endline than baseline (Figure 2). There were also differences between host and refugee households, as well as between farmer and pastoralist households; returnee and farmer households showed greater upward trends in food consumption status at endline compared to baseline than host or pastoralist households. Additionally, returnee and male-headed households both showed statistically significant increases in the rCSI (Figure 4), which reflects increased food insecurity. All other categories of project participant households had slight or no changes in rCSI. 1.3. Did the project have any unintended consequences – either positive or negative? Few unintended consequences – either positive or negative – were reported in FGDs or KIIs. There was some indication from FGD participants and KIs that the project raised expectations – for some – that were not met in terms of where activities were implemented; there were people not targeted by the project (or Taadoud) who also wanted support. One key informant in West Darfur suggested that, “One of the positive results achieved by the project is that the beneficiaries are writing proposals and submitting them to regional organizations for some services, and some of them have been approved, such as a proposal for UNAMID to provide support to the flood victims.” 5. Livelihood Outcomes Household livelihood and income generating activities information was collected to estimate the logframe indicator “Outcome 2.2: Percentage of beneficiaries reporting net income from their livelihood.” Net income is simply defined here as revenues greater than costs. It is synonymous with “profitability.” This indicator measures what percentage of people self-report that they are earning more from the livelihood activity supported by USAID/OFDA than they are spending on expenses (i.e., inputs, rent, transport, fees).18 Sample households were asked about their livelihood or income generating activities for the past 12 months and whether they had net income, i.e., profit from these income generating activities. RISING II-supported livelihood activities included SILC and assets-based saving supports, Private Service Provider (PSP) activation, training on small scale business planning and linkage support to male- and female-headed households with the SILC and PSP groups and capacity building to farmers and pastoralists on value chain improvement, packaging, input supports and market linkages.19 1.4. How have household livelihoods changed as a result of the project? Farming/crop production and sales was the main livelihood activity in which most households participated at both baseline (78%) and endline (90%) (Figure 16; ANNEX VIII). The three most common income generating activities were: 1) farming/crop production and sales; 2) livestock/livestock product production and sales; and 3) handicrafts (at baseline) or non-agricultural businesses (at endline). Households engaging in crop and in livestock production/sales activities increased significantly from baseline to endline (p<0.05), while those engaging in handicrafts decreased (3.7 to 1.6% at baseline and endline, respectively; p<0.10) and those engaging in non-agricultural businesses increased (1.8 to 2.8% at baseline and endline, respectively; no statistical difference). 18 USAID/OFDA Proposal Guidelines, Economic Recovery & Market Systems PIRS, February 2018. P.1. 19 RISING II project PIRS. Page | 24 1.4.1. What types of income-generating activities, particularly for women, are available now that were not available before RISING II? According to quantitative data, female-headed households increased their engagement in crop and livestock production / sales, agricultural daily wage labor, and non-agricultural business (e.g., petty trade, small restaurants) (Figure 18; ANNEX VIII). Qualitative data suggest that while farming and livestock rearing are the primary livelihoods across all five states, local differences exist. For example, FGD participants in East Darfur indicated there has been an increase in people seeking jobs in mining and the military, over farming or livestock production, as well as for day labor. Day labor (as well as cash-for-work) was reported as particularly important for women and youth (e.g., West Darfur, Central Darfur). However, women’s ability to engage in wage labor may be tied – in some cases – to its proximity to their village. In East Darfur, female FGD participants noted, “If we get a chance for laboring on cash for work, we are ready for that, especially the young people, but unfortunately, we do not have this chance. Only [young] men can go out of the village seeking jobs [in] daily work in towns.” “Summer farming” was indicated as a “new” income-generating activity for women (e.g., West Darfur). For example, female FGs in Central Darfur indicated that women’s access to seeds (okra, groundnut, vegetables) has helped them become traders “like men” at market days and that overall women are much more engaged in markets than in the past. Young men and women, in particular, have difficulty securing stable income and need support with capital (as well as skills training) to start businesses, such as mobile charging or small commodities trading businesses. According to FGDs in East and West Darfur, new project-promoted income generating activities for women include working in the market (e.g., selling food, juice, “Ageena”, tea), farming and livestock (grazing), value added processing (e.g., pressing groundnut oil or making paste; making and selling porridge from sorghum), as well as daily work. According to a FGD with mothers in West Darfur, project inventions have “cured unemployment”, which led to an increase in household income and fostered “joint work” within the household. 1.4.2. How has household income changed as a result of the project? Figure 5: Households Reporting Net Income/Profit, by Survey Rounds Statistical significance for the difference between endline and baseline are denoted as: *p<0.10; **p<0.05; and ***p<0.01. No stars signify no statistically significant difference. Overall, between baseline and endline the percentage of households self-reporting any net income (i.e., did you have any net income/profit?) decreased from 72.9% to 67.1%, although the difference was not statistically significant (Figure 5). The project did not achieve the EOP target of 92.9% of households reporting net income, or profit, from their livelihood activities. Page | 25 In particular, significantly fewer female-headed households reported a profit at endline (58.9%) than baseline (72.6%). Households reporting net profit decreased for the three main income generating activities at baseline (i.e., farming, livestock, handicrafts), although the difference is statistically significant for crop production/sales only (Figure 19; ANNEX VIII). At endline, one-third (34.1%) of RISING II project participant households responded that the change in their overall household income involved an increase over the 12 months prior to the survey (Figure 20; ANNEX VIII), which was a significant change from baseline (18.6%). This is consistent with a reduction in the number of households at endline reporting their income had decreased in the last 12 months; fewer households reported a decrease in income at endline (48.5%) than baseline (67.4%). There was unanimous agreement among all FGDs and KIIs that one of the project’s biggest impacts was on household income, primarily as a result of increased production. In particular, FGD participants across all five states felt that farmers had strengthened their income-generating capacity “because of crop production and livelihood skills gained in SILC groups.” Savings/SILCs, together with training/ extension services, access to inputs, information, transport, market access, and new skills have improved women’s ability to generate income, according to both men and women FGD participants in West Darfur. According to lead farmers in East Darfur, “With increased income, people are able to buy a horse or donkey cart.” FGs in other states, such as Central Darfur, indicated that increased household income now provided the funds for school fees and that their children have better access to schools and an education. Many families are now also able to purchase livestock, such as goats and donkeys, including for ploughing and transport purposes. In a number of areas, FGs and KIs noted that the production year had been extended as a result of RISING II activities; some people indicated they were now able to cultivate during two growing seasons (e.g., winter and summer) whereas they had previously only cultivated during one season. In West Darfur, women FGD participants indicated they were now able to grow vegetable gardens, which they had not been able to do prior to RISING II. They attributed this primarily to the provision of seeds by the project, including vegetables (e.g., tomatoes, watercress, okra, radish, watermelon, eggplant). Prior to RISING II, most did not have access to such seeds. In other cases, people felt they were able to increase their sales of cash crops, such as groundnut, which might be sold at higher prices than other crops.. Again, it is noted that much of the qualitative data presented above seems to contradict the quantitative results presented. For example, income clearly did not increase over time (e.g., Figure 5, Figure 19, Figure 20) yet FGD participants felt that their income had improved as a result of the project. Such a discrepancy could result from differences in the nature of quantitatively and qualitatively collected data. Quantitative results represent the average across all sampled households and qualitative results represent the opinions of the individuals who provided them and may not be generally applicable across their community or beyond. Though they may “differ”, both types of data are important to consider when interpreting results. Readers should also recall that over the course of the project, Sudan experienced tremendous inflation, price fluctuations, and devaluation of its currency, all of which would likely have a negative effect on household income and profit, even if productivity or volume of sales increased. 1.4.3. How has agricultural production/productivity changed as a result of the project? As part of the project’s activities to improve livelihoods, a number of improved agricultural practices/ technologies were promoted to increase household access to food through better utilization of their agricultural lands. The following three outcome indicators are used to measure this activity: 1. OUTCOME 2.1: Number of months of household food self-sufficiency as a result of improved agricultural production programming. 2. OUTCOME 2.1.1: Number of hectares under improved agricultural methods. 3. OUTCOME 2.1.2: % of households with access to sufficient seed to plant. Page | 26 Indicators were calculated for project participating households from farmers and pastoralist (agro￾pastoralist) categories who reported having access to farmland.20 Overall, household access to farmland showed a small but statistically significant decrease over time, from 96% at baseline to 90% at endline (Figure 24; ANNEX VIII). This was consistent for all categories of project participants, except for pastoralist and MNF households, which did not change. Utilization of the farmland to which a household had access did not change from baseline to endline (Figure 25; ANNEX VIII) for most categories of household. There was, however, a significant increase in utilization for pastoralist households (79% at baseline and 91% at endline) and a significant decrease for MNF households (89% at baseline and 72% in endline). Overall, RISING II promoted 11 improved production practices.21 Figure 26 (ANNEX VIII) shows results for seven improved practices promoted by the project, six of which increased in use between baseline and endline. Only the use of Integrated Pest Management (IPM) remained essentially unchanged. When considering uptake of improved practices, it is important to also assess the number of practices/ technologies used by farmers, as they have greater impact when used in combination with each other. The percentage of households using at least three (Figure 27; ANNEX VIII) and at least six (Figure 28; ANNEX VIII) improved practices increased significantly from baseline to endline for all categories of participant household (p<0.10, p<0.05, or p<0.01), except in the case of pastoralists and MNF households using at least six practices, which increased but was not statistically significant (Figure 28; ANNEX VIII). Notably, over one￾half of households applied at least three types of improved practices/technologies at endline while the percentage of households using at least six types remained quite low (13%). Thus, there is significant room for continued improvement in the uptake of practices and technologies that lead to increased production. 1.4.4. How has the amount of land on which improved practices are used changed as a result of the project? Figure 6: Number of Hectares of Land Under Improved Agricultural Methods, by Survey Rounds Statistical significance for the difference between endline and baseline are denoted as: *p<0.10; **p<0.05; and ***p<0.01. No stars signify no statistically significant difference. Outcome 2.1.1 measures the total hectares of land on which at least one of the seven RISING II project￾promoted technologies or practices were applied in the last season or past 12 months.22 Farmers were first asked whether they applied any of the individual technologies/practices in the last season or past 12 months. If the farmer answered “yes,” then s/he was asked how much land they used for the specific technology/practice in the reporting year. This means that if a farmer applied promoted technologies/ 20 Access to farm land is defined as “the farmer can use the land for farming irrespective of his or her own land or leased land or gift land.” 21 Quality seeds, seedling preparation/planting, crop rotation, intercropping, minimum tillage, mulching, Integrated Pest Management (IPM), fertilizer tree plantation, soil and water conservation, vetiver/elephant grass plantation, plant density. 22 Calculation of Outcome 2.1.1 involves only the land on which any of the seven improved practices (see Figure 21) were used; regardless of how many practices were applied to the same piece of land. See the Feed the Future Indicator Handbook, October 2014. Page | 27 practices to his or her plot several times in the year (e.g., using the same piece of land to cultivate crops more than once per year), the total amount of land to which they applied the technologies/practices involved the area each time the improved practice/technology was applied. Since information for all seven practices was collected for the last season, it implies that the particular piece of land was used at least once in the reporting year for any improved practice. The sum of the hectares of land used for individual technologies for all seven practices was obtained from sampled households (i.e., those who had applied improved practices to their land). The total amount of land (in hectares) from the sample was extrapolated (weighted23) to the 13,501 total RISING II project participants (12,448 farmers and 1,053 agro-pastoralists). Figure 6 shows that the total hectares of land under improved agricultural production practices increased significantly from 2,758 hectares at baseline to 4,007 hectares at endline, well surpassing the EOP target of 3,034 hectares. This is consistent with the increased use of improved practices noted above (Figure 27 and Figure 28, ANNEX VIII). With one exception, there were more hectares under improved practices for all types of household categories, though the differences were not statistically significant for pastoralist and male-headed households. Returnee household was the only household category for which the hectares decreased between baseline and endline, though the difference was not statistically significant. Readers should recall that this result suggests that farmers were using improved practices on the same piece of land more than once over the course of the year, which is in line with increased ability to cultivate during both the summer and winter growing seasons. According to FGDs, a key result of the RISING II project was that many families have increased the number and/or size of their cultivated plots and have experienced an increase in crop production through implementation of various new farming techniques learned through trainings conducted by the project. In Central Darfur, cultivating fodder for livestock was an important new practice promoted through the project; seeds were provided at seed fairs by RISING II. In West Darfur, it was reported that “improved seeds were distributed and planted close to communities to improve pasture and water sources.” In targeted areas of North Darfur, FGD participants indicated that the RISING II project introduced “many” project participant households to cultivating groundnuts and sesame for the first time. For one sesame farmer, very high productivity led to enough income for him to purchase 23 goats. According to another farmer, “Farmers have adopted the new techniques and there is apparent impact on income.” As much as anything, increased income will facilitate longer-term sustainability of these practices, according to FGD participants and government/project partner KIs as well. There was widespread agreement by FGs and KIs that agricultural inputs (e.g., seeds, tools) provided to farmers by the RISING II project enabled them to increase their production, primarily by increasing their use of improved practices (e.g., more than one practice at a time on the same piece of land, more than one cultivation with improved practices per year), but also through well-trained and accessible agricultural extension workers. Noted one farmer in East Darfur, “These are good methods and techniques that farmers will continue to use.” In West Darfur, household training on the nutritional value of crops, use of improved seeds, storage, crop rotations, and cultivating in more than one season have benefitted agricultural production in the area. According to one FGD participant, “People like the new cultivation and want more training and improved seeds.” According to FGDs in West Darfur, cultivation was traditionally done by hand previously, on small plots with specific varieties such as corn and millet in the summer, and for personal consumption only (i.e., not for selling). In the past, for example, buckets were used to draw water from the “haffir” but now, as a result of the RISING II project, farmers use a generator to draw water for agriculture. As a result, this has led to increased cultivated area and corn production, and households have begun selling products in the market, such as new varieties of corn (Tabit, Wad Ahmed) and other crops. 23 Participant-Based Survey Sampling Guide for Feed the Future Annual Monitoring Indicators, Diana Maria Stukel, PhD, September 2018 Page | 28 In winter, people are able to grow tomatoes, watercress, eggplant, cucumber, root, beet, radish, potato, watermelon, and cantaloupe. The productivity of an acre has increased from 2 to 8 shawwal, on average, at least according to one KI. Both the ability to sell surplus crops (and in turn purchase inputs) and store grain in warehouses contributes to the sustainability of the project, according to FGD participants. KIIs in North Darfur suggest the same; improved agricultural techniques led to large increases in production. From a demonstration plot of 1 mukhama, sesame production (measured by sacks) increased by more than 162% and as much as 950% for groundnuts, according to an agricultural extension worker. Likewise in West Darfur, where one feddan typically yielded approximately two sacks, not only are households now cultivating an average of three feddans, but they are producing 7-8 sacks per feddan, according to a KI. This, in turn, has contributed to local markets with better and more diverse products. As previously seen, qualitative insights do not always align with quantitative results. In some cases this may result from a perception by participants in qualitative FGDs or KIIs that they must provide “good” opinions of projects. While possible, no evidence exists to suggest this is the case here. In the absence of relevant data, such discrepancies should be viewed as differences in the average household response to questions in the household quantitative survey and the opinions of a very small subset of sample households recruited for FGDs and KIIs. Further study may be needed to get more consistent data between quantitative and qualitative sources. 1.4.5. How has household access to sufficient seed changed as a result of the project? The indicator for Outcome 2.1.2 measures the uninterrupted and sufficient supply of seeds for RISING II project participants. Access to sufficient seed means a household either has seed in hand (own stocks) or has the ability or means to get it elsewhere (e.g., from neighbors, markets, agro-dealers) in time to plant, according to the crop calendar. This is a general measure of seed security at the household level and takes into account the full range of channels – including receipt of relief seed – through which farmers procure seeds to meet their household needs. Seeds distributed by the RISING II project included millet, sorghum, groundnuts, and various vegetables. Seed distributions occurred through project-supported local vendors and DiNER Fairs and involved a voucher system. Respondents were first asked which project-promoted crops and vegetables they produced in the past 12 months, followed by whether the seed/seedling supply was adequate (i.e., more or the same) for those individual crops and vegetables. Overall, more households reported they had access to sufficient seed supplies over the 12 months prior to the endline (55%) than at baseline (33.1%) (Figure 7). This represents an EOP achievement rate of 95% for the logframe indicator Outcome 2.1.2. All categories of project participant households, except MNF households, reported statistically significant increases in their access to sufficient seed supplies at endline. Figure 7: Households with Access to Sufficient Seeds for Planting, by Survey Rounds Statistical significance for the difference between endline and baseline are denoted as: *p<0.10; **p<0.05; and ***p<0.01. No stars signify no statistically significant difference. Page | 29 Throughout the RISING II operational area, a lack of access to quality seed of improved varieties – and other inputs – is a key challenge. Traders in East Darfur, for example, reported lack of access to a diversity of crop seeds as their primary challenge related to procuring inputs for farmers. A female farmer in Central Darfur said, “Through [the] NCA/RISING project we have access to quality and improved seeds and tools for both rainy and winter seasons through seed fairs conducted many times in our community.” According to women FGD participants in both East and West Darfur, access to improved seed (as well as inputs and information) has resulted in increased productivity and income. A female farmer in Central Darfur observed, “The vegetable seeds we received from [RISING II] has helped us to plant various crops like onion, tomatoes, okra, watermelon, radish, and other vegetables. We have good harvest for both summer crops like millet, sorghum, and groundnut for both the last two years. Many families here have their income increased and some even are able to buy new animals like goats and donkeys for ploughing and transport.” According to qualitative data, project beneficiaries generally felt that training in post-harvest practices has also increased the effectiveness of the project. Farmers in East Darfur suggested that female-headed households and returnees have not enjoyed the same benefits from agricultural services (and SILCs) as others in the community “because they cannot do what the other[s] can do.” For example, women may be constrained in leaving their villages (e.g., to purchase seed in a larger town) or from conversing with a male extension agent without a male escort (e.g., male family member). Various cultural or religious constraints can limit women’s access to services, information, and other assets (e.g., financial). Returnee households are often marginalized and their access to services and other resources limited. According to other qualitative data, returnee communities often lack access to seeds, tools, and knowledge on good farming techniques. 1.4.6. How has household food self-sufficiency changed as a result of the project? The success of the Outcome 2.1 indicator “Number of months of HH food self-sufficiency as a result of improved agricultural production programming” depends on improvements in the indicators for Outcome 2.1.1 and Outcome 2.1.2. Food self-sufficiency is defined as physical and economic access to adequate food for all household members. The number of months of food self-sufficiency is an estimate of a households’ ability to produce, purchase, store, or otherwise maintain a level of food consumption, without external The challenges of …groundnut and sorghum seeds, agricultural tools, capacity…., and social cohesion has changed after the CRS/RISING project provided …seeds to the community. -FGDs with M/F farmers and community leaders, East Darfur Figure 8: Number of Months of Household Food Self-Sufficiency, by Survey Rounds Statistical significance for the difference between endline and baseline are denoted as: *p<0.10; **p<0.05; and ***p<0.01. No stars signify no statistically significant difference. Page | 30 assistance or inputs, such that members of the household are not going hungry that is achieved with a degree of plausible attribution to USAID/OFDA-funded activities for improving food access. Sample households were asked how many months they expected their household to have adequate food from consuming or selling their own crop and livestock production (i.e., without support from others). The indicator was computed from the self-reported number of months that respondents believed they would be food self-sufficient. Figure 8 shows that the overall average number of months of household food self-sufficiency increased by one month between baseline (4.6 months) and endline (5.5 months). Although this represents a statistically significant increase, it falls slightly short of the EOP target of 5.9% (i.e., 95% achievement). With the exception of pastoralist households, all other types of households showed improvement in months of food self-sufficiency between baseline and endline, though the differences were not significant for host households. While qualitative discussions did not specifically reveal information about improvements in the number of months of food self-sufficiency per se, many FGD and KI participants reported increased production and household income resulting from project activities. Both of these changes would augment availability of food for household consumption over the course of a year. In particular, the project supported the most vulnerable households, providing them with basic farming inputs, seeds, and training to help them cultivate more crop area using improved farming techniques that result in increased harvests. Despite a lack of rain and/or flooding in some areas, many households benefited by providing more – and greater diversity of – foods for their consumption. Introducing agricultural training and providing inputs and seeds have improved production (for sale and consumption) and, in some areas, specifically for youth and women, who often have lower levels of crop production. Most households produce most of what they consume; approximately three-fourths of participant households at both baseline and endline reported their main source of food was from their own agricultural production (Figure 29; ANNEX VIII). Purchasing foods with cash from household income was the second most common source of food. All other sources were used by less than 10 percent of households; including food aid or cash transfers from humanitarian assistance initiatives. 1.5. How has access to markets, particularly for women, changed as a result of the project? There were no statistically significant changes in accessing markets within 5 km between baseline and endline (Figure 30; ANNEX VIII) for any type of household, with the exception of households in host communities, which improved their access from 29.7% at baseline to 42.2% at endline. However, there is an overall trend in improvement between baseline and endline. Project activities facilitated better linkages between households and local traders / markets, which allowed households better access to agricultural inputs, for example, through a voucher system. Perspectives from FGDs and KIIs, particularly with women, suggest that women have good access to markets generally, though it can be difficult during the rainy season. A trader in one East Darfur community indicated that women’s ability to access markets has not fundamentally changed, but they are “forced” by circumstances “to contribute more [to] household income to cover daily needs: women cannot stay at home as previously.” At the same time, many FGD participants (male and female) felt that better access to markets for women would augment their ability to earn income. According to interviewees in North Darfur (and elsewhere), markets could be improved by providing shelter and security, providing/improving water sources, latrines for women, and better organizing the market space. Qualitative data from FGDs indicates people’s access to markets was limited by COVID-19 restrictions/ market closures and transport costs/availability, particularly for women, youth and the elderly. Youth, people with disabilities, and the elderly need additional support with IGAs, including business skills training, savings/credit, and transport, for example, to help them engage in market activities. Traders in West Darfur indicated the pandemic – and efforts to curb its spread – also had a negative impact on daily workers. Other FGs felt that many price increases were attributable to COVID-19. Page | 31 1.5.1.Have there been any changes in availability of productive inputs, food, or other necessities as a result of the project? Across the five states, people noted the high and still fluctuating prices of many goods (e.g., inputs, fuel, food, clothes, medicine), which undermines their purchasing power regardless of increased income. Traders in East Darfur indicated “Prices can change each week or every day sometimes”, sometimes with “huge” weekly price increases. In West Darfur, FGD participants noted improved availability of food commodities at the village level, and at competitive prices, as well as the emergence of new types of foods that were not present in the past, including watercress, roots, leafy vegetables, radishes, melons, and watermelon. 1.5.2.Has the extent to which the local market operates in cash or vouchers changed as a result of the project? Unfortunately, qualitative insights on transfer modality are mostly absent from the qualitative data. A few FGs in West Darfur suggest that the vouchers provided by CRS were based on the economic and health status of households (including targeting PLWs and people with disabilities) and that food transfers are preferred over cash because “food transfers better fill the food gap” and “cash is used for other things.” One KI in East Darfur indicated that the government did not allow cash transfers and suggested that future projects negotiate with the government to introduce electronic vouchers, as traders did not appear to like the paper vouchers. 1.6. How has access to loans and savings changed as a result of the project? In West Darfur, participants noted that loans were difficult to access due to people’s inability to repay, coupled with a general lack of financial capital and limited supply of inputs. Large traders may still be able to provide loans to people for cultivation, which they repay with groundnuts or sorghum during/after harvest time, but this has become hard for farmers, who often experience crop failure due to water shortages. Nonetheless, many FGD participants (e.g., in Central Darfur), felt that SILC activities have contributed much to people’s ability to save and use money for such productive purposes and facilitates their ability to engage in markets. For example, SILC members are able to get loans (e.g., to purchase agricultural inputs) and repay them after harvest. In one area of West Darfur, perceptions from qualitative interviews are that the project was able to improve livelihood opportunities for returnees by increasing the area they were able to cultivate (e.g., by providing seeds and training), which increased production and created market surpluses. As a result, incomes increased and people became better able to contribute to social funds (e.g., SILCs). These social funds, in turn, contributed to livelihood improvements by creating resources for loans that were invested in trade and achieved some profit. According to both traders and farmers in East Darfur, seeking loans/credit from microfinance institutions or banks to support livelihood and business activities is a difficult process (e.g., too much paper work and conditions). FGD and KII participants indicated that loans from group-based savings and lending programs (e.g., SILC) are used to buy crops and then sell when the prices are high, increasing their earnings to pay school fees, health expenses and cover household needs. As noted by one FGD participant in West Darfur, “Income increased from market sales, beggary disappeared and women were better able to support their families.” 6. Access to Natural Resources Outcomes Information was collected in both baseline and endline surveys on the availability of natural resources in the RISING II project communities and whether project participating households have access to those resources year-round to measure point estimates of the indicator “Outcome 1.1: % of HHs with equitable and uninterrupted access to (at least three) shared natural resources (water, pastures, etc.) throughout the year.” The Outcome 1.1 indicator refers to every household in a community having equal choice to use, withdraw and manage the available shared or open natural resources and facilities with uninterrupted use Page | 32 by any individual or group.24 The RISING II project considers the following natural resources for computing this indicator: i. Open or communally established water source (water from the stream/river, borehole, check dam, watershed). ii. Open or shared pastureland/grazing land for agriculture production and livestock rearing. iii. Access to sharecropping/lease and procuring agriculture land. iv. Using forest products for livelihood and livestock. v. Participation in natural resources inputs supported by the Government and others (e.g., material support, community management committees). A household with year-round (both dry and other seasons) access to at least three of the five natural resources is required for this indicator. First, information was collected on whether the natural resources were available in respondents’ communities. Second, respondents were asked whether their household had access to those natural resources year-round without any interruptions (by any individual or groups). Additionally, information was collected on household use of some improved natural resources for “Outcome 1.3.1: % of HHs using improved natural resources” (see below). 1.7. How has household/community access to natural resources changed as a result of the project? Availability of open/community-shared forest/forest products, pasture/grazing land, and government input support (e.g., organizing community management communities, materials for infrastructure construction / rehabilitation activities) increased between baseline and endline (Figure 21; ANNEX VIII). That is, more households at endline than baseline perceived forest, pasture, and other resources were available to the community. In contrast, fewer households reported that crop land (e.g., sharecropping, lease) was available in the community at endline (59.4%) than at baseline (72.2%). Households reporting they had year-round access to the natural resources available in their communities showed a similar pattern, with statistically significant increases for forest/forest products, pasture/grazing, and government/other inputs, and decrease in access to land for cultivation (Figure 22; ANNEX VIII). Figure 9 shows year-round access to at least three types of available natural resources increased significantly from 45.9% at baseline to 61.3% at endline, with an achievement of 93% of the EOP target (65.9%). Figure 9: Household Year-round Access to at Least Three Natural Resources, by Survey Rounds Statistical significance for the difference between endline and baseline are denoted as: *p<0.10; **p<0.05; and ***p<0.01. No stars signify no statistically significant difference. 1.7.1.Has access changed equally for different types of households? More project participant households in host communities and pastoralist households reported year-round access to at least three natural resources at endline than baseline (Figure 9); there was no significant 24 Taadoud II M&E Guideline/RISING II PIRS Page | 33 difference for households in returnee households, though the trend was upward. Households headed by males or by M&F also improved their year-round access to at least three types of natural resources. There were no statistically significant changes for any other type of household. 1.8. How has management of community natural resources (including conflict mitigation) changed as a result of the project? The Outcome 1.3.1 indicator measures the percentage of households using/applying at least two of the RISING II project-promoted improved natural resources practices in the past 12 months.25 The improved natural resources management (NRM) practices used to calculate this indicator are: i. Crop culture: seedling production and transplantation, intercropping, mulching, etc. ii. Permaculture practices (soil and water conservation): erosion prevention and control with mechanical and biological methods such as gabions, vetiver grass, elephant grass, riverbank management. iii. Agroforestry: integrating fodder and fruit tree production, other nitrogen fixing trees. iv. Natural regeneration such as Farmer Managed Natural Regeneration (FMNR): regeneration of natural landscapes (reseeding), management of forest plantations (planting, cutting, pruning), sustainable harvesting of forest products (wood, firewood, charcoal production). v. Water management: non-irrigation-based water management such as water harvesting, improved mechanical and physical land preparation such as building of swales or ditches, sub surface dams and watershed management. vi. Climate mitigation: technologies selected because they minimize carbon emissions relative to alternative technologies. Examples include low- or no-till practices and solar-powered irrigation. Survey respondents were asked whether they used any of the NRM practices promoted by the project over the 12 months prior to the survey, which were then grouped into the six categories described above. Figure 23 (ANNEX VIII) shows significant improvement in use of NRM practices between baseline and endline for two of the six categories. Use of improved crop cultural practices increased from 24.4% at baseline to 52.6% at endline and use of permaculture practices also doubled (17.2% at baseline to 34.0% at endline). Although climate mitigation, agroforestry, and water management practices all showed upward trends, none of the changes were statistically significant. Use of FNMR practices decreased slightly between baseline and endline, though again, this difference was not significant. Figure 10: Households Using at Least Two Improved NRM Practices, by Survey Rounds Statistical significance for the difference between endline and baseline are denoted as: *p<0.10; **p<0.05; and ***p<0.01. No stars signify no statistically significant difference. Figure 10 shows households reporting they used at least two of the six categories of NRM practices (Outcome 1.3.1). Overall, more households used at least two of the NRM practices at endline (52.5%) than 25 RISING II project PIRS. Page | 34 at baseline (33.8%). This represents an achievement of 82% of the EOP target (63.8%). Significant increases were reported for all other category of project participant household, with the greatest increases for pastoralist and Male with No Adult Female (MNF) households. Insights from FGD participants and KIIs with project and government partners suggest that most communities experienced better management of - and access to - their natural resources as a result of the RISING II project. There was widespread agreement in FGDs and from KIIs that the RISING II project provided communities with good training on natural resource management and disaster risk reduction (DRR), both of which help them better manage their natural resources and deal with risks from hazards, such as drought. The project targeted different groups of users, including farmers, pastoralists, and nomads, and included males, females, youth, and elders. With support from RISING II, communities formed and were trained in community action groups (CAGs) for natural resource management, conflict resolution (e.g., ajawid) for peace building, DRR, and development of community action plans. According to a KI in West Darfur, training involved how to manage and preserve natural resources and the environment as well as the concept of “equal access”; natural resources belong to everyone and “must be divided among them”. Key informants in North and South Darfur also indicated the project supported establishment of and training on early warning systems (EWS) in communities. According to qualitative data from West Darfur, training on natural resources management allowed for better access and use by different groups of people, which increased production and filled markets with agricultural and dairy products. Overall, this improved the standard of living for everyone. However, many agreed that, generally, access to farmland is more of a challenge for returnees than for other farmers or pastoralists. In most areas, natural resource constraints were one of the main causes of conflicts between farmers and pastoralists. For example, in West Darfur, FGD participants indicated that the entry of livestock into agricultural lands and the destruction of crops led farmers to occupy lands and prevented access by herders. One community in West Darfur recalled how the RISING II project helped them form a committee to plan and organize pastures, opening grazing paths (i.e., migratory routes), and building earth dams to capture water. The committee also developed a “charter” to settle disputes over natural resources that was inclusive of different critical groups within the community (e.g., the elderly, women, returnees, pastoralists). These committees, or community-based conflict resolution mechanisms (CB-CRM), became a model for the government in terms of settling disputes, which resulted in additional training for the committees in terms of keeping records on the issues and proposed solutions to different types of conflict. In East Darfur, FGD participants in one locality felt that control and management of available farmland and pasture has shifted from the CAGs to the Ajaweeds. According to them, conflicts between pastoralists and farmers, and between host communities and returnees, have declined over the last two years because of conflict resolution mechanisms established by the RISING II project. In at least one area, reduced conflict and strengthened peacekeeping resulted in the settlement of several nomadic groups due to the availability of water and pastures. Although most FGDs and KIIs indicated that no one is excluded from participating in conflict resolution mechanisms, women do not participate because of “cultural norms”. Water remains a challenge, including for both people and livestock. Women are especially affected by the need to venture to more distant sources of water, exposing them to potential dangers as well as adding to their time and labor burden. FGD Participants in one FGD in Central Darfur suggested the need to develop water sources near communities that currently lack them and providing training on how to deal with/prevent conflicts. Project training in conflict management also contributed to building social cohesion and strengthened social relationships among users of local natural resources (especially between farmers and pastoralists) both within and between communities. A KI in Central Darfur recalled how the RISING II project helped diffuse conflict between pastoralists and farmers through community capacity building in conflict management, which also helped build social cohesion between the two groups and within the community Page | 35 more generally. In South Darfur, pastoralists and farmers, united in their understanding of the importance of natural resources for their survival, banned the unorganized cutting of trees for the benefit of everyone. 7. Health and Nutrition Outcomes Quantitative data related to water, sanitation, and hygiene (WASH) practices was collected for the RISING II logframe indicator “Outcome 3.1.2: percentage of people targeted by the hygiene promotion program who know at least three (3) of the five (5) critical times to wash hands.” Data for measuring hand-washing practices was collected for respondents (disaggregated by gender); data for measuring access to water and sanitation was collected at the household level. 1.9. How has community access to safe water and sanitation facilities changed as a result of the project? Access to safe drinking water and hygienic sanitation facilities are important to ensure improved hygiene practices. Overall, the percentage of households with improved drinking water sources26 decreased slightly from 64.4% at baseline to 57.4% at endline, though the difference is not statistically significant (Figure 31; ANNEX VIII). This pattern was the same for the majority of household types (i.e., farmers, pastoralists, M&F, and both female- and male-headed households). However, statistically fewer returnee and FNM households had access to improved drinking water sources at endline than at baseline. Overall, the percentage of households using unsafe drinking water decreased between baseline and endline (Figure 32; ANNEX VIII); approximately one-third (30.6%) of households reported that their drinking water was unsafe at baseline while less than one-fourth (22.3%) did at endline. There was a noticeable reduction in using unsafe water sources for drinking across all categories of household but was a statistically significant change only for host, farmer, MNF, and male-headed households. Among those households reporting their drinking water was from an unsafe source at endline, three-fourths of them (75.1%) reported doing nothing to make it safe, which was slightly lower than at baseline (Figure 33; ANNEX VIII). The most common way households treated water to make it safe for drinking was through filtration. There was a statistically significant increase in use of solar disinfection to treat unsafe water between baseline and endline. RISING II considers ring-slab/offset latrines (water seal), pit latrines (covered), and septic latrines as improved toilet facilities. Figure 34 (ANNEX VIII) shows that household use of improved toilet facilities improved at endline. Overall, significantly more households at endline (32.6%) than at baseline (19%) reported they had access to improved toilet facilities. As indicated in Figure 34, more host, farmer, FNM, M&F, and both male- and female-headed households at endline report access to improved toilet facilities than at baseline. Returnee and MNF households reported improvements, though the differences are not statistically significant. Access decreased by approximately one-third for pastoralist households but, again, the difference is not statistically significant. More than one-half of all households had no toilet facility, which did not change between baseline and endline (Figure 35; ANNEX VIII). Of those households who have access, the most common type of latrine is a covered pit latrine, followed by uncovered pit latrines. Between the implementation of the BL and EL surveys, households appear to have shifted from using uncovered to covered pit latrines, a statistically significant change for both measures. That is, fewer households use uncovered – and more households used covered – pit latrines at endline than baseline. 1.10. How have household hand-washing behaviors changed as a result of the project? Outcome indicator 3.1.2 measures individual knowledge of hand washing practices at critical times for preventing the spread of pathogens along the fecal-oral cycle. Information for hand washing practices was collected from individual respondents (disaggregated by male and female). The five critical times to wash hands with soap or cleaning agents are defined as: 1) After defecation/using the toilet; 2) Before eating; 3) 26 Improved water sources include hand pumps, boreholes, piped water into the household compound, piped water in a public place/donkey, berkad (underground tank), improved hand-dug wells, and trucked-in water. Page | 36 After changing diapers or cleaning a child’s bottom; 4) Before preparing food; and 5) Before feeding an infant. Survey respondents were targeted by RISING II’s hygiene promotion activities and were asked about the most important times to wash one’s hands with soap or cleaning agents. The question included 10 occasions (five critical and five other times, to minimize bias in responses). Data analysis included only the five critical times defined above. Only respondents who demonstrated that they knew at least three of the five critical times to wash hands were included in the calculation of the value for this indicator. Figure 11: Awareness of at Least Three of the Five Critical Times to Wash Hands, by Survey Rounds Statistical significance for the difference between endline and baseline are denoted as: *p<0.10; **p<0.05; and ***p<0.01. No stars signify no statistically significant difference. Overall, most people were aware of the importance of washing hands before eating (Figure 36; ANNEX VIII), which showed a statistically significant increase between baseline (85.4%) and endline (91.8%). Awareness of the importance of hand washing before feeding children also improved between the two surveys (17.4% and 24% at baseline and endline, respectively; p<0.10). There were no statistically significant changes in awareness of the other critical times for washing hands. The percentage of respondents targeted by the hygiene promotion activities who are aware of at least three out of the five critical times for washing hands increased slightly from 31.1% at baseline to 35.6% at endline (Figure 11), which is not statistically significant. This falls far short of the EOP target of 51.1% for this logframe indicator (i.e., Outcome 3.1.2). Although awareness increased slightly for most types of households, there are no statistically significant differences between baseline and endline for any household type. When disaggregated by gender, more male respondents were aware of at least three of the five critical times for hand washing at baseline (20.8%) than at endline (31.4%) (Figure 37; ANNEX VIII). Among female respondents, there is no change between baseline and endline (Figure 38; ANNEX VIII). Interestingly, male respondents from female-headed households reported a significant increase in awareness between baseline and endline (Figure 37) and female respondents in male-headed households showed improvement but it was not statistically significant (Figure 38; ANNEX VIII). However, female respondents in female￾headed households show a decline in awareness of at least three critical times for hand washing (Figure 38; ANNEX VIII), which suggests a possible “shift” of females out of female-headed households between baseline and endline and into other household types, or at least out of the sampled RISING II project participating households. According to FGDs and KIIs across the five states, there is“much more” awareness and practice of washing hands at critical times, such as before eating, after defecation, etc. now than previously, including washing with soap when available and with ash if not. In East Darfur, both community health workers (CHW) and farmers indicated there had been “massive” awareness-raising campaigns by CRS and the Ministry of Health through radios and mobile phones, especially as part of the COVID-19 response. According to FGDs and KIIs in West Darfur, the state-promoted COVID-19 response included establishing hand washing Page | 37 stations in communities, which were supplied with soap for hand washing and chlorine for disinfection and water treatment. Although the qualitative data are seemingly more positive regarding handwashing awareness than quantitative results show (Figure 11), there is an overall positive trend in the quantitative results over time, even though there are not statistically significant differences. In West Darfur, community members were also sensitized through targeted community social mobilization with enhanced adherence to WASH guidelines. Until recently, water and sanitation facilities were either completely inadequate or non-existent in most areas. People were getting drinking water from unprotected sources of water and practicing open defecation, making it difficult to maintain good sanitation practices. Lack of water generally also makes hand washing – and other health practices – difficult. FGD participants in West Darfur indicated that the largest proportion of child deaths can be attributed to diarrhea, which in turn is mainly due to poor environmental sanitation, water and hygiene services. Children suffer from acute malnutrition, with 50% of these cases associated with frequent diarrhea or worm infestation. According to most KIIs with health￾related stakeholders, the public health and water program has achieved remarkable progress through training and the commitment of mothers to general hygiene, taking care of their children, boiling water before drinking it, and washing hands after defecation. In particular, project activities have contributed to a reduction in illness and common diseases (e.g., malaria, diarrhea) among children. 1.11. How has the nutritional status of children under five (CU5) changed as a result of the project? To measure Outcome Indicator 3.1: Proportion of children 6-23 months of age who receive foods from 4 or more food groups, mothers or caregivers of children 6-23 months of age were asked about the food or food groups fed to the child during the 24 hours prior to the survey (previous day). Information was collected for 19 foods or food groups, which were then reclassified into the following seven food groups according to the WHO/UNICEF IYCF indicator guidelines:27 1) Grains, roots and tubers; 2) Legumes and nuts; 3) Dairy products (milk, yogurt, cheese); 4) Flesh foods (meat, fish, poultry and liver/organ meats); 5) Eggs; 6) Vitamin-A rich fruits and vegetables; and 7) Other fruits and vegetables. Figure 12: Children 6-23 Months Consuming Four or More Food Groups During the Previous Day, by Survey Rounds Statistical significance for the difference between endline and baseline are denoted as: *p<0.10; **p<0.05; and ***p<0.01. No stars signify no statistically significant difference. 27 Indicators for assessing infant and young child feeding practices, Part 1, Definitions, Conclusions of a consensus meeting held 6–8 November 2007 in Washington, DC, USA. “Now women use soap when washing hands and when cooking food; [this] has become a habit among women, girls, and men. Most people wash their hands with soap. This has become common practice, particularly due to the mobilization of COVID-19 and preventive measures undertaken by the village.” -FGDs with female CHWs, Central Darfur Page | 38 Consumption by children 6-23 months of any amount of food from each of the food groups is sufficient to “count”; there is no minimum quantity in order to be included in the calculations except if an item is only used as a condiment. Children 6-23 months of age who received foods from four or more (≥4) food groups28 during the previous day were included in the computation of this indicator. Figure 12 shows that the percentage of children with diverse diets (i.e., consumed four or more out of seven food groups in the past 24 hours) increased from 19.3% at baseline to 25.3% at endline, though the difference is not statistically significant. The EOP target of 39.3% was not met, with only 64% achievement of EOP Target. Significant improvement in diverse diets for young children were reported for children from host and female-headed households. Overall, more young boys and girls ate diverse diets at endline than baseline, though the differences were not statistically significant. Across household types, girls tended to have slightly higher rates of consuming at least four of the seven food groups (Figure 40; ANNEX VIII) than boys (Figure 39; ANNEX VIII), with the exception of young girls in female-headed households. Boys in female-headed households showed statistically significant improvement in diet diversity between baseline (9.6%) and endline (31.6%), which is in stark contrast with diet diversity for young girls from female-headed households, which declined from 14.9% at baseline to 11.3% at endline. This pattern was reversed for young boys and girls from male￾headed households; girls did better in male-headed households than did boys. There were no qualitative data to help explain this pattern. At the same time, the sample sizes for this indicator are quite small and care should be taken in interpreting these results. Qualitative data suggest that food baskets provided by the RISING II project likely played a significant role in the better nutrition outcomes. Across all states, FGDs and KIIs suggest that there has been great improvement in the nutritional status of young children (CU5), due to eating more meals per day as well as eating more diverse foods (e.g., from vegetable production). According to one FGD participant in Central Darfur, “Here we grow different vegetables, which are good for our health.” In East Darfur, KIIs with CHWs suggest that referrals to feeding centers for malnourished CU5 have dropped from six to two per month in one sample community, and is, in their words, “one of the most critical impacts of the RISING II activity.” According to KIs in North Darfur, SAM cases dropped from 141 to 42 across the 16 targeted communities and MAM cases dropped from 334 to 152 during the first year of the project. CHWs in West Darfur also indicated that malnutrition had been the greatest challenge over the last 18 months but that the overall impact of the project was the disappearance of critical cases of malnutrition and increased awareness and interest in ensuring good nutrition for infants, young children, and PLWs. In North Darfur, KIs noted that one of the first interventions implemented involved MUAC screening and treatment of SAM cases, including a referral voucher for lentils, vegetable oil, and cereal (e.g., sorghum, millet). In East Darfur, KIs indicated that women now know how to use MUAC tapes, where to take their children if severely malnourished, and how many times during a day infants need to breastfeed. 1.12. How have attitudes about or practices in exclusive breastfeeding of babies for the first six months changed as a result of the project? To measure data for Outcome Indicator 3.1.1: Proportion of infants 0-5 months of age who are fed exclusively with breast milk, the WHO/UNICEF IYCF guidelines were used. Although child feeding also included breast feeding children 0-23 months, this indicator only used information for children aged 0-5 months. “Fed Exclusively” is defined for infants (0-5 months) that were fed breastmilk (including milk expressed from a wet nurse) and nothing else, not even water, in the previous day (past 24 hours).29 Infants were also counted if they had taken oral rehydration salts (ORS) and/or drops/syrups (vitamins, minerals, 28 “Breast milk” has been included on the list of number food groups and cut-off point has been changed from 4+ to 5+ in 2021 IYCF guideline. But both the RISING II baseline and endline analyses followed the previous guideline because the baseline indicator analysis was conducted using the previous guideline. The EOP target was also set based on the baseline analysis. 29 Indicators for assessing infant and young child feeding practices, Part 1, Definitions, Conclusions of a consensus meeting held 6–8 November 2007 in Washington, DC, USA Page | 39 medicines). Thus, children 0-5 months who were fed only breastmilk with the exception of vitamins/minerals/medicine in the past 24 hours (previous day) were counted toward this indicator. Figure 13: Children Aged 0-5 Months Exclusively Breastfed During the Previous Day (past 24 hours), by Survey Rounds Statistical significance for the difference between endline and baseline are denoted as: *p<0.10; **p<0.05; and ***p<0.01. No stars signify no statistically significant difference. Overall, there was no significant difference between baseline and endline in children 0-5 months exclusively breast fed in the past 24 hours, although the percentage increased from 45.7% at baseline to 50.7% at endline (Figure 13). This is true for infants from all categories of participant households, with the exception of infants from returnee households, which declined from 55.6% at baseline to 42.4% endline but was not statistically significant. The RISING II EOP target of 65.7% was not achieved (77% achievement of EOP Target) for Outcome 3.1.1. Boys tended to be exclusively breast fed more than girls at both baseline and endline (Figure 41 and Figure 42; ANNEX VIII), but both groups show only slight improvement (i.e., not statistically significant) between the two surveys. Across most types of participant household, there was an upward trend (i.e., not statistically significant) in exclusive breastfeeding for both infant boys and girls. However, exclusive breast feeding for both girls and boys in returnee households declined at endline, though again, the difference is not significant. Regardless of the lack of significance, which may simply be due to small sample sizes, the overall trends are positive for the three indicators reported in this section. CHWs reported that attitudes around breastfeeding have started to change, mainly among women and mothers who are now exclusively breastfeeding just after birth and for the first six months. While most mothers understand the importance of providing breastmilk, some still have not applied the practice. Some FGD participants suggested that cultural factors can limit a woman’s choice in breastfeeding (e.g., their husbands might decide otherwise). Overall, KIs in East Darfur indicated that women have improved knowledge about hygiene, hand washing, sanitation practices (use of toilets rather than open defecation), and the importance of nutritious and diverse foods. However, challenges remain, including many communities without any medical services or health centers, a lack of staff in existing centers, and limited success changing some behaviors. 8. Project Management Outcomes: OECD-DAC The RISING II endline evaluation collected qualitative data from project, partner, and government staff to assess the effectiveness of project management, systems, and processes established by the project, including key questions around relevance, effectiveness, coherence, impact, and sustainability. It should be noted that this section provides results from qualitative inquiry only and does not include findings from the quantitative household survey, which does not include any questions about relevance, effectiveness, coherence, impact, sustainability, or any other project management, systems or processes established by the project. Page | 40 As previously mentioned, qualitative data collection was interrupted both by the unexpected passing of the lead national qualitative consultant in Sudan, as well as a coup at the beginning of field work. The closure of state government and NGO offices meant that many key informant interviews with NGO partner and government staff were not possible. As a result, there is somewhat of a general lack of detail across some of the qualitative data relative to the management of and processes used in the RISING II project. Nonetheless, some general lessons learned can be gleaned. 1.13. Relevance Questions regarding relevance center on whether FGD participants and KIs felt the RISING II project was, in fact, implementing the right activities. Overall, qualitative insights suggest the RISING II project was highly relevant. 1.13.1. To what extent did project interventions address the most critical needs of project participants? It was generally agreed by partner stakeholders that the RISING II project addressed the most critical needs of its target groups; i) NRM and associated activities ensured equitable access to resources and helped prevent conflicts, ii) training on improved agricultural practices – along with provision of inputs – and income-generating activities improved household production and increased income, iii) SILC activities facilitated access to credit – as well as savings – for investing in livelihood opportunities, and iv) nutrition and WASH activities directly addressed malnutrition through increasing awareness and knowledge of mothers and PLW on the importance of health and nutrition-related issues, and improved nutritional status among young children. Even when results were not statistically significant, trends were in the right direction. 1.13.2. To what degree were interventions sensitive to the constraints faced by households in trying to make a living? The project was perceived by KIs to have been designed to accommodate various contextual factors within the implementation areas. For example, most KIs appreciated CRS using vouchers rather than cash (and adjusting their value) because of severe inflation and other macro-economic conditions outside the control of the project. Other KIs felt that using a catchment approach to implementation contributed to broader impact, for example, the entire community benefits from peace building activities in terms of accessing natural resources. The project design also strengthened social networks and cohesion through “groups”, including SILC groups and community-based committees (e.g., conflict resolution committees). One KI in East Darfur noted that nutrition activities focused on local food ingredients, which she considered to be quite fitting relative to the economic conditions and capacity of the targeted community, and stated, “Accessibility of the ingredients is for all and is socially acceptable.” 1.14. Effectiveness This section addresses the extent to which FGD participants and KIs felt the intervention achieved its objectives and results, including any differential results across groups (e.g., men, women, youth, pastoralists, farmers). From the perspective of FGD and KII participants, the RISING II project was highly effective. Though there is always room for improvement, there were very few complaints about its effectiveness. 1.14.1. What have been the effects of project interventions on households/communities? All key informants considered that the project had achieved most of its objectives around improved livelihoods, food security and nutrition outcomes, and community governance of natural resources. Qualitative results clearly suggest that people felt their crop production had increased, in part from use of improved seeds distributed through seed fairs and tools/improved practices; households were producing “90% of target communities are returnees who are, in fact, in real need for all types of services, not only livelihoods and agriculture but also health, nutrition, and water and sanitation. However, according to what I have seen at different communities that I visited, [the] RISING project provided them with the most important and practical farming techniques and knowledge besides providing them with seeds and tools necessary for cultivation and crop production.” -KII in Central Darfur Page | 41 their own quality seed on demo and seed multiplication farms; income had increased as a result of better production and links with markets; communities had improved their capacity to resolve local conflicts through community-based conflict mitigation mechanisms, which had also contributed to better social cohesion within communities; successful establishment of SILC groups had a knock-on effect in communities by motivating others to start their own SILC groups even without project support; and households genuinely felt that malnutrition among young children had decreased and that families were consuming more diverse and nutritious foods as a result of the project. For other FGD and KI participants, the project had the most effect on household production and income. For example, farmers in East Darfur felt that agricultural activities (e.g., seeds, trainings, demo plots) and associated improvements in nutrition that resulted, have contributed most to the success of the project. Specifically, they felt the project greatly improved people’s ability to feed themselves; households produced more because they were cultivating more land, using inputs, and applying improved practices and techniques that resulted in higher production and productivity (e.g., seed storage). In turn, there was more food for household consumption and for selling in the market, which generated income for other household needs as well. No KIs felt the project lacked positive effects on any outcomes. Although the quantitative data show no statistically significant improvements for several outcome indicators, most results show positive trends. 1.14.2. Were different groups affected differently? For the most part, FGD participants and KIs felt that all project-targeted groups benefitted equally from project activities. Farmers and pastoralists felt they had improved their livelihoods through inputs and trainings provided to them by the RISING project. Vendors and traders who participated in seeds fairs felt they had earned good profits and their capital and businesses increased. According to some KIIs, project interventions encouraged many IDPs to return to their villages to begin rebuilding their lives. Mothers and other caregivers learned – and acknowledged the importance of – exclusive breastfeeding, hand washing, and consumption of diverse and nutritious foods for their children. One KI in East Darfur felt that farmers and children benefitted more than other groups because of “the personal benefit (i.e., incentive) associated with these two interventions.” Even where differences might occur, it is extremely uncommon for FGD participants or KIs to report that certain groups are differentially affected – either in a positive or negative sense. Overall, the qualitative data are simply too limited to adequately address this question. Analysis of quantitative data did not include comparisons between different types of groups (e.g., comparison of female- and male-headed households). 1.14.3. Which interventions, or combination of activities, have contributed most to the success of the project? Effectiveness also considers which, if any, interventions have the most effect on outcomes, at least according to FGD participants and KIIs. Broadly, most FGs and KIs perceive the project as having three strategic objectives: (i) agriculture and food security, (ii) natural resource management and (iii) DRR/peace￾building, and nutrition assistance. Overall, the outcomes on which project interventions were perceived by FGD participants and KIIs as having the most impact varied somewhat on the background, or area of expertise, of the respondent. For example, some stakeholders with a nutritional focus felt that the combination of food security and nutrition activities contributed most to the success of the project (e.g., success rated as 80% by project staff in North Darfur), though they also felt it could have been more successful if the project had better adapted to changes in the economic and environmental contexts (e.g., inflation, pests, low rainfall). According to others, it was the food basket distribution (including during critical food gaps from August through November) that integrated food security/livelihoods with nutrition, which in turn reduced rates of severe- and moderate-acute malnutrition that had the greatest positive effect. Other stakeholders perceived the improved agricultural production – and its effect on income – to have contributed most to the project’s successes, including more diverse diets and better health/nutrition outcomes. Alternatively, Page | 42 one KI in West Darfur suggested that, “The biggest impact of the project is the training of group elders who direct people towards the right actions that they must do, such as caring for forests, preventing excessive logging, taking care of water sources, as well as community peace between the different groups, which enhanced the achievement of goals.” 1.14.4. What have been the challenges to project implementation? In most project implementation areas, KIs indicated there were challenges to the effective (and efficient) implementation of project activities. Though the specific issues raised by KIs varied somewhat across states, delays in project implementation were widely noted and generally attributed to insufficient staff (e.g., engagement with both RISING II and Taadoud at the same time), political instability (e.g., public demonstrations), fuel shortages, market inflation, budget issues/revisions, and the COVID-19 pandemic. In Eastern Darfur, KIs noted that implementation of the RISING II project coincided with a change in local government, including the new administration “insisting on changing even the government’s field agents.” In some areas, termination – and replacement – of existing agents delayed implementation. Most partners also experienced challenges specifically related to transfers/vouchers. Qualitative data suggests that people’s preference for food versus cash transfers varies. In some cases, food is preferred as it “fills food gaps”. Others prefer the flexibility afforded by cash; households can prioritize how cash is used for household needs. Personal preferences notwithstanding, cash may be most appropriate when markets function well and inflation is low, whereas food transfers may be more appropriate when inflation is high, or prices fluctuate greatly, and markets do not function well. At the very least, some KIs indicated that CRS adjusted the value of their transfers in response to the unstable economic context (e.g., inflation, currency devaluation) in which it operated. Others suggested the project needs to be more flexible in order to adjust when inflation is an issue. Given the limited details (e.g., lack of follow-up questioning) in most interviews, it is not possible to determine whether this slight discrepancy is a result of i) individual KIs simply being unaware of changes in the value of transfers made by CRS, ii) misunderstanding by some KIs, or iii) confusion (e.g., translation error) with exchange rates (see below). Qualitative data also suggest that a lack of available cash in central banks at the beginning of project implementation resulted in delays. According to Kis with state-level project staff, several processes internal to CRS also resulted in delays, including the time needed for CRS to deal with rate changes regarding the cost of vehicle rentals and the long process required to print vouchers, which affected timely implementation of seed fairs, and in turn, the timeliness of planting. A KI in Central Darfur suggested the need for timely printing and provision of vouchers to consortium member field offices in order to allow seed fairs in suitable time for farming. Several KIs with project staff in different states noted that delays in budget transfers (e.g., between national and state offices) resulted in less time to actually implement activities. Delays – for whatever reason – can be particularly detrimental to projects of shorter timeframes, such as the 25-month LOA for RISING II. Lack of water was also considered a real challenge to project implementation by some FGD and KII participants. The winter season, when water is scarce, is the main season for growing vegetables. As one KI put it, “We cannot talk about kitchen gardens due to a shortage of water.” Increased access to water was considered a top priority by KIIs with project staff in East Darfur. In particular, they felt activities could focus more on rainwater harvesting, irrigation, and incorporating fish farming and vegetable growing into the project’s natural resource management activities. Strong recommendations were made to include constructing boreholes and haffirs in future project cycles to support vegetable gardening as well as strengthened activities for provision of clean water for health objectives. In East Darfur, staffing challenges apparently arose for project staff as a result of the departure of a consortium member (UMCOR) from the Taadoud project. Given CRS’s shared responsibilities and resources (e.g., staff) between the two projects, some staff previously with UMCOR where integrated into the RISING II project. According to a KI in East Darfur, the two organizations differed in their standards, staff capacity/quality, policies/procedures, etc. and “it took some time for CRS to reach stability.” Page | 43 1.14.5. What improvements can be made to address the challenges / improve the project? People’s perspectives on how the project can be improved varied widely, in part based on their areas of expertise and/or engagement with RISING II. For example, health / nutrition stakeholders suggest establishing health centers near villages. Other stakeholders recommend increasing the number of targeted households, especially among the poorest. Overall, however, qualitative data suggest high degrees of praise for the project design rather than criticisms. Some challenges should have been predicted, including inaccessible roads during the rainy season that hindered access to more remote communities and the likelihood of increased insecurity/ physical safety issues. Some suggestions taken from FGD participants and KIs for consideration in future programming are provided. ● Develop an early procurement and rapid response plan with better coordination between departments ● Ensure sufficient staff capacity (i.e., number of staff, skill sets of project employees) ● Expand targeting to additional communities and households, especially among the most needy ● Add visual aids, films, and posters in health/nutrition training ● Reduce the number of indicators (e.g., aggregate some indicators) ● Provide project officer for each component (i.e., FSL, NRM, SILC, health/nutrition) ● Build capacity of national partner(s) ● Improve stakeholder engagement, especially in design phase ● Provide more hardware in project design; more construction and / or repair of infrastructure is needed to offset damage and loss from decades of war 1.14.6. How has COVID-19 affected project implementation? Most KIIs with partner project staff and government partners, as well as many FGDs, suggest that COVID-19 had little significant effect on project implementation or outcomes. There were, however, some who perceived that COVID-19 and restrictions mandated by the government delayed implementation of some activities, as well as monitoring. In response to the pandemic, CRS incorporated specific COVID-19 protocols into project implementation. In Central Darfur, KIs indicated they conducted orientation sessions on COVID￾19 protocols for participants before – and during – voucher distribution at seed fairs, used social distancing for queues, and provided hand-washing facilities in waiting areas. KIs in other states indicated similar measures were taken to minimize risk to staff and beneficiaries. 1.15. Efficiency Overall, this section addresses questions about how well resources were used. There are very few insights from the qualitative data that specifically address the extent to which the project delivers results in an economic manner. In large part, this is due to the limited number of KIIs conducted with relevant project staff due to the coup and subsequent shutdown of government and NGO offices. One KI in Eastern Darfur noted, “The level of efficiency is very high for timely delivery of goods, but the cost per project participant is low due to high needs in the community.” In the absence of detailed information on budgets and activity costs from project staff across the five states, it is unclear how widely applicable this sentiment might be across the project. Another KI indicated that the amount spent per beneficiary improved after the “change in exchange rate” (i.e., to USD); project participating households received more money in SDG (when based on the USD), which helped them afford more seeds/tools during seed fairs. 1.15.1. How did project management adapt the project design or implementation? Numerous KIs indicated that the RISING II project made strategic and purposeful changes to implementation in response to challenges that arose. For example, challenges presented by COVID-19 were managed by adapting protocols for interacting within communities, such as setting health conditions, social distancing, and using a ToT approach to train people in their villages. Additionally, CRS held weekly and monthly meetings with local partners to reflect on the project’s activities, at least in some states. To address project beneficiary concerns, suggestion boxes and / or other means of communication were Page | 44 provided to project beneficiaries, though it is not clear to what degree they may have been used. Most FGD participants and KIIs indicated there were few, if any, issues to be dealt with as a result of beneficiary input. For example, CRS sent a team of project staff to resolve a lack of representation in one community CAG in East Darfur. Although it is likely other adaptations were made in response unforeseen challenges, monitoring, or beneficiary input, insights from other FGDs and KIIs are limited to only a few specific examples: • In North Darfur, a KI indicated the project changed the variety of millet seed provided to beneficiaries, from Ashana to Dembi, which project beneficiaries preferred. • Similarly, in West Darfur, feedback from beneficiaries resulted in a shift to “improved grains that were preferred by the target audience (Tabat corn and Wad Ahmed).” • In South Darfur, one FGD mentioned that services delivery (e.g., inputs) and distribution were said to have improved after beneficiaries complained. • Also in East Darfur, a FGD indicated that “the farming activities changed to application of agricultural techniques this rainy season” but no follow-up helped clarify exactly what this meant. For example, was this “change” in response to concerns raised by project beneficiaries (e.g., implementation of agricultural activities was moving too slowly) or does the response not truly match the question (i.e., was the question clearly understood by members of the FGD)? 1.15.2. To what degree was implementation and delivery of goods and services conducted in a timely manner? There was an overall awareness on the part of both project beneficiaries and partner staff that some delays occurred, whether from COVID-19, safety / security concerns, or processes internal to consortium partners. One state-level project officer listed high prices, the security situation, lack of sufficient liquidity in banks, and the non-neutrality of local leaders in terms of tribal differences (e.g., decision-making that preferentially affects individuals within a community) as challenges to timely project implementation. KIs mostly noted that the project timeframe was too short. According to KIs in East and North Darfur, the project should be implemented for 3-4 years in order to build resilience. Others agreed that a longer period of implementation would allow for more capacity building of partners (e.g., NGO and government). Many KIs felt that COVID-19 delayed delivery of goods and services by restricting movement and travel by project staff to communities. 1.16. Coherence 1.16.1. How did project activities align with government strategies/policies? With other projects/NGOs? Qualitative insights from all KIIs suggest that project activities are well-aligned with the priorities and policies of the government’s line ministries, particularly the Ministry of Agriculture and Forests (MoA) and Ministry of Health (MoH). One of the MoA’s top priorities is to increase crop production and productivity, although it also has certain health objectives, which are aligned with health and nutrition activities implemented under RISING II. Addressing malnutrition, including through behavior change, is in line with the MoH’s priorities and strategies. A KI in Central Darfur said the project is, “in line with government line ministries because it focuses on supporting households to be food secure, have better income and means of livelihoods.” Another KI, in West Darfur, noted that “The objectives of the program stem from food security and peace building stipulated in the peace agreement in Sudan…” As noted by numerous KIs, RISING II activities are implemented in coordination with and with “support from line ministry technical staff.” A government KI indicated that staff from the MoA are now taking the lead in most project trainings at community levels and will continue to provide necessary extension support they need even after the conclusion of RISING II. 1.16.2. What improvements could be made to partner coordination to leverage better results? KIs mentioned the good relationship – and coordination – that exists between project and government partners, although this can be difficult to maintain as administrations are frequently changing. There also Page | 45 appears to be good coordination among RISING II consortium and other partners, due in part to planning, monitoring, and monthly and quarterly reporting. In East Darfur, a KI indicated there is, “Good coordination with partners such as HAC, line ministries, local NGOs, and UN agencies, which contributed to project achievements.” At the same time, there is some thought that coordination could be improved through earlier involvement of partners (e.g., during the project launch) and through regular or “continuous” coordination between partners. 1.17. Sustainability This section focuses on whether the benefits gained from project activities are likely to last into the future, especially if external project support ends. Overall, the likelihood of outcomes being sustained after the project ends are unclear, though most of those interviewed are hopeful. 1.17.1. To what extent are the project’s outcomes likely to be sustained after the project ends? The issue of whether – or to what degree – outcomes will be sustained once a project ends is difficult to truly gauge. Project participants want to be hopeful; implementing partners/project staff are often overly doubtful. Qualitative data suggest that people value the skills and knowledge they gained from the project and that they do not end with the end of a project. For example, project activities strengthened SILC groups and members’ ability to save, which means they have better opportunities for starting new businesses even if the project phases out. Likewise, people felt that they will continue applying improved practices and producing more diverse foods even when the project ends. Strengthened local capacity – within both the community and government line ministries – will help ensure many outcomes continue into the future. The CAG, an important local structure, will remain functioning and help in addressing community needs and in planning. At the same time, more community involvement and participation is needed to ensure long-term sustainability of collective action that benefits all community members equally and continues to build social cohesion. Insights from qualitative interviews in East Darfur noted that including traditional healers in nutrition activities will help ensure mothers continue to be aware of, and act on, nutritional issues in their children. However, the project trained and raised the awareness of traditional healers (e.g., to use MUAC tapes), who now advise mothers when to take their malnourished children for treatment. Additionally, focusing on local foods and ingredients in cooking demonstrations makes it easier for caregivers to continue this practice after the project ends. 1.17.2. What are the challenges to ensuring gains will continue after the project ends? There is some thought that sustaining the gains in production will require continued access to agricultural inputs and training, neither of which are likely to be realized – at least to any large degree – in the absence of external assistance. Governments and their priorities change; staffing and budgets are usually insufficient relative to the need. The general feeling is that although the project has improved livelihoods and knowledge overall, the lack of improved seeds, farming equipment, livestock (e.g., goats), and financial capital continue to be challenging. Additionally, lack of rain and flooding often lead to crop failure. Factors that may challenge people’s ability to sustain positive outcomes include the security situation in Sudan. Political instability is likely to negatively impact the ability – and certainly the timeliness and reach – of development actors in the country. Unstable macro-economic conditions are also challenging, in terms of people’s purchasing power and ability to save or obtain loans. D. CONCLUSIONS AND RECOMMENDATIONS Quantitative and qualitative evidence from the final evaluation shows that the RISING II project had significant and positive impacts on beneficiary households across all five states in Darfur. Although not all end of project targets were achieved, tremendous progress was made. Thus, there remains much opportunity for future programs to learn from the RISING II final evaluation in order to have more impact on improving food and nutrition security for vulnerable populations in Darfur. Several lessons learned have “Forever, we will continue to keep the learning and experience from CRS in terms [of the] application of agricultural techniques…, NUT/health modules learned [by] women, and SILC activities.” -FGD with M/F CHWs, East Darfur Page | 46 emerged, particularly from qualitative interviews, and are presented below along with some recommendations. Project Activities ❖ Livelihood activities: Shocks from climate change (e.g., drought) are likely to continue negatively impacting households’ livelihoods (primarily farming and livestock-rearing) and putting households and communities at recurring risk of food insecurity. Future programs should identify and promote alternative and innovative income-generating activities that are at less risk from the effects of climate change. ❖ Access to Financial services: Although the SILC activities are effective and should be continued/expanded, participants from the FGDs and KIIs recommend more support for financial service providers and links with financial institutions in future programming, including for women. ❖ Nutrition and Health: According to project stakeholder interviews, there is currently an over-emphasis on referring children with severe/moderate acute malnourishment to treatment centers. Future programs should better balance project activities that promote nutritional referrals with those that promote other key health interventions and messaging (e.g., exclusive breastfeeding, diet diversity, handwashing). Messaging strategies can be strengthened by introducing hardware components (e.g., toilet parts, materials/parts for water points) with hygiene promotion activities and training lead mothers at the community level for IYCF messaging. Project Management ❖ Data collection: The timing of data collection can complicate interpretation of project outcomes. Data collection activities should be timed and coordinated to occur during the same time of year (ideally during the lean season) so that food security and other indicators with potential seasonal effects (e.g., coping strategies) can be measured and interpreted accurately. ❖ End of Project targets: The End of Project (EOP) targets for the 12 logframe indicators reported here were overly ambitious given the short project timeframe. The baseline and endline surveys were conducted approximately one year apart. EOP targets should be based on findings from project baseline data combined with what project technical staff feel is realistic to achieve over the LOA rather than using targets from projects with longer timeframes (i.e., Taadoud). ❖ Activities appropriate to project duration: Even with a NCE, 25 months is too short a timeframe to facilitate some types of change, particularly for many nutritional outcomes, behavior change related to culturally sensitive issues (e.g., breastfeeding, women’s engagement outside the house), and people’s coping strategies in response to shocks and stressors. It also takes time to strengthen resilience capacity and for households to adopt – and adapt to – new ways of growing crops and rearing livestock or taking on new types of livelihoods. An 18- or 25-month project is more in line with humanitarian response programming whereas development programs more typically involve a minimum of four years for implementation. If donors are not able or willing to extend project timelines, then implementing partners need to prioritize activities to those for which improvements can be made and easily measured within a short timeframe. ❖ Internal processes: Based on feedback from state-level project staff and implementing partners, CRS should streamline and/or strengthen their procurement process(es) to improve efficiency, logistics, and timeliness of delivering goods and services. They should also review staffing levels and capacity to inform hiring additional staff at the state level. ❖ Project visibility: Insights from qualitative data suggest the need to increase visibility of the project to better engage beneficiaries, communities, and government partners (e.g., government ministries), particularly as part of its launch. Government, community, and other partners need to be engaged in all phases of the project, including the design phase and community leaders can help mobilize their communities. Page | 47 E. ANNEXES ANNEX I Sample Size Estimation Following are the three project goal level indicators are critical to measure RISING II project impact and outcome: ● Food Consumption Score (FCS): Percentage of households with poor, borderline, and adequate food consumption score ● Reduced Coping Strategy Index (rCSI): Required: Mean, standard deviation, confidence interval (95%), and median ● Household Hunger Score (HHS): Prevalence of households with moderate or severe hunger The sample size estimated for FCS and HHS indicators provide sufficiently large samples to provide precise estimates for the other household level indicators. Also following are the two outcome level nutrition indicators that are important to measure for children 0-5 months and 6-23 months. 1. Proportion of children 6-23 months of age who receive foods from 4 or more food groups. 2. Proportion of infants 0-5 months of age who are fed exclusively with breast milk. BHA/USAID recommends the following sample size estimation formula and parameters30 with a fixed sample size 678 households. The recommended estimation procedure used to detect 10 percentage-points change from baseline to endline and assume 50% prevalence at baseline (if such an estimate is not available from prior surveys). Where: 𝑛initial = is the initial sample size required by the surveys for each of the two time points. a. 𝛿 = 𝑃1,est − 𝑃2,𝑒st = minimum effect size to be achieved over the time frame specified by the two surveys. b. 𝑃1,est = represents a survey estimate of the true population proportion 𝑃1 at baseline [If such an estimate is not available from prior surveys, 0.5 will be used]. c. 𝑃2,st = represents a survey estimate of the true population proportion 𝑃2 at evaluation. d. P = mean of 𝑃1,est and 𝑃2,st = 0.5 e. 𝑧1−∝ is the value from the normal probability distribution corresponding to a confidence level 1−𝛽. For 1−𝛽 = 0.95, the corresponding value is 𝑧0.95= 1.64 (two-tailed test). f. 𝑧1−𝛽 is the value from the normal probability distribution corresponding to a power level of 1−𝛽.For 1−𝛽 = 0.80, the corresponding value is 𝑧 0.80 = 0.84. g. 𝐷est is the estimated design effect (DEFF) of the survey. Also, the finite population correction factor FPC = 1/ (1+ninitial/N), where ninitial is the initial sample size and N is the total population of beneficiaries , should be used to adjust the final sample size if initial sample size (ninitial) is greater than 5% of the total population of beneficiaries (i.e. ninitial/N is greater than 5%). However, information for the three main indicators (FCS, rCSI and HHS) is available from 2020 RISING II Baseline survey. Thus, actual proportions for these indicators have been used to calculate the sample size using the BHA/USAID recommended formula. Also, based on the experience from Taadoud II and RISING II surveys, non-response rate is expected to be higher than the BHA recommended non-response rate 10%. Therefore, the non-response rate has been considered as 15% for RISING II beneficiary-based endline survey. 30 FFP Guidance for an Abbreviated Statement of Work (SoW) for a Baseline Study and Final Evaluation for Emergency Programs (Revised on December 2019) Page | 48 The estimated sample size for household level indicators needs to be inflated for the two nutrition indicators (exclusive breast feeding and child diet diversity), so that the number of households to be interviewed will ensure the required child sample sizes for children 0-5 months and 6-23 months will be attained. The approach involves inflating the required sample size using the following formula that the proportion of households that have at least one eligible child.31 nchild= ninitail X Where, Ƴ = eligible children per household32 The RISING II endline SoW includes estimated sample sizes for the indicators applying the BHA recommended formula and parameters, except the recommended threshold to detect 10 percentage points changes from baseline to endline. Using the threshold to detect 15% points change might not have adequate sample that the BHA recommended threshold (10% points change). However, the logframe indicator target for the FCS indicator has been set to be increased 15% points from baseline to endline. The sample size was estimated to detect 15% points change for FCS indicator and 10% points change for the HHS indicator according to the targets given in the RISING II logframe. TANGO carefully reviewed the estimated sample sizes given in the RISING II endline SoW and proposed slightly larger sample sizes using the actual RISING II baseline values and standard parameters given in Table 2. Table 2: Sample size by five critical outcome indicators Indicators Baseline (P1,est) Assumed for endline (P2,est)) Z1-α at 95% CI level Z1-ᵦ 80% power Design effect (Dest) ninitial Inflation factore Non Response (%) Sample HHs (n)d 1. Percentage of households with poor, borderline, and adequate food consumption score 0.471a 0.621b 0.165 0.84 2.0 270 15% 318 2. Percentage of households with moderate or severe hunger 0.208a 0.108b 0.165 0.84 2.0 327 15% 384 3. Percentage of children 0-5 months who are exclusively breastfed 0.457a 0.657b 1.282c 0.524c 2.0 80 11.307 15% 1,058 4. Proportion of children 6-23 months of age who receive foods from 4 or more food groups 0.193a 0.393b 0.165 0.84 2.0 126 05.201 15% 770 a RISING II Baseline 2020 b Actual endline targets from RISING II Logframe c The required sample size for children 0-5 months is significantly higher using the same level of significance (95% CI) and statistical power (80%) used for other indicators. For this reason the RISING II project has recommended to use lower confidence level (90%) and statistical power (70%) since this is not a critical indicator in the RISING II logframe. d The initial estimated sample sizes ninitial were found below 5% of the total beneficiary households 28,060. Finite Population Correction Factor is not applicable. e Inflation factors to adjust for children 0-5 months (exclusive breastfeeding), and 6-23 months (diet diversity) 31 i) Feed the Future Population-Based Survey Sampling Guide, Diana Maria Stukel, PhD, April 2018 ii) Addendum to FANTA Sampling Guide by Robert Magnani (1999): Correction to Section 3.3.1, Determining the Number of Households That Need to be Contacted, Diana Stukel Megan Deitchler , March 2012 32 Calculated from 2020 RISING II Baseline: Beneficiary family size 6.6 and beneficiary population proportions for children 0-5 months 1.4% and 6-23 months 3.2%, i.e. Ƴ=6.6X1.4% for children 0-5 months and Ƴ=6.6X3.2% for 6-23 months Page | 49 The minimum required sample sizes have been estimated for two household level indicators 318 for FCS and 384 for HHS. The minimum required sample size for the HH level indicators has been considered the largest estimated sample size of these two indicators i.e. 384. But the required sample size for children 0-5 months is 1,058 households. Therefore, the final sample size is rounded to 1,040, which will be the final sample size for RISING II endline survey. According to the PaBS guideline33 the range of cluster sample size is recommended 15 to 35 sampling units (households). Larger cluster sample size may have larger design effect of the estimates. A larger number of clusters in the sample ensures high level of statistical efficiency. Thus, 20 beneficiary households will be selected randomly from the list of RISING II Beneficiary sampling frame in each of 52 PPS sample clusters (out of 92 communities) to survey 1,040 households across all five Darfur program states and localities. 33 Feed the Future “Participant-Based Survey Sampling Guide for Feed the Future Annual Monitoring Indicators”, Diana Maria Stukel, PhD, September 2018 Page | 50 ANNEX II Data Analysis and Tabulation Plan Indicator Type of respondents Main Disaggregation Method Question Number Baseline Target Data Tabulation/Analysis 1. Impact i: Percentage of HH with poor, borderline, and adequate Food Consumption Score (FCS). Household member who is primarily responsible for food preparation and distribution ● HH compositi on (FNM, MNF, M&F, CNA)34 ● The Food Consumption Score (FCS) is a composite score based on dietary diversity, food frequency, and the relative nutritional importance of different food groups. To collect FCS data, a questionnaire is used to ask respondents about the frequency of their households' consumption of 16 food groups over the previous seven days. ● Sixteen food groups regrouped into 9 main categories: Main staples, Pulses, Fruits, vegetables, Meat/Fish/eggs, Milk, Sugar/honey, Oil and condiments. ● To calculate the FCS, the consumption frequencies (number of days in last 7 days) are summed and multiplied by the standardized food group weight (Main staples 2, Pulses 3, Fruit 1, vegetables 1, Meat/Fish/Eggs 4, Milk/Milk products 4, Sugar/honey/products 0.5, Oil/fats/butter 0.5 and condiments 0), and households are then be classified into three groups based on their weighted scores – poor (FCS 0-28), borderline (FCS 28.5 to 42), or acceptable (FCS >42) - using the World Food Program's/EFSP indicator guideline recommended cutoff points35 . ● Reference: FFP/ WFP/FAO Guideline, FFP/EFSP Indicator guideline and RISING II PIRS B101 ● Poor: 23.5% ● Bo: 29.4% ● Ac: 47.1% ● Poor: 0% ● Ac: 62.1% ● 16 foods/food groups B101A to B101O are re-grouped into 9 food groups with maximum number of days consumed in 7 days for any food in each food group. For example: Cereals/grain consumed 7 days (B101A2) and roots/tubers consumed 5 days (B101B2). Main staples = max(B101A2, B101B2) = 7 days ● Weighted consumption is obtained multiplying number days consumed by the respective food group weights. ● Weighted total of 9 food groups yield FCS score. 2. Impact ii: Prevalence of HH with moderate or severe hunger Household member who is primarily responsible for ● HH Compositi on (FNM, MNF, ● The HHS is a food deprivation scale that measures the percent of households experiencing hunger. To collect data for this indicator, the person in the household in charge of food preparation is asked about B102-B104 20.8% ● 10.8% ● If B102, B103 and B104 “No=2” recoded to “0” and “Don’t know=3” recoded to system missing. 34 Adult Female no Adult Male (FNM), Adult Male no Adult Female (MNF), Male and Female Adults (M&F), Child no Adults (CNA) 35 Category cult-off points are considered for daily consumption of oil and sugar Page | 51 Indicator Type of respondents Main Disaggregation Method Question Number Baseline Target Data Tabulation/Analysis (Household Hunger Scale, HHS) food preparation and distribution M&F, CNA) the frequency with which three events were experienced by any household member in the last four weeks: 1. No food at all in the house 2. Went to bed hungry 3. Went all day and night without eating ● If the event is reported as having not been experienced in the last four weeks, the response is coded as “never” (value = 0). ● If the event is reported as having been experienced in the last four weeks, a frequency of occurrence question is asked to determine how often the event was experienced. ● For each frequency of occurrence question, the following responses are possible: “rarely” (value = 1), “sometimes” (value = 2), and “often” (value = 3). For tabulation purposes, the responses are then recoded into three frequency categories: “never” (new recoded value = 0), “rarely or sometimes” (new recoded value = 1), and “often” (new recoded value = 2). ● Values for the three questions are summed for each household, producing a HHS raw score ranging from 0 to 6. The raw scores are then tabulated into the following categories: HHS score 0-1 = little to no hunger HHS score 2-3 = moderate hunger 1.18. HHS score 4-6 = severe hunger ● FFP requires reporting of the percentage of households with a score of “moderate” to “severe” hunger. ● Reference: FFP/FANTA Guideline, RISING II PIRS ● B102, B103 and B104 “Yes=1”, recoded variables B102a, B103a and B104a for the responses Rarely 1=1, Sometimes 2=1 and Often 3=2. ● Sum of the recoded variables B102a, B103a, B104a is HHS score. ● If recoded all B102 & B103 & B104=0, HHS score=0. 3. Outcome 1.1: Percentage of HHs with equitable and Household head and other knowledgeable HH members ● HH Compositi on (FNM, MNF, ● The indicator ‘households have equitable and uninterrupted access to shared resources’ refers to every household in a community should have equal choice to use, withdraw D101-D105 45.9% (at least 3 out of 5) ● 65.9% ● Access to water: if (D101b1=1 & D101b2=1) and/or D101c=1 Page | 52 Indicator Type of respondents Main Disaggregation Method Question Number Baseline Target Data Tabulation/Analysis uninterrupted access to shared natural resources (water, pastures etc.) throughout the year. M&F, CNA) and manage the available shared or open natural resources and facilities that will be not interrupted by any individual or group. ● Each household in a community if want to use natural following resources without any interruption year-round (dry and other seasons): vi. Open or communally established water source (water from the stream/river, borehole, check dam, watershed) vii. Open or shared pastureland, grazing land for agriculture production and livestock rearing viii. Access to sharecropping/lease and procuring agriculture land ix. Use forest products for livelihood and livestock x. Use of ponds or open waterbody for fishing xi. Participation in natural resources inputs supported by the Government and others. ● Percentage of household with uninterrupted access to shared natural resources year￾round will be reported to this indicator. ● Reference: Taadoud II M&E Guideline/RISING II PIRS ● Access to pastureland: if D102a1=1 & D102a2=1 ● Access to share￾cropping/ leasing/procuring agricultural land: if D103a=1 ● Access to forest/forest product: if D104a=1 &/or D104b=1 ● Government/Others inputs supported natural resources: if D105a=1 ● Sum of the 5 access score ranges from 0 to 5. ● A household with at least 336 out of 5 access scores will be counted for this indicator. 4. Outcome 1.2.1: Reduced Coping Strategies Index (rCSI). Household member who is primarily responsible for food preparation and distribution ● HH Compositi on (FNM, MNF, M&F, CNA) ● The rCSI is a proxy indicator of household food insecurity that is based on a list of behaviors (coping strategies). The index reflects both the frequency of each behavior (i.e. how many days over the last 7 days the coping strategy was used by any member of the household) and severity (i.e. how serious the strategy). The rCSI is based on a list of five food-related coping strategies (Rely on B105 11.7 ● 8.8 ● rCSI1=0 if B105a1=2 and rCSI1=B105a2 ● rCSI2=0 if B105b1=2 and rCSI2=B105b2 ● rCSI3=0 if B105c1=2 and rCSI3=B105c2 ● rCSI4=0 if B105d1=2 and rCSI4=B105d2 36 The threshold “at least 3” is not mentioned in PIRS. This need to be confirmed by the RISING II project MEAL team prior to start data analysis Page | 53 Indicator Type of respondents Main Disaggregation Method Question Number Baseline Target Data Tabulation/Analysis less preferred/less expensive food, Borrow food or rely on friends/relatives, limit portion size at mealtime, restrict adult consumption and reduce number of meals in a day) that the household used in the seven days prior to the survey. ● The rCSI raw scores are calculated by multiplying the frequency (number of days) with which a behavior was used by the universal severity weight (Less preferred/less expensive food 1, Borrow food or rely on others 2, limit portion size 1, restrict adult consumption 3 and reduce number of meals in a day 1), then summing the weighted scores for each coping strategy. ● The maximum raw score for the rCSI is 56, i.e. a household that used all five strategies every day for the last 7 days would have a raw score of 56. ● Mean of rCSI score will be used for this indicator. ● Reference: FFP/WFP Guideline, RISING II PIRS ● rCSI5=0 if B105e1=2 and rCSI5=B105e2 ● Weighted rCSIwi scores obtained multiplying individual rCSIi score by the respective severity weights ● Sum of rCSIwi is the composite rCSI score 5. Outcome 1.3.1: Percentage of HH using improved natural resources. HH Head and others knowledgeable HH members ● HH Compositi on (FNM, MNF, M&F, CNA) ● This indicator measures percentage of households using/applied at least two of the RISING II project promoted improved natural resources in the past 12 months. ● The improved natural resources are related to use of land or water or management innovations including those that address ecosystem restoration, climate change adaptation and mitigation. Following are the improved natural resources management practices are considered to calculate this indicator: E301b,E304,E 308,E318, E318a, E320A, E302B-E302D, E321 (A,B,D), E307, E321 33.8% ● 63.8% vii. Crop culture: seedling production and transplantation, intercropping, mulching etc. E301b, E304, E308 ● Crop culture: if E301b=1 or E304=1 or E308=1 Page | 54 Indicator Type of respondents Main Disaggregation Method Question Number Baseline Target Data Tabulation/Analysis viii. Permaculture practices (Soil and water conservation): erosion prevention and control with mechanical and biological methods such use of gabions, vetiver grass, elephant grass, riverbank management E318, E318a ● Permaculture: if (E318=1 or E318=2 or E318=3 or E318a=4 or E318=5 or E318=6) or E318a=1 ix. Agroforestry: Integrating fodder and fruit trees production, other nitrogen fixing trees. E320A ● Agroforestry: if E320=1 x. Natural regeneration such as Farmer Managed Natural Regeneration: Regeneration of natural landscapes (Reseeding), Management of forest plantation (planting, cutting, pruning etc., Sustainable harvesting of forest products (wood, firewood, charcoal production etc. E302B-E302D ● FNMR: if E320=2 or E320=3 or E320=4 xi. Water management: non-irrigation-based e.g., water harvesting, improved mechanical and physical land preparation such as building of swales or ditches, sub surface dams and watershed E321 (A,B,D) ● Water management: if E321=2 or E321=3 or E321=4 xii. Climate mitigation: technologies selected because they minimize emission intensities relative to other alternatives. Examples include low- or no-till practices and use alterative (e.g, could be solar) for irrigation E307, E321 ● Climate mitigation: if E321=3 or E307=1 6. Outcome 2.1: Number of months of HH food self￾sufficiency as a result of improved agricultural production programming HH Head and others knowledgeable HH members ● HH Compositi on (FNM, MNF, M&F, CNA) ● Food self-sufficiency is defined as physical and economic access to adequate food for all household members. The number of months is an estimate of the ability to produce, purchase, store, or otherwise maintain a level of food consumption, without external assistance or inputs such that members of the household are not going hungry achieved with a degree of plausible attribution to USAID/OFDA-funded activities to improve ● The household will be asked for how many months they expect to cover their household with adequate consumption from own agriculture and livestock production and selling without supports from others. E207 4.6 months ● 5.9 month s ● Average from E207 Page | 55 Indicator Type of respondents Main Disaggregation Method Question Number Baseline Target Data Tabulation/Analysis ● The average number of moths will be reported for this indicator. ● Reference: OFDA Guideline/RISING II PIRS 7. Outcome 2.2: Percentage of beneficiaries reporting net income from their livelihood. HH Head and others knowledgeable HH members ● Sex: Male and Female ● ERMS/New Livelihoods Development activities aim to help people start an activity they have not done before to help them earnings. ● Livelihoods Restoration activities, by contrast, help people to resume the activities they were doing pre-disaster or were already practicing to earn a living. ● Net income is simply defined here as revenues greater than costs. It is synonymous with “profitability.” ● This indicator measures what percent of people self-report that they are earning more from the livelihood activity supported by USAID/OFDA than they are spending on costs (i.e., inputs, rent, transport, fees). ● This indicator should be measured at or near the end of the USAID/OFDA-supported project. Example: Someone who was earning a net income shortly after receiving inputs halfway through the project period, but is no longer earning a profit, should not be counted as profitable. ● RISING II supported livelihood activities are SILC and assets based saving supports, PSP activation, training on small scale business planning and linkages supports to the male and female headed households with the SILC and PSP groups and capacity building to the farmers and pastoralists on value chain improvement, packaging, inputs supports and market linkages. ● Percentage of households with increased net income (profit) from the RISING II project supported livelihood activities will be reported for this indicator. ● Reference: OFDA Guideline/RISING II PIRS C102A and C103 72.9% ● 92.9% ● HH will be counted with increased income if C102a1 to C102h1=1 and C103=1 ● Denominator (C101): Total number of HH reported with livelihood activities for agriculture, livestock and other income generating activities in the past 12 months. Page | 56 Indicator Type of respondents Main Disaggregation Method Question Number Baseline Target Data Tabulation/Analysis 8. Outcome 2.1.1: Number of hectares under improved agricultural methods HH member who is involve/ knowledgeable in agriculture and farming activities ● No disaggrega tion mentione d in Logframe ● This indicator measures the number of hectares that receive treatment using improved agricultural methods due to USAID/OFDA-funded activities. ● For USAID/OFDA, the designation of “improved agricultural method” indicates methods to enhance productivity and at the same time soil/water/nutrient quality with the goal of long-term system protection. ● These may include soil and water conservation measures (e.g., low till, no till, conservation agriculture), mechanical treatment or application of soil improvements or enrichments. ● This indicator captures the total hectares of land with at least one RISING II project promoted technology or practice applied in the past 12 months. It does not matter how many total improved technologies or practices are applied to an area. ● Land with individual technologies or a practice will be collected by first asking whether the farmer used or applied the specific technologies or practices in the past 12 months. If the farmer answered “yes,” he or she will then be asked how much land had been used for this specific technology or practice and how many times the land was used in the reporting year. If the farmer applied promoted technologies to his or her plot several times in the year, then the area of the plot would be counted for the number of times used under this indicator37 . ● The sum of the hectares of land that was used for individual technologies is the indicator value and thus the same land could be reported more than one times in some cases. E301-E301a E301b-E301c E302-E303a E304-E305a E306-E307a E308-E309a E313a-E313b 2,758 hectares ● 3,034 hectar es ● If a HH applied correct practice will be counted for this indicators ● Mukhamas of land will be converted into hectares ● Sum of land that was used with correct practices will be reported to this indicator 37 Feed the Future Indicator Handbook, October 2014 Page | 57 Indicator Type of respondents Main Disaggregation Method Question Number Baseline Target Data Tabulation/Analysis ● Reference: FFP/OFDA Guideline/RISING II PIRS 9. Outcome 2.1.2: Percentage of HH with access to sufficient seed to plant. HH member who is involve/ knowledgeable in agriculture and farming activities ● HH Compositi on (FNM, MNF, M&F, CNA) ● Access to sufficient seed indicates that a household either has seed in hand (own stocks) or has the ability or means to get it elsewhere (from neighbors, market, agro￾dealers) in a timely manner to plant the farmer’s normal intended planting area. ● This is a general measure of seed security at the household level and takes into account the full range of channels through which farmers procure seed to meet their household needs. One of these channels may be the receipt of relief seed. ● For RISING II the seeds promoted are millets, sorghum, groundnuts, and some vegetables and shall access from the channels of local vendors and private sectors which the project will facilitated through DiNerFair with a voucher distribution program. ● Reference: OFDA Guideline/RISING II PIRS E204, E204a 33.1% ● 58.1% ● HH with responses 1 (adequate) or 2 (more) in E204a will be counted for this indicator. ● Denominator: HH with production of crops in the past 12 months reported in E204 10. Outcome 3.1: Proportion of children 6-23 months of age who receive foods from 4 or more food groups. Mother/Caregive r of children 6-23 months ● Sex: Male and Female ● Information will be collected for 19 foods or food groups for children 0-23 months. These 19 foods or food groups will re-groups into following 7 food groups based on the WHO/UNICEF IYCF indicator guideline: - Grains, roots and tubers - Legumes and nuts - Dairy products (milk, yogurt, cheese) - Flesh foods (meat, fish, poultry and liver/organ meats) - Eggs - Vitamin-A rich fruits and vegetables - Other fruits and vegetables. ● Consumption of any amount of food by children 6-23 months from each food group is sufficient to “count”. I.e., there is no minimum quantity, except if an item is only used as a condiment. ● This is based on recall of the previous day. G113 19.3% ● 39.3% ● Recode 1 if “yes” in g113a to g113t and 0 if “no”. ● 19 foods are grouped into 7 food groups and are coded with 1 if any of the items in the individual group is with code 1, otherwise 0. ● Sum of the 7 food groups will have scores ranges from 0 to 7. ● If the some of the score is greater than or equal to 4 will be counted for this indicator. Page | 58 Indicator Type of respondents Main Disaggregation Method Question Number Baseline Target Data Tabulation/Analysis ● Children 6–23 months of age who received foods from ≥4 food groups during the previous day will be counted to compute percentage to report this indicator. ● Reference: WHO/IYCF Indicator Guideline, RISING II PIRS ● Denominator: Number of children 6-23 months in the sample. 11. Outcome 3.1.1: Proportion of infants 0-5 months of age who are fed exclusively with breast milk Mother/Caregive r of children 0-5 months ● Sex: Male and Female ● Fed exclusively: Infant is fed breast milk (including milk expressed from a wet nurse) and nothing else, not even water. ● Infants may be given ORS, drops, syrups (vitamins, minerals, medicines). ● This is based on recall of the previous day ● Children 0-5 months fed only breastmilk with an exception of vitamins/minerals/medicine in the past 24 hours will be counted to calculate percentage for this indicator. ● Reference: WHO/IYCF Indicator Guideline, RISING II PIRS G106, G109, G112 and G113 45.7% ● 65.7% ● “Yes” for G106, G109 and “No” for G112 and no food was fed listed in G113 in the past 24 hours ● Denominator: number of children 0-5 months in the sample 12. Outcome 3.1.2: Percent of people targeted by the hygiene promotion program who know at least three (3) of the five (5) critical times to wash hands All Adult HH members ● Sex: Male and Female ● This indicator measures individuals’ knowledge of the hand washing practices which are most effective at preventing the spread of pathogens along the fecal-oral cycle. The five critical times to wash hands are defined as 1. After defecation/using the toilet; 2. Before eating; 3. After changing diapers or cleaning a child’s bottom; 4. Before preparing food; and 5. Before feeding an infant. ● Respondents who demonstrate that they know at least three (3) of the (5) critical times to wash hands will be counted to calculate percentage for this indicator. ● Reference: OFDA Guideline, RISING II PIRS F111, F111a 31.1% (at least 3 out of 5) ● 51.1% ● Each of the 5 critical times will be coded with 1 if the respondent mention as critical time for handwashing in f111 ● Sum of the score for 5 critical times will ranges from 0 to 5. ● Respondent with score 3 or more will be counted for this indicator ● Denominator: Total number of respondent for the question F111 ● Disaggregation: sex of the respondent from F111a Page | 59 ANNEX III Detailed results for RISING II Logframe Indicators Page | 60 ANNEX IV Qualitative Research Questions Topic Areas of inquiry Relevance: is the intervention doing the right things? • To what extent the intervention’s objectives and design respond to beneficiaries’, states, country, and partner/institution needs, policies, and priorities, and will they continue to do so if circumstances change? • To what extent the objectives and design of the intervention are sensitive to the economic, environmental, equity, social, political economy, and capacity conditions in which it takes place? • To what extent the intervention can be (or has been) adapted to remain relevant? Effectiveness: is the intervention achieving its objectives? • To what extent has the intervention achieved, or is expected to achieve, its objectives, and its results, including any differential results across groups? • Which interventions appear to have the most effect on outcomes? How do these changes correspond to those hypothesized by the activity’s Logframe? Efficiency: how well are resources being used? • To what extent does the intervention deliver, or is likely to deliver, results in an economic and timely way? How well was the intervention was managed? • How has management adapted the project design or implementation based on monitoring information and feedback from the target population? • What lessons were learned regarding program design and implementation? • What was the level of efficiency with regards to cost-per-project participant, timely delivery of the goods or services, and adjusting the transfer amount based on price and need changes? Impact: what difference does the intervention make? • To what extent the intervention has generated or is expected to generate significant positive or negative, intended or unintended, higher-level effects/change? • What factors appear to facilitate or inhibit these changes? Sustainability: will the benefits last? • To what extent do the net benefits of the intervention continue or are likely to continue after the project ends? (Focus on the financial, economic, social, environmental, and institutional capacities of the systems needed to sustain net benefits over time; involves analyses of resilience, risks and potential trade-offs) Coherence: how well does the intervention fit within the project area? • To what extent do other interventions (and policies) support or undermine the intervention, and vice versa? Includes internal coherence and external coherence? Page | 61 ANNEX V Proposed Field Schedule for Qualitative Data Collection Location Team October 18-24, 2021 Monday 18 Oct Tuesday 19 Oct Wednesday 20 Oct Thursday 21 Oct Saturday 23 Oct Sunday 24 Oct Tora C Abu Nabgia 1 North Darfur National 3 KIIs with HAC, gov’t (ag, nutrition, ranges/pastures, etc.) 2-3 KIIs with consortium and local partner staff Data entry; travel 3 KIIs with HAC, gov’t (ag, nutrition, ranges/pastures, etc.) 3 KIIs with HAC, gov’t (ag, nutrition, ranges/pastures, etc.) Data entry 2 Field teams 3 FGDs (e.g., farmers, leaders, CAGS, etc.) 2-3 FGDs or KIIs (e.g., mothers, CHW/CAHW, traders, etc) 3 FGDs (e.g., farmers, leaders, CAGS, etc.) 2-3 FGDs or KIIs (e.g., mothers, CHW/CAHW, traders, etc) Barka Tolei Kalaka 3 South Darfur National 3 KIIs with HAC, gov’t (ag, nutrition, ranges/pastures, etc.) 2-3 KIIs with consortium and local partner staff Data entry; travel 3 KIIs with HAC, gov’t (ag, nutrition, ranges/pastures, etc.) 2-3 KIIs with consortium and local partner staff Data entry 4 Field teams 3 FGDs (e.g., farmers, leaders, CAGS, etc.) 2-3 FGDs or KIIs (e.g., mothers, CHW/CAHW, traders, etc) 3 FGDs (e.g., farmers, leaders, CAGS, etc.) 2-3 FGDs or KIIs (e.g., mothers, CHW/CAHW, traders, etc) Azomry Almashageya Abu Oud 5 East Darfur National 3 KIIs with HAC, gov’t (ag, nutrition, ranges/pastures, etc.) 2-3 KIIs with consortium and local partner staff Date entry; travel 3 KIIs with HAC, gov’t (ag, nutrition, ranges/pastures, etc.) 2-3 KIIs with consortium and local partner staff Data entry Page | 62 Location Team October 18-24, 2021 Monday 18 Oct Tuesday 19 Oct Wednesday 20 Oct Thursday 21 Oct Saturday 23 Oct Sunday 24 Oct 6 Field teams 3 FGDs (e.g., farmers, leaders, CAGS, etc.) 2-3 FGDs or KIIs (e.g., mothers, CHW/CAHW, traders, etc) 3 FGDs (e.g., farmers, leaders, CAGS, etc.) 2-3 FGDs or KIIs (e.g., mothers, CHW/CAHW, traders, etc) Salima Waigo 7 West Darfur National 3 KIIs with HAC, gov’t (ag, nutrition, ranges/pastures, etc.) 2-3 KIIs with consortium and local partner staff Date entry; travel 3 KIIs with HAC, gov’t (ag, nutrition, ranges/pastures, etc.) 2-3 KIIs with consortium and local partner staff Data entry 8 Field teams 3 FGDs (e.g., farmers, leaders, CAGS, etc.) 2-3 FGDs or KIIs (e.g., mothers, CHW/CAHW, traders, etc) 3 FGDs (e.g., farmers, leaders, CAGS, etc.) 2-3 FGDs or KIIs (e.g., mothers, CHW/CAHW, traders, etc) Sowh Takjow 9 Central Darfur National 3 KIIs with HAC, gov’t (ag, nutrition, ranges/pastures, etc.) 2-3 KIIs with consortium and local partner staff Data entry; travel 3 KIIs with HAC, gov’t (ag, nutrition, ranges/pastures, etc.) 2-3 KIIs with consortium and local partner staff Data entry 10 Field teams 3 FGDs (e.g., farmers, leaders, CAGS, etc.) 2-3 FGDs or KIIs (e.g., mothers, CHW/CAHW, traders, etc) 3 FGDs (e.g., farmers, leaders, CAGS, etc.) 2-3 FGDs or KIIs (e.g., mothers, CHW/CAHW, traders, etc) Page | 63 ANNEX VI List of Qualitative Interviews Name of KI Position Date of interview Community No./ gender Date received by TANGO West Darfur Government/Institutional Stakeholders Osman Ahmed Hussein Director General of the Ministry of Production and Economic Resources October 18 El Geneina. Ministry of Production and Economic Resources 1 M. No data (assume included in report?) Taher Muhammad Suleiman Deputy Commissioner October 18 El Geneina - HAC 1M. No data (assume included in report?) Imam Adam Moussa Youssef Nutrition Officer October 25 Forbranga locality - Ministry of Health 1 F. Nov 15 Adam Musa Yusuf Agricultural inspector October 25 Forbranga locality - Ministry Of Agriculture 1M. Nov 15 Khaled Khamis Ahmed Humanitarian Aid Commissioner October 25 Forbranga locality - humanitarian aid commission HAC 1M. No data (assume included in report?) RISING II Project Staff Abdel Razek Ahmed Adam first project officer October 19 El Geneina - Crs 1M. No data (assume included in report?) Noureddine Hassan Mohamed M&E RISING11 October 26 October 19 1M. No data (assume included in report?) Consortium partner staff (KIs) Salaheddine Abdullah Adam Program Manager West Darfur Youth Organization for Development October 18 El Geneina - Crs Office 1M No data (assume included in report?) Al-Tijani Shartay Sharif Peace Lights Organization￾Organization manager October 19 El Geneina - Crs Office 1M No data (assume included in report?) Mahdi Hamdan Hassan Peace Lights Organization￾Program Coordinator October 24 Forbranga locality - Waigo 1M. No data (assume included in report?) Community leaders Ismail Abdullah Abdulkarim Chieftain October 24 Forbranga locality - Waigo 1M. No data (assume included in report?) Hajar shirar Chieftain October 24 Forbranga locality - Waigo 1M. No data (assume included in report?) Focus Group Discussions/Key Informant Interviews Comm leaders, CAG, etc. October 19 Salima 3 females; 4 males October 29 Comm leaders, CAG, etc. October 22 Waigo 2 females; 5 males October 29 Community Health workers October 20 Salima 1 female; 2 males October 29 Community Health workers October 21 Waigo 2 females; 1 male October 29 Page | 64 Name of KI Position Date of interview Community No./ gender Date received by TANGO Traders, wholesalers, market actors October 20 Salima 3 females October 29 Traders, wholesalers, market actors October 21 Waigo 2 females; 1 male October 29 Farmers, pastoralists, etc. October 19 Salima 3 females; 5 males October 29 Farmers, pastoralists, etc. October 21 Waigo 2 females; 4 males October 29 Mothers, caregivers, etc. October 20 Salima 8 females October 29 Mothers, caregivers, etc. October 22 Waigo 6 females October 29 Central Darfur RISING Partner Staff Issa Adam Abdallah NCA NRM Officer November 11 Zalingei November 15 Mohammed Ismail NCA Ag Extension Officer November 15 Government/Institutional stakeholders Hawa Ibrahim Adam Min of Agr; Head of Ag Office, Central Darfur State Zalingei November 15 Focus Group Discussions/Key Informant Interviews Farmers, CAG, etc. November 7 Hilat Borrow/Host community 8 females November 15 Farmers, CAG, etc. November 6 Hilat Borrow/Host community 4 females; 4 males November 15 Health workers November 4 Arokum 2 females; 4 males November 15 Traders November 4 Urokum 5 males November 15 Traders November 7 Hilat Barnow 2 females; 2 males November 15 East Darfur Project/Partner Staff Hawa Zaharia Eisa SILC Project Officer October 24 CRS Office AlDein November 10 Francis Kyobe RISING Project Manager October 22 CRS Office AlDein November 10 Ishag Shaib Hamza Senior MEAL Officer October 26 CRS Office AlDein November 10 Eisa Hamid ALTAMAS (NNGO); Program manager October 20 CRS Office AlDein November 10 Emad Eldin Mohamed Ahmed Nutrition Project Officer; RISING II October 24 CRS Office AlDein November 10 Fathelrahman Osman NRM Project Officer; RISING II October 25/27 CRS Office AlDein November 10 Salih Musa Livelihood Project Officer; RISING II n.d. November 10 Page | 65 Name of KI Position Date of interview Community No./ gender Date received by TANGO Government/Institutional Stakeholders HAC-ED; Acting Commissioner/Head of Tech. Dept October 21 CRS Office AlDein November 10 Khalida Khaleel Head of Rural Women Dev./ Rep for DG-MOPER October 25 Min. of Production and Economic Resources November 10 Focus Group Discussions/Key Informant Interviews Community leader October 20 Azomry Elferdous 1 male November 10 Female community leader October 21 Azomry Elferdous 1 female Combined with above CHW October 23 Mashagia 1 male Traders October 20 Azomry Elferdous/Host 3 females; 7 males November 10 Farmers October 20 Azomry Elferdous/Host 4 females; 6 males November 10 Farmers, CAGs October 23 Elmashagia Abu Oud￾Assalaya/Host 3 females; 5 males November 10 Traders October 24 Elmashagia Abu Oud￾Assalaya/Host 4 females; 5 males November 10 Healthworkers, caregivers October 23 Elmashagia Abu Oud￾Assalaya/Host 8 females; 1 male November 10 Ajawids, CAG October 21 Azomry 3 females; 7 males November 10 Community Health workers October 21 Azomry Elferdous/Host 5 females; 3 males November 10 SILC group October 21 Azomry Elferdous/Host 15 females North Darfur Summary report draft report submitted November 10 Project/Partner Staff Ibrahim Hassan Adam CAFOD Program Accompanier Nutrition Ahmed Yasin CAFOD Program Accompanier Nutrition Government/Institutional Stakeholders Siddiq Ibrahim Haroun Manager of Ag Techniques Transfer and Extension Dept October 21 Alyaa Hadi Suliman Ag Engineer (Trainer) October 21 Latifa Adam M. Elayyid Manager of Ag Planning Dept Batoul Ahmed Manager of Planning and Animal Statistics Dept October 24 Rajaa Adam Hasan CAHWs Coordinator October 24 Page | 66 Name of KI Position Date of interview Community No./ gender Date received by TANGO Fatima Elrasheed Ag Engineer and Manager Rural Fashir Sabah Adam Idrees M&E Manager, HAC October 24 South Darfur Summary report draft No data; report submitted November 10 Page | 67 ANNEX VII Analysis Tables for Demographics and Background Characteristics Table 3 Survey Sample and Respondent's Profile Page | 68 Table 4 Household Demographics Page | 69 ANNEX VIII Additional Results Table 5: RISING II Sample and Non-response, by Survey Rounds SAMPLE AND NON-RESPONSE ALL Baseline 2020 Endline 2021 Total # of beneficiaries 15,347 16,442 Target sample size (HHs) 980 1,040 No. of HHs interviewed 924 938 Non-response Rate (%) 5.7 9.8 Table 6: Reasons for non-response, by Survey Rounds SAMPLE AND NON-RESPONSE ALL BL 2020 EL 2021 REASONS OF NON-RESPONSE A. Interview incomplete 1.8 3.9 B. HH present, but no adult 10.7 C. HH absent 71.4 46.1 D. Interview postponed 1.8 E. Dwelling vacant 5.4 17.6 F. Dwelling not found 5.4 11.8 G. Team did not collect data - 20.6 H. Other 3.6 - Figure 14: Primary Occupation of Household Head, by Survey Rounds Table 7. Nutritional Weights 4.0 1. Milk/Dairy foods 0.5 6. Fruits 4.0 2. Meat/Eggs/Fish 0.5 7. Sugar/Honey 3.0 3. Pulses/legumes 0.0 8. Oil/Fats 2.0 4. Cereals/Tubers 0.5 9. Condiments/others 1.0 5. Vegetables Page | 70 Table 8. Food-based Coping Strategies and Severity Weights 4.0 1. Limit/reduce portion size at mealtimes 4.0 2. Reduce number of meals eaten per day 3.0 3. Borrow food or rely on help from friend or relatives 2.0 4. Rely on less expensive or less preferred foods 1.0 5. Reduce adult consumption so children can eat Figure 15: Consumption of 9 Food Groups in the Past 7 Days, by Survey Rounds Figure 16: Household Livelihood Activities in the Past 12 months, by Survey Rounds Page | 71 Figure 17: Household Coping Strategies in the Past 7 Days, by Survey Rounds Figure 18: Household Income Generating Activities, by Survey Rounds Figure 19: Households Reporting Net Profit, by Livelihood Activity and Survey Rounds Page | 72 Figure 20: Change in Household Income Over the Last 12 Months, by Survey Rounds Figure 21: Availability of Natural Resources, by Survey Rounds Figure 22: Year-Round Household Access to Natural Resources, by Survey Rounds Page | 73 Figure 23: Households Using Improved NRM Practices in the Past 12 months, by Survey Rounds Figure 24: Household Access to Farmland, by Survey Rounds Figure 25: Household Utilization of Farmland to Which They Have Access, by Survey Rounds Figure 26: Household Use of Improved Practices in the Last Planting Season, by Survey Rounds Page | 74 Figure 27: Households Using at Least Three Improved Production Practices, by Survey Rounds Figure 28: Households Using at Least Six Improved Production Practices, by Survey Rounds Figure 29: Households’ Main Source of Food in the Past 12 Months, by Survey Rounds Figure 30: Access (easy/very easy) to Markets within 5km, by Survey Rounds Page | 75 Figure 31: Households With Improved Drinking Water Sources, by Survey Rounds Figure 32: Households Using Unsafe Drinking Water, by Survey Rounds Figure 33: Household Actions to Make Unsafe Drinking Water Safe, by Survey Rounds Figure 34: Households With Improved Toilet Facilities, by Survey Rounds Figure 35: Household Toilet Facilities, by Survey Rounds Page | 76 Figure 36: Respondents’ Perception of the Five Critical Times to Wash Hands, by Survey Rounds Figure 37: Males Who Know at Least Three of Five Critical Times to Wash Hands, by Survey Rounds Figure 38: Females Who Know at Least Three of Five Critical Times to Wash Hands, by Survey Rounds FNM and MNF households implies adults at least 18 years old whereas household heads can be 15 or more years old. Males in FNM and females in MNF households may include household heads between the ages of 15 and 18. Page | 77 Figure 39: Boys Aged 6-23 Months Consuming Four or More Food Groups During the Previous Day, by Survey Rounds Figure 40: Girls Aged 6-23 Months Consuming Four or More Food Groups During the Previous Day, by Survey Rounds Figure 41: Boys Aged 0-5 Months Exclusively Breastfed During the Previous Day (past 24 hours), by Survey Rounds Figure 42: Girls Aged 0-5 Months Exclusively Breastfed During the Previous Day (past 24 hours), by Survey Rounds