THE SOMALIA HEALTH, PROTECTION AND NUTRITION 2 (SHARPEN 2) PROGRAM FINAL EVALUATION Program Implemented By Catholic Relief Services (CRS) Somalia Submitted on 29th January 2022 Page 2 of 97 ABSTRACT This is a final evaluation report for the Somalia health, protection and nutrition 2 (SHARPEN 2) program. The evaluation was to assess whether the response achieved the desired outcomes and produced evidence-based recommendations to inform future programming. The evaluation sought to: determine achievement against performance targets of select indicators; identify to what extent were beneficiaries actively consulted and engaged in the project; identify program strategies and structures which contributed to or impeded project impact; draw lessons from the project and results achieved to inform future similar programming. The evaluation was conducted through a mixed methods approach (desk review of program documents, qualitative and quantitative interviews). The program relevance was found to be strong while on effectiveness, satisfactory performance was documented in the health sector, unsatisfactory performance was noted in the nutrition, WASH and protection sectors. Program efficiency and consultation and engagement of beneficiaries were strong under the program. Lastly, lessons learned and best practices from the program implementation have been documented and so has recommendations based on the evaluation findings. Cover photo credits: The SOS Children’s Villages Hospital in Garasbaley, Afgooye region which was key in providing health and nutrition services to program beneficiaries. Photo credits @CRS Somalia (2021). Page 3 of 97 LIST OF ABBREVIATIONS AND ACRONYMS AFDB African Development Bank ANC Antenatal Care CFS Child Friendly Spaces CHW Community Health Worker CNW Community Nutrition Worker COVID-19 Coronavirus Disease 2019 CRS Catholic Relief Services CVRS Civil Registration of Vital Statistics DAC Development Assistance Committee DSA Detailed Site Assessment FGD Focus Group Discussion FGM/C Female Genital Mutilation/Cut GAM Global Acute Malnutrition GBV Gender Based Violence GDP Gross Domestic Product GSIYCF Global Strategy on Infant and Young Child Feeding HH Household HIV Human Immunodeficiency Virus HIPC Heavily Indebted Poor Countries HSED Health and Social Economic Development Consortium Limited HSSP Health Sector Strategic Plan IDA International Development Association IDP Internally Displaced Person IMF International Monetary Fund IOM International Organization for Migration IPC Integrated Phase Classification IYCF Infant and Young Child Feeding KII Key Informant Interview MICS Multiple Indicator Cluster Survey MOH Ministry of Health NCD Non Communicable Disease NGO Non-Governmental Organization OCHA United Natation’s Office for the Coordination of Humanitarian Affairs OECD Organization of Economic Cooperation and Development OFDA Office of the United States Foreign Disaster Assistance PNC Post Natal Care PPS Probability Proportional to Size SAM Severe Acute Malnutrition SDG Sustainable Development Goal SHARPEN Somalia Health, Protection and Nutrition SHDS Somali Health Demographic Survey SSWC Save Somali Women and Children TOR Terms of Reference UN United Natation UNCRC United Nations Convention on the Rights of the Child UNICEF United Nations International Children's Fund US United States USAID United States Agency for International Development US$/USD United States Dollar VIP Ventilated Improved Pit WASH Water, Sanitation and Hygiene WFP World Food Program Page 4 of 97 TABLE OF CONTENTS ABSTRACT ...................................................................................................................................................... 2 LIST OF ABBREVIATIONS AND ACRONYMS .......................................................................................... 3 LIST OF TABLES ............................................................................................................................................ 5 LIST OF FIGURES........................................................................................................................................... 5 EXECUTIVE SUMMARY............................................................................................................................... 6 Evaluation Purpose........................................................................................................................................ 6 Evaluation Methodology............................................................................................................................... 6 Evaluation Findings ...................................................................................................................................... 6 Lessons Learned From the SHARPEN II Program Implementation .......................................................... 11 Best Practices in the SHARPEN II Program Implementation .................................................................... 11 Recommendations....................................................................................................................................... 12 SECTION ONE: EVALUATION PURPOSE ................................................................................................ 13 1.1 Introduction........................................................................................................................................... 13 1.2 Purpose of the End Term Evaluation .................................................................................................... 13 1.3 End Term Evaluation Objectives.......................................................................................................... 13 1.4 End Term Evaluation Questions ........................................................................................................... 13 SECTION TWO: BACKGROUND................................................................................................................ 13 2.1 Contextual Background......................................................................................................................... 13 2.2 The SHARPEN II Program................................................................................................................... 15 SECTION THREE: EVALUATION METHODS AND LIMITATIONS...................................................... 16 3.1 End Term Evaluation, Approach, Design and Data Collection Methods.............................................. 16 3.2 End Term Evaluation Data Collection Methods................................................................................... 16 3.3 End Term Evaluation Target Population and Samples Selection.......................................................... 17 3.4 Ethical Considerations in the End Term Evaluation Exercise............................................................... 17 3.5 Data Management and Analysis............................................................................................................ 17 3.6 Challenges and Limitations................................................................................................................... 18 SECTION FOUR: FINDINGS, CONCLUSION AND RECOMMENDATIONS......................................... 19 4.1 Introduction........................................................................................................................................... 19 4.2 Demographic Information of Respondents........................................................................................... 19 4.3 Relevance of the SHARPEN II Program .............................................................................................. 21 4.4 Program Effectiveness .......................................................................................................................... 23 4.5 Program Efficiency ............................................................................................................................... 41 4.6 Beneficiaries Consultation and Engagement......................................................................................... 43 4.7 Lessons Learned From the SHARPEN II Program Implementation..................................................... 45 4.8 Best Practices in the SHARPEN II Program Implementation............................................................... 46 4.9 Conclusion............................................................................................................................................. 46 4.10 Recommendations............................................................................................................................... 47 SECTION FIVE: LIST OF ANNEXES .......................................................................................................... 49 Annex 1: List of Key Informants................................................................................................................ 49 Annex 2: Distribution of beneficiaries and quantitative household survey sample size............................. 49 Annex 3: Analysis of indicators.................................................................................................................. 49 Annex 4: Health-Quantitative Household Survey Tool .............................................................................. 49 Annex 5: Nutrition-Quantitative Household Survey Tool .......................................................................... 49 Annex 6: Protection-Quantitative Household Survey Tool ........................................................................ 49 Annex 7: Water, Sanitation and Hygiene (WASH)-Quantitative Household Survey Tool ........................ 49 Annex 8: Focus Group Discussion (FGD) Guide for Program Beneficiaries............................................. 49 Annex 9: Key Informant Interview Guide for Program Staff and Partners ................................................ 49 Annex 10: Data Quality Control Measures................................................................................................. 49 Annex 11: Training of the Enumerators...................................................................................................... 49 Annex 12: Terms of Reference ................................................................................................................... 49 Page 5 of 97 LIST OF TABLES Table Page Table 1: The SHARPEN II program indicator performance tracking table 8 Table 1.1: Evaluation questions 14 Table 2.1: Description of project goals, objectives and key activities 15 Table 4.1: Distribution of the interviewed beneficiaries by region and sector 18 Table 4.2: Distribution of beneficiaries by gender, age group and position in the household 18 Table 4.3: Education levels among the program beneficiaries 19 Table 4.4: Marital status of the program beneficiaries 19 Table 4.5: Residence of the beneficiaries 20 Table 4.6: Forms of disability among program beneficiaries 20 Table 4.7: Community needs 20 Table 4.8: Source of the nutrition message you received 24 Table 4.9: IYCF knowledge 24 Table 4.10: Presence of a child/children aged below 2 years in the households 24 Table 4.11: Breastfeeding of children aged below 2 years 26 Table 4.12: Food stuff consumed by children aged 6-23 months in the 24 hours preceding the evaluation survey 26 Table 4.13: Fluids consumed by children aged 6-23 months in the 24 hours preceding the evaluation survey 28 Table 4.14: Sources of health messages 28 Table 4.15: Barriers to health care services access 29 Table 4.16: Critical moments for hand washing known by the program beneficiaries 30 Table 4.17: Sanitation facility used in the households 31 Table 4.18: Number of persons sharing a single toilet 31 Table 4.19: Household’s main source(s) of water for drinking, cooking, and hygiene 32 Table 4.20: Satisfaction with WASH items/kits provided 33 Table 4.21: Water and sanitation services supply and utilization 33 Table 4.22: Protection concerns 34 Table 4.23: Post rape/sexual violence services known to beneficiaries 35 Table 4.24: Places where SGBV cases would be reported 35 Table 4.25: Safe spaces 38 Table 4.26: Summary of efficiency in the program 38 Table 4.27: Respect of beneficiaries rights 39 Table 4.28: Awareness of the free hotline 39 Table 4.29: Preferred feedback channels 39 LIST OF FIGURES Figure Page Figure 2.1: Map of Somalia showing household distribution by region 13 Figure 4.1: Prevalence of disability 20 Figure 4.2: Beneficiaries who have received nutrition messages 23 Figure 4.3: Exclusive breastfeeding of children aged 0-5 months 26 Figure 4.4: Consumption of atleast 4 food stuffs by children aged 6-23 months 26 Figure 4.5: Health messaging 27 Figure 4.6: Putting health messages into use 28 Figure 4.7: Awareness of atleast 3 critical moments for hand washing 29 Figure 4.8: Proportion of households using improved sanitation facilities 30 Figure 4.9: Sharing of toilets/latrines 30 Figure 4.10: Proportion of respondents indicating that their source of water had improved over the preceding year 31 Figure 4.11: Households that received WASH kits from the SHARPEN II program 32 Figure 4.12: Percentage of functional hand washing stations in health facilities 32 Figure 4.13: Awareness of places to seek help in case of sexual violence and rape 33 Figure 4.14: Access to SGBV services and reporting of SGBV services 34 Figure 4.15: Households with children aged below 15 years 35 Figure 4.16: Use of the free hotline 39 Page 6 of 97 EXECUTIVE SUMMARY Evaluation Purpose The purpose of the evaluation was to assess whether the response achieved the desired outcomes and produced evidence-based recommendations to inform future programming. The evaluation sought to: determine achievement against performance targets of select indicators; identify to what extent were beneficiaries actively consulted and engaged in the project; identify program strategies and structures which contributed to or impeded project impact; draw lessons from the project and results achieved to inform future similar programming. Evaluation Methodology • A mixed methods approach to data collection was employed for this evaluation and it entailed: o An inception meeting with program staff to get a deeper understanding of the program; o A review of existing secondary source literature and documentation, including program records from its WASH, nutrition, protection and health interventions; o To generate qualitative data, the team led 21 Key Informant Interviews (KIIs) with program staff, partner organizations, health care workers, and district officials (3 females and 18 males); as well as 13 Focus Group Discussions (FGDs) with program beneficiaries (7 female-only and 6 male-only); and o To generate quantitative data, in particular for the relevant SHARPEN logframe indicators, the team carried out a household survey with a total of 2,188 program beneficiaries (1888 females and 300 males) distributed by sector as follows: 286 health beneficiaries (212 females and 74 males), 824 nutrition beneficiaries (782 females and 42 males), 567 protection beneficiaries (512 females and 55 males), and 511 WASH beneficiaries (382 females and 129 males). • To analyze the data generated, the team: o Transcribed and analyzed all qualitative data using flow chart matrices to establish convergence and divergence of themes. A deductive qualitative data analysis approach was used to deconstruct, interpret and reconstruct the responses. o Exported all quantitative data from tablets to MS. Excel sheets and then analyzed the data set using the Statistical Package for the Social Sciences (SPSS) version 23.0. Evaluation Findings Program Relevance [Evaluation Rating: Strong] • From the visited households, sectoral areas of focus under the program were named as major needs for the beneficiaries as follows: health (57.5%), nutrition (47.5%), water (53.1%), hygiene and sanitation (39.9%), protection and security (16.2%), children playgrounds and safe spaces (7.3%), which means that the program contributed to meeting the needs of the targeted beneficiary groups. • In FGDs, beneficiaries shared stories of how the program had reduced and/or eliminated the sufferings they endured in search of water, sanitation facilities, health, and nutrition services an indication of the program’s relevance to the targeted population. • In most KIIs with program staff, partner organizations, health care workers, nutritionists and district officials, stakeholders pointed out that SHARPEN 2 was a continuation of SHARPEN 1 due to the needs observed in the communities even toward the end of the previous phase of the program. In addition, needs assessments were conducted to establish priority areas for interventions and geographic scopes were discussed and agreed upon with the various cluster and technical working groups in Somalia prior to the implementation of SHARPEN 1 program. • The interventions under this program were also found to be in line with the Sustainable Development Goals (SDGs 2,3,5 and 6), 0F 1, 1F 2 the Somalia Humanitarian Response Plan (2021),2F 3 the 9th National Development Plan for Somalia (NDP-9, 2020-2024 pillar 4),3F 4 Somalia WASH Cluster Strategic Operational Framework (SOF), which prioritizes WASH improvements for all Somali nationals,4F 5 the 1Public Private Pact.2020. Somalia and SDG 2030.< https://www.ppp￾sdg.com/services/Somalia%20and%20SDG/index.html#:~:text=MEETING%20THE%20SDGS%20IN%20SOMALIA,aspires%20to%20achieve%20by%202030.&text=This%20partnership%20can% 20be%20used,ambitions%20of%20the%202030%20Agenda. > 2 UNESCO.2020. Claiming Human Rights - in Somalia. 3 OCHA.2021. Somalia: Humanitarian Response Plan 2021 (February 2021). < https://reliefweb.int/report/somalia/somalia-humanitarian-response-plan-2021-february-2021> 4 The Ministry of Planning, Investment and Economic Development, Federal Government of Somalia.2020.Somalia national development plan.<2020 to 2024 http://mop.gov.so/wp￾content/uploads/2019/12/NDP-9-2020-2024.pdf> 5 WASH Cluster Somalia.2018. Guide to WASH Cluster Strategy and Standards also, known as Strategic Operational Framework (SOF).< https://www.humanitarianresponse.info/sites/www.humanitarianresponse.info/files/documents/files/180502_guide_to_wash_cluster_strategy_and_standards_sof.pdf > Page 7 of 97 Essential Package of Health Services (EPHS) and primary health care approach,5F 6 the Astana declaration on primary health care,6F 7 the Somalia interim country strategic nutrition plan (2019-2021),7F 8 and the Somalia national GBV Strategy (2018 – 2020) currently being updated.8F 9 Therefore this program was contributing to the global and national measures to meet the needs of the Somali population. • Lastly, there was relevance in choosing to work with the three local partners due to their extensive local networks and geographical coverage across the country. Program Effectiveness [Evaluation Rating: Unsatisfactory Performance] Nutrition Interventions [Evaluation Rating: Unsatisfactory Performance] • Exclusive breastfeeding (EBF) for children aged 0-5 months in the 24 hours preceding the survey was reported in 61.5% of the households. The baseline score for this indicator was 68.1% (confidence interval of 60.6% to 74.9%) and the target was 75.0%, as such, the program target was underachieved by 13.5%. and this was largely due to the widespread negative cultural practices in the communities. FGDs with program beneficiaries further documented barriers to EBF as: unavailability of food for mothers, engagement in livelihoods activities by mothers, strong beliefs and cultural practices such as feeding babies with water and animal fats, and the perception that mothers are not able to produce adequate milk to exclusively breastfed babies for six months. Some of these barriers have been previously documented in the Somalia Nutrition Strategy (2020-2025).9F 10 • The program had a target of having at least 75% of the children aged 6-23 months receiving at least 4 different food stuffs per day with a baseline score of 47.1% (confidence interval of 39.5% to 54.8%). However, from the end term evaluation, only 43.3% of the households indicated that children in this age category had consumed at least 4 different food stuffs in the 24 hours preceding the survey. Therefore, the program target was underachieved by 31.7%. From the FGDs, the worsening drought in the horn of Africa, food unavailability, inflation and lack of livelihoods opportunities were blamed for limited dietary diversity in the households, all of which have been previously documented in the 2021 Somalia humanitarian needs overview.10F 11 Health Interventions: Good (Evaluation Rating: Satisfactory Performance) • Health awareness campaigns through various forums and channels were conducted under the program with the objective of improving preventive and promotive behaviors and practices in the program sites. From the evaluation, 90.8% of the program beneficiaries could recall three or more health messages against a baseline figure of 69.4% (confidence interval of 63.17% to 75.14%) and an end term target of 85.0% meaning that the program had surpassed the intended target by 4.8%. FGDs indicated that community groups’ membership increased exposure to health messages, providing a critical pathway to influence health promotion and, thus, better health outcomes. WASH Interventions (Evaluation rating: Unsatisfactory Performance] • The program targeted to increase awareness of the five critical moments for hand washing from a baseline score of 78.7% (confidence interval of 72.1% to 84.4%) to a minimum 80.0%. From the end term evaluation, 88.5% of the interviewed WASH beneficiaries were aware of at least three of the five critical moments for hand washing. Therefore, the target for this indicator was surpassed by 8.5%. The critical moments known by the respondents were as follows: after defecation/visiting the toilet (96.9%), after cleaning a child's bottom or changing nappies (77.5%), before feeding a child (81.8%), before eating (93.3%) and before touching and preparing food (65.6%). From the FGDs, a strong linkage between poor perception of safety and food handling and the hygiene of babies was noted with fire expected to kill microorganisms during food preparation and changing of babies being considered to have no food safety risks. • On average, 9 persons shared a single latrine against a baseline figure of 22 and a target of 30 indicating that this program target was surpassed 21 users (70.0%). With latrines having been initially constructed 6 Ministry of Health and Human Services, Federal Government of Somalia, Ministry of Health, Puntland; and Ministry of Health, Somaliland.2014. Somali health policy, prioritization of health policy actions in Somali health sector.< http://www.mohpuntland.com/wp-content/uploads/2016/03/FINAL-Somali_Health_Policy_Directions_and_Priorities-Dec-2014-2.pdf > 7 WHO.2018.New global commitment to primary health care for all at Astana conference.< https://www.unicef.org/press-releases/new-global-commitment-primary-health-care-all-astana￾conference#:~:text=The%20Declaration%20of%20Astana%2C%20unanimously,4)%20align%20stakeholder%20support%20to > 8Food Agricultural Organization of the United Nations-FAO.2019. Somalia interim country strategic plan (2019–2021) .< https://docs.wfp.org/api/documents/536e0ee1ec2e424cb5fab8b177f6d33c/download/> 9 GBV Sub-Cluster Somalia.2018.Somalia National GBV Strategy 2018 – 2020.< https://reliefweb.int/sites/reliefweb.int/files/resources/Somalia%20-%20National%20GBV%20strategy%202018- 2020.pdf> 10Federal government of Somalia.2020.Somalia Nutrition Strategy (2020-2025).< https://www.unicef.org/somalia/media/1756/file/Somalia-nutrition-strategy-2020-2025.pdf> 11 OCHA.2021.2021 Somalia Humanitarian Needs Overview.< https://reliefweb.int/report/somalia/2021-somalia-humanitarian-needs-overview > Page 8 of 97 under SHARPEN 1 program and additional toilets constructed under SHARPEN 2, the number of households sharing latrines was significantly reduced. • Satisfaction with the contents of the WASH kits was 95.4% against a baseline figure of 93.4% and a target of 96%, hence an underachievement of 2.0%. Given that most households used hard water from boreholes, soaps that lathered well were required while beneficiaries did not like the issuance of collapsed types of jerricans. • Satisfaction with the quantity of WASH kits issued was 93.5% against a baseline figure of 82.4% and a target of 90%, an overachievement of 3.5%. Equal quantities of WASH kits were distributed to households regardless of the number of household members hence dissatisfaction by households with large number of beneficiaries (WASH kits recommended by the Somalia WASH cluster serve an average of 6 persons in each household). • Satisfaction with the quality of WASH kits issued was 92.9% against a baseline figure of 93.9% and target of 95% which is a 2.1% underachievement. This dissatisfaction was attributed to low lathering when used with hard water from the boreholes as well as the issuance of collapsed jerricans which beneficiaries did not like. • From the interviews with health care workers and visits in the health facilities, 72.5% of the hand washing stations in health facilities were still against a baseline figure of 100.0% and a target of 95% meaning that this target was underachieved by 27.5%. Follow ups by WASH staff to repair these WASH stations were not factored in the program design while individual health facilities did not take any repairs actions on the hand washing stations. • From the constructed and rehabilitated water sources, households collected an average of 32.5 liters per person per day against a baseline figure of 15 liters and a target of 15 liters indicating that this target was surpassed by 7 liters (46.7%). This was largely due to additional water points constructed and rehabilitated under SHARPEN 2 program which supplemented those under SHARPEN 1 program. • The volume of water supplied per person per day was 15 liters against a baseline figure of 15 liters and a target of 20 liters meaning that this target for this indicator was underachieved by 3.7 liters (24.7%). IDPs movements across regions and program sites increased the need for water and these vulnerable populations could not be denied this precious resource. • 11,900 people were directly utilizing improved water services from the program against an anticipated 14,000, hence an underachievement of 2,100 (15.0%). Although this indicator target appears not to have been met, there were IDPs outside the targeted populations who accessed water and they were not documented as target populations. • 12,900 persons were directly utilizing improved sanitation services provided by under the program against an anticipated figure of 8,000 indicating that the target was surpassed by 4,900 (61.3%). This population of beneficiaries went beyond the target due to newly displaced populations who could not be denied sanitation services available in the IDP camps. Protection Interventions (Evaluation Rating: Unsatisfactory Performance] • Safe spaces established under SHARPEN 1 program were handed over to the communities for management as a sustainability measure and as such, SHARPEN 2 program did not have activities to promote safe spaces uptake. However, the evaluation team followed up on the safe spaces, in 49.5% of the household where children accessed safe spaces, safety and welfare of children was reported to have increased against baseline figure of 47.9% and a target of 75.0% hence an under achievement of 25.5%. From the FGDs, the importance of safe spaces was not fully understood and most caregivers reported them to be useful only for children who were at protection risk or survivors of abuse. Summary of the Program Indicators • As illustrated in Table 1 below, the program met 7 of the 14 program indicators (50.0%) targets assessed; only the health sector met the intended target while the WASH, protection and nutrition sectors either met their targets partially or did not meet them at all. Table 1: The SHARPEN 2 program indicator performance tracking table Sect or Indicators Baseline Value Expe cted chan ge End term evaluation score Conclusi on Comments Sco re 95% confidence interval Target Score 95% confiden ce interval Page 9 of 97 Heal th Percentage of community members who can recall target health education message 69.4 % 63.17% to 75.14% 85% + 90.8% (86.6% males and 90.5% females) 86.7% to 93.9% Target met Community groups membership increased exposure to health messages Nutri tion Proportion of infants 0-5 months of age who are fed exclusively with breast milk (Disaggregated by gender) 68.1 % 60.6% to 74.9% 75% + 61.5% (58.4% males and 65.5% females) 53.1% to 69.4% Target not met The target was not met largely due to social cultural practices, unavailability of food for mothers, engagement in livelihoods activities Proportion of children 6-23 months of age who receives foods from 4 or more food groups, disaggregated by: male and female 47.1 % 39.5% to 54.8% 75% + 43.3% (43.0% males and 43.6% females) 39.4% to 47.4% Target not met Target not met due to worsening drought in the horn of Africa, food unavailability, inflation and lack of livelihoods opportunities WA SH Percent of people targeted by the hygiene promotion program who know at least three (3) of the five (5) critical times to wash hands (Disaggregated by gender) 78.7 % 72.1% to 84.4% 80% + 88.5% (67.4% males and 95.5% females) 85.5% to 91.0% Target met A strong linkage between poor perception of safety and food handling and the hygiene of babies noted Number of people directly utilizing improved sanitation services provided with OFDA funding (Disaggregated by gender) 5,60 0 N/A 8,000 + 12,900 N/A Target met (Data on disaggreg ation by gender not provided) Additional IDPs populations were accessing sanitation facilities Average number of users per functioning toilet 22.1 16.3 to 28.8 30 - 9 6.3 to 12.3 Target surpassed Number of people directly utilizing improved water services provided with OFDA funding (Disaggregated by gender) 11,0 76 N/A 14,000 + 11,900 N/A Target not met (Data on disaggreg ation by gender not provided) There was no documentation of additional IDP populations collecting water from the constructed/rehabilitated water sources Average liters/person/day collected from all sources for drinking, cooking, and hygiene 15 liter s 10.2 to 21.0 liters 15 liters + 32.5 liters 30.9 to 34.1 liters Target met Water sources from SHARPEN 1 program were supplemented by those constructed and rehabilitated under SHARPEN 2 Estimated safe water supplied per participant in liters/person/day 15 liter s 10.2 to 21.0 liters 20 liters + 15 liters 14.25 to 15.75 liters Target not met Additional IDPs populations were accessing water under the program Percent of hand washing stations built or rehabilitated in health facilities that are functional 100. 0% 77.7% to 99.8% 95% + 72.5% 68.3% to 76.3% Target not met Follow ups by WASH staff to repair these WASH stations were not factored in the program design while individual health facilities did not take any repairs actions on the hand washing stations Percent of households targeted by WASH program that are collecting all water for drinking, cooking, and hygiene from improved water sources (Disaggregated by gender) 21.5 % 15.8% to 28.2% 70% + 70.3% (males: 69.0% and 70.7%) 66.1% to 74.2% Target met Water sources from SHARPEN 1 program were supplemented by those constructed and rehabilitated under SHARPEN 2 Percent of households reporting satisfaction with the contents of the WASH NFIs received through direct distribution (i.e. kits) or voucher (Disaggregated by gender) 93.4 % 88.6% to 96.6% 96% + 95.4% (female: 96.6% and males: 94.7%) 92.7% to 97.2% Target not met This dissatisfaction was attributed to low lathering when used with hard water from the boreholes and issuance of the Percent of households reporting collapsed type of jericans satisfaction with the quality of WASH NFIs received through direct distribution (i.e., kits), vouchers, or cash (Disaggregated by gender) 93.9 % 89.2% to 97.0% 95% + 92.9% (female: 93.8% and male: 91.7 %) 89.9% to 95.3% Target not met Percent of households reporting satisfaction with the quantity of WASH NFIs received through direct distribution (i.e., kits), vouchers, or cash (Disaggregated by gender) 82.4 % 75.9% to 87.8% 90% 93.5% (female: 95.6% and male: 92.9 %) 90.5% to 95.7% Target met Equal quantities of soaps were distributed to households regardless of the number of household members hence dissatisfaction by households with large number of beneficiaries Page 10 of 97 Program Efficiency [Evaluation Rating: Strong] Time Efficiency • No delays were reported in the engagement of partner organizations and an initial phase of the grant disbursement from CRS but did not affect the timeliness of the program activities. However, delays in supplies of health products largely caused by the COVID 19 pandemic restrictions were reported in Garsabaalay (Afgooye region). Similarly, in Dollow, there were delays in kick off of some program activities for up to 5 months since community members wanted to take control of car hire for project activities. In addition, in June 2021, there were inter-clan clashes in Dollow among the riverine populations and this led to the suspension of health and nutrition activities and relocation of some of the services to villages occupied by IDPs. • CRS provided the leadership for the consortium and consortium partners reported no challenges in working jointly under the program. The partnership gained greater visibility, by participating sectoral cluster meetings in Somalia. Time Inefficiency • Across all the program sites, ambulance services were appreciated but not timely due to geographical vastness and insecurity at night. As such there were inevitable delays in accessing health services especially at night. • In nutrition, health and WASH interventions, a high number of staff was required especially due to the parallel COVID-19 mitigation measures hence putting strain on staff. This high demand for health and nutrition services was in some instances associated with long queues and long waiting time in health facilities, nutrition clinics and WASH services access points. Cost Efficiency • Each consortium partner was responsible for its own total share of the budget and allocated across the respective work packages and budgets were “locked in”, hence little flexibility to manoeuvre the budgets since approved work plans and budget lines were strictly followed. • The implementation of individual sectoral activities was high but the transformation of inputs to outputs was suboptimal given that only half of the targeted program outcomes were achieved. • Several cost cutting measures were noted in the program including: set up of safe spaces in the health facilities rather than constructing new one; hygiene kits were obtained from the Somalia WASH cluster’s Regional Supply Hub instead of procuring them from a central store which would have been costly; the project met only the cost of transporting them; RUSF was procured from Ethiopia instead of Kenya to reduce the cost of transportation; and distribution of interventions according to the strengthen and geographical presence of the partners ensured that programming was cost effective by leveraging on the existing networks, infrastructure, staff and facilities. However, in Hudur, transportation costs were higher than anticipated due to security lock downs. Cost Inefficiency • Under the partnership, there were no protocols and practices to ascertain that good practices and lessons learned were recognized and integrated into work practices. • Results-based management principles were not fully exploited by the project and some commitments were not properly followed up on, for example, nutrition and WASH behaviour change messages required follow ups to ascertain whether they were being put into action and the gaps thereof, however, this was not done. Consultation and Engagement of Beneficiaries [Evaluation Rating: Strong] • The household survey revealed that 82.4% of program beneficiaries indicated that their rights were fully respected and upheld under the program, 14.6% indicated partial respect and upholding of their rights while 3.0% were of their opinion that their rights were not upheld and not respected (including 6.9% of the protection beneficiaries). • From the KIIs and FGDs, IDP camp leaders and village committee leaders were used to get feedback from the beneficiaries on the various services offered by the partner organizations • Under the program, a hotline was available for provision of feedback on the services offered as well as to call for emergency assistance. However, only 65.1% of the program beneficiaries (69.0% females and 40.7% males) reported being aware of this hotline. Of those aware of the free hotline, only 46.7% had used it (47.7% females and 40.3% males). Given that the hotline was only used in emergency cases and/or when in need of information, the uptake of this platform was sub-optimal. • Preferred feedback channels were reported as the hotline (53.2%), community and camp leaders (50.3%), program staff (24.6%), and 9.8% indicated phone short message services-SMSs (9.8%) and suggestion Page 11 of 97 boxes (2.3%) all of which point out to demand for other complementary feedback platforms in addition to the hotline. Lessons Learned From the SHARPEN 2 Program Implementation Nutrition Interventions • Nutrition interventions without food security measures in drought affected populations will improve knowledge but not practices if food sources remain unavailable or inaccessible. There are strong knowledge levels on IYCF and good nutrition in general, but those practices are not followed due negative coping strategies employed due to food shortages in the households. • Context-specific nutrition messaging for men ought to be developed and rolled out, following their daily routines and socialization patterns in order to ensure effective uptake. Men showed lower knowledge levels of ICYF and good nutrition practice and, unlike their female counterparts, there were no support groups for awareness creation or education for male beneficiaries. Health Interventions • Identification of training courses for health workers need to be guided by demand rather than implementers perceived training gaps. Across the health facilities, health care workers asked for these trainings to be guided by their preferences and needs. • Contingency measures for procurement and supply of medicines ought to be put in place in pandemics and fragile nations with rapidly changing markets. Delays in supply of drugs for health facilities were noted in Baidoa and the Afgooye corridor due to the COVID-19 pandemic related restrictions. • In view of sparse distribution of health facilities, unavailability of transport services, insecurity and the absence of a working health emergency response system, community health workers are useful in supplementing static health facilities. WASH Interventions • Use of hard water influences the preferred soaps by beneficiaries while the satisfaction with WASH kits is influenced by user preferences of beneficiaries as well as the number of household members in families. Protection Interventions • Protection shelters should not only target females but also males. From qualitative interviews in Adaado and Luuq, both females and males called for protection shelters for boys who are being forcefully recruited into militia groups. All Interventions • Empowerment of community structures (e.g. gatekeepers in the IDP camps, village committees, and water management committees) through capacity building, consultation and collaboration enhances awareness, involvement and buy in of communities increased chances of sustainability. • Conflict sensitive programming – to ensure delays are averted in future and avoid harm to beneficiaries due to our interventions, there is need to have a conflict sensitive lens when designing new projects. Project teams should understand the tensions that exist and potential connectors. In Dollow, delays were noted due to conflict of interest over hire of project vehicles. • Geographic shifting of activities during crises to meet needs when possible helps provide critical services to those who need it most; for example, SHARPEN II shifted interventions to the riverine IDPs at their new displacement villages in Laascaanood from Dollow. This further indicates the need for regular review of the program activities vis-à-vis the community needs and flexibility in the program plans and finances. Best Practices in the SHARPEN 2 Program Implementation • Use of regular patients’ feedback surveys in the SOS Children’s Villages health facilities helped in documenting health service delivery gaps and improving on the same (e.g. long waiting time, stock outs and poor communication by health care workers). This was reported by mothers seeking delivery services in SOS Children’s Villages health facilities. • Establishment of child friendly spaces in health facilities further supported the protection of children while lowering barriers to addressing their health and psychosocial needs. • Mobile health teams able to deliver services closer to hard to reach areas and also provide an avenue for follow up of children under treatment. This is supported by community surveillance mechanisms more strongly when compared to static health facilities. • Holding regular review meetings with various stakeholders and partners including other international NGOs, local NGOs, sectoral cluster groups and the Ministry of Health to evaluate progress and share experiences was found to be a catalyst for decision making to address emerging issues in the Somaliland drought mitigation. Page 12 of 97 Recommendations Nutrition • Explore the best approaches to improve nutrition messaging targeting males. This could include religious leaders who have much respect from males and other male dominated forums. • Inclusion of crisis modifier budgets and consideration of resilience activities as part of the project to ensure sustainability of nutrition interventions that are linked to food security. Health • Ensure that trainings offered to health care workers are aligned to their needs by conducting training needs assessments. • Ensure stock pre-positioning systems are in place to respond when there is an emergency, such as the COVID-19 pandemic, which may hinder fast procurement of health products and technologies. • Continue supplementing static health services with community outreaches and primary health care services through community workers in view of sparse distribution of health facilities, unavailability of transport services, insecurity and the absence of a working health emergency response system. Protection • Come up with outcome indicators for monitoring under the protection sectors-Under SHARPEN 1 there was only one outcome indicator and under SHARPEN 2 program there was no outcome indicator thus difficulties in evaluation the performance of this sector. • Establish safe spaces for youthful males who are being forcefully recruited into militias against their wishes in Adaado, Luuq and Dollow areas. Safe spaces were not targeted by programming under SHARPEN 2 and as such, awareness on the same should be prioritized. • Factor in family strengthening and kinship interventions in view of the high number of children either at risk of losing parental care or those who have already lost parental care. • Create more awareness on the dangers of physical and humiliating punishment for children which remains rampant in the visited program sites. • Create more awareness on legal and psychosocial services available for survivors of sexual violence. As it is, there is little information on these. • Create more awareness on available protection shelters and safe spaces. There is demand for these services but awareness on the same is lacking. Safe spaces were not targeted by programming under SHARPEN 2 and as such, awareness on the same should be prioritized • Ensure services offered in safe spaces and child friendly centers meet the needs of the beneficiaries. Regular satisfaction survey will help document gaps in these services for upfront remedy. • Improve record keeping and data management practices-Data on psychosocial services was not provided though a request for the same (disaggregated by gender and age) was made. • Support policy dialogue sessions to address the longstanding practice of early and forced marriages. WASH • During the program design stage, factor in the influx of IDPs populations which may increase the utilization of water and sanitation services. • Prioritize solar lamps on latrines and locks on latrine doors especially in IDPs camps. Darkness and the insecurity that comes with accessing latrines was described a major contributor to open defecation especially in Baidoa and the Afgooye corridor. • Establish strategic waste disposal pits in the IDP camps for ease of management of household solid wastes. Environmental health was not targeted by the SHARPEN 2 program interventions, but the beneficiaries have made a request for waste disposal pits. • Ensure that WASH kits supplied are informed by preferences of the targeted beneficiaries for uptake and satisfaction purposes. Beneficiaries had their own preferences for soaps that lather well when used with hard water while the number required should be guided by the actual number of household members rather than an estimated average of family size. • Replace the collapsed forms of jerricans with the non-collapsed ones which are more preferred in the Somalia context. • Create more awareness on the need to wash hands before touching and preparing food which are poorly understood by the program beneficiaries. Page 13 of 97 SECTION ONE: EVALUATION PURPOSE 1.1 Introduction This is a draft report for the end term evaluation of the “Somalia Health, Protection and Nutrition 2 (SHARPEN 2) program” end term evaluation. The program was implemented in Somalia by Catholic Relief Services (CRS) in partnership with Save Somalia Women and Children (SSWC), SOS Children’s Villages Somalia (SOS), and Trócaire Somalia from August 1, 2020, to September 30, 2021 in Mogadishu, Afgooye Corridor, Baidoa, Cadaado, Dollow, and Luuq, while expanding the successful, integrated approach to Xudur, Garbaharey and Burdhubo. The end term evaluation was conducted in the month of December 2021 by the HSED Group Africa, a Horn of Africa based research advisory firm (http://www.hsed.co.ke/). 1.2 Purpose of the End Term Evaluation The primary purpose of the end term evaluation was to provide a concise assessment of the achievement of the project against project objectives, outcomes and outputs and subsequently form the basis for the project closure decision in addition to producing evidence based recommendation for future programming.11F 12&12F 13 1.3 End Term Evaluation Objectives Specifically, the end term evaluation sought to: 1) Determine achievement against performance targets of select indicators; 2) Identify to what extent were beneficiaries actively consulted and engaged in the project; 3) Identify program strategies and structures which contributed to or impeded project impact; and 4) Draw lessons from the project and results achieved to inform future similar programming. 1.4 End Term Evaluation Questions To achieve the objectives of this assignment, the evaluation team targeted to review and answer questions on relevance, effectiveness, efficiency, and lessons learnt in the program implementation as detailed in Table 1.1 below. Table 1.1: Evaluation questions Component Evaluation Questions Source of Data Relevance • To what degree did the project meet the needs of target beneficiary populations? • Program documents • Feedback data • Beneficiaries • Program staff • Camp leaders Effectiveness • To what extent were the sector specific objectives of the project achieved? • To what extent were beneficiaries actively consulted and engaged in the project especially in their ability to provide feedback through partner’s accountability mechanisms? • Beneficiaries • Program staff • Program partners • Camp leaders Efficiency • To what extent did the implementation process, including delivery options and models, ensure efficient use of value for money; including: management structures, partner roles and coordination, humanitarian coordination between other actors? • Program documents • Program staff • Program partners Lessons learnt • Were there lessons learnt in the program implementation? • Program staff • Program partners • Camp leaders SECTION TWO: BACKGROUND 2.1 Contextual Background The Federal Republic of Somalia is a long, narrow country that wraps around the Horn of Africa.13F 14 Somalia is bounded by the Gulf of Aden to the north, by the Indian Ocean to the east, by Kenya (684 km) and Ethiopia 12Thomson, G. & Hoffman, J. 2003. Measuring the success of EE programs. Canadian Parks and Wilderness Society. 13Patton, M.Q. 1987. Qualitative Research Evaluation Methods. Thousand Oaks, CA: Sage Publishers. 14Lewis T.2009.Somalia.EthnoMed. Page 14 of 97 (1640 km) to the west, and by Djibouti to the northwest (61 km).14F 15 The country’s total area is 637 657 km2 , with a coastline of 3,025 km which is the longest coast of any African nation, bordering on both the Red Sea and the Indian Ocean.15F 16 The inland areas are predominantly plateaus, with the exception of some rugged mountains in the far north. The northern region is more arid, whereas the southern portion of the country receives more rainfall. Many Somalis are nomadic or semi-nomadic herders, some are fisher people, and some farmers.16F 17 The year is subdivided into four seasons as follows: Jilal, Gu, Hagga and Deyr.17F 18 Somalia’s current population is estimated at 15,442,905 and women represent nearly half of the adult population and 4 of 10 households are headed by females.18F 19The territory of Somalia is de facto divided into three distinct administrative areas: Somaliland (a selfdeclared independent state, not recognised by the international community), Puntland (a -selfdeclared- autonomous state of Somalia) and the area south of Puntland, from Mudug region to the south, referred to as South/Central Somalia (Figure 1.1).19F 20 Figure 2.1: Map of Somalia showing household distribution by region20F 21 Plagued by recurrent natural disasters and decades of armed conflict and compounded by widespread and protracted displacement, Somalia has seen the disruption of critical infrastructure and even the most basic services including health and education. Services and opportunities are projected to remain severely strained in the years to come in most parts of Somalia – especially for internally displaced families and rural communities.21F 22 Somalia’s path to political and security stabilization and development trajectory faces many challenges and multiple shocks. 22F 23 A sustained period of political and institutional progress reflects a country transitioning out of fragility and protracted crisis. Somalia reached the Decision Point of the Heavily Indebted Poor Countries (HIPC) initiative on March 25, 2020, restoring the country’s access to regular concessional financing and launching the process toward debt relief. It cleared its arrears to the African Development Bank (AFDB), the International Monetary Fund (IMF) and the International Development Association (IDA), and reduced its external debt to $3.9 billion (78% of the revised 2020 gross domestic product (GDP) from $5.3 billion.23F 24However, an incomplete political settlement, vulnerability to shocks (such as climate related disasters, locust’s infestation and floods) are jeopardizing the recovery from fragility.24F 25 15Central Intelligence Agency (CIA), World Factbook.2021.Somalia (Geography), last updated: 24 November 2020.< https://www.cia.gov/the-world-factbook/countries/somalia/> 16European Asylum Support Office (EASO).2014. EASO Country of Origin Information report: South and Central Somalia Country overview. 17Lewis T.2009.Somalia.EthnoMed. 18Food Security and Nutrition Analysis Unit (FSNAU).2021.Somalia climate. 19The World Bank.2020. Population, total – Somalia.< https://data.worldbank.org/indicator/SP.POP.TOTL?locations=SO> 20UNHCR.2010.Eligibility Guidelines for Assessing the International Protection needs of Asylum Seekers from Somalia, 5 May 2010. 21 https://www.worldatlas.com/maps/somalia 22UN High Commissioner for Refugees.2019. UNHCR Somalia Factsheet: 1 - 30 September 2019.< https://reliefweb.int/report/somalia/unhcr-somalia-factsheet-1-30-september-2019>. 23UNICEF.2016.Situation analysis of children in Somalia 2016. 24The World Bank.2021. The World Bank in Somalia: Overview. 25The World Bank.2021. The World Bank in Somalia: Overview. Page 15 of 97 2.2 The SHARPEN 2 Program Guided by the successful implementation of SHARPEN 1 program, CRS proposed SHARPEN 2 which has provided an integrated package of basic life-saving services to crisis-affected Somalis in Mogadishu, Afgooye Corridor, Baidoa, Xudur, Cadaado, Luuq, Dollow, Garbaharey and Burdhubo. This package includes access to primary health care through static and mobile facilities; a complete Integrated Management of Acute Malnutrition (IMAM) program; prevention of gender-based violence (GBV) and comprehensive services for survivors, including children who are victims of GBV; child friendly spaces that promote protection, health and nutrition of children; integrated hygiene, health and nutrition sensitization; critical water, sanitation, and hygiene (WASH) infrastructure, and distribution of hygiene supplies coupled with hygiene messaging. CRS had been overseeing and coordinating three implementing partners: Save Somalia Women and Children (SSWC), SOS Children’s Villages Somalia (SOS) and Trócaire Somalia. This project implemented from August 1, 2020, to September 30, 2021, continues essential service provision funded by OFDA in Mogadishu, Afgooye Corridor, Baidoa, Cadaado, Dollow, and Luuq, while expanding the successful, integrated approach to Xudur, Garbaharey and Burdhubo. Specific activities and interventions were chosen for each catchment area based on the strengths and capacities of each organization, the needs of the targeted communities, complementarity with funding received from other donors, lessons learned through consultation with participants of past and ongoing projects and the feasibility of interventions. In addition, construction and rehabilitation of Health and WASH infrastructure provides facilities that meet SPHERE and national guidelines. The project had four broad multi sectoral objectives, these being to ensure: target populations access comprehensive primary care and have reduced morbidity; malnutrition levels in young children (under 5) and Pregnant/ Lactating Women (PLWs) are decreased; the risk of GBV among vulnerable groups (including children) is reduced and survivors of GBV access comprehensive services; and vulnerable populations access safe water, improved sanitation and hygiene practices, and behaviors that decrease malnutrition. The project goal, sectors, objectives, and activities are summarized in Table 1.1 below. Table 2.1: Goals, objectives and key activities under SHARPEN 2 program Sector (Beneficiaries) Health (164,441) Nutrition (87,097) Protection (44,100) WASH (69,686) Objective Access to comprehensive primary care and have reduced morbidity. Malnutrition levels in young children (<5) and pregnant/lactating women (PLWs) are decreased. The risk of GBV against vulnerable groups (including children) is reduced and survivors of GBV access comprehensive services. Vulnerable populations access clean water and have improved hygiene. Approach CRS and its partners support static and mobile health clinics, who use community health workers to extend service provision to hard-to-reach IDP populations who cannot easily access existing facilities. CRS and its partners use the Basic Nutrition Service Package for Somalia protocol (as recommended by the Somalia Nutrition Cluster). CRS uses a combination of strategies to decrease malnutrition, including household level screening, treatment at health facilities, IYCF and mother-to-mother support groups and targeted nutrition messaging. CRS and its partners work within prevention and response to gender-based violence and child protection and psychosocial support services to achieve this objective. GBV survivors access comprehensive services including medical, legal and psycho-social. Furthermore, the project works to raise GBV awareness and mitigation strategies among target communities. Partners also participate in child protection through the provision of child friendly spaces (CFS). CRS and its partners undertake extensive hygiene promotion campaigns, integrated with nutrition messaging at facilities and HH-level through leveraging partner networks of CHWs and CNWs to deliver integrated hygiene messaging. The project supports the construction of latrines, hand washing stations and shallow wells to benefit both IDP and host community populations. SECTION THREE: EVALUATION METHODS AND LIMITATIONS 3.1 End Term Evaluation, Approach, Design and Data Collection Methods 3.1.1 End Term Evaluation Approach A mixed methods approach was employed for this evaluation entailing a desk review of e literature and program documents, Focus Group Discussions (FGDs), quantitative interviews with program beneficiaries and Key Informant Interviews (KIIs) with program staff and representatives of partner organizations with these multiple methods being aimed at ensuring triangulation of findings25F 26&26F 27 3.1.2 End Term Evaluation Design This summative program was of a non-experimental design with individual indicators analysis and comparison against set targets.27F 28 A descriptive cross-sectional study was used to establish the snapshot program outcomes among the beneficiaries following exposure to interventions.28F 29 The end term evaluation was conducted among program beneficiaries, staff, partners and other stakeholders across program sites in Mogadishu, Afgooye Corridor, Baidoa, Cadaado, Luuq, Dollow, Hudur and Burdhubo regions of South Central Somalia where the program was implemented. 3.2 End Term Evaluation Data Collection Methods 3.2.1 Desk Review of Program Documents A desk review of documents preceded field visits and this was useful in fine-tuning the evaluation methodology, formulation of evaluation questions and development of evaluation data collection tools. 3.2.3 Quantitative Data Collection: Household Survey of Program Beneficiaries Quantitative data was collected from 2,188 program beneficiaries (1888 females and 300 males) distributed by sector as follows: 286 health beneficiaries (212 females and 74 males), 824 nutrition beneficiaries (782 females and 42 males), 567 protection beneficiaries (512 females and 55 males), and 511 WASH beneficiaries (382 females and 129 males). Simple random sampling was used in the identification of program beneficiaries in all the program sites. Thirty enumerators (15 male and 15 female) were trained for two days (Annex 11) and they conducted the quantitative data collection for ten days using the KoBoCollect mobile data collection platform.29F 30 3.2.4 Qualitative Data Collection: Key Informant Interviews (KIIs) & Focus Groups Discussions (FGDs) 21 Key Informant Interviews (KIIs) with program staff, partner organizations, health care workers, and district officials (3 females and 18 males) were conducted in the field (Annex 1). KIIs respondents were identified through purposive sampling aimed at ensuring that only persons knowledgeable with the program activities were included in the evaluation. A total of 13 FGDs (7 female-only and 6 male-only) were conducted with program beneficiaries across the program sites. Each FGD had 6 participants and was conducted by a moderator and a note taker. FGDs participants were selected through snowball sampling with beneficiaries being asked to identify other persons in the3 villages or OIDP camps who had received interventions offered under the program. 3.2.4 Direct Observation Direct observation was used to examine and validate and document completion and use of infrastructure in health facilities, water points and other sanitation facilities. 26Dopp A.R., Mundey, P., Beasley, L.O. et al.2019. Mixed-method approaches to strengthen economic evaluations in implementation research. Implementation Sci 14, 2 (2019). https://doi.org/10.1186/s13012-018-0850-6. 27Grey literature is published research materials and documents while white literature refers to routine reports and technical documents. 28Thompson C B.2007.Research Study Designs: Non-experimental .Air Medical Journal Associates 26:1, doi:10.1016/j.amj.2006.10.003. 29Setia MS.2016. Methodology Series Module 3: Cross-sectional Studies. Indian J Dermatol. 2016 May-Jun; 61(3): 261–264. Doi: 10.4103/0019-5154.182410 30www.kobotoolbox.org/tags/kobocollect Page 17 of 97 3.3 End Term Evaluation Target Population and Samples Selection From program records, CRS and partner organizations reached 438,916 beneficiaries with various interventions as follows: 176,267 beneficiaries with health interventions, 129,370 with nutrition interventions, 24,876 with protection interventions and 108,403 with WASH interventions as illustrated in Annex 2. Sample sizes for various respondents across thematic interventions areas were computed using the Feed the Future Survey Implementation Guidelines provided by CRS, a 10% change in indicators and 10% non-response rate.30F 31 From the guidelines, 1,020 households from the nutrition beneficiaries were required while 680 respondents from the protection, WASH and Health beneficiary groups are required (total number of respondents=3,060). In the absence of village/IDP camps wise (cluster level) beneficiary data, the samples are distributed across the districts based on the population of beneficiaries (Annex 2). Respondents were identified through simple random sampling across the program sites. 3.4 Ethical and Safety Considerations in the End Term Evaluation Exercise The research team adhered to the CRS beneficiaries’ safeguarding protocols, the USAID evaluation policy, CRS – MEAL policies and procedures and other universally accepted research ethical measures including: independence and impartiality; culturally meaningful approaches to informed consent process, detailed enough to provide information on risks and benefits of participation in the study will be developed; voluntary participation without coercion will be ensured; confidentiality of the responses and the data will be ensured; there will be no risks the end term evaluation interview participants; the cultural, religious and traditions of study populations and communities will be respected; and feedback will be provided to the study participants and community respondents. In addition, the team of enumerators received training on the ethics with emphasis on issues of being sensitive in the questioning process and framing questions within the acceptable cultural values and norms, - free from judgmental phrasing. The enumerators made sure that the respondents fully understood the background and the objectives of the evaluation exercise before starting the interviews. Verbal informed consent was obtained from all qualitative and quantitative questionnaires respondents. All participants were informed about the interview procedures and the voluntary nature of their participation; assured of confidentiality; and informed that no adverse consequences would arise if they declined participation. No identifying markers were listed on any of the data collection tools but the names of KIIs participants were recorded. 3.5 Data Management and Analysis 3.5.1 Data Quality Control Measures and Training of Enumerators Details on data quality control measures adopted in the evaluation and training of the enumerators are provided as Annexes 10 and 11 respectively. 3.5.2Qualitative Data Management and Analysis Qualitative data was transcribed and analyzed using flow chart matrices to establish convergence and divergence of themes. A deductive qualitative data analysis approach was used where a predetermined structure based on research questions guided the analysis process. The following steps were followed: transcription; translation of the responses; deconstruction; interpretation; reconstruction; and establishing convergence and divergence in patterns and themes. 3.5.3 Quantitative Data Management and Analysis The quantitative survey data set from the households’ survey was exported to MS. Excel sheets and then exported into the Statistical Package for the Social Sciences (SPSS) version 23.0. Labeling of variables was be done, data cleaning carried out including checking of outliers, missing data imputation and variable transformation. This was an iterative procedure that took place throughout the entire analysis. All data cleaning steps were documented on a syntax file. Descriptive analyses was conducted to extract frequencies, percentages, means, medians and standard deviations computed in the analysis. Exploratory analyses statistic included 95% confidence intervals (CI), cross tabulations and correlations of the program outcomes and comparison with baseline and target values. 31Diana Maria Stukel. 2018. Feed the Future Population-Based Survey Sampling Guide. Washington, DC: Food and Nutrition Technical Assistance Project, FHI 360. Page 18 of 97 3.6 Challenges and Limitations 1) Under SHARPEN 2 program, additional program sites received interventions (Dollow, Luuq and Burdhubo regions). The beneficiaries in these regions were exposed to program interventions for one year unlike those in other sites that had interventions for a duration of 2 years (SHARPEN 1 program and SHARPEN 2 program interventions). Therefore, this may be an explanation for why the scores on program indicators in the new sites was largely lower than the sites where SHARPEN 2 continued from SHARPEN 1. 2) In some program sites, the number of respondents targeted with quantitative interviews was not achieved due insecurity and migration of IDPs households. This low sample size resulted in commensurately lower statistical power and increased margin of error. 3) In parts of Caadado and Hudur, the research teams were ambushed by militias and as such were not able to facilitate FGDs. As such, only 13 of the 16 planned FGDs were conducted hence reduced variability in the qualitative responses. 4) The scores on the nutrition indicators are largely affected by the current drought in Somalia. The FSNAU-FEWS NET 2021 Post Gu Technical Release dated September 9, 2021 approximated that 3.5 million people in Somalia faced acute food insecurity Crisis (IPC Phase 3) or worse outcomes in late 2021. Specifically, the report indicates that 7,178,500 persons in Somalia faced food insecurity between October and December 2021 with 3,712,900 persons (51.7%) in the stressed phase, 2,824,960 (39.4%) in the crisis phase and 640,730 (8.9%) in the emergency phase.31F 32 5) In Dollow and Burdhubo, conflicts displaced populations and thus partner organizations had to follow them up in IDP camps thus slowing down the implementation of program activities. Insecurity in these two sites also limited the number of nutrition beneficiaries interviewed given that the specific age groups required for EBF and food diversity (specifically 38 and 20 nutrition beneficiaries were interviewed in Dollow and Burdhubo sites). 32FSNAU-FEWS NET.2021.FSNAU-FEWS NET 2021 Post Gu Technical Release - September 9, 2021.< https://fsnau.org/node/1891> Page 19 of 97 SECTION FOUR: FINDINGS, CONCLUSION AND RECOMMENDATIONS 4.1 Introduction The findings presented in this section are based on: a desk review of program documents, a quantitative survey with 2,188 program beneficiaries (1888 females and 300 males) distributed by sector as follows: 286 health beneficiaries (212 females and 74 males), 824 nutrition beneficiaries (782 females and 42 males), 567 protection beneficiaries (512 females and 55 males), and 511 WASH beneficiaries (382 females and 129 males) as illustrated in Table 4.1 below; 13 FGDs with program beneficiaries (7 with female groups and 6 with male groups); 21 KIIs (3 females and 18 males)with program staff, partner organizations, health care workers and district officials; and triangulation of secondary literature. Table 4.1: Distribution of the Evaluation respondents, by region and sector Sector Region Total Mogadishu Afgooye Cadaado Baidoa Luuq Dollow Hudur Burdhubo Health 1.0% 11.7% 0.0% 25.4% 25.8% 17.6% 26.1% 40.0% 13.1%(286) Nutrition 56.6% 52.2% 0.0% 13.7% 32.0% 16.0% 73.9% 20.0% 37.7%(824) Protection 22.0% 30.0% 100.0% 40.2% 0.0% 0.0% 0.0% 0.0% 25.9%(567) WASH 20.4% 6.1% 0.0% 20.6% 42.3% 66.4% 0.0% 40.0% 23.4%(511) Total 100.0% (677) 100.0% (410) 100.0% (178) 100.0% (291) 100.0% (194) 100.0% (250) 100.0% (88) 100.0% (100) 100.0% (2188) 4.2 Demographic Information of Respondents From the household survey, a total of 2,188 beneficiaries were interviewed (translating to a response rate of 71.5%) these being 86.3% females and 13.7% males, 27.8% in the age group 18-25 years, 38.1% in the age group 26-35 years, 20.4% in the age group 36-45 years, 6.9% in the age group 46-55 years and 6.8% above the age of 55 years. By position in the households, 79.6% were heads of households while 20.4% were senior most females. Given the high number of female respondents and the high number of heads of households, there was a significant number of female headed households in the program sites while gender breakdown aligns in part with the type of services offered in each location (Table 4.2). Table 4.2: Distribution of Respondents Characteristics Mogadish u Afgooye Caadad o Baidoa Luuq Dollow Hudur Burdhub o Total Gen der Female 94.4% 98.0% 92.1% 93.5% 90.2% 47.2% 98.9% 31.0% 86.3% Male 5.6% 2.0% 7.9% 6.5% 9.8% 52.8% 1.1% 69.0% 13.7% Total 100.0% (677) 100.0% (410) 100.0% (178) 100.0% (291) 100.0% (194) 100.0% (250) 100.0 % (88) 100.0% (100) 100.0% (2188) Age Cat egor y 18-25 Years 41.7% 40.2% 20.2% 30.6% 0.0% 0.4% 17.0% 20.0% 27.8% 26-35 Years 42.4% 43.2% 37.6% 35.4% 52.1% 27.6% 1.1% 28.0% 38.1% 36-45 Years 11.1% 10.5% 25.8% 23.0% 41.2% 46.4% 0.0% 19.0% 20.4% 46-55 Years 3.0% 2.7% 13.5% 5.8% 6.7% 20.4% 0.0% 16.0% 6.9% 55+ Years 1.9% 3.4% 2.8% 5.2% 0.0% 5.2% 81.8% 17.0% 6.8% Total 100.0% (677) 100.0% (410) 100.0% (178) 100.0% (291) 100.0% (194) 100.0% (250) 100.0 % (88) 100.0% (100) 100.0% (2188) HH posi tion Head of the household (male or female) 85.7% 73.9% 96.1% 66.7% 100.0% 80.4% 2.3% 96.0% 79.6% Senior most female in the household 14.3% 26.1% 3.9% 33.3% 0.0% 19.6% 97.7% 4.0% 20.4% Total 100.0% (677) 100.0% (410) 100.0% (178) 100.0% (291) 100.0% (194) 100.0% (250) 100.0 % (88) 100.0% (100) 100.0% (2188) In terms of place of residence, 60.2% of the households visited for interviews were those of IDPs while 39.8% were those of host communities. Caadado and Luuq program sites had the highest proportion of IDP households (97.8% and 100.0% respectively) as illustrated in Table 4.3. Page 20 of 97 Table 4.3: Residence of the beneficiaries Residence Mogadis hu Afgooye corridor Cadaad o Baidoa Luuq Dollow Hudur Burdh ubo Total IDPs 42.8% 94.4% 97.8% 77.0% 100.0% 0.8% 34.1% 16.0% 60.2% Host community 57.2% 5.6% 2.2% 23.0% 0.0% 99.2% 65.9% 84.0% 39.8% The education levels were relatively low among the respondents, with 47.3% having never been to school, 34.5% having been to religious/Koranic schools only, 6.8% having attended informal schools, 5.9% having been to primary schools without completion, 2.9% having completed primary school level of education, 1.2% having been to secondary schools without completion and 1.4% having completed secondary schools (Table 4.3) with females recording poorer levels of education. Table 4.4: Education levels among the program beneficiaries Education level None Total Informal educatio n Religious education Primary school incomplete Primary school complete Secondary school incomplete Secondary school complete Sit e Mogadi shu 45.2% 6.9% 39.3% 5.2% 1.8% 0.7% 0.90% 100.0%(677) Afgooy e corridor 70.0% 0.5% 22.0% 5.9% 1.7% 0.0% 0.00% 100.0%(410) Cadaad o 23.6% 20.8% 27.5% 15.2% 8.4% 3.9% 0.60% 100.0%(178) Baidoa 60.5% 5.8% 23.4% 8.6% 0.7% 0.3% 0.70% 100.0%(291) Luuq 90.7% 6.7% 2.6% 0.0% 0.0% 0.0% 0.00% 100.0%(194) Dollow 7.6% 2.8% 80.8% 4.8% 4.0% 0.0% 0.00% 100.0%(250) Hudur 19.3% 21.6% 53.4% 4.5% 1.1% 0.0% 0.00% 100.0% (88) Burdhu bo 13.0% 7.0% 28.0% 2.0% 16.0% 13.0% 21.00% 100.0%(100) Lo cat ion IDP camps 62.0% 6.3% 22.4% 6.2% 2.0% 0.8% 0.40% 100.0%(1317) Host commu nity 25.3% 7.6% 52.8% 5.4% 4.2% 1.8% 2.90% 100.0% (871) Ge nd er Female 52.8% 6.8% 31.5% 5.5% 2.0% 0.8% 0.60% 100.0%(1888) Male 13.3% 7.0% 53.3% 8.3% 8.3% 3.7% 6.00% 100.0%(300) Total 47.3% 6.8% 34.5% 5.9% 2.9% 1.2% 1.40% 100.0%(2188) On marital status, 82.6% of the beneficiaries reported to be married, 8.4% were divorced (9.5% females and 1.7% males), 4.1% were widowed, 2.6% were separated and 2.3% were single (Table 4.4). Table 4.4: Marital status of the program beneficiaries Single Married Separated Divorced Widowed Total Site Mogadishu 2.1% 84.2% 0.6% 10.5% 2.6% 100.0% (677) Afgooye corridor 3.7% 81.5% 0.2% 9.5% 4.9% 100.0% (410) Cadaado 3.9% 75.8% 8.4% 11.8% 0.0% 100.0% (178) Baidoa 1.7% 83.5% 0.7% 6.2% 7.9% 100.0% (291) Luuq 0.0% 84.0% 8.8% 5.7% 1.5% 100.0% (194) Dollow 0.0% 84.0% 5.2% 5.6% 5.2% 100.0% (250) Hudur 0.0% 97.7% 0.0% 1.1% 1.1% 100.0% (88) Burdhubo 10.0% 67.0% 4.0% 9.0% 10.0% 100.0% (100) Location IDP camps 2.6% 80.5% 2.7% 9.6% 4.5% 100.0% (1317) Host community 2.0% 85.9% 2.4% 6.5% 3.2% 100.0% (871) Gender Female 2.0% 81.6% 2.4% 9.5% 4.5% 100.0% (1888) Male 4.7% 89.3% 3.3% 1.7% 1.0% 100.0% (300) Total 2.3% 82.6% 2.6% 8.4% 4.1% 100.0% (2188) The prevalence of disability among the interviewed program beneficiaries was 10.4% with Cadaado and Hudur program sites recording the highest figures (25.3% and 35.2%)-Figure 4.1.32F 33 Given that disability was one of the cross cutting themes considered in selection of beneficiaries, the regions of Cadaado and Hudur which saw additional beneficiaries recruited under SHARPEN 2 program, had a 33Disability was self-reported based on the Washington group of questions hence inclusion and exclusion errors. Page 21 of 97 higher proportion of persons with disabilities. The types of disabilities reported in the visited households were: physical (47.5%), vision (22.4%), hearing (16.0%), mental (11.4%), acquired brain injuries (1.8%) and intellectual disabilities (0.9%)-Table 4.6. Generally, physical disabilities were reported to be high due to low immunization coverage and injuries from fights as well as long term untreated accidents cases resulting from low health services access. Figure 4.1: Prevalence of disability Table 4.6: Forms of disability among program beneficiaries Nature of disability Mogad ishu Afgoo ye Cadaa do Baido a Luuq Dollo w Hudu r Burdh ubo Total Vision impairment 12.5% 33.3% 20.5% 19.6% 9.1% 61.1% 12.9% 50.0% 22.4% Hearing 6.3% 22.2% 25.0% 4.3% 27.3% 5.6% 22.6% 50.0% 16.0% Mental health conditions 10.4% 0.0% 27.3% 10.9% 18.2% 0.0% 3.2% 0.0% 11.4% Intellectual disability 0.0% 0.0% 2.3% 2.2% 0.0% 0.0% 0.0% 0.0% 0.9% Acquired brain injury 0.0% 0.0% 0.0% 0.0% 0.0% 22.2% 0.0% 0.0% 1.8% Physical disability 70.9% 44.4% 25.0% 63.0% 45.5% 11.1% 61.3% 0.0% 47.5% Total 100.0 %(48) 100.0 %(9) 100.0 %(44) 100.0 %(46) 100.0 %(11) 100.0 %(18) 100.0 %(31) 100.0 %(12) 100.0%(219) 4.3 Relevance of the SHARPEN 2 Program [Evaluation Rating: Strong] On program relevance of the SHARPEN II program, the evaluation team sough to establish the extent to which the intervention objectives and design responded to beneficiaries’ needs, global priorities, country needs, and partner organizations mandates. 4.3.1 Relevance to the Beneficiaries’ Needs [Evaluation rating: strong] In assessing the relevance of the program interventions at the community level, respondents were asked of their major household needs over the last two years. The responses were as follows: food (67.6%), health (57.5%), nutrition (47.5%), water (53.1%), shelter (42.1%), hygiene and sanitation (39.9%), non￾food items (24.7%), livelihoods (27.2%), protection and security (16.2%), children playgrounds and safe spaces (7.3%), education (7.7%), animal health services (5.7%) and identification documents (2.0%)-Table 4.7. This is an indication that the program was addressing the top most community needs across all the program sites. Table 4.7: Community needs Need Program site Location Gender Overal Moga l dishu Afgooy e corrido r Caadad o Baidoa Luuq Dollow Hudu r Burdhu bo IDP camps Host Commun ity Female Male Health 60.1% 65.6% 26.4% 61.5% 96.9% 45.6% 3.4% 51.0% 62.3% 50.2% 57.3% 59.0% 57.5% Nutriti on 44.0% 55.9% 22.5% 45.7% 84.0% 35.6% 8.0% 80.0% 50.3% 43.3% 46.5% 54.0% 47.5% Food 61.7% 82.9% 39.9% 74.9% 89.7% 34.8% 88.6% 92.0% 78.7% 50.7% 67.8% 65.7% 67.6% Water 40.3% 69.0% 21.9% 74.9% 93.3% 50.8% 34.1% 11.0% 68.7% 29.5% 53.9% 48.0% 53.1% Mogadishu Afgooye corridor Cadaado Baidoa Luuq Dollow Hudur Burdhubo Male Female Total 7.1% 2.2% 25.3% 15.8% 5.7% 10.4% 35.2% 12.0% 11.0% 10.3% 10.4% Percentage of the respodents Page 22 of 97 Hygie ne and sanitat ion 33.4% 48.0% 5.6% 60.1% 75.3% 23.2% 31.8% 33.0% 50.0% 24.6% 40.1% 38.3% 39.9% Nonfo od items 17.4% 25.1% 38.2% 26.8% 2.1% 40.0% 6.8% 64.0% 24.2% 25.5% 22.5% 38.7% 24.7% Prote ction 10.3% 12.9% 0.6% 43.0% 8.8% 35.2% 0.0% 0.0% 17.5% 14.2% 15.3% 22.0% 16.2% Shelte r 25.4% 52.2% 33.1% 27.1% 93.3% 50.0% 8.0% 85.0% 52.0% 27.2% 39.8% 57.0% 42.1% Identif ication docum ents 0.3% 3.4% 3.9% 4.1% 0.0% 3.6% 0.0% 0.0% 2.3% 1.6% 2.1% 1.3% 2.0% Anima l health servic es 6.8% 0.5% 20.8% 1.4% 0.0% 12.0% 0.0% 5.0% 5.5% 6.0% 5.2% 8.3% 5.7% Liveli hood/ Sourc e of incom e 35.0% 34.1% 33.7% 39.9% 2.6% 15.2% 0.0% 0.0% 31.4% 20.9% 28.7% 18.3% 27.2% Childr en play groun ds and safe spaces 5.3% 1.7% 6.2% 33.7% 2.6% 0.4% 0.0% 1.0% 10.7% 2.1% 7.8% 3.7% 7.3% Educa tion 12.9% 12.2% 0.0% 2.1% 0.0% 0.4% 28.4% 0.0% 6.2% 10.0% 8.5% 3.0% 7.7% FGDs with beneficiaries across the program sites documented the sufferings they endured in search of water, sanitation facilities, health and nutrition services which further indicates that the program interventions responded to the immediate needs of the targeted population. Below is a summary of quotes from FGD participants in the field during the evaluation exercise, further supporting the program’s relevance to their needs: ……………………. “We lacked clean toilets and there was a shortage of water, water is very expensive we couldn't afford to buy it regularly, as such, infections were also high due to poor hygiene” ……………… [Male FGD participant, Mogadishu] ……………………. “The common protection cases in this camp include children separated from their biological parents, defilement of young girls, negligence by caregivers and early and forced marriages” ……………….. [Female FGD participant, Adaado] ……………. “Over the year we have endured shortages in water supply, latrines, food insecurity and malnutrition” ………….. [Female FGD participant, Burdhubo] ……………………. “The most common needs in this community include toilets and latrines, shelter, food, water supply and sanitation facilities, garbage disposal pits and unavailability of livelihoods opportunities” ………………. [Male FGD participant, Afgoye corridor] From KIIs with stakeholders including program staff, partner organizations, health care workers, nutritionists and district officials, SHARPEN II was a continuation of SHARPEN 1 due to the need observed in the communities during the previous phase of the program. In addition, needs assessments were conducted to establish priority areas for interventions and geographic scopes were discussed and agreed upon with the various cluster and technical working groups in Somalia. Below is a summary of quotes from KIIs with program staff and partners as captured during the evaluation exercise. Page 23 of 97 …………………..“Due to the existing need of health WASH nutrition and protection we initiated activities for intervention so that we could bring a sustainable outcome for the community” …………. [KII respondent, Dollow] …………….“Due to drought, famine , displacement , insecurity , harassment and inter clan wars there are high levels of malnutrition, children deaths, mothers deaths, disease outbreaks and poor sanitation and hygiene practices. Therefore this program was addressing the priority needs of our community members” …………………. [KII respondents, Bakool] …………. “There was an assessment that was made by these partners which necessitated response to widespread violence among women and children” ……………….. [KII respondent, Adaado] ……………“Due insecurity, clan clashes and high cases gender based violence the number of people who are in need of protection and psychosocial support services in Bay region has been increasing” …………………. [KII respondent, Baidoa] 4.3.2 Relevance to the International and National Priorities [Evaluation Rating: Strong] The intervention areas under this program were also found to be in line with the Sustainable Development Goals (SDGs 2,3,5 and 6), 33F 34, 34F 35 the Somalia Humanitarian Response Plan (2021),35F 36 the 9th National Development Plan for Somalia (NDP-9, 2020-2024 pillar 4),36F 37 Somalia WASH Cluster Strategic Operational Framework (SOF), which prioritizes WASH improvements for all Somali nationals,37F 38 the Essential Package of Health Services (EPHS) and primary health care approach,38F 39 the Astana declaration on primary health care,39F 40 the Somalia interim country strategic nutrition plan (2019- 2021),40F 41 and the Somalia national GBV Strategy (2018 – 2020) currently being updated.41F 42 Therefore, the interventions under SHARPEN 2 program were relevant to the national and international priorities of Somalia. 4.3.3 Relevance to CRS and the Local Implementing Partners [Evaluation Rating: Strong] The SHARPEN II program was designed and implemented in line with the CRS programming priorities in Somalia which are to work with local government and communities to meet basic nutritional and sanitation needs, ensure protection of women, children and vulnerable groups, build resilience, and lay the foundation for peace among communities in conflict.42F 43 SOS Children’s villages implements programs in the thematic areas of health, education, protection and family strengthening and under this program SOS was strong in health services provision.43F 44 Trócaire largely works in the thematic areas of nutrition, health and WASH; therefore, the role they played under this program was in line with their core mandate.44F 45 The Save Somali Women and Children (SSWC) is strong in protection programming an indication of broad merit in partnering with CRS under SHARPEN II program.45F 46 4.4 Program Effectiveness [Evaluation Rating: Unsatisfactory] On the effectiveness of SHARPEN II program, the evaluation sought to assess the extent to which the targeted objectives and results were met, as well as factors facilitating or restricting the achievement of the program’s goals. Comparison of final evaluation scores with baseline figures and program targets was done with qualitative data providing explanations for the attained evaluation sores. 34Public Private Pact.2020. Somalia and SDG 2030.< https://www.ppp￾sdg.com/services/Somalia%20and%20SDG/index.html#:~:text=MEETING%20THE%20SDGS%20IN%20SOMALIA,aspires%20to%20achieve%20by%202030.&text=This%20partnership%20can%20be%20used,ambitions %20of%20the%202030%20Agenda. > 35 UNESCO.2020. Claiming Human Rights - in Somalia. 36OCHA.2021. Somalia: Humanitarian Response Plan 2021 (February 2021). < https://reliefweb.int/report/somalia/somalia-humanitarian-response-plan-2021-february-2021> 37The Ministry of Planning, Investment and Economic Development, Federal Government of Somalia.2020.Somalia national development plan.<2020 to 2024 http://mop.gov.so/wp-content/uploads/2019/12/NDP-9-2020- 2024.pdf> 38WASH Cluster Somalia.2018. Guide to WASH Cluster Strategy and Standards also, known as Strategic Operational Framework (SOF).< https://www.humanitarianresponse.info/sites/www.humanitarianresponse.info/files/documents/files/180502_guide_to_wash_cluster_strategy_and_standards_sof.pdf > 39Ministry of Health and Human Services, Federal Government of Somalia, Ministry of Health, Puntland; and Ministry of Health, Somaliland.2014. Somali health policy, prioritization of health policy actions in Somali health sector.< http://www.mohpuntland.com/wp-content/uploads/2016/03/FINAL-Somali_Health_Policy_Directions_and_Priorities-Dec-2014-2.pdf > 40WHO.2018.New global commitment to primary health care for all at Astana conference.< https://www.unicef.org/press-releases/new-global-commitment-primary-health-care-all-astana￾conference#:~:text=The%20Declaration%20of%20Astana%2C%20unanimously,4)%20align%20stakeholder%20support%20to > 41Food Agricultural Organization of the United Nations-FAO.2019. Somalia interim country strategic plan (2019–2021) .< https://docs.wfp.org/api/documents/536e0ee1ec2e424cb5fab8b177f6d33c/download/> 42GBV Sub-Cluster Somalia.2018.Somalia National GBV Strategy 2018 – 2020.< https://reliefweb.int/sites/reliefweb.int/files/resources/Somalia%20-%20National%20GBV%20strategy%202018-2020.pdf> 43Catholic Relief Services.2020. CRS in Somalia. 44SOS Children’s villages international.2020. SOS Children's Village Mogadishu.< https://www.sos-childrensvillages.org/where-we-help/africa/somalia/mogadishu> 45 Trocaire.2020. Where we work-Somalia. 46SSWC.2020. Who is SSWC? Page 24 of 97 4.4.1 Effectiveness in the Nutrition Interventions [Evaluation Rating: Unsatisfactory] Nutrition education and behaviour change messages were relayed to program beneficiaries through several channels including face to face sessions, outreaches, radio messages and posters in health facilities. Overall, 93.2% of the beneficiaries had received nutrition messages from the partner organizations. The interviewed nutrition beneficiaries in the host community had not received any messages while only 54.8% of the males had received such messages (Figure 4.2). FGDs indicated a low knowledge and interest in nutrition activities by men while community groups’ membership increased exposure to health messages, providing a critical pathway to influence health promotion and, thus, better health outcomes. Figure 4.2: Beneficiaries who have received nutrition messages The source of nutrition messages was health care workers in health facilities (85.7%), community health care workers (40.8%), community health groups such as mothers support groups (24.3%), mobile phones (13.2%), relatives and family members (10.2%), radio (5.6%), television (2.7%), brochures and other reading materials (1.3%), door to door visits (0.7%) and the internet (0.1%)-Table 4.8. Table 4.8: Source of the nutrition message you received Source of the message Program site Location Gender Total Mogadishu Afgooye corridor Baidoa Luuq Dollow Hudur IDP camps Host Community Female Male Health care workers in health facilities 88.4% 67.5% 97.4% 100.0% 100.0% 100.0% 83.6% 88.2% 85.5% 91.3% 85.7% Community health care workers 27.8% 39.9% 94.9% 100.0% 100.0% 1.5% 49.5% 30.2% 39.3% 87.0% 40.8% Community health groups such as mothers support groups and community health units 17.6% 31.5% 2.6% 80.6% 20.7% 1.5% 34.5% 12.1% 24.2% 30.4% 24.3% Radio 0.8% 13.8% 38.5% 19.4% 0.0% 0.0% 9.0% 1.4% 5.8% 0.0% 5.6% Television 0.3% 0.5% 2.6% 6.5% 0.0% 0.0% 4.5% 0.6% 2.7% 4.3% 2.7% Mobile phones (calls or messages) 4.9% 20.2% 2.6% 66.1% 0.0% 0.0% 22.4% 2.0% 13.0% 17.4% 13.2% Internet including social media 0.5% 3.4% 2.6% 0.0% 0.0% 0.0% 0.2% 0.0% 0.1% 0.0% 0.1% Brochure/Reading materials 7.0% 18.7% 35.9% 0.0% 0.0% 0.0% 1.9% 0.6% 1.3% 0.0% 1.3% Relatives and family members 1.1% 0.5% 97.4% 0.0% 0.0% 0.0% 12.6% 7.2% 10.3% 4.3% 10.2% Door to door visits by social workers 88.4% 67.5% 0.0% 0.0% 0.0% 0.0% 0.5% 0.9% 0.7% 0.0% 0.7% 96.6% 94.9% 97.5% 100.0% 72.5% 100.0% 95.5% 90.6% 0.0% 95.3% 54.8% 93.2% Mogadishu Afgooye corridor Baidoa Luuq Dollow Hudur Burdhubo IDP Camp Host Community Female Male Total Percentage of the respondents Page 25 of 97 IYCF knowledge was high among the program beneficiaries with 89.0% correcting reporting that breastfeeding ought to be initiated within one hour of baby delivery, 71.5% correctly describing the length of baby breast-feeding (24 months) and a further 96.1% correctly describing that exclusive breast-feeding needs to be done for 6 months while 97.8% correctly had the knowledge on the timing of introduction of complementary feeds (Table 4.9). Table 4.9: IYCF knowledge Program site Gender Location Tota Mog l adis hu Afgoo ye corrid or Baido a Luu q Doll ow Hud ur Burdh ubo Femal e Mal e IDP camp s Host commu nity Correct knowledge on introduction of breastfeeding following baby delivery 83.3 % 97.2% 97.5% 100. 0% 80.0 % 100. 0% 60.0% 90.0% 78.6 % 93.0 % 85.4% 89.4 % Correct knowledge on length of baby breastfeeding 71.5 % 60.7% 87.5% 98.4 % 52.5 % 73.8 % 100.0% 71.2% 76.2 % 67.3 % 76.3% 71.5 % Correct knowledge on length of exclusive breast-feeding 93.7 % 98.6% 92.5% 100. 0% 97.5 % 100. 0% 95.0% 96.0% 97.6 % 95.7 % 96.6% 96.1 % Correct knowledge on introduction of complementary feeds 96.6 % 99.1% 100.0 % 100. 0% 95.0 % 100. 0% 95.0% 98.2% 90.5 % 97.7 % 97.9% 97.8 % In 18.9% of the households visited they had a child/children under 18 years. From the 18.9% of households with children of this age, 91.0% had one child in this age category, 8.9% had two children in this age category while 0.2% had three children of this age group (Table 4.10). Table 4.10: Presence of a child/children aged below 2 years in the households <2 years olds presence in the HHs Program site Gende r Location Total Moga dishu Afgooy e corrido r Baido a Luuq Dollo w Hudur Burdh ubo Femal e Male IDPM camps Host comm unity HH with a child aged below 2 years 5.7% 27.6% 40.0% 30.6% 72.5% 7.7% 30.0% 17.4% 47.6% 22.3% 15.1% 18.9% Number of undue 2 years olds in HHs Site Gender Location Total Moga dishu Afgooy e corrido r Baido a Luuq Dollo w Hudur Burdh ubo Femal e Male IDPM camps Host comm unity 1child 92.8% 88.4% 79.2% 86.0% 100.0% 91.7% 100.0% 90.7% 100.0% 89.2% 92.9% 91.0% 2 children 6.9% 11.6% 20.8% 14.0% 0.0% 8.3% 0.0% 9.2% 0.0% 10.5% 7.1% 8.9% 3 children 0.3% 0.0% 0.0% 0.0% 0.0% 0.0% 0.0% 0.2% 0.0% 0.3% 0.0% 0.2% Only 67.7% (confidence interval of 53.1% to 69.4%) of the children aged below 24 months were breastfed in the 24 hours preceding the survey with the lowest proportion being in the host community (50.0%) and the highest being in Luuq (77.6%). Among the children aged 0-5 months, 95.6% were breastfed in the 24 months preceding the survey with the lowest proportions being in Burdhubo (33.0%) and Dollow (50.0%)-Table 4.11. Continuation of breastfeeding in Somalia (to 24 months) is poor due to social cultural practices that hinder optimal IYCF practices, misconceptions that it is biologically impossible for a mother to produce enough milk up to 24 months after delivery, a woman cannot breastfeed while pregnant, low mothers’ education and delivery in early ages including adolescence.46F 47 FGDs indicated that IYCF messages had been received but they were not being put into use due to the aforementioned negative cultural beliefs. 47Ministry of Health and Human Services, Federal Republic of Somalia.2020. Somalia nutrition strategy (2020 -2025). < https://www.unicef.org/somalia/media/1756/file/Somalia-nutrition￾strategy-2020-2025.pdf> Page 26 of 97 Table 4.11: Breastfeeding of children aged below 2 years Age Site Location Gender Total Mogadish u Afgooy e corrido r Baido a Luuq Dollo w Hudu r Burdhub o IDP camp s Host communi ty Femal e Male 2 years < 66.8% 65.3% 72.4% 77.6% 66.7% 73.8% 50.0% 67.6 % 67.8% 69.9% 65.6 % 67.7 % 0-5 month s 98.4% 100.0% 85.7% 100.0 % 50.0% 100.0 % 33.3% 98.6 % 91.9% 94.8% 96.1 % 95.6 % The question on EBF focused on the 24 hours preceding the evaluation interviews and not the entire age of the child. Exclusive breastfeeding for children aged 0-5 months in the 24 hours preceding the survey was reported in 61.5% of the households with the lowest proportions being in Dollow (0.0%) and Baidoa (7.1%)-Figure 4.3. The baseline score for this indicator was 68.1% (confidence interval of 60.6% to 74.9%), as such, there was no change in this indicator following nutrition interventions. Due to insecurity only 38 nutrition beneficiaries were interviewed in Dollow hence a very low score on EBF in this program site. From the FGDs, food unavailability for mothers, engagement in livelihoods activities by mothers, strong beliefs and cultural practices such as feeding babies with water and animal fats, and the perception that mothers do not have the ability to exclusively breastfed for the first six months were picked as some of the barriers to exclusive breastfeeding despite a high awareness of the importance of the same. Figure 4.3: Exclusive breastfeeding of children aged 0-5 months Mogadishu Afgooye corridor Baidoa Luuq Dollow Hudur Burdhubo Female Male IDP camps Host community All sites All sites All sites Evaluation Basel ine score Targe t 59.7% 74.5% 7.1% 100.0% 0.0% 100.0% 100.0% 65.5% 58.4% 60.3% 62.9% 61.5% 68.1% 75.0% Percentage Page 27 of 97 Through nutrition messaging the importance of diet diversity in the reduction of stunting, wasting and being underweight in children was brought to the attention of nutrition beneficiaries. Thus, the program had a target of having atleast 75% of the children aged 6-23 months receiving atleast 4 different food stuffs per day with a baseline score of 47.1% (confidence interval of 39.5% to 54.8%). From the end term evaluation, only 43.3% of the households (confidence interval of 39.4% to 47.4%) indicated that children in this age category had consumed atleast 4 different food stuffs in the 24 hours preceding the survey (Figure 4.4). Due to insecurity only 20 nutrition beneficiaries were interviewed in Burdhubo program sites. As such, the program target was not met. From the FGDs, food unavailability, inflation and lack of livelihoods opportunities were blamed for limited dietary diversity in the households. The FSNAU-FEWS NET 2021 Post Gu Technical Release dated September 9, 2021 approximated that 3.5 million people in Somalia faced acute food insecurity Crisis (IPC Phase 3) or worse outcomes in late 2021. Specifically, the report indicates that 7,178,500 persons in Somalia faced food insecurity between October and December 2021 with 3,712,900 persons (51.7%) in the stressed phase, 2,824,960 (39.4%) in the crisis phase and 640,730 (8.9%) in the emergency phase.47F 48 Figure 4.4: Consumption of atleast 4 food stuffs by children aged 6-23 months in the 24 hours preceding the evaluation survey 48FSNAU-FEWS NET.2021.FSNAU-FEWS NET 2021 Post Gu Technical Release - September 9, 2021.< https://fsnau.org/node/1891> Mogadishu Afgooye corridor Baidoa Luuq Dollow Hudur Burdhubo Female Male IDP camps Host community All sites All sites Al sites Evaluation Baseli ne survey Target 46.5% 31.0% 86.7% 85.1% 28.6% 18.3% 0.0% 43.6% 43.0% 47.6% 38.8% 43.3% 47.1% 75% Percentage Page 28 of 97 The food stuffs largely consumed in the households in the 24 hours preceding the evaluation survey were grains (78.5%), tubers (73.1%), Vitamin A rich fruits and vegetables (60.9%), fats (60.0%), condiments (57.6%0, flesh foods (42.9%), legumes and seeds (39.3%), eggs (28.9%), dairy products (14.9%) and sugars (15.8%)-Table 4.12. Economic access is one of the most significant barriers to achieving a nutritious diet, particularly in rural areas across Somalia. Somalian meals consist of mainly staple commodities (maize, sorghum, rice, wheat, and pasta), oil and, with limited consumption of nutritious foods, such as fruits and vegetables. The majority of the Somali population consumes more frequently an energy-based diet because of their affordability and accessibility. It consists primarily of starchy carbohydrates and minimum nutritional values. A healthy and balanced meal globally costs approximately six (6) times more to purchase compared to an energy-based diet. According to the World Food Programme (WFP): Fill the Nutrition Gap, the cost of diet is roughly seven (7) dollars per day per household in Somalia. Because a nutritionally dense food is not affordable to the majority of the population; as a result, their health is impacted with hidden hunger, iron deficiency anaemia and vitamin A. The prevalence of Global Acute Malnutrition (GAM) continues to be reported as a serious factor affecting the overall health of the Somali population. The median (GAM) prevalence has remained severe or between (10– 14.9%) for the past three consecutive seasons.48F 49 The Minimum Expenditure Basket (MEB) in Somalia represents a set of essential food items representing 2,100- kilocalories per person per day.49F 50 Table 4.12: Food stuff consumed by children aged 6-23 months in the 24 hours preceding the evaluation survey Program site Location Gender Total Mogadi shu Afgooye corridor Baidoa Luuq Dollow Hudur Burdhu bo IDP camps Host Communit y Fem ale Male Grains 83.5% 64.1% 66.7% 87.2% 83.3% 75.0% 100.0% 75.3% 82.2% 80.8%76.1% 78.5% Tubers 79.5% 73.5% 66.7% 97.9% 66.7% 26.8% 0.0% 74.2% 71.8% 73.4%72.7% 73.1% Vitamin A rich fruits & vegetables 68.3% 48.7% 86.7% 97.9% 50.0% 16.1% 0.0% 61.2% 60.6% 63.3%58.3% 60.9% Flesh foods 43.9% 40.2% 86.7% 53.2% 33.3% 28.6% 0.0% 44.0% 41.7% 40.6%45.5% 42.9% Eggs 35.6% 13.7% 6.7% 34.0% 16.7% 30.4% 0.0% 24.7% 33.6% 28.7%29.2% 28.9% Legumes, nuts and seeds 41.3% 29.9% 60.0% 85.1% 16.7% 7.1% 33.3% 42.6% 35.5% 39.5%39.0% 39.3% Dairy products 12.2% 12.8% 80.0% 17.0% 33.3% 14.3% 0.0% 16.2% 13.5% 13.3%16.7% 14.9% Fats 57.8% 60.7% 80.0% 38.3% 83.3% 87.5% 0.0% 61.9% 57.9% 58.7%61.4% 60.0% Condiments 60.4% 45.3% 80.0% 78.7% 16.7% 55.4% 0.0% 60.1% 54.8% 58.7%56.4% 57.6% Sugars 17.5% 6.8% 80.0% 19.1% 33.3% 5.4% 0.0% 17.9% 13.5% 13.6%18.2% 15.8% The fluids taken by consumed by children aged 6-23 months in the 24 hours preceding the evaluation survey were plain water (92.6%), milk (79.1%), porridge (55.3%), formula milk (47.4%), juice (39.6%), broth (38.4%) and yoghurt (7.9%)-Table 4.13. Table 4.13: Fluids consumed by children aged 6-23 month (24 hours preceding the interviews) Program site Location Gender Total Mogadishu Afgoo ye corrid or Baidoa Luuq Dollow Hudur Burdhubo IDP camps Host Community Female Male Plain water 89.7% 98.3% 100.0% 97.9% 85.7% 94.9% 0.0% 96.3% 88.7% 94.2% 90.9% 92.6% Formula 54.9% 30.6% 100.0% 80.9% 28.6% 5.1% 0.0% 45.9% 48.9% 47.8% 46.9% 47.4% Milk 78.1% 76.9% 100.0% 93.6% 100.0% 69.5% 100.0% 79.4% 78.8% 78.6% 79.6% 79.1% Juice 43.3% 28.9% 100.0% 66.0% 42.9% 6.8% 0.0% 37.8% 41.6% 42.0% 37.1% 39.6% Broth 43.9% 23.1% 6.7% 36.2% 28.6% 52.5% 0.0% 37.5% 39.4% 36.3% 40.7% 38.4% Yoghurt 6.3% 7.4% 73.3% 0.0% 42.9% 3.4% 0.0% 6.1% 9.9% 8.1% 7.6% 7.9% Porridge 61.1% 39.7% 46.7% 63.8% 28.6% 54.2% 50.0% 56.4% 54.0% 57.6% 52.7% 55.3% 49Somalia Nutrition Cluster.2021.Nutrition-Sensitive Diet in Somalia. < https://reliefweb.int/sites/reliefweb.int/files/resources/nutrition_sensitive_diet_somalia_snc_vf.pdf> 50Food security and Nutrition Analysis Unit.2020. Somalia Market Update: May 2020 Update (Issued June 16, 2020). . Page 29 of 97 4.4.2 Effectiveness in the Health Interventions [Evaluation Rating: Satisfactory] Health awareness campaigns through various forums and channels were conducted under the program with the objective of improving preventive and promotive behaviours and practices in the program sites. From the end term evaluation, 87.4% of the respondents had received health messages from the partner organizations with the least proportion being in the Afgooye corridor (62.5%)-Figure 4.5. In addition, of those who had received health messages, 90.8% (confidence interval of 86.7% to 93.9%) could recall three or more health messages against a baseline figure of 69.4% (confidence interval of 63.17% to 75.14%) and an end term target of 85.0% meaning that the program had achieved the intended target. From the FGDs, the received health messages were: safe water storage in clean containers, drinking clean water, breastfeeding for children and mothers, proper hand washing at critical times, attending regular ANC clinics, pregnancy complications and their management, immunization of children against diseases, maintenance of personal hygiene, proper food preparation and handling, seeking treatment in health facilities, deworming practices, the importance of skilled delivery services and usefulness of post natal visits care. Figure 4.5: Health messaging Mogadishu Afgooye Corridor Baidoa Luuq Dollow Hudur Burdhubo Female Male IDP camps Host Community Total Evaluation Baselin e Survey Target 71.4% 86.7% 98.6% 94.0% 90.3% 77.3% 87.5% 92.3% 86.6% 90.4% 91.2% 90.8% 69.4% 85.0% Can recall three or more health messages Received any health messages Page 30 of 97 From the evaluation 96.0% of the respondents indicated that they had put the received messages into use including 95.5% males and 96.2% females (Figure 4.6). The sources of the received health messages were large health café workers in static health facilities (80.4%), community health care workers (68.4%), community groups such as mother support groups and community health units (24.0%), television (19.2%), mobile phones (19.2%), radio (12.4%), relatives and family members (5.2%) and brochures and reading materials (0.8%)-Table 4.14. From the FGDs and KIIs, men received health messages mostly through health facilities but nutrition messaging was largely done through community outreaches and mothers’ support groups which men were not part of. Figure 4.6: Putting health messages into use Table 4.14: Sources of health messages Program site Gender Location Tota Mogadis l hu Afgooy e Corrid or Baido a Luuq Dollo w Hudu r Burdhu bo Fema le Male IDP cam p Host Communi ty HCWs 28.6% 93.3% 88.6 % 100.0 % 90.3 % 100.0 % 22.5% 89.1% 56.7 % 85.1 % 76.5% 80.4 % CHWs 57.1% 40.0% 54.3 % 96.0% 96.8 % 0.0% 97.5% 62.8% 83.6 % 68.4 % 68.4% 68.4 % Communi ty health groups 42.9% 36.7% 2.9% 68.0% 32.3 % 0.0% 0.0% 26.2% 17.9 % 39.5 % 11.0% 24.0 % Radio 57.1% 3.3% 2.9% 48.0% 0.0% 0.0% 0.0% 15.3% 4.5% 26.3 % .7% 12.4 % Televisio n 14.3% 0.0% 62.9 % 0.0% 0.0% 13.6% 0.0% 22.4% 10.4 % 13.2 % 24.3% 19.2 % Phones calls or messages 42.9% 0.0% 0.0% 90.0% 0.0% 0.0% 0.0% 23.0% 9.0% 42.1 % 0.0% 19.2 % Brochure s 0.0% 3.3% 0.0% 0.0% 3.2% 0.0% 0.0% 1.1% 0.0% .9% .7% 0.8% Relatives 14.3% 0.0% 17.1 % 0.0% 0.0% 0.0% 0.0% 6.0% 3.0% 5.3% 5.1% 5.2% 100.0% 100.0% 100.0% 86.0% 96.8% 100.0% 95.0% 96.2% 95.5% 93.9% 97.8% 96.0% Percentage Page 31 of 97 Challenges in accessing health services were described as long distance to health facilities (62.2%), unavailability of transport services to health centers (46.9%), insecurity (13.3%), cost barriers (6.3%), unavailability of health care workers (2.1%), stock outs in health facilities (1.7%) and health care workers’ negative attitude (1.7%)-Table 4.15. FGDs documented distance to health facilities, unavailability and/or unaffordability of transport services and insecurity as the main barriers to health care services utilization. Table 4.15: Barriers to health care services access Program site Gender Location Total Mogadis hu Afgooye Baid oa Lu uq Dol low Hu du r Burd hubo Fe mal e Ma le IDP camp Host Comm unity Insecurity 42.9% 6.3% 9.5% 2.0 % 54. 5% 0.0 % 0.0% 9.4 % 24. 3% 6.3% 18.9% 13.3% Unavailability of transport services to health facilities 57.1% 50.0% 37.8 % 98. 0% 56. 8% 0.0 % 10.0 % 50.5 % 36. 5% 62.2 % 34.6% 46.9% Long distance to facilities 71.4% 68.8% 87.8 % 54. 0% 18. 2% 0.0 % 100.0 % 62.3 % 62. 2% 64.6 % 60.4% 62.2% Unavailability of medicines (stock outs) 14.3% 4.2% 0.0% 4.0 % 0.0 % 0.0 % 0.0% 1.9 % 1.4 % 3.9% 0.0% 1.7% Unavailability of some health services 14.3% 8.3% 9.5% 2.0 % 0.0 % 0.0 % 0.0% 5.7 % 1.4 % 6.3% 3.1% 4.5% High/Unaffordable cost of health services 14.3% 14.6% 10.8 % 2.0 % 2.3 % 0.0 % 0.0% 8.0 % 1.4 % 7.9% 5.0% 6.3% Unavailability of health care workers 0.0% 8.3% 0.0% 4.0 % 0.0 % 0.0 % 0.0% 2.4 % 1.4 % 3.9% 0.6% 2.1% Health care workers negative attitude 14.3% 8.3% 0.0% 0.0 % 0.0 % 0.0 % 0.0% 1.9 % 1.4 % 3.1% 0.6% 1.7% 4.4.3 Effectiveness in the WASH Interventions [Evaluation Rating: Not Satisfactory] Under the WASH components of the program, latrines and water points were constructed and rehabilitated, and behavior change was achieved through effective hygiene promotion campaigns. Specifically, awareness creation on the benefits of using soap for hand washing and critical times for hand washing was done. Thus, the program targeted to increase awareness of the five critical moments for hand washing from a baseline score of 78.7% (confidence interval of 72.1% to 84.4%) to a minimum 80.0%. From the end term evaluation, 88.5% (confidence interval of 85.5% to 91.0%) of the interviewed WASH beneficiaries were aware of at least three critical moments for hand washing with the lowest proportions being in Burdhubo (50.0%) and among males (67.4%)-Figure 4.7. Therefore, the target for this indicator was achieved. The critical moments known by the respondents were as follows: after defecation/visiting the toilet (96.9%), after cleaning a child's bottom or changing nappies (77.5%), before feeding a child (81.8%), before eating (93.3%) and before touching and preparing food (65.6%)- Table 4.16. From the FGDs, a strong linkage between poor perception of safety and food handling and babies was noted with fire expected to kill microorganisms during food preparation and changing of babies being considered to have no safety risks. Page 32 of 97 Figure 4.7: Awareness of at least 3 critical moments for hand washing Mogadishu Afgooye corridor Baidoa Luuq Dollow Burdhubo IDP camps Host communities Female Male Total All sites All sites Evaluation Baseline Survey Target 96.4% 98.7% 95.0% 100.0% 81.3% 50.0% 96.8% 83.5% 95.5% 67.4% 88.5% 78.7% 80.0% Percentage Page 33 of 97 Table 4.16: Critical moments for hand washing known by the program beneficiaries Program site Location Gender Total Mogadishu Afgooye corridor Baidoa Luuq Dollow Burdhubo IDP camp Host Community Female Male After defecation/visiting the toilet 100.0% 88.0% 85.0% 100.0% 97.6% 100.0% 93.7% 98.8% 97.1% 96.1% 96.9% After cleaning a child's bottom or changing nappies 92.8% 88.0% 41.7% 100.0% 73.5% 42.5% 77.8% 77.3% 83.8% 58.9% 77.5% Before feeding a child 87.0% 100.0% 81.7% 97.6% 75.9% 45.0% 90.5% 76.7% 88.7% 61.2% 81.8% Before eating 89.9% 100.0% 98.3% 100.0% 91.6% 87.5% 97.9% 90.7% 94.0% 91.5% 93.3% Before touching and preparing food 84.1% 96.0% 76.7% 52.4% 53.6% 42.5% 69.8% 63.0% 72.8% 44.2% 65.6% From the WASH beneficiary households visited, 89.8% were using improved sanitation facilities (least being in the Afgooye corridor)-Figure 4.8. The specific type of sanitation facilities used were pit latrines (65.4%), toilets of composting and hanging types (24.5%) and buckets (5.1%) while open defecation was reported in 5.1% of the households (Table 4.17). Secondary literature indicates that, open defecation is common practice with eleven to 28 percent of the population defecating in the open. 50F 51, 51F 52 FGDs on the other hand indicated that lack of locks on the latrine doors and unavailability of light on the latrines contributed to open defecation at night. Figure 4.8: Proportion of households using improved sanitation facilities 51UNICEF Somalia.2021. Water, sanitation and hygiene.< https://www.unicef.org/somalia/water-sanitation-and-hygiene > 52UNICEF.2021.Water, Sanitation, and Hygiene Assessment. < https://reliefweb.int/sites/reliefweb.int/files/resources/REACH_SOM_Report_Somalia-WASH-Report_February-2021.pdf> Mogadishu Afgooye corridor Baidoa Luuq Dollow Burdhubo IDP camps Host communities Female Male Total 93.5% 20.0% 100.0% 100.0% 88.0% 92.5% 88.9% 90.4% 90.1% 89.1% 89.8% Percentage of HHs Page 34 of 97 Table 4.17: Sanitation facility used in the households Program site Location Gender Total Mogadish u Afgooy e corrido r Baido a Luuq Dollo w Burdhub o IDP camps Host commun ity Femal e Male Pit latrine 73.2% 8.0% 83.3% 6.1% 83.7% 38.6% 38.6% 92.5% 61.3% 77.5 % 65.4 % Toilet (composting and hanging types) 20.3% 12.0% 16.7% 93.9 % 4.2% 50.3% 50.3% 0.0% 28.8% 11.6 % 24.5 % Bucket52F 53 4.3% 80.0% 0.0% 0.0% 0.0% 10.6% 10.6% 0.0% 6.8% 0.0% 5.1% Bush/Field/o pen defacation 2.2% 0.0% 0.0% 0.0% 12.0% 0.5% 0.5% 7.5% 3.1% 10.9 % 5.1% Total 100.0% (138) 100.0% (25) 100.0 % (60) 100.0 % (82) 100.0 % (166) 100.0% (189) 100.0% (322) 100.0% (40) 100.0 % (382) 100.0 % (129) 100.0 % (511) Latrines were constructed under the program in both IDP camps and the host community villages. From the field visits during the evaluation, an average of 59.4% households reported sharing of toilets (67.6% in the IDP camps and 54.2% in the host community)-Figure 4.9. At baseline, 22 persons shred a latrine while at end term, on average, 9 persons shared a single latrine (confidence interval of 6.3 to 12.3 users) with the highest and lowest number of persons sharing a toilet being in the Afgooye corridor (56) and Luuq (4)-Table 4.18. KIIs indicated that the Afgooye corridor had the largest number of new IDPs population who utilized the available latrines with program beneficiaries hence a higher proportion of households sharing latrines. Figure 4.9: Sharing of toilets/latrines 53Buckets were used at night when females and children feared walking to and using latrines. Mogadishu Afgooye corridor Baidoa Luuq Dollow Burdhubo IDP camps Host community Female Male Total 36.3% 88.0% 61.7% 65.9% 63.0% 91.9% 67.6% 54.2% 55.1% 73.0% 59.4% Percentage of the HHs Page 35 of 97 Table 4.18: Number of persons sharing a single toilet Variable Variable description Mean number of persons sharing a single toilet Program site Mogadishu 7 Afgooye corridor 56 Baidoa 8 Luuq 4 Dollow 5 Burdhubo 15 Location IDPs 5 Host 11 Overall Total 9 At baseline, 21.5% (confidence interval of 15.8% to 28.2%) were accessing water from improved sources. Under the program water sources were repaired and constructed, thus, overall, 59.3% (confidence interval of 66.1% to 74.2%) of the respondents indicated that their source(s) water for drinking, cooking, and hygiene had improved in the preceding year (Figure 4.10) against a target of 70.0% an indication that this program outcome target was not met. The mains sources of water for drinking, cooking, and hygiene in the visited households were piped water on premises inside dwelling, plot or yard (70.5%), Other improved sources such as public taps, standpipes, tube wells, boreholes, protected springs or rain water (30.3%), surface water from either river, dam, lake, pond, stream, canal or irrigation channels (20.7%) and unimproved sources like unprotected dug well, unprotected spring, carts, trucks, bottled water (9.0%)-Table 4.19. Figure 4.10: Improved water source over the preceding year Mogadishu Afgooye corridor Baidoa Luuq Dollow Burdhubo IDP camps Host community Female Male Total All sites All sites Evaluation Baseli ne survey Target 65.2% 32.0% 33.3% 50.0% 65.7% 46.0% 67.1% 57.1% 65.9% 87.5% 59.3% 21.5% 70.0% Percentage Page 36 of 97 Table 4.19: Household’s main source(s) of water for drinking, cooking, and hygiene Program site Location Gender Total Moga dishu Afgooy e corrido r Baido a Luuq Dollow Burdh ubo IDP camp Host Communi ty Fem ale Mal e Piped water on premises (inside dwelling, plot or yard) 89.9%8.0% 60.0% 41.5% 77.7% 87.5% 47.1% 84.2% 67.0% 80.6% 70.5% Other improved sources (public taps, standpipes, tube wells, boreholes, protected springs or rain water) 13.8%60.0% 38.3% 89.0% 15.1% 0.0% 59.8% 13.0% 34.3% 18.6% 30.3% Unimproved sources (unprotected dug wells, unprotected spring, carts, trucks, bottled water) 2.2% 36.0% 35.0% 14.6% 0.6% 0.0% 23.3% 0.6% 11.0% 3.1% 9.0% Surface water (river, dam, lake, pond, stream, canal or irrigation channels) 0.7% 8.0% 15.0% 41.5% 33.1% 12.5% 24.9% 18.3% 18.3% 27.9% 20.7% Of the interviewed WASH beneficiaries, 75.1% reported that their households had received WASH kits containing soaps, aqua tabs and water jerricans in the year preceding the evaluation survey and these kits were issued for an average 5 times (Figure 4.11). Figure 4.11: Households that received WASH kits from the SHARPEN 2 program WASH kits from the Somalia WASH cluster were distributed in Mogadishu, Afgooye corridor, Baidoa and Burdhubo and they contained aqua tabs to treat water, collapsed type of jerricans and soaps. At baseline the satisfaction with contents of the WASH kits was 93.4% (confidence interval of 88.6% to 96.6%), satisfaction with the quality of WASH kits was 93.9% (confidence interval of 89.2% to 97.0%) and the satisfaction with the quantity of WASH kits was 82.4% (confidence interval of 75.9% to 87.8%). Satisfaction with the contents of the kits was 95.4% (confidence interval of 92.7% to 97.2%) while satisfaction with the quantity issued was 93.5% (confidence interval of 90.5% to 95.7%) and satisfaction with the quality was 92.9% (confidence interval of89.9% to 95.3%) against targets of 96%, 90% and 95% indicating that these three indicators were not met (Table 4.20). The key issues of dissatisfaction were: issuance of collapsed jerricans which beneficiaries did not like, provision of soaps that could not lather well when used with hard water and the number of kits distributed was based on the estimated family size in Somalia rather than the actual size of the benefitting households under the program. 50.7% 28.0% 100.0% 95.1% 78.3% 97.5% 84.7% 69.6% 70.9% 87.6% 75.1% Percentage of HHs Page 37 of 97 Specifically, WASH kits obtained from the Somalia WASH cluster were meant to serve six members in each household. Table 4.20: Satisfaction with WASH items/kits provided Program site Location Gender Total Mogadi shu Afgooye corridor Baidoa Burdh ubo IDP Host Female Male Satisfaction with the contents of the kits 100.0% 85.7% 96.7% 97.4% 95.1% 97.8% 96.6% 94.7% 95.4% Satisfaction with the quantity of the kits 100.0% 85.7% 93.3% 94.9% 97.4% 83.9% 95.6% 92.9% 93.5% Satisfaction with the quality of the kits 100.0% 85.7% 88.3% 97.4% 93.3 91.5% 93.8% 91.7% 92.9% Under SHARPEN II program, hand washing stations were constructed in health facilities. All the 100.0% hand washing stations constructed in health facilities were handed over to the health facilities within the program life. From the interviews with health care workers, 72.5% of the hand washing stations in health facilities were still functional (confidence interval of 68.3% to 76.3%) against a target of 95% meaning that this target was not met (Figure 4.10). KIIs with health care workers indicated mechanical breakdown of the hand washing stations which were largely used by patients. Follow up KIIs with program staff further indicated that breakdown of the hand washing facilities were not reported for repairs and the program staff did not make factor in frequent visits and repairs in the program design. Figure 4.12: Percentage of functional hand washing stations in health facilities From the constructed and rehabilitated water sources, at baseline 15 liters of water per person was collected from the water sites while at end term, households collected an average of 32.5 liters per person per day (confidence interval of 30.9 to 34.1 liters) against a target of 15 liters indicating that this target was met. At baseline, 15 liters of water per person per day was supplied in the WASH program sites, however, the volume of water supplied per person per day at end term was 15 liters (confidence interval of 14.25 to 15.75 liters) against a target of 20 liters meaning that this target for this indicator was not met; this was due low awareness on the volume of water required per person per day among WASH beneficiaries. Although WASH program staff ensured 15 liters of water was available per person per day, uptake of this volume was not done by the beneficiaries. In addition, at baseline, 11,076 persons were accessing water provided under the program while at end term, 11,900 people were directly utilizing improved water services provided by OFDA against an anticipated 14,000, an indication that the target was missed. Lastly, at baseline, 5,600 persons were accessing sanitation services from the program and at end term, 12,900 persons were directly utilizing improved sanitation services provided by OFDA against an anticipated figure of 8,000 indicating that the target was met Cadaado Luuq Total All facilities All facilities Baseline Baseline Target 70% 75% 72.5% 100.0% 95.0% Pecentage Page 38 of 97 (Table 4.21). There were influxes of IDPs in the camps with the worsening of drought and the insecurity dynamics in the south-central Somalia hence a higher need for water and sanitation facilities and utilization of these WASH services by IDPs who were not captured in the routine programming report. Table 4.21: Water and sanitation services supply and utilization Indicator Program site Evaluat ion Baselin e survey Targe t Luuq Dollow No. of people directly utilizing improved water services provided with OFDA funding 4800 2,100 (Male: 903, Female: 1,197) 11,900 11,076 14,000 Average liters/person/day collected from all sources for drinking, cooking, and hygiene 50 liters 15 liters 32.5 liters 15 liters 15 liters Estimated safe water supplied per beneficiary in liters/person/day 15 liters 15 liters 15 liters 15 liters 20 liters No. of people directly utilizing improved sanitation services provided with OFDA funding 10,500 2, 400 (Male: 1,032,Female: 1,368) 12,900 5,600 8,000 4.4.4 Effectiveness of the Protection Interventions [Evaluation Rating: Unsatisfactory] From the visited protection program sites, households reported the major safety burdens as: forced marriages (63.7%), sexual abuse and rape (49.0%), early marriages (48.7%), physical disciplining of children (42.0%), emotional abuse (32.5%), neglect (24.2%), FGM/C (30.3%), separation of children from parents (28.7%), trafficking (5.1%), recruitment into militias (7.8%) and abduction (4.4%)-Table 4.22. From KIIs with program staff in Cadaado and Luuq, an average of 20 GBV cases were reported each month in each of these program sites. Table 4.22: Protection concerns Program site Gender Location Total Mogadis hu Afgooye Corridor Cadaa do Baidoa Femal e Male IDP camp Host Community Physical disciplining 40.3% 82.9% 4.5% 58.1% 43.4% 29.1% 42.2% 16.7% 42.0% Sexual abuse and rape 36.2% 69.1% 24.7% 81.2% 49.2% 47.3% 49.4% 16.7% 49.0% Emotional abuse 32.9% 67.5% 6.7% 34.2% 31.6% 40.0% 32.8% 0.0% 32.5% Neglect 28.9% 35.0% 14.6% 21.4% 21.7% 47.3% 24.1% 33.3% 24.2% Early marriages 38.3% 37.4% 50.0% 71.8% 49.2% 43.6% 48.8% 33.3% 48.7% Forced marriages 29.5% 42.3% 19.7% 64.1% 37.5% 25.5% 63.6% 66.7% 63.7% FGM/C 27.5% 19.5% 23.6% 55.6% 31.3% 21.8% 30.7% 0.0% 30.3% Trafficking 10.7% 10.6% 0.0% 0.0% 5.1% 5.5% 5.2% 0.0% 5.1% Abduction 11.4% 4.9% 0.6% 0.9% 4.7% 1.8% 4.3% 16.7% 4.4% Recruitment into militia 2.0% 22.0% 7.3% 0.9% 7.8% 7.3% 7.5% 33.3% 7.8% Separation from parents 14.8% 27.6% 26.4% 51.3% 29.3% 23.6% 28.9% 16.7% 28.7% Various protection interventions were implemented including awareness creation, psychosocial support services, hygiene kit and safe spaces. From the evaluation interviews, 82.7% of the respondents were aware of places to seek help in case of sexual violence and rape (80.0% males and 83.0% females) with the least proportion being in Mogadishu (53.7%) and the highest being in Baidoa (98.3%)-Figure 4.13. Figure 4.13: Awareness of places to seek help in case of sexual violence and rape Mogadishu Afgooye Corridor Cadaado Baidoa Female Male IDP camps Host Community Total 53.7% 82.1% 97.2% 98.3% 83.0% 80.0% 82.7% 83.3% 82.7% Percentage of the respondents Page 39 of 97 Post rape/sexual violence services known to the protection beneficiaries were medical treatment (73.7%), psychosocial support (34.4%), and referral to other organizations (29.6%), legal services (27.5%), and protection shelters (29.5%)-Table 4.23. From the FGDs there was an information gap on the availability of protection shelters with a high demand for such services for boys who were being recruited into militias. Awareness of hygiene kits was found to be very high among all the FGDs respondents in all the protection program sites. Table 4.23: Post rape/sexual violence services known to beneficiaries Site Gender Location Total Mogadishu Afgooye Corridor Cadaado Baidoa Female Male IDP camps Host Community Medical treatment 69.8% 91.1% 47.8% 100.0% 73.0% 80.0% 74.3% 16.7% 73.7% Legal services 30.2% 31.7% 29.8% 16.2% 27.0% 32.7% 27.6% 16.7% 27.5% Protection shelters 12.8% 48.8% 29.8% 29.9% 30.1% 23.6% 29.1% 66.7% 29.5% Psychosocial support 37.6% 50.4% 12.9% 46.2% 33.2% 45.5% 34.6% 16.7% 34.4% Referral to other organizations 11.4% 26.0% 31.5% 53.8% 28.9% 36.4% 29.8% 16.7% 29.6% From the interviewed protection beneficiaries, 36.2% indicated that they or their family members had accessed SGBV services in the 6 months preceding the evaluation (29.1% males and 36.9% females). Once again, Mogadishu recorded the least cases of respondents assessing SGBV services (15.4%) and this was attributed to multiple organizations offering protection services in Benadir region-Figure 4.14. In addition, 76.7% of the respondents indicated that they would report SGBV cases if they came across them in their households and communities. Figure 4.14: Access to SGBV services and reporting of SGBV services Mogadishu Afgooye Corridor Cadaado Baidoa Female Male IDP camps Host Community Total 15.4% 30.9% 25.3% 84.6% 36.9% 29.1% 36.5% 0.0% 36.2% 55.7% 69.1% 95.5% 82.9% 76.8% 76.4% 76.6% 83.3% 76.7% Would report a case of sSGBV if any member of your household encountered it Respondent or or any member of your family accessed SGBV response services in the preceding 6 months Page 40 of 97 Asked about where they would report the SGBV cases, the interviewed program beneficiaries indicated to camp leaders (68.3%), local policemen (40.5%), humanitarian workers in CBOs and NGOs (33.3%), community leaders (27.8%), religious leaders (23.0%), relatives and family members (18.4%), health workers (13.6%) and social workers (6.7%)-Table 4.24. This is an indication of the community members’ trust in both formal and informal protection mechanisms. Table 4.24: Places where SGBV cases would be reported Program site Gender Location Total Mogadishu Afgooye corridor Cadaado Baidoa Female Male IDP camps Host Communit y Camp leader 68.7% 87.1% 48.8% 85.6% 67.2% 78.6% 68.8% 20.0% 68.3% Clan/community leader 66.3% 51.8% 8.2% 8.2% 26.7% 38.1% 27.9% 20.0% 27.8% Local police men 55.4% 18.8% 45.9% 37.1% 40.5% 40.5% 40.5% 40.0% 40.5% Religious leaders 19.3% 12.9% 18.2% 43.3% 23.2% 21.4% 22.6% 60.0% 23.0% Relative or family member 20.5% 41.2% 14.7% 3.1% 18.6% 16.7% 18.6% 0.0% 18.4% Humanitarian workers (NGOs and CBOs) 26.5% 58.8% 29.4% 23.7% 32.1% 45.2% 33.5% 20.0% 33.3% Health care workers 4.8% 30.6% 4.1% 22.7% 13.0% 19.0% 13.7% 0.0% 13.6% Social workers 1.2% 8.2% 0.6% 20.6% 5.3% 19.0% 6.7% 0.0% 6.7% In 89.2% of the visited protection beneficiary households, there were children below 15 years (Figure 4.15). Only 48.8% of the households with children were aware of safe spaces for children with the least proportion being in Afgooye (22.6%). Among caregivers who were aware of safe spaces, only 44.9% reported that their children used them with the least proportion being in Cadaado (18.9%). In 49.5% of the household where children accessed safe spaces, safety and welfare of children was reported to have increased with the least increase being in Mogadishu (38.1%)-Table 4.25. After SHARPEN 1, the safe spaces were handed over to the community members as part of the exit plan and as such there were no activities under SHARPEN 2 program to promote use of spaces and service improvement in these safe spaces. Figure 4.15: Households with children aged below 15 years Mogadishu Afgooye Corridor Cadaado Baidoa Female Male IDP camps Host Community Total 92.6% 93.5% 94.4% 72.6% 89.6% 85.5% 89.3% 83.3% 89.2% Percentage of the HHs Page 41 of 97 Table 4.25: Safe spaces Program site Gender Location Total Mogadishu Afgooye Corridor Cadaado Baidoa Female Male IDP camps Host Communit y Awareness of safe spaces 52.9% 22.6% 53.6% 68.2% 48.4% 53.2 % 48.5% 80.0% 48.8% Access to safe spaces by children 57.5% 57.7% 18.9% 63.8% 44.6% 48.0 % 45.3% 25.0% 44.9% Improvement in the safety and welfare of the child when he/she accessed a child-friendly space 38.1% 60.0% 47.1% 59.5% 50.5% 41.7 % 49.1% 100.0% 49.5% The much success documented on the program indicators was attributed to dedicated staff, good feedback mechanisms, and a good partnership with IDPs’ gatekeepers and other stakeholders in the country. The one unforeseen outcome under the program was the displaced households from Laascaanood, who came to the project sites following eviction and 294 of these households were provided with medicine and nutrition supplies. Here are some of the quotes from KII respondents on program effectiveness: ………………………“SOS Children’s Villages Somalia brought emergency ambulance services that have been responding to health emergencies at night and when patients cannot travel to the hospitals” ………..…… [KII respondent, Garsabaalay] …………………. “Many mothers with severe malnutrition and children with chronic malnutrition have been reached with lifesaving services, we would have lost them”…………………. [KII respondent, Baidoa] ………“Under these program we have constructed latrines, hand washing stations and wells where a lot of households can now access water from” ……….. [KII respondent Luuq] Here some quotes from program beneficiaries on program effectiveness: …………..“Girls and women are able to express themselves in case of any incident since we got female protection workers in the facilities” ……………. [Female FGD respondent, Adaado] ……………… “I have travelled from Bardale District because my daughter was suffering from malnutrition. She has since been treated here at the SOS Children’s Villages hospital in Baidoa. I am grateful for the services received” ……………….. [Female FGD participant, Baidoa] ………………. “Under this program we have received nutrition biscuits for children, and porridge for pregnant women” …………….……………….. [Female FGD participant, Mogadishu] 4.5 Program Efficiency [Evaluation Rating: Strong] On the efficiency in the implementation of the SHARPEN II program, the evaluation sought to assess whether the objectives were achieved economically by the development intervention. Efficiency in the implementation of individual interventions is usually assessed through a comparison of the output level indicators achieved in the projects against planned targets and in cost efficiency through a comparison of budget to expenditure and costs to outputs. The efficiency of individual sectoral interventions was relatively high given the operating context in Somalia.53F 54 However, only half of the portfolio outcome level indicators targets were met upon comparison of baseline and end term figures. On decision￾making process of budget distribution, at the inception phase, budget allocations across each sector and partner were set by CRS. Each consortium partner was responsible for its own total share of the budget and allocated across the respective work packages. As the budgets were “locked in”, there was generally little flexibility to manoeuvre the budgets. The results were then examined in relation to the proportion of realized outputs versus planned outputs and the evaluation team was not satisfied in the scope of outputs from the program (50.0%). The consortium partners reported no challenges in working jointly under the program. In addition, the partnership gained greater visibility, by participating sectoral cluster meetings in Somalia. Findings indicate that the organisations complemented each other quite well as each had their own role and specific expertise that was suitable to the intervention areas. From the KIIs, at no time did individual organizations’ priorities take precedence over the partnership’s goals. One 54Proportion of targeted activities implemented versus outcomes. Page 42 of 97 aspect that was no strong under the partnership was protocols and practices to ascertain that good practices and lessons learned were recognized and integrated into work practices; there was no evidence of documentation of best practices and lessons learn by the partners. In addition, results-based management principles were not fully exploited by the project and some commitments were not properly followed up on, for example, nutrition and WASH behaviour change messages were not followed up to establish challenges their adoption by program beneficiaries Overall, no delays were reported in the engagement of partner organizations. There was a delayed grant disbursement from CRS but it did not affect the timeliness of the program activities. However, delays in supplies of health products largely caused by the COVID 19 pandemic restrictions were reported in Garsabaalay (Afgooye region). This was in the following drugs: ORS, Amoxicillin syrup, Paracetamol, metronidazole and Phenoxymethylpenicillin. In Dollow, there delays in kick off of some program activities for up to 5 months since community members wanted to take control of car hire for project activities. In addition, in June 2021, there were inter-clan clashes in Dollow among the riverine populations and this led to the suspension of health and nutrition activities and relocation of some of the services to villages occupied by IDPs. Similarly, across all the program sites, ambulance services were appreciated but not timely due to geographical vastness and insecurity at night. Distribution of interventions according to the strength and geographical presence of the partners ensured that kick off was timely. In nutrition, health and WASH interventions, a high number of staff was required especially due to the parallel COVID-19 mitigation measures hence putting strain on staff; as such, there were inevitable delays in accessing health services especially at night while the high demand for health and nutrition services was in some instances associated with long queues and long waiting time in health facilities, nutrition clinics and WASH services access points. In Hudur, transportation costs were higher than anticipated due to security lock downs. Several cost cutting measures were noted in the program including: set up of safe spaces in the health facilities rather than constructing new one; hygiene kits were obtained from the Somalia WASH cluster’s Regional Supply Hub instead of procuring them from a central store which would have been costly; the project met only the cost of transporting them; RUSF was procured from Ethiopia instead of Kenya to reduce the cost of transportation; and distribution of interventions according to the strengthen and geographical presence of the partners ensured that programming was cost effective by leveraging on the existing networks, infrastructure, staff and facilities. Table 4.26 below summarized the program efficiency as follows: the budget allocated for various activities was well and adequately distributed, all the planned program activities were implemented, the consortium of partners was lively and synergistic, there was flexibility to adapt to the changing dynamics in Somalia including insecurity as documented in the change of targeted villages as explained above, the partner organizations had a heavy presence in the sectoral cluster meetings, most services/interventions under the program were offered in a timely manner, there was no documentation of best practices and lessons learnt internally by the partner organizations and only 50.0% of the program outcomes were realized following the implemented interventions. Table 4.26: Summary of efficiency in the program Efficiency aspect Not satisfact ory Least satisfact ory Partiall y satisfac tory Satisf actory Highly satisfact ory Comments Overall budget design and allocation process  • Budgets were largely adequate Efficient implementation of activities  • Efficiency measures in procurement and set up of safe spaces were noted • Partners strength in the sectors and geographic coverage of interventions was optimal Adaptability/flexibility during implementation  • Change of sites were done including in Caadado and Dollow due to implementation challenges Page 43 of 97 Functioning of the consortium  • Regular meetings were helpful and no disagreements were reported Partnership with/presence in the national humanitarian space  • Active presentations in the sectoral cluster meetings were noted Provision of services and support in a timely and reliable manner  • Challenges noted did not compromise the effectiveness and overall outcomes of the project in relation to its established expected accomplishments Presence of protocols and practices to ascertain that good practices and lessons learned are recognized and integrated into work practices  • There was no evidence of good practices and lessons capturing or replication Comparison of outputs verses inputs  Only 50.0% of the program indicators targets were met Here are some quotes on program efficiency from KII respondents and FGD participants: …………“Towards the end of the program, we end a shortage of several drugs due to travel restrictions” ………….. [KII respondent, Afgooye corridor] ………….. “The drugs we lacked were ORS, Amoxicillin syrup, Paracetamol, metronidazole and Phenoxymethylpenicillin” ……………. [KII respondent, Afgooye corridor] ………….. “So far we have not experienced any delays in receiving any program interventions” …………….. [Female FGD participant, Mogadishu] …………. “We only had the initial delays due to agreements execution, after that all services have been delivered on time” ……………… [KII respondent, Luuq] 4.6 Beneficiaries Consultation and Engagement [Evaluation Rating: Strong] Overall, 82.4% of the program beneficiaries indicated that they were consulted (directly or through community and camp leaders and their views given consideration) rights were fully respected and upheld under the program, 14.6% indicated partial respect and upholding of their rights while 3.0% were of their opinion that their rights were mot upheld and not respected (including 6.9% of the protection beneficiaries)-Table 4.27. From the KIIs and FGDs, IDP camp leaders and village committee leaders were used to get feedback from the beneficiaries on the various services offered by the partner organizations. Table 4.27: Respect of beneficiaries rights Variable Variable description Fully Partially Not at all Total Program site Mogadishu 80.20% 12.60% 7.20% 100.0% (677) Afgooye corridor 88.0% 9.8% 2.2% 100.0% (410) Caadado 94.9% 5.1% 0.0% 100.0% (178) Baidoa 76.3% 22.7% 1.0% 100.0% (291) Luuq 66.0% 34.0% 0.0% 100.0% (194) Dollow 80.8% 17.6% 1.6% 100.0% (250) Hudur 100.0% 0.0% 0.0% 100.0% (88) Burdhubo 90.0% 10.0% 0.0% 100.0% (100) Gender Female 82.3% 14.9% 2.8% 100.0%(1888) Male 83.3% 12.7% 4.0% 100.0%(300) Sector Protection 76.0% 17.1% 6.9% 100.0%(567) Health 87.4% 11.9% 0.7% 100.0%(286) Nutrition 88.7% 10.3% 1.0% 100.0%(824) WASH 76.5% 20.4% 3.1% 100.0%(511) Location IDP camps 78.6% 17.8% 3.6% 100.0%(1317) Hot community 88.2% 9.9% 2.0% 100.0%(871) Total 82.4% 14.6% 3.0% 100.0% (2188) Page 44 of 97 Under the program a hotline was available for provision of feedback on the services offered as well as to call for emergency assistance. However, only 65.1% of the program beneficiaries (69.0% females and 40.7% males) reported to be aware of this hotline (Table 4.28). Of those aware of the free hotline, 46.7% had uses it (47.7% females and 40.3% males)-Figure 4.16. KIIs indicate that feedback provided was on emerging needs, water points breakdown, difficulties in using the soaps provided, inadequate WASH kits, discomfort regarding the jericans provided, progress of malnourished children and request for ambulance services and protection shelters. In contrast however, the program staff indicated that the use of the hotline went down under SHARPEN 2 program when compared to the SHARPEN 1 program. Table 4.28: Awareness of the free hotline Varia ble Program site Sector Gender Mogadi shu Afgo oye corri dor Caad ado Baid oa Luu q Doll ow Hud ur Burdh ubo Protec tion Heal th Nutrit ion WA SH Fem ale Mal e Yes 64.5% 72.9 % 65.7% 64.3 % 97.9 % 48.4 % 80.7 % 3.0% 53.8% 68.2 % 76.2% 58.1 % 69.0 % 40.7 % Figure 4.16: Use of the free hotline Mogadishu Afgooye corridor Caadado Baidoa Luuq Dollow Hudur Burdhubo Protection Health Nutrition WASH Female Male IDP Host Total 23.5% 43.2% 93.3% 60.8% 92.8% 51.2% 33.0% 6.0% 50.6% 54.2% 41.0% 47.4% 47.7% 40.3% 55.3% 33.8% 46.7% Percentage of the beneficiaries Page 45 of 97 Asked about their preferences for feedback in future, 53.2% named the hotline, 50.3% named community and camp leaders, 24.6% preferred program staff, and 9.8% indicated phone short message services (SMSs) and 2.3% opted for suggestion boxes. By gender, 56.7% of the females and 31.3% of the males preferred hot lines, 45.8% of the females and 79.0% of the males preferred community and/or camp leaders, 24.4% of the females and 25.7% of the males named program staff, 8.4% females and 18.3% males opted for short messages via phones while 2.6% females and 0.7% males opted for feedback boxes. (Table 4.29). Table 4.29: Preferred feedback channels Variable Variable description Hot line Community/camp leaders Program staff Messages Feedback boxes Program site Mogadishu 81.2% 26.9% 12.7% 2.5% 2.7% Afgooye corridor 55.9% 54.1% 25.4% 12.4% 6.3% Cadaado 39.9% 65.2% 55.1% 15.2% 1.7% Baidoa 77.3% 21.6% 7.9% 17.2% 0.3% Luuq 0.0% 100.0% 48.5% 0.5% 1.0% Dollow 1.2% 100.0% 0.0% 0.0% 0.0% Hudur 51.1% 69.3% 1.1% 1.1% 2.3% Burdhubo 41.0% 74.0% 72.0% 67.0% 0.0% Sector Protection 54.1% 60.3% 27.9% 11.1% 1.1% Health 47.6% 50.7% 31.5% 12.6% 1.4% Nutrition 65.0% 32.6% 26.7% 7.5% 4.5% WASH 36.2% 67.5% 13.7% 10.4% 0.8% Location IDP camps 51.6% 56.2% 29.0% 10.8% 2.7% Host community 55.7% 41.4% 17.9% 8.3% 1.7% Gender Female 56.7% 45.8% 24.4% 8.4% 2.6% Male-W 31.3% 79.0% 25.7% 18.3% 0.7% Total 53.2% 50.3% 24.6% 9.8% 2.3% 4.7 Lessons Learned From the SHARPEN II Program Implementation Nutrition Interventions • Nutrition interventions without food security measures in drought affected populations will improve knowledge but not practices if food sources remain unavailable or inaccessible. There are strong knowledge levels on IYCF and good nutrition in general, but those practices are not followed due negative coping strategies employed due to food shortages in the households. • Context-specific nutrition messaging for men ought to be developed and rolled out, following their daily routines and socialization patterns in order to ensure effective uptake. Men showed lower knowledge levels of ICYF and good nutrition practice and, unlike their female counterparts, there were no support groups for awareness creation or education for male beneficiaries. Health Interventions • Identification of training courses for health workers need to be guided by demand rather than implementers perceived training gaps. Across the health facilities, health care workers asked for these trainings to be guided by their preferences and needs. • Contingency measures for procurement and supply of medicines ought to be put in place in pandemics and fragile nations with rapidly changing markets. Delays in supply of drugs for health facilities were noted in Baidoa and the Afgooye corridor due to the COVID-19 pandemic related restrictions. • In view of sparse distribution of health facilities, unavailability of transport services, insecurity and the absence of a working health emergency response system, community health workers are useful in supplementing static health facilities. WASH Interventions • Use of hard water influences the preferred soaps by beneficiaries while the satisfaction with WASH kits is influenced by user preferences of beneficiaries as well as the number of household members in families. • There are movements of IDPs populations (influx) which end up increasing the utilization of water and sanitation services. Page 46 of 97 Protection Interventions • Protection shelters should not only target females but also males. From qualitative interviews in Adaado and Luuq, both females and males called for protection shelters for boys who are being forcefully recruited into militia groups. All Interventions • Empowerment of community structures (e.g. gatekeepers in the IDP camps, village committees, and water management committees) through capacity building, consultation and collaboration enhances awareness, involvement and buy in of communities increased chances of sustainability. • Conflict sensitive programming – to ensure delays are averted in future and avoid harm to beneficiaries due to our interventions, there is need to have a conflict sensitive lens when designing new projects. Project teams should understand the tensions that exist and potential connectors. In Dollow, delays were noted due to conflict of interest over hire of project vehicles. • Geographic shifting of activities during crises to meet needs when possible, helps provide critical services to those who need it most; for example, SHARPEN II shifted interventions to the riverine IDPs at their new displacement villages in Laascaanood from Dollow. This further indicates the need for regular review of the program activities vis-à-vis the community needs and flexibility in the program plans and finances. 4.8 Best Practices in the SHARPEN II Program Implementation • Use of regular patients’ feedback surveys in the SOS Children’s Villages health facilities helped in documenting health service delivery gaps and improving on the same (e.g. long waiting time, stock outs and poor communication by health care workers). This was reported by mothers seeking delivery services in SOS Children’s Villages health facilities. • Establishment of child friendly spaces in health facilities further supported the protection of children while lowering barriers to addressing their health and psychosocial needs. • Mobile health teams able to deliver services closer to hard-to-reach areas and also provide an avenue for follow up of children under treatment. This is supported by community surveillance mechanisms more strongly when compared to static health facilities. • Holding regular review meetings with various stakeholders and partners including other international NGOs, local NGOs, sectoral cluster groups and the Ministry of Health to evaluate progress and share experiences was found to be a catalyst for decision making to address emerging issues in the Somaliland drought mitigation. 4.9 Conclusion This evaluation was conducted to document the relevance, effectiveness and efficiency of the program. From the evaluation findings, there was broad merit in implementing health, nutrition, WASH and protection interventions under the SHARPEN II program. The SHARPEN II program was found to be in line with the Somalia National Development Plan, the Somalia Humanitarian Response Plan (2021), SGDs and the nutrition, health and WASH and GBV sectoral priorities. The voices of the beneficiaries do confirm that the program was addressing four of their top most needs and was implemented for their best interest and has addressed part of their household needs. There was relevance in choosing to work with the three local partners due to their extensive local networks and offices across the country as well as expertise in implementing health, nutrition, and protection and WASH programs in various districts of Somalia. This partnership was strong in terms of ease of communication through regional offices; partners have local presence in the areas of operation; partners have on-going relationship with communities, thereby making the mobilization of local resources for implementation easier; partners have indigenous technical knowledge and understanding of local conditions, local culture and local coping strategies; the implementing partner staff were mostly locals, and therefore faced no language difficulties, and; partners have developed networking and collaborative relationships with other agencies and government departments. On effectiveness, the planned interventions were implemented and in some cases with wide reach, but the program produced mixed results. Only seven of the fourteen (fifty percent) program indicators s have been met with gaps in the WASH sector, protection sector and nutrition sector. Specifically, Page 47 of 97 satisfaction with WASH kits was low in terms of quantity, quality and contents while the number of people utilizing water services provided with OFDA support was low, the functionality of hand washing stations in health facility was suboptimal, exclusive breastfeeding was not done in the target households, children aged six to twenty four months were not having the expected dietary diversification and we didn’t have enough parents reporting improvement in sense of safety and well-being of children as a result of accessing a child-friendly spaces. In addition, nutrition and WASH knowledge was still poor amongst males and in the host communities. On efficiency, the program was found to be timely in implementing the various interventions as well as in adapting to the changing dynamics in Somalia including the COVID-19 pandemic and displacement of households in Laascaanood. However, the program was not efficient in mitigation the delay sin the supply of medicines in Afgooye and Baidoa regions. Similarly, the program lacked protocols and practices to ascertain that good practices and lessons learned are recognized and integrated into work practices. Overall, the evaluation team was further satisfied with the following efficiency aspects: the overall budget design and allocation process; efficient implementation of activities; adaptability/flexibility during implementation; a functioning of the consortium ; partnership with/presence in the national humanitarian space; provision of services and support in a timely and reliable manner. The program’s hotline remains relevant for feedback relays and so do community leaders and IDP camps gate keepers. However, ye hotline is not fully known by the program beneficiaries hence low utilization. Lastly, several sectoral best practices and lessons learnt have been documented for improving the design and implementation of future look alike programs in Somalia and other similar contexts. 4.10 Recommendations 4.10.1 Nutrition Recommendations for CRS • Explore the best approaches to improve nutrition messaging targeting males. This could include religious leaders who have much respect from males and other male dominated forums. Recommendations for the Donor • Inclusion of crisis modifier budgets and consideration of resilience activities as part of the project to ensure sustainability of nutrition interventions that are linked to food security. 4.10.2 Health Recommendations for Partners • Ensure that trainings offered to health care workers are aligned to their needs by conducting training needs assessments. • Ensure stock pre-positioning systems are in place to respond when there is an emergency, such as the COVID-19 pandemic, which may hinder fast procurement of health products and technologies. • Continue supplementing static health services with community outreaches and primary health care services through community workers in view of sparse distribution of health facilities, unavailability of transport services, insecurity and the absence of a working health emergency response system. 4.10.3 Protection Recommendations for Partners • Come up with outcome indicators for monitoring under the protection sectors-Under SHARPEN 1 there was only one outcome indicator and under SHARPEN 2 program there was no outcome indicator thus difficulties in evaluation the performance of this sector. • Establish safe spaces for youthful males who are being forcefully recruited into militias against their wishes in Adaado, Luuq and Dollow areas. Safe spaces were not targeted by programming under SHARPEN 2 and as such, awareness on the same should be prioritized. • Factor in family strengthening and kinship interventions in view of the high number of children either at risk of losing parental care or those who have already lost parental care. • Create more awareness on the dangers of physical and humiliating punishment for children which remains rampant in the visited program sites. Page 48 of 97 • Create more awareness on legal and psychosocial services available for survivors of sexual violence. As it is, there is little information on these. • Create more awareness on available protection shelters and safe spaces. There is demand for these services but awareness on the same is lacking. Safe spaces were not targeted by programming under SHARPEN 2 and as such, awareness on the same should be prioritized • Ensure services offered in safe spaces and child friendly centers meet the needs of the beneficiaries. Regular satisfaction survey will help document gaps in these services for upfront remedy. • Improve record keeping and data management practices-Data on psychosocial services was not provided though a request for the same (disaggregated by gender and age) was made. Recommendations for CRS • Support policy dialogue sessions to address the longstanding practice of early and forced marriages. 4.10.4 WASH Recommendations for CRS and the Donor • During the program design stage, factor in the influx of IDPs populations which may increase the utilization of water and sanitation services. Recommendations for Partners • Prioritize solar lamps on latrines and locks on latrine doors especially in IDPs camps. Darkness and the insecurity that comes with accessing latrines was described a major contributor to open defecation especially in Baidoa and the Afgooye corridor. • Establish strategic waste disposal pits in the IDP camps for ease of management of household solid wastes. Environmental health was not targeted by the SHARPEN 2 program interventions, but the beneficiaries have made a request for waste disposal pits. • Ensure that WASH kits supplied are informed by preferences of the targeted beneficiaries for uptake and satisfaction purposes. Beneficiaries had their own preferences for soaps that lather well when used with hard water while the number required should be guided by the actual number of household members rather than an estimated average of family size. • Replace the collapsed forms of jerricans with the non-collapsed ones which are more preferred in the Somalia context. • Create more awareness on the need to wash hands before touching and preparing food which are poorly understood by the program beneficiaries. Page 49 of 97 SECTION FIVE: LIST OF ANNEXES Annex 1: List of Key Informants Annex 2: Distribution of beneficiaries and quantitative household survey sample size Annex 3: Analysis of indicators Annex 4: Health-Quantitative Household Survey Tool Annex 5: Nutrition-Quantitative Household Survey Tool Annex 6: Protection-Quantitative Household Survey Tool Annex 7: Water, Sanitation and Hygiene (WASH)-Quantitative Household Survey Tool Annex 8: Focus Group Discussion (FGD) Guide for Program Beneficiaries Annex 9: Key Informant Interview Guide for Program Staff and Partners Annex 10: Data Quality Control Measures Annex 11: Training of the Enumerators Annex 12: Terms of Reference Page 50 of 97 SECTION SIX: LIST OF ANNEXES Annex 1: List of Key Informants S.No Names Gender Title Region 1) Mohamed Ahmed Waydow Male District Official Garsabaalay 2) Dahir Mayow Mukhtar Female Nutrition Officer Baidoa 3) Muhdin Adan Ibrahim Male Health Ministry Official Hudur 4) Yussuf Dugow Male Humanitarian Coordinator Dollow 5) Halima Ahmed Daleel Female Health Care Worker Mogadishu 6) Abdiwahit Ahmed Jamac Male Program Manager Luuq 7) Adan Abdirahman Male Ministry Of Planning Bakool 8) Abdirizack Ibrahim Mohamed Male District Medical Officer Luuq 9) Farhan Arab Omar Female Nutritionist Dollow 10) Abdirahman Omar Abdi Male Medical Officer Garsabaalay 11) Abdi Noor Mohamed Farah Male Nurse Adaado 12) Hussein Ismail Ali Male Field team leader Baidoa 13) Dahabo Abdi Aden Male Nutrition Nurse Luuq 14) Fartun Abdulkhadir Mohamed Female GBV Coordinator Adaado 15) Abbas Tahlil Male Team Leader Adaado 16) Roda Jama Rooble Female Medical Officer Mogadishu 17) Yasmin Salad Omar Female Nurse Mogadishu 18) Abdirahman Ibrahim Abdullahi Male Nutrition Officer Mogadishu 19) Mustaf Maalim Issack Male Health Care Worker Dollow 20) Hassan Mohamed Ibrahim Male Project Coordinator Baidoa 21) Mohamed Moalim Dhaqane Male Hospital Director Baidoa Page 51 of 97 Annex 2: Distribution of beneficiaries and quantitative household survey sample size Thematic area Region Partner No. of Beneficiaries Sample Size Protection Mogadishu SSWC 5,070 139 Baidoa SSWC 7,512 205 Adaado SSWC 7,714 211 Afgoye Corridor SSWC 4,580 125 Sub-Total 24,876 680 WASH Mogadishu SOS 21,127 132 Afgoye Corridor SOS 3,660 23 Baidoa SOS 17,575 110 Dollow Trócaire 32,274 203 Luuq Trócaire 26,449 166 Burdhubo Trócaire 7,318 46 Sub-Total 108,403 680 Health Mogadishu SOS 90,158 317 Afgoye Corridor SOS 13,158 51 Baidoa SOS 32,236 124 Hudur SOS 7932 31 Dollow Trócaire 7,825 30 Luuq Trócaire 20,945 81 Burdhubo Trócaire 11,945 46 Sub-Total 176,267 680 Nutrition Mogadishu SOS 48,378 381 Afgoye Corridor SOS 26,830 212 Baidoa SOS 10,931 86 Hudur SOS 12,456 98 Dollow Trócaire 12,256 97 Luuq Trócaire 16,187 128 Burdhubo Trócaire 2,332 18 Sub-Total 129,370 1020 Total 438,916 3,060 Annex 3: Analysis of indicators Objective statement Indicator Source of data Data collection method Unit of Analysis Denominator Numerator How will the data be analyzed Results presentation Target populations access comprehensive primary care and have reduced morbidity. Number and percentage of community members who can recall target health education message Household heads/senior most females in the households Health quantitative survey tool question 17 Health interventions beneficiaries Total number of health interventions beneficiaries Health beneficiaries who can recall target health education message Frequency distribution Summary table/graphics with cross tabulation by region, residence (IDPs and non IDPs) and gender Malnutrition levels in young children (<5) and pregnant/lactatin g women (PLWs) are decreased. Proportion of infants 0-5 months of age who are fed exclusively with breast milk Nutrition beneficiaries in the households with children o-5 months Nutrition quantitative survey tool question 22 Nutrition beneficiaries with children 0- 5months Infants 0-5 months of age in the surveyed households Infants 0-5 months of age who received only breast milk during the previous day Frequency distribution Summary table/graphics with cross tabulation by region and residence (IDPs and non IDPs) Proportion of children 6- 23 months of age who receive foods from 4 or more food groups Nutrition beneficiaries in the households with children 6-23 months Nutrition quantitative survey tool question 27 Nutrition beneficiaries in the households with children 6- 23 months Total number of children 6-23 months in the surveyed households Total number of children 6-23 months fed on 4 or more food stuffs in the preceding 24 hours Frequency distribution Summary table/graphics with cross tabulation by gender, region and residence (IDPs and non IDPs) The risk of GBV against vulnerable groups (including children) is reduced and survivors of GBV access comprehensive services. Percentage of children whose parents report improvement in sense of safety and well-being of children as a result of accessing a child-friendly space Protection beneficiaries with children below 15 years Protection quantitative survey tool question 24 Protection beneficiaries with children below 15 years Total number of children below 15 years in the surveyed households Children below 15 years in the surveyed households whose parents report improvement in sense of safety and well-being of children as a Frequency distribution Summary table/graphics with cross tabulation by gender, region and residence (IDPs and non IDPs) Page 53 of 97 result of accessing child friendly spaces Vulnerable populations access clean water and have improved hygiene. People targeted by the hygiene promotion program who know at least three (3) of the five (5) critical times to wash hands WASH beneficiaries WASH quantitative survey tool question 15 Wash beneficiaries _________ ___________ Numerical count Summary table/graphics with cross tabulation by gender, region and residence (IDPs and non IDPs) Number of people directly utilizing improved sanitation services provided with OFDA funding WASH beneficiaries Phone Survey Wash beneficiaries ___________ ___________ Numerical count Summary table with cross tabulation by gender, region and residence (IDPs and non IDPs) Average number of users per functioning toilet WASH beneficiaries WASH quantitative survey tool question 18 Wash beneficiaries Enumerated number of functional toilets Estimated population of toilet users Mean Summary table with cross tabulation by gender, region and residence (IDPs and non IDPs) Number of people directly utilizing improved water services provided with OFDA funding WASH beneficiaries KII and monitoring data from Trócaire staff Wash beneficiaries Total volume of water supplied per day in liters for drinking, cooking, and hygiene Estimated size of the population supplied with water per day for drinking, cooking, and hygiene Mean Summary table with cross tabulation by gender, region and residence (IDPs and non IDPs) Average liters/person/day collected from all sources for drinking, cooking, and hygiene WASH beneficiaries KII and monitoring data from Trócaire staff Wash beneficiaries Total volume of water supplied per day in liters for drinking, cooking, and hygiene Estimated size of the population supplied with water per day for drinking, cooking, and hygiene Mean Summary table with cross tabulation by gender, region and residence Page 54 of 97 (IDPs and non IDPs) Estimated safe water supplied per beneficiary in liters/person/day WASH beneficiaries KII and monitoring data from Trócaire staff Wash beneficiaries Total volume of safe water supplied to beneficiaries per day in liters Estimated size of the population supplied with safe water per day Mean Summary table with cross tabulation by region and residence (IDPs and non IDPs) Percent of hand washing stations built or rehabilitated in health facilities that are functional Health care workers KII and monitoring data from Trócaire and SOS staff Health facilities Total number of health facilities that had hand washing stations rehabilitated Total number of functional hand washing stations after rehabilitation Frequency distribution Summary table/graphics with cross tabulation by type of health facility, region and residence (IDPs and non IDPs) Percent of households targeted by WASH program that are collecting all water for drinking, cooking, and hygiene from improved water sources WASH beneficiaries WASH quantitative survey tool question 22 Wash beneficiaries Total number of WASH beneficiaries surveyed Number of WASH beneficiaries who report collecting all water for drinking, cooking, and hygiene from improved water sources Frequency distribution Summary table/graphics with cross tabulation by gender, region and residence (IDPs and non IDPs) Percent of households reporting satisfaction with the contents of the WASH NFIs received through direct distribution (i.e. kits) or voucher WASH and NFI beneficiaries WASH quantitative survey tool question 26 Wash beneficiaries Total number of WASH and NFI beneficiaries who received WASH NFI kits, vouchers or cash Total number of WASH and NFI beneficiaries who received WASH NFI kits, vouchers or cash and were satisfied with their contents Frequency distribution Summary table/graphics with cross tabulation by gender, region and residence (IDPs and non IDPs) Percent of households reporting satisfaction with the quantity of WASH and NFI beneficiaries WASH quantitative Wash beneficiaries Total number of WASH and NFI beneficiaries who Total number of WASH and NFI beneficiaries who Frequency distribution Summary table/graphics with cross Page 55 of 97 WASH NFIs received through direct distribution (i.e. kits), vouchers, or cash survey tool question 27 received WASH NFI kits, vouchers or cash received WASH NFI kits, vouchers or cash and were satisfied with the quantities supplied tabulation by gender, region and residence (IDPs and non IDPs) Percent of households reporting satisfaction with the quality of WASH NFIs received through direct distribution (i.e. kits), vouchers, or cash WASH and NFI beneficiaries WASH quantitative survey tool question 28 Wash beneficiaries Total number of WASH and NFI beneficiaries surveyed Total number of WASH and NFI beneficiaries reporting satisfaction with the quality of received WASH NFI kits, vouchers or cash Frequency distribution Summary table/graphics with cross tabulation by gender, region and residence (IDPs and non IDPs) Annex 4: Health-Quantitative Household Survey Tool CATHOLIC RELIEF SERVICES SOMALIA: THE SHARPEN II PROGRAM END TERM EVALUATION SURVEY IN SOUTH CENTRAL SOMALIA End Term Evaluation Household Survey on Health Interventions -Questions to be answered by the Head of the Household or the senior most female in the household phone calls. Informed consent: Hello, my name is ________________ and I am making a phone call on behalf of SOS Somalia/Trócaire Somalia/Save Somali Women and Children (read the applicable partner organization) to conduct an assessment regarding health services they have been offering to persons like yourself in your community over the last two years. As we embark on this this assessment, I would like to talk to a few people like yourself who have benefited from the program to understand your experiences from the services you received. Benefits of the study: The information you provide will be adopted for the improvement of similar services in future. Risks of the study: There are no direct or indirect risks in your participation in this interview. Confidentiality: Your identity and responses shall be treated with confidentiality and all the information you give will only be used for the purposes of this study. The information you share today is confidential. We will not share your information with anyone else without your permission. Consent to participate in the study: You may choose not to participate in the study since participation in the study is voluntary. Would you be willing to allow me to continue with the interview? [1] No (If No: Thank him/her and end the interview) [2] Yes If yes: With your permission, I hope I can now start the discussion. The interview should take no more than 15 minutes. Answer the questions as accurately as possible. But before I begin, I am ready to answer any questions that you may have about this interview. S/NO Question Options/Responses Coding instructions Identifiers and Social Demographic Background 1) Questionnaire number ___ ___ ___ -A maximum of three digits -Mandatory 2) Interviewers code ____ -Numerical values 1 t0 10 --Mandatory 3) Date of the interview __ __ /__ __/ __ __ __ __ -To be picked automatically by the laptop or tablet --Mandatory 4) Start time ____ ____ ___ ___ -To be picked automatically by the laptop or tablet --Mandatory 5) Location of the respondent 1) Mogadishu 2) Afgooye Corridor 3) Cadaado 4) Baidoa 5) Luuq 6) Dollow 7) Hudur 8) Burdhubo -Single response -Mandatory 6) What is your age in years as at last birthday? ___ ___ __ -A maximum of three digits -Mandatory Page 57 of 97 7) What is your gender? 1-Female 2-Male -Single response -Mandatory 8) What is your position in the household? 1-Head of the household (male or female) 2-Senior most female in the household -Single response -Mandatory 9) What is the highest level of Education you have achieved? A-Low Education Bracket 1) None 2) Informal education 3) Religious education 4) Primary School Incomplete B-Moderate Education Bracket 5) Primary school complete 6) Secondary School incomplete 7) Secondary School Complete C-High Education Bracket 8) TVET (technical vocation education) 9) Tertiary/college 10) Other (Specify)________ -One response only -Mandatory 10) What is your current marital status? 1) Single 2) Married 3) Separated 4) Divorced 5) Widowed/widower 6) Other (specify)________ -One response only -Mandatory 11) How would you describe your place of residence? 1-IDP camp 2-Host Community -Single response -Mandatory 12) Are you currently living with a disability? 1-Yes 2-No -Single response -Mandatory 13) Is any other member of your household living with a disability? 1-Yes 2-No -Single response -Mandatory If Yes move to 14 If No move to 15 14) If Yes, What type of disability? 1-Vision Impairment. 2-Deaf or hard of hearing. 3-Mental health conditions. 4-Intellectual disability. 5-Acquired brain injury. 6-Physical disability. 7-Other (specify)____________ Multiple response Relevance 15) Over the last two years what have been your households’ greatest needs? [Do not read responses: Tick all mentioned answers] 1-Health 2-Nutrition 3-Food 4-Water 5-Hygiene and sanitation 6-Nonfood items 7-Protection and security 8-Shelter 9-Identification documents 10-Animal health services 11-Livelihood/Source of income 12-Children play grounds and safe spaces 13-Other (Specify):_______ -Multiple response -Mandatory Effectiveness Page 58 of 97 16) Have you received any health education message over the last 6 months? 1-Yes 2-No -Single response -Mandatory If Yes move to 16, If No move to 20 17) What message did you receive? [Do not read responses; tick all responses given] 1) Safe water storage in clean containers 2) Drinking clean water 3) Breastfeeding for children and mothers 4) Proper hand washing at critical times 5) Attending regular ANC clinics 6) Pregnancy complications and their management 7) Immunization of children against diseases 8) Maintenance of personal hygiene 9) Proper food preparation and handling 10) Seeking treatment in health facilities 11) Deworming practices 12) Use of skilled delivery services 13) Use of post natal visits or care 14) Other (specify)____________ Multiple response 18) What was the source of the health message you received? 1) Health care workers in health facilities 2) Community health care workers 3) Community health groups such as mothers support groups and community health units 4) Radio 5) Television 6) Mobile phones (calls or messages) 7) Internet including social media 8) Brochure/Reading materials 9) Relatives and family members 10) Other (specify) ___________ Multiple response 19) Have you put the messages you received into use? 1-Yes-fully 2-Yes-partially 3-No Single response 20) What challenges do you still face in accessing health services? 1) Insecurity 2) Unavailability of transport services to health facilities 3) Long distance to health facilities 4) Unavailability of medicines in health facilities (stock outs) 5) Unavailability of some health services 6) High/Unaffordable cost of health services 7) Unavailability of health care workers 8) Health care workers negative attitude 9) Unfavourable health facility operation time (opening and closing) Multiple response Page 59 of 97 10) Health services are not culturally sensitive 11) Language barriers 12) Availability of traditional healers in the community 13) Others (specify)____________ Engagement of beneficiaries/Accountability to beneficiaries 21) In your opinion, were your rights respected/upheld in receiving the health services? 1-Fully 2-Partially 3-Not at all Single mandatory response 22) Were you aware of the hotline for providing feedback to program staff regarding areas of concern, satisfaction and dissatisfaction over the services provided to you? 1-Yes 2-No Single mandatory response 23) Did you at any point provide feedback or complains regarding health services received under this program? 1-Yes -No Single response 24) What mechanism would you feel safe and confident to provide feedback over the services you receive? 1-Hot line 2-Community/camp leaders 3-Program staff 4-Messages 5-Feeback boxes 6-Other (specify)______________ Multiple response 25) Interview end time: _____:_____ To be picked automatically by tablet/laptop for all respondents Thank you very much for your time and your valuable information, it will help SOS, Trócaire and Save Somali Women and Children improve health services delivery in this region Page 60 of 97 Annex 5: Nutrition-Quantitative Household Survey Tool CATHOLIC RELIEF SERVICES SOMALIA: THE SHARPEN II PROGRAM END TERM EVALUATION SURVEY IN SOUTH CENTRAL SOMALIA End Term Evaluation Household Nutrition Survey Questions to be answered by the Head of the Household or the senior most female in the household phone calls. Informed consent: Hello, my name is ________________ and I am making a phone call on behalf of SOS Somalia/Trócaire Somalia/Save Somali Women and Children (read the applicable partner organization) to conduct an assessment regarding nutrition services they have been offering to persons like yourself in your community over the last two years. As we embark on this this assessment, I would like to talk to a few people like yourself who have benefited from the program to understand your experiences from the services you received. Benefits of the study: The information you provide will be adopted for the improvement of similar services in future. Risks of the study: There are no direct or indirect risks in your participation in this interview. Confidentiality: Your identity and responses shall be treated with confidentiality and all the information you give will only be used for the purposes of this study. The information you share today is confidential. We will not share your information with anyone else without your permission. Consent to participate in the study: You may choose not to participate in the study since participation in the study is voluntary. Would you be willing to allow me to continue with the interview? [1] No (If No: Thank him/her and end the interview) [2] Yes If yes: With your permission, I hope I can now start the discussion. The interview should take no more than 15 minutes. Answer the questions as accurately as possible. But before I begin, I am ready to answer any questions that you may have about this interview. S/No Question Options/Responses Coding instructions Identifiers and social demographic markers 1) Questionnaire number ___ ___ ___ -A maximum of three digits -Mandatory 2) Interviewers code ____ -Numerical values 1 to 10 --Mandatory 3) Date of the interview __ __ /__ __/ __ __ __ __ -To be picked automatically by the laptop or tablet --Mandatory 4) Start time ____ ____ ___ ___ -To be picked automatically by the laptop or tablet --Mandatory 5) Location of the respondent 1) Mogadishu 2) Afgooye Corridor 3) Cadaado 4) Baidoa 5) Luuq 6) Dollow 7) Hudur 8) Burdhubo -Single response -Mandatory Page 61 of 97 6) What is your age in years as at last birthday? ___ ___ __ -A maximum of three digits -Mandatory 7) What is your gender? 1-Female 2-Male -Single response -Mandatory 8) What is your position in the household? 1-Head of the household (male or female) 2-Senior most female in the household -Single response -Mandatory 9) What is the highest level of Education you have achieved? A-Low Education Bracket 1) None 2) Informal education 3) Religious education 4) Primary School Incomplete B-Moderate Education Bracket 5) Primary school complete 6) Secondary School incomplete 7) Secondary School Complete C-High Education Bracket 8) TVET (technical vocation education) 9) Tertiary/college 10) Other (Specify)________ -One response only -Mandatory 10) What is your current marital status? 1) Single 2) Married 3) Separated 4) Divorced 5) Widowed/widower 6) Other (specify)________ -One response only -Mandatory 11) How would you describe your place of residence? 1-IDP camp 2-Host community -Single response -Mandatory 12) Are you currently living with a disability? 1-Yes 2-No -Single response -Mandatory 13) If yes, what type of disability? 1-Vision Impairment. 2-Deaf or hard of hearing. 3-Mental health conditions. 4-Intellectual disability. 5-Acquired brain injury. 6-Physical disability. 7-Other (specify)____________ -Multiple response Relevance 14) Over the last two years what have been your households’ greatest needs? [Do not read responses: Tick all mentioned answers] 1-Health 2-Nutrition 3-Food 4-Water 5-Hygiene and sanitation 6-Nonfood items 7-Protection and security 8-Shelter 9-Identification documents 10-Animal health services 11-Livelihood/Source of income 12-Children play grounds and safe spaces 13-Other (Specify):_______ -Multiple response -Mandatory Effectiveness Page 62 of 97 15) Have you received any nutrition messages from SOS Somalia/Trócaire Somalia/Save Somali Women and Children over the last two years? 1-Yes 2-No -Single mandatory response If NO move to 17, If Yes move to 16 16) What was the source of the nutrition message you received? 1) Health care workers in health facilities 2) Community health care workers 3) Community health groups such as mothers support groups and community health units 4) Radio 5) Television 6) Mobile phones (calls or messages) 7) Internet including social media 8) Brochure/Reading materials 9) Relatives and family members 10) Other (specify) _______________ Multiple response 17) How long after baby delivery should breastfeeding be introduced? 1-Within 1 hour 2-After one hour (including days and weeks) Single response 18) For how long should babies be breastfed in months? ____ ____ 0 to 24 months as options 19) For how long should babies be exclusively breastfed in months?? ______ _____ 0 to 24 months as options 20) At what age should complimentary feeds be introduced in months ____ _______ 0 to 24 months as options 21) Does your household have a child aged below 2 years (below 24 months) of age? 1-Yes 2-No 3-Don’t Know/Not sure Mandatory If Yes move to 22, If No or Don’t Know/Not sure move to 29 22) How many children below 2 years in your household? Number of children: Numerical The subsequent questions should have room for these number of children (as many as the number of children) Page 63 of 97 23) What is the age of the child (NAME) in months? Age of child in months: Instruction to the enumerator: Please verify the age with any certificates like birth certificate or immunization card if available Mandatory If the age in months is more than 24, go to the next child or move to 29 24) What is the sex of the child (NAME)? 1-Male 2-Female Mandatory 25) Was (NAME) breastfed yesterday during the day or at night? 1-Yes 2-No 3-Don’t Know/Not sure -Single response -Mandatory If Yes move to 27 If No or Don’t Know move to 26 26) Sometimes babies are fed breast milk in different ways, for example by spoon, cup or bottle. This can happen when the mother cannot always be with her baby. Sometimes babies are breastfed by another woman, or given breast milk from another woman by spoon, cup or bottle or some other way. This can happen if a mother cannot breastfeed her own baby. Did (NAME) consume breast milk in any of these ways yesterday during the day or at night? 1-Yes 2-No 3-Don’t Know/Not sure 27) Next I would like to ask you about some liquids that (NAME) may have had yesterday during the day or at night. Did (NAME) have any of these? A-Plain water? 1-Yes 2-No 3-Don’t Know/Not sure Single response Single response Page 64 of 97 [Read the list of liquids starting with ‘plain water’] B-Infant formula such as [discuss examples during training]? A-Yes 2-No 3-Don’t Know/Not sure C-Milk such as tinned, powdered, or fresh animal milk? A-Yes 2-No 3-Don’t Know/Not sure Single response D-Juice or juice drinks? A-Yes 2-No 3-Don’t Know/Not sure Single response E-Clear broth? A-Yes 2-No 3-Don’t Know/Not sure Single response F-Yogurt? A-Yes 2-No Single response G-Thin porridge? A-Yes 2-No 3-Don’t Know/Not sure Single response H-Any other liquids such as [water based liquids in local settings] A-Yes 2-No 3-Don’t Know/Not sure Single response I-Any other liquids? A-Yes 2-No Single response 28) Please describe everything that (NAME) ate yesterday during the day or night, whether at home or outside the home. [Read one by one] A-Porridge, bread, rice, noodles, or other foods made from grains? 1-Yes 2-No 3-Don’t Know/Not sure Single response for each B-Pumpkin, carrots, squash, or sweet potatoes that are yellow or orange inside? 1-Yes 2-No 3-Don’t Know/Not sure C-White potatoes, white yams, manioc, cassava, or any other foods made from Roots? 1-Yes 2-No 3-Don’t Know/Not sure D- Any dark green leafy vegetables? 1-Yes 2-No 3-Don’t Know/Not sure Page 65 of 97 E-Vitamin A rich fruits (such as ripe mangoes, ripe papayas etc.] o 1-Yes 2-No 3-Don’t Know/Not sure F-Any other fruits or vegetables? 1-Yes 2-No 3-Don’t Know/Not sure G- Liver, kidney, heart, or other organ meats? 1-Yes 2-No 3-Don’t Know/Not sure H-Any meat, such as beef, pork, lamb, goat, chicken, or duck? 1-Yes 2-No 3-Don’t Know/Not sure I-Eggs? 1-Yes 2-No 3-Don’t Know/Not sure J-Fresh or dried fish, shellfish, or seafood? 1-Yes 2-No 3-Don’t Know/Not sure K-Any foods made from beans, peas, lentils, nuts, or seeds? 1-Yes 2-No 3-Don’t Know/Not sure L-Cheese, yogurt, or other milk products? 1-Yes 2-No 3-Don’t Know/Not sure M-Any oil, fats, or butter, or foods made with any of these? 1-Yes 2-No 3-Don’t Know/Not sure N-Any sugary foods such as chocolates, sweets, candies, pastries, cakes, or biscuits? 1-Yes 2-No 3-Don’t Know/Not sure O-Condiments for flavor, such as chilies, spices, herbs, or fish powder? 1-Yes 2-No 3-Don’t Know/Not sure P-Grubs, snails, or insects? 1-Yes 2-No 3-Don’t Know/Not sure Q-Foods made with red palm oil, red palm nut, or red palm nut pulp sauce? 1-Yes 2-No 3-Don’t Know/Not sure Engagement of beneficiaries/Accountability to beneficiaries Page 66 of 97 29) In your opinion, were your rights respected/upheld in receiving the nutrition services? 1-Fully 2-Partially 3-Not at all Single mandatory response 30) Were you aware of the hotline for providing feedback to program staff regarding areas of concern, satisfaction and dissatisfaction over the nutrition services provided to you? 1-Yes 2-No Single mandatory response 31) Did you at any point provide feedback or complains regarding nutrition services received under this program? 1-Yes -No Single mandatory response 32) What mechanism would you feel safe and confident to provide feedback over the services you receive? 1-Hot line 2-Community/camp leaders 3-Program staff 4-Messages 5-Feeback boxes 6-Other (specify)______________ Multiple response 33) Interview end time: _____ _____ To be picked automatically by tablet/laptop for all respondents Thank you very much for your time and your valuable information, it will help SOS, Trócaire and Save Somali Women and Children improve nutrition services delivery in future programs. Annex 6: Protection-Quantitative Household Survey Tool CATHOLIC RELIEF SERVICES SOMALIA: THE SHARPEN II PROGRAM END TERM EVALUATION SURVEY IN SOUTH CENTRAL SOMALIA End Term Evaluation Household Nutrition Survey Questions to be answered by the Head of the Household or the senior most female in the household phone calls. Page 67 of 97 Informed consent: Hello, my name is ________________ and I am making a phone call on behalf of SOS Somalia/Trócaire Somalia/Save Somali Women and Children (read the applicable partner organization) to conduct an assessment regarding protection services they have been offering to persons like yourself in your community over the last two years. As we embark on this this assessment, I would like to talk to a few people like yourself who have benefited from the program to understand your experiences from the services you received. Benefits of the study: The information you provide will be adopted for the improvement of similar services in future. Risks of the study: There are no direct or indirect risks in your participation in this interview. Confidentiality: Your identity and responses shall be treated with confidentiality and all the information you give will only be used for the purposes of this study. The information you share today is confidential. We will not share your information with anyone else without your permission. Consent to participate in the study: You may choose not to participate in the study since participation in the study is voluntary. Would you be willing to allow me to continue with the interview? [1] No (If No: Thank him/her and end the interview) [2] Yes If yes: With your permission, I hope I can now start the discussion. The interview should take no more than 15 minutes. Answer the questions as accurately as possible. But before I begin, I am ready to answer any questions that you may have about this interview. S/No Question Options/Responses Coding instructions Identifiers and social demographic markers 1) Questionnaire number ___ ___ ___ -A maximum of three digits -Mandatory 2) Interviewers code ____ -Numerical values 1 t0 10 --Mandatory 3) Date of the interview __ __ /__ __/ __ __ __ __ -To be picked automatically by the laptop or tablet --Mandatory 4) Start time ____ ____ ___ ___ -To be picked automatically by the laptop or tablet --Mandatory 5) Location of the respondent 1) Mogadishu 2) Afgooye Corridor 3) Cadaado 4) Baidoa 5) Luuq 6) Dollow 7) Hudur 8) Burdhubo -Single response -Mandatory 6) What is your age in years as at last birthday? ___ ___ __ -A maximum of three digits -Mandatory 7) What is your gender? 1-Female 2-Male -Single response -Mandatory 8) What is your position in the household? 1-Head of the household (male or female) 2-Senior most female in the household -Single response -Mandatory 9) What is the highest level of Education you have achieved? A-Low Education Bracket 1) None 2) Informal education 3) Religious education -One response only -Mandatory Page 68 of 97 4) Primary School Incomplete B-Moderate Education Bracket 5) Primary school complete 6) Secondary School incomplete 7) Secondary School Complete C-High Education Bracket 8) TVET (technical vocation education) 9) Tertiary/college 10) Other (Specify)________ 10) What is your marital status? 1) Single 2) Married 3) Separated 4) Divorced 5) Widowed/widower 6) Other (specify)________ -One response only -Mandatory 11) How would you describe your place of residence? 1-IDP camp 2-Host community -Single response -Mandatory 12) Are you currently living with a disability? 1-Yes 2-No -Single response -Mandatory 13) If yes, what type of disability? 1-Vision Impairment. 2-Deaf or hard of hearing. 3-Mental health conditions. 4-Intellectual disability. 5-Acquired brain injury. 6-Physical disability. 7-Other (specify)____________ Multiple response Relevance 14) Over the last two years what have been your households’ greatest needs? [Do not read responses: Tick all mentioned answers] 1-Health 2-Nutrition 3-Food 4-Water 5-Hygiene and sanitation 6-Nonfood items 7-Protection and security 8-Shelter 9-Identification documents 10-Animal health services 11-Livelihood/Source of income 12-Children play grounds and safe spaces 13-Other (Specify):_______ -Multiple response -Mandatory Effectiveness 15) What are the common forms of child abuse and exploitation in your region? [Do not read responses, tick all responses given] 1) Physical disciplining/assault 2) Sexual abuse and rape 3) Emotional abuse 4) Neglect 5) Early marriages 6) Forced marriages 7) Female Genital Mutilation/Cut 8) Trafficking 9) Abduction 10) Recruitment into militias 11) Separation from parents -Multiple response 16) In case of sexual abuse or rape, do you know of a place where you can seek help in your community? 1-Yes 2-No -Single response Page 69 of 97 17) What help is available for survivors of sexual abuse or rape in your community? 1-Medical treatment 2-Legal services 3-Protection shelters 4-Psychosocial support 5-Referral to other organizations -Multiple response 18) Have you or any member of your family accessed sexual or gender violence response services in the last 6 months? 1-Yes 2-No -Single response 19) Would you report a case of sexual or gender violence if any member of your household encountered it? 1-Yes 2-No -Single response -If Yes move to 20, If No move to 21 20) Where would you report the sexual assault, exploitation or rape? 1-Camp leader 2-Clan/ommunity leader 3-Local police men 4-Religious leaders 5-Relative or family member 6-Humanitarian workers (NGOs and CBOs) 7-Health care workers 8-Social workers 9-Other (Specify)__________ Multiple response 21) Does your household have any child/children below 15 years? 1-Yes 2-No -Single response If Yes move to 22, If No move to 25 22) Are you aware of a children’s safe space in this community? [safe spaces are places where kids spend time and safe from neglect, physical, sexual or emotional harm or abuse] 1-Yes 2-No -Single response 23) Do your children have access to the safe space available in this community? 1-Yes 2-No -Single response 24) Would you say that that there was an improvement in the safety and welfare of your child when he/she accessed a child-friendly space? 1-Improved 2-Remained the same 3-Deteroriated -Single response Engagement of beneficiaries/accountability to beneficiaries 25) In your opinion, were your rights respected/upheld in receiving the nutrition services? 1-Fully 2-Partially 3-Not at all Single mandatory response 26) Were you aware of the hotline for providing feedback to program staff regarding areas of concern, satisfaction and dissatisfaction over the nutrition services provided to you? 1-Yes 2-No Single mandatory response 27) Did you at any point provide feedback or complains regarding nutrition services received under this program? 1-Yes -No Single mandatory response 28) What mechanism would you feel safe and confident to provide 1-Hot line 2-Community/camp leaders Multiple response Page 70 of 97 feedback over the services you receive? 3-Program staff 4-Messages 5-Feeback boxes 6-Other (specify)______________ 29) Interview end time: _____:_____ To be picked automatically by tablet/laptop for all respondents Thank you very much for your time and your valuable information, it will help SOS, Trócaire and Save Somali Women and Children improve protection services delivery in future programs. Annex 7: Water, Sanitation and Hygiene (WASH) -Quantitative Household Survey Tool CATHOLIC RELIEF SERVICES SOMALIA: THE SHARPEN II PROGRAM END TERM EVALUATION SURVEY IN SOUTH CENTRAL SOMALIA End Term Evaluation Household Nutrition Survey Questions to be answered by the Head of the Household or the senior most female in the household phone calls. Informed consent: Hello, my name is ________________ and I am making a phone call on behalf of SOS Somalia/Trócaire Somalia/Save Somali Women and Children (read the applicable partner organization) to conduct an assessment regarding Water, Hygiene and Sanitation (WASH) services they have been offering to persons like yourself in your community over the last two years. As we Page 71 of 97 embark on this this assessment, I would like to talk to a few people like yourself who have benefited from the program to understand your experiences from the services you received. Benefits of the study: The information you provide will be adopted for the improvement of similar services in future. Risks of the study: There are no direct or indirect risks in your participation in this interview. Confidentiality: Your identity and responses shall be treated with confidentiality and all the information you give will only be used for the purposes of this study. The information you share today is confidential. We will not share your information with anyone else without your permission. Consent to participate in the study: You may choose not to participate in the study since participation in the study is voluntary. Would you be willing to allow me to continue with the interview? [1] No (If No: Thank him/her and end the interview) [2] Yes If yes: With your permission, I hope I can now start the discussion. The interview should take no more than 15 minutes. Answer the questions as accurately as possible. But before I begin, I am ready to answer any questions that you may have about this interview. S/No Question Options/Responses Coding instructions Identifiers and social demographic markers 1) Questionnaire number ___ ___ ___ -A maximum of three digits -Mandatory 2) Interviewers code ____ -Numerical values 1 t0 8 --Mandatory 3) Date of the interview __ __ /__ __/ __ __ __ __ -To be picked automatically by the laptop or tablet --Mandatory 4) Start time ____ ____ ___ ___ -To be picked automatically by the laptop or tablet --Mandatory 5) Location of the respondent 1) Mogadishu 2) Afgooye Corridor 3) Cadaado 4) Baidoa 5) Luuq 6) Dollow 7) Hudur 8) Burdhubo -Single response -Mandatory 6) What is your age in years as at last birthday? ___ ___ __ -A maximum of three digits -Mandatory 7) What is your gender? 1-Female 2-Male -Single response -Mandatory 8) What is your position in the household? 1-Head of the household (male or female) 2-Senior most female in the household -Single response -Mandatory 9) What is the highest level of Education you have achieved? A-Low Education Bracket 1) None 2) Informal education 3) Religious education 4) Primary School Incomplete -One response only -Mandatory Page 72 of 97 B-Moderate Education Bracket 5) Primary school complete 6) Secondary School incomplete 7) Secondary School Complete C-High Education Bracket 8) TVET (technical vocation education) 9) Tertiary/college 10) Other (Specify)________ 10) What is your marital status? 1) Single 2) Married 3) Separated 4) Divorced 5) Widowed/widower 6) Other (specify)________ -One response only -Mandatory 11) How would you describe your place of residence? 1-IDP camp 2-Host community -Single response -Mandatory 12) Are you currently living with a disability? 1-Yes 2-No -Single response -Mandatory 13) If yes, what type of disability? 1-Vision Impairment. 2-Deaf or hard of hearing. 3-Mental health conditions. 4-Intellectual disability. 5-Acquired brain injury. 6-Physical disability. 7-Other (specify)____________ Multiple response Relevance 14) Over the last two years what have been your households’ greatest needs? [Do not read responses: Tick all mentioned answers] 1-Health 2-Nutrition 3-Food 4-Water 5-Hygiene and sanitation 6-Nonfood items 7-Protection and security 8-Shelter 9-Identification documents 10-Animal health services 11-Livelihood/Source of income 12-Children play grounds and safe spaces 13-Other (Specify):_______ -Multiple response -Mandatory Effectiveness 15) What are critical moments for washing your hands? [Do nor read choices; tick all responses given] 1-After defecation/visiting the toilet 2-After cleaning a child's bottom or changing nappies 3-Before feeding a child 4-Before eating 5-Before touching and preparing food -Multiple response -Mandatory 16) What sanitation facility does your household use? 1-Pit latrine 2-Toilet (composting and hanging types) 3-Bucket 4-Bush/Field/open defacation 5-Other (specify)_________ -Multiple response -Mandatory 17) Do you share this facility with others who are not members of your household? 1-Yes 2-No -Single response -Mandatory Page 73 of 97 18) How many households in total use this toilet facility, including your own household? ___ ____ _________ _________ -Numerical value -Mandatory 19) Has your source of water for drinking, cooking, and hygiene from improved over the last one year? 1-Yes, improved 2-No, remained the same 3-No, worsened -Single response 20) What is/are your household’s main source(s) of water for drinking, cooking, and hygiene? A-Improved source 1-Piped water on premises (inside dwelling, plot or yard) 2-Other improved sources (public taps, standpipes, tube wells, boreholes, protected springs or rain water) B-Unimproved source 3-Unimproved sources (unprotected dug well, unprotected spring, carts, trucks, bottled water) 4-Surface water (river, dam, lake, pond, stream, canal or irrigation channels) Multiple response 21) Are there times when water is unavailable from this source/ these sources? 1-Yes 2-No -Single response -If yes move to 22 -If No, move to 23 22) If yes, where do you collect water for drinking, cooking, and hygiene when it is unavailable from these sources? A-Improved source 1-Piped water on premises (inside dwelling, plot or yard) 2-Other improved sources (public taps, standpipes, protected tube wells, protected boreholes, protected springs or rain water) B-Unimproved source 3-Unimproved sources (unprotected dug well, unprotected spring, carts, trucks, bottled water) 4-Surface water (river, dam, lake, pond, stream, canal or irrigation channels) Multiple resp0nse 23) How many liters of water is your household able to access/collect from the water point in a day? ________________________ -Numerical value -Mandatory 24) Has your household received any water, hygiene and sanitation items/kits over the last one year? 1-Yes 2-No -Single response -Mandatory 25) How many times has your household received water, hygiene and sanitation items/kits over the last one years? ___ ___ ____ -Numerical value -Mandatory 26) Are you satisfied with the contents of the water, hygiene and sanitation items/kits provided to you? 1-Satisfied 2-Not satisfied -Single response -Mandatory 27) Are you satisfied with the quantity of these water, hygiene and sanitation items/kits issued? 1-Satisfied 2-Not satisfied -Single response -Mandatory Page 74 of 97 28) Are you satisfied with the quality of these water, hygiene and sanitation items/kits issued? 1-Satisfied 2-Not satisfied -Single response -Mandatory Engagement of beneficiaries/Accountability to beneficiaries 29) In your opinion, were your rights respected/upheld in receiving the Water, Hygiene and Sanitation services and kits provided to you? 1-Fully 2-Partially 3-Not at all Single mandatory response 30) Were you aware of the hotline for providing feedback to program staff regarding areas of concern, satisfaction and dissatisfaction over Water, Hygiene and Sanitation services and kits provided to you? 1-Yes 2-No Single mandatory response 31) Did you at any point provide feedback or complains regarding Water, Hygiene and Sanitation services and kits provided to you? 1-Yes -No Single mandatory response 32) What mechanism would you feel safe and confident to provide feedback over the services you receive? 1-Hot line 2-Community/camp leaders 3-Program staff 4-Messages 5-Feeback boxes 6-Other (specify)______________ Multiple response 33) Interview end time: _____:_____ To be picked automatically by tablet/laptop for all respondents Thank you very much for your time and your valuable information, it will help SOS, Trócaire and Save Somali Women and Children improve Water, Hygiene and Sanitation (WASH) services delivery in future programs. Annex 8: Focus Group Discussion Guide for Program Beneficiaries CATHOLIC RELIEF SERVICES SOMALIA: SHARPEN 11 PROGRAM FINAL EVALUATION Moderator’s Name: __________________ Recorder’s Name: ____________________ Date of interview: ___________________ Region: Village/IDP Camp: ___________________ Number of Participants: ______ ______ Ages of Participants: Page 75 of 97 Welcome and thank you for volunteering to take part in this FGD. You have been asked to participate as your point of view is important. We realize you are busy and I appreciate your time. Introduction: We are (1) __________ and (2) ___________ and we are here on behalf of Catholic Relief Services, SOS Children’s Villages International, Save Somali Women and Children (SSWC) and Trócaire to conduct to carry out an interview; this interview is designed to assess your current thoughts and opinions on the services you have received under the SHARPEN II program. Your responses will be very useful in helping us to plan support activities for this community. This FGD will take no more than 30 minutes, and I will be taking notes on what we are discussing Anonymity: Despite the discussion being noted down, I would like to assure you that the discussion will be anonymous. The notes will contain no information that would allow individual subjects to be linked to specific statements. Please answer as accurately and truthfully as possible. If there are any questions or discussions that you do not wish to answer or participate in, you do not have to do so; however please try to answer and be as involved as possible. Ground rules • The most important rule is that only one person speaks at a time. There may be a temptation to jump in when someone is talking but please wait until they have finished. • There are no right or wrong answers • You do not have to speak in any particular order • When you do have something to say, please do so. There are many of you in the group and it is important that I obtain the views of each of you • You do not have to agree with the views of other people in the group • Does anyone have any questions? (Answers). OK, let’s begin Guiding Questions 1) What the most common needs in this community? 2) What assistance has been received in your households through the SHARPEN II program? 3) Are you satisfied with how beneficiaries were selected for this program? 4) What needs has the program addressed and which ones has it not addressed? 5) Were there any delays in receiving any program interventions? 6) How have the benefits from this program changed your household? [Food, nutrition, health, WASH, protection etc.] 7) Were you involved on decision making under the program? [How?] 8) Are you satisfied with your level of involvement in the program? [How would you want to be involved in decisions making regarding the program activities] 9) What challenges do you still face in accessing health services? 10) What challenges do you still face in accessing nutrition services? 11) What challenges do you still face in accessing water and sanitation services? Page 76 of 97 12) What protection challenges children and women still face in this region? 13) What protection challenges do children still face in this region? 14) What recommendations would you make to improve similar programs in this area in future? Thank you for your time and your useful responses; they will be helpful in improving the services we offer to you in future Annex 9: Key Informant Interview Guide for Program Staff and Partners CATHOLIC RELIEF SERVICES SOMALIA: THE SHARPEN II PROGRAM END TERM EVALUATION IN SOUTH CENTRAL SOMALIA End Term Evaluation Key Informant Interview Guide for Program Staff and Representatives of Partner Organizations Welcome and thank you for volunteering to take part in this KII. You have been asked to participate as you are a key stakeholder in the SHARPEN program implemented by CRS Somalia, SOS Children’s Villages Somalia, Trócaire Somalia and Save Somali Women and Children. This interview is designed to assess your current thoughts and opinions on the program. This end of program evaluation intends to assess the relevance, effectiveness of the project design, and achievements of its results and objectives. It will also assess the efficiency of the implementation process. In addition, it will determine lessons learned to benefit future programming. Page 77 of 97 This KII will take no more than 20 minutes, and I will be taking notes on what we are discussing. Anonymity: Despite the discussion being noted down, I would like to assure you that the discussion will be anonymous. The notes will contain no information that would allow individual subjects to be linked to specific statements. Please answer as accurately and truthfully as possible. Interviewer’s name: Respondents name: Gender: Position: Organization: Date of interview: Relevance/ Appropriateness 1. How was the scope of the program arrived at? [Nutrition, Health, WASH and Protection] 2. How does the goal (aim) of the project relate to, or fit into, the needs of the Somali community/beneficiaries 3. How does the program fit into your organizations scope of work and mandate?? 4. How well does the program align with government and agency priorities? 5. How does the program address global needs and priorities? Effectiveness 6. To what extent have the goals of this program been met? [Nutrition, Health, WASH and Protection] Which goals have been met, which ones have not been met? 7. What factors facilitated the success? 8. Which of these changes are directly attributed to the project? 9. Are there any unforeseen/unintended outcomes? Which ones? Are they positive or negative? How were the unforeseen consequences addressed? 10. For health care, WASH and protection workers:54F 55 Responde nt Indicator Program site Mogadish u Afgooye Corrido r Baidoa Cadaad o Luuq Dollow WASH staff What is the number of people directly utilizing improved water services provided with OFDA funding? Male: Female: Total: Male: Female: Total: Male: Female: Total: Male: Female: Total: Male: Female: Total: Male: Female: Total: WASH staff What is the average liters/person/day collected from all sources for drinking, cooking, and hygiene WASH staff What is the estimated safe water supplied per beneficiary in liters/person/day WASH staff What is the number of people directly utilizing improved sanitation services provided with OFDA funding Male: Female: Total: Male: Female: Total: Male: Female: Total: Male: Female: Total: Male: Female: Total: Male: Female: Total: Health care workers/m What percent of hand washing stations built or 55KIIs respondents filled this table on quantitative data for WASH and protection. Page 78 of 97 anagers/hea ds of health facilities rehabilitated in health facilities that are functional Protection workers What is the average number of SGBV cases reported every month Protection counsellors Percentage of people reporting improvements in their feelings of well-being and ability to cope at the end of the program Total:____ _ Male: ____ Female:___ ___ <5 years:____ _ 5-9 years:____ 10-14 years:____ __ 15-19 years:____ _ 20-49 years:____ 50+ years:____ __ Total:___ __ Male: ____ Female:_ _____ <5 years:___ __ 5-9 years:___ _ 10-14 years:___ ___ 15-19 years:___ __ 20-49 years:___ _ 50+ years:___ ___ Total:__ ___ Male: ____ Female:_ _____ <5 years:__ ___ 5-9 years:__ __ 10-14 years:__ ____ 15-19 years:__ ___ 20-49 years:__ __ 50+ years:__ ____ Total:___ __ Male: ____ Female:_ _____ <5 years:___ __ 5-9 years:___ _ 10-14 years:___ ___ 15-19 years:___ __ 20-49 years:___ _ 50+ years:___ ___ Total:_ ____ Male: ____ Female: _____ _ <5 years:_ ____ 5-9 years:_ ___ 10-14 years:_ _____ 15-19 years:_ ____ 20-49 years:_ ___ 50+ years:_ _____ Total:__ ___ Male: ____ Female:_ _____ <5 years:__ ___ 5-9 years:__ __ 10-14 years:__ ____ 15-19 years:__ ___ 20-49 years:__ __ 50+ years:__ ____ Program Efficiency 11. What informed the budget distribution across various sectors? 12. Were finances adequate for the implementation of all program activities? [Why? Why not?] 13. What cost cutting measures were adopted under this program? 14. Were staff to implement the program adequate? [Why? Why not?] 15. What platform was used for cash transfers? [Why was this platform chosen? Did it have any challenges, if yes name them? 16. Were there any delays in the implementation of any program activity? [Which ones? What caused the delays? How were the delays mitigated?] 17. Could a different approach in the program have produced better results for a lower cost? [Name them and justify] 18. Who were the partners under the program how did they contribute to the achievement or non-achievement of results? Conclusion, Best Practices and Lessons Learnt 19. What are some of the success stories that can be shared on the positive impact or influence of the project’s services on beneficiaries? 20. What would you say were the strengths of the program? What about the weaknesses? 21. What international best practices/innovations were incorporated or experienced in this program? 22. What are some of the lessons learnt from this project, and how have they been used to strengthen the project/programme? [Strong justification needed] Page 79 of 97 23. What would you recommend to improve in future design and implementation for similar projects in future? We have come to the end of our interview, thank you so much for your time, the information you have given us will be used to improve the similar program activities in future. Annex 10: End Term Evaluation Data Quality Assurance Plan The following data quality assurance measures were be put in place before, during and after data collection: Quantitative Data Quality Control Measures Before Data Collection • Coding of the quantitative data collection tool on KoBoCollect to ensure mandatory filling of all questions before proceeding to subsequent questions. • Training of enumerators and supervisors. • Pretesting of data collection tools. • Pilot testing of data collection procedures. • Provision of common instructions on common errors. Page 80 of 97 • Defining the minimum duration for completing a quantitative interview. During the Data Collection Exercise • Over the shoulder supportive supervision of enumerators • Pre-filled, pre-loaded or auto-complete list e.g. for the clusters, gender etc. • Skip/piping logic- questions that are not applicable are not displayed. • Mandatory questions- these questions cannot be left blank or skipped. • Sequential, single question display so that an enumerator focuses on 1 question at a time. • Input masks- control the number and types of characters that can be entered. • Validation rules ensuring keying in ‘valid’ responses, e.g. age limits, pregnant males. • Regularly tracking the errors that field staffs make in their SMS formats or answer values. • Answer confirmation: Prompting to confirm the answer that has been answered. • Error feedback- if answers are incorrect, we will provide details of the error type. • Post-completion review after completion before sending data to server. • Collection of GPS coordinates of the location of interviewers (the office in Mogadishu). • Collection of start and completion time to analyze time taken in each interview. • Record interview where consent will be granted by respondents. After Field Work • Post completion review of the data set. • Analysis of survey completeness/errors. • Deletion of incomplete errors and questionnaires for respondents who declined interviews. • Analysis of time taken per questionnaire-where short time below the standard set time for each questionnaire is encountered, that questionnaire will be discarded. • Recording syntax steps for data manipulation, labeling and analysis. • Triangulation of findings-using findings from various data collection methods and from the various interviewers. Qualitative Data Quality Control Measures • Conceptualization of research questions was guided by the TOR and the OECD/DAC criteria. • Inclusion and exclusion criteria-Only scientifically published documents and official partner organizations related literature and program data were included in the evaluation report. • Data acquisition-Combining desk review and KIIs as a strategy for increasing the validity of results will be done. • Selection of respondents-Key knowledgeable respondents in the program and partner organizations were targeted with KIIs. • Data analysis-To avoid subjective elements, triangulation of findings from different sources will be done during data analysis for every study theme. • Validity and sorting-Categorization, classification, sorting and labeling was used to build themes around each research question. Page 81 of 97 Annex 11: Recruitment and Training of Research Assistants for the End Term Evaluation The HSED Group Africa recruited 30 enumerators (15 male and 15 female) from its pool of research assistants who have been conducting mobile data collection in South Central Somalia. The qualifications for selection were as follows: • Possession a minimum of a university degree; • Fluency in spoken and written English and Somali language; • Prior experience working the HSED Group Africa and CRS programs assessments; and • Experience on mobile data collection and recording of data on the Kobo Toolbox. A two days training for the enumerators was conducted across the program sites and it focused on: • Covid-19 safety precautions. • CRS beneficiaries’ safeguarding protocols. • USAID evaluation policy. • CRS – MEAL policies and procedures. • Objectives of the program end-term evaluation. • Roles and responsibilities of the enumerators. • Confidentiality protocols. • Interviewing techniques and methodology. • Communication skills during interviews. • Mobile phones use in data collection. • Orientation with the evaluation tools including skip patterns. • Pre-testing of the questionnaire (enumerators to interview each other). • Pilot testing of procedures- sampling techniques/respondent selection procedures. • Content and use of the questionnaires, survey forms and materials. • Work plan and targeted respondents per enumerators every day. • Selection of respondents and handling non-response in telephone interviews. Annex 12: Inclusion and Exclusion Criteria for the End Term Evaluation Interview Respondents KIIs and FGDs targeted only knowledgeable respondents with first hand exposure to the program hence possession of valuable knowledge on the program implementation. The selection of household beneficiaries will be guided by the following inclusion and exclusion criterion: Inclusion Criteria • Households that had benefited from services offered by the three implementing partners. • Head of households or senior most females in the households. • Provision of verbal consent to participate in the end-term evaluation survey. Page 82 of 97 Exclusion Criteria • Households that had not benefitted from the services offered by the three implementing partners. • Households where respondents were not heads or senior most females. • Households where heads or senior most females did not provide verbal consent to participate in the end-term evaluation survey interviews. Annex 13: Terms of Reference Evaluation Terms of Reference: SHARPEN II Final Evaluation Catholic Relief Services, Somalia Sept 08, 2021 I. INTRODUCTION I.A. Introduction Catholic Relief Services Somalia is seeking to engage an external consultant to conduct a final evaluation for SHARPEN II program. The aim of the evaluation is to determine the overall merit and value of the response through using meta-questions to assess the project’s relevance and effectiveness as well as to provide evidence-based recommendations that will inform future programming. Non-experimental evaluation design will be used to measure the changes brought about by the project intervention to the target beneficiaries in Mogadishu, Afgooye Corridor, Baidoa, Xudur, Cadaado, Luuq, and Dollow. Probability sampling will be used for the household quantitative survey to ensure the subjects of the population get an equal opportunity to be selected as respondents. A stratified two stage cluster sampling approach will be used to select study participants. For the qualitative data collection, a purposive sampling method will be used to select study respondents, based on the role they played in the emergency response. Quantitative data collection will sample beneficiary households in project target locations in Mogadishu, Afgooye Corridor, Baidoa, Xudur, Cadaado, Luuq, and Dollow that have benefited from the project interventions and will be designed to collect data from household heads and/or their spouses on demographic, socio-economic characteristics of the households and the performance of the selected indicators. The data will be collected using structured questionnaire and rely on a mobile data collection application via a remote modality. Qualitative data collection will target the key stakeholders who helped contribute to project design and implementation. This information will be used to supplement and complement the quantitative data collected from the household interviews to provide an in-depth knowledge on how the intervention has been able to support conflict and drought affected IDPs in the project target locations. The targeted respondents will include the health centers staff, village committee members, representatives of the community and the other project staff involved in the implementation of the response. The qualitative data collection will be done through Key Informant Interviews (KIIs, field observation, beneficiaries’ interviews such as water management committees, camp leaders, and community leaders The findings of the report would be used by CRS and it’s implementing partners to determine achievement against performance indicators and to draw lessons to inform future intervention designs and similar programming. I.B. Background: CRS and Implementing Partners Catholic Relief Services (CRS) - An international non-governmental organization supporting relief and development work in over 99 countries around the world. CRS programs assist persons based on need, regardless of creed, ethnicity, or nationality, and works through local church and non-church Page 83 of 97 partners to implement programs. CRS carries out the commitment of the Bishops of the United States to assist the poor and vulnerable overseas. CRS currently addresses food, water, hygiene, health, nutrition, and protection needs of vulnerable Somalis. CRS has been working closely with and channeling resources and support to local organizations inside Somalia since the 1990s. CRS has been implementing activities in Mogadishu since August 2011, in Baidoa since April 2012, and in Gedo region since 2014 with OFDA, FFP, and other private and external donor funding. SOS Children’s Villages International - An independent, non-governmental international development organization which has been working to meet the needs and protect the interests and rights of children since 1949. SOS in Somalia began in 1983; and a property provided by the government was chosen as the site for the first SOS Children's Village and its adjoining kindergarten. The SOS School was later converted into an emergency clinic during the war, and the mother and child clinic became part of emergency relief programming. Today it remains the only functioning maternity ward and gynecological care facility in the country. More recently, the SOS Vocational Training Center was established, which offers training courses for nurses and midwives. SOS has worked with CRS since 2011 on emergency programs to provide livelihood recovery, basic health, and nutrition services to vulnerable IDPs and host communities. Save Somali Women and Children (SSWC) - SSWC was founded in 1992 by a group of Somali female intellectuals from a cross section of the community and has a longstanding history of promoting women’s rights and advocacy. SSWC has worked in the areas of protection, WASH, and livelihoods, and prioritizes supporting grassroots economic projects for women, enhancing their capacity for advocacy on the issue of Female Genital Mutilation (FGM), providing training to Non-Governmental Organizations (NGOs) and Community Based Organizations (CBOs) on women’s rights, and raising awareness on the conditions of women and girls in Somalia in Baidoa, Mogadishu, Afgooye, Caadado, Dusamareeb and Kismayu. Trocaire – Trocaire is an international non-governmental organization that works with local partners to support communities in over 20 developing countries with a focus on food and resource rights, women's empowerment and humanitarian response. Trocaire has been operational in Somalia since 1992 and is one of the few organizations that continued to provide life-saving interventions without pause throughout the chaotic decades of civil war. Trocaire employs a unique, community led approach through all its work that has ensured access and safety for its staff and operations. I.D. Background: Project Goal and Objectives Following on the success of SHARPEN, CRS proposed SHARPEN 2 which provides an integrated package of basic life-saving services to crisis-affected Somalis in Mogadishu, Afgooye Corridor, Baidoa, Xudur, Cadaado, Luuq, Dollow, Garbaharey and Burdhubo. This package includes access to primary health care through static and mobile facilities; a complete Integrated Management of Acute Malnutrition (IMAM) program; prevention of Gender-Based Violence (GBV) and comprehensive services for survivors, including children who are victims of GBV; child friendly spaces that promote protection, health and nutrition of children; integrated hygiene, health and nutrition sensitization; critical water and sanitation (WASH) infrastructure, and distribution of hygiene supplies coupled with hygiene messaging. CRS oversees and coordinates three implementing partners: Save Somalia Women and Children (SSWC), SOS Children’s Villages Somalia (SOS) and Trócaire Somalia. The project works to increase access to Health, WASH, Nutrition, and Protection and WASH services to reach the following sector objectives: 1. Target populations access comprehensive primary care and have reduced morbidity. 2. Malnutrition levels in young children (under 5) and Pregnant/ Lactating Women (PLWs) are decreased. 3. The risk of GBV among vulnerable groups (including children) is reduced and survivors of GBV access comprehensive services. Page 84 of 97 4. Vulnerable populations access safe water, improved sanitation and hygiene practices, and behaviors that decrease malnutrition. Specific activities and interventions were chosen for each catchment area based on the strengths and capacities of each organization, the needs of the targeted communities, complementarity with funding received from other donors, lessons learned through consultation with participants of past and ongoing projects and the feasibility of interventions. Construction and rehabilitation of Health and WASH infrastructure provides facilities that meet SPHERE and national guidelines. This project implemented from August 1, 2020, to September 30, 2021, continues essential service provision funded by OFDA in Mogadishu, Afgooye Corridor, Baidoa, Cadaado, Dollow, and Luuq, while expanding the successful, integrated approach to Xudur, Garbaharey and Burdhubo. The project goal, sectors, objectives by sector, and activities are presented in the table below. Table 1.2. Objective and Approach per Sector Sector Health 164,441 Nutrition 87,097 Protection 44,100 WASH 69,686 Objective Target populations access comprehensive primary care and have reduced morbidity. Malnutrition levels in young children (<5) and pregnant/lactating women (PLWs) are decreased. The risk of GBV against vulnerable groups (including children) is reduced and survivors of GBV access comprehensive services. Vulnerable populations access clean water and have improved hygiene. Approach CRS and its partners support static and mobile health clinics, who use community health workers to extend service provision to hard-to-reach IDP populations who cannot easily access existing facilities. CRS and its partners use the Basic Nutrition Service Package for Somalia protocol (as recommended by the Somalia Nutrition Cluster). CRS uses a combination of strategies to decrease malnutrition, including household level screening, treatment at health facilities, IYCF and mother-to￾mother support groups and targeted nutrition messaging. CRS and its partners work within prevention and response to gender-based violence and psychosocial support services to achieve this objective. GBV survivors access comprehensive services including medical, legal and psycho-social. Furthermore, the project works to raise GBV awareness and mitigation strategies among target communities. CRS and its partners undertake extensive hygiene promotion campaigns, integrated with nutrition messaging at facilities and HH￾level through leveraging partner networks of CHWs and CNWs to deliver integrated hygiene messaging. Furthermore, the project supports the construction of latrines, hand washing stations and shallow wells to benefit both IDP and host community populations. Page 85 of 97 II. PURPOSE OF THE EVALUATION II.A. Purpose of the Evaluation The purpose of the evaluation is to assess whether the response achieved the desired outcomes and produced evidence-based recommendations to inform future programming. The evaluation will seek to: • Determine achievement against performance targets of select indicators. • Identify to what extent were beneficiaries actively consulted and engaged in the project. • Identify program strategies and structures which contributed to or impeded project impact • Draw lessons from the project and results achieved to inform future similar programming. II.B. Key Audiences and Uses Table 2.1. Summary of Stakeholder Data Needs and Evaluation Role STAKEHOL DERS STAKEHOLDER EVALUATION DATA NEEDS AND USE STAKEHOLDERS’ ROLE IN THE EVALUATION JUSTIFICAT ION FOR STAKEHOL DER ROLE USAID • To establish the impact of the BHA project on the target beneficiaries. • Suggestions/recommendatio ns with the potential to shape future programming. • Provision of funds to undertake the evaluation. • Review and approval of the evaluation TOR and report. This is the donor for the project. CRS • To establish the impact of OFDA on the target beneficiaries. • Suggestions/recommendatio ns with the potential to shape future programming. • To assess and evaluate the relevance, efficiency, and effectiveness of the response interventions in terms of its implementation approach and strategies. • Development of the evaluation TOR. • Recruitment of the consultant. • Reviewing and approval of the evaluation design and tools upon agreement with the consultant. • Provision of relevant project background materials. • Supervision of the consultant. • Review and approval of the final the evaluation report. • Processing payments for the consultant(s) upon receiving of the final report. • Make travel, accommodation and per diem arrangements for consultant and team. • Ensure smooth flow of consultancy engagement processes including contractual obligations. • Share the final evaluation report with all key stakeholders, including key project staff, partners, donor representative and government. This is the project prime. Local Implementi ng Partners: SOS, SSWC and Trocaire • To measure the outcome and impact of the intervention. • Suggestions/recommendatio ns with the potential to shape future programming. • Assist the consultant in the implementation of the evaluation methodology as appropriate i.e., recruitment of research assistants, participation in sampling, mobilizing sampled communities, scheduling meetings, interviews, etc. These are the project sub￾grantees. Page 86 of 97 • Timely procurement of logistics as agreed with the consultant. District Health Boards (DHBs) • Status of health, nutrition, and WASH indicators in their constituencies • Receive key indicator results through regular consultation channels with Local Implementing Partners DHBs are local oversight mechanisms for health care service provision III. EVALUATION QUESTIONS / OBJECTIVES The evaluation will use meta-questions around relevance, effectiveness, and efficiency of project interventions, in assessing whether the project met its objectives. These evaluation meta-questions are provided in table 3.2 below. Below (table 3.1) is the list of required indicators, baseline benchmarks and end of targets to be measured by the evaluation. Table 3.1. Performance Indicators OBJECTIVE STATEMENT INDICATORS with required disaggregates BASELINE VALUE 95% confidence interval TARGET PROPOSED METHOD Target populations access comprehensive primary care and have reduced morbidity Number and percentage of community members who can recall target health education message, disaggregated by: N/A 69.4% 63.17% to 75.14% 123,353; 85% Survey response data Malnutrition levels in young children (<5) and pregnant/lactating women (PLWs) are decreased Proportion of infants 0-5 months of age who are fed exclusively with breast milk, disaggregated by: Male and Female 68.1% 60.6% to 74.9% 75% Survey response data Proportion of children 6-23 months of age who receives foods from 4 or more food groups, disaggregated by: Male and Female 47.1% 39.5% to 54.8% 75% Survey response data Page 87 of 97 Vulnerable populations access clean water and have improved hygiene Percent of people targeted by the hygiene promotion program who know at least three (3) of the five (5) critical times to wash hands, disaggregated by: Male & Female 78.7% 72.1% to 84.4% 80% Survey response data Number of people directly utilizing improved sanitation services provided with OFDA funding, disaggregated by: Male & Female 5,600 N/A 8,000 Household Survey Average number of users per functioning toilet, disaggregated by: N/A 22.1 16.3 to 28.8 30 Household Survey Number of people directly utilizing improved water services provided with OFDA funding, disaggregated by: Male & Female 11,076 N/A 14,000 Survey response data Average liters/person/day collected from all sources for drinking, cooking, and hygiene, disaggregated by: N/A 15 10.2 to 21.0 15 Survey response data Estimated safe water supplied per participant in liters/person/day 15 10.2 to 21.0 20 Survey response data Percent of hand washing stations built or rehabilitated in health facilities that are functional, 100.0% 77.7% to 99.8% 95% KII IP Page 88 of 97 disaggregated by: N/A Percent of households targeted by WASH program that are collecting all water for drinking, cooking, and hygiene from improved water sources, disaggregated by: Male & Female 21.5% 15.8% to 28.2% 70% Household Survey Percent of households reporting satisfaction with the contents of the WASH NFIs received through direct distribution (i.e. kits) or voucher, disaggregated by: Male & Female 93.4% 88.6% to 96.6% 96% Household Survey Percent of households reporting satisfaction with the quantity of WASH NFIs received through direct distribution (i.e., kits), vouchers, or cash, disaggregated by: Male & Female 82.4% 75.9% to 87.8% 90% Household Survey Percent of households reporting satisfaction with the quality of WASH NFIs received through direct distribution (i.e., kits), vouchers, or cash, disaggregated by: Male & Female 93.9% 89.2% to 97.0% 95% Household Survey Page 89 of 97 All performance indicators must be reported with appropriate disaggregation’s as specified in USAID/OFDA’s Performance Indicator Reference Sheets. Table 3.2. Summary of Evaluation Questions COMPONENTS OF THE PROGRAM WE WOULD LIKE TO LEARN MORE ABOUT QUESTIONS WE HAVE THAT WE WOULD LIKE ANSWERED EXISTING DATA TO HELP ANALYZE THIS QUESTION FURTHER DATA NEEDED WHO SHOULD BE INVOLVED? Relevance 1. To what degree did the project meet the needs of target beneficiary populations? • Project proposal • Project monthly and quarterly reports • Feedback mechanism data • Beneficiary and other stakeholder perspectives on the relevance of the project in meeting their needs. CRS, SOS, SSWC, Trocaire, beneficiaries and other relevant stakeholders Effectiveness 2. To what extent were the sector specific objectives of the project achieved? 3. To what extent were beneficiaries actively consulted and engaged in the project especially in their ability to provide feedback through partner’s accountability mechanisms? • Project proposal • Project monthly and quarterly reports • Feedback mechanism data, including design documentation of mechanisms • Previous donor submitted reports • Partner indicator tracking tables • CRS partner accountability assessment documents • Population level data on required performance indicators (detailed in tabled 4.1 below) • Feedback from beneficiaries regarding awareness, accessibility and use of accountability mechanisms • Beneficiary feedback regarding project outputs, outcomes and community consultation around the project implementation • Analysis of project performance against indicator targets and changes from the baseline CRS, SOS, SSWC, Trocaire, beneficiaries and other relevant stakeholders Efficiency 4. To what extent did the implementation process, including delivery options and Review of Program financial reports, Detail implementation Detail Implementation plans SSWC, Trocaire, SOS Page 90 of 97 models, ensure efficient use of value for money; including: management structures, partner roles and coordination, humanitarian coordination between other actors? plans, Budget comparison report IV. EVALUATION METHODOLOGY IV.A. Evaluation Design and Approach The final evaluation will employ a non-experimental design for simple pre-post comparison of results using a mixed-methods approach involving both quantitative and qualitative data. Data collection will involve a quantitative beneficiary household survey; document reviews, including routine monitoring data and project reports; beneficiary and stakeholder interviews. The consultant will use a comparative analysis approach to report on project achievements for selected indicator values. Quantitative HH survey will be conducted through structured questionnaire, with relevant and appropriate questions, that will generate quantitative information that will be captured numerically and can produce summary statistics such as frequency distributions, means, medians, ranges and other measures of variation which describe the beneficiary in an aggregate way. The quantitative HH survey will be complemented by qualitative methods in the form of KIIs. Secondary data from routine project MEAL system and the previous final evaluation and needs assessments conducted in the project should also be used to inform the findings, conclusions, and recommendations in this evaluation. The survey will generate end-line data for the outcome indicators of the OFDA project, which could be used as baseline data for follow-on program where appropriate. In this design, the researcher will not control, manipulate, or alter the predictor variables or project beneficiaries, but will instead rely on interpretation, observation and interactions to conclude, through correlations. IV.B. Sources of Data and Data Collection Methods To answer the key evaluation questions associated with this assignment, both qualitative and quantitative data will be elicited by the consultant, through primary and secondary sources. First, the evaluation will assess and draw from secondary data, which will be obtained through review of key project documents: quarterly assessments, monthly reports, monitoring data and project proposal. The evaluation will also generate primary data. The consultant will design and conduct a household survey administered in-person to randomly selected beneficiaries in the target locations, to generate primary quantitative data. To generate primary qualitative data, the consultant will conduct key informant interviews, and any other methods considered appropriate, such as case studies and most significant change stories. Stakeholder interviews, field observations will also be conducted. Key informant interviews will be purposively selected from project beneficiary and project stakeholder populations. The consultant will conduct key informant interview with SOS, SSWC, Trocaire, and CRS Somalia staff, as well as other key project stakeholders. If possible, all qualitative data will be recorded during the interview process and translated from Somali to English, as needed. The consultant should include all raw qualitative key informant interview data, as appendices to the Final Evaluation Report, and categorize, summarize, interpret, and highlight key findings and conclusions from all this data in the Final Report itself. All the qualitative data collected through the above methods will be used to triangulate, explain, and create context for the trends and data collected through the quantitative methods. In the Final Evaluation Report, the analysis of all data (primary, secondary, quantitative, qualitative) and findings Page 91 of 97 should be organized around and presented to answer the key Evaluation Questions as summarized in Table 3.2. To abide by the “Do No Harm” principle especially in the context of COVID-19, the lead consultant must always ensure protection and safety of partner staff, data collection team, as well as members of local communities, and ensure all mandatory measures (based on both national and global guidance) for protection of staff members and local communities are being taken. Limitations • Korey Reverine outreach services was suspended after long-lasting clan conflict which resulted heavry fitghting between the DIR clan and Gabawayn minority clan in Dollow, Gedo • Somalia presidential and parlimentary elections are due and may take place when data collection began which may affect travelling • According to COVID-19 prevention measures, new arrivals in the country may have to complete 2 weeks quarantine, this will affect if international consultant is engaged on the assignment. IV.C. Sampling Strategy The survey will employ a stratified two-stage cluster sampling as it is the most efficient way to sample the population given that the beneficiaries are in different districts receiving different interventions. The districts would be the strata to ensure that every district would be proportionately represented in the evaluation. The first stage of sampling would select sample villages/IDP camps from each district determined by Probability Proportional to Size (PPS) sampling. Households, registered as beneficiaries, would then be selected from these villages/IDP camps by simple random sampling and would be the primary units for the survey. The beneficiary lists from the implementing partners would be the sampling frame. The lists would contain household contact information and intervention(s) received. For KIIs, there should be at least one interview from a key CRS staff (MEAL & Program), one interview from a key staff of each of the implementing partners, and one interview from the camp leaders or key persons in each of the village/IDPs surveyed. If possible, KIIs should be recorded during the interview process and translated from Somali to English, as needed. The consultant shall determine any other KIIs that are feasible within the framework of mitigating the effects of COVID-19 and that would contribute to answering the Evaluation Questions above. IV.D. Data Analysis Procedures The study proposes to collect data using household questionnaire coded in mobile application. Descriptive statistics (frequencies and percentages, means, medians, and standard deviations) will be used to describe the evaluation findings on the project indicators. Qualitative data obtained using KIIs will be captured using KII guides, beneficiaries’ interview, field observations and FGDs translated into English and later typed in Microsoft Word templates. Analysis for this data will be mainly through content analysis and establishment of themes. The findings from qualitative data will be used to triangulate the primary quantitative data and secondary data (from project documents and performance reports) for the final conclusions of the evaluation findings. IV.D. Considerations/Recommendations on COVID-19 Page 92 of 97 As COVID-19 affects the communities and stakeholders we work with, CRS is putting first the safety and well-being of the staff, beneficiaries, consultants, and other stakeholders in project regions. The consultant is expected to adapt their evaluation plans and methodologies according to the changing situation, prioritizing safety and informing CRS of all changes. The consultant must respect all COVID￾19 directives issued by the Federal Government of Somalia and federal member states where data collection is taking place. The consultant is allowed to utilize virtual meeting platforms in conducting some key informant interviews, especially for the target respondents from CRS, partners, and stakeholders with enough digital literacy. Lastly, the consultant is also expected to be transparent on the possible effects of the COVID-19 situation on the data gathered. V. EVALUATION TEAM Evaluation (Lead) Consultant: Will plan and coordinate data collection, review data, analyses it and prepare a high-quality report. The consultant shall report to CRS Somalia MEAL Manager who will also work closely with him/her during the evaluation. Key working relations: Somalia MEAL Manager, Somalia Emergency Coordinator, Somalia Program Manager Key Responsibilities  Develop an inception report, detailing the agreed upon study design, methodology, indicators, data- gathering tools, work plan schedule and budget to carry out the assignment, in consultation with CRS.  Conduct desk‐review of relevant project documents and secondary data  Develop quantitative and qualitative data gathering tools in consultation with CRS  Plan and coordinate quantitative and qualitative data collection  Conduct training for the data collection teams including pre‐testing of data collection tools  Organize and facilitate team interactions  Provide support to evaluation team members to fulfil their obligations  Conduct Key Informant Interviews  Review, clean and analyze data collected  Write report on the findings and recommendations  Present preliminary findings to project stakeholders for validation  Incorporate input from project stakeholders and submit final report  Carry out or assist in additional duties assigned by the project staff Desired Qualifications and abilities  A minimum educational qualification of a Master’s degree in Monitoring and Evaluation, Emergency response, Social Sciences, Economics or relevant field from recognized university  Must have a proven research experience in the Somalia context.  Has undertaken similar evaluations in the past 3 years in Somalia. This includes demonstrated ability to manage field procedures in the evaluation area.  Previous evaluation experience for a USAID project is an added advantage.  Solid experience in qualitative and quantitative studies.  Experienced in use of ICT4D solutions in data gathering and remote data collection and management.  Computer proficiency with good knowledge of MS office (Word, Excel, PowerPoint) and data analysis applications e.g., SPSS, STATA.  Excellent analytical and report writing skills  Excellent written and spoken English. Knowledge of Somali language will be an added advantage  Excellent communication and interpersonal skills  Excellent time management skills  Ability to work promptly and accurately, and pay attention to detail Page 93 of 97  Ability to work well both independently and in a team  Available to be engaged during the entire survey period Field Supervisors: Will take part in enumerator training, guide and supervise data collection Report to Lead Consultant Key responsibilities  Obtain sampling lists for each area in which his/her team will be working  Assign work to enumerators.  Maintain fieldwork control sheets and make sure assignments are carried out  Communicate any problems to the Lead Consultant and/or project staff  Foster a positive team spirit  Conduct regular spot-checks and re￾interviews  Conduct regular review sessions with each enumerator  Receive data from enumerators (questionnaires, focus group guides etc.) at the end of each day  Produce a summary observation report detailing daily achievements, general observations, challenges and summary findings/ emerging themes.  Oversee entry of data into established data entry templates as necessary by team members at the end of each data collection day  Ensure that all evaluation procedures and protocols are followed  Carrying out or assist in additional duties assigned by the Lead Consultant Desired Qualifications and abilities  Minimum post-secondary college or university education  Familiarity with the Somalia context (specifically in project areas)  Fluency in written and spoken Somali and English  Experienced in team management.  Familiarity with data collection using mobile technologies  Understands surveys ethics and protocols.  Understanding of data confidentiality issues.   Ability to work with minimum supervision  Excellent communication and interpersonal skills  Ability to multitask  Social Perceptiveness- Aware of other reactions and understands them  Ability to work quickly and accurately, and pay attention to detail  Ability to work well both independently and in a team  Available to be engaged during the entire survey period. Enumerators: Will administer questionnaires to respondent Report to the Field Supervisors. Key responsibilities  Locate households and identify respondents  Explain survey and/or focus group objectives and procedures to interviewees  Ask questions in accordance with instructions to obtain various specified information  Interpret questions to help interviewees' comprehension  Identify and resolve inconsistencies in interviewees' responses by means of appropriate questioning and/or explanation  Review data obtained from interview for completeness and accuracy.  Identify and report problems in obtaining valid data  Produce a daily observation report detailing daily achievements, general observations, Desired Qualifications and Abilities  Minimum secondary education  Prior experience conducting data collection for government programs and/or international NGOs in Somalia  Excellent verbal and written communication skills in English and Somali  Familiarity with mobile data collection technologies  Understands survey protocols and ethics.   Familiarity with Afgooye, Baidoa and Mogadishu districts geography  Knowledge of mobile based data collection  Excellent communication and interpersonal skills  Social perceptiveness- Aware of other reactions and understands them Page 94 of 97 challenges and summary findings/emerging themes.  Make and honor appointments made with respondents in cases where the respondent was not available for interview  Meet with supervisor daily to submit completed assignments and discuss progress  Carry out or assist in the additional procedures for data collection, as requested by the field supervisor  Excellent time management skills  Ability to work quickly and accurately, and pay attention to detail  Ability to work well both independently and in a team  Respectful and friendly in all interactions  Available to be engaged during the entire survey period Role of CRS Role of Implementing Partners  Recruit external consultant for the evaluation  Provide consultant with project documents, reports and available secondary data for review  Review data collection tools and inception report  Make travel, accommodation and per-diem arrangements for consultant and team  Oversee the recruitment of enumerators by the consultant  Provide oversight for the data collection  Review and provide input on the consultant’s deliverables  Organize validation workshop for the presentation of preliminary findings to the project stakeholders  Ensure smooth flow of consultancy engagement processes including contractual obligations  Share the final evaluation report with all key stakeholders, including key project staff, partners, donor representative and government  Provide sample frames from which respondents for the quantitative data will be drawn  Assist in the identification, contacting and locating of project beneficiaries and stakeholders  Support recruitment of enumerators  Conduct community sensitization and mobilization of respondents  Participate in the validation workshop The composition of the evaluation team is flexible to change, depending on the approach of the consultant to address the limitations posed by the COVID-19 pandemic. VI.REPORTING AND DISSEMINATION PLAN VI.A. Final Evaluation Report The consultant must submit two hard copies and a soft copy of the final evaluation report which is expected to be within 35 pages (without annexes) and with the following components:  Preliminary Pages (Title page, Table of Contents including a list of annexes, tables and figures, Acknowledgement, List of Acronyms and Abbreviations, Definition of Terms and Concepts)  Executive Summary that includes IPTT  Introduction describing the project’s background and context, a description of the program, including the results framework and theory of change  Purpose and Objectives of Evaluation  Key evaluation questions (or objectives) and a statement of the scope of the evaluation Page 95 of 97  An overview of the evaluation approach, methodology and data sources, as well as limitations and delimitations of the evaluation itself  Evaluation findings, organized around the four key Evaluation Questions  Conclusions based on evaluation findings, organized around the four key Evaluation Questions  Lessons learned based on the evaluation findings  Recommendations based on conclusions, organized by audience: 1.) Donor/OFDA and 2.) CRS and its LIPs, as well as future Implementing Partners, etc.  Appendices (including all data collection tools, respondent lists, raw data collected, data analysis files, list of secondary documents reviewed etc.) VI.B. Dissemination Plan STAKEHOLDE R/ AUDIENCE KEY FINDINGS CHANNEL(S) OF COMMUNICATIO N PRODUCT(S ) TO SHARE USAID/OFDA • To establish the impact of the project components on the target beneficiaries. • Suggestions/recommendatio ns with the potential to shape future programming. • Email communication • Final Evaluation Report CRS and local implementing partners (LIPs) • To establish the impact of the project component on the target beneficiaries. • Suggestions/recommendatio ns with the potential to shape future programming • To assess and evaluate effectiveness of the response interventions in terms of its implementation approach and strategies • Email communication • Dissemination and reflection meeting • Hard copies • Final Evaluation Report and Appendices • PowerPoint presentatio n from validation event VII. SCHEDULE AND LOGISTICS The consultant should prepare a detailed work plan document, in which he/she describes the evaluation’s overall schedule (i.e., duration, phasing, timing) as well as work hours, required preparation work, conditions that might affect data collection, meeting-arranging procedures, and needed and available office space, cars, equipment, and local services (e.g., interviewers). VIII. DELIVERABLES AND TIMELINE The following items will be expected to be delivered during implementation, analysis, and reporting on the evaluation: • Inception Report with data collection tools • A detailed work plan, with target dates and deliverables identified and highlighted • All data collection tools • Sampling guidance and sample, as well as list of participants for KIIs • Cleaned quantitative dataset (for quantitative data collection methods) • Related codebooks, and data analysis files (Excel format other software (SPSS) syntax files required) • Key informant interview forms/reports (MS word) • All photos and field notes with quotation from beneficiary verbatim • The final report shall be submitted in two (2) hard copies and one (1) soft copy. Page 96 of 97 DELIVERABLES ESTIMATED NUMBER OF DAYS NEEDED TO COMPLETE TARGET DATES TO COMPLETE Initial meeting between CRS and the consultant to agree on the evaluation methodology and data collection tools 1 20, Sept 2021 Consultant reviews project documents and submits an inception report with all data collection tools and guidance 3 23, Sept, 2021 Revision of the inception report and data collection tools by the consultant based on feedback provided by CRS 2 25, Sept, 2021 Translation and coding of tools 3 28, Sept, 2021 Training for enumerators (including pretest; only if needed depending on the approach of the consultant on the COVID-19 situation) 2 30, Sept, 2021 Data collection (Qualitative and quantitative concurrently) Translation of qualitative data. 8 08, Oct, 2021 Data cleaning and analysis with submission of quantitative database and command files (SPSS or Stata), qualitative descriptive files, data tables and submission of draft report. 3 11, Oct, 2021 Revise and finalize report based on feedback provided by CRS. Submit any final datasets or annexes. 2 14, Oct, 2021 Results dissemination meeting with CRS leadership, Partners and Consultant to validate the final Report 1 15, Oct, 2021 Total consultant engagement days (Estimated) 25 days IX. BUDGET CRS will provide for the following costs for the consultant team: payment for translators, data collectors, data processors (as needed), and secretarial services; equipment, etc. CRS will procure the consultant services based on a competitive rate and ability to complete the assignment. X. ETHICAL CONSIDERATIONS The below ethical considerations will be adhered to during the evaluation process: i.) The evaluation will be conducted by an independent and impartial external evaluator. ii.) Participation in the survey will be voluntary. iii.) Anonymity, confidentiality and safeguarding of survey data will be guaranteed. iv.) There will be no risks and benefits for individual participants v.) The culture, norms and traditions of study populations will be respected. Page 97 of 97 vi.) Consent will be sought prior to commencing data gathering vii.) Views and Opinions of the different survey subjects will be respected. viii.) Abide by the “Do No Harm” principle especially in the context of COVID-19. ix.) The consultant should limit the risk of spreading COVID-19 to communities and implementing partner organizations staff by avoiding in-person data collection. x.) The consultant should always adhere to the COVID-19 directives issued by the Federal Government of Somalia and federal member states where the data collection is taking place. xi.) Comply with USAID evaluation procedures by ensuring external consultant has been listed under Section 1.4.b.2.B of your award entitled "Sub-Award, Transfer, or Contracting Out of Any Work" XX. Key evaluation compliance requirements See the USAID evaluation policy (https://www.usaid.gov/evaluation/policy) and CRS – MEAL policies and procedures (available on request). XXX. QUOTATION REQUIREMENTS Interested applicants to send a technical and financial proposal for the work in line with the following guidance:  Capability statement: How the consultant or firm is structured for the assignment, the role each staff will play including the CVs of the key personnel who will take part in the consultancy  Technical Proposal: The consultant’s understanding and interpretation of the Terms of Reference (TOR), a detailed methodology and plan on how the data collection and analysis will be done and detailed implementation schedule for the evaluation.  Financial proposal: Itemized budget proposal that should include the consultancy fees and operational costs.  References: Names, addresses, telephone numbers of three organizations that you have conducted evaluations for within the last three years, that will act as professional referees  Evaluation reports: Final reports for the evaluations conducted for the three reference organizations provided XXL. APPLICATION PROCESS Subject Line: End of Project Evaluation Consultancy Services for CRS Somalia Send to crskenya-procurement@crs.org. Deadline for the application is 19, Sept 2021 Applications received after this date will not be considered.