September 2014 This publication was produced at the request of the United States Agency for International Development (USAID). International Business & Technical Consultants, Inc. (IBTCI) prepared it independently under order number AID-OAA￾M-13-00005. The authors are Team Leader, Gayla Cook, Mesfin Beyero, Patricia Bonnard, Phillips Lukwiya, Eunice Wavomba, and Steven Hansch. EVALUATION Formative Evaluation of USAID’s Office of U.S. Foreign Disaster Assistance and Office of Food for Peace Formative Evaluation of the Office of U.S. Foreign Disaster Assistance and Office of Food For Peace Programs in Yemen Cover photo: Families receive food items as part of Global Communities’ Yemen Food for Asset Development Program, in Ibb governorate, Jeblah village food distribution point, April 14, 2014, funded by USAID’s Office of Food for Peace. Photographer: Mohammed Al-Salmy. FORMATIVE EVALUATION OF USAID’s OFFICE OF U.S. FOREIGN DISASTER ASSISTANCE AND OFFICE OF FOOD FOR PEACE A PROCESS ASSESSMENT OF HOW IMPLEMENTING PARTNERS REACH AND SERVE THE NEEDIEST BENEFICIARIES September 26, 2014 Order No.: AID-OAA-M-13-00005 Contract No.: GS-10F-0309P DISCLAIMER The author’s views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. ACKNOWLEDGMENTS Many people and organizations contributed to this assessment of the effectiveness of processes of USAID’s Office of U.S. Foreign Disaster Assistance (OFDA) and Office of Food for Peace (FFP) to assist those people who are most in need in Yemen. Inaccessibility in Yemen – which can be associated with gender, physical terrain, civil conflict, language differences, low education levels, or cultural norms – is the greatest challenge to humanitarian aid, and to monitoring and evaluation efforts generally. As they respond to the challenges of access, foreign humanitarian workers in Yemen face the risks of political and criminal kidnapping or attack, and indirect exposure to armed clashes, as do their even larger numbers of Yemeni colleagues. Therefore, appreciation goes to all the NGOs and other international implementing partners referred to in this report, and to their many local partners (IBTCI could not name them all) for their dedication, and for their cooperation in conducting this independent evaluation. Appreciation goes to Prodigy Systems, the Yemeni company that managed teams of data collectors throughout the country to successfully survey and interview partners and beneficiaries, including in very hard-to￾reach areas, and in areas with conflict nearby, and for their admirable technical acumen. IBTCI also recognizes the similarly hard-working staff of YOFMEP, both Yemeni and international, who provided the programmatic and administrative support to this evaluation. This assessment witnesses the dedicated, hard-working Yemenis at the national, governorate, district and community levels, who – as workers or volunteers – daily strive to assist the neediest children, women and men among the population, lack adequate food, health services, water and sanitation facilities, and face the dangers of land mines. Their efforts and persistence in finding longer-term solutions engender hope for the future. This report is an act of accountability to the American taxpayers, whom IBTCI acknowledges for providing their generous resources to alleviate the suffering of the most vulnerable Yemenis, through the offices of OFDA and FFP at USAID. It also represents accountability to the Yemeni beneficiaries to listen to and reflect their views on how well the programs operate and the effectiveness of the programs. The Evaluation Team CONTENTS Acronym List Executive Summary 1 Introduction 9 Evaluation Background 9 Formative Evaluation Purpose 9 Evaluation Lines of Inquiry: Target Implementing Partners and Approach 9 OFDA & FFP Strategy in Yemen 10 Formative Evaluation Selection Choices: Target Implementing Partners & Sectors 14 The Formative Evaluation Approach 15 Evaluation Methods and Limitations 17 Sample Frame and Site Selection 19 Evaluation Limitations 24 Findings 28 Agriculture/Food Security and Economic Recovery and Market Systems 28 Health and Nutrition 43 Water, Sanitation and Hygiene 61 Protection 76 Conclusions and Recommendations 84 Agriculture/Food Security and Economic Recovery and Market Systems 84 Health and Nutrition 86 Water, Sanitation and Hygiene 90 Protection: Mine Risk Education 94 Gender Implications 98 Portfolio Review: OFDA and FFP 101 Tables Table 1: Numbers of Persons Surveyed .................................................................................................................... 20 Table 2: Agriculture and ERMS Indicator Results Summary .................................................................................. 29 Table 3: Agriculture and ERMS Implementation Status by Project (as of March 2013) ................................. 30 Table 4: Sampling and Units of Analysis .................................................................................................................... 33 Table 5: OFDA Principal Beneficiary Selection Criteria ......................................................................................... 34 Table 6: FFP Principal Beneficiary Selection Criteria .............................................................................................. 36 Table 7: Health and Nutrition Indicator Results Summary ................................................................................... 44 Table 8: Health and Nutrition Project Implementation Status ............................................................................. 47 Table 9: Health & Nutrition Sampling and Units of Analysis ............................................................................... 50 Table 10: WASH Indicator Results Summary ........................................................................................................... 62 Table 11: WASH Implementation Status by Project ............................................................................................... 63 Table 12: Interviews and Surveys ............................................................................................................................... 64 Table 13: Who Owns and Controls Water Resources ........................................................................................ 69 Table 14: Protection Indicator Results Summary .................................................................................................... 77 Table 15: MRE Implementation Status....................................................................................................................... 78 Table 16: MRE Sample Frame and Interviews ........................................................................................................... 79 Figures Figure 1: Active USG Humanitarian Programs in Yemen 12 Figure 2: Map of Data Collection Locations 23 Figure 3: Current Forms of Support from IPs to Health Centers/Partners 51 Figure 4: Health Care Outreach Activities to Get Care Out into Their Community 52 Figure 5: Health Facility Concerns of Target Populations 55 Figure 6: Nutrition Services Gaps or Concerns Expressed 56 Figure 7: Beneficiary Views about Food Rations 57 Figure 8: Income Sources of WASH Target Households 65 Figure 9: Household Concerns about Their Challenges to Water Access 68 Figure 10: Latrine Access in WASH Target Areas 69 Figure 11: Association between Hygiene and Level of Education 70 Figure 12: Where Households in Program Areas Say They Obtain Their Water Supplies 71 Figure 13: Perceptions about Water Management Committees 73 Figure 14: Children's Time Use Related to UXOs 79 Figure 15: Responses from Children about What They Learned in MRE 81 Figure 16: Gender Distribution within Each Survey Group in Formative Evaluation 98 Figure 17: General Level of Support by OFDA and FFP in the Different Investigations in Yemen 101 Annexes Annex I: Evaluation Statement of Work Annex II: Formative Evaluation Timetable Annex III: Lines of Inquiry and Corresponding Indicators Annex IV: Evaluation Team Members, Disclosure of Conflict of Interest Forms/Statement of Differences Annex V: OFDA and FFP Strategic Targets Annex VI: Results of the Field Inquiry Annex VII: Data Collection Instruments Annex VIII: Bibliography of Documents Reviewed Annex IX: OFDA Partners' Statement of Differences ACRONYMS ADRA Adventist Development and Relief Agency ANC Antenatal Care BCC Behavior Change Communication CAMC Community Asset Management Committee CFSS Comprehensive Food Security Survey CHW Community Health Worker CLC Community Livelihood Committee CLTS Community-led Total Sanitation CMAM Community-based Management of Acute Malnutrition CRC Community Resilience Committee CSO Central Statistics Organization CSSW Charitable Society for Social Welfare CVC Community Voluntary Committee CY Calendar Year DCHA USAID Bureau of Democracy, Conflict and Humanitarian Assistance DFID Department for International Development, UK DHO District Health Office DQA Data Quality Assessment DRC Danish Refugee Council DRR Disaster Risk Reduction EA Enumeration Area EASE Emergency Assistance to Support Yemeni Communities EAVP-Y Emergency Assistance to Vulnerable Populations in Yemen EBF Exclusive Breastfeeding EFSP Emergency Food Security and Resilience Programming EMMA Emergency Market and Mapping Assessment EmOC Emergency Obstetric Care ERMS Economic Recovery and Market Systems ERW Explosive Remnants of Wars ESN Emergency Safety Net FARA Food Assistance, Resilience Achieved FEWS NET Famine Early Warning Systems Network FFA Food for Assets FFP Office of Food for Peace FFT Food for Training FFW Food for Work FGD Focus Group Discussion GC Global Communities GIS Geographic Information System GHO Governorate Health Office GPS Global Positioning System H/N Health and Nutrition HH Household HIS Health Information System IBTCI International Business and Technical Consultants, Inc. IDA International Disaster Assistance IDP Internally Displaced Person IEC Information, Education and Communication IMC International Medical Corps IMCI Integrated Management of Childhood Illnesses INGO International Non-Governmental Organization IOM International Organization for Migration IP Implementing Partner IRD International Relief and Development IYCF Infant and Young Child Feeding KII Key Informant Interview M&E Monitoring & Evaluation MAI Ministry of Agriculture and Irrigation MAM Moderate Acute Malnutrition MC Mercy Corps MCHN Maternal Child Health and Nutrition MOH Ministry of Health MOPIC Ministry of Planning and International Cooperation MOSAL Ministry of Social Affairs and Labor MPHP Ministry of Public Health and Population MRE Landmine Risk Education MTEVT Ministry of Technical Education and Vocational Training MTMSG Mother-to-Mother Support Group MWE Ministry of Water and Environment n/a Not Applicable NGO Non-Governmental Organization OCHA United Nations Office for the Coordination of Humanitarian Affairs OFDA Office of U.S. Foreign Disaster Assistance OTP Outpatient Therapeutic Program PHAST Participatory Hygiene and Sanitation Transformation PHC Primary Health Care PLW Pregnant and Lactating Women PPS Probability Proportional to Size RoYG Republic of Yemen Government RUSF Ready to Use Supplementary Food RUTF Ready to Use Therapeutic Food SAM Severe Acute Malnutrition SCI Save the Children International SFD Social Fund for Development SHA Senior Humanitarian Advisor of USAID in Yemen SMART Standardized Monitoring and Assessment of Relief and Transitions SOW Statement of Work SWF Social Welfare Fund TFC Therapeutic Feeding Center ToT Training of Trainers TPM Third Party Monitoring U-FE Utilization Focused Evaluation UN United Nations UNDP United Nations Development Programme UNHCR United Nations High Commissioner for Refugees UNICEF United Nations Children’s Fund USAID United States Agency for International Development UXO Unexploded Ordnance of War VCHW Voluntary Community Health Worker VFA Voucher for Assets WASH Water, Sanitation and Hygiene WB The World Bank WFP UN World Food Programme WHO World Health Organization WMC Water Management Committee WSB Wheat Soy Blend WfH Weight for Height YALLA Yemen, Abyan and Lahj Livelihood Assistance YEMAC Yemen Executive Mine Action Center YFAD Yemen Food for Asset Development YOFMEP Yemen OFDA/FFP Monitoring & Evaluation Program 1 EXECUTIVE SUMMARY In order to inform the U.S. Government’s (USG) humanitarian programming in Yemen, a formative evaluation was commissioned by the Office of U.S. Foreign Disaster Assistance (OFDA) and Office of Food for Peace (FFP), which are both part of the US Agency for International Development (USAID) Bureau for Democracy, Conflict and Humanitarian Action (DCHA). This evaluation was conducted to look at the OFDA and FFP supported humanitarian programs, processes and target populations in emergency and conflict-affected areas of Yemen. International Business and Technical Consultants, Inc. (IBTCI) is the contractor for the Yemen OFDA and FFP Monitoring and Evaluation Program (YOFMEP), which includes this evaluation as one component. The implementing partners (IPs) who were the subjects of this evaluation included the Adventist Development and Relief Agency (ADRA), Global Communities (GC), International Medical Corps (IMC), International Relief and Development (IRD), Mercy Corps (MC), Save the Children International (SCI), the International Organization for Migration (IOM), the UN World Food Programme (WFP), and the United Nations Children’s Fund (UNICEF). Collectively they channeled roughly $200 million in USAID assistance in Yemen since 2012. These agencies are working with internally displaced persons (IDPs), communities affected by food shortages, communities that have been marginalized, and communities affected by armed conflict. Each of these IPs have integrated programs that attempt to address priority needs across the technical sectors where the most urgent gaps were identified. The core sectors to which OFDA and FFP are giving priority are food security (including food access, markets, and agriculture), water, sanitation and hygiene (WASH), and health, nutrition and protection (with a focus on landmine awareness). Each IP also has different relations with local partners, many being governmental or semi-governmental entities (health centers), village committees and indigenous non-governmental organizations (NGOs) such as the Yemen Executive Mine Action Center (YEMAC) or the Charitable Society for Social Welfare (CSSW). The geographic scope of 80 percent of IP activities is primarily rural, emphasizing those locations (governorates, districts and villages) which are most adversely affected by conflict, including those that have hosted large numbers of displaced people and those with higher levels of food insecurity within the general population. The Formative Evaluation approach applied a Utilization-Focused Evaluation (U-FE) approach, to assist the intended users - USAID OFDA and FFP managers in the field and in Washington, D.C. - in making decisions related to complex program design and implementation issues. Three overarching lines of inquiry framed the approach: Line of Inquiry 1: What is the project doing to ensure that it reaches the beneficiaries most in need? What are the challenges to reaching the beneficiaries most in need? How have IPs overcome those challenges? Line of Inquiry 2: Are there significant needs within OFDA and FFP’s operational sectors that are not currently being addressed or met by OFDA and FFP programming? Line of Inquiry 3: What are the most significant lessons learned about the implementation process of OFDA and FFP programming from the monitoring process to date? What are the most significant strengths of and areas of improvement for OFDA and FFP programming? 2 Methodology The formative evaluation adopted a mix of complementary methods including document review, interviews with numerous officials and experts, field-based focus group discussions (FGDs), and a battery of structured surveys. The YOFMEP formative evaluation generated original data from interviews with beneficiaries and with partner institutions. The sample was derived using a Proportionate-to-Population approach. In all, 1,492 interviews were conducted. OFDA and FFP approved the evaluation plan in December 2013. Field data collection took place in April 2014 after security issues had created delays in the arrival of external technical specialists in Sana’a. A team of expatriate and local technical specialists conducted the formative evaluation.  For partner interviews, 14 teachers and 20 health facility officers were interviewed, covering protection and health issues. In the course of other structured interviews and FGDs, the evaluation team and Arabic-speaking interviewers drew original evidence from the perspectives of 140 people.  To gather evidence from beneficiary households, the evaluation included 126 health/nutrition interviews, 98 WASH interviews, and 1,194 food distribution interviews using structured survey instruments. The sample of the formative evaluation was constructed using a Probability Proportional to Size (PPS) approach, proportionately including the target populations of program sectors funded by OFDA and FFP. As the numbers reflect, alleviating hunger through food distribution with FFP funding is by far the largest USAID humanitarian investment in Yemen. Limitations detailed in the report include denial of access to some areas by insecurity, lack of knowledge of very vulnerable and marginalized groups and gaps in data from the Government or counterparts. Attribution is limited as many IPs receive parallel streams of funding, and each IP intervention is at a different point of its project cycle. FINDINGS, CONCLUSIONS, AND RECOMMENDATIONS Following are the main findings, organized by key sectors: Agriculture/Food Security and Economic Recovery and Market Systems Findings and Conclusions  In identifying target groups, FFP uses the Social Welfare Fund (SWF) lists, which are based on poverty measures, as the initial step in the identification of beneficiaries for Title II food assistance and selection of priority geographic areas.  WFP is assisting 3.8 million people monthly, via its safety-net program. Food assistance was verified as given routinely, often as monthly rations, particularly targeted to IDPs. People came to distribution points and transported their rations home by food, donkey, wheelbarrows, cars and trucks. WFP distributions are calculated in relation to SPHERE minimum standards, with a target of 2,100 kilocalories (or 450 grams) per person per day.  Several International non-governmental organizations (INGO) apply FFP assistance to implement WASH, early recovery and community resilience.  The beneficiary selection used by IPs is appropriately needs-based, and the criteria used are well-defined and understood by administrators, though the criteria are not well understood among recipients. Sixty-nine percent of survey respondents said that they did not know what ration size or composition that they were entitled to. Current distribution includes 10 3 kilograms of wheat soy blend (WSB) and one liter of vegetable oil, whereas recipients would prefer 25kgs of WSB and 4 liters of oil.  There has been a shift from item-based to value-based food baskets (i.e. through vouchers denominated in local currencies, not weight of food), where the beneficiaries will have the freedom to choose from the 14 food items available, potentially valuable for capturing in the summative evaluation.  IPs actively involve many local community leaders (religious leaders – imams, Sheikhs and Mullahs, local council members) who assist in reaching out to those in need.  For food distribution, the house-to-house validation process seems to encourage dialogue, leads to more accurate lists and is appreciated by the target populations.  The weaker government systems have resulted in the beneficiaries to be entirely dependent on the good will of the service providers, i.e. the health workers in the facilities and local leaders managing the food distribution points (FDPs). A lack of transparency and accountability in systems has been reported.  Cluster (UN) coordination of food security works better at regional level than national.  Credit is a constraint to economic recovery, according to many respondents.  Focus group respondents articulated a desire/need for seeds, tools and fertilizer for recovery.  IPs are responding by building the capacity of local indigenous NGOs who can access and can stay in the areas of implementation longer, putting different innovative and technologically advanced tools in place for monitoring of project activities, hiring temporary staff, and using mobile teams for outreach.  Youth are increasingly recognized by IPs as important though neglected target group.  The Emergency Food Security and Resilience Program (EFSP) project of MC serves as a replicable example of service delivery that is well- coordinated with the local community. The communities select the community assets (e.g., WASH infrastructure) to be constructed or rehabilitated after going through a prioritization process that may ensure sustainability. Agriculture/Food Security and Economic Recovery and Market Systems Recommendations  IPs should share positive and successful lessons among themselves about how communities were engaged in the development of criteria and a sense of community ownership.  As WFP draws back on the delivery of large-scale food ration distribution, WFP might consider to simultaneously transition a greater share of its portfolio to targeted/supplementary feeding of malnourished children in order to expand the proportion of children who benefit from a coherent referral system with UNICEF. All children who are discharged from UNICEF-supported therapeutic feeding (management of severe acute malnutrition (SAM)) or are found in the course of UNICEF case finding and surveillance ought to be entered into supplementary feeding centers and remain there until after the child recovers above the threshold (-2 standard deviations) to “mild” status.  Further research should be conducted about how specialty foods are used post-distribution, including leakage or sharing, nor what the cost-effectiveness comparisons are among commodities. 4  Voucher schemes help beneficiaries while stimulating private markets. The extent of these value-chain and supply-chain benefits need to be better measured and documented.  Economic Recovery and Market Systems (ERMS) programs should be scaled up faster and in concert with the shift from relief to development, but building on which models work. The use of vouchers should be carefully studied to understand its effect on local markets and the pace of value chain recovery. Health and Nutrition Findings and Conclusions  The use of mobile health teams to reach out to inaccessible areas is effective.  There is a systematic lack of capacity in the national health care system and an absence of health workers at health facilities for much of each day. There is a lack of family planning services available to target populations, which IPs are not meeting.  Tracking of individual malnourished children does not occur and there is an absence of appropriate monitoring tools that indicate not only how much of the therapeutic products have been provided but also how the therapeutic foods are administered to malnourished children on a daily basis.  The designation of confirmed cholera cases as something else, i.e. “acute watery diarrhea” inhibits effective technical attention from the international community, including the mobilization of World Health Organization (WHO) resources. In recent years, outbreaks of cholera have led an increasing number of national governments to formally shy away from allowing use of the formal term cholera, believing that it may scare away trade, tourism, and confidence in the government.  Despite considerable effort by IPs in capacity building, health facilities ask for more training.  There is evident self-exclusion by marginalized groups who are afraid of interaction with authorities and NGOs at health facilities. Nomadic people are very hard to reach for program interventions.  There is a lack of proper mechanism for getting beneficiaries’ feedback.  The monthly health cluster meetings allow them to exchange updates about who is doing what and where, and help to avoid duplication of efforts and wastage of resources.  Projects require sufficient time at the outset to raise the level of awareness of the community on what the project is about. Grievance/complaint channels were absent from most IP projects. Health and Nutrition Recommendations  IPs should give particular attention to urban areas, where it is often, wrongly, assumed the default rates are lower.  IPs should train voluntary community health workers (VCHWs) on proper counseling techniques to make communication effective for behavioral change. The counseling should be timely, based on the problem at the time of visit. USAID/OFDA needs to monitor that IPs are closely involved in the selection process of health workers for the different trainings.  Monthly joint supervision with the Ministry of Public Health and Population (MPHP) can further facilitate whether those in need are reached and humanitarian workers take action in the earliest time possible 5 WASH Findings and Conclusions  Notably, every household surveyed had a water source within 500 meters of where they lived, which represents progress for this population.  Sanitation and hygiene results are not being met as seen in the finding that 96 percent of those surveyed said that hygiene campaigns were not beneficial.  While water supply is addressed well, there is correspondingly little attention to water quality. Four-fifths of respondents indicated that there was no treatment of water supply at any level (community or household), and IPs are not addressing point of consumption water quality.  Most WASH services addressed gender-sensitive privacy needs such as gender segregated sanitation facilities at markets and health centers.  Formation and use of water management committees (WMCs) is a good example of coordinating service delivery with the communities and is critical not only for the purposes of ownership and sustainability, but also as a means to empower local structures on governance issues. The local, community-level WMCs, with membership selected through rigorous processes, include some women for minimal gender balance, as the implementing agencies require it.  In the WASH sector, UNICEF has built networks with partners, resulting in better coordination in rapidly meeting emergency needs of communities. Also, it has prepositioned WASH supplies at community levels, as part of emergency preparedness plans and to better respond to urgent needs of affected communities.  In areas where UNICEF works in WASH activities, local key informants stated that there is no “exit strategy” for longer-term transition. This is a particular dilemma where international aid supports the ongoing trucking expenses of basic water to IDP camps. This form of emergency water supply provision without a long-term sustainable solution is globally understood to be cost ineffective.  Water trucking is frequently used only as a short-term stop gap in emergencies, but in Yemen the recurrent costs of trucking, including fuel, exceeds the cost of any other sustainable water infrastructure that could have been built WASH Recommendations  For a long-term sustainable solution to sanitation, donors should encourage IPs to adopt a sustainable model for Sanitation Marketing and Community-led Total Sanitation (CLTS) approaches in the promotion of sanitation and hygiene as a viable mechanism for increasing sanitation coverage by supporting efforts to enhance the capacity of the private sector to supply desirable sanitation products. Sanitation marketing should focus on demand creation through media and communications campaigns.  Research should look into the medium-term costs and benefits of aid support to privately￾owned water sources (which charge beneficiaries for access) as compared to other, public or newly-established water sources.  Considering the inability of the government departments to take the lead in coordinating WASH activities with IPs and the low capacities of the community groups with limited abilities to manage and sustain WASH services/facilities, coupled with low level of participation of women in WASH activities due to low level literacy and marginalization, future intervention priority should be given to strengthening capacity of local joint communities, local 6 organizations and local government in program cycle management, including financial management, gender and rights based approaches, conflict sensitive programming, and participatory approaches to service delivery. Protection Findings and Conclusions  Landmine awareness by UNICEF and SCI has good coverage and good uptake of messages. Among those surveyed, every child had had some landmine risk education (MRE). Almost all learned at their schools.  Sub-award partners play key roles. For landmine education, UNICEF implements through the Danish Refugee Council (DRC), which delivers landmine risk education to children through their interactive participation. DRC uses games to raise awareness and to heighten the interest of children in MRE. This approach seems effective in engaging children, indeed, more effective than approaches that depend on teachers. A UNICEF government counterpart also requested that UNICEF include interactive materials such as games in the MRE kits. Protection Recommendations  Programs supported by OFDA to reduce the hazards of explosive remnants of war (ERW) could be better integrated across the five key components: education, humanitarian demining (landmine demarcation of fields is also important), victims’ assistance including rehabilitation and reintegration, stock pile destruction and advocacy against the use of antipersonnel mines.  In particular, given the extent of the problem with mines and unexploded ordnances of war (UXOs), UNICEF should start engaging with the government in developing an action plan for including MRE in to the overall school curriculum.  Based on the expressed needs of IPs, additional items such as cameras and games for children should be included in the MRE training kits. Support should also allow IPs to include MRE as a regular component in the school curriculum as this would ensure sustainability of risk awareness. PORTFOLIO LEVEL FINDINGS  IPs are deploying more staff out in the field to cover wider geographical areas and monitor projects. • Yemen offers too few credible local/national partners for delivery of humanitarian assistance. IPs are supported to work with national partners for outreach, necessary in many remote areas. For example, because of insecurity in Abyan Governorate, GC built partnership with the local NGO, CSSW, to reach out to communities there. • Poor security conditions sometimes confound any quality in monitoring and therefore feedback. • The physical topography of the nation makes program access difficult. While projects are effective in meeting gaps among those reached, they are not achieving the scale to meet the larger needs of the overall vulnerable population. • Both OFDA and FFP activities continue to follow relief modalities, i.e. classic, short-term approaches to direct delivery of many services. Even as programs shift to “recovery”, there is a growing need to shift to resilience building, i.e. long-term risk reduction. • Integration: As so often is the case, opportunities are missed due to a lack of integration between programs. 7 • In all development and humanitarian projects, there is a need for conscious and systematic integration of accountability principles in all actors at all levels, and provision of tools for dialogue and engagement with service providers needs to be in place. Gender:  Many OFDA and FFP-funded programs target women; nevertheless, women have not always benefited fully from the programs. Even when a lot of programs focus on women, women do not take advantage of these programs. They do not take advantage of repeat visits, as they do not have the same mobility as males. Women tend to be less informed about how most programs are designed or targeted. Eighty-four percent of women said that they had not been communicated with by IPs, as opposed to 17 percent of men surveyed, in the large-sample survey.  Key informants from the IPs did not confirm the use of any gender marker system/matrix to track gender balance and participation in WASH activities. Beneficiaries are not informed by IPs about upcoming development projects and beneficiaries’ selection processes are not always inclusive of community members.  From the evidence reviewed, there was a notable lack of attention to reproductive health.  IPs should establish clear theories of change about what works in involving women in activities beyond merely receipt of services. The WMCs include women in these committees only when the IP pressed for their participation as a requirement of the project. Further research should ask how effective this is for scaling up, replication, or long-term change. PORTFOLIO LEVEL RECOMMENDATIONS Decisions to Inform: Inform about transparency, accountability and effectiveness of beneficiary selection methodology and implementation mechanism; also assess and compare effectiveness of the different outreach (targeting) approaches of IPs and challenges encountered.  OFDA and FFP should require, and IPs need, to put in place effective feedback/ grievance/complaint channels.  While IPs have selection criteria that target the neediest, within that large mass are even more marginalized groups like the elderly, disabled, and youth. OFDA and FFP should ask that IPs take measures to include them in programming, and IPs then should monitor progress.  For food distribution, the house-to-house validation process seems to encourage dialogue, leads to more accurate beneficiary lists and is appreciated by the target populations. OFDA and FFP should continue to support community-based methods to identify and validate beneficiaries.  The weaker government systems have resulted in the beneficiaries becoming dependent on the goodwill of the service providers, i.e. the health workers in the facilities, and local leaders managing the FDPs. In turn, a lack of transparency and accountability systems have been reported. OFDA and FFP should encourage wider use by IPs of local accountability structures, such as committees, or grievance mechanisms. Decisions to Inform: The appropriateness of technical programming by sector to accomplish sector objectives and convey potential gaps that became evident as a result of the current sector programming. 8  Both OFDA and FFP are primarily supporting relief modalities, and the transition into recovery or resilience programming is either slow or not evident. Newer projects that have a resilience objective should be monitored against refined resilience indicators.  While projects are effective, they are not achieving the scale to meet the needs of the vulnerable. This suggests possibly re-thinking portfolio funding targets and proportions: fewer targets with more impact, as in the MRE sector.  OFDA and FFP Program Guidelines should incorporate principles for shifting the focus of development, from needs-servicing to building the capacity of individuals and communities to understand, claim and fulfil their entitlements, to be integrated systematically.  OFDA and FFP should require that IPs strive for gender balance in program implementation and operation, which they and YOFMEP should monitor.  IPs (with OFDA and FFP support) need to plan for and spend sufficient time in the initial phases of projects to raise the level of awareness of the community on what the project is about. Decisions to Inform: Lessons learned and best practices of IPs implementation and coordination processes.  Donors can support national-level interagency sharing. The monthly sectoral United Nations (UN) cluster meetings allow IPs to exchange updates on who is doing what and where, and help to avoid duplication of efforts and wastage of resources. These meetings function better in the field than in the national capital.  Monthly joint supervision among the cluster, donors and MPHP can facilitate whether those in need are reached and humanitarian workers take actions in the earliest time possible.  OFDA and FFP can promote best practices seen in specific areas by IPs. Projects that are well coordinated with local communities – such as those of MC, ADRA, and others - serve as replicable examples of processes that may ensure sustainability. Coordination can include beneficiary selection and program elements such as identification of community assets for rehabilitation or food for work (FFW) or food for assets (FFA) projects.  IPs are responding to a sustainability concern by building the capacity of local indigenous NGOs who can access and stay in the areas of implementation longer, putting different innovative and technologically advanced tools in place for monitoring of project activities, hiring temporary staff, and using mobile teams for outreach. OFDA/FFP should encourage this through knowledge sharing about how to best do it.  IPs have experimented with various ways to mitigate capacity challenges of government partners, with mixed results. OFDA/FFP might encourage knowledge sharing about these experiences, and those of other development partners, to consider what’s working, what’s not, and viable strategies. 9 1. INTRODUCTION EVALUATION BACKGROUND OFDA and FFP contracted International Business & Technical Consultants, Inc. (IBTCI) in October 2013 to implement YOFMEP in collaboration with IBTCI’s local subcontractor, Prodigy Systems. The primary objective of YOFMEP is to monitor and evaluate ongoing humanitarian projects in Yemen that are funded by OFDA and FFP, which both are under DCHA at USAID. As the dangerous security risks in Yemen limit OFDA’s and FFP’s own direct access to the many parts of the country where they might observe IPs in the conduct of these projects, YOFMEP provides a solution in the form of third party monitoring (TPM) and evaluation (M&E) services, which report back to OFDA and FFP. This is described in the Statement of Work (SOW), available in Annex I. On an ongoing basis IBTCI, as the implementer of YOFMEP, collects data about and verifies selected humanitarian activities that are supported by OFDA and FFP across Yemen. The mandate of YOFMEP is to monitor the progress of these activities and to evaluate the effectiveness and sustainability of the portfolios of OFDA and FFP in Yemen. These portfolios address immediate needs in: WASH; nutrition, health, logistics and relief commodities; humanitarian information and information management; protection, ERMS; and agriculture, food security, and food assistance. While OFDA’s and FFP’s IPs maintain responsibility for monitoring their own activities, YOFMEP provides an independent M&E mechanism. The results of these supplementary M&E support services assist OFDA and FFP in their ongoing programming decision-making and future program design. The M&E contract stipulated that IBTCI would conduct at least two evaluations: a first Formative Evaluation set early in the contractual period, and a second to allow for some comparison and estimation of results. The Formative Evaluation was designed to be somewhat shorter in duration, with deliverables to be completed near the midpoint of the contract (April - May 2014). The Summative Evaluation will be more substantial and will gather data toward the end of 2014. FORMATIVE EVALUATION PURPOSE Assessing the effectiveness of OFDA and FFP’s implementation processes is the main concern of this Formative Evaluation. It is intended to be a short-term evaluation to give OFDA, FFP, and their IPs the opportunity to actively engage with the findings to guide any needed mid-course corrections. While encouraging learning, the evaluation is not intended to require adjustments to current project designs. Instead, the aim of this Formative Evaluation is to appraise the effectiveness, sustainability, and implementation process of USAID’s portfolio of investments so as to inform the design of future USAID programming in Yemen. EVALUATION LINES OF INQUIRY: TARGET IPs AND APPROACH Objective: Effectiveness. The SOW stated that the contractor would evaluate the effectiveness of OFDA and FFP programs in Yemen, along the following lines of inquiry: 1. What is the project doing to ensure that it reaches the beneficiaries most in need? What are the challenges to reaching the beneficiaries most in need? How have IPs overcome those challenges? 10 2. Are there significant needs within OFDA and FFP’s operational sectors that are not currently being addressed or met by OFDA and FFP programming? 3. What are the most significant lessons learned about the implementation process of OFDA and FFP programming from the monitoring process to date? What are the most significant strengths of and areas of improvement for OFDA and FFP programming? These three lines of inquiry will appear repeatedly in the presentation below as the framework for the clustering of findings, conclusions and recommendations. OFDA & FFP STRATEGY IN YEMEN The USG has spent almost $200 million in humanitarian and stabilization funding in Yemen during the past two years.1 The USG has been responsible for 38 percent of all donor humanitarian funding to Yemen during 2013-2014. But the persistent Yemeni conflict and population displacements began to subside somewhat from 2012 to 2013, and local authorities began to open new geographic areas for access by humanitarian aid agencies, greatly facilitating humanitarian action to meet unmet needs. In response to large scale population returns, humanitarian assistance shifted in scope to include more “early recovery” activities such as mine clearance, agriculture, non-agricultural livelihood recovery, infrastructure repair, and capacity building. As well, the IPs have been able to conduct more frequent and more geographically inclusive assessments to improve their targeting of vulnerable groups and allow for the refinement of their response efforts. Program coordination and integration were enhanced and refined. Despite improvements in the political climate in 2013, Yemen remains a humanitarian crisis. The UN Office for the Coordination of Humanitarian Affairs (OCHA) estimates that out a population of 25 million, 15 million (58 percent) need humanitarian assistance due a combination of conflict, displacement, economic setbacks, high food prices, food scarcity, landmine risks and refugees from neighboring countries.2 The UN further estimates that 10.5 million people are food insecure and 4.5 million are severely food insecure in Yemen. Among the food insecure, more than one million children under five years of age suffer from acute (wasting) malnutrition, more than one quarter of who suffer from life￾threatening severe malnutrition.3 Yemen continues to receive flows of refugees and migrants from neighboring countries, particularly Somalia. Following a change in labor policy, nearly one half million Yemeni laborers, many with their families, were forcibly returned, unemployed, to Yemen from Saudi Arabia in mid-2013. More expulsions are expected in 2014, creating economic pressures in crisis zones. 1 The US Ambassador to Yemen re-issued a formal disaster declaration for the country in September 2013, covering fiscal year 2014. 2 OCHA (Feb, 2014). 2014 Strategic Response Plan: Yemen Humanitarian Response Plan for Jan 2014 to Dec 2015. 3 Defined as the child falling three standard deviations below the mean weight for height (WfH) based on a cohort reference table. Children with severe acute malnutrition have the highest relative risk of death of any group in emergencies. 11 At the time of this report, much of the country is in the process of transitioning away from violent conflict toward recovery.4 The political turmoil in southern Yemen has abated in recent years. Consequently nearly 95 percent of the internally displaced from this area have returned to their areas of origin and begun the process of reintegration. Infrastructure in those areas was damaged (i.e. homes, irrigation systems, wells, terraces and other buildings), personal property looted, and livestock stolen or killed during the conflict. In the near term, rebuilding, rehabilitation and restocking are needed. Meanwhile, conflict continues in parts of Yemen, including the north where Houthi rebel groups fight.5 Consequently, the humanitarian aid networks in Yemen are gradually shifting from emergency assistance distributed without conditionality to more narrowly targeted and conditional assistance that can restore community infrastructure and household livelihoods and improve market mechanisms with greater attention toward sustainability. The humanitarian community aims to “build back better” and promote more resilient households and communities. NGOs are advised by OCHA to pursue these strategic objectives: “Provide effective and timely life-saving assistance to the most vulnerable people in Yemen. Assist and protect people affected by crisis, including refugees, migrants and returning Yemenis. Strengthen the capacity of national actors to plan for and respond to humanitarian emergencies. With development partners, including the Government of Yemen, address the underlying causes of vulnerability to reduce the need for continued humanitarian assistance and increase resilience. Ensure meaningful participation and equitable access to services, resources, and protection measures for women, girls, boys, and men.” The Formative Evaluation took into account the five elements of OCHA’s strategic response plan. Consistent with OCHA and the international humanitarian community, USAID has sought to reorient its program from emergency food assistance to a resilience-based approach. For USAID, resilience is defined as the “…ability of people, households, communities, countries, and systems to mitigate, adapt to, and recover from shocks and stresses in a manner that reduces chronic vulnerability and facilitates inclusive growth.”6 USAID’s characterization of the drivers of food insecurity in Yemen includes: “…internal conflict and political instability, high food and fuel prices, rising levels of poverty and the effects of climate change” …as well as “cultural factors including hygiene, and infant and child feeding practices.”7 A December 2013 United Nations Development Programme (UNDP) livelihood assessment found that insecurity, road blocks, banditry, high prices of productive resources (water included) and market closures each affected more than 30 percent of the rural population in conflict affected areas.8 UNICEF’s Social Protection Monitoring panel survey found a strong correlation at the household level of conflict and food insecurity. 4 Yemen has suffered conflicts for decades. Notable milestones were the revolution of 1962 and the unification of the independent states of North Yemen and South Yemen in 1990. 5 The Houthi or Huthis in northern Yemen have sought to expand their territorial influence. The Huthis are allied with nearby tribes associated with Saleh’s General People’s Congress. In recent years the conflict has been cyclical. See: June 2014 The Huthis: From Saada to Sana’a, The International Crisis Group (ICG). 6 USAID, 2013 Building Resilience to Recurrent Crisis: USAID Policy and Program Guidance, Washington, DC. 7 Amendment No. 01. USAID/DCHA/FFP Annual Program Statement No. FFP-13-000001. International Emergency Food Assistance. 8 UNDP, Dec 2013. Multidimensional Livelihoods Assessment in Conflict Areas of Yemen: Integrated Summary Report. Sana’a. 12 During the 2014 Fiscal Year (FY), USAID programs in Yemen have evolved and continue to evolve from free distribution of general food rations to an increased distribution of vouchers (i.e. Vouchers for Assets, or VFA, to be redeemed for food),9 following the USAID/DCHA guidelines: Proposed interventions with planned linkages to USAID agriculture and food security programming should make receipt of food vouchers conditional upon participation in Food for Assets (FFA) activities coordinated with USAID initiatives to construct or to rehabilitate small‐scale infrastructure related to rainwater harvesting in Sana’a, Dhamar, Raymah, Ibb, Ta’izz, Al Dhale’e and Lahij. Proposed activities with planned linkages to USAID nutrition programming should make receipt of food vouchers conditional upon beneficiary participation in behavior‐change activities. These include either infant and child feeding practices or health care incentive programming coordinated with USAID’s Yemen Maternal Child Health Integrated Program (MCHIP) initiative, currently being implemented in the governorates of Sana’a and Dhamar, and/or with Yemen’s Scaling Up Nutrition (SUN) movement, in which USAID is a key stakeholder.10 The programmatic shift from relief to recovery aims to augment community and household resilience to natural and man-made shocks, rebuild more durable livelihoods, and improve nutritional and health status 9 Vouchers for food and/or agriculture and vocational inputs. 10 Amendment No. 01. USAID/DCHA/FFP Annual Program Statement No. FFP-13-000001. International Emergency Food Assistance. Figure 1: Active USG Humanitarian Programs in Yemen 13 while maintaining appropriate food consumption levels through conditional food vouchers that support local market response and recovery. In addition, all proposed interventions are expected to:  Recognize and reinforce existing Republic of Yemen Government (RoYG) systems that are developing to be able to address food insecurity, agriculture, livelihoods, and nutrition needs, and provide safety nets for the most vulnerable.  Leverage the participation of relevant national ministries (Ministry of Agriculture and Irrigation (MAI), Ministry of Planning and International Cooperation (MOPIC), MPHP, Ministry of Social Affairs and Labor (MOSAL)).  Strengthen Yemen’s social contracts under the SWF and Social Fund for Development (SFD) at different levels of governance.  Use up-to-date, data-driven lists of the most vulnerable households. The strategic themes for OFDA in FY 2014 include these elements: 1. Maintain the capacity to quickly respond to any new displacements, address the needs of IDPs living in camps, and provide support to returnees. OFDA will support multi-sector programming that establishes early recovery and builds resilience to achieve these aims. 2. Coordinate with the USAID Mission in Yemen on water and nutrition, two of the Mission’s cross cutting issues, as well as leverage Development Objective 2, which deals with social development. 3. Continue to support TMP and establish a Famine Early Warning Systems Network (FEWS NET) office in Sana’a in order to reach inaccessible areas. 4. Remain flexible and able to provide assistance in the dynamic environment of Yemen. 5. Remain committed to efforts that mainstream protection in all programming. In identifying target groups, FFP uses the SWF lists11 as the initial step in the identification of beneficiaries for Title II food assistance. The Yemeni government unit responsible for management and administration of the national social safety net program created these SWF lists, drawing on poverty measures. The lists are used to identify geographic areas and, where possible, begin the process of beneficiary selection. In collaboration with the communities where they are working, IPs were tasked to update and validate the lists. USAID/FFP supports ADRA, GC, SCI, and MC for the implementation of community mobilization and for behavior change communication (BCC) outreach. These programs are in their first year of implementation. Also supported by FFP, the WFP has been implementing large scale, unconditional food distributions. But, like the NGOs, in mid-2014 (approximately July) WFP is due to begin a shift to conditional distribution tied to FFW, FFA and food for training (FFT), along with added safety-net components. Overall, WFP has been assisting 3.8 million people as part of its Emergency Safety Net (ESN) program. SCI also implements a substantial WASH program, EFSP. ADRA and GC implement agriculture/food security and ERMS programs with economic recovery and community resilience components. The distribution of vouchers that can be redeemed for food and vocational toolkits with the purchase of locally sourced food and other goods is meant to stimulate local market recovery and growth. 11 There is one national list but area specific lists are being used by the IPs so the word “lists” is used. 14 USAID/OFDA supports ADRA, GC, IMC, IOM, IRD, SCI, and UNICEF. At the time of the formative evaluation, the IPs that had completed a full year of programming were IMC, SCI and UNICEF. Three OFDA programs have Agriculture and Food Security, and ERMS as primary objectives – ADRA, GC, and IOM. Health, nutrition and WASH were the sector emphases for IMC, IRD, SCI, and UNICEF. In addition to nutrition and WASH, UNICEF is the only IP addressing MRE – the Protection Sector – with OFDA support. OFDA uses the SFD household vulnerability lists as the initial step in the identification of OFDA target beneficiaries. These SFD lists are created and managed by a semi-governmental body, which uses nutritional status as a determination of inclusion. OFDA primarily funds activities in the sectors of WASH, Health and Nutrition (H/N), agriculture/food security and ERMS. For these, they fund GC, IMC, IOM, SCI and UNICEF to implement programs to rehabilitate water infrastructure, such as wells, catchment structures and irrigation systems. In addition, these IPs provide training in improved sanitation and hygiene practices and management of water systems. UNICEF, SCI, and IMC also implement H/N interventions in conjunction with the MPHP. Infrastructure rehabilitation activities create desired temporary employment while improving the basis for longer-term livelihood options that are reliant on water, especially agriculture and animal husbandry. ADRA, GC and IOM support the restoration of livelihoods primarily with the distribution of small ruminants and/or chickens, creation of keyhole gardens, vocational and veterinary training, and provision of business startup kits. OFDA continues to support UNICEF’s protection activities and humanitarian coordination and communication as well. FORMATIVE EVALUATION SELECTION CHOICES: TARGET IMPLEMENTING PARTNERS & SECTORS In 2012, OFDA and FFP funded approximately $100 million of humanitarian programs in Yemen. When the Formative Evaluation planning began in November 2013, OFDA and FFP had contributed an additional $100 million to the humanitarian response in the country. For the purpose of M&E, OFDA and FFP selected the specific projects and partners to target within YOFMEP M&E activities. The tables in Annex IV catalogue most of OFDA and FFP funded projects in Yemen, which – except for IMC, SCF, UNICEF and WFP – had recently completed the mobilization phase and were beginning activities in the first quarter of calendar year (CY) 2013. Therefore, at the time of the Formative Evaluation field data collection in March - April 2014, only IMC, SCF, UNICEF and WFP had beneficiaries (the end recipients of assistance, whether nutrition, health, training, awareness or site rehabilitation) to be included in the Formative Evaluation assessment. Thus, OFDA had more projects that had completed a 12-month project funding cycle, than did FFP, though FFP’s assistance had reached a larger number of end beneficiaries, given the size of the WFP coverage. In preparing the Formative Evaluation, IBTCI found that the majority of projects funded by OFDA and FFP each had objectives in more than one sector. For the purpose of assigning the evaluation’s Technical Sector Specialists, and for analysis purposes, the team assigned each project to a sector according to the US level of investment, relative to all projects funded. The levels of USAID investment in Yemen, by sector, are depicted in Figure 17 in Section 4 Conclusions and Recommendations. 15 THE FORMATIVE EVALUATION APPROACH A. Portfolio Analysis Approach Discussions with OFDA/FFP confirmed the need for an evaluation approach that leads to actionable recommendations on the portfolio levels of OFDA and FFP assistance in Yemen. The evaluation design combined data collected from IP projects individually and then considered the entire dataset to generate findings to formulate conclusions and recommendations at the portfolio level, for FFP and OFDA. In order to inform future decisions about the selection and design of interventions in Yemen, IBTCI’s contribution will allow both FFP and OFDA to gain a better understanding of the comparative effectiveness of the projects and procedures it has supported. B. Utilization Focused Evaluation The Formative Evaluation approach applies a U-FE approach. U-FE is a process for making decisions related to complex programs in collaboration with the primary users of the evaluation findings, focusing on the intended uses of the evaluation. The M&E contract states that the intended users are USAID OFDA and FFP managers in the field and in Washington, D.C. Therefore, the Formative Evaluation took a highly iterative process of consultations with the Yemen based Senior Humanitarian Advisor (SHA) and OFDA and FFP managers through each phase of the evaluation design, planning and implementation. This was written for the audience of OFDA and FFP and, secondarily, the IPs. C. Gender Consideration Conforming to USAID Evaluation Policy, the Formative Evaluation, including the design of field questions and how they were conducted, as well as the analysis took gender disaggregation and gender issues into account wherever possible.. The evaluation aimed to observe the effects of the interventions from the perspectives of women and men, both, as well as of girls and boys – including how the differences have a positive or negative impact on project objectives. The field data collectors of Prodigy Systems – IBTCI’s subcontractor – included female professional staff to ensure access to all community strata and to facilitate the meeting and interviewing of female beneficiaries, while respecting Yemeni cultural norms. There were additional measures used to ensure that male community leaders were informed of the field assessment objectives and the importance of inclusion of women in the surveyed population. Only women enumerators interviewed women; focus group meetings were often gender-specific and private where necessary to insure confidentiality of the information exchanged and limit the influence of men. Moreover, in consideration of the limited literacy of the women target population, the field enumerators read the interview/survey questions to the survey participants to elaborate on the objective of each question and to seek informed responses to the evaluation’s lines of inquiry. The evaluation analysis is disaggregated by gender wherever appropriate. D. Do No Harm IBTCI took steps to ensure that the evaluation assessment would ‘do no harm’ to beneficiaries or to IPs’ activities and their relationship with the community, beneficiaries, partners and government stakeholders. IBTCI coordinated with IPs to inform the target stakeholders (ministries, local governors, local councils, community leaders and beneficiaries) of data collection in advance and its purpose ‘to evaluate in order to learn and improve delivery of the humanitarian assistance.’ Coordination and consultations stopped short of IPs’ participation in the field assessment, to steer clear of potential influence on stakeholders’ opinions. Additionally, IBTCI’s evaluation approach incorporated the basic ‘Do No Harm’ evaluation ethics where participants in the evaluation engage willingly without fear of penalty (possible exclusion), and are fully informed of the evaluation’s purpose. Care has been taken in the evaluation reporting to not mention beneficiaries by name. 16 E. Conflict & Context Awareness The humanitarian assistance funded by OFDA and FFP through IPs’ projects are implemented in diverse geographic regions of Yemen where the socio- political ‘implementation context’ varies widely. Intrinsic to the predominately-Yemeni staff of IBTCI and Prodigy Systems is their awareness of the differences among the regions’ socio-political and cultural dynamics and levels of conflict. While maintaining objectivity and neutrality, these differences needed to be anticipated. For instance, due to local conflict, some locations initially selected for the survey sample had to be replaced with others. In support of this and the Do No Harm concepts, IBTCI drew from USAID’s guidance on complexity-aware monitoring.12 F. Communication and Coordination with the Implementing Partners The evaluation team liaised continuously with the IPs to plan and implement the Formative Evaluation. IPs proved to be cooperative and accommodating with regard to their provision of activity records and notification of their stakeholders of the Formative Evaluation activities, even when these requests put additional pressure on the IPs’ workloads. An early challenge in obtaining beneficiary lists from IPs was a key issue that was later solved (see limitations section below). G. Building on Monitoring Data The evaluation activities built on YOFMEP monitoring data of IP programs. The Evaluation Project Specialist reviewed the lines of inquiry of the Formative and Summative Evaluations with the monitoring team early in the contract; the monitoring team then aligned some of the data to collect during site visit verification and output monitoring. The evaluation activities make use of monitoring reports and data quality assessments as objective, external data to guide and support its analysis of the lines of inquiry. H. Reinforcing Local Capacity The YOFMEP implementation design by IBTCI, including both Formative and Summative Evaluations, are consistent with the USAID Evaluation Policy that requires: “aims of capacity building and respectful engagement with all partners [be achieved] by involving appropriate expertise from partner countries.” The IBTCI evaluation design approved by OFDA and FFP required that IPs provide contacts with all stakeholder groups in Yemen, engaging them actively throughout the inquiry and exposing them to best practices in M&E. The YOFMEP subcontractor, Prodigy Systems, a highly qualified research firm, provided the planning and execution of the complex and challenging quantitative and qualitative data collection requirements of the Formative Evaluation. 12 “Complexity-aware monitoring is appropriate for aspects of strategies or projects where cause and effect relationships are poorly understood, thereby making it difficult to identify solutions….” In USAID 2013 Discussion Note: Complexity Aware Monitoring. 17 2. EVALUATION METHODS AND LIMITATIONS Overall, IBTCI used a mixed-methods approach, drawing on available data and conducting a battery of original research at field sites. All in all, the evaluation encompassed a wide array of agencies and interventions, large and small, each with their own theories of change, timelines, assumptions and matching resources. The portfolio level approach described in the introduction was instrumental in determining the evaluation sample for the methodology for this Formative Evaluation. The Formative Evaluation encompasses:  Nine IPs: ADRA, GC/CHF,13 IMC, IRD, SCI, MC (five NGOs), IOM, UNICEF and WFP (three International Organizations).  12 projects: at the time that the evaluation’s data collection began in March 2014, eight IPs were just recently past their mobilization phase and starting activities, while four had been operational for more than nine months and reporting the numbers of their beneficiaries or participants reached.  The main sectors of assistance that informed this evaluation were: H/N, WASH, Protection (education for landmine awareness), and agriculture/food security.  OFDA and/or FFP supported project activities are being implemented in 18 governorates: Abyan, Lahj, Aden, Taiz, Ibb, Sana’a, Hadramawt, Al Maharah, Sa’ada, Hodeidah, Al Bayda, Al Dhalee, Al Mahwait, Amran, Dhamar, Hajja, Mareb, Raymah; the UNICEF support projects are countrywide. Evaluation Timetable: YOFMEP staff submitted and OFDA/FFP approved the Evaluation Plan in December 2013. A dialogue with both OFDA and FFP over alternate options for sampling and sample sizes continued until March, 2014. The technical team was in country in late February and early March, 2014 to conduct the initial interviews, complete the instrument design, and work with YOFMEP Monitoring Specialists and Prodigy Systems to plan the larger part of the fieldwork. The evaluation subcontractor, Prodigy Systems, conducted field research surveys, FGDs, and SIs from in March and April, 2014. The evaluation team presented the findings and submitted the draft report a month later, after translating the Arabic qualitative data and analyzing and synthesizing the findings. Lines of Inquiry, Indicators and Decisions to Inform: The IBTCI/YOFMEP Scope of Work specified the three main lines of inquiry (cited above in the introduction) for the Formative Evaluation. The table in Annex II associates each line of inquiry with a corresponding set of assessment questions, indicators, data sources, and the OFDA and FFP decisions that the Formative Evaluation seeks to inform. This chart was the basis for the preparation of the evaluation tools (all tools are in Annex VI.) The lines of inquiry and sub-questions focus on the specific activities and outputs of the IPs and the patterns of recovery observed among their beneficiaries. However, the Formative Evaluation did not seek to explore other 13 Global Communities (GC) is often still known by their former name, CHF, originally standing for the “Cooperative Housing Foundation.” 18 forces or funding streams, though it might be feasible to blend the data in the future to estimate the ‘contribution to change’ of the activities.14 Methodology: The evaluation applied quantitative and qualitative research methods and tools to the evaluation lines of inquiry and with stakeholders’ groups that were consulted in the evaluation process. The evaluation research instruments included SIs, focus groups meetings, site visits (observation), and quantitative household beneficiary surveys. The evaluation instruments were designed and developed to respond to the evaluation questions and to collect data on the proposed indicators under each line of inquiry. The quantitative instrument-surveys targeted the end beneficiaries of OFDA/FFP assistance, while the qualitative tools – SIs and focus group meetings – were addressed to the other stakeholders groups as listed in the table in Annex II-A. That table shows the number of interviews associated with each of the stakeholder groups, which included IP main office and project staff, government counterparts, local IPs, other donors, and beneficiaries with implementing roles, such as committee members. Data Sources: Introductory meetings with OFDA/FFP IPs and desk review of the IP project reporting and documents facilitated the identification of the stakeholders’ groups (see data sources listed in Annex II-A). IBTCI requested from IPs and collected the following detailed information: (1) the name, location and contacts for the stakeholders relevant to each project, (2) beneficiaries/trainees lists, location and contacts relevant to the proposed Formative Evaluation sample framework, and (3) site locations, particularly in the cases of water infrastructure and rehabilitation. The data sources and tools included: Focus Groups & SIs: SIs and focus group meetings were conducted with the relevant stakeholders of the 12 projects covered under the Formative Evaluation as identified in Annex II-A. Site Visits: Site visits were conducted for selected water infrastructure projects and sites where WASH facilities were being rehabilitated, as in IDP return areas. Surveys: Covered six of the 18 governorates targeted through IP programs and obtained evidence directly from the beneficiaries of IMC, SCI, UNICEF and WFP. The projects’ main sector(s) of assistance were considered when selecting the survey sample of beneficiaries (end recipients), such as WFP (Food distribution), UNICEF (WASH and MRE), IMC (H/N), and SCI (H/N). Reaching out to the beneficiaries (end-recipients) was conducted through the main focal points of the IPs’ assistance (i.e. for IMC and SCI through the health dispensaries, for UNICEF through their IPs (YEMAC and NGOs), and for WFP within the areas/regions neighboring the points of distribution). In summary, the “portfolio of assistance” perspective guided the evaluation’s identification and selection of the sectors and intermediate stakeholders and ultimate beneficiaries to incorporate all of the groups needed for this consultation along the lines of inquiry of the Formative Evaluation. The local target geographic area is defined as the local enumeration area, or “EA.” The Central Statistics Organization (CSO) of Yemen defines an EA as a geographic settlement in which 80-180 families live together and can be classified as an urban or rural area. SAMPLE FRAME and SITE SELECTION 14 See: Roger Few, Daniel McaVoy et al. 2014 Contribution to Change: An approach to Evaluating the Role of Intervention in Disaster Recovery. GB: Oxfam. 19 The goal of the sample frame was to use randomization to achieve representation. Sample size alone is not as effective at achieving representativeness as the use of randomized selection across clusters or, as is the case here, geographic EAs.15 In effect, the sampling approach was a modified, two-stage random cluster sample. The sample of the Formative Evaluation was derived using a PPS approach, which was deemed appropriate in order to have a representative sample. The sampling expert and statistician consultant calculated the number of beneficiaries to achieve this target point and the nature of the programs in consideration and applied the statistical principle that for very large numbers in the underlying population, significance/power is reached at a certain plateau level. Thus the sample strategy was to reach a sample of sufficient size by sector to draw fairly evidence-based conclusions in relation to the lines of inquiry of the Formative Evaluation, based on process assessment. The total claimed beneficiary population of the two UN IPs - WFP and UNICEF - which had completed 12 month implementation cycles, was over four million each. Another roughly 800,000 more were added from IMC and SCI beneficiary totals. The calculation for the sample size was as follows: a. Total number of intervention districts = 333 (all EAs in Yemen). b. The Sample Strategy was to capture the variety of types of programs and target populations through Population Proportional Sampling (PPS). c. The target Confidence level for the calculations was 95 percent. d. The Confidence interval was 10 percent (~ 0.10312). e. The Standard Error was five percent (~ 0.05261). Based on the above parameters, the sample size computed to be 72 randomly selected EAs, with 12 households to be sampled within each EA, which is the same as a district. To be on the safe side, to increase the inclusion of different distribution areas and counter the “design effect,” the sample was expanded to 74 EAs, with the resulting total of 888 households, thus allowing for at least a 95 percent level of confidence, an acceptable margin of error of five percent, based on an estimated non-response rate of 10 percent. The EAs were selected in proportion to the beneficiary numbers funded per sector, and the number of EAs where sector activities were implemented. It was understood that it would be a statistical inevitability that some households in some EAs would benefit from activities in more than a single sector. But the beneficiary household was counted just once, while capturing the number of sector benefits received. Stratification allowed the process to ensure inclusion of the project areas of interest, such as rural populations, or where there are cultural differences. Through the first round of random selection of EAs, communities with these priority attributes had an equal chance of being represented. Then, after randomly selecting the EAs, those communities with operational OFDA and FFP projects and project￾created committees, were included for qualitative data collection.16 Within each EA, the sampling units were the household (HH) and the individual. A target of 12 households was surveyed within each EA. Following the training of the field enumerators on the data collection 15 W Paul Vogt, Dianne Gardner, Lynn Haeffele. 2012 When to Use What Research Design. NY: The Guilford Press. 16 Early consideration was given to employing lot quality assurance sampling (LQAS) methods, which has advantages for small sets of real-time management questions about coverage or outcomes, but is not as effective at providing point estimates or reliable statistical distributions for the many variables of interest. 20 instruments, Prodigy Systems pilot tested a small sample of respondents to assess the instruments and make corrections to the instruments. Table 1: Numbers of Persons Surveyed Tool Minimum target sample17 Completed Percent female Health & Nutrition Beneficiary Interviews conducted 117 126 94% Heath Center Worker Interviews conducted 20 20 30% Food Distribution Interviews conducted 978 1,194 60% WASH Interviews conducted 79 98 84% MRE Student Interviews conducted 27 40 40% MRE Teacher Interviews conducted 7 14 29% Total 1,228 1,492 In the end, the initial sample numbers were increased (32.6 percent) proportionately to each category of the supported project. The total sampling units were 926, which included 10 percent of the non￾respondent. The CSO of Yemen estimates that the average household size in Yemen is 7.14. This was taken as the coefficient for estimating the overall beneficiary population. Thus the completed sample of 1,492 interviewed individuals represent, through their households, a total population of roughly at least 10,650 men, women and children. This is a conservative estimate, in recognition that it is commonly known that the more rural and the lower income the household, the larger is the average household size. Among target households, the average household size probably is more than eight persons. Therefore, the total beneficiary population included in the survey probably exceeds 10,650 and would be closer to 12,000. A smaller sample size would have reduced the confidence degree and increased the sampling error level. The limitation of this strategy was that the smaller sub-programs may be under-represented, such as the MRE sub-project of UNICEF. The solution was to triangulate with the qualitative data. With intentional oversampling, the overall statistical power of the Formative Evaluation is greater than the minimum requirement. Original survey data was collected by a number of mobile teams, through Prodigy Systems. Program beneficiaries were interviewed at their homes, not at the distribution point. Qualitative Data Collection: SIs and FGDs were valuable in that discussants were able to go into greater depth by explaining and providing examples. There were 57 SIs and 33 FGDs. In addition, there were extensive interviews with 49 IP staffers. IP staff were useful for explaining what activities were undertaken, while beneficiary interviews elucidated the benefits received and ongoing barriers they face. For the qualitative surveys, all 12 OFDA and FFP projects that had been identified by OFDA and FFP in the sampling plan were included. 17 This was the calculated goal to achieve the statistical power mentioned earlier, with an extra 10 percent increase on top of that. In each case, the completed sample equaled or exceeded the target + 10 percent.. 21 To gain evidence from beneficiaries in areas not included in the quantitative survey, the eight target projects were targeted through 46 SIs and 17 focus groups; this is a purposive and representative sampling to focus on those areas where community committees are already formed and operational. The selection of projects sites for the qualitative evaluation was from the 74 EAs sample of the quantitative evaluation where the generated list of the 74 EAs was used as a sample frame for the qualitative evaluation. The community committees were formed in some of the programs and not in all of them based on each program’s design. To accommodate this, using a matrix of the 74 EAs and communities where committees were formed, the statistician matched sites with formed community committees and created a list of project sites to be all visited for the qualitative evaluation. A map of the data collection locations is at the end of this chapter. Gender Consideration: In planning for data collection, there were several measures intended to overcome bias against women’s participation. The majority of interviewers were women, and training specifically addressed requesting that women participate in FGDs and other interviews. The interviewers – all Yemenis – were knowledgeable about cultural norms such as arranging separate spaces to interview women. Nevertheless, despite these measures, there were more than twice as many men interviewed as women: 102 versus 37. IPs had more men than women program staff responsible for project implementation. OFDA and FFP Representation, Double-Counting, Marginal Groups: OFDA and FFP give support to specific projects that were readily identified in the sample selection process. Yet, there appears to be some double-counting in the process of the randomized selection of EAs, in that there were some households who received benefits from more than one IP or sector. The interviewers were trained to use each sector tool that applied. Therefore the beneficiary HH could be counted once and indicate the number of benefits they received. The randomized selection process ensured an equal (inclusive) opportunity for the representation of remote project sites and for remote populations, including IDPs and marginalized groups with cultural differences. Then, after randomly selecting EAs, communities with operational OFDA and FFP projects and committees and projects, were included for qualitative data collection. Records from the many interviews, largely conducted in Arabic, were later translated to English. The data quality was found to be excellent in terms of completeness and consistency. The surveys and FGDs speak louder than the secondary data (project monitoring plans or the literature). While the expatriate evaluation team was unable to travel to speak directly with beneficiaries, the trained Yemeni interviewers did so. Means of Analysis: The first steps of the data analysis occurred among field partners, including Prodigy Systems. Geographic locations of the field assessment research were plotted on MAPs through Geographic Information Systems (GIS) tools, particularly Arc-Info (from ESRI Corporation) and incorporated the locations into all of the analysis. The statistical analysis correlated data by project, by region, by sector, and according to whether the funding was from OFDA or from FFP. The team’s data cleaning, review and synthesis examined the evidence and compared it for its validity, reliability, precision and generalizability. Data verification occurred to a large degree through triangulation – collecting and comparing data on the same questions from different stakeholders and using different methods. Another means of verification was Prodigy Systems’ method of tracking the position of data collectors and direct entry of data into an online system, thus reducing data entry errors, where a supervisor checked the data for completeness 22 and accuracy on a real-time basis. Later when technical experts had queries, Prodigy Systems supervisors re-checked the data to explain the reason and made adjustments where warranted. The final team analysis has been oriented to weigh the levels of evidence against the alternative answers under the lines of inquiry framed by their usefulness to OFDA and FFP, using Utilization-Focused distinctions among findings, outcomes, goal clarification.18 The technical experts analyzed and synthesized their respective mixed-method data for their respective sectors, and then shared their results among one another and began a larger synthesis across sectors. The sample frame adopted did not include sufficient sample sizes, or power, in specific areas to be able to generalize about the effect size in each, or compare them. But where patterns are seen, this can inform follow-up inquiry with greater power.19 To be clear, the Formative Evaluation was not primarily intended to estimate the impact or results of programs, but to identify important patterns of achievements as well as unmet needs. As well, it was crafted to discern patterns of program progress affecting success or bottlenecks. These can be further explored in the Summative Evaluation. The analysis addressed issues that spanned from the micro (how individual children are tracked in health care), to the facility level, the agency level, and the national inter-agency coordination level. Evidence was teased out about the effects and program lessons at each level. The broad, representative spread of the data collection sites, both north and south and rural and urban, is seen in the map in Figure 2. 18 Michael Quinn Patton writes in the 2008, 4th edition of Utilization-Focused Evaluation (Los Angeles: Sage publishers) that UFE are designed and judged by their usefulness by the intended users, here being OFDA and FFP. 19 A similar discussion can be found in S Banks, G McHugo et al. 2002 “A Prospective Meta-Analytic Approach in a Multisite Study of Homelessness Prevention” in Conducing Multiple Site Evaluations in Real￾World Settings, James Herrell and Roger Straw editors, San Francisco Josey-Bass. 23 Figure 2: Map of Data Collection Locations based on where IP activities occur 24 25 Evaluation Limitations Limitations Due to the Operating Environment 1. Due to security conditions when the evaluation technical specialists were in Yemen, they were not able to meet directly with the beneficiaries. This prevented the team from getting first-hand information and impressions to get a sense of the aid recipients’ feelings and emotions while expressing their views and ideas in the FGDs. Evaluation Response: Local (national) team members traveled to the project sites and met directly with the beneficiaries instead. 2. The interruption and delay in the evaluation’s original schedule due to an unexpected four-week evacuation of YOFMEP expatriates from Yemen, followed by adjusting to evaluation team members’ revised availability had ripple effects in adjusting the time-frames in the Formative Evaluation schedule. Evaluation Response:: The Formative Evaluation kept to a comparable timeframe, but was several weeks delayed. The follow-up occurred at a later time than originally thought, but the time delay allow for a better comparison of program lessons over time. 3. The physical access by teams to meet with many vulnerable households posed a potential bias as the locations of the beneficiaries were scattered and some were hard to reach. Evaluation Response: Where the team could not reach a location selected for the sample, the protocol was to select the next closest location. This was the case for two locations. Data collection was cancelled due to security conditions in one location: Abyan governorate, Al-Wadhea’a district. Enumerator teams also gave priority to overcoming roadside bias by getting out to remote areas. 4. Being able to interview target women interviewees was frequently difficult due to cultural conventions and low literacy and education levels. Evaluation Response: Priority was given to greater inclusion of women and women’s groups, including the provision of privacy in their FGDs. Limitations Intrinsic to IPs and Projects 5. The target group for the UNICEF protection outreach was school children and other community members needing MRE and involved the training of teachers and community volunteers. The UNICEF project also aimed to enhance coordination of MRE with the UN’s early recovery and education clusters to ensure more effective planning, monitoring and response. However, the evaluation methodology was limited to only inquire about the MRE activities for children and teachers at the primary school level. Evaluation Response: The scope of the evaluation findings focused on the awareness and training and not on ancillary activities or theories of change. 6. It was hard to get information from projects that were run by the Government of Yemen and from governmental counterparts, including challenges communicating with them and their limited availability. 26 Evaluation Response: Triangulation with multiple sources of data, including site visits, were used together. 7. Overlapping projects by different Government and international agencies made attribution of results a challenge to calculate for any one intervention, funding stream or IP. At best in these areas, “contribution to change” is the best inference. Evaluation Response: The Formative Evaluation provides some basis for a before-and￾after comparison via the Summative Evaluation. However, strict attribution is not a key part of the analysis. 8. As discussed in the methods section above, the activities by the IPs are not all at the same phase of implementation, some projects having barely begun their activities. Evaluation Response: This was taken into account in the sample frames and actual beneficiaries were targeted, not future beneficiaries for prospective activities. 9. In some cases, sampling frames by program areas lack good demographic data and details of the population/beneficiaries. Evaluation Response: The two-staged stratified, random selection of sites is robust given approximations of population sizes in target communities. Within the communities, data is based on the percentage of respondents and is not contingent on the overall community size. The overall sample size is liberal; the sampling strategy took into account the need to examine the OFDA and FFP portfolios separately. For the qualitative data collection, all 12 target OFDA and FFP projects were priorities, as were committees that exist as part of the implementation strategy of the associated activity. 10. A lot of important data about the associated the target communities and concurrent programs affecting them was not known or available. These included the particular barriers to food security and reliance at the village and micro levels, the burdens of disease seen at the clinic level or the actual density of landmine risk, or pollution associated with the areas of MRE programming. Evaluation Response: Again, triangulation was used across a wide array of information sources. Limitations Due to Methodology, Including Sampling 11. Matching data from program areas with EAs was difficult and time consuming because of differences with regard to how the project data were organized. Evaluation Response: This was a learning experience for the team that will inform the Summative Evaluation. 12. Tracing the true beneficiaries at the sites was difficult. Evaluation Response: Given leakage effects, it is not clear that it matters much. Many questions are about what took place in the project, not necessarily about how the beneficiary’s individual benefit was measured. 13. Pure randomization was not achieved in the selection of all sites. Some selected EAs replaced others, unavoidably, because of conflict conditions. 27 Evaluation Response: For this reason, the overall sample size was intentionally increased by over 30 percent to compensate for any “design effect” decreases in statistical power. The statistical power of the final sampling was still high. 14. As in any such survey, there is potential blurring of concepts when translating and re-translating back. The translation of the survey instruments into Arabic was critical for asking questions but the survey tools were completed in English. Evaluation Response: This was an iterative process of testing the meanings used and received back. 15. Whenever basing findings on respondent answers, there is the question of how honest the respondent is and how well they are able to report and remember with accurate recall. Evaluation Response: To compensate for these limitations, issues were addressed through different simultaneous types of questions, and from qualitative (FGDs) and quantitative (survey) methods. Limitations Resulting from Bias 16. Bias can be introduced in many ways, including many of the points listed in this section. Bias can influence how enumerators hear or record answers. Observational bias can be introduced via the selection of times to visit sites (favorable weather, non-holiday, etc.). Self-selection bias is introduced both in which Yemenis chose to come participate in assistance projects and via which beneficiaries make themselves available for surveys. Self-selection bias in the first case limits the ability of IPs from knowing which at risk populations they are not reaching and should instill humility in the aid communities about our knowledge of how well we are covering the overall problem. Self-selection in being available to answer surveys is a challenge both in implementing the surveys and in their subsequent interpretation. Evaluation Response: Each line of inquiry involved multiple sources of evidence to cross-check one another, such as students and teachers, both, when examining the implementation of landmine risk education. More attention will be given to tracking the degree to which target interviewees may have opted-out from being available for interviews in the summative evaluation. 17. IP staff introduce a bias, even unintentionally or unknowingly, in how they frame their answers to questions. This also invariably occurs with the staff of IP partner agencies. Evaluation Response: In this report, IP data in reports is complemented, not verified, by surveys, the later being more about understanding the reasons why projects unfolded the way they did and in their timing. Many IP interviews were interpreted as suggestive, the way FGDs yield insights about perspectives, but were not taken as fact. 18. Beneficiaries certainly introduce bias in their answers about the adequacy, fairness, transparency and timeliness of assistance. This is particularly pronounced in aid programs where there is perceived ongoing or significant future prospects for valuable assistance. So, for example, in food assistance programs, it is common worldwide that beneficiaries, however pleased to receive the food, will argue that it was the wrong kind, of low quality and in insufficient quantity and packaging. Because this is a broad, generic perspective, it’s hard to draw insights about whether, for example, food distributions actually satisfy minimum caloric standards (e.g. 2,100 kcal/person/day). Evaluation Response: in the course of interpreting and triangulating the evidence, the evaluation team learned over the course of the formative evaluation the limits of interpretation of survey questions about beneficiary satisfaction, fairness, etc. The levels of response were found to be 28 perhaps less revealing than the patterns within the responses, male versus female, location, IDPs versus non-IDPs, etc. The key response was to recognize that just because a large quantitative survey was implemented does not overcome the fact that what was measured were perceptions and attitudes which were influenced beneficiary calculations to influence increases in future aid. Similarly, in KIIs and smaller surveys (water, health), survey answers among beneficiaries are best at discerning their understanding of the aid program and their attitudes about what they would like to have. As in a FGD, they help to reveal issues related to the implementation of programs, but are not strong evidence about its technical quality. Limitations Related to Practical Considerations 19. The SOW required that the evaluation take a utilization-focused approach that required that YOFMEP staff engage in an iterative process with OFDA and FFP managers. It was efficacious for the work but required more time than anticipated. Evaluation Response: Evaluation team members put in more time. 20. The evaluation specialist responsible for managing the evaluation, and who developed the Evaluation Plan, departed unexpectedly at a critical stage of the evaluation. Evaluation Response: Fortunately, there was redundancy and backup, and a strong overall project leader. 21. Translation took longer than planned due to the subjective questions in the survey and the effort not to distort the meaning of the interviewees and to reflect the real answers of beneficiaries. Evaluation Response: Analysis, compilation, and report writing was shifted in time, resulting in a slightly later report to USAID. 22. Tools were prepared in English, translated to Arabic and then tested: answers were translated to English for analysis. Evaluation Response: As mentioned above, priority was given to capturing the meaning and content validity. 23. Time constraints prevented Prodigy Systems from conducting SIs in Hadramout because the distance to travel required three days. Evaluation Response: Instead, the evaluation drew from knowledge based on a monitoring visit to the IP in that area. 29 3. FINDINGS 3.1. AGRICULTURE/FOOD SECURITY AND ECONOMIC RECOVERY AND MARKET SYSTEMS SECTOR INTRODUCTION With recent peace agreements in Yemen, a large number of IDPs have returned home, including almost all IDP families who had fled Abyan. Yet many IDPs in the north are protracted. Both OFDA and FFP have allocated the lion’s share of their support in emergencies to the food security and livelihood needs of at-risk populations. Both increasingly support rehabilitation under the rubric of ERMS, which acknowledge the importance of value chains that determine how populations are able to re￾establish viable production, storage and trade for both income and food supplies. The evidence shows that the IPs they fund are exhibiting strengths in rigorously identifying the most vulnerable to participate in the interventions. It is too early to see service delivery results, but there are instructive lessons and indications of gaps in this sector for OFDA, FFP and IPs to watch as implementation continues. Both OFDA and FFP are experimenting, through their IPs with innovative ways to address household livelihood restoration, which includes both an attention to agriculture, including livestock and pastoralism, as well as the adroit re-establishment of key market linkages that unleash productivity. One of the largest implementers in Yemen, WFP, began piloting cash transfers to the poorest families in Hajjah and Ibb, aiming to give cash rations roughly equivalent to 50 kg of wheat and five liters of oil. Thus, each IP is gradually experimenting with transitional ways of moving from food to livelihoods. Sector Results Summary Table 2: Agriculture and ERMS Indicator Results Summary 1. What is the project doing to ensure that it reaches the beneficiaries most in need? What are the challenges to reaching the beneficiaries most in need? How have implementing partners overcome those challenges? Indicator OFDA FFP Defined selection criteria adopted by IPs. 3 of 3 IPs 2 of 2 IPs Percentage of survey respondents confirming knowledge of selection criteria. 100% 100% Monitoring mechanism for ‘selection criteria’ compliance put in place by IPs. 3 of 3 2 of 2 Percentage of respondents concurring about compliance with selection criteria. 20 100% 20 The output of the survey interviews did not distinguish between OFDA-funded and FFP-funded assistance, since IPs implement their programs jointly. 30 Percentage of respondents confirming above average effectiveness of targeting methodology in reaching out to the neediest groups. 21 100% 2. Are there significant needs within OFDA and FFP’s operational sectors that are not currently being addressed or met by OFDA and FFP programming? Indicator OFDA FFP Percentage of respondents confirming technical appropriateness of IP programming of AG and ERMS sector. Not applicable (N/A) (N/A) 3. What are the most significant lessons learned about the implementation process of OFDA and FFP programming from the monitoring process to date? What are the most significant strengths of and areas of improvement for OFDA and FFP programming? Indicator OFDA FFP IP projects are on plan with no major delays, problems or complaints recorded. Yes, according to current plan Yes, according to current plan Percentage of respondents22 confirming an above average level of satisfaction with the projects’ implementation mechanism. (N/A) (N/A) Percentage of respondents confirming above average satisfaction with the quality of services and goods. (N/A) (N/A) Background - Status OFDA often funds projects on the basis of one-year commitments in Yemen. Therefore, implementation occurs under the assumption of a six to12-month span of time. The comments presented in the “implementation status” table number 3, below draw out the status at the time of the field visit in March/April 2013. If the source was an IP report the information presented here may represent the status at an even earlier point in the project implementation and project cycle. All agriculture and market recovery projects that were included were in the startup phase at the time of the evaluation field visit, so they had only just begun activities and therefore there was little or no experience by the beneficiaries to collect. Some FFP programs (e.g. FARA, GC) had September/October 2013 start dates. OFDA program start dates are more varied. As an example, IOM was in the process of establishing the Community Voluntary Committees (CVCs) while the Yemen, Abyan and Lahj Livelihood Assistance (YALLA) detailed implementation plan indicates that livestock activities were planned for December 2013 with the initiation of additional, layered livelihood, irrigation and beekeeping activities over the few months into April 2014. Table 3: Agriculture and ERMS Implementation Status by Project (as of March 2013) OFDA/ FFP IP Project Name Start & End Date Implementation Status ADRA Yemen, Abyan and Lahj Livelihood Assistance (YALLA) Aug 2013- Aug 2015 Baseline completed. Livestock distribution delayed to prevent destocking during Eid. Livelihood Coordinator was not yet recruited. 21 The question was not stated in terms of below average, average and above average. All respondents were satisfied with the effectiveness of the targeting methods; however, some exclusions were noted (see text). 22 Surveyed and interviewed stakeholders. 31 O F D A GC Emergency Assistance to Support Yemeni Communities (EASE) Jul 2013-Jul 2014 EASE I finished in July 2013. EASE II builds on previous project. Constructed keyhole garden structures. Procured livestock with plans to immediately distribute IOM Enhancing the Resilience of Vulnerable Communities Through Integrated Livelihoods and WASH Programming Sep 2013-Feb 2015 Completed an assessment of water points and conducted community sensitization as initial implementation. In Al Mafah, IOM continues to provide emergency relief due to inaccessibility and lack of local institutions. Livestock distribution delayed to prevent de-stocking over Eid. F F P ADRA Food Assistance, Resilience Achieved Project (FARA) Sep 2013-Sep 2014 Baseline completed. GC Yemen Food For Asset Development (YFAD) Oct-2013- Sep 2016 Was still in the start-up and recruitment phase (No agriculture specialist yet). Baseline tools have been developed but survey not yet executed. Electronic voucher system designed and tested. It is now ready to roll out to other IPs. SCI Emergency Food Security and Resilience Programming (EFSP) Sep 2013-Sep 2015 Was still in the start-up and recruitment phase. The Agriculture, Food Security, and Market Context The work environment in Yemen is both geographically and temporally varied. Some areas of employment are extremely remote (Al Mafah in Abyan) and others lie just outside an urban center (Sana’a and Aden). The main contributor to temporal variability and implementation challenges is the violent conflict which varies across regions and programs, from local tribal disputes, banditry, and Al Qaeda bombardments, to political instability fostered by secessionist pressures. Climatic hazards such as drought and flooding (IOM areas) are both geographic and temporal in nature. Additionally challenging contextual characteristics are the feudal tenure system combined with widespread exploitation of landless and near landless peasants, rapid expansion of khat production at the expense of food and other traditional crops, damaged and deteriorated agricultural infrastructure (e.g. terraces and water catchment and irrigation systems), limited availability and access to improved inputs and microfinance, the decimation of livestock, and the growing scarcity of water23. Throughout much of Yemen, non-farm income (wages from causal labor and income from microenterprises and fishing along the coastline) is more common and contributes significantly more to household livelihoods and wealth creation than does work in agriculture. Food access is, therefore, largely determined by sources other than own-production. Tragically, the conflict destroyed much of the business capital and inventories. According to a WFP 2013 assessment, 95 percent of Yemeni households purchase food with their own resources and 23 percent rely on credit (the sum of these two figures exceeds 100 percent because some households utilize both access strategies). The assessment found that the percentage of households that are dependent on credit grew from 2011 to the end of 2013, principally in the southern Governorates.24 Indebtedness is a serious, growing problem. Not surprisingly therefore, a WFP survey notes that 80 percent of Yemeni households are indebted. With food prices identified as 23 Sources of information on the project context were program proposals and interviews with IPs and FAO. 24 WFP, (Sep 2013). Updated Food Security Monitoring Survey: Yemen. Sana’a. 32 the most common shock, 45 percent of all households surveyed reported food purchases as the main reason for their debt. Much of the population is recovering from acute conflict and the majority of those displaced Yemenis have returned to their homes. Consequently, the humanitarian community is gradually shifting from unconditional (grant) emergency relief to more strategic and conditional assistance aiming to restore community productive infrastructure and household livelihoods. The humanitarian community aims to “build back better” and to promote greater, more resilient households and communities (OCHA 2014 Strategic Response Plan).25 Additional challenges for IPs are the poor telecommunication capabilities and infrastructure among the IP program staff (field staff to headquarters) and between the target communities and the IPs, the slow pace of government engagement (including MOPIC), and delays in the scheduling of training through CSSW because of CSSW’s overloaded schedule. OFDA and FFP Objectives Evidence from the literature was limited, including documentation about OFDA and FFP strategies, assumptions and rationale. The findings presented in this report about food and agriculture draw on evidence primarily from the available IP reporting documents. The decision was made to shift from emergency to recovery aims to augment community and household resilience to natural and man-made shocks, rebuild more durable livelihoods, and improve nutritional and health status while maintaining appropriate food consumption levels through conditional food vouchers that support local market response and recovery. FFP continues to work to a large extent through the WFP. In July 2014, WFP intends to pivot from a large-scale general distribution program to the distribution of conditional vouchers that are tied to FFW, FFA and FFT. Similarly, SCI, through its EFSP, implements a substantial WASH program using conditional distribution and community asset restoration. A particular focus is water resources: wells and structures to harvest and distribute water. These programs are expected to have significant effects on new agricultural production and other water-dependent livelihoods. The food vouchers will also increase household food access. The use of food and vocational toolkit vouchers for the purchase of locally sourced food and other goods is meant to stimulate local market recovery and growth. ADRA and GC implement agriculture and ERMS programs with economic recovery and household and community resilience components. They will be distributing locally and regionally purchased small ruminants (sheep and goats) and chickens. Beneficiaries will also receive short-term training in animal husbandry. Via its FFA support, GC is introducing improved rainwater catchment in arid areas and constructing keyhole gardens within household compounds. As does FFP, OFDA provides support for AG and ERMS activities. GC, IMC, the IOM, SCI, and UNICEF implement programs to rehabilitate water infrastructure necessary for food production, such as wells, catchment structures, and irrigation systems. In addition, these IPs provide training in the improved management of water systems. These AG and ERMS projects, involving water infrastructure rehabilitation activities, create significant, albeit temporary, employment and restore livelihoods that are reliant on water supply, especially agriculture and animal husbandry. ADRA, GC, and IOM support the restoration of 25 OCHA. (Feb 12, 2014). 2014 Strategic Response Plan. Yemen Humanitarian Response Plan, Sana’a, prepared by OCHA in collaboration with the humanitarian community. 33 livelihoods primarily with the distribution of small ruminant and/or chickens, creation of keyhole gardens, vocational and veterinary training, and provisions of business startup kits. Methods and Data Collection Data used for the Formative Evaluation included secondary data from IP and USAID reports and primary data collection through SIs of key informants and FGDs with community members and beneficiary populations. Additional information sources are included in the reference list. Table 4: Sampling and Units of Analysis Number of IPs met: 4 ADRA, GC, IOM, SCF Number of beneficiaries surveyed in the sector HHs: none Individuals: none AG and ERMS were not included in the quantitative survey Number of FGDs and structured interviews: 26 32 FGDs/groups: 22 Individuals: 10 12 structured interviews included the headquarters of IPs. Structured field interviews: 4. SECTOR FINDINGS Line of Inquiry 1: Coverage A review of the output of the field data collection suggests that a number of respondents combined elements of various selection criteria together (e.g. the inclusion of previous ownership of livestock). This was not required of every beneficiary, only those who received small ruminants or chickens. In some instances most, but not all, the criteria were identified. It is not possible to determine whether the outcome represents interview error or the respondents’ limited understanding or experience related to the question. One field staff-person had only recently been hired and therefore did not have sufficient experience to answer the questions. The beneficiary selection process used by USAID IPs comprised several steps: 1. Geographic or site selection. 2. Definition of household selection criteria – households to receive food vouchers. 3. Selection process to establish the list of household beneficiaries, including creation of CVCs, use of either SFW or SFD lists, surveys of households, etc. 4. Verification process. 5. Definition of households to participate in specific activities (FFW, FFA, FFT, livestock restocking, agriculture, gardens, livelihood restoration and training) process to establish the list of participant beneficiaries. Each step made a contribution to the ultimate inclusion or exclusion of any given household in a program. In general, the identification of activities reflected USAID priorities, regulations and guidance as well as IP design. If the target of the program was nutrition and health, the livelihood intervention tended to be for 26 There were no focus group discussions in the areas where IOM implements. 34 women or household members who could contribute to community asset rehabilitation as well as the women’s willing to participate in BCC training. While the basic selection process is essentially the same for OFDA and FFP projects, each IP tends to use its own distinctive geographic criteria to determine where they will work. IP key informants explained that FFP uses the SFW while OFDA prefers the SFD in defining the most vulnerable communities.27 Several IP key informants who work in southern governorates were of the opinion that the SFD lists provide a more accurate measure of food insecurity and vulnerability, particularly in the highly politicized south. Where OFDA and FFP resources are combined, each IP uses a consistent household selection criteria for their OFDA and FFP activities (e.g. ADRA YALLA or GC/CHF). In these programs, the selection of beneficiaries who will participate in specific activities, such as livestock restocking and vocational training, is tailored to the intervention: access to irrigated land for agricultural production, current or previous livestock ownership, possession of sufficient property to establish keyhole gardens or previous experience in a particular non-farm vocation. IOM used one additional geographic (community) selection criteria. Since the design and objective of their program - enhancing community resilience and water resources (both quality and quantity) - can be variously based on a hazard (scarcity and flooding) or a determinant of the capacity to cope and thrive, IOM takes a more holistic approach. It opts to select communities that have water wells, a health post, a market place and the potential to enhance vocational opportunities. There are three OFDA-funded projects with AG and/or ERMS components: ADRA’s “YALLA” project, GC’s EASE II and IOM’s Enhancing Resilience of Vulnerable Communities. The first two are implemented in conjunction with other FFP-funded programs. IOM does not receive FFP funding in Yemen, but does have complementary UK Department for International Development (DFID) resources for some components of their livestock activities. The following table presents the IP headquarter staff’s explanation of the selection criteria used. The criteria listed in the table are used to select recipients of food vouchers. There are additional criteria used for specific interventions such as restocking livestock, vocational training, which are included in the text. Table 5: OFDA Principal Beneficiary Selection Criteria IP￾PROJECT INTERVENTION SELECTION CRITERIA ADRA￾YALLA Livestock, repair of irrigation channels or structures, beekeeping  Female-Headed HHs  Children under five  Family size (more than five children)  Presence of a disabled person in HH  HHs with no income 27 SFW is derived from the government institution responsible for the social safety net program and the one FFP uses. The list is considered to be inaccurate, missing many qualifying poor households and including many erroneous households that were added as personal and political favors. ADRA also noted that it is less representative of nutritional status compared to the SFD. The IPs that use this list have been helping to update and correct it. The SFD was supported by the EU and executed by a semi-governmental entity. It is considered to be more accurate and a better measure of need. OFDA uses this list. 35  khat sets the criteria based on the activity GC-EASE and YFAD Keyhole Gardens  Women with land (small area – two meters squared – around the compound)28  Income rank Livestock  Women  Experience with livestock  Willingness to participate in training IOM General Household Section  Female headed households  Family size  Presence of a disabled adult who would otherwise have earned income  Presence of a disabled child  Health conditions Vocational Support  Women with prior experience or willingness to train ADRA receives funding from both OFDA and FFP: YALLA is the ADRA-designed ERMS project funded by OFDA. The focus of the project is to support fisheries (both men and women), livestock (sheep and goats), beekeeping and irrigation depending on the local livelihood context. GC receives funding from both OFDA (EASE II) and FFP (YFAD). While staff are funded separately by either the OFDA or FFP funding streams, project activities are implemented jointly. Because OFDA did not require that GC identify its geographic areas based on the SFD, they were able to utilize the SFW for both funding streams and thereby integrate the areas of intervention. One exception to this is where the beneficiaries are refugees, which is outside of OFDA’s remit. While Yemen Food for Asset Development (YFAD) includes some refugees as part of its target beneficiaries, EASE II does not provide support for them because OFDA has not approved their inclusion.29 EASE II is primarily about market recovery (ERMS). The target population for the keyhole gardens is women with children under seven years old. Women are also targeted for the distribution of livestock (small ruminants and chickens) and a basic two-week training in animal husbandry, which includes a minimum amount of business training. Both men and women participate in the vocational training. Given the short timeframe of one year, GC pre-selected two vocations for women (hairdressing and retail sales), and two for men (electronics and auto mechanics). The team was not able to validate how valuable the skillsets for these vocations may be, or whether they fit in terms of beneficiary interests or gaps needing to be filled based on a market analysis. Over a period of time, monitoring may provide such data. IOM has OFDA funding, but not FFP support. Their project, Enhancing the Resilience of Vulnerable Communities in Abyan Governorate Through Integrated Livelihoods and WASH Programs, uses the rehabilitation of community water systems as an entry point for other activities including the provision of livestock to lower-income women without alternative non-agricultural incomes and the dissemination of business kits along with training to restore livelihoods. To better address livestock health and management, IOM is helping communities to forge stronger relationships with the MAI as well as arranging for MAI training of Associate Veterinarians, individuals who 28 Male heads of households can participate, though the focus is meant to be for women. 29 Within the USG, funding for refugee assistance falls under the State Department’s Bureau for Population, Refugees and Migration, which supports UNHCR’s work in Yemen. This is a separate pipeline of funding, distinct from OFDA. 36 receive basic training and apprentice with local private-sector veterinarians. One exception is Al Mafah, where they continue to provide emergency assistance. a. FFP Projects There are two FFP projects with AG and/or ERMS components: ADRA-FARA and GC-YFAD. Table 6: FFP Principal Beneficiary Selection Criteria IP-PROJECT INTERVENTION SELECTION CRITERIA ADRA-FARA30 General Household Selection IDP Returnee PLW Malnourished child Children under five Disabled person in the HH Income rank GC/CHF-YFAD BCC PLW Malnourished child Children under five Women willing to attend all trainings Family size Disabled person in the HH Income rank AG Own land Willing to work in groups All IPs understand that they should solicit criteria and targeting inputs from the communities themselves. Each IP begins with a starting set of criteria, which are then tailored through engagement with their communities starting with an explanation to the community. ADRA’s Food Assistance, Resilience Achieved (FARA) programming is well integrated with its YALLA activities. The latter is their primary agriculture and ERMS component and provides a mechanism to begin the transition from strictly emergency assistance to conditional assistance. The FARA conditionality mechanism requires households that receive food vouchers to participate in community livelihood restoration activities (see YALLA in the OFDA section above). GC’s YFAD project hinges on achieving: 1) BCC and training that is tied to the receipt of food vouchers for four months, and 2) conditional participation in the FFA efforts to restore critical community infrastructure and hence build the resilience of the community. Participants will be required to work two days/week for a few hours each day over the four-month period. They will also receive training in participatory planning so that the community can continue to plan and work collectively on future food security threats and challenges beyond the life of the project. Survey respondents did confirm that IPs based selection criteria on original surveys and not simply on the SWF. In general, respondents said that the process of listing beneficiaries and verification was effective at identifying vulnerable households. 30 FARA is not actually an AG or ERMS program other than that it is complemented by YALLA, which has a focus on livelihoods. FARA participant households contribute to community asset rehabilitation, which is a fundamental input to both agriculture and some businesses. 37 However, some respondents expressed their perception that the one criterion, “large household size” is biased against small vulnerable households, leading to the systematic exclusion of small households (e.g. retirees, disabled adults and widow with few dependents). Because this issue was revealed through the SIs and FGDs, the team is unable to report whether this issue has been resolved. But according to IPs, the small household size criterion is not absolutely binding; rather, one flag among many. MC is working with FFP support focuses on livelihoods via an iterative community dialogue to reach the poorest of the poor. MC does not select beneficiaries based on the lists from the SWF but instead works from survey data generated by the CCSW for the selection of the beneficiaries of its food and nutritional outreach. In addition, MC draws on data from the SFD for the implementation of asset-based programming. MC has looked at the market environment in order to creatively anticipate how to use its activities to kick-start businesses during community reintegration. MC conducted market analysis consistent with a Emergency Market and Mapping Assessment (EMMA), a new set of standards used by NGOs working in crises. MC considered that cash distributions might place women in a disadvantaged position vis-à-vis their husbands, while food aid was expensive to move, at the same time that Yemen has markets that ought to be encouraged. Unlike other neighboring countries, the mobile phone penetration was not at a stage for replicating partnerships with telecoms, as MC had done in Haiti, and other NGOs have experimented in other countries. The conclusion was that the greatest benefit, coverage, and efficiency could be achieved through a voucher system. 38 Combined OFDA and FFP Projects31 Observations from the field are roughly consistent across both OFDA funded and FFP funded activities. Several respondents noted that the most vulnerable households are those who are the poorest. But, in general, all IPs and CVC respondents equated the vulnerability of households with those households that fell within their selection criteria. IP headquarter staff noted that they had a set of criteria that would be modified with input from the communities through the dialogue with the CVCs. Ultimately, the broader community had influence over who was included among the beneficiaries, whether they added a new criterion to the set of criteria not. It was an iterative process. Only one focus group specifically mentioned that the community contributed to the determination of the criteria. GC and ADRA focus groups with CVCs said that they were given criteria by the respective IPs (no FGD participants said that IPs did or did not include the communities input in the selection of specific criteria). Overall, respondents stated that they felt that the criteria were appropriate, that the process was clear and fair and that the IPs adequately facilitated, monitored and validated the process. Most of the disagreements and errors were resolved through the various steps in the selection process. There were, however, a few complaints regarding households on the list. Several focus groups noted that the overall number of beneficiaries was too limited and that some households were excluded simply because there were too many other qualifying households competing for a limited and an insufficient number of spots to fill on the lists. Some concern was raised over households that were not included or were removed from the list. Several respondents in the field (field coordinators and other staff) and members of focus groups noted that the household size criteria (to receive food aid or vouchers) was the most problematic in terms of disqualifying otherwise deserving households (ADRA and GC, both of which have OFDA and FFP funding). WFP also uses household size in its criteria, applied when the head of household is male and unemployed and there are six or more family members. Illness and disability can override that criterion in some cases. But retired couples, elderly household heads with several dependent children, disabled people living in small households and widows who headed relatively small households were all said to be removed from the beneficiary lists due to small household sizes, despite the communities’ recommendation that they be included. These observations are not at odds with the finding that the CVC FGDs found the selection process effective, fair and transparent. These in-need households were identified through the process and put on the beneficiary lists. The concern is that worthy small-sized households were ultimately removed from the list because they didn’t meet one criterion (i.e. having a large household).32 ADRA (both with OFDA and with FFP funding) field staff were well informed about the selection criteria, verification process and distribution site management and supervision. The government District Officer, on the other hand, was unable to answer most interview questions but expressed interest in being more informed by ADRA. 31 There is some double counting across OFDA and FFP programs because it was not possible to distinguish whether all respondents were referring to OFDA or FFP or both. These funding sources are comingled. 32 Specifically, one filter for male-headed households is that they would need to have six or more members to qualify for program inclusion. 39 From the field surveys, 69 percent of respondents said that they did not know what ration size/composition that they were entitled to. Right or wrong, only two percent of recipients identified themselves as IDPs. Almost all, 99 percent, referenced themselves as beneficiaries of the ESN. The impressions of CVC focus group participants were mixed. Some focus group participants were well informed and expressed confidence in ADRA’s use of household surveys and satisfaction with the overall process. However, others noted that they were not informed of the selection criteria nor did they know how ADRA had selected to include certain interventions.33 One group provided an example of a village that had created their own criteria in lieu of receiving clear instructions from ADRA. This caused some challenges, which have since been resolved. The focus group participants complained about the household size criteria because it limited the inclusion of vulnerable households that happen to be small. It was not that households that should not be on list were able to register, but rather that some households in need were not included on the lists and they should have been. GC (Both OFDA and FFP) – GC employed an open CVC election process whereby the community showed a raise of hands in favor of a set of nominated CVC members. Despite apparent opportunities for political censuring, focus groups participants were satisfied with the process. They said that they were informed of the criteria. GC visited each household to verify their eligibility. GC beneficiaries also noted that the size-of-household criteria excluded some worthy households from the beneficiary lists. IOM (OFDA only) – At the time of the fieldwork IOM was still in the process of implementing assessments and setting up their operations. One IOM staff informant was only recently hired and unable to effectively answer the questions. No focus group interviews with beneficiaries were possible. Key informants from all IPs, regardless of funding source, and the key informant from the Food Security Working Group noted that it is customary for Yemeni people take care of each other. IOM stressed that shared responsibility is key to community resilience and reducing vulnerability. Yemenis will assist their family members, village and community members and those belonging to their tribe. What remains a significant challenge is the inequality of wealth and power within communities. Sheiks tend to own most of the land and, by extension, the water because the Sheik’s controls the irrigation equipment and his fields are irrigated first and the chances of receiving sufficient and regular water diminishes for those fields situated further along the irrigation system. Agriculture and ERMS activities that require water are directly influenced by this institutional arrangement. The more vulnerable households such as those who are poorer and less powerful and/or who have different family and tribal backgrounds than that sheik or the other more powerful people in the community will receive less benefit from the restoration of community irrigation infrastructure. Under both OFDA and FFP funding streams, the IPs sensitize the communities: they introduce themselves, explain what the intention of their program is, facilitate the process of establishing inclusive and representative CVCs and guide the CVCs about what criteria to use in the selection of vulnerable households and participants for particular program activities such as vocational training. IOM mentioned that they systemically visit each tribe within their project area and hold discussions with the leaders and community members. Publically posted lists are shared with the entire community and everyone is encouraged to give their impressions. All IPs stressed that is absolutely necessary to take time to build trust, reach agreement and win the endorsement of leaders and the community. 33 These focus groups included a range of community members, and within FGDs, views were not attributed to individuals. As such it is not typically possible to discern their role or standing in the community. 40 All IPs noted that while they employed house-to-house visits and conducted beneficiary validation surveys, it was still necessary to engage in an iterative process of mutual validation. 34 Typically the SFD or SFW lists are used in beneficiary selection. However, the communities validate the lists, and households are either added or removed from the lists according to the CVCs’ assessment of need and appropriateness. This process of updating SFD and SFW lists should contribute to the validity and accuracy of government and other donor-funded programs. To better ensure the inclusion of the most vulnerable groups in project activities, the IPs advise communities that are forming their CVCs to include sheiks, local council members, majors, religious leaders, women, teachers, health workers, youth, representatives from all villages and members of all relevant community committees (e.g. water or irrigation committees). Where there is strong suspicion of government (e.g. some southern districts), IPs refrain from recommending the inclusion of government officials and do not share the SFW lists with the community. Instead, they compare the lists to the SFW in private. The idea is to make the process transparent and trustworthy and, therefore, acceptable to the broader community. The IPs validate and work from those beneficiary lists that are pulled together by CVCs using the IP’s own house-to-house visits and by posting the list for public review and scrutiny. ADRA uses a beneficiary validation questionnaire. GC mentioned that they post the lists of which people will participate in each intervention. According to several IP key informants, the selection process is iterative and protracted, but necessary. The full process can take more than a month depending on the level of trust and cohesion within a community and how quickly the IP can establish community trust. The Livelihoods Officer at GC noted one incident that well illustrates how fragile some relationships are between IPs and their target communities. They had been working with a community for several weeks, building trust, defining the beneficiary lists, prioritizing interventions and preparing to start project activities. At that point, the community got into a severe altercation with another GC-assisted community, resulting in deaths – the dispute had nothing to do with GC. Nevertheless, the community quickly reassessed its relationship with GC and terminated their participation, as GC had a relationship with that other community. Key informants assured the evaluation team that these types of disputes and disruptions of an activity are not common. Line of Inquiry 2: Programming Gaps These were the specific gaps noted in AG and ERMS project designs. OFDA a. A lack of accessible credit (small business and household consumption). Local respondents spoke to the weak infrastructure for financial access, particularly for poor and peasant producers seeking small scale loans for economic recovery investments. b. Lack of interventions/support tailored to nomadic people. Though OFDA has unusual experience addressing emergency needs in pastoral populations (Niger, Ethiopia, Somalia…), it remains difficult for IPs to design interventions outside the rubric of settled villages, with 34 IOM’s public posting of lists with beneficiary names has been deemed a best practice by IOM, worth consideration by other IPs. However experience from other parts of the world leads IBTCI to question the protection implications of this exception to confidentiality. 41 infrastructure, clinics, etc. IP attention to the options for more dispersed rural communities, including nomads, remains a gap. c. Need to continue attention on disaster risk reduction (DRR). OFDA promotes attention to mainstreaming risk-reduction among its implementers, but the formative evaluation found minimal reference to it in the project reporting, implementation and designs of the IPs. d. Lack of interventions/support tailored to youth. School-age students and young workers were encountered in all project sites, but IP programs did not appear to intentionally consider the opportunities or needs specific to youth. e. More households require assistance than the program quotas can reach. In each project site, it was evident that there were populations appearing to be in need of assistance who did not receive assistance, were not on eligibility lists or who could not travel to the areas. Because ADRA and GC projects received funding from both OFDA and FFP, applicable many findings may be drawn from those listed below for FFP. FFP The quantitative survey revealed perceptions on the part of beneficiaries about gaps on the receiving end of programs intended to address food security head-on (i.e. Title II distributions). Credit as a Constraint to Resilience: Consistent with the UN Humanitarian Response Plan for 2013,35 IOM and GC noted that credit is a critical constraint to the rehabilitation of household and community productive assets, such as irrigation canals, wells, water catchment equipment, terraces, business equipment and inventories as well as livestock herds and flocks. It is also recognized that, faced with high prices of food and other basic commodities, Yemeni households rely on available credit to obtain these basic necessities and now they are accumulating significant debt.36 To achieve sustainable outcomes and enhance resiliency, households need access to credit and saving schemes. In order to devise a plan or program to address this gap, it will first be necessary to conduct more interviews with additional key informants familiar with credit and borrowing in Yemen. It may also be necessary to commission a more in-depth study about patterns of financial access during recovery, particularly for returning IDPs. Nomadic People’s interventions: Livestock replenishment in agriculture and ERMS projects in the portfolio work of settled agriculture, but the nomadic people of Yemeni do not fit this profile. Building Disaster Risk Reduction Capacity: During the KIIs, IOM and GC discussed the importance of building household and community capacity to manage risk related to drought, floods, access to water, food price and general inflation and political instability. Further discussion revealed that though projects may lack components designated “DRR,” in fact, the OFDA and FFP portfolios in Yemen consist of many projects comprising infrastructure construction for food and water, livelihood rehabilitation, and improving community management of these projects. These activities that anticipate and mitigate risks to vulnerable populations are approaches to DRR. 35 UNHCR, 2013, UNHCR’s Humanitarian Response Plan. 36 WFP. (Sep 2013). Updated Food Security Monitoring Survey: Yemen. Sana’a. 42 Why Including Youth is Critical: IOM and GC noted the gap in programming for youth. Yemeni youth (ages 14 to 24) represent 23 percent of the total population, while youth unemployment runs as high as 40 percent.37 GC mentioned that the youth in the south are easily recruited into the political conflicts and are regularly involved in the fighting between communities, tribes, etc. IOM believes that they need to provide livelihood opportunities and some form of mediation services.38 Addressing the Assistance Gap: The FGDs with beneficiaries suggested that there were significantly more households requiring assistance than can be reached with the resources available. This is not an unexpected claim to hear from beneficiaries. In planning for the Summative Evaluation, YOFMEP will see if OFDA/FFP wishes to know how much its IPs cover in terms of percentage of total needs. This might require data collection from other donors. Line of Inquiry 3: Implementation Lessons, Strengths and Areas for Improvement IMPLEMENTATION LESSONS Mostly, the AG and ERMS interventions reviewed in the Formative Evaluations were still plans and not yet activities with outputs or results. The Formative Evaluation occurred too early in the project cycle for solid lessons to be drawn. Nevertheless, IPs shared some lessons that they felt they had learned:  The beneficiary selection and potentially protracted validation process of some IPs is critical in a post-conflict context such as Yemen in that it forms the basis for building trust between the community and the IP.  Vulnerability (as equated with food insecurity) is a stagnant (status) measure rather than one associated with a household’s evolving capacity to manage risk. It was noted (e.g IOM) that returnees and households where the primary income earner(s) died or became disabled tended to be among the most vulnerable. This is another reason for having alternative beneficiary identification methods that do not rely exclusively on SWF and SFD lists.  Water catchment systems are critically important resources to the entire community and to agriculture production in most areas of Yemen, but vulnerable household access to sufficient water for agriculture depends on where they are situated within the system and how resources are managed in the system as a whole.  Credit and savings schemes should be included.  Coordination of the UN food security cluster is effective at the regional offices but not at the national Sana’a level.  There were many UN clusters and meetings for a limited number of IP staff to adequately engage in all of the efforts; this proved to be a challenge for IPs trying to coordinate. STRENGTHS/PROMISING PRACTICES 37 World Bank. (April 2013) Press Release, World Bank Grant Supports Employment Creation for Yemen’s Neediest Youth and Women. 38 Source: United Nations, World Population Prospects: The 2012 Revision, Available on: http://esa.un.org/unpd/wpp/unpp/panel_indicators.htm. 43  IPs use community-based approaches to targeting, including the use of community-based assessments and publicly-posted lists that are shared with the entire community for comment. These methods promote transparency and help to correct inaccuracies and omissions from recipient information generated via the CSW, CSSW, and SFD.  By combining OFDA and FFP funding, refugees can be better served, since OFDA doesn’t provide assistance for refugees. AREAS FOR IMPROVEMENT Are the projects’ organization, technical staff and service delivery adequate for the provision of the output level and quality of services proposed? The Formative Evaluation occurred too early in the project cycle for a meaningful assessment of adequate service delivery by the agriculture and ERMS interventions; the Summative Evaluation will assess results at that point. Findings were that IPs were demonstrating organizational and technical staff capability and resource allocation commensurate with OFDA and FFP funding. This will be a key question for the Summative Evaluation and will be given greater attention in the field surveys. 44 3.2. HEALTH AND NUTRITION SECTOR INTRODUCTION Emergency-affected populations in Yemen face a wide range of diseases and high rates of malnutrition, requiring infectious disease control, health systems operation, and the community or outpatient treatment of severe malnutrition, complemented by broad scale food rations provided to displaced households. OFDA and FFP and their IPs are involved in a wide array of H/N interventions, mostly oriented around health facilities. The combination of health, nutrition and food aid is the largest sector in expense and volume of activities. These interventions are oriented largely to the provision of supplies to meet the immediate needs of the target populations, including medical equipment and specialty foods for recovery from malnutrition. Malnutrition and disease compound one another in a child. At a national level, food availability also a dynamic relationship with conflict, landmine-polluted agricultural lands, higher fuel prices, returned migrants and urban riots. In January 2014, locust swarms had destroyed 90 percent of sesame and millet crops in western governorates. Sector Results Summary Narratives of the brief indicators below follow the chart. Table 7: Health and Nutrition Indicator Results Summary 1- What is the project doing to ensure that it reaches the beneficiaries most in need? What are the challenges to reaching the beneficiaries most in need? How have implementing partners overcome those challenges? Indicator OFDA FFP Defined selection criteria adopted by IPs. The selection within UNICEF, IMC or SCI programs is based on nutritional status, and the health beneficiaries are selected based on illness. In other words, the criteria are individualized by diagnosis. Yes Percentage of respondents confirming knowledge of the assistance selection criteria. The selection for UNICEF, IMC or SCI programs is based on their nutritional status, measured by anthropometrics using weight-for-height. Health beneficiaries are selected based on health needs. The individual diagnosis is complex and difficult to compare for consistency. FOOD DISTRIBUTION More than half of the beneficiaries (i.e. 55 percent of male beneficiaries (219/402) and 58 percent of female beneficiaries (316/540)) of food distribution knew why they were selected. The majority of the non-beneficiaries interviewed (i.e. 83 percent of the males (68/82) and 78 percent of the females (135/173)) did not know why they were excluded and were not aware of the selection criteria. Review and or monitoring mechanism for conformity to selection criteria put in place by IPs. The selection for UNICEF, IMC or SCI programs is based on their nutritional status, as the health beneficiaries are selected FOOD DISTRIBUTION Almost all of the beneficiaries (i.e. 97 percent (391/402) of the male and 98percent (529/540)) of the female respondents reported that appropriate 45 based on health needs. It is difficult to describe selection criteria since it is dependent on individual diagnosis and thus not applicable. grievance/complaint structures or channels were not in place for cases of abuse or misuse of food distribution. Among the non-beneficiary respondents, 94 percent (74/79) of the male and 97 percent (166/172) of the female respondents reported that appropriate grievance/complaint structures or channels were not in place for cases of abuse or misuse of food distribution. Percentage of respondents concurring beneficiary compliance with selection criteria. It was evident that beneficiaries met selection criteria since services provided by UNICEF, IMC or SCI programs were based on their nutritional/health status. FOOD DISTRIBUTION There was gender disparity with respect to transparency of the beneficiaries’ selection process: most male beneficiaries (i.e. 60 percent (239/402)) but less than half of the female beneficiaries (i.e. 45 percent (243/540)) said that the beneficiary selection was “transparent,” reflecting lesser female involvement in liaising with IPs, lower literacy, and attention to other things. Among the non-beneficiaries interviewed, however, only 28 percent of the male (22/79) and 30.2 percent of the female (52/172) reported that the selection process was transparent. Among those who knew why they were selected, only one-third of them (i.e. 32 percent of male beneficiaries (70/219)) and 33 percent of female beneficiaries (105/316) were aware WFP or IPs use of selection criteria but that they were unfamiliar with the details of the criteria and verification. Similarly among the non-beneficiary respondents, only a small proportion (i.e. six percent of both males and females) were aware that any beneficiary verification process was going on, not surprising in many non￾camp settings. Percentage of respondents confirming above average effectiveness of targeting methodology in reaching out to the neediest groups. The selection by UNICEF, IMC or SCI programs is based on their nutritional status, as the health beneficiaries are selected based on diagnoses, presentation to the clinic, health needs and illness. FOOD DISTRIBUTION Most of the male beneficiary respondents (i.e. 58 percent (234/402)) and half of the female beneficiary respondents (i.e. 49 percent (263/540)) said that the beneficiary selection was fair. As noted above, the majority of females reported a lack of transparency in the selection process suggesting that women are kept more outside of meetings and may be less literate. Among the non-beneficiaries interviewed, similar levels of males and females (17 percent of men (13/79) and 19 percent of women (32/172)) reported that the selection process was fair. As they were not beneficiaries, it stands to reason that more may have considered the process fairer had they been included for assistance. 46 2. Are there significant needs within OFDA and FFP’s operational sectors that are not currently being addressed or met by OFDA and FFP programming? Indicator OFDA FFP Percentage of respondents confirming technical appropriateness of IP programming by sector. HEALTH Among those who utilized the health services of the facilities, 57 percent (60/106) of the beneficiaries said that there are gaps in the services. NUTRITION Among those who utilized the nutrition services of the health facilities, nearly half (i.e. 48 percent (38/79)) of the beneficiaries said that there are gaps in the services. Recipients described gaps in the amount of food rations being received (cooking oil, beans, wheat), and the frequency of distribution. In other words, when asked, they said they wanted more of everything, in larger lots, and more often. 3. What are the most significant lessons learned about the implementation process of OFDA and FFP programming from the monitoring process to date? What are the most significant strengths of and areas of improvement for OFDA and FFP programming? Indicator OFDA FFP IP Projects are on plan with no major delays, problems or complaints recorded. 3 out of 3 4 out of 4 Percentage of respondents39 confirming an above average satisfaction with the projects’ implementation mechanism. Most beneficiaries expressed satisfaction, while at the same time noting the unreliability of health services and other service gaps. FOOD DISTRIBUTION The majority of the respondents (i.e. 84 percent (339/402) of the male and 75 percent of the female (406/540)) respondents reported that they are satisfied with the food distribution services. Percentage of respondents confirming above average satisfaction with the quality of services and goods. HEALTH Almost all of the beneficiaries (i.e. 92 percent (35/38)) who attended the health education sessions at the facilities reported that the sessions have influenced their health-related practices. NUTRITION All of the respondents (i.e. 100 percent (45/45)) who attended the nutrition education sessions at the facilities reported that the sessions have influenced their practices. Nearly two-thirds (i.e. 63 percent (254/402)) of the male and 66 percent (358/540)) of the female respondents reported that they are satisfied with the quality of the food received. 39 Surveyed and interviewed stakeholders. 47 Background - Status Table 8: Health and Nutrition Project Implementation Status IP Project Name Start & End Date Status O F D A IMC Emergency Assistance to Vulnerable Populations in Yemen (EAVP-Y) 2/1/2013 – 1/31/14 Completed SCI Child-Focused Health, Nutrition and WASH Emergency Response 3/1/ 2013 – 2/28/2014 Completed UNICEF Scaling up the integrated management of severe acute malnutrition (CMAM) among girls and boys under five years of age in the most vulnerable communities 1/1/ 2013 – Dec. 31st, 2013 Refunded for 2014 Completed F F P ADRA Food Assistance, Resilience Achieved (FARA) Project 9/15/2013 - 9/14/2014 Mobilization Phase completed: selected districts, formed community committees, selected and verified beneficiaries. In the 1 st quarter of CY 2014, began distributing food vouchers and food distribution, and nutrition awareness training Mercy Corps Food Security and Resilience Building Program (EFSP) 10/1/13 – 9/30/2016 In the 1 st quarter of CY 2013, began food voucher distribution. SCI Emergency Food Security and Resilience Programming (EFSP) 10/1/2013 - 3/31/2016 In the 1 st quarter of CY 2014, began distribution of food vouchers for work, mobilizing health facilities & Mother-to￾Mother support groups. WFP WFP - Emergency Food & Nutrition Support to Food￾Insecure and Conflict-Affected People (EMOP 200451) January 1st, 2013 – Dec. 31, 2013 Re-funded for 2014 Completed The Health and Nutrition Sectors Context Yemen has one of the highest rates of acute (wasting) childhood malnutrition in the world,40 compounded in recent years by food price shocks, which have led to food riots.41 According to a comprehensive study completed in mid-year 2013, nearly half of Yemeni children under five are chronically malnourished42, one million children are acutely malnourished, and some 500,000 pregnant and lactating women (PLW) are at risk of malnutrition. Poor feeding practices, household food insecurity, lack of education, diarrhea or other diseases caused by unsafe drinking water and poor access to H/N programs all contribute to the 40 In general, levels of underweight in children are modest in the Middle East and Western Asia, with the one exception of Yemen. For weight-for-height, Yemen may have the highest rate of wasting anywhere in in the world. 41 WFP reports from its 2013 Food Security Monitoring Survey that most Yemeni households are pursuing “destructive consumption” (i.e. eating fewer meals, smaller meals and other coping). 42 Updated Food Security Monitoring Survey Yemen, World Food Programme, September 2013. 48 elevated prevalence of malnutrition. Malnutrition is most prevalent in the densely populated Northwest, along the Red Sea coast and along the Arabian Sea coast and into Abyan. In addition to 30,000 cases of cholera confirmed between 2010 and 2011, and peaking in the summer of 2011 mostly in Abyan, Lahj Al-Dhale’a and Aden43, Yemen has seen a number of key lethal and debilitating disease outbreaks of concern to the aid community. Yemen is also a priority country for monitoring polio during the global eradication efforts. Ten years ago Yemen had 180 cases confirmed, after it was reintroduced via Sudan, compelling international aid agencies working often with INGO partners to mount large-scale immunization programs,. Polio has become a renewed concern in 2014 as polio has been seen for the first time in many years and has been spreading in nearby Somalia and Syria. UN Health Cluster attention is also on the Middle East respiratory syndrome coronavirus (MERS) that had a confirmed case in Yemen in early 2014. Yemen Surveillance for Yemen Rift Valley Fever44 was also found to have affected over 650 in 2000. In 2012, Medecins Sans Frontieres reported both measles and dengue outbreaks in the conflict-affected areas of Abyan. The political upheaval and accompanying violence following the change in government in 2011 have complicated the ongoing humanitarian situation. This has resulted in the destruction and disruption of local primary health care facilities, a pervasive shortage of qualified personnel, a reduction in the provision of primary health care, and a lack of adequate medical equipment and referral systems, particularly in Sa'ada, Hajjah, Amran, Al-Jawf, and Abyan Governorates. According to the 2013 UN Health Cluster report, about 8.6 million people lack access to sufficient health services. The conflict not only has driven up food prices (as around the Salafist conflict) and interrupted health services, but has also created distrust between institutions. IOM reports about its work in conflict zones in the recent HPN: “In Abyan, as in Al-Jawf, IOM had to devise remote monitoring strategies and mechanisms that could ensure effective programming while minimizing corruption and the misuse of IOM resources. As field teams comprised individuals from mutually hostile tribes or opposing political factions, IOM often received contradictory accounts of events in the field and uncovered corruption or abuses of power perpetrated by field staff that either misunderstood or wilfully misrepresented IOM's objectives and priorities. Where there was animosity among field staff, baseless accusations of corruption proved difficult to verify or disprove without time- and labor-intensive investigations.” Many respondents in the field research repeated concerns about the low capacity of the government to achieve health goals. The MPHP designed a 15-year national health strategy to be implemented between 2010 and 2025.45 The strategy aims to ensure the provision of preventive, therapeutic and rehabilitation health care, paying attention to the balance in allocation of resources among the various types of services, levels of health system, and the rural and urban areas in order to further approach the regional standards in governmental health expenditure. USAID has been assisting the sector through a number of IPs, including NGOs and UN agencies. WHO and UNICEF receive USAID funding for immunizations in conflict-affected zones in the north. The US 43 Vibrio Cholera 01 was diagnosed in an index case of a Somali refugee in Yemen in Hodeidah in early 2010. Yemen has had one of the largest though least reported cholera epidemics in the world during the last few years. Cholera previously spread in war-affected southern Yemen in 1994. 44 An acute hemorrhagic fever syndrome, which also killed hundreds goats, sheep, cattle and camel, in northern Yemen. 45 National Health Strategy 2010 – 2025, Ministry of Public Health and Population, Republic of Yemen. 49 Department of State funding through the United Nations High Commissioner for Refugees (UNHCR) also targets health issues countrywide. OFDA and FFP Objectives Objectives of USAID/OFDA to address H/N gaps and needs in Yemen are broken down here by nutrition and health. Health  Health system strengthening through working with the MPHP, including capacity building trainings for MPHP staff to ensure the sustainability of activities.  Primary health care and mobile health facility support to prevent and treat communicable diseases and increase vaccination coverage through personnel and community education (i.e. through IOM and NGOs).  Disease surveillance system improvement for early detection of epidemics and outbreaks.  Maternal and child health support, including clinical care for victims of sexual violence.  Support coordination efforts through the World Health Organization, who head the health “cluster.” Nutrition USAID/Yemen has identified nutrition as a cross-cutting theme in its Development Objectives in its Country Development Cooperation Strategy for 2014 – 2016. USAID/OFDA supports nutrition activities that are linked with health services and encourages increased integration with other sectors like WASH/hygiene promotion. The nutrition specific objectives include:  Prevention of malnutrition through the promotion of infant and young child feeding (IYCF) education and practices.  Interventions to address moderate acute malnutrition (MAM) and SAM.46  Training of nutrition workers to improve their knowledge and skills relevant to working with children.  Where appropriate and feasible, OFDA collaboration and joint planning with the Mission on resilience-building nutrition programs in the areas of malnutrition. The objectives of USAID/FFP to address H/N gaps and needs in Yemen are to screen for and reduce malnutrition, address general food insecurity among the 1.2 million food insecure, and build resilience among 378,000 people in Al Dhale’e, Dhamar, Ibb, Lahj, Raymah, Sana’a and Taizz Governorates. These FFP objectives that address nutrition gaps and needs in Yemen focus on the reduction of food insecurity among vulnerable populations. Through its Emergency Programs, FFP meets this objective by 46 In emergencies and recovery, this is short-term (acute) wasting malnutrition, as measured by Weight-for￾Height and/or by Mid-Upper Arm Circumference. Some nutritional recovery programs use both measures as criteria. 50 providing food assistance to save lives, reduce suffering, and support the early recovery of populations affected by emergencies. Specific interventions include:  Direct food assistance through blanket rations to women and children who have additional vulnerability in most emergency situations due to their status.  Therapeutic and/or supplementary feeding to acutely malnourished individuals.  Cash transfers and food vouchers as appropriate given the context and market conditions. For the last decade, NGOs, with OFDA and FFP support, have been standardizing an outreach approach to reduce malnutrition referred to as CMAM, or Community-based Management of Acute Malnutrition, which includes treatment at homes and regular child screenings at the outpatient wings of health centers (hence, this part is referred to as the Outpatient Therapeutic Program or OTP). These allow for coverage of rural and marginalized populations who cannot get to or attend Therapeutic Feeding Centers (TFC). FFP has contributed 78,000 metric tons to WFP since the start of FY 2013. This summer, WFP will commence a new Protracted Relief and Recovery Operation, which will include FFW and cash assistance. FFP partnered with MC, GC and SCI in late September 2013 to support more than 378,000 of the food insecure in Al Dhale’e, Dhamar, Ibb, Lahi, Raymah, Sana’a and Ta’izz through voucher for assets, food and nutrition, including DRR. Methodology & Data Sources The following table shows the number of study participants for key informant interviews (KIIs) and quantitative HH surveys: Table 9: Health & Nutrition Sampling and Units of Analysis Health & Nutrition Sampling and Units of Analysis Seven IPs IMC, IOM, MC, SCI, UNICEF, WFP, WHO Individuals: 19 HHs surveyed 125 for Health & Nutrition 1,192 for Food Distribution FGDs and Structured Interviews 20 Health Workers SECTOR FINDINGS Line of Inquiry 1: Coverage IPs deployed more staff in the field to cover wider geographical areas and expand the monitoring of projects. The monthly UN cluster meetings allowed them to exchange updates on who is doing what and where and helped to avoid duplication of efforts and wastage of resources. They could have gone further and provided more of a forum about which approaches and methods were effective. Use of mobile teams to reach out to the inaccessible areas has been successful. Periodic visits to these areas also have increased the amount of dry rations given. Moreover, monthly joint supervision with MPHP also have facilitated accessibility to those in need and ensure timely disease control efforts. IPs cite many challenges to their being able to actually reach their intended beneficiaries, including: the weakness of the health system and the absence of health workers from the facilities for most of the day; 51 a lack of credible national partners; poor quality of monitoring; the rugged physical topography of Yemen; a lack of proper mechanisms for obtaining feedback from the beneficiaries; and the self-exclusion by marginalized groups who fear coming to get services from the health facilities. There was a lack of coordination between IPs, and especially between UNICEF and WFP, regarding the management of SAM and MAM caseloads of malnutrition, respectively, as continuity of care is critical in recovery. There was a low level of awareness among government officials and beneficiaries about the type and purpose of the therapeutic and supplementary food products and inadequate overlap between food security conditions and malnutrition levels so as to link the differing WFP vs. UNICEF programs. There exist significant numbers of marginalized groups who fear coming to health facilities. These groups in many instances excluded themselves, often for cultural reasons. This is due to fear, long distances to travel, lack of knowledge, or language issues; thus, it was difficult to serve them. This question of how to interpret the level of needs and barriers to participation among specific groups remains an ongoing conundrum for OFDA’s and FFP’s portfolios. Transparency and Accountability The Formative Evaluation identified that the process of beneficiary selection was not transparent within the communities being targeted, as nearly half of the beneficiaries and three-fourths of the non￾beneficiaries expressed an inability to account for the selection. Almost all of the respondents, beneficiaries and non-beneficiaries, males and females, reported that appropriate grievance/complaint structures or channels were not in place for cases of abuse or misuse of food distribution. Among the few who acknowledged the existence of such structures, the majority reported that the organization was not responsive to their complaints. SCI found that there was a high level of illiteracy among the general population, which prevented information flow and made access to services difficult. Nearly half of the beneficiaries and three-fourths of non-beneficiaries did not know why they were selected for (or excluded from) the program. This means that although the community is aware of the selection criteria, there was no systematic mechanism in place to enable them to hold service providers (i.e. community leaders and IPs) accountable for their decisions and make them responsive to their complaints. Much of the challenge with full coverage and inclusion of malnourished children is the lack of awareness among Yemenis of the risks and causes of malnutrition, or how to measure it in a child. There was a low level of awareness among Yemeni senior officials about the different levels of malnutrition and their implications, which has prevented easily reaching those in need. Malnutrition has been considered a normal phenomenon for a long period of time among Yemenis. Their low level of awareness or ability to recognize malnutrition as an urgent medical condition has prevented parents/caretakers from bringing malnourished children to medical attention. The IPs are also struggling against entrenched behaviors regarding infant feeding. Many young mothers in IP activity areas are pressured by family members to introduce solid foods to newborns and under six month old infants based on local traditions of introducing solid foods at early ages. One local preparation is called ‘Chebissa,’ which is prepared using a mixture of flours. International guidelines however prohibit international NGOs from encouraging any supplementation to infants below six months of age, who should be exclusively breastfed by their mother. 52 a. OFDA In MC activity areas, community members themselves collectively select the type of community asset to be constructed or rehabilitated after a process of prioritization in the FFA project. There is a community asset management committee (CAMC) for identification and registration of the beneficiaries. There is a memorandum of understanding (MOU) where no committee member will have influence on the ownership of the project. The surveys among the local health facilities revealed the main ways they see the IPs currently providing support, principally in training and drug supply, as seen in Figure 3. Current forms of support provided by the IPs, but coming out of interviews with health center staff, were: training, drugs, nutrition, furniture, salaries, and budget support. It is important to distinguish between the mobile health teams that travel out into districts to provide assistance and are wholly uncovered and the community outreach to the population around a health center as part of routine primary health care extension work. Yemen includes (and needs) both. Use of mobile health teams to ensure that the most marginalized and hardest-to-reach were identified as providing some assistance, as for example in the Saada directorate. In the last year, some mobile teams ceased working, as the USAID-funded CLP project ended its support. Figure 3: Current Forms of Support from IPs to Health Centers/Partners 53 The health facilities that receive support from IPs also mount their own community outreach in most cases, as seen in Figure 4. For more remote groups, IMC sends mobile teams that close the gap between the facility and the community, thereby helping to increase health care coverage. In principle, outreach also can reduce defaulter rates from the currently very high levels, seen to be as high as 60 percent. IMC also recruits and trains people who are able to read and write, then serving as VCHWs to educate the community about key messages, screen children for malnutrition, and provide vitamin A pills. Clinics have identified multiple avenues to extend healthcare to populations in outlying areas. Currently, community health workers (CHWs) have been the most widely used vehicle to improve health care. However, Mobile Health Teams and “house calls” by health center staff have been utilized as well. UNICEF has developed guidelines that prescribe the admission and discharge criteria of children into the programs. However, UNICEF apparently has no mechanisms in place for receiving beneficiaries’ feedback for their H/N programming. More than two thirds of health centers stated that they used CHWs to reach out into their communities with primary health care. Different strategies were used by Governorate Health Offices (GHOs), District Health Offices (DHOs), and IPs to reach out to women and children in remote locations, including outreach campaigns, field visits by staff, mobile units, and VCHWs. In one project, IMC, deployed mobile teams to increase the breadth of coverage and reduce the rate of default from the current high levels of 60 percent. In addition, IMC and SCI recruited and trained people who are able to read and write as VCHWs to educate the community on key messages, screen children for malnutrition, and help distribute vitamin A capsules. Some organizations, like UNICEF, worked to build the capacity of local indigenous NGOs. Figure 4: Health Care Outreach Activities to Get Care Out into Their Community 54 With OFDA support, IMC finds that in its partner health facilities offices were closed for much of the day because the health workers did not adhere to official working hours. Particularly in the health sector, new, technologically advanced, sophisticated tools are being used in Yemen to improve the quality of monitoring, which IPs can share among one another. SMS messaging is one example for BCC. This may be critical to overcome community and cultural misconceptions that can impair the project outcomes that were observed. Some community members reported that vaccines cause infertility because they came from Western countries or Israel, and that some of the foods contained pork because the distributing organization was from America. FFP The lists of food assistance beneficiaries were not routinely updated on a regular basis. Some of the IPs use the SWF list that is more than five years old, while others identify beneficiaries based on the 2012 CFSS. 47 Among the IPs, the amount of livelihood transfers and the modalities used (i.e. cash or food vouchers) also differ, for example between WFP and MC. Unfortunately, the level of awareness of the selection criteria for food distribution was found to be low. Non-beneficiaries were more likely to be aware of the criteria for food distribution than the beneficiaries themselves, possibly reflecting a correlation among illiteracy, displacement, fragmented households and meeting the vulnerability criteria for assistance. WFP, working with support from FFP, identified its targets with the use of its 2011 Comprehensive Food Security Survey. At present, WFP does not intend to re-register or verify the levels of needs of its target households. While WFP manages the large-scale distribution of specialty foods for moderate malnutrition, there was a low level of understanding about how to appropriately use these foods properly, including “Plumpy’Doz”48 and “Plumpy’Sup.” These are both Ready to Use Supplementary Foods (RUSF). Because these products are sometimes shared with and eaten by family members who are not malnourished, their effectiveness in saving lives of the most vulnerable is diminished. WFP’s nutrition objective for their support to feeding centers is to treat MAM cases among children aged six to 23 months. They use Plumpydoz for younger children, Plumpy’Sup for children six to 59 months, and Plumpy’Sup and a blanket distribution of a WSB, oil and sugar for PLW. In order to ensure that these foods are actually consumed by the intended individuals, and not spread among household members, additional dry rations are given for the rest of the family. There was no finding about whether the extra rations intervention worked for the intended purpose, affirming the hypothesis. It would be instructive for WFP and/or YOFMEP monitoring to collect and analyze data to find out. In MC’s FFP programs, the beneficiary lists included numerous individuals who do not meet the selection criteria and should not have received assistance. Beneficiary selection in MC’s programs are being followed with a secondary verification process. MC has striven in recent years to become more compliant with the Humanitarian Accountability Partnership in terms of its accountability procedures, and in Yemen established a grievance reporting mechanism where people can complain of abuses. 47 Comprehensive Food Security Survey, World Food Programme, 2012. 48 Plumpy’Doz is a popular, brand name for one form of Ready to Use Therapeutic Food (RUTF). Though a proprietary brand name from one set of providers, the name is often used at the field level inter￾changeably with the overall category, RUTF. In this report the generic term will be used from now on. 55 There was an ongoing discussion among the nutrition clusters on the integration of IYCF programs in all health facilities. The staff will first be trained to orient the Mother-to-Mother Support Groups (MTMSG) on the subject. Many households believed that sending older children to school may lead to long-term malnutrition (i.e. stunting) of their younger siblings still at home. SCI is focusing on broad-scale change through counseling messages developed based on the need of the area and messages created for the particular week, to be decided ahead of time. Line of Inquiry 2: Programming Gaps Gaps in current OFDA and FFP programming both at the political and institutional level include:  Lack of integration or continuity-of-care for case-management, within and between sectors.  Inadequate planning of programs, such as ESN, to seize opportunities.  Concerns about the packaging of food aid. How IPs have overcome challenges IPs have built the capacity of local indigenous NGOs who can access and can stay in the areas of implementation longer and put different innovative and technologically advanced tools in place to monitoring project activities. Currently, there is an ongoing dialogue to coordinate and provide the continuum of care needed by malnourished children. Some IPs are deploying temporary staff and mobile teams to oversee, for example, how the food distribution is taking place and whether the selection criteria and procedures are being followed. a. OFDA Despite the existence of health facility-based community outreach (i.e. through CHWs), the health programs lack a strong community component in terms of reaching out to the beneficiaries with the desired quality messages and frequency of contact. In particular there is a lack of integration across activities (e.g. between IYCF and CMAM), between treatment of different degrees of malnutrition, and between ESN and Health Education, which reflects many missed opportunities for the provision of continuum of care that is needed for better outcomes. The preventive approach to malnutrition always needs to be integrated with the CMAM programs that are basically curative. This will help mothers/caretakers to adopt better caring practices for their children and reduce relapse. Moreover, there is a lack of coordination at the facility level, where there are lots of missed opportunities by health professionals. For example, a child who is coming for vaccination services could get his or her nutritional status assessed while the child is at the facility. 56 The main gaps reported by the respondents were that there are no drugs or medical supplies (71 percent reported this), followed by inadequate health facility staffing because health workers do not show up or stay in the facility (30 percent reported this). The proportions who said that the health workers are not skilled enough or that there was inadequate health education given were minimal. Figure 5 reports beneficiary findings that they find gaps in the availability of health services, often due to health workers not being at facilities. Overall, a constant theme from the surveys was that beneficiaries found health workers did not show up at staff offices when they were supposed to, an accountability issue. But it also hints at targeting of medical personnel. As reported by Michael Neuman of Medecins Sans Frontieres: “Yemeni health workers are extremely worried about their security… an additional risk factor is the gap between the reality of care in Yemen and the high expectations patient have of doctors… Across the country, accounts of doctors trying to avoid treating highly complex medical cases and referring patients for security reasons abound. In 2012 and 2013 three… Yemeni doctors left…after having been either threatened or involved in an incident. All three gave the same reason for leaving: a general lack of motivation to continue working stemming from insecurity.” 49 The health projects are supporting different types of reproductive health services. The lack of family planning services has been a critical gap as well as antenatal care (ANC). Some health facilities have requested laboratory equipment to improve the diagnostic and case management services. Repeated shortage of medical supplies and stock-outs of Ready to Use Therapeutic Foods (RUTF) for the management of SAM cases were also commonly cited frustrations of staff. In this formative inquiry, half of the health of health workers reported that they have been supervised by the GHO or DHO on a monthly basis, while 20 percent reported that they have never been supervised. Among those supervised, one-third (31 percent) reported that the government officials did not come with a supervision checklist. In general, governmental support to health facilities is poor. 49 Michael Neuman, MSF 2014 “Managing the Risks to Medical Personnel Working in MSF Projects in Yemen” Humanitarian Exchange Magazine: The Humanitarian Situation in Yemen London: Overseas Development Institute Page 10. Figure 5: Health Facility Concerns of Target Populations 57 At the IPs coordination level, the projects are well staffed with technical expertise. At the level of rural health facilities, where the service delivery occurs, there is a pervasive shortage of qualified staff and a lack of commitment to the working hours, and moreover some health workers are considering nutrition work as an additional task, which is not part of their job description. At the community level, the teams of VCHWs were not large enough in number and were not being trained with a standardized module to equip them with the knowledge and skill that they need to provide counseling for behavioral change. These limitations reduce the outputs and quality of services to which the project is committed. The H/N programs lack sufficiently strong community grounding in terms of the outreach to the beneficiaries with key public health messages and referrals. Moreover, contact between HHs and their outreach oriented VCHWs was infrequent. What outreach did exist too often was targeting only mothers. The BCC messages to the population were not attuned to the specific needs or understandings of the HHs. Moreover, the VCHWs were not skilled enough to follow effective counseling steps and techniques and instead were just passing on information. UNICEF sees numerous gaps in the integration of health systems and services as organized by the government, especially at the facility level. Moreover, the level of awareness or execution among health workers about the importance of integration was low. For example, a child who is reached for vaccination could be screened for malnutrition. The opportunity is missed for enrolling many malnourished children to the OTP, but UNICEF did not appear to routinely do this, despite its lead role in offering vaccines. There is high turnover of staff, which creates the recurring need for new staff training and refresher trainings. If there were more auto-didactic materials, such as wall charts that clearly showed the details of the case management of diseases, it would help plug the gaps. In IMC programs, the BCC component is ineffective. Counseling was provided based on pre-defined topics for a certain period of time and was not attuned to the specific needs of the HH on the day of visit. Basic nutritional recovery feeding is also challenged by the demands of health workers who request additional compensation and claim that nutrition is not part of their job requirement. At the same time, the nutrition recovery standards manual, which is available at all field sites to guide CMAM programs, is larger and more complicated than some staff find useful. There is a need to generate shorter summary versions for easier reference to basic and simple procedures. More than half (52 percent) of those who visited health facilities believe that there are gaps in the nutrition services provided Figure 6: Nutrition Services Gaps or Concerns Expressed among Beneficiary Households 58 at the facility, and the majority (71 percent) reported that there are no RUTF supplies. More than a quarter surveyed reported that nutrition education is not given at the facility (26 percent), while the level of awareness of the community about the causes of malnutrition and the nutrition therapeutic products used for treatment was low. Operationally, there is little mention of Essential Medicines Lists that are required by international standards to harmonize the work of outside agencies in conflict zones.50 b. FFP In the general ration, WFP imports WSB brought in bags of 25 kilograms (kgs) and vegetable oil is imported in four-liter cans. Distribution is taking place in smaller units: 10 kgs and one liter respectively. This leads to inefficiencies and food loss (spillage) and introduces some difficulties for the recipients with regard to their post-distribution transport of the food home. From the evidence reviewed, WFP does not appear to integrate its programs well, either in terms of case referrals or with the community. This is not unexpected for an organization managing interventions of such large scale. In most cases, WFP’s food ration distribution, under the ESN program, was not linked to any health or nutrition education, either as an offer or a condition. Based on the formative evaluation surveys, there is a notable difference by gender in terms of who receives the food assistance. The male in the family collects the food 52 percent of the time, the mother 21 percent, and other 27 percent. In general, however, there was no statistically significant difference in utilization of food, whether it was collected by the father, mother or other family members: 97%, 97% and 98% of men, women, and others who collected food said they utilized it for household consumption. It was only the 2 to 3% of the households who sold food rations, irrespective of who collected it. One 50 The Government has an “Essential Medicines List” and tracks pharmaceutical needs in the country. See: MPHP 2012 Pharmaceutical Country Profile. Figure E: Responses about Gaps in Food Distribution (n = 853)Figure F: Nutrition Services Gaps or Concerns Expressed among Beneficiary Households Figure G: Nutrition Services Gaps or Concerns Expressed among Beneficiary Households Figure H: Responses about Gaps in Food Distribution (n = 853)Figure I: Nutrition Services Gaps or Concerns Expressed among Beneficiary Households Figure 7: Beneficiary Views about Food Rations. 59 consideration might be that UNICEF has been targeting only women about children’s diets with the promotion of IYCF messages, but does not message with males who also play a role in household diets. The management of SAM and MAM was integrated in only 71 of the nearly 400 WFP sites by March 2014 and there is a plan to increase the coverage gradually to the remaining sites. At the household level, it is important to track malnourished children enrolled in supplementary feeding. But for WFP activities there were notable gaps in monitoring these young beneficiaries at their homes. For instance, it was difficult to observe whether supplementary food was being consumed by the malnourished individuals, as intended, or consumed by other family members (referred to as “leakage”). Other gaps that were observed were an absence of proper monitoring tools to follow up on not only on how much food is provided but also on how the foods are administered to the malnourished child. In the case of food distribution, a lack of small packages of foods smaller than the current 25kg and liters of oil in smaller units than the current four liters also pose problems for beneficiary planning. Line of Inquiry 3: Implementation Lessons, Strengths and Areas for Improvement LESSONS LEARNED There is valuable flexibility in some projects. For example, MC is implementing a three -ear Food Security and Resilience Building Program, but the project keeps changing every year depending on the lessons generated, learned, and reincorporated by MC. STRENTHS/PROMISING PRACTICE Essential H/N packages were defined in 2013 in order to make integration a routine aspect of the government system. This can be supported for pharmaceuticals (essential drug adherence), workforce training, and population surveillance. The opportunity exists to bring the government in to the conversation now. The UN nutrition cluster, chaired by UNICEF, meets monthly, but the MPHP ultimately controls which agencies work in different parts of the country where there are relevant inter-agency nutrition sub-clusters meetings (in Aden, Hodeida, Saada and Taiz). The clusters help to standardize protocols and coordinate nutrition surveys. There is a successful case of facility level integration in Aden where, for example, a child coming for immunization will go through IYCF counseling, get weighed, and leave. Outreach is the key challenge to get beyond clinic-based models or camp-based assumptions of population access. UNICEF has attempted to scale up its systems approach, under OFDA programming and has H/N specialists who are required to make field visits monthly or quarterly. UNICEF cross-checks the data and mentor the field offices and has health/nutrition specialists in each of their district offices. These officers are responsible for monitoring all the UNICEF activities that are implemented through their government and NGO partners. Additional staff, including an information management officer, are allocated in each of the field offices to minimize the reporting lag, which had been in the range of three to four months. UNICEF also works frequently with Food Distribution. April 15, 2014. 60 the government: There is a weekly teleconference to review the plans and listen to feedback from the field. Good practices include the EFSP project of MC where their service delivery is well coordinated with the local community. The communities selected the public works to be constructed or rehabilitated after going through a prioritization process to ensure sustainability. In terms of coverage, MC’s initial registration constituted only 50 percent of the beneficiaries. The next 50 percent are intended to be identified later, for there will be more people who are needy and not selected. This is an iterative method that should be watched and perhaps replicated. The governorates, the districts and the sheiks are being notified about the limited resources available so as to give priority to those in real need. MC’s food￾assisted activities are implemented in coordination with the government to ensure ownership by authorities of the ongoing sustainability of program. For example, MC convenes a kick-off meeting with the line ministries in each of the governorates. MC also weaves in a conflict mitigation approach to its food assistance programs to resolve disputes while also extending services to those in need. The ESN created an opportunity to educate people with regard to different issues, such as child caring practices including breastfeeding, hand washing, family planning and so on. The women who are the targets did not show up in most cases. Many innovative approaches are being adopted by the IPs for reaching out within communities that are inaccessible. The most cost-effective and context-specific outreach should be adopted by other IPs and taken to scale. As Yemen transitions from conflict, more “early recovery” programs are being designed and supported by donors in the health field. DFID shifted to three-year funding cycles for its nutrition programming, combining internal humanitarian and development funding streams. IMPROVEMENTS When asked how could health services be improved at health facilities, the local health staff gave top votes to providing a physician (four), providing medical equipment (three), providing an ambulance (two), and several mentioned infrastructure and refrigerators. The VCHWs who are mainly engaged in giving out health information could counsel better by taking into account the level of literacy of the beneficiaries. Vulnerable populations remain critically dependent on external actors like NGOs and the UN for life￾saving drugs, micronutrients, therapeutic foods and vaccines:. There is little progress toward building resilient supply chains for these items in lieu of external intervention. A more comprehensive approach to health is required. Currently, there is an ongoing dialogue to coordinate and provide the continuity of care needed for the management of malnutrition as a recovering child is discharged from therapeutic feeding and should enroll in a supplementary feeding program. Guidelines emphasizing this comprehensive approach are being developed and there is a call for an integrated collaboration involving all partners. Another survey finding was the request for the training of midwives. All health facilities are submitting to IPs monthly reports that appropriately disaggregate health conditions by gender. Family planning and reproductive health are beginning to be scaled up, but too slowly. The cultural status of women has kept this sector behind. The International Rescue Committee (not an IP to OFDA) has focused on promoting transport and around-the-clock maternity ward electricity to enable several thousand Yemeni women to 61 give birth in clinics with skilled assistance, which has helped 2,700 deliveries to occur with medical supervision.51 One path to better coverage of neglected populations is via stepped-up coordination among IPs, between IPs and government line offices, across the different sectors, and even at the facility level. There is a need for the standardization of trainings of VCHWs in terms of content. The training should equip them with the knowledge and counseling skills to facilitate the behavioral change process at the HH level. MAIN FINDINGS The government health system is weak, with a shortage of health workers, a lack of commitment by the health workers to the full, official working hours set by the government, and a shortage of drugs, medical supplies and therapeutic foods. All of the H/N activities of the IPs are operating on this limited platform of the government health system, which limits effective access by the beneficiaries. UNICEF is building the capacity of local/national partners on the assumption that international NGOs will probably exit eventually. Currently, there are about 50 NGOs that have relevant experience in emergency programs management. There was a disconnect between the management of the different stages of malnutrition (between acute and severe), which is required for the provision of the continuity of care necessary for each child’s full recovery. In addition, the IYCF interventions that are necessary for the prevention of malnutrition were not integrated with CMAM programs for prevention of relapse. Those children who were being treated for malnutrition were not properly followed over time to monitor how the therapeutic products are being administered and whether the children are resuming their growth trajectories, as recorded on their road to health cards. H/N programs require referral systems and a continuum of care whereby each individual passes through different categories or phases of attention and assistance. Often, that’s where problems occur in emergencies, particularly where people are displaced. As a good example, in the SCI operations, children discharged from the therapeutic feeding programs are referred to the supplementary feeding (MAM) centers, as per protocol, and then later are followed up at their dwellings by the VCHW. The follow-up is to inform caretakers about the diet necessary for the child to prevent recidivism, which is common in cases of malnourished children. 51 Dr. Abdelhadi Eltahir, Nathaly Spilotros, Kate Hesel. 2014 “Family planning and post-abortion care in emergency response - IRC’s experience in Yemen” Humanitarian Exchange Magazine: The Humanitarian Situation in Yemen London: Overseas Development Institute. Page 11. 62 3.3 WATER, SANITATION AND HYGIENE SECTOR INTRODUCTION As Yemen is a poor, arid, water-scarce country with minimal infrastructure for WASH, an estimated 13 million Yemenis live without access to a minimally acceptable (‘improved’) water source (access is only 31 percent) and 12 million are without access to minimal (“improved”) sanitation (access is only 21 percent).52 Second only to food security interventions, the WASH sector has the highest number of interventions (seven) funded by OFDA and FFP. Three OFDA projects launched early in 2013, and had end beneficiaries who provided useful feedback for this Formative Evaluation. In general, the IPs have been resourceful and successful in reaching the neediest beneficiaries. In the WASH sector there are numerous promising practices and lessons learned, as well as programming gaps and areas for improvement for OFDA and FFP managers to consider. Sector Results Summary Table 10 introduces the key indicators examined, with explanations following. Table 10: WASH Indicator Results Summary 1- What is the project doing to ensure that it reaches the beneficiaries most in need? What are the challenges to reaching the beneficiaries most in need? How have implementing partners overcome those challenges? Indicator OFDA FFP Defined selection criteria that are adopted by IPs. 6 out of 6 1 out of 1 Percentage of survey respondents confirming knowledge of selection criteria. 100% 100% Monitoring mechanism confirming ‘selection criteria’ put in place by IPs. 6/6 100% Percentage of respondents confirming compliance with selection criteria. 100% 100 % Percentage of respondents confirming above average effectiveness of targeting methodology in reaching out to the neediest groups. 100 % 100 % 2. Are there significant needs within OFDA or FFP WASH activity areas that are not being addressed or met by OFDA or FFP Programming? Indicator OFDA FFP Percentage of respondents confirming technical appropriateness of IP programming of WASH sector. 80 % 0% 3. What are the most significant lessons learned about the implementation process of OFDA and FFP programming from the monitoring process to date? What are the most significant strengths of and areas of improvement for OFDA and FFP programming? Indicator OFDA FFP IP Projects are on plan with no major delays, problems or complaints recorded. 5 out of 6 1 out of 1 Percentage of respondents53 confirming above average satisfaction with the projects’ implementation mechanism. 90 % 0 % Percentage of respondents confirming above average satisfaction with the quality of services and goods. 90 % 0% Sector Background – Status 52 UNOCHA 2014 Strategic Humanitarian Response Plan, Yemen 2014. United Nations. 53 Surveyed and interviewed stakeholders. 63 As the table below shows, OFDA funded six out of seven of the projects with WASH components. FFP funded an SCI project with WASH that overlapped an earlier OFDA-funded project on the list. Table 11: WASH Implementation Status by Project IP Project Name Start & End Date Implementation Status O F D A GC Emergency Assistance to Support Yemeni Communities EASE – Phase 2 8/10/2013 - 8/10/2014 Mobilization Phase: Formation of volunteer committees and selection of beneficiaries for various assistance activities. IMC Emergency Assistance to Vulnerable Populations in Yemen (EAVP-Y) 2/1/2013 - 1/31/2014 Project has been in operation for at least 12 months at time of evaluation and is reporting on major results indicators. IOM Enhancing resilience of vulnerable communities, livelihoods and WASH 8/29/2013- 2/28/2015 Mobilization phase for selection of 15 communities within the targeted districts and identification of water infrastructure sites for rehabilitation and beneficiary selection. IRD Community Water Resilience through Sand Dam Construction - Improved Hygiene Practices in Yemen 8/22/2013 - 2/21/15 Community mobilization done and four sites excavated; 70 percent of works were completed in Hadramout Governorate, Districts of Aldherbeen, Aidem and Shatroten Romah, while 40 percent work completed in Wadi Tuftuf. SCI Child-Focused Health, Nutrition and WASH Emergency Response 3/1/2013 - 2/28/2014 Project has been in operation for at least 12 months at time of evaluation and is reporting on major results indicators. UNICEF WASH 1/1/2013- 12/31/2013 refunded for 2014 Activities under the grants for 2013 completed 12 months of operations. A new 12-month implementation round began in January 2014. F F P SCI Emergency Food Security and Resilience Programming - EFSP 10/1/2013 - 3/31/2016 Mobilization Phase: No WASH activities started. Technical assessment and sites selection and verification of beneficiaries and formation of CRC (Community Resilience Committees) on-going. WASH Sector Context Reports from OCHA and UNICEF indicate that in Yemen, 140m³ of water is available per person per year, compared to an average of 1,000m³ in the Middle East and North Africa. Of this 140m³, only seven percent covers personal and household requirements; the rest is used for agriculture, including khat cultivation. More than three million people live in high risk, water-shortage prone situations in 70 percent of the districts, and 3.76 million in 85 percent of the districts are in critical emergency situations. Approximately 30 percent of the water-supply infrastructure in rural areas does not function due to lack of adequate water supplies, disrupted power supplies due to fuel shortages and sabotage, or a lack of financial resources to pay for maintenance and repairs. An estimated 73 percent of the population in rural areas does not have access to adequate sanitation, substantially increasing the population’s day to day exposure to waterborne and life-threatening diseases. The periodic reports from the UN-led WASH Cluster, which assesses needs in Yemen highlight that poor hygiene practices lead to increased communicable disease transmission and poor health outcomes. Similarly, Standardized Monitoring and Assessment in Relief and Transition (SMART) surveys have 64 confirmed the expected correlation between poor hygiene practices and increased malnutrition, particularly among children under five years of age who are the most vulnerable. According to UNICEF, unsafe water, poor sanitation and unhygienic practices are responsible for 88 percent of diarrheal disease in Yemen. The various WASH needs assessments that UN OCHA compiles indicate correlations between populations affected by cholera and malnutrition and a lack of piped drinking water, open defecation and poor hygiene practices. The assessments reinforce the recognition that young children in Yemen have multiple bouts of diarrhea a year, typical of the least developed areas of the world. Several other water and sanitation related diseases pose ongoing, endemic, life-threatening risks in Yemen, especially to children, such as cholera, malaria, schistosomiasis54 and parasitic worm infections. Chemical pollution such as fluorosis and run-off from agricultural chemicals, such as nitrate, are additional, unaddressed health risks in parts of Yemen. UNICEF works mainly through government and partners as opposed to implementing programs directly. Given its experience, UNICEF reported the inability of government departments to lead the provision of sustainable services and develop policy guidelines/framework in the WASH sector due to weak institutional capacity. For example, there are poor information management systems (e.g. no available data on water resources (hydrogeology information)) and no national guidelines to regulate and monitor water quality and sustainably manage water resources. The UN WASH cluster, chaired by UNICEF, reports that it has improved the ability of humanitarian organizations to deliver timely access to WASH services for IDPs and other target communities in hard to reach locations. The WASH Cluster developed a response plan that aims to address their needs by providing durable solutions for both those IDPs who can return and for those who cannot, in the north and the south of the country as well as for the communities hosting them. The most vulnerable communities suffering from chronic WASH under￾development were targeted by OFDA or FFP intervention. USAID/DCHA’s IPs in Yemen are responding to water needs in a variety of ways, recognizing that water plays many roles in the communities. OFDA and FFP Objectives The USAID Humanitarian and Cooperation Strategy for 2014 – 201655 emphasized WASH as a priority, in recognition of the importance of water supply in the context of water scarcity. Pursuant to this priority, OFDA provided nearly $1.9 million in FY 2013 for an innovative WASH program to build water storage capacity through sand-dams, with a significant “software” (local knowledge) component. In addition, the anticipated USAID-funded FEWS NET will be able to provide the WASH cluster with the monitoring of underground water tables and deep aquifers. This will improve on the information management system of water resources for Yemen and protection of ground water usage. Unlike with OFDA, no FFP strategy addressing WASH was found in the course of the evaluation. Data Sources Table 12: Interviews and Surveys Six IPs: GC, IMC. IOM, IRD, SCI, UNICEF Individuals in IPs: 19 54 The World Bank, with WHO, supports the Ministry of Public Health and Population to control schistosomiasis including the distribution of 46 million doses of praziquantel, targeting school age children. 55 USAID-OFDA Yemen Complex Emergency, Fiscal Year (FY) 2014 Implementation Plan. 65 HHs surveyed: 589 Individuals surveyed: 209 HH from UNICEF (CSSW and Al Khair NGOs) and 380 HHs from IMC and SCI Water Management committees from two FGDs and structured interviews, from IRD sand dam projects (three Hadramout villages) and GC East￾phase two (six Abyan, Aden and Lahj villages) Total number of individuals in FGDs and SIs: 19 SECTOR FINDINGS Line of Inquiry 1: Coverage In general, the IPs defined their target, vulnerable groups as being female-headed households, the disabled (elderly, mental and physical handicap), poor families, and by location (IDPs, vulnerable communities, arid, water-scarce areas and Nomadic communities), and by activity groups (e.g. schools in IDP camps). To be considered ‘poor,’ a family’s income status was estimated by the use of criteria that included access to land, number of livestock, disability and being a female-headed household. During project design phases, 100 percent of key informants from the IPs confirmed they used needs￾assessments and baseline surveys by the individual IPs and by others (e.g. UN) to identify and select beneficiaries who are the most in need. UNICEF, for example, worked together with and supported its partners in the selection of beneficiaries. UNICEF selected beneficiaries on the basis of location (IDPs, vulnerable communities) and activity groups (e.g. schools in IDP camps). In making these decisions, UNICEF based its selection on assessment data and reports done by IPs, UNICEF field staff, and UNHCR. Examining how vulnerability was defined, 84 percent of beneficiaries in UNICEF’s WASH areas (of who 81 of were female) responded that they have no means of income or livestock, while another 16 percent said that they did have income. This confirms that IPs did successfully define their vulnerable groups and beneficiaries using the expected set of criteria that were based on groups such as female headed households, the disabled (elderly, mental and physical handicap), and poor families. A breakdown of how recipients pursue their income and livelihoods, based on the Formative Evaluation’s survey work, appears below in Figure 8. Agriculture, government, and day labor are the primary means of income for the majority of the population in areas where WASH is active. With 24 percent of people relying on farming and livestock for a livelihood, there is an extreme need to provide easily accessible clean water and proper sanitation services and infrastructure in order to support an agricultural economy and stymie food-borne diseases that have the potential of limiting growth. 66 Key Informants from IOM and SCI (in FFP-funded projects) defined vulnerable groups as being female￾headed households, the disabled, (elderly, mental and physical handicap) or very poor families. As did other IPs, GC engaged local community members, program participants, and a range of key informants to corroborate these criteria (needs), as described in key secondary sources, to more accurately gauge the state of humanitarian needs in their program areas and to best target resources to those beneficiaries most in need. Evaluation interviews, supported by monitoring data, found that the approach of targeting based on geography used by 80 percent of IPs had a primarily rural focus, emphasizing locations (governorates, districts and villages) that are most adversely affected by conflicts, including those regions that have hosted large numbers of displaced people, along with other indices such as the level of water scarcity and level of food insecurity. The key constraints in reaching out to vulnerable households most in need, according to 100 percent of the respondents from the IPs, were the physical terrain, physical insecurity, and inter-tribal conflicts. All parts of Yemen – with the exception of the western part – have areas where it was difficult to reach beneficiaries. GC initiated digital short-message-service (SMS, or cellphone based text messages) broadcast technology to disseminate key hygiene messages to target communities via the mobile phone Figure 8: Formative Evaluation Survey Results. Between one-third and one-half of WASH target households depend on agricultural work, including livestock, for their livelihood 67 network.56 This was critical where either physical distance or insecurity prevented the IP staff from convening community workshops. Virtually all (100 percent) of key informants said that local community leaders, such as religious leaders, were engaged in both site selection and in beneficiary selection, based on the results of needs assessments that were coordinated with the IPs. Thus, most WASH activities were targeted to IDPs in camps and to vulnerable communities in arid, water-scarce areas, such as the IRD project in Hadramout, or activities targeting mainly the nomadic communities in urgent need of water for their livestock to survive. Respondents from five out of six the IPs who were interviewed also confirmed that working through local leaders, including imams, local council members, Sheikhs, and Mullahs, helped to reach out to those prospective beneficiaries who are most in need, and obtain community buy-in for the WASH activities. Key UNICEF respondents confirmed that UNICEF is reaching its beneficiaries through government established structures at lower levels and through local partners who have good relationships with local communities. GC acknowledged that they rely on their partnership with the CCSW a local nonprofit group, to reach beneficiaries in certain districts in Abyan that are too dangerous for GC staff to travel to. A respondent from GC said working through the local leadership “created venues for the community leadership and project management to meet and show solidarity, while noting that community leadership support at the local level increases trust and access.” This Formative Evaluation confirmed that strategic alliances were frequently struck among IPs, local NGOs, and local leaders - including imams, local council members, Sheikhs, and Mullahs - in order to identify the neediest, to interact with beneficiaries in more remote, conflict-affected and hard-to-reach areas, and to obtain community buy-in for the WASH activities. Cooperation with local government appears to facilitate WASH activities in Yemen. UNICEF engaged with local water authorities for on-site selection and design specifications of water point rehabilitation. This enhanced government involvement in reaching out to populations most in need of WASH services and possibly in longer-term maintenance of the sites. Interviewees said local councils helped the IPs to identify vulnerable HHs. To address the potential exclusion of other vulnerable groups, IPs reported that they sought to be inclusive in the selection and implementation process through dialogue with different sections of each community. This also allowed the community to understand the importance of WASH services. Key informants from IPs reported that working collectively with government and local community leaders helped to mitigate against tensions and tribal conflicts over water rights. Transparency and Accountability Of the key informants interviewed from the IPs and local community leaders (e.g. local councils), 100 percent of respondents confirmed their knowledge of the selection criteria as being based on estimates of income status (i.e. land, number of livestock), disability and female-headed families. 56 As in many other parts of the developing world, the affordability and pervasive use of mobile phones among rural and poor populations is a recent phenomenon that aid agencies have been quickly experimenting to capitalize on. Mobile phone use in Yemen has increased from four million in 2007 to 17 million in 2014, roughly half the population. Bringing clean water home from a water collection point in Raymah Mountains 68 This contrasted with beneficiary feedback: 84 percent of females and 17 percent of males respondents surveyed who said that they were not aware of the targeting process, the selection process, or the selection criteria and not involved in the beneficiary selection process. Similarly, 84 percent of females and 17 percent of males who participated in the Formative Evaluation’s survey said that IPs did not contact them during the targeting process and so they did not agree that the targeting methodology was effective. IOM formed WMCs with the help of government counterparts and local communities, with the assumption that they were best placed to identify those most in need. However, there is often too little inclusion or participation of women in these committees, unless the IP pressed for their participation as a requirement of the project. All key informants from WASH IPs stated their field teams later verified that those targeted based on their vulnerability, did indeed participate (they used house-to-house visits, for example). This followed naturally, they said, after there was maximum participation of community members, including their leaders, during the selection processes. Line of Inquiry 2: Programming Gaps Several gaps in WASH project designs were noted:  Inadequate monitoring of WASH interventions by some IPs.  Beneficiaries rely on privately owned, rather than public water sources.  Human needs57 in most IP project locations exceed the capacity of the interventions.  Inequitable participation of women.  Dearth of DRR approaches to WASH. Lack of attention to structural, long-term water infrastructure as the funding stream and time-horizons discourage planning for long-term resilience and cost-effectiveness. Lack of Adequate IP Monitoring of WASH Interventions This evaluation, through KIIs and review of available records, established that UNICEF, especially, does not have adequate internal monitoring mechanisms or monitoring plans for tracking the progress of its program implementation or activities against the project indicators and work plans. Also, YOFMEP monitoring activities had been unable to satisfactorily verify IOM progress against the indicators and work plans. Beneficiaries Rely on Privately Owned Rather than Public Water Sources The Formative Evaluation examined whether beneficiaries were getting water for free or whether they paid for the water from these sources. More than 60 percent of respondents said that they paid for water while 31 percent said they did not. While more than 60 percent said they pay for water, about half (47 percent) of respondents said their water comes from privately owned water sources; 13 percent reported that the water they use comes from sources owned by the community. Table 13 below shows a breakdown of the ownership of water sources and beneficiaries by their use, from the surveys. 57 The minimum standard to which most IPs are committed through their sign-on to the Sphere standards is 15 liters of water per person, per day. 69 Table 13: Who Owns and Controls Water Resources Who owns water sources/points Percent Community Private owner Government Not applicable 13 50 18 21 Each IP informant said that some of the water resources they rehabilitated belonged to privately owned households, in communities where many individuals pay for the water, as reported below.58 Human Needs in Most IP Project Locations Exceed the Capacity of the Interventions The evaluation ascertained that the scale of the needs in most IP project locations is greater than these interventions can cover. For instance, only 30 percent of respondents from the IPs (IMC, IOM and SCI) reported that they are implementing some level of institutional WASH programs (mainly hygiene promotion activities at health facilities and around markets), while UNICEF is supporting partners for WASH in schools on a limited scale, because of limited resources. Figure 9 displays respondents’ primary worry that there is a shortage of water. A dried￾up water supply not only affects Yemenis’ health, but may also causes food shortages in the future. The limited accessibility of water due to high costs and distance between water points also threatens the livelihoods of Yeminis. Unequal Participation of Women Limited options for education and high levels of illiteracy among women reduces the possibility of recruiting and involving women volunteers (a key component of the project design) in WMCs, but does not necessarily preclude women from being effectively trained either vocationally or technically. Some of the key informants’ from IRD, for example cited that one of the criteria for selection into WMCs is a 58 The IPs might explore how their assistance for the rehabilitation of privately owned water sources affects the beneficiaries, including their livelihoods. A market analysis of the water economy may identify unexpected outcomes on general standards of living of the beneficiaries. Figure 9: Households Concerns about Their Challenges to Water Access 70 person’s ability to read and write. However, because few target women can read and write, there were only three women out of 18 members in the three WMCs formed in the three locations in Hadramount governorate, Romah District. IRD plans to develop simple, tailored training programs to target rural and poorly educated populations in order to effectively prepare people to work on development of both hardware-infrastructure and software-WASH programs. Water Quality Most beneficiaries (82 percent) said that they did not treat their household water supply in any way. Among those taking water from a reservoir or bladder (presumably IDPs) none (0 percent) claimed to have treated it (perhaps believing it to be already treated, which it may have already been, while in the bladder). Given that most water sources in Yemen are from below ground water, which does not typically require as much treatment, this may influence the 82 percent who reported in the surveys that they did not treat their drinking water. Meanwhile, IPs do not appear to be offering point-of-consumption or point-of-use options (i.e. water-purification goods, such as aqua-tabs or chlorine). Research could be conducted to identify the bacterial contamination levels in household-level water storage vessels. Sanitation Sanitation may represent a larger gap than water supply, despite the arid environment. As seen in Figure 10, 15 percent of respondents stated that they either had no access to any latrine or as in nine percent of the cases had a latrine without sewage. Latrines for displaced populations are slow to build, particularly for former IDPs in areas of return,. Hygiene was conducted primarily through outreach campaigns, though few beneficiaries appeared to have changed any behaviors because of the campaigns. Among the households surveyed in this Formative Evaluation, 96 percent stated that the hygiene campaigns were not helpful. Interestingly, as seen in Figure 11, underlying the relationship between whether good hygiene practices (i.e. good hand-washing) appears an inversely relationship literacy, which is counter to what might be expected. From all the areas surveyed, breaking down those who say they practice hand-washing by education level, those with education consistently say they do not wash their hands, whereas the majority of those who are illiterate do wash their hands. This raises questions, which can be pursued in the Figure 10: Latrine Access in WASH Target Areas 71 Summative Evaluation, about whether there is covariance in this indicator by IDP status, or some other common independent variable. A Dearth of DRR & Innovative Approaches to WASH Ninety percent of key informants from the IPs felt that there was a lack of DRR, risk-mitigation approaches or other innovative approaches in WASH programming. This suggests a relative lack of attention to longer-term, durable solutions to address the WASH gaps in minimum standards of vulnerable groups. Cross-cutting innovation and longer-term planning, OFDA-supported projects are mixed insofar as any indications of recovery or transition changes from emergency WASH to long-term sustainable models. Line of Inquiry 3: Implementation Lessons, Strengths/Promising Practices and Areas for Improvement LESSONS LEARNED Sufficient Quantity but Bad Water Quality A whopping 82 percent of respondent households said they depend on water that is not treated in any way, and none (0 percent) of those respondents who rely on a reservoir or bladder as their water source said that they treat it in any way. Yet, most of the target populations appear to cope adequately with the arid condition where they reside: only 13 percent said that the supply of water was insufficient. A majority of respondents for this survey were households living around health centers. A strategy of many OFDA funded programs has been to install or repair water points at health centers. Twenty-seven percent of host communities get water Figure 11: Pattern of Association between Hygiene and Education among Respondents. An inverse relationship was observed in the data between hygiene practice and level of education. This might be explained by an unseen covariant factor. 72 from a pipeline connection to their houses (through taps) or they supplement their supply by purchasing water from private sources (60 percent). Thirteen percent of beneficiaries surveyed are living in camp settlements where they obtain water through water trucks. Almost all respondents had points of water within 500 meters of their dwellings, which represents progress for this indicator. Figure 12 below describes the statistical responses from beneficiary surveys about their use of water sources. At water access points, most respondents felt free from threats. Only four percent of respondents said they suffered harassment, insecurity, and bullying, and 13 percent claimed some form of conflict or risk around water points. The UN Cluster Helps Compensate for Policy Framework Vacuum Even though there is a lack of a coherent policy framework in the WASH sector, there have been efforts by IPs to stimulate information exchange and operational collaboration between central and local government agencies, NGOs, and donors through the WASH Cluster system. The Formative Evaluation found that IPs are pursuing a measured, step-by-step approach to build understanding, acceptance, and coordination among their different WASH stakeholders. This allows unique access to affected populations that are beyond the reach of many IPs due to security and access issues. STRENGTHS/PROMISING PRACTICES Figure 12: Where households in program areas say they obtain their water supplies 73 Cell Phone Technology Innovation for WASH BCC There is now an emerging SMS-broadcast technology option, introduced by GC. This is being tested to disseminate key hygiene messages to target communities for BCC purposes via the mobile phone network and overcome challenges posed by distance and insecurity. Results of this pilot from GC would be an interesting innovation to replicate in other areas by other IPs. Community-based Networks Help Emergency Preparedness UNICEF has built networks with partners, resulting in better coordination in responding especially to emergency needs of communities. In addition, it has prepositioned WASH supplies at many community levels as part of emergency preparedness plans to respond more quickly to urgent needs among affected communities. Local Involvement Supports Successful Implementation Most IPs (UNICEF, IRD and IOM, with OFDA support) are working with communities to establish grassroots mechanisms to solve water resource needs, such as village level WMC. The committees are selected through community consultation and approval of local community leaders and were charged with ensuring gender balance and inclusiveness of all groups. The strategy was to help to create a feeling of ownership and strengthen local capacity to solve problems like involving vulnerable groups like Bedouins and/or migrants in projects like IRD’s in Hadramout. IPs say that active involvement of IPs in the local communities in which they are operating created reliable and participatory community networks and a strong understanding of the fluid environment of the Yemeni context and its effect on vulnerable families, enabling more efficient implementation and more effective targeting of program activities. For instance, IRD staff facilitated meetings with the representatives and members of the community. Each committee is composed of six to eight elected members, with at least one being female. IRD involved these WMC members in carrying out baseline surveys about how water is used within that community. Fifty-six percent of households surveyed confirmed the existence of the WMC and their responsibility for maintaining the dam in good condition and for prolonging its life as a sound, useful water resource. The Formative Evaluation team further asked water users how their water facilities are operated and maintained and whether they are satisfied with roles of community WMC. Across the surveyed WASH sites, 61 percent stated they are satisfied with the formation and roles of WMCs. Another 34 percent said they are doing very good work and one percent indicated that they did not know if there was anyone who manages and maintains the water. Figure 13 gives a summary of responses received concerning operation and maintenance of water points. WASH Training in Aden by Global Communities. February 16, 2014. 74 The WASH survey also sought to establish the nature of community involvement in the selection of WMCs and the specific role they play in the management of water infrastructures. Given that between five and seven members of the community are typically elected to manage a water facility, this can be considered a critical proportion of the population. However, 80 percent are made up of men and only 20 percent of women. The nature of beneficiary involvement ranged from membership in WMCs, cleaning water point areas, fencing, community mobilization, regulating water usage, and participating in the decision making process. Ethnic Tensions Can Inhibit IP Work For one IP (IOM), Southern sentiment against Northern citizens contributed to difficulties finding suitable field staff. After recruiting available local staff, the IP gave them close support and follow up from its offices in Aden and Sana’a and established good working relationships with local authorities at the governorate, district, and local levels to enhance activity implementation. Also, there were issues of tribal disputes in program areas regarding water rights. Shortages of Skilled Human Resource Weaken Service Delivery A shortage of skilled human resources in health facilities led to shortened working hours at centers (only being there from nine to noon each day). Meanwhile, most IPs reported limited government involvement to take the lead in coordinating WASH maintenance alongside IPs. IMPLEMENTATION IMPROVEMENTS NEEDED OFDA and FFP-funded WASH interventions were being implemented according to work plans, with few major delays, problems recorded, or complaints. Any delays were due to security conditions. Qualitative and quantitative data confirmed satisfaction with most of the projects’ implementation mechanisms and mostly with the quality of the services and goods. Key informants among the target populations and water user groups confirmed that their IPs have adequate technical capacity and staff to deliver their intended Figure 13: Perceptions of how well the Water Management Committees manage water resources, according to users 75 WASH services. An analysis of all of the data by the evaluators found that IP staff have been generally adequate for certain tasks, including support for the water supply and storage needs of populations, but less so for crafting effective hygiene interventions. All IPs, except UNICEF local partners, have adequate technical staff (e.g. engineer and hygiene promoters) and have technical knowledge and capacity in the delivery of quality WASH services. Technical Design Review of IRD Sand Dams Project The IRD project to build sand dams in the Governorate of Hadramut is said to be the most cost-effective form of rainwater harvesting and provides targeted communities with a clean, local, and reliable source of water even during periods of drought. IRD uses a local engineering team and partner, the SFD, with experience based on having implemented similar dams for rainwater harvesting. However, key informants among IRD staff confirmed59 that there has been a change from the original technical design of actual sand dams to what is called a “water reservoir," known locally as a water “caravan”. This evaluation proposes that the design be reviewed to understand the environmental sustainability aspects of the water caravan Infrastructure that had compelled IRD to change their original design. Prioritize Hygiene Promotion Interventions Considering the “buy in” of the WASH packages (hard infrastructure plus local knowledge) by the beneficiaries/communities, 100 percent of the key informants from the IPs acknowledged that the WASH packages are implemented commensurate with the relevant needs of the targeted communities and standard practices (i.e. software packages were implemented after the hardware components). Examples are the civic-minded individuals within the targeted communities who were identified and trained as community volunteers by IMC, SC and IRD in hygiene promotion and education. For instance, IMC in cooperation with local authorities and community leaders identified and trained about 400 community hygiene volunteers in the high￾risk areas of Al Haymah Al Kharijiyah, Jahana districts. The training focused on good hygiene and water use practices and included basic messages that promote good hygiene behaviors such as regular hand-washing, proper collection, storage and use of drinking water, proper waste disposal, and proper handling of food. Specific, culturally relevant messages were developed for different target groups including those at risk (children, mothers of young babies, adolescents and people preparing food) and those who can influence behavior change (imams, community leaders, teachers, health professionals). Compared to other WASH interventions (such as infrastructure), hygiene promotion is generally a lower cost set of activities, based primarily on communications. As elsewhere in the world, the hygiene promotion interventions in Yemen encouraged good daily household practices such as hand washing with 59 The OFDA Technical Advisor in Washington and OFDA Yemen team were informed about the background in technical design changes. This was supported by a letter from Social Fund for Development (SFD), who are the biggest Sand Dams implementers in Yemen. Health center staff and beneficiaries at Al-Sha'abaniah, Taiz. WASH activity by Save the Children. February, 2014. 76 soap, confirmed by the communities (48 percent of households surveyed), which is effective in reducing diarrheal morbidity. Evidence from various studies found that hand washing with soap cuts the incidence of diarrhea by nearly half. Thus, some might argue that hygiene promotion intervention, as opposed to WASH hardware, should be elevated in priority within the WASH sector in order to bring about significant change with minimum investment. IMC established hygiene promotion corners within clinics, through hygiene promotion leaflets, posters, and other information, education and communication (IEC) materials. The Formative Evaluation determined that IMC distributed hygiene kits that included essential materials for ensuring good hygiene practices. Only the intended beneficiaries (severely malnourished children) who visited the 12 health centers were targeted. Distribution did not occur at household levels at home nor in the communities more widely. This may help to explain why 87 percent of respondents said they do not participate in hygiene kit distributions or receive WASH supplies. MC distributed about 2,400 hygiene kits, in total, to families of malnourished children. Each kit contained the following items: Item Quantity Soap for bathing (125 gm.) 20 pieces (pcs) Dish Detergent (450 gm.) 2 pcs Shampoo 2 liters Sponge for cleaning dishes 4 pcs Tooth Brush 6 pcs Tooth Paste 4 tube Hand towels 4 pcs Water storage vessels 2 pcs Menstrual hygiene materials 6 pcs Key informants from IMC, SC and IRD confirmed the involvement of community leaders in hygiene promotion through established community groups, schools, mosques, community centers, etc. However, looking at the beneficiary levels of participation and knowledge, 68 percent of those surveyed said they had never participated in any hygiene promotion activities as opposed to 32 percent who said they had. When those who said they had never participated in hygiene promotion were asked why, 39 percent said there were no hygiene promotion activities while 26 percent responded they had never been contacted. This probably reflects the IP’s strategy for implementing hygiene promotion activities by targeting some groups and not the whole population. SC and IMC, for example, targeted specific cohorts of the community in their WASH outreach. SC targeted expectant mothers during pregnancy who often come to the health facilities and advised them on how to protect the health of their children and themselves. Special attention was also provided to catchment areas within the health facilities, especially in areas where there was high incidence of diarrhea (or water washed diseases). Such a specific targeted approach has limited efforts to apply Participatory Hygiene and Sanitation Transformation (PHAST) methods in all hygiene promotion activities. 77 3.4 PROTECTION SECTOR INTRODUCTION Landmines and UXOs present a persistent, day to day threat to conflict-affected populations in Yemen. Children in conflict affected areas are at daily risk as they are prone to mistake UXOs for toys to pick up when found. Boys are at the greatest risk due to their movements and exposure outside the home; culturally, girls are more restricted to their homes. This sector is distinctive in the OFDA portfolio in that the only protection project is one award to UNICEF for MRE. It is part of a largely macro-award to UNICEF, under its general “appeal.” The protection component is the smallest of the three UNICEF components that USAID funded in 2013-14, the other two being nutrition and WASH. Significantly, OFDA funding constituted almost all of the UNICEF appeal for MRE. UNICEF appealed for $1.5 million and OFDA provided $1.5 million in 2013. Later, Japan donated on top of this, so, while OFDA funded 100 percent of the appeal, their contribution represented 85 percent of the donor pool. The landmine/UXO issue falls into the larger category of protection. In Yemen, a large share of the other protection efforts are conducted by UNHCR, which is primarily funded by the US Department of State, not USAID.60 Sector Results Summary Narratives of the brief indicators below follow the chart. Note that FFP does not provide support for protection activities. Table 14: Protection Indicator Results Summary 1- What is the project doing to ensure that it reaches the beneficiaries most in need? What are the challenges to reaching the beneficiaries most in need? How have implementing partners overcome those challenges? Indicator OFDA FFP Defined selection criteria adopted by IPs. 8 out of 8 n/a Percentage of respondents confirming knowledge of selection criteria. 78% n/a Review and/or monitoring mechanism for conformity to ‘selection criteria’ put in place by IPs. 100% n/a Percentage of respondents concurring beneficiary compliance with selection criteria. 100% n/a Percentage of respondents confirming above average effectiveness of targeting methodology in reaching out to the neediest groups. 95% n/a 2. Are there significant needs within OFDA and FFP’s operational sectors that are not currently being addressed or met by OFDA and FFP programming? Indicator OFDA FFP Percentage of respondents confirming technical appropriateness of IP programming by sector. 100% n/a 3. What are the most significant lessons learned about the implementation process of OFDA and FFP programming from the monitoring process to date? What are the most significant strengths of and areas of improvement for OFDA and FFP programming? 60 While Yemen is a signatory to the 1951 Refugee Convention and 1967 Protocol and hosts more than 240,000 refugees from neighboring countries, the assistance to refugees is coordinated by UNHCR and funded by the US Department of State and distinct from the assistance programs for the 230,000 IDP Yemenis. 78 Indicator OFDA FFP IP projects are on plan with no major delays, problems or complaints recorded. 7 out of 8 n/a Percentage of respondents61 confirming above average satisfaction with the projects’ implementation mechanism. 95% of UNICEF’s IPs expressed above average satisfaction with the project implementation mechanisms. n/a Percentage of respondents confirming above average satisfaction with the quality of services and goods. 95% of the children and 60% of the teachers. n/a Background - Status Most target outreach plans in the MRE projects funded in 2013 were completed as of the Formative Evaluation fieldwork as described in the table below. Funding was continued in 2014 but was not underway at the time of the Formative Evaluation. Table 15: MRE Implementation Status IP Project Name Start & End Date Implementation Status UNICEF Mine Risk Education for Conflict Affected Populations Jan 01- Dec 30 2013 Completed DRC* Mine Risk Education for Conflict Affected Populations Jan 01- Oct 30 2013 Completed YEMAC* Mine Risk Education for Conflict Affected Populations Completed Democratic Schools * Mine Risk Education for Conflict Affected Populations Completed Bureau of Social Affaires* Mine Risk Education for Conflict Affected Populations Completed Yemen Women’s Union* Mine Risk Education for Conflict Affected Populations Completed Child Protection Department * n/a Completed * UNICEF’s implementing partner The Protection and Landmine/MRE Context Government and rebel forces have been accused of burying landmines throughout Yemen between 2004 and 2012, including in the Sa’ada Governorate in the north and Abyan in the south. The Yemen Ministry of Defense has recently reported landmines casualties in the country’s northwest coast governorate on Hajjah and reportedly, in 2011 and 2012, anti-personnel mines were used in and around Sana’a. According to UN reports, the number of civilian casualties due to contact with UXOs increased significantly in the third quarter of 2012, particularly in Abyan Governorate in the south, following an announcement by the government that they had ousted militants, which resulted in a large number of people fleeing the conflict to return home to heavily mined areas. Yemen’s population is young, with an estimated 42 percent of the population under the age of 15. According to UNICEF, at the beginning of 2012, 49 children were victims of mines/ UXOs and ERW (10 boys and four girls verified killed, and 34 boys and one girl verified maimed). This number far exceeds the figures reported in 2011, when15 children were reported affected (10 killed and five maimed). 61 This included those surveyed and interviewed stakeholders. 79 During the SIs, a YEMAC official in Hajjah governorate reported that within the past 12 months, three children had sustained injuries from playing with a UXO. Also, in Aden governorate, a government official reported that five children sustained injuries and one child had died within the past month from a UXO. Both officials felt that the curious nature of children puts them most at risk. The Formative Evaluation included questions to teachers involved in the program about where landmines are believed to exist and where children might come in contact with the mines. Five out of 14 said landmines exist “in the fields” or open areas. One cited water collection points. The others appeared to have no awareness of where landmine hazards might exist. OFDA Objectives In accordance with USAID/OFDA’s global mandate of saving lives and alleviating the suffering from conflict among affected populations, USAID/OFDA funded UNICEF $1.5 million in FY 2013 to provide training and awareness about the risks associated with landmines, UXOs and other ERWs. UNICEF had proposed to target 200,000 children (equal numbers of girls and boys) and 150,000 adults (equal numbers of women and men) to receive landmine awareness training and advocacy, and to train up to 300 teachers and community volunteers (both men and women) about how to teach MRE. They also proposed to extend the integration and coordination of MRE activities in early recovery and education clusters and ensure effective planning, monitoring and response. Data Sources Table 16: MRE Sample Frame and Interviews Type of Interviewee Name of Organization/Number of individuals Eight IPs 3 government partners (YEMAC, Bureau of Social Affairs, Child Protection Department), 3 local NGOs (Democratic Schools, Yemen Women’s Union, DRC), and UNICEF 40 students and 14 teachers No. of Individuals: 54 Structured interviews with IPs No. of Individuals: 6 PROTECTION SECTOR FINDINGS Line of Inquiry 1: Coverage UNICEF’s objective was to disseminate information on the dangers of landmines and ERW to 200,000 children. UNICEF reported that they had met this target. UNICEF is implementing its MRE program in collaboration with the YEMAC with international NGOs (i.e. the DRC, and with local NGOs (i.e. Democratic Schools). To ensure that the beneficiaries who are most in need are reached, UNICEF has also adopted various strategies including broadening partnerships with other key government ministries, such as the MOSAL. YEMAC, DRC and Democratic School (local NGO) are UNICEF’s IPs conducting MRE for teachers. In areas outside of government control, UNICEF and IPs are engaging with locally appointed rural councils and leaders to promote MRE. In order to increase access to MRE for girls and women, UNICEF‘s partners are making efforts to recruit and train female community volunteers as trainers. Similarly, some IPs adopted a house-to-house approach when providing MRE to conflict affected populations. This approach not only ensures access to girls and women but also to school age children who are not enrolled in school. 80 Furthermore, UNICEF and IPs have also developed criteria for selecting MRE committee members so as to enhance the committee acceptance in the local community. MRE programs gave priority to IDP camps first and then schools in areas that are considered most affected by landmines or UXOs. Vulnerable populations are classified as those of IDPs or returnees or those that are conflict-affected. In the summertime, the program also targets khat sessions. Cultural norms have also played a significant role in the implementation of MRE. In some culturally conservative districts, MRE teams have faced fierce opposition from religious leaders opposed to MRE activities. However, within these groups, boys are considered more vulnerable due to their tendencies to be more mobile. However, with regards to training teachers, the selection criteria seem to vary from one governorate to the next. For example, in Aden Governorate, it was reported that both the head teachers and YEMAC were responsible for selecting teachers to participate in MRE while in Hajjah Governorate, it is the Bureau of Social Affairs that is responsible for selecting field staff and teachers. The teachers are selected on the basis of their availability to engage in MRE and their ability to have a relationship with the local council leadership. Seventy-eight percent of the teachers interviewed confirmed knowledge of the selection criteria; 28 percent confirmed selection through a committee process. Among the 14 teachers surveyed, six reported the selection process as good and five as not good. A review and monitoring mechanism for MRE appears to be in place. UNICEF reported that it conducts regular spot checks on projects implemented by its IPs. In addition, UNICEF’s IPs submit monthly project status reports to UNICEF through an agreed upon format. The reports are then shared with OCHA for wider circulation to the humanitarian community. This evaluation mechanism is effective in measuring the quantity of the targets reached but not the quality. Transparency and Accountability The inclusion of local communities in the design of the interventions affecting them has been inadequate. Communities are not aware of the selection criteria for teachers or schools to participate in MRE. According to a government informant, an official from YEMAC was responsible for selecting schools to participate in MRE. He said the procedure for selecting schools was transparent, yet the community was not involved in the process. Daily Exposure by Children The formative research explored the daily activities of children and how these might shape their exposure to landmine risk. Of the children interviewed, 68 percent spoke about assisting with chores around the house while 25 percent Figure 14: Children's Time Use 81 said they assisted with chores such as collecting water. In addition, nine percent of the children reported playing in open fields while 43 percent play in either designated areas or do not play outside at all. A total of 40 children were interviewed, 60 percent male and 40 percent female. Line of Inquiry 2: Programming Gaps The Formative Evaluation identified these gaps:  Missing the Opportunity to Educate the Whole Family  Coordinating MRE with De-Mining  Not Obligating Teachers to Use MRE Training and Appear to be Ineffective Missing the Opportunity to Educate the Whole Family Given the high proportion of Yemen’s population who are adolescents and children, it is understandable that the MRE outreach targets largely school age primary school children under 16. However, the UNICEF proposal targets boys and girls (200,000) along with women and men (150,000) for MRE education in 2013. Meanwhile none of the IPs have put in place programs that target the disabled or elderly. Coordinating MRE with De-Mining According to UNDP, de-mining activities should be conducted in parallel with MRE. There were instances reported by YEMAC where community leaders refused MRE unless de-mining happened at the same time. However, YEMAC lacks the capacity and financial support to maintain parallel de-mining activities. While UNDP is a lead player in demining, its leadership in cluster coordination is lacking. None of the IPs are involved in demining, in large part because the Government has not permitted outside NGOs from being involved. Not Obligating Teachers to Use MRE Training UNICEF enlisted YEMAC, the DRC and the Democratic School (local NGO) for MRE training for teachers. YEMAC trains groups of community volunteers, NGO staff and teachers to conduct MRE campaigns. Teachers work during campaigns. They are not obligated to conduct regular MRE awareness sessions for students at their schools. However, they are encouraged to do so. The DRC conducts trainings in school for children. Children are trained in small groups within the presences of their teachers. This training methodology exposes teachers to MRE. Teachers are then a focal point for follow up on any issues raised locally, after the training. UNICEF explained that with this approach, if a child sees a UXO on the way to school, the child knows which teacher to contact and then the teacher can notify the DRC. The Democratic School implementing activities in Hajjah governorate facilitates training for teachers in schools. The actual training is conducted by YEMAC and teachers are requested to conduct regular MRE sessions for students in their schools. Of the teachers interviewed, 36 percent reported having received training on the risks associated with MRE and ERW. However, 40 percent did not know the organization that provided the training. While 60 percent of the teachers who received training rated the overall training as “good,” 80 percent reported that the length of the overall training was not adequate. Most (75 percent) responded that three days of training would be adequate; 60 percent reported that the training was less than two hours in duration. Furthermore, 64 percent of the teachers who received MRE training apparently had not engaged 82 in any MRE related activities; their newly acquired knowledge and skills had not been used. The extent to which teachers are involved in MRE activities after the trainings appears to depend on the objectives of the organization that is facilitating a given training. The training objectives seem to adhere to UNICEF’s IPs’ objectives. For example Democratic Schools trains teachers and then requests the teachers to transfer the knowledge to students in their schools; however, YEMAC does not have the same requirements. Line of Inquiry 3: Implementation Lessons, Strengths and Areas for Improvement LESSONS LEARNED/STRENGTHS/PROMISING PRACTICE Overall, the MRE program seems to have gotten the message across effectively to the target children. UNICEF’s model works. As seen in Figure 145, almost every child gave appropriate answers in the survey. Working with Government: Opportunities and Challenges The three government agencies that are collaborating with UNICEF in the implementation of MRE reported an overall positive interaction with UNICEF. They also reported that UNICEF supported them with trainings for their staff, teaching aids, training guides, brochures, posters, promotional items and per diem for their field teams. However, they felt that this support was not always adequate, such as when they requested cameras to enable the teams to document their activities (these were not then provided). Furthermore, while the teaching aids were highly appreciated, there was concern expressed regarding the delay in delivering the materials. It was also recommend that the teaching aid materials should include games for children since games can be very engaging for children and therefore, heighten their overall interest in learning about the dangers of mines and UXOs. A government official recommended that MRE should be introduced into the school curriculum given the extent of the problem and the extent of harm Figure 15: Responses from Children about what they learned about how to react to possible landmines or UXOs 83 evident in the daily lives of Yemenis. This could be a real opportunity to institutionalize MRE so that it is sustainable and address the finding that teachers who received MRE training do not necessary transfer the knowledge. IMPLEMENTATION IMPROVEMENTS NEEDED According to interviews with IPs of UNICEF and beneficiaries, they are mainly on target with what they had planned. Teachers and IPs expressed high satisfaction (90 percent affirmative) with the MRE-related services that the IPs provided. Teachers’ Views about Benefits Of the 14 teachers interviewed, all felt that students are improving or have improved in their awareness of and now understand landmine risk, and will adjust their behavior to avoid landmines. As one teacher said, “students received a lot of benefits and they are becoming aware.” UNICEF has field offices throughout Yemen to support the delivery of the proposed activities. UNICEF supplies teaching aids for MRE though there is a shortage of materials associated with delays in UNICEF receiving the items in country. UNICEF enlisted, YEMAC, DRC and Democratic Schools to provide MRE for teachers. In general, the MRE IPs (sub-grantees to UNICEF) are satisfied with their level of coordination with UNICEF. Key findings from the formative field evaluation are:  97 percent of the children reported receiving information on mines and ERW at school.  60 percent of teachers who received training rated the overall training as “good.”  64 percent of the teachers who received MRE are not engaged in any MRE related activities.  80 percent of the teachers who received training reported that the length of the training was inadequate.  64 percent of the teachers who received MRE are not engaged in any MRE related activities.  95 percent of the children reported knowing about mines and ERW.  97 percent of the children reported receiving information on mines and ERW at school. Government entities responsible for selecting teachers and children for MRE vary from one governorate to another. Communities participate in MRE activities but the participation is limited to project implementation only, and communities are not involved in the selection of teachers and schools for the inclusion of MRE. MRE is Effective but Could Improve Further As seen in from the quantitative survey data, in Figure 15, the impact of the MRE outreach to children led to the desired results. 84 Aggregating their answers, a high proportion (95 percent) of the children reported knowing about mines and ERW and 90 percent responded that they would avoid contact with a suspicious foreign object lying on the ground. A higher proportion (97 percent) of the respondent children reported receiving MRE information at school and 26 percent reported receiving information on mines from a teacher; 65 percent reported receiving information from additional or other sources. Even though 95 percent of the children interviewed felt that the MRE training was effective, an official from the Bureau of Social Affairs reported that including activities such as games in the trainings might increase MRE training effectiveness. A relatively high percentage of children reported receiving MRE from schools and also felt that the training was adequate. Along similar lines, 60 percent of the teachers who received MRE were generally satisfied with the overall experience. However, the overall objective for training teachers is not well defined. The UNICEF project document aims to have “300 teachers and community volunteers both men and women trained on MRE”, but their expectations after the training are not indicated. The theory of how this activity will lead to changes in landmine outcomes is not articulated, and therefore difficult to measure or evaluate. 85 4. CONCLUSIONS, LESSONS, AND RECOMMENDATIONS Based on an interpretation of the findings detailed in the previous sections of this report, this section presents the conclusions, which synthesize and draw inferences from those findings. The conclusions include an evaluation of the development hypothesis and assumptions stated or implied in launching the interventions. This section further lists the associated recommendations that the Formative Evaluation makes to OFDA and FFP, based on the evidence associated here and with the related conclusions. Where appropriate, the conclusions include distilled lessons, which are actionable principles for how aid programs may be carried out in the future.62 The conclusions blend the various strands of evidence and weigh the strength of the evidences. Many of the raw findings from the Formative Evaluation inquiries may entail some degree of bias due to the perceptions and motivations of the respondents who may view a link between their answers and the prospects for more future aid resources. This is frequently the case, for example, when asking beneficiaries about food assistance, where vegetable oil, wheat and other products can be monetized by the family and therefore may be seen as an income transfer for livelihood support. 4.1. AGRICULTURE/FOOD SECURITY AND ECONOMIC RECOVERY AND MARKET SYSTEMS Line of Inquiry: Coverage Because the planned agriculture and ERMS interventions have been in the initial stages of roll out, there are as of yet few lessons learned regarding inclusion of those most in need. However, both IOM and GC noted that while their project designs have not specifically targeted at youth, they now recognize that the youth are a critically important vulnerable population, and both IPs have begun to more systematically incorporate them in CVCs and plan to include them in project activities. As assistance agencies shift over, a gap is evident in the employment and livelihood-engagement needs of young people. Approximately 23 percent of the population in conflict areas is between 15 and 24 years old.63 Surveys helped identify additional vulnerable households not on the Government lists, and in turn improved the quality of these lists. In the end, the selection process was deemed adequate, transparent and fair, although respondents felt that more beneficiaries should be included overall. Recommendation 1. The beneficiary selection criteria should allow for greater flexibility in terms of household size and the dependency ratio. The current, iterative process of household 62 A “lesson learned” is a generalization that does not refer to a specific circumstance but to a class of situations (e.g., to livestock projects for the rural poor in the arid areas). It points out what is very likely to happen and/or on what should be done for something to take place (or to prevent it). The ADS Glossary, Chapter 540 (2012) adds that lessons learned are “conclusions extracted from reviewing a …program or activity by participants, managers, customers or evaluators with implications for effectively addressing similar issues/problems in another setting.” 63 UNDP (2013). Desk Study – Multi-Dimensional Livelihoods Assessment in Conflict Affected Areas. 86 selection should be expanded to new areas where it is not currently being used, but allow small households meeting specific poverty and vulnerability criteria to be included on the beneficiary lists. Line of Inquiry: Unaddressed Operational Needs Vulnerable households within the existing program areas are excluded from key activities. At the same time, expansion in many of these regions into livelihoods and markets will be of interest to more powerful local stakeholders. Currently, the main obstacles to reaching the most vulnerable groups are the continued violent conflict alongside a generalized climate of distrust, particularly among the conservative, obstructionist tribal governance structures. Recommendation 2. Because so many vulnerable households are excluded, OFDA or FFP may encourage IPs to design programs to assist a greater number of households. The USG may: 1) increase funding levels to cover more households in need, or 2) encourage or require that IPs identify gaps/shortfalls and connect households with other relief/humanitarian assistance, whether from a local government program or from a donor funded program. Focus group respondents articulated a desire/need for seeds, tools and fertilizer for recovery. There also were apparent gaps in access to financial services to cover the borrowing needs of new investors in agricultural production and processing. Such micro-investments are a core dimension of OFDA’s ERMS portfolio and emphasized in the SEEP Minimum Economic Recovery Standards.64 Recommendation 3. Limited support for seeds, tools, fertilizer and financing may effectively leverage the kick-starting of local supply and value chains, particularly for returnees, which would allow displaced and dislocated households to re-establish a viable income. Line of Inquiry: Significant Implementation Lessons (Strengths and Areas for Improvement) Vulnerable communities are now in a position to adopt enhanced market-recovery interventions, assisting with seeds, locally-appropriate livestock, veterinary services, etc. Recommendation 4. ERMS programs should be scaled up faster and in concert with the shift from relief to development, but building on the models that work. The use of vouchers should be carefully studied to understand its effect on local markets and the pace of value chain recovery. WFP intends to move to voucher systems, having done it elsewhere in the world. In Yemen, NGOs have new and useful experience about how and where vouchers do and do not work. 64 These standards were developed by NGOs with OFDA funding, via the inter-agency small enterprise, “SEEP” network. See: www.seepnetwork.org/minimum-economic-recovery-standards-resources-174.php. Food Voucher given to a beneficiary, taken in Modea, January 2014. 87 Recommendation 5. FFP should support WFP, IOM and IPs to come together to share experiences and models in ERMS, including market-based (voucher) mechanisms to address livelihood needs. 4.2 HEALTH AND NUTRITION Line of Inquiry: Coverage in Food Assistance Beneficiary lists are largely effective, as a way to engage the communities, and promote transparency. However, IPs did not adequately inform their target communities about their rights and eligibilities related to the project activities. The low level of awareness of community members of the selection criteria for food distribution indicates that proper awareness raising activities were not accomplished. Specifically, IPs and local leaders did not disclose the selection criteria to the community members so that they were aware of why they were included in or excluded from food distribution. Recommendation 6. IPs should share positive and successful lessons among themselves about how communities were engaged in the development of criteria and a sense of community ownership. A key issue for discussion is the appropriateness of publicly posting the names of beneficiaries selected, which involves tradeoffs between transparency and protection-associated￾with-anonymity. Better communication of rights-based entitlements (i.e. what to expect in ration distributions) can be achieved. For example, 69 percent of respondents said that they did not know what ration size/composition that they were entitled to. Recommendation 7. Not all beneficiaries need to be identified and formally selected in the initial phase of assistance. Rather, by delaying the selection of a certain proportion of beneficiaries to a later date, the project can accommodate those who inevitably had been uncounted or otherwise wrongly excluded. Line of Inquiry: Unaddressed Operational Needs related to Food Assistance and Nutrition Large gaps remain in the proper identification and inclusion of malnourished children into targeted feeding, and their subsequent referrals between levels of recovery feeding. Tracking of individual malnourished children does not occur and there is an absence of appropriate monitoring tools that indicate not only how much of the therapeutic products have been provided, but also how the therapeutic foods are administered to malnourished children on a daily basis. Some projects with strong IYCF components will not be able to meet their objectives and produce results in one year. Overall referral systems for malnourished children do not exist in most locations. There has been inadequate communication among and between USAID’s IPs and coordination gaps between IPs and government offices. For example, the beneficiary identification process and the subsequent transfers of support vary among IPs, and IPs used different strategies to reach communities, particularly those in remote locations. The lack of health program integration, such as between IYCF and CMAM, SAM and MAM, and ESN and Health Education, has resulted in missed opportunities for the continuity of care needed for better outcomes. 88 Recommendation 8. More support for nutrition products can help meet the needs of the vulnerable children. As WFP draws back on the delivery of large-scale food ration distribution, WFP might consider simultaneously transitioning a greater share of its portfolio to targeted/supplementary feeding of malnourished children in order to expand the proportion of children who benefit from a coherent referral system with UNICEF. All children who are discharged from UNICEF-supported therapeutic feeding (management of SAM or are found in the course of UNICEF case finding and surveillance ought to be entered into supplementary feeding centers and remain there until after the child recovers above the threshold (two standard deviations) to “mild” status. This Formative Evaluation found that many children fell through the cracks and did not get properly admitted or retained in an integrated system of therapeutic and supplementary feeding, along with home follow-up or family education. Too little is known about the urban/rural and cultural dimensions of default. Sphere states that the minimum standard for default is 15 percent. However, countrywide, the default rate in Yemen is estimated at 34 percent.65 IPs should give particular attention to urban areas, where it is often wrongly assumed the default rates are lower. In fact, distance to market, or distance to feeding center is not the over-arching force influencing the default.66 Many beneficiaries cited the small lot size of food when distributed as a challenge for them. Current distribution includes 10 kilograms of WSB and one liter of vegetable oil, whereas recipients would prefer 25kgs of WSB and four liters of oil. There may be a lack of communication to beneficiaries about international norms for large scale, population based, emergency food programs. Recommendation 9. Communications should be created to explain widely accepted SPHERE minimum standards, the rationale for the lot size given, and the reasons why food is bagged and shipped one way, and distributed another way. Research may be undertaken about the feasibility of distribution to groups of households, instead of each household, and whether this can create efficiencies with regard to keeping bags and cans intact for transport and loss-reduction purposes. The research issue is whether whole-bag or whole-vegetable-oil-container distribution would have the primary effect of increasing the commercial sale or leakage of these items by the recipients. Not enough is known about how specialty foods are used post-distribution, including leakage or sharing, nor what the cost-effectiveness comparisons are in Yemen between different commodities. Recommendation 10. USAID/OFDA and FFP may request that IPs, and its own monitors (YOFMEP), put in place a monitoring mechanism to ensure that beneficiaries are utilizing the therapeutic products and the food sources as recommended. One such system may be a form of post-distribution monitoring that tracks back to the household level. Line of Inquiry: Significant Implementation Lesson (Strengths and Areas for Improvement) 65 The Coverage Monitoring Network (CMN), 2014. 66 Saul Guerrero, Koki Kyalo et al. “Debunking Urban Myths; Access and Coverage of SAM-treatment Programs in Urban Contexts.” Field Exchange Dublin: Emergency Management Network. 89 Targeted feeding for children who are acutely malnourished is a case of complementarity between OFDA and FFP. OFDA adds to FFP’s population-based ration feeding with support for more specialized and recovery feeding that is targeting the most malnourished. There has been a shift from item-based to value-based food baskets (i.e. through vouchers denominated in local currencies, not weight of food), where the beneficiaries have the freedom to choose from the 14 food items available. This will be an area for further exploration in the Summative Evaluation. Recommendation 11. As programs continue the shift from relief to recovery, a more rights￾based approach would make use of vouchers that confer more decision-making, as well as purchasing power, to the HH as opposed to IPs or community leaders. Again, USAID should encourage IPs to communicate lessons about beneficiary selection models. Line of Inquiry: Coverage Related to Health Remote populations are the key target group that are being reached, but need greater penetration, with IP assistance to health clinic partners. The designation of confirmed cholera cases as something else (i.e. “acute watery diarrhea”) inhibits effective technical attention from the international community, including the mobilization of WHO resources. In recent years, outbreaks of cholera have led an increasing number of national governments to formally shy away from allowing use of the formal term cholera, believing that it may scare away trade, tourism, and confidence in the government. The downside is that while Yemen has seen one of the world’s worst cholera epidemics in recent years, it has received minimal attention. This tradeoff has been similarly seen in countries like Zimbabwe and Ethiopia. Recommendation 12. OFDA and FFP should encourage frank discussion among IPs about which specific health conditions are not being adequately addressed, either in stand-alone vertical campaigns, or in the inclusion of integrated health programming. Cholera is only one example of this larger question. Project implementation begins abruptly and not always before advising the community about the project. Engaging the community is necessary for the success of the program, and that proper coordination should been done with government line offices. Recommendation 13. Health projects should allocate some greater effort in the start-up phase to amply explain to the target communities their objectives and the selection criteria of beneficiaries and should establish well-advertised grievance mechanisms. This will help to minimize the level of complaints and systematically handle grievances that will come along with exclusion from programs. Line of Inquiry: Unaddressed Operational Needs Related to Health There is a systematic and pervasive lack of capacity in the national health care system and an absence of health workers at health facilities for a substantial part of each day. In addition, the government is inadequately involved in interagency cluster meetings for health or nutrition. Building the local capacity is the first step in ensuring sustainability in the long-term, as Yemenis are in a better position to understand the local context. 90 Recommendation 14: USAID/OFDA and IPs need to coordinate with the federal government to bolster the harmonization of essential drug protocols and other standards that are implemented by government health workers, before they make home visits on a standardized package of topics, including IYCF, to improve the exclusive breastfeeding (EB) rate that is crucial for child nutrition, which is currently as low as 13 percent. Comprehensive referral systems and continuity-of-care are persistent gaps at all levels, within villages, between IPs, and between clinics and caregivers. Recommendations 15: As mentioned above with regard to food aid, the continuum of medical care, which includes malnutrition, is a failure between sectors too. USAID/OFDA needs to take into account the integration of various programs that are necessary for the provision of the continuum of care for better H/N outcomes. The integration needs to take place at all levels (i.e. at GHO, DHO, health facilities and FDPs) where the opportunities will be utilized for providing the continuum of care needed for the beneficiaries. USAID and IPs should support area-wide tracking of children, appropriate referrals, and use of road-to-health cards. Engaging the leadership at different levels (GHOs and DHOs) is needed to make sure that health authorities take responsibility for steering the entire health sector. This involves development of supervision checklists, regular monitoring, and follow up with incentive mechanisms in place for outstanding performances. Despite considerable effort by IPs in training, health facilities have asked for more training. Most IPs are working on system-strengthening mainly through the training of health workers. However, there is a lack of family planning services available to the target populations and VCHWs are mainly engaged in giving out information rather than conducting the proper counseling that takes the level of literacy into consideration. The information, unfortunately, is not based on the needs at the time of the visit. While a primary role of the VCHW’s has been communication, some of the VCHWs are inadequately qualified to educate communities (e.g. the quality of the information they gave was poor, and important topics like IYCF were ignored among some of those trained). Recommendation 16: As assistance transitions more toward recovery, a larger share of the assistance portfolio may shift toward improving the human capacities and primary care technical skills at the health clinics where IPs partner. IPs should train VCHWs on proper counseling techniques to make communication effective for behavioral change. The counseling should be timely, based on the problem at the time of visit. USAID/OFDA needs to monitor that IPs are closely involved in the selection process of health workers for the different trainings. USAID/OFDA should request that IPs train VCHWs with the knowledge and skills to be able to conduct effective counseling, and not just pass on information. In addition, USAID/OFDA needs to make sure that not only those who practice the behavior (i.e. the mothers) are counseled, but also those who can influence the behavior (e.g. grandmothers). USAID/OFDA has to make sure that the selection of trainees should not be left to the DHOs alone and IPs need to be involved to make sure that relevant staff are trained. The health cluster lacks sufficient guidance for IPs on topics such as essential drug lists, the containment of communicable diseases, or differential diagnosis in distinctive parts of the country. 91 Recommendation 17: Donors should together encourage ministry engagement in humanitarian country team meetings. Increasingly this should include, for advancing innovation and resilience, the Ministry of Technical Education and Vocational Training (MTEVT). Line of Inquiry: Significant Implementation Lessons (Strengths and Areas for Improvement) The flexibility that OFDA and FFP demonstrate in making project revisions every year is appropriate. Recommendation 18: One area where multi-year funding is called for is regarding behavioral changes in IYCF practices, which are unlikely to be met within a single year. As programs end or shift, USAID/OFDA should support a conscious review by IPs and specialists to document the cost-effectiveness of the models and activities conducted in the extreme conditions of Yemen, and recommend context specific lessons about how the government may take them to scale in the future. 4.3 WASH Line of Inquiry: Coverage In general, WASH programs reached many communities with successful increases in the reliability of water supply. However, hygiene interventions did not have an observable influence. Notably, every household surveyed had a water source within 500 meters of where they lived, which represents progress for this population. The approaches by the IPs are fundamentally sound for delivering on emergency water supply needs. IPs mostly have adequate technical staff to ensure effective WASH programs implementation. Recommendation 19: OFDA is expected to and ought to continue its focus on water scarcity as it relates to humanitarian goals in Yemen by promoting and raising awareness on integrated water resource management. Working in partnership with the IPs will effectively address water issues in the short and longer-terms. IP program target beneficiaries are primarily in rural areas with a specific emphasis on areas most adversely affected by conflict, including those that have hosted large numbers of displaced people and those with levels of food insecurity, water scarcity, and limited service providers (government or NGOs). Recommendation 20: OFDA should request that IPs verify beneficiaries through household surveys conducted at the outset of the program, validated through baseline data and a secondary rapid assessment in the targeted areas. As part of their projects, each IP has their own, internal, well-defined criteria to select and verify beneficiaries most in need. From their reporting, IPs appear to conduct an appropriate level of verification of beneficiaries. WASH IPs had among them common sets of criteria and definitions of vulnerable groups 92 of beneficiaries, including female-headed households, the disabled (elderly, mental and physical handicap), and poor families. Transparency and accountability is not observed as there is lack of awareness by the beneficiaries/ communities about incoming projects, including selection processes. Recommendation 21: OFDA should encourage IPs to adopt methodologies of engaging communities themselves in the selection of most needy beneficiaries and, where possible, in the selections of sites/location of projects to achieve more transparency and accountability. Recommendation 22: Considering the inability of the government departments to take the lead in coordinating WASH activities with IPs and the low capacities of the community groups that have limited ability to manage and sustain WASH services/facilities, coupled with low levels of participation of women in WASH activities due to low literacy levels and marginalization, future intervention priority should be given to strengthening capacity of local joint communities, local organizations and local government in program cycle management, including financial management, gender and rights based approaches, conflict sensitive programming, and participatory approaches to service delivery. Inadequate resources (funding) limit the ability of IPs to meet the larger extent of WASH needs. Inadequate institutional and individual capacity at government levels (governorates, districts, and local levels) is a key factor in the poor delivery of services and the unsustainable management of WASH facilities. Most IPs are assisting to rehabilitate or expand water resources within the private enterprise nature of water ownership. The ownership of water points tends to cluster by location, which is to be expected. There are a number of cases of mixed ownership. In six or more cases, all or almost all water access is through a private owner. Recommendation 23: IPs and OFDA should monitor the impact of water markets on livelihoods based upon the selling of water in the program area (market analysis of water) and any negative or positive impacts of the project on beneficiaries. Line of Inquiry: Unaddressed Operational Needs Technically, the approach taken by IPs is making beneficiaries more, not less, dependent on emergency assistance, which is freely distributed during the crisis. They are not prepared to move towards a foundation for successful long-term recovery. The evaluation considered whether the IPs have an explicit strategy for longer-term WASH interventions. In areas where UNICEF works in WASH activities, local key informants stated that there is no “exit strategy” for longer-term transition. This is a particular dilemma where international aid supports the ongoing trucking expenses of basic water to IDP camps. This form of emergency water supply provision without a long-term sustainable solution is globally understood to be cost ineffective. It has occurred many times before in emergencies where protracted displacement puts aid agencies into “perpetual short-term programming:” supporting operations that are expensive but being unable to implement systems approaches that would yield cost savings in the long￾term. Water trucking is frequently used only as a stop gap in emergencies, but in Yemen the recurrent costs of trucking, including fuel, exceeds the cost of any other sustainable water infrastructure that could have been built. Technical designs gave little consideration to recovery elements or DRR approaches to WASH programming and implementation. 93 Recommendation 24: IPs and cluster leads such as UNICEF may be encouraged to develop long-term cost effective solutions and exit strategies for emergency WASH programs such as water trucking. Recommendation 25: IPs and OFDA should consider implementing environmentally integrated approaches to water supply and sanitation, for example in the context of integrated water resource management. Recommendation 26: OFDA should emphasize and follow up with IPs to ensure the response is well linked to longer-term programs and set clear frameworks particular in terms of links to DRR work and building communities’ resilience to inevitable future shocks as well as addressing short and long-term community needs in WASH. While water supply is addressed well, there is correspondingly little attention to water quality. Four￾fifths of respondents indicated that there was no treatment of water supply at any level (community or household), and IPs are not addressing point of consumption water quality. Inadequate community-focused and participatory approaches in sanitation and hygiene promotion is a critical gap in the effort to generate community demand and leadership for improved sanitation and behavior change and encourage communities to develop mechanisms that align with their own local practice, addressing the diverse needs of all their members including vulnerable groups, people with disabilities, and women and girls. The estimation of water needs or gaps is not based on evidence. IPs are not making direct calculations using SPHERE minimum standards. There is no baseline information or primary data from the IPs pointing out that before their intervention, households or individuals were getting less water per capita consumption (i.e. below standards). While reading through the available project documents from the IPs, every agent seems to have relied on general information of water scarcity in Yemen without specific details at the households or individual levels. Hence, it was difficult to consider this in survey tools of this Formative Evaluation. The primary efforts to improve hygiene (i.e. BCC) appear to have been unsuccessful. Curiously, there appears to be an unexpected, inverse relationship between level of literacy and adaptation of hand-washing messages. Hand-washing messages are not well received in the general population but are remembered in IDP camps. Recommendation 27: For a long-term solution to sanitation, OFDA may encourage IPs to adopt a sustainable model based on Sanitation Marketing and CLTS approaches in the promotion of sanitation and hygiene, which have offered a viable mechanism for increasing sanitation coverage via incentives and support to building capacity of the private sector to supply minimal sanitation products. Sanitation marketing should focus on the creation of demand by the target population, through IP communications campaigns. Lessons learned should be re-incorporated into on-going OFDA funded participatory and community-wide hygiene programs. The WASH networks are an appropriate program mechanism; however, the dearth of monitoring data indicates a need for better monitoring systems. Coordination: Government authorities are limited in their ability to lead the coordination of interventions. 94 Recommendation 28: IPs such as UNICEF, with OFDA/FFP support, should engage government authorities preferably using a bottom up approach, to build strong relationships with local authorities at lower levels. IPs can actively invite representatives from the Ministry of Water and Environment (MWE) to UN cluster meetings. Line of Inquiry: Significant Implementation Lessons (Strengths and Areas for Improvement) In the WASH sector UNICEF has built networks with partners, resulting in better coordination especially in the response to the emergency of needs of communities. Also, it has prepositioned WASH supplies at community levels as part of emergency preparedness plans and to better respond to the urgent needs of affected communities. Recommendation 29: Even though IP targeting processes are effective, emphasis should be on addressing challenges in reaching out to beneficiaries in hard to reach rural and remote locations. Formation and use of WMCs is a good example of coordinating service delivery with the communities and is critical not only for the purposes of ownership and sustainability, but also as a means to empower local structures on governance issues. Recommendation 30: Beyond water supply, IPs should continually promote and ensure effective community participatory approaches such as PHAST, especially in hygiene promotion and sanitation activities. Both OFDA and FFP should encourage community participatory approaches as a condition for funding. Conscious of the role water supply plays not only in local resilience, but also in conflict in arid lands,67 the evaluation looked for, but did not see evidence of aid-associated water supply as a driver of new or excess community tensions. Recommendation 31: IPs should creatively explore ways to measure the relationship between water supply, land ownership, grazing rights and other dimensions that influence periodic conflict. Generally all IP WASH interventions are supply-driven in the context of relief WASH services. Recommendation 32: OFDA should ensure that IPs adopt more demand-led approaches that empower the affected beneficiaries to address needs with respect to the provision, operation and maintenance, repair and cost recovery of water supplies (e.g. from water trucking to permanent water source to consumption), on-site sanitation (household, communal and institutional latrines) and hygiene promotion to encourage safe personal hygiene behavior (primarily hand-washing with soap and safe water handling). Thus a transition from relief to recovery should start with a gradual shift from a supply-driven to a demand-led approach. 4.4 Protection 67 USAID’s 2013-2018 Water Strategy observes: “Growing demands on limited fresh water, degradation of fresh water quality and greater variability in rainfall patterns are potential drivers of tension. Competition and disputes over water and watersheds exist in many places around the world.” (pg 4). 95 The MRE program seems to have met its objectives in large part because OFDA provided sufficient funding support per the budget and activities. Every child surveyed learned the lessons to avoid contact with unknown objects. Every child queried (24 boys, 16 girls; mostly six to12 years of age) had had some landmine education. Almost all learned at their schools. Even though the teachers were trained, some felt they did not learn. SCI, YEMAC, and UNICEF assisted in the training. UNICEF’s objective was to disseminate information on the dangers of mines and ERW to 200,000 children. UNICEF reported that they had met this target. Recommendation 33: The UNICEF MRE funding proposal from the humanitarian response plan presents a comprehensive overview of the needs assessment, activities and targets. In the future, OFDA may ask UNICEF about overall expected targets. OFDA could then apply this information when monitoring the project. For financial accountability, OFDA should ask UNICEF to share more information about how their programs compare to their costs. Line of Inquiry: Coverage and Targeting Landmine awareness outreach by UNICEF and SCI has good coverage and good uptake of messages. The sub-award partners have played key roles. For landmine education, UNICEF implements MRE through the DRC, which provides MRE information to children through interactive participation. This approach seems effective in engaging children. UNICEF’s teaching aids to its partners are in demand, and one government counterpart requested that UNICEF include interactive materials such as games in the MRE kits. DRC uses games to raise awareness and to heighten the interest of children in MRE. Recommendation 34: OFDA should continue targeting children for MRE goals, and to accomplish this further support UNICEF for MRE working in schools and with teachers. UNICEF should clearly define the objects of the MRE program for teachers including the percentage of teachers who are expected to engage in MRE activities after the trainings. Given the relative ineffectiveness of teachers in the training process (observed in the surveys here), OFDA should suggest that UNICEF better define the purpose or theory behind the training of teachers. Within the group of targeted teachers, OFDA should ask UNICEF to monitor the percentage of teachers targeted for awareness as well as those who are expected to transfer MRE knowledge in their schools. The UNICEF humanitarian appeal for MRE in Yemen for 2013 was $1.5 million. OFDA itself funded 100 percent of this appeal. UNICEF reported that this generous support from OFDA enabled UNICEF to reach beyond their targets in FY 2013. According to the USAID/OFDA SHA in Yemen, usually OFDA does not fund 100 percent of UN agency appeals. OFDA usually requires these organizations to seek balanced burden-sharing funds from other donors as well. Recommendations 35: OFDA may consider adhering to this guidance in the future. Rather than fully funding the appeal, OFDA could advocate for other donors to support the appeal. OFDA’s support to the UNICEF-led MRE efforts is a rare instance where one donor almost fully funds the UN appeal for a sector. To extend the current progress, other donor support may be essential when collective pressure is required to address system-wide issues with the UN or with the local government. Donors can use their leverage to influence policy and to bring about change. Because UNICEF’s IPs are solely dependent upon UNICEF for funding of MRE activities, OFDA should work with UNICEF to engage the Yemeni government to provide a percentage of funding for MRE. 96 In particular, given the extent of the problem with mines and UXOs, UNICEF should start engaging with the government in developing an action plan for including MRE in to the overall school curriculum. According to UNDP, in Yemen MRE is adequately funded, while the other dimensions of landmine action are grossly underfunded. Recommendation 36: Programs supported by OFDA to reduce the hazards of ERW should be better integrated across the five key components: education, humanitarian demining (landmine demarcation of fields is also important), victims’ assistance including rehabilitation and reintegration, stock pile destruction and advocacy against the use of antipersonnel mines. How OFDA shares these obligations among donors should be more explicit. All the IPs working under UNICEF have well-defined selection criteria for their project objectives. However, the inclusion of the local communities in program design as protagonists in MRE is weak. The selection criteria for teachers and schools to participate in MRE is not standardized. UNICEF is providing good but insufficient attention to monitoring and feedback. Recommendation 37: As continued insecurity in Yemen may diminish key staffing at the various UNICEF sub-offices, OFDA should monitor UNICEF’s staffing levels periodically to ensure that programs are managed and monitored adequately. Based on the expressed needs of IPs, additional items such as cameras and games for children should be included in the MRE training kits. Support should also allow IPs to include MRE as a regular component in the school curriculum as this would ensure sustainability of risk awareness. UNICEF should survey its IPs on the effectiveness of the teaching aids supplied. UNICEF might consider updating the teaching aids for children to include creative, interactive games. While UNICEF targeted teachers for MRE, the objective of their trainings is unclear and seems dependent upon the organization providing the trainings. UNICEF should monitor the IP selection criteria with the aim of increasing the overall level of satisfaction with the selection process to above 50 percent. Recommendations 38: Future inquiry is needed to understand why teachers who received the MRE training do not necessarily transfer the knowledge. Some element of the theory of change of UNICEF’s training with OFDA funding appears flawed. Because schools are seen as a key mechanism for exposing new generations of children to long￾term UXO risks, UNICEF should advocate for the government to develop a standardized process for selecting teachers and schools to participate in MRE. To begin to measure the impact of these interventions, better data would be needed about baseline/Knowledge, Attitudes, Practices, Beliefs (KAPB). Evaluating behavior change can be very difficult and hard to measure. In addition, one of the criticisms of MRE globally is that it has not demonstrated its effectiveness in a tangible way. Line of Inquiry: Unaddressed Operational Needs Insecurity is a major deterrent that limits the fuller implementation of MRE in Yemen. UNICEF and its IPs have adopted various strategies to reach out to the most vulnerable groups. In areas outside of government control, UNICEF established partnerships with locally appointed officials to facilitate MRE activities. 97 Communities are recipients of MRE and not part of the process. The target communities are not involved in project implementation or in project design. Community participation is limited to the implementation phase of assisting with raising awareness. However, the community is involved in spreading awareness on the risks associated with mines and UXOs. Recommendation 39: UNICEF may be asked, alongside their partners to develop an effective community based MRE program. MRE is likely to be far more effective with community involvement. UNICEF’s IP’s should communicate better with the target communities during the design of activities, their implementation and the end of project evaluation. MRE is likely to be far more effective if communities are fully involved in the analysis of attitudes and behavior mapping, and feel a sense of ownership. Recommendation 40: OFDA should request a project monitoring plan from UNICEF. UNICEF should review the overall project design including indicators and targets. While UNICEF targeted teachers for MRE, the objective of their training was unclear and seems to have depended upon the organization proving the training. YEMAC trains teachers to participate in MRE campaigns including campaigns at school; however, the teachers are not required to conduct MRE sessions in their schools. DRC trains children and teachers are present in the classroom during the training, and Democratic Schools trains teachers and requests that they conduct MRE. Additional time may be required to complete the training module. UNICEF’s IPs require additional financial support to adequately implement MRE. For greater outreach and sustainability, the inclusion of MRE in the school curriculum is recommended, although teachers who received MRE training are not required to transfer the knowledge. Line of Inquiry: Significant Implementation Lessons (Strengths and Areas for Improvement) The single-prime partner (UNICEF) for the whole country appears to be an effective modality. UNICEF has five offices that are supported by a central office in Sana’a. These offices are responsible for coordinating and monitoring UNICEF activities implemented through the government and UNICEF’s IPs. UNICEF provides ongoing capacity building for its partners and evaluates their performance through periodic spot checks. UNICEF’s partners submit monthly reports on the status of the programs. UNICEF employed different modalities for training teachers for their MRE work. The main theory of change, that schools are an effective way of reaching children, has been demonstrated, although teachers who have received MRE training do not appear to transfer the key knowledge. MRE education in schools seems to be effective. UNICEF’s technical staff and service delivery are adequate for the provision of output level and quality of services proposed. Teachers are satisfied with the quality of the training. Recommendation 41: OFDA can ask UNICEF to evaluate all three of their MRE training modalities for their effectiveness. UNICEF ought to review these modules now being used to train teachers for their effectiveness and include the time allocated per module. Working with UNICEF and the other IPs, OFDA might encourage a review of their current standard operating procedures to identify ways to expand community level action. UNICEF should review the project design of its MRE work, including its indicators and targets. As an intermediate measure between the IPs and the teachers, UNICEF may want to explore expanded use of Training of Trainers (ToT) which an scale up the net amount of teaching on landmine risk, and also allow for it to continue on a more sustainable basis. 98 The success of the training is based upon the percentage of students who reported having received MRE in schools. A total of 40 students were interviewed (24 males and 16 female). When the students were asked if they knew what mines and ERW were, 95 percent said they did. Out of this group, 97 percent said they had received the information on mines and ERW from school. Caveat: It is possible that some of the students interviewed might have had prior knowledge of the dangers associated with mines since UNICEF and IPs conducted MRE is some of the same governorates the previous year. Prior knowledge should be assessed in future inquiry, including the Summative Evaluation. To ensure the sustainability of the MRE efforts, UNICEF has concentrated support to enhance the capacity of local NGO and government counterparts. UNICEF also supported IPs with teaching aid materials and per diem for field staff. However, there is a lack of financial assistance from the government for MRE. Local NGOs and government entities implementing MRE activities depend solely upon UNICEF for financial support. The level of financial assistance provided by UNICEF is unsustainable because it depends on donor support. Recommendation 42: UNICEF should periodically review its supply chain for teaching materials. This will enable UNICEF to address bottlenecks in a timely manner and to make the necessary corrective actions. The DRC actively provides MREs to children through interactive participation. DRC uses games to raise awareness and to heighten the interest of children in MRE, an approach that seems effective in engaging children. A UNICEF government counterpart also requested that UNICEF consider including interactive materials such as games in the MRE kits. UNICEF should also develop further interactive approaches to include activities such as puppet shows and peer education workshops. Recommendation 43: In governorates where UNICEF does not have a presence, OFDA should explore replicating UNICEF’s model of coordination structures (e.g. those chaired by one of its IPs with activities in the governorate). 4.5 Gender The evaluation team was ever-conscious of the gender dimensions in the core needs, in the IP programming and in the nature of field research. The majority of respondents overall were female (63 percent), particularly among the beneficiaries (health and WASH). The survey ensured the inclusion of women in the institutional surveys (schools, clinics) where the staff are predominantly male. Figure 16 graphically depicts the same data as seen earlier, in Table 1, repeated here to indicate that to understand the gender implications of programs (women are not well represented in staffing), an intentional approach toward gender inclusion in samples is required. Line of Inquiry: Coverage 99 Women are routinely reached by programs, often benefiting more than men (as in food and water access, despite a lack of targeting). They are directly engaged in most nutrition and medical programs, and WCMs are often composed of women. Interestingly, women and men appeared to have roughly the same knowledge of their entitlements in food distributions. There were also large differences between women and men in their access to food, none of which is surprising, given what is known about how culture and religion marginalize women in Yemen. Women tend to be less informed about how most programs are designed or targeted. Eighty-four percent of women said that they had not been communicated with by IPs, as opposed to 17 percent of men surveyed, in the large-sample survey. Even when a lot of programs focus on women, women do not take advantage of these programs. They do not take advantage of repeat visits, as they do not have the same mobility as males. ADRA found that at health posts in its areas, the health posts excluded women (not children) who could have qualified for PLW care. Recommendation 44: Across the sectors, the evaluation suggests that IPs could do considerably more to engage their communities in defining gender-specific patterns of vulnerability and risk, and build more focus on gender balance into economic recovery efforts. Sometimes, preference for males is necessary. For instance, in landmine awareness education, which reaches more boys who happen to attend school in greater proportions, boys are actually the priority group as they are the more likely to travel into open fields and pick up objects. Thus, boys are also more represented in the schools where the MRE activities occur as the result of the activity, which favors boys proportionally, and coincides with the intent. Line of Inquiry: Unaddressed Operational Needs Figure 16: Gender Distribution within Each Survey Group in Formative Evaluation 100 There is evident gender segregation in the involvement in water activities in WASH. In the course of WASH interventions, men are more often involved in the physical digging of wells, production of piping systems, etc., whereas women are more often involved in the day to day management of household water supplies, from fetching water to its use in cleaning, cooking and hygiene. Reports from FGDs of WMC showed that a majority of the respondents (seven out of seven) are men, an indication of gender imbalance. Key informants from the IPs did not confirm the use of any gender marker system/matrix to track gender balance and participation in WASH activities. Beneficiaries are not informed by IPs about upcoming development projects and beneficiaries’ selection processes are not always inclusive of community members. Recommendation 45: OFDA and FFP should require that IPs strive for gender balance in program implementation and operation, which they and YOFMEP should monitor. IPs should establish clear theories of change about what works in involving women in activities beyond merely receipt of services. The WMCs include women in these committees only when the IP pressed for their participation as a requirement of the project. Further research should ask how effective this is for scaling up, replication, or long-term change. From the evidence reviewed, there was a notable lack of attention to reproductive health, except that it came up frequently in inter-agency assessments, when referenced in broad brush strokes. 68 Recommendation 46: In the Summative Evaluation and other inquiries, specific attention should be given to understanding both the need for and the level of programming for reproductive health services, including emergency obstetric care (EmOC). The shift from food distribution to voucher distribution may create inter-gender tensions within households and may put some women at a disadvantage for traveling to sometimes distant retailers to redeem the vouchers. Recommendation 47: Gender disaggregated impact assessments should explore the implications of creative distribution modes, taking into account distance to markets and cultural barriers to travel. 68 The UN IASC Joint Country Mission to Yemen Needs Assessment of the crisis in May 2010 concluded that of all the sectors it reviewed for “health interventions required for scale-up, reproductive health was identified as a priority area.” Woman receives food voucher in Al-Makha District, Hesi bin Alwan, Jan. 2014 101 Line of Inquiry: Significant Implementation Lessons (Strengths and Areas for Improvement) Too few of the IPs had robust theories for how women in particular would overcome their disadvantages within communities as part of helping them become resilient to the risks at the heart of the disaster assistance. Recommendation 48: IPs should ensure full involvement from women, as representative as possible to promote good governance where decision making and responsibilities for water and sanitation are being shared equally by beneficiary women and men. OFDA/FFP should direct grants to those activities that clearly demonstrate results for women and girls and that can monitor and quantify gender-disaggregated results. 4.6 PORTFOLIO REVIEW OF OFDA AND FFP 102 This section synthesizes portfolio-wide and multi-sector conclusions and recommendations. Figure 17 depicts all the activities that were discussed throughout this evaluation, with the color intensity connoting the level or intensity of support to specific activities by sectors. While FFP resources are reaching a greater number of vulnerable households, OFDA’s resources are targeted to a range of specific sectors and gaps. A Note on the Formative Evaluation This evaluation generated many findings, lessons and recommendations, though less can inferred about “results” for either OFDA or FFP funding streams. Because many beneficiaries received aid from different programs and different agencies, the most appropriate impact analysis in the future would be “Contribution to Change.” As violence calms down, there may seem to be “regression to the mean” among health statistics - a normal statistical drift that may be mistaken for changes due to aid. The statistical power of the data generated in this Formative Evaluation is generally strong. Among the food recipients, the sample size was large enough to make many inferences. Among other surveys, such as health facilities and schools, the concurrence across different respondents in different locations suggests highly significant coverage. In the Summative Evaluation, more attention to geography as a controlling variable should be built into the design and analysis. In Yemen, Governorates each encompass distinct cultures. For instance, in some mountainous Governorates, the target population is often dispersed, communication infrastructure is very poor, and the health, education and livelihood of its people are lower than elsewhere in Yemen. School based programs provide some options for outreach. In these areas, because of the topography, target Figure 17: General level of support by OFDA and FFP in the different investigations in Yemen 103 groups may travel to get assistance such as food aid, but others do not travel to receive food or other assistance because their transportation costs are just too onerous. The Summative Evaluation should control for distances and migration as well as migratory lifestyles. Nomadic people are very hard to reach for program interventions. Their vulnerabilities relate closely to the physical topography of the nation, which also makes program access difficult. Integration In general, opportunities are missed due to a lack of integration across sectors and between different IPs’ programs, such as not providing health behavior change at the time of food distribution, offering family planning in health outreach, or providing protection messages when working with farmers in livelihood outreach, where landmine exposure is limiting which lands come under production. Water and Livelihood: The cost of water to vulnerable households, which has a critical impact on their livelihoods, is an issue that is not clearly addressed by IPs in sector programming. WASH and Environment: Limited attention by the IPs and by OFDA has been given to the long￾term environmental sustainability of water supply programs, especially in the context of climate change impact and adaptation. Integrated Health/Nutrition/Water: IMC and SCI are implementing integrated health programs (Health/Nutrition/WASH) in 12 health units/facilities targeting children under five years old and their mothers. The IPs focus most of their activities at health centers, community levels (mainly communities living around the health centers), and governorate levels (Ministry of Health). Hence, children under five years who are in the communities and their caregivers are the vulnerable groups in this case. IOM targets those children who present at the health centers. Inter-Agency Coordination: The monthly cluster meetings allow IPs to exchange updates on who is doing what and where, and help to avoid the duplication of efforts and wasting of resources. These meetings function better in the field than in the national capital. Monthly joint supervision with MPHP also facilitates whether those in need are reached and humanitarian workers take actions in the earliest time possible. IP Program Adaptation and Evolution: Skills transfer, demonstration effects and social capital are reasonable outcomes from some of the current service delivery activities. IPs are responding by building the capacity of local indigenous NGOs who can access and can stay in the areas of implementation longer, putting different innovative and technologically advanced tools in place for monitoring project activities, hiring temporary staff, and using mobile teams for outreach. The EFSP project of MC serves as a replicable example of service delivery that is well coordinated with the local community. The communities select the outputs to be constructed or rehabilitated after going through a prioritization process that may ensure sustainability. Local Engagement for Buy-in: IPs actively involve many local community leaders (religious leaders including imams, Sheikhs and Mullahs, and local council members) for the purpose of reaching out to those in need who may not understand or trust authorities or how to participate in aid programs. Projects require sufficient time at the outset to raise the level of awareness of the community on what the project is about. 104 Decisions to Inform: Inform about transparency, accountability and the effectiveness of the beneficiary selection methodology and implementation mechanism; also assess and compare the effectiveness of the different outreach (targeting) approaches of IPs and challenges encountered. Recommendation 49: OFDA and FFP should require, and IPs need to put in place, proper feedback/grievance/complaint channels. Grievance/complaint channels were absent from most IP projects investigated. Proper mechanisms for getting beneficiaries’ feedback are scarce. Recommendation 50: Questions about marginalized groups should be answered. While IPs have selection criteria that target the neediest, within that large mass are even more marginalized groups like the elderly, disabled, and youth. OFDA and FFP should ask that IPs take measures to include them in programming, and IPs then should monitor progress. Meanwhile, there is evident self-exclusion by entire marginalized groups who are afraid of interaction with authorities and NGOs at health facilities. Future assessments and evaluations should attempt to determine the consequences of their exclusion in terms of excess mortality, disability, malnutrition, and missed opportunities. Recommendation 51: Explore highlighting validation, best practices or standards. For food distribution, the house-to-house validation process seems to encourage dialogue, leads to more accurate beneficiary lists and is appreciated by the target populations. In general, the beneficiary selection used by IPs appears to be appropriately needs-based, and the criteria used are well￾defined and understood by administrators, although those criteria are not well understood among recipients. IPs (OFDA and FFP) need to plan for and spend sufficient time in the initial phases of the projects to raise the level of awareness of the community on what the project is about. Recommendation 52: Be cautious of the transfer of authority. The weaker government systems have resulted in the beneficiaries becoming entirely dependent on the good will of the service providers (i.e. the health workers in the facilities and local leaders managing the FDPs). The downside is that few transparency and accountability systems have been reported. Meanwhile, IPs are supported to work with national partners for outreach, necessary in many remote areas. For example because of insecurity in Abyan Governorate, GC built a partnership with the international NGO, CSSW, to reach out to communities there. Recommendation 53: Protection should be integrated in agriculture and food security. Greater information is needed about the impact of landmines and protection challenges in sectors such as trade, water supply and livelihoods. Regardless of how many farmers or pastoralists have been injured by landmines, it would be worth knowing, and therefore researching, how many Yemenis are prevented from trying to farm or herd, due to limitations from landmines. There is evidence that water sources may be adjacent to landmines as well. Prior to the distribution of agriculture inputs, IPs should ensure that beneficiaries have received MRE. IPs working in agriculture and food security should explore other farming models that can increase yield and productivity while ensuring the safety of famers. Decisions to Inform: The appropriateness of technical programming by sector to accomplish sector objectives and convey potential gaps that became evident as a result of the current sector programming. Recommendation 54: Programming should draw on theories of change that result in resilience. Both OFDA and FFP appear in many activities to be still oriented toward relief modalities, whereas recovery or resilience programming are less visible. Newer projects that do promote recovery and resilience should be measured against newer milestones and results. 105 Recommendation 55: As recovery, return and reintegration are achieved in some areas, donors should be nimble to shift gears. While projects are effective, they are not achieving the scale to meet the needs of the vulnerable. This suggests possibly re-thinking portfolio funding targets and proportions, such as fewer targets with more impact, like in the MRE sector. Recommendation 56: OFDA and FFP Program Guidelines should incorporate principles for shifting the focus of development from servicing needs to building the capacity of individuals and communities to understand, claim and fulfil their entitlements, to be integrated systematically. Decisions to Inform: Lessons learned and best practices in implementation and coordination: Promising Practice: Projects that are well coordinated with local communities – such as those of MC, ADRA, and others – serves as replicable examples of processes that may ensure sustainability. Coordination can include beneficiary selection and program elements, such as the identification of community assets for rehabilitation for FFW or assets projects. Promising Practice: The monthly cluster meetings allow IPs to exchange updates on who is doing what and where, and help to avoid the duplication of efforts and wastage of resources. These meetings function better in the field than in the national capital. Monthly joint supervision with MPHP would help track whether those in need are reached and coordinate humanitarian workers to act in a timely manner. Recommendation 57: OFDA/FFP should encourage this through knowledge sharing about how to best to build on the successes of collaboration with the many Yemeni partners. IPs are responding to a sustainability concern by building the capacity of local indigenous NGOs that can access and stay in the areas of implementation longer, putting different innovative and technologically advanced tools in place for monitoring the project activities, hiring temporary staff, and using mobile teams for outreach. Meanwhile, IPs have tried different ways to mitigate capacity challenges of government partners, with mixed results. OFDA/FFP might encourage knowledge sharing about these experiences, and those of other development partners, to consider what’s working, what’s not, and viable strategies. Recommendation 58: OFDA and FFP may rethink assumptions about how to concentrate resources. While projects are effective, they are not achieving the scale to meet the needs of the vulnerable. This suggests possibly re-thinking portfolio funding targets and proportions, such as fewer targets with more impact, like in the MRE sector. Recommendation 59: Activities can re-orient from charity to rights-based orientations. OFDA and FFP Program Guidelines should incorporate principles for shifting the focus of development from servicing needs to building the capacity of individuals and communities to understand, claim and fulfil their entitlements, to be integrated systematically. Overall, the Formative Evaluation generated a greater than average set of insights about an entire portfolio of programs, projects and activities. Use of the YOFMEP monitoring team was a successful example of data-gathering in a crisis zone that OFDA can learn from for future research, surveys, and verifications. Future field inquiry, including the 106 Summative Evaluation, can build on this to document the important Yemeni case of relief to development transition and the rich cross-sector lessons from the NGOs, UN agencies and other partners. 1 ANNEXES YOFMEP FORMATIVE EVALUATION SEPTEMBER 26, 2014 FOR OFDA AND FFP 2 ANNEX I: EVALUATION STATEMENT OF WORK A.4. STATEMENT OF WORK USAID YEMEN MONITORING & EVALUATION OF USAID OFDA AND FOOD FOR PEACE PROGRAMS A.5. PURPOSE The purpose of this contract is to establish a third‐party monitoring and evaluation system to conduct ongoing collection and verification of humanitarian programs in Yemen funded by USAID OFDA and the USAID FFP. The Contractor shall monitor the progress of activities and evaluate the effectiveness and sustainability of OFDA and FFP programs. The programs focus in the following sectors: water, sanitation and hygiene (WASH), nutrition, health, logistics and relief commodities, humanitarian information and information management, protection, economic recovery and market systems (ERMS), agriculture, food security, and food assistance. While OFDA and FFP implementing partners maintain responsibility for monitoring their activities, the Contractor shall provide independent monitoring and evaluation support services. The results of these supplementary M&E services will be instrumental in improving overall OFDA and FFP programming and determining appropriate future program design. A.6. INTRODUCTION A.6.1. Overview This contract supports the Monitoring & Evaluating of USAID Office of Foreign Disaster Assistance and the Office of Food for Peace programs in Yemen. The Contractor shall monitor the progress of activities and evaluate the effectiveness and sustainability of OFDA and FFP programs. The programs focus in the following sectors: water, sanitation and hygiene (WASH), nutrition, health, logistics and relief commodities, humanitarian information and information management, protection, economic recovery and market systems (ERMS), agriculture, food security, and food assistance. While OFDA and FFP implementing partners maintain responsibility for monitoring their activities, the Contractor shall provide independent monitoring and evaluation support services. The results of these supplementary M&E services will be instrumental in improving overall OFDA and FFP programming and determining appropriate future program design. A.6.2. Background Although Yemen became a unified state in 1990, the country remains a patchwork of groups. The country’s leadership, headed by President Abdo Rabo Mansour Hadi, is tasked with a balancing act—managing the competing interests of local tribes, neighboring governments, non‐ state military forces, and divided Islamic factions. As the Middle East’s least developed nation, Yemen has experienced a series of secession movements, including a 1994 civil war, which reflect these tensions and hinder efforts to promote economic growth and social stability. In 2004, fighting broke out between Shia opposition groups—the al‐Houthi—and Republic of Yemen (RoYG) forces. Since then, the protracted conflict has affected more than 1 million people and repeatedly displaced populations in northern Yemen. The humanitarian crisis deepened in the 3 summer of 2009 as RoYG troops, Sunni tribes, and the al‐Houthi again battled in the north, resulting in hundreds of deaths and displacing a quarter of a million people. Later that year, the conflict took on an international dimension following clashes between northern al‐Houthi rebels and Saudi security forces along the countries' shared border. In early 2011, conflict between these groups in the wake of the Arab Spring again limited the capacity of the RoYG to provide basic services, exacerbated deteriorating humanitarian conditions among impoverished populations, and resulted in displacement throughout the country. Economic collapse and civil unrest also made Yemen an important base for Islamic militants pushed out of Al Qaeda strongholds in Afghanistan and Pakistan. As of March 2013, intertribal conflict in the north, fighting between the RoYG and armed groups in the southern governorates, and civil unrest in urban centers of the west and central governorates have further weakened the central government’s capacity to respond to the needs of affected populations. Such chronic insecurity has limited humanitarian access and hindered procurement of accurate information relating to humanitarian needs. It is estimated, however, that 13 million people—or 55 percent of the Yemeni population—are currently in need of humanitarian assistance. Among them are approximately 350,000 internally displaced persons (IDPs) and more than 238,000 refugees, most of whom originated in the Horn of Africa. In addition, political and economic instability, rising food and fuel prices, high unemployment, and conflict‐related displacement have left approximately 10.5 million people in Yemen food insecure. In response to the humanitarian needs in Yemen, OFDA and FFP have funded programs in WASH, health, nutrition, agriculture & food security, protection, humanitarian coordination, and emergency food assistance. In 2012, OFDA and FFP funded approximately $100 million of humanitarian programs in Yemen; thus far in 2013, OFDA and FFP have contributed nearly $55 million to the humanitarian response in Yemen. The programs funded by OFDA and FFP are implemented primarily in the western regions of Yemen. A.6.3. Objective The principle objective of this contract is to monitor and evaluate ongoing projects funded by OFDA and FFP in Yemen. The security situation in Yemen limits OFDA’s and FFP’s access to the implementing partners in the field working throughout Yemen. The Contractor shall provide monitoring and evaluation services to report on the progress of projects in Yemen. The Contractor shall be responsible for verifying activities, monitoring the outputs of activities, and conducting a data quality assessment. Two evaluations will be conducted as part of this contract: a formative evaluation and a summative evaluation. The formative evaluation will be short in duration and deliverables and will be completed near the midpoint of the contract. The summative evaluation will be more substantial and will be completed near the end of the contract. A.6.4. Tasks The monitoring aspect of this task order contains three types of activities: 1. Activity verification; 2. Output monitoring; and 3. Data quality assessment OFDA and FFP shall provide the contractor with secondary data information from sources listed below. Additional key information will be provided by the Government and implementing 4 partners following award. Additional information may be provided in the field. This shall include, but will not be limited to the following: ‐ Partner Awards and Proposals ‐ Quarterly Performance Reports and Financials ‐ Partner M&E Plans ‐ Available Baseline Data ‐ Yemen Fact Sheets ‐ Available Sector Reports Monitoring tasks described above will be conducted on a regular basis by Contractor staff based in Yemen at the contractor’s office. Monitoring staff are expected to work full‐time throughout the period of performance of the contract. The Contractor shall maintain consistent and timely reporting for each of the monitoring tasks. For more information on the reporting deliverables for the monitoring tasks, please see Sections C.6 and F.2. A.6.4.1. Activity Verification The Contractor shall use innovative techniques to verify the existence and progress of activities funded by OFDA and FFP in Yemen. The precise methods of verification will be agreed upon during project start‐up and will result in OFDA and FFP having a clear idea of the status of activities they fund throughout Yemen. The Contractor shall report the results of activity verification in the Monthly Monitoring Reports. Please refer to Sections C.6.8 and F.2 for further information on the deliverables associated with this aspect of the contract. A.6.4.2. Output Monitoring The Contractor shall monitor the outputs of OFDA and FFP programs in Yemen. OFDA and FFP will select the projects to be monitored and will give the Contractor the program design and monitoring documents necessary to carry out the output monitoring tasks. The Contractor shall report the results of output monitoring in the Monthly Monitoring Reports. Please refer to Sections C.6.8 and F.2 for further information on the deliverables associated with this aspect of the contract. A.6.4.3. Data Quality Assessment The Contractor shall conduct two data quality assessments of the monitoring data gathered by OFDA’s and FFP’s implementing partners in Yemen. OFDA and FFP will select the projects that will be included in the data quality assessment. The data quality assessment shall examine the validity, reliability, precision, integrity, and timeliness of the monitoring data. The Contractor will submit two data quality assessment reports during the period of performance of the contract. Please refer to Sections C.6.9 and F.2 for further information on the deliverables associated with this aspect the contract. A.6.5. EVALUATION OBJECTIVES, LINES OF INQUIRY, APPROACH, AUDIENCE, AND METHODS The Contractor shall conduct two separate evaluations which have different objectives and lines of inquiry. The objective of the formative evaluation is understanding effectiveness of OFDA and FFP programs and is intended to be a short‐term evaluation that will give OFDA, FFP, and their implementing partners the opportunity to actively engage with the findings so that mid‐course 5 corrections to increase effectiveness can be made if necessary. The summative evaluation will still be conducted with learning and use in mind, but will not be used to adjust current programming. The objectives of the summative evaluation are effectiveness, sustainability, and implementation process appraisal. The summative evaluation will inform the design of future USAID programming in Yemen. The formative and summative evaluations will employ a utilization focused approach. All evaluation work and deliverables will have end use in mind so that OFDA and FFP managers can utilize the information to improve current and future programming in Yemen. Inherent in this approach is time spent with the users of the evaluation to ensure that the methods, deliverables, and process will result in maximum use of the evaluation findings. The users of both evaluations will be OFDA and FFP managers in the field and in Washington, DC. Both the formative and summative evaluations will use quantitative and qualitative methods. The Contractor will ensure that the methods they use throughout this contract do no harm to respondents. A.6.5.1. Formative evaluation1 Objective: Effectiveness. The Contractor shall evaluate the effectiveness of OFDA and FFP programs in Yemen. Lines of Inquiry: 1. What is the project doing to ensure that it reaches the beneficiaries most in need? What are the challenges to reaching the beneficiaries most in need? How have implementing partners overcome those challenges? 2. Are there significant needs within OFDA and FFP’s operational sectors that are not currently being addressed or met by OFDA and FFP programming? 3. What are the most significant lessons learned about the implementation process of OFDA and FFP programming from the monitoring process to date? What are the most significant strengths of and areas of improvement for OFDA and FFP programming? A.6.5.2. Summative Evaluation Objective 1: Effectiveness. The Contractor shall evaluate the effectiveness of OFDA and FFP programs in Yemen. Lines of Inquiry: 1. To what extent did the project achieve the intended goal, objectives, and results as defined by the project’s Results Framework? a. Were there any important unintended outcomes, either positive or negative? b. What were the main reasons that determined whether intended outcomes were or were not achieved, and whether there were positive or negative unintended outcomes? Which unintended outcomes were under the control of the program and which were not? 2. What is the project doing to ensure that it reaches the beneficiaries most in need? What are the challenges to reaching the beneficiaries most in need? How have implementing partners overcome those challenges? 3. How did the gender dimensions of program activities impact implementation and results? a. In what ways did men, women, boys and girls participate in project activities? 6 b. Were gender considerations included in program design and implementation? In what ways was gender considered? Were there any challenges when considering gender in program design and implementation? Objective 2: Sustainability. The Contractor shall evaluate the sustainability of OFDA and FFP activities and results in Yemen. Lines of Inquiry: 1. How sustainable are the OFDA and FFP interventions in Yemen? a. If sustainability is an issue, what challenges do implementing partners face in ensuring their interventions are sustainable? b. How have they overcome these challenges? c. What are the main factors that affect, either positively or negatively, the sustainability of project outcomes? 2. Have the OFDA and FFP programs built the capacity of government officials, civil society, or communities to mitigate, adapt, and recover from future disasters, shocks or stresses? Objective 3: Implementation Process Appraisal. The Contractor shall evaluate the implementation process of OFDA’s and FFP’s programs in Yemen. Lines of inquiry: 1. What are the strengths of the implementation process for OFDA and FFP programs in Yemen? How could OFDA and FFP best build upon these strengths? 2. What are the weaknesses of the implementation process for OFDA and FFP programs in Yemen? How could OFDA and FFP avoid these challenges in the future? A.6.6. Activities & Deliverables A.6.6.1. Pre Kick‐Off Meeting Conference Call A teleconference call will be conducted with the Contractor team and OFDA staff to finalize the kick‐off meeting agenda and to provide any needed clarifications on the draft work plan and the draft monitoring plan. The Pre Kick‐Off Meeting Conference Call shall occur no later than seven (7) days after the contract award. A.6.6.2. Draft Work Plan The Contractor shall submit a work plan that will include the following elements: ‐ a schedule for completion of all of the deliverables contained in this task order, including time for preparatory work, consultation, field work, data analysis, report‐writing, report revisions, and presentations; ‐ an explanation of the roles and responsibilities of the Contractor’s team members; and ‐ a communication plan explaining points of contact between OFDA, FFP, the Contractor, and implementing partners. The Contractor shall submit the work plan no later than ten (10) days after the contract is awarded. 7 A.6.6.3. Kick‐Off Meeting (Washington, D.C.): A kick‐off meeting will be held in Washington, DC to review the work plan and discuss other deliverables of the contract. The Contractor team shall meet with staff from OFDA. The Kick‐Off Meeting will take place no later than fourteen (14) days after the contract is awarded. A.6.6.4. Final Work Plan The Contractor shall incorporate feedback on the draft work plan from OFDA and FFP to create the Final Work Plan. Once the Final Work Plan is approved, any changes to it must be submitted to the COR for further approval. A.6.6.5. Draft Monitoring Plan The Contractor shall submit a Draft Monitoring Plan that explains how they plan to carry out the activity verification, output monitoring, and data quality assessments. OFDA and FFP will send the Contractor a list of projects to be monitored so that the Contractor can submit a detailed Draft Monitoring Plan. The Draft Monitoring Plan shall include three sections: activity verification, output monitoring and data quality assessment. The activity verification section shall include, at a minimum, the following elements for each of the activities to be verified: ‐ project name ‐ project location ‐ verification indicator ‐ data collection methods ‐ data sources ‐ timeline for data collection ‐ conflict considerations ‐ key assumptions The output monitoring section shall include, at a minimum, the following elements for each output indicator in each of the projects to be monitored: ‐ output indicator ‐ definition ‐ data collection method ‐ data source ‐ location of data collection ‐ frequency of data collection ‐ timeline for data collection ‐ means of analysis ‐ conflict considerations ‐ key assumptions The data quality assessment section will include, at a minimum, the following elements for each of the projects to undergo a data quality assessment: ‐ project name ‐ data collection methods 8 ‐ data collection locations ‐ very brief outline for the assessment report The Draft Monitoring Plan will also address the following questions: ‐ How will the Contractor verify the data it gathers? ‐ How will the Contractor incorporate the Do No Harm principles in its data collection? ‐ How will the Contractor ensure that the data collected in each of the three types of monitoring activities is reported in a way that will maximize use of the results? ‐ How will the Contractor effectively communicate with the implementing partners, OFDA (in Yemen and Washington, D.C.), and FFP (in Yemen and Washington, D.C.) to ensure the monitoring aspect of the project is implemented as efficiently and effectively as possible? The due date for this deliverable will be determined at the Kick‐Off meeting. A.6.6.6. Final Monitoring Plan The Final Monitoring Plan shall incorporate OFDA and FFP feedback. The due date for this deliverable will be set at the Kick‐Off meeting. A.6.6.7. Draft Evaluation Plan for the Formative and Summative Evaluations The Contractor shall provide a Draft Evaluation Plan for the Formative and Summative Evaluations that will detail the proposed methods and timeline for the two evaluations. The Draft Evaluation Plan will have two sections: Formative Evaluation and Summative Evaluation. The Formative Evaluation section will include the following elements: o Evaluation objectives o Lines of inquiry with corresponding indicators o Decisions to inform o Methods o Data sources o Locations of data collection o Conflict and gender considerations o Means of analysis o Time (days) The Summative Evaluation section will include the following elements: o Evaluation objectives o Lines of inquiry with corresponding indicators o Decisions to inform o Methods o Data sources o Locations of data collection o Conflict and gender considerations o Means of analysis o Time (days) The Draft Evaluation Plan will also address the following points: 9 ‐ How will the Contractor verify the data it collects? ‐ How will the Contractor incorporate the Do No Harm principle in its data collection and reporting? The due date for the Draft Evaluation Plan will be determined at the Kick‐Off Meeting. A.6.6.8. Monthly Monitoring Report The Contractor shall provide a Monthly Monitoring Report that summarizes the results of the activity verification and output monitoring. The Monthly Monitoring report will include, at a minimum, the following elements: ‐ quantitative and qualitative results from activity verification from each project monitored; ‐ quantitative and qualitative results from output monitoring from each project monitored ‐ recommendations for improvement based on data gathered The Contractor shall submit the Monthly Monitoring reports on the same date each month; the due dates will be set at the Kick‐Off Meeting. A.6.6.9. Monthly Teleconference The Contractor shall conduct a monthly teleconference with the COR and other identified participants no later than 7 days after the Monthly Monitoring Report submission. The Contractor shall provide a overview of the Monthly Monitoring Report and address any outstanding concerns. A.6.6.10. Data Quality Assessment Report The Contractor shall complete two Data Quality Assessments during the period of performance of the contract. The report will include, at a minimum, the following sections: ‐ Methods ‐ Findings ‐ Recommendations The due dates for the two Data Quality Assessment Reports will be set at the Kick‐Off Meeting. A.6.6.11. Final Evaluation Plan for the Formative and Summative Evaluations The Final Evaluation Plan shall incorporate OFDA and FFP feedback. The due date for the Final Evaluation Plan will be set at the Kick‐Off Meeting. A.6.6.12. Presentation of Findings: Formative Evaluation The Contractor shall present findings from the formative evaluation to OFDA and FFP staff in Yemen. The Washington, D.C. team will join this meeting via conference call or video teleconference, therefore all documentation (i.e. PowerPoint slides) must be submitted to the COR 2 days prior to the meeting and address the following: 10 ‐ Findings for each of the lines of inquiry in the formative evaluation ‐ Actionable and practical recommendations The due date for this deliverable will be set at the Kick‐Off Meeting. A.6.6.13. Draft Formative Evaluation Report The Contractor shall submit a Draft Formative Evaluation Report, which shall contain the following sections: ‐ Executive Summary ‐ Table of Contents ‐ Introduction ‐ Methodology ‐ Analysis/Results ‐ Findings and Conclusions ‐ Recommendations ‐ References (include all documents reviewed, including background documentation and records of technical data application and decision‐making) ‐ Annexes may include: the Statement of Work; any “statements of differences” regarding significant unresolved difference of opinion by funders, implementers, and/or members of the evaluation team; all tools used in conducting the evaluation, such as questionnaires, checklists, survey instruments, and discussion guides; sources of information, properly identified and listed; disclosure of conflicts of interest forms for all evaluation team members, either attesting to a lack of conflict of interest or describing existing conflict of interest. The Contractor shall address the following performance standards: ‐ The reports shall include an Executive Summary that is between 3 and 5 pages, and summarizes the purpose, background of programs being evaluated, evaluation questions, methods, findings, conclusions, recommendations, and lessons learned. ‐ The recommendations will be related to each of the evaluation questions will be actionable. ‐ Relationship between findings (facts and figures), conclusions (inference), and recommendations (courses of action) will be clearly identified so that readers understand the logic and recommendations are evidence‐based. ‐ Report will be grammatically correct and contains no spelling or punctuation errors. ‐ Report will be branded and marked as required. ‐ OFDA and FFP shall review and approve all evaluation reports. ‐ All monitoring and evaluation products will be produced in English. Furthermore, the reports shall meet the following relevant performance criteria as stated in USAID’s Evaluation Policy Guide and Section F.2. Please see the following link: http://www.usaid.gov/sites/default/files/documents/1868/USAIDEvaluationPolicy.pdf ‐ The evaluation reports should represent a thoughtful, well‐researched and well organized effort to objectively evaluate what worked in the projects, what did not, and why. ‐ The evaluation reports should address all evaluation questions included in the scope of work. 11 ‐ The evaluation reports should include the scope of work as an Annex. All modifications to the scope of work, whether in technical requirements, evaluation questions, evaluation team composition, methodology or timeline shall be agreed upon in writing by OFDA and FFP. ‐ Evaluation methodology shall be explained in detail and all tools used in conducting the evaluations such as questionnaires, checklists, and discussion guides will be included in an Annex to the final report. ‐ Evaluation findings will assess outcomes and impacts using gender disaggregated data. ‐ Limitations to the evaluations shall be disclosed in the report, with particular attention to the limitations associated with the evaluation methodology (selection bias, recall bias, unobservable differences between comparator groups, etc.). ‐ Evaluation findings should be presented as analyzed facts, evidence and data and not based on anecdotes, hearsay, or the compilation of people’s opinions. ‐ Findings should be specific, concise, and supported by strong quantitative or qualitative evidence. ‐ Sources of information need to be properly identified and listed in an Annex, including a list of all individuals interviewed. ‐ Recommendations need to be supported by a specific set of findings. ‐ Recommendations should be action‐oriented, practical, and specific, with defined responsibility for the action. A.6.6.14. Final Formative Evaluation Report The Final Formative Evaluation Plan shall incorporate OFDA and FFP feedback. The due date for this deliverable will be determined at the Kick‐Off meeting. A.6.6.15. Draft Summative Evaluation Report The Contractor shall submit a Draft Summative Evaluation Report, with the same standards and outline as detailed in Section A.6.6.13. A.6.6.16. Presentation of Findings: Summative Evaluation (Yemen and Washington, D.C.) The Contractor shall present findings from the draft summative evaluation to OFDA and FFP staff in Yemen and in Washington, D.C. All documentation (i.e. PowerPoint slides) must be submitted to the COR 2 days prior to the meeting and address the following: ‐ Findings for each of the lines of inquiry in the formative evaluation ‐ Actionable and practical recommendations The due date for this deliverable will be determined at the Kick‐Off Meeting. A.6.7. Final Summative Evaluation Report The Final Summative Evaluation Plan shall incorporate OFDA and FFP feedback. The due date for this deliverable will be determined at the Kick‐Off meeting. 12 A.6.8. Deliverables In accordance with the tasks stated and deliverable requirements of this task order statement of work, the contractor shall deliver the following: A.6.8.1. Delivery / Deliver Methods All deliverables shall be submitted in accordance with the delivery schedule distribution. Electronic copies shall be delivered using Microsoft Office suite of tools (for example, MS WORD, MS EXCEL, MS POWERPOINT, MS PROJECT, or MS ACCESS format), unless otherwise specified by the COR. Electronic submission shall be made via email, unless otherwise agreed to by the COR. The Contractor shall use the U.S. Postal Service standard delivery for delivery of materials, equipment, or required hardcopy documents. The contractor may submit deliverables by messenger with the consent of the COR. The COR may determine alternate methods of delivery not specified in the delivery schedule. A.6.8.2. Government Acceptance Period The COR will have a reasonable period to review all contractor draft deliverables and make comments. Upon receipt of the final deliverables, the COR will have a reasonable period for final review prior to acceptance or providing documented reasons for non‐acceptance. The final submission should be deemed approved if the Government has not rejected it in thirty (30) calendar days. The COR will have the right to reject or require correction of any deficiencies in accordance with the contract inspections clause. In the event of a rejected deliverable, the COR will notify the Contractor in writing of the specific reasons for rejection. The Contractor shall have two (2) workdays to correct the rejected deliverable and return it per delivery instructions. A.6.8.3. Delivery Schedule Abbreviations The following abbreviations are used in the delivery/deliverable schedule: Abbreviation Definition CO Contracting Officer COR Contracting Officer's Representative CS Contract Specialist DA Days after DACA Days after contract award (award of this order) DAEOM Days after the end of the month CADAYS Calendar Days E Electronic Copy FFP Food for Peace Staff H Hard Copy IPM In Person Meeting M Monthly NLT Not Later Than OFDA Office of Foreign Disaster Assistance Staff TBD To be determined after consultation and approval by COR TC Teleconference VC Video Conference 13 ANNEX II: FORMATIVE EVALUATION TIMETABLE  Submit Evaluation Plan to OFDA/FFP / Approved 12/13/13 – 12/21/13  Dialogue on Evaluation Sample 12/20/14 – 3/18/14  Literature review/design evaluation instruments 3rd and 4th week of December  Submit sample for OFDA/FFP comments/approval 1/12/14  Submit evaluation instruments 1/12/14  OFDA/FFP approve sample design 3/18/14  Translate & tested instruments, train enumerators, o plan field research 3/19/14 - 3/27/14  Field research-International Technical Team in Yemen 2/22-3/4/14  Field research-surveys, FGDs, structured interviews 3/30-4/16/14  Translate Arabic qualitative data, analyze data,  draft report 4/17 – 5/9/14  Submit draft report & presentation outline 5/16/14  Present formative evaluation findings 5/13/14  Comments from OFDA/FFP 5/22/14 etc.  Submit final evaluation report 8/26/2014 14 ANNEX III: LINES OF INQUIRY AND CORRESPONDING INDICATORS Line of Inquiry 1- What is the project doing to ensure that it reaches the beneficiaries most in need? What are the challenges to reaching the beneficiaries most in need? How have implementing partners overcome those challenges? Corresponding Assessment Questions Proposed Indicators Data Source1 This line of inquiry will understand and assess IPs ‘selection criteria’ and targeting mechanism for identifying, reaching out, and verifying vulnerability of beneficiaries and assessing targeting effectiveness. Identification: - How is the term ‘vulnerable’ defined? Is there a specific and clearly identified selection criterion? Methodology: - What is the established mechanism – methodology to outreach to the most vulnerable groups? Verification: - How are IPs verifying beneficiaries’ conformity to the selection criteria? Transparency and Accountability - Are communities and beneficiaries aware – informed of the targeting / selection criteria? What is the opinion of the community (beneficiaries and leaders) regarding the targeting criteria and outreach mechanism? Is it inclusive of the most vulnerable populations? Does it exclude non-vulnerable groups? Are there ‘vulnerable groups’ who are not targeted by the current programming? Who are they? Challenges: What are IPs challenges in reaching out to those ‘most in-need’? How are IPs addressing these challenges? Defined selection criteria adopted by IPs. 1. (a, b, c, d) & 6. Percentage of respondents2 confirming knowledge of selection criteria. 2.(a, b, c, d), 3.(a, b, c), & 5. Review and or monitoring mechanism for conformity to ‘selection criteria’ put in place by IPs. 1.(a, b, c, d) 2.d 3.(a, b, c) & 5 Percentage of respondents concurring that beneficiaries met their selection criteria. 1. (a, b, c, d), 2.d, 3. 5. Percentage of respondents confirming an above average3 effectiveness of the targeting methodology in reaching out to the neediest groups. 1. (a, b, c, d), 2.d, 3. 5. Decisions to Inform: Analyses aim to inform about transparency, accountability and effectiveness of beneficiary selection methodology and implementation mechanism. It will also assess and compare effectiveness of the different outreach (targeting) approaches of IPs and challenges encountered. 2. Are there significant needs within OFDA and FFP’s operational sectors that are not currently being addressed or met by OFDA and FFP programming? This line of inquiry will understand and review the technical aspects of IP programming with respect to the problems addressed by sector-technical area. Are there critical gaps in sectors’ programming that became evident as a result of IPs implementation of past and current programs? Are there aspects of technical programming that needs to be re-evaluated for improved results on the target sectors Index developed to evaluate operational sectors’ needs to be addressed. YOFMEP Sector Specialists 1 The data source referenced is presented in table 2- data source section. 2 Respondents refer to all interviewed or surveyed stakeholders for a specific line of inquiry. 3 The survey scale rates respondents’ opinion with regard to i) above average, ii) average and iii) less than average. The Evaluation report summarizes survey results on the spectrum of the scale. This indicator is meant to convey the degree of effectiveness as measured by the percentage of surveyed respondents who confirm above average effectiveness. 15 objectives? Is the type and quality of services provided through IP programming (goods, training…) adequate to meet identified sector objectives? Are there critical gaps in sectors’ programming which are not being addressed by the current OFDA/FFP assistance projects? Percentage of respondents confirming technical appropriateness of IP programming by sector. 1.c 2.b 3. (a, b) 4. 6. Decisions to Inform: Analysis will inform about the appropriateness of technical programming (IP and OFDA/FFP) by sector to accomplish sector objectives and convey potential gaps that became evident as a result of the current sector programming. 3. What are the most significant lessons learned about the implementation process of OFDA and FFP programming from the monitoring process to date? What are the most significant strengths of and areas of improvement for OFDA and FFP programming? Are the projects’ organization, technical staff and service delivery adequate for the provision of the output level and quality of services proposed? Is the service delivery process coordinated with the local community? What are the lessons learned from IPs different community coordination processes. What are the implementation processes (for the delivery of humanitarian assistance) challenges? How did the IP address these challenges? What are the major strengths of IP programming? What are the lessons learned in terms of an effective implementation process? Are these lessons replicable? IP Projects are on plan with no major delays, problems or complaints recorded. 1. 6. Percentage of respondents confirming above average satisfaction with the projects’ implementation mechanism. 2. (a, b, c, d) Percentage of respondents confirming above average satisfaction with the quality of services and goods. 2. (a, b, c, d) 3. (b, c) 5. Decisions to Inform: Lessons learned and best practices of IPs implementation and coordination processes. 16 Annex IV: Evaluation Team Members, Disclosure of Conflict of Interest Forms, and Statement of Differences Monitoring and Evaluation Team Leader, Ms. Gayla Cook, is an M&E Specialist with over 30 years of experience designing, managing, assisting, and evaluating international donor-funded development projects in several sectors and in nearly 30 countries in the Middle East, Africa, and the Caribbean. As Principal of M&E Practice at IBTCI, Ms. Cook’s most recent assignments include leading a Social Accountability Assessment on select USAID/Ethiopia emergency humanitarian assistance and conflict mitigation programs, including the Development Food Assistance Program (DFAP); serving as Project Director on the USAID Multi-Year Assistance Program (MYAP) Program Performance Evaluation aimed at optimizing the effectiveness of future Food for Peace programming in Haiti; and leading the summative evaluation of the Somalia Youth Livelihoods Program. From 2002-2008, as Project Director for the Africa Bureau of USAID’s Office of Sustainable Development, she oversaw a contract for the design and management of the M&E reporting system of the Africa Education Initiative, a $600 million investment to improve education access and quality in 40 countries, overseeing data quality reviews, baseline studies and monitoring activities and producing technical reports, special studies, publications, and policy papers. As Chief of Party on the USAID/South Africa Basic Education Reconstruction (SABER) Project from 1995- 1997, she was responsible for fielding teams responding to task orders that addressed the capacity gaps in a newly democratic country that included: facilitating strategic planning and change management in provincial departments; technical assistance for policy formulation; establishing education management information systems; and coaching and mentoring senior managers. Ms. Cook holds a master’s degree in Communications from Syracuse University and a bachelor’s degree in English Literature and African Studies from Cornell University. Evaluation Project Specialist, Ms. Leyla Moubayed, is an evaluation expert with over 18 years of experience in project evaluation, project management, community building, and leadership training for local municipal and religious leaders in Yemen in the greater Middle East. Through her varied experience evaluating international donor-funded programs, Ms. Moubayed has engaged in all aspects of evaluation including designing methodology, supervising and managing evaluation teams, overseeing implementation, conducting and managing quantitative and qualitative surveys, conducting key informant interviews, leading focus group discussions, developing evaluation tools and instruments such as focus group guides, orienting and training local researchers and data collectors, implementing stakeholder workshops, and writing, compiling, and presenting final reports. Previously with IBTCI, she served as Team Leader and lead evaluator for the USAID/Yemen Promoting Youth for Civic Engagement project, Team Leader for the Military Information Support Team (MIST) project in Yemen and worked on the review of the Office of Middle East Programs (OMEP), which covered programs in Yemen, Jordan, Morocco, Lebanon, and Palestine. Ms. Moubayed’s other evaluation experience includes working as lead evaluator of the Yemen Community-based Conflict Mitigation (Y-CCM) project and serving as Subject Matter Expert on the evaluation of the USAID-funded Small Villages Wastewater Treatment Systems (SVWTS) project in Lebanon. As Program Director for the $14 million USAID -funded Community Action Program in Iraq, she managed teams of over 200 technical experts, community mobilizers, coordinators and field officers in the development and execution community infrastructure projects in the education, health, economic growth, and environmental sectors. Ms. Moubayed speaks fluent English, Arabic, and French and holds a master’s degree in business administration from the Lebanese American University. Monitoring Project Specialist, Ms. Emilly Kemigisha, is and accomplished M&E specialist with ten years of evaluation, research, and capacity building experience on development programs focused on gender analysis, health, HIV/AIDS, education, and democracy and governance and extensive experience designing project M&E plans and data collection methodologies, developing training manuals for performance monitoring and utilization, collecting data, and assessing data quality in fragile northern Uganda. Most recently, Ms. Kemigisha served as M&E Specialist on an evaluation of a crop value chain development project, led teams on two data quality assessments (DQAs) for USAID, and served as a Team Leader on the evaluation of a health communication partnership. From 2004 to 2012, as M&E Specialist for the 17 USAID-funded Monitoring and Evaluation Management Services (MEMS) project, Ms. Kemigisha spearheaded an assessment of gender integration into USAID/Uganda activities, led a team in conducting DQAs for the President’s Malaria Initiative (PMI), and developed, refined, and documented the Mission’s Performance Monitoring Plans (PMPs) with a special focus on quantitative indicators. In addition, she led training for Mission staff and implementing partners in performance management, DQA, and evaluative research. Ms. Kemigisha holds a master’s degree in Public Health from the University of Uganda. M&E Technical Sector Expert (Agriculture and Food Assistance), Dr. Patricia Bonnard, is an agriculture and food assistance expert with nearly 30 years of experience working on agriculture and food security issues. Most recently, Dr. Bonnard advised on testable hypotheses related to food security for the USAID Global Climate Change Initiative and designed and implemented an evaluation of the World Food Program regional program in Central America. Since 2011 she has been a member of the OFDA Technical Assistance Group’s surge team serving as an Agriculture Specialist, creating assessments and making recommendations for response options, providing technical assistance to Disaster Assistance Response Teams and Response Management Teams, overseeing agriculture response activities, and coordinating with other responders and host governments. Also with OFDA, she served four years as Agricultural Recovery Evaluator. Ms. Bonnard served for five years as Senior Advisor for the Famine Early Warning Systems Network (FEWS NET), writing and guiding the development of numerous tools to build the FEWS NET knowledge base. She has served as a technical expert for a number of other relevant projects, including Evaluation Specialist/Agriculture Sector for OFDA, two years as Research Analyst for the International Food Policy Research Institute, and four years as a Senior Agriculture and Food Security Advisor for the Food and Nutrition Technical Assistance Project. Dr. Bonnard holds a PhD in Agricultural Economics from Michigan State University and is fluent in English, Portuguese, Spanish, French, and Italian. M&E Technical Sector Expert (Health and Nutrition), Dr. Mesfin Beyero, is a food security, malnutrition prevention, and humanitarian aid expert with 20 years of health sector experience, including extensive experience in project planning, implementation, coordination, and monitoring and evaluation. Recently, Dr. Beyero served as Nutrition and Public Health Consultant for CARE’s mid-term evaluation of the MUSKOKA project for Improving Health and Nutrition of Vulnerable Women and Children in Ethiopia and served as Health and Nutrition Consultant for IBTCI’s evaluation of the USAID/Ethiopia Pastoral Livelihoods Initiative II project. Previously, as Director of the Alive & Thrive Project in Ethiopia, he was responsible for program design, implementation, monitoring, and providing guidance in all areas of operational research with the aim of reducing stunting among infants and young children. As Nutrition Consultant for UN World Food Program, in 2007 he reviewed critical documents related to food insecurity in Ethiopia including national agricultural and food security strategies and programs, agriculture and food security elements of the Poverty Reduction Strategies, and programs focusing on HIV/AIDS, access to basic education, and emergency preparedness and response. From 2004 to 2006, he served as M&E Training Officer for USAID’s LINKAGES initiative to support the government of Ethiopia and its partners in addressing malnutrition through support to child survival programs and HIV/AIDS programs. In the role, he monitored all activities through community-based baseline surveys and impact assessment studies. Dr. Beyero holds a master’s degree in Public Health from Franzens University of Innsbruck and is fluent in Amharic and English. M&E Technical Sector Expert (Protection), Mr. Steven Hansch, is an evaluation methodologist and humanitarian aid expert specializing in learning and disseminating lessons to improve humanitarian action with a focus on protection, nutrition, water and shelter. In 2001, Mr. Hansch initiated and organized an international conference on protection in emergencies which brought together donors, UN agencies, NGOs, scholars, and the Red Cross to discuss how protection can be a more formalized sector in the development community. He has conducted field research in a dozen refugee communities to examine the feasibility of protection in emergencies and has led and designed protection interventions in Ethiopia, Sudan, Somalia, and Darfur and has been a keynote speaker at over 50 public events, courses, and conferences on humanitarian aid with a focus on the implementation of protection. Mr. Hansch has worked in evaluation at many levels, gathering evidence, conducting interviews, and culling lessons from dozens of emergencies in sub-Saharan Africa, Latin America, Asia, the Middle East and Balkans. He has spent the 18 majority of these years designing and implementing a range of evaluation methodologies for international humanitarian interventions. At the Refugee Policy Group, for six years he conducted, led and managed qualitative evaluations covering a range of refugee, IDP and other crises in close collaboration with UNHCR, IOM, WHO, WFP, the Red Cross, USAID, and NGOs. He led two different evaluation teams after the 2004 tsunami, two in Somali famines, an extensive multi-stakeholder evaluation after the Rwanda genocide, and three evaluations or assessments since the 2010 Haiti earthquake. He has designed and implemented surveys in Sudan, Somalia, and the DRC for Rwandan and has extensive experience running focus groups and semi-structured surveys and running qualitative data analysis and reporting. Mr. Hansch holds a master’s degree in Public Health – Epidemiology and Biostatistics from Boston University. M&E Technical Sector Expert (Water, Sanitation, and Hygiene), Mr. Peter Lukwiya, has ten years of experience in the evaluation and implementation of international donor-funded water, sanitation and hygiene (WASH) projects with significant experience working in complex environments and circumstances. Recently, Mr. Lukwiya completed an assignment as WASH Consultant for Pact South Sudan, in which he conducted an overall analysis of South Sudan’s WASH sector through comprehensive literature reviews, interviews and discussions with relevant stakeholders from government, NGOs and the private sector. In addition, he recently completed an assignment as WASH Technical Advisor for Save the Children International (SCI) in Somalia, contributing to SCI’s capacity to meet its objectives through the provision of quality emergency water, sanitation and hygiene programming. In Ethiopia, as Emergency WASH Specialist for UNICEF, he led the development of response planning for WASH preparedness and operational framework. In Uganda, Mr. Lukwiya developed participatory M&E strategies and introduced social inclusion and gender issues into the WASH sector in his role as WASH Program Manager for Concern Worldwide. As Evaluation Team Leader for the International Federation of Red Cross and Red Crescent Societies, he was responsible for all aspects of the evaluation design and methodology, led field work, presented findings and recommendations, and led the drafting of the reports for the Ethiopia Emergency Drought Appeal evaluation. Mr. Lukwiya holds a master’s degree in Environmental Sanitation from Ghent University in Belgium. M&E Technical Advisor/Statistician, Mr. Md. Aman Ullah, is a statistician with 20 years of experience in M&E, Management Information Systems (MIS), project management, web management, statistical analysis, research and knowledge management, demography, statistics, biostatistics, data management, and stochastic processes and has performed multiple evaluations in Yemen, Ethiopia and Bangladesh for UNICEF, UNFPA, DANIDA, UNDP, and other international organizations. Currently M&E Consulting Manager for Yemen’s Ministry of Public Health and Population Schistosomiasis Control Project, Mr. Ullah is responsible for independent performance evaluation, coverage verification, mid-term evaluation, resource mobilization, communication, capacity building and internal control of mass drug administration. He has also conducted two national surveys on campaign sites and households covering 3.7 million target populations in 18 Governorates and 43 districts and administered technical and financial verification in 18 Governorates and 263 districts. Previously, he worked with USAID's largest NGO networks in Bangladesh, where he developed an eMonitoring system and Health MIS integrating 168 urban and 155 rural clinics and 26 NGOs and applied quantitative data analysis for clinical quality control and client satisfaction surveys. Mr. Ullah holds a master’s degree in Statistics from Jahangirnagar University. Protection and Information Management Specialist, Ms. Eunice Wavomba, is a humanitarian aid expert with field experience in the mainstreaming of OFDA protection initiatives in conflict environments and experience working on protection programs in Sudan and Iraq for OFDA and the UN. Recently, Ms. Wavomba served as Program Officer for OFDA in Sudan, responsible for the management and provision of technical assistance on programs in the protection, shelter, nutrition, information management, and sanitation sectors and worked with OFDA and Food for Peace officers to coordinate and manage protection program design and integration for the largest OFDA programming in Africa, valued at over $100 million annually. Before that, she served as Team Leader for the UN Sudan Returns, Reintegration and Recovery Unit, responsible for the development of protection referral systems for IDPs in transit to mitigate the risk of harm, exploitation and abuse. Additionally she actively participated in the development of Mine Risk Education (MRE) activities aimed at reducing the risk of injury from unexploded ordinances 19 and mines by raising awareness and promoting behavioral changes through information campaigns and training. Previously, Ms. Wavomba was a humanitarian observer with the UN Office of Peacekeeping Operations where, among other duties, she led field assessment teams in gathering data, drafting field trip reports, and coordinating with state-level government officials and key stakeholders on humanitarian observation missions. Ms. Wavomba holds a master’s degree in sociology form the Washington International University. Prodigy Systems is a Sana’a-based Information and Communications Technology (ICT) company specializing in research and data collection for M&E projects in Yemen and the wider region. Utilizing cutting edge technology to facilitate efficient project activities, over the past seven years Prodigy has built its expertise and capacity in managing complete research projects for several international NGOs and bilateral and multilateral donors, including USAID, UNHCR, RTI International, WHO, and the World Bank. Consisting of 22 fulltime staff and a poll of 200 enumerators with more than two years of data collection experience, its core competencies include developing research methodology, constructing sample frames and drawing samples, developing survey tools and training materials, developing data quality protocols, conducting survey fieldwork, automating data collection tools, and developing data entry and management systems. Prodigy has conducted surveys for several recent large-scale M&E projects in Yemen, including baseline, mid-term, and end line studies for the USAID/Yemen Early Grade Reading Program (YEGRP) in 2012 and 2013 and the WFP and UNHCR-funded IDP Verification and Registration Project in 2012. On both projects, Prodigy developed the survey methodology, work plan and mobile survey instruments, pretested survey tools, and recruited, trained and provided technical support for enumerators, processed and cleaned collected data, and published a website to view the database of all collected data. Please see following pages for key USAID Disclosure of Conflict of Interest Forms. 20 21 22 23 24 STATEMENT OF DIFFERENCES There were no differences of opinion among members of the evaluation team. 25 ANNEX V: OFDA AND FFP STRATEGIC TARGETS OFDA Target Projects/Status as at Summative Evaluation start January 2013 IP Project Title – Amount, Start & End Dates Sectors - Geography Project Status as of December 2013 ADRA (YALLA) Yemen, Abyan and Lahj Livelihood Assistance (YALLA) - Agriculture & Food Security - Economic Recovery & Market System (ERMS) -Mobilization phase: Formation of community committees and selection and verification of beneficiaries lists. -In 1st quarter CY 2013, they reported beginning to register beneficiaries and to distribute livestock. - $ 4,184,694 - 8/15/2013 (start date) - 6/30/2015 Abyan and Lahj Governorates GC/CHF Emergency Assistance to Support Yemeni Communities EASE – Phase 2 - Agriculture & Food Security - WASH Abyan (Zinjibar, Lawdar, Khanfar), Aden (Khormakser, Dar Saad, Al Mansoora), Lahj (Tuban), Taiz (Taiziah), Ibb (Hazm Al Qdain, De-Al Sefal) -Mobilization Phase: Formation of volunteer committees and selection of beneficiaries for various assistance activities. -In 1st quarter CY 2013, they began activities in all sectors - $2,200,000 - 8/10/2013 - 8/10/2014 - Economic Recovery & Market System (ERMS) Abyan (Zinjibar, Lawdar, Khanfar), Aden (Khormakser, Dar Saad, Al Mansoora), Taiz (Taiziah) IMC Emergency Assistance to Vulnerable Populations in Yemen (EAVP-Y) - Health - Nutrition - WASH -Project was operational for all of 2013 and reported on activity results. - $1,943,498 - 2/1/2013 - 1/31/2014 Sana’a: Balad Al Roos, Sanhan, Bani Matar, Al Haymah Al Kharijiyah, and Jahana districts IOM IOM Enhancing the resilience of vulnerable communities via integrated livelihood & WASH - Agriculture & Food Security - Economic Recovery & Market System - WASH -Completed site assessments to identify water infrastructure sites for 15 communities within the targeted districts for rehabilitation. - In 1st quarter CY 2014, began WASH activities. - $2,315,300 - 8/29/2013 - 2/28/15 Abyan (Khanfir, Sarar, Rasad, Jayshan, Sibah, Al Mahfad and Al Wade’a 26 IP Project Title – Amount, Start & End Dates Sectors - Geography Project Status as of December 2013 districts). IRD Community Water Resilience through Sand Dam Construction & Improved Hygiene - Infrastructure - WASH -Mobilized communities, conducted sand dam site selection, began dam construction, and provided WASH training and promotion activities in Hadrawawt -After completion in Hadramawt will move - $1,895,647 to Maharah governate - 8/22/13 - 2/21/15 Hadramawt (Rimah, Thamood, AlQaf), Al Maharah (AlGhaydah), SCI Child-Focused Health, Nutrition and WASH Emergency Response - Health - WASH Sa’ada ( Sehar, Hydan, Sagain Alsafra); Al Hodeidah (Al Hali, Bagil and biet Alfagih); Taiz (Altaiziyah, Almuzaffa) Project completed a 12 month cycle and reported on major results indicators. - $3,999,975 - 3/1/2013 – - 2/28/2014 - Nutrition Sa’ada (Sehar, Hydan, Sagain Alsafra); Hodeidah (Al Hali, Bagil and biet Alfagih) UNICEF Award # AID￾OFDA-IO-13- 00020-02 Award # AID￾OFDA-IO-13-00020 - Nutrition, - Protection (MRE) - WASH -Annual UNICEF report process. -Activities under the grants for 2013 were completed. The focus of formative evaluation was sites from 2013. For the 1 st quarter of CY 2014, each sub-project has determined adjustments in target beneficiaries, exact locations,& processes (i.e. local & other government partners). -UNICEF works mainly through Yemeni government agencies (around 80% of funds as reported during our meeting of 11/26/13) such as YEMAC and local and international NGOs. - $2,184,693 - $6,000,000 - 1/1/2013 - - 12/31/2013 (Refunded as of 1/2014) Countrywide 5 NGOs + 2 UN Agencies US$ 24,723,807 10 Governorates & countrywide 6 sectors of assistance -4 projects recently completing mobilization phase & starting activities -3 awards completed full 12 month project cycle 27 FFP Target Projects/Status as at Summative Evaluation start January 2013 IP Project Title – Amount, Start & End Dates Sectors- Geography Project Status as at December 2013 WFP Emergency Food & Nutrition Support to Food Insecure and Conflict-Affected (EMOP 200451) Nutrition, Food Security -Ongoing implementation throughout 2013 -Summative evaluation focus on 2013 implementation -2014 implementation follows the same processes as 2013 - $46,389,400 - 1/1/2013 - 12/31/2013 Refunded in January 2014 Abyan, Aden, Al-Bayda, Al-Dhalee, Al-Hodeida, Al-Mahwait, Amran, Dhamar, Hadramout, Hajja, Ibb, Lahj, Mareb, Raymah, Sana'a, Taiz ( 16 governorates) ADRA (FARA) Food Assistance, Resilience Achieved (FARA) Project Food Security, Health & Nutrition -Mobilization Phase completed: selected districts, formed community committees, selected and verified beneficiaries -In 1st quarter CY 2014, began distributing food vouchers and food distribution, and nutrition awareness training - $6,400,000 - 9/15/2013 - 9/14/2014 Abyan (Lawdar, Sarar, Khanfir, Zinjibar, Al Wade'a, Ahwar &Mudiya) Mercy Corps Food Security and Resilience Building Program -EFSP Food Distribution, Nutrition -Mobilization Phase completed: Identification and verification of beneficiaries and formation of CAMC (Community Asset Management Committees) - In 1st quarter CY 2013, food voucher distribution began, WASH volunteer training, identified community assets for rehabilitation, and began rehabilitation, selected communities to form Disaster Risk Reduction (DRR) training and form DRR committees -Implementation is at various stages because the start-up is being phased in the three governates - $5,000,000 (for year 1) - 10/1/2013 - 9/30/2016 Taizz (Al Mukha, Mawza, Dhubab); Sana'a (Alhaima Dakhilya, AlhiamaKharijya, Nehm); Lahj (Al Maqaqtirah, AlmadarebahWa Al Arah) 28 GC (YFAD) Yemen Food for Asset Development (YFAD) program Agriculture, Food Security -Mobilization Phase completed: Selection and verification of beneficiaries and formation of CVC (Community Volunteer Committees) - In 1st quarter CY 2014, began distribution of vouchers, and agriculture and food security activities - $5,000,000(for year 1) - 10/1/2013 - 9/30/2016 Taiz (At Ta’iziyah); Ibb (Hazm Al Qdain, Dhi As Sufal, Ibb, Jiblah); Raymah (Al Jabin, Al Jafariyah, Kusmah) SCI (EFSP) Yemen: Emergency Food Security and Resilience Programming - EFSP Resilience, Nutrition -Mobilization Phase mainly completed: Selection and verification of beneficiaries and formation of CRC (Community Resilience Committees) -In 1st quarter CY 2014, began distribution of food vouchers for work, identified community assets for rehabilitation, began vocational training registration, mobilizing health facilities & Mother-to-Mother support groups - $5,000,000 (for year 1) - 10/1/2013 - 3/31/2016 Sana'a (BiladArRus); Dhamar (Utmah, MaghiribAns) 4 NGOs & 1 UN agency US$ 67,789,400 16 governorates 5 sectors of assistance - 4 projects completed mobilization phase - ongoing implementation of WFP activities since beginning of 2013 29 Summary of SIs MRE 2 4 Health 19 16 WASH 6 0 Agriculture – Livelihood and Food Security 8 2 TOTAL 35 22 FGDs summary TOTAL OF SIs & FGDs = 90 Sector M F MRE Health & Nutrition FGD 1 3 1 WASH FGD 1 4 0 FGD 2 3 0 Agriculture – Livelihood and Food Security FGD 1 3 0 FGD 2 2 0 FGD 3 4 2 FGD 4 3 3 FGD 5 5 0 Total 27 6 IP Personnel Interviewed 40 9 OVERALL 102 37 139 individuals SUMMARY OF PERSONS INTERVIEWED (details are in a separate document for confidentiality) Sector M F 57 49 33 No Participants ANNEX VI: RESULTS OF THE FIELD INQUIRY 30 Field-Work Summary of Interviews for Formative Evaluation Number One-on-One Interviews Completed: 1,488 Breakdown by Sector and Target Group Males Females Percent Female Health & Nutrition Interviews with Beneficiaries 8 117 94% Heath Worker Interviews at Centers 14 6 30% Food Distribution Interviews 481 711 60% WASH Interviews conducted 16 81 84% MRE Student Interviews 24 16 60% MRE Teacher Interviews 10 4 29% Total: 553 935 63% 31 ANNEX VII: DATA COLLECTION INSTRUMENTS QUALITATIVE INSTRUMENTS HEALTH AND NUTRITION - Structured Interviews Stakeholders Sample Size Governorate Health Office (GHO) 1 per governorate Sana’a, Taiz and Hodeida District Health Office (DHO) 1 per district Sana’a (Bani Matar, Al Hayma Al Kharijia, Jahan, Sanhan) Hodaidah (Alhali, Bajil, Bait Alfaqeeh, Taiz (Altaiziah) Community Health Workers 8 in total 2 men and 2 women per IP sites (IMC, SCF) Question and Filters 1. Gover norate Health Office (GHO) 2. District Health Office (DHO) 3. Volunteer Community Health Workers Comments Quality of Services and Support 1 What support (health and nutrition) are you getting from the IPs? (IMC, SCF) x x x Ask for Health and Nutrition separately 2 Is this support adequate in terms of type of health and nutrition services? Why or why not? x x x 3 What additional health and nutrition services should be provided to meet the needs of the population? x x x Ask for Health and Nutrition separately 4 Is there a national protocol for the management of severe acute malnutrition? Is the government monitoring its use? x x Training and staffing 5 Is there is a high turnover of staff in the health system? If ‘Yes’, what are the reasons? How can it be improved? x x 6 Is staffing sufficient in numbers and competencies for the functions that must be performed? If ‘No’, what are the gaps? x x 32 7 Do you agree that the profile of health care providers is considered for the selection of trainees for specific training programs? x x 8 Have you been trained? How long have you been trained? Which topics were covered in the training? Was the training adequate to teach/counsel the community? x Drugs and Medical Supplies 9 Are the medical supplies (e.g., medical kits, equipment, consumables) distributed appropriate to the need of the health facilities in the governorate/district? x x 10 If ‘No’, what are the gaps? x x Accessibility 11 Are most children and women in the community making use of these health and nutrition services? If NOT, what are the reasons? x x Ask for Health and Nutrition separately 12 What is being done to increase access to health and nutrition services by women and children? x x Ask for Health and Nutrition separately 13 In areas where it is difficult and/or hard to reach, how are you ensuring that these populations are served? x x x 14 Do you make home visits? Are you targeting men and women separately for health and nutrition education? x Ask for Health and Nutrition separately 15 What strategies do you use for targeting the illiterate women and men in the community? x x x 16 Who in the household/community are influential of Infant Young Child Feeding practices? x 17 Who are the targets for the behavior change interventions for Infant and Young Child Feeding? x 18 Who in the household/community are influential of Health care seeking? x 33 19 Do you know any children/women severely malnourished in your community? x 20 Can you identify malnourished cases in your community? x Management oversight, regulation and accountability 21 How often do you meet with IPs to discuss? Is the frequency of the interaction with the IPs team adequate? What do you discuss during these meetings? x x 22 Do you provide reports to IPs? If so how frequently? What type of information is contained in the report? Do you have a reporting format or template from the IPs? x x 23 Do you supervise the health facilities? If ‘Yes’, how frequently? Do you have a supervision checklist? x x 24 Have the health and nutrition services made a difference in your health and that of your community? How? x Challenges 25 What are the challenges in implementing health and nutrition programs? x x x Ask for Health and Nutrition separately 26 How are you addressing them? x x x 34 HEALTH AND NUTRITION – Focus Group Discussions Interviews with Community Asset Management Committees Stakeholders Sample Size 1 per governorate Mercy Corps Governorates Taiz Name of district Al Mukha Question and Filters Comments Committee selection 1 How were the members of the Community Asset Management Committee (CAMC) selected? 2 Do you feel this group is representative of the community? If not, why not? Food Voucher Beneficiary selection 3 What was the process followed to construct, verify and finalize the beneficiary list for the Mercy Corps project? 4 Was this a good process or would you change it somehow, and, if so, how? 5 Were there differences of opinions as to who should be on list, and, if so, how did you resolve these differences? (Give examples) 6 Are there vulnerable groups or households who were NOT included on the beneficiary list that should be included? (Give examples)? 7 Are there groups or households who were included on the beneficiary list that should NOT be included? (Give examples)? Food voucher 8 Was the voucher you received adequate for your family?  Quantity  Quality Training 9 Are the nutrition and hygiene promotion trainings adequate to bring about health and nutritional benefits? Community Asset Building 10 How did the community decide on the types of community projects? 35 PROTECTION- MRE Structured Interviews Stakeholders Sample Size Government Counterparts ( Local Government Unit, Rural Council ) a) Department of Social Affairs and Labour -Faiza abdu majid 770624583- Khormaksar district, Aden governorate b) Education Office- Lola Saeed Ali 770760126- Kraiter district, Aden governorate 2 Local or Implementing Partners a) AL Mehraq NGO- Anwer abdu ali 777134322 - Altwahi district, Aden governorate b) Yemen Women Union- Majida Saleh Moh 733749329, Al￾Mullah district, Aden governorate 2 Question and Filters 1. Government Counterpart 2. Local or Implementing Partners Comments 1 Do most children attend school or only a few? What percentage? x x 2 Is there a difference between the number of boys and girls who attend school x X 3 What impact do mines and ERW have on the daily lives of people in the community? Have they stopped doing things they used to do in the area? x X 4 In the last 12 months, are you aware of any children who have suffered injuries from explosive devices? If so, where? How many? x 5 In the last 12 months, are you aware of any animals that have been killed or injured by explosive devices? If so, where? How many? x 6 What groups (farmers, herders, etc) of the community do you feel may be at x X 36 risk from a mines and ERW injury? (probe who, why, activities of these people) 7 How much local knowledge is there about landmines? How does this information get passed on? x X 8 How do you select beneficiaries for MRE? What role does the affected population have in selecting? x x 9 Is the beneficiary selection available to the local government authorities x 10 In which districts were the needs - based assessments conducted prior to the project implementation? x 11 In areas where you don’t have access, how are you ensuring that targeted populations are reached? x 12 For MRE school based programs, how are the schools selected for the program? Who is responsible for selecting these schools? x 13 Is the selection procedure for these schools transparent and inclusive? x x 14 Who is responsiblefor selecting teachers to participate in MRE training? x x 15 Is the selection process for teachers transparent? x x 16 How are landmines a threat to youth? x x 17 Within the targeted population, does the program prioritize specific groups?(if yes, go to # 18) x X 18 Which groups? Why are they prioritized? x X 19 In what aspects of the process is the community involved in the program? x X 20 How do women and girls access MRE? x X 21 Are your field reports disaggregated by gender, age, disability and vulnerability X 37 criteria? 22 What measures are you taking to ensure that those who are illiterate, and those with disabilities such as the blind and deaf have access to information? x x 23 What type of assistance are you receiving from UNICEF to support MRE? x X 24 Is this assistance adequate? Why or why not? x x 25 How often do you meet with UNICEF to discuss? Is the frequency of the interaction with the UNICEF team adequate? What do you discuss during these meetings? x X 26 Do you provide reports to UNICEF? If so how frequently? What type of information is contained in the report? Do you have a reporting format or template from UNICEF? x X 27 What would you say were key challenges in implementing MRE and how have you been able to overcome them. x X 38 AGRICULTURE-LIVELIHOODS AND FOOD SECURITY – STRUCTURED INTERVIEWS (ADRA YALLA/FARA) Organization ADRA YALLA and FARA projects 1. IP Field Staff ADRA staff working with communities 2. Government Local Councils where ADRA projects are operational Questions and Filters 1. IP Field staff 2.Gov ernment Project Background – Comments from ADRA interviews TARGET POPULATION 1-Who is the primary target population for ADRA projects? Question format: The question will be open-ended: Who is the target population and allow the respondent to answer x x ADRA-Committees includes district council, teacher ADRA selected Committees and did some voucher distributions only …did not distribute agriculture inputs yet nor done training (?) The target population can be women, female head of household, households that are without the means to earn income, returnees. The response should explain why these households were targeted 2-What were the selection criteria or characteristics of the beneficiaries (what characteristics determined whether a household or person would be included on the beneficiary lists? Question format: What were the characteristics or criteria for selecting beneficiaries? Should the selection process include any other household or individual characteristics to better identify the right beneficiaries? Yes, No, No opinion If yes, which characteristics? x x ADRA- selection criteria = Conflict affected areas only. Returnees, Pregnant and Lactating Women, malnourished child in the household, children under 5, size of household, disabled person in household, income (rank). 3- Who participated in the selection process? Question Format Who participated in creating the beneficiary list? Who was most influential? x x ADRA-Beneficiaries decide if interested in the activities ADRA has to offer. 39 Do you think these were the most representative people to carry out the selection? Yes, No, No opinion If no, why not and who should have been included? 4-Were there any households or people who were NOT selected who you think should have been? Question Format Was there any type of household that did not get included who should have been? x x Note: Ask respondent to indicate any type of person/household that was excluded (e.g., IDP, returnee, landless, disabled, widows, nomadic people, specific tribes, and include an “other” category with a blank to add other types of people. Let them select all that apply 5- Did the ADRA effectively verify the list of beneficiaries? Question Format: Did the ADRA verify the beneficiary list? Yes, No, No opinion Do you think that the verification process was effective – do you have confidence in the verification process? Yes, No, No opinion If no, what was the problem: And, do you have suggestions on how to improve the process? x x 6- Do you think the selection criteria and process was fair? Question Do you think the selection process was fair and representative? Yes, No, No opinion If not, why not? And, what could be done to improve it? x x Comment I want to find out what they thought of the process – did they think it was fair. This question is not about the criteria but the process VOUCHERS/INPUTS DISTRIBUTED 7- Was the voucher you received adequate for your family?  Quantity field teams ADRA has only selected distribution sites and will distribute agricultural inputs this week or next ADRA has procured some livestock but has not yet distributed it. 40  Quality Question What is the purpose of the vouchers and kits? Is the quantity and quality of the items distributed (animals, inputs, etc) suitable to restore the beneficiaries’ livelihood/income? Yes, No, No opinion If no, why not? And, how would you change the distribution package? Comment I want to understand if others such as ADRA field staff and government council, etc think the distribution is adequate for the beneficiaries given the aim to restore their livelihoods 8- Were the right type of animal and breed of animal distributed? Is the right type of animal and breed of animal being distributed? Yes, No, No Opinion If no, what type/breed would have been better? x Question ADRA Aden technical specialist has identified animals but not yet distributed 9- Are the benefits and training adequate to restore households’ pre-conflict standard of living Question: Is the planned training adequate to change the beneficiaries’ practices in the intended ways and to restore the beneficiaries’ standard of living/income? Yes, No, No opinion If no, what is needed to create the desired change and to fully restore the standard of living?? Comment ADRA-hasn’t started training…so this question is not applicable to ADRA projects IP PREFORMANCE 10- Did ADRA monitor the distribution process well – did the right people get the right vouchers/inputs/goods at the expected time? Is ADRA adequately monitoring the distribution process? Yes, No, No opinion If no, how are they not? And, what could they do to improve their management? x x ADRA-Food voucher distributions only start this week/next week 41 11- Does ADRA have adequate technical capacity to deliver the project goods and services? Questions Did ADRA consult you on the type and animals to distribute? Yes, No, No opinion If yes, did they take your advice? Did ADRA consult you on the type of feed and/or fodder to distribute? Yes, No, No opinion If yes, did they take your advice? x x ADRA-haven’t started. They did not consult Ministry of Agriculture, but rather the committee and other locals. Comment I just want to get a sense if ADRA is seeking technical advice especially since none have agriculture staff ______________________ 12- How well are ADRA’s projects and services coordinated with the local community? Question In the planning of activities, has ADRA adequately collaborated with the communities? Very well, well, not very well, no opinion If very well, what was particularly good about it? If not well, give an example and describe how ADRA resolved it x x ADRA – hasn’t started activities so the committee’s impressions would be limited to organization and planning 13- How well are ADRA programs and services coordinated with the local government and authorities? In the planning of activities, has ADRA adequately collaborated with the communities? Very well, well, not very well, no opinion If very well, what was particularly good about it? If not well, give an example and describe how ADRA resolved it x x ADRA – haven’t started activities so respondent’ impressions would be limited to organization and planning. Works with committee on procurement of agricultural inputs 14- How well are ADRA projects and services coordinated with other organizations’ programs in the area? Question How well has the ADRA collaborated with other x x ADRA – located in Aden where there is a regional Agriculture and Food Security Cluster IOM – ADRA is livestock focal point/common activities 42 organizations and projects in the area? Very well, well, not very well, no opinion If very well, what was particularly good about it? If not well, give an example and describe how ADRA resolved it 15- Were there any GAPS in terms of the type of services? Question Are there aspects to the specific interventions such as livestock restocking, restoring livelihoods and keyhole gardens that are lacking in the design or implement of the program? Yes, No, No opinion If yes, what was lacking and how could the gap be filled? x x ADRA-not yet delivering services so the impressions will be based on impressions about current design, inclusivity, procurement, security 16-Are there lessons learned that should be shared? Question Are there any lessons learned about the administration of the program thus far that you like to share? Yes, No, No opinion If yes, what are they? x x ADRA-not yet delivering services so the impressions will be based on impressions about current design, inclusivity, procurement, security 43 AGRICULTURE-LIVELIHOODS AND FOOD SECURITY – STRUCTURED INTERVIEWS (GC/CHF YFAD) Organization GC/CHF YFAD project 1. IP Field Staff GC staff working with communities (Taiz, Ibb, Raymah) Contact: Muna Mohammed Hashem – Field Coordinator 738-434-378 Fawzi Abdulhafedh Hamoud 738-434-379 2. Government Social Welfare Fund (MOU in process) Rahama al-Sofi, Manager of Conditional Assistance at Social Welfare Fund; contacts cell:771206771) Contact reference for GC Abdul Salam Al Kohlani – Deputy Program Director 737-700-163 Ayid Sharyan – Monitoring and Evaluation Specialist 737-789-010 Questions and Filters 1. IP Field staff 2.Gov ernment Project Background – Comments from GC/CHF interviews TARGET POPULATION 1-Who is the primary target population for GC project? Question format: The question will be open-ended: Who is the target population and allow the respondent to answer x x GC – works with local committees. Selection is primarily based on Social Welfare Fund lists. The target population can be women, female head of household, households that are without the means to earn income, returnees. The response should explain why these households were targeted 2-What were the selection criteria or characteristics of the beneficiaries (what characteristics determined whether a household or person would be included on the beneficiary lists? Question format: What were the characteristics or criteria for selecting beneficiaries? Should the selection process include any other household or individual characteristics to better identify the right beneficiaries? Yes, No, No opinion If yes, which characteristics? x x GC – selection criteria = Social Welfare Fund and income rank GC – selection criteria for Agriculture (not keyhole garden) – own land 3- Who participated in the selection process? x x GC-Beneficiaries express interest in the activities that GC 44 Question Format Who participated in creating the beneficiary list? Who was most influential? Do you think these were the most representative people to carry out the selection? Yes, No, No opinion If no, why not and who should have been included? has to offer. 4-Were there any households or people who were NOT selected who you think should have been? Question Format Was there any type of household that did not get included who should have been? x x GC-notes that in south the communities don’t like government involvement so GC informs the local councils but the local councils doesn’t participate in the project committees ------------------------------------------------------ Note: Ask respondent to indicate any type of person/household that was excluded (e.g., IDP, returnee, landless, disabled, widows, nomadic people, specific tribes, and include an “other” category with a blank to add other types of people. Let them select all that apply 5- Did GC/CHF effectively verify the list of beneficiaries? Question Format: Did GC/CHF verify the beneficiary list? Yes, No, No opinion Do you think that the verification process was effective – do you have confidence in the verification process? Yes, No, No opinion If no, what was the problem: And, do you have suggestions on how to improve the process? x x GC-Social Welfare Fund isn’t involved in this part, they just provided the lists. 6- Do you think the selection criteria and process was fair? Question Do you think the selection process was fair and representative? Yes, No, No opinion If not, why not? And, what could be done to improve it? x x GC-Social Welfare Fund isn’t involved in this part, they just provided the lists. Comment I want to find out what they thought of the process – did they think it was fair. This question is not about the criteria but the process 45 VOUCHERS/INPUTS DISTRIBUTED 7- Was the voucher you received adequate for the family?  Quantity  Quality Question What is the purpose of the vouchers and kits? Is the quantity and quality of the items distributed (animals, inputs, etc) suitable to restore the beneficiaries’ livelihood/income? Yes, No, No opinion If no, why not? And, how would you change the distribution package? x GC-have distributed vouchers so far. Comment I want to understand if others such as GC/CHF field staff and government council, etc think the distribution is adequate for the beneficiaries given the aim to restore their livelihoods 8- Were the right type of animal and breed of animal distributed? Is the right type of animal and breed of animal being distributed? Yes, No, No Opinion If no, what type/breed would have been better? Question GC has distributed vouchers so far This question is not be relevant to GC/CHF 9- Are the benefits and training adequate to restore households’ pre-conflict standard of living Question: Is the planned training adequate to change the beneficiaries’ practices in the intended ways and to restore the beneficiaries’ standard of living/income? Yes, No, No opinion If no, what is needed to create the desired change and to fully restore the standard of living?? Comment GC-hasn’t started training…so this question is not applicable to GC project GC – hasn’t started training IP PREFORMANCE 10- Did GC/CHF monitor the distribution process well – did the right people get the right vouchers/inputs/goods at the expected time? Question Is GC/CHF adequately monitoring the distribution process? Yes, No, No opinion If no, how are they not? And, what could they do to improve their management? x x GC-only vouchers distributed 46 11- Does GC/CHF have adequate technical capacity to deliver the project goods and services? Question Did GC/CHF consult you on the type of inputs to distribute? Yes, No, No opinion If yes, did they take your advice? Did GC/CHF consult you on the type of feed and/or fodder to distribute? Yes, No, No opinion If yes, did they take your advice? x x Comment I just want to get a sense if GC/CHF is seeking technical advice especially since none have agriculture staff ______________________ 12- How well are GC/CHF s projects and services coordinated with the local community? Question In the planning of activities, has GC/CHF adequately collaborated with the communities? Very well, well, not very well, no opinion If very well, what was particularly good about it? If not well, give an example and describe how GC/CHF resolved it x GC – haven’t started activities so Social Welfare impressions would be limited to organization and planning 13- How well are GC/CHF programs and services coordinated with the local government and authorities? Question In the planning of activities, has GC/CHF adequately collaborated with the communities? Very well, well, not very well, no opinion If very well, what was particularly good about it? If not well, give an example and describe how GC/CHF resolved it x x GC– haven’t started activities so the Social Welfare impressions would be limited to organization and planning (some distributions of vouchers have taken place recently) 14- How well are GC/CHF projects and services coordinated with other organizations’ programs in the area? Question How well has the GC/CHF collaborated with other organizations and projects in the area? Very well, well, not very well, no opinion If very well, what was particularly good about it? x GC – developing e-voucher system that other organizations will use. Please ask for names of these organizations 47 If not well, give an example and describe how GC/CHF resolved it 15- Were there any GAPS in terms of the type of services? Question Are there aspects to the specific interventions such as livestock restocking, restoring livelihoods and keyhole gardens that are lacking in the design or implement of the program? Yes, No, No opinion If yes, what was lacking and how could the gap be filled? x GC-not yet delivering other than vouchers so impressions will be based on impressions about current design, inclusivity, procurement, security 16-Are there lessons learned that should be shared? Question Are there any lessons learned about the administration of the program thus far that you like to share? Yes, No, No opinion If yes, what are they? x x GC-not yet delivering services so the impressions will be based on impressions about current design, inclusivity, procurement, security GC is developing the e-voucher system and may have lessons related to that already ________________________ Comment Can only assess the selection and first delivery process but it gives some feedback 48 AGRICULTURE-LIVELIHOODS AND FOOD SECURITY – STRUCTURED INTERVIEWS (IOM) Organization IOM 1. IP Field Staff IOM staff working with communities (Aden Office) Livelihoods: Mageed Alkaldi Senior Field Coordinator +967 737 888 764 Water sites rehabilitation: Eng. Waleed Ali Adhban - WASH specialist +967 736 856 280 Ali Al Jeffry- Emergency Operations Assistant +967 736 333 502 2. Government - GARW (General Authority of Rural Water) is consulted because IOM starts by identifying water projects - community water points. - Ministry of Public Health and Population (MOPHP) is consulted because the water points are often health facilities (??) MOPHP is consulted also because there are human/animal multiuse best practices to consider. - Ministry of Agriculture and Irrigation extension (MOAI). - Local Council is merely to keep informed. They don’t tend to be in the Committees Contact reference for IOM Marco Chimenton IOM Aden Sub-Office – Officer in Charge +967 736 900 068 (Aden) Ahmed Amin Abdo - Monitoring & Evaluation Assistant +967 737 888 772 Note: To interview government institutions listed under the government category in the regions selected for the field research. The only list of contact provided by IOM is attached with the email. Other contact for government institutions collaborating with IOM to be obtained directly from IOM field staff. To conduct the structured interviews with at least 3 to 4 people for every government category. Questions and Filters 1. IP Field staff 2. Gov ernment Background Comments TARGET POPULATION 1 - Who is the primary target population for IOM Program? Question format: The question will be open-ended: Who is the target population and allow the respondent to answer x x IOM is using a community approach and starts by identifying water activities because these resources are often community resources and it is a way to orient the program toward community action. Health facilities are included because the water points are often health facilities . IOM starts with community water points. The target population might be e.g., women, Female head of household, households that are without the means to earn income, returnees. A response should be more specific than just “vulnerable household” 49 2-What were the selection criteria or characteristics of the beneficiaries (what characteristics determined whether a household or person would be included on the beneficiary lists? Question format: What were the characteristics or criteria for selecting beneficiaries? Should the selection process include any other household or individual characteristics to better identify the right beneficiaries? Yes, No, No opinion If yes, which characteristics? x x IOM – selection criteria = Female head of households, large families, disabled heads of households, or households with disabled who would be earning income, health considered. Comment Note: Enumerator checks off the various characteristics for the beneficiaries. Use the criteria listed above as response options and allow the respondent to list other criteria. 3- Who participated in the selection process? Question Format Who participated in creating the beneficiary list? Who was most influential? Do you think these were the most representative people to carry out the selection? Yes, No, No opinion If no, why not and who should have been included? x x IOM form committees for beneficiary list and beneficiaries select their actual activity. Committees say what is important, but if a beneficiary is included on the list and has a vocation, they get a micro enterprise kit. Comment I want to know what types of people were included (village leaders, sheik, teacher, women, etc…others ) 4-Were there any households or people who were NOT selected who you think should have been? Question Format Was there any type of household that did not get included who should have been? x x Note: Ask respondent to indicate any type of person/household that was excluded (e.g., IDP, returnee, landless, disabled, widows, nomadic people, specific tribes, and include an “other” category with a blank to add other types of people. Let them select all that apply 50 5- Did the IOM effectively verify the list of beneficiaries? Question Format: Did the IP verify the beneficiary list? Yes, No, No opinion Do you think that the verification process was effective – do you have confidence in the verification process? Yes, No, No opinion If no, what was the problem: And, do you have suggestions on how to improve the process? x x 6- Do you think the selection criteria and process was fair? Question Do you think the selection process was fair and representative? Yes, No, No opinion If not, why not? And, what could be done to improve it? x x Comment I want to find out what they thought of the process – did they think it was fair. This question is not about the criteria but the process VOUCHERS/INPUTS DISTRIBUTED 7- Was the voucher you received adequate for your family?  Quantity  Quality Question What is the purpose of the vouchers and kits? Is the quantity and quality of the items distributed (animals, inputs, etc) to restore the beneficiaries’ livelihood/income? Yes, No, No opinion If no, why not? x x Background IOM – will distribute actual (in kind) livelihoods kits and animals, depending on the household income earning history. IOM-will distribute a stipend (small amount of money) to the livelihoods beneficiaries (only) as a method to protect assets/what they just received in their vocational kit. IOM-Ministry of Agriculture and Committees input for fodder, feed and animal selection. They have not distributed yet, but they have selected what and how much they will distribute so that respondents can comment on that. IOM is only at procurement stage so can only ask about quality of 51 And, how would you change the distribution package? vouchers from those who gave technical input. Comment I want to understand if others such as IOM field staff and government council, etc think the distribution is adequate for the beneficiaries given the aim to restore their livelihoods. 8- Were the right type of animal and bread of animal distributed? Question Is the right type of animal and breed of animal being distributed? Yes, No, No Opinion If no, what type/breed would have been better? x IOM-Ministry of Agriculture Committees input for fodder, feed and animal selection IOM is only at procurement stage so can only ask about quality of vouchers from those who gave technical input. 9- Are the benefits and training adequate to restore households’ pre-conflict standard of living? Question Is the planned training adequate to change the beneficiaries’ practices in the intended ways and to restore the beneficiaries’ standard of living/income? Yes, No, No opinion If no, what is needed to create the desired change and to fully restore the standard of living? x x IOM – haven’t started vocational training IOM-Ministry of Agriculture working on veterinary training, livestock management for beneficiaries IOM-Private sector will provide practicum but this is for the future Comment I want to understand what those who have been working with the IOM and helping to create training materials and sessions think of the adequacy of the training – will it result in desired changes in behavior? IP PREFORMANCE 10- Did IOM monitor the distribution process well – did the right people get the right vouchers/inputs/goods at the expected time? Question Did the IOM consult you on the type and animals to distribute? Yes, No, No opinion IOM – haven’t started distributions (does not apply to IOM) 52 If yes, did they take your advice? Did the IOM consult you on the type of feed and/or fodder to distribute? Yes, No, No opinion If yes, did they take your advice? 11- Does IOM have adequate technical capacity to deliver the project goods and services? Questions Did the IP consult you on the type and animals to distribute? Yes, No, No opinion If yes, did they take your advice? Did the IP consult you on the type of feed and/or fodder to distribute? Yes, No, No opinion If yes, did they take your advice? x x Background IOM – haven’t started training Ministry of Agriculture experience would be limited to selection of animals and fodder and feed and training design and discussions of veterinary services design. Comment I just want to get a sense if IOM is seeking technical advice especially since none have agriculture staff 12- How well are the IOM programs and services coordinated with the local community? Question In the planning of activities, has IOM adequately collaborated with the communities? Very well, well, not very well, no opinion If very well, what was particularly good about it? If not well, give an example and describe how IOM resolved it x x IOM – haven’t started activities so the respondent impressions would be limited to organization and planning. 13- How well are IOM programs and services coordinated with the local government and authorities? Question In the planning of activities, has IOM adequately x x IOM – haven’t started activities so the respondent’ impressions would be limited to organization and planning. 53 collaborated with the communities? Very well, well, not very well, no opinion If very well, what was particularly good about it? If not well, give an example and describe how IOM resolved it 14- How well are IOM programs and services coordinated with other programs in the area? Question How well has IOM collaborated with other organizations in the area? Very well, well, not very well, no opinion If very well, what was particularly good about it? If not well, give an example and describe how IP resolved it x x IOM-district councils; IOM-ADRA; IOM-Oxfam; IOM-ICRC (all these organizations are working in IOM project regions. IOM – ADRA is livestock focal point IOM – Oxfam is the coordination focal point IOM – ICRC on WASH (consider this because water is the entry point of activity) 15- Were there any GAPS in terms of the type of services? Question Are there aspects to the specific interventions such as livestock restocking, restoring livelihoods and keyhole gardens that are lacking in the design or implement of the program? Yes, No, No opinion If yes, what was lacking and how could the gap be filled? x x IOM-not yet delivering services so the respondent’ impressions will be based on current design, inclusivity, procurement, security… 16-Are there lessons learned that should be shared? Question Are there any lessons learned about the administration of the program thus far that you like to share? Yes, No, No opinion If yes, what are they? x x IOM-not yet delivering services so the respondent’ impressions will be based on current design, inclusivity, procurement, security… Comment Can only assess the selection and first delivery process but it gives some feedback 54 AGRICULTURE-LIVELIHOODS AND FOOD SECURITY –Focus Groups المنظمة المشروع المجموعات المركزة / ألعدد / لوائح اللجان group focus facilitate to Contact Faryal Minhas 771772389 Mr. Omer Omer 777920769 - 3 Focus groups – total of 15 participants - Suggested location governorate of Abyan - List of committee and members attached with email YALLA & FARA projects ADRA Abdul Salam Al Kohlani 737-700-163 Ayid Sharyan 737-789-010 3 focus groups- total of 15 participants (2 FG in Raymah) and 1 FG in Taiz (north). Do not have committee list…newly formed committees. To arrange through local office of YFAD YFAD Project GC/CHF No Focus groups Only structured interviews Enhancing resilience of vulnerable communities IOM Focus Group Questions 1. How were the members of this Committee selected? 2. Do you feel that this group is representative of the community? If not, why not? 3. What was the process followed to construct, verify and finalize the beneficiary list? 4. Was this a good process or would you change it somehow, and, if so, how? 5. How did the Committee decide on the list of beneficiaries? Were there some specific characteristics you used to select households? (Give examples) 6. Were there differences of opinions as to who should be on list, and, if so, how did you resolve these differences? (Give examples) 7. Are there vulnerable groups or households who were NOT included on the beneficiary list that should be included? (Give examples)? 8. Are there groups or households who were included on the beneficiary list that should NOT be included? (Give examples)? 9. How did the community decide what types of livelihood (livestock restoring, keyhole gardening, vocational training, beekeeping, etc) would be supported? 10. How was it decided which type of livelihood assistance a beneficiary would receive? 11. Is there other assistance related to agriculture and income earning that vulnerable households need in order to restore their livelihoods, and, if so, what? 55 WATER, SANITATION AND HYGIENE STRUCTURED INTERVIEW QUESTIONNAIRE INSTRUCTIONS: The interview contains 3 question areas. The interview should last no more than ___ minutes. We recommend that one interviewer records the candidate responses. The question areas to be addressed are directly linked to the Person Specification. Each question area should be covered in turn. Interviewer 1 asks the questions while the candidate responses are recorded by interviewer 2. Ensure that this report form has the candidate’s name recorded and the date and time of the interview. The following shows the interview question matrix. No Questions and Filters 1. Government counter parts 2. Local Partners(CSSW and Al Khair) 3. Local Councils 4.Comments A Beneficiary (Communities) , Project site identification and selection 1. Who are the primary beneficiaries of the Water, Sanitation and Hygiene project? (IDP, host communities) x x 2. Are specific categories of the community (Age, Gender, Disabled and Elderly) targeted to benefit from the water, sanitation and hygiene project? x x 3. If, yes, why are these members of the population specifically targeted? x x 4. What criteria were used to select the community group/project areas? x x 5. Is there any community group/areas left out in the project selection process? If so, why? x x 6. What was the level of participation of the community during project sites’ selection, planning and implementation process? x x x 7. Have there been any x x x 56 challenges in selection of communities and sites for the project? 8. If yes, how did you address these challenges? x x 9. What measures were put in place to ensure needs of communities is taken into consideration? x B Water, Sanitation and Hygiene Programming design 1. Is your organization implementing all water, sanitation and hygiene activities? And why? x 2. What technical aspects were taken into consideration during the designs of the water, sanitation and hygiene project? x 3. In terms of water, do the water sources in the area provide sufficient quantity of water to meet communities ‘needs? During all period s of the year? x x 4. If yes, Is it sufficient for short term and long term needs? x x 5. Is there a considerable number of livestock in the project area? If yes, what is the provision for drinking water for the livestock? x x 6. In your opinion, what are the gaps in the current water, sanitation and hygiene programming? x x 7. If any gaps have been identified, what are the best ways to address these gaps x x 57 in a timely way? C Programme Implementation process 1. How does your organization monitor progress, challenges/bottlenecks during the activities implementation? x 2. Does your project have some elements of Disaster Risk Reduction management? x x 3. In your opinion, does your organization have appropriate technical human resources and capacity to effectively implement the water, sanitation and hygiene activities? x 4. How does your organization coordinate with external actors, particularly the UN (the Water sanitation and hygiene cluster system), INGOs? x x D Risks and challenges 1. What main factors helped or hindered water, sanitation and hygiene service deliveries (security events, logistics, infrastructure, procedures, access, etc.)? x x 2. How does your organization mitigate these risks? x x 58 Lesson Learned Questions- for LOCAL NGOs ONLY Questions in the final stage of an interview should cover general learning, and can be direct or indirect, as follows. The actual questions asked will depend on the answers to previous questions. 1. a). When you look back on the Water sanitation and hygiene project implementation, what do you think went well along all phases of the implementation? b). what hasn’t gone well? c). and why? 2. Looking back, what are the 2- 3 things that you would change and which would improve the water sanitation and hygiene project implementation? 3. Is there any question that you were expecting and which I have not asked? WATER SANITATION AND HYGIENE FOCUS GROUPS DISCUSSION QUESTIONNIARE 1. How were you identified as beneficiaries? 2. Were you satisfied with the identification process? If not why? 3. What is your current level of participation in the project? 4. How many times are you meeting? 5. Did you participation in project site selection and are you aware of the criteria used? 6. How was your committee members group formed? 7. Were you satisfied with the committee group formation process? If not why? 8. Do the feel that your committee is representative the community, in terms of gender, or tribe? 9. Are you receiving any additional support like trainings from the organization? 59 Quantitative Instruments Yemen OFDA/FFP Monitoring & Evaluation Project (YOFMEP) - Formative Evaluation, March 2014 استبيان - وال تغذي ه ال صحة QUESTIONNAIRE WORKER HEALTH – NUTRITION & HEALTH ال صحي ال مرك ز موظفي HNW1. Governorate : :المحافظة HNW2. District : :المديرية HNW3. Health Facility (Name): :)االسم )الصحي المرفق HNW4. Interviewer’s name: :المبحوث اسم HNW5. Date: :التاريخ HNW6. Time: :الوقت HNW7. Implementing Partner : 1. IMC 2. Save the Children Save the Children .2 IMC1 :المنفذة المنظمة General Background خلفية عامة HNW8 Sex of the respondent 1 ذكر Male جنس المبحوث 2 أنثى Female HNW9 Respondent age العمر Age عمر المبحوث HNW9.01 Don't know/ Refuse رفض /اعلم . 888 60 HNW10 Respondent’s position/title وظيفة المجيب HNW11 Number of technical staff in the facility عدد الموظفين الفنيين في المرفق HNW12 How is this number made up? مكونات هذا العدد HNW12.01 Doctors أطباء HNW12.02 Nurses الممرضات HNW12.03 Midwives قابالت HNW12.04 Medical Assistants طبيين مساعدين العاملين في المختبرات personnel Laboratory 12.05HNW HNW12.06 Pharmacy صيادلة Health الصحة Health and Nutrition support في مجال دعم الصحة و التغذية HNW13 How do you rate the government’s budget allocation to the health care system? 1 جيد جداً good Very 2 جيدة Good 3 ضعيفة Poor كيف تقيم الميزانية الحكومية المخصصة لنظام الرعاية الصحية؟ HNW14 What support (health and nutrition) are you getting from the IPs? (IMC, SCF) HNW14.01 Training التدريب 1 ما هو الدعم )الصحي و الغذائي( الذي تحصل علي ه من المنظمات 14.02HNW .)IMC/SCF( الداعمة؟ 1 ألدوية و المستلزمات الطبية supplies medical and Drugs HNW14.03 Nutrition supplies تغذية مواد 1 HNW14.04 Budget support الميزانية دعم 1 HNW14.05 Others أخرى 1 61 HNW14.06 Others (specify) )حدد )أخرى؟ HNW15 Is the support adequate in terms type of health and nutrition services ? Yes نعم 1 هل الدعم كافي من حيث نوع الخدمات الصحيةاألساسية و الغذائية؟ 2 ال No Don't know/Refuse رفض /اعلم ال 888 HNW16 Why or why not? لماذا نعم و لماذاال؟ HNW17 What additional primary health and nutrition services should be provided to meet the needs of the population? HNW17.01 Health services خدمات صحية أساسية HNW17.02 Nutrition services خدمات غذائية أساسية ما هي الخدمات الصحيةاألساسية و الغذائية اإلضافية التي يجب تقديمها لتلبية احتياجات السكان؟ Leadership oversight, regulation and accountability اإلدارة، التنظيم و المسؤولية HNW18 How often do you meet with IPs to discuss? 1 شهري Monthly 2 ربع سنوي Quarterly 3 نصف سنوي Biannually كم غالباً تلتقي بالمنظمة المنفذة للمناقشة؟ Not at all علىاإلطالق 4 If not at all, SKIP TO HNW 21 ←) انتقل لسؤال رقم 12في حال كانت على ا طالق( 5 أخرى Others HNW18.01 Others (specify) )حدد )أخرى؟ 62 HNW19 Is the frequency of interaction with the IPs team adequate? Yes نعم 1 2 ال No هل كان التفاعل المتكرر مع فريق المنظمة كافي؟ HNW20 What do you discuss during these meetings? ماذا تناقش خالل هذه االجتماعات؟ HNW21 Do you provide reports to IPs? Yes نعم 1 2 ال No هل تقوم بتقديم التقارير للمنظمة المنفذة؟ ←)انتقل لسؤال رقم 52في حال كانتال( 25 HNW TO SKIP ,No If HNW22 1 شهري Monthly If ‘Yes’, how frequently? Quarterly سنوي ربع 2 3 نصف سنوي Biannually إذا "نعم"، كم عدد المرات؟ Not at all علىاإلطالق 4 HNW23 Is the report data disaggregated by gender and type of services? Yes نعم 1 2 ال No هل بيانات التقرير مصنفة حسب الجنس و نوع الخدمات؟ HNW24 Do you have a reporting format or template from the IPs? Yes نعم 1 2 ال No هل لديك إطار أو شكل للتقرير تم تزويدك ب ه من الشركاء المنفذين؟ HNW25 Have frequently have you been supervised by ministry of health staff from Governorate Health Office/District Health Office? Monthly شهري 1 63 2 ربع سنوي Quarterly كل كم يتم اإلشراف عليك من قبل موظفي وزارة الصحة و من قبل مكتب الصحة في المحافظة/مكتب الصحة في المديرية؟ 3 نصف سنوي Biannually 4 علىاإلطالق )انتقل إلى سؤال رقم 52( all at Not If not at all, SKIP TO HNW 27 HNW26 Does the Governorate Health Office/District Health Office come with a supervision checklist? Yes نعم 1 هل يأتي مكتب الصحة في المحافظة/مكتب الصحة في المديرية بقائمة المراجعة؟ 2 ال No Accessibility of health services إمكانية الحصول على الخدمات الصحية HNW27 Are most children and women in the community making use of these health and nutrition services? Yes ( if yes skip to 29) )52 الى انتقل نعم حال في ) نعم 1 هل أغلب األطفال و النساء في مجتمعك تمكنوا من استخدام هذه الخدمات الصحية و الغذائية؟ 2 ال No HNW28 If No, what are the reasons? إذا "ال"، ما هياألسباب؟ HNW29 Are you targeting men and women separately for health and nutrition education? Yes نعم 1 هل تستهدف الرجال و النساء بشكل منفصل من أجل التثقيف الصحي و الغذائي؟ 2 ال No HNW30 Do you make home visits through voluntary community health workers to target your beneficiaries? Yes نعم 1 هل تقوم بالزيارات المنزلية بواسطة العاملين المتطوعين في صحة المجتمع من أجل استهداف المستفيدين؟ 2 ال No HNW31 Who in the household/community are the most influential of Infant and Young Child Feeding practices? 1 الم Mother 2 الب Father 64 مناألشخاص سواء في ألسرة/المجتمع يعتبر األكثر تأثيراً على ُ ممارسات تغذية الرضع و األطفال الصغار؟ Mother-in-law الحماة 3 4 الجدة Grandmother HNW32 Who in the household/community are the most influential of Health care seeking? Mother الم 1 2 الب Father مناألشخاص سواء في ألسرة/المجتمع يعتبر األكثر تأثيراً من أجل السعي للحصول على الرعاية الصحية؟ Mother-in-law الحماة 3 4 الجدة Grandmother HNW33 Who are the targets for the behavior change interventions for Infant and Young Child Feeding? HNW33.01 (Multiple responses possible) الم Mother 1 HNW33.02 Father الب 1 ُ من هم المستهدفين من برامج تغيير سلوك التغذية لدى الرضع و 33.03HNW ألطفال الصغار؟ )من الممكن وضع ردود متعددة( Mother-in-law الحماة 1 HNW33.04 Grandmother الجدة 1 HNW34 What is being done to increase access to health and nutrition services for women and children? ما الذي تم القيام به لزيادة وصول النساء واألطفال للخدمات الصحية و الغذائية؟ HNW35 In areas where it is difficult and/or hard to reach, how are you ensuring that these populations are served? List the strategies. Outreaches by health workers from زيارتهم بواسطة العمال الصحيين في المنشأة facility the 1 1 عمال صحة المجتمع workers health Community في المناطق التي يصعب الوصول إليها، كيف تتأكد من أن السكان تم تقديم الخدمات لهم؟ )أورداالستراتيجيات( 1 الفرق المتنقلة teams Mobile 1 ال شيء مما سبق above the of None Training of staff and quality of health service تدريب الموظفين و جودة الخدمة الصحية HNW36 Is there a national protocol for the management of 65 severe acute malnutrition? Yes نعم 1 2 ال ) في حال ال انتقل الى 83( (38 to skip no if ( No هل هناك برتوكول وطني لعالج سوء التغذية الحاد؟ HNW37 Is the facility applying this national protocol? 1 نعم Yes 2 ال No هل المرفق يطبق هذا البروتوكول الوطني؟ HNW38 Are international standards being followed to admit Children, pregnant women and lactating mothers into the Moderate Acute Malnutrition and Severe Acute Malnutrition management programs? .1 نعم في حال نعم ) انتقل الى 04( (40 to skip yes if ( Yes 1 هل المعايير الدولي ة متبعة لقبول األطفال، النساء الحوامل و األمهات المرضعات في برامج سوء التغذية المتوسط و سوء التغذية الوخيم؟ 2 ال No HNW39 If ‘No’, what are the challenges? إذا "ال"، ما هي التحديات؟ HNW40 Do you have job aids posted on the wall to assist you in case management? Yes نعم 1 هل لديك ملصقات مساعدة للعمل معلقة على الحائط لمساعدتك في الج الحاالت؟ 2 ال No HNW41 Is there a high turnover of staff at the health facility? 1 نعم Yes 2 ال ) في حال ال انتقل الى 00( (44 to skip no if (No هل معدل تغيير الموظفين في المرفق الصحي عالي؟ HNW42 If ‘Yes’, what are the reasons? إذا "نعم"، ما هي األسباب؟ 66 HNW43 How can it be improved? كيف يمكن تحسينه )استقرار الموظف(؟ HNW44 Is staffing sufficient in numbers and competencies for the functions that must be performed at the health facility? Yes (if yes skip to 46) )04 الى انتقل نعم حال في ) نعم 1 هل الموظفين من حيث العدد و الكفاءة كافيين للمهام التي يجب أن تنفذ في المنشأة الصحية؟ 2 ال No HNW45 If ‘No’, what are the gaps? إذا "ال"، ما هي النواقص؟ HNW46 Did you receive any training in the last 12 months through the project? Yes نعم 1 هل حصلت انت او احد الطاقم على أي تدريب في الـ 15شهر الماضي ة من خالل المشروع؟ 2 ال ) في حال ال انتقل الى02( (49 to skip no if (No HNW47 If ‘Yes, which trainings did you get in the last 12 months? List the trainings. إذا "نعم"، ما هي التدريبات التي حصلت عليها في الـ 15شهر الماضية؟ أورد الدورات التدريبة. HNW48 Is the training topic relevant to the work that you do in the health facility? Yes نعم 1 هل موضوع التدريب كانت له عالقة بالعمل الذي تقوم به في المرفق الصحي؟ 2 ال No HNW49 Do you agree that job profile of health care providers is considered for the selection of trainees for specific training programs? Yes (if yes skip to 51) )21 الى انتقل) نعم 1 ُخذ بعين االعتبار المهام الوظيفية للعاملين في الرعاية الصحية 2 ال No هل أ 67 عند اختيار المتدربين لبرامج تدريبي ة محددة؟ HNW50 If ‘No’, what are the criteria for selection? إذا "ال"، ما هي معاييرلالختيار؟ Drugs and medical supplies األدوية و المستلزمات الطبية HNW51 Are the medical supplies (e.g., medical kits, equipment, consumables) distributed appropriate to the need of the health facility? Yes ( if yes skip to 53) )28الى انتقل) نعم 1 هل المستلزمات الطبية )على سبيل المثال الحقائب الطبية، المعدات و المواداالستهالكية( التي وزعت مناسبةالحتياج المرفق الصحي؟ 2 ال No HNW52 If ‘No’, what are the gaps? إذا "ال"، ما هي النواقص؟ Gaps and Challenges الفجوات و التحديات HNW53 Considering the total services offered by the health facility, do you believe that there are gaps in certain types of services that need improvement? Yes نعم 1 بالنظر إلى مجموع الخدمات المقدم ة من قبل المرفق الصحي، هل تعتقد أن هناك نواقص في أنواع محددة من الخدمات و التي تحتاج إلى تحسين؟ 2 ال ) في حال ال انتقل الى 24( (56 to skip no If (No HNW54 If ‘Yes’, which services? إذا "نعم"، أي خدمات؟ HNW55 How can it be improved? 68 كيف يمكن تحسينها؟ HNW56 What are the challenges in implementing health and nutrition programs? ما هي التحديات التي تواج هها في تنفيذ برامج الصحة و التغذية؟ HNW57 How are you addressing them? كيف تقوم بالتعامل معها؟ HNW58 Comments: الحظات Monitor general comments in this interview الحظات عامة للمقابلة Yemen OFDA/FFP Monitoring & Evaluation Project (YOFMEP) - Formative Evaluation, March 2014 HEALTH & NUTRITION - BENEFICIARY SURVEY QUESTIONNAIRE األ سره رب مقاب لة - وال تغذي ة ال صحة 69 HNH1 Project Implementer: IMC / SCI SCI / IMC : المشىروع منقذ HNH2 Project Name: Health and Nutrition والتغذية الصحة :المشىروع اسم HNH3 Governorate: المحافظه HNH4 District: المديريه HNH5 Sub-district العزله HNH6 City/Village: القريه المرفق الصحي )االسم( :Cente Health 7 HMH المعلومات الديمغرافية و االجتماعية Socio Demographic HNH8 Respondent gender 1 ذكر Male جنس المبحوث 2 أنثى Female HNH9 Level of literacy 1 امي None مستوى معرفة القراءة و الكتابة Basic reading & writing ويكتب يقرأ 2 3 اساسي Primary 4 ثانوي Secondary 5 أخرى Other أ خرى ) حدد( :(specify (Other 9.01HNH Don't know/ Refuse رفض /اعلم ال 888 HNH10 Respondents age العمر عمر المجيب HNH11 Number of people in the household عدد ألفراد في ألسرة HNH12 How is this number made up? إلناث )أكبر من 13سنة( years 18than greater Female Adult حدد الفئات العمرية لهذا العدد 12.01HNH 70 HNH12.02 Adult Male greater than 18 years )سنة13 من أكبر )الذكور إلناث ما بين 13-2سنة Female years 18to 5 12.03HNH الذكور ما بين 13-2سنة Male years 18to 5 12.04HNH HNH12.05 Less than 5 years Female سنوات2 من اصغر الناث HNH12.06 Less than 5 years Male سنوات2 من اصغر الذكور HNH13 Does the household have a means of income? 1 الثروة الحيوانية Livestock هل ألسرة تملك وسائل للدخل؟ 13.01HNH HNH13.02 Daily labour اليومي ألجر 1 HNH13.03 Business owner عمل صاحب 1 HNH13.04 Govermental job حكومية وظيفة 1 HNH13.05 Other أخرى 1 HNH13.06 Other (specify) )حدد )أخرى؟ Don't know/ Refuse رفض /اعلم ال 888 الصحة Health HNH14 Have you ever used any of the health services of the health centre? 1 نعم Yes هل سبق لك استخدام أي من الخدمات الصحية في المركز الصحي؟ 2 ال No انتقل الى السؤال رقم 12 20HNH TO SKIP HNH15 If ‘Yes, what are the reasons that you have visited the health centre the last time? Reproductive Health إلنجابية الصحة 1 انتقل الى السؤال رقم 21 16HNH TO SKIP 2 ألمراض المعدية diseases Communicable إذا نعم، ما هي األسباب التي جعلتك تزور المركز الصحي في آخر مرة؟ انتقل الى السؤال رقم 21 17HNH TO SKIP HNH16 If it was Reproductive Health 1 رعاية ما قبل اوالدة Care Antenatal اذا كان االختيار الصحه النجابيه 16.01HNH HNH16.02 Delivery اوالدة 1 1 رعاية ما بعد اوالدة Care Postnatal 16.03HNH HNH16.04 Violence العنف 1 71 HNH17 If it was Communicable diseases 1 كنت مريض sick were You اذا كان االختيار المراض المعدية 17.01HNH HNH17.02 Child was sick مريض كان طفلي 1 1 فرد آخر من ألسرة كان مريض sick member family Other 17.03HNH HNH18 Are there gaps in the health services of the facility? 1 نعم Yes هل هناك نواقص في خدمات المرفق الصحية؟ 2 ال No انتقل الى السؤال رقم 12 20HNH TO SKIP HNH19 HNH19.01 what are the gaps? Health workers do not show up/not available متوفرين غير/يظهرون ال الصحيون العاملون 1 1 العاملون الصحيون ليسوا مؤ هلين بما يكفي enough skilled not are workers Health 19.02HNH HNH19.03 ما هي النواقص؟ ) ال تقرأ االجابات فقط اشر على االحابة المطابقة( 1 ال توجد أدوية أو مستلزمات طبية supplies medical or drugs No 1 لم يُعطى تثقيف صحي given education health No 19.04HNH HNH19.05 Other أخرى 1 HNH19.06 Other (specify) )حدد )أخرى؟ Don't know/ Refuse رفض /اعلم ال 888 HNH20 Have you attended health education sessions in the facility? 1 نعم Yes هل حضرت محاضرات التثقيف الصحي في المرفق او في اي مكان اخر؟  2 ال No ←انتقل الى السؤال رقم 12 24HNH TO SKIP HNH21 When was the last time you attended health االشهر الماضية ?education متى كانت آخر مرة حضرت في ها التثقيف الصحي؟ HNH22 Which health topic was covered in the education session? Reproductive Health إلنجابية الصحة 1 2 ألمراض المعدية diseases Communicable ما الموضوع الصحي الذي تم تناوله في دورة التثقيف؟ HNH23 Have the health education influenced your 72 practices? Yes نعم 1 هل أثر التثقيف الصحي على عاداتك؟ 2 ال No  HNH24 Have you been visited by a voluntary community health worker? 1 نعم Yes هل تمت زيارتك من قبل عامل الصحة المجتمعي المتطوع؟ 2 ال No ←انتقل الى السؤال رقم 11 26HNH TO SKIP HNH25 If ‘Yes, was the visit frequent enough? 1 نعم YES إذا نعم، هل كانت الزيارة متكررة بما فيه الكفاية؟ 2 ال NO  HNH26 Who is most influential in deciding health facility visits? Mother ألم 1 2 ألب Father 3 الحماة law-in-Mother من الذ يتخذ على ألغلب قرار تحديد الزيارات للمرفق الصحي؟ 4 الجدة Grandmother HNH27 What additional services should be provided to meet your health needs? ما هي الخدمات إلضافية التي ينبغي أن تقدم لتلبية احتياجاتك الصحية؟ التغذية Nutrition HNH28 Have you ever used any of the nutrition services of the health centre? 1 نعم Yes هل سبق لك استخدام أي من خدمات التغذية في المركز الصحي؟ 2 ال No ←انتقل الى السؤال رقم 21 32 HNH TO SKIP HNH29 If ‘Yes, what are the reasons that you have visited the health centre the last time? Counseling on Infant and Young Child الصغار ألطفال و الرضع تغذية بشأن المشورة 1 73 Feeding إذا نعم، ما هي األسباب التي دفعتك لزيارة المركز الصحي في المرة 1 الج سوء التغذية لدى ألطفال/النساء women/child malnourished of Treatment األخيرة؟ HNH30 Are there gaps in the nutrition services in the facility? 1 نعم Yes  2 ال No هل يوجد أي نواقص في خدمات التغذية في المرفق؟ ←انتقل الى السؤال رقم 21 32HNH TO SKIP HNH31 If ‘Yes’, what are the gaps?  Health workers do not show up/not available متوفرين غير/يداومون ال الصحيون العاملون 1 1 العاملون الصحيون ليسوا مؤ هلين بما يكفي enough skilled not are workers Health 1 ال يوجد إمدادات الغذاء التعويضي الجا هز الستخدام (Nut Plumpy (supplies RUTF No 1 لم يُعطى تثقيف غذائي given education nutrition No إذا "نعم"، ما هي النواقص؟ 1 اخرى Other HNH31.06 Other (specify) )حدد )أخرى؟ Don't know/ Refuse رفض /اعلم ال 888  HNH32 Have you attended nutrition education sessions in the facility? 1 نعم Yes هل حضرت محاضرات التثقيف الغذائي في المرفق او في اي مكان اخر؟ 2 ال No ←انتقل الى السؤال رقم 23 35HNH TO SKIP HNH33 If ‘Yes’, which nutrition topic was covered in the HNH33.01 education session? Breastfeeding الطبيعية الرضاعة 1 1 التغذية التكميلية feeding Complementary إذا "نعم"، ما الموضوع الغذائي الذي تم تناوله في دورة التثقيف؟ 33.02HNH HNH34 Have the nutrition education influenced your practices? 1 نعم Yes هل أثر التثقيف الغذائي على عاداتك؟ 2 ال No  HNH35 Who in the family is most influential in dictating what the child should be fed Mother ألم 1 2 ألب Father 74 ً في إمالء ما يجب أن يُغذى به الطفل؟ 3 الحماة law-in-Mother من هو في األسرة األكثر تأثيرا 4 الجدة Grandmother HNH36 Have you been visited by a voluntary community health worker? 1 نعم Yes هل تمت زيارتك من قبل عامل الصحة المجتمعي المتطوع؟ 2 ال No ←انتقل الى السؤال رقم 23 39 HNH TO SKIP HNH37 Was the visit frequent enough? 1 نعم Yes هل كانت الزيارة متكررة بما فيه الكفاية؟ 2 ال No  HNH38 Did the voluntary community health worker talk to anybody else in the family other than you? 1 نعم Yes هل عمال صحة المجتمع المتطوعين تحدثوا مع أي فرد آخر من ألسرة 2 ال No سواك؟  HNH39 What additional nutrition services should be provided to meet the needs of the population? ما هي خدمات التغذية إلضافية التي يجب تقديم ها لتلبي احتياجات السكان؟ HNH40 Comments: الحظات Monitor general comments in this interview الحظات عا مة للمقابلة 75 Yemen OFDA/FFP Monitoring & Evaluation Project (YOFMEP) - Formative Evaluation, March 2014 FOOD DISTRIBUTION - BENEFICIARY SURVEY QUESTIONNAIRE المستفيد استبيان – األغذية توزيع FD 1 Governorate : :المحافظة FD 2 District : :المديرية FD 3 village: :القريه FD 4 Implementing Partner : WFP WFP :المنفذة المنظمة FD 5 Location of Food Distribution Point (Name): الغذائية المواد توزيع نقطة موقع )االسم(: FD 6 Interviewer’s name: :الباحث اسم Socio Demographic المعلومات الديمغرافية و االجتماعية FD7 Sex of the respondent 1 أنثى Female جنس المبحوث 2 ذكر Male FD8 Respondent age العمر Age عمر المبحوث Don't know/ Refuse رفض /اعلم ال 888 FD9 Number of people in the household عدد األفراد في األسرة FD10 How is this number made up? حدد الفئات العمرية لهذا العدد FD10.01 Female greater than 18 years )سنة81 من أكبر )إلناث 1 76 FD10.02 Male greater than 18 years )سنة81 من أكبر )الذكور 1 1 إلناث ما بين 81-5سنة Female years 18to 5 10.03FD 1 الذكور ما بين 81-5سنة Male years 18 to 5 10.04FD FD10.05 Less than 5 years Female سنوات5 من أقل إلناث 1 FD10.06 Less than 5 years Male سنوات5 من أقل الذكور 1 FD11 1 الثروة الحيوانية Livestock 11.01FD FD11.02 Does the household have a means of income? Daily labour اليومي الجر 1 1 صاحب العمل owner Business هل األسرة تملك وسائل للدخل؟ 11.03FD FD11.04 Govermental job حكومية وظيفة 1 FD11.05 Others (specify) )حدد )أخرى؟ 1 2.0 Beneficiary selection criteria .2.2معايير اختيار المستفيدين FD12 Are you receiving food parcel through World Food Programme? Yes نعم 1 Section beneficiary-non the to go ,’No ‘If : No هل استلمت المواد الغذائية م خالل برنامج الغذاء العالمي؟ (Section 4) ال : إذا "ال"،اذ هب إلى قسم الغير مستفيدين )قسم رقم 4( 2 FD13 Which category of beneficiaries do you belong to? Internally Displaced People (IDP) ً 1 النازحون داخليا 2 برنامج شبكة ألمان و الطوارئ (ESN (Net Safety Emergency إلى أي فئة من المستفيدين تنتمي؟ FD14 Do you know why you were selected? 1 نعم Yes 2 ال )في حال كانت الجاب ال انتقل الى 81( (17 to skip no If ( No هل تعرف لماذا تم اختيارك؟ FD15 If ‘Yes’, what is the criteria that applied for your selection? إذا "نعم"، ما هي المعايير تم تطبيقها في اختيارك؟ FD16 Are you aware if WFP or implementing partner verified 77 beneficiaries’ conformity to selection criteria? Yes نعم 1 هل أنت على علم إذا برنامج الغذاء العالمي أو المنظمة المنفذة تحققوا من مطابقة المستفيدين لمعايير الختيار؟ 2 ال No FD17 Was the beneficiary selection transparent? 1 نعم ) في حال كانت نعم انتقل الى 81( (19 to skip Yes If ( Yes 2 ال No هل كانت عملية اختيار المستفيدين شفافة؟ FD18 If ‘No’, please provide specific examples? إذا "ال"، يُرجى إعطاء أمثلة محددة. FD19 Was the beneficiary selection fair? 1 نعم ) في حال كانت نعم انتقل الى 18( (21 to skip Yes If ( Yes 2 ال No هل كانت عملية اختيار المستفيدين عادلة؟ FD20 If ‘No’, please provide specific examples? إذا "ال"، يُرجى إعطاء أمثلة محددة. FD21 Are appropriate grievance/complaint structures or channels in place for cases of abuse or misuse of food distribution? Yes نعم 1 هل هناك قنوات مناسبة للتظلم/الشكوى في حاالت العتداء أو سوء توزيع ألغذية؟ 2 ال ) في حال كانت نعم انتقل الى 12 ( (23 to skip No If ( No FD22 What is the degree of responsiveness of the organization? 1 جيدة Good 2 متوسطة Average 3 ال توجد استجابه response No ما هي درجة استجابة المنظمة؟ FD23 Do you know any people who have been recently added to the beneficiary list? Yes نعم 1 ً لقائمة المستفيدين؟ 2 ال No هل تعرف أي أشخاص تمت إضافتهم مؤخرا 78 FD24 Do you know any people who have been recently removed from the beneficiary list? Yes نعم 1 2 ال No هل تعرف أي أشخاص تمت إزالتهم من قائمة المستفيدين؟ 3.0 Food Distribution .0.2توزيع األغذية FD25 Mother الم 1 Who collects the food? Father االب 2 3 فرد اخر من السرة member family Other من الذي يقوم باستالم المواد الغذائية من مركز التوزيع؟ 4 اخرى Other FD25.01 Others (specify) )حدد )أخرى FD26 Do you know the size and composition of ration you are entitled to? Yes نعم 1 2 ال No هل تعرف حجم حصتك الغذائية الي تستحقها؟ FD27 Are you receiving the same amount regularly? 1 نعم ) في حال كانت نعم انتقل الى 11 ( (29 to skip yes if (Yes 2 ال No هل تستلم الحصة نفسها باستمرار؟ FD28 If ‘No’, what are the gaps? إذا "ال"، ما هي النواقص؟ FD29 What do you do with the food? Consume it ( skip to 31) تستهلكها ) في حال اختار هذه الجابة انتقل الى )28 1 All Sold out تبيعها 2 3 تبيع جزء منها sold Partially ماذا تفعل بألغذية؟ FD30 If you sell the food, why do you sell it? Don’t like the composition مكوناتها تعجبني ال 1 Want to buy a non-food item غذائية غير مواد شراء أريد 2 79 3 أخرى؟ )حدد( في حال انك تقوم ببيع المواد أو جزء منها فلماذا تقوم بذبك؟ 30.01FD FD31 Are you satisfied with the food distribution service? 1 نعم )في حال كانت نعم انتقل الى 22 ( (33 to skip yes if ( Yes 2 ال No هل أنت راض عن خدمة توزيع الغذاء؟ FD32 If ‘No’, why you are not satisfied? إذا "ال" ٍ ، لماذا لست راض؟ FD33 The food distributor was very courteous? 1 نعم Yes 2 ال No هل اسلوب موزع الغذاء جيد؟ FD34 The waiting time for getting the food ration was reasonable. Yes نعم 1 2 ال No هل كان وقت االنتظار للحصول على الحصة الغذائية مناسب؟ FD35 Are you satisfied with the quality of food received? 1 نعم )في حال كانت االجابة نعم انتقل الى 21( (37 to skip yes if ( Yes 2 ال No هل أنت راض عن جودة المواد المستلمه؟ FD36 If ‘No’, what are the reasons? إذا "ال"، ما هي أسبابك؟ FD37 Will you come to the food distribution point again? 1 نعم Yes 2 ال No هل سوف تأتي إلى نقطة توزيع الغذاء مرة أخرى؟ 4.0 Non-Beneficiary Survey .4.2 استبيان غير المستفيدين 80 FD38 Do you know about beneficiary selection criteria? 1 نعم Yes 2 ال No هل تعرف معايير اختيار المستفيدين؟ FD39 In your opinion, was beneficiary selection transparent? 1 نعم )في حال كانت نعم انتقل الى 48 ( (41 to skip yes if ( Yes 2 ال No في رأيك، هل كانت عملية اختيار المستفيدين شفافة؟ FD40 If ‘No’, please provide specific examples? إذا "ال"، يُرجى إعطاء أمثلة محددة. FD41 In your opinion, was beneficiary selection fair? 1 نعم )افي حال كانت نعم انتقل الى 42 ( (43 to skip yes if (Yes 2 ال No في رأيك، هل كانت عملية اختيار المستفيدين عادلة؟ FD42 If ‘No’, please provide specific examples? إذا "ال"، يُرجى إعطاء أمثلة محددة. FD43 Are appropriate grievance/complaint structures or channels in place for cases of abuse or misuse of food distribution? Yes نعم 1 هل هناك قنوات مناسبة للتظلم/الشكوى في حاالت العتداء أو سوء توزيع ألغذية؟ 2 ال )في حال كانت نعم انتقل الى 45 ( (45 to skip no if ( No FD44 What is the degree of responsiveness of the organization? 1 تستهلكها Good 2 تبيعها Average 3 تبيع جزء منها response No ما هي درجة استجابة المنظمة؟ FD45 Are you aware of any beneficiary verification process going on? Yes نعم 1 ً؟ 2 ال No هل أنت على علم بأي عملية تحقق من المستفيد تحدث حاليا 81 FD46 Do you know any people who have been recently added to the beneficiary list? Yes نعم 1 ً لقائمة المستفيدين؟ 2 ال No هل تعرف أي أشخاص تمت إضافتهم مؤخرا FD47 Do you know any people who have been recently removed from the beneficiary list? Yes نعم 1 2 ال No هل تعرف أي أشخاص تمت إزالتهم من قائمة المستفيدين؟ Yemen OFDA/FFP Monitoring & Evaluation Project (YOFMEP) - Formative Evaluation, March 2014 مقاب لة - األل غام مخاطر من ال توعية Interview Children - Education Risk Mine األط فال MREC1 Project Implementer: 1- Unicef 2. Women Union 3.YEMAC المشىروع منقذ MREC2 Project Name: المشىروع اسم MREC3 Governorate: المحافظه MREC4 District: المديريه MREC5 Sub-district العزله MREC6 City/Village: القريه MREC7 Interviewer Name: المبحوث اسم MREC8 Interview Loction: المفابلة مكان و ال دي مغراف ية ال معلومات Demographic Socio 82 االج تماعية MREC9 Sex of the respondent 2 انثى Female جنس المجيب 1 ذكر Male MREC10 Respondents age بين 15-4سنوات years 6-12 عمر المجيب بين 15الى 12سنه years 12-15 أكبر من 12سنه years 15 Above General Information معلومات عامة MREC11 Which chores do you assist with at home? 1 الرعي Herding ما هي ألعمال التي تقوم بها لمساعدة ألسرة؟ 1 العمل في الحقل work Farm 1 جمع المياه water Collecting MREC11.01 Other (specify) )حدد )أخرى 1 MREC12 At home, at what locations do you and your friends usually play? ما هي األ ً ماكن التي تلعب فيها عادة أنت و 1 في الحقول المفتوحة fields open the In أصدقاؤك؟ 2 في مناطق مخصصة للعب areas play designated In Don’t play out side الخارج في تلعب ال 3 4 اخرى Other MREC12.01 Other (specify) )حدد ) اخرى أسئلة حول جودة البرنامج to respond that Question program quality MREC13 What do you do if you see a foreign object laying on the ground when Pick it up بيدي ألتقطه 1 83 you are working or play outside? ماذا تفعل لو رأيت جسم غريب مُلقى على ألرض أثناء عملك أو لعبك في الخارج؟ Kick it out of the way الطريق عن ً 2 أركله بعيدا Avoid contact with all unknown objects الغريبة ألجسام مع االتصال أتجنب 3 Don’t know أعرف ال 888 MREC14 Do you know what mines and explosive remnants of war (ERW) are? Yes نعم 1 هل تعرف ما هي مخلفات الحرب من األلغام و المتفجرات؟ )إذا "ال" إنهاء الستبيان و شكر المشارك( No ( if no END the interview) )المقابله انهي حالال في ) ال 2 MREC15 If yes, where did you receive the information? At the school المدرسة في 1 2 شخص أتى إلى منزلي house my to come Someone في حال نعم، أين تلقيت المعلومات؟ 3 اخرى Other MREC15.01 Other (specify) )حدد )اخرى MREC16 Who delivered the information to you? من الذي نقل إليك المعلومات؟ A teacher المعلم 1 2 مدير المدرسة أو مديرة المدرسة Headmistress or Headmaster The 3 اخرى Other MREC16.01 Other (specify) )حدد )اخرى MREC17 Was the information useful ? 1 نعم Yes هل كانت المعلومات مفيدة؟ 2 ال ) في حا ال قم بإنهاء المقابله( (interview the finish no if ( No MREC18 How was the information useful? Please explain? كيف وجدت ان المعلومات كانت مفيدة؟ يُرجى التوضيح MREC19 Enumerator Comments Comments: الحظات 84 الحظات عامة للمقابلة Yemen OFDA/FFP Monitoring & Evaluation Project (YOFMEP) - Formative Evaluation, March 2014 - األل غام مخاطر من ال توعية Interview Children - Education Risk Mine األط فال مقاب لة MREC1 Project Implementer: 1- Unicef 2. Women Union 3.YEMAC المشىروع منقذ MREC2 Project Name: المشىروع اسم MREC3 Governorate: المحافظه MREC4 District: المديريه MREC5 Sub-district العزله MREC6 City/Village: القريه MREC7 Interviewer Name: المبحوث اسم MREC8 Interview Loction: المفابلة مكان و ال دي مغراف ية ال معلومات Demographic Socio االج تماعية MREC9 Sex of the respondent 2 انثى Female جنس المجيب 85 1 ذكر Male MREC10 Respondents age بين 15-4سنوات years 6-12 عمر المجيب بين 15الى 12سنه years 12-15 أكبر من 12سنه years 15 Above General Information معلومات عامة MREC11 Which chores do you assist with at home? ما هي ألعمال التي تقوم بها لمساعدة Herding ألسرة؟ الرعي 1 1 العمل في الحقل work Farm 1 جمع المياه water Collecting MREC11.01 Other (specify) )حدد )أخرى 1 MREC12 At home, at what locations do you and your friends usually play? ما هي األ ً ماكن التي تلعب فيها عادة أنت 1 في الحقول المفتوحة fields open the In و أصدقاؤك؟ 2 في مناطق مخصصة للعب areas play designated In Don’t play out side الخارج في تلعب ال 3 4 اخرى Other MREC12.01 Other (specify) )حدد ) اخرى أسئلة حول جودة البرنامج respond that Question to program quality MREC13 What do you do if you see a foreign object laying on the ground when you are working or play outside? Pick it up بيدي ألتقطه 1 ماذا تفعل لو رأيت جسم غريب مُلقى على ألرض أثناء عملكأو لعبك في Kick it out of the way الطريق عن ً 2 أركله بعيدا Avoid contact with all unknown objects الغريبة ألجسام مع االتصال أتجنب 3 86 888 ال أعرف know t’Don الخارج؟ MREC14 Do you know what mines and explosive remnants of war (ERW) are? Yes نعم 1 هل تعرف ما هي مخلفات الحرب من األلغام و المتفجرات؟ )إذا "ال" إنهاء الستبيان و شكر المشارك( No ( if no END the interview) )المقابله انهي حالال في ) ال 2 MREC15 If yes, where did you receive the information? At the school المدرسة في 1 2 شخص أتى إلى منزلي house my to come Someone في حال نعم، أين تلقيت المعلومات؟ 3 اخرى Other MREC15.01 Other (specify) )حدد )اخرى MREC16 Who delivered the information to you? من الذي نقل إليك المعلومات؟ A teacher المعلم 1 2 مدير المدرسة أو مديرة المدرسة Headmistress or Headmaster The 3 اخرى Other MREC16.01 Other (specify) )حدد )اخرى MREC17 Was the information useful ? 1 نعم Yes هل كانت المعلومات مفيدة؟ 2 ال ) في حا ال قم بإنهاء المقابله( (interview the finish no if ( No MREC18 How was the information useful? Please explain? كيف وجدت ان المعلومات كانت مفيدة؟ يُرجى التوضيح MREC19 Enumerator Comments Comments: الحظات الحظات عامة للمقابلة 87 Yemen OFDA/FFP Monitoring & Evaluation Project (YOFMEP) - Formative Evaluation, March 2014 ال صحي وال صرف ال مياه QUESTIONNAIRE SURVEY HYGIENE AND SANITATION ,WATER وال نظاف ة WASH1 Project Implementer: IMC,SCF, CSSW, ةسسؤم ريخلا المشىروع منفذ WASH2 Project Name: المشىروع اسم WASH3 Governorate: المحافظه WASH4 District: المديريه WASH5 Sub-district العزله WASH6 City/Village: القريه WASH7 Location of Activity: النشاط موقع WASH8 Name of interviewer المبحوث اسم المعلومات الديمغرافية و االجتماعية Socio Demographic WASH9 Respondent gender 1 ذكر Male جنس المبحوث 2 أنثى Female WASH10 Level of literacy 1 امي None مستوى معرفة القراءة و الكتابة Basic reading & writing ويكتب يقرأ 2 3 اساسي Primary 4 ثانوي Secondary 5 أخرى Other أ خرى ) حدد( :(specify (Other 10.01WASH 88 Don't know/ Refuse رفض /اعلم ال 888 WASH11 Respondent age العمر age Respondent عمر المجيب WASH12 Number of people in the household عدد ألفراد في ألسرة Number of people ألفراد عدد WASH13 How is this number made up? إلناث )أكبر من 13سنة( years 18than greater Female Adult حدد الفئات العمرية لهذا العدد 13.01WASH WASH13.02 Adult Male greater than 18 years )سنة13 من أكبر )الذكور إلناث ما بين 13-2سنة Female years 18to 5 13.03WASH الذكور ما بين 13-2سنة Male years 18to 5 13.04WASH WASH13.05 Less than 5 years Female سنوات2 من اصغر الناث WASH13.06 Less than 5 years Male سنوات2 من اصغر الذكور WASH14 Does the household have a means of income? 1 الماشية Livestock هل ألسرة تملك وسائل للدخل؟ 14.01WASH WASH14.02 Daily labour اليومي ألجر 1 WASH14.03 Business owner عمل صاحب 1 WASH14.04 Government Official حكومية وظيفة 1 WASH14.05 Other أخرى 1 WASH14.06 Other (specify) )حدد )أخرى؟ Don't know/ Refuse رفض /اعلم ال 888 المياه Water WASH15 What is your primary source of drinking water? (this refers to the main water point/location used for water collection) Bladder/tank خزان 1 Reservoir/Well بئر 2 3 تجويف مائي )كريف( Water hole Bore ما هو المصدر األساسي لمياه الشرب؟ 89 4 شبكة المياه موصلة الى داخل المنزل house the inside tab Water ( هذا يشير إلى نقطة المياه الرئيسية/الموقع المستخدم لجمع المياه( 5 أخرى Other WASH15.01 Other (specify) )حدد )أخرى؟ Don't know/ Refuse رفض /اعلم ال 888  WASH16 How far away is this primary source of drinking water from this house? Less than 100m away متر 144 من أقل 1 100m to 500m away متر 244-144 بين 2 3 أكثر من 244 متر away m500 than More كم يبعد مصدر مياه الشرب الرئيسي من المنزل؟ 4 أخرى Other WASH16.01 Other (specify) )حدد )أخرى؟  WASH17 How long do you have to wait to collect water from your primary source? Less than 15 minutes دقيقة 12 من أقل 1 2 بين 84-12دقيقة minutes 30 - 15 3 أكثر من 84 دقيقة minutes 30 than More كم من الوقت يجب عليك النتظار لجمع المياه من مصدرك الرئيسي؟ 4 أخرى Other WASH17.01 Other (specify) )حدد )أخرى؟ WASH18 If more than 30 minutes, why? Reasons? 1 كثير من الناس ينتظرون waiting people many Too 2 انتظار الشاحنات لتمأل الخزانات tanks the fill to trucks the for Waiting إذا كان أكثر من 84 دقيقة، ما هي األسباب؟ The flow of water is too slow from the source ً 3 تدفق المياه من المصدر بطيء جدا WASH18.01 Other أخرى 4 أخرى؟ )حدد( (specify (Other Don't know/ Refuse رفض /اعلم ال 888 WASH19 Is the water collected sufficient for your needs? 1 نعم Yes 21 TO SKIP هل المياه التي يتم جمع ها كافيةالحتياجاتك؟ 2 ال No  WASH20 If no, then why is this? Not enough water كافية غير المياه مصادر 1 The water is not always available ً 2 المياه ليست متوفرة دائما 90 لماذا؟ ،ال إذا I am not able to collect more المزيد جمع على ً 3 أنا لست قادرا 4 أخرى Other WASH20.01 Other (specify) )حدد )أخرى؟ Don't know/ Refuse رفض /اعلم ال 888 WASH21 Do you encounter any conflicts at the water collection point? Yes نعم 1  2 ال No هل واجهت أي نزاعات في نقطة تجميع المياه؟ SKIP TO 23 WASH22 If yes, what type of conflicts? 1 إنسان و إنسان Human and Human 2 إنسان و ماشية livestock and Human 3 إنسان و حيوانات مفترسة wildlife and Human 4 أخرى Other إذا نعم، أي نوع من النزاعات؟ WASH22.01 Other (specify) )حدد )أخرى؟ Don't know/ Refuse رفض /اعلم ال 888 WASH23 Do you encounter any risks at the water collection point? 1 نعم Yes هل واجهت أي مخاطر في نقطة تجميع المياه؟ 2 ال No SKIP TO 25 WASH24 If yes, can you tell me what these risks might be? (DO NoT PROMPT these responses. Encourage responses by asking” are there any others??”. Tick any risks mentioned) WASH24.01 Bullying البلطجة 1 WASH24.02 Physical harassment الجسدية المضايقات 1 1 انعدام األمن Insecurity 24.03WASH WASH24.04 Animals الحيوانات 1 إذا نعم، هل تستطيع أن تخبرني ماهي هذه المخاطر؟ 24.05WASH )ال تقرأ هذه اإلجابات. شجع المجيب عن طريق سؤاله "هل يوجد أخرى؟". أشر إلى أي مخاطر مذكورة(. 1 أخرى Other WASH24.06 Other (specify) )حدد )أخرى؟ Don't know/ Refuse رفض /اعلم ال 888  WASH25 Are there any problems / challenges with your regular water source? 1 نعم Yes 91 هل هناك أي مشاكل/تحديات في تأمين احتياجك من مصادرالمياه بشكل 2 ال No مستمر؟ SKIP TO 27 WASH26 If so, can you name some? WASH26.01 Distance too far ً 1 المسافة بعيدة جدا WASH26.02 Water polluted/dirty قذرة/ملوثة المياه 1 1 إمدادات المياه غير كافية insufficient supply Water 26.03WASH WASH26.04 Other أخرى 1 أخرى؟ )حدد( (specify (Other إذا كانت اإلجابة "نعم"، فما هي هذه التحديات؟ 26.05WASH Don't know/ Refuse رفض /اعلم ال 888 WASH27 Who owns water sources/points around this area? 1 المجتمع Community 2 ملك خاص owner Private 3 الحكو مة Government من الذي يملك مصدر المياه في المنطقة؟ WASH28 Do you pay for water? Yes – always ً 1 دائما 2 نعم – أثناء موجة الجفاف spell dry during – Yes هل تدفع مقابل الحصول على المياه؟ 3 ال No SKIP TO 30 4 أخرى Other أخرى؟ )حدد( (specify (Other  WASH29 If you pay, how much do you pay for a container? WASH29.01 Money مال حجم الحاويات )ليتر( ltr container of Volume إذا كانت إجابتك "نعم"، كم تدفع مقابل الحاوية؟ 29.02WASH WASH29.03 Other (specify) أخرى WASH30 Is there any water source management committee for your community? Yes نعم 1  2 ال No هل يوجد أي لجنة إلدارة مصدر المياه في مجتمعك؟ SKIP TO 32 Don't know/ Refuse رفض /اعلم ال 888 92 WASH31 If yes, how would you rate their performance in managing/maintaining the water sources?  Very good ً 1 جيد جدا 2 جيد Good 3 ضعيف Poor إذا نعم، كيف تقيم أداؤها في إدارة/الحفاظ على مصادر المياه؟  مشاركة المجتمع Participation Community WASH32 Have you ever participated in community hygiene promotion campaigns activities? 1 نعم Yes هل سبق لك او احد افراد السرة المشاركة في أنشطة حمالت الترويج 2 ال No للنظافة في المجتمع؟ SKIP TO 35 WASH33 If yes, what activity? 1 تدريبات على النظافة trainings Hygiene 33.01WASH WASH33.02 Hygiene promotion activities للنظافة الترويج أنشطة 1 1 توزيع حقيبة مستلزمات النظافة distribution kits Hygiene إذا نعم، ما هوالنشاط؟ 33.03WASH WASH33.04 Other أخرى 1 WASH33.05 Other (specify) )حدد )أخرى؟ Don't know/ Refuse رفض /اعلم ال 888 WASH34 How were you contacted? 1 بواسطة متطوعين المجتمع volunteers community By 2 بواسطة موظفين المنظ مة staffs organisational By كيف تم الوصول إليك؟ 3 السلطات المحلية authorities Local 4 أخرى Other WASH34.01 Other (specify) )حدد )أخرى؟ Don't know/ Refuse رفض /اعلم ال 888  WASH35 If no, why didn’t you? I have not been contacted بي التصال يتم لم 1 2 ال يوجد نشاط ترويج للنظافة activity promotion hygiene No إذ ال, لماذا لم تشارك؟ 3 أنا رفضت refused I 4 أخرى Other WASH35.01 Other (specify) )حدد )أخرى؟ 93 Don't know/ Refuse رفض /اعلم ال 888 أنشطة تعزيز النظافة activities promotion Hygiene WASH36 What hygiene promotion messages did you acquire? (DO NoT PROMPT these answers. Allow interviewee to respond and encourage further answers by saying; “can you tell me any?”) WASH36.01 Hand Washing اليدين غسل 1 1 التعامل معسال مة المياه handling safety Water 36.02WASH WASH36.03 Latrine maintenance المراحيض صيانة 1 ما هي رسائل تعزيز النظافة التي حصلت عليها؟ 36.04WASH )ال تقرأ هذه اإلجابات. شجعه عن طريق سؤاله " هل بإمكانك أن تخبرني ما هي؟"( 1 الحفاظ على المواد الغذائية نظيفة clean food Keep WASH36.05 Other أخرى 1 WASH36.05_1 Other (specify) )حدد )أخرى؟ Don't know/ Refuse رفض /اعلم ال 888  WASH37 How did you access these information/messages? WASH37.01 Use of posters/brochures الكتيبات أو الملصقات استخدام 1 1 زيارات من منزل إلى منزل visits house to House ما هي الوسائل التي استخدمت لتوصيل الرسائل؟ 37.02WASH 1 ح الت التوعية العا مة campaigns awareness HP 37.03WASH WASH37.04 Other أخرى 1 WASH37.05 Other (specify) )حدد )أخرى؟ Don't know/ Refuse رفض /اعلم ال 888  WASH38 Was the hygiene information beneficial to you/household? 1 نعم Yes هل كانت المعلومات الصحية مفيدة لك/ألسرتك؟ 2 ال No SKIP TO 40 WASH39 If yes, how beneficial was the hygiene awareness to WASH39.01 you/household? Change hygiene behaviors الصحية السلوكيات تغيرت 1 WASH39.02 Improved hygiene/health of household ألسرة صحة/نظافة تحسنت 1 1 غير مفيد beneficial Not إذا "نعم"، كيف؟ 39.03WASH WASH39.04 Other أخرى 1 WASH39.05 Other (specify) )حدد )أخرى؟ Don't know/ Refuse رفض /اعلم ال 888 94 WASH40 Do you ever drink treated water or treat water to make it safe for drinking? 1 نعم Yes هل قمت بمعالجة مياه الشرب لجعل ها آمنة للشرب؟ 2 ال No SKIP TO 42 WASH41 If yes; which method do you use? WASH41.01 Boiling الغلي 1 WASH41.02 Use chemical treatments (aqua tabs, PUR, Waterguard) استخدام ال الجات الكيميائية ) ,tabs aqua )الكلور,PUR, Water guard 1 1 التقطير Filtration إذا نعم، أي طريقة تستخدم؟ 41.03WASH WASH41.04 Other أخرى 1 WASH41.05 Other (specify) )حدد )أخرى؟ Don't know/ Refuse رفض /اعلم ال 888 الصرف الصحي Sanitation WASH42 Does your household use latrine? 1 نعم Yes هل أسرتك تستخدم المراحيض؟ 2 ال No SKIP TO 45 WASH43 Do you share it with other Households 1 ال No 2 نعم, مع أسرة واحدة household other 1 with ,Yes Yes, with 2 other households أسرتين مع ,نعم 3 4 نعم, مع الث أسر households other 3 with ,Yes هل تشاركها مع آخرين؟ 5 نعم, مع أكثر من 8 أسر households 3 than more with ,Yes 6 أخرى Other WASH43.01 Other (specify) )حدد )أخرى؟ Don't know/ Refuse رفض /اعلم ال 888 WASH44 Where do you defecate? 1 بين الشجيرات Bush 44.01WASH 95 1 في أرض مفتوحة ground open أين تقضي حاجتك؟ 44.02WASH WASH44.03 Other أخرى 1 WASH44.04 Other (specify) )حدد )أخرى؟ Don't know/ Refuse رفض /اعلم ال 888 WASH45 How were you provided with the latrine? 1 شيدت من قبل منظ مة organization by Constructed 2 شيدت ها بنفسي myself by Constructed 3 شيدت من قبل المجتمع community by Constructed من جهز لك المراحيض؟ 4 الحكو مة Govt by 5 أخرى Other WASH45.01 Other (specify) )حدد )أخرى؟ Don't know/ Refuse رفض /اعلم ال 888 WASH46 Do you have any knowledge/information on proper sanitation? 1 نعم Yes هل لديك أي معرفة/معلومات عن الصرف الصحي السليم؟ 2 ال No SKIP TO WASH48 WASH47 If yes, what knowledge/information? 1 عدم قضاء الحاجة في مكان مفتوح Open in defecate to Not 1 استخدام المراحيض بشكل صحيح usage Latrine Proper 1 أخرى Other أخرى؟ )حدد( (specify (Other إذا نعم، ما هي المعلومات؟ 47.01WASH Don't know/ Refuse رفض /اعلم ال 888 WASH48 Have you ever participated in any Sanitation campaigns/activities in your community 1 نعم Yes هل سبق و أن شاركت في ح الت للصرف الصحي/أنشطة في مجتمعك؟ 2 ال No SKIP TO 50 WASH49 Who organized the campaigns? 1 منظ مة Organization 49.01WASH WASH49.02 Local authority المحلية السلطة 1 WASH49.03 Community members المجتمع أفراد 1 96 WASH49.04 Other أخرى 1 أخرى؟ )حدد( (specify (Other من الذي نظم هذه الح الت؟ 49.05WASH Don't know/ Refuse رفض /اعلم ال 888 WASH50 How were you contacted? 1 بواسطة متطوعين المجتمع volunteers community By 1 بواسطة موظفين المنظ مة staffs organization By 1 السلطات المحلية authorities local By كيف تم الوصول إليك؟ 1 أخرى Other WASH50.01 Other (specify) )حدد )أخرى؟ Don't know/ Refuse رفض /اعلم ال 888 WASH51 How beneficial were the campaigns to WASH51.01 you/household? Change sanitation behavior of household members ألسرة الصحيألفراد السلوك تغير 1 WASH51.02 Made household have latrines مراحيض تمتلك األسرة جعل 1 1 غير مفيدة beneficial Not كيف كانت الحمالت مفيدة لك/ألسرتك؟ 51.03WASH WASH51.04 Other 1 WASH51.05 Other (specify) )حدد )أخرى؟ Don't know/ Refuse رفض /اعلم ال 888 WASH52 Comments: الحظات Interviewer general comments in this interview الحظات عا مة للمقابلة 97 ANNEX VIII: BIBLIOGRAPHY OF DOCUMENTS REVIEWED GENERAL DOCUMENTS Baal, S. and Holladay, S. (2012). Joint IDP Profiling Service (JIPS). Yemen Profile at a Glance 2010.” United Nations Office for the Coordination of Humanitarian Affairs (OCHA). (2013). “Humanitarian Bulletin Yemen.” Issue 19, September 5 – October 5, 2013. United Nations Office for the Coordination of Humanitarian Affairs (OCHA). (2013). “Humanitarian Bulletin Yemen.” Issue 20, October 6 – November 5, 2013. United Nations Office for the Coordination of Humanitarian Affairs (OCHA). (2013). “Humanitarian Bulletin Yemen.” Issue 21, November 6 – December 4, 2013. United Nations Office for the Coordination of Humanitarian Affairs (OCHA). (2014). “Humanitarian Bulletin Yemen.” Issue 22, December 5, 2013 – January 7, 2014. United Nations Office for the Coordination of Humanitarian Affairs (OCHA). (2014). “Humanitarian Bulletin Yemen.” Issue 23, January 8 – February 7, 2014. United Nations Office for the Coordination of Humanitarian Affairs (OCHA). (2014). “Humanitarian Bulletin Yemen.” Issue 24, February 8 – March 5, 2014. United Nations Office for the Coordination of Humanitarian Affairs (OCHA). (2014). “Humanitarian Bulletin Yemen.” Issue 25, March 6 – April 4, 2014. United Nations Office for the Coordination of Humanitarian Affairs (OCHA). (2013). “Humanitarian Response Plan 2013.” United Nations Office for the Coordination of Humanitarian Affairs (OCHA). (2014). “Strategic Response Plan 2014: Yemen Humanitarian Response Plan.” United Nations Office for the Coordination of Humanitarian Affairs (OCHA). (2014). “Yemen Humanitarian Needs Overview 2014.” USAID. (2012). “ADS Chapter 308 Awards to Public International Organizations.” OFDA/FFP DOCUMENTS USAID. (2013). “FFP Annual Program Statement for International Food Assistance.” May 6, 2013. USAID. (2013). “FFP Annual Program Statement for International Food Assistance, Amendment 1.” May 13, 2013. USAID. (2014). “OFDA Complex Emergency, Fiscal Year (FY) 2014 Implementation Plan for Yemen.” USAID. (2012). “OFDA Guidelines for Proposals.” 98 OFDA IMPLEMENTING PARTNER DOCUMENTS Adventist Development and Relief Agency (ADRA), Food Assistance, Resilience Achieved (FARA) ADRA (2013). “Abyan and Lahj Food Assistance Project Baseline Survey Draft Report.” ADRA. (2013). “FARA July 2013 – June 2016 Indicator Performance Tracking Table.” ADRA. (2013). “FARA Results Framework.” ADRA. (2013). “FARA Technical Proposal.” ADRA. (2013). “YALLA FARA Detailed Implementation Plan: September – November 2013.” Global Communities, Emergency Assistance to Support Yemeni Communities (EASE) – Phase 2 Global Communities. (2013). “EASE Quarterly Report.” July 1 – September 30, 2013. Global Communities. (2013). “EASE – Phase 2 Yemen Modification Technical Proposal.” Global Communities. “EASE Target Areas” International Medical Corps (IMC), Emergency Assistance to Vulnerable Populations in Yemen (EAVP-Y) IMC. (2013). “EAVP-Y 2013 Work Plan.” IMC. (2013). “EAVP-Y Quarterly Report.” April – June 2013. IMC. (2013). “EAVP-Y Quarterly Report.” January – March 2013. IMC. (2013). “EAVP-Y Quarterly Report.” July – September 2013. IMC. (2013). “EAVP-Y Quarterly Report Version 2.” July – September 2013. IMC. (2013). “EAVP-Y Technical Proposal Revised Submission.” January 16, 2013. IMC. (2013). “EAVP-Y Technical Proposal Revised Submission.” January 17, 2013. IMC. (2013). “IMC Work Plan.” December 2013 – January 2014. International Organization for Migration (IOM), Enhancing the Resilience of Vulnerable Communities in Abyan Governorate through Integrated Livelihoods and WASH Programming IOM. (2013). “OFDA Resilience Indicator Performance Tracking Table.” IOM. (2013). “OFDA Resilience Location Table.” IOM (2013). “OFDA Resilience Results Matrix.” IOM (2013). “OFDA Resilience Workplan.” 99 International Relief and Development (IRD), Community Water Resilience through Sand Dam Construction, Improved Hygiene Practice in Yemen IRD. (2013). “Community Water Resilience through Sand Dam Construction, Improved Hygiene Practice in Yemen Technical Proposal.” IRD. (2013). “OFDA Quarterly Program Performance Report.” August 22 – September 30, 2013. IRD. (2013). “Photo Gallery of Interventions from July – September 2013 Quarterly Report.” IRD. (2013). “SGP – Program Management Report: Yemen-Increased Access to Potable Water and Improved Hygiene Practice in Hodeidah Governorate.” USAID. (2013). “Agreement No. AID-OFDA-G-13-00144.” Save the Children, Child-Focused Health, Nutrition and WASH Emergency Response Save the Children. (2013). “Child-Focused Health, Nutrition and WASH Emergency Response Quarterly Report.” July – September 2013. Save the Children. (2013). “Child-Focused Health, Nutrition and WASH Emergency Response Proposal.” Save the Children. (2013). “Request to USAID/OFDA for a New Award.” United Nations Children’s Fund (UNICEF), WASH Emergency Project for Yemen UNICEF. (2013). “Yemen Humanitarian Response Plan 2013.” FFP IMPLEMENTING PARTNER DOCUMENTS Adventist Development and Relief Agency (ADRA), Yemen, Abyan and Lahj Livelihoods Assistance (YALLA) ADRA. (2013). “YALLA 1st Quarterly Report.” August 15 – September 30, 2013.” ADRA. (2013). “YALLA FARA Detailed Implementation Plan: September – November 2013.” ADRA. (2013). “YALLA Technical Proposal.” Global Communities (GC), Yemen Food for Asset Development (YFAD) GC. (2014). “Emergency Food Security Program – Baseline Report.” GC. (2013). “YFAD Results Framework.” GC. (2013). “YFAD Technical Proposal.” Mercy Corps, Food Security and Resilience Building (FSRB) Mercy Corps. (2013). “FSRB 1st Quarterly Report.” October – December 2013. 100 Mercy Corps. (2013). “FSRB Annual Work Plan (Year 1).” Mercy Corps. (2014). “FSRB Bi-weekly Report.” February 1 – 15, 2014. Mercy Corps. (2014). “FSRB Bi-weekly Report.” February 16 – 28, 2014. Mercy Corps. (2014). “FSRB Bi-weekly Report.” January 16 – 31, 2014. Mercy Corps. (2014). “FSRB Bi-weekly Update.” April 1 – 15, 2014. Mercy Corps. (2014). “FSRB Bi-weekly Update.” April 16 – 30, 2014. Mercy Corps. (2014). “FSRB Bi-weekly Update.” March 1 – 15, 2014. Mercy Corps. (2013). “FSRB Indicator Performance Tracking Table Final.” Mercy Clrops. (2013). “FSRB Indicator Performance Tracking Table Revised.” Mercy Corps. (2013). “FSRB Performance Monitoring Plan.” Mercy Corps. (2013). “FSRB Proposal.” Mercy Corps. (2013). “FSRB Results Framework Final.” Mercy Corps. (2013). “FSRB Results Framework Revised.” Save the Children, Emergency Food Security Program (EFSP) Save the Children. (2013). “EFSP Branding and Marking Plan.” Save the Children. (2013). “EFSP Budget.” Save the Children. (2013). “EFSP Budget Notes.” Save the Children. (2013). “EFSP IPTT.” Save the Children. (2013). “EFSP Map of Yemen.” Save the Children. (2013). “EFSP Monitoring and Evaluation Plan.” Save the Children. (2013). “EFSP Past Performance Review.” Save the Children. (2013). “EFSP Proposal Narrative.” Save the Children. (2013). “EFSP Response to Issues Letter.” Save the Children. (2013). “EFSP Results Framework.” Save the Children. (2013). “EFSP Workplan and Beneficiary Calculations.” Save the Children. (2013). “Yemen Emergency Food Vouchers Final Report.” 101 USAID. (2013). “Negotiated Indirect Cost Rate Agreement.” USAID. (2013). “Reps and Certs.” USAID. (2013). “SF424.” World Food Programme (WFP), Emergency Food and Nutrition Support to Food-Insecure and Conflict-Affected People WFP. (2013). “Emergency Operation Yemen 200451.” WFP. (2014). “Post-distribution Report – ESN.” WFP. (2014). “Post-distribution Report – IDPs.” WFP. (2014). “Projects for Executive Board Approval Agenda Item 6, Protracted Relief and Recovery Operations – Yemen 200636: Safeguarding Lives, Improving Food Security and Nutrition, and Building Resilience.” WFP. (2013). “Activity Verification Data.” WFP. (2014). “Activity Verification Report – ESN.” WFP. (2014). “Activity Verification Report – IDPs.” IBTCI YOFMEP DOCUMENTS Al-Salihi, F. (2013). “EASE YOFMEP Implementing Partner November Monitoring Status Report.” Al-Salihi, F. (2013). “Enhancing the Resilience of Vulnerable Communities in Abyan Governorate through Integrated Livelihoods and WASH Programming YOFMEP Implementing Partner November Monitoring Status Report.” Al-Salihi, F. (2013). “YFAD YOFMEP Implementing Partner November Monitoring Status Report.” Assalahy, A. (2013). “Child-Focused Health, Nutrition and WASH Emergency Response YOFMEP Implementing Partner November Monitoring Status Report.” Assalahy, A. (2013). “Community Water Resilience through Sand Dam Construction, Improved Hygiene Practice in Yemen YOFMEP Implementing Partner Notember Monitoring Status Report.” Assalahy, A. (2013). “EAVP-Y YOFMEP Implementing Partner November Monitoring Status Report.” Assalahy, A. (2013). “EFSP YOFMEP Implementing Partner November Monitoring Status Report.” Assalahy, A. (2013). “WASH Emergency Project for Yemen YOFMEP Implementing Partner November Monitoring Status Report.” Cook, G., Kemigisha, E., Kulaib, K., and Assalahy, A. IBTCI. (2014). “YOFMEP December 2013 Monitoring Report.” Cook, G., Kemigisha, E., Kulaib, K., and Assalahy, A. IBTCI. (2014). “YOFMEP February 2014 Monitoring Report.” 102 Cook, G., Kemigisha, E., Kulaib, K., and Assalahy, A. IBTCI. (2014). “YOFMEP January 2014 Monitoring Report.” Cook, G., Kemigisha, E., Kulaib, K., and Assalahy, A. IBTCI. (2014). “YOFMEP March 2014 Monitoring Report.” Cook, G., Kemigisha, E., and Assalahy, A. IBTCI. (2013). “YOFMEP November 2013 Monitoring Report.” Kemigisha, E. (2013). “Emergency Food and Nutrition Support to Food-Insecure and Conflict-Affected People YOFMEP November Implementing Partner Monitoring Status Report.” Kemigisha, E. (2013). “FARA YOFMEP Implementing Partner November Monitoring Status Report.” Kemigisha, E. (2013). “FSRB YOFMEP Implementing Partner November Monitoring Status Report.” IBTCI. (2013). “Child-Focused Health, Nutrition and WASH Emergency Response YOFMEP Output Monitoring Report.” IBTCI. (2013). “FSRB Verification Monitoring Report.” ANNEX IX: OFDA PARTNERS' STATEMENT OF DIFFERENCES 1. Feedback on evaluation report from Save the Children (pages 104 -107) II. Feedback on the evaluation report from UNICEF Yemen (pages 108 - 116) 108 Feedback Received from UNICEF Yemen on October 25th, 2014 We would like to express our thanks for the opportunity to comment on the OFDA Formative Evaluation Report for Yemen. We note with appreciation and value the findings and recommendations made on the UNICEF implemented programmes. The report is timely as we embark on the end year review of our 2014 programmes and begin the 2015 programme planning, including the humanitarian response plan. This is an opportunity for us to reflect on the observations and recommendations internally as well as with our Government and NGO partners and commit on the way forward to implement them where possible. We remain thankful for the continuous and generous support of the US Government. With regards to the report, we would like to offer the following clarifications and observations. On Nutrition Overall the distinction between the different types of nutrition food namely those used for SAM and MAM programme respectively is clearly stated in the overall objective and first part of the document. However, such a distinction is not consistently maintained in the document making it at times difficult to distinguish whether the findings and recommendations are related to SAM or MAM programmes. Page 4 Bullet 3: In recognition of this problem UNICEF & MoPH has been training community volunteer to track individual malnourished children. Monitoring tool for tracking malnourished children were developed and shared with OTPs and TFCs. In addition, tracking tools have been shared to indicate how much therapeutic products and other medicines are provided to children. Instructions are also given to mothers with regards to how the therapeutic foods should be administered to the malnourished child on a daily basis if he/she has been admitted to the TFC level or enrolled at an OTP level. Each child nutrition status is also assessed to ensure that the required weight for recovery has been gained. Efforts are ongoing to strengthen the health staff capacity to treat malnourished children, follow the protocols including in supply management. On-job mentoring is increasingly being provided during supervisory visits to ensure better compliance. Bullet 6. UNICEF in close coordination with the Social Welfare Fund has recently conducted a mapping of Mouhamasheen in Taiz (a marginalised group in Yemeni society) to better understand their needs and exclusion from basic social services. This will help inform UNICEF programming in the future. UNICEF is also scaling up community based integrated preventive and curative health, 109 nutrition and WASH interventions through community volunteer network (1 volunteer to 50 HH) to bring the service closer to the communities, including to marginalised groups. Bullet 7. As part of the scale up of the community based integrated package of services, UNICEF will establish a Village Development Committees (VDCs). These committees will serve as the voice of communities to raise complains and concerns, and will be linked to multi-sectoral coordination bodies at district and governorate levels. At HH level, community volunteers will act as the conduit to raise complaints. They will be linked to the VDC and to health facilities through monthly meetings. Page 4 under Health and Nutrition Recommendations, bullet 2. In order to strengthen the counselling skills of voluntary community health workers, the Nutrition and Health Education Directorate at MoPH, with support from UNICEF, has recently developed three integrated training manuals to make behavioural change communication more effective. The modules focus respectively on IYCF; IMCI; and growth monitoring and promotion as well as reproductive health. The training and capacity building have already started in some Governorates. Page 46, table 2, under Nutrition: With reference to the gaps in services, it would be have been useful to have a better sense from the beneficiaries on the types of services they feel is missing. In general, gaps in services are noticeable especially for PLW and MAM. UNICEF and WFP are working hard to integrate geographically SAM and MAM programmes as well as services to PLWs. In areas where WFP is not present UNICEF is providing plumpy nut to severe malnourished children until full recovery. UNICEF also initiated integrated community based prevention interventions in a few districts in 2014 with the plan to scale up interventions in 2015-2016 to reach 63 districts presenting the highest stunting and wasting rates. These interventions will also target children under 2 and LPWs, focusing on the first 1000 days with the objective of improving feeding and caring practices and preventing malnutrition. Page 51 - To strengthen the coordination and identify the role and responsibilities UNICEF and WFP have signed a Memorandum of understating (MOU) to strengthen coordination both at county office level and in the field. This has helped improve the geographic and programmatic convergence. Both organisations are increasingly working with the same implementing partners, merging training packages and working on the logistic side of the program by supporting MOH both technically and financially to accelerate supply distribution and avoid interruption of services, which is done in most of the targeted districts. Monthly meetings are conducted to ensure better coordination and complementarity of the programmes. 110 As part of UNICEF efforts to increase awareness and commitment of senior officials about understanding and addressing malnutrition have supported the government to develop a 5 years costed National Multisectoral Nutrition Action Plan with the objectives of addressing immediate and underlying causes of malnutrition through a comprehensive and integrated community based approach. An advocacy strategy is under development which objective is to help increase awareness and commitment as well as ensure accountability during the implementation of the Action Plan. Page 55 - As mentioned above UNICEF is scaling up the community based integrated package using community volunteer with the ratio of 1 to 50 HH to ensure increased and regular contact with HHs and to provide curative care i.e. treatment from childhood illnesses and malnutrition as well as preventive care i.e. monthly growth monitoring and promotion as well as age specific IYCF counselling services and referral to other health and livelihood interventions when needed. In 2014, to help improve the integration and coverage of health services UNICEF in 2014 supported capacity building of health staff as well as the implementation of quarterly integrated outreach services (vaccination, MUAC screening, supplementation and postnatal and antenatal care services for children and mothers). Pages 56 & 58 - There is interruption in the supply chain that requires an improvement in coordination within MoPH on which UNICEF (field and central level) are continuously working on resolving. The monthly report UNICEF receives from its field offices helps ensure follow up on any reported disruption in supplies. Delays of clearance at the port of entry to Yemen has been also an issue. UNICEF and WFP are working with MoPH to ensure that the supply management system is strengthened up to the facility level, including the improvement of transport and overall logistics management of supplies. Page 57 - regarding integration of vaccination and nutrition services, please see above comment on page 55. As for simplifying the nutrition recovery standard manual for health workers, a summary job aids for CMAM for nutrition recovery standards manual has been developed. It covers among other things admission and discharge criteria, anthropometric measurements and appetite test, quantities of the RUTF to be given to the child for treatment. UNICEF will ensure that these materials are distributed to health facilities. 111 Page 58 - UNICEF is supporting MoPH to build the capacity of health staff on IYCF counselling supported by a network of 1-50 HH community volunteer, using the newly developed IYCF counselling manuals, the training is being rolled out to districts. Page 59 - The community based integrated interventions will target HHs to ensure inclusion of fathers as well as other influential HH members such as grandmothers. Page 61 - An affected child by Severe and Moderate Acute Malnutrition is treated at the level of the TFC if he/she has complication or at the level of the OTP if the child has no complication. If the Child is screened and found moderately malnourished and there are no MAM services available, UNICEF is providing treatment to malnourished children until full recovery (i.e. treating them for MAM). This is done to ensure continuity of care especially for the severe cases. In addition UNICEF and WFP are conducting joint mapping of the affected areas and working towards improving the geographic and programmatic convergence of SAM & MAM services. However integration of SAM, MAM and IYCF services are sometimes constrained by lack of resources, particularly for the management of MAM. Efforts to ensure a continuum of care continue to be deployed and new guidelines have recently been finalised in that regard. Under the Recommendations Section - The Nutrition programme has benefited from the generous support of OFDA. Other donors have also contributed generously to the programme these include DFID, ECHO, the EU, Sweden, UNICEF National Committees, the Consolidated Emergency Response Fund. With these funds, UNICEF was only able to reach 60% of the SAM affected children to date with an integrated package of nutrition, health and WASH services. USAID contributed generously with more than a third of the total funding requirement to reach 60% of affected children. This included a partial contribution to purchase some of the needed RUTF. Page 87 – UNICEF appreciates OFDA’s recommendations including those in regards to the disruption of the supply chain, the need for a continuum of care for SAM and MAM children, and the improvement required to the referral system. As stated above efforts are being made to treat SAM cases until full recovery where treatment for MAM children is not available. UNICEF together with MoPH and WFP will continue working towards ensuring better convergence at health facility level of both SAM and MAM services, with an integrated approach to WASH and health services. Page 88 - UNICEF will also consider a revision to its communication strategy to ensure better awareness with regards to malnutrition and the Sphere standards within all levels of Government, with NGO partners and at the community. 112 Page 89 - UNICEF will continue to advocate with donors for increased funds to support the scaling￾up of the integrated community based package and to implement the newly developed national Multisectoral Action Plan. This will also require advocacy with other sectors such as WASH, Health, Education and Child Protection to implement integrated nutrition sensitive programmes, including to reach the most marginalised and hard to reach communities. On WASH Page 5, bullet 2 - UNICEF has been regularly monitoring the hygiene conditions in the IDP camps. Hygiene campaigns are conducted on regular basis for IDPs and hosting communities. There has been no major water related outbreaks reported during the last three years amongst displaced population. This serves as a good indicator with regards to the effectiveness of hygiene campaigns. In addition the campaigns specifically targeting IDPs, national campaigns for hygiene such as the global hand washing days also target displaced population. Page 64. Indeed, it is recognised that the government capacity is weak in managing and regulating the water supply sector. UNICEF is investing through its regular and humanitarian programmes in strengthening government capacity. Humanitarian programmes have generally benefitted from better Government support than development programme. National Water Quality Guidelines are available but enforcement and monitoring of their application is unfortunately not taking place. Page 68 - UNICEF has detailed plans and processes in place to monitor the implementation of its programs projects. For supply of materials in humanitarian situations, UNICEF field staff and partners monitor to ensure receipt by every beneficiary. For physical works, monitoring is even better because of the works being physical in nature. The estimation/costing is monitored by UNICEF staff and verified with the market. UNICEF is happy to explain further its monitoring mechanism to OFDA. Page 69 - UNICEF does not invest in privately owned water sources. Well protection is in progress in 75 locations with a commitment to allow access to villagers. Page 70 and Pages 71 - Most of the water that UNICEF is supplying, is regularly treated in IDP areas. Where water is not treated, UNICEF supplies water filters (but not from OFDA grant). The water treatment is done at source where communities are not involved that’s why the respondents might not be aware. UNICEF will ensure that all water users are fully informed about the treatment of water supplied to them. 113 Page 73 - UNICEF will study the experience of GC and would welcome discussion for introduction in UNICEF BCC campaigns. Pre-positioning of materials to address the needs of 25,000 people is part of UNICEF’s regular programme planning and will continue. Page 75 - UNICEF continues to invest in developing the technical capacity of its local NGO and Government partners who lead the WASH response to IDPs. UNICEF will further investigate any capacity gaps of its NGO and their staff to help further strengthen their technical skills. Page 92 - Line of inquiry - UNICEF uses different approaches in its emergency assistance. Although water trucking operation is still ongoing it has been substantially reduced by 60% during the last 12 months. The IDPs particularly in Hajjah governorate, where bulk of water trucking takes place, are now accessing water through water networks supported by UNICEF. Efforts are ongoing to handover those water networks, boreholes and pumping operations to host communities. This is expected to be completed in the coming months. Page 92, Recommendation No 22. The recommendation is well noted. However, complete community involvement is challenging in humanitarian situation. The recommendation aligns with UNICEF WASH regular programme where work in that regard is ongoing. In the 1022 villages targeted for CLTS, water supply component is part of the incentive for community that achieves Open Defecation Free status. The investment on this incentive though is limited, but communities become more efficient once mobilised. Page 93, Recommendation No 27. In 2014, UNICEF has rolled out CLTS in 1022. Regular development resources are being used for this purpose. CLTS cover also areas with humanitarian. UNICEF is partnering with seven new partners to implement this programme. To help build built national capacity in CLTS national guidelines and trainer manuals have also been developed. About 70 master trainers are already trained. The training is being rolled out to CLTS facilitators in 15 districts. On Child Protection Page 79 - UNICEF has two additional Implementing Partners (IPs) in MRE activities (mainly in southern governorates): Save the Children International (SCI) and Intersos. Though we acknowledge and agree that we need to prioritise teachers in conveying MRE messages to student, UNICEF is aiming also at reaching also children who are not enrolled in schools through community based MRE activities. 114 (Coverage): UNICEF faced challenges in accessing some conflict locations like Abyan, Dhale, and Saada as access was denied by parties to the conflict. Page 80 - (community involvement): Although YEMAC has involved community members in the selection of high risk districts and schools, UNICEF will consider specific involvement of community in future MRE activities. However, UNICEF believes that as per current practice, YEMAC needs to continue the selection of suitable teachers in coordination with MOSAL, Education offices and local authorities. Page 81 - (suggestion of using teachers as trainers as well as to conduct MRE activities) UNICEF had originally included TOT training for teachers. In order to keep consistency in the messages, UNICEF Government counterpart, YEMAC, insisted to directly train staff who are conducting MRE awareness activities. However further advocacy with YEMAC enabled Save the Children to train teachers to conduct MRE awareness in schools under the supervision of YEMAC staff. Furthermore Democratic School trained teachers in Sada’a as YEMAC did not have access to the Governorate. (monitoring mechanisms) UNICEF is collecting quarterly technical reports not only with quantitative data but also with qualitative data. In addition as a way to improve quality, YEMAC trains NGO staff and teachers to conduct MRE activities together with one YEMAC experienced staff. DRC follows the same approach when implementing the training for community based trainers (CBT) where one staff from DRC accompanies the CBTs and checks the quality of MRE activities through regular meetings with the CBTs. Moreover, UNICEF staff at central and field level conduct regular monitoring visits to ensure quality. YEMAC has developed a monitoring systems to supervise and review MRE activities under their supervision. UNICEF however recognises that further improvement on monitoring activities is required. (inclusion of disabled and elderly): The house-to- house approach of DRC aims at including the whole family in MRE awareness in interventions. Experience however proves that the house-to-house approach is very expensive, this is why it is not implemented on a larger scale. YEMAC does encourage participation of disabled children and gives them priority when distributing MRE materials. YEMAC also coordinates with teachers of deaf and dumb children to provide MRE sessions using the signal language. (MRE parallel to de-mining): UNICEF fully agrees on the importance of conducting MRE parallel to de-mining, and strongly advocates with relevant authorities to coordinate MRE and de-mining activities. UNICEF appreciates the support of OFDA in advocating with other donors in that regard. 115 (Use of cameras) UNICEF provided YEMAC with cameras to be used in MRE activities in Hajjah, Sana’a and Amran. Provision of cameras in southern governorates is scheduled for 2014. Cameras were also part of the support provided by UNICEF to DRC and InterSoS. It is worth mentioning that UNICEF through the generous funding of its donors has supported the cost of all MRE activities in country. Page 95, Recommendations - OFDA generously contributed to the 95% of the MRE related activities. ($1,423,114.64 out of 1.5 million), UNICEF also received funds from other donors such as CERF. UNICEF appreciates all the recommendations provided by OFDA including to integrate MRE in school curriculum; to have TOT especially for teachers; financial contribution of MRE by the Government; increased involvement of communities; conducting baseline/KAPB. UNICEF will review its supply chain for teaching materials. UNICEF will also consider two additional areas in its future programming namely: - UNICEF is planning to build the capacity of YEMAC to establish an accreditation system as per the MRE standards to include a wider and diversified range of MRE partners to respond in a timely manner to the needs on the ground. YEMAC shall ensure good and quality MRE messages by providing thorough training to partners and implementing close monitoring and supervision activities. - UNICEF will advocate with donors for the need to include MRE teaching supplies (such as leaflet, posters, booklets etc.) for children as well as the need to support house-to-house approaches to ensure reaching the family as whole. On Overall Behaviour Change Communication Health and Nutrition: Page 51 & Page 55; WASH p 70; Child Protection P81. UNICEF recognises the fact that deeply rooted social norms in Yemeni societies greatly influence family and individual knowledge and attitudes. This presents major bottlenecks in decision-making and the adoption of positive care practices as well as in demand for and uptake of services. In this regard UNICEF Yemen has established a fully-fledged communication for development (C4D) programme whose primary responsibility is to strengthen delivery of an integrated behaviour and social change interventions at the household and community level, with focus on 11 key life-saving, care and protective services. These include promotion of uptake of antenatal care, routine immunization, infant and young child feeding, integrated management of childhood illnesses, hygiene promotion, child protection and gender sensitive on-time enrollment, among others. 116 The interventions include capacity building and systems strengthening of government and CSO partners to effectively deliver SBCC interventions as well as an extensive community engagement programme, with the household as the unit of intervention for change. An integrated C4D package is being finalized to ensure coherent messaging and interactive approaches among partners as they engage with communities. OFDA recommendations for improving behaviour change interventions are well aligned with UNICEF’s own efforts at a holistic approach to behaviour change. Page 90, Recommendation no 16 - UNICEF has streamlined and improved its training curriculum for CHWs integrating a 3 day interpersonal communication/counselling skills with a further 2 days training on messaging around the 11 key care, lifesaving and protective practices, reporting and work planning. It has also included strengthening CHV coordination, reporting and supervision systems at community and district level through partnerships with civil society organisations and capacity building of district health education staff of the ministry of health. Page 97, Recommendation 39 & 40 - UNICEF is already expanding its approach to community engagement prioritizing partnerships that privilege community voices and participation of the most vulnerable and often excluded members of the community ( through focus group discussions, individual interviews and testimonials, etc) in addressing pertinent community attitudes and practices that impact on effective care practices for children. U.S. Agency for International Development Office of Foreign Disaster Assistance and Office of Food for Peace 1300 Pennsylvania Avenue, NW Washington, DC 20523