UNITED STATES AGENCY FOR INTERNATIONAL DEVELOPMENT OFFICE OF FOOD FOR PEACE PUBLIC LAW 480, TITLE II DEVELOPMENT FOOD ASSISTANCE PROJECT LIVELIHOODS, AGRICULTURE AND HEALTH INTERVENTIONS IN ACTION (LAHIA) PROJECT MID-TERM EVALUATION REPORT MARADI, NIGER AUGUST 2012 – AUGUST 2017 Evaluation Dates: September 2015 – November 2015 Report Author: Kyung Endres Evaluation Team Members: Kyung Endres, Chaibou Dadi, Aboubacar Souley, Zalika Ayouba Abdou, Aichatou Guero, Mousthapha Brah Mamadou, and Amadou Sahirou LAHIA Midterm Evaluation Report PUBLICATION PAGE i PUBLICATION PAGE We would like to thank all of the contributors to this report, the Livelihoods, Agriculture and Health Interventions in Action Project monitoring, evaluation and learning team in Niger, Maradi officials of the Government of Niger, LAHIA project staff, Save the Children U.S. headquarters staff, partner representatives and project beneficiaries. Without the participation and input from these people, the evaluation would not have been possible. Special appreciation goes out to the evaluation team specialists, Chaibou Dadi and Aboubacar Souley for their leadership and counsel and to the evaluation team members, Zalika Ayouba Abdou, Aichatou Guero, Mousthapha Brah Mamadou, and Amadou Sahirou for their hard work and dedication. This evaluation was performed under contracts between Save the Children Federation and True Panacea, LLC, Save the Children International and Souley Aboubacar, and Save the Children International and Chaibou Dadi. Preferred citation: Endres, Kyung. Livelihoods, Agriculture and Health Interventions in Action Project Mid￾Term Evaluation Report. Rep. Washington: Save the Children U.S., 2015. Print. LAHIA Midterm Evaluation Report TABLE OF CONTENTS 67 ii TABLE OF CONTENTS PUBLICATION PAGE ....................................................................................................................................... 1 TABLE OF CONTENTS ..................................................................................................................................... 2 TABLE OF FIGURES AND TABLES .................................................................................................................... 4 ACRONYMS .................................................................................................................................................... 5 EXECUTIVE SUMMARY ................................................................................................................................... 1 Project Description .................................................................................................................................... 1 Evaluation Description ............................................................................................................................... 1 Evaluation Methods .................................................................................................................................. 2 Evaluation Findings .................................................................................................................................... 2 Recommendations .................................................................................................................................... 3 INTRODUCTION: PROJECT DESCRIPTION ....................................................................................................... 4 Food Security in Niger ............................................................................................................................... 4 Overview of Food for Peace Security Projects ........................................................................................... 4 LAHIA Project Results Framework and Activities ....................................................................................... 6 LAHIA Project Geographic Targeting ......................................................................................................... 7 LAHIA Project Beneficiary Targeting .......................................................................................................... 9 LAHIA Project Implementing Partners ....................................................................................................... 9 LAHIA Project Community Organizations ................................................................................................10 OBJECTIVES OF THE MID-TERM EVALUATION .............................................................................................12 EVALUATION METHODS AND LIMITATIONS ................................................................................................13 Interpreting the Evaluation Questions ....................................................................................................13 Evaluation Team ......................................................................................................................................13 Evaluation Methods ................................................................................................................................14 Literature Review ..................................................................................................................................14 Qualitative Interviews ...........................................................................................................................14 Evaluation Processes ...............................................................................................................................15 Site Selection Criteria and Process ........................................................................................................15 Respondent Selection Criteria and Process ..........................................................................................16 Evaluation Plan .....................................................................................................................................17 Evaluation Limitations .............................................................................................................................17 ACTIVITY DESCRIPTION, FINDINGS AND DISCUSSION ..................................................................................19 Overall Program.......................................................................................................................................19 Technical Approach by Sector and Cross-Cutting ....................................................................................19 Agriculture and Livelihoods ..................................................................................................................19 Health, Nutrition, Water Sanitation and Hygiene .................................................................................25 Cross-Cutting Gender............................................................................................................................31 Monitoring & Evaluation .......................................................................................................................33 Commodity Management .....................................................................................................................34 Environmental Safeguard and Compliance ...........................................................................................35 Implementation of Activities ...................................................................................................................35 Targeting ...............................................................................................................................................35 Progress ................................................................................................................................................38 Quality ..................................................................................................................................................41 Sustainability .........................................................................................................................................41 Management ........................................................................................................................................42 LAHIA Midterm Evaluation Report TABLE OF CONTENTS 67 iii Partnership ..............................................................................................................................................45 Implementing Partners .........................................................................................................................46 Partnership with Government Services ................................................................................................46 Technical Implementing Partners .........................................................................................................47 Other Development Actors ...................................................................................................................47 Institutional Capacity ............................................................................................................................48 RECOMMENDATIONS ..................................................................................................................................50 Invest in Relationships for Sustainability ................................................................................................. 50 Focus on Institutional Strengths ..............................................................................................................50 Sub-Contract Activities Outside of Institutional Strengths ......................................................................50 CONCLUSIONS .............................................................................................................................................51 REFERENCES AND APPENDICES ...................................................................................................................52 Appendix A: Indicator Performance Tracking Table ................................................................................53 Appendix B: List of Villages Where LAHIA Works ....................................................................................54 Appendix C: Scope of Work .....................................................................................................................55 Appendix D: List of Respondents .............................................................................................................56 Appendix E: LAHIA Mid-Term Evaluation Plan .........................................................................................57 Appendix F: Evaluation Calendar .............................................................................................................58 Appendix G: Tables of Evaluation Questions ...........................................................................................59 LAHIA Midterm Evaluation Report TABLE OF FIGURES AND TABLES iv TABLE OF FIGURES AND TABLES Figure 1 Food For Peace Conceptual Framework .......................................................................................... 5 Figure 2 USAID/FFP Niger Implementation Area ........................................................................................... 8 Figure 3 LAHIA Target Groups from Proposal ..............................................................................................36 Figure 4 Save the Children LAHIA Organizational Chart ..............................................................................44 Table 1 LAHIA Project Strategic Objectives, Intermediate Results and Activities .......................................... 6 Table 2 Population Participating in the LAHIA Project ................................................................................... 8 Table 3 LAHIA Implementing Partner List and Associated Organizations (with activities)............................. 9 Table 4 LAHIA Community-Based Organizations .........................................................................................10 Table 5 LAHIA Mid-Term Evaluation Key Staff .............................................................................................13 Table 6 LAHIA Mid-Term Evaluation List of Villages ....................................................................................16 Table 7 Number of Respondents by Type of Respondent ...........................................................................16 Table 8 Subjects Respondents Reported Learning through LAHIA Extension Agents ..................................27 Table 9 LAHIA Targeting ..............................................................................................................................36 Table 10 Output Indicators, Actual & Targeted Values ...............................................................................40 Table 11 LAHIA Mid-Term Evaluation Questions .........................................................................................59 Table 12 Overall Evaluation Questions ........................................................................................................63 Table 13 SO2 Agriculture Evaluation Questions ..........................................................................................63 Table 14 SO2 Livelihoods Evaluation Questions ..........................................................................................63 Table 15 Ration Mid-Term Evaluation Question ..........................................................................................64 Table 16 Health & Nutrition Mid-Term Evaluation Questions .....................................................................64 Table 17 Community-Led Total Sanitation Mid-Term Evaluation Questions ...............................................64 Table 18 Water and Sanitation, Hygiene Mid-Term Evaluation Questions .................................................64 Table 19 Gender Mid-Term Evaluation Questions ......................................................................................64 Table 20 Monitoring and Evaluation Mid-Term Evaluation Questions ........................................................64 Table 21 Commodity Management Mid-Term Evaluation Questions .........................................................64 Table 22 Environmental Compliance Mid-Term Evaluation Questions .......................................................65 Table 23 Targeting Mid-Term Evaluation Questions ...................................................................................65 Table 24 Progress Mid-Term Evaluation Questions .....................................................................................65 Table 25 Quality Mid-Term Evaluation Questions .......................................................................................65 Table 26 Sustainability Mid-Term Evaluation Questions .............................................................................65 Table 27 Management Mid-Term Evaluation Questions .............................................................................65 Table 28 Implementing Partner Mid-Term Evaluation Questions ...............................................................65 Table 29 Government Mid-Term Evaluation Questions ..............................................................................66 Table 30 Local Implementing Partner Mid-Term Evaluation Questions ......................................................67 Table 31 Other Development Actor Mid-Term Evaluation Questions .........................................................67 Table 32 Institutional Capacity Mid-Term Evaluation Questions ................................................................. 67 LAHIA Midterm Evaluation Report ACRONYMS v ACRONYMS ACRONYM ORIGINAL LANGUAGE TRANSLATION ADS Automated Directives System CLTS Community Led Total Sanitation DFAP Development Food Assistance Project FFP Food for Peace FP Family Planning HHLD Household INRAN Institut National des Recherches Agronomiques du Niger National Institute of Agronomical Research of Niger IPTT Indicator Performance Tracking Table IR Intermediate Result IYCF Infant and Young Child Feeding KM Kilometer LAHIA Livelihoods, Agriculture and Health Interventions in Action M&E Monitoring and Evaluation MCHN Maternal Child Health and Nutrition MEAL Monitoring, Evaluation and Learning MMD Mata Masu Dubara Revolving Credit Group MTE Mid-Term Evaluation ONG Organisation non-gouvernementale Non-governmental organization OSV Observatoire de Surveillance de la Vulnérabilité Risk Assessment Team of Commune PLW Pregnant and Lactating Women REGIS AG Resilience and Economic Growth in the Sahel – Accelerated Growth RISE Resilience in Sahel Enhanced RNA Régénération Naturelle Assistée Natural Resources Management Committee SBC Social and Behavior Change SCUS Save the Children United States SCAP/RU Systeme Communautaire d'Alerte Precoce/Reponse aux Urgences Community Early Warning System/Emergency Response SO Strategic Objective SONGES L’Association d’Appui aux ONG du Sud Association for Supporting Southern NGOS SOW Scope of Work USAID United States Agency for International Development VDC Village Development Committee Comité de Développement Villageois VIP Ventilation Improved Pit LAHIA Midterm Evaluation Report ACRONYMS vi WASH Water, Sanitation and Hygiene WVUS World Vision United States LAHIA Midterm Evaluation Report EXECUTIVE SUMMARY 1 EXECUTIVE SUMMARY Project Description Save the Children Federation, Inc. (SCUS), with partner World Vision, manages and implements the Livelihoods, Agriculture and Health Interventions in Action (LAHIA)1 project in the Marardi region of Niger. The project is funded by the United States Agency for International Development (USAID) Office of Food for Peace (FFP). It started in fiscal year 2012 and is scheduled to finish at the end of fiscal year 2017. This is a food security project with the goal to reduce food insecurity and malnutrition among poor rural households in the Aguié and Guidan Roumdji Districts of the Maradi Region of Niger. It has four strategic objectives: SO1: Nutritional status of children under 5 years of age and pregnant and lactating women improved SO2: Access to food by vulnerable households increased SO3: Vulnerability to food security shocks reduced SO4: Meaningful participation by women at the household and community levels improved (cross-cutting) Evaluation Description The mid-term evaluation contained three key research questions. 1. Assess the strengths and challenges of project implementation and outputs in terms of meeting terms agreed with FFP and their acceptability and usefulness to the targeted communities, and to identify factors that appear to enhance or detract from the efficiency, quality, acceptability, and usefulness of the activities’ implementation and outputs. 2. Present evidence of changes (intended and unintended) associated with project activities and outputs, and identify factors in the implementation or context that impede or promote the observed and intended changes. 3. Recommend adjustments to the project design, resource allocation, or implementation that could improve the likelihood of achieving desired results by the project’s end – based on the findings from evaluation objectives above. To provide for greater clarity of the research questions, the LAHIA program specified over 100 qualitative evaluation questions. These questions were classified into four main areas of questioning. Under three of the classifications, 17 sub-categories of questions were included as listed below. 1) Overall Evaluation Questions 2) Technical Approach a. agriculture and livelihoods b. health, nutrition, water sanitation and hygiene c. gender d. monitoring and evaluation e. commodity management f. environmental safeguard and compliance 1 Hausa “LAHIA” means good health LAHIA Midterm Evaluation Report EXECUTIVE SUMMARY 2 3) Implementation of Activities a. targeting b. progress c. quality d. sustainability e. management 4) Partnerships a. with implementing partners b. with government technical services c. with government municipal authorities d. with government administrative authorities e. with technical implementing partners f. with other development actors Evaluation Methods Save the Children hired seven consultants to conduct the mid-term project evaluation. Data were collected from representative of all types of project stakeholders and beneficiaries. A total of 85 key informant interviews and 39 focus group discussions were conducted by the seven consultants over a period of six days. Interviews and discussions were conducted in French with government officials and Hausa, Djerma, and French with community members. The primary level of data analysis was project beneficiary data. Project staff interviews, partner interviews, and project reports verified findings of beneficiaries. The principle investigator and two local researchers organized and analyzed the data in response to the 126 evaluation questions. Within each of the evaluation questions the three research questions were addressed. The report presents the findings, discussion, and recommendations in accordance with the organizations of the evaluation questions. Evaluation Findings As a general rule, the evaluation team found that the project was very good at implementing strategic objective (SO) 1 activities which had discrete starting and ending points with clear participant targeting, objectives, and processes for ensuring expected outcomes would be met. The most well planned and organized activities were also the most successful from the beneficiary point of view. These activities included: drilling wells; handing out rations; and educating people on health, nutrition and hygiene. It would be expected that Save the Children, a global leader in promoting women and children’s health, should excel at these types of activities. Outside of the goat rotation scheme, the SO2, SO3, and SO4 activities were not expected to yield immediately tangible results. Most of the efforts required a longer-term vision and investment of time and resources to achieve the sought after gains. For example; the farmer field schools required an entire growing season, if not two to see the benefits of the technologies being promoted; the cereal banks entailed waiting several months until the lean season to sell produce while simultaneously earning off￾farm income; village committees were expected to prepare to manage unforeseen shocks for an undetermined time in the future; and community members were asked to question gender norms for an unknown future which the project promises will be better than today. In all of these cases, beneficiaries were asked to suspend disbelief, risk their meager resources on the unknown, and take a leap of faith into the unknown. Beneficiary expectations for the unknown were not effectively managed. The project staff and the beneficiaries made attempts at implementing the non-tangible activities with a limited degree of success. LAHIA Midterm Evaluation Report EXECUTIVE SUMMARY 3 Several project staff mentioned that the target population was accustomed to receiving hand-outs from projects. Community members were not as interested in longer term investments, despite the project having explained that the LAHIA project was a facilitator and not a gift-giver. Recommendations Activities which require improvement include farmer’s field schools, value chain promotion, monitoring and evaluation. These activities can be improved by following the evaluation’s recommendations for program improvements, which are: a) invest in relationships; b) focus on institutional strengths; and c) sub-contract activities for which LAHIA is performing poorly. LAHIA Midterm Evaluation Report INTRODUCTION 4 INTRODUCTION: PROJECT DESCRIPTION Food Security in Niger “Nigerien children are among the most malnourished globally.2 The prevalence of chronic malnutrition (stunting) is estimated at 51% nationally with the highest levels found in Zinder Region (64.8%) followed by Maradi, where stunting rates are estimated at 63% with 61.4% in girls and 64.8% in boys respectively.34 This is surprising given that 40% of Niger’s food production takes place in the two regions which comprise Niger’s agro-pastoralist belt, Maradi and Zinder. Poor infant and young child feeding (IYCF) practices, limited access to quality health care, and poor sanitation and hygiene linked to water access are primary contributors. Clearly, food access is not the only cause of food insecurity. Purchasing power is challenged by regular environmental and economic shocks, ever-decreasing land plots due to population growth, and poor access to and knowledge of sustainable farming and marketing practices. Half the population cannot afford a balanced diet, even though sufficient food is locally available.5 Nearly 85% of children admitted to nutrition treatment centers for severe malnutrition come from poor or very poor families.6 Gender inequities and several decades of political instability cut across the picture, and girls and women are particularly marginalized. Cultural norms linked to land tenure, asset ownership and accumulation, education access, early marriage, and household decision-making leave women without safety nets, skills, or economic opportunities. While the current (new) government has acknowledged the importance of women’s rights, policies for access to health care, strengthened production and marketing, and early warning information systems are not in place, nor have government positions been established or funded to carry forward the new priorities.”7 Overview of Food for Peace Security Projects The United States Agency for International Development (USAID) Office of Food for Peace (FFP) provides food and cash assistance in support of Development Food Assistance Projects (DFAPs). Their mandate funds food security projects in low-income food-deficit countries. The FFP conceptual framework, from the FFP 2006-2010 strategy, presented in Figure 1 provides an overview of office priorities and the relationships between types of activities, risks and intended outcomes. 2 Approximately 41% of all children in Niger are underweight (<2 weight/age per WHO Child Growth Standards), the 4 th highest level of malnutrition in the world. UNICEF State of the World’s Children 2011, http://www.unicef.org/sowc2011 3 MOH-INSN. (June 2011) Rapport de l’Enquête SMART. 4 SC’s August 2011 nutritional survey conducted in six districts, including Aguié, found similar results. 5 Save the Children. (2007) Household Economy Approach Survey. 6 Save the Children UK, Niger. (2008) Study on the Causes of Malnutrition, Tessaoua District. 7 Save the Children US (April 2015) Terms of Reference Midterm Evaluation LAHIA Program. LAHIA Midterm Evaluation Report INTRODUCTION 5 Figure 1 Food For Peace Conceptual Framework8 Note: Figure 1 is the USAID/Bureau for Democracy, Conflict and Humanitarian Assistance /Food for Peace conceptual framework. It shows the relationships between food security risks, desired program outcomes, desires 8 Food for Peace. (May 2005) Strategic Plan for 2006-2010. Food security risks to be tackled Desired Program Outcome Food Security Outcome Strategic Goal Adequate Food Availability Adequate Food Access Appropriate Food Utilization Food Security Human Capital Nutrition; Health and sanitation Maternal and child care; Dignity, Education; Skills; Political voice; Capacity, Indigenous knowledge Consumption Equity in intra￾household food distribution; Food quantity, quality, diversity Income Market Integration, Purchasing power; Savings potential; Credit access Productivity Labor productivity, Livelihood stability and diversification Resources Natural resource sustainability; Productive assets; Secure livelihoods Enhanced Community Resiliency Enhanced Livelihood Capacity Enhanced Human Capital ‘Natural’ Shocks Climatic shocks; Natural resource mining and degradation; Yield volatility; Asset depletion; Neglect of natural hazard mitigation Economic Risks Income fluctuation; Collapsed terms of trade Savings depletion Employment insecurity; Price volatility; High transactions costs; Information asymmetry Inflation Social and Health Risks Epidemics, HIVAIDS; Widespread untended undernutrition Risk perceptions Corruption; Crime Social disintegration; Predatory extraction by armed forces; Conflict; Ethnic and social discrimination Political Risks: Poor governance (national and local); Lack of legal recourse; Inadequate representation; Lack of accountability; Inadequate provision of services and creation of public goods; Adverse regulations; Lack of recognition of human rights; Political instability; Ineffective institutions. Food Insecurity LAHIA Midterm Evaluation Report INTRODUCTION 6 food security outcomes, and the strategic goal of food security. LAHIA Project Results Framework and Activities The LAHIA project, started 17 August 2012, goal is to reduce food insecurity and malnutrition among poor rural households in the Aguié and Guidan Roumdji Districts of the Maradi Region of Niger. The Results Framework, Table 1, serves as a roadmap for the program and reflects a set of strategies to build upon and address the challenges which lead to food insecurity and malnutrition among poor rural households in Maradi. The Results Framework represents the logical and sequential order in which the program implements activities, and the achievements which result from the various efforts. Table 1 is a synthesis of the indicator performance tracking table (IPTT) strategic objectives (SOs), intermediate results (IRs), and activities. The project has five years to achieve targets they have set as measured by outcome and output indicators listed in Appendix A: Indicator Performance Tracking Table. Table 1 LAHIA Project Strategic Objectives, Intermediate Results and Activities SO1: NUTRITIONAL STATUS OF CHILDREN UNDER FIVE YEARS OF AGE AND PREGNANT AND LACTATING WOMEN IMPROVED IR 1.1: Adoption of Key Maternal Child Health and Nutrition (MCHN) Practices at Household and Community Level by Men and Women Increased Sub-IR 1.1.1: Knowledge of key health and nutritional practices increased Activity 1.1.1.1: Design comprehensive social and behavior change (SBC) strategy Activity 1.1.1.2: Implement SBC strategy Activity 1.1.1.3: Reinforce family planning (FP) promotion and linkages with services Activity 1.1.1.4: Support for exclusive breastfeeding at community, commune, and district levels Sub-IR 1.1.2: Protect most vulnerable pregnant and lactating women (PLW) and children under 2 Activity 1.1.2.1: Distribute MCHN rations to vulnerable PLWs and children under 2 Activity 1.1.2.2: Distribute protective family rations during the lean season to households receiving individual rations IR 1.2: Utilization of Key MCHN Services at Community and Health Facility Levels Increased Activity 1.2.1: Reinforce delivery of services at health facility and community levels IR 1.3: Access to Potable Water and Sanitation Facilities for Men, Women, Girls, and Boys Increased Activity 1.3.1: Construct at least 99 new and rehabilitate 40 existing water points Activity 1.3.2: Construct 261 household ventilation improved pit (VIP) latrines and 25 public gender￾segregated VIP Activity 1.3.3: Promote treatment options for drinking water collected from open wells and surface sources and good hygiene practices Activity 1.3.4: Train two groups: local masons and pump maintenance/repair technicians Activity 1.3.5: Establish water, sanitation and hygiene (WASH) Management Committees for all public water and sanitation facilities SO2: ACCESS TO FOOD BY VULNERABLE HOUSEHOLDS INCREASED IR 2.1: Women’s Roles in Livelihoods Diversification and Protection Enhanced Activity 2.1.1: Establish and /or strengthen women’s savings groups Activity 2.1.2: Identify and strengthen women’s income generating activities (agricultural and non￾agricultural) Activity 2.1.3: Provide small ruminants, inputs and services for animal husbandry Activity 2.1.4: Create women's gardens and small livestock enterprises IR 2.2: Use of Ecologically Sound Agriculture and Natural Resource management (NRM) Practices by Women and Men Increased Activity 2.2.1: Train staff and partners in good agricultural techniques and management practices. Activity 2.2.2: Disseminate good agricultural techniques and management practices at the village level Activity 2.2.3: Facilitate access and linkages to essential inputs and services LAHIA Midterm Evaluation Report INTRODUCTION 7 Table 1 LAHIA Project Strategic Objectives, Intermediate Results and Activities IR 2.3: Agricultural Marketing Improved for Women and Men Activity 2.3.1: Strengthen capacity in agricultural business planning, knowledge and skills Activity 2.3.2: Establish mechanism to access market information SO3: VULNERABILITY TO FOOD SECURITY SHOCKS REDUCED IR 3.1: Community-Level Capacity to Respond to and Mitigate Shocks Improved Activity 3.1.1: Create new and strengthen existing village development committees (VDCs) Activity 3.1.2: Pilot food and nutritional security surveillance system IR 3.2: Commune Management Structure Related to Crisis Prevention and Response Improved Activity 3.2.1: Provide regular training and equipment to commune-level units for crisis prevention and response Activity 3.2.2: Strengthen commune-level plans for preparedness and management of crises SO4: MEANINGFUL PARTICIPATIION BY WOMEN AT THE HOUSEHOLD AND COMMUNITY LEVELS IMPROVED (CROSS-CUTTING) IR 4.1: Women’s Active Participation in Community Activities Promoted Activity 4.1.1: Train staff and partners on gender awareness Activity 4.1.2: Conduct a gender audit in target communities and ensure integration of gender-specific needs into program activities Activity 4.1.3: Address practical and strategic gender needs across SOs 1-3 Activity 4.1.4: Encourage the participation of women across all activity areas IR 4.2: Community Leaders’ Engagement in Promoting Women’s Status is Improved Activity 4.2.1: Organize gender reflection days and international women's days with community leaders Activity 4.2.2: Train VDCs in gender awareness (links SO3, Activity 3.1.1) Note: The information in Table 1 was taken from the LAHIA IPTT. The list of LAHIA activities are categorized by strategic objectives (SOs) and intermediate results (IRs). LAHIA Project Geographic Targeting The LAHIA project targeted five communes in the Maradi Region; Aguié, Gangara,Guidan Sori, Saé Saboua, and Tchadoua. Save the Children Federation, Inc. (SCUS) 9 implements activities in Aguié, Gangara, and Tchadoua. World Vision (WV) works in Guidan Sori and Saé Saboua. The targeted communes where LAHIA works are shown in Figure 2. As of June 2015, they work in 72 villages with just over 14,000 households. This is an estimated 70% of the total population of the targeted villages10 . Table 2 provides an overview of the scope of the LAHIA project by commune. Aguié is the most active commune containing 30% of all LAHIA households. The commune of Tchadoua includes the smallest number of households (1,828) but an equal proportion of the targeted population (70%) as the other communes. The full list of LAHIA villages in contained in Appendix B: List of Villages Where LAHIA Works. 9 SCUS is one of 30 national organizations who have agreed to implement non-domestic programs through a single global program delivery unit, Save the Children International (SCI). 10 [14,038 project households in 72 villages x 5.7 household members (World Bank. 2014. The World Economic Factbook.) = 80,017 people] / [113,708 population of 72 villages] LAHIA Midterm Evaluation Report INTRODUCTION 8 Figure 2 USAID/FFP Niger Implementation Area Note: USAID/FFP Niger DFAP program areas. The LAHIA program areas are highlighted in yellow. The program works in 72 villages in the Aguié, Gazaoua and Guidan Roumdji Departments. Table 2 Population Participating in the LAHIA Project COMMUNE # LAHIA VILLAGES/ COMMUNE # HHLDS in LAHIA VILLAGES TOTAL POPULATION in LAHIA VILLAGES # LAHIA BENEFICIARY # LAHIA BENEFICIARY/ TOTAL POPULATION % DISTRIBUTION of LAHIA BENEFICIARIES by COMMUNE Aguié 19 4293 34773.3 24470.1 70.37% 30.58% Gangara 12 3321 26900.1 18929.7 70.37% 23.66% Tchadoua 14 1828 14806.8 10419.6 70.37% 13.02% Guidan Sori 10 2002 16216.2 11411.4 70.37% 14.26% Saé Saboua 17 2594 21011.4 14785.8 70.37% 18.48% Total 72 14038 113707.8 80016.6 70.37% 100.00% Note: The numbers of villages, households (HHLDs), and population included in the LAHIA project as of June 2015. SC is the community lead in the first three communes and WV in the last two. The last column shows that the SC manages just under 65% of the LAHIA beneficiaries. The second to the last column shows that project beneficiary numbers are estimated as 70.37% of the total population for each commune. LAHIA Midterm Evaluation Report INTRODUCTION 9 LAHIA Project Beneficiary Targeting The LAHIA project works with the poor and the very poor. Rations are distributed to pregnant and lactating women (PLW), caretakers of children under two years old, children under two years old, and their families. Other activities are open to any person residing in the targeted village. Participation is voluntary in all activities. Food rations are conditional on women’s attendance at required pre and post￾natal consultations and children’s vaccinations and check-ups. LAHIA Project Implementing Partners The LAHIA project is implemented by SC and WV. SC is the prime cooperative agreement holder and WV is a sub recipient. Local organizations L’Association d’appui aux ONG du Sud (SongES) and Initiative de Sécurité des Conditions de Vie (ISCV) have conducted husband school activities. The project engages with local partners to implement targeted activities. The list of local partners and the activities in which they are involved are listed in Table 3. Government technical services provide some degree of oversight and supervision as related to their respective mandates. Their involvement as overseers ensures that project activities meet government requirements. Health and rural engineering technical services also provide technical services to LAHIA beneficiaries. The project uses non-governmental organizations (NGOs) and community members, to deliver social and behavior change (SBC) messages and to provide a range of services and improved seeds. Table 3 LAHIA Implementing Partner List and Associated Organizations (with activities) NIGERIEN IMPLEMENTING PARTNERS LAHIA ACTVITIES Project Administration Governor of region Cooperation: information sharing and feedback Magistrate of commune (Préfet) Coordination: government technical services Mayor of communities Coordination: village entry and outreach SO1: Nutritional Status of Children Under Five Years of Age and Pregnant and Lactating Women Improved Government Health technical services Health & nutrition life skills prioritization & oversight Government Integrated health center Health & nutrition clinical services Government Hydraulic technical services Borehole water testing & oversight Government Rural Engineering technical services Borehole targeting, specifications & oversight L’Association d’appui aux ONG du sud (SongES)*+ SBC for health and nutrition life skills delivery Initiative de Sécurité des Conditions de Vie (ISCV) *+ SBC for health and nutrition life skills delivery Community radios* Health and nutrition SBC message delivery Pump repairmen Pump repair and maintenance Community led total sanitation (CLTS) masons Borehole apron & latrine construction SO 2: Access to Food by Vulnerable Households Increased Government Agricultural technical services Agriculture technical skills prioritization & oversight Government Livestock technical services Livestock technical oversight Government Environmental technical services Renewable resource regeneration oversight Asusu (Hausa term for “saving box”)* Warehouse receipts lending Institut national des recherches agronomiques du Niger (INRAN)* Farmer field school technical skills trainers LAHIA Midterm Evaluation Report INTRODUCTION 10 Table 3 LAHIA Implementing Partner List and Associated Organizations (with activities) NIGERIEN IMPLEMENTING PARTNERS LAHIA ACTVITIES Halal* Seed supplier & multipliers Improved cook stove artisans Fabricate improved cook stoves SO3: Vulnerability to Food Security Shocks Reduced Government Planning technical services Emergency response support system development Risk Assessment Team of Commune (OSV) Commune Risk Assessment Team SO4: Meaningful Participation by Women at the Household and Community Levels Improved Government Population technical services Gender equity prioritization Government Women’s promotion technical services Gender equity prioritization Note: Table 3 lists the Nigerien implementing partners for the LAHIA project and provides a brief description of the types of activities to which the partners contribute. * Indicates non-governmental organizations (NGOs) + Indicates organizations who have received funding as sub recipients from the LAHIA project. LAHIA Project Community Organizations The LAHIA project implements a large number of their activities though community-groups that they have adopted, strengthened, or created. Table 4 lists these groups and the activities with which they are active. Some groups contribute to achievement of more than one objective, such as the village development committees (VDCs). Table 4 LAHIA Community-Based Organizations COMMUNITY-BASED ORGANIZATIONS LAHIA ACTVITIES Project Administration Village chief Cooperation: information sharing and feedback Village Development Committee (VDC) Cooperation: implement activities in village SO1: Nutritional Status of Children Under Five Years of Age and Pregnant and Lactating Women Improved Food management committee Ration distribution oversight Water and sanitation, hygiene management committee Borehole use and maintenance management Communicator network Health and nutrition SBC delivery SO 2: Access to Food by Vulnerable Households Increased Natural Resources Management Committee (RNA) Natural resource promotion & activity management Women’s savings groups (MMD) Rotate money SO3: Vulnerability to Food Security Shocks Reduced VDC Warehouse receipts management Community early warning system/Emergency response (SCAP/RU) Food security surveillance and response SO4: Meaningful Participation by Women at the Household and Community Levels Improved Cross-cutting for all committees Gender Note: Table 4 lists the Nigerien community-based organizations of the LAHIA project and provides a brief description of the types of activities to which the organizations contribute LAHIA Midterm Evaluation Report INTRODUCTION 11 LAHIA uses community groups as a key sustainability strategy. Some of the groups they work with were already in existence at the start of the project, others were reinvigorated by the project, and others were created from scratch. The case varies by village and group. For example, there were remnants of CARE International’s revolving credit groups in the targeted communities. The government of Niger started village development, natural resource management, and emergency response committees. LAHIA’s community agricultural workers regrouped women and retrained them on how to save and provide credit to each other. LAHIA Midterm Evaluation Report OBJECTIVES OF THE MID-TERM EVALUATION 12 OBJECTIVES OF THE MID-TERM EVALUATION Per FFP guidance, the objective of a Midterm Evaluation (MTE) is to measure the progress from the start of a program. The Midterm Evaluation provides an opportunity to assess the effectiveness of the program delivery and approach. Specifically, the aim of an MTE11 is to: 1. Assess the strengths and challenges of project implementation and outputs in terms of meeting terms agreed with FFP and their acceptability and usefulness to the targeted communities, and to identify factors that appear to enhance or detract from the efficiency, quality, acceptability, and usefulness of the activities’ implementation and outputs. 2. Present evidence of changes (intended and unintended) associated with project activities and outputs, and identify factors in the implementation or context that impede or promote the observed and intended changes. 3. Recommend adjustments to the project design, resource allocation, or implementation that could improve the likelihood of achieving desired results by the project’s end – based on the findings from evaluation objectives above. As stated in the LAHIA MTE Scope of Work “The purpose of the midterm evaluation is to review the implementation processes of LAHIA program, describe the advancement of the LAHIA program to date , to identify whether the program is on track to achieving its goals, and provide recommendations on the strengths (to consolidate the achievements) and areas for improvement. The Midterm evaluation will help determine whether all the program objectives can be achieved for the rest of the program duration and make recommendations in this regard.” The MTE team evaluated the progress made by the LAHIA program in the attainment of program objectives, provided evidence on the effectiveness and shortcomings of LAHIA’s strategies and approaches, and made recommendations to improve future LAHIA programming. In addition, the MTE team discussed LAHIA’s ability to reach program targets and evaluate the quality of partnership with technical and implementation partners of the program. The full evaluation scope of work is contained in Appendix C: Scope of Work. This document contains the client’s language of the evaluation objectives, questions, terms and conditions, and consultant deliverables. 11 USAID/Food For Peace. (October 2014) Scope of Work for Mid-Term Evaluation of Food for Peace, Title II Development of Food Aid Projects. LAHIA Midterm Evaluation Report EVALUATION METHODS AND LIMITATIONS 13 EVALUATION METHODS AND LIMITATIONS Interpreting the Evaluation Questions The list of questions, from the mid-term evaluation scope of work, has been copied into Table 11. Questions are grouped into categories; a) general questions applied to all activities; b) specific questions relating to SO1, SO2, SO4, commodity management, and monitoring and evaluation (M&E); c) more general questions applied to all activities; and d) partnerships. The 126 evaluation questions are contained in Appendix G: Tables of Evaluation Questions . Questions are organized into tables corresponding to each sub-section of this chapter. Note that no evaluation questions were included in the SOW regarding SO3: vulnerability to shocks reduced, natural resource management under SO2 or about increased use of health center services under SO1. Implementation of Activities and Partnership sections includes some discussion regarding these activities. There is redundancy in many of the questions. Evaluation questions asking the same questions, using different words were answered collectively. For example, questions 118 through 120 use the terminology, constraints, challenges, and successes of failures to identify problems. Only one list of problem areas is included in the report, not three. Redundant questions remain on the list so the developers of the questions will know that the questions have been incorporated into the evaluation design by the MTE team. Compound questions have been cleaned up and separated into single answer questions. Compound questions were separated so they could be answered methodically. As an example take question 30. “Are the water points that have been constructed sustainable and appropriate?” This question was divided into two, Are water points that have been constructed appropriate? and Are water points that have been constructed sustainable?. The first part of the compound question relates to the on-going care and maintenance of water points after the project closes. The second part of this compound question asks whether the identification of community needs were correct before activities started, during the program design phase. Evaluation Team The evaluation team consisted of seven (7) independent consultants, three (3) key staff and four (4) assistants. The key evaluation positions are described in Table 5. The three key staff are Chaibou Dadi, Kyung Endres, and Aboubacar Souley. Ms. Endres is a foreign national with previous experience in Niger, conducting mixed methods evaluations, and working with food security programs. Mr. Souley is based in Niger and has evaluated maternal and child health and nutrition, WASH, family planning, behavior change, and gender projects. Mr. Dadi, also based in Niger. He is an expert in program evaluations, food security, natural resource management, income generating activities, and agriculture. Table 5 LAHIA Mid-Term Evaluation Key Staff NAME TITLE RESPONSIBILITIES12 Kyung Endres Team Leader  Lead the development of qualitative tools in technical area and coordinate cross-cutting themes  Lead the development of sampling and evaluation methods  Oversee the hiring of data collectors  Develop and facilitate training of data collectors  Lead the qualitative analysis 12 Save the Children US. (April 2015) Terms of Reference Midterm Evaluation LAHIA Program. LAHIA Midterm Evaluation Report EVALUATION METHODS AND LIMITATIONS 14  Responsible for the overall report draft (in English) and deliverables Chaibou Dadi Food security, livelihoods (FSL) and resilience consultant  Review the methods and provide feedback on data collection and schedule  Lead development of qualitative tool in French for each technical area  Review tools for local adaptation  Provide oversight on writing  Conduct analysis on key themes  Provide writing support as needed in deliverable Aboubacar Souley Health and nutrition consultant Assistants 2 FSL 2 health/nutrition  Support International and National consultants  Qualitative data collection, including efficient note taking Note: Table 5 is a list of the evaluation team roles and responsibilities as presented in the terms of reference for the mid-term evaluation and subsequent conversations between SC headquarters and the lead consultant. The Nigerien team was finalized by SC while the lead consultant was in Niger. After traveling to Maradi the three key staff spent two days finalizing the sampling framework and data collection tools. The research assistants were all present the day before data collection began. The key staff presented roles and responsibilities to the assistants. Roles were to facilitate qualitative interviews, take notes of discussions, and share findings and impressions with team. Responsibilities included informed consent with all respondents, conducting qualitative interviews in LAHIA communities, typing interview notes, presenting analyses of each interview in rolling debrief form, submitting all written notes, informed consent signatures, and rolling debriefs to the lead consultant. We reviewed the program activities, MTE the scope of work, and discussion guides. Approximately one hour was spent on facilitation techniques as all team members reported to have had significant interviewing experience. The lead consultant conducted all interviews with LAHIA staff, partners, and government officials based in Maradi. All of these interviews were in French. The two Nigerien key evaluators held interviews with government officials at the commune level in French, Hausa or Djarma. Community interviews were conducted in Hausa and French by the entire evaluation team. Each evaluator conducted an average of six interviews per day. Two communities were visited each day by the entire team. Evaluation Methods Literature Review The evaluation team reviewed project documents, planning documents, beneficiary databases, and related government policy documents. The literature review provided the evaluation team with context, in order to design the sampling framework and develop discussion guides. Included was the approved project proposal, quarterly and annual reports, assessments, meeting notes, strategies and partner agreements. The evaluation team used the IPTT and project reports to answer the evaluation questions asking about progress to date. Qualitative Interviews The evaluation team conducted interviews with LAHIA project staff, government representatives, implementing partners, community groups, direct beneficiaries, and non-beneficiaries. A full list of respondents can be found in Appendix D: List of Respondents. The sampling framework ensured that women were represented to observe any gender-differentiated impacts of the project, especially regarding agricultural productivity and empowerment. Respondents fall into five categories: LAHIA Midterm Evaluation Report EVALUATION METHODS AND LIMITATIONS 15 1. Regional, District and Commune government officials. Government officials were selected as key informants as implementing partners. The evaluation team understands that some officials acted in an oversight capacity and others directly contributed to the project. 2. Community leaders who are intimately familiar with the project as managers or activities and have regular contact with the project participants. 3. Prime project implementers (SC & WV) who planned, implemented and managed all project activities. 4. Service delivery partners such as health centers, community management groups, and non￾governmental partners. 5. Participating community members namely, project beneficiaries, women who were pregnant or breastfeeding, caretakers of children under five, household members who participated in activities. Seven (7) qualitative evaluators facilitated focus group discussions and key informant interviews with key project stakeholders and beneficiaries. The discussion guides are contained in Appendix E: LAHIA Mid￾Term Evaluation Plan. For focus group discussions, when possible, one evaluator facilitated the discussion while the second took notes. In three instances focus group discussions was managed by only one person who facilitated and took notes simultaneously. For key informant interviews, one evaluator facilitated and simultaneously took notes. Evaluators recorded conversations either using paper and wrote their notes up using word processing software. All qualitative interviews employed open-ended questions to elicit undirected responses. Interviewers were instructed to allow the respondent to speak 90% of the time. The qualitative evaluators met three times during the course of data collection to exchange unexpected findings and topics needing additional information. Questions that had been thoroughly answered were discarded, the evaluation team added new questions to the discussion guides to answer evaluation questions more thoroughly. Interviews recorded successes, failures, and lessons learned from the LAHIA approach to project implementation. The evaluation team asked questions about women’s participation in project activities, what their major interests in the project were and how the project affected them in terms of capacity building, self-esteem, and income generation. Each interview started with the evaluator(s) introducing themselves and reading the informed consent form, in Appendix E: LAHIA Mid-Term Evaluation Plan. If the respondent agreed, the interviewer signed the consent form, respondents signed the sign in sheet, and the documents were exchanged. Evaluation Processes Site Selection Criteria and Process Niger’s administrative boundaries are delineated into Regions, Departments, Communes and Villages. LAHIA is active in 72 villages. SC works in a total of 45 villages, 19 in Aguié, 12 in Gangara, and 14 in Tchadoua. WV works in a total of 27 villages, 10 in Guidan Sori and 17 in Saé Saboua. The LAHIA monitoring and evaluation team selected two villages from each of the five project communes. To select villages, first those made inaccessible by the rains were not considered. Secondly villages with a variety of LAHIA water, sanitation, hygiene, health, nutrition, agricultural, income generation, and resilience activities were selected. The final list of villages included in the evaluation is in Table 6. The evaluation team visited all ten villages as planned in Table 6. Not all 72 LAHIA communities need to be visited by the qualitative team as new responses decrease after a few interviews with the same respondent type and in ethnographic research it is standard practice to discontinue interviewing LAHIA Midterm Evaluation Report EVALUATION METHODS AND LIMITATIONS 16 respondents of the same type when answers become redundant. A full calendar of evaluation activities is in Appendix F: Evaluation Calendar. Table 6 LAHIA Mid-Term Evaluation List of Villages CODE DEPARTMENT COMMUNE VILLAGE NUMBER OF LAHIA ACTIVITIES IMPELEMENTED (OF 31) APPROXIMATE DISTANCE FROM MARADI TOWN (KM) AG1 Aguié Aguié Naki Karfi 14 120 AG2 Aguié Aguié Guidan Tango 18 100 GA1 Aguié Gangara Malam Daweye Na Hantsi 17 150 GA2 Aguié Gangara Guidan Kiata 17 115 TC2 Madarounfa Tchadoua Farga 17 60 TC3 Madarounfa Tchadoua Algafia 15 65 GS1 Guidan Roumdji Guidan Sori Koumboula 24 22 GS2 Guidan Roumdji Guidan Sori Guidan Bagwari 21 59 SS1 Guidan Roumdji Saé Saboua Magaji Arzitaou 17 28 SS2 Guidan Roumdji Saé Saboua Takalahia 18 22 Note: Table 6 is the list of villages where primary qualitative data were collected. WV works in the Communes of Guidan Sori and Saé Saboua. SC works in the other three communes. Maradi is the regional capital and the base of operations for the project. The distance from the regional capital to the village is shown in kilometers (KMs). Respondent Selection Criteria and Process The ten villages included in the evaluation have a total population of approximately 23,300 people. The project estimates that 70% of villagers are beneficiaries, or 16,300 people. The evaluation team interviewed 85 key informants and 39 focus groups. Key informant interviews were with individuals. Each focus group included between two and 20, with an average of five participants, for a total of just under 200 focus group participants. All interviews were planned to last less than 60 minutes. The research assistants’ interviews on the first day took up to 90 minutes. During the evaluation team’s debrief we stressed the importance of concise interviews and discussed strategies for quality control and time management. Qualitative respondents were selected based on their association with the project. The types and number of planned respondents is broken down in Table 7. Table 7 Number of Respondents by Type of Respondent KEY INFORMANT INTERVIEWS FOCUS GROUP INTERVIEWS TYPE NUMBER TYPE NUMBER Community Leaders 10 Community Management Groups 4 Individual Beneficiaries 42 Beneficiary Groups Health & Nutrition 14 Suppliers 4 Beneficiary Groups Livelihoods 10 Nigerien Implementing Partners 4 Suppliers 1 LAHIA Staff 7 Government Stakeholders 18 TOTAL 85 TOTAL 39 Note: The evaluation team interviewed 85 individuals and 39 groups during the course of primary data collection. LAHIA Midterm Evaluation Report EVALUATION METHODS AND LIMITATIONS 17 SC and WV assisted with respondent recruitment. The projects made primary contact with community leaders, implementing partners and government officials to facilitate scheduling meetings. In each village, the evaluation team met with LAHIA stakeholders to identify the individuals based on desired characteristics such as women who received rations. Key staff persons of government institutions and implementing partners were contacted and appointments made to meet in their offices. Evaluation Plan The introductory material included in this report was originally introduced in the Evaluation Plan. Some of the information has been updated in the Evaluation Report to more accurately describe the project. The Evaluation Plan is included in Appendix E: LAHIA Mid-Term Evaluation Plan. The biggest difference between the submitted plan and the information contained in this report is that the Evaluation Conceptual Framework is not being used. Instead of applying an evaluation framework, Save the Children requested that evaluation findings be organized by activity area. Evaluation Limitations 1. Identifying and contracting qualified international and local consultants was very challenging. The international consultant was contracted on July 27th. Save the Children shared three rounds of resumes with the international consultant for review given that there were challenges with commitments on the part of local consultants identified by SC. They started working on September 4th. The hiring delays are a limitation to the evaluation in that the local consultants did not have time to read the background documents or substantially participate in the evaluation design. 2. The scope of work for the evaluation was overly ambitious given the level of effort and time allocated to conducting the evaluation. Including over 120 questions diverges from the USAID Automated Directives System (ADS) 203 which states, “Identify a small number of key questions and specific issues answerable with empirical evidence.” – ADS 203, v2004 & v2010 In their evaluation training of USAID staff and contractors, USAID promotes including three to four evaluation questions. The TIPS worksheet number 3 talks about how to prepare an evaluation scope of work (SOW). They warn specifically against including too many evaluation questions. “One of the most common problems with evaluation SOWs is that they contain a long list of poorly defined or “difficult to answer” questions given the time, budget and resources provided. While a participatory process ensures wide ranging input into the initial list of questions, it is equally important to reduce this list to a manageable number of key questions. Keeping in mind the relationship between budget, time, and expertise needed, every potential question should be thoughtfully examined by asking a number of questions.” - USAID TIPS Number 3, 2010 The consultant strongly encouraged Save the Children to focus their evaluation questions on what would be essential advice for the program out-years. The SOW document was unchanged; LAHIA Midterm Evaluation Report EVALUATION METHODS AND LIMITATIONS 18 however, SC and the consultant had verbal conversations regarding the consolidation of questions. Fewer questions allow evaluation teams to investigate root causes of successes and challenges. Responding to over 100 questions meant that the evaluators were only able to answer the questions but not underlying causes and effect. 3. The amount of time dedicated to the evaluation was uncharacteristically short. This limited the quality of team planning (4 days), data collector training (1 day), data collection (6 days), and data analyses (2 days). The consultant expressed concerns about the team’s ability to produce quality work in the given timeframe. Save the Children extended the consultant’s scope of work by two weeks to include report writing time. 4. Communities made inaccessible by the rains were not included in the sampling plan. This follows guidance given by USAID/FFP and Save the Children to “reduce travel time, and maximize time engaging with program participants”. As a result, the sampling plan excludes more remote villages and may introduce a sampling bias due to levels of contact with the LAHIA project due to their isolation. 5. Despite being included in the consultant’s SOW, no comparison group of non-beneficiaries was included in the sampling plan. This was due to limited amount of time in Niger. 6. The required report outline, as contained within the scope of work did not include a discussion section. It is standard in any type of research to present findings and then discuss the implications of the findings. The consultant has therefore included some discussion in the ACTIVITY DESCRIPTION, FINDINGS AND DISCUSSION section. LAHIA Midterm Evaluation Report ACTIVITY DESCRIPTION, FINDINGS AND DISCUSSION 19 ACTIVITY DESCRIPTION, FINDINGS AND DISCUSSION The ACTIVITY DESCRIPTION, FINDINGS AND DISCUSSION chapter of this report answers the evaluation questions using the same organization and order as in the MTE SOW. Each sub-section of this chapter contain brief descriptions of project activities, evaluation findings, discussion of the findings, and recommendations. Overall Program The SOW contained key evaluation questions under the heading, “Overall”. They are found in Table 12. Ten questions applied to all project activities and three questions were specific to value chains, natural resource management (NRM), and livestock activities. The first 10 questions have been answered in subsequent sections as they are easier to understand in context with other similar questions. They are therefore not discussed again in this section. Description: The LAHIA project works with farmers in cowpea, millet, goats, moringa, vegetables, and seed multiplication as their main value chains. LAHIA field agents encourage farmers to use NRM techniques to increase soil fertility in fields and conserve grazing lands for goats. They grant goats to groups of women with the understanding that goat recipients will reimburse the project with a kid to share with another woman. Findings: The Resilience and Economic Growth in the Sahel – Accelerated Growth (REGIS AG) program is slated to increase household incomes by developing the cowpea and small livestock value chains in Maradi . The LAHIA project has already begun discussions with REGIS AG project to transfer business skills to warehouse receipts and women’s goat rotation groups. Discussion: There may be many possibilities for learning and potential alignment. The evaluation questions ask how the LAHIA project could align with the resilience projects. It would make more sense for the new projects to align to the existing projects. The specific adjustments can only be determined after the REGIS-AG and Resilience in Sahel Enhanced (RISE) program has solidified their activities and thoroughly shared with others. Recommendations: Some ideas for collaboration might be that food for work can be used to rehabilitate REGIS AG or other partner sites, as was successfully completed under LAHIA, to remove invasive species and sow animal fodder. Additionally, the project could share para vet training materials and conduct joint training sessions. REGIS AG para vets could shadow those already trained by LAHIA. Technical Approach by Sector and Cross-Cutting Agriculture and Livelihoods The evaluation questions relating to agricultural productivity and livelihoods are in Table 13 and Table 14, respectively. Five questions asked about the acceptance and uptake of project-promoted agricultural technologies and techniques. Four evaluation questions asked about income generating activities. Description: The agriculture and livelihoods activities boost the availability of home grown food and access to purchased food by producing more and raising disposable incomes, respectively. By combining agribusiness incomes and growing vegetables with nutritious feeding knowledge learnt in SO1 activities, the LAHIA project is arming beneficiaries with the tools to be able to make and financially support the healthiest eating choices. LAHIA agricultural extension agents organize community members into topical groups, or business units. Participation in any group is voluntary. All business activities, except for the warehouse receipts program, are completely community owned. The warehouse receipts program is overseen by the village development committee (VDC), a government-promoted institution. Communities determine which LAHIA Midterm Evaluation Report ACTIVITY DESCRIPTION, FINDINGS AND DISCUSSION 20 groups interest them. If interest is great in a certain thematic area, one community may have multiple groups of the same type. For example, the project works in 72 communities but reported working with 112 unique producer organizations and 200 different MMD groups at the end of the second fiscal year. The main groups that have been created under the rubric of agriculture and livelihoods are bulleted below. The women’s MMD savings group may or may not have an agricultural component to them. All of the other groups center on crop or animal agricultural value chains.  Rainy season millet farmer field schools  Rainy season cowpea farmer field schools  Dry season gardening farmer field schools  Moringa planting groups  Warehouse receipts groups (SO3 activity but also contributes to achieving SO2)  Women’s goat rotation groups  Women’s savings groups (MMD) Members of agriculture and livelihoods groups may also be members of health and nutrition groups. As intended, it is not uncommon to find that a behavior change peer educator is also a member of a farmer field school. Two groups, as noted in the list above, allow only female members. The others may have male and female members within the same business unit. LAHIA field agents and implementer, INRAN, teach farmers agricultural growing techniques through farmer field schools (FFS). FFS participants were granted improved seeds for their communal test plots. These improved seeds were made available through the HALAL distribution system. Some FFS farmers were HALAL distributors. Moringa is promoted by organizing nurseries to grow seedlings and sell to individual households who are taught how to plant and care for them in their concessions. The project also promotes consuming moringa leaves for their nutritional value. The warehouse receipts program linked cereal banks to credit. In order to receive the initial disbursement from the microfinance institution, community members needed to agree to training and oversight by the project. Communities build or rehabilitate storage structures and create cereal bank groups. The LAHIA project guarantees loans from Asusu, a Nigerien private-sector credit provider, for cereal bank groups. Cereal bank members are to use the loan money for entrepreneurial activities while the harvest is stored until market prices rise and the harvest can be sold. The cereal bank members must repay the loan, with interest, before Asusu releases the stored goods. After repaying the loan, farmers can access their commodities and then use them as they see fit. Groups are encouraged by the project to include male and female members. The two types of LAHIA project groups with exclusively female members are 1) women’s savings groups, or MMD’s, and 2) goat rotation groups. The MMD’s are a circle of trusted friends who create a group with the intention of saving money together and rotating the pot of money to different members. Members are expected to repay the MMD loan and when another group member needs the money, members determine if they will give her the loan. Each group creates their own operating procedures, rules, and means of enforcement. The idea behind MMD’s is that a larger sum of money at one time will give its members greater purchasing power for more significant business activities. For the goat rotation activity, the project identifies the most vulnerable from amongst the women who receive rations to participate in a goat rotation group. Goat rotation may be an income generating activity of some MMDs. The project purchases goats in the local markets, and government technical services inspect and certify the goats before they are distributed to women. LAHIA grants some women goats with the understanding that their offspring will be given to another member of her group. The project trained a cadre of women, five 15-20 year old and seven 21-49 year old women as para￾veterinarians to ensure the health of the goats and equipped them to provide para-veterinary services to their surrounding community. LAHIA agriculture and livelihoods activities seek to smooth food consumption by increasing long-term ability of beneficiaries to better absorb agricultural production and economic shocks. Given this LAHIA Midterm Evaluation Report ACTIVITY DESCRIPTION, FINDINGS AND DISCUSSION 21 justification, these activities could also be categorized as resilience activities. Like the other livelihoods activities, warehouse receipts can fall under both SOs. It is officially categorized by LAHIA under the resilience SO. As the business components are the same as the other livelihoods activities, warehouse receipts will be discussed in the report as a livelihood activity. Findings: Farmers reported learning about bio-pesticides, improved seeds, and planting techniques for beans through farmer field schools. They said the techniques promoted by LAHIA are too labor intensive and are applied to the test plots only. Of the approximately 100 beneficiaries interviewed who participated in farmer field schools, only one understood the purposes of test plots and the difference between a test plot and a regular field. The technologies and practices promoted through the farmer field schools were deemed inappropriate by all of the focus groups. Trainings were viewed by farmers as pedagogical versus practical. The quote below is a gentle example of how most farmers felt about the learning. Q: Quelles sont les innovations promues par le projet que vous maîtrisez ? R: Je maîtrise la nutrition des enfants, mais les techniques agricoles ça nous reste encore car pour certaines, nous avons vu et nous imitons et donc, il y a beaucoup de lacunes. Nous espérons des formations pour maîtriser tout. Q: What LAHIA innovations do you use? R: I follow child nutrition, but the agricultural techniques are still missing because for some, we saw and we imitated and, so, there are a lot of shortcomings. We want training so we can master all. - Male farmer field school participant Farmers wanted concrete solutions which address the expressed concerns of shrinking plot size and declining soil fertility. Some seed varieties, such as those requiring a shorter growing season, were viewed favorably. Farmers reported that the project promoted seed varieties were not available in the local marketplace. Planting multiple seeds per hole and then reducing the number of shoots was also popular. Students who participated in farmer field schools reported not applying project-promoted techniques to their fields such as measuring and cordoning plot borders, and did not intend to. Farmers were not convinced of the efficacy of bio-pesticides or organic fertilizers and asked for chemical pesticides and fertilizers. Farmers felt that the local trainers were unqualified. According to INRAN, farming students asked the lead farmers highly technical questions that college graduates might be expected to know, such as the life cycle of certain pests. It should be noted that lead farmers had a greater appreciation for the techniques promoted by INRAN, a LAHIA technical assistance provider, than farming students. Of the 100 farmers interviewed, none mentioned value chain promotion. Women and men spoke of learning to prepare corn-soya blend rations, but not value addition, or promotion of key LAHIA value chain in cowpeas. Processed cowpeas were identified as having high income potential for women in the SC and WV value chain analysis13. People were already growing cowpeas but there was room for production techniques to improve. Cowpeas were grown in farmer field school plots using new seed varieties promoted by LAHIA. Improved seeds were reported as unavailable despite HALAL’s distribution efforts. One women’s group had received a mill to make cowpea flour and many more asked for one. No marketing or distribution channels were discussed. 13 CESAO PRN (March 2014) Rapport Étude des Chaines de Valeurs Niebe dans la Zone d’Intervention du Programme LAHIA. [Translation: Study Report on Cowpea Value Chains in the LAHIA Program Area]. LAHIA Midterm Evaluation Report ACTIVITY DESCRIPTION, FINDINGS AND DISCUSSION 22 Moringa installations were planned for in 30 villages. Evaluators spoke with only two households who planted moringa in their compounds through the LAHIA project. One was very positive and the other thought caring for the plants and harvesting the leaves was too much work. Findings are inconclusive. Project beneficiaries voiced several concerns with the warehouse receipts activities; a) the crops warehoused are male-dominated; b) the warehouses are controlled by village development committees leaving little room for women’s voices; c) the warehousing facilities are of poor quality; and d) the conditions for loan repayment were not aligned with market realities. The most loudly voiced issue was the inability of the group to repay their loan. The terms of the agreement with ASUSU were that groups would store their commodities for six month, receive a loan, use the loan in profitable business ventures, repay the loan, and then have ASUSU would unlock the warehouse and group members could retrieve their goods. Warehouse groups reported borrowing money to repay the loan, defeating the purpose of a project-sponsored loan. The warehouse groups felt they should be able to store the commodities, use the loan for living expenses, sell the commodities at a profit and then repay the loan. The only small grants for women that were universally mentioned were goat donations. Some women were given one goat and expected it to reproduce another for rotation back into the pool. Women intended to share offspring with other group members but had not yet earned money from the goats. According to the SC second quarterly report of fiscal year 2015, 1,080 goats were distributed and 85 were recirculated to other group members. Government officials, charged with assessing animal health, said the project purchased and distributed sick animals after inspectors refused to certify the animals for donation. They also said that the LAHIA project’s standard of providing one animal per woman was outside of the government’s recommendations for three nanny goats and one billy goat per beneficiary. The Ministry of Agriculture and Animal Husbandry technical services was a strong supporter of the para￾vet training program and encouraged the project to send them for government training in Maradi. Evaluators spoke held two focus group discussions with MMD groups but held key informant interviews with individuals who were MMD members. Project participants were very eager to participate in LAHIA’s income generating activities. They listed making and selling prepared foods as a way to make money. MMD members were active in health and agriculture activities. Groups that had not received direct grants from the project were asking for inputs to start their businesses. Across the board, women wanted free animals and access to credit. When asked for recommendations, typical responses are as follows. LAHIA staff stated that some MMD groups had accessed credit with Asusu independently of the project after having been introduced to the credit facility through the warehouse receipts activity. This is evidenced by the two excerpts below: "Les recommandations sont: Les femmes ont besoin des petits ruminants pour l’élevage, des subventions pour exercer les activités génératrices de revenus, une pompe/forage, une machine à moudre, les cours d’alphabétisation. Toutes ces activités nous les voulons pour notre groupe. " "The recommendations are: Women need small livestock to raise, a grant to start income generating activities, a borehole with pump, a mill, literacy classes. All these activities we want for our group.” - Women’s savings group focus group discussion Discussion: Disconnects between the lead farmers and the students were telling. Teaching is a skill that few innately possess. The farming trainers struggle to pass on information so their students learn as much as they did. INRAN proposed some possible reasons why farmers were resistant to the 19 lessons they provided;  No fact sheets accompanied lessons,  No reference materials in local languages,  There are not enough field agents. They cited 1 field agent per 50 villages, LAHIA Midterm Evaluation Report ACTIVITY DESCRIPTION, FINDINGS AND DISCUSSION 23  Farming students have questions outside of the 19 training topics that local trainers are unprepared to answer,  Rapidly producing millet = immediate adoption  2-3 years for farmers to see the results of bio-pesticides and adopt The mistrust of the agricultural field agents’ knowledge base was palpable during focus group discussions with farmer field school members. Their thirst for knowledge was not satiated. Farmers stated that the information was impractical to apply, meaning it was hard work and they did not see the benefits of messages other than improved seeds. This response could mean that the information is not useful to these farmers, the information was not well packaged for the trainers, or transfer from the trainers to the farmers was poorly executed. Any one of these reasons may be a consequence of contracting a research institution to train farmers. One of the challenges the project faces is that there are too many value chain activities to manage all successfully with current staffing levels. Value chain activities being promoted include; millet, cowpeas, vegetables, moringa, goats, and anything else beneficiaries suggest. This large number of value chains also has negative repercussion on the project’s ability to strengthen the input supply chain. The HALAL distribution model is community-based. They are very satisfied with their LAHIA distributors and expressed the desire for expansion. This is an opportunity for the project to strengthen the availability of improved seed varieties in local market, to increase their uptake, and to build the livelihoods of distributors. The value-chain approach that LAHIA is promoting is unclear from the activity strategy papers.14, 15, 16 A value chain approach is to “produce value added products or services for a market, by transforming resources and by the use of infrastructures – within the opportunities and constraints of its institutional environment.”17 In order for a project to promote a specific value chain, the chain should be defined and the constraints identified for each link in the chain. The cowpea study recommended activities all along the value chain from production to sales. The project strategies included procurement production activities which the project has done. The strategies and activities stopped short of post-procurement and post-harvest handling. This should be what is in the strategy, solutions. FAO first introduced a warehouse receipts program to Niger in 1999.18 For the past 25 years development organizations have been promoting the endeavor as a best practice. Farmers have not widely adopted the practice. A project should then ask itself, why farmers do not all participate. The literature describes the “critical failure factors” in warehouse receipts programs as the lack of legislation governing the valuation of movable collateral, poor quality warehouses (infrastructure and management), and decreased profitability due to the leveling out of market prices due to regular supply of the stored commodity.19 14 Save the Children & World Vision. (2014) Document de la Stratégie. Mise en Œuvre de 72 Champs Ecoles Pluviales FY14. [Translation: Strategy Document. Implementation of 72 Rainy Season Field Schools Fiscal Year 2014]. 15 Save the Children & World Vision. (2014) Stratégie de Promotion du Petit Élevage au Profit des Ménages Pauvres. [Translation: Small Animal Husbandry Promotion Strategy for Poor Households.] 16 Save the Children & World Vision. (2014) Stratégie de Mise en Œuvre de Promotion du Moringa. [Translation: Implementation Strategy for the Promotion of Moringa]. 17 Trienekens, Jacques. “Agricultural Value Chains in Developing Countries A Framework for Analysis.” International Food and Agribusiness Management Review Volume 14, Issue 2 (2011). 18 Wampfler, Betty. Le développement du Warrantage au Niger: Communication du projet Intrants FAO au colloque Farm, Paris. Décembre 2007. 19 International Fund for Agricultural Development. The Rural Finance Knowledge Management Partnership. (2013) Warehouse Receipt System: Providing Collateral and Maximizing Value. Nairobi, Kenya. LAHIA Midterm Evaluation Report ACTIVITY DESCRIPTION, FINDINGS AND DISCUSSION 24 Specific to animal husbandry, the activity was admired by beneficiaries as a source of insurance against destitution and a sign of prosperity. This activity merits more attention. The Government of Niger recommends that rotating animal schemes start beneficiaries with three nanny goats and one billy goat based on past experience. Animals die and pregnancy is difficult to achieve when ownership is relegated to one lone nanny goat. Training para-vets, an activity under LAHIA, was a good way for more owners to access animal health services. If the para-vets are not incorporated into the government support system, the project will need to create a parallel system so practitioners can refresh skills, equipment, and supplies. Recommendations: The income generating activities selected should be promoted with MMD groups and the project should consider facilitating financing with a small business lender, such as is done with the warehouse receipts project. MMDs need business skills and the evaluation team encourages the expansion of the functional business literacy classes for these groups. The sub-contract for agricultural extension training to INRAN should be reconsidered. The project should include adult learning methods in their agricultural training of trainers sessions, not just training on technical content. To improve service quality, the project should redesign its staffing strategy along with the changes to extension services. Ensure proper training of staff or sub-contract these services to an organization specializing in meeting the needs of farmers. This may be an opening for the LAHIA project to engage more closely with other USAID projects. The project should choose one or two of the strongest strategies for building livelihoods and integrating into value chains for sustainability and move forward with focus and clear implementation parameters. After selecting the focal businesses the project should define the skills that all business people should have to be successful. LAHIA field agents should be trained to pass these skills on to entrepreneurs, customizing them to the context of the LAHIA communities. Once objectives and the skills have been set, the project can create beneficiary inclusion criteria to identify those households that are ready for market integration. If the project feels that time and budget planning are pre-requisites to being a successful business owner, then people should be trained in functional business literacy. For example, if the project selects goat husbandry as one of their value chains, the project should conduct a market analysis to understand market demand (for ensuring proper focus on production without potentially oversupplying the marketplace and impacting sales prices negatively). It should then consider donating three nanny goats and one billy goat to each beneficiary as well as training additional para-vets and developing a supportive network for these para-vets. LAHIA should consider investing more resources on land reclamation to ensure a supply of food for the goats. Furthermore, empowering women to treat their goats as an investment and training them to manage them as a business would encourage sustainability and profitability. Male partners must also be included in this activity; this is the perfect opportunity to concretely influence gender relations within households. The project can hold role playing sessions with goat owners and their spouses to tackle difficult issues about ownership and revenue. Microfinance is used to promote entrepreneurship. Running a business requires business skills and an understanding of the market. The only way to know the market is to conduct a thorough market analysis for the targeted value chains. Lead farmers should be involved so they can internalize the data and develop a knowledge base from which they can conduct similar research for other value chains. While designing the analysis, keep in mind that it should be feasible for a rural farmer to conduct. This is no small task and requires the guidance from experts in the field. Business groups should work with field agents to develop business plans which include market analyses and budgets. If the warehouse receipts activity continues, pre-requisites must be set for participation. Either include only business people who can repay the loan before selling their produce or renegotiate credit terms with ASUSU to allow farmers to sell their crop before repaying the loan. Either way, the project LAHIA Midterm Evaluation Report ACTIVITY DESCRIPTION, FINDINGS AND DISCUSSION 25 should discuss terms with ASUSU to extend the loan period past six months to ensure better returns. The project will also need to invest in physical infrastructure to measure and store dry goods. Health, Nutrition, Water Sanitation and Hygiene The evaluation question relating to women and children’s rations is in Table 15. The six evaluation questions relating to social and behavior change messages are contained in Table 16. The two evaluation questions relating to community led total sanitation (CLTS) are contained in Table 17. The four evaluation questions relating to water and sanitation, hygiene (WASH) are contained in Table 18. Description: The activities for improving the nutritional status of women and children are to; a) distribute rations; b) provide communities with consistent and targeted messages from multiple sources; c) promote hygiene; and d) provide access to clean drinking water and health care. The project provides rations to PLW if they are registered at the health center as having had three prenatal visits or have children less than two years old. Rations were also used as food for work to rehabilitate public lands for grazing and as supplemental foods to PLW households during the lean seasons. LAHIA health field agents, local NGO SONGES, and health centers educate women and men to care for their health and the health of their children. SONGES trains male and female peer educators to provide health, nutrition and hygiene messages during ration distributions, community gatherings, and home visits. Messages may include;  Description of LAHIA project;  Objectives of providing rations;  Conditions of receiving rations;  Ration cooking demonstrations and storage;  Personal, child, home, and food hygiene;  Post-natal care for child and mother;  Exclusive breastfeeding for six months;  Complementary feeding after six months; or  Testing children for malnutrition (upper arm circumference).20 In the last year, of implementation, the project started new SBC outreach activities; a community radio public service announcement and call in campaign, a “network of communicators”, and community￾based video component (working with the USAID-funded SPRING project and the NGO Digital Green). The SBC campaigns also included one-off events to promote healthy lifestyles. The activities are implemented via groups, which are identified by the program by functions (positive deviance, peer educator groups) and SBC health messages are transmitted to groups. This approach allows for larger reach, rather than individual contact. The WASH infrastructure activities address the lack of physical infrastructure as a barrier to potable water and sanitary environments. The construction or repair of wells, boreholes, pumps, water towers and public latrines target entire villages or public localities, such as schools. WV invested heavily in gathering environmental and social evidence for thorough planning and execution. They also trained local drillers. The promotion of household latrines targets individuals and their households. World Vision plans and manages all aspects of water system construction in partnership with the Ministries of Hydraulics and of Rural Engineering. The LAHIA project pays for all construction equipment and materials. A 20 Save the Children and World Vision (no date) «Les Activités Santé/Nutrition Mises en Œuvres sur les Sites de Distribution des Vivres». [Translation: Health/Nutrition Food Distribution Implementation Plan]. LAHIA Midterm Evaluation Report ACTIVITY DESCRIPTION, FINDINGS AND DISCUSSION 26 representative of Rural Engineering is present during construction and Hydraulics tests water quality upon completion of the structures. Government officials participate in inaugural ceremonies. World Vision trains a community-based water management committee and repair persons to maintain the water system. Rations Findings: Nearly all respondents, whether they received rations or not, understood and agreed with how food rations were distributed. When asked about the timing of ration delivery, they said the food came when scheduled. There was one who suggested providing rations to households unable to feed themselves such as the elderly, handicapped, and very poor. Everyone loved the rations. People observed that more, better quality, and consistent food resulted in better health. Beneficiaries reported that due to the rations and hygiene education that children did not get sick, as they had in the past. People understood that food was provided as payment for non￾compulsory work and felt this was appropriate. The only complaint about the rations was that some women who met the selection criteria were not selected as beneficiaries. People understood the selection criteria very well. They described that Women who met the criteria are put on a beneficiary ration list by health center staff. Co-wives and husbands explained that when a pre-determined number of women had been registered for ration distributions, the remaining women were turned away from the health facility. These same people explained that the wives were reported to have met the criteria of being pregnant or breastfeeding a child less than two years of age. Feelings were strong as in the quote below. “Il y a également une énorme défaillance au niveau de l’élaboration de la liste des bénéficiaires, il y a vraiment trop d’omissions." "There is an enormous failure in the creation of beneficiary lists. There are truly too many omissions.” - Female ration beneficiary In mid-2014, the LAHIA program identified the issue of non-inclusion and began making adjustments in targeting. Project staff said that should have been no set number of PLW ration beneficiaries. The fiscal year 2015 Pipeline and Resource Estimate Proposal (PREP) reflected this. LAHIA staff said that women were turned away only in cases where they did not meet the inclusion criteria. Counterfeited cards were identified by the LAHIA M&E staff and women with these cards not accepted into the program. In all cases of potential fraud, women were advised of the proper channels. Discussion: After potable water activities, the rations were the second most well regarded LAHIA activity. The selection criteria for ration beneficiaries are well delineated making inclusion and graduation well understood and accepted by all. There is an indication that some pregnant women and children under five are not receiving rations even if they meet the criteria. The LAHIIA staff is working to remedy the situation. This is an issue the village management committees in charge of rations should be reporting to the project and LAHIA should continue to address. Recommendations: Senior management of LAHIA should continue to investigate where there are still issues of non-inclusion of eligible beneficiaries. Village management committees should be included in the investigation and the solution. Reassess information flows and decision making processes and parameters between LAHIA and community-based management organizations. Make sure the communications processes between the village-based committees and the LAHIA staff are understood and feasible. The village committees should be sending reports to LAHIA after ration distributions. A problem section could be included in this regular report. LAHIA Midterm Evaluation Report ACTIVITY DESCRIPTION, FINDINGS AND DISCUSSION 27 Health and Nutrition Messages Findings: When asked where they heard health messages, beneficiaries listed peer educators and LAHIA field agents as the number one sources. Respondents reported hearing health messages on the topics contained in Table 8. Table 8 Subjects Respondents Reported Learning through LAHIA Extension Agents REPORTED HEALTH EDUCATION TOPICS  Family planning  Birth spacing  Antenatal care  Clinician-assisted births  Post-natal care  Exclusive breastfeeding  Child health and nutrition  Recipes  Personal hygiene  Household cleanliness  Community sanitation  Potable water Note: During interviews with community members, evaluators asked in which LAHIA activities they participated and what they learned from LAHIA. This is a comprehensive list of technical and life skills topics, grouped by subject area. For nearly all of the female respondents the health and nutrition messages were ones they had heard before. The same messages had been heard at health centers and through other projects but they were not adopted. More males said that the health messages were new for them. This quote shows that the outreach strategy is reaching people and doing so in a way that may be changing their behavior. “Bien avant l’arrivée de LAHIA, nous avons déjà entendu parler de ces pratiques au centre de santé mais ceux que nous n’accordons pas beaucoup d’importance certes on les pratique mais pas comme avec la venue de LAHIA car avec les sensibilisations et les contrôles qui se font lors des visites à domicile nous mettons plus d’attention pour les appliqués." “Long before LAHIA came we had already heard talk of these practices at the health center. But, we didn’t pay attention like with LAHIA. The awareness raising and verification that they do in the home causes us to pay more attention and use what we have learned.” - Female ration beneficiary & farmer field school participant The benefits of health messages were evident to project beneficiaries with regards to reduced childhood morbidity and more prenatal visits to the health center for pregnant women. When asked by evaluators, project beneficiaries expressed interest in learning more. Project beneficiaries trusted that LAHIA field agents were acting in the best interest of the community. Below is a typical exchange with regards to the benefits of LAHIA messages. Q: Est-ce que quelque chose a changé pour les situations des femmes et des enfants avec LAHIA? R: Beaucoup a changé, Toutes les femmes se font consulter et le taux de malnutrition des enfants a chuté. Les maladies ont diminué aussi, naissances espacées… Q: Avez-vous l’intention de continuer ce que LAHIA vous a appris? R: Evidemment Q: Qu’est-ce qui vous motive dans le cadre de votre collaboration avec LAHIA? Q: Has anything changed for women and children with LAHIA? R: A lot has changed. All women have consultations and the children’s malnutrition rates have plummeted. Illnesses have decreased too, birth spacing… Q: Will you continue what LAHIA has taught you? R: Of course. Q: What motivates you to work with LAHIA? R: For me it’s the consideration they give us, the attention and regular supervision. No other project has been as close to beneficiaries as LAHIA Midterm Evaluation Report ACTIVITY DESCRIPTION, FINDINGS AND DISCUSSION 28 R: Vraiment pour moi c’est la considération et la supervision régulière qu’ils font à notre égard, aucun projet n’a été vraiment aussi proche des bénéficiaires que LAHIA. LAHIA. - Focus group discussion with infant and young child feeding group IYCF groups, comprised of men and women, said they felt proud to receive training and to promote healthy living to friends and family. This group, unlike village management group members, did not feel they were sacrificing individual freedoms in search of universal health, nor did they complain about bearing an undue burden. The IYCF groups were large and the responsibilities to transmit messages to the wider community shared. All women with young children are contributing to learning even if they were not tasked with educating others. Men and women reported sharing the information they learned from LAHIA with others they cared about. Likewise, peer educators were proud of their standing and accomplishments. They were enthusiastic about what they learned and about sharing information with others. They did not give indications that they felt taken advantage of. The peer educators who were interviewed were very active and committed. They often participated in other LAHIA activities in their village such as being a member of the village development committee and participating in the farmer field schools. These men seemed to be the most committed type of respondents, outside of LAHIA staff, who participated in the MTE. Beneficiaries reported health messages were well delivered, well understood, and applied. They trusted the messengers and the messages. Respondents asked for more information and new health messages, particularly on the topic of hygiene. The evaluators interpret this consistently positive feedback as adequately trained and knowledgeable staff and facilitators. Project beneficiaries reported that they practiced all feeding recommendations provided by the LAHIA field agents. For infants they listed exclusive breastfeeding, weaning foods, number of meals, and maternal hygiene as important practices. "Tous les messages sont importants parce que c’est quelque chose de nouveau et qui nous aide. Ils sont tous maitrisés et sont partagés avec les maris dans les ménages et les amies, voisines dans le village. Toutes les pratiques seront poursuivies à la fin du projet parce que je connais l’importance et j’y accorde un grand intérêt. Ces activités nous permettent de venir en aide à nos maris dans la gestion du foyer." ”All the messages are important because it’s something new that helps us. They are all followed and shared with husbands and friends, neighbors in the village. All of the practices will be followed after the project finishes because I understand the importance and they interest me. These activities allow us to help our husbands to manage the household.” - Female ration beneficiary, IYCF group member, and warehouse receipts participant Project staff reported reviewing health center records, checking in with peer educators, and visiting with beneficiaries to verify message delivery. Discussion: The peer educator training and outreach system is very efficient and reaches anyone who will listen during community gatherings and home visits. While listening to key informants, evaluators came to understand that it is nearly impossible to avoid hearing health messages in LAHIA communities. By providing widespread and frequent public service announcements, all villagers were reached. After three years of participating in the LAHIA project, beneficiaries have come to appreciate the LAHIA messages. The relative value of the commodity of health seems to have gained importance. They reported being able to relate to the messages in a way that they did not before. Messages being delivered by friends and peers are effective modes of transmission in the communities where LAHIA works. Most LAHIA Midterm Evaluation Report ACTIVITY DESCRIPTION, FINDINGS AND DISCUSSION 29 respondents said that they discuss LAHIA health messages with friends, family and neighbors. People asked for new information about how to stay healthy. There is nearly always a discrepancy between reported practices and actual adoption of new behaviors. People over-report behavior change. These data reflect knowledge and intention to change. These findings indicate that the program is moving forward in the right direction. Recommendations: Institutionalize and normalize the data quality assurance activities using spot checks and secondary records. Spot checks of health center growth monitoring records or direct observation of household activities were not conducted as part of the MTE, but could be incorporated into existing monitoring by field agents. The direct observation of household activities would be useful to get a sense for which health messages are being practiced and which are not, despite what people report. A very concrete example of how the project could use direct observation would be to record actual food diaries of women and children over a 24 hour period. The same food consumption questionnaire used during the baseline to record foods eaten could be used to answer questions about exclusive breastfeeding and child feeding habits. This would be an important exercise for the project to better understand seasonal differences in food security and to develop appropriate messages related to seasonally-induced stresses and shocks. Community-Led Total Sanitation Findings: The project promoted community-led total sanitation (CLTS) through the promotion of household ventilated improved pit (VIP) latrines and delivering sanitation and hygiene behavior change messages. One-third of villagers interviewed said that they presently had latrines or had had a latrine in the past year. The beneficiaries who claimed to have constructed household latrines from local materials, in response to LAHIA sanitation messages, all stated that they were destroyed by the last recent rains. Respondents were waiting for the rains to pass to reconstruct or until they had enough money to purchase cement with which to build a more durable latrine. Those respondents who had not built a household latrine of their own volition stated that they would accept a latrine if it were given to them. The WV quarterly report from October to December 2014 reported that other projects working in Maradi gave cement to households to build latrines.21 Some LAHIA beneficiaries and project staff with whom evaluators spoke said villagers were waiting for this project to do the same. The interviewer questions (Q) and interviewee responses (R) for those that did not have a latrine contained nearly identical dialogue. Q: Avez-vous dans votre ménage une latrine moderne? R: Non, nous n’avons pas encore de latrine moderne dans le ménage. Nous partons dans la brousse en cas de besoin. Q: Vous trouvez que c’est approprié le fait d’aller dans la brousse déféquer? R: Nullement! ce n’est pas approprié, c’est pourquoi j’ai programmé dans les 2 mois à venir (coïncidant avec la fin de la saison pluvieuse) de construire une latrine bien moderne, cimentée. Q: Do you have a modern latrine at your home? R: No. We do not yet have a modern latrine in our home. We go in the bush when we need to go. Q: Do you this it is appropriate to defecate in the bush? R: Not at all! It is not appropriate. That is why I plan, in two months, to construct a good modern latrine, cemented. - Male farmer field school participant & recipient of protective ration 21 World Vision Niger. (2015) Livelihoods Agriculture Health Intervention in Action Quarter One Narrative Report. LAHIA Midterm Evaluation Report ACTIVITY DESCRIPTION, FINDINGS AND DISCUSSION 30 Latrines were a higher priority in communities with little natural foliage to ensure some degree of privacy. This is in line with the shame that respondents expressed at having to go to the bathroom in a place where they might be seen by others. Respondents did not talk about the shame of filth but did identify village cleanliness as desirable. One sampled village was strikingly immaculate when the evaluators arrived. No village has been certified by LAHIA as CLTS friendly. Discussion: Community-led total sanitation (CLTS) is another example of a technique used to change attitudes. Whether the immaculate village was cleaned for the visitors or it is always clean at least shows that the community cleaned to make a good impression. That means that some degree of importance has been attached to being defecation-free. Recommendations: There are no recommendations to modify CLTS activities. Water and Sanitation, Hygiene Findings: The evaluators spoke with management committees in two villages and two pump repairmen. Findings are also taken from WV quarterly reports and interviews with government officials Community and government involvement was more time consuming than initially planned. WV organized, mobilized and trained water management committees on operating procedures and responsibilities. Water management committees are made up of men and women. The management committees interviewed said that the women were responsible for daily cleaning around the well and men looked after the bank account and operations. No one complained about the division of labor. As of the WV quarterly report covering January through March 2014, 35 committees had saved an average of $300.00 USD each, for borehole maintenance.22 The committees had bank accounts for the maintenance fund and kept the well aprons clean and the pumps functional. Communities that have not received improved potable water systems requested that LAHIA dig them a well. In communities that did receive a borehole, they asked for an additional water source and for a mechanized pump. Community members had heard stories of LAHIA providing water towers in the region and requested those as well. There were no questions raised by respondents regarding inaccessibility to water sources except to say that pumps were slow to deliver water, physically demanding to operate, and wait times for water high. There was a suggestion that the project donate motorized pumps rather than hand pumps. The rural engineering and hydrology technical services were clear about their responsibilities from inception to post-LAHIA. The evaluation team spoke with two pump repairmen. The pump repairmen have already replaced parts and report greasing the pumps each month. One said that he trained his brother who has taken over pump maintenance. This means they have the skills as the pumps are still functional. They are not paid for their services. Discussion: This activity is very appropriate because it responds to a real need as defined by both the project and its beneficiaries. The evaluation leader believes that what was not said is just as important as what was said during beneficiary interviews. There were no complaints. Construction is very efficient and the rural engineering and hydrology technical services were clear about their responsibilities from inception to post-LAHIA. The involvement of water management committees at the community level and government technical services at the commune, district and regional levels has set this activity up for enduring success. Ensuring that water points would be respected and maintained by the host community, before beginning construction, seems to have paid off and been important for sustainability. Water management committees seemed to be functioning and taking their jobs seriously. They are relatively new and still norming. After one year the boreholes are still giving water, have maintenance 22 World Vision Niger. (2015) Livelihoods Agriculture Health Intervention in Action Quarter Two Narrative Report. LAHIA Midterm Evaluation Report ACTIVITY DESCRIPTION, FINDINGS AND DISCUSSION 31 plans in place, and have money set aside for maintenance. Government oversight roles are clear and respected by project staff and government employees. Recommendations: Supportive structures in the communities were well trained but could use on￾going support and encouragement from the project. Supporting the norming process for established committees will be an important sustainability strategy. Included in this should be discussion about term limits or, at minimum, re-election schedules. By rotating public servant volunteers, they avoid burnout and ineffectual members can be voted out. Continually visiting villages with LAHIA water points will encourage committees and villagers to maintain standards which could turn into habit. It should be noted that without being paid, not many skilled repairmen will continue providing services. There should be a strategy to either organize communities to pay them or to train replacements who can take on the repairs and maintenance when the volunteer repair person can no longer volunteer. Cross-Cutting Gender The 15 evaluation questions relating to gender are contained in Table 19. Description: The gender analysis report conducted for the Niger DFAPs defines gender activities as promoting female participation in community activities.23 The project, in collaboration with the two other DFAPs being implemented in Niger, conducted a gender analysis, “Gender Analysis in the Regions of Maradi and Zinder” which catalogued women’s roles within the household and community. LAHIA trains project staff, SongES peer educators, VDCs, and communicator network members. They hold special sessions to discuss women’s issues, such as access to land and reflecting on women’s contributions to society. Women are members on all village-based management committees. Youth are management committee members to a lesser extent. The husband schools were created to directly include husbands in to promote and protect women’s health. Findings: According to quarterly and annual reports, LAHIA conducted a gender analysis, trained LAHIA staff and partners on gender sensitivity quarterly, and encouraged the participation of beneficiary women in all project activities. This has translated into increasing the representation of women in community leadership groups. Some activities excluded men such as goat rotation groups and savings groups. The project sponsored an exchange for National Women’s Day in one commune. The project has started to realize some of recommendations from the gender analysis, namely promoting household hygiene and sanitation, teaching literacy, and promoting women in leadership positions. Beneficiaries were asked to discuss gender roles in their households and community and how they had changed with their participation in the LAHIA project. Reactions to including women in all activities were mixed. During individual interviews with project beneficiaries, a few men reported that they included their wives in household decision making. During focus group discussions, male project participants were more vocal about asserting their authority over women. They made statements about how women were included on LAHIA committees because the project required it and that they would not allow their wives to consort with men in public as the project forced others to do. When directly asked about the changing gender roles, responses are epitomized by the following mixed gender focus group discussion. Q: Quelle est la signification de la présence de femmes parmi vous? Homme: C’est obligatoire. Q: Why is it notable that women participate in this farmer field school? Male: It’s obligatory. 23 Marie Angelique Yande Faye. (Nov 2014) Analyse Genre dans les Régions de Maradi et Zinder. [Translation: Gender Analysis in the Regions of Maradi and Zinder]. LAHIA Midterm Evaluation Report ACTIVITY DESCRIPTION, FINDINGS AND DISCUSSION 32 Femme: Nous avons aussi besoin de renforcement de capacités. Homme: C’est ce que l’évolution a apporté, nous prenons patience, nous observons l’évolution se dérouler. Homme: Si nous n’avions pas consenti à ce qu’ils voulaient, est-ce que nous aurions eu ce bienfait, le forage? Female: We too need capacity building. Male: This is what comes with modern times. We are being patient. We observe the changes unfold. Male: If we didn’t consent to what they want, would we have this good thing, the borehole? - Farmer field school focus group Communities felt that the gender approach used by the project was imposed on them. Meaning that project staff determined that communities needed to include women in public activities and women and men should be equal partners in household decision making. Interviewees have not yet been persuaded that women should be public figures or to work alongside men. Peer educators were very proud that they were selected as model husbands and of the work they were doing to help women in their communities. They actively promoted women’s health issues and husbandry. They also had consistent support and supervision from SongES coaches, also referred to as facilitators. Discussion: The quote above was selected because it shows a male resisting changing gender norms and a female embracing her empowerment. Findings were inconclusive. The verbosity of men in focus group discussions could have been influenced by the facilitators’ preconceptions of social norms or by peer pressure to resist acknowledging women’s participation in public activities as acceptable. Changing social belief systems is a complicated issue. Social norms are not necessarily functionalist24 meaning that gender roles do not necessarily serve a practical purpose such as male power consolidation or labor efficiencies such as women can clean better than men. They are followed to avoid social sanctions or exclusion. The gender analysis did not identify the forces at work which were responsible for marginalizing women or what the consequences of non-compliance would be. These are the things you need to know to impact social change. If the project can convince and demonstrate to men and women that the risk to change the status quo is worth taking then the opportunity exists to impart lasting change. A true approach to gender equity encourages men to participate in activities traditionally dominated by women. The peer educator activity is a good example of how gender lines can be successfully crossed. The success of the peer educator activities points to well-trained coaches who have a mastery of the subject matter and adult learning techniques and their supervisory support. The gender analysis report was shared in November of 2014 and the project has started to implement the recommendations. As data collection occurred less than one year after the report was released, the three pages of recommendations are still in the process of being mainstreamed into activities. Some of the recommendations from the gender analysis report were reinforced during this mid-term evaluation.  Développer des activités mixtes  Develop co-educational activities  Par rapport à certains gros villages réétudier la pertinence des pompes manuelles.  Revisit the appropriateness of manual pumps  L’approche genre n’étant pas une inversion des rôles traditionnels du travail  The gender approach is not an inversion of traditional roles 24 Stanford University. Social Norms. Published 2011. http://plato.stanford.edu/entries/social-norms/. . LAHIA Midterm Evaluation Report ACTIVITY DESCRIPTION, FINDINGS AND DISCUSSION 33 Recommendations: To fully investigate the effects of LAHIA’s gender work a focused evaluation will need to be conducted. The project could benefit from an evaluation to determine whether the gender action plan drafted by project staff was followed and if it continues to be appropriate. The gender evaluation could investigate the reluctance voiced during the MTE to at least question gender norms. Specifically look at the ways in which the promotion of gender equity is enhancing or hindering project outcomes. Monitoring & Evaluation The two evaluation questions relating to M&E are contained in Table 20. Description: The monitoring and evaluation (M&E) system tracks data required to tabulate 63 indicators. The project uses three impact indicators. Two are standard USAID Bureau of Food Security indicators and all are monetary measures of poverty. The rest of the indicators are a mix of input, output, and outcome. 21 indicators were collected during the joint DFAP baseline survey and will be collected again at the end of the five year project. The other indicators are populated through an annual survey or routine data collection. The last annual survey was conducted by a private sector firm for three indicators. Regular monitoring data are collected by LAHIA field agents. The regular monitoring is a paper-based set of forms that are used by LAHIA field agents to collect monthly data. Data from these forms are manually entered into spreadsheets and indicator values tabulated. Some indicator values are reported in quarterly reports. The indicator performance tracking table (IPTT) is updated annually. SC and WV use the same indicators, definitions, and data collection forms and processes. Studies and evaluations are planned and executed jointly. The monitoring, evaluation and learning (MEAL) staff from both organizations work closely to complete tasks and problem solve. Data are shared with government and non-governmental partners during Task Force meetings. The Task Force initially met monthly to share progress, discuss plans, and problem solve. During the last year of the program meetings were reduced to once a quarter. LAHIA shared qualitative and quantitative monitoring data with partners during the meetings. Findings: The project shared a number of monitoring and evaluation documents with the evaluation team. These include;  Niger Title II Baseline Report (Jan 2014)  Reporting Reference Guide with Data Collection Form Appendix (Jun 2015)  Indicator Performance Tracking Table (Jan 2015)  Fiscal Year 2014 Annual Results Report  Fiscal Year 2014 Quarter 1, 2 & 3 Reports  Fiscal Year 2015 Quarters 1 & 2 Reports LAHIA’s Monitoring, Evaluation and Learning (MEAL) team has created several documents which they collectively refer to as the “M&E Plan”. These documents include: Performance Management Plan (PMP), Performance Indicator Reference Sheets (PIRS) and the Indicator Performance Tracking Table (IPTT). The Reporting Reference Guide contained 18 performance indicator reference sheets that would normally be one portion of the M&E plan, was not provided to the evaluators. The performance indicator reference sheets provided definitions and instructions for collecting and tabulating 32 indicators. All indicators were to be collected on a monthly basis by LAHIA field agents and ultimately submitted to the M&E team by the technical coordinators. The Reference Guide appendix contained 16 data collection documents. They were a mixture of data collection forms, tally sheets, and reporting formats. LAHIA Midterm Evaluation Report ACTIVITY DESCRIPTION, FINDINGS AND DISCUSSION 34 World Vision reported on these indicators in all their quarterly reports. Save the Children included some of these monitoring data in the last quarterly report (FY15 Q2) shared with the evaluation team. Both organizations included rich qualitative information about the activities and about project operations. Monitoring, evaluation and learning (MEAL) staff in Niger and at SCUS felt the monitoring and evaluation system was not as useful as it could be. Staff cited possible areas of improvement could be to include quality indicators, involving technical coordinators in data use, and moving from a paper-based to an electronic data capture system. The project has invested time and effort into creating the current system and brainstorming possible ways to improve it. Discussion: The M&E documents provided to the evaluation team contained the most essential information necessary for collecting and tabulating data and reporting indicator values to USAID in a consistent manner. Quite a bit of effort went into creating the indicator performance reference sheets in the Reporting Reference Guide. A full M&E plan already has a foundation upon which to build. A full M&E plan is a culminating document that brings together the why, what, where, when and how of project M&E. An effective monitoring system is based on careful planning, thorough documentation (i.e., protocols), thoughtful allocation of effort, and accurate record keeping. Project staff asked about adding quality indicators to the already existing list of 63 USAID-required indicators, 40 of which are reported on annually. However, process indicators which monitor quality are currently absent in the M&E plan. The existing indicators do little to help managers make programmatic decisions which may be one of the reasons the program coordinators are not more involved. Although the required indicators are numerous and onerous to collect, it does the project a disservice not to include at least one quality indicator per intermediate result. Recommendations: The next step for LAHIA is to document the current system and its strengths and shortcomings, and share the formal gap analysis with program staff. As important as documentation is, developing technical staff’s appreciation for how data can help them manage project is equally as vital. This can only be done if the data are relevant. Leadership should explicitly tie data collected to individual and project achievements and decision making. This could be done by using existing staff who are already carrying a heavy burden, bringing in technical assistance from member organizations’ headquarters, or by hiring short-term consultants to help select quality indicators and to build the monitoring system. LAHIA should reach out to the other DFAPs implemented in Niger to learn about their M&E systems, challenges and accomplishments. Highlight areas that could benefit from development, draw staff into the process, and obtain outside assistance. Train program staff to record keep as documentation of accomplishments and learning opportunities rather than a reporting requirement for funding purposes. Technical coordinators should embrace monitoring data as proof of their accomplishments. Commodity Management Description: The LAHIA project monetized one tranche of commodities and has since only used commodities for direct distribution activities. They used the port of Cotonou for importation of commodity during the first year. The late shipment delayed full implementation during Year 1. Lomé was used as the port of call in the second year. SC rents a warehouse where the stock of food is kept for both SC and WV. Rations are distributed once every two months at distribution points. The evaluation questions relating to commodity management are contained in Table 21. Findings: Ration recipients were asked if there were any problems with receiving received rations. All who were scheduled to receive rations said they were received as they expected and there were no problems. Discussion: The project ran into logistical difficulties procuring the commodities in the first year and adjusted accordingly by changing the distribution schedule. There were no ration distribution problems reported from over 100 beneficiaries. This implies that LAHIA is managing the commodities well and ensuring recipients receive their rations. LAHIA Midterm Evaluation Report ACTIVITY DESCRIPTION, FINDINGS AND DISCUSSION 35 Recommendations: There are no recommendations for commodity management. Environmental Safeguard and Compliance The nine evaluation questions relating to environmental safeguards and compliance are contained in Table 22. Description: The project conducted an initial environmental assessment and maintains environmental status reports and the environmental mitigation monitoring plan in compliance with Regulation 216. Environmental stewardship is an integrated part of project activities, including partnering with community-based natural resource management committees. Findings: LAHIA staff were trained on the foreign relations Title 22 Code of Federal Regulation 216 in January of 2014. The food security and livelihoods coordinators are responsible for environmental monitoring and reporting. Borehole water quality is tested by government officials and maintains a closed system to protect water cleanliness. The project teaches water management committees how to maintain cleanliness of the water points. The project used Food for Work to reclaim land for grazing and teaches farmers about ecologically sound farming practices. The project does not promote synthetic pesticides in any of its agricultural activities. Instead, the project has promoted organic compounds concocted from neem tree seeds. No cases of illnesses have been reported in the LAHIA catchment area as a result of pesticide use. In the commodity warehouse, fumigation equipment is handled by a professional fumigation company following safety norms and standards. According to LAHIA staff the project exceeds the bar set by environmental compliance regulations. Discussion: The evaluation questions in the scope of work were closed with regards to environmental compliance. As the project is reportedly complying and surpassing regulations, there are not discussions in response to the evaluation questions. The MTE was not an environmental assessment. Recommendations: There are no recommendations for environmental compliance. Outstanding Bureau Environmental Officer conditions should be handled separately by an environmental evaluation expert. Implementation of Activities This category of questions has no overarching theme tying the sub-sections together. Sub-sections include; a) targeting; b) progress; c) quality; d) sustainability; and e) management. When questions are answered in the Technical Approach by Sector and Cross-Cutting section the responses are not duplicated here. Targeting Three targeting questions are contained in Table 23. Description: The program was designed to involve the same individuals in activities under all of the SOs. SO3 activities (white sphere) target all community members as indirect beneficiaries. The activities build capacity of early warning and emergency response government teams in the five communes. The team members are the direct beneficiaries. Activities under SO2: Agriculture and livelihoods (blue sphere) were intended to directly include just over 126,000 people, all of whom are indirect SO3 beneficiaries. SO1: Health, Nutrition, WASH (green sphere) activities were to include 125,000 people, a subset of the SO2 and SO3 project participants. Figure 3 was copied from the SC and WV proposal to USAID/FFP. It visually shows the target populations as they were conceived in 2011, during the proposal design period. LAHIA Midterm Evaluation Report ACTIVITY DESCRIPTION, FINDINGS AND DISCUSSION 36 Figure 3 LAHIA Target Groups from Proposal Note: This graphic pictorially represents the LAHIA target population. The three circles represent the numbers of people to be involved in activities carried out under each SO. The population targeted by SO3 activities is represented by the largest circle, SO2, by the blue circle, and SO1 by the green. SO1 (green): at least 17,972 households with children less than 2 years of age or handicapped, approximately 125,804 SO2 (blue): all SO1 beneficiaries plus at least 350 participants who do not qualify for SO1 activities, approximately 126,151 SP3 (white): total population of five communes, approximately 445,570 Findings: In reality, LAHIA targets are more nuanced. Each activity targets a different type of person or group of people. To provide a picture of activity targets, Table 9 is a copy of the activities list from the IPTT with beneficiary types listed in the right-hand column. Activities beginning with the number “1” fall under SO1. Note that each activity targets a different set of people. Many individuals participate in multiple different groups. The project adapted as new opportunities were identified. The activity list in Table 9 is still being revised as new demands are identified. The intention of including a breakdown by activity of project targets was to illustrate the complexity of the LAHIA program. Table 9 LAHIA Targeting LAHIA ACTIVITY LAHIA TARGET Activity 1.1.1.1: Design comprehensive social and behavior change (SBC) strategy LAHIA health staff Activity 1.1.1.2: Implement SBC strategy LAHIA health staff Activity 1.1.1.3: Reinforce family planning (FP) promotion and linkages with services All community members Activity 1.1.1.4: Support for exclusive breastfeeding at community, commune, and district levels All community members Activity 1.1.2.1: Distribute MCHN rations to vulnerable PLWs and children under 2 PLWs & children < 2 Activity 1.1.2.2: Distribute protective family rations during the lean season to households receiving individual rations Households of PLWs & children < 2 Activity 1.2.1: Reinforce delivery of services at health facility and community levels Health center staff Activity 1.3.1: Construct at least 99 new and rehabilitate 40 existing water points GoN Ministry of Water prioritized villages in the LAHIA intervention zone Activity 1.3.2: Construct 261 household ventilation improved pit (VIP) latrines and 25 public gender-segregated VIP Activity 1.3.3: Promote treatment options for drinking water collected from open wells and surface sources and good hygiene practices Activity 1.3.4: Train two groups: local masons and pump maintenance/repair SO3 (en blanc): Population totale de 5 communautés cibles : approx. 445.570 SO1 (en vert): au moins 17,972 ménages avec des enfants < 2et/ou des handicapés approx. 125,804 SO2 (en bleu): Tous les bénéficiaires SO1 ciblés et ±350 personnes qui participent dans les activités de SO2, mais qui ne remplissent pas les critères d’éligibilité de SO1: approx. 126.151 LAHIA Midterm Evaluation Report ACTIVITY DESCRIPTION, FINDINGS AND DISCUSSION 37 Table 9 LAHIA Targeting LAHIA ACTIVITY LAHIA TARGET technicians Activity 1.3.5: Establish water, sanitation and hygiene (WASH) Management Committees for all public water and sanitation facilities Activity 2.1.1: Establish and /or strengthen women’s savings groups All women Activity 2.1.2: Identify and strengthen women’s income generating activities All men and women Activity 2.1.3: Provide small ruminants, inputs and services for animal husbandry All women Activity 2.1.4: Create women's gardens and small livestock enterprises All men and women for gardening All women for livestock Activity 2.2.1: Train staff and partners in good agricultural techniques and management practices. LAHIA staff Activity 2.2.2: Disseminate good agricultural techniques and management practices at the village level All men and women Activity 2.2.3: Facilitate access and linkages to essential inputs and services All men and women Activity 2.3.1: Strengthen capacity in agricultural business planning, knowledge and skills All men and women Activity 2.3.2: Establish mechanism to access market information All men and women Activity 3.1.1: Create new and strengthen existing village development committees VDC members Activity 3.1.2: Pilot food and nutritional security surveillance system SCAP/RU & OSV members Activity 3.2.1: Provide regular training and equipment to commune-level units for crisis prevention and response OSV Activity 3.2.2: Strengthen commune-level plans for preparedness and management of crises OSV Activity 4.1.1: Train staff and partners on gender awareness LAHIA staff, peer educators, VDC members, communicator network members Activity 4.1.2: Conduct a gender audit in target communities and ensure integration of gender-specific needs into program activities LAHIA staff Activity 4.1.3: Address practical and strategic gender needs across SOs 1-3 All men and women Activity 4.1.4: Encourage the participation of women across all activity areas All men and women Activity 4.2.1: Organize gender reflection days and international women's days with community leaders All men and women Activity 4.2.2: Train VDCs in gender awareness (links SO3, Activity 3.1.1) VDC members Discussion: The discussion sub-section will only discuss targeting for ration beneficiaries. The original idea of the project was that SO1 beneficiaries would benefit from all other activities. This approach to coverage was intended to create synergies between activities at an individual level. The logical conclusion is that by the end of the project, those beneficiaries who participated in the full suite of LAHIA activities would be relatively better off than others, such as women who were never pregnant. The formative research report25 said that in Maradi there was already a high level of consciousness about using health services for all child birthing care. But, according to the 2012 Demographic and Health Survey, 73% of women delivered at home and only 11% had no prenatal care. These statistics, in and or themselves, are only jarring when coupled with the 36% perinatal death rate in Maradi and the 39% 25 MOUSSA Abari , SANI Ismael , AGALI Mouhmoud. (Jun 2013) Rapport de Recherche Formative. [Translations: Formative Research Report]. LAHIA Midterm Evaluation Report ACTIVITY DESCRIPTION, FINDINGS AND DISCUSSION 38 maternal mortality rate estimated for Niger during the past seven years.26 This means that more than one in three pregnant women die due to child birth. We know that skilled care before, during, and after pregnancy can greatly reduce mortality rates. This establishes that encouraging prenatal visits and health center deliveries is important. To encourage health center visits, LAHIA rations entice families to comply. To be selected as a ration beneficiary the mother must have made three documented antenatal care visits. In reality, the program must wait for the women to confirm pregnancy at the health center and become enrolled in the system. To answer the evaluation questions, this target population was appropriately identified based on the consequences of food insecurity. The preconditions on enrolling in the maternal ration program don’t align with the science. The author therefore does not think LAHIA has reached its target population early enough in the pregnancy to avert negative health outcomes during adulthood which is the premise upon which the 1,000 day approach is based. The ideals of the 1,000 day approach are not practical in a food distribution program where rations are provided only upon proof of pregnancy. In the history of food aid, selection criteria are very difficult to enforce. Title II programs switched from using “vulnerability” as the main inclusion criteria to “pregnant or lactating women and children under 2 years of age” is understandable. The latter set of criteria is easier to prove than the former. Recommendations: The selection criteria for ration beneficiaries should be maintained as it currently is to avoid confusion and to maintain consistency of beneficiary expectations. If the current criterion are preserved, say that your project takes into consideration the 1,000 day approach. The alternative is to reconsider preconditions for acceptance into the maternal ration program. Ration distributions could be approved after the first visit and subsequent rations predicated on subsequent visits. It is more administratively complicated but scientifically sound. Progress The evaluation questions associated with this sub-section can be found in Table 24. The “Progress” sub-section evaluation questions are partially answered in the Technical Approach by Sector and Cross-Cutting sub-section. To avoid overlap in answers between sub-section the evaluator reserved the monitoring data assessment for this sub-section. Descriptions have been foregone as the project has been described above. Monitoring data are contained in Appendix A: Indicator Performance Tracking Table. Agriculture and Livelihoods Findings & Discussion: Data referenced in this sub-section are contained in Appendix A: Indicator Performance Tracking Table. The life of project target for number of people trained in agricultural techniques is set for more than 19,000 people. The total population in the catchment area is 113,700 people. 44% of the population of Niger is of productive age, between 15 and 54 years old. The total population multiplied by the percentage of adults able to work in the fields is 50,000 people. This means that the project intends to train 38% of all eligible adults. This seems to be a reasonable target in terms of outreach levels. All projects working to increase agricultural production must have two essential indicators, area and quantity produced. The gross margin indicator (indicator #46) for cowpeas appears to be incorrect. The second fiscal year reported a gross margin of $4,439 per kilogram of cowpeas. INRAN may have yield data for the varieties the project promotes that can be used to update targets. 26 Institut National de la Statistique (INS) et ICF International, 2013. Enquête Démographique et de Santé et à Indicateurs Multiples du Niger 2012. Calverton, Maryland, USA: INS et ICF International. LAHIA Midterm Evaluation Report ACTIVITY DESCRIPTION, FINDINGS AND DISCUSSION 39 The number of farmers practicing value chain activities (indicator #49) may contain typographical errors as the numbers are inconsistent and should be reviewed. The actual number of farmers went from nearly 40,000 in year one to 3,500 in year two. If the project plans for 7,500 women to receive goats (indicator #33) and only 26 para-vets are trained (indicator #34), each para-vet will be responsible for about 290 women. This might be high mainly considering that it may be difficult for women to travel independently to outside villages to access para-vets, or vice versa. Accessing services might therefore become problematic. The numbers of women in savings groups (indicator #30) and participating in income generating activities (indicator #32) exceeds targets while the numbers of women provided with goats lags behind expected numbers. The average increase in the sales value of agricultural products (indicator #29) for LAHIA participants was $US 855 last fiscal year. Incremental change should be carefully interpreted. The value stated does not take into account fluctuating exchange rates or supply and demand due to over or under production. The main outcome of agriculture and livelihood activities is to increase beneficiaries’ stable assets. This is not measured by LAHIA monitoring indicators, but in the baseline and endline surveys. Using the available monitoring data in the IPTT, it is not possible for the evaluators to assess changes in household assets. Qualitatively, evaluation respondents did not report making money as a result of LAHIA. They reported great anticipation for the possibility of making money with LAHIA’s help. At the same time, project beneficiaries were not provided with any financial literacy training so they may have made money due to LAHIA livelihoods activities but are unaware of the increase in income. Participating in the project increased individuals’ assets but not because they were engaged in successful business ventures but because the project gave them those assets. Unfortunately, no field agents were included in the evaluation so beneficiary commentary that they were not knowledgeable in business topics went unconfirmed. Recommendations: Farmers should be taught how to collect and evaluate the most basic of agricultural statistics. Without understanding yields, farmers will not be able to assess their degree of success. The project team may want to revisit the number of para-vets trained. Targets for the number of women in savings groups and participating in income generating activities should be reviewed and reconsidered. The project should invest in the knowledge and skills of their field agents as business advisors. Health, Nutrition, Water Sanitation and Hygiene Findings & Discussion: Appendix A: Indicator Performance Tracking Table. The numbers of community health workers delivering family planning messages (indicator #11) is set for approximately 1,250 people. As the project is in 72 villages, the project plans to work with about 17 (1250/72) health workers per village. That seems like a large number of health workers for one village. Even with replacements more than one dozen people trained to educate others about family planning seems excessive. The evaluator assumes that these are numbers of direct trainees and not indirectly trained by community volunteers. Just over 900 people were trained in health and nutrition (#10). 5,400 people were trained in agricultural techniques (#37). By looking at the numbers and knowing that health and nutrition education is the major thrust of the program there seems to be a disconnect between the reported number of people trained in health and nutrition. These may be numbers reported by the local partners, SongES for health and nutrition peer educators. The cascade of second tier students may not have been counted. The percentage of women receiving four antenatal check-ups (#14) is lower than targeted. It will most likely not reach the 100% coverage rate set for the final year of the project. To add context to these LAHIA Midterm Evaluation Report ACTIVITY DESCRIPTION, FINDINGS AND DISCUSSION 40 statistics, the United States Department of Health and Human Services, has set the target for the same indicator at 77% to be reached by 2020 (from 70% in 2007). Constructing 99 water systems and rehabilitating 40 water systems over the life of the project is aggressive (#17). That implies that during the 48 months of activity implementation, the project will build or repair three water systems each month.27 The targeted numbers of wells and boreholes may be overly optimistic considering preparation time with communities to ensure they are ready to respects the norms of using the water system, the limited equipment and site supervisors during the construction period, and the investment in time to transform a group of individuals into a functional and durable water management committee. As of the writing of this report, it is unlikely that the five-year targets will be met. Given that LAHIA is working in approximately 70 villages and the government has scheduled multiple water systems in some localities, it is conceivable that by the end of the project, all LAHIA villages will have improved water systems. This is a laudable legacy in a water-poor area. Interviewees in all villages expressed the need for potable water sources and attributed the identification of this need to LAHIA behavior change messages. Constructing 286 latrines (#21) over the life of the project is reasonable. This translates into six latrines per month during the 48 month activity implementation period. Construction has advanced on schedule and if it continues at the same pace, should meet the expected targets. The project monitors no outcome measures for WASH activities. Interviewees were asked what changes s/he saw in the village that s/he attributed to LAHIA. A common answer was that children were clean and did not get sick. When probed these outcomes were accredited to having more food to eat, thanks to rations, and having clean water for bathing and consumption. These outcomes are as expected and gratifying that the intended positive results have been realized. Recommendations: The project should reanalyze the numbers of people trained in health and nutrition and agricultural techniques for accuracy. LAHIA should revisit its last year target for the percentage of women receiving four antenatal check-ups given that, according to the World Health Organization, only 64% of women in the world four antenatal visits. Numbers should be adjusted downwards to estimate the actual numbers of new health workers that would be found in each village. The same logic should be applied to the 4,500 child health and nutrition trainees. The antenatal checkup and number of water point indicator targets should also be reassessed. Cross-Cutting Gender Findings: Data referenced in this sub-section are contained in Table 10 27 99 water systems constructed + 40 water points rehabilitated = 139 water systems repaired or constructed / 48 months of construction and rehabilitation = 2.90 water systems repaired or constructed Table 10 Output Indicators, Actual & Targeted Values INDICATOR FY1 (actual target) FY2 (actual target) FY3 (actual target) FY4 (actual target) FY5 (actual target) LOA (actual target) 32. Percentage of women using at least one market￾led IGA strategy 0% 0% 92% 30% N/A 40% N/A 50% N/A 60% 92% 70% 57. Percentage of women in leadership positions in community groups 0% 25% 33% 30% N/A 35% N/A 40% N/A 45% N/A 50% LAHIA Midterm Evaluation Report ACTIVITY DESCRIPTION, FINDINGS AND DISCUSSION 41 Discussion: The percentage of women in leadership positions in community groups (indicator #57) appears to be advancing as planned at the village level. These people were not selected in elections where LAHIA could dictate the composition of the winning team but educating community members on the value on gender-inclusivity could contribute to most equitable gender representativeness of village committees. The proportion of female participants participating in economic activities (indicator #32) meets or surpasses the target. The only question is that one strata includes children as young as 10 years old. The project should ask itself why it is promoting child labor. The final indicator to measure women’s participation asks community leader’s about women making joint decisions in their households (indicator #60). The targets appear to be on track. Recommendations: Disaggregating the joint decision making data by gender would help project managers to better fashion gender messages. Unless LAHIA plans on getting people fired and replaced with women, targets should be adjusted accordingly. Alternatively these institutions should not even be included in the disaggregation list since the project may not have control over the gender of members. Continue tracking disaggregated indicators by gender. Quality This question listed in Table 25 is answered in section Technical Approach by Sector and Cross-Cutting. Sustainability Evaluation questions about sustainability are in Table 26. Description: Sustainability is the continuation of activities after external support has been withdrawn. One focus group summed up what evaluators deduced after analyzing over one hundred interviews and two years’ worth of project documents. The lists below were generated during a focus group discussion with farmer field school participants so that context should be taken into consideration as responses are internalized. Q: Ce qui va rester après le projet LAHIA? R: Selon les hommes  Forage  Salubrité et les latrines  Moringa R: Selon les femmes  Petit élevage  Forage  Moringa Q: What will stay after the LAHIA project? R: According to the males  Borehole  Hygiene and latrines  Moringa R: According to females  Animal husbandry  Borehole  Moringa Q: Ce qui va disparaitre après le projet LAHIA? R: Selon les hommes Q: What will disappear after the LAHIA project? R: According to the males 60. Percentage of community leaders who report that in their own households, women make their own decisions or joint decisions with their husbands on health, nutrition, and agricultural issues 0% 10% 41% 20% N/A 30% N/A 44% N/A 45% N/A 50% Note: These indicators and data were copied from the LAHIA project’s indicator performance tracking table (IPTT). The top numbers of each line are the reported achievements for the first two years, FY2013 and FY 2014 .Data for out years are not available (N/A). A target has been set for each fiscal year (FY) and for the five year life of the activity (LOA). LAHIA Midterm Evaluation Report ACTIVITY DESCRIPTION, FINDINGS AND DISCUSSION 42  Traitement Sida cordifolia et ensemencement des herbacées  Maraichage  AGR R: Selon les femmes  AGR  Champ école paysan  Distributions de farines  Sida cordifolia removal and reseeding  Gardening  Income generating activities R: According to females  Income generating activities  Farmer field school  Rations Farmer field school focus group discussion From the perspective of the evaluation team leader, the top two most sustainable LAHIA activities are, in order of longevity are; a) WASH boreholes and pumps, and b) health and nutrition messages. The evaluator selected only two because beneficiary reactions to the other activities was mixed. Water and Sanitation, Hygiene For the purposes of this evaluation WASH infrastructure sustainability is not whether or not communities will continue to dig boreholes but if the structures will be maintained and used as intended. They should be considered a public good, accessible to all members of the community, kept clean, and pumps maintained and repaired. The evaluation team visited several communities with water management committees. They all had bank accounts and members knew the account balance. Communities care for and maintain LAHIA water infrastructure and the village management systems appears to be sustainable. This activity has the greatest likelihood for sustainability. Health and Nutrition Messages Project beneficiaries said they would continue to practice LAHIA-promoted health behaviors after the project ended. Peer educators, ration recipients and random community members seemed to be genuinely pleased at having gained some control over their lives and their destinies. There will certainly be some messages that people continue to adhere to after the project leaves. They may even continue to visit health centers. Management Management-related evaluation questions are found in Table 27. Description: SC and WV both have field offices in Maradi and their own staff. The SC LAHIA organigram, Figure 4 shows the Save team should have 18 extension agents, 8 technical staff, and 4 managers of the technical and extension staff. Findings: Staff turnover at SC has been high at all levels. The most pronounced example is that the project has had four chiefs of party in the first three years of implementation. Project staff reported having difficulty maintaining the required number of field agents for complete coverage. Despite having personal convictions that align with the Save mission statement, there was a malaise and dissatisfaction with the working environment. Examples include salaries that are under market and not investing in the professional development of personnel. WV employees cited personal growth opportunities and feelings confidence in the organization and of job security. Discussion: The field agents are the primary point of contact between community members and the project. Without them, LAHIA would be ineffective at engaging communities and individuals. Field agents were absent from the evaluation activities. One was occasioned upon by the evaluation team during a LAHIA Midterm Evaluation Report ACTIVITY DESCRIPTION, FINDINGS AND DISCUSSION 43 site visit but the contact was unplanned. In retrospect, it would have been important to hold at least two focus group discussions with field agents, one group of health promoters and another with agricultural extension agents. This was unfortunately not planned for in the evaluation design. Dissatisfaction with the workplace most often translates into staff feeling undervalued. That could be related to low salaries, divergent opinions not being heard, or professional expertise being respected. Recommendations: The causes of the high turnover and vacancy rates of field agents should be investigated by the senior management. The project would benefit greatly from a management evaluation by SC headquarters. It could be that the field agents are overqualified and find better work with the newly gained skills from LAHIA. Or, it could be found that field agents are overworked and under compensated and leave out of frustration. Whatever the reason, without field agents, the project has no reach and will have very limited interactions with the villages they serve. The end result will be that objective cannot be met. LAHIA Midterm Evaluation Report ACTIVITY DESCRIPTION, FINDINGS AND DISCUSSION 44 Figure 4 Save the Children LAHIA Organizational Chart HR officer Sub office coord Aguié Commodity Manager Finance Officer Program Manager Chief Of Party Rural & LH coordinator HNcoord M&E coordi Gender spécialist Gcoordinato Lahia accounta SBC Specialist MealOfficer 2 Rural dev agent Ag supervisor 3 Ag ext agent 6 Food monitor Finance assistan Field Operations Manager Procurement and Logistic Officer Commodity Accountant Warehouse manager Warehous e keeper MMD officer HN supervisor Drivers RH/AdminAss Cleaners 6 ACB Postes Pourvus Postes Non pourvus Nouveau poste 09 Guards Cleaners IGA Specialist Senior commodity accountant Data, information & com officer RH/Admin assistant Logistic assistant 03 Guards LAHIA Midterm Evaluation Report ACTIVITY DESCRIPTION, FINDINGS AND DISCUSSION 45 Partnership Description: Partners are people or organizations that work together towards a common goal. They share risks and benefits derived from the union. The evaluation scope of work listed the types of partners that the project wanted assessed. They include: Sub-recipients:  implementing partners (World Vision and Save the Children);  technical implementing partners (SongES for the Husband’s Schools) Collaborators:  government technical services(agriculture, health, livestock, environment, rural engineering, and hydraulics); municipal authorities (local elected officials and town representatives); administrative authorities (préfets, Governors, Initiative 3N - les Nigeriens Nourrissent les Nigeriens) ;  ASUSU (Saving Association in Hausa) for micro-credit activities and INRAN (Institut National des Recherches Agronomiques du Niger) for improved seeds and distribution networks); and  other development actors (example, other Food for Peace-funded programs). Findings: The LAHIA-led Task Force meeting was universally popular with partners. All of the participating partners felt it was an effective way to learn about the project and share feedback. Despite the support for the Task Force, public and private partners wanted greater information exchange. Government asked for written reports and to participate in supervisory visits. All partners wanted to revisit agreements with the LAHIA project as the work and expectations have shifted from the original planning period. It gave them a regular opportunity to feel included in the project. When asked how relationships between the LAHIA project and partner organizations could be improved, without exception, all partners interviewed mentioned that the Task Force should meet monthly. All partners appreciated the opportunity to learn about the project and provide feedback. They noted with aversion that the frequency had decreased. They were told the switch to quarterly meetings was due to a lack of new information to impart, but were not convinced. In general partners felt that there was not enough communication or sharing about progress or issues. All of the partners, whether implementers or government supervisors, said that LAHIA needed to respect their agreements. All wanted to renegotiate their agreements with the LAHIA project. Discussion: The project may feel that organizing and preparing for monthly meetings was cumbersome but it fostered good will. All healthy relationship requires nurturing and effort. The project expects that government will continue to provide services that are currently provided by LAHIA. This assumption is contingent on partners being prepared financially and managerially to assume those responsibilities. The project has done nothing to transfer the requisite skills or funding for the partners to adopt new activities. Recommendations: LAHIA should create a partnership policy and strategies to foster good will, healthy relationships, and to promote the sustainability of activities. The strategies should include how and when the handover of activities will occur. Invest more time on your relationship with partners. Reinstate monthly Task Force meetings. Project managers should use preparation for the Task Force presentations as an opportunity to take stock of past events and create short-terms plans. If a full presentation is too onerous to develop each month, a full program presentation could be prepared quarterly and in the intermittent months one activity could be highlighted in depth. Since each partner is different and the cooperation, coordination, collaboration paradigm does not apply to each partnership in the same way, LAHIA should treat each relationship as unique and agree upon the level of engagement that both organizations are prepared to invest. LAHIA Midterm Evaluation Report ACTIVITY DESCRIPTION, FINDINGS AND DISCUSSION 46 Implementing Partners The seven evaluation questions are found in Table 28. Description: Save the Children established itself in Niger as an emergency responder in 2005 in establishing a presence in Maradi. Save is a world leader in conceptualizing and delivering child-centric relief and development programs. WV brings 20 years of experience in Niger to the partnership. They have a history of planning and constructing WASH infrastructure. Findings: The relationship between Save and World Vision is close at the functional level and diverges up the hierarchy. Technical coordinators informally share thoughts and ideas with their counterparts. The SCUS headquarters staff has no direct management relationship for the LAHIA program as the contractual relationship is between SCI and WV, although SCUS is ultimately responsible for the project vis a vis the donor. Communications occur as needed. The SC Chief of Party and WV Program Manager in Niger have needed to jump start relationships with the arrival of each of the two Save COPs. SC and WV program heads in country are working to build a functional, if not productive, relationship. The most recent Save COP arrived one week before the evaluation team. The relationships between the two organizations have suffered from the lack of a consistent Save COP. Discussion: There is good will in both organizations to improve the working relationship and both sides understand that regular communication will be the cornerstone for a strong union. Recommendations: There are no recommendations as new relationships have just begun with recent staff turnover and will take time to develop. Partnership with Government Services The seven evaluation questions are found in Table 29 Description: Different government ministries are considered partners to the project. Their role is to ensure activities comply with government standards. In this capacity the government provides oversight as related to their mandates. Findings: The evaluation team met with government officials including the governor, department and regional technical service representatives, and mayors. Their level of support for the project mirrored the findings of this evaluation in such that technical services working in the sectors that were well organized and executed had more positive things to say about the project and collaboration. For example, the WASH infrastructure activities were well organized and executed. The hydrology and rural engineering representatives had intimate knowledge of project activities and locations. They were well informed and engaged. The same can be said for the health services. The health centers are linked to rations and behavior change communications. These are likewise well delineated in terms of project beneficiaries, entry and exit criteria. The only government ministry involved in income generating activities is the animal husbandry technical services. They are supportive of the para-vet training and suggested para-vets formally enter into the national system by receiving the government’s training. The animal husbandry representatives reported a positive relationship with the project despite the project not following national guidelines for animal grants. The evaluators recommend a regular meeting of para-vets to support and encourage dialogue and knowledge sharing. These meetings should be coordinated with the technical services of the communes where the para-vets live and work. On the other hand, the agriculture knowledge-based activities were less structured and Ministry of Agriculture representatives at the regional, department and commune levels did not feel engaged or part of the project. The early warning system unit at the government said they received computer equipment and training from the LAHIA project. Other Government interviewees did not report having capacity built but LAHIA Midterm Evaluation Report ACTIVITY DESCRIPTION, FINDINGS AND DISCUSSION 47 the project is not currently in the position to build the capacity of government institutions. Generalized capacity building was not a part of the original project plan. Discussion: As with all large organizations, different administrative levels have overlapping responsibilities. While speaking with representatives at the various levels of government, it became clear to evaluators that there are internal ownership disputes that make problem solving very challenging and time consuming for the projects such as LAHIA. When working with governments, projects should keep all levels involved abreast of operations. One of the sustainability strategies of the project is to have the government continue supporting LAHIA communities in line with their respective ministries. Given this expectation for the exit strategy, firm foundations need to be laid soon. Expectations for who will be responsible for what after the project closes will be a negotiation and should be included in the revised agreements. The evaluation team had access to one of the agreements with the sanitary district. It was very general and focused mainly on administrative and legal obligations. It did not go into details about who was responsible for what, when. A partnership agreement with roles and responsibilities detailed should have been included. Recommendations: Revisit agreements with government at regional, departmental, and commune levels. Technical Implementing Partners The seven evaluation questions are found in Table 30. Description: The project considers the service providers as development partners and some as beneficiaries. The evaluators classify all organizations or individuals who provide goods or services as suppliers, whether they live and work in LAHIA targeted communities or in Maradi, the regional capital. Findings: The service providers, except the para-vets, were interviewed. They were all clear on their relationships with the project and very supportive of how their services were being used. They reported exceptional returns on their investments with the LAHIA project. The for-profit ventures unequivocally want more business from LAHIA and its beneficiaries. Despite Halal’s network of sales agents in villages and towns, farmers asked where they could find improved seeds. This indicates to evaluators that there is room for growth in the distribution, sales, and/or marketing network for improved seeds. Recommendations: Direct relationships between communities and suppliers should be actively encouraged. The project should not act as intermediary between the suppliers and the communities to encourage beneficiary independence from the project staff and sustainable commercial relationships after the project ends. Other Development Actors The seven evaluation questions are found in Table 31. Findings: The clearest outcome of the community development committees was that women and men reported that “everyone” had pre-natal visits and birthed at health centers. No respondent said they were visiting the health center because they had community or social support to do so. They attributed the change in behavior to the convincing arguments of the LAHIA project. The IYCF groups said that the groups facilitated learning and was more conducive than hearing messages through health centers or mass media to instigating change. Management committee members were asked what training they had received from LAHIA. In the eyes of committee members, they were trained in hygiene, nutrition, and agriculture. There was no mention of management training by LAHIA. The LAHIA project strategies provided to the evaluation team group activities focused on technical messages and not on group structuring, management or problem resolution. The latter are necessary to function effectively. LAHIA Midterm Evaluation Report ACTIVITY DESCRIPTION, FINDINGS AND DISCUSSION 48 During interviews with village development committees, they kept referring to their work as “the LAHIA project”. VCD members of both focus groups, one community level early warning committee, and two village chiefs, asked for compensation from the project and let evaluators know they were “fed up with volunteering”. The IPTT contains no indicators to measure expected results for building supportive community structures. If the project did have such indicators they could measure attitudes and reported changes in behaviors. Discussion: At this juncture the groups are young and still forming. Management groups are not committed to public service. Committee members do not yet have ownership of the development activities associated with the LAHIA project. From the perspective of these groups the consequence of failure could be social ostracization. They are taking a potentially life changing risk by supporting an unknown entity. Approaching the delicate relationship through this lens will strengthen bonds of trust. If true partners share risks, they need to be made aware of them to truly own them and take responsibility. The alternative is when organizations use coping mechanisms to reduce risk. A common fall back option is to avoid taking any responsibility. This allows for deniability when problems arise. If this is the attitude that management groups are taking then it is very unlikely that they will rise to the occasion and take on responsibilities once the project leaves. For the committees and groups to remain functional, i.e. sustain themselves, the project needs to do everything possible to transmit LAHIA values to the groups. Stated another way, the groups need to own the values that may have started externally from them to increase the probability that they operate after the project closes. Monitoring organizational development, using qualitative data, would help project managers to attribute behavior and attitude changes to specific management-strengthening activities. Recommendations: The evaluation team urges the project to consider all types of village management committees as partners. The project should develop standard organizational management guides and templates for training across various types of organizations. LAHIA should have internal working sessions to develop communications and outreach procedures and processes to involve all partners to the degree that they feel comfortable. It may be useful to hold conferences in Maradi for committees to interact with committee members from other LAHIA villages. These conferences could be organized with learning sessions, activities and exchanges. It will reinvigorate members. For example, a learning activity on committee operating agreements would reinforce the standards by which groups operate, such the practicality of term limits. The activities could be to revisit and revise committee operating agreements and then share ideas with committees from other villages. Institutional Capacity The seven evaluation questions are found in Table 32. Description: The LAHIA project intended to build early warning systems of the government at the commune and community levels. “SC’s efforts to integrate Household Economic Approach (HEA) into the national early warning system Systeme d’Alerte Précoce/Cellule de Crise Alimentaire will continue, with GoN staff, to strengthen commune level early warning systems, plans and crisis response units, improve information systems, and provide Food for Work (FFW) resources for community-prioritized mitigation activities. Community-level emergency preparedness and mitigation capacity will be built by training and support to Village Development Committees (VDCs).”28 28 Save the Children. (Jun 2012) Livelihoods, Agriculture and Health Interventions in Action LAHIA) Multi-Year Assistance Program Proposal. LAHIA Midterm Evaluation Report ACTIVITY DESCRIPTION, FINDINGS AND DISCUSSION 49 Findings: The SCAP/RU said they received computer equipment from LAHIA as capacity building. Neither local government nor local NGOs reported receiving skills building from LAHIA. Project reports documented gender training. The only example uncovered during the evaluation was that one VDC used the early warning system to alert commune officials of a seed shortage and received seeds as requested. Discussion: There may be a disconnect between what commune early warning system staff and LAHIA considers training, or respondents forgot, even when probed. Providing computer equipment and one training does contribute towards the capacity building efforts approved in the project proposal but does not integrate SC’s HEA approach into the government early warning system. More work is needed to reach proposed objectives for strengthening the early warning system. Recommendations: FFP programs should consider hiring an early warning system specialist (shared position). This specialist’s salary could be shared amongst all of the DFAPs with duties extending to all districts in which Title II activities are funded. This specialist would be tasked with conducting a needs assessment of targeted communes, creating an implementation plan and ultimately training early warning system staff. Care should be taken to coordinate with other donors and the appropriate government agencies. LAHIA Midterm Evaluation Report RECOMMENDATIONS 50 RECOMMENDATIONS The report is peppered with recommendations as they pertain to specific evaluation questions. The three overarching recommendations are; a) invest in relationships; b) focus on institutional strengths; and c) sub-contract activities for which LAHIA is performing poorly. The specific programmatic recommendations presented in the ACTIVITY DESCRIPTION, FINDINGS AND DISCUSSION are summarized below each of the related overarching recommendations. Invest in Relationships for Sustainability The relationships with government and community-based management organizations (CDV, water point, warehouse receipts, etc.) are the foundation for any sustainability. These relationships need to be fostered so the project can ensure continued food security. Clarify standard operating procedures, expectations, and responsibilities with all partners. Do not get stuck in the middle of government disputes during these negotiations. Commune, department, and regional levels of government technical services should all agree on how to work with LAHIA. One way to do this is to negotiate agreement with representatives of all sitting together in one room Treat community management groups as partners. They share the risks and benefits of project outcomes with Save the Children and World Vision. Treating community management groups as partners means that they have clear responsibilities for project implementation, as does LAHIA. These partners are true collaborators. Hold management committee conferences with time for learning workshops and information sharing sessions. Workshops should cover topics such as group formation, group management, and reporting to LAHIA. Information sessions can cover topics that are of interest and importance to the groups such as member retention, accountability, and election. Focus on Institutional Strengths Go long and go far by continuing to do stellar work in transmitting health and nutrition messages and building WASH infrastructure. Keep doing what you already know how to do. Sub-Contract Activities Outside of Institutional Strengths Select two or three types of businesses to support and find a partner with value chain experience to implement the activities. Sub-contract agricultural extension to a reputable organization with experience providing farmers in the Sahel with practical skills. Sub-contractors could be used to train LAHIA farmers directly or to train LAHIA field agents. Both the training for farmers and value chain work could go to the same organization. LAHIA Midterm Evaluation Report CONCLUSIONS 51 CONCLUSIONS As with nearly all projects, this project has strengths and weaknesses. The strengths lay in three activities; building WASH infrastructure; distributing food rations; and changing health behaviors. These three activities have strong planning and management in common. Beneficiary inclusion criteria are clear, the activity goals are well defined, and the definition of a successfully completed activity are laid out. They should continue to change lives through these activities. If the project only had three activities it would still be positively impacting the lives of the people it touches. On a personal level, health and nutrition messages can be considered sustainable. One a community level, WASH infrastructure is sustainable. These activities have social community and government support networks which enhances the possibilities for continuing to follow behavior change messages and for caring for pumps. The other LAHIA activities were less well defined and not as advanced. These activities included long￾term asset growth, agriculture technical skills building, gender life skills, and developing community management groups. These activities lacked clear focus and direction. The project needs to focus in scope and depth. If the project cannot figure out how to implement these activities with the same level of quality and consistency as the successful activities, they should be abandoned. The project has raised expectations, in the communities they serve, to help in these ways. LAHIA Midterm Evaluation Report REFERENCES AND APPENDICES 52 REFERENCES AND APPENDICES LAHIA Midterm Evaluation Report REFERENCES AND APPENDICES 53 Appendix A: Indicator Performance Tracking Table LAHIA Midterm Evaluation Report REFERENCES AND APPENDICES 54 Appendix B: List of Villages Where LAHIA Works LAHIA Midterm Evaluation Report REFERENCES AND APPENDICES 55 Appendix C: Scope of Work LAHIA Midterm Evaluation Report REFERENCES AND APPENDICES 56 Appendix D: List of Respondents LAHIA Midterm Evaluation Report REFERENCES AND APPENDICES 57 Appendix E: LAHIA Mid-Term Evaluation Plan LAHIA Midterm Evaluation Report REFERENCES AND APPENDICES 58 Appendix F: Evaluation Calendar LAHIA Midterm Evaluation Report REFERENCES AND APPENDICES 59 Appendix G: Tables of Evaluation Questions Table 11 LAHIA Mid-Term Evaluation Questions OVERALL 1. At the time of the MTE, what are LAHIA program’s largest successes, and main obstacles the program is facing in meetings its overall program objective of “Food insecurity and malnutrition reduced among poor rural households in Aguié and Guidan Roumdji Districts of Maradi Region, Niger” 2. At the time of the MTE, what are LAHIA program’s largest successes, and main obstacles the program is facing in meetings its overall program objective of “Food insecurity and malnutrition reduced among poor rural households in Aguié and Guidan Roumdji Districts of Maradi Region, Niger” 3. Which current strategies should be continued to meet the overall program objective(s)? 4. Which current LAHIA strategies should be altered and/or course corrected to meet the overall program objective(s)? 5. Within each strategy, and across the program, how well has the program implemented cross-cutting activities (commodity management, gender, M&E)? 6. Within each strategy, and across the program, how well has the program implemented cross-cutting activities (commodity management, gender, M&E)? 7. Within each strategy, and across the program, how well has the program implemented cross-cutting activities (commodity management, gender, M&E)? 8. Within each strategy, and across the program, how well has the program implemented cross-cutting activities (commodity management, gender, M&E)? 9. Within each strategy, and across the program, how well has the program implemented cross-cutting activities (commodity management, gender, M&E)? 10. Within each strategy, and across the program, how well has the program implemented cross-cutting activities (commodity management, gender, M&E)? 11. What adjustments (in terms of value chain, natural resource management, livestock) must be done to align LAHIA DFAP with the USAID resilience program in Niger? 12. What adjustments (in terms of value chain, natural resource management, livestock) must be done to align LAHIA DFAP with the USAID resilience program in Niger? 13. What adjustments (in terms of value chain, natural resource management, livestock) must be done to align LAHIA DFAP with the USAID resilience program in Niger? TECHNICAL APPROACH BY SECTOR AND CROSS-CUTTING Agriculture and Livelihoods 14. What are farmers’ (men, women, and youth) perceptions about the LAHIA-promoted technologies and practices and their perceived benefits? 15. What are farmers’ (men, women, and youth) perceptions about the LAHIA-promoted technologies and practices and their perceived benefits? 16. To what extend do the beneficiary farmers (men, women and youth) implement the techniques proposed by the program in their own fields? 17. What are the factors that limit or promote the adoption of those techniques? 18. What are the factors that limit or promote the adoption of those techniques? 19. Is there evidence that LAHIA is on track for promoting its selected value chains? 20. LAHIA uses the value chain development approaches (facilitation, vertical/horizontal integration input access); are LAHIA’s small grants for women in terms of improving individual women’s revenue? 21. Can microfinance respond to the cash needs of value chain actors? 22. Are there alternative solutions? Health, Nutrition, Water Sanitation and Hygiene 23. Do targeted beneficiaries receive appropriate health and nutritional messages? 24. Are the project staff and volunteers adequately trained on the subject matter? 25. Are the project staff and volunteers adequately trained on the subject matter? 26. Do the facilitators have adequate knowledge on the subject matter? LAHIA Midterm Evaluation Report REFERENCES AND APPENDICES 60 Table 11 LAHIA Mid-Term Evaluation Questions 27. To what extent are infant feeding practices being adopted? 28. Is there evidence that some pregnant women and children under five are not receiving rations? 29. Is the LAHIA communication strategy for behavior change efficient? 30. Are the water management committees that have been formed in villages functioning and effective? 31. Are CLTS interventions proving effective in motivating beneficiaries to improve their proposal and communal hygiene and sanitation practices? 32. Are community led total sanitation (CLTS) interventions proving effective in motivating beneficiaries to improve their proposal and communal hygiene and sanitation practices? 33. Are the water points that have been constructed sustainable and appropriate? 34. Are the water points that have been constructed sustainable and appropriate? 35. Do the trained artisans have the skills necessary to fix the new pumps/water systems? Cross-Cutting: Gender 36. What has been the effect of LAHIA programming on the empowerment of women and adolescent girls within various social systems (within the community, in management organizations, within households and between couples)? 37. What has been the effect of LAHIA programming on the empowerment of women and adolescent girls within various social systems (within the community, in management organizations, within households and between couples)? 38. What has been the effect of LAHIA programming on the empowerment of women and adolescent girls within various social systems (within the community, in management organizations, within households and between couples)? 39. What has been the effect of LAHIA programming on the empowerment of women and adolescent girls within various social systems (within the community, in management organizations, within households and between couples)? 40. Have gendered relationships within the households been impacted? 41. Are there specific gendered components of household decisions which have changed? 42. Do women feel they have more, less or the same level of decision-making ability, or voice? 43. How has participation in project activities impacted women (positively and negatively) 44. How has participation in project activities impacted women (positively and negatively) 45. How have project activities impacted boys and girls? 46. What is the progress to-date on partners implementing and influencing gender norms? 47. What is the progress to-date on partners implementing and influencing gender norms? 48. Have there been any unexpected barriers, or unintended consequences, and how well have partners been able to address any challenges? 49. Have there been any unexpected barriers, or unintended consequences, and how well have partners been able to address any challenges? 50. Have there been any unexpected barriers, or unintended consequences, and how well have partners been able to address any challenges? Cross-Cutting: Monitoring & Evaluation 51. How harmonized is the program’s M&E system? 52. Are there areas which can be improved? Cross-Cutting: Commodity Management 53. How well is LAHIA managing commodities and ensuring the recipients are receiving their rations? 54. How well is LAHIA managing commodities and ensuring the recipients are receiving their rations? Cross-Cutting: Environmental Safeguard and Compliance 55. How effectively is fumigation equipment being used in commodity warehouses? 56. How is water quality in wells being protected? 57. Are animals and humans sharing the same water wells? 58. Are buffer zones being used? 59. Are pesticides being used for pest management and, if so, have any illnesses been observed or reported? 60. Are pesticides being used for pest management and, if so, have any illnesses been observed or reported? LAHIA Midterm Evaluation Report REFERENCES AND APPENDICES 61 Table 11 LAHIA Mid-Term Evaluation Questions 61. Have LAHIA implementers received training on environmental safeguards and compliance? 62. What type of environmental expertise does the program team have on staff? 63. How have community members been engaged for long-term environmental/sustainable management of program implementation? IMPLEMENTATION OF ACTIVITIES Specifically through the lens of targeting, progress-to-date, quality, sustainability and commodity management Targeting 64. How well has LAHIA identified and reached its target population? 65. How well has LAHIA identified and reached its target population? 66. How successful has the targeting during the “lean season” programming? Progress 67. To what extent has LAHIA made progress and achievements relative to the set objectives, including cross￾cutting components and intermediate results? 68. To what extent has LAHIA made progress and achievements relative to the set objectives, including cross￾cutting components and intermediate results? 69. To what extent has LAHIA made progress and achievements relative to the set objectives, including cross￾cutting components and intermediate results? 70. Is the program on-track to meet its expected results and achievements? 71. Are the program targets realistic and achievable? Quality 72. To what extent are LAHIA interventions providing quality services that are appropriate to the needs of the households? Sustainability 73. Which activities or programmatic aspects of each purpose have the greatest likelihood for sustainability? 74. Are there areas of importance that need to be strengthened in order to achieve greater sustainability? 75. How has the program strengthened capacities of locally elected representatives, members of the government technical service to better meet the requirements of the service? 76. How has the program strengthened capacities of locally elected representatives, members of the government technical service to better meet the requirements of the service? Management 77. What are efficiencies in the management of the program, resource allocation and the intervention area that can be improved? 78. What are efficiencies in the management of the program, resource allocation and the intervention area that can be improved? 79. What are efficiencies in the management of the program, resource allocation and the intervention area that can be improved? Partnership The LAHIA program has several activity implementing partners as described below. For each partner category, as well as other developmental actors, the MTE should assess the strengths and weaknesses of these partnerships; Identify factors which contributed more to the successes of failures of these partnerships; assess partner satisfaction with the LAHIA program; and identify constraints and challenges which consortia members face. Implementing Partners World Vision International is a co-implementing partner for LAHIA program activities 80. assess partner satisfaction with the LAHIA program 81. assess the strengths of these partnerships 82. assess the weaknesses of these partnerships 83. identify constraints which consortia members face 84. identify challenges which consortia members face 85. identify factors which contributed more to the successes of failures of these partnerships 86. analyze the effectiveness of this partnership Government Technical Services LAHIA Midterm Evaluation Report REFERENCES AND APPENDICES 62 Table 11 LAHIA Mid-Term Evaluation Questions agriculture, health, livestock, environment, rural engineering, and hydraulics as essential partners to the success of the activities for a sustainability approach; thorough scrutiny of these partners must be conducted by the MTE team 87. assess partner satisfaction with the LAHIA program 88. assess the strengths of these partnerships 89. assess the weaknesses of these partnerships 90. identify constraints which consortia members face 91. identify challenges which consortia members face 92. identify factors which contributed more to the successes of failures of these partnerships 93. analyze the effectiveness of this partnership Government Municipal Authorities local elected officials and town representatives as essential partners to the success of the activities for a sustainability approach; thorough scrutiny of these partners must be conducted by the MTE team 94. assess partner satisfaction with the LAHIA program 95. assess the strengths of these partnerships 96. assess the weaknesses of these partnerships 97. identify constraints which consortia members face 98. identify challenges which consortia members face 99. identify factors which contributed more to the successes of failures of these partnerships 100.analyze the effectiveness of this partnership Government Administrative Authorities préfets, Governors, Initiative les Nigeriens Nourrissent les Nigeriens (3N) as essential partners to the success of the activities for a sustainability approach; thorough scrutiny of these partners must be conducted by the MTE team 101.assess partner satisfaction with the LAHIA program 102.assess the strengths of these partnerships 103.assess the weaknesses of these partnerships 104.identify constraints which consortia members face 105.identify challenges which consortia members face 106.identify factors which contributed more to the successes of failures of these partnerships 107.analyze the effectiveness of this partnership Technical Implementing Partners SongES for the Husband’s Schools, Asusu (Saving Association in Hausa) for micro-credit activities and INRAN (Institut National des Recherches Agronomiques du Niger) for improved seeds and distribution networks, are important for the LAHIA program. 108.assess partner satisfaction with the LAHIA program 109.assess the strengths of these partnerships 110.assess the weaknesses of these partnerships 111.identify constraints which consortia members face 112.identify challenges which consortia members face 113.identify factors which contributed more to the successes of failures of these partnerships 114.analyze the effectiveness of this partnership Other Development Actors including but not limited to other FFP funded programs 115. The quality and extent of the cooperation and coordination between LAHIA and other development partners, the, in the region must be assessed. 116.Suggested question: How well has LAHIA communicated, collaborated, layered, sequenced, and/or integrated with other USAID/FFP programs? 117.assess partner satisfaction with the LAHIA program 118.assess the strengths of these partnerships 119.identify constraints which consortia members face 120.identify challenges which consortia members face 121.identify factors which contributed more to the successes of failures of these partnerships LAHIA Midterm Evaluation Report REFERENCES AND APPENDICES 63 Table 11 LAHIA Mid-Term Evaluation Questions 122.assess the weaknesses of these partnerships 123.analyze the effectiveness of this partnership Institutional Capacity 124. To what extent has LAHIA made progress in strengthening local Government technical services, local NGOs and governance? 125.To what extent has LAHIA made progress in strengthening local Government technical services, local NGOs and governance? 126.Institutional capacity. To what extent has LAHIA made progress in strengthening local Government technical services, local NGOs and governance? Note: The mid-term evaluation scope of work contains approximately 120 evaluation questions when applied to the client’s sub-topics and after separating compound questions. Some questions ask whether or not the project activities should have been included in the design but most ask about process and outcomes. Table 12 Overall Evaluation Questions 1. At the time of the MTE, what are LAHIA program’s largest successes in meetings its overall program objective of “Food insecurity and malnutrition reduced among poor rural households in Aguié and Guidan Roumdji Districts of Maradi Region, Niger” 2. At the time of the MTE, what are LAHIA program’s main obstacles in meetings its overall program objective of “Food insecurity and malnutrition reduced among poor rural households in Aguié and Guidan Roumdji Districts of Maradi Region, Niger” 3. Which current strategies should be continued to meet the overall program objective(s)? 4. Which current LAHIA strategies should be altered and/or course corrected to meet the overall program objective(s)? 5. Within each strategy, how well has the program implemented (commodity management)? 6. across the program, how well has the program implemented (commodity management)? 7. Within each strategy, how well has the program implemented (gender)? 8. across the program, how well has the program implemented (gender)? 9. Within each strategy, how well has the program implemented (M&E)? 10. across the program, how well has the program implemented (M&E)? 11. What adjustments (in terms of value chain) must be done to align LAHIA DFAP with the USAID resilience program in Niger? 12. What adjustments (in terms of natural resource management) must be done to align LAHIA DFAP with the USAID resilience program in Niger? 13. What adjustments (in terms of livestock) must be done to align LAHIA DFAP with the USAID resilience program in Niger? Table 13 SO2 Agriculture Evaluation Questions 14. What are farmers’ (men, women, and youth) perceptions about the LAHIA-promoted technologies and practices? 15. What are farmers’ (men, women, and youth) LAHIA-promoted technologies and practices perceived benefits? 16. To what extend do the beneficiary farmers (men, women and youth) implement the techniques proposed by the program in their own fields? 17. What are the factors that limit the adoption of those techniques? 18. What are the factors that promote the adoption of those techniques? Table 14 SO2 Livelihoods Evaluation Questions 19. Is there evidence that LAHIA is on track for promoting its selected value chains? 20. LAHIA uses the value chain development approaches (facilitation, vertical/horizontal integration input access); are LAHIA’s small grants for women in terms of improving individual women’s revenue? 21. Can microfinance respond to the cash needs of value chain actors? LAHIA Midterm Evaluation Report REFERENCES AND APPENDICES 64 22. Are there alternative solutions? Table 15 Ration Mid-Term Evaluation Question 28. Is there evidence that some pregnant women and children under five are not receiving rations? Table 16 Health & Nutrition Mid-Term Evaluation Questions 23. Do targeted beneficiaries receive appropriate health and nutritional messages? 24. Are the project staff adequately trained on the subject matter? 25. Are the volunteers adequately trained on the subject matter? 26. Do the facilitators have adequate knowledge on the subject matter? 27. To what extent are infant feeding practices being adopted? 29. Is the LAHIA communication strategy for behavior change efficient? Table 17 Community-Led Total Sanitation Mid-Term Evaluation Questions 30. Are CLTS interventions proving effective in motivating beneficiaries to improve their proposal? 31. Are CLTS interventions proving effective in motivating beneficiaries to improve their communal hygiene and sanitation practices? Table 18 Water and Sanitation, Hygiene Mid-Term Evaluation Questions 30. Are the water management committees that have been formed in villages functioning and effective? 32. Are the water points that have been constructed appropriate? 33. Are the water points that have been constructed sustainable? 34. Do the trained artisans have the skills necessary to fix the new pumps/water systems? Table 19 Gender Mid-Term Evaluation Questions 36. What has been the effect of LAHIA programming on the empowerment of women and adolescent girls within (within the community)? 37. What has been the effect of LAHIA programming on the empowerment of women and adolescent girls within (in management organizations)? 38. What has been the effect of LAHIA programming on the empowerment of women and adolescent girls within (households)? 39. What has been the effect of LAHIA programming on the empowerment of women and adolescent girls (between couples)? 40. Have gendered relationships within the households been impacted? 41. Are there specific gendered components of household decisions which have changed? 42. Do women feel they have more, less or the same level of decision-making ability, or voice? 43. How has participation in project activities impacted women (positively) 44. How has participation in project activities impacted women (negatively) 45. How have project activities impacted boys and girls? 46. What is the progress to-date on partners implementing gender norms? 47. What is the progress to-date on partners influencing gender norms? 48. Have there been any unexpected barriers? 49. Have there been any unintended consequences? 50. How well have partners been able to address any challenges? Table 20 Monitoring and Evaluation Mid-Term Evaluation Questions 51. How harmonized is the program’s M&E system? 52. Are there areas which can be improved? Table 21 Commodity Management Mid-Term Evaluation Questions 53. How well is LAHIA managing commodities? 54. How well is LAHIA ensuring the recipients are receiving their rations? LAHIA Midterm Evaluation Report REFERENCES AND APPENDICES 65 Table 22 Environmental Compliance Mid-Term Evaluation Questions 55. How effectively is fumigation equipment being used in commodity warehouses? 56. How is water quality in wells being protected? 57. Are animals and humans sharing the same water wells? 58. Are buffer zones being used? 59. Are pesticides being used for pest management? 60. if so, have any illnesses been observed or reported? 61. Have LAHIA implementers received training on environmental safeguards and compliance? 62. What type of environmental expertise does the program team have on staff? 63. How have community members been engaged for long-term environmental/sustainable management of program implementation? Table 23 Targeting Mid-Term Evaluation Questions 64. How well has LAHIA identified its target population? 65. How well has LAHIA reached its target population? 66. How successful has the targeting during the “lean season” programming? Table 24 Progress Mid-Term Evaluation Questions 67. To what extent has LAHIA made progress and achievements relative to the set objectives? 68. To what extent has LAHIA made progress and achievements relative to cross-cutting components? 69. To what extent has LAHIA made progress and achievements relative to intermediate results? 70. Is the program on-track to meet its expected results and achievements? 71. Are the program targets realistic and achievable? Table 25 Quality Mid-Term Evaluation Questions 72. To what extent are LAHIA interventions providing quality services that are appropriate to the needs of the households? Table 26 Sustainability Mid-Term Evaluation Questions 73. Which activities or programmatic aspects of each purpose have the greatest likelihood for sustainability? 74. Are there areas of importance that need to be strengthened in order to achieve greater sustainability? 75. How has the program strengthened capacities of locally elected representatives to better meet the requirements of the service? 76. How has the program strengthened capacities of members of the government technical service to better meet the requirements of the service? Table 27 Management Mid-Term Evaluation Questions 77. What are efficiencies in the management of the program that can be improved? 78. What are efficiencies in, resource allocation that can be improved? 79. What are efficiencies in the management of the intervention area that can be improved? Table 28 Implementing Partner Mid-Term Evaluation Questions 80. assess partner satisfaction with the LAHIA program 81. assess the strengths of these partnerships 82. assess the weaknesses of these partnerships 83. identify constraints which consortia members face 84. identify challenges which consortia members face 85. identify factors which contributed more to the successes of failures of these partnerships 86. analyze the effectiveness of this partnership LAHIA Midterm Evaluation Report REFERENCES AND APPENDICES 66 Table 29 Government Mid-Term Evaluation Questions 87. assess partner satisfaction with the LAHIA program 88. assess the strengths of these partnerships 89. assess the weaknesses of these partnerships 90. identify constraints which consortia members face 91. identify challenges which consortia members face 92. identify factors which contributed more to the successes of failures of these partnerships 93. analyze the effectiveness of this partnership LAHIA Midterm Evaluation Report REFERENCES AND APPENDICES 67 Table 30 Local Implementing Partner Mid-Term Evaluation Questions 108. assess partner satisfaction with the LAHIA program 109. assess the strengths of these partnerships 110. assess the weaknesses of these partnerships 111. identify constraints which consortia members face 112. identify challenges which consortia members face 113. identify factors which contributed more to the successes of failures of these partnerships 114. analyze the effectiveness of this partnership Table 31 Other Development Actor Mid-Term Evaluation Questions 115. The quality and extent of the cooperation and coordination between LAHIA and other development partners, the, in the region must be assessed. 116. Suggested question: How well has LAHIA communicated, collaborated, layered, sequenced, and/or integrated with other USAID/FFP programs? 117. assess partner satisfaction with the LAHIA program 118. assess the strengths of these partnerships 119. identify constraints which consortia members face 120. identify challenges which consortia members face 121. identify factors which contributed more to the successes of failures of these partnerships 122. assess the weaknesses of these partnerships 123. analyze the effectiveness of this partnership Table 32 Institutional Capacity Mid-Term Evaluation Questions 124. To what extent has LAHIA made progress in strengthening local Government technical services? 125. To what extent has LAHIA made progress in strengthening local NGOs? 126. Institutional capacity. To what extent has LAHIA made progress in strengthening governance?