July 19, 2018 This report was produced for review by ACDI/VOCA and the United States Agency for International Development. It was prepared by NORC at the University of Chicago. DRAFT FINAL REPORT FINAL EVALUATION OF VICTORY AGAINST MALNUTRITION (ViM) PROJECT, BURKINA FASO July 19, 2018 This report was produced for review by ACDI/VOCA and the United States Agency for International Development. It was prepared by NORC at the University of Chicago. DRAFT FINAL REPORT FINAL EVALUATION OF VICTORY AGAINST MALNUTRITION (ViM) PROJECT, BURKINA FASO July 19, 2018 This report was produced for review by ACDI/VOCA and the United States Agency for International Development. It was prepared by NORC at the University of Chicago. DRAFT FINAL REPORT FINAL EVALUATION OF VICTORY AGAINST MALNUTRITION (ViM) PROJECT, BURKINA FASO July 19, 2018 This report was produced for review by ACDI/VOCA and the United States Agency for International Development. It was prepared by NORC at the University of Chicago. FINAL REPORT FINAL EVALUATION OF VICTORY AGAINST MALNUTRITION (ViM) PROJECT, BURKINA FASO FINAL REPORT: FINAL EVALUATION OF VICTORY AGAINST MALNUTRITION (ViM) PROJECT, BURKINA FASO July 19, 2018 Contracted under ACDI/VOCA WORK ORDER NO: J0679-06 Prepared by: Dr. Lauren Persha, NORC at the University of Chicago Dr. Shanto Sadhu, NORC at the University of Chicago Dr. Audra Grant, NORC at the University of Chicago Dr. Eric Weiss, Consultant Phocas Ntahorugiye, Consultant Qualitative and quantitative research support provided by: Sarah Lord, NORC at the University of Chicago Letitia Onyango, NORC at the University of Chicago Samantha Downey, NORC at the University of Chicago Ilse Paniagua, NORC at the University of Chicago Alexander Rigaux, NORC at the University of Chicago Stacy Pancratz, NORC at the University of Chicago Submitted by: NORC at the University of Chicago Cover Photo Credit: Phocas Ntahorugiye DISCLAIMER The authors’ views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. FINAL REPORT – VICTORY AGAINST MALNUTRITION (ViM) FINAL EVALUATION Contents LIST OF FIGURES....................................................................................................................................4 LIST OF TABLES......................................................................................................................................4 EXECUTIVE SUMMARY................................................................................................................................................... I INTRODUCTION AND EVALUATION BACKGROUND................................................................................................. I EVALUATION QUESTIONS ................................................................................................................................. I PROGRAM DESCRIPTION................................................................................................................................. II THEORY OF CHANGE ...................................................................................................................................... II EVALUATION METHODOLOGY..........................................................................................................................III EVALUATION LIMITATIONS AND CONSTRAINTS................................................................................................... IV SUMMARY FINDINGS AND CONCLUSIONS........................................................................................................... V SECTOR-SPECIFIC RECOMMENDATIONS: MCHN AND WASH.............................................................................. IX SECTOR-SPECIFIC RECOMMENDATIONS: AGRICULTURE AND LIVELIHOODS............................................................... X 1. INTRODUCTION, EVALUATION PURPOSE AND QUESTIONS.................................................. 1 1.1 INTRODUCTION AND EVALUATION BACKGROUND....................................................................................1 1.2 EVALUATION QUESTIONS ....................................................................................................................1 2. PROJECT BACKGROUND......................................................................................................................... 3 2.1 COUNTRY CONTEXT ...........................................................................................................................3 2.2 PROGRAM DESCRIPTION.....................................................................................................................4 2.3 THEORY OF CHANGE ..........................................................................................................................5 2.4 BASELINE AND MIDTERM CONTEXT ......................................................................................................6 3. EVALUATION METHODOLOGY ............................................................................................................ 8 3.1 QUANTITATIVE DATA COLLECTION AND ANALYSIS: PBS ...........................................................................8 3.1.1 Sampling and Data Collection...............................................................................................8 3.1.2 Analysis of Quantitative Data .............................................................................................10 3.2 QUALITATIVE DATA COLLECTION AND ANALYSIS ...................................................................................12 3.2.1 Group Discussions...............................................................................................................13 3.2.2 Key Informant Interviews....................................................................................................13 3.2.3 Village Selection for Qualitative Data Collection ................................................................13 3.2.4 Desk Review of IP and Secondary Documentation..............................................................13 3.2.5 Qualitative Data Analysis....................................................................................................13 3.3 EVALUATION LIMITATIONS AND CONSTRAINTS......................................................................................14 3.3.1 Lack of Valid Counterfactual for Quantitative PBSs............................................................14 3.3.2 Small Sample Size for Target Value Chain Crops.................................................................14 3.3.3 Difference in Seasonal Timing of Data Collection between Baseline and Endline PBSs......14 3.3.5 Recall Bias in Qualitative Data Collection ...........................................................................16 3.3.6 Selection Bias for Qualitative Data Collection.....................................................................16 3.4 DESCRIPTIVE SUMMARY OF POPULATION CHARACTERISTICS.....................................................................17 4. FINDINGS AND CONCLUSIONS ........................................................................................................17 4.1 EQ 1: TO WHAT EXTENT WAS THE VIM PROJECT EFFECTIVE IN ACHIEVING STRATEGIC OBJECTIVES AND INTERMEDIATE RESULTS?..................................................................................................................17 4.1.1 Integrated Quantitative and Qualitative Findings by Sector and Strategic Objective: Project Achievements and Key Factors, Overall and by Subgroups....................................................17 Strategic Objective 1: Increased and Diversified Agricultural Production ..........................................19 FINAL REPORT – VICTORY AGAINST MALNUTRITION (ViM) FINAL EVALUATION Strategic Objective 2: Increased household incomes..........................................................................26 Strategic Objective 3: Reduced chronic malnutrition among children under five years of age and pregnant and lactating mothers (PLW) ..............................................................................................30 Crosscutting: Gender...........................................................................................................................38 Crosscutting: Environment..................................................................................................................40 4.2 EQ2: EFFECTIVENESS OF VIM IN PROMOTING IMPROVED AGRICULTURAL AND LIVESTOCK PRACTICES AND INCREASING AND DIVERSIFYING SMALLHOLDER AGRICULTURAL PRODUCTION. ............................................41 4.2.1 Descriptive statistical analysis ............................................................................................42 Beneficiary / Non-Beneficiary Comparisons .......................................................................................42 4.2.2 Econometric Analysis..........................................................................................................44 4.2.3 Surveyed Farmer Perceptions of Factors Influencing Crop Yields and Income ..........................50 CONCLUSIONS .....................................................................................................................................53 4.3 EQ3: EFFECTIVENESS OF VIM PROJECT IN PROMOTING PRACTICES AROUND INFANT AND YOUNG CHILD FEEDING, MATERNAL AND CHILD HEALTH; WATER, SANITATION AND HYGIENE IN REDUCING MALNUTRITION AMONG CHILDREN UNDER FIVE, AND PREGNANT AND LACTATING WOMEN............................................................54 ECONOMETRIC ANALYSIS....................................................................................................................56 CONCLUSIONS .....................................................................................................................................63 4.4 EQ 4: EFFECTIVENESS OF THE STRATEGIES IMPLEMENTED BY VIM PROJECT TO ENSURE THAT PRIVATE/PUBLIC PROVIDERS OF AGRICULTURE, LIVESTOCK, HEALTH, WOMEN'S EMPOWERMENT, AND NUTRITION SERVICES, COMMUNITY ACTORS INVOLVED IN HEALTH AND NUTRITION PROMOTION WILL CONTINUE TO PROVIDE SERVICES AFTER THE END OF VIM?..................................................................................................................63 4.4.1 What is the level of motivation of the service providers to continue providing services after the project ends, and likelihood of sustained service provision after the end of ViM? ......................63 4.4.2 What are the technical, organizational, and management capacities of the producer groups developed by ViM project and their potential effectiveness to provide sustained service delivery after the end of ViM? ............................................................................................................64 4.4.3 What has been done by ViM project to ensure that the service providers will have continuous access to resources to enhance their capacities?.............................................................65 CONCLUSIONS .....................................................................................................................................66 4.5 EQ 5: LEVEL OF DEMAND AMONG THE COMMUNITY MEMBERS AND WILLINGNESS TO PAY FOR ESSENTIAL AGRICULTURAL, LIVESTOCK, HEALTH AND NUTRITION SERVICES PROVIDED BY THE PROJECT...........................66 CONCLUSIONS .....................................................................................................................................70 4.6 EQ6: EFFECTIVENESS OF THE INTERVENTIONS IMPLEMENTED BY VIM TO STRENGTHEN HOUSEHOLDS’ AND COMMUNITIES’ RESILIENCE TO IDIOSYNCRATIC AND CO-VARIATE SHOCKS?.................................................70 CONCLUSIONS .....................................................................................................................................72 4.7 EQ7: HOW WELL DID VIM TAKE ADVANTAGE OF THE OTHER USG INVESTMENTS THAT PROVIDE COMPLEMENTARY SERVICES NECESSARY TO ACHIEVE VIM’S GOAL?..........................................................72 4.8 EQ8: HOW EFFECTIVE ARE THE APPROACHES USED BY VIM PROJECT TO ADDRESS GENDER GAPS AND EMPOWER WOMEN?.......................................................................................................................................73 ECONOMETRIC ANALYSIS....................................................................................................................74 CONCLUSIONS .....................................................................................................................................75 4.9 EQ9: PARTNERSHIPS AND EXIT STRATEGY: HOW EFFECTIVE WERE THE PARTNERSHIPS DEVELOPED WITH GOBF AT NATIONAL, COMMUNE, AND VILLAGE LEVEL AS WELL AS OTHER DEVELOPMENT ACTORS IN DELIVERING INTENDED RESULTS ALIGNED WITH COUNTRY DEVELOPMENT OBJECTIVES? WAS THERE A WELL-DEVELOPED EXIT STRATEGY EFFECTIVELY COMMUNICATED?..........................................................................................................75 4.10 EQ10: UNINTENDED CONSEQUENCES AND LESSONS LEARNED: WHAT ARE THE UNINTENDED POSITIVE AND/OR NEGATIVE CONSEQUENCES OF THE VIM PROJECT? WHAT LESSONS CAN BE LEARNED TO MINIMIZE UNINTENDED NEGATIVE CONSEQUENCES IN THE DESIGN OF FUTURE PROJECTS? ............................................................78 FINAL REPORT – VICTORY AGAINST MALNUTRITION (ViM) FINAL EVALUATION 4.11 EQ 11: HOW EFFECTIVE ARE THE VALUE CHAIN APPROACHES USED BY VIM PROJECT TO PROMOTE COWPEAS AND SMALL LIVESTOCK VALUE CHAIN IN THE TARGET COMMUNITIES? .......................................................79 Cowpea Value Chain Activities............................................................................................................80 Beneficiary / Non-Beneficiary Comparisons .......................................................................................80 Livestock Value Chain Activities..........................................................................................................81 Beneficiary / Non-Beneficiary Comparisons .......................................................................................82 5. RECOMMENDATIONS..............................................................................................................................83 5.1 SECTOR-SPECIFIC RECOMMENDATIONS: MCHN AND WASH.................................................................83 5.2 SECTOR-SPECIFIC RECOMMENDATIONS: AGRICULTURE AND LIVELIHOODS.................................................84 FINAL REPORT – VICTORY AGAINST MALNUTRITION (ViM) FINAL EVALUATION LIST OF FIGURES Figure 1: ViM Intervention Zone.................................................................................................................................... 5 Figure 2: Burkina Faso Seasonal Calendar (Typical Year)......................................................................................15 Figure 3: Changes in Crop Production (by Percentage of Farmers reporting production)...........................22 Figure 4: Comparison of Agricultural Practices Adopted......................................................................................23 Figure 5: Comparison of Household Food Consumption by Individual Food Groups...................................26 Figure 6: Comparison of Prevalence of Moderate to Severe Hunger................................................................27 Figure 7: Comparison of Business Development Services.....................................................................................29 Figure 8: Comparison of Stunted and Underweight Children..............................................................................30 Figure 9: Stunting and Underweight Differences across Gender.........................................................................32 Figure 10: Comparison of Exclusively Breastfed Components.............................................................................33 Figure 11: Comparison of Women and Child Health and Nutrition Practices................................................34 Figure 12: Comparison of Quality of Health Services............................................................................................35 Figure 13: Comparison of Adoption of Hygiene and Sanitation Practices.........................................................37 Figure 14: Comparison of Female Empowerment...................................................................................................38 Figure 15: Comparison of Environmentally Sustainable Practices........................................................................40 Figure 16: Relative importance of different factors affecting crop yields...........................................................51 Figure 17: Relative importance of different factors in affecting income.............................................................52 LIST OF TABLES Table 1: Implementing Partner Zones of Coverage, and Number of Villages and Beneficiaries....................5 Table 2: Endline Household Survey Sample Structure............................................................................................10 Table 3: Beneficiary Status Determination by Indicator.........................................................................................11 Table 4: ViM Baseline and Endline Indicator Mean Values, and Baseline to Endline Indicator Comparisons. ..............................................................................................................................................................................................19 Table 5: Comparison of Endline Indicator Values for Beneficiaries and Non-beneficiaries...........................43 Table 6: The Effect of ViM Interventions on Outcome Indicators......................................................................46 Table 7: The Effect of ViM Interventions on the Adoption of GAP....................................................................48 Table 8: The Effect of the Adoption of GAP and Livestock Practices on Outcome Indicators...................49 Table 9: Effect of ViM Interventions on Malnutrition..............................................................................................59 Table 10: Effect of ViM Interventions on Hygiene Practices.................................................................................62 Table 11: Technologies or practices households intend to use after the end of ViM activities...................67 Table 12: Trainings households expect to continue to attend after the end of ViM intervention ..............68 Table 13: Effect of ViM Interventions on Women Empowerment......................................................................75 Table 14: Cowpea Value Chain Activity Rates.........................................................................................................80 Table 15: Comparison of Cowpea Value Chain Responses for Beneficiaries and Non-Beneficiaries.........80 Table 16: Livestock Value Chain Activity Rates.......................................................................................................81 Table 17: Comparison of Livestock Value Chain Responses for Beneficiaries and Non-Beneficiaries.......82 FINAL REPORT – VICTORY AGAINST MALNUTRITION (ViM) FINAL EVALUATION ACRONYMS ASBC Agent de Santé à Base Communautaire AGRODIA Association des Grossistes et Détaillants d’Intrants Agricoles APIL Action pour la Promotion des Initiatives Locales ATAD Alliance Technique d’Assistance au Développement AZND Association Zood Nooma pour le Développement CAPI Computer Assisted Personal Interviewing CBDF Coalition Burkinabè des Droits de la Femme COP Chief of Party CRDA Centre de Recherche sur le Développement Appliqué CTV Continuous Treatment Variable DFAP Development Food Assistance Program DH Demonstration Herd DO Development Objective ET Evaluation Team EBF Exclusive Breastfeeding EQ Evaluation Question FFP Food For Peace FFS Farmer Field Schools FY Fiscal Year GD Group Discussion GoBF Government of Burkina Faso HH Household IM Impact INERA Institut de l’Environnement et de Recherche Agricole IP Implementing Partners IR Intermediate Results IRB Internal Review Board KII Key Informant Interviews LIP Local Implementing Partner MCHN Mother and Child Health and Nutrition MLA Mother Leader Animator MYAP Multi Year Assistance Program FINAL REPORT – VICTORY AGAINST MALNUTRITION (ViM) FINAL EVALUATION M&E Monitoring and Evaluation NGO Non-Governmental Organization NORC NORC at the University of Chicago OC Outcome OCADES Organisation Catholique pour le Développement Economique et la Solidarité PBS Population-Based Survey PE Performance Evaluation PLW Pregnant and Lactating Women PM2A Preventing Malnutrition Under 2 Approach SC Save the Children SILC Saving and Internal Lending Community SNV Netherlands Development Organization SO Strategic Objectives SQ Sub-question SSL Secure Sockets Layer TL Team Leader USAID United States Agency for International Development USG United States Government ViM Victory Against Malnutrition VVV Volunteer Village Vaccinator WASH Water, Sanitation and Hygiene WHO World Health Organization i FINAL REPORT – VICTORY AGAINST MALNUTRITION (ViM) FINAL EVALUATION EXECUTIVE SUMMARY INTRODUCTION AND EVALUATION BACKGROUND This report details the findings and recommendations of an end-of-project performance evaluation (PE) of the Victory Against Malnutrition (ViM) project, as commissioned by ACDI/VOCA under funding provided by the United States Agency for International Development (USAID) Office of Food for Peace (FFP). ViM is implemented in four communes of Sanmatenga province in the Central North Region of Burkina Faso. The purpose of the final PE is to determine to what extent the food security status of the targeted population has changed and how the ViM project has contributed to the resiliency of the targeted communities. The PE has two components: (1) a quantitative component which draws on the results from a Population-Based Survey (PBS) administered at project baseline (July-August 2012) and endline (Sept-Oct 2017); and (2) a qualitative component which collects data on and analyzes changes in perception, knowledge and behavior of target beneficiaries. The evaluation was conducted from October 2017 – April 2018 by NORC at the University of Chicago (NORC). The findings, conclusions, and recommendations of the evaluation may be used by USAID to extract lessons learned and generate insights to inform the design of follow-on programming in Burkina Faso. EVALUATION QUESTIONS I. To what extent was the ViM project effective in achieving strategic objectives and intermediate results? II. Using quantitative analysis, evaluate the effectiveness of the ViM project in promoting improved agricultural and livestock practices and increasing and diversifying smallholder agricultural production. III. Assess the effectiveness of the ViM project in promoting practices around infant and young child feeding, maternal and child health; water, sanitation and hygiene (WASH) in reducing malnutrition among children under five, and pregnant and lactating women. IV. Assess the effectiveness of the strategies implemented by the ViM project to ensure that private/public providers of agriculture, livestock, health, women's empowerment, and nutrition services will continue. Will community actors involved in health and nutrition promotion continue to provide services after the end of the ViM? V. What has been done by ViM to increase the motivations to acquire essential services of the beneficiaries and service providers? What is the likelihood of continued demand for these services? VI. Assess the effectiveness of the interventions implemented by ViM to strengthen households’ and communities’ resilience to idiosyncratic and co-variate shocks. VII. How well did ViM take advantage of the other USG investments that provide complementary services necessary to achieve ViM’s goal? What are the missed opportunities? What key lessons learned and best practices related to coordination should inform future FFP programming in Burkina Faso? VIII. How effective are the approaches used by ViM to address gender gaps and empower women? ii FINAL REPORT – VICTORY AGAINST MALNUTRITION (ViM) FINAL EVALUATION IX. How effective were the partnerships developed with the Government of Burkina Faso (GoBF) at national, commune, and village level as well as the partnerships with other development actors in delivering intended results aligned with country development objectives? Was there a well￾developed exit strategy effectively communicated? X. What are the unintended positive and/or negative consequences of the ViM project? What lessons can be learned to minimize unintended negative consequences in the design of future projects? How can FFP and its partners design strategies to systematically capture positive consequences? XI. How effective are the value chain approaches used by ViM to promote cowpeas and small livestock value chain in the target communities? Did the ViM project apply the value chain principles for improving small farmers’ access to inputs, financial services and markets? PROGRAM DESCRIPTION The ViM project is a seven-year USD $31.4 Millon USAID/FFP funded Title II Multi Year Assistance Program (MYAP) that started in August 2011 and will end in September 2018. ViM is managed by ACDI/VOCA, in partnership with Save the Children (SC) and the Netherlands Development Organization (SNV) and three Local Implementing Partners (LIPs). ViM also collaborates with government and private actors. ViM’s primary goal under the MYAP is to reduce food insecurity among vulnerable rural populations in four targeted communes of Sanmatenga Province in the Central North Region of Burkina Faso. ViM activities and intermediate results (IR) are designed around three integrated Strategic Objectives (SO): • Strategic Objective 1 (SO1): Increased and diversified agricultural production • Strategic Objective 2 (SO2): Increased household incomes • Strategic Objective 3 (SO3): Reduced chronic malnutrition among children under five years of age and pregnant and lactating women (PLW) The project also addressed issues related to gender equity and women's empowerment and environmental practices that cut across these three SOs. ACDI/VOCA was the lead organization responsible for interventions related to SO1 and SO2 objectives, together with the LIPs. SC was responsible for implementation of interventions related to SO3. SNV undertook capacity building of the LIPs to facilitate implementation of SO1 and SO2 activities. THEORY OF CHANGE The overall goal of ViM is to reduce food insecurity among vulnerable rural populations in the Sanmatenga province of Burkina Faso. ViM interventions for SO1 focus on increasing productivity and food availability through improved practices and technologies and enhancing value chains of key agricultural products. The intermediary results for these interventions includes adopting improved agricultural and livestock practices and improving smallholders’ access to agricultural inputs. ViM interventions for SO2 are based on improvement to value chains, including building links between producers and buyers, supporting development of small food production units, and facilitating access to credit. These interventions are intended to lead to sustainable relationships between beneficiaries and public/private stakeholders, and to link smallholders to profitable domestic/regional markets. The intermediary results for these interventions include improving linkages to markets and access to business development services. iii FINAL REPORT – VICTORY AGAINST MALNUTRITION (ViM) FINAL EVALUATION ViM interventions for SO3 focus on breaking down barriers of chronic malnutrition that have a direct impact on food security by focusing on children under the age of 5 and PLW. Food rations were distributed to children and PLW and to their respective households, to try and reduce vulnerability to food insecurity and improve their caloric, protein, and micronutrient intake. Additionally, ViM interventions included promoting health, nutrition and WASH best practices through the Care Group approach and Mother Leader Animatrices (MLA) to disseminate awareness as well as building latrines and hand washing stations. The intermediary results for this SO include a focus on improving mother and child health and nutrition practices (MCHN), quality of health services, and hygiene/sanitation practices. Project activities to address crosscutting issues on gender and the environment focus on gender integration and increasing women’s empowerment and the use of environmentally sustainable agricultural practices through the ViM interventions under each SO. EVALUATION METHODOLOGY The end-of-project PE of ViM utilizes a mixed-methods design that integrated data from multiple sources. The evaluation drew on data collected via a PBS to measure current status and change over time for 20 key FFP indicators and a companion qualitative data collection effort conducted at endline to provide: additional context and understanding of beneficiary perceptions of project achievements; changes in key outcomes and reasons why; and additional information from project Implementing Partners (IPs) and related stakeholders on issues of project effectiveness, sustainability, unintended consequences, and lessons learned. In addition to the quantitative PBS data and primary qualitative data collected from project beneficiaries and other key stakeholders, the final PE also drew on secondary data from IP performance monitoring data, key IP documents, and the midterm evaluation. The availability of pre/post quantitative data on FFP indicators substantially increased the ability for the Evaluation Team (ET) to derive evidence-based findings, conclusions, and recommendations for this evaluation. The mixed-methods design also utilized qualitative data collection protocols to collect primary qualitative data from ViM beneficiaries and other key stakeholders to help interpret findings and provide support for recommendations. Following the method adopted in the baseline, the endline quantitative household survey is a PBS generating endline indicator values representative of the overall project area for ViM. The target population for the endline PBS for the ViM final evaluation is all households in the ViM project implementation area, consisting of all villages that received the ViM project intervention. This consists of 201 villages, across four communes in Sanmatenga province, covering approximately 35,000 households. The quantitative survey was piloted from November 22- 24, 2017, and data was collected in the field from November 25, 2017 through December 16, 2017. The follow up interviews were conducted from December 17-21, 2017. The final PE analyzed quantitative data collected via a population based survey of households from selected ViM project villages to identify the changes in key outcome indicators between the baseline and endline. The quantitative analysis also investigates the role of ViM in influencing the observed changes and in promoting good practices. The quantitative analysis included descriptive statistical analyses consisting of indicators calculation and baseline to endline comparative analysis, and supplemental econometric analyses to assess how various interventions supported by ViM affected key outcomes of interest. To obtain a deeper understanding of how results vary across subgroups and extent of exposure to different ViM activities, the descriptive analysis also compared indicators between disaggregated iv FINAL REPORT – VICTORY AGAINST MALNUTRITION (ViM) FINAL EVALUATION beneficiary and non-beneficiary groups1 and between male and female beneficiaries. Beneficiary or non￾beneficiary status was defined separately for different sets of indicators, and based on whether surveyed respondents directly participated in a set of pre-defined ViM activities or trainings related to that indicator. The qualitative component of the ViM PE drew on three data collection methods: • Desk review of IP quarterly and annual results reports, monitoring and evaluation (M&E) data, and relevant secondary information; • 31 gender-segregated group discussions (GDs) conducted at endline with project beneficiaries across a total of 16 villages; and • 37 Key informant interviews (KIIs) conducted across village, commune, prefect, and national levels. Qualitative data collection utilized purposive sampling and semi-structured GD and KII protocols. To help provide additional context and interpretation for quantitative results, the qualitative tools design took into account preliminary results from the quantitative PBS. The qualitative fieldwork for the endline PE was conducted in January 2018. GDs were recorded, with consent from participants, and transcribed from Mooré to French. Transcribed GD data and detailed GD notes were then coded in NVivo according to a pre-defined codebook coding text segments according to key themes of interest for the Evaluation Questions (EQs). KII notes were summarized using standard content analysis techniques. For both KIIs and GD data, analyses summarized common trends and patterns to highlight project-, sector-, and gender-differentiated trends, and to identify examples of positive deviance and most significant change. EVALUATION LIMITATIONS AND CONSTRAINTS Key constraints associated with this pre-post mixed methods performance evaluation are: lack of a valid counterfactual for the quantitative PBS; small sample size for some of the target value chain crops; difference in seasonal timing of data collection between baseline and endline for the PBS; unreliability of the farmer self-reported approach for yield data collection at baseline and endline; and potentials for recall and selection bias for the qualitative data collection. To help address the absence of a valid counterfactual, the companion qualitative data collection at endline aims to: help corroborate the PBS-based results; capture information on potential confounders; and understand beneficiary perceptions on if and how project activities helped to elicit change in key outcomes. In addition, supplemental econometric analyses were conducted to test for correlations between beneficiary participation in or exposure to ViM activities and select outcome indicators, to help isolate contributions of ViM activities to observed outcomes. The seasonal difference between baseline and endline could affect baseline to endline differences for some indicators. In particular, dietary diversity, prevalence of household hunger, and prevalence of minimum acceptable diet for children may be more sensitive to this timing difference. Non-nutrition indicators that may be affected are the household dietary diversity score and minimum acceptable diet. For agricultural indicators, because the main harvest season was underway during endline, there may also be a seasonal influence due to differences in accuracy of farmer recall for self-reported harvests at baseline vs. endline. To help control for the timing difference, questions on household hunger in July (the middle of the lean season, and consistent with the timing of baseline) were added to the endline questionnaire. 1 See Annex 3 for detailed charts on how beneficiary/non-beneficiary status was determined for each indicators, based on the types of interventions directly received by the surveyed respondents. v FINAL REPORT – VICTORY AGAINST MALNUTRITION (ViM) FINAL EVALUATION Statistical tests of difference between baseline and endline for crop yield estimates are not reported for the final evaluation, as yield estimates were obtained using farmer self-reported yields, and the baseline values for these indicators were considered unreliable by ViM. There are well-known challenges associated with the reliability of farmer self-reporting on both farm size and crop production data. Use of GPS technology to obtain accurate plot areas, and crop-cut methods to obtain yields, are associated with more accurate measurements of crop production, but are also typically cost-prohibitive for large￾scale surveys. After discussion with ACDI/VOCA and ViM, the endline evaluation team calculated the baseline to endline change using the original baseline data, but has not reported statistical tests of significance for baseline to endline change in this document for three indicators, due to the project’s communicated view that the baseline values for these indicators were considered invalid. ViM has used the GoBF, Agristat, FAOstat or internal ViM project annual M&E data as a substitute for the baseline data for these indicators, however the evaluation team was not able to use those numbers as a baseline substitute for the purposes of conducting statistical tests of baseline to endline change, because these substitute data were obtained using different sampling frames and data collection approaches than the baseline and endline data collection and would not constitute a valid statistical analyses of baseline to endline changes in the ViM project area over the project lifetime. SUMMARY FINDINGS AND CONCLUSIONS Key findings and conclusions for EQs one through four are summarized below. The main take-away points from the statistical analysis of PBS baseline (2012) and endline (2017) data for the ViM project are: • For SO1 indicators, quantitative and qualitative data provide an indication of widespread improvements in adoption of agricultural practices, increased crop yields, as well as some diversification. While the PBS and t-test results for a change in sorghum and cowpeas between baseline and endline are not reported due to methods limitations, qualitative data provided strong support for widespread improvements in crop yields in the ViM project area. The use of at least four sustainable agricultural practices at endline was 74% of farmers. The percentage of farmers who used improved storage increased from 48.1% at baseline to 74.3% at endline. Qualitative data from GDs and KIIs indicate that improved storage for cowpeas was seen as instrumental for improved crop preservation, and for facilitating improvements in marketing and sales. • Several indicators in SO2 show improvements in household income over the project lifetime. These indicators include a decrease in household hunger and improvements in market linkages, business development, use of financial services, and the percentage of farmers practicing value chain activities. • For SO3, there have been large and statistically significant improvements in mother and child health and nutrition, including for stunting, underweight, women’s dietary diversity, quality of health services, and sanitation/hygiene. • Women’s empowerment has seen substantial improvement through women’s decision-making power and changed perceptions of gender-based violence. • There are mixed results for improvements in crosscutting environmental issues, but a large and statistically significant improvement in the percentage of farmers using environmental mitigation principles in diversified income generating activities. Summary tables of endline and statistical results are presented in Annex I, and discussion of results by indicators is presented in detail in chapter four of the report body. Qualitative results suggest that overall ViM activities were impactful for ViM beneficiaries and well-implemented by the project. Beneficiaries cited a variety of ViM activities deemed helpful, including and not limited to literacy courses (three GDs), implementation of learned agricultural production techniques (seven GDs), safe conservation of crops (five GDs), hygiene awareness raising activities (two GDs), increased access to vi FINAL REPORT – VICTORY AGAINST MALNUTRITION (ViM) FINAL EVALUATION health care (six GDs), livestock raising and care techniques (five GD participants), and porridge making (one GD). Supplemental econometric analysis was used to determine the effect of ViM interventions on agricultural production and income. To assess the effectof ViM promoted good agricultural practices (GAP) trainings and extension services on agricultural production, statistical analysis was done to estimate the effectof ViM on the average number of agricultural products produced and the yield of cowpeas and sorghum based on multivariate regression techniques. Beneficiaries who received/attended ViM promoted GAP training or were served by extension service providers produced greater numbers of crops/livestock products on average compared those who did not participate in such trainings. Similarly, those who participated in GAP training had higher cowpeas yields. Participants in GDs across several village also noted substantial improvements to their crop yields, and often tied these specifically to some of the trainings on good agricultural practices that were provided by ViM. To assess the effectof ViM interventions on income, HDDS, assets value, and cowpea and livestock revenue and gross margins measures were constructed. The findings indicate ViM training had a positive effect in increasing HDDS, and on domestic and productive asset values. There was also a positive effect on cowpea and livestock revenues and gross margins from GAP trainings and extension services. With respect to the role of ViM in improving the use of good agricultural practices, econometric analyses found there was a positive effect of ViM promoted GAP training and extension service providers on the adoption of at least four GAPs for sorghum, as well as for cowpeas. ViM livestock training and extension services had a positive effect on both measures of improved livestock practices. Assessment of the role of ViM on nutritional outcome and MCHN practices indicated that beneficiaries of ViM interventions were relatively better off with respect to key malnutrition indictors and more likely to adopt improved practices. Overall, ViM project beneficiaries reported changes in MCHN and WASH outcomes within their communities as a direct result of the implementation of ViM-promoted practices, while the quantitative and qualitative data alike provide broad support for substantially increased prevalence of MCHN and WASH practices (and indeed, positive impact-level changes) in ViM intervention areas during the project lifetime. Together, this evidence suggests substantial effectiveness of the project in promoting these practices and facilitating positive MCHN and WASH outcomes. Respondents in GDs and key informant interviews also provided several anecdotes outlining the ways in which the implementation of practices around birth spacing, hand washing, food variation, health centers, and latrine use have positively affected their lives. Results of the econometric analysis indicate that after controlling for various potential confounders, the beneficiaries of ViM interventions were significantly better off compared to the non-beneficiaries with respect malnutrition indicators. The ViM interventions considered for these analyses were food rations and MCHN training. The outcomes considered were WDDS, stunting rate and underweight for children under five years of age. Provision of ViM food rations had negative (reducing) effect on the stunting rate, indicating that the prevalence of malnutrition was lower among beneficiaries who received food rations than for individuals (non-beneficiaries) who did not receive food rations. Thus, provision of food rations was effective in reducing malnutrition among children under five years of age in the project area. In addition to this positive effect of ViM’s food rations activity on childhood malnutrition, results also suggest that the ViM promoted MCHN training was also successful, as children in households that were exposed to ViM MCHN training had a lower stunting rate and underweight rate than households that did not directly participate in this training. Econometric results also show that ViM WASH training was also effective in reducing stunting rates, as the stunting rate among households that received WASH training was lower than households with similar characteristics that did not receive WASH training. vii FINAL REPORT – VICTORY AGAINST MALNUTRITION (ViM) FINAL EVALUATION Moreover, households with a neighborhood group member had a higher percentage of children receiving MAD than the households that did not have a neighborhood group member. Finally, the econometric results indicate that ViM promoted WASH training was effective in improving knowledge about the importance of hygiene practices and also influenced households to adopt improved handwashing practices. With respect to the effectiveness of ViM’s strategies to ensure continued service provision after the lifetime of the project, qualitative results from KIIs suggest there is some likelihood that ViM activities could be continued independently of USAID support. However, several respondents added that activity continuation or continued service provision would require increased or repeated trainings, partner organizations would have to keep the equipment they have received from the project, or that organizations would need to seek out other donors for support. Among those in-depth interview respondents who were less convinced of sustained service delivery, several stated limited alternatives for funding as the reason for their concern. Others noted that partner organizations lack sufficient training, and that institutional knowledge was too limited at this stage for sustainability upon project withdrawal. When asked about what key elements of support are needed for organizations to continue offering ViM services, 19 of the 36 key informants said that multiple types of training would be essential, while six respondents mentioned capacity building in particular. Among the cowpeas and onions producer group representatives who were interviewed at endline, all noted large gains from the ViM project, within their organizations and among the targeted producer groups for the selected value chain crops (cowpeas, sorghum, tomoatoes and onions), in terms of capacity building for agricultural production, and crop conservation, marketing and storage techniques. Some of the key areas of capacity strengthening that they noted were around using a group sales system for products, and a more consolidated approach to purchasing required inputs for production. Still, a need for additional traning and supervision of groups after the project lifetime was commonly highlighted, as was a need for funding to support training and monitoring activities beyond the project lifetime. In addition, respondents mentioned remaining gaps in groups’ capacity to obtain loans for inputs and large capital purchases, access to marketing equipment such as scales and sieves, and overall organizational management skills. Overall, evaluation results suggest that technical, organizational and/or other institutional capacity constraints are likely to challenge continued service delivery after ViM for many project-supported groups, despite their interest to continue providing the project-promoted services. This does not appear to be due to respondent dissatisfaction with the ViM-provided support and/or capacity building to individual providers or service groups. Rather, there is a need for continued support for some additional time, before such organizations are in a position to provide services and function wholly independently. At the same time, qualitative findings at endline indicated that the vast majority of participants reported very positive feelings and high motivation to continue applying the methods and practices they learned, while the quantitative results provided further support for a high level of continued demand for ViM￾promoted services and trainings. Results also suggest that ViM has contributed to the strengthening of social cohesion within communities, which is an important factor in the resilience of people at community level. The high likelihood for continued demand for ViM-promoted services by households in the project area is also supported by related results from the household survey, with respect to household expectations to continue ViM-promoted technologies or practices, and continued participation in related trainings. GD results indicate an improved ability of respondents to absorb shocks – a capacity that beneficiaries largely attributed to ViM project interventions, although the evidence for this is somewhat mixed. In GDs, participants mentioned specifically, increased capacity to store crops, which contributes to longer viii FINAL REPORT – VICTORY AGAINST MALNUTRITION (ViM) FINAL EVALUATION periods of food conservation and contributes to household income smoothing in the face of idiosyncratic or covariate shocks. This conclusion is also supported by the PBS results, in which the percentage of farmers using improved crop storage methods rose by 26 percentage points. Enhanced crop storage, for example, lead to greater food availability for longer periods of the year for households, and also enabled farmers to sell food at more favorable prices, which boosted income in a manner that ultimately strengthens resilience. Beneficiaries also said that the knowledge they acquired through ViM training on vaccination for livestock, information on protection of crops against pests, and using improved seeds has also bolstered resilience against shocks. The quantitative SO2 results provide additional quantitative indication of increased household resilience and capacity to absorb shocks. In particular, the prevalence of household hunger declined substantially (though some of this decline could be due to the difference in seasonality between baseline and endline), while the value of household assets and access to formal was significantly higher at endline, and some market linkages were strengthened (for example, greater participation in value chains among households in the project area). Lastly, the significantly improved status of children with respect to stunting and prevalence of underweight children suggest there is likely greater stability of household food security at endline, although it is noted that women’s dietary diversity did not change over the project lifetime. Desk review of ViM project reports indicates that the project engaged in substantial collaboration with other USG investments, and aimed to leverage those engagements to provide complementary services beyond what the project could likely achieve on its own. This may be particularly the case for the way the project worked with additional USAID-funded programs in later years of the project. There are several examples of ways in which the project was able to work under the RISE initiative to help try to ensure sustained service delivery and overlapping coverage in capacity building to try to ensure continued achievements of ViM goals beyond the project lifetime. Although the quantitative data did not provide as much evidence of the effectiveness of ViM project on gender, the complementary qualitative data suggests that ViM activities have improved women’s empowerment, their access to and control over productive resources, and increased decision-making over a range of issues and financial resources. Moreover, the quantitative results indicate that ViM activities clearly contributed to increasing the agency of women with respect to their own health, children’s health, and productive resources. ViM activities that centered on the promotion of women as leaders, the economic empowerment of women, and the sensitization of men are likely key contributors to this shift. Lastly, qualitative results provide strong evidence that Care/Neighborhood Groups were extremely important to ViM activities in mobilizing community members, promoting improved MCHN practices, and sensitizing men and women on a range of important issues, including strengthening women’s empowerment and changing deleterious attitudes and behaviors among men. Overall, the evaluation found few strongly negative unintended consequences or barriers that substantially affected the achievement of ViM’s intended outcomes. On the positive side, it is noted that ViM’s use of peer-led learning groups and networks had synergistic effects for behavior change in villages, and can be leveraged to provide synergies with other initiatives. Qualitative results highlighted three key areas of constraints that contributed to negative unanticipated consequences or likely constrained higher potential achievement of some project objectives: a lack of sufficient water in some villages, which several GD participants noted hindered their engagement in ViM activities, both for agricultural and WASH programs; inadequate market access in some areas, which may have contributed to constrained achievement of value chain and household livelihoods objectives in some project areas; and challenges for women associated with implementing exclusive breastfeeding. Lastly, with respect to the effectiveness of ViM approaches to promote cowpeas and livestock value chains, the evaluation found that ViM was effective in promoting the cowpea value chain activities. 82.7% of farmers (717 of 867) who grew cowpeas and were ViM beneficiaries used cowpea value chain ix FINAL REPORT – VICTORY AGAINST MALNUTRITION (ViM) FINAL EVALUATION activities, compared to 64.2% of non-beneficiary (1,369 of 2,131) cowpea farmers. Of the 2,086 farmers, 52.3% were male and 47.6% were female. For cowpea farmers who practiced joint purchase of inputs, 84.9% of beneficiaries said that they had improved access to inputs compared to 46.4% of non￾beneficiaries. Beneficiaries who practiced bulk sale through farmer’s groups were more likely to say they had improved access to market information (82.8%) than non-beneficiaries (60.0%). Additionally, beneficiaries who practiced bulk sale through farmer’s groups were less likely to see remote sales centers or bad road conditions as a constraint on marketing cowpeas than non-beneficiaries. Finally, there was no difference between beneficiaries and non-beneficiaries who practiced marketing skills in terms of seeing improved access to market information, which was around 97% for both. ViM was also effective in promoting the livestock value chain activities; 73.4% of farmers (259 out of 353) who own livestock and were ViM beneficiaries used livestock value chains, compared to 55.0% of non-beneficiary (1,339 out of 2,433) livestock owners. Of these 1,598 farmers, 65.3% were male and 34.7% were female. A higher percentage of beneficiaries got at least some animals vaccinated as compared to non￾beneficiaries (92.5% vs 84.3%). However, when the sources of the vaccinations are considered, the beneficiaries had a higher rate of getting vaccinations from NGOs compared to the non-beneficiaries who had higher rates of getting animal vaccinations from CBOs or private organizations. SECTOR-SPECIFIC RECOMMENDATIONS: MCHN AND WASH The ViM project has contributed substantial efforts that addressed both immediate and underlying causes of malnutrition in the project area’s population of children under five years of age. The project implemented several best-practices activities that reflect the current evidence base for effective woman and child health and nutrition programming, including support for or the use of multiple delivery platforms (for example making use of market-driven service provisionining and layered delivery platforms at community, commune and broader levels) across a fairly comprehensive package of high￾impact interventions. Given that this groundwork has been laid, recommendations for future programming emphasize developing and further strengthening longer term strategies to enhance the provision of basic health services, together with a focus on enhancing sustainable strategies for livelihood support and social protection mechanisms.Specific recommendations are elaborated below. Immediate / short term interventions • Consider establishing and strengthening a provincial level forum to systematically guide, support and monitor community-based nutrition programs. Establishment of a community based nutrition program, which has already been piloted by the ViM project in 4 communes, would enable the government and development partners to continue to address the challenge of malnutrition in all communes of Sanmatenga province meaningfully. Supporting nutrition-sensitive interventions such as those within agriculture, food security, outreach immunization, nutrition and health education on hygiene, water and sanitation and livelihoods sectorcould then be integrated into this overarching platform for sustainable nutrition interventions. As part of this process, strengthening of a provincial multi￾sectoral forum to develop a province-focused strategy for eradication of malnutrition and to provide oversight in implementation of nutrition and linked nutrition-sensitive interventions is also recommended. As piloted by the ViM project, there will be a need for continued monitoring of both facility and community based interventions to track progress. • Continue to support and strengthen programs and strategies currently addressing infant and young child nutrition, with a view to building sustained support for the protection, promotion, and support of optimal infant and young child feeding practices. This could be addressed through regular, frequent and updated trainings of community volunteers to counsel and provide on-going support to caregivers. Mothers should continue to be trained in preparation of nutritious foods for infants and young children using foods that are locally available. Community based volunteers should be trained to x FINAL REPORT – VICTORY AGAINST MALNUTRITION (ViM) FINAL EVALUATION conduct regular screening and growth monitoring and to make referrals of malnourished children for follow-up at local health facilities. Medium / longer-term interventions • To improve feeding practices, underlying cultural factors and beliefs hindering uptake of recommended feeding behaviors identified through the ViM evaluation endline survey will need to be addressed. Engaging influential household members such as grandmothers and grandfathers is strongly recommended. • Improved coverage for child health programs is recommended, especially for Vitamin A supplementation and de-worming. This is feasible through strengthening of the outreach component, ensuring that these are distributed concurrently during vaccination campaigns. SECTOR-SPECIFIC RECOMMENDATIONS: AGRICULTURE AND LIVELIHOODS Evaluation results shows that ViM was effective in promoting cowpea and livestock value chain activities, and cowpea farmer beneficiaries who practiced bulk sales through organized farmer’s groups appear to have been more likely to have had improved access to market information. The evaluation analyses also suggest that beneficiaries who received ViM promoted GAP training or extension services produced a greater number of crops or livestock products on average compared with those who did not participate in such trainings. Similarly, participation in GAP training was associated with a higher yield for some crops, including cowpeas. This may suggest a stronger and continued focus on cowpea value chains relative to other commodities in future programming, but such results can also be sensitive to specific context and market factors present during program time frame and thus new programs should conduct and factor their own up-to-date crop-specific market and value chain analyses into crop-specific decisions for future programming. Participation in ViM trainings was also found to have a positive effect on household domestic and productive asset values. The overall conclusion is that the ViM project has been effective in promoting smallholders’ access, affordability and use of agriculture and livestock extension services, and of project-sponsored high quality agricultural and livestock inputs including veterinary services. However, many of these gains detected through supplemental econometric analyses by the evaluation team are not reflected in the PBS-based baseline to endline analyses. This may indicate that many of the positive agricultural and livelihoods benefits in the project’s intervention area are more restricted to direct participants in ViM’s programs, while not yet achieving widespread spillover to the intervention area population as a whole. Based on this, the key recommendation for future programming within agricultural and livelihoods spheres is to consider building program strategies for faster scale-up and reaching a greater number of beneficiaries within project timeframes. In addition, it is important to ensure that training and service provision knowledge, managerial capacity, and adequate resource transfer to local institutions has been substantially achieved prior to the end of project close-out, such that continued support to existing beneficiaries and expansion of benefits to additional populations is ensured beyond the project lifetime. 1 FINAL REPORT – VICTORY AGAINST MALNUTRITION (ViM) FINAL EVALUATION 1. INTRODUCTION, EVALUATION PURPOSE AND QUESTIONS 1.1 INTRODUCTION AND EVALUATION BACKGROUND This report details the findings and recommendations of an end-of-project PE of the ViM project, as commissioned by ACDI/VOCA under funding provided by the United States Agency for International Development (USAID) Office of Food for Peace (FFP). ViM is implemented in four communes of Sanmatenga province in the Central North Region of Burkina Faso. The purpose of the final PE is to determine to what extent the food security status of the targeted population has changed and how the ViM project has contributed to the resiliency of the targeted communities. The PE has two components: (1) a quantitative component which draws on the results from a Population-Based Survey (PBS) administered at project baseline (July-August 2012) and endline (Sept-Oct 2017); and (2) a qualitative component which collects data on and analyzes changes in perception, knowledge and behavior of target beneficiaries. The quantitative component of the PE measures change and assesses the results achieved by the ViM project by statistically comparing the end￾line survey results against the baseline survey results in ViM operational areas. The qualitative component is designed to complement and expand on the quantitative PBS results, to better understand project implementation processes and challenges, reasons for observed outcomes, and issues related to project effectiveness, sustainability and lessons learned to inform future program design. The evaluation was conducted from October 2017 – April 2018 by NORC at the University of Chicago (NORC). The evaluation’s results are aimed at multiple audiences. The findings are expected to have primary accountability and learning value to USAID (FFP/Washington, USAID/Burkina Faso, FFP West Africa Regional Office, and the FFP learning network) and ACDI/VOCA and its sub partners. Additional stakeholders include government officials from key collaborating Ministry offices in Burkina Faso and USAID West Africa Sahel Regional Office. The findings, conclusions, and recommendations of the evaluation may also be used by USAID to extract lessons learned and generate insights to inform the design of follow-on programming in Burkina Faso. 1.2 EVALUATION QUESTIONS The end-of-project PE provides a substantive analysis of four key topical areas, according to the evaluation Statement of Work (Annex X). I. To what extent was the ViM project effective in achieving strategic objectives and intermediate results? Assess the performance of the ViM project based on a comparative statistical analysis2 of outcome and impact indicator data, collected through the endline and the baseline surveys. For key outcome and impact indicators, use econometric analysis to identify plausible factors associated with indicator change3. 2 Using t-tests of differences. 3 Note that as with all performance evaluations, the final evaluation indicator change will not be able to establish attribution of observed changes to the ViM project itself. Results are also net of any macro-level factors or policy effects that may have taken place during the project lifetime, such as nationwide free health care for children under five or enhanced access to health care services for pregnant or lactating women. 2 FINAL REPORT – VICTORY AGAINST MALNUTRITION (ViM) FINAL EVALUATION II. Using quantitative analysis, evaluate the effectiveness of the ViM project in promoting improved agricultural and livestock practices and increasing and diversifying smallholder agricultural production. Assess smallholders’ access to, affordability, quality of, and use of agriculture and livestock extension services, and agricultural and livestock inputs. Assess the extent to which these services were available and beneficial for both men and women beneficiaries. III. Assess the effectiveness of the ViM project in promoting practices around infant and young child feeding, maternal and child health; water, sanitation and hygiene (WASH) in reducing malnutrition among children under five, and pregnant and lactating women. How prevalent are these practices in the ViM target communities? IV. What is the level of motivation among the potential service providers4 and the likelihood of their sustained service provision after the end of the ViM project? What are the incentives? Assess the effectiveness of the strategies implemented by the ViM project to ensure that private/public providers of agriculture, livestock, health, women's empowerment, and nutrition services. Will community actors involved in health and nutrition promotion will continue to provide services after the end of the ViM? How and where will these providers get resources to provide these critical services? How would their access to resources be different for services that are considered public vs private? What has been done by the ViM project to ensure that the service providers will have continuous access to enhance their capacities? Assess the technical, organizational, and management capacities of the producer groups developed by the ViM project and their effectiveness to provide sustained service delivery after the end of the ViM project. V. Assess the level of demand among the community members and willingness to pay for essential agricultural, livestock, health and nutrition services provided by the project. What has been done by ViM to increase the motivations to acquire essential services of the beneficiaries and service providers? What is the likelihood of continued demand for these services? VI. Assess the effectiveness of the interventions implemented by ViM to strengthen households’ and communities’ resilience to idiosyncratic and co-variate shocks. How do households’ and communities perceive their capacity to manage these shocks? How well did ViM implement the recommendations from the formative evaluation to integrate with RISE initiatives? How well have the management (e.g. monitoring and evaluation, partnership, and resource management), approaches (e.g. mothers’ groups, and farmer field schools), and cross cutting areas (e.g. community participation, gender, governance, and natural resource management) of the ViM project has been integrated with the RISE initiatives, and what are the results? VII. How well did ViM take advantage of the other USG investments that provide complementary services necessary to achieve ViM’s goal? What are the missed opportunities? What key lessons learned and best practices related to coordination should inform future FFP programming in Burkina Faso? VIII. How effective are the approaches used by ViM project to address gender gaps and empower women? To what extent have ViM activities affected women's 4 Agriculture, livestock, health and nutrition. 3 FINAL REPORT – VICTORY AGAINST MALNUTRITION (ViM) FINAL EVALUATION empowerment, specifically empowerment in terms of access to and control over productive resources; increased decision-making roles over income and purchase/use of productive resources; increased leadership as evidenced by participation in community and social institutions, and comfort speaking in public? What has been or could be done to sustain the positive gender related outcomes that are achieved by ViM? What are the lessons learned? IX. How effective were the partnerships developed with GoBF at national, commune, and village level, as well as the partnerships with other development actors in delivering intended results aligned with country development objectives? Was there a well-developed exit strategy effectively communicated? X. What are the unintended positive and/or negative consequences of the ViM project? What lessons can be learned to minimize unintended negative consequences in the design of future projects? How can FFP and its partners design strategies to systematically capture positive consequences? XI. How effective are the value chain approaches used by ViM project to promote cowpeas and small livestock value chain in the target communities? Did the ViM project apply the value chain principles for improving small farmers’ access to inputs, financial services and markets? 2. PROJECT BACKGROUND 2.1 COUNTRY CONTEXT Burkina Faso is a poor landlocked country in West Africa with fragile food security and a prevalence of acute and chronic malnutrition. The ViM project is implemented in the Central North Region of Burkina Faso, Sanmatenga Province, in the communes of Kaya, Barsalogho, Namissiguima and Pissila. Food Insecurity The economy of Burkina Faso is based primarily on agriculture and livestock due to limited livelihood opportunities. Household food supply is generally dependent on subsistence agricultural production that is concentrated in one rain-fed season, which contributes to structural food insecurity every year during the lean season from May to September. Additionally, Burkina Faso is susceptible to cyclical food crises due to low and variable rainfall leading to droughts and flooding, land degradation and deforestation, insect invasions, climate change, and other environmental shocks. In 2012 there was a severe food crisis in Burkina Faso due to drought with low rainfall and poor spatial-temporal distribution, and Sanmatenga province was one of the most affected.5 Prevalence of Malnutrition Food insecurity in Burkina Faso is closely linked to malnutrition, particularly in children and pregnant and lactating women (PLW) with poor and undiversified diets. Burkina Faso’s National Nutrition Survey of 2011 estimated the prevalence of chronic malnutrition in the country to be 35%. Malnutrition is estimated to be a major contributor to the country’s high under five child mortality rate, estimated at 5 USAID. Agriculture and Food Security; Burkina Faso. 4 FINAL REPORT – VICTORY AGAINST MALNUTRITION (ViM) FINAL EVALUATION 176 per 1,000 live births in 20106. Malnutrition is driven by poor sanitation practices, poor women and child health and nutrition practices, and a lack of quality health services, among other factors. Child malnutrition is worsened by poor sanitation leading to diarrheic diseases, poor infant and child feeding practices, and barriers to assessing health services for young children and PLW. Malnutrition for PLW is particularly affected by the poor access or use of health services leading to anemia, vitamin A deficiencies, and lack of prenatal visits. 2.2 PROGRAM DESCRIPTION The ViM project is a seven-year USD $31.4 Millon USAID/FFP funded Title II Multi Year Assistance Program (MYAP) that started in August 2011 and will end in September 2018. ViM is managed by ACDI/VOCA, in partnership with Save the Children (SC) and the Netherlands Development Organization (SNV) and three Local Implementing Partners (LIPs), and also in collaboration with government and private actors. ViM’s primary goal under the MYAP is to reduce food insecurity among vulnerable rural populations in four targeted communes of Sanmatenga Province in the Central North Region of Burkina Faso. ViM activities and intermediate results (IR) are designed around three integrated Strategic Objectives (SO): Strategic Objective 1 (SO1): Increased and diversified agricultural production IR 1.1 Improved agricultural practices adopted IR 1.2 Improved smallholder access to agricultural inputs IR 1.3 Improved livestock practices adopted Strategic Objective 2 (SO2): Increased household incomes IR 2.1 Market linkages improved IR 2.2 Access to business development services improved Strategic Objective 3 (SO3): Reduced chronic malnutrition among children under five years of age and pregnant and lactating women (PLW) IR 3.1 Improved mother and child health and nutrition (MCHN) practices IR 3.2 Improved quality of health services IR 3.3 Improved hygiene and sanitation practices. The project also addressed issues related to gender equality and women's empowerment and environmental practices that cut across these three SOs. In terms of main consortium partner roles, ACDI/VOCA was the lead organization directly responsible for interventions related to SO1 and SO2 objectives, together with LIPs, and for project monitoring and evaluation activities. SC, in collaboration with ACDI/VOCA, was responsible for implementation of interventions related to SO3. SNV undertook capacity building of LIPs to facilitate implementation of SO1 and SO2 activities. 6 United Nations Children’s Fund, The State of the World’s Children 2012: Children in an urban world, UNICEF, New York, 2012 5 FINAL REPORT – VICTORY AGAINST MALNUTRITION (ViM) FINAL EVALUATION Table 1: Implementing Partner Zones of Coverage, and Number of Villages and Beneficiaries Target Zone of Coverage for ViM Project Number of Villages Number Commune of Beneficiaries Barsalogho 55 65,985 Kaya 71 57,136 Namissiguima 11 9,964 Pissila 64 79,589 Total 201 212,674 Figure 1: ViM Intervention Zone Source: ViM baseline study 2.3 THEORY OF CHANGE The overall goal of the ViM intervention is to reduce food insecurity among vulnerable rural populations in the Sanmatenga province of Burkina Faso. The ViM project involves interventions revolving around the three SO; increasing and diversifying agricultural production (SO1), increasing household income (SO2), and reducing chronic malnutrition (SO3), while paying particular attention to the issues of gender inequality and sustainable development. ViM interventions for SO1 focus on increasing productivity and food availability through improved practices and technologies and enhancing value chains of key agricultural products. The intermediary results for these interventions includes adopting improved agricultural and livestock practices and improving smallholders’ access to agricultural inputs. 6 FINAL REPORT – VICTORY AGAINST MALNUTRITION (ViM) FINAL EVALUATION ViM interventions for SO2 are based on improvement to value chains, which focuses on building links between producers and buyers, supporting development of small food production units, and facilitating access to credit. These interventions are intended to lead to sustainable relationships between beneficiaries and public/private stakeholders, and to link smallholders to profitable domestic/regional markets. The intermediary results for these interventions include improving linkages to markets and access to business development services. ViM interventions for SO3 focus on breaking down barriers of chronic malnutrition that have a direct impact on food security by focusing on children under the age of five and PLW. Food rations were distributed to children and PLW and their respective households, to try and reduce vulnerability to food insecurity and improve the caloric, protein, and micronutrient intake for children and PLW. Additionally, ViM interventions included promoting health, nutrition and WASH best practices through the Care Groups approach and Mother Leader Animatrices (MLA) to disseminate awareness as well as building latrines and hand washing stations. The intermediary results for this SO include a focus on improving mother and child health and nutrition practices (MCHN), quality of health services, and hygiene/sanitation practices. Project activities to address crosscutting issues on gender and the environment focus on gender integration and increasing women’s empowerment, and the use of environmentally sustainable agricultural practices through the ViM interventions under each strategic objective. 2.4 BASELINE AND MIDTERM CONTEXT The ViM baseline report, midterm evaluation, and formative evaluation reports provide useful project and background context that inform the final evaluation. The ViM baseline report (2012) outlined several recommendations for the achievement of each SO and the corresponding intermediate results. The baseline report highlighted that achievement of IRs is related to increased and diversified agricultural production (SO1) rested on two elements. First, the development of women’s producer groups was deemed necessary to obtain improvements in income generation. Although existing agricultural groups showed promise towards achieving indicators listed in SO1, the baseline report mentioned a need for skills development related to the effective use of inputs, group membership optimization, profit and profit-sharing analysis, and product diversification. Second, the baseline results indicated an overwhelming desire by villagers for water resource development, including the development of dams and other water supply infrastructure. Procurement of additional resources was recommended to help facilitate the IRs related to SO1. With regard to the achievement of IRs under SO2 (increasing household incomes), the baseline report recommended that ViM prioritize interventions that are anticipated to have greater implications for the most vulnerable households, and also to focus on lesser-known food products, particularly those that are local or only used during the lean season. The baseline report further recommended that leveraging partnerships related to market projects and savings and loans programs would be important for addressing gaps in the value chains prioritized by the project, and for mobilizing producers to save money. Lastly, the baseline report included several recommendations related to the achievement of the IRs related to SO3 (the reduction of chronic malnutrition among children under five years of age, and pregnant/lactating mothers). The report reiterated the importance of the formation of women’s groups, so that there was an opportunity for women’s agricultural groups to broaden their focus to include nutrition and health issues, since targeted communities had few activities centered on the nutrition and health of women. In this vein, activities that improve the potable water supply in targeted communities were also highlighted, including support for water testing. In addition to potable water, the promotion of latrines was recommended as another means to achieve SO3. Although projects in target communities 7 FINAL REPORT – VICTORY AGAINST MALNUTRITION (ViM) FINAL EVALUATION previously supported the construction of latrines, it was noted there had been little change in the motivation to use them, possibly related to a lack of sensitization on hygiene issues. In response, the baseline report suggested that ViM collaborate with NGO sponsored activities to promote discussion about the importance of hygiene. The use of community health workers in target communities was viewed as an important element in increasing sensitization about women’s health, improved water, and hygiene. At midline, several of the baseline recommendations were implemented, and appeared to produce positive results. To achieve SO1 and SO2, ViM targeted six priority value chains (cowpea, sorghum, small ruminants, poultry, tomato, and onion). ViM also selected groups in targeted communities based on criteria aiming to reach the most vulnerable populations. The groups underwent technical training and organizational strengthening. As recommended at baseline, several partnerships were leveraged in order to implement ViM’s value-chain approach, which involved agro dealers, technical support from government offices, and other linkages. Some of the key achievements for SO1 and SO2 that were highlighted in the midterm report after two years of implementation included: • 600 groups supported to increase and diversify agriculture production and receive subsidies. • 2,873 producers involved in onion and tomato value chains. • 9,952 producers involved in sorghum and cowpea value chains. • 3,169 breeders involved in small ruminant value chains. • Improved cowpea and sorghum seeds distributed. • Administration of several well-received trainings, with plans for producers to integrate new practices into their production. • Successful establishment of partnership with FCP to foster women’s access to credit. • Improved animal health and growth through vaccination campaigns. • 346 breeders involved in poultry value chains and vaccination. Despite these achievements, the midline report noted several challenges. Some were project-related, such as efforts to facilitate organizational strengthening among LIPs, who were generally noted to have grown in capacity over the project lifetime but still had skills gaps at midline. Others, such as moderate participation in livestock fairs and drops in inputs subsidies, were caused by fluctuating prices that had implications for access to livestock and fertilizer for the most vulnerable producers. Midline recommendations included strengthening group support and functioning, promoting farmers’ knowledge and practices, and monitoring the framework for providing credit access to women. At midline, there were several noteworthy achievements for SO3, including the implementation of care groups led by MLA, neighborhood groups, and implementation of 46 village committees tasked with the management and distribution of food in community warehouses. In addition, the midline results revealed significant achievements related to the effectof MLA, including increased awareness of breastfeeding importance, sanitation, pre-natal consultations, hygiene, and nutrition. Recommendations for achieving additional gains centered on: increasing the knowledge capital of mothers, providing specific support to mothers for food availability, and increased autonomy for care groups and community contributions. Specific nominal achievements for SO3 highlighted in the midterm report included: • 2,325 Mother Leaders trained, 23,694 mothers involved, and 60 community health volunteers trained. • Implementation of 46 village committees tasked with the management and distribution of food in community warehouses who conducted: o Monthly distribution of food rations for 27,801 children and 21,555 pregnant and lactating women o Distribution of family rations for 20,868 households o 42 village sanitary committees (425 people) established 8 FINAL REPORT – VICTORY AGAINST MALNUTRITION (ViM) FINAL EVALUATION o 102 family latrines and 235 hand washing stations built Midline findings suggested some notable challenges for cross-cutting issues. With regard to gender, expected results from training on women’s rights were viewed to have not yet materialized. As a result, the report recommended awareness-raising efforts on gender that integrate interactive tools and strengthened training for community leaders. There were also concerns regarding the sustainability of some of the project activities and interventions after the end of the project, specifically, how groups would access high quality and affordable products after project close, how obstacles faced by groups would be resolved, and the provision of technical services from government officers. To address these concerns, midline recommendations emphasized promoting sustainability, including reinforcement of group support, strategic partnerships, and identification of groups with greatest potential for development. The current final evaluation seeks to determine the extent to which ViM was successful in achieving its original objectives and the extent to which the integration of recommendations from baseline and midline helped achieve objectives and / or affected other outcomes. 3. EVALUATION METHODOLOGY The end-of-project PE of ViM utilizes a mixed-methods design that integrated data from multiple sources. The evaluation drew on data collected via a PBS to measure current status and change over time for 20 key FPP indicators and a companion qualitative data collection effort conducted at endline to provide: additional context and understanding of beneficiary perceptions of project achievements; changes in key outcomes and reasons why; and additional information from project Implementing Partners (IPs) and related stakeholders on issues of project effectiveness, sustainability, unintended consequences, and lessons learned. In addition to the quantitative PBS data and primary qualitative data collected from project beneficiaries and other key stakeholders, the final PE also drew on secondary data from IP performance monitoring data, key IP documents, and the midterm evaluation. The availability of pre/post quantitative data on FFP indicators substantially increased the ability for the Evaluation Team (ET) to derive evidence-based findings, conclusions, and recommendations for this evaluation. The mixed-methods design also utilized qualitative data collection protocols, which were designed to take into account preliminary findings from the quantitative PBS, to collect primary qualitative data from ViM beneficiaries and other key stakeholders to help interpret findings and provide support for recommendations. 3.1 QUANTITATIVE DATA COLLECTION AND ANALYSIS: PBS 3.1.1 Sampling and Data Collection Following the method adopted in the baseline, the endline quantitative household survey is a PBS generating endline indicator values representative of the overall project area for ViM. The target population for the endline PBS for the ViM final evaluation is all households in the ViM project implementation area, consisting of all villages that received the ViM project intervention. This consists of 201 villages, across four communes in Sanmatenga province, covering approximately 35,000 households. The baseline sample size estimation was done with a target of detecting a six percentage-point reduction in the stunting rate among children below five years of age between the baseline and endline, the endline sample size was adjusted to be sufficient to detect a smaller effect size of five percentage-point reduction. As a result, the endline sample size was estimated taking into consideration the new effect size sought and the sample size achieved at the baseline. 9 FINAL REPORT – VICTORY AGAINST MALNUTRITION (ViM) FINAL EVALUATION Using a process described in detailed in Annex V, the endline evaluation team found that a total 2,217 households needed to be surveyed from 50 villages to achieve the target sample size of 1,843 children in the age group 0-59 months to detect a five percentage-point reduction in the stunting rate between the baseline and endline. Sampling of Villages for Endline Survey The endline evaluation team replicated the multi-stage sample selection method used for the quantitative survey in the baseline. Accordingly, a fresh sampling exercise drew a new sample of 50 villages from the 201 villages that received ViM intervention during 2012-2017 using Probability Proportional to Size (PPS) sampling method to ensure a reasonable balance between statistical and operational efficiency. The “size measure” was the number of households of each village from the most recent census in Burkina Faso (RGPH 2006). Household Listing In order to identify the households to be surveyed in the endline survey, following the method adopted in the baseline survey, the field teams completed a listing of all households in each of the 50 selected endline sample villages during October 13-19, 2017. The coverage of this household listing needed to be accurate and comparable across baseline and endline, to avoid geographical bias and other possible sources of bias. The endline evaluation team was alerted of several villages where the geographical borders shifted from 2012 to 2017; following ViM project staff’s preference, the endline survey teams performed the listing exercise based on borders active in 2012. For this evaluation, NORC used the following definition of a household from Burkina Faso’s National Institute of Statistics and Demography (INSD): A household is a group of persons, related or not, living under the same roof (or on the same plot), sharing meals, recognizing the authority of one individual as the head of household, and for which the resources and spending are pooled, or at least partly. The listing data was collected using tablets and the Survey Solutions tablet application. At each identified household, enumerators gathered the following information: date, community name, neighborhood name, number of structures on the plot, number of housing structures on the plot, name of household head, number of household members, names of spouses, number of children under five years, address/directions to household, and telephone number of household head. Before moving to the next household, enumerators placed a number written in chalk on the main dwelling of the household, as an indicator to any other field team member that this household was already listed. In all, over 17,000 households were enumerated during the listing exercise. The number of households listed during the exercise was compared to the 2006 census estimate of the village by NORC and the field teams were alerted to any major differences between them. The field supervisors were responsible for confirming the differences with the village leaders to ensure that the listing data accurately reflect the current village population7. Sample Selection After completion of household listing, NORC conducted a random selection of households for survey administration. To conduct the random selection, all households in a village were assigned a random number and the 45 households with the smallest random numbers were selected for the endline sample. In addition, the 10 households with the next largest random numbers were selected to serve as 7 Village populations at endline compared to baseline ranged from 70% to about 150%. The main contributors to the changes in village populations was population movement in and out of villages, and changes in village boundaries since the 2006 census. 10 FINAL REPORT – VICTORY AGAINST MALNUTRITION (ViM) FINAL EVALUATION replacement households in the event that a sample household was not available for interview. The random selection of households was conducted using Stata software, and used a fixed starting seed number insuring that the randomization process was replicable. In the final sample and replacement list, each household was assigned a unique household identification number, in order to track that the interview data captured information for the selected households. Data Collection The quantitative survey was piloted from November 22- 24, 2017, and data was collected in the field from November 25, 2017 through December 16, 2017. Follow up interviews were conducted from December 17-21, 2017. Table 2: Endline Household Survey Sample Structure Commune Number of Villages Number of households Selected Actual sample N % achievement Selected Actual sample N % achievement KAYA 14 14 100% 630 622 99% BARSALOGHO 16 16 100% 720 716 99% NAMISSIGUIMA 2 2 100% 90 90 100% PISSILA 18 18 100% 810 811 100% TOTAL 50 50 100% 2250 2239 99% Quality Control Quality control measures taken during endline survey fielding included high-frequency validation and reliability checks, daily data uploads during quantitative data collection and weekly production reports. The evaluation team regularly reviewed the survey data for quality and provided necessary feedback to the local data collection firm during survey fielding. To ensure the quality of anthropometry measurements, field supervisors were trained in anthropometry measurement processes, and conducted oversight and spot checks of the anthropometry measurements made by each of the teams. Lastly, a local coordinator member of the evaluation team (independent of the data collection firm), provided in￾country oversight and served as an additional backstop to overall quality and enumerator comprehension and conduct of survey implementation during survey fielding. Quality measures taken during the data processing stage, included standard data quality review and cleaning to ensure the correct entry of household unique identification numbers, checks for missing values on critical items, and checks inconsistencies between questions., range checks on numeric and text responses, detecting and correcting, removing, or flagging incorrect data, errors in format, and incomplete or inconsistent data. 3.1.2 Analysis of Quantitative Data The final PE analyzed quantitative data collected via a population based survey of households from selected ViM project villages to identify the changes in key outcome indicators between the baseline and endline. The quantitative analysis also investigates the role of ViM in influencing the observed changes and also in promoting good practices. The quantitative analysis included descriptive statistical analysis consisting of calculation of indicators and documenting the change in indicators between baseline and endline through comparative analyses. To obtain a deeper understanding of how results vary across subgroups and extent of exposure to different ViM activities, the descriptive analysis also compared indicators between disaggregated 11 FINAL REPORT – VICTORY AGAINST MALNUTRITION (ViM) FINAL EVALUATION beneficiary and non-beneficiary groups8 and between male and female beneficiaries. Beneficiary or non￾beneficiary status was defined separately for different sets of indicators, and based on whether surveyed respondents directly participated in a set of pre-defined ViM activities or trainings related to that indicator. Many indicators may encompass multiple types of trainings, activities, and/or the receipt of food rations. A respondent was considered to be a beneficiary if he/she was involved in at least one training, activity, or received food rations. For all individual level indicators, the individual’s beneficiary status rather than the household beneficiary status was used in analysis. Table 3 provides an overview of the key project activities or trainings that were used to determined status as a direct beneficiary, for indicators under each of the SOs. A more detailed description of how beneficiary status was defined for each indicators in listed in Annex III. Farmer level interventions that determine beneficiary status included receipt of: agricultural inputs (for crops or livestock), trainings (on plants, animals, NRM, value chains, financial management, or farming as a business), crop equipment, or a plot of land. Beneficiary status for the woman level indicators was determined by whether the woman was a member of a Care/Neighborhood Group or received food rations. Beneficiary status for child level indicators beneficiary status was determined by whether the child received food rations. Household level variables use whether anyone in the household received training (on agriculture, livestock, women’s health and nutrition, child health and nutrition, SILC, gender, or natural resources) or rations (for children, PLW, or household) that relate to the indicator. In addition to the descriptive statistical analyses described above, the evaluation team also undertook econometric analyses to assess how various interventions supported by the ViM project affected key outcome indicators of interest. After controlling for potential confounders, the econometric analyses explored how these interventions implemented by ViM contributed to observed outcomes. Table 3: Beneficiary Status Determination by Indicator Indicator Beneficiary Status Determination SO1: Increased and diversified agricultural production IM1 Agricultural inputs crops/livestock, extension services plant/animal, crop equipment, or plot of land IM2 Agricultural inputs crops, extension services plant, crop equipment, or plot of land IR. 1.1: Improved Agricultural Practices Adopted OC1 Agricultural inputs crops/livestock/NRM, extension services plant/animal, crop equipment, plot of land OC2-OC4 Agricultural inputs crops, extension services plant, crop equipment, or plot of land IR 1.2: Smallholder access to agricultural inputs improved OC6 Agricultural inputs crops, extension services plant, crop equipment, or plot of land IR 1.3: Improved livestock practices adopted OC7-OC8 Agricultural inputs livestock or extension services animal SO2: Increased household incomes IM3-IM4 Women’s health & nutrition and child health & nutrition trainings, or child, PLW, or household rations OC9 SILC training 8 See Annex III for detailed charts on how beneficiary/non-beneficiary status was determined for each indicators, based on the types of interventions directly received by the surveyed respondents. 12 FINAL REPORT – VICTORY AGAINST MALNUTRITION (ViM) FINAL EVALUATION IR 2.1: Market linkages improved OC9_2-OC10 Agricultural inputs crops, extension services plant, crop equipment, or plot of land OC11 Extension services, financial management and farming as a business OC12 Agricultural inputs, crops/livestock, extension services plant/animal/value chain, crop equipment, or plot of land IR 2.2: Access to business development services improved OC13-OC14 Extension services, financial management and farming as a business SO3: Reduced chronic malnutrition among children under five years and PLWs IM5-IM6 Child rations OC15 Care/neighborhood groups, or woman rations IR 3.1: Improved MCHN practices OC16 Child rations IR 3.2: Improved quality of health services OC19 Care/neighborhood groups OC20-OC21 Child rations OC22-OC23 Care/neighborhood groups OC24-OC25 Child rations IR 3.3: Improved hygiene and sanitation practices OC27-OC29 WASH training Crosscutting: Gender OC33 Care/neighborhood groups Crosscutting: Environment OC36-OC38 Agricultural inputs crops/livestock/NRM, extension services plant/animal, crop equipment, plot of land 3.2 QUALITATIVE DATA COLLECTION AND ANALYSIS The qualitative component of the ViM Final Evaluation drew on three data collection methods: • Desk review of IP quarterly and annual results reports, monitoring and evaluation (M&E) data, and relevant secondary information; • 31 gender-segregated group discussions (GDs) conducted at endline with project beneficiaries across a total of 16 villages; and • 36 Key informant interviews (KIIs) conducted across village, commune, prefect, and national levels. KIIs were conducted with a range of government officials, members or leaders of ViM￾established or supported groups or committees (such as producers’ unions, CLTS and CVD committees), and local implementer or other partner groups9. Qualitative data collection utilized purposive sampling and semi-structured GD and KII protocols. To help provide additional context and interpretation for quantitative results, the qualitative tools design 9 See Annex VII for the qualitative survey instruments. 13 FINAL REPORT – VICTORY AGAINST MALNUTRITION (ViM) FINAL EVALUATION took into account preliminary results from the quantitative PBS. The qualitative fieldwork for the endline PE was conducted in January 2018. 3.2.1 Group Discussions The evaluation team conducted 31 GDs at endline, consisting of two gender-segregated GDs per sampled village across 16 villages. GDs comprised 325 participants in total (118 men and 207 women)10. Fifteen of the 16 villages selected for qualitative fieldwork were selected at random from a list 25 villages sampled at endline for the quantitative data collection that were sufficiently accessible to allow for completion of the village GDs and KIIs within one day of fieldwork. The 16th village was selected to purposively include a village from rural Namissiguima, at ViM’s request. GDs were organized with the assistance of community leaders and ViM field agents, and were comprised of a maximum of 12 participants. A French and Mooré speaking moderator conducted the GDs in Mooré, the local language predominantly spoken in the ViM implementation zones. GDs were generally held outside, at common gathering areas, in the respective villages. Semi-structured instruments with questions organized by evaluation theme were used to guide the discussion (see Annex VII). 3.2.2 Key Informant Interviews At endline, a total of 36 KIIs were conducted by members of the evaluation team, with oversight by the team’s MCHN subject matter expert. These KIIs include 11 at the district/province level, 12 at the district/province and commune/department levels, and 13 at the village level. The interviews followed a semi-structured format to allow for follow-up questions and flexibility in the discussion. The KII protocol was structured to gather information on: the extent to which activities have been achieved, and on what factors promoted or inhibited project activities and outcomes; perspectives on the effectiveness of project interventions and targeted groups; quality of services provided; motivations and capacity to demand and sustain services; and the sustainability of project interventions and outcomes (see Annex VII for KII protocols). 3.2.3 Village Selection for Qualitative Data Collection Village selection for qualitative data collection was based on maximizing coverage on a range of different project activities, drawn from IP data on activity implementation by village, and maximizing variation on village distance to a regional town (within a two-hour travel time, for logistical feasibility). Village selection was stratified by commune, and done in collaboration with inputs from the ViM M&E team. 3.2.4 Desk Review of IP and Secondary Documentation The purpose of the desk review of IP and available secondary documentation was to identify key findings and explanatory factors from IP reports and internal M&E data pertaining to each of the outlined Evaluation Questions (EQs). Examination of key documents was also used as a source of triangulation for qualitative data provided by project beneficiaries and KIIs or to help interpret or provide explanatory context for both PBS and qualitative results. Desk review for this evaluation primarily consisted of reviews of ViM quarterly and annual reports, exit strategy documents, and M&E data and methods write-ups. 3.2.5 Qualitative Data Analysis GDs were recorded, with consent from participants, and transcribed from Mooré to French. Transcribed GD data and detailed GD notes were then coded in NVivo according to a pre-defined codebook coding text segments according to key themes of interest for the EQs. KII notes were summarized using standard content analysis techniques. For both KIIs and GD data, analyses summarized 10 30 GDs were analyzed, because the Women’s GD from the Tibi-Yarce village was not delivered. The participant total for each gender listed here reflects this. 14 FINAL REPORT – VICTORY AGAINST MALNUTRITION (ViM) FINAL EVALUATION common trends and patterns to highlight project-, sector-, and gender-differentiated trends, and to identify examples of positive deviance and most significant change. 3.3 EVALUATION LIMITATIONS AND CONSTRAINTS 3.3.1 Lack of Valid Counterfactual for Quantitative PBSs The pre/post quantitative design without a valid counterfactual at both time points cannot definitively attribute changes in program indicators to the ViM project activities. However, companion qualitative data collection at endline aims to: help corroborate the PBS-based results; capture information on potential confounders; and understand beneficiary perceptions on if and how project activities helped to elicit change in key outcomes. In addition, supplemental econometric analyses were conducted to test for correlations between beneficiary participation in or exposure to ViM activities and select outcome indicators, to help isolate contributions of ViM activities to observed outcomes. 3.3.2 Small Sample Size for Target Value Chain Crops There were four targeted value chain crops for analysis on yields, intercropping, gross margin calculations; sorghum, cowpeas, onions, and tomatoes. However, sample size limitations for onions and tomatoes prevented analysis for these crops. Onions and tomatoes were not major crops at baseline with 42 farmers growing onions and 22 farmers growing tomatoes out of the 1,916 farmers that grew crops (2.2% and 1.1% of farmers respectively). At endline, 54 farmers grew onions and 33 farmers grew tomatoes out of the 3,810 farmers who grew crops (1.4% and 0.9% respectively). This small sample size made testing differences and significance levels difficult, so these crops were not included in the yields, intercropping, or gross margin indicators. 3.3.3 Difference in Seasonal Timing of Data Collection between Baseline and Endline PBSs The baseline data collection for the PBS took place in July-August (2012), while the endline survey was conducted in December (2017). This difference in seasonality across the two rounds of data collection could contribute to differences in some of the indicator estimates.11 In terms of a typical seasonal calendar, July-August is the lean season, where households engage in peak wild foods harvest. In December, farmers are in the midst of the main harvest season (see Figure 2). In particular, dietary diversity, prevalence of household hunger, and prevalence of minimum acceptable diet for children may be more sensitive to this timing difference. Dietary diversity and prevalence of a minimum acceptable diet would be expected to be higher in the harvest season, and prevalence of household hunger would be expected to be lower. To help control for the timing difference, questions on household hunger in July (the middle of the lean season, and consistent with the timing of baseline data collection) were added to the endline questionnaire. In addition to this, it is noted that the agricultural data collection at endline refer to the 2016/2017 agricultural campaign, while the baseline data collection similarly refered to the 2011/2012 agricultural campaign. The seasonal shift of endline data collection from July-August to December could be associated with a higher level of recall bias on crop harvests by respondents, due to the longer time length since harvest relative to baseline. However, some level of recall bias is expected in both rounds. 11 Non-nutrition indicators that may be affected are HDDS and MAD. For agricultural indicators, because the main harvest season was underway during endline, there may also be a seasonal influence due to differences in accuracy of farmer recall for self-reported harvests at baseline vs. endline, since baseline was conducted during the off￾season. 15 FINAL REPORT – VICTORY AGAINST MALNUTRITION (ViM) FINAL EVALUATION Figure 2: Burkina Faso Seasonal Calendar (Typical Year) (Source: Famine Early Warning Systems Network (FEWS NET), accessed at: http://www.fews.net/west-africa/burkina-faso 3.3.4 Methodological Approach for Yield Data Collection at Baseline and Endline For the yield of targeted value chain crops (sorghum, cowpeas, onions, and tomatoes12), methodological issues contributed to difficulties for the endline evaluation team to accurately assess changes between baseline and endline. The baseline and endline survey collected farmer self-reported estimates of the total area cultivated for each of nine crops13, and corresponding production. Based on the self-reported measures, yield estimates were constructed for each of the crops for baseline and endline. Given the lack of precision in many of the production units and in farmer self-reported field size, the yield figures are necessarily best approximations. However, they are based on a consistent data collection approach for both rounds that was also consistent with the time and budget constraints on the ViM evaluation. At endline, the conversion factor from local units to kilograms for crop production was updated to improve accuracy of the production data, by requiring field teams to obtain measurement conversions in the field by crop, however this can also introduce comparability issues between survey rounds14. There are well-known challenges associated with the reliability of farmer self-reporting on both farm size and crop production data, and such data are inherently expected to have high variability15. Recent work in other African smallholder contexts has found that farmers tend to systematically over-estimate their yields on smaller plots, and under-estimate yields on larger plots16. Use of GPS technology to obtain accurate plot areas, and crop-cut methods to obtain yields, are both recommended to obtain more accurate measurements of crop production. But, these methods also have some level of measurement error, including sampling error stemming from the extrapolation of yields from a small 12 Crop yields for onion and tomatoes could not be calculated, due to small sample sizes in both the baseline (42 farmers grew onions and 22 grew tomatoes) and endline survey rounds (54 farmers grew onions and 33 grew tomatoes). 13 The 9 crops were; sorghum, millet, maize, rice, cowpeas, onions, tomatoes, sesame, and peanuts. 14 The endline evaluation team considers the yield data produced at endline to have improved reliability over the baseline data. 15 See Carletto, C., S. Savastano, and A. Zezza (2013) Fact or artifact: The impact of measurement errors on the farm size-productivity relationship. Journal of Development Economics 103:254-261. 16 Desiere, S. and D. Joliffe (2018). Land productivity and plot size: Is measurement error driving the inverse relationship? Journal of Development Economics 130:84-98. 16 FINAL REPORT – VICTORY AGAINST MALNUTRITION (ViM) FINAL EVALUATION crop square to a larger and typically heterogenous farm area17. Despite the gains to reliability in yield data, the use of such methods are much more costly than standard self-reporting and are often cost￾prohibitive in practice for large household surveys due to the substantial additional time and financial resources required to conduct them at scale18. This discussion on methods is relevant for the ViM final evaluation, because of perceived inaccuracies by ViM in the baseline self-reported agricultural data, which were communicated to the endline evaluation team during the final evaluation. According to the evaluation team’s understanding, ViM determined the baseline agricultural data to be sufficiently unreliable, and instead considered GoBF national statistics obtained through Agristat or FAOstat as the baseline source of yield data for sorghum, cowpeas, onions and tomatoes, together with ViM project internal M&E data obtained from a sample of beneficiary household using GPS-based plot measurements or the crop squares method. For one other agricultural indicator, the percentage of farmers using improved agricultural inputs, the evaluation team understanding is that ViM also considered the baseline data to be unreliable, and drew instead on GoBF Ministry of Agriculture data as the project baseline for this indicator. For these four indicators, after discussion with ACDI/VOCA and ViM, the endline evaluation team calculated the baseline to endline change using the original baseline data, but has not reported statistical tests of significance for baseline to endline change in this document, due to the project’s communicated view that the baseline values for these four indicators were considered invalid. The evaluation team has noted these instances throughout the report. The evaluation team is not able to use the GoBF, Agristat, FAOstat or internal ViM project annual M&E data as a substitute for the baseline data for these indicators, for the purposes of conducting statistical tests of baseline to endline changes, because these alternative data were obtained using different sampling frames and data collection approaches than the baseline and endline data collection. In particular, in addition to the difference in methods to obtain the data, the sampling frames were also not designed to be representative and generalizable to the population level in the ViM project area (regardless of ViM beneficiary status). As a result, these data unfortunately cannot be used to conduct a valid statistical analysis of change on these indicators in the ViM project area over the project lifetime. 3.3.5 Recall Bias in Qualitative Data Collection Some evaluation topics, such as perceptions about the beneficiary selection processes that occurred early in program implementation, may be difficult for respondents to remember accurately as time passes. Careful construction of question wording on interview guides, probing for clarification, and triangulation across GDs and KIIs were used to mitigate the potential for recall bias to influence results. PE teams had little indication of serious issues related to this during the qualitative data collection. 3.3.6 Selection Bias for Qualitative Data Collection For the qualitative component of the PE, village selection was purposive, stratified by commune, and aimed to maximize coverage on the breadth of project activities in selected villages while including villages across a range of accessibility from Kaya (based on distance from regional town in kilometers and total travel time). Village selection for the qualitative data collection aimed for representativeness to the extent possible, but it is noted that purposive qualitative data collection is inherently non￾representative. Moreover, respondents who participated in GDs and were willing to share their views may not be representative of all project participants, or may be different in key observable or 17 Carletto, C, S. Gourlay and P. Winters (2015). From Guesstimates to GPStimates: Land area measurement and implications for agricultural analysis. Journal of African Economies 24:593-628. 18 Kilic, T., A. Zezza, C. Carletto, and S. Savastano (2017). Missing(ness) in action: Selectivity bias in GPS-based land area measurements. World Development 92:143-157. 17 FINAL REPORT – VICTORY AGAINST MALNUTRITION (ViM) FINAL EVALUATION unobservable ways. The ET sought to mitigate the potential for biased results by recruiting respondents with a range of experiences and beneficiary roles for the qualitative data collection, and also by triangulating information across different types of project beneficiaries and stakeholders. 3.4 DESCRIPTIVE SUMMARY OF POPULATION CHARACTERISTICS Prior to presenting evaluation findings by evaluation question, key demographic and related characteristics of surveyed population are briefly summarized. The survey sample consisted of 2,239 households, 4,032 individual farmers, 3,400 women aged 15-49, and 3,308 children under five years of age. Nintey percent of households in the sample were male-headed, and the mean age of the household head was 45.6 years of age. Eighty-five percent of household heads reported no formal schooling, while nine percent reported primary schooling. The average household size was 7.7 people. Eighty-one percent of surveyed households reported at least one household member who had participated in a ViM training or received food rations through the project, and fifty-three percent of households in the sample reported participation in both of these key project activities. 4. FINDINGS AND CONCLUSIONS 4.1 EQ 1: TO WHAT EXTENT WAS THE VIM PROJECT EFFECTIVE IN ACHIEVING STRATEGIC OBJECTIVES AND INTERMEDIATE RESULTS? FINDINGS 4.1.1 Integrated Quantitative and Qualitative Findings by Sector and Strategic Objective: Project Achievements and Key Factors, Overall and by Subgroups This chapter summarizes the endline to baseline statistical results for key outcome and impact level indicators, and provides a discussion of those results by SO. A condensed version of the baseline to endline indicator values and statistical results are presented in Table 4 below. The full set of summary tables of endline indicator values and the endline to baseline statistical results are presented in a series of tables in Annex I, The main take-away points from the statistical analysis of PBS baseline (2012) and endline (2017) data for the ViM project are as follows: • For SO1 indicators, quantitative and qualitative data provide an indication of widespread improvements in adoption of agricultural practices, increased crop yields as well as some diversification. While the PBS and t-test results for a change in sorghum and cowpeas between baseline and endline are not reported due to methods limitations, qualitative data provided strong support for widespread improvements in crop yields in the ViM project area. The use of at least four sustainable agricultural practices at endline was 74% of farmers. The percentage of farmers who used improved storage increased from 48.1% at baseline to 74.3% at endline. Qualitative data from GDs and KIIs indicate that improved storage for cowpeas was seen as instrumental for improved crop preservation, and for facilitating improvements in marketing and sales. • Several indicators in SO2 show improvements in household income over the project lifetime. These indicators include a decrease in household hunger and improvements in market linkages, business development, use of financial services, and the percentage of farmers practicing value chain activities. • For SO3, there have been large and statistically significant improvements in mother and child health and nutrition, including for stunting, underweight, women’s dietary diversity, quality of health services, and sanitation/hygiene. 18 FINAL REPORT – VICTORY AGAINST MALNUTRITION (ViM) FINAL EVALUATION • Women’s empowerment has seen substantial improvement through women’s decision-making power and changed perceptions of gender-based violence. • There are mixed results for improvements in crosscutting environmental issues, but a large and statistically significant improvement in the percentage of farmers using environmental mitigation principles in diversified income generating activities. The qualitative data collected for this evaluation provides a complementary set of information to further understand elements of ViM project effectiveness in certain sectors. Qualitative results suggest that ViM activities were impactful for beneficiaries overall, and well-implemented by the project. Beneficiaries cited several ViM activities they deemed to be helpful, including and not limited to literacy courses (three GDs), implementation of learned agricultural production techniques (seven GDs), safe conservation of crops (five GDs), hygiene awareness raising activities (two GDs), increased access to health care (six GDs), livestock raising and care techniques (five GD participants), and porridge making (one GD). Qualitative results are discussed in further detail, by broad sector (Agriculture vs. MCHN and WASH) in EQ2 and EQ3, with additional results by topic incorporated into the ensuing report discussions on EQs 4 – 11. “All the activities carried out by the ViM Project have been beneficial to us. Thanks to the training and equipment, we benefitted from things have changed positively. In order of preference what has helped us a lot is the chemical fertilizer [and] the carts. We cannot count the benefits. Even the technique of zaï, stones bunds and half-moons are all activities that have had a considerable impact on the improvement of our productions.” [Firka Men, GD] 19 FINAL REPORT – VICTORY AGAINST MALNUTRITION (ViM) FINAL EVALUATION Table 4: ViM Baseline and Endline Indicator Mean Values, and Baseline to Endline Indicator Comparisons. Average or % Difference (EL-BL) Significa nce Level Number of observations Baseline Endline Baseline Endline Strategic Objective 1: Increased and diversified agricultural production IM1 Average number of different types of agricultural products produced per farmer 4.36 4.32 -0.03 ns 1,916 3,923 IM2 Average yield of [crop] per farmer (kg/ha) Sorghum 68319 515 -168 *** 978 1,884 Cowpeas 21720 462 245 *** 905 2,961 IR. 1.1: Improved Agricultural Practices Adopted OC1 Percentage of farmers who used at least FOUR sustainable agriculture (crop/livestock and/or NRM) practices and/or technologies in the most recent season 75.9% 74.5% -1.4% ns 1,891 3,940 OC2 Percentage of farmers using intercropping Sorghum 23.7% 22.2% -1.5% ns 1,308 2,137 Cowpeas 36.2% 21.5% -14.7% *** 1,317 2,998 OC3 Percentage of farmers using thinning for sorghum 34.0% 33.3% -0.7% ns 1,308 2,137 OC4 Percentage of farmers who used an improved storage technique in the most recent season for cowpeas 48.1% 74.3% 26.2% *** 924 2,221 IR 1.2: Smallholder access to agricultural inputs improved OC6 Percentage of farmers using improved agricultural inputs 73.8%21 70.7% -3.1% ** 1,839 3,822 IR 1.3: Improved livestock practices adopted OC7 Percentage of farmers using at least veterinary care, complementary feeding and habitat construction at the same time 10.2% 3.5% -6.6% *** 1,389 2,786 OC8 Percentage of farmers accessing government or private sector veterinary care and vaccinations 35.9% 35.8% -0.2% ns 1,389 2,786 Strategic Objective 2: Increased household incomes IM3 Average Household Dietary Diversity Score (HDDS) 4.019 3.942 -0.077 ns 1,002 2,153 IM4_0 Prevalence of households with moderate or severe hunger 43.2% 3.6% -39.6% *** 1,057 2,235 IM4_1 Prevalence of households with moderate or severe hunger in July 43.2% 11.2% -32.0% *** 1,057 2,235 OC9_0 Average value of a set of assets (not including livestock) per household (CFA) 170,182 228,980 65,281 *** 1,076 2,206 IR 1.3: Improved livestock practices adopted OC10_0 Value of purchases from smallholders of targeted commodities as a result of USG assistance 29,940 58,367 29,567 *** 833 2,910 OC10_1 Percentage change in value of purchases from smallholders of targeted commodities as a result of USG assistance N/A 48.7% N/A N/A N/A N/A OC11 Percentage of rural smallholders reporting more than THREE sources of revenue 1.1% 1.7% 0.6% * 1,916 3,990 OC12_0 Percentage of farmers who practiced the value chain activities promoted by the project in the most recent season (crops only) 37.4% 70.3% 32.8% *** 1,840 3,803 IR 2.2: Access to business development services improved OC13_0 Percentage of farmers who used financial services in the most recent season 28.6% 35.5% 6.9% *** 1,906 3,990 19 The ViM project did not accept this baseline value, and instead used 718.7 kg/ha as the baseline value for this indicator, obtained from Agristat data, from 2006 to 2010; per ViM’s approved 2017 IPTT. 20 The ViM project did not accept this baseline value, and instead used 851 kg/ha as the baseline value for this indicator, obtained from Agristat data, from 2006 to 2010; per ViM’s approved 2017 IPTT. 21 The ViM project did not accept this baseline value, and instead used 10.0% as the baseline value, which was from the Ministry of Agriculture; per ViM’s approved 2017 IPTT. 20 FINAL REPORT – VICTORY AGAINST MALNUTRITION (ViM) FINAL EVALUATION Average or % Difference (EL-BL) Significa nce Level Number of observations Baseline Endline Baseline Endline OC14 Percentage of farmers reporting access to up-to-date pricing information 50.2% 89.0% 38.7% *** 1,220 2,882 Strategic Objective 3: Reduced chronic malnutrition among children under five years of age and pregnant and lactating mothers IM5 Prevalence of stunted children under five years of age 31.2% 25.6% -5.6% *** 1,115 3,021 IM6 Prevalence of underweight children under five years of age 24.7% 19.7% -5.0% *** 1,130 3,051 OC15 Women’s Dietary Diversity: Mean number of food groups consumed by women of reproductive age 2.86 3.29 0.43 *** 1,458 3,400 IR 3.1: Improved MCHN practices OC16 Prevalence of exclusive breastfeeding of children under six months of age 36.8% 60.3% 23.5% *** 155 325 OC17 Percentage of children born in the last 24 months who were put to the breast within one hour of birth 39.5% 68.0% 28.4% *** 612 1,243 OC18 Prevalence of children 6-23 months receiving a minimum acceptable diet (MAD) 4.9% 8.6% 3.7% ** 469 927 IR 3.2: Improved quality of health services OC19 Percentage of mothers with children age 0-23 months who had four or more antenatal visits when they were pregnant with their youngest child 42.3% 62.5% 20.2% *** 572 1,101 OC21 Percentage of children 6-23 months who received vitamin A supplementation within the past 6 months 64.6% 72.8% 8.2% *** 460 900 OC23 Percentage of women who received vitamin A supplementation post-partum 54.1% 79.2% 25.1% *** 764 1,100 OC24 Percentage of children 12-23 months of age children fully immunized 34.5% 50.5% 16.0% *** 290 774 OC25 Percentage of children 0-23 months who have a growth monitoring card showing at least one weighing in the last 2 months 59.2% 49.8% -9.3% ** 262 588 IR 3.3: Improved hygiene and sanitation practices OC27 Percentage of households with children aged 0-23 months that have water and soap or a locally available cleansing agent at a hand washing place 1.2% 20.4% 19.2% *** 591 1,094 OC27a Percentage of households with soap and water at a hand washing station commonly used by family members 1.0% 17.0% 16.0% *** 1,267 2,237 OC28 Percentage of households with access to an improved sanitation facility 0.6% 23.5% 22.8% *** 1,094 2,237 OC29 Percentage of respondents who know all critical moments for hand washing to prevent diarrheal disease 1.8% 40.7% 39.0% *** 617 1,252 Crosscutting: Gender OC33 Percentage of currently married women age 15-49 reporting that she makes decisions either by herself or jointly with her spouse regarding seeking health services for her own health; seeking health services for her children's health; how to spend money she herself has earned 62.4% 72.4% 10.0% *** 1,337 2,923 OC34 Percentage of women who agree that it is justifiable for a man to hit his wife under one or more circumstances 91.0% 50.6% -40.3% *** 1,338 3,400 Crosscutting: Environment OC36 Percentage of beneficiaries adopting at least THREE environmentally sustainable technologies 84.7% 84.3% -0.4% ns 1,891 3,940 OC37 Percentage of rural smallholders reporting use of at least TWO environmental mitigation principals in diversified Income Generating Activities 53.0% 66.2% 13.3% *** 1,123 2,248 OC38 Percentage of smallholder farmers demonstrating knowledge of two sound pest management practices 31.6% 52.3% 20.6% *** 1,904 3,990 21 FINAL REPORT – VICTORY AGAINST MALNUTRITION (ViM) FINAL EVALUATION SO1: Increased and Diversified Agricultural Production ViM’s SO1 focused on increasing and diversifying agricultural production. Two impact indicators were used to assess changes in agricultural production during the project lifetime: the average number of agricultural products produced per farmer (Impact 1 (IM1)), and the average yield of value chain crops per farmer (IM2). In addition, OC indicators for adoption of improved agricultural practices, livestock practices and smallholder access to agricultural inputs were also assessed. For the average number of agricultural products produced per farmer (IM1), which includes both crop and livestock22, farmers on average produced 4.3 products at endline, and there was no significant change in this indicator between baseline and endline. However, the endline data indicates that beneficiary farmers did produce a significantly higher number of different types of agricultural products than non-beneficiaries, producing 4.8 products on average compared to 4.0 products for non￾beneficiaries. Additionally, male farmers produced a significantly greater number of agricultural products than female farmers at endline (5.4 products for men, compared to 3.3 products for women), although there were no differences by gender between baseline and endline. Comparison of types of crop produced by farmers reveals a potentially important change in the cropping pattern as well. The percentages of surveyed farmers growing each of the three primary staple crops of Burkina Faso, sorghum, millet, and maize, all decreased at endline (Figure 3). These crops are primarily used for household consumption; at baseline and endline, only around 3-5% of farmers who grew these crops indicated that they sold some portion of the harvest. Noting that the percentage of farmers growing cowpeas and sesame increased over the same period, the decrease in sorghum, millet and maize planting may reflect a shift by some farmers to grow cash crops instead of food crops. The endline data collection necessarily focused on the same set of crops that were prioritized at baseline. The data indicates a doubling in the percentage of farmers growing sesame, a traditional cash crop, (24% at endline, compared to 12% at baseline), providing additional support for a possible shift towards cash crops in the ViM intervention area during the project lifetime. Additionally, only 41% of farmers who grew sesame at baseline reported that they sold sesame, compared to 80% of sesame farmers at endline, which also suggests increased focus on cash crops and commodity production. 22 See Annex II for list of crops and livestock. 22 FINAL REPORT – VICTORY AGAINST MALNUTRITION (ViM) FINAL EVALUATION Figure 3: Changes in Crop Production (by Percentage of Farmers reporting production) 23 *** p<0.01, ** p<0.05, * p<0.1 The percentage of farmers who owned livestock did not change between the two survey rounds, where approximately 70% of households owned livestock in each round. However, the number of types of livestock owned increased slightly from 2.2 to 2.4 and the average total number of livestock units owned also increased from 10.3 to 11.5. Additionally, the number of livestock sold per farmer increased from 3.2 at baseline to 7.5 at endline, which suggests that total number of livestock owned over the course o the years was much higher at endline. The percentage of farmers owning cattle, donkeys, shoats, and poultry all increased, and the average number of poultry per farmer owning poultry also increased. Overall, the results suggest an expansion in livestock holdings among households in the ViM project area during the ViM project lifetime. The PBS and t-test results for a change in sorghum yield (IM2) between baseline and endline are not reported (see section 3.3.4).The reported yields based on farmer self-reporting are 683 kg/hectare at baseline and 515 kg/hectare at endline. For ViM beneficiary farmers, sorghum yield was significantly higher at endline (576 kg/hectare) than for non-beneficiaries (461 kg/hectare). Male farmers had higher sorghum yields at endline (558 kg/hectare) than female farmers (437 kg/hectare). Overall, the area cultivated for sorghum decreased from 1.6 hectares at baseline to 1.3 hectares at endline. Taking into account the potential unreliability of farmer self-reported crop areas discussed earlier in the report, it is also possible that this decline could be associated with an increased trend in cash cropping, as discussed above. The PBS and t-test results for a change in cowpea yield (IM2) between baseline and endline are also not reported (see section 3.3.4). The reported yields based on farmer self-reporting are 217 kg/hectare at baseline and 462 kg/hectare at endline. At endline, there was no difference in yields for ViM beneficiaries 23 For all charts in this chapter, results show the statistical test of differences between endline and baseline values obtained through the ViM baseline and endline population-based household surveys. Statiscal significance is denoted as: *** p<0.01, ** p<0.05, * p<0.1. Results marked with *** are considered highly statistically significant. Where results have no stars, the difference between endline and baseline values is not statistically significant. 0% 10% 20% 30% 40% 50% 60% 70% 80% Sorghum Millet Maize Rice Cowpeas Onion Tomatoes Sesame Peanuts Percent of Farmers Growing Baseline Endline *** *** *** *** 23 FINAL REPORT – VICTORY AGAINST MALNUTRITION (ViM) FINAL EVALUATION and non-beneficiaries, while male farmers had higher cowpea yields than female farmers. Keeping in mind the potential unreliability of farmer self-reported crop areas, no difference in the area cultivated for cowpeas was found between baseline and endline. The yields at endline for the other main crops that farmers grew were 514 kg/hectare for millet, 473 kg/hectare for maize, 191 kg/hectare for sesame, and 304 kg/hectare for peanuts24. These are also not compared to the baseline values due to the issues discussed in section 3.3.4. IR. 1.1: Improved Agricultural Practices Adopted The four indicators under IR 1.1 are focused on capturing changes in the adoption of improved agricultural practices: percentage of farmers who used at least four sustainable agriculture practices (OC1), percentage of farmers using intercropping (OC2), percentage of farmers using thinning (OC3), and the percentage of farmers using improved storage (OC4). The baseline to endline results are shown in Figure 4, and indicate a significant increase in the percentage of farmers using improved crop storage practices over the project lifetime. Figure 4: Comparison of Agricultural Practices Adopted *** p<0.01, ** p<0.05, * p<0.1 The first indicator, OC1, is the percentage of farmers who used at least four sustainable agriculture practices and/or technologies, which includes crop and livestock practices as well as natural resource management (NRM) practices25. There was no change between endline and baseline, with around 75% of farmers adopting such practices (Figure 4). However, ViM beneficiary farmers used significantly more practices at endline (82.3%) compared to non-beneficiaries (68.4%). Men also used significantly more practices (83.9%) than women (64.8%) at endline, but men used less practices at endline than baseline while the percentage of women who did so remained constant across both survey rounds. 24 Note that ViM considered the endline yield values reported for sesame and peanuts to be low, as the values were 48 and 76 percent lower, respectively, than 2017 yield data for Sanmatenga province collected by the Ministry of Agriculture’s National Agricultural Survey. See report discussion on inherent unreliability of farmer self￾reported yields. 25 See Annex II for a full list of all practices and technologies considered. 76% 24% 36% 34% 48% 75% 22% 22% 33% 74% 0% 10% 20% 30% 40% 50% 60% 70% 80% OC1: 4+ sustainable ag practices OC2: Sorghum intercropping OC2: Cowpea intercropping OC3: Sorghum thinning OC4: cowpea storage Percent of Farmers Baseline Endline *** *** 24 FINAL REPORT – VICTORY AGAINST MALNUTRITION (ViM) FINAL EVALUATION The total number of sustainable agricultural practices and/or technologies used increased from 5.9 at baseline to 6.2 at endline. This change was primarily driven by an increase in adoption of different practices relating to crops and NRM, while the total number of livestock practices remained constant across baseline and endline. These increases are not reflected in the percentage of farmers using four or more practices (OC1), because most of the increases are accounted for by famers shifting from a lower number of practices to a higher number of practices, but not necessarily from below four practices to above four practices. Qualitative data also demonstrated the use of improved agricultural practices, noting that the qualitative results are not subject to the minimum threshold used in quantitative data. During GDs, participants broadly discussed the use of improved agricultural practices, challenges and successes in adoption of improved practices, and the effects of improved practices on their agricultural outcomes. GD participants in six groups made specific references to production gains they felt they had gained through implementing the zai technique and the use of chemical fertilizers. There were also some references to agricultural benefits from the use of the stone bunds and the half-moon technique. In addition, KIIs with cowpea union leaders indicated that beneficiaries found trainings on improved agricultural practices to be useful, and would continue to seek support due to the improved agricultural outcomes they experienced. The second indicator considered is the use of intercropping of targeted value chain crops sorghum, cowpeas, onions, and tomatoes (OC2). Similar to yields, intercropping for onion and tomatoes could not be calculated due to sample size limitations. The use of intercropping on sorghum remained constant between baseline and endline (Figure 4), ViM beneficiaries and non-beneficiaries, and men and women, at around 22% of farmers. The use of intercropping on cowpeas decreased between baseline and endline from 36.2% to 21.5% of farmers (Figure 4), and 19.3% of ViM beneficiary farmers used intercropping, compared to 23.4% of non-beneficiary farmers. Intercropping is typically used to mitigate risk for farmers, however the use of improved drought tolerant short cycle crop varieties has reduced the need for producers to engage in intercropping as a risk reduction strategy. In addition, cowpeas have grown in importance as a cash crop over the years, which has led to farmers adopting this crop as monoculture more commonly than in the past. Thus, this result for OC2 may reflect a greater number of farmers in the ViM intervention area adopting cowpea cash cropping as a strategy. This result is also supported by a substantial increase in the percentage of farmers who sold cowpeas at endline, which adds further evidence of a shift towards cowpea cash cropping at endline. At baseline, only 39.7% of cowpea growers sold cowpeas, compared to 76.0% at endline. Qualitative data also support this finding, as GD participants in three GDs highlighted cowpea production and harvesting as as the most beneficial activity they engaged in through ViM. These participants noted that the sale of cowpeas has been profitable, and beneficial to their families. The third indicator is the percentage of farmers using thinning (OC3) on sorghum. Thinning is a risk mitigation strategy where extra seeds are planted, and if many seedlings survive farmers reduce the number to reduce the competition for nutrients and thereby increase the development of the remaining seedlings. The percentage of farmers who used thinning on sorghum remained constant at around 33% between baseline and endline (Figure 4). However, ViM beneficiaries used thinning on sorghum significantly more than non-beneficiaries, at 37.8% and 29.4% respectively. There was no significant difference between men and women at baseline or endline in terms of use of thinning on sorghum. The evaluation team notes that thinning as a crop management practice is not necessary unless the density of the crops is too high. To help farmers save seeds and reduce labor used by farmers’ thinning, the ViM project trained farmers on proper crop density per hectare, which reduced the need for farmers to thin their crops. Thus, the lack of increase in sorghum thinning is not viewed as a negative trend since some 25 FINAL REPORT – VICTORY AGAINST MALNUTRITION (ViM) FINAL EVALUATION farmers may be using the correct initial number of seeds required at planting. For this indicator, there were limited data in the qualitative findings. The fourth indicator is the percentage of farmers using an improved storage technique for cowpeas, which is a triple deepening bag (OC4). There is a significant improvement achieved in the ViM project areas, as the percentage of farmers who used improved storage increased from 48.1% at baseline to 74.3% at endline (Figure 4). Similarly, a much larger percentage of ViM beneficiaries were using such practices at endline (84.3%), as compared to non-beneficiaries (66.2%). At endline males used improved storage (77.9%) more than women (70.0%), and use of improved storage increased within both genders between baseline and endline. Qualitative data from GDs and KIIs with cowpea union leaders demonstrated that improved storage for cowpeas was important to beneficiaries. These data reveal that improved crop storage was important for quality preservation, and for facilitating marketing and sales. IR 1.2: Smallholder access to agricultural inputs improved A key intermediate indicator for improved and diversified farmers’ agricultural products is whether the quality of the agricultural inputs used by smallholder farmers has improved. The indicator for IR 1.2 is the percentage of farmers using improved agricultural inputs (OC6), which is defined as using one of the practices including chemical fertilization, organic fertilization, improved seeds or seed treatment on any crop. The PBS and t-test results for a change in that the use of improved agricultural inputs between baseline and endline are not reported (see section 3.3.4). The endline value was 70.7% percent of farmers. At endline, the use of improved agricultural inputs was higher among ViM beneficiaries (77.6%) compared to non-beneficiaries (65.8%). The percentage of males using improved agricultural inputs at endline was 78.1%, compared to 63.1% of females. Among various categories of inputs, the percentage of famers using chemical fertilization increased from 28% to 37% and using improved seeds increased from 4% to 23%, while farmers using organic fertilization decreased from 64% to 58%. IR 1.3: Improved livestock practices adopted A second key intermediate indicator for improved and diversified agricultural products focuses on whether livestock practices improved. The indicators for IR 1.3 are the percentage of farmers using improved livestock practices (OC7) and the percent of farmers accessing government or private sector veterinary care and vaccinations (OC8). For OC7, use of improved livestock practices is defined as using at least veterinary care, complementary feeding, and habitat construction at the same time. The PBS data indicates that a smaller percentage of smallholders adopted improved livestock practices at endline as compared to the baseline (from 10.2% to 3.5%) (Figure 5), and use of improved livestock practices was higher among the beneficiaries as compared to the non-beneficiaries (at 8.8% and 2.5%, respectively). The main decreases came from a decline in the use of veterinary care (60% at baseline to 32% at endline) and habitat construction (17% at baseline to 9% at endline), while complementary feeding did not change. Men used improved veterinary care (4.8%) more than women (1.5%) at endline, but there was no change between men and women between baseline and endline. This result may be explained by a vaccine shortage experienced in the project area in recent years. OC8 focuses on the percent of farmers accessing government or private sector veterinary care and vaccinations, instead of less formal care that is often lower quality. There was no change in this indicator between baseline and endline, with values remaining constant at about 36% (Figure 5). However, ViM beneficiaries at endline used government or private sector veterinary care and vaccinations more than 26 FINAL REPORT – VICTORY AGAINST MALNUTRITION (ViM) FINAL EVALUATION non-beneficiaries, at 50.3% and 32.8% respectively. Additionally, men used such services from the government or private sector (42.1%) more frequently than women (25.7%), although there was no difference within gender between baseline and endline. This lack of significant difference can be explained by the vaccine shortage experienced in recent years on the one hand and on other hand by the low level of extension services coverage (maximum of one extension agent/commune of more than 55 villages). Strategic Objective 2: Increased household incomes ViM’s SO2 focused on increasing household income. Three indicators are used to assess household income: Household Dietary Diversity Score (IM3), prevalence of moderate and severe hunger (IM4), and the value of a set of assets (OC9). For IM3, the Household Dietary Diversity Score (HDDS) was used, which is a measure of the number of food groups eaten by members of the household in the last 24 hours, with a maximum possible score of 1226. HDDS is used as a proxy for a household’s socioeconomic status. The PBS results indicate there was no improvement in HDDS over the ViM project lifetime, with the average household consuming approximately four food groups at baseline and at endline (Figure 6). However, ViM beneficiaries were better off than non-beneficiaries at endline. ViM beneficiaries’ average HDDS was 4 at endline, compared to non-beneficiaries, who on average consumed fewer food groups (3.7). Overall, there was a decrease in the percentage of households who consumed fruits and vegetables, which was somewhat compensated for by increases in the proportion of households who consumed pulses/legumes/nuts, oil/fats, and sugar/honey food groups (Figure 6). The change in the composition of food groups eaten may be related to differences in the seasonal timing of the survey between baseline in 2012 (July￾August) and endline in 2017 (December). Figure 5: Comparison of Household Food Consumption by Individual Food Groups The prevalence of moderate and severe hunger in the household (IM4) is calculated on the basis of how often there was no food in the house, members went to bed hungry, or went all day and night without eating. There was a dramatic decrease in the incidence of hunger in the project area over the ViM project lifetime, falling from 43.2% of households at baseline to 3.6% of households at endline (Figure 6). There was no difference at endline between ViM beneficiaries and non-beneficiaries. The difference in 26 The list of food groups can be found in Annex II. 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Percent oh Households Baseline Endline 27 FINAL REPORT – VICTORY AGAINST MALNUTRITION (ViM) FINAL EVALUATION seasonal timing of the baseline and endline survey could contribute to this dramatic difference, because baseline was conducted during the lean season (July-August) and endline was conducted after the harvest in December. To help mitigate the seasonal difference in timing of data collection, the endline survey also asked a recall question about the household’s hunger level in July (IM4_1), to more closely follow the baseline reference period. Prevalence of hunger based on the July recall question at endline was somewhat higher, at 11.2%, but still much lower than the baseline prevalence and did not differ by beneficiary status. This reduction in the prevalence of hunger over the project lifetime is especially significant given that endline occurs within a drought context in 2017. Figure 6: Comparison of Prevalence of Moderate to Severe Hunger *** p<0.01, ** p<0.05, * p<0.1 The final income indicator under SO2, OC9, focused on the change in average value of all domestic, transportation, and productive assets (not including livestock) over the project lifetime27. Assets increased to 229,000 CFA at endline from 170,000 CFA28 at baseline, an increase of 35%. To ensure that the values were not driven by outliers, the median values and the values Winsorized at the 90th percentile were compared. The median increased from 95,000 CFA at baseline to 145,000 CFA at endline (50% increase), and the Winsorized value increased from 150,000 CFA to 205,000 CFA (37% increase). The large increases observed between the baseline and endline values, even after controlling for outliers, suggests that the values truly were higher at endline. The increase in the value of transportation assets (Winsorized values) between baseline and endline was the main driver of the increase, while the Windsorized values of the domestic and productive asset values did not significantly change. The endline version of the indicator (OC9_1) included additional assets29, as well as the value of the livestock. The value for all assets at endline was 450,000 CFA, and ViM beneficiary households had higher asset values (590,000 CFA) than non-beneficiary households (415,000 CFA). IR 2.1: Market linkages improved One of ViM’s primary strategies to increase household income was through improving market linkages, so that producers and buyers could form strong relationships and increase the ease of sales, sales, and 27 See Annex II for list of all assets considered. 28 Baseline asset values were adjusted up for inflation. 29 See Annex II for additional assets added at endline. 43% 4% 11% 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% IM4: moderate to severe hunger Percent of Households Baseline Endline Endline July *** 28 FINAL REPORT – VICTORY AGAINST MALNUTRITION (ViM) FINAL EVALUATION sustainable linkages. The four indicators for improved market linkages (IR 2.1) are: gross margin for crops (OC9_2), value of purchases from smallholders (OC10), smallholders reporting more than three sources of revenue (OC11), and farmers who practiced value chain activities (OC12). The first indicator, OC9_2, is the gross margin sales of the targeted value chain crops - sorghum, cowpeas, onions, and tomatoes. Onions and tomatoes were again not included in this assessment due to insufficient sample size (see evaluation limitations in section 3.3). Gross margin is a measure of the profitability of crops, and is calculated by taking the difference between the sales income for a crop and the amount spent on inputs for those crops. This indicator could not be calculated at baseline, however the endline value for sorghum was 25,500 CFA and 32,000 CFA for cowpeas. ViM beneficiaries did not have higher gross margins for sorghum relative to non-beneficiaries, but did have higher gross margins for cowpeas (39,000 CFA) compared to non-beneficiaries (25,500 CFA). The second indicator, OC10, is the value of purchases from smallholders of targeted commodities as a result of USG assistance, which is the total value of all crops sold by farmers who grew crops. PBS results indicate there was an increase from 30,000 CFA at baseline to 58,500 CFA at endline, an increase of 49%. The increase is primarily driven by a shift at endline toward more cash cropping, with a larger proportion of farmers who grew a given crop selling some of that crop at endline relative to baseline. There were sizable increases from baseline to endline in the percent of farmers who sold a given crop, such as for cowpeas (40% at baseline compared to 76% at endline), onions (60% at baseline compared to 96% at endline), sesame (41% at baseline compared to 80% at endline), and peanuts (44% at baseline compared to 80% at endline). The increase in the percentage of farmers selling crops could be tied to higher crop production, which can be supported by cowpea yields and the percentage of farmers selling cowpeas both significantly increasing. The amount of the crop sold per farmer selling the crop did not increase, which adds additional support that this shift is primarily driven by more farmers engaged in selling some proportion of their crop. Beneficiary farmers had a higher value of purchases from smallholders of targeted commodities at 75,700 CFA compared to 43,900 CFA for non￾beneficiaries. The third indicator, OC11, captures diversification of income by reporting the percentage of rural smallholders who reported more than three sources of revenue30. The PBS data indicates that most of the smallholders are entirely dependent on agriculture as the sole source of revenue, as less than 2% had income from non-farm sources. This indicator remained constant over the project lifetime, at about 1% for baseline and endline (Figure 7). The percentage of ViM beneficiaries using at least three sources of revenue was higher than non-beneficiaries, at 5.6% and 1.4% respectively, and there were no changes between men and women. The total number of sources of income remained constant between baseline and endline at around 1.7, and there were decreases in livestock sales and gold mining. The fourth indicator, OC12, is the percentage of farmers who practiced specified value chain activities31 on crops (not including livestock) promoted by ViM. There was a significant improvement in practicing value chain activities within the project areas, as signified by a dramatic increase from 37.4% to 70.3% of farmers between baseline and endline (Figure 7). At endline, livestock value chain activities were included in this indicator (OC12_1), and the percentage of farmers using crop or livestock value chains was 75.3%32. ViM beneficiaries used value chain activities on crops or livestock more than non￾beneficiaries (85.6% and 67.4% respectively). Male farmers practiced value chain activities (77.2%) more 30 The full list of different sources of revenue can be found in Annex II. 31 A full list of value chain activities can be found in Annex II. 32 This measure could not be calculated at baseline because there were no questions on value chain activities for livestock at baseline. 29 FINAL REPORT – VICTORY AGAINST MALNUTRITION (ViM) FINAL EVALUATION frequently than female farmers (73.3%) and the practice of value chain activities was more common among both men and women at endline compared to baseline. IR 2.2: Access to business development services improved Figure 7: Comparison of Business Development Services *** p<0.01, ** p<0.05, * p<0.1 Small businesses can benefit from access to business development services such as financial services and market information. The two indicators for IR 2.2 are the percentage of farmers who used financial services (OC13) and the percentage of farmers who used up-to-date pricing information (OC14). The first indicator, OC13, is the percentage of farmers who used financial services, which includes agricultural credit and savings from formal and informal institutions. More farmers were using formal financial services at endline (35.5%) compared to baseline (28.6%) (Figure 8). The percentage of farmers using agricultural credit decreased from 8% at baseline to 5% at endline, however a greater percentage of smallholders saved at endline (33%) compared to the baseline (23%). At endline, with crop warrantage also included (OC13_1), the percentage of farmers using financial services was 39.0%33. When crop warrantage is taken into account, a greater proportion of ViM beneficiaries used financial services (66.2%) compared to non-beneficiaries (36.5%). Men and women were not significantly different in their use of financial services, at approximately 39% for each. The second indicator, OC14, is the percentage of farmers who reported having access up-to-date pricing information34. There was a dramatic increase in this indicator, from 50.2% at baseline to 89.0% at endline (Figure 7), which clearly indicates that farmers in the project area are much better informed about market prices than at baseline. For each of the five sources of pricing information considered for this indicator, there was also a significant increase in the percent of farmers who reported accessing this source between baseline and endline. The overall percentage of farmers who obtained up-to-date pricing information from cell, radio, or government sources, rather than from other producers or collectors, is still relatively low at endline, but the increase within each information source category adds 33 This measure could not be calculated at baseline because there was no question on crop warrantage at baseline. 34 See Annex II for a full list of sources of up-to-date pricing information. 1% 37% 29% 50% 2% 70% 36% 89% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% OC11: 3+ revenue sources OC12: value chain activities OC13: formal financial services OC14: up-to-date pricing info Percent of Farmers Baseline Endline *** *** * *** 30 FINAL REPORT – VICTORY AGAINST MALNUTRITION (ViM) FINAL EVALUATION further support to the overall trend of increased information access. At endline, ViM beneficiaries had better access to up to date pricing information, with 95.5% reporting access to up-to-date pricing information compared to 88.2% of non-beneficiaries. A higher percentage of female farmers (90.9%) reported access to up-to-date pricing information than male farmers (87.1%) at endline. A larger percentage of both men and women had access to up-to-date pricing information at endline compared to baseline, 52.8% and 46.9% respectively at baseline. Strategic Objective 3: Reduced chronic malnutrition among children under five years of age and pregnant and lactating mothers (PLW) The goal of the ViM project is to reduce food insecurity among vulnerable populations. ViM’s SO3 focused on decreasing chronic malnutrition among children under five years of age and PLW. The different types of interventions implemented by the ViM project are expected to reduce malnutrition in general, and in particular, chronic malnutrition among children under five years of age. The primary indicators were used to assess change in malnutrition: prevalence of stunted children (IM5) under five years of age, prevalence of underweight child (IM6) under five years of age, and PLW’s dietary diversity (OC15). Height for age (stunting) is an indicator of chronic (long-term) malnutrition arising from deprivation usually related to a persistently poor food security situation, micronutrient deficiencies, recurrent illnesses and other factors which interrupt normal growth. Unlike wasting, it is not affected by seasonality but it is related to the long-term effects of socio-economic development and a long-standing food insecurity situation. A low height-for-age reflects deficits in linear growth and is referred to as stunting. Figure 8: Comparison of Stunted and Underweight Children *** p<0.01, ** p<0.05, * p<0.1 31% 26% 25% 20% 0% 5% 10% 15% 20% 25% 30% 35% IM5: Stunted IM6: Underweight Percent of Chidlren Baseline Endline *** *** 31 FINAL REPORT – VICTORY AGAINST MALNUTRITION (ViM) FINAL EVALUATION The PBS results show that there was a significant decrease in the prevalence of stunted children (IM5) in the ViM project areas – the stunting rate decreased from 31.2% at baseline to 25.5% at endline35 (Figure 8), which indicates a large drop in malnutrition of children in that target age group. Reducing chronic malnutrition among child populations typically requires a comprehensive approach to improve food security and social behavior change communication (SBCC), such as increasing food production, food accessibility, food use and food stability in vulnerable households. Health and nutrition related practices such as optimal breastfeeding practices, complementary feeding practices, child caring practices and health environment are considered to be among some of the most important factors to decrease the prevalence of stunting among children under five years of age. However, there was no difference in the stunting rate between ViM beneficiaries and non-beneficiaries at endline. This finding likely relates to the broad coverage of some types of food rations across households in the ViM project area, such that even if an individual child did not receive a supplemental food ration directly, according to the caregiver’s response on the household survey at endline, often they were still in a household that received a food ration at some point, such as directly to the woman. In addition, the stunting rate at endline among male children (28.3%) was higher than for female children (22.9%) (Figure 9). However, there was a reduction in stunting rate between the baseline and endline for both genders (from 33.0% to 28.3% for boys and from 29.4 to 22.9% for girls). The second indicator for SO3 is the prevalence of underweight children under five years of age (IM6), as measured by weight-for-age (WFA). The WFA index provides a composite measure of wasting and stunting and is commonly used to monitor the growth of individual children using child health cards, since it enables mothers to easily visualize the trend of a child’s weight against age. There was a significant decrease in the prevalence of underweight children as well between the baseline and endline (from 24.7% at baseline to 19.7% at endline) (Figure 8). Thus, both of these impact-level indicators show that there was a significant decrease in malnutrition for children under the age of five in the ViM intervention area (Figure 8), over the ViM project lifetime. Similar to the stunting rate, the prevalence of underweight children was higher among male children than female children at endline (Figure 9), 21.9% of boys compared to 17.5% of girls). There was no significant decrease in underweight boys between baseline and endline, but the percentage of underweight girls decreased from 24.0% at baseline (to 17.5%, as noted). 35For additional context, the 2016 National Nutrition Survey reported a national stunting rate of 27.3% in 2016, down from 35.1% in 2009. The 2016 region-wide rate for Centre-Nord Region (in which the ViM project area of Sanmatenga Province is located) was 25.5 % (95% confidence interval: 21.4 – 30.0%) (GoBF, Ministere de la Sante, 2016. Rapport de l’Enquete Nutritionelle National SMART 2016). In terms of stunting trends in Sanmatenga Province during the project lifetime, the National SMART survey reported a stunting prevalence of 33.3% in 2012 and 23.3% in 2017. 32 FINAL REPORT – VICTORY AGAINST MALNUTRITION (ViM) FINAL EVALUATION Figure 9: Stunting and Underweight Differences across Gender *** p<0.01, ** p<0.05, * p<0.1 The third SO3 indicator assessed is Women’s Dietary Diversity Score (WDDS), OC15, which gathers information on how many food groups women of reproductive age consumed in the 24 hours before the survey, from among nine nutrient-rich food groups36. Reduction of PLW malnutrition is considered to be an important measure for reducing malnutrition among their children. The PBS data indicates that of the nine nutrient-rich food groups included, on average, WDDS increased marginally from 2.9 food group at baseline to 3.3 at endline. While there was some improvement in the dietary scores, the extent of change over the project lifetime was marginal, which may result from an interplay of several factors such as climate influenced changes in harvests and consequent food shortages in households. ViM beneficiary women had a slightly higher WDDS (3.3) than non-beneficiary women (3.2), although this difference is not significant. IR 3.1: Improved MCHN practices Poor woman and child health and feeding practices can contribute to malnutrition since they may lead to a situation where children do not receive enough calories or macronutrients. The three indicators for IR 3.1 are exclusive breastfeeding (OC16), children breastfed within one hour of birth (OC17), and children receiving a minimally acceptable diet (OC18). Overall, the evaluation results suggest a robust improvement in MCHN practices during the ViM project lifetime (Figure 11). 36 See Annex II for full list of food groups. 28% 23% 22% 18% 0% 5% 10% 15% 20% 25% 30% Stunted Underweight Percent of Children Boys Girls *** *** 33 FINAL REPORT – VICTORY AGAINST MALNUTRITION (ViM) FINAL EVALUATION Figure 10: Comparison of Exclusively Breastfed Components The first indicator under IR 3.1 is the prevalence of exclusive breastfeeding of children who are under six months of age (OC16). Breast milk is the optimal source of nutrients for infants. Children who are exclusively breastfed receive only breast milk. Exclusive breastfeeding is recommended during the first six months of a child’s life because it limits exposure to disease agents and provides all of the nutrients that a baby requires. Exclusive breastfeeding (EBF) for six months has been shown to be the most effective preventive intervention for ensuring child survival, and is estimated to prevent 13 percent of deaths of children younger than five years. Appropriate complementary feeding has also been estimated to prevent an additional six percent of all deaths in this age group (Jones et al., 2003). Information on infant and young child feeding practices was obtained based on a 24-hour recall, in line with the WHO guidelines to minimize recall bias and thus obtain more valid information. There was a significant increase in exclusively breastfed children over the project lifetime, from 36.8% at baseline to 60.3% at endline (Figure 11). The main driver of the increase in exclusively breastfed children was in the decrease in percentage of children who consumed water (Figure 10). There was no difference in the prevalence of exclusive breastfeeding between ViM beneficiaries and non-beneficiaries, which may be attributed to the small sample size of children under the age of six months37, or speak to effective spillover of project messaging around the importance of EBF. This increase in EBF is explained by the sensitization to mothers on the IYCF optimal practices including exclusive breastfeeding during the first six months of the life of the infant. During the ViM project intervention, the quantitative and qualitative data suggest that the majority of mothers adopted best practices for child feeding, including exclusive breastfeeding during the first six months of age for infants. 37 There were 155 children under six months of age at baseline and 325 at endline. 0% 20% 40% 60% 80% 100% Percent of Children Baseline Endline 34 FINAL REPORT – VICTORY AGAINST MALNUTRITION (ViM) FINAL EVALUATION Figure 11: Comparison of Women and Child Health and Nutrition Practices *** p<0.01, ** p<0.05, * p<0.1 The second indicator under IR 3.1 is the prevalence of children under two years of age who were put to the breast within one hour of birth, OC17. Global recommendations on infant feeding require that breastfeeding commences immediately after birth, or within the first hour after delivery, and that no pre-lacteal feds are given to newborn infants. This first milk is rich in colostrum and anti-infective properties, hence nourishing and protecting new born infants from common infections. There was a dramatic increase in the percentage of children who were put to the breast during the ViM project lifetime, from 39.5% at baseline to 69.5% at endline (Figure 11). However, there was no significant difference in this indicator between ViM beneficiaries and non-beneficiaries. The third indicator, OC18, is the prevalence of children 6-23 months receiving a minimum acceptable diet (MAD)38. This includes continued breastfeeding, frequency of feeding and dietary diversity. Continued breastfeeding up to the age of two years is recommended for continued optimal growth and development of young children. Typically, the onset of malnutrition in infants and young children coincides with the initiation of complementary feeding from the age of six months, and peaks at 18-24 months. The prevalence of children receiving a MAD increased from 4.9% at baseline to 8.9% at endline (Figure 11), and there was once again no significant difference between ViM beneficiaries and non￾beneficiaries. At endline, the reasons for lack of MAD achievement varied by child’s age. For children 6-8 months of age at endline, 26.9% received the minimum meal frequency while only 4.9% of them received the minimum dietary diversity. For children 9-23 months of age at endline, 31.1% received the minimum meal frequency and the diversity of foods eaten was also more diversity, with 31.4% of them receiving the minimum dietary diversity. IR 3.2 Improved Quality of Health Services In November 2016, The World Health Organization (WHO) issued a new series of recommendations to improve quality of antenatal care to reduce the risk of stillbirths and pregnancy complications and give women a positive pregnancy experience. By focusing on a positive pregnancy experience, these new guidelines seek to ensure not only a healthy pregnancy for mother and baby, but also an effective 38 See Annex II for the requirements for a MAD. 37% 40% 5% 60% 70% 9% 0% 10% 20% 30% 40% 50% 60% 70% 80% OC16: exclusively breastfed OC17: breastfed within one hour OC18: MAD Percent of Children Baseline Endline *** *** ** 35 FINAL REPORT – VICTORY AGAINST MALNUTRITION (ViM) FINAL EVALUATION transition to positive labor and childbirth and ultimately to a positive experience of motherhood. There are seven indicators for IR 3.2: antenatal visits (OC19), postnatal health check (OC20), vitamin A supplements for children (OC21), iron and folic acid supplementation (OC22), vitamin A supplements for mothers (OC23), fully immunized children (OC24), and growth monitoring weigh-ins (OC25). Overall, the final evaluation results indicate substantial, positive changes in the quality of antenatal care and health services for children and mothers in ViM intervention areas during the project lifetime (Figure 12). Figure 12: Comparison of Quality of Health Services *** p<0.01, ** p<0.05, * p<0.1 The first indicator under IR 3.2 is the percentage of mothers with children aged 0-23 months who had four or more antenatal visits when they were pregnant with their youngest child (OC19). Recent evidence indicates that a higher frequency of antenatal contacts by women and adolescent girls with a health provider is associated with a reduced likelihood of stillbirths. This improvement is because of the increased opportunities to detect and manage potential complications. There was a significant increase in the percentage of mothers attending four or more antenatal visits (Figure 12), from 42.3% at baseline to 62.5% at endline, however there was no difference between ViM beneficiaries and non-beneficiaries39. The main contributing factor mentioned by respondents during the qualitative data collection, and which was noted to limit achievement of the recommended number of visits, was the long distance between households and health facilities. The second indicator, OC20, is the percentage of newborns receiving postnatal health check within two days of birth. The postnatal period is a critical phase in the lives of mothers and newborn babies. Most maternal and infant deaths occur during this time. Yet, this is also the most neglected period for the provision of quality care. Recommendations focus on skilled attendants working at the primary level of 39 Comparisons between beneficiaries and non-beneficiaries for the health services indicators are informative for understanding whether direct participation in care groups is associated with different outcomes at endline. However, it is also noted that several community-wide activities conducted by ViM, such as growth monitoring and vaccination campaigns, also contribute to observed health services outcomes regardless of care group participation. 42% 65% 54% 35% 59% 63% 73% 79% 64% 51% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% OC19: 4+ antenatal visits OC21: child vitamin A supplement OC23: mother vitamin A supplement OC24: fully immunized OC25: growth monitoring card Percent of PLW or Children Baseline Endline *** *** ** *** *** 36 FINAL REPORT – VICTORY AGAINST MALNUTRITION (ViM) FINAL EVALUATION health care, either at the facility or in the community. They apply to all women attending antenatal care, in delivery, postpartum or post abortion care, or who come for emergency care, and to all newborns at birth for routine and emergency care. This is a new indicator that could not be calculated at baseline, but at endline 51.9% of newborns received a postnatal check within two days of birth. There was no difference in this indicator between ViM beneficiaries and non-beneficiaries. The third indicator, OC21, is percentage of children 6-23 months who received vitamin A supplements within the past six months. Vitamin A is essential for the functioning of the immune system and the healthy growth and development of children; children with vitamin A deficiency suffer an increased risk of visual impairment (night blindness), illness and death from childhood infections such as measles and those causing diarrhea40. The Burkina Faso Ministry of Health policy states that children under the age of five years should receive Vitamin A supplementation every six months. There was a significant increase in the percentage of children who received vitamin A supplementation in the ViM intervention areas, from 64.6% at baseline to 72.9% at endline (Figure 12). However, there was no difference in this indicator between ViM beneficiaries and non-beneficiaries. The fourth indicator, OC22, is the percentage of PLW who received iron and folic acid supplementation. WHO estimated in January 2017 that more than 40% of pregnant women worldwide are anemic; at least half of this anemia burden is assumed to be due to iron deficiency. Pregnant women require additional iron and folic acid to meet their own nutritional needs as well as those of the developing fetus. Deficiencies in iron and folic acid during pregnancy can potentially negatively impact the health of the mother, her pregnancy, as well as fetal development. Evidence has shown that the use of iron and folic acid supplements is associated with a reduced risk of iron deficiency and anemia in pregnant women. Because of this new recommendation from WHO, this indicator on daily iron and folic acid supplementation during pregnancy was included in the evaluation at endline. This indicator was not calculated at baseline, but it is disaggregated to the last pregnancy (93.6% at endline) and the last week (48.1% at endline). There was a significant increase for the last pregnancy between beneficiaries (97.0%) and non-beneficiaries (90.6%), and slightly higher for beneficiaries than non-beneficiaries for the last week, 51.8% and 44.7% respectively. The fifth indicator, OC23, is the percentage of women who received vitamin A supplementation post￾partum. According to WHO, January 2017, Vitamin A is important for visual health, immune function and fetal growth and development. Vitamin A deficiency is a public health problem in many parts of the world, particularly Africa and South-East Asia. It can cause visual impairment in the form of night blindness and, in children, may increase the risk of illness and death from childhood infections, including measles and those causing diarrhea. Vitamin A supplementation in postpartum women might be expected to improve maternal vitamin A status, thereby increasing the vitamin A content of breast milk and improving the health of mother and infant. There was a significant increase in women receiving vitamin A supplements post-partum from 54.1% to 79.2% (Figure 12), and ViM beneficiary women (84.8%) were more likely to received vitamin A supplements than non-beneficiary women (74.1%). The sixth indicator, OC24, is the percentage of children 12-23 months of age children who are fully immunized. Immunization is a proven tool for controlling and eliminating life-threatening infectious diseases and is estimated to avert between two and three million deaths each year. It is one of the most cost-effective health investments, with proven strategies that make it accessible to even the most hard￾to-reach and vulnerable populations. It has clearly defined target groups; it can be delivered effectively through outreach activities; and vaccination does not require any major lifestyle change. There was a significant increase in fully immunized children from 34.5% at baseline to 50.5% at endline (Figure 12), 40 WHO. Global prevalence of vitamin A deficiency in populations at risk 1995-2005. WHO Global Database on Vitamin A Deficiency. Geneva, World Health Organization, 2009. 37 FINAL REPORT – VICTORY AGAINST MALNUTRITION (ViM) FINAL EVALUATION however beneficiary children had a lower immunization rate (36.6%) than non-beneficiary children (54.8%). The seventh indicator, OC25, is the percentage of children 0-23 months who have a growth monitoring card showing at least one weighing in the last two months. The Growth Monitoring (GM) is the process of following the growth rate of a child in comparison to a standard by periodic, frequent anthropometric measurements in order to assess growth adequacy and identify faltering early. There was a slight decrease from baseline (59.2%) to endline (49.8%) on this indicator (Figure 12), and also no difference between ViM beneficiaries and non-beneficiaries. During the GDs and KIIs conducted at endline, the reasons that respondents cited for the decrease on this indicator included several different factors. First, it was noted that the Ministry of Health (MoH) has not yet adopted the new health card for children, which highlights growth monitoring from 0-23 months of age. This has a negative impact on the growth monitoring program because mothers are obliged to end this process at the end of the immunization program. In addition, mothers are not aware of the importance of continuing the growth monitoring of their infants after the immunization program, and health providers appear not to adequately sensitize on this issue because there is no GoBF guideline on this issue. IR 3.3: Improved hygiene and sanitation practices Poor sanitation practices and hygiene can lead to malnutrition through the spread of diarrheal diseases. The four indicators for IR 3.3 are percentage of households with soap at a hand washing station (OC27 and OC27a), access to improved sanitation (OC28), and knowing the critical moments for hand washing (OC29). As for other indicators under IR3, the overall evaluation results suggest that there were substantial positive improvements on hygiene and sanitation practices within the ViM project areas during the project lifetime (Figure 13). Figure 13: Comparison of Adoption of Hygiene and Sanitation Practices *** p<0.01, ** p<0.05, * p<0.1 The first set of indicators under IR 3.3 includes the percentage of households with children aged 0-23 months that have water and soap or a locally available cleansing agent at a hand washing place (OC27). There was a dramatic increase from baseline to endline on this indicator, from 1.2% to 20.4% (Figure 13). Additionally, ViM beneficiary households were more likely to have water and soap or a locally 1% 1% 1% 2% 20% 17% 23% 42% 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% OC27: child under 2, soap and water OC27a: soap and water OC28: improved sanitation OC29: know critical handwashing Percent of Households Baseline Endline *** *** *** *** 38 FINAL REPORT – VICTORY AGAINST MALNUTRITION (ViM) FINAL EVALUATION available cleansing agent at a hand washing place (24.0%) than non-beneficiary households (14.8%). The second related indicator, OC27a, is the percentage of households with soap and water at a hand washing station commonly used by family members. There was a similar increase from 1.0% at baseline to 17.0% at endline as with the households with children aged 0-23 months. Beneficiary households were more likely (21.8%) than non-beneficiary households (10.8%) to have soap and water at a hand washing station commonly used by family members. The third indicator, OC28, is the percentage of households with access to an improved sanitation facility41. There was a dramatic increase in the percentage of households with access to an improved sanitation facility, from 0.6% at baseline to 23.5% at endline (Figure 13). Additionally, a greater percentage of ViM beneficiary households had improved sanitation facilities (27.6%) relative to non￾beneficiary households (18.2%). The fourth indicator, OC29, is the percentage of respondents who know all critical moments for hand washing to prevent diarrheal disease42. There was a large increase in the percentage who know all critical moments for hand washing, from 1.8% to 41.6% (Figure 13). Also, the percentage of ViM beneficiary households who knew the critical moments of hand washing (44.4%) was greater compared to that of non-beneficiaries (31.2%). Crosscutting: Gender Across ViM activities it was critical that each activity made considerations for existing gender gaps through gender integration and women's empowerment. To assess the progress in this realm, three indicators were used: women who make decisions by themselves or jointly with their spouse (OC33), women who agree it is justifiable for a man to hit his wife (OC34), and MLA who feel more respected (OC35). The evaluation results suggest significant improvements on each of these crosscutting indicators during the ViM project lifetime (Figure 14). Figure 14: Comparison of Female Empowerment *** p<0.01, ** p<0.05, * p<0.1 41 See Annex II for the definition of an improved sanitation facility. 42 See Annex II for the list of critical moments for hand washing. 62% 91% 72% 51% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% OC33: woman decision making OC34: justifiable to hit woman Percent of Women Baseline Endline *** *** 39 FINAL REPORT – VICTORY AGAINST MALNUTRITION (ViM) FINAL EVALUATION The first indicator, OC33, is the percentage of married women aged 15-49 who reported they made decisions by themselves, or jointly with their spouse. This indicator focused on the extent to which women were involved in decision making regarding seeking health services for their own health, seeking health services for their children's health, and how to spend money they themselves had earned. Although many ViM activities specifically focused on the unique health and economic needs of women, the level of agency women have in their own health, their children’s health, and productive resources has implications for ViM activities’ effectivenessand sustainability. There was a significant increase in the percentage of women who report making decisions alone or jointly with their spouses (from 62.4% at baseline to 72.4% at endline) (Figure 14). Beneficiary women reported making decisions alone or jointly with their spouse (75.5%) more often than non-beneficiary women (69.9%). These results indicate that ViM activities contributed to increasing the agency of women in their health, children’s health, and productive resources. The second indicator, OC33, is the percentage of women who agree that it is justifiable for a man to hit his wife in one or more circumstances43. Women’s attitudes about gendered social norms can reveal important information regarding their perceptions of the power dynamics that exist between men and women in their communities. Attitudes about domestic violence, in particular, may indicate that in a given community women are regarded as subordinates, and not equals. According to WHO44, women in communities where poverty is prevalent are at higher risk of intimate partner violence than others. Further, communities can reinforce individual beliefs about the acceptability of intimate partner violence and contribute to its prevalence. The percentage of women who agreed that it was acceptable for a man to hit his wife in one or more circumstances dropped drastically from baseline (91.0%) to endline (50.6%) (Figure 14). This reduction demonstrates that ViM activities were successful in changing attitudes about intimate partner violence and the role of women within the context of marriage. Activities centered on the promotion of women as leaders, the economic empowerment of women, and the sensitization of men are likely key contributors to this shift. However, ViM beneficiary women were more likely to agree it is acceptable for a man to hit a women (60.0%) than non-beneficiary women (44.6%). It is noted, however that this statistical result was not corroborated during qualitative data collection. The third indicator, OC35, is the percentage of Care Group members (also known as MLAs) who indicate they are more respected after participating in the program. MLA were extremely important to ViM activities in mobilizing community members, promoting improved MCHN practices, and sensitizing men and women. This is a new indicator that could not be calculated at baseline, and cannot be compared between beneficiaries and non-beneficiaries because all MLAs are by definition beneficiaries. At endline 91.4% of MLA members reported feeling more respected after participating in the program. Of the MLA members who reported feeling more respected, 81.0% reported feeling more respected by their husbands, 75.9% by women in the household, 56.7% by men in the household, 67.8% by women outside the household, and 56.3% by men outside the household. This result was also supported in the qualitative findings, where both men and women discussed the importance of MLAs in facilitating change within their communities, and men in particular referenced the role of MLAs in changing attitudes and behaviors among men45. 43 See Annex II for full list of instances. 44 WHO. Intimate partner violence. WHO Global Database on Understanding and addressing violence against women. Geneva, World Health Organization, 2012. 45 Please see the results for EQ8 for additional discussion on this issue. 40 FINAL REPORT – VICTORY AGAINST MALNUTRITION (ViM) FINAL EVALUATION Crosscutting: Environment Across ViM agricultural interventions, increasing and diversifying agricultural production in an environmentally sustainable way was a primary project goal. To assess the progress in this realm, three indicators were used: adoption of environmentally sustainable technologies (OC36), use of environmental mitigation principles (OC37), and knowledge of pest management practices (OC38). The evaluation results suggest positive improvements in two of these indicators (OC37 and OC38), while the OC36 measurement at baseline was considered to be erroneously high and this may explain the lack of change in the baseline to endline analysis46 (Figure 15). Figure 15: Comparison of Environmentally Sustainable Practices *** p<0.01, ** p<0.05, * p<0.1 The first indicator, OC36, is the percentage of farmers adopting at least three environmentally sustainable technologies, which includes the same crop, livestock, and NRM practices47 from OC1. The PBS based results indicate a high percentage of farmers who adopted at least three environmentally sustainable technologies at endline, with approximately 84% of the farmers adopting at least three such techniques (Figure 15). A greater percentage of farmers are using improved seeds, chemical fertilizers, adopting techniques such as ploughing, chemical fertilizers, crop rotation, zai techniques while there is not much improvement in thinning or seed treatment. Weeding, hoeing, intercropping and organic fertilizer were less frequently practiced in the endline among the farmers, which may be explained by the greater adoption of several of the improved farming practices listed above. Beneficiaries were more likely to use at least three environmentally sustainable technologies (89.9%) than non-beneficiaries (80.0%), suggesting greater adoption among those who more directly participated in project agricultural activities. Additionally, men (90.4%) were more likely than women (78.2%) to use at least three environmentally sustainable technologies, but men were less likely than at baseline (95.0%). The second indicator, OC37, is the percentage of rural smallholders reporting use of at least two environmental mitigation principles in diversified income generating activities. Environmental mitigation principles are important in protecting the farmers’ crops from a negative environment shock that could 46 The baseline value was considered erroneously high and was not included in ViM’s official baseline statistics. 47 See Annex II for full list of practices. 85% 53% 32% 84% 66% 52% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% OC36: 3+ sustainable ag practices OC37: 2+ envirnmental mitigation principles OC38: 2+ pest management practices Percent of Farmers Baseline Endline *** *** 41 FINAL REPORT – VICTORY AGAINST MALNUTRITION (ViM) FINAL EVALUATION greatly impact their food security. This indicator was defined to only include NRM methods that are considered to embody environmental mitigation principles48. The number of farmers using at least two environmental mitigation principles increased from 53.0% at baseline to 66.2% at endline (Figure 15). Beneficiaries were more likely to use these methods (71.4%) than non-beneficiaries (61.5%). A higher percentage of men used environmental mitigation principles (71.7%) than women (58.4%) at endline, and both male and female farmers’ usage increased usage from baseline (62.2% and 39.5% at baseline respectively). The third indicator, OC38, is the percentage of smallholder farmers demonstrating knowledge of two sound pest management practices. Pest management practices are important for safeguarding farmers’ crops against a potential insect infestation that could threaten farmers’ livelihood. Note that this indicator calls for the number of surveyed farmers demonstrating knowledge of (at least) two “sound pest management practices.” The questions in the survey instrument only asked about which pest management methods were used by the surveyed farmers, which is somewhat different from what was specified in the indicator. Therefore, the results presented below only reflect the data collected against the questions that were actually asked rather than the exact questions specified in the indicator definition. Overall, the percentage of surveyed farmers using at least two pest management methods increased from 31.6% at baseline to 52.3% at endline indicating a significant increase in use of improved pest management practices among the farmer in the ViM project areas (Figure 15). Beneficiary farmers were more likely to use pest management practices (63.0%) than non-beneficiaries (44.0%), which may also help to explain higher yields at endline for beneficiary farmers relative to non-beneficiary farmers for some crops, such as sorghum. Male farmers used pest management practices (58.3%) more than female farmers (46.1%), and the use of pest management practices increased between baseline and endline for men and women (41.8% and 21.1% at baseline respectively). 4.2 EQ2: EFFECTIVENESS OF VIM IN PROMOTING IMPROVED AGRICULTURAL AND LIVESTOCK PRACTICES AND INCREASING AND DIVERSIFYING SMALLHOLDER AGRICULTURAL PRODUCTION. FINDINGS Approaches are defined as effective if they have reached their project objectives and achieved broad impact in improving the health, nutrition, hygiene, or food security outcomes of ViM project beneficiaries and/or non-beneficiaries. One of the major focuses of ViM is to support rural smallholders’ agriculture and livestock production and diversification. ViM also supports income-generating interventions to develop sustainable livelihood opportunities for rural poor. Towards that goal, one of the primary objectives of this evaluation is to assess how effective ViM-promoted interventions were in achieving the project defined strategic objective of increasing and diversifying smallholders’ agricultural production and increasing farmers’ income. In addition, this section aims to assess smallholders’ access, affordability and use of agriculture and livestock extension services, and of project-sponsored high quality agricultural and livestock inputs including veterinary services. To address these questions, we first present a descriptive statistical analysis to identify how outcomes differ between the beneficiary and non-beneficiary groups at endline, followed by a rigorous econometric analysis with an aim of providing more robust evidence of association. 48 See Annex II for the list of environmental mitigation principles. 42 FINAL REPORT – VICTORY AGAINST MALNUTRITION (ViM) FINAL EVALUATION 4.2.1 Descriptive statistical analysis To address the question of effectiveness, we first identify whether ViM beneficiaries were performing better than non-beneficiaries at endline with respect to key agricultural outcome indicators of interest. It must be noted that while the quantitative data collected at endline can illuminate some differences in the outcomes related to agriculture between ViM beneficiaries and non-beneficiaries, assessing the effectiveness of the ViM project in enabling and even causing these changes requires attribution of the differences in outcomes to the ViM project. Establishing such attribution is not feasible for this evaluation design, due to the absence of valid comparison group and experimental or quasi-experimental evaluation design. Thus, the evaluation team notes, that the observed differences discussed for these analyses are only indicative of correlations between project participation and outcomes. Beneficiary / Non-Beneficiary Comparisons This subsection compares the available endline data for ViM beneficiaries and non-beneficiaries and draws the conclusions that can be supported by such analyses. Table 5 presents the comparisons of the values of key indicators at endline. All of the presented data were collected at the individual farmer level rather than at the household level. The comparisons reported in Table 5 indicates that the ViM beneficiaries49 had achieved better diversification – both in terms of number of different types of products produced as well as sources of livelihood. Data reported in Table 5 indicate beneficiaries performed better with respect to most of the other agricultural indicators such as yield of crops, gross margin of crops, and ability to market crops. Good agricultural and livestock practices are important for increasing agricultural production and income. Good agricultural practices (GAP) are linked to more efficient ways to farm and obtain the most out of the land. Types of good agricultural practices include sustainable agricultural practices, natural resource management practices, thinning and intercropping50 farming techniques, and improved agricultural inputs. Good livestock practices are important for the health or animals used for production, consumption, and sale. Types of good livestock practices include veterinary care from qualified government and private sector sources, habitat construction, complementary feeding, and value chain activities51. 49 Farmer level interventions that determine beneficiary status include; received agricultural inputs (for crops or livestock), received trainings (on plants, animals, NRM, value chains, financial management, or farming as a business), received crop equipment, or received a plot of land. See Annex III for farmer level table on determining beneficiary status for each indicator. 50 Thinning and intercropping may not always be considered GAP, see discussion of OC2 and OC3 in section 4.1. 51 See Annex III indicator definition for OC12. 43 FINAL REPORT – VICTORY AGAINST MALNUTRITION (ViM) FINAL EVALUATION Table 5: Comparison of Endline Indicator Values for Beneficiaries and Non-beneficiaries Indicator Indicator Values Difference (BF – NBF) Significance52 Beneficiary Non￾Beneficiary Average number of different types of agricultural products produced per surveyed farmer 4.80 3.97 0.83 *** Percentage of rural smallholders reporting more than THREE sources of revenue 5.6% 1.4% 4.3% *** Average yield of [crop] per surveyed farmer (kg/ha) Cowpeas 519 415 104 ns Sorghum 576 461 115 *** Millet 477 564 -87 ns Maize 516 424 92 ** Gross margin sales of crops per farmer (FCFA)53 Cowpeas 39,253 25,589 13,664 *** Millet 12,335 12,250 85 ns Maize 11,573 14,000 -2,427 ns Value of purchases from smallholders of targeted commodities as a result of USG assistance (FCFA) 75,681 43,894 31,787 *** Agricultural and Livestock Practices Percentage of farmers who used at least FOUR sustainable agriculture (crop/livestock and/or NRM) practices and/or technologies in the most recent season 82.3% 68.4% 13.9% *** Percentage of farmers using intercropping Sorghum 21.6% 22.8% -1.2% ns Cowpeas 19.3% 23.4% -4.1% *** Percentage of farmers using thinning: sorghum 37.8% 29.4% 8.4% *** Percentage of farmers who used an improved storage technique for cowpeas in the most recent season 84.3% 66.2% 18.1% *** Percentage of farmers using improved agricultural inputs 77.6% 65.8% 11.8% *** Percentage of farmers using at least veterinary care, complementary feeding and habitat construction at the same time 8.8% 2.5% 6.4% *** Percentage of farmers accessing government or private sector veterinary care and vaccinations 50.3% 32.8% 17.5% *** Percentage of farmers who practiced the value chain activities promoted by the project in the most recent season (livestock included) 85.6% 67.4% 18.3% *** *** p<0.01, ** p<0.05, * p<0.1 52 The significance level is noted as not significant (ns), significant at the 10% level (*), significant at the 5% level (**), or significant at the 1% level (***). 53 Sorghum gross margin not reported due to small sample size (less than 10% of producers reporting sale of sorghum in the most recent year). 44 FINAL REPORT – VICTORY AGAINST MALNUTRITION (ViM) FINAL EVALUATION While the “naïve” comparison of indicators between the beneficiary and non-beneficiary farmers presented in Table 5 shows that for most of the agriculture-related indicators, the beneficiaries achieved better outcomes compared to the non-beneficiaries, it is important to note that these observed differences are not sufficient to make robust inference about the effectiveness of ViM in influencing these outcomes because these outcomes can also be influenced by other confounding factors. To help address this, in the next section, we provide a more rigorous econometric analyses to more robustly identify whether ViM interventions had statistically significant influence on the observed outcomes of interest, after controlling for other observable factors that can also influence these outcomes. 4.2.2 Econometric Analysis Making inferences about the effectiveness of ViM based on conclusions generated from the comparison of outcomes of interest between the beneficiary and non-beneficiaries typically produces biased inferences, as it suffers from a problem known as “self-selection”. The problem of self-selection arises due to the fact that the beneficiaries, who either decide themselves to participate in a program (such as GAP training), or are identified by the implementers as target beneficiaries, may be inherently different from those who either decide not to participate in a program made available to them, or are involuntarily “excluded” by program implementers. To generate more robust conclusions about the program’s effectiveness, we use statistical modelling to address potential selection issues, and more rigorous econometric analysis to identify the effectiveness of ViM. In the next subsection, we present the results from these econometric analyses aimed to provide more robust conclusions about the effectiveness of ViM in increasing and diversifying smallholders’ agricultural production and income. The econometric analyses utilizes a model-based approach to estimate how various outcome variables are influenced by ViM interventions, while correcting for the self-selection issue and potential influence of other covariates by using control variables. There were three steps to the analysis: determining the effect of ViM interventions on agricultural production and income, determining the effect of ViM interventions on the adoption of GAP and livestock practices, and determining the effect of the adoption of GAP and livestock practices on agricultural production and income. Through this analysis it is possible to determine not only if ViM was effective in increasing agricultural production and income, but also if the adoption of GAP was the channel for causing improvements, if any. The analysis, as a routine protocol, first identified whether the problem of self-selection is a valid concern for a given outcome variable of interest, and then utilized statistical methods to correct for the self-selection issue. Finally, the analysis identifies whether, after controlling for other observable confounders, ViM interventions were effective in increasing and diversifying smallholders’ agricultural production and income. Effectiveness of ViM Trainings on Diversification and Agricultural Income The first part of the econometric analysis was determining the effect of ViM interventions on agricultural production and income. To assess the effectiveness of ViM promoted GAP trainings and extension services on increasing and diversifying agricultural production, statistical analysis was done to estimate the effect of ViM on the average number of agricultural products produced and the yield of cowpeas and sorghum based on multivariate regression techniques. Econometric Model: In order to test whether ViM trainings affect diversification and agricultural income, we used a statistical model that accounts for potential correlation between the unobservable factors that affect ViM training participation and the unobservable factors that affect outcomes variables of interest (diversification and agricultural income). The model used is known as an endogenous treatment regression model in which 45 FINAL REPORT – VICTORY AGAINST MALNUTRITION (ViM) FINAL EVALUATION participation in ViM training is estimated using a logistic (probit) model and the effect of training on the potential outcome is estimated using a linear model that accounts for the self-selection problem. 54 In the logistic regression model ViM training participation has been expressed as function of various observable farmer, farm and household characteristics that might have influenced training participation (including age, gender, education, whether the farmer was a member of a farmer group, whether the farmer was a subsistence farmer, an indicator of household economic status and farm assets). Then, the effect of participation in ViM training on diversification and income has been estimated, correcting for potential self-selection bias. The regression method controlled for observable covariates that typically influence the outcomes of interest in addition to ViM training participation, such as farmer/household demographic characteristics (age, gender), measure of household assets, number of plots etc. The description of the covariates used in the regression and regression results have been presented in Annex VI. To estimate the effect of ViM training/extension services on the diversification measure (average number of different types of agricultural products produced), treatment has been defined as farmers who received extension services, trainings or support from the ViM project (on plant production, animal production, natural resource management, agricultural or livestock value chains, and farming as business) during the period 2012-2017. Similarly, to estimate the effect of ViM training/extension services on the yield of cowpeas and sorghum, training/extension services related to crop production has been used. Econometric results reported in Table 6 (based on Table 10.2- Table 13.3 of Annex VI) show that the beneficiaries who received/attended ViM promoted GAP training or were served by extension service providers, produced greater numbers of crops/livestock products on average compared those who did not participate in such trainings. On average, the number of agricultural products produced by the beneficiaries exposed to ViM trainings were 0.8 more than the non-beneficiaries with similar economic, and demographic characteristics (95% confidence interval: 0.45 -1.15 products). Similarly, those who participated in GAP training/received extension services enjoyed higher yields of cowpeas, producing on average 234 kg more cowpeas per hectare compared to the farmers who did not receive ViM training, which was approximately 70% higher than the average yield of non￾beneficiaries (95% confidence interval: 116-352 kg). However, for sorghum, econometric results indicate there was wide variation in treatment effect (large standard error), and there was no statistically significant effect (at the 10% level of significance) of GAP trainings/extension services on sorghum yields. In addition, we also estimated the effect of access to formal credit on crop diversification and yield. The results presented in Table 6 indicate that those who received a loan from formal financial institutions (banks, microfinance institutions) in the most recent crop season produced a greater number of products (0.43, with a 95% confidence interval of 0.19- 0.67 products), but there was no statistically significant effect of loans on yields. Thus, the results suggest that access to formal credit is linked to improved diversification. 54 The endogenous treatment regression model estimates average treatment effect based on a linear potential￾outcome model that allows for a specific correlation structure between the unobservables that affect the treatment and the unobservables that affect the potential outcomes. 46 FINAL REPORT – VICTORY AGAINST MALNUTRITION (ViM) FINAL EVALUATION To test the effect of ViM interventions on income, HDDS, assets value, and cowpea and livestock revenue and gross margins measures were constructed. We used an endogenous treatment regression model to estimate how ViM trainings/extension services affected the outcome indicators. The results reported in Table 6 (based on regression results of Table 11.2 of Annex VI) show that ViM trainings/extension services positively affected household income. At endline, those who were exposed to ViM trainings/extension services had greater dietary diversity and consumed 0.42 additional food groups than the non-beneficiaries (95% confidence interval: 0.20 - 0.63 additional food groups), which was equivalent to 11% more than the average HDDS value of non-beneficiaries.55 The results indicate ViM training had a positive effect on domestic and productive asset value as the average value of assets of beneficiary households was 189,052 CFA more than the households who did not receive ViM training (95% confidence interval: 86,526 – 291,578 CFA) which is equivalent to 50% more than the asset value of non-beneficiaries. There was a considerable effect of ViM training/ extension services on cowpea revenue as beneficiaries’ average revenue was approximately 22,400 CFA more than the non-beneficiaries (95% confidence interval: 9,766 – 35,034 CFA). Controlling for other factors, this was 69% and 81% above the non-beneficiaries’ average value of revenue. A similar trend was found for gross margin, in which beneficiaries’ average value was 36,140 CFA above the average for non￾beneficieries (95% confidence interval: 13,337 – 58,943 CFA). We estimated the effect of formal credit on cowpeas revenue and gross margin. Results presented in Table 12.2 and 12.3 show that while cowpeas revenue was significantly higher among the farmers who received credit from formal sources, there was no effect of credit on gross margin. We also explored how crop warrantage affected cowpeas revenue and gross margin, but found no statistically significant effect. Similarly, livestock revenues and gross margins of recipients of GAP trainings and extension services were 124,000 and 112,000 CFA more than the non-beneficiaries respectively. Overall, the regression results presented in Table 6 indicate that the beneficiaries who received ViM training/extension services were significantly better off than the non-beneficiaries with respect to their livelihood outcomes. Table 6: The Effect of ViM Interventions on Outcome Indicators Level of Analysis Outcome / Impact Indicators Indicator of ViM Intervention Multivariate Regression Results Interpretation Role of ViM in increasing and diversifying agricultural production Farmer Average number of different types of agricultural products produced per farmer GAP training, extension services, including livestock Positive effect Beneficiaries of GAP training, extension services from ViM produce greater number of agricultural products compared to non-beneficiaries Farmer Yield of Cowpea per farmer (kg/ha) GAP training, extension services (crops) Positive effect Yield of cowpea farmers who received GAP training, extension services from ViM higher compared to non￾beneficiaries 55 We found no statistically significant effect of formal credit, use of warrantage for crops and use of market price information from formal sources on HDDS. 47 FINAL REPORT – VICTORY AGAINST MALNUTRITION (ViM) FINAL EVALUATION Level of Analysis Outcome / Impact Indicators Indicator of ViM Intervention Multivariate Regression Results Interpretation Farmer Yield of Sorghum per farmer (kg/ha) No effect Yield of sorghum farmers was not effected by GAP training and extension services. Role of ViM in increasing Household Incomes Household Household Dietary Diversity Score (HDDS) GAP training, extension services, including livestock Positive effect Beneficiaries of GAP training, extension services from ViM consume greater number of nutrient rich food groups compared to non-beneficiaries Household Value of a set of assets (livestock, and other agricultural and non-agricultural assets) GAP training, extension services, including livestock Positive effect Beneficiaries receiving GAP training, extension services from ViM had larger asset values compared to non￾beneficiaries Farmer Total revenue: Cowpea GAP training, extension services (crops) Positive effect Farmers who received GAP training, extension services from ViM had larger revenue from sales of cowpea compared to non-beneficiaries Farmer Gross margin (sales) per-hectare: Cowpea Positive effect Farmers who received GAP training, extension services from ViM generated higher gross margin from sales of cowpea compared to non￾beneficiaries Farmer Total revenue: livestock Livestock training, extension services Positive effect Farmers who received livestock training, extension services from ViM enjoyed a greater revenue from sales of livestock compared to non￾beneficiaries Farmer Gross margin (sales): livestock Farmers who received livestock training, extension services from ViM generated higher gross margin from sales from livestock compared to non-beneficiaries The second step of the econometric analysis was determining the effect of ViM interventions on the adoption of GAP. To assess the role of ViM in improving agricultural practices, the adoption of at least four GAP for sorghum or cowpeas was used. Econometric Model: In order to test the effectiveness of ViM trainings on adoption of GAP and livestock practices, we used a logistic regression (probit model). The probit model estimated the likelihood of adoption of GAP practices as a function of ViM training and other covariates that affect adoption of such practices including farmer characteristics (such as age, gender, education, farmer group membership, household economic status, etc.) and other household level parameters. The regression model, with all included covariates and results, is presented in Annex VI. 48 FINAL REPORT – VICTORY AGAINST MALNUTRITION (ViM) FINAL EVALUATION The summarized results presented in Table 7 (based on Table 14.2 – 14.5 in Annex VI) show that there was a positive effect of ViM promoted GAP training and extension service on the adoption of at least four improved practices in sorghum and cowpeas production56. The estimation results indicated that those who received GAP training and extension service were 19 percent more likely to adopt at least four improved practices for sorghum and 17 percent more likely to adopt at least four improved practices for cowpeas production compared to those who did not receive ViM promoted GAP training and extension service57. To assess the role of ViM in improving livestock practices, two measures were considered (including use of at least veterinary care, complementary feeding, and habitat construction at the same time, and use of government or private sector veterinary care and vaccinations). The summarized regression results reported in Table 7 show that livestock training and extension services had a positive effect on both measures of improved livestock practices. The marginal effect estimation results indicate that those who received livestock training and extension services were 2 percent more likely to adopt improved livestock practices and 11 percent more likely to use vaccination/veterinary care from formal providers (government or private sector) compared to those who did not receive livestock training and extension services. Table 7: The Effect of ViM Interventions on the Adoption of GAP Level of Analysis Adoption of GAP Indicator of ViM Intervention Multivariate Regression Results Interpretation Farmer Adoption of at least 4 good practices in sorghum production GAP training, extension services (crops) Positive effect Beneficiaries of GAP training, extension services from ViM are more likely to adopt ViM promoted good agricultural practices in sorghum production compared to non-beneficiaries Farmer Adoption of at least 4 good practices in cowpea production Positive effect Beneficiaries of GAP training, extension services from ViM are more likely to adopt ViM promoted good agricultural practices in cowpea production compared to non-beneficiaries Farmer Adoption of good livestock practices (at least veterinary care, complementary feeding and habitat construction at the same time) Livestock training, extension services Positive effect Beneficiaries of livestock training, extension services from ViM are more likely to adopt ViM promoted good livestock practices compared to non-beneficiaries Farmer Adoption of good livestock practices (using government or private sector veterinary care and vaccinations) Positive effect Beneficiaries of livestock training, extension services from ViM are more likely to adopt ViM promoted good livestock practices compared to non-beneficiaries 56 The improved practices could be any four from the following list: Ploughing; Line planting; Sarclage/binage; Le buttage/ridging; Chemical fertilization; Organic fertilization; Crop rotation; Improved seeds; Plant treatment; seed treatment; Intercropping; Thinning; Zai-Anti erosion bunds. 57 The results communicated here are based on marginal effect estimations. 49 FINAL REPORT – VICTORY AGAINST MALNUTRITION (ViM) FINAL EVALUATION The third step of the econometric analysis was determining the effect of adoption of GAP on agricultural production and income. Econometric Model: The effect of adoption of GAP and livestock practices on agricultural outcome is estimated using an instrumental variable approach that accounts for the potential correlation between the unobservable factors that affect adoption of GAP and livestock practices and the unobservable factors that affect outcomes variables of interest. The instruments used to correct for endogeneity (potential correlation among unobservable factors) include participation in GAP training, and other farmer/household characteristics. The regressions used to estimate the effect of adoption on agricultural outcomes controlled for other covariates that typically affect the outcome, as presented in Annex VI. To assess the effect of adoption of GAP on agricultural production and income related to crops, the yields of sorghum and cowpeas were examined. Statistical results presented in Table 8 (based on Table 15.2 – Table 15.3) indicate that while there was no effect of ViM promoted GAP training on sorghum yields (at 10% level of significance), there was a positive effect of GAP training on cowpea yields. The results indicate that after controlling for observable covariates and potential endogeneity, adoption of GAP increased cowpea yield by 228 kg per hectare (95% confidence interval: 63-393 kg). To assess the effect of the adoption of good practices related to livestock, practice of using at least veterinary care, complementary feeding and habitat construction at the same time and use of government or private sector veterinary care and vaccinations were considered. The results reported in Table 16.2 show that adoption of good livestock practices had a statistically significant and positive effect on livestock revenue: those who adopted improved livestock practices earned 17,245 CFA more than those who (with similar observable characteristics) did not adopt such practices. Table 8: The Effect of the Adoption of GAP and Livestock Practices on Outcome Indicators Level of Analysis Effect of Adoption of Good Practices Outcome Indicator Multivariate Regression Results Interpretation Farmer Adoption of at least 4 good practices in sorghum production Yield of Sorghum per farmer (K.G/ha) No effect No difference in yield of sorghum per farmer from the adoption of at least 4 good practices in sorghum production. Farmer Adoption of at least 4 good practices in cowpea production Yield of Cowpea per farmer (K.G/ha) Positive effect The smallholders who adopted at least 4 ViM promote good practices in cowpea production experienced higher yields than those who didn’t adopt 4 or more practices Farmer Adoption of good livestock practices (at least veterinary care, complementary feeding and habitat construction at the same time) Livestock revenue Positive effect The smallholders who adopted good livestock practices experienced higher livestock revenue than those who didn’t adopt 50 FINAL REPORT – VICTORY AGAINST MALNUTRITION (ViM) FINAL EVALUATION Farmer Adoption of good livestock practices (using government or private sector veterinary care and vaccinations) Livestock revenue Positive effect The smallholders who adopted good livestock practices experienced higher livestock revenue than those who didn’t adopt The quantitative analyses provide statistical evidence that the smallholder farmers who received ViM promoted GAP/livestock and/or extension services were significantly more likely to adopt good agricultural and livestock practices. The beneficiaries who received training and adopted such practices enjoyed higher yield of cowpeas, generated higher revenue and had a greater gross margin from sales of cowpeas. A similar trend was noticed among the beneficiaries who received ViM promoted livestock training. Statistical results indicate that the beneficiaries accumulated more assets and consumed more nutrient rich food – all of which is indicative of the finding that the beneficiaries had greater income compared to the non-beneficiaries. 4.2.3 Surveyed Farmer Perceptions of Factors Influencing Crop Yields and Income The perceptions and opinions of surveyed farmers were captured in the quantitative endline survey and through the qualitative GDs. The these two datasets cannot provide strong statistical confirmation of attribution of improvements in crop yields and income directly to the ViM project, for the reasons cited earlier in this report, but they can provide useful insights into whether farmers in the ViM project area perceived their crop yields and household incomes to have changed over the project lifetime, and the extent to which they viewed ViM trainings and other activities to have contributed to those changes. The endline survey asked the farmers to rate the relative importance of various factors in bringing about a change in the household’s yield for most important crops and for their household’s farm income. The various factors considered were: o Weather, pests, production conditions o Market conditions: availability or cost of inputs, etc. o Me (i.e., the farmer) and my family and community: skills, hard work, collaboration o Training, information, input supply, market linkages offered by ViM interventions o Training, information, input supply, market linkages offered by other programs o Training on gender offered by ViM o Training on natural resource management offered by ViM The surveyed farmers were asked to rate the importance of each of the above factors. The responses to these questions regarding the importance of the different factors on yields and income are summarized in Figures 16 and 17. This figure shows that farmers considered each of the factors to be either important, very important, or essential by roughly similar percentages, including those that noted specific trainings by ViM. Participants in GDs across several village also noted substantial improvements to their crop yields, and often tied these specifically to some of the trainings on good agricultural practices that were provided by ViM. As one GD participant from Tamsin noted, “Yields almost doubled thanks to [use of] zai.” A woman from the GD held in Silmiougou noted, “Before the project, we just dumped the manure in the fields and we sowed in an archaic way. Now we practice zai. As for the conservation of crops, chemicals (kouka) were used. Now we keep it in sachets. These benefits are huge. This training helps [us] to better preserve our crops.” In another GD, women noted their views on the most important benefits they had received as a result of the ViM project: “There is increased production and better crop conservation [such as for] cowpeas. Practices such as zai and PICs have allowed us to increase our productions.” [Women’s GD, Konean]. In Nagroaogo Foulce village, men mentioned improved yields for their 51 FINAL REPORT – VICTORY AGAINST MALNUTRITION (ViM) FINAL EVALUATION cowpeas and millet crops, which they said resulted in part from the improved spacing at planting that they learned through ViM trainings, among other agricultural practices they learned through the project. Overall, GD participants mentioned increases in crop yields for at least one crop type, in each of the GDs held on agricultural topics, lending further anecdotal support to ViM’s effectiveness in promoting improved agricultural practices and increasing crop production. In terms of the perceived effects of such changes on income, there is also some qualitative support from the group discussions for a link between improved farmer practices, yields, increased farm earnings, and positive effects on household economic well being. For example, womein in Barsalogho village said they had seen an increase in their cowpeas yield, which enabled them to earn more from the same plot of land they had farmed in the past. They said the increased earnings could be used for children’s school fees, medicines and other family needs. Figure 16: Relative importance of different factors affecting crop yields 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Weather, pests, production conditions Market conditions Me and my family and community Training, information from ViM Training, information from other Training on gender by ViM Training on NRM by ViM Percent Factors Not Important at All Not Very Important Important Very Important Essential 52 FINAL REPORT – VICTORY AGAINST MALNUTRITION (ViM) FINAL EVALUATION Figure 17: Relative importance of different factors in affecting income However, qualitative results also indicate that some of the ViM activities were considered less effective, even if the main reasons cited were somewhat external to the project’s scope. Participants in two GDs from Konean and Nagraogo Foulce, for instance, cited a water shortage in their area that prevented successful implementation of certain activities. Overall, the constraint of insufficient water on various ViM-promoted agricultural activities was commonly noted, and discussed at length in nine of the GDs held. Various GD participants mentioned lack of sufficient water as a specific constraint on the effectiveness of or ability of villagers to implement agricultural trainings they received by ViM on, variously: growing moringa, production of off-season crops, promotion of vegetable farming by women, greater engagement in market gardening, and overall improvements to agricultural production in general. It is recognized by the evaluation team that addressing water constraints and improving the water supply was outside the scope of ViM’s activities. However, this external constraint to the potential effectiveness of some of ViM’s agricultural activities was also noted at the mid-term evaluation and clearly remained an issue for the realization of some of ViM’s agricultural activities in at least some of the project villages. It was also clear from the GDs held that beneficiaries had in some cases requested assistance for water infrastructure or water supply improvements from ViM, and had understood from ViM that this was outside the scope of ViM’s activities. For example, a participant in one GD noted, “We talked about wells but they [ViM] say it does not belong to their activities.” [Men’s GD, Gabou]. Despite this recognition of the limitations on ViM’s overall scope of assistance, the below responses from two GDs, to a question on what respondents thought the ViM project could have done better to ‘work with community members to help improve agricultural and livelihoods activities’ sums up beneficiaries views on the critical importance of addressing water-related constraints to achieving some of ViM’s core objectives: “These challenges related to improving agricultural production, they persist, not because the activities carried out by the project are ineffective, but because the central problem is the availability of water and this [problem] remains. Even ViM knows that without water you cannot do anything!” [Men’s GD, Firka] “Solve the water problem, and diversify activities for women.” [Women’s GD, Nagraogo Foulce] 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Weather, pests, production conditions Market conditions Me and my family and community Training, information from ViM Training, information from other Training on gender by ViM Training on NRM by ViM Percent Factors Not Important at All Not Very Important Important Very Important Essential 53 FINAL REPORT – VICTORY AGAINST MALNUTRITION (ViM) FINAL EVALUATION In terms of more minor constraints on the effectiveness of ViM’s agricultural activities, two GDs also noted that use of carts was limited by lack of appropriate animals to pull them. Equipment shortage was raised in one GD, where participants noted this had prevented the adoption of agricultural practices among beneficiaries. One GD mentioned that the lack of financial means had limited their ability to engage in gardening. The same group explained that certain markets were geographically inaccessible for them to sell sheep, requiring beneficiaries to resell their livestock locally. Finally, participants in another GD noted problems during the second phase of fertilizer provision, although this issue was not commonly raised across the GDs. According to the experience of participants in this GD, the fertilizer they received was not satisfactory, and both the product and the system failed to meet their needs. As a result, beneficiaries incurred losses, as illustrated below: “The second phase of donating fertilizer did not benefit us at all. At the beginning of the project, they told us about a system to give us fertilizer, but in the end when you observe the amount injected and the fertilizer was not satisfactory. If we had taken our money to get that at the market, it would be more beneficial…” [Tibi-Yarce, Men, GD] Roughly half of GDs (12 of 31) suggested helpful activities or further assistance that ViM or another organization could provide, although the evaluation team acknowledges that several of these activities were outside of ViM’s scope. For example, GD participants suggested additional support for dams and water retention (6 GDs), to assist villages with the creation of additional water reservoirs for gardening and drinking. “The challenges were the problem of water and marketing of agricultural products. In the whole village there is only one water point. Fetching water is very tiring. For lack of water we do not practice off￾season crops. The water problem remains.” [Konean Women] Beneficiaries also suggested a need for assistance with animal enclosures for their livestock (2 GDs), and two other groups requested support for growing rice. Beneficiaries from yet another two GDs asked for grants for agricultural credits in order to improve activities in which they were already engaged. Other requests were for support growing soumbala (1 GD), the creation of a health center (1 GD), the use of draft animals to farm (1 GD), affordable chemical fertilizer (1 GD), and basket weaving (1GD). A few GDs were more general in their request for additional assistance, requesting help with water, food, machines, training, seeds, and equipment. Some of these activities, including support for growing rice and soumbala, were implemented by ViM in other areas. CONCLUSIONS Overall, the GD qualitative data provides support that many of the ViM beneficiaries strongly believe that the ViM project has substantially improved crop yields and thus, consequently, incomes. While such a conclusion does not have the high confidence level that statistical conclusions might have, when taken together with the comparison of baseline and endline results presented in EQ1, the supplemental econometric results and the endline comparison of key indicator values for ViM beneficiaries and non￾beneficiaries presented in this section, the overall conclusion is that the ViM project has been effective in promoting smallholders’ access, affordability and use of agriculture and livestock extension services, and of project-sponsored high quality agricultural and livestock inputs including veterinary services. The outcomes for the ViM beneficiaries on their crop yields and income have also been positive, further confirming the overall effectiveness of the ViM project on agricultural issues. 54 FINAL REPORT – VICTORY AGAINST MALNUTRITION (ViM) FINAL EVALUATION 4.3 EQ3: EFFECTIVENESS OF VIM PROJECT IN PROMOTING PRACTICES AROUND INFANT AND YOUNG CHILD FEEDING, MATERNAL AND CHILD HEALTH; WATER, SANITATION AND HYGIENE IN REDUCING MALNUTRITION AMONG CHILDREN UNDER FIVE, AND PREGNANT AND LACTATING WOMEN. Overall, ViM project beneficiaries reported changes in MCHN and WASH outcomes within their communities as a direct result of the implementation of ViM-promoted practices, while the quantitative and qualitative data alike provide broad support for substantially increased prevalence of MCHN and WASH practices (and indeed, positive impact-level changes) in ViM intervention areas during the project lifetime. Together, the qualitative and quantitative evidence suggests substantial effectiveness of the project in promoting these practices and facilitating positive MCHN and WASH outcomes. Respondents in GDs and key informant interviews related several anecdotes which provided additional depth of understanding on the ways in which the implementation of practices around hand washing, food variation, health centers, latrine use and birth spacing have positively affected their lives. Breastfeeding Women across villages reported that ViM sensitizations on breastfeeding was effective in promoting changes to their child feeding practices. In some villages, women spoke specifically about the importance of exclusively breastfeeding for the first six months, and then slowly integrating millet and ground porridge into babies’ diets, for the overall health of their children. Although the majority of women did not have challenges with breastfeeding, some noted various constraints. Some women reported that juggling multiple responsibilities posed a challenge to breastfeeding, especially throughout the day. In one region, a Mother Leader noted that the women who had issues breastfeeding during the day were constrained by time. She explains: “There is a lack of time, especially among the Mossi [ethnic group] during the rainy season. They give themselves much more to the work of the fields than to take care of their children and often even the husbands demand they leave their children and come to cultivate.” [Mother Leader, Barsalogo] In other cases, respondents in one village reported issues with breastfeeding due to their own inability to maintain a proper diet for themselves. KIIs with CLTS committee members and volunteer village vaccinators (VVVs) in two villages indicate that although there have been many successes in getting women to understand the importance of optimal breastfeeding practices, there is still some work to be done in ensuring that all women understand and prioritize this. Hand washing/Latrine Use GD respondents also found training on sanitation and latrine use to be effective. Hand-washing trainings were found to be particularly effective, as some women respondents noted that prior to ViM activities, they did not understand the importance of hand washing before breastfeeding and food preparation. In this regard, respondents reported that they now understood the importance of hand washing, particularly in reducing the prevalence of disease amongst themselves and their children. KIIs with CLTS committee members and community health officials also corroborate that hand washing was understood to be an important element in reducing illness within their communities. Although respondents discussed having been trained on the importance of latrine use, the implementation of this practice was inconsistent. In villages where the means and materials for latrine construction were present, community members successfully implemented this practice. However, in 55 FINAL REPORT – VICTORY AGAINST MALNUTRITION (ViM) FINAL EVALUATION other villages, respondents noted that although the desire to implement the practice was strong, the means to do so made it impossible. This sentiment was also echoed in KIIs with Mother Leaders, as they noted that additional sensitization and additional financial support may be necessary to implement latrine use more broadly. Nutrition/Food Diversification ViM beneficiaries across villages considered food preparation demonstrations, trainings, and sensitization efforts around nutrition to be very effective. Women respondents noted that the training they received was a critical part in implementing changes to their children’s diets, particularly in preparing porridge. In three villages, respondents perceive that malnutrition has significantly decreased or almost disappeared in their communities as a result of ViM activities. Further, women in some villages also note the relationship between good nutrition and strong immune systems in children. In two villages, women explained how their training sensitized them to ways that good nutrition can help minimize diseases in their children. Women also demonstrated a clear understanding of the role of food diversification in nutrition. As such, women provided specific examples of the ways in which they had diversified their children’s feeding practices and diets through ViM promoted-practices: “Things have changed a lot: there is the abandonment of old methods in favor of new methods. We received sensitization sessions on different forms of nutrition (maintenance and growth). For example, the child is entitled to three meals a day. The child needs rich foods for its growth [such as] bean leaves, fish and meat. “[Konean, Women] Some challenges with food diversification were also outlined in the GDs and KIIs. In cases where respondents indicated that consumption of varied or vitamin-A rich foods was limited, they noted that a key reasons was that the means to consume such foods was also limited in their households. KIIs with MLA revealed that in instances where MAD practices were not being implemented, individuals often lacked the means to purchase vitamin-A rich foods. In one village, an MLA noted that there was still some additional sensitization that needs to be done around the consumption of vitamin-A rich foods, as she observed that some women do not fully understand their importance. Visiting Health Centers Across GDs, respondents noted that sensitization on the importance of visiting health centers was extremely effective. In many cases, respondents in GDs and KIIs noted that prior to ViM’s arrival, many women were giving birth at home, and without any antenatal consultations. Currently, many respondents noted that they visited health centers for antenatal visits, giving birth, vaccinations, and post-birth checkups for weighing children. As one respondent noted: “The health side helped us a lot because today we are going to the clinic for our health problems. Women give birth less at home, vaccination is more regular. There is no one who goes to term with pregnancy without doing a single weighing. We know that at three months we must start weighing it and after delivery, we must weigh and vaccinate.” [Kodibito, Women] Women across project areas also report that their husbands have also been sensitized to the need for visiting health centers. Consequently, they have become more supportive in accompanying them to health centers, and assisting in finding transportation when necessary. Women also reported that antenatal visits in particular have been important in helping husbands become more invested in their 56 FINAL REPORT – VICTORY AGAINST MALNUTRITION (ViM) FINAL EVALUATION child’s health. In one village, men respondents noted that the Mother Leader in their area was instrumental in changing their attitudes about visiting health centers, and supporting their wives in ensuring the health of their children. These men noted that prior to ViM, women often managed their pregnancies alone, from inception to birth. Respondents also reported challenges with accessing health centers consistently, especially for weighing. Although this was not discussed in-depth during GDs, KIIs with Mother Leaders revealed that the few women who were not taking their children for regular weighing may have been reluctant to do so because they could not afford the prescriptions they would receive at the time of the visit. Birth spacing/family planning58 In 15 of 19 GDs with specific probing on this, both men and women reported that birth spacing and family planning sensitization and training was effective. In some GDs, respondents noted that prior to ViM, many women were giving birth as little as one year apart. Now, women are giving birth as far as three years apart. GDs respondents reported that birth spacing and family planning had many benefits, and aided in promoting women’s health, and the health of their children. With additional time between births, women were more able to dedicate attention to the health and nutrition of their other children. In one village, men and women respondents reported that birth spacing was an essential element to increasing the economic participation of women. According to respondents in this village, birth spacing increased the availability of women to take on domestic work outside the home, and work in the fields. As one respondent notes: “Thanks to training around birth spacing, women are not giving birth randomly. They benefit not only from sensitization but also accompaniments. Family planning is well respected and it helps the households a lot. Thanks to spacing of births moms are more and more available for housework and country.” [Women’s GD, Basma.] Men (four GDs) and women (seven GDs) across villages noted that in some cases, men were not always immediately receptive to the adoption of family planning practices. However, with increased levels of training, sensitization, and observing the benefits of their peers whose wives had begun birth spacing/family planning, they became more amenable to the idea. Key informant interviews with MLAs, CVDs and Community Health Officials also demonstrate the efficacy of sensitization around birth spacing and family planning. Notably, CVDs note that sensitization around birth spacing and family planning have been the most useful of all activities because women are able to work, take better care of their children, and have fewer children. ECONOMETRIC ANALYSIS Using the quantitative data collected in the endline survey, econometric analysis was conducted to assess whether ViM activities had any statistically significant influence on malnutrition and other key MCHN outcome indicators tracked by the program. Econometric Model: In order to assess whether various ViM interventions affect MCHN indicators, the evaluation team used multivariate regression analysis to test whether: 1. ViM food rations affected stunting rates. 2. MCHN training affected WDDS, stunting and underweight rates. 58 While this was discussed by GD participants, it is noted that this was not developed as a specific activity under ViM. 57 FINAL REPORT – VICTORY AGAINST MALNUTRITION (ViM) FINAL EVALUATION 3. MCHN outcomes were affected by MLA and care group/neighborhood group participation. 4. ViM trainings/group participation affected adoption of improved health practices. The multivariate regressions checked and controlled for potential correlation between the unobservable factors that affect ViM training participation (or targeting) and the unobservable factors that can affect MCHN outcome variables of interest and estimated the influence of ViM training on the MCHN outcome indicators, after controlling various individual and household characteristics that might have influenced the outcomes (referred to as covariates). The regression models and results, together with the covariates used, are presented in Annex VI. ViM interventions provided food rations to children and PLW in the study areas until September 2016. Thus, it was important to assess whether provision of food rations had any influence on the malnutrition indicators. We used multivariate regression models to test the influence of ViM rations on stunting and underweight rates, controlling for the covariates that typically influence the outcomes. The results are presented in Annex VI. Table 9 presents a summary of the econometric analysis results (based on Tables 17.2- Table 18.7 in Annex VI), and indicate that after controlling for various potential confounders, those who were exposed to different ViM MCHN interventions were significantly better off compared to the non￾beneficiaries, with respect malnutrition indicators. The ViM interventions considered were food rations and MCHN training. Since the MCHN training was delivered through the MLA/care groups and neighborhood groups, we also estimated how participation in these groups affected the outcomes. The outcomes considered were WDDS, stunting rate and underweight rate for children under five years of age. For estimating whether ViM MCHN training affected WDDS, we used a multivariate linear regression model where the effect of MCHN training on WDDS was estimated controlling for various demographic (household size, gender and education of household head, number of educated adults) and economic status factors (including index of farm assets and total value of assets). The results presented in Table 9 (based on regression results in the annex Table 17.2) indicate that WDDS was marginally higher (by 0.19) among households who participated in MCHN training. Next, we estimated the effect of ViM interventions on child malnutrition indicators, where we estimated the effect of ViM food rations, MCHN training, adoption MAD and also participation in ViM promoted groups while controlling for household demographic and economic characteristics as well as hygiene facilities that affect such outcomes. While the effect of food rations was estimated using liner regression, endogenous treatment regression model was used to estimate the effect of participation in training and MLA and neighborhood groups. The regression results and covariates used have been provided in Table 18.1- 18.7 in Annex VI. An objective of the ViM food rations intervention was to provide temporary support to prevent malnutrition, whileViM livelihood interventions that focus on improving households’ economic conditions are in progress, so that households have better resources to prevent malnutrition at the conclusion of the food distribution period. Econometric results presented in Table 9 show that provision of ViM food ration had a negative (reducing) effect on the stunting rate. After controlling for other factors, households that received food rations had a 6 percent lower stunting rate than those who did not receive food rations. This result indicates that the prevalence of malnutrition was lower among beneficiaries who received food rations than for individuals (non-beneficiaries) who did not receive food rations. Thus, provision of food rations was effective in reducing malnutrition among children under five years of age in the project area. 58 FINAL REPORT – VICTORY AGAINST MALNUTRITION (ViM) FINAL EVALUATION In addition to this positive effect of ViM’s food rations activity on childhood malnutrition, results also suggest that the ViM promoted MCHN training was also successful: the households that were exposed to ViM MCHN training had 46 percent lower stunting rate than households that did not directly participate in this training (95% confidence interval: 33- 59%). Similarly, MCHN training had positive contributions in reducing prevalenc of underweight: the underweight rate in households that were exposed to ViM MCHN training was 46 percent lower than the non-participant households (95% confidence interval: 38- 53%) However, there was no effect of food rations on the underweight indicator.Since the MCHN training was delivered through different platforms, such as the MLA groups and neighborhood groups, we explored how participation in the MLA and neighborhood groups affected the MCHN outcomes. Similar to the econometric methods used to assess the effect of MCHN training based on endogenous treatment effect regressions, we explored the effect of participation in MLA and neighborhood groups on stunting outcomes, controlling for the effect of other covariates. The regression results presented in Table 18.4 and 18.5 indicate that these groups were effective in reducing stunting rates as households with members in MLA and neighborhood groups had a lower stunting rate than households that did not have members in MLA group and neighborhood groups. The estimation results indicate that stunting rate in the household with members in MLA group was 16 percent lower than the households that did not have a members in MLA group (95% confidence interval: 2- 31%). Similarly, household with members in neighborhood groups was 19 percent lower than the households that did not have a members in MLA group (95% confidence interval: 4- 33%). Thus, our analysis indicates that MCHN training delivered through the MLA and neighborhood groups had significant effect on reduction of malnutrition among children below 5 years of age. Econometric results also show that ViM WASH training was effective in reducing stunting rates, as the stunting rate among households that received WASH training was lower than households with similar characteristics that did not receive WASH training. The estimation results indicate that the stunting rate in households that received WASH training was 45 percent lower than for households that did not receive WASH training (95% confidence interval: 31- 59%). It is important to note that the regression results also indicate that the stunting and underweight rates in households that had soap at hand washing facility were significantly lower compared to the households with similar demographic and economic profiles but did not have soap for hand washing. The estimation results indicate that the stunting rate among households with soap at hand washing facility was approximately 4 percent lower than the households that did not have soap. This finding demonstrates that improving availability of soap for hand washing practices can play a significant role in reducing malnutrition among children below five years of age. 59 FINAL REPORT – VICTORY AGAINST MALNUTRITION (ViM) FINAL EVALUATION Table 9: Effect of ViM Interventions on Malnutrition Level of Analysis Outcome/Impact Indicators Indicator of ViM Intervention Multivariate Regression Results Interpretation Role of ViM in Reducing Malnutrition Woman WDDS ViM MCHN training Positive effect Women from households that received ViM MCHN training had higher dietary diversity scores than the women who did not receive the training. Household Stunting rate ViM Food Rations for Children Positive effect Stunting rate among children in the households where children received ViM food rations were lower than the households that didn’t receive such rations. Household Stunting rate ViM MCHN training Positive effect59 Stunting rate among children in the households that received ViM MCHN training were lower than the households that didn’t receive such training. Household Underweight rate ViM MCHN training Positive effect Underweight rate among children in the households that received ViM MCHN training were lower than the households that didn’t receive such training. Household Stunting rate ViM WASH training Positive effect Stunting rate among children in the households that received ViM WASH training were lower than the households that didn’t receive such training. Household Stunting rate Participation in Neighborhood Groups Positive effect60 Stunting rate among children in the households with women in Neighborhood Groups were lower than the households without members in such groups. Household Stunting rate Participation in MLA group Positive effect Stunting rate among children in the households that had a woman participating in MLA groups was lower than the households without members in such groups. Role of ViM in Improving Adoption of Improved Health Practices Household Prevalence of children 6-23 months receiving a minimum acceptable diet (MAD) ViM MCHN training Positive effect Households that received ViM MCHN training had a higher percentage of children receiving MAD than the households that didn’t receive such training. 59 Note that a positive effect of ViM on stunting is indicated by a negative sign on the coefficient for this variable (i.e., a reduction in the stunting rate). 60 As above. 60 FINAL REPORT – VICTORY AGAINST MALNUTRITION (ViM) FINAL EVALUATION Level of Analysis Outcome/Impact Indicators Indicator of ViM Intervention Multivariate Regression Results Interpretation Household Prevalence of children 6-23 months receiving a minimum acceptable diet (MAD) MLA group participation NO effect There was no difference in MAD scores in households that had a woman participating in MLA groups compared to the households with no MLA member. Household Prevalence of children 6-23 months receiving a minimum acceptable diet (MAD) Neighborhood group participation Positive effect Households with members in Neighborhood Groups had higher percentage of children receiving MAD than the households that didn’t participate in neighborhood groups. Household Prevalence of children 6-23 months receiving a minimum acceptable diet (MAD) ViM Food Rations for Children Positive effect Households that received ViM food rations for children had higher percentage of children receiving MAD than the households that did not receive ViM food rations. Household Likelihood of households with newborns receiving a postnatal health check within two days of birth ViM MCHN training Positive effect Households that receive ViM MCHN training had a higher percentage of newborns receiving a postnatal health check within two days of birth. Household Likelihood of households with newborns receiving postnatal health check within two days of birth Participation in Neighborhood Groups Positive effect Households with members in Neighborhood Groups had higher percentage of children who received postnatal health check than households that did not participate in Neighborhood Groups. Household Likelihood of households with newborns receiving postnatal health check within two days of birth MLA group participation Positive effect Households with MLA members had higher percentage of children who received postnatal health check than households that didn’t have MLA members. Household Likelihood of households with children 12-23 months of age fully immunized ViM MCHN training Positive effect Households that receive ViM MCHN training had a higher percentage of children12-23 months of age fully immunized. Household Likelihood of households with children 12-23 months of age children fully immunized Participation in Care Groups or Neighborhood Groups Positive effect Households with members in Neighborhood Groups had higher percentage of fully immunized children than the households that did not participate in Neighborhood Groups. Household Likelihood of households with children 12-23 months of age children fully immunized MLA group participation Positive effect Households with MLA members had higher percentage of children who were fully immunized than households that didn’t have MLA members. Next, we explored how the ViM interventions affected MCHN practices using multivariate regression analysis that considered the effect of MCHN training on Minimum Acceptable Diet (MAD), postnatal health checks and immunization practices. We used linear regression to estimate whether participation 61 FINAL REPORT – VICTORY AGAINST MALNUTRITION (ViM) FINAL EVALUATION in MCHN training affects the prevalence of children receiving a MAD, controlling for household demographic and economic factors. The results presented in Table 9 (based on regression results in Table 19.2 -19.5 in Annex VI) show that households that received MCHN training had a higher prevalence of children receiving a MAD. While exploring the effect of group participation, we found that households with a neighborhood group member had a higher percentage of children receiving MAD than the households that did not have a neighborhood group member. However, there was no such effect of participation in MLA groups. Thus, that the findings suggest that the neighborhood groups were effective in influencing households to adopt improved child feeding practices. Also, we tested the effect of child rations on MAD. The results reported in Table 19.5 indicate that the households that received child rations from ViM had greater prercentage of children receiving MAD. In addition, we also tested whether households where a greater percentage of children received MAD had lower stunting rate using an instrumental variable model that accounts for the potential correlation between the unobservable factors that affect adoption of MAD and the unobservable factors that affect stunting rate. The instruments used to correct for endogeneity (potential correlation among unobservable factors) include participation in MCHN training, and neighborhood groups, and other farmer/household characteristics. The regressions results presented in Table 19.6 in Annex VI, indicate that adoption of MAD has no statistically significant effect on stunting rate. In other words, we found no contemporaneous correlation between MAD and stunting rate. Postnatal health check and immunization of young children are two important indicators of quality of health services for children. We used a logistic regression to test whether MCHN training and group participation affected adoption of improved MCHN practices (postnatal health checks of new-borns and full immunization of children below 2 years), after controlling for household demographic and economic factors. Results presented in Table 9 (based on Table 20.2 – 20.4 in the Annex) indicate that MCHN training was effective in influencing the likelihood that a household will take newborns for postnatal checks. The results of estimation indicate that households that received MCHN training was 5 percent 61 more likely to have new-born children taken for postnatal health checks wihin two days of birth than the households that did not receive MCHN training (95% confidence interval: 1- 10%). We also estimated the correlation between group participation and postnatal health checks. The estimation results show that MLA and neighborhood group participation were positively correlated with the likelihood of undertaking a postnatal health check. However, MLA group participation was significant only at the 10% level of significance. Also, estimation results indicate households with a neighborhood group member were 10 percent more likely62 to have postnatal health checks of new borns, while households with a MLA group member were 5 percent more likely to have postnatal health checks of new borns as compared to the households that did not have a member in such groups. We did a similar analysis of the effect of MCHN training and group participation on immunization practices of children under 24 months of age. Table 9 (based on regression results presented in Table 21.2-21.4 in the Annex) demonstrates similar results: MCHN training increases the likelihood that family has a fully immunized child below 24 months (by 6 percent63). Households with an MLA member and neighborhood group member were respectively 7 and 5 percent more likely64 to have a child below 2 with complete immunization as prescribed for ensuring protection against life threatening diseases. 61 Based on marginal effect estimation. 62 Based on marginal effect estimation. 63 Based on marginal effect estimation. 64 Based on marginal effect estimation. 62 FINAL REPORT – VICTORY AGAINST MALNUTRITION (ViM) FINAL EVALUATION Results of the econometric analysis as presented above indicate that overall, ViM promoted interventions, controlling for other covariates, improving mother and child health practices. Improving sanitation practices and hygiene is an important strategy adopted by ViM to improve child health outcomes, as it could prevent the spread of diarrheal diseases which significantly increases malnutrition. Next, we present the results of the analysis of the effectiveness of WASH training in improving sanitation and hygiene practices. We used multivariate regression techniques to test whether participation in ViM WASH trainings had any influence on knowledge of beneficiaries about critical moments of hand washing and influence behavior change relating to their hygiene practices (use of hand washing), controlling for other covariates that typically influence households’ hygiene practices as described in Annex VI. Table 9 demonstrates the results of econometric analysis exploring the effectiveness of WASH training based on the regression results presented in Table 22.2- 22.7 in Annex VI. The results show that participation in ViM WASH training improved beneficiaries’ knowledge about the importance of hand washing: households which participated in ViM WASH training were 11 percent65 more likely to know all the critical moments of handwashing (95% confidence interval: 4-19%) than those who did not participate in ViM WASH training. Simailarly, we also tested the effect of participation in ViM WASH training on adoption of hygiene practices. The results indicate that ViM WASH training also influenced adoption of improved hygiene practices (captured by presence of soap and water in handwashing facility). The econometric results indicate that households which participated in ViM WASH traininig were 7 percent66 more likely to have soap and water in handwashing facility (95% confidence interval: 4-19%) than those who did not participate in ViM WASH training. Thus, the above set of results indicate that ViM promoted WASH training was effective in improving knowledge about the importance of hygiene practices and also influenced households to adopt improved handwashing practices. Table 10: Effect of ViM Interventions on Hygiene Practices Level of Analysis Outcome/Impact Indicators Indicator of ViM Intervention Multivariate Regression Results Interpretation Role of ViM in Influencing Hygiene Practices Household Percentage of respondents who know all critical moments for hand washing to prevent diarrheal disease WASH training Positive effect Households that received ViM WASH training were more likely to be aware of all critical moments of hand wash compared to the households that did not receive ViM WASH training. Household Percentage of households with soap and water at a hand washing station commonly used by family members WASH training Positive effect Households that received ViM WASH training were more likely to have soap and water at a hand washing station compared to the households that did not receive ViM WASH training. 65 Based on marginal effect estimation. 66 Based on marginal effect estimation. 63 FINAL REPORT – VICTORY AGAINST MALNUTRITION (ViM) FINAL EVALUATION CONCLUSIONS The results from the quantitative analyses indicate that ViM project activities significantly lowered malnutrition rates among beneficiary households and positively affected the adoption of improved health and hygiene practices in the ViM project area. This is corroborated by qualitative data provided by ViM project beneficiaries, in which respondents reported changes in MCHN and WASH outcomes within their communities as a direct result of the implementation of ViM-promoted practices, while the quantitative and qualitative data alike also provide broad support for substantially increased prevalence of MCHN and WASH practices (and indeed, positive impact-level changes) in ViM intervention areas during the project lifetime. Together, this evidence suggests substantial effectiveness of the project in promoting these practices and facilitating positive MCHN and WASH outcomes. GD and KII respondents related several anecdotes outlining the ways in which the implementation of such practices has positively affected their lives. 4.4 EQ 4: EFFECTIVENESS OF THE STRATEGIES IMPLEMENTED BY VIM PROJECT TO ENSURE THAT PRIVATE/PUBLIC PROVIDERS OF AGRICULTURE, LIVESTOCK, HEALTH, WOMEN'S EMPOWERMENT, AND NUTRITION SERVICES, COMMUNITY ACTORS INVOLVED IN HEALTH AND NUTRITION PROMOTION WILL CONTINUE TO PROVIDE SERVICES AFTER THE END OF VIM? FINDINGS 4.4.1 What is the level of motivation of the service providers to continue providing services after the project ends, and likelihood of sustained service provision after the end of ViM? Thirty-three of the 36 KII participants interviewed67 reported that they would be motivated and interested in continuing all ViM activities once the project is over. Of those that reported being motivated and interested to continue activities, 23 reported that activities could continue without USAID financing, while nine reported that they could not, for reasons specified below (of these nine KIIs, four were health agents of ViM focal points on health and nutrition issues at commune or region level; the remainder were village vaccinators (1), mother leaders within villages (2) or members of CLTS groups (2)) . However, eight key informants mentioned that motivation and interest to continue activities would be impaired by resource constraints, including insufficient equipment (three KIIs), limited funds (four KIIs), and limited access to trainings (one KII). “Yes, I think that the organizations that implement the ViM project activities are motivated and interested in continuing to offer some or all of these activities, once the project is completed: they themselves can attest to the success of the project activities wherever it has gone, and will also want make this change elsewhere.” [Maman Leader, Nagraogo Foulce] 67 See Annex VIII for an anonymized list of KII respondents and service provider types. Of the 36 KIIs conducted, 11 were at Region level, 12 were at Commune level, and 13 were at village level. Within villages, KIIs were conducted with four mother leaders, two volunteer village vaccinators, members of three CVDs, and members of four CLTS committees. Beyond village level, KIIs were held with four health agents; six different producer unions; livestock production and nutrition focal points; regional directors, zonal heads and/or ViM focal points within ministries of health, animal resource, environment, women and family issues (with respect to agriculture, livestock and health services); veterinary service providers and chefs de poste veterinaires; and agricultural inputs dealers. 64 FINAL REPORT – VICTORY AGAINST MALNUTRITION (ViM) FINAL EVALUATION The indication that 23 of 36 KII respondents (64%) believed that ViM activities could be continued independently of USAID financing suggests good motivation and a possibility for sustained service provisioning after the ViM lifetime, but there is also an indication that additional resources or support will be required for this sustained service to happen in practice. For example, five of these KII respondents qualified their answers, saying that activity continuation would require increased or repeated trainings (three KIIs, across volunteer village vaccinators and producer group members); that organizations would need to be able to keep the equipment they have received from the project (one KII); or that the organization would need to resort to funding from other donors (one KII). In addition, 11 of the 36 KIIs (30%) did not believe that ViM activities could continue without USAID funding (this view was expressed primarily by health sector KIIs at commune or region levels, by village vaccinators, and by members of CLTS groups within villages) . Besides the implied lack of alternatives for funding that prompted five of the KII respondents to answer as they did above, some respondents also mentioned that they did not feel that their organization was sufficiently trained, and that institutional knowledge was not sufficient for independent service delivery without project support (four KIIs). Some KIIs described their organizations as young or fairly new and, according to one quote, likening the organization to a child learning to walk. One health sector key informant expressed a viewpoint that while some of the health and nutrition projects or service provisioning could continue, others would have to be stopped due to limited resources or lack of funding on the part of the state, while another key informant within the veterinary services sector similarly noted that the state has limited means to subsidize livestock products or provide equipment for service provisioning. When asked about which challenges would most significantly threaten the continuation of ViM activities after the project lifetime, seven out of 36 KIIs said there was nothing significant impairing sustained ViM activities (two CLTS representatives, two producer union representatives, one mother leader, one health agent and one SILC mobilizer). However, the majority of respondents, spanning all service provisioning sectors covered by the KIIs, mentioned challenges and their responses highlighted common themes of insufficient trainings (12 KIIs) and funds (nine KIIs) to continue service provision and ensure sustainability. Lack of state government support for activities was also cited as an issue by five KIIs (three KIIs within health and nutrition sectors, and two KIIs within livestock or animal production and veterinary services sectors). Equipment needs was mentioned by four KIIs, as well as means for transportation and distant health and educational facilities (three KIIs). Another challenge to sustained service delivery, noted by three KIIs (two within animal production or livestock services and one nutrition focal point), stemmed from difficulties to sufficiently involve beneficiaries and obtain participants commitment and interest in the programming. Two respondents, both in the health and nutrition sector, mentioned research related challenges: one noted that there were few information tools for data collection, and another spoke about how the systems and techniques that ViM has helped facilitators master would not function as well without USAID backing. 4.4.2 What are the technical, organizational, and management capacities of the producer groups developed by ViM project and their potential effectiveness to provide sustained service delivery after the end of ViM? When asked about what key elements of support are needed for organizations to continue offering ViM services, 19 of the 36 key informants said that multiple types training would be essential, while six respondents mentioned capacity building in particular. These viewpoints spanned all sectors covered by the KIIs, and were fairly equially distributed across village-level and commune-level KIIs. The types of needed trainings mentioned were sanitation training (two respondents) and hygiene training (one respondent), with organization specific training implied by the other respondents. Six respondents said that financial support would be crucial. Various forms of technical support were also mentioned as necessary to ensure that organizations could continue to offer ViM services. This included the provision of supervisory and guidance services for target groups like livestock owners and farmers (four KIIs); 65 FINAL REPORT – VICTORY AGAINST MALNUTRITION (ViM) FINAL EVALUATION support for regular monitoring activities to commence or continue (six KIIs); and help with continued or newly initiated consultations with beneficiaries (three KIIs). Support for awareness raising and women empowerment activities (one KII) and advocacy for producer union members (one KII), were also specified. Other necessary support that was mentioned include: Material support, including helping the organization to obtain seeds, fertilizers, equipment, funds, medicine/vaccines, and the building of storage units for beneficiaries (13 KIIs); and transportation support, to help facilitators reach project sites (one KII). Some of the respondents answered this question by specifying the services they already provided and would continue after the project’s end, which may indicate better organizational preparation for the removal of USAID financing. Overall, these results suggest that technical, organizational and/or other institutional capacity constraints are likely to challenge continued service delivery after ViM for many such project-supported groups, despite their interest to continue providing the project-promoted services. « Yes, they are learning to put in place their own initiatives. At the stage that we are currently, we have not reached a high degree of maturity to evolve on our own; Yes, I think they are quite interested and able to partially support the costs associated with these services: if someone knows the importance of what he wants (he will be able to do so).” [CVD member Barsalogho KII] « Yes, it is in their interest because they are going to continue and their capacities have been strengthened to do so and they have acquired the necessary materials to do so.” [PV Barsalogho KII] « “Without lying it is difficult to continue for lack of means. Even if we continue, it will not be all of the activities.” [Focal Point MRAH KII] “No, we have not reached a good level of maturity to be autonomous, so not all activities can continue. We still need support.” [VVV Pissila, KII] “Ha! Impossible. We look like a child who crawls on all fours and learns to walk, if he/she is not followed he will fall. We ask for training and financial support.” [CLTS member Namissigma, KII] Among the cowpeas and onions producer group representatives who were interviewed at endline, all noted large gains from the ViM project, within their organizations and among the targeted producer groups for the selected value chain crops (cowpeas, sorghum, tomoatoes and onions), in terms of capacity building for agricultural production, and crop conservation, marketing and storage techniques. Some of the key areas of capacity strengthening that they noted were around using a group sales system for products, and a more consolidated approach to purchasing required inputs for production. Still, a need for additional traning and supervision of groups after the project lifetime was commonly highlighted, as was a need for funding to support training and monitoring activities beyond the project lifetime. In addition, respondents mentioned remaining gaps in groups’ capacity to obtain loans for inputs and large capital purchases, access to marketing equipment such as scales and sieves, and overall organizational management skills. 4.4.3 What has been done by ViM project to ensure that the service providers will have continuous access to resources to enhance their capacities? When asked about what key elements of support are needed for organizations to continue offering ViM services and strengthen their capacity to do so, KII respondents emphasized essential trainings, particularly in the areas of sanitation and hygiene. Various forms of technical support were also recommended to ensure continued organizational provision of ViM services, including: offering supervisory and guidance services for target groups such as livestock owners and farmer groups; conducting regular monitoring activities; and continued or newly initiated consultations with 66 FINAL REPORT – VICTORY AGAINST MALNUTRITION (ViM) FINAL EVALUATION beneficiaries. Additional areas specified as necessary for support included awareness raising and women’s empowerment activities and advocacy for producer union members. Respondents, in the same vein, cited a number of inputs that would be critical for continuation of ViM services. Support for acquiring seeds, fertilizers, equipment, funds, medicine/vaccines, and help building storage units for beneficiaries were each identified as key areas of potential material support. Some KII respondents, however, noted the services they are already providing and would continue to do so after the ViM project’s end, which indicate improved organizational preparation for the end of USAID funding. “We especially want training because it is more beneficial than money. Knowledge is the greatest wealth. If we are given money, we will finish spending without doing anything, but it is training, we learn something that does not end.” – Agent de Santé à Base Communautaire (ABSC) Delga, KII CONCLUSIONS According to qualitative results from KIIs, there is some likelihood that ViM activities could be continued independently of USAID support. However, several respondents added that activity continuation or continued service provision would require increased or repeated trainings, partner organizations would have to keep the equipment they have received from the project, or that organizations would need to seek out other donors for support. Among those in-depth interview respondents who were less convinced of sustained service delivery, several stated limited alternatives for funding as the reason for their concern. Others noted that partner organizations lack sufficient training, and that institutional knowledge was too limited at this stage for sustainability upon project withdrawal. Overall, these results suggest that technical, organizational and/or other institutional capacity constraints are likely to challenge continued service delivery after ViM for many such project-supported groups, despite their interest to continue providing the project-promoted services. This does not appear to be due to respondent dissatisfaction with the ViM-provided support and capacity building to individual providers or service groups. Rather, there is a need for continued support for some additional time, before such organizations are in a position to provide services and function wholly independently. 4.5 EQ 5: LEVEL OF DEMAND AMONG THE COMMUNITY MEMBERS AND WILLINGNESS TO PAY FOR ESSENTIAL AGRICULTURAL, LIVESTOCK, HEALTH AND NUTRITION SERVICES PROVIDED BY THE PROJECT. FINDINGS The demand and willingness to pay for essential services provided by the project on the part of project beneficiaries is tied to beneficiaries’ perceptions of the quality, effectiveness and suitability of those services. Qualitative findings at endline indicated that the vast majority of participants reported very positive feelings and high motivation to continue applying the methods and practices they learned, while the quantitative results provided further support for a high level of continued demand for ViM￾promoted services and trainings. This motivation was particularly high in the case of ViM agricultural and WASH activities. An increase in agricultural production was noted in 16 GDs. There was also some indication of spillover effects to non-project beneficiaries, a further indication of perceived effectiveness of project activities and continued demand for project services beyond the project lifetime. Participants in 11 GDs reported sharing the knowledge they had learned during training with non-beneficiaries, whether it be in their own village or their neighboring village. Knowledge-sharing behavior was reported both for agriculture and WASH activities. “The project has helped us hold meetings between community members. A few years before ViM was here, we could go years without seeing each other. But now, we see each other almost every month. The 67 FINAL REPORT – VICTORY AGAINST MALNUTRITION (ViM) FINAL EVALUATION community is gathering. For example, in all the surrounding villages we are known and we know them too. People are imitating us, other women want to be like our women here.” [Nagraogo Foulce – Men] This comment indicates that ViM has contributed to the strengthening of social cohesion which is an important factor in the resilience of people at community level. The high likelihood for continued demand for ViM-promoted services by households in the project area is also supported by related results from the household survey, with respect to household expectations to continue ViM-promoted technologies or practices, and continued participation in related trainings. These results are summarized in the sections below. Beneficiary Expectations to Continue ViM-promoted Technologies or Practices The majority of surveyed households indicated that they planned to continue to use at least one practice or technology (83%) or continue at least one training (84%), which suggests that there is strong continued demand for these practice/technologies and trainings. Female-headed households were somewhat less likely to continue using the practices or technologies (78%) than male-headed households (89%), although this indicator of continued demand is high for both genders. For each practice or technology, men were also more likely to indicate they intended to continue using it. The same trend is observed for continuing trainings (78% for men and 73% for women). Agricultural trainings and practices were the most likely to be continued for each gender. WASH or child health and nutrition practices and trainings were indicated as the next most likely to be continued. Participation in Saving and Internal Lending (SILC) groups was the least likely to be continued for both practices and training. Respondents indicated that farmers or Care/Neighborhood Groups were the most common source of help adopting practices and continued trainings, followed by the government. Overall, the high number of households who said they planned to continue using ViM-promoted practices, or would continue to participate in offered trainings offered through farmers’ groups or Care/Neighborhood Groups suggests a high likelihood for continued demand for ViM-promoted activities after the project lifetime. Table 11: Technologies or practices households intend to use after the end of ViM activities Technologies/Practices Percent Who will help households adopt practices after ViM project end Farmer groups Care/ Neighbor￾hood Groups Govern￾ment Private institution Other NGO Myself Improved agriculture 59.9% 73.7% 27.5% 45.0% 10.8% 16.6% 14.3% Improved crops seed 37.7% 69.0% 33.9% 48.2% 11.9% 18.6% 14.8% Improved fertilizers/pesticides 34.7% 65.3% 24.0% 50.3% 10.2% 21.2% 18.0% Improved livestock 24.6% 62.9% 35.6% 56.6% 17.5% 24.9% 11.6% NRM 48.6% 61.4% 23.3% 37.6% 9.8% 16.8% 26.1% Child health/nutrition 44.0% 19.8% 69.1% 46.6% 9.2% 17.9% 16.9% WASH 51.7% 25.7% 58.1% 43.7% 10.4% 22.7% 21.1% Participating in SILC group 14.8% 45.1% 64.2% 49.1% 25.3% 26.2% 4.9% NONE 12.5% N/A N/A N/A N/A N/A N/A The majority of households surveyed said they planned to continue using at least one practice or technology (87.5%), which also suggests that the majority of households continue to find these practices/technologies useful. The practices or technologies that households indicated they planned to continue using the most were (in decreasing order): improved agricultural practices, WASH practices, NRM practices, child health and nutrition practices, improved crop seeds, and improved livestock practices. Less than a quarter of households said they planned to continue using improved livestock 68 FINAL REPORT – VICTORY AGAINST MALNUTRITION (ViM) FINAL EVALUATION practices (24.6%) or participate in a SILC group (14.8%), an indication of beneficiary perception of lower effectiveness, practices not reaching most of the beneficiaries, or lack of knowledge of practices for each of these areas. The anticipated source of help to continue to use the practices/technologies varied, but farmers’ groups or Care/Neighborhood Groups were the most common response for each practice, with government training as the next most common for each. Less than a third of farmers planned to rely on private institutions, other NGOs, or themselves for help adopting these technologies/practices after the project lifetime. The practice that households indicated they intend to continue using the most was improved agricultural practices (59.9%), and they planned to rely on farmers groups (73.7%) or the government (45.0%) to provide adoption support. WASH practices were the next most likely practice to be continued, at 51.7%, and most households planned to rely on Care/Neighborhood Groups (58.1%) or the government (43.7%) for this support. Additionally, 48.6% of households intended to continue using NRM, and planned to rely on farmers groups 61.4% of the time. Forty-four percent of households indicated they planned to continue child health and nutrition practices, and they planned to rely on Care/Neighborhood Groups (69.0%) or the government (48.2%) for this support. Improved seeds (38%) and fertilizers/pesticides use (34.7%) were planned to be continued with help from famers groups (69.0% and 65.3%, respectively). Table 12: Trainings households expect to continue to attend after the end of ViM intervention Technologies/Practices Percent Who will help households adopt practices after ViM project end Farmer groups Care/ Neighbor￾hood Groups Govern￾ment Private Institutions Other NGO Agriculture- crops 65.6% 82.1% 28.6% 49.8% 12.4% 22.9% Livestock 44.5% 63.0% 31.2% 62.0% 18.7% 31.4% Sanitation and WASH 44.6% 31.8% 74.1% 50.2% 12.2% 26.5% Literacy 23.6% 31.2% 69.0% 57.9% 17.3% 27.9% Gender 15.0% 27.1% 78.1% 53.2% 15.5% 25.8% NRM 29.6% 78.6% 43.2% 57.8% 12.7% 23.3% Child health and nutrition 36.2% 20.3% 80.1% 60.0% 11.5% 29.4% Participating in SILC group 15.8% 36.9% 62.5% 64.0% 36.3% 36.0% NONE 13.5% N/A N/A N/A N/A N/A Beneficiary Interest in Continued Trainings As for ViM-promoted practices, the majority of households also indicated that they planned to continue at least one form of training after the ViM project lifetime (86.5%), which suggests they found the trainings useful, and there is a high likelihood of continued demand for them. The trainings that households indicated they were the most likely to attend were on (in decreasing order): agriculture, sanitation, livestock, child health and nutrition practices, and NRM. The source for continued training varied across the different trainings, but farmers groups or Care/Neighborhood Groups were the most common response for each question (except SILC groups), with government training as the next most common for each. Less than a third of farmers planned to receive training from private institutions or other NGOs (except SILC groups). 69 FINAL REPORT – VICTORY AGAINST MALNUTRITION (ViM) FINAL EVALUATION Less than a quarter of households plan to continue attending trainings in literacy (23.6%), gender (15.0%), or SILC groups (15.8%), which suggests these trainings were potentially viewed as less valuable or perhaps households felt they did not have the required skills to take advantage of them. Households expected to rely most on the government for SILC groups practices (64.0%), followed by Care/Neighborhood Groups (62.5%). Additionally, about 36% of people expected to receive continued trainings from farmer groups, private institutions or other NGOs, which was the highest percent for private institutions and other NGOs of all the practices. The training that the most households indicated they planned to continue to attend was agricultural crop training (65.6%), and most households expected to rely on farmers groups (82.1%) or the government (49.8%) to receive this training. Households also expected to continue trainings in livestock or sanitation, at 45% of households for each. The majority of households expected to receive livestock training from farmers groups (63.0%) or the government (62.0%), and sanitation training from Care/Neighborhood Groups (74%) or the government (50%). Additionally, 36.2% of households expected to continue child health and nutrition practices, and they expected to receive this training from Care/Neighborhood groups (80.1%) or the government (60.0%). NRM trainings were expected to be continued by 29.6% of households, and they planned to receive this training from farmers groups (78.6%) or the government (57.8%). Willingness to Pay With respect to willingness to pay, in three of three GDs in which follow up probing directly asked about this, participants universally reported that they would be willing to pay for activities after the project lifetime, particularly for training on WASH topics. Willingness to pay is typically derived from motivation and household resource capacities, but the ability to pay is typically a function of each household’s current situation and outside of their direct and immediate control. Realized demand, therefore, requires a combination of both willingness and ability to pay; and this is often where some important sustainability challenges are noted. Across the GDs, respondents stated that they understood the value of ViM activities, and would be willing to pay for services provided by ViM activities beyond the life of the project. However, many respondents also noted that they currently did not have the financial means to pay for project activities, and that they could not anticipate when they would have the means to do so. GD and KII respondents noted that where challenges with implementation of practices occurred, it was often due to the lack of financial means. Service Demand and Beneficiary Targeting Given ViM’s objective to reduce food insecurity and malnutrition, and improve incomes and livelihoods of the poorest and most vulnerable households in project areas, GD and KII respondents were also asked whether they felt that project beneficiaries and those targeted for participation in different project activities were selected equitably and appropriately targeted. This targeting and selection also provides insights into beneficiary perceptions of the suitability of services provided, their perceived quality and effectiveness, and their ultimate views on continued demand for services beyond the project lifetime. Respondents were also asked about the general perceptions of the beneficiary selection process and targeting criteria for participation or selection. On these issues, respondents observed that ViM was widely considered effective, according to GDs and KIIs. Most KII respondents (33 of 35 individuals) believed that targeted ViM beneficiaries were well-chosen, and the majority of GDs thought that assistance was distributed equally amongst program beneficiaries (24 out of 31 GDs). Amid these 70 FINAL REPORT – VICTORY AGAINST MALNUTRITION (ViM) FINAL EVALUATION viewpoints, however, is the assertion that program activities could target other groups as well, such as unemployed youth (five GDs/KIIs), the elderly (three KIIs), those who raised livestock (three KIIs), traditional practitioners of agriculture (one KII), handicapped people (one KII), and religious leaders (one KII). CONCLUSIONS The demand and willingness to pay for essential services provided by the project on the part of project beneficiaries is also tied to beneficiaries’ perceptions of the quality, effectiveness and suitability of those services. Overall, the high number of households who said they planned to continue using ViM￾promoted practices, or would continue to participate in offered trainings offered through farmers groups or Care/Neighborhood Groups suggests a high likelihood for continued demand for ViM￾promoted activities after the project lifetime. This was particularly the case for ViM agricultural and WASH activities, and also links to beneficiary perceptions of the relatively strong effectiveness of these project-promoted activities (see additional discussion of activity effectiveness in EQ1, EQ2 and EQ3). The anticipated post-project source for continued service support or training varied somewhat across sectors / service type, but households most commonly indicated they anticipated this to come from farmers groups (for agricultural support) or Care/Neighborhood Groups (for WASH, nutrition and health-related support), followed by government. However, these results for EQ5 are juxtaposed against that of EQ4, in which there is an indication of a number of institutional and resource challenges for continued service provision on the part of providers, particularly in the absence of ViM or alternative donor-funded support. 4.6 EQ6: EFFECTIVENESS OF THE INTERVENTIONS IMPLEMENTED BY VIM TO STRENGTHEN HOUSEHOLDS’ AND COMMUNITIES’ RESILIENCE TO IDIOSYNCRATIC AND CO-VARIATE SHOCKS? FINDINGS GD results indicate an improved ability of respondents to absorb shocks – a capacity that beneficiaries largely attributed to ViM project interventions, although the evidence for this is somewhat mixed. In GDs, participants mentioned specifically, increased capacity to store crops, which contributes to longer periods of food conservation and contributes to household income smoothing in the face of idiosyncratic or covariate shocks. This conclusion is also supported by the PBS results, in which the percentage of farmers using improved crop storage methods rose by 26 percentage points. Enhanced crop storage, for example, lead to greater food availability for longer periods of the year for households, and also enabled farmers to sell food at more favorable prices, which boosted income in a manner that ultimately strengthens resilience. Beneficiary participants attributed this behavior to ViM training. “Before we could have a bountiful harvest, but conservation was a problem. But currently we manage to keep the crops for two to three years.” [Female, Barsalogo] “There is a place here where, after the harvests, we can deposit millet, beans, sesame and take the money we want until the prices rise to an acceptable level. This works for us because traders plunder us here. They often know that we need money to solve our problems.” [Male, Firka] Beneficiaries also said that the knowledge they acquired through ViM training on vaccination for livestock, information on protection of crops against pests, and using improved seeds has also bolstered resilience against shocks. ViM strengthened the storage capacity for beneficiaries' agricultural products by building 14 storage facilities with a total capacity of 1450 MT, and rehabilitated four warehouses with a total capacity of 200 71 FINAL REPORT – VICTORY AGAINST MALNUTRITION (ViM) FINAL EVALUATION MT. This allowed beneficiaries to do warrantage in order not to sell their production at the time of harvest. However, some participants noted in GDs that they were nevertheless forced to sell their crops at prices that were not ideal, due to pending bills or other unexpected household shocks. Some beneficiaries who had not engaged in warrantage noted they were forced to sell crops at unfavorable times, even though they were aware that prices were too low. “We were shown how to keep crops. For example, between October and November, the price of beans is very low. But where do we find money to pay for this? If you decide to keep your harvest to sell afterwards, who will refund you now? So you are obliged to sell at a very low price which will cause losses. If you do not repay the money on time, the police will pick you up. If ViM can help us in this way it would make us happy.” [Male, Tibi-Yarcé] Training on cowpea storage methods was provided for members of producer groups but not for other beneficiaries of the ViM project who are not members of such groups. Therefore, not all beneficiaries received all training systematically. Although not common, participants in a small number of GDs also noted a mismatch at times, for some of the ViM-provided materials relative to their crop production or storage needs. In this and similar cases raised in some GDs, beneficiaries felt this could stem from a misunderstanding of beneficiary capacity and needs, which they felt could be resolved through greater consultation by ViM staff, as illustrated in the quotes below: “The problem with ViM is that to help us, they do not ask us what we want. They make materials that are not always good. For example, we paid 20,000 francs each for storage cases that did not even serve us. We did not know the techniques and we did not even have the equipment. … the methods that the project taught us were good but in their application, the technicians did not always come to us to have our point of view and suggestions to adapt the tools. They did what they wanted. As a result the storage crates are too big and cannot be used easily at our homes. Besides that, the material used is very fragile. We did not end up using these.” [Male, Nagroago Foulce] “There are tools that they came to give us and that do not serve us because it is not adapted to our situation. But if they asked for our opinion, we can say what we do not need. … They have solved many of our concerns but they do things without [consulting in our community] and that is not perfect. You can have a problem of 2500F and someone comes to give you something worth 10,000F but it does not solve your problem. … Before paying for materials one can consult with us so that we explain what will help us.” [Male, Tibi-Yarcé] Despite these few issues, the qualitative results on household resilience are generally positive, and also suggest more positive effects compared to the available quantitative findings. Overall, participants in eight GDs mentioned a positive effect of the ViM project on their household’s resilience, citing examples such as increased availability of food and less food spoilage over the year due to the crop storage and conservation technicaues they learned, and learning how to obtain market prices and consider price favorability before selling some of their stores in order to increase incomes. Productivity results from the PBS do not provide additional quantitative support for yields increases, likely due to the methods limitations noted in section 3.3.4, however the qualitative data indicate a strong perception of substantially increased yields among respondents. The PBS data do suggest that a number of farmers in the project area have switched their priority from crops that are traditionally intercropped in favor of putting more of their farmland into cash crops, such as cowpeas or sesame. The ViM project has also introduced inputs (e.g. seed and fertilizers) that has reduced the need for thinning. Quantitative results also suggest a reduction in the rates of farmers using veterinary care and complementary feeding and habitat construction for livestock, however it is noted that farmers adopting 72 FINAL REPORT – VICTORY AGAINST MALNUTRITION (ViM) FINAL EVALUATION one or two practices are not counted. In addition, in FY17, there was a breakdown of veterinary and other livestock inputs in ViM intervention areas and KIIs suggested that many breeders did not have access to inputs that would enable application of best practices. In contrast, the quantitative SO2 results do provide some additional quantitative indication of increased household resilience and capacity to absorb shocks. In particular, the prevalence of household hunger declined substantially (though some of this decline could be due to the difference in seasonality between baseline and endline), while the value of household assets and access to formal was significantly higher at endline, and some market linkages were strengthened (for example, greater participation in value chains among households in the project area). Lastly, the significantly improved status of children with respect to stunting and prevalence of underweight children suggest there is likely greater stability of household food security at endline, although it is noted that women’s dietary diversity did not change over the project lifetime. CONCLUSIONS Taken together, the quantitative and qualitative results for the evaluation provide an indication of improved household ability to weather shocks overall, despite that the PBS results for particular indicators, such as crop yields, could not be confirmed. The stronger anecdotal evidence from the qualitative results may stem to some extent from the use of GD questions that were more targeted on these issues than the household survey, by specifically asking participants their views on adjusting to shocks, which allowed for more detailed and nuanced responses. 4.7 EQ7: HOW WELL DID VIM TAKE ADVANTAGE OF THE OTHER USG INVESTMENTS THAT PROVIDE COMPLEMENTARY SERVICES NECESSARY TO ACHIEVE VIM’S GOAL? Desk review of ViM project reports indicates that the project engaged in substantial collaboration with other USG investments, and aimed to leverage those engagements to provide complementary services beyond what the project could likely achieve on its own. This may be particularly the case for the way the project worked with additional USAID-funded programs in later years of the project. The project was able to work under the RISE initiative to help ensure sustained service delivery and overlapping coverage in capacity building to try to ensure continued achievements of ViM goals beyond the project lifetime. Examples include: • Collaboration with REGIS-ER in the training of local implementing partner field officers (Quarterly Report, FY16, Q3), under the RISE collaboration. • Engaging in collaborative partnerships with REGIS-AG and REGIS-ER to enhance market linkages, and organize trade fairs between producers and potential buyers for cowpea, poultry and small ruminant markets (FY16, Q4). • Taking advantage of training and workshops offered by other USG investments, such ViM staff participating in training of trainers workshops offered by SPRING, to learn how to more effectively use produce community videos for use as a behavior change tool in communities (FY17, Q1 and ongoing afterwards). • Procurement and dissemination of small ruminant and poultry health care tool kits to community livestock auxiliaries, undertaken together with REGIS-AG (FY17, Q1). • Videos created in collaboration with SPRING, for example on handwashing and complementary feeding, which also showcased men as important behavior change agents (Quarterly Report Fy17, Q2). 73 FINAL REPORT – VICTORY AGAINST MALNUTRITION (ViM) FINAL EVALUATION • Collaborative support for warrantage activities by the cowpea producers’ union in Barsalogho community, together with REGIS-AG and REGIS-ER (FY17, Q2). Other examples, in which the project leveraged non-USG collaborations include that ViM undertook collaborative vaccination campaigns and worked to reinforce linkages between livestock producers and groups within both the public and private sectors (Quarterly Report FY17, Q3). In addition, ViM created producer links with public and private suppliers through its voucher program, and supported local improved seeds growers to help ensure continued availability and access of quality improved seeds to meet producers’ needs (Quarterly Report FY17, Q3). Lastly, it is noted that in ViM’s exit strategy the project aimed to work with several RISE actors, particularly REGIS-ER and REGIS-AG, to transfer all agricultural and livestock production support activities to the regional and provincial directorates before project close-out and the final quarter of FY18 (see EQ9 for additional discussion). In FY18, ViM also aimed to increase joint field visits with the REGIS field officers, aiming at a progressive transfer of some activities and support to service providers such as lead farmers, VVVs, and local seed growers. Moreover, in close collaboration with the FASO program and REGIS-ER, ViM planned to recommend that the government include care groups promoted by the RISE actors into its new health and nutrition policy through the GASPA (PREP, IY7). Thus, the evaluation finds evidence that both USG and non-USG collaborating partners have been engaged by the project to take on a range of collaborative training and capacity building services at community and different levels of government, and this appears to have been done across the project’s MCHN, WASH, agriculture, and livelihoods sectors. 4.8 EQ8: HOW EFFECTIVE ARE THE APPROACHES USED BY VIM PROJECT TO ADDRESS GENDER GAPS AND EMPOWER WOMEN? FINDINGS The ViM project has facilitated positive changes in gender norms and practices. Both male and female GD participants noticed that male beneficiaries are more supportive of taking pregnant women and children to the health center, nutrition for women and children, and family planning and birth spacing issues. “There have been positive changes in the attitudes of husbands in supporting women in pregnancy, food and family planning. MLA have played a big role at this level. They made us husbands realize that we have to help our wives. Even those who were reluctant and did not let their wives out had to let them join other women. There are even men who accompany their wives to the hospital and this was not done before. Before, women had to manage their pregnancies until giving birth but this is no longer the case. Now the support depends on the means of each man, because there are people who mean well but do not have the money.” [Male GD, Firka] “Thanks to the various trainings, women do not give birth at home anymore. Family planning is well respected and it helps the households a lot. Thanks to birth spacing, the mothers are more and more available for the domestic and field jobs.” [Female GD, Basma] “The positive changes are mainly the change of attitude of our husbands, who were initially reluctant. Now it is them who are giving us advice. Family planning makes it easier to take care of children.” [Female GD, Sangro] Importantly, quantitative results are consistent with these favorable qualitative assessments. In a statistically significant change since the project’s inception, about 70% of currently married women 74 FINAL REPORT – VICTORY AGAINST MALNUTRITION (ViM) FINAL EVALUATION (72.4% up from 62.4%), ages 15 to 49 years old, now report that they make decisions either by themselves or jointly with their spouses on seeking health services for her own health; seeking health services for her children's health; and on how to spend money she has earned. Survey findings are even more dramatic on a measure of normative change: a large majority (91.0%) of women surveyed now disagree that it is acceptable for a man to hit his wife under one or more circumstances (down 40 percentage points from 50.6%), in another statistically signification change in findings. In addition, 91.4% of Mother Leader Animatrices believe they are more respected after participating in the ViM project. Sex-disaggregated indicator results also provide a window into ViM’s overall acheivements with respect to gender integration. While male farmers typically had higher values than female farmers for several of the individual-level indicators assessed for this evaluation (see results presented for EQ4, and Table 3 of Annex II), there are several indicators for which female farmers improved from baseline to endline either by a similar amount, relative to their male counterparts, or by a greater amount during the project lifetime. This was the case, for example, for the percentage of female farmers who use improved storage techniques for cowpeas (in which the increase for female farmers from baseline to endline was 13 percentage points greater than for male farmers and substantially closed the gap on this across male and female farmers that had been present at baseline), the percentage of female farmers who practiced value chain activities promoted by the project (in which the percentage of female and male farmers engaging in this increased by similar amounts across the project lifetime), and the percentage of farmers who used financial services in the most recent season (in which the gain realized by female farmers over the project lifetime was also 13 percentage points higher than for men and brought female farmers on par with men by endline) (see Table 4 in Annex II for additional details). ECONOMETRIC ANALYSIS While comparison of baseline and endline data indicated that there were significant improvements in indicators of women’s empowerment at endline, analysis of endline data indicated that ViM interventions were not as effective in improving women’s empowerment. The ViM project attempted to improve women’s empowerment through administering gender trainings among the beneficiaries. To assess whether trainings affected empowerment, endline survey collected data on women’s participation in household decision making. Econometric Model: We used multivariate regression techniques to test whether participation in ViM gender trainings affected women’s participation in household decision making controlling for other covariates that typically influence women’s participation as described in Table 23.1 Annex VI. We used instrumental variable method to control for the self-selection in participation (or targeting) in gender training, and controlled for households and individual level covariates that affect women participation in decision making. It is important to note that the econometric analysis utilized here has some limitations, as the quantitative endline survey did not collect behavioral data from women, and thus the models used to estimate the effect utilized only economic and demographic data. Due to this, the estimated results could also be influenced by some unobserved omitted variables that the evaluatin team is not able to control for in these models. Results presented in Table 13 (based on the regression results reported in Table 23.2 in Annex VI) indicate that women from households that participated in ViM gender training were more involved in household decision making processes. In addition to considering whether gender training had any effect, we also explored whether participation in the neighborhood and MLA groups had any influence on women’spariticapation in decision making. However, results presented in Table 13 (based on Table 23.3 in Annex VI) indicate that 75 FINAL REPORT – VICTORY AGAINST MALNUTRITION (ViM) FINAL EVALUATION participation in neighborhood/MLA groups was not correlated with women participation in household decision making. Table 13: Effect of ViM Interventions on Women Empowerment Level of Analysis Outcome/Impact Indicators Indicator of ViM Intervention Multivariate Regression Results Interpretation Role of ViM in improving Women empowerment Woman Women participation in household decision making (either by herself or jointly with her spouse) Household members received ViM gender training Positive effect Women from households that received ViM gender training were more likely to participate in household decision making Woman Women participation in household decisions making (either by herself or jointly with her spouse) Household members’ participation in MLA/ neighborhood groups No effect Household members’ participation in MLA/neighborhood groups was not correlated with women participation in decision making CONCLUSIONS The quantitative data provide some evidence of the effectiveness of ViM project on gender, as results find support for ViM effects on women’s participation in household decision-making (as an indicator of women’s empowerment). The complementary qualitative data provide some additional depth of understanding on this, suggesting that ViM activities have improved women’s empowerment, their access to and control over productive resources, and increased decision-making over a range of issues and financial resources. Moreover, the quantitative results indicate that ViM activities clearly contributed to increasing the agency of women with respect to their own health, children’s health, and productive resources. ViM activities that centered on the promotion of women as leaders, the economic empowerment of women, and the sensitization of men are likely key contributors to this shift. Lastly, qualitative results provide strong evidence that Care/Neighborhood Groups were extremely important to ViM activities in mobilizing community members, promoting improved MCHN practices, and sensitizing men and women on a range of important issues, including strengthening women’s empowerment and changing deleterious attitudes and behaviors among men. 4.9 EQ9: PARTNERSHIPS AND EXIT STRATEGY: HOW EFFECTIVE WERE THE PARTNERSHIPS DEVELOPED WITH GOBF AT NATIONAL, COMMUNE, AND VILLAGE LEVEL AS WELL AS OTHER DEVELOPMENT ACTORS IN DELIVERING INTENDED RESULTS ALIGNED WITH COUNTRY DEVELOPMENT OBJECTIVES? WAS THERE A WELL-DEVELOPED EXIT STRATEGY EFFECTIVELY COMMUNICATED? FINDINGS Analysis of partnerships and sustainability draws from the Sustainability and Exit Strategies Conceptual Framework that was developed under Food for Peace’s Food and Nutrition Technical Assistance III 76 FINAL REPORT – VICTORY AGAINST MALNUTRITION (ViM) FINAL EVALUATION Project (FANTA)68. Findings and conclusions derive from four key components or pillars of the framework that are examined throughout this report: • Sustained source of resources (EQ4) • Sustained technical and managerial capacity (EQ4) • Sustained motivation of beneficiaries and service provider (EQ4 and EQ5) • Linkages to governmental organizations and/or other entities (EQ7 and EQ9) To address partnership effectiveness, the evaluation examines what vertical linkages have been created or leveraged to augment access to resources, strengthen capacity, and motivate beneficiaries and service providers to demand, deliver, and use products and services in a sustainable manner. In addition, issues related to capacity, motivation, resources, and institutional linkages were explored through GD and KII sessions and were presented in previous sections of this report (EQ4 and EQ5). Evaluation findings on the technical and managerial capacity of project beneficiaries and service providers was also discussed under EQ4. Most importantly, as noted in those sections, their ability to apply gained knowledge and skills after the ViM project lifetime is highly dependent on their capacity to leverage resources externally through local government services and/or other relevant actors. The motivation of beneficiaries to demand services, their ability and willingness to pay, and the motivation of service providers to sustain the provision of resources was also discussed previously in this report under EQ4 and EQ5. According to qualitative results from KIIs, ViM activities could be continued independently of USAID supporting. However, several respondents added that either activity continuation would require increased or repeated trainings, partner organizations would have to keep the equipment they have received from the project, or that organizations would need to seek out other donors. “We especially want training because it is more beneficial than money. Knowledge is the greatest wealth. If we are given money, we will finish spending without doing anything, but it is training, we learn something that does not end.” – ABSC Delga, KII Among those in-depth interviews in which respondents were less convinced of outcome sustainability, several stated a lack of alternatives for funding as the reason for their concern. Others observed that partner organizations lack sufficient training, and that institutional knowledge was too limited for sustainability. Exit Strategy Building sustained technical and managerial capacity, through group as well as more individualized approaches, is intended to be carried out in phases as elaborated in ViM’s Sustainability Plan and Exit Strategy. An exit strategy generally entails three phases: 1. Phase down – progressive reduction in program resources and support; 2. Phase over – transfer of responsibility to local, permanent institutions; and 3. Phase out – program interventions will end. The ViM project was in the process of phasing down and building ownership and capacity among partners for phasing over the transfer of responsibilities at the time of the data collection for the endline evaluation. ViM’s FY17 PREP (PREP 7) and FY17 quarterly reports also provide an indication of the substantial extent to which ViM has conceptualized and engaged in planning and implementing its exit 68 Rogers, Beatrice Lorge and Coates, Jennifer. 2016. Sustaining Development: A Synthesis of Results from a Four￾Country Study of Sustainability and Exit Strategies among Development Food Assistance Projects-Executive Summary. Washington, DC: FHI 360/Food and Nutrition Technical Assistance III Project (FANTA). 77 FINAL REPORT – VICTORY AGAINST MALNUTRITION (ViM) FINAL EVALUATION strategy. This planning includes organizing workshops and trainings across multiple levels of partnerships, such as with community-based groups, commune and district-level authorities, commune mayors and head nurses from primary care centers (Quarterly Report FY17, Q3). The project reports indicate that the process of reinforcing community ownership of ViM activities started in implementation year (IY) five and was planned to continue throughout 2018. In FY17, ViM conducted one-day meetings for project stakeholders at the community level, including VHNCs, MLAs, WASH committees, representatives from local government, and members of the committees formed for managing institutional latrines. Other participants included health personnel, representatives of village chiefs and leaders, technicians from applicable ministries, and USAID-funded RISE initiative project staff. During these meetings, ViM collected suggestions from the staff of public services and NGOs, and related stakeholders about how to continue collaborating and supporting community groups in order to ensure activities continue beyond the end of the project. A total of 120 meetings were planned (PREP 7), and it appeared that ViM achieved this target. For example, the project reported that health and nutrition promoters held monthly meetings with 111 village health and nutrition committees during the second quarter of FY17, to provide technical support for the proper implementation of their action plan activities (Quarterly Report FY17, Q2). ViM’s exit strategy provided specific attention to bolstering the capacity of a variety of project stakeholders, with the aim of ensuring that they have the necessary technical and managerial skills to provide continued services to producers for sustained improvement in food production, income generation, health, and nutrition. The key handover action that ViM reported conducting to reinforce capacity for continued service provision included ongoing reinforcement of the GoBF’s agencies and service providers’ operating capacity in the project area. ViM also aims to boost the involvement of influential community members, such as grandmothers and fathers, in project nutrition activities. For example, in every project village, ViM promoters aimed to organize three meetings with groups of grandmothers and three with groups of fathers (PREP 7). As also noted in EQ7, ViM exit strategy entails continued work with RISE actors, particularly REGIS-ER and REGIS-AG, to transfer all agricultural and livestock production support activities to the regional and provincial directorates before the final quarter of FY18. Through the partnership agreements signed with these regional and provincial directorate agencies and service providers, ViM aimed to continue to reinforce the skills and capabilities of their field officers to provide appropriate and complementary training. In FY18, ViM also aimed to increase joint field visits with the REGIS field officers, aiming at a progressive transfer of some activities and support to service providers such as lead farmers, VVVs, and local seed growers. Moreover, in close collaboration with the FASO program and REGIS-ER, ViM planned to recommend that the government include care groups promoted by the RISE actors into its new health and nutrition policy through the GASPA (PREP IY7). Lastly, to sustain health and nutrition activities at the village level, ViM worked to ensure that its care groups were integrated into the Burkina Faso GASPA policy, so that the data from their activities can be collected by the national health data system. It is planned that health and nutrition promoters will work with CHAs and MLAs to discuss and create action plans to continue screenings after project closeout. CHAs will then share these action plans with health facility staff to encourage MLAs to continue their good work and ensure accountability at the community level. ViM also is working with the Ministry of Water and Sanitation for the recognition and integration of the WASH committees in their community￾based approaches69 (PREP IY7). 69 However, ViM communicated to the ET that this will no longer take place, as apparently the activity ultimately did not receive donor approval. 78 FINAL REPORT – VICTORY AGAINST MALNUTRITION (ViM) FINAL EVALUATION The time frame over which ViM has planned to transfer project responsibilities appears to be reasonable, however it is noted that despite these efforts, at the time of qualitative data collection in early 2018, KIIs with local government and technical staff indicated that additional training and capacity building was still desired before at least some of the targeted partners would feel confident in their ability to continue service delivery and support to sustain the impacts achieved under ViM. 4.10 EQ10: UNINTENDED CONSEQUENCES70 AND LESSONS LEARNED: WHAT ARE THE UNINTENDED POSITIVE AND/OR NEGATIVE CONSEQUENCES OF THE VIM PROJECT? WHAT LESSONS CAN BE LEARNED TO MINIMIZE UNINTENDED NEGATIVE CONSEQUENCES IN THE DESIGN OF FUTURE PROJECTS? FINDINGS A small number of positive and negative unintended and unanticipated consequences were identified during the qualitative data collection, and these arose primarily through KII meetings with collaborating partners and service providers. The analyses in this section focuses on major unintended consequences, rather than smaller issues that appeared to have had only minor bearing on project activities or post￾project sustainability. Where available, strategies or corrective measures that were taken to address negative consequences and adjust implementation, management, monitoring, or other related areas are also discussed. A major positive and unintended consequence of ViM identified via the qualitative data collection relates to the potential to use ViM’s SBCC approach to leverage additional social and behavior change with other initiatives operating in the province, • Synergistic Behavior Change and Leveraging Opportunities through Organized Peer Groups: The ViM project has used an organized and effective approach to elicit behavior change on important MCHN and WASH issues, through the use of peer-led learning groups and networks (for example, the Care/Neighborhood Groups). This approach, and the existence of the ViM-organized peer network, can be leveraged to provide synergies with other initiatives in the four communes of Sanmatenga province. For example, this leveraging can provide a forum to dialogue on local causes of malnutrition and how these can be overcome, further reinforce key messages on nutrition and childcare, and provide additional synergies with a range of related development objectives. Qualitative results highlight three key areas of constraints that contributed to negative unanticipated consequences or likely constrained higher potential achievement of some project objectives: • Lack of Water: Participants in 16 GDs reported that a lack of water hindered their engagement in ViM activities, and they noted this with respect to both agricultural and WASH programs. The contributions of insufficient water access and infrastructure on the perceived effectiveness of some of ViM’s agricultural activities is discussed under EQ2. The quote below illustrates additional perceptions by ViM beneficiaries of its links to achieving WASH objectives. Despite that assistance on water activities was outside of ViM’s scope, the attention to this issue from the ViM MTE and at this endline evaluation underscore that in future, program design for 70 This section addresses “unanticipated” as well as “unintended” consequences. The former refers to project under- or over-performance due to unforeseen circumstances. The latter refers to actual outcomes that were unexpected. 79 FINAL REPORT – VICTORY AGAINST MALNUTRITION (ViM) FINAL EVALUATION similarly conceptualized projects may benefit from consideration of how water assistance may be more directly incorporated into project strategies, to help reduce a key barrier to the attainment of integrated agriculture, health and nutrition project objectives in Burkina Faso. “We cannot practice sanitation due to a lack of water. They tell us to be clean but we do not even have water. We thought they were going to help us with that. But we are still waiting. Our seven main neighborhoods do not even have water. For example, in this area there is no borehole except for the pump at the school but everyone can not draw water with the students! More drilling is needed.” [Nagraogo Foulbe – Men] • Inadequate market access: An inadequate access to markets was reported as an unresolved issue by participants in four GDs. “Our challenges were the lack of adequate markets. They offer very low prices, even similar to a donation during the harvest period. But we have no choice because it is the period where we have to pay the school fees of children.” [Gabou- Women] • Exclusive breastfeeding challenges: In qualitative follow-ups on EBF, some women noted that they had difficulty breastfeeding during the day because they were doing work outside of the home. While EBF did increase substantially over the project lifetime, at endline the EBF prevalence was still a somewhat moderate 60.3%. This type of external challenge on exclusive breastfeeding likely contributes to the gap in EBF among women in the project area, and may suggest space for future projects to incorporate knowledge of the particular constraints on EBF among beneficiary populations into targeted messaging and technical strategies. CONCLUSIONS Overall, the evaluation found few strongly negative unintended consequences or barriers that substantially affected the achievement of ViM’s intended outcomes. Those that are noted above may provide a useful indication of key areas that may benefit from additional consideration during future project design stages, including a constraints analysis and alternative strategies that similar projects may use in future to better address them. 4.11 EQ 11: HOW EFFECTIVE ARE THE VALUE CHAIN APPROACHES USED BY VIM PROJECT TO PROMOTE COWPEAS AND SMALL LIVESTOCK VALUE CHAIN IN THE TARGET COMMUNITIES? This section addresses the effectiveness of the value chain approaches used by ViM to promote cowpeas and small livestock value chain activities71 in the targeted communities. Quantitative and qualitative data is used to evaluate outcomes related to the value chain approach between VIM beneficiaries and non￾beneficiaries, but, as mentioned for EQ2, a robust assessment of the effectiveness of the VIM project requires attribution of the differences in outcomes to the VIM project. The limitations on establishing attribution, as noted earlier, including that survey questions on value chains did not ask where the knowledge and behaviors were learned, the inability to use a quasi-experimental approach for this evaluation, and contamination risk. The phrasing of the household survey question on value chains was: “Did you practice any of the following value chain activities directly related to your cowpea production during the most recent agricultural year?” The question does not ask if the listed activities were practiced through ViM farmer groups or support, so it 71 See Annex II for the full list of activities considered value activities. 80 FINAL REPORT – VICTORY AGAINST MALNUTRITION (ViM) FINAL EVALUATION is possible surveyed farmers may have practiced these activities (joint purchase of inputs, bulk sale through farmers’ groups, or bulk transport through farmers’ groups) before ViM began. Note that these three activities relate more to aggregation to the farmer and farmer group level, rather than establishing relationships with buyers and input providers that would also be core to a value chain approach. Although the endline survey sample did include non-beneficiaries, both beneficiary and non-beneficiary farmers may reside in the same village. Thus, for the cowpea and livestock value chain activities, non￾beneficiary farmers living in the project villages are able to and may have purchased inputs or sold or transported cowpeas through a ViM-sponsored farmers’ group. In the context of this report, it will be assumed that any beneficiary cowpea or livestock value chain farmer that reported practicing any of the cowpea or livestock value chain activities did so through ViM farmer groups support process. It will also be assumed that non-beneficiary cowpea or livestock value chain farmer that reported practicing any of the cowpea or livestock value chain activities did so through groups or processes that were independent of ViM. This assumption is necessary in order to disaggregate the data so that it is possible to compare the effectiveness of the cowpea and livestock value chain activities supported by ViM to those not supported by ViM. Cowpea Value Chain Activities ViM was effective in promoting the cowpea value chain activities; 82.7% of farmers (717 of 867) who grew cowpeas and were ViM beneficiaries used cowpea value chain activities, compared to 64.2% of non-beneficiary (1,369 of 2,131) cowpea farmers (Table 11). There were 2,086 surveyed cowpea value chain farmers (69.6% of cowpea farmers), which is any farmer who grew cowpeas and practiced at least one value chain activity on cowpeas. Of the 2,086 farmers, 52.3% were male and 47.6% were female. Table 14: Cowpea Value Chain Activity Rates Is the Farmer a “Cowpea Value Chain Farmer?1 ViM Beneficiary (Number) ViM Non￾Beneficiary (Number) Total Yes 717 1,369 2,086 (69.6%) No 150 762 912 (30.4%) Total 867 (28.9%) 2,131 (71.1%) 2,998 Beneficiary / Non-Beneficiary Comparisons This section will compare the beneficiary and non-beneficiary farmers for the adoption and effectiveness of cowpea value chain activities. Four of the value chain activities correspond to a follow-up question about perceptions of the activity; joint purchase of inputs, bulk sale through farmer’s groups, bulk transport through farmer’s groups, and marketing skills. Each of these activities are broken up between beneficiaries and non-beneficiaries to see if beneficiaries were better off (Table 12). Table 15: Comparison of Cowpea Value Chain Responses for Beneficiaries and Non-Beneficiaries Cowpea Value Chain Activity and Responses Indicator Values Differences Beneficiary Non￾Beneficiary Difference Difference Direction Cowpea Value Chain Activity: Joint Purchase of Inputs Practice of cowpeas value chain activities improved: access to inputs 84.9% 46.4% 38.4% Beneficiary Cowpea Value Chain Activity: Bulk Sale Through Farmer’s Groups 81 FINAL REPORT – VICTORY AGAINST MALNUTRITION (ViM) FINAL EVALUATION Cowpea Value Chain Activity and Responses Indicator Values Differences Beneficiary Non￾Beneficiary Difference Difference Direction Practice of cowpeas value chain activities improved: Access to market information 82.8% 60.0% 22.8% Beneficiary Cowpea Value Chain Activity: Bulk Transport Through Farmers’ Groups Major constraints in the marketing of cowpeas: Remote sales center 58.8% 62.5% -3.7% Beneficiary Majors constraints in the marketing of cowpeas: Bad road condition 58.8% 75.0% -16.2% Beneficiary Cowpea Value Chain Activity: Marketing skills Practice of cowpeas value chain activities improved: Access to market information 97.3% 97.6% -0.3% N/A For cowpea farmers who practiced joint purchase of inputs, 84.9% of beneficiaries said that they had improved access to inputs compared to 46.4% of non-beneficiaries. Beneficiaries who practiced bulk sale through farmer’s groups were more likely to say they had improved access to market information (82.8%) than non-beneficiaries (60.0%). Additionally, beneficiaries who practiced bulk sale through farmer’s groups were less likely to see remote sales centers or bad road conditions as a constraint on marketing cowpeas than non-beneficiaries. Finally, there was no difference between beneficiaries and non-beneficiaries who practiced marketing skills in terms of seeing improved access to market information, which was around 97% for both. The other source of information regarding the effectiveness of ViM cowpea value chain activities comes from the KII of the Cowpea Union Barsalogo. The respondent noted that the only trainings beneficiaries found effective specific to cowpea production dealt with treatment products in which training farmers learned that there are two treatment products for cowpeas at each stage (systemic product and contact product). However, there was no mention made of the cowpea value chain activities or their effectiveness. Livestock Value Chain Activities Table 16: Livestock Value Chain Activity Rates Is the Farmer a “Livestock Value Chain Farmer?72 ViM Beneficiary (Number) ViM Non￾Beneficiary (Number) Total Yes 259 1,339 1,598 (42.6%) No 94 1,094 1,188 (57.4%) Total 353 (12.7%) 2,433 (87.3%) 2,786 ViM was effective in promoting the livestock value chain activities; 73.4% of farmers (259 out of 353) who own livestock and were ViM beneficiaries used livestock value chains, compared to 55.0% of non￾beneficiary (1,339 out of 2,433) livestock owners (Table 13). There were 1,598 surveyed livestock value chain farmers (42.6% of livestock farmers), which is any farmer who owned livestock and practiced at least one value chain activity on livestock. Of these 1,598 farmers, 65.3% were male and 34.7% were female. 72 The operational definition in this report of a “livestock value chain farmer” is a surveyed farmer that practices at least one of the listed livestock value chain activities. 82 FINAL REPORT – VICTORY AGAINST MALNUTRITION (ViM) FINAL EVALUATION Beneficiary / Non-Beneficiary Comparisons The comparison of beneficiary/non-beneficiary results for livestock follow the same methodology as were used for the cowpea analysis above. There were two livestock value chain activities that correspond to a follow up question about perceptions of the activity; joint purchase of vaccinations and marketing skills. Each of these activities are broken up between beneficiaries and non-beneficiaries to see if beneficiaries were better off (Table 14). Table 17: Comparison of Livestock Value Chain Responses for Beneficiaries and Non￾Beneficiaries Livestock Value Chain Activity and Responses Indicator Values Difference Metrics Beneficiary Non￾Beneficiary Difference Difference Direction Livestock Value Chain Activity: Joint Purchase of Vaccinations In the most recent agricultural year, have any of your animals obtained any vaccinations? 92.5% 84.3% 8.2% Beneficiary If vaccinations were obtained for animals: Vaccinations were obtained from: NGO 15.3% 8.4% 6.9% Beneficiary Vaccinations were obtained from: CBO 14.3% 20.6% -6.3% Non-Beneficiary Vaccinations were obtained from: Private organization 1.0% 7.8% -6.8% Non-Beneficiary Livestock Value Chain Activity: Bulk Sale Through Farmer’s Groups The main destination for the livestock products sale: Regional market 28.6% 24.0% 4.6% Beneficiary The main destination for the livestock products sale: Group sale 0.0% 3.0% -3.0% Non-Beneficiary Table 15 shows that a higher percentage of beneficiaries got at least some animals vaccinated as compared to non-beneficiaries (92.5% vs 84.3%). However, when the sources of the vaccinations are considered, the beneficiaries had a higher rate of getting vaccinations from NGOs compared to the non￾beneficiaries who had higher rates of getting animal vaccinations from CBOs or private organizations. The questions were not phrased to be able to determine if the vaccinations obtained from these sources were bulk vaccinations or not, so the effectiveness of ViM in promoting this specific livestock value chain activity cannot be determined. For surveyed farmers that reported practicing bulk sales of animals through farmer’s groups, follow-up questions asked about the destination of the livestock sales and not whether these were bulk sales or not. It is not clear whether or not villages or local markets would support bulk sales of animals, so it is not possible to comment about these sales destinations. Regional markets might support such sales, in which case a somewhat higher percentage of ViM beneficiary livestock value chain famers used these venues for sales as compared to non-beneficiary livestock value chain (28.6% vs 24.0%). When group sales are explicitly mentioned none of the ViM beneficiary livestock value chain famers used this venue, compared to 3.0% of their non-beneficiary counterparts. The KII and GD data do not bring additional 83 FINAL REPORT – VICTORY AGAINST MALNUTRITION (ViM) FINAL EVALUATION insights to bear on these issues, and the quantitative data is unclear in determining the effects with the data available. 5. RECOMMENDATIONS Synthesis of the evaluation team’s findings and key conclusions leads to the following recommendations, centered on overall program design and sector-specific recommendations. 5.1SECTOR-SPECIFIC RECOMMENDATIONS: MCHN AND WASH The ViM project has contributed substantial efforts that addressed both immediate and underlying causes of malnutrition in the project area’s population of children under five years of age. This is notable, given recognized complexities to implementing effective integrated child health and nutrition programs, and important knowledge gaps on evidence-based interventions and resources required to implement them successfully or at scale, best practices on most effective delivery mechanisms, and most effective ways to couple or integrate nutrition programming with supporting interventions in agriculture, WASH, governance and social protection sectors73. The ViM project implemented several best-practices activities that reflect the current evidence base for effective woman and child health and nutrition programming74, including support for or the use of multiple delivery platforms (for example making use of market-driven service provisionining and layered delivery platforms at community, commune and broader levels) across a fairly comprehensive package of high-impact interventions. Given that this groundwork has been laid, recommendations for future programming emphasize developing and further strengthening longer term strategies to enhance the provision of basic health services, together with a focus on enhancing sustainable strategies for livelihood support and social protection mechanisms. Specific recommendations are elaborated below. Immediate / short term interventions • Consider establishing and strengthening a provincial level forum to systematically guide, support and monitor community-based nutrition programs. Establishment of a community based nutrition program, which has already been piloted by the ViM project in 4 communes, would enable the government and development partners to continue to address the challenge of malnutrition in all communes of Sanmatenga province meaningfully. Supporting nutrition-sensitive interventions such as those within agriculture, food security, outreach immunization, nutrition and health education on hygiene, water and sanitation and livelihoods sectorcould then be integrated into this overarching platform for sustainable nutrition interventions. As part of this process, strengthening of a provincial multi￾sectoral forum to develop a province-focused strategy for eradication of malnutrition and to provide oversight in implementation of nutrition and linked nutrition-sensitive interventions is also recommended. As piloted by the ViM project, there will be a need for continued monitoring of both facility and community based interventions to track progress. • Continue to support and strengthen programs and strategies currently addressing infant and young child nutrition, with a view to building sustained support for the protection, promotion, and support of optimal 73 Shekar, M., et al 2017. An Investment Framework for Nutrition: Reaching the Global Targets for Stunting, Anemia, Breastfeeding, and Wasting. World Bank Group, Washington DC. 74 For example, see Eozenou, P. H, and M. Shekar 2017. Stunting Reduction in Sub-Saharan Africa. World Bank Group, Washington DC. 84 FINAL REPORT – VICTORY AGAINST MALNUTRITION (ViM) FINAL EVALUATION infant and young child feeding practices. This could be addressed through regular, frequent and updated trainings of community volunteers to counsel and provide on-going support to caregivers. Mothers should continue to be trained in preparation of nutritious foods for infants and young children using foods that are locally available. Community based volunteers should be trained to conduct regular screening and growth monitoring and to make referrals of malnourished children for follow-up at local health facilities. • Stengthen behavior change communication messages targeting mothers and caregivers to highlight and address key barriers to appropriate feeding practices identified through the ViM baseline and endline evaluation surveys. Those barriers include: pre-lacteal feeds, early introduction of supplementary foods, low frequency of feeding during the day and low variety (diversity) of foods fed to young children. Delivery of key nutrition messages above could use the following channels: community￾based promotion through mother-leaders, community health animators, peer-led mother support groups, VVV, CDV, other community groups/meetings and religious forums. Facility-based promotion would include delivery of context and culturally specific IYCN (infant and young child nutrition) messages. Medium / longer-term interventions • To improve feeding practices, underlying cultural factors and beliefs hindering uptake of recommended feeding behaviors identified through the ViM evaluation endline survey will need to be addressed. Engaging influential household members such as grandmothers and grandfathers is strongly recommended. • Improved coverage for child health programs is recommended, especially for Vitamin A supplementation and de-worming. This is feasible through strengthening of the outreach component, ensuring that these are distributed concurrently during vaccination campaigns. 5.2 SECTOR-SPECIFIC RECOMMENDATIONS: AGRICULTURE AND LIVELIHOODS Evaluation results shows that ViM was effective in promoting cowpea and livestock value chain activities, and cowpea farmer beneficiaries who practiced bulk sales through organized farmer’s groups appear to have been more likely to have had improved access to market information. The evaluation team’s analyses also suggest that beneficiaries who received/attended ViM promoted GAP training or served by extension service providers, produce greater number of crops/livestock products on average compared those who did not participate in such trainings. Similarly, those who participated in GAP training, enjoyed a higher yields of cowpeas, however, there was no such relation with yields of sorghum. This may suggest a stronger and continued focus on cowpea value chains relative to other commodities in future programming, but such results can also be sensitive to specific context and market factors present in a given program time frame and thus new programs should conduct and factor their own up￾to-date crop-specific market and value chain analyses into crop-specific decisions for future programming. ViM trainings were also found to have a positive effect in increasing a household’s domestic and productive asset values. The overall conclusion is that the ViM project has been effective in promoting smallholders’ access, affordability and use of agriculture and livestock extension services, and of project-sponsored high quality agricultural and livestock inputs including veterinary services. However, many of these gains detected through supplemental econometric analyses by the evaluation team are not reflected in the PBS-based baseline to endline analyses, which may indicate that many of the positive agricultural and livelihoods benefits in the project’s intervention area are more restricted to direct participants in ViM’s programs, while not yet achieving widespread spillover to the intervention area population as a whole. 85 FINAL REPORT – VICTORY AGAINST MALNUTRITION (ViM) FINAL EVALUATION Based on this, the key recommendation for future programming within agricultural and livelihoods spheres is to consider building program strategies for faster scale-up and reaching a greater number of beneficiaries within project timeframes, and/or to ensure that training and service provision knowledge, managerial capacity, and adequate resource transfer to local institutions has been substantially achieved prior to the end of project close￾out, such that continued support to existing beneficiaries and expansion of benefits to additional populations is ensured beyond the project lifetime.