DISCLAIMER The author’s views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. YAAJEENDE FINAL IMPACT EVALUATION REPORT An Impact Evaluation of the Yaajeende Nutrition-Led Agriculture Program in Senegal (2011-2017) YAAJEENDE FINAL IMPACT EVALUATION REPORT An Impact Evaluation of the Yaajeende Nutrition-Led Agriculture Program in Senegal (2011-2017) September 22, 2018 Contracted under AID-685-C-15-00003 Prepared by: Dr. Lauren Persha, NORC at the University of Chicago Gregory Haugan, NORC at the University of Chicago Qualitative and quantitative research support provided by: Souleymane Barry, Senegal Monitoring and Evaluation Project Mawadda Damon-Gartner, NORC at the University of Chicago Samantha Downey, NORC at the University of Chicago Alexandre Monnard, NORC at the University of Chicago May Nourredine, NORC at the University of Chicago Stacy Pancratz, NORC at the University of Chicago Ilse Paniagua, NORC at the University of Chicago Submitted by: NORC at the University of Chicago for Management Systems International, a Tetra Tech Company Cover Photo: Yaajeende Service Provider Agent in Matam who runs an agricultural products store which he started through the support of the project. Photo Credit: Souleymane Barry DISCLAIMER The authors’ views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. i | YAAJEENDE FINAL IMPACT EVALUATION REPORT USAID.GOV CONTENTS EXECUTIVE SUMMARY...................................................................................................................... V INTRODUCTION .................................................................................................................................1 COUNTRY AND SECTOR BACKGROUND................................................................................1 FINAL EVALUATION PURPOSE, AUDIENCE AND INTENDED USES .................................8 EVALUATION QUESTIONS...............................................................................................................9 METHODS AND LIMITATIONS........................................................................................................9 FINDINGS............................................................................................................................................. 22 CONCLUSIONS AND RECOMMENDATIONS ........................................................................ 73 ANNEXES ............................................................................................................................................. 81 LIST OF TABLES TABLE 1. YAAJEENDE IMPLEMENTATION ACTIVITIES IN PROJECT YEAR 1. .................6 TABLE 2: YAAJEENDE ENDLINE IMPACT EVALUATION: IMPACT/OUTCOME MEASURES ................................................................................................................................................................. 11 TABLE 3: YAAJEENDE QUANTITATIVE EVALUATION SAMPLE AT BASELINE, MIDLINE AND ENDLINE.................................................................................................................................... 18 TABLE 4. QUALITATIVE SAMPLE TARGETS AT ENDLINE: YAAJEENDE TREATMENT VILLAGES .............................................................................................................................................. 21 TABLE 5. SUMMARY TABLE OF EVIDENCE FOR YAAJEENDE PROGRAM IMPACTS . 25 TABLE 6. OUTCOME FAMILY 1 VILLAGE FIXED EFFECTS DID RESULTS: INDIVIDUAL￾LEVEL OUTCOMES, ML-EL.............................................................................................................. 28 TABLE 7. OUTCOME FAMILY 1 VILLAGE FIXED EFFECTS DID RESULTS: INDIVIDUAL￾LEVEL OUTCOMES, ML-EL, VILLAGE-LEVEL TREATMENT INTENSITY EFFECTS......... 35 TABLE 8.OUTCOME FAMILY 2 VILLAGE FIXED EFFECTS DID RESULTS: HOUSEHOLD￾LEVEL OUTCOMES............................................................................................................................ 40 TABLE 9. OUTCOME FAMILY 2 VILLAGE FIXED EFFECTS DID RESULTS: HOUSEHOLD￾LEVEL OUTCOMES, ML-EL; VILLAGE-LEVEL TREATMENT INTENSITY EFFECTS......... 45 TABLE 10. OUTCOME FAMILIES 3 VILLAGE FIXED EFFECTS DID RESULTS: HOUSEHOLD LEVEL OUTCOMES, ML-EL.............................................................................................................. 49 TABLE 11. OUTCOME FAMILY 3 VILLAGE FIXED EFFECTS DID RESULTS: HOUSEHOLD￾LEVEL OUTCOMES, ML-EL. VILLAGE-LEVEL TREATMENT INTENSITY EFFECTS......... 54 TABLE 12. OUTCOME FAMILY 4 VILLAGE FIXED EFFECTS DID RESULTS: HOUSEHOLD￾LEVEL OUTCOMES............................................................................................................................ 57 ii | YAAJEENDE FINAL IMPACT EVALUATION REPORT USAID.GOV TABLE 13. OUTCOME FAMILY 4 VILLAGE FIXED EFFECTS DID RESULTS: HOUSEHOLD￾LEVEL OUTCOMES............................................................................................................................ 63 TABLE 14. OUTCOME FAMILY 2 VILLAGE FIXED EFFECTS DID RESULTS: HOUSEHOLD￾LEVEL OUTCOMES, ML-EL. MOTHERS GROUP PARTICIPATION VILLAGE INTENSITY REGRESSIONS. .................................................................................................................................. 112 LIST OF FIGURES FIGURE 1. TRENDS IN CHILD ANTHROPOMETRIC INDICATORS IN SENEGAL, 2000–14 ....................................................................................................................................................................3 FIGURE 2. SENEGAL SEASONAL CALENDAR, TYPICAL YEAR.............................................3 FIGURE 3. YAAJEENDE INSTITUTIONAL PLATFORMS AT VILLAGE AND COMMUNE LEVELS ......................................................................................................................................................8 FIGURE 4. MAP OF YAAJEENDE FINAL IMPACT EVALUATION VILLAGE SAMPLE..... 17 FIGURE 5. MAP OF YAAJEENDE ZONES OF INTERVENTION........................................... 18 FIGURE 6. OUTCOME FAMILY 1 COMPARING TREATMENT EFFECTS BY MODEL: INDIVIDUAL-LEVEL OUTCOMES ................................................................................................. 30 FIGURE 7. OUTCOME FAMILY 1 COMPARING TREATMENT INTENSITY EFFECTS BY MODEL: INDIVIDUAL-LEVEL OUTCOMES................................................................................ 37 FIGURE 8. OUTCOME FAMILY 2 COMPARING TREATMENT EFFECTS BY MODEL: HOUSEHOLD-LEVEL OUTCOMES............................................................................................... 41 FIGURE 9. OUTCOME FAMILY 2 COMPARING TREATMENT INTENSITY EFFECTS BY MODEL: HOUSEHOLD-LEVEL OUTCOMES ............................................................................. 46 FIGURE 10. OUTCOME FAMILY 3 COMPARING TREATMENT EFFECTS BY MODEL: HOUSEHOLD-LEVEL OUTCOMES............................................................................................... 50 FIGURE 11. OUTCOME FAMILY 3 COMPARING TREATMENT INTENSITY EFFECTS BY MODEL: HOUSEHOLD-LEVEL OUTCOMES ............................................................................. 55 FIGURE 12. OUTCOME FAMILY 4 COMPARING TREATMENT EFFECTS BY MODEL: HOUSEHOLD-LEVEL OUTCOMES............................................................................................... 58 FIGURE 13. OUTCOME FAMILY 4 COMPARING TREATMENT INTENSITY EFFECTS BY MODEL: HOUSEHOLD-LEVEL OUTCOMES ............................................................................. 64 iii | YAAJEENDE FINAL IMPACT EVALUATION REPORT USAID.GOV Acronyms APS Agent de service préstataire (Service Provider Agent) AMEP Activity Monitoring and Evaluation Plan ANSD Agence nationale de la statistique et de la démographie (National Agency for Statistics and Demography) BFS Bureau for Food Security CAADP Comprehensive Africa Agriculture Development Program CBSP Community-Based Solution Provider CLM Cellule de la lutte contre la malnutrition (Cell Against Malnutrition) CNV Community Nutrition Volunteer COR Contracting Officer’s Representative CWG Citizen Working Group DID Difference-in-Difference DHS Demographic Health Survey DO Development Objective DPV Direction de la protection des végétaux (Directorate of Plant Protection) EDR Evaluation Design Report EGO Economic Growth Team GD Group Discussion GDG Groupe Debbo Galle (“Excellent Mothers” Group) GHI Global Hunger Index FCFA Franc CFA (Senegalese currency) FIE Final Impact Evaluation FLO First-Level Objective FRA Flood Recession Agriculture HDDS Household Dietary Diversity Score IE Impact Evaluation ITT Intent to Treat KII Key Informant Interviews iv | YAAJEENDE FINAL IMPACT EVALUATION REPORT USAID.GOV MAD Minimum Acceptable Diet MCHN Mother-Child Health and Nutrition MDES Minimum Detectable Effect Size MEP Monitoring and Evaluation Project MIE Midterm Impact Evaluation MNCH Maternal Child Health and Nutrition NCBA CLUSA National Cooperative Business Association’s Cooperative League of the USA NLA Nutrition-Led Agriculture PE Performance Evaluation POC Point of Contact PSM Propensity Score Matching RCT Randomized Controlled Trial USAID United States Agency for International Development VSC Village Steering Committee WASH Water, Sanitation and Health WDDS Women’s Dietary Diversity Score v | YAAJEENDE FINAL IMPACT EVALUATION REPORT USAID.GOV EXECUTIVE SUMMARY PURPOSE AND BACKGROUND This report details the findings, conclusions and recommendations of a mixed-methods quasi-experimental final impact evaluation (FIE) of the United States Agency for International Development’s (USAID) $50 million, seven-year Feed the Future Nutrition-Led Agriculture Project for Food Security in Senegal, known as “Yaajeende.” The National Cooperative Business Association’s Cooperative League of the USA (NCBA CLUSA) implemented the project. To combat poverty and child malnutrition, Yaajeende sought to accelerate the participation of the very poor in rural economic growth and improve the four dimensions of food security: availability, access, utilization and stability. Yaajeende worked in 790 villages across 49 municipalities (“communes” in French) and nine departments in the Matam, Tambacounda, Kédougou and Kolda regions. The project’s implementation period was November 1, 2010, to September 30, 2017. The FIE aims to provide USAID with an evidence base on the impacts of the nutrition-led agriculture (NLA) approach that the project utilized on its key objectives, including reduced poverty and malnutrition. The findings are expected to provide accountability and learning value to USAID, including both the Senegal Mission and USAID/Feed the Future. Additional stakeholders include the Government of Senegal, implementing partners and other agencies, donors and practitioners active in nutrition, health, agriculture and integrated sectors. EVALUATION QUESTIONS The Yaajeende FIE used a mixed-methods approach to answer seven overarching evaluation questions: 1. (Overall program impacts) What are the impacts of the Yaajeende NLA package on the prevalence of poverty and malnutrition six years after the start of program implementation, across four thematic categories: women’s and children’s nutrition; household food security and poverty/economic well￾being; household water, sanitation and hygiene practices; and household agricultural practices. 2. (Drivers of impacts) What major factors or processes contributed to observed impacts, including the role and importance of Yaajeende-supported local institutions such as community-based solution providers (CBSPs), citizen working group (CWG) / village steering committees (VSCs) and Debbo Gallé groups 1 (GDGs)? 3. (Heterogeneity of impacts) How do program impacts differ for two key subgroups of interest across key outcomes: households in the northern regions (Matam/Bakel) vs. southern regions (Kédougou) and poorest households2 vs. other households? 4. (Drivers of heterogeneous impacts) What are potential explanatory reasons for variations in key outcomes across subgroups? 1 Mother-to-mother groups formed through the project. Yaajeende documentation described their primary function as: “groups of women of child-bearing age who support and promote community nutrition activities and nutrition education.” 2 Defined for this evaluation as households below the median raw poverty score at baseline. vi | YAAJEENDE FINAL IMPACT EVALUATION REPORT USAID.GOV 5. (Moderating context factors) How do key individual and household characteristics shape program impacts? (Household: family size, maximum level of education, age of household head; Individual: mother’s age at time of first birth, GDG participation.) 6. (Targeted follow-up analyses) What characteristics of households and mothers appear to be associated with successful poverty and malnutrition reduction for children under age 5 and women of reproductive age? 7. Were there any unintended broader consequences of the intervention, positive or negative, beyond those related to activity objectives? METHODS The FIE is a quasi-experimental, mixed-methods study presenting the sum of evidence for Yaajeende program impacts, heterogeneity of impacts, and key drivers and moderating factors. The statistical impact analysis uses a difference-in-differences (DID) regression approach, coupled with statistical matching via entropy weighting, to estimate the project’s impact on outcomes of interest. This is done for an analytic sample of 2,470 households surveyed in 157 Yaajeende treatment or comparison group villages. The sample includes 1,830 households in 115 villages in the treatment group, and 640 households in 42 villages in the comparison group. Yaajeende impacts are assessed and reported for both the midline to endline (ML-EL) period (2015 to 2018), and across baseline to endline (BL-EL, 2011 to 2018). Statistical results on Yaajeende impacts are coupled with qualitative data collection at endline to expand on and help interpret the impact results. The qualitative analysis draws on group discussion and key informant interview data collected in 18 Yaajeende villages (six per region) and nine comparison group villages (three per region). In total, qualitative data collection consisted of 45 group discussions at endline with Debbo Gallé group members, other women in Yaajeende and comparison villages and men in Yaajeende villages, together with 70 key informant interviews with project stakeholders at village and higher administrative levels. The FIE measures project impacts for 19 outcome measures across four outcome families: women’s and children’s nutritional status and diet; household food security and economic well-being; household water, sanitation and health (WASH) practices; and household agricultural practices. Heterogeneity in project effects is examined for two beneficiary subgroups: (1) households in the northern regions (Matam/Bakel) vs. the southern region (Kédougou); and (2) poorest households vs. other households. Key limitations to this impact evaluation are typical for quasi-experimental IEs: potential for biased estimates of project effects due to selection bias on where implementation occurred, and recall bias with respect to processes that may have occurred early in project implementation. Contamination of comparison areas by other donor programming arose as an important constraint on comparability at endline. Given the evidence for similar donor programming in comparison villages, the comparison case for this evaluation changed to measuring the effects of Yaajeende programming relative to households that were exposed to similar other donor programming during the same time period. FINDINGS AND CONCLUSIONS A key overarching issue for the interpretation of the impact results from this evaluation is the wide range of overlapping donor initiatives apparent in the study area, further corroborated through qualitative data at endline. While the FIE often finds moderate to no evidence of Yaajeende program impacts beyond the comparison situation, the interpretation for many of the FIE results is not necessarily that Yaajeende had vii | YAAJEENDE FINAL IMPACT EVALUATION REPORT USAID.GOV no impact, but rather that the Yaajeende program’s effects were similar to those of other programs implemented in comparison group villages. Evaluation Question 1: What are the impacts of the Yaajeende NLA package on the prevalence of poverty and malnutrition six years after the start of project implementation, across four thematic categories (women’s and children’s nutrition; household food security and poverty / economic well-being; household water, sanitation and hygiene practices; and household agricultural practices)? Over midline to endline, the FIE finds beneficial impacts as a result of the project in four areas: two key women’s and children’s nutritional status indicators (reduction in prevalence of women underweight, and a 2.5-8.0 percentage point increase in the prevalence of minimum acceptable diet, or MAD), a 0.8 to 2.8 percentage point reduction in the likelihood of poverty at the household level, an increase in agricultural investment and an increase in agricultural production. In most cases, the magnitude of these increases is moderate, but these impacts are generally against an overarching time trend of gains on the same (for which Yaajeende programming had additional impacts above the background trends) or within a context of decline (in which Yaajeende programming shows evidence of having helped households mitigate overarching negative stresses). Relative to a comparison situation of similar programming efforts on women’s and children’s health, nutrition, WASH and agricultural support, the FIE finds no evidence for added Yaajeende project impacts for healthy household practices such as common use of a handwashing station and use and proper storage of iodized salt, where Yaajeende and comparison households alike improved on these indicators during the project lifetime, but gains were higher in comparison areas. Although the FIE focuses on midline to endline results due to power limitations and lower reliability of the baseline data, the baseline to endline impact results confirm and follow the same trend on the outcomes for healthy household practices and agricultural practices, and confirm and find stronger impacts than the midline to endline period for household economic well-being results. The baseline to endline results do not find evidence of positive Yaajeende effects for any women’s nutritional status and diet outcomes, but the analyses are underpowered to detect a small significant effect. Many of the FIE findings are consistent with the pattern of outcomes found at midline through the MIE analyses, including a similar set of constraints on wider impacts, as obtained through qualitative data collection. The FIE shows some evidence that varying household-level exposure to project trainings and activities is a moderator of overall project impacts across Yaajeende villages. For villages with higher average village￾wide exposure to and participation in trainings on issues promoted by Yaajeende, the prevalence of children underweight declined by a 3 to 6 percentage points, the stunting rate dropped by 3 percentage points, and the likelihood of poverty at the household level experienced a stronger reduction. In general, intensity of treatment results may suggest that for integrated agriculture, health and nutrition programs, a programming approach that achieves higher saturation of direct participation in multiple trainings across households in a village may be associated with a higher likelihood of achieving statistically significant change on key women and children’s nutrition and diet outcomes. viii | YAAJEENDE FINAL IMPACT EVALUATION REPORT USAID.GOV Evaluation Question 2: What major factors or processes contributed to observed impacts, including the role and importance of Yaajeende-supported local institutions? The qualitative data shed some light on certain contributing reasons for overarching impacts. There was a widespread view of positive changes in children’s health, and a reduction in children’s malnutrition, as a result of Yaajeende project activities. This had similar levels of support across each of the three regions covered by the evaluation. But, key drivers of impacts, or lack thereof, focused on continued lack of materials and financial resources at the household level to implement or sustain Yaajeende-promoted activities, particularly with respect to inputs needed for effective agricultural production and gardening activities. Sufficient and reliable water access remained a key constraint for broader agricultural production and market gardening gains throughout the project areas. While respondents widely viewed the introduction of community or microgardens as beneficial, they reported several challenges with a focus on the lack of or insufficient access to several required inputs and insufficient water, a near ubiquitous limitation mentioned for all agricultural activities assessed. In some areas, particularly in Kédougou, respondents viewed market oversaturation as a constraint on higher agricultural revenues. Respondents identified insufficient breast milk production by mothers and lack of time to comply with optimal feeding practices due to women’s schedules and labor needs as key factors that continue to stymy wider implementation of optimal breastfeeding practices, despite a strong knowledge of associated health benefits. The FIE found modest gains in reducing household poverty likelihood through the project and suggestive evidence that this relates to increased agricultural production. But these effects varied widely and evidence of increased revenue as a result of production gains or stronger value chain participation is limited. With respect to the village and higher-level structures that the Yaajeende program put in place to help disseminate knowledge and institutionalize behavior change, qualitative data at endline indicated a view that these institutions and their communications mechanisms were effective for transmitting knowledge and permitting wide knowledge-sharing within villages. However, respondents in beneficiary villages largely indicated that household means primarily drove the ability to put this knowledge into practice, especially with respect to improving diets of women and children, and a household’s lack of means to grow or buy sufficient quantities of nutritious foods is a key limiting factor for many. With respect to a child’s access to enriched foods, the communal system supported by the project for child health screenings and production of such foods, where these systems are maintained, appears beneficial. With respect to evidence on broader contributing roles of Yaajeende-supported local institutions, the FIE found some evidence, though varying, that non-standardized processes for GDG creation and rules on membership could have contributed to lower participation rates or selective membership only by certain individuals in villages. If widespread, this could be a contributing factor to lower overall achievement on some outcomes, such as nutrition and diet, since the project envisioned the GDGs to have a primary responsibility of supporting and promoting community nutrition activities and nutrition education, and utilized GDGs as the primary vehicles to disseminate information and select participants for a range of project activities. The network of service delivery provider agents (APS, or in some places APS/VNC, for community nutrition volunteers) acted as a key facilitator for efficient distribution of agricultural inputs and monitoring progress with clients to ensure that they receive their inputs on time each season. Many saw the APS role as important for bridging the gap with suppliers at the village level, and to provide follow￾on technical advice to villagers on the use of new agricultural technologies or inputs they had obtained. But KIIs with several APSs suggested that the system is not yet profitable for many such agents to sustain service delivery on their own. Of the roles created through the project, endline qualitative data collection ix | YAAJEENDE FINAL IMPACT EVALUATION REPORT USAID.GOV suggests that the citizen working group (GTC) may have experienced the most challenges for functionality, pointing to coordination issues between GTCs and local authorities that may have limited their effectiveness, and insufficient financial and material autonomy to succeed in practice across all of their intended responsibilities. Evaluation Question 3: How do program impacts differ for key subgroups of interest across key outcomes? The evaluation will assess two subgroups: northern regions (Matam and Bakel) vs. southern region (Kédougou); and poorest households vs. other households. For a small number of outcomes, the FIE finds evidence that program impacts differed by region or household wealth status. In Kédougou Region, the program achieved moderate reductions in stunting and prevalence of children underweight that were not observed in Matam or Bakel. Those areas saw greater decline in poverty likelihood and increased agricultural production. There are also some differences in impacts for poorest households. Poorer households had stronger gains on agricultural investment and agricultural production, but the overall program effects observed on women’s and children’s nutritional and diet indicators were driven by gains on these outcomes in less-poor households. Evaluation Question 4: What are potential explanatory reasons for variations in key outcomes across the subgroups? Themes identified in the qualitative data provide do not elucidate reasons for observed regional differences or those by household wealth status. Yaajeende households in all regions and across both wealth groups reported similar levels of participation in nutrition and health trainings. Yaajeende households in Kédougou reported higher rates of training participation relative to northern region households for agricultural, livestock and WASH trainings, but they also have a less diverse diet, a lower percentage of garden use and no observed gains on agricultural yields or revenue relative to Matam and Bakel households. Poorer households reported similar levels of training participation to those of less-poor households across all training categories. This may have contributed to their gains on agricultural outcomes, but it is possible that these gains may not yet have reached a sufficient level to realize corresponding improvements in nutritional and diet indicators. In general, the available evidence through this evaluation suggests that different results pathways are likely at work for achieving impacts in Kédougou relative to the two northern areas of Yaajeende implementation, but the current analyses does not point to strongly obvious reasons that are able to explain the regional or wealth status differences. Evaluation Question 5: How do key individual and household characteristics shape program impacts? Children’s age (measured in days) is associated with higher prevalence of stunting and underweight and lower likelihood of being exclusively breastfed. But an increase in child’s age also associates with a higher likelihood that the child receives a minimum acceptable diet. For adult women, age (measured in years) works in the opposite direction and is associated with a lower underweight prevalence. The effect of the household head’s level of education goes in the expected direction, with children in households where the head has at least a primary education approximately 4.5 percent and 6.4 percent less likely to be stunted or underweight, respectively. x | YAAJEENDE FINAL IMPACT EVALUATION REPORT USAID.GOV Larger households appear to exhibit higher household dietary diversity, reduced lean season duration, lower likelihood of poverty and greater revenue from agriculture. Similarly, households where the head has at least an elementary education have household dietary diversity score (HDDS) measures that are 0.391 points higher, on average, than households with uneducated household heads; they experience lean seasons that, on average, are 0.336 months (around 10 days) shorter, are 1.457 percentage points less likely to fall below the poverty line and have agricultural revenue that is FCFA 18,788.074 (approx. USD $33)3 higher. Households with a greater number of members are more likely to have soap-and-water handwashing stations, but there is no evidence of a relationship between household size and iodized salt usage and storage. There is no evidence of an association between the head of household’s education status and household healthy practices outcomes, or between prevalence of handwashing stations and the age of the head of household. A small negative relationship exists between the head of household’s age and the use and proper storage of iodized salt. Larger households show slightly higher agricultural investment and have greater agricultural production. The household head’s education status has no association with agriculture investment or use of CBSP, though households where the head has at least an elementary education have agriculture production that is 223.9 kg higher per year, on average, than when the households head has no education. Evaluation Question 6: What characteristics of households and mothers appear to be associated with successful poverty and malnutrition reduction for children under age of 5 and women of reproductive age? The key factors identified through the FIE analyses are overwhelmingly household characteristics that much existing literature identifies as strong determinants of household poverty and malnutrition status: women’s age (mother or primary caregiver), household head level of education and household size. The FIE finds no evidence of statistically significant treatment effects for participation in mothers’ groups in Yaajeende treatment villages, relative to trends for individuals in Yaajeende village households where no household member participated in such a group. The exception is a statistically significant increase in the prevalence of stunting for children in households with participation in a mothers’ group, a result that is unexpected and difficult to interpret. Given that women self-select into mothers’ groups, one potential explanation for such a result could be that women in households experiencing greater negative shocks disproportionately joined mothers’ groups during the midline to endline period. The FIE also finds no evidence for greater effects among participating households in villages where mothers’ group participation is stronger, nor evidence of statistically significant treatment effects for participation in mothers’ groups in treatment villages, relative to trends for individuals in Yaajeende village households where no one participated in a mothers’ group. Evaluation Question 7: Were there any unintended broader consequences of the intervention, positive or negative, beyond those related to the activity objectives? In terms of broader unintended positive consequences of Yaajeende programming, the FIE points to unexpected but synergistic empowerment and capacity building for self-reliance, seen to contribute 3 USD $1 = FCFA $564.81. xi | YAAJEENDE FINAL IMPACT EVALUATION REPORT USAID.GOV positively to post-project sustainability on several activities. Respondents mentioned the following key activities: continued sensitization activities in their communities, community meals and preparation of enriched flour. Negative unintended consequences included some perceptions of lost interest and information overload related to the time-consuming nature of participation in project activities. RECOMMENDATIONS • Include targeted efforts to ease household resource constraints. As highlighted in the MIE findings, financial and material resource constraints at the household level continue to serve as a key barrier for low household adoption or sustained uptake of key Yaajeende-promoted activities, including market gardening, more productive and varied agriculture in general and some income￾generating activities. Future projects should consider strategies by which they can make such inputs more affordable and accessible to smallholders earlier in project time frames, and include targeted efforts aimed toward poorer households for which such gains are likely to have the strongest short￾term boost in food security, nutrition and health benefits through agricultural programming. • Consider partnerships and/or strategic planning with water infrastructure programs during program design stages, to ensure that program rollout of agricultural and gardening activities takes places in areas with sufficient and reliable water access. • Provide follow-on support to CBSP / APS networks to overcome key barriers to sustained activity and growth. Respondents widely recognized the APS network as instrumental for efficient distribution of agricultural inputs and for monitoring progress with clients to ensure that they receive their inputs on time each season, bridging an important gap with suppliers at the village level and providing needed technical guidance to beneficiaries on use of new agricultural technologies or inputs they had obtained. But, KIIs with several APSs suggested that by project end, the system is not yet profitable for many such agents to sustain service delivery on their own. At endline, multiple respondents noted dissolution of APS relationships in each of the three regions. More support of the APS system appears necessary before it can be considered fully functional and sustainable on its own. This seems particularly warranted in a follow-on activity, given the substantial investment in establishing and maintaining the system during the Yaajeende program lifetime. • Consider more streamlined nested governance structures and building earlier and stronger linkages to government or other existing structures that are necessary partners for post-project sustainability. Yaajeende focused on establishing locally led governance structures and institutions for knowledge-sharing, activity rollout and service provision. The project created several interconnected institutional structures within village and higher administrative levels to help embed communications systems and coordination for project activities, disseminate information and establish functional platforms for sustained knowledge transfer and service delivery post-project. While the FIE finds this approach beneficial in general, it is possible that the layered and overlapping nature of responsibilities resulted in an overly complicated institutional structure, with relatively weaker and stronger linkages in different parts of the system, that may be difficult to sustain in whole without continued project support. Future projects may benefit from a more streamlined and hierarchical structure, as well as earlier and more dedicated efforts to identify vulnerabilities in the system and building linkages between newly established institutions and the broader government systems they are likely to rely on post-project. Such efforts may also help to identify opportunities for cross-program synergies and leverage opportunities, as well as efficient human and other resource xii | YAAJEENDE FINAL IMPACT EVALUATION REPORT USAID.GOV allocation, across the multiple actors in the donor-supported mother-child health and nutrition (MCHN), WASH and agriculture space in country. • Consider bifurcated strategies that provide more direct targeting and dedicated support for the most vulnerable households to increase impacts for the poorest households. The FIE finds that poorer households had stronger gains on agricultural investment and production, yet they failed to achieve impacts on nutritional status and diet indicators observed for the program overall. Yaajeende achieved great progress in increasing community knowledge on appropriate feeding and diets and links to overall health and in establishing a system by which community members could lead monitoring and identification of malnutrition cases and provide enriched foods. However, more dedicated and systematized efforts appear to be needed to ensure that such services are provided on a regular basis for the poorest households. • Consider developing region-specific strategies that take into account strong context differences across implementation zones, given evidence of differences in impacts for Kédougou Relative to the northern areas covered by this evaluation. • Consider theory of change and evaluation learning by measuring impact through targeted “mini”-randomized controlled trial (RCT) impact evaluation activities. Given the large scale of Feed the Future programs in Senegal, there is opportunity to design and conduct smaller-scale mini-RCTs targeting specific learning questions for subsets of program activities. Especially in areas with high levels of similar or overlapping donor activity, RCT approaches are better suited to provide reliable impact estimates and learning on effective program interventions. RCTs also often require smaller sizes to achieve desired statistical power, but involve more upfront work on the design end and work best when they are designed and conducted in close collaboration with program implementers at the start of new programs. 1 | YAAJEENDE FINAL IMPACT EVALUATION REPORT USAID.GOV INTRODUCTION This report details the findings, conclusions and recommendations of a final impact evaluation (FIE) of the United States Agency for International Development’s (USAID) $50 million, seven-year Feed the Future Nutrition-Led Agriculture Project for Food Security in Senegal, known as “Yaajeende.” The National Cooperative Business Association’s Cooperative League of the USA (NCBA CLUSA) implemented this project from 2010 to 2017. To combat poverty and child malnutrition, Yaajeende sought to accelerate the participation of the very poor in rural economic growth and improve the four dimensions of food security: availability, access, utilization and stability. Yaajeende worked in 790 villages across 49 municipalities (“communes” in French) and nine departments in the Matam, Tambacounda, Kédougou and Kolda regions. The FIE aims to provide USAID with an evidence base on the impacts of the nutrition-led agriculture (NLA) approach utilized by the project on its key objectives, including reduced poverty and malnutrition. The FIE applies USAID’s evaluation policy guidance on using the most rigorous evaluation design and methods possible to identify impacts, establish attribution to program activities and demonstrate accountability for achieving results. The FIE follows up on key findings and questions identified during a 2015 midterm impact evaluation (MIE) of Yaajeende, and aims to provide an evidence base on the impacts of the “nutrition-led agriculture” (NLA) approach on poverty and malnutrition objectives, as well as capture practical lessons from USAID’s experience using the NLA to achieve key project objectives. The FIE measures project impacts on development outcomes of interest, provides corresponding qualitative evidence to explain how and why the project achieved observed impacts on project indicators, identifies drivers of variation in results across key subgroups of interest and aims to contribute to the evidence base and inform learning that may be useful for the design of future activities that aim to use integrated agricultural, health and nutrition synergies to improve poverty and malnutrition. Primary audiences for this evaluation are USAID, project implementing partners and government agencies involved in Yaajeende. Secondary audiences include other non-governmental organizations, government agencies and the members of the broader donor community in Senegal and globally who are involved in the nutrition, health and poverty reduction sectors. COUNTRY AND SECTOR BACKGROUND ECONOMIC STATUS, POVERTY AND MALNUTRITON IN SENEGAL Senegal’s overall economic context is characterized by punctuated and uneven economic growth, coupled with widely disparate trajectories across geographic regions and population subgroups. The country experienced a period of generalized growth from 1995 to 2005, but has seen stagnant per capita gross domestic product (GDP) growth in recent years, and experienced a series of food, climatic and financial shocks over the past decade. 4 Economic fluctuations in the country haven taken place against a backdrop of significant population expansion since 2002, including an average annual growth in population of 2.5 percent, 5 and large movements of the population from rural to urban areas that have contributed to 4 World Bank. 2015. Senegal Poverty Assessment: Report No: ACS10625. Washington, DC: World Bank. 5 2014. Rapport Définitif: Recensement Général de la Population et de l’Habitat, de l’Agriculture et de l’Elevage (RGPHAE) 2013. Dakar, Sénégal: ANSD. 2 | YAAJEENDE FINAL IMPACT EVALUATION REPORT USAID.GOV significant population disparities between a small number of urban centers relative to rural areas in the country. Despite a reduction of poverty in urban areas over the past decade, the poverty headcount in Senegal has been stagnant since 2005. With respect to malnutrition, Senegal has one of the lowest rates of chronic malnutrition in sub-Saharan Africa6 and has received attention for its fairly strong policy approach and government commitment toward developing a multi-sectoral strategic plan focused on nutrition and improving the country’s nutrition service delivery system. Senegal has achieved important gains in reducing malnutrition as a whole, however many challenges remain and concern is growing over gains on nutrition indicators in earlier decades that have remained stagnant. 7 For example, although the prevalence of child under-5 stunting of 18.7 percent in 2014 is one of the lowest on the continent, 8 the past decade has seen poor economic growth and an increase in the incidence of poverty in the eastern and southern regions, no change in the prevalence of child wasting and one of the highest rates of underweight women aged 15-49 on the continent. Moreover, there is evidence that the reduction in the stunting rate disproportionately affects wealthier and more urban households, and gains have been greater among male children than female children. In general, Senegal has also seen substantial and widening geographic disparities across regions on health and nutrition indicators, with higher rates of stunting in regions in the country’s South and East. This includes no change over time in Bakel, which has a higher rate of child wasting. On net, Senegal’s nutrition profile is characteristic of a country in transition, 9 following a somewhat common trajectory for countries undergoing an abrupt economic adjustment, in which poverty-linked malnutrition and associated health issues remain pervasive in the face of a smaller but growing trend in prevalence of obesity and overnutrition indicators that tend to be more commonly seen in high-income countries. 10 6 According to the 2016 Demographic Health Survey (DHS), the prevalence of child stunting in Senegal was 17.0 percent, wasting was 7.2 percent, child under-5 underweight was 13.5 percent, and the percentage of children under the age of 6 months who were exclusively breastfed was 36.4 percent. Based on data from DHS program STAT compiler, USAID, (accessed in September 2018) https://www.statcompiler.com/en/. 7 Nene, Marc. 2018. “Nutrition Situation in Senegal.” Analysis & Perspective: 15 Years of Experience in the Development of Nutrition Policy in Senegal. World Bank, Washington, DC; CLM, Dakar, Sénégal. 8 According to Nene 2018, Senegal ranks second-lowest in sub-Saharan Africa on the prevalence of stunting in children under the age of 5 but is in the middle of the pack with respect to prevalence of women underweight, child wasting and exclusive breastfeeding. 9 Popkin, B.M., L.S. Adair and S.W. Ng. 2012. “Global Nutrition Transition and the Pandemic of Obesity in Developing Countries.” Nutrition Reviews 70 (1): 3–21. 10 Popkin, B. 2001. “The Nutrition Transition and Obesity in the Developing World.” Journal of Nutrition 131 (3): 871S–873S. 3 | YAAJEENDE FINAL IMPACT EVALUATION REPORT USAID.GOV FIGURE 1. TRENDS IN CHILD ANTHROPOMETRIC INDICATORS IN SENEGAL, 2000–14 Source: Nene, Marc. 2018. “Nutrition Situation in Senegal.” Analysis & Perspective: 15 Years of Experience in the Development of Nutrition Policy in Senegal. World Bank, Washington, DC; CLM, Dakar, Sénégal. FOOD SECURITY In 2017, Senegal ranked 67th of 99 countries in the Global Hunger Index (GHI). Although the country has seen substantial improvement in its GHI score since 2000s, indicating an improving trend on the country’s hunger situation, the level of severity of hunger is considered to be moderate to serious. In 2016, 11.3 percent of the population was considered to be undernourished. 11 The World Food Program estimates that 17 percent of the population in rural areas of Senegal are food insecure. Causes of food insecurity are wide-ranging and have been exacerbated by recurring droughts, floods and changing timing and severity of climatic stresses that have negatively impacted household food production. Other factors include high food prices and additional recurring economic shocks, as well as low household capacity to weather food and economic shocks. FIGURE 2. SENEGAL SEASONAL CALENDAR, TYPICAL YEAR Source: Famine Early Warning Systems Network (FEWS NET), http://www.fews.net/west-africa/senegal. 11 International Food Policy Research Institute. 2017. “2017 Global Hunger Index: The Inequalities of Hunger.” IFPRI. Washington, DC. 4 | YAAJEENDE FINAL IMPACT EVALUATION REPORT USAID.GOV In terms of the broader knowledge base on underlying causes, direct determinants and effective programming to reduce maternal and child malnutrition, current logic frameworks on maternal and child nutrition consider the direct causes of malnutrition to include inadequate food intake (both quantity and quality of food), coupled with recurring incidences of disease that compromise intake and absorption of nutrients. Key underlying causes of malnutrition include food security (food availability and food diversity), access to and use of health services and a general hygienic environment. At the macro level, studies point to a range of economic, social, political and environmental factors, including country wealth status, poverty and wealth distribution, conflict and climate events. 12 A comprehensive 2017 analysis by the World Bank highlights seven nutrition-specific interventions, for which there is strong evidence of their efficacy for reducing key malnutrition indicators. These interventions are targeted either to pregnant women and those with infants, or directly to infants and young children. Interventions to reduce child stunting that are targeted to pregnant women and those with infants include antenatal micronutrient supplementation (including iron and folic acid supplements), individual or group-based counseling on infant and young child nutrition (including promotion of exclusive breastfeeding for children 0-5 months of age and appropriate quantity and quality of complementary foods for children 6-23 months of age), nutrition supplementation for pregnant women who are food insecure and intermittent presumptive treatment of malaria in malaria-endemic regions. Interventions directly for children include vitamin A and zinc supplements for children 6-59 months of age and public provision of supplemental foods for children 6-23 months of age living under the poverty line, delivered through a community-based nutrition program or social safety net programs. 13 However, analyses also indicate that current financing for such interventions in poor countries is generally insufficient to meet current global nutrition objectives and targets established by the World Health Organization, 14 and substantial scale-up of such investments would be required to achieve malnutrition reductions at scale. FINAL IMPACT EVALUATION ACTIVITY DESCRIPTION The purpose of the FIE is to evaluate the impact that the Feed the Future Nutrition-Led Agriculture Project for Food Security in Senegal (known as “Yaajeende”) has had on reducing malnutrition and poverty in its intervention area. The National Cooperative Business Association’s Cooperative League of the USA (NCBA CLUSA) received a five-year cooperative agreement in November 2010 to implement the project, and then a two-year extension in September 2015. Yaajeende was implemented in 790 villages across 49 municipalities (“communes” in French) in the regions of Matam, Kédougou and Kolda and the department of Bakel in Tambacounda. According to project documentation, the geographic coverage of the project was substantial, reaching 84 percent of Matam, 70 percent of the department of Bakel and 40 percent in each of Kédougou and Kolda. By project’s end in October 2017, Yaajeende reported reaching nearly 1 million people across 101,000 households. 15 The project aimed to work in vulnerable areas, and communes were selected for project implementation on 12 Eozenou, Patrick Hoang-Vu; Shekar, Meera. 2017. Stunting Reduction in Sub-Saharan Africa. Washington, D.C., World Bank Group. 13 Bhutta, Z.A., J.K. Das, A. Rizvi, M.F. Gaffey, N. Walker, S. Horton, P. Webb, A. Lartey and R.E. Black. 2013. “Evidence-Based Interventions for Improvement of Maternal and Child Nutrition: What Can Be Done and at What Cost?” The Lancet 382 (9890): 452–77. Shekar, Meera, Jakub Kakietek, Julia Dayton Eberwein, Jon Kweku Akuoku, Audrey Pereira and Mary D’Alimonte. 2017. An Investment Framework for Meeting the Global Nutrition Target for Stunting. Washington, DC: World Bank Group. 14 WHO (World Health Organization). 2012. Global Targets 2025. http://www.who.int/nutrition/topics/nutrition_globaltargets2025/en/ 15 NCBA CLUSA, September 2017. Feed the Future Yaajeende Final Report to USAID. 5 | YAAJEENDE FINAL IMPACT EVALUATION REPORT USAID.GOV the basis of two key criteria: (1) incidence of malnutrition and (2) presence of sufficient water, arable land and human resources to strengthen civil society, governance and private local institutions situated to address project objectives over the longer term. 16 The core target population within selected communes was women of reproductive age (ages 15-49) and children under 5 years old. Smallholders and commercial producers were considered additional population targets by the project. A midterm impact evaluation was conducted in May and June 2015 in six of the nine departments where the project is active. Three departments in the Kolda Region were excluded because field operations did not start there until 2014. Prior to the project’s extension, some intervention areas had received the full nutrition-led agriculture (NLA) package, while others received only the nutrition or the agriculture package. During the extension phase, Yaajeende focused on scaling up the full NLA approach in all of the intervention areas. USAID/Senegal EGO commissioned this Final Impact Evaluation (FIE) of the Yaajeende project, in order to provide an evidence base on the impacts of the NLA approach utilized by the intervention on key project objectives including reduced poverty and malnutrition. The FIE follows up on key findings and questions identified during a 2015 midterm impact evaluation (MIE) of Yaajeende, and aims to capture practical lessons from USAID’s experience using the NLA to achieve key project objectives. YAAJEENDE DEVELOPMENT HYPOTHESIS AND PATHWAYS TO IMPACT Yaajeende addressed three of the four first-level objectives (FLOs) of the Mission’s Economic Growth Results Framework via several intermediate results to be attained by the activity: • FLO 1: Inclusive Agriculture Sector Growth • FLO 3: Improved Nutritional Status, especially of women and children • FLO 4: Improved Management of Natural Resources The project’s stated overarching goal was to improve the economic and nutritional status of the very poor in four regions of Senegal using a holistic food systems strategy. The project theory of change posited that mutually supporting interventions of nutrition and agriculture will be more efficacious in improving nutritional status than either component on its own. The goals of Yaajeende were to (a) integrate the very poor into agricultural markets and the rural economy; (b) improve the nutritional status of women and children; and (c) increase household assets and income among those who are not participating in rural economic activities, including those who are unable to participate. YAAJEENDE PROJECT IMPLEMENTATION Yaajeende viewed the NLA approach as a structural food system approach based on the four pillars of food security: availability, access, utilization and governance. It focused on sustainable production, broad distribution, trade, informed consumption and transparent governance of high-quality, nutritious foods that have the ability to resolve nutritional deficiencies. The project aimed to encompass issues related to governance, health, market, behavior change and demand creation by integrating government, civil society and public health actors, as well as private sector and nonprofit actors. 16 Yaajeende documentation does not elaborate on how this second criterion was assessed. 6 | YAAJEENDE FINAL IMPACT EVALUATION REPORT USAID.GOV Yaajeende employed an NLA approach to address entrenched food insecurity issues in the project zone. Per Yaajeende’s description, “NLA is a dynamic food systems approach that promotes sustainable food and nutrition interventions through strong local governance and a responsive private sector. Because women are drivers of healthy food production and consumption in the community, Yaajeende places special emphasis on ensuring that women can fully participate in each of its NLA projects, including livestock, horticulture, resilient farming and bio-fortified crops, community-based solution provision, nutrition education, hygiene and food security governance.”17 To achieve project objectives, Yaajeende emphasized institutional capacity building and private sector strengthening to integrate agriculture and nutrition and contribute to overall sustainability. The key local institutions that the project emphasized were citizen working groups (CWGs), mother-to-mother groups (known as Debbo Gallé, or “excellent mothers” in Wolof) and through networks of community-based service providers (CBSPs) that were linked to private sector suppliers inputs, with the aim of facilitating improved access to products, information, services and technical assistance needed for the production, marketing and consumption of nutritious food. Per the project description, NLA was viewed to rest on the following key “development levers”: • Introduce innovative agricultural technologies to reduce nutritional micronutrient deficiencies among populations, including use of bio-fortified seed varieties, and innovative management techniques or methods to enhance agricultural productivity. • Develop a strong private sector network of CBSPs to facilitate access to products, information, services and technical assistance needed for the production, marketing and consumption of nutritious food. In turn, this was viewed to elicit changes in the agricultural production choices by farmers, as well to improve the ability for producers and consumers to make informed choices. • Build strong local institutions the capacity for local resource persons to “take charge of food security and nutrition issues after the project has ended.” Yaajeende implementation occurred through a series of “baskets” of activities aimed to address each of the four pillars of food security: availability, access, utilization and governance. Activities were designed to be mutually reinforcing across the pillars, with the aim of achieving integrated and sustainable results. The project streamlined its approach over the seven years of implementation, narrowing the activities from 19 to a set of seven interventions. Table 1 summarizes the Year 1 activities as described at project close. 18 TABLE 1. YAAJEENDE IMPLEMENTATION ACTIVITIES IN PROJECT YEAR 1. AVAILABILITY ACCESS UTILIZATION GOVERNANCE • Identify production and improve resiliency of rain-fed agriculture through improved natural resource management, agroforestry and conservation agriculture • Strengthen producer organizations • Promote the emergence of • Create the foundation for large-scale nutrition activities at the local level • Improve capacity of rural commune (RC) staff in food security issues 17 NCBA CLUSA, 2017. Feed the Future Yaajeende Final Report to USAID. (September 2017) 18 NCBA CLUSA, 2017. Feed the Future Yaajeende Final Report to USAID. (September 2017) 7 | YAAJEENDE FINAL IMPACT EVALUATION REPORT USAID.GOV AVAILABILITY ACCESS UTILIZATION GOVERNANCE • Promote new ISRA19/ICRISAT20 varieties of rice, millet, sorghum and maize • Improve availability of quality seeds • Improve irrigation and rural infrastructure • Improve the productivity of nutritious vegetables and fruits through gardening and arboriculture • Promote the creation of livestock-based enterprises • Improve animal nutrition community-based products and service providers • Improve the regional marketing of produce • Improve post￾harvest activities • Improve access to financial services • Conduct formative research to orient nutrition activities • Social behavior change communications and social marketing • Reintegrate wild foods to fortify local diets • Reduce incidence of diarrhea diseases by improving hygiene behaviors around drinking water and food • Improve the RCs use of transparent, participative techniques for planning, budgeting, implementation and monitoring activities Project activities by the final year of the project broadly consisted of: flood recession agriculture and quality grains project; commercial horticulture project; livestock enterprise project; CulitVert social franchise / CBSP project; Debbo Gallé project; WASH project; and a governance and food security project. By project close in 2017, Yaajeende reported implementation in a total of 790 villages, of which two-thirds had received a comprehensive package of interventions across the three conceptualized ‘levers’ of NLA, consisting of at least two or more interventions in nutrition and agriculture and at least one related to governance. The project’s management approach aimed for active involvement and community ownership of the process, and was based on a layered governance structure of stakeholders established at village, commune and regional levels, as summarized in Figure 3. Yaajeende viewed these platforms as “owners and frontline implementers” of the project, and aimed for increasing ownership and leadership of project activities by these groups over the Yaajeende lifetime. At project end, Yaajeende highlighted several achievements as measured through project monitoring, including improvements in women and children’s nutritional status (including reduced prevalence of acute malnutrition in children, improvements in minimum acceptable diet (MAD), and reduced malnutrition in women), and increased agricultural production and household income, such as through sale of agricultural products at local markets and other activities promoted by the project. 19 Institut Senégalais de Recherches Agricoles/Senegalese Institute of Agricultural Research 20 International Crops Research Institute for the Semi-Arid Tropics 8 | YAAJEENDE FINAL IMPACT EVALUATION REPORT USAID.GOV FIGURE 3. YAAJEENDE INSTITUTIONAL PLATFORMS AT VILLAGE AND COMMUNE LEVELS21 FINAL EVALUATION PURPOSE, AUDIENCE AND INTENDED USES The purpose of this FIE is to provide USAID with an evidence base on the impacts of the nutrition-led agriculture (NLA) approach utilized by the project on key project objectives, including reduced poverty and malnutrition. The results of the evaluation are aimed at several audiences. The findings are expected to have accountability and learning value to USAID, including the Senegal Mission and USAID/Feed the Future. Additional stakeholders include the Government of Senegal, implementing partners and other agencies, donors and practitioners active in nutrition, health, agriculture and integrated sectors. The findings, conclusions and recommendations of the evaluation may help inform the design of future activities that aim to use integrated agricultural, health and nutrition synergies to improve poverty alleviation and malnutrition. 21 Figure excerpted from NCBA CLUSA, 2017. Feed the Future Yaajeende Final Report to USAID. (September 2017) 9 | YAAJEENDE FINAL IMPACT EVALUATION REPORT USAID.GOV EVALUATION QUESTIONS The Yaajeende FIE used a mixed-methods approach to answer seven overarching evaluation questions: 1. (Overall project impacts) What are the impacts of the Yaajeende NLA package on the prevalence of poverty and malnutrition six years after the start of project implementation? Impacts will be examined across four thematic categories: a. Women and children’s nutrition; b. Household food security and poverty / economic well-being; c. Household water, sanitation and hygiene practices; and d. Household agricultural practices. 2. (Drivers of impacts) What major factors or processes contributed to observed impacts, including the role and importance of Yaajeende-supported local institutions such as CBSPs, CWG/VSCs and GDGs? 3. (Heterogeneity of impacts) How do project impacts differ for key subgroups of interest across key outcomes? The evaluation will assess how impacts vary for the following two sets of subgroups: a. Households in the Northern Region (Matam/Bakel) vs. Southern Region (Kédougou) and b. Poorest households22 vs. other households. 4. (Drivers of heterogeneous impacts) What are potential explanatory reasons for variations in key outcomes across subgroups? 5. (Moderating context factors) How do key individual and household characteristics shape project impacts? Characteristics to be assessed will include: a. Household: Family size, maximum level of education, age of household head. b. Individual: Mother’s age at time of first birth, GDG participation. 6. (Targeted follow-up analyses) What characteristics of households and mothers appear to be associated with successful poverty and malnutrition reduction for children under age 5 and women of reproductive age? 7. Were there any unintended broader consequences of the intervention, positive or negative, beyond those related to activity objectives? METHODS AND LIMITATIONS EVALUATION DESIGN The IE design and analytic approach used by the endline evaluation team maintains the quantitative sample size and sampling approach for data collection that was used at midline across three regions of project implementation (Matam, Bakel and Kédougou). The FIE maintains a mixed-methods quasi-experimental 22 Defined for this evaluation as households below the median raw poverty score at baseline. 10 | YAAJEENDE FINAL IMPACT EVALUATION REPORT USAID.GOV impact evaluation design, using a difference-in-difference (DID) approach coupled with statistical matching as the primary analytic method to establish project impacts. PROJECT OUTCOME FAMILIES, INDICATORS AND SUBGROUPS This FIE measures project impacts for 19 outcomes grouped across four families, as listed in Table 2: women and children’s nutritional status and diet; household food security and economic well-being; household WASH practices; and household agricultural practices. The FIE also examines heterogeneity in project effects for two beneficiary subgroups, which are: • Households in the northern regions (Matam/Bakel) vs. the southern region (Kédougou) and • Poorest households vs. other households. As supplemental analyses, the evaluation team conducted multi-variate regression analyses to better understand how Yaajeende impacts vary across a set of household and individual-level characteristics that may also shape project impacts. DEFINITION OF TREATMENT Two definitions of treatment were adopted for the endline analyses: (1) binary treated status based on whether households were located in a village that had received any Yaajeende implementation package by midline and (2) a household-level treatment intensity or dosage variable that was constructed from the sum of the total number of types of Yaajeende or similar development-programming trainings that a household directly participated in, across the following eight sets of issues: agriculture; livestock; health and nutrition; potable water and WASH; savings, loans and/or business skills and entrepreneurship; modern poultry farming, agroforestry, and food storage and processing. 11 | YAAJEENDE FINAL IMPACT EVALUATION REPORT USAID.GOV TABLE 2: YAAJEENDE ENDLINE IMPACT EVALUATION: IMPACT/OUTCOME MEASURES FIE NO. INDICATOR DESCRIPTION HH OR INDIVIDUAL LEVEL SUBGROUP ANALYSES WEALTH STATUS REGION WOMEN AND CHILDREN’S NUTRITIONAL STATUS AND DIET 1.1 Wasting Prevalence of wasting in children aged 6-59 months IND ✓ ✓ 1.2 Stunting Prevalence of stunting in children aged 6-59 months IND ✓ ✓ 1.3 Underweight, under 5 years Prevalence in children under age 5 IND ✓ ✓ 1.4 Underweight, 15-49 years Prevalence in women aged 15-49 years IND ✓ ✓ 1.5 Minimum acceptable diet Prevalence for children aged 6-23 months IND ✓ ✓ 1.6 Exclusive maternal breastfeeding Prevalence of exclusive breastfeeding of children under 6 months of age (recorded for children aged 0-24 months) IND ✓ ✓ 1.7 Women’s dietary diversity score, 25-49 years Women’s dietary diversity score, 25-49 years (New at endline) HH ✓ ✓ HOUSEHOLD FOOD SECURITY AND POVERTY/ ECONOMIC WELL-BEING 2.1 Food diversity score Household food diversity score HH ✓ ✓ 2.2 Soudure (Hunger season) Duration in months of reduced food intake (soudure) reported by the household HH ✓ ✓ 2.3 Poverty estimate Likelihood that household falls under the poverty line, based on poverty score card HH ✓ ✓ 2.4 Agricultural Revenue Per capita agricultural revenue (FCFA23) HH ✓ ✓ 2.5 Subjective HH well-being indicator Index of household satisfaction with current financial situation, level of food consumption, level of food expenditures, and ability to provide food and basic necessities over the next 12 months (New at endline) HH ✓ ✓ HEALTHY HOUSEHOLD PRACTICES 3.1 Handwashing station Prevalence of households with a handwashing station in common use HH ✓ ✓ 3.2 Salt iodation and storage Prevalence of households using iodized salt HH ✓ ✓ HOUSEHOLD AGRICULTURAL PRACTICES 4.1 Agricultural investment Index of agricultural technology adoption, scale ranging from 0-17 HH ✓ ✓ 4.2 Use of CBSP (community-based service provider) Prevalence of household purchase of goods or services from a community-based service provider HH ✓ ✓ 4.3 Production of targeted commodities Per capita yield of each of 4 target commodities (Kg/ha) (New at endline) HH ✓ ✓ 4.4 Garden access Prevalence of households with access to a home or community garden (New at endline) HH ✓ ✓ 4.5 Value chain participation Prevalence of households that practice value chain activities promoted by the activity in the past 12 months (New at endline) HH ✓ ✓ 23 Franc CFA, Senegal’s unit of currency. 12 | YAAJEENDE FINAL IMPACT EVALUATION REPORT USAID.GOV CONSTRUCTION AND VIABILITY OF COMPARISON GROUP Under a quasi-experimental impact evaluation design, project impacts are determined by drawing on outcome information collected from a group of project beneficiaries (the treatment group, or in this case, households in Yaajeende project villages), and the same set of information collected from a group of comparable households and individuals that did not receive the treatment (known as the comparison group). The comparison group serves as the counterfactual for the treatment group, providing information on what would have happened to households and individuals in the treatment group, had they not received the project intervention. For the impact analysis to be credible and robust, households in the comparison group should be as similar as possible to those in the treatment group across key characteristics that also influence the outcomes of interest under the project. In the context of the Yaajeende evaluation, examples of such characteristics include household factors such as the household’s poverty status prior to the start of project activities, overall household size and the maximum level of education in the household. All of these characteristics may influence the likelihood of household or individual interest or ability to participate in Yaajeende activities, as well as the extent to which project activities may bring about the desired changes in outcomes for the household. It is also important for comparison group villages to have contexts that are broadly similar to treatment group villages for a similar set of reasons, and because development projects often choose to implement activities in areas that meet certain implicit or explicit characteristics or criteria, which may also influence project results. For example, Yaajeende was targeted to areas with higher poverty and malnutrition. This so-called selection bias can be a source of confounding around the true effect of a project, if strong differences exist in the underlying context for the treatment and comparison groups and the available analytic steps to correct for these differences are not effective. To account for nonrandom treatment assignment at midline, the MIE team used a sampling approach in which comparison villages were selected from rural communities (communautés rurales, or CRs) adjacent to treatment CRs. In doing so, the midline team assumed that the parallel trends assumption required for a DID design, in which the treatment and comparison units follow similar broad trends over time, would be more likely to hold due to the geographic proximity of the treatment and comparison villages. The final evaluation team assessed the viability of the comparison group to serve as a valid counterfactual and determined that although some differences in household context exist across the treatment and comparison group, common statistical matching techniques effectively produce a balanced and comparable sample for a credible DID analysis. At endline, the final evaluation team also included village-level context factors in the matching approach, which approximate broader village context. However, contamination of comparison areas by other donor projects arose as an important constraint on comparability at endline. A new survey module the FIE team added at endline provided strong indication that many households in comparison group villages had been exposed to similar activities conducted by other donors during the midline-endline period. This was further confirmed by qualitative data collected in comparison villages, while qualitative data collected in treatment villages also provided evidence that many Yaajeende villages are also affected by other agriculture and food security, WASH, and/or MCHN programming by other donors during the Yaajeende project lifetime (see annex I for a summary of overlapping donor activity in Yaajeende villages, and similar programming in comparison villages). Given the evidence for alternative donor programming in comparison villages, the comparison case for this evaluation changes from one focused on determining the effects of NLA Yaajeende programming relative 13 | YAAJEENDE FINAL IMPACT EVALUATION REPORT USAID.GOV to comparable households that received no programming, to measuring the effects of Yaajeende programming relative to households that were exposed to similar types of agricultural, WASH, and/or MCHN donor programming during the same time period. STUDY POWER Power calculations indicated that the Yaajeende FIE is sufficiently powered to detect programming￾relevant effect sizes24 for nearly all of the impact/outcome indicators listed in Table 2, and for the two subgroup analyses. The final evaluation has lower power to detect small program effects, meaning that it may not be sufficiently powered to distinguish a small, true program impact from no impact for many of the outcome variables. This is not viewed as a major limitation, given USAID’s expectation of relatively large program impacts at endline. ATTRITION The attrition rate for households was 7.8 percent from midline to endline, and 16.5 percent from baseline to midline. In the event of household non-response or inability to be located at endline, the team replaced that household with a randomly selected household from the village that met at least one of the following eligibility criteria: (1) presence of at least one child aged 0-59 months or (2) presence of at least one woman between the ages of 15-49 years old. Randomized selection of replacement households was done using the household list for the village obtained from the village chief, or using a random village walk method specified for enumerators if the village list was unavailable. ANALYTIC SAMPLE The analytic sample for this evaluation consists of 2,470 households surveyed across 157 communities, distributed as Table 3 shows. Annex II describes demographic characteristics of the sample. At endline, treatment group heads of households averaged 56.3 years old and household size was 13.8 people. Female￾headed households accounted for 17.8 percent of the sample. For the comparison group, average age for heads of household at endline was 50.3 years old, the average household comprised 10.8 individuals, and females headed 8.9 percent of households. In total, the sample comprised 31,316 individuals at endline, including 24,268 individuals in Yaajeende villages and 7,048 in comparison villages. This includes 5,648 women aged 15-49 in Yaajeende villages, versus 1,513 in the comparison group. There were 3,756 children under 5 years old in treatment villages, and 1,167 in comparison villages. ANALYTIC APPROACH DIFFERENCE-IN-DIFFERENCES The evaluation team used a difference-in-differences (DID) regression approach, coupled with statistical matching, to estimate the program’s impact on outcomes of interest. The FIE adopts three statistical approaches to estimate average treatment effects of the Yaajeende interventions on the four outcome families: a difference-in-differences (DID) approach using village fixed effects and a binary treatment variable, a non-parametric entropy balancing DID approach, and a DID approach with village fixed effects 24 Defined as medium to large effective sizes, with minimum detectable effect size (MDES) ranging from .30 and greater. 14 | YAAJEENDE FINAL IMPACT EVALUATION REPORT USAID.GOV and a continuous treatment intensity variable based village-level exposure to different training types covered by the Yaajeende program and similar donor efforts present in Yaajeende or comparison villages. The DID models include covariates to control for observed differences in treatment and comparison groups, and village fixed effects to control for time-invariant unobservables. The endline analyses adopt a standard cross-sectional regression approach to take advantage of all available observations. Such models estimate the population average change due to the program, and treats the panel observations in the dataset as repeated cross-sections. Rerunning the models with only the panel households in the dataset may further corroborate results or yield additional insights that are not captured by the cross-sectional analysis. However, it is also possible that the lower statistical power of the panel models, due to fewer observations, may limit the possibility of detecting statistically significant treatment effects for the panel dataset. This is particularly a concern for comparisons done from baseline to endline, rather than midline to endline, because the baseline sample was determined to be underpowered at midline, and the sample was substantially expanded at that time. The sample consists of 1,330 households at baseline, 2,514 households at midline, and 2,470 households at endline. Of these, 1,036 households form a panel from baseline to endline and 2,318 households form a panel from midline to endline. Due to the expansion of the household and village sample at midline, a greater portion of households in the sample only constitute a panel across the midline and endline survey rounds. Given these limitations, the evaluation team ran analyses on the panel dataset as a form of robustness check on the cross-sectional results. The DID model takes the following generic form: 𝑌ℎ𝑣𝑡 = 𝛼 + 𝛾(𝑇𝑟𝑒𝑎𝑡𝑚𝑒𝑛𝑡𝑣 ) + 𝜆(𝐸𝑛𝑑𝑙𝑖𝑛𝑒𝑡 ) + 𝛿𝐷𝑣𝑡 + 𝜖ℎ𝑣𝑡 (1) Where subscript h denotes household, v is village and t is time. 𝑌ℎ𝑣𝑡 is the value of the outcome for household h, in village v, and at time t. 𝛾 is the effect of treatment at baseline. Treatment is 1 for program beneficiaries and 0 for comparison households, with variation at the village level. 𝐸𝑛𝑑𝑙𝑖𝑛𝑒𝑡 is a dummy to denote the endline time period and λ is the effect of the time dummy. Dst is a dummy variable defined by interacting treatment and endline; the variable takes a value of 0 for households in control villages and all observations in the baseline period, and a value of 1 for endline treatment households. Under standard assumptions, δ provides an unbiased estimate of the causal impact of the Yaajeende activity on the outcome, 𝑌ℎ𝑣𝑡, and measures the intent to treat (ITT) estimate for the Yaajeende analysis. 𝜖ℎ𝑣𝑡 is an error term and 𝛼 is a constant. The workhorse model takes the generic DID form in Equation 1 and adds in a fixed effect for village: 𝑌ℎ𝑣𝑡 = 𝛼 + 𝜆(𝑇𝑖𝑚𝑒𝑡 ) + 𝛿𝐷𝑣𝑡+𝜕𝑣 + 𝜖ℎ𝑣𝑡 (2) where 𝜕𝑣 is the village-level fixed effect for village v. The term 𝑇𝑟𝑒𝑎𝑡𝑚𝑒𝑛𝑡𝑣 is absorbed by the village fixed effect and drops out; 𝛿 continues to be the primary coefficient of interest, estimating the ITT treatment effect of the program. This model has the same basic intuition as a standard DID model in Equation 1, but has the added benefit that the village fixed effect term absorbs all time-invariant village-level characteristics that may influence outcomes, such as cultural practices or soil quality. ENTROPY-WEIGHTED DID To overcome the confounding effects of potential selection bias, both in terms of villages selected for Yaajeende interventions and household self-selection into Yaajeende activities within those villages, the 15 | YAAJEENDE FINAL IMPACT EVALUATION REPORT USAID.GOV evaluation team used entropy-balancing as a form of matching, to improve comparability of treatment and comparison groups and mitigate observable bias. 25 The aim of pre-processing the data via entropy￾weighting is to improve covariate balance between the treatment and comparison groups, so that the comparison group has a more similar distribution to the treatment group on observed characteristics that are viewed to influence outcomes. The evaluation team matched on a set of household-level characteristics that could be related both to a household’s interest to participate in and benefit from Yaajeende interventions, and village-level characteristics that indicate broader village context and market access. Household-level covariates used were: household head’s education, whether or not the household head is literate, age of the household head, and number of household members. Travel time from the village to the nearest population center of at least 20,000 people indicated market access. 26 For individual-level regressions, the study also included individual’s age and gender as individual-level covariates in the matching. SUBGROUP DIFFERENCES IN TREATMENT EFFECT To explore heterogeneity in program effects, the evaluation team tested for differences in binary treatment effect across two key beneficiary subgroups of interest: region (the two northern areas of Matam Region and Bakel Department in Tambacounda Region vs. the southern region of Kédougou), and household poverty status (poorest households vs. remaining households). Such analyses are important to conduct, because it is possible that even if overall treatment effects are not significant, some significant effects may still exist for certain types of program areas or households, relative to others. Examination of regional differences is particularly of interest for this evaluation because Kédougou Region is a substantially different implementation context relative to the two northern regions of implementation, particularly in terms of climate, economy, language, infrastructure and international borders. Subgroup analyses with respect to household poverty status provide an understanding of if and how program effects may vary for poorest households relative to other households. This is important to explore because Yaajeende is designed around improving development outcomes for most vulnerable households. To determine how impacts vary by binary subgroups of interest, a separate set of models are estimated that include an additional interaction term between treatment assignment and a dummy to indicate subgroup status. Next, a triple interaction term is created by multiplying the endline time dummy, treatment, and the subgroup dummy; the coefficient on this interaction term is used to obtain the point estimate of the difference in the average treatment effect across households in the two subgroups. DOSE-RESPONSE EFFECT OF PROGRAM PARTICIPATION To gain additional understanding of how the Yaajeende project impacted households in Yaajeende villages that directly participated in project activities, the evaluation’s modeling approach uses a continuous, rather than binary, treatment estimator. This enables examination of the dose-response effect of Yaajeende project participation using a treatment variable that represents participation intensity at the household 25 Hainmueller, J. 2012. “Entropy Balancing for Causal Effects: A Multivariate Reweighting Method to Produce Balanced Samples in Observational Studies”. Political Analysis, 20(1):25-46; Imai, K., and M. Ratkovic. 2014. “Covariate Balancing Propensity Score”. Journal of the Royal Statistical Society 76:243-263; Zubizarreta, J. 2015. “Stable Weights that Balance Covariates for Estimation With Incomplete Outcome Data.” Journal of the American Statistical Association, 110(511):910-922. 26 HarvestChoice, 2015. “Travel time to nearest town over 20K (mean, hours, 2000).” International Food Policy Research Institute, Washington, D.C., and University of Minnesota, St. Paul, MN. Available online at http://harvestchoice.org/data/tt_20k. 16 | YAAJEENDE FINAL IMPACT EVALUATION REPORT USAID.GOV level, rather than simple presence or absence of Yaajeende implementation in the village. This dose￾response model takes into account different levels of exposure and participation in Yaajeende activities at the household level, and may be able to isolate differences in program effects, which results from increasing levels of exposure to Yaajeende activities. The household-level intensity variable draws on questions added to the endline household survey that ask about household participation in different types of training activities, and constructs for each household a variable ranging from 0 to 8, indicating the total number of different training types the household directly participated in over the project time frame. This specification takes the following general form: 𝑌ℎ𝑣𝑡 = 𝛼 + 𝜆(𝐸𝑛𝑑𝑙𝑖𝑛𝑒𝑡 ) + 𝛿𝐷𝑣𝑡 + 𝜌𝐸ℎ𝑣𝑡 + 𝜑𝑅ℎ𝑣 + 𝜏𝑇ℎ𝑣𝑡 + 𝜕𝑣 + 𝜖ℎ𝑣𝑡(3) Parameters 𝑌ℎ𝑣𝑡, 𝐸𝑛𝑑𝑙𝑖𝑛𝑒𝑡 , 𝐷𝑣𝑡, 𝜕𝑣 , and 𝜖ℎ𝑣𝑡 were as defined previously. 𝜌 measures the effect of 𝐸ℎ𝑣𝑡, which is a term that interacts 𝑇𝑟𝑎𝑖𝑛𝑖𝑛𝑔𝑠ℎ𝑣 , the number of training types a household attended\, and 𝐸𝑛𝑑𝑙𝑖𝑛𝑒𝑡 . 𝜑 measures the effect of 𝑅ℎ𝑣 , which is a term that interacts 𝑇𝑟𝑎𝑖𝑛𝑖𝑛𝑔𝑠ℎ𝑣 and 𝑇𝑟𝑒𝑎𝑡𝑚𝑒𝑛𝑡𝑣 . Finally, our main parameter of interest is 𝜏, measuring the effect of 𝑇ℎ𝑣𝑡, a triple interaction term that interacts 𝑇𝑟𝑎𝑖𝑛𝑖𝑛𝑔𝑠ℎ𝑣 , 𝑇𝑟𝑒𝑎𝑡𝑚𝑒𝑛𝑡𝑡 , and 𝐸𝑛𝑑𝑙𝑖𝑛𝑒𝑡 . Thus, 𝜏 measures the impact of having attended an additional training type on the outcome of interest. QUANTITATIVE DATA COLLECTION SAMPLE The target population for the FIE data collection at endline was households in villages that were surveyed at midline. At baseline, the analytic sample for the Yaajeende evaluation consisted of 1,110 households across 68 activity and 63 comparison villages in Bakel, Kédougou and Matam. Village selection was stratified by zone and CR, with 10 households surveyed per selected village. At midline, due to concerns about study power, 27 additional intervention villages were randomly selected from activity CRs and included in the midline sample. 27 Also, the number of households surveyed per village was expanded from 10 to 17 to improve study power. This resulted in an analytic sample at midline of 2,514 households across 94 Yaajeende villages and 63 comparison villages. Figure 4 shows the geographic distribution of villages in the sample, and the expansion of villages across waves. Comparison villages are in neighboring communes to Yaajeende implementation areas, but are geographically clustered up to 100 kilometers from Yaajeende treatment villages in the sample in Matam and Bakel. In Kédougou, comparison villages in the sample are more interspersed with Yaajeende treatment villages, separated by an average distance of around 10 kilometers. At endline, the treatment status for 23 “partially treated” villages was updated from comparison group to treatment group, on the basis on additional information provided by program implementers. 28 The net effect of this was an increased imbalance in treatment group villages and households relative to the comparison group. But the inclusion of the partially treated villages in the treatment group had no material effect on results. 27 Power analyses conducted at endline confirm that the baseline sample was underpowered to detect program effects for many outcomes. 28 Sixteen of these were villages that received Yaajeende interventions for one to three years during 2011-2014, before the program withdrew due to implementation challenges. Such villages are considered partially treated for this evaluation, which aims to understand average program effects across the range of typical implementation contexts for such programming, and not just those where implementation proceeds smoothly. The evaluation team ran endline results with and without the partially treated villages in the sample, and found no material differences in results. 17 | YAAJEENDE FINAL IMPACT EVALUATION REPORT USAID.GOV FIGURE 4. MAP OF YAAJEENDE FINAL IMPACT EVALUATION VILLAGE SAMPLE 18 | YAAJEENDE FINAL IMPACT EVALUATION REPORT USAID.GOV Figure 5 is a map that the Yaajeende project compiled, showing the Yaajeende zones of intervention across 49 communes, and distinguishing between communes where implementation covered 100 percent of villages (dark green), 76-99 percent of villages (light green), 50-75 percent of villages (orange) or less than 50 percent of villages in the commune (red). Sampling for the IE covered all intervention zones except those in Kolda Region. FIGURE 5. MAP OF YAAJEENDE ZONES OF INTERVENTION29 TABLE 3: YAAJEENDE QUANTITATIVE EVALUATION SAMPLE AT BASELINE, MIDLINE AND ENDLINE DEMOGRAPHIC BASELINE (2011) MIDLINE (2015) ENDLINE (2018) Bakel Matam Kédougou Overall Bakel Matam Kédougou Overall Bakel Matam Kédougou Overall TREATMENT GROUP Village N 20 33 35 88 25 49 41 115 25 49 41 115 Household N 200 330 351 881 411 776 656 1843 407 775 648 1830 Children 0-6 months (infants) 50 46 52 148 112 166 92 370 75 98 99 272 Children 6-23 months (under 2) 135 229 168 532 377 544 329 1250 323 533 339 1195 Children 6-59 months (under 5) 412 580 458 1450 1234 1781 1042 4057 1052 1631 1073 3756 Women 15-49 years 632 856 710 2198 1642 2297 1224 5163 1600 2632 1416 5648 COMPARISON GROUP Village N 12 15 15 42 12 15 15 42 12 15 15 42 Household N 120 150 150 420 183 228 244 655 180 223 237 640 29 NCBA CLUSA. 2017. “Feed the Future Yaajeende Final Report”. USAID report. 19 | YAAJEENDE FINAL IMPACT EVALUATION REPORT USAID.GOV DEMOGRAPHIC BASELINE (2011) MIDLINE (2015) ENDLINE (2018) Bakel Matam Kédougou Overall Bakel Matam Kédougou Overall Bakel Matam Kédougou Overall Children 0-6 months (infants) 24 21 14 59 63 39 45 147 42 27 27 96 Children 6-23 months (under 2) 60 68 41 169 187 107 108 402 182 89 79 350 Children 6-59 months (under 5) 198 207 139 544 609 394 342 1345 528 339 300 1167 Women 15-49 years 246 227 257 730 621 375 396 1392 644 431 438 1513 Table 3 shows the baseline, midline and endline sample distribution, overall and by region, in the number of villages, households and individuals in each of the four age categories in the treatment and comparison groups. Sample numbers in Table 3 are based on the revised treatment status for several villages that took place at endline, and accounts for information on partially treated villages, thus reflecting a greater sample imbalance across treatment and comparison groups than the initial design for this evaluation allowed. The household sample at endline replicated the same multi-stage clustered sampling design used at baseline and midline, and revisited the villages that were surveyed at midline in the three regions. Per the midline sample structure, 17 households were surveyed per village, with the aim of revisiting the households in each village that were surveyed at midline. In the event of household non-response or inability to locate, households were replaced following a randomized replacement protocol. TIMING Endline data collection took place during a six-week period in March and April 2018. The baseline and midline data collection took place during May and June during a post-harvest period prior to the rainy season and the onset of the lean season. Although it is preferable to replicate this timing for the endline data collection, this was not possible for the endline data collection, due to the timing of evaluation activities. Interpretation of the endline results takes into account that the endline data collection occurs slightly earlier into the lean season than the baseline and midline. As a result, some food consumption and dietary diversity indicators under the status quo could be more positive at endline, depending on household situations and program effectiveness, although the impact estimates are not sensitive to this difference since it affects both the treatment and comparison groups. Longer-term impacts such as child stunting are not anticipated to be strongly sensitive to this difference in the timing of data collection. HOUSEHOLD SURVEY, ADMINISTRATION AND SURVEY PLATFORM The Yaajeende endline data collection used the same household survey instrument that was used at midline, consisting of the following modules: household roster, household assets, revenue sources, surface area cultivated, agriculture and livestock production, debts and financial services, participation in Yaajeende activities, food consumption, nutrition and health and anthropometry. The modules target different household respondents, including head of household, woman in charge of household and women in household with children under age 5. Based on an exploratory analysis of the baseline and midline data during Phase I and instruments review, the endline household survey instrument was modified to add gap-filling and clarification questions to 20 | YAAJEENDE FINAL IMPACT EVALUATION REPORT USAID.GOV improve the ability for the endline analysis to accurately measure and detect program impacts. Changes were also made with the intent of reducing the overall survey burden time for respondents. QUALITY CONTROL AND DATA CLEANING The evaluation team was in regular communication with the survey teams during survey fielding, and tracked sample accumulation against the master sample list through an established reporting system. Quality control measures employed during survey fielding included daily data uploads and review during quantitative data collection, high-frequency validation and reliability checks, checks for enumerator inconsistencies and weekly summary production reports. In total, the evaluation team produced 11 data quality review reports during the six weeks of survey fielding. In addition, field supervisors were required to conduct spot checks on 20 percent of surveys in each village, and a separate back-check quality control process required that an independent team re-interview 10 percent of the household survey sample across 58 of the 158 villages in the sample. Back-check data audits occurred for consistency against the corresponding survey responses and assessed overall survey quality on key questions and track enumerator performance. Additional quality measures took place during the data processing stage. QUALITATIVE DATA COLLECTION AND ANALYSES Qualitative data collection for the FIE occurred in May and June 2018, consisting of 45 group discussions (GDs) with project beneficiaries or other men and women in surveyed villages, and 70 key informant interviews (KIIs) in villages or at the commune level with various Agent de service préstataire (APS), Volontaire pour la nutrition communautaire (VNC), Groupe de travail citoyen (GTC), Comités locaux du pilotage (CLP) and government nutrition or health focal points. In total, qualitative data collection took place in 18 Yaajeende villages (six per region) and nine comparison group villages (three per region). Village selection was stratified by region and village treatment status, so that six Yaajeende villages and three comparison group villages were visited per region. Table 4 describes the target qualitative sample at endline in Yaajeende treatment villages. GD targets in Yaajeende villages were generally met, while the number of KIIs conducted was exceeded, with an average of 20 KIIs in each region. Qualitative data collection also took place in comparison group areas. GDs and KIIs were conducted in three comparison group villages per region, for a total of nine comparison group villages in the qualitative sample. In each comparison group village, one GD took place with women in the village, a community profile and KII occurred with the village head and one or two additional KIIs were conducted with volunteers from alternative donor programs that may have been present in the village, where applicable. Annex III contains an anonymized list of all GD and KIIs conducted at endline. Village selection for the qualitative data collection was done purposively, informed by the household survey data, and aimed to maximize variation in qualitative data coverage across key program participation, context factors and village-wide averages on key women’s and children’s nutritional status and diet outcomes. This ensured that the evaluation team obtained information from respondents across the range of village contexts covered by the project, and for which village-wide project participation and outcomes at endline represented a distribution across high, low and average outcomes. With participants’ consent, the evaluation team recorded GDs and then transcribed them from the local language into French. Transcribed GD and KII data were then coded in NVivo according to a predefined codebook, designating text segments according to key themes of interest with respect to project implementation and the evaluation questions. The data were then summarized using standard content 21 | YAAJEENDE FINAL IMPACT EVALUATION REPORT USAID.GOV analysis techniques, focused on common themes and patterns to highlight project-, sector-, and gender￾differentiated trends, and to identify examples of positive deviance and significant change. TABLE 4. QUALITATIVE SAMPLE TARGETS AT ENDLINE: YAAJEENDE TREATMENT VILLAGES REGION / ZONE YAAJEENDE TREATMENT GROUP: TARGET SAMPLE (18 Yaajeende Treatment Villages ; 6 villages per region) KEY INFORMANT INTERVIEWS GROUP DISCUSSIONS Bakel 12-15 KIIs with village-level and commune-level project stakeholders, prioritizing APS, VNC and GTC members, and nutrition and health focal points in government 6 GD with women Debbo Gallé members 6 GD s with other women in Yaajeende villages 3 GDs with men in Yaajeende villages Matam 12-15 KIIs with village-level and commune-level project stakeholders, prioritizing APS, VNC and GTC members, and nutrition and health focal points in government 6 GD with women Debbo Gallé members 6 GD s with other women in Yaajeende villages 3 GDs with men in Yaajeende villages Kédougou 12-15 KIIs with village-level and commune-level project stakeholders, prioritizing APS, VNC and GTC members, and nutrition and health focal points in government 6 GD with women Debbo Gallé members 6 GD s with other women in Yaajeende villages 3 GDs with men in Yaajeende villages Treatment Sample 36-45 KIIs 36 Group Discussions LIMITATIONS SELECTION BIAS The primary drawback of quasi-experimental designs is that they involve a risk of selection bias, such that the differences in outcomes between the treatment and control groups may be the result of unobserved systematic differences between the two groups rather than a causal impact of the intervention. A limitation of this approach is that the estimation of impacts can be biased if unobserved trends selectively affect only the treatment or comparison group. The FIE team took several steps to assess and mitigate this potential limitation, including collecting additional information on the endline household survey to assess whether major time-varying confounders may be present; employing alternative model specifications to test robustness of results; and combining the DID analysis with statistical matching, a common approach to reduce sources of bias and improve the precision of the impact estimate. Village selection for the qualitative data collection aimed for representativeness to the extent possible, but purposive qualitative data collection is inherently non-representative. Moreover, respondents who participated in GDs and were willing to share their views may not be representative of all project participants, or may have key observable or unobservable differences. The ET sought to mitigate the potential for biased qualitative results by recruiting respondents with a range of experiences and beneficiary roles for the qualitative data collection and by triangulating information across types of project beneficiaries and other stakeholders. RECALL BIAS Recall and response bias are potential limitations for any qualitative or quantitative data collection effort. Some evaluation topics, such as perceptions about the beneficiary selection processes that occurred early 22 | YAAJEENDE FINAL IMPACT EVALUATION REPORT USAID.GOV in program implementation, may be difficult for respondents to remember accurately as time passes. Recall bias may lead to exaggerated negative or positive perceptions of past experiences, as people tend to remember only key aspects or feelings over time. Careful construction of the wording of questions on interview guides, probing for clarification and triangulation across GDs and KIIs mitigated the potential for recall bias to influence results. OTHER THREATS TO DID VALIDITY The complications apparent in the DID design for this impact evaluation, including selection bias in the types of households and villages selected for Yaajeende project implementation, and contamination of comparison group villages by similar development programming by other donors, renders the DID results less reliable on their own. To help overcome some of these limitations, the FIE team drew on an iterative modeling approach to build greater confidence in results and their consistency across alternative specifications. Entropy-weighting is used as a statistical matching approach to improve similarity between treatment and comparison villages and households, and yields more reliable results with respect to determine Yaajeende project effects. In addition, the evaluation team’s primary alternative approach, based on intensity of treatment exposure, provides a complementary and more nuanced understanding of how varying levels of household participation and average village participation for this type of integrated programming relates to changes in the outcomes. For the baseline to endline analyses of impacts, the results are considered to have lower reliability due to the smaller number of households and villages in the baseline sample, resulting in an underpowered baseline, and differing village sample and survey instrument used at baseline. FINDINGS This section presents the integrated quantitative impact results and companion qualitative findings for the Yaajeende FIE. It begins with an overview of the key findings, and presents a summary table of evidence for results by each of the four outcome families. Results for the impact of Yaajeende programming on key outcomes of interest are presented based on three sets of difference-in-differences (DID) models: (1) DIDs using village fixed effects and controlling for key household characteristics; (2) entropy-weighted DIDs using village fixed effects and controlling for key household characteristics; and (3) DIDs that exploit the household panel by employing household-level fixed effects. Entropy-weighting is implemented as a form of matching to additionally control for observable confounding factors and selection bias related to the village contexts where Yaajeende was chosen for implementation, and is done to improve the comparability between Yaajeende and comparison villages for the DID approach. The results summarized by outcome family aim to concisely present findings for evaluation questions (EQs) 1 through 5 for each of four outcomes families, in the following order: • (EQs 1 and 2) Overall DID and alternative model results and potential drivers of impact for each outcome family. This section presents the analytic impact results (EQ1) for midline to endline across the DID and entropy-weighted DID models, for both the binary Yaajeende treatment variable and the treatment variable based on intensity of treatment at the household level. Results focus on midline to endline effects, but baseline to endline results are also presented. The impact 23 | YAAJEENDE FINAL IMPACT EVALUATION REPORT USAID.GOV results are integrated with qualitative results on drivers or factors that contributed to observed results (EQ2). • (EQs 3 and 4) Results on subgroup analyses for heterogeneity of impacts and potential drivers: Impact results are reported by region and household wealth status (EQ3), integrated with qualitative results on potential drivers of heterogeneous impacts (EQ4). • (EQ 5) Moderating context factors: Presents a summary of if and how key individual or household characteristics are associated with program effects. • Overarching conclusions by outcome family. The ensuing sections in this chapter present findings for EQ6, which summarizes analyses to identify characteristics of households associated with household-level poverty reduction, and characteristics of households and mothers that are associated with malnutrition reduction for children under age of give and women of reproductive age; and EQ7, on unintended positive or negative broader consequences of the Yaajeende interventions as identified through qualitative data collection. OVERVIEW OF KEY FINDINGS In general, the DID results find little evidence of positive improvements for several of the FIE outcomes as a result of Yaajeende programming, relative to comparison group villages in nearby communes with varying levels of similar integrated WASH, nutrition and agricultural programming. In some cases, outcomes did improve in Yaajeende areas between midline and endline, but households in comparison areas experienced similar or greater levels of improvement on those outcomes during the same time period. The net of this trend through a DID analyses is either no additional impact as a result of Yaajeende programming or, for a small number of outcomes, a negative effect from the project relative to the comparison areas. The FIE finds beneficial impacts as a result of the program on two key women’s and children’s nutritional status indicators (a 5.6 percentage point decrease in prevalence of women underweight, and a 2.5- 8.0 percentage point increase in the prevalence of MAD), a 0.8 to 2.8 percentage point reduction in the likelihood of poverty at the household level, an increase in agricultural investment and an increase in agricultural production. In most cases, the magnitude of these increases are moderate. But these impacts usually occur against an overarching context of general gains on the same (for which Yaajeende programming had additional impacts above the background trends), or within a context of decline, in which Yaajeende programming shows evidence of helping households mitigate overarching negative stresses. Relative to a comparison situation of similar programming efforts on women’s and children’s health, nutrition, WASH and agricultural support, no evidence for added Yaajeende project impacts is found for healthy household practices such as common use of a handwashing station or use and proper storage of iodized salt. Yaajeende and comparison households alike improved on these indicators during the project lifetime, but gains were higher in comparison areas. Some evidence indicates that varying household-level exposure to program trainings and activities moderates overall program impacts across Yaajeende villages. The FIE finds that villages with higher average village-wide exposure and participation in trainings on issues promoted by Yaajeende experience a 3 to 6 percentage point decline in the prevalence of children underweight, a 3-percentage point decline in the stunting rate and a stronger reduction in the likelihood of poverty at the household level. 24 | YAAJEENDE FINAL IMPACT EVALUATION REPORT USAID.GOV Although the FIE focuses on midline to endline results due to power limitations and lower reliability of the baseline data, the baseline to endline impact results confirm and follow the same trend on the outcomes for healthy household practices and agricultural practices. They also confirm and find stronger impacts than did the midline to endline period for the household economic well-being results. The baseline to endline results do not find evidence of positive Yaajeende effects for any women’s nutritional status and diet outcomes, but the analyses are underpowered to detect a small significant effect if it is present. Also, many FIE findings are consistent with the pattern of outcomes found at midline through the MIE analyses, including a similar set of constraints on wider impacts, as obtained through qualitative data collection. The qualitative data sheds some light on potential contributing reasons for overarching impacts. There was a widespread view of positive changes in children’s health, and a reduction in children’s malnutrition, as a result of Yaajeende project activities. This had similar levels of support across each of the three regions covered by the evaluation. Key drivers of impacts or lack thereof, focused on continued lack of financial and material resources at the household-level to implement or sustain Yaajeende-promoted activities, particularly with respect to inputs needed for effective agricultural production and gardening activities. Sufficient and reliable water access remained a key constraint for broader agricultural production and market gardening gains throughout the project areas. While respondents widely viewed the introduction of community or microgardens as beneficial, they reported several challenges with a focus on the lack of or insufficient access to several required inputs and insufficient water, a near ubiquitous limitation mentioned for all agricultural activities assessed. In some areas, particularly in Kédougou, respondents viewed market oversaturation as a constraint on higher agricultural revenues. Respondents identified insufficient breast milk production by mothers and lack of time to comply with optimal feeding practices due to women’s schedules and labor needs as key factors that continue to stymy wider implementation of optimal breastfeeding practices, despite a strong knowledge of associated health benefits. The FIE found modest gains in reducing household poverty likelihood through the project and suggestive evidence that this relates to increased agricultural production. But these effects varied widely and evidence of increased revenue as a result of production gains or stronger value chain participation is limited. Table 5 provides a summary of the body of evidence for Yaajeende project impacts available through the FIE analyses. Summary statistics for outcomes by survey round and treatment group, on which the impact analyses are conducted, are in Annex II. That annex also provides outcome means and t-tests for difference across baseline, midline and endline for Yaajeende treatment villages only, showing trends on indicators in Yaajeende villages over the project lifetime. 25 | YAAJEENDE FINAL IMPACT EVALUATION REPORT USAID.GOV TABLE 5. SUMMARY TABLE OF EVIDENCE FOR YAAJEENDE PROGRAM IMPACTS OUTCOME FAMILY QUANTITATIVE EVIDENCE QUALITATIVE EVIDENCE Women’s nutritional status and diet Prevalence of women underweight WDDS30 Children’s nutritional status and diet Wasting Stunting Underweight MAD EBF / Optimal breastfeeding Fairly strong evidence for a modest decrease in prevalence of women underweight and some evidence for a modest increase in prevalence of MAD, as a result of Yaajeende programming. For villages with higher village-wide exposure and participation in trainings (regardless of Yaajeende-led or other), there is evidence of a modest decline in prevalence of children underweight (3-6 percentage points), and in prevalence of stunting (3 percentage points). For remaining indicators in this outcome family, there is limited or no evidence of positive impact on these outcomes as a result of Yaajeende programming, relative to a comparison situation of similar programming on women and children’s health, nutrition and WASH by other donors. There is evidence of differences by region. Effects show moderate reductions in stunting and underweight in children for Kédougou, and a small decrease in prevalence of underweight in women for Matam and Bakel. There is evidence of differences by household poverty status. Impacts are seen only for less poor households. BL-EL results find no evidence of positive treatment effects. The impacts seen in ML-EL results for underweight and MAD were not confirmed in BL-EL analyses. Respondents expressed a view of positive change with respect to children’s health and reduced malnutrition as a result of project activities. Some respondents noted that vitamin-rich foods are not always available, or accessible only to families that have money to buy them at market. Lack of means as a limiting factor to realizing health and nutrition gains was commonly expressed. There is strong evidence of knowledge gain on optimal breastfeeding practices and health effects, but insufficient milk production and lack of time required to implement this practice are key reasons for limited implementation. Respondents in beneficiary villages largely indicated that putting this knowledge into practice, especially with respect to improving diets of women and children, was largely driven by household means. Lack of means to grow or buy sufficient quantities of nutritious foods is still a key limiting factor for many households. Household food security and economic well-being HDDS31 Hunger season (soudure) Likelihood of being below poverty line Agricultural revenue Perceived household well￾being Strong evidence for a small decrease in likelihood of poverty (0.8-2.8 percentage points). For households with more direct exposure to Yaajeende trainings, there is moderate evidence for stronger reductions in likelihood of poverty. For remaining indicators in this outcome family, there is weak or no evidence of positive program impacts. There are differences by region, with evidence of stronger declines in poverty as a result of the program in Matam and Bakel (2.1 percentage points) than in Kédougou (no effect). There is no evidence that Yaajeende had differential impacts on poorer households for indicators in this outcome family. There is moderate evidence that households with more direct exposure to Yaajeende-led trainings experienced an increase in agriculture revenue. Respondents report uneven gains on expanded access to different types of nutritious foods throughout the year, although the introduction of community or microgardens is widely seen as beneficial for improving access and follow-on benefits to nutrition and health. Respondents report widespread challenges with gardens, mainly focused on lack of required inputs and insufficient water. Ongoing challenges with hardship during the lean season continue to be noted, but Yaajeende established systems such as greniers des enfants were viewed as helping ease malnutrition. Reports of increased agricultural revenue were generally limited, and it is still dependent on whether the harvest is successful or not. Other constraints included marketing and 30 Women’s Dietary Diversity Score 31 Household Dietary Diversity Score 26 | YAAJEENDE FINAL IMPACT EVALUATION REPORT USAID.GOV OUTCOME FAMILY QUANTITATIVE EVIDENCE QUALITATIVE EVIDENCE BL-EL results find stronger impacts than ML-EL results. BL-EL confirms the finding for an impact on likelihood of poverty with a stronger predicted reduction (5 percentage points). BL-EL results also find a moderate increase in agriculture revenue. transport challenges, while market oversaturation was also noted to limit potential revenues. Healthy household practices Handwashing station Use and storage of iodized salt There is no evidence of positive impact on these outcomes as a result of Yaajeende programming, relative to a comparison situation of similar programming on women and children’s health, nutrition and WASH by other donors. There is no evidence for program effects on indicators in this outcome family by region or household wealth status. There is no evidence that the program had differential impacts in villages or households more exposed to Yaajeende-led trainings. BL-EL findings confirm the null results detailed above for this outcome family. Qualitative data collection at endline indicated that respondents in Yaajeende villages had been well-sensitized on WASH issues and had put many of the practices in place, including with respect to testing, use and storage of iodized salt. However, there was ample evidence from comparison group villages of similar activities and perceived improvements on WASH issues, tied to other donor programs in those villages. Yaajeende beneficiaries and stakeholders indicated that tippy￾taps were not always seen as the preferred option, and many eventually came into disuse. Upkeep and maintenance was seen as time-consuming, and household distance to water points was a constraint for keeping the taps replenished. Health focal points indicated that additional and more dedicated follow-ups were needed to institutionalize their use more widely. Household agricultural practices Agricultural investment Use of CBSP Production of targeted commodities Garden access Value chain participation There is moderate to strong evidence that Yaajeende increased investment in agriculture, relative to trends in comparison villages. There is weak to moderate evidence that Yaajeende increased agriculture production. No evidence showed that Yaajeende increased use of CBSPs. Regional regressions show an increase in agriculture production in Matam and Bakel as a result of the program (524 kg), but not in Kédougou. Moderate to strong evidence for stronger effects in poorer households on agriculture investment and production. There is no evidence that the program had outsized positive impacts in villages or households most exposed to Yaajeende-led trainings. BL-EL analysis can be done only for agriculture production and it shows a large increase in treatment households relative to trends in comparison villages (631 kg), confirming an effect. Fairly widespread reports of increased agricultural yields that they attributed to farming practices they learned through Yaajeende interventions, but more commonly stated in Kédougou and Matam than in Bakel. Participants in Yaajeende programming reported that members of their community know how to farm more effectively as a result of project-led trainings. Respondents in comparison areas report a similar level of exposure and perceived improvements on agricultural practices and availability of inputs through other donor programming. Water constraints are universally noted. YAAJEENDE FINAL IMPACT EVALUATION REPORT ANNEXES 27 OUTCOME FAMILY 1: WOMEN’S AND CHILDREN’S NUTRITONAL STATUS AND DIET Outcome Family 1 contains six individual-level outcomes: the prevalence of wasting, stunting, underweight (kids), underweight (women), minimum acceptable diet (MAD) and exclusive breastfeeding (EBF). EVALUATION QUESTIONS 1 AND 2: OVERALL PROGRAM IMPACTS AND DRIVERS OF IMPACTS Evaluation Question 1: What are the impacts of the Yaajeende NLA package on the prevalence of poverty and malnutrition six years after the start of program implementation across four thematic categories (women’s and children’s nutrition; household food security and poverty / economic well-being; household water, sanitation and hygiene practices; and household agricultural practices)? Evaluation Question 2: What major factors or processes contributed to observed impacts, including the role and importance of Yaajeende-supported local institutions? For all results reported, the evaluation team considers the entropy-balanced DID model, which uses a matching approach to improve similarity between treatment and comparison villages and households, to yield the most reliable results with respect to determining Yaajeende project impacts.32 On the basis of the entropy-weighted DID model, the endline results suggest the Yaajeende project had a statistically significant impact on two of the six outcomes in Family 1 (Table 6), relative to comparison group households, where there is varying but widespread evidence of other donor-supported MCHN/WASH programming implemented during the same time frame. 33 These are the prevalence of underweight in women and minimum acceptable diet in children; both of these go in the expected direction, are statistically significant at the 95 percent level and have modest effect sizes at -0.153 and 0.331, respectively. The estimates for these two results suggest the Yaajeende project led to a 5.6 percentage point decrease in the likelihood of women being underweight and an 8.1 percentage point increase in the likelihood of children having a minimum acceptable diet, relative to the trends in comparison group villages. For Family 1 outcomes in general, the estimate for the Yaajeende treatment effect goes in the expected direction, but the effect is not statistically significant for all outcomes. The estimated coefficients for the Yaajeende project effect are negative for the prevalence of wasting and children underweight, which signify a reduction in the likelihood of wasting and underweight, and there is a positive coefficient for MAD. The prevalence of EBF is a notable exception where the treatment effect estimate goes opposite the expected direction, despite a positive secular trend on the indicator. Effect sizes for each of the women’s and children’s nutritional status and diet outcomes are generally small, at below 0.2 — except for MAD and EBF, where the effect size is moderate from a policy-relevance standpoint (approximately 0.3 and -0.4, respectively). In the latter case, the effect goes in the unexpected direction by predicting a decline in EBF. The overall pattern of results for Outcome Family 1 suggests that while many of these indicators in Yaajeende villages may have improved overall during the project lifetime, the magnitude of improvements was not large enough to detect an effect relative to the trends in comparison villages. They also show substantial improvement 32 In simplified terms, entropy-balancing is a statistical matching technique that looks for observations in the comparison group that are most similar to the treatment group in terms of their observable characteristics (household size, education, poverty status, etc.). By assigning these observations more importance, it constructs a comparison group that more closely mirrors the treatment group than does the original (unweighted) comparison group. Thus, the evaluation team is more confident that the model captures the treatment effect, avoiding the noise generated when comparison and treatment groups differ substantially. 33 See Annex I for a summary of similar programming by other donor projects obtained through qualitative data collection at endline in comparison villages. YAAJEENDE FINAL IMPACT EVALUATION REPORT ANNEXES 28 over the same time period, likely owing to their exposure to similar programming. At endline, the mean prevalence of wasting among children under 5 in Yaajeende households was 11 percent, stunting was 21.5 percent and underweight was 18.4 percent. Also, 30.7 percent of children were optimally breastfed during 0-5 months of age, and 8.1 percent of children aged 6-23 months received a minimum acceptable diet. For all age groups under 5 years, insufficient minimum dietary diversity was the key reason for lack of achievement of MAD. Among Yaajeende households at endline, only 5.5 percent of children aged 6-8 months achieved minimum dietary diversity, although 24.9 percent had a minimum meal frequency. For children aged 9-23 months, 12.9 percent achieved minimum dietary diversity, while 45.1 percent had a sufficient meal frequency. Among women aged 15-49 years, 19.2 percent were underweight, and the mean dietary diversity score (WDDS) was 4.0. At endline, WDDS among women in Yaajeende villages was 0.80 units higher than for women in comparison villages, a statistically significant difference (see additional results in Annex II). The estimate for the endline time variable in these analyses (Endline) generally shows progress on women’s and children’s nutritional and diet outcomes between midline and endline. This overarching time trend points to an overall decline of 2.9 percentage points in the prevalence of wasting and a decline of 2.0 percentage points in the prevalence of kids underweight during 2015-2018. Similarly, a 20.8 percentage point increase occurred in the prevalence of EBF over the period, significant at the 90 percent level. This pattern, with its overall positive trend on the outcome over time but no statistically significant treatment effect for households in Yaajeende project villages, is also consistent with an explanation that could stem from widespread exposure of comparison group households to other development programs that focused on similar issues and also had some degree of effectiveness. Such a situation is supported through household training exposure data in the endline household survey and through qualitative data collection in comparison villages at endline. Under it, households in comparison villages — while not beneficiaries of Yaajeende — were exposed to other development programs that may have led to improvements in these outcomes. Thus, the interpretation for the endline results for this outcome family are not necessarily that Yaajeende had no impact, but rather that the Yaajeende project’s effects were similar to those of other programs implemented in comparison group villages, making Yaajeende’s impact relative to trends in comparison villages undetectable. The midline to endline findings for this outcome family are similar to the pattern of baseline to midline findings reported by the MIE team on these outcomes. 34 Baseline to endline analyses conducted by the endline evaluation team found somewhat similar results, although the BL-EL results are considered less robust by the evaluation team than the ML-EL findings are, due to power limitations with the baseline sample and concerns about baseline data reliability. In general, that analysis does not find evidence for effects of Yaajeende on the individual-level women’s and children’s nutritional status and diet outcomes from baseline to endline overall. The estimated effects generally go in the same direction as in the midline-endline estimates, suggesting positive but not statistically significant improvements, and the unexpected negative impact on EBF is smaller for the baseline-endline estimates (see Annex II for additional presentation of the baseline to endline results). TABLE 6. OUTCOME FAMILY 1 VILLAGE FIXED EFFECTS DID RESULTS: INDIVIDUAL-LEVEL OUTCOMES, ML-EL. VARIABLES 1.1 1.2 1.3 1.4 1.5 1.6 1.6A 34 The MIE found no significant treatment effects for any of these outcomes in Outcome Family 1. YAAJEENDE FINAL IMPACT EVALUATION REPORT ANNEXES 29 Wasting: z￾score below - 2 on reference weight-for￾length curve. Stunting: z￾score below -2 on reference length-for￾age curve. Underweight: z￾score below -2 on reference weight-for-age curve. Underweight: body mass index (BMI) below 18.5. Minimum acceptable diet (MAD) for children ages 6-23 months Exclusively breast-fed (binary) Exclusively breast-fed (Revised) Yaajeende Treatment -0.033 -0.015 -0.012 -0.056** 0.081** -0.198* -0.195* Effect (0.031) (0.038) (0.027) (0.023) (0.032) (0.112) (0.117) Gender = Female -0.014 -0.011 -0.020 -0.004 -0.019 -0.036 (0.013) (0.024) (0.019) (0.015) (0.024) (0.022) ln(Age) -0.034*** 0.046*** 0.013 -0.224*** 0.045*** -0.090*** -0.090*** (0.012) (0.017) (0.011) (0.024) (0.017) (0.021) (0.020) Household Head Has At Least Elementary -0.004 -0.055*** -0.063*** -0.010 0.050 -0.040 -0.080** Education (0.015) (0.014) (0.024) (0.035) (0.045) (0.036) (0.034) Endline -0.029 0.040 -0.020 0.009 -0.058** 0.208* 0.157 (0.028) (0.034) (0.022) (0.021) (0.028) (0.108) (0.113) Observations 8,442 8,631 8,447 10,526 2,428 3,399 3,399 Treatment N 6500 6649 6506 8352 1896 2609 2609 Control N 1942 1982 1941 2174 532 790 790 Treatment Effect 95% CI [ -0.093; 0.027] [ -0.089; 0.059] [ -0.065; 0.041] [ -0.102; - 0.011] [ 0.017; 0.144] [ -0.418; 0.022] [ -0.424; 0.033] Treatment Effect Size -0.110 -0.034 -0.031 -0.153 0.331 -0.412 -0.402 *** p<0.01, ** p<0.05, * p<0.1 Note: Robust standard errors in parentheses clustered at the village level. Effect size calculated as the estimated coefficient divided by the standard deviation of the outcome at endline. Age is measured in days for children and years for women. Figure 6 shows the estimate and 95 percent confidence interval for the treatment effect of the Yaajeende project on each of the outcomes for Outcome Family 1. To facilitate comparison and check the robustness of the estimates across model specifications, this figure plots the side-by-side effect sizes from the DID model with village fixed effects, with village fixed effects and entropy weighting, and with household fixed effects. The estimates are generally similar, demonstrating robustness to different model specifications. In other words, regardless of which model is used, the magnitude of the estimated Yaajeende project effect on the outcome is similar and the estimates across the three-model specification tend to be in the direction expected for a positive program effect. Across all specifications, evidence is consistently fairly low for a statistically significant effect of the Yaajeende project on women’s and children’s nutritional status and diet outcomes, relative to the comparison case of households in areas exposed to similar MCHN programming by other donors during the Yaajeende project lifetime. 35 35 See Annex I for a summary of similar programming by other donor projects, obtained through qualitative data collection in comparison villages. YAAJEENDE FINAL IMPACT EVALUATION REPORT ANNEXES 30 FIGURE 6. OUTCOME FAMILY 1 COMPARING TREATMENT EFFECTS BY MODEL: INDIVIDUAL-LEVEL OUTCOMES The qualitative data collected through group discussions and KIIs in Yaajeende villages and in comparison areas largely corroborate the statistical results and provide an understanding of drivers of these results. These data also shed light on the role of the project in improving knowledge and knowledge transfer on these issues, the nature of behavior changes and challenges to wider uptake and effectiveness of behavior change. “Before (the Yaajeende project), we did not know how to take care of our children, the importance of prenatal visits, how to set up gardens — and the project trained us in this way and that helped to reduce our difficulties.” (Kédougou, GDG GD) The qualitative data suggests widespread sensitization and trainings provided by Yaajeende, focused on knowledge sharing and transfer to others in the village on the importance of varied diet for pregnant women and young children, improved child feeding and nutrition practices, health visits, child immunizations and so on. Women indicated they received training on making enriched porridge for children to improve their nutrition and highlighted that a system was in place to weigh children in the village, monitor their health and recommend enriched foods if their pace of growth raised concerns, and refer children to health centers in cases of severe malnutrition. Overall, respondents expressed a view of positive change with respect to children’s health and reduced malnutrition as a result of project activities. In general, the qualitative data from Yaajeende villages at endline suggests that women’s knowledge on issues related to their children’s and their own health, the importance of eating more diverse and vitamin￾rich foods, and the benefits of childbirth and pre- and ante-natal visits to health clinics or hospitals has substantially improved, along with behavior change to put this knowledge into practice. The view that positive changes had occurred in children’s health, as well as a reduction in children’s malnutrition, as a result of Yaajeende project activities was fairly widespread. This had similar levels of support across each of the three regions covered by the evaluation. In -0.087 0.069 -0.016 -0.068 0.092 -0.218 -0.222 -0.110 -0.034 -0.031 -0.153 0.331 -0.412 -0.402 -0.108 0.069 -0.059 -0.098 0.400 -0.096 -0.024 1.1 Wasting (Children) 1.2 Stunting (Children) 1.3 Underweight (Children) 1.4 Underweight (Women) 1.5 Min. Acceptable Diet (Children) 1.6 Exclusive Maternal Breastfeeding (Children) 1.6a Revised EBF (Children) -1.0 -0.5 0.0 0.5 1.0 Unbalanced DID Entropy Balancing DID Household Panel Note: Plot shows the estimated effect sizes and 95% confidence intervals for the treatment effect of the Yaajeende program from regressions with controls. The effect size is the estimated effect divided by the standard deviation of the outcome at endline. For example, an effect size of 0.2 means the treatment increased the outcome by 0.2 standard deviations, relative to trends in the control group. Effect sizes of 0.2 are generally considered 'small', 0.4 as 'medium', and 0.6 as 'large'. Women and Children's Nutrition Outcomes YAAJEENDE FINAL IMPACT EVALUATION REPORT ANNEXES 31 each region, this view was more commonly expressed by women who were GDG members, but it was also — albeit less commonly — noted by men in Yaajeende villages and women in Yaajeende villages who were not GDG members. APS, VNC and GTC members and health officials at commune or higher administrative levels expressed similar views, particularly noting a substantial reduction in cases of severe malnutrition, reduced malnutrition in general and a reduction in diarrhea rates. These views were represented across KIIs from each of the three regions covered by the evaluation. In some cases, respondents linked their perceptions of change to what they saw through community monitoring and child screenings they participated in, and there was a widespread view that the screening and referral system in place has contributed to better monitoring and improvements on children’s health. One GTC member described the system to weigh and track malnutrition cases as follows: “We track children from 0 to 5 years in age. Green means that the child is healthy, the yellow color is lacking vitamins and should take Plumpy’nut, for lack of enriched flour, for 15 days normally and that everyone can take two to three times a day in a clean container. Now, the red zone indicates severe malnutrition, which is taken care of at the level of the health post.” (Matam, GTC KII) GDG members also described group activities that Yaajeende organized with them and helped them institutionalize within the community, with respect to community members producing enriched foods for children and a structured system to weigh and screen children to identify those who should receive enriched foods and/or be referred to the health post, for cases of more severe malnutrition. “There is a group that meets every month to make flour enriched with some varieties of millet mix that is given to children and there is a follow-up, next month we weigh the children who had benefited from this flour to assess their nutritional status and weight, and with that there has been an improvement in children's health.” (Bakel, GDG GD) “They sometimes organize weighing days for children, if they find that a child is not the right weight, he is immediately referred (to the health post) or the child is forced to take vitamins.” (Bakel, GDG GD) Women learned to make vitamin-enriched porridges and other foods for children, and conveyed knowledge on the importance of these foods for improving child health. But some respondents also noted that vitamin-rich foods are not always available, and at times accessible only to families with money to buy them at market. “Yaajeende has also improved the nutritional aspect. Indeed, even if malnutrition is not totally eradicated in the area, it is considerably reduced. Before, when I was screening children if I have 60 yellows, I would have perhaps 14 or 15 reds. … With the arrival of Yaajeende, we were taught how to make flour enriched with simply the mixture of millet, peanut, cowpea, kethiakh and iodized salt. We were trained and then we formed groups (to continue the practice). If your child is sick, you are asked to make him the enriched flour and give it to your child for two weeks; when you come to the weighing, you will see that the child is in the green (healthy).” (Matam, APS-VNC KII) “For carrots and squash, our kids were not used to it (before). We eat vegetables now and have a varied diet with the project. Children also eat it.” (Matam, GDG GD) “When you have money, you can find [vegetables] at the weekly market. At the market you can find carrot, turnips, cabbages, eggplant. We can then make a puree for children, and it gives them vitamins. Yaajeende taught us that.” (Matam, GDG GD) YAAJEENDE FINAL IMPACT EVALUATION REPORT ANNEXES 32 “We eat our vegetables rich in vitamins such as carrots, cabbage, etc. In winter time, all these vegetables are available in the village garden, but at the moment we buy them.” (Matam, non-GDG women’s GD) But not all groups at endline felt that the Yaajeende project had been effective in increasing knowledge or changing women’s behavior with respect to health and nutrition issues. A minority of respondents felt that information and sensitization meetings were only available to some people in the villages, while others felt the project had not really done anything concrete to improve situations for women and children’s health, thus little had changed on these issues as a result of the project activities. Across Yaajeende village households surveyed at endline, 63.5 percent said they had regularly participated in at least one Yaajeende activity, while 36.5 percent said they had not. Of the participating households, 85 percent said that someone from the household had participated in a Debbo Gallé group, and 13.9 percent had not. Across all households surveyed at endline, 68.5 percent said their village had a mother’s group. Among households with a respondent who regularly participated in mother’s group meetings, 65.8 percent said they knew about nutrition trainings held through the mother’s group, but 34.2 percent said they were not aware of such trainings. Men in one GD noted that pregnant women’s diet had not changed much, despite sensitization on this and the health benefits to women and children. The reason they gave for lack of implementation in practice is because varied foods are not available to them. “Yes, they sensitized them, they said that especially the consumption of the pregnant woman who must be rich and varied for the health of the woman and the child, but the problem is that they do not apply because the means are lacking, even if they want to do it, they do not have these means.” (Bakel, Men’s GD) Lack of means as a limiting factor to realizing health and nutrition gains was commonly expressed, mentioned in 10 of the 33 GDs in Yaajeende villages. This view was represented across the three regions, and by GDG members, other women beneficiaries and men. “They have improved our knowledge, but the effectiveness of this change depends on the means at our disposal. We cannot speak of change without means.” (Bakel, non-GDG women’s GD) “For women, most of them have not changed their practice due to lack of resources. … We do not always have the means to buy all the ingredients for a vitaminized diet, otherwise we just eat the usual dishes.” (Bakel, non-GDG women’s GD) Another contributing factor for lower than expected change on nutrition and diet outcomes relates to labor and time costs required to implement the behavior changes that were promoted by the project. This was noted by some GDG and other beneficiary women, who expressed a sentiment that one can learn new things about the health and nutrition of pregnant women, but the behaviors that were promoted are time-consuming and it is difficult to implement this in practice if one does not have the time or resources available. This includes a sentiment that in reality it is difficult to add meat, fish and vegetables to household diets, as these are not always widely available, even if one has the knowledge that it is beneficial to do so. One of Yaajeende’s aims was to design and leverage community-level processes and resources available within communities to enable villagers to solve development problems on their own. One such system was the greniers des enfants, or children’s granaries, that the project established, with the aim of setting up a communal system by which villages could produce enriched foods for children. Coupled with systematic child health screenings in the village, ultimately this system aimed to work toward reducing child malnutrition through their community self￾driven initiatives. This was indeed described at endline as a collaborative and community-led process. Some YAAJEENDE FINAL IMPACT EVALUATION REPORT ANNEXES 33 respondents mentioned that they liked the solidarity of this approach, and that it relies on each villager to work and contribute to it, rather than an outside group simply providing money. However, while this system was seen as an important contributor to reducing malnutrition, at least one KII suggested that this activity only began to take hold toward the end of the project, and this delay may have limited its effectiveness. “We made a granary where we have cowpeas, millet, rice, peanuts. Every week, I call people to the village chief to prepare the enriched flour. We wash millet, pestle with peanut. We do it every week. It is me who passes in the houses to inform those who are part of the Debbo Gallé group and others who are not part of it. There is no discrimination. Since the children are numerous, we have formed three subgroups in the village. For example, we are preparing today for a first group, next week it will be the turn of the second and the other week for the last group.” (Matam, GDG group) “I think that the child's granary could have been more effective if it had been established in time. They (the project) did it, but at the last moment. If we had started with this in the beginning we could have avoided many cases of severe malnutrition. Because the battle is not at the level of structures, it is not at the macro level, but instead at the household level and at the community level. If we managed to establish the granaries and follow up and especially to ensure the sustainability of these granaries, I think we could have fought effectively against moderate acute malnutrition which is really the bedrock of all problems.” (Matam, Nutrition Focal Point) Men in one group in Bakel noted that they had not received training on malnutrition issues and were not aware of anyone who had been. Although uncommon, this view is represented in each of the three regions covered by the evaluation, and it is in line with a view expressed by some male respondents that Yaajeende activities on malnutrition were primarily targeted toward women. Members of one men’s group in Matam said they were not very involved in the women’s health and nutrition issues because this was the domain of women, but they felt there had been positive improvements for women’s and children’s health. However, they noted that the trainings on this were organized for women only. “Since the project arrived, I have not received any training on malnutrition and I don’t know anyone who was trained in this domain.” (Bakel, Men’s GD) In general, the qualitative data on issues related to women and children’s nutrition and diet provides some indication that Yaajeende trainings and program activities on this were primarily targeted to women in Yaajeende villages, rather than village-wide sensitization that focused on men and women equally. Still, there are examples at endline of knowledge transfer to men and men’s awareness and involvement on issues related to children’s health and nutrition. For example: “The child who has received exclusive breastfeeding growth is different from the child who has not received it, and this is beneficial. My wife respected EBF because the VNC showed me when to give the child milk and how to prepare it. I make the milk myself and give it to his mother to give to the child.” (Kédougou, men’s GD) Although the evaluation does not find strong support for a statistically significant change in optimal breastfeeding practices as a result of the project, the qualitative data provided widespread indication that women in Yaajeende villages are aware of the importance of these practices for the health, nutrition and growth of the child. At the same time, women provided several reasons that help explain why this knowledge gain did not translate into more widespread behavior change on this practice. The most common difficulty noted with respect to exclusive breastfeeding is that women still often do not produce enough milk to consistently breastfeed. (This also corroborates the lower-than-anticipated improvements on women’s nutritional status in the quantitative results.) Women noted in some cases that this simply wasn’t a common practice they had been used YAAJEENDE FINAL IMPACT EVALUATION REPORT ANNEXES 34 to, so at times they forget and give the child water. Men in Yaajeende villages, and VNC members also noted this ongoing challenge, and added that a lack of time to practice optimal breastfeeding continues to be an issue. “I have not seen a change in breastfeeding. For there to be a change, women must have a good vitamin-rich diet so that the milk is rich in the evening. And our women do not eat rich foods.” (Kédougou, men’s GD) However, there was also an indication of men’s growing awareness on optimal breastfeeding practices, and of growing acceptance and support for it through sensitization and their own experiences with nutritional improvements in children as a result: “I have seen women who stay five to six months without giving their children water, they only give milk. I saw that here in this village. I was (initially) against this practice, but I was told it's healthier. I am sincere in what I say because I have fasted. I remember that well. 36 … [But] they are healthy, unlike the other children.” (Bakel, men’s GD) EVALUATION QUESTIONS 3 AND 4: HETEROGENEITY OF IMPACTS AND POTENTIAL DRIVERS Evaluation Question 3: How do program impacts differ for key subgroups of interest across key outcomes? The evaluation will assess two subgroups: Northern regions (Matam and Bakel) vs. Southern region (Kédougou); and poorest households vs. other households. Evaluation Question 4: What are potential explanatory reasons for variations in key outcomes across the subgroups? Is there evidence of a difference in Yaajeende project effects across regions? Annex II presents regression results that estimate Yaajeende impacts by geographic region. The results for Outcome Family 1 suggest a significant increase (5.2 percentage points) in children’s stunting prevalence in Yaajeende villages in Matam and Bakel, relative to comparison villages with similar programming, but a reduction in the prevalence of women underweight of 8.6 percentage points. Matam and Bakel show no indication of other significant impacts as a result of Yaajeende programming. In Kédougou, results suggest a statistically significant decrease of 11.4 percentage points in the prevalence of stunting in children, and a 9.7 percentage point decrease in the prevalence of underweight in children as a result of Yaajeende programming. There is no indication of other regional differences in the impact of Yaajeende programming. Is there evidence of a difference in Yaajeende project effects for poorest households relative to others? Results for the effect of the program by household poverty status are also in Annex II. This evaluation defines “poorer” households as those whose likelihood of poverty was above the median likelihood of poverty in 2018, while “less-poor” households have a likelihood of poverty at or below the 2018 median. Overall, the results show larger effects in the expected direction for less-poor households than for poorer households. Some of these effects for less-poor households are statistically significant and economically important, such as a drop of 20.8 percentage points in the prevalence of wasting in children, relative to trends in comparison areas. Similarly, a reduction of 22.6 percentage points is apparent in the prevalence of underweight in women from less-poor households, relative to trends for women from less-poor households in comparison villages. However, these 36 The evaluation team interprets the respondent’s statement here as justification for why he was initially against the practice of exclusive breastfeeding. He notes that through his own fasting, he has experienced the weakening effects of abstaining from food and some types of drink. Therefore, he does not consider it unreasonable that he initially had concerns over potential negative effects that an infant might experience if the infant is not allowed to drink water and is provided only breast milk. YAAJEENDE FINAL IMPACT EVALUATION REPORT ANNEXES 35 effects do not appear in poorer households, where no treatment effects are detected. Additionally, while some of these effects appear quite large, they are not especially robust to using different model specifications, and therefore the poverty status regressions for this outcome family specifically should be taken with caution. Is there evidence of stronger impacts for households that directly participated in Yaajeende activities? The endline household survey asked household heads a series of questions about whether they or any member of their household participated in trainings focused on eight sets of issues that the Yaajeende project covered over the past six years. The questions are administered to all respondents across the sample, and they provide information on household participation in different types of trainings regardless of whether Yaajeende or other donor programs provided the trainings. The evaluation team created two measures of participation intensity based on these questions. First was a household-level measure of participation intensity, defined as the total number of topic areas for which the respondent noted someone from the household had participated in a training session. Next was a village-level measure of participation intensity, defined as the village average of the household-level measure. These measures are defined for both treatment and comparison group households, who may have attended training sessions affiliated with programs other than Yaajeende. They provide a proxy measure of the intensity of household-level exposure to program training sessions that were designed to disseminate knowledge and help facilitate behavior change by beneficiaries toward improved agricultural, food security, health, nutrition, WASH and economic outcomes. To estimate the effect of direct participation in trainings on issues promoted by the project, the evaluation team ran the DID regressions with village fixed effects, adding in a triple interaction term between treatment, endline and treatment intensity. Table 7 shows the results using the village-level treatment intensity measure. The estimate on the coefficient of interest, the triple interaction term, goes in the expected direction for the children’s biometric outcomes in columns 1.1 to 1.3, significant for the effect on stunting and underweight prevalence in children (outcomes 1.2 and 1.3). The interpretation of the coefficient implies that an increase of one in the average number of training types attended by households in the village leads to an 8.8 and 9.8 percentage point decrease in the likelihood a child is stunted and underweight, respectively, at endline. The estimates for the effect on underweight in women and MAD do not go in the expected direction, but the effect sizes are extremely small and the magnitude of the estimates are close to zero. Comparing these results based on the average village-level participation intensity to those using the household￾level measure of treatment intensity, the coefficients using the household-level measure are much smaller and see no statistical significance. These results are shown in Figure 7, which plots the estimates on the triple interaction term using both the household- and village-level treatment intensity measures, for both the standard DID and DID with entropy weighting. Regardless of the intensity measure used, outcomes 1.1 to 1.3 consistently have effects in the expected direction, with consistent statistical significance for underweight prevalence in children, in addition to a statistically significant effect on the stunting prevalence due to an increase in the average village-wide training exposure received by households. The effect sizes on these outcomes for the entropy weighted models are approximately twice as large as in the standard models, and results may suggest that for integrated agriculture, health and nutrition programs, a programming approach that achieves higher saturation of direct participation in multiple different trainings across households in a given village may be associated with a higher likelihood of achieving statistically significant change on key women’s and children’s nutrition and diet outcomes. TABLE 7. OUTCOME FAMILY 1 VILLAGE FIXED EFFECTS DID RESULTS: INDIVIDUAL-LEVEL OUTCOMES, ML-EL, VILLAGE-LEVEL TREATMENT INTENSITY EFFECTS YAAJEENDE FINAL IMPACT EVALUATION REPORT ANNEXES 36 VARIABLES 1.1 1.2 1.3 1.4 1.5 1.6 1.6A Wasting: z￾score below - 2 on reference weight-for￾length curve. Stunting: z-score below -2 on reference length-for￾age curve. Underweight: z￾score below -2 on reference weight-for-age curve. Underweight: body mass index (BMI) below 18.5. Minimum acceptable diet (MAD) for children ages 6-23 months Exclusively breast-fed (binary) Exclusively breast-fed (Revised) Intensity Differential Effect (Treat*Endline* -0.021 -0.088** -0.098*** 0.043* 0.044 -0.082 -0.106 Intensity) (0.050) (0.035) (0.024) (0.025) (0.047) (0.153) (0.148) Yaajeende Treat. 0.012 0.094*** 0.085*** -0.041* -0.077* 0.118 0.150 Effect (0.049) (0.032) (0.021) (0.023) (0.044) (0.150) (0.143) Endline*Intensity 0.000 0.064* 0.084** -0.088*** 0.066 -0.145 -0.130 (0.057) (0.033) (0.041) (0.031) (0.056) (0.172) (0.164) Gender = Female -0.015 -0.011 -0.021 -0.007 -0.027 -0.043* (0.013) (0.024) (0.019) (0.015) (0.024) (0.022) ln(Age) -0.034*** 0.047*** 0.010 -0.215*** 0.037** -0.097*** -0.097*** (0.011) (0.014) (0.010) (0.023) (0.015) (0.020) (0.018) Household Head Has At Least Elementary Education -0.008 -0.049*** -0.065*** -0.006 0.053 -0.031 -0.071* (0.013) (0.012) (0.023) (0.036) (0.040) (0.038) (0.037) Endline -0.048 -0.049** -0.096*** 0.037** 0.013 0.090 0.012 (0.049) (0.022) (0.031) (0.017) (0.046) (0.160) (0.148) Observations 8,442 8,631 8,447 10,526 2,428 3,399 3,399 Treatment N 6500 6649 6506 8352 1896 2609 2609 Control N 1942 1982 1941 2174 532 790 790 Diff. Effect 95% CI [ -0.119; 0.077] [ -0.158; - 0.019] [ -0.145; - 0.050] [ -0.007; 0.093] [ -0.048; 0.135] [ -0.382; 0.218] [ -0.396; 0.183] Effect Size -0.070 -0.206 -0.245 0.119 0.180 -0.172 -0.220 *** p<0.01, ** p<0.05, * p<0.1 Note: Robust standard errors in parentheses clustered at the village level. Effect size is for the triple interaction term, calculated as the estimated coefficient divided by the standard deviation of the outcome at endline. Age is measured in days for children and years for women. YAAJEENDE FINAL IMPACT EVALUATION REPORT ANNEXES 37 FIGURE 7. OUTCOME FAMILY 1 COMPARING TREATMENT INTENSITY EFFECTS BY MODEL: INDIVIDUAL￾LEVEL OUTCOMES EVALUATION QUESTION 5: MODERATING CONTEXT FACTORS Evaluation Question 5: How do key individual and household characteristics shape program impacts? In terms of moderating context factors, children’s age (measured in days) is associated with higher prevalence of stunting and underweight and lower likelihood of being exclusively breastfed. However, the child’s age is also associated with a higher likelihood that she or he receives a minimum acceptable diet. Since stunting effects are cumulative and indicative of chronic malnutrition over time, these outcomes may require longer time periods to observe significant impacts at scale. For adult women, age (measured in years) works in the opposite direction and is associated with a lower underweight prevalence. The effect of the household head’s level of education goes in the expected direction, with children in households headed by someone with at least a primary education being approximately 4.5 percent and 6.4 percent less likely to be stunted or underweight, respectively. Regarding the village and higher-level structures that the Yaajeende project put in place to help disseminate knowledge and institutionalize behavior change related to knowledge on MCHN issues in general, access and health benefits of varied and nutritious foods and best practices for infant and young child feeding, qualitative data collected at endline indicated that the institutions established and communications mechanisms were viewed as -0.036 -0.022 -0.029 -0.013 -0.144 -0.008 0.010 -0.063 -0.072 -0.083 -0.034 -0.132 -0.007 0.011 -0.113 -0.091 -0.159 0.012 -0.016 -0.365 -0.289 -0.070 -0.206 -0.245 0.119 0.180 -0.172 -0.220 1.1 Wasting (Children) 1.2 Stunting (Children) 1.3 Underweight (Children) 1.4 Underweight (Women) 1.5 Minimum Acceptable Diet (Children) 1.6 Exclusive Maternal Breastfeeding (Children) 1.6a Revised EBF (Children) -1.0 -0.5 0.0 0.5 Household-Level Intensity Household-Level w/ Entropy Balancing Village-Level Intensity Village-Level w/ Entropy Balancing Note: Plot shows the estimated effect sizes and 95% confidence intervals for the effect on the triple interaction term between treatment, endline, and treatment intensity, from regressions with controls. Women and Children's Nutrition Outcomes YAAJEENDE FINAL IMPACT EVALUATION REPORT ANNEXES 38 effective for transmitting knowledge and permitting wide knowledge-sharing within villages. However, respondents in beneficiary villages largely indicated that putting this knowledge into practice, especially with respect to improving women’s and children’s diets, was largely driven by household means. Household lack of means to grow or buy sufficient quantities of nutritious foods is still a key limiting factor for many. With respect to child’s access to enriched foods, the communal system supported by the project for child health screenings and production of such foods, where these systems are maintained within the community, appears beneficial. One GTC member, describing the integrated approach that the project aimed for, also conveys that nutritional and diet gains ultimately rest on households benefiting and taking up a constellation of linked practices together. In reality, this may be accessible to only a portion of the population who are best situated to do this in practice. “The services we offer to Debbo Gallé groups, for example, help them create what is called the children's granary. In terms of nutrition, the VNC groups them together for training on processing into enriched flour, mobilizing local products by making awareness caravans to promote local consumption and also to support nutrition with the children's loft to support children from 0 to 5 years old. On the farm level with the introduction of PDCO (orange-flesh sweet potato), which is very rich in vitamin and iron. The approach on which Yaajeende works is for the development of a nutritional agriculture in addition to the fortified Bio products, such as orange sweet potato, sorbic sorghum, fortified organic millet, obotambayel maize. It supports producers to popularize these products and that the population can benefit from them.” (Matam, GTC KII) The role of GDGs is also noted here, and GDG members themselves highlighted it, but other women in Yaajeende villages also did in some cases. Some respondents saw GDGs as the key motivators, noting that they provided advice to women in the village on the importance of breastfeeding exclusively until the age of 6 months. GDG members, other women in Yaajeende villages and men mentioned VNCs as playing a key role, together with Badiene gokh37, in organizing women in Yaajeende villages for sensitizations/trainings on malnutrition and child health issues, and this was recognized as beneficial for changing the health and nutrition status of children in Yaajeende villages. One Kédougou VNC member described the network’s role as being specific only to women’s and children’s malnutrition, sanitation, gardening and hygiene issues. In other words, in some Yaajeende areas, a distinction appeared to be linked to gender for the roles and topics covered by the VNC and the GTC, with GTC activities being focused on agricultural inputs and service provisions, improved seed varieties, plowing, animal vaccinations, and so on. APS was uncommonly mentioned (in just one men’s GD) as having played a role in training on women’s and children’s nutritional issues. OUTCOME FAMILY 1 CONCLUSIONS In conclusion, the results for this outcome family suggest that the Yaajeende project had an impact on improving the percentage of children receiving a minimum acceptable diet and reducing the prevalence of underweight women. Specifically, the evaluation team estimates that the program led to an increase of between 2.5 and 8 percentage points in children receiving the minimum acceptable diet, compared to the trend in control villages; however, due to the small size of this effect, it does not show consistent statistical significance across all models. Further downstream in the causal chain, the program appears to have led to a reduction in the prevalence of underweight and stunted children in households exposed to higher-intensity levels of treatment. An increase of one in the average number of training types attended by households in the village leads to a decrease of between 37 An intiative promoted through Senegal’s Ministry of Health, these are women who sensitize women on health issues. YAAJEENDE FINAL IMPACT EVALUATION REPORT ANNEXES 39 3 and 6 percentage points in the prevalence of children underweight, and a decrease of approximately 3 percentage points in the prevalence of stunting. Qualitative data support that the project improved knowledge on breastfeeding and obtaining a varied and nutritious diet for women. Women demonstrated knowledge of which vitamin-rich foods they should eat for a more nutritious diet (such as carrots, mangoes and oranges), not only for their own health but also for the health of a developing baby (for pregnant and lactating women). But some respondents in Yaajeende villages did not see project activities as having effectively changed health and diet status in practice at scale, while the time investment of related activities promoted by the project and a lack of resources to buy varied foods were seen as key limiting factors to wider achievements. Respondents in beneficiary villages largely indicated household means primarily drove the trend of putting knowledge into practice, especially with respect to improving women’s and children’s diets. Household lack of means to grow or buy sufficient quantities of nutritious foods is still a key limiting factor for many, and likely contributes to lower-than-expected gains on improved diets and nutrition indicators for women and children. OUTCOME FAMILY 2: HOUSEHOLD FOOD SECURITY AND ECONOMIC WELL-BEING Outcome Family 2 contains four household-level outcomes: household dietary diversity score (HDDS), Soudure, poverty likelihood and agricultural revenue. EVALUATION QUESTIONS 1 AND 2: OVERALL PROGRAM IMPACTS AND DRIVERS OF IMPACTS Evaluation Question 1: What are the impacts of the Yaajeende NLA package on the prevalence of poverty and malnutrition six years after the start of program implementation, across four thematic categories (women and children’s nutrition; household food security and poverty / economic well-being; household water, sanitation and hygiene practices; household agricultural practices)? Evaluation Question 2: What major factors or processes contributed to observed impacts, including the role and importance of Yaajeende-supported local institutions? Overall, evidence is mixed regarding the effects of the Yaajeende project on household food security and economic well-being outcomes since midline (Table 8). Results from the entropy-weighted DID model suggest that HDDS in treatment villages declined with respect to trends in comparison villages, while the duration of the hungry season increased by approximately half a month. While results also suggest a small decline (0.822 percentage points) in the likelihood of poverty, and a moderate increase in agriculture revenue, these results are not statistically significant in the entropy-weighted model. 38 As was the case for Outcome Family 1, the effect of the overall time trend generally shows progress on each outcome and is statistically significant. For example, between 2015 and 2018, the HDDS measure showed an overall increase of 1.134 food group units, while the length of the lean season decreased by 0.78 months, and the likelihood of poverty decreased by 6.886 percentage points. While the time trend for agricultural revenue shows a decline in revenue over time rather than moving in the expected direction, this declining trend was also present from baseline to midline, while inherent variability in self-reported agricultural revenue and production data, and differences in data collection between ML and EL, could also contribute to this. 38 The basic DID without entropy weighting suggests a 2.8 percentage point reduction in the likelihood of a household being under the poverty line as a result of Yaajeende programming in the village, corresponding to a policy-relevant effect size of 0.28. YAAJEENDE FINAL IMPACT EVALUATION REPORT ANNEXES 40 For Yaajeende households at endline, average HDDS was 6.6, the length of the hunger season was reported to last 3.3 months on average, and the likelihood of a household being below the poverty line was 25.9 percent. Mean total household agricultural revenue was FCFA 8,269 higher than in comparison households at endline. Yaajeende households at endline had a mean score of 2.1 (of 5) with respect to their subjective financial satisfaction, which is low overall but was 0.15 points higher than comparison group households at the same point, a statistically significant difference based on a t-test of difference on endline means. Although Yaajeende households reported a somewhat higher mean score for their perceived change in their subjective financial satisfaction over the past six years, relative to comparison group households at endline, the different is not statistically significant (see Annex II for additional results). The midline to endline findings for this outcome family are similar to the pattern of baseline to midline findings reported by the MIE team on these outcomes. 39 Baseline to endline analyses conducted by the endline evaluation team, although considered less robust by the evaluation team than the midline to endline findings due to power limitations and concerns on baseline data reliability, found quite different results from the ML-EL findings. The BL-EL results for the effect on HDDS and duration of the hungry season are near zero and statistically insignificant. The estimates do find a statistically significant effect for Yaajeende on likelihood of poverty, suggesting that the program reduced this likelihood by 5.012 percentage points from baseline to endline in treatment villages, with respect to trends in the comparison group. Similarly, the team estimates that annual agriculture revenue increased by FCFA 37,746 over the same period as a result of the program, statistically significant at the 95 percent level (see Annex II for additional presentation of the BL-EL results). TABLE 8. OUTCOME FAMILY 2 VILLAGE FIXED EFFECTS DID RESULTS: HOUSEHOLD-LEVEL OUTCOMES VARIABLES 2.1 2.2 2.3 2.4 Household Dietary Diversity Score - Past 24 hrs Soudure: Duration of reduced food intake (months per year) Likelihood of poverty at the $1.25 2005 PPP threshold (%) Total household agriculture revenue Yaajeende Treatment -0.929*** 0.511* -0.822 15,014.825 Effect (0.343) (0.293) (0.786) (10,842.390) ln(Household Size) 0.465*** -0.193** -2.530*** 10,194.244*** (0.094) (0.078) (0.422) (2,609.349) ln(Head Age) -0.023 -0.192 0.372 -4,998.660 (0.169) (0.160) (0.828) (8,759.890) Head Education Level: Household Head Has At Least Elementary Education = 1 0.391** -0.336*** -1.457 18,788.074** (0.153) (0.094) (0.900) (8,709.571) Endline 1.134*** -0.780*** -6.886*** -22,558.495** (0.328) (0.237) (0.624) (9,324.611) Observations 4,791 4,804 4,903 4,827 Treatment N 3563 3570 3638 3583 Control N 1228 1234 1265 1244 39 The BL-ML impact results reported at midline found a decline in HDDS and overarching declining trend in dietary diversity over time; a reduction in soudure of 0.3 months for villages receiving the Yaajeende agricultural treatment; a 2.9 percentage point effect in “high-intensity” Yaajeende villages; and a positive increase in agricultural revenue of FCFA 27,000 in the context of a similar magnitude of overall declines in agricultural revenues over time in the study area. YAAJEENDE FINAL IMPACT EVALUATION REPORT ANNEXES 41 VARIABLES 2.1 2.2 2.3 2.4 Household Dietary Diversity Score - Past 24 hrs Soudure: Duration of reduced food intake (months per year) Likelihood of poverty at the $1.25 2005 PPP threshold (%) Total household agriculture revenue Treatment Effect 95% CI [ -1.601; -0.258] [ -0.063; 1.085] [ -2.363; 0.718] [-6235.868; 36265.519] Treatment Effect Size -0.519 0.269 -0.091 0.216 *** p<0.01, ** p<0.05, * p<0.1 Note: Robust standard errors in parentheses clustered at the village level. Effect Size calculated as the estimated coefficient divided by the standard deviation of the outcome at endline. Figure 8 shows the effect size and 95 percent confidence interval for the treatment effect of the Yaajeende project on each of the outcomes for Outcome Family 2. To facilitate comparison and check the robustness of the estimates across model specifications, we plot the side-by-side effect sizes from the DID model with village fixed effects; with village fixed effects and entropy weighting; and with household fixed effects. As for Outcome Family 1, general consistency exists across the estimates from the different model specification, and the estimates for the effect on HDDS and likelihood of poverty are particularly robust. FIGURE 8. OUTCOME FAMILY 2 COMPARING TREATMENT EFFECTS BY MODEL: HOUSEHOLD-LEVEL OUTCOMES Qualitative data collected at endline largely support these statistical results and provide an understanding of contributing factors to observed results. On dietary diversity and expanded access to different types of nutritious foods throughout the year, the qualitative data suggest some gains on this, but it is unevenly reported -0.278 0.028 -0.273 0.134 -0.519 0.269 -0.091 0.216 -0.315 0.092 -0.240 0.167 2.1 Dietary Diversity Score (Past 24 hrs) 2.2 Duration of Hungry Season (Months per year) 2.3 Likelihood of Poverty 2.4 Total Agricultural Revenue -1.0 -0.5 0.0 0.5 Unbalanced DID Entropy Balancing DID Household Panel Note: Plot shows the estimated effect sizes and 95% confidence intervals for the treatment effect of the Yaajeende program from regressions with controls. Household Food Security YAAJEENDE FINAL IMPACT EVALUATION REPORT ANNEXES 42 across the Yaajeende villages in the qualitative sample. Two of 33 GDs from Yaajeende villages indicated that households consume a wider variety of foods as a result of project activities. There is some indication of an increased variety of crops grown. For example, GD respondents mentioned that successful crops (many introduced by the project, or improved seed varieties) included watermelon, corn, salad greens, turnips, sorrel, peanuts, jujubes, red potatoes, moringa and sorghum. The introduction of community or microgardens into Yaajeende villages by the project was widely seen as beneficial (15 of 33 GDs, including seven from Kédougou, five from Matam and three from Bakel), including with respect to improved access to diverse foods and follow-on benefits for household nutrition and health. In four GDs with men, women and Debbo Gallé members and four KIIs in Matam and Kédougou regions, respondents noted that the introduction of gardening activities and gardens run by women had enabled them to diversify their foods and ultimately contributed to better community health and reduced malnutrition in their communities. However, respondents also reported a range of community or microgarden challenges and failures. Market gardening in the dry season was said to be infeasible due to water demands and insufficient supply (one GD with Debbo Gallé members in Matam and one with other women beneficiaries in Bakel), and distance of gardens from the village was also cited as a challenge. Yaajeende respondents widely reported lack of inputs as the prevailing difficulty for gardening, and agricultural activities in general (23 of 33 Yaajeende GDs; eight Kédougou GDs, nine Matam GDs and six Bakel GDs). These included lack of seeds, fertilizer, water, fences, land, soil, forage, diesel and general materials and financial support. Of these, the need for water was the most commonly referenced; 20 of the 33 GDs in Yaajeende villages noted it, half of them Debbo Gallé members (six GDs each from Kédougou and Bakel, and eight from Matam). Reasons cited for the lack of water included climate (meaning that sources such as wells dried up) and sources of water being too far away to access. In two KIIs and one GD with women in Kédougou and Matam, respondents indicated they had abandoned gardening activities due to insufficient water. Men’s GD and Debbo Gallé groups mentioned a lack of sufficient fencing material to protect gardens from pests and/or livestock, as did KIIs in Kédougou and Bakel. Regarding land, some communities noted that they did not have enough available land to establish a community garden, although this mentioned only by respondents in Bakel (three of 33 GDs in Yaajeende villages). Parcels of land allotted for women were at times noted to be too small. Soil quality and issues with forage repairs to provide adequate water were less commonly noted concerns (one GD each with women or GDG members, respectively). In general, a number of groups spoke to an overall need for materials to be provided to them, as inputs were expensive. Financial support was also stated repeatedly as a factor impeding garden success, or for agricultural more generally (14 of 33 GDs in Yaajeende villages; with nearly equal representation across the three regions, and by the beneficiary group). Community gardens were not restricted only to Yaajeende villages. The qualitative data collection in comparison to group villages provided evidence that that households in comparison group areas were also exposed to and taking on similar activities. Similar to Yaajeende villages, respondents in comparison group GDs noted that gardens were generally beneficial, provided them with additional quantities and types of food for home consumption and helped reduce malnutrition in children. One of three comparison villages in Matam and one of three in Kédougou noted this. These respondents also noted that household diets had become more diverse as a result of such gardens in their communities, and that they had gained an additional source of revenue due to sales of garden produce. Before, individuals had to travel to buy vegetables, but establishment of gardens in the community had contributed now to household self-sufficiency in the village (one control GD with women in Matam). Control groups spoke of the same challenges for gardens that Yaajeende respondents noted, including issues with lack of inputs such as water and fencing, and a general lack of agricultural inputs that have made market gardening a more YAAJEENDE FINAL IMPACT EVALUATION REPORT ANNEXES 43 limited activity. Given the evidence for gardening activities in comparison areas, it is not necessarily unexpected that the statistical DID results find little evidence for an added effect of Yaajeende on these outcomes. With respect to the hunger season, respondents mentioned varying difficulties across households and communities, but provided a general sense of ongoing hardship during this period, despite project activities aimed at easing the length and severity of the lean season that households experienced. Two GDs in Yaajeende villages in Matam and Kédougou stated that villagers generally still consume all of their food during the hunger season, while a third Kédougou GD noted an ongoing need for financial support during this period to help the household manage food needs. KIIs with VNC and GTC members noted that the government used to provide food coupons for households during the hunger season, but this program is no longer in place, which creates additional hardship. More commonly, and particularly in Matam and Bakel, respondents noted that the severity of the hunger season was tied to whether sufficient rains had fallen during the agricultural season. The period of hunger was described by some as typically lasting around two months. Some respondents saw the “greniers des enfants” and micro- or community gardens that Yaajeende established as helpful for easing the severity of the hunger season (nine of 33 GDs in Yaajeende villages, and nearly equal representation across region and respondent category). However, a VNC member in Matam noted that while Yaajeende put microgardens in place to help secure household access to nutritious foods, it was not sufficient for securing food year-round, especially during the hunger season. Yaajeende also provided guidance on agricultural practices to avoid hunger season, but some APS agents and men’s group respondents in Kédougou Region and Bakel said that villagers generally did not have enough money to implement those practices. In terms of similarities with comparison group villages, at least one comparison village in Bakel indicated they had received supplemental food provisions during periods of hunger, in contrast to Yaajeende villages. However, this does not appear to be widespread, and there were no other key differences with respect to experiences related to hunger season. In terms of agricultural revenue, qualitative data provided a range of complementary information as to why the project may have only achieved modest impacts on this. Much of this relates to change, or lack thereof, in agricultural practices and production, discussed in greater detail under Outcome Family 4. While many respondents indicated consuming a portion of their harvest and selling another portion, conditions for selling — and hence any potential for generating agricultural revenues — depend on whether yield has been sufficient to retain some product for sale rather than using all of it for home consumption. A small number of Yaajeende GDs stated that increased yields due to practices or inputs provided by the project had allowed more people to sell agricultural goods and gain some financial autonomy (two men GD and one GDG in Matam; one KII in Kédougou). In general, when the harvest is good, households indicated that they retain part of it for home consumption and sell what they can at market. But the qualitative data suggest that this is not widespread. Respondents in three of six Kédougou Region Yaajeende villages indicated that their yields still barely cover household food needs, and they generally are not in a position to transform or sell some portion of their harvest. GDG members from one GD in Matam Region similarly noted that their agricultural yields had increased, but not to the point where their harvest can be sold, as the yields are still not sufficient to cover household food needs. Respondents in some GDs indicated an increase in revenue that they attributed to Yaajeende activities. Three of 33 GDs in Yaajeende villages mentioned an increase in revenue due to the sale of agricultural products, which they attributed to Yaajeende support. This was mentioned in Bakel primarily, with onions, potatoes and YAAJEENDE FINAL IMPACT EVALUATION REPORT ANNEXES 44 chili peppers noted as the most profitable crops. However, one GTC in Bakel also noted that the market is saturated, due to the entire village growing crops that need to be sold. More commonly, however, GDG and GTC KIIs noted that they are not able to sell all of their agricultural products, because demand is insufficient, and the market is too far. They noted that they don’t have a market in their own villages and cannot effectively store the products for later; as a result, their harvest goes bad. GD respondents in four of the six Yaajeende villages visited in Bakel, and two of the six Kédougou Region villages, expressed this. Marketing and transporting the agricultural products to the markets also poses a challenge. “Since there is no market to sell the products, the products are either given to the sheep or consumed by the HH.” (Bakel, GDG group) “We don’t have enough clients who want our onions; all the villages grow onions.” (Bakel, GDG group) With respect to community gardens established by the project, markets were also highlighted as an issue limiting their potential. Two KIIs in Kédougou and Bakel reported that markets were oversaturated due to the overall success of gardening activities. A GD with women beneficiaries from Bakel also reported that there was no place to sell goods produced from gardens. In one Kédougou Region comparison village, respondents mentioned having difficulty finding buyers or clients for their agricultural products. Comparison group villages also mentioned challenges in producing sufficient yields to be able to sell some portion for revenue and indicated that this was a deterioration from earlier situations. “Nowadays, we only consume what we harvest. In the past, we were able to consume and sell from our harvest.” (Matam, Comparison Village GD) EVALUATION QUESTIONS 3 AND 4: HETEROGENEITY OF IMPACTS AND POTENTIAL DRIVERS Evaluation Question 3: How do program impacts differ for key subgroups of interest across key outcomes? The evaluation will assess two subgroups: northern regions (Matam and Bakel) vs. southern region (Kédougou); and poorest households vs. other households. Evaluation Question 4: What are potential explanatory reasons for variations in key outcomes across the subgroups? Is there evidence of a difference in Yaajeende project effects across regions? Regressions showing the differential effects of the Yaajeende project by geographic region are in Annex II. The most important regional difference observed is that the program appears to have been effective at reducing the likelihood of poverty in the northern regions of Matam and Bakel, with an estimated 2.089 percentage point reduction as a result of the program, statistically significant at the 95 percent level. The estimate in the southern region of Kédougou is an increase of 1.063 percentage points in the likelihood of poverty, which is not statistically significant. On the other hand, the estimate predicting an increase in the duration of reduced food intake is significant only in Matam and Bakel, while the effect in Kédougou is a much smaller predicted increase of 0.134 months, which is not statistically significant. Similarly, the effect of the program appears much more positive for increasing agriculture revenue in Kédougou than in the northern regions, though the estimates here are not statistically significant. Is there evidence of a difference in Yaajeende project effects for poorest households relative to others? YAAJEENDE FINAL IMPACT EVALUATION REPORT ANNEXES 45 Regressions showing the differential effects of the Yaajeende project by household poverty status are in Annex II. The negative impact on dietary diversity and the increase in duration of the hungry season seen previously appears stronger in less-poor households, although the effects are also present and remain statistically significant in poorer households. No statistically significant effects are seen for either likelihood of poverty or agriculture revenue, neither in terms of the total effect on poorer households nor in terms of the difference in effect between poorer and less poor households. Is there evidence of stronger impacts for households that directly participated in Yaajeende activities? Using the village-level treatment intensity measure defined previously, the evaluation team ran the DID regressions with village fixed effects, adding in a triple interaction term between treatment, endline and treatment intensity. Results using entropy weighting and the village-level treatment intensity measure are in Table 9. The estimate on the triple interaction term goes in the expected direction for the effect on duration of the lean season and total household agricultural revenue, showing marginal significance for agricultural revenue. The estimates are not statistically significant for HDDS and likelihood of poverty but do go in the direction opposite of what was expected. With respect to the likelihood of poverty, the results suggest an overall time trend was a reduction in the likelihood of poverty of 5.541 percentage points, together with an additional 1.289 percentage point reduction for treatment villages, and an additional reduction of 1.36 percentage points for each unit increase in the village-level intensity indicator, the latter statistically significant at the 90 percent level. In other words, the results suggest that the villages in the Yaajeende project saw a fall in the likelihood of poverty that was in addition to the general trend in declining poverty seen overall in the study area. However, the impact of greater training participation for households in Yaajeende villages is less effective than the level of poverty reduction achieved through higher levels of participation intensity by outside programming in comparison villages. Comparing these results based on the average village-level participation intensity to those using the household￾level measure of treatment intensity, the coefficients using the household-level measure are generally smaller but provide some important additional detail on how participation intensity at the household level relates to evidence of impact. Results indicate a significant, positive effect of additional training types attended by the household on the likelihood of poverty, and on agricultural revenue, which may suggest a beneficial effect to households gained through the layering or ability to participate in multiple types of trainings offered through donor-supported programs. Figure 9 shows these results, plotting the estimates on the triple interaction term using both the household- and village-level treatment intensity measures, for both the standard DID and DID with entropy weighting. TABLE 9. OUTCOME FAMILY 2 VILLAGE FIXED EFFECTS DID RESULTS: HOUSEHOLD-LEVEL OUTCOMES, ML-EL; VILLAGE-LEVEL TREATMENT INTENSITY EFFECTS VARIABLES 2.1 2.2 2.3 2.4 Household Dietary Diversity Score - Past 24 hrs Soudure: Duration of reduced food intake (months per year). Likelihood of poverty at the $1.25 2005 PPP threshold (%) Total household agriculture revenue Intensity Differential -0.559 -0.372 0.891 26,389.831* Effect (Treat*Endline*Intensity) (0.421) (0.356) (0.813) (14,493.637) Yaajeende Treatment Effect -0.383 0.811 -1.289 -11,804.755 (0.448) (0.546) (1.017) (10,857.737) Endline*Intensity 0.609 0.429 -1.360* -26,769.138* (0.407) (0.319) (0.745) (14,309.127) YAAJEENDE FINAL IMPACT EVALUATION REPORT ANNEXES 46 VARIABLES 2.1 2.2 2.3 2.4 Household Dietary Diversity Score - Past 24 hrs Soudure: Duration of reduced food intake (months per year). Likelihood of poverty at the $1.25 2005 PPP threshold (%) Total household agriculture revenue ln(Household Members) 0.428*** -0.183** -2.585*** 10,346.572*** (0.092) (0.071) (0.426) (2,673.763) ln(House Head Age) -0.064 -0.199 0.023 -4,133.456 (0.175) (0.155) (0.983) (7,444.319) Household Head Has At Least Elementary Education 0.406*** -0.339*** -1.515* 19,075.252** (0.139) (0.094) (0.902) (8,271.137) Endline 0.499 -1.184*** -5.541*** 4,933.314 (0.398) (0.439) (0.753) (7,627.573) Observations 4,791 4,804 4,903 4,827 Treatment N 3563 3570 3638 3583 Control N 1228 1234 1265 1244 Diff. Effect 95% CI [ -1.384; 0.267] [ -1.071; 0.326] [ -0.703; 2.485] [-2017.176; 54796.837] Effect Size -0.314 -0.196 0.098 0.378 *** p<0.01, ** p<0.05, * p<0.1 Note: Robust standard errors in parentheses clustered at the village level. Effect Size is for the triple interaction term, calculated as the coefficient divided by the standard deviation of the outcome at endline. FIGURE 9. OUTCOME FAMILY 2 COMPARING TREATMENT INTENSITY EFFECTS BY MODEL: HOUSEHOLD￾LEVEL OUTCOMES -0.036 -0.042 -0.048 0.064 -0.044 -0.013 -0.085 0.106 -0.497 -0.008 0.159 0.156 -0.313 -0.195 0.097 0.378 2.1 Dietary Diversity Score (Past 24 hrs) 2.2 Duration of Hungry Season (Months per year) 2.3 Likelihood of Poverty 2.4 Total Agricultural Revenue -1.0 -0.5 0.0 0.5 1.0 Household-Level Intensity Household-Level w/ Entropy Balancing Village-Level Intensity Village-Level w/ Entropy Balancing Note: Plot shows the estimated effect sizes and 95% confidence intervals for the effect on the triple interaction term between treatment, endline, and treatment intensity, from regressions with controls. Household Food Security YAAJEENDE FINAL IMPACT EVALUATION REPORT ANNEXES 47 EVALUATION QUESTION 5: MONITORING CONTEXT FACTORS Evaluation Question 5: How do key individual and household characteristics shape program impacts? Household-level control variables show effects that go in the expected direction and are statistically significant. Larger households appear to exhibit higher household dietary diversity, reduced lean season duration, lower likelihood of poverty and greater revenue from agriculture. Similarly, households where the head has at least an elementary education have HDDS measures that are 0.391 points higher, on average, than those with uneducated household heads. The former experience lean seasons that are 0.336 months shorter, they are 1.457 percentage points less likely to fall below the poverty line and they have agricultural revenue that is FCFA 18,788.074 (approximately USD $33) 40 higher, on average. With respect to the village and higher-level structures that the Yaajeende project put in place to help disseminate knowledge and institutionalize behavior change with respect to the Family 2 outcomes, a small number of Debbo Gallé groups spoke about the positive effects of Yaajeende’s support for agricultural revenues. One Matam Region GDG noted that Yaajeende had taught them on how to package local cereals for sale at markets. Members of a second GDG in Matam described training they received from Yaajeende; it helped them determine which products to sell at market and which to distribute to their members for home consumption, as well as how to calculate profits and revenues earned from the sale of their agricultural products. The same GDG also established a fund to allow them to buy livestock to raise and resell. OUTCOME FAMILY 2 CONCLUSIONS Overall, the evidence suggests that the Yaajeende project contributed to a reduction in the likelihood of poverty in treatment villages, representing a decrease on the order of between 0.822 and 2.766 percentage points. Notably, the estimates show that households that participated in more trainings saw a greater reduction in the likelihood of poverty. Evidence from regional regressions shows that the program was more effective at reducing poverty in the northern regions of Matam and Bakel, where the coefficients estimate that the likelihood of poverty for households in treatment villages fell by 2.089 percentage points, while no decrease is seen in Kédougou. The qualitative data supported a notion of respondents reporting uneven gains by households on expanded access to different types of nutritious foods throughout the year, although the introduction of community or microgardens is widely seen as beneficial for improving access and follow-on benefits to nutrition and health. Respondents report widespread challenges with gardens, mainly focused on lack of or insufficient access to several required inputs, and insufficient water, which was a near ubiquitous limitation for all agricultural activities assessed. Ongoing challenges with hardship during the lean season continue to be noted, but Yaajeende established systems such as greniers des enfants were viewed as helping ease malnutrition. Reports of increased agricultural revenue was generally limited, and still dependent on whether harvest is successful or not. Other constraints included marketing and transport challenges, while market oversaturation was also noted to limit potential revenues for agricultural products. 40 US$1=FCFA$564.81. YAAJEENDE FINAL IMPACT EVALUATION REPORT ANNEXES 48 OUTCOME FAMILY 3: HOUSEHOLD HEALTHY HYGIENE PRACTICES Outcome Family 3 contains two household-level outcomes: prevalence of handwashing stations and prevalence of iodized salt use. EVALUATION QUESTIONS 1 AND 2: OVERALL PROGRAM IMPACTS AND DRIVERS OF IMPACTS Evaluation Question 1: What are the impacts of the Yaajeende NLA package on the prevalence of poverty and malnutrition six years after the start of program implementation, across four thematic categories (women’s and children’s nutrition; household food security and poverty / economic well-being; household water, sanitation and hygiene practices; household agricultural practices)? Evaluation Question 2: What major factors or processes contributed to observed impacts, including the role and importance of Yaajeende-supported local institutions? The evaluation does not find evidence of statistically significant, positive effects of the Yaajeende project for household healthy hygiene outcomes since midline, relative to comparison group households where there is varying but widespread evidence of other donor-supported MCHN/WASH programming implemented during the same time frame (see Table 10). While there was improvement on these outcomes over time during the lifetime of the Yaajeende project, the pattern of statistical results for this outcome family suggest that programming in comparison areas may have been more effective, leading to negative, statistically significant estimates for the effect of Yaajeende relative to the change observed in comparison areas over the same timeframe. Relative to the trend in comparison villages, households in Yaajeende villages were 7 percentage points less likely to have a soap-and￾water handwashing station in use. They were also 6 percentage points less likely to use recommended practices for iodized salt storage, although this result is not statistically significant. As for other outcomes families, the results also show that the overall time trend from 2015 to 2018 for both Yaajeende and comparison villages was positive for these outcomes. For salt storage in particular, the overall trend for all villages was a 22.4 percentage point increase in the likelihood that households had properly stored iodized salt between midline and endline. Thus, it is likely that Yaajeende did have some impact on this outcome, but the impact is not detectable relative to trends in comparison villages over the same time frame, given similar programming that occurred there which appears to have led to greater improvements on these outcomes for households in those areas. At endline, a verified soap and handwashing station was in common use for 48.6 percent of the surveyed Yaajeende households, relative to 81.9 percent of comparison group households. Thirty-five percent of Yaajeende households used iodized salt that was properly stored, which was similar to the percentage of comparison group households that had the same at endline. For both of these outcomes, the percentage of comparison group households that was positive for these indicators at midline was nearly half that the Yaajeende treatment group, and the gains in the comparison group over the midline to endline period were substantially greater. YAAJEENDE FINAL IMPACT EVALUATION REPORT ANNEXES 49 The midline to endline findings for this outcome family are similar to the pattern of baseline to midline findings reported by the MIE team on these outcomes41. Baseline to endline analyses conducted by the endline evaluation team, although considered less robust by the ET than the ML-EL findings due to power limitations and concerns on baseline data reliability, found effects that tend to be near zero, are statistically insignificant, and do not consistently go in the expected direction, suggesting no impact relative to the trends in comparison villages over the same period. However, similar to the ML-EL estimates, we estimate positive trends between baseline and endline over time for both treatment and comparison villages, with a 7.7 percentage point increase in the prevalence of handwashing stations, and a 14.2 percentage point increase in the likelihood households had properly stored iodized salt, both statistically significant (see Annex II for additional presentation of the baseline to endline results). TABLE 10. OUTCOME FAMILIES 3 VILLAGE FIXED EFFECTS DID RESULTS: HOUSEHOLD LEVEL OUTCOMES, ML-EL VARIABLES 3.1 3.2 Verified soap and water handwashing station (binary) Iodized salt properly obtained and stored Yaajeende Treatment -0.070 -0.060 Effect (0.063) (0.054) ln(Household Size) 0.032** 0.018 (0.014) (0.016) ln(Head Age) 0.004 -0.088* (0.022) (0.046) Head Education Level: Household Head Has At Least Elementary Education = 1 0.015 -0.019 (0.054) (0.022) Endline 0.031 0.224*** (0.059) (0.046) Observations 4,903 4,568 Treatment N 3638 3401 Control N 1265 1167 Treatment Effect 95% CI [ -0.194; 0.053] [ -0.166; 0.047] Treatment Effect Size -0.210 -0.125 *** p<0.01, ** p<0.05, * p<0.1 Note: Robust standard errors in parentheses clustered at the village level. Effect Size calculated as the estimated coefficient divided by the standard deviation of the outcome at endline. As for the previous outcome families, the evaluation team checked the robustness of the DID estimates through alternative model specifications that incorporated entropy balancing and household-level fixed effects. To facilitate comparison, Figure 10 plots the effect size estimates for the treatment effect of the Yaajeende project from each of the three models. Overall, the estimates are fairly consistent across the models, and show negative estimates of Yaajeende project effect for both outcomes that are statistically insignificant at the 95 percent confidence level. There is little variation in the estimate for the effect of Yaajeende on the likelihood that households have a soap- 41 The midline impact results found that Yaajeende was associated with an 11-percentage point lower likelihood of having a handwashing station in use relative to comparison areas, against an overall large secular improvement of 26 percentage points over time. At midline, no effect of Yaajeende was found on the use and proper storage of iodized salt, and it was noted that iodized salt was widely purchase but often not stored properly. YAAJEENDE FINAL IMPACT EVALUATION REPORT ANNEXES 50 and-water handwashing station in use. The effects are reported in Figure 10 in terms of effect sizes (for all of the outcome families), to facilitate comparison of the magnitude of Yaajeende project effects across different outcomes. For this outcome, the effect varies in size between -0.21 and -0.263 standard deviations of the outcome at endline, which corresponds to a reduction of approximately 7 percentage points relative to trends in comparison villages. Only the estimate from the standard DID model is significant at the 90 percent level. Slightly more variation is seen in the estimates across models for the effect on the likelihood that households use and properly store iodized salt, with an effect size that varies between -0.032 and -0.125 standard deviations. This corresponds to a reduction of 1.5 (household panel model) and 6 (entropy balanced model) percentage points in the likelihood that Yaajeende households use and properly stored iodized salt, relative to households in comparison villages, but the estimate is not statistically different from zero. FIGURE 10. OUTCOME FAMILY 3 COMPARING TREATMENT EFFECTS BY MODEL: HOUSEHOLD-LEVEL OUTCOMES These statistical results are largely supported by the qualitative data collected at endline, which informs on drivers of impacts. While data collection at endline indicated that respondents in Yaajeende villages had been well-sensitized on WASH issues and had implemented many of the practices — including with respect to testing, use and storage of iodized salt — ample evidence from comparison group villages showed similar activities and perceived improvements on WASH issues tied to other donor programs in those villages. Moreover, the qualitative data shed light on issues with eventual disuse of tippy-taps (simple, economical and effective handwashing stations) promoted by Yaajeende, and provided additional explanation for some reasons why. In this sense, the qualitative data largely corroborate the evaluation team’s interpretation of the endline statistical results on Yaajeende project impacts for Outcome Family 3. This appears to describe a situation in which households in both Yaajeende and comparison group villages improved on these outcomes -0.263 -0.051 -0.210 -0.125 -0.255 -0.032 3.1 Soap and Water Handwashing Station 3.2 Iodized Salt Properly Obtained & Stored -0.8 -0.6 -0.4 -0.2 0.0 0.2 Unbalanced DID Entropy Balancing DID Household Panel Note: Plot shows the estimated effect sizes and 95% confidence intervals for the treatment effect of the Yaajeende program from regressions with controls. Household Healthy Practices YAAJEENDE FINAL IMPACT EVALUATION REPORT ANNEXES 51 during the midline to endline period, but the magnitude of improvement in comparison villages was greater. Some of the contributing reasons for lower potential change in Yaajeende villages are described below. With respect to use and storage of iodized salt, qualitative data collected in Yaajeende villages provided fairly strong evidence of the perceived effectiveness of Yaajeende project efforts to sensitize beneficiaries on the health importance of using iodized salt, coupled with installation of a system of regular testing and monitoring of salt within homes and businesses in the community. Beneficiaries highlighted that the project had brought salt test kits for the communities to determine if the salt was iodized, and described a testing system established by Yaajeende in which VNCs received training in and conducted salt testing for women and businesses in the village. The process introduced by the project was seen as effective not only for sensitizing people on this issue, but also for establishing a system to determine if salt was iodized or not and to inform households accordingly. GDG members indicated that the Yaajeende project organized discussions on the importance of using iodized salt. Through this process, women learned how to use iodized salt and how to conserve it so that the iodide would not evaporate. The key role of the VNC in this process was widely acknowledged by GD respondents in Yaajeende villages. In KIIs, VNC members indicated that they were sensitized on the importance of pregnant women especially using iodized salt, to help strengthen the fetus, as well as for health benefits in general, and trained on the use of the iodized salt test kits. Men in Yaajeende communities also expressed knowledge on the use of iodized salt and described the VNC role in testing salt for women in the community and bringing about change on use of iodized salt. “We went door to door, we visited each household to check the iodized salt. See if they ate healthy. Anyone we found who did not eat iodized salt, we told them to leave this salt. To bring other salt (for us to test). When she brings the salt, I did the test to see (if it was iodized). When (the salt) was good, I told her it was good and that she could consume it. I also entered the shops — in every shop that was here, I tested the salt. When I found that it was good (iodized) salt, I told (the shopkeeper) to sell it. If not, I told (the shopkeeper) not to sell it.” (Matam, VNC KII) “I saw that the VNC asked the women to bring their salt which they used in their kitchens for verification, if it is of the good quality they will say it and advise also people to use this type of salt.” (Bakel, men’s GDI) “They gave us salt testers. Many used poor-quality salt, but with these testers many realized that the salt was not good. So you see there is an improvement.” (Bakel, APS KII) With respect to handwashing and use of handwashing stations, Yaajeende beneficiaries expressed substantial qualitative corroboration that the project had conducted widespread sensitization on a range of WASH issues, including handwashing and its relationship to reducing illnesses. Respondents in Yaajeende villages expressed a general feeling that Yaajeende had brought much-improved knowledge to communities on health, hygiene and WASH issues. (Specifics included washing hands before meal preparation, washing children’s hands, washing hands after use of the toilet, properly washing children, avoiding stagnating water around households, using sand to avoid mosquito larvae, general cleanliness around households and access to potable water.) Tangible changes in the community resulted from this behavior change. “Now there have been changes, soap is made available to the people, they are trained on handwashing and food hygiene. With the sensitization and talks, I felt that there were a lot of changes in some areas. And these families know that malnutrition is caused by insalubrity. It's not just in the diet but in general. Now people are sensitized in this way. This has helped to reduce malnutrition.” (Matam, GTC KII) YAAJEENDE FINAL IMPACT EVALUATION REPORT ANNEXES 52 GDGs were at times mentioned as a key vehicle for this, while other GDs in Yaajeende villages noted that the VNC was the primary way through which they obtained trainings and knowledge on WASH issues in general. One men’s group noted the role of GDG as follows: “With the Debbo Gallé, they sensitized the population on the hygiene, they even distributed liquid soap for the cleaning of the toilets. They also sensitized how to wash their hands after leaving the toilet with ashes mixed with bleach. They also sensitized on the hygiene of the houses.” (Matam, men’s GDI) There is also some indication that many Yaajeende villages had received sensitization on these issues through other means as well, which may have helped to reinforce take up of behavior change. This may be particularly so for villages in Kédougou Region, where previous handwashing campaigns during the 2014-16 West Africa Ebola outbreak were also frequently mentioned by respondents, and may have had a reinforcing influence on their WASH behavior. “With the advent of Ebola, they told people to wash their hands well. They even created water bottles (tippy-tap) and they distributed soap, which was good for us. All this to increase the healthiness in the village. That's what I can say about the project.” (Kédougou, men’s GD) On the use of tippy-taps promoted by the project, respondents in Yaajeende villages and KIIs with Yaajeende volunteers and health agents indicated that these were not always seen as the preferred option, and many eventually came into disuse. Some respondents indicated that people stopped using tippy-taps because their upkeep and maintenance was seen as too time-consuming. This appears to be driven at least in part by general water shortages in villages, long wait times at wells and the long distance from households to water points that adds to household labor to keep the tippy-tap water replenished. KIIs with health focal points at commune and higher administrative levels indicated a need for longer sensitization and more dedicated follow-up to ensure that communities were embedding their use into their daily lives. Less commonly, it was also suggested that people remove tippy-taps from their households during the rainy season, because water stagnates there or children break them too easily. “There are no ineffective activities but rather misunderstood activities and a lack of follow-up. All the activities we did with them (Yaajeende) are effective, but it depends on their level of understanding of the beneficiaries and the follow￾up at the end of the project. If I take the example of the [tippy-tap] and improved canary tap (des canaries à robinet améliorée), the population knows the interest of these activities but there is no follow-up. And if there is no continuity, people return to their old habits.” (Bakel, health focal point KII) “When setting up the [tippy-taps], they (the project) promised to follow up, but since then they have not been seen again, so there is no will to continue (using them) on the part of the villagers.” (Matam, non-GDG women’s GS) “The problem with [tippy-taps] is maintenance. They are put in place, but people do not renew the water often. The other problem too, we put soap but it does not stay in place, it disappears after a while. Even at the level of health facilities, people had put handwashing stations, but it is sustainability that is a problem.” (Matam, nutrition focal point KII) EVALUATION QUESTIONS 3 AND 4: HETEROGENEITY OF IMPACTS AND POTENTIAL DRIVERS Evaluation Question 3: How do program impacts differ for key subgroups of interest across key outcomes? The evaluation will assess two subgroups: northern regions (Matam and Bakel) vs. southern region (Kédougou); and poorest households vs. other households. YAAJEENDE FINAL IMPACT EVALUATION REPORT ANNEXES 53 Evaluation Question 4: What are potential explanatory reasons for variations in key outcomes across the subgroups? Is there evidence of differences in results patterns across regions? Evidence from region-specific regressions shows that results were quite variable across regions (see regional results in Annex II). The estimates for the effect of the program on the prevalence of handwashing stations in the northern regions of Bakel and Matam are positive, but near zero and not statistically significant. The differential impact for the Kédougou Region, however, is negative and statistically significant. This effect is quite strong, and when adding the small, positive Yaajeende treatment effect to the much larger, negative differential effect to obtain the estimated treatment effect of Yaajeende in Kédougou, the estimate predicts a 22.2 percentage point decline in the prevalence of handwashing stations, relative to trends in comparison villages. On the other hand, estimates for the effect of Yaajeende on iodized salt usage and storage were near zero and show no signs of statistical significance, regardless of the region. Is there evidence of differences in results for the poorest households? Some differences can be observed comparing poorer households to those that are less poor. For poorer households, the estimates suggest a statistically significant 15.9 percentage point decline in the likelihood that a household in a Yaajeende village has a soap-and-water handwashing station, relative to the trend for poorer households in comparison villages. The coefficient for the effect on the likelihood of proper use and storage of iodized salt is also negative for poorer households, but the result is not statistically significant. Less-poor households show no evidence of statistically significant impacts for either outcome. The time trend for poorer households is positive and larger in magnitude than for less-poor households, and is statistically significant for the likelihood of proper iodized salt use and storage. This can be seen by examining the differential time trend for poor households (Poor*Endline), measuring the average difference between poorer and less poor in the change between midline and baseline, and adding it to the global time trend (Endline). This produces the estimated time trend for all poorer households, and suggests that poorer households had larger improvements on salt usage than less-poor households since midline. Given the substantial qualitative and household survey evidence that WASH programming also occurred in comparison villages, one interpretation of these results is that while training on household healthy practices (Yaajeende or otherwise) may have led to some progress toward these outcomes, outside programming in the comparison areas may have more effectively targeted or elicited behavior change for poorer households during this time frame. Is there evidence of stronger impacts for households that participated in Yaajeende activities? Results using the village-level treatment intensity measure and triple interaction term between treatment, endline and treatment intensity are in Table 11. No evidence suggests that households in treatment villages with higher training participation experienced outsized benefits relative to those in comparison villages with similar levels of participation intensity. This is demonstrated by the small, statistically insignificant coefficients on the triple interaction term for both outcomes. The estimates on the triple interaction term in Table 11 have little variation across alternative model specifications. Figure 11 presents the estimates from Table 11 alongside the estimates using the household- and village-level treatment intensity measures, for both the standard DID and DID with entropy weighting. While the overall size of the estimates generally shows consistency, greater statistical significance exists for the household-level measure of training intensity. In all cases, the results continue to suggest a consistent trend of relatively higher impacts in YAAJEENDE FINAL IMPACT EVALUATION REPORT ANNEXES 54 comparison areas: despite overall improvements on these outcomes over time and accounting for variation in treatment intensity across villages, households in Yaajeende villages with greater exposure to trainings showed relatively poorer performance on these outcome indicators compared to households with similar exposure to (non-Yaajeende) training sessions in comparison villages during the same time frame. TABLE 11. OUTCOME FAMILY 3 VILLAGE FIXED EFFECTS DID RESULTS: HOUSEHOLD-LEVEL OUTCOMES, ML-EL. VILLAGE-LEVEL TREATMENT INTENSITY EFFECTS. VARIABLES 3.1 3.2 Verified soap and water handwashing station (binary) Iodized salt properly obtained and stored Intensity Differential -0.067 0.016 Effect (Treat*Endline*Intensity) (0.082) (0.102) Yaajeende Treatment Effect -0.003 -0.147 (0.066) (0.139) Endline*Intensity 0.069 0.065 (0.079) (0.099) ln(Household Members) 0.025** 0.010 (0.010) (0.017) ln(House Head Age) 0.005 -0.096* (0.020) (0.052) Household Head Has At Least Elementary Education 0.003 -0.034 (0.054) (0.021) Endline -0.039 0.163 (0.041) (0.126) Observations 4,903 4,568 Treatment N 3638 3401 Control N 1265 1167 Diff. Effect 95% CI [ -0.228; 0.093] [ -0.185; 0.216] Effect Size -0.204 0.033 *** p<0.01, ** p<0.05, * p<0.1 Note: Robust standard errors in parentheses clustered at the village level. Effect Size is for the triple interaction term, calculated as the estimated coefficient divided by the standard deviation of the outcome at endline. YAAJEENDE FINAL IMPACT EVALUATION REPORT ANNEXES 55 FIGURE 11. OUTCOME FAMILY 3 COMPARING TREATMENT INTENSITY EFFECTS BY MODEL: HOUSEHOLD￾LEVEL OUTCOMES 00 EVALUATION QUESTION 5: MODERATING CONTEXT FACTORS Evaluation Question 5: How do key individual and household characteristics shape program impacts? Households with a greater number of members are more likely to have soap-and-water handwashing stations, but no evidence suggests a relationship between household size and iodized salt usage and storage. Perhaps counterintuitively, there is no evidence of an association between head of household’s education status and either of the household healthy practices outcomes. Similarly, there appears to be no association between prevalence of handwashing stations and the age of the head of household, while some signs point to a small, negative relationship between the head of household’s age and the use and proper storage of iodized salt. OUTCOME FAMILY 3 CONCLUSIONS In conclusion, the results for Outcome Family 3 suggest that the Yaajeende project had no impact on the outcome indicators related to household healthy practices. The time trends between 2015 and 2018 show that improvements did occur in these outcome indicators, particularly for usage and proper storage of iodized salt and particularly for households with greater exposure to training sessions, and that these time trends were strongest for poorer households. However, because these improvements occurred in both treatment and comparison villages, perhaps owing to WASH programming in comparison villages from other outside programming, the DID model does not allow for conclusive attribution of these improvements to this programming (Yaajeende or otherwise), rather than changes that may have occurred regardless. -0.150 -0.124 -0.227 -0.150 -0.182 -0.095 -0.204 0.032 3.1 Soap and Water Handwashing Station 3.2 Iodized Salt Properly Obtained & Stored -1.0 -0.5 0.0 0.5 Household-Level Intensity Household-Level w/ Entropy Balancing Village-Level Intensity Village-Level w/ Entropy Balancing Note: Plot shows the estimated effect sizes and 95% confidence intervals for the effect on the triple interaction term between treatment, endline, and treatment intensity, from regressions with controls. Household Healthy Practices YAAJEENDE FINAL IMPACT EVALUATION REPORT ANNEXES 56 OUTCOME FAMILY 4: HOUSEHOLD AGRICULTURAL PRACTICES Outcome Family 4 has three outcomes: agricultural investment index, use of community-based service providers (CBSPs) and agricultural production (of the four most important crops for the household). EVALUATION QUESTIONS 1 AND 2: OVERALL PROGRAM IMPACTS AND DRIVERS OF IMPACTS Evaluation Question 1: What are the impacts of the Yaajeende NLA package on the prevalence of poverty and malnutrition six years after the start of program implementation across four thematic categories (women’s and children’s nutrition; household food security and poverty / economic well-being; household water, sanitation and hygiene practices; and household agricultural practices)? Evaluation Question 2: What major factors or processes contributed to observed impacts, including the role and importance of Yaajeende-supported local institutions? Results from the DID model with village fixed effects show that the Yaajeende project increased agricultural investment in households in Yaajeende villages relative to comparison group households in villages exposed to varying integrated MCHN/WASH and agricultural programming during the same time frame (Table 12). The agricultural investment index ranges from 0 to 11, measuring the number of areas in which men and women in the household adopted agricultural technologies for improving crop production, including preparing fields with a tractor, implementing erosion control and making compost. The estimated effect suggests that Yaajeende programming resulted in an average increase of 0.262 practices adopted per household, statistically significant at the 95 percent confidence level. While the corresponding effect size of 0.336 is moderate from a policy standpoint, the mean number of technologies used by households is generally low and remained so by endline. The evaluation does not find evidence for statistically significant Yaajeende project impacts on use of CBSPs or agricultural production, although the estimated effect on these two outcomes is positive and in the expected direction. At endline, Yaajeende households had a mean agricultural investment score of 0.5 (on a scale of 0-11), and 15.6 percent of households reported use of a CBSP. Garden access was reported by 55.9 percent of households in Yaajeende villages, relative to 30.8 percent of comparison village households. Value chain participation and the number of value chain activities that a household participated in was nearly equal across households in Yaajeende and comparison group villages, at 40 percent and one activity, respectively. The total agricultural production reported by households in Yaajeende villages at endline was nearly five times higher than for households in comparison group villages at endline, although Yaajeende villages also had somewhat higher production at midline as well (see Annex II for additional results). In contrast to the previous three outcome families, the overarching time trend for outcomes related to household agricultural practices suggests a decline in all three outcomes over midline to endline, and this trend is statistically significant for agricultural production. This suggests that households in both treatment and control villages produced approximately 1,224 kg less in the 12-month period preceding the 2018 interview than in the same period before the 2015 survey. A similar decline is also observed between baseline and midline, and for both time periods, the decline appears driven by an increased percentage of households that reported farming but with no production (interpreted as a failed harvest). The midline to endline findings for this outcome family could be conducted only for agriculture production, as agriculture investment and CBSP data were not collected at baseline. Baseline to endline analyses conducted by the endline evaluation team, although considered less robust by the ET than the ML-EL findings due to power limitations and concerns on baseline data reliability, do show evidence for a positive, statistically significant impact of the Yaajeende project YAAJEENDE FINAL IMPACT EVALUATION REPORT ANNEXES 57 on agriculture production. The evaluation team estimates that between baseline and endline, households in Yaajeende villages increased production of their four most important crops by 631.094 kg per year as a result of the program, relative to trends in comparison villages (see Annex II for additional presentation of the baseline to endline results). TABLE 12. OUTCOME FAMILY 4 VILLAGE FIXED EFFECTS DID RESULTS: HOUSEHOLD-LEVEL OUTCOMES VARIABLES 4.1 4.2 4.3 Agriculture investement index Household uses CBSP / APS (binary) Total household agriculture production Yaajeende Treatment 0.262** 0.038 355.508 Effect (0.112) (0.039) (220.250) ln(Household Size) 0.064 0.008 347.488*** (0.047) (0.012) (46.702) ln(Head Age) 0.062 0.003 54.790 (0.076) (0.025) (144.970) Head Education Level: Household Head Has At Least Elementary Education = 1 0.017 -0.008 223.899** (0.048) (0.016) (98.835) Endline -0.113 -0.036 -1,223.969*** (0.084) (0.023) (207.416) Observations 4,422 4,424 4,818 Treatment N 3250 3252 3576 Control N 1172 1172 1242 Treatment Effect 95% CI [ 0.042; 0.482] [ -0.038; 0.115] [ -76.175; 787.190] Treatment Effect Size 0.336 0.134 0.555 *** p<0.01, ** p<0.05, * p<0.1 Note: Robust standard errors in parentheses clustered at the village level. Effect size calculated as the estimated coefficient divided by the standard deviation of the outcome at endline. As for the previous outcome families, the evaluation team checked the robustness of the DID estimates through alternative model specifications that incorporate entropy balancing and household-level fixed effects. To facilitate comparison, Figure 12 plots the size estimates of the treatment effect of the Yaajeende project from each of the three models. The result from the DID model with entropy weighting and village fixed effects for the effect of Yaajeende on agricultural investment is corroborated by the standard DID model, which estimates a slightly smaller increase of 0.182 technologies adopted per household, again statistically significant at the 95 percent confidence level. In terms of effect sizes, the estimate varies between 0.213 standard deviations of the outcome variable in the standard DID model, compared to 0.336 standard deviations for the DID model with entropy balancing. The magnitude of the effect from the household panel model is similar to the standard DID model, but the estimate is not statistically significant. This is likely due to the loss in statistical power from fewer observations in the household panel and a much larger number of covariates (i.e., one fixed effect per household, instead of one per village in the standard model). Regardless of the model used, the results find no consistent evidence for a statistically significant effect of Yaajeende programming on the other two outcome indicators in this family. YAAJEENDE FINAL IMPACT EVALUATION REPORT ANNEXES 58 FIGURE 12. OUTCOME FAMILY 4 COMPARING TREATMENT EFFECTS BY MODEL: HOUSEHOLD-LEVEL OUTCOMES In terms of the qualitative findings at endline, the results present a mixed set of evidence on reasons for observed impacts, that at times corroborates and otherwise contradicts the statistical results. Twenty of 33 GDs in Yaajeende villages reported increased agricultural yields that they attributed to farming practices they learned through Yaajeende interventions, although this was more commonly stated in Kédougou and Matam than in Bakel (11 GDs in Matam, seven in Kédougou and two in Bakel). Half of the GDs that noted an increased yield were with GDG members. Some men’s group respondents noted the caveat that large yields can be realized if the farming practices and directions introduced by the program are followed. Increased harvests under Yaajeende have allowed people to feed themselves throughout the year. In some cases, respondents noted that the increase in yields allowed them to sell some portion of their harvest and have greater financial autonomy. In general, participants in Yaajeende programming reported that members of their community know how to farm more effectively as a result of trainings held by the project. “Today, thanks to the experience we gained, we can cultivate everything we want. If you see how we grow onions today in our community, you will know that it is totally different from the how it was done before. And this thanks to Yaajeende, the experience is never lost.” (Matam, GDG GD) In addition, respondents from Yaajeende villages highlighted improvements on a range of agricultural technologies, including: access to and use of improved seed varieties, better knowledge of how to protect crops from pests, how to cultivate effectively during the rainy season and the off-season, increased fertilizer use and use of composting and improved access to agricultural machinery and equipment due to Yaajeende subsidization of machinery such as tractors. 0.213 0.066 0.045 0.336 0.134 0.555 0.187 0.106 0.068 4.1 Agricultural Investment Index 4.2 Uses CBSP/APS 4.3 Total Agricultural Production -0.5 0.0 0.5 1.0 1.5 Unbalanced DID Entropy Balancing DID Household Panel Note: Plot shows the estimated effect sizes and 95% confidence intervals for the treatment effect of the Yaajeende program from regressions with controls. Household Agricultural Practices YAAJEENDE FINAL IMPACT EVALUATION REPORT ANNEXES 59 Communal or group savings and credit systems introduced by Yaajeende, though uncommonly mentioned, were in some cases noted by respondents to be useful in enabling participation in some agricultural activities. Members of two GDG GDs across Matam and Kédougou noted that this helped facilitate purchases of materials like fertilizer and obtaining tractor services. However, qualitative data collection in comparison group villages at endline indicate that many of these agricultural activities were also underway in comparison group villages over the same time frame. This similar level of exposure to and apparent perceived improvements on agricultural practices and availability of inputs in comparison areas, ostensibly through other donor programming also active in these areas, is likely a strong reason why the statistical results find little to no evidence of a comparative effect from Yaajeende programming. For example, GDs with women in comparison group villages across the three regions also mentioned the introduction of machinery as a reason for improved yields, as well as increased use of herbicides, community gardens and learning practices to more effectively farm during the cold season. Regarding crop loss or harvest failures, the quantitative findings with respect to an increased number of farmers reporting failed harvests is also present to some extent in the qualitative data at endline. For example, respondents from three Yaajeende GDs and one comparison village GD, all in the Kédougou Region, noted difficulties in sustaining a harvest throughout the year, while a men’s GD in Matam noted that harvest losses were a common occurrence in general. Another Yaajeende GD in Kédougou noted limited uptake of new farming methods that the project promoted, and suggested that farmers had returned to their traditional farming practices because they had not seen improved results using the new techniques. General issues with poor soil quality and water constraints were also commonly noted as contributing factors. “All I can add is that we owe all this knowledge to Yaajeende. All of the losses on crop harvests are attributable to a low level of knowledge (beforehand). We used to use five tubs (of seed), but the harvest was not good. Our volunteer passed on the knowledge he had received, and it allowed us to understand the (appropriate) watering cycle after sowing. Now we see increased interest in cultivating the land, and we are hoping to have a good harvest. All this is thanks to the training we received.” (Matam, GDG GD) Some GDs in Yaajeende villages attributed their poor or lost harvests to a delayed receipt of seeds by the project, although this was not independently confirmed through the endline data collection. Four of 33 GDs in Yaajeende villages said that the seeds Yaajeende provided took too long to grow (three GDs in Bakel; one in Matam). In another three of 33 GDs in Yaajeende villages (two in Matam; one in Kédougou), respondents said that seeds were not available on time, and they felt this negatively impacted the trajectory of their growing season. “This year the harvests were worse. Nobody has harvested even a small spike. We will even have a seed problem (for next year). The problem is in this whole area. In addition, if we do not receive seed on time, we will not be able to sow. We seek help on time. We ask for support in seeds and agricultural equipment.” (Matam, men’s GD) One treatment group with men from Matam and one KII respondent from Bakel said the seeds they received from the project were not good quality, and one treatment group said the new seeds have not been linked to good harvests. One KII respondent from Kédougou noted that in some cases, there had been no adoption at all of new seeds, and the older seed varieties were used instead. In other cases, respondents attributed poor harvests to their lack of following new practices that Yaajeende brought, placing the responsibility for crop loss on themselves: “I did not notice improvement at this level (on crop harvests). For two years now, our harvests are bad. They (Yaajeende) taught us a lot, but the problem is that in practice we do not do as they request. That's why for these past two years, we did not harvest anything. If we had respected the recommendations, we would have good harvests. We want to YAAJEENDE FINAL IMPACT EVALUATION REPORT ANNEXES 60 stop growing souna millet and continue with the cultivation of watermelon. Now, almost all farmers grow watermelons. With the souna, the harvest can last for four months, while the l’hivernage no longer lasts for four months. We would like to have new adapted seeds or support for watermelon growing.” (Matam, men’s GD) The overarching limitation on stronger improvements to agricultural yields, however, was water constraints, which was mentioned in all but one of the GDs and KIIs conducted at endline, irrespective of Yaajeende or comparison village status. Other common reasons cited for poor or low harvests included pests and animals that destroyed crops in unprotected fields. Related to yields, Yaajeende beneficiaries also mentioned crop storage issues and continuing challenges, with implications for food availability over the year. Seven of 33 GDs held in Yaajeende villages noted difficulties successfully storing their crops after collection. Reasons for these challenges included pests, lack of space to store items communally, climate-related and other type of spoilage and certain crops like onions spoiling more quickly. With respect to Yaajeende trainings related to agriculture, trainings conducted by Yaajeende included instruction on both general and crop-specific planting techniques, and these were seen as useful (four of 33 GDs in Yaajeende villages). A similar number of GDs in Yaajeende villages indicated that learning and new practices uptake from the trainings was not always extensive. One men’s GD in Kédougou noted that trainings took place in their village, but they did not adopt the planting practices because they did not have all of the tools required to do it. A women’s GD in Bakel said that the trainings they received reinforced their existing capacity, but they did not feel that they learned anything new. More rarely, GD respondents noted that others in their village had been trained on new agricultural techniques through the project, but this knowledge transfer had not been passed on to them (one women’s GD in Matam). With respect to livestock, beneficiaries indicated they received training on issues including livestock breeding, enclosure construction and cleaning, land management for livestock, animal health, vaccinations and related care. GDs with Yaajeende beneficiaries mentioned several areas of training where respondents expressed a desire for additional trainings to further strengthen their capacity to sustain and benefit from the knowledge and farming activities brought by the project. Topics mentioned for additional training included: textile dyeing (teinture); linking producers to vendors; processing and other activities related to product transformation and value chain participation; how to calculate production costs to gauge profitability; knowledge on how to estimate maximum potential yields and revenue; and amount of seed required for a given area of land and crop grown. In addition, some indications emerged of a perception of insufficient project support by some of the GDs with Yaajeende beneficiaries, which could also indicate a degree of variability in project support or effectiveness across the large number of villages covered by the project. A Kédougou Region KII noted that no materials had been provided to the village, and the project also did not provide follow-up services after activities were underway. In four of 33 GDs in Yaajeende villages, respondents said they could not implement the agricultural techniques they learned through the project because they didn’t have the associated materials required to do it (one GD in Kédougou; two in Bakel; one in Matam). More uncommonly, some respondents expressed a feeling of broken promises from Yaajeende, whereby they felt that certain services or material support had been assured by the project but did not arrive (one KII in Kédougou; one women’s GD in Bakel). Although it was not commonly expressed, respondents in one Bakel women’s GD expressed a sentiment somewhat akin to information overload: “We limit ourselves only to what they showed us, we have no other choice. On the livestock side, they brought calendars, they made us aware of how to maintain / care for the cattle and their pens; they also showed us how to make compost with animal waste. We found it all too difficult for us. We did it ourselves once but afterwards we had to abandon all YAAJEENDE FINAL IMPACT EVALUATION REPORT ANNEXES 61 these practices. We also received training on good animal feed practices, and at the same time they advised us to use veterinarians to monitor their health.” (Bakel, non-GDG women’s GD) Finally, some respondents said relatively few people in their village directly benefited from Yaajeende activities, or that the assistance provided by the project simply did not make a noticeable difference for their agricultural production and related farming needs, due to broader limitations on distribution of inputs and overarching constraints from a lack of water. This sentiment was expressed in five of 33 GDs in Yaajeende villages and spanned discussions with men, GDG members and other women in the village (three GDs in Matam; two in Bakel). “There was a project staff who came from the Ndouloumdji area, and they told us about agriculture. But, personally, I did not notice a change. There are those who have benefited from fertilizer, but I have not received it yet. Before, I farmed close to 9 hectares, but this year I did not harvest anything. It's divine will. Personally I have not yet received support from Yaajeende. I did not receive fertilizer. Those who have not received pumps, insects destroy our crops. It's just two or three houses they give (benefits to), which is minimal to bring about real change. I do not know how to inquire (about participation).” (Matam, men’s GD) Overall, farmers pointed to many existing challenges to production, and conveyed a sense of varied success despite engaging with Yaajeende, that in some ways was seen as outside of the project’s control. “It is said in Pular that 10 will cultivate, three will win and seven will lose.” (Matam, men’s GD) With respect to use and access to community or microgardens, women in GDs across the three regions indicated that the introduction of gardening activities by Yaajeende had brought them tangible benefits. In some cases, women indicated that the project had helped their women’s group to secure land for a community garden, three of 33 GDs in Yaajeende villages viewed gardening activities to be among those that would be sustainable after the project ended. One men’s group in Bakel noted that gardening activities had provided women with a livelihood and a source of income, which allowed them to achieve some independence over their purchases. While community gardens appear to have been widely implemented and generally well-received in several Yaajeende villages, evidence also suggested that such gardens had been implemented through other donor activities in area comparison villages, again contributing to a lack of statistical evidence for an increase in garden access at endline through the project. In addition, both comparison group and Yaajeende village GDs highlighted insufficient inputs, including water resources, as a limiting factor on gardening success. Other input constraints noted included: seeds, fertilizer, water, fences, land, soil, forage, diesel and general materials and financial support. Insufficient financial support was stated repeatedly as a key factor impeding gardening success. With respect to value chain participation, household survey data at endline found no difference in value chain participation between Yaajeende households and those in comparison villages, which was universally low across respondents. Counterfactual treatment effects are not estimated for this outcome due to the absence of midline data. Qualitative data indicated interest and some level of uptake of value chain activities as a result of involvement with Yaajeende, particularly for soap-making, which was seen as somewhat more profitable. Other transformation activities that were mentioned by Yaajeende participants included bissap juice production, shea butter processing, jam and cheese making and processing of sweet potatoes. However, respondents also mentioned several challenges to greater realization of benefits through value chain activities, including lack of resources to procure inputs or related cost barriers, difficulties accessing buyers, lack of local markets (in Bakel) and market saturation (in Kédougou) that was seen to limit profits. While respondents indicated they had received a number of trainings from Yaajeende on these activities, some GD respondents expressed a YAAJEENDE FINAL IMPACT EVALUATION REPORT ANNEXES 62 desire for additional trainings (particular noted for soap-making), or noted that follow-up was needed because people would forget how to do the activity after the training is completed. While some value chain activities and participation in trainings were noted in comparison group villages, these were generally more limited (noted with respect to soap-making and production of beignets), and were restricted to a smaller proportion of comparison group villages visited (one village of three visited in Matam and in Bakel). EVALUATION QUESTIONS 3 AND 4: HETEROGENEITY OF IMPACTS AND POTENTIAL DRIVERS Evaluation Question 3: How do program impacts differ for key subgroups of interest across key outcomes? The evaluation will assess two subgroups: northern regions (Matam and Bakel) vs. southern region (Kédougou); and poorest households vs. other households. Evaluation Question 4: What are potential explanatory reasons for variations in key outcomes across the subgroups? Is there evidence of differences in results patterns across regions? Comparing the results from region-specific regressions, some differences are evident, generally showing greater program effects in Matam and Bakel, with null or muted effects in Kédougou. The significant estimate for the effect of Yaajeende on agricultural investment in Table 12 appears to be driven by changes in the northern regions of Matam and Bakel, where the estimate suggests the Yaajeende project increased agricultural investment by 0.260 technologies per household on average. This result is not statistically significant at conventional levels, but it shows some signs of marginal significance (p=.11). The estimated effect on investment in Kédougou is notably smaller and shows no signs of statistical significance. For agriculture production, the evaluation team found that the program had an impact on Matam and Bakel, where the estimated effect is an increase of 524.196 kg in the production of the four main crops per household per year, relative to trends in comparative villages between midline and endline. This is slightly below, but still in line with, the overall estimate from the baseline to endline regressions described previously. On the other hand, the differential treatment effect for Kédougou seen in the triple interaction term is strongly negative, leading to an overall treatment effect for Kédougou that is near zero and not statistically significant. Is there evidence of differences in results for the poorest households? The significant estimate for the effect of Yaajeende on agricultural investment in Table 12 appears driven by poorer households. For poorer households, the estimated effect is an additional 0.226 new methods or technologies adopted per household, statistically significant at the 95 percent level. On the other hand, for less-poor households, the estimated effect is near zero and shows no signs of statistical significance. Some evidence shows that the program had some impact on poorer households in terms of agriculture production. For less-poor households, the estimated coefficient suggests a reduction in agricultural production of 290.979 kg associated with Yaajeende villages, which shows some borderline statistical significance, but is not significant at conventional levels. However, the estimates show that the impact in poorer households is statistically different from the impact in less-poor households, estimating an increase of 530.282 kg on top of the effect for less-poor households. However, the combined effect for poorer households obtained by adding the Yaajeende treatment effect and the differential effect for poorer households, while positive, is not statistically significant. Is there evidence of stronger impacts for households that directly participated in Yaajeende activities? YAAJEENDE FINAL IMPACT EVALUATION REPORT ANNEXES 63 The evaluation does not find evidence for stronger impacts on agricultural outcomes for households with greater exposure to Yaajeende trainings. Table 13 presents the results using the village-level measure of treatment intensity, adding a triple interaction term between treatment, endline, and intensity to the DID model with entropy weighting. The coefficient on the triple interaction term is small for the effect on both agriculture investment and use of CBSP, and is not statistically significant. The estimate for the effect on agriculture production is larger, but does not go in the expected direction, and also shows no signs of statistical significance. TABLE 13. OUTCOME FAMILY 4 VILLAGE FIXED EFFECTS DID RESULTS: HOUSEHOLD-LEVEL OUTCOMES VARIABLES 4.1 4.2 4.3 Agriculture investment index Household uses CBSP / APS (binary) Total household agriculture production Intensity Differential 0.072 0.011 -111.888 Effect (Treat*Endline*Intensity) (0.170) (0.051) (383.791) Yaajeende Treatment Effect 0.166 0.021 456.930 (0.242) (0.077) (440.046) Endline*Intensity -0.077 -0.004 104.088 (0.160) (0.042) (376.323) ln(Household Members) 0.071 0.008 351.553*** (0.046) (0.013) (50.543) ln(House Head Age) 0.039 0.005 62.672 (0.070) (0.025) (140.534) Household Head Has At Least Elementary Education 0.012 -0.009 220.410** (0.049) (0.016) (93.746) Endline -0.009 -0.030 -1,311.395*** (0.212) (0.057) (407.649) Observations 4,422 4,424 4,818 Treatment N 3250 3252 3576 Control N 1172 1172 1242 Diff. Effect 95% CI [ -0.262; 0.406] [ -0.089; 0.110] [ -864.103; 640.328] Effect Size 0.091 0.037 -0.175 *** p<0.01, ** p<0.05, * p<0.1 Note: Robust standard errors in parentheses clustered at the village level. Effect Size is for the triple interaction term, calculated as the estimated coefficient divided by the standard deviation of the outcome at endline. The null result for the differential impact by treatment intensity holds across alternative model specifications as well. Regardless of whether the village-level measure of treatment intensity is used or the household-level measure, and whether using the standard model or the model with entropy weighting, the estimate for the differential impact of Yaajeende programming for households with greater exposure is near zero and shows no signs of statistical significance. YAAJEENDE FINAL IMPACT EVALUATION REPORT ANNEXES 64 FIGURE 13. OUTCOME FAMILY 4 COMPARING TREATMENT INTENSITY EFFECTS BY MODEL: HOUSEHOLD￾LEVEL OUTCOMES EVALUATION QUESTION 5: MODERATING CONTEXT FACTORS Evaluation Question 5: How do key individual and household characteristics shape program impacts? Household-level control variables generally go in the expected direction, but are statistically significant only for the effect on agriculture production. Larger households show slightly higher agricultural investment and have greater agricultural production. The household head’s education status appears to have no association with agriculture investment or use of CBSP, though households where the head has at least an elementary education have agriculture production that is 223.899 kg higher per year, on average, than households where the head has no education, and this difference is statistically significant at the 95 percent confidence level. In terms of the role of agents established through Yaajeende, support from project local partners appears to have been seen as beneficial in several villages. GDGs reportedly served an important role in enabling women to become involved in activities, bring community members together and playing a role in knowledge transfer through organized sensitization meetings and trainings. APS and VNCs were noted as playing useful roles as well, and APS partners appear to have been viewed as being well-trained to support agricultural producers. “The Yaajeende project trained the APS so that they know, for example, that this type of salad or carrot should be used at certain times of the year. Thus they orient the customers taking into account this aspect. APS are trained to be able to advise producers.” (Matam GTC KII) In general, for GDs in Yaajeende villages, VNCs were described as having supported micro- and community gardens and providing materials for villagers, while GTC members were also noted to have changed women’s growing habits and provided material inputs such as seeds and foods. -0.036 0.031 0.016 0.002 0.070 -0.017 -0.001 0.107 0.121 0.089 0.038 -0.173 4.1 Agricultural Investment Index 4.2 Uses CBSP/APS 4.3 Total Agricultural Production -1.5 -1.0 -0.5 0.0 0.5 1.0 Household-Level Intensity Household-Level w/ Entropy Balancing Village-Level Intensity Village-Level w/ Entropy Balancing Note: Plot shows the estimated effect sizes and 95% confidence intervals for the effect on the triple interaction term between treatment, endline, and treatment intensity, from regressions with controls. Household Agricultural Practices YAAJEENDE FINAL IMPACT EVALUATION REPORT ANNEXES 65 OUTCOME FAMILY 4 CONCLUSIONS The results from Outcome Family 4 suggest the Yaajeende project had an impact on increasing investment in agriculture. Specifically, the results imply households in treatment villages increased their use of agriculture delivery services, such as preparing fields with tractors, pumping water and using mechanical harvesters, and made additional investments in crop production practices, such as investing in erosion control, crop protection, soil water conservation or compost-making. On an investment index ranging from 0 to 11, measuring the number of areas in which the household made investments, the estimates for the effect of Yaajeende suggest the project increased investment by 0.182 to 0.262 additional areas per household, on average. Additional evidence suggests that the effect may be driven by poorer households and households in the northern regions of Matam and Bakel. Together, the results provide no evidence to suggest that Yaajeende had any effect on usage of CBSP. Similarly, no evidence indicates that households with greater exposure to Yaajeende experienced greater program effects, in terms of agricultural investment, use of CBSP or agricultural production. The results for the impact on agriculture production are somewhat less conclusive, but suggest that some effects did occur. None of the evaluation team’s models show a statistically significant impact for the effect of Yaajeende on the overall agricultural production of treatment villages from midline to endline, relative to comparison villages. However, the results from regressions estimating the impact from baseline to endline do show evidence for positive, statistically significant increases in production as a result of the program. This, combined with results from the regional midline-endline regressions, showing the program had a positive impact on production in Matam and Bakel, support the conclusion that the program was effective at increasing household agricultural production, at least in some regions. For Matam and Bakel, we estimate Yaajeende increased the agriculture production of households’ four most important crops by approximately 524 kg per year, relative to comparison villages. EVALUATION QUESTION 6: HOUSEHOLD CHARACTERISTICS ASSOCIATED WITH POVERTY AND MALNUTRITION REDUCTION Evaluation Question 6: What characteristics of households and mothers appear to be associated with successful poverty and malnutrition reduction for children under age of 5 and women of reproductive age? This section summarizes the results of targeted follow-up analyses conducted by the endline evaluation team to explore characteristics of households associated with household-level poverty reduction, and characteristics of households and mothers that are associated with malnutrition reduction for children under 5 years and women of reproductive age. The key factors pointed to through these analyses are overwhelmingly household characteristics that much existing literature identifies as strong determinants of household poverty and malnutrition status. These are women’s age, household head’s level of education and household size, generally all statistically significantly associated with the outcomes tested across all outcomes families, in the expected direction. With respect to the role of project participation factors, in addition to the broader Yaajeende treatment status and household-level participation intensity variables reported previously, the evaluation team also tested for an association between Debbo Gallé group participation and positive change on women and children’s nutritional status and diet indicators, as well as HDDS and likelihood that the household is below the poverty line. The GDG groups facilitated more intensive and comprehensive collaboration with Yaajeende project’s core target populations (i.e., women of reproductive age and children under 5 years old). Therefore, it may be likely that YAAJEENDE FINAL IMPACT EVALUATION REPORT ANNEXES 66 those who participated in these groups experienced greater improvements in outcomes from midline to endline than those who did not participate did. To test whether households with GDG participants experienced improvements in key outcomes, the evaluation team restricted the sample to households in Yaajeende treatment villages only and ran a DID model that replaces the Yaajeende treatment variable with a dummy variable, indicating whether or not anyone in the household participated in a mothers group. The regressions were run with entropy weighting. The resulting estimate gives the effect on the outcome of mothers’ group participation in the household, relative to trends among non-GDG households in Yaajeende villages. In general, the evaluation team found no evidence of statistically significant treatment effects for participation in mothers’ groups in Yaajeende treatment villages, relative to trends for individuals in Yaajeende village households where no household member participated in a mothers’ group. The exception is a statistically significant increase in the prevalence of stunting, a result that is unexpected and difficult to interpret. Since participation in these groups was voluntary, one possibility is that individuals who faced some sort of negative household-level shock between midline and endline self-selected into GDG participation. Under such a scenario, participation would be correlated with negative household- and individual-level trends, which would give a biased impact of the effect of participation. In any case, the results overall fail to detect an impact for GDG group participation. The evaluation team additionally asked whether participation in women’s groups is more effective in villages where participation was greater. This could be the case, because participating in a strong mothers’ group is likely to be qualitatively different from a group in a village where participation is weak. The team constructed a village-level intensity variable, defined as the percentage of households in the sample that indicated they were participating in mothers groups’ in each village at endline. Again using only households in Yaajeende treatment villages, the team ran a regression analogous to the village intensity regressions described in the main text of the paper, this time interacting mothers’ group intensity with household-level participation in mothers’ groups. In general, for the individual-level outcomes, the results show no evidence for greater effects for participating households in villages where mothers’ group participation is stronger. Similarly, the evaluation team found no evidence of statistically significant treatment effects for participation in mothers’ groups in treatment villages, relative to trends for individuals in Yaajeende village households where no one participated in a mothers’ group. As for the individual-level outcomes, the team also saw no evidence for differential effects based on the village￾level strength of participation, while the overall treatment effect of participation is smaller for the effect on HDDS and is no longer statistically significant, compared to the previous regressions on these outcomes (see Annex II for related statistical results). PROJECT IMPLEMENTATION The FIE is not tasked explicitly with an assessment of project implementation, but, as with any impact evaluation, an understanding of beneficiary views and experiences with respect to implementation of the project is a key piece of the puzzle to interpret the statistical impact results. On the basis of KIIs and GDs conducted at endline, it is clear that Yaajeende used several communications mechanisms to bring knowledge and promote behavior change in project villages. These include hands-on trainings (often conducted by VNC, APS and other relais), for example on the production of enriched flour, soap-making, jam production, plant grafting and zai and gardening techniques; group discussions and community meetings on a YAAJEENDE FINAL IMPACT EVALUATION REPORT ANNEXES 67 range of topics42 targeted to various demographic groups such as youth or GDG members; door-to-door visits to raise awareness or conduct child screenings and nutrition monitoring; community meal organization, including the use of boite a image to facilitate discussions; provision of booklets and other information resources on foods, meal preparation, nutrition and infant and young child feeding best practices; competitions for students on nutrition issues; and mass communication methods such as sensitization through broadcasting via cars, village announcements, songs and awareness-raising (e.g., through caravans) and through the use of government services on the same. The project also held trainings specifically for VNC and APS members on topics such as interpersonal communication skills, and topic-specific trainings on nutrition, WASH, horticulture, product transformation, savings and loans and others. Despite this broad and well-rounded approach, KIIs and GDs noted some challenges that may have served to limit project effectiveness on some outcomes. Noted challenges included: too few APS and VNC members in large villages to adequately spread information throughout the village, lack of knowledge or uneven awareness of village group discussions or meetings held by the project, insufficient frequency of awareness-raising or sensitization efforts, coordination challenges for village gatherings and general lack of interest to attend among beneficiaries. On beneficiary selection and targeting, the qualitative data confirm that women in GDG were the focus of project activities and appear to have been targeted with more resources relative to men and other villagers, such as women who did not join or become active members of Debbo Gallé groups. However, APSs, VNCs and GTCs reported working with other women’s groups in their communities as well, so most women were sensitized and informed on various issues the project promoted. Several communities indicated that older women were also targeted, particularly on nutrition for young children, because they often are secondary caregivers for young children and yet may fall back on outdated beliefs and practices that may be harmful to children, such as giving water to newborns. They were also viewed to have important counseling skills and influence in the community, and experience on pregnancy and birthing. In terms of appropriateness of targeting, some communities noted that youth in general had been neglected by the project, particularly regarding agriculture-related activities, and this was a missed opportunity, despite motivational challenges also being noted for youth, who are seen to have limited interest in agriculture and are more interested to migrate abroad or to cities. Some VNCs, however, reported working with youth on nutrition activities. While men were generally not considered to be the primary target of the project, they frequently reported involvement in agricultural activities and trainings. Generally speaking, the focus on women and young children was viewed as appropriate and relevant. Some noted that it was the best choice to get results or to achieve the project’s objectives. Rationales most commonly expressed in support of targeting women include that they know best how their children live and can influence their husband; and that they typically work and spend for their households, so men benefit too. In one case, men said that targeting their own demographic would not have worked, as they travel frequently and would not be able to reliably participate. The selection process by which individuals in Yaajeende villages came to be involved in project roles or activities was typically reported to be the responsibility of the GTC and was based on who in the community was present and willing to volunteer for positions. Key criteria used by the GTCs to make the selection were broadly shared across communities. They include being literate and smart, willing to document information received and able to 42 For example, on optimal breastfeeding practices, infant and young child feeding, vaccinations, use of iodized salt, WASH issues, use of mosquito nets and several others. YAAJEENDE FINAL IMPACT EVALUATION REPORT ANNEXES 68 share it with others. Trust and honesty were also reported as important criteria, as well as having time to commit to the project. Selection for these roles often took place during a community meeting, sometimes with Yaajeende project staff present. In some cases, these meetings were not well-publicized enough within the villages, leading to frustration and a lack of understanding about how the process worked. In general, project stakeholders indicated that they thought the process used was fair, well-done and equitable. They also agreed with the criteria used and thought they were important. The selection of the GTC members was reportedly done by leaders or members of other influential groups in the community, as well as CLPs, village chiefs, elected officials, imams and elders during general assemblies spanning many villages. In several locations, participants mentioned that the goal was specifically to have representation from each village the GTC would cover and for staff to come from different villages. Joining a GDG seemed a bit less standardized across communities, as was the number of such groups reported. Smaller communities seem to indicate having a few groups, whereas larger villages commonly noted having 10 or more GDGs, sometimes organized on a per-neighborhood basis. In some cases, it was reported that pre-existing groupings had been repurposed and reinforced by Yaajeende to become a GDG. In one location, GDG membership was reportedly driven by how well these previous groupings were performing in their previous activities. While it was commonly reported that anybody who wanted to join GDGs as welcome if they followed the rules and were engaged, GD respondents in some communities reported issues with exclusion. For example, non-GDG participants in one community reported that only 20 women had been selected for each GDG and additional groups had not been created, so that many who were interested in joining had not been able to. In another case, women who were already members of other community groups reportedly had not been allowed to join GDGs if they held a position in the existing group, and a cap on membership was also enforced. More commonly, GDs with women in Yaajeende villages who were not GDG members seemed to indicate a lack of awareness about meetings to select GDG members as a primary reason for their exclusion. Most commonly, it appears that the GTC and CLP would decide about allocation of project resources and who could participate in what activities. However, in some cases the president of GDGs also played an important role. GTCs reported supporting people ready to work as a priority, focusing on motivated people, underpinned by the logic was that those will yield visible results that might convince the skeptics to also become involved or to stay involved and engaged and apply the practices. Alternatively, random drawings were held in some communities, especially for passage du don43. In one case, the VNC reported creating lists of GDG women and selecting a certain number to get seeds and seedlings during meetings attended by GDGs so everyone understood. The logic cited was that GDG women all know each other and knew who would be most likely to succeed. It wasn’t clear whether worst-off or most-vulnerable households were given priority for project benefits. This was both lamented and celebrated by various project stakeholders, but on balance it seems that they were mostly satisfied with the fact that everyone was able to benefit. However, this lack of priority for the poorest households was likely compounded by the fact that certain activities required a contribution (passage de don, latrines, savings and loans), which created an additional hurdle for these households to be beneficiaries. 43 The project’s Pass on the gift program : placement of subsidized animals to vulnerable households, who must in tern pass on the same number of female animals, and technical knowledge on proper animal raisin methods, to another vulnerable person in the community.. YAAJEENDE FINAL IMPACT EVALUATION REPORT ANNEXES 69 This is somewhat in contrast to comparison villages, where GDs and KIIs with respondents described several projects that had specifically targeted those who had the fewest means and were the most vulnerable. Government services also noted that they specifically target and provide resources to the poorest, based on village-level official data and then within these communities, by asking chiefs, elders and other leaders about who is most in need. Discrepancies in terms of project activities and ability to participate were noted occasionally. They were primarily attributed to three reasons: poor or unfair project implementation and oversight, lack of means to partake in certain activities viewed as most valuable, or lack of interest or refusal to participate by some households. A lack of interest or refusal to participate by some households was the most commonly cited reason for these discrepancies. This was reported as a key driver of discrepancies between households in project villages, because most project activities and roles require one to volunteer or express interest to get involved. For example, GDGs were often formed early on and did not always accept new members. Furthermore, motivation and dynamism were noted as key criterion for selection of APS, VNC and other roles in the project. Individuals also must have sufficient interest and motivation and interest in learning to take the time to attend trainings and group discussions. In terms of selection for activities within GDGs, in some cases KIIs noted that selection for participation is based on who in GDGs is expected to succeed. On inclusion and participation, some women in Yaajeende villages who were not GDG members said that women were selected among those who came to a meeting. Many women weren’t there because they were not given advance notice and did not receive direct outreach. In a small number of cases, KIIs indicated that a women had to do at least one training to participate in the GDG. In at least four villages, GDGs were reported to have already ceased operation, or were still active but formed late and never really got going. Some had stopped operating in mid-2017, mostly because people weren’t coming to meetings and the savings and loans portion was having issues in terms of people contributing. Many VNCs were “converted” to APS-VNCs toward the end of the project. Qualitative data collection at endline encountered several instances where the same individual reportedly held multiple project positions as VNC, APS and in charge of GTC communications. Consolidating so many roles into a single individual, particularly if this is the only individual in a village with strong connections to project processes, selection of beneficiaries for sub￾activities, or inputs/oversight into distribution of resources, could contribute to potential concerns on elite capture or inequitable benefits distributions, or simply result in fewer individuals of households, comprising those with stronger connecting to the single key project point of contact in the village, benefiting from project activities. The APS network was recognized to have been a key facilitator for efficient distribution of agricultural inputs, and monitoring progress with clients to ensure that they receive their inputs on time each season. The APS was also noted for planning a key role in gathering product needs and expectations from villagers and passing this information on to GTCs. Many saw the APS role as important for bridging the gap with suppliers at the village level, and providing follow-on technical advice to villagers on the use of new agricultural technologies or inputs they had obtained. But KIIs with several APS suggested that the system is not yet profitable for many such agents to sustain service delivery on their own. One such APS noted that he had narrowed his focus to livestock care, due to insufficient demand for other agricultural products that resulted in a loss. This was particularly noted for seed varieties that the state also provides at a subsidized cost. In KIIs with two APSs, individuals noted they had stopped selling products when Yaajeende ended, such as selling seeds, enriched flour, iodized salt, dried fish and pain de singe, because they did not have the resources to continue doing this after the project closed. APSs noted that maintaining sufficient funds to operate and be able to buy new stocks of inputs or supplies was a challenge, especially because many clients request credit advances and then are late on repayment, or eventually unable to repay it at all. In each of the three regions, APS KIIs noted some YAAJEENDE FINAL IMPACT EVALUATION REPORT ANNEXES 70 instances of collapse due to lack of repayment, or cases where orders were received but the individuals later bought the goods elsewhere prior to delivery by APS. This resulted in surplus inventory for the APS and a lack of cash flow. In such cases, lack of storage or secure storage for surplus inventory was an added challenge. High transportation costs were also noted as a substantial problem that affects APS product pricing. From the perspective of Yaajeende beneficiaries, APSs were seen to provide useful services, trainings and access to inputs, but delays in receipt of orders were also noted (especially in earlier years) and prices were often noted to be high — although the quality of cheaper products was also recognized to be lower. In other cases (especially mentioned in Kédougou), respondents noted that prices were higher compared to local markets, but buying from the APS was easier than going to market and more accessible because the APS could provide inputs on credit. In Matam, some respondents felt that APS prices were substantially lower than market prices, while in other areas prices were noted as too high. On net, the competitiveness of product pricing by APS appears to be fairly variable across locations, but there tended to be general agreement that the quality of product is high, as was the knowledge and technical advice provided by APSs. Yaajeende established Cultivert to professionalize the APS network, improve supply and linkages with finance institutions, and help create financial autonomy for the system. Some KIIs noted that Cultivert appears to have improved APSs’ knowledge of sales and entrepreneurship and improved their activities and work conditions as well as the delivery time of goods. But KIIs in Kédougou and Bakel suggested that many APSs either chose not to join or were unhappy with the transition. In terms of overall governance structures established through the project and perceptions on how they functioned in practice, KIIs and GDs indicated some common themes that likely help to explain some of the variations on outcomes observed through this evaluation. Key points are: • Coordination issues between GTC and local authorities such as mayors appears to be relatively common. The GTC is supposed to interface with the local government, but doing this effectively requires support and willingness on the part of local authorities. GTCs are external to local government, but must be able to work with mayors and commune administrative leadership. Some KIIs reported challenges obtaining financial or resource from local authorities (mayors, primarily), such as for resources to travel or to facilitate meetings. Some KIIs felt that GTCs did not have sufficient financial or material autonomy to succeed in their activities. • At higher levels of government, some KIIs indicated their belief that coordination and collaboration with Yaajeende on activities was not very high. They were accustomed to greater coordination through other donor projects operating in the area (PINKK44 and NEEMA were cited specifically). This was expressed by six KIIs with such authorities, and spanned KIIs in each of the three regions. • Some APS and GTC member also noted lower integration by the project with government services, which they saw as potentially problematic for the sustainability of activities post-project, due to the lower level of buy-in from related authorities during the project lifetime. 44 Projet Intégré de Nutrition dans les régions de Kolda et Kédougou YAAJEENDE FINAL IMPACT EVALUATION REPORT ANNEXES 71 • The project was perceived to have done a good job of training and imparting the necessary knowledge to all project beneficiaries (including VNC, APS and GTC), and in almost all cases, respondents felt the training was of sufficient quality that those trained could train others on what they learned. In terms of continuing activities in practice, a portion of the beneficiaries – at the village level and the VNC, APS and a small number of GTCs — appear to be continuing activities that do not require money or purchase of material goods. Lack of means (money or material) is the main reason that the full intervention package has not been sustained • Of the different roles created through the project, endline qualitative data collection suggests that the GTC may have experienced the most challenges for functionality. Often they do not appear to be well￾supported financially by the mayors, so they are unable to travel or operate as intended. The level of collaboration/cooperation at this level of the system appears fairly low, and there are indications that the GTC has not been well-integrated into the government system (for example, having strong links to mayors and ministries). • A large number of respondents spoke of the need to follow up to sustain activities — either through funding to enable travel to oversee activities, holding group meetings or providing refresher trainings or agricultural extension services. In terms of project implementation challenges, many KII and GD participants voiced two main complaints: (1) livestock that the project provided died in large numbers due to a lack of vaccination or illness, whether or not related to being vaccinated (eight of 33 GDs and nine KIIs); and (2) beneficiaries, in addition to some APSs, VNCs and GTCs, felt that the project had promised them certain activities, inputs or services that they did not receive (eight of 33 GDs and three KIIs). These were more concentrated in Kédougou than other provinces. Less commonly noted issues included late delivery of seed or seedlings, such as during the dry season (three of 33 GDs), and bad quality of seed or old animals (four of 33 GDs). EVALUATION QUESTION 7: UNINTENDED CONSEQUENCES Evaluation Question 7: Were there any unintended broader consequences of the intervention, positive or negative, beyond those related to the activity objectives? This section briefly summarizes any key unintended or unexpected positive or negative broader consequences of the Yaajeende interventions as identified through qualitative data collection. Positive unintended or unexpected consequences include: • General feelings of empowerment through knowledge and capacity building, expressed by several individuals who took on direct roles through the project. There was a feeling that through Yaajeende trainings and skills they received, such individuals have skills that will allow them to continue to find other work and gain income as result. “I was a simple shepherd, I went to school until the third class, but I did not know anything, but with the arrival of Yaajeende, I had a lot of knowledge thanks to the meetings I was doing and trading, we had a lot of skills.” (Bakel GTC member) • Feeling of relationship-building, working together and building greater capacity for self-reliance. Some KIIs also saw this as contributing to post-project sustainability. YAAJEENDE FINAL IMPACT EVALUATION REPORT ANNEXES 72 “Previously everyone was going their own way, but since Yaajeende came, we have meetings, everyone has their idea and we work in the union and in confidence.” (Matam GTC member) “When the Yaajeende project was here, they accompanied the producers by helping them with transportation or other activities. Our worries were: Would we be able to continue these activities after the departure of Yaajeende? But the opposite has happened. It was after the departure of Yaajeende that our results became better because it was a challenge that needed to be addressed.” (Matam GTC member) Negative unintended consequences included: • Perception that the meeting structure and sensitization process was time-consuming and people lose interest to participate. This may also contribute to lower sustainability or continuation of activities after the project has ended. “Since Yaajeende left, me personally, I stopped holding meetings because the women don’t have time available, and the service is gone.” • The project model, typical of many multi-sectoral development programs, of several sub-activities implemented in a given village and relatively small numbers of direct participants per activity, appears to have left some beneficiaries with a perception that project resources and activities were distributed too sparsely among the community to really see a change or make a large difference. “Yaajeende came here many times, they worked, helped some, others did not. They repaired many things and destroyed others. The first time they called us to the hospital for discussions like this. They called a dozen or 20 people to show them how to prepare enriched flour for children and lactating women. They insisted that we had mastered the cooking techniques and they had stopped there. Afterwards, they came back and said they were going to build a garden for us women. We even had to dig a hole under their guidance. Two years ago, they came back to train us in the manufacture of soap to have a revenue generating activity, but we consider this technique too difficult. They came back to give some sheep, only 10 households. The rest has not received anything, but I know that everyone cannot receive it.” (Bakel, women’s GD) • In addition, some respondents described a situation in which they saw a general lack of means to practice activities or behavior changes promoted by the project, or to continue them beyond project lifetime. Examples mentioned included the price of fertilizer, new seed varieties and inputs required for agricultural processing. “They trained us on agricultural techniques, but did not give us the opportunity to practice them. This is the case of fertilizers, it is easy to explain, if we do not have the means to acquire it we will not be able to apply the techniques introduced by the project.” (Matam, men’s GD) “We were brought fertilizer, but it's just a part that has benefited. The rest used cattle waste as fertilizer, as my younger brother had said. Lack of resources sometimes constitutes a brake on access to fertilizer. The priority for some is to feed their families rather than buying fertilizer for a field of 3 or even 4 hectares.” (Matam, men’s GD) • Some agricultural practices promoted by the project were perceived to be too labor- and time-intensive to continue. Zai methods were particularly noted in this respect, as well as making compost using livestock waste, because it requires water, which is often too far away for households to see the effort as realistically feasible. YAAJEENDE FINAL IMPACT EVALUATION REPORT ANNEXES 73 With respect to the zai method promoted, one GTC member noted: “It's hard work. Holes must be dug throughout the field and well before the rainy season. So people can not apply it to large areas and have abandoned it. But people still continue to do all the other activities.” (Matam GTC member) • Water issues: Water is far away from fields and from households, and must be transported by hand or on one’s head in buckets. Water issues also make it difficult to practice some of the agricultural techniques promoted by the project, such as use of zai, and making compost with livestock waste. In addition, water taps are often shared with livestock, and the wait time to get water is long. “There have been changes in this area but we lack the means and the water to do the gardening activities. For agriculture, the lack of seeds and agricultural equipment prevents us from doing our business well.” (Bakel, women’s GD) CONCLUSIONS AND RECOMMENDATIONS The purpose of the Yaajeende final evaluation is to evaluate the program’s impact on malnutrition and poverty reduction in its intervention areas. This final evaluation examines whether and how the full nutrition-led agriculture (NLA) approach impacted poverty and malnutrition after nearly seven years of implementation. The results find little evidence of positive improvements for several of the FIE outcomes as a result of Yaajeende programming, relative to comparison group villages in nearby communes with varying levels of similar integrated WASH, nutrition and agricultural programming. In some cases, outcomes did improve in Yaajeende areas between midline and endline, but households in comparison areas experienced similar or greater levels of improvement on those outcomes during the same period. The net of this trend through the DID analyses with statistical matching is either no additional impact as a result of Yaajeende programming or, for a small number of outcomes, a negative effect from the program relative to the comparison areas. The baseline to endline results do find greater support for positive impacts of Yaajeende programming on agricultural production and revenue. A key overarching issue for the interpretation of the impact results is the wide range of overlapping donor initiatives apparent in the study area, further corroborated through qualitative data at endline. This results in a comparison between Yaajeende and similar donor programming, rather than Yaajeende activities against a comparison case of no activities. However, such levels and types of contamination by other donor programming is not uncommon in Senegal and across low-income countries, and well-known methodological challenges exist in isolating program impacts attributable to a single donor activity under such conditions. This highlights a key impact evaluation issue that the development community grapples with as whole. In many cases, it may be desirable to compare one type of programming to an alternative approach or method. In other cases, where development interventions consist of several integrated components or sub-activities, it may be more desirable to use impact evaluations to provide additional evidence on if and how certain activities work or affect beneficiaries, which kinds of beneficiaries and under which context conditions. When programs are large in scope, operating in several hundred villages, as was the case for Yaajeende, some expectation of varied program effectiveness generally exists. But programs can also take advantage large numbers of implementation villages to design more targeted, smaller￾scale impact studies that aim to specifically learn or fill knowledge gaps on the effectiveness of program subcomponents. In situations with high levels of donor activity across multiple sectors, randomized controlled trial (RCT) designs are better suited to providing strong evidence on impacts attributable to the program of interest. YAAJEENDE FINAL IMPACT EVALUATION REPORT ANNEXES 74 CONCLUSIONS The analyses performed for this FIE estimates impacts for Yaajeende programming for the three-year period from midline to endline (2015 to 2018) and baseline to endline (2011 to 2018). It also provides insights from qualitative and quantitative survey data into drivers of impacts (where present), how impacts vary across different regions of program implementation and for poorest households relative to others, and sheds light on reasons why and other key characteristics of individuals and households that contribute to moderating change on desired outcomes. Evaluation Question 1: What are the impacts of the Yaajeende NLA package on the prevalence of poverty and malnutrition six years after the start of program implementation across four thematic categories (women’s and children’s nutrition; household food security and poverty / economic well-being; household water, sanitation and hygiene practices; and household agricultural practices)? The FIE finds beneficial impacts as a result of the program on two key women’s and children’s nutritional status indicators (reduction in prevalence of women underweight, and a 2.5-8.0 percentage point increase in the prevalence of MAD), a 0.8 to 2.8 percentage point reduction in the likelihood of poverty at the household level, an increase in agricultural investment and an increase in agricultural production. In most cases, the magnitude of these increases are moderate, but that these impacts are typically against an overarching context of general gains on the same (for which Yaajeende programming had additional impacts above the background trends), or within a context of decline, in which Yaajeende programming shows evidence of helping households mitigate overarching negative stresses. Relative to a comparison situation of similar programming efforts on women’s and children’s health, nutrition, WASH and agricultural support, no evidence for added Yaajeende project impacts is found for healthy household practices such as common use of a handwashing station45 and use and proper storage of iodized salt, where Yaajeende and comparison households alike improved on these indicators during the project lifetime, but gains were higher in comparison areas. Although the FIE focuses on midline to endline results due to power limitations and lower reliability of the baseline data, the baseline to endline impact results confirm and follow the same trend on the outcomes for healthy household practices and agricultural practices. They also confirm and find stronger impacts than the midline to endline period for the household economic well-being results. The baseline to endline results do not find evidence of positive Yaajeende effects for any women’s nutritional status and diet outcomes, but the analyses are underpowered to detect a small significant effect if it is present. Many of the FIE findings are consistent with the pattern of outcomes found at midline through the MIE analyses, including a similar set of constraints on wider impacts, as obtained through qualitative data collection. Some evidence indicates that varying household-level exposure to program trainings and activities moderates overall program impacts across Yaajeende villages. For villages with higher average village-wide exposure and 45 With respect to effective approaches toward behavior change on handwashing with soap (and related WASH interventions), 3ie conducted a recent systematic review of the existing evidence base. It highlighted that while community-based approaches and social marketing tend to be effective for promoting behavior change (moving beyond knowledge gain to sustained uptake of promoted behaviors), often a combination of approaches is needed and these should be tailored to specific implementation contexts. The review also found that approaches based on health messaging and health benefits gained from handwashing, although commonly implemented by WASH interventions, are not effective. To increase effectiveness, programs should aim to involve community members in intervention design and implementation, use social marketing to help determine beneficiary needs and preferences and to stimulate demand for desired handwashing options, take consumer preferences into account (including with respect to available options, practical constraints on functionality in a given context, and so on), and work with local builders and entrepreneurs (3ie. 2017. “Handwashing and sanitation behavior change in WASH interventions.” Systematic Review Brief. International Initiative for Impact Evaluation. Washington, D.C). YAAJEENDE FINAL IMPACT EVALUATION REPORT ANNEXES 75 participation to trainings on issues promoted by Yaajeende, a decline of 3 to 6 percentage points occurred in the prevalence of children underweight, as well as a 3 percentage point decline the stunting rate and a stronger reduction in the likelihood of poverty at the household level. In general, intensity of treatment results may suggest that for integrated agriculture, health and nutrition programs, a programming approach that achieves higher saturation of direct participation in multiple different trainings across households in a village may be associated with a higher likelihood of achieving statistically significant change on key women’s and children’s nutrition and diet outcomes. Evaluation Question 2: What major factors or processes contributed to observed impacts, including the role and importance of Yaajeende-supported local institutions? The qualitative data shed some light on potential contributors to overarching impacts. Key drivers of impacts or lack thereof included: focus on a continued lack of resources at the household level to implement or sustain Yaajeende-promoted activities, particularly with respect to inputs needed for effective agricultural production and gardening activities that beneficiaries widely recognized as contributing to a reduction in food insecurity; opening access to a wider range of nutritious foods for households; and potentially leading to income gains if they could be implemented. In some areas, market oversaturation was also viewed as constraint on higher agricultural revenues. With respect to optimal breastfeeding practices, a mother’s insufficient breast milk production and lack of time to comply with optimal feeding practices, given women’s schedules and labor needs, were key factors continuing to stymy wider implementation of these practices, despite knowledge of strong associated health benefits. There were modest gains in reducing household poverty likelihood through the project, and evidence suggests that this relates to increased agricultural production. However, wide variation exists on these effects and evidence suggests increased revenue as a result of production gains or stronger value chain participation are limited. With respect to the village and higher-level structures that the Yaajeende project put in place to help disseminate knowledge and institutionalize behavior change, qualitative data at endline indicated that these institutions and their communications mechanisms were viewed as effective for transmitting knowledge and permitting wide knowledge-sharing within villages. However, respondents in beneficiary villages largely indicated that putting this knowledge into practice, especially with respect to improving diets of women and children, was largely driven by household means, and household lack of means to grow or buy sufficient quantities of nutritious foods is still a key limiting factor for many. With respect to children’s access to enriched foods, the communal system supported by the project for child health screenings and production of such foods, where these systems are maintained, appears beneficial. With respect to evidence on broader contributing roles of Yaajeende-supported local institutions, some evidence, albeit varying, shows that non-standardized processes for GDG creation and rules on membership could have contributed to lower participation rates, or selective membership by certain individuals in villages. Given that GDGs were also primary vehicles by which the project sought to disseminate information and select participants for a range of project activities, such a situation could be a contributing factor to lower overall achievement on some outcomes, if widespread. The APS (in some places APS/VNC) network was recognized as a key facilitator for efficient distribution of agricultural inputs and monitoring progress with clients to ensure that they receive their inputs on time each season. Many saw the APS role as important for bridging the gap with suppliers at the village level, and to provide follow-on technical advice to villagers on using new agricultural technologies or inputs they had obtained. But KIIs with several APSs suggested that the system is not yet profitable for many such agents to sustain service delivery on their own. Of the roles created through the project, endline qualitative data YAAJEENDE FINAL IMPACT EVALUATION REPORT ANNEXES 76 collection suggests that the GTC may have experienced the most challenges for functionality, pointing to coordination issues between GTC and local authorities that may have limited their effectiveness, and insufficient financial and material autonomy to succeed in practice across all intended responsibilities. Evaluation Question 3: How do program impacts differ for key subgroups of interest across key outcomes? The evaluation will assess two subgroups: northern regions (Matam and Bakel) vs. southern region (Kédougou); and poorest households vs. other households. Evaluation Question 4: What are potential explanatory reasons for variations in key outcomes across the subgroups? For a small number of outcomes, there is evidence that program impacts differed by region or household wealth status. In Kédougou Region, the program achieved moderate reductions in stunting and prevalence of children underweight that was not seen in Matam or Bakel. The two northern regions saw a greater decline in poverty likelihood and increased agricultural production. Some differences also existed in impacts for poorest households. Poorer households had stronger gains on agricultural investments and agricultural production. In contrast, the overall program effects observed on women’s and children’s nutritional and diet indicators were driven by gains on these outcomes in less-poor households, despite that poorer households did experience improvements to agricultural production that are expected to help lead to improved nutritional outcomes. Themes from the qualitative data provide few clear differences by region that might enable a stronger understanding of reasons for these regional differences. A positive view on the introduction of community or microgardens as beneficial to improving access to diverse foods was somewhat more represented by GDs in Kédougou relative to the two northern regions. However, only 40 percent of Yaajeende households in Kédougou reported using a home or communal garden, relative to 63.5 percent of Yaajeende households in Matam and Bakel. Kédougou region households report a shorter hunger period than households in the northern region, but the reasons for this are not strongly apparent as they also have a less diverse diet at the household level, while significantly higher agricultural yields and revenue are not observed. On the other hand, participation in value chain activities by Yaajeende households was greater in Kédougou (56.8 percent in Kédougou at endline, and 32.4 percent of Yaajeende households in Matam and Bakel). At the same time, market saturation was more commonly mentioned in Kédougou as reason for lower profitability of income￾generating activities. Yaajeende households in Kédougou reported higher rates of training participation relative to northern region households for agricultural, livestock and WASH trainings, while Yaajeende households in all regions reported similar levels of participation in nutrition and health trainings. Given the higher level of WASH training exposure in Kédougou, along with some Kédougou respondents mentioning strong handwashing practices and corresponding perceived improvements in health through Ebola campaigns in that region, one suggestion is that stronger handwashing behavior in Kédougou in recent years — while not attributable to Yaajeende through this analysis — could be a contributing factor for the reductions in child stunting and underweight observed in that region. In general, the available evidence through this evaluation suggests likely different results pathways at work for achieving impacts in Kédougou relative to the two northern regions of Yaajeende implementation, but the current analysis does not point to strong obvious reasons to explain the regional differences. Evaluation Question 5: How do key individual and household characteristics shape program impacts? YAAJEENDE FINAL IMPACT EVALUATION REPORT ANNEXES 77 Children’s age (measured in days) is associated with higher prevalence of stunting and underweight and lower likelihood of being exclusively breastfed. But increased child’s age is also associated with a higher likelihood that the child receives a minimum acceptable diet. For adult women, age (measured in years) works in the opposite direction and is associated with a lower underweight prevalence. The effect of the household head’s level of education goes in the expected direction, with children in households where the head has at least a primary education approximately 4.5 percent and 6.4 percent less likely to be stunted or underweight, respectively. Larger households appear to exhibit higher household dietary diversity, reduced lean season duration, lower likelihood of poverty and greater revenue from agriculture. Similarly, households where the head has at least an elementary education have HDDS measures that are 0.391 points higher, on average, than households with uneducated household heads; they experience lean seasons that are 0.336 months shorter, are 1.457 percentage points less likely to fall below the poverty line and have agricultural revenue that is FCFA 18,788.074 (approximately USD $33)46 higher, on average. Households with a greater number of members are more likely to have soap-and-water handwashing stations, but there is no evidence of a relationship between household size and iodized salt usage and storage. There is no evidence of an association between the head of household’s education status and either of the household healthy practices outcomes, nor between prevalence of handwashing stations and the age of the head of household. There is a small negative relationship between the head of household’s age and the use and proper storage of iodized salt. Larger households show slightly higher agricultural investment and have greater agricultural production. The household head’s education status has no association with agriculture investment or use of CBSP, though households where the head has at least an elementary education have agriculture production that is 223.9 kg higher per year, on average, than households where the head has no education. Evaluation Question 6: What characteristics of households and mothers appear to be associated with successful poverty and malnutrition reduction for children under age of 5 and women of reproductive age? Across the board, a set of individual and household characteristics consistently showed as important moderators of outcomes, in many cases more so than Yaajeende treatment status. The key factors pointed to through these analyses are overwhelmingly household characteristics that existing literature repeatedly identifies as strong determinants of household poverty and malnutrition status. These are women’s age, household head level of education and household size. 47 For this FIE, these characteristics are all statistically significant in their associations with the outcomes tested across all outcomes families, in the expected direction. Given that households with these characteristics tend to be better off, these results may suggest a role for adaptive programming that differentiates and provides additional or altered programming support to households that face additional challenges to achieve gains on project outcomes, for example on the basis of lower household and/or mother’s education status. 46 USD $1 = FCFA $564.81. 47 Note that the mother’s level of education is also typically found to be strongly associated with reduction in child stunting, although not tested for in these analyses. YAAJEENDE FINAL IMPACT EVALUATION REPORT ANNEXES 78 The FIE finds no evidence of statistically significant treatment effects for participation in mothers’ groups in Yaajeende treatment villages, relative to trends for individuals in Yaajeende village households where no household member participated in a mothers’ group. The exception is a statistically significant increase in the prevalence of stunting, a result that is unexpected and difficult to interpret. Given that women self-select into mothers’ groups, one potential explanation for such a result could be that women in households experiencing greater negative shocks disproportionately joined these groups during the midline to endline period. The FIE also found no evidence for greater effects for participating households in villages where mothers’ group participation is stronger. Evaluation Question 7: Were there any unintended broader consequences of the intervention, positive or negative, beyond those related to the activity objectives? In terms of broader unintended positive consequences of Yaajeende programming, the FIE points to unexpected but synergistic empowerment and capacity building for self-reliance, seen to contribute positively to post-project sustainability on several activities. Respondents mentioned the following key activities: continued sensitization activities in their communities, community meals and preparation of enriched flour. There was also a general feeling of empowerment through knowledge and capacity building, expressed by several individuals who took on direct roles through the project, which they felt would enable them to be able to find new work and gain income after the project had finished. Negative unintended consequences included some perceptions of lost participation interest and information overload related to time-consuming nature of participation in project activities, particularly noted for the number of meetings held and what was seen as a time-consuming sensitization process. RECOMMENDATIONS Include targeted efforts to ease household resource constraints. As the MIE findings also highlighted, resource constraints at the household level continue to serve as a key barrier for low household adoption or sustained uptake of key activities promoted by Yaajeende, including for market gardening, more productive and varied agriculture in general and some income-generating activities. Ongoing lack of means (money or material) also appears to be the main reason that the full intervention package has not always been sustained at project end. There is evidence for progress on many project outcomes over the lifetime of the project through the FIE, despite a lack of strong impact evidence at this stage (i.e., progress has been made over time, although the results from the impact evaluation mean we cannot always attribute this progress to Yaajeende). Given this, it may be that additional dedicated effort to reducing these key barriers for activities, which are not only critical to the program’s theory of change but also occur early in the envisioned causal chain, can serve to springboard households more strongly onto a pathway for sustained access to varied nutritious foods and improved revenue sources early enough in the project’s lifetime to achieve substantial impacts at scale by the project’s end. Future projects should consider strategies by which they can make such inputs more affordable and accessible to smallholders earlier in project time frames, and include targeted support aimed at poorer households (for example, extended time periods for subsidized agro-inputs), for whom such gains are likely to have the strongest short-term boost for realizing food security, nutrition and health benefits through agricultural programming. Consider partnerships and/or strategic planning with water infrastructure programs during program design stages to ensure that program rollout of agricultural and gardening activities takes places in areas with sufficient and reliable water access. Sufficient and reliable water access remains a key constraint for broader agricultural production and market gardening gains throughout the project areas. If water infrastructure improvements are outside the project’s scope, the project might consider dedicated efforts to build partnerships or collaborations with initiatives that focus there. YAAJEENDE FINAL IMPACT EVALUATION REPORT ANNEXES 79 Provide follow-on support to CBSP / APS networks to overcome key barriers to sustained activity and growth. The APS network was widely recognized as instrumental for efficient distribution of agricultural inputs, and monitoring progress with clients to ensure that they receive their inputs on time each season, bridging an important gap with suppliers at the village level and providing needed technical guidance to beneficiaries on the use of new agricultural technologies or inputs they had obtained. But KIIs with several APSs suggested that by the project’s end, the system is not yet profitable for many such agents to sustain service delivery on their own. There are also issues with insufficient or insecure storage for surplus inventory and high transportation costs were also noted as persisting problems that affect APS product pricing and profitability of their activities. KIIs with APSs suggested that many of them are benefiting financially, despite their potential exposure to debt, but there is a view that financial support from the project would have helped them better establish their business prospects before the project’s end, and particularly to enable access to credit, transport and logistical equipment such as trucks. At endline, multiple respondents in each of the three regions noted dissolution of APS relationships. More support of the APS system appears necessary before it can be considered fully functional and sustained on its own. This would appear to be particularly warranted in a follow-on activity, given the substantial investment in establishing and maintaining the system during the Yaajeende project lifetime. Consider more streamlined nested governance structures, and building earlier and stronger linkages to government or other existing structures that are necessary partners for post-project sustainability. Yaajeende focused on establishing locally led governance structures and institutions for knowledge-sharing, activity rollout and service provision. The Yaajeende project created several interconnected institutional structures within village and higher administrative levels to help embed communications systems and coordination for project activities, disseminate information and establish functional platforms for sustained knowledge transfer and service delivery after the project had ended. The approach is seen as beneficial in general, but it is possible that the layered and overlapping nature of responsibilities resulted in an overly complicated institutional structure, with weaker or stronger linkages in different parts of the system, that may be more difficult to sustain in whole without continued project support. Endline qualitative data collection suggests that of the institutions that were established and supported, the GTC may have experienced the most challenges for functionality. The level of collaboration/cooperation at this level of the system appears fairly low, and there are indications that the GTC has not been well integrated into the government system, with weaker linkages to mayors and ministries. Future projects may benefit from considering a more streamlined and hierarchical structure, and earlier and more dedicated efforts to identifying vulnerabilities in the system and building linkages between newly established institutions and the broader government systems they are likely to rely on post-project. Such efforts may also help to identify opportunities for cross-program synergies and leverage opportunities, as well as efficient human and other resource allocation across the multiple actors apparent in the donor-supported MCHN WASH, and agriculture space in country. Consider bifurcated strategies that provide more direct targeting and dedicated support for most vulnerable households, to better expand on impacts for the poorest households. This evaluation found that poorer households had stronger gains on agricultural investment and production, yet they failed to achieve impacts on nutritional status and diet indicators that were observed for the program overall. Current best practice evidence for effective interventions to reduce child malnutrition highlight community-based delivery of supplemental foods for women and children living in households under the poverty line, irrespective of their malnutrition status48. While the Yaajeende project achieved great progress in increasing community knowledge 48 Shekar, Meera, Jakub Kakietek, Julia Dayton Eberwein, and Dylan Walters. 2017. An Investment Framework for Nutrition: Reaching the Global Targets for Stunting, Anemia, Breastfeeding, and Wasting. Directions in Development. Washington, DC: World Bank. YAAJEENDE FINAL IMPACT EVALUATION REPORT ANNEXES 80 on appropriate feeding and diets and links to overall health, and establishing a system by which community members could lead monitoring and identification of malnutrition cases and provide enriched foods, it may be that more dedicated and systematized efforts are needed to ensure that such services are provided on a regular basis for the poorest households. Given evidence of differences in impacts for Kédougou relative to the northern regions covered by this evaluation, future programs may want to consider developing region-specific strategies that take into account strong context differences across implementation zones. This could include conducting assessments early in project timelines to identify how to uniquely leverage major context differences in each zone to target key constraints and focus resources on issues or objectives that may need additional support. In this evaluation, achievements on stunting and prevalence of children underweight in Kédougou appear to have occurred without a corresponding significant gain in agricultural production. This may provide suggestive evidence for different pathways to impacts across these two regions, which is not unrealistic given the agro￾ecological and socio-economic context in Kédougou relative to Matam or Bakel. Consider theory of change and evaluation learning and measuring impact through targeted “mini”- RCT impact evaluation activities. This evaluation also suggests some important learning issues for future evaluation design and implementation for this type of broad integrated program approach across multiple sectors, for which a given village has access to multiple implemented sub-activities, and potentially different direct beneficiaries who may be involved in any one of those activities. Well-designed randomization of who receives different types or timing of program activities can also provide a powerful approach for learning about the effects of subsets of project activities in general, how those effects might vary for different types of beneficiaries and why. Given the large scale of Feed the Future programs in Senegal, operating across several hundred villages and multiple regions, ample opportunity is likely to be available to design and conduct smaller-scale sets of mini-RCTs targeted around specific learning questions for subsets of program interventions or activities across a program’s broad intervention zone. Especially in areas with high levels of similar or overlapping donor activity, RCT approaches are more strongly placed to provide reliable impact estimates and learning on effective program interventions than a matched comparison design, such as the one this evaluation used. RCTs also often require smaller sizes to achieve desired statistical power, but require more upfront work on the design end and work best when they are designed and conducted in close collaboration with program implementers at the start of new programs. YAAJEENDE FINAL IMPACT EVALUATION REPORT ANNEXES 81 ANNEXES ANNEX I – EVALUATION METHODS SUPPLEMENTAL METHODS AND INDICATOR CALCULATIONS Indicator 1.1 is wasting among children aged 6-59 months. Raw data in the survey report children's height, weight, sex, and age. Software published by the World Health Organization, igrowup, calculates the individual child's z-score on the weight for length curve. The definition of wasting in use by USAID is a z-score below -2. The software igrowup flags observations with z-scores below -5 and above 5 as biologically implausible; again using global reference data and not the sample distribution. The WHO software will also estimate the local prevalence of wasting at the same threshold, using a vector of sample weights and excluding observations with edema. Indicator 1.2 is stunting among children aged 6-59 months. Raw data in the survey report children's height, weight, sex, and age. Software published by the World Health Organization, igrowup, calculates the individual child's z-score on the length for age curve. The definition of stunting in use by USAID is a z-score below -2. The software igrowup flags observations with z-scores below -5 and above 5 as biologically implausible; again using global reference data and not the sample distribution. The WHO software will also estimate the local prevalence of stunting at the same threshold, using a vector of sample weights and excluding observations with edema. Indicator 1.3 is underweight among children aged 6-59 months. Raw data in the survey report children's height, weight, sex, and age. Software published by the World Health Organization, igrowup, calculates the individual child's z-score on the weight for age curve. The definition of underweight in use by USAID is a z￾score below -2. The software igrowup flags observations with z-scores below -5 and above 5 as biologically implausible; again using global reference data and not the sample distribution. The WHOsoftware will also estimate the local prevalence of underweight at the same threshold, using a vector of sample weights and excluding observations with edema. Indicator 1.4 is underweight among women aged 15-49. Raw data in the survey report women's height, weight, sex, and age. Underweight is a clinical condition defined by a body mass index (BMI) beneath 18.5. Body mass index is calculated as the ratio of weight (kg) to height squared (m). For example, a woman of 150 cm in height and 54 kg in weight would have a body mass index of 24. The units of BMI (kg/m^2) do not have any intuitive physical meaning, but a higher number indicates heavier weight at any given height. Despite the existence of a mature literature on proposed BMI adjustments to account for sex, age, and body type, this study uses only simple BMI. Indicator 1.5 is minimum acceptable diet (MAD) for children aged 6-23 months. The specific measure of MAD for non-breastfed children 6-23 months of age who had at least the minimum dietary diversity and the minimum meal frequency during the previous day and the proportion of breastfed children 6-23 months of age who received at least two milk feedings and had at least the minimum dietary diversity and the minimum meal frequency during the previous day. Dietary diversity scores are the number of food groups consumed by the child in the previous 24 hours out of the following list: grains, roots and tubers; legumes and nuts; dairy products; flesh foods; eggs Vitamin-A rich fruits and vegetables; other fruits and vegetables. The minimum meal frequency criterion is two (2) meals for breastfed children aged 6-8 months; three (3) meals for breastfed children aged 9-23 months; and four (4) meals for non-breastfed children aged 6-23 months. Indicator 1.6 is optimal maternal breastfeeding practices for children under six months of age. The variable is a binary indicatory for whether or not a child was exclusively breastfed, and is reported for children aged 0-23 months, with a time horizon of birth until six months of age for children 6-23 months old, and from YAAJEENDE FINAL IMPACT EVALUATION REPORT ANNEXES 82 birth until present for children 0-5 months old. It is based on a set of survey questions and requires that each of the following conditions are met: (1) breastfeeding begins within 24 hours of birth; (2) No water, sweet water, or flavored water before 6 months of age; (3) No formula, powdered milk, porridge, or solid food before 6 months of age; (4) Breast fed at least six months or currently. Indicator 1.6a is a revised calculation for Exclusive Breastfeeding EBF. Indicator 1.6 was the calculation used in the midline evaluation. For indicator 1.6a, we remove the condition that breastfeeding begin within 24 hours of birth. Indicator 1.7 is women’s dietary diversity score (WDDS), defined for women ages 15-49 years old. It is calculated from survey questions regarding the food consumed by the woman in the day before the interview. The variable is a discrete index ranging from 0 to 9, and corresponds to the number of food groups consumed. Indicator 2.1 is household dietary diversity score (HDDS). It is a discrete index ranging from 0 to 12. It corresponds to the number of food categories consumed in the past 24 hours, including cereals, roots and tubers, vegetables, fruits, meat, eggs, fish and seafood, lugumes, dairy products, oils and fats, sugar or honey, and other miscellaneous foods. Indicator 2.2 is the duration of reduced food intake (soudure) reported by the household. It is common in rural Senegal for households to reduce food intake for a period of time each year, typically during the rainy season. The variable is calculated from a series of questions in the survey asking in which specific months during the past year the household reduced its food intake, and is measured as months per year. Indicator 2.3 is the likelihood that a household suffers from poverty. It is based on the 2011 Poverty Scorecard, outlined in the document “Simple Poverty Scorecard Poverty-Assessment Tool - Senegal” available at www.simplepovertyscorecard.com/SEN_2011_ENG.pdf. The scorecard calibrates the estimated poverty rate based on eleven questions with simple qualitative responses. The responses are weighted to provide a raw score that takes values between 0 and 100. The raw score can be calibrated to poverty rates at any of a menu of poverty lines: such as the $1.25 World Bank daily income poverty line, the national Senegalese poverty line, and the USAID extreme poverty line. For this study, we use the $1.25 World Bank daily income poverty line. Indicator 2.4 is the total revenue from agriculture (FCFA). It includes both rainy and dry season plantings, and up to three crops planted by the respondent, with the highest surface areas planted first. The list of crops suggested for responses are as follows, with the respondent permitted to replace these with others at his discretion: rice, sorghum, millet, maize, fonio, manioc, yam, tomato, onion, squash, cabbage, cauliflower, lettuce, sweet potato, okra, beans, potato, gourds, groundnuts, sesame, palm (oil), cashew, hibiscus, papaya, melon, watermelon, tobacco, mango, and citrus. Indicator 2.5a is a subjective household well-being indicator, measuring the perceived change in household well-being over the last two years. It ranges from 1 (much worse) to 5 (much better), and is based on an average of 6 separate questions regarding the change in financial situation, ability to manage unforeseen expenses, food availability, agricultural output, per acre yield of most important crops, and agricultural income. Indicator 2.5b is a subjective household well-being indicator, measuring household financial satisfaction at endline. It ranges from 1 (highly unsatisfied) to 5 (highly satisfied), and is based on a single survey question. Indicator 2.5c is a subjective household well-being indicator, measuring the perceived change in household financial satisfaction over the last six years. It ranges from 1 (much worse) to 5 (much better), and is based on a single survey question. YAAJEENDE FINAL IMPACT EVALUATION REPORT ANNEXES 83 Indicator 3.1 is a binary indicator for a handwashing station in common use. It is based on a short battery of questions about handwashing, with visual verification of the handwashing station, soap, and water. Indicator 3.2 is a binary indicator for the use of iodized salt, including both purchase and storage, and verified with a field chemical test. Indicator 4.1 is an index of agriculture technology adoption and agriculture service delivery use. It adds one point for each specific technology or service adopted since 2011. These include tractor use, water pump use, mechanical harvester use, investing in erosion control, investing in protection for crops, compost making, soil water conservation, and SRI. The final five items in the list are counted for men and women separately, so the index ranges from 0 to 11. Indicator 4.2 is a binary indicator for use of a community-based service provide (CBSP) for agriculture. The variable is equal to one if the household used a CBSP to acquire fertilizer, prepare their fields with a tractor, pump water, use mechanical harvester services, invest in erosion control, protect crops, make compost, implement soil water conservation, or SRI. Indicator 4.3 is the total production from agriculture, in kg. It includes both rainy and dry season plantings, and up to three crops planted by the respondent, with the highest surface areas planted first. The list of crops suggested for responses are as follows, with the respondent permitted to replace these with others at his discretion: rice, sorghum, millet, maize, fonio, manioc, yam, tomato, onion, squash, cabbage, cauliflower, lettuce, sweet potato, okra, beans, potato, gourds, groundnuts, sesame, palm (oil), cashew, hibiscus, papaya, melon, watermelon, tobacco, mango, and citrus. Indicator 4.4 is garden access. It is a binary indicator at the household level which describes whether the household has access to and uses a home or communal garden. Indicator 4.5 is value chain participation. It is a binary variable measuring whether or not the household participated in at least one of eight value chain activities, including joint purchase of inputs, bulk sale through farmers’ groups, bulk transport through farmers’ groups, sorting/grading, processing, record keeping, marketing skills, and delayed sales. Indicator 4.5a is a second measure of value chain participation. It is an index ranging from 0 to 8, corresponding to the total number of value chain activities participated in. YAAJEENDE FINAL IMPACT EVALUATION REPORT ANNEXES 84 SUMMARY OF COMPARISON GROUP DONOR-SUPPORTED ACTIVITY Contamination of comparison areas by other donor programming arose as an important constraint on comparability at endline. A new survey module the FIE team added at endline provided strong indication that many households in comparison group villages had been exposed to similar activities conducted by other donors during the midline-endline period. This was further confirmed by qualitative data collection in comparison villages, while qualitative data collection in treatment villages also provided evidence that many Yaajeende villages are also affected by other agriculture and food security, WASH, and/or MCHN programming by other donors during the Yaajeende project lifetime. Given the evidence for alternative donor programming in comparison villages, the comparison case for this evaluation changes from one focused on determining the effects of NLA Yaajeende programming relative to comparable households that received no programming, to measuring the effects of Yaajeende programming relative to households that were exposed to similar types of agricultural, WASH, and/or MCHN donor programming during the same time period. In general, the DID results find little evidence of positive improvements for most of the FIE outcomes as a result of Yaajeende programming, relative to comparison group villages in nearby communes with varying levels of similar integrated WASH, nutrition and agricultural programming. In some cases, outcomes did improve in Yaajeende areas between midline and endline, but households in comparison areas experienced similar or greater levels of improvement on those outcomes during the same time period. The net of this trend through a DID analyses is either no additional impact as a result of Yaajeende programming, or, for a small number of outcomes, a negative effect from the program relative to the comparison areas. The strongest evidence for fairly widespread exposure by comparison group households to other donor￾programming related to agriculture, food security, WASH, and/or MCHN issues comes from the endline household survey instrument, which included a new survey module on household participation in various types of trainings since 2011, across eight broad set of issues that Yaajeende provided trainings or sensitization on. In general, the results on these questions, listed below, suggest that Yaajeende households had received a wider range of trainings, and for each topic category the percent of Yaajeende households that reported participation in such a training was significantly higher than comparison group households that received the same. But, the results also show fairly substantial training exposure for comparison group households, particularly for health and nutrition issues (24 percent of comparison group households relative to 45 percent of Yaajeende households), WASH (32 percent of comparison group households relative to 52 percent of Yaajeende households), and entrepreneurship, business skills, and/or savings and loans trainings (9 percent of comparison households, relative to 16 percent of treatment group households). Endline Survey Round Treatment (Yaajeende Villages) Comparison Group T-C Difference Mean SD Obs Mean SD Obs Endline Diff. Selected program participation indicators: household participation in any… Agricultural trainings during 2011-2017 (I6) 0.270 0.444 1774 0.088 0.283 609 0.183*** Livestock trainings during 2011-2017 (I6) 0.155 0.362 1767 0.048 0.213 608 0.108*** Health and Nutrition trainings during 2011-2017 (I6) 0.451 0.498 1770 0.240 0.427 607 0.211*** WASH trainings during 2011-2017 (I6) 0.517 0.500 1771 0.316 0.465 609 0.202*** Enterpreneurship / Business Skills / Savings and Loans trainings during 2011-2017 (I6) 0.161 0.368 1762 0.088 0.283 608 0.073*** Modern Poultry Farming trainings during 2011-2017 (I6) 0.043 0.203 1768 0.013 0.114 609 0.030*** Agroforestry trainings during 2011-2017 (I6) 0.091 0.287 1770 0.039 0.194 607 0.051*** YAAJEENDE FINAL IMPACT EVALUATION REPORT ANNEXES 85 Food Storage and Processing trainings during 2011-2017 0.216 0.412 1766 0.049 0.215 609 0.167*** Other types of trainings during 2011-2017 (I6) 0.033 0.179 1770 0.028 0.165 609 0.005 The qualitative data provides additional and richer indication of the types of related programming that comparison group villages were exposed to, on the basis of the nine comparison group villages that were visited for endline qualitative data collection. Below, we provide a summary list of overlapping donor activity that was mentioned by respondents in comparison villages. In terms of maternal and child health and nutrition, WASH, or linked agricultural programs, other donor programming activities that were specifically mentioned included: • Sensitization on EBF, MCHN, WASH issues in general • Providing iron or other vitamin supplements to infants and pregnant women • Child vaccinations • Conducting or facilitating child weighing within communities and referrals to health posts based on their nutrition status • Providing supplemental food rations to families with young children or pregnant women were also described in some comparison group areas, primarily mentioned in Matam • Garden establishment and production support • Funding for livestock breeing • Cereal banks construction • Provisioning of fruit trees like mango, guava, and papaya • Distribution of improved seed varieties and other agricultural inputs • Farmer training to help farmers plan harvests and determine the amount of harvest to maintain for home consumption vs sale • Establishment of tontines or other group-lending structures Specific projects (often known to KIIs or village respondents simply by a donor name) that were mentioned in comparison areas included the following, although this likely cannot be considered a comprehensive list: ACTETE, AFRICARE, CARITAS, Counterpart, Croix Rouge, La LUMIERE, NEMA, Oxfam, PINKK, PRODAM, and TOSTAN. YAAJEENDE FINAL IMPACT EVALUATION REPORT ANNEXES 86 ANNEX II – SUPPLEMENTAL DATA Table 1. Summary Statistics and Mean Differences for Yaajeende Treatment Group Sample, EL-BL and EL-ML. Treatment Group Summary Statistics on Outcomes (Yaajeende Villages) Mean / [Obs N] Mean Difference & Significance Level† Outcome Variable Baseline Midline Endline EL - BL EL - ML Women and Children's Nutritional Status and Diet 1.1 Wasting prevalence (children aged 6-59 months) 0.145 0.161 0.110 -0.034* -0.051*** [1059] [3542] [3338] 1.2 Stunting prevalence (children aged 6-59 months) 0.238 0.186 0.215 -0.023 0.029** [1058] [3549] [3481] 1.3 Underweight prevalence (children aged 6-59 mos) 0.231 0.208 0.184 -0.047* -0.024 [1058] [3548] [3338] 1.4 Underweight prevalence (women aged 15-49 years): body mass index (BMI) below 18.5 0.272 0.249 0.192 -0.079*** -0.056*** [988] [4096] [4315] 1.5 Prevalence of children receiving a Minimum acceptable diet (MAD) , children aged 6-23 months 0.132 0.065 0.081 -0.051 0.016 [346] [856] [1134] 1.6 Prevalence of best breastfeeding practices under 6 months of age (recorded for children aged 0-24 months) 0.014 0.240 0.307 0.293*** 0.067* [680] [1550] [1467] 1.6a Exclusively breast-fed, under 6 months of age (recorded for children aged 0-24 months; binary) (Revised Calculation) 0.018 0.288 0.315 0.296*** 0.026 [680] [1550] [1467] Household Food Security and Poverty / Economic Well-being 2.1 Household Dietary Diversity Score 7.062 6.354 6.638 -0.424*** 0.284*** [879] [1820] [1769] 2.2 Soudure (hunger season): Duration of reduced food intake (months per year). 2.522 3.539 3.296 0.774*** -0.243 [881] [1843] [1762] 2.3 Likelihood of poverty at $1.25 2005 PPP threshold (%) 35.380 33.821 25.879 -9.501*** -7.942*** [876] [1843] [1830] 2.4 Total household agriculture revenue (FCFA) 27018 20143 1381349 -13205.851** -6330.303 [881] [1841] [1777] Healthy Household Practices 3.1 Verified soap and water handwashing station in 0.044 0.117 0.07650 0.032* -0.041* common use (binary) [881] [1843] [1830] 3.2 Iodized salt properly obtained and stored 0.182 0.186 0.35351 0.172*** 0.168*** 49 Two reasons for the substantial decline at endline relate to differences in data collection methods at endline: (1) The endline data has a greater percentage of households where production and revenue is 0 for a given season. At endline, the production and revenue questions were only asked to households who reported farming in the relevant season (e.g., rainy season production was only asked to households who reported rainy season farming in a previous question). This logic filter was not used at midline, such that households could have reported a production value and also said they did not farm in that season. (2) Endline collected data on up to the four most important crops, whereas the midline collected data on all crops cultivated. This contributes to higher overall production (and corresponding revenues reported) for all households at midline, compared to endline. 50 A large contributor to this at endline is that when the household was asked, "Does the household have a corner/place designated for handwashing?", only 11% of respondents said yes. Of those that did, about 4% met the soap and water requirements with a functioning tippytap and soap, and about 61% met the soap and water requirements with some other water source. 51 This indicator has two elements: (1) Is salt iodized or not?; and (2) Is it stored properly? At endline, 80% of households had iodized salt, 15% had non￾iodized salt, and 5% had no salt. However, among households with iodized salt, only 42% of them stored it properly. YAAJEENDE FINAL IMPACT EVALUATION REPORT ANNEXES 87 [841] [1745] [1681] Household Agricultural Practices 4.1 Agriculture investment index (0-7 score) N/a 0.298 0.457 0.159** N/a [1632] [1643] 4.2 Household use of CBSP / APS (binary) N/a 0.146 0.156 0.010 N/a [1634] [1643] 4.3 Total household agriculture production (Kg) 1189.188 937.904 108.910 -1080.278*** -828.995*** [881] [1831] [1780] *** p<0.01, ** p<0.05, * p<0.1 † Results are based on a t-test of difference across survey rounds, and indicate whether indicator change within Yaajeende treatment villages during the project lifetime is statistically significant. This analysis does not include a counterfactual and T-test results are not indicative of change in outcomes attributable to Yaajeende programming. YAAJEENDE FINAL IMPACT EVALUATION REPORT ANNEXES 88 Table 2. Outcome Summary Statistics by Survey Round and Treatment Group. Midline Endline Treatment (Yaajeende Villages) Comparison Group Treatment (Yaajeende Villages) Comparison Group T-C Difference Outcome No. Outcome Variable Name Mean SD Obs Mean SD Obs Mean SD Obs Mean SD Obs Endline Diff. Women and Children's Nutritional Status and Diet 1.1 Wasting prevalence (children aged 6-59 months) 0.161 0.368 3542 0.136 0.343 1097 0.110 0.313 3338 0.110 0.314 1000 0.000 Males 0.187 0.390 1740 0.153 0.361 557 0.123 0.329 1673 0.106 0.308 500 0.017 Females 0.137 0.344 1802 0.117 0.322 540 0.097 0.296 1665 0.115 0.319 500 -0.018 Child (24-59 months) 0.156 0.363 2365 0.108 0.311 750 0.119 0.324 2203 0.104 0.305 674 0.015 Toddler (6-23 months) 0.173 0.378 1177 0.195 0.396 347 0.094 0.292 1135 0.124 0.330 326 -0.029 1.2 Stunting prevalence (children aged 6-59 months) 0.186 0.389 3549 0.229 0.420 1099 0.215 0.411 3481 0.251 0.434 1038 -0.037 Males 0.180 0.384 1746 0.224 0.417 557 0.221 0.415 1736 0.288 0.453 519 -0.067** Females 0.191 0.394 1803 0.234 0.424 542 0.208 0.406 1745 0.215 0.411 519 -0.007 Child (24-59 months) 0.189 0.391 2371 0.261 0.439 749 0.224 0.417 2289 0.279 0.449 699 -0.055* Toddler (6-23 months) 0.180 0.384 1178 0.162 0.369 350 0.197 0.398 1192 0.197 0.398 339 0.000 1.3 Underweight prevalence (children aged 6-59 months) 0.208 0.406 3548 0.217 0.413 1096 0.184 0.388 3338 0.205 0.404 1000 -0.021 Males 0.218 0.413 1744 0.218 0.414 556 0.199 0.399 1673 0.215 0.411 500 -0.016 Females 0.199 0.399 1804 0.216 0.412 540 0.169 0.375 1665 0.195 0.397 500 -0.027 1.4 Underweight (women aged 15-49 years): body mass index (BMI) below 18.5 0.249 0.432 4096 0.185 0.388 1023 0.192 0.394 4315 0.158 0.364 1165 0.035 1.5 Minimum acceptable diet (MAD), children 6- 23 months 0.065 0.246 856 0.077 0.267 238 0.081 0.273 1134 0.081 0.273 325 0.000 Males 0.073 0.260 415 0.064 0.245 118 0.081 0.273 553 0.087 0.282 160 -0.005 Females 0.057 0.232 441 0.091 0.288 120 0.081 0.273 581 0.075 0.265 165 0.005 1.6 Prevalence of best breastfeeding practices under 6 months of age (recorded for children aged 0-24 months; binary) 0.240 0.427 1550 0.209 0.407 508 0.307 0.461 1467 0.373 0.484 432 -0.066 Males 0.260 0.439 755 0.212 0.410 258 0.319 0.466 720 0.333 0.472 219 -0.014 Females 0.221 0.415 795 0.206 0.405 250 0.295 0.456 747 0.413 0.494 213 -0.118** 1.6a Exclusively breast-fed, under 6 months of age (recorded for cildren aged 0-24 months; binary) (Revised Calculation) 0.288 0.453 1550 0.272 0.445 508 0.315 0.464 1467 0.404 0.491 432 -0.089* Males 0.313 0.464 755 0.292 0.456 258 0.325 0.469 720 0.369 0.484 219 -0.044 Females 0.264 0.441 795 0.251 0.435 250 0.304 0.460 747 0.439 0.497 213 -0.134** YAAJEENDE FINAL IMPACT EVALUATION REPORT ANNEXES 89 Midline Endline Treatment (Yaajeende Villages) Comparison Group Treatment (Yaajeende Villages) Comparison Group T-C Difference Outcome No. Outcome Variable Name Mean SD Obs Mean SD Obs Mean SD Obs Mean SD Obs Endline Diff. 1.7 Women's dietary diversity score (0-9 range) . . 0 . . 0 3.986 1.556 1545 3.190 1.469 528 0.796*** Household Food Security and Poverty / Economic Well-being 2.1 Household Dietary Diversity Score - Past 24 hrs 6.354 2.246 1820 4.934 2.030 638 6.638 1.849 1769 5.755 1.683 601 0.883*** 2.2 Soudure (hunger season): Duration of reduced food intake (months per year). 3.539 2.303 1843 3.690 1.871 646 3.296 2.156 1762 3.334 1.806 599 -0.038 2.3 Likelihood of poverty at the $1.25 2005 PPP threshold (%) 33.821 10.652 1843 38.145 4.140 646 25.879 10.509 1830 32.916 5.674 631 -7.037*** 2.4 Total household agriculture revenue (FCFA) 201433 80776 1841 22741 73290 646 13813 93893 1777 5543 46568 609 8269.310* 2.5a Household Subjective Financial Satisfaction in 2018 (1=Highly unsatisfied,5=Highly satisfied ) . . 0 . . 0 2.142 0.745 1780 1.990 0.747 608 0.152** 2.5b Mean Change in Household Wellbeing, Last 2 years (1=Much worse,5=Much better) . . 0 . . 0 2.675 0.834 1537 2.697 0.776 557 -0.023 2.5c Mean Change in Subjective Financial Satisfaction, Last 6 years (1=Much worse,5=Much better) . . 0 . . 0 3.098 1.000 1779 3.056 1.013 609 0.042 Healthy Household Practices 3.1 Verified soap and water handwashing station in common use (binary) 0.117 0.322 1843 0.066 0.249 646 0.486 0.501 266 0.819 0.389 57 -0.026 3.2 Iodized salt properly obtained and stored 0.186 0.389 1745 0.106 0.308 610 0.353 0.478 1681 0.292 0.455 567 0.061 Household Agricultural Practices 4.1 Agricultural investment index (0-7 score) 0.298 0.656 1632 0.220 0.579 603 0.457 0.911 1643 0.194 0.534 577 0.264*** 4.2 Household uses of CBSP / APS (binary) 0.146 0.353 1634 0.053 0.223 603 0.156 0.363 1643 0.033 0.178 577 0.124*** 4.3 Total household agriculture production (Kg) 937.9 1366.2 1831 871.1 1182.4 644 108.9 899.7 1780 20.8 111.5 609 88.089** 4.4 Garden Access: Household use of home or communal garden (binary) . . 0 . . 0 0.559 0.497 1754 0.308 0.462 601 0.251*** 4.5 Value chain participation: Household participation in any value chain activity . . 0 . . 0 0.408 0.492 1643 0.392 0.489 577 0.016 4.5a Number of value chain activities that household participated in . . 0 . . 0 1.111 1.920 1643 1.209 2.476 577 -0.098 Selected program participation indicators: household participation in any… Agricultural trainings during 2011-2017 (I6) . . 0 . . 0 0.270 0.444 1774 0.088 0.283 609 0.183*** Livestock trainings during 2011-2017 (I6) . . 0 . . 0 0.155 0.362 1767 0.048 0.213 608 0.108*** Health and Nutrition trainings during 2011- 2017 (I6) . . 0 . . 0 0.451 0.498 1770 0.240 0.427 607 0.211*** WASH trainings during 2011-2017 (I6) . . 0 . . 0 0.517 0.500 1771 0.316 0.465 609 0.202*** YAAJEENDE FINAL IMPACT EVALUATION REPORT ANNEXES 90 Midline Endline Treatment (Yaajeende Villages) Comparison Group Treatment (Yaajeende Villages) Comparison Group T-C Difference Outcome No. Outcome Variable Name Mean SD Obs Mean SD Obs Mean SD Obs Mean SD Obs Endline Diff. Enterpreneurship / Business Skills / Savings and Loans trainings during 2011-2017 (I6) . . 0 . . 0 0.161 0.368 1762 0.088 0.283 608 0.073*** Modern Poultry Farming trainings during 2011-2017 (I6) . . 0 . . 0 0.043 0.203 1768 0.013 0.114 609 0.030*** Agroforestry trainings during 2011-2017 (I6) . . 0 . . 0 0.091 0.287 1770 0.039 0.194 607 0.051*** Food Storage and Processing trainings during 2011-2017 . . 0 . . 0 0.216 0.412 1766 0.049 0.215 609 0.167*** Other types of trainings during 2011-2017 (I6) . . 0 . . 0 0.033 0.179 1770 0.028 0.165 609 0.005 YAAJEENDE FINAL IMPACT EVALUATION REPORT ANNEXES 91 Table 3. Outcome Summary Statistics by Survey Round and Treatment Group, Region and Wealth Status Sub-groups. Midline Endline Treatment (Yaajeende) Comparison Group Treatment (Yaajeende) Comparison Group Outcome Variable Name Mean SD Obs Mean SD Obs Mean SD Obs Mean SD Obs Women and Children's Nutritional Status and Diet 1.1 Wasting prevalence (children aged 6-59 months) Southern Region (Kedougou) 0.123 0.329 946 0.121 0.327 298 0.072 0.258 939 0.099 0.300 258 Northern Region (Matam & Tambacounda) 0.175 0.380 2596 0.142 0.349 799 0.124 0.329 2399 0.115 0.319 742 Less Poor (Poverty Score > Endline Median) 0.194 0.396 828 0.000 0.000 7 0.115 0.319 2337 0.083 0.276 369 Poorer (Poverty Score <= Endline Median) 0.152 0.359 2714 0.136 0.343 1090 0.101 0.301 1001 0.128 0.335 631 1.2 Stunting prevalence (children aged 6-59 months) Southern Region (Kedougou) 0.301 0.459 948 0.201 0.401 301 0.281 0.450 974 0.248 0.433 272 Northern Region (Matam & Tambacounda) 0.145 0.353 2601 0.241 0.428 798 0.192 0.394 2507 0.253 0.435 766 Less Poor (Poverty Score > Endline Median) 0.144 0.352 828 0.295 0.492 7 0.191 0.393 2451 0.247 0.432 388 Poorer (Poverty Score <= Endline Median) 0.198 0.398 2721 0.229 0.420 1092 0.270 0.444 1030 0.254 0.436 650 1.3 Underweight prevalence (children aged 6-59 months) Southern Region (Kedougou) 0.238 0.426 948 0.194 0.396 298 0.157 0.364 939 0.180 0.385 258 Northern Region (Matam & Tambacounda) 0.198 0.398 2600 0.227 0.419 798 0.194 0.395 2399 0.216 0.412 742 Less Poor (Poverty Score > Endline Median) 0.214 0.410 828 0.158 0.394 7 0.175 0.380 2337 0.166 0.373 369 Poorer (Poverty Score <= Endline Median) 0.207 0.405 2720 0.218 0.413 1089 0.206 0.405 1001 0.231 0.422 631 1.4 Underweight (women aged 15-49 years): body mass index (BMI) below 18.5 Southern Region (Kedougou) 0.172 0.378 968 0.097 0.296 318 0.122 0.327 1101 0.097 0.296 350 Northern Region (Matam & Tambacounda) 0.271 0.445 3128 0.231 0.422 705 0.213 0.410 3214 0.189 0.392 815 Less Poor (Poverty Score > Endline Median) 0.217 0.413 1201 0.000 0.000 6 0.183 0.387 3246 0.146 0.354 468 Poorer (Poverty Score <= Endline Median) 0.262 0.440 2895 0.185 0.389 1017 0.220 0.414 1069 0.166 0.372 697 1.5 Minimum acceptable diet (MAD) , children aged 6-23 months Southern Region (Kedougou) 0.092 0.290 220 0.155 0.365 60 0.082 0.275 327 0.158 0.367 74 Northern Region (Matam & Tambacounda) 0.056 0.230 636 0.046 0.211 178 0.080 0.272 807 0.050 0.218 251 Less Poor (Poverty Score > Endline Median) 0.114 0.318 186 0.000 0.000 2 0.101 0.302 792 0.094 0.294 120 Poorer (Poverty Score <= Endline Median) 0.050 0.217 670 0.077 0.268 236 0.031 0.175 342 0.072 0.259 205 1.6 Prevalence of best breastfeeding practices under 6 months of age (recorded for children aged 0-24 months, binary) Southern Region (Kedougou) 0.212 0.409 399 0.131 0.338 146 0.400 0.490 438 0.484 0.502 106 YAAJEENDE FINAL IMPACT EVALUATION REPORT ANNEXES 92 Midline Endline Treatment (Yaajeende) Comparison Group Treatment (Yaajeende) Comparison Group Outcome Variable Name Mean SD Obs Mean SD Obs Mean SD Obs Mean SD Obs Northern Region (Matam & Tambacounda) 0.249 0.433 1151 0.246 0.431 362 0.272 0.445 1029 0.328 0.470 326 Less Poor (Poverty Score > Endline Median) 0.256 0.437 352 0.366 0.590 3 0.301 0.459 1025 0.321 0.468 163 Poorer (Poverty Score <= Endline Median) 0.235 0.424 1198 0.208 0.406 505 0.321 0.467 442 0.406 0.492 269 1.6a Exclusively breast-fed, under 6 months of age (recorded for cildren aged 0-24 months; binary) (Revised Calculation) Southern Region (Kedougou) 0.326 0.469 399 0.220 0.416 146 0.412 0.493 438 0.508 0.502 106 Northern Region (Matam & Tambacounda) 0.276 0.447 1151 0.296 0.457 362 0.279 0.449 1029 0.362 0.481 326 Less Poor (Poverty Score > Endline Median) 0.292 0.455 352 0.366 0.590 3 0.310 0.463 1025 0.352 0.479 163 Poorer (Poverty Score <= Endline Median) 0.287 0.453 1198 0.271 0.445 505 0.326 0.469 442 0.437 0.497 269 1.7 Women's dietary diversity score Southern Region (Kedougou) . . 0 . . 0 3.566 1.419 550 3.599 1.377 194 Northern Region (Matam & Tambacounda) . . 0 . . 0 4.204 1.579 995 2.889 1.464 334 Less Poor (Endline Poverty Score > Endline Median) . . 0 . . 0 4.309 1.553 1033 3.573 1.468 195 Poorer (Endline Poverty Score <= Endline Median) . . 0 . . 0 3.385 1.371 512 2.944 1.418 333 Household Food Security and Poverty / Economic Well-being 2.1 Household Dietary Diversity Score Southern Region (Kedougou) 4.791 2.259 647 4.547 2.304 239 5.556 1.794 609 5.673 1.669 216 Northern Region (Matam & Tambacounda) 7.155 1.766 1173 5.231 1.737 399 7.163 1.636 1160 5.815 1.694 385 Less Poor (Endline Poverty Score > Endline Median) 8.046 1.720 366 4.136 0.971 5 7.128 1.752 1187 6.110 1.631 216 Poorer (Endline Poverty Score <= Endline Median) 5.925 2.159 1454 4.939 2.034 633 5.697 1.657 582 5.540 1.680 385 2.2 Soudure (hunger season): Duration of reduced food intake (months per year). Southern Region (Kedougou) 2.900 1.673 656 3.226 1.552 244 2.478 1.334 609 2.806 1.183 215 Northern Region (Matam & Tambacounda) 3.868 2.506 1187 4.049 2.015 402 3.693 2.358 1153 3.720 2.067 384 Less Poor (Endline Poverty Score > Endline Median) 3.058 2.164 369 4.621 1.749 5 3.308 2.308 1181 3.104 1.692 216 Poorer (Endline Poverty Score <= Endline Median) 3.661 2.322 1474 3.684 1.871 641 3.274 1.830 581 3.474 1.860 383 2.3 Likelihood of poverty at the $1.25 2005 PPP threshold (%) Southern Region (Kedougou) 37.472 6.473 656 38.638 3.604 244 30.640 7.482 648 31.756 4.465 237 Northern Region (Matam & Tambacounda) 31.947 11.828 1187 37.763 4.479 402 23.471 10.989 1182 33.803 6.311 394 Less Poor (Endline Poverty Score > Endline Median) 15.540 8.429 369 23.980 5.624 5 20.820 9.737 1195 28.171 5.410 219 Poorer (Endline Poverty Score <= Endline Median) 38.434 4.307 1474 38.228 3.989 641 34.922 3.291 635 35.670 3.642 412 2.4 Total household agriculture revenue (FCFA) YAAJEENDE FINAL IMPACT EVALUATION REPORT ANNEXES 93 Midline Endline Treatment (Yaajeende) Comparison Group Treatment (Yaajeende) Comparison Group Outcome Variable Name Mean SD Obs Mean SD Obs Mean SD Obs Mean SD Obs Southern Region (Kedougou) 27112.032 88544.577 655 41679.867 1.02e+05 244 16197.424 87120.745 617 11627.589 70617.601 220 Northern Region (Matam & Tambacounda) 16573.035 76282.939 1186 8092.421 31592.836 402 12642.838 97054.706 1160 1053.664 7224.863 389 Less Poor (Endline Poverty Score > Endline Median) 20068.536 67938.291 367 0.000 0.000 5 18260.320 1.12e+05 1190 12994.823 74463.725 219 Poorer (Endline Poverty Score <= Endline Median) 20161.423 83703.531 1474 22873.922 73484.077 641 5282.325 39966.675 587 1004.425 8215.917 390 2.5a Household Subjective Financial Satisfaction in 2018 (1=Highly unsatisfied,5=Highly satisfied ) Southern Region (Kedougou) . . 0 . . 0 2.289 0.780 617 2.015 0.773 220 Northern Region (Matam & Tambacounda) . . 0 . . 0 2.071 0.717 1163 1.971 0.728 388 Less Poor (Endline Poverty Score > Endline Median) . . 0 . . 0 2.165 0.715 1192 2.114 0.773 218 Poorer (Endline Poverty Score <= Endline Median) . . 0 . . 0 2.099 0.799 588 1.915 0.722 390 2.5b Mean Change in Household Wellbeing, Last 2 years (1=Much worse,5=Much better) Southern Region (Kedougou) . . 0 . . 0 3.122 0.777 543 2.861 0.722 212 Northern Region (Matam & Tambacounda) . . 0 . . 0 2.458 0.772 994 2.565 0.794 345 Less Poor (Endline Poverty Score > Endline Median) . . 0 . . 0 2.662 0.832 1022 2.888 0.833 205 Poorer (Endline Poverty Score <= Endline Median) . . 0 . . 0 2.699 0.837 515 2.574 0.711 352 2.5c Mean Change in Subjective Financial Satisfaction, Last 6 years (1=Much worse,5=Much better) Southern Region (Kedougou) . . 0 . . 0 3.301 0.963 616 3.314 0.925 220 Northern Region (Matam & Tambacounda) . . 0 . . 0 2.999 1.003 1163 2.866 1.034 389 Less Poor (Endline Poverty Score > Endline Median) . . 0 . . 0 3.122 1.013 1191 3.193 1.036 219 Poorer (Endline Poverty Score <= Endline Median) . . 0 . . 0 3.051 0.974 588 2.973 0.990 390 Healthy Household Practices 3.1 Verified soap and water handwashing station in common use (binary) Southern Region (Kedougou) 0.223 0.417 656 0.143 0.351 244 0.084 0.278 648 0.212 0.409 237 Northern Region (Matam & Tambacounda) 0.063 0.243 1187 0.007 0.084 402 0.072 0.259 1182 0.018 0.134 394 Less Poor (Endline Poverty Score > Endline Median) 0.140 0.347 369 0.000 0.000 5 0.092 0.290 1195 0.092 0.290 219 Poorer (Endline Poverty Score <= Endline Median) 0.111 0.315 1474 0.067 0.250 641 0.048 0.213 635 0.108 0.311 412 3.2 Iodized salt properly obtained and stored Southern Region (Kedougou) 0.192 0.394 622 0.132 0.339 230 0.392 0.489 586 0.413 0.494 207 Northern Region (Matam & Tambacounda) 0.183 0.386 1123 0.086 0.281 380 0.334 0.472 1095 0.201 0.401 360 Less Poor (Endline Poverty Score > Endline Median) 0.244 0.430 353 0.000 0.000 5 0.355 0.479 1123 0.287 0.453 208 YAAJEENDE FINAL IMPACT EVALUATION REPORT ANNEXES 94 Midline Endline Treatment (Yaajeende) Comparison Group Treatment (Yaajeende) Comparison Group Outcome Variable Name Mean SD Obs Mean SD Obs Mean SD Obs Mean SD Obs Poorer (Endline Poverty Score <= Endline Median) 0.171 0.376 1392 0.107 0.309 605 0.350 0.477 558 0.295 0.457 359 Household Agricultural Practices 4.1 Agriculture investement index (0-7 score) Southern Region (Kedougou) 0.326 0.746 627 0.291 0.682 243 0.261 0.659 601 0.224 0.467 216 Northern Region (Matam & Tambacounda) 0.282 0.599 1005 0.159 0.466 360 0.560 1.003 1042 0.170 0.580 361 Less Poor (Endline Poverty Score > Endline Median) 0.332 0.621 305 0.000 0.000 5 0.497 0.946 1082 0.198 0.543 207 Poorer (Endline Poverty Score <= Endline Median) 0.290 0.664 1327 0.221 0.580 598 0.383 0.836 561 0.191 0.530 370 4.2 Household use of CBSP / APS (binary) Southern Region (Kedougou) 0.066 0.248 627 0.044 0.206 243 0.062 0.241 601 0.041 0.199 216 Northern Region (Matam & Tambacounda) 0.191 0.393 1007 0.060 0.237 360 0.206 0.405 1042 0.026 0.160 361 Less Poor (Endline Poverty Score > Endline Median) 0.225 0.418 305 0.000 0.000 5 0.181 0.386 1082 0.030 0.170 207 Poorer (Endline Poverty Score <= Endline Median) 0.126 0.332 1329 0.053 0.224 598 0.109 0.312 561 0.035 0.183 370 4.3 Total household agriculture production (Kg) Southern Region (Kedougou) 1225.978 1675.022 644 980.979 1099.829 242 67.076 460.219 618 28.730 154.080 220 Northern Region (Matam & Tambacounda) 792.487 1153.697 1187 786.410 1237.216 402 129.404 1049.506 1162 14.985 63.879 389 Less Poor (Endline Poverty Score > Endline Median) 967.434 1293.484 368 366.097 386.149 5 144.568 1072.323 1192 32.316 161.403 219 Poorer (Endline Poverty Score <= Endline Median) 930.446 1384.248 1463 874.056 1184.954 639 40.440 386.729 588 13.819 63.832 390 4.4 Garden Access: Household use of home or communal garden (binary) Southern Region (Kedougou) . . 0 . . 0 0.403 0.491 608 0.366 0.483 216 Northern Region (Matam & Tambacounda) . . 0 . . 0 0.635 0.482 1146 0.265 0.442 385 Less Poor (Endline Poverty Score > Endline Median) . . 0 . . 0 0.611 0.488 1177 0.343 0.476 216 Poorer (Endline Poverty Score <= Endline Median) . . 0 . . 0 0.459 0.499 577 0.286 0.453 385 4.5 Value chain participation: Household participation in any value chain activity Southern Region (Kedougou) . . 0 . . 0 0.568 0.496 601 0.500 0.501 216 Northern Region (Matam & Tambacounda) . . 0 . . 0 0.324 0.468 1042 0.309 0.463 361 Less Poor (Endline Poverty Score > Endline Median) . . 0 . . 0 0.363 0.481 1082 0.429 0.496 207 Poorer (Endline Poverty Score <= Endline Median) . . 0 . . 0 0.492 0.500 561 0.369 0.483 370 4.5a Number of value chain activities that household participated in Southern Region (Kedougou) . . 0 . . 0 2.026 2.570 601 1.900 3.350 216 YAAJEENDE FINAL IMPACT EVALUATION REPORT ANNEXES 95 Midline Endline Treatment (Yaajeende) Comparison Group Treatment (Yaajeende) Comparison Group Outcome Variable Name Mean SD Obs Mean SD Obs Mean SD Obs Mean SD Obs Northern Region (Matam & Tambacounda) . . 0 . . 0 0.631 1.219 1042 0.676 1.254 361 Less Poor (Endline Poverty Score > Endline Median) . . 0 . . 0 0.940 1.790 1082 1.250 2.001 207 Poorer (Endline Poverty Score <= Endline Median) . . 0 . . 0 1.433 2.107 561 1.184 2.730 370 Selected program participation indicators: household participation in any… Household participation in any Agricultural trainings during 2011-2017 (I6) Southern Region (Kedougou) . . 0 . . 0 0.423 0.494 616 0.137 0.345 220 Northern Region (Matam & Tambacounda) . . 0 . . 0 0.196 0.397 1158 0.051 0.221 389 Less Poor (Endline Poverty Score > Endline Median) . . 0 . . 0 0.289 0.453 1187 0.136 0.343 219 Poorer (Endline Poverty Score <= Endline Median) . . 0 . . 0 0.235 0.424 587 0.059 0.235 390 Household participation in any Livestock trainings during 2011-2017 (I6) Southern Region (Kedougou) . . 0 . . 0 0.224 0.417 612 0.044 0.205 220 Northern Region (Matam & Tambacounda) . . 0 . . 0 0.122 0.328 1155 0.050 0.219 388 Less Poor (Endline Poverty Score > Endline Median) . . 0 . . 0 0.157 0.364 1182 0.049 0.217 218 Poorer (Endline Poverty Score <= Endline Median) . . 0 . . 0 0.153 0.360 585 0.047 0.211 390 Household participation in any Health and Nutrition trainings during 2011-2017 (I6) Southern Region (Kedougou) . . 0 . . 0 0.534 0.499 614 0.347 0.477 220 Northern Region (Matam & Tambacounda) . . 0 . . 0 0.411 0.492 1156 0.160 0.367 387 Less Poor (Endline Poverty Score > Endline Median) . . 0 . . 0 0.458 0.498 1184 0.292 0.456 218 Poorer (Endline Poverty Score <= Endline Median) . . 0 . . 0 0.438 0.497 586 0.209 0.407 389 Household participation in any WASH trainings during 2011-2017 (I6) Southern Region (Kedougou) . . 0 . . 0 0.651 0.477 616 0.454 0.499 220 Northern Region (Matam & Tambacounda) . . 0 . . 0 0.452 0.498 1155 0.213 0.410 389 Less Poor (Endline Poverty Score > Endline Median) . . 0 . . 0 0.526 0.500 1184 0.366 0.483 219 Poorer (Endline Poverty Score <= Endline Median) . . 0 . . 0 0.501 0.500 587 0.285 0.452 390 Household participation in any Enterpreneurship / Business Skills / Savings and Loans trainings during 2011-2017 (I6) Southern Region (Kedougou) . . 0 . . 0 0.200 0.401 613 0.111 0.315 220 Northern Region (Matam & Tambacounda) . . 0 . . 0 0.142 0.349 1149 0.070 0.256 388 Less Poor (Endline Poverty Score > Endline Median) . . 0 . . 0 0.175 0.380 1180 0.118 0.324 218 Poorer (Endline Poverty Score <= Endline Median) . . 0 . . 0 0.135 0.342 582 0.069 0.254 390 YAAJEENDE FINAL IMPACT EVALUATION REPORT ANNEXES 96 Midline Endline Treatment (Yaajeende) Comparison Group Treatment (Yaajeende) Comparison Group Outcome Variable Name Mean SD Obs Mean SD Obs Mean SD Obs Mean SD Obs Household participation in any Modern Poultry Farming trainings during 2011-2017 (I6) Southern Region (Kedougou) . . 0 . . 0 0.074 0.262 615 0.010 0.098 220 Northern Region (Matam & Tambacounda) . . 0 . . 0 0.028 0.165 1153 0.016 0.124 389 Less Poor (Endline Poverty Score > Endline Median) . . 0 . . 0 0.040 0.195 1182 0.025 0.156 219 Poorer (Endline Poverty Score <= Endline Median) . . 0 . . 0 0.050 0.217 586 0.006 0.077 390 Household participation in any Agroforestry trainings during 2011-2017 (I6) Southern Region (Kedougou) . . 0 . . 0 0.150 0.357 616 0.076 0.265 219 Northern Region (Matam & Tambacounda) . . 0 . . 0 0.062 0.241 1154 0.012 0.111 388 Less Poor (Endline Poverty Score > Endline Median) . . 0 . . 0 0.095 0.294 1183 0.048 0.215 219 Poorer (Endline Poverty Score <= Endline Median) . . 0 . . 0 0.082 0.274 587 0.034 0.180 388 Household participation in any Food Storage and Processing trainings during 2011-2017 (I6) Southern Region (Kedougou) . . 0 . . 0 0.229 0.420 616 0.101 0.302 220 Northern Region (Matam & Tambacounda) . . 0 . . 0 0.210 0.407 1150 0.010 0.099 389 Less Poor (Endline Poverty Score > Endline Median) . . 0 . . 0 0.262 0.440 1181 0.077 0.267 219 Poorer (Endline Poverty Score <= Endline Median) . . 0 . . 0 0.127 0.334 585 0.031 0.174 390 Household participation in any Other types of trainings during 2011-2017 (I6) Southern Region (Kedougou) . . 0 . . 0 0.032 0.176 617 0.038 0.193 220 Northern Region (Matam & Tambacounda) . . 0 . . 0 0.034 0.180 1153 0.020 0.140 389 Less Poor (Endline Poverty Score > Endline Median) . . 0 . . 0 0.036 0.187 1183 0.048 0.213 219 Poorer (Endline Poverty Score <= Endline Median) . . 0 . . 0 0.027 0.162 587 0.016 0.125 390 YAAJEENDE FINAL IMPACT EVALUATION REPORT ANNEXES 97 Table 4. Components of Minimum Acceptable Diet (MAD) by Survey Round and Treatment Group. Minimum Acceptable Diet (MAD) components by survey round and treatment group Overall ML Comparison Group ML Treatment (Yaajeende Villages) EL Comparison Group EL Treatment (Yaajeende Villages) Breastfed (children aged 6-8 months) Number of Children 648 72 276 68 232 Minimum Meal Frequency 43.5% 27.6% 43.8% 53.7% 44.4% Minimum Dietary Diversity 4.3% 3.1% 4.8% 0.0% 4.9% Cereals, Roots, and Tubers 49.7% 40.3% 56.4% 53.1% 43.9% Legumes and Nuts 12.2% 9.6% 13.3% 10.7% 11.9% Milk and Dairy 25.8% 10.8% 18.7% 22.0% 36.6% Meat, Poultry, and Fish 7.2% 4.7% 13.0% 0.0% 3.1% Eggs 1.7% 4.5% 2.5% 0.0% 0.7% Vitimin A-Rich Fruits and Vegetables 14.5% 3.4% 18.0% 3.4% 15.2% Other Fruits and Vegetables 3.1% 3.4% 3.1% 1.0% 3.3% Breastfed (children aged 9-23 months) Number of Children 1623 200 557 208 658 Minimum Meal Frequency 44.7% 35.1% 42.1% 61.7% 45.1% Minimum Dietary Diversity 12.1% 8.8% 9.2% 13.4% 15.0% Cereals, Roots, and Tubers 78.0% 81.6% 75.5% 73.3% 80.2% Legumes and Nuts 31.5% 27.5% 32.8% 46.1% 28.0% Milk and Dairy 32.8% 16.3% 21.1% 34.2% 44.5% Meat, Poultry, and Fish 12.5% 13.3% 13.3% 8.2% 12.7% Eggs 2.6% 1.2% 3.4% 2.3% 2.4% Vitimin A-Rich Fruits and Vegetables 26.4% 14.6% 18.2% 29.6% 34.1% Other Fruits and Vegetables 6.8% 4.9% 9.7% 7.2% 4.7% Not Breastfed (children aged 6-23 months) Number of Children 654 59 274 51 270 Minimum Meal Frequency 36.2% 26.5% 32.0% 37.0% 41.4% Minimum Dietary Diversity 20.1% 6.6% 23.9% 5.5% 19.7% Cereals, Roots, and Tubers 88.8% 85.2% 90.6% 92.6% 87.2% Legumes and Nuts 37.7% 54.5% 41.3% 36.4% 32.9% Milk and Dairy 48.6% 14.5% 41.3% 29.0% 60.7% Meat, Poultry, and Fish 26.3% 7.5% 32.1% 16.3% 24.4% Eggs 6.8% 10.4% 11.7% 0.0% 2.9% Vitimin A-Rich Fruits and Vegetables 32.7% 9.3% 32.2% 30.2% 36.0% Other Fruits and Vegetables 9.0% 3.8% 12.6% 0.0% 7.3% * Note: The small sample sizes within these age sub-groups may contribute to unreliability of the results shown. YAAJEENDE FINAL IMPACT EVALUATION REPORT ANNEXES 98 Figure 1. Minimum Acceptable Diet (MAD) Components at Midline, by Treatment Group 0 20 40 60 80 Percent Cereals, Roots, and Tubers Legumes and Nuts Milk and dairy Meat, Poultry, and Fish Eggs Fruits and Vegetables (Vitamin A-Rich) Other Fruits And Veg. Min. Meal Frequency Min. Dietary Diversity MAD Components Midline Control Sample Breastfed (6-8 months) Breastfed (9-23 months) Not Breastfed (6-23 months) 0 20 40 60 80 100 Percent Cereals, Roots, and Tubers Legumes and Nuts Milk and dairy Meat, Poultry, and Fish Eggs Fruits and Vegetables (Vitamin A-Rich) Other Fruits And Veg. Min. Meal Frequency Min. Dietary Diversity MAD Components Midline Treatment Sample Breastfed (6-8 months) Breastfed (9-23 months) Not Breastfed (6-23 months) YAAJEENDE FINAL IMPACT EVALUATION REPORT ANNEXES 99 Figure 2. Minimum Acceptable Diet (MAD) Components at Endline, by Treatment Group 0 20 40 60 80 100 Percent Cereals, Roots, and Tubers Legumes and Nuts Milk and dairy Meat, Poultry, and Fish Eggs Fruits and Vegetables (Vitamin A-Rich) Other Fruits And Veg. Min. Meal Frequency Min. Dietary Diversity MAD Components Endline Control Sample Breastfed (6-8 months) Breastfed (9-23 months) Not Breastfed (6-23 months) 0 20 40 60 80 Percent Cereals, Roots, and Tubers Legumes and Nuts Milk and dairy Meat, Poultry, and Fish Eggs Fruits and Vegetables (Vitamin A-Rich) Other Fruits And Veg. Min. Meal Frequency Min. Dietary Diversity MAD Components Endline Treatment Sample Breastfed (6-8 months) Breastfed (9-23 months) Not Breastfed (6-23 months) YAAJEENDE FINAL IMPACT EVALUATION REPORT ANNEXES 1 0 0 Figure 3. Household Financial Satisfaction at Endline, and Perceived Change since 2011. 0 20 40 60 80 100 Percent Treatment (Yaajeende Villages) Comparison Group Satisfaction with Current Financial Situation Fully Satisfied Rather Satisfied Less Than Satisfied Not At All Satisfied 0 20 40 60 80 100 Percent Treatment (Yaajeende Villages) Comparison Group Change In Financial Situation Over Past 6 Years Improved a Lot Somewhat Improved Remained the Same Somewhat Deteriorated Deteriorated A Lot YAAJEENDE FINAL IMPACT EVALUATION REPORT ANNEXES 1 0 1 Figure 4. Household Perceived Change in Subjective Well-being over Past Two Years, at Endline. 0 20 40 60 80 100 Percent Agricultural Income Crop Yield Agricultural Output Hunger Season Food Availibility Management of Unforeseen Expenses Financial Situation Treatment Comparison Treatment Comparison Treatment Comparison Treatment Comparison Treatment Comparison Treatment Comparison Household Well-being: Change in Past 2 Years Improved a Lot Improved Somewhat About the Same Somewhat Worse A Lot Worse YAAJEENDE FINAL IMPACT EVALUATION REPORT ANNEXES 1 0 2 Figure 5. Household and Neighbors’ Perceived Financial Well-being. Note: The figure shows the results to a series of questions asking respondents to imagine a ten-step ladder, where the first step represents the poorest people and the tenth step represents the richest. The questions ask the respondent where they and their neighbors are located currently, and where they were located in 2011. The graph can be read in two ways. First, it shows the percentage of respondents on each step. Looking at “Your Household Today”, 20% of respondents in the comparison group said they were on the poorest step, compared to about 18% in the treatment group. About 25% of respondents in the comparison group said they were on the second poorest step, compared to about 22% in the treatment group. Second, it shows the percentage of respondents falling into a range of steps. Looking at “Your Neighbors Today”, we can see that about 19% of households in the treatment group said their neighbors were on one of the five highest steps (gray or black-shaded), compared to about 10% of households in the comparison group. 0 20 40 60 80 100 Percent Your Neighbors in 2011 Your Household in 2011 Your Neighbors Today Your Household Today Treatment Comparison Treatment Comparison Treatment Comparison Treatment Comparison Financial Stepladder (Scale of 1-10) 1 (Poorest) 10 (Richest) YAAJEENDE FINAL IMPACT EVALUATION REPORT ANNEXES 1 0 3 Table 5. Poverty Status Sub-Group Analysis for Outcome Family 1: Village Fixed Effects DID Results, Individual-Level Outcomes, ML-EL. 1.1 1.2 1.3 1.4 1.5 1.6 1.6a VARIABLES Wasting: z-score below -2 on reference weight￾for-length curve. Stunting: z-score below -2 on reference length￾for-age curve. Underweight: z￾score below -2 on reference weight￾for-age curve. Underweight: body mass index (BMI) below 18.5. Minimum acceptable diet (MAD) for children ages 6-23 months Exclusively breast-fed (binary) Exclusively breast-fed (Revised) Poor Differential Effect 0.170*** 0.192*** 0.368*** 0.201*** 0.080 0.231 0.195 (Poor*Treat*Endline) (0.051) (0.060) (0.054) (0.046) (0.069) (0.203) (0.188) Yaajeende Treatment Effect Treatment -0.208*** -0.188*** -0.411*** -0.226*** -0.042 - 0.430*** -0.425*** (0.039) (0.054) (0.030) (0.029) (0.054) (0.083) (0.080) Poor*Endline -0.137*** -0.165*** -0.337*** -0.222*** -0.102** -0.130 -0.125 (0.038) (0.051) (0.045) (0.040) (0.048) (0.189) (0.172) Treatment*Poor -0.199*** -0.169*** -0.456*** -0.130*** -0.167*** - 0.299*** -0.325*** (0.031) (0.032) (0.027) (0.025) (0.047) (0.085) (0.088) Gender = Female -0.014 -0.012 -0.021 -0.004 -0.017 -0.033 (0.013) (0.024) (0.019) (0.015) (0.024) (0.022) ln(Age) -0.034*** 0.046*** 0.013 -0.225*** 0.042** - 0.090*** -0.090*** (0.012) (0.017) (0.011) (0.023) (0.017) (0.022) (0.021) Household Head Has At Least Elementary Education -0.005 -0.054*** -0.065*** -0.009 0.046 -0.039 -0.080** (0.014) (0.014) (0.025) (0.034) (0.048) (0.035) (0.036) Poor 0.162*** 0.173*** 0.417*** 0.178*** 0.117*** 0.236*** 0.261*** (0.015) (0.018) (0.016) (0.012) (0.016) (0.062) (0.064) Endline 0.118*** 0.206*** 0.350*** 0.206*** 0.048 0.379*** 0.335*** (0.027) (0.047) (0.023) (0.023) (0.030) (0.069) (0.066) Observations 8,442 8,631 8,447 10,526 2,428 3,399 3,399 Diff. Effect 95% CI [ 0.070; 0.271] [ 0.075; 0.309] [ 0.263; 0.473] [ 0.112; 0.291] [ -0.054; 0.215] [ -0.166; 0.628] [ -0.174; 0.563] Effect Size 0.567 0.447 0.926 0.547 0.329 0.480 0.401 Treatment Effect in -0.037 0.004 -0.043 -0.025 0.038 -0.199 -0.230 Poorer Households (0.036) (0.033) (0.040) (0.031) (0.038) (0.158) (0.156) *** p<0.01, ** p<0.05, * p<0.1 Note: Robust standard errors in parentheses clustered at the village level. Effect Size is for the triple interaction term, calculated as the estimated coefficient divided by the standard deviation of the outcome at endline. Treatment effect in poorer households calculated by adding the differential effect and the Yaajeende treatment effect. Age is measured in days for children and years for women. YAAJEENDE FINAL IMPACT EVALUATION REPORT ANNEXES 1 0 4 Table 6. Poverty Status Sub-Group Analysis for Outcome Families 2-4: Village Fixed Effects DID Results, Household-Level Outcomes, ML-EL. 2.1 2.2 2.3 2.4 3.1 3.2 4.1 4.2 4.3 VARIABLES Household Dietary Diversity Score - Past 24 hrs Soudure: Duration of reduced food intake (months per year). Likelihood of poverty at the $1.25 2005 PPP threshold (%) Total household agriculture revenue Verified soap and water handwashing station (binary) Iodized salt properly obtained and stored Agriculture investement index Household uses CBSP / APS (binary) Total household agriculture production Poor Differential Effect 2.051*** -1.922*** -2.827 -5,255.336 -0.219* -0.093 0.225* 0.048 530.282** (Poor*Treat*Endline) (0.456) (0.570) (2.169) (7,978.074) (0.116) (0.097) (0.128) (0.045) (212.633) Yaajeende Treat. Effect Treatment -3.130*** 2.459*** 2.412 12,084.094 0.060 -0.022 0.001 -0.015 -290.979 (0.243) (0.572) (1.836) (9,433.735) (0.042) (0.061) (0.114) (0.047) (184.519) Poor*Endline -1.491*** 1.339*** -4.189** -7,463.994* 0.180 0.159** -0.263** -0.020 -610.480*** (0.419) (0.511) (2.075) (3,984.584) (0.112) (0.075) (0.105) (0.019) (155.971) ln(Household Size) 0.407*** -0.156* -1.070*** 10,297.785*** 0.036** 0.017 0.066 0.006 350.461*** (0.089) (0.081) (0.233) (2,725.534) (0.015) (0.015) (0.046) (0.012) (48.138) ln(Head Age) -0.032 -0.186 0.169 -5,005.031 0.000 -0.092* 0.061 0.003 52.151 (0.166) (0.159) (0.722) (8,708.742) (0.023) (0.048) (0.076) (0.025) (144.238) Head Education Level: At Least Elementary Education = 1 0.340** -0.300*** -0.420 18,913.449** 0.019 -0.019 0.018 -0.010 226.011** (0.145) (0.087) (0.573) (8,812.532) (0.050) (0.021) (0.049) (0.016) (98.742) poor = Yes 1.481*** -1.364*** 11.611*** 13,217.590*** -0.021 -0.035 0.313*** 0.020* 765.230*** (0.188) (0.487) (1.455) (4,744.974) (0.032) (0.028) (0.050) (0.012) (115.027) Endline 2.618*** -2.127*** 0.493 -12,525.494** -0.077** 0.122*** 0.171** -0.016 -546.866*** (0.198) (0.516) (1.731) (6,248.101) (0.031) (0.034) (0.080) (0.015) (102.600) Observations 4,791 4,804 4,903 4,827 4,903 4,568 4,422 4,424 4,818 95% CI [ 1.157; 2.944] [ -3.039; - 0.805] [ -7.079; 1.424] [-2.09e+04; 10381.402] [ -0.446; 0.008] [ -0.283; 0.096] [ -0.027; 0.476] [ -0.040; 0.135] [ 113.529; Effect Size 1.145 -1.010 -0.312 -0.076 -0.657 -0.195 0.288 0.167 947.035] 0.828 Treatment Effect in -1.079** 0.537* -0.415 6828.758 -0.159* -0.116 0.226** 0.033 239.302 Poorer Households (0.426) (0.322) (0.802) (9439.687) (0.093) (0.073) (0.111) (0.037) (226.635) *** p<0.01, ** p<0.05, * p<0.1 Note: Robust standard errors in parentheses clustered at the village level. Effect Size is for the triple interaction term, calculated as the estimated coefficient divided by the standard deviation of the outcome at endline. Treatment effect in poorer households calculated by adding the differential effect and the Yaajeende treatment effect. YAAJEENDE FINAL IMPACT EVALUATION REPORT ANNEXES 1 0 5 Table 7. Regional Sub-group Analysis for Outcome Family 1: Village Fixed Effects DID Results, Individual-Level Outcomes, ML-EL. 1.1 1.2 1.3 1.4 1.5 1.6 1.6a VARIABLES Wasting: z￾score below - 2 on reference weight-for￾length curve. Stunting: z￾score below -2 on reference length-for￾age curve. Underweight: z-score below - 2 on reference weight-for-age curve. Underweight: body mass index (BMI) below 18.5. Minimum acceptable diet (MAD) for children ages 6-23 months Exclusively breast-fed (binary) Exclusively breast-fed (Revised) Kedougou Differential Effect 0.010 -0.166** -0.135*** 0.071* 0.010 -0.077 -0.187 (Treatment*Endline*Kedougou) (0.066) (0.068) (0.047) (0.041) (0.065) (0.217) (0.219) Yaajeende Treatment Effect Treatment -0.036 0.052*** 0.038 -0.086*** 0.064 -0.115 -0.075 (0.032) (0.019) (0.035) (0.026) (0.048) (0.126) (0.121) Treatment*Kedougou 0.001 0.123** 0.082** -0.074** -0.057 0.271 0.299 (0.058) (0.055) (0.035) (0.037) (0.056) (0.202) (0.199) Gender = Female -0.014 -0.011 -0.019 -0.003 -0.019 -0.035 (0.013) (0.024) (0.018) (0.015) (0.024) (0.022) ln(Age) -0.034*** 0.045*** 0.013 -0.225*** 0.045*** -0.092*** -0.092*** (0.012) (0.017) (0.011) (0.024) (0.016) (0.019) (0.018) Household Head Has At Least Elementary Education -0.004 -0.054*** -0.062*** -0.011 0.051 -0.040 -0.078** (0.015) (0.012) (0.023) (0.035) (0.045) (0.040) (0.037) Endline -0.030 -0.016 -0.057* 0.040* -0.030 0.070 0.005 (0.028) (0.014) (0.031) (0.022) (0.044) (0.118) (0.114) Observations 8,442 8,631 8,447 10,526 2,428 3,399 3,399 95% CI [ -0.119; 0.139] [ -0.299; - 0.032] [ -0.227; - 0.043] [ -0.009; 0.151] [ -0.118; 0.138] [ -0.503; 0.348] [ -0.617; 0.243] Effect Size 0.033 -0.386 -0.340 0.193 0.042 -0.161 -0.385 Treatment Effect -0.026 -0.114* -0.097*** -0.015 0.075* -0.192 -0.262 in Kedougou (0.058) (0.066) (0.031) (0.031) (0.044) (0.175) (0.182) *** p<0.01, ** p<0.05, * p<0.1 Note: Robust standard errors in parentheses clustered at the village level. Effect Size is for the triple interaction term, calculated as the estimated coefficient divided by the standard deviation of the outcome at endline. Treatment effect in Kedougou calculated by adding the differential effect and the Yaajeende treatment effect. Age is measured in days for children and years for women. YAAJEENDE FINAL IMPACT EVALUATION REPORT ANNEXES 1 0 6 Table 8. Regional Sub-group Analysis for Outcome Families 2-4: Village Fixed Effects DID Results, Household-Level Outcomes, ML-EL. 2.1 2.2 2.3 2.4 3.1 3.2 4.1 4.2 4.3 VARIABLES Household Dietary Diversity Score - Past 24 hrs Soudure: Duration of reduced food intake (months per year). Likelihood of poverty at the $1.25 2005 PPP threshold (%) Total household agriculture revenue Verified soap and water handwashing station (binary) Iodized salt properly obtained and stored Agriculture investement index Household uses CBSP / APS (binary) Total household agriculture production Kedougou Dif. Effect -0.081 -0.680 3.153** 31,971.487 -0.251** -0.034 -0.089 0.011 -514.747 (Kedougou*Treat.*End.) (0.587) (0.556) (1.517) (22,326.911) (0.114) (0.111) (0.213) (0.070) (439.775) Yaajeende Effect -0.782** 0.814** -2.089** -1,637.990 0.029 -0.037 0.260 0.031 524.196** (0.341) (0.410) (0.995) (7,837.149) (0.028) (0.083) (0.164) (0.060) (255.435) Kedougou *Endline 0.811 0.513 -1.386 -40,435.842** 0.099 0.076 -0.236 -0.021 -18.340 (0.559) (0.457) (1.283) (19,182.853) (0.106) (0.088) (0.170) (0.047) (416.910) ln(Household Size) 0.443*** -0.202** -2.512*** 11,042.869*** 0.030** 0.016 0.071 0.009 351.649*** (0.090) (0.080) (0.428) (2,631.694) (0.012) (0.016) (0.048) (0.012) (48.064) ln(Head Age) -0.029 -0.193 0.370 -4,792.078 0.005 -0.088* 0.063 0.003 57.286 (0.166) (0.159) (0.835) (8,651.489) (0.021) (0.046) (0.075) (0.025) (145.513) Head Education Level: At Least Elementary Education = 1 0.388** -0.335*** -1.470 18,728.754** 0.016 -0.020 0.016 -0.008 226.945** (0.150) (0.095) (0.902) (8,667.590) (0.054) (0.023) (0.046) (0.016) (98.154) Endline 0.746** -1.026*** -6.218*** -3,150.289** -0.017 0.187** 0.003 -0.025 -1,215.652*** (0.325) (0.332) (0.727) (1,354.204) (0.022) (0.075) (0.130) (0.039) (236.026) Observations 4,791 4,804 4,903 4,827 4,903 4,568 4,422 4,424 4,818 95% CI [ -1.231; 1.069] [ -1.771; 0.411] [ 0.180; 6.126] [-1.18e+04; 75731.429] [ -0.475; - 0.028] [ -0.251; 0.183] [ -0.506; 0.328] [ -0.126; 0.149] [-1376.691; Effect Size -0.045 -0.357 0.348 0.460 -0.753 -0.071 -0.114 0.039 347.197] -0.804 Treatment Effect -0.863* 0.134 1.063 30333.498 -0.222** -0.071 0.171 0.043 9.449 in Kedougou (0.478) (0.375) (1.133) (20913.433) (0.111) (0.073) (0.135) (0.036) (357.862) *** p<0.01, ** p<0.05, * p<0.1 Note: Robust standard errors in parentheses clustered at the village level. Effect Size is for the triple interaction term, calculated as the estimated coefficient divided by the standard deviation of the outcome at endline. Treatment effect in Kedougou calculated by adding the differential effect and the Yaajeende treatment effect. YAAJEENDE FINAL IMPACT EVALUATION REPORT ANNEXES 1 0 7 Table 9. Baseline – Endline Impact Analysis for Outcome Family 1: Village Fixed Effects DID Results, Individual-Level Outcomes. 1.1 1.2 1.3 1.4 1.5 1.6 1.6a VARIABLES Wasting: z￾score below - 2 on reference weight-for￾length curve. Stunting: z￾score below -2 on reference length-for￾age curve. Underweight: z-score below - 2 on reference weight-for-age curve. Underweight: body mass index (BMI) below 18.5. Minimum acceptable diet (MAD) for children ages 6-23 months Exclusively breast-fed (binary) Exclusively breast-fed (Revised) Yaajeende Treatment -0.028 0.015 0.011 -0.021 0.039 -0.081 -0.105* Effect (0.027) (0.045) (0.036) (0.036) (0.074) (0.063) (0.063) Gender = Female -0.037*** -0.007 -0.025** -0.007 -0.001 -0.001 (0.013) (0.013) (0.011) (0.024) (0.024) (0.025) ln(Age) 0.003 0.048*** 0.048*** -0.215*** 0.056*** -0.118*** -0.122*** (0.009) (0.013) (0.010) (0.023) (0.021) (0.012) (0.012) Household Head Has At Least Elementary -0.038*** -0.060** -0.088*** -0.011 0.054 -0.020 -0.023 Education (0.014) (0.025) (0.023) (0.025) (0.033) (0.048) (0.048) Endline -0.001 -0.046 -0.061** -0.053** -0.099** 0.345*** 0.366*** (0.021) (0.039) (0.029) (0.025) (0.050) (0.051) (0.051) Observations 5,655 5,833 5,653 6,722 1,860 2,441 2,441 Treatment N 4296 4437 4295 5289 1440 1875 1875 Control N 1359 1396 1358 1433 420 566 566 Treatment Effect 95% CI [ -0.080; 0.024] [ -0.074; 0.103] [ -0.060; 0.082] [ -0.091; 0.049] [ -0.105; 0.183] [ -0.204; 0.043] [ -0.228; 0.019] Treatment Effect Size -0.089 0.035 0.028 -0.054 0.143 -0.173 -0.222 *** p<0.01, ** p<0.05, * p<0.1 Note: Robust standard errors in parentheses clustered at the village level. Effect Size calculated as the estimated coefficient divided by the standard deviation of the outcome at endline. Age is measured in days for children and years for women. YAAJEENDE FINAL IMPACT EVALUATION REPORT ANNEXES 1 0 8 Table 10. Baseline – Endline Impact Analysis for Outcome Families 2-4: Village Fixed Effects DID Results, Household-Level Outcomes. 2.1 2.2 2.3 2.4 3.1 3.2 4.1 4.2 4.3 VARIABLES Household Dietary Diversity Score - Past 24 hrs Soudure: Duration of reduced food intake (months per year). Likelihood of poverty at the $1.25 2005 PPP threshold (%) Total household agriculture revenue Verified soap and water handwashing station (binary) Iodized salt properly obtained and stored Agriculture investement index Household uses CBSP / APS (binary) Total household agriculture production Yaajeende Effect -0.033 -0.041 -5.012*** 37,745.783** -0.044 0.018 631.094*** (0.221) (0.236) (1.122) (14,790.917) (0.039) (0.056) (215.449) ln(Household Size) 0.457*** -0.216*** -2.816*** 18,079.920** 0.025** -0.009 0.086 0.033* 393.752*** (0.079) (0.067) (0.553) (7,009.523) (0.010) (0.018) (0.057) (0.017) (86.281) ln(Head Age) 0.150 -0.207 0.332 4,740.168 -0.017 -0.025 0.017 -0.013 205.111** (0.127) (0.141) (0.704) (9,273.419) (0.014) (0.032) (0.073) (0.033) (89.120) Head Education Level: At Least Elementary Education = 1 0.402*** -0.202 -1.772** 15,260.127* -0.028** -0.017 -0.077 -0.048 138.897 (0.111) (0.125) (0.855) (9,052.151) (0.014) (0.025) (0.060) (0.042) (88.441) Endline -0.498*** 0.858*** -4.100*** -55,352.276*** 0.077** 0.142*** -1,792.542*** (0.181) (0.182) (0.700) (14,454.406) (0.036) (0.047) (203.716) Observations 3,640 3,633 3,725 3,658 3,732 3,456 2,210 2,210 3,661 95% CI 2631 2626 2689 2641 2694 2509 1635 1635 2644 Effect Size 1009 1007 1036 1017 1038 947 575 575 1017 Treatment Effect [ -0.466; 0.400] [ -0.504; 0.421] [ -7.212; - 2.812] [ 8756.118; 66735.448] [ -0.120; 0.033] [ -0.092; 0.127] [ 0.000; 0.000] [ 0.000; 0.000] [ 208.823; 1053.366] in Kedougou -0.018 -0.020 -0.495 0.436 -0.159 0.038 0.000 0.000 0.783 *** p<0.01, ** p<0.05, * p<0.1 Note: Robust standard errors in parentheses clustered at the village level. Effect Size calculated as the estimated coefficient divided by the standard deviation of the outcome at endline. YAAJEENDE FINAL IMPACT EVALUATION REPORT ANNEXES 1 0 9 SUPPLEMENTAL ANALYSES OF DIFFERENTIAL EFFECTS BASED ON PARTICIPATION IN A MOTHER’S GROUPS Respondents to the endline survey in treatment villages were asked about their participation in Yaajeende-affiliated mothers groups (known as Debbo Gallé, or “Excellent Mothers” groups). These groups facilitated more intensive and comprehensive collaboration with the Yaajeende activities’ core target populations (i.e., women of reproductive age and children under 5 years old). Therefore, it may be likely that those who participated in these groups experienced greater improvements in outcomes from midline to endline than those who did not participate. To test whether households with GDG participants experienced improvements in key outcomes, we restrict the sample to households in Yaajeende treatment villages only, and run a DID model which replaces the Yaajeende treatment variable with a dummy variable indicating whether or not anyone in the household participated in a mothers group. The regressions are run with entropy weighting. The resulting estimate gives the effect on the outcome of mothers group participation in the household, relative to trends among non-GDG households in Yaajeende villages. Table 11 shows the results for the impact on individual-level outcomes. In general, we find no evidence of statistically significant treatment effects for participation in mothers’ groups in treatment villages, relative to trends for individuals in Yaajeende village households where no one participated in a mothers group. The exception is a statistically significant increase in the prevalence of stunting, a result that is unexpected and difficult to interpret. Since participation in these groups was voluntary, one possibility is that individuals that faced some sort of negative household-level shock between midline and endline self￾selected into participation. Under such a scenario, participation would be correlated with negative household- and individual-level trends, which would give a biased impact of the effect of participation. In any case, the results overall fail to detect any impact for group participation. Table 11. Mothers Group Participation Regression Analyses for Outcome Family 1: Village Fixed Effects DID Results, Individual-Level Outcomes, ML-EL. 1.1 1.2 1.3 1.4 1.5 1.6 1.6a VARIABLES Wasting: z￾score below - 2 on reference weight-for￾length curve. Stunting: z￾score below -2 on reference length-for￾age curve. Underweight: z-score below - 2 on reference weight-for-age curve. Underweight: body mass index (BMI) below 18.5. Minimum acceptable diet (MAD) for children ages 6-23 months Exclusively breast-fed (binary) Exclusively breast-fed (Revised) Mothers Group 0.033 0.051** 0.013 -0.017 0.037 0.032 0.028 Treatment Effect (0.022) (0.021) (0.028) (0.019) (0.038) (0.055) (0.054) Mothers Group -0.027 -0.012 -0.003 0.012 0.007 -0.007 -0.006 Participant (0.026) (0.021) (0.019) (0.021) (0.028) (0.048) (0.048) Gender = Female -0.031** 0.002 -0.022 -0.007 -0.052* -0.058** (0.012) (0.015) (0.016) (0.022) (0.027) (0.028) ln(Age) 0.005 0.029*** 0.019 -0.260*** 0.036* -0.106*** -0.107*** (0.008) (0.010) (0.012) (0.026) (0.021) (0.018) (0.018) Household Head Has At Least Elementary -0.011 -0.028** -0.058** -0.012 0.013 -0.022 -0.026 Education (0.019) (0.012) (0.022) (0.032) (0.025) (0.056) (0.060) Endline -0.082*** 0.003 -0.042* -0.040*** 0.006 -0.018 -0.055 (0.017) (0.018) (0.023) (0.014) (0.026) (0.042) (0.042) Observations 6,309 6,458 6,315 8,097 1,850 2,538 2,538 YAAJEENDE FINAL IMPACT EVALUATION REPORT ANNEXES 1 1 0 Participant N 2960 3029 2964 3771 877 1212 1212 Non-Participant N 3349 3429 3351 4326 973 1326 1326 Treatment Effect 95% CI [ -0.011; 0.077] [ 0.010; 0.093] [ -0.041; 0.068] [ -0.054; 0.021] [ -0.039; 0.112] [ -0.076; 0.140] [ -0.079; 0.134] Treatment Effect Size 0.106 0.125 0.034 -0.042 0.133 0.069 0.060 *** p<0.01, ** p<0.05, * p<0.1 Note: Robust standard errors in parentheses clustered at the village level. Effect Size calculated as the estimated coefficient divided by the standard deviation of the outcome at endline. Age is measured in days for children and years for women. The Mother’s Group Treatment Effect is the interaction between having a mother’s group participant in the household and endline (i.e., the DID estimator). Table 12 shows the results for the impact on two household-level outcomes most related to areas targeted in mothers’ groups: household dietary diversity and likelihood of poverty. Having someone in the household participating in the mothers’ groups in Yaajeende villages is significantly correlated with a modest improvement in household dietary diversity, predicting a 0.406 increase in the HDDS scale, relative to trends in non-participating households in treatment villages. Similarly, participation is correlated with a reduction of 1.162 percentage points in the likelihood of poverty, a result that is marginally significant (p=0.11) Table 12. Outcome Families 2 Village Fixed Effects DID Results: Household-Level Outcomes, ML-EL. Mothers Group Participation Regressions. 2.1 2.3 VARIABLES Household Dietary Diversity Score - Past 24 hrs Likelihood of poverty at the $1.25 2005 PPP threshold (%) Mothers Group 0.406** -1.162 Treatment Effect (0.167) (0.722) Mothers Group -0.015 0.168 Participant (0.108) (0.564) ln(Household Size) 0.439*** -3.072*** (0.088) (0.397) ln(Head Age) 0.073 0.003 (0.168) (0.999) Head Education Level: At Least Elementary Education = 1 0.598*** -3.130** (0.131) (1.228) Endline -0.061 -7.370*** (0.118) (0.503) Observations 3,409 3,419 Participant N 1530 1534 Non-Participant N 1879 1885 Treatment Effect 95% CI [ 0.077; 0.734] [ -2.576; 0.252] Treatment Effect Size 0.222 -0.109 *** p<0.01, ** p<0.05, * p<0.1 Note: Robust standard errors in parentheses clustered at the village level. Effect Size calculated as the estimated coefficient divided by the standard deviation of the outcome at endline. We additionally ask whether participation in womens groups is more effective in villages where there was greater participation. This could be the case, because participating in a strong mothers group is likely to be qualitatively different than participating in a group in a village where participation is weak. We construct a village-level intensity variable, defined as the percentage of households in the sample who indicated they were participating in mothers’ groups in each village at endline. Again using only YAAJEENDE FINAL IMPACT EVALUATION REPORT ANNEXES 1 1 1 households in Yaajeende treatment villages, we run a regression analogous to the village intensity regressions described in the main text of the paper, this time interacting mothers group intensity with household-level participation in mothers groups. Table 13 shows the results for the impact on individual-level outcomes. In general, the results show no evidence for greater effects for participating households in villages where mothers group participation is stronger. Similarly, we find no evidence of statistically significant treatment effects for participation in mothers groups in treatment villages, relative to trends for individuals in Yaajeende village households where no one participated in a mothers group. Table 13. Outcome Family 1 Village Fixed Effects DID Results: Individual-Level Outcomes, ML-EL. Mothers Group Participation Village Intensity Regressions. 1.1 1.2 1.3 1.4 1.5 1.6 1.6a VARIABLES Wasting: z￾score below - 2 on reference weight-for￾length curve. Stunting: z￾score below -2 on reference length-for￾age curve. Underweight: z-score below - 2 on reference weight-for-age curve. Underweight: body mass index (BMI) below 18.5. Minimum acceptable diet (MAD) for children ages 6-23 months Exclusively breast-fed (binary) Exclusively breast-fed (Revised) Intensity Dif. Effect -0.073 -0.108 -0.234 -0.179 0.082 0.215 0.097 (Partic.*Intensity*Endline) (0.141) (0.116) (0.143) (0.113) (0.176) (0.341) (0.336) Mothers Group 0.079 0.105* 0.137* 0.061 -0.002 -0.087 -0.036 Treatment Effect (0.077) (0.060) (0.074) (0.053) (0.098) (0.177) (0.171) Endline*Intensity -0.025 0.057 0.074 0.168** -0.064 -0.037 0.048 (0.087) (0.096) (0.118) (0.068) (0.096) (0.193) (0.196) Participant*Intensity 0.043 0.097 0.205* 0.161 0.014 -0.289 -0.123 (0.113) (0.120) (0.122) (0.116) (0.151) (0.302) (0.310) Mothers Group -0.053 -0.060 -0.109* -0.063 -0.002 0.144 0.064 Participant (0.056) (0.063) (0.065) (0.056) (0.085) (0.141) (0.147) Gender = Female -0.031** 0.002 -0.022 -0.007 -0.051* -0.057** (0.012) (0.015) (0.015) (0.022) (0.027) (0.028) ln(Age) 0.005 0.029*** 0.019 -0.260*** 0.036* -0.106*** -0.107*** (0.008) (0.010) (0.012) (0.026) (0.021) (0.018) (0.018) Household Head Has At Least Elementary -0.011 -0.028** -0.058** -0.012 0.013 -0.023 -0.026 Education (0.018) (0.012) (0.023) (0.032) (0.024) (0.056) (0.059) Endline -0.072** -0.021 -0.074 -0.112*** 0.034 -0.003 -0.076 (0.036) (0.036) (0.052) (0.024) (0.036) (0.084) (0.090) Observations 6,309 6,458 6,315 8,097 1,850 2,538 2,538 Participant N 2960 3029 2964 3771 877 1212 1212 Non-Participant N 3349 3429 3351 4326 973 1326 1326 Treatment Effect 95% CI [ -0.350; 0.204] [ -0.336; 0.120] [ -0.515; 0.047] [ -0.401; 0.043] [ -0.262; 0.427] [ -0.454; 0.884] [ -0.561; 0.755] Treatment Effect Size -0.237 -0.264 -0.599 -0.455 0.301 0.464 0.207 *** p<0.01, ** p<0.05, * p<0.1 Note: Robust standard errors in parentheses clustered at the village level. Effect Size calculated as the estimated coefficient divided by the standard deviation of the outcome at endline. Age is measured in days for children and years for women. YAAJEENDE FINAL IMPACT EVALUATION REPORT ANNEXES 1 1 2 Table 14 shows the results for the impact on the two household-level outcomes. As was the case for the individual-level outcomes, we see no evidence for differential effects based on the village-level strength of participation, while the overall treatment effect of participation is smaller for the effect on HDDS and is no longer statistically significant, compared to the previous regressions on these outcomes seen in Table X. TABLE 14. OUTCOME FAMILY 2 VILLAGE FIXED EFFECTS DID RESULTS: HOUSEHOLD-LEVEL OUTCOMES, ML-EL. MOTHERS GROUP PARTICIPATION VILLAGE INTENSITY REGRESSIONS. 2.1 2.3 VARIABLES Household Dietary Diversity Score - Past 24 hrs Likelihood of poverty at the $1.25 2005 PPP threshold (%) Intensity Dif. Effect (Participant* 0.654 3.700 Intensity*Endline) (0.878) (3.975) Mothers Group 0.046 -2.894 Treatment Effect (0.465) (2.444) Endline*Intensity -0.053 -2.211 (0.571) (2.384) Participant*Intensity 0.120 -2.752 (0.641) (2.969) Mothers Group -0.051 1.467 Participant (0.308) (1.704) ln(Household Size) 0.438*** -3.076*** (0.089) (0.398) ln(Head Age) 0.067 0.007 (0.167) (1.000) Head Education Level: At Least Elementary Education = 1 0.596*** -3.131** (0.131) (1.236) Endline -0.039 -6.473*** (0.257) (1.152) Observations 3,409 3,419 Participant N 1530 1534 Non-Participant N 1879 1885 Treatment Effect 95% CI [ -1.066; 2.374] [ -4.091; 11.491] Treatment Effect Size 0.357 0.348 *** p<0.01, ** p<0.05, * p<0.1 Note: Robust standard errors in parentheses clustered at the village level. Effect Size calculated as the estimated coefficient divided by the standard deviation of the outcome at endline. YAAJEENDE FINAL IMPACT EVALUATION REPORT ANNEXES 1 1 3 ANNEX III – DATA COLLECTION INSTRUMENTS ENDLINE HOUSEHOLD SURVEY INSTRUMENT Yaajeende Endline Survey Questionnaire_FINAL16mar_forExport.xlsx Please double click on the object to view the full household survey instrument. YAAJEENDE FINAL IMPACT EVALUATION REPORT ANNEXES 1 1 4 ENDLINE GD AND KII PROTOCOLS EVALUATION FINALE DU PROJECT YAAJEENDE, SENEGAL PROTOCOL #1 : MEMBRES DE DEBBO GALLE ET AUTRES BENEFICIARES FEMININS ; AUTRES BENEFICIARES FEMININS (VILLAGES DE TRAITEMENT) INTRODUCTION ET CONSENTEMENT [POUR DEBUTER, LE MODERATEUR DOIT LIRE LE SCRIPTE DE CONSENTEMENT QUI SUIT] : Bonjour et merci d’avoir accepté de me parler. Je m’appelle (nom d’intervieweur/intervieweuse) ____ et voici mon collègue ___ qui prendra des notes pendant la conversation. Nous travaillons pour CRDH, en collaboration avec MSI et NORC à l’Université de Chicago, basé aux Etats Unis. USAID nous a chargés de mener une étude pour évaluer les impacts du projet Yaajeende. Dans le cadre de cette étude, nous voudrions parler avec vous sur vos expériences sur ce projet et ses activités dans votre communauté. La discussion durera au maximum une heure et demi. L’objectif de cette discussion est de vous écouter et de recueillir vos opinions et expériences du projet Yaajeende dans votre communauté, ainsi que votre perception des changements qui en auraient découlé. Notez qu’il n’y a pas de bonnes ou mauvaises réponses. Sentez-vous libre de partager vos expériences et réactions, positives ou négatives. Si c’est possible, donnez des exemples pour illustrer ce que vous dites. Notre rôle ici est de susciter la discussion et les échanges, pour que chacun participe et puisse donner son avis et son opinion. Nous souhaitons enregistrer cette discussion afin de noter fidèlement le contenu de nos échanges pour ne rien oublier de ce qui sera dit ici. Vos identités ne seront pas divulguées. Les informations qui seront collectées à travers cette discussion seront traitées de façon anonyme et confidentielle. Votre participation est entièrement libre et vous pourrez choisir de ne pas répondre à une question ou d’interrompre votre participation à tout moment si vous trouvez les discussions gênantes ou vous vous sentez mal à l’aise. Néanmoins, votre contribution est très importante pour aider l’USAID à rendre ses programmes de nutrition et de sécurité alimentaire plus efficaces et mieux adaptés aux besoins de la population du Sénégal. Si vous avez des questions sur l’étude, vous pouvez nous les poser maintenant, ou contacter M. Souleymane BARR au 77.448.27.13 ou au 33.820.82.08. Etre-vous d’accord de participer à la discussion ? OUI : __/ NON :____/ [SI LES INTERVIEWÉS DISENT OUI, CONTINUEZ LA DISCUSSION. SI NON], REMERCIEZ LA PERSONNE ET ARRETRER L’INTERVIEW Parfait, dans ce cas, commençons ! Une dernière petite note : Veuillez s’il vous plait mettre vos téléphones en mode vibreur et ne pas répondre a moins d’une urgence pour éviter de déranger les discussions. [DEMARRER L’ENREGISTEUR] YAAJEENDE FINAL IMPACT EVALUATION REPORT ANNEXES 1 1 5 Type de Répondent dans le Groupe de Discussion :  Membres de groupes Debbo Galle (GDGs)  Autres bénéficiaires masculins  Autres bénéficiaires féminins  Femmes dans les villages de comparaisons Facilitateur : ____________________ Preneur de notes : _____________________ Début : ___ :___ AM/PM (Encerclez) Fin: ___ :___ AM/PM (Encerclez) Numéro de l’enregistrement :________________________________ Evaluation finale du projet Yaajeende Région : Matam / Bakel / Kedougou (Encerclez) Date : Mois: _____________ Jour: _______ Année : 2018 Département:___________________________________ Commune : _________________________________ Village :_________________________________ Parti cipa nt Occupation principale (eg. Fermier, Bétail, Eleveur bovin, Autre (Veuillez préciser)) Genre (H/F) Age Etat civil (Marié, Veuve, Célibataire, Divorcé) Niveau d’éducation (Nombre d’années) Ethnie Nombre d’enfants sous l’âge de 5 ans dans le ménage 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13. 14. 15. YAAJEENDE FINAL IMPACT EVALUATION REPORT ANNEXES 1 1 6 Commentaires sur le déroulement du groupe de discussion (Niveau de discussion– eg. haut niveau d’activité, peu d’engagement, consensus général, contentieuse, individu(s) dominant, etc.) : YAAJEENDE FINAL IMPACT EVALUATION REPORT ANNEXES 1 1 7 Evaluation finale du projet Yaajeende : GUIDE DE DISCUSSION EN GROUPE #1 : Utilise Ce Guide Pour : (1) Membres des Groupes Debbo Galle ; ou (2) Autres Bénéficiaires Féminins THEMES / SUJETS D’INVESTIGATION : • REALISATIONS / IMPACT DU PROGRAMME ET BIEN-ÊTRE DES MÉNAGES) • EFFICACITÉ • CIBLAGE ET DISTRIBUTION DES BENEFICES DANS LE GROUPE • DURABILITÉ DES RÉSULTATS Les répondants peuvent avoir participé à plusieurs activités de Yaajeende, y compris : • Groupes de Mères à Mères ou groupes Debbo Galle (GDG) • Micro-jardin de marché, de l’école, du ménage, ou communautaire • Passage du don • Repas communautaires de démonstration organisés par Yaajeende • Membres d’un groupe de producteurs soutenus par Yaajeende • Formations sur les systèmes et technologies d’agriculture (y compris l’agriculture de conservation, parcelles de démonstration, utilisation de trous zaï, utilisation de services de labour motorisée, semences améliorées, engrais, compostage, et/ou assurance agricole) • Programmes d’agriculture irriguée et de décrue (y compris formations sur l’espacement des plantations, le traitement des semences, le fumier organique, et l’application d’engrais en micro-doses) • Activités d’horticulture commerciale et de biorestauration de terres dégradées • Activités d’élevage et/ou formation à la production avicole moderne • Formations en transformation agroalimentaire et/ou participation aux activités agro-industrielles • Activités de financement et d’assurance des récoltes ou du bétail, y compris la formation au crédit et l’accès/l’accord de crédit • Fournisseurs communautaires de services agricoles ou nutritionnels et/ou bénévoles en nutrition • Activités de mécanisation, de post récolte et de commercialisation/marketing (y compris l’accès et l’achat d’équipement agricole, d’irrigation, ou de post récolte) • Activités et formations lies à l’eau-assainissement-hygiène • Membres de groupes de travail citoyen • Utilisation de services fournis par un agent prestataire de services (APS) YAAJEENDE FINAL IMPACT EVALUATION REPORT ANNEXES 1 1 8 Questions Instructions Supplémentaires pour Modérateur A. QUESTIONS D’INTRODUCTION 1. Connaissez-vous ce projet ? Est-ce-que vous vous en rappelez ? a. Par quel nom appelez-vous le projet dans votre village ? b. Pouvez-vous décrire brièvement votre participation au projet Yaajeende ? c. Faites-vous ou faisiez-vous parti d’un groupe Debbo Galle ? (Utilisez-vous un autre nom pour ces groupes ‘Debbo Galle’ ?) a. Si oui, depuis combien de temps ce groupe est-il actif ? b. Si le groupe n’est pas actif, pourquoi pas ? MODÉRATEUR : [SE REFERER A LA LISTE DES ACTIVITÉS CONTENUE CI-DESSUS] ➔ OBTIEN LES ACTIVITES PRINCIPLES SANS ENTRER DANS TROP DE DETAILS. Pour évaluer la familiarité des participants avec le programme et assurer l'utilisation de la terminologie du programme tel que connu par les participants tout au long du reste de la discussion. B. IMPACT DU PROGRAMME SUR LE BIEN-ÊTRE DES MÉNAGES MAINTENANT, NOUS ALLONS DISCUTER EN DETAIL LES ACTIVITES DU PROJET ET COMMENT CES ACTIVITES ONT AFFECTE VOS CONDITIONS DE VIE ET QUELLES SONT VOS IMPRESSIONS GENERALES DU PROJET. POUR CHAQUE SUJET, NOUS VOULONS VOTRE AVIS SUR LES ACTIVITES DU PROJET QUI ONT LE PLUS AIDE VOTRE MENAGE ET COMMUNAUTE, ET QUELLES EN SONT LES RAISONS. 2. Le projet Yaajeende a-t-il conduit à des changements dans vos pratiques agricoles ou d’élevage ? Si oui, qu’est ce qui a changé dans ces pratiques ? Si non, pourquoi ? a. Cultivez-vous des cultures nouvelles ou différentes ? (Pourquoi ou pourquoi pas ? Si oui, lesquelles ?) b. Utilisez-vous des variétés de semences différentes ? Utilisez-vous de nouveaux intrants que vous n’utilisiez pas avant (comme des engrais par exemple) ? (Pourquoi ou pourquoi pas ? Si oui, lesquelles et comment vous les procurez-vous ?) c. Avez-vous changé ou modifié vos pratiques agricoles ? (Pourquoi ou pourquoi pas ? Si oui, de quelle manière ?) d. Avez-vous fait de l’élevage de nouveaux types d’animaux ? Si oui, lesquels ? e. Qui a été à l’origine de ces changements ? Quel a été le rôle des groupe(s) Debbo Galle, des prestataires de service communautaires (CBSPs), et des groupes de travail citoyen (CWGs) et comités de direction locaux (VSCs) dans ces changements ? D’autres groupes d’importances ? (Avez-vous des exemples ?) f. Quels défis continuent d’exister par rapport à l’agriculture dans la communauté ? 3. Y a-t-il eu des pratiques agricoles ou d’élevage promues par le projet que la communauté ne voulait pas ou ne pouvait pas pratiquer ? (Lesquelles ? Pourquoi ?) 4. Y avait-il des cultures et techniques sur lesquelles la communauté voulait être formée et qui n'étaient pas couvertes par les activités du projet ? (Lesquelles ?) Les 2 premières questions de ce guide ne devraient pas prendre plus de 15 minutes. YAAJEENDE FINAL IMPACT EVALUATION REPORT ANNEXES 1 1 9 Questions Instructions Supplémentaires pour Modérateur 5. Avez-vous constaté un changement dans les rendements ou les revenus agricoles ou d’élevage à la suite de l'adoption d'une des pratiques agricoles ou d’élevage promues par le projet ? a. [Si oui], Quelles sont les cultures qui ont bénéficié de ces nouvelles pratiques ? b. [Si oui], Quelles nouvelles pratiques ont engendrées la plus grande amélioration ? c. [Si non] Pourquoi ? 6. Avez-vous utilisé ou acheté des intrants agricoles, des services vétérinaires, des produits nutritionnelles ou d’hygiène chez un des prestataires de service communautaires soutenus par Yaajeende ? (Si non, pourquoi ?) a. [Si oui], que pensez-vous de la qualité et disponibilité de ces produits et services ? Comment se présente la situation actuelle par rapport à avant Yaajeende ? b. [Si oui], que pensez-vous des prix de ces produits et services ? Sont-ils à la portée de la majorité de la communauté ? Comment se présente la situation actuelle par rapport à avant Yaajeende ? c. [Si non] Pourquoi ? d. Que pensez-vous des conseils, formations, et autre assistance technique apportés par ces prestataires de services communautaires ? 7. Y a-t-il eu des activités promues par le projet que vous et/ou votre ménage vouliez mettre en œuvre mais qui n'ont pas pu se faire en raison des coûts financiers ? Lesquelles ? a. Y avait-il des activités que vous et/ou votre ménage vouliez mettre en œuvre mais qui n’a pas pu se faire pour des raisons autres que le coût ? Lesquelles et pourquoi ? 8. Avez-vous constaté des changements dans la santé et la productivité du bétail ou d'autres animaux de ferme, et dans leurs progénitures, à la suite de l'adoption d'une des pratiques d’élevage promues par le projet ? a. [Si oui], Quelles sont les animaux qui ont bénéficié de ces nouvelles pratiques ? b. [Si oui], Quelles nouvelles pratiques ont engendrées la plus grande amélioration ? c. [Si non] Pourquoi ? 9. Les activités de Yaajeende vous ont-elles aidé à avoir assez à manger durant l’année ? a. [Si oui], quelles activités ont été particulièrement utiles ? Lesquelles ne l’étaient pas ? (Pourquoi ?) b. [Si oui], est-ce le cas pour les femmes enceintes ? Les bébés et les enfants ? c. Quel a été le rôle des groupe(s) Debbo Galle, des prestataires de service communautaires (CBSPs), et des groupes de travail citoyen (CWGs) et comités de direction locaux (VSCs) dans ces changements ? d. Quels défis continuent d’exister par rapport à la quantité de nourriture disponible ? 10. Les activités de Yaajeende ont-elles changées les pratiques d’allaitement des femmes de la communauté ? a. [Si oui], de quelles manières ? Quelles activités ont été particulièrement utiles ? Lesquelles ne l’étaient pas ? (Pourquoi ?) YAAJEENDE FINAL IMPACT EVALUATION REPORT ANNEXES 1 2 0 Questions Instructions Supplémentaires pour Modérateur b. Quel a été le rôle des groupe(s) Debbo Galle, des prestataires de service communautaires (CBSPs), et des groupes de travail citoyen (CWGs) et comités de direction locaux (VSCs) dans ces changements ? c. Quels défis continuent d’exister par rapport à la diversité et qualité de nourriture disponible ? 11. Les activités de Yaajeende vous ont-elles aidées à avoir à une alimentation plus diverse et nourrissante ? a. [Si oui], quelles activités ont été particulièrement utiles ? Lesquelles ne l’étaient pas ? (Pourquoi ?) b. Préparez-vous des repas différents de ceux que vous prépariez avant le projet dans vos ménages ? Utilisez-vous des aliments différents ? c. Utilisez-vous une alimentation spécifique/différente pour les groupes les plus vulnérables, notamment les femmes enceintes, les bébés et les enfants ? (Si oui, en quoi cela consiste ?) d. Quel a été le rôle des groupe(s) Debbo Galle, des prestataires de service communautaires (CBSPs), et des groupes de travail citoyen (CWGs) et comités de direction locaux (VSCs) dans ces changements ? e. Quels défis continuent d’exister par rapport à la diversité et qualité de nourriture disponible ? 12. Quelles informations sur la nutrition et l’allaitement ont été communiquées par Yaajeende ? Avez-vous des exemples ? a. Y a-t-il eu des formations ou des discussions dans le village sur l’allaitement exclusif et son importance ? (Si oui, qui parmi vous y a participé ?) b. Pensez-vous être assez bien informées sur le sujet de l’allaitement exclusif et à la nutrition des bébés/enfants ? c. Quel a été le rôle des groupe(s) Debbo Galle, des prestataires de service communautaires (CBSPs), et des groupes de travail citoyen (CWGs) et comités de direction locaux (VSCs) dans ces formations et discussions ? d. Les mères de votre communauté mettent-elles en pratique les informations et méthodes enseignées liées à l’allaitement exclusif et à la nutrition des bébés ? (Pourquoi ou pourquoi pas ?) e. En pratique, quels défis existent encore dans la communauté par rapport à l’allaitement exclusif des enfants de moins de 6 mois ? (Citez des exemples ?) 13. En générale, quels sont les défis auxquels les femmes sont confrontées pour allaiter leurs enfants tout au long de la journée, dans ce village ? [Facilitateur : Question de suivi additionnel, s’il n’y a pas suffisamment de réponse dans la question ci-dessous] 14. En quoi la vie des femmes a-t-elle changé grâce aux activités de Yaajeende ? Et la vie des jeunes enfants ? Pouvez-vous donner quelques exemples ? 15. Les activités du projet Yaajeende ont-elles permis d’améliorer la santé et le bien-être des ménages dans votre village/communauté ? YAAJEENDE FINAL IMPACT EVALUATION REPORT ANNEXES 1 2 1 Questions Instructions Supplémentaires pour Modérateur a. [Si oui], quelle a été la contribution la plus importante de Yaajeende à la sante et le bien-être des ménages ? b. [Si non], pourquoi le projet n’a-t-il pas eu d’effet sur la santé et le bien-être des ménages selon vous ? c. Comment cela a-t-il été réalisé ? d. Quels groupes (ex., GDGs, CBSPs, CWGs or VSCs) et individus ont été les plus utiles pour accomplir ces changements ? e. En quoi ces acteurs clés ont-ils rendu l’intervention si efficace ? 16. Quels sont les plus grands défis dans votre communauté par rapport à : a. La santé des femmes et celle de vos enfants ? b. La participation dans la chaîne de valeur ? c. L’accès des femmes au crédit ? d. Autres activités qui améliorent vos moyens de subsistance ? Chaîne de valeur inclus : production, stockage, transformation (eg. farine, concentrés, confitures, pâtes, séchage, et torréfaction), emballage, et marketing/ distribution/ vente des légumes et céréales ; transformation de lait en fromage ou yogourt, etc. C. CIBLAGE ET DISTRIBUTION DES BENEFICES DANS LA COMMUNAUTE 17. Qui a été choisi pour participer aux différentes activités de Yaajeende dans la communauté ? a. De quelle manière ces groupes et individus ont-ils été choisis pour participer aux différentes activités ? b. Pourquoi pensez-vous que ces groupes ou individus ont été choisis ? c. A votre avis, Yaajeende et les responsables de mise en œuvre dans votre communauté ont-ils bien choisi les groupes et individus ciblés ? (Pourquoi ou pourquoi pas ?) 18. Certains ménages ont-ils plus bénéficié que d’autres des activités et retombées du projet Yaajeende dans votre village/communauté ? Pour quelles activités ? a. Pourquoi pensez-vous que ces ménages ou groupes ont plus bénéficié ? 19. Qui a bénéficié le moins des activités de Yaajeende dans votre communauté ? a. Pourquoi pensez-vous qu’ils ont moins bénéficié ? b. Y a-t-il des ménages qui n’ont pas bénéficiés du tout ? (Si oui, pourquoi ?) c. Quelles suggestions avez-vous pour que ces ménages en profitent davantage ? YAAJEENDE FINAL IMPACT EVALUATION REPORT ANNEXES 1 2 2 Questions Instructions Supplémentaires pour Modérateur 20. Quoi d’autre le projet aurait-il pu faire pour mieux travailler avec les membres de cette communauté ? D. PÉRENNITÉ 21. Selon votre expérience, est-ce que les gens qui ont appris de nouvelles pratiques agricoles, nutritionnelles, WASH, ou de moyens de subsistance à travers Yaajeende les ont partagées avec d'autres dans votre communauté, par exemple ceux qui n'ont pas reçu ces formations ? a. Comment cela s’est-il passé ? Pour quels types d’activités ? (Donner des exemples si possible) b. Pensez-vous que des personnes hors de cette communauté / en dehors des communautés touchées par Yaajeende ont pu bénéficier des activités de Yaajeende ? c. [Si oui], Donnez quelques exemples si possible ; comment cela s’est-il produit ? Comment ont-ils pu apprendre ou bénéficier efficacement ? 22. Pensez-vous avoir reçu assez de formations et autres ressources pour continuer les pratiques introduites par Yaajeende par vous-même ? a. [Si oui], quelles pratiques pensez-vous continuer, et pourquoi ? b. [Si non], Pourquoi pas ? Qu’est-ce qui vous manque ou vous empêche de continuer ces pratiques ? c. Y a-t-il des raisons culturelles pour lesquelles des activités ou pratiques introduites par Yaajeende n’ont pas été suivies où adoptées ? Qu’est ce qui pourrait être fait pour accroitre l’adoption de ces pratiques ? d. Que pensez-vous que Yaajeende pourrait faire d'autre pour améliorer la nutrition et la sécurité alimentaire dans votre communauté ? E. QUESTIONS DE CLOTURE 23. Y a-t-il quelque chose que vous aimeriez dire à propos du projet Yaajeende dont nous n'avons pas déjà parlé ? Maintenant notre discussion est terminée. Nous avons beaucoup appris et nous vous remercions vivement pour votre participation. Avant de partir, avez-vous des questions pour moi ? YAAJEENDE FINAL IMPACT EVALUATION REPORT ANNEXES 1 2 3 EVALUATION FINALE DU PROJET YAAJEENDE, SENEGAL PROTOCOL #1 : MEMBRES DE DEBBO GALLE ET AUTRES BENEFICIARES FEMININS ; AUTRES BENEFICIARES FEMININS (VILLAGES DE TRAITEMENT) INTRODUCTION ET CONSENTEMENT [POUR DEBUTER, LE MODERATEUR DOIT LIRE LE SCRIPTE DE CONSENTEMENT QUI SUIT] : Bonjour et merci d’avoir accepté de me parler. Je m’appelle (nom d’intervieweur/intervieweuse) ____ et voici mon collègue ___ qui prendra des notes pendant la conversation. Nous travaillons pour CRDH, en collaboration avec MSI et NORC à l’Université de Chicago, basé aux Etats Unis. USAID nous a chargés de mener une étude pour évaluer les impacts du projet Yaajeende. Dans le cadre de cette étude, nous voudrions parler avec vous sur vos expériences sur ce projet et ses activités dans votre communauté. La discussion durera au maximum une heure et demi. L’objectif de cette discussion est de vous écouter et de recueillir vos opinions et expériences du projet Yaajeende dans votre communauté, ainsi que votre perception des changements qui en auraient découlé. Notez qu’il n’y a pas de bonnes ou mauvaises réponses. Sentez-vous libre de partager vos expériences et réactions, positives ou négatives. Si c’est possible, donnez des exemples pour illustrer ce que vous dites. Notre rôle ici est de susciter la discussion et les échanges, pour que chacun participe et puisse donner son avis et son opinion. Nous souhaitons enregistrer cette discussion afin de noter fidèlement le contenu de nos échanges pour ne rien oublier de ce qui sera dit ici. Vos identités ne seront pas divulguées. Les informations qui seront collectées à travers cette discussion seront traitées de façon anonyme et confidentielle. Votre participation est entièrement libre et vous pourrez choisir de ne pas répondre à une question ou d’interrompre votre participation à tout moment si vous trouvez les discussions gênantes ou vous vous sentez mal à l’aise. Néanmoins, votre contribution est très importante pour aider l’USAID à rendre ses programmes de nutrition et de sécurité alimentaire plus efficaces et mieux adaptés aux besoins de la population du Sénégal. Si vous avez des questions sur l’étude, vous pouvez nous les poser maintenant, ou contacter M. Souleymane BARR au 77.448.27.13 ou au 33.820.82.08. Etre-vous d’accord de participer à la discussion ? OUI :__/ NON :____/ [SI LES INTERVIEWÉS DISENT OUI, CONTINUEZ LA DISCUSSION. SI NON], REMERCIEZ LA PERSONNE ET ARRETRER L’INTERVIEW Parfait, dans ce cas, commençons ! Une dernière petite note : Veuillez s’il vous plait mettre vos téléphones en mode vibreur et ne pas répondre a moins d’une urgence pour éviter de déranger les discussions. [DEMARRER L’ENREGISTEUR] YAAJEENDE FINAL IMPACT EVALUATION REPORT ANNEXES 1 2 4 Type de Répondent dans le Groupe de Discussion :  Membres de groupes Debbo Galle (GDGs)  Autres bénéficiaires masculins  Autres bénéficiaires féminins  Femmes dans les villages de comparaisons Facilitateur : ____________________ Preneur de notes : _____________________ Début : ___ :___ AM/PM (Encerclez) Fin: ___ :___ AM/PM (Encerclez) Numéro de l’enregistrement :________________________________ Evaluation finale du projet Yaajeende Région : Matam / Bakel / Kedougou (Encerclez) Date : Mois : _____________ Jour: _______ Année : 2018 Département:___________________________________ Commune : _________________________________ Village :_________________________________ Parti cipa nt Occupation principale (eg. Fermier, Bétail, Eleveur bovin, Autre (Veuillez préciser)) Genre (H/F) Age Etat civil (Marié, Veuve, Célibataire, Divorcé) Niveau d’éducation (Nombre d’années) Ethnie Nombre d’enfants sous l’âge de 5 ans dans le ménage 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13. 14. 15. YAAJEENDE FINAL IMPACT EVALUATION REPORT ANNEXES 1 2 5 Commentaires sur le déroulement du groupe de discussion (Niveau de discussion– eg. haut niveau d’activité, peu d’engagement, consensus général, contentieuse, individu(s) dominant, etc.) : YAAJEENDE FINAL IMPACT EVALUATION REPORT ANNEXES 1 2 6 Evaluation finale du projet Yaajeende : GUIDE DE DISCUSSION EN GROUPE #1 : Utilise Ce Guide Pour : (1) Membres des Groupes Debbo Galle ; ou (2) Autres Bénéficiaires Féminins THEMES / SUJETS D’INVESTIGATION : • REALISATIONS / IMPACT DU PROGRAMME ET BIEN-ÊTRE DES MÉNAGES) • EFFICACITÉ • CIBLAGE ET DISTRIBUTION DES BENEFICES DANS LE GROUPE • DURABILITÉ DES RÉSULTATS Les répondants peuvent avoir participé à plusieurs activités de Yaajeende, y compris : • Groupes de Mères à Mères ou groupes Debbo Galle (GDG) • Micro-jardin de marché, de l’école, du ménage, ou communautaire • Passage du don • Repas communautaires de démonstration organisés par Yaajeende • Membres d’un groupe de producteurs soutenus par Yaajeende • Formations sur les systèmes et technologies d’agriculture (y compris l’agriculture de conservation, parcelles de démonstration, utilisation de trous zaï, utilisation de services de labour motorisée, semences améliorées, engrais, compostage, et/ou assurance agricole) • Programmes d’agriculture irriguée et de décrue (y compris formations sur l’espacement des plantations, le traitement des semences, le fumier organique, et l’application d’engrais en micro-doses) • Activités d’horticulture commerciale et de biorestauration de terres dégradées • Activités d’élevage et/ou formation à la production avicole moderne • Formations en transformation agroalimentaire et/ou participation aux activités agro-industrielles • Activités de financement et d’assurance des récoltes ou du bétail, y compris la formation au crédit et l’accès/l’accord de crédit • Fournisseurs communautaires de services agricoles ou nutritionnels et/ou bénévoles en nutrition • Activités de mécanisation, de post récolte et de commercialisation/marketing (y compris l’accès et l’achat d’équipement agricole, d’irrigation, ou de post récolte) • Activités et formations lies à l’eau-assainissement-hygiène • Membres de groupes de travail citoyen • Utilisation de services fournis par un agent prestataire de services (APS) YAAJEENDE FINAL IMPACT EVALUATION REPORT ANNEXES 1 2 7 Questions Instructions Supplémentaires pour Modérateur F. QUESTIONS D’INTRODUCTION 24. Connaissez-vous ce projet ? Est-ce-que vous vous en rappelez ? d. Par quel nom appelez-vous le projet dans votre village ? e. Pouvez-vous décrire brièvement votre participation au projet Yaajeende ? f. Faites-vous ou faisiez-vous parti d’un groupe Debbo Galle ? (Utilisez-vous un autre nom pour ces groupes ‘Debbo Galle’ ?) a. Si oui, depuis combien de temps ce groupe est-il actif ? b. Si le groupe n’est pas actif, pourquoi pas ? MODÉRATEUR : [SE REFERER A LA LISTE DES ACTIVITÉS CONTENUE CI-DESSUS] ➔ OBTIEN LES ACTIVITES PRINCIPLES SANS ENTRER DANS TROP DE DETAILS. Pour évaluer la familiarité des participants avec le programme et assurer l'utilisation de la terminologie du programme tel que connu par les participants tout au long du reste de la discussion. G. IMPACT DU PROGRAMME SUR LE BIEN-ÊTRE DES MÉNAGES MAINTENANT, NOUS ALLONS DISCUTER EN DETAIL LES ACTIVITES DU PROJET ET COMMENT CES ACTIVITES ONT AFFECTE VOS CONDITIONS DE VIE ET QUELLES SONT VOS IMPRESSIONS GENERALES DU PROJET. POUR CHAQUE SUJET, NOUS VOULONS VOTRE AVIS SUR LES ACTIVITES DU PROJET QUI ONT LE PLUS AIDE VOTRE MENAGE ET COMMUNAUTE, ET QUELLES EN SONT LES RAISONS. 25. Le projet Yaajeende a-t-il conduit à des changements dans vos pratiques agricoles ou d’élevage ? Si oui, qu’est ce qui a changé dans ces pratiques ? Si non, pourquoi ? g. Cultivez-vous des cultures nouvelles ou différentes ? (Pourquoi ou pourquoi pas ? Si oui, lesquelles ?) h. Utilisez-vous des variétés de semences différentes ? Utilisez-vous de nouveaux intrants que vous n’utilisiez pas avant (comme des engrais par exemple) ? (Pourquoi ou pourquoi pas ? Si oui, lesquelles et comment vous les procurez-vous ?) i. Avez-vous changé ou modifié vos pratiques agricoles ? (Pourquoi ou pourquoi pas ? Si oui, de quelle manière ?) j. Avez-vous fait de l’élevage de nouveaux types d’animaux ? Si oui, lesquels ? k. Qui a été à l’origine de ces changements ? Quel a été le rôle des groupe(s) Debbo Galle, des prestataires de service communautaires (CBSPs), et des groupes de travail citoyen (CWGs) et comités de direction locaux (VSCs) dans ces changements ? D’autres groupes d’importances ? (Avez-vous des exemples ?) l. Quels défis continuent d’exister par rapport à l’agriculture dans la communauté ? 26. Y a-t-il eu des pratiques agricoles ou d’élevage promues par le projet que la communauté ne voulait pas ou ne pouvait pas pratiquer ? (Lesquelles ? Pourquoi ?) 27. Y avait-il des cultures et techniques sur lesquelles la communauté voulait être formée et qui n'étaient pas couvertes par les activités du projet ? (Lesquelles ?) Les 2 premières questions de ce guide ne devraient pas prendre plus de 15 minutes. YAAJEENDE FINAL IMPACT EVALUATION REPORT ANNEXES 1 2 8 Questions Instructions Supplémentaires pour Modérateur 28. Avez-vous constaté un changement dans les rendements ou les revenus agricoles ou d’élevage à la suite de l'adoption d'une des pratiques agricoles ou d’élevage promues par le projet ? a. [Si oui], Quelles sont les cultures qui ont bénéficié de ces nouvelles pratiques ? b. [Si oui], Quelles nouvelles pratiques ont engendrées la plus grande amélioration ? c. [Si non] Pourquoi ? 29. Avez-vous utilisé ou acheté des intrants agricoles, des services vétérinaires, des produits nutritionnelles ou d’hygiène chez un des prestataires de service communautaires soutenus par Yaajeende ? (Si non, pourquoi ?) e. [Si oui], que pensez-vous de la qualité et disponibilité de ces produits et services ? Comment se présente la situation actuelle par rapport à avant Yaajeende ? f. [Si oui], que pensez-vous des prix de ces produits et services ? Sont-ils à la portée de la majorité de la communauté ? Comment se présente la situation actuelle par rapport à avant Yaajeende ? g. [Si non] Pourquoi ? h. Que pensez-vous des conseils, formations, et autre assistance technique apportés par ces prestataires de services communautaires ? 30. Y a-t-il eu des activités promues par le projet que vous et/ou votre ménage vouliez mettre en œuvre mais qui n'ont pas pu se faire en raison des coûts financiers ? Lesquelles ? b. Y avait-il des activités que vous et/ou votre ménage vouliez mettre en œuvre mais qui n’a pas pu se faire pour des raisons autres que le coût ? Lesquelles et pourquoi ? 31. Avez-vous constaté des changements dans la santé et la productivité du bétail ou d'autres animaux de ferme, et dans leurs progénitures, à la suite de l'adoption d'une des pratiques d’élevage promues par le projet ? a. [Si oui], Quelles sont les animaux qui ont bénéficié de ces nouvelles pratiques ? b. [Si oui], Quelles nouvelles pratiques ont engendrées la plus grande amélioration ? c. [Si non] Pourquoi ? 32. Les activités de Yaajeende vous ont-elles aidé à avoir assez à manger durant l’année ? e. [Si oui], quelles activités ont été particulièrement utiles ? Lesquelles ne l’étaient pas ? (Pourquoi ?) f. [Si oui], est-ce le cas pour les femmes enceintes ? Les bébés et les enfants ? g. Quel a été le rôle des groupe(s) Debbo Galle, des prestataires de service communautaires (CBSPs), et des groupes de travail citoyen (CWGs) et comités de direction locaux (VSCs) dans ces changements ? h. Quels défis continuent d’exister par rapport à la quantité de nourriture disponible ? 33. Les activités de Yaajeende ont-elles changées les pratiques d’allaitement des femmes de la communauté ? d. [Si oui], de quelles manières ? Quelles activités ont été particulièrement utiles ? Lesquelles ne l’étaient pas ? (Pourquoi ?) YAAJEENDE FINAL IMPACT EVALUATION REPORT ANNEXES 1 2 9 Questions Instructions Supplémentaires pour Modérateur e. Quel a été le rôle des groupe(s) Debbo Galle, des prestataires de service communautaires (CBSPs), et des groupes de travail citoyen (CWGs) et comités de direction locaux (VSCs) dans ces changements ? f. Quels défis continuent d’exister par rapport à la diversité et qualité de nourriture disponible ? 34. Les activités de Yaajeende vous ont-elles aidées à avoir à une alimentation plus diverse et nourrissante ? f. [Si oui], quelles activités ont été particulièrement utiles ? Lesquelles ne l’étaient pas ? (Pourquoi ?) g. Préparez-vous des repas différents de ceux que vous prépariez avant le projet dans vos ménages ? Utilisez-vous des aliments différents ? h. Utilisez-vous une alimentation spécifique/différente pour les groupes les plus vulnérables, notamment les femmes enceintes, les bébés et les enfants ? (Si oui, en quoi cela consiste ?) i. Quel a été le rôle des groupe(s) Debbo Galle, des prestataires de service communautaires (CBSPs), et des groupes de travail citoyen (CWGs) et comités de direction locaux (VSCs) dans ces changements ? j. Quels défis continuent d’exister par rapport à la diversité et qualité de nourriture disponible ? 35. Quelles informations sur la nutrition et l’allaitement ont été communiquées par Yaajeende ? Avez-vous des exemples ? f. Y a-t-il eu des formations ou des discussions dans le village sur l’allaitement exclusif et son importance ? (Si oui, qui parmi vous y a participé ?) g. Pensez-vous être assez bien informées sur le sujet de l’allaitement exclusif et à la nutrition des bébés/enfants ? h. Quel a été le rôle des groupe(s) Debbo Galle, des prestataires de service communautaires (CBSPs), et des groupes de travail citoyen (CWGs) et comités de direction locaux (VSCs) dans ces formations et discussions ? i. Les mères de votre communauté mettent-elles en pratique les informations et méthodes enseignées liées à l’allaitement exclusif et à la nutrition des bébés ? (Pourquoi ou pourquoi pas ?) j. En pratique, quels défis existent encore dans la communauté par rapport à l’allaitement exclusif des enfants de moins de 6 mois ? (Citez des exemples ?) 36. En générale, quels sont les défis auxquels les femmes sont confrontées pour allaiter leurs enfants tout au long de la journée, dans ce village ? [Facilitateur : Question de suivi additionnel, s’il n’y a pas suffisamment de réponse dans la question ci-dessous] 37. En quoi la vie des femmes a-t-elle changé grâce aux activités de Yaajeende ? Et la vie des jeunes enfants ? Pouvez-vous donner quelques exemples ? 38. Les activités du projet Yaajeende ont-elles permis d’améliorer la santé et le bien-être des ménages dans votre village/communauté ? YAAJEENDE FINAL IMPACT EVALUATION REPORT ANNEXES 1 3 0 Questions Instructions Supplémentaires pour Modérateur f. [Si oui], quelle a été la contribution la plus importante de Yaajeende à la sante et le bien-être des ménages ? g. [Si non], pourquoi le projet n’a-t-il pas eu d’effet sur la santé et le bien-être des ménages selon vous ? h. Comment cela a-t-il été réalisé ? i. Quels groupes (ex., GDGs, CBSPs, CWGs or VSCs) et individus ont été les plus utiles pour accomplir ces changements ? j. En quoi ces acteurs clés ont-ils rendu l’intervention si efficace ? 39. Quels sont les plus grands défis dans votre communauté par rapport à : e. La santé des femmes et celle de vos enfants ? f. La participation dans la chaîne de valeur ? g. L’accès des femmes au crédit ? h. Autres activités qui améliorent vos moyens de subsistance ? Chaîne de valeur inclus : production, stockage, transformation (eg. farine, concentrés, confitures, pâtes, séchage, et torréfaction), emballage, et marketing/ distribution/ vente des légumes et céréales ; transformation de lait en fromage ou yogourt, etc. H. CIBLAGE ET DISTRIBUTION DES BENEFICES DANS LA COMMUNAUTE 40. Qui a été choisi pour participer aux différentes activités de Yaajeende dans la communauté ? d. De quelle manière ces groupes et individus ont-ils été choisis pour participer aux différentes activités ? e. Pourquoi pensez-vous que ces groupes ou individus ont été choisis ? f. A votre avis, Yaajeende et les responsables de mise en œuvre dans votre communauté ont-ils bien choisi les groupes et individus ciblés ? (Pourquoi ou pourquoi pas ?) 41. Certains ménages ont-ils plus bénéficié que d’autres des activités et retombées du projet Yaajeende dans votre village/communauté ? Pour quelles activités ? b. Pourquoi pensez-vous que ces ménages ou groupes ont plus bénéficié ? 42. Qui a bénéficié le moins des activités de Yaajeende dans votre communauté ? d. Pourquoi pensez-vous qu’ils ont moins bénéficié ? e. Y a-t-il des ménages qui n’ont pas bénéficiés du tout ? (Si oui, pourquoi ?) f. Quelles suggestions avez-vous pour que ces ménages en profitent davantage ? YAAJEENDE FINAL IMPACT EVALUATION REPORT ANNEXES 1 3 1 Questions Instructions Supplémentaires pour Modérateur 43. Quoi d’autre le projet aurait-il pu faire pour mieux travailler avec les membres de cette communauté ? I. PÉRENNITÉ 44. Selon votre expérience, est-ce que les gens qui ont appris de nouvelles pratiques agricoles, nutritionnelles, WASH, ou de moyens de subsistance à travers Yaajeende les ont partagées avec d'autres dans votre communauté, par exemple ceux qui n'ont pas reçu ces formations ? d. Comment cela s’est-il passé ? Pour quels types d’activités ? (Donner des exemples si possible) e. Pensez-vous que des personnes hors de cette communauté / en dehors des communautés touchées par Yaajeende ont pu bénéficier des activités de Yaajeende ? f. [Si oui], Donnez quelques exemples si possible ; comment cela s’est-il produit ? Comment ont-ils pu apprendre ou bénéficier efficacement ? 45. Pensez-vous avoir reçu assez de formations et autres ressources pour continuer les pratiques introduites par Yaajeende par vous-même ? e. [Si oui], quelles pratiques pensez-vous continuer, et pourquoi ? f. [Si non], Pourquoi pas ? Qu’est-ce qui vous manque ou vous empêche de continuer ces pratiques ? g. Y a-t-il des raisons culturelles pour lesquelles des activités ou pratiques introduites par Yaajeende n’ont pas été suivies où adoptées ? Qu’est ce qui pourrait être fait pour accroitre l’adoption de ces pratiques ? h. Que pensez-vous que Yaajeende pourrait faire d'autre pour améliorer la nutrition et la sécurité alimentaire dans votre communauté ? J. QUESTIONS DE CLOTURE 46. Y a-t-il quelque chose que vous aimeriez dire à propos du projet Yaajeende dont nous n'avons pas déjà parlé ? Maintenant notre discussion est terminée. Nous avons beaucoup appris et nous vous remercions vivement pour votre participation. Avant de partir, avez-vous des questions pour moi ? YAAJEENDE FINAL IMPACT EVALUATION REPORT ANNEXES 1 3 2 EVALUATION FINALE DU PROJET YAAJEENDE, SENEGAL PROTOCOL #3 : FEMMES (VILLAGES DE COMPARAISON) INTRODUCTION ET CONSENTEMENT [POUR DEBUTER, LE MODERATEUR DOIT LIRE LE SCRIPTE DE CONSENTEMENT QUI SUIT] : Bonjour et merci d’avoir accepté de me parler. Je m’appelle (nom d’intervieweur/intervieweuse) ____ et voici mon collègue ___ qui prendra des notes pendant la conversation et _____ qui lui servira d’interprète. Nous travaillons pour CRDH, en collaboration avec MSI et NORC à l’Université de Chicago, basé aux Etats Unis. USAID nous a chargés de mener une étude pour évaluer les impacts du projet Yaajeende, qui été mise en œuvre dans d’autres villages de la région. Dans le cadre de cette étude, nous voudrions parler avec vous sur vos expériences avec des activités qui sont concerné avec l’agriculture, la nutrition, la sante, WASH et hygiène dans votre communauté. La discussion durera au maximum une heure et demi. L’objectif de cette discussion est de voir si dans votre village, il y’a eu des activités ou interventions similaires à celles que le projet Yaajeende a mis en œuvre dans les villages où il est intervenu. Notez qu’il n’y a pas de bonnes ou mauvaises réponses. Sentez-vous libre de partager vos expériences et réactions, positives ou négatives, et d’être précis. Si c’est possible, donnez des exemples pour illustrer ce que vous dites. Notre rôle ici est de poser des questions et d’écouter vos avis et expériences. Nous allons enregistrer cette discussion afin de noter fidèlement le contenu de la conversation, et de ne rien oublier de tout ce qui a été dit. Vos identités ne seront pas divulguées. Les informations qui vont être collectées à travers cette discussion seront conserver en sécurité et sont considérées comme confidentielles, elles ne seront partagées avec USAID que de façon anonyme. Votre participation est entièrement libre et vous pouvez choisir de ne pas répondre à une question ou d’interrompre votre participation à tout moment si vous trouvez les discussions gênantes ou vous vous sentez mal à l’aise. Néanmoins, votre contribution est très importante pour aider l’USAID à rendre leurs programmes de nutrition et sécurité alimentaire plus efficace et mieux adapté aux besoins de la population du Sénégal. Si vous avez des questions sur l’étude, vous pouvez nous les poser maintenant, ou contacter [Nom] par téléphone : [numéro de téléphone]. Etre-vous d’accord de participer à la discussion d’aujourd’hui que nous allons enregistrer ? OUI : __/ NON :____/ [SI LES INTERVIEWÉS DISENT OUI, CONTINUEZ LA DISCUSSION. SI NON], REMERCIEZ LA PERSONNE ET ARRETRER L’INTERVIEW Parfait, dans ce cas, commençons ! Une dernière petite note : Veuillez s’il vous plait mettre vos téléphones en mode vibreur et ne pas répondre a moins d’une urgence pour éviter de déranger les discussions. [DEMARRER L’ENREGISTEUR] YAAJEENDE FINAL IMPACT EVALUATION REPORT ANNEXES 1 3 3 Type de Répondent dans le Groupe de Discussion :  Membres de groupes Debbo Galle (GDGs)  Autres bénéficiaires masculins  Autres bénéficiaires féminins  Femmes dans les villages de comparaisons Facilitateur : ____________________ Preneur de notes : _____________________ Début : ___ :___ AM/PM (Encerclez) Fin: ___ :___ AM/PM (Encerclez) Numéro de l’enregistrement :________________________________ Evaluation finale du projet Yaajeende Région : Matam / Bakel / Kedougou (Encerclez) Date : Mois : _____________ Jour: _______ Année : 2018 Département:___________________________________ Commune : _________________________________ Village :_________________________________ Parti cipa nt Occupation principale (eg. Fermier, Bétail, Eleveur bovin, Autre (Veuillez préciser)) Genre (H/F) Age Etat civil (Marié, Veuve, Célibataire, Divorcé) Niveau d’éducation (Nombre d’années) Ethnie Nombre d’enfants sous l’âge de 5 ans dans le ménage 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13. 14. 15. YAAJEENDE FINAL IMPACT EVALUATION REPORT ANNEXES 1 3 4 Commentaires sur le déroulement du groupe de discussion (Niveau de discussion– eg. haut niveau d’activité, peu d’engagement, consensus général, contentieuse, individu(s) dominant, etc.) : YAAJEENDE FINAL IMPACT EVALUATION REPORT ANNEXES 1 3 5 Evaluation finale du projet Yaajeende : GUIDE DE DISCUSSION EN GROUP #3 : Femmes dans les villages de comparaison THEMES / SUJETS D’INVESTIGATION : • PARTICIPATION ET REALISATIONS DES PROGRAMMES SIMILAIRE A YAAJEENDE ET BIEN-ÊTRE DES MÉNAGES • EFFICACITÉ • CIBLAGE ET DISTRIBUTION DES BENEFICES DANS LE GROUPE • DURABILITÉ DES RÉSULTATS Les Répondants peuvent avoir participé à plusieurs activités similaires à celles de Yaajeende, par exemple : • Groupes de Mères à Mères ou groupes Debbo Galle (GDG) • Micro-jardin de marché, de l’école, du ménage, ou communautaire • Passage du don • Repas communautaires de démonstration organisés par Yaajeende • Membres d’un groupe de producteurs organisés par Yaajeende • Formations sur les systèmes et technologies d’agriculture (y compris l’agriculture de conservation, parcelles de démonstration, utilisation de fosses zaï, utilisation de services de labour motorisée, semences améliorées, engrais, compostage, et/ou assurance agricole) • Programmes d’agriculture irriguée et de décrue (y compris formations sur l’espacement des plantations, le traitement des semences, le fumier organique, et l’application d’engrais en micro-doses) • Activités d’horticulture commerciale et de biorestauration de terres dégradées • Activités d’élevage et/ou formation à la production avicole moderne • Formations en transformation agroalimentaire et/ou participation aux activités agro-industrielles • Activités de financement et d’assurance des récoltes ou du bétail, y compris la formation au crédit et l’accès/l’accord de crédit • Fournisseurs communautaires de services agricoles ou nutritionnels et/ou bénévoles en nutrition • Activités de mécanisation, de post récolte et de commercialisation/marketing (y compris l’accès et l’achat d’équipement agricole, d’irrigation, ou de post récolte) • Activités et formations lies à l’eau-assainissement-hygiène • Membres de groupes de travail citoyen • Utilisation de services fournis par un agent prestataire de services (APS) YAAJEENDE FINAL IMPACT EVALUATION REPORT ANNEXES 1 3 6 Questions Instructions Supplémentaires pour Modérateur A. QUESTIONS D’INTRODUCTION 1. Participez-vous à un ou plusieurs projets qui concerne l’agriculture, la nutrition, santé, l’hygiène ou WASH dans ce village ? a. Comment s’appellent ce ou ces projets et depuis quand ont-t-ils été mis en œuvre dans votre village ? b. Pouvez-vous décrire brièvement les activités de ce ou ces projets ainsi que votre participation au projet(s) ? Durant quelle année ces activités ont commencé ? g. Faites-vous ou faisiez-vous parti d’un groupe de mères mis en place par ce ou ces projets ? Ou, de tout autre activité de groupe ? Si oui, quelle groupe(s) ? Depuis combien de temps ce groupe est-il actif ? Si le groupe n’est pas actif, pourquoi pas ? MODÉRATEUR : [SE REFERER A LA LISTE DES ACTIVITÉS DE YAAJEENDE CONTENUE CI-DESSUS COMMES EXAMPLES DE CE QUI NOUS INTERESSE] Pour comprendre participation dans des activités qui sont similaire au projet Yaajeende, et assurer l'utilisation de la terminologie du programme tel que connu par les participants tout au long du reste de la discussion. B. IMPACT DU PROGRAMME ET BIEN-ÊTRE DES MÉNAGES MAINTENANT, NOUS ALLONS DISCUTER EN DETAIL DES ACTIVITES DU PROJET ET COMMENT CES ACTIVITES ONT AFFECTE VOS CONDITIONS DE VIE ET QUELLES SONT VOS IMPRESSIONS GENERALES DU PROJET. POUR CHAQUE SUJET, NOUS VOULONS VOTRE AVIS SUR QUELLES ACTIVITES DU PROJET ONT LE PLUS AIDE VOTRE MENAGE ET COMMUNAUTE, ET QUELS EN SONT LES RAISONS. 2. Description du projet et de ses activités a. Formations et sensibilisations sur des pratiques qui concerne l’agriculture, l’élevage, la nutrition, la santé, l’hygiène ou WASH b. Fournitures d’intrants agricoles (semences, engrais, etc.) ou d’élevage c. Fournitures de matériel et d’équipement agricole ou d’élevage d. Aménagement des terres et de périmètres e. Autres activités menées par le projet dans votre village/communauté 3. Qui dans la communauté a été choisi pour participer aux différentes activités de ce (ces) projet(s) ? a. De quelle manière ces groupes et individus ont-ils été choisis pour participer aux différentes activités ? b. Pourquoi pensez-vous que ces groupes ou individus ont été choisis ? c. A votre avis, le projet et les responsables de sa mise en œuvre dans votre communauté ont-ils bien choisi les groupes et individus ciblés ? (Pourquoi ou pourquoi pas ?) 4. Pendant les dernières années, avez-vous changé certains aspects de vos pratiques agricoles ou d’élevage ? Si oui, qu’est ce qui a changé dans ces pratiques ? Si non, pourquoi ? Les 2 premières questions de ce guide ne devraient pas prendre plus de 15 minutes. YAAJEENDE FINAL IMPACT EVALUATION REPORT ANNEXES 1 3 7 Questions Instructions Supplémentaires pour Modérateur a. Cultivez-vous des cultures nouvelles ou différentes ? (Pourquoi ou pourquoi pas ? Si oui, lesquelles ?) b. Utilisez-vous des variétés de semences différentes ? Utilisez-vous de nouveaux intrants que vous n’utilisiez pas avant (comme des engrais par exemple) ? (Pourquoi ou pourquoi pas ? Si oui, lesquelles et comment vous les procurez-vous ?) c. Avez-vous changé ou modifié vos techniques agricoles ou d’élevage ? (Pourquoi ou pourquoi pas ? Si oui, de quelle manière ?) d. Avez-vous fait de l’élevage de nouveaux types d’animaux ? Si oui, lesquels ? e. Qui a été à l’origine de ces changements ? D’autres groupes d’importance ? (Avez-vous des exemples ?) f. Quels défis continuent d’exister par rapport à l’agriculture ou l’élevage dans la communauté ? 5. Pendant les dernières années, avez-vous constaté des changement dans les rendements ou les revenus agricoles ou d’élevage à la suite de l'adoption de nouvelles pratiques agricoles ou d’élevage? a. [Si oui], Quelles sont les cultures qui ont bénéficié des nouvelles pratiques ? b. [Si oui], Quelles nouvelles pratiques ont engendrées la plus grande amélioration ? c. [Si non] Pourquoi ? 6. Pendant les dernières années, avez-vous utilisé ou acheté des intrants agricoles, des services vétérinaires, des produits nutritionnels ou d’hygiène que vous n’utilisiez pas avant dans votre village/communauté ? a. [Si oui], comment avez-vous obtenu ces produits et services ? b. [Si oui], que pensez-vous de la qualité et disponibilité de ces produits et services ? c. [Si oui], que pensez-vous des prix de ces produits et services ? Sont-ils à la portée de la majorité de la communauté ? d. [Si non] Pourquoi ? e. Que pensez-vous des conseils, formations, et autre assistance technique apportés par ces prestataires de services communautaires ? 7. Y a-t-il eu des activités que vous et/ou votre ménage vouliez mettre en œuvre mais qui n'ont pas pu se faire en raison des coûts financiers ? Lesquelles ? a. Y avait-il des activités que vous et/ou votre ménage vouliez mettre en œuvre mais qui n’ont pas pu se faire pour des raisons autres que le coût ? Lesquelles et pourquoi ? 8. Pendant les dernières années, avez-vous constaté des changements dans la santé et la productivité du bétail ou d'autres animaux de ferme, et dans leurs progénitures, à la suite de l'adoption de l'une ou l'autre des pratiques d’élevage ? a. [Si oui], Quels sont les animaux qui ont bénéficié des nouvelles pratiques ? b. [Si oui], Quelles nouvelles pratiques ont engendrées la plus grande amélioration ? c. [Si non] Pourquoi ? YAAJEENDE FINAL IMPACT EVALUATION REPORT ANNEXES 1 3 8 Questions Instructions Supplémentaires pour Modérateur 9. Pendant les dernières années, est-ce qu’il y a eu des activités qui vous ont aidées à avoir assez à manger durant l’année ? a. [Si oui], quelles activités ont été particulièrement utiles ? Lesquelles ne l’étaient pas ? (Pourquoi ?) b. [Si oui], est-ce le cas pour les femmes enceintes ? Les bébés et les enfants ? c. Qui a été responsable de ces changements ? D’autres groupes d’importance ? (Avez-vous des exemples ?) d. Quels défis continuent d’exister par rapport à la quantité de nourriture disponible ? 10. Pendant les dernières années, est qu’il y a des activités qui avez changées les pratiques d’allaitement des femmes de la communauté ? a. [Si oui], de quelle manière ? Quelles activités ont été particulièrement utiles ? Lesquelles ne l’étaient pas ? (Pourquoi ?) b. Qui a été responsable pour ces changements ? D’autres groupes d’importance ? (Avez-vous des exemples ?) c. Quels défis continuent d’exister par rapport à la diversité et qualité de nourriture disponible ? 11. Pendant les dernières années, est qu’il y a eu des activités qui vous ont aidé à avoir à une alimentation plus diverse et nourrissante ? a. [Si oui], quelles activités ont été particulièrement utiles ? Lesquelles ne l’étaient pas ? (Pourquoi ?) b. Préparez-vous des repas différents de ceux que vous prépariez avant le projet dans vos ménages ? Utilisez-vous des aliments différents ? c. Utilisez-vous une alimentation spécifique/différente pour les groupes les plus vulnérables, notamment les femmes enceintes, les bébés et les enfants ? (Si oui, en quoi cela consiste ?) d. Qui a été responsable de ces changements ? D’autres groupes d’importance ? (Avez-vous des exemples ?) e. Quels défis continuent d’exister par rapport à la diversité et qualité de nourriture disponible ? 12. Pendant les dernières années, est qu’il y a eu des activités de sensibilisation sur la nutrition et l’allaitement dans cette communauté ? Avez-vous des exemples ? a. Y a-t-il eu des formations ou des discussions dans le village ou la commune sur l’allaitement exclusif et son importance ? (Si oui, qui parmi vous y a participé ?) b. Pensez-vous être assez bien informées sur le sujet de l’allaitement exclusif et à la nutrition des bébés ? c. Qui a été responsable pour ces changements ? D’autres groupes d’importance ? (Avez-vous des exemples ?) d. Les mères de votre communauté mettent-elles en pratique les informations et méthodes liées à l’allaitement exclusif et à la nutrition des bébés ? (Pourquoi ou pourquoi pas ?) e. En pratique, quels défis existent encore dans la communauté par rapport à l’allaitement exclusif des enfants de moins de 6 mois ? (Avez-vous des exemples ?) YAAJEENDE FINAL IMPACT EVALUATION REPORT ANNEXES 1 3 9 Questions Instructions Supplémentaires pour Modérateur f. En général, quelles sont les raisons qui empêchent les ménages d'incorporer différents types d'aliments nutritifs dans leurs repas (y compris des raisons culturelles) ? 13. En générale, quels sont les défis auxquels les femmes sont confrontées pour allaiter leurs enfants tout au long de la journée, dans ce village ? [Facilitateur : Question de suivi additionnel, s’il n’y a pas suffisamment de réponse dans la question ci-dessous] 14. Quels sont les plus grands défis dans votre communauté par rapport à : a. La santé des femmes et celle des enfants ? b. La participation dans la chaîne de valeur ? c. L’accès des femmes au crédit ? d. Autres activités qui améliorent vos moyens de subsistance ? Chaîne de valeur inclus : production, stockage, transformation (eg. farine, concentrés, confitures, pâtes, séchage, et torréfaction), emballage, et marketing/ distribution/ vente des légumes et céréales ; transformation de lait en fromage ou yogourt, etc. C. QUESTIONS DE CLOTURE 15. Y a-t-il quelque chose que vous aimeriez dire dont nous n'avons pas déjà parlé ? Maintenant notre discussion est terminée. Nous avons beaucoup appris et nous vous remercions vivement pour votre participation. Avant de partir, avez-vous des questions pour moi ? YAAJEENDE FINAL IMPACT EVALUATION REPORT ANNEXES 1 4 0 EVALUATION FINALE DU PROJET YAAJEENDE, SENEGAL PROTOCOLES POUR LES KIIs (Autorités Régionales et Communales ; Agents du projet et relais) Introduction et Consentement Pour débuter, l’interviewer doit lire le scripte de consentement qui suit : Bonjour et merci d’avoir accepté de me parler. Je m’appelle (nom d’intervieweur/intervieweuse) ____ et voici mon collègue ___ qui prendra des notes pendant la conversation et _____ qui lui servira d’interprète. Nous travaillons pour CRDH, en collaboration avec MSI et NORC à l’Université de Chicago, basé aux Etats Unis. USAID nous a chargés de mener une étude pour évaluer les impacts du projet Yaajeende. Dans le cadre de cette étude, nous voudrions parler avec vous sur vos expériences sur ce projet et ses activités dans votre communauté. La discussion dura au maximum 1 heure. L’objectif de cette discussion est de vous écouter et de capturer vos expériences des activités liées du projet Yaajeende, ainsi que de votre perception des changements qui en ont découlés. Notez qu’il n’y a pas de bonnes ou mauvaises réponses. Sentez-vous libre de partager votre expériences et réactions, positives ou négatives, et d’être précis. Si c’est possible, donnez des exemples pour illustrer ce que vous dites. Notre rôle ici est de poser des questions et d’écouter votre avis et expériences. Nous allons enregistrer cette discussion afin de noter fidèlement le contenu de la conversation, et de ne rien oublier de tout ce qui a été dit. Vos identités ne seront pas divulguées. Les informations qui vont être collectées à travers cette discussion seront conserver en sécurité et sont considérées comme confidentielles, elles ne seront partagées avec USAID que de façon anonyme. Votre participation est entièrement libre et vous pouvez choisir de ne pas répondre à une question ou d’interrompre votre participation à tout moment si vous trouvez les discussions gênantes ou vous vous sentez mal à l’aise. Néanmoins, votre contribution est très importante pour aider l’USAID à rendre leurs programmes de nutrition et sécurité alimentaire plus efficace et mieux adapté aux besoins de la population du Sénégal. Si vous avez des questions sur l’étude, vous pouvez nous les poser maintenant, ou contacter M. Souleymane BARRY au 77.448.27.13 ou au 33.820.82.08. Etre-vous d’accord de participer à la discussion d’aujourd’hui ? OUI NON INFORMATION SUR L’INTERVIEW Département et Commune ____________________________ Village ____________________________________________  Village de Yaajeende (Traitement)  Village de Comparaison Facilitateur _______________________________ Enregistreur _______________________________ Nom de l’interviewé ____________________________ Employeur ou structure affilié ___________________ Date ________________________________ Heure de début d’interview ____________ YAAJEENDE FINAL IMPACT EVALUATION REPORT ANNEXES 1 4 1 Titre du Poste _________________________ Heure de fin d’interview _____________ Note pour facilitateur : Dans les villages de comparaison, commencez l’entretien avec un brève discussion du (des) projet(s) dans lesquels le relais est impliqué dans ce village. Ensuite, n’oubliez-pas de remplacer l’expression ‘du projet Yaajeende’ avec le nom de ces autres projets, pour les questions dans ce guide. (N’oubliez-pas que dans les villages de comparaison, les questions dans ce guide sont posées dans le contexte de ces autres projets, pas du projet Yaajeende). Questions d’introduction 1. Pourriez-vous donner un aperçu de votre fonction dans le cadre du projet Yaajeende / du projet ? o Quand et comment avez-vous commencé à travailler avec le projet Yaajeende / le projet ? o Quel type de services offrez-vous à la communauté ? o [Facilitateur : Ne posez pas cette question dans les villages de comparaison] Faites-vous partie d’une ou plusieurs structures de gouvernance locales établies par Yaajeende (e.g., groupe de travail citoyen [CWGs], comité de direction locaux [VSCs]) ? Ciblage et répartition des avantages pour les bénéficiaires du Projet 2. Quels sont les groupes qui sont principalement ciblés par les activités du projet, dans les communautés où Yaajeende (le projet) est mis en œuvre ? o Pensez-vous que les groupes ciblés sont ceux qui conduisent à un meilleur impact du projet ? o Y aurait-il d’autres groupes qui ne sont pas touchés par le projet mais qui aurait amélioré l’impact actuel du projet s’ils étaient bénéficiaires ? Lesquelles et pourquoi ? [Si organismes communautaires ; e.g. APS ; VSCs] 3. A quels groupes et individus dans les communautés offrez-vous la majorité de vos services ? o Sont-ils en majorité ceux qui ont reçu d’autres services offerts par Yaajeende / par le projet ? o Y a-t-il des membres de la communauté qui ne sollicitent jamais vos services ? Savez-vous pourquoi ? 4. Quel est le plus grand défi qui fait face aux membres des communautés qui cherchent à se procurer vos services ? o Pourquoi est-ce un défi majeur et comment pourrait-on y remédier ? Efficacité des interventions 5. Parmi les différentes activités mises en œuvre par le projet Yaajeende / le projet, pensez-vous que certaines était particulièrement efficace pour améliorer l’agriculture, l’élevage, la nutrition, la santé, et les moyens de subsistance ? Si oui, lesquelles ? o Quelles approches considérez-vous comme les plus efficaces, et pourquoi ? o A votre avis, y a-t-il eu des activités qui ont été inefficaces ou qui ont eu moins d’effets ? Si oui, lesquelles ? Quelles en sont les raisons ? o D’après vous, quelles sont les facteurs ou les contraintes qui ont contribué à une faible efficacité ou performance de ces activités ? 6. Avez-vous identifié des leçons particulières à tirer et ou découvert des effets inattendus liés aux actions / activités de Yaajeende / du projet ? Si oui, lesquelles ? YAAJEENDE FINAL IMPACT EVALUATION REPORT ANNEXES 1 4 2 7. Comment qualifieriez-vous la qualité et la quantité de formations que Yaajeende / le projet vous a offert ? 8. Avez-vous eu des interactions avec CultiVert ? o Si oui, de quelle manière avez-vous bénéficié de cette initiative ? o [Facilitateur : Posez cette question seulement dans les villages de comparaison] Avez-vous entendu parler du projet Yaajeende ? Si oui, de quelle manière avez-vous interagi avec ce projet ? 9. A votre avis, comment ont évolué le processus de commande, la vitesse de livraison, et la qualité des stocks que les agents de services offrent ? o Attribuez-vous ces changements au projet ? Pourquoi ou pourquoi pas ? 10. Facilitateur : Ne posez pas cette question dans les villages de comparaison Que pensez-vous de l’accès et du niveau de prix des services que les agents de services offrent à la communauté ? Contribution à l’atteinte des objectifs du projet 11. Selon vous, depuis le début du projet, y a-t-il eu des changements au niveau de la malnutrition et de l’état de santé général parmi : o Les groupes le plus vulnérables dans votre communauté (e.g., les femmes enceintes, les jeunes enfants) ? o Ceux que vous connaissez qui ont bénéficié des interventions de Yaajeende / du projet ? o La population plus généralement ? Possibilités d’amélioration 12. Y aurait-il des défis ou contraintes spécifiques qui limitent les résultats potentiels des activités mises en œuvre dans le cadre du projet Yaajeende / du projet ? Si oui, lesquels ? 13. Avez-vous des recommandations pour améliorer la mise en œuvre des activités du projet Yaajeende / du projet auxquels vous avez pris part ou contribué à sa réalisation ? Si oui, lesquelles ? Pérennité des acquis du projet 14. Selon vous, est-ce-que les bénéficiaires ont reçu les formations et ressources nécessaires et ont acquis des connaissances suffisantes pour pérenniser certaines ou la totalité des pratiques par eux-mêmes ? Pourquoi ? o [Si oui] Pensez-vous que les bénéficiaires sont en mesure de partager certaines de ces pratiques avec d’autres qui n’ont pas participé au projet ? Pourquoi ou pourquoi pas ? o [Si non] ? Quelles sont les principales raisons qui empêchent les bénéficiaires de maintenir durablement ces pratiques ? Y a-t-il des exceptions selon vous ? 15. Pensez-vous que les organisations qui mettent en œuvre les activités du projet Yaajeende / du projet sont motivées et intéressées à continuer d’offrir certaines ou la totalité de ces activités, une fois le projet terminé ? o Y a-t-il des exceptions notables ? o [Si non] Quels sont les principaux obstacles qui les en empêchent ? YAAJEENDE FINAL IMPACT EVALUATION REPORT ANNEXES 1 4 3 16. Pensez-vous que les bénéficiaires vont continuer à demander certains ou la totalité des services, le cas échéant ? Pensez-vous qu’ils soient assez intéressés et à même de supporter en partie les couts associés à ces services ? o [Si non] Quels sont les principaux obstacles qui les en empêchent ? 17. Facilitateur : Ne posez pas cette question dans les villages de comparaison Selon vous, est-ce-que certaines ou la totalité des activités pourraient être continuées indépendamment du financement de USAID ? Si ou, lesquelles et de quelle manière ? o Y a-t-il des exceptions à votre réponse ? Pourquoi considérez-vous ces activités comme des exceptions ? 18. Facilitateur : Ne posez pas cette question dans les villages de comparaison Quels sont les éléments de soutient clés (p. ex., des fonds, un soutien en nature, rendre les services payants, la formation et le renforcement des capacités, ou la promotion) qui seraient nécessaires pour que votre organisation continue d'offrir certaines des services de Yaajeende ? o Qu'est-ce qui a été fait par le projet Yaajeende pour s'assurer que les fournisseurs de services auront un accès continu pour améliorer leurs capacités ? 19. Que considérez-vous être les défis les plus important pour pérenniser les activités de Yaajeende ? *** Questions additionnelles au niveau de la commune / département / région : 1. Quels types de formations et de changement de comportements sur la nutrition et l’alimentation les bénéficiaires ont-ils reçus ? a. Le message a-t-il été compris / perçu d’une façon efficace ? b. De quelles façons les ménages et les femmes ont-ils augmenté les différents types d'aliments qu'ils mangent ? c. À quels défis sont-ils confrontés pour accéder aux différents types d’aliments ? d. Quelles sont les autres raisons qui empêchent les ménages d'incorporer différents types d'aliments nutritifs dans leurs repas (y compris des raisons culturelles) ? 2. De quelle manière l’accès des ménages à la nourriture a-t-il changé au cours de l'année et surtout pendant la période de soudure ? a. Quelles sont les raisons de ces changements ? Question de clôture 20. Auriez-vous d’autres commentaires que vous voudriez bien ajoutés ? YAAJEENDE FINAL IMPACT EVALUATION REPORT ANNEXES 1 4 4 ENTRETIEN CLÉ - PROFIL DE LA COMMUNAUTÉ Description : un questionnaire semi-structuré avec le chef du village (et jusqu’à 3-5 dirigeants communautaires ou anciens si c’est préféré par le chef). Dans la mesure du possible, ces questions seront posées aux anciens et aux chefs de village qui resteront ensuite hors des discussions des groupes de discussion (GD) et de toutes les entrevues avec les informateurs clés (ESS), permettant ainsi aux GD et aux ESS de se dérouler de façon indépendante et sans interjections ou interférences de ces leaders. Objectifs : • Comprendre le contexte de la communauté et obtenir des informations sur les principales caractéristiques socio￾culturelles, démographiques, environnementales, d'infrastructure et de moyens de subsistance de la communauté ; • Mieux comprendre les principaux défis liés à la sécurité alimentaire et aux moyens de subsistance, et les menaces naturelles ou humaines auxquels la communauté fait face. INTRODUCTION ET CONSENTEMENT [POUR DEBUTER, LE MODERATEUR DOIT LIRE LE SCRIPTE DE CONSENTEMENT QUI SUIT] : Bonjour et merci d’avoir accepté de me parler. Je m’appelle (nom d’intervieweur/intervieweuse) ____ et voici mon collègue ___ qui prendra des notes pendant la conversation et _____ qui lui servira d’interprète. Nous travaillons pour CRDH, en collaboration avec MSI et NORC à l’Université de Chicago, basé aux Etats Unis. USAID nous a chargés de mener une étude pour évaluer les impacts du projet Yaajeende, qui été mise en œuvre dans d’autres villages de la région. Dans le cadre de cette étude, nous voudrions parler avec vous de vos expériences avec des activités qui sont concerné avec l’agriculture, la nutrition, la santé, WASH et hygiène dans votre communauté. La discussion durera au maximum une heure. L’objectif de cette discussion est de mieux comprendre le contexte de la communauté, dont son infrastructure et ses principales caractéristiques socio-culturelles et environnementales ainsi que ses principaux défis liés à la sécurité alimentaire et aux moyens de subsistance. Notez qu’il n’y a pas de bonnes ou mauvaises réponses. Sentez-vous libre de partager vos expériences et réactions, positives ou négatives, et d’être précis. Si c’est possible, donnez des exemples pour illustrer ce que vous dites. Notre rôle ici est de poser des questions et d’écouter vos avis et expériences. Nous allons enregistrer cette discussion afin de noter fidèlement le contenu de la conversation, et de ne rien oublier de tout ce qui a été dit. Vos identités ne seront pas divulguées. Les informations qui vont être collectées à travers cette discussion seront conservées en sécurité et sont considérées comme confidentielles, elles ne seront partagées avec USAID que de façon anonyme. Votre participation est entièrement volontaire et vous pourrez choisir de ne pas répondre à certaines questions ou d’interrompre votre participation à tout moment si vous trouvez les discussions gênantes ou vous vous sentez mal à l’aise. Néanmoins, votre contribution est très importante pour aider l’USAID à rendre ses programmes de nutrition et de sécurité alimentaire plus efficaces et mieux adaptés aux besoins de la population du Sénégal. Si vous avez des questions sur l’étude, vous pouvez nous les poser maintenant, ou contacter M. Souleymane BARR au 77.448.27.13 ou au 33.820.82.08. Etre-vous d’accord de participer à la discussion ? OUI : ___/ NON : ____/ [SI LES INTERVIEWÉS DISENT OUI, CONTINUEZ LA DISCUSSION] YAAJEENDE FINAL IMPACT EVALUATION REPORT ANNEXES 1 4 5 INFORMATIONS SUR LE OU LES RÉPONDANT(S) Facilitateur _________________________________ Preneur de notes _______________________- ________ Département _____________________________ Commune ________________________________ Village ___________________________________ Date _____________________________________ Heure de début ____________ Heure de fin _____________ Nombre de Répondants _____ Titre des répondants __________________________ __________________________ __________________________ __________________________ __________________________ __________________________ A. Démographie 1. Quel et la population totale de ce village ? _____________ Combien de ménages vivent dans le village ? ____________ 2. Quels sont les principaux groupes ethniques de la population du village ? B. Infrastructure 3. Y a-t-il une école primaire dans le village ? ___________________________________________________________ 4. A quelle distance se trouve l'école secondaire la plus proche ? ______________________________________________ 5. A quelle distance se situe la structure sanitaire la plus proche, et dans quel village se trouve-t-elle ? __________________________________________________________________________________________ ____ a. Existe-il une sage-femme dans le village ? _____________________________________________________ b. Existe-il une matrone dans le village ? ________________________________________________________ 6. Le village dispose-t-il d’électricité ? ____________________________ a. Si oui, depuis quelle année ? ___________________________ b. Quelle est la source (ex., réseau, générateur, énergie solaire) ? ____________________________________ c. Pour les ménages qui n’ont pas de compteur électrique, quelle est leur source primaire d’électricité (ex., réseau, générateur, énergie solaire, utilisation du compteur d’un voisin) ? 7. Quelle est la source d'eau principale pour ce village ? _______________________________________________ YAAJEENDE FINAL IMPACT EVALUATION REPORT ANNEXES 1 4 6 a. Quelles sont les autres sources d’eau (puits ouverts, forages, eau courante, rivières / cours d'eau, étangs, autres), et combien existent de chaque type ? 8. Existe-t-il une couverture de téléphone cellulaire dans le village ? Si oui, depuis quelle année ? ____________________ 9. A quelle distance est la route bitumée la plus proche ? ____________________________________________________ 10. Disposez-vous d’un marché dans le village ? a. [SI NON] A quelle distance environ se trouve le marché hebdomadaire le plus proche ? __________________________ C. Moyens de Subsistance 11. Quels sont les principaux moyens de subsistance dans le village, pour la majorité de ménages vivant dans ce village (agriculture, élevage, pêche, agroforesterie, petit commerce, autres) ? a. Lequel est pratiqué principalement par les hommes ? b. Lequel est pratiqué principalement par les femmes ? c. Lequel est pratiqué principalement par les adolescentes ? d. Lequel est pratiqué principalement par les adolescents ? 12. Y a-t-il eu des changements majeurs dans ces moyens de subsistance au cours des 5 dernières années ? [Donnez 1-2 exemples pour mieux expliquer la question] a. [SI OUI] Pourquoi ces changements ont-ils eu lieu ? 13. Certains membres de cette communauté pratiquent-ils la migration saisonnière ou à long-terme ? Type de migration ? (Saisonnière ou à long￾terme) Qui migre ? (Jeunes hommes, familles entières, etc.) Raisons pour la migration Durée typique d’absence ? Principales destinations ? Mois / saison typique de retour ? 1 YAAJEENDE FINAL IMPACT EVALUATION REPORT ANNEXES 1 4 7 2 3 D. Catastrophes ; Assistance au développement 14. Depuis 2015, quels sont les 3 plus grand chocs ou catastrophes, naturelles ou autres, qui ont affectés un grand nombre de ménages dans votre communauté ? (Ex., sécheresses exceptionnelles, inondations, grandes pertes de récoltes et/ou d'animaux en raison de conditions météorologiques ou de maladie, insécurité) Type de catastrophes Quelles en sont les causes / raisons ? Effets sur les gens de cette communauté ? 1 2 3 15. Depuis 2011, quels types d’assistance externes qui concerne l’agriculture, l’élevage, la santé, la nutrition ou WASH ont été mises en œuvre dans la communauté et comment cela a-t-il fonctionné ? Note : REMPLISSEZ LA TABLE CI-DESSOUS AVEC LE DETAIL DE CHAQUE PROJET, SES ACTIVITES PRINCIPALES, SA PERIODE D’INTERVENTION ET L’ORGANIZATION EN CHARGE (GOUVERNEMENT, ONG, AUTRES) Assistance de Développement Externe Nom du projet et Organisation de Mise en Œuvre Services fournis et principales cibles Année de début du projet Année de fin du projet 1 YAAJEENDE FINAL IMPACT EVALUATION REPORT ANNEXES 1 4 8 2 3 4 5 6 7 8 COMMENTAIRES : YAAJEENDE FINAL IMPACT EVALUATION REPORT ANNEXES 1 4 9 ANONYMIZED LIST OF GDS CONDUCTED AT ENDLINE Date Yaajeende Treatment Status Group Interviewed Region Commune Village 26-May Comparison Group Female non-beneficiaries Kedougou Ethiolo Ekesse 26-May Comparison Group Female non-beneficiaries Matam Oudalaye Mbelogue 27-May Intervention Group Debbo Galle Members Kedougou Dar Salam Dar Salam 27-May Intervention Male beneficiaries Kedougou Dar Salam Dar Salam 27-May Comparison Group Female non-beneficiaries Matam Oudalaye Gasse Diabe 28-May Intervention Group Debbo Galle Members Kedougou Dar Salam Bilele 28-May Intervention Male beneficiaries Kedougou Dar Salam Bilele 28-May Comparison Group Female non-beneficiaries Matam Oudalaye Loumbol Samba Abdoul 29-May Comparison Group Female non-beneficiaries Kedougou Bandafassi Sylla Counda 29-May Intervention Group Debbo Galle Members Matam Ogo Thainconehieye 30-May Intervention Group Debbo Galle Members Kedougou Dakately Longueniane 30-May Intervention Other female beneficiaries Kedougou Dakately Longueniane 30-May Intervention Group Debbo Galle Members Matam Agnam Civol Agnam Yéroyabé 30-May Intervention Other female beneficiaries Matam Agnam Civol Agnam Yéroyabé 31-May Intervention Group Debbo Galle Members Kedougou Sabodala Faloumbou 31-May Intervention Group Debbo Galle Members Matam Thilogne Thilognetokossel 1-Jun Intervention Group Debbo Galle Members Kedougou Fongolembi Sodiore 1-Jun Intervention Male beneficiaries Kedougou Fongolembi Sodiore 1-Jun Intervention Male beneficiaries Matam Ogo Thainconehieye 2-Jun Intervention Male beneficiaries Matam Thilogne Thilognetokossel 3-Jun Intervention Other female beneficiaries Kedougou Sabodala Sabodala Niakafiri 3-Jun Intervention Group Debbo Galle Members Kedougou Sabodala Sabodala Niakafiri 4-Jun Comparison Group Female non-beneficiaries Kedougou Ethiolo Cote 4-Jun Intervention Group Debbo Galle Members Matam OuroSidy Foumikharadiobe 4-Jun Intervention Other female beneficiaries Matam OuroSidy Foumikharadiobe 5-Jun Intervention Group Debbo Galle Members Matam Orkadiere Sinthiou Polèle 5-Jun Intervention Male beneficiaries Matam Orkadiere Sinthiou Polèle 6-Jun Intervention Group Debbo Galle Members Tambacounda Belle Nayé 7-Jun Comparison Group Female non-beneficiaries Tambacounda Gathiary Sabou Cire Gathiary 8-Jun Intervention Group Debbo Galle Members Matam Bokiladji Bondjiwaly 8-Jun Intervention Other female beneficiaries Matam Bokiladji Bondjiwaly 8-Jun Comparison Group Female non-beneficiaries Tambacounda Madina Foulbe Samba Yayé 9-Jun Intervention Group Debbo Galle Members Tambacounda Balou Sébou 9-Jun Intervention Male beneficiaries Tambacounda Balou Sébou 10-Jun Intervention Male beneficiaries Tambacounda Belle Nayé 10-Jun Intervention Group Debbo Galle Members Tambacounda Sadatou Sinthiou Njimbé 10-Jun Intervention Male beneficiaries Tambacounda Sadatou Sinthiou Njimbé 11-Jun Intervention Group Debbo Galle Members Tambacounda Bélé Bélé YAAJEENDE FINAL IMPACT EVALUATION REPORT ANNEXES 1 5 0 Date Yaajeende Treatment Status Group Interviewed Region Commune Village 11-Jun Intervention Group Debbo Galle Members Tambacounda Gabou Samba Yidé 12-Jun Intervention Other female beneficiaries Tambacounda Gabou Samba Yidé 12-Jun Intervention Other female beneficiaries Tambacounda Sinthiou Fissa Yéri Malé ANONYMIZED LIST OF KIIS CONDUCTED AT ENDLINE Date of interview Party Interviewed Region Commune (if applicable) 26-May Agricultural Relais Kedougou Ethiolo 27-May APS Kedougou Dar Salam 27-May APS Kedougou Dar Salam 27-May GTC Kedougou Dar Salam 27-May VNC Kedougou Dar Salam 28-May APS/VNC/GTC Kedougou Dar Salam 28-May Village Relais Matam Oudalaye 29-May VNC / Relais Matam Ogo 30-May GTC/VNC/APS Kedougou Dar Salam 30-May GTC Kedougou Dakately 30-May APS Kedougou Dakately 30-May VNC Kedougou Dakately 30-May CLP Kedougou Dakately 30-May Livestock Relais/APS Kedougou Dakately 30-May Veterinary Agent & Water and Forest Agent Kedougou Dakately 31-May GTC Kedougou Sabodala 31-May VNC/APS Kedougou Sabodala 31-May Agricultural Relais/APS/GTC/CLP Kedougou Sabodala 31-May VNC Matam Thilogne 31-May Veterinary Agent Kedougou Sabodala 1-Jun APS/VNC Kedougou Fongolembi 1-Jun GTC Kedougou Fongolembi 1-Jun Veterinary Agent Kedougou Fongolembi 1-Jun Grafting Relais Kedougou 2-Jun GTC Matam Agnam Civol 2-Jun 3 GTC members Matam Thilogne 2-Jun APS Matam Thilogne 3-Jun APS/VNC Kedougou Sabodala 4-Jun Village Relais Kedougou Ethiolo 4-Jun GTC Matam OuroSidy 4-Jun APS Matam OuroSidy 4-Jun CLP Matam OuroSidy YAAJEENDE FINAL IMPACT EVALUATION REPORT ANNEXES 1 5 1 Date of interview Party Interviewed Region Commune (if applicable) 4-Jun APS Matam OuroSidy 4-Jun Point Focal Nutrition Kedougou n/a 4-Jun SDDR Coordinator Kedougou n/a 4-Jun P2RS Agent Kedougou n/a 4-Jun DRDR Matam n/a 4-Jun CLM Matam n/a 4-Jun Nutrition Focal Point Matam n/a 4-Jun Nutrition Focal Point Kedougou n/a 4-Jun SDDR Coordinator Kedougou n/a 4-Jun SDDR Coordinator Matam n/a 5-Jun APS Matam Orkadiere 5-Jun CLP Matam Orkadiere 5-Jun GTC Matam Orkadiere 5-Jun 1 Relais & 1 VNC Matam Orkadiere 6-Jun GTC Matam Orkadiere 6-Jun APS Matam Orkadiere 6-Jun VNC Tambacounda Belle 6-Jun APS Tambacounda Belle 6-Jun Relais Tambacounda Belle 7-Jun VNC / Relais Matam Bokiladji 7-Jun Village Relais Tambacounda Gathiary 9-Jun Nutrition Focal Point Tambacounda n/a 9-Jun Hygiene Focal Point Tambacounda n/a 9-Jun Village Relais Tambacounda Sadatou 9-Jun VNC / Relais Tambacounda Balou 10-Jun SDDR Coordinator Tambacounda n/a 10-Jun CLP Tambacounda Sadatou 10-Jun VNC / Relais Tambacounda Sadatou 10-Jun CLP Tambacounda Balou 11-Jun 2 GTC members Tambacounda Balou 11-Jun 3 GTC members Tambacounda Bélé 11-Jun VNC / Relais Tambacounda Gabou 11-Jun APS Tambacounda Gabou 11-Jun APS Tambacounda Balou 12-Jun GTC Tambacounda Gabou 12-Jun 3 GTC members Tambacounda Sinthiou Fissa 12-Jun APS Tambacounda Sinthiou Fissa 12-Jun CLP Tambacounda Sinthiou Fissa YAAJEENDE FINAL IMPACT EVALUATION REPORT ANNEXES 1 5 2 ANNEX IV – EVALUATION SOW YAAJEENDE FINAL EVALUATION STATEMENT OF WORK APRIL 17, 2017 This publication was produced for review by the United States Agency for International Development. It was prepared by Management Systems International, a Tetra Tech Company, for the USAID/Senegal Mission-Wide Monitoring and Evaluation Project. YAAJEENDE FINAL IMPACT EVALUATION REPORT ANNEXES 1 5 3 YAAJEENDE FINAL EVALUATION STATEMENT OF WORK Contracted under AID-685-C-15-00003 USAID Senegal Monitoring and Evaluation Project DISCLAIMER The author’s views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. YAAJEENDE FINAL PERFORMANCE EVALUATION STATEMENT OF WORK cliv ACRONYMS AMEP Activity Monitoring and Evaluation Plan ANSD Agence nationale de la statistique et de la démographie BFS Bureau for Food Security CAADP Comprehensive Africa Agriculture Development Program CBSP Community Based Solution Provider CLM Cellule de la lutte contre la malnutrition CNV Community Nutrition Volunteer COR Contracting Officer Representative CWG Citizen Working Group DID Difference-in-difference DGG Debbo Gallé Group DO Development Objective DPV Direction de la protection des végétaux EDR Evaluation Design Report EGO Economic Growth Team FGD Focus Group Discussion FIE Final Impact Evaluation FLO First Level Objectives FtF Feed the Future FRA Flood Recession Agriculture IE Impact Evaluation KII Key Informant Interviews MEP Monitoring and Evaluation Project MIE Midterm Impact Evaluation NLA Nutrition-Led Agriculture PE Performance Evaluation POC Point of contact PSM Propensity Score Matching USAID United States Agency for International Development VCS Village Steering Committee WASH Water, Sanitation and Health YAAJEENDE FINAL PERFORMANCE EVALUATION STATEMENT OF WORK 1 1. STATEMENT OF WORK DETAIL USAID SOW Manager Megan Kyles, EGO Activity COR/AOR Megan Kyles, EGO MEP SOW Manager Souleymane Barry Activity Title Yaajeende Activity Period November 1, 2010 – September 30, 2017 Award/Contract # AID-685-A-00-11-00002 Funding $49,799,066 Implementing Organization NCBA CLUSA Geographic Coverage National (Matam, Bakel Dept of Tambacounda, Kedougou, Kolda) Task Final Impact Evaluation Task Start and End Dates March 15-December 30, 2017 2. ACTIVITY DESCRIPTION The purpose of this study is to evaluate the impact that the Feed the Future Nutrition-led Agriculture Project for Food Security in Senegal (known as “Yaajeende”) has had on reducing malnutrition and poverty in its intervention area. NCBA CLUSA received a 5-year cooperative agreement in November 2010, and then a 2-year extension in September of 2015. To combat poverty and child malnutrition, Yaajeende was designed to accelerate the participation of the very poor in rural economic growth and improve the four dimensions of food security – availability, access, utilization, and stability. As one of the original programs of the Feed the Future Initiative, Yaajeende is predicated on the United Nation’s Rome Principles for Sustainable Global Food Security, and employs an innovative, country-led, integrated approach to tackle the underlying issues which hold back the very poor from becoming integral and active members of the rural, agricultural marketplace and accessing the foods they need for a diverse, healthy diet year round. Over the years, Yaajeende has expanded its operations, ultimately working in 790 villages across 49 municipalities (“collectivités locales” or “communes” in French) and 9 departments of Matam, Tambacounda, Kedougou, and Kolda. 3. DEVELOPMENT HYPOTHESIS AND IMPLEMENTATION The goal of Yaajeende is to accelerate the participation of the very poor in rural economic growth and to improve nutritional status (primarily stunting). Interventions under this project aim to (a) integrate the very poor into agricultural markets and the rural economy; (b) improve the nutritional status of women and children; and (c) increase household assets and income among those who are not or are unable to participate in rural economic activities. YAAJEENDE FINAL PERFORMANCE EVALUATION STATEMENT OF WORK 2 Yaajeende addresses endemic food insecurity with its Nutrition-Led Agriculture (NLA) approach, promoting actions that improve the quantity, quality, price, use, and governance of nutritious foods to respond to nutritional deficiencies in and beyond project areas. NLA is a dynamic food systems approach that strengthens local governance and private sector to solve the root causes of food insecurity. As women are drivers of healthy food production and consumption in the community, Yaajeende places special emphasis on ensuring that women can fully participate in each of its NLA programs (livestock, horticulture, resilient farming and bio-fortified crops, community-based service provision, empowerment and nutrition education, hygiene, and food security governance). In May/June 2015, a Midterm Impact Evaluation was conducted in 6 of the 9 departments. (The 3 departments located in Kolda Region were excluded as field operations only began there in 2014.) Prior to the extension, Yaajeende implemented a mixed approach, with some intervention areas receiving the full NLA approach, while others received only the nutrition or agriculture package. During the extension phase, Yaajeende focused on scaling up the full NLA approach in all of its villages of intervention. The validity of this management decision was later confirmed by the Midterm Impact Evaluation (MIE), which found that villages receiving the full package to be clearly better off. Therefore, the final evaluation will focus on how the full NLA approach impacted poverty and malnutrition after nearly seven years of implementation. DEVELOPMENT HYPOTHESIS: Food insecurity continues to be a serious problem throughout Senegal and the region, leaving large segments of the population vulnerable to famine and hunger. USAID Senegal is working with the Government of Senegal to promote greater agricultural productivity and improve regional food security. USAID’s development strategy is aligned within the Comprehensive Africa Agriculture Development Program (CAADP), an integrated framework implemented in West Africa under the auspices of ECOWAS, that promotes agricultural development as a critical means to eliminate hunger, reduce poverty and food insecurity, and increase trade. The Yaajeende Agricultural Development Project for Food Security of USAID Senegal falls under the Feed the Future Initiative and was designed to improve food security and incomes. This is the Goal of the project. It coincides with the Development Objective (DO) of the Mission’s Economic Growth program. Yaajeende addresses three of the four First Level Objectives (FLOs) of the Economic Growth Results Framework (presented on the second page following) via several intermediate results to be attained by the Program, which taken together are to contribute to the DO/Goal. These are: FLO 1 Inclusive Agriculture Sector Growth FLO 3 Improved Nutritional Status, especially of women and children FLO 4 Improved Management of Natural Resources YAAJEENDE FINAL PERFORMANCE EVALUATION STATEMENT OF WORK 3 FLO 2, Increased trade, is not directly targeted by Yaajeende, which is focused on poor, food-deficit areas. Additional production is expected to be auto-consumed or marketed within Senegal. The Program is designed based on an understanding that food security is made up of three elements— availability, access and utilization. The development hypothesis of Yaajeende is that increased crop and livestock production leads directly to greater availability for both producers and consumers and it leads to improved access for producers. Increased marketing and marketing services lead to greater access for both producers and consumers as products demanded are delivered in the time, place and quality needed and as better prices develop (via reduced marketing margins) for both producers and consumers. Also, the higher incomes among the poorest of producers improve access and reduce vulnerability to food insecurity. Utilization, which examines the nutritive value of consumption, depends both on consumption of an adequate and diversified diet and the health status of the population. Waterborne diseases are particularly detrimental to absorption of nutrients and they are unfortunately very common and often serious. Yaajeende addresses this serious constraint through its potable water, sanitation and hygiene component. Finally, by establishing an effective institutional environment capable of addressing food insecurity, availability, access and utilization of food may be improved on a sustainable basis. Together, the increased production and marketing are expected to lead to improved incomes and greater availability and access to food, while the improved hygiene leads further to improved nutrition. In short, the population will be less food insecure. For sustainability, Yaajeende’s approach is to assist the very poor and vulnerable to obtain productive assets such as irrigation facilities or livestock that will “Feed the Future”. CRITICAL ASSUMPTIONS The Yaajeende approach is based on a few assumptions which are critical for the program to achieve its targets and objectives. One is that poor farmers/livestock producers are willing to adopt technologies that are new to them. While Yaajeende intends to buy down some of the risk of new technologies and also to provide technical assistance and training, the project will not fully subsidize any activity. The Project does not have the resources to fully subsidize the large target population nor is it inclined to do so. It is well￾established that subsidies are not sustainable, they do not allow scaling up, and they do not encourage ownership by the clients. For irrigation, potable water and livestock watering schemes, the project assumes the ground and surface water regimen will remain relatively stable or at worst trend gradually toward greater scarcity. Therefore, while some allowance will be made for climate change in the planning and design of schemes, true effects of global warming on limited water supplies in semi-arid environments are not yet known and they could be of a quantum character in localized areas rather than being gradual. The consequence could be that investments would not yield the expected results. Specific intervention(s) to be examined and the effects/outcome indicator changes those interventions are expected to produce are shown in the tables below along with actual achievements where available. [Results Framework in separate attachment] YAAJEENDE FINAL PERFORMANCE EVALUATION STATEMENT OF WORK 4 USAID|YAAJEENDE INDICATORS FOR FIE AND THEIR TARGETS The tables below presents an initial list of USAID’s priority outcome, output and impact indicators for the Yaajeende project. The Evaluation Team will review and finalize the list of project indicators to be measured and evaluated at endline, in discussion with USAID, during the endline evaluation design phase. This determination will be informed by the baseline and midline impact evaluation (MIE) results and sampling constraints, and will also include consideration of the inclusion of additional indicators proposed by USAID for endline. Impact Indicators 52 Data for Impact Indicators 1, 2, and 8, are from the July 2016 draft of the “USAID Yaajeende Program Midterm Impact Evaluation”, Table 14, pp. 56-57 (completed by the International Development Group for USAID). Type Ind. No. per PMP Project Indicator Subset Ref No. Five-Year Cumulative Result 2010-2015 Seven-Year Cumulative Target 2010-2017 Impact 1 Prevalence of stunted children under five years of age (Reduction) 6-59 months of age 3.1.9-11 FTF 30% Reduction (from 23% to 16%)52 20% (from 23% to 18.4%) Impact 2 Prevalence of underweight children under five years of age (Reduction) 6-59 months of age 3.1.9-16 FTF No Significant Difference* 25% (from 23% to 17.25%) Impact 3 Prevalence of wasted children under five years of age 6-59 months of age 3.1.9-12 FTF Track only (15-17%) Track only Impact 4 Percentage of households that have increased income by 10% or more custom New Indicator 80% Impact 5 Percentage of households that have diversified their income without a decline in overall income custom New Indicator 80% Impact 6 Percentage of households that have increased dietary diversity score custom New Indicator 80% YAAJEENDE FINAL PERFORMANCE EVALUATION STATEMENT OF WORK 5 Outcome and Output Indicators Impact 7 Percentage of direct female beneficiaries of nutrition-sensitive agricultural activities consuming a diet of minimum diversity custom New Indicator 80% Impact 8 Prevalence of children 6-23 months receiving a Minimum Acceptable Diet (Increase) 3.1.9.1(1)/ RIA FTF 53% reduction (from 13% to 6%)* 40% Increase (from 13% to 18.2%) Impact Underweight females 15-49 years of age Base period PMP indicator Track only 13% reduction (from 30% to 26%) Impact Months of lean season (soudure) in months Base period PMP indicator Track only Indeterminate result at MIE (base period target was 30% reduction from 2.7) Impact Poverty rate Not an indicator in either PMP Track only 6% reduction (from 35% to 33%) Impact Agriculture revenue (including crops, livestock, poultry) Base period PMP indicator with revision Track only Type Ind. No. per PMP Project Indicator Ref No. Five-Year Cumulative Result (2010-2015) Extensio n Target PY6 Extensio n Target PY7 Seven-Year Cumulative Target (2010-2017) Achieved through Sep 2016 (PY6) Outcome 3 Percentage of households supported by USAID|Yaajeende purchasing commercial inputs or services through service providers (CBSPs) custom New Indicator 50% 60% 60% Not yet assessed. Outcome 4 Percentage of people with increased production of targeted commodities custom New Indicator 60% 70% 70% Not yet assessed. YAAJEENDE FINAL PERFORMANCE EVALUATION STATEMENT OF WORK 6 IMPLEMENTATION: Yaajeende’s NLA Approach: NLA is a structural “Food System” approach to food security that promotes the emergence of an agricultural sector that focuses on the sustainable production, broad distribution, lucrative trade, informed consumption, and transparent governance of high quality, nutritious foods that have the ability to resolve nutritional deficiencies. Like a value chain, a food system is a systems-based approach that entails conducting well-coordinated actions across multiple sectors with a diverse range of actors in order to ensure sustainable progress in 53 Historical data from project records was used to generate the figure for this new indicator. 54 This cumulative figure includes double counting as some individuals were reached in more than one reporting year. Output 6 Number of households with access to a home or community garden custom 11,05053 New Indicator 10,000 5,000 26,050 22 518 Outcome 7 Number of women/care givers/community members reached with nutrition behaviors change messages custom New Indicator 50,000 80,000 130,00054 88 153 Outcome 8 Value of incremental sales collected at farm￾level attributed to FTF 4.5.2(23)/ RIA FTF $7,588,832 (of $12,170,197 in total sales) $2,100,000 $900,000 $10,590,000 TBD Output 9 Number of rural households benefiting directly from USG interventions 4.5.2(13)/S FTF 98,533 30,000 20,000 148,500 131,592 Outcome Salt iodization and storage Base period PMP indicator Not reported (target was 30% increase in HH use) Outcome Exclusive maternal breastfeeding Track only – Not an indicator in either PMP Outcome Handwashing station in common use Base period PMP indicator Not reported (target was 30% of HH) YAAJEENDE FINAL PERFORMANCE EVALUATION STATEMENT OF WORK 7 any one area. NLA is a food system based on the “four pillars” of food security: Availability, Access, Utilization, and Governance. A Food System is not only attuned to market dynamics and economic priorities. It encompasses governance issues, health issues, market issues, behavior change issues, demand creation issues, and thus is typically broader than value chains by integrating government, civil society, and public health actors as well as private sector actors and nonprofit actors. Food systems, unlike value chains, are “directed” systems, meaning that they are not focused only on fulfilling the demands of large market actors, but also are concerned with the social and physical needs of the general population as well as those who are economically and nutritionally vulnerable. Below is a schematic of Yaajeende’s Nutrition Led Agriculture food systems approach: Pillar 1 of the NLA food system represents a menu of appropriate technologies provided by private sector firms as well as by leading research institutes such as ICRISAT, ISRA, IFPRI/HarvestPLUS, Ainoma, AfricaRICE, etc. that is offered to communities in the Zone of Intervention (ZOI). These technologies are selected because they increase the AVAILABILTY of products that improve nutrition, increase wealth, or enhance resilience. Pillar 2 is composed of the various local private sector “distribution” networks that provide farmers and vulnerable people with ACCESS to the technologies in Pillar 1. The project utilizes three main types of local businesses to distribute technologies and knowledge to farmers: 1) producer organizations and farmer groups; 2) Community Based Solution Providers that provide both agricultural inputs and services; a subset of CBSPs called CBSP/Nutrition that focuses efforts on providing nutritional hygiene and sanitation related products; 3) nutrition oriented enterprises: local group-owned businesses that produce products such as compost, enriched flours, fonio, baobab powder and other transformed or fortified foods Conservation Agriculture Market Gardens Iodized Salt Improved Breed Livestock Enriched Flours Improved Seeds Bio-fortified Crops Arboriculture BDL Micro-gardens Moringa Gardens DUP Livestock Products Compost Equipment Pillar 1: Availability (Technologies): Research Institutions /Firms G Firms/Community Based Solution Providers (CBSPs) Nutrition Enterprises (Processing, Storage, Fortification…etc. ) CNV/CBSP Pillar 2: Access Businesses that provide Products/Services/Knowledge Producer Groups/Agricultural Cooperatives Pillar 4: Local Governance/Civil Society: (Leadership, Coordination, Oversight) Pillar 3: Utilization Behavior Change/ Education/Training/Demand Creation through Training and Communications MTM CR, Citizen Working Group, LSC USAID|Yaajeende YAAJEENDE FINAL PERFORMANCE EVALUATION STATEMENT OF WORK 8 for markets. Pillar 3 represents the activities conducted with the general population in order to educate the public about the need for changes in key behaviors around key nutrition actions, agricultural production, food preparation, consumption, sanitation and hygiene and potable water use that thereby enhances the public’s UTILIZATION of food and water resources and increases the public demand for improved technologies. These activities center on conducting Behavior change communications and social marketing to increase awareness and build demand for the technologies in Pillar 1 (provided by Pillar 2), as well as organizing “Debbo Gallé” (Excellent Mothers) Groups which facilitate more intensive and comprehensive collaborations with the project’s core target populations (women of reproductive age and children under 5). Pillar 4 centers on local governments and civil society--notably the municipal-level (“collectivité locale”) Citizen Working Groups (CWG) and their constituent Village Steering Committees (VSC) that coordinate and plan food security initiatives with local governments and harmonize the actions of the other institutions within the food system, including the CBSPs and the Debbo Gallé Groups. Project staff mirror these local actors to build up their capacity to plan, monitor, implement, advocate and evaluate local initiatives. These governance institutions represent the “head” of the Food System by observing, analyzing, evaluating and coordinating action of the CBSPs, Debbo Gallé Groups (DGGs) and the Community Nutrition Volunteers (CNV) who represent the “hands”—or implementation institutions-- of the NLA system. Citizen Working Group for Food Security (CWG) Local Government (Conseil Rural) Local Development Plans and Budgets Local Steering Committee Local Steering Committee Local Steering Committee Local Steering Committee Research & Private Sector Providing Technologies, Products, Services and Knowledge Utilization Availability Governance Access Public Health Agencies and NGOs helping to Educate, inform, Build Skills and Change Behaviors Community Based Solution Providers & Producer Organizations Community Based Solution Providers & Community Nutrition Volunteers Regional Government (Conseil Régional) Communauté Rurale (CR) Level Village Level Local Food Security Plans Food Security Pillars 1 & 2: Availability and Access Food Security Pillar 3: Utilization Food Security Pillar 4: Governance Nutrition Led Agriculture: A Food Systems Approach to Food Security and Nutrition YAAJEENDE FINAL PERFORMANCE EVALUATION STATEMENT OF WORK 9 THE NUTRITION LED AGRICULTURE PROCESS Because NLA is a structural response to address intractable issues of food security and malnutrition, the approach unfolds in a series of carefully conceived steps: 1) Selecting the Target Zones of Intervention (ZOI) USAID Yaajeende’s Zone of Intervention for Phase 2 includes 49 communes across Matam, Kedougou, and Kolda regions as well as the Departments of Bakel (Tambacounda). The communes have been selected by weighing two main criteria: 1) the incidence of malnutrition as evidenced in Severe Malnutrition (Wasting) or Chronic Malnutrition (Stunting); 2) the presence of sufficient resources in terms of water, arable land, and human resources to build or reinforce local institutions (businesses, civil society, governance, markets) that can resolve these issues over the long term. 2) Prioritizing Target Populations Core Target: USAID Yaajeende considers its “core target” populations to be women of reproductive age (ages 15-49) and their children under 5 years old. Not only is this population particularly vulnerable when food crisis occur, but women are the primary producers of micronutrient rich foods (vegetables and fruits), the main purchasers and preparers of food, and the main providers of care to children; thus women are the center of food security and the project seeks to resolve their issues of availability, access, utilization, and voice in governance in a comprehensive and lasting fashion. Smallholders: These are producers that are undertaking primarily subsistence agriculture activities on around 1 hectare of land and who can benefit from an increased professionalization of their activities to intensify yields, or expand and diversify their production or to begin to market a surplus part of their production. They can benefit from a modest infusion of credit and improved access to technologies and services. Commercial Producers: These are producers who are already undertaking commercial farming activities on a larger scale than the mid-level producers. Generally they are literate or partially literate and have basic skills in running a larger scale operation and work 5-10 hectares of land. Typically they employ other laborers on their farms. They demonstrate a commercial mindset and increased sophistication in their approach to agriculture; they are ready to adopt more innovative technologies and can benefit greatly from an infusion of credit and from an integrated approach to agricultural production. They are key to producing for local markets. 3) Targeting Specific Nutritional Deficiencies that Impact Health In Senegal, interventions focus on two major types of malnutrition: 1) Acute malnutrition: to resolve this kind of malnutrition the project is working to improve/diversify macronutrient production including carbohydrates through cereals and proteins through legumes and meat and lipids through animal products (dairy, meat). [It is important to note, however, that Yaajeende does not have a performance target for wasting, but rather tracks this data only.] 2) Chronic malnutrition: Stunting (height-for-age value to be less than two standard deviations of the WHO Child Growth Standards median) among children results from long-term, chronic micronutrient and macronutrient deficiencies that have permanent effects on health as well as on cognitive and physical development. Thus, the project has targeted the following five YAAJEENDE FINAL PERFORMANCE EVALUATION STATEMENT OF WORK 10 micronutrient deficiencies to resolve within the target zones which are at the root of many prolific health concerns. Vitamin A: Poor ocular health and a low resistance to diseases constitute the consequences of Vitamin A deficiency. An untreated chronic Vitamin A deficiency can lead to a complete loss of sight and even death. Eggs, milk, butter, fish and liver constitute the best sources of Vitamin A. However, palm oil, orange flesh sweet potatoes (OFSP), tomatoes, mangos, papayas, yellow corn, Shea butter, and oils constitute excellent sources of beta-carotene that the body can easily convert into Vitamin A. USAID Yaajeende, in addition to its “Pass on the Gift” strategy, focuses on the annual production of foods rich in Vitamin A/beta-carotene through market and micro-gardens and household arboriculture production of Mango, Papaya, Moringa, and Pomme du Sahel. Iron: The consequences of anemia due to iron deficiency are in many cases serious: high risk of morbidity and mortality of the fetus, premature birth and low-weight at birth (caused by underweight and growth delays), disturbance of mental and physical development, often irreversible, in infants and young children; less resistance to infections, fatigue and reduction in physical capacities in older children and adults. Meat, liver, offal, milk and egg are among animal products that are rich in iron. Legumes (cowpea, lentils, pea), and fresh and dried green leaves (moringa, cassava leaves, sweet potato leaves) are among vegetable products rich in iron. USAID Yaajeende, through its agricultural production activities will direct crops towards those rich in iron and Vitamin C such as orange, lemon and grapefruits to improve iron absorption. Vitamin C: Vitamin C is essential to aid the absorption of Iron and other micronutrients. It is critical for maintaining good health and a strong immune system. Foods high in Vitamin C include: Madd, Citrus, Jujube, Moringa, Mango and Pomme du Sahel. Zinc: Zinc is found in a large variety of food. Red meat and poultry make up the majority of zinc in a diet. There are other good sources of zinc such as beans, lentils and dairy products. However, the absorption of zinc is more important in a diet that includes many animal proteins. Delayed growth is the best known consequence and the most easily measured negative consequence associated to zinc deficiency found in populations. The other signs of a zinc deficiency are a severity and a higher frequency of diarrhea and a weakening of the immune system, leaving children more prone to disease. Thus, USAID Yaajeende promotes the consumption of animal products (poultry, eggs, milk) made available and accessible through “Pass on the Gift” Livestock placements. Iodine: Iodine is essential to the development of the brain of the fetus. Indeed, the results of various studies show that approximately 3% of infants born from mothers who are deprived of iodine during the pregnancy suffer from cretinism, 10% show a severe mental retardation and 87% an intellectual deficiency. In other words, all children born from mothers presenting an iodine deficiency will suffer from the repercussions of this deficiency in an irreversible way. In the communities where the iodine deficiency is endemic, the intelligence quotient is reduced on average by 13.5 points. In addition to the effects on the development of the brain, iodine deficiency could be the source of low weight at birth (one of the causes of chronic malnutrition) and miscarriages. In partnership with UNICEF and the Senegalese Cellule de Lutte Contre la Malnutrition (CLM), USAID Yaajeende, through its CBSP network ensures a better availability and access to iodized salt in areas of intervention. Households are made aware on the importance of iodine on the cognitive development of children and the risks of miscarriage during pregnancy. Furthermore, YAAJEENDE FINAL PERFORMANCE EVALUATION STATEMENT OF WORK 11 households are informed on how to store and conserve the iodized salt in order to better preserve its qualities. To date, this storage is an ongoing challenge the project will work to better address in year 7. In addition, the project works to bolster the availability and access to key macro-nutrients such as Carbohydrates (Cereals), Lipids (Animal and Vegetables) and Proteins (Animal and Vegetable) to offset malnutrition in mothers and children who are undernourished. 4) Conducting Operational Research: Food Resource Inventories, Nutritional Profiles, Key Behavior Barrier Analyses, Quantifying Local Production Targets and Nutrition Led Agricultural Calendars. After targeting specific macronutrient and micronutrient deficiencies, it is important to take a census of the locally available food resources –including both domesticated crops and wild foods found in the surrounding countryside--that can adequately resolve these priority deficiencies. A nutritional profile is created for each food to discern its actual macronutrient/micronutrient content to understand the food’s potential contribution to improved local diets. Simultaneously, an analysis of dietary behaviors, a register of local recipes and an analysis of consumption patterns is created to help managers arrive at a list of priority crops that can be effectively promoted taking into account both market demand as well as potential contribution to resolving the key deficiencies. Studies of Behaviors around the Key Nutrition Actions as well as Barrier Analyses to Behavior Changes are created at the outset to help craft communications and behavior change strategies. Once the nutritional profiles of the different foods are available, it is possible to approximate the needs of each individual community based on the kinds of foods available locally and their contribution to the daily recommended allowance of a specific deficiency and production. Using simple tools, communities can fix production targets for each crop based on the particular needs of a community. These production targets also help inform a year round “NLA Calendar” developed for each commune and disseminated in each village that tracks which foods are available and which should to be grown or purchased during each month of the year and helps identify gaps to ensure that all major deficiencies are covered year round either through production activities, food conservation/stocking, or market based activities (ie. purchases). 5) Mapping Severe Malnutrition To better target its resources, the project is using data provided by the Government of Senegal to map severe malnutrition (MAS) cases on mapping software. The data is collected by the GoS health posts and the maps are created by the Community Nutrition Volunteers using GPS units. The maps are accurate down to the household level and can help the project identify “hot spots” of malnutrition so that causality can be established. In addition, the maps can be used to identify and observe changes in larger commune and region wide trends. 6) Implementing Integrated Packages to Respond to the Priority Deficiencies Within the project’s design, eight key categories of activities lead to a measurable and sustained impact on the nutritional status of a population. Yet, certain “nutrition specific” activities are essential and prioritized for quick impacts and while others bolster these nutrition specific activities and ensure their long-term sustainability and resilience. YAAJEENDE FINAL PERFORMANCE EVALUATION STATEMENT OF WORK 12 PACKAGE A: NUTRITION PACKAGE This package of “nutrition specific” activities is essential for any change in nutrition and without it, impacts on malnutrition are unlikely. These activities are foundational. The implementation of Package A results in quick impacts and is primarily directed at the core target demographics through the “Debbo Gallé” institutions. 1) Key Behavior Changes: 13 Essential Nutrition Actions, Community Meals, Grandmother activities, Social Marketing, Nutrition Caravans. 2) Access to Fortified Foods: Locally made Enriched Flours, Iodized Salt sales, Bio-fortified Crop introduction, Wild Foods Education, and Household fortification 3) Access to Clean Potable Water and Hygienic Conditions: WASH, Potable Water, CLTS PACKAGE B: AGRICULTURE PRODUCTION PACKAGE While implementation of Package A provides quick impacts, those impacts are more likely to be sustained over the medium and long term through implementation of the Agricultural Production package. The Agriculture Package improves the diversity and quality of food production within the communities that, in turn, impacts health. Although slower in achieving impact and more indirect in its effects on health than the Nutrition Package, outputs from these agricultural activities feed into the nutrition specific interventions in the core nutrition package (such as inputs for locally created enriched flours) and make the key changes in behavior possible. 4) Improved Sustainable Production of Energy Dense Cereals (Carbohydrates): Flood Recession Agriculture, Improved Varieties, Bio-fortified cereals, Conservation Agriculture. 5) Improved Sustainable Production of Micronutrient Rich Vegetables/Fruits: Market and Micro Gardening, Arboriculture, Bio-fortified Vegetables and Legumes. 6) Improved Animal Assets and Access to Animal Protein and Lipids: Livestock Placements and Passing; Aviculture, Improved breeds to maximize milk, meat and egg production. PACKAGE C: SUSTAINABILITY AND RESILIENCE PACKAGE This package ensures increased market access to commercial products and services that improve the overall food and water supply and improves the enabling environment, maximizing the use of food/water resources and the increasing access to resources and assets. Governance improves anticipation of shocks and provides a mechanism to plan, coordinate and evaluate commune level interventions. Implementation of this package increases long-term sustainability, improves resistance to shocks, provides options in the event of crises and builds the capacity of local institutions to carry out food security interventions in the future. 7) Improved Dynamic Private Sector: Community Based Solution Providers in both Agriculture and Nutrition, Irrigation Services, Nutrition oriented group enterprises, Livestock Enterprises, and enhanced Producer Organizations. YAAJEENDE FINAL PERFORMANCE EVALUATION STATEMENT OF WORK 13 8) Improved Local Governance: Strengthened Local Government Capacity, Dynamic Civil Society, Equitable Land Tenure and Local capacity to coordinate food security actions. 7. Achieving Scale through the NLA Approach Achieving the scale we seek necessitates a strategy that is predicated on the following tactics: 1) Decentralized implementation through local institutions and people: the project now pushes the majority of the project implementation out to the local institutions including Local Governments, Civil Society Groups (CWGs/VSCs), Community Based Solution Providers (CBSPs), Community Nutrition Volunteers, Producer Groups, local NGOs. Each of the 49 communes in the ZOI will have its own annual strategy, work plan and budget that will be created and monitored by its CWGs and implemented by its Local Resource Persons (LRP). Project staff focus on coordinating these local people and institutions, building up their capacities, and helping them in monitoring the indicators of their success. 2) Leveraging partnerships: achieving desired scale will surpass the ability of the project to finance all activities. The project has developed strategies for each of its seven programs on how to leverage funds from the Government of Senegal, other bilateral and multilateral partners, and financial services to fund broader implementation and increase the impact within project sites. As the project has matured, it is increasingly engaging with other major actors including UNICEF, the World Food Program and the Food and Agriculture Organization as well as smaller NGOs such as Action Against Hunger, and Helen Keller International. USAID Yaajeende is orienting these actors towards promising sites and helping them connect to the local institutions (Debbo Gallés, CBSPs, CWGs) to facilitate collaborations at the commune level. Finally, the project will continue its established collaborations with other USAID interventions including the Naatal Mbay, and ERA, along with newer projects, like the NLA adaptation, USAID SPRING, the WASH work of ACCES, and the unifying decentralized governance project, GOLD. 3) Graduation of communes: After 6 years, there are communes in each major zone that are ready to take matters into their own hands. The project has developed a graduation strategy for communes whose primary institutions demonstrate a level of capacity needed to continue under its own leadership. At project close out, a matrix of where each commune stands in terms of its capacities, strengths and challenges will be developed for tracking and follow up. 4) Institutionalization of USAID Yaajeende Programs into Senegalese Government Policies and Programs: USAID Yaajeende has been approached by several government agencies that have articulated the desire to institutionalize USAID Yaajeende programs into their ongoing activities. The project is working with the Cellule de Lutte Contre la Malnutrition (CLM) on biofortification strategies and is currently working on a protocol to conduct activities with Debbo Gallé groups. The Ministry of Livestock and the Delegation General de la Protection Social et la Solidarite Nationale (DGPS-SN) are working on institutionalizing the Livestock Placements program throughout Senegal, along with the Association of Mayors of Senegal. The Direction de la Protection des Végétaux (DPV) and the Ministry of Agriculture are working with the project to improve and greatly expand Flood Recession Agriculture (FRA) along the Senegal River basin. YAAJEENDE FINAL PERFORMANCE EVALUATION STATEMENT OF WORK 14 4. EXISTING PERFORMANCE INFORMATION SOURCES USAID will provide the endline Evaluation Team with a package of Activity background materials, including: • Cooperative Agreement and modifications. • Activity quarterly reports, work plans, Activity Monitoring and Evaluation Plans (AMEPs) and field visit reports; • Raw and cleaned baseline and midline datasets, including all household and individual (if applicable) identifiers, and codebooks; • Final survey instruments (in all languages available) and sampling protocols used for the baseline survey and the Midterm Impact Evaluation (MIE); • Data cleaning and analysis scripts or do-files used for baseline and midline variable construction and analyses, if available; and • Any relevant studies or background reports used to support the activity. All background documents will be provided to the MEP Team at the start of the evaluation design phase, and within five days of SOW approval. 5. TASK PURPOSE, AUDIENCE, AND INTENDED USES Purpose USAID is commissioning this evaluation to conduct a mixed methods Final Impact Evaluation (FIE) of the $50-million, 7-year Yaajeende program, a component of USAID/Senegal’s Feed the Future program. The FIE will provide USAID with an evidence base on the impacts of the “Nutrition Led Agriculture” (NLA) approach utilized by the program, on key program objectives, including reduced poverty and malnutrition. The FIE is anticipated to consist of a quasi-experimental impact evaluation, which employs a valid counterfactual, to estimate the impact of the Yaajeende program on a set of outcomes that are of interest to USAID. The FIE will also include calculation of several project performance indicators, which will provide data on the level of achievement of these indicators and complement the analysis of individual or household level impacts of Yaajeende that are observed through the endline impact analysis. The FIE will apply USAID’s Evaluation Policy guidance on using the most rigorous evaluation design and methods possible to identify impacts, establish attribution to program activities, and demonstrate accountability for achieving results. The FIE is also designed to capture practical lessons from USAID’s experience using the NLA to achieve key program objectives. The FIE will follow-up on key findings and questions identified during the MIE conducted in 2015 and will particularly aim to provide corresponding qualitative evidence to explain how and why observed impacts on program indicators were achieved, and drivers of variation in results across key sub-groups of interest. Given potential limitations to the validity of the quasi-experimental IE approach that were noted in the Yaajeende MIE, the FIE will take a phased approach. The initial stage of the FIE, Phase I, will consist of an endline evaluation design phase to recommend the most feasible approach for rigorous evaluation at endline, based on the evaluation team’s review of the baseline and midline data, and constraints on sample size and statistical power. The Evaluation Design Report produced during the evaluation design phase, as the final deliverable for Phase I, will present the available options and the details of each YAAJEENDE FINAL PERFORMANCE EVALUATION STATEMENT OF WORK 15 approach, and make a recommendation on the most feasible and rigorous approach to adopt for the endline evaluation, given existing constraints. Regardless of whether a performance or impact evaluation is recommended, it is anticipated that both quantitative and qualitative data will be collected and analyzed for the endline evaluation. If the evaluability assessment conducted during the evaluation design phase in Phase I suggests that a quasi-experimental impact evaluation approach will be infeasible to maintain for the endline, it is anticipated that it will still be possible to move forward with a mixed methods performance evaluation that undertakes statistical analysis across the baseline, midline and endline data. Phase II will be conducted pending USAID approval, noting that the phase II budget will be revised based on the evaluation option that USAID chooses. USAID may also choose not to continue with Phase II, pending consideration of the evaluation options and associated budgets proposed in the Evaluation Design Report produced at the end of Phase I. Audiences The evaluation will likely be of interest to several audiences. The findings are expected to have accountability and learning value to USAID, including the Senegal Mission and USAID/FtF. Findings are also likely to be of interest to the Government of Senegal, implementing partners, and other agencies, donors and practitioners active in nutrition, health, agriculture and integrated sectors. Primary audiences for this evaluation are USAID, program implementing partners, and government agencies involved in Yaajeende. Secondary audiences include other nongovernmental organizations, government agencies and the broader donor community in Senegal and globally who are involved in nutrition, health and poverty reduction sectors. Intended Uses USAID will use this evaluation to inform the design and programming decision-making of future activities that aim to use integrated agricultural, health and nutrition synergies to improve poverty alleviation and malnutrition. 6. TASK REQUIREMENT Illustrative evaluation questions are presented below and based on those proposed by USAID. The evaluation team will further refine these evaluation questions during the evaluation design phase and in discussion with USAID, pending review of baseline and midline data and evaluation options and constraints. The final proposed evaluation questions will be included in the Evaluation Design Report produced at the culmination of the endline evaluation design phase (Phase I). 8. What are the impacts of the full NLA package on the prevalence of poverty and malnutrition, 6 years after the start of program implementation? Impacts will be examined across several categories of outcomes, such as: nutritional status, dietary and household practices, agricultural practices, and household economic wellbeing. Outcome categories and specific outcome variables to be assessed within each category will be finalized at the Evaluation Design Report phase. 9. What were the major factors or processes that contributed to observed impacts? For example, what role and importance did the Yaajeende-supported local institutions (CBSPs, CWG/VSCs, DGGs) have in producing the observed impacts? 10. How do program impacts on poverty and malnutrition differ by key sub-groups of interest? The key sub-groups of interest for this evaluation are: a. Northern region (Matam/Bakel) v. Southern region (Kedougou) b. Households below the poverty line c. Women 11. What are potential explanatory reasons for variations in key outcomes across sub-groups? YAAJEENDE FINAL PERFORMANCE EVALUATION STATEMENT OF WORK 16 12. How do key individual, household, and village characteristics shape program impacts? Examples of village, household, and individual factors that could be assessed quantitatively as drivers of outcome heterogeneity include: a. Village: Distance from major market, distance from health clinic. b. Household: Family size, maximum level of education, age of household head. c. Individual: Mother age at time of first birth, DGG participation. 13. Do any patterns emerge through more targeted follow-up analyses, related to household characteristics for successful poverty and malnutrition reduction? 14. Were there any unintended broader consequences (positive and negative) of the intervention, beyond those related to project objectives? The FIE will aim to use a mixed methods approach to answering evaluation questions. While specific methods details will be specified in the Evaluation Design Report, pending the evaluation team’s review of the baseline and midline survey instruments and data, the following broad approaches are envisioned for each of the evaluation questions outlined above: Evaluation Question Broad Focus Qualitative or Quantitative Approach 1 Program impacts Quantitative 2 Program processes contributing to observed impacts Qualitative 3 Program impact variation by key sub-groups Quantitative 4 Explanatory reasons for outcome variation by sub-groups Qualitative 5 Individual, household and village drivers of program impacts Quantitative 6 Exploratory follow-up analyses of patterns related to poverty and malnutrition Quantitative 7 Unintended or broader consequences Qualitative (potentially supported by quantitative) 7. GENDER DISAGGREGATION AND GENDER DIFFERENTIAL EFFECTS In line with USAID’s Gender Equality and Female Empowerment Policy and Automated Directives System 203.3.1.5, the evaluation will consider gender-specific and differential effects of the Yaajeende program on the outcomes covered by this FIE, where it is feasible to do so. The endline surveys and qualitative data collection will be structured to enable gender-disaggregated analysis on key outcomes, where possible, and to identify gender differences with respect to program access and outcomes, as well as lessons learned from female community members. Qualitative data collection and analysis will also devote particular attention to investigating differential impacts by gender. In addition, the evaluation team will conduct further inquiry on gender themes as they emerge during data analysis. A detailed description of the anticipated gender disaggregated analyses, specified by outcome, will be included in the Evaluation Design Report produced during the endline evaluation design phase. 8. APPROACH 1. Design YAAJEENDE FINAL PERFORMANCE EVALUATION STATEMENT OF WORK 17 The MEP Team is required to conduct a Final Impact Evaluation of the Activity entitled “Yaajeende”. The main source of data for this evaluation will be collected from the main stakeholders in the Activity - - mainly households in treatment villages; direct beneficiaries and their families (those who participated in trainings or are members of the Mother-to-Mother groups); service providers (Large Producers, Community-Based Service Providers, Community Nutrition Volunteers, Veterinarians, Citizen Working Groups, and Emerging Breeders); implementing partner staff; and civil society leaders, traditional leaders and local authorities. While the final description of data collection methods, and draft instruments, will be specified in the Evaluation Design Report, it is anticipated that the endline data collection will include both a quantitative household survey, and qualitative data collection. The FIE anticipates using a mixed-methods quasi-experimental difference-in-difference (DID) design, drawing on quantitative household survey data, and qualitative data to complement the findings of the quantitative research. The DID approach will be coupled with statistical matching, to improve the precision of the impact estimates. Incorporating qualitative analysis will play an important role, by enabling the evaluation team to understand the “how” and “why” questions that can provide important context and an explanation for quantitative findings. In addition, qualitative analysis enables examination of outcomes and impacts of interest that are not amenable to quantitative analysis, as well as to provide illustrative cases and anecdotes that provide breadth of understanding for quantitative results. The mixed-methods approach relies on multiple sources of quantitative and qualitative data to triangulate information, ensure robust findings, and provide sound recommendations. The FIE will (1) measure impacts attributable to the project (2) identify reasons for observed impacts and any observed differences in impacts across different sub-groups of interest; and (3) answer final evaluation questions. The final impact evaluation will inform on: impacts that are attributable to Yaajeende; how impacts may vary across different types of beneficiaries, and the reasons why; and broader learning for effective programming. The goal of an IE is to generate objective, scientifically valid evidence of the causal impact of an intervention. The central methodological consideration for an IE is its approach to establishing causality. The challenge in this regard arises because for most interventions, the outcomes of interest are affected by a range of factors in addition to the intervention itself. It is not sufficient for the evaluation to simply measure changes in outcomes for beneficiaries. To be credible, the impact evaluation must be able to establish the extent to which the observed changes are due to the effects induced by the Yaajeende program activities, as opposed to other factors, over the evaluation timeframe. To represent what would have happened to program beneficiaries, had they not participated in the program, IEs use a control (or “comparison”) group to represent the counterfactual, i.e., the hypothetical outcomes for the beneficiaries in the absence of the activity. Per the USAID’s Evaluation Policy, the use of a counterfactual is the defining feature of an IE that distinguishes it from a performance evaluation. An important methodological consideration for IEs is the approach to selecting the comparison group. For this final evaluation, it is noted that the comparison group sample was established at baseline, and therefore the endline design and analysis must work within the constraints of that sample. A second consideration is the analytic approach used to measure impacts that are attributable to the program itself rather than other confounding factors. In this study, a comparison group constructed from households in villages who did not receive Yaajeende program activities serves as the counterfactual. It is anticipated that statistical matching will be used in the final impact analysis to YAAJEENDE FINAL PERFORMANCE EVALUATION STATEMENT OF WORK 18 improve the comparability of comparison group households and villages with those in the treatment group. Selection factors for Yaajeende program participation will be taken into account for the matching, to the extent possible, in order to strengthen the comparability of treatment and comparison group households. This statement of work requires that the Evaluation Team develop and submit for approval an Evaluation Design Report (EDR) at the end of Phase I, prior to the start of the evaluation fieldwork in Phase II. The EDR will summarize the findings of the IE evaluability assessment conducted during Phase I, include the finalized evaluation questions, the evaluation team’s recommended evaluation approach and detailed methodology, a description of alternative evaluation design options and their associated strengths and limitations, the proposed evaluation fieldwork and deliverables schedule, team composition and estimated budget; and the draft qualitative and quantitative instruments. 2. Data Collection Methods The data collection methodology to be used by the team will broadly consist of two phases: Phase 1: Document review and pre-analysis of baseline and midterm survey data to develop the Evaluation Design Report The team will review a wide range of documentation including the Activity baseline and midterm evaluation reports and data, Activity agreements/contracts, modifications, Activity Monitoring and Evaluation Plans (AMEPs), project databases, and reports as means of obtaining depth of understanding around program implementation, how implementation may have differed in practice from implementation plans (this is crucial for strong IE design, determining appropriate analyses, and accurate interpretation of IE results) The review of baseline and midterm methodology, survey data and instruments by the evaluation team during the endline evaluation design phase will enable the evaluation team to determine the most rigorous and feasible evaluation approach at endline, and how endline sampling protocols and instruments can be modified to add gap-filling questions, clarifying questions, or improve the ability for the endline analysis to accurately measure, detect and explain program impacts. This will also enable determination of most appropriate analytic methods for the quantitative analysis. Phase 2: Quantitative and qualitative field research for the endline evaluation The final description of data collection methods and draft instruments will be included in the Evaluation Design Report. It is anticipated that the endline data collection will include both a quantitative household survey, and qualitative data collection. For the quantitative and qualitative research, the team anticipates using the following research instruments: Quantitative Data Collection: • Household survey of program beneficiaries and households in the comparison group sample: Closed questions to determine beneficiary satisfaction, access to services, socio-economic inclusion, improved capacity, and quality delivery. The endline household survey will use the same survey instrument that was implemented at baseline and midline, consisting of the following modules: household roster, household assets, revenue sources, surface area cultivated, YAAJEENDE FINAL PERFORMANCE EVALUATION STATEMENT OF WORK 19 agriculture and livestock production, debts and financial services, participation in Yaajeende activities, food consumption, nutrition and health, and anthropometry. Different modules target different household respondents, including head of household, woman in charge of households, and women in household with children under the age of five. Based on an exploratory analysis of the baseline and midline data during the evaluation design phase (see section on Strength and Limitations below), and instruments review, the endline household survey may be modified slightly to add gap-filling or clarification questions that will improve the ability for the endline analysis to accurately measure and detect program impacts. Qualitative Data Collection: • Key informant interviews: Open questions for interviewing program stakeholders: (implementing partner) project staff, participating national and local authorities, service providers, and USAID representatives. The endline data collection will use the same KII protocols as implemented at baseline and midline, with updates made as needed to improve the ability for the endline data collection to meet the endline evaluation priorities. Pending the exploratory analysis of the baseline and midline data during the evaluation design phase, and instruments review, the endline KII protocol may be modified slightly to add gap-filling or clarification questions that will improve the ability for the endline analysis to explain reasons for observed impacts and variation in impacts across different groups of interest. • Focus group discussions: Open questions for group discussion with different types of program beneficiaries and non-beneficiaries to discuss program effects on individuals and institutions, and obtain depth of understanding around reasons for impacts and their variation across different sub-groups of interest. The endline data collection will draw on the same FGD protocols as implemented at baseline and midline, with updates made as needed to improve the ability for the endline data collection to meet the endline evaluation priorities. Pending the exploratory analysis of the baseline and midline data during the evaluation design phase, and instruments review, the endline FGD protocols may be modified to add gap-filling or clarification questions that will improve the ability for the endline analysis to explain reasons for observed impacts and variation in impacts across different groups of interest. Other data sources, such as secondary data available from the projects itself, research studies, or from the Government of Senegal will also be considered. A wide variety of data sources should be considered as part of the data collection process. Sampling A detailed approach to quantitative data collection will be described in the Evaluation Design Report. The household survey sampling for the endline data collection is expected to follow the same broad multi-stage cluster sampling approach that was utilized for the baseline and midline data collection. At baseline, 2,690 households were sampled across 167 project villages and 102 comparison group villages, stratified by zone and CR, with 10 households surveyed per village. At midline, 27 “new intervention villages” were randomly selected from project CRs and included in the sample. Additionally, the number of households per village (in both treatment and comparison villages) was increased from 10 per village to 17 per village in order to provide sufficient power for statistical analysis of impacts on children under 5 years of age. For the qualitative data collection, the evaluation team will develop semi-structured interview and FGD guides tailored for each respondent category. All qualitative instruments and guides will be consistent YAAJEENDE FINAL PERFORMANCE EVALUATION STATEMENT OF WORK 20 with the indicators and objectives of the evaluation, such that questions map to corresponding project indicators and overall evaluation questions. Focus groups and key informant interviews (KIIs) will be stratified across project locations, types of beneficiaries (including attention to gender balance), and key stakeholder groups including project implementing staff and partners, government institutions, private sector representatives, and project beneficiaries. A detailed approach to qualitative data collection will be described in the Evaluation Design Report. Data Analysis Pending completion of the assessment of IE evaluability feasibility during the evaluation design phase (Phase I), the Evaluation Design Report will propose the detailed analysis methods to be used for answering each evaluation question, based on the evaluation team’s recommendation on the most feasible evaluation option available. If an impact evaluation is determined to be feasible, it is anticipated that the evaluation team will use a Difference in Difference (DID) regression approach, coupled with statistical matching, to estimate the program’s impact on the selected outcomes of interest. Under the DID approach, an estimate of the program’s impact on each outcome indicator is obtained from the average difference in outcomes between matched treated and comparison group households, at baseline and endline. We combine the DiD design with statistical matching to further reduce sources of bias and improve the precision of the impact estimate. Several different matching approaches are available. Per best practices, analysts will select the matching approach that yields the strongest comparability across the treatment and comparison group units (i.e., balance across treatment and comparison group households on key baseline characteristics). If an impact evaluation is still determined to be feasible, the endline analysis will likely adopt a standard panel regression model to estimate treatment effects in the context of a difference-in-difference approach. Panel regression models are a standard approach to estimating treatment effects in the context of a different-in-difference set-up. The model includes a range of covariates to control for observed differences in the treatment and control groups, as well as fixed or random effects that can control for time-invariant unobserved factors. The treatment effect is estimated by a regression coefficient on a dummy variable that interacts time and treatment. For continuous outcome variables at the household level, the panel regression model takes the following generic form:55 𝑌𝑖𝑡 = 𝛾0 + 𝛾1𝑋𝑖𝑡 + 𝛾3𝛿𝑡 + 𝛽(𝛿𝑡 ∗ 𝑇𝑖 ) + 𝛾𝑖 + 𝜀𝑖𝑡 Where: Yit is the outcome of interest for household i at time t, Xit is a vector of covariates, δt is a dummy variable equal to 1 at the endline, 55 Probit or logit models will be used for binary outcome variables. The team will determine during endline analysis whether to adopt a fixed or random effects model, based on what is most statistically appropriate given the data. YAAJEENDE FINAL PERFORMANCE EVALUATION STATEMENT OF WORK 21 T is a dummy variable equal to 1 for members of the treatment group, γi is a vector of household-level fixed or random effects εit is a random error term, and the γ and β are parameters to be estimated. The estimate of program impact is given by β, which measures the intent to treat (ITT) estimate for this analysis. Observations will be weighted by their inverse probability of selection into the survey sample. To determine how impacts vary by sub-groups of interest, a separate set of models will be estimated, which include an interaction between treatment assignment and a dummy to indicate sub-group status. The interpretation of the sub-groups analysis results will pay particular attention to power issues, as the smaller sample sizes that are typically available for sub-groups analyses can introduce additional limitations on statistical power to detect significant results. For the qualitative data analysis, transcribed data from the focus group discussions (FGDs) and KIIs will be analyzed using content analysis techniques, coding text according to key themes of interest. Responses related to each theme will be summarized, and quotations from respondents will be included to illustrate key findings. This includes highlighting “outlier” responses and experiences to capture the range of responses. Descriptive and inferential analysis will also be disaggregated by sub-groups of interest. PRIVACY AND CONFIDENTIALITY CONSIDERATIONS Informed Consent The evaluation will obtain informed consent from respondents before carrying out any data collection in households. A consent form will be used that will be translated into appropriate local languages. Scripts for interacting with participating households, survey instruments, focus group scripts (if applicable), and all other data collection materials are subject to ethical approval before use. Careful attention will be paid to ensure that respondents understand that their responses will be used for research purposes and are expected to be made public without compromising their confidentiality and anonymity. Ethical Approvals In-country approvals from the Government of Senegal, via the Agence National de la Statistique et de la Demographie (ANSD), will be obtained by the evaluation team. 9. STRENGTHS AND LIMITATIONS The Evaluation Design Report will include a discussion of the strengths and limitations of the evaluation team’s recommended evaluation approach, and each of the alternative approaches included in the EDR. Here, we broadly note the following strengths and limitations of a DID quasi-experimental design, which the evaluation team will aim to conduct if the evaluation design phase suggests that this approach is still the most feasible and rigorous option available to quantify Yaajeende program impacts and meet USAID’s learning objectives. Carefully designed quasi-experimental studies can be used to generate findings that are widely recognized as scientifically valid for causal inference. When experimental methods such as randomized controlled trials are not available, as is the case for the Yaajeende FIE, quasi￾experimental designs remain the best available choice for robust and rigorous attribution of observed impacts to project interventions. An additional strength of the proposed mixed methods design for the YAAJEENDE FINAL PERFORMANCE EVALUATION STATEMENT OF WORK 22 Yaajeende endline evaluation is the strong reliance on complementary qualitative data collection to inform on how impacts may vary across different types of beneficiaries, and the reasons why; and broader learning for effective programming. The primary drawback of quasi-experimental designs is that they involve a risk of selection bias—i.e., that the differences in outcomes between the treatment and control groups may be the result of unobserved systematic differences between the two groups rather than the causal impact of the intervention. A limitation of this approach is that the estimate of impacts can be biased if there are unobserved trends that selectively affect only the treatment or comparison group. To help with this limitation, the DID analysis will be combined with statistical matching, a common approach to reduce sources of bias and improve the precision of the impact estimate. The evaluation team will also draw on available project monitoring and secondary data, to understand broad trends in areas where treatment and comparison group households are located, and confirm there are no processes that selectively affected households or villages only in one of the two groups. A second potential limitation is that the sample size and availability of comparable units across treatment and comparison group households for this study could limit the statistical power for the analysis to detect impacts on certain indicators. The MIE for this study indicate that insufficient study power, and selection bias/noncomparability of treatment and comparison group households, are both potential limitations for a proposed impact evaluation approach to evaluate the Yaajeende program. Since the sample size and construction across treatment and comparison group clusters was already established at baseline, it will not be possible to fully overcome some of these limitations for the endline analysis. However, to help mitigate these limitations, the evaluation team will conduct exploratory analysis of the baseline and midline data during the evaluation design phase to identify how power to detect effects varies across outcome indicators, and explore whether the use of more sophisticated matching techniques may be able to improve comparability of the treatment and comparison group sample. In addition, the evaluation team will use the understanding of baseline and midline sample and data limitations derived during this phase to determine whether and how the household survey and qualitative protocols can be modified to provide stronger coverage on key outcomes and explanatory processes. Finally, recall and response bias are potential limitations for any qualitative or quantitative data collection effort. To mitigate response bias, the evaluation team will rigorously test its discussion templates and interview instruments and protocols to ensure that there are no leading questions, that the purpose of the evaluation is clear, that respondents are not primed with information that could skew their responses, and that respondents feel comfortable speaking truthfully. Recall bias may lead to exaggerated negative or positive perceptions of past experiences, as people tend to remember only key aspects or feelings over time. Follow-up interviews, a well-crafted survey instrument, appropriate follow-up questions, and the use of secondary data will help the evaluation team mitigate some of the challenges of recall bias. 10. DELIVERABLES The deliverables for this evaluation will include: Phase I: YAAJEENDE FINAL PERFORMANCE EVALUATION STATEMENT OF WORK 23 Deliverables 1 and 2: Draft and Final Evaluation Design Report: The Evaluation Design Report will indicate the proposed evaluation methodology, data collection plan, and draft data collection tools for the endline evaluation, together with alternative evaluation options that may be available if the anticipated quasi-experimental impact evaluation approach is determined not to be the most appropriate or feasible option. The draft Evaluation Design Report will be submitted on or around 35 days after SOW approval and receipt by the evaluation team of the cleaned and identified baseline and midline data, codebooks, analysis do-files and other supporting documentation, and the survey instruments used at baseline and at midline. This will give the evaluation team adequate time to conduct exploratory analysis on the existing data, review baseline and midline tools and sampling, and propose the most rigorous and feasible approach for the endline data collection and evaluation. The EDR will summarize the findings of the IE evaluability assessment conducted during Phase I, include the finalized evaluation questions, the evaluation team’s recommended evaluation approach and detailed methodology (indicators/variables to be measured, sampling methods, data collection and analysis methods), a description of alternative evaluation design options and their associated strengths and limitations, the proposed evaluation fieldwork and deliverables schedule, team composition and estimated budget; and the draft qualitative and quantitative instruments. The production of the EDR is inclusive of the evaluation team conducting an assessment of the feasibility of conducting an impact evaluation at endline (rather than a performance evaluation), and presenting USAID with its recommendation on the most rigorous evaluation approach that is feasible for the Yaajeende endline evaluation. In assessing the feasibility of conducting an impact evaluation at endline (rather than a mixed methods performance evaluation), the EDR will particularly take into consideration the following: (1) potential limitations on detecting program impacts due to limitations on sample size, statistical power, and threats to the validity of the DID approach due to potential noncomparability, contamination and/or treatment spillover into comparison group areas; and (2) the implications of the available options for the timing of the endline household survey data collection in 2017, given that it will not be possible to conduct the endline data collection at the same time as baseline and midline data collection efforts, which were both conducted during May-June post-harvest season typically prior to the onset of the lean season. In addition, the timing of Ramadan in 2017 during late May to late June, with potential residual impacts on certain dietary and nutrition indicators of interest for Yaajeende, will also be taken into account. The available options for the endline data collection are either during the 2017 lean season in July – September, or during the October-November harvest season. Phase II (pending USAID approval to move forward with Phase II): Deliverable 3: Final quantitative household survey instrument and qualitative instruments. Deliverable 4: Final data collection protocols and enumerator training materials. Deliverable 5: Data quality and cleaning plan. Deliverable 6: Weekly production reports during survey fielding: A written report of the data collection progress made in the field covering key scheduled activities for data collection, status of completion, data quality checks, and constraints encountered during the data collection process. Deliverable 7: Draft Evaluation Report: The team will submit a draft report to the MEP COR and Technical POC who will provide comments for revision and finalization of the report ten working days YAAJEENDE FINAL PERFORMANCE EVALUATION STATEMENT OF WORK 24 following the draft submission. This is anticipated to be a 60 page report, excluding annexes and attachments. Deliverable 8: Final Evaluation Report: A written and electronic document that includes an executive summary, table of contents, methodology, findings, conclusions, lessons learned, and recommendations. The report will be submitted in English and translated into French. Deliverable 9: Abstract: A 2-page document including the purpose, questions, methodology and outcome of the evaluation/research; i.e. findings, conclusions and recommendations. The Abstract will be submitted in English and French and will be used to reach a broader audience of donor and implementing partners about the research findings. Deliverable 10: Results presentation: A PowerPoint presentation on findings, conclusions, and recommendations of the evaluation. The presentation should not be more than 15 slides. Additional presentations can be held in order to involve larger level stakeholders within the GOS, implementing partners, or relevant technical and financial partners. Deliverable 11: Household Survey Data: Raw and cleaned versions of the survey database will be delivered with corresponding metadata documentation allowing use of the data by a third party. This includes survey data cleaning and analysis do files. Deliverable 12: Qualitative data: Qualitative data transcripts in French. The table below provides estimated due dates for each of these deliverables: Table 1: Deliverables List and target Dates Deliverable Estimated Due Date Target Date PHASE I 1. Draft Evaluation Design Report (EDR), based on an assessment of evaluability feasibility conducted through exploratory analysis of the baseline and midline data. The EDR will summarize the findings of the IE evaluability assessment conducted during Phase I, include the finalized evaluation questions, the evaluation team’s recommended evaluation approach and detailed methodology, a description of alternative evaluation design options and their associated strengths and limitations, the proposed evaluation fieldwork and deliverables schedule, team composition and estimated budget; and the draft qualitative and quantitative instruments. o/a 35 days following approval of SOW and receipt by the evaluation team of the baseline and midline datasets and supporting data files (codebooks; cleaning and analyses do-files) and baseline and midline survey instruments May 30, 2017 YAAJEENDE FINAL PERFORMANCE EVALUATION STATEMENT OF WORK 25 Deliverable Estimated Due Date Target Date 2. Final Evaluation Design Report, including final data collection and analysis methods, draft evaluation instruments, team composition, and proposed timeline o/a 14 days following receipt of USAID comments on Draft Evaluation Design Report June 27, 2017 (assumes a 2-week comment window by USAID) PHASE II 3. Final quantitative and qualitative instruments o/a 14 days following USAID selection of final evaluation approach, and USAID approval to move forward with Phase II TBD based on Phase II approval 4. Final data collection protocols and enumerator training materials o/a 5 days prior to start of enumerator training TBD based on timing of data collection 5. Data quality and cleaning plan o/a 15 days after data collection launch TBD based on timing of data collection 6. Weekly production reports during survey fielding, including data quality checks weekly, during data collection TBD based on timing of data collection 7. Draft Evaluation Report o/a 70 days following completion of endline data collection TBD based on timing of data collection 8. Final Evaluation Report o/a 21 days following receipt of USAID comments on Draft Evaluation Report TBD based on timing of data collection 9. 2-page Abstract including the purpose, questions, methodology and outcome of the evaluation/research, submitted in English and French o/a 21 days following receipt of USAID comments on Draft Evaluation Report TBD based on timing of data collection 10. Initial presentation of findings o/a 10 days following completion of Final Evaluation Report TBD based on timing of data collection 11. Fully cleaned, redacted, and documented endline quantitative household survey data submitted o/a 30 days following approval of final evaluation report TBD based on timing of data collection 12. Cleaned qualitative data transcripts submitted in French o/a 30 days following approval of final evaluation report TBD based on timing of data collection 11. TEAM COMPOSITION The team will be led by an evaluation specialist with experience conducting impact evaluations utilizing mixed methods. The team will include individuals with expertise in econometrics; advanced qualitative data collection and analysis; survey data collection; and nutrition subject matter expertise. CVs of YAAJEENDE FINAL PERFORMANCE EVALUATION STATEMENT OF WORK 26 proposed members of the team will be sent to MEP Contracting Officer Representative (COR) and Technical point of contact (POC) for approval. All finalized team members are required to provide a signed statement attesting that they have no conflict of interest or describing any existing conflict of interest. YAAJEENDE FINAL PERFORMANCE EVALUATION STATEMENT OF WORK 1 Table 2: TASKS and estimated Levels of Effort Tasks Team Leader / IE Specialist Evaluation Specialist Survey Specialist Quanti￾tative Analyst Statisti￾cian Research Analyst MCH & Nutrition Specialist Anthropo metry Specialist Quali￾tative Specialist Local SME Consultan t NORC Home Office Manager MEP Task Mgr MEP Data Analyst PHASE I Desk review of program reports / documentation 1 1 1 Evaluability and EDR: • Baseline and midline data exploration for endline IE feasibility • Develop Evaluation Design Report with recommended evaluation approach and alternate options; • Update and finalize instruments and sampling protocols for endline; • Finalize EDR 12 10 5 5 2 4 5 3 4 1 5 3 Data Collection Firm Solicitation 1 1 0.5 1 PHASE II Work Plan Development 2 1 1 0.5 1 Finalize quantitative and qualitative instruments and protocols 1 2 2 1 Data Collection Firm Selection, Contracting and Management 2 2 1 1 0.5 2 1 YAAJEENDE FINAL PERFORMANCE EVALUATION STATEMENT OF WORK 2 Develop enumerator training materials; program and pre-test CAPI survey 0.5 4 12 4 3 3 1 2 Survey piloting, enumerator training, and participation in initial data collection 10 12 12 12 12 7 5 Data collection oversight 1 2 2 1 1 0.5 7 5 Qualitative data analysis 2 5 10 0.5 7 5 Quantitative data cleaning and analysis 8 8 3 13 2 8 3 2 0.5 Initial results write-up and discussion with USAID 7 5 1 1 5 5 2 2 2 Follow-up analyses 5 3 5 1 3 3 2 Prepare draft final report, 2-page abstract, and presentation of findings 12 5 1 4 5 4 0.5 2 1 Data set preparation 0.5 2 3 5 5 Respond to comments; Finalize evaluation report, abstract, and presentation 5 2 2 3 2 0.5 2 1 Total LOE in days: 401 70 40 20 35 4 45 42 20 35 20 5 40 25 *Note that the project budget also includes 10 days LOE for a NORC Research Assistant and 4 days LOE for NORC’s financial manager. These inputs are not disaggregated by task here as they entail broad-based management support for NORC’s activities and reporting responsibilities under the project. Table 3: Estimated Timing of evaluation Completion by Broad Activity Activity Mar Apr May June July Aug Sept Oct Nov Dec YAAJEENDE FINAL PERFORMANCE EVALUATION STATEMENT OF WORK 3 PHASE I SOW Desk review of program reports Evaluation Design Phase: • Baseline and midline data exploration and IE evaluability assessment • Develop and finalize Evaluation Design Report • Update qualitative and quantitative instruments and sampling protocols (including IRB preparation) Deliverable 1: Draft Evaluation Design Report (target: May 30, 2017) Deliverable 2: Final Evaluation Design Report (target: June 27, 2017) Data collection firm solicitation PHASE II Workplan Development Deliverable 3: Final quantitative and qualitative instruments (target: 14 days after Phase II approval) Data collection firm selection and contracting; instruments translation Enumerator training and data collection Deliverable 4: Final data collection protocols and enumerator training materials (5 days prior to launch) Deliverable 5: Data quality and cleaning plan (15 days after launch) Deliverable 6: weekly production reports during data collection (weekly) YAAJEENDE FINAL PERFORMANCE EVALUATION STATEMENT OF WORK 4 Data cleaning Data analysis Initial results & discussion with AID Follow-up analyses Report writing Data set preparation Deliverable 7: Draft Evaluation Report Deliverable 8: Final Evaluation Report Deliverable 9: 2-page Abstract Deliverable 10: Results presentation Deliverable 11: Quantitative data and documentation Deliverable 12: Qualitative data transcripts *Data collection is currently anticipated for either during July-Aug 2017, or Oct-Nov 2017, and will be determined during the evaluation design phase. If data collection takes place during the latter period, the timing of subsequent activities would shift accordingly. YAAJEENDE FINAL PERFORMANCE EVALUATION STATEMENT OF WORK 1 12. PARTICIPATION OF USAID STAFF AND PARTNERS It is expected that the USAID/Senegal Economic Growth Team (EGO) and the Program Office Data Analyst will work closely with the evaluation team throughout the process of planning, preparing for fieldwork, data collection, preparation of findings and dissemination of the report. Both the EGO Team and Program Office Data Analyst will review the IE Design Report and provide approval prior to the launch of fieldwork. In addition, a technical committee comprised of USAID EGO and Program Office staff as well as GOS stakeholders and the ANSD will be developed. They will review the methodology, tools and data analysis plan. MEP will also seek a formal “visa” approval from the ANSD for the endline data collection. As part of the evaluation, USAID/EGO Team, key Bureau for Food Security (BFS) staff in Washington and Ministry of Agriculture and other key stakeholders will also be interviewed. At the completion of the fieldwork, it is expected that USAID, Yajeende staff, and other stakeholders in the nutrition and agriculture sector will participate in a presentation of the evaluation’s initial findings, conclusions, and recommendations. MEP will work closely with USAID to discuss the development of a steering committee to help promote the study results to a much larger national audience to ensure that the study results are used by the GOS to inform other agriculture and nutrition programs. 13. SCHEDULING AND LOGISTICS MEP Senegal will arrange all logistics for fieldwork. MEP Senegal (for the USAID/Senegal EGO Team) will request introductory communications for the evaluation team. All appointments will be made by MEP Senegal staff and team members. 14. DISSEMINATION Copies of the final report in French will be made available to all stakeholders participating in the initial findings workshop. MEP will work with USAID/EGO and Program Office to invite a key number of GOS representatives involved in agriculture and nutrition to serve on a steering committee to help in the dissemination of findings. Copies of the final report in English will be shared with relevant USG offices within USAID FtF and any other relevant USG agencies. Finalized copies of both the French and English report will be uploaded to the DEC. An executive summary of the final results in research brief format will be written for development practitioners and policy maker audiences. Finally, a 25-50 person presentation workshop open to GoS stakeholders will be held to present findings of the report. 15. REPORTING REQUIREMENTS It is expected that this report will be drafted and finalized in English and then translated into French. The report itself should not be longer than 60 pages total, excluding the Annexes. The report will be aimed to communicate key results to a non-technical audience, but will also provide sufficient details on methods and results to meet technical standards for impact evaluation reporting. The report will be branded with the standard USAID branding requirements and will be formally submitted to the DEC upon approval. 16. ATTACHED REFERENCE DOCUMENTS Please check all that apply below. X Budget YAAJEENDE FINAL PERFORMANCE EVALUATION STATEMENT OF WORK 2 Document review matrix Results framework Response matrix X Gantt chart X CVs Conflict of Interest Statements USAID evaluation policy USAID evaluation report structure YAAJEENDE FINAL PERFORMANCE EVALUATION STATEMENT OF WORK 3 17. AUTHORIZATIONS The undersigned hereby authorize the following items (checked below) for the Statement of Work (SOW) described above: Completion of the SOW, as described above; SOW staffing, as described above; Concurrence with Contracting Officer’s Travel Approval for the Consultant(s), requested above (if received prior to review). [COR to either sign below or indicate approval in a return email] __________________________________________ ________________ Contracting Officer’s Representative (COR) Date Roy Geiser, or designate __________________________________________ ________________ Office Director Date YAAJEENDE FINAL PERFORMANCE EVALUATION STATEMENT OF WORK 1 APPENDIX I: BUDGET Not applicable. YAAJEENDE FINAL PERFORMANCE EVALUATION STATEMENT OF WORK 1 APPENDIX II: GANTT CHART Activity Mar Apr May June July Aug Sept Oct Nov Dec PHASE I SOW / Workplan Development Desk review of program reports Evaluation Design Phase: • Baseline and midline data exploration and IE evaluability assessment • Develop and finalize Evaluation Design Report • Update qualitative and quantitative instruments and sampling protocols Deliverable 1: Draft Evaluation Design Report (target: May 30, 2017) Deliverable 2: Final Evaluation Design Report (target: June 27, 2017) Data collection firm solicitation PHASE II Deliverable 3: Final quantitative and qualitative instruments (target: 14 days after Phase II approval) Data collection firm selection and contracting; instruments translation Enumerator training and data collection Deliverable 4: Final data collection protocols and enumerator training materials (5 days prior to launch) Deliverable 5: Data quality and cleaning plan (15 days after launch) YAAJEENDE FINAL PERFORMANCE EVALUATION STATEMENT OF WORK 2 Deliverable 6: weekly production reports during data collection (weekly) Data cleaning Data analysis Initial results & discussion with AID Follow-up analyses Report writing Data set preparation Deliverable 7: Draft Evaluation Report Deliverable 8: Final Evaluation Report Deliverable 9: 2-page Abstract Deliverable 10: Results presentation Deliverable 11: Quantitative data and documentation Deliverable 12: Qualitative data transcripts *Data collection is currently anticipated for either during July-Aug 2017, or Oct-Nov 2017, and will be determined during the evaluation design phase. If data collection takes place during the latter period, the timing of subsequent activities would shift accordingly. YAAJEENDE FINAL PERFORMANCE EVALUATION STATEMENT OF WORK 3 APPENDIX III: CONFLICT OF INTEREST STATEMENTS YAAJEENDE FINAL PERFORMANCE EVALUATION STATEMENT OF WORK 4 YAAJEENDE FINAL PERFORMANCE EVALUATION STATEMENT OF WORK 5 YAAJEENDE FINAL PERFORMANCE EVALUATION STATEMENT OF WORK 6 APPENDIX IV: USAID CRITERIA TO ENSURE THE QUALITY OF THE EVALUATION REPORT Per ADS 201, Criteria to Ensure the Quality of the Evaluation Report, draft and final evaluation reports will be evaluated against the following criteria to ensure the quality of the evaluation report: • The evaluation report should represent a thoughtful, well-researched and well-organized effort to objectively evaluate the strategy, project or activity. • The evaluation report should be readily understood and should identify key points clearly, distinctly and succinctly. • The Executive Summary of the evaluation report should present a concise and accurate statement of the most critical elements of the report. • The evaluation report shall adequately address all evaluation questions included in the statement of work. Or the evaluation questions subsequently revised and documented in consultation and agreement with USAID. • Evaluation methodology shall be explained in detail and all tools used in conducting the evaluation such as questionnaires, checklists and discussion guides will be included in an Annex in the final report. • Evaluation findings will assess outcomes and impact on males and females. • Limitations to the evaluation shall be disclosed in the report, with particular attention to the limitations associated with the evaluation methodology (selection bias, recall bias, unobservable differences between comparator groups, etc.). • Evaluation findings should be presented as analyzed facts, evidence and data and not based on anecdotes, hearsay or the compilation of people’s opinions. Findings should be specific, concise and supported by strong quantitative or qualitative evidence. • Sources of information need to be properly identified and listed in an annex. Recommendations need to be supported by a specific set of findings and should be be action￾oriented, practical and specific, with defined responsibility for the action.