Final Performance Evaluation of the Amalima Development Food Assistance Project in Zimbabwe IMPEL Brief March 2020 | Volume I IMPEL | Implementer-Led Evaluation & Learning Associate Award ABOUT IMPEL The Implementer-Led Evaluation & Learning Associate Award works to improve the design and implementation of Food for Peace (FFP) funded development food security activities (DFSAs) through implementer-Led evaluations and knowledge sharing. Funded by the USAID Office of Food for Peace (FFP), the Implementer-Led Evaluation & Learning Associate Award will gather information and knowledge in order to measure performance of DFSAs, strengthen accountability, and improve guidance and policy. This information will help the food security community of practice and USAID to design projects and modify existing projects in ways that bolster performance, efficiency and effectiveness. The Implementer-Led Evaluation & Learning Associate Award is a two-year activity (2019-2021) implemented by Save the Children (lead), TANGO International, and Tulane University in Haiti, the Democratic Republic of Congo, Madagascar, Malawi, Nepal, and Zimbabwe. RECCOMENDED CITATION IMPEL. (2020). Final Performance Evaluation of the Amalima Development Food Assistance Project in Zimbabwe (Vol. 1). Washington, DC: The Implementer-Led Evaluation & Learning Associate Award PHOTO CREDITS Sara Alexander. Disaster risk reduction focus group in Gwanda district. DISCLAIMER This report is made possible by the generous support of the American people through the United States Agency for International Development (USAID). The contents are the responsibility of the Implementer-Led Evaluation & Learning (IMPEL) award and do not necessarily reflect the views of USAID or the United States Government. CONTACT INFORMATION IDEAL Activity c/o Save the Children 899 North Capitol Street NE, Suite #900 Washington, DC 20002 www.fsnnetwork.org info@fsnnetwork.org Prepared by: Final Performance Evaluation of the Amalima Development Food Assistance Project in Zimbabwe Acknowledgments i Acknowledgments The members of the endline evaluation team would like to gratefully acknowledge the support provided by the staff of the Amalima program. Special thanks go to David Brigham, Chief of Party, Nqobile Ncube, Deputy Chief of Party, and Personal Sithole, the Senior M&E Coordinator for Amalima, for their diligent support and patience throughout the endline evaluation process. We also want to thank all of the field staff from Amalima at district and ward offices, and the implementing partners who went out of their way to provide logistical support to enable the team to implement an intensive schedule over a relatively short period of time. We also greatly appreciated the time they set aside for interviews that provided insightful observations in their responses to our questions. Innumerable program participants and beneficiaries also offered a wealth of information on their experiences with the Amalima project. Their perceptive responses regarding the challenges they face in their lives, and the stimulating discussions we had with them addressing how Amalima helped to improve the quality of their lives, provided invaluable information. Our gratitude goes out to them as well. We also thank our partner Jimat Development Consultants, including Munhamo Chisvo, Gideon Gavera, and in particular Tinashe Sande, who provided invaluable assistance in organizing the staff, training, and logistics for the quantitative and qualitative surveys. In addition, our thanks go out to Deveria Banda, Arnold Damba, Jacob Kandayi, Munjira Mutambwa, Lazarus Mutizwa, Tasiana Nyadzayo, Mikiri Shingirai, and to independent survey monitor Kuziwa Makamanzi. The qualitative team wishes to acknowledge the excellent support of our field assistants: Pride Bhebhe, Monica Gavela, Loveness Gwesela, Ndumiso Mathuthu, Nhlanhla Mbele, Nomathemba Ngwenya, Fransisca Nyagato, and Renias Tasaranarwo. Behind the scenes at TANGO HQ we have appreciated the guidance and support of Jeanne Downen, Mark Langworthy, Monica Mueller, Elizabeth Cuellar, Lynn Michalopoulos, Stephanie Martin, and Carrie Presnall. Finally, we appreciate the guidance and interest from USAID’s Office of Food for Peace/Washington in this project: Arif Rashid and Adam Trowbridge. Hopefully, we've listened well, our observations are grounded in reality, our assessment is accurate, and our recommendations will be useful. Sincerely, Dr. Sara E. Alexander, Anthropology, Baylor University (Team Leader) Bernard Crenn, Independent Consultant Dr. Daniel K. Musoke, International Research Consortium George Nhunhama, Independent Consultant IMPEL | Implementer-Led Evaluation and Learning ii Table of Contents Table of Contents Acknowledgments ............................................................................................................................. i Figures..............................................................................................................................................iv Tables...............................................................................................................................................iv Acronyms ..........................................................................................................................................v Executive Summary..........................................................................................................................vii 1. Introduction ................................................................................................................................1 1.1 Program Background.....................................................................................................................1 1.2 Theory of Change ..........................................................................................................................3 2. Evaluation Overview....................................................................................................................5 2.1 Evaluation Purpose........................................................................................................................5 2.2 Evaluation Questions ....................................................................................................................6 3. Evaluation Methods.....................................................................................................................8 3.1 Quantitative Data Collection.........................................................................................................8 Overview ........................................................................................................................................8 Population-Based Sample Design...................................................................................................8 Data Analysis................................................................................................................................10 3.2 Qualitative Data Collection .........................................................................................................12 Overview ......................................................................................................................................12 Evaluation Team...........................................................................................................................12 Sample Design ..............................................................................................................................13 Methods.......................................................................................................................................13 Analysis, Coding and Interpretation Methods .............................................................................14 3.3 Integration of Quantitative and Qualitative Findings.................................................................14 3.4 Limitations...................................................................................................................................15 4. Evaluation Findings....................................................................................................................18 4.1 Targeting .....................................................................................................................................18 4.2 SO1: Household access to and availability of food improved.....................................................19 IR 1.1: Agricultural productivity increased...................................................................................19 Conclusions...................................................................................................................................25 IR 1.2: Agricultural marketing improved......................................................................................26 IR 1.3 Post-harvest losses reduced...............................................................................................28 Conclusions...................................................................................................................................28 4.3 SO2: Community resilience to shocks improved.........................................................................28 IR 2.1: Basic agricultural infrastructure and other production assets developed/rehabilitated .29 IR 2.2: Community-managed disaster risk reduction systems strengthened ..............................30 IR 2.3: Community social capital leveraged .................................................................................31 Conclusions...................................................................................................................................33 4.4 SO3: Nutrition and Health among Pregnant and Lactating Women and Boys and Girls under 2 Improved...........................................................................................................................................34 IR 3.1 Consumption of diverse and sufficient foods for pregnant and lactating women and boys and girls under 2 improved ..........................................................................................................34 IR 3.2 Health and hygiene and caring practices of pregnant and lactating women, caregivers, boys and girls under 2 improved..................................................................................................36 IR 3.3 Accessibility to and effectiveness of community health and hygiene services improved.39 Conclusions...................................................................................................................................42 Final Performance Evaluation of the Amalima Development Food Assistance Project in Zimbabwe Table of Contents iii 4.5 Unintended Outcomes................................................................................................................42 4.6 Factors Contributing to Outcomes..............................................................................................43 Project Integration .......................................................................................................................43 Capacity Building of Communities ...............................................................................................44 Gender Equality and Empowerment............................................................................................44 Community Cohesion ...................................................................................................................44 Environmental Sustainability........................................................................................................45 Collaboration with Key Stakeholders...........................................................................................45 4.7 Contribution of Activities to Mitigation, Adaptation to, and Recovery from Food Security Shocks and Stresses ..........................................................................................................................45 4.8 Beneficiary Satisfaction...............................................................................................................46 4.9 Coordination................................................................................................................................47 4.10 Gender Considerations..............................................................................................................48 4.11 Environmental Considerations..................................................................................................49 4.12 Sustainability .............................................................................................................................51 Conclusions...................................................................................................................................54 4.13 Lessons Learned ........................................................................................................................54 5. Recommendations.....................................................................................................................56 Annex A: References........................................................................................................................58 Annex B: Evaluation Statement of Work ...........................................................................................61 Annex C: Training, Data Collection, and Quality Assurance................................................................82 Annex D: Imputing Missing Data.......................................................................................................86 Annex E: Data Sources: Interviews, Focus Groups, and Asset Observations........................................88 Annex F: Comparison of Baseline and Endline Data...........................................................................93 Annex G: Comparison of Baseline and Endline Indicators by Project Participation Status (Amalima).101 Annex H: Supplemental Tables.......................................................................................................107 Annex I: Analysis of “Adequacy” Indicators.....................................................................................113 Annex J: Multiple Regression Analysis ............................................................................................116 Volume II Annexes: Annex K: Data Collection Instruments Annex L: Disclosure of Conflicts of Interest IMPEL | Implementer-Led Evaluation and Learning iv Table of Contents Figures Figure 1: Zimbabwe food insecure population trends (in millions)..............................................................1 Figure 2: Amalima asset provisioning .........................................................................................................21 Figure 3: Prevalence of underweight, stunted, or wasted CU5 at baseline and endline ...........................38 Figure 4: WASH indicators at baseline and endline (Amalima) ..................................................................40 Figure 5: FEWSNET seasonal calendar and critical events timeline for Zimbabwe ....................................62 Figure 6: Percentage of farmers using sustainable agriculture (crop, livestock, NRM) practices in the last 12 months at baseline and endline (Amalima) ..................................................................................107 Figure 7: Livestock practices in the past 12 months (Amalima) ...............................................................107 Figure 8: Cropping practices in the past 12 months (Amalima) ...............................................................108 Figure 9: Percentage of farmers who practiced value chain activities promoted by the project in the past 12 months (Amalima) .............................................................................................................108 Figure 10: Gender indicators at baseline and endline for women’s adequacy of ownership of assets, adequacy of decision-making about assets, and adequacy of decisions on credit (Amalima).................109 Figure 11: Gender indicators at baseline and endline for men’s adequacy of ownership of assets, adequacy of decision-making about assets, and adequacy of decisions on credit (Amalima).................109 Figure 12: Percentage of farmers who used financial services in the past 12 months (Amalima)...........110 Figure 13: Average Household Dietary Diversity Score (HDDS) (Amalima)..............................................110 Figure 14: Prevalence of households with moderate or severe hunger (HHS) (%) (Amalima).................110 Figure 15: Percentage of households with poor, borderline, or adequate Food Consumption Scores (FCS) at baseline (2014) and endline (2019) (Amalima)................................................................111 Figure 16: Child feeding practices and dietary diversity (Amalima) .........................................................112 Tables Table 1: Primary evaluation questions and methods...................................................................................6 Table 2: Information used to compute sample size......................................................................................9 Table 3: FFP endline indicators...................................................................................................................10 Table 4: Survey response rates (Amalima) .................................................................................................12 Table 5: Summary of qualitative data distribution, by district...................................................................13 Table 6: Personnel employed for Zimbabwe quantitative survey training and data collection.................82 Table 7: Per capita daily consumption: expenditures over a 30-day period (USD 2014)...........................87 Table 8: Key informant interviews..............................................................................................................88 Table 9: Summary data for focus group discussions conducted ................................................................90 Table 10: Summary data for asset observations.........................................................................................92 Table 11. Baseline-endline comparison of household sanitation and drinking water, sanitation facility, source and treatment of drinking water (Amalima) ....................................................................111 Table 12: Joint decision-making on credit, households that reported cash or in-kind borrowing (Amalima)..................................................................................................................................................115 Table 13: Regression results for use of financial services and adoption of sustainable crop practices...117 Table 14: Regression results for child nutritional variables, underweight and stunting of CU5 ..............118 Table 15: Regression results for household food security status (adequate food security based on HHS) .....................................................................................................................................................119 Final Performance Evaluation of the Amalima Development Food Assistance Project in Zimbabwe Acronyms v Acronyms AGRITEX Department of Agricultural Technical and Extension Services AMC Asset Management Committee ANC Ante natal care BCC Behavior change communication BL/EL Baseline/endline CA Conservation agriculture CBF Community-based facilitator CFA Cash for Assets CGV Care group volunteer CHC Community Health Club CMDRR Community-Managed Disaster Risk Reduction CSI Coping Strategies Index CU2 Children under two CU5 Children under five DFAP Development Food Assistance Project DFSA Development Food Security Activity DRR Disaster Risk Reduction EA Enumeration Area FaaB Farming as a Business FEWSNET Famine Early Warning System Network FFA Food for assets FFP Food for Peace GoZ Government of Zimbabwe HCC Health Center Committee IEC Information, education and communication IGA Income- generating activity IP Implementing partner IPM Integrated pest management IR Intermediate result IYCF Infant and young child feeding KAP Knowledge, Attitude and Practice MAD Minimum Acceptable Diet (indicator) MCH Maternal and child health M&E Monitoring and evaluation MIS Management information system IMPEL | Implementer-Led Evaluation and Learning vi Acronyms MoHCC Ministry of Health and Child Care MoWAGCD Ministry of Women’s Affairs, Gender and Community Development ODF Open-defecation-free ODK Open Data Kit PHH Post-harvest handling PLW Pregnant and lactating women SBCC Social and behavior change communication SO Strategic objective TIP Trials of improved practices TOC Theory of Change TPCPDL Total per capita poverty datum line VAC Village Agricultural Coordinator VS&L Village Savings and Lending WASH Water, sanitation and hygiene WEAI Women's Empowerment in Agriculture Index WFP World Food Program ZimVAC Zimbabwe Vulnerability Assessment Committee Final Performance Evaluation of the Amalima Development Food Assistance Project in Zimbabwe Executive Summary vii Executive Summary Evaluation Purpose and Evaluation Questions This report details the findings and recommendations of the endline evaluation of the Amalima project, a five-year Development Food Assistance Project (DFAP) in Zimbabwe funded by the United States Agency of International Development (USAID) Office of Food for Peace (FFP). The evaluation’s broad objective is to determine conditions in targeted areas prior to the start of new DFAPs. It is comprised of a representative population‐based household survey that collected data for required impact and outcome Indicators for Title II program intervention areas, and a qualitative study that provides depth and context, and serves to triangulate quantitative survey findings and analysis. The endline evaluation is designed as the second step in a two‐part evaluation process, following the baseline study at the beginning of the program. The evaluation’s main objectives are to:  Determine endline values of key impact and outcome-level indicators (disaggregated by awardee, age, and sex as appropriate), demographics in target areas, and appropriate independent variables;  Conduct bivariate and multivariate analyses of impact and outcome indicators with independent variables identified for inclusion in the survey as appropriate, with results provided by awardee and the overall Title II country program area;  Gather qualitative data to ground‐truth survey data and provide contextual information on the overall food insecurity and malnutrition situation; and  Assess progress toward end‐of‐program targets for impact and outcome indicators. The evaluation results have multiple audiences. The findings are expected to have primary accountability and learning value for USAID (FFP/Washington, USAID/Zimbabwe, FFP Southern Africa Regional Office, and FFP learning network), and implementing partners and their sub-partners. Additional stakeholders include Zimbabwean government officials from key collaborating offices such as the Matabeleland North and Matabeleland South provincial Drought Relief and Food and Nutrition Council. USAID/FFP will use the evaluation findings, conclusions, and recommendations to inform future FFP activity design in Zimbabwe and the region. FFP may use the evaluation recommendations and findings to refine future proposal guidelines and program policy. FFP provided evaluation questions to guide the design and development of the endline evaluation. The evaluation team also referred to the baseline quantitative and qualitative data for comparison with endline findings. The team assessed the technical viability of the evaluation questions and incorporated specific elements in the design and methodology of the endline (both the quantitative and qualitative components) to ensure the evaluation provides valid and reliable data and directly addresses the evaluation questions. In some cases, this method involved incorporating additional variables or strata in the design of the household survey and the qualitative component. The primary questions are:  To what extent did the programs achieve the intended goal, objectives and results as defined by the Results Framework?  How satisfied were beneficiaries with the program? How did program activities improve the ability of beneficiary households and communities to mitigate, adapt to, and recover from food security shocks and stresses?  To what extent did the program coordinate with other food security and humanitarian programming, the host country government, and the donor?  How sustainable are the programs’ outcomes? IMPEL | Implementer-Led Evaluation and Learning viii Executive Summary  What lessons can be learned for future FFP and USAID Title II in Zimbabwe? Evaluation Design, Methods and Limitations The evaluation used a mixed-methods design that integrated data from multiple sources. It drew on quantitative data collected via a population-based survey (PBS) to measure current status and change over time for 21 key FFP indicators, and a qualitative study to provide context and understanding of unexpected changes in key outcomes, participant perceptions of project achievements, and additional information from implementing partners and related stakeholders about project effectiveness, sustainability, unintended consequences, and lessons learned. The statistically representative PBS sample of 486 households in the four Amalima districts was selected using a multi-stage clustered sampling approach. The quantitative analysis follows a pre-post design in which the same questionnaire was administered at the start of program implementation in 2014 and after program completion in 2019. The endline indicator calculation methods are the same as those for the baseline. Bivariate analyses were applied to survey data to compare changes in indicators from baseline to endline and differences between Amalima participant and non-participant households. Qualitative data collection used purposive sampling and semi-structured focus group discussion (FGD) and key informant interview (KII) protocols. The qualitative study conducted 47 FGDs (419 participants; 86 males; 333 female) across the four project districts; 70 KIIs (42 males; 28 females) at village, ward, and district levels and with implementing partners and the private sector in Bulawayo; 34 asset observations in the four districts; and desk review. Possible attribution error, selection bias, recall error, and bias in self￾reported data were mitigated by carefully constructing interview guides and triangulating data across FGDs and KIIs. Findings and Conclusions SO1: HOUSEHOLD ACCESS TO AND AVAILABILITY OF FOOD IMPROVED The adjustments made in the planning reassessment in 2013 to place less emphasis on crop cultivation and more on livestock and irrigated gardens proved successful in terms of supporting a secure food supply. The focus on addressing the largely unmet demand for water for livestock and crops resulted in an increased commitment to livestock herding and lowered risk perceptions associated with establishing new nutrition gardens, which ultimately meant improvements in dietary diversity. Project participants perceived that the improved dip tank infrastructure and treatment procedures, reinforced by the paravet trainings and toolkits, helped improve health conditions, which they then explain, resulted in a decline in livestock death rates over the course of the project. The collective benefits of SO1 activities are community-based, and sustainability is projected to be secure for at least a few years. These activities were conducted in more sustainable ways by improving soil and water conservation works and securing more participation and buy-in from communities ahead of activity implementation. The knowledge and skill sets were taught to communities in ways that engendered confidence in their own knowledge base and ability to sustain project activities once Amalima exits. Agricultural marketing improved generally—a little for most households at the community level (goats, farming inputs) and significantly for a few hundred at a larger level (horticulture and matching grant groups). Village Savings and Lending (VS&L) groups, agro-dealers, and toward the latter half of the project, Village Agricultural Coordinators have been key to the project’s agricultural marketing success. Final Performance Evaluation of the Amalima Development Food Assistance Project in Zimbabwe Executive Summary ix Value chain and marketing outcomes are strongly influenced by external factors, and while the project addressed them well with dedicated knowledge, coaching, and financial support, it was at the cost of an ultimately high project dependency. It was very relevant for the project to promote better post-harvest handling to store crops, both for home use and for marketing surplus, but cyclical drought conditions were an impediment to improved adoption rates. The strong integration of project activities created an enabling environment for behavior change. The interventions in agriculture, VS&L, and gender mainstreaming were designed as a comprehensive package dealing with crop and livestock and aimed at boosting the knowledge and skills of both male and female farmers to intensify production and realize income through sales. This integrated package was designed intentionally, to deliberately strengthen the linkages between VS&L and agriculture on the one hand, and VS&L, health and nutrition on the other. Amalima promoted the VS&Ls, which ultimately increased incomes, thus enabling people to purchase inputs for better agricultural production and healthier diets. SO2: COMMUNITY RESILIENCE TO SHOCKS IMPROVED The disaster risk reduction activities were highly relevant as the area is prone to both slow- and rapid￾onset disasters, but their effectiveness is mixed. The Community-Managed Disaster Risk Reduction groups are dependent on the strength of the committees and community leadership. Generally, the impact and sustainability of community initiatives and assets were good, though there was some variation regarding early results. The emphasis on more proactive approaches during the extension, such as gully prevention and soil and water conservation measures, improved capacity and should consolidate efforts over the longer term. The project approach of leveraging social capital for systemic change is very relevant, as resilience in more remote, poorer areas with severe environmental stresses is less able to depend on technical solutions. This improved social capital is sustainable and there are many cases reported of benefits spreading beyond the direct project participants (e.g., kitchens, gardens, VS&Ls). The VS&Ls are a strong cross-sector link and cover the majority of areas/communities. New groups are forming and they have been spreading throughout the communities over the project’s five years. SO3: NUTRITION AND HEALTH AMONG PREGNANT AND LACTATING WOMEN AND BOYS AND GIRLS UNDER 2 IMPROVED Amalima increased the availability and consumption of different nutritious foods at the household level partly due to supplemental rations, which improved the complementary feeding of children under two, and to improved value chains promoted by the project. Engagement in VS&Ls also increased households’ ability to purchase nutritious foods for their families. The Healthy Harvest trainings promoted the importance of producing and consuming diverse and nutritious crops and vegetables. In addition, communities learned how to organize and maintain nutrition gardens and how to prepare nutritious foods using locally available ingredients. The increased availability and consumption of a diverse and nutritious diet led to reductions in stunting, wasting, and underweight in children under five. The evaluation showed improved health and hygienic practices among the targeted beneficiaries including antenatal care attendance, exclusive breastfeeding, quality and quantity of foods for children 6-23 months, feeding habits during illness, handwashing practices, and other hygienic practices such as safe play areas for infants. These improvements were attributed to the effective behavior change communication messaging facilitated by the care group approach. The integration of the Healthy IMPEL | Implementer-Led Evaluation and Learning x Executive Summary Harvest trainings into the conservation agriculture, horticulture, and care group trainings helped participants understand how the various initiatives of the project are integrated. The Healthy Harvest materials helped train lead mothers and lead farmers on a diverse, nutritious, and locally available diet. The increased male involvement in childcare and maternal support at home, and use of eco-stoves, provided caregivers and pregnant and lactating women more time for child caring. The implementation of Amalima’s water, sanitation and hygiene component in the four districts of Matabeleland was welcomed by the participating communities, and led to remarkable improvements in the accessibility and efficiency of health services. The project activities were clearly relevant and effective in meeting community needs and expectations, and left the communities satisfied. These particular efforts resulted in positive behavior change where the outcomes were readily incorporated in participants’ daily lives. Prospects for the sustainability of improved household hygiene after the main phase of project implementation are quite high, since the communities have realized that these practices strongly contribute to improved primary health for their family members. There remains, however, a need to strengthen government support to communities to scale up this intervention to new districts. Recommendations R1: Project planning. In order to develop complex interventions that are more likely to be effective, sustainable and scalable, project planners need to understand not just whether, but how and why an intervention has a particular effect, and which parts of a complex intervention have the greatest impact on outcomes. This requires a prospective, theory-driven process of intervention design and evaluation (DeSilva, Breuer and Patel 2014). Future projects should use a rigorous Theory of Change (TOC) approach that will ultimately indicate how and why the initiative works, empirically testing for every expected step on the path to impact. Detailed, sectoral TOCs should be developed based on community needs assessments conducted in the project planning phase, and in collaboration with stakeholders. The TOCs should be modified throughout the intervention and evaluation process through an ongoing process of reflection to explore change and how it happens. R2: Timeline. For a five-year project, an effective timeline would be: Year One: inception phase to involve training, trust and ownership building, setting up systems for sustainability and quality implementation (studies, pilot projects). Years Two and Three: full implementation with large community support. Year Four: project steps back and lets participants and stakeholders manage the majority of the work. Year Five: actively work on sustainability/replicability and filling gaps, no new activities, refresher activities but no new training. In a five-year program, where sustainability is a primary goal, the first two years should not place undue pressure to reach certain quantitative goals to the detriment of quality and sustainability, but rather lay down a solid basis for participants and stakeholder participation and ownership. Allow the first year to be an inception phase to adjust the original design and planning to better fit the reality and put all systems in place. R3: Trainings. Develop specific leadership, governance, and conflict resolution training, especially for community-based facilitators, Disaster Risk Reduction/Asset Management Committees, and traditional and religious leaders. Produce these training materials in the same manner as completed trainings, with simple text, in local languages, and with good graphic support. R4: Quality management. Establish a Quality Management department at the beginning of the project, to be responsible for monitoring, evaluation, accountability and learning (e.g., Knowledge, Attitude and Practice and barrier studies); project-wide accountability; administrative and technical compliance; sectoral integration; relevant policies and strategies; a theory of change that is regularly revisited; taking sustainability and self-replication into account from the start of the project; managing project data both Final Performance Evaluation of the Amalima Development Food Assistance Project in Zimbabwe Executive Summary xi to meet contractual obligations and to be useful to managers (e.g., via integrated databases, unique IDs, data quality assurance); and providing real-time information to feed into current and future programming. Considering that FFP development activities are similar across countries, it could be useful to have a standardized but customizable system for data acquisition and management, as each project struggles to put something basic in place, usually in the second year and still with gaps. A well-designed system could also be useful for project integration, quality management, and evaluation. R5: Communications. Scale up the use of phone technology in training, networking, and reporting. The use of WhatsApp by agro-dealers provided higher efficiency with effective communication, and lead farmers in Tsholotsho were using WhatsApp for notification of trainings, meetings, and other activities. R6: Water provisioning. The most recurrent community request was for perennial community water points for households, horticulture, or livestock use. Considering the increasing severity of drought conditions and the positive impact of the dams and other water infrastructure, future projects should prioritize such water amenities with local management and sustainability at the core. R7: Asset ownership. To ensure better community buy-in and sustainability, the branding of assets and documents (e.g., infrastructure, training material, disaster risk reduction plans) should highlight first and foremost the engagement of participants and stakeholders, with financial and implementing support less prominent. R8: Scale up care group model. The Ministry of Health and Child Care has started rolling out the care group methodology in non-project districts based on evidence of effectiveness from the project. However, there are no clear data on the costs of scaling up sustainable, innovative, community-based incentives, enhancing adolescent and young mother participation in care groups, or how to harness health for greater effectiveness. In order to support a more strategic approach for scaling up the care group model to other districts and countries, evidence should be documented to establish the causal impact of the care group model on maternal and child health outcomes; determine the cost benefit of scaling up the model compared to the standard of care; and identify innovative approaches for enhancing adolescent and young mother participation. Final Performance Evaluation of the Amalima Development Food Assistance Project in Zimbabwe Introduction 1 1. Introduction 1.1 Program Background Context: When the Amalima project was being formulated in 2012-13, ZimVAC estimated that 1.6 million Zimbabweans were food insecure—including 369,175 in Matabeleland North and South—and predicted at that time to be unable to access sufficient food during the next peak hunger period (January – March 2013) (ZimVAC 2012). Food insecurity (above 20 percent in the 13 combined districts of Matabeleland North and South) was at its highest level in three years. Moreover, nationwide, fewer than 10 percent of children 6-23 months of age consumed the minimum acceptable diet, as defined by the Ministry of Health and Child Welfare (MoHCW) (Food and Nutrition Council 2011). These rates were due in part to food insecurity, but also to beliefs and constraints dictating infant feeding practices. The 2012 ZimVAC Report also indicated a 33.8 percent stunting rate in children ages 6-59 months, representing widespread chronic malnutrition that risked worsening if food security conditions and household nutritional practices did not improve. The Amalima project was originally planned for 2014-2018 and was extended to 2020. The project targets four districts located across Matabeleland North (Tsholotsho) and South (Bulilima, Mangwe, and Gwanda). A number of significant challenges have persisted in this region over the duration of the project, some of which have inhibited positive results for certain activities designed under each of the strategic objectives. Figure 1 above shows the oscillation of food insecurity in Zimbabwe over the years of the project, also indicating the severity of the drought periods, which the project aimed to address through a focus on agricultural productivity and food and nutrition security for vulnerable populations in the four districts. By 2018 and well into the project, the number of severely food insecure people was expected to increase from about 567,000 between April and June 2018 to 2.4 million between July 2018 and March 2019, translating to 28 percent of the rural population (ZimVAC 2018, 2019). Over the course of the project, devaluation of local currency and shortages of US dollars inflated food production costs. Over 70 percent of Zimbabweans lived below the national poverty line. Remittances from the Zimbabwean diaspora constituted a major source of income as US$2 million, on average, entered the country every day in 2018 (FSIN 2019). Over the latter part of the project, late onset of rains and long mid￾season dry spells led to localized household food production shortfalls. By December 2018, food prices were more than 50 percent higher than 2017 prices. This economic and agricultural situation combined to reinforce chronic malnutrition trends: 234,000 children under five years(CU5) were acutely malnourished in 2018, with 26.5 percent of children ages 0-59 months living in rural districts stunted (ZimVAC 2018). Figure 1: Zimbabwe food insecure population trends (in millions) Source: ZimVac 2018 2.2 0.6 2.8 4.1 1.1 2.4 Year 2013/14 2014/15 2015/16 2016/17 2017/18 2018/19 IMPEL | Implementer-Led Evaluation and Learning 2 Introduction In 2012/13, team members were convinced a sustainable solution to long-term food and nutrition insecurity in Zimbabwe would require a multi-faceted approach that addressed a number of specific problems in the four districts if the overarching goal was to be achieved (Amalima 2014a). These main issues included:  Limited availability of food, in particular the staple crop maize, due largely to insufficient access to water resources, particularly to support crop cultivation and livestock herding;  Sub-optimal land use where maize was being cultivated in areas unsuitable for its production given poor soil quality and erratic rainfall;  Ineffective planning and preparation for severe drought conditions in terms of low maize yields and lack of safety nets to address shocks when they occurred;  Inaccessible inputs for crop production, largely in terms of drought-resistant seed varieties that were beyond the purchasing power of most households;  Low and undiversified household incomes where poorer households’ livelihoods were reliant on labor and remittances, with limited employment and income-generation opportunities;  Poor infant and young child feeding (IYCF) practices. The Zimbabwe National Nutrition Surveys (Food and Nutrition Council 2011, 2012) reported that in almost all rural districts, less than 10 percent of Zimbabwean infants were exclusively breastfed during the first six months. In targeted districts, only a small minority of children were fed according to IYCF practices.  Micronutrient deficiencies among pregnant and lactating women (PLW). At the beginning of the project, 32 percent of pregnant women in Zimbabwe suffered from anemia, reflecting widespread iron deficiencies. In Matabeleland South, this figure reached 45 percent (Zimbabwe National Statistics Agency 2012). Micronutrient deficiencies were compounded by unequal intra-household rationing practices.  High prevalence of childhood diarrhea, where diarrhea episodes resulted from insufficient access to clean water and sanitation, the prevalence of parasites, unsafe food preparation and storage, and poor knowledge of hygiene (Ibid.). Project goals and objectives: In FY 2013, the United States Agency of International Development (USAID) Office of Food for Peace (FFP) awarded Cultivating New Frontiers in Agriculture (CNFA) a five￾year (2013-2018) Title II Development Food Assistance Project (DFAP) in Zimbabwe called Amalima to be conducted in Matabeleland North and South, located in the western and southwestern regions of Zimbabwe. In 2017, CNFA and implementing partners (IPs) proposed a two-year extension (Amalima 2017a) to enable the program to respond to specific recommendations made in the Midterm Evaluation (CNFA and World Vision 2016), consolidate gains to date, and maximize the sustainable impact of activities. The FY 2019-20 extension was granted. The project was designed to offer a set of innovative approaches to address the causes of food and nutrition insecurity as outlined above, building on existing communal initiatives and solidarity to strengthen resilience. Amalima is implemented by CNFA and five IPs: the Organization for Rural Associations for Progress (ORAP), International Medical Corps (IMC), The Manoff Group, Africare, and the Dabane Trust. With these partners and local community members, Amalima sought to improve agricultural productivity and marketing, reduce disaster risk, and strengthen the adaptive capacity of households and communities in Matabeleland North and South. The project has three strategic objectives (Amalima 2014a): SO 1: Household access to and availability of food improved, with emphasis on improved crop and livestock production, enhanced soil fertility, adoption of conservation agriculture (CA) techniques, increased crop diversity, improved agricultural marketing, and enhanced value chain activities. SO 2: Community resilience to shocks improved, with emphasis on rehabilitation or construction of basic infrastructure that supports agricultural activities (e.g., dams, dip tanks, pump systems, nutrition Final Performance Evaluation of the Amalima Development Food Assistance Project in Zimbabwe Introduction 3 gardens), the strengthening of Disaster Risk Reduction (DRR) systems, improved leveraging of community social capital, improved access to savings especially for women, and promoting effective local group leadership structures. SO 3: Nutrition and health among PLW; and boys and girls under 2 improved, with emphasis on consumption of diverse and sufficient foods for PLW and CU2; improved health, hygiene and caring practices for caregivers, PLW and CU2; improved knowledge and skills of child health and nutrition; improved male involvement in child health and maternal nutrition; optimized time for child caring; and improved effectiveness of community health and hygiene services. By the end of the project in June 2020, the Amalima team’s vision for the targeted districts is to sustainably increase food security and reduce malnutrition, with a reduction in average food insecurity rates from 36 percent to 18 percent and a reduction in average stunting rates from 34 percent to 19 percent. This vision entails increased and improved production of nutritious foods in good and bad years; increased cereal “imports” from surplus areas and enhanced purchasing power of vulnerable households; and improved nutrition practices, including a quadrupling of exclusive breastfeeding rates and an increase in dietary diversity among PLW and CU2 from two food groups to five (Amalima 2014a, 2017a). 1.2 Theory of Change The concept Amalima is the coming together of households to pool resources (e.g., labor, money, assets) toward shared objectives. Essentially, Amalima refers to a social contract that “no one should go hungry if we can come together.” Together, communities reflect on problems, determine action, create systems to deal with challenges, and ensure accountability amongst themselves. Amalima interventions have been strongly supported by evidence from formative research undertaken in the project areas. Since livestock production (cattle, goats, sheep and indigenous poultry) is the mainstay of the Amalima districts, Amalima has placed a strong focus on building the knowledge and skills of farmers in livestock production, e.g., on practices such as pre-season deworming, dehorning, castration, supplementary feeding using fodder banks, vaccination, dipping, improved animal housing, and breed improvement, designed to strengthen farmers’ capacity to improve livestock production techniques (USAID, CNFA, Amalima. 2016c). Climate-smart cultivation practices, such as CA, are at the core of the crop production component of Amalima and have been proven effective in Natural Regions IV and V, where mean annual rainfall is low and erratic. Anecdotal evidence suggests that the relevance of CA became even more prominent in the 2015/16 season, which was severely affected by an El-Niño-induced drought. Farmers in Ward 7 Tsholotsho, who grew sorghum, confirmed that they had been able to harvest the crop more than once: as long as there was moisture retained in the ground, the sorghum continued to yield (CNFA and World Vision 2016). In addition, interventions under Nutrition and Health have a strong theory of change linked to global empirical evidence on stunting, and to the national policy and strategic plan on nutrition, which emphasize that interventions should be targeted at children and their mothers in the first 1,000 days of the child‘s life if stunting prevalence is to be reduced. The nutrition model addresses both the immediate and the underlying causes of malnutrition. While providing direct nutrient-rich food handouts to children and their mothers to improve dietary intake is not sustainable in the long run, it provides immediate relief to drought-stricken households, while the other project components seek to build resilience of these target communities to such shocks, as well as improve their knowledge and capacities to produce a more-diversified basket of food commodities at the farm level. IMPEL | Implementer-Led Evaluation and Learning 4 Introduction The Amalima program designed the following principles of implementation to form the foundation for the project initiatives:  Strengthen household and communal resilience by melding traditional concepts—Amalima—with innovation. Through an innovative twist on traditional communal solidarity, the project worked to strengthen the ability of individuals, households and communities in the targeted districts to mitigate, adapt to and recover from shocks and stresses in a manner that reduces chronic vulnerability and facilitates inclusive growth, per USAID’s new resilience policy. The traditional communal solidarity practices of Amalima were revived and expanded in new ways in order to strengthen labor-poor households’ resilience alongside that of the entire community.  Apply innovative methodologies to increase uptake of improved practices. The Amalima team built on the groundwork laid by the USAID/FFP Promoting Recovery in Zimbabwe (PRIZE) program and focused on the uptake of improved practices, through two complementary approaches. The first was to understand and address the “why.” Why do farmers not adopt CA despite its benefits? Why do poor households not save in anticipation of the next shock? Why do mothers not exclusively breastfeed their children or use proper hygiene and sanitation practices? The answers to these questions lie in the practices’ convenience, affordability, feasibility, or labor intensity—all of which are different for women and girls than for men and boys. The project applied Trials of Improved Practices (TIPs), a formative research technique developed by the Manoff Group for nutrition communication, to nutrition as well as agriculture interventions. The second approach was to facilitate self-replication by leveraging social capital for systemic change. The project aimed to benefit not only the targeted households and communities but also their neighbors, through self￾replication of improved practices. The intent was for behavior change and its positive outcomes to spread from targeted wards to neighboring wards and even other districts through information sharing and cross-pollination, facilitated by project “look and learn” tours.  Mainstream sustainability. The project aimed to help create the enabling environment required to bring about equitable, inclusive, and sustainable change for communities. Hence, a focus on the sustainability of outcomes is woven throughout the project’s design. The community-led approach allowed households and communities to take ownership of their challenges and participate as drivers of their solutions. The empowerment of community members to take responsibility for working together and training each other in production techniques, and health and nutrition, ensured that knowledge was retained within the community. Reliance on local partners such as ORAP and Dabane Trust, and local service providers such as ZimAHEAD, brought local knowledge into the consortium while strengthening local implementing organizations to work in Zimbabwe long after the project’s end.  Actively promote gender equality and women’s empowerment. The project leveraged the social capital of Amalima collective action (which is most often applied by women) and built on strong, trust-based relationships between women to promote improved practices such as savings and exclusive breastfeeding. The gender specialist led a gender analysis and advised technical staff on ways to promote women’s empowerment and gender equality. With guidance from the gender specialist, technical staff members were responsible for ensuring that project technical assistance activities were tailored to the realities of women’s workloads, childcare responsibilities, and relatively low levels of financial literacy and business skills. This initiative included ensuring that trainings were given within villages and at convenient times for women to attend; that female agro￾dealers were trained, empowered and equipped to provide inputs and information to women farmers; and that labor-saving technologies were prioritized. Final Performance Evaluation of the Amalima Development Food Assistance Project in Zimbabwe Evaluation Overview 5 2. Evaluation Overview 2.1 Evaluation Purpose This report details the findings and recommendations of the endline evaluation of the Amalima project. The evaluation’s broad objective is to determine conditions in targeted areas prior to the start of new Title II programs. It is comprised of a representative population‐based household survey that collected data for required impact and outcome Indicators for Title II program intervention areas, and a qualitative study that provides depth and context, and serves to triangulate quantitative survey findings and analysis. The purpose of the endline evaluation is to measure the performance and development outcomes of the Amalima project. It is designed as the second step in a two‐part evaluation process, following the baseline at the beginning of the program (USAID 2015). Data collection occurred during peak harvest and gardening periods. The specific objectives of the endline evaluation are the following:  Determine the endline values of key impact and outcome-level indicators—disaggregated by awardee, age, and sex as appropriate—in addition to endline values of demographics in target areas and appropriate independent variables;  Conduct bivariate and multivariate analyses of impact and outcome indicators with independent variables identified for inclusion in the survey as appropriate, with results provided by awardee and the overall Title II country program area;  Gather qualitative data to ground‐truth survey data and provide contextual information on the overall food insecurity and malnutrition situation; and  Assess progress toward end‐of‐program targets for impact and outcome indicators. The endline evaluation was conducted by TANGO International with assistance from Jimat Consultants. Staff from FFP and the USAID Mission in Zimbabwe provided input and were involved throughout the process. The Evaluation Team consulted with the Amalima awardees to understand the program description and theory of change, obtain inputs for the quantitative survey instrument and qualitative study, and receive contextual information to properly develop a sampling and logistics plan. In discussion and coordination with FFP, TANGO provided draft and final versions of specific deliverables to the awardees for review and information. The evaluation results have multiple audiences. The findings are expected to have primary accountability and learning value for USAID (FFP/Washington, USAID/Zimbabwe, FFP Southern Africa Regional Office, and the FFP learning network), IPs, and their sub-partners. Additional stakeholders include Zimbabwean government officials from key collaborating ministry offices and regional committees, including the Matabeleland North and Matabeleland South provincial Drought Relief and Food and Nutrition Council. The findings, conclusions, and recommendations of the evaluation will be used by USAID/FFP to extract lessons learned and generate insights to inform the design of follow-on FFP activities in Zimbabwe and the southern region of Africa. Evaluation recommendations and findings may also be used by FFP internally to refine future Development Food Security Activity (DFSA) proposal guidelines and project policy. IMPEL | Implementer-Led Evaluation and Learning 6 Evaluation Overview 2.2 Evaluation Questions FFP provided evaluation questions to guide the design and development of the endline evaluation. In response to these questions, the evaluation team also referred to the baseline quantitative and qualitative data as a basis for comparison. The team assessed the technical viability of the evaluation questions and incorporated specific elements in the design and methodology of the endline (both the quantitative and qualitative components) to ensure the evaluation provides valid and reliable data and directly addresses the evaluation questions. In some cases, this method involved incorporating additional variables or strata in the design of the household survey and the qualitative component. Table 1 provides the primary questions and sub-questions used for the endline evaluation. Table 1: Primary evaluation questions and methods Criteria Main evaluation questions Sub-questions Evaluation method Impact 1. To what extent did the programs achieve the intended goal, objectives and results as defined by their Results Framework? 2. How did program activities improve the ability of beneficiary households and communities able to mitigate, adapt to, and recover from food security shocks and stresses? 1.1 Were there any important unintended outcomes, either positive or negative? 1.2 What were the main reasons that determined whether intended outcomes were or were not achieved, and whether there were positive or negative unintended outcomes? Which reasons were under control of the programs and which were not? 1. Quantitative bivariate analysis 2. Quantitative and qualitative Beneficiary satisfaction 3. How satisfied were beneficiaries with the programs? 3.1 What issues were most important to beneficiaries forming their perceptions of the programs? What were the key successes and challenges of the programs? Qualitative Relevance 4. How relevant was beneficiary targeting, considering the needs of the target population? 4.1 Were beneficiary targeting criteria and processes appropriate, transparent, and properly implemented? 4.2 Were the scale, type, and timing of the program activities appropriate to the needs of the target population? Qualitative Final Performance Evaluation of the Amalima Development Food Assistance Project in Zimbabwe Evaluation Overview 7 Criteria Main evaluation questions Sub-questions Evaluation method Effectiveness 5. How well were program activities planned and implemented? 5.1. What were the main factors that contributed to whether activities resulted in intended outputs and outcomes? 5.2. What quality standards were defined? How did the programs develop those standards? Quantitative and qualitative Coordination 6. To what extent did the programs coordinate with other food security and humanitarian programming, the host country government, and the donor? Qualitative Sustainability and Replicability 7. How sustainable are the programs’ outcomes? 7.1. What exit strategies were incorporated into program design? Were such strategies implemented, how were they perceived by the beneficiary population, and what were the strengths and weaknesses of the exit strategies adopted? Qualitative Cross‐cutting issues 8. How well were gender and environmental considerations integrated into program design and implementation? 8.1. Were they successful in meeting their stated objectives? How? Quantitative and qualitative Lessons Learned 9. What lessons can be learned future FFP and USAID Title II in Zimbabwe? Quantitative and qualitative IMPEL | Implementer-Led Evaluation and Learning 8 Evaluation Methods 3. Evaluation Methods This section describes the evaluation methods, with additional detailed information provided in Annexes C, D and E. It describes the mixed-methods approach, the analysis methods and how they have been used to answer the evaluation questions, and a discussion of limitations associated with the evaluation. The evaluation utilized a mixed-methods design that integrated data from multiple sources. It drew on data collected via a population-based survey (PBS) to measure current status and change over time for 21 key FFP indicators, and a companion qualitative data collection effort to provide context and understanding of participant perceptions of project achievements; unexpected changes in key outcomes and explanations for these results; and additional information from IPs and related stakeholders on project effectiveness, sustainability, unintended consequences, and lessons learned. The evaluation also used secondary data including IP performance monitoring data, key IP documents, and the midterm evaluation and project FY 2018 Detailed Implementation Plan Extension document conducted for the DFAP. The mixed-methods design utilized data collection protocols to collect primary qualitative data from DFAP participants and other key stakeholders, and additionally drew on key IP documents, monitoring information, and the mid-term evaluation to help interpret findings and provide support for recommendations. Table 1 in Sec. 2.2 indicates the evaluation method that corresponds to each evaluation question. 3.1 Quantitative Data Collection OVERVIEW The objectives of the quantitative portion of this evaluation are to provide endline estimates of FFP program indicators, to measure changes in indicators over the five-year program cycle, and to provide evidence to prioritize and refine interventions. The quantitative analysis follows a pre-post design in which the same questionnaire was administered in 2014, at the start of program implementation, and in 2019, following program completion. Pre-post designs provide for measurement and statistical tests of changes in indicators between the baseline and endline, but do not allow for attribution or causation. The data were gathered via an in-person PBS of 486 households in the four Amalima districts (see survey tool in Volume II, Annex K). Survey fieldwork took place from May 21 to June 6, 2019, as close as possible to the baseline data collection timeframe (late March through May). Data collection was scheduled close to the end of the program given weather constraints, namely, that the lean season coincides with the rainy season; the timing of data collection was thus designed to allow for probable access to all project areas. TANGO International and Jimat Development Consultants collaborated for survey training, household listing, and survey fieldwork. Surveys were translated into the most common local languages, Shona and Ndebele. Annex C describes the training and fieldwork in detail. POPULATION-BASED SAMPLE DESIGN The statistically representative sample was selected using a multi-stage clustered sampling approach (USAID 2015). The sampling frame for the endline study was constructed from the 2012 Zimbabwe census enumeration areas (EAs).1 Amalima project staff provided TANGO with a list of wards in each district and TANGO used these wards to identify all EAs for inclusion in the sampling frame. Stunting, one of several key measures of food insecurity, was used to compute sample size in the baseline and 1 The EA is the lowest census administrative level and typically includes 100-200 households. Final Performance Evaluation of the Amalima Development Food Assistance Project in Zimbabwe Evaluation Methods 9 endline surveys. Sample size is the minimum number of households necessary to detect whether stunting decreased to the project target rate of 25.2 percent (baseline value: 31.7 percent), a reduction of 6.5 percentage points. As shown in Table 2, the total target sample size is 540.2 Table 2: Information used to compute sample size Percentage of stunting at baseline (actual) 31.7 Expected percentage of stunting at endline 25.2 Design effect at baseline (actual) 0.8 Percentage of CU5 of the total population at baseline (actual) 13.8 Household size at baseline (actual) 5.5 Minimum required sample size (# CU5; computed) 294 Minimum required sample size adjusted for the number of CU5 per household (# HH; computed) 504 Non-response rate (estimated) 7% Final target sample size (# HH) (computed) 540 The minimum required sample sizes for the baseline and endline surveys were computed to provide estimates of key project indicators (stunting in particular) with similar levels of statistical precision over the two surveys. However, the minimum required sample size for the endline sample has been computed to be significantly smaller than what was estimated for the baseline for two reasons. First, at the time of the baseline, there was less available information about characteristics of project populations, so conservative estimates of key parameters were adopted. At the time of the endline, more accurate estimates of key parameters were available from the baseline results. In particular, the actual design effect (a parameter in the sample size calculation formula) was 0.89, less than one-half of the value of 2.0 used in the baseline calculation. The second reason is that the formula used at the baseline to estimate the number of households to achieve a sufficient number of CU5 resulted in a much larger number of CU5 being surveyed than was actually required for statistical purposes. For this reason, the required sample of households to be interviewed in the endline was adjusted downward to compensate for the oversampling of CU5 at baseline. These adjustments to the minimum required sample for the endline have resulted in significantly smaller required samples of households to attain indicator estimates that still have the desired level of statistical precision. For this reason, even though the endline sample is smaller than the baseline, the comparison of results with the baseline are statistically valid. Note: FFP quantitative performance evaluations use a PBS that is drawn from the general population in a DFAP implementation area. Accordingly, beneficiaries who directly participate in DFAP activities are not specifically targeted in the quantitative survey; rather, the sample is designed to be statistically representative of the entire population within the project implementation area, which includes DFAP participants and non-participants. This report includes an annex (Annex G) on participant versus non-participant data for key indicators for illustrative purposes only. It is important to note that the baseline and endline surveys are independent population-based samples, and there may be systematic, non-random differences between participants and non-participants. As a result, observed differences between participant and non-participant groups, whether positive or negative, cannot be directly attributed to DFAP activities. Further, as the PBS was not designed to allow comparisons between participants and non-participants, the interpretation of differences in indicator results must done judiciously. In the case of the Amalima survey, 54.9 percent of 2 Refer to Table 4 for non-response rates. IMPEL | Implementer-Led Evaluation and Learning 10 Evaluation Methods sampled households self-identified as directly participating in any project activity. However, experience from past FFP surveys suggests that self-reporting of participation may not be accurate, which weakens the validity of any comparison of outcomes. The analysis has sought to present more accurate information about project participants by consulting project performance monitoring data. DATA ANALYSIS The endline indicator calculation methods are the same as those for the baseline. The data to compute the indicators were collected using a questionnaire with separate modules for each indicator topic. Table 3 shows indicators, disaggregates and corresponding questionnaire module. See Volume II, Annex K for the quantitative survey questionnaire. Child stunting and underweight indicators were derived using WHO child growth standards and associated software (WHO 2011). Household, women’s and farmer’s indicators were computed following FFP guidelines (FANTA III 2015). Expenditures and poverty indicators follow World Bank guidelines (World Bank n.d.). Bivariate analyses were applied to the survey data, to compare changes in indicators from baseline to endline and differences between Amalima participant households and non-participants. Module I collected information about program participation, which was used to categorize households and individuals. Differences in means or proportions, as appropriate, test whether the change over time or between groups is statistically significant (at levels ranging from p<0.10 to less than p<0.001). Note that comparisons over time of monetary indicators are difficult because of the extremely high and variable rate of price inflation, large fluctuations in currency exchange rates, and multiple currencies used in Zimbabwe over the life of the project.3 Table 3: FFP endline indicators Food security indicators (Module H) Average Household Dietary Diversity Score (HDDS) Prevalence of households with moderate or severe hunger (HHS), overall and by gendered household type Average Coping Strategies Index (CSI) Food Consumption Score (FCS) Poverty indicators (Module H) Per capita expenditures (USD 2014) Percentage below the Total Per Capita Poverty Datum Line (TPCPDL)(a) Mean depth of poverty (using the TPCPDL) Sanitation and hygiene (WASH) indicators (Module F) % of households using an improved source of drinking water % of households using improved sanitation facilities % of households with soap and water at a handwashing station % of households practicing correct use of recommended household water treatment technologies % of households practicing safe storage of drinking water % of households with a handwashing station near a sanitation facility(b) Agricultural indicators (Module G) % of farmers who used financial services in past 12 months, overall and by sex % of farmers who practiced project-promoted value chain activities in past 12 months, overall and by sex 3 See relevant discussion of the poverty analysis in Section 3.4 Limitations. Final Performance Evaluation of the Amalima Development Food Assistance Project in Zimbabwe Evaluation Methods 11 % of farmers who used at least five sustainable agriculture (crop, livestock, NRM) practices and/or technologies in past 12 months, overall and by sex % of farmers who used at least five sustainable crop practices and/or technologies in past 12 months % of farmers who used at least three sustainable livestock practices and/or technologies in past 12 months % of farmers who used at least three sustainable NRM practices in past 12 months % of farmers who used improved storage practices in the past 12 months, overall and by sex Women’s health and nutrition indicators (Module E and Anthropometry) Prevalence of underweight women Women’s Dietary Diversity Score (WDDS) Average number of antenatal care visits by pregnant women Number of months pregnant at time of first ANC visit Children’s health and nutrition indicators (Module D and Anthropometry) Prevalence of underweight children under 5 years of age, overall and by sex Prevalence of stunted children under 5 years of age, overall and by sex Prevalence of wasted children under 5 years of age, overall and by sex % of children under age 5 with diarrhea in last two weeks, overall and by sex % of children under age 5 with diarrhea treated with oral rehydration therapy (ORT), overall and by sex Prevalence of exclusive breastfeeding of children under six months of age, overall and by sex Prevalence of children 6-23 months of age receiving a minimum acceptable diet, overall and by sex Gender indicators (Module J) % who achieve adequacy in ownership of assets, by sex % who achieve adequacy in decision-making for purchase, sale or ownership of assets, by sex % who achieve adequacy in decisions on credit, by sex (a) Based on Zimbabwe's Total Per Capita Poverty Datum Line (b) The denominator includes households with access to a sanitation facility. Sample Weights Sample weights were computed for each indicator, corresponding to a unique sampling scheme. The sampling weight is the inverse of the product of the probabilities of selection from each stage of sampling (EA selection and household selection). Separate weights were derived and adjusted to compensate for household and individual non-response, as shown in Table 4. For modules that asked questions at household level (Modules C, F, and H), weights were the inverse of the probability of EA selection, multiplied by the inverse of the probability of household selection, multiplied by the household inverse of the household response rate. For Modules D, E, G, and J that asked questions at the individual level, all eligible individuals were selected for the sampling weights also include the inverse of the individual response rate. IMPEL | Implementer-Led Evaluation and Learning 12 Evaluation Methods Table 4: Survey response rates (Amalima) Number Sampled Number Interviewed Response Rate (%) Households (Modules C, F and H) 540 486 90.01 Children 0-59 months of age (Module D) 342 315 92.1 Women 15-49 years of age (Module E) 480 325 67.72 Non-pregnant women 15-49 years of age (Module E Women’s Anthropometry) 379 352 92.9 Farmers (Module G) 604 599 99.2 Primary male decision-maker (Module J) 298 239 80.23 Primary female decision-maker (Module J) 453 434 95.8 1 Non-responses were mainly due to household members temporarily migrating or being out of the house from morning until late at night and unavailable for interviews. The non-response rate of 10 percent was higher than the 7 percent anticipated in the sample size estimations. However, the sample size for CU5 exceeded the target of 294 children for the Amalima project. 2 Testing did not show any systematic bias that would affect results. There were no statistically significant differences in project participation, HDDS, CSI, improved water or improved sanitation between women ages 15 to 29 who participated in the survey and those who did not. 3 Testing showed some differences between respondents and non-respondents. Respondents were better off than non￾respondents in that they had higher HDDS. Program participants were more likely to respond. There were not statistically significant differences in CSI, improved water or improved sanitation. 3.2 Qualitative Data Collection OVERVIEW The endline qualitative study was conducted between July 15 and August 16, 2019.4 The study involved four data collection methods: focus group discussions (FGDs), key informant interviews (KIIs) at village, ward, and district levels and with IPs and individuals in the private sector in Bulawayo; asset observations; and desk review. Qualitative data collection utilized purposive sampling and semi-structured FGD and KII protocols. This section describes these methods as well as team composition and data analysis methods. EVALUATION TEAM The evaluation team was comprised of four evaluators, each of whom had one translator and one notetaker, for a total of 12 team members. Three of the evaluators were international (U.S., France, Uganda) and one was from Zimbabwe. Three of the four were male. The technical specializations represented by the four evaluators include: applied environmental anthropology (emphasis on food security, agriculture development, climate change, natural resource management); infrastructure engineering (emphasis on marketing systems, post-harvest technology, irrigation and drainage, and soil and water conservation); health economics and services (emphasis on nutrition, public health management, and monitoring and evaluation); and groundwater geophysics and hydrogeology (emphasis on groundwater development, water quality, and sustainable WASH projects). All eight translators and notetakers (four females; four males) were from Zimbabwe and were fluent in Ndebele. 4 The fieldwork portion in the districts took place from July 19 – August 10. Final Performance Evaluation of the Amalima Development Food Assistance Project in Zimbabwe Evaluation Methods 13 SAMPLE DESIGN Village/ward selection for qualitative data collection was done using purposive sampling. It aimed to maximize coverage of a range of different project activities5 and variation in village distance (in kilometers and total travel time) to a regional town, extending from Gwanda, Plumtree, or Tsholotsho Centre regional towns. Sampling was also opportunistic in terms of field and timing logistics: given the team was travelling to and from regional towns or district centers, field visits were restricted to a maximum of two-hour one-way travel time. An additional consideration was project personnel availability, as in some cases the team needed Amalima’s assistance to gain approval of Village Heads to conduct the study, and most of the remote areas do not have gravel or sand roads, so each team had to be guided by field personnel for the duration of the day’s field activities. The sampling design used for the FGDs and those KIIs that were conducted with project participants was purposive in two ways: i) the team held FGDs with project participants only, and ii) in terms of breadth of coverage of types of project activities. The evaluation team also requested that the district and field￾level personnel arranging FGDs do so to reflect the range of strengths and weaknesses in programming across the four districts comprising the project. Village selection for the qualitative data collection aimed for geographic, spatial representation to the extent possible, but it is noted that purposive qualitative data collection, particularly within short timeframes, is inherently non-representative (Bernard 2017). Moreover, participants who took part in FGDs and were willing to share their views are not representative of all project participants, or may be different in key observable or unobservable ways. The evaluation team sought to mitigate the potential for biased results by asking field-level personnel to recruit project participants with a range of experiences and beneficiary roles for the qualitative data collection, and also by triangulating information across different types of project beneficiaries and stakeholders. Field and district personnel were also interviewed as a means of or triangulating information gathered from project participants. Table 5 summarizes the distribution across districts of KIIs, FGDs, and asset observations. Complete details by district, including a break-out of types of KIIs and FGDs (both lists indicating number of males/females), and assets observed, are found in Annex E. Table 5: Summary of qualitative data distribution, by district Activity Districts Tsholotsho Mangwe Buililima Gwanda Totals KIIs 13 9 9 14 45 FGDs 18 12 10 7 47 Asset observations 11 8 9 6 34 HQ/private sect. KIIs 25 METHODS Focus group discussions. The evaluation team conducted a total of 47 FGDs across the four project districts in 34 villages located in 21 wards. The FGDs comprised 419 participants in total (86 males and 333 females). Working with the assistance of the Deputy Chief of Party and district supervisors who helped with contact information, the Evaluation Team organized types of FGDs such that the majority of project activities would be represented in these group discussions and across the four project districts. The FGDs were organized with the assistance of district-level project personnel, community leaders, and DFAP field agents, and on average were comprised of 8-12 participants. Working with their respective 5 Based on IP data on activity implementation by village IMPEL | Implementer-Led Evaluation and Learning 14 Evaluation Methods team evaluator, one translator along with one notetaker conducted the FGDs in Ndebele, the predominant language in each of the DFAP implementation areas. Responses to questions were translated for the team evaluator during the course of the KII or FGD.6 FGDs were generally held outside at common gathering areas in the respective villages. Semi-structured instruments with questions organized by the evaluation team were used to guide the discussion (see Volume II, Annex K). All FGDs were recorded with participants’ verbal consent. The recordings of the FGDs were used in data processing when a question arose from the two sources of notes (interviewer and notetaker).7 Key informant interviews. The evaluation team conducted 45 KIIs in the villages and wards, and an additional 25 with IPs and the private sector in Bulawayo for a total of 70 KIIs (42 males and 28 females). The interviews followed a semi-structured format to allow for follow-up questions and flexibility in the discussion. The KII protocol was structured to gather information on the extent to which activities have been achieved; any gaps or challenges the project experienced over the course of the five years; what factors promoted or inhibited project activities and outcomes; perspectives on the effectiveness of project interventions and targeted groups; quality of services provided; motivations and capacity to demand and sustain services; and the projected sustainability of project interventions and outcomes (see Volume II, Annex K for KII protocols). Most KIIs were not recorded, as the one-on-one pace of the conversation is slower and much easier to follow and record. Asset observations. The team made 34 asset observations throughout the four districts, which included site visits to infrastructure projects and demonstrations of eco-stove use, livestock herding practices, lead mother visit observations, and training illustrations. See Annex E for summary details. Desk review. The IPs provided a collection of documents to the evaluation team, including quarterly and annual reports, training documents, field studies addressing particular initiatives of the project (e.g., gully reclamation, gender mainstreaming and farming as a business), monitoring formats, maps, and learning units. The purpose of the desk review of IP and available secondary documentation was to identify key findings and explanatory factors from IP reports and internal M&E data pertaining to each of the outlined evaluation questions. Examination of key documents before data collection assisted in the design of some of the evaluation questions. The secondary information was also used as a source of triangulation for qualitative data provided by project beneficiaries and KIIs, or to help interpret or provide explanatory context for both PBS and qualitative results. ANALYSIS, CODING AND INTERPRETATION METHODS The FGDs were recorded, with consent from participants, and transcribed from Ndebele into English. Transcribed FGD data and detailed FGD notes were then provided to all team members according to key themes, evaluation questions, and ultimately, sections of the report. KII notes were summarized using standard content analysis techniques. For both KIIs and FGD data, analyses summarized common trends and patterns to highlight project-, sector-, and gender-differentiated trends, and to specifically identify examples of perceived strong areas of probable sustainability, unexpected outcomes, positive deviance, and most significant changes and impacts. 3.3 Integration of Quantitative and Qualitative Findings As detailed in the previous sections, mixed-methods endline evaluation utilized both quantitative data collected through the PBS and qualitative data collected through KIIs, FGDs, asset and process (e.g., eco- 6 See qualitative tools in Volume II, Annex J. 7 Final transcripts were not made due to budget constraints. Final Performance Evaluation of the Amalima Development Food Assistance Project in Zimbabwe Evaluation Methods 15 stove cooking) observations, and document review. This collective information is integrated in several ways in the evaluation to interpret findings and provide support for recommendations. Given some delay in receiving the preliminary data results during the qualitative data collection phase of the evaluation,8 the more systematic integration of various data sets occurred during the process of composing the report, after all field data collection was complete. When the qualitative team received the preliminary data results toward the end of their field data collection process, the team reviewed the data at that time and used the information as a starting point for the analysis of what was learned during the qualitative study. The process of integrating the quantitative and qualitative findings was as follows: First, the qualitative team reviewed the IPTT indicator results by SO, focusing initially on the implications of the results relative to each objective and their respective intermediate results (IRs). This review included the Minimum Acceptable Diet (MAD) data and additional analyses conducted to explore data addressing particular IRs. For example, a breakdown of the frequency of use of individual cropping and livestock practices was included in assessing results for IR 1.1.2 (livestock management improved) and IR 1.1.3 (soil fertility and soil moisture improved). The IPTT indicator asks respondents to indicate whether they have “used at least five improved cropping practices in the past 12 months,” but this does not provide the breakdown of their use of each specific practice. In this example, certain practices address the IR more effectively than others, hence, the need for the breakdown of the data used in the indicator. Second, at the end of the qualitative field study, when the team was still in Zimbabwe and the preliminary data results were released, the team held discussions about the IPTT results and their implications for the broader intent of the project, i.e., how participants who were mostly engaged in SO3 activities were impacted by project results of SO1 or SO2 activities and vice versa. These team discussions and their conclusions are reflected specifically throughout Sec. 4.2-4.4 of this report. Third, the integrated analysis of quantitative and qualitative data, along with consideration of other information sources, is synthesized in the report sections that address evidence of cross-integration of initiatives within the project (Sec. 4.5-4.12). For example, how the difficulties of recurring drought conditions and low yields in turn impacted diet diversity and child stunting—this question is analyzed in reference to gender empowerment issues, environmental considerations, and unintended outcomes. How DRR activities influenced livestock and crop production, and the willingness of traditional leaders to support the Male Champion initiative that ultimately provided women more time to participate in SO1 activities, particularly goat husbandry, is a second example of exploring evidence of cross-fertilization. These discussions, oriented around assessing the broader impacts of synthesized project initiatives (intentional or not), are specifically discussed throughout the latter sections of the report. 3.4 Limitations ODK programming errors. ODK programming errors caused skips over parts of several questions. Consequently, data are missing from three modules. i) In the household consumption module (Module H), which collects data on weekly, monthly and annual consumption and expenditures, data are missing for six out of seven categories of monthly expenditures. Analysts used baseline data to impute missing monthly data, then to estimate per capita daily expenditures and poverty indicators. Refer to Appendix D for imputation methods. ii) In the children’s nutrition module (Module D), one of the variables used to compute Minimum Adequate Diet (MAD), meal frequency, was skipped. Analysts imputed meal 8 See Sec. 3.4 – Limitations. IMPEL | Implementer-Led Evaluation and Learning 16 Evaluation Methods frequency and used the imputed values to estimate MAD. To be comparable, imputations include both baseline and endline values for MAD. iii) In the gender module (Module J), one of the asset categories, mechanized farming equipment, was missing from the endline survey. This did not affect indicator values. Analysts re-estimated baseline values of the indicator percentage who achieve adequacy in ownership of assets omitting the missing variable, and found only a very small difference (less than 0.01). This brought survey response rates below the 10 percent estimate for survey Module E, Women 15-49 years of age, and for Module J, Primary male decision-maker (see Volume II). Timing of quantitative data results. Given the short timeline from completion of the PBS to the beginning of the qualitative survey, the evaluation team received preliminary results of the PBS during the qualitative data collection period, and those findings were used to help direct follow-up probing on some issues, e.g., types of sustainable agricultural practices. Poverty analysis. As noted above in the Data Analysis section, comparisons over time of monetary indicators are problematic because of the extremely high and variable rate of price inflation, large fluctuations in currency exchange rates, and multiple currencies used in Zimbabwe over the life of the project. Expenditure and poverty indicators at endline should be interpreted with caution for several reasons: (1) Reporting issues: Expenditure totals may underestimate the contribution of food and other items that households receive from remittances or barter. Data collection methods follow World Bank guidelines and are not specifically designed to measure contributions of remittances or barter. For each food item, enumerators ask whether it was consumed, then for each item, how much of what was consumed came from purchases, own-production, or gifts. Foods received from bartering or remittances should be counted as gifts. However, survey questions are not worded to explicitly ask about consumption items from barter or remittances. (2) The unstable financial situation in Zimbabwe contributes to issues with the indicator computation. At the time of the endline, Zimbabweans were experiencing sharp price increases and fluctuating currency exchange rates, and the government had recently changed the official currency, moving from the US dollar (USD) to the Zimbabwe dollar. Data on all of these elements are part of the calculations. If these are changing, the estimated value is unstable and not very reliable. (3) Between baseline and endline, along with changing the official currency, the Total Per Capita Poverty Datum Line (TPCPDL) changed from being denominated in USD to Zimbabwe dollars and the poverty line was increasing rapidly. During endline survey fieldwork the daily per capita poverty line increased by almost 4 Zimbabwe dollars. This also contributed to the unreliability of expenditure and poverty indicator estimates. Difference in seasonal timing of data collection between baseline and endline PBSs. The baseline data collection for the PBS took place March 24 – May 1, 2015, and the endline household survey was conducted May 21 – June 6, 2019. It is possible that this slight difference in seasonality across the two rounds of data collection could contribute to differences in some of the indicator estimates. The main variation is in the green harvest, which is more available in March/April than in May/June. Also, depending on the crop, in May some farmers begin land preparation in anticipation of the rainy season and planting; the lean season for farmers generally occurs somewhat later in the year. In particular, dietary diversity, prevalence of household hunger, and prevalence of diarrhea indicators may be more sensitive to this difference. In addition, the end of March through June are harvest months in which it is not uncommon for seasonal migrants to have migrated out of villages to seek wage work. Biases in qualitative data collection. Some project target goals are inflated due to the self-reporting nature, biased questionnaire design, knowledge of expected responses, and pressure from staff to produce good results. Some evaluation topics, such as perceptions about the beneficiary selection processes that occurred early in program implementation, may be difficult for respondents to remember accurately as time Final Performance Evaluation of the Amalima Development Food Assistance Project in Zimbabwe Evaluation Methods 17 passes. Careful construction of question wording on interview guides, probing for clarification, and triangulation across FGDs and KIIs were used to mitigate the potential for recall bias that could influence results. The qualitative evaluation teams had little indication of serious issues related to this possible constraint during the qualitative data collection. IMPEL | Implementer-Led Evaluation and Learning 18 Evaluation Findings 4. Evaluation Findings 4.1 Targeting The targeting criteria used to identify participants were sound and appropriate relative to the project’s objectives. The project targeted 65 wards in four districts identified in Matabeleland North and Matabeleland South Provinces – Tsholotsho, Gwanda, Mwange, and Bulilima – to reach an estimated 60 percent of the population. Amalima is designed to improve food and nutrition security. The selection criteria for project participants aligned with these goals and included food insecure households, stunting percentages among young children, productivity potential for improvement interventions to would-be successful households based on prevalence of farming and livestock rearing households, market access in terms of road networks and proximity to towns, hydrological characteristics that would support the planned water and sanitation interventions, and more broadly, agro-ecological and livelihood zones that allowed for crop and livestock husbandry improvements. At project start, all four districts had a child stunting rate of at least 32 percent, a range of 25-57 percent food insecure households, fair access to markets, high dependence on crop cultivation, and some subsistence livestock rearing (Amalima 2014a). With this being said, there were some challenges in terms of targeted populations choosing to participate and some adjustments in targeting certain subgroups of the populations over the course of the project. First, there were early negative perceptions of Amalima in communities as residents came to realize it was not an emergency “handout” project (which is what they were accustomed to) and required active participation. While the activities benefitted those with basic resources (land, labor, enough money to invest in Village Savings and Lending [VS&L], etc.), the benefit was less for those furthest away from activities or with transport issues for SO1, especially women with young children and less household help. No one was actively excluded by the project or its personnel, and no one was worse-off due to the project, but some benefitted more than others, and occasionally resentment was indicated in FGDs, most often about the structure of the voucher program. Project participants did not find the selection process for the Household Asset Voucher activity clear. While the areas to be covered were decided with government stakeholders, on a most-acute-needs basis, the selection process for individuals to receive subsidized vouchers was unclear to local community members. In addition, after observing low levels of participation from youth and males early in the project, Amalima devised additional and more￾focused targeting approximately mid-way through the project for these two subgroups. While young mothers were targeted early in the project for the nutrition initiatives associated with SO3, they were targeted much later in the project in terms of participating in SO1 activities, particularly goat livestock groups and the nutrition gardens. The Male Champion activities were also initiated later in the project; 9 the demographic and gender role issues that were addressed in this initiative have been well entrenched in society in these districts for many decades and could have been included from project onset. 9 This is discussed in detail in Sec. 4.4. A lead farmer in Tsholotsho district, Ward 9, explained project benefits: Amalima has trained us so that we can now provide for our families and have surplus to sell to the grain marketing boards. The project trained us to work together as one and we have developed in many ways, for example, what we get from the VS&L groups. We are now independent in terms of food production. Final Performance Evaluation of the Amalima Development Food Assistance Project in Zimbabwe Evaluation Findings 19 4.2 SO1: Household access to and availability of food improved In a region that has experienced recurrent drought for over a decade, sustained availability of nutritious foods requires an increase in yields of appropriate crops and nutritious garden vegetables, an improvement in livestock production, and a strengthening in household and community capacities to reduce risk and effectively respond to shocks. Amalima’s food production approach was watershed￾based, where the NRM/DRR coordinators worked with the water team and communities to facilitate land-use planning. They identified watershed areas requiring rehabilitation and management, applied cost- and labor-effective rehabilitation approaches, and worked to improve watershed management. Vulnerable households’ access to food depends on its availability and price in local markets and the purchasing power households may or may not have to secure it. The project worked to improve food distribution from surplus to deficit areas and to increase the incomes of poor households through push and pull approaches that included increases in livestock production and improvements and/or establishment of new nutrition gardens. To strengthen resilience and mitigate risk, the project trained communities in a range of strategies to diversify household incomes, and through the formation of VS&L groups, to build their capacity to save and to access financial services. Indicators relevant to this SO are Household Dietary Diversity Score (HDDS), Household Hunger Score (HHS) and Food Consumption Score (FCS). The HDDS is a count of foods from 12 food groups consumed by household members in the 24 hours prior to the survey (FAO 2010). HDDS ranges from 0 to 12, with lower numbers indicating less dietary diversity, and can be interpreted as an indicator of food access and a proxy for socioeconomic status. While the survey data indicate that HDDS decreased slightly from 5.3 at baseline to 5.0 at endline (p<0.1) (Figure 13, Annex H), in contrast to the quantitative data, PLW FGDs indicated increased consumption of nutritious foods including green leafy vegetables, livestock and poultry by some households during the project period. In addition, the interviews showed that communities had dropped taboos prohibiting children’s consumption of meat and eggs. The percentage of households with moderate or severe hunger significantly decreased from 29.3 percent at baseline to 20.1 percent at endline (p<0.01) (Figure 14, Annex H). The largest decrease was among female-headed households (Annex F). Food Consumption Score (FCS) is an indicator of dietary quality and how regularly people eat. The percentage of households with poor and borderline FCS increased from 31.4 percent at baseline to 42.1 percent at endline (p<0.05), thereby indicating worsening food security (Figure 15, Annex H). The increased prevalence of households with low and borderline FCS showed that Amalima was constrained in its efforts to improve household diets. This was attributed to the low purchasing power for many households given their undiversified household incomes, national economic policies, and currency adjustments. Additionally, the recurring droughts (in four of the project’s five years) led to limited availability of foods, minimal agricultural and livestock productivity, as well as low HDDS. IR 1.1: AGRICULTURAL PRODUCTIVITY INCREASED While Amalima was well intentioned in improving crop production, 10 with the recurring droughts (two two-year periods over the course of the five-year project), households were able to hold steady over the course of the project but with little additional gain in yields, as reported in the annual results reports over course of the project. This result is considered positive given the serious challenges both the drought and the economic situation in Zimbabwe posed. Conditions for raising livestock were 10 To be discussed in greater detail under IR 1.1.3 and IR 1.1.4. IMPEL | Implementer-Led Evaluation and Learning 20 Evaluation Findings undeniably improved, particularly in the areas of water provisioning, improved grazing management, and disease control. Participants across all four districts and in every farming or livestock-associated FGD cited the project’s extensive training programs on soil fertility, CA, vegetable cultivation, livestock grazing and health management, and bookkeeping, as the most successful initiative Amalima offered household members and local communities over the course of the project. As one female farmer in Gwanda professed, “I benefitted a lot from the agriculture trainings, mostly to learn to have hardier response plants to deal with the dryness—better products all around in terms of agriculture. I thank Amalima for all they have given us. We have been learning more how to help ourselves and I am very happy for this.” Lead farmer FGDs acknowledged the positive contributions the voucher program made given the materials provided, including fencing materials, wheelbarrows, and basic gardening tools, though some expressed disappointment that the voucher program, as they perceived it, did not distribute vouchers equitably across all communities. IR 1.1.1 Access to water resources for agricultural production improved The project chose to emphasize on water provisioning for livestock and crop production and gardens, given the recurrent chronic drought conditions in this part of Zimbabwe and the well-documented shortages and declining levels of food and nutrition security over the 10 years prior to Amalima’s implementation (Zimbabwe National Statistics Agency 2013). All activities around water provisioning for livestock (dam rehabilitation or new construction) were well designed and extended, with high to moderate success in terms of the logistics of physical location (newly constructed dams) relative to community access, and effective function. Figure 2 shows the assets the project has either rehabilitated or newly constructed. Given the extended drought conditions, the provisioning of water through a range of irrigation systems for nutrition gardens provided a sustainable food supply in terms of vegetables and some grains, and hence, a degree of diversity of food sources for local (community and ward-level) households. Over the course of the project there was a transition from provisioning gardens with drip irrigation systems to installing solar-based systems for pumping water from nearby water sources and/or boreholes. This alteration occurred given technical issues with the drip irrigation systems installed initially in Tsholotsho district (HQ staff KII). 11 11 Solar and drip irrigation are not mutually exclusive. There are some irrigation schemes where the pumps are solar powered and the water is dispersed using drip irrigation. Final Performance Evaluation of the Amalima Development Food Assistance Project in Zimbabwe Evaluation Findings 21 Figure 2: Amalima asset provisioning Source: Amalima. 2019a. Amalima Wards and Asset Maps. Bulawayo: Amalima project, USAID, CNFA, Amalima. A female lead farmer FGD in Tsholotsho district indicated the group’s strong appreciation for the project’s hands-on training in agriculture activities. They explained their improved confidence in being able to provide for their families and to have surplus to sell in the local markets. The water supply emphasis was given dams for watering livestock and irrigation systems for nutrition gardens. Water provisioning for agricultural field crop production was not part of the project; rather, agricultural crops and techniques that address issues of soil fertility, erosion, and drought conditions were introduced through trainings and demonstration plots. These met with mixed results, as cultivation on the demonstration plots early on did not do well. As explained by HQ staff, given the recurrent droughts, roughly two years into implementation, the project made the decision to transition from supporting largely maize and sorghum production to also supporting groundnuts, along with livestock and foods cultivated in the gardens. IR 1.1.2: Livestock management improved The BL/EL (baseline/endline) data indicate a statistically significant increase in the percentage of farmers who used at least three sustainable livestock practices and/or technologies in the past 12 months, from 28.2 percent in 2014 to 49.6 percent in 2019 (see Annex F). The percentage of farmers using sustainable agriculture (crop, livestock, NRM) practices and/or technologies increased from baseline to endline (see Figure 6, Annex H). The biggest gains were in the percentage of farmers using at least five sustainable crop practices and/or technologies, and farmers using at least three sustainable livestock practices IMPEL | Implementer-Led Evaluation and Learning 22 Evaluation Findings and/or technologies. Both increased to about half at endline. These results are clearly validated through the emphasis placed on providing infrastructure, feedstock guidance, grazing management instruction, and veterinary services throughout the four districts. Figure 7, Annex H, provides the breakdown and prevalence of various livestock practices over the previous 12-month period; four of the practices were used by more than 30 percent of livestock owners. All livestock and paravet FGDs acknowledged the project’s intent to develop a (livestock) asset base from which households could draw when coping strategies were required in the case of a shock or an immediate need for cash. The trainings provided for livestock improvement are extensive, thorough and well-organized (USAID, CNFA, Amalima 2016a-e). All participants of all FGDs conducted with livestock groups (cattle and goats) reported they had participated in every livestock training provided by Amalima. The wide range of topics—from feedstock and nutrition management, salt lick production, castration, illness recognition and disease management, as well as recordkeeping—instilled confidence in their own capacity to manage their livestock more effectively. As one lead livestock farmer in Tsholotsho district explained, “I have the skills and knowledge. Amalima came and has filled the knowledge gap. I am happy because I can function on my own, I can do the trainings on my own.” Dam AMC (Asset Management Committee) FGDs in Mangwe and Gwanda districts revealed strong support for the rehabilitation of dams or construction of new weirs given the substantial increase in secure and relatively stable water supply for livestock, particularly during drought periods. They reported they were no longer forced to travel long distances to water livestock; the livestock remained close to villages and homes, and residents also had access to water for various agricultural purposes. One FGD in Mangwe district reported problems with leakages in one corner of the dam they manage, which they attributed to faulty construction. They calculated the resources needed to repair the dam but explained they had no funding for the work at that time. They also expressed their concerns about buildup of siltation above a dam in their area, attributing this problem to movement of livestock into the river. The siltation then had to be removed, which took time and labor away from their other work. Amalima’s parallel focus on the rehabilitation of existing dip tanks or the construction of new ones (see Figure 2 above) is deeply appreciated by livestock owners as expressed in livestock FGDs across all project districts. A number of older dip tanks have been non-functional for up to 5-8 years (some as long as 15 years) so they were committing significant travel time to protect their cattle. Ideally, they dip cattle twice a month in winter and four times a month in summer, so time necessary to reach a dip tank competes heavily with other labor requirements (dip tank FGD in Mangwe District and dip tank observations in Bulilima and Gwanda districts). The dip tank FGDs reported the payment of 5 percent to the AMCs—from levies paid to the Department of Veterinary Services—to be satisfactory for minor maintenance tasks, but commented this amount did not meet the cost of upgrading facilities. Amalima also provided resources to increase the number of paravets across all wards of the project. Amalima’s required paravet trainings were extensive; the opportunity to become a paravet was extended to lead livestock farmers. Given the depth of technical training, being a paravet carries a certain status locally and recognized responsibility. Every individual paravet (three FGDs, one each in Bulilima, Mangwe, and Gwanda) acknowledged the level of commitment they felt toward their role, District Veterinary Services staff (Gwanda district in particular) explained the difficulties of choices made regarding which dip tanks to rehabilitate where, as is still the case, new dip tanks may not be approved given limited resources: Everyone in these districts is vulnerable and deserves water and veterinary services, but there is only a certain amount of resources to go around. (KII, VET Extension Supervisor, Gwanda). Final Performance Evaluation of the Amalima Development Food Assistance Project in Zimbabwe Evaluation Findings 23 given the opportunity to gain the knowledge and skills the project’s comprehensive training had provided them. Paravets interviewed by the qualitative team report they do not charge for their services (this was also stated in an Agritex KII) and would find it particularly difficult to do so in their local communities, although Amalima encouraged them to collect a small fee and to have livestock owners pay for medicines and the chemicals for injections. Although other sources report that the paravets do charge for their services, the qualitative data suggest that receiving compensation from clients is not consistent. The original paravet toolkit was described by new trainees as functional, but it was noted the consumables require replenishment by their own means once initial supplies are expended; most acknowledged they do not have the means to do this, at least not during their first year. IR 1.1.3: Soil fertility and soil moisture improved All four project districts have dryland soils that are deficient in phosphate, organic materials, and other nutrients. The project was designed to improve soil fertility through practices devised to conserve soil and water such as U-shaped planting pits, tied ridges, terracing, and mulching with crop residues, grass and mopane leaves (see Figure 8 in Annex H for use levels of these practices). The project’s chief agronomist and NRM/DRR coordinators developed training materials for CA to promote those techniques and other improved production practices for homesteads, and for rehabilitation of communal arable and grazing land. When asked about improving soil fertility, the farmer FGD participants talked most about potholing, and were able to adequately explain what it was and its benefits. For one lead farmer, “It serves to put fertilizer where it is needed the most for plant growth, so it helps the plant but also the soil” (Tsholotsho district). Potholing is a traditional technique in these districts; for this particular technique the project was working to improve a well-accepted and commonly used technique, which may be why farmers talked about it more often than other techniques they were learning for the first time. It was puzzling to see indicated in the survey data that potholing was only used by 12 percent of the farmers (see Figure 8 in Annex H: according to the quantitative data, the practices listed above were all used by relatively low percentages of farmers. This result is discussed in more detail in the section below relative to low adoption rates of certain CA techniques. IR 1.1.3.1 Practice of conservation agriculture increased The BL/EL data results indicate a statistically significant increase in those farmers who used at least five sustainable crop practices and/or technologies in the past 12 months, from 28.2 percent in 2014 to 50.2 percent in 2019 (see Figure 6, Annex H). 12 Amalima’s intent to increase the practice of CA was supported 12 We conducted additional analysis (not shown) comparing males to females for each of these five indicators: value chain activities, sustainable agriculture practices, sustainable crop practices, sustainable livestock practices, and sustainable NRM practices. We also compared males to females in terms of counts for any of those practices (“count” refers to the average number of practices adopted per farmer). There was no statistically significant result for any of these comparisons. A lead farmer in Tsholotsho district who transitioned to a VAC explains his challenges over the course of the project: As a lead farmer, I was responsible for training over 110 farmers and I have trained 11 farmers to also be trainers. I have been to many trainings and I have held a lot of trainings. But this is the best from Amalima, the knowledge. My biggest challenge is that a few people aren’t motivated, so I spend some of my time encouraging everyone to take part through their successes and share with those who even didn’t work, as an example. A person can request something from you today, but there is an understood system of reciprocity where you can’t keep taking only. You must also have to give. When I was chosen to be a VAC, that kind of example became even more important. We are supposed to figure out why farmers don’t want to do certain things that are good for them. Some of them only want to do what they know they’ll get the most, and not what’s good for the long term. IMPEL | Implementer-Led Evaluation and Learning 24 Evaluation Findings through both the provision of four training manuals on CA (USAID, CNFA, Amalima 2016 f-i) and hands￾on training to introduce new skills demonstrating CA techniques. Lead farmer FGDs demonstrated enthusiasm for CA trainings and demonstrated knowledge of certain CA techniques, in particular mulching, pot holing, and fertilizer application. Figure 8 (Annex H) indicates the breakdown of techniques used by farmers across all four districts over the past 12 months (relative to administration of the PBS). Conversations with district and HQ staff indicate high levels of participation in the CA trainings yet annual project reports indicate low adoption rates, due most likely to low yields associated with chronic drought conditions, compounded by inaccessible inputs. Ultimately, the Village Agriculture Coordinator (VAC) role was created to focus on behavior change approaches in an attempt to fill the gap between trainings and adoption of CA techniques, which were largely due to input accessibility and input availability. The CA trainings were well attended but adoption rates were estimated between 20-40 percent across the four districts (HQ staff KII). In FY 2018, Amalima trained approximately 400 lead farmers to become VACs (USAID and CNFA 2017a), whose initial task was to identify the challenges to CA adoption, e.g., knowledge, behavior, input access, or social factors. The VAC was to determine what the behavioral issues were locally; they also served as focal points for communication and coordination of project agricultural activities with the Amalima Field Officer and ward-level AGRITEX agricultural extension officer (USAID and CNFA 2017a). As members of the local community, VACs were more available to support lead farmers than project Field Officers and extension officers, who visit regularly but are not present on a daily basis (KII AGRITEX, Mangwe and Bulilima districts). They also planned to serve as long-term community resources for good agriculture practices, work with AGRITEX, and future agricultural projects in their areas after Amalima (VAC FGDS, Bulilima and Gwanda districts; KII AGRITEX, Bulilima). FGDs with farmers indicated farmers respected VACs and were willing to work with them. They also showed a clear understanding of the role of the VAC relative to the roles of the agriculture extension agents who work for AGRITEX. A VAC FGD in Gwanda district indicated they felt there were still some significant challenges as the project nears its end, in terms of delineation of roles. VACs were to also assist extension workers as “foot soldiers,” but initially, there was resentment from the AGRITEX extension workers as the VACs had more knowledge gained through Amalima trainings on a range of topics. During the qualitative study, ward-level meetings had begun with extension workers and VACs; they met to exchange information, review proposals for action, and coordinate their work. IR 1.1.3.2 Use of organic and inorganic fertilizers by male and female farmers increased Of the 50.2 percent of farmers who reported using five or more sustainable cropping practices over the past 12 months (Annex F), 44.5 percent reported using manure, and 25.6 percent reported using micro￾dosing13. The project also promoted the application of small, affordable quantities of fertilizer with the seed at planting time or as top dressing 3-4 weeks after emergence. Micro-dosing reduces fertilizer costs and is an appropriate technique for use in moisture-stressed crops (KII lead farmer, Tsholotsho; lead farmer FGDs, Tsholotsho and Bulilima districts). Compost was rarely used for dry grain cultivation, but rather was sometimes applied in the nutrition gardens (garden FGDs, Tsholotsho and Mangwe). 13 The baseline evaluation did not provide a breakdown of the crop practices data by practice, thus comparing specific practices, in terms of increase of use, over the course of the project is not possible. Final Performance Evaluation of the Amalima Development Food Assistance Project in Zimbabwe Evaluation Findings 25 IR 1.1.4: Cultivation of a diverse range of improved crop varieties by farmers increased While maize remains Zimbabwe’s staple crop and is grown by approximately 80 percent of smallholder farmers in the country (Food and Nutrition Council 2011), including many farmers in the targeted project districts, yields have remained very low over the last decade. The project developed and implemented a drought response strategy during FY 2016 that scaled up livestock management and irrigated-crop trainings to mitigate the effects of the drought, and deemphasized CA trainings when rain-fed demonstration plots failed (USAID, CNFA, Amalima 2017b). Even yields for drought-tolerant millet and sorghum failed to meet targets due to poor rains, so the project decided to continue to promote the production of animal-source foods and nutritious crops through the gardens and Healthy Harvest training (Food and Nutrition Council et al. 2015) to meet the target of increased consumption of locally available, nutrient-rich foods. Participants in the lead farmer FGD in Tsholotsho explained, given the recurring droughts, they preferred to focus their activities on garden cultivation and not invest in new techniques to produce different varieties of grain crops because the lack of rain meant this labor was largely a waste of time. Amalima also supports vegetable gardens at homesteads so people have better diets, as well as chicken farming, both home and commercial (meat and eggs). Many participants have vegetable gardens and everyone is encouraged by the project to have one or to share labor and resources with a neighbor or family to participate in a garden. A lead farmer in Gwanda explained how she decided to have a garden at her home in addition to the crop she cultivates away from the house. With assistance from Amalima, when she was secure in her transition to an agro-dealer, she then had the resources to put in a garden at her home—onions, tomatoes, and chomolia (KII Agro-dealer, Gwanda). She describes a circular garden with a fruit tree in the middle—either mango, lemon or orange as examples, that is planted from seed and then vegetables are planted around and under the tree. This is one design from the project. The project’s efforts to provide matching grants to groups to aid in use of technologies that facilitate sorghum processing has gone mostly untested given low yields over the course of the project. One VS&L group in Tsholotsho district that had formed in 2014, however, transitioned to a thresher group in May 2018 to process sorghum, millet, and maize for local communities. They reported making good profits their first year and intend to buy a grinding mill if all goes well and they continue to earn profits. Positive results in increases in diet diversity14 were largely due to the project’s efforts to improve already-established nutrition gardens or to assist communities in establishing new gardens. The foods cultivated in each garden vary some across the districts; the crops most commonly reported by the garden FGDs and by Field Officers included sugar beans, tomatoes, carrots, onions, spinach, and chomolia, as well as some maize, sorghum and millet. These were produced for both subsistence and local (community and ward) markets. CONCLUSIONS Amalima should be applauded for initiating the project around encouraging communities to transition to drought-tolerant small-grain crops including sorghum, millet, and groundnuts, given the exceptionally low maize yields for over a decade. No project personnel could have known prior to project onset that over the course of the project there would be extensive drought periods four of the five years of the project. Moreover, the economic situation in Zimbabwe when the project was being designed (2011- 2012) was favorable and planners had no strong reason not to assume a healthy trajectory over the course of the project. The adjustments made in the planning reassessment during 2013 to place less emphasis on crop cultivation and more on livestock and irrigated gardens proved to be successful in 14 See Sec. 4.4 for specific discussion. IMPEL | Implementer-Led Evaluation and Learning 26 Evaluation Findings terms of supporting a secure food supply. The focus on addressing the largely unmet demand for water for livestock and crops resulted in an increased commitment to livestock herding, especially goats, and lowered risk perceptions associated with establishing new nutrition gardens, which ultimately meant improvements in diet diversity. Project participants perceived that the improved dip tank infrastructure and treatment procedures, reinforced by the paravet trainings and toolkits, helped improve health conditions, which they then relate, resulted in a decline in livestock death rates over the course of the project. The lower death rate overall does not exclude fluctuations that happened in certain periods, e.g., some livestock owners reported higher death rates in 2019 relative to 2018. While the adoption rates of CA techniques were low, the addition of the VACs should improve this situation, particularly if precipitation improves and farmers are motivated to focus on grains and are able to secure inputs at reasonable cost from local agro-dealers. The collective benefits of these activities are community-based and sustainability is projected to be secure for at least a few years. 15 Over the course of the project, these activities were conducted in more sustainable ways, by including better soil and water conservation works, and securing more participation and buy-in from communities ahead of the implementation of the activity. The knowledge and skill sets were provided to the local communities in ways such that their confidence levels were high in terms of their own knowledge base and being able to sustain project activities once Amalima exits. Given the trainings on maintenance of infrastructure and technologies (e.g., solar pumps for gardens, water tanks, solar panel units), in conjunction with the VS&Ls working to accrue savings for repairs and maintenance, the management committees do have in place procedures and means for addressing most maintenance issues and should be able to sustain efforts from a technical standpoint as well. IR 1.2: AGRICULTURAL MARKETING IMPROVED The BL/EL statistics show no significant change in the percent of farmers who practiced value chain activities promoted by the project in the past 12 months (Figure 9, Annex H). This result is largely corroborated through different VS&L and farmer FGDs and lead farmer KIIs, where participants were most concerned about producing enough crops for subsistence and having livestock to sell, both for income and as a coping mechanism in their response to the recurring drought conditions. There could be a case for stating that it is remarkable that agricultural production has not dropped further, considering the exacerbating natural and manmade conditions. A number of factors—the persistent droughts, the high cost of transport, the lack of ease of transaction (due to currency, price volatility, and inflation issues)—all compounded to create a non-market-friendly environment. These conditions resulted in mixed project effectiveness. Due to its natural dryness, the region’s primary agricultural marketable products were livestock and drought-resistant small grains. As the region is sparsely populated with dispersed communities, inferior transport infrastructure, and weak local and regional marketing networks, farmers are generally oriented toward local markets. The project supported output agricultural marketing well through Farming as a Business (FaaB) trainings, VS&Ls, Community-based Facilitators (CBFs), matching grant recipients, and agro-dealers, but at a cost of high project dependency with few opportunities for this structure to continue after the project, outside of local informal markets. The lack of opportunities outside local informal markets is due largely to the national economic/monetary situation, incurring high risk, high transaction costs, and weak marketing networks. One major and lasting positive impact of the project has been to establish a widespread and largely self￾sustaining input supply chain for most households through the strong working relationships of lead 15 See Sec. 4.12 for in-depth discussion. Final Performance Evaluation of the Amalima Development Food Assistance Project in Zimbabwe Evaluation Findings 27 farmers, VACs and agro-dealers. There is a narrow and sustainable positive impact for output marketing for the several hundred grant and horticulture participants, but this has not extended beyond the range of the matching grant recipients nor the project (Matching Grant, Horticulture FGDs; HQ staff KII). Smartphone technology (e.g., WhatsApp groups) has been useful for participants and staff; further uses may be possible when more locals have smartphones and there is consistent network coverage. IR 1.2.1: Business skills for men and women improved While many SO1 participants received some form of business training (FaaB, AMCs, matching grant and horticulture participants, VS&Ls, CBFs), the challenges of the frequent droughts and economic conditions were a significant deterrent to putting that knowledge into practice, especially given the lack of agricultural products surplus and limited marketing options. The main beneficiaries have been the horticulture and matching grant groups and the agro-dealers, thanks to consistent and dedicated project support. It was commonly found that it is difficult for CBFs to earn even a modicum of income from providing products and services, as it is sociologically a challenge for local residents to charge fairly for one’s products or service in one’s home community. One CBF explained that this is because one’s acquaintances are also poor and it is difficult to overcome traditional customs of group solidarity. The most obvious examples were paravets and VS&L cluster facilitators; the project could have made an effort from the beginning to change this. It was found that project participants had a good understanding and practice of recordkeeping, but lacked a deeper understanding of markets and marketing (VS&L and agro-dealer FGDs). For example, very few of the matching grant groups knew what a business plan was. The four FaaB training modules (Amalima 2017b) provide thorough coverage of key basic business and marketing principles, recordkeeping, and financial management, but they do not cover business planning, or how markets and value chains work. For participants to attain a solid understanding of the concepts and practices involved in markets and marketing, further training and coaching is required. However, regardless of knowledge, it is difficult for participants to engage in markets given the precariousness of the Zimbabwe economy and challenges related to communication (required for access to market information) and transportation—though the latter might be easier in or near urban/trading centers. IR 1.2.2: Access to business capital for men and women improved Access to a small amount of business capital has improved for many through the VS&L groups16 and after the first few years, some VS&L groups reported they were able to move into larger value-chain activities and IGAs. Only a few cases of linkages to financial institutions for larger loans were reported, such as the seven chicken groups in Gwanda district: the 51 farmers in these groups each took a US$395 loan from Metbank. This agreement was retained with a high dependency on the project, as the national economic situation was unfavorable for a more independent relationship. For both women and men, the percentage who achieved adequacy in ownership of assets and of decisions on credit decreased at endline (see Figure 10 and Figure 11, Annex H). Factors contributing to this may be that the men who were living in Zimbabwe were those who had not immigrated to neighboring countries for work. They were younger and older males, i.e., not of working age, with little likelihood of owning assets or being involved in household economic decisions. Other challenges, such as chronic drought conditions, had lowered 16 See Sec. IR 2.3.1. IMPEL | Implementer-Led Evaluation and Learning 28 Evaluation Findings people's feelings of asset adequacy and reduced the options for credit decisions. The biggest change is the decrease in percentage of men and women who achieved adequacy in decisions on credit. Compared to men, a higher percentage of women achieved adequacy in credit and in ownership of assets at endline. See Annex I for a detailed discussion and further analysis of the “adequacy” indicators.17 IR 1.3 POST-HARVEST LOSSES REDUCED IR 1.3.1: Post-harvest handling of agricultural produce improved The percentage of farmers (15.8) who used improved storage practices in the past 12 months did not change from baseline to endline (see Annex F); there were no gender differences. A factor affecting the lack of change in storage practices is that there were good harvests at baseline and poor ones at endline. From the DRR FGDs across all districts and KIIs with lead farmers, it is clear that while there is widespread knowledge about improved post-harvest handling (PHH), including the use of Integrated Pest Management (IPM), participants perceive a low to mixed impact depending on the specific crop and the appropriate or necessary practice required. The strategies most used are those with low labor requirements and minimum resource content, or high and predictable benefit/cost such as not drying meat in the sun, using chemical treatment for seed, using plastic bag packaging for storage, and selling and boiling milk (agriculture FGD, field staff KII). The training materials for PHH are accurate and reinforce IPM strategies, but some of the IPM strategies are not always effective; their usefulness may be crop- and pest-dependent as well as influenced by a range of abiotic factors with seasonal variance complicating appropriate formulas. Participants were previously aware of many of the PHH practices; the project reinforced existing knowledge and provided additional skills training. CONCLUSIONS Agricultural marketing has improved generally, a little for most households at the community level (goats, farming inputs) and significantly for a few hundred at a larger level (horticulture and matching grant groups). VS&L groups, agro-dealers, and towards the latter half of the project VACs, have also been the key to the project’s agricultural marketing success. Value chain and marketing outcomes are strongly influenced by external factors and while the project addressed them well with dedicated knowledge, coaching, and financial support, it was at the cost of an ultimately high project dependency. It was very relevant for the project to promote better PHH, both for home use and for marketing surplus, but the cyclical drought conditions were the main impediment to better adoption rates. A Knowledge, Attitude and Practice (KAP)/barrier study could have assisted both in the promotion effort and with increasing adoption rates; such studies were done by the Learning Unit with other project initiatives, such as for increasing youth participation (Amalima 2018b). 4.3 SO2: Community resilience to shocks improved This section discusses specific project initiatives designed by Amalima to improve the capacity of households and communities to providing safety nets to effectively respond to shocks or climate-related events. Activities focus on infrastructure improvements, strengthening social capital/networks, and improving financial security, particularly for women. 17 Adequacy of asset ownership, adequacy of decision-making about asset disposition, and adequacy of decision-making about use of credit Final Performance Evaluation of the Amalima Development Food Assistance Project in Zimbabwe Evaluation Findings 29 An important indicator relevant to resilience to shocks is the Coping Strategies Index (CSI). The decrease in household hunger (shown in Figure 14, Annex H and discussed elsewhere in this report) is reflected in the reduction in the average CSI from 33.8 at baseline to 25.0 at endline (p<0.001) (see Annex F). This decrease is encouraging but surprising, given worsening FCS (discussed earlier under SO1, in Section 4.2). However, the decrease in CSI over the project period was confirmed by the PLW FGDs, which showed a reduction in coping behaviors that were very common before the Amalima project such as going without meals, limiting portion sizes, and harvesting immature food crops. It was also supported by comments from interviewed government stakeholders concerning people’s current quality of life relative to before the project. IR 2.1: BASIC AGRICULTURAL INFRASTRUCTURE AND OTHER PRODUCTION ASSETS DEVELOPED/REHABILITATED The main infrastructure initiatives have been supported through the use of Cash for Assets (CFA), specifically for the construction or rehabilitation of dams, primarily for livestock watering; the construction or rehabilitation of dip tanks; irrigated gardens; and the rehabilitation of severely eroded zones and damaged grazing lands. The benefits of these assets extend beyond the project participants and are also appreciated by AGRITEX with their recent labor challenges (e.g., Bulilima District currently has 27 agriculture extension field staff for 22 wards but they had 67 field officers before the recent government freeze) (KII AGRITEX, Bulilima). In a few cases, dam FGDs (Mangwe and Gwanda districts) indicated frustration around the timeline to secure approval for new dams or rehabilitation work for existing but nonfunctional dams. All the infrastructure assets visited were of good quality and the participants appreciated the CFA associated with their construction, but sustainability is mixed,18 partly because the project implemented the activity before the AMC had the management capacity to maintain the asset. A few common themes emerged from the information gained in the lead farmer FGDs and field staff KIIs. The first concerned the project approach whereby many assets had informal community management, especially in the first three years, when it was more important to have the asset built than to consider its sustainability or management capacity. Second, each asset has its own characteristics that needed special treatment, not always taken into proper consideration from the beginning. For example, dams in sandy soils need regular “scooping” as sediment flows in and fills it up, and it is either labor-intensive to remove the sediment or expensive if a digger is used. The better the soil and water management upstream of the dam, the less need for scooping. A number of dam AMCs reported issues around being able to meet labor demands for scooping activities; their frustration centered on their perceived lack of planning on the part of the project for the need for scooping. Dams with water troughs also need a strong fence to prevent animals from eroding the banks and falling in, but this component is both expensive and needs 18 See Sec. 4.12 for in-depth discussion. In Mangwe, one dam project began in 2013 when there was such a severe drought, they had to move the livestock significantly further away. The village held a water assembly where it was decided there were such serious problems with water, they needed to construct a dam. They submitted a plan to the Ward Council and the development leader who recorded the plans. That office decided which problems are most important—then the development office selected projects. We were lucky that our dam project was approved in that process at that time. Our initial plan was for water for livestock. But we also wanted to have an agricultural project and use water from the dam. That project hasn’t yet been approved by the council because they need to know the water capacity of the dam—a study must be done. This has now taken a long time and we are having to wait for the news. – Leader, dam AMC IMPEL | Implementer-Led Evaluation and Learning 30 Evaluation Findings maintenance. Observations of the constructed natural fences indicated they were not animal proof (AMC FGD, Mangwe District). Livestock death rates have declined due to the protection afforded by paddocking and other animal shelters, though one DRR FGD in Mangwe discussed the challenges of securing labor and materials for construction of quality paddocks, neither of which is being subsidized by the project. Irrigated gardens with a wire fence, a large branding billboard, high project visibility, and solar pumping are prone to the theft of the solar panels, regular and sometimes high maintenance of the irrigation system (especially if drip), a level of technical skill requiring occasional costly inputs (pump, electricity), and thus are more challenging to manage. IP staff reported adapting the initial solar panel system roughly halfway through the project to eliminate theft. Any asset requiring labor-intensive measures in the public space (eroded zones, grazing lands) will face ownership and maintenance issues. These challenges suggest a clear-cut need, not only for competent AMCs, but also for proactive leadership, which is not consistent across all districts. IPs report that training modules for management of infrastructure projects include budgetary oversight and basic maintenance but do not include leadership and consensus-building skills. Dip tanks are perceived as a government initiative, since they are used primarily by the Department of Veterinary Services when they have mass dipping, vaccination, or dosing campaigns. When they collect fees they are required to provide roughly five percent to the AMC for maintenance. Some DRR committees have also conducted some work (mainly dams) with no material support from Amalima, such as in Bulilima Wards 5 and 11 (DRR FGDs). Local perceptions indicate villages prefer to do construction or rehabilitation with project support, but it should be noted that successful dam weir projects have been completed with no project support. IR 2.2: COMMUNITY-MANAGED DISASTER RISK REDUCTION SYSTEMS STRENGTHENED All of the Community-Managed Disaster Risk Reduction (CMDRR) committees have completed the appropriate trainings, which address nutrition, behavior change, and infrastructure stability. Amalima specifically trained on a range of topics addressing the hazards of droughts and where to get assistance when there is an extended drought. FGDs with CMDRRs indicate that for roughly half, their main activity is to hold monthly or quarterly meetings. The other half are more proactive; they organize community information dissemination or maintenance activities that may include stabilizing or securing roofs for personal or ward structures. Only two were found to have a stable financial system or long-term plans. Some DRR committees are not very active, meeting less than once/month; their work is therefore challenging for the project to monitor as they do not necessarily identify themselves as a group and do not report progress on a regular basis. In many cases, there are many similarities and a strong linkage between the project-based assets and the CMDRR and AMC committees, where many people are members of both. The main issues found were a lack of ownership and sustainability due to a less participatory project approach, labor and/or resource intensiveness of the DRR activities, the issue of how to manage public spaces (grazing land, eroded spaces), and the national system of DRR management. Exacerbating the issues around managing public spaces is the absence of the majority of males of working age from this population (as they have emigrated to South Africa or Botswana for work) and strong cultural barriers that do not allow women to make these types of decisions (KIIs, district offices, Department of Social Welfare, Ministry of Women’s Affairs). The local DRR system was found to be top-down, reactive, and weakly linked between village, ward, and district levels. The District Administrator is the head of the Civil Protection Unit (KII field staff) with Final Performance Evaluation of the Amalima Development Food Assistance Project in Zimbabwe Evaluation Findings 31 support from the councilors and various technical government agencies (AGRITEX, Veterinary Services, and Ministry of Health). While the training material and observed implementation were thorough and largely effective, it was repeatedly noted that there was too little done to incorporate fodder production along with soil and water conservation measures. The DRR FGD in Mangwe expressed appreciation for the trainings that addressed techniques designed to guarantee roofs are secure in strong storms and high winds. IR 2.3: COMMUNITY SOCIAL CAPITAL LEVERAGED Community social capital has been effectively leveraged over the course of the project. In every type of FGD, many participants relay their involvement in a number of project activities. The cross-integration is most evident within activities of one of the SOs, e.g., lead farmers are also on AMCs for dams or CMDRRs, and more recently the VACs. Lead mothers are involved in Community Health Clubs (CHCs). It is critical to note that what explicitly strengthened social capital across the board is the universal link through the VS&Ls: this connection provided a model of accountable participative responsibility reinforced by positive socio-economic gain. Another factor is that the CBFs function across a number of project initiatives, such as the lead farmers and lead mothers, or the CMDRRs and AMCs. Across all of the participant and stakeholder FGDs and village-level KIIs, it is clear there has been growing interest in the project since its onset, and in the end, immense satisfaction with the project and how it has been implemented. People highly value how the trainings brought them together in a new way, with widespread and good inter- and intra- group cohesiveness. They express high levels of confidence they can sustain the improved practices and apply the new knowledge and skills once the project is finished. Intentional efforts to engage the local leadership in the project activities, particularly through the mechanisms for community dialogue (e.g., discussion guides for livestock management), were effective in basic ways. Fewer gains in efforts to improve leadership skills and gaps were noted, particularly when leaders or managers could not address conflicts that arose within their group effectively and did not understand the broader impacts the project could have outside their districts. It was also widely reported within SO3 activities that gender relations improved, with a generally higher respect for women shown by men, acknowledged by females. 19 IR 2.3.1 Access to savings improved, particularly for women The BL/EL statistics reveal a significant and marked increase in the use of financial services in the past 12 months, from a general 5.4 percent to 24.5 percent (Figure 12, Annex H), which can largely be attributed to the use of VS&Ls. The use of financial services increased from 5.3 percent of male farmers at baseline to 19.4 percent at endline. The increase is even greater for women: 5.5 percent to 28.1 percent (see Annex F). The VS&L model used in Zimbabwe is a simple and robust one: VS&Ls typically meet monthly, keep no money in a box, and have a high success rate, with an average group size of 10 members and a monthly interest rate of 10 percent. In Year Three, as explained by the VS&L facilitator, the project created clusters of VS&Ls to mobilize more funds to enable larger IGAs. Each month, one group is allowed to use the pooled fund for profitable bulk buying of merchandise for resale or for addressing a need associated with their cattle. Women form the large majority of VS&L membership and their lives have been transformed by the VS&Ls on several levels, as related in VS&L FGDs across all districts. First, members experience new economic 19 See Sec. 4.4 for more in-depth discussion. IMPEL | Implementer-Led Evaluation and Learning 32 Evaluation Findings freedom with the ability to invest in productive activities, but also to allow financial resilience. Second, members have increased in-family and in-community respect due to their improved economic and management capacity. Third, members benefit from the group dynamics and the internal support they provide each other. Most women in VS&L FGDs mentioned being able to purchase more livestock, especially goats; being able to send their children to school for more years; being able to pay for medicines when needed; and having money for home improvements. With increasing regularity over the course of the project, men now either form their own groups or are willing to join a group with women. VS&Ls have been the key component of project activity integration, partly because many were formed from the Community Health Clubs, a consistently strong element of the project, but also because they allowed a short-term flexible funding source for nutrition, health, and farming activities, and a once-a￾year larger investment in cattle or latrines as the most common practice. Though not tracked by the standard MIS (management information system), it is estimated that 5-15 percent of members belong to more than one group and at least 10 percent of households have two members in VS&L groups (usually the same one), so that the number of households participating in VS&Ls is perhaps 80 percent of the number of accounts tracked (HQ staff KII). Due to currency and inflation issues, many groups in the project districts use the South African Rand, and at times, groups resorted to using chickens as currency. Globally, the groups have been very creative in overcoming hardships and using their savings for productive or health-protective activities, such as financing latrine construction. IR 2.3.2 Local social support mechanisms functional From a majority of lead farmer, livestock, and more recently formed VAC FGDs and KIIs, it is clear that the project has worked diligently to integrate government, private sector services, and traditional leadership into project activities and has created or reinforced CBFs and their roles in their communities. The exception to this achievement was the sometimes contentious relationship Amalima experienced with the Ministry of Agriculture, and in turn AGRITEX district staff due to issues outside of the project’s control. Amalima was conscientious throughout the project to extend support and trainings to the agriculture extensionists working for AGRITEX, with mixed results. Lead farmers and VACs expressed willingness to work with the agriculture extension workers, but tensions persisted in some wards more than others, due to the history of politics with the government not consistently supporting AGRITEX extension agents to participate in Amalima trainings (AGRITEX KII). This integrated leadership structure is reasonably functional while the project is operating, but there are reservations as to the ongoing functionality once the project is complete, as government services are short of resources and have their own goals and objectives (most FGDs, stakeholder KIIs). Although CBFs have been motivated with project support, very few of them are compensated for the services they provide, and thus over time are less likely to continue to offer the same level of support. It was clear that across all components, the CBFs were appreciated and were energized by the project (due to communications, planning, data collection, local focal points, training, local recognition, etc.) but that when the project ended, they planned to perform the same functions but in a more limited manner, also because they need to feed their families. This is especially the case for the paravets, who must replenish their toolkit supplies and are not comfortable charging fees locally for their services (paravet FGDs, Mangwe and Gwanda districts). The strength of local social support is expressed by participants in their descriptions of how relationships and interactions with fellow community and ward members have strengthened over time, the weaving of all of the project activities under the guidance of the CBFs, and the better inter-gender relations, with the VS&Ls as the glue that holds it together. Almost everyone who is a member of an AMC is also a member of at least one VS&L, if not two or three. Final Performance Evaluation of the Amalima Development Food Assistance Project in Zimbabwe Evaluation Findings 33 In the project extension, there has been a specific emphasis on the functionality of local social support and sustainability, with a focus on SBCC (Social and Behavior Change Communication), but each component does not have a TOC (Theory of Change) so as to clearly state the model and test the assumptions. The Learning Unit has provided specific assistance with the few studies it conducted (youth, gender, asset ownership and decision making, input supply, VS&L) (Amalima 2016d-f), though not all were completed at the time of the evaluation. The non-use of TOC is reflected by the project’s late inclusion of the Male Champion initiative, youth participation, some of the young mothers’ activities, and sustainability initiatives around their participation. IR 2.3.3 Local group leadership structures in place and effective From the majority of FGDs across all strategic objectives and KIIs with field staff and IPs, it is clear that all of the AMCs and group management structures have been well trained and organized for the management of their activity, but that their effectiveness depends on the capacity of the committee’s leadership. Amalima has weakly addressed this component by not offering specific trainings on leadership (compared to management), consensus building, and conflict resolution. While elderly males are recognized as traditional leaders, they serve this capacity at the local village and ward levels (without specific tasks relative to asset management) and it is a role they have grown into without any training. The traditional leaders interviewed expressed the desire for such trainings. While the project has greatly improved the status of women into small and medium leadership roles (within groups or as CBFs), there was little evidence of women becoming community leaders. CONCLUSIONS The resources committed to SO2 initiatives have been extremely effective, as there is a large unmet demand for water for crops and livestock; the benefits are community-based and sustainability is projected to be substantial and secure for at least a few years. 20 In the latter years of the project, these activities were conducted in a more sustainable manner, by including better soil management around water conservation works, and securing more participation and buy-in from communities ahead of activity implementation. The DRR activities were very relevant as the area is prone to both slow- and rapid-onset disasters, but their effectiveness is mixed. The CMDRRs are very dependent on the strength of the committees and community leadership. Generally, the impact and sustainability were good around project assets, but there was wide geographical variation and a beginning of more proactive approaches with preventive activities since the extension, such as gully prevention and other soil and water conservation measures. Leveraging social capital for systemic change is very relevant as resilience in more remote, poor areas with severe environmental stresses is less dependent on technical solutions. This improved social capital is sustainable and there are many cases of benefits spreading beyond the direct current project participants (e.g., kitchens, gardens, VSLAs,). Field Officers need to be more engaged to help “light initial fires” but challenges around workload and skills in this area may prevent solid advances here. The VS&Ls are a strong cross-sector link and cover the majority of areas/communities. New groups are forming, they are very sustainable, and they have been spreading throughout the communities over the five years. Unlike other VS&L models, this project did not create the position of cluster facilitator at the beginning of the project or the practice of their receiving compensation from the assisted groups to ensure continuous improvements and ongoing self-dissemination. 20 See Sec. 4.12 for in-depth discussion. IMPEL | Implementer-Led Evaluation and Learning 34 Evaluation Findings 4.4 SO3: Nutrition and Health among Pregnant and Lactating Women and Boys and Girls under 2 Improved The Amalima project focused on reducing malnutrition through increased dietary diversity and quality, improved care practices for women and children, increased access to water and sanitation, improved hygiene behaviors, and integration of nutrition into health care services (Amalima/USAID 2015b). The project has made great improvements in maternal and child health and nutrition as a result of interconnected health, nutrition and agricultural interventions that were highly relevant and responsive to identified problems and needs of the targeted beneficiaries, which included those CU2 with documented high levels of stunting and PLW with little knowledge of recommended maternal and child health practices (USAID 2015). IR 3.1 CONSUMPTION OF DIVERSE AND SUFFICIENT FOODS FOR PREGNANT AND LACTATING WOMEN AND BOYS AND GIRLS UNDER 2 IMPROVED Interviews with almost all lead mothers and caregivers showed an increase in the availability and consumption of different nutritious foods at the household level over the project period. This helps explain the improvements in child malnutrition indicators, which show a reduction in stunting in CU5 from 31.7 percent at baseline to 24.5 percent at endline (p <0.05) and a reduction in underweight from 14.6 percent to 6.5 percent (p<0.001) (see Annex F and Figure 3). Many clinic health workers reported that cases of stunting and underweight CU2 were rarely seen or not seen at health facilities beginning in 2017 extending to the present. They also reported optimal growth for almost all of the children whose growth was being assessed during the food distribution sessions. At the same time, there was no significant change in the prevalence of children 6-23 months who received a minimum acceptable diet21 or in the Women’s Dietary Diversity Score (WDDS): 22 women 15- 49 years of age consumed an average of 2.8 of the nine basic food groups at both baseline and endline. However, many caregivers and lead mothers reported they now consume a variety of foods including animal-source foods, especially during harvest periods. Many community members reported consuming a wider variety of foods including maize, rice, millet, fresh and dried vegetables, meat and poultry, small dried fish, caterpillars, beans, peas, groundnuts—all of which was not the case before the project. They have also increased consumption of fruits like melons, pumpkins, tomatoes, and indigenous nuts. In interpreting these seemingly mixed findings, it is important to keep several things in mind regarding measures like the various dietary diversity scores (HDDS, MAD, MDD-W, WDDS) and measures of stress and coping (HHS, CSI). First of all, they are very responsive indicators and therefore impacted by changes in the current context. Because they are based on current food consumption and other behaviors during the preceding day or month (depending on the indicator), they are best interpreted as a group rather than individually (Maxwell et al., 2013). This is in contrast to the anthropometric indicators, which are not so responsive to immediate circumstances and so give a better idea of the general trend over time. All project staff attributed the increased consumption of nutritious foods partly to the monthly supplemental feeding rations distributed by Amalima to PLW and CU2. These consisted of 5.5 kg of Corn Soya Blend (CSB+) together with 1.38 kg of vegetable oil per month for PLW, and 3 kg of CSB+ together with 0.92 kg of fortified vegetable oil for children 6-23 months. The protective ration consisted of 10kg of 21 The MAD indicator measures the percentage of children 6-23 months of age who receive a minimum acceptable diet, apart from breast milk. It measures both the minimum feeding frequency and minimum dietary diversity. 22 The WDDS is computed based on nine critical food groups. This indicator measures the micronutrient adequacy of the diet and reports the mean number of food groups consumed in the previous day by women of reproductive age (15-49 years of age). The indicator is tabulated by averaging the number of food groups consumed out of the nine food groups for all women. Final Performance Evaluation of the Amalima Development Food Assistance Project in Zimbabwe Evaluation Findings 35 sorghum, 3.3kg of lentils and 0.92kg of fortified vegetable oil per household per month. The evaluation findings suggest that the rations contributed to improved dietary diversity and also improved complementary feeding for CU2. The protective ration helped to reduce intra-household sharing of the supplemental feeding ration during the lean season, which was exacerbated by drought. However, a few caregivers reported that food rations were insufficient, usually lasting for 2-3 weeks in spite of the additional protection family ration introduced in FY 2016 to address increased food needs during the drought-induced lean seasons. A few young mothers reported not participating in the supplementary feeding program because they did not understand the preventive aims of the supplementary rations— thinking they were food for severely malnourished children. Lead mothers report that the messaging around child nutrition and childcare provided to assembled mothers prior to each food distribution was beneficial to the targeted population. Members of some caregiver FGDs explained they learned about other project activities at the sessions prior to the ration distributions, including the VS&L groups and dams being constructed for water provisioning for livestock, and became aware of the diversity of project activities spread throughout the districts over the beginning years of the project. The interview findings are consistent with the reduction in severe coping behaviors (see CSI discussion under Sec. 4.3). Most caregivers and lead mothers reported they no longer limit portion sizes, reduce number of meals, go without food for entire days, or harvest immature crops, which were all common practices during times of food scarcity prior to the Amalima project. For women in the Amalima activity area who gave birth in the past two years, the average number of ANC visits was 4.9 at endline, not a statistically significant difference from baseline. This result aligns with the WHO guideline that pregnant women have four or more ANC visits. IR 3.1.1. Knowledge and skills on diverse crops by PLW and caregivers improved The majority of care group members strongly appreciated the knowledge acquired through different health harvest trainings that were conducted by the project field officers and other volunteers (lead mothers and caregivers) on the methods used to maintain nutrition gardens, and how to prepare and consume nutritious rations using locally available indigenous foods. Project staff reported that the Healthy Harvest trainings were integrated with trainings on conservation agriculture, irrigated horticulture, and care groups. Participants reported they not only consider cattle, poultry, and goats as assets now, but they also regard them as key sources of nutritious food for both children and adults. Many caregivers reported that children’s consumption of milk from livestock and eggs from poultry increased since the project began. Many caregivers credited the different trainings and knowledge acquired during the care group sessions as contributing to a number of important changes in the community. Community members no longer hold beliefs or taboos that prohibit consumption of certain foods, including eggs and meat, by pregnant mothers and CU2, which was the case previous to the project. Many caregivers reported they no longer feed CU2 the dishes the rest of the family eats. They have learned to prepare porridge enhanced with protein-rich foods like fish powder23 and peanut butter. Some beneficiaries are now able to use the savings from VS&Ls to rear animals and poultry and also purchase additional foods they did not usually have in their homes prior to the knowledge they gained in the nutrition trainings. This complements comments from livestock and paravet FGDs, all of which acknowledged the project’s intent to ultimately improve the amount of protein available to households. 23 Fish powder is very rich in proteins and was used to fortify/enhance locally-prepared porridge. IMPEL | Implementer-Led Evaluation and Learning 36 Evaluation Findings IR 3.2 HEALTH AND HYGIENE AND CARING PRACTICES OF PREGNANT AND LACTATING WOMEN, CAREGIVERS, BOYS AND GIRLS UNDER 2 IMPROVED PLW FGDs across the four districts indicated a number of positive changes in health and hygienic practices. ANC attendance increased greatly and almost none of the caregivers were seeking care from traditional birth attendants. Most mothers practice exclusive breastfeeding and provide highly nutritious foods to their babies. Most caregivers reported accurate knowledge of the most acceptable hygiene and sanitation practices. Over the last two years of the project, with the Male Champion initiative, caregivers reported increased male involvement in childcare and maternal support at home. Lead mother FGDs across the project districts report increased community support and care for malnourished children and reduction in diarrheal cases among children in the community. Very few respondents did not practice the recommended practices. These participants exhibited limited knowledge of the promoted behaviors and low exposure to the care group trainings and home visits by lead mothers. The project was well designed to reach all targeted beneficiairies, however, these exceptions may have resulted because many communities are sparsely populated and some households are located very far and isolated from others. This made it difficult for lead mothers to visit all of them and deliver key messages. The care group model was used to create effective peer networks of caregiver volunteers, lead mothers, and caregivers. These networks effectively relayed BCC messages on maternal health, healthy harvests, exclusive breastfeeding, and complementary feeding. Formative research (Amalima 2015b) identified barriers for behavior change and addressed the key drivers of stunting in the region. This research identified some critical barriers that informed the BCC messaging: i) caregivers had heavy workloads which made it difficult to breastfeed for an appropriate length of time or to prepare individual meals for infants and young children; ii) caregivers were not feeding children animal source foods and were unaware of locally available nutritious foods; and iii) caregivers had no knowledge of feeding children differently when they were ill. The cooking classes conducted during care group sessions provided caregivers firsthand knowledge on how to prepare nutritious meals using locally available ingredients. The recipe book helped to reinforce the training received in the cooking classes. Lead mothers reported that use of interactive flip charts and the recipe books greatly facilitated the trainings. The Healthy Harvest trainings helped to strengthen nutrition/agriculture linkages. These training modules were well integrated into horticultural practices, and care group trainings helped to teach lead mothers and lead farmers about the consumption of a diverse, nutritious, locally available diet. Many caregivers reported to have learned how to grow successful small-scale home nutrition gardens from the lead mothers. Some caregivers reported to have attended those trainings conducted close to their homes, provided by lead farmers. IR 3.2.1 Knowledge and skills of child health and maternal nutrition by caregivers improved The majority of caregiver FGDs and all lead mother FGDs reported they gained knowledge about the recommended child health and nutrition and hygienic practices, specifically i) the prevalence and duration of exclusive breastfeeding for infants under the age of six months; ii) the duration of Most caregivers reported that interpersonal communication was very effective in learning the messages delivered during the home group counselling sessions. This initiative was strengthened when lead mothers used the home visit sessions to tailor the BCC messages to the unique situations of each caregiver. Home visits provided a unique opportunity for reaching out to other key decision makers in the homes such as men, grandmothers, and mothers-in-law, with similar messages. Final Performance Evaluation of the Amalima Development Food Assistance Project in Zimbabwe Evaluation Findings 37 breastfeeding for children ages 6-23 months; iii) the diets of children ages 6-23 months in terms of quantity and quality of particular foods, including enriched porridge with nuts, legumes and vegetables; iv) improved feeding habits for infants and young children during illness as a means to sustain nutrition and restore health; and v) how to reduce diarrhea frequency through increased hand washing with soap or ash, and actions to keep babies from contacting feces, such as creating safe play areas. Exclusive breastfeeding of children under six months increased dramatically from under half at baseline to over three-quarters at endline (Annex F and Figure 16, Annex H). Feeding practices for children 6-23 months old remained low in spite of a sound understanding of child feeding practices by caregivers and a good project design. Amalima accomplished different activities aimed at improving infant and young child feeding practices via: (i) effective BCC messages, tailored to the information needs of beneficiaries, which helped to address the key barriers and motivators for improved child feeding practices; (ii) cooking classes conducted during care group sessions, which gave caregivers firsthand knowledge on how to prepare nutritious meals using locally available ingredients; (iii) home nutrition gardens, which helped to increase household access to green leafy vegetables; and (iv) increased male involvement in child health and nutrition activities at household level through the male champions. This combination of activities created an appreciation of a nutritious diet for the household and also emphasized the role of men in ensuring that their families have access to nutritious food. Across all districts, many caregivers reported having more time for child care and feeding because they were receiving support from their husbands in undertaking household chores and gardening. Some caregivers adopted use of the environmentally friendly and fuel-efficient eco-stoves, which cook faster and consume less firewood compared to the traditional open-fire method. This helped to reduce the time women spent on food preparation and collecting firewood. IR 3.2.2 Male involvement in child health and maternal nutrition improved All Amalima staff reported the Male Champion initiative, which began in FY 2016, increased male involvement in care group activities. This activity was triggered by formative research findings showing low male involvement in project activities. FGDs with most lead mothers and Male Champions indicated a general increase in male involvement in child health and nutrition activities at the household level. The evaluation showed that men who wanted to participate in the Male Champion groups were reached at various community meetings/functions (not necessarily linked to distributions or home visits). Soccer tournaments were also used for motivation and wider community engagement rather to identify/recruit new Male Champions, who had already been selected by that time. Amalima used a highly consultative and participatory process involving groups of men and women, which generated information used to develop a strategy for male involvement in IYCF, centered on Male Champions. FGDs with Male Champions in Gwanda and Tsholotsho districts showed that male participation was greatly facilitated by the use of village heads and village health workers to identify and mobilize men to serve as Male Champions. Potential participants were identified at soccer tournaments at ward and district levels. Men gave strong positive feedback about the drama sessions, which helped them share The majority of Male Champion participants valued the project-led trainings because they were highly interactive and encouraged their discussion of issues that affect them from a male perspective. The trained Male Champions used interpersonal communication effectively to support their wives in improving IYCF, nutrition and care. Many Male Champions reported they strongly acknowledge that it was important for their families to receive a nutritious diet and they now have a clear understanding of this need and support their improved role in ensuring that their families have access to nutritious food. IMPEL | Implementer-Led Evaluation and Learning 38 Evaluation Findings testimonials on their experiences and stimulated community discussions about male involvement in other project activities. IR 3.2.3 Time available for child caring by PLW and caregivers optimized Many caregiver FGDs across all districts reported having more time for childcare and feeding because they were receiving support from their husband in household chores and gardening. Some caregivers adopted use of the environmentally friendly and fuel-efficient eco-stoves, which cook faster and consume less fuel wood compared to the traditional open fire method. Some women reported that eco￾stoves are user-friendly and were able to carry them to their gardens, where they could use them while gardening. Women in one Irrigation Scheme FGD reported issues around the stoves being fragile and breaking in transport. Appropriate clay soils for constructing eco-stoves were found in only 22/63 targeted wards and therefore, the project was not able to scale up the eco-stoves to many other targeted locations (Amalima 2017a). IR 3.2.4 Community-based management of malnutrition among PLW and boys and girls under 2 improved The evaluation showed that Amalima contributed to improvements in the nutritional status of the targeted beneficiaries over the course of the project. This was attributed largely to the supplementary food rations targeted to PLW and CU2. This section provides details of the baseline – endline changes. There was great improvement in the use of recommended IYCF because of Amalima’s effective BCC messaging. The prevalence of exclusive breastfeeding for children under 6 months old significantly increased from 44.9 percent at baseline to 75.3 percent at endline (p<0.05) (Annex F and Figure 16, Annex H). While the prevalence of children 6-23 months receiving a minimum acceptable diet increased from 2.6 percent at baseline to 4.8 percent at endline, this was not a statistically significant result ( Figure 16, Annex H). The percentage of CU5 with diarrhea in the two weeks prior to the survey decreased from 15.8 percent at baseline to 10.4 percent at endline (p<0.1). The evaluation generally showed that Amalima contributed to the positive changes in the nutritional status of CU5, as shown in Figure 3 below. Figure 3: Prevalence of underweight, stunted, or wasted CU5 at baseline and endline The prevalence of underweight, stunted, or wasted CU5 declined from baseline (2014) to endline (2019) ns = not significant, + p<0.1, * p<0.05, ** p<0.01, *** p<0.001. Final Performance Evaluation of the Amalima Development Food Assistance Project in Zimbabwe Evaluation Findings 39 Conclusions Amalima increased the availability and consumption of different nutritious foods at the household level partly due to the supplemental feeding rations, which improved on the complementary feeding of CU2, and improved value chains promoted by the project. Engagement in VS&Ls also increased the ability of households to purchase nutritious foods for their families. The Healthy Harvest trainings promoted the importance of producing and consuming diverse and nutritious crops and vegetables. In addition, communities learned how to organize and maintain nutrition gardens and how to prepare nutritious foods using locally available ingredients. The increased availability and consumption of a diverse and nutritious diet led to reductions in the levels of stunting, wasting, and underweight in CU5. The evaluation showed improved health and hygienic practices among the targeted beneficiaries including ANC attendance, exclusive breastfeeding, quality and quantity of food for children 6-23 months, feeding habits during illness, handwashing practices, and other hygienic practices such as safe play areas for infants. These improvements were attributed to the effective BCC messaging facilitated by the care group approach. The integration of the Healthy Harvest trainings into the CA, horticulture, and care group trainings helped participants understand how the various initiatives of the project are integrated. The Healthy Harvest materials helped train lead mothers and lead farmers on a diverse, nutritious, and locally available diet. The increased male involvement in childcare and maternal support at home, and use of eco-stoves, provided caregivers and PLW more time for child caring. IR 3.3 ACCESSIBILITY TO AND EFFECTIVENESS OF COMMUNITY HEALTH AND HYGIENE SERVICES IMPROVED Across the four districts, WASH activities made a significant contribution to improvements in the accessibility and efficiency of health services during the project. These activities were guided by the national WASH development framework and participatory community needs identification with community leaders (UNICEF 2015). The project outputs indicate improved health and hygiene facilities throughout all four project districts, particularly in reference to the health centers. This success strengthened community satisfaction and prospects for sustainability. The evaluation team found that WASH activities were anchored in enhancing the knowledge and skills of service providers and improved facilities for childbearing mothers and participating communities (clinic FGDs, all districts). Health centers were specifically targeted by the project in the provision of improved drinking water sources; these efforts were not extended to communities. Consequently, the household-based quantitative survey did not indicate statistically significant improvement on this indicator (see Figure 4). However, other positive smaller WASH improvements were noted over the life of the project, largely as a result of the strong hygiene education and promotion components of the project. These efforts were further enhanced by the well-trained and active CBFs and CHCs. The endline survey assessed standard indicators for household WASH practices. From baseline to endline, the percentage of households using an improved source of drinking water did not change (see Figure 4). 24 The most common improved source of potable water is the tube well or borehole (used by around 60 percent of households at both baseline and endline), and the most common unimproved source is surface water (20-25 percent of households, also with no significant change). Of those 24 It bears noting that in the analysis of the survey data for this indicator, only households that meet both of the following two conditions are considered to have access to an improved water source: (1) has access to one of the water sources on the list (piped water into dwelling or yard, piped tap/standpipe, tube well or borehole, protected well, protected spring, and rainwater) AND (2) water was generally available without any interruptions of a day or more in the two weeks prior to the survey. The mere existence of the water source structure is not sufficient. IMPEL | Implementer-Led Evaluation and Learning 40 Evaluation Findings households using unimproved sources, almost 90 percent did not treat their water for drinking, with no baseline-endline change (see Table 11, Annex H). There was, however, improvement in the percentage of households using safe storage of drinking water, which rose from 49.7 percent at baseline to 98.4 percent at endline (Figure 4). Improvement is also seen in the use of improved sanitation facilities, driven by increased use of ventilated improved pit latrines, which went from 26.1 percent of households at baseline to 36.0 percent at endline (p<0.01) Table 11, Annex H).25 There was also some improvement in the percentage of households with soap and water at a handwashing station, though the endline percentage is still a small proportion of households (7.8 percent). Annex F shows additional WASH indicators. IR 3.3.1 Improved knowledge and skills of community health and hygiene to service providers KIIs conducted with project staff and district nutrition and nursing officers indicated that trainings were well organized and led to improved knowledge and skills of community health and hygiene. Public Health staff provided extensive training and orientation using well-developed training materials that had been tested over a number of years. The project then identified lead mothers, caregivers, Health Club Volunteers, Male Champions, and other related CBFs throughout the districts who would lead the training-of-trainer sessions related to SO3 activities. Capacity enhancement of health center staff, extension workers, and community volunteers was extended in an efficient manner as demonstrated by the noted improvement in health services provision and engendering positive behavior change within communities. While the Health Center Committees (HCC) offered much-needed community support to the clinics, the HCC’s linkages to government support agencies such as the Public Works Department need to be clarified.26 The project targeted initial trainings of health club volunteers and caregivers who were then able to form CHCs whose ripple effects improved hygiene practices in all four districts. Strengthening of 25 It bears noting that in the analysis of the survey data for this indicator, only households that meet both of the following two conditions are considered to have access to an improved water source: (1) has access to one of the water sources on the list (piped water into dwelling or yard, piped tap/standpipe, tube well or borehole, protected well, protected spring, and rainwater) AND (2) water was generally available without any interruptions of a day or more in the two weeks prior to the survey. The mere existence of the water source structure is not sufficient. 26 Discussions at Health Centers revealed that the Health Center staff reported infrastructure-related maintenance problems to both the HCC and Public Works Department, but repair work was a sole responsibility of the latter. Thus, reporting to HCC appeared to be for their information only and the HCC would not take remedial measures. Figure 4: WASH indicators at baseline and endline (Amalima) The largest improvement was in use of safe storage of drinking water. Final Performance Evaluation of the Amalima Development Food Assistance Project in Zimbabwe Evaluation Findings 41 activities such as CHC competitions reinforced hygiene practices and positive behavior change among participating villages. KIIs with clinic personnel and FGDs with community members at selected health centers throughout the districts showed that the project interventions provided improvements, as the knowledge and skills of community health and hygiene to service providers had been steadily declining over the preceding years. The capacity enhancement activities in health services provision significantly rejuvenated health delivery in participating communities. IR 3.3.2 Improved community health and hygiene services for pregnant and lactating women, boys and girls FGDs with clinic staff indicated that rehabilitation and construction of safe water infrastructure was successfully carried out in the 27 targeted clinics where solar pumping units were installed. This initiative also incorporated rehabilitation of clean and safe water conveyance systems, storage tanks, and reticulation to standpipes in most participating clinics. However, the evaluation team noted that the scope of the rehabilitation did not cover repairs to the commonly damaged wastewater conveyance systems, which in turn restricted the availability of running water inside the clinics, bathrooms, and flush toilets. There was also a need to assess the capacity of the solar units, which appeared to have excess capacity that could be used to supply basic electrical power to the clinic and staff houses. Because of the limited scope of WASH infrastructure, some desirable WASH benefits were not realized such as having running water inside the clinics, flush toilets, or access to bathing facilities such as bathtubs or showers. Apparently, these needs had not been foreseen in the design and budgeting of the project. Sustainability of WASH infrastructure was enhanced by the use of solar pumping units that required only minimal maintenance. In addition, at the end of the project the responsibility for maintaining infrastructure constructed at the government health centers will automatically fall to the Public Works Department. The HCCs will also assist in mobilizing resources for sustaining the facilities. One of the Amalima project’s unexpected (and positive) outcomes was helping to improve service delivery statistics at health facilities, which enabled the facilities to access more funds through results￾based financing mechanisms. Ultimately, this enabled many health facilities to improve the health facility infrastructure and quality of services offered. Clinic staff FGDs indicate the completion of rehabilitation/construction of waiting mothers’ shelters at a number of health centers, which improved the comfort of expectant mothers as they awaited child delivery. Construction of model VIP latrines at the same centers, as well as user-friendly latrines for pregnant mothers, disabled, and the elderly, provided an improved health environment at these clinics. The provision of chlorination pumps at rehabilitated water points provided safe water for clinic staff, patients, and neighboring villagers. Clinic staff acknowledged the positive results of the installation of hygiene promotion facilities like tippy taps at health centers that enhanced community uptake of positive hygiene practices and handwashing after using the toilet. According to clinic staff, chlorination of water at point source contributed to a significant reduction in diarrheal episodes in CU2 and beneficiary communities. Pregnant mothers waiting for child delivery had a reduced burden as improved access to essential facilities is assured at health centers. Retention of experienced health staff has been enhanced, resulting in improved health services, as the participating institutions now offer improved staff living conditions. Established WASH facilities at health centers served as important training and demonstration units and are being replicated at the household level, particularly through the CHCs. Excess drinking water at clinics was IMPEL | Implementer-Led Evaluation and Learning 42 Evaluation Findings utilized for nutrition gardening27 that ultimately led to improved nutrition for clinic staff, waiting mothers and patients. These gardens were also acting as demonstration plots for neighboring villages. Improved access to safe water and sanitation was achieved in communities through infrastructure rehabilitation and hygiene promotion. These included the construction or rehabilitation of sand abstraction water points for both horticultural and household use in communities; trainings for health center staff and community volunteers in purification of drinking water by boiling, chlorine-based chemicals, and slow sand filtration at household level; safe storage of drinking water and its proper use; the creation of model homes and kitchens, including the energy conserving eco-stove, and regular CHC competitions to encourage a high level of hygiene in communities. FGDs with the majority of district WASH staff showed that linkages between WASH and other project components were established. CHCs are now incorporating VS&L and IGAs to improve their resource base for advancing construction of hygiene-enabling facilities. The problem of open defecation in grazing areas is now being jointly resolved by CHCs and lead livestock farmers through a scaling up of VIP latrine construction.28 In all WASH FGDs, participants expressed willingness to continue with improved WASH provision since they had gained valuable knowledge regarding improvements to their health. All FGDs made a commitment to sustain the WASH activities through the formation of more CHCs; the construction of additional household sanitation facilities using VS&L; and income-generating projects so as to attain open-defecation-free (ODF) status in all participating villages. CONCLUSIONS The implementation of the WASH component of the Amalima project in the four districts of Matabeleland was welcomed by the participating communities with remarkable improvements in accessibility and efficiency of health services. The project activities were clearly relevant and effective in meeting community needs and expectations, and left the communities satisfied. These particular efforts resulted in positive behavior change where the outcomes were readily incorporated in their daily life. Prospects for the sustainability of improved household hygiene after the main phase of project implementation are quite high, since the communities have realized that these practices strongly contribute to improved primary health for their family members. There remains, however, a need to strengthen government support to communities to scale up this intervention to new districts. 4.5 Unintended Outcomes The project yielded a number of unintended positive outcomes: VS&Ls have been the mainstay of project integration, economic improvements, improved resilience, and women’s empowerment. It is not evident from KIIs with IPs that this result was intentional, at least not to the extent that it occurred. There was also good cross-fertilization of knowledge and practices through the CBFs, such as VACS promoting WASH and CHCs encouraging young mothers to participate in nutrition gardens. Even though the project’s activities largely involved women’s lives and seemed to make more work for them, they resoundingly stated that presently they were more productive, had more assistance from men and a lot of group support, were in a better economic situation, and had more flexibility in terms of 27 The Evaluation Team's understanding is that chlorination was done at the point of collection, i.e., at the drinking water tap, while gardening water was collected straight from the tank without chlorination. 28 Attainment of open-defecation-free status is a national policy and strategy for WASH that encourages all rural households to own and use a latrine the disposal of human excreta. This strategy been progressively implemented over the past six years as part of improving access to improved sanitation facilities the at household level. Final Performance Evaluation of the Amalima Development Food Assistance Project in Zimbabwe Evaluation Findings 43 time relative to daily tasks, without feeling an additional burden. All of the women reported experiences indicating fundamental shifts to improved gender equality in household labor and decision making and reduction in number of cases of gender-based violence. Some women reported they were happier, with an overall improvement in their sexual relationships. Most clinic nurses reported that the supplementary feeding rations attracted many people to the health facilities who previously shunned health services (KIIs clinic nurses). They were also provided additional health services including growth monitoring, vitamin A supplements, early ANC bookings, and immunizations, in addition to the food rations. This unplanned result boosted the service delivery statistics for clinics, which enabled health facilities to access more funds through results-based financing. Nurses reported these supplementary funds were used to improve the overall quality of services by purchasing essential drugs, MCH equipment, and setting up maternity waiting shelters. Excess water at clinics was utilized for nutrition gardening, leading to improved nutrition for clinic staff, waiting mothers, and patients. These gardens were also acting as demonstration plots for surrounding villages. Communities in the vicinity of the clinics benefitted from the nearby safe water at the clinic. A few unintended negative outcomes were associated with the project: The majority of FGDs for SO1 conveyed that the people who had least benefited from project activities were those who were economically poorest and those who lived furthest away from activity centers. The former group did not have sufficient labor, livestock, or the financial means (VS&Ls) to participate, and the latter had relatively higher transport costs in terms of time, energy, and money. During VS&L FGDs it was explained that, especially at the onset of the project, there was social stigma associated with participating in Amalima as the project appeared to largely be designed for women. Amalima was the first development project in this region that had a more participatory approach and did not have a routine of handouts. These concerns gradually dissipated as the initial non-participants acknowledged the longer-term benefits of the project. Until later in the project, the project lacked appeal for youth, as most activities involved having access to land and other productive resources and the sought-after benefits (livestock, cropping, VS&L) would only be realized over a longer period of time. For example, over time, some youth came to realize that pen fattening was the best option for their age group. The project also experienced challenges around securing the participation of young mothers outside of SO3 activities. Two areas where young mothers have begun to participate are in the nutrition gardens and the livestock groups oriented around goats. Goat herding is more attractive to younger females given lower labor requirements and also because traditionally, adult males are not the decision makers for goat herding as they are for cattle. 4.6 Factors Contributing to Outcomes This section describes factors the evaluation team assessed as contributing to project outcomes, based on the information and perspectives gathered from KIIs with project stakeholders and FGDs with project participants. PROJECT INTEGRATION The extent of integration of project activities created an enabling environment for behavior change, as reported by Amalima staff and project participants consistently in all four project districts. The interventions in agriculture, VS&L, and gender mainstreaming were designed as a comprehensive package dealing with both crop and livestock and aimed at boosting the knowledge and skills of both male and female farmers to intensify production and realize income through sales. They are an IMPEL | Implementer-Led Evaluation and Learning 44 Evaluation Findings integrated package that was intended to strengthen the linkages between VS&L and agriculture on the one hand, and VS&L and health and nutrition on the other. As reported by the vast majority of VS&L members who participated in FGDs across all four districts, they used the increased income they received from the VS&L activities to purchase inputs for better agricultural production, to provide healthier diets for themselves and members of their households, and to cover school fees and health care costs. CAPACITY BUILDING OF COMMUNITIES The project design was strong in that it sought to build capacities of communities – through well￾designed trainings oriented to improve knowledge and management skills relevant to all three SOs – to take charge of building their own resilience. Qualitative feedback from the majority of participants confirmed that they recognized the value and importance of the knowledge and skills they gained in the trainings. Trainings were on-point with community needs and provided accurate technical information and guidance in targeted behavior changes to support program initiatives. Communities learned to identify interventions to reduce disaster risk and have the knowledge and skills to implement them with support from government stakeholders at the district and ward levels. Participation of beneficiary communities in project implementation, monitoring and evaluation is a key element that supports sustainability of these initiatives as well as those developed in the future. GENDER EQUALITY AND EMPOWERMENT The integration of gender into program activities helped to break through social and cultural norms that affect consumption of nutritious foods, especially in reference to food taboos discouraging CU5 from eating meat and eggs. While VS&L and gender equality objectives were not integrated at the start of the project, over time the synthesis of a number of project components has resulted in VS&L and gender playing pivotal roles in broadening the impacts of the nutrition and agriculture interventions to include women‘s empowerment both outside and within the home, particularly in regards to the roles women have in maintaining and managing the nutrition gardens. Knowledge and skills gained in relevant trainings has led to increased capacity on the part of both males and females to ultimately improve food and nutrition security for their families. Females have been empowered through participation in nutrition gardens, livestock groups, and DRR committees, as well as health and cooking initiatives. VS&Ls were found to be especially popular among females; VS&L FGDs across all districts included many endorsements of the positive results achieved, as members were able to save sufficient funds or access loans to acquire productive assets. Related VS&L group activities created social capital for participants that opened more opportunities for them to meet their economic and social needs. The Male Champions meetings organized by Amalima increased male involvement in promoting and influencing key maternal and IYCF practices at the household level. Most men who were interviewed reported they no longer feel ashamed to do household work including caring for the child, cooking, and cleaning the backyard in the presence of other people. COMMUNITY COHESION Engagement and buy-in of participant communities toward the Amalima concept strengthened community cohesion. Support from the project in the provision of new knowledge and skills, tools, materials, and technical assistance gave participants conviction and confidence to address head-on the major challenges they face in food productivity and malnutrition. Committee structure reinforced the Amalima approach and groups worked together with assistance from project staff to develop activities that would effectively meet the critical needs of food security and also provide healthy and sufficient diets. Access to financial services, in particular savings, was a key step in getting their children through Final Performance Evaluation of the Amalima Development Food Assistance Project in Zimbabwe Evaluation Findings 45 to school graduation. The exercise of saving through groups helped not only to build social capital among group members, but to foster financial management skills and promote long-term asset accumulation. ENVIRONMENTAL SUSTAINABILITY Amalima’s greatest environmental challenge was to develop appropriate and effective measures for improving food and livestock production while combating the effects of chronic drought conditions on soil fertility, erosion, and water quality and supply. Amalima’s efforts to mitigate the effects of drought included water provisioning for livestock, CA practices that involved water harvesting and moisture conservation technologies, the cultivation of better-adapted small grain crops such as sorghum, pearl millet, and cowpeas, and the promotion of small-scale irrigation practices (Amalima 2014a). COLLABORATION WITH KEY STAKEHOLDERS KIIs with government stakeholders (MoHCC, Ministry of Gender) showed strong collaboration in the planning, implementation and monitoring of interventions. They were fully engaged in the formative research, gender analysis, and development and pre-testing of information, education and communication (IEC) materials, training, and monitoring of activities. MoH staff were engaged in pre￾testing and final adaptation of IEC materials and the integration of food distribution into child health activities, which provided an opportunity for child growth monitoring, identification, and enrolment of severely malnourished children into care and more effective BCC messaging to the caregivers. 4.7 Contribution of Activities to Mitigation, Adaptation to, and Recovery from Food Security Shocks and Stresses The Amalima project has contributed well to all three aspects of food security resilience both in direct (farming) and indirect (VSLA, networking) ways. VS&L funds provided safety nets, and the project’s Amalima approach—along with high levels of group participation—improved social connectedness. Considering the region’s agro-ecological potential and its recent climate challenges, mitigation of drought impacts were boosted through the emphasis on drought-tolerant small grains, enhanced PHH, improved livestock grazing management, and more effective VS&Ls, though there were some gaps in practice, particularly for PHH. Most SO1 and SO2 activities were aimed at improving resilience to food security shocks and stresses and improving nutrition through the garden harvests. The best results were demonstrated in the pervasive gains in technical knowledge and skills, widespread CBFs and VS&L support, effective targeting of technical groups and AMCs, networking linkages to government services, access to inputs through the agro-dealers, and better gender equity and women’s empowerment, all boosted by increased self￾confidence and better social relationships. This improved resilience is most marked at the household and group level and less at the village, ward, and district levels. Of the physical assets, dams, water harvesting works, and dip tanks contributed significantly but at small scale, only in their vicinity, and in the near to intermediate term. The DRR activities also made an impact, mostly at the planning level for mitigation, but in the few physical examples, recovery with dams and water harvesting worked. The project distributed monthly supplementary feeding rations comprised of fortified vegetable oil and corn soya blend (CSB+) to PLW and children ages 6 to 23 months. All project staff reported this was a timely response to the consecutive droughts which caused severe food shortages. The ration IMPEL | Implementer-Led Evaluation and Learning 46 Evaluation Findings distribution increased access to nutritious meals for PLW and supplementary foods for children ages 6 to 23 months. In addition to the primary distribution points which were set up within the Primary Health Centers, Amalima also set up secondary distribution points, which increased access by reducing the long travel distances that would have been necessary for some of the targeted beneficiaries. All project staff credited the Healthy Harvest trainings for building community capacity to grow, prepare and process healthy foods. The majority of community health FGDs reported improved knowledge of enriched porridge preparation and increased consumption of local nutritious foods. Most lead mothers and caregivers reported having more time to prepare meals for their children due to the environmentally friendly and fuel-efficient eco-stoves which they acquired with support from Amalima. These stoves entail shorter cooking times and reduced the time women spent collecting fuelwood. Subsequently, women had more time for childcare and completing other household chores. The project also effectively addressed the underlying beliefs and constraints to the adoption of the recommended IYCF practices. Through formative research (Amalima 2015b), the project identified the key barriers and facilitators to the recommended IYCF practices and used them to inform the different BCC materials. The engagement of community leaders and other key household decision makers (including men, grandmothers, and mothers-in law) helped to address the underlying barriers and constraints and facilitated adoption of the recommended IYCF practices. Poverty indicators capture a household’s ability to meet its basic survival needs such as food, clothing, and shelter. Households have two types of resources they can draw on to meet these needs: current income, and assets they can rely on to generate future income. This report uses measures of income poverty, which indicates whether a household currently has sufficient resources to meet basic needs. The poverty indicators used here include the percentage of households below the total per capita poverty datum line (TPCDL), and depth of poverty. For both measures, the poverty line below which a household is considered poor is the 2014 national poverty line of US$3.35 per person per day. Household income is measured using total per capita expenditures, including food and non-food items. Per capita daily expenditures increased from US$1.20 at baseline to US$1.30 at endline (USD 2014) (p<0.1) (Annex F). Using the 2014 national poverty line of US$3.35 per person per day, the percentage of households below the TPCPDL decreased from 97.1 percent at baseline to 89.9 percent at endline (p<0.001) (see Annex F). There was no statistically significant change in the mean depth of poverty, which measures how far households are below the poverty line. On average, households were at about 65 percent below the poverty line at both baseline and endline (Annex F).29 4.8 Beneficiary Satisfaction In KIIs with district-level ministry personnel, satisfaction was reported regarding processes used in the initial planning and decisions on ward targeting. In the vast majority of FGDs across all four districts, villagers expressed their gratitude to the Amalima project for affording them the opportunity to fully participate in initiatives designed to ultimately resolve their problems. They considered all project interventions as relevant, timely executed, and significantly addressing their needs. Household-based benefits such as improved food security and incomes and improved nutrition and hygiene were found to give high levels of satisfaction as testified in CHC FGDs whose members were also participating in additional project activities. CHC and VS&L activities were specifically mentioned in the 29 https://www.usaid.gov/sites/default/files/documents/1866/Zimbabwe%20Baseline%20Study%20Report%2C%20June%202015. pdf Final Performance Evaluation of the Amalima Development Food Assistance Project in Zimbabwe Evaluation Findings 47 majority of SO3-oriented FGDs as having direct benefits to households and communities in terms of improvements in nutrition and economic security. CHC members applauded the project for improved support from CBFs and regular CHC competitions that encouraged members to uphold household hygiene. Mothers of childbearing age expressed their sincere gratitude to the support from caregivers and Village Health Workers as they felt empowered to better care for their children and families. In most FGDs across all four districts, women claimed that the knowledge gained in preparing nutritious foods greatly assisted in fighting stunting among small children. In addition, women were grateful to the Amalima project for including traditional leaders as active participants since this approach resulted in the consideration of cultural gender norms, and spouses were consequently enjoying better interaction in and around their households. Participants in a Male Champions FGD acknowledged the encouragement and support for their efforts in promoting women’s empowerment in the households and community. Communal irrigated gardens were well received by community members; the garden produce contributed immensely to providing a more nutritious diet for households. Community members strongly believed that the nutritional uptake of the readily available fruits and vegetables significantly contributed to the decline of malnutrition and stunting among CU5. The construction of dams and livestock watering facilities were highly valued by participants as they provided sustainable and easily accessible water for both crop production and livestock. The rehabilitation of cattle dip tanks was welcomed by community members and agricultural extension staff given their importance as DRR measures. Many participants were impressed by transformations within their local communities as dams, gardens, and improved access to new markets expanded opportunities for selling goods in local markets and to boarding schools. Improved livestock management practices (e.g., grazing and improved enclosures) and the readily available paravets resulted in improved health and general condition of cattle and goats, both of which contributed to good financial returns on the local market. Families were also grateful as higher incomes helped with school fees and health care costs. With improved animal husbandry practices, there were consequently reduced livestock losses from the extended droughts and associated diseases. The incorporation of VS&L as a source of financing across all SOs positively transformed the well-being of both men and women; goat livestock FGDs expressed their feelings of pride given their dramatic increase of goats over only a few years. Throughout most FGDs with women regarding SO3 activities, they intend to continue VS&L activities as they helped financially empower them in asset acquisition despite the harsh macro-economic environment. However, in order to sustain this positive momentum, concerted and more systematic efforts must be exerted to link VS&Ls to financial lending institutions. In sum, the evaluation team found that across all three SOs, the project components provided consistent levels of good to excellent satisfaction. Participating communities welcomed the project and felt positive regarding its outcomes. Participants felt that the interventions were appropriately designed to benefit their respective communities. 4.9 Coordination Interviews with district nutritionists, district nursing officers and ward nutrition officers showed they were highly involved in the planning and implementation of different project activities, review of project progress and field monitoring and supervision visits, gender analysis, formative research, development and pre-testing of IEC materials and trainings. IMPEL | Implementer-Led Evaluation and Learning 48 Evaluation Findings For SO1, Amalima collaborated with AGRITEX, the Department of Mechanization, Department of Livestock Production and Development, and Department of Veterinary Services to train farmer groups on crop production (dryland and horticulture), soil and water conservation, and livestock production. Amalima also participated in the Matabeleland North and Matabeleland South provincial Drought Relief and Food and Nutrition Council meetings. Amalima actively coordinated SO2 activities with external stakeholders including weekly participation in coordination meetings with Drought Relief Committees30 and quarterly District Food and Nutrition Security Council ward-level and provincial-level meetings. Amalima participated in the Provincial Health Executive meeting for Matabeleland South and shared key programmatic information on Integrated Management of Acute Malnutrition trainings and the status of active screening. Amalima continued to support and strengthen linkages between CHCs and government extension workers. Environmental health technicians continued to actively support participatory health and hygiene education and monitor model households and CHC activities (Amalima 2016a). All field officers who were interviewed reported that Amalima conducted community dialogues and engaged all stakeholders in the review of program activities and shared ideas on possible improvements. The field officers reported that these dialogues helped to enhance the community leaders’ appreciation and understanding of the role of Amalima community volunteers. As a result, many community leaders recognized volunteers during community gatherings, which increased their level of intrinsic motivation. The program also emphasized the role that traditional leaders played in advocating for women to hold leadership positions in project activities and in the community generally. During gender dialogues, participants identified the roles traditional leadership played to promote community acceptance of women in leadership positions, and to build women’s confidence to take on these roles (Amalima 2018a). The evaluation found evidence of strong working relationships between MoHCC staff, care group volunteers (CGVs) and CHC participants, all of which helped avoid creation of parallel structures. The MoH was involved in selecting beneficiaries and making decisions on how to engage community health structures comprised of village health workers and their adherence to national standards. MoH staff were also involved in pre-testing and final adoption of the IEC materials. Effective coordination in the health sector improved the integration of food distribution into child health activities and provided opportunities for child growth monitoring, identification and enrollment of severely malnourished children into care systems, and more effective BCC messaging to caregivers. Although collaboration with government counterparts was strong, full participation of government counterparts in Amalima activities was negatively affected by the project’s contractual inability to provide allowances/per diems to facilitate participation of government stakeholders in key activities. One example is inconsistencies in AGRITEX extension officers receiving transportation and meal support from AGRITEX to attend certain Amalima trainings (KIIS with district AGRITEX staff and Amalima IPs). 4.10 Gender Considerations Amalima mainstreamed gender in all project activities and encouraged equal participation of all men and women in all activities. This was guided by the well-designed gender strategy (Amalima 2015a) that was informed by a systematic gender analysis study (Amalima 2014b). 30 Drought Relief Committees are governmental committees convened at the district level and chaired by the District Administrator, which discuss drought-related issues such as the availability of stock feed. Final Performance Evaluation of the Amalima Development Food Assistance Project in Zimbabwe Evaluation Findings 49 The evaluation showed a significant change in self-perception among women. Most women in the SO3- related FGDs reported that they were beginning to see themselves as possible leaders. Their perceptions were there are increasingly more women in leadership positions on village committees including the VS&Ls and dam or gardening irrigation scheme AMCs. More women were participating in joint decision making on financial matters and asset sales unlike in the past when they were excluded. They also talked about the reductions in their workloads at home as being positive and they acknowledged that more men were participating in household chores and other activities such as gardening and caring for babies at home. Interviews with most project staff, as well as male and female beneficiaries showed that the project increased women’s participation and decision making in households and the community by mainstreaming gender across all project interventions, using monthly gender messages to enhance gender awareness in communities, training women on how to participate in decision-making, and increasing male participation in activities favored by women such as VS&Ls and CHCs. A number of project initiatives were designed to address the heavy workload of women in these districts, which existed due partially to deeply-entrenched social and cultural traditions, but also to the high rates of male emigration to South Africa, Botswana and other countries for employment. KIIs with field officers and FGDs with a range of male and female beneficiaries indicated a number of positive advancements: eco-stoves have shortened cooking times; representation of women on DRR and CHC committees was strengthened; women were empowered to own assets, particularly livestock; men and grandmothers were targeted to improve their knowledge of child and health care for household members, including children; many men were now accompanying their partners to food distribution points and participated in free health education sessions; male involvement in household chores was promoted in the Male Champions peer-to-peer approach; and participation of young mothers in care groups increased through sporting and cooking class demonstrations and competitions. 4.11 Environmental Considerations The project provided both technical and material support to augment water supplies based on specific requirements of ward and community needs but also on available resources, local geophysical characteristics, and water availability. Water provisioning for livestock and nutrition gardens was a critical element of sustainability as water is essential for agricultural production, livestock rearing, and household health. The Dabane Trust, working in collaboration with AGRITEX, identified aquifers for local community use and the most appropriate abstraction system. The history of water-provisioning infrastructure initiatives in these districts proved challenging as many older dams had fallen into disrepair and the macro-economic problems of 2007-09 seriously challenged communities’ ability to maintain equipment (dam FGDs in Tsholotsho and Mangwe). Complicating matters further was the lack of buy-in from traditional community leaders who regarded water projects as gifts from governments and NGOs (FGD in Gwanda). From the very beginning, the project applied the Amalima approach in efforts to strengthen local capacities to make strategic investments in water points and irrigation, and to build the financial and technical capacity to maintain them. The majority of dam AMC FGDs indicated satisfaction with Amalima’s approach in their decisions regarding rehabilitation of older dams, choice of location for newly constructed dams, and the process by which communities request approval for dam projects and assistance for technical and/or material support for any water provisioning initiative. A few participants expressed some frustration with the timeline for securing approval for improvements targeting water provisioning, but most were understanding of the demands both Amalima and the Ward Councils had in terms of requests made. IMPEL | Implementer-Led Evaluation and Learning 50 Evaluation Findings While the project participants in these districts overwhelmingly reported support for all of the water provisioning projects and attended trainings on maintenance and monitoring, some communities (in Gwanda and Mangwe districts) reported weariness at having to spend their labor time working on siltation issues, given its competition with tasks they deemed more imminent on their seasonal calendar such as land preparation, planting and harvest. A number of irrigation schemes, where water was drawn from alluvial aquifers in the dryland areas, (usually surface-dry sand riverbeds), were set up to support the nutrition gardens. These systems were designed to be small-scale and located where there were water deficits. The project took precautionary measures—through river catchment and site investigations and site-specific environmental impact assessments—to guard against negative environmental impacts downstream (HQ staff KII). All systems are small-scale; no massive or terminal dams were constructed that would cause cessation of river flow. Dabane Trust worked to select perennial sites so community residents would not need to seek other water sources for part of the year (Amalima 2014a). Over the course of the project, Dabane conducted other pathological and chemical assessments of water, including testing for chemicals released from unregulated mining enterprises and toxic substances such as cadmium. Testing procedures involved Dabane and representatives from the Ministry of Health (HQ staff KII). Where gullies formed, the project worked with soil and water conservation engineers who formulated specific plans for the area in which the gully had formed, and a DRR committee was trained to maintain constructed physical structures and biological barriers (DRR FGD, Tsholotsho). The DRR committee was responsible for continuing to improve soil and water conservation in the gully catchment area, working with the community to construct mechanical conservation works and erecting physical barriers in developing gullies (HQ staff KII). The NRM/DRR coordinators worked with the project’s agronomist to develop a series of locally appropriate CA practices to enhance soil fertility and support the rehabilitation of communal arable and grazing lands. FGDs with lead farmers and livestock groups indicated the following techniques as to what comprised CA: ridges between rows, terracing along contours accompanied by planting of trees and forages, and mulching of crop residues. Gaps existed between the more extensive range of topics the project offered in its four CA training modules and what was reported in the FGDs, even though participants had attended all of the CA trainings. Low CA adoption rates were attributed to labor intensity requirements and input availability and accessibility limitations (HQ and district-level staff). Farmers must repeat CA techniques every year as a result of livestock grazing on their land and destroying planting pits. With the assistance of the HQ Behavior Change Communication Specialist, the project developed strategies to achieve higher adoption rates. VAC FGDs in Mangwe and Gwanda reported they felt the discussion guides (Amalima 2018c) effectively reached farmers in improving their level of understanding as to the concrete benefits of CA. The Malondolo Gully Reclamation in Ward 18 of Tsholotsho demonstrates the effectiveness of Amalima’s gully reclamation process. This involved district-level stakeholders including the Rural District Council, Environmental Management Agency, the LPD (Livestock Production and Development) department, and AGRITEX, working with qualified and experienced engineers who designed the reclamation process and structural and biological measures to reclaim the gully (Amalima 2014d). Ultimately a masonry weir was constructed as the most significant physical barrier whose foundation is solid bedrock that also reduces the probability of dam failure through undercutting (DRR FGD, Tsholotsho). The DRR committee at the site is actively involved in conducting regular inspection and maintenance (Amalima 2014d). Final Performance Evaluation of the Amalima Development Food Assistance Project in Zimbabwe Evaluation Findings 51 The Amalima project assisted communities, traditional leaders, and Rural District Councils with efforts to ensure optimal allocation of land for livestock grazing and agricultural production. The project focused its efforts in livestock husbandry—the primary component of local livelihoods—across all four districts (HQ staff KII). The Amalima Livestock and NRM/DRR coordinators guided field officers to assist communities to develop management systems for rotational grazing as well as grazing paddocks. Community facilitators addressed the rehabilitation of degraded lands, targeting reduction of rainfall runoff and increased infiltration to retain soil and land surface cover (DRR FGDs in Mangwe, Gwanda), all of which was designed to improve land productivity. Livestock FGDs in Mangwe and Bulilima reported gaining important information that addressed pasture reclamation through soil preparation, planting of indigenous grass seeds, and construction of water harvesting bunds. DRR FGDs in Mangwe, Gwanda, and Tsholotsho districts were trained on paddock construction for more effective containment and protection of livestock, especially at night. Several invasive species were either introduced into the project region (prior to project onset) or are native to southern Africa. In either case, understanding the dangers of invasive species, deforestation, and environmental degradation ensures acceptance and participation in environmental rehabilitation activities (CNFA and World Vision 2016). For example, in areas where communities removed lantana camara, the people realized that this paved the way immediately for growth of grass important for livestock grazing. However, in areas where the recognition of the benefits of this labor was low, communities were not motivated to participate (HQ staff KII). Overall, the training for removal of invasive species was based on using IPM techniques designed for each specific species, such as the Opuntia Factsheet (Amalima 2016b), and the Lantana Camara Factsheet (Amalima 2016c), which addressed this perennial, thicket-forming shrub could completely stall the regeneration of rangelands for several years and is classified as a noxious weed in the Zimbabwe Noxious Weeds Act (GoZ 1996). 4.12 Sustainability A number of factors are key to achieving sustainability of practices and impacts in a development program; these factors are resources, capacity and motivation, and in some cases, linkages. As found in a recent Tufts study examining sustainability in DFAPs, when a project exits, sustained sources of resources for each input that was provided by the project should be in place for ensured sustainability (FANTA III 2015b). In the case of Amalima, the financial resources generated by the VS&Ls (some VS&Ls were in place prior to the project) provided the means, as reported by VS&L members, to secure inputs needed for crop production, the nutrition gardens, and the purchase of livestock (primarily goats), as well as pay school fees and some health care costs. Given that some VS&Ls were functioning prior to the project, the boost Amalima provided created more groups, larger groups, and a stronger system of VS&Ls that, given the widespread membership, popularity and sound management of these groups, as recognized by members, the VS&L provides a sustainable resource for continued support of the agriculture production activities. In terms of marketing resources, interviews with participants suggest the newly established local network of agro-dealers may be a promising project outcome, as farming input supplies are now available – and more accessible with the strong support from VS&Ls. The rehabilitation of assets was naturally more sustainable, as the communities were already engaged in the process and the AMCs were at least partially active before the physical work began. Newly constructed assets did not always have an AMC in place at time of completion so management was sometimes slow to initiate or ineffective; sustainability is questionable in these cases. IMPEL | Implementer-Led Evaluation and Learning 52 Evaluation Findings Building quality technical and managerial capacity throughout the service delivery process and creating mechanisms to sustain that capacity are also critically important for sustainability. Improved capacity is essential to sustained behavioral change (FANTA III 2015b). Amalima provided an extensive range of high-quality, systematic, and technical training workshops and complementary modules under all three strategic objectives. Asset management committees (e.g., dams, gardens, dip tanks, etc.) were also trained to manage finances. Amalima was also structured to provide capacity building among individual participants – to improve child care, feeding practices, hygiene, and agricultural practices, and to manage their resources to be able to support these necessities. Improved capacity is critical to sustained behavioral change. The primary gaps in trainings were on leadership skills and conflict resolution competence. Project participants and government stakeholders perceived that project outcomes will be largely sustainable, especially those that are based on capacity building initiatives to foster improved knowledge. Participants felt strongly that the trainings provided knowledge and skill sets that would take them into the future in terms of agricultural production and improved nutrition and health conditions; however, there was no mechanism created or strengthened to periodically refresh participants’ knowledge, and the record on the adoption of CA techniques is mixed. While CA trainings were deemed successful in terms of knowledge and skills gained, they will not be effective if reasons behind slow adoption rates are not addressed head on. The VAC initiative and the development and use of discussion guides (Amalima 2018c) were designed to address issues around the low adoption rates. Yet some report they have not adopted CA techniques because the inputs are expensive. Perhaps earnings from the VS&L groups will improve this situation. While dams and dip tanks have been extended well with high impact and activities around them; mechanisms were not set up to specifically support the sustainability of these efforts. Gender was mainstreamed in the different trainings using the gender discussion booklet (USAID, CNFA, Amalima 2016k). By engaging men and other key decision makers, the program has removed barriers that would prevent continued adoption of key health and nutrition practices. Most of the male and female beneficiaries strongly appreciated the benefits of the increased participation of women in joint decision making, and the involvement of men in child health activities and household chores. They appreciate the benefits of gender mainstreaming and are likely to continue the improved behaviors. Care groups have strengthened cohesion among community members. Care groups engaged in VS&L activities were more cohesive than those groups who were not engaged in VS&L activities and showed high likelihood of continued functionality after the project. Interviews with care group members who participated in VS&L activities indicated stronger social cohesion; they were more united, assisted each other and frequently came together to support each other, for example during latrine construction, garden preparation, and harvest periods. Guaranteeing continued sources of motivation for service providers and beneficiaries is also a critical element of sustainability. Financial incentives and in-kind benefits are usually the most successful motivators for service providers. While personal commitment and prestige are important, they are not sufficient on their own to sustain active service delivery over the long term. For beneficiaries, a tangible and immediate benefit is the most effective motivation to continue using practices learned and making use of services (FANTA III 2015b). The Amalima project created a large pool of community-based volunteers who are considered to be a hub of knowledge within their communities. Most of the interviewed volunteers exhibited high levels of intrinsic motivation and many reported they planned to continue their work after the program ends. Nevertheless, the evaluators agree that without some form of compensation, even if a token one, interest of the volunteers will wane within a relatively short period after the project exits. The paravets are one such example. In FGDs, paravets relayed their pride Final Performance Evaluation of the Amalima Development Food Assistance Project in Zimbabwe Evaluation Findings 53 in having specialized knowledge and having completed what they perceived to be rigorous training. They expressed their desire to conduct their work, but that to do so they not only need to be able to replace consumables in their toolkits, they also need some compensation, as serving in this role means time away for their own farms and livestock, which requires them to pay others to do that work for them. Amalima created strong linkages between clinic nurses, caregiver volunteers, lead mothers and caregivers. Their personal perceptions are they will sustain activities after the project, yet to date, no tangible benefit has been put in place that would reinforce their motivation to continue in their positions. However, the strong working relationships between MoHCC staff, caregiver volunteers and CHC participants may help to sustainably promote the adoption of improved nutrition and hygiene practices. Gender was mainstreamed in the different trainings using the gender discussion booklet (USAID, CNFA, Amalima 2016k). By engaging men and other key decision makers, the program has removed barriers that would prevent continued adoption of key health and nutrition practices. Most of the male and female beneficiaries strongly appreciated the benefits of the increased participation of women in joint decision making, and the involvement of men in child health activities and household chores. They appreciate the benefits of gender mainstreaming and are likely to continue the improved behaviors. Care group FGDs reported a strengthening of cohesion among community members. Care group members discussed how care groups engaged in VS&L activities were more cohesive than those groups who were not engaged in VS&L activities, and they indicated a good likelihood of continued functionality after the project, which is possible given the benefit of the consequent income. KIIs with care group members who participated in VS&L activities indicated they felt their groups had strong social cohesion; they were united, assisted each other and frequently came together to support each other, for example during latrine construction, garden preparation, and harvest periods. In these examples, the Amalima concept does reinforce a mindset of working together for the greater good for the group. The project began to place greater emphasis on sustainability in Year Three, with a very specific weighting during the 2018-19 extension (Amalima 2017a), whereas it should have been planned for and monitored from project onset. The main gaps in more effectively addressing sustainability/replicability, even during the extension, were that local project leaders (e.g., lead farmers, lead mothers, VACs, etc.) were not sufficiently equipped to exercise better leadership and governance. While the project offered business-related skills to aid in leadership of AMCs, the trainings did not provide guidance centered on team building and conflict resolution. It was mentioned several times during FGDs and KIIs with field staff that project branding had a negative and significant effect on local ownership identification and fostering project dependency, particularly when there was a large USAID/Amalima/CNFA sign placed close to the gate of a fenced-in area (irrigation gardens, dams, dip tanks, and gully reclamation) or on DRR community plans or most of the training materials. The relative positioning and sizes of the logos and text can give an erroneous impression of project “ownership.” Some participants of varied FGDs asked, for example, whether garden projects would continue once Amalima left given they were “owned” by project donors. This level of confusion may result in less motivation and commitment to certain project initiatives over time unless clear indication is given that all initiatives were meant to benefit community and ward members over the long term and after the project is complete. Considering that youth are key for sustainability and the difficult and late project engagement with youth, it is unfortunate that there was a contractual obligation not to involve the school system, especially at upper secondary level, where students will soon be young adults and face the issues addressed by the project. In addition, the Male Champion initiative, which largely involved older males, IMPEL | Implementer-Led Evaluation and Learning 54 Evaluation Findings most likely will not be sustained unless younger males who emigrated for work return to Zimbabwe, as currently there are few younger males in the region to learn these knowledge and skill sets. Adding garden membership to male-based initiatives could step up the probability of sustainability. In addition, although the Male Champions are strongly linked with community leaders, they do not have a direct link with the health facility nurse or environmental health technician, which will also affect sustainability of their activities. CONCLUSIONS In sum, sustainability will be directly influenced by the ability and motivation of local community members to want to maintain the activities and assets provided by the project, and given their capacity to secure the resources that will be required to do so (FANTA III 2015b). Part of their motivation to act after the project exits will be determined by the nature of the return on their investment, be it economic or social or both. Most probable initiatives include the garden groups, which are small and easier to manage but have a high return in terms of contribution to food and nutrition security; VS&Ls, which provide opportunities for financial investment and access to resources for improvement and as safety nets; locally based agro-dealers, who provide easy access to agricultural inputs; the strong working partnerships established with CGVs and CHC participants to achieve improved nutrition and hygiene practices; and those gender initiatives that removed pre-existing barriers that had prevented the adoption of improved health, child care, and nutrition practices. 4.13 Lessons Learned Having a concept such as Amalima that encourages working together toward common goals proved instrumental in meeting the objectives of this project. The ideas of the Amalima approach transcended all project initiatives. The revival and restructuring of the traditional Amalima concept is highly commendable since the resultant activities strongly aided in achieving enhanced food security for many, and improved nutrition and hygiene in participating communities. The consortium structure in program planning and implementation and the collaboration mechanism with government was usually effective across the course of the project and should be replicated. Although the mixture of expertise acts as an advantage, it can also be a challenge at times to have the team arrive at a collective decision or reach general consensus. Processes for decision-making should be determined prior to project onset; fostering this across all activities was challenging (HQ KIIs), particularly on the agricultural side. Confusion and misunderstandings or conflicts sometimes threatened the success of certain initiatives, particularly early on in the project. Communication channels were effective; directing project information from district offices through to field officers and lead mothers and farmers, as well as AMCs and CHCs once they were formed, was regarded as efficient and successful. Participants and stakeholders expressed satisfaction with the project’s communication practices and generally knew when the trainings, CFAs, and other project activities were scheduled as well as when the project would be completed. Instrumental in this success were the roles community members played as intermediate leaders whose communications were trusted and respected by participants. Over time, this structure also contributed to the enhancement of social capital and reinforced the Amalima project approach of working together that was evidenced throughout the FGDs and community and district-level KIIs. The process used for developing training materials was particularly effective and should be replicated. These materials—learning units, topical modules in CA, DRR, and the recipe book, for example—were well organized and constructed, socially and culturally appropriate, and presented in local languages Final Performance Evaluation of the Amalima Development Food Assistance Project in Zimbabwe Evaluation Findings 55 accompanied with pictorial illustrations for those who were not literate. Cascade training could be effective but the project had better quality control on the SO3 side of the project than for SO1 and SO2; this should have been replicated across the entire project. Cascade trainings would not work well for FaaB or any new subject or something needing experimentation, as such topics require regular reinforcement and knowledgeable coaching. Inter-sectoral integration was realized through the work of CBFs, VACs later in the project, and the VS&Ls. The VS&Ls were the core activity that linked economic, social, gender, environmental and health outcomes. VS&Ls were particularly effective for improving household-level and community resilience during deleterious climate and economic conditions. The vast majority of FGD participants across all three strategic objectives were members of at least one VS&L; many were members of two or more VS&Ls. In the Zimbabwe context, mobile phone technology was very useful for both participants and implementers (communication, training/learning, measuring, reporting) and its role should be expanded, especially in an area with low radio coverage and few regular markets. As reported in Male Champion FGDs, the inclusion of traditional leadership in project implementation assisted in adapting to negative cultural norms in communities, particularly around the Male Champions initiative. The project gave a solid effort at empowering community-based volunteers who took pride in their work. As discussed in Sec. 4.12, sustainability of these efforts may be threatened without the community-based volunteers and others, including paravets, receiving compensation. Retaining incentive over longer time periods may prove to be challenging without at least token payments. The peer recognition model is good, but a minimal degree of incentive (from communities) may aid the probability of longer-term sustainability. KIIs with most lead mothers and caregivers indicated that home visits provided a unique opportunity of reaching out to other key decision makers in the homes such as the men, grandmothers and mothers-in￾law with similar messages, and cooking classes conducted during care group sessions provided caregivers firsthand knowledge on preparing nutritious meals using locally available ingredients. There is a traditional tendency to target women for MCH intervention in public health systems. Men are critical targets for enhanced uptake of MCH intervention because women are not key decision makers. It is critical to always engage the other spheres of influence who are key decision makers at home when rolling out BCC messaging. Linking agriculture with nutrition interventions maximized the intervention impact on key nutrition outcomes. Amalima promoted CA, improved post-harvest handling, better livestock practices, increased diet diversity, and nutrition gardens among the care groups. The combined effect of all of these approaches resulted in a significant reduction in stunting levels among children, as detailed in Sec. 4.4. The consortium concept in program planning and implementation and the collaboration mechanism with the GoZ should be strengthened and replicated. Strong coordination with respective government structures is key to project success. It helped to make early refinements on the project design, prevented creation of parallel structures, promoted integration, and helped to promote ownership and scaling up of effective interventions. Implementation research helps to improve the quality and effectiveness of programs. A comprehensive learning and research agenda should be incorporated in program design. IMPEL | Implementer-Led Evaluation and Learning 56 Recommendations 5. Recommendations R1: Project planning. In order to develop complex interventions which are more likely to be effective, sustainable and scalable, project planners need to understand not just whether, but how and why an intervention has a particular effect, and which parts of a complex intervention have the greatest impact on outcomes. For this, a prospective, theory-driven process of intervention design and evaluation is required (DeSilva, Breuer and Patel 2014). Theory of Change (TOC) approaches ultimately indicate how and why an initiative works which is empirically tested for every expected step on the path to impact. It should be developed in collaboration with stakeholders and modified throughout the intervention and evaluation process through an ongoing process of reflection to explore change and how it happens (DeSilva, Breuer and Patel 2014). It is a pragmatic framework which describes how the intervention affects change. The Amalima project should have organized community needs assessments in the planning formulation phase, as well as developed a detailed, sectoral TOC from which to justify and formulate specific project activities. Community needs assessments are conducted prior to taking action and are used to determine current situations and identify issues for action, establishing the essential foundation for vital planning. The process is an invaluable tool for involving the beneficiaries in solving problems and developing goals and is a key component of sustainable outcomes (Ervin 2005). Research shows if there is no community-level involvement in planning for community development, then the likelihood of sustainability of programs is very low (Kedia and van Willigen 2005). In the Amalima proposal, the primary objectives were stated clearly in terms of project activities, but the planners neglected to define a comprehensive set of strategies to ensure sustainability of project initiatives under a number of economic and political scenarios. This approach was not undertaken during the planning phase; the extension document has some discussion of this component, but it is mostly a listing of what additional activities they want to carry out prior to exit. When asked about project planning, the participants were unaware of any planning that involved community members prior to the project onset. R2: Timeline. For a five-year project, an effective timeline would be: Year One: inception phase to involve training, trust and ownership building, setting up systems for sustainability and quality implementation (studies, pilot projects). Years Two and Three: full implementation with large community support. Year Four: project steps back and lets participants and stakeholders manage the majority of the work. Year Five: actively work on sustainability/replicability and filling gaps, no new activities, refresher activities but no new training. In a five-year program, where sustainability is a primary goal, the first two years should not place undue pressure to reach certain quantitative goals to the detriment of quality and sustainability, but rather lay down a solid basis for participants and stakeholder participation and ownership. Allow the first year to be an inception phase to adjust the original design and planning to better fit the reality and put all systems in place. R3: Trainings. Develop specific leadership, governance, and conflict resolution training, especially for CBFs, DRR/AMC, and traditional and religious leaders. Produce these training materials in the same manner as completed trainings, with simple text, in local languages, and with good graphic support. R4: Quality management. Establish a Quality Management department at the beginning of the project, to be responsible for monitoring, evaluation, accountability and learning (e.g., KAP and barrier studies); project-wide accountability; administrative and technical compliance; sectoral integration; relevant Final Performance Evaluation of the Amalima Development Food Assistance Project in Zimbabwe Recommendations 57 policies and strategies; a theory of change that is regularly revisited; taking sustainability and self￾replication into account from the start of the project; managing project data both to meet contractual obligations and to be useful to managers (e.g., via integrated databases, unique IDs, data quality assurance); and providing real-time information to feed into current and future programming. Considering that FFP development activities are similar across countries, it could be useful to have a standardized but customizable system for data acquisition and management, as each project struggles to put something basic in place, usually in the second year and still with gaps. A well-designed system could also be useful for project integration, quality management, and evaluation. R5: Communications. Scale up the use of phone technology in training, networking, and reporting. The use of WhatsApp by agro-dealers provided higher efficiency with effective communication, and lead farmers in Tsholotsho were using WhatsApp for notification of trainings, meetings, and other activities. R6: Water provisioning. The most recurrent community request was for perennial community water points for households, horticulture, or livestock use. Considering the increasing severity of drought conditions and the positive impact of the dams and other water infrastructure, future projects should prioritize such water amenities with local management and sustainability at the core. R7: Asset ownership. To ensure better community buy-in and sustainability, the branding of assets and documents (e.g., infrastructure, training material, disaster risk reduction plans) should highlight first and foremost the engagement of participants and stakeholders, with financial and implementing support less prominent. R8: Scale up care group model. The Ministry of Health and Child Care has started rolling out the care group methodology in non-project districts based on evidence of effectiveness from the project. However, there are no clear data on the costs of scaling up sustainable, innovative, community-based incentives, enhancing adolescent and young mother participation in care groups, or how to harness health for greater effectiveness. In order to support a more strategic approach for scaling up the care group model to other districts and countries, evidence should be documented to establish the causal impact of the care group model on maternal and child health outcomes; determine the cost benefit of scaling up the model compared to the standard of care; and identify innovative approaches for enhancing adolescent and young mother participation. IMPEL | Implementer-Led Evaluation and Learning 58 Annex A: References Annex A: References Alkire, Sabina, Ruth Meinzen-Dick, Amber Peterman, Agnes R. Quisumbing, Greg Seymour and Ana Vaz. 2012. The Women’s Empowerment in Agriculture Index. 2012. IFPRI Discussion Paper 01240 International Food Policy Research Institute. December. https://www.ifpri.org/publication/women’s-empowerment-agriculture-index Amalima. 2014a. Amalima Project Proposal document (AID-FFP-A-13-00004 2013-2000). Cultivating New Frontiers in Agriculture (CNFA). Amalima. 2014b. Amalima Gender Analysis Report. Learning Unit document. Bulawayo: Cultivating New Frontiers in Agriculture (CNFA). Amalima. 2014c. Amalima Annual Project Report. Cultivating New Frontiers in Agriculture (CNFA). Amalima. 2014d. Report on Reclamation of the Malondo Gully – Ward 18 of Tsholotsho District. Bulawayo: CNFA, Amalima. Amalima. 2015a. Gender Strategy. Cultivating New Frontiers in Agriculture. Bulawayo: CNFA. Amalima/USAID. 2015b. Strategic objective three formative research report on infant and young child feeding practices. Bulawayo: CNFA, Amalima, February. Amalima. 2016a. Amalima Annual Project Report. CNFA, November. Amalima. 2016b. Opuntia Fact Sheet. Bulawayo: CNFA, Amalima. Amalima. 2016c. Lantana Camara Fact Sheet. Bulawayo: CNFA, Amalima. Amalima. 2016d. Asset ownership and decision-making study draft report. Learning Unit document. Bulawayo: Amalima. Amalima. 2016e. Learning Report Input Supply Models. Learning Unit document. Bulawayo: Amalima. Amalima. 2016f. VS&L Study Final Report. Learning Unit document. Bulawayo: Amalima. Amalima. 2017a. Amalima FY 18 Detailed Implementation Plan (DIP) and Summarized FY 19 & FY 20 DIP. Bulawayo: Amalima. Amalima. 2017b. FaaB: What is a Business? Project Training Manuals. Bulawayo: CNFA (Amalima). Amalima. 2018a. Amalima Annual Project Report. Cultivating New Frontiers in Agriculture (CNFA), January 2018. Amalima. 2018b. Study Youth Participation. Learning Unit document. Bulawayo: Amalima. Amalima. 2018c. Conservation Agriculture and Livestock: Discussion Guides. Bulawayo: Amalima. Amalima. 2019a. Amalima Wards and Asset Maps. Bulawayo: Amalima project, USAID, CNFA, Amalima. Amalima. 2019b. Subsidized HHAV [Household Asset Voucher] PDM [Post-Distribution Monitoring] draft report. Learning Unit document. Bulawayo: Amalima. Bernard, H. Russell. 2017. Research Methods in Anthropology: Qualitative and Quantitative Approaches. Lanham, MD: Rowman & Littlefield, Publishing Group. CARE. 2010. Access Africa Technical Learning Series: No. 1. Internet source. CARE. https://www.careinternational.org.uk/linking-for Final Performance Evaluation of the Amalima Development Food Security Activity in Zimbabwe Annex A: References 59 change/images/technical%20learning%20series%20publication%201%20- %20%20village%20agent.pdf CNFA and World Vision. 2016. Mid-Term Evaluation Report for the Development Food Assistance Programs: ENSURE and Amalima. Amalima Agreement No. AID-FFP-A-13-00004. Washington D.C.: USAID, December. DeSilva, Mary J., E. Breuer, and Vikram Patel. 2014. Theory of Change: a Theory-driven approach to enhance the Medical Research Council’s framework for complex interventions. Trial 15: 267. Ervin, Alexander. 2005. Applied Anthropology: Tools and Perspectives for Contemporary Practice. Boston: Allyn and Bacon. FANTA III. 2015. FFP Indicators Handbook Part I: Indicators for Baseline and Final Evaluation Surveys. April. Washington, DC: Food and Nutrition Technical Assistance III Project (FANTA III). FANTA III. 2015b. Sustaining Development: A Synthesis of Results from a Four-Country Study of Sustainability and Exit Strategies among Development Food Assistance Projects, Beatrice Rogers and Jennifer Coates. Boston: Gerald J. and Dorothy R. Friedman School of Nutrition Science and Policy, Tufts University. FAO. 2010. Guidelines for measuring household and individual dietary diversity. Rome: FAO. FEWS NET. 2018. Food Security Outlook Update: Increasing non-staple food and other commodity prices further strain poor households’ access to food, September 2018. Available at http://fews.net/southern-africa/zimbabwe/food-security-outlook-update/september-2018 Food and Nutrition Council. 2018. Zimbabwe National Nutrition Survey – 2018: Preliminary Findings. Harare: Food and Nutrition Council. Food and Nutrition Council. 2012. Zimbabwe National Nutrition Survey – 2012: Preliminary Findings. Harare: Food and Nutrition Council. Food and Nutrition Council. 2011. Zimbabwe National Nutrition Survey – 2010: Preliminary Findings. Harare: Food and Nutrition Council. Food and Nutrition Council, Ministry of Health and Child Care, USAID, Amalima, 2015. A Healthy Harvest: An Adapted Illustrated Guide for Amalima Community Trainers, Module 1: Nutrition. Harare: Zimbabwe Ministry of Health and Child Care. FSIN (Food Security Information Network). 2019. Global Report on Food Crisis: Zimbabwe. Rome: UN FSIN. GoZ. (Government of Zimbabwe). 1996. Noxious Weeds Act. Harare: Government of Zimbabwe. Joint initiative of IUCN, UNEP, FAO. Kedia, Satish and John van Willigen. 2005. Applied Anthropology: Domains of Application. Westport, CT: Praeger Publishers. Maxwell, D., J. Coates and B. Vaitla. 2013. How Do Different Indicators of Household Food Security Compare? Empirical Evidence from Tigray. Medford, USA: Feinstein International Center, Tufts University. UNICEF. 2015. Water Supply and Sanitation in Zimbabwe Turning Finance into Services for 2015 and Beyond. Rome: UNICEF. IMPEL | Implementer-Led Evaluation and Learning 60 Annex A: References USAID Office of Food for Peace. 2012. Zimbabwe USAID-BEST Analysis: Bellmon Estimation Studies for Title II. Washington D.C.: FFP USAID. USAID. 2015. Baseline study of the Title II Development Food Assistance Programmes in Zimbabwe. Contract #: AID-OAA-13-00022. Washington, D.C.: USAID/ICF, June 3. USAID, CNFA, Amalima. 2016a. Animal Health Fact Sheet for Lead Farmers. Bulawayo: Amalima. USAID, CNFA, Amalima. 2016b. Cattle Management Calendar. Bulawayo: Amalima. USAID, CNFA, Amalima. 2016c. Livestock Rearing Module 1: Livestock Nutrition. Bulawayo: Amalima. USAID, CNFA, Amalima. 2016d. Livestock Quality Improvement Fact Sheet for Lead Farmers. Bulawayo: Amalima. USAID, CNFA, Amalima. 2016e. Chicken Production, Module 1: Indigenous Chickens. Bulawayo: Amalima. USAID, CNFA, Amalima. 2016f. Module 1: Land Preparation and Planting. Bulawayo: Amalima. USAID, CNFA, Amalima. 2016g. Module 2: Manure, Compost and Fertilizer Application. Bulawayo: Amalima. USAID, CNFA, Amalima. 2016h. Module 3: Planting, Thinning, Weeding and Top Dressing. Bulawayo: Amalima. USAID, CNFA, Amalima. 2016i. Module 4: Mulching, Inter-cropping, Crop Rotation, Harvesting and Post￾harvest Handling. Bulawayo: Amalima. USAID, CNFA, Amalima. 2016j. Nutrition Meals and Snacks for Young Children: Tickle your child’s taste buds with locally available foods. Bulawayo: Amalima. USAID, CNFA, Amalima. 2016k. Let’s Talk about Gender: Gender Discussion Guide. Bulawayo: Amalima. USAID, CNFA. 2017a. Amalima FY 18 and Extension DIP Narrative, and Summarized FY 19 & FY 20 DIP. Washington D.C.: U.S. Agency for International Development Food for Peace Development Food Aid Program. Cooperative Agreement No. AID-FFP-A-13-0 0004. USAID, CNFA, Amalima. 2017b. Amalima Annual Results Report FY 2016. Washington D.C.: U.S. Agency for International Development Food for Peace Development Food Aid Program. Cooperative Agreement No. AID-FFP-A-13-00004. World Bank. N.d. Living Standards Measurement Study of the World Bank. Available at http://web.worldbank.org/archive/website00002/WEB/INDEX-5.HTM WHO (World Health Organization). 2011. WHO Anthro and Macros, version 3.2.2. Available at http://www.who.int/childgrowth/software/en/ Zimbabwe National Statistics Agency. 2012. Zimbabwe Demographic and Health Survey 2012. Harare: Zimbabwe National Statistics Agency. Zimbabwe National Statistics Agency. 2013. Poverty and Poverty Datum Line Analysis in Zimbabwe 2011/12. Harare: ZimStat. ZimVAC. 2012. Zimbabwe Vulnerability Assessment Results. Harare: Regional Vulnerability Assessment and Analysis Programme. ZimVAC. 2018. Vulnerability Assessment Committee Results: Rural Livelihoods Assessment. Harare: Regional Vulnerability Assessment and Analysis Programme. Final Performance Evaluation of the Amalima Development Food Security Activity in Zimbabwe Annex B: Evaluation Statement of Work 61 Annex B: Evaluation Statement of Work Statement of Work for Endline Study: Title II Development Food Assistance Programs in Zimbabwe I. Introduction A. OVERVIEW The final evaluation of the Title II Office of Food for Peace (FFP) Development Food Assistance Programs in Zimbabwe is the second and final phase of a pre-post evaluation strategy. In FY 2013, FFP awarded Cultivating New Frontiers in Agriculture (CNFA) and World Vision two five-year Title II projects in Zimbabwe: (1) the Amalima I Program in western and southwestern Zimbabwe, implemented by Cultivating New Frontiers in Agriculture (CNFA) and partners: the Organization for Rural Associations for Progress (ORAP), International Medical Corps (IMC), The Manoff Group, Africare, and Dabane Trust, and (2) Enhancing Nutrition, Stepping Up Resiliency and Enterprise (ENSURE) in eastern Zimbabwe, implemented by World Vision and partners: CARE, SNV USA, Southern Alliance for Indigenous Resources and International Crops Research Institute for the Semi-Arid Tropics. The baseline study was conducted by ICF International from January through August 2014 and employed a mixed methods approach.31 The study investigated population characteristics; household hunger and coping strategies; dietary diversity and food consumption; poverty; water, sanitation and hygiene practices; agricultural practices; women’s health and nutrition; children’s health and nutrition; and gender equality. Through this solicitation, FFP seeks a firm (referred to in this document as “the Contractor”) to conduct an endline study to determine conditions in targeted areas of Zimbabwe prior to the start of new Title II programs. FFP requires a representative population‐based household survey focused on the collection of data for the required impact and outcome indicators for Title II program intervention areas. The study will also include a qualitative component that will add depth, richness, and context and serve to triangulate information from survey findings and analysis. B. EVALUATION PURPOSE AND OBJECTIVES The purpose of the endline study is to measure the development outcomes of the Amalima and ENSURE projects. The endline study is designed as the first step in a two‐part evaluation process, following the baseline at the beginning of the program. Endline data should be conducted at approximately the same time of the year as the baseline, which was late March through May. Further, endline data should be collected as close as possible to the end of the program. Given that the lean season coincides with the rainy season, the Contractor should be aware that certain areas where data collection will occur may be difficult to access. The Famine Early Warning System Network (FEWSNET) graph below shows the seasonal calendar and critical events timelines for Zimbabwe. Note that this figure corresponds to the country in general; the specific zones in which the Title II programs are working may have a seasonal calendar that varies 31https://www.usaid.gov/sites/default/files/documents/1866/Zimbabwe%20Baseline%20Study%20Report%2C%20June%20201 5.pdf IMPEL | Implementer-Led Evaluation and Learning 62 Annex B: Evaluation Statement of Work slightly from this graph. After contract issuance, the Contractor should confirm with FFP and the USAID Mission in Zimbabwe when data collection will take place. Figure 5: FEWSNET seasonal calendar and critical events timeline for Zimbabwe The specific objectives of the endline study are the following:  Determine the endline values of key impact and outcome level indicators—disaggregated by awardee, age, and sex as appropriate— in addition to endline values of demographics in target areas and appropriate independent variables;  Conduct bivariate and multivariate analyses of impact and outcome indicators with independent variables identified for inclusion in survey as appropriate, with results provided by awardee and the overall Title II country program area;  Use qualitative data to ground‐truth survey data and provide contextual information on the overall food insecurity and malnutrition situation; and  Assess progress toward end‐of‐program targets for impact and outcome indicators. While the endline study will be externally designed, led, and reported on by the Contractor, staff from FFP and the USAID Mission in Zimbabwe will provide input and be involved during all stages. The Contractor will consult with Title II awardees to understand the program description and theory of change, obtain inputs for the quantitative survey instrument and qualitative study, and receive contextual information to properly develop a sampling and logistics plan. In discussion and coordination with FFP, the Contractor will provide draft and final versions of specific deliverables to the awardees for review and information. II. Indicators for Collection and Endline Evaluation Questions A. INDICATORS FOR COLLECTION The Contractor will be responsible for collecting data on all applicable indicators listed below, plus a limited number of additional indicators for each Title II development food aid program awardee, including women’s status and empowerment indicators. The final list of indicators to be collected will be discussed and agreed upon in consultation with FFP, the USAID Mission in Zimbabwe, and each of the FY 2013 Title II awardees. The FFP Indicators for the baseline and final evaluation surveys are:  Prevalence of underweight children under five years of age  Prevalence of Poverty: Percent of people living on less than US$1.25/day Final Performance Evaluation of the Amalima Development Food Security Activity in Zimbabwe Annex B: Evaluation Statement of Work 63  Mean depth of poverty  Per capita expenditures (as a proxy for income) of USG targeted beneficiaries  Prevalence of stunted children under five years of age  Prevalence of underweight women (of reproductive age)  Percentage of farmers who used at least [a project‐defined minimum number of] sustainable agriculture (crop/livestock and/or NRM) practices and/or technologies in the past 12 months  Percentage of farmers who used at least [a project‐defined minimum number of] improved storage techniques in the past 12 months  Percentage of farmers who used financial services (savings, agricultural credit, and/or agricultural insurance) in the past 12 months  Percentage of farmers who practiced the value chain activities promoted by the project in the past 12 months  Household Hunger Scale (HHS): Prevalence of households with moderate or severe hunger  Average Household Dietary Diversity Score (HDDS)  Percentage of children 6‐23 months of age receiving a minimum acceptable diet  Women’s Dietary Diversity Score (WDDS): Mean number of food groups consumed by women of reproductive age  Prevalence of exclusive breastfeeding of children under six months of age  Percentage of children under age five who had diarrhea in the prior two weeks  Percent of children under five years old with diarrhea treated with Oral Rehydration Therapy (ORT) 18. Percentage of households using an improved drinking water source  Percentage of households with access to an improved sanitation facility  Percent of households with soap and water at a handwashing station commonly used by family members  Women’s status and empowerment indicator(s) and/or awardee gender objectives as identified in the results frameworks The Contractor will closely follow the guidance on the FFP Standard Indicators Handbook for Baseline and Final Evaluation for indicator definition, collection, and analysis for the indicators listed above.32 In several instances, the Contractor will have to refer to the source documents used to develop the FFP Standard Indicators Handbook for Baseline and Final Evaluation for instructions on adapting questionnaires to the local context, as well as other important details on data collection and tabulation. The Contractor will also work closely with FFP, the USAID Mission in Zimbabwe, and Title II awardees to develop questionnaires and tabulation instructions for the agriculture indicators (#7‐10), gender indicator(s), and any additional program‐specific indicators not specified in the Handbook. For the poverty prevalence indicator, the Contractor will closely follow FTF guidance for indicator definition, collection, and analysis.8 To derive the mean depth of poverty indicator, the Contractor will use the same per capita expenditure data used for the poverty prevalence indicator. The Contractor will work closely with FFP to develop tabulation and analysis instructions for this indicator. The Contractor will ensure that rigorous practices are used to collect, tabulate, and analyze the indicator data. Refer to Section III of this SOW for further information on the required quantitative methodology. 32 The FFP Standard Indicators Handbook for Baseline and Final Evaluation can be found at http://www.usaid.gov/what‐we‐ do/agriculture‐and‐food‐security/food‐assistance/guidance/food‐peace‐information‐bulletins IMPEL | Implementer-Led Evaluation and Learning 64 Annex B: Evaluation Statement of Work B. EVALUATION QUESTIONS FFP has identified preliminary evaluation questions that will guide the design and development of endline study. To answer the evaluation questions, the Contractor will be responsible for referring to baseline quantitative and qualitative data as a basis for comparison. In concert with the Title II program awardees, the Contractor is expected to assess the technical viability of the evaluation questions and incorporate specific elements in the design and methodology of the baseline study (both the quantitative and qualitative components) to ensure that the endline study provides valid and reliable data and responds to the evaluation questions. This might involve incorporating additional variables or strata in the design of the household survey and the qualitative component. The following table lists the primary evaluation questions: Criteria Main evaluation questions Sub-questions Impact 1. To what extent did the programs achieve the intended goal, objectives and results as defined by their Results Framework? 2. How did program activities improve the ability of beneficiary households and communities able to mitigate, adapt to, and recover from food security shocks and stresses? 1.1 Were there any important unintended outcomes, either positive or negative? 1.2 What were the main reasons that determined whether intended outcomes were or were not achieved, and whether there were positive or negative unintended outcomes? Which reasons were under control of the programs and which were not? Beneficiary satisfaction 3. How satisfied were beneficiaries with the programs? 3.1 What issues were most important to beneficiaries forming their perceptions of the programs? What were the key successes and challenges of the programs? Relevance 4. How relevant were program activities beneficiary targeting, considering the needs of the target population? 4.1 Were beneficiary targeting criteria and processes appropriate, transparent, and properly implemented? 4.2 Were the scale, type, and timing of the program activities appropriate to the needs of the target population? Effectiveness 5. How well were program activities planned and implemented? 5.1. What were the main factors that contributed to whether activities resulted in intended outputs and outcomes? 5.2. What quality standards were defined? How did the Final Performance Evaluation of the Amalima Development Food Security Activity in Zimbabwe Annex B: Evaluation Statement of Work 65 Criteria Main evaluation questions Sub-questions programs develop those standards? Coordination 6. To what extent did the programs coordinate with other food security and humanitarian programming, the host country government, and the donor? Sustainability and Replicability 7. How sustainable are programs’ the outcomes? 7.1. What exit strategies were incorporated into program design? Were such strategies implemented, how were they perceived by the beneficiary population, and what were the strengths and weaknesses of the exit strategies adopted? Cross‐cutting issues 8. How well were gender and environmental considerations integrated into program design and implementation? 8.1. Were they successful in meeting their stated objectives? How? Lessons Learned 9. What lessons can be learned future FFP and USAID Title II in Zimbabwe? III. Endline Study Design and Methodology The endline study will consist of the following data collection activities, described below: A. Representative population‐based household survey B. Qualitative data collection activities A. REPRESENTATIVE POPULATION‐BASED HOUSEHOLD SURVEY The Contractor is expected to take responsibility for the design and execution of all aspects of a representative, population‐based household survey, including sampling plan; questionnaire instrument development; field procedure manuals for enumerators and supervisors; training of enumerators, supervisors, and anthropometrists; piloting of the questionnaire instrument; organization of field work; pre‐testing of the survey rollout; data collection, cleaning, manipulation, and analysis. Sampling Design: Before embarking on designing the sample survey, the Contractor should become familiar with the FANTA Sampling Guide (1997)33 and Addendum (2012).34 The former provides an overview of the recommended design features for Title II baseline and final evaluation surveys. The 2012 Addendum provides important corrections to the guide, which should be followed closely. The 33 Although the FANTA Sampling Guide presents random walk as an acceptable sampling method, it is no longer considered acceptable and will not be accepted as a proposed second stage method. 34 The FANTA Sampling Guide and Addendum can be found at http://www.fantaproject.org/publications/sampling.shtml IMPEL | Implementer-Led Evaluation and Learning 66 Annex B: Evaluation Statement of Work survey population should be limited to those living in geographic areas where program implementation is intended to take place and the sampling frame should reflect this constraint. The Contractor should plan to conduct one survey and two strata, with each awardee area representing one stratum in the survey design. The sample size will be approximately 1600 households. A multi‐stage cluster sampling design should be used. FFP requires that the final evaluation for the program be a performance evaluation (rather than an impact evaluation). This implies that a simple pre‐post design without control groups will be used at both baseline and final evaluation. The Contractor should provide initial indication of the sampling design for the endline survey in a Sampling Plan document in advance of field implementation. This document should include all of the following elements:  The base sample size at both the awardee and overall combined levels. The equation used to drive the calculation of the sample size should also be indicated, where the basis of the calculation should be a test of differences of proportions over two time points. The parameters used in the equation, including the design effect, confidence level, and statistical power assumed should be given. The Contractor should provide a table showing a comparison of sample sizes across “candidate indicators” under consideration for taking on the role of “principal indicator to drive the overall sample size”. The Contractor should carry out sample size calculations separately for each awardee and then sum them to obtain the total sample size for the country survey.  The final choice of principal indicator that will drive the sample size calculation for the entire survey (and associated target group) along with a rationale for the choice of indicator. In terms of associated target group, if stunting is the principal indicator, the target group will be children 0‐59 months, for example.  The number of households to be sampled in order to achieve the desired sample size for the target group (assuming that households may contain more than one or no eligible members from the target group). The Contractor should give an indication of how the base sample size will be adjusted to account for the number of households that need to be visited. See the FANTA 2012 Addendum for more details.35  The number of households to be sampled to account for anticipated household non‐ response. The Contractor should indicate by how much the number of households to be sampled will be pre‐inflated to account for household non‐response.  Geographic or other stratification along with the associated sample allocation scheme (optional). Note that at a minimum, the sample will be stratified by awardee if two awards are made. Additional strata are not required but may be considered. Note that estimates must be produced at both the awardee and combined Title II country program level. Also note, while additional stratification can be considered in the design, estimates do not have to be produced at the level of the lower strata and are likely not feasible given limited survey resources.  The number of stages of sampling to be used. Explanation of how the number of clusters and of households per cluster in the sample will be determined.  Explanation on the source of the information for the sampling frame, e.g., census lists or other national or internationally‐sponsored surveys, such as the Demographic Health Surveys (DHS). The Contractor should indicate how reliable and recent the frame information is.  A Probability Proportionate to Size (PPS) sampling mechanism should be used to randomly select the clusters. The Contractor should use the number of households per cluster as the size 35 Ibid. Final Performance Evaluation of the Amalima Development Food Security Activity in Zimbabwe Annex B: Evaluation Statement of Work 67 measure and include a table of size measures and another table showing the final list of selected clusters along with their probabilities of selection.  Indication that the Contractor will use systematic sampling (or some other probability‐ based sampling technique such as Simple Random Sampling) to select dwellings within clusters. This implies that for the sampled clusters, a list of all households, with household identification and location indicated, must be obtained through either a mapping and listing operation in the cluster prior to interviewing (preferred), or through other existing reliable sources.  The Contractor should collect geographic information system (GIS) information using GPS equipment to locate dwellings during the listing process. GPS units should be used to capture the precise longitude and latitude of each household to be surveyed. These values may then be randomly displaced by a given distance or aggregated up to a higher administrative unit as needed.  Explanation of how households are defined by the Census office in the country in question. In cases where there are multiple households per dwelling, the Contractor should adopt a “take‐ all‐households” approach. The Contractor should specify how polygamous households will be treated as polygamy is prevalent in Zimbabwe.  Indication that the Contractor will adopt a “take‐all‐individuals” approach to select individuals within households from whom to collect data for each target group, particularly for target groups that are more rare in the population, such as children ages 0‐5 months in the case of the exclusive breastfeeding indicator, for example. Questionnaire Instrument: FFP expects the Contractor to develop a questionnaire instrument in English and the local languages, Ndebele and Shona, in which the survey will be conducted, incorporating modules specified in the FFP Standard Indicators Handbook for Baseline and Final Evaluation (referenced above) to respond to the data collection needs of the Title II development food aid programs and USAID. Some of the modules associated with various FFP Indicators, such as HDDS, will require country‐specific adaptation which should be done in consultation with FFP and the Title II awardees.11 Given the limited time and resources for development, it is recommended that the Contractor limit the instrument to a paper and pencil version. The questionnaire should include an informed consent statement for each respondent and commence with a set of questions to establish a household roster. The questions within the questionnaire should be organized by respondent type36 and questions should follow international standard format, e.g. DHS, wherever possible. In general, the Contractor should ensure that questions are written following established questionnaire design principles and that rigorous practices are used to collect, tabulate, and analyze indicator data. These practices should include adding identifiers, such as cluster number, household number, and respondent identification number (line number from household roster) to each page of the questionnaire(s). This helps to ensure that pages can be correctly associated with a given household and respondent if separated, and enable the derivation of household‐level sampling weights and a household non‐ response adjustment to be incorporated into the sampling weights for use in all data analyses. The Contractor should ensure that the questionnaire is piloted and validated in communities not included in the sample frame prior to commencement of data collection. Field Procedure Manuals for Enumerators and Supervisors: FFP expects that the Contractor will develop two field manuals to be used as part of the training materials and serve as reference material 36 Note that a respondent is an individual or set of individual(s) identified as most appropriate to respond to a set of questions on behalf of a specific target group. Such respondents can be the actual sampled members of the target group themselves (e.g., adults providing direct responses on behalf of themselves) or can be individuals not part of the target group providing proxy responses on behalf of sampled individuals in the target group (e.g., caregivers on behalf of young children). IMPEL | Implementer-Led Evaluation and Learning 68 Annex B: Evaluation Statement of Work for staff in the field conducting the survey: one for enumerators and one for supervisors of enumerators. The field manual for enumerators should give recommended best practices for conducting interviews and dealing with specific challenging situations, e.g. households that refuse to participate, and provide a household and individual respondent non‐response follow‐up strategy. It should also contain a detailed explanation of how to properly administer each question in the questionnaire. The field manual for supervisors can contain some of the same material as the field manual for enumerators, The supervisor field procedure manual should also describe the roles and responsibilities of the field staff and outline the chronology of field work, including training, piloting the questionnaire, pre‐testing the survey, data collection, etc. It should also include instructions on mapping and listing clusters, use of GPS equipment, enumerator quality assurance monitoring, questionnaire editing procedures, re‐ interviewing procedures, and procedures for sampling dwellings within clusters, households within dwellings, and individuals within households. Anthropometry Training Materials: The Contractor will provide a short guide and/or other materials to support the training of anthropometrists in the measurements required for the stunting and underweight indicators. This will include instructions on how to take measurements on height and weight for both women and children under five years of age, citing a reference for the methodology that will be used. It will also include a section on methods (event calendars, e.g.) that will be used to ascertain the age of the individuals whose measurements are being taken. Finally, the training materials should include a section on standardization testing of the anthropometrists, which should cover anthropometrical measurement taking and testing of precision and validity of measurements taken by each anthropometrist. Data Treatment and Analysis Plan: The Contractor will prepare a data treatment and analysis plan to address the following elements:  Indication of how and when data will be entered into the database, as well as the software to be used for data entry. Double‐data entry is required;  Data quality checks and edits (data cleaning) planned to ensure logical consistency and coherence across records, as well as an indication of the software to be used;  Sampling weights to be included on the data file. The formulae used to calculate the sampling weights should be included as part of a data dictionary document. Different sampling weights will need to be calculated for separate analysis of each awardee area and of the aggregate Title II program data for the country. Note that a household non‐response adjustment should be made to the sampling weights as part of the final weighting system;  Indicator tabulation plan. Estimates should be produced for each awardee stratum and for the overall level;  Sub‐groups (e.g., age, sex or other geographic breakdowns), if any, for which the Contractor will produce estimators (provided the associated precision levels are sufficient);  Any other planned data analyses. The Contractor should specify all intended bivariate and multivariate analyses here;  Confidence intervals associated with the indicators that will be produced alongside the indicator estimates, and assurance that that these will take into account the design effect associated with the complex sampling design; and  Software to be used for data analysis and for conversion of anthropometric data into Z‐ scores (WHO’s Anthro is recommended but not mandatory).  Upon completion of the survey, location information and associated data collected as part of this award will be delivered to FFP. The Contractor should specify how location data will be adjusted to protect personally identifiable information in accordance with the research protocol Final Performance Evaluation of the Amalima Development Food Security Activity in Zimbabwe Annex B: Evaluation Statement of Work 69 submitted to the Institutional Review Board (IRB). Note that the Contractor will be responsible for adhering to and obtaining all necessary US and host country IRB approvals. The Contractor will ensure that the labeling and architecture of all datasets is consistent to help facilitate meta‐analyses of datasets across Title II development programs and countries at a later date. B. QUALITATIVE DATA COLLECTION FFP will discuss with the Contractor specific details with respect to the requested architecture of the datasets. The meta‐analysis of data is not part of this SOW. B. Qualitative Data Collection. The Contractor will undertake qualitative data collection as part of the endline study. The main objective of the qualitative study is to provide a deeper understanding of the overall food security situation in the program implementation area as perceived by communities and potential beneficiaries. Qualitative information adds depth, richness, and context and will serve to triangulate and interpret information from the household survey. Quantitative and qualitative results should be combined to provide a more complete picture of the overall food security situation. The qualitative study described in this SOW is not expected to replace any in‐depth qualitative assessments or formative research that implementing partners may conduct at the beginning of a program to inform specific aspects of their program design. The Contractor is expected to take responsibility for the design and execution of all aspects of the qualitative study. The Contractor should submit a proposed methodology for the qualitative study that clearly shows how it will complement the quantitative survey and includes the following elements:  Purpose and objectives of the qualitative study;  Research questions the qualitative study will answer;  Conceptual framework presenting the themes that are thought to be relevant to answer the research questions;  Detailed methodology presenting data collection methods to be used, e.g., rapid appraisal/participatory rural appraisal, focus groups, key informant interviews, structured/semi‐ structured interviews, anecdotal evidence, organizational capacity assessments, observations, or seasonal calendars;  Description of the instruments that will be developed and the type of questions to be asked, e.g., key informant interview guides, focus group guides, or organizational capacity assessment questionnaires;  Sampling design and approach for selecting sites, key informants, focus group discussion participants, and/or direct observation sites for the qualitative component;  Timeline and overall approach to data collection, i.e. will it take place prior, in parallel, or subsequent to the household survey, and any potential timeline constraints. (Note that it is highly recommended to conduct the qualitative data collection after the quantitative data collection has been completed and at least partially analyzed to better inform the questions that the qualitative component will set out to answer); and  Plan for data management, coding, and analysis specifying how collected data will be translated, transcribed, coded, and analyzed, the time required for each, and the specific software to be used. IMPEL | Implementer-Led Evaluation and Learning 70 Annex B: Evaluation Statement of Work IV. Endline Study Deliverables and Report Outline A. DELIVERABLES The Contractor is responsible for the following deliverables: Details Deliverables Deadline Pertinent permissions, insurance, and other required permits a) Obtain all necessary permissions for implementing the baseline data collection. b) Adhere to Governments of the U.S. and Zimbabwe national and local formalities. Obtain all required permits related to data collection from human subjects and logistics of survey implementation, including necessary IRB approvals, health and accident insurance, salary and taxes for all enumerators, supervisors and anthropometrists. Pertinent permissions, insurance, and other required permits Evidence submitted and approved prior to FFP granting permission to Contractor to commence pre‐data collection activities, including training of enumerators, supervisors and anthropometrists Inception report and project management tool a) Inception report: specify details for critical tasks, anticipated outputs, date-bound timelines, resource needs, and responsible person(s). Composition of a standard field survey team, including expected tasks and responsibilities of each team member, should also be described. b) Project management tool: an online project management tool should be set up and accessible by FFP and the Contractor. The tool should include a breakdown of key tasks and activities with agreed‐upon deadlines, as well as a Gantt/flow chart of activities over the lifetime of the study. Inception report and project management tool Draft of inception report submitted four weeks after contract issuance. Draft reviewed, revised, finalized, and approved within eight weeks of signing contract. Launch of project management tool four weeks after contract issuance. In‐country endline workshop a) Organize, develop materials for, and conduct a three‐ to four‐day in‐ country workshop in English that brings together the Contractor, Title II awardees, FFP, and the USAID Mission in Zimbabwe. b) Purpose is to glean information on program implementation and country‐ specific ground realities in relation to survey sampling and fieldwork logistics planning; define questions for Two months after the conclusion of the M&E in‐country workshop Final Performance Evaluation of the Amalima Development Food Security Activity in Zimbabwe Annex B: Evaluation Statement of Work 71 Details Deliverables Deadline qualitative component, and vet quantitative instrument and qualitative methodology plan. c) Contractor staff who must attend include those responsible for developing the sampling plan, quantitative instrument, and qualitative methodology, and responsible for overseeing fieldwork. Staff from sub‐contractor firms must also attend the workshop. d) Participants from FFP, USAID, and Title II awardees will fund their attendance at the workshop. However, the Contractor will bear the costs of travel and attendance, in addition to the costs of venue rental, catering, simultaneous translation for the workshop, etc. Quantitative survey questionnaire instrument a) Draft a questionnaire instrument in English adapted to Zimbabwe context that responds to the elements specified in Section III A above. b) Translate the approved questionnaire instrument from English into the local languages, Ndebele and Shona, in which the survey will be administered. If oral (non‐written) languages are needed, a phonetic translation will be required and additional training of enumerators will be necessary. c) Back‐translate the questionnaire from the local languages to English with a second translator to ensure accurate translation. d) Pilot the survey instrument in all the languages in which the survey will take place. (More details under deliverable #9). Final English, corresponding local language, and back‐ translated questionnaires approved by FFP Draft English version of instrument submitted two weeks after conclusion of in‐ country workshop conducted by Contractor (see Deliverable 3). Local language versions of questionnaire instrument to be submitted after English version approved. Date TBD. Draft versions reviewed, revised, finalized, and approved by FFP prior to granting permission to Contractor to commence pre‐data collection activities, including training of IMPEL | Implementer-Led Evaluation and Learning 72 Annex B: Evaluation Statement of Work Details Deliverables Deadline enumerators, supervisors and anthropometrists Qualitative data collection methodology a) Draft a detailed qualitative data collection methodology that responds to the elements specified in Section III B. Qualitative data collection materials approved by FFP Draft materials to be submitted to FFP 3 weeks after conclusion of in￾country workshop conducted by Contract (see Deliverable 3). Draft version of materials reviewed, revised, and approved by FFP prior to granting permission to the Contractor to commence qualitative data collection. Sampling plan a) Draft sampling plan for the household survey that responds to the elements specified in Section III Sampling approved by FFP Draft to be submitted two weeks after in‐ country workshop. List of sampled and replacement villages may follow as a separate appendix but to be submitted and approved prior to FFP granting permission to Contractor to commence pre‐data collection activities, including training of enumerators, supervisors, and anthropometrists. Field procedure manuals for a) enumerators and b) supervisors a. Draft two field procedure manuals for the quantitative population‐based household survey that respond to the elements specified in Section III A. Two field procedure manuals—one for enumerators and another for Drafts of both manuals submitted three weeks after conclusion of in‐ country workshop. Final Performance Evaluation of the Amalima Development Food Security Activity in Zimbabwe Annex B: Evaluation Statement of Work 73 Details Deliverables Deadline supervisors— approved by FFP Data treatment and analysis plan a. Detailed data treatment and analysis plan that responds to the elements specified in Section III A. Data treatment and analysis plan approved by FFP Draft submitted two weeks after conclusion of in‐ country workshop conducted by Contractor (see Deliverable 3). Draft reviewed, revised, finalized and approved prior to FFP granting permission to the Contractor to commence pre‐data collection, including training of enumerators, supervisors and anthropometrists. Training curriculum and pre‐data collection activities a) Develop training materials to address the household survey and the qualitative components, including anthropometry training and standardization testing materials, as outlined in Section III A. b) Pilot test the survey instrument in each of the local languages following enumerator and supervisor training with a small number of non‐sampled households. This will serve as an opportunity to verify that skip patterns, flow, wording, and translation of the questionnaire instrument are working well. Each enumerator team should interview at least two households during the pilot test. c) Pre‐test the survey procedures using the finalized survey instrument in all languages in which the questionnaire will be administered in a small number of households in non‐sampled communities, prior to starting data Training materials approved by FFP Draft training materials submitted at least four weeks prior to commencement of pre‐ data collection activities, including training of enumerators, supervisors and anthropometrists. Draft training materials reviewed, revised, finalized, and approved prior to FFP granting permission to the Contractor to commence pre‐data collection activities, including training of enumerators, supervisors and anthropometrists. IMPEL | Implementer-Led Evaluation and Learning 74 Annex B: Evaluation Statement of Work Details Deliverables Deadline collection. This will serve as an opportunity to verify that enumerators and supervisors have understood their roles and responsibilities as well as all of the survey procedures, prior to “going live”. Each enumerator team should interview at least two households during the pre‐test. d) Develop field movement plan indicating clear intended chronology of interviewing through list of sampled villages, as well as associated assignments of enumerator teams to sampled villages. Sampling frame, data sets and data files  Sampling frame  Raw data set  Edit rules for cleaning data  Data dictionary/codebook  Syntax and output for all analyses and variable transformations  Final data set including cleaned data, sampling weights at each stage, final sampling weights, and all derived indicators Programming specifications for data cleaning to be submitted and approved prior to commencement of programming. Final submission of the data sets must be in the format required by FFP Information Bulletin 11‐02 (August 11, 2011). a) Sampling frame b) Raw data set c) Edit rules d) Data dictionary codebook e) Syntax f) Final data set All files submitted six weeks after completing survey data collection. Briefings a) Weekly phone briefings with FFP and other stakeholders identified by FFP to include a progress report and discussion on any difficulties related to the baseline study. During data collection period, electronic material accompanying briefings should include short field progress reports with number of clusters completed, non‐response rates, re‐interview rates, enumerator drop‐out rates, etc. Template for field progress reports Weekly phone briefings with FFP and other stakeholders. Monthly phone briefing and final in‐country briefings with the USAID Mission in Zimbabwe, FFP, and Title II awardees. Schedule of briefings to be determined jointly by Contractor and FFP. Final Performance Evaluation of the Amalima Development Food Security Activity in Zimbabwe Annex B: Evaluation Statement of Work 75 Details Deliverables Deadline to be determined jointly by FFP and Contractor. b) Monthly phone briefings with the USAID Mission in Zimbabwe and FFP. These briefings should follow the same format as the weekly briefings. c) Formal, final in‐country briefing to the USAID Mission in Zimbabwe , FFP, and Title II awardees to include a PowerPoint presentation and cover the contents of the baseline study report, including findings, conclusions, lessons learned, and recommendations Draft baseline study report a) Draft final report, not to exceed 50pages, excluding appendices and attachments. The report must be presented in English and must include the results of both the quantitative and qualitative components of the study b) Must follow the report outline in this Scope of Work Draft report reviewed by FFP Submitted 14 weeks after completing data collection in the field (and eight weeks after submission of data set as per Deliverable 10). Contractor should allocate sufficient time to allow for several rounds of review by FFP, the USAID Mission in Zimbabwe , and awardees prior to issuing a final report Final baseline a) A revised version of the draft report that incorporates the comments of FFP and the USAID Mission in Zimbabwe b) The final report must be presented in English and follow the reporting format given in Section IV B of this SOW c) FFP expects that the final report will adhere to the USAID Evaluation Policy’s criteria to ensure the quality of the evaluation report (refer to USAID Final report reviewed and approved by FFP and submitted to the DEC Submitted two weeks after receiving comments from FFP and the USAID Mission in Zimbabwe on draft final report (see Deliverable 12) IMPEL | Implementer-Led Evaluation and Learning 76 Annex B: Evaluation Statement of Work Details Deliverables Deadline Evaluation Policy, page 11, Appendix 1) d) The approved final report must be submitted to USAID’s Development Experience Clearinghouse (DEC) and a cover sheet attached indicating the type of evaluative work conducted and design e) The completed baseline study report must include a three‐ to five‐ page summary of the purpose, background of the project, methods, findings, and, if applicable recommendations Lessons learned and best practices document a) Draft a lessons learned and best practices document, not to exceed five pages, related to the Contractor’s overall experience in conducting the baseline study as an independent third‐party to FFP and the Title II awardees. The document should include recommendations for FFP on areas of improvement for future baseline studies and final evaluations. A 5-page lessons learned and best practices document Submitted one week after FFP approval of the final evaluation report B. OUTLINE OF ENDLINE STUDY REPORT The recommended endline study report outline follows: Cover page, Table of Contents, List of Acronyms; Executive Summary should be a clear and concise stand‐alone document that states the most salient findings, conclusions, and recommendations of the study and gives readers the essential contents of the endline report in three to five pages. The Executive Summary helps readers to build a mental framework for organizing and understanding the detailed information within the report; Introduction should include purpose, audience, and synopsis of task; Methodology and Study Design should describe the methodology and design of the household survey and qualitative component, constraints and limitations to the study process and rigor, and issues in carrying out the study; Overview of the Current Food Security Situation should provide a brief overview of the current food security situation in Zimbabwe related to food availability, access, and utilization; current and anticipated programming; and stakeholders. A desk review of information already available will suffice; Final Performance Evaluation of the Amalima Development Food Security Activity in Zimbabwe Annex B: Evaluation Statement of Work 77 Tabular summary of quantitative survey results should present findings of the household survey in table form for all the indicators by awardee and for the aggregate Title II program area in Zimbabwe; Findings should present results from the household quantitative survey and qualitative study. Results from the quantitative survey should be analyzed and discussed, using findings from the qualitative study to complement and help triangulate them. The qualitative study findings should also provide a deeper understanding of the overall food security situation in the program implementation area. Any bivariate and multivariate analysis undertaken should also be included; Conclusions and Recommendations should provide high‐level conclusions from the endline study and recommendations for the design and implementation of future programming in Zimbabwe. Recommendations must be relevant to program and context and include concrete and realistic steps for implementing or applying the recommendation; Issues should provide a list of key technical and/or administrative issues, if any, that the Title II programs for which the baseline study was conducted should consider; and Annexes should document the following and be succinct, pertinent, and readable:  References, including bibliographical documentation;  List of meetings, including key informant interviews and focus group discussions, with number, type, and date of interactions;  Quantitative survey instruments in English and applicable local languages, Ndebele and Shona in Zimbabwe;  Sampling Plan for the quantitative survey;  Qualitative study methodology and instruments developed and used;  Quantitative data sets and qualitative data transcripts in electronic format;  Data dictionary and program files used to process the data in electronic format;  Baseline study SOW; and  Other special documentation identified as necessary or useful V. Contractor Qualifications The selected firm/consortium shall possess the following qualifications:  Legal status recognized to work in the country, enabling the organization to perform the above‐ mentioned tasks;  Demonstrated experience and strong internal capacity in designing, organizing, and managing the implementation of large‐scale population‐based household surveys in developing countries within the past five years;  Demonstrated experience and strong internal capacity in designing, organizing, and conducting qualitative research, data collection, and analysis in developing countries within the past five years;  Demonstrated experience and strong internal capacity in the statistical analysis of complex survey data and in analyzing data from mixed‐method studies;  Good network of experienced enumerators, supervisors, anthropometrists, and data entry clerks in Zimbabwe, or demonstrated ability to effectively recruit skilled enumerators, supervisors, and data entry clerks in developing countries  Experience engaging and managing statistical or evaluation firms and/or institutions in Zimbabwe or other developing countries; and IMPEL | Implementer-Led Evaluation and Learning 78 Annex B: Evaluation Statement of Work  Ability to deliver high‐quality written and oral products. VI. Team Composition and Qualifications For planning purposes, the team for this study will consist of key personnel with defined technical expertise, a mix of consultants that will provide varying technical and subject matter expertise, and support staff. The team should include local consultants with expertise, knowledge, and experience in Zimbabwe. Offerors may propose an alternative personnel configuration to implement the study based on the approach provided in their proposals. The required areas of technical and subject matter expertise represented on the team should reflect the multi‐sectoral nature of Title II food assistance and the expertise required to conduct qualitative research and quantitative population‐based household surveys:  Expertise in food security programming;  Expertise in agriculture;  Expertise in maternal and child health and nutrition;  Expertise in gender integration;  Expertise in qualitative data collection methods and analysis; and  Expertise in the design and execution of population‐based household surveys, and in the analysis of complex survey data. Key Personnel: Endline Study Team Leader: This individual will serve as team leader in a full‐time position for the duration of the study. S/he will be the primary point of contact between USAID and the endline study team and have responsibility for the overall compilation of the final endline study report. The incumbent must meet the following criteria:  At least 10 years of food security programming in senior management positions; Master’s or PhD degree in development studies, management, program evaluation, or other relevant field of study;  Directly managed the design and implementation of at least two food security‐related, large‐ scale, population‐based household surveys with complex designs;  Broad range of subject matter expertise and demonstrated experience in the areas of food security, agriculture development, nutrition, and health;  Excellent organization and writing skills and a demonstrated ability to deliver a quality written product (Evaluation Report and PowerPoint)  Excellent oral communication, presentation, and inter‐personal skills;  Technical and management skills to manage budget resources (dollars and staff) for the evaluation, as well as assist and support the team with field logistics (e.g., coordinating with USAID and/or a government ministry to set up initial appointments for interviews); and  Experience on past Title II baseline surveys or final evaluations would be a plus. Senior Survey Specialist: This individual will be responsible for designing, managing, and coordinating the population‐based household survey and analysis of the survey data. The incumbent must meet the following criteria:  At least eight years of experience designing, managing, leading, and coordinating representative population‐based household surveys in developing countries; Final Performance Evaluation of the Amalima Development Food Security Activity in Zimbabwe Annex B: Evaluation Statement of Work 79  Master’s degree or PhD in statistics, survey methodology, epidemiology or other relevant field of study;  Extensive knowledge of and experience in sample design for complex surveys and complex survey data analysis;  Extensive experience with the design and development of quantitative survey questionnaire instruments; Extensive experience with data management and database organization, including developing data entry programs and supervising data entry, cleaning, and quality control;  Strong working knowledge of SPSS, STATA, SAS or other statistical package; Excellent writing and organization skills and a demonstrated ability to deliver a high‐quality written product ; and  Experience on past Title II baseline surveys or final evaluations would be a plus. Qualitative Research Specialist: This individual will be responsible for designing, managing, and supervising qualitative data collection and analysis. The incumbent must meet the following criteria:  At least eight years of experience designing and implementing qualitative research studies in developing countries;  Experience with a diverse range of qualitative methodologies, such as rapid appraisal/participatory rural appraisal, focus groups, key informant interviews, structured/semi‐ structured interviews, anecdotal evidence, organizational capacity assessments, observations, and seasonal calendars;  Excellent writing and organization skills and a demonstrated ability to deliver a high‐quality written product;  Familiarity with a broad range of subject matter in the areas of food security, agriculture development, nutrition, and health; and  Experience on past Title II baseline surveys or final evaluations would be a plus Field Operation Manager: This individual will be responsible for planning, managing, and supervising the household survey data collection in‐country. The incumbent must meet the following criteria:  Undergraduate degree in one of the social science disciplines;  Eight years of experience supervising large‐scale survey field work in developing countries, preferably involving anthropometric data collection;  Experience hiring, training, and overseeing field supervisors and enumerators; coordinating field logistics, schedules, and equipment; and managing data quality control in the field; and Fluency in relevant national language required. As per the criteria presented above and given the multi‐sectoral approach of Title II programs, the Contractor will be expected to involve sectoral experts in the areas of agriculture, livelihoods, health, and nutrition, as needed. These experts can either be external consultants engaged on a full‐ or part‐ time basis or members of the selected firm with the necessary skills. The required skills of the agriculture and health and nutrition experts are outlined below; however, additional sectoral experts may be needed based on the country context and Title II program activities: Agriculture Expert: This expert will provide technical guidance related to agriculture and agribusiness during the study. The incumbent must meet the following criteria:  At least five years of food security implementation experience in developing countries; Master’s or PhD degree in agriculture‐related field of study;  Strong knowledge of agriculture indicators, agriculture extension, conservation agriculture, input management, post‐harvest handling, livestock management, and agricultural marketing;  Excellent writing and organization skills; IMPEL | Implementer-Led Evaluation and Learning 80 Annex B: Evaluation Statement of Work  Excellent oral communication, presentation, and inter‐personal skills;  Excellent analytical and technical skills; and Strong knowledge of Title II programming, experience on past Title II baseline surveys or final evaluations would be a plus. Health and Nutrition Expert: This expert will provide technical guidance related to maternal and child health and nutrition during the study. The incumbent must meet the following criteria:  At least five years of maternal and child health and nutrition expertise in developing countries;  Master’s or PhD degree in international public health, international nutrition or other relevant field of study;  At least three years of emergency or development food security implementation experience;  Strong knowledge of health and nutrition indicators, supplementary and vulnerable group feeding practices, positive deviance, care group, and community healthcare methodologies;  Excellent writing and organization skills;  Excellent oral communication, presentation, and inter‐personal skills;  Excellent analytical and technical skills; and  Strong knowledge of Title II programming, experience on past Title II baseline surveys or final evaluations would be a plus. Other team members: The Offeror will need to consider and budget accordingly to what extent the team will require junior or mid‐level support (e.g., to assist in collecting, analyzing, and cleaning data, and preparing tabular or graphic materials). As per the USAID Evaluation Policy, all endline study team members will provide a signed statement attesting to a lack of conflict of interest or describing an existing conflict of interest relative to the program for which the endline study is being conducted. VII. Endline Study Management A. LOGISTICS FFP will provide overall direction to the Contractor, identify key documents, and assist in facilitating a work plan. FFP staff in Washington and the USAID Mission in Zimbabwe will assist in arranging meetings with key stakeholders as identified by USAID prior to the initiation of field work. The Contractor is responsible for arranging other meetings as identified during the course of this study and advising FFP prior to each of those meetings. The Contractor is also responsible for arranging vehicle rental and drivers as needed for site visits and field work. The Contractor will be responsible for making hotel arrangements, procuring its own work/office space, computers, internet access, printing, and photocopying. The Contractor will be required to make its own payments. Staff from FFP and the USAID Mission in Zimbabwe will be made available to the team for consultations regarding sampling, geographical targeting, sources, and technical issues before and during the evaluation process. B. SCHEDULE/TIMELINE Offerors must submit a timeline of activities as part of their proposals, which should follow the timeline set forth in Section IV A of this Scope of Work. Final Performance Evaluation of the Amalima Development Food Security Activity in Zimbabwe Annex B: Evaluation Statement of Work 81 C. BUDGET A firm bidding on this activity must, in addition to a technical proposal, submit a Budget in Excel showing the projected Level of Effort (LOE) for each proposed full‐time and/or short‐time member of the Team, including subject matter expertise and administrative (logistical) support. Other costs that should be included are international travel and per diem, in‐country costs for data collection and interviewing, communications, report preparation and reproduction, and other costs as appropriate. A six‐day work week is authorized when working in‐country. Offeror proposals will be evaluated on the merit of the proposed approach including the following criteria:  30% Technical Approach as illustrated in the description of proposed methodology.  25% Timeline reflecting proposed activities, which emphasizes the ability to meet the proposed deadlines.  25% Key personnel and composition of the technical team, including CVs and commitment of availability. FFP will also consider the offeror’s ability to engage and use local firms.  20% Past performance, including a sample document (preferably of a baseline or final evaluation with quantitative and qualitative methodologies) provided as a writing sample to evaluate this criteria. The offeror should also include in the submission a list of references, preferably in USAID, related to the completion of a baseline study or final evaluation for a Title II or food security project. VIII. Intellectual Property USAID shall, solely and exclusively, own all rights in and to any work created in connection with this agreement, including all data, documents, information, copyrights, patents, trademarks, trade secrets or other proprietary rights in and to the work. The Contractor is not allowed to withhold any information related to this agreement, as this will become public information. IMPEL | Implementer-Led Evaluation and Learning 82 Annex C: Training, Data Collection, and Quality Assurance Annex C: Training, Data Collection, and Quality Assurance Training TANGO organized and led enumerator training in preparation for the ENSURE and Amalima endline quantitative survey. The training took place from April 22 to May 4, 2019. It was led by two TANGO consultants with assistance from Jimat Development Consultants. The Jimat team included a Survey Director, Survey Coordinator, and two Population-Based Survey (PBS) Specialists. An independent Anthropometry Specialist led the anthropometry training and an Independent Survey Monitor provided support to the TANGO team and to all supervisors. Table 6 shows the number of different personnel employed in the training and data collection phases, by personnel category. Table 6: Personnel employed for Zimbabwe quantitative survey training and data collection Household survey enumerator training A team of 47 household survey enumerators and nine field team leaders participated in the 11-day training. The training covered: study objectives and sampling methodology, human subject research, interview norms and survey implementation guidance. It also included a thorough review of the household survey instrument, instruction how to conduct household listing, and the use of tablets and data collection through ODK. During the course of the training, enumerators and field team leaders practiced administering the household survey, using both paper and tablet versions in order to familiarize themselves with different scenarios they could encounter in the field. Throughout the course of the training, participants maintained a list of questions and issues to review with TANGO. Listing enumerator training The listers and lister supervisors attended the first two days of the household enumerator training (April 22-23, 2019) for overall orientation. On the third day, the listing group split away for its own training and practice before beginning the listing exercise on April 25. The listing team was comprised of 30 listers and 10 lister supervisors. The listers received training on the listing survey and on developing sketch maps for use by the household survey enumerators. An exercise was developed to encourage listers and household enumerators to develop and interpret sketch maps, using the local venue as an example. This ensured enumerators and listers had a good understanding of how the data collected by the two individual surveys (household and listing) were linked and how enumerators’ work contributed to their peers’ work. The lister supervisors were trained on processing listing surveys, overseeing the listing data collection, and quality control checks. The training reviewed protocol to introduce the project to the local leadership, as the listing teams were the first point of contact between survey teams, households and communities. # listers # lister supervisors # enumerators, HH survey # enumerators, anthropometry # team leaders Training 30 10 47 12 9 Data collection 18 10 46 9 9 Final Performance Evaluation of the Amalima Development Food Security Activity in Zimbabwe Annex C: Training, Data Collection, and Quality Assurance 83 Anthropometry enumerator training A team of 12 anthropometry enumerators also participated in the first two days of the training (April 22- 23, 2019) alongside the household enumerators and listing enumerators. The anthropometry enumerators then participated in separate training from April 24 to May 4, 2019 to focus on the anthropometry survey. Training included sessions on i) measurement procedures for women and children on stunting and underweight indicators; ii) introduction to using tablets and data collection through ODK; and iii) anthropometry quality control measures to be covered with field team leaders. Jimat invited women and children to participate as volunteers for the anthropometry training. Household survey enumerators assisted the anthropometry enumerators by positioning children so that they could be measured correctly. The Anthropometry Specialist instructed enumerators on how to avoid recording errors when measuring women’s height and weight and children’s standing or recumbent height and weight. Supervisor training In addition to the 11-day training, field team leaders participated in a one-day supervisor training which covered roles and responsibilities of supervisors and the fieldwork work plan. The training was led by the TANGO team; participants were the Jimat personnel (Survey Director, Study Coordinator, and PBS Quality Controllers), Independent Survey Monitor, and Anthropometry Specialist. The TANGO team discussed responsibilities for each type of supervisor to ensure role clarity and optimal quality control over the data collection process and data management. This was necessary given the layered approach to supervision that was established for data collection: Jimat team members, independent consultants, and field team leaders each had specific roles to play. The team of 15 field team leaders, responsible for directly managing survey and anthropometry enumerators, were trained on the necessary procedures to follow when arriving at a cluster (EA), including communication with local leadership, the identification of households, and the assigning of households to enumerators. All supervisors were instructed on procedures for data quality control and troubleshooting through the use of control sheets, spot checks, and re-check processes. Field team leaders were instructed on monitoring household survey and anthropometry enumerators’ data collection closely, on verifying questionnaire completeness, and on data management. This included creating backup copies of data, data archiving, and transferring complete and verified questionnaires to the TANGO server. Training location and pre-testing All trainings took place in Harare. During the course of the training, the household survey enumerators, anthropometry enumerators, and field team leaders had the opportunity to role-play data collection measures with volunteer members of the public who Jimat Consultants invited to the training. This was done so they could practice introductions, gather practice survey data and enter it into tablets, and ensure coordination among data collectors. A field pre-test was organized on May 2, near the end of the training. It was conducted in a rural community within the boundaries of the projects but outside the sample, so teams could have the opportunity to gather information in an environment that closely resembled the area where actual data collection would take place. The pre-test allowed the enumerators and field team leaders to practice the procedures to follow when arriving in each EA. Household enumerators were asked to complete one household survey, and anthropometry enumerators were asked to measure at least one child and one woman. Field team leaders supervised each enumerator during a portion of their interview and IMPEL | Implementer-Led Evaluation and Learning 84 Annex C: Training, Data Collection, and Quality Assurance provided feedback on the conduct of the interview. In addition to serving as a practice for the enumerators and a test of the survey tool, the pre-test allowed enumerators to practice coordinating the logistics of household interviews and anthropometric measurements. It also served as a test of the anthropometric equipment, and was helpful to understand the time needed to complete the survey, measurements, and data quality procedures. The last two days of training for household survey and anthropometry enumerators in Harare were reserved for reviewing obstacles faced during the pre-test, reviewing definitions and terms in local languages, and discussing issues that needed further clarity. Issues encountered during the pre-test led TANGO to add an extra day of training (May 4) that was not originally planned. Translation and back-translation Following the baseline survey procedure, the household survey questions were translated and entered into ODK in Shona and Ndebele. The translation and back-translation of the household survey questionnaire were done by three enumerators hired by Jimat: one translated questions from English to Shona and one translated the questions from English to Ndebele. A third translator back-translated the household survey from the local language to English to ensure accuracy. As in the baseline, anthropometry and listing surveys were in English. The translation process was monitored by the TANGO team and closely verified by the Independent Survey Monitor to ensure accuracy. Household survey enumerators spent a total of seven days role-playing in English/Shona/Ndebele with other enumerators and with the invited volunteers. Anthropometry enumerators also practiced in local languages with women and child volunteers throughout their training. Field procedure manuals for enumerators and supervisors TANGO produced a series of manuals to guide and support the teams throughout the data collection process. The manual for field team leaders includes:  Information on household and anthropometry surveys, including explanations for every question and instructions;  Terminology on agriculture, WASH practices, and food security;  Description of the anthropometry survey and measurement that was covered during training;  Instructions for operating tablets, understanding ODK, and uploading data to the TANGO server; and  Quality control sheets for leaders to conduct checks on enumerators’ work. The household survey manual and anthropometry manual focus on detailed explanations of questions from each survey and on working with ODK. The anthropometry manual describes procedures adapted from the DHS biomarker manual for all DHS surveys worldwide. Reinforcing information from the training, it also includes enumerator instructions for cases where a child is suffering from wasting or exhibiting bilateral pitted edema. Survey programming TANGO staff converted the baseline survey questionnaire to an Excel version that was readable by ODK software. This included typing out more than 900 rows in Excel and adding columns for three languages (English, Shona, and Ndebele), with codes for skip patterns and constraints that would allow the survey logic to run appropriately. Prior to the team’s departure for fieldwork, TANGO performed a final check Final Performance Evaluation of the Amalima Development Food Security Activity in Zimbabwe Annex C: Training, Data Collection, and Quality Assurance 85 and the Independent Survey Monitor also did a quality control check to verify the ODK logic in all three languages before finalizing the household survey on May 20. The programming of the listing survey and the anthropometry survey were also done using the questions from the baseline surveys; a similar process was followed for ODK programming. Listing Listing began on April 25 while household and anthropometry survey trainings continued in Harare. Jimat obtained detailed boundary maps for each sampled EA from the Zimbabwe National Statistics Agency, which included household counts from the 2012 census. Lister enumerators used these maps to develop sketch maps, which included the official EA borders and sketches of infrastructure, forests, bridges, and any other natural and physical key points that would help the household and anthropometry teams locate sampled households. The listing team included a mapper and a lister working together to collect listing data and develop the maps. Listing supervisors traveled with the teams, introduced teams to village leaders, and followed all procedures, including quality control checks. Each lister team recorded GPS coordinates at the center of the EA they listed. Each listing team gathered household-identifying information from each dwelling in the EA, including the name of the head of household. The teams worked closely with their supervisors to avoid duplications and missing households. The listing data were uploaded to the TANGO server, where the TANGO team verified them for completeness and accuracy. The Survey Director at TANGO conducted the sampling of households (described in Section 3.1 of main report). The selected households were provided to the Independent Survey Monitor in Harare, who distributed lists of households by EA to field team leaders. The field team leaders used these lists to assign households to individual household survey and anthropometry enumerators. Household survey and anthropometric data collection The household survey enumerators collected data from their assigned households and worked in coordination with the anthropometry enumerators to ensure that the criteria for measuring children and women were applied. In the rare cases where household survey enumerators finished their interview and moved to another household before the anthropometry enumerators arrived (sometimes they were delayed at the previous household because they had to measure multiple individuals), the teams communicated with each other on which children and women that needed to be measured. The field team leader, anthropometry enumerators, and household survey enumerators debriefed at the end of each data collection day to plan the logistics for the next day and allow the leader to perform quality control checks. Quality assurance The field team leaders provided the first level of quality control by implementing spot checks and directly observing enumerators. The Survey Director, Survey Coordinator, PBS Quality Controllers, and the two independent consultants provided quality oversight to the teams in the field. The TANGO team monitored data uploaded to the TANGO secure server and provided feedback to the teams. This process ensured questionnaires were reviewed daily for completeness and accuracy. In the analysis stage, data were cleaned using STATA statistical software; identifying information was removed from the final dataset. IMPEL | Implementer-Led Evaluation and Learning 86 Annex D: Imputing Missing Data Annex D: Imputing Missing Data There are two areas where data needed to be imputed to compute FFP indicators: children’s meal frequency and household monthly expenditures. Each is discussed in turn, below. Children’s meal frequency The ODK program skipped the question in the children’s nutrition module about meal frequency. This information is required to estimate the Minimum Acceptable Diet (MAD) indicator. Analysts therefore used information from the baseline survey to impute meal frequency, compared baseline imputed values to actual values, imputed meal frequency for the endline, and then computed MAD for baseline and endline using the imputed values. The analysis estimated separate Poisson37 regression equations for children who are breastfeeding and children who are not. The dependent variable (y) in the equation is meal frequency. Explanatory (x) variables are the child’s dietary diversity, child’s age in months, milk feeds, whether or not the child had diarrhea, whether the biological mother resides in the same household, HDDS, improved water source, and household food expenditures. Household type, child’s sex, household size and geographic district were included as control variables. Imputed values of meal frequency are equal to the predicted values from each equation. Analysts estimated several sets of equations. The first set used each explanatory variable by itself, plus the control variables. The second combined all the variables that were statistically significant in the first set into one equation (one for breastfed and one for non-breastfed children), plus the control variables. The third included all the explanatory variables, regardless of their statistical significance in the first set of equations. The next step was to compute correlations of predicted values and actual values from the baseline survey. Predicted values of meal frequency from the final equation, with all the variables, had the highest correlation with actual values. Coefficients from that equation were applied to the endline data to impute endline meal frequency. Analysts estimated baseline and endline values of MAD using imputed meal frequency. Household monthly expenditures Most of the data to estimate monthly expenditures were not collected in the Zimbabwe endline survey. Skip patterns in ODK limited responses to questions about utilities. Data are missing for six categories of monthly expenditures:  Vehicle-related expenses  Transport and communications  Health care  Personal care and effects  Household operations  Recreation and entertainment 37 Poisson regression equations are used where the dependent variable is a count, in this case, meals per day. Final Performance Evaluation of the Amalima Development Food Security Activity in Zimbabwe Annex D: Imputing Missing Data 87 Estimation equations to impute endline monthly expenditures use baseline data. The estimation equations are of the form: 𝑙𝑛𝑌𝑖 = 𝛼 + 𝛽𝑋𝑖 + 𝜖𝑖 Where 𝑙𝑛𝑌 is the natural log of per capita daily expenditures over a 30-day period. Log values are appropriate to use when data are highly skewed. In this case, most households report zero or near zero monthly expenditures and a few have high values. X variables are per capita daily food expenditures, per capital daily annual expenditures and the per capita value of each household’s consumption assets. The equations also include household size as a demographic control variable. Imputation methods take account of the high correlations among these monthly expenditures, food expenditures, yearly expenditures, and household consumption assets. Coefficients from the equations were applied to the endline variables to impute monthly consumption expenditures. Imputations were done separately for each of the 10 districts to account for different geographic conditions, such as access to markets, infrastructure, and livelihoods. Coefficients from the estimation equations are presented in Table 7. Table 7: Per capita daily consumption: expenditures over a 30-day period (USD 2014) Province/District code 101 102 103 506 602 603 604 801 803 807 PCD food consumption, 0.13** 0.13** 0.10 0.25*** 0.14 -0.62 0.20*** 0.30*** 0.21** 0.18** PCD other-yearly consumption 0.83*** 0.01** 2.20*** 1.60*** 0.45** 4.54** 2.04*** 0.41*** 0.06** 0.22*** PCD asset value 1.32*** 1.12*** 0.14 0.13*** 1.47*** 0.33 0.28*** 0.50*** 1.59*** 0.17** Household size -0.05 -0.04 -0.02 0.01 -0.04 -0.02 0.09 -0.04 -0.04 -0.10* Constant -1.82** - 1.45*** - 1.86*** - 2.46*** - 1.96*** - 1.80*** - 2.89*** - 1.71*** - 1.97*** -1.04** Observations 376 496 1156 966 760 532 685 496 545 557 * p<0.05, ** p<0.01, *** p<0.001 Imputed values of monthly expenditures were added to food expenditures, annual expenditures, and assets to estimate total per capita daily expenditures. IMPEL | Implementer-Led Evaluation and Learning 88 Annex E: Data Sources: Interviews, Focus Groups, and Asset Observations Annex E: Data Sources: Interviews, Focus Groups, and Asset Observations Table 8: Key informant interviews Location Stakeholder type/title M F Name* Tsholotsho Distr Lead Farmer X VAC (president) X DRR (president) X Paravet X Ministry of Women’s Affairs X Vet Extension Office X Social Welfare Officer X DRR/NRM Distr Officer X Distr Field Coordinator X AGRITEX X Distr Nutritional Officer X Distr WASH Officer X Nutrition/Health field officer X Mangwe Distr Agro-dealer X Social Welfare Officer X Ministry Women’s Affairs X District CEO X AGRITEX (n=2) X X Clinic Nurse X Nutrition/Health field officer X Distr WASH Officer X Bulilima Distr Field Officer (Ag) X Field Officer (VS&L) X SO2 Team Leader X Social Welfare Officer X VET Services X AGRITEX X Distr Nutrition Officer X Field Officer (Health) X Field Officer (WASH) X Gwanda Distr AGRITEX X RDDC X SO1 Officer X VET Extension Supervisor X EMA, Env. Officer X Dip tank secretary X Final Performance Evaluation of the Amalima Development Food Security Activity in Zimbabwe Annex E: Data Sources: Interviews, Focus Groups, and Asset Observations 89 Location Stakeholder type/title M F Name* Agro-dealer 1 X Agro-dealer 2 X Distr Field Coordinator/Amalima X Field Officer 1 (Livestock) X Field Officer 2 (VS&L) X Field Officer 3 (CA) X Distr Nutritional Officer X Clinic Nurse X Bulawayo Input Supplier 1 X Input Supplier 2 X Input Suppliers (n=2) X X Amalima HQ COP X D. Brigham DCOP X N. Ncube Agriculture and Livestock Officer X T. Chivere Environment and Resilience Officer X J. Karuma Social Behavioral Change X M. Dzimba M&E Coordinator X P. Sithole Marketing Facilitator X T. Nyoni Engineer X E. Moffat Village Savings & Lending X M. Siakwenga Sanitation and Hygiene X W. Sibanda Social Behavior Change & Nutrition X M. Maphosa HH Asset Vouchers & Matching Grants X B. Shinda Community Mobilization X S. Kamanga Gender Specialist X D. Sikosana Water Supplies Specialist X L. Nkomo Agriculture & Livelihoods Manager X N. Nyathi Manager, Nutrition & Health X P. Murakwani Nutrition Coordinator X S. Moyo Monitoring & Evaluation (Nutrition) X S. Dube Compliance Officer X C. Sibutha Database Administrator X B. Kgwadi Totals 70 KIIs 42 M 28 M *As per IRB stipulations, the evaluation team did not collect names of any project participants, district￾level field officers or other district-level personnel/staff. Names were only collected for Amalima HQ staff, as indicated. IMPEL | Implementer-Led Evaluation and Learning 90 Annex E: Data Sources: Interviews, Focus Groups, and Asset Observations Table 9: Summary data for focus group discussions conducted District Ward# Village Type of FGD # female participants # male participants Total # participants Tsholotsho 18 Nkunzi VS&L 6 2 8 18 Nkunki VS&L 7 0 7 9 Siyabalandela VS&L 17 2 19 11 Mayeza Livestock (goats) 7 0 7 11 Ngubomlilo VS&L 6 0 7 11 Ngubomlilo VS&L 5 0 5 19 Dikili East Dam 3 3 6 19 Denge VS&L (Ag) 14 2 16 18 Dlula DRR 3 1 4 19 Denge Horticulture 10 0 10 9 Mpanedziba Lead Farmers 10 0 10 18 Nkunzi CHC 7 1 8 9 Mpumelelo Lead Mothers 7 0 7 9 Mpanedziba Care Givers 7 0 7 11 Ngubomlilo CHC 7 1 8 19 Five Male Champ. 0 5 5 18 Nkunzi Lead Mothers 10 0 10 9 Mpumelelo CHC 16 9 25 Mangwe 3 Madabe VS&L, VC (goats) 10 0 10 3 Dukwe DRR 6 4 10 3 Mapholisa VS&L 12 0 12 9 Mambale Agro-dealers (matching grt) 4 0 4 7 Phathisanani DRR 2 7 9 13 Embakwe Dip Tank 0 4 4 1 Kwite Dam AMC 3 2 5 4 Tshitshi Livestock 9 2 11 15 Mabuledi Nutr Garden 10 0 10 3 Mapholisa CHC 6 3 9 9 Mambale CHC 4 6 10 15 Makhubu WASH (CHC) 10 2 12 Bulilima 11 Madlambuzi General￾Benes 11 1 12 5 Matjinge Livestock (goats) 5 0 5 11 Madlambuzi VAC/Paravets 7 4 11 Plumtr ee MoH Dist Hospital Care Givers 6 2 8 4 Nlinwane Food Ration Recipients 5 0 5 21 Ndiweni Voucher Final Performance Evaluation of the Amalima Development Food Security Activity in Zimbabwe Annex E: Data Sources: Interviews, Focus Groups, and Asset Observations 91 District Ward# Village Type of FGD # female participants # male participants Total # participants Recipients 5 0 5 20 Hikwa Care Group 6 2 8 20 Hikwa CHC 8 0 8 4 Tokwana/Lulo Lead Mothers 12 0 12 20 Tjompani CHC (WASH) 8 0 8 Gwanda 13 Garanyemba Trad. Leaders 0 4 4 13 Garanyemba VS&L, Cluster Facilitators 10 0 10 13 Garanyemba DRR 8 3 11 13 Garanyemba VAC 1 3 4 6 Sibona Male Champions 0 6 6 4 Stanmore Lead Mothers 7 0 7 6 Mtshabezi CHC 10 5 15 Totals 21 wards 34 villages 47 FGDs 333 F 86 F 419 total IMPEL | Implementer-Led Evaluation and Learning 92 Annex E: Data Sources: Interviews, Focus Groups, and Asset Observations Table 10: Summary data for asset observations District Ward# Village Asset Tsholotsho 19 Denge Irrigation scheme/garden 19 Denge Solar powered water pump 9 Moyeni Moyeni dam, drinking area for livestock 11 Mayeza Goat herding project 11 Nqubomililo VC thresher machine 19 Dikili East Pateni dam, CFA group dam construction 18 Dlamdam Dlamdam contours 11 Nqubomililo Lead Mother visit observation 3 Nqubomililo Lead Mother training Care Group 3 Nqubomililo Model Home 8 Nkunzi WASH (BVIPs, solar pump, water testing) Mangwe 3 Madhabe Goat project 3 Madhabe Nutritional garden 9 Manbadi Ag supply store (m grant) 13 Embakwe Dip tank 1 Empandemi Dam (new) 15 Mapubedi Nutritional garden (Siyabalandela) 15 Makhubu Model home 4 Tshitshi WASH (BVIPs, solar pump) Bulilima 10 Bango WASH (solar pump, latrines) 21 Ndiweni WASH (latrines) 4 Ndwolani Food ration distribution 11 Madlambuzi Dip tank 11 Madlambuzi Sand abstraction 20 Hikwa Eco-Stove demo Gwanda 7 Simbumbumbu WASH (sand abstraction, pump, latrines) 6 Sibona Nutritional garden 13 Garanyemba Dam 13 Garanyemba Power pump 20 Mkhaliphe Dip tank 20 Mkhaliphe Gully reclamation Totals 20 wards 22 villages N = 34 Final Performance Evaluation of the Amalima Development Food Security Activity in Zimbabwe Annex F: Comparison of Baseline and Endline Data 93 Annex F: Comparison of Baseline and Endline Data Baseline 95% CI Endline 95% CI Difference Sample size 2014 Baseline Lower Upper 2019 Endline Lower Upper Endline - Baseline Sig. Level Baseline Endline FOOD SECURITY INDICATORS Average Household Dietary Diversity Score (HDDS) 5.3 5.2 5.4 5.0 4.7 5.2 -0.3 * 2,374 4201 Prevalence of households with moderate or severe hunger (HHS) 29.3 26.4 32.2 20.1 14.8 25.4 -9.2 ** 2,426 483 Male and female adults 28.9 25.7 32.0 19.5 13.5 25.5 -9.3 ** 1,663 303 Adult female, no adult male 32.0 27.4 36.7 21.8 14.7 28.9 -10.3 * 627 136 Adult male, no adult female 21.9 13.6 30.3 22.6 5.5 39.6 0.6 ns 125 38 Child, no adults NA NA 11 6 Average Coping Strategies Index 33.8 31.2 36.4 25.0 22.3 27.7 -8.8 *** 2,426 483 Food Consumption Score Percentage of households with FCS =< 21 (Poor) 4.0 3.1 4.9 7.8 4.3 11.2 3.7 * 2,475 483 Percentage of households with FCS > 21 and FCS =< 35 (Borderline) 31.4 28.8 34.0 42.1 35.0 49.2 10.7 * 2,475 483 Percentage of households with FCS > 35 (Adequate) 64.6 61.6 67.6 50.1 42.6 57.7 -14.5 ** 2,475 483 WASH INDICATORS Percentage of households using an improved source of drinking water 44.5 39.4 49.6 38.1 23.6 52.7 -6.4 ns 2,452 486 Percentage of households using improved sanitation facilities 40.6 36.7 44.5 54.0 45.2 62.8 13.4 ** 2,452 486 Percentage of households with soap and water at a handwashing station 1.6 0.9 2.2 7.8 5.4 10.3 6.3 *** 2,452 485 Percentage of households practicing correct use of recommended household water treatment technologies 8.6 6.9 10.3 9.5 5.6 13.4 0.9 ns 2,482 486 IMPEL | Implementer-Led Evaluation and Learning 94 Annex F: Comparison of Baseline and Endline Data Baseline 95% CI Endline 95% CI Difference Sample size 2014 Baseline Lower Upper 2019 Endline Lower Upper Endline - Baseline Sig. Level Baseline Endline Percentage of households practicing safe storage of drinking water 49.7 43.9 55.5 98.4 97.1 99.8 48.8 *** 2,481 486 Percentage of households with a handwashing station near a sanitation facility2 2.6 1.4 3.8 12.9 9.0 16.9 10.3 *** 1,168 290 AGRICULTURAL INDICATORS Percentage of farmers who used financial services in the past 12 months 5.4 4.0 6.8 24.5 19.8 29.2 19.1 *** 3,025 597 Male farmers 5.3 3.6 7.1 19.4 13.6 25.2 14.1 *** 1,190 254 Female farmers 5.5 4.0 6.9 28.1 22.9 33.4 22.6 *** 1,835 343 Percentage of farmers who practiced value chain activities promoted by the project in the past 12 months 71.8 67.7 75.9 68.4 63.5 73.3 -3.3 ns 3,025 597 Male farmers 67.8 62.8 72.9 65.6 58.7 72.4 -2.3 ns 1,188 254 Female farmers 74.4 70.4 78.3 70.5 65.5 75.5 -3.8 ns 1,837 343 Percentage of farmers who used at least five sustainable agriculture (crop, livestock, NRM) practices and/or technologies in the past 12 months 56.8 52.6 60.9 67.5 60.9 74.2 10.8 ** 2,999 597 Male farmers 58.8 53.5 64.0 69.1 61.3 77.0 10.4 * 1,180 254 Female farmers 55.4 51.3 59.6 66.4 58.7 74.0 10.9 * 1,819 343 Percentage of farmers who used at least five sustainable crop practices and/or technologies in the past 12 months 28.2 25.1 31.4 50.2 43.1 57.3 22.0 *** 3,021 597 Percentage of farmers who used at least three sustainable livestock practices and/or technologies in the past 12 months 28.2 25.4 31.0 49.6 42.5 56.6 21.4 *** 3,026 597 Percentage of farmers who used at least three sustainable NRM practices in the past 12 months 8.7 6.4 11.1 5.7 2.9 8.5 -3.0 ns 3,024 597 Final Performance Evaluation of the Amalima Development Food Security Activity in Zimbabwe Annex F: Comparison of Baseline and Endline Data 95 Baseline 95% CI Endline 95% CI Difference Sample size 2014 Baseline Lower Upper 2019 Endline Lower Upper Endline - Baseline Sig. Level Baseline Endline Percentage of farmers who used improved storage practices in the past 12 months 15.0 12.8 17.2 15.8 10.8 20.9 0.8 ns 3,000 522 Male farmers 16.5 13.7 19.3 19.5 13.0 26.1 3.0 ns 1,180 227 Female farmers 14.0 11.6 16.4 13.0 8.2 17.9 -1.0 ns 1,820 295 WOMEN'S HEALTH AND NUTRITION INDICATORS Prevalence of underweight women 13.9 11.9 15.8 11.7 8.1 15.4 -2.1 ns 1,430 353 Women’s Dietary Diversity Score (WDDS) 2.8 2.7 2.9 2.8 2.6 2.9 0.0 ns 1,579 326 Average number of antenatal care (ANC) visits by pregnant women3 4.7 4.4 5.0 4.9 4.5 5.2 0.2 ns 419 120 Number of months pregnant at time of first ANC visit Percentage <4 months pregnant 25.6 20.7 30.5 42.6 33.9 51.4 17.0 ** 425 126 Percentage 4-5 months pregnant 43.6 38.9 48.2 40.8 31.1 50.4 -2.8 ns 425 126 Percentage 6-7 months pregnant 23.9 19.2 28.5 12.6 7.7 17.6 -11.2 ** 425 126 Percentage 8 or more months pregnant 3.0 1.0 5.0 3.1 0.4 5.9 0.2 ns 425 126 Percentage with no antenatal care 4.0 1.5 6.4 0.8 -0.8 2.4 -3.2 * 425 126 CHILDREN'S HEALTH AND NUTRITION INDICATORS Prevalence of underweight children under 5 years of age (Total) 14.6 12.6 16.5 6.7 3.5 10.0 -7.8 *** 1,609 292 Male 15.4 12.5 18.3 7.3 4.0 10.7 -8.1 *** 839 143 Female 13.7 11.1 16.2 6.2 1.8 10.6 -7.5 ** 770 149 Prevalence of stunted children under 5 years of age (Total) 31.7 29.2 34.3 24.5 20.7 28.3 -7.2 ** 1,609 292 IMPEL | Implementer-Led Evaluation and Learning 96 Annex F: Comparison of Baseline and Endline Data Baseline 95% CI Endline 95% CI Difference Sample size 2014 Baseline Lower Upper 2019 Endline Lower Upper Endline - Baseline Sig. Level Baseline Endline Male 34.6 31.1 38.1 28.7 22.9 34.4 -5.9 + 839 143 Female 28.6 25.1 32.1 20.7 16.0 25.4 -7.9 ** 770 149 Prevalence of wasted children under 5 years of age (Total) 3.6 2.6 4.6 1.7 0.1 3.4 -1.9 * 1,609 292 Male 3.8 2.4 5.2 2.1 -0.1 4.2 -1.8 ns 839 143 Female 3.4 2.1 4.8 1.4 -0.4 3.3 -2.0 + 770 149 Percentage of children under age 5 with diarrhea in the last two weeks (Total) 15.8 13.4 18.2 10.4 5.5 15.2 -5.5 * 1,881 315 Male 16.5 13.7 19.2 12.3 6.0 18.7 -4.1 ns 986 153 Female 15.1 11.6 18.7 8.6 2.2 14.9 -6.6 + 894 162 Percentage of children under age 5 with diarrhea treated with ORT (Total) 70.3 64.3 76.3 66.9 41.7 92.2 -3.4 ns 294 34 Male 66.8 58.6 74.9 NA 160 19 Female 74.6 67.7 81.5 NA 134 15 Prevalence of exclusive breast-feeding of children under six months of age 44.9 36.8 53.0 75.3 59.2 91.4 30.4 ** 167 32 Male 45.1 35.0 55.1 NA 92 11 Female 44.7 32.0 57.4 NA 75 21 Percentage of children 6-23 months who receive foods from 4 or more groups 12.4 9.0 15.8 11.2 4.5 17.8 -1.2 ns 514 104 Male 12.2 7.8 16.7 15.6 5.9 25.2 3.3 ns 259 58 Final Performance Evaluation of the Amalima Development Food Security Activity in Zimbabwe Annex F: Comparison of Baseline and Endline Data 97 Baseline 95% CI Endline 95% CI Difference Sample size 2014 Baseline Lower Upper 2019 Endline Lower Upper Endline - Baseline Sig. Level Baseline Endline Female 12.6 8.0 17.2 6.0 -2.8 14.8 -6.6 ns 255 46 Prevalence of children 6-23 months of age receiving a minimum acceptable diet (MAD)4 2.6 1.2 4.1 4.8 1.1 8.5 2.2 ns 482 104 Male 2.6 0.7 4.4 7.6 1.1 14.2 5.1 ns 246 58 Female 2.7 0.5 4.8 1.5 -1.4 4.5 -1.1 ns 236 46 GENDER INDICATORS Females Percentage who achieve adequacy in ownership of assets5 89.9 88.2 91.6 78.6 74.2 83.1 -11.3 *** 2,207 434 Percentage who achieve adequacy in decision-making for purchase, sale or ownership of assets 80.8 79.0 82.7 76.0 71.5 80.6 -4.8 + 2,205 427 Percentage who achieve adequacy in decisions on credit 37.3 34.4 40.3 14.6 10.9 18.2 -22.8 *** 2,165 434 Males Percentage who achieve adequacy in ownership of assets 91.6 89.7 93.5 75.7 67.9 83.5 -15.9 *** 1,161 239 Percentage who achieve adequacy in decision-making for purchase, sale or ownership of assets 82.9 79.3 86.4 75.6 67.4 83.9 -7.2 ns 1,159 228 Percentage who achieve adequacy in decisions on credit 30.6 27.2 34.0 7.3 3.6 11.1 -23.3 *** 1,141 239 ns = not significant, + p<0.1, * p<0.05, ** p<0.01, *** p<0.001 NA : Not available, cell has less than 30 observations. 1 At endline, 63 households reported that the day prior to the survey was a holiday. 2 Denominator includes households with access to a sanitation facility. 3 Women ages 15-49 with a live birth in the past two years. 4 Baseline values have been recomputed to adjust for an error in calculations. 5 The meal frequency component of the MAD indicator was imputed. This information was not collected at endline. The table shows imputed values for both baseline and endline. There are fewer observations at baseline for children 6 to 23 months because not all child data could be matched onto the household file. 6 Baseline values were adjusted to remove mechanical agricultural equipment from assets, this information was not collected at endline. IMPEL | Implementer-Led Evaluation and Learning 98 Annex F: Comparison of Baseline and Endline Data 95% CI 95% CI Difference Number of observations 2014 Baseline 2014 Baseline, corrected1 Lower Upper 2019 Endline2 Lower Upper Endline - Baseline, corrected Sig. Level Baseline Baseline, corrected Endline POVERTY INDICATORS Per capita expenditures (USD 2010) 0.45 0.50 0.47 0.52 0.54 0.50 0.58 0.04 + 2,473 2,473 482 Male and Female Adults 0.44 0.49 0.46 0.52 0.57 0.51 0.62 0.07 * 1,694 1,694 303 Adult Female no Adult Male 0.45 0.49 0.45 0.53 0.43 0.34 0.52 -0.06 ns 638 638 135 Adult Male no Adult Female 0.67 0.74 0.62 0.86 0.78 0.68 0.88 0.04 ns 130 130 38 Child No Adults NA NA NA 11 11 6 Per capita expenditures (Zim$ 2019) 12.92 12.20 13.65 14.05 12.99 15.11 1.13 + 2,473 482 Male and Female Adults 12.77 11.98 13.55 14.72 13.27 16.16 1.95 * 1,694 303 Adult Female no Adult Male 12.81 11.80 13.83 11.22 8.78 13.66 -1.59 ns 638 135 Adult Male no Adult Female 19.33 16.20 22.46 20.35 17.68 23.02 1.02 ns 130 38 Child No Adults NA NA NA 11 6 Percentage below the Total Per Capita Poverty Datum Line (TPCPDL), Zim$ 20193 44.0 40.7 47.3 52.1 46.5 57.8 8.14 * 2,473 482 Male and Female Adults 43.6 40.0 47.3 51.3 44.1 58.6 7.71 + 1,694 303 Adult Female no Adult Male 46.1 41.3 50.9 57.9 47.0 68.9 11.81 + 638 135 Adult Male no Adult Female 34.0 22.1 45.9 36.3 23.0 49.6 2.32 ns 130 38 Child No Adults NA NA NA 11 6 Final Performance Evaluation of the Amalima Development Food Security Activity in Zimbabwe Annex F: Comparison of Baseline and Endline Data 99 95% CI 95% CI Difference Number of observations 2014 Baseline 2014 Baseline, corrected1 Lower Upper 2019 Endline2 Lower Upper Endline - Baseline, corrected Sig. Level Baseline Baseline, corrected Endline Mean depth of poverty (using the TPCPDL), Zim$ 2019 15.2 13.6 16.9 20.0 16.7 23.3 4.73 * 2,473 484 Male and Female Adults 15.4 13.5 17.3 19.8 16.3 23.3 4.39 * 1,694 303 Adult Female no Adult Male 15.0 12.9 17.2 22.0 16.0 27.9 6.93 * 638 136 Adult Male no Adult Female 11.2 5.9 16.5 13.2 5.8 20.7 2.04 ns 130 39 Child No Adults NA NA NA 11 6 Per capita expenditures (USD 2014) 1.20 1.13 1.27 1.30 1.20 1.40 0.10 + 2,473 482 Male and Female Adults 1.18 1.11 1.26 1.36 1.23 1.50 0.18 * 1,694 303 Adult Female no Adult Male 1.19 1.09 1.28 1.04 0.81 1.27 -0.15 ns 638 135 Adult Male no Adult Female 1.79 1.50 2.08 1.89 1.64 2.13 0.09 ns 130 38 Child No Adults NA NA NA 11 6 Percentage below the Total Per Capita Poverty Datum Line, USD2014 (TPCPDL)4 98.2 97.1 96.3 97.9 89.9 88.1 91.7 -7.2 *** 2,473 2,473 482 Male and Female Adults 98.3 97.3 96.3 98.3 88.4 85.5 91.3 -8.9 *** 1,694 1,694 303 Adult Female no Adult Male 98.7 97.6 96.1 99.1 95.4 90.7 100.1 -2.2 ns 638 638 135 Adult Male no Adult Female 88.6 85.8 79.7 92.0 82.3 74.3 90.2 -3.6 ns 130 130 38 Child No Adults NA NA NA 11 11 6 Mean depth of poverty (using the TPCPDL) 68.5 65.8 64.2 67.4 65.3 62.9 67.7 -0.5 ns 2,473 2,473 484 IMPEL | Implementer-Led Evaluation and Learning 100 Annex F: Comparison of Baseline and Endline Data 95% CI 95% CI Difference Number of observations 2014 Baseline 2014 Baseline, corrected1 Lower Upper 2019 Endline2 Lower Upper Endline - Baseline, corrected Sig. Level Baseline Baseline, corrected Endline Male and Female Adults 68.9 66.2 64.6 67.9 64.3 61.3 67.3 -1.9 ns 1,694 1,694 303 Adult Female no Adult Male 68.1 65.5 63.0 68.0 70.3 64.9 75.6 4.8 ns 638 638 136 Adult Male no Adult Female 55.5 53.0 46.7 59.3 50.5 44.1 56.9 -2.5 ns 130 130 39 Child No Adults NA NA NA 11 11 6 ns = not significant, + p<0.1,* p<0.05, ** p<0.01, *** p<0.001 NA : Not available, cell has less than 30 observations 1 Corrections to baseline: Monthly and annual expenditures, missing recoded to zero which allowed for a more accurate sum. 2 Endline monthly expenditures were imputed. Inflation adjusted price per kilogram from the baseline dataset were applied to some baseline food expenditures. 3 Based on Zimbabwe's Total Per Capita Poverty Datum Line (TPCPDL) 2019 in Zimbabwe dollars 4 Based on Zimbabwe's Total Per Capita Poverty Datum Line (TPCPDL) 2014 US dollars Final Performance Evaluation of the Amalima Development Food Security Activity in Zimbabwe Annex G: Comparison of Baseline and Endline Indicators by Project Participation Status (Amalima) 101 Annex G: Comparison of Baseline and Endline Indicators by Project Participation Status (Amalima) Participants 95% CI Non-participants 95% CI Sample size Participants Lower Upper Non￾partici￾pants Lower Upper Differ￾ence Sig. Level Partici￾pants Non￾partici￾pants FOOD SECURITY INDICATORS Average Household Dietary Diversity Score (HDDS)1 5.0 4.9 5.2 4.8 4.4 5.2 -0.2 ns 238 182 Prevalence of households with moderate or severe hunger (HHS) 18.2 12.3 24.1 22.4 14.5 30.3 4.3 ns 272 211 Average Coping Strategies Index 26.6 22.6 30.6 23.1 20.3 25.8 -3.5 * 272 211 Borderline or poor Food Consumption Score (FCS) 51.0 42.0 60.0 48.5 39.9 57.0 -2.5 ns 272 211 POVERTY INDICATORS2 Per capita expenditures (USD 2010) 0.52 0.45 0.58 0.58 0.51 0.64 0.06 ns 272 210 Per capita expenditures (USD 2014) 1.25 1.09 1.41 1.39 1.23 1.55 0.15 ns 272 210 Percentage below the Total Per Capita Poverty Datum Line (TPCPDL) USD 20143 90.4 87.5 93.3 89.1 85.0 93.2 -1.31 ns 272 210 Mean depth of poverty (using the TPCPDL USD 2014) 66.7 63.1 70.2 63.2 60.0 66.5 -3.44 ns 272 210 Per capita expenditures (Zim$ 2019) 13.44 11.73 15.16 15.03 13.32 16.75 1.59 ns 272 210 Percentage below the Total Per Capita Poverty Datum Line (TPCPDL) Zim$ 20194 54.3 45.5 63.0 48.6 42.0 55.3 -5.63 ns 272 210 Mean depth of poverty (using the TPCPDL Zim$ 2019) 21.8 17.5 26.1 17.1 13.7 20.5 -4.64 * 272 210 WASH INDICATORS Percentage of households using an improved source of drinking water 40.5 26.2 54.8 35.3 22.3 48.3 -5.2 ns 273 213 Percentage of households using improved sanitation facilities 56.0 46.9 65.0 51.6 44.3 58.9 -4.4 ns 273 213 IMPEL | Implementer-Led Evaluation and Learning 102 Annex G: Comparison of Baseline and Endline Indicators by Project Participation Status (Amalima) Participants 95% CI Non-participants 95% CI Sample size Participants Lower Upper Non￾partici￾pants Lower Upper Differ￾ence Sig. Level Partici￾pants Non￾partici￾pants Percentage of households with soap and water at a handwashing station 6.7 3.0 10.4 9.1 5.0 13.2 2.4 ns 272 213 Percentage of households practicing correct use of recommended household water treatment technologies 12.8 6.9 18.6 5.6 2.4 8.7 -7.2 + 273 213 Percentage of households practicing safe storage of drinking water 98.7 97.1 100.3 98.1 96.1 100.1 -0.6 ns 273 213 Percentage of households with a handwashing station near a sanitation facility5 12.1 5.8 18.4 14.0 8.0 20.0 1.9 ns 167 123 AGRICULTURAL INDICATORS Percentage of farmers who used financial services in the past 12 months 29.8 23.9 35.6 17.5 12.5 22.4 -12.3 ** 349 248 Male farmers 24.4 15.9 32.9 14.3 10.0 18.6 -10.1 * 133 121 Female farmers 32.9 26.5 39.2 20.4 11.6 29.1 -12.5 * 216 127 Percentage of farmers who practiced value chain activities promoted by the project in the past 12 months 74.0 66.8 81.3 61.0 54.5 67.4 -13.1 ** 349 248 Male farmers 71.5 62.8 80.2 59.5 48.4 70.7 -11.9 + 133 121 Female farmers 75.6 67.0 84.1 62.3 55.0 69.6 -13.3 * 216 127 Percentage of farmers who used at least five sustainable agriculture (crop, livestock, NRM) practices and/or technologies in the past 12 months 74.7 68.0 81.5 57.9 48.6 67.2 -16.8 ** 349 248 Male farmers 77.4 70.2 84.6 60.7 49.4 72.1 -16.7 * 133 121 Female farmers 73.2 63.8 82.5 55.2 44.7 65.8 -17.9 ** 216 127 Final Performance Evaluation of the Amalima Development Food Security Activity in Zimbabwe Annex G: Comparison of Baseline and Endline Indicators by Project Participation Status (Amalima) 103 Participants 95% CI Non-participants 95% CI Sample size Participants Lower Upper Non￾partici￾pants Lower Upper Differ￾ence Sig. Level Partici￾pants Non￾partici￾pants Percentage of farmers who used at least five sustainable crop practices and/or technologies in the past 12 months 56.3 49.5 63.2 42.1 33.2 51.0 -14.2 * 349 248 Percentage of farmers who used at least three sustainable livestock practices and/or technologies in the past 12 months 53.0 48.2 57.7 45.0 36.3 53.8 -7.9 + 349 248 Percentage of farmers who used at least three sustainable NRM practices in the past 12 months 7.1 3.3 10.9 3.8 0.9 6.8 -3.2 ns 349 248 Percentage of farmers who used improved storage practices in the past 12 months 17.4 11.5 23.4 13.5 6.9 20.1 -3.9 ns 310 212 Male farmers 20.8 12.1 29.6 18.1 9.1 27.1 -2.8 ns 123 104 Female farmers 15.3 8.9 21.8 9.1 2.5 15.8 -6.2 + 187 108 WOMEN'S HEALTH AND NUTRITION INDICATORS Prevalence of underweight women 12.1 7.1 17.1 11.1 5.2 17.0 -1.0 ns 229 124 Women’s Dietary Diversity Score (WDDS) 2.7 2.5 2.8 2.9 2.7 3.2 0.3 * 218 108 Average number of antenatal care visits by pregnant women6 4.9 4.4 5.3 4.8 4.3 5.4 0.0 ns 90 30 Number of months pregnant at time of first ANC visit Percentage <4 months pregnant 41.9 30.9 52.8 45.0 27.6 62.3 3.1 ns 95 31 Percentage 4-5 months pregnant 41.9 29.1 54.7 39.4 25.1 53.7 -2.5 ns 95 31 IMPEL | Implementer-Led Evaluation and Learning 104 Annex G: Comparison of Baseline and Endline Indicators by Project Participation Status (Amalima) Participants 95% CI Non-participants 95% CI Sample size Participants Lower Upper Non￾partici￾pants Lower Upper Differ￾ence Sig. Level Partici￾pants Non￾partici￾pants Percentage 6-7 months pregnant 13.0 7.5 18.4 9.8 2.5 17.1 -3.2 ns 95 31 Percentage 8 or more months pregnant 2.1 -1.0 5.2 5.8 -2.3 13.9 3.7 ns 95 31 Percentage with no antenatal care 1.2 -1.4 3.8 0.0 0.0 0.0 -1.2 ns 95 31 CHILDREN'S HEALTH AND NUTRITION INDICATORS Prevalence of underweight children under 5 years of age (Total) 7.8 4.8 10.7 4.0 0.2 7.8 -3.8 * 217 75 Male 9.5 6.3 12.7 1.8 -2.2 5.9 -7.7 * 103 40 Female 6.2 1.6 10.8 6.2 -1.2 13.7 0.0 ns 114 35 Prevalence of stunted children under 5 years of age (Total) 26.3 21.5 31.1 19.7 12.4 26.9 -6.6 ns 217 75 Male 28.2 23.1 33.3 29.9 15.1 44.6 1.6 ns 103 40 Female 24.6 17.3 31.9 9.2 -1.7 20.0 -15.4 * 114 35 Prevalence of wasted children under 5 years of age (Total) 2.0 0.0 4.1 0.9 -1.2 3.1 -1.1 ns 217 75 Male 2.1 -0.2 4.5 1.8 -2.2 5.9 -0.3 ns 103 40 Female 1.9 -0.9 4.7 0.0 0.0 0.0 -1.9 ns 114 35 Percentage of children under age 5 with diarrhea in the last two weeks (Total) 8.8 3.8 13.8 14.3 5.1 23.4 5.4 ns 229 86 Male 10.5 3.5 17.4 17.0 8.1 25.9 6.5 ns 110 43 Female 7.3 1.3 13.4 11.7 -1.9 25.3 4.4 ns 119 43 Final Performance Evaluation of the Amalima Development Food Security Activity in Zimbabwe Annex G: Comparison of Baseline and Endline Indicators by Project Participation Status (Amalima) 105 Participants 95% CI Non-participants 95% CI Sample size Participants Lower Upper Non￾partici￾pants Lower Upper Differ￾ence Sig. Level Partici￾pants Non￾partici￾pants Percentage of children under age 5 with diarrhea treated with ORT (Total) NA NA 20 14 Male NA NA 11 8 Female NA NA 9 6 Prevalence of exclusive breast-feeding of children under six months of age 85.2 74.7 95.8 NA 24 8 Male 88.4 62.4 114.4 NA 9 2 Female 83.4 68.4 98.5 NA 15 6 Percentage of children 6-23 months who receive foods from 4 or more groups 11.2 3.2 19.1 11.2 0.6 21.7 0.0 ns 76 28 Male 15.2 7.2 23.2 NA 38 20 Female 7.4 -5.4 20.1 NA 38 8 Prevalence of children 6-23 months of age receiving a minimum acceptable diet (MAD)7 7.1 2.1 12.1 0.0 0.0 0.0 -7.1 ** 76 28 Male 12.6 3.3 22.0 NA 38 20 Female 1.9 -2.1 5.9 NA 38 8 GENDER INDICATORS Females Percentage who achieve adequacy in ownership of assets8 82.4 75.3 89.5 73.5 65.2 81.7 -8.9 ns 256 178 Percentage who achieve adequacy in decision-making for purchase, sale or ownership of assets 80.1 74.4 85.8 70.5 63.0 77.9 -9.6 + 252 175 IMPEL | Implementer-Led Evaluation and Learning 106 Annex G: Comparison of Baseline and Endline Indicators by Project Participation Status (Amalima) Participants 95% CI Non-participants 95% CI Sample size Participants Lower Upper Non￾partici￾pants Lower Upper Differ￾ence Sig. Level Partici￾pants Non￾partici￾pants Percentage who achieve adequacy in decisions on credit 19.3 14.1 24.5 8.0 5.5 10.5 -11.3 ** 256 178 Males Percentage who achieve adequacy in ownership of assets8 79.7 68.8 90.6 72.1 63.0 81.2 -7.6 ns 119 120 Percentage who achieve adequacy in decision-making for purchase, sale or ownership of assets 79.6 70.2 89.1 72.0 62.4 81.6 -7.6 ns 114 114 Percentage who achieve adequacy in decisions on credit 9.9 4.8 15.0 5.0 1.8 8.3 -4.8 ns 119 120 ns = not significant, † p<0.1, * p<0.05, ** p<0.01, *** p<0.001 NA : Not available, cell has less than 30 observations. 1 The denominator for HDDS is smaller than for other food security indicators because 63 households reported that the day prior to the survey was a holiday. 2 Monthly expenditures were imputed. Inflation adjusted price per kilogram from the baseline dataset were applied to some endline food expenditures. This method was used where endline prices were implausible and sample size was too small to use median or mean values. 3 Based on Zimbabwe's Total Per Capita Poverty Datum Line (TPCPDL), denominated in 2014 USD 4 Based on Zimbabwe's Total Per Capita Poverty Datum Line (TPCPDL), denominated in 2019 Zimbabwe dollars 5 Denominator includes households with access to a sanitation facility. 6 Women ages 15-49 with a live birth in the past 2 years. 7 The meal frequency component of the MAD indicator was imputed. This information was not collected at endline. 8 Values do not include mechanical agricultural equipment as an asset, this information was not collected at endline. Final Performance Evaluation of the Amalima Development Food Security Activity in Zimbabwe Annex H: Supplemental Tables 107 Annex H: Supplemental Tables Figure 6: Percentage of farmers using sustainable agriculture (crop, livestock, NRM) practices in the last 12 months at baseline and endline (Amalima) Figure 7: Livestock practices in the past 12 months (Amalima) 0.00% 10.00% 20.00% 30.00% 40.00% 50.00% 60.00% 70.00% Improved animal shelters Vaccinations Deworming Castration Dehorning Home made animal feeds made of locally available… Animal feed supplied by stock feed manufacturer Artificial Insemination Pen feeding Fodder production and/or veld reinforcement with… Used the service of community animal health… No livestock practices The percentage of farmers using sustainable agriculture and livestock practices increased from baseline (2014) to endline (2019) Percentage of farmers who in the past 12 months used… ns = not significant, + p<0.1, * p<0.05, ** p<0.01, *** p<0.001. IMPEL | Implementer-Led Evaluation and Learning 108 Annex H: Supplemental Tables Figure 8: Cropping practices in the past 12 months (Amalima) Figure 9: Percentage of farmers who practiced value chain activities promoted by the project in the past 12 months (Amalima) 0.00% 10.00% 20.00% 30.00% 40.00% 50.00% 60.00% 70.00% 80.00% Micro-dosing Manure Compost Planting basins Mulching Weed Control Dry planting Ripping into residues Clean ripping Tied ridges Pot-holing Crop rotations Intercropping Integrated pest menagement (IPM) Early planting and planting with first rains Use of improved crop varieties Dead level contours Rigging into separate lines The percentage of farmers who practiced value chain activities promoted by the project did not change from baseline (2014) to endline (2019) ns = not significant, + p<0.1, * p<0.05, ** p<0.01, *** p<0.001. Final Performance Evaluation of the Amalima Development Food Security Activity in Zimbabwe Annex H: Supplemental Tables 109 Figure 10: Gender indicators at baseline and endline for women’s adequacy of ownership of assets, adequacy of decision-making about assets, and adequacy of decisions on credit (Amalima) Figure 11: Gender indicators at baseline and endline for men’s adequacy of ownership of assets, adequacy of decision-making about assets, and adequacy of decisions on credit (Amalima) The percentage of women who achieved adequacy in ownership of assets and for decisions on credit decreased from baseline (2014) to endline (2019) ns = not significant, + p<0.1, * p<0.05, ** p<0.01, *** p<0.001. The percentage of men who achieved adequacy in ownership of assets and for decisions on credit decreased from baseline (2014) to endline (2019) ns = not significant, + p<0.1, * p<0.05, ** p<0.01, *** p<0.001. IMPEL | Implementer-Led Evaluation and Learning 110 Annex H: Supplemental Tables Figure 12: Percentage of farmers who used financial services in the past 12 months (Amalima) The percentage of farmers who used financial services in the past 12 months increased from baseline (2014) to endline (2014) ns = not significant, + p<0.1, * p<0.05, ** p<0.01, *** p<0.001. Figure 13: Average Household Dietary Diversity Score (HDDS) (Amalima) HDDS declined from baseline (2014) to endline (2019) ns = not significant, + p<0.1, * p<0.05, ** p<0.01, *** p<0.001. Figure 14: Prevalence of households with moderate or severe hunger (HHS) (%) (Amalima) The prevalence of moderate or severe hunger decreased from baseline (2014) to endline (2019) ns = not significant, + p<0.1, * p<0.05, ** p<0.01, *** p<0.001. Final Performance Evaluation of the Amalima Development Food Security Activity in Zimbabwe Annex H: Supplemental Tables 111 Figure 15: Percentage of households with poor, borderline, or adequate Food Consumption Scores (FCS) at baseline (2014) and endline (2019) (Amalima) Table 11. Baseline-endline comparison of household sanitation and drinking water, sanitation facility, source and treatment of drinking water (Amalima) Amalima 2014 Baseline 2019 Endline Sig. Improved, not shared sanitation facility Flush toilet 0.3 0.6 ns Ventilated improved pit latrine 26.1 36.0 ** Pit latrine with slab 14.6 17.5 ns n 2,483 486 Improved, shared sanitation facility Flush toilet 0.0 0.3 ns Ventilated improved pit latrine 3.3 3.7 ns Pit latrine with slab 2.2 2.2 ns n 2,483 486 Non-improved sanitation facility Open pit 3.6 0.9 *** No facility 49.7 39.0 ** n 2,483 486 Improved source of drinking water1 Piped water into dwelling 0.1 0.2 ns Piped water into yard 0.4 0.4 ns Piped tap/standpipe 0.9 1.4 ns Tube well or borehole 62.2 59.0 ns Protected well 8.3 6.2 ns Protected spring 0.4 0.5 ns Rainwater 0.3 0.2 ns n 2,482 485 Unimproved source of drinking water Surface water 19.9 24.7 ns Unprotected well 5.7 4.9 ns Unprotected spring 1.4 0.8 ns Cart with small tank 0.1 0.0 ns The percentage of households with poor and borderline FCS increased from baseline (2014) to endline (2019) IMPEL | Implementer-Led Evaluation and Learning 112 Annex H: Supplemental Tables Amalima 2014 Baseline 2019 Endline Sig. Tanker truck 0.0 0.4 ns Bottled water 0.0 0.0 ns Other 0.2 1.2 ns n 2,482 485 Water availability Water is generally available from source 67.9 68.5 ns Water not available a day or more - last 2 weeks 19.9 22.4 ns n 2,482 485 Water treatment prior to drinking2 Boil 7.9 9.3 ns Filter 0.8 0.6 ns Bleach/chlorine added 3.6 1.9 ns Stand and settle 0.6 0.7 ns No treatment 87.5 87.8 ns n 1,381 288 ns not significant, + p<0.1,* p<0.05, ** p<0.01, *** p<0.001 1 To be counted as “improved,” a household needs access to one of the sources on the improved list AND water needs to be generally available without any interruptions of a day or more over the last two weeks. 2 Includes only HH using unimproved water source(s). Totals sum to more than 100 because of multiple responses. Figure 16: Child feeding practices and dietary diversity (Amalima) The prevalence of exclusive breastfeeding for children under 6 months old increased from baseline (2014) to endline (2019) ns = not significant, + p<0.1, * p<0.05, ** p<0.01, *** p<0.001. *The meal frequency component of the MAD indicator was imputed. This information was not collected at endline. Indicator includes imputed values for both baseline and endline Final Performance Evaluation of the Amalima Development Food Security Activity in Zimbabwe Annex I: Analysis of “Adequacy” Indicators 113 Annex I: Analysis of “Adequacy” Indicators The household survey included a series of questions from the resources domain of the Women's Empowerment in Agriculture Index (WEAI) (Alkire et al., 2013). The questions provide information for three values reported in the baseline and endline reports: adequacy of asset ownership, adequacy of decision-making about asset disposition, and adequacy of decision-making about use of credit. Males and females who self-identified as decision-makers in their households were asked the following questions: Asset ownership: Who owns most of [item]? (Agricultural land, livestock, farm equipment, business equipment, house, large and small durables, cell phone, transport) Disposition of assets: Who would you say can decide whether to sell, give away, rent or mortgage [item]? Use of credit (cash or in-kind): Who made the decision to borrow, what to do with money, item borrowed from [source]? (NGO, informal lender, formal lender [bank], friends or relatives, savings or credit group) Response codes for all three:  Self  Partner/spouse  Self and partner/spouse jointly  Other household members  Self and other household member(s)  Partner/spouse and other household member(s)  Someone (or group of people) outside the household  Self and other outside people  Partner/spouse and other outside people  Self, partner/spouse and other outside people  None of these items Based on the response codes, males and females were categorized as achieving or not achieving “adequacy” or not in each of the three indicators. Adequacy is equal to 100 for response codes 1, 3, 5, 8 or 10 (which all include “self”). Adequacy is equal to 0 for other codes, or if the household does not own assets. The analysis made statistical comparisons between baseline and endline, but not between men and women. We did, however, conduct some additional analysis of decision-making. The original indicator is from the WEAI. TANGO modified and recomputed the indicators in two ways. The first was to estimate whether joint decision-making in the three measured values (as defined above) increased from baseline to endline. This modification changes the numerator used in the WEAI-based indicator: joint decision-making is defined as self with partner/spouse and uses only response codes 3 and 10. The analysis showed that for both men and women, joint decision-making about credit significantly decreased (worsened) from baseline to endline, dropping from 11.8 to 4.6 percent for men (p<0.01) and from 8.7 to 4.1 percent for women (p<0.01). There were no significant changes for men or women between baseline and endline in asset ownership or decision-making about assets using the criteria for joint decision-making. IMPEL | Implementer-Led Evaluation and Learning 114 Annex I: Analysis of “Adequacy” Indicators The second modification focused on decision-making regarding the use of credit. This analysis included only households that had borrowed either cash or in-kind. Households without any debt were omitted instead of being coded equal to 0 or inadequate. (Borrowing was much lower at endline when 10.4 percent of households reported borrowing, compared to 40.5 percent at baseline.) The results are shown in Table 12 in Annex H. There were no statistically significant differences for females in either adequacy (using the indicator definition) or joint decision-making (using codes 3 and 10). Differences might have been statistically significant in a larger sample. For males, sample sizes were too small to test for differences. Final Performance Evaluation of the Amalima Development Food Security Activity in Zimbabwe Annex I: Analysis of “Adequacy” Indicators 115 Table 12: Joint decision-making on credit, households that reported cash or in-kind borrowing (Amalima) Baseline 95% CI Endline 95% CI Difference Sample size 2014 Baseline Lower Upper 2019 Endline Lower Upper Endline - Baseline Sig. Level Baseline Endline Joint ownership of assets and joint decision-making on assets and credit (n = all households) Females Percentage who jointly own assets 29.5 27.0 31.9 31.6 25.4 37.9 2.2 ns 2,188 433 Percentage making decisions jointly for purchase, sale or ownership of assets 29.3 26.5 32.2 31.7 25.5 37.9 2.4 ns 2,205 434 Percentage who make decisions jointly on credit 8.7 7.0 10.4 4.1 1.7 6.5 -4.6 ** 2,165 434 Males Percentage who jointly own assets 35.4 31.7 39.1 43.9 34.3 53.4 8.4 ns 1,152 239 Percentage making decisions jointly for purchase, sale or ownership of assets 37.0 32.9 41.2 42.4 32.9 51.9 5.4 ns 1,159 239 Percentage who make decisions jointly on credit 11.8 9.1 14.5 4.6 1.3 7.9 -7.2 ** 1,141 239 Joint decision-making on credit (n = only households reporting borrowing cash or in-kind) Females Percentage who achieve adequacy in decision-making in decisions on credit (self or joint) 88.5 86.2 90.8 91.4 85.5 97.4 2.9 ns 908 74 Percentage making decisions jointly on credit 20.6 16.9 24.4 25.9 12.3 39.5 5.3 ns 908 74 Males Percentage who achieve adequacy in decision-making in decisions on credit (self or joint) 83.2 79.0 87.3 NA 419 27 Percentage making decisions jointly on credit 32.0 25.5 38.4 NA 419 27 ns = not significant, † p<0.1,* p<0.05, ** p<0.01, *** p<0.001 NA : Not available, cell has less than 30 observations. IMPEL | Implementer-Led Evaluation and Learning 116 Annex J: Multiple Regression Analysis Annex J: Multiple Regression Analysis Multiple regression analysis was undertaken to further explore the underlying factors associated with changes in several of the key project outcome and impact variables. The specific variables that were examined in this analysis are:  Farmers’ use of financial services  Farmers’ adoption of at least 5 sustainable agricultural practices  Households with adequate food consumption (FCS)  Underweight of CU5  Stunting of CU5 The regression analysis measured the contribution of a number of variables to explain variation in these outcome and impact variables. General categories of explanatory variables were applied in all the regression analyses:  Survey round: a dummy variable for survey round (0=baseline, 1 = endline) was included to measure the changes in the dependent variables independent of any of the other explanatory variables in the model;  Program participation: this variable was included to measure the extent to which changes in the dependent variables are associated with the respondents’ participation in project-supported activities;  Gender variables that measure gender characteristics of the respondents, including the reported participation of women in relevant decision-making;  Household characteristics that measure household demographic characteristics, including gendered household type, and education characteristics of household members;  Non-food assets as a measure of household wealth; and  District: dummy variables for districts (Tsholotsho is the excluded comparison district) to account for any geographic factors not captured in other explanatory variables. Table 13 reports the results from the regressions estimating the probability that a farmer used financial services and the probability that farmers adopted at least five sustainable agricultural practices. Adoption of agricultural practices showed significant increases from baseline to endline, controlling for all the other explanatory variables in the equations. Participation in agricultural trainings is also positively associated with increased rates of adoption of both types of practices, and participation in value-chain activities is associated with greater use of financial services. Note that adoption of sustainable agricultural practices and participation in value-chain activities are not included as explanatory variables for the adoption of sustainable agricultural practices, as they are used in the definition of the dependent variable. Female farmers are more likely to adopt sustainable practices. Households in which women participate in joint decisions about credit are more likely to use financial services and adopt sustainable practices, and joint decision-making over assets is also positively associated with adoption of sustainable practices. Information about gendered household type provides more information about female decision-making. Households without female decision-makers are less likely to use financial services, while households with no male decision-makers are more likely to adopt sustainable agricultural practices. In these regression models, a variable measuring non-food assets was included as an explanatory variable to measure the effect of wealth on use of financial services or adoption of sustainable practices. Final Performance Evaluation of the Amalima Development Food Security Activity in Zimbabwe Annex J: Multiple Regression Analysis 117 This wealth variable is positively associated with adoption of sustainable agricultural practices, suggesting that access to savings is a requirement to adopt these practices. Table 13: Regression results for use of financial services and adoption of sustainable crop practices Dependent variable Use of financial services in the past 12 months Adopt sustainable crop practices (5 or more) Survey round (Baseline) Endline 1.46*** 0.14 Program participation Sustainable agricultural practices/technologies 0.04 Participated in value-chain activities 0.95*** Participated in agriculture trainings 1.01*** 1.32*** Gender variables Female farmer 0.29 -0.20+ Joint asset ownership -0.47 -0.05 Joint right to one or more assets 0.56 0.52** Joint decisions about credit 0.68** 0.22+ Household characteristics Household size 0.04 0.06** Gendered household type Adult males no adult female -1.71** -0.19 Adult female no adult male -0.31 0.34+ Share of adults with more than primary education 0.54* 0.17 Non-food assets 0.11 0.89** District (Tsholotsho) Bulilima 0.82* 0.16 Mangwe 0.89** 0.29 Gwanda 0.66* 1.15*** Constant -5.50*** -2.20*** Observations 3543 3586 + p<0.1; * p<0.05; ** p<0.01; *** p<0.001 Table 14 provides estimates from the regressions of child nutritional variables: underweight and stunting. The probability of stunting decreased from baseline to endline, after controlling for all other variables in the model, while the probability that a child is underweight did not change significantly. With respect to program participation, only receipt of cash transfers is significantly associated with stunting: children in households that received cash transfers are less likely to be stunted. Access to improved drinking water source and use of cleansing agent and water for washing is associated with lower likelihood of underweight, but these WASH variables are not associated with stunting. Child age is strongly associated with a higher likelihood of both underweight and stunting (the negative coefficient on the squared age term means that this effect is relatively less for older children than for younger). Children who had diarrhea in the last two weeks are more likely to be underweight. IMPEL | Implementer-Led Evaluation and Learning 118 Annex J: Multiple Regression Analysis If the child’s natural mother is in the household the child is less likely to be stunted, and surprisingly, the education level of the child’s caregiver is positively associated with stunting. On the other hand, higher education level of the household is associated with lower levels of both nutritional indicators. Table 14: Regression results for child nutritional variables, underweight and stunting of CU5 Dependent variable Underweight (%<-2sd) Stunting (%<-2sd) Survey round (Baseline) Endline -0.80 -0.97* Program participation Child rations -0.33 0.16 Cash transfer -1.12 -0.81* Nutrition training -0.04 0.48 WASH practices Using an improved drinking water source -0.35+ 0.22 Have cleansing agent and water -2.60* -0.76 Child characteristics Child age (months) 0.06** 0.12*** Child age (months) squared -0.00** -0.00*** Female child -0.13 -0.34** Had diarrhea in the last two weeks 0.45+ -0.10 Caregiver's education 0.31 0.25+ Child's natural mother lives in same HH 0.12 -0.39+ Gender variables Joint asset ownership -0.06 -0.18 Joint right to one or more assets -0.11 0.07 Joint decisions about credit -0.00 -0.13 Household characteristics Household size 0.02 -0.05 Count of children under 5 in household 0.30* 0.38** Gendered household type Adult males no adult female -0.18 -0.46 Adult female no adult male 0.01 0.05 Share of adults with more than primary education -0.75* -0.60* Non-food assets -0.55 -0.22 District (Tsholotsho) Bulilima 0.64** 0.05 Mangwe 0.61* -0.24 Gwanda 0.15 -0.25 Constant -3.44*** -1.94*** Observations 1892 1892 + p<0.1; * p<0.05; ** p<0.01; *** p<0.001 Table 15 presents regression results for household food security. The dependent variable is households reporting adequate food security (i.e., not moderately or severely food insecure) based on the FCS. Overall, the probability that a household reports adequate food security decreased from baseline to endline, controlling for other factors. Final Performance Evaluation of the Amalima Development Food Security Activity in Zimbabwe Annex J: Multiple Regression Analysis 119 Households in which women had joint-decision making regarding the use of one or more assets are more likely to be food secure, all else being equal. Household education and wealth levels (as measured by non-food assets) are more likely to be food secure, all else being equal. Table 15: Regression results for household food security status (adequate food security based on HHS) Dependent variables % HH with adequate food security Survey round (Baseline) Endline -0.58*** Program participation Food or cash assistance (0-2) 0.22 Nutrition or agriculture training (0-2) -0.20 WASH practices Using an improved drinking water source 0.07 Have cleansing agent and water 0.47 Gender indicators Joint asset ownership 0.16 Joint right to one or more assets 0.33* Joint decisions about credit -0.08 Household characteristics Household size 0.05† Gendered hh type/Male and female headed 0.00 Adult males no adult female 0.20 Adult female no adult male -0.12 Share of adults with more than primary education 0.57** Non-food assets (USD 2014) 1.19*** ENSURE Districts (Tsholotsho) Bulilima -0.54** Mangwe 0.45* Gwanda 0.91*** Constant -0.68* Observations 2897 + p<0.1; * p<0.05; ** p<0.01; *** p<0.001