MID-TERM PERFORMANCE EVALUATION OF USAID/NEPAL’S SUAAHARA II INTEGRATED NUTRITION PROGRAM December 2019 This publication is made possible by the support of the American people through the United States Agency for International Development (USAID). The contents of this publication are the sole responsibility of the authors and do not necessarily reflect the views of USAID or the United States Government. ii MID-TERM PERFORMANCE EVALUATION OF USAID/NEPAL’S SUAAHARA II INTEGRATED NUTRITION PROGRAM December 2019 Evaluation Mechanism Number: AID-367-C DISCLAIMER This publication is made possible by the support of the American people through the United States Agency for International Development (USAID). The contents of this publication are the sole responsibility of the authors and do not necessarily reflect the views of USAID or the United States Government. iii TABLE OF CONTENTS ABBREVIATIONS................................................................................................................................................ V TEAM MEMBERS ............................................................................................................................................VIII ABSTRACT ......................................................................................................................................................... X EXECUTIVE SUMMARY .....................................................................................................................................XI RECOMMENDATIONS FOR FUTURE USAID NUTRITION PROGRAMMING .......................................................................... XVII 1 EVALUATION PURPOSE ............................................................................................................................. 1 2 PROGRAM BACKGROUND......................................................................................................................... 1 3 EVALUATION QUESTIONS AND METHODOLOGY....................................................................................... 2 3.1 METHODOLOGY ........................................................................................................................................... 3 3.1.1 Data Collection Methods..................................................................................................................... 3 3.1.2 Geographic Sites Selected ................................................................................................................... 4 3.1.3 Data Analysis....................................................................................................................................... 4 3.1.4 Validation of Findings.......................................................................................................................... 5 3.2 LIMITATIONS................................................................................................................................................ 5 4 FINDINGS .................................................................................................................................................. 6 4.1 EVALUATION AREA 1: EFFECTIVE COVERAGE OF PROGRAMMING........................................................................... 6 4.1.1 Nutrition and Health Behaviors and Service Utilization ...................................................................... 6 4.1.2 Access to Diverse and Nutrient-rich Foods.......................................................................................... 9 4.1.3 Suaahara II’s Contribution to Closing Gaps Between Socio-economic Groups and to Changes in Social Norms..................................................................................................................................................... 9 4.1.4 Implementation of Integrated Nutrition Programming Approach....................................................11 4.1.5 Continuum of Care and Life Cycle Approach. ....................................................................................12 4.1.6 Utilization of Existing Platforms........................................................................................................13 4.1.7 Beneficiaries’ Understanding of Suaahara........................................................................................16 4.1.8 Knowledge and skills of Suaahara FLWS and FCHVs.........................................................................16 4.2 EVALUATION AREA 2: MULTI-SECTORAL AND MULTI-LEVEL STAKEHOLDER COORDINATION, COLLABORATION, AND ENGAGEMENT: ........................................................................................................................................................17 4.2.1 Rollout of the Multi-Sector Nutrition Plan (MSNP-II) ........................................................................17 4.2.2 Convening, Coordinating, and Facilitating the Implementation of MSNP ........................................18 4.2.3 Linkage and Collaboration with Other Relevant USAID Activities and Other Nutrition Partners......19 4.2.4 Private Sector Engagement...............................................................................................................21 4.3 EVALUATION AREA 3: CAPACITY STRENGTHENING AND SUSTAINABILITY APPROACH: .................................................22 4.3.1 Contribution to Multi-sectoral Nutrition and Health Programming..................................................22 4.3.2 Transitioning Technical and Programmatic leadership to Nepal institutions [GON and Civil Society Organizations (CSOs)].....................................................................................................................................24 4.3.3 Technical Assistance (TA) for Planning, Implementation, and Monitoring of the MSNP at the Local Levels 24 4.3.4 Challenges Suaahara II Faces in Implementing Multi-sector Nutrition Interventions.......................25 4.4 EVALUATION AREA 4: MONITORING, EVALUATION, RESEARCH, AND LEARNING: ......................................................26 4.4.1 Robustness of MER System ...............................................................................................................27 4.4.2 Data Sharing......................................................................................................................................28 4.4.3 Demonstrating Innovation ................................................................................................................29 4.4.4 Alignment with the Global Nutrition Agenda....................................................................................30 4.4.5 Evaluation Area 5: Program Management .......................................................................................31 4.4.6 Program Management Structure, Coherence, and Relationship.......................................................31 4.4.7 Building Local NGOs Capacity ...........................................................................................................32 4.4.8 Potential Areas for Improvement......................................................................................................33 iv 5 RECOMMENDATIONS ............................................................................................................................. 33 5.1 PROGRAMMATIC: SCALE, QUALITY AND SUSTAINABILITY ....................................................................................33 5.2 TECHNICAL ................................................................................................................................................35 5.3 PARTNERSHIP.............................................................................................................................................37 5.4 RECOMMENDATIONS FOR FUTURE USAID NUTRITION PROGRAMMING ................................................................37 APPENDIX 1: EVALUATION STATEMENT OF WORK.......................................................................................... 39 APPENDIX 2: GETTING TO ANSWERS MATRIX ................................................................................................. 57 APPENDIX 3: ADDITIONAL TABLES AND FINDINGS (SCONDARY ANALYSIS OF 2017 & 2018 ANNUAL HH SURVEY DATA).............................................................................................................................................................. 74 APPENDIX 4: WORK PLAN ............................................................................................................................... 88 APPENDIX 5: ETHICAL APPROVAL LETTER........................................................................................................ 89 APPENDIX 6: RESULTS FRAMEWORK AND RESEARCH QUESTIONS .................................................................. 90 APPENDIX 7: ADDITIONAL TABLES FOR METHODOLOGY................................................................................. 93 APPENDIX 8: DISCLOSURE OF ANY CONFLICT OF INTEREST ............................................................................. 95 LIST OF TABLES Table 1: Changes in Behaviors and Service Utilization Among Beneficiaries................................................. 6 Table 2: Use of Modern Methods of Family Planning Among Mothers with a Child Under the Age of 2 ............................................................................................................................................................................... 75 Table 3: ANC Visits at Least 4 Times During Pregnancy................................................................................. 76 Table 4: PNC Checkup 3 or More Times within 7 Days-Post Delivery....................................................... 77 Table 5: Appropriate Method of Water Treatment Among Households with Children Under Age 2 78 Table 6: Handwashing in Six Critical Times Among Mothers with a Child Under Age 2 ........................ 79 Table 7: HMG Meeting Among Mothers Who Have a Child Under the Age of 2 (ever participated) . 80 Table 8: Exclusive Breastfeeding Among Mothers with a Child Under Age 2 ............................................ 81 Table 9: Minimum Acceptable Diet Among Mothers with a Child Under Age 2....................................... 82 Table 10: Dietary Diversity Among Children 6-23 Months of Age (foods from 4 or more of 7 food groups) ................................................................................................................................................................. 83 Table 11: DD Among Mothers with a Child Under Age 2 (Foods from 5 or more of 10 food groups) ............................................................................................................................................................................... 84 Table 12: Growth Monitoring Information (only available for 2018)............................................................ 85 Table 13: IR 3 Relevant Indicators......................................................................................................................... 85 Table 14: Evaluation Questions.............................................................................................................................. 90 Table 15: Mid-term Performance Evaluation Sites............................................................................................. 93 Table 16: List of Selected Municipalities and Wards......................................................................................... 93 Table 17: Overview of Data Collection Effort.................................................................................................... 94 LIST OF FIGURES Figure 1: Ratio of Highest Quintile with Lowest Quintile Between 2017 and 2018 (Lower the better) ............................................................................................................................................................................... 85 Figure 2: Difference in Highest and Lowest Quintile Between 2017 and 2018 (Lower the better)...... 86 v ABBREVIATIONS ANC Antenatal Care BA Bhanchhin Aama (Mother Knows Best) radio program CB-IMNCI Community Based - Integrated Management of Neonatal and Childhood Illnesses CDCS Country Development Cooperation Strategy CHD Child Health Division CHSB Community Health Score Board CMC Community Mapping Census CNV Community Nutrition Volunteer CLA Collaborating, Learning, and Adapting CPR Contraceptive Prevalence Rate COP Chief of Party CSO Civil Society Organization DADO District Agricultural Development Office DAG Disadvantaged Groups DC District Coordinator DHO District Health Office DHS Demographic and Health Survey DHIS District Health Information System DIP Detailed Implementation Plan DLSO District Livestock Services Office DMDO District Monitoring and Documentation Officer DPHO District Public Health Office DQA Data Quality Assessment DWSSO District Water Supply and Sanitation Office representative DEC Development Experience Clearinghouse DO Development Objective DRRO Disaster Risk Reduction Office (USAID/Nepal) ENPHO Environment and Public Health Organization ET Evaluation Team FC Field Coordinator FCHV Female Community Health Volunteer FGD Focus Group Discussion FHD Family Health Division FHI 360 Family Health International FLW Frontline Worker FM Frequency Modulation FP Family Planning FPAN Family Planning Association of Nepal FS Field Supervisor FTF Feed the Future FY Fiscal Year GESI Gender Equity and Social Inclusion GMP Growth Monitoring and Promotion GON Government of Nepal HFP Homestead Food Production HFPB HFP Beneficiary HH Household HKI Helen Keller International HMIS Health Management Information System HMG Health Mothers’ Group HR Human Resource HP Health Post vi HSS Health Systems Strengthening HTSP Healthy timing and spacing of pregnancy IEC Information Education and Communication IFA Iron and Folic Acid IMAM Integrated Management of Acute Malnutrition IPC Interpersonal Communication IR Intermediate Result IP Implementing Partner IT Information Technology JTA Junior Technical Assistant KII Key Informant Interview KISAN Knowledge-based Integrated Sustainable Agriculture in Nepal (USAID/Nepal Activity) KPI Key Performance Indicator LAM Lactational Amenorrhea Method LG Local Government LRP Local Resource Person MAM Moderate Acute Malnutrition M&E Monitoring and Evaluation MEL Monitoring, Evaluation, and Learning (USAID/Nepal Activity) MER Monitoring Evaluation and Research MIYCN Maternal Infant and Young Child Nutrition MNCH Maternal, Newborn, and Child Health MSNP Multi-Sector Nutrition Plan (GON) MSNS Multi-Sectoral Nutrition Strategy (USAID) MTE Mid-Term Performance Evaluation MOALD Ministry of Agriculture and Livestock Development MOE Ministry of Education MOFAGA Ministry of Federal Affairs and General Administration MOHP Ministry of Health and Population MOU Memorandum of Understanding MTA Mid-term Assessment MUAC Mid Upper Arm Circumference NACS Nutrition Assessment, Counseling and Support NDHS Nepal Demographic and Health Survey NFSSC Nutrition and Food Security Steering Committee NGO Non-governmental Organization NHEICC National Health, Education, Information and Communication Center NHRC Nepal Health Research Council NIL Nutrition Innovation Lab (Feed the Future) NPC National Planning Commission NTAG Nepali Technical Assistance Group ODF Open Defecation Free OFSP Orange Fleshed Sweet Potato ORC Outreach Clinic ORS Oral Rehydration Solution OTC Outpatient Therapeutic Center PAHAL Promoting Agriculture, Health and Alternative Livelihoods (USAID/Nepal Activity) PCA Program Constraints Assessment PHCC Primary Health Care Centre PNC Postnatal Care PMP Performance Management Plan PNGO Partner Non-Governmental Organization PPP Public Private Partnership RM Rural Municipality RUTF Ready to Use Therapeutic Food vii SABAL Sustainable Action for Resilience and Food Security (USAID/Nepal Activity) SAM Severe Acute Malnutrition SATH Self-Applied Technique for Quality Health SBCC Social and Behavior Change Communication SSBH Strengthening Systems for Better Health (USAID/Nepal Activity) SMS Short message service (texting) SLT Senior Leadership Team SOW Scope of Work STTA Short Term Technical Assistance TA Technical Assistance TOC Theory of Change TOT Training of Trainers UM Urban Municipality UNICEF United Nations Children’s Fund USAID United States Agency for International Development/ Mission USD United States Dollar VDRC Vijaya Development Resource Center VMF Village Model Farmer WASH Water, Sanitation and Hygiene WASH-CC WASH Coordination Committee WRA Women of Reproductive Age viii TEAM MEMBERS Team Members Mellen Duffy Tanamly, Evaluation Team Lead Dr. Amit Bhandari, Health Sector Specialist Dr. Subas Risal, Evaluation Expert Dr. Ramesh Adhikari, Senior Nutrition Advisor Sabina Sigdel, Field Researcher and Interpreter/Translator Pankaj Sharma, Field Research Coordinator Srijana Giri, Field Researcher Laxmi Thapa, Field Researcher Maya Chetri, Field Researcher Goma Lama, Field Researcher Nishu Aryal, Field Researcher USAID/Nepal’s Monitoring, Evaluation, and Learning (MEL) Support Kshitiz Shrestha, Evaluation Specialist (Senior Level) Manorama Adhikari, Deputy Chief of Party Swadesh Gurung, Monitoring & Evaluation Specialist Ganesh Sharma, Statistician and Data Analyst ix Acknowledgments This evaluation could not have been possible without the contributions of dedicated U.S. Agency for International Development (USAID) and implementing partner (IP) staff whose inputs were instrumental to the evaluation team’s efforts. The team expresses its deepest gratitude to Nutrition Specialist Debendra Adhikari and Strategic Information and Research Advisor Sabita Tuladhar Pradhan at the Health Office, and Monitoring and Evaluation Specialists Murari Adhikari and Prakash Gnyawali at Program and project Development Office for their tremendous support and guidance throughout the evaluation. The evaluation team also wishes to acknowledge the support it received from the Suaahara II Activity’s staff. Specifically, the team expresses its appreciation for the generosity of Deputy Chief of Party, Programs Pooja Pandey, Senior Technical Advisor Kenda Cunningham, and their extensive team for their willingness to welcome the evaluation team, provide critical input to numerous inquiries, supply project data, and facilitate introductions to all related stakeholders at the central level and the field level. The evaluation team is grateful to the staff of all organizations, including senior government officials, that were sampled for this evaluation, as well as the beneficiaries who made themselves available to participate in focus group discussions and interviews. These staff and beneficiaries were always welcoming and cheerful, and generously provided their time to share valuable information upon which the findings, conclusions, and recommendations are based. X ABSTRACT The Mid-term Performance Evaluation of the Suaahara II Integrated Nutrition Program employed qualitative and quantitative methods to inform future programming and improve the effectiveness and efficiency of implementation during the remainder of the program. USAID/Nepal provided five evaluation questions relating to effective coverage; multi-sectoral coordination; capacity strengthening and sustainability; monitoring, evaluation, research, and learning; and program management. The evaluation team gathered information in 7 of Suaahara’s 42 districts, central and provincial levels, conducted focus group discussions, key informant interviews and group interviews among beneficiaries and stakeholders, observed health facilities, carried out an online survey of partner non-governmental organizations, performed secondary analysis of Suaahara data, and reviewed the relevant literature. Findings indicate that scale is being achieved without losing focus on quality and program results are largely on track. Suaahara is recognized as a strong nutrition partner and has adapted well to the massive federalism shift. Suaahara staff are using data from its strong M&E system to drive program improvements. Though the targeting for gender equity and social inclusion is too broad, Suaahara has demonstrated success in reducing gaps between different socio-economic groups. Suaahara has prioritized implementing its package of interventions which represents a major part of the national multi-sectoral nutrition plan. While there is considerable progress in project implementation, sustainability efforts are not sufficiently strategic, and participating NGOs are not incorporated as long-term nutrition champions. The evaluation team recommends that Suaahara scales down operations and continues to focus in neediest areas with core plus interventions to increase reach and impact among the most marginalized; keeps capacity strengthening central to maintain gains and improve quality; rethinks approaches to improve key behaviors resistant to change; continues and re-strategizes engagement with local bodies to enhance sustainability; and engages partner NGOs beyond their current mandate. The evaluation team also recommends pro-active engagement to enhance nutrition monitoring within the national Health Management Information System; continues working with the Government of Nepal to implement the national multi-sectoral plan at local levels; and expanding linkages for homestead food production beneficiaries with markets and private input providers. XI EXECUTIVE SUMMARY A. Evaluation Purpose The Suaahara II mid-term performance evaluation (MTE) is designed to inform future programming and to evaluate and improve the effectiveness and efficiency of SII implementation during the remainder of the program. The MTE team has systematically gathered qualitative information and considered available quantitative data to determine what the project has achieved to date or is likely to achieve during the life of program, how it is being implemented; the challenges that the program has faced; how it is perceived and valued; and how it is managed. B. Program Background Suaahara II Integrated Nutrition Program (2016-2021) builds on Suaahara I (2011-2016) accomplishments. The overall goal is to improve nutritional status of women and children under five years in Nepal through an increased emphasis on the 1,000-day period between pregnancy and a child’s second birthday. Led by Helen Keller International (HKI) with a consortium of partners, the project works with the Government of Nepal (GON) in 389 Nepal’s 753 municipalities across 42 of 77 districts using evidence-based, multi￾sectoral nutrition interventions that include nutrition-specific and nutrition-sensitive activities at the policy (national and sub-national), community, and household levels. Specifically, Suaahara focuses on improving nutrition services; maternal, newborn, and child health services; reproductive health/family planning services; water, sanitation and hygiene (WASH); homestead food production (HFP) - kitchen gardening, poultry rearing, and marketing; and multi-sectoral nutrition governance. It is working to achieve its objectives through multi-sector partnerships with different levels of actors throughout the government structure, private sector, and other development stakeholders, as well as via public-private partnerships. Exhibit 1: Evaluation Areas/Questions 1. Effective coverage of programming: To what extent are Suaahara II interventions reaching the intended target populations (1,000-day mothers and households, adolescent girls, frontline service providers, Female Community Health Volunteers (FCHVs), village model farmers), particularly food insecure and disadvantaged communities, and resulting in improved health and nutrition behaviors and utilization of nutrition, MCH, FP, WASH, and Homestead Food Production (HFP) services? 2. Multi-sectoral and multi-level stakeholder coordination, collaboration, and engagement: To what extent have GON counterparts, other USAID implementing partners, and other stakeholders, at national and subnational levels and across sectors, been engaged in program planning, implementation, and monitoring for Suaahara II? 3. Capacity strengthening and sustainability approach: To what extent and how is Suaahara II programming contributing to improved capacity, ownership, and motivation to implement multi-sectoral nutrition and health programming within the GON at different levels? 4. Monitoring, evaluation, research, and learning: How are Suaahara II’s monitoring, evaluation, and research systems and activities contributing to data-driven program implementation? 5. Program management: How effective are the current program management structure and operations to implement this complex multi-sectoral program? XII C. Evaluation Questions and Methodology The MTE was guided by five evaluation questions (Exhibit1). Qualitative methods were primarily utilized, along with analysis of secondary data, an online mini survey, and literature review. Key Informant Interviews, Focus Group Discussions and Group Interviews were conducted with direct and indirect beneficiaries, USAID/Nepal, GON stakeholders at multiple levels, consortium partners, partner NGOs and other development agencies. Data was collected in seven Suaahara II districts from April to June 2019. Districts were selected purposively based on specific criteria which included intensity of program, presence of other USAID/Nepal programs and geographic diversity. Ethical approval was obtained from Nepal Health Research Council. Qualitative data were analyzed using Nvivo 12 whereas secondary analysis was conducted in Stata15. D. Findings and Conclusions 1. Effective Coverage of Programming USAID’s Multi-Sectoral Nutrition Strategy (MSNS) 2014–2025 aims to “scale up effective, integrated, nutrition-specific and nutrition-sensitive interventions, programs, and systems across humanitarian and development contexts.” The MTE finds that Suaahara is consistently implementing a strong multi-sector program well aligned with priorities set forth in the MSNS and the Nepal country context. Through scaling up proven and cost-effective interventions— nutrition specific such as improved infant and young child feeding and iron and folic acid supplementation (IFA), with strategically selected nutrition￾sensitive interventions such as homestead food production, enhanced dietary diversity for women and children, WASH, and promotion of healthy timing and spacing of pregnancy (HTSP), in the same communities and households—Suaahara is helping to extend effective coverage of much needed nutrition programming throughout target districts, especially to underserved areas. Suaahara is expanding nutrition interventions to promote behavior change and use of nutrition services over a wide geographic area through multi-sectoral involvement while maintaining high levels of quality and equity. Suaahara is employing smart GESI strategies, reaching women and marginalized groups; but targeting can be sharpened. Overall, Suaahara is taking the right approach to change priority nutrition-related practices – a multi￾pronged, comprehensive Social and Behavior Change Communication strategy, starting with household visits, and a variety of media used to reach mothers and other family members. Knowledge about ten priority behaviors has increased significantly; and there is a positive trend in practice of seven priority behaviors: dietary diversity (both mothers and children), hand washing at all six critical times, households practicing correct use of recommended household water treatment technology; 4+ ante-natal care (ANC) visits by pregnant women, and IFA tablets taken for180 days during pregnancy; immediate and exclusive breastfeeding rates increased slightly. There were however a few behaviors that showed stagnancy such as oral rehydration solution with therapeutic zinc treatment for children with diarrhea, and the use of modern family planning methods. Coverage of evidence-based nutrition interventions for key life cycle stages, such as pregnancy and infancy, is increasing through health providers at community level and health facilities. The importance of maternal nutrition factors in the life cycle is coming to the fore globally, and Suaahara is promoting good maternal nutrition to break the cycle of malnutrition. Suaahara has recently added a new initiative to improve nutrition and health among adolescent girls – another significant stage in the life cycle. Suaahara’s XIII HFP activities have expanded access to diverse and nutrient-rich foods. Village Model Farmers (VMFs) are successfully supporting 3902 active HFP groups in 1500 Core+ wards1, but need more linkages with private input suppliers and markets, GON services and resources, especially for chicken vaccination, drip irrigation and other climate-smart technologies, and marketing. The program has integrated GESI strategy and principles into its programming, which has led to positive impact in terms of reducing the gaps between socioeconomic groups and changing social norms. The secondary analysis of Suaahara II Annual Household Surveys 2017 and 2018 data shows gaps are closing in majority of health and nutrition indicators, for example egg consumption among mothers and children, attendance of at least four ANC visits, and meat consumption. The analysis shows that the blanket approach (DAG vs. non-DAG areas) is working relatively well to equitably and consistently improve the Suaahara related behaviors and service use among those at the lower wealth quintiles and the socially excluded groups. This is despite the fact that about a quarter of DAG population are not covered with core plus interventions. 2. Multi-sectoral and Multi-level Stakeholder Coordination, Collaboration, and Engagement Suaahara II has positioned itself well as a key and trusted nutrition partner both at the national and sub￾national levels. The program has successfully engaged government at federal as well as local levels. In year 2 of the program, Suaahara’s main points of coordination in delivering the program changed from 40 District Health offices to 389 municipalities (urban and rural) in 42 (2 Suaahara districts split) program districts. This called for a major shift in the way the program engaged with the government to which Suaahara rapidly responded, adapting well without major disruption to the program activities or outputs. There is considerable engagement and support of local governments – municipalities and wards – in planning, implementation and monitoring of the program. There is intensive engagement with elected and technical officials and frontline workers, including FCHVs, at the local levels, thanks to Suaahara’s network of Field Supervisors (FS) and Community Nutrition Volunteers (CNV). Suaahara relies primarily on their one to one engagement with key stakeholders for coordination and collaboration needed to deliver the program effectively. Mutual cooperation between Suaahara II and local governments was evident in the sampled districts. Local leaders and officials are highly appreciative of technical support provided by Suaahara II on planning and budget disbursement. Some Suaahara II innovations / interventions have been replicated with local government funding, for example, establishment of nutrition corners in all health facilities, scaling up of “letter to father” in non Suaahara II districts and use of Suaahara II BCC materials. Continued engagement with and support by Suaahara has resulted in local elected officials being increasingly sensitised on the value of nutrition and health, as reflected in the gradual prioritisation and increasing allocation of funding by local bodies to multi sector nutrition activities. Suaahara was closely involved in the development of the second national Multi-Sector Nutrition Plan (MSNP II) and the support has been highly recognized by the government and other partners. Suaahara is active in national multi-sector nutrition fora, working closely with the National Planning Commission (NPC, the apex body for the MSNP II), relevant sector ministries and other partners. Suaahara is helping to fill gaps in Nepal in implementation of the MSNP II and is influential in country in keeping nutrition in the policy makers view. Suaahara II has helped establish MSNP coordination structures at local levels in many districts; however, continued support by the GON is lacking. Many of these committees are not yet fully functional as evident in the sampled municipalities and wards. 1 Wards with majority poor vulnerable and socially vulnerable and/or marginalized population XIV Suaahara II collaborates with relevant USAID programs – notably with KISAN-II and Strengthening Systems for Better Health - identifying some joint areas of work and mutual support. KISAN II and Suaahara II are both contributing to CDCS, USAID Multisector Nutrition Strategy and Feed the Future objectives and exchange technical assistance and training for each other’s beneficiaries. This partnership is particularly important for sustaining HFP groups through private sector linkages. Suaahara also works closely with UNICEF at the central level facilitating joint working and minimized duplication, to the extent possible, in the 14 districts where both organisations overlap. Potential partnerships have been explored with the private sector; a couple of good examples of centrally brokered partnerships are in place in the WASH and agriculture sectors. The MTE generally found that private sector engagement and partnership are little understood particularly at the implementation level, with no formalized partnerships as yet in any of the sampled districts. Suaahara II has utilized its partnership with KISAN-II to facilitate some private sector engagement on a limited scale; e.g. initiatives to link HFP groups with private input suppliers and vendors to ensure local availability of required inputs and markets to sell surplus produce. More awareness of private sector potential for expanding and sustaining development work is needed, including in the health sector. Suaahara II has the potential and scope to define the benefits of private sector partnerships through small to medium scale private sector engagement that it has facilitated or is in the process of negotiating. 3. Capacity Strengthening and Sustainability Approach Suaahara’s programming is centered around support and facilitation of capacity development within Nepal institutions. Suaahara’s training for technical, managerial and local government officials has been intensive and well received. The MTE finds that more attention and a nuanced approach to sustainability is needed in the remaining two years of the program to ensure that capacity improvements and gains in nutrition status are maintained and expanded. National level GON officials emphasize that Suaahara should intensively focus on increasing “ownership” and “governance” at local levels for MSNP. In order to institutionalize these, Suaahara will need to explore opportunities with the GON to strengthen national capacity to orient, facilitate, coordinate and provide an enabling environment for local governments to own and manage MSNP. Suaahara has played an important role in building the capacity of local elected leaders, officials and frontline workers (e.g. health, agriculture, WASH) in areas related to nutrition and health by orienting them on the integrated nutrition program and also on the broader multi-sector nutrition and development agenda. Local respondents urged for sharper targeting of marginalized groups. This may necessitate USAID and Suaahara II categorizing districts for phased graduation and focusing on newer districts and the districts that have high levels of malnutrition. Suaahara’s FLWs are considered active and well qualified, knowledgeable about their work, tools and activities, and trusted by communities. These cadres are valuable resources for future community nutrition programming; Suaahara II may negotiate hiring CNVs and FS for local government roles as part of close out. Local NGO partners are not viewed beyond their implementation role. Nevertheless, PNGOs have taken steps to build their capacity through involvement with Suaahara. They have developed or upgraded their GESI, HR, and administrative policies which demonstrates their intention to progress. Their presence at district and local level is an untapped asset for sustainability. For instance, the board of directors, well￾known leaders in their districts, could be oriented as nutrition champions in their communities. 4. Monitoring, Evaluation, Research, and Learning Monitoring, Evaluation, Research and Learning (MER) is a cross-cutting area robustly focused on by the program – much bigger in scope, scale and intensity than has been seen in nutrition and health programs XV of this size in Nepal. Suaahara II has a strong program-specific MER system whose data is regularly used for planning and management. But this data is not systematically shared with or used by GON programs at sub-national level. Suaahara II is however, helping to improve the quality of information collected by strengthening the capacity of the government through routine data quality assessment. It is not possible to obtain the breadth and depth of data generated by the program from the national information systems, except for limited population level and service coverage data. Suaahara MER systems are technology driven and human resource intensive. Quite large annual household surveys, covering more than 3,500 HHs, and other research (e.g. formative) are done through contracted research agencies. The information and knowledge generated by Suaahara II is used by the program to continuously learn and improve as well as to create global public goods. An internal platform for sharing amongst the field staff could be useful not only in improving internal learning, but also for adaptation and scale up of local solutions and innovations some districts might have discovered. Suaahara is also focusing on research in partnership with other global institutions and universities. Global sharing of results and learning from the program (webinars, seminars, workshops, presentations and publications) has been a key feature of the program. 5. Program Management Suaahara’s management structure is well setup given the number of entities involved in implementation. The program has been successful in demonstrating itself as one entity i.e. Suaahara. Although delayed release of budget required “Slow Down” of certain activities and deferral of others in Year 2, the program management has been effective in delivering a complex program in a rapidly changing socio￾political context. Its joint program management and operations structure has been an advantage for coherence and uniformity in delivery both at headquarters and districts. There is a senior management “Consortium Review Committee” which reviews program progress and discusses key issues, and a Program Advisory Committee, comprised of all relevant government ministries and USAID in place to galvanize support for Suaahara, ensure linkages with GoN programs, share learning, and increase the ownership and sustainability of program activities. Some consortium partners feel that major decisions are sometimes made by a couple of “big partners” without consultation with all partners. Suaahara II district teams work as one, though staff are administratively under different consortium partners. There are some grievances about different remuneration and benefits packages for the same level of job as each consortium partner has its own benefit packages and terms and conditions for its employees. A few operational and management issues were reported that affect implementation of activities, and were not adequately addressed, such as delayed disbursements to a sub-awardee by the prime or to PNGOs by a partner. In general, PNGOs were found to have positive working relations with the consortium partners; most of the PNGOs interviewed had been engaged since Suaahara I and with their continuation in Suaahara II, there has been positive growth of PNGOs. On paper the planning is bottom-up with districts preparing Detailed Implementation Plans (DIP) within the parameters set by the center. However, in practice, it is strongly top-down planning in which districts have little flexibility to propose anything contextual or innovative. Suaahara II and PNGO staff are stretched due to the new Federal structure as they now have to coordinate with 389 municipalities and over 3000 wards. The addition of CNVs as a cadre has helped free up FS to focus on local coordination; however, the district level staff now have almost ten times more local government entities with whom they must coordinate. Moreover, the Suaahara district structure is being gradually tapered down with one field team responsible for more than one district. Suaahara II needs to carefully consider how district teams are pulled out of the districts with minimal disruption to the coordination and influence it has in those localities. XVI E. Recommendations Recommendations for Suaahara II have been grouped into three areas: Programmatic, Technical and Partnership. Recommendations for future USAID/Nepal nutrition programming are also included. PROGRAMMATIC: Scale, Quality and Sustainability 1. Scale down and continue with additional focus in certain areas to make higher impact in the neediest areas. Focus on gradually reducing scale (breadth as well as depth – geographic and interventions) while continuing with sharper targeting of disadvantaged populations and doing more on strengthening capacity, improving quality and sustainability. 2. Keep capacity strengthening central to maintain gains and improve quality. This may not always require expensive training courses but continue providing individual and group coaching and mentoring. 3. Continue and re-strategize engagement with local bodies to enhance sustainability of resources, critical activities and systems for improved nutritional outcomes. The strength for nutrition action is in local bodies. Engagement and ownership of the Suaahara program and agenda by the local government, as evident in all sampled districts, present a huge opportunity to engrain nutrition as a top development agenda locally wherever the program operates. TECHNICAL: 1. Engage pro-actively with MOHP to support improvements in GON’s Health Management Information System (HMIS) for nutrition data. 2. Rethink alternatives for areas where little improvement is seen: Suaahara II would need to rethink, perhaps research further, and try alternative approaches, interventions or innovations to achieve improvements in the remaining two years in three key behaviors (sick child feeding, use of ORS and Zinc to treat diarrhea, and treatment of drinking water) and service use (e.g., growth monitoring and promotion and IMAM), which have been found challenging to change. 3. Ensure new initiatives of the program are timely assessed and lessons incorporated and shared. a. Take stock of the Suaahara Integrated Adolescent Nutrition Program, including the full Reproductive Health, Nutrition and WASH education experience and learnings in the four pilot districts, and share results with USAID and other relevant partners through appropriate knowledge sharing platforms. b. Continue to improve quality of IFA supplementation activity and consider scaling-up based on findings of municipality level reviews held after the first year of implementation. 4. Use alternative media platforms to increase the reach amongst the family of 1000-day mothers. PARTNERSHIP: 1. Continue work with the National Planning Commission (NPC) to increase awareness and to promote the value of the MSNP II at local levels in order to garner more interest and resources for multi-sector nutrition interventions. 2. Engage the partner NGOs beyond the current mandate of the program, building their capacities for sustainability of the multisector nutrition agenda 3. Link beneficiaries (VMFs and HFPB groups) to market and private suppliers to increase access to inputs as well as to sell surplus produce. XVII Recommendations for Future USAID Nutrition Programming 1. Remaining Period of Current Program a) Continue work with GON and other donors to agree on a minimum set of nutrition-specific and nutrition-sensitive interventions that should be implemented in target geographic areas to satisfy MSNP II requirements. b) Take stock of the Suaahara II Integrated Adolescent Nutrition Program Ensure its results and learnings are used in other USAID programs and shared with other partners for potential replication. c) Continue advocating with the GON to engage the government health personnel (particularly health managers) to build their capacity in planning and management of nutrition and health programs. 2. For Future a Multi-sectoral/Integrated Nutrition Program a) Consider developing true collaboration among programs by having joint goals and work plans with nutrition objectives. b) Think about how target areas are better selected for future nutrition investments. c) Consider having better coordination with other multi/bi-lateral agencies working in the same geographic areas to avoid duplication and to ensure programming synergy, where more than one partners are present. 3. Other Agenda Beyond the Remit of Current Program a) Consider opportunities to advocate for and support developing broader nutrition capacity. Increasing the number and quality of nutrition professionals across all sectors and enhancing the technical knowledge and skills of these professionals are critical to improving the quality of nutrition services. The expansion of professionals and frontline workers should include measures to ensure that women have the opportunity and ability to gain the knowledge and skills needed to join the nutrition workforce. b) Consider instilling awareness of private sector potential and value of private sector engagement and partnerships amongst its implementing partners. 1 1 EVALUATION PURPOSE The mid-term performance evaluation (MTE) of Suaahara II2 is designed to inform future programming and to evaluate and improve the effectiveness and efficiency of Suaahara implementation during the remainder of the program. The MTE team consulted with Suaahara implementation staff, the USAID/Nepal program management team, and the Government of Nepal (GON); reviewed existing evidence and literature relevant to Suaahara implementation; and solicited feedback from program participants and key stakeholders on the quality of Suaahara implementation, adjustments to be made, and opportunities for improvement. The intended audience and primary users of the MTE are the USAID/Nepal Mission and Suaahara staff and partners. The GON, beneficiaries of Suaahara, and other offices in the USAID Global Health and Asia Bureaus also are target audiences. Additionally, the results of the MTE are applicable and useful to other USAID bureaus and missions implementing health and nutrition programs, other organizations implementing in Nepal, and other organizations working in nutrition, health, and nutrition-sensitive agriculture programming more broadly. 2 PROGRAM BACKGROUND Suaahara is a comprehensive program that aims to improve the nutritional status of women and children under five years in Nepal, with an emphasis on the 1,000 days between pregnancy and a child’s second birthday3. It works with the GON in 389 of Nepal’s 753 municipalities across 42 of 77 districts using evidence-based, multi-sectoral nutrition interventions that include nutrition-specific and nutrition￾sensitive activities at the policy (national and sub-national), community, and household (HH) levels. Specifically, Suaahara focuses on improving nutrition services; maternal, newborn, and child health (MNCH) services; reproductive health/family planning (RH/FP) services; water, sanitation and hygiene (WASH); homestead food production (HFP) (home gardening and poultry rearing); and multi-sectoral nutrition governance. It is working to achieve its objectives through multi-sector partnerships with different levels of actors throughout the government structure, private sector, and other development stakeholders, as well as via public-private partnerships. Additionally, Suaahara emphasizes gender equity and social inclusion (GESI) as a cross-cutting theme; implements activities via diverse social and behavior change efforts, and has extensive monitoring, evaluation and research (MER) for learning data systems. Suaahara II aims to reach 1.5 million pregnant and lactating women and children under two years. In addition, the program is targeting other HH members, including children under five, mothers-in-law, and husbands, who play a critical role in improving maternal and child health and nutrition as part of the HH approach. In some areas, Suaahara also targets adolescent girls as part of a learning agenda. It implements its Core package in all 3,353 wards of 389 municipalities and Core+ activities in 1,504 wards in disadvantaged communities (classified in the lowest three to six categories by the Ministry of Federal Affairs and Local Development prior to the start of Suaahara) of select municipalities. The Core intervention package includes social and behavior change (SBC) interventions (nutrition, WASH, FP, MNCH); maternal, infant and young child nutrition assessment and counseling support; integrated management of acute malnutrition (IMAM); governance activities, and GESI activities. The Core+ package 2 Hereinafter referred by “Suaahara” unless otherwise stated. 3 1,000-day mother is used for a woman who is in the 1,000 days between the start of a pregnancy and the second birthday of the child born out of that pregnancy. Similarly, 1,000-day HH is used for a HH with a 1,000-day mother. Suaahara II Partners HKI (Prime) CARE FHI 360 ENPHO VDRC NTAG Equal Access PNGOs in each district 2 adds enhanced HFP (home gardening and poultry rearing) that includes improving agriculture and livestock services, income generation activities, as well and more intensive social and behavior change communication (SBCC), health system strengthening (HSS), WASH, and GESI components. Helen Keller International (HKI) and six consortium partners implement Suaahara II. A chief of party (COP) leads the program, and each consortium partner has deputed technical staff in the Suaahara Kathmandu office as well as in Suaahara field offices. Further, for the first half of Suaahara II, a small team operated out of Nepalgunj, as a Program Linkages office under the leadership of the program operations managers, which was responsible for linkages between Suaahara and other USAID-funded programs. Suaahara consortium partners jointly implement activities in an integrated way to demonstrate coherence in programming and convergence with 1,000-day HHs at the community level. Also, HKI partners with local NGOs (partner non-governmental organizations, PNGOs) in each district to implement Suaahara activities, with the aim of supporting cost-effective and locally responsive nutrition and health services in partnership with the local government and other stakeholders. Results Framework Suaahara II seeks to achieve four key intermediate results: i) improved household nutrition and health behaviors; ii) increased use of quality nutrition and health services by women and children; iii) improved access to diverse and nutrient-rich foods by women and children; and iv) accelerated rollout of the Multi￾Sector Nutrition Plan (MSNP) through strengthened local governance. (See Appendix 6 for program associated goal and outcomes.) 3 EVALUATION QUESTIONS AND METHODOLOGY Altogether, five broad themes through which evaluation questions were developed were considered (also see Appendix 6 for associated sub-questions for each evaluation question) that included: 1. Effective coverage of programming - To what extent are Suaahara II interventions reaching the intended target populations (1,000-day mothers and HHs, adolescent girls, frontline service providers, FCHVs, village model farmers), particularly food insecure and disadvantaged communities, and are resulting in improved health and nutrition behaviors and utilization of nutrition, MCH, FP, WASH, and HFP services? 2. Multi-sectoral and multi-level stakeholder coordination, collaboration, and engagement - To what extent have GON counterparts, other USAID implementing partners, and other stakeholders, at national and sub-national levels and across sectors, been engaged in program planning, implementation, and monitoring for Suaahara II? 3. Capacity strengthening and sustainability approach - To what extent and how is Suaahara II programming contributing to improved capacity, ownership, and motivation to implement multi-sectoral nutrition and health programming within the GON at different levels? 4. Monitoring, evaluation, research, and learning - How are Suaahara II’s monitoring, evaluation, and research systems and activities contributing to data-driven program implementation? 5. Program management - How effective are the current program management structure and operations to implement this complex multi-sectoral program? 3 3.1 Methodology The MTE for Suaahara II is a performance evaluation largely relying on primary qualitative data collection, secondary analysis of quantitative monitoring data, and a review of existing literature, evidence, and findings from relevant research. The MTE team has attempted to ensure inclusion of all key stakeholders to establish a comprehensive understanding of beneficiaries’ perspectives across local to national levels, ecological regions, the health system, and different sectors and influencers on nutrition, health, and food security. The evaluation has leveraged existing qualitative and quantitative data and findings from recent and concurrent surveys and research activities in Nepal, including but not limited to the Suaahara program’s monitoring, evaluation, and learning activities, the applied research on multi-sectoral nutrition interventions and policy of the Nutrition Innovation Lab (NIL), and the 2016 Nepal Demographic and Health Survey results. The Nepal Health Research Council approved the MTE design on April 24, 2019. 3.1.1 Data Collection Methods 1. Desk Review of Documents and Secondary Data a. Document review: The MTE team conducted a thorough review of program documents and technical strategies and secondary literature to understand the nutrition and health situation in Nepal. The review included population-based health, nutrition, and food security surveys, information collected by other USAID programs in implementation areas (e.g., NIL, Health 4 Life, KISAN II4, PAHAL5), and other available relevant data from government programs and other non-USAID initiatives in the region. b. Analysis of secondary data: The MTE team conducted a thorough review of Suaahara annual household survey data related to nutrition, health status, food security, and health services in implementation areas. c. Research activities: The evaluation team also familiarized itself with the activities and findings of the Feed the Future NIL, which conducts applied research in Nepal focused on nutrition policy and multi-sectoral nutrition interventions. 2. Primary Data Collection a. Focus group discussions (FGD), key informant interviews (KIIs), and group interviews: The MTE team interacted with beneficiaries to assess the quality of program intervention and identify progress, successful approaches, barriers to adoption of optimal nutrition and health practices, and opportunities for program improvement. These beneficiaries and target populations/groups include 1,000-day HH members; adolescent girls; FCHVs; other community frontline workers and volunteers; health facility staff; health mother’s groups (HMGs); and Suaahara frontline workers, including village model farmers (VMF), field supervisors, community nutrition/WASH facilitators, and peer facilitators; nutrition and food security coordination committee members, and other relevant local leaders. The MTE also conducted discussions and interviews with key program personnel, USAID/Nepal representatives, relevant implementing partners, government ministries, and staff at multiple levels (national, district, and municipality), and other development partner agencies and organizations operating in the area. Semi-structured interview tools were tailored to particular categories of respondents. Altogether 147 key informant interviews, 30 FGDs, 23 group interviews, and five observations were undertaken. b. Field observation: The MTE team observed field activities in the Terai, hills, and mountain ecological regions during site visits to collect findings on the implementation of program activities. It visited at least two government health facilities in each municipality. Client-provider interaction was observed to track the behavior of health service providers towards service users, adherence 4 Knowledge-based Integrated Sustainable Agriculture in Nepal second phase. 5 Promoting Agriculture, Health and Alternative Livelihoods. 4 to the quality protocols in providing services, and record keeping. Additionally, observations were conducted to assess various influencing factors in the health facilities including cleanliness of health facilities, stock of micronutrient supplements, and contraceptives. c. Mini survey: A online survey of PNGOs of the 42 program districts was conducted using SurveyMonkey to understand capacity development and the management aspects of PNGOs. Of the 39 PNGOs (covering 42 districts), 27 responded. The survey questionnaires were sent to and responded by the executive directors of the PNGOs (the response was provided by other staff, too, for 5 of the 27 responding PNGOs). 3.1.2 Geographic Sites Selected A purposive sampling approach was used for the selection of study sites. A total of six districts were selected for the evaluation, namely Panchthar, Sindhupalchowk, Lamjung, Rupandehi, Surkhet, and Kailali. Additionally, the MTE team visited the Banke district to carry out interviews with individuals from district and regional offices and to explore the Adolescent Health and Nutrition Program and the IMAM program. The team selected districts based on the following criteria (see Appendix 7 for more details): • Geographic diversity. • Locations where other relevant USAID programs are active, including PAHAL, KISAN, and SSBH. • Newly added districts in Suaahara. Within each district, two municipalities (one rural and one urban), totaling 12 from 6 districts, were selected based on presence of disadvantaged communities. In each municipality, at least one ward was selected for conducting ward level data collection. Out of 12 wards across 12 municipalities, 6 Core+ wards and an equal number of Core wards were selected. 3.1.3 Data Analysis The data collected above have been analyzed to respond to the evaluation questions listed above and assess the soundness of Suaahara’s theory of change and implementation process to achieve the intended outcomes. The evaluators have used predefined criteria for determining the effectiveness and efficiency of programming (See Appendix 2: Getting to Answers Matrix). Use of multiple methods, insights, and existing data and evidence sources allowed the MTE team to triangulate findings and produce more robust evaluation results. • Quantitative secondary data analysis: Changes in specific health and nutrition behaviors have been assessed by analyzing existing data from the Suaahara annual household monitoring survey data for 2017 and 2018. • Qualitative data analysis: A systematic approach to coding, interpreting, and synthesizing the collected qualitative data has been applied. FGDs and in-depth interviews were recorded with the consent of participants and fully transcribed and translated into English before being analyzed, using NVivo software. • Gender and social inclusion considerations: Collected data and analysis have been disaggregated by sex, socio-economic status, age, geographic region, and other characteristics as necessary and where possible to discern differences and similarities in experiences, perceptions, needs, barriers, and other findings. 5 3.1.4 Validation of Findings • Once the preliminary data analyses were completed, the MTE team presented initial observations to USAID/Nepal and Suaahara leadership and relevant staff in a debrief session to validate findings and interpretations. It held a recommendations workshop on July 18, 2019 with USAID and Suaahara team to review the recommendations in the draft MTE report. 3.2 Limitations The evaluation encountered potential areas of bias and other limitations, which it addressed through methodological or analytic means. Limitations experienced consist of the following: • Potential response bias, a common problem for evaluations, where respondents give the interviewer positive remarks about an activity because they would like to receive future support, might have affected this MTE too. A broad range of key informants, program participants, stakeholders, and beneficiaries were included in the sample and triangulation of findings from the different groups of interviewees was done to minimize the impact of this potential bias. Also, the MTE team used direct observation based on nuanced knowledge of the evaluation environment to assist in identifying potential bias in responses. • The MTE team relied to a certain extent on Suuahara personnel to suggest sites to visit and coordinate some of the field logistics. To reduce potential selection bias in this case, the MTE team communicated criteria and standards to the field coordinators and asked for more than one option for Core and Core+ sites. The evaluation team made the final selection. • Another concern has been the availability of contacts. Data collection with direct and indirect beneficiaries was somewhat difficult to schedule because of existing demands on their time. This was particularly true with national GON stakeholders. To mitigate this concern, our team remained flexible to the extent possible to accommodate as many key informants and FGD participants as possible. 6 4 FINDINGS 4.1 Evaluation Area 1: Effective Coverage of Programming Evaluation Question 1: To what extent are Suaahara II interventions reaching the intended target populations (1,000-day mothers and HHs, adolescent girls, frontline service providers, FCHVs, village model farmers), particularly food insecure and disadvantaged communities, and resulting in improved health and nutrition behaviors and utilization of nutrition, MCH, FP, WASH, and HFP services? 4.1.1 Nutrition and Health Behaviors and Service Utilization Suaahara aims to reduce undernutrition among 1,000-day women and children under the age of two, while addressing gender inequality and social/cultural exclusion. Through scaling up6 proven and cost￾effective interventions—such as improved infant and young child feeding, iron and folic acid (IFA) supplementation, WASH practices, and enhanced dietary diversity for women and children—Suaahara’s technical and financial support is helping to extend effective coverage of much-needed nutrition programming throughout the target districts, especially to underserved areas. Suaahara is expanding nutrition interventions with proven efficacy to promote behavior change and use of nutrition services over a wider geographic area while maintaining high levels of quality and equity through multi-sectoral involvement. In year two, through its various activities, Suaahara reached 1,642,275 children under the age of 5, 544,772 children under the age of 2, and 373,600 pregnant women. Suaahara managed to reach over 1.5 million HHs through its Community Mapping Census (CMC) HH visits. In general terms, improvements in most nutrition and health-related behaviors, except a few, can be noted among the intended beneficiaries. Both the primary and available secondary data indicate the improvements. Improved behaviors mainly include the dietary diversity (both mothers’ and children’s), hand washing at all six critical times, HHs practicing correct use of recommended HH water treatment technology, 4+ antenatal care (ANC) visits by pregnant women, and IFA tablets taken for 180 days during pregnancy (see table1 below). Immediate and exclusive breastfeeding rates increased slightly. There were, however, a few behaviors that showed a decline, such as oral rehydration solution (ORS) with therapeutic zinc treatment for children with diarrhea and the use of modern FP methods. Table 1: Changes in Behaviors and Service Utilization Among Beneficiaries Indicators 2017 2018 P-value Prevalence of exclusive breastfeeding of children under six months of age 70.6% 71.1% 0.86 Breastfeeding within one hour of birth 67.5% 69.3% 0.03 Minimum dietary diversity among children 6-23.9 (4+ of food group) 46.7% 53.5% 0.001 Minimum dietary diversity among mothers 35.6% 41.6% <0.000 Practice of sick children 6-23 months of age fed more during illness 38.5% 38.8% 0.75 Practice of sick child (diarrhea) 2 months or more given ORS and Zinc 22.6% 21.9% 0.71 Handwashing with soap and water at all 6 critical times always 7.8% 19.0% <0.000 6 USAID Multi-Sectoral Nutrition Strategy 2014-2025 Discussion Paper: Nutrition Scale-up: Learning from Experience. Accessed: https://www.usaid.gov/sites/default/files/documents/1864/Scaling-Up-Discussion-Paper-508.pdf. 7 Percentage of HH in target area practicing correct use of recommended HH water treatment technology 14.3% 19.0% <0.000 4+ ANC among mothers who received any 79.5% 85.5% <0.000 IFA tablets taken for 180 days during pregnancy 52.4% 59.1% <0.000 Use of a modern method of FP, among those who don’t want another child and whose husband has not migrated 62.9% 59.6% 0.05 Weight taken in most recent ANC, among mothers who received any 86.7% 93.4% <0.000 Percent of pregnant women weighed during most recent ANC visit 86.7% 93.4% 0.000 Percent of children under age 2 weighed in the past month 14.6% 22.2% 0.000 Percent of births attended by a skilled birth attendant 73.2% 77.3% 0.004 Percent of newborns receiving a postnatal health check within 24 hours of birth 73.5% 79.1% 0.000 Percent of births receiving at least 4 ANC visits during pregnancy 79.5% 85.5% 0.000 Percent of reproductive age women in union who are currently using a modern method of contraception 34.2% 33.2% 0.400 Data Source: Suaahara II Year Two Annual Report and 2017 and 2018 Annual Surveys Growth Monitoring and Promotion (GMP), an intervention package implemented nationally within the Ministry of Health and Population’s (MOHP’s) nutrition program for nearly two decades, is possibly the least effective intervention supported by Suaahara. Nationally GMP coverage is low and participation is poor in Suaahara implementation areas . Other national data confirm low utilization rates7. The GON promotes frequent health visits during the first two years of life, and Suaahara’s annual survey data do show a significant increase in children 0-2 years weighed in the past month. However, only 22 percent of the target group have been weighed within the last month, and of those whose weight was measured, only 35 percent of mothers reported being informed about their child’s growth. GMP has been found to be extremely challenging to improve (availability, quality as well as use) despite Suaahara’s efforts to support both demand and supply. Health managers and health workers in some of the sampled districts suggested that GMP, even when conducted, has not been effective and questioned whether this was the best approach to improve nutrition of children who are younger than five years old8. Nepal is consistent with global findings on the challenges to effective GMP. Given the complexity of Suaahara programming and multiple challenges related to successful GMP9, it may be more cost-effective for Suaahara to focus on nutrition assessment and counseling of children as a consistent part of all health service contacts per nutrition assessment counseling and support (NACS) guidelines. Counseling was not considered a nutrition intervention by many, but it’s changing. – Suaahara national level staff Suaahara rolled out GON-endorsed maternal infant and young child nutrition (MIYCN) and NACS intervention packages to train and mentor government health workers and FCHVs as well as Suaahara frontline workers [Field Supervisors (FS) and Community Nutrition Volunteers (CNVs)]. In addition, Suaahara provided need-based equipment and materials, technical support, and followed-up support. These have been integrated with existing maternal and child health programs of the government (e.g., CB-IMNCI, IMAM, safer motherhood). Although MIYCN and NACS training is an important step for improving frequency and quality of assessment, counseling and support, health workers largely have been 7 Ministry of Health and Population, Nepal; New ERA; UNICEF; EU; USAID; and CDC. 2018. Nepal National Micronutrient Status Survey, 2016. Kathmandu, Nepal: Ministry of Health and Population, Nepal. 8 In a sampled district where IMAM had been implemented for just over a year, only one-fourth of the UNICEF estimated cases of children with severe acute malnutrition (SAM) were found/enrolled in the first year of its implementation. A large proportion of these cases was found through other avenues of screening (e.g., children brought to a facility for treatment or vaccination, some focused screening campaigns, screening done by FCHVs using MUAC tape), and not through routine GMP sessions. 9 Ashworth A, Shrimpton R. Jamil K Review Article Growth monitoring and promotion: review of evidence of impact The Authors. Journal compilation © 2008 Blackwell Publishing Ltd. Maternal and Child Nutrition (2008), 4, pp. 86–117 8 reported doing it selectively only (anthropometric assessments and counseling done only in certain cases). Unlike most nutrition programs globally, Suaahara has demonstrated a good focus on maternal malnutrition too. This is evident from an increase in attendance of 4 ANC visits and improvement in nutrition elements of ANC (e.g., consumption of 180 IFA tablets, weighing, and counseling). On-site coaching and mentoring, an innovative approach developed and successfully tested by the program itself, has been found to be very effective in refreshing health workers’ knowledge and skills (e.g., on CB-IMNCI, IMAM and MIYCN, as attested by most stakeholders). This is demonstrably useful in assessing the knowledge and skills gaps amongst health service providers and providing tailored coaching at the facilities they work in. Lack of improvement/progress is noted in the treatment of diarrhea with ORS despite both supply and demand-side interventions. Almost half of the children with diarrhea seek treatment from the private medical sector (48 percent); about one third do not seek any advice or treatment; and of those who seek any advice or treatment, nearly three-fourths do so from the private sector (74 percent) (NDHS 2016). Since the program doesn’t engage with the private medical sector, it’s possibly a missed opportunity to improve ORS and zinc uptake among children with diarrhea. Despite the significant decrease in the total fertility rate over the last two decades, the contraceptive prevalence rate (CPR) stagnated over the last few years, and unmet need has remained high in Nepal (NDHS 2016). A similar trend has been seen in Suaahara areas, too, as shown by the first two annual HH surveys. Further analysis of annual HH survey data (2017 and 2018) showed no improvement in use of modern methods of FP among mothers with child under the age of two. However, there has been a reduction in FP use in 2018 compared to 2017 among disadvantaged groups (DAGs) and among mothers belonging to lowest two wealth quintiles. Similarly, further analysis of the second annual survey 2018 shows that CPR stood at 30 percent among the surveyed mothers with a child under two years of age which, though not directly comparable, is much lower than the national average for married women aged 15-49 years (43 percent, NDHS 2016). Health workers and Suaahara FLWs confirmed that among those who use a FP method, modern behavioral methods, such as the Lactational Amenorrhea Method (LAM), though included in the MIYCN and related guidelines, are not practiced. Practicing LAM10 Village Model Farmer Improves Nutrition Practices in Terai Community Sarala KC (name changed), a VMF selected and trained in Suaahara I, is a young mother with a four-year old and an eight-month old child. She formed a Homestead Food Production (HFP) group of twenty-four 1,000-day mothers. She and the FCHV organize and lead meetings in which she shares her knowledge about topics, such as the importance of a good diet during pregnancy and child feeding or demonstrates how to make a nutritious jaulo. She provides eggs, leafy vegetables, seeds, and saplings to the members, who all have poultry and kitchen gardens now. She received five chicks, vegetable seeds, and orange fleshed sweet potato (OFSP) vines from Suaahara. Her kitchen garden is now better managed. She has adopted drip irrigation, and she has a compost pile. Sarala KC grows different vegetables according to the Suaahara seasonal calendar, including three new items introduced by Suaahara (Kang Kong, Swiss chard, and OFSP), and sells the surplus to small vendors who come to buy or in a haat bazar. She earned 18,000 to 19,000 rupees last year, which she uses for HH expenses, including buying fruit and paying school fees for her child. Sarala received a 10-day poultry raising training one year ago. Her chickens are housed in a “semi-intensive coop,” which she promotes to other 1,000-day mothers, along with regular vaccination by the local agro-vet. This VMF and community have benefited from a partnership of Suaahara and KISAN II, which provides support for selling surplus production, training for moving toward more semi￾commercial efforts, and linking VMFs with agro-vets. 10 The three criteria for LAM effectiveness are: 1) The baby is “only/exclusively breastfed,” meaning the woman breastfeeds her baby day and night and does not give any other food, water or liquids (except for medicine, vitamins, or vaccines); 2) The woman’s menstrual bleeding has not returned since her baby was born; and 3) The baby is less than six months old. 9 enhances exclusive breastfeeding with its protective effects for babies and mothers and serves as a first step in family planning practice. 4.1.2 Access to Diverse and Nutrient-rich Foods In its first two years of implementation, Suaahara II has reached 114,504 vulnerable families11 to help make nutritious foods more accessible and promote diverse diets for 1,000-day mothers and children through nutrition-sensitive and nutrition-specific approaches. Suaahara uses HFP interventions and SBCC to improve production and consumption of diverse and nutrient-rich foods, primarily in DAG communities. Research documents that home garden ownership and production diversity in the home garden are positively associated with dietary diversity of HHs and women.12 Suaahara is scaling up state of the art approaches to nutrition-sensitive agriculture among intended beneficiaries, primarily women and children. Suaahara II is working with nearly 4,000 VMFs, including those trained during the first phase of Suaahara and still active, as well as additional ones in two new districts (Panchthar and Dhading). The 1,000-day households which received HFP inputs from VMFs and/or graduated HFP beneficiaries almost doubled between 2017 and 2018 (from 17.4% to 30%). Suaahara’s Annual Survey 2018 data reveals increased adoption of improved practices and increased production of diverse vegetables and eggs within vulnerable HHs, and that awareness of the benefits of production and consumption of key diverse foods for mothers and children has increased since 2017 among mothers and HH heads. However, there were challenges of chicken mortality due to lack of vaccination against New Castle and other diseases. A campaign to promote regular vaccination is planned, and linkages with agro-vets for regular vaccination services are being developed. Suaahara II has helped to increase business and marketing capacity among enhanced HFP households by linking some of them with private sector entities for additional capacity building and other supports (like market, loan, credit and subsidies). These inputs are likely to bear fruit during the remainder of the program in terms of both income and sustainability. In one example, from the 103 VMFs in a Terai district, 11 Local Resource Persons (LRP) have been trained - some in poultry rearing and some in vegetable production; they in turn are increasing their own production and service to their community. 4.1.3 Suaahara II’s Contribution to Closing Gaps Between Socio-economic Groups and to Changes in Social Norms Positive changes in certain behaviors among the DAG beneficiaries, as shown by Suaahara annual surveys, could, to some extent, suggest the narrowing of the gaps between socioeconomic groups. There has been improvement in some of the key behaviors among the DAG communities as evident in sampled districts. Improved behaviors mainly are in relation to dietary diversity, ANC visits, IFA tablets, and handwashing practices. These days we feed Lito (nutritious baby food), make Jaulo (porridge) putting Spinach and coriander. I add egg to it. But it is better to add yolk part later because if we put at first it might smell, and child might not eat. If we mash before feeding the child, they eat it. I add salt later only. – A 1,000-day mother from the DAG community, Terai district 11 The number indicated is only of the VMFs, HFPB groups, and HFP HHs reached; not the total beneficiaries. 12 Abu Hayat Md. Saiful Islam --- Patterns, determinants and food and nutrition security implications of home gardens in Bangladesh: Evidence from nationally representative household panel data. Presentation Ag2Nut Academy in India Jun 26, 2019 10 If a woman with low weight gives birth to a child it may be malnourished, low weight and ill. Therefore, we eat nutritious food, do ANC visit regularly, take iron tablets regularly. – A 1,000-day mother from DAG community, Terai district The above-discussed findings are to a large extent consistent with the information gathered from service providers in DAG wards. In the last 3 years pregnant women take regular ANC checkups. They (1,000-day mothers) feed nutritious food to their children and themselves. They also have started drinking purified water and consuming green and diverse vegetables. – A FCHV from the DAG community, Hill/Mountain district. The results are broadly supported by the work of other studies in the area13. However, there still exist serious issues that need to be addressed such as gender discrimination within certain communities: There is gender discrimination. If a mother gives birth to a son, she is properly cared after and is provided with proper meals, but if a mother gives birth to a girl child she is not provided with proper meals. There are malnourished children. Tendency of giving birth to many children has made it difficult to provide nutrition-rich foods in Muslim families. Due to weak economic status it is difficult to provide nutrition-rich foods. – A 1,000-day mother, Terai district Further analysis carried out using the data obtained from AHH (2017) and AHH (2018) could be useful in gaining insights into the differences between wealth (highest and lowest quintiles) and social groups in terms of adopting some of Suaahara-related behaviors. Many behaviors, for example, egg consumption among mothers and children, attendance of at least four ANC visits, and meat consumption among children have largely improved across equity quintiles, caste/ethnic groups, and agro ecological zones. The most encouraging reduction among them is seen in egg consumption among both mothers and children. Similarly, remarkable improvement in hand washing practices and decreased disparities in the hand washing can be seen across ethnic groups (see Appendix 3 for more details). Dietary diversity (DD) among children between 6-23 months of age had increased in 2018 compared to 2017 (from 47 percent to 54 percent). As with child DD, mothers DD also increased from 35 percent in 2017 to 42 percent in 2018. However, further analysis of the 2018 (single year) annual survey data shows that there are still big gaps between DAG and non-DAG communities in some behaviors; for example, minimum dietary diversity of child aged 6-23 months for DAG community was significantly different from non-DAG (47.4 percent for the DAG and 57.8 percent for non-DAG; p-value: <0.0001). Although there has been improvement in consumption of meat, eggs, and iron-rich food among Dalits and disadvantaged Janajatis, much less improvement is seen among HHs belonging to the lowest equity quintiles (See Appendix 3). As per the Suaahara II team, their DAG population in their CMC data matches 76 percent with that based on the old (GON) criteria. This means one-fourth of the disadvantaged populations are not receiving core plus inputs from the program. Also as outlined above, the secondary analysis of data shows a relatively positive picture in relation to closing gaps between socioeconomic groups, which is indicative of the fact that the blanket approach (DAG vs. non-DAG areas) is working relatively well to equitably and consistently improve the Suaahara related behaviors and service use among those at the lower wealth quintiles and the socially excluded groups. Social Norms Although social norms are not readily shaped or changed as evident from several studies14, Suaahara appears to have contributed to some extent towards bringing gradual changes to this end. There are 13 Cunningham, K., Singh, A., Pandey Rana, P., Brye, L., Alayon, S., Lapping, K., ... & Klemm, R. D. (2017). Suaahara in Nepal: An at‐scale, multi‐sectoral nutrition program influences knowledge and practices while enhancing equity. Maternal & child nutrition, 13(4), e12415. 14 Lessig, L. (1995). Social meaning and social norms. U. Pa. L. Rev., 144, 2181. 11 instances of gradual changes in social norms due to Suaahara interventions. One frequently encountered example was the changes in behavior among the people belonging to the Brahmin community in terms of raising chickens. Many of them (people belonging to Brahmin community) have started raising chicken in their home for the children even though they don’t eat meat. – APNGO official, Hill/Mountain district. Likewise, other norms appeared to be gradually changing, such as lactating mothers in some study locations not being allowed to have salt for several days following the delivery of the child. “The social norms are also changing after Suaahara interventions. Before, the lactating mothers were not allowed to consume salt for some days. They even had to hide themselves in a dark room for a few days (stay in isolation). At the time of giving birth they did not come to hospital. Now, they can consume salt, eat nutritious food, they don’t have to hide, and they come to hospital to give birth. – A government official, Terai district There are, however, practices that still exist, such as mobility restriction among married women, particularly in Terai program districts. Such social norms are beyond the program’s control but have significant bearing on program interventions. Additionally, beliefs that taking iron tablets or eating eggs during pregnancy will make the baby grow large in the womb resulting in difficulties during delivery are still prevalent in some of the program districts15 Suaahara has focused on GESI as a key cross-cutting area. The program has trained/oriented its PNGO staffs and board members on GESI and ensured a GESI perspective in its activities. Its approach and activities are targeted at reducing gender-related barriers for 1,000-day mothers to care for them and their children. For example, Suaahara has taken a whole family approach, rather than a mother/child dyad focus, and attempted to engage men in HH work and other non-traditional roles. Male GESI champions have been trained in how to change thinking. Expansion of HFP is a viable strategy to achieve GESI objectives in Nepal as it both empowers women and reaches marginalized groups with opportunities to increase incomes while expanding the availability of nutritious foods. Technologies to reduce women’s work burden are being introduced, including drip irrigation and water harvesting for kitchen gardens and women-friendly agriculture equipment. Availability of nutrient-dense foods is coupled with behavior change messaging for consumption of diverse diets, which is resulting in changing dietary norms [e.g., “harek baar, khaana chaar (every time, four food￾groups)”]. Before (HFP activities) we were eating, but we didn’t know about nutrition. – A VMF, Terai district Women themselves are involved in kitchen gardening and chicken rearing, which have increased their decision-making capability regarding nutrition. – A 1,000-day mother, Terai district. 4.1.4 Implementation of Integrated Nutrition Programming Approach Suaahara is an integrated nutrition program, implementing a mix of proven nutrition-specific interventions (e.g., optimal infant and young child feeding, IFA supplementation, with strategically selected nutrition￾sensitive interventions, including HFP, WASH, and promotion of healthy timing and spacing of pregnancy (HTSP), in the same communities and households). Multi-sectoral programming is essential to address the many direct and underlying factors that affect nutrition status. This design is aligned with both the USAID 15 This information was provided by 1000 day mothers during one focus groups in one hill study district. 12 Multi-Sectoral Nutrition Strategy (MSNS) 2014-2025 and the GON Multi-Sector Nutrition Plan II (MSNP II) 2018-2022. The 2017 NIL Annual Report illustrates the need for integrated programming. Its findings show a statistically significant association between cropping and livestock diversity (the range of items produced) and dietary diversity of children in Nepal. The effects are most strong for poorer HHs and children above two years of age, less strong for younger children in the poorest HHs only, and no correlation for infants 6-18 months. Expecting an impact during the first 1,000 days from any one intervention alone may, therefore, be misplaced and require other complementary targeted nutrition interventions.16 In Suaahara, integration has been rightly considered at different aspects of programming. The program leadership fosters integration of interventions/sectors within work plans and approaches. For example, Bhanchhin Aama radio program and job aides integrate different subjects in materials; NACS is integrated into health services with other MCH services (e.g. FP, ANC, CB-IMNCI, MIYCN); HFP groups cover nutritious food consumption and other nutrition messages; HMGs include messages on WASH, dietary diversity, FP, and 1,000-day nutrition-specific messages; income is generated through savings and credit activities; FCHVs are frequently trained as VMFs, and Suaahara has involved them as key partners for the HFP beneficiary groups, seeking to link them with HMGs. FCHVs support VMFs in delivering key behavior change related messages and promote HFP during key community events, such as food demonstrations and important day celebrations. All Suaahara staff are co-located, both at the central and the district levels, to foster more collaboration and cross-discipline thinking. This is elaborated on further under Question 5 below. The program also has integrated a GESI strategy and principles into its programming, which has led to positive impact in terms of reducing the gaps between socioeconomic groups and changing social norms, as discussed in the section above. 4.1.5 Continuum of Care and Life Cycle Approach. Suaahara II’s interventions are focused on critical phases in the life cycle: pregnancy, newborns, post￾partum, and children under two years of age – the most nutritionally vulnerable stages in the life cycle. The importance of maternal nutrition factors in the life cycle is coming to the fore globally, and Suaahara is promoting good maternal nutrition to break the cycle of malnutrition. Analysis of national data in Nepal, document that maternal factors, including maternal height and education, were generally the strongest individual‐level risk factors for stunting.17 Findings from recent work in Nepal18 suggest that pre-conception (adolescence), pregnancy and early postpartum, represent windows of opportunity for tackling child wasting, not only stunting. Rates of weight gain in pregnancy are associated with infant weight for age, length for age, and weight for length and are predictive of postnatal growth at six months of age. The study examines factors linked with low mid-upper arm circumference (MUAC) in pregnant women in Banke Nepal.19 Suaahara has recently added a new initiative to improve nutrition and health among adolescent girls – another significant stage in the life cycle. Weekly IFA supplementation is ongoing for girls in 14 districts. 16Mulmi P, Masters WA, Ghosh S, Namirembe G, Rajbhandary R, Manohar S, et al. (2017) Household food production is positively associated with dietary diversity and intake of nutrient-dense foods for older preschool children in poorer families: Results from a nationally-representative survey in Nepal. PLoS ONE 12(11): e0186765. https://doi.org/10.1371/journal.pone.0186765 17 Jamie L. Dorsey, et. Al, Individual, HH, and community level risk factors of stunting in children younger than five years old: Findings from a national surveillance system in Nepal. Maternal and Child Nutrition Volume14, Issue1January 2018 18 Nutrition Innovation Lab 2017 Annual Report. 19 https://www.nutritioninnovationlab.org/publication/factors-associated-mid-upper-arm-circumference-pregnant-women-banke￾nepal 13 In one of the observation districts, IFA distribution in most schools is being implemented satisfactorily according to Suaahara field staff. However, schools have faced challenges, which were documented, discussed in a review meeting, and plans have been made to rectify the situation. GON IFA distribution guidelines are being updated based on the initial experience, including how to manage the distribution during school vacation, or an absence, and how best to reach out of schoolgirls. In addition, Suaahara, through its integrated adolescent nutrition package, also works with boys in four districts. This package addresses water purification, handwashing, sanitation, and menstrual hygiene management, and other relevant prevention topics. For Suaahara, the continuum of care starts at the household with the Community Mapping Census (CMC) by frontline workers (FLWs) and home visits, including behavior change promotion and interpersonal counseling as well as nutrition assessment and appropriate referrals to health facilities and outreach clinics. After appropriate care, including counseling at the health facility, the mother/child returns to the community. However, follow up and support at home is a weak link. The FCHV may not be fully informed about health/nutrition condition and treatment, and so is unprepared to follow up in the community. For example, loss to follow up/defaulters are very common among children enrolled in the IMAM program, as evident in some of the sampled districts, due to lack of follow up and support at home. Though screening happens at households, community outreaches and all health facilities, diagnosis and provision of ready-to-use-therapeutic-food (RUTF) is available at limited outpatient therapeutic centers (OTCs) only. A main reason for parents’ not going for follow up and for replenishment of RUTF is because of the distance to OTC. While it may not be cost-effective to increase the number of OTCs, alternative measures need to be considered. Directly addressing the causes of high levels of acute malnutrition and defaulters in certain communities is essential to reducing this serious health risk among young children. The IMAM focal person at an outpatient therapeutic center (OTC) in a Terai district where levels of acute malnutrition are high, when asked about the causes of SAM and MAM in community, responded that mothers have to go to the field to work even at one month post-partum, leaving her baby with other kids, and “she also has to take care of the family.” The new mother’s obligation to return to the field affects breastfeeding (unless other HH members bring the infant to her) and limits regular follow up visits to the OTC, often aggravating the child’s condition. Another continuum of care issue that stands out as needing more attention is immediate and exclusive breastfeeding in facility births. Immediate and exclusive breastfeeding saves lives and helps prevent all forms of malnutrition. Suaahara II is supporting the promotion of breastfeeding in ANC visits and in the community, but there doesn’t seem to be focused support for immediate and exclusive breastfeeding in births that occur in facilities. Recent data20 on immediate breastfeeding indicate the need for focus on breastfeeding support and encouragement at delivery. The Suaahara 2018 annual survey also shows that only 69 percent of children 0-23 months were breastfed within one hour of birth (an increase from 67.5 percent in 2017). This is a missed opportunity as facility-based births are increasing (three in four births - NDHS 2016), and health workers attending these births can be trained and supervised to support and encourage newly delivered mothers to successfully initiate breastfeeding and avoid pre-lacteal feeds. 4.1.6 Utilization of Existing Platforms Suaahara is providing support to the GON in integrating nutrition services to exiting service delivery platform,s such as HMG meetings, health facilities, outreach clinics, schools, local markets, VMF-led HFP groups, and agriculture service centers. Although utilization has not been to the desired extent, Suaahara has been attempting to promote a GMP platform. There is need for continuous focus on quality of counseling and service delivery, including ensuring integration of nutrition education and counseling services through MCH/FP clinics, birthing centers, immunization outreach clinics, and other existing venues. 20 Nepal National Micronutrient Survey 2016 14 HMGs: Most beneficiaries noted HMGs being used as a platform by Suaahara to improve knowledge on health and nutrition among the 1,000-day mothers. In our community, HMG meetings are being held on the 20th of every month. In this meeting we talk about nutrition and sanitation. This meeting is organized by FCHV. Sometimes the sisters (FLWs) from Suaahara attend this meeting too. – A 1,000-day mother, Hill/Mountain district We conduct meetings on the 24th of every month. We discuss on health and hygiene related issues. In HMG meetings they teach us how to make Jaulo, using grains and pulses. The discussions in the meeting are mainly focused on health and nutrition. – A 1,000-day mother, Terai district Although, there has been an increase in FCHV facilitated HMGs in the community between 2017 and 2018 (Suaahara Annual Reports 2017 and 2018), there are still areas that can be improved. These areas mainly include lack of time to attend and inconvenient times of the meetings. The Suaahara Mid-term Assessment done in 2018 found that lack of incentives/snacks to the mothers is a constraint to participation. It was also evident in this evaluation. There was a provision of snacks in mothers group meeting before. Now 1,000-day mothers complain for not giving snacks during any meetings. There should be provision of some snacks. They don’t show any interest in meetings because of the snacks issue. – A FLW, Hill/Mountain district Though HMG meetings are mainly intended for 1,000-day mothers, the members are not always of this category. HMG members mostly elderly do not want to retire because savings is involved. An attempt is being made by Suaahara to include 1,000-day mothers as members of HMGs. It is not the aama samuha but hajur aama samuha (not 1,000-day mothers’ but grandmothers’ groups). – A FCHV, Hill/Mountain district Another challenge associated with HMG meeting participation concerns religion. But some women from Muslim community and lower castes they don’t like to come to HMG meetings, and they don’t participate in other activities. That’s why they don’t know about nutrition. – A 1,000-day mother, Terai district To regularize the HMG meetings, a specific mapping tool as part of Self-Applied Technique for Quality Health (SATH) is applied in the selected HMGs in the Core+ areas (poor vulnerable and socially vulnerable and/or marginalized group). SATH is intended to help for the discussion on essential behaviors and motivate the mothers to meet regularly. GON: Government institutions including ward/municipalities and health facilities also are being used as important platforms by Suaahara to reach its intended beneficiaries; for example, the availability and quality of services, such as ANC/post-natal care (PNC) check-up, IMNCI, and nutrition delivered through the health system, has been improved with Suaahara support. “Earlier they (mothers) did not remember which vaccine to be given, when to take for weighing the child, but now they talk about it all the time in HMG, while meeting FCHV. They are taking their children to health post regularly.” – A HH member, Hill/Mountain. Similarly, a few municipalities have decided to allocate or at least are planning to allocate the budget for nutrition and health (discussed in detail in relevant section). 15 We have planned to allocate separate budget for their nutrition and development of economic status on the budget of 2019. If we are able to encourage those who have 1-2 katthas of land for growing vegetables and nutritious food on their own, it would help to improve the nutrition status of the mother and child. – A government official, Terai district Local Market: Over the last decade, vegetable farming in Nepal has gained importance as it is not done just for consumption by the family, but also for income-generating purposes.21 Suaahara has been utilizing local markets to build linkages between the beneficiaries and local businesses mainly on agriculture and WASH components. With the help of Suaahara staff we are coordinating with local buyers (businessmen) and selling vegetables (potatoes, cauliflower, cabbage, tomatoes, etc.) ourselves. It is not difficult to do this as the buyer directly visits our village. That way we have direct coordination in between sellers (village farmers) and the buyer(s). – A VMF, Hill/Mountain district Likewise, Suaahara has been facilitating the linkages between local businesses that deal with the WASH products and its beneficiaries. Suaahara has motivated private business sectors to keep WASH materials. Process of WASH mart is going on. Some of the shops sell WASH materials upon the request of ours. Suaahara has been doing social marketing; there is a need of further motivation. – A Suaahara official, Hill/Mountain district Agriculture Services: The next platform that emerged from the evaluation was government agriculture and livestock offices. The government offices have been providing technical assistance to the beneficiaries. (We) facilitate in the course of training provided to VMF of Suaahara program. Active VMF takes necessary suggestion to coordinate with agriculture…In case of necessity in the work field of Suaahara, we facilitate to provide training on kitchen gardening in the community. – A government official, Hill/Mountain district. “There are model farmers who are doing vegetable production. Others are also doing vegetables in their kitchen garden; some sell the leftovers from their household consumption. Suaahara has definitely played an important role, last year they had distributed vegetable seeds in 3 lots, at that time our role was to provide technical assistance; people from livestock also did similar kind of coordination. We selected areas for them so that the programs would not duplicate.” – A government official, Terai district. Suaahara II also is leveraging local level resources (block grants) to provide mini-hatcheries, vaccines, and gender-friendly agriculture tools (e.g., mini power tiller, thresher, and grinder) that contribute to increased production for VMFs and HFP groups. Suaahara links beneficiaries with private suppliers for a range of agriculture inputs, including seeds, chicks and poultry care supplies, vaccines for chickens, materials for irrigation and farming tools and with markets. These result in options for VMFs and HFP beneficiaries to sustain and expand their horticulture and poultry production. At the time of purchasing seeds, JTA (Junior Technical Assistants for agriculture) from nearby village suggested to me how to grow vegetables in good amount on a small farm and how we can take care of our homestead production. – 1,000-day mother, Hill/Mountain district VMFs and HFP beneficiary groups were created by Suaahara since its first phase as a platform to promote the availability of diverse and nutrient-rich foods locally. The program is now trying to sustain them by helping them register with the local government and linking them with local markets. Technology: Suaahara has recently started utilizing short message service (SMS) to spread the health and nutrition-related messages across its intended beneficiaries. 21 Rai, M. K., Paudel, B., Zhang, Y., Khanal, N. R., Nepal, P., & Koirala, H. L. (2019). Vegetable farming and farmers’ livelihood: Insights from Kathmandu Valley, Nepal. Sustainability, 11(3), 889. 16 Some of us are using SMS service. We are getting messages about nutritious foods, ways to take care of our baby when they get sick from diarrhea. Only the person who connects to the server can get messages in their mobile. It’s especially for 1,000- day mothers. – A 1,000-day mother, Hill/Mountain district Nevertheless, the messages sent through mobile phones often aren’t received, mainly due to the frequency of individuals switching sim cards. We hardly pay attention to messages send through mobile phones. If we change the sim cards, we get better offer than topping up. We don’t get the messages because, once we change the sim card, the phone number changes. – A 1,000-day mother, Hill/Mountain district Additionally, Suaahara has been using Frequency Modulation (FM) radio and social media, particularly to air the program called Bhanchin Aama. The program also is posted on social media websites, such as YouTube. Use of media is often considered as an effective way of disseminating nutrition messages. Despite increase in listenership, it appears that it has not been utilized to the desired extent. There is a potential to increase the listenership of the Bhanchin Aama radio program. 4.1.7 Beneficiaries’ Understanding of Suaahara Suaahara is well known and appreciated by GON officials and stakeholders in the areas where it is implemented. Most beneficiaries understand Suaahara as “Harek Baar Khana Chaar” (every time, four food groups) and handwashing. Different stakeholders relate Suaahara to one or the other behavior promoted by the program. For example, 1,000-day mothers: egg consumption, food consumption, nutritious food, food diversity, distribution of chicks; FCHVs: to prevent malnutrition; and Health providers: quality training. However, there is a lack of clarity among Suaahara field staff and government staff regarding the eligibility for intensive interventions among beneficiaries. More than us the farmers who are appointed as VMF (mostly Dalit or marginalized ones) have a problem, they get agriculture input tools which is not possible to be distributed to all the villagers. It becomes an issue of envy among the villagers. The VMFs use to produce in a large scale commercially and they have access to the market and have good income. To this the villagers make an issue saying they took all the support and used for their own personal good only. – A GON official, Terai district 4.1.8 Knowledge and Skills of Suaahara FLWS and FCHVs Suaahara is well known for providing quality training. The program carries out a wide variety of training programs for frontline workers and higher-level staff, mainly through Training of Trainers (TOT). Much of the training is conducted as in-service training; however, the program has begun to implement onsite coaching. Suaahara training sessions and materials are considered as high quality and, most importantly, those who have received training from Suaahara generally feel competent to provide nutrition-related services for which they have been trained. Furthermore, Suaahara’s job aids are valued and well-utilized in health facilities, home visits, and community meetings by Suaahara FLWs and FCHVs. There is good participation of and collaboration among local officials in conducting Suaahara-supported training: FCHVs are provided with training on different topics: IMAM, CB-IMNCI, FP, etc. The training is delivered by the government officials at district level. – A health worker, Terai district The program has worked with the GON and development partners at the central level to integrate NACS into different national programs; for example, IMAM, MIYCN, CB-IMNCI, GMP and basic FCHV training package – an important step for sustainability. Job aids have been developed to guide health providers in conducting NACS with mothers and children. Essential anthropometric equipment have been provided to PHCCs, HPs, and referral hospitals to fill gaps, which were identified in a survey of all health 17 facilities at the start of Suaahara II, and to enable a basic nutritional assessment. Although direct observation of FCHVs or health providers using NACS skills was not possible, the building blocks were found to be in place for enhanced nutritional services. Suaahara’s FLWs are considered active and well-qualified, knowledgeable about their work, tools, and activities, and trusted by communities. Program officials are confident of the capacity and knowledge of FLWs: Given their role, they (FLWs) are adequately trained, and they know the details of the program. – A PNGO official, Hill/Mountain district In certain instances, Suaahara FLWs reported not having received adequate training for the job. For example, some FLWs with a health or other non-agriculture background feel they lack adequate agriculture knowledge to fully support VMFs. FCHVs rely heavily on Suaahara FLWs for nutrition-related duties. As such, FLWs work closely with FCHVs to improve the latter’s knowledge and skills as some FCHVs are not yet confident in their ability to fulfill nutrition￾related responsibilities, such as taking measurements using MUAC tape. 4.2 Evaluation Area 2: Multi-sectoral and Multi-level Stakeholder Coordination, Collaboration, and Engagement Evaluation Question 2: To what extent have GON counterparts, other USAID implementing partners, and other stakeholders, at national and sub-national levels and across sectors, been engaged in program planning, implementation, and monitoring for Suaahara II? 4.2.1 Rollout of the Multi-Sector Nutrition Plan (MSNP-II) “Suaahara is a very strong partner in the scene of Nutrition in Nepal.” – A national-level stakeholder Suaahara has positioned itself well as a key and trusted nutrition partner, both at the national and sub￾national levels. The program has successfully engaged government at federal and local levels. In year two of the program, there was a major change in the external politico-administrative environment. With the promulgation of the Constitution in 2015, Nepal formally adopted federalism, subsequently restructuring the state into a federal government, seven provincial governments, and 753 local governments (293 urban municipalities and 460 rural municipalities). The local governments now have the authority and are solely tasked with delivering basic services, including health and nutrition services, while functions, such as setting national-level goals and standards, are under the jurisdiction of the federal government. Therefore, Suaahara’s main point of coordination in delivering the program changed from 40 District Health offices to 389 municipalities (urban and rural) in 42 (two Suaahara districts split) program districts, which led to a major shift in the way the program engaged with the government. Suaahara rapidly responded to the shift, adapting well without a major disruption to the program activities or outputs. I think Suaahara 2 is working effectively in changing structure. They are doing coordination with local government as well as with other sector also. – A ward chair, Rural Municipality, Hill/Mountain District Federalism has made it even easier. It has helped SUAAHARA to work in the community with the support of elected representatives. – A health official, Terai District USAID’s MSNS 2014–2025 aims to “scale up effective, integrated, nutrition-specific and nutrition-sensitive interventions, programs, and systems across humanitarian and development contexts.” The MTE finds 18 that Suaahara II is consistently implementing a strong multi-sector program well aligned with priorities outlined in the MSNS. Suaahara coordinates and works with the National Planning Commission (NPC, the apex body for the development of MSNP II) and relevant sector ministries. Suaahara is helping to fill gaps in Nepal in implementation of the MSNP II. They advocate within MSNP framework and have brought in key players and are seen as Influential in country in keeping nutrition on the agenda, high in policymakers view. - NIL leadership Suaahara collaborates closely with NPC, the Ministries of Health and Agriculture, and, to some extent, the Ministries of Education and Water. Other partners engage more closely with the Ministry of Federal Affairs and General Administration (MOFAGA), but Suaahara II is not required to do so since they are not an implementing body. According to some stakeholders, this skewed engagement with certain sectors/sector ministries only can result in isolation and lack of ownership of the program by some government counterparts. There is considerable engagement and support of local governments – municipalities (denoting both urban and rural) and wards – in planning, implementing, and monitoring the multisector nutrition program. There is evidence of consultations by USAID with the national level government in developing Suaahara II, a follow-on of Suaahara I. Suaahara has been a consistent presence in nutrition-related activities The program has made earnest efforts to collaborate with relevant USAID programs – notably with KISAN-II (KISAN and Suaahara overlap in 23 districts) and the Strengthening Systems for Better Health (SSBH) project (Suaahara and SSBH overlap in 8 districts) – and identified some joint areas of working and mutual support. Suaahara also works closely with UNICEF, a key partner in multi-sector nutrition, bilaterally as well as through multi-sector platforms. This has resulted in joint working and minimized duplication, to the extent possible, in the 14 districts where both organizations overlap. The program has explored potential partnerships with the private sector, with a couple of good examples of centrally brokered partnerships in place, particularly for agriculture and WASH. However, private sector engagement and partnership is little understood, particularly at the implementation level; there are no formalized partnerships currently in any of the sampled districts. 4.2.2 Convening, Coordinating, and Facilitating the Implementation of MSNP Suaahara II was closely involved in the development of the second national MSNP II. The support extended by the program, primarily in analyzing and providing local and global evidence, has been highly recognized by the government and other partners. Suaahara II has been active in the national multi-sector nutrition fora, working closely with the government and other partners. However, as noted above, apart from the health and agriculture ministries and the planning commission, coordination with others is inadequate. Though a key development partner supporting MSNP II development and rollout, Suaahara II is criticised for focusing narrowly on certain MSNP 2 components only. Suaahara II is not seen as a “full MSNP program” by few other partners. Whether a partner supporting the implementation of MSNP II should be covering all sectors and the full MSNP package in a given geography is a key question to be pondered. Government counterparts stress to partners to implement the full MSNP package. Suaahara was by design limited to only certain components of the MSNP II, the argument being that it would not be possible to maintain the scale it has now without spreading the resources too thinly and risking poor quality of delivery. Partners working on or supporting MSNP II are few. Given the range of sectors it 19 involves, it may not be possible for a partner to implement or support the whole package alone. Some stakeholders suggested that it will be judicious if multiple partners worked together in a given geographic area to provide full package of services. Suaahara has been closely involved with the NPC and provincial government in establishing MSNP structures and orienting key stakeholders at the province level. The program is playing an increasingly active role with the Social Development Ministry in furthering the MSNP planning and implementation, as evident in at least one province. Among the three levels, the role of provincial level for the basic service delivery is the least clear. The program may want to limit their role to building bridges between the municipalities and provincial authorities to ensure increasing allocation of provincial grants on multi￾sector nutrition activities implemented by municipalities. The crux of Suaahara’s role in facilitating the implementation of MSNP II has been at the local level. With the formation of three levels of government, the program shifted their engagement from the districts to the municipalities and wards. There is strong evidence of intensive engagement with elected and technical officials and frontline workers, including FCHVs, at the local levels, thanks to the network of FS and CNVs. Suaahara has helped establish MSNP coordination structures [e.g., the Nutrition and Food Security Steering Committee (NFSSC) and the WASH Coordination Committee (WASHCC)] at local levels in many districts; however, continued support is lacking. Many of these committees are not yet fully functional as evident in the sampled municipalities and wards. Those at the district levels do not see the importance of coordination as their role in implementation is nearly non-existent. At the municipality level, poor leadership of the MSNP agenda and lack of resources to conduct meetings have been cited as major causes. Unless there is partner’s support, these committees do not meet regularly. Suaahara relies primarily on their one-to-one engagement with the stakeholders for the coordination and collaboration that is needed to deliver the program effectively. Cooperation (support for each other’s program/activities) between Suaahara and local governments was evident in the sampled districts. Local leaders and officials are highly appreciative of technical support provided by Suaahara on planning and budget disbursement. Some of the Suaahara innovations/interventions have been replicated with local government funding (e.g., key life events, food demos, printing copies of letters to my father, and MUAC screenings). Suaahara teams also are equally appreciative of the support and involvement of local leaders and officials in implementing and monitoring program activities. For example, they go on joint monitoring visits; and local and district level technical officers (health, livestock, agriculture) support as resource persons in many of the program’s activities. Continued engagement with and support by Suaahara II has resulted in local elected officials being increasingly sensitized on the value of nutrition and health. Many local leaders and officials attested to this, and this is also reflected in the gradual prioritization (over infrastructure) and increasing allocation of funding by local bodies to multi-sector nutrition activities. 4.2.3 Linkage and Collaboration with Other Relevant USAID Activities and Other Nutrition Partners As a part of this evaluation, officials of other USAID-funded programs were interviewed, mainly to explore their collaboration with Suaahara II. In a few instances, the collaboration between HKI and other USAID funded programs can be noted. For example, SSBH in Surkhet has been conducting its activities jointly with the Suaahara II team as needed. SSBH, working in systems strengthening, including quality of care, and Suaahara II, working at the community level, complement each other. They hold meetings regularly in Karnali Province with all related USAID programs, including Suaahara II, to exchange information on progress. 20 We work together when needed. I think the collaboration is quite good. We hold a meeting at least once monthly and the intention is to avoid duplication of activities. We are new and Suaahara has been here quite long. Their experience has helped us to do the budgetary planning. - A SSBH staff, Hill/Mountain district However, it is quite interesting to note that household-level data that Suaahara II collects as a part of their monitoring has not been shared yet among other USAID funded programs. The data can be helpful particularly for planning for activities. HH level data that Suaahara collection has not been shared to us. Dissemination of such data would be helpful in planning for activities. This will help us to avoid duplication of activities. – A SSBH staff, Hill/Mountain district KISAN II and Suaahara II are both contributing to USAID Multisector Nutrition Strategy and Feed the Future objectives in Nepal. KISAN II aims to increase resilience, inclusiveness, and sustainability of income growth in the Feed the Future zone of influence; their target is 200,000 farming HHs in the zone of influence. KISAN II and Suaahara II overlap in 23 districts. One of their objectives meshes with Suaahara’s objectives: Enable vulnerable communities to act on business opportunities within selected market systems. There are several examples of collaboration between the Suaahara team and the KISAN II team. One is the utilization of the latter’s platform to raise awareness about nutrition and health among the 1,000-day mothers. In one of the villages (where the majority belong to Badi and Dalit communities), in some meetings organized by KISAN, Suaahara had instilled awareness on nutrition. Health service seeking behavior among the 1,000-day mothers there has improved. – A Suaahara district staff, Terai district KISAN II field staffs invite Suaahara VMFs to orient KISAN farmer groups on the importance of the “1000 golden days”, and kitchen gardens for improving dietary diversity. During group formation by private sector, KISAN contacts field supervisors of Suaahara, who provide detailed information about progressive VMFs and some leader farmers (Suaahara households producing a surplus) that can be linked to KISAN private partners (e.g. agrovets). Suaahara and KISAN II are also collaborating to link VMFs and HFP groups with local markets and introduce new technologies, such as drip irrigation. Work planning can be done jointly (rather than reviewing work plans for potential areas of overlap after the fact). It’s not just about coordination, should move toward common goal in collaboration. Both programs are at an ideal stage now so both can take advantage of expertise of the other. – A KISAN II regional staff Although there is no “true collaboration” between Suaahara and other USAID funded programs, there are instances of collaboration as and when required. It also is important to gain an understanding of the collaboration between Suaahara and UNICEF, both of whom are working in similar programs in Nepal. It appears that although there is a close working relationship between Suaahara and UNICEF, there still exist areas for continued development. For example, overlapping of 14 districts could have been avoided if UNICEF was consulted at the design stage. Overlapping could have been completely avoided had we been consulted at the design stage of Suaahara. - A national-level stakeholder It also appears that coordination with other implementing agencies can be improved, particularly to avoid duplications of activities. It should be noted that the UNICEF’s working modality is significantly different from Suaahara. UNICEF 21 funds into the Red Book, which is channeled through MOFAGA. Unlike Suaahara, there is no district and community level presence as UNICEF relies on GON for planning, implementing, and reporting. 4.2.4 Private Sector Engagement “Promote increased, responsible private sector engagement in targeted countries to encourage the production and consumption of nutritious and safe foods and harness the expertise of the private sector to shape healthy consumption patterns.” – A USAID MSNS Private sector engagement and partnership with development programs is an increasing priority for USAID. Generally, private sector engagement seems to be little understood, particularly at the implementation level. Even at the central level, the understanding is varied – with concepts ranging from simple identification of and linkages to local suppliers and buyers to a meaningful partnership with shared objectives between the program/beneficiaries and the private sector to even a public-private-partnership. The central Suaahara II program team has dedicated considerable time and resources to explore strategic interests of private sector actors, their goals for corporate social responsibility and potential business models that match Suaahara target groups and activities, with a particularly focus on agriculture/HFP and WASH. They have engaged with a multitude of private-sector agencies exploring and testing potential partnership options with shared objectives. The program has been successful in establishing some meaningful partnerships. For instance, in a few districts, a memorandum of understanding was signed with Shreenagar Agro Company, a large-scale agribusiness value chain company, and partnership is ongoing to benefit VMF and local poultry farmers in a couple of districts by supplying poultry production inputs and guaranteeing a buyback of ready to sell supplies. Although this is a good example of a mutually beneficial partnership between a major private sector player and program beneficiaries, the private sector partner is not interested in expanding this to more districts, possibly due to high costs and limited profit opportunities. A true public-private partnership – between the Ministry of Agriculture and Livestock Development (MoALD), Nepal Poultry Federation and Suaahara II – was tried in 2018 for an egg promotion campaign but did not materialize. Recently, MOALD has endorsed the egg campaign, and the Suaahara II team is putting together an execution plan. There is evidence of efforts made by some district program teams to broker private sector engagement to make agriculture or WASH inputs available and/or affordable with a view of promoting adoption of healthy behaviors. For example, some have negotiated a discounted rate for water filters of a particular brand, and some have advocated and facilitated establishments of WASH marts so that WASH supplies are available in remote villages with small markets. However, no private sector partnership has been formalized yet in any of the sampled districts. Suaahara II also has utilized its partnership with KISAN-II to facilitate private sector engagement at a limited scale. For example, there have been initiatives to link HFPBs with private input suppliers and vendors to ensure both the availability of required inputs locally as well as the market to sell their produce. Beneficiary women who want to sell their surplus production seem to welcome such linkages. The program could focus more on KISAN-II’s model of small companies or vendors working with small stakeholders and developing trust and beneficial relationships. We are coordinating with local buyers and selling vegetables (potatoes, cauliflower, cabbage, tomatoes, etc.) ourselves. It is not difficult to do this as the buyer directly visits our village. That way we have direct coordination between sellers (village farmers) and the buyer(s). – A VMF, hill district. 22 Engaging with private sector in social sectors/development programming isn’t well understood generally, and there are few examples of such partnerships in development programs in Nepal. …Some have the concept that private engagement is a failure of government. – A national stakeholder More awareness of private sector potential for expanding and sustaining development work is needed. Suaahara has the potential and scope to define the benefits of private-sector partnerships through small to medium scale private sector engagement that it has facilitated or is in the process of negotiating. Forging a private sector partnership in a context where both private businesses and development work are done conventionally can be time-intensive, as experienced by the program. It’s clear that private sector’s primary motivation is profit, so it’s hard to engage them for a joint cause/objective unless profitability is assured. Suaahara and other development partners may consider allocating some dedicated resources to promote private partnerships (e.g., providing seed money or subsidizing certain costs to attract private companies to join hands). Private companies engaged in all facets of food systems play an important role in ensuring food safety. For instance, a NIL study in Banke found exposure of aflatoxin is widespread (94 percent) among pregnant women22, a serious food hazard risk. Commercial entities involved in the management of food supplies at all levels need to be involved in preventing this dangerous toxin from entering the food system. Opportunities for private sector engagement in the health sector are abundant. For example, private pharmacies can sell ORS with therapeutic zinc to expand community availability. Sale of drinking water treatment methods is a natural role for small enterprises. 4.3 Evaluation Area 3: Capacity Strengthening and Sustainability Approach Evaluation Question 3: To what extent and how is Suaahara II programming contributing to improved capacity, ownership, and motivation to implement multi-sectoral nutrition and health programming within the GON at different levels? 4.3.1 Contribution to Multi-sectoral Nutrition and Health Programming National-level GON officials emphasize that Suaahara II should increase focus on “ownership” and “governance” at local levels for MSNP. The program has done a lot at the local level, but Suaahara may need to look at opportunities to strengthen national capacity, mainly around its role to orient, facilitate, coordinate, and provide an enabling environment for local governments to implement MSNP. Suaahara’s programming is centered around support and facilitation of capacity development within Nepal institutions. Section 4.1 describes efforts to build technical and programmatic leadership for implementation of quality nutrition interventions. Suaahara has played an important role in building the capacity of elected leaders, officials and frontline workers (e.g., health, agriculture, WASH) at local levels in areas related to multi-sector nutrition and health. For example, nearly all health workers and FCHVs have been trained on the MIYCN package via a full training course, refreshers, and/or onsite coaching and mentoring. A large number of elected and administrative officials of local governments, members of different coordinating committees, and technical coordinators have been oriented on Suaahara as well as the broader multi-sector nutrition and development agenda. This coupled with continued engagement and support has resulted in more sensitized elected officials and gradually increasing allocation to multi￾sector nutrition and health programming by the local governments. The elected local government officials are excited about improving nutrition for their constituents, especially the female officials, and seemed to be well acquainted with Suaahara FLWs. 22 NIL 2017 Annual Report 23 Suaahara II has provided technical support to local governments in identifying issues, local solutions, and planning and budgeting for multi-sector nutrition and health planning. In districts where government has other funds for MSNP programming (e.g., from UNICEF through MOFAGA), Suaahara II has provided technical assistance on-demand to help them plan and implement. Suaahara II also has helped orient officials and stakeholders at the provincial level on multi-sector nutrition and health programming. The result of this, in terms of increased multi-sector nutrition and health allocations by provinces to the local governments, is yet to be seen. There is evidently good sense of ownership of Suaahara II among the local government officials, but still knowledge on MSNP as a concept and national program is lacking. I feel Suaahara is our own program… – An elected local government official, Hill/Mountain district In general, elected local government officials seem to have the intention, authority, resources, and political incentive to do more for women and children and disadvantaged communities, but lack the capacity to plan and implement, and, hence, rely heavily on Suaahara. For instance, most of the health coordinators of the municipalities have clinical experience only, with little exposure to planning, budgeting and managing public health and nutrition programs. In the context of massive transition in the civil service (re-adjustment process), the majority of the health coordinators are likely to be new, with a real danger of losing the institutional memory, capacity and gains made. Suaahara needs to consider building their capacity for nutrition and MSNP as they are critical to planning and implementing nutrition-specific activities. In most of the sampled districts, onsite coaching and mentoring was done independently by Suaahara district technical staff (MNCH or NSBCC Officers) without engaging government health managers, supervisors and coordinators (from district or municipality level offices). This was necessitated by a lack of resources to pay government officials to travel to facilities. In one of the sampled districts, however, the Provincial Health Office (previously DHO) – learning from the Suaahara model – had used some of their unallocated provincial funds to have government health managers, supervisors and coordinators in conjunction with Suaahara II district technical staff deliver onsite coaching and mentoring to health workers. Provincial health officials were highly impressed with the effectiveness of the approach. It’s a missed opportunity that Suaahara has not been able to ensure government staff engagement (as mentors) in most of their program districts. Suaahara II needs to strategically scale-up and expand their onsite coaching and mentoring approach, involving government health managers and workers to ensure the capacity of health workers and FCHVs are maintained and sustained in the long run. Through Suaahara II advocacy, local governments and provincial health offices are likely to allocate funding to cover some of the costs of mentoring and coaching by government staff, if Suaahara is unable to fund them fully. Including MIYCN and WASH content in the GON pre- and in-service curricula for frontline health workers is important for building capacity and sustainability of nutrition improvements once Suaahara has concluded. Despite being in the program proposal (Updated Program Description), not much has been done in this area, except for including NACS in the FCHV national training curriculum. The remaining years of the program might not be enough to ensure change in curricula. However, Suaahara could engage and advocate with the Ministry of Education (MOE) and other relevant ministries and aid their efforts to update curricula. 24 4.3.2 Transitioning Technical and Programmatic leadership to Nepal institutions [GON and Civil Society Organizations (CSOs)] As MSNP-II drives nutrition-sensitive and nutrition-specific programs, it is essential for USAID and the GON to orient, build capacity, and support local governments in generating a sense of ownership of the MSNP-II. At the local level, health coordinators are the key to improve technical and programmatic leadership, but most lack the capacity for planning, budgeting, and management. They should be empowered by giving authority and enhancing their managerial capacity through training and orientations. Additionally, there is a need to inform all levels of government officials about their responsibility in implementing MSNP-II and follow up. The government should engage more actively in the roll-out of MSNP-II and orient provincial-level officials and advocate for them to work at the local level. This will align with Suaahara’s upcoming approach to focus on sustainability and strengthen local government in uptake of integrated nutrition programs. Also, expediting the re-adjustment process is necessary so that local government could be striding towards being technically self-sufficient, which is lacking at present. On the other hand, USAID and Suaahara II should consider categorizing districts for phased graduation. For instance, those districts that are active for a long time and with low levels of malnutrition could be considered for phased graduation and instead focus on existing and newer districts that have high level of malnutrition. As one respondent said, “‘True Karnali’ must be the focus of Suaahara.” The local NGO partners should be viewed beyond their implementation role. Their presence at the district and local levels is an asset that can play a crucial role in the sustainability of activities. Even when PNGO staff leave when their programs end, their board members carry on the program’s institutional memory and legacy23. Most of the PNGO board members in the sampled districts were found to be well￾known and socially influential people, who could be nutrition champions in their communities. There have been steps taken from PNGOs to build their capacity through involvement with Suaahara. They have developed or upgraded their GESI, HR, and administrative policies, which shows their intention to progress. Suaahara’s FLWs are considered active and well qualified, knowledgeable about their work, tools, and activities, and trusted by communities as multi-sector nutrition service providers, agents for change and effective community mobilizers. Program officials are confident of the capacity and knowledge of the field supervisors. Apart from districts where UNICEF hires Nutrition officers and facilitators through their ‘Poshan ko Laagi Hatemalo’ program24, municipalities and wards do not have the technical staff or skilled focal points for multi-sector nutrition. Existing FSs and CNVs are recognized assets in their communities who will exist beyond the program and could sustain program gains should they continue in those roles. These cadres are valuable resources in their communities for future nutrition programming. Part of the closeout planning for Suaahara II must be to negotiate hiring CNVs and FS for local government roles. 4.3.3 Technical Assistance (TA) for Planning, Implementation, and Monitoring of the MSNP at the Local Levels Suaahara II has helped form NFSSC and WASHCC at different levels and helped orient a large number of elected and administrative officials of local governments on Suaahara as well as the broader multi-sector nutrition and development agenda. Local government leaders are increasingly sensitized, and there is strong evidence that they are gradually prioritizing nutrition and health in their plans and budget. For instance, 27 PNGOs, in their response to the online survey, stated that out of 252 municipalities that they currently work under Suaahara II, 84 percent of these municipalities had allocated budget and made 23 Some PNGOs were found to have used their knowledge, skills, learning and successful approaches and interventions from their previous work / programs into their current work / programs. 24 Multi-sector nutrition program supported by UNICEF 25 expenditures under nutrition activities in 2018/19. Local governments also have accepted and adopted some of the Suaahara interventions. However, these are skewed towards subsidies and distributions [e.g., top-up resources for pregnant women, such as for institutional delivery and 4 ANCs, celebrating key life events, and some for innovations (such as “letter to my father,” replicating food demos), but very little towards capacity and systems strengthening (e.g., quality training, onsite coaching and mentoring, policy development, improving quality of data collection and use, etc.), except for an encouraging trend of replicating models, such as SATH and Community Health Score Board (CHSB), which help improve quality and utilization of services]. The selection of the adopted interventions or products by municipalities and wards have been random generally. As per Suaahara’s reporting, through intensive advocacy efforts, newly elected local GON officials allocated more than 8 million USD to nutrition-related activities in their 2018-19 plan (Suaahara II Year 2 Annual Report, 2018). Suaahara also is helping HFP groups and VMFs to register within municipalities to facilitate technical support, local grants, and subsidies. Among 3,904 groups, 808 groups are already registered at municipal offices to access additional trainings, loans, and other services provided to women farmers from MoAD. Some of the support that they received were for mini-hatcheries, Newcastle disease vaccines and gender-friendly agriculture tools (e.g., mini power tiller, thresher, grinder) that contribute to increased production for VMFs and HFP groups. 4.3.4 Challenges Suaahara II Faces in Implementing Multi-sector Nutrition Interventions The MSNP II being a complex approach, there are a few, but potent challenges that Suaahara has been facing. Three main challenges that emerged included coordination with government officials at multiple levels, lack of capacity among the municipal level government officials, and sustainability. Coordination challenge: Before the change in structure, Suaahara staff, including the FLWs, did not have as many government officials to deal with. Thus, coordination was less time intensive. Now, coordination requires interacting with multiple local governments, which involve both time and cost implications. There was confusion among the Suaahara staff regarding with whom (government officials) they should coordinate. The coordination with the government officials has become very challenging due to the change in structure. Before we used to coordinate with DHO and other sectors (at district level) but now we have to coordinate with more offices (government), municipalities and rural municipalities. It is really time consuming and hard to manage. – A Suaahara official, Hill/Mountain district It is not only the Suaahara staff, but also the government officials themselves who appear to be confused about their roles and responsibilities. We ourselves are not clear about our roles. I used to have good working relationships with Suaahara before but now they don’t approach us; neither we are interested. – A government district official, Hill/Mountain district Suaahara, despite, the challenge discussed above, is making attempts to adapt to the context. Suaahara is coordinating with local government very well, that’s why they are working effectively in the changed structure as well. The coordination part of Suaahara is very good. – A government official, Hill/Mountain district Lack of capacity among local government officials to deliver nutrition and health-related services: With resources at hand, the decision-making power now rests with the local government. The resources before the change in structure used to be channeled through the district, which was in the position to make decisions. One grave concern, expressed mostly by the district and provincial levels officials, was the lack of capacity among officials at the local level to properly use resources allocated to 26 them. Available studies also indicate inadequate numbers of civil servants, many of whom have low technical skill, as one of the major constraints of change in structure.25 Health related program shouldn’t be given directly to local government because many of the local leaders haven’t understood the health and nutrition so it should be taken care by the health sector only. It was easier before in district level. Now health has gone to local government and has become difficult for us to work with local government because they don’t understand what we want.”– A government official, Terai district. 4.4 Evaluation Area 4: Monitoring, Evaluation, Research, and Learning Evaluation Question 4: How are Suaahara II’s monitoring, evaluation, and research systems and activities contributing to data-driven program implementation? Monitoring, evaluation, and research (MER) is a cross-cutting area robustly focused on by the program – much more significant in scope, scale, and intensity than has been seen in nutrition and health programs of this size in Nepal. The breadth and depth of data generated by the program is not possible to obtain from the national information systems, except for limited population level and service coverage data. Hence, Suaahara has established program-specific information management systems. This is also in response to the lessons learned from the first phase of the program. The MER team at the central office is larger than other technical (thematic) teams (Reference: See the Organizational Structure/Organogram given in Suaahara II Year Three Work Plan).The MER systems of the program are technology-driven (e.g., uses the mobile platforms CommCare and DHIS2) as well as human resource-intensive (e.g., for community monitoring census (CMC) and monthly monitoring checklist, large proportion of the staff time of FSs, CNVs, data management and documentation officers (DMDOs), and district coordinators is spent in data collection and/or its supervision/quality assurance; Yes, I am involved in collecting data for the project. At first, we have to visit every household. It is very difficult to visit every household because of geographical barriers. This is hilly region and houses are scattered here and there. Whenever we go to visit household, most of mother are busy in their work and hesitate to give time and on other hand it takes 2 hours with 1,000-day mother household, as we have 5 checklists to be filled. We have to observe, whether the toilet is clean or not, whether there is soap available or not etc. – A Suaahara FLW, Hill/Mountain district Large annual household surveys, covering more than 3,500 HHs, and other research (e.g., formative) are done through contracted research agencies. The Suaahara II internal Mid-Term Assessment: Qualitative Research Using the Program Constraints Assessment (2018) was conducted by an external team to gather perceptions on Suaahara accomplishments, constraints inhibiting program effectiveness, and suggestions to overcome these constraints. The assessment also sought to capture beneficiaries’ perceptions on primary benefits from Suaahara interventions and their greatest remaining problems. A final impact evaluation is planned for the end of Suaahara II, which will cover Suaahara programming from 2011-2021. Suaahara also has conducted multiple rounds of formative research to refine and improve the effectiveness of its interventions. The information and knowledge generated by Suaahara is used very well by the program to continuously learn and improve as well as to create global public goods. There are several examples in the sampled districts of how data generated by the program has been used to make changes to the program approach 25 Bajracharya, P., & Grace, C. (2014). The Nepal Civil Service and Re-structuring of the State. Option Paper produced for the Project to Prepare the Public Administration for State Reforms (PREPARE) jointly conducted by the Government of Nepal Ministry of General Administration and the United Nations Development Program. 27 or plans. For example, the sampling of the HH for monthly monitoring checklist, which are done during a home visit based on the data from the CMC; added focus in the HMGs on behaviors that are found hard to change in that particular district; design of the adolescent initiative based on the finding that almost all adolescent girls are in school and not many of the younger ones have phones (making mobile-based approaches not viable). Similarly, the program has responded to the evidence from the monitoring system to make appropriate and aggregate changes. Examples include the introduction of CNVs in response to the workload anticipated from the CMC data collection and to increase coverage of Suaahara through HH visits and interpersonal counseling; the introduction of the SMS intervention, which wouldn’t have been possible without the CMC generating phone numbers; the move to add Facebook and YouTube as additional ways of the Bhanchhin Aama radio program reaching HHs given annual survey data showing the prevalence of smartphones among beneficiaries and decreasing trend of radio use; and the approach of private sector partnership with Baltra for establishing WASH marts across 500 outlets was based on the results of willingness to pay for filters and other WASH items by the HHs. Suaahara is one of USAID’s first and largest multi-sector nutrition programs globally, and there is a great amount of attention and interest from USAID as well as the global nutrition community in its implementation and impact. Suaahara harnesses science and data to inform cutting-edge approaches in nutrition. The lessons learned, successes, and results of this program have the potential to shape integrated nutrition programming in other countries too. Apart from the rich data from the census, annual surveys, and monthly monitoring, Suaahara also is focusing on research in partnership with other global institutions and universities. An adolescent longitudinal cohort study and a randomized control trial for use of SMS reminder messages to HHs with young children are being undertaken by Suaahara as a part of learning. Global sharing of results and learning from the program (webinars, seminars, workshops, presentations, and publications) and through collaboration with universities has been a key feature of the program’s plan to use data as global goods. 4.4.1 Robustness of MER System Suaahara II’s MER system builds upon the learning from the first phase of Suaahara and sits at the core of generating evidence for programmatic improvement. Except for a few, minor issues, the MER system was found to be systemic and of adequate quality in terms of coverage, measurement, and the generation of evidence to improve program activities. There is clarity regarding recording and reporting processes across all levels of program implementation. For instance, across the sampled districts, there are standardized reporting and recording formats available which are used with a consistent understanding of the reporting process and the timeline of reporting. However, some data capturing formats and protocols were received late by a few PNGOs. Also, a few of them had confusion in understanding and capturing some data elements and instructions. The routine monitoring activities are tied to monthly activities that would mainly cover input, process, and output level indicators, based on monthly targets from the approved work-plans. There is a system for ensuring targets are reviewed and tallied against achievements through monthly, bimonthly, and quarterly meetings involving Suaahara FLWs, PNGO staffs, and Suaahara district staffs. Such practice was consistent throughout the visited districts. However, there was irregularity in maintaining meeting minutes, decisions, and discussions made during the meetings. Additionally, PNGOs’ monitoring staff reported that they didn’t have any manuals or guidelines to maneuver and assist data collection, reporting, and recording processes. The field level staffs needed to contact focal persons if any issue arises that might lead to data inconsistency, besides increasing the unnecessary workload to focal persons. There was a gap in circulating information regarding the use of newly introduced tools and processes that occur yearly. The DMDO in one district, who is responsible for district-level reporting, didn’t know 28 about the IMAM activity, who the beneficiaries were, what the role of health staff and their staff was, and how to record and report the data. There was inconsistent verbal guidance from Suaahara regarding the collection and management of data. However, DMDOs shared that the decision channel has been clarified recently, which has made it easier to seek and obtain information on issues surrounding data recording and reporting. One of the good practices observed was the responsiveness of Suaahara to undertake proper modification of MER processes and tools to enhance efficiency and effectiveness. For instance, the number of tools to be filled by FLWs was reduced in year 2 in response to feedback from FLWs and district teams (including PNGOs) and based on the usability of the information obtained from tools. Use of information and data obtained from routine monitoring, evaluation, and research activities was found to be systematic and built into the programmatic decision-making practice to some extent. The central teams who represent various sectors (such as WASH, the private sector, SBCC) are well versed with the results, and most of them mentioned the use of the results to make programmatic changes. The MER team also was receptive of the programmatic need to undertake changes in their tools to capture the information to fulfill needs of the program. Such a collaborative approach has been successful in meeting programmatic needs and use of the MER system to provide evidence and information. For example, engagement with the private sector partner, Baltra, for promoting water filters was based on willingness to pay information from the HH survey. On the contrary, Suaahara II’s annual survey, due to its design to be powered across intervention areas, does not capture intensive program vs. non-intensive areas to allow disaggregation with sufficient precision. The random sampling only includes six WASH intensive wards in its 85 total wards. One of the program managers at Suaahara was vocal about this issue stating that this sample was inadequate to show the effectiveness of the WASH intensive intervention. This highlights a challenge for Suaahara, which has blanketed program approaches (Core) and other targeted approaches (Core+), and even sub-areas within Core+ for some interventions. To power sufficiently to track trends for both areas would nearly double the sample size (and required resources). Some mobile devices related hardware issues have been hindering recording and reporting processes, such as overheating, limited storage, and short battery life of mobile devices. Respondents reported that these issues not only delay the recording of data but also hamper their regular programmatic activities. Including this, the mobiles allocated for CNVs, who were responsible for filling more forms in CommCare, are old and have low storage capacity in comparison to those used by FS. The MER system captures multi-sectoral HH-level information that is needed to have the holistic nature and at the same time simple enough so that community-level staff can use it. A separate and focused review of the MER system conducted in 2017 also commended Suaahara II’s M&E system by stating it is “creative, robust, and innovative.” Thus, despite a few issues, Suaahara II’s M&E system is adequately organized and maintained to measure progress of the program. 4.4.2 Data Sharing Data is not systematically shared at local and national levels for use by MSNP implementing partners. Citing the ethical need to protect the identities of the beneficiaries, raw data has not been shared with in￾country partners and stakeholders. Suaahara shares analyzed data at different levels, presented verbally at various meetings and fora, or through sharing of program reports; however, there is no format or consistent approach across the districts or levels of implementation. There is anecdotal evidence of data with identifiers (e.g., of 1,000-day mothers) being shared at some local levels to enable the local government (municipality or ward) to, for example, distribute benefits to certain target populations (such as, pregnant women). One can surmise that the potential of Suaahara data to influence national and local MSNP activities beyond those implemented by the program itself has hardly been realized. 29 It’s a lost opportunity that a program, all four key results areas of which are focused on using existing systems and platforms and strengthening the capacities of governments and people, has by design left out any capacity strengthening activities for or collaboration with the government (all levels) on such a key, cross-cutting area. For example, there is little evidence that the program has helped strengthen HMIS or data management broadly at the provincial and local levels; more so in a context where federal transition has negatively affected the HMIS data flow and quality. There is room to collaborate with HMIS in cooperation with SSBH to improve HMIS nutrition indicator reporting. Suaahara II is active in Nepal and globally to share results and learnings from the program (webinars, seminars, workshops, presentations and publications) and through collaboration with universities. For example, Suaahara staff regularly deliver presentations on their work at the annual Scientific Symposium sponsored by the NIL; participants include GON officials, academics, students, and researchers from Nepal and many other countries. Suaahara staff participated in several sessions of the 2018 Food, Diets, and Nutrition: 25 Years of Progress in Nepal 6th Annual Scientific Symposium on Agriculture - Nutrition Pathways and 25 Years of Nepal’s Progress in Nutrition November 2018, Kathmandu, Nepal. (Leadership of the Suaahara MER system) is thoughtful and rigorous in approach; have set the bar high. Suaahara is seen as consistently involved. Suaahara I and II have done a better job from a learning perspective - have upped the bar… About learning and sharing. – A NIL official. In addition, two workshops were offered to participants on “Agriculture Interventions: Challenges, Implementation, and Tools Applied for Monitoring in USAID Funded Suaahara Program” and “Integrating Family Planning in Agriculture and Nutrition Programming: Experiences from Suaahara II.” On May 9, 2019, Suaahara was featured in a global webinar sponsored by the USG Global Nutrition Coordination Plan’s Implementation Science sub-group. It was an interactive learning exchange with practitioners and researchers on what an implementation science approach is, and Suaahara was presented as the real￾world example of how global nutrition programs have engaged with and applied implementation science. Suaahara staff also presented Suaahara findings at the Nutrition and Nurture in Infancy and Childhood conference in England in 2019. Suaahara staff frequently publish articles in peer-reviewed journals on aspects of their work and lessons learned; collaboration with masters and doctoral students globally make this possible and simultaneously provide networking and capacity building opportunities for Suaahara staff. For example, an article published in 2019 used Suaahara I data to show WASH as a linkage in the association between women’s empowerment and child’s nutritional outcome (Cunnigham et al., 201926). Kathmandu and field-based Suaahara II staff are currently engaged in using Suaahara II data to help fill local and global knowledge gaps by writing manuscripts focused on association between exposure to Suaahara II and key outcomes, GMOP, HMGs, parent depression and household diets, health facility readiness for serviced delivery, and integrated programming for adolescents in school. 4.4.3 Demonstrating Innovation Suaahara II uses mobile technology for routine monitoring of program activities and implementation. Field level staff who reach out to beneficiaries use the CommCare platform to record data and facilitate their visit and counseling. With monthly HH visits and reporting of 1,000-day mother household information, 26 Cunningham K, Ferguson E, Ruel M, et al. Water, sanitation, and hygiene practices mediate the association between women's empowerment and child length‐for‐age z‐scores in Nepal. Matern Child Nutr. 2019;15: e12638. https://doi.org/10.1111/mcn.12638 30 Suaahara II’s learning approach captures vital programmatic information. This information is available simultaneously to field staff via their mobile phones for reference. It was demonstrated several times when field coordinators checked their database on the spot to respond to a programmatic question. Suaahara has several innovations/interventions, which are exciting and frequently replicated with local government funding. These, to name a few examples, include celebrations of key life events and “a letter to my father.” The “letter to my father,” based on the belief men want to be good fathers and have healthy families, has been a sensation with men throughout Suaahara II’s target districts. The introduction of orange-fleshed sweet potato as a new crop has not only brought an easy-to-grow, nutritious alternative for homestead production, but the vegetable has become a popular item in markets. 4.4.4 Alignment with the Global Nutrition Agenda USAID MSNP urges programming to “Strengthen the evidence base for and scale up (1) proven nutrition￾sensitive agriculture interventions and (2) NACS as a component of routine clinical health care.” Suaahara is doing both. Suaahara’s HFP interventions are examples of best practices on innovative approaches to expanding agricultural production, increasing rural incomes and the poor’s access to nutritious food, and increasing inclusion in commercial markets. Suaahara has adopted NACS as a package to ensure the basic elements of nutrition services become part of routine health/nutrition services. NACS was originally developed for HIV/AIDS nutrition services but has been taken up in a few cases to strengthen nutrition care for chronic diseases and maternal and child health. Suaahara is one of the first large scale programs to train health providers on NACS as a component of routine clinical health care. This aspect of programming will be of immense importance to the global nutrition community to learn whether and how NACS can strengthen the quality of nutrition services in the health care setting. Suaahara’s adolescent initiative is filling a gap in the global community with respect to formative research, strategic design, and the scaling up of adolescent programs. Suaahara should document the full IFA supplementation intervention including the evolution of planning, implementation, monitoring, learning, and quality improvement processes in which the program is engaging. The health and nutrition education curriculum experience piloted in four districts will be of interest to the global community as will the findings from the adolescent longitudinal panel and affiliated study on adolescent aspirations led by a researcher from Wageningen University. It is suggested that USAID/Nepal sponsor a seminar or conference in collaboration with the MOE and MOHP on the results of Suaahara’s contribution to the Revitalized School Health and Nutrition Program, which is currently under revision by a committee that includes USAID and Suaahara. Suaahara’s adolescent initiative promises to be a rich source of evidence to guide other similar programs. The USAID Multi-Sectoral Nutrition Strategy 2018 Periodic Assessment is a participatory and forward￾looking process to monitor the status of MSNS implementation in USAID programming and country context, monitor MSNS implementation in countries (2022, 2025), and engage stakeholders and inform learning around MSNS implementation (2018, 2022, 2025). The assessment gathered information on several domains and factors in the implementation of the MSNS. Eleven countries participated in the 2018 assessment, including Nepal. Some of the findings were: • Number of MSNS nutrition-specific services by country: Nepal = seven, one of the highest in USAID country programs. 31 • Presence of Nutrition-sensitive programming: Number of MSNS nutrition-sensitive programming types by country: Nepal = six, at the top of all the USAID MSNS nutrition-sensitive programming. (These include: economic strengthening, livelihoods, and social protection; family planning, healthy timing, and spacing of pregnancy; food safety and food processing; girls’ and women’s education; water, sanitation, hygiene; and nutrition-sensitive agriculture.) The USAID MSNS Monitoring and Learning (M&L) Plan has recently been launched. It will track the progress of the implementation of the Agency's strategy. Building on existing monitoring and learning systems, the M&L Plan includes an approach and tools to conduct periodic assessments. These assessments, scheduled for 2018, 2022, and 2025, will monitor nutrition outcomes and reach across select countries to examine the effect and utility of a multi-sectoral strategy on nutrition programming and results. After each assessment, findings will be incorporated into action plans to continue and improve the implementation of the Agency's MSNS. Suaahara II’s robust MER system will provide an enormous amount of high-quality data for the first and second assessments, which will impact all of USAID’s global nutrition learning. The global nutrition community needs documented experiences in the scale-up of multi-sectoral nutrition programs. Scale-up can be defined as a process of expanding nutrition interventions with proven efficacy to more people over a wider geographic area that maintains high levels of quality, equity, and sustainability through multi-sectoral involvement. The multi-sectoral nature of nutrition and the need for coordination, in addition to the inconsistency in definitions of scale-up and approaches to it, highlight the importance of sharing experiences and what has been learned. Suaahara, through its rigorous documentation, is facilitating learning and experience sharing on large scale-ups. 4.4.5 Evaluation Area 5: Program Management Evaluation Question 5: How effective are the current program management structure and operations to implement this complex multi-sectoral program? The program management structure is well set up, given the number of entities involved in implementation. The program has been successful in demonstrating itself as one entity: Suaahara. There were a few issues, however, that were identified, including that the implementation was affected by budgetary constraints which required a “slow down” of certain activities and cancelation of others in year 2, with spillover effects in the following years. Nonetheless, the program management has been effective in delivering such complex program in the rapidly changing socio-political context. 4.4.6 Program Management Structure, Coherence, and Relationship The joint program management and operations structure and the visibility of Suaahara team as one entity, as opposed to multiple agencies implementing Suaahara, have been an advantage for coherence and uniformity in delivery (planning and implementation) both at the center and district levels. These have helped brand the program and are critical to the recognition it enjoys. There is an overarching “Consortium Review Committee,” represented by the senior management of the program and the head of each consortium partner (or designated representative), which reviews the program progress and discusses key issues. Some consortium partners feel that major decisions are made by a couple of “big partners” without consultation with all partners. Inclusive and participatory decision is important that will support to get better result both at central and community level. – A central-level Suaahara II personnel 32 The Program Advisory Committee, comprised of all relevant government ministries and USAID is in place to galvanize support for Suaahara, ensure linkages with GON programs, share lessons learned, and increase the ownership and sustainability of program activities. In the districts, the Suaahara district team works as one, though the staff are administratively under different consortium partners. There are some grievances about the different remuneration and benefits package for the same level of job as each consortium partner has its own benefits packages and terms and conditions for its employees. Despite structures joined up horizontally at the Centre and district level, the joining up is less pronounced vertically between the Centre and the district teams. On paper the planning is bottom-up with districts preparing a detailed implementation plans (DIP) within the parameters set by the Centre. However, in practice, the most district teams and PNGOs in the sampled districts expressed that it is strongly top-down planning. Everything is standard and districts have little flexibility to propose anything contextual or innovative. Anything novel from the district is always cut by the Centre in the final plans. – A Suaahara district official, Hill/Mountain district 4.4.7 Building Local NGOs Capacity In general, PNGOs were found to have positive working relationships with the consortium partners. Most of the PNGOs in the sampled districts had been engaged since Suaahara I, and with their continuation in Suaahara II, there has been positive growth of PNGOs. The PNGO survey conducted as a part of the MTE revealed that PNGOs had timely received support from Suaahara, including technical assistance for coaching and mentoring for field-level staff, onsite coaching to FCHVs and health workers, MSNP-NFSSC formation, and capacitating. Various thematic personnel from Suaahara were supportive in the program implementation, such as for the implementation of activities (e.g., SATH, CHSB, key life events), documentation, M&E, planning, and coordination with local bodies and stakeholders. PNOs have observed enhancements in the skills of field-level staff who work continuously with Suaahara staff. …we have received more knowledge on WASH, nutrition and mobile use due to multi-sectoral nature of the program. We have flip chart, zinc card, wheel card, IVR to counsel the mother. These trainings have increased our confidence to work in the community. All the trainings received are adequate and effective. – Suaahara FLWs, Hill district. There has been capacity building support to PNGOs that are directly related with the program implementation from Suaahara. Capacity building was mainly focused in providing training and orientation on GESI to board members and staff of PNGOs. As a result, PNGOs have been able to formulate or update GESI policies and practice it. …Suaahara has given organizational level GESI orientation so it has supported to develop organization level GESI policies… – A PNGO official, Terai district Capacity building support also covered administrative improvement, compliance, and overall organizational enhancement, such as improved financial and administration management, strengthened organizational portfolio in nutrition work, strengthened organizational policies, and improved HR management. Some PNGOs even shared that working with Suaahara has helped them to improve the visibility and recognition of their organization in the district and supported them to improve their capacity to work at the community level. 33 …engagement with Suaahara has supported to increase the capacity of our organization working in Nutrition related work in community level. – A PNGO official, Hill/Mountain district None of the PNGOs, by design, receive reimbursement for overhead or management costs while the lead and consortium partners do. This has evidently affected PNGO morale and their ability to finance internal improvements to their organizations. In the remainder of the program, PNGOs expect to enhance their leadership capacity, more capacity development activities (such as, data management and documentation/analysis/interpretation training and data use training, program management training, and a computer skills training) that would help overall organizational development. In addition, some PNGOs were seeking to enhance report/proposal writing skills of their organization, which, according to them, would support the sustainability of their organization. However, the capacity needs and gaps are varying, and all may not be within the scope of the program to support. 4.4.8 Potential Areas for Improvement An internal platform for sharing amongst the field staff could be useful not only in improving learning but also for adapting and scaling up of the local solutions and innovations that some districts might have discovered. It is prudent to timely address any operational and management issues (e.g., delayed disbursement to a sub-awardee by the prime or to PNGOs by a partner), especially when it is affecting the implementation of activities. A shared understanding and uniform approach in managing crisis are important. For example, during the “slow down” of activities in year 2, due to a two- to three-month delay in the budget release, not all partners were fully onboard regarding which activities should be slowed down, moved forward, or dropped altogether. Some partners perceived that it disproportionately affected them for reasons not clear to them. The field level Suaahara and PNGO staff are stretched due to the new federal structure as they now must coordinate with 389 municipalities and over 3000 wards. The addition of CNV as a cadre has helped free some time for FS to focus on local coordination. However, the district level structures that were designed to coordinate with district-level stakeholders and line agencies have now almost ten times more local government entities to coordinate with. Moreover, the Suaahara district structure is gradually tapered with program teams being pulled out of districts gradually, with one team responsible for more than one district. Suaahara needs to carefully consider how districts teams are pulled out of the districts with minimal disruption to the coordination and influence it has at the district and local levels 5 RECOMMENDATIONS Recommendations for Suaahara II are grouped into three areas: Programmatic, Technical and Partnership. Recommendations for future USAID/Nepal nutrition programming are included. 5.1 Programmatic: Scale, Quality and Sustainability 1. Scale down and continue with additional focus in certain areas to make higher impact in the neediest areas. Focus on gradually reducing scale (breadth as well as depth – geographic and interventions) while continuing with sharper targeting of disadvantaged populations and doing more on strengthening capacity, improving quality and sustainability. For example, 34 a. Continue non-budgetary activities27 in all areas, but re-focus budgetary activities in disadvantaged areas and population groups identified together with the local government and stakeholders (not the usual blanket approach - DAG vs non-DAG areas based on decades￾old GON criteria, but find focus on the disadvantaged within each and every ward). b. Provide technical assistance to local government in systematically identifying the disadvantaged groups, be open about resources Suaahara can provide, and what local government should contribute. c. Use the final program period to work continuously with municipality and ward level NFSSCs to identify and target disadvantaged groups in all areas – both current DAG and non-DAG – sharpening the focus on GESI. Targeted services should also be delivered in partnership with Provincial / District technical leaders. II. Keep capacity strengthening central to maintain gains and improve quality. This may not always require expensive training courses but continue providing individual and group coaching and mentoring. a. Advise and support different tiers of government to ensure a ‘culture’ of quality training, which Suaahara is well known for. b. Focus on training smaller numbers and building the capacity of trainers, always including someone local. c. Help GON to create a repository of training materials and also to prepare and keep records of staff trained in nutrition (through Suaahara and other programs too). d. Continue working on and, if possible, scale up in needier areas onsite coaching and mentoring to end-user health workers, working with other partners where they exist. e. Ensure government staff engagement (as mentors) in onsite coaching and mentoring, and advocate with local and provincial governments to allocate funding to cover all or part of the cost of engaging them. This will enable building a local pool of trainers and mentors – consisting of health managers, supervisors and coordinators, as well as peers from other health facilities in the municipality (in-charges, health workers) who demonstrate leadership and good knowledge and skills. f. Dedicate additional resources and time to engage the Health Coordinators coming in after the civil servants’ readjustment process and build their capacity through individual or group coaching, mentoring. This will also apply to new or transferred in health workers who are not aware of Suaahara or skilled in nutrition. This will help avert the risks of losing the institutional memory, capacity and gains made. III. Continue and re-strategize engagement with local government to enhance the sustainability of resources, critical activities, and systems for improved nutritional outcomes. The strength for nutrition action is in local bodies. The engagement and ownership of the Suaahara program and agenda by the local government, as evident in all sampled districts, present a huge opportunity to engrain nutrition as a top development agenda locally wherever the program operates. For example, a. Come up with a clear menu of (say 10) options -- from among Suaahara interventions, innovations, products, policies, strategies, activities, cadres such as Field Supervisors and Community Nutrition Volunteers, etc. -- that seem to be effective, or have the potential, to 27 These are activities at community and household levels which require FLWs’ time but no separate budget to conduct. E.g. home visits, follow-up, data collection, HMG meetings, etc. 35 improve quality, strengthen systems and promote sustainability. For example: the increasing trend of replicating models such as SATH and CHSB need to be continued. b. Make strategic efforts across all districts and municipalities to advocate their adoption in local government plans and budgets. c. Strongly advocate to create Nutrition focal person positions, such as FS and CNV of Suaahara (who have been well recognized at local levels as multi-sector nutrition service providers, agents for change and effective community mobilizers), through local government funding for multi-sector nutrition coordination and support. d. Continue engage coordination committees (WASH-CC) at the local level and improve their capacity to implement WASH activities (such as Total Sanitation), focusing on areas and communities where WASH indicators are not improving. e. Continue advocate for formation / revitalization of NFSSCs and WASHCCs at least at the municipality level, and support them, where possible, in planning and monitoring multi-sector nutrition activities. f. Continue working with NPC and sector ministries at the central level to make policy provisions to make these committees effective and accountable. 5.2 Technical I . Engage pro-actively with MOHP to support improvements in HMIS for nutrition data. MER is a cross-cutting area robustly focused on by the program. The information and knowledge generated by Suaahara is used effectively by the program to continuously learn and improve and serve as models for other programs. There is an opportunity to collaborate with and strengthen the capacity of officials at all levels of government to improve their data systems. For example, a. Use the learning from the program to inform and expand, as necessary, the number of nutrition program indicators and data sources within the GON’s HMIS. b. Share the experience and learnings from Suaahara’s use of DHIS2 with the government to include and track inputs and resources, in addition to the service availability, coverage and usage data that GON’s HMIS focuses on. c. Work closely with MOHP’s M&E and HMIS sections, and relevant staff at province, district and local levels to learn and benefit from Suaahara experience through training, shadowing, onsite coaching and mentoring. d. Collaborate with and leverage another USAID/Nepal program, Strengthening Systems for Better Health, to strengthen GON’s M&E / HMIS / DHIS2 systems, particularly to both improve reporting of HMIS nutrition indicators and introduce additional nutrition indicators that are important for the MOHP and the MSNP II. II. Continue successful approaches and rethink alternatives for areas where little improvement is seen. Suaahara would need to rethink, perhaps research further, and try alternative approaches, interventions or innovations to achieve improvements in the remaining two years in three key behaviors (sick child feeding, use of ORS and Zinc to treat diarrhea, and treatment of drinking water) and service use (e.g., growth monitoring and promotion and IMAM), which have been found challenging to change. a. Consider working with private sector in a few program districts or municipalities as a learning agenda to see if that would help improve the use of ORS and Zinc to treat diarrhea n children. 36 b. Conduct formative research on current breastfeeding practice in various levels of health facilities to inform training, job aids, and pre-service curriculum modifications. Include specific messaging and guidance in its tools and approaches (e.g. SBCC tools, onsite coaching and mentoring) to strengthen counseling and promote early initiation of breastfeeding at facilities. c. Consider, as part of the research work, proven effective alternative approaches to ensure the nutrition status of children under two is routinely monitored and counseling and support are provided on a routine basis. For example, testing a community or home-based alternative that would use a low-cost height measurement tool (such as a child length mat28, a stick with length/height milestones or a wall chart) to both assess a child’s linear growth and promote the concept of “tall and healthy” within the entire family. This could be complemented by using MUAC tapes by FCHVs at the community and HH level to screen for acute malnutrition, particularly in the Terai where wasting rates are double. d. Consider additional measures under IMAM, such as adjusting the distribution of OTCs, robust counseling on importance treatment compliance and adding more RUTF stocking/dispensing sites to improve access for replenishment of RUTF, and thereby minimize loss to follow up / defaulters of children under treatment. III. Ensure new initiatives of the program are timely assessed and lessons incorporated and shared. a. Take stock of the Suaahara Integrated Adolescent Nutrition Program, including the full Reproductive Health, Nutrition and WASH education experience and learnings in the four pilot districts, and share results with USAID and other relevant partners through appropriate knowledge sharing platforms. This could include, for example, holding a seminar or conference in collaboration with the MOE and MOHP on the results of Suaahara’s contribution to the Revitalized School Health and Nutrition Program and develop plans for future revision and expansion. b. Continue to improve quality of IFA supplementation activity and consider scaling-up based on findings of municipality level reviews held after the first year of implementation. Areas to be addressed include coverage of private school students, logistics of supplement delivery from the MOH to schools, and ensuring accurate information is consistently provided by teachers and in peer exchanges. IV. Use alternative media platforms to increase the reach amongst the family of 1000- day mothers. Qualitative inquiries suggest to some extent that the knowledge and practices of other HH members has increased but reaching these other HH members still has remained lower than reach to mothers. Encourage 1000-day household members to listen to Bhanchin Aama (e.g. through advertisement in cable Television, a popular media) or promoting alternative media (e.g. listening Bhanchin Aama through YouTube, Facebook links, etc.) to broaden the access beyond local FMs. 28Accessed 8/10/19: https://www.manoffgroup.com/signature-solutions/child-length-mat/ 37 Partnership I. Work with the National Planning Commission (NPC) to increase awareness and to promote the value of the MSNP II at local levels in order to garner more interest and resources for multi-sector nutrition interventions. a. Consider having “big and visible” events, such as district or provincial nutrition conferences, bringing together key local leaders (e.g., mayors) and civil servants to educate and advocate with them to support multi-sector nutrition activities. This should be targeted at selected districts and provinces with high needs (high levels of stunting and wasting) and/or little political commitment. b. Orient, build capacity, and support local governments to generate sense of ownership of MSNP II, and help them advocate to leverage other resources e.g. from provincial government, other partners, private sector, etc. c. Make more focused efforts and more regular engagement (technical assistance) with Provincial government to highlight the agenda of MSNP and ensure funding from different sector directorates / offices for MSNP interventions. d. Continue engaging and advocating with Family Welfare Division of MOHP to increase including Suaahara initiated activities for non-Suaahara districts in the GON Redbook and provide them technical support in developing and revising policies, strategies, guidelines and manuals to effectively implement the activities. II. Engage the Partner NGOs beyond the current mandate of the program. a. Make efforts to engage PNGO board members and build their capacity in the multi-sector nutrition agenda to potentially serve as nutrition champions in their respective districts and help sustain the nutrition gains made. b. Consider providing capacity building resources to PNGOs to help them with institutional strengthening and sustain their activities beyond Suaahara. As PNGOs’ vary in their strengths, the support could be customized based on individual PNGO’s capacity needs and gaps. III. Link beneficiaries (VMFs and HFPB groups) to market and private suppliers to increase access to inputs as well as to sell surplus produce. 5.3 Recommendations for Future USAID Nutrition Programming I. Remaining period of current program a) Continue work with GON and other donors to agree on a minimum set of nutrition-specific and nutrition-sensitive interventions that should be implemented in target geographic areas to satisfy MSNP II requirements. The interventions could be supported by more than one partner depending on the partners’ interest and presence, but the focus should be on ensuring the selected districts and municipalities implement an agreed upon set of MSNP interventions fully. Context specific activities could be added for areas with particular challenges. b) Take stock of the Suaahara Integrated Adolescent Nutrition Program. Ensure its results and learnings are used in other USAID programs and shared with other partners for potential replication. c) Continue advocating with the GON to engage the government health personnel (particularly health managers) to build their capacity in planning and management of nutrition and health programs. 38 II. For future multi-sectoral/integrated nutrition program a) Consider developing true collaboration among programs, e.g. joint goals and work plans. These could be health, agriculture, education or other sectoral activities that include nutrition programming. In the agriculture sector, there is a need for collaborative action to improve dietary diversity while ensuring food safety particularly for animal source foods and fresh produce and to build resilience in at-risk communities. b) Think about how target areas are better selected for future nutrition investments E.g. based on nutrition levels, such as Terai for high SAM levels and many other factors contributing to poor nutrition status; ‘True’ Karnali districts for poor levels of health services and food availability, etc. c) Consider having better coordination with multi/bi-lateral agencies working in the same area to avoid duplication and ensure programming synergy, where more than one partners are present, to offer a full package of multi-nutrition. III. Other agenda beyond the remit of such program a) Consider opportunities to advocate for and support developing broader nutrition capacity. Increasing the number and quality of nutrition professionals across all sectors and enhancing the technical knowledge and skills of these professionals are critical to improving the quality of nutrition services. The expansion of professionals and frontline workers should include measures to ensure that women have the opportunity and ability to gain the knowledge and skills needed to join the nutrition workforce. Opportunities to advocate with and support the GON, for example, could be to: • Develop or revise curriculum: Nutrition modules with robust content and up-to-date evidence base must be an integral part of the curricula in medical, nursing, and agricultural educational institutions, as well as certificate programs for frontline community workers. Higher learning institutions that support professional training in nutrition across sectors, as well as the overall systems for training, recruitment, deployment, and retention of competent professionals need to be strengthened to help create leaders and researchers for nutrition innovation and technological advances. Pre-service education, service-entry induction and in￾service training for nurses, doctors, agricultural technicians, and frontline workers should include up-to-date practical nutrition information and needed competencies. • Develop an equitable human resources plan, that engages women and minorities, to address gaps related to the number and skills mix of nutrition professionals and technicians within key sectors at each relevant level • Strengthen academic institutions’ capacity to anticipate national technical gaps and develop appropriate curricula to meet needs in nutrition-related sectors through degree programs at all academic levels • Train facility-based staff in quality assurance and quality improvement methodologies • Assess and strengthen managerial competencies at all levels within key programs and systems in relevant sectors • Strengthen country and regional research capacity to set research agendas and conduct research on relevant nutrition issues b) Consider instilling awareness of private sector potential and value of private sector engagement and partnerships amongst its implementing partners. Private sector engagement and partnership with development programs is an increasing priority for USAID. 39 APPENDIX 1: EVALUATION STATEMENT OF WORK MIDTERM PERFORMANCE EVALUATION OF USAID/NEPAL’S SUAAHARA II INTEGRATED NUTRITION PROGRAM STATEMENT OF WORK I. BACKGROUND Nepal has seen a reduction in stunting among children under five years of age over the last twenty years from 57% to 36%, yet the 2016 rate of more than one-third of children being stunted is of continuing concern. The 2016 NDHS data show that 10% of children under five are wasted and 2% severely wasted. Underweight in women of reproductive age (WRA) is still high at 17%, but down from 26% in 2006. Anemia is also a concern at 53% among children 6-59 months of age, 41% among all WRA (an increase of 5% from 2011), 46% among pregnant women, and 44% among adolescent girls (NDHS 2016). Poor Maternal, Infant, and Young Child Nutrition (MIYCN) practices, including insufficient dietary diversity and quantity and low consumption of animal source foods, are key determinants of under-nutrition and vary by geographic zone and various socio-demographic and economic factors. Gender discrimination is another contributing factor in under-nutrition among women and children, including intra-familial food distribution favoring men, women’s heavy workload, and early marriages and pregnancies. This perpetuates an intergenerational cycle of malnutrition. Poor water, sanitation and hygiene (WASH) services and practices are also important factors in health and nutrition outcomes. Nepal has made great strides in ending open defecation, but a third of households still do not have a latrine and only 23% use an effective water treatment method (NDHS 2016). Use of healthcare services is suboptimal, although improving. The proportion of women who had the WHO-recommended minimum of four antenatal care (ANC) visits increased from 29 to 69% between 2006 and 2016. Approximately 24% of women wish to space or limit births but are not using contraception. USAID has provided substantial contributions to health and nutrition improvements in Nepal in partnership with the Government of Nepal (GON) and other external development partners. USAID supported to implement Action Against Malnutrition through Agriculture (2008-2012) in 3 districts in the Far-western region and an Integrated Nutrition program - Suaahara - in 20 districts (2011-2016) that expanded into 41 districts across all geographic zones of Nepal and developed productive working relationships with the government at all levels including other stakeholders. As part of its strategy to continue to strengthen the country’s health and nutrition programs, USAID, in consultation with the Ministry of Health and Population (MOHP) and relevant partners, designed Suaahara II. Suaahara II is a five-year, $63 million integrated nutrition program dedicated to improving the health and nutrition status of women and children, particularly in the 1,000 days period from pregnancy to a child’s second birthday. This period is recognized as the crucial time to prevent malnutrition. Suaahara II collaborates with existing programs and the private sector towards achieving its shared objectives. The Suaahara II mid-term performance evaluation is designed to inform future programming and to evaluate and improve the effectiveness and efficiency of Suaahara II implementation during the remainder of the program. The mid-term evaluation team will consult with Suaahara II implementation staff, USAID program management team, and the GON; review existing evidence and literature relevant to Suaahara II 40 implementation; and solicit feedback from program participants and key stakeholders on the quality of Suaahara II implementation, adjustments to be made, and opportunities for improvement. II. DESCRIPTION OF THE PROGRAM Activity Name Suaahara II (Good Nutrition Program) Implementer Helen Keller International Cooperative Agreement/Contract # AID-367-A-16-00006 Total Estimated Ceiling of the Evaluated Program/ Activity (TEC) $63,254,184 Life of Strategy, Program, or Activity April 01, 2016 to March 31, 2021 Active Geographic Regions 389 municipalities in 42 districts Development Objective(s) (DOs) Increased human capital USAID Office Health Office, USAID Nepal Key Interventions and Implementation Strategies Suaahara II is a comprehensive program that aims to improve the nutritional status of women and children under five years in Nepal, with an emphasis on the 1,000 days period between pregnancy and a child’s second birthday. Suaahara II works with the GON in 389 municipalities of 42 districts using evidence-based, multi-sectoral nutrition interventions that include nutrition-specific and nutrition￾sensitive activities at the policy (national and sub-national), community, and household levels. Specifically, Suaahara II focuses on improving nutrition services; maternal, newborn, and child health (MNCH) services; reproductive health/family planning services; water, sanitation and hygiene (WASH); homestead food production (kitchen gardening and poultry rearing); and multi-sectoral nutrition governance. It is working to achieve its objectives through multi-sector partnerships with different levels of actors throughout the government structure private sector, and other development stakeholders, as well as via public-private partnerships. Additionally, Suaahara II emphasizes gender equity and social inclusion as a cross-cutting theme; implements via diverse social and behavior change efforts and has extensive monitoring, evaluation and research (MER) for learning data systems. The Core intervention package includes social and behavior change interventions (Nutrition, WASH, FP, MNCH); maternal, infant and young child nutrition assessment and counseling support; integrated management of acute malnutrition; governance activities, and gender equity and social inclusion (GESI) activities. The Core Plus package adds enhanced homestead food production (home gardening and poultry rearing) and more intensive social and behavior change, health system strengthening, WASH and GESI components. 41 Targeted Groups and Geographic Coverage Suaahara II will reach 1.5 million pregnant and lactating women, and children under two years. In addition, Suaahara II will target adolescent girls and other household members, including children under five, mothers-in-law, and husbands, who play a critical role in improving maternal and child health and nutrition as part of the household approach. Suaahara II works in 389 municipalities (3,353 wards) of 42 districts. Suaahara II implements its Core package in all wards of 389 municipalities and Core Plus activities in 1,504 wards in disadvantaged communities (classified in the lowest 3 to 6 categories by the Ministry of Federal Affairs and Local Development prior to Suaahara) of select municipalities. Description of Key Partners and How Activities are Coordinated Suaahara II is implemented by Helen Keller International (HKI) with its six consortium partners. Suaahara II is led by a Chief of Party (COP) and each consortium partner has deputed technical staff in the Suaahara II Kathmandu office as well as technical staff based in Suaahara II field offices. Further, a small team operates out of Nepalgunj, as a Program Linkages office and under the leadership of the Program Operations Managers, responsible for linkages between Suaahara II and other USAID funded programs. Suaahara II consortium partners jointly implement activities in an integrated way to demonstrate coherence in programming and convergence of 1,000-day households at the community level. Also, HKI partners with local NGOs in each district to implement its activities, with the aim of supporting cost￾effective and locally responsive nutrition and health services in partnership with the local government and other stakeholders. Below are the roles of each of the consortium partners in Suaahara II: Suaahara II Partners Roles in the Suaahara II Program HKI (Prime) ▪ Overall program management and administration ▪ Overall technical direction and leadership for IRs 1, 3 and 4 (see Results Framework) ▪ Lead Monitoring Evaluation and Research component ▪ Capacity strengthening for local NGOs to deliver nutrition interventions ▪ Liaison with government, USAID and other stakeholders ▪ Management of all NGO sub-awards CARE ▪ Leadership for IR 2 and technical input for increased availability and quality of MNCH and adolescent health services, GESI, and increased resilience of communities and households to potential nutrition shocks FHI360 ▪ Technical assistance for improved capacity of health service providers to do nutrition assessment counseling and support and improved healthy timing and spacing of pregnancy counseling through promotion and more accessible FP outreach services ENPHO ▪ Program and technical guidance for WASH for improved essential WASH actions VDRC ▪ Program and technical guidance for village model farmers and marketing for increased and sustained homestead production of nutrient-rich foods ▪ Linkages with the Feed the Future program, KISAN II’s services and to markets for selling surplus homestead food production NTAG ▪ Training and capacity development for maternal infant and young child nutrition and demand creation Equal Access ▪ Mass media, particularly radio program with expansion to additional audiences ▪ Design and implementation of SMS campaign 42 Suaahara II collaborates with the GON to plan and implement its activities at the national and sub￾national level by participating in technical working groups, task forces, and inter-agency sub-committees under National Planning Commission, MOHP, Ministry of Agriculture Development and Ministry of Water Supply. These include: the Nutrition Technical Group; the Family Planning Sub-Committee; the Community Based-Integrated Management of Nutrition and Childhood Illness technical working group; the Female Community Health Volunteer (FCHV) Sub-Committee; and the Nutrition and Food Security Coordination Committee at the national and sub-national level. Suaahara II also coordinates with key USAID programs (Strengthening System for Better Health, Breakthrough Action, KISAN II, PAHAL, FTF Nutrition Innovation Lab, WASH activities) and other development programs in its intervention areas. Implementation History and Issues to Date Suaahara I (2011-2016), achieved both scale and promising results of its implementation in 41 districts across all geographic zones of Nepal. Suaahara I made considerable progress in improving nutrition, health and hygiene practices in the target districts, primarily using FCHVs to reach mothers with children in the 1,000-day critical window through monthly home visits and health mothers’ groups to promote adoption of essential nutrition, health and WASH practices. Despite remarkable gains in awareness and practices, based on Suaahara I process evaluation the program exposure and adoption of practices findings, there is still room for improvement. Suaahara I’s 2015 process evaluation showed that several other key behavioral indicators, including dietary diversity, appropriate feeding and treatment of sick children, and mothers having at least three postpartum care visits did not show significant improvement over time. Suaahara II was designed to build on Suaahara I with continued focus on strengthening health systems; improving health, nutrition, and agriculture service quality; and improving household level behaviors in all Suaahara II thematic areas. Suaahara II’s approach is similar to Suaahara I with its continued emphasis on improving women’s decision-making power as a potential means for attaining improvements in key health and nutrition behaviors. But Suaahara II has a family rather than mother/child dyad focus, added new thematic areas including adolescent nutrition and integrated management of acute malnutrition, and expanded to new intervention areas such as the use of text messaging to reach additional households. Nepal is undergoing tremendous challenges and transition, including a new constitution, moving to a federal system, and continued recovery from the devastating earthquakes of 2015 and recent floods. The implementation of federalism, including decentralization of power to the municipal level, is changing much of the authority, responsibility and implementation structure within the government system. Municipalities, many with first time mayors, are now in charge of determining funding levels and authorizations for government resources to be spent on health, agriculture and other services. In Suaahara II intervention areas, almost all decisions and processes will be carried out by 389 municipalities rather than 42 districts, creating the very real possibilities of even greater fluctuation in services. Map of the Implementation Area 43 Goal, Objectives, Theory of Change/Development Hypothesis, Results Framework, and Description The goal of Suaahara II is to improve the nutritional status of women and children under five years through an increased emphasis in the 1,000 days period between pregnancy and a child’s second birthday in 42 districts of Nepal. The objectives of Suaahara II are to improve household nutrition and health behaviors; increase use of quality nutrition and health services by women and children; improve access to diverse and nutrient-rich foods by women and children; and support the accelerated rollout of the GON’s Multi-Sectoral Nutrition Plan (MSNP) through strengthened local governance. Suaahara II’s theory of change articulates how key inputs and intervention under these four objectives will lead to shifts in the capacity of households, community and service provider levels across the program life to achieve the goal of improved nutrition status of women and children. Suaahara II activities will drive change across five categories: a) health and nutrition; b) WASH; c) service quality and reach; d) food production/security; and e) stronger governance. Activities across these five areas will in turn drive measurable results in improved nutrition and health behaviors of women, adolescents and young children, increased use of quality nutrition and health services, increased household access to diverse and nutrient￾rich foods, increased linkages with markets and private sector, and the accelerated implementation of the MSNP through strengthened local governance. Expected key results include: • Reduced stunting, underweight, and wasting prevalence among children under five in 42 target districts. • Improved household health and nutrition behaviors. • Increased use of quality maternal, newborn, and child health services; family planning services. • Improved water, sanitation and hygiene behavior and practices. • Increased consumption of diverse and nutritious foods by women and their families. 44 Suaahara II Results Framework III. PURPOSE OF THE EVALUATION The purpose of this mid-term performance evaluation is to inform future programming and to evaluate and improve the effectiveness and efficiency of Suaahara II implementation during the remainder of the program. The objectives of this evaluation are to: • Assess the effective coverage and quality of Suaahara II services and interventions. • Identify progress, successful approaches, and application of innovation and lessons learned to date in program implementation. • Identify challenges, weaknesses, bottlenecks, and constraints to produce planned outputs; • Review the programming logic (results framework and theory of change) to determine whether the program is on course to achieve the intended results and impact. • Examine the management structure and coordination practices with stakeholders. • Examine the sustainability of program interventions in line with its current coordination practices. • Recommend solutions for improving program implementation and results. To achieve these objectives, the mid-term evaluation will work closely with Suaahara II staff and USAID program management team, review existing evidence and literature relevant to Suaahara II implementation, and solicit feedback from program participants and key stakeholders on the quality of Suaahara II implementation, adjustments to be made, and opportunities for improvement. The intended audience and primary users of the mid-term evaluation will be the USAID/Nepal mission and Suaahara II staff and partners. The Government of Nepal, beneficiaries of Suaahara II, and other offices in the USAID Global Health and Asia Bureaus will also be target audiences. The results of the mid￾term evaluation will also be applicable and useful to other USAID bureaus and missions implementing 45 health and nutrition programs, and to other organizations implementing in Nepal and in nutrition and health programming more broadly. IV. EVALUATION QUESTIONS Evaluation Area Evaluation Sub-Questions Effective coverage of programming: To what extent are Suaahara II interventions reaching the intended target populations (1,000-day mothers and households, adolescent girls, frontline service providers, FCHVs, village model farms), particularly food insecure and disadvantaged communities, and resulting in improved health and nutrition behaviors and utilization of nutrition, MCH, FP, WASH, and HFP services? - What evidence is there that Suaahara II’s nutrition sensitive and nutrition specific programs might have contributed to closing the gaps between socio-economic groups and to changes in social norms related to nutrition behaviors and practices among those exposed? - How effectively is Suaahara II implementing an integrated nutrition programming approach, including its gender equity and social inclusion strategy and principles? - To what extent is Suaahara II implementing with regard to the continuum of care and life cycle approach? - To what extent is Suaahara II utilizing existing platform to reach its target beneficiaries? How effectively is Suaahara II integrating its program activities into existing government programs? - To what extent do target beneficiaries have a clear understanding of what Suaahara is, including the services offered by Suaahara II and who is eligible to receive them? - To what extent are Suaahara II frontline workers (field supervisors; community nutrition facilitators; and peer facilitators) and female community health volunteers knowledgeable and skilled to provide the nutrition-related services? To what extent do field staff feel adequately trained, supported, and equipped with the appropriate materials and tools to carry out the services? - What challenges does Suaahara II face for effective implementation of complex multi-sector nutrition interventions in Nepal (e.g. targeting, financing, conflicting interest of sectoral ministries, technical quality, new federal structure, human resources)? Multi-sectoral and multi-level stakeholder coordination, collaboration, and engagement: To what extent have GON counterparts, other USAID implementing partners, and other stakeholders, at national and subnational levels and across sectors, been engaged in program planning, - To what extent has Suaahara II succeeded in convening, coordinating, and facilitating implementation of the national Multi-Sectoral Nutrition Plan, including capacity development and coordination at different levels? What have been the results of this coordination and facilitation? - How effectively do Suaahara II activities link and collaborate with other relevant USAID activities? (including FTF KISAN II, PAHAL, HSS, FP and 46 implementation, and monitoring for Suaahara II? WASH, and contributions to UNICEF, AFSP, and other organizations)? - How has the program engaged with the private sector and leveraged partnerships with the private sector? What are the lessons learned from this engagement regarding the opportunities and challenges of this approach? Capacity strengthening and sustainability approach: To what extent and how is Suaahara II programming contributing to improved capacity, ownership, and motivation to implement multi-sectoral nutrition and health programming within the GON at different levels? - How USAID and Government can better support to Suaahara II in the next next two years in order to ensure necessary capacity strengthening, with an eye to transition technical and programmatic leadership to Nepal institutions (GON, CSOs) and individuals? - To what extent and how is Suaahara II providing support to leverage local grants for multi-sectoral nutrition activities and providing TA for planning, implementation and monitoring of the MSNP at local levels? Monitoring, evaluation, research, and learning: How are Suaahara II’s monitoring, evaluation, and research systems and activities contributing to data-driven program implementation? - How robust is the program’s M&E system, including that of sub-awardees, to track progress against targets, adequately monitor the quality of activities, and generate lessons for improving programs? - To what extent is Suaahara II coordinating with MSNP implementing partners, including GON sectoral authorities at national, municipal, and local levels, to share monitoring data, and how is that data being utilized and influencing MSNP activities? - To what extent is Suaahara II sharing findings generated on implementation of integrated nutrition programming with other stakeholders to inform other government and development partner efforts to improve the nutritional and health status of women and children in Nepal? - To what extent is Suaahara II promoting research and learning by demonstrating innovation in terms of the use of science and technology for assessing gaps and reaching targeted beneficiaries and populations efficiently (such as, innovation in monitoring approaches, use of data, and communication) ? - How do Suaahara II interventions and data systems align with the global nutrition agenda, specifically in terms of addressing knowledge gaps and providing new evidence for the global community? Program management: - To what extent does the current program management structure promote constructive 47 How effective are the current program management structure and operations to implement this complex multi-sectoral program? relationships between the prime implementing partner and sub-awardees? - Does the consortium of partners demonstrate coherence? (i.e. common vision; efficient decision￾making; clarity of roles, policies, and procedures; set of common criteria for excellence regarding program management; financial and managerial accountability; quality of service delivery; compliance to donors; accountability to communities, beneficiaries, and each other) - To what extent is the program management building local NGOs’ capacity to manage technical and administrative functions? - What is lacking in the joint management and operations structure that could improve Suaahara II programming for the next two years? V. EVALUATION DESIGN AND METHODOLOGY The mid-term evaluation for Suaahara II will be a performance evaluation largely relying on primary qualitative data collection, secondary analysis of quantitative monitoring data, and a review of existing literature, evidence, and findings from relevant research. The evaluation contractor will further refine a rigorous methodology to yield meaningful insights to respond to the evaluation questions. The contractor will submit the preliminary evaluation design to MEL/CAMRIS in response to the SOW, which will be reviewed by USAID. The design should include detailed methodology and an evaluation design matrix with a data collection and analysis plan and tools (e.g. questionnaires, interview guides, frameworks, data extraction templates) for each of the main evaluation question areas. The evaluation team, in collaboration with USAID, will finalize the evaluation methodology before fieldwork begins. The Evaluation Contracting Officer’s Representative (COR) will approve the finalized evaluation design two weeks prior to the team’s arrival in country. At a minimum, USAID expects that the evaluation team will conduct the following: • Review and analyze existing program information and plans, relevant data, literature, and evidence from the program and other research activities in the implementation areas. • Analyses quantitative Suaahara II monitoring data, as needed. • Field test data collection instruments. • Collect qualitative data from Suaahara II beneficiaries and staff, USAID staff, the Government of Nepal counterparts at appropriate levels, partner organization staff, and other relevant technical and research experts. The evaluation team will spend approximately 135 calendar days in Nepal to carry out this mid-term evaluation. Upon award, the evaluation team will familiarize themselves with program documentation, which USAID will provide prior to the arrival of the team in the region and/or commencement of the team’s field work. The evaluation team will work with USAID and Suaahara II program staff to agree on a list of site visit locations and schedule, as well as a list of key informant interviewees. 48 Data Collection Methods The mid-term evaluation will require participatory methods inclusive of all key stakeholders to establish a comprehensive understanding of beneficiaries’ perspectives across local to national levels, ecological regions, the health system, and different sectors and influencers on nutrition, health, and food security. The evaluation will leverage existing qualitative and quantitative data and findings from recent and concurrent surveys and research activities in Nepal, including but not limited to the Suaahara II program’s own monitoring, evaluation, and learning activities, and the Nutrition Innovation Lab’s applied research on multi-sectoral nutrition interventions and policy. 1. Desk review of documents and secondary data: a. Document review: A thorough review of program documents (Section VII) and technical strategies, and secondary literature in order to understand the nutrition and health situation in Nepal. b. Review of secondary data: A thorough review of existing data related to nutrition, health status, food security, and health services in implementation areas such as Suaahara II annual household survey data and other relevant data. c. Research activities: Review of existing research activities in Suaahara II implementation areas and available data and evidence, including data generated by Suaahara II’s program monitoring systems. In particular, the evaluation team must be familiar with the activities and findings of the Feed the Future Nutrition Innovation Lab, which conducts applied research in Nepal focused on nutrition policy and multi-sectoral nutrition interventions. 2. Primary data collection: a. Focus group discussions and key informant interviews: The evaluation team will interact with beneficiaries of Suaahara II to assess the quality of program intervention and identify progress, successful approaches, barriers to adoption of optimal nutrition and health practices, and opportunities for program improvement. These beneficiaries and target populations/groups include 1,000-day household members; female community health volunteers; other community frontline workers and volunteers; health facility staff; mothers groups; and Suaahara II frontline workers including village model farmers, field supervisors, community nutrition/WASH facilitators, and peer facilitators; nutrition and food security coordination committee members and other relevant local leaders. Discussions and interviews will also be conducted with key program personnel, program implementation staff, USAID/Nepal representatives, relevant implementing partners, government ministries and staff at multiple levels (national, municipality, village), and other donor agencies and organizations operating in the area. b. Field observation: The evaluation team will observe field activities in the terai, hills and mountain ecological regions during site visits to collect findings on the implementation of program activities. Data Analysis The data collected above will be analyzed to respond to the evaluation questions in Section IV and assess the soundness of Suaahara II’s theory of change and implementation process to achieve the intended outcomes. The evaluators will clearly define criteria for determining effectiveness and efficiency of programming. Use of multiple methods, insights, and existing data and evidence sources will allow the evaluation team to triangulate findings and produce more robust evaluation results. 49 • Quantitative secondary data analysis: Changes in specific health and nutrition behaviors will be assessed by analyzing existing data from Suaahara II annual household monitoring survey data, population-based health, nutrition, and food security surveys, information collected by other USAID programs in implementation areas (e.g. Nutrition Innovation Lab, Health 4 Life, KISAN II, PAHAL), and other available relevant data from government programs and other non-USAID initiatives in the region. • Qualitative data analysis: systematic approach to coding, interpreting, and synthesizing the collected qualitative data. Focus group discussions and in-depth interviews will be recorded with the consent of participants and fully transcribed and translated into English before being analyzed, using appropriate software. • Gender and social inclusion considerations: Collected data and analysis will be disaggregated by sex, socio-economic status, age, geographic region, and other characteristics as necessary and where possible to discern differences and similarities in experiences, perceptions, needs, barriers, and other findings. Validation Workshop: Once the preliminary data analyses have been completed, the evaluation team will present initial observations to USAID/Nepal and Suaahara II leadership and relevant staff to validate findings and interpretations. In the event of strong disagreements, the evaluation team may need to revisit data collected or communities to ensure findings are based on valid and reliable information. Methodological Strengths and Limitations The evaluation team can include explicit description of anticipated limitations in data collection and analysis. Ethical Review The evaluation must operate in line with current USAID policy on treatment of human subjects when applicable. The SOW can request that the offeror account for ethical considerations such as: Institutional Review Board (IRB) requirements (USAID and Nepal Health Research Council), protocols for ensuring respondents are not harmed (i.e., “Do No Harm” principles—especially relevant if there are sensitive questions involved and/or in non-permissive environments), collection and safety of personally identifiable information (PII). (See USAID Scientific Research Policy.) The evaluation team must receive ethical approval from Nepal Health Research Council before the data collection. VI. EVALUATION TEAM The evaluation team will be composed of at least three individuals, all demonstrating the following characteristics: • Substantial and demonstrated knowledge and experience in multisectoral nutrition or agriculture￾nutrition-health programming by at least one team member will be beneficial. • Expertise in evaluation and strong quantitative and qualitative analysis skills. • Master’s degree or higher level of education in a relevant technical area (such as public health, nutrition, epidemiology, agricultural economics, etc.). • Knowledge and experience with USAID award and reporting requirements, policies and initiatives, and tools (e.g. performance monitoring plans, results frameworks, program evaluation) 50 • Advanced written and oral communication skills in English; at least one team member should speak Nepali. • Expertise working in low-resource settings, with preference for experience in health and nutrition in Nepal. • Experience working in the international donor environment and international health programs, preferably in South Asia. The evaluation team should include one Team Leader, one Health Specialist, and one Evaluation Specialist with the following additional responsibilities and competencies: TEAM LEADER Responsibilities: • Serve as the primary investigator of the mid-term evaluation, providing oversight to the evaluation throughout the entire evaluation process. • Work with USAID and Suaahara II management to plan evaluation and logistics. • Work with evaluation team to allocate evaluation responsibilities based on their skills and capacity. • Support evaluation team members to ensure that all team members fulfill obligations, communications, and deliverables in a timely manner. • Manage evaluation team’s activities and organize and facilitate team interaction. • Serve as liaison between USAID and the evaluation team. • Lead interpretation of findings and both oral and written presentations of findings. Qualifications: • Minimum of 10 years of experience in public health or nutrition, with at least 5 years of experience in designing, implementing, managing, monitoring, and evaluating international health programs. • Prior experience working on at-scale and/or complex, multi-sectoral, integrated programs or monitoring and evaluation. • Strong evaluation, organizational, and management skills, with a minimum of 8 years of experience with evaluation tools and methods. • Prior experience in leading evaluation teams; excellent skills in planning, facilitation, and consensus building. • Excellent interpersonal skills, including experience successfully interacting with host government officials, local communities, civil society partners, and other stakeholders. • Strong oral and written communication skills, with extensive report writing experience • Experience working in Nepal or the South Asia region preferred. HEALTH SPECIALIST Responsibilities: • Serve as a member of the evaluation team, providing expertise in maternal and child health and nutrition. • Participate in planning and briefing meetings, data collection, data analysis, and development of presentations and reports. 51 Qualifications: • Master’s degree or higher level of education in a relevant technical area such as public health, nutrition, epidemiology or related technical areas. • At least 5 years’ experience with maternal and child health and/or nutrition programs; USAID program implementation experience preferred. • Familiarity with WASH, agriculture food security, resilience, and/or family planning programs is desirable and integrated, multi-sectoral program experience preferred. • Excellent interpersonal skills, including experience successfully interacting with host government officials, civil society partners, and other stakeholders. • Experience in health/nutrition program implementation and monitoring and evaluation, including design and implementation of evaluations. • Familiarity with USAID health programs/programs, primary health care or health systems strengthening preferred. • Good presentation and writing skills, including report writing experience. EVALUATION SPECIALIST Responsibilities: • Serve as a member of the evaluation team, providing expertise and quality assurance on evaluation methods and issues. • Participate in planning and briefing meetings, data collection, data management, data analysis, and development of presentations and reports. Qualifications: • Master’s degree or higher level of education in a relevant technical area such as epidemiology, statistics, survey methods, economics, etc. • Experience in health/nutrition program monitoring and evaluation, including design and implementation of evaluations. • Familiarity with WASH, agriculture food security, resilience, and/or family planning programs is desirable and integrated, multi-sectoral program experience preferred. • Experience in health/nutrition program evaluation. • Familiarity with USAID health programs/programs, primary health care or health systems strengthening preferred. • Familiarity with USAID M&E policies and practices, including evaluation policies, results frameworks, and performance monitoring plans. • Strong qualitative evaluation and analytical skills. • Good presentation and writing skills, including report writing experience. • Experience working in Nepal or the South Asia region preferred. 52 VII. KEY DOCUMENTS FOR REVIEW USAID will provide following documents for review: Suaahara II Documents GON and Other Documents 1. Program Description 2. MER plan 3. Semi-annual/Annual reports 4. Health Facility Assessment report 5. Annual Household Survey reports (2017 and 2018) 6. Technical briefs (ex: Nutrition, WASH, MCH, FP, Governance, SBCC) 7. Technical Strategies (ex: GESI, PPP, and SBCC) 8. Annual work plans (years 1, 2, and 3) 9. Routine monitoring findings 10. Suaahara peer-reviewed publications to date 11. Multisector Nutrition Plan II (MSNP) 12. Nepal Health Sector Implementation Plan 2016-2021 13. MoHP NHSS Strategy 14. Demographic and Health Survey 2016 15. Annual Report, DoHS 16. DHS Further analysis reports 2018 17. Nepal SDG targets 18. MSNP II implementation guideline 19. USAID Multi-Sectoral Nutrition Strategy 2014-2025 20. Articles published by FTF Nutrition Innovation Lab (Policy and panel study) 21. Nutrition and Food Security Plan of Action 22. IYCF strategy 23. KISHAN II document 24. PAHAL program reports 25. SABAL program report 26. Maternal undernutrition strategy for health sector 27. Adolescent IFA supplementation strategy and guideline 28. WASH Strategy and Master plan In addition, the evaluation team should be very familiar with the USAID Evaluation Policy (ADS 201) and Evaluation Toolkit: https://usaidlearninglab.org/evaluation-toolkit. VIII. KEY STAKEHOLDERS TO BE CONSULTED The following stakeholders should be interviewed as part of the mid-term evaluation: a. USAID i. USAID Health Office (FP, MCHN, WASH, ME) ii. FTF Team Leader and COR of KISAN II b. Government of Nepal i. Nutrition Section Chief ii. MCH and FP Section Chiefs iii. Program Director /Joint Secretary, National Planning Commission iv. National Health Education and Information Communication Center (NHEICC) v. Department of Water Supply and Sewerage vi. Department of Agriculture vii. Department of Livestock viii. Public health officers/nutrition focal points at select districts ix. Mayors/chair of municipalities c. Suaahara II program staff: Chief of Party, Deputy COPs, Senior Technical Advisor, Thematic team leads, M&E Sr. Managers, and consortium partner focal persons 53 d. Other development partners/stakeholders i. UNICEF ii. World Bank iii. ACF iv. WFP v. KISAN II vi. PAHAL vii. USAID Strengthening Systems for Better Health viii. WASH activities ix. AFSP/FAO x. FTF Nutrition Innovation Lab xi. Suaahara II beneficiaries IX. PROGRAM RESPONSIBILITIES The Suaahara II management/staff will be engaged in all aspects of the mid-term evaluation, including the following responsibilities: • Ensure effective coordination of mid-term evaluation logistics as needed, such as scheduling appointments with stakeholders and key informants and coordinating with consortium members and partners. • Provide consultants with background documents, reports, data, and other program materials. • Provide consultants with map and list of program sites, classified by implementation timeline and progress (e.g., phase of implementation, performance), ecological zone, and types of interventions • Respond to evaluation questions, including sharing successes and challenges and recommending changes. • Validate methodology and data collected by evaluation team and support interpretation of results. • Review draft reports and provide timely, consolidated feedback. In addition, USAID/Nepal will provide technical guidance to the evaluation team throughout the assignment, providing support with the following tasks: • Respond to queries about the SOW or the evaluation assignment in general. • Identify and prioritize key documents and materials for the evaluation team to review, and provide them, in electronic format, to the evaluation team once the contractor has been identified. • Work with the Suaahara II program management to develop a list of field visit locations, key contacts, and any other logistics support. • Designate a USAID/Nepal Point of Contact for constant availability and technical guidance throughout the evaluation process. • Assist the evaluation team in organizing meetings with stakeholders. • Facilitate introductions and interviews with other USAID implementing partners. • Provide timely review of draft documents and approval of deliverables. 54 X. TIMELINE FOR EVALUATION Total duration of evaluation: April 2019 to July 2019 Key evaluation activities Timeframe Pre-Planning • Meeting with the partners to plan for the Mid-Term evaluation, finalize Scope of Work and finalize core competencies and level of experience of the evaluation team members and the number of members • Contact possible Team Leader / member candidates/ firms • Finalize the evaluation team (members and the team leader) • Organize all documents and make them available to the evaluation team • Develop a list of all operational municipalities (by partner) and classify them by ecozone and type of interventions (Core and core plus) • Hire contracted evaluation Team Leader and members from MEL program By mid-December 2018 Mid-January 2018 February 2019 March 2019 Planning • Consult with National Planning Commission and Ministry of Health and Population • Review of existing reports, documents and data (quantitative survey, program records) • Develop qualitative survey tools • Develop evaluation design (see Section XI for components) • In-briefing with USAID/Nepal to discuss plan and design • Develop and share evaluation design with stakeholders • Identify program staff who will participate in the review process • The evaluation team selects sample communities/municipalities • Arrange all logistics • Discussion with Suaahara II to ensure that appropriate steps are being taken before implementation April 2019 Implementation • Introductory meeting between the evaluation team and the stakeholders (partners, GON, key program staff, donor) • Briefings with USAID/Nepal: mid-term briefing to discuss desk review findings; periodic briefings as agreed upon during initial in-briefing • Field work (interviews, FGDs, observations, analyses, triangulations) • Make presentation to USAID/Nepal and Suaahara II program staff separately on the preliminary observations to validate the findings and interpretations • Re-visit sites (if there are extreme disagreements on evaluation team findings between the evaluation team and the program staff) • Make a detailed presentation of the results including recommendations to Suaahara II program staff • Make a summary presentation of the key findings to USAID and Government counterparts Starting from 3rd week of April 2019 - 1st week of June 2019 Reporting • Prepare a draft report following the guideline and submit to USAID/ Nepal and Suaahara II program staff • Address the comments and incorporate inputs from USAID, and any other reviewers • Hold final presentation and analysis workshop June – Mid August 2019 55 • Finalize the mid-term report XI. DELIVERABLES 1. Evaluation Design: Following the discussions of plan and design with USAID/Nepal, the evaluation team must submit an evaluation design (which will become an annex to the Evaluation report). The evaluation design will include: (1) Background and Program Description including Program Theory of Change; (2) Key Evaluation Questions; (3) Methodology (including data collection methods, sampling, and data analysis); (4) a detailed evaluation design matrix that links the Evaluation Questions in the SOW to data sources, methods, and the data analysis plan; (5) draft questionnaires and other data collection instruments or their main features; (6) the list of potential interviewees and sites to be visited and proposed selection criteria and/or sampling plan (must include calculations and/or a justification of sample size, plans as to how the sampling frame will be developed, and the sampling methodology, where applicable); (7) Evaluation work plan including the anticipated schedule and logistical arrangements and a list of the members of the evaluation team, delineated by roles and responsibilities; (8) known limitations to the evaluation design; and (9) a dissemination plan. 2. Discussion of plan and design: An in-briefing with USAID/Nepal upon the Team Leader’s arrival in Nepal for introductions and to discuss the team’s understanding of the assignment, initial assumptions, evaluation questions, methodology, and work plan, and/or to adjust the Statement of Work (SOW), and the evaluation design, if necessary. 3. Desk Review Analysis of data and Briefings: The evaluation team is expected to hold a mid-term briefing with USAID after completing the desk review component on the findings of the desk review and its implications on the evaluation questions, including potential challenges and emerging opportunities. The team will also provide the evaluation COR/manager and Suaahara II AOR with periodic briefings and feedback on the team’s findings, as agreed upon during the in-briefing. If desired or necessary, weekly briefings by phone can be arranged. 4. Presentation of Initial findings to the Mission and Suaahara II program staff: At the end of the field work the evaluation team is expected to present their initial findings to USAID/Nepal and the implementing partner separately. This presentation will provide an opportunity for the Mission and IP to validate any factual inaccuracies and develop better ownership of the outcomes of the evaluation later. The presentation should also provide an overview to the likely recommendations that seem to be emerging based on the team’s reflection of data collection work. 5. Draft Evaluation Report: The contractor will submit a draft evaluation report of not more than 30 pages in length, single-spaced in TNR 12-point font, excluding annexes, with an executive summary of not more than 3-5 pp. in length, by mid-May 2019. As per USAID Evaluation Report Requirements ('A Mandatory Reference for ADS Chapter 201, 201mah' available at: https://www.usaid.gov/sites/default/files/documents/1868/201mah.pdf) the report must also include an abstract of not more than 250 words briefly describing what was evaluated, evaluation questions, methods, and key findings or conclusions. The abstract should appear on its own page immediately after the evaluation report cover. 6. Final Presentation and Analysis Workshop: The evaluation team is expected to hold a final presentation in person/by virtual conferencing software to discuss the summary of findings and recommendations to USAID. This presentation will be scheduled as agreed upon during the in-briefing. 7. Final Evaluation Report and Presentation: A final evaluation report of not more than 30 pages in length, single-spaced in TNR 12 point font, excluding annexes, with an executive summary of not more 56 than 3 pages in length, within 10 working days of receipt of consolidated comments in electronic format from USAID. A Comments Matrix should also be included with a list of comments from USAID and responses from the evaluation team addressing each comment. All the qualitative data collected as part of this evaluation must be submitted as an annex to the final report, either in summarized format or transcripts, with PII removed. The Final approved evaluation report must be submitted to the Development Experience Clearing House (DEC). XII. EVALUATION REPORT CRITERIA The Evaluation Final Report must follow USAID’s Criteria to Ensure the Quality of the Evaluation Report (found in Appendix I of the USAID Evaluation Policy: https://www.usaid.gov/evaluation/policy). The report should not be more than 30 pages (excluding executive summary, table of contents, acronym list and annexes). The structure of the report should follow the Evaluation Report template, including branding (https://www.usaid.gov/branding). Draft reports must be provided electronically, in English, to MEL who will then submit it to the USAID Health Office for review. For additional Guidance, please see the Evaluation Reports and How-To Note on preparing Evaluation Draft Reports found at: https://usaidlearninglab.org/evaluation-toolkit. USAID Criteria to Ensure the Quality of the Evaluation Report (USAID ADS 201): • Evaluation reports should be readily understood and should identify key points clearly, distinctly, and succinctly. • The Executive Summary of an evaluation report should present a concise and accurate statement of the most critical elements of the report. • Evaluation reports should adequately address all evaluation questions included in the SOW, or the evaluation questions subsequently revised and documented in consultation and agreement with USAID. • Evaluation methodology should be explained in detail and sources of information properly identified. • Limitations to the evaluation should be adequately disclosed in the report, with attention to the limitations associated with the evaluation methodology 57 APPENDIX 2: GETTING TO ANSWERS MATRIX Questions Measures Data Sources Methodology / Tool Data Analysis 1. Effective coverage* of programming To what extent are Suaahara II (Suaahara) interventions reaching the intended target populations, particularly food insecure and disadvantaged communities, and resulting in improved health and nutrition behaviors and utilization of nutrition, MCH, FP, WASH, and HFP services? *Effective coverage includes the availability of services, use of services, and quality of services provided. Primary beneficiaries are 1,000- day mothers, children and households, adolescent girls, frontline service providers, female community health volunteers (FCHV(, and village model farmers living in Suaahara target communities. Suaahara coverage data by intervention in 42 target districts disaggregated by food security and socioeconomic status Behavior-related indicators and indicators of the utilization of services including but not limited to: • Percent of women consuming all 180 tablets of IFA during pregnancy • Percent of births attended by a skilled birth attendant • Percent of births receiving at least 4 antenatal care (ANC) visits during pregnancy • Percent of newborns receiving postnatal health check within 24 hours of birth Suaahara monthly monitoring data Suaahara annual reports Health sector reports Nepal demographic and health Surveys Key informants at the national, district, municipality, and wards levels Stakeholders in national and local multi-sectoral nutrition committees 1,000-day mothers and other household members (husbands and mothers-in￾law) Adolescent girls Frontline workers (FLW) Health care providers FCHVs Village model farmers (VMFs) Document review National data Suaahara M&E system Suaahara Formative and operational research data/results KIIs/FGDs Field observations of the quality of health and nutrition services in health centers and communities Secondary analysis of Suaahara quantitative data Qualitative analysis of FGD and KII discussions; thematic coding of responses using the software package Nvivo Comparative analysis of Suaahara supported district indicators with national statistics Triangulation with qualitative views/perspectives on quality of care and accessibility of services. 58 Questions Measures Data Sources Methodology / Tool Data Analysis • Prevalence of exclusive breastfeeding of children < 6 months of age • CPR among WRA • GMP attendance • Engagement of HMGs • Listenership of Bhanchhin Aama programming • # of eggs produced in the previous month by HHs who own chickens • Percent of HH who practice handwashing at 6 critical times • Treatment of water before drinking • # of ODF communities • Percent of HH that used revenue earned by selling HFP surplus for nutrition in the previous year Thematic Suaahara staff (egg consumption, HMG, GMP, water treatment, and Bhanchin Aama) Field supervisors Community nutrition facilitators WASH triggers DWASHCCs and VWASHCCs 1.1 What evidence is there that Suaahara II’s nutrition-sensitive and nutrition-specific programs29 might have contributed to closing Indicators of the provision of services disaggregated by socio￾economic group including but not limited to: Suaahara program staff PNGOs GON officials (district/local) KII/FGDs Suaahara M&E system Secondary data analysis of MIYCN indicators disaggregated by DAG and non-DAG HHs 29 Nutrition-specific interventions address the immediate determinants of malnutrition and include Interventions or programs that address the immediate determinants of fetal and child nutrition and development—adequate food and nutrient intake, feeding, caregiving and parenting practices, and low burden of infectious diseases. Examples: adolescent, preconception, and maternal health and nutrition; maternal dietary or micronutrient supplementation; promotion of optimum breastfeeding; complementary feeding and responsive feeding practices and stimulation; dietary supplementation; diversification and micronutrient supplementation or fortification for children; treatment of severe acute malnutrition; disease prevention and management; nutrition in emergencies. Nutrition-sensitive interventions address the underlying and systemic causes of malnutrition and include interventions or programs that address the underlying determinants of fetal and child nutrition and development— food security; adequate caregiving resources at the maternal, household and community levels; and access to health services and a safe and hygienic environment—and incorporate specific nutrition goals and actions. Nutrition-sensitive programs can serve as delivery platforms for nutrition-specific interventions, potentially increasing their scale, coverage, and effectiveness. Examples: agriculture and food security; social safety nets; early child development; maternal mental health; women’s empowerment; child protection; schooling; water, sanitation, and hygiene; health and family planning services for healthy timing and spacing of pregnancies. (Maternal and Child Nutrition 3, The Lancet 2013; 382: 536–51) / 59 Questions Measures Data Sources Methodology / Tool Data Analysis the gaps between socioeconomic groups and to changes in social norms related to nutrition behaviors and practices among those exposed? • Percentage of HHs with a child 0-2 years old who had contact with the FCHV in the previous month • Percent of reproductive age women in union who are currently using a modern method of contraception • Changes in dietary diversity among 1,000-day mothers Formative research defining social norms and SBCC program results for 10 priority health behaviors SMS messaging to 1,000-day HH and FLWs SBCC activities and messaging to promote the use of income to improve food security and nutrition Community Mapping Census (CMC) data use in Suaahara target areas Training records Suaahara annual reports FCHVs Members of HMGs HFP groups and VMFs SBCC materials 1,000-day mothers Document review Gender and socio-economic group analysis – percentages engaged, trends, types of users (e.g., HHs where husbands migrate vs. non-migration), frequency of visits by FCHVs and to health facilities Secondary data analysis of SBCC data 1.2 How effectively is Suaahara II implementing an integrated nutrition programming approach, including its gender Municipalities receiving Integrated package of Core and Core + services Suaahara program staff PNGOs Document review KII and FGDs Qualitative analysis of FGD, KII and IDIs; thematic coding of responses using the software package called Nvivo. 60 Questions Measures Data Sources Methodology / Tool Data Analysis equity and social inclusion strategy and principles? # and percentage of DAG Wards with full Core + package of services and: • Women’s Dietary Diversity • Percent of HH with homestead gardens meeting minimum criteria • Percent of HH with chickens • # of people trained in homestead food production (HFP) • Percent of HH with a child < 2 years that received HFP inputs from VMFs and/or graduated HFP beneficiaries Use of CB-IMNCI materials, and equity and access guidelines by FCHVs Effectiveness of Self Applied Technique for quality Health (SATH) in encouraging and motivating mothers to seek health services in areas with below average MCH indicators Evidence of quality of care improvements in poor￾performing health facilities Assistance to women to overcome barriers and gain equal access to tools, resources and opportunities GON officials (district/ local) Suaahara annual reports Suaahara M&E system CMC data FGDs with HMGs, HFP groups IDIs with VMFs and HH members Gender equality and social inclusion (GESI) strategies Observations of HMGs Secondary analysis of M&E data regarding women’s empowerment and social inclusion Triangulation of GON data with Suaahara coverage of DAG groups and qualitative findings 61 Questions Measures Data Sources Methodology / Tool Data Analysis Evidence of women’s participation in HH decision making Evidence of resilience-building including, but not limited to: • OFSP production and marketing in Suaahara Districts • # of DAG VDCs received training on drought-resistant vegetables IMAM referral mechanism OFSP production data from FTF programs and/or MOAD Training records 1.3 To what extent is Suaahara II implementing activities with regard to the continuum of care and life cycle approach? Maternal nutrition in national and subnational development agendas, sectoral plans and budgets in Suaahara target areas Operationalization of national guidelines on maternal nutrition care during the first 1,000 days at the subnational level Adolescent girl programming activities: Strategy, IFA, and SBCC mass media campaign KIIs, IDIs, and FGDs with Health care officials, providers, and FCHVs GON National MIYCN guideline reviews Suaahara Adolescent Strategy KIIs with school officials FGDs with adolescent girls in targeted schools Document review KII and FGDs Observations of HMGs and health facilities Qualitative analysis of FGD, KII, and IDI discussions; thematic coding of responses using the software package Nvivo Secondary analysis of M&E data on care during key stages in life cycle 1.4 To what extent is Suaahara II utilizing existing platforms to reach its target beneficiaries? How effectively is Suaahara II integrating its program activities into existing government programs? MIYCN/NACS package implementation at the local level Constraints to participation in HMGs addressed; participation in HMGs in DAG communities Use of CB-IMNCI services by Suaahara beneficiaries Observations at Health facilities KII with health officials, health care providers KII with Nutrition Food Security Steering Committees (NFSSC) KIIs, IDIs, FGDs, data review, structured observations Qualitative analysis of FGD, KII, and IDI discussions and observations; thematic coding of responses using the software package Nvivo Triangulation of Suaahara services data and qualitative findings 62 Questions Measures Data Sources Methodology / Tool Data Analysis Provision of CB-IMNCI services according to GON guidelines Prevalence of children born in the last 24 months put to the breast within one hour of birth by delivery location Percent of women consuming all 180 tablets of IFA during pregnancy Integration of nutrition interventions, e.g NACS, into routine mother, infant and childcare services offered by GON health services and FCHVs, Quality of NACS services at OTCs in Suaahara IMAM Districts Availability and use of NACS job aids for facility-based workers and counseling wheel card for FCHVs Provision of services by MOAD agriculture extension agents in Suaahara target areas to Suaahara HFP clients Water supply and other WASH services supplied by GON programs FGDs with FCHVs and HMGs IDI with users of CB￾IMNCI services and IMAM services KII with agriculture staff in Suaahara Districts KII with WASH officials in Suaahara Districts Suaahara monitoring data 63 Questions Measures Data Sources Methodology / Tool Data Analysis Knowledge of availability of voluntary family planning services, including LAM, in Suaahara supported municipalities 1.5 To what extent do target beneficiaries have a clear understanding of what Suaahara is, including the services offered by Suaahara II and who is eligible to receive them? Community leaders, health care providers and 1,000 days mothers’ knowledge and awareness of Suaahara interventions FGDs with 1,000-day beneficiaries KII with community leaders FGDs and KII Qualitative data analysis (thematic analysis) 1.6 To what extent are Suaahara II frontline workers (field supervisors; community nutrition facilitators; and peer facilitators) and female community health volunteers knowledgeable and skilled to provide the nutrition-related services? To what extent do field staff feel adequately trained, supported, and equipped with the appropriate materials and tools to carry out the services? Field Supervisors (FS) prepared and supported to accomplish their many tasks effectively In-service training to strengthen FS and FLW’s knowledge and skills for nutrition tasks Capacity for correct and regular use of anthropometric equipment for 1,000-day clients Guidelines and standards and job aids are available for reference in health centers and for FCHVs. Satisfaction of FLWs re-training and confidence to offer quality nutrition care Quality of nutrition assessment and counseling provided by FCHVs and health facility staff IDIs with field supervisors, community nutrition facilitators, peer facilitators, FCHVs Observations of FCHVs and health care providers implementing NACS with 1,000 days community members and health facility clients Records of joint quality monitoring visits to PHC/ORCs KII with health officials and health center staff Observations of health center activities for 1,000 days clients FGDs with FCHVs and HMGs Review of training materials Review of supervisory reporting IDIs, KIIs, and FGDs Structured observations in health centers Qualitative data analysis of IDIs, KIIs, FDGs, and observations using the software package Nvivo Secondary analysis of Suaahara’s training records 64 Questions Measures Data Sources Methodology / Tool Data Analysis Support for FLWs by supervisors to ensure effective performance 1.7 What challenges does Suaahara II face for effective implementation of complex multi-sector nutrition interventions in Nepal? New federal structure Targeting Financing Administrative efficiency Conflicting interest of sectoral ministries Technical quality Human resources availability and capacity KIIs with USAID a KIIs with Suaahara staff KIIs with PNGOs Suaahara’s reporting and M&E system KIIs with GON officials implementing MSNPII at all levels KIIs with NPC, Nutrition Technical Committee, and other relevant stakeholders KIIs with GON counterparts at national and subnational levels in all relevant sectors Document reviews KIIs with stakeholders Qualitative data analysis (thematic analysis) Triangulation with NIL data on governance for multi￾sectoral nutrition in Nepal 2. Multi-sectoral and multi-level stakeholder coordination, collaboration, and engagement of programming To what extent have GON counterparts, other USAID implementing partners, and other stakeholders, at national and subnational levels and across sectors, been engaged in program planning, implementation, and monitoring for Suaahara II? Frequency of planning and implementation review meetings and field visits Number of sectors represented in MSNP II meetings in target municipalities (regular attendance) No. of municipalities where other USAID implementing partners attend multi-sectoral Suaahara program reports and M & E data KII with GON counterparts at national and subnational levels in all relevant sectors Suaahara II and MSNP II monitoring system data Document review KIIs Secondary analysis of Suaahara and GON quantitative data Qualitative analysis of KIIs (thematic analysis) Triangulation with NIL data on nutrition governance in Nepal 65 Questions Measures Data Sources Methodology / Tool Data Analysis meetings and engage in joint nutrition programming Evidence of new or improved nutrition-specific and nutrition￾sensitive interventions scaled up in Suaahara districts KII with National Planning Commission representatives Activity reports 2.1 To what extent has Suaahara II succeeded in convening, coordinating, and facilitating the implementation of the National Multi-Sectoral Nutrition Plan, including capacity development and coordination at different levels? What have been the results of this coordination and facilitation? Number of people trained to assess, plan, and manage the MSNP at sub-national levels Amount of targeted DDC and VDC funds leveraged for health, agriculture, environment, education, and/or GESI activities Suaahara II role in National Nutrition Food Security Steering Committee (NFSSC) meetings and in Suaahara Districts Effect of new federal structure on roll out of MSNPII Use of available Suaahara monitoring data by local NFSSC members for decision making and planning Annual GON expenditures on MSNP implementation at national and subnational levels Suaahara’s and the GON’s training records GON national and sub￾national budgets Minutes of NFSSC meetings and other documentation KIIs with NFSSC members at national and local levels. Document review KIIs Qualitative data analysis (thematic analysis) Secondary analysis of Suaahara and GON quantitative data and other information 66 Questions Measures Data Sources Methodology / Tool Data Analysis Evidence that nutrition has been prioritized in subnational programming 2.2 How effectively do Suaahara II activities link and collaborate with other relevant USAID activities? (including FTF KISAN II, PAHAL, HSS, FP and WASH, and contributions to UNICEF, AFSP, and other organizations)? Support to HFP groups/VMFs by PAHAL, KISAN II and reciprocal technical assistance by Suaahara Joint FP programming in Suaahara Districts Collaboration with H4L on PHC/ORC strengthening and other HPN activities Suaahara districts where nutrition-specific and nutrition￾sensitive interventions supported by USAID converge in the same communities and/or integrate KIIs with representatives of other relevant USAID activities KII with Suaahara staff and PNGOs in target localities KII with representatives of other stakeholders working in the same localities KIIs Program reports of other relevant USAID activities Qualitative data analysis (thematic analysis) Review of program documentation 2.3 How has the program engaged and leveraged partnerships with the private sector? What are the lessons learned from this engagement regarding the opportunities and challenges of this approach? Private sector in WASH Feedback regarding private sector service provision and gaps from VMFs and HFP groups KII with private WASH￾related business persons operating in Suaahara target localities FGDs with VMFs and HFP groups with private sector linkages for training, supplies, and services KIIs FGDs Qualitative data analysis through Thematic analysis 3. Capacity strengthening and sustainability approach To what extent and how is Suaahara II programming contributing to improved capacity, ownership, and motivation to implement multi￾sectoral nutrition and health programming within the GON at Integrated nutrition orientation/support for newly elected ward committee members and staff from concerned line agencies facilitated by DC/FC/thematic IDIs with newly elected GON officials IDIs with nutrition champions IDIs KIIs Document review Qualitative data analysis (Thematic analysis) Analysis of NIL data on nutrition governance in Nepal 67 Questions Measures Data Sources Methodology / Tool Data Analysis different levels (National, district, municipalities, and wards)? officers at PNGO-level, in each district Capacity building of nutrition champions and leadership development for MSNP II Awareness among CACs, HMGs, HFP Groups, communities, and municipalities on budget allocation and effective demand for proper resource allocation KIIs with MSNP II national leaders IDIs with relevant community group leaders and local GON officials 3.1 How can USAID and government better support to Suaahara II in the next two years in order to ensure necessary capacity strengthening, to transition technical and programmatic leadership to Nepal institutions (GON and CSOs) and individuals? Positions in health facilities filled and turnover reduced Adequate number of FCHVs recruited and trained Regular and adequate contraceptives, micronutrients and medical supplies available to health care providers Adequacy of ag. extension agents, agro-vets and other technical support for HFP groups and VMFs USAID continues to support joint programming between Suaahara and FTF implementors for the sustainability of HFP and contributions to FTF outcomes USAID continues to advocate for nutrition integration within KII and FGDs with USAID/Nepal staff, Suaahara II staff, PNGO staff, and NIL staff KII with key USAID programs operational in Suaahara districts Suaahara annual reports Suaahara M&E system Observations of supply chains in health facilities KIIs with MSNPII implementors at local levels Review of MSNP records at local levels KIIs FGDs Document review Structured observations Qualitative data analysis (thematic analysis) Review of GON human resource information 68 Questions Measures Data Sources Methodology / Tool Data Analysis the health sector and WASH programs Specific capacity needs in GON institutions, CSOs, and individuals Recommendations for the sustainability of technical and programmatic leadership 3.2 To what extent and how is Suaahara II providing support to leverage local grants for multi￾sectoral nutrition activities and providing TA for planning, implementation, and monitoring of the MSNP at local levels? # of local grants for multi￾sectoral nutrition in Suaahara targeted municipalities Evidence of Suaahara support to local grant implementation and monitoring Description of Suaahara’s role in leveraging local grants and list of such grants Documented examples of Suaahara’s TA for planning, implementation, and monitoring of MSNP at local levels KIIs with PNGOs and MSNPII implementors at local levels Suaahara reports Document review KIIs Qualitative data analysis (thematic analysis) Secondary analysis of Suaahara quantitative data 4. Monitoring, evaluation, research, and learning How are Suaahara II’s monitoring, evaluation, and research systems and activities contributing to data-driven program implementation? Examples of Suaahara data use by Suaahara and other partners Contributions to accountability and continual learning KIIs with USAID staff, Suaahara consortium members, the GON, and other stakeholders Suaahara’s reports and publications KIIs Document review Qualitative data analysis (thematic analysis) 69 Questions Measures Data Sources Methodology / Tool Data Analysis 4.1 How robust is the program’s M&E system, including that of sub-awardees, to track progress against targets, adequately monitor the quality of activities, and generate lessons for improving programs? Adequacy of data collection to measure results for each of Suaahara II‘s four intermediate results30 and sub-results Documentation of outcomes with a focus on gender and equity Completeness and accuracy of the monitoring system to report on quality and utilization of activities in each component of Suaahara Data on the implementation of activities that enable USAID and GON to make decisions on the cost-effectiveness and sustainability of different approaches to nutrition improvements KII with USAID staff, Suaahara consortium members, GON and other stakeholders Review of M&E system and reports KII with GON officials at all implementing levels KII with representatives of stakeholders including other donors, civil society KII with NIL staff KIIs Document review Qualitative data analysis (thematic analysis) 4.2 To what extent is Suaahara II coordinating with MSNP implementing partners, including GON sectoral authorities the at national, municipal, and local levels, to share monitoring data, and how is that data being utilized and influencing MSNP activities? Modes and frequency of data sharing by Suaahara with MSNP partners at various levels Evidence of Suaahara data use by MSNP officials at various levels KII with MSNPII officials at national, district and municipal levels Copies of Suaahara generated reports/information in hard or soft copy KIIs Document review Qualitative data analysis (thematic analysis) 30 1: Improved Household Nutrition and Health Behaviors. 2: Increased Use of Quality Nutrition and Health Services by Women and Children. 3: Improved Access to Diverse and Nutrient-Rich Foods by Women and Children. 4: Accelerated Rollout of the Multisector Nutrition Plan through Strengthened Local Governance. 70 Questions Measures Data Sources Methodology / Tool Data Analysis Evidence of MSNP activities influenced by Suaahara provided information 4.3 To what extent is Suaahara II sharing findings generated on implementation of integrated nutrition programming with other stakeholders to inform other government and development partner efforts to improve the nutritional and health status of women and children in Nepal? Examples of Suaahara program personnel sharing information on integrated nutrition programming with other partners in public fora or via other means of communication Evidence of Suaahara findings influencing investments of other development partners in Nepal to improve nutritional and health status of women and children. KII with Suaahara Staff and other stakeholders Suaahara Annual Reports KII with USAID and GON officials KIIs Document review Qualitative data analysis (thematic analysis) 4.4 To what extent is Suaahara II promoting research and learning by demonstrating innovation in terms of the use of science and technology for assessing gaps and reaching targeted beneficiaries and populations efficiently? (such as innovation in monitoring approaches, use of data, and communication) Examples of research and innovation carried out by Suaahara or with Suaahara support Documentation and dissemination of models for effective implementation of integrated activities Research reports and publications Suaahara Annual Reports KII with Suaahara staff KII with National Medical Research Council and NIL KIIs Document reviews Qualitative data analysis (thematic analysis) 4.5 How do Suaahara II interventions and data systems align with the global nutrition agenda, specifically in terms of addressing knowledge gaps and providing new evidence for the global community? Increased availability of quality data on multi-sectoral nutrition programming at scale Contributions to global evidence base to fill gaps in knowledge Knowledge exchange with USAID/W and other Missions KII with USAID/Nepal and USAID/W staff KII with Suaahara staff KII with NIL Staff Publications KIIs Document review Qualitative data analysis (thematic analysis) 71 Questions Measures Data Sources Methodology / Tool Data Analysis 5. Program Management How effective are the current program management structure and operations to implement this complex multi-sectoral program? Organogram and overall management structure including field operational structure Clear lines of authority and responsibility for each component Adherence to program timelines Budgets for prime and sub￾awardees Expenditures by IR Job descriptions and time spent on tasks of central office and field staff KII with USAID KII and FGDs with Suaahara staff at all levels Suaahara reporting Suaahara financial reports Organogram Management and operational policies and procedures KIIs and FGDs Document review Qualitative data analysis (thematic analysis) Review of management documentation 5.1 To what extent does the current program management structure promote constructive relationships between the prime implementing partner and sub-awardees? Frequency and means of communication among Suaahara consortium – formal meetings, agenda, informal communications Quality and regularity of communication with Program Manager at USAID Location of sub-awardee staff at national and local levels Processes to resolve management challenges and conflicts KII with staff of prime and sub-awardee partners KII and FGDs with representatives of PNGOs KII with USAID KIIs and FGDs Document review Qualitative data analysis (thematic analysis) 72 Questions Measures Data Sources Methodology / Tool Data Analysis Roles and responsibilities of each partner in the consortium (differentiated by intervention, province, etc.) Evidence of a shared vision, efficient decision-making; clarity of roles, policies, and procedures; set of standard criteria for excellence regarding program management; financial and managerial accountability; quality of service delivery; compliance to donors; and accountability to communities, beneficiaries, and each other IDIs with staff of prime and sub-awardee partners IDIs and FGDs with representatives of PNGOs KII with USAID Program communications and processes IDIs and FGDs Document review Qualitative data analysis (thematic analysis) 5.2 Does the consortium of partners demonstrate coherence? (i.e., a shared vision; efficient decision￾making; clarity of roles, policies, and procedures; a set of standard criteria for excellence regarding program management; financial and managerial accountability; quality of service delivery; compliance to donors; accountability to communities, beneficiaries, and each other) PNGO satisfaction with Suaahara participation Evidence of management capacity development with PNGOs IDIs with staff of prime and sub-awardee partners IDIs and/or FGDs with representatives of PNGOs KII with USAID IDIs with staff of prime and sub-awardee partners IDIs and FGDs with representatives of PNGOs KII with USAID Mini survey with PNGOs KIIs, IDIs, and FGDs IDIs, FGDs, and KIIs Mini-survey Qualitative data analysis (thematic analysis) Qualitative data analysis (thematic analysis) 73 Questions Measures Data Sources Methodology / Tool Data Analysis 5.3 To what extent is the program management building local NGOs’ capacity to manage technical and administrative functions? Gaps or deficiencies in management or operations structure or processes that may inhibit Suaahara outputs Recommendations for management and operational improvements that could expedite Suaahara efficiency and effectiveness IDIs with staff of prime and sub-awardee partners IDIs and/or FGDs with representatives of PNGOs KII with USAID IDIs with staff of prime and sub-awardee partners IDIs and FGDs with representatives of PNGOs KII with USAID Mini-survey with PNGOs KIIs, IDIs and FGDs Mini-survey Qualitative data analysis (thematic analysis) 5.4 What is lacking in the joint management and operations structure that could improve Suaahara II programming for the next two years? Gaps or deficiencies in management or operations structure or processes that may inhibit Suaahara outputs Recommendations for management and operational improvements that could expedite Suaahara efficiency and effectiveness IDIs with staff of prime and sub-awardee partners IDIs and/or FGDs with representatives of PNGOs KII with USAID IDIs with staff of prime and sub-awardee partners IDIs and FGDs with representatives of PNGOs KII with USAID Mini-survey with PNGOs KIIs, IDIs and FGDs Mini-survey Qualitative data analysis (thematic analysis) 74 APPENDIX 3: ADDITIONAL TABLES AND FINDINGS (SCONDARY ANALYSIS OF 2017 & 2018 ANNUAL HH SURVEY DATA) General Impression of the Findings • No improvement in the use of modern methods of FP among mothers with a child under two years old. Moreover, there has been a reduction in FP use in 2018 compared to 2017 among DAG groups and among mothers belonging to lowest wealth quintile. • Pregnant mothers going for at least four ANC checkups was found to be improved across all categories. Exposure to Suaahara personnel might have influenced the practice as more women attended at least four ANC checkups if they had met with any Suaahara personnel within the last six months. Also, mothers with no education had substantial improvement in going for at least four ANC checkups. • Likewise, there was an improvement in PNC checkups, but the proportion remains only at 33 percent in 2018 (an increase from 28 percent in 2017). Compared to 2017, PNC increased among the lowest and highest wealth quintiles by around 10 percentage points in 2018 for both the sub-groups; however, PNC checkups among mothers belonging to lowest wealth quintile is the lowest (at 22 percent). It also is low among mothers who are not educated and who belong to the mature Suaahara implementation district. • Appropriate water treatment remains low at around 20 percent; however, it has increased from 13.5 percent in 2017. Water treatment with appropriate methods remains low among DAG households, households in lower wealth quintiles, and households in Terai and Mountain regions. • Compared to 2017, there has been a significant improvement in HMG participation by mothers who have a child under the age of two. Continuation of participation and discussion in the meeting might be important to improve health and nutrition-related awareness. This also could indicate a successful approach Suaahara is taking to encourage mothers to participate in HMG. However, it should be dealt cautiously. • There are improvements in exclusive breastfeeding among mothers in 30-34 years age category, 2nd lowest and 2nd highest equity quintiles, brahmin/chhetri caste group, and mountain region, but in total there’s a very small change between year 1 and 2. • Minimum acceptable diet for children has improved from 37 to 46 percent in total, with increments across most groups. • Dietary diversity (DD) among children between 6-23 months of age had increased in year two compared to 2017 (from 47 percent to 54 percent). DD remains low among children aged 6-12 months of age (38 percent) compared to higher age groups (66 percent among children between 18-24 months old). There has been considerable increment in DD among children belonging to socially excluded groups; however, only slight improvement was observed among lowest wealth quintile. • As with child DD, mothers DD also increased from 35 percent in 2017 to 42 percent in 2018. However, mothers who are aged 35 and above and who are uneducated, have low DD (below 30 percent). There has been considerable improvement in DD of mothers who belong to lowest wealth quintile and socially excluded group. Women belonging to the Terai region had comparatively lower DD than those who belong to the Mountain and Hill regions. 75 • Only about 4 percent of children under two years old had their height measured at least once in last six months; however, 90 percent of those children had their weight measure at least once in the last six months before the survey. Table 2: Use of Modern Methods of Family Planning Among Mothers with a Child Under the Age of 2 Explanatory Variables Year 1 Year 2 Chi-square and p- value % N % N Met with Suaahara personnel in last six months Pearson chi2(1) = 69.5639 Pr = 0.000 No 30.1% 1633 29.0% 1315 Yes 34.4% 215 33.2% 594 Age category of mothers Pearson chi2(4) = 8.8459 Pr = 0.065 15-19 25.8% 217 29.4% 255 20-24 27.4% 758 27.7% 837 25-29 34.3% 540 32.7% 544 30-34 33.1% 239 28.9% 187 35 and above 38.5% 96 46.0% 87 DAG and Non-DAG based on 2016 equity quintiles Pearson chi2(1) = 16.4881 Pr = 0.000 DAG 32.2% 916 28.6% 814 Non-DAG 28.9% 934 31.6% 1096 Mothers education Pearson chi2(3) = 7.1766 Pr = 0.066 No education 34.0% 329 34.5% 252 Primary 31.6% 370 30.4% 352 Secondary 28.7% 917 30.0% 1031 Above secondary 31.2% 234 27.6% 275 Equity quintiles based on 2016 Pearson chi2(4) = 17.2161 Pr = 0.002 Lowest 30.2% 388 25.9% 343 2nd lowest 33.7% 528 30.6% 471 Middle 30.0% 430 33.4% 488 2nd highest 28.3% 389 29.9% 455 Highest 27.0% 115 30.7% 153 Caste categories Pearson chi2(2) = 3.0883 Pr = 0.213 Socially excluded 33.6% 897 33.9% 992 Brahmin/chhetri 28.5% 740 26.1% 766 Others 24.9% 213 28.3% 152 Residence Pearson chi2(1) = 0.5780 Pr = 0.447 Urban 30.3% 934 29.1% 951 Rural 30.8% 916 31.5% 959 Ecological belt Pearson chi2(2) = 0.3423 Pr = 0.843 Mountain 42.0% 238 40.1% 237 Hill 28.7% 1018 28.0% 1090 Terai 29.1% 594 30.7% 583 Total 30.5% 1,850 30.3% 1,910 76 Table 3: ANC Visits at Least 4 Times During Pregnancy Explanatory Variables Year 1 Year 2 Chi-square and P-value % N % N Met with Suaahara personnel in the last six months Pearson chi2(1) = 202.3612 Pr = 0.000 No 79.4% 1633 82.8% 1315 Yes 80.0% 215 91.4% 594 Age category Pearson chi2(4) = 8.8547 Pr = 0.065 15-19 82.0% 217 86.7% 255 20-24 83.1% 758 88.2% 837 25-29 78.5% 540 84.9% 544 30-34 75.7% 239 81.3% 187 35 and above 59.4% 96 69.0% 87 DAG and Non-DAG based on 2016 equity quintiles Pearson chi2(1) = 13.1824 Pr = 0.000 DAG 74.3% 916 80.0% 814 Non-DAG 84.5% 934 89.6% 1096 Mothers education Pearson chi2(3) = 12.4533 Pr = 0.006 No education 66.6% 329 74.2% 252 Primary 74.1% 370 76.7% 352 Secondary 83.5% 917 89.0% 1031 Above secondary 90.2% 234 93.8% 275 Equity quintiles based on 2016 Pearson chi2(4) = 14.8914 Pr = 0.005 Lowest 76.3% 388 79.0% 343 2nd lowest 72.9% 528 80.7% 471 Middle 83.5% 430 87.5% 488 2nd highest 85.1% 389 91.4% 455 Highest 86.1% 115 90.8% 153 Caste categories Pearson chi2(2) = 18.0890 Pr = 0.000 Socially excluded 75.5% 897 83.5% 992 Brahmin/chhetri 83.9% 740 89.2% 766 Others 80.8% 213 80.3% 152 Residence Pearson chi2(1) = 0.1627 Pr = 0.687 Urban 77.1% 934 82.9% 951 Rural 81.9% 916 88.1% 959 Ecological belt Pearson chi2(2) = 3.8027 Pr = 0.149 Mountain 82.8% 238 90.7% 237 Hill 76.5% 1018 84.3% 1090 Terai 83.2% 594 85.6% 583 Suaahara II implementation level Pearson chi2(1) = 0.0652 Pr = 0.798 Mature 75.9% 945 82.6% 973 Non-mature 83.2% 905 88.5% 937 Total 79.5% 1850 85.5% 1910 77 Table 4: PNC Checkup 3 or More Times within 7 Days-Post Delivery Explanatory Variables Year 1 Year 2 Chi-square and P- value % N % N Met with Suaahara personnel in the last six months Pearson chi2(1) = 74.6063 Pr = 0.000 No 28.3% 1633 32.5% 1315 Yes 26.5% 215 34.5% 594 Age category Pearson chi2(4) = 4.7707 Pr = 0.312 15-19 24.0% 217 32.9% 255 20-24 29.2% 758 33.1% 837 25-29 28.9% 540 33.1% 544 30-34 30.5% 239 38.0% 187 35 and above 18.8% 96 24.1% 87 DAG and Non-DAG based on 2016 equity quintiles Pearson chi2(1) = 0.0098 Pr = 0.921 DAG 19.2% 916 26.5% 814 Non-DAG 36.8% 934 38.0% 1096 Mothers education Pearson chi2(3) = 1.9983 Pr = 0.573 No education 16.7% 329 21.0% 252 Primary 23.2% 370 28.1% 352 Secondary 31.6% 917 35.6% 1031 Above Secondary 38.0% 234 41.5% 275 Equity quintiles based on 2016 Pearson chi2(4) = 9.2041 Pr = 0.056 Lowest 11.6% 388 22.4% 343 2nd lowest 24.8% 528 29.5% 471 Middle 33.0% 430 34.8% 488 2nd highest 40.6% 389 37.6% 455 Highest 38.3% 115 49.7% 153 Caste categories Pearson chi2(2) = 9.7060 Pr = 0.008 Socially excluded 27.4% 897 31.4% 992 Brahmin/chhetri 28.0% 740 35.9% 766 Others 31.5% 213 30.9% 152 Residence Pearson chi2(1) = 0.5932 Pr = 0.441 Urban 27.3% 934 31.1% 951 Rural 28.9% 916 35.1% 959 Ecological belt Pearson chi2(2) = 4.6975 Pr = 0.095 Mountain 18.1% 238 26.6% 237 Hill 24.8% 1018 30.7% 1090 Terai 37.9% 594 40.3% 583 Suaahara II implementation level Pearson chi2(1) = 0.6639 Pr = 0.415 Mature 23.4% 945 26.1% 973 Non-mature 33.0% 905 40.4% 937 Total 28.1% 1850 33.1% 1910 78 Table 5: Appropriate Method of Water Treatment Among Households with Children Under Age 2 Explanatory Variables Year 1 Year 2 Chi-square and p- value % N % N Met with Suaahara personnel in last six months Pearson chi2(1) = 14.6576 Pr = 0.000 No 11.9% 1633 18.1% 1315 Yes 25.6% 215 23.1% 594 Age category Pearson chi2(4) = 11.4040 Pr = 0.022 15-19 7.4% 217 18.4% 255 20-24 12.8% 758 17.8% 837 25-29 15.6% 540 23.9% 544 30-34 17.6% 239 20.3% 187 35 and above 10.4% 96 12.6% 87 DAG and Non-DAG based on 2016 equity quintiles Pearson chi2(1) = 6.4645 Pr = 0.011 DAG 8.4% 916 10.1% 814 Non-DAG 18.4% 934 26.7% 1096 Mothers education Pearson chi2(3) = 3.9519 Pr = 0.267 No education 3.6% 329 9.1% 252 Primary 6.8% 370 14.8% 352 Secondary 16.0% 917 18.8% 1031 Above Secondary 27.8% 234 38.5% 275 Equity quintiles based on 2016 Pearson chi2(4) = 9.4252 Pr = 0.051 Lowest 7.5% 388 7.9% 343 2nd lowest 9.1% 528 11.7% 471 Middle 20.5% 430 26.6% 488 2nd highest 13.9% 389 24.8% 455 Highest 26.1% 115 32.7% 153 Caste categories Pearson chi2(2) = 5.1154 Pr = 0.077 Socially excluded 12.0% 897 17.9% 992 Brahmin/chhetri 14.7% 740 22.1% 766 Others 15.0% 213 18.4% 152 Residence Pearson chi2(1) = 1.7844 Pr = 0.182 Urban 11.1% 934 18.6% 951 Rural 15.8% 916 20.6% 959 Ecological belt Pearson chi2(2) = 0.1559 Pr = 0.925 Mountain 9.7% 238 14.8% 237 Hill 17.2% 1018 24.6% 1090 Terai 8.6% 594 12.3% 583 Suaahara II implementation level Pearson chi2(1) = 0.4471 Pr = 0.504 Mature 12.4% 945 17.1% 973 Non-mature 14.6% 905 22.3% 937 Total 13.5% 1850 19.6% 1910 79 Table 6: Handwashing in Six Critical Times Among Mothers with a Child Under Age 2 Explanatory Variables Year 1 Year 2 Chi-square and p- value % N % N Met with Suaahara personnel in last six months Pearson chi2(1) = 41.8000 Pr = 0.000 No 8.1% 1633 17.9% 1315 Yes 5.1% 215 22.6% 594 Age category Pearson chi2(4) = 10.1647 Pr = 0.038 15-19 6.0% 217 23.9% 255 20-24 9.2% 758 20.1% 837 25-29 6.9% 540 18.6% 544 30-34 8.8% 239 15.0% 187 35 and above 3.1% 96 13.8% 87 DAG and Non-DAG based on 2016 equity quintiles Pearson chi2(1) = 0.8319 Pr = 0.362 DAG 5.9% 916 15.1% 814 Non-DAG 9.6% 934 22.5% 1096 Mothers education Pearson chi2(3) = 18.2270 Pr = 0.000 No education 5.2% 329 11.5% 252 Primary 7.8% 370 9.9% 352 Secondary 7.0% 917 22.6% 1031 Above Secondary 14.5% 234 26.5% 275 Equity quintiles based on 2016 Pearson chi2(4) = 2.9793 Pr = 0.561 Lowest 5.2% 388 10.5% 343 2nd lowest 6.4% 528 18.5% 471 Middle 8.8% 430 23.8% 488 2nd highest 10.5% 389 21.3% 455 Highest 9.6% 115 22.2% 153 Caste categories Pearson chi2(2) = 7.5020 Pr = 0.023 Socially excluded 7.0% 897 16.7% 992 Brahmin/chhetri 8.8% 740 24.4% 766 Others 7.5% 213 11.2% 152 Residence Pearson chi2(1) = 0.4742 Pr = 0.491 Urban 6.3% 934 17.2% 951 Rural 9.3% 916 21.5% 959 Ecological belt Pearson chi2(2) = 3.9275 Pr = 0.140 Mountain 7.1% 238 13.1% 237 Hill 7.4% 1018 20.8% 1090 Terai 8.8% 594 19.2% 583 Suaahara II implementation level Pearson chi2(1) = 0.0226 Pr = 0.880 Mature 4.6% 945 11.6% 973 Non-mature 11.2% 905 27.4% 937 Total 7.8% 1850 19.4% 1910 80 Table 7: HMG Meeting Among Mothers Who Have a Child Under the Age of 2 (ever participated) Explanatory Variables Year 1 Year 2 Chi-square and p- value % N % N Met with Suaahara personnel in last six months Pearson chi2(1) = 2.5972 Pr = 0.107 No 0.7% 1633 17.7% 1315 Yes 9.3% 215 35.9% 594 Age category Pearson chi2(4) = 3.9384 Pr = 0.414 15-19 0.9% 217 16.5% 255 20-24 2.1% 758 20.4% 837 25-29 1.1% 540 26.8% 544 30-34 2.5% 239 31.0% 187 35 and above 2.1% 96 33.3% 87 DAG and Non-DAG based on 2016 equity quintiles Pearson chi2(1) = 4.9412 Pr = 0.026 DAG 2.5% 916 28.3% 814 Non-DAG 1.0% 934 19.7% 1096 Mothers education Pearson chi2(3) = 1.1973 Pr = 0.754 No education 2.1% 329 26.6% 252 Primary 1.6% 370 22.7% 352 Secondary 1.6% 917 22.5% 1031 Above Secondary 1.7% 234 24.4% 275 Equity quintiles based on 2016 Pearson chi2(4) = 10.6359 Pr = 0.031 Lowest 1.8% 388 30.3% 343 2nd lowest 3.0% 528 26.8% 471 Middle 1.9% 430 22.1% 488 2nd highest 0.3% 389 20.4% 455 Highest 0.0% 115 9.8% 153 Caste categories Pearson chi2(2) = 8.8814 Pr = 0.012 Socially excluded 1.1% 897 21.3% 992 Brahmin/chhetri 2.4% 740 28.9% 766 Others 1.9% 213 9.2% 152 Residence Pearson chi2(1) = 2.0085 Pr = 0.156 Urban 1.8% 934 18.9% 951 Rural 1.6% 916 27.7% 959 Ecological belt Pearson chi2(2) = 5.4811 Pr = 0.065 Mountain 3.8% 238 30.0% 237 Hill 2.1% 1018 26.4% 1090 Terai 0.3% 594 14.9% 583 Suaahara II implementation level Pearson chi2(1) = 1.3161 Pr = 0.251 Mature 2.1% 945 23.8% 973 Non-mature 1.3% 905 22.8% 937 Total 1.7% 1850 23.4% 1910 81 Table 8: Exclusive Breastfeeding Among Mothers with a Child Under Age 2 Explanatory Variables Year 1 Year 2 Chi-square and p- value % N % N Met with Suaahara personnel in last six months Pearson chi2(1) = 24.4282 Pr = 0.000 No 69.8% 394 69.6% 319 Yes 75.4% 61 74.8% 131 Age category Pearson chi2(4) = 0.2257 Pr = 0.994 15-19 70.5% 78 66.3% 83 20-24 71.4% 199 72.8% 195 25-29 69.2% 107 70.6% 109 30-34 73.6% 53 78.3% 46 35 and above 61.1% 18 58.8% 17 DAG and Non-DAG based on 2016 equity quintiles Pearson chi2(1) = 1.7012 Pr = 0.192 DAG 70.0% 237 71.6% 208 Non-DAG 71.1% 218 70.7% 242 Mothers education Pearson chi2(0) = . Pr = . No education 73.1% 67 73.1% 67 Primary 79.3% 87 79.3% 87 Secondary 70.6% 248 70.6% 248 Above Secondary 52.8% 53 52.8% 53 Equity quintiles based on 2016 Pearson chi2(4) = 3.5148 Pr = 0.476 Lowest 74.5% 106 70.6% 85 2nd lowest 66.4% 131 72.4% 123 Middle 75.0% 96 71.3% 108 2nd highest 71.1% 97 76.2% 101 Highest 56.0% 25 51.5% 33 Caste categories Pearson chi2(2) = 4.9276 Pr = 0.085 Socially excluded 78.9% 223 75.5% 237 Brahmin/chhetri 59.8% 179 65.0% 183 Others 71.7% 53 73.3% 30 Residence Pearson chi2(1) = 0.5629 Pr = 0.453 Urban 69.2% 224 69.8% 235 Rural 71.9% 231 72.6% 215 Ecological belt Pearson chi2(2) = 5.0078 Pr = 0.082 Mountain 58.7% 63 70.7% 58 Hill 71.5% 249 72.6% 274 Terai 74.1% 143 67.8% 118 Suaahara II implementation level Pearson chi2(1) = 0.0387 Pr = 0.844 Mature 67.2% 232 69.9% 226 Non-mature 74.0% 223 72.3% 224 Total 70.5% 455 71.1% 450 82 Table 9: Minimum Acceptable Diet Among Mothers with a Child Under Age 2 Explanatory Variables Year 1 Year 2 Chi-square and p- value % N % N Met with Suaahara personnel in last six months Pearson chi2(1) = 77.0838 Pr = 0.000 No 36.9% 1232 43.8% 996 Yes 42.4% 151 49.7% 463 Age category Pearson chi2(4) = 7.4398 Pr = 0.114 15-19 35.0% 137 47.1% 172 20-24 38.7% 555 44.9% 642 25-29 39.2% 431 49.7% 435 30-34 36.8% 185 41.8% 141 35 and above 24.7% 77 32.9% 70 DAG and Non-DAG based on 2016 equity quintiles Pearson chi2(1) = 9.7353 Pr = 0.002 DAG 35.4% 673 40.6% 606 Non-DAG 39.5% 712 49.3% 854 Mothers education Pearson chi2(0) = . Pr = . No education 24.3% 259 24.3% 259 Primary 32.4% 281 32.4% 281 Secondary 42.0% 664 42.0% 664 Above Secondary 47.5% 181 47.5% 181 Equity quintiles based on 2016 Pearson chi2(4) = 11.8208 Pr = 0.019 Lowest 35.1% 279 35.7% 258 2nd lowest 35.5% 394 44.3% 348 Middle 38.0% 332 48.2% 380 2nd highest 37.9% 290 51.1% 354 Highest 50.0% 90 47.5% 120 Caste categories Pearson chi2(2) = 4.6684 Pr = 0.097 Socially excluded 35.1% 667 43.6% 755 Brahmin/chhetri 41.3% 559 49.7% 583 Others 34.0% 159 39.3% 122 Residence Pearson chi2(1) = 1.0393 Pr = 0.308 Urban 35.8% 704 42.5% 716 Rural 39.2% 681 48.8% 744 Ecological belt Pearson chi2(2) = 0.2596 Pr = 0.878 Mountain 37.7% 175 45.3% 179 Hill 40.3% 762 47.8% 816 Terai 32.6% 448 42.2% 465 Suaahara II implementation level Pearson chi2(1) = 0.1679 Pr = 0.682 Mature 37.1% 709 44.2% 747 Non-mature 37.9% 676 47.3% 713 Total 37.5% 1385 45.7% 1460 83 Table 10: Dietary Diversity Among Children 6-23 Months of Age (foods from 4 or more of 7 food groups) Explanatory Variables Year 1 Year 2 Chi-square and p- value % N % N Age group of child Pearson chi2(2) = 1.0676 Pr = 0.586 6-11.9 months 32.0% 531 37.9% 501 12-17.9 months 52.1% 403 57.1% 475 18-23.9 months 59.2% 451 66.1% 484 Met with Suaahara personnel in last six months Pearson chi2(1) = 97.0124 Pr = 0.000 No 46.2% 1232 51.4% 996 Yes 51.0% 151 57.9% 463 DAG and Non-DAG based on 2016 equity quintiles Pearson chi2(1) = 7.5324 Pr = 0.006 DAG 42.2% 673 47.4% 606 Non-DAG 51.0% 712 57.8% 854 Mothers education Pearson chi2(3) = 3.3718 Pr = 0.338 No education 29.7% 259 36.6% 205 Primary 42.0% 281 46.4% 276 Secondary 50.9% 664 55.5% 771 Above Secondary 63.0% 181 72.1% 208 Equity quintiles based on 2016 Pearson chi2(4) = 8.2128 Pr = 0.084 Lowest 41.9% 279 43.8% 258 2nd lowest 42.4% 394 50.0% 348 Middle 49.1% 332 58.2% 380 2nd highest 50.3% 290 58.5% 354 Highest 60.0% 90 55.0% 120 Caste categories Pearson chi2(2) = 6.4382 Pr = 0.040 Socially excluded 42.7% 667 50.9% 755 Brahmin/chhetri 52.6% 559 58.3% 583 Others 42.8% 159 46.7% 122 Residence Pearson chi2(1) = 1.4617 Pr = 0.227 Urban 45.3% 704 50.3% 716 Rural 48.2% 681 56.6% 744 Ecological belt Pearson chi2(2) = 0.3451 Pr = 0.842 Mountain 44.0% 175 56.4% 179 Hill 49.9% 762 55.5% 816 Terai 42.4% 448 48.8% 465 Suaahara II implementation level Pearson chi2(1) = 0.0017 Pr = 0.967 Mature 44.9% 709 51.3% 747 Non-mature 48.7% 676 55.8% 713 Total 46.7% 1385 53.5% 1460 84 Table 11: DD Among Mothers with a Child Under Age 2 (Foods from 5 or more of 10 food groups) Explanatory Variables Year 1 Year 2 Chi-square and p- value % N % N Met with Suaahara personnel in last six months Pearson chi2(1) = 86.4474 Pr = 0.000 No 34.5% 1633 40.0% 1315 Yes 40.5% 215 47.0% 594 Age category Pearson chi2(4) = 4.8723 Pr = 0.301 15-19 31.8% 217 41.6% 255 20-24 37.7% 758 42.4% 837 25-29 35.6% 540 44.7% 544 30-34 32.2% 239 41.2% 187 35 and above 28.1% 96 28.7% 87 DAG and Non-DAG based on 2016 equity quintiles Pearson chi2(1) = 6.9564 Pr = 0.008 DAG 31.7% 916 37.3% 814 Non-DAG 38.7% 934 45.8% 1096 Mothers education Pearson chi2(3) = 7.2087 Pr = 0.066 No education 23.4% 329 26.6% 252 Primary 30.5% 370 35.8% 352 Secondary 37.7% 917 43.3% 1031 Above Secondary 49.1% 234 60.7% 275 Equity quintiles based on 2016 Pearson chi2(4) = 11.0569 Pr = 0.026 Lowest 26.5% 388 37.0% 343 2nd lowest 35.4% 528 37.6% 471 Middle 38.4% 430 46.5% 488 2nd highest 37.5% 389 41.8% 455 Highest 43.5% 115 55.6% 153 Caste categories Pearson chi2(2) = 8.2811 Pr = 0.016 Socially excluded 31.1% 897 39.3% 992 Brahmin/chhetri 41.1% 740 47.1% 766 Others 31.9% 213 36.2% 152 Residence Pearson chi2(1) = 0.0903 Pr = 0.764 Urban 34.9% 934 43.1% 951 Rural 35.5% 916 41.3% 959 Ecological belt Pearson chi2(2) = 0.6037 Pr = 0.739 Mountain 41.2% 238 56.1% 237 Hill 37.8% 1018 43.3% 1090 Terai 28.3% 594 34.5% 583 Suaahara II implementation level Pearson chi2(1) = 0.0731 Pr = 0.787 Mature 37.5% 945 45.6% 973 Non-mature 32.8% 905 38.6% 937 Total 35.2% 1850 42.2% 1910 85 Table 12: Growth Monitoring Information (only available for 2018) N Mean Min Max SD Height measured at least once in last six months 64 (3.6%) 1.7 1 8 1.4 Weight measured at least once in last six months 1620 (90%) 2.7 1 9 1.6 Table 13: IR 3 Relevant Indicators Indicator 2017 Results 2018 Results Percent of households with homestead gardens meeting minimum criteria (HFP areas) 8.6% 23.3% Percent of households with chickens (HFP areas) 47.8 % 59% Percent of households with a child 0-2 years who received HFP inputs from VMFs and/or graduated HFP beneficiaries 17.4% 29.8% Number of nutrient dense vegetables cultivated by household, in the previous year (HFP areas) 8.7 11.1 Number of eggs produced in the previous month by households who own chickens (HFP areas) 11.8 9.5 Percent of households with surplus HFP sold (vegetable/egg production) in the past year (HFP areas) 21.7% 18.7% Percent of households that used revenue earned by selling HFP surplus for nutrition, in the previous years31 17.4% 23.2% Figure 1: Ratio of Highest Quintile with Lowest Quintile Between 2017 and 2018 (Lower the better) 0.0 1.0 2.0 3.0 4.0 5.0 6.0 7.0 8.0 9.0 Egg consumption among mothers Egg consumption among children Attended ANC at least 4 times Took at least 180 IFA tablets during pregnancy Appropriate drinking water treatment among HH with children <2 years Ratio of Highest to lowest quintile 2017 Ratio of Highest to lowest quintile 2018 31 Uses of income in 2017: Food Security: 16.7%; Nutrient-dense food: 5.8%; Health/FP: 3.5%; WASH: 11.3%; Education: 5.1%; Savings: 3.8%; Other: 3.1%. Uses of income in 2018: Food Security: 17.7%; Nutrient-dense food: 7.2%; Health/FP: 5.3%; WASH: 13.9%; Education: 5.0%; Savings: 4.7%; Other: 3.5% 86 Figure 2: Difference in Highest and Lowest Quintile Between 2017 and 2018 (Lower the better) 0.0 5.0 10.0 15.0 20.0 25.0 30.0 Egg consumption among mothers Egg consumption among children Attended ANC at least 4 times Took at least 180 IFA tablets during pregnancy Appropriate drinking water treatment among HH with children <2 years Difference in highest to lowest quintile 2017 Difference in highest to lowest quintile 2018 87 Severe Acute Malnutrition Related Case Study Management of Acute Malnutrition: Parents Learn About Nutrition while Caring for Their Son Shyam Chaudhari (30) and Radha Chaudhari (22) of a rural municipality in a Terai district were very upset and afraid when their nine-month-old son, Raju Chaudhari, was diagnosed with severe acute malnutrition 10 months ago. Until then he had not started proper complementary feeding and he only weighed 6 kg. “I did not know that we should start complementary feeding by rhe sixth month. I used to breastfeed him and feed rice and dal once or twice a day. I thought it was enough for a baby of his age,” said his mother. Since the child’s weight was so low, the OTC referred him to Seti Zonal Hospital, Nutrition Rehabilitation Home (NRH) for admission. Within a month of admission in NRH Raju gained 1.5 kgs. During her stay in the hospital, Radha learned that a child’s diet needs to be inclusive. “The food provided to him in NRH used to include mashed carrots, leafy vegetables, meat, eggs, milk lentils, and pickles too. I did not use to feed him so,” says Radha. She herself noticed an increase in her breast milk flow with the balanced diet the NRH provided to her. After returning from the NRH, Raju had diarrhea. “I was really afraid then, so I went to the Sister (nurse at NRH), she told me that things will be fine, but I need to be very careful with cleanliness and hygiene,” said Raju’s father. According to the Suaahara field supervisor who was involved, the child was unable to adapt to the environment back home because his parents were not much conscious of his cleanliness. The couple was counseled very well time and again regarding healthy hand washing, and cleanliness of the surroundings. Radha then started visiting the HMG meetings regularly where she learned about maternal child nutrition, cleanliness and hygiene, home food production, water purification, and started applying this knowledge in her everyday activities. According to her husband, these days, they try to make their diet balanced, including leafy vegetables, fruits, eggs, and meat. The couple who only ran a tea shop earlier now maintain their own kitchen garden and have kept three chickens. Raju does not like eating eggs, but his mother does not keep it aside saying he does not like, as earlier. Now she has her own tricks like hiding it in flour, or lito or jaulo. In the last GMP session, he weighed 8.5 kgs. “The madam in health post and the (NRH) Sister said it is ok for this age, but I want him to gain more weight so I am going to be more careful about his diet – after all it is now that he will have 80% of his brain developed,” said Radha. (All names have been changed for anonymity.) 88 APPENDIX 4: WORK PLAN 17 24 1 7 14 21 28 4 11 18 25 4 11 18 25 1 8 15 22 29 6 13 20 27 3 10 17 24 1 8 15 22 29 5 12 19 26 2 9 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 A. Evaluation Management Review and finalize SOW Recruitment and onboarding team leader Recruitment and onboarding of other team members LOE-A B. Evaluation Planning and Data Collection Preparation Evalution planning Compile and organize all relevant documents, reports, records, etc. Review relevant documents, reports, records, etc. Initial drafting of evaluation design Internal review of draft design and revisions USAID review of evaluation design Design and planning meetings with USAID, IP and govt. NHRC approval Finalization of evaluation design X Data collection preparation Drafting tools, internal review, revisions USAID review of data collection tools Preparation of field data collection plan Travel of team leader to Nepal USAID in-brief X Finalization of data collection tools Orientation and training on the tools LOE-B C. Fieldwork Primary data collection (central and field level) Desk review of secondary information, data and analysis (includes report review and analysis of secondary quantitative data) LOE-C D. Data Analysis Debrief of early high level findings to the Mission, IP and govt. X Transcription and translation of field notes Data collation, compilation and coding Initial report drafting Team leader leaves Nepal LOE-D E. Draft Report Preparation of draft report, Internal review, revision Copy editing, final revisions Submission of draft report to USAID X Recommendations workshop X LOE-E F. Final Report and presentations USAID reviews, compiles, comments Revision by team, create response matrix, team leader addresses comments and finalizes the report Internal review, revisions Presentations to IP, Mission and GON Final copy editing Submission of final report to USAID X LOE-F JUNE JULY AUG SEP NMEL Estimated Timeline: MIDTERM PERFORMANCE EVALUATION OF USAID/NEPAL’S SUAAHARA II INTEGRATED NUTRITION PROGRAM DEC JAN FEB MAR APR MAY 89 APPENDIX 5: ETHICAL APPROVAL LETTER 90 APPENDIX 6: RESULTS FRAMEWORK AND RESEARCH QUESTIONS The mid-term performance evaluation questions appear below. Table 14: Evaluation Questions Evaluation Areas and Questions Evaluation Sub-Questions 1. Effective Coverage of Programming: To what extent are Suaahara II interventions reaching the intended target populations [1,000- day mothers and households, adolescent girls, frontline service providers, Female Community Health Volunteers (FCHVs), village model farmers], particularly food insecure and disadvantaged communities, and resulting a. What evidence is there that Suaahara II’s nutrition-sensitive and nutrition-specific programs might have contributed to closing the gaps between socioeconomic groups and to changes in social norms related to nutrition behaviors and practices among those exposed? b. How effectively is Suaahara II implementing an integrated nutrition programming approach, including its gender equity and social inclusion strategy and principles? c. To what extent is Suaahara II implementing activities with regard to the continuum of care and life cycle approach? d. To what extent is Suaahara II utilizing an existing platform to reach its target beneficiaries? How effectively is Suaahara II integrating its program activities into existing government programs? e. To what extent do target beneficiaries have a clear understanding of what Suaahara is, including the services offered by Suaahara II and who is eligible to receive them? 91 in improved health and nutrition behaviors and utilization of nutrition, MCH, FP, WASH, and Homestead Food Production (HFP) services? f. To what extent are Suaahara II frontline workers (field supervisors; community nutrition facilitators; and peer facilitators) and female community health volunteers knowledgeable and skilled to provide the nutrition-related services? To what extent do field staff feel adequately trained, supported, and equipped with the appropriate materials and tools to carry out the services? g. What challenges does Suaahara II face for effective implementation of complex multi-sector nutrition interventions in Nepal (e.g. targeting, financing, conflicting interest of sectoral ministries, technical quality, new federal structure, human resources)? 2. Multi-sectoral and multi-level stakeholder coordination, collaboration, and engagement: To what extent have GON counterparts, other USAID implementing partners, and other stakeholders, at national and subnational levels and across sectors, been engaged in program planning, implementation, and monitoring for SUAAHARA? a. To what extent has Suaahara II succeeded in convening, coordinating, and facilitating the implementation of the National Multi-Sectoral Nutrition Plan, including capacity development and coordination at different levels? What have been the results of this coordination and facilitation? b. How effectively do Suaahara II activities link and collaborate with other relevant USAID activities (including FTF KISAN II, PAHAL, HSS, FP and WASH, and contributions to UNICEF, AFSP, and other organizations)? c. How has the program engaged and leveraged partnerships with the private sector? What are the lessons learned from this engagement regarding the opportunities and challenges of this approach? 3. Capacity strengthening and sustainability approach: To what extent and how is Suaahara II programming contributing to improved capacity, ownership, and motivation to implement multi-sectoral nutrition and health programming within the GON at different levels? a. How can USAID and the government better support Suaahara II in the next two years in order to ensure necessary capacity strengthening, to transition technical and programmatic leadership to Nepal institutions (GON and CSOs) and individuals? b. To what extent and how is Suaahara II providing support to leverage local grants for multi-sectoral nutrition activities and providing technical assistance for planning, implementation, and monitoring of the MSNP at local levels? 4. Monitoring, evaluation, research, and learning: How are Suaahara II’s monitoring, evaluation, and research systems and activities contributing to a. How robust is the program’s M&E system, including that of sub￾awardees’ systems, to track progress against targets, adequately monitor the quality of activities, and generate lessons for improving programs? b. To what extent is Suaahara II coordinating with MSNP implementing partners, including GON sectoral authorities at the national, municipal, and local levels, to share monitoring data, and how is that data being utilized and influencing MSNP activities? c. To what extent is Suaahara II sharing findings generated on implementation of integrated nutrition programming with other 92 data-driven program implementation? stakeholders to inform other government and development partner efforts to improve the nutritional and health status of women and children in Nepal? d. To what extent is Suaahara II promoting research and learning by demonstrating innovation in terms of the use of science and technology for assessing gaps and reaching targeted beneficiaries and populations efficiently (such as, innovation in monitoring approaches, the use of data, and communication)? e. How do Suaahara II interventions and data systems align with the global nutrition agenda, specifically in terms of addressing knowledge gaps and providing new evidence for the global community? 5. Program management: How effective are the current program management structure and operations to implement this complex multi-sectoral program? a. To what extent does the current program management structure promote constructive relationships between the prime implementing partner and sub-awardees? b. Does the consortium of partners demonstrate coherence (i.e., a common vision; efficient decision-making; clarity of roles, policies, and procedures; a set of standard criteria for excellence regarding program management; financial and managerial accountability; quality of service delivery; compliance to donors; and accountability to communities, beneficiaries, and each other)? c. To what extent is the program management building local NGOs’ capacity to manage technical and administrative functions? d. What is lacking in the joint management and operational structure that could improve Suaahara II programming for the next two years? 93 APPENDIX 7: ADDITIONAL TABLES FOR METHODOLOGY Table 15: Mid-term Performance Evaluation Sites Suaahara II MTE Province Province 1 Province 3 Gandaki Province Province 5 Karnali Province Province 7 Proposed Districts Panchthar Sindhupalchok Lamjung Rupandehi Banke** Surkhet Kailali Sampling Criteria Newly added district in S II (Level of exposure- post 2016) Level of Exposure- Pre￾2014 Level of Exposure￾Pre-2014 Level of Exposure￾Pre-2014 Level of Exposure￾Post-2014 Level of Exposure￾Post-2014 Level of Exposure￾Post-2014 Blend of Hill and Mountain Blend of Mountain and Hill Blend of Mountain and Hill Terai Terai Hill Terai CB-IMNCI district Presence of KISAN and SABAL CB-IMNCI district Presence of KISAN Presence of KISAN Presence of SSBH, KISAN and PAHAL Presence of KISAN Earthquake affected district IMAM district IMAM district IMAM district All four intensive programs All four intensive programs All four intensive programs Three out of four intensive programs Adolescent girls program Three out of four intensive programs All four intensive programs ** The team visited the Banke district to interview individuals from district and regional offices (of Suaahara II and other partners, such as KISAN II and SSBH) and looked into Adolescent Girl’s Program (Health and Nutrition and IMAM program). No other ward level FGDs were conducted in Banke. Table 16: List of Selected Municipalities and Wards District Municipality Type of ward Wards Lamjung Kwoholasothar Rural Municipality Core Plus Wards 2 Besisahar Municipality Core Wards 11 Rupandehi Devdaha Municipality Core Ward 17 Rohini Rural Municipality Core Plus Ward 5 Surkhet Lekabesi Municipality Core Ward 7 Barahtal Rural Municipality Core Plus Ward 7 (5, 6) Kailali Kailari Rural Municipality Core Plus Ward 7 (5) Gauriganga Municipality Core Ward 11 (9, 5 ,4) Panchthar Fidim Municipality Core Ward 11 Phalgunanda or Falelung Rural Municipality Core Plus Ward 4 Sindhupalchok Chautara Municipality Core Ward 7 Lisankhu Rural Municipality Core Plus Ward 7 Banke Khajura and Baijanath Rural Municipality - IFA, IMAM, KISAN II and VMF/HFP 94 Table 17: Overview of Data Collection Effort District FGD GI KII Observation Total Lamjung 5 3 23 1 32 Panchthar 4 2 22 0 28 Rupandehi 5 5 21 1 32 Surkhet 5 2 20 0 27 Kailali 7 1 14 0 22 Banke 0 3 9 1 13 Sindhupalchok 4 3 20 2 29 Kathmandu - 4 18 - 22 Total 30 23 147 5 205 District Total Types of Participant 1,000 Days Mother Other HH FCHV VMF/ HFP Girls/ Teacher Health Facility GON PNGO Suaa￾hara (FLWs & staffs) USAID partner Radio USAID/ Nepal Suaa￾hara Kathma ndu Office National stakeholders Lamjung 32 2 2 1 1 0 4 13 4 5 0 0 - - - Panchthar 28 2 1 2 1 0 2 11 2 6 0 1 - - - Rupandehi 32 2 2 1 2 0 4 12 1 7 0 1 - - - Surkhet 27 2 2 1 4 0 1 8 2 5 1 1 - - - Kailali 22 2 2 1 3 0 0 9 2 2 0 1 - - - Banke 13 0 0 0 1 2 1 2 3 3 1 0 - - - Sindhupalch ok 29 2 1 1 2 0 4 10 2 5 2 0 - - - Kathmandu 22 - - - - - - 2 - - - - 3 15 2 Total 205 12 10 7 14 2 16 65 16 33 4 4 3 15 2 Note: For the ypes of participant, other households include; mother-in law, fathers, and USAID partners: KISAN, SSBH, SEBAC-Nepal. Radio: interview with radio partner. Girls/teacher: Interview with adolescent girls and teacher in Banke district. Suaahara: Total interview with FLWs (FS, CNV) and Suaahara district staffs. 95 APPENDIX 8: DISCLOSURE OF ANY CONFLICT OF INTEREST 96