FINAL PERFORMANCE EVALUATION FEED THE FUTURE TAJIKISTAN HEALTH AND NUTRITION ACTIVITY 2015-2020 December 2019 This publication was produced at the request of the United States Agency for International Development. It was prepared independently by ME&A, Inc. and RTI as subcontractor. FINAL PERFORMANCE EVALUATION FEED THE FUTURE TAJIKISTAN HEALTH AND NUTRITION ACTIVITY 2015-2020 This publication was produced at the request of the United States Agency for International Development Developed under Agreement Number: AID-176-LA-15-00001. It was prepared independently by: Jean Capps, Team Leader Sabir Kurbanov, Technical Specialist Manzura Mirsaidova, Technical Specialist Contractor: ME&A, Inc. 4350 East-West Highway, Suite 210 Bethesda, MD 20814 Tel: 240-762-6296 www.MEandAHQ.com DISCLAIMER The author’s views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. ABSTRACT The United States Agency for International Development Central Asia (USAID/CA)-funded Tajikistan Health and Nutrition Activity (THNA) is a five-year project designed to improve the health and nutrition status of women and children in 12 southwestern districts of the Khatlon region of Tajikistan. THNA is seeking to achieve this goal through implementing a range of interventions and strategies that focus on the 1,000-day window of opportunity—between conception and a child’s second birthday—to make a lasting impact on the child’s growth, learning, and future productivity. This performance evaluation was commissioned to determine THNA’s results achieved to date and identify lessons learned and best practices and make recommendations for future USAID maternal, newborn, and child health (MNCH) and nutrition programs. The Evaluation Team (ET) conducted desk review of materials, interviews with key stakeholders, focus group discussions with direct beneficiaries, client exit interviews with women that use the health facilities (HFs) supported by THNA, and a survey of health workers (HWs). The ET found that THNA’s interventions have: 1) improved MNCH and nutritional status; 2) developed national capacity to assess, plan, train, provide supportive supervision, and monitor programs; and 3) provide a package of health, hygiene, and nutrition Social and Behavior Change Communication interventions at the household level linking communities to improved MNCH and nutrition health services. The ET recommends supporting increased access to MNCH services in rural communities, expanding quality improvement to neonatal and child health programs, increasing attention to maternal nutrition and anemia, and developing a Community Health Worker program. APPRECIATION The THNA Evaluation Team would like to express appreciation to the THNA managers that organized our interviews and field visits on very short notice and in often very challenging circumstances. We are grateful for MOHSPP representatives, health facility managers, health workers, and donor organization managers that took time from their very demanding schedules to meet with the team and gave us very thoughtful and detailed answers to our questions. We would especially like to thank the groups of pregnant women, mothers of children under 2 years of age, their mothers-in-law, and community leaders for their warm welcomes into their communities and taking the time to answer important questions about the benefits they derived from the improved health services and community-based health promotion that they received from the project. CONTENTS EXECUTIVE SUMMARY..................................................................................................................................i 1.0 EVALUATION PURPOSE AND QUESTIONS............................................................................1 1.1 Evaluation Purpose......................................................................................................................1 1.2 Evaluation Questions..................................................................................................................1 2.0 PROJECT BACKGROUND .............................................................................................................2 3.0 EVALUATION METHODOLOGY.................................................................................................2 3.1 Sampling .........................................................................................................................................3 3.1.1 Health Worker Survey ....................................................................................................3 3.1.2 Client Exit Interview ........................................................................................................3 3.1.3 Site Visits.............................................................................................................................4 3.2 Data Collection Methods ..........................................................................................................4 3.2.1 Desk Review ......................................................................................................................4 3.2.2 Key Informant Interviews................................................................................................4 3.2.3 Focus Group Discussions................................................................................................4 3.2.4 Health Worker Surveys and Client Exit Interviews..................................................5 3.3.5 Data Analysis......................................................................................................................5 4.0 Findings by Evaluation Question......................................................................................................6 4.1 EQ1: To what extent has the activity increased access and improved to (quality) MNCH services including nutrition, sanitation, and hygiene in the ZOI, particularly in the first 1,000 days?.....................................................................................................................6 4.1.1 Improved Access and Quality of Health Services......................................................6 4.1.2 Constraints in Improving Clinical Health Care Services....................................... 18 4.1.3 Specific Barriers to Addressing Maternal and Child Anemia ............................... 21 4.1.4 Follow-Up Information Received After the ET Fieldwork and Draft Report.. 23 4.1.5 Sustaining Community Volunteers............................................................................. 24 4.2 EQ2: To what extent has the activity increased access to a diverse set of nutrient rich foods throughout the year and why, especially the dietary diversity of pregnant and lactating women and children under 2 years of age?................................................. 27 4.2.1 Response From the Project to Draft Report.......................................................... 29 4.2.2 School Peer Educator and Demonstration Gardens............................................. 30 4.3 EQ3: To what extent were (program-supported) evidence-based guidelines and procedures institutionalized by the health systems in the FTF ZOI? And what if any, were the barriers?..................................................................................................................... 32 4.3.1 Constraints...................................................................................................................... 34 4.4 EQ4: Going forward, how, and in which areas, should USAID-funded MCH & Nutrition activities be conducted to support the health system and/or population in improving the nutrition and health services to Mothers and Children?....................... 35 ANNEXES ....................................................................................................................................................... 39 Annex 1: Evaluation Statement of Work.................................................................................... 40 Annex 2: Evaluation Schedule........................................................................................................ 46 Annex 3: Evaluation Design Matrix.............................................................................................. 49 Annex 4: List of Documents Reviewed....................................................................................... 59 Annex 5: List of Key Informants for Interviews and FGDs.................................................... 61 Annex 6: Data Collection Tools................................................................................................... 66 Annex 7: Bios and Summary Information of Team Members................................................ 90 LIST OF FIGURES Figure 1: Level of Women/Clients Satisfaction (n=164)..........................................................................9 Figure 2: Type of Health Workers Surveyed .......................................................................................... 11 Figure 3: Average Scoring of Opinion for Trainings (n=149) .............................................................. 12 Figure 4: Percentage of Respondents by Type of Changes Identified (n=123)................................ 12 Figure 5: Infection Control and Clinical Safety Indicators at the Hospital Level...................... 14 Figure 6: Percentage of Respondents by Type of Infection Preventing Procedure Identified (n=124)............................................................................................................................................ 14 Figure 7: Infection Control Practices Mentioned by HWs (n=124)................................................... 15 Figure 8: Maternity Departments EPC Scorecards................................................................................ 19 Figure 9: Reduction in PPH Cases by Cohort in THNA-Supported HF........................................... 20 Figure 10: QI Indicators on ANC .............................................................................................................. 20 Figure 11: Percentage of Women with Children Sick with Diarrhea Who Responded Yes to the Following Questions (n=20)..................................................................................................... 21 Figure 12: Anemia Among WRA (2017 DHS)........................................................................................ 21 Figure 13: Anemia in Children Under 5 Years (DHS 2017) ................................................................ 22 Figure 14: Women’s Knowledge of Foods High in Vitamin A............................................................. 28 Figure 15: Percentage of Women Reaching Minimum Dietary Diversity (MDD), %, and Average Number of Food Groups Consumed by Women, RHS Rounds 2-5 and Control Communities in 2019................................................................................................................. 29 Figure 16: Breastfed and Non-Breastfed Children 6-23 Months Who Achieve MDD, Minimum Meal Frequency (MMF,) and Minimum Acceptable Diet (MAD), RHS Rounds 1-5 and Control Communities in 2019 (%).......................................................................................... 29 Figure 17: Respondents Applying Improved Agricultural Technologies, as Confirmed By THNA Enumerators, By Type of Technology, 2017 and 2019 ( %).............................................. 30 Figure 18: Proposed Modified WHO Health Systems Framework.................................................... 34 LIST OF TABLES Table 1: Sample Health Facilities, Health Workers, and Clients ...........................................................3 Table 2: Protocols, Guidelines, and Standards Developed with THNA Support by Subject and Type(s) of Assistance..................................................................................................................... 32 ACRONYMS AMSTL Active Management of the Third Stage of Labor ANC Antenatal Care ARI Acute Respiratory Infection BTN Beyond the Numbers CA Cooperative Agreement CE Community Educator CEI Client Exit Interview CHP Community Health Promoter CHT Community Health Team CHV Community Health Volunteer CHW Community Health Worker CIP International Potato Center CME Continuing Medical Education DEC Development Experience Clearinghouse DH District Hospital DHC District Health Center/District Primary Health Center DHS Demographic and Health Survey EBF Exclusive Breastfeeding EmONC Emergency Obstetric and Newborn Care EPC Effective Perinatal Care EQ Evaluation Question ET Evaluation Team FGD Focus Group Discussion FM Family Medicine FP Family Planning FTF Feed the Future GAIN Global Alliance for Improved Nutrition GBV Gender-Based Violence GIZ Deutsche Gesellschaft für Internationale Zusammenarbeit (German Agency for International Cooperation) GOT Government of Tajikistan HF Health Facility HH Health House HLSC Healthy Lifestyle Center HMIS Health Management Information System HSS Health Systems Strengthening HW/HCW Health Worker HWS Health Worker Survey ICATT IMCI Computerized Adaptation and Training Tool IFA Iron and Folate Tablets IMNCI/IMCI Integrated Management of Newborn and Child Illness IYCF Infant and Young Child Feeding IP Implementing Partner IPC Infection Prevention and Control IPCS Institute of Pediatric and Child Surgery KfW Kreditanstalt für Wiederaufbau (German Development Bank) KI Key Informant KII Key Informant Interview km Kilometer M&E Monitoring and Evaluation MAD Minimum Acceptable Diet MCH Maternal and Child Health MCHN Maternal and Child Health and Nutrition MDD Minimum Dietary Diversity MHSP Ministry of Health and Social Protection of Tajikistan MICS Multiple Indicator Cluster Survey MiL Mother-in-Law MNCH Maternal, Newborn, and Child Health MOHSPP Ministry of Health and Social Protection of the Population NRC Nutrition Resource Center OB/GYN Obstetrics and Gynecology ORS Oral Rehydration Salts PGMI Post-Graduate Medical Institute PHC Primary Health Care/Primary Health Care Center PPH Postpartum Hemorrhage PMNCH Partnership for Maternal, Newborn, and Child Health QI Quality Improvement RF Results Framework RH Reproductive Health RHC Reproductive Health Center RHFA Rural Health Facility Assessment RHH Rural Health House RMNACH Reproductive, Maternal, Newborn, Adolescent, and Child Health SAM Severe Acute Malnutrition SBCC Social and Behavior Change Communication SDC Swiss Development Council SDG Sustainable Development Goal SRIOG Scientific Research Institute of Obstetrics and Gynecology SUN Scaling Up Nutrition TA Technical Assistance TAWA Tajikistan Agriculture and Water Activity TDHS Tajikistan Demographic and Health Survey THNA Tajikistan Health and Nutrition Activity TOT Training-of-Trainers TWG Technical Working Group UNFPA United Nations Population Fund UNICEF United Nations Children’s Fund USAID United States Agency for International Development VHC Village Health Center VIP Ventilated Improved Pit (toilet) WASH Water, Sanitation, and Hygiene WB World Bank WFP World Food Programme WG Women’s Group WHO World Health Organization WRA Women of Reproductive Age ZOI Zone of Influence i EXECUTIVE SUMMARY EVALUATION PURPOSE The purpose of the performance evaluation of the United States Agency for International Development (USAID) Feed the Future (FTF) Tajikistan Health and Nutrition Activity (THNA) is: 1) to learn to what extent the activities’ results have been achieved as described in the results framework; and 2) to inform the design of future maternal, newborn, and child health (MNCH) activities. USAID will use the lessons learned and challenges identified to develop future projects, with the intention to replicate successful approaches. USAID will share results within the USAID mission, implementing partners (IP), the Ministry of Health and Social Protection (MOHSPP), and different stakeholders. PROJECT BACKGROUND THNA is a MNCH and nutrition project implemented in the FTF Zone of Influence (ZOI) in the Khatlon region by IntraHealth and Abt Associates in collaboration with the Government of Tajikistan (GOT) MOHSPP. THNA combines an existing community social and behavior change communication (SBCC) approach with clinical MNCH quality improvement (QI) and uses participatory methods to involve MOHSPP officials to undertake participatory Rapid Health Facility Assessments (RHFAs) that identify the major gaps in access to quality health care for pregnant women and infants in the “1,000 days” between conception and when the child is 2 years of age. THNA also uses a training-of-trainers (TOT) cascade training methodology with the same MOHSPP and other national health experts to train trainers in District hospitals (DHs) and primary health centers (PHCs) to implement evidence-based interventions in the “window of opportunity” to prevent chronic malnutrition and decrease maternal and child morbidity and mortality. EVALUATION METHODOLOGY The evaluation was conducted between August and December 2019 and used a mixed-methods approach that included: 1) key document and available data review; 2) key informant interviews (KIIs) with MOHSPP officials, IP staff, and District health facility (HF) managers and staff, staff at the Research Institute of Obstetrics Gynecology and Perinatology, the National Reproductive Health Center, and others (94 total); focus group discussions (FGDs) with beneficiaries and their families (26 FGDs; 225 participants); surveys with 124 health workers (HWs); and client exit interviews (CEIs) with 164 women who use the HFs supported by the project. FINDINGS EQ1: To what extent has the activity increased access and improved to (quality) MNCH services including nutrition, sanitation, and hygiene in the ZOI, particularly in the first 1,000 days? The evaluation found that THNA’s QI focus involving all levels of the health system, cascade TOT methodology, strategic supply of MNCH clinical and training equipment and supplies, and specific attention to infection prevention and control (IPC) improved the quality and availability of services known to reduce maternal, newborn, and child morbidity and mortality from baseline levels measured in 2016. QI scorecards of 11 indicators in both DHs and PHCs showed progressive improvements over time, (with variations between districts). Community health ii promoters (CHPs) trained and supported by the project promoted health, nutrition, and hygiene behaviors using effective SBCC methods to promote healthy pregnancy outcomes, child nutrition and methods to prevent diseases and/or mitigate their effects. CHPs were credited with motivating mothers and their families to improve maternal and child household behaviors and successfully promoting care-seeking for prevention and danger signs in pregnancy and childhood illness. As a result of THNA’s support, care-seeking for prevention and treatment services for pregnant women and infants increased, early pregnancy registrations increased, home births decreased and, except for pre-eclampsia, beneficiaries identified danger signs in pregnancy requiring attention. CHPs often facilitated referrals in emergencies. Diarrhea and acute malnutrition cases have decreased, and beneficiaries attribute this to better hygiene, especially handwashing with soap, and better Infant and Young Child Feeding (IYCF) practices, thanks to training received from the project. Young mothers say they now exclusively breastfeed more than they did in the past and they are not giving infants sweet tea, which led to early breastfeeding cessation in the past. They reported they and their children are consuming a wider variety of foods and “cooking demonstrations” provided by the project taught them how to prepare home-based pureed foods for complementary feeding. Using Mid-Upper Arm Circumference (MUAC) tapes during home visits, CHPs identified and referred acutely malnourished children to DHs where THNA strengthened case management capacity. This was confirmed by HF representatives. EQ2: To what extent has the activity increased access to a diverse set of nutrient rich foods throughout the year and why, especially the dietary diversity of pregnant and lactating women and children under 2 years of age? The evaluation found there was increased access and consumption of a diverse diet by mothers, which was achieved through project interventions. Some of these interventions were directly linked to: food preparation; improved agricultural techniques; cooking demonstrations with locally available and affordable foods, including foods from the beneficiaries’ own gardens; and introduction of techniques for preparing appropriate complementary foods for infants. Recurrent household surveys conducted by the project measured significant improvements in multiple standard indicators of improved diets in both mothers of young children and in children under 5 years of age. The project taught improved preservation techniques to make a wider variety of foods available throughout the year and ways of protecting household livestock (particularly chickens) from disease. School children were taught how to improve their diets and demonstration gardens were started for children to learn improved agricultural practices that they would teach to other students and share with their families. On the other hand, introduction of new types of unfamiliar foods such as sweet potatoes, Chinese cabbage, and home cheese were not popular and are unlikely to be pursued. Locally available and affordable foods are also deficient in iron to a large extent and a large percentage of mothers and children have iron deficiency anemia (IDA). iii EQ3: To what extent were (program-supported) evidence-based guidelines and procedures institutionalized by the health systems in the FTF ZOI? And what if any, barriers were there? THNA provided the ET with a list of 19 protocols, guidelines, and standards at the national and regional level that were supported, to various extent, by the project. The MOHSPP Chief Pediatrician confirmed THNA’s involvement in 15 of them, of which 10 have been approved and the other five are expected to be approved soon. MOHSPP’s representatives specifically cited project contributions in maternal care guidelines, IPC, Integrated Management of Child Illness (IMCI) and the IMCI Computerized Adaptation and Training Tool (ICATT), and Nutrition in Pregnancy. The project and stakeholders said that the approval and dissemination process is slow, but necessary to implement new policies and interventions. The Scaling Up Nutrition (SUN) group strategy development has stalled. National stakeholders told the ET that the group working on it has not been very active, the health sector represents the minority of members and therefore has little ability to advocate for more action. At the time of the evaluation, the “1,000 Days SBCC Strategy” was close to approval, but that is the work of many partners and was not led by THNA. The Chief Pediatrician, however, credited the project for many of the approaches adopted in the new strategy. CONCLUSIONS Overall, THNA has implemented the important activities tied to its Results Framework (RF). The dual health facility and community approaches, along with strengthening access to and consumption of nutritious foods complimented and reinforced the effects of each other. All respondents stated the MNCH health and nutritional status visibly improved as a result of project interventions. More work remains to be done in the areas of anemia prevention and control, nutrition during pregnancy, and household sanitation, especially latrines. THNA developed national capacity to assess, plan, train, provide supportive supervision, and monitor programs. Coverage of quality health services was very low at baseline and access to improved services for most of the resource-poor population is far from complete. Further progress will require scaling-up to new areas and strengthening several Health Systems Strengthening (HSS) components that were not THNA’s focus to be sustainable and/or scaled￾up. Progress in several areas is hindered by factors beyond the project’s control, including poor data for decision-making at the local level, centralized management of the MOHSPP, human resources for health challenges and weaknesses in the supply chain for drugs, supplies, and equipment that could be improved with better leadership and management at the regional and district levels. HFs continue to have serious water, hygiene and sanitation problems; however, infrastructure issues are best addressed by the regional governments. EQ4: Going forward, how, and in which areas, should USAID-funded MCH and Nutrition activities be conducted to support the health system and/or population in improving the nutrition and health services to Mothers and Children? The evaluation’s recommendations prioritize strategic investments to target funds and technical assistance to specifically address the most important gaps in maternal and child health and nutrition (including anemia) during the “window of opportunity” to prevent chronic malnutrition in the first 1,000 days. iv • Address gaps in nutrition during pregnancy and faltering maternal and child anemia prevention and control with special attention to: a) cultural and outdated medical advice to restrict intake in late pregnancy; b) overcoming barriers to effective anemia prevention and control in mothers and children; c) expanding emphasis on birth spacing as a way to support MNCH and nutrition; and d) maximizing USAID’s influence in their leadership roles as Chair of the Food Security and Nutrition Group and Co-Chair of the national Scaling Up Nutrition (SUN) movement to motivate members of these groups to take action to address these serious threats to MNCH nutrition and survival. • Support sustainability of THNA QI approaches by advocating that QI be linked to HF accreditation processes and to Continuing Medical Education (CME) credits for individual health professionals. • Support the global “A Promise Renewed” movement and expand QI to Neonatal and Child Health programs. • Support increased access to MNCH health services in rural communities with advocacy to task shift some antenatal care (ANC) services to health houses (HHs). • Consider supporting sustainable country ownership of health systems reform with Leadership and Management training for the regional and district health managers. • Consider supporting a SBCC program devoted to the “1,000 Days and MNCH and nutrition.” • Provide support to the design and implementation of the national community health workers (CHWs) strategy through the healthy lifestyle centers (HLSCs). 1 1.0 EVALUATION PURPOSE AND QUESTIONS 1.1 EVALUATION PURPOSE The purpose of the Tajikistan Health and Nutrition Activity (THNA) performance evaluation is to: 1) determine to what extent the activity’s intended results have been achieved to date; and 2) identify lessons learned, success stories, best practices, and challenges and use them to inform recommendations for the design of future maternal and child health and nutrition (MCHN) and food security activities, particularly those related to implementation of the 1,000 Days Strategy. The results of the evaluation can also inform other programs that will be implemented by other United States Agency for International Development (USAID) Feed the Future (FTF) activities and also contribute to the Multisectoral Nutrition Strategy. The evaluation is intended to provide an independent view of how the activity has been implemented from a wide variety of perspectives, especially the extent that coverage with evidence-based nutrition and maternal and child health (MCH) interventions has been achieved. The evaluation will also address the activity’s flexibility and effectiveness in adjusting to any changes that took place in its operational areas within the Government of Tajikistan (GOT) or within implementing partners (IPs) during the life of the activity. Results from this evaluation are intended to be used by USAID/Tajikistan, as well as the health, nutrition, and development community within Tajikistan and across the region. Evaluation results will further be shared with local stakeholders, such as THNA IPs, the Ministry of Health and Social Protection of the Population (MOHSPP), members of the National Nutrition Forum and other organizations working to implement the 1,000 Days Strategy in Tajikistan, the United Nations Children’s Fund (UNICEF), the World Health Organization (WHO), the German Agency for international cooperation (GIZ), the World Bank (WB), the World Food Programme (WFP), United Nations Population Fund (UNFPA), and other donor or advocacy organizations working in on poverty reduction, agricultural development, and nutrition. 1.2 EVALUATION QUESTIONS The THNA performance evaluation report provides answers to the following three evaluation questions (EQs): 1. To what extent has the activity increased access to and improved MCHN services, including nutrition, sanitation, and hygiene in the FTF Zone of Influence (ZOI), particularly in the first 1,000 days? 2. To what extent has the activity increased access to a diverse set of nutrient-rich foods throughout the year, especially the dietary diversity of pregnant and lactating women and children under two years of age, and why or why not? 3. To what extent were (activity-supported) evidence-based guidelines and procedures institutionalized by the health systems in the FTF ZOI? And were there barriers? 4. Going forward, how and in which areas should USAID-funded MCHN activities be conducted to support the health system and/or population in improving nutrition and 2 health services to mothers and children? 2.0 PROJECT BACKGROUND THNA is a five-year (2015-2020) project under USAID FTF initiative with the goal of improving the health and nutrition status of women and children in 12 southwestern districts of the Khatlon Region of Tajikistan. THNA is implemented by IntraHealth International and Abt Associates in partnership with the MOHSPP for a total commitment of $13.15 million. The overall project rationale is to integrate improved health care service providers with improved health care knowledge, practices, and evidence-based decision-making of families, communities, and policymakers, thus significantly increasing the likelihood of better-quality health service access to healthier populations, particularly women and children, thereby decreasing infant mortality and child stunting. THNA aimed to accomplish this goal through four intermediate results: 1) improved quality of health care services for maternal, newborn, and child health (MNCH); 2) increased access to a diverse set of nutrient-rich foods; 3) increased practices of health behaviors around MNCH; and 4) institutionalization of evidence-based MNCH services through national-level policies and standards. To achieve these results, THNA implemented a range of interventions and strategies that specifically focused on the 1,000-day “window of opportunity”—between conception and a child’s second birthday—to make a lasting impact on the child’s growth, learning, and future productivity. To improve the quality of health care services, the project provided MNCH equipment and presented a package of the WHO’s “effective perinatal technologies,” or Effective Perinatal Care (EPC) to improve the quality and outcomes of care for mothers and newborns by updating and improving professional and managerial MNCH knowledge and skills for MNCH medical personnel targeting District Hospitals (DHs) and Primary Health Care Centers (PHCs). Promoting maternal and child nutrition and healthy behaviors around MNCH also utilized a family and community￾centered approach that collaborated with Community Health Promoters (CHPs), community educators (CEs), health providers, and family decision-makers to adopt MNCH, hygiene, sanitation, and overall nutrition knowledge and best practices. Finally, the institutionalization of evidence-based MNCH services was manifested through the project support and advocacy for policy reforms on poverty reduction, agriculture, nutrition, and health to better facilitate a conducive environment for project MNCH best practices and achievements to take hold and be supported by a diverse coalition of national policies and standards. The performance evaluation was conducted by an evaluation team (ET) of three MNCH professionals, consisting of one international team leader and two local experts. The evaluation methodology, results, and findings, conclusions, and recommendations are presented in the following sections. 3.0 EVALUATION METHODOLOGY The ET used a mixed-methods evaluation design, employing a combination of qualitative and quantitative methods. Qualitative methods included desk review of key project documents; review of secondary sources of information and analysis, including documents published by the 3 government and other donors, and nutrition and health systems technical documents; and key informant interviews (KIIs) and focus group discussions (FGDs) with health providers, beneficiaries, community members, and other activity stakeholders. A list of key documents consulted can be found in Annex 4. Quantitative methods included collecting quantitative data from the project’s reports and other sources and conducting two surveys: a health worker survey (HWS) and client exit interviews (CEIs). 3.1 SAMPLING The ET used a multi-stage sample design for the surveys. In the first stage, purposive sampling was employed to select six of the 12 THNA operational districts, reflecting the geographic distribution of the project activities that can be visited within the timeframe of the evaluation. Some of the selected districts had also been supported by the Tajikistan Agriculture and Water Activity (TAWA), and other USAID FTF projects, and/or other health and nutrition activities funded by other donors, including the WB, German Development Bank (KfW), WFP, GIZ, and UNICEF. The second stage of sampling consisted of the selection of health facilities stratified by type: DH, PHC, and village health center (VHC). The DHs and PHCs in each sample district were automatically included in the sample. VHCs were selected using proportionate sampling weighted by the number of VHCs in each district. The third and final stage of sampling involved random selection of respondents for the health worker (HW) survey and client exit interview within the selected health facilities. Two communities, located at least 5 kilometers (km) from a health facility (HF), were selected within each district for beneficiary and community FGDs and KIIs. 3.1.1 Health Worker Survey The sample survey was administered to staff at THNA-supported health facilities, which were selected randomly. Overall, 124 HWs were surveyed. 3.1.2 Client Exit Interview These interviews were conducted with women of reproductive age (WRA) at the THNA￾supported sample health facilities that include the District Primary Health Care Centers (DPHCs) and VHCs. Mothers of children under 5 were selected for CEIs related to child health services. Enumerators interviewed the WRA as they were exiting the HF immediately after the visit. The survey team targeted four sub-groups of WRA: pregnant women; mothers with sick child (diarrhea and acute respiratory infection [ARI]); mothers with healthy child (accessing preventive care services); and women receiving family planning (FP) services. Overall, 164 WRA were interviewed. Table 1: Sample Health Facilities, Health Workers, and Clients Type of Health Facility Sample Facilities HWs CEIs DH 6 72 - DPHC 6 30 120 VHC 11 22 44 Total 23 124 164 The quantitative survey was complemented by a qualitative component that, using purposive sampling, sought to gain insights into the project’s results and outcomes from a diverse group of 4 beneficiaries and stakeholders, including pregnant women and mothers of children under 2 years of age as well as their key family members, especially mothers-in-law (MiLs), that culturally have the most influence on preventive and health care and nutrition behaviors of young mothers, HWs, and stakeholders at community and district levels (e.g., CEs, CHPs, community leaders, and influential decision-makers). While the quantitative survey focused on DPHCs and VHCs. The team visited a small number of rural health houses (RHHs), but since they were not the target of the project, they were not included in either the quantitative or qualitative components of the evaluation. 3.1.3 Site Visits Maternity and pediatric departments were visited in all DHs and Family Medicine (FM) and Reproductive Health (RH) departments were visited in each DHCs of the six districts. Systematic random sampling was done from the lists of VHCs that were provided by the project (see Annex 6). In cooperation with USAID, the proposed list was finalized, considering potential replacement HFs, if needed. See Annex 6 for the final evaluation samples. 3.2 DATA COLLECTION METHODS 3.2.1 Desk Review The desk review started with a review of project documents provided by USAID and secondary data from documents pertaining to: relevant aspects of the activity and MCHN services in Tajikistan; the latest global health, nutrition, and food security materials on evidence-based MCHN and food security interventions that have been proven to improve child nutrition status, especially stunting and anemia; and reports from other health, nutrition, and food security projects in Tajikistan, Central Asia, and the world. The review included evaluation reports from similar and related projects, such as: The Final Evaluation of TAWA (2018), USAID’s Food for Peace (FFP) program evaluations, the Child Survival and Health Grants program, and the MCHN technical support documents from USAID’s centrally-funded MCHN and nutrition technical programs. A list of documents reviewed is presented in Annex 4. 3.2.2 Key Informant Interviews The ET conducted KIIs with a range of activity stakeholders at the national, regional, and health facility levels to gain insights into their experiences with THNA, their perceptions of the activity, how the activity is being implemented, its relevance to priority health and nutrition needs of the country, the effectiveness of the project approaches, and the likelihood that changes supported by the project will be sustained. In Khatlon, the ET conducted KIIs in DHs and DPHCs in all six districts and at the Regional level, including relevant district and regional health and nutrition government stakeholders and THNA staff. Group interviews were conducted with project CEs and CHPs, and recipients of THNA MCHN training, including trainers and quality improvement (QI) teams. (See Annex 5 for a list of individuals and/or offices contacted and interviewed). 3.2.3 Focus Group Discussions FGDs were conducted in a total of 12 communities (two communities per district), each located at least 5 km from the nearest health facility. In each district, at least one FGD was conducted in each activity-supported community with beneficiary clients including pregnant women, mothers of children under 2, and “influencers” of pregnant women and young mothers such as husbands, 5 MiLs, and community leaders. In all districts, more than one category of beneficiaries was interviewed. In most cases, more than one group per category was interviewed per district. Due to seasonal agricultural activity, few men were available for interviews. The ET was able to interview one group of men with young children or grandchildren as well as two (male) community leaders. Topics covered during FGDs were directly related to THNA’s efforts reflected in the project’s Results Framework (RF). Both the KII and FGD discussion tools used pre￾approved, standardized tools translated into the local language (Tajik or Uzbek). One community randomly selected by the ET was Turkmen-speaking but was successfully interviewed using Uzbek and Russian. Participation in the KIIs and FDGs was voluntary, and the ET received verbal informed consent prior to proceeding with the discussions. 3.2.4 Health Worker Surveys and Client Exit Interviews Quantitative tools were translated into Tajik and Uzbek by Z-Analytics, the local firm that conducted the survey, and reviewed by native speakers on the ET and pre-tested in project HFs not selected in the evaluation sample. HWSs were administered to staff that had participated in THNA-supported trainings in sampled project health facilities. In addition to collecting information on types of trainings and attitudes on the usefulness and quality of training using “yes” or “no” questions, in the survey questions the ET also included the use of Likert-style numeric codes in order to judge the intensity of the HWS opinions. CEIs asked about women’s experience with specific health and/or nutrition services (e.g., antenatal care [ANC], FP, child health, etc.) supported by THNA. Survey instruments were drafted in English and translated into the relevant languages. Participation in the surveys was voluntary and responses were anonymous to encourage frank and open answers. 3.3.5 Data Analysis The ET conducted analysis of the quantitative and qualitative data gathered to answer the three evaluation questions. Qualitative Data Data collected were systematically summarized to draw out trends and key findings and analyzed by identifying thematic areas that emerged from the aggregated responses from individual and groups to common questions asked to each kind of informant. Where outliers were found, they are identified separately in the evaluation results. The ET triangulated these key findings from the qualitative survey with results from the quantitative survey to identify areas of agreement and any differences. Quantitative Data The HW survey and CEI survey data analysis was conducted using Excel software to generate frequency and summary statistics, including measures of central tendency and contingency tables and charts/graphs showing how different respondent groups responded to different survey questions and whether these differences are significant. Evaluation Limitations • Finding WRA clients of the HFs was difficult as the evaluation field work coincided with the cotton harvesting season. There were not as many women visiting the HFs at the time 6 of the ET’s visit to the facilities. • Due to seasonal employment in the cotton harvest and migration to Russia for employment, the ET was able to speak to only a limited number of men. • Difficulties in obtaining permission letter from the Ministry of Health of The Republic of Tajikistan to conduct the data collection in the Khatlon regions. • Of the major international partners, the WHO was unavailable. • Due to previously planned field work, the ET was unable to attend the National Nutrition Review held in Dushanbe in early October 2019. • With the exception of a few late adolescent pregnant women that participated in community FGDs, the ET was able to access adolescents (girls and boys) at only at a few schools. Students in school may not be representative of all adolescents. 4.0 FINDINGS BY EVALUATION QUESTION (Note: this report focuses on findings that most accurately respond to the EQs and should not be considered a comprehensive list of all THNA activities in the project RF. These can be found in detail in the THNA quarterly and annual reports submitted to USAID. Findings are listed by priority to USAID). 4.1 EQ1: TO WHAT EXTENT HAS THE ACTIVITY INCREASED ACCESS AND IMPROVED TO (QUALITY) MNCH SERVICES INCLUDING NUTRITION, SANITATION, AND HYGIENE IN THE ZOI, PARTICULARLY IN THE FIRST 1,000 DAYS? FINDINGS 4.1.1 Improved Access and Quality of Health Services THNA is a MNCH and nutrition project intended to build MOHSPP’s national capacity to design, implement and monitor MNCH and nutrition health programs at the district, HF, and community levels. IntraHealth with partner Abt Associates were not direct implementers; they provided technical support to develop training, monitoring, coaching, mentorship, on-the-job training, and QI approaches from initial problem analysis through all MOHSPP processes that can result in sustainable change. THNA supported the MOHSPP to conduct a Rapid Health Facility Assessment (RHFA) that: 1) used adapted WHO indicators and scorecards in EPC, Infection Prevention and Control (IPC), and Integrated Management of Childhood Illness (IMCI); and 2) conducted KIIs with national-level experts and authorities from the MOHSPP (2), the Republican Family Medicine Center (1), the MCH Center (Research Institute of Obstetrics, Gynecology, and Perinatology) (6), the National Reproductive Health Center (1), and the Pediatric Research Center (1). The RHFA’s tools supported self-assessments and participatory analysis that identified baseline conditions at the DHs and PHCs. Joint analysis of the findings helped the participants to determine the priority 7 MNCH and Nutrition areas and applicable evidence-based interventions, as well as the steps needed to implement them. In KIIs with RH and Child Health national-level MOHSPP managers, key informants (KIs) told the ET that when they compared the RHFA findings with “official” data submitted through the national health management information system (HMIS), they “were shocked to discover that (official statistics) seemed to reflect the situation as it should be, rather than the real situation.” They said that these findings motivated them to engage with the project and their MOHSPP colleagues to do what they could within their positions to improve conditions and services. KIIs conducted by the ET confirmed that the findings in project reports, according to which data analyzed, were used were collected from participatory RHFAs and QI self-assessments conducted by the national-level experts trained by the project. Data were analyzed by facility, aggregated by district, and shared with the MOHSPP and stakeholders for planning capacity building and monitoring progress against standard international MNCH indicators. MOHSPP KIIs said that the RHFAs and other assessments (including infection control) were “real eye-openers” for national managers and academics who had primarily relied on the national reporting mechanisms and periodic surveys (like the Demographic and Health Survey [DHS] or Multiple Indicator Cluster Survey [MICS]) to understand the reality of the situation in HFs and client behaviors. (Neither the DHS nor the MICS examine HFs at the specific district level). Analyzing the findings led MOHSPP authorities to conclude that the data they received through the national health statistical system “may not be accurate.” Data analysis helped the MOHSPP and the project identified the key policies, standards, and protocols that would need to be modified to allow new evidence-based interventions and processes to be introduced into district HFs. (See EQ3 for a list supplied by the project). KIIs with the PHC and DH managers and deputy managers at the district level revealed that at the facility level, routine data from HFs are collected and forwarded to the national level for analysis but not analyzed locally, or even at the regional level. During the ET’s field visits to district HFs, the team confirmed that aside from QI data developed later as part of the project activities, locally available data were of poor quality, or not easily accessible. MOHSPP’s RH and Infection Control Obstetrics and Gynecologists (OB/GYNs) told the ET that data analysis from the RHFAs revealed large discrepancies between the “real” and “reported” situation related to conditions in HFs’ and HWs’ performance. They gave an example where the national standard was “100 percent of newborns received all four elements of essential (newborn) care” and that is what DH maternities reported. However, follow-up interviews with patients and medical personnel revealed that was not true.1 According to THNA reports, the national MOHSPP KIIs stated that “the findings were bleak.” DH maternities were chronically understaffed and underequipped, experienced ongoing shortages of most medical supplies, and suffered from poor quality control and managerial oversight. 2 In addition, most facilities had problems with water supply, handwashing practices were not diligently followed, and infection rates were higher than they should be. Hazardous waste management was universally poor and, in almost all facilities, laundries and incinerators were either not available or malfunctioning. Medical equipment was old, in poor condition, or missing altogether, and there were chronic shortages 1 THNA project, First Annual Report, 2017. 2 Ibid. 8 of disposables. 3 Standard disciplinary practices were used (and continue to be used in some places) to discipline HWs, including termination for unfavorable outcomes, such as maternal deaths. This happened to the manager of one hospital visited by the ET who was fired because of a maternal death that may not have been avoidable. The lack of supportive supervision and mentoring provides incentives for managers to avoid revealing problems or to submit false reports. KIIs with national level MOHSPP authorities also revealed that IPC at DHs and PHCs was one of the most striking deficiencies identified in the baseline RHFA and other facility assessments conducted when the program started. Some deficiencies identified, especially major water and sanitation infrastructure improvement, were beyond the scope of the THNA project. IPC findings led to a specific QI focus that included intensive training on handwashing, separating “clean” and “dirty” instruments and putting them into separate rooms, and following disinfection protocols and proper use of autoclaves provided by the project. Curiously, despite the deficiencies, RHFAs also found that “even though chronically overworked and understaffed, health care workers (HCWs) were still motivated to provide patient care and improve their individual professional qualifications,” which is indicative of a positive enabling environment for training effectiveness targeting front line HCWs. “When I looked at the baseline RHFA and some investigations I did on my own, I wept….I could not believe that conditions were actually that bad (unsanitary). With the support of the (THNA) project, we put a plan into action to address the deficiencies.” (KII with National MCH Center OB/GYN responsible for Infection Control) MOHSPP and IP managers said that they jointly identified skilled and experienced internationally qualified national experts from across the country to develop a multi-year curriculum and serve as a training-of-trainers (TOT) for key MNCH topics. THNA participated in national Technical Working Groups (TWGs) and facilitated meetings of the MOHSPP and stakeholders to raise national awareness about key MNCH areas for intervention. Several MOHSPP national, regional, and district officials confirmed that they had attended these meetings. The project also trained a team of national MOHSPP experts and a team from the Scientific Research Institute of Obstetrics, Gynecology (SRIOG) to conduct the baseline and recurrent RHFAs of district and provincial hospital maternities and six PHCs. This group of experts became the trainers and mentors that conducted trainings at the DHs and PHCs. KIIs with MOHSPP personnel and HF managers and HW trainees confirmed activities and approaches reported in project reports. The project provided support for training activities for TOTs at each level of the cascade and provided limited MNCH equipment and supplies to enable HWs to implement what they had learned. Later in the project, THNA helped refurbish and equip rooms in HFs to be designated as Nutrition Resource Centers (NRCs) (sometimes referred in documents as Training Resource Centers). After the district-level TOTs were completed, national-level trainers were responsible for follow-up mentoring, coaching, and on-the-job training in periodic follow-up visits. HF technical support for quality health services primarily targeted maternity units with targeted training to support inpatient IMCI and management of severe acute malnutrition (SAM) in the pediatric wards. At the PHC level, the project focused on 3 Ibid. 9 FM and RH departments where WRA and children receive routine outpatient preventive and curative services. Cascade PHC TOT was also extended to HWs from VHCs. Improvements of clinical practices and procedures, measured through periodic RHFAs and (later) QI team self-assessments, were made by the HF employees themselves, supported with mentoring and coaching from the national-level trainers. HF managers and staff members said that even though NRCs provided by the project have only recently opened in some HFs, they are very valuable, especially the mannequins.4 Access to an NRC provides opportunities to practice new skills when it is convenient for the HW. NRCs are equipped with sinks and soap to emphasize the importance of handwashing for infection prevention. National-level trainers also provide support to IMCI centers with updated WHO IMCI recommendations and IMCI Computerized Adaptation and Training Tool (ICATT)/IMCI software packages and online video conferences. The ET noted examples of clinical improvements supported by the project during HF visits and KIIs and group interviews with HWs. These and satisfaction with the services received were also confirmed by the CEIs with 164 women accessing services supported by the THNA project (73 percent in DHs and 27 percent in PHCs). Over 98 percent of them expressed great satisfaction with the overall experience receiving services (see Figure 1). 5 Their satisfaction was echoed in community FGDs with beneficiaries. Figure 1: Level of Women/Clients Satisfaction (n=164) Health Worker Training The ET conducted a survey of 124 randomly selected HWs trained by the project that were present on the day the surveyors visited the HF. Type of HWs surveyed is represented below. 4 Infant mannequins provide opportunities for staff to practice newborn resuscitation and improve their skills. 5 Z-Analytics, THNA Client Exit Interviews, Final Performance Evaluation, October 2019. 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Overall satisfaction with the services received Waiting time to see the health care provider Cleanliness of the health facility Privacy at the exam room Time given to you by the health provider Health provider attitude with you Satisfied Uncertain Not Satisfied Photo 1: DH NRC 10 11 Figure 2: Type of Health Workers Surveyed Two-thirds of trainees were nurses or midwives; 82 percent had three or more years of training in their area of specialty; and 67 percent had been in their positions for more than three years. A minority of trainees said that they had been trained by THNA staff (including national trainers) or “colleagues” (consistent with a cascade training methodology) and the trainings were “appropriate and useful.” “Increased knowledge and improved clinical skills” was cited as the primary benefit. The survey participants recommended that future trainings be longer and more frequent (including refreshers). By Year 4, project reports said approximately 50 percent of eligible workers across all project districts had been trained in nutrition counseling and/or received clinical on-the- job training; however, only 25 percent of eligible PHC providers in FTF districts had been trained in prevention and management of anemia. (The Year 4 Quarter 3 report cites additional anemia trainings planned by THNA that may not be reflected in survey findings). More specific feedback obtained from KIIs with HWs and DH Maternal and Pediatric managers confirmed that topics covered were consistent with evidence-based priorities identified in the beginning of the project. Survey participants and HF KIIs both said that maternal and newborn mortality and morbidity as well as cases of (acute) malnutrition had decreased since the project began. PHC FM doctors said they thought reduction of acute malnutrition cases requiring hospitalization reduced as a result of earlier identification and referral of malnutrition done by CHPs. In the HWS, sample respondents were asked to rate the various MCH and Nutrition training topics6 that THNA provided, on a scale of 1 to 5, one being the worst and five the best score, using criteria that include extent of learning, the length of training, content of training and usefulness of training. Overall, the trainings scored high across all the four criteria ranging from a minimum of 4.4 for length of training to a maximum of 4.7 for usefulness of training. A summary of the ratings against the four criteria is summarized in Figure 3. 6 Training topics include: Emergency Obstetric and Newborn Care (EmONC); Kangaroo Mother Care; Partogram management; EPC; Infection control; ANC; Anemia; Child nutrition/Infant and Young Child Feeding; Integrated Management of Malnutrition (IMAM); IMCI Integrated Management of Childhood Illnesses; Supportive supervision; and Quality improvement/Beyond The Numbers. 3.2% 32.3% 29.0% 1.6% 7.3% 8.9% 11.3% 6.5% Obs/Gyn Hospital Nurse Midwife Feldscher Other Family medicine doctor Family nurse at PHC Pediatrician 12 Figure 3: Average Scoring of Opinion for Trainings (n=149) In collaboration with the Institute of Pediatrics and Child Surgery (IPCS), the project also trained 24 PHC pediatricians using adapted WHO guidelines on clinical management for pediatric triage and emergency conditions. Both DH and PHC managers said referrals from PHCs to DHs and coordination between facilities had improved as a result of this training. HWs were asked if there had been any changes related to the health and nutrition in pregnant women and children under 2 years of age since the project began providing support to their HF. Figure 4 below indicates that reduced morbidity (sickness) rate and reduced malnutrition were the most frequently mentioned. Figure 4: Percentage of Respondents by Type of Changes Identified (n=123) KIIs with DH managers and/or deputy managers (N=6), Heads of Maternity Departments, chief pediatricians, midwives (N=22), and nurses (N=18) all said that early registration for pregnancy (ANC within the first 12 weeks) had risen dramatically as a result of project activities. They confirmed that increased referrals for acute malnutrition reported elsewhere had occurred due to screening and referrals from CHPs that identified both sick and malnourished children and referred them to care when their conditions were more easily treated. Home births, which were already low, have further decreased due to promotion of HF births by HWs and CHPs. All participants interviewed said that EPC training and mentorship, especially consistent and correct use of the partogram and Active Management of the Third Stage of Labor (AMTSL), have reduced life-threating postpartum hemorrhage (PPH), cases of pre- and eclampsia and hemorrhagic shock, and they improved the newborn resuscitation skills. Informants also specifically mentioned Criteria Low Score (1) 2 3 4 High Score (5) Extent of learning Learned nothing Learned a lot Length of training Not appropriate Very appropriate Content of training Not appropriate Very appropriate Usefulness of training Not useful Very useful 4.5 4.4 4.7 4.5 67% 41% 53% 27% 24% 10% 4% 1% 35% 0% 10% 20% 30% 40% 50% 60% 70% 80% Reduced morbidity rate Reduced mortality rate Reduced malnutrition Decrease in the number of home deliveries Improved quality of services Staff Acquired valuable knowledge and skills Strengthened providers networks Other Reduced Anemia 13 MNCH specific equipment and supplies provided by the project (such as autoclaves, oxygen concentrators, etc.) and the introduction of NRCs. Project reports, supported by district-level KII informants interviewed by ET, indicate that national trainers trained by THNA, helped form QI teams, primarily in maternity units and PHCs. QI teams collected and analyzed data on 11 standard indicators for each type of facility, and developed action plans to correct deficiencies. QI teams, which included many staff trainers, explained how they were able to analyze data they had collected on specific QI indicators. Maternity QI analysis using the reviews of individual cases of maternal “near-misses” in facilities in the frame of the WHO “Beyond the Numbers” (BTN) methodology, strengthened continuous improvement of the quality of maternal and newborn care and outcomes and BTN cross visits to share best practices with other HF QI teams. QI teams said that the QI approach “brought together” the various kinds of support services provided by the project (training, equipment, supplies, mentoring, applying clinical standards, record keeping, analyzing data, etc.) and enabled them to identify and focus on specific priority MNCH problems. The HW survey and KIIs and group interview feedback on training and follow-up support at all levels, including national MOHSPP officials, HF managers, and HWs, indicated the quality of training and technical support provided by project was high-quality and effective. Multiple clinical MNCH indicators showed improvement toward measurement targets. Specific improvements presented in project reports and confirmed in KIIs included: 1) early registration for pregnancy increased significantly at the PHCs; 2) DHs are now better staffed and equipped (emergency obstetric and newborn care, or EmONC); and 3) newborn resuscitation is performed better and the availability of infant mannequins provides opportunities for staff to practice and improve their skills. Midwives and OB/GYNs stated they more consistently perform routine (physiological) and complicated deliveries according to existing national protocols and are more competent in EmONC procedures. QI teams and their monitoring were cited by HF managers as helping hospital maternity units to identify quality issues and formulate quarterly action plans to address deficiencies. QI was introduced only in maternity units, but according to THNA, all 12 pilot DHs they support have expanded QI to their other departments after they observed the impact of the approach. Cross-visits between districts have allowed staff from weaker performing facilities to observe and learn from stronger performing facilities. Both DH and PHC managers confirmed findings reported elsewhere that coordination between the PHC (outpatient) facilities and the DH (inpatient) facilities has improved. DH HWs specifically identified ICATT training, along with laptops as particularly useful for self-directed refresher training. The project also provided training in improved treatment for children hospitalized with SAM in hospital pediatric units. Informants, however, said that some significant gaps remain in hygiene and sanitation in the DH overall. (The ET observed some serious deficiencies in toilet cleanliness and handwashing facilities in DHs other than inside maternity units). Water, sanitation, and hygiene (WASH) activities to DH facilities is beyond the focus of the project. HWs and managers also mentioned the important contributions of autoclaves and oxygen concentrators to support IPC and EmONC. 14 Infection Prevention and Control (Sanitation and Hygiene) Figure 5 below shows an assessment of infection control and clinical safety indicators at 12 DHs. Figure 5: Infection Control and Clinical Safety Indicators at the Hospital Level Source: THNA Year 3 Annual Report. While all districts showed improvements, only one district (Yovon) exceeded an average score of 70 percent. The evaluation survey of HWs (n=124) asked them about their IPC behaviors. The findings yielded low results, with only 65 percent of HWs saying that they followed the procedures of handwashing with soap (see Figure 6 below). Some indicators, such as clean water storage or improved waste management were particularly low. These findings could inform the trained national experts that provide mentoring support to trained HWs to determine whether they perform all of their infection prevention responsibilities as they were trained. Figure 6: Percentage of Respondents by Type of Infection Preventing Procedure Identified (n=124) Source: Evaluation Quantitative Health Worker Survey October 2019. When HWs were asked about their infection control (WASH) behaviors, the most frequently mentioned (88 percent) was washing hands with soap. Handwashing with soap at key times is necessary to prevent disease in the child and some handwashing behaviors impact disease 51% 22% 65% 75% 19% 18% 0% 10% 20% 30% 40% 50% 60% 70% 80% Improved disinfection Improved waste management Washing hands with soap Proper glove use Clean water storage Preparation and use of antiseptics 15 transmission to the whole family. In the survey, responses from some showed lower counseling on some handwashing behaviors than on others. Figure 7: Infection Control Practices Mentioned by HWs (n=124) Use of improved latrines was not mentioned and observations of household latrines revealed that very few had covers to prevent flies from entering. The project had constructed some demonstration “improved” latrines. One community leader showed the improved (ventilated improved pit or VIP) latrine at the closest VHC. He indicated that it was highly acceptable and affordable to households (approximately $250) and he thought some families would probably construct them. WASH components of the project were measured in the periodic household survey of women with children under 5 years and tracked “increased soap observed at handwashing locations” (a global WASH indicator) and separated the 2018 findings by households trained by the project. These findings corresponded with community FGD findings with pregnant women, mothers of children under 2 years old, and MiL interviews conducted by the ET. WASH hygiene behaviors, specifically handwashing was supported by CEs at the community level. This reinforced HWs counseling for beneficiaries as well as the social and behavior change communication (SBCC) activities promoted by CHPs during home visits. Access to quality MNCH and Nutrition services through community-based SBCC The community SBCC approach introduced by Mercy Corps and expanded by THNA is a cross￾cutting approach that supported results for both EQ1 and EQ2. Mercy Corps’ original MCH program followed a methodology implemented in USAID-supported community MNCH programs in resource challenged environments for over three decades. It goes well beyond the more familiar Information, Education, and Communication (IEC) “health education approach” and supports uptake and sustaining desired behaviors by having repeated contacts with beneficiaries or those targeted for behavior change through home visits and participation in community health and nutrition campaigns such as World Breastfeeding Week. In the original design, CEs’ focus was more devoted to increasing access to supplies of foods for a diversified diet, improved techniques for improving food storage for access to fruits and vegetables during the winter 88% 40% 44% 56% 35% 28% 1% 6% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Washing hands with soap Washing hands after using toilets Washing hands before preparing food for the baby Washing hands before feeding the child Washing hands after cleaning the baby Using hand sanitizers Other Using latrines Percent Response 16 months, and improved animal husbandry, particularly protecting poultry from diseases. CHPs and CEs often worked together to support behavior change in beneficiaries and their families. They identified households with pregnant women and women with children under 2 years of age and facilitated, reinforced, and encouraged positive health and nutrition behaviors within these households. Many volunteers of both types are mature women and MiLs who are the household members with the most influence on mothers. They also have credibility with households that they visit. Health managers and deputy managers in KIIs at PHCs said that CHPs are the ones that have done the most to identify pregnant women and get them into early registration, which then provides access to several MNCH and nutrition services. CHPs also disseminate information about “danger signs of pregnancy and delivery” and contributed to timely referral of pregnant women to HF and decreasing the number of emergency cases in hospitals. They also received recognition in the KIIs to have used Mid-Upper Arm Circumference (MUAC) tapes to identify and refer malnourished children for treatment earlier than in the past and when intervention is more likely to be successful. They promote household hygiene and sanitation, especially handwashing at key times and keeping the environment around the houses clean. FGDs with pregnant women in seven communities (with seven to 12 participants each) showed that they changed their behaviors and registered early for pregnancy, went for blood tests, and took iron tablets (when they were prescribed). They said the CHPs encouraged them to eat a varied diet and not to eat “junk food” and “Russian noodles.” They were encouraged to eat “five or six times a day” and eat “a diversified diet.” The CHPs also used the “Maternal and Child Health Handbook,” which was reproduced by the project to cover a variety of MNCH topics including danger signs of pregnancy requiring treatment and how many ANC visits need to be made. Pregnant women interviewed in FGDs were able to accurately mention most dangers signs of pregnancy requiring treatment, especially bleeding, fever and abdominal pain, and headache but were able to remember “swelling” (a sign of pre-eclampsia) without probing. Although nutrition during pregnancy is a focus of the project and nutrition counseling was reported by HWs and in CEIs, FGDs with pregnant women in all districts showed that women believed, and CHPs supported, the advice that pregnant women should “reduce the amount of bread” (the staple food) during the last trimester of pregnancy “in order to avoid having a fat baby and a difficult delivery.” When asked where they got this information they said, “from health workers.” OB/GYNs on a QI team in one hospital confirmed that is the information they give during ANC. The ET followed up with the MOHSPP and reviewed the English version of “Nutrition in Pregnancy” developed in collaboration with the project. The instructions in the revised guidelines are correct, but the new guidelines have not been disseminated. Two FGDs with pregnant women told the ET information about iron and folate tablets (IFAs) during ANC that contradicted information from other sources. Only two out of 10 or 11 pregnant women were receiving them, even though PHCs did not report any stock-outs of supplies. The two women that were receiving tablets only got them after they “had a blood test” and “were found to be anemic.” Other FGD respondents said that they were not advised to take IFAs, even if to purchase them. In other community FGDs with both pregnant women and mothers of children under 2 years old, they said that “in the past (2016-2018), they received IFA for anemia prevention and treatment. But starting in 2019, the HWs now only provide IFA after the woman has had a blood test that says they are anemic.” HWs interviewed after these FGDs 17 confirmed that what the women in the FGDs told the ET was true and it was due to lack of IFA stocks. Without district-specific data, the ET could not determine whether the women were not taking IFAs because they were not given them, if they did not understand the importance of taking them, or for other reasons. The CEI conducted by the ET quantitative team indicated that women said they were given IFA and counseled on the importance of taking them. Further probing revealed that anemia in pregnancy or in children is not perceived to be a problem at the household level. “I feel fine and my Mother-in-Law doesn’t think (IFA) tablets are necessary.” Dietary iron sources were not emphasized in nutrition counseling from HWs or by CHPs and CEs. The reasons for this could not be identified by the ET. (More details about issues surrounding anemia in pregnant women and children can be found later in this report). While the 2017 DHS found some increases in initiation of breastfeeding within the first hour of birth, breastfeeding practices had changed little since the 2012 DHS and some indicators had actually declined. 7 FGDs held in six communities with mothers of children under 2 years of age (with eight to 12 participants each) found that mothers reported that both CHPs and HWs promoted exclusive breastfeeding (EBF), which they said they are doing, and that they are no longer giving sweet tea to their children, 8 which were the causes of decreased milk supply and early cessation of breastfeeding due to “insufficient milk supply” in the past. Additionally, they noted that EBF knowledge increased the income of the family through not procuring the additional dry milk “Malish.” They said that cooking demonstrations done by the project had showed them how to prepare “pureed” foods to feed their children 6-24 months more frequently. Both CHPs and CEs introduced new foods, especially apples, eggplants, pumpkins, and produce from home gardens, to their children and how to properly use the land every year. In the household hierarchy, MiLs are the strongest influence on maternal behaviors. Daughters￾in-law are expected to do most of the housework and cooking for the extended family. MiLs in seven FGDs (with nine to 11 participants each) reported that in the past they did not make the connection between nutrition and the health and development of the child, which they do consider part of their responsibility as MiLs. Young mothers reported they received dramatically more support from their MiLs after both the mothers and the MiLs were trained by the project. Some mothers reported that their MiLs helped facilitate early ANC registration, accessing health services, “eating five to six times per day” during pregnancy and some helped to prepare the “puree” complementary feeding introduced by the project. Daughters-in-law (in FGDs with pregnant women and mothers under 2) said that training by CHPs had improved their relationships with their MiLs. “The CHPs in the project achieved a ‘peace accord’ between us and our Mothers-in-Law. They now help us to prepare the ‘puree’ for our children and encourage us to go for health care for ourselves and our children.” (Mother of Child under 2 years of age in Community FGD). All respondents in the three types of FGDs organized by the ET were asked if they had noticed any change in the health and nutrition status in mothers and children since the project began. They said that there was “less diarrhea and less malnutrition,” “better hygiene,” and “fewer emergency trips due to pregnancy danger signs.” They also said they went for medical care more frequently because health workers “were nicer to them” and “gave them more useful information 7 DHS 2017. 8 In the KII with WFP, the representative identified the traditional practice of providing sweet tea to infants instead of breastmilk undermined breastmilk production and was a leading cause of early cessation of EBF. 18 and answered their questions.” They also said that HFs were in better shape and “cleaner” than in the past. This information was triangulated with the CEI responses where beneficiaries found HFs to be clean. The effectiveness of the THNA community SBCC approach was recognized by the MOHSPP at each level. The Regional health office told the ET that they would like to replicate the community￾based approach in other districts in Khatlon. The national Chief Pediatrician described the importance of the community-based approach through SBCC. Specifically, he identified: 1) CHPs’ role in identifying pregnant women and motivating them to register in the first trimester; 2) CHPs’ role identifying and referring malnourished children for treatment; and 3) CHP household-level focus on nutrition, disease prevention (handwashing, immunization), and care-seeking for danger signs. PHC directors in KIIs said that the role of volunteers (CHPs) in giving information to women and their families and motivating them to change behaviors and use health services was particularly valuable and that THNA was the first to implement the right “results-based approach” at the community level. The MOHSPP told the ET that there were many donors that work with communities, but in their opinion, THNA used the right approach and was much more successful. He also said that they are planning to scale up the (community) approach nationally and will put many aspects of the project’s SBCC strategy into the National Health Strategy (2020-2030). 4.1.2 Constraints in Improving Clinical Health Care Services Managers complained that lack of funds prevent them from fulfilling their responsibilities to provide quality health services, especially in upkeep and repairs to DH water, hygiene and sanitation systems outside of maternity departments. One director said, “The funds I get for cleaning the hospital for a year are used up in a month.” Water shortages in challenged districts of the Khatlon region also limit the ability of large facilities to conduct routine cleaning of toilets used by patients, facility staff, and visitors. Several facilities visited by the ET had no handwashing station available near the toilets other than those found in the maternity unit. Child health departments of the DHs, where THNA provided training in inpatient IMCI and treatment of SAM, do not have appropriate bathrooms and toilets. This has implications for IPC in those departments. Midwives and OB/GYNs on one QI hospital team complained that they had been promised an incinerator to burn placentas from deliveries but had not yet received one, so they continue to bury them. (The project reported that they provided a few small incinerators to some hospitals.) Cost and distance to obtain key ANC diagnostic tests to identify and treat anemia and pre￾eclampsia, and supply stockouts of IFAs hinder implementation of evidence-based interventions intended to prevent life-threatening conditions in pregnant women and their babies during the window of opportunity (the first 1,000 days). PHC managers and project reports cite shortages of family doctors lead to heavy workloads and responsibilities. Turnover of the highest qualified medical practitioners is high as many doctors quit their jobs in the Khatlon Region and leave the country to work in Russia. PHC managers are often overwhelmed with forms to fill out and usually do not have access to their HF data presented in ways that facilitate decision-making at their level. PHCs, especially in remote areas, often lack basic equipment and essential medicines. Aside from that provided by THNA during the project, staff rarely receive on-the-job training. National MOHSPP informants, including the Chief Neonatologist, said that some family doctors continue to prescribe unnecessary procedures and medications, so sick infants during the early neonatal period are often not treated correctly. 19 Preventable neonatal deaths continue to be a problem and many of these deaths are due to sepsis or the poor condition of the mother prior to birth. 9 Chlorhexidine umbilical care is an evidence￾based neonatal intervention supported in USAID MNCH programs in other countries that has been proven to reduce neonatal deaths due to sepsis but it is not currently MOHSPP policy. Continuing HF IPC challenges mean that neonates continue to remain at risk for sepsis, even when maternity protocols are followed. The poor health and nutrition of the mother also contributes to poor neonatal outcomes. Results of recurrent RHFAs conducted by national experts trained by the project, using WHO EPC scorecard composite scores from 0 to 3, are presented in the chart below. Findings demonstrate significant change, even though more is needed to be done to achieve a top score of 3 in all indicators measured. Figure 8: Maternity Departments EPC Scorecards Full quantitative performance measurement for all 11 DH and (separate) 11 PHC indicators was beyond the scope of the evaluation and some factors (such as drug and human resource availability) that contribute to scores are beyond the focus of the project. Results are collected by national-level trainers, aggregated and reported by district, shared with stakeholders, and included in annual reports. Other routine hospital data are sent directly to the national MOHSPP for analysis but was not available to the ET. QI self-assessments conducted by HFs themselves collected data on the indicators. This was supervised by the trained mentors from the national level. Looking at specific maternity QI indicators, PPH and hemorrhagic shock are two indicators related to a life-threatening condition during and after delivery. The findings in Figure 9 below reflect changes in average number of PPH cases for approximately a 15-month period. Cases of hemorrhagic shock decreased from an average of 4.9 to 1.8 per quarter in 15 months and reduced the resources required for blood transfusions. 9 KII with MoHSPP National Chief Neonatologist. 20 Figure 9: Reduction in PPH Cases by Cohort in THNA-Supported HF QI self-assessments for ANC at PHCs revealed mixed results. Some indicators showed continuous improvements, while others did not increase, or decreased. Two of the five indicators (pre-eclampsia screening and appropriate iron therapy) are linked to life-threatening conditions in pregnant women. They are commodity-dependent and DH and PHC informants said that tests were not performed because reagents and equipment are not available. This is beyond the control of the project, but measuring the indicators allows priority deficits to be identified. Training and/or demand creation approaches on important MNCH components have limited effectiveness when there are weaknesses (in this case the supply chain) in the underlying health system. These health system limitations are identified in project reports and were confirmed by ET during field interviews. Figure 10: QI Indicators on ANC In child health, THNA used a WHO scorecard to track IMCI indicators and pediatric emergency care and rated the overall performance on a 0-5 scale, indicating significant improvements are needed in IMCI performance. Due to time limitations, the ET was unable to investigate all possible factors in the IMCI assessments that kept scores in pneumonia and diarrhea case management from being higher. However, in the CEIs with mothers that had taken their children for diarrhea treatment, 25 percent were not given both oral rehydration salts (ORS) and zinc. While 90 21 percent were given ORS, 25 percent did not receive zinc and 25 percent of women said they did not receive instructions how to administer ORS and/or zinc (see Figure 11 below). This showed that diarrhea case management, a key component of IMCI, was not always provided according to WHO guidelines. Figure 11: Percentage of Women with Children Sick with Diarrhea Who Responded Yes to the Following Questions (n=20) 4.1.3 Specific Barriers to Addressing Maternal and Child Anemia Anemia has been and remains a serious threat to the health of mothers and children as well as the physical and cognitive development of children.10 The World Health Assembly adopted a target of 50 percent reduction of anemia in WRA by 2025. 11 The 2017 DHS anemia findings for WRA and children under 5 years in FTF districts, which showed very high anemia levels, are presented below. No studies have been done since then and are not expected until at least 2021. 12 Figure 12: Anemia Among WRA (2017 DHS) 10 http://web.worldbank.org/archive/website01213/WEB/0__CO-50.HTM, accessed November 10, 2019. 11 Multi-sectoral Nutrition Strategy 2014-2025, USAID May 2014. 12 Tajikistan DHS, 2017. 90.0% 75.0% 75.0% 95.0% 95.0% 0.0% 10.0% 20.0% 30.0% 40.0% 50.0% 60.0% 70.0% 80.0% 90.0%100.0% Was your child provided oral rehydration salts (ORS) today? Was your child provided Zinc today? Were you instructed how to administer ORS or Zinc? Were you counselled on the need to increase fluids (breast milk, soup, water) for your sick child? Were you instructed to continue feeding your child when sick? 41.1 47.3 57.3 42.8 2.6 National Khatlon region FtF district Took any iron supplementation during pregnancy Took 60+iron pills during pregnancy 22 Figure 13: Anemia in Children Under 5 Years (DHS 2017) Anemia in pregnancy caused by deficiencies of iron and other micronutrients in the diet can cause pre-term births, low birthweights, developmental delays, and poor resistance to infection in children. Iron deficiency anemia (IDA) in pregnant women contributed to an estimated 420 perinatal deaths nationally in 2016.13 Multiple factors, including diet and health conditions and pregnancy can contribute to anemia in women and children. 14 Iron supplementation with IFAs in pregnancy are usually provided during ANC to mitigate the effects of dietary deficiencies, bleeding, or other conditions. Official MOHSPP guidelines for anemia prevention and treatment during pregnancy require IFA supplements to be provided for all pregnant women. In reality, however, according to FGDs with beneficiaries and KIIs with MOHSPP RH managers at the time of the evaluation, only pregnant women that had received blood tests and were diagnosed with anemia were receiving IFAs in 2019. This was reportedly due to supply shortages. In community FGDs, beneficiary women said that this had not always been the case. In the years corresponding with most of the THNA project (2016-2018), there were sufficient supplies and women received IFAs for prevention without the need for a blood test. No data were available to indicate if women actually consumed all of the tablets they were provided during those years, but evidence from the 2017 DHS would indicate that they did not (see Figure 12). Low consumption of IFAs can be due to many factors including side effects, insufficient supply, or not understanding the importance of taking them. There are significant financial and geographical barriers for women in poor project communities to travel to district centers and pay for the blood tests that are required to diagnose anemia. Several women beneficiaries in the community FGDs said they did not go, even when instructed to do so in ANC. (Most women will become more anemic as their pregnancy progresses regardless of the results of tests for anemia performed in early pregnancy. Even if they were to receive a “normal” result from a hemoglobin test in early pregnancy, it would not indicate they were not at risk for the impact of anemia.) Evaluation findings related to anemia control in pregnancy in the project area from multiple sources yielded mixed—and sometimes contradictory—findings, only a few of which could be attributed to the project. Aside from data the 2017 DHS, no objectively verifiable data were 13 Global Alliance for Nutrition Improved Nutrition (GAIN), Food Fortification in Tajikistan: A Cost-Effective Strategy for Sustainable Growth, 2016; Haas, J. and Brownlie T., Iron Deficiency and Reduced Work Capacity: A Critical Review of the Research Journal of Nutrition. 2001;131 in GAIN, 2016; and Horton & Ross, The Economics of Iron Deficiency Food Policy 28 (2003) 51-75 in GAIN, 2016. 14 USAID, Multi-Sectoral Nutrition Strategy 2014-2025, May 2014. 41.5 46.4 50.8 31.3 39.0 35.6 National Khatlon region FtF district Consumption of iron rich foods FtF districts Received iron supplementation in Khatlon region Received iron supplementation in FtF 23 available to the ET for analysis (see Figure 12). The ET discussed anemia in KIIs with district RH managers before information on IFAs was collected from beneficiary FGDs or results of the CEIs from ANC were available. Directors of district RH centers at all levels are responsible for IFA stocks and distribution to HWs for them to give to pregnant women. They usually send their IFA requests and report stockouts or challenges to the regional level MOHSPP. This year, however, they told the ET that due to budget limitations, UNICEF did not supply enough IFAs for all pregnant women to cover both prevention and treatment. This required them to limit IFA tablets to only women with anemia confirmed by a blood test. In a follow-up KII, the National RH Deputy Director told the ET that UNICEF was planning to continue to provide IFAs, but only in limited quantities. She also said that due to the reduced IFA supplies this year (2019) the RH center would (as of October 2019) distribute IFA tablets to poor and vulnerable pregnant women with anemia who are not able to buy them and also provide them to women with severe anemia, but not to all pregnant women. She confirmed that they (the MOHSPP) had a limited stock but were expecting another supply soon. UNICEF confirmed what the RH Deputy Director said that (as of October 2019), budget constraints meant that they will have to provide a lower quantity of IFAs to Tajikistan than they have in the last few years. In CEIs conducted with women receiving ANC services in the six evaluation district PHCs (N=42) the majority said they received iron tablets. Off those, the vast majority (95 percent) were told the importance of taking them and how to avoid side effects (90 percent). (Due to the cotton harvest season, women seeking health services during the evaluation may not be representative of the population of beneficiary women as a whole). However, as described earlier, FGDs with pregnant women and women with children less than 2 years in communities, however, showed that the reduced provision of IFAs was already in place as only approximately 20 percent had been given them. Some women said that in ANC they were told that they had a normal blood test and were only told to buy IFA in a pharmacy for prevention of anemia. But they said that they did not procure them due to financial problems and they also did not consider it important. Additional probing during FGDs with pregnant women and MiLs revealed a lack of awareness that “anemia was a problem” for either pregnant women or children or that taking iron tablets during pregnancy was important, especially if they “felt well.” When questioned about diversifying diets as promoted by THNA, low iron foods like “fruits and vegetables” were most frequently mentioned by respondents. When asked about foods to prevent anemia, a few women responded “pomegranates.” Even though the DHS indicated the majority of women ate either fish, poultry, or meat the night before, being questioned about eating meat evoked laughter because they said that “meat is so expensive it is only for special occasions like festivals.” Women said that they eat eggs but did not know that egg yolks can be a good source of iron. Even though IFAs are inexpensive, the country purchases none of its own supplies. GIZ, showed the ET recently arrived stocks of IFAs intended for the districts where they are working in the Kulob Region. They have also procured hemoglobinometers. They said that when the hemoglobinometers arrive, they will distribute them and the IFAs to the districts, in their region. 4.1.4 Follow-Up Information Received After the ET Fieldwork and Draft Report The ET responded to questions about the IFA supply in the draft report and followed up with the MOHSPP. The MOHSPP Deputy RH Director told the ET member that they have started to 24 raise the issue of iron provision with the Ministry of Health and all development partners. In October, all RH centers received IFA supplies and there are no stock outs. (This could explain why there were sufficient stocks at the time of the ET visits, but not earlier when many beneficiary women in communities had gone to ANC. She added that they had asked GIZ to see if they would provide IFAs to their 10 target districts, while UNICEF tries to find a way to cover the whole country. Data on anemia from the 2017 DHS, however, indicate that even in the years prior to 2019, when women’s IFA supplies were supposedly adequate and given for both prevention and treatment, very few women took the recommended quantities (see Figure 12). MNCH national experts agree that these stopgap measures will not be enough to make a dent in the massive anemia problem and that a separate anemia program that involves all relevant government sectors, including the Parliament should be launched. 4.1.5 Sustaining Community Volunteers All informants, from the central MOHSPP to regional and district health authorities to community and household FGD and KII participants, expressed appreciation for the important role that community volunteers have played in reducing morbidity and mortality in mothers and children in the Khatlon Region FTF districts. There is now a national framework for community health workers (CHWs) that is defined through MOHSPP orders #153 from March 9, 2017, and #114 from February 25, 2019, and protocol #1 from January 23, 2018, by the MOHSPP Council on Publications. More regulatory documents are still expected. Under the current framework, government Healthy Lifestyle Centers (HLSC) are expected to: 1) facilitate the formation of “community health teams” (CHTs) from among community members; 2) train CHT “facilitators” from among village-level primary healthcare staff; and 3) support CHTs and their facilitators in their work on community health. According to USAID, discussions between the THNA and the GOT and other stakeholders were supposed to start as early as the beginning of the project, but took place only toward the end of the fourth year. USAID also confirms that there is a National Program Strategy in place and approved by MOHSPP, with THNA that some important orders and coordination among donors are still lacking. Analysis of the national strategy and the ultimate objectives of how CHWs will link with the overall health system was beyond the scope of the evaluation. THNA said that it has been: engaging HLSC staff as trainers for community volunteers since Spring 2018; fostering links between the volunteers and village-level primary healthcare clinics to support the continuum of services for pregnant women and children with signs of malnutrition; and engaging HLSC and village PHC staff into all monthly meetings with volunteers to provide refresher trainings and facilitate cooperation between the volunteers and health providers. THNA has been making efforts to transfer its CHW cadre under the HLSC management within the government framework. THNA says that it suggested to Photo 2: Group Interview with CHPs, Dusti District, Khatlon Region October 2019. 25 USAID (that they) support training of village-level PHC providers as “CHT facilitators” and transform THNA volunteers into CHTs under the government framework in selected communities in the Khatlon regional and 12 district HLSCs. This would help the HLSCs absorb THNA community volunteers under the new government framework. USAID did not support this idea at the time. The short time frame, lack of supportive government policies and donor coordination likely would have made this approach impractical. THNA suggested that it might be considered for the next phase of the MNCH and nutrition programs. THNA said that it employed an alternative approach to support the transfer of managing volunteers to the HLSCs and is engaging the regional and district-level HLSC staff into all program planning, implementation, monitoring and evaluation (M&E), and decision-making discussions concerning community volunteers in an attempt to transfer the management expertise to HLSCs One DPHC manager told the ET that he had recently attended a regional meeting on the topic of continuing support for community volunteers. 15 Each district has HLSCs and there is a national office in Dushanbe. The ET met with two HLSC trainers, one who was able to describe in detail the training on adult education techniques that he received from the project. The long-term strategy for turning over volunteer support to the HLSCs was not yet fully developed at the time of the evaluation. When CHPs and CEs were asked about whether they will continue their work without external support, they said that they would. They would like to continue learning and developing more skills. CHPs and CEs said that recognition (and respect) from family and community, better quality of lives for themselves and their families (they were taught to “begin with our own household”), and the opportunities to continue learning are more motivating than the small “incentives” (e.g., cooking pots). CONCLUSIONS THNA is an integrated capacity building program where multiple project activities are designed to support the effectiveness and results of the others. When the project began, performance in multiple MNCH areas, including IPC was very low. The THNA methodology of addressing quality though QI capacity building, supportive supervision, and mentoring clinical HWs, combined with effective SBCC approaches delivered through volunteers to beneficiaries and their families in the community increased demand for health services while also changing household and community behaviors has been largely successful and has changed household MNCH health and nutrition behaviors. Community MNCH SBCC was specifically mentioned at the MOHSPP central and district levels as a major contributor to reductions in mortality and morbidity. While THNA does not have a separate sustainability and exit strategy, their current work supporting regional and district HLSCs to assume responsibility for volunteers will mean the volunteers will likely to continue to function after the end of the project. Although some indicators have not yet reached targets, it must be kept in mind that some behaviors take longer to change, some are more resistant to a QI approach, and others are impacted by factors outside of the influence of the project (such as human resources or commodity supply chains). Women and their families (especially MiLs) have demonstrated that 15 Clarification information provided by USAID and THNA to the ET, December 2019. 26 they are willing and able to change their attitudes and behaviors in response to SBCC methodologies promoted by the project, especially when the quality of health services and their daily lives have improved and when the same information and encouragement is delivered through multiple channels. This is especially true when the information channels are through individuals and systems that support each other and are respected by beneficiaries and communities. Community volunteers are credited with contributing to many positive changes. They are highly accepted and influence attitudes and behavior changes in the most influential people on maternal health and nutrition behaviors. Emphasis on maternal nutrition will remain essential to have impact on children before they are born and avoid chronic malnutrition. Anemia in mothers and children remains a major threat to accomplishing the objectives of the 1,000 Days Strategy. Neonatal and child health programs deserve significantly more attention. There is evidence that THNA succeeded in increasing access of mothers and children to a diverse diet of nutritious foods. The approach was multi-sectoral and determining clear links between some project activities and these achievements was not possible; however, qualitative investigation during the evaluation pointed to “cooking demonstrations” done by CHPs and CEs that showed women how to prepare different local foods, especially pureed complementary foods for infants and teaching mothers improved agricultural and food preservation techniques were important. Improved hygiene, especially handwashing with soap, was credited with decreased amounts of diarrhea by women in communities. The THNA project is integrated vertically and horizontally, meaning interventions and support were given from the central level MOHSPP to District HFs and to beneficiaries through CHPs and CEs. Health houses (HHs) were not included, so the HFs closest to the population were not part of the program. Future programs could benefit by task shifting some health promotion and treatment to those local facilities. Project quality improvement activities will continue if the GOT or Regional government can provide funding for mentors to conduct their periodic visits from the capital city. Community￾level activities are likely to continue if volunteer support through HLCS, or other sources, is of good quality and supported. The process of developing a national community health strategy that includes volunteers has started but will not be completely implemented by the end of the project. THNA’s decision to strengthen HLSC managerial capacity to absorb and continue the community MNCH work is appropriate. Implementation of the national strategy would be appropriate in future USAID MNCH programs in Tajikistan (see EQ4) RECOMMENDATIONS EQ 1 recommendations for the remainder of the project: The project developed two technical documents, one related to quality of clinical care improvement in HFs and the other focused on community SBCC approaches. Each document includes several project results plus recommendations for consideration in future programs. If USAID approves, these documents could be disseminated on the Internet, including on the Development Experience Clearinghouse (DEC). The ET concurs with the project’s identification of constraints and recommendations in the documents and recommends that they be discussed with the MOHSPP and stakeholder decision-makers before the program ends, if they have not already done so. 27 THNA should have an exit (or handover) strategy especially if community volunteers will have a significant delay before the HLSCs are able to resume support to them. THNA should review and refresh existing trainings, especially on nutrition during pregnancy, anemia prevention and control, and household sanitation (most latrines are not covered) before the end of the project and ensure copies of training materials and job aids remain available for future projects. Nutrition in pregnancy counseling should directly reinforce the importance of increasing and not decreasing food consumption in late pregnancy when chronic malnutrition in children often begins, while at the same time reassuring pregnant women that good nutrition will not lead to a big baby and a difficult delivery. Similarly, THNA and MOHSPP should undertake an investigation of the true barriers to effective anemia control in mothers and children in the FTF districts. From the findings, THNA should provide recommendations for actions that the MOHSPP can take until iron fortified wheat is available and consumed by target populations. It is unlikely that there is enough time remaining in the project to act on many of the findings, but there is time to raise awareness among HWs, beneficiaries, and their families of the detrimental impact of anemia on maternal and child health and development and make recommendations to the MOHSPP for future programs. (Recommendations for future USAID programs are included in the answer to EQ4.) 4.2 EQ2: TO WHAT EXTENT HAS THE ACTIVITY INCREASED ACCESS TO A DIVERSE SET OF NUTRIENT RICH FOODS THROUGHOUT THE YEAR AND WHY, ESPECIALLY THE DIETARY DIVERSITY OF PREGNANT AND LACTATING WOMEN AND CHILDREN UNDER 2 YEARS OF AGE? FINDINGS The MOHSPP Chief Pediatrician confirmed that THNA worked, using a multilevel strategy that included the agriculture and education sectors, to improve personal hygiene, especially handwashing. KIIs with DH and PHC managers in each district indicated that cases of acute malnutrition referred for care had increased, while the overall levels of malnutrition had decreased indicating early malnutrition is identified and referred sooner. Health managers, however, said that they, they did not have direct knowledge of THNA agricultural or food diversity activities. CEs charged with activities intended to improve access to dietary diversity also promoted pregnant women and women with children under 2 years to adopt MNCH preventive and health care-seeking behaviors (additional support for these activities is addressed in EQ1). Consistent with USAID’s global multisectoral Food Security and Nutrition approaches within FTF, the project strategy included partnering with other FTF program partners and collaborated on joint targeted interventions to increase dietary diversity and positive MNCH approaches. Activities included to promote dietary diversity focused on agricultural and household budget skills in particular, along with food consumption (including breastfeeding). They also included cross-trainings between projects such as TAWA, which assessed soil quality, trained THNA communities on cheese production and introduced new crops, and the International Potato Center (CIP), which introduced sweet potatoes as a new crop with high nutritional value for pregnant women and infants 6-24 months. TAWA, CIP, and THNA technical staff met regularly to plan joint activities with school gardening projects, farmers’ markets and fairs, and TAWA’s open field days. CEs also promoted consumption of a diverse diet during the “cooking demonstrations” on how to prepare complementary foods for children 6-24 months, and how to diversify diets using available and affordable foods, often from women’s own gardens. They 28 also demonstrated improved preservation methods to ensure that seasonal fruits and vegetables would be available for mothers and children during the winter months. FGDs with participants in these activities revealed that beneficiaries and their families were very enthusiastic about improved agricultural techniques for familiar and available food, ways to protect their chickens and cows from illness, and the improved preservation techniques. They said they found the information about MNCH and nutrition helpful and new techniques informative. On the other hand, new crops such as okra, Chinese cabbage, and sweet potatoes that were largely introduced by project FTF partners, TAWA and CIP, were unfamiliar foods and were not well accepted during the short time period of the project. Instead, women preferred to prepare pumpkin, which is high in Vitamin A, which was familiar and acceptable to both adults and children and available during certain times of the year. Women said they were constrained from continuing some practices they learned by lack of water and grass for their cattle, which impeded milk production; as a result, home cheese was not a popular option. Two group interviews with CEs (with 3-10 participants) stated that they (and the beneficiaries) were enthusiastic to learn more about how to increase poultry production and, surprisingly, mentioned they would like to raise rabbits, even though there are some cultural resistance to rabbit consumption by pregnant women. THNA conducted periodic surveys of women in households with children under 5 years of age. One indicator measured knowledge of dietary Vitamin A sources. Findings were documented in the Year 3 annual report. After two years of implementation, fewer than 25 percent could name sweet potato as a source of Vitamin A. The same survey measured women with under children under 5 dietary diversity (defined as consuming five or more of 10 types of foods). Figure 14: Women’s Knowledge of Foods High in Vitamin A 29 Figure 15: Percentage of Women Reaching Minimum Dietary Diversity (MDD), %, and Average Number of Food Groups Consumed by Women, RHS Rounds 2-5 and Control Communities in 2019 Source: THNA comments to USAID in response to draft evaluation report, December 2019. Figure 16: Breastfed and Non-Breastfed Children 6-23 Months Who Achieve MDD, Minimum Meal Frequency (MMF,) and Minimum Acceptable Diet (MAD), RHS Rounds 1-5 and Control Communities in 2019 (%) Source: THNA comments to USAID in response to draft evaluation report, December 2019. 4.2.1 Response From the Project to Draft Report In feedback on the draft ET report, THNA provided updated graphs from a more recent RHH (2019) and added the results to those tracked over the previous years of the project. These were incorporated into the body of the report and are represented in the women and children’s MDD graphics. To provide quantitative finding on the topics covered in the FGDs with CEs conducted by the ET with agriculture volunteers, THNA provided the results of the recurring agricultural practices survey seen below. These results became available after the fieldwork had been completed by the ET and were also provided to USAID in their Year 4 progress report. They provide quantitative measurement of THNA’s performance to improve access to nutritious foods from the perspective of the project. The findings confirm what the ET heard from CEs and 30 beneficiaries during their fieldwork. The ET did not see or hear anything that would not support these reported findings. Figure 17: Respondents Applying Improved Agricultural Technologies, as Confirmed By THNA Enumerators, By Type of Technology, 2017 and 2019 ( %) Source: THNA comments to USAID in response to draft evaluation report, December 2019. 4.2.2 School Peer Educator and Demonstration Gardens The ET visited two project-supported schools, conducted KIIs with the school directors and teachers and group interviews with students selected to be peer educators, and also viewed demonstration gardens. In three group interviews of approximately 20 student peer educators, the ET posed questions about the health, hygiene, and diet diversification activities of their program. Peer educators said they are trained as trainers in health, nutrition, and hygiene and are then expected to share what they have learned with their families and train other students. THNA provided training materials and posters with FTF messages for classrooms and invited TAWA to assess soil quality in the gardens. They also provided materials for improved latrines (labor was to be provided by matching funds raised by the school). The ET observed latrines under construction at two schools with matching funds supplied by parents, local businesses, or school staff. The ET also saw the demonstration gardens that teach students to diversify household food supply with improved growing techniques. Products of the gardens contribute to the food served to schools. The ET found the content of the peer educator training to be sound and peer educators and their teacher trainers said they enjoyed learning the content and sharing it. The value and reach of the peer educator program in relation to the project target population, contributions to MNCH and nutrition indicators beyond dietary diversity, and hygiene with adolescent girls will 31 have to be viewed from a long-term perspective. Activities are not directly linked to pregnant women, mothers of children under 2, or their families currently living in the project target area. CONCLUSIONS THNA would be considered a very ambitious program even if it were working solely within the MNCH sector. Several community-based approaches implemented by the project (some in partnership with other FTF projects) resulted in measurable increases of dietary diversity and MADs in mothers and their children under 5 years of age as captured by the project’s recurrent household surveys. Identification of Vitamin A sources increased as documented in these surveys, but project documents do not show awareness of dietary sources of iron. New foods take a long time to be accepted, so findings that the uptake of new foods was low were not surprising and may increase over time The project successfully increased consumption of familiar, affordable, and available foods and more mothers and children under 5 consume MADs. There was overlap and complementarity between MNCH and nutrition-oriented efforts to improve access to nutritious foods in community-level activities addressing health household behaviors and demand for services. Many activities complimented, and probably supported, results in the other areas of emphasis. However, the THNA IP and partners are largely health specialists. Agriculture and economic areas of emphasis, that are certainly worthy of support within FTF, are difficult for health professionals to design and manage, especially within a complex and multifaceted health program like THNA. Youth peer educator programs are difficult to evaluate for effectiveness and MNCH indicators directly linked to those programs are very few. Adolescent nutrition and hygiene behavior change, especially for girls, is valuable, but measuring impact relative to the 1,000 Days Strategy to reduce stunting, or FTF goals and objectives, requires developing indicators and measurement methods that fall outside of the MOHSPP reach. Routine household surveys of mothers with children under 5 years of age that provide the project with quantitative measurement of efforts to increase dietary diversity cannot capture the effects of project efforts directed to students. The value of demonstration gardens might be that the students in the future will share what they have learned with their parents and apply them when they become adults. This his will not, however, lead to measurable impact on the 1,000 Days Strategy target population: pregnant and lactating women and children under the age of 2 years within the timeframe of a single project. Sustaining these activities may be best linked to the education, youth, or agricultural development programs. RECOMMENDATIONS As it draws to a close, THNA should analyze the multiple project activities under this part of the program and identify activities that appear to contribute the most to improving access to a diverse diet for pregnant and lactating women and children under 2 years old. Positive nutrition and health behaviors (including hygiene), captured in their recurrent household survey of mothers with young children, should be shared with stakeholders and other GOT ministries to capture the lessons learned from non-health-focused project efforts. To the extent possible, the project should highlight THNA activities in the agriculture, adolescent health and hygiene, WASH, and women’s economic support that seemed to be promising and share them with those stakeholders in other sectors—including within USAID—that may be able to incorporate some of the strategies into their programs. 32 4.3 EQ3: TO WHAT EXTENT WERE (PROGRAM-SUPPORTED) EVIDENCE￾BASED GUIDELINES AND PROCEDURES INSTITUTIONALIZED BY THE HEALTH SYSTEMS IN THE FTF ZOI? AND WHAT IF ANY, WERE THE BARRIERS? FINDINGS Institutionalizing new evidence-based approaches requires several steps. After situation analysis and consensus priority setting, changes in Tajikistan required analysis of the corresponding policies and regulations to identify those that must be changed, along with any new ones that would be need to be developed to allow new approaches to be introduced and HW practices adopted. The MOHSPP Chief Pediatrician and Head of the Pediatric Unit of the MOHSPP told the ET that THNA contributed (along with GIZ and UNICEF) to 15 national MNCH, nutrition, IPC, and QI protocols that were revised and submitted for approval by the MOHSPP. Of those that were submitted, 10 have been approved and five should be approved soon. In some cases, THNA reproduced and disseminated copies of materials based on approved protocols and policies. The 1,000 Days Strategy SBCC strategy was developed with several partners and the THNA community SBCC approach was incorporated into the draft. A list of policies, guidelines, and standards with THNA contributions is listed in the table below. Table 2: Protocols, Guidelines, and Standards Developed with THNA Support by Subject and Type(s) of Assistance Title Level of Contribution THNA Contribution (participated in development, revision, approval, implementation, indirect support, advocacy) Supportive Supervision guideline* National, Regional • Participated in development • Approval • Implementation • Indirect support • Advocacy Kangaroo Mother Care (Tutorial) National, Regional • Participated in development • Approval • Implementation • Indirect support • Advocacy Prevention, diagnosis, and treatment of anemia in children, adolescent girls, and women of reproductive age (National Training module) National, Regional • Participated in development • Approval • Implementation • Indirect support • Advocacy ICATT/IMCI (training tool)* National, Regional • Revision • Approval • Implementation • Indirect support • Advocacy National guideline on Infection Control (prikaz #1119) National, Regional • Revision • Implementation • Indirect support Healthy nutrition and nutrition of pregnant women guideline National, Regional • Participated in development • Approval 33 Title Level of Contribution THNA Contribution (participated in development, revision, approval, implementation, indirect support, advocacy) • Implementation • Indirect support • Advocacy Methodical manual on “Care of Healthy Child of 0-24 months” National, Regional • Participated in development • Approval • Implementation • Indirect support • Advocacy Antenatal Care (Standards) Regional • Implementation • Indirect support • Advocacy MCH handbook Regional • Implementation • Indirect support Hospital IMCI pocketbook Regional • Implementation • Indirect support Integrated management of acute and moderate malnutrition National guideline National, Regional • Implementation • Indirect support • Advocacy Management of complications during pregnancy and delivery National standards National, Regional • Implementation • Indirect support • Advocacy Beyond the numbers approach National guideline National, Regional • Implementation • Indirect support Management of bleedings in delivery National standards National, Regional • Implementation • Indirect support Management of hypertension conditions during pregnancy and delivery National standards Regional • Implementation • Indirect support Law on Food Fortification National • Advocacy • Implementation National Health Strategy National • Development • Ongoing SUN Action Plan National • Development Healthcare Waste Management National • Advocacy Source: THNA Clinical Manager. True institutionalization can only take place when updated protocols are standardized and disseminated, HWs are trained, and new approaches implemented, supervised, and sustained. Some approved documents on the list are still undergoing that process. Determining the extent to which they meet true institutionalization will not be ascertained until after the project is over, but all changes intended to be institutionalized must first undergo the same processes. KIIs with the national MOHSPP and stakeholders confirmed universal appreciation for the overall approach used by the project where they were substantially involved in developing some of them (Nutrition in Pregnancy, Maternal Care, IMCI/ICATT upgrade, and IPC) and the contributions the project has provided in identifying and improving procedures and guidelines related to MNCH and nutrition QI. THNA does not take full credit for any of these efforts and says that progress is the 34 result of many partners. Some documents, especially related to SUN and the 1,000 Days Strategy, require other ministries beside the MOHSPP to be involved. National stakeholders reported that technical meetings of the groups with responsibility to move these guidelines forward are infrequent and sparsely attended, so little progress has been made towards obtaining approval. 4.3.1 Constraints The QI methodology and training, along with specific MNCH technical trainings were not developed to be integrated into national medical pre-service and postgraduate programs. Sustainability and expansion are not assured until these programs train on the latest approved documents and changes are supported by all components of the health system. These programs were largely not involved in developing project strategy that would have had the objective of making sustainable improvements in MNCH health and nutrition services. Many facility managers lack enough management, leadership, and problem-solving skills to identify problems from their own data for decision-making and action without a formal QI team supported from the outside. According to national experts on the ET, health managers outside of the central level also have very few options to make changes on their own. Overall management and leadership skill-building was not a focus of the THNA project, so this is not a reflection on THNA’s performance. Institutionalization, scale-up, maximization of available resources, and sustainability capacity building were also not focus areas of the project. Strengthening other essential components of the system, or implementation of the WHO’s health system “building blocks” needs to be a goal for future programs, and not necessarily limited to those with USAID support. THNA largely focused on only two of the WHO Health Systems Strengthening (HSS) building blocks depicted in the figure below, but results were impacted by weaknesses in the other building blocks. Figure 18: Proposed Modified WHO Health Systems Framework The weaknesses in of HF managers leadership and management skills will make it difficult for them to make maximum use of technical assistance (TA) and resources, including those provided by donors and the district and regional levels of the government. Medical specialists are often placed in positions with significant management responsibilities without any management, budget, or leadership training, nor training on using data for decision-making. Health and nutrition projects in other countries have found it necessary to provide capacity building in those areas in order for the technical assistance provided to result in sustainable change. 35 CONCLUSIONS Approval of policy and standards documents is just one of the early, but necessary, steps to improving the quality of health care services to the whole country, including HFs in the FTF ZOI. Dissemination and capacity building take place after approvals have been secured. THNA does not have sole responsibility for the adoption, implementation, or dissemination of any MOHSPP policy documents. They have been credited with contributions to 15 draft documents, of which 10 have been approved and the others are expected to be approved soon. They have played active roles in developing several specific clinical guidelines in maternity, IMCI/ICATT, and nutrition during pregnancy. Some documents, such as the SUN Strategy require the members of those groups to become more active in developing them. SUN is a multisectoral movement where health partners are minority members. On the other hand, according to UNICEF, USAID is Co-Chair of the SUN movement, and could use its influence to motivate that group more active at the national level. RECOMMENDATION USAID should exercise its leadership role in the SUN movement to engage members of the group to move forward with developing the national SUN strategy. Future USAID MNCH and nutrition programs should continue to play a role in providing technical content, advocacy, dissemination, and capacity building for these upgraded policies, standards, and guidelines. 4.4 EQ4: GOING FORWARD, HOW, AND IN WHICH AREAS, SHOULD USAID-FUNDED MCH & NUTRITION ACTIVITIES BE CONDUCTED TO SUPPORT THE HEALTH SYSTEM AND/OR POPULATION IN IMPROVING THE NUTRITION AND HEALTH SERVICES TO MOTHERS AND CHILDREN? The evaluation recommendations prioritize strategic investments to target funds and TA that could specifically address the most important gaps in MCHN (including anemia) during the window of opportunity to prevent chronic malnutrition in the 1,000 Days Strategy. RECOMMENDATIONS Recommendation 1: Address gaps in nutrition during pregnancy and faltering maternal and child anemia prevention and control. Anemia a serious public health problem, especially for WRA and young children, and requires much more emphasis and urgent actions at the national level (including international partners). The Global Alliance for Improved Nutrition (GAIN) project provided TA to Tajikistan to develop wheat flour fortification, but it was still under development at the time of the evaluation. The ET is confident that fortified flour will eventually be available throughout the country and contribute to the long-term reduction of the threat that anemia poses in MNCH and nutrition, but this will not happen for women, adolescent girls, and children in FTF districts soon. USAID should use its influence as Chair of the Food Security and Nutrition Group and Co-Chair of the national SUN movement to: bring ministries and groups engaged in relevant food security and health activities together to: 1) raise the profile of nutrition in pregnancy in general, including harmful practices; 2) focus attention on the multiple factors involved in high maternal and child anemia prevalence and steps that can be taken to reduce them; and 3) devote specific attention to disseminating and implementing the new Nutrition in Pregnancy guidelines that were developed with project assistance. Specific meetings should be called to raise awareness that anemia 36 prevention and treatment are failing in the short run and will impact maternal and child morbidity and the growth of children for several years, especially in poor rural districts targeted by FTF. USAID’s centrally-funded technical support programs in MNCH and nutrition have the expertise needed to advise the Mission in these areas. Poor maternal nutrition and maternal and child anemia are major threats to national development overall, achieving the sustainable development goals (SDGs) as well as the objectives of the 1,000 Days strategy. USAID can mobilize SUN and food security partners for action. As a matter of some urgency, USAID should work with the MOHSPP and other donors to focus on a sustainable iron/folate supply for both prevention and treatment by getting anemia testing, prevention, and treatment commodities as line items in the national budget. UNFPA’s successful example of cost-sharing with the GOT and the government of Japan to get contraceptives into the national budget can serve as a model for how to achieve this. Intrauterine devices (IUDs) are now entirely funded in the GOT budget. The ET concurs with THNA’s recommendation that future MNCH and nutrition programs should increase emphasis on birth spacing and access to contraceptives as important for improving MCHN health and nutrition status (including anemia). Recommendation 2: Support sustainability of THNA QI approaches by advocating QI be linked to HF accreditation processes and to Continuing Medical Education (CME) credits for individual health professionals. Future programs seeking to institutionalize new protocols, guidelines, and standards should work with the Republican and Regional Post-Graduate Medical Institutes (PGMIs) and the Republican Health Lifestyles Centers in the design of the program and link these institutions to the NRCs introduced by THNA and to the HLSCs. They should build upon existing policies on distance education and adult education methods (already started by the project for some topics) so that health managers and facilities will have continuous access to updated information and policies. Working with Republican PGMIs (national and regional) can support institutionalization of improvements. QI efforts should be extended to neonatal and pediatric care and linking QI (including IPC) to hospital accreditation is likely to be a strong motivator to sustain changes. To sustain QI and Clinical Safety technical support efforts, future QI projects should intentionally link to the DH accreditation process. Since MOHSPP regulations require every facility to be accredited according to specific accreditation standards that include large QI and Clinical Safety components national MNCH experts told the ET that would be a strong motivation to sustain those technical investments USAID made in the THNA project. Facilities must conduct self￾assessments during the preparation period before they can pass accreditation. QI skills learned in the project can help HFs for this process using these skills if the skills are maintained and supported. THNA has recommended, and the ET agrees, that if IP trainings qualified for CME credits, it would be a strong motivator for individual health professionals to participate in trainings. This would require linkages to be developed with the PGMI responsible for CMEs. Recommendation 3: Support the Global “A Promise Renewed” movement and expand QI to neonatal and child health programs. Since the overall QI and SBCC approaches of THNA were successful, but heavily focused on maternity care, they could be scaled up to other MNCH areas if additional health resources were available. If funds for MNCH QI expansion become available, QI processes should be expanded 37 to child health, including improved water and sanitation in the child health wards, beginning as soon as babies are born and extending through early childhood, with special emphasis on children up until the age of two when they are most vulnerable and interventions are most likely to be effective. IMCI already covers the newborn period, but doctors continue to prescribe treatments that are not evidence based. This emphasis on IMCI should extend to monitoring and measuring progress in IMCI implementation in the country. If possible, USAID should advocate with the MOHSPP and stakeholders to introduce chlorhexidine umbilical cord treatment for neonates. This is an evidence-based intervention supported by USAID MNCH programs in other countries that is proven to substantially reduce neonatal deaths due to sepsis. Recommendation 4: Support access to MNCH health services in rural communities with advocacy for some ANC services to be adopted by RHHs. USAID should consider supporting increased access to important ANC services by joining with other stakeholders, especially the WB, to advocate with the MOHSPP to allow task adoption for hemoglobin and urine tests and IFA supplies for pregnant women at the RHH level. Tajikistan has had successful experience with this approach previously. The WB told the ET that their new program will put more emphasis on HHs which are the HFs that are located closest to the communities the health system serves. HHs were not targeted in the THNA health capacity building training. Recommendation 5: Consider supporting sustainable country ownership of health systems reform with leadership and management training for the regional and district health managers. USAID should consider constraints for implementation and recommendations for future programs identified in the two recent program technical documents produced in September 2019 and consider their analysis and recommendations in the design of future programs. Small donor programs with limited coverage relative to the national population are still the norm in Tajikistan. That will not change until the health systems are strengthened with better leadership and management skills, along with the authority and resources to make informed decisions without direction from the central MOHSPP. This strengthened capacity will require changes in how health services are financed to become effective. Analysis of the Tajik health system was beyond the scope of the evaluation. Globally, however, USAID has many years of experience funding leadership and management training designed to promote country ownership and management skills development to take charge of all of the health system, by strengthening the WHO “building blocks” for sustainable health programs within the context of overall health reform, USAID/Washington can advise the Mission about whether a leadership and management program (either as a stand-alone, or a buy-in to an existing program) is advisable. Experience by other donors in Tajikistan has shown that sponsoring health managers for courses conducted abroad is not effective in building and sustaining capacity in this area. Recommendation 6: Consider supporting a SBCC program devoted to the 1,000 Days Strategy, MNCH, and nutrition. Support the design and implementation of the national CHW strategy linked to the national HSLCs. USAID should consider supporting additional SBCC projects as USAID is a global leader in these types of programs. The successful THNA SBCC is highly respected in Tajikistan. SBCC will play a prominent role in the national 1,000 Days Strategy and is anticipated to be included in the 38 National Health Strategy for 2020-2030. However, the ET does not recommend that a major investment in mass media campaigns be the major primary emphasis of any new SBCC program, but some media interventions can be used to support behavior change at the community and household level. The “THNA model” that focuses on interpersonal communication at the community level has already been recognized at the community, district, regional, and national levels as appropriate in the Tajikistan context. Projects intended to support community SBCC components should be designed to link with the National Republican HLSCs and regional governments for sustainability when projects turn over volunteer support responsibility. District HLSCs might consider including some experienced CHPs to help guide volunteer training and support. The ET concurs with THNA’s emphasis on building regional and district capacity to work with HLSCs to prepare them to implement the new CHW strategy when all of the necessary approvals and guidelines and donor support are in place.USAID should support the MOHSPP to coordinate with other donors as they develop their national CHW program. 39 ANNEXES 40 ANNEX 1: EVALUATION STATEMENT OF WORK SECTION C – DESCRIPTION/SPECIFICATIONS/STATEMENT OF WORK C.1 TITLE OF ACTIVITY The title of the Activity is Tajikistan Health and Nutrition (THNA) Evaluation. C.2 PURPOSE The purpose of this statement of work is to procure an end of project performance evaluation for the Tajikistan Health and Nutrition Activity (THNA), under the Feed the Future (FTF) initiative. The end of project evaluation will serve a dual purpose: 1) to learn to what extent the activities’ results have been achieved as described in the results framework; and 2) to inform the design of future Maternal and Child Health and Nutrition activities. Lessons learned and challenges disclosed from the evaluation results will be considered when programming future projects while success stories will be replicated going forward. The results of the performance evaluation will be shared with different stakeholders including the Ministry of Health and Social Protection of the Population (MOHSPP), USAID mission, the THNA Implementing Partners, and will be posted on the DEC (Development Experience Clearinghouse) for public access within 90 days of report completion. C.3 BACKGROUND Tajikistan is the poorest of the five Central Asian republics, with almost half the population living below the poverty line. One-third of the population is affected by food insecurity, and more than 30 percent of these households are severely food insecure. Employment opportunities in Tajikistan are scarce, and one in four households has a family member working abroad. Remittances to Tajikistan are the highest in the world. With increased economic growth in the past decade due to the economic recovery in the agriculture, mining, and natural resources sectors, the Government of Tajikistan (GOT) has supported poverty reduction efforts, aligning multilateral and donor support. Included in this critical support are the U.S. Government Feed the Future (FTF) and Global Health Initiative prioritize maternal, newborn, and child health (MNCH) care addressing the underlying causes of malnutrition. The Ministry of Health and Social Protection of the Population (MOHSPP) has set national health goals to reduce the aggregate morbidity rate by 30 percent, with specific benchmarks applied to the strengthening of MNCH and adolescent health and reducing maternal and child morbidity. One of the priority goals of the National Development strategy till 2030 is to reduce stunting by 40 percent. Feed the Future Tajikistan Health and Nutrition activity builds on the successes of projects such as the USAID Quality Health Care Project (QHCP) and Maternal and Child Health Project (MCHP) that have taken important steps to improve access to quality maternal and child health care at clinical and community levels. USAID-funded activities implemented by the Global Alliance for Improved Nutrition (GAIN) in wheat flour fortification (FY 2013-FY 2017), as well as the USAID-funded Universal salt iodization project through UNICEF (FY 2015-FY 2020), have worked to address nutrition at the national level. The remote Khatlon oblast, the focus region for the FTF Tajikistan Health and Nutrition Program (THNA), has some of the country’s poorest nutrition and maternal and child health statistics. Child mortality in Khatlon Province decreased from 61 to 40 per 1,000 live birth between 2012 and 2017, but the region still has the highest under-five mortality rate in the country. The stunting 41 rate decreased from 25 percent to 18 percent between 2012 and 2017. Cost and cultural practices are barriers to regular consumption of protein, and consistent access to fruit and vegetables is limited, particularly in the winter. Additionally, fewer than half of families have access to a central water system, and latrines are frequently poorly maintained in both private homes and health facilities. Changing family health and nutrition practices requires nuanced and tailored efforts, as young mothers have little power, while mothers-in-law and husbands are the family’s key decision-makers. Additionally, health systems need support to sustain gains in applying best practices. To address these challenges and support the country to meet its health and nutrition goals, THNA works to strengthen its efforts to integrate health care and nutrition at the family, community, clinical, and national levels. THNA’s technical approach applies the following technical strategies to achieve results: • Promote the use of evidence-based practices, particularly in the first 1,000 days window of opportunity between a woman’s pregnancy and her child’s second birthday within the home, community, and health facility; • Improve healthy behaviors and practices among adolescent girls, women, and children in their homes and communities and improve their access to health care services; • Strengthen links among health facilities, communities, and FTF initiatives; • Incorporate gender-equitable and culturally sensitive approaches that empower women to improve their health and the health of their children; • Leverage stakeholder partnerships across sectors (health, education, agriculture), with donors and through public–private partnerships. THNA expects the following results: Result 1: Improved quality of health care services for MNCH, in nutrition, sanitation, and hygiene in the Feed the Future Zone of Influence. Result 2: Increased access to a diverse set of nutrient-rich foods throughout the year. Result 3: Increased practice of healthy behaviors around MNCH, including increased consumption of nutrient-rich foods among adolescent girls, women, and children 6-24 months of age; improved sanitation and hygiene behaviors; and increased appropriate health care services in the Feed the Future Zone of Influence. Result 4: Institutionalize advances made in Tajikistan’s health sector to provide evidence-based MNCH services, including nutrition, sanitation, and hygiene, at the national level. A. Description of the Problem, Development Hypothesis(es), and Theory of Change The remote Khatlon oblast, the focus region for the FTF Tajikistan Health and Nutrition Program (THNA), has some of the country’s poorest agriculture and maternal and child health statistics. The region has the highest under-five mortality rate in the country (61 per 1,000 live birth), twice as high as in the capital, Dushanbe. Over one-third of children in the region are stunted. The development hypothesis for the Health and Nutrition Program, in support of the goal, objective, and purpose, is: If USAID improves the quality and use of health care services and improves access to and consumption of diverse diets, then health status and nutrition will improve, particularly for women and children. 42 This hypothesis is based on four core theories of change for the Health and Nutrition Program: 1. IF health providers know and use evidence-based practices, and IF they have supplies of basic equipment needed to carry out these practices, THEN they will deliver better quality services and maternal and infant morbidity and mortality will decline. The above development hypothesis corresponds to Result 1: Improved Health Services “Improved quality of health care services for MNCH, including nutrition, sanitation, and hygiene health care services” and relevant indicators. 2. IF access and utilization of more diverse and nutrition foods and improved sanitation facilities are in place for the rural population, especially women of reproductive age and children under 2, THEN rates of stunting and wasting of children under five will decline. The above development hypothesis corresponds to Result 2: Dietary Diversity “Increased access to diverse nutrient-rich foods throughout the year” and relevant indicators. 3. IF women, family decision-makers, and community members know and recognize the danger signs in pregnancy, delivery, and post-delivery; the danger signs of child illnesses; the consequences of poor nutrition during the first “1,000 days” window; and the benefits of good water/hygiene/sanitation behaviors, THEN they will practice healthy behaviors and seek health care when needed and maternal and infant morbidity and mortality will decline. The above development hypothesis corresponds to Result 3: Behavior Change “Increased practice of healthy behaviors, including consumption of nutrient-rich foods; improved sanitation and hygiene; and increased use of health care services” and relevant indicators. 4. IF the evidence-based guidelines and standards are utilized by all levels of the health system, THEN quality of health care will be sustained beyond the life of the Program and health outcomes will improve over the long term. The above development hypothesis corresponds to Result 4: Policy “Institutionalize evidence based MNCH services, including nutrition, sanitation, and hygiene through national level policies and standards” and relevant indicators. The THNA team proposes an innovative, comprehensive, responsive, and sustainable program framework that will result in improved integration of health care at the family, community, clinical, and national levels related to good MNCH, with an emphasis on nutrition, sanitation, and hygiene, thereby improving health status and nutrition, particularly for women and children living in the Khatlon oblast. C.4 PROJECT INTENT On September 25, 2015, USAID signed a bilateral cooperative agreement under Feed the Future to fund the “Tajikistan Health and Nutrition Activity,” implemented primarily by IntraHealth International Ltd. ($13,158,832; End date – September 28, 2020). The award was designed to address the challenges faced in Khatlon oblast. In October 2018, the program description of the activity was revised mainly to reflect termination of one of the sub-grantees—Mercy Corps. To improve the health and nutrition of the Khatlon population, the THNA award started in September 2015 with the following objectives: • Improved quality of health care services for MNCH, in nutrition, sanitation, and hygiene in the Feed the Future Zone of Influence. 43 • Increased access to a diverse set of nutrient-rich foods throughout the year. • Increased practice of healthy behaviors around MNCH, including increased consumption of nutrient-rich foods among adolescent girls, women, and children 6-24 months of age; improved sanitation and hygiene behaviors; and increased appropriate health care services in the Feed the Future Zone of Influence. • Institutionalize advances made in Tajikistan’s health sector to provide evidence-based MNCH services, including nutrition, sanitation, and hygiene, at the national level. THNA focuses on the thousand-day window of opportunity—between conception and a child’s second birthday—to make a lasting impact on a child’s growth, learning, and future productivity. Interventions include: • Training health workers in antenatal care, essential perinatal care, and integrated management of childhood illnesses; • Improving household nutrition through developing families’ skills and knowledge in nutrition, food preservation, backyard gardening, household budgeting, and essential hygiene behaviors; • Providing small grants for equipment and sanitation upgrades in health facilities and the community; • Supporting advocacy for policy reforms on poverty reduction, agriculture, nutrition, and health. Gender Consideration: THNA provides initial gender-sensitivity training and education to program staff to gain a unified understanding of gender-related program challenges and ensure a cohesive gender strategy across facility and community-level interventions. THNA uses a family- and community-centered approach to engage both women and men and girls and boys to adopt practices that improve the health and nutrition of the entire family and community. Activities take gender norms into consideration to ensure appropriate family members are targeted with tailored messages. However, the program is mindful that these activities do not amplify stereotypical gender roles or exacerbate inequities. Rather, program activities aim to promote shared decision-making, increase women’s self-efficacy, particularly for young women and girls, and offer expanded and more equitable opportunities for both women and men. C.5 EVALUATION QUESTIONS This evaluation is an end of project performance evaluation for the THNA activity that is scheduled to come to an end in September 2020. USAID will use the results of the report of the evaluation to inform implementation of ongoing and new activities. It is expected that THNA stakeholders will have the opportunity to discuss the results of the evaluation. Evaluation Questions for THNA in Order of Importance 1. To what extent has the activity increased access and improved to Maternal Newborn Child Health services including nutrition, sanitation, and hygiene in the Zone of Influence, particularly in the first 1,000 days? 44 2. To what extent has the activity increased access to a diverse set of nutrient rich foods throughout the year and why, especially the dietary diversity of pregnant and lactating women and children under 2 years of age? 3. To what extent were (program supported) evidence-based guidelines and procedures institutionalized by the health systems in the FTF ZOI? And what if any, were there barriers? 4. Going forward, how, and in which areas, should USAID-funded MCH & Nutrition activities be conducted to support the health system and/or population in improving the nutrition and health services to Mothers and Children? C.6 EVALUATION DESIGN AND METHODOLOGY Evaluation Design This is an end of activity performance evaluation and will focus on how the activity interventions implemented, and what they achieved in regard to the expected results as outlined in the cooperative agreement and in line with the development hypothesis. Additionally, this performance evaluation will examine how activities are perceived by local stakeholders, valued, and likely to be sustained. Evaluators will use a mix of quantitative and qualitative data collection and analysis methods as appropriate. The evaluation must follow the principles and guidelines for high quality evaluations outlined in the USAID Evaluation Policy (Updated October 2016) https://www.usaid.gov/sites/default/files/documents/1870/USAIDEvaluationPolicy.pdf 1. Data collection methodology and corresponding data sources: The Contractor should consider a range of possible methods and approaches for collecting and analyzing information and data that are required to answer the evaluation questions. The Contractor must share data collection tools with USAID for feedback and/or discussion, providing sufficient time for USAID’s review before they are applied in the field. The data collection tools should draw upon both subjective and objective input of the programs’ stakeholders and should be disaggregated to the relevant level. The data collection methodology will include a mix of tools appropriate to the evaluation questions and include a document review, in-depth interviews with the key stakeholders, surveys, and focus group discussions with beneficiaries. USAID/Central Asia will provide Contractor with electronic access to key program-related documents prior to the start of the in￾country work. All team members should review these documents in preparation for the initial Team Planning Meeting. Relevant documents include: • FTF strategy and amendments; • FTF baseline and mid-term evaluations; • THNA cooperative agreement and modification; • THNA work plans; • Quarterly and annual reports; • Monitoring and evaluation plan and results; • THNA baseline data; • THNA midterm review report; • DHS 2017 report; 45 • Micronutrient Status Survey 2016 report. In addition to the above list, the document review must consider other secondary literature determined relevant by the Contractor. The Contractor will conduct site visits to Khatlon oblast. Surveys, key informant interviews, and focus group discussions will be conducted with counterparts, stakeholders, and beneficiaries according to the sample size highlighted in the below table. The Contractor should calculate a representative sample size for an appropriate sample, and the method for determining the representative sample must be discussed with USAID. 2. Interviews, Surveys, and Site Visits: The Contractor will conduct in-depth interviews, surveys, and focus group discussions, at a minimum, with the following organizations/staff: • Ministry of Health and Social Protection (MOHSPP); • Department of Health of Khatlon Khukumat; • Staff of the pilot central district maternities and child departments; • Staff of central district health centers; • Staff of the sub-district Primary Health Care facilities; • Representatives of district khukumats; jamoat; mahalla committees; • USAID Staff (AOR, health team lead, program officer, program assistant); • Project beneficiaries; • International donors and partners working in the MCH and Nutrition sectors in Tajikistan; • Any other relevant stakeholders of THNA. 3. Data analysis plan: Prior to the start date of data collection, the Contractor must develop and present, for USAID/Central Asia review and approval, a data analysis plan that details how focus groups and key informant interviews will be transcribed and analyzed; what procedures will be used to analyze qualitative and quantitative data from key informant and other stakeholder interviews; and how the evaluation will weigh and integrate qualitative data from these sources with quantitative data from performance indicators and the activity performance monitoring records to reach conclusions about the areas of this evaluation. The Contractor must submit a complete table with proposed data collection and analyses methods, as convenient. To the extent possible, data and information need to be disaggregated by gender, age, and location. 46 ANNEX 2: EVALUATION SCHEDULE Tajikistan Health and Nutrition Activity (THNA) Evaluation Evaluation Schedule ~ August – November 2019 ~ Sun Mon Tue Wed Thu Fri Sat Aug. 25 26 Review materials 27 Review materials 28 Review materials 29 Conference call with USAID 30 Review materials & prepare Work Plan 31 Sept. 1 2 Review materials & prepare Work Plan 3 Review materials & prepare Work Plan 4 Review materials & prepare Work Plan 5 Review materials & prepare Work Plan 6 Review materials & prepare Work Plan 7 8 9 Review materials & prepare Work Plan 10 Review materials & prepare Work Plan 11 Review materials & prepare Work Plan 12 Review materials & prepare Work Plan 13 14 15 16 Review materials & prepare Work Plan 17 Review materials & prepare Work Plan 18 Review materials & prepare Work Plan 19 Review materials & prepare Work Plan 20 Submit Draft Work Plan 21 22 23 Incorporate comments 24 USAID approves travel and comments on Work Plan 25 Incorporate comments 26 Incorporate comments Translation of selected data collection tools 27 Incorporate comments Translation of selected data collection tools 28 Translation of selected data collection tools 29 30 Submit final Work Plan Oct. 1 Team leaves for Dushanbe 2 Team arrives in Dushanbe 3 Internal team meeting Work Plan Approved by USAID Dushanbe 4 In-Brief with USAID Finalize quantitative data tools Surveyor training Dushanbe 5 Pilot test of survey KIIs and FGDs Finalize quantitative data tools 47 Evaluation Schedule ~ August – November 2019 ~ Sun Mon Tue Wed Thu Fri Sat 6 7 KIIs and FGDs 8 KIIs and FGDs 9 Leave to Khatlon Meet and de￾brief with survey team in Bokhtar KIIs and FGDs 10 Beneficiary survey in selected locations KIIs and FGDs 11 Beneficiary survey in selected locations KIIs and FGDs 12 Beneficiary survey in selected locations KIIs and FGDs 13 14 Beneficiary survey in selected locations KIIs and FGDs 15 Beneficiary survey in selected locations KIIs and FGDs 16 Beneficiary survey in selected locations KIIs and FGDs 17 Beneficiary survey in selected locations Team returns to Dushanbe as a base for Yovon. KIIs and FGDs 18 Beneficiary survey in selected locations KIIs and FGDs 19 Beneficiary survey in selected locations KIIs and FGDs Qualitative data collection in Districts ends. 20 21 Beneficiary survey in selected locations KIIs and 22 Beneficiary survey in selected locations KIIs and 23 Beneficiary survey in selected locations KIIs and 24 KIIs 25 KIIs Follow-up meeting with THNA HQ staff. 26 Initial Data analysis and Out-Brief preparation 27 28 Initial Data analysis and Out-Brief preparation 29 Initial Data analysis and Out-Brief preparation 30 Out-Brief with USAID 31 Team leaves for home Nov. 1 Team arrives home 2 3 4 Data analysis and report writing 5 Data analysis and report writing 6 Data analysis and report writing 7 Data analysis and report writing 8 Data analysis and report writing 9 10 11 Data analysis and report writing 12 Data analysis and report writing 13 Data analysis and report writing 14 Data analysis and report writing 15 Data analysis and report writing 16 17 18 Data analysis and report writing 19 Data analysis and report writing 20 Data analysis and report writing 21 Data analysis and report writing 22 Submit Draft Evaluation Report to USAID 23 24 25 26 27 28 29 30 48 Evaluation Schedule ~ August – November 2019 ~ Sun Mon Tue Wed Thu Fri Sat Dec. 1 2 Incorporate Comments 3 Incorporate Comments 4 Incorporate Comments 5 Incorporate Comments 6 Incorporate Comments 7 8 9 Submit Final Evaluation Report to USAID 10 11 12 13 14 49 ANNEX 3: EVALUATION DESIGN MATRIX THNA Evaluation Matrix 16Evaluation Questions (EQ) w/ Illustrative KII, FGD & Survey Questions Data Sources Methodologies Data Analysis EQ 1: To what extent has the activity increased access and improved Maternal Newborn Child Health services including nutrition, sanitation, and hygiene in the Zone of Influence, particularly in the first 1,000 days? Illustrative KII and FGD Questions • Have health care providers changed practices related to MNCH/nutrition High Impact Evidence-Based practices? If so, which of these practices are a result of TA from THNA? • Which specific changes in health provider practices have been measured? • What are barriers to changing health provider practices? Has the activity done anything to address these barriers? • Have any related policies, standard operating procedures (SOPs), or protocols been changed as a result of THNA interventions? Are there those that are in-progress, but not yet implemented? • What specific aspects related to project training have been the most useful? Are aspects of health provider performance that still need to be improved? • Are there other donors or programs working with the same health providers in MNCH and nutrition? If yes, does THNA coordinate activities for synergies and cost efficiencies? • Are there client-related factors impacting access to MNCH and nutrition services? Has the project taken measures designed to reduce barriers to access and utilization of services? • Activity documentation (narrative report, M&E plan, results framework, etc.) • USAID interviews • IP briefings and documents • Local & national authorities • National health data and health information system • National policies, standards and protocols • International standards and protocols • Other Thousand Days donors and members of the Nutrition Forum • MOHSPP and other health officials • Other stakeholders • Desk review • KIIs • FGDs • Review of secondary performance data • Review of secondary contextual data from other sources such as stakeholder and staff KII • Client exit interviews • HF Checklists • Content analysis of beneficiary perceptions for key three dimensions of capacity building, adoption factors, and impacts • Analysis of activity performance and contextual data • Statistical analysis of mini￾survey results • Triangulate results and compare results to evidence-based Partnership for Maternal, Newborn, and Child Health (PMNCH) and nutrition interventions KEY STAKEHOLDERS TO BE INTERVEIWED • MOHSPP staff and managers • District Hospital staff • PHC managers • Head of Maternity units • Reproductive Health Center (RHC) – head of facility • RHC –Midwives, ,Nurses • Project staff 16 Evaluation questions were taken from the THNA Request for Task Order Proposal (RFTOP) solicitation. 50 THNA Evaluation Matrix 16Evaluation Questions (EQ) w/ Illustrative KII, FGD & Survey Questions Data Sources Methodologies Data Analysis Illustrative Survey Questions for tools • Which MNCH, nutrition (Infant and Young Child Feeding [IYCF]), and/or WASH technologies or practices17 have you received training? Adopted? • If there are key evidence-based practices to which you have not introduced or included, why not? • Are there skills in which you have been trained, but not yet used? • Have you benefitted from these new technologies or practices in the following areas: • Will you continue to use these new technologies or practices in the future? If no, why not? • How satisfied are you with the quality of activity training you received? Do you have recommendations that could improve the training? • Are there factors beyond THNA focus that inhibit your ability to practice what you have learned? • Has the THNA helped to strengthen provider networks? If yes, how? If not, why not? • Other MNCH projects & donors • Secondary data • Female & male groups and/or community group members • Tajikistan Demographic and Health Survey (TDHS) (for comparison) • Health staff from DHC, RHC, and HH • Mothers of children under 2 • Pregnant women • FP Clients (if any) • MiLs • Community leaders • Teachers • Village Development Committees (if available) • Student Peer Educators • Teachers • School Heads EQ2: To what extent has the activity increased access to a diverse set of nutrient rich foods throughout the year and why, especially the dietary diversity of pregnant and lactating women and children under 2 years of age? Illustrative KII & FGD Questions • What activities did the project select to enhance access to a diverse set of nutrient-rich foods? How were activities selected? Have they changed over the life-of-project (LOP)? If yes, why? • Were activities and target groups selected by the project consistent with evidence-base for increased consumption of these foods? • Activity documentation • Project baseline and monitoring data • IP • Local & national authorities • Desk document review • KIIs • FGDs • Review of secondary performance data • Analysis of community group formation, structure, scale, and coverage (if relevant) • Content analysis of perceptions on the type, quality, and quantity of training 17 Specific technologies and practices will be inserted based on focus activities of the project determined from background information provided to develop the workplan. 51 THNA Evaluation Matrix 16Evaluation Questions (EQ) w/ Illustrative KII, FGD & Survey Questions Data Sources Methodologies Data Analysis • What linkages (if any) did the THNA project establish with other FTF activities designed to increase availability of diverse nutrient￾rich foods? • What specific collaborative activities did THNA undertake to coordinate efforts to improve access to and consumption of nutrients at the beneficiary level? How were they measured? • Did THNA provide nutrition capacity building to all the same communities where TAWA was supposed to increase capacity for households to produce more nutritional foods? • Specifically, has access and consumption of nutritious foods changed in families of farmers targeted with FTF agriculture interventions? Agricultural interventions of other programs? • What tools did THNA use to assess household dietary consumption of specific nutrients by pregnant and lactating women and/or children under 2? Is THNA measuring changes in access and/or consumption patterns? • What specific activities did THNA do to promote immediate and exclusive breastfeeding in newborns and infants under the age of 6 months? • Wasting of infants has reportedly increased in Tajikistan in recent years, with the highest rates in infants under 6 months of age? Was this trend noted in the THNA catchment area? Did the project undertake specific measures to respond to this trend? • How has the activity affected the dietary habits and health-related behaviors of families with young mothers, children, and adolescent girls? • What were the barriers to increased consumption within households? Within communities? Did the project implement measures designed to overcome these barriers? Are there barriers that have not yet been overcome? If yes, what would be needed to overcome them? • How does THNA monitor and document improvements in food diversity in pregnant/lactating mothers, children less than 2 years old, and adolescents? How often? What does THNA do with this • Other stakeholders and related projects/programs • Pregnant and lactating women; Mothers of children less than 2 years; Adolescent girls; Fathers; Influential members of communities, including MiLs; MNCH health providers, adolescent girls, CHPs, CEs • Mini-surveys • Content analysis of importance and value of community-based approach for ensuring uptake and benefits from new practices and technologies • Analysis of activity performance data • Statistical analysis of survey results KEY STAKEHOLDERS TO BE INTERVEIWED • Ministry of Health and Social Protection of Tajikistan (MHSP) staff • Head of MCH department • Chief specialists (in charge for pediatrics and women’s health) of Regional Health department • PHC managers • RHC – head of facility • RHC – FD; Nurses • Project staff from Intrahealth, Mercy Corp, Abt • Health staff from RHC, HH • Community leaders • School authorities • School students Key partners • UNICEF 52 THNA Evaluation Matrix 16Evaluation Questions (EQ) w/ Illustrative KII, FGD & Survey Questions Data Sources Methodologies Data Analysis information? Is THNA an active member of the Tajik National Nutrition Forum? Did THNA participate in developing the (upcoming) Golden Thousand Days Nutrition Communication Strategy? If yes, how? Will THNA have a specific role in the implementation of the communication strategy in the FTF focus areas? • Are pregnant women provided with adequate supplies of Iron/Folate during pregnancy? If not, are they available for purchase in the private sector? Is there a way to determine if foods fortified (by other programs) are consumed by women and children in the THNA beneficiary population? • What additional measures could the project take to increase uptake of nutrition-related MNCH behaviors?? • Are IYCF behaviors documented in health facility records? If yes, where? Are there gaps (e.g., IYCF indicators not measured/documented)? Why? • How can IYCF-target behaviors be measured for progress in the 1,000 days strategy? • Does the THNA project participate in national-level 1,000 coalitions, working groups such as the National Nutrition Forum? • For activities that have been established, how many do you think will be sustained after the THNA project ends? Are there measures that THNA can take to promote sustainability or scale￾up of successful strategies? • What lessons learned does the THNA project have for reaching adolescent girls about nutrition? Are there strategies that are could to be successful that the project did not use? Illustrative Survey Questions Providers • Has the nutritional status of mothers and young children changed since THNA began working here in 2015? Has this been measured? Have you noticed changes in nutrition behaviors within families? If yes, do you know why? • WHO • WFP • WB • GIZ • Health staff – doctors and nurses from DHCs, RHC, and HH • Community educators ?? • Project engaged Volunteers 53 THNA Evaluation Matrix 16Evaluation Questions (EQ) w/ Illustrative KII, FGD & Survey Questions Data Sources Methodologies Data Analysis • Did you receive training in IYCF behaviors in the context of your work? What specifically were you trained in? Have you been able to implement this training? • How effective do you think the training was? • Did you have to make changes in protocols, procedures, and supervision as a result? • How do you rate the training on specific topics (list)? • How likely is it that you will continue to include IYCF promotion activities after external support ends? • Did your training specifically address the nutritional needs for adolescent girls? Were you able to reach adolescent girls to implement what you learned? • Were you involved in promoting health and nutrition-related behavior change in communities? If so, how? If not, who is responsible for SBCC promotion in communities? • Are nutrient-fortified foods accessible to pregnant and lactating women and small children in your area? • Do you have adequate supplies of Iron/Folate tablets for pregnant women and Vitamin A for small children? • Are young children routinely dewormed in your facilities? • Are newborns born in your facility put to breastfeed immediately after birth? If not, when? Clients/Beneficiaries • Since 2015, have you perceived there been any changes in the health nutritional status of pregnant women and young mothers? Children under age 5? Adolescent girls? • Have there been specific activities to teach you about improving health feeding behaviors (including breastfeeding) and providing support to young mothers and children to improve their nutrition? Can you list the topics and what you learned? • Have you been able to implement what you learned? • Are there any cultural factors that prevent you from implementing what you have learned about feeding yourself during and after • Mothers of children under 2 • Community leaders • Village community activists 54 THNA Evaluation Matrix 16Evaluation Questions (EQ) w/ Illustrative KII, FGD & Survey Questions Data Sources Methodologies Data Analysis pregnancy or feeding your baby in the first 2 years? Does your family and/or community support you to practice good nutrition practices? • Are there aspects of improving feeding practices (IYCF) that were not included or are there gaps in your knowledge? • Did you learn anything in specific about the health and nutrition needs of adolescent girls? • Are you able to implement what you have learned? • In your opinion, do you feel families now have access to nutritional foods including sources of protein and fruits/vegetables that they did not have prior to 2015? If yes, please give examples. EQ3: To what extent were (program supported) evidence-based guidelines and procedures institutionalized by the health systems in the FTF ZOI/ ? And what if any, were therebarriers Illustrative KII & FGD Questions • Have any evidence-based MNCH and nutrition guidelines and procedures have been targeted by THNA? At what level? Have they been adopted? • Are there evidence-based Global MNCH and nutrition practices that THNA has not been able to promote? Why? • Have any program-supported evidence-based guidelines and procedures in MNCH, nutrition, or WASH been institutionalized since 2015? If yes, what were they? What role (if any) did THNA play? • How has the activity affected the health and nutrition-related daily activities and commitments of female members of farming families targeted in the TAWA project? Other non-farming families? • How as the activity affected the intra-household decision-making within female farmers’ households? Other non-farming households? • Did the THNA project establish community-structures for activities at that level, or did they work within existing structures, including those established within the TAWA project? If THNA worked with community-based groups, has the activity affected • Activity documentation • IP • Local & national authorities • Policy documents • National policies and protocols • International evidence-based guidelines and protocols • Other stakeholders • Secondary data • Female & male • TDHS (for reference) • Health providers • Pre-service curricula • Desk review • KIIs • FGDs • Review of secondary performance data • Review of secondary contextual data • Mini-surveys • Health facility checklists • Analysis of nodes in theory of change • Assessment of changes in guidelines and procedures related due to THNA activities • Content analysis of how and whether provider and client behaviors have changed • Content analysis of pass￾through effects of THNA intervention as well as alternative explanations for changes • Analysis of activity performance and contextual data • Statistical analysis of mini￾survey results 55 THNA Evaluation Matrix 16Evaluation Questions (EQ) w/ Illustrative KII, FGD & Survey Questions Data Sources Methodologies Data Analysis women’s group (WG) members’ participation in community leadership positions? • Did THNA experience challenges working on national-level policy change while simultaneously working in a remote Rayon? • What can be done to increase the impact of activities like THNA on nutritional outcomes in diet diversity, child health and feeding behaviors, and health care-seeking behaviors? • Did THNA coordinate with TAWA and other FTF and development activities intended to synergize the impact of all programs? Were these efforts intended to reach the entire beneficiary population of these other projects? How did THNA monitor any results of synergy of multiple FTF approaches? • Have protocols and guidelines developed through THNA been endorsed by the GOT MHSP? Illustrative Survey Questions • Have you benefited from the activity in terms of improved performance in your job? (Please give examples.) Are there some things that are needed that did not benefit from THNA activities? Are their factors outside of the influence or control of the THNA project that impede progress in this area? • Has the availability of specific health and nutrition services increased as a result of the THNA project policy-level efforts? • Are there any other benefits (professional or personal that you have received from the activity? • Health facility members • Health facility data • Community members • Activity documentation • USAID interviews • Local & national authorities • Other stakeholders • Other projects & donors • National PMNCH strategies and frameworks • National related SDG strategies and frameworks • USAID Multi￾sectoral Nutrition Strategy • USAID Maternal and Newborn Health policies and guidelines • Global PMNCH guidelines • National PMNCH roadmap TO BE INTERVEIWED • Deputy Minister of MOHSPP • Head of MCH department Head of Safe motherhood unit of MHSP • Staff from MCH institute • Staffs from Pediatric research center • Head of Regional Health department • Chief specialists (in charge for pediatrics and women health) of Regional Health department • DHC managers Key partners • UNICEF • WHO • WFP • WB • GIZ • Health staff doctors, nurses, and midwifes from health centers of villages 56 THNA Evaluation Matrix 16Evaluation Questions (EQ) w/ Illustrative KII, FGD & Survey Questions Data Sources Methodologies Data Analysis EQ 4: Going forward, how, and in which areas, should USAID-funded MNCH & Nutrition activities be conducted to support the health system and/or population in improving the nutrition and health services to Mothers and Children? Illustrative KII & FGD Questions • What is the level of resources/funding that can be attributed to activity facilitation efforts? Did the project allocate activities according to priority needs? Are there important factors outside of the project’s control that are or will impact on the effectiveness of project efforts? • To what extent are stakeholders and other FTF projects involved in coordinating in activity-related areas? Why? • Are there other Ministries or national stakeholders that should be involved in future programs that were not involved with THNA? • Did the THNA project engage in Oblast and National-level advocacy activities related to the 1,000 Days Strategy and/or MNCH and nutrition policies? • What joint efforts exist across possible levels of coordination (community, district, and national levels)? How effective have they been? Have sufficient project activities been devoted to the cultural and behavioral factors in MNCH and nutrition? • What community, district, or national-level mechanisms/platforms exist that can support coordination? How have they functioned? How can they be made to function better? • What incentives/barriers are there for stakeholders to coordinate in activity-related areas? • What specific role would THNA and partners (and future USAID-supported) programs take to support the work of the National Nutrition Forum (supported by UNICEF) until the end of the project and beyond? • Are their gaps in implementation of the “1,000 Days Strategy” that should specifically be included in plans for future FTF/USAID programs? • Activity documentation • USAID • IP • Local & national authorities • Other stakeholders • Other projects & donors • National PMNCH strategies and frameworks • National related SDG strategies and frameworks • USAID Multi￾sectoral Nutrition Strategy • USAID Maternal and Newborn Health policies and guidelines • Global PMNCH guidelines • National PMNCH roadmap • Desk review • KIIs • Review of secondary performance data • Coordination: Content analysis of coordination levels, forms, mechanisms, and related incentives/barrier • Analysis of activity performance contextual data • Incorporate contextual barriers to multi-sectoral approaches to health and nutrition strategies • Capture concurrent efforts by the National Nutrition Forum and the project’s role in coordinating with them • Compare findings with national and international levels TO BE INTERVIEWED • Deputy Minister of MOHSPP • Head of MCH department Head of Safe motherhood unit of MHSP • Staff from MCH institute • Staffs from Pediatric research center • Head of Regional Health department 57 THNA Evaluation Matrix 16Evaluation Questions (EQ) w/ Illustrative KII, FGD & Survey Questions Data Sources Methodologies Data Analysis • Are there other groups (including civil society) that THNA (or future programs collaborate with to further the MNCH and/or nutrition objectives of THNA and USAID programs? • Chief specialists (In-charge for pediatrics and women health) of Regional Health department • DHC managers • Health staff from health centers of the districts Key partners • UNICEF • WHO • WFP • World Bank • GIZ Gender What activities did the project undertake that were designed to increase women and girls? Example could include: • Shared decision-making within the family • Self-efficacy of girls in changing behavior and health care-seeking • Improved access to existing health services • Expanded and more equitable opportunities for both men and women • Example-setting with project HR and training plans • Prevention and care for gender-based violence (GBV) How successful were these activities? Are there remaining challenges the project was unable to address? Has the project taken intentional efforts to include men and boys in MNCH and nutrition (including water and sanitation) activities of the project? • What is the gender-balance of project staff (including partners and community volunteers, if any?) If there were imbalances, have they been identified by the project? Any measures taken? Reports of the GOT Women’s Committee TDHS • Mission Gender Strategy • IP • TDHS • GBV reports • Adolescent Girl Health Studies • Nutrition studies • Desk Review • Project Data and Reports • FGDs • KII • Gender Reports • Health facility checklists • Client exit interview • Project HR and Training Reports Analysis will include health and nutrition-specific factors as well as general considerations such as female education and decision-making ability within families. • Deputy Minister of MOHSPP • Head of MCH department Head of Safe motherhood unit of MHSP • Staff from MCH institute • Head of Regional Health department • Chief specialists (in charge for pediatrics and women health) of Regional Health department 58 THNA Evaluation Matrix 16Evaluation Questions (EQ) w/ Illustrative KII, FGD & Survey Questions Data Sources Methodologies Data Analysis • What lessons learned should the project, partners, and stakeholders use to make continued improvements in future programs, including those implemented by the MOHSPP? • Did the project include activities in advocacy and/or capacity building related to violence against women? Were there any barriers to improving care for adolescent girls in health and nutrition? Has the project undertaken any activities to overcome these barriers? • What is the gender ratio within THNA IP staffing? If women are under-represented, did THNA take any measures to improve representation of women on staff or with partners? • DHC managers • RHC – heads and staff (doctors and nurses) • Community leaders 59 ANNEX 4: LIST OF DOCUMENTS REVIEWED 1. National Plan for reproductive health, mother, newborn, child and adolescent health up to 2020, Tajikistan. 2. MOHSPP, National program for Reproductive Health and Family Planning up to 2019- 2022. 3. Recommendations on maternal health, guidelines approved by the WHO guidelines review committee, WHO 2017. 4. Joint annual review of Tajikistan Health Strategy 2010-2020. 5. Clinical recommendations “Healthy Food and Nutrition of pregnant women” MOH and USAID, 2016. 6. Assessment Tool for Quality of hospital and primary care for mothers and child health adapted version, Khatlon oblast, Kurgan-tyube region, MOH and USAID, Tajikistan, 2016. 7. Supportive supervision guidelines MOH Tajikistan and WHO, 2017. 8. Clinical protocols for antenatal care, EmONC and newborn health care, Prevention and Infection Control guidelines Tajikistan, 2016, 2017. 9. Demographic and Health Survey 2016. 10. Care of a healthy child at the age of 0-24 months, MOH and USAID, 2018. 11. Joint Annual Report, 2016: Implementation review of national health strategy for 2010- 2020 Joint Annual Report, 2016: Implementation review of national health strategy for 2010-2020. 12. National development strategy of the republic of Tajikistan for the period up to 2030. 13. A national 1,000 golden days communication strategy and plan for the period of 2019- 2025, 2018. 14. Common results framework for improving nutrition in Tajikistan 2018-2022. 15. National micronutrient status survey in Tajikistan, UNICEF, 2016. 16. National action plan of the republic of Tajikistan: providing of the population with micronutrient fortified food for 2017-2020 (not endorsed yet). 17. Situational analysis improving economic outcomes by expanding nutrition programming in Tajikistan. UNICEF/WB, 2014. 18. Assessment of Hospital care for mothers and newborn babies in Qurghonteppa area of Khatlon region, USAID/Intrahealth/MHSP, 2016. 19. Nutrition and physical activity strategy for republic of Tajikistan 2015-2024. 20. National development strategy of the republic of Tajikistan for the period up to 2030. 21. WHO, Building Blocks of Health Systems Strengthening (2010 and revisions). 22. Tajikistan Gender Assessment, Asian Development Bank, 2016. 60 23. WHO, Essential Nutrition Actions, 2013. 24. Kavale, J., et al, Addressing barriers to exclusive breast-feeding in low- and middle￾income countries: a systematic review and programmatic implications, Public Health Nutrition, August 2017. 25. USAID Maternal and Child Survival Program, Maternal Nutrition Programming in the Context of the 2016 WHO ANC Guidelines for a Positive Pregnancy Experience, 2018. 26. WHO Antenatal Care Guidelines: USAID Maternal and Child Survival Project, 2017 27. WHO Beyond the Numbers: Reviewing Maternal Deaths and Complications to Make Pregnancy Safer, 2004 28. WHO, Assessment Tool for Quality of hospital and primary care for mother, newborn and child health, 2013 29. Global Alliance for Nutrition Improved Nutrition (GAIN), Food Fortification in Tajikistan: A Cost-Effective Strategy for Sustainable Growth, 2016. 30. Haas, J. and Brownlie T., Iron Deficiency and Reduced Work Capacity: A Critical Review of the Research Journal of Nutrition. 2001;131 in GAIN, 2016. 31. Horton & Ross, The Economics of Iron Deficiency Food Policy 28 (2003) 51-75 in GAIN, 2016. 32. Bitar, S. et al, Collaborative Quality Improvement in Maternal, Newborn and Child Health in Tajikistan, IntraHealth, 2019 33. Bitar, S. et al, Community-Based Approaches to Improve Maternal and Child Health in Tajikistan, IntraHealth, 2019. 34. Partnership for Maternal, Newborn and Child Health, A Global Review of the Key Interventions Related to Reproductive, Maternal, Newborn and Child Health, PMNACH, 2011 35. USAID, Tajikistan Nutrition Profile, 2018 36. USAID, Multi-Sectoral Nutrition Strategy 2014-2025, May 2014. 37. World Food Programme, Country Brief, Tajikistan, 2019 38. World Food Programme, Fill the Nutrition Gap, Tajikistan 2018. 39. Intrahealth, Tajikistan Health and Nutrition Activity, Annual Reports, Year 1-3, 2016- 2018; Quarterly Reports, 2016-2019. 61 ANNEX 5: LIST OF KEY INFORMANTS FOR INTERVIEWS AND FGDS # Name Position Ministry of Health and Social Protection of Population 1 Dr. Rahmatulloeva Sanavbar Head of Pediatric Unit, MHSP 2 Dr. Rakhmatulloev Sharali Chief Pediatrician, MHSP 3 Dr. Rakhimova Rano Head, International Relations Department, MHSP Republican Family Medicine Training Center 4 Dr. Bandaev Ilkhom Director, Family Medicine Center Research Institute of Obstetrics, Gynecology, and Perinatology (MCH Institute) 5 Dr. Mardonova Salomat Deputy Director 6 Dr. Kamilova Mavjuda OBGYN/EPC Specialist 7 Dr. Zakirova Firuza Neonatologist/EMONC Specialist 8 Dr. Nodirshoeva Roza OBGYN/Infection Control 9 Dr. Abdurakhmanova Gulsun OBGYN/EPC Specialist 10 Dr. Yunusov Abdugani Chief Neonatologist National Reproductive Health Center 11 Dr. Ahmedjanova Gulnora Chief RH Specialist Pediatric Research Center 12 Dr. Kayumova Dilshoda Nutrition Specialist, Pediatric Research Center Regional Health Department 13 Dr. Rajabzoda Mirali Head of Khatlon Regional Health Department 14 Dr. Tursunov Kholbozor Chief Pediatrician, Khatlon Region International Agencies 15 Miralibekova Shamsiya WFP, Coordinator 16 Bridget Jobson UNICEF, Head of Health/Nutrition Section 17 Shabanova Malohat UNICEF, Nutrition Specialist 18 Bolschman Natasha GIZ, Manager of Health and Nutrition Project 19 Latipova Mutriba WB, Health Project 20 Afendieva Gulara SINO Swiss Development Council (SDC) Project Director 21 Karimova Gulzira SINO (SDC) Project National Project Coordinator 22 Tobias Schüth SINO (SDC) Project, International Consultant on Community Health Team 23 Egamberdiev Maksud SINO (SDC) Project, Local Consultant, Community Health Team 24 Rakhimova Nargis UNFPA Country Manager Project Staff (THNA) 25 Roman Yorik Chief of Party, THNA 26 Gulnora Razykova Deputy Chief of Party, THNA 27 Tahmina Jabbarova Clinical Director, THNA, Abt 28 Faridun Khudonazarov M&E Manager, THNA, 29 Halima Boboeva Technical Specialist, MCH Hospital, THNA, Abt 30 Surayo Pulatova Technical Specialist, MCH PHC, THNA, Abt 31 Ato Tabarov Regional Manager, Khatlon Region, THNA 31 Mavjuda Maksudhojaeva Regional Coordinator, THNA 32 Qodirov Nosirjon Regional WASH Specialist, THNA District Project Coordinators (THNA) 33 Fozilova Marhabo Shahrituz/N. Khusrav 34 Karimova Mavjuda Dusti 35 Rahmatulloeva Havasmoh Yovon 36 Jaborova Zamira Qubodiyon 37 Hakimov Jamoliddin Khuroson 62 Persons Interviewed From the Selected FTF Districts of the ZOI # Name Position Dusti District 1 Mallazoda Saidullo Hospital Manager 2 Aliberdiev Murod Head, Maternity Unit 3 Khudoiberdiev Jumageldi Local Trainer/Hospital-Level NRC Responsible Person 4 Mahmudova Barno Chief Midwife 5 Bosorboy Sharipov PHC Manager 6 Mulkiev Merdan Deputy PHC Manager 7 Zulfiya Mirzoeva Local Trainer/PHC-Level NRC Responsible Person 8 Firuza Shamsova Deputy Hospital Manager 9 A. Murtazov Director, HLSC 10 Zulfiya Director, RHC N. Khisrav District 1 Nazar Kholmurodov Hospital Manager 2 Minoyat Qarshieva Head, Maternity Unit/Local Trainer/Hospital-Level NRC Responsible Person 3 Gulnora Chief Midwife 4 Salomat Midwife 5 Parvina Midwife 6 Ergashoy Midwife 7 Umed Khujanov PHC Manager 8 Mirzoeva Khosiyat Local Trainer/PHC-Level NRC Responsible Person 9 Shoimardonqulova Sayoat Director, RHC Shaartuz District 1 Boboeva Husnoro Deputy Hospital Manager 2 Kakharova Ranokhon Local Trainer/Hospital-Level NRC Responsible Person 3 Dilafuza Usmonova Head, Maternity Unit 4 DIlrabo Chief Midwife 5 Malohat Midwife 6 Nazira Midwife 7 Qodirova Munavara Local Trainer/PHC-Level NRC Responsible Person 8 Turamurodov Yigitali IMCI Center Manager 9 Lugmonov Jaloliddin Director, HLSC 10 Nuralieva Niroga Head, RHC Qabadiyon District 1 Oimatov Kholmurod Hospital Manager 2 Lailo Alimatova Head, Maternity Unit 3 Bobogulova Mashura Deputy Hospital Manager In-Charge MCH 4 Karomathon Chief Midwife 5 Jamilabony Midwife 6 Sanavbar Midwife 7 Karim Juraev PHC Manager 8 Bobiev Jamshed Local Trainer/PHC-Level NRC Responsible Person 9 Panji Qarshiev Director, District HLSC 10 Toshev Staff, District HLSC (ex-Hospital Manager) Yovon District 1 Burhonidin Khalilov Hospital Manager 2 Abduholiq Jurabekov Head, Maternity Unit 3 Sharipov Ismoil Local Trainer/Hospital-Level NRC Responsible Person 4 Nargiz Abdulloeva IMCI 5 Gulshanoy Chief Midwife 63 # Name Position 6 Khadicha Midwife 7 Gulrehsor Midwife 8 Zikrullo Ahmadulloev PHC Manager 9 Rahimov Makhmurod Local Trainer/PHC-Level NRC Responsible Person Khuroson District 1 Rajabali Gulov Hospital Manager 2 Pardahol Saidmurodova Head, Maternity Unit 3 Turaev Jamshed Local Trainer/Hospital-Level NRC Responsible Person 4 Parviz Muhamad PHC Manager 5 Saidova Guljahon Local Trainer/PHC-Level NRC Responsible Person 6 Ahmedova Jamila Head, RHC 7 Saidova Farida Chief Midwife 8 Gulrukhsor Turkmenova Director, HLSC 64 Numbers of FGD Participants by Type, Gender District Community Village Target Women Men Dusti Nuri Vaksh Navobod Pregnant women 12 - MiL 9 - Dusti Gulmurod Kahramon Pregnant women 8 - Mothers of children under 2 0 - MiL 9 - Dusti Dusti DHC CE 10 3 DHC CHP 11 - Dusti Pasariq School # 11 - 6 Total 59 9 N. Khisrav Istiqlol Bishkent CE/CHP 14 5 N. Khisrav Istiqlol Arabkhona MiL 9 0 Pregnant women 10 - Total 33 5 Qabadiyon 20 solagi Istiqloliyat Sohibkor Pregnant women 9 - Mother in Law (MiL) 11 - Total Qabadiyon Zarkamar Zarkamar Mothers of children under 2 9 - MiL CE/CHP - - Qabadiyon Chorboq School # 9 1 3 Total 30 3 Shaartuz Sayod Kisil Askar Pregnant women - - Mothers of children under 2 - - MiL - - Mixed 9 0 CE/CHP 2 0 Shaartuz Kh. Kholmatov B. Qaraboev Pregnant women - - Mothers of children under 2 11 - MiL Mixed - - Total 22 0 Khuroson Aini Istiqlol Mixed 12 - - - Khuroson Hiloli Somoniyon Mothers of children under 2 8 - MiL 7 - Total 27 0 Yovon Obshoron Tagoyobod 2 Mothers of children under 2 11 0 MiL 8 0 65 District Community Village Target Women Men CE/CHP 3 0 Yovon G. Yusupova Dashtobod Mothers of children under 2 10 MiL Yovon G. Yusupova Buston-2 School #49 1 4 Total 23 4 GRAND TOTAL 194 21 66 ANNEX 6: DATA COLLECTION TOOLS 67 Client Exit Interview Tool GENERAL INFORMATION (FILL IN THIS SECTION BEFORE THE INTERVIEW) Interviewer _______________________________________________________ Health Facility Name ________________________________ Location________________________District_________________________ Date_____/_____/_____ (day/month/year) CLIENT INFORMED CONSENT (READ WORD FOR WORD): Hello, I am______________. We are studying the quality of care that health personnel in this clinic provide to clients. I will ask you questions regarding the interaction and treatment of the persons who attended you during your visit as well as the services you received. I do not need to know your name, and your answers will be completely confidential. Do you agree to have this interview? (IF THE CLIENT DOES NOT ACCEPT, THANK HIM OR HER AND END THE INTERVIEW.) In order to ask the woman respondent, the correct questions, be sure to ask her the reason for today’s visit and only ask the questions that relate to that reason (service). Interview # 1 2 3 4 5 Client Satisfaction: Mix of ANC, FP, and Child Health Clients Clients Satisfaction: All clients S= Satisfied UC= Uncertain NS= Not Satisfied Q1. How would you describe your overall satisfaction with the service you received? S UC US S UC US S UC US S UC US S UC US Q2. How long did you wait to see the health care provider? (in minutes) ___ ___ ___ ___ ___ Q3. Are you satisfied with this waiting time? S UC US S UC US S UC US S UC US S UC US Q4. Are you satisfied with the overall cleanliness of the health facility? S UC US S UC US S UC US S UC US S UC US Q5. Are you satisfied with the privacy at the exam room? S UC US S UC US S UC US S UC US S UC US Q6. Are you satisfied with the time given to you by the health providers? S UC US S UC US S UC US S UC US S UC US 68 Q7. Are you satisfied with the health provider attitude with you? S UC US S UC US S UC US S UC US S UC US ANC Clients: Pregnant Women Maternal Health Services: Antenatal Care (ANC) Clients Q8. How many ANC visits should you have before you have your baby? Q9. Did the provider discuss danger signs in pregnancy (bleeding, severe headache, dizziness, etc.)? Y/N Y/N Y/N Y/N Y/N Q10. Please let us know what services you received today. For example, was your weight taken today? Y/N Y/N Y/N Y/N Y/N Q11. Was your blood pressure taken today Y/N Q12. Was your blood test taken today or during a previous visit in this pregnancy? Y/N Q13. Was your urine test taken today or during a previous visit during this pregnancy? Y/N Q14. Were you provided with iron/folate tablets? Y/N Q15. How many iron/folate tablets were you given? Q16. Did your provider explain to you why it is important to take iron/folate tablets? Y/N Q17. If you were not provided with iron/folate tablets, were you told to get them somewhere else (e.g., pharmacy, another facility)? Y/N Y/N Y/N Y/N Y/N Q18. If you were prescribed and/or provided with iron tablets did provider explain how minimize annoying side effects? Y/N Y/N Y/N Y/N Y/N Q19. Were you provided with advice on what foods or beverages to consume or not consume with iron/folate tablets (such as fruit juices or tea)? Separate question, please insert and renumber “Did your health worker tell you if you should reduce the amount of bread you eat during your pregnancy?” Y/N Y/N Y/N Y/N Y/N Q20. Were you provided counseling on what foods and in what quantity that you should eat during your pregnancy? Q21. Did you discuss how you will feed your baby after he/she is born? Specifically, did your health worker discuss breastfeeding? Did your health worker advise against giving tea or water or sugar to your newborn (separate question, please insert and renumber) Y/N Y/N Y/N Y/N Y/N 69 Q22. Did you discuss where you will deliver your baby? Y/N Q23. Will you deliver at a health facility? Y/N Q24. Did your provider discuss how many ANC visits in total that you should have before the baby is born? Y/N Well-Baby Clients Child Health Care Services: Mothers of Well-Baby <2 yrs Q25. What is the age of your child (in months)? Q26. Was your child’s weight taken today? Q27. Was your child’s height taken today? Q28. If weight or height were taken, did the health worker (provider) explain whether the baby’s growth is normal? If not, were you counseled on what to do (feeding)? Was your child referred for treatment of malnutrition? Q29. Were you given any verbal and/or printed information on: Ask each topic separately: • Exclusive breast feeding, • Child feeding practices (child over 6 months), • Vitamin A supplementation, or • Immunization? Y/N Y/N Y/N Y/N Y/N Q30. Was your child’s immunization record checked? If the child was missing any vaccinations (per schedule) was the child offered the missing vaccine? Y/N Y/N Y/N Y/N Y/N Q31. Were you prescribed and/or provided with Vitamin A/D drops or capsules for your child at this or at any visits in the last 6 months? Y/N Y/N Y/N Y/N Y/N Q32. Has your child ever been dewormed? Y/N Q33. If yes, when? Sick-Baby Clients Child Health Care Services: Diarrheal Diseases (only if facility visit due to Diarrheal Disease) Q34. Was your child provided zinc today? Y/N Q35. Was your child provided oral rehydration salts (ORS) today? Y/N Q36. If neither were provided, were you advised where to get them? Y/N Q37. Were you instructed how to administer ORS or zinc? Y/N Y/N Y/N Y/N Y/N Q38. Were you counseled on the need to increase fluids (breast milk, soup, water) for your sick child? Y/N Y/N Y/N Y/N Y/N Q39. Were you instructed to continue feeding, including breastfeeding your child when sick? Child Health Care Services: Acute Respiratory Infection (only if facility visit due to ARI) Q40. Was your child’s ARI determined to be pneumonia? Y/N Y/N Y/N Y/N Y/N Q41. Were you given antibiotics for your child? Y/N Y/N Y/N Y/N Y/N 70 Q42. If yes, what is the name? (look at the medicine or prescription and write the name down) Q43. If you were not given antibiotics (but were told your child should have them) were you told where to get them? Y/N Y/N Y/N Y/N Y/N Q44. Were you instructed how to administer antibiotics? Y/N Y/N Y/N Y/N Y/N Child Health Services: Nutrition Counseling Q45. Was your child weighed today? (if not answered above) Y/N Q46. If yes, did the health worker explain whether your child had the correct height or weight for their age? Y/N Q47. Did the health worker counsel you on how to feed your baby? Y/N Q48. Did the health worker explain the different types of foods that you should give your baby? Y/N Q49. If you are breastfeeding, have you been counseled about when to introduce solid foods and what foods and how to introduce them? Y/N Q50. Did the health worker explain to you the amount of food to provide the child and how often? If you had questions, did the health worker answer them? Y/N Q51. Did the health worker provide information to you on how to have increased access to quality nutritious foods for you and your baby? Y/N Women Health: Family Planning Women Health Services: Postpartum, Family Planning or Birth Spacing (whenever FP services are offered. These questions can be asked to postpartum clients) If 10 clients are not available, answer for those that are available. If no clients are available write “No Postpartum or FP Clients at this facility today” Q52. Did your health worker discuss family planning or birth spacing with you? If no, skip the rest of the questions in this section. Q53. If yes, what methods were available? Was more than one method of contraception discussed with you for you to choose? Q54. Were you provided a family planning method today? If yes, write down the methods Q55. (If yes to Q54) During your consultation today, did the provider explain how to use the method? Y/N Y/N Y/N Y/N Y/N Q56. Did the provider talk about possible side effects? Y/N Y/N Y/N Y/N Y/N Q57. Did the provider tell you when to return for follow￾up or if you have any side effects? Y/N Y/N Y/N Y/N Y/N Q58. Did the provider (health worker) discuss the benefits of family planning or child spacing for your health and or the health of your baby? Q59: Did the health worker answer any questions that you had about FP/Birth Spacing and/or the FP/Birth Spacing method? 71 General Guidelines: 1. Identify a woman of reproductive age (15-49) who is leaving the clinic. If she is pregnant, ask if the purpose of her visit was for herself or for one of her children. Women who have come for ANC should be included in that interview. Women coming for well or sick child care should be asked those questions. If the woman is not pregnant but has a child under 2 years of age and came for family planning services, ask her those questions. 2. Introduce yourself and ask if you can ask her few questions, which will not take more than 5 minutes. Explain that the questions are not personal and are anonymous. Questions are to aid planning and improving services at this facility. Though clinic staff are informed of your work, you are not part of the facility staff and the clinic staff will not know individual responses. 3. If the woman agrees to be interviewed, determine if she fits one of the categories of interest. If not thank her and find another woman: Pregnant women (ANC) (n=5) Mothers of U5 children: clinic visit due to DD or ARI (n=5) Mother of U5 children: well-baby visit (n=5) Women/Girls for FP services 4. Complete interviews for five (5) women in EACH of the above categories. NOTE: A maximum of 20 client exit interviews may need to be completed at each health facility in each category. However, it is possible that a pregnant woman will be attending clinic for another under two child (sick or well-baby visit), though she can also be asked the ANC questions. Thus, sometimes fewer than 10 individual interviews might be required. 72 THNA Health Worker Survey Date: Interviewer’s name/ID: Name of Facility: District: Jamoat: Facility Type (choose one): 1. District Hospital 2. District Health Center 3. Rural Health Center 4. Health House Instructions for the interviewer: Only ask questions relevant to the area of the health workers direct responsibility. Mark N/A next to questions that do not pertain to their area of expertise. Health worker questionnaires will be provided in larger facilities with sufficient numbers of staff in project focus areas of support. (Informed Consent on purpose of the questionnaire and request for voluntary participation, including explanation of how data will be used.) “Hello, my name is ____________, we are interviewers from Z-Analytics (Тахлил ва Машварат), working together with USAID, we are conducting a survey on the THNA project implemented in your area on health and nutrition of mothers and children. You were randomly selected to take part in this survey. We would like to have your time helping us answering these survey questions. We ensure that your information will be kept confidential. Your individual responses will be combined with the responses of other health workers in this Region to provide an overall assessment of what went well in the THNA project, what was not so successful and provide suggestions to USAID, other donors, and the Ministry of Health to use in designing future programs.” Background Questions Questions 1. Have you participated in Maternal and Child Health and/or Nutrition trainings provided by the THNA (Tajikistan Health and Nutrition Activity) project? a. Yes (If yes more on to the next question) b. No (Stop the survey and thank the respondent for their time) 2. Who trained you? a. THNA/USAID/IntraHealth Project staff b. Colleagues 3. Where did the training take place? a. At district level (Resource center) b. At RHC 73 4. What is your name? _______ 5. Technical Area: (Check all that apply) a. Maternity in Hospital b. Newborn Care in Hospital c. Pediatric department Hospital d. Family medicine in PHC (КАТС) e. Other (specify) 6. Type of Health Worker: a. Obs/Gyn b. Pediatrician c. Family medicine doctor d. Other (What type?_______) e. Hospital Nurse f. Family nurse at PHC g. Midwife h. Feldscher i. Other (specify) 7. Years of training in specialty a. Less than 2 years b. Three years c. 5-6 years d. Other 8. Length of time in this position a. Less than one year b. 1-3 years c. More than 3 years 9. What type of trainings have you participated in? (Multiple answers possible) a. EmONC (Emergency Obstetric and Newborn Care) Ask only OB/GYN b. Kangaroo Mother Care Ask only postpartum/neonatology workers c. Partogramme management d. EPC 74 e. Infection control f. Antenatal care (Ask Family Medicine Only) g. Anemia h. Child nutrition/ Infant and Young Child Feeding, IYCF (complementary feeding) i. IMAM (Integrated Management of Malnutrition) (pediatrics only) j. IMCI Integrated Management of Childhood Illnesses (pediatrics only) k. Supportive supervision l. Quality improvement/Beyond The Numbers m. Other Quality of training On a scale of 0 to 5, with 0 being the worst and 5 being the highest, please provide your opinion of the quality of the trainings according to the questions provided below: (LIST each type of THNA training for each type of training listed above. Fill this table for each training mentioned above. ) Questions Answers 1. To what extent do you feel you have learned from the TNHA project training received? (Please circle the score number that you feel most closely represents your views) Learned a lot 5 4 3 2 1 Learned nothing 2. Was the length of the training appropriate enough? Very appropriate 5 4 3 2 1 Not very appropriate 3. Was the content of the training appropriate enough? Very appropriate 5 4 3 2 1 Not very appropriate 4. How useful was the training for your day-to-day work? Very useful 5 4 3 2 1 Useless 5. Was it theoretical-oriented or practical-oriented training? Practical-oriented 5 4 3 2 1 Theoretical-oriented 6. What have you NOT learned that you needed to and/or expected to learn during the training? Please describe fully any items 7. Was there any pre- and post-test evaluation of the training? Yes / No 8. Please let us know if there are any proposals or offers you might have with regards to improving this training? 1. Increase the training period 2. Repeat the training again or at regular intervals 3. Less theory and more practice during the training 4. More support after the training 5. Follow up with other activities (mentoring/monitoring/coaching) 6. Other 10. From your point of view, has the health and/or nutritional status of mothers and young children in your area changed since THNA began working here in 2015? 75 a. Yes (If yes go to the next question) b. No (If no skip to question 10) c. Don’t know/No answer 10.1 If yes, how? a. Reduced morbidity rate b. Reduced mortality rate c. Reduced malnutrition d. Reduced Anemia e. Decrease in the number of home deliveries f. Improved quality of services g. Staff Acquired valuable knowledge and skills h. Strengthened providers networks i. Other 11. Did the training you received specifically address the nutritional or health needs of adolescent girls? a. Yes b. No 12. Were you able to reach adolescent girls to implement what you learned? a. Yes b. No c. Partially d. Not always e. Other 13. Do you have adequate supplies of Iron/Folate tablets for pregnant women? a. Yes b. No (- question 13) 13.1 If yes, do they (pregnant women) take them? a. Yes b. No c. Sometimes d. I don’t know 14. Do you have adequate supplies of Vitamin A for small children? a. Yes 76 b. No ( question 14) 14.1 If yes, do they (children) take them? a. Most of the time b. Sometimes c. Never d. I don’t know e. Other 15. Are young children routinely dewormed in your facility? (Primary Health only) a. Yes b. No c. Don’t know d. Other 16. Were you involved in promoting health and nutrition-related behavior change in communities and RHC /HH centers? a. Yes b. No ( go to question 16) c. Don’t know/Don’t remember ( go to question 16) 16.1 If yes, how? (Please choose every option the respondent mentions a. I am involved in supervising community health volunteers (CHVs) trained by THNA. b. I coordinate with THNA for community-level activities. c. I am directly responsible for services to clients at the health center . d. I collaborate/supervise the food security Community Educators (CE) trained by the project. e. I receive reports from CHVs. f. I receive reports from CEs. g. I supervise health workers working at smaller health facilities. h. Other ____ 17. Are you following infection prevention procedures? a. Yes b. No ( go to question 17) 17.1 If yes, how? a. Improving of disinfection, washing, sterilizing, sorting, packaging, and sterilizing the medical instruments 77 b. Improved waste management, proper disposal of hospital waste c. Washing hands with soap, following the procedures of hand washing d. Proper glove use e. Clean water storage f. Preparation and use of antiseptics g. Other 18. Do you counsel mothers on good hygiene? a. Yes b. No ( go to question 18) 18.1 If yes how? (Multiple answers possible, tick all answer options that the respondent mention) a. Washing hands with soap b. Washing hands with soup after using toilets c. Washing hands with soup before preparing food for the baby d. Washing hands with soup before feeding the child e. Washing hands with soup after cleaning the baby f. Using hand sanitizers g. Using latrines h. Other 19. Are there any other benefits (professional or personal that you have received from the activity? a. Yes, b. No (if no go to question 19) 19.1 If yes, Please provide examples. ________________________ 20. Do you have any recommendations that you would like to share with us for similar projects in future, especially related directly to the work that you do? __________ 78 Beneficiary KII and FGD Questionnaire Interviewer_______________ Date___________________ District ____________________ Nearest Health Facility Village _____________ Community______________ Type of Beneficiary: Pregnant women __________ Mothers of Children Under 2 _________ Informed Consent Obtained Introduction We will be asking questions related to THNA’s work in this community and a few questions about TAWA, if they were working there. Clients/Beneficiaries 1. Since 2015, have you perceived there been any changes in the health or nutritional status of pregnant women and young mothers? Children under age 5? Adolescent girls? 2. Have there been any changes in your closest health facilities (including health workers) that have increased your access to MNCH or nutrition services? 3. Are you familiar with the activities of the THNA in your community and local health facilities? If yes, can you name the activities of THNA? Did you participate in any of these activities? 4. Has the quality of health services available to you improved since the THNA project began? If yes, how? If no, why not? 5. Are you able to independently decide to go to seek health services? 6. If yes, where do you normally go for health and nutrition advice? If no, what prevents you from acting to seek care from a health worker? 7. Have there been specific activities to teach you about improving health feeding behaviors (including breastfeeding) and providing support to young mothers and children to improve their nutrition? Can you list the topics and what you learned? 8. When is the most important time for mothers and children to receive the right kind and amount of nutrition? 9. Have you been able to implement what you learned? If yes, would you provide examples? If not, can you explain why not? 10. Are there any cultural factors that prevent you from implementing what you have learned about feeding yourself during and after pregnancy or feeding your baby in the first 2 years? Does your family and/or community support you to practice good nutrition practices? 11. Are there aspects of improving infant and child feeding practices (IYCF) that were not included or are there gaps in your knowledge about how to have good nutrition for you 79 or your child? 12. Did you learn anything in specific about the health and nutrition needs of adolescent girls? 13. Are you able to implement what you have learned? 14. In your opinion, do you feel families now have access to nutritional foods including sources of protein and fruits/vegetables that they did not have prior to 2015? If yes, please give examples. 15. Are the men in your family supportive of any efforts you make to implement improved health or nutrition behaviors? 16. Did you and/or members of your family participate in the agricultural or economic support from the TAWA project? If yes, what activities were you involved in? Are there remaining challenges, aside from your knowledge, that make it difficult to increase nutritious foods in your family’s diet, especially when you are pregnant or feeding a young child? 80 Key Informant or Group Interview: CE and CHP (done separately) (1) health and (2) food security Respondent Name_______________________________ Title____________________ Jamoat ________________________ Village __________________________ Oblast________________ 1. What is your current place of work or institution? 2. Tell me about your involvement in the community? a. What types of activities have you been involved in? b. What tools/ information have you used that have been particularly effective in reaching and motivating family members to change their eating habits? 3. Have you participated in the trainings conducted by the USAID/THNA project? If yes, please specify the subject of the training: The duration of the training was: less than 3 days, more than 3 days 4. Who facilitated these trainings: a) Local trainers b) Project trainers c) RHC doctors 5. Were these trainings useful in your daily volunteer work: a) yes, b) in some degree 6. Did someone from project supervised your activities at village level: YES; NO: DNK 7. Did you submit reports to your supervisor: YES; NO; DNK If yes, to whom ___________and how often: weekly; monthly; other _____ 8. Is there any difficulties to gather HH member in your community: Yes, No DNK? If yes what type tactic you have used to reach your beneficiaries? 9. What was an average length of your meeting with beneficiaries: lest than 10 min; 10-14 min; more than 20 min; 10. How many hours a day do you spend on volunteer work in your village/community 11. From your knowledge and experience in the community, what are some of the most and least effective approaches to change attitude and practices in nutrition/ food preparation? 12. Do you believe that poor diet is a problem in your community? YES NO DNK If yes, How does poor diet seem to be a problem in your community? 13. Did you see any change, improvement in nutrition/died diversity among your beneficiaries? 81 Key informant Interview: Government Health, Nutrition and Food Security Officials (Questions will be adapted according to the type of Stakeholders and areas of expertise in a separate questionnaire.) Introduction Thank you very for taking time to participate in the THNA program final performance evaluation interview. My name is______________ and I will be asking you some questions which will guide the interview process and help us to understand what went well in the THNA and what could be changed or strengthened in future reproductive, maternal, newborn, adolescent, and child health (RMNACH) and nutrition programs in this region. The information provided will be handled confidentially. Please limit your response to specific activities of the THNA program. This questionnaire asks about: background, capacity building, and human resources for health, community activities, and gender-related issues as well as specific questions about THNA activities related to the health and nutrition of mothers and children. Please provide answers to the best of your ability. If I ask a question about something you are not familiar with, please say so and we will move on to another question. I will be taking notes that will be combined with many interviews at each level as well as survey data from health workers to try to get a comprehensive perspective on the program that will help provide USAID, the GOT, and stakeholders and help them plan the way forward in similar programs. We will not identify respondents individually in our reports, so please feel free to be candid. Section One: Background inform ation Date:______________________ District (if relevant):__________________________ Type of Respondent (position): ________________________ Sex: Male____ Female_______ Program Area (Health, Nutrition, Food Security, Other):________________ Interviewer(s):________________________ EQ = Corresponding Evaluation Question EQ Section 1: Progress Towards Results 1,2,3 01 Overview: As a result of work done by THNA activities, have you noticed any improvements in improving access to and/or utilization of maternal, newborn, and child health (MNCH), women’s RH, and nutrition and/or food security of mothers and children in in THNA target districts especially with regard to the “1,000 Days Strategy”? Please give examples: From the Matrix: EQ1 • Have health care providers changed practices related to MNCH/nutrition High Impact Evidence-Based practices? If so, which of these practices are a result of technical assistance from THNA? • Which specific changes in health provider practices have been measured? • What are barriers to changing health provider practices? Has the activity done anything to address these barriers? 82 • Have any related policies, standard operating procedures, or protocols been changed as a result of THNA interventions? Are there those that are in-progress, but not yet implemented? • What specific aspects related to project training have been the most useful? Are aspects of health provider performance that still need to be improved? • Are there other donors or programs working with the same health providers in MNCH and nutrition? If yes, does THNA coordinate activities for synergies and cost efficiencies? • Are there client-related factors impacting access to MNCH and nutrition services? Has the project taken measures designed to reduce barriers to access and utilization of services? EQ2 • What activities did the project select to enhance access to a diverse set of nutrient-rich foods? How were activities selected? Have they changed over the life of project? If yes, why? • Where activities and target groups selected by the project consistent with evidence-base for increased consumption of these foods? • What linkages (if any) did the THNA project establish with other FTF activities designed to increase availability of diverse nutrient-rich foods? • What specific collaborative activities did THNA undertake to coordinate efforts to improve access to and consumption of nutrients at the beneficiary level? How were they measured? • Did THNA provide nutrition capacity building to all the same communities where TAWA was supposed to increase capacity for households to produce more nutritional foods? • Specifically, has access and consumption of nutritious foods changed in families of farmers targeted with FTF agriculture interventions? Agricultural interventions of other programs? • What tools did THNA use to assess household dietary consumption of specific nutrients by pregnant and lactating women and/or children under 2? Is THNA measuring changes in access and/or consumption patterns? • What specific activities did THNA do to promote immediate and exclusive breastfeeding in newborns and infants under the age of 6 months? • Wasting of infants has reportedly increased in Tajikistan in recent years, with the highest rates in infants under 6 months of age? Was this trend noted in the THNA catchment area? Did the project undertake specific measures to respond to this trend? • How has the activity affected the dietary habits and health-related behaviors of families with young mothers, children, and adolescent girls? • What were the barriers to increased consumption within households? Within communities? Did the project implement measures designed to overcome these barriers? Are there barriers that have not yet been overcome? If yes, what would be needed to overcome them? • How does THNA monitor and document improvements in food diversity in pregnant/lactating mothers, children less than 2 years old, and adolescents? How often? What does THNA do with this information? Is THNA an active member of the Tajik National Nutrition Forum? Did THNA participate in developing the (upcoming) Golden Thousand Days Nutrition Communication Strategy? If yes, how? Will THNA have a specific role in the implementation of the communication strategy in the FTF focus areas? • Are pregnant women provided with adequate supplies of iron/folate during pregnancy? If not, are they available for purchase in the private sector? Is there a way to determine if foods fortified (by other programs) are consumed by women and children in the THNA beneficiary population? 83 • What additional measures could the project take to increase uptake of nutrition-related MNCH behaviors?? • Are IYCF behaviors documented in health facility records? If yes, where? Are there gaps (e.g., IYCF indicators not measured/documented)? Why? • How can IYCF-target behaviors be measured for progress in the 1,000 days strategy? • Does the THNA project participate in national-level 1,000 days coalitions, working groups such as the National Nutrition Forum? • For activities that have been established, how many do you think will be sustained after the THNA project ends? Are there measures that THNA can take to promote sustainability or scale-up of successful strategies? • What lessons learned does the THNA project have for reaching adolescent girls about nutrition? Are there strategies that are could to be successful that the project did not use? EQ3 • Have any evidence-based MNCH and nutrition guidelines and procedures been targeted by THNA? At what level? Have they been adopted? • Are there evidence-based partnership for maternal, newborn, and child health (PMNCH) practices that THNA has not been able to promote? Why? • Have any program-supported evidence-based guidelines and procedures in MNCH, nutrition, or WASH been institutionalized since 2015? If yes, what were they? What role (if any) did THNA play? • How has the activity affected the health and nutrition-related daily activities and commitments of female members of farming families targeted in the TAWA project? Other non-farming families? • How as the activity affected the intra-household decision-making within female farmers’ households? Other non-farming households? • Did the THNA project establish community-structures for activities at that level, or did they work within existing structures, including those established within the TAWA project? If THNA worked with community-based groups, has the activity affected WG members’ participation in community leadership positions? • Did THNA experience challenges working on national-level policy change while simultaneously working in a remote Rayon? • What can be done to increase the impact of activities like THNA on nutritional outcomes, diet diversity, child health and feeding behaviors, and health care-seeking behaviors? • Did THNA coordinate with TAWA and other FTF and development activities intended to synergize the impact of all programs? Were these efforts intended to reach the entire beneficiary population of these other projects? How did THNA monitor any results of synergy of multiple FTF approaches? EQ4 Training : Would you comment on specific training and capacity building activities provided by THNA, including gender-specific health issues like GBV? ------------------------------------------------------------------------------------------------------------------------------------- ------------------------------------------------------------------------------------------------------------------------------------- ------------------------------------------------------------------------------------------------------------------------------------- ------------------------------------------------------------------------------------------------------------------------------------- ------------------------------------------------------------------------------------------------------------------------------------- ----------------------------------- 84 a) b) Developing training m anuals/resources (if relevant): ------------------------------------------------------------------------------------------------------------------------------------- ------------------------------------------------------------------------------------------------------------------------------------- ------------------------------------------------------------------------------------------------------------------------------------- ------------------------------------------------------------------------------------------------------------------------------------- ------------------------------------------------------------------------------------------------------------------------------------- ----------------------------------- c) d) Developing policies, strategies, and guidelines: ------------------------------------------------------------------------------------------------------------------------------------- ------------------------------------------------------------------------------------------------------------------------------------- ------------------------------------------------------------------------------------------------------------------------------------- ------------------------------------------------------------------------------------------------------------------------------------- ------------------------------------------------------------------------------------------------------------------------------------- ----------------------------------- e) f) Conducting assessm ents; using data for decision-making : ------------------------------------------------------------------------------------------------------------------------------------- ------------------------------------------------------------------------------------------------------------------------------------- ------------------------------------------------------------------------------------------------------------------------------------- ------------------------------------------------------------------------------------------------------------------------------------- ------------------------------------------------------------------------------------------------------------------------------------- ----------------------------------- Supportive Supervision: 1) Health Workers (by levels) 2) Community Volunteers (by type) 3) Other g) h) Planning and m anagem ent of health, nutrition and food security services (specify type and level): ------------------------------------------------------------------------------------------------------------------------------------- ------------------------------------------------------------------------------------------------------------------------------------- ------------------------------------------------------------------------------------------------------------------------------------- ------------------------------------------------------------------------------------------------------------------------------------- ------------------------------------------------------------------------------------------------------------------------------------- -------------------------------- i) j) Com m unity m obilization for m aternal, neonatal, child health, and nutrition outcom es and food security activities: ------------------------------------------------------------------------------------------------------------------------------------- ------------------------------------------------------------------------------------------------------------------------------------- ------------------------------------------------------------------------------------------------------------------------------------- ------------------------------------------------------------------------------------------------------------------------------------- ------------------------------------------------------------------------------------------------------------------------------------- ----------------------------------- k) l) Other, specify: ------------------------------------------------------------------------------------------------------------------------------------- ------------------------------------------------------------------------------------------------------------------------------------- ------------------------------------------------------------------------------------------------------------------------------------- ------------------------------------------------------------------------------------------------------------------------------------- ------------------------------------------------------------------------------------------------------------------------------------- ----------------------------------- In your view, have THNA activities affected any changes in access and utilization of reproductive, maternal, newborn, child health (including adolescent, or RMNACH), and nutrition behaviors and food security information and services in its target districts since the THNA project began? 1. Coordination of key stakeholders 2. Ongoing clinical mentoring and supportive supervision 3. Ongoing technical support 4. Building institutional and human capacity for provision quality services 85 5. Providing financial resources (excluding grants) 6. Participation and provision of TA support in national, provincial and district MOHSPP planning and budgeting activities 7. Implementation of grants program/capacity building of grantees in grants management and organizational development • Did health facilities in your area of responsibility receive any grants from THNA to improve WASH? If so, how are you able to know about these activities? Were you involved in planning or implementing any of these activities? If yes, how? If not, what in your view are the key challenges for improving the health and nutrition and health status of mothers and children in your area? 1. Improving communication for RMNACH and nutrition behavior change 2. Improving access to health services 3. Improving access to improved health security 4. Improving quality of health services Specifically, what in your view are the key challenges in improving utilization of these health, nutrition, and food security services? 1. What specific strategies did THNA take to address these challenges? 2. Were they effective? 3. If yes, why do you think they were effective? 4. If not, why do you think they were not effective? Was it something under the control of the project or something(s) external to the project? a) b) What do you think future programs should do to improve access to and utilization of information and services for these health, nutrition, and food security areas? c) d) If you described improvements in access to and utilization of health information and services, do you think any improvements made by will be sustained after the project? If not, what do you think could have been done to ensure sustainability? Section 2: Capacity Building 3 What THNA strategies or activities are you aware of that have helped to strengthen community participation in improving health, nutrition and food security? In your view, have these activities contributed to a positive change? Are you familiar with any collaborative activities between THNA and the TAWA project related to food security and nutrition? Can you comment on whether these activities were effective? If yes, what do you think contributed to the effectiveness? If not, why do you think they were not effective? Based on your experiences or observations, how have THNA community-focused activities addressed gender issues (e.g., household decision-making, gender imbalances among health workers or volunteers), if at all? Specifically, can you comment on: Participation as volunteers in program activities Leadership roles in community groups Participation in community health planning Ability to seek care at health facilities Other, specify Are there groups that have community members have benefited more than others from THNA activities? If yes, why do you think this is the case? 86 Are there groups that have benefited least? Why do you think this is so? Section 3: Human Resources for Health 3 Has THNA contributed to improved health worker performance to provide quality services? In RMNACH and nutrition? If yes, please specify what services and what THNA did to improve them? Did THNA contribute to national strategies and/or processes to strengthen health systems, especially human resources for health, and/or supply chain management? Please give examples. Were these efforts effective? Please give examples. 3 In this context how would you assess specific capacity building programs promoted by THNA: Did THNA provide any capacity building in the following areas? 1. Involving district/regional teams in ongoing analysis of performance gaps and improvements 2. Incentive packages for community-based providers 3. Job-based training and mentoring program for rural and facility health workers 4. Developing a workforce performance system 5. Other: 4 What are the most important remaining gaps to address for providing access and delivery of quality RMNACH and nutrition/food security in the THNA project area? Section 4: Coordination and Integration 3 From your knowledge, what has been the coordination of THNA with other related projects in the region: 1. Other USAID FTF projects: (TAWA and any others) 2. Other donor projects, (World Bank, etc.) 3. Other GOT projects (specify) 4. Other: For any collaborative and/or integrated activities listed above: 1. Was the integration or collaboration effective? 2. If yes, why? If no, why not? Please give examples. 4 What could future programs working in the areas that THNA targeted do to increase access and utilization and quality of health and nutrition/food security services especially in the “1,000 Days?”: At national level At provincial level At district level At community level Section 5: Country Ownership and Sustainability 3 What do you think is the perception of GOT ministries responsible for services supported by THNA and GOT counterparts to plan, design, implement, manage, monitor, and evaluate health programs at the national, regional, and district levels? (Please be specific.) 1,2,3 What about (ask about each type of health or food security worker on separately)? What specific activities has capacity building of each specialist done that has moved your programs forward? 3,4 What are the most important human resources for health/nutrition/food security issues that should be the focus for future programs in order to decrease mortality and morbidity in mothers and children an improve their nutritional status? 3,4 Were protocols and guidelines developed by THNA endorsed by the GOT? Section 6: Gender Integration 1,2,3,4 To what extent have women been empowered to act in support of health behaviors and interventions in THNA target districts? 1. Participation as volunteers 2. Leadership roles in community groups 3. Participation in community health planning 4. Take independent action at the household level (e.g., feeding behaviors, health practices, etc.) 87 5. Take independent action to seek health, nutrition, and/or food security services 6. Address issues of GBV. 7. How much is male migration for employment a factor in deterring or encouraging mothers of young children to participate in health and nutrition services for themselves and their children? 8. Do you think there was a balance of activities focused on agricultural food security and household nutrition behaviors and practices? Were the agricultural/food security activities targeted to both men and women equally or more towards one gender? 9. Are there remaining barriers for women for them to take up healthy household behaviors and care-seeking for themselves and their children? For each of the issues listed above, can you comment on whether THNA directly addressed each gender-related factor? For those that THNA addressed, would you comment on the effectiveness? Who are the major influencers of mothers’ household and care-seeking behaviors? Did THNA work with all or most of these influencers in their project strategies? If yes, what were these strategies? If not, why not? Can you comment on whether THNA specifically targeted men for activities intended to lead to improvement in maternal and child health and nutrition indicators? If yes, what were they? If no, why? Are there strategies or activities that future programs could incorporated to ensure gender￾related health and food security issues would be addressed more effectively? Progress towards program goals and objectives • Have any evidence-based MNCH and nutrition guidelines and procedures been targeted by THNA? At what level? Have they been adopted? • Are there evidence-based PMNACH practices that THNA has not been able to promote? Why? • Have any program-supported evidence-based guidelines and procedures in MNCH, nutrition, or WASH been institutionalized since 2015? If yes, what were they? What role (if any) did THNA play? • How has the activity affected the health and nutrition-related daily activities and 88 commitments of female members of farming families targeted in the TAWA project? Other non -farming families? • How as the activity affected the intra -household decision -making within female farmers’ households? Other non -farming households? • Did the THNA project establish community - structures for activities at that level, or did they work within existing structures, including those established within the TAWA project? If THNA worked with community -based groups, has the activity affected WG members’ participation in community leadership positions? • Did THNA experience challenges working on national -level policy change while simultaneously working in a remote Rayon? • What can be done to increase the impact of activities like THNA on nutritional outcomes , diet diversity, child health and feeding behaviors , and health care -seeking behaviors? • Did THNA coordinate with TAWA and other FTF and development activities intended to synergize the impact of all programs? Were these efforts intended to reach the entire beneficiary population of these other projects? How did THN A monitor any results of synergy of multiple FTF approaches? EQ4 • What is the level or resources/funding that can be attributed to activity facilitation efforts? Did the project allocate activities according to priority needs? Are there important factors outside of the project’s control that are or will impact on the effectiveness of project efforts? • To what extent are stakeholders and other FTF projects involved in coordinating in activity - related areas? Why? • Are there other Ministries or national stakeholders that should be involved in future programs that were not involved with THN A ? • Did the THNA project engage in Oblast and National -level advocacy activities related to the 1,000 Days Strategy and/or MNCH and nutrition policies? • What joint efforts exist across possible levels of coordination (community, district, and national levels)? How effective have they been? Have sufficient project activities been devoted to the cultural and behavioral factors in MNCH and nutrition? • What community, district , or national -level mechanisms/platforms exist that can support coordination? How have they functioned? How can they be made to function better? 89 • What incentives/barriers are there for stakeholders to coordinate in activity-related areas? • What specific role would THNA and partners (and future USAID-supported) programs take to support the work of the National Nutrition Forum (supported by UNICEF) until the end of the project and beyond? • Are their gaps in implementation of the “1,000 Days Strategy” that should specifically be included in plans for future FTF/USAID programs? Are there other groups (including civil society) that THNA (or future programs collaborate with to further the MNCH and/or nutrition objectives of THNA and USAID programs? 90 ANNEX 7: BIOS AND SUMMARY INFORMATION OF TEAM MEMBERS Name: Jean Meyer Capps Position: Team Leader Key Qualifications: Jean Capps is an evaluation team leader and public health expert with more than 35 years of experience in public health; maternal and child health; health systems; M&E; Ebola, community health, and WASH programs; HIV/AIDS; FP; and nutrition/food security programs. Ms. Capps has served as team leader on 16 midterm and final evaluations, case studies, and program design for USAID and is experienced in all aspects of evaluation including evaluation design, tool development, qualitative and quantitative data collection, methods of analysis, stakeholder engagement, and presentation of results. As team leader for the USAID Pakistan MNCH/FP/Nutrition/Immunization Project Midterm Evaluation in 2016, she was responsible for leading all aspects of the evaluation design, including tool development, qualitative and quantitative data collection, methods of analysis, stakeholder engagement, and presentation of results to the USAID ET as well as production of the evaluation report. In 2014, as team leader for the USAID Georgia SUSTAIN MNCH/FP Final Evaluation, she led a team of three evaluation specialists in a review of TA provided to the public and private sector programs in maternal and newborn health and strengthening access to FP through pharmacies and private providers. In 2014, she led the USAID Global Health Child Survival and Health Project Final Evaluation in Liberia, an evaluation that included a 30-cluster survey to review project interventions including FP EPI integration, Community IMCI, and Water and Sanitation components integrated with HIV/AIDS and malaria programming. Quantitative methods included statistical interpretation and outcome analysis of survey results and qualitative group interviews. Ms. Capps’ responsibilities included tool development, managing field data collection, group analysis, and facilitating consensus findings. As Maternal/Child Health Specialist with the Asian Development Bank in Kyrgyzstan (2006-2007), Ms. Capps also designed an early childhood development program. 91 Name: Sabir Kurbanov Position: Senior Technical Advisor: Nutrition Key Qualifications Dr. Sabir Kurbanov is a Tajik nutrition and public health expert with more than 45 years of experience in public health; health policy; maternal and child health and RH; HIV/AIDS and PMTCT; nutrition; project and survey design and implementation; M&E; research; and teaching. As Health Specialist with UNICEF in Tajikistan, he was responsible for policy formulation and development; program design, management, implementation, and M&E of the health and nutrition projects (MCH, EPI, HIV, Nutrition) supported by UNICEF; strategic planning and program development; analysis of the country situation related to child health and nutrition; data gathering, analysis, and presentation for appropriate policy and strategy formulation; development of national laws and legislations related to child and newborn health/nutrition; and management of human and financial resources of over $2 million per year. Dr. Kurbanov has extensive experience designing and implementing surveys having worked on numerous surveys in Tajikistan including the USAID/Feed the Future Cost of the Diet (CotD) Survey in the Khatlon province; Health Policy Unit of the MOH household-based survey to assess access to ANC services; and MOH/World Bank Baseline Nutrition Survey in the Khatlon region where he developed research modules and materials, trained interviewers, monitored data collection, analyzed data, and wrote the report. In addition, as the nutrition consultant for the joint Food and Agriculture Organization of the United Nations (FAO) and Agency of Statistics (AOS) assessment of the Women’s Dietary Diversity Score Survey (WDDS), Dr. Kurbanov assessed the nutrition situation, analyzed the nutrition status of women and children, and performed data analysis and interpretation, working closely with AOS staff to develop proposals, implement training, and facilitate the process of integration of the minimum dietary diversity of women (MDD-W) into household budget survey modules. As the Chief Pediatrician of the MOHSPP in Tajikistan, he designed, implemented, and monitored MCH Programs/Projects, namely CDD/ARI, Breastfeeding, and EPI and conducted trainings and seminars related to child health and nutrition; and the Chief of the Infectious Department of the Mother and Child Health (MCH) Institute in Dushanbe, he designed and implemented MCH research studies and trained health workers. Dr. Kurbanov is also the former Deputy Dean of the Pediatric Faculty of the Tajik Medical University (1976-1990) where he trained medical students in pediatrics and developed the training and teaching materials/curriculum. He speaks Tajik, Uzbek, Russian, and English. 92 Name: Manzura Mirsaidova Position: Senior Technical Advisor: Maternal and Child Health Key Qualifications Dr. Manzura Mirsaidova is a Tajik doctor and maternal and child health expert with more than 18 years of experience in public health; maternal and child health; obstetrics and gynecology; RH and FP; nutrition; health systems; health policy; and training. As team leader for the Regional Health/Food Security/Nutrition, Education, and Social Program with GIZ in Tajikistan, Kyrgyzstan, and Uzbekistan, Dr. Mirsaidova was responsible for the strategic development and coordination of the project as well as global project planning and management. In addition to providing technical support to Ministries and the Government in promoting the project activities, indicators, and outcomes, she managed quality improvement and assurance and designed, prepared, and implemented conferences, workshops, and training courses. As Manager of the United Nations Population Fund’s (UNFPA) Reproductive Health, Advocacy, and Relationship Program in Tajikistan, Dr. Mirsaidova provided TA and capacity building to the GOT and policymakers in development of advocacy strategies and policies; was responsible for planning, budgeting, management, monitoring, and evaluation of the program for RH/safe motherhood, FP, and HIV/AIDS; coordinated development of the National Advocacy Policies and Strategies; improved the effectiveness and quality of national RH programs; and set up local capacity to address crucial health problems through trainings and workshops. As Professor at the Tajik State Medical University Department of Obstetrics and Gynecology, she developed and introduced clinical protocols and national guidelines in obstetrics and gynecology based on evidence-based medicine and taught courses on obstetrics and gynecology to medical students among other duties. Dr. Mirsaidova speaks Farsi, Russian, English, and German.