Innovation for Scale: Enhancing Ethiopia's Health Extension Package in the Southern Nations and Nationalities People's Region (SNNPR) Shebedino and Lanfero Woredas “The death of children has reduced, so we no longer waste our time or land burying the dead.” Cooperative Agreement No.: GHS-A-00-07-00023-00 Dates: 1 October 2007 – 30 September 2012 REPORT OF THE FINAL EVALUATION Submitted to USAID/GH/HIDN/CSHGP December 2012 Submitted by: Save the Children Federation, Inc. 54 Wilton Road, Westport, CT 06880 Telephone: (203) 221-4000 Fax: (203) 221-4056 Contact Persons: Eric A. Swedberg, Senior Director, Child Health and Nutrition Carmen Weder, Associate Director, Department of Health & Nutrition Principal writers and editors: External Evaluator/Team Leader: Dr. Peter Waiswa; Save the Children Home Office: Dr. David Marsh, Senior Child Evaluation Advisor/CCM Global Team Leader; Karen Z. Waltensperger, Senior Advisor, Health-Africa; Sharon Lake-Post, Editorial Consultant; Save the Children Ethiopia: Dr. Hailu Tesfaye, Child Survival Advisor; Getenet Kebede, Health Program Coordinator; Worku Tefera, M&E Coordinator; Partners: Asres Bedaso, MNCH Officer, Lanfero DHO; Bedio Badego, Shebedino DHO; Tofik Handegeba, MNCH Officer, Silti Zone Health Department; Agaro Godanam, Disease Prevention & Health Promotion Officer, Sidamo Zone Health Department; Demisse Denebo, Disease Prevention & Health Promotion Officer, SNNPR Regional Health Bureau; Luwei Pearson, UNICEF Ethiopia, Health Unit Head This report is made possible by the generous support of the American people through the United States Agency for International Development USAID). The contents are the responsibility of Save the Children and do not necessarily reflect the views of USAID or the United States Government. CS-23 Ethiopia, Final Evaluation Report, Save the Children, December 2012 2 Table of Contents ACRONYMS .................................................................................................................................. 3 A. PRELIMINARY INFORMATION (Executive Summary) ..................................................... 5 B. OVERVIEW OF THE PROJECT, OR STRUCTURE AND IMPLEMENTATION ............. 7 C. EVALUATION ASSESSMENT METHODOLOGY AND LIMITATIONS ....................... 12 D. DATA QUALITY AND USE ............................................................................................... 13 E. PRESENTATION OF PROGRESS TOWARD ACHIEVING PROJECT RESULTS ........ 15 F. DISCUSSION OF PROGRESS TOWARDS ACHIEVEMENT OF RESULTS ................. 17 G. CONCLUSIONS AND RECOMMENDATIONS ................................................................ 29 Annexes: Annex 1: Learning Briefs: Evidence Building…………………………………………………..30 Annex 2: List of Publications and Presentations Related to the Project………………………...49 Annex 3: Project Management Evaluation……………………………………………………....52 Annex 4: Work Plan Table……………………………………………………………………....54 Annex 5: Rapid CATCH Table………………………………….…………………………..…..60 Annex 6: Final KPC Report……………………………………………………………….…….62 Annex 7: CHW Training Matrix (sample)………………………………………………….…..130 Annex 8: Evaluation Team Members and Their Titles……………………………………..…..132 Annex 9: Evaluation Assessment Methodology………………………………………………..133 Annex 10: List of Persons Interviewed and Contacted During Final Evaluation…………...….151 Annex 11: Final OR Report (required for Innovation Grants)…………………………………154 Annex 12: Special Publications/Reports…………………………………………………..…....155 Annex 13: Project Data Form………………………………………………………………..…208 Annex 14: Grantee Plans to Address Final Evaluation Findings………………………………223 Annex 15: Grantee Response to Final Evaluation Findings (optional)…………….…….…….224 CS-23 Ethiopia, Final Evaluation Report, Save the Children, December 2012 3 ACRONYMS ACT Artemisinin Combination Therapy ANC Antenatal Care BC Behavior Change BCC Behavior Change Communication CCM Community Case Management CCM/P Community Case Management/Pneumonia C-IMNCI Community-Integrated Management of Newborn and Childhood Illnesses CS Child Survival CS-23 Child Survival-23 (USAID CSHGP 23rd cycle project) CSHGP Child Survival and Health Grants Program DHO District Health Office DIP Detailed Implementation Plan EDHS Ethiopia Demographic and Health Survey ENC Essential Newborn Care EPI Expanded Program of Immunization FE Final Evaluation FMOH Federal Ministry of Health GOAL Irish Non-governmental Organization HC Health Center HDA Health Development Army HFA Health Facility Assessment HEP Health Extension Package HEW Health Extension Worker HH Household HMIS Health Management Information System HP Health Post HSDP-III Health Sector Development Programme III iCCM Integrated Community Case Management IDI In-depth Interviews IEC Information Education Communication IFHP Integrated Family Health Program IMNCI Integrated Management of Newborn and Childhood Illnesses IMR Infant Mortality Rate IR Intermediate Result ITN Insecticide Treated Bednets IYCF Infant and Young Child Feeding Kebele Village, neighborhood, smallest administrative unit, Peasant Association KPC Knowledge, Practices and Coverage M & E Monitoring and Evaluation mHealth Mobile Health Technology MNC Maternal and Newborn Care MNCH Maternal Newborn and Child Health MTE Mid-term Evaluation MTMCG Mother-to-Mother Care Groups CS-23 Ethiopia, Final Evaluation Report, Save the Children, December 2012 4 NGO Non-Governmental Organization NNMR Neonatal Mortality Rate OPV Oral Polio Vaccine ORS Oral Rehydration Salt ORT Oral Rehydration Therapy OTP Outpatient Therapeutic Program PCM Pneumonia Case Management PMF Pregnant Mothers Forum PNC Post-natal Care PSI Population Services International PVO Private Voluntary Organization RDT Rapid Diagnostic Test RHB Regional Health Bureau R-HFA Rapid Health Facility Assessment SNL Saving Newborn Lives SNNPR Southern Nations Nationalities People’s Region SO Strategic Objective TBAs Traditional Birth Attendants TOT Training of Trainers TT Tetanus Toxoid UNICEF United Nations Children’s Fund USAID United State Agency for International Development vCHW Volunteer Community Health Worker(s) WHO World Health Organization Woreda District WRA Women of Reproductive Age ZHD Zonal Health Department CS-23 Ethiopia, Final Evaluation Report, Save the Children, December 2012 5 A. PRELIMINARY INFORMATION (Executive Summary) Background: Save the Children was awarded a five-year Standard USAID/CSHGP Child Survival Project (CS-23) - Innovation for Scale: Enhancing Ethiopia's Health Extension Package in the Southern Nations and Nationalities People's Region (SNNPR) - to address four main causes of child death: (1) pneumonia, (2) malaria, (3) diarrheal diseases (that together account for 68% of under-five mortality); and (4) neonatal infection, responsible for half of all neonatal mortality. The project was implemented in the SNNPR in the districts of Shebedino (Sidama Zone) and Lanfero (Silti Zone) and reaches 69, 491 children 0-59 months of age; and 87,496 women of reproductive age (WRA). The overall goal of the project was to enhance the government iCCM strategy in order to contribute to reduced childhood mortality, with a strategic objective to increase use of key childhood services and behaviors. Four intermediate results (IRs) IR-1: Access and availability of child health services and supplies increased; IR-2: Quality of child health services increased; IR-3: Knowledge and acceptance of key child health services and behaviors increased; IR-4: Child health social and policy environment enabled. Principal project strategies (1) Capacity-building, training, and supervision for improved systems and provider performance; (2) health extension package (HEP)/community-integrated management of neonatal and childhood illness (c-IMNCI) for behavior change (BC) delivered at health post (HP) and household (HH) levels by health extension workers (HEWs) and volunteer community health workers (vCHWs) (now the Health Development Army/HDA); (3) Technical communication and advocacy directed at government, professional associations, civil society, and the Federal Ministry of Health (FMOH) for policy change. The project focused on the implementation of the three pillars of the Integrated Management of Neonatal and Childhood Illness (IMNCI) strategy in health centers (HCs), including: clinical IMNCI, health systems support, and community and family practices. Main conclusions of the evaluation In general, the project was successful in implementing all pillars of IMNCI, including; clinical IMNCI training of HC staff, and HEWs in HPs; provision of supervision and supplies for IMNCI; and training and support to vCHWs and others to improve family practices through c-IMNCI. The project also successfully operationalized zinc and pneumonia treatment for childhood diarrhea and pneumonia respectively at HCs and HPs. The treatment of diarrhea with zinc, and pneumonia with antibiotics, was the first at scale in Ethiopia, resulting in the project contributing to operationalizing the national policy. These achievements were accomplished as a result of a comprehensive strategy with strong partnership and collaboration with local health authorities in project implementation. However, main gaps existed in care for newborn babies, both in terms of care practices at home and care seeking, and care for sick newborn babies; each of which were low. Main recommendations of the evaluation Based on the findings, the following were the main conclusions endorsed by the regional and district partners: 1) Engage traditional and spiritual healers as they are still trusted by the community in case of certain illnesses; CS-23 Ethiopia, Final Evaluation Report, Save the Children, December 2012 6 2) Strengthen the capacity of the district to sustain regular integrated and clinical supervision for HCs, HPs and community volunteers; 3) Revise and distribute behavior change communication (BCC) materials to include key IMNCI messages; 4) Strengthen maternal and newborn care (MNC) with special focus to skilled and clean births, and care for both the well and sick newborn baby; 5) Strengthen HPs to be able to provide 24 hour services by ensuring that each HP has the recommended two HEWs and other key requirements such as solar/electricity and water; 6) In the long term, strengthen the pharmaceutical supply chain to ensure that HPs are adequately stocked with key commodities for IMNCI; 7) Expand iCCM to include treatment of the sick, young infant at the HP level. 8) Save the Children should continue fostering a strong partnership at the local, regional and national levels to ensure that lessons learnt are scaled-up all over the country. Summary of Major Project Accomplishments Table 1: Summary of Major Project Accomplishments Strategic Objective: Improved use of key child health services and behaviors Project Inputs Activities Outputs Outcome IR-1: Availability and access to child health services and supplies increased IMNCI training packages Training of HEWs and HC staff in IMNCI; 84% (103/121) of HEWs trained in IMNCI (1 HP per 1000 U5s); 90% (9/10) HCs with IMNCI trained staff; 14,700 U5s treated with antimalarials annually (291 malaria/fever cases treated per 1000 U5s) ; 10,346 U5s treated with antibiotics annually (205 pneumonia cases treated per 1000 U5s); 7,017 U5s treated with ORS annually (1,927 with ORS+zinc) annually (139 diarrhea cases treated per 1000 U5s) IMNCI and other supplies (referral slips, timers, chartbooks, registers, furniture, medical equipment) Provision of IMNCI supplies and drugs to HCs and HPs (initial & through supervision visits) >80% of HPs with IMNCI supplies (except timer); IMNCI drugs (ORS, zinc, ACTs, ABs, CQ) & logistics support (transport, petrol) 91% of HPs with zinc, 100% with ORS, 100% with chloroquine, 18% with ACTs on day of assess. visit IR-2 Quality of child health services improved Transport, supervision tools/checklists, joint planning; Joint supervision Job aids to Supervision visits, provision of job aids 100% of HEWs/HPs report supervision in previous month; 97% of HPs meet FMOH “functional” criteria In case scenario of pneumonia at 11 HPs: 100% of HEWs would classify & refer/treat correctly; 82% classify correctly & 9% reported CS-23 Ethiopia, Final Evaluation Report, Save the Children, December 2012 7 improve adherence to protocols full assessment; 93% of U5 cases at HPs with complete & consistent classification & treatment recorded IR-3: Knowledge and acceptance of key child health services and behaviors improved c-IMNCI training packages and IEC materials Training of vCHWs and HEWs in c-IMNCI 72% (1080/1500) of target vCHWs trained; 68% (82/121) of target HEWs trained Caretakers report knowledge of key family practices & illness danger signs* IR-4: Child health social and policy environment enabled Key partnerships with health authorities at regional, zonal, district and local levels Meetings, technical working groups, joint planning, trainings and supervisions, etc Substantial engagement and buy-in for CS and IMNCI activities at all levels Technical updates, policy briefs, publications, presentations (evidence & feasibility of CCM/P) Policy dialogue & advocacy for CCM/P; Participate in orientation, development & training for CCM/P at national level (‘10) Pneumonia management at the community level now allowed per FMOH policy, Save the Children and other NGOs to implement in ~600 districts in 2010 *Focus group reports not representative and a convenience sample of caretakers chosen by HEWs and HDA leaders B. OVERVIEW OF THE PROJECT, OR STRUCTURE AND IMPLEMENTATION 1. Goals and objectives Save the Children Ethiopia was awarded a five-year Standard USAID/CSHGP Child Survival Project (CS-23) - Innovation for Scale: Enhancing Ethiopia's Health Extension Package in the Southern Nations and Nationalities People's Region (SNNPR). The project was designed to address four main causes of child death: (1) pneumonia, (2) malaria, (3) diarrheal diseases (that together account for 68% of under-five mortality); and (4) neonatal infection, responsible for half of all neonatal mortality. The overall goal of the project was to reduce childhood mortality, with a strategic objective to increase use of key childhood services and behaviors. The project focused on the implementation of the three pillars of the Integrated Management of Neonatal and Childhood Illness (IMNCI) strategy in HCs and HPs including: 1) clinical IMNCI; 2) health systems support; and 3) community and family practices. The strategic objective was “Use of key child health services and behaviors improved.” Based on the project’s Results Framework (Figure 1) the project had four Intermediate Results (IRs):  IR-1: Access and availability of child health services and supplies increased;  IR-2: Quality of child health services increased; CS-23 Ethiopia, Final Evaluation Report, Save the Children, December 2012 8  IR-3: Knowledge and acceptance of key child health services and behaviors increased; and  IR-4: Child health social and policy environment enabled. 2. Project location and estimated project area population This project was implemented in the SNNPR in the districts (woredas) of Shebedino (Sidama Zone) and Lanfero (Silti Zone). The project reached a total population of 366,898 in Shebedino (255,209) and Lanfero (111,689) districts, including 16,645 infants 0-11 months of age; 13,948 children 12-23 months of age; 40,815 children 24-59 months of age; 69,491 children 0-59 months of age; and 87,496 WRA. When project implementation commenced, SNNPR reported Ethiopia’s second highest IMR and U5MR, at 107/1,000 and 157/1,000, respectively. Its neonatal mortality rate (NNMR) – contributed nearly half of the infant mortality of 49/1,000, which was considerably higher than the national NNMR of 39/1,000. TT2+ coverage for pregnant women was 61.4%; while for non-pregnant women it was 36.9%. 3. Technical and cross-cutting interventions  During this project, Save the Children enhanced an existing government package already operating at national scale. The main implementation strategy - CCM in the context of the existing health extension Program (HEP) and IMNCI/c-IMNCI - is designed to enhance the package of evidence-based assessment, classification, and curative interventions for common, serious, childhood infections (pneumonia, diarrhea, malaria, and neonatal infection) delivered at the HP (community) level. This strategy was supported by EPI, capacity-building, and BC at the household level. The project reinforced the potential of an existing government vehicle (HEP) to improve access and availability (IR1), quality (IR2), demand at scale (IR3), and to strengthen the social and policy enabling environment (IR4). Specifically, the technical interventions addressed in CS-23 included: Pneumonia case management (PCM) (35%): Initially the project promoted management of pneumonia with antibiotics at HCs, assessment and referral at HPs, promotion of early care-seeking; and advocacy at the regional and national levels CS-23 Ethiopia, Final Evaluation Report, Save the Children, December 2012 9 for inclusion of pneumonia management at HPs within the HEP. Following policy change to allow community treatment of pneumonia with oral antibiotics, HEWs started treating pneumonia in the second half of the project implementation period.  Prevention and Treatment of Malaria (20%): Prevention through appropriate use of insecticide treated nets (ITNs), early care seeking, appropriate case management at HP, including rapid diagnostic tests (RDTs) and Artemisnin Combination Therapies (ACTs);  Newborn Care (25%): Recognition of danger signs, birth preparedness, promotion of use of antenatal care (ANC), delivery and postnatal (PNC) at HPs, HCs and in the community; and  Immunization (5%): Promotion of immunization through HEWs and vCHWs. In order to deliver the above, interventions were integrated across the main technical areas and these included:  Capacity building (of districts, HCs, HPs and communities), training and supervision for improved systems and provider performance;  Strengthening the integrated management of neonatal and childhood illness (IMNCI) and improving the Expanded Program of Immunization (EPI) in the community, at HPs and HCs;  Promotion of HEP for 16 key behaviors at the community and household levels delivered by HEWs and vCHWs (HDA);  Technical communication, policy dialogue and advocacy at the regional and national levels for CS activities, IMNCI, and PCMt at the community level; and  Strengthening monitoring and evaluation (M&E) of the progress toward objectives in conjunction with local health systems, the local community and other key stakeholders. 4. Project Design In this project, Save the Children enhanced an existing government package already operating at national scale and is a priority strategy. The HEP is the government’s pro-poor strategy that ensures increased efficiency, expanded coverage, and equitable access. Through the HEP, the FMOH strives to bring a set of evidence-based promotive, preventive, and limited curative interventions closer to the household level. Under the current Ethiopian government HSDP-III, the HEP is being taken to scale nationally. Delivered by trained community-based government￾salaried HEWs – who assisted with BC interventions by vCHWs - the HEP comprises “16 packages” including HIV/AIDS prevention, water and sanitation, hygiene, immunization, and best practices for maternal, newborn, and child health at the household level. HEWs are the pillars of the program and are responsible for working with householders in the community to create “Model Families” with the 16 packages in place. According to guidelines, 96 hours per month, per HEW are dedicated to “Model Families”. It was expected that each of the two HEWs assigned to a HP would spend approximately two days at the HP and the other three days in the community. However, due to attrition and evolving responsibilities, this arrangement has been changing. Prior to this project in the two districts, the limited set of curative interventions delivered by HEWs included use of oral rehydration solution (new formula ORS) and zinc therapy (as yet not operationalized) for diarrhea; Rapid Diagnostic Testing (RDT) for malaria with Coartem® for falsiparum and chloroquine for vivax; and assessment and referral of pneumonia, dysentery, and CS-23 Ethiopia, Final Evaluation Report, Save the Children, December 2012 10 neonatal infection. Government policy did not yet authorize use of antibiotics at the HP level. At the time of project initiation, HEWs were unable to treat pneumonia, dysentery, or neonatal infection. Working with many national and international partners, Save the Children advocated for policy change so that HEWs were able complete a basic but full package of life-saving community case management (CCM) interventions at the HP level. The result was that HEWs were authorized to be trained to assess and treat pneumonia with oral antibiotics. In Lanfero and Shebedino districts, Save the Children through the CS-23 project and with support from partners enhanced the existing HEP system by implementing and supporting the three pillars of IMNCI. This was accomplished in close coordination with local health authorities. These two districts were among the first in Ethiopia to implement all three pillars. Save the Children provided initial clinical IMNCI training to HEWs working in rural kebeles, including the diagnosis of malaria with RDTs and treatment with either ACTs (falciparum) or chloroquine (vivax); treatment of diarrhea with oral rehydration salts (ORS) and zinc and initially, the assessment of respiratory illness and referral for pneumonia. After policy change, the HEWs were trained and started assessing and treating pneumonia with oral antibiotics. The IMNCI trainings for HC staff and HEWs had a high facilitator-to-participant ratio (1:4 for HC staff, 1:5 for HEWs). Trainings used the Ethiopian-adapted WHO IMCI training package which includes participatory teaching methods and four to six clinical practice sessions. In order to support clinical IMNCI, Save the Children provided BCC and other guidelines in the local language, including IMNCI registers, chartbooks, timers, referral slips and other supplies to all HCs and HPs. HC staff also received clinical IMNCI training, including treatment of pneumonia with antibiotics, and job aids. Save the Children staff, in coordination with the Regional Health Bureau (RHB) and District Health Offices (DHO), provided regular, ongoing support and supervision to health workers providing IMNCI clinical services. The Outpatient Therapeutic Program (OTP), which manages acute severe malnutrition, was not integrated with IMNCI services. In 2010, Save the Children worked with the DHOs to provide on-the-job training to HEWs in the integration of OTP and IMNCI services. In promoting community and family practices, HEWs coordinated with vCHWs in the communities to promote BC in the use of available services and early care seeking, immunization, growth promotion and appropriate feeding practices, hygiene and sanitation, and home management of illness. The HEWs meet with vCHWs on a bi-monthly or monthly basis to coordinate activities. Save the Children provided initial trainers’ training in community-IMNCI to HEWs who then trained 1080 vCHWs. Save the Children also provided ongoing support through community visits and supervision meetings. The vCHWs and HEWs received Information, Education and Communication (IEC) materials and counseling cards from CS-23 to support this work. Additionally, the CS-23 project coordinated with the RHB and DHOs to support preventive practices, such as distribution of ITNs, EPI and sanitation campaigns, etc. The CS-23 project promoted health systems support for IMNCI services. In addition to providing ongoing support for supervision and training, Save the Children also assisted with supplies and drug stocks. These activities included working with the RHB and DHO to ensure adequate drug supplies and the purchase of ORS, ACTs and chloroquine for HPs and HCs when adequate stocks were not available. Save the Children facilitated the introduction of zinc for diarrhea CS-23 Ethiopia, Final Evaluation Report, Save the Children, December 2012 11 management in IMNCI algorithms in coordination with PSI. Lanfero and Shebedino were among the first districts in Ethiopia to pilot the introduction of zinc for diarrhea. In addition to the full implementation of IMNCI, Save the Children conducted advocacy at the international, national and regional levels to promote CS activities, with an emphasis on policy change to include pneumonia management in the community (within HEWs’ responsibilities). Based on the project documents and findings from the final evaluation, it is evident that the technical package was clear, comprehensive, well implemented, and well documented. 5. Partnerships and collaboration The key partners of the project were collectively the local health authorities, including: 1) the SNNRP Regional Health Bureau (RHB), especially the Family Health Department, the Child Health and Nutrition Team, the RHB HEP and Planning and Programming Department; 2) the Sidama and Silti Zone Health Departments; and 3) the Lanfero and Shebedino DHOs. These partners were involved since project start-up, through briefing meetings, the DIP workshop, the baseline Knowledge, Practices and Coverage (KPC) survey, dissemination workshops, district￾based planning and capacity building trainings for health professionals (facilitated by experts from the FMOH, and integrated supportive supervision, as well as participating in the MTE and in the final evaluations. The key implementers of the IMNCI strategy were HEWs and vCHWs, which ensured local partnership and capacity building at the community level. Save the Children also strengthened local partnerships by participating in the Regional Child Survival Task Force, the Technical Advisory Group (TAG) meetings and the EPI working group chaired by the RHB. Save the Children also worked in collaboration with other non-governmental organizations (NGOs) and development partners at the local and national levels. HCHPUNICEF was a principal national and regional partner, especially in Lanfero District. Other partners included JSI/IFHP, L10K, UNICEF, WHO, GOAL, Plan Ethiopia, Population Services International, and the Malaria Consortium. 6. Partnerships and Relationship with USAID in Ethiopia There was a strong collaboration with the USAID-bilateral Integrated Family Health Program (IFHP) in the use of IEC tools, IMNCI training for HEWs and sharing key CS job aids for health facilities. IFHP, managed by John Snow International (JSI), provided training of trainers (TOT) for the Save the Children team to build its capacity in the facilitation of IMNCI training for HEWs. Save the Children collaborated with UNICEF to provide essential medical supplies to HPs and with WHO on the joint effort for policy influence on CCM/P. PSI provided zinc for piloting in Lanfero and Shebedino districts, which were among the first in the country to introduce zinc into IMNCI protocols. Save the Children worked closely with GOAL Ethiopia to share ideas and organize joint trainings (i.e., zinc treatment). This partnership was evidenced in the final evaluation interviews and at the dissemination of the preliminary final evaluation findings where partners and stakeholders were visibly present. The USAID Mission, Ethiopia was engaged in the project since its initial stages through the provision of technical advice and revision of the project document. The Save the Children national health unit head periodically met with USAID-Ethiopia’s child survival (CS) focal person, the Health Population and Nutrition Officer at the Mission, to provide updates on the status of the project and interventions. CS-23 Ethiopia, Final Evaluation Report, Save the Children, December 2012 12 C. EVALUATION ASSESSMENT METHODOLOGY AND LIMITATIONS The final evaluation of the CS-23 project was conducted by a team led by an external evaluator. The team included two senior representatives from Save the Children headquarters; Save the Children national and regional offices; members from the RHB, Zonal Health Department and DDHO staff; CS-23 project staff; and National and Regional representatives from UNICEF. The final evaluation was conducted between September 18 and September 30, 2012. On the first day of the final evaluation, the team met to agree on the focus of the evaluation, to review and agree on tools, and to form and orient field teams. Four principal methods were used for the final evaluation: 1) Document review, including policy documents, program reports, technical reports, reports of evaluations or study findings, training and health education materials. 2) Field visits to Shebedino and Lanfero Districts. The team spent four days in the field making site visits to district headquarters, HCs and HPs, and to conduct in-depth interviews (IDIs) with district staff, HC staff, HEWs and community members (both IDIs and FGDs). 3) Observation of HCs and HPs (HEWs); and 4) In-depth interviews with regional and national stakeholders. Some intended national level interviews that were planned, were not carried out due to a public holiday following the death of the Ethiopian PM (in the case of PSI) or because the responsible officer was out in the field (in the case of USAID staff). (Annex 8 shows a summary of the contacts and respondents in the final evaluation.) Following the field work, both field teams met in a joint meeting to discuss and synthesize the findings. A final summary of main findings and recommendations was reviewed and discussed with CS-23 program staff and the staff of UNICEF, FMOH/RHB and districts, and all the evaluation team members. At the meeting, key recommendations were outlined. These were later presented in a half-day stakeholders’ feedback meeting in Awassa on August 29, 2012. Program data, documents and reports were generally available to the evaluation team, and interviews were conducted with key stakeholders at all levels. In addition to the baseline and endline KPC surveys, other assessments conducted by the grantee included:  Baseline and endline health facility assessments (HFA) (see Annex 12);  Assessment of the potential of mHealth to support HEW Supervision in Ethiopia’s SNNPR (Annex 1);  Exploration of causes for low utilization in Shebedino vs. Lanfero District (Annex 1);  How did USAID’s Child Survival and Health grant’s CS-23 Project to Save the Children Contribute to CCM of Pneumonia Policy Change and iCCM Scale Up in Ethiopia? (Annex 1); and  Mother-to-Mother Care Group, Pregnant Mothers Forum, and Increased Institutional Delivery in Lanfero District, Ethiopia (Annex 1). Taken together, these documents provided rich data that enabled the final evaluation team to understand the project impact and effectiveness. However, there were a few gaps and limitations. Although at the time of the final evaluation the KPC assessment had been completed and analyzed, the narrative report was not ready. During CS-23 Ethiopia, Final Evaluation Report, Save the Children, December 2012 13 presentations by the project team of the KPC results, we noticed some findings that were not clear, especially those related to care seeking for pneumonia-like symptoms. The data suggested that care givers do not make high use of HPs for treatment. We followed up on this issue during the field visits, and discovered that the finding was incorrect and likely due to the fact that community members are not always able to differentiate HPs from HCs when such questions are asked in the local language. In addition, whereas we found that the IMNCI/iCCM recordings at health facilities and at HPs had generally improved, there were still gaps such as in recording of age, weight, etc., in HCs and HPs. D. DATA QUALITY AND USE The final evaluation As already stated, the final evaluation included field visits. Although we achieved a high response rate, we also had some problems that affected the field work. During the period of the evaluation, we learnt of the passing of the Ethiopian Prime Minister. As a result, the field team in Shebedino was only able to spend three of the planned four days in the field. In addition, the survey period took place during the rainy season and many roads, especially in the more rural areas, were not accessible. Thus the final areas for the field visits were purposively selected to exclude inaccessible areas. Population-based baseline and endline surveys The standard CSHGP KPC survey instrument and methodology were adapted for use. The KPC is designed as a before and after study with no control. This has limitations of causality attribution as similar changes could be occurring elsewhere. However, several evaluation techniques were implemented in order to triangulate the findings, and clearly these showed a dose-response in the relationship of findings to interventions. Background characteristics of the sample population at baseline and endline were similar. The sampling frame for the baseline survey comprised the entire population of the two districts. It was a random cluster household survey with 600 respondents (300 per district). Because there was no complete report of the final KPC at the time of the evaluation, it is difficult to judge the methodological challenges and sampling assumptions used. Given the differences in population size, it would have been expected that the sample size per district would have been weighed so that the larger district proportionately contributes more to the total sample than the smaller one. However, this was taken care of at analysis. Certain endline indicators were not collected at baseline. For these indicators it is not possible to determine whether changes in knowledge or practices are associated with project activities. Matching surveys to ensure that they collect the same key data is important for project comparisons. More attention to this issue is recommended in the future. Another challenge encountered in the endline KPC was that the communities tended to refer to HCs and HPs the same way, that is, they could not differentiate HCs from HPs. We later found out that the use of “HP” in the local language is not common. Because of this, the final KPC results erroneously show that HPs are not used especially for pneumonia treatment. CS-23 Ethiopia, Final Evaluation Report, Save the Children, December 2012 14 Health Facility Assessments HFAs on the quality of MNC were conducted at baseline and endline. Surveys measured availability of facility supports including essential medicines, supplies and equipment, availability of services and some aspects of health worker knowledge and clinical practice. Standard facility assessment tools were used and adapted for local use. The surveyors conducted a census of all health facilities in the two districts and a random survey of half the HPs. None of the two districts had a hospital. The final evaluation and process evaluation data Routine monitoring data: The project used routine data to monitor progress, presented in internal quarterly and annual reports, as well as annual reports to USAID. The monitoring plan included reporting on: 1) health communication activities; 2) findings (successes and gaps) observed during integrated supervision; 3) use of curative child health services by illness at HCs and HPs; and 4) advocacy activities. The project found that the data regarding processes and health service utilization collected through the routine Ethiopian Health Management Information System (HMIS) needed strengthening and definitions of collected information often did not correspond with standard IMNCI or maternal, neonatal and child health (MNCH) definitions. In response, the project organized a five-day training with the RHB to ensure coordination of CS-23 activities and supportive supervision with the planned improvements to the FMOH HMIS. During field visits in the final evaluation, we found that at the time health workers were using HMIS for children under-five that is based on IMNCI definitions. This has greatly improved the quality of the data for sick children under-five coming to HCs or HPs. Health Information System (HIS) Data The community-based surveillance system was not fully operational during the project implementation period. Availability of data from the community would greatly enhance decision making and programming. Improving availability, quality and use of community based data should be considered as a key priority if future. Data from the routine HMIS were available and used for following trends in some key indicators, including ANC visits, timing of ANC visits, facility deliveries and PNC visits. Special studies and operational research To address operational issues that have arisen in the course of the CS-23 project, Save the Children conducted formative research to assess the existing HEP supervision plans and actual implementation. The special embedded studies conducted have been outlined above. Briefly they included: Assessment of the potential of mHealth to support Health Extension Worker Supervision; Exploration of causes for low utilization in Shebedino vs. Lanfero Districts; and studies on use of zinc by HEWs to treat diarrhea. Use and dissemination of routine project M&E information Results from the baseline KPC survey were used to set targets, and the HFA was used to better understand the context and target project activities. The project used routine data to document project progress and to identify gaps that could be addressed through project activities. The M&E data from the various sources (KPC, HFA, routine monitoring) was collected in collaboration with FMOH partners and shared through written reports, review meetings and workshops. In addition, preliminary implementation experiences from the project were shared at national and global meetings and were used to pilot the global iCCM indicators (Annex 2). The preliminary findings from the endline evaluation have already been presented to the regional stakeholders and also discussed CS-23 Ethiopia, Final Evaluation Report, Save the Children, December 2012 15 with Save the Children and the head of Child Health in Ethiopia. In addition, the team has already published one paper in a peer reviewed journal: Degefie T, Marsh D, Gebremariam A, Tefera W, Osborn G, Waltensperger K. Community Case Management Improves Use of Treatment for Childhood Diarrhea, Malaria and Pneumonia in a Remote District of Ethiopia. Ethiop. J. Health Dev. 2009; 23(2). E. PRESENTATION OF PROGRESS TOWARD ACHIEVING PROJECT RESULTS Table 2 presents the M&E matrix from the DIP and updated based on the final survey findings. The FE used the results framework to guide data collection and was a participatory process. Table 2: M&E Matrix Goal, Objective, Intermediate Result Indicators Data Source/ Method of Measurement Baseline Value Final Value* Final Target Goal: Under-five morality reduced Strategic Objective: Use of key child health services and practices increased Appropriate hand washing practices % of mothers of children 0-23 months who live in a household with soap or a locally appropriate cleanser at the place for hand washing and who washed their hands with soap at least 2 of the appropriate times during the day or night before the interview KPC 28% 60% 45% Increased feeding during diarrheal episode % of children aged 0-23 months with diarrhea in the last two weeks who were offered the same amount or more food during the illness KPC, 2011 EDHS 29% 25% 43% Increased fluid intake during diarrheal episode % of children 0-23 months with diarrhea in the last two weeks who were offered more fluids during the illness KPC, 2011 EDHS 20% 59% 36% Appropriate care seeking for pneumonia % of children age 0-23 months with chest-related cough and fast and/ or difficult breathing in the last two weeks who were taken to an appropriate health provider KPC, 2011 EDHS 32% 45% 60% ORT use % of children age 0-23 months with diarrhea in the last two weeks who received ORS and/or recommended home fluids. KPC, 2011 EDHS 57% 55% 72% Zinc therapy % of children 0-23 months with diarrhea in the last two weeks who were treated with zinc supplements KPC, 2011 EDHS, DHO/RHB service data 7% 34% 25% ITN use by child % of children age 0-23 months who slept under an insecticide-treated bed net (in malaria risk areas, where bed net use is effective) the previous night KPC, 2011 EDHS 40% 39% 65% Postnatal visit to check on newborn within first 3 days after birth % of children age 0-23 who received a post-natal visit from an appropriate trained health worker within three days after the birth of the youngest child KPC, 2011 EDHS 4% 14% 30% Immediate and exclusive breastfeeding of newborns) % of newborns who were put to the breast within one hour of delivery and did not receive prelacteal feeds KPC, 2011 EDHS 62% 93% 69% Exclusive breastfeeding (0-5 months) % of children age 0-5 months who were exclusively breastfed during the last 24 hours KPC, 2011 EDHS 3% 29% 25% IR-1: Access and availability of child health services and supplies increased Access to immunization % of children age 12-23 months who received a DPT1 vaccination before they reached 12 months KPC, 2011 EDHS 80% 97% 80% CS-23 Ethiopia, Final Evaluation Report, Save the Children, December 2012 16 Goal, Objective, Intermediate Result Indicators Data Source/ Method of Measurement Baseline Value Final Value* Final Target Clinical IMNCI coverage % of HEWs/VCHWs trained in IMNCI RHB/ZHD/DHO documentation, project training records 0% 100% 60% Community IMNCI coverage % of HEWs/VCHWs trained in c-IMNCI RHB/ZHD/DHO documentation, project training records 0% 100% 60% Availability of zinc % of HPs that report no stock-out of zinc in previous month RHB/ZHD/DHO documentation, rapid inventories of HPs, stock-out reports 0% 100% 75% IR-2: Quality of child health services improved Health system performance regarding immunization % of children age12-23 months who received a DPT3 vaccination before they reached 12 months KPC 47% 71% 75% Measles vaccination % of children age 12-23 months who received a measles vaccination regardless of age KPC, 2011 EDHS 60% 84% 75% Child with fever receives appropriate anti￾malarial % of children age 0-23 months with a febrile episode during the last two weeks who were treated with an effective anti-malarial drug within 24 hours after the fever began KPC, R-HFA, health facility record review 17% 47% 60% Use of medicine during diarrhea % of children 0-23 months with diarrhea in the last two weeks who were no treated with anti-diarrheals or antibiotics KPC, health facility record review 41% 4% 22% HEW performance % of trained HEWs who followed correct IMNCI steps to assess, classify, treat, refer childhood illness R-HFA, performance observations, supervisory records TBD 81% 60% Functional supervisory system % of HPs that have received supportive supervision 1x/mo in past quarter (according to FMOH criteria) R-HFA, monthly woreda reports 80% 100% Target not set Functional health system % of HPs meeting FMOH “functional” criteria (refer to Annex 18) RHB/ZHD/ DHO records, reports 80% 100% Target not set Functional health system % of HPs that have met all reporting requirements in past quarter (according to FMOH criteria) RHB/ZHD/DHO reports 80% 100% Target not set IR-3: Knowledge and Acceptance of key child health services and practices improved Neonatal danger signs % of mothers report knowledge of at least 2 neonatal danger signs needing treatment KPC 29% 28% Target not set Child danger signs % of mothers who know at least 2 signs of illness in children needing treatment KPC, 2011 EDHS 51% 74% 75% IR-4: Policy and social environment enabled Policy change HSDP-IV includes CCM as HEP strategy at level of HP - including antibiotics for treatment pneumonia, dysentery, neonatal sepsis FMOH/RHB policy documents and operational guidance 100% Target not set Joint planning for sustainability Joint planning takes place on annual basis with RHB/ZHD/DHO, Save the Children, ESHE, and relevant key community stakeholders RHB/ZHD/DHO records, project documentation N/A Yes Yes *Final value = Endline Household Survey Result. CS-23 Ethiopia, Final Evaluation Report, Save the Children, December 2012 17 F. DISCUSSION OF PROGRESS TOWARDS ACHIEVEMENT OF RESULTS 1. Contribution Toward Project and OR Objectives We use the results framework to appraise the findings. Overall, the final evaluation findings indicate that this project has been a great success. Although no data was collected on mortality, the intermediate results show marked increases in key mortality proxies such as coverage of high impact interventions, care seeking, quality of care, and decreased morbidity. The section that follows discusses the progress towards achieving the intermediate results (IR) and strategic objective (SO) as outlined in the results framework. We discuss the remaining challenges and make recommendations to inform further scale-up or to sustain the project and make it even more effective. Contribution Toward Objectives- IR-1: Access and availability of child health services and supplies increased a. Access to IMNCI services for sick children at HPs and HCs, and c-IMNCI through vCHWs Overall access and referral: The project has achieved its targets for improving access to IMNCI services for sick children at HCs and HPs. All HCs in the two districts offer full IMNCI services and 100% (11 out of 11) have at least one IMNCI nurse on-staff, although turnover of IMNCI￾trained HC staff has been a challenge especially in Lanfero. Approximately 90% (109/121) of the targeted HEWs are trained in IMNCI (management of malaria and diarrhea, and now pneumonia, assess –classify and refer newborn infection and ENC), with approximately one IMNCI functional HP for every 1000 children under five years of age. There has been an increase in the number of children seeking care at either HCs or HPs (see Annex 6, Final KPC Report, Table 1 for more details). National policy now permits the management of pneumonia in the community, and implementation started in the fourth quarter of 2010. Key stakeholders, local health authorities, project staff and HEWs considered the IMNCI training for HC staff and HEWs, as well as the ongoing support to trained health workers and HEWs to be one of the most significant achievements of the project to date. Likewise, community members in focus groups also expressed appreciation for the availability of services through HPs. A referral system is in place at each level in the community; vCHWs promote the use of HEWs and HPs and refer children to HPs using improvised referral slips provided by CS-23. At the next level, HEWs refer sick newborn babies and severely ill older children to HCs using referral slips provided by Save the Children. Although in the MTE it was found that HEWs also refer children from HPs to HCs because of drug stock-outs, especially CoArtem®, resulting in unnecessary referral, this practice has been reversed as stock-outs are now rare. Recently, the two districts acquired ambulances from the FMOH to refer women who cannot deliver at a HP or a HC. The HPs are now networked and if they got such a case, they just need to make a call and an ambulance will appear. The addition of the ambulance has further motivated communities to seek facility delivery. However, the free ambulance services are currently for women in labor; people with other medical emergencies can use the ambulance at a cost. Thus, referral from HCs to hospitals for severely ill children is a large challenge due to costs and transportation. Back referral from HCs to HPs happens infrequently and lack of feedback was reported to demoralize CS-23 Ethiopia, Final Evaluation Report, Save the Children, December 2012 18 some HEWs. Save the Children, through another resource, will provide additional ambulances to the districts which may address the challenge of sick child referral. Challenges and recommendations: A significant number of HEWs have left or transferred to other posts, and many HPs currently have only one HEW. This one HEW has an increasing workload, especially the curative aspect. Although they are currently very enthusiastic and well￾motivated, it is likely that they will soon face burn-out. This situation will be worse especially once the CS-23 project ends, as supported supervision and mentorship are likely to be reduced. Interviews with the RHB head revealed that the government is aware of these problems and strategies are being put in place to improve the staffing of HPs by addressing attrition. Another challenge affecting access to IMNCI services for sick children at HPs was the availability of drugs and supplies. Although this is currently not a big problem in the two districts, district stakeholders, health workers and HEWs were concerned that there was no clear mechanism for continued high quality services at the HPs. This is because HPs lacked a budget and some of the drugs (zinc, antibiotics) were not part of a regular kit. A package of iCCM drugs will be provided to each HP by FMOHFMOH/UNICEF and this may solve the problem. Overcoming challenges linked to referral for sick children is difficult; one ambulance for the whole district is not enough to do all the delivery referrals and also add children. Even if this was possible, lack of resources within communities and in the health system is a large constraint. A more workable solution will be to empower HCs to be able to deal with most of the sick children who would have needed referral. This will involve staff training and equipping of facilities so that HCs have the skilled staff, drugs and equipment to manage very sick children. b. Availability of IMNCI supplies and drugs at HPs and HCs HPs in both districts were well equipped with most IMNCI equipment and supplies on the day of the assessment during the final evaluation, including IMNCI chartbooks and registers, referral slips, thermometers, MUAC strips, scales, counseling cards and RDTs. These findings corroborated the final evaluation HFA findings which reported that HPs had all essential drugs, supplies and records. HCs. The percentage of the population with year-round geographical access (within 5Km or one hour access) increased from 57% at baseline to 100% at endline; and most of these HCs had improved availability of all three child health services (increase from 38% at baseline to 91% at endline). These HCs witnessed marked improvements in infrastructure, with an increase from 38% at baseline to 100% at endline in terms of having all essential infrastructure present and functional. Staff availability also seems to have increased slightly; 27% of HCs did not have all the clinical staff on the day of the endline HFA. In terms of medical drugs and supplies, there was marked improvement at endline compared to baseline for availability of essential supplies (increase from 42% to 100%); and all essential child drugs (14% to 100%). However, performance on maternal and especially neonatal commodities was poor; only 37% of HC had all basic neonatal and delivery supplies and 73% had all required basic maternal and neonatal drugs. HPs. In addition, all HPs had excellent availability of RDTs. This was confirmed during field visits as we found a good supply and no stock-outs. Zinc was introduced and supplied in coordination with PSI, with PSI providing zinc in-kind for pilot testing in the CS-23 districts. Using matching funds, CS-23 supplied ORS, zinc, chloroquine, CoArtem® and amoxicillin CS-23 Ethiopia, Final Evaluation Report, Save the Children, December 2012 19 syrups and capsules, as adequate drug supply through government health systems is an ongoing challenge in Lanfero and Shebedino Districts. However, private funds for drug supply will soon be depleted. The FMOH/UNICEF iCCM drugs package will address future needs. Additionally, the Save the Children project often provided supervision and logistical support, such as transportation resources, to ensure adequate drug supplies in peripheral HPs and HCs. In interviews, we learned from the deputy head of the RHB that the government is already working on a system to sustain the medical supplies and drugs through provision of HP kits. He said “HPs are a government policy so sustainability is not an issue. The government, with the help of its partners, is working on strengthening the pharmaceutical supply chain, and are working on supplies using a pool system that is based on need”. The distribution of these kits has been tasked to the supervising HCs. However, we learnt from the field interviews that unless more is done, there will be stock-outs mainly due to: 1) Lack of adequate funds for the districts to routinely distribute the kits; 2) Poor supply chain information systems; and 3) General budget limitations in the national health sector. Although not formally assessed, most HPs and HCs appeared to have a functioning ORT corner. HPs generally had very well maintained buildings, furniture and clean water. This finding differed from that at the MTE visit, where the infrastructure was reported to be poor. After the MTE, Save the Children received support from Save the Children Korea which provided funds for renovations and for furniture, thus improving the situation. Contributions of the CS-23 project: Overall, almost all HPs and HCs have adequate supplies to provide IMNCI services, as well as ORT corners, and this can be attributed to the CS-23 project support and supervision activities. Almost all interviewees at the RHB, district, HCs and HPs attributed this success mainly to the project’s efforts to strengthen the existing health system. It was reported that through CS-23, health workers, HEWs and vCHWs were trained and supervised routinely and also during more specific technical supervision. Save the Children also provided transport support to districts to distribute the commodities. However, this active involvement and support by Save the Children has consequences that might affect the project’s sustainability. During interviews, it did seem like Save the Children might have created some kind of dependency syndrome, as districts appeared unlikely to be able to adequately maintain support supervision and mentorship. Key stakeholders and almost every service provider noted the provision and follow-up on supplies as a large contribution of the CS-23 project. One of the District Health Officers said, “When the project ends, we will have problems with support supervision, review meetings and distribution of medical supplies”. Most of the key informants from the regional and zonal levels to vCHWs and caretakers in the community, reported maintaining adequate drug supplies as one of the largest challenges to child health activities. Although this is currently not a big problem in the two districts, district stakeholders, health workers and HEWs were concerned that there was no clear mechanism for continued high quality services at the HPs. This is because HPs lacked a budget and some of the drugs (zinc, antibiotics) were not part of a regular kit. Challenges and recommendations: Relatively weak stock management systems within government structures and a lack of drug supplies at all levels of the health system are ongoing challenges faced by the CS-23 project. These threatened the sustainability of progress during the project and especially upon project completion. Additionally, the shortage of transportation and petrol for activities within the government health system negatively impacts logistics and the CS-23 Ethiopia, Final Evaluation Report, Save the Children, December 2012 20 provision of supplies to HPs and HCs. The sustainability of the zinc supply is also a large challenge. Thus, currently all zinc must be procured through private channels. The iCCM kit will include zinc in the package but the challenge is that this kit is not yet reaching all HPs. However, Shebedino District has received and distributed the iCCM kit to all HPs recently. c. Access to (and use of) maternal and neonatal services at HPs and HCs Overall: Access to maternal and neonatal services within Lanfero and Shebedino Districts, although more limited than for IMNCI services, did register some improvements. HPs have started providing delivery care, a service which was not available at that level before. Women report using ANC services at the HP and HC levels, and the promotion of these services is also reported at all levels. The promotion of assisted delivery at HCs was reported to occur at all levels—HCs, HPs (by HEWs) and in the community (by vCHWs). The endline KPC showed that the institutional delivery rate increased from 2% at baseline to 25% (15% skilled birth attendance), which is far higher than the average for Ethiopia (10%). However, postnatal/partum visit (25% mother, 17% newborn) is low. Knowledge of two or more danger signs is also low (13% maternal, 27% newborn). Surprisingly, apart from immediate breastfeeding and giving colostrum to the newborn, the final KPC shows that essential newborn care (ENC) practices slightly worsened (or at least did not improve). In addition, during the field visits conducted as part of the final evaluation, it was noted that there is a generally low use of both HC and HPs for the care of the sick newborn. Several reasons were sited including cultural barriers that discourage a newborn from “crossing an imaginary border”, the fear of “the evil eye”, newborns are delicate to take out, communities not being aware of services for the newborn at HPs, and communities being unsatisfied with the care for newborn babies at the HP as the care is limited to only assessment and referral (and no treatment). The consequences are that some newborns developed danger signs and died at home without care seeking, or that care seeking was delayed. Other babies bled to death due to poor cord cutting and tying; or care givers would apply dangerous substances such as cow dung to the cord. The use of vCHWs working under the supervision of the HEWs was been one of the key cornerstones of the maternal and newborn BC. The community health workers (vCHWs) were trained by Save the Children and given IEC materials on MNCH. Empowered by these, the vCHWs and HEWs made home visits or organized meetings to mobilize communities on MNCH care. It was noted in the MTE that often messages about assisted delivery and newborn care almost exclusively targeted young women. Yet it is fathers and older women (grandmothers) that often make key decisions about delivery and newborn care; however, they are not specifically targeted for key messages. As a result, the project came up with innovations to address these demand side issues. Other channels were developed including mother-to-mother groups, pregnant women groups, and engagement of community elders in MNCH. In Lanfero district, following a pilot, mother-to-mother groups have been scaled-up as an initiative of the district. One of the strategies which led to recent surges in institutional birth especially at HPs is the use of the Pregnant Mothers Forum (PMF). The MTE (August 2010) recommended strengthening maternal and newborn health (MNH) promotion by pilot-testing Mother-to-Mother Care Groups (MTMCG) in three kebeles in Lanfero District. The district CS-23 Ethiopia, Final Evaluation Report, Save the Children, December 2012 21 health team observed that, after the MTMCG, a pilot kebele experienced increased deliveries at its HP. Thus, the team revised and scaled up the approach as a PMF in 25 of 27 kebeles. A PMF is a group of five to 15 pregnant mothers who support each other and are facilitated by the vCHW and the HEW. Each group has a team leader, also a pregnant mother and an active community member. Each PMF meeting has a coffee ceremony and a “porridge ceremony,” a cultural ceremony of eating porridge with close friends and relatives when a mother gives birth. The DHO showed innovative leadership in using a special group of mothers, in this case pregnant mothers, to address their own issues in a network with local resources. It was reported that most mothers now give birth at HPs or are planning to do so. (Annex 1, Learning Brief 7) The changes in maternal and delivery care practices were also reinforced by a couple of health systems strengthening interventions. These included renovation and equipping of HPs, training of health workers and HEWs in MNC care and in clean delivery practices, provision of drugs and supplies, and support supervision and mentorship. These were implemented by the district but facilitated and funded by the CS-23 project. A combination of these community and health facility (demand and supply) interventions have led to drastic changes in long-held cultural practices that were documented in the MTE such as: families citing home delivery as a more culturally appropriate practice, and mothers reporting detesting the delivery tables; and families not being able to afford razors, gloves, towels and drugs, which were a barrier to delivery at HCs. Despite these achievements, care seeking and postnatal care for sick and well newborn babies is still poor. Once the newborns reach the health facility, the only care is assessment, counseling and advice on referral to HCs or to the hospital. HEWs have not been trained in treating sick newborn babies nor does national policy allow them to do so. As a result, most mothers with sick newborns do not see a reason to seek care. This is different for mothers with older infants and children who are treated at HPs when they are sick. Challenges and recommendations: The neonatal technical component of the CS-23 has made the least progress; assisted delivery and newborn care seeking are unfortunately low due to a myriad of factors (see utilization section below), including cultural and health systems barriers. Management of sick newborns, ENC and post-natal care messages are included for one day in the IMNCI training packages, although treatment of the sick newborn is not allowed at the HP level. However there seems to be hope that this will change, although it will take some time. It is anticipated that the conclusion of SNL’s sepsis management research will provide further evidence to reinforce advocacy efforts so that HEWs will be allowed to treat sick newborn babies with antibiotics. Another strategy that the government is embarking on is the upgrading of HEWs to Level III which would allow them to treat sick newborns. It is reported that so far about 5000 have been trained, but the process is slow due to a lack of available resources. Another potential challenge of this process is that unless well managed, it will contribute to further ‘attrition’ of HEWs as they will be absent from the already over-burdened HPs for a long time. In the meantime, care for newborn babies should be improved through strengthening home visits and referral by CS-23 Ethiopia, Final Evaluation Report, Save the Children, December 2012 22 vCHWs to HPs. The government should also explore diversifying the use of the ambulance to transport referred sick newborns. d. Contribution toward objectives IR-2: Quality of child health services increased This section details different attributes that either influenced quality or demonstrated the quality of services provided in the project. Within each section, we discuss the challenges and recommendations to improve health systems support and quality of services. Supervision to ensure quality of services at HPs and HCs Supervision: The final KPC showed that supervision was very effective. Nearly all (100%) health workers and HEWs reported to have been supervised at least once in the three months prior to the survey. The CS-23 team and FMOH partners identified supervision as a large challenge within the implementation of IMNCI. Therefore, a detailed assessment of the supervision system for the IMNCI services was done and detailed in the MTE. The supervision of HEWs and HPs within the HEP system included joint supervision from the DHO, from the HCs and weekly supervision by HEP supervisors. HEP supervisors were supervised monthly by the DHO HEP Coordinator. Save the Children supports and joins many of these supervision visits, and also conduct supervision visits independent of those conducted by the districts. It is estimated that Save the Children provided over 50% of supervision to HPs. The joint supervision with local health authorities was reported to be a large contribution of the CS-23 project, contributing to high supervision completion rates at HC and HP levels. HCs are also implementing the government policy of being responsible for supervising five HPs. In one interview, we were told that each kebele/HP is allocated a health worker to supervise and to be responsible for its performance. This serves as a great link between HCs and the HPs. During interviews, most respondents identified supervision as one of the major contributors to the observed performance of the project. Because districts faced constraints in terms of capacity to supervise (lack of functional vehicles/motorcycles and funds for fuel and maintenance), Save the Children assisted with the provision of most of these services or resources. Other challenges included high workloads of government staff that presented challenges to completing scheduled joint supervisions. During the FE field visits, HEWs reported that they were concerned that the ending of the project might mean less supervision, and yet this supervision was a basis for their motivation and improving skills. In both the MTE and the FE, most HEWs from both districts reported that the supervisor checked records, corrected errors, and gave training. Challenges and recommendations: Supervision was a great success especially towards the end of the project. However, because of the challenges outlined above, it is doubtful whether the frequency and quality of supervision can be maintained or scaled-up in routine district health systems. There are also concerns about whether the HEP supervisors can be motivated enough to continue the work. As scale-up of the project takes place, strengthening integrated support supervision in the districts will be key. CS-23 Ethiopia, Final Evaluation Report, Save the Children, December 2012 23 Quality of services at HPs and HCs The performance of health workers and HEWs was assessed at baseline, mid-term and at endline. It was found that HW performance on assessment was quite poor in both districts. However, they were able to give the right treatment based on the diagnosis or classification made. The main gaps – less commonly observed than at HPs – were checking for general danger signs, chest indrawing or duration of cough. On the other hand, HEWs did well when evaluated on both criteria (assessment and treatment). Thus, it is clear HEWs were able to follow guidelines. During the FE field visits, we found reasonably good quality of registers at HCs, however they lacked completeness especially related to the recording of age and weight. Overall, HP registers were generally complete. Challenges and recommendations: Health workers are the referral point and supervisors for HEWs. However, their performance is below standard in terms of following the IMNCI guidelines. It is recommended that once trained, supervision should be comprehensive and should emphasize the use of guidelines. e. Contribution toward objectives IR-3: Knowledge and acceptance of key child health services and behaviors increased Overall: The project activities to promote the knowledge of key services and practices were a success for some services but disappointing for others. Communities were aware of the services provided at HCs and HPs, and the work that HEWs and vCHWs were doing in the community. This awareness was attributed mainly by the community mobilization performed by the trained HEWs and the vCHWs. The CS-23 project trained 109 HEWs and 1080 vCHWs in c-IMNCI, and provided job aids and counseling cards to support their community mobilization. To further support the vCHWs and HEWs, the CS-23 project recruited two Save the Children project field staff per district to support the community work and to allow for more intensive follow-up of c￾IMNCI activities in communities. Key quantitative indicators of increased knowledge by community members are those of danger sign awareness. According to the KPC findings, the percentage of mothers who knew at least two signs of illness in older children increased from 51% at baseline to 74% at endline; but there was no change in awareness of neonatal danger signs (29% at baseline to 28% at endline). These findings clearly show that more work in improving neonatal care awareness is needed. Challenges and recommendations: Gaps in awareness still remain especially with regard to newborn care. Future programs at scale should consider a special focus on implementation and monitoring of the newborn component. Possible strategies include additional training of HEWs and vCHWs, having more targeted supervision, and strengthening or expanding the use of pregnant women and mother-to-mother groups. f. Contribution toward objectives IR-4: Child health social and policy environment enabled Save the Children has engaged in policy dialogue and advocacy at the international, national and regional/local levels in order to foster a positive policy environment. At the international level, Save the Children advocates for CS programming and best practices. Health workers from Shebedino and the CS-23 project team were featured in a US-based CS-23 Ethiopia, Final Evaluation Report, Save the Children, December 2012 24 campaign for child and neonatal survival, sponsored by the Ad Council. This campaign aims to garner support and funding for MNCH services. An Italian donor group visited the project in 2010, and based on this successful visit supported Ethiopia’s EveryOne Campaign and implementation of a three-year MNCH project in three remote districts of Southern Ethiopia. Save the Children Korea also provided support to bridge gaps in the CS-23 project. This included renovation and equipping of HPs with delivery care equipment and procurement of furniture, motorbikes, ambulances and essential drugs and medical supplies. The CS-23 project also contributed to the development of the international iCCM benchmarks and indicators of implementation strength. At the national level, Save the Children is a member of the National Child Survival and CCM Task Forces, and has presented experiences from CS-23 and other related projects. Much of this advocacy has focused on fostering policy change to permit the management of pneumonia with antibiotics at the community (HEW) level. In late 2009, the government of Ethiopia changed the HEP policy to allow pneumonia management with antibiotics in the community. This achievement in the policy environment was likely influenced, in combination with political and contextual factors, by a myriad of advocacy activities by many development partners, including Save the Children’s. This re-emphasizes the well-known fact that policy change takes time. Local evidence (including experience from CS-17 published during CS-23), demonstrates that strong and strategic partnerships, site visits, and persistent multi-channel advocacy collectively has a “big voice to reach big ears.” IMNCI at the HP level (CCM), including treatment of pneumonia, was introduced at-scale in Ethiopia through support from UNICEF. Save the Children was awarded a grant from UNICEF-Ethiopia to implement iCCM with pneumonia treatment in 100 districts in the Oromia [64] and SNNP [36] regions, including Shebedino District. At the regional and local levels, Save the Children has a strong partnership with the regional and local health authorities and is a member of the Regional Child Survival Task Force. It also facilitated the formation of district-level Child Survival Task Forces. The CS-23 Program Manager and the Save the Children Health Unit Head have played a crucial role in the revision and development of national strategic documents and guidelines plus training materials in nutrition, IMNCI and iCCM. g. Contribution toward objectives Strategic Objective: Use of services Table 2 (page 15 of this report), which is the M&E matrix, summarizes the overall utilization of services. In addition, according to HMIS records, by the end of 2011, 13,035 children under five had been treated with antimalarials (or 211 malaria/fever cases per 1000 under-fives); 8,200 were treated with antibiotics (133 pneumonia cases treated per 1000 U5s); 9,137 were treated with ORS of which 3,712 were treated with ORS + zinc (148 diarrheal cases 1000 under five) at either HCs or at HPs. Based on the baseline and final KPCs, the following services were doubled during the project period or exceeded the set targets: 1) appropriate hand washing practices (28% to 60%); 2) increased fluid intake during diarrheal disease episode (20 to 59); 3) zinc therapy (7% to 34%); 4) immediate and exclusive breastfeeding of the newborn (62% to 93%); 5) exclusive CS-23 Ethiopia, Final Evaluation Report, Save the Children, December 2012 25 breastfeeding of infants 0-5 months old (3% to 29%); and 6) access to immunization (80% to 97%). However, the following services use was not markedly increased or performance was below the set target: 1) increased feeding during diarrheal disease episode (29% to 25%); 2) appropriate care seeking for pneumonia treatment (32 to 45%); 3) ITN use by child (40% to 39%); and 4) postnatal visit in the first three days (4% to 14%, target 30%). Thus, whereas use of some services increased (diarrhea management, breastfeeding, immunization services, others were quite low (pneumonia treatment, malaria prevention by using ITNs and newborn care services). An assessment of how HEW spend their time found that although they spend most of their time (Annex 1, Learning Brief 1) at the HP, they spend little time on CCM (2.7% of HP and 1.6% of total time). Interpretation and recommendations: CS-23 staff investigated reasons for low utilization in Shebedino (Annex 1, Learning Brief 5) and found the following possible explanations: under￾reporting (malnourished children registered in Outpatient Treatment Program but not in IMNCI Register, HEWs treating sick children during household visits without registers, high patient loads, and forgetfulness); Geographic and financial access barriers were common for services at HCs, but not at HPs – sometimes aggravated by seasonal harvest responsibilities and flooding; The technical quality of case management at all health facilities (especially at HCs) was not high; limited demand for evidence-based treatment due to:1) lack of awareness of illness signs; 2) reliance on a variety of home treatments for multiple syndromes; 3) belief that illness is self￾limited; 4) reluctance to bring young infants out of the home, fearing “evil eye” or shame; 5) preference for prayer or traditional healing and resorting to “western” care only if conditions worsened; 6) use of pharmacies or private clinics; and 7) lack of mothers’ autonomy to seek care outside the home if there is a financial implication. Field visits during the FE in Lanfero found similar reasons. Strengthening both service supply and demand should increase utilization. Studying the behavioral determinants of a selected few recent adopters of prompt evidence-based treatment could help to refine and strengthen the demand strategy. Very few cases of neonatal illness were seen at HCs and few were seen at HPs. This evidence of appallingly low care seeking for newborns is likely related to low rates of assisted delivery and lack of awareness that health services are available for newborns. Lessons from the SNL research now being carried out in SNNPR and Oromiya Region could further inform efforts to scale-up care for sick newborn babies. 1. Contextual Factors Contextual factors influence the implementation, sustainability and potential impact of the CS-23 project. Many of the implementation-related contextual factors have been discussed above. The contextual factors that might have positively influenced the CS-23 project included: 1) strong policy and program context – the project enhanced an already existing government system; 2) positive synergies with other projects present in Shebedino District, where local NGOs support maternal and neonatal health programming-Lanfero has few complementary health projects; and 3) the policy change to allow pneumonia management also was a timely shift that enabled HEWs to begin treating pneumonia. This motivated both the HEWs, the vCHWs and the communities as antibiotics were now available at HPs. CS-23 Ethiopia, Final Evaluation Report, Save the Children, December 2012 26 Contextual factors that might have negatively impacted the project included: 1) The rural nature of the districts, especially Lanfero, meant that access and care seeking were already low, and made program implementation more difficult; 2) Famine and malnutrition are common problems in the districts, especially in Lanfero-these not only affect the risk of childhood illnesses but also influence care seeking as families may be struggling to find food; 3) The lag in approving the pneumonia policy which allowed community pneumonia treatment delayed the introduction of this important intervention in the project areas; 4) Current policy does not allow neonatal sepsis management at the HEW level; this has constrained efforts to promote care for sick newborns. Merely assessing, counseling and referring is not attractive to either the HEWs nor the community members as referral is often impossible; 5) The availability and access to many drug shops and private clinics, especially in Shebedino, could have served as alternative points for accessing treatment. This would be especially helpful when the communities were unsatisfied with the care they were getting at the HPs (e.g. when a care giver preferred a certain treatment or drug that is not available at the HP). Yet these drug shops and HPs are not part of routine HMIS nor are they part of district/FMOH quality improvement projects; 6) The chronic lack of logistical resources for support activities—e.g., neither of the DHOs has a car, but they do have only one or two old motorbikes for all their activities. Unfortunately, the cost of petrol is often not included in the operating budget. These factors threatens the sustainability of progress at the close of project activities, constrain the day-to-day functioning of DHOs, and challenge the CS￾23 project coordination with FMOH partners; and 7) Health staff in both districts have many competing demands on their time with many staff frequently absent from their posts to engage in other, sometimes non-health related activities. 2. Role of Key Partners As discussed above, the FMOH at all levels is the main partner in project implementation. Other PVOs and multi-lateral agencies were also partners. The role of each project partner, the results of the collaboration and suggestions for improvements are presented in Table 3 below. As discussed above, the FMOH at all levels is the main partner in project implementation. Table 3. CS-23 Ethiopia Key Partners Partners Role in Project Result of Overall Collaboration Activities/ Suggestions for Improvements Regional Health Bureau and Zonal Health Offices and Lanfero and Shebedino District Health Offices  Approval and support for CS-23 activities, particularly with HEWs and communities.  Participate in joint planning and progress review.  Lead and participate in CS Task Force and TAG meetings.  Participate in training activities and provide follow-up on service provision after training.  Conduct joint supportive supervision of HEWs periodically.  Ensure that HPs have essential supplies and medicines for maternal and child health. Results:  Activities for building the capacity of HEWs and vCHWs have proceeded as planned.  Introduction of IMNCI supports and strategies have proceeded as planned.  High level of buy-in for IMNCI and CS activities. Suggestions for improvements:  Increase coordination and frequency of joint supportive supervision suggested.  Increase coordination on monitoring data and use of HMIS systems for CS interventions.  Develop jointly, a transition plan for the end of project. CS-23 Ethiopia, Final Evaluation Report, Save the Children, December 2012 27  Appropriate distribution to HPs within the target area of any equipment donated and/or essential medicines. Regional Bureau of Finance and Economic Development  Oversee the overall coordination of the project at all levels in cooperation with DPPB, RHB and SC/US.  Perform midterm and terminal evaluation of the project according to GO-NGO guideline.  Link SC/US to relevant offices and institutions for securing supports needed in program.  Strong collaboration to monitor the implementation of activities with all partners.  SC/US facilitated the midterm evaluation and covered all expenses.  Support letter to relevant offices to facilitate the implementation of project activities. Population Services International in Ethiopia  Provide orientation training for zinc treatment and provide initial stocks of zinc.  Introduction of zinc has proceeded as planned; no suggestions for improvement. GOAL￾Ethiopia  Provide assistance for orientation training for zinc treatment.  Collaborate in sharing plans and results for CS programming.  GOAL staff have readily collaborated with SC, CS-23, sharing available information and experiences. UNICEF  Support iCCM kit and delivery kit for HPs to implement case management.  Monitor the implementation of IMNCI/iCCM in collaboration with FMOH/RHB.  iCCM kits and delivery kits supported HPs to deliver quality health services.  Integrated supportive supervision to give on￾the-job training for HEWs, to improve IMNCI/iCCM services. JSI/IFHP  Developed IMNCI/iCCM registers and chart booklets in collaboration with FMOH/RHB.  Give IMNCI training for HWs and HEWs in collaboration with SC/US, DHO and RHB.  TOT training for SC/US CS staff to enable them to give training for HEWs.  IMNCI/iCCM registers and chart booklets distributed to all health facilities.  All HPs and HCs are implementing IMNCI/iCCM services and child health services improved.  CS health staffs able to give training foe HEWs and contributed for quality improvement at HPs and HCs.  Knowledge and skill of the health workers and HEWs improved to save the life of children. JSI/L10K  Implementing iCCM activity in collaboration with DHO and CS-23 field staff.  Supportive supervision to update the knowledge and skill of HEWs.  All HPs and HCs are implementing IMNCI/iCCM activities.  Improved knowledge and skills of HEWs to manage cases at HP level. Plan International  Participated on baseline, midterm and final evaluation debriefing workshop to share experiences and to strengthen our partnership.  Participated on all debriefing workshops to adapt lessons learned from CS project to their organization. They have contributed to the generation of ideas during discussion.  Strengthened partnership. CS-23 Ethiopia, Final Evaluation Report, Save the Children, December 2012 28 3. Overall Design Factors that Influenced Results The CS-23 project’s choice to implement simultaneously all three components of IMNCI (clinical, community and health systems) at HCs and HPs for one of the first times in Ethiopia is commendable. This design serves as a model for future programming in caring for the sick child in the community in Ethiopia and beyond. This design is considered best practice to achieve results in improving access and utilization of sick child services, although it is rarely implemented in practice. The newborn care technical component received less attention in design and implementation. The CS-23 project planned to train HEWs in management of sepsis and neonatal infections based on results and lessons learnt in an SNL-funded randomized control trial to be carried out in SNNPR and Oromia, but this activity was delayed and did not release its findings within the life of CS-23. Because of this delay and the need for stronger emphasis on the neonate in order to achieve the CS-23’s ultimate goal of impacts on under-five mortality, the project re-worked the original design and strategy for improving neonatal health by trying to strengthen postnatal visitation, recognition of danger signs, assessment at HPs and referral. However, as discussed before, success was limited. It is evident that changing long held cultural briefs, practices take time. a. Contribution to Global Learning The implementation of IMNCI at the HPs, especially with the addition of pneumonia management, is one of the first in the SNNPR and serves as a learning experience at the regional and national levels. Lanfero and Shebedino Districts are among the first in the country to introduce zinc treatment for management of diarrhea. The project also served as a pilot for developing the global iCCM indicators. CS-23 conducted formative research to assess and propose potential improvements to supervision within the HEP system; a full operational research proposal for HEP supervision strategies is under development. Project staff have also participated and presented the project experiences at international meetings and conferences. Linking with USAID, MCHIP and ACCESS, strategies tested by MCHIP were further expanded by these projects to four additional districts in Malawi, and technical and training materials were shared and replicated more widely. The project also linked with Save the Children’s SNL global country programs. Experiences with implementation have been used to inform approaches to community-based newborn programming in other SNL countries. Finally, one peer reviewed article was published. b. Dissemination and information use In addition to the global learning activities described above, information about project activities and research findings have been disseminated using a number of mechanisms, including:  Local meetings, workshops, trainings and technical updates;  Presentations at international and regional conferences and meetings; Malaria Consortium  Participated on baseline, midterm and final evaluation debriefing workshop to share experiences and to strengthen our partnership.  Participated on all debriefing workshops to adapt lessons learned from CS project to their organization. They have contributed to generate ideas during discussion.  Strengthened partnership. CS-23 Ethiopia, Final Evaluation Report, Save the Children, December 2012 29  Local exchange study tours – staff visit other districts to observe local practices;  International exchange study tours. Save the Children staff, the FMOH and partners visited the SEARCH project in India; and  Program reports, technical documents, training material, facilitators guidelines, and health education materials; all have been available for review and use by the FMOH and other donors and partners. A full list of project publications and presentations is presented in Annex 2. After the field work of the FE, preliminary findings and recommendations were presented to regional stakeholders in a half-day meeting. The CS-23 FE report will be shared with all stakeholders. G. CONCLUSIONS AND RECOMMENDATIONS Overall, the CS-23 project has successfully supported the implementation of the complete package of IMNCI in facilities and the community. Its activities have, and will, serve as a model for implementation of comparable initiatives in Ethiopia. The grant to Save the Children from UNICEF to implement IMNCI/iCCM in the community in 100 districts using a similar approach, is a good measure of the CS-23 project’s success. However, utilization of maternal, child and neonatal services remains a challenge. The project should build on its success in introducing and supporting INMCI, as well as reinforcing capacity and relationships with the FMOH at all levels, in order to introduce and implement stronger strategies to improve neonatal health. In summary, the primary recommendations at endline include: 1. According to the regional and district partners, it is important to engage traditional and spiritual healers as they are still trusted by the community in the case of certain illnesses; 2. Strengthen the capacity of the districts to sustain regular integrated and clinical supervision for HCs, HPs and community volunteers; 3. Revise and distribute BCC materials to include key IMNCI messages; 4. Strengthen MNC with special focus on skilled and clean births, and care for both the well and sick newborn baby; 5. Strengthen HPs to be able to provide 24-hour services by ensuring that each HP has the recommended two HEWs and other key requirements such as solar/electricity and water; 6. In the long term, strengthen the pharmaceutical supply chain to ensure that HPs are adequately stocked with key commodities for IMNCI; 7. Strengthen both the demand and supply side to be able to effectively increase care seeking and treatment for pneumonia at HPs. 8. Expand iCCM to include treatment of the sick young infant at HP level; and 9. Save the Children should continue fostering a strong partnership at the local, regional and national levels to ensure that lessons learnt are scaled-up all over the country. CS-23 Ethiopia, Final Evaluation, Save the Children, December 2012 30 Annex 1: Learning Brief(s): Evidence Building Learning Brief 1: How Do Health Extension Workers Spend Their Time? David Marsh, Peter Waiswa, Hailu Tesfaye, Worku Tefera, et al. Background CS-23 is a five-year (October 2007 to September 2012), USAID-funded project in which Save the Children/US supported government health partners to deliver integrated community case management (CCM) to two districts (Shebedino and Lanfero) in Ethiopia’s Southern Nations, Nationalities and Peoples Region. We aimed to save lives by increasing the coverage of curative interventions (cotrimoxazole for pneumonia, ORS and zinc for diarrhea, and chloroquine or artemether-lumafantrine for confirmed malaria) for sick children under age five. We accomplished this through training existing Health Extension Workers (HEW) in case management, assuring medicine availability, supporting structured supervision, mobilizing demand through community volunteers, and engaging in and informing the national CCM policy, planning and programming dialogue through the Child Survival Working Group. During the midterm evaluation of CS-25 in Zambia Save the Children developed and tested methods to characterize how Community Health Workers (CHW) spent their working time. In short, this volunteer cadre (n=4) reported working full-time (176 hours/month), of which 65 h/m (37%) were devoted to delivering the new CCM strategy. During the final evaluation of CS-23 in Ethiopia, we further refined the methodology to answer the following questions: (1) How much time do paid HEWs spend in the community? In the clinic? Delivering iCCM? Conducting non￾health activities? (2) How does clinic vs. community time allocation change if a health post is staffed by a pair vs. a single HEW? (3) How much time do volunteers spend? (4) How is the time apportioned between service and travel? Methods We first interviewed program staff and government district partners to list common HEW health and non-health activities. On consecutive days, we interviewed four HEWs each from different health posts and one volunteer in Shebedino District, further adapting the list for each informant. For each activity we determined the activity frequency (per week, month or year), duration (minutes, hours, day), and associated preparation and travel time (minutes, hours). We probed to avoid double-counting. Two HEWs worked alone since their partners were receiving extended additional training; the remaining two were members of two-person teams, normally staffing a health post. The latter two reported the collective time for their teams, so we divided the results by two to calculate the time per HEW. Results The four HEWs were female, residents of the health posts’communities, age 20-24 years, and on the job for four to six years. The female volunteer of two years duration was 36 years old. The two solo HEWs reported working longer hours than the paired HEWs (217 vs. 170 h/m). HEWs spent most of their time (62.4%, range 41.2-92.7%) at the health post. Time for preparation and travel for health post activities was minimal, but substantial for community activities (26.8%, range 10.9-55.6%). The main HEW community activities were: training and meeting with volunteers; household visits; vector control, and group health education. HEWs spent little time on CCM (2.7% of health post and 1.6% of total time). Non-health activities (meetings with local Cabinet, Development Team, Women & Children Affairs and facilitating CS-23 Ethiopia, Final Evaluation, Save the Children, December 2012 31 non-health community gatherings) accounted for less than 10% of HEW time. The volunteer reported worked 44 h/m, of which 35.5% was in preparation and especially travel. Discussion HEWs reported working long days, especially if alone at the health post. Whether solo on in pairs, they spent much more time at the health post than the desired norm of 25%, perhaps relying on volunteers for some community duties. The methods are susceptible to reporting bias; however, the overall working time derived from totaling each activity is remarkably close to “full time” (173 h/m), thereby lending credibility. CS-23 Ethiopia, Final Evaluation, Save the Children, December 2012 32 Learning Brief 2: The Status of Ethiopia’s Integrated Community Case Management Strategy and the Contribution of USAID’s Child Survival and Health Grant’s “CS-23” Project to Save the Children Hailu Tesfaye, Tedbabe Degefie, Karen Waltensperger, David Marshet al. Background CS-23 is a five-year (October 2007 to September 2012), USAID-funded project in which Save the Children/US supported government health partners to deliver integrated community case management (CCM) to two districts (Shebedino and Lanfero) in Ethiopia’s Southern Nations, Nationalities and Peoples Region. We aimed to save lives by increasing the coverage of curative interventions (cotrimoxazole for pneumonia, ORS and zinc for diarrhea, and chloroquine or artemether-lumafantrine for confirmed malaria) for sick children under age five. We accomplished this through training existing Health Extension Workers (HEW) in case management, assuring medicine availability, supporting structured supervision, mobilizing demand through community volunteers, and engaging in and informing the national CCM policy, planning and programming dialogue through the Child Survival Working Group. CCM is a widely endorsed strategy to increase the coverage of life-saving interventions in communities that have low access to facility-based curative care for sick children. Planners and programmers increasingly view the strategy through a health systems perspective with three program phases: pre-introduction, introduction, and scale up – and eight components: (1) organization, coordination and policy setting; (2) budgeting, costing and financing; (3) human resources; (4) supply chain management; (5) service delivery and referral; (6) behavior and social change, sensitization and advocacy; (7) supervision and performance; and (8) monitoring, HMIS, evaluation and research – for which 70 benchmarks have been proposed. With support from USAID’s Child Survival and Health Grants Program, we developed and applied a methodology to “map” the status of a country’s strategy. Methods In April 2012, the Project Manager (HT)completed a structured data collection form to characterize each benchmark in terms of national achievement (no/partial/yes) and project activities that helped achieve the national benchmark (no/yes – with supporting evidence). We scored responses for national achievement as 0 (no), 1 (partial), or 2 (yes). We vetted the Project Manager’s responses through a UNICEF/Ethiopia respondent, which resulted in adjusting <10% of responses. We analyzed CCM benchmark achievement by health system component and NGO role in helping to achieve these – also by health system component and country. Results At the time of the assessment, Ethiopia had a mature iCCM strategy with most (83%) benchmarks achieved. Three of eight components were more than 90% achieved (human resources, behavior and social change, and monitoring). Supply chain (75%), service delivery (72%) and supervision (61%) had the most room for improvement. Save the Children appeared to have facilitated achieving several national iCCM benchmarks in all components. Discussion Benchmarks cannot describe the totality of a health system, even for a single strategy; however, they are consensus “markers” for emerging best practices regarding the essential elements of a functional system. Ethiopia is one of a few countries globally to have achieved near national scale-up this life-saving strategy – and by far the largest country (by CS-23 Ethiopia, Final Evaluation, Save the Children, December 2012 33 population) to have done so. USAID – through a CSHGP grantee – played a catalytic role. Benchmark maps convey a host of complex information while instructing the audience. CS-23 Ethiopia, Final Evaluation, Save the Children, December 2012 34 Learning Brief 3: CCM Indicators: Feasibility, Validity, Usefulness, and Sustainability Worku Tefera, Hailu Tesfaye, Karen Waltensperger, David Marsh et al. Background CS-23 is a five-year (October 2007 to September 2012), USAID-funded project in which Save the Children/US supported government health partners to deliver integrated community case management (CCM) to two districts (Shebedino and Lanfero) in Ethiopia’s Southern Nations, Nationalities and Peoples Region. We aimed to save lives by increasing the coverage of curative interventions (cotrimoxazole for pneumonia, ORS and zinc for diarrhea, and chloroquine or artemether-lumafantrine for confirmed malaria) for sick children under age five. We accomplished this through training existing Health Extension Workers (HEW) in case management, assuring medicine availability, supporting structured supervision, mobilizing demand through community volunteers, and engaging in and informing the national CCM policy, planning and programming dialogue through the Child Survival Working Group. The midterm evaluation of CS-23 (August 2010) coincided with the emerging consensus on indicators to track CCM. Given the lack of experience with these new indicators, we decided to measure 27 of them at a project level and review experience during the final evaluation (August 2012). Methods After the midterm evaluation, the project Monitoring and Evaluation Officer developed and implemented a monitoring framework for the indicators. During the final evaluation, we assessed each indicator according to: required effort, usefulness, likely validity, and likely sustainability. We assigned values of 3, 2 or 1 to each variable, 3 being the best (i.e., low effort, or high usefulness, likely validity and likely sustainability). We weighted each variable equally and totaled the scores for an overall value with maximum possible value, 12. Results Most indicators (18/27) required a low level of effort to obtain, were likely valid (23/27) and useful (22/27); but only a third (9/27) were likely routinely sustainable, even with partner support. Overall indicator value scores ranged from 6 (one indicator) to 12 (six indicators) of 12. Eighteen indicators scored 10 or higher, and 10 scored 11 or higher. Emerging “Implementation Strength” indicators scored highly: CHW functionality (score: 12), community coverage (12), routine supervision coverage (11), clinical supervision coverage (11), CHW density (10), and drug availability and stock-out (9 each). The lowest value scores were appropriate referral (6), timely treatment (7), knowledge of illness signs (8) and referral ratio (8). Discussion The list of global CCM indicators has evolved somewhat in the past two years, but the phenomena measured are similar and in many cases identical. The experience of CS-23 and this validation exercise confirmed emerging global consensus on implementation strength indicators. The likely sustainability of several CCM quality indicators assumed continued partner support. USAID and grantees are well positioned to continue to advance the experience- and evidence-base of indicator use, and especially approaches to sustain, measure and report frontline health worker skills. CS-23 Ethiopia, Final Evaluation, Save the Children, December 2012 35 CS-23 Global CCM Benchmark Indicators (Implementation Strength indicators in bold) Components Indicator Score (Low, Med, High) Effort to Obtain Likely Validity Useful￾ness Sustainable Coordination &Policy Setting Policy – CCM incorporated in national MNCH policy or guidelines (yes/no) Low High High High Costing and Finance Costing – comprehensive costing for all components established, i.e. supply chain mgt, training, supervision, etc.) Low High High High Human Resources CHW Functionality (% active/ deployed) Low High High High Retention Ratio (% active/trained) (% HEWs who provide iCCM on 1 April who are still on the job on 31 May) Low High High High CHW Density (#/1000 total population) Low High High Low Community Coverage (% targeted areas with CCM) Low High High High CHW Training Plan: new skills (yes/no) Low High High Low Continuing Education Plan: refresher (yes/no) Low High High Low Supply Chain Management Drug Availability (% implementation sites with all iCCM drugs on day of observation) Medium High High Low Stock-out Ratio (% implementation sites with stock-outs) Medium High High Low Timely Treatment (% treatments within 24 hours) High Medium High Low Treatment Ratio (actual treatments/expected cases/yr) Medium High Medium Medium Case Load (# cases/site/month or quarter) Low Medium Medium Medium Service Delivery, Referral Referral Ratio (% total cases referred) Low Medium Medium Low Appropriate Referral Ratio (% cases with DS or severe disease referred) High Medium Medium Low Communication and Social Mobilization Communication Strategy – plan for communication developed and messages and materials for health staff and community tested and available (yes/no) Low High High Low Knowledge of Illness Signs (% of caregivers who know > 2 signs of childhood illness) High High High Low Supervision Supervisory Plans and Tools (checklists, guidelines, training materials, plans and SOPs available) (yes/no) Low High High Low Routine Supervision Coverage (% CHWs receiving > 1 supervisory visit in the prior 3 months with registers and/or reports review) Low High High Medium CS-23 Ethiopia, Final Evaluation, Save the Children, December 2012 36 Clinical Supervision Coverage (% CHWs receiving > 1 supervisory visit in the community in the prior 3 months where case management observed and coaching provided) Low High High Medium CHW/Supervisor Ratio (# CHWs / # supervisors ) Low High High High Consistent Case Management (% registered cases with consistent assessment, classification, treatment) Medium High High High* Treatment Knowledge (% CHWs correctly managing case scenario) Medium High High High* Treatment Practice (% CHWs correctly managing sick children or % sick children correctly managed) High High High High* Monitoring and Evaluation, HIS CCM in HMIS – one or more CCM indicators in HMIS (yes/no) Low High Medium Low Standard Reporting – standardized CCM registers and reporting documents available for CHWs and HF (yes/no) Low High High Low District CCM Monitoring Ratio (% implementing districts providing CCM monitoring data) Low High High Low *sustained through partners. CS-23 Ethiopia, Final Evaluation, Save the Children, December 2012 37 Learning Brief 4: The Potential of mHealth to Support Health Extension Worker Supervision in Ethiopia’s Southern Nations, Nationalities and Peoples Region Hailu Tesfaye, Rita Nemero, David Marsh, et al. Background CS-23 is a five-year (October 2007 to September 2012), USAID-funded project in which Save the Children/US supported government health partners to deliver integrated community case management (CCM) to two districts (Shebedino and Lanfero) in Ethiopia’s Southern Nations, Nationalities and Peoples Region. We aimed to save lives by increasing the coverage of curative interventions (cotrimoxazole for pneumonia, ORS and zinc for diarrhea, and chloroquine or artemether-lumafantrine for confirmed malaria) for sick children under age five. We accomplished this through training existing Health Extension Workers (HEW) in case management, assuring medicine availability, supporting structured supervision, mobilizing demand through community volunteers, and engaging in and informing the national CCM policy, planning and programming dialogue through the Child Survival Working Group. Save the Children, in partnership with UNICEF, also supported the training of HEWs from 935 health posts in iCCM in 35 districts and one sub-city in SNNPR, including the two CS-23 districts. Teams were challenged to conduct the post-training follow-up and supportive supervision per set standards. Thus, we surveyed HEW trainees regarding their mobile phone use, as a possible alternative approach for some follow-up. Methods We surveyed 325 HEW trainees from Sidama and Gurage Zones of SNNPR during 11 rounds of training. We sought information about mobile phone ownership, phone type, SMS use, network availability and past use of mobile phones for job-related communication. Results Most HEWs (89%, 289/325) reported owning mobile phones, most commonly (80%, 232/289) Nokia. Nearly all (95%, 275/289) reported that their phone could send and receive text messages. Most could access a network at their health post (74%, 213/289) or in their community (77%, 221/289) – which was often the same place. Two-thirds (68%) of local networks were available Monday to Friday. Most (84%) of the HEWs without local access to a network reported traveling three to eight days to reach a network. Few (7%) HEWs can charge their mobile phone at the HP, and for more than half (57%, 165/289) the mobile phone will work for three or fewer days, once fully charged. Most HEWs (87%, 250/289) reported being supervised or asking for work-related information using their mobile phone. Three in ten (29%, 84/289) HEWs could send SMS in English, but six in ten (60% 172/289) could understand messages sent to them in English. Discussion Most HEWs have mobile phones and already use them informally to support their duties. The potential for “mHealth” to plug the supervision and support gap in Ethiopia is great, notwithstanding the challenges of electricity, network and English language. Not surprisingly, the Bill & Melinda Gates Foundation is supporting a national consultation to systematically explore this. CS-23 Ethiopia, Final Evaluation, Save the Children, December 2012 38 Learning Brief 5: Multi-factorial Causes for Low Utilization in Shebedino vs. Lanfero District Hailu Tesfaye, Worku Tefera, Abeba Bekele, Karen Waltensperger, David Marsh et al. Background CS-23 is a five-year (October 2007 to September 2012), USAID-funded project in which Save the Children/US supported government health partners to deliver integrated community case management (CCM) to two districts (Shebedino and Lanfero) in Ethiopia’s Southern Nations, Nationalities and Peoples Region. We aimed to save lives by increasing the coverage of curative interventions (cotrimoxazole for pneumonia, ORS and zinc for diarrhea, and chloroquine or artemether-lumafantrine for confirmed malaria) for sick children under age five. We accomplished this through training existing Health Extension Workers (HEW) in case management, assuring medicine availability, supporting structured supervision, mobilizing demand through community volunteers, and engaging in and informing the national CCM policy, planning and programming dialogue through the Child Survival Working Group. The August 2010 midterm evaluation noted marked differences in curative service utilization – through register review – between the two districts. Lanfero treated ten times more cases of pneumonia than Shebedino (465 versus 46 cases per 1000 estimated population of under-fives per year, respectively). The trend was similar for malaria and diarrhea. We aimed to understand why. Methods Using a Results Framework to inform our approach, we investigated use (explanations for under-reporting); access (geographical, financial, cultural); quality (real or perceived, availability of medicines); and demand (knowledge of illness signs and illness recognition, home care, labeling, decision-making, patterns of care-seeking outside the home). Resources limited our investigations to Shebedino only. We conducted: (1) a population-based household survey using LQAS sampling (n=114 mothers of children under 5); (2) key informant interviews with the Senior Health Extension Worker (HEW) in six kebeles, three each with high and low access to a health center; (3) focus group discussions with mothers of sick children within the last three months (n=6) from the same kebeles; (4) key informant interviews with health workers at all six health centers; and (5) health facility assessments of all six health centers and at the health posts in the same six kebeles. Results Reasons for under-reporting actual use of treatments were: malnourished children registered in Outpatient Treatment Program but not in IMNCI Register, HEWs treating sick children during household visits without registers, high patient loads, and forgetfulness. Geographic and financial access; barriers were common for services at health centers, but not at health posts – sometimes aggravated by seasonal harvest responsibilities and flooding. The technical quality of case management at all health facilities (especially at health centers) was not high. Indeed, some mothers suspected HEW ability and some were uncomfortable with some health worker attitudes, but the main concern was medicine shortages. A host of factors limited demand for evidence-based treatment: (1) lack of awareness of illness signs; (2) reliance on a variety of home treatments for multiple syndromes; (3) belief that illness is self-limited; (4) reluctance to bring young infants out of the home, fearing “evil eye” or shame; (5) preference for prayer or traditional healing and resorting to “western” care only if conditions worsened; (6) use CS-23 Ethiopia, Final Evaluation, Save the Children, December 2012 39 of pharmacies or private clinics; and (7) lack of mothers’ autonomy to seek care outside the home if there is a financial implication. Discussion The reasons for sub-optimal use of evidence-based treatments for serious childhood infection in Shebedino are multi-factorial, and the differences from Lanfero are not known since we did not study both districts. Nonetheless, strengthening both service supply and demand should increase utilization. Studies of the behavioral determinants of a few recent adopters of prompt evidence-based treatment could refine the demand strengthening. CS-23 Ethiopia, Final Evaluation, Save the Children, December 2012 40 Learning Brief 6: How Did USAID’s Child Survival and Health Grant’s “CS-23” Project to Save the Children Contribute to Community Case Management of Pneumonia Policy Change and iCCM scale up in Ethiopia? Hailu Tesfaye, Abeba Bekele, David Marsh, Karen Waltensperger, Rita Nemero, et al. Background CS-23 is a five-year (October 2007 to September 2012), USAID-funded project in which Save the Children/US supported government health partners to deliver integrated community case management (CCM) to two districts (Shebedino and Lanfero) in Ethiopia’s Southern Nations, Nationalities and Peoples Region. We aimed to save lives by increasing the coverage of curative interventions (cotrimoxazole for pneumonia, ORS and zinc for diarrhea, and chloroquine or artemether-lumafantrine for confirmed malaria) for sick children under age five. We accomplished this through training existing Health Extension Workers (HEW) in case management, assuring medicine availability, supporting structured supervision, mobilizing demand through community volunteers, and engaging in and informing the national CCM policy, planning and programming dialogue through the Child Survival Working Group. Save the Children’s previous CS-17 project in Ethiopia’s Oromia Region demonstrated the feasibility of CCM – including CCM of pneumonia (CCM/P) – through community health workers in a pre-Health Extension Program era. These workers treated 2.5 times more children than all health workers in Liben District and demonstrated high quality case management. At CS-23 project start-up, Ethiopia’s policy forbade CCM/P although it was intended to be a key CS-23 strategy. Thus, in agreement with the Southern Regional Health Bureau, Save the Children supported the training of 109 HEWs in IMNCI for the first time in Shebedino and Lanfero Districts to assess, classify and treat diarrhea and malaria plus assess, classify and refer pneumonia. Advocacy Methods The project’s fourth intermediate result included advocacy for CCM/P. During project planning, Save the Children briefed Ministry of Health Officials including Regional Health Bureau Heads on the global status of CCM/P and its need in Ethiopia. We held similar meetings with UNICEF, WHO and JSI of the USAID bilateral. We also drafted “Pneumonia Control Memo: Control of Childhood Pneumonia in Shebedino and Lanfero Woredas in SNNPR and in Ethiopia, Generally” which described pneumonia epidemiology, pneumonia control status globally and in Ethiopia, and opportunities in Ethiopia to implement CCM/P and to show policy leadership. We shared the memo with the State Ministry of Health and Southern Ethiopia Regional Health Bureau. We also published the Liben CS-17 CCM/P experience in the Ethiopian Journal of Health and Development and presented findings at the Ethiopian Pediatric Society’s (EPS) 10thAnnual Conference in May 2008. The Society endorsed the findings and recommended “the need to revise the current national policy on CCM to allow HEWs to treat pneumonia at community level and development partners, NGOs and professional societies to support the government in training, supervision, logistic and supplies while rolling out this initiative of CCM of pneumonia.” JSI’s ESHE Project amplified the experience, reporting that IMNCI-trained HEWs in Bolososore District of Southern Ethiopia– when trained, supervised and supported – could implement CCM according to protocol at health posts (implementing IMNCI within the HEP, Nov 2009). CS-23 Ethiopia, Final Evaluation, Save the Children, December 2012 41 A second advocacy approach was a partnership of UNICEF, WHO, JSI/ESHE, EPS, Save the Children and others who advocated for CCM/P through national and regional Child Survival Technical Working Groups (TWG), professional forums and annual meetings. This partnership organized field visits for MoH officials and policy-makers to Liben and Bolososore Districts, and UNICEF organized a visit for senior federal MoH and RHB officials to the Society for Education, Action and Research in Child Health (SEARCH) in Gadchiroli, India in October 2009. Results In light of the above-mentioned multiple approaches, plus other global activities (UNICEF-WHO Joint Statement for CCM/P [2004], Global Action Plan for Pneumonia [2008], the “Tipping Point” paper in the Bulletin of WHO [2008], and World Pneumonia Day and Summit [2009]), the Government of Ethiopia adopted a CCM/P policy in February 2010, thereby allowing HEWs to treat childhood pneumonia at the community level. As a member of the TWG, the CS-23 team then helped develop and review the three-year iCCM implementation plan launched in the four big regions with support from UNICEF and USAID. The CS-23 team facilitated and participated in the first pilot iCCM HEW training and facilitated the national master and regional trainings of trainers (TOT). CS-23’s Shebedino District was a national and regional iCCM “learning district” since it was the only available iCCM implementing district at the time; thus it was the site of practical training for the national and regional TOTs. Save the Children/US is an implementing partner to scale up iCCM in 100 districts (36 in SNNPR and 64 in Oromia), and Save the Children/UK supports 25 districts in Amahara Region. To date Save the Children has trained 6111 HEWs to deliver iCCM – including pneumonia treatment – at 3368 health posts, each serving a population of about 5000, i.e., serving a total population of about 16 million. Discussion Policy change takes time. Local evidence (including experience from CS-17 published during CS-23), strong and strategic partnerships, site visits, and persistent multi￾channel advocacy collectively has a “big voice to reach big ears.” To date Ethiopia has implemented CCM/P in 500 districts of SNNPR, Oromia, Tigray, Amhara, Benshangul Gumez and Gambela; has trained 21,500HEWs in more than 10,000 health posts; and has delivered life￾saving curative interventions to more than 250,000 children. CS-23 Ethiopia, Final Evaluation, Save the Children, December 2012 42 Learning Brief 7: Mother-to-Mother Care Group, Pregnant Mothers’ Forum and Increased Institutional Delivery in Lanfero District, Ethiopia Hailu Tesfaye, Getenet Kebede, Habtamu Tilahun, Chiksa et al. Background CS-23 is a five-year (October 2007 to September 2012), USAID-funded project in which Save the Children/US supported government health partners to deliver integrated community case management (CCM) to two districts (Shebedino and Lanfero) in Ethiopia’s Southern Nations, Nationalities and Peoples Region. We aimed to save lives by increasing the coverage of curative interventions (cotrimoxazole for pneumonia, ORS and zinc for diarrhea, and chloroquine or artemether-lumafantrine for confirmed malaria) for sick children under age five. We accomplished this through training existing Health Extension Workers (HEW) in case management, assuring medicine availability, supporting structured supervision, mobilizing demand through community volunteers, and engaging in and informing the national CCM policy, planning and programming dialogue through the Child Survival Working Group. The third intermediate result of the project aimed to increase demand for using high impact interventions through increased knowledge and acceptance of evidence-based health behaviors promoted by volunteer community health workers. The midterm project evaluation (August 2010) recommended strengthening maternal and newborn health promotion by pilot-testing Mother-to-Mother Care Group (MTMCG) in three kebeles in Lanfero District. The district health team observed that, after the MTMCG, a pilot kebele experienced increased deliveries at its health post. Thus, the team revised and scaled up the approach as a Pregnant Mothers Forum (PMF) in 25 of 27 kebeles. A PMF is a group of five to 15 pregnant mothers. Each group has a team leader, also a pregnant mother and an active community member. The leader takes all PMF members for antenatal care (ANC), ensuring that each takes iron-folate. She conducts PMF meetings fortnightly for members near term, or monthly for others. The meetings occur at the health post so that the HEW and the community volunteers can join the meeting. HEWs are trained in safe, clean delivery, and volunteers know and communicate/share related messages. During the meeting the members learn about regular ANC, nutrition, birth preparedness, maternal and newborn danger signs, and the advantages of giving birth in health facilities. Each PMF meeting has a coffee ceremony and a “porridge ceremony,” a cultural ceremony of eating porridge with close friends and relatives when a mother gives birth. The PMF leader follows all members and ensures that they give birth at the health post, not at home. If labor occurs after hours, the PMF has the HEW cell phone number to call for help. When a PMF member delivers at the HP, other members gather therefor at least six hours for the porridge ceremony, having contributed corn flour. The six-hour stay gives the HEW an opportunity to conduct early postnatal care for the mother and newborn, reinforcing breastfeeding positioning and attachment, immediate breastfeeding, and other essential newborn care messages. Ethiopia has a very high maternal mortality ratio of 676/100,000 live births (EDHS, 2011) which has not shown progress since the 2005 DHS. Institutional delivery (skilled birth attendance) is only 10%. Only 34% received antenatal care, 48% received TT vaccine, and very few (7%) received postnatal care. The major barriers to access health services were lack of transport, lack of money and distance to health center (EDHS, 2011). CS-23 Ethiopia, Final Evaluation, Save the Children, December 2012 43 Methods The PMF was established in 2004 (Ethiopian Calendar [EC]), so we viewed delivery reports and registers at the 25 health posts to compare the number of institutional deliveries conducted at health posts by the HEWs in 2003 and 2004 (EC). We also compared the number of deliveries attended by HEWs at the 25 HPs three months before and three months after the establishment of the PMF (Oct 11, 2011 to Jan 7, 2012 and April 9, 2012 to July 7, 2012, respectively). Results The number of health post deliveries doubled in 2004 (EC) from 394 in 2003 (EC) to 802. Moreover, the number of health post deliveries increased nearly five-fold from the three months before to the three months after the establishment of the PMF (74 vs. 345). Applying Ethiopia’s crude birth rate for rural populations (36.2 live births per 1000 total population per year) to the estimated catchment area population that received the PMF (122,319), we estimate that the proportion of deliveries occurring at health posts increased from 7% in the quarter before to 31% in the quarter after PMF was introduced. Discussion This before-after study strongly suggests that the PMF approach was effective, given the timing of the results and the content of the approach. The District Health Office showed innovative leadership in using a special group of mothers, in this case pregnant mothers, to address their own issues in a network with local resources. Most mothers now give birth at health posts or are planning to do so. It is likely that use of ANC, TT immunization, iron-folate, and immediate breastfeeding are also increasing. This innovative strategy has begun to address gaps in maternal and newborn health and can be replicated throughout the zone and beyond. Delivery at a health post is not the same as delivery by a skilled birth attendant, but it is an improvement over home delivery and should save lives. CS-23 Ethiopia, Final Evaluation, Save the Children, December 2012 44 Learning Brief 8: Developing and Applying a “Fill-in-the-Blanks” Case History Template Saving Lives and Informing the Introduction and Scale-up of a National Integrated Community Case Management Strategy for Ethiopia Hailu Tesfaye, Karen Waltensperger, David Marsh et al. Situation Despite progress, Ethiopia still has a high under-five mortality rate (106 deaths per 1000 live births [Countdown to 2015, 2012]), with many preventable child deaths due to pneumonia (21%), diarrhea (14%) and malaria (2%). Baseline levels of coverage of curative interventions were 32% and 28% for care-seeking and treatment of ARI needing assessment, 57% for treatment of diarrhea with ORS, and 47% care-seeking for fever (household survey, 2007). Thus, Save the Children’s five-year project “Innovation for Scale – Enhancing Ethiopia’s Health Extension Program” (2008-2012) aimed to improve the health and survival of 56,000 children under five years of age through increasing the use of high impact treatment interventions delivered through the Community Case Management Strategy (CCM). CCM Context The impact area was comprised of two densely-populated districts, Shebedino and Lanfero, in Southern Nations, Nationalities and Peoples Region (SNNPR). Shebedino (total population 223,000, 461/square km) is mainly Protestant; Lanfero (total population 139,000, 248/square km) is mainly Muslim. The CCM package includes curative interventions (cotrimoxazole for pneumonia, ORS and zinc for diarrhea, chloroquine for vivax and Coartem® for falciparum malaria – distinguished by multispecies rapid diagnostic tests) for sick children age 2-59 months of age and assessment and referral for sick young infants aged 0-1.9 months. The Health Extension Workers (HEW) who deliver CCM are an official, salaried, female, class￾10 educated cadre who receive a year’s training and who generally return to their community, usually assigned as pairs at each health post. Ethiopia introduced CCM in February 2010 when pneumonia case management was added to HEW skills, and began scale-up in July 2010 through CIDA, USAID and other partners. Strategies Project strategies and approaches to increase access to case management included: (1) training all existing HEWs (approximately two per 5000 total population) in CCM, (2) training at least four health workers per health center in IMNCI, (3) introducing a referral slip for HEW and vCHWs – later replaced by the national CCM referral slip, and (4) occasionally providing urgent referral via a project vehicle in Lanfero. Approaches to assure CCM quality through assuring medicine availability included: (1) providing medicines for the first four years, (2) introducing stock and bin cards at health posts, and (3) supporting the government to assume responsibility for logistics in year 5. Approaches to assure the CCM quality through training included: (1) supporting training of master trainers in Shebedino; (2) supporting national competency-based CCM training with pre- and post-tests, role-plays, videos, and classroom and practical training; and (3) in-depth structured supportive supervision within six weeks of training. Approaches to assure the CCM quality through supervision included: (1) joint supervision with government partners; (2) occasional transport; and (3) support to three supervision approaches: on-going weekly HEW supervision from health center staff, on-going quarterly integrated supportive (ISS) supervision, and quarterly CCM mentoring and review meetings supported by UNICEF. CS-23 Ethiopia, Final Evaluation, Save the Children, December 2012 45 Strategies to increase demand for CCM services included: (1) social and behavior change through volunteer Community Health Promoters in years 1 to 3; (2) formation of care groups, Mother2Mother Groups and Elder Groups after the midterm evaluation; (3) support to the national Health Development Army strategy which absorbed the foregoing in year 4; (4) a revised Family Health Card reconciled with the 20 IMNCI key messages; and (5) a child-to-child approach whereby school children bring messages to their family. Approaches to enable the national environment for CCM included, as a member of the Child Survival Technical Working Group: (1) participating in mapping and defining HEW roles, reviewing policies and guidelines and launching CCM strategy; (2) informing the national monitoring framework (adapted from CS-23) and action research agenda; (3) reviewing the HEW retention strategy; and (4) contributing to drug supply implementation plan. In addition, CS-23 (5) established the first CSTWG in SNNPR; (6) was first to introduce CCM for pneumonia and thus inform the training strategy; (7) supported HEWs who received the “best performance” awards; and (8) facilitated CCM guideline pre-testing, training, and review. Measures of success included indicators of intervention use (ORS and zinc for diarrhea, antibiotics for ARI needing assessment, antimalarial for fever), access (density of HEW), quality (availability of medicine), demand (knowledge of danger signs), and an enabled environment (supportive policy for CCM of pneumonia). Tools Key CCM documents include: two sick child registers for infants and children <2 and 2-59 months old, referral forms from volunteers to HEWs and from HEWs to health centers, monthly reports, ISS checklist, and the UNICEF “Form C” CCM checklist. The two patient registers recapitulate each case management step (ask, look, decide, treat, refer, follow-up). The ISS checklist tracked CCM and other services (family planning, antenatal care, etc.). Form C focused on CCM and tracked consistency of register recording, referrals, outcomes, supplies, medicines, HEW knowledge, and more. Results The quantitative results are in Table 1 below. Overall, the project met most targets. In addition, the project is generally sustainable since it worked within government policy, plans, programs and structures. What mostly distinguished this project is how much it informed national CCM. The main challenges – common in most CCM settings – are assuring continued access to (i.e., deploying additional HEWs to compensate for the annual 10% attrition rate) and quality of (sustained medicine supply and case management skills) CCM services. CS-23 Ethiopia, Final Evaluation, Save the Children, December 2012 46 Table 1: Main Project Results Result Indicator Global Baseline Endline Implementation Strength Routine supervision to Health Post in last qtr Yes Lanfero: 100% Shebedino: 100% Clinical supervision to Health Post in last qtr Yes Lanfero: 100% Shebedino: 100% HEW density (per 1000 <5s) Yes Lanfero: 2.1 Shebedino: 1.6 Access Increased HEWs trained and delivering iCCM Yes Lanfero: 0% Shebedino: 0% Lanfero: 87% Shebedino: 93% Health Posts delivering CCM 0% 100% Quality Assured Health Posts with all CCM medicines Yes 0% 100% Demand Mobilized Knowledge of 3+ childhood illness danger signs Yes 15% 51% Environment Enabled Supportive policy for CCM of pneumonia Yes No Yes Use (Coverage) Increased Appropriate care-seeking for cough and fast or difficult breathing Yes 32% 45% Use of antibiotic for cough and fast or difficult breathing Yes 28% 53% Appropriate care-seeking for fever Yes 47% 72% Use of antimalarial for fever Yes 17% 47% Use of ORS for diarrhea Yes 57% 55% Use of zinc for diarrhea Yes 7% 34% CS-23 Ethiopia, Final Evaluation, Save the Children, December 2012 47 Summary Template for Projects Delivering Community Case Management1 (draft April 20, 2012) Situation XXXi has a high under five mortality rate (XXXii [XXX]iii), with many preventable child deaths due to pneumonia (##iv%), malaria (##%) and diarrhea (##%). Baseline levels of coverage of curative interventions were XXXv and XXX for care-seeking for and treatment of ARI needing assessment, XXX for treatment of diarrhea with ORS, and XXX for treatment of fever/malaria (XXXvi). Thus, XXX’svii #-year “XXXviii Project” (20##-20##ix) aims/aimed to improve the health and survival of ##,000 children under five years of age through increasing the use of high impact treatment interventions delivered through the Community Case Management Strategy (CCM). CCM Context The impact area in ##x districts in XXxiProvince is XXXxii-populated (total population/square kilometer: ##xiii); the terrain is XXXxiv; and the population is XXX.xv The CCM package includes curative interventions (XXXxvi for XXX,xvii XXX for XXX, XXX for XXX) for sick children age ##-##xviii months of age. The community health workers (called XXX) who deliver CCM are an XXXxix, XXXxx, XXXxxi cadre of worker, generally XXX.xxiiOther contextual factors for CCM included XXXxxiii. Strategies Strategies and approaches to increase access to CCM include(d): (1) XXX,xxiv (2) XXX, (3) XXX, (4) XXX, and (5) XXX. Approaches to assure CCM quality through medicine availability include(d): (1) XXX, (2) XXX, and (3) XXX.xxv Approaches to assure the CCM quality through training include(d): (1) XXX,xxvi (2) XXX, (3) XXX, (4) XXX, and (5) XXX. Approaches to assure the CCM quality through supervision include(d): (1) XXX,xxvii (2) XXX, (3) XXX, (4) XXX, and (5) XXX. Strategies to increase demand for CCM services include(d): (1) XXX,xxviii (2) XXX, (3) XXX, (4) XXX, and (5) XXX. Strategies to enable the community environment for CCM include(d): (1) XXX,xxix (2) XXX, (3) XXX, (4) XXX, and (5) XXX. Strategies to enable the national environment for CCM include(d): (1) XXX,xxx (2) XXX, (3) XXX, (4) XXX, and (5) XXX. Measures of success include(d) indicators of use (XXXxxxi), access (XXXxxxii), quality, (XXXxxxiii), demand (XXXxxxiv), and an enabled environment (XXXxxxv). Tools Key CCM documents were: XXX.xxxvi The sick child recording form XXX.xxxvii The patient register XXXxxxviii. The referral form XXX.xxxix The periodic report XXX.xl The supervision checklist tracked                                                              i Name of country ii Level of under‐five mortality iii Source of U5MR iv Proportionate mortality value – from MGD report v Indicator values vi Source of indicator values. vii Name of NGO viii Name of project                                                              1 This note can either be framed as a summary for completed projects or a progress report for on‐going projects. CS-23 Ethiopia, Final Evaluation, Save the Children, December 2012 48                                                                                                                                                                                                 ix Project dates x Number of districts xi Name of province xii Sparsely vs. densely xiii TP/square km xiv Ecology, i.e., drought‐prone, flood‐prone, mountainous, semi‐arid, arid, hilly, fertile, etc. xv Settled in fixed communities vs. agro‐pastoralist vs. nomadic, etc. xvi Name of medicine xvii Name of syndrome (diarrhea, fast breathing, fever or RDT‐positive fever xviii Specify age in months xix Official vs. unofficial cadre xx Salaried vs. volunteer xxi Literate vs. semi‐literate vs. non‐illiterate xxii Mostly male vs. balanced male and female vs. mostly female. xxiii Specify any other factors, i.e., ecological, political, policy, etc. xxiv List up to 5 (e.g., mapping, CHW selection for access, CHW deployment, CHW retention, referral strengthening, etc.) xxv Specify if the project relied on government supplies or purchased medicines, along with logistical support approaches. xxvi List up to 5 (e.g., CHW selection for quality, competency‐based training, training package selected, training duration, % clinical, competency‐based certification, competency‐based job aids, etc.) xxvii List up to 5 (e.g., competency‐based supervisor training, deploying supervisors, competency‐based supervision of CHWs, frequency of supervision [plan vs. actual], supervision content, supervision locus, supervision of supervisors, etc.) xxviii List up to 5 (e.g., sensitization, messages, targets, channels, products, etc.) xxix List up to 5 xxx List up to 5 (e.g., advocate, demonstrate, evaluate, conduct research, contribute to technical advisory group, etc.) xxxi Specify 1‐3 indicators. xxxii Specify 1‐3 indicators. xxxiii Specify 1‐3 indicators. xxxiv Specify 1‐3 indicators. xxxv Specify 1‐3 indicators. xxxvi Specify from: sick child recording form, patient register, referral form, periodic report, supervision checklist, mentoring checklist, etc. xxxvii Characterize: literate vs. non‐literate, adaptations from WHO/UNICEF, etc. xxxviii Specify: literate vs. non‐literate, open‐ended vs. tick‐based, # columns, and general contents (columns for identifying data, assessment, classification, treatment, referral, follow‐up, outcome). xxxix Specify: literate vs. non‐literate; format (open‐ended vs. pre‐formatted); and back‐ or counter‐referral (yes or no) xl Specify: who completed it, what it tracked (syndromes, age groups, sex, referrals, outcomes, medicines used and supplied), and how it was used (monitoring, coaching, discussion with community, not used much) CS-23 Ethiopia, Final Evaluation, Save the Children, December 2012 49 Annex 2: List of Publications and Presentations Related to the Project List of Presentations/Publications To Whom Date 1.CS-23 project overview for partners Regional Health Bureau, Sidama and Silti Zone Health Departments, to Lanfero and Shebedino DHOs, UNICEF, JSI/ESHE, GOAL, Plan International March 2008 2.CS-23 project overview Regional CS task force which includes RHB, UNICEF,WHO, JSI and international NGOs in SNNPR March 2008 3.Community Case Management Improves Use of Treatment for Childhood Diarrhea, Malaria and Pneumonia in a Remote District of Ethiopia Annual conference of the Ethiopian Pediatric Society, close to 100 pediatricians in attendance. (SC also distributed articles on pneumonia CCM) May 2008 4.CS-23 project overview CS Technical Advisory group which includes MoH, UNICEF, GOAL Ethiopia, JSI/IFHP, Malaria Consortium November 2008 5.CS-23 project overview and achievements presentation Regional, Zonal and District Health offices review meetings June 2009, March 2010 6.Dissemination workshops on KPC and HFA baseline findings RHB, Sidama and Siliti ZHD, Lanfero and Shebedino DHO August 2009 7.Experience of operational zinc treatment in CS-23 project GOAL CS MTE workshop September 2009 8.Community Case Management Improves Use of Treatment for Childhood Diarrhea, Malaria and Pneumonia in a Remote District of Ethiopia International Multilateral Initiative on Malaria symposium November 2-6, 2009 in Nairobi Kenya Nov 2009 9.CS-23 project overview and achievements presentation Africa Regional Pan-Africa Every One Campaign Workshop, February 18-20, 2010 Addis Ababa Feb 2010 10. CS-23 project overview and achievements presentation National Orientation and launching workshop on CCM: Nazareth, Ethiopia. Donors, implementing partners and RHBs Feb 2010 11. Presentation on CCM esp. of Pneumonia Regional level workshops and CS task force April 2010 12. Evidence, Advocacy, and Partnerships for Community Case Management of Childhood Infection in Ethiopia: The End of the Beginning Save the Children Program Learning Group Norwalk, CT, USA. June 2010 13. CS-23 project overview and achievements presentation for debriefing workshop of the final evaluation MoH partners from Region, Zone and Districts, Bureau of Finance partners, UNICEF, Plan Ethiopia JSI/IFHP, Engender Health, Malaria Consortium August 2012 14.The final KPC and HFA findings presentation for the debriefing workshop MoH partners from Region, Zone and Districts, Bureau of Finance partners, UNICEF, Plan Ethiopia, JSI/IFHP, Engender Health, Malaria Consortium August 2012 List of Reports/Publications 1.Formative research on the HEP supervision (unpublished report) September 2009 CS-23 Ethiopia, Final Evaluation, Save the Children, December 2012 50 2.Degefie T, Marsh D, Gebremariam A, Tefera W, Osborn G, Waltensperger K. Community Case Management Improves Use of Treatment for Childhood Diarrhea, Malaria and Pneumonia in a Remote District of Ethiopia Ethiop. J. Health Dev. 2009;23(2) Peer-review publication in Ethiopian Journal of Health and Development from previous project to advocate for inclusion of pneumonia treatment 2009 3.Quarter and annual activity and financial reports to government partners RHB, Sidama and Siliti ZHD, Lanfero and Shebedino DHO and to Regional Bureau of Finance Jan, Apr, Jul and Oct 2008- 12 4.KPC and HFA baseline reports to key partners RHB, Sidama and Siliti ZHD, Lanfero and Shebedino DHO August 2009 5.Integrated/Joint supervision reports Sidama and Siliti ZHD, Lanfero and Shebedino DHO Aug, Oct, Mar 2009- 2012 6.Training reports in IMNCI RHB, Sidama and Siliti ZHD, Lanfero and Shebedino DHO July 2009 7.Training report in C-IMNCI RHB, Sidama and Siliti ZHD, Lanfero and Shebedino DHO Feb and Nov 2009 8.AWD outbreak reports Lanfero RHB, Siliti ZHD and Lanfero DHO Dec 2009 9.Mid Term Evaluation Report Submited to USAID and all relevant partners Oct 2010 10.iCCM training report for HEWs RHB, Sidama and Shebedino DHO Dec 2010 11.Experience in Zinc treatment and follow up in Lanfero and Shebedino Brief report document July 2012 12.Mobile phone profile of HEWs in iCCM implementation districts and its potential use for mHealth Brief analysis and report in two iCCM implementation zones July 2011 13.KPC and HFA final evaluation reports to key partners RHB, Sidama and Siliti ZHD, Lanfero and Shebedino DHO September 2012 List of Learning Briefs for FE 1.How Do Health Extension Workers Spend Their Time? David Marsh, Peter Waiswa, Hailu Tesfaye, Worku Tefera, et al. Learning debrief document for CS-23 FE report August 2012 2.The Status of Ethiopia’s Integrated Community Case Management Strategy and the Contribution of USAID’s Child Survival and Health Grant’s “CS-23” Project to Save the Children Hailu Tesfaye, Tedbabe Degefie, Karen Waltensperger, David Marshet al. Learning debrief document for CS-23 FE report August 2012 3.CCM Indicators: Feasibility, Validity, Usefulness, and Sustainability Worku Tefera, Hailu Tesfaye, Karen Waltensperger, David Marsh et al. Learning debrief document for CS-23 FE report August 2012 4.The Potential of mHealth to Support Health Extension Worker Supervision in Ethiopia’s Southern Nations, Nationalities and Peoples Region Learning debrief document for CS-23 FE report August 2012 CS-23 Ethiopia, Final Evaluation, Save the Children, December 2012 51 Hailu Tesfaye, Rita Nemero, David Marsh, et al. 5.Multi-factorial Causes for Low Utilization in Shebedino vs. Lanfero District Hailu Tesfaye, Worku Tefera, Abeba Bekele, Karen Waltensperger, David Marsh et al. Learning debrief document for CS-23 FE report May 2011 6.Saving Lives and Informing the Introduction and Scale-up of a National Integrated Community Case Management Strategy for Ethiopia Hailu Tesfaye, Karen Waltensperger, David Marsh et al. Learning debrief document for CS-23 FE report August 2012 7. How Does USAID’s Child Survival and Health Grant’s “CS-23” Project to Save the Children contribute to community case management of pneumonia policy change and iCCM scale up in Ethiopia? Hailu Tesfaye, Abeba Bekele, David Marsh, Karen Waltensperger, Rita Nemero, et al. Learning debrief document for CS-23 FE report August 2012 8. Mother to Mother Care Group, Pregnant Mothers’ Forum and Increased Institutional Delivery in Lanfero District, Ethiopia Hailu Tesfaye, Getenet Kebede, Habtamu Tilahun, Chiksa Sultan et al. Learning debrief document for CS-23 FE report August 2012 CS-23 Ethiopia, Final Evaluation, Save the Children, December 2012 52 Annex 3: Project Management Evaluation This annex discusses: 1) Planning and Implementation; 2) Human Resources and Staff Management; 3) Financial Management; 4) Logistics, Technical and Administrative Support; and 5) Transition to Save the Children International. 1. Planning and Implementation The final evaluation team included the woreda Health Officers from Lanfero and Shebedino, as well as a representative from Sidamo Zonal Health, attesting to Save the Children’s strong partnership approach. Review of project documentation and interviews with government partners and other key stakeholders confirm that the project’s planning process has been inclusive since inception, engaging at the national, regional, zonal, and woreda (district) levels. Development of the original application and DIP, the MTE, and final evaluation were all participatory. The project’s DIP and annual work plans have been coordinated with district planning and serve to guide implementation in the two districts. The priority recommendations of the final evaluation were drafted by the government partners themselves, as it is they who will take the interventions forward. In addition to the above, Save the Children participates on the SNNPR Regional Child Survival Task Force and serves on several technical working groups at the national level. UNICEF/Ethiopia Head of Health and Nutrition, Luwei Pearson, also joined the final evaluation team in the field for several days; and Martine Le Fûr, who heads up the UNICEF field office in SNNPR, joined the team at several critical points of synthesis and discussion. 2. Human Resources and Staff Management There were no changes in key personnel during the term of the cooperative agreement. As detailed in the MTE report, the project added two field positions for Community Mobilization Officers (CMOs) in early 2010. The aim was to strengthen coordination with local community leadership and provide follow-up of the vCHWs trained in c-IMNCI. The presence of CMOs in the field has helped to facilitate the transition over the past year to Ethiopia’s new Health Development Army (HDA) strategy, which has incorporated most community health volunteers, as well as to pilot the Care/Mother2Mother Groups and Elders Groups in select kebeles. Shortly after the MTE, Dr. Abeba Bekele assumed leadership of the EtCO’s health programs. The Deputy Project Manager was also replaced, the position assumed by Getenet Kebede. There have also been some turnovers noted in field-based officers. These changes do not appear to have disrupted program operations, and it is evident that the team functions well. According to the Save the Children Country Director Ned Olney, most of the project staff will be incorporated into other current field-based programs or as opportunity arises. 3. Financial Management Save the Children’s financial management system, as detailed in the DIP, appears to have been adequate and accountable at headquarters, country, and field levels. Close-out of the cooperative agreement is now taking place, and Save the Children believes all funds will be expended per plan. CS-23 Ethiopia, Final Evaluation, Save the Children, December 2012 53 4. Logistics As reported at midterm, a delay in raising private funds for procurement of the project vehicle was a major challenge. This delay lasted nearly a year until Save the Children’s Survive to Five Campaign (now “Every Beat Counts”) agreed to raise the necessary funds for the vehicle, as well as to cover procurement of zinc and other essential CCM drugs to fill supply gaps. Save the Children was able to fund the CCM drug supply until the second quarter of FY12. With support from the JSI Supply Chain for CCM (SC4CCM) and UNICEF, the government is now rolling out CCM semi-annual “kits” for the HSAs. During the initiation period, some disruption and stock-outs have been noted and are described elsewhere in this report. 5. Technical and Administration Support As at midterm, technical and administrative support were found to have been satisfactory to meet project needs. Dr. David Marsh, Save the Children’s Senior Child Survival Advisor and CCM Team Leader, has provided consistent project backstopping, Additional programmatic support has been provided by Karen Z. Waltensperger, Senior Advisor, Health-Africa, based in South Africa, and by Save the Children staff at home office who assist with finance, documentation, and coordination. 6. Transition to Save the Children International The transition of the EtCO to Save the Children International (SCI) by all six current Save the Children members in Ethiopia is proceeding with minimal disruption. Ethiopia is scheduled to “go live” on 1 October 2012. CS-23 Ethiopia, Final Evaluation, Save the Children, December 2012 54 Annex 4. Work Plan Table MAJOR ACTIVITIES OBJECTIVE MET ACTIVITY STATUS Intermediate Result 1. Access and availability of services and supplies increased YES Activity 1. Train health workers including Health Extension Workers in IMNCI Completed 109 HEWs and 58 health workers trained in IMNCI Activity 2. Provide standard IMNCI algorithms to assess, classify and treat symptoms of diarrhea, malaria and pneumonia for health centers and health posts Completed All HCs and HPs equipped in 2009 with IMNCI job aids Activity 3. Train community health promoters in C￾IMNCI Completed 1080 vCHWs trained c-IMNCI Activity 4. Facilitate prompt referral of sick children from community to health post and severe cases from HP to health center Completed Referral system is in place from community to health post and from health post to health center Activity 5. Strengthen prompt and effective assessment and appropriate treatment of diarrhea and malaria by trained HEWs Completed Activity 6. Strengthen prompt and effective assessment and referral of pneumonia by trained HEWs Completed New FFMOH permits the treatment of pneumonia with antibiotics by HEWs; HEWs treat pneumonia Activity 7. Provide/facilitate for health centers and health posts with essential IMNCI drugs to treat diarrhea and malaria Completed IMNCI drugs procured and distributed; Stock shortages and financial resource constraints threaten on￾going support Activity 8. Provide/facilitate availability of first line antibiotic to treat pneumonia at health centers Completed Activity 9. Ensure adequate supply of antimalarial and new formula ORS at health center and health post level Completed Activity 10. Ensure adequate zinc supply/stock at health post (HP) and health center (HC) Completed Activity 11. Start zinc treatment for diarrhea at HC and HP Completed Activity 12. Advocate at regional and national level through established child survival groups and UN organization to start CCM/pneumonia by trained HEWs Completed Policy change in 2010 to include CCM/P by HEWs; CCM/pneumonia is treated by trained HEWs Activity 13. Ensure adequate supply/stock of first line antibiotic for pneumonia at health posts Completed FMOH policy changed and allowed HEWs to treat pneumonia; started in 2010 Activity 14. Start community case management of pneumonia at health posts by trained HEWs Completed Activity 15. Follow up for adequate supply and distribution of ITN at HC and HP Completed Support for ITN distribution in transportation; LL ITNs primarily supplied through campaigns Activity 16. Follow up for adequate supply and stock of childhood vaccines at HC and HP Completed Supportive supervision; supply is by FMOH Activity 17. Promote routine and outreach immunization Completed Done by HEWs/vCHWs;support from FMOH partners and CS-23 CS-23 Ethiopia, Final Evaluation, Save the Children, December 2012 55 MAJOR ACTIVITIES OBJECTIVE MET ACTIVITY STATUS Activity 18.Avail a trained HEW and a health professional in essential new born care and assessment of sick new born Completed Activity 19. Avail a trained HEW and health professional in safe delivery, newborn care, assessment, resuscitation and postnatal care Completed HEWs in both child survival project districts trained on safe and clean delivery Activity 20. Strengthen the link between TBAs and HEWs in follow up of deliveries, newborns to provide essential newborn care and postnatal care Completed HEWs liaising with TBAs well for follow-up; TBAs educate mothers on the need of institutional delivery Activity 21. Strengthen the referral link of sick newborns to health centers for early and prompt management Partial Few sick newborns seeking care at either HPs or HCs Activity 22. Follow up for availability of safe delivery kit and newborn resuscitation equipment at health post and health centers Partial Supplied by UNICEF; safe and clean delivery kits are available Activity 23. Support TT immunization and availability at health post Completed Through supportive supervision Activity 24. Avail standard Job aids (IMNCI reference materials, wall charts, teaching aids, IEC materials) in child health Completed All standard IMNCI job aids and registers distributed and Activity 25. Avail standard registers and reporting available formats Completed Activity 26. Monitor and follow up for essential drugs and supply and facilitate corrective actions Completed Through supervision and coordination Intermediate Result 2. Quality of services increased Activity 1. Train health professionals in IMNCI case management skill (how to assess, classify, treat and counsel) pneumonia, malaria and diarrhea, newborn and sick young infant Completed 58 health workers trained in IMNCI Activity 2. Train HEWs in IMNCI case management and referral skill (assess, classify, treat, counsel) of pneumonia, diarrhea, malaria, newborn/sick young infant Completed 109 HEWs trained in IMNCI; HEWs trained in pneumonia or sick newborn mgmt after government policy change on pneumonia treatment Activity 3. Facilitate/provide with IMNCI essential drugs and supplies for health centers and health posts. Completed Activity 4. Follow up for appropriate drug treatment at HP and HC level Completed Activity 5. Follow up for proper counseling and follow up at health post and HC Completed Provided with IEC job aids Activity 6. Facilitate rehydration therapy with the new ORS formula Completed ORT corners set-up and ORS formula available Activity 7. Facilitate zinc treatment for diarrhea Completed In coordination with PSI; after ICCM scale up zinc is provided by UNICEF Activity 8. Give regular on the job trainings and technical assistance in IMNCI implementation at health post and health centers Completed During joint supervisions and visits by SC staff Activity 9.Train HEWs in assessment of sick newborn, in essential newborn care messages (TT immunization, Completed This was provided to HEWs as part of the IMNCI/ICCM CS-23 Ethiopia, Final Evaluation, Save the Children, December 2012 56 MAJOR ACTIVITIES OBJECTIVE MET ACTIVITY STATUS cord care, thermal management and recognition of newborn danger signs ) training Activity 10. Support/build capacity of health workers and District Health Office staff in proper supervision and routine monitoring, in sustaining facility and community level activities Completed Regular joint supervision and review meeting occurring Activity 11. Review quality improvement options with partners in delivery of MCH services and design OR protocol Completed Operations Research (OR) conducted Activity 12. Review regional data collection/HMIS in standard documentation and reporting to make them user friendly Completed FMOH developed standard HMIS; SC data collection formats coincide with FMOH HMIS Activity 13. Conduct baseline rapid health facility assessment survey Completed Activity 14. Strengthen existing supportive supervision for HEWs jointly with district health office health center staff Completed Joint supervision for IMNCI conducted throughout project Activity 15. Facilitate use/adoption of standard supervision checklists inclusive of curative services and counseling services Completed Supervision checklists developed and in use Activity 16. Avail Standard job aids for reference and documentation (registers, reference job aids for key messages delivery, IMNCI reference materials/algorithm) Completed Supplies distributed and available at HCs and HPs Activity 17. Conduct joint review meetings and feed backs on performance on regular basis (recognize best performances) Completed Conducted regularly throughout project Activity 18. Ensure use of standard reporting formats and registers Completed Standard reporting formats registers are in use Activity 19. Strengthen the link between HEWs and TBAs/TTBAs in essential newborn care and post natal care Completed Links established Activity 20. Support and facilitate child immunization Completed Promotion on immunization Activity 21. Promote on early treatment of sick child for fever, diarrhea and pneumonia Completed Promotional activity by vCHWs Activity 22. Planning, design monitoring and evaluation and KPC training for child survival M&E Officer Completed Activity 23. Facilitate annual technical updates with professional associations on child survival Completed Membership, participation and presentations at national, regional and district child survival task forces Activity 24. Annual progress review and planning meeting Completed Close relationship with RHB and local health authorities Activity 25. Performance progress monitoring survey Completed Activity 26. Participatory midterm evaluation led by external consultant Completed Activity 27. Review midterm assessment and MTE findings and recommendations with FMOH and partners, prioritize and schedule actions to address recommendations, and plan for required actions Completed MTE was participatory, with results and recommendations workshop CS-23 Ethiopia, Final Evaluation, Save the Children, December 2012 57 MAJOR ACTIVITIES OBJECTIVE MET ACTIVITY STATUS Activity 28. Conduct end line rapid health facility assessment Completed Final Evaluation was participatory, with results and recommendation workshop Activity 29. Conduct endline KPC survey Activity 30. Final evaluation led by external consultant Intermediate Result 3. Knowledge and acceptance of key services and behaviors increased YES Activity 1. Train HEWs and VCHWs in delivery of key messages in child health, nutrition, care seeking behaviors/practices, child/maternal/newborn danger signs, essential newborn care, postnatal care, hygiene and sanitation Completed Activity 2. Adopt/develop education materials/teaching aids for key messages in child health, in appropriate behaviors and practices Completed Counseling cards, leaflets and booklets distributed Activity 3. Conduct community leaders sensitization workshop Completed Activity 4. Provide health promotion activities in health facilities in child health, nutrition, care seeking, hygiene and sanitation during one to one sessions or during health education Completed Health Education sessions; using IEC materials Activity 5. Deliver key behaviors and practices in appropriate care seeking for ill child, in recognition of signs needing proper treatment, in recognition of danger signs through trained HEWs and VCHWs at household level Completed Activity 6. Promote on appropriate hand washing practices Completed Promotional activity by vCHWs Activity 7. Promote standard immunization services during health facility visits Completed Activity 8. Promote on proper oral rehydration at health facilities and home during diarrhea Completed All health facilities have ORT corner Activity 9. Counsel/advise caretakers on proper feeding and fluid during diarrhea episodes Completed Activity 10. Counsel caretakers in proper breast feeding, proper feeding for infant and young child and feeding during illness Completed Activity 11. Counsel caretakers in one to one and in groups about household sanitation and hygiene (proper hand washing, safe waste disposal and safe water storage/treatment) using trained HEWs and VCHWs Completed Activity 12. Inform community on ITN availability and proper utilization by children and pregnant women Completed Activity 13. Ensure caretakers understanding of importance of referral and follow up of sick child Completed Intermediate Result 4. Social and policy environment enabled and sustainability of all activities improved YES Activity 1. Develop project agreement with Regional Health Bureau (RHB) and key stakeholders Completed Activity 2. Collaborate with RHB and other partners on Completed CS-23 Ethiopia, Final Evaluation, Save the Children, December 2012 58 MAJOR ACTIVITIES OBJECTIVE MET ACTIVITY STATUS child survival working groups at regional and national level Activity 3. Work with GoE/NGO/UN partners for policy improvement in HSDP-IV, esp. CCM of pneumonia by HEWs Completed Activity 4. Document and disseminate evidence based best practices in CCM using FMOH guidelines and documents Completed 17 CCM best-practices disseminated Activity 5. Conduct joint planning with relevant and key partners and community stakeholders in CCM (develop detailed implementation plan) Completed Activity 6. Lead regular partners coordination, advocacy and policy dialogue in CCM and evidence based new born and child practice Completed Activity 7. Adopt proven child health interventions and strategies in to regional and national policies and programs Completed Activity 8. Conduct joint and integrated supportive supervisions and TAs Completed Activity 9. Follow up of health facilities functionality in IMNCI implementation and reporting Completed All HCs and HPs implement IMNCI and report Activity 10.Facilitate proper health service delivery by health posts in collaboration with FMOH and partners Completed Activity 11. Support and facilitate standard documentation and regular reporting by health posts and health centers Completed Documentation and reporting is implemented both at HCs and HPs Activity 12. Standardize the referral link between HEWs and TBAs/VCHWs in safe and clean delivery and postnatal visit Completed HEWs trained in safe and clean delivery, links between TBAs/vCHW and HEWs Activity 13. Participate and advocate through Save the Children Health and Nutrition (PR3) Program Learning Group Completed Activity 14. Conduct first regional dissemination workshop (Awassa) Completed Activity 15. Final dissemination workshop Completed Participants were government offices and Save the Children partner non-government organizations. Activity 16. Participate in periodic regional child survival taskforce meetings Completed Monthly meeting Activity 17. Document and share child survival interventions and updates in regional review meetings Completed Reports shared regularly with RHB, ZHD and DHO Activity 18. Support and participate in national child survival taskforce to advocate for policy change in CCM Completed Policy changed to treat pneumonia by HEWs Activity 19. Establish regional TAG (Technical Advisory Group with representation from RHB, Regional Child Survival Coordinator, Hawassa University, UNICEF, WHO, ESHE, Malaria Consortium) Completed Activity 20. Conduct regional TAG meeting Completed CS-23 Ethiopia, Final Evaluation, Save the Children, December 2012 59 MAJOR ACTIVITIES OBJECTIVE MET ACTIVITY STATUS Activity 21. Establish district level child survival team (FMOH, SC, others working on CS) Completed Established in 2009; meet on monthly basis and review performance progress Activity 22.Document of the CCM/P experience to inform regional level scale up Completed Documentation/presentations of experiences in Liben (CS-17); CCM/P scaled up Activity 23. Advocacy and policy dialogue to promote uptake at the regional level Completed CS-23 Ethiopia, Final Evaluation, Save the Children, December 2012 60 Annex 5: Rapid CATCH Table1 CSHGP Intervention Area Rapid CATCH Indicator Baseline Estimate MTE Estimate* Final Estimate Maternal Newborn Care (2) Maternal TT Vaccination: Percentage of mothers with children age 0-23 months who received at least two tetanus toxoid vaccinations before the birth of their youngest child. 91.7% N/A 93.5% (2) Skilled Delivery Assistance: Percentage of children age 0-23 months whose births were attended by skilled personnel. 1.4% N/A 14.8% (3) Post-natal visit to check on newborn within the first 3 days after birth: Percentage of children age 0-23 months who received a post￾natal visit from an appropriate trained health worker within three days after the birth of the youngest child. 3.5% N/A 14.4% Breastfeeding (4) Exclusive Breastfeeding: Percentage of children age 0-5 months who were exclusively breastfed during the last 24 hours. 2.5% N/A 28.8% Nutrition (5) Infant and Young Child Feeding: Percent of infants and young children age 6-23 months fed according to a minimum of appropriate feeding practices. 45.3% N/A 60.6% Vitamin A (6) Vitamin A Supplementation in the last 6 months: Percentage of children age 6-23 months who received a dose of Vitamin A in the last 6 months: card verified or mother’s recall. 60.1% N/A 91.1% Immunization (9) Measles vaccination: Percentage of children age 12-23 months who received a measles vaccination. 59.7% N/A 83.6% (7) Access to immunization services: Percentage of children aged 12-23 months who received DTP1 according to the vaccination card or mother’s recall by the time of the survey. 79.9% N/A 96.9% (8) Health System Performance regarding Immunization Services: Percentage of children aged 12-23 months who 46.7% N/A 71.4% 1 NOTE: THESE VALUES REPRESENT WEIGHTED AVERAGES FOR THE TWO DISTRICTS TOGETHER. Rapid CATCH data broken out for each of the two districts can be found in Annex 13, Project Data Form. CS-23 Ethiopia, Final Evaluation, Save the Children, December 2012 61 CSHGP Intervention Area Rapid CATCH Indicator Baseline Estimate MTE Estimate* Final Estimate received DTP3 according to the vaccination card or mother’s recall by the time of the survey. Malaria (11) Treatment of Fever in Malarious Zones Percentage of children age 0-23 months with a febrile episode during the last two weeks who were treated with an effective anti-malarial drug within 24 hours after the fever began. 33.2% N/A 65.2% Control of Diarrheal Diseases (12) ORT Use: Percentage of children age 0- 23 months with diarrhea in the last two weeks who received oral rehydration solution (ORS) and/or recommended home fluids. 37.1% N/A 54.8% Pneumonia Case Management (13) Appropriate Care Seeking for Pneumonia: Percentage of children age 0-23 months with chest-related cough and fast and/or difficult breathing in the last two weeks who were taken to an appropriate health provider. 31.9% N/A 44.8% Control of Diarrheal Diseases (14) Point of Use (POU): Percentage of households of children age 0-23 months that treat water effectively. 13% N/A 31.6% (15) Appropriate Hand Washing Practices: Percentage of mothers of children age 0-23 months who live in households with soap at the place for hand washing. 28.2% N/A 60.0% Malaria (10) Child sleeps under an insecticide-treated bednet: Percentage of children age 0-23 months who slept under an insecticide-treated bednet (in malaria risk areas, where bednet use is effective) the previous night. 39.9% N/A 39.3% Nutrition (17) Underweight: Percentage of children 0-23 months who are underweight (-2 SD for the median weight for age, according to WHO/NCHS reference population). 18.8% N/A 15.4% * No data collected at midterm.    CS-23 Ethiopia, Final Evaluation, Save the Children, December 2012 62 Annex 6: Final KPC Report Caregiver Knowledge and Reported Practice Regarding Life-Saving Child Survival Interventions – Results of Two Population-Based Household Surveys in Shebedino and Lanfero Districts, SNNPR, Ethiopia March/April 2012 This report is made possible by the generous support of the American people through the United States Agency for International Development USAID). The contents are the responsibility of Save the Children and do not necessarily reflect the views of USAID or the United States Government. CS-23 Ethiopia, Final Evaluation, Save the Children, December 2012 63 Table of Contents Page A. Summary ............................................................................................................................ 65 B. Objectives of the endline KPC Survey .............................................................................. 67 C. Methods.............................................................................................................................. 67 D. Results ................................................................................................................................ 71 E.      Discussion .......................................................................................................................... 86 ANNEXES .................................................................................................................................... 92 Annex 1: Questionnaire ............................................................................................................ 93 Annex 2: Tabulation Plan ....................................................................................................... 115 Annex 3: Schedule – Lanfero ................................................................................................. 128 Annex 4: Schedule – Shebedino ............................................................................................. 129 CS-23 Ethiopia, Final Evaluation, Save the Children, December 2012 64 Acronyms ARI Acute Respiratory Tract Infection CATCH Core Assessment Tool for Child Health CS Child Survival CS-23 Child Survival-23 CSTS Child Survival Technical Support D Denominator DIP DPT Detailed Implementation Plan Diphtheria, Pertussis, Tetanus EPI Expanded Program on Immunization HEW Health Extension Worker HF HP ICCM IMNCH IPT Health Facility Health Post Integrated Community Case Management Integrated Management of Newborn and Child Health Intermittent Prophylactic Treatment ITN Insecticide-treated bed-nets KPC Knowledge, Practice and Coverage MAMAN Minimum Activities for Mothers and Newborns N Numerator ORS Oral Re-hydration Solution ORT Oral Re-hydration Therapy PPS Probability Proportional to Sample Size RHB Regional Health Bureau SC Save the Children SD Standard Deviation SNNPR SPSS Southern Nations, Nationalities, and People’s Region Statistical Package for Social Sciences TBA Traditional Birth Attendant TT Tetanus Toxoid WAZ Weight-for Age-Z Score WHO Woreda Health Office WHO/NCHS World Health Organization/National Center for Health Statistics ZHD Zonal Health Desk CS-23 Ethiopia, Final Evaluation, Save the Children, December 2012 65 A. Summary Save the Children completed two consecutive, population-based, (n=300 each), randomized cluster, district-wide, household surveys in Shebedino and Lanfero Districts in Ethiopia’s Southern Nations, Nationalities, and People’s Region (SNNPR) in March and April 2012 (vs. in April 2008 for the baseline). The purpose of the household survey was to determine accomplished results versus the baseline results and project targets. The samples were similar; but more children in Shebedino were below the age of six months while in Lanfero most were between 12-23 months old. Lanfero’s mothers were less likely to have had schooling. Families had good access to health facilities (HFs) in both districts although these were almost always health posts (HPs), which provided limited services during limited hours of operation (25% effort in HP). Currently, all HPs provide treatment for pneumonia. Most families (94%) owned land, but usually small amounts. The Shebedino District Agricultural and Development Office reported that land ownership ranking was as follows: Poor: 0-¼ hectare, medium: ¼-2 hectare, rich: 2-4 hectare, and very rich: ≥5 hectare. Accordingly the weighted average land ownership for both districts was poor for 9%, medium 32%, rich 21%, and very rich 32%. The weighted average ownership of household appliance for both districts was; radio 32%, tape recorder 24%, and television 6%. Most households do not have their own means of transportation and transportation ownership was less than 7%. Maternal and Newborn Care All indicators have improved from the baseline results except knowledge of newborn danger signs. TT2 coverage (baseline 92% vs. 94%), delivery at health facility (HF) (baseline: 2% vs. 26%), skilled birth attendant (baseline: 1% vs. 15%), postpartum maternal visit within three days of delivery (baseline: 2% vs. 27%), knowledge of two or more postpartum maternal danger signs (baseline: 8% vs. 17%), and postnatal newborn visit within three days of delivery (baseline: 4% vs. 14%). On the other hand, knowledge of two or more newborn danger signs was below the baseline (baseline: 29% vs. 28%). Essential Newborn Care Nearly three-quarters of deliveries (75%) occurred at home. More than two-thirds (68%) of birth attendants were relatives, friends, traditional birth attendants (TBAs), trained community health workers, untrained community health workers or the mother herself. Most essential newborn care practices were poor in both districts compared to the baseline results: clean cord cut (baseline: 98% vs. 69%), immediate drying and wrapping (baseline: 82% vs. 67%), and infant placement with mother before the placenta delivered (baseline: 62% vs. 42%). Immediate breastfeeding and colostrum given to newborns have improved in both districts from the baseline; immediate breastfeeding (baseline: 62% vs. 93%); colostrum given to newborns (baseline: 82% vs. 94%). Infant and Young Child Nutrition Exclusive breastfeeding practice improved from the baseline as follows: infants <6 months (baseline: 3% vs. 29%) and <4 months (baseline: 11% vs. 35%). Complementary feeding practice improved in both districts from baseline, and was better in Shebedino than Lanfero; infants 6-8 months (baseline: 30% vs. 57%) and 9-23 months (baseline: 42% vs. 64%). Vitamin A supplementation coverage improved from baseline in both districts (60% vs. 91%), and Shebedino was better than Lanfero. CS-23 Ethiopia, Final Evaluation, Save the Children, December 2012 66 Immunizations Access to and coverage of, EPI services has improved from the baseline (Penta 1 baseline: 80% vs. 97%); and (Penta 3 baseline: 47% vs. 71%). Morbidity The proportion of sick children with fever and cough/fast or difficult breathing two weeks prior to the survey has decreased from the baseline, while those with diarrhea did not show an improvement. However, in Lanfero the proportion of sick children with fever, diarrhea and cough/fast or difficult breathing has increased from the baseline. Lanfero is a more food insecure and malarious district than Shebedino. The overall proportion of reported sick children has declined (baseline: 53% vs. 43%), with fever (baseline: 35% vs. 24%), diarrhea (baseline: 29% vs. 31%), and cough/fast or difficult breathing (baseline: 25% vs. 12%). Malaria Ownership of Insecticide-Treated Bed Nets (ITNs) in both districts declined from the baseline (baseline: 55% vs. 47%). Most (85%) ITNs were used by children the previous night (baseline 73%) while overall usage of ITN by children was almost the same as the baseline result (baseline: 40% vs. 39%). Careseeking for fever has improved from the baseline (baseline: 47% vs. 71%); two-thirds of those seeking care (65%) received an antimalarial than at baseline (33%). Coartem is the main antimalarial used. ARI Care seeking for cough and difficult or fast breathing has improved from the baseline (baseline: 32% vs. 45%), with the overall careseeking behavior higher in Lanfero than Shebedino. Those who sought care received an antibiotic, but less than at baseline (89% vs. 44%). The majority of people sought care for ARI from HCs (77%), but a few (4%) sought care from a pharmacy in Shebedino District and from a hospital (1%) in Lanfero District. Three quarters (75%) of caregivers knew signs of pneumonia (baseline: 49% vs. 76%); two or more childhood danger signs (baseline: 51% vs. 74%); and more than half knew three or more childhood danger signs (baseline: 15% vs. 52%). Diarrhea Preventive practices also improved (hand-washing [baseline: 28% vs. 60%] and water treatment [baseline: 13% vs. 32%]). ORT use for diarrhea declined from the baseline (baseline: 57% vs. 55%), but improved in Lanfero. Additionally increased food intake during illness/diarrhea declined from the baseline (baseline: 29% vs. 25%) while increased fluid intake during diarrhea improved (baseline: 20% vs. 59%). Zinc treatment for diarrhea has increased from baseline (baseline 7% vs. 34%). Nutritional Status The proportion of moderately underweight children (weight for age [WFA] < -2 z-score) decreased from the baseline (baseline: 19% vs. 8%); while severely underweight (WFA<-3 z-score) has increased (baseline: 3% vs. 8%]). This is better than the Ethiopia DHS 2011 household survey report (moderate: [29%]; severe: [9%]). There were slightly more moderately underweight children in Shebedino than Lanfero, but more children were severely underweight in Lanfero than their counterparts in Shebedino. Lanfero is a food insecure district. In both districts more boys were either moderately or severely malnourished than girls. Mean weight-for-age z-score was worse for older children (aged 0-5 and 18-23 months of age: -0.05 and -1.1 in Shebedino, and -0.33 and -1.77 in Lanfero). In general, most of the health and nutritional status indicators have improved from baseline in both districts except for those related to essential newborn care practices (cord cutting, drying CS-23 Ethiopia, Final Evaluation, Save the Children, December 2012 67 and wrapping, and immediate placement on mothers’ abdomen), ownership of ITN, ORT use, increased food and fluid intake for diarrhea, and caretakers’ knowledge of two or more neonatal danger signs. EPI access and coverage has shown improvement. The illness prevalence has been particularly reduced for fever and cough/fast or difficult breathing. Care seeking behavior has also shown improvement, while treatment practices for ARI and diarrhea need to be strengthened. B. Objectives of the endline KPC Survey The main purpose of the endline KPC survey was to determine results achieved and compare them with baseline results and targets. As set in the Detailed Implementation Plan (DIP), we have conducted the endline KPC survey using the same tools and methodology to measure changes in key indicators (including Rapid CATCH). The endline KPC survey was conducted in each district of the CS-23 project in order to compare results. C. Methods Questionnaire Design For the endline survey, we used the same questionnaire used for the baseline survey. We adapted the baseline questionnaire from KPC 2000+ modules, Revised Rapid CATCH (December 16, 2007), Minimum Activities for Mothers and Newborns (MAMAN), and Ethiopia Demographic and Health Survey – 2005 (equity questions). We developed a tabulation plan, and interviewer instructions to match the questionnaire. The questionnaire has 12 modules: (1) Introduction; (2) Maternal and Newborn Care; (3) Breastfeeding and Infant and Young Child Feeding; (4) Vitamin A Supplementation; (5) Child Immunizations; (6) Malaria–Treatment of Fever; (7) Malaria–ITN Use; (8) Control of Diarrhea; (9) ARI/Pneumonia; (10) Water and Sanitation; (11) Household Wealth; and (12) Anthropometrics. The questionnaire addresses the mother of the youngest child in the household as well as the child. We have translated the questionnaire into Amharic for use in both districts. Data collectors had samples of the following drugs to facilitate identification: ORS packet, vitamin A capsule, zinc tablet and Coartem. During the baseline survey, Save the Children’s CS-23 Manager, the Head of the Ethiopia Country Office’s Health and Nutrition Unit, the Africa Regional Health Advisor, and the Senior Child Survival Advisor from Save the Children’s Home Office reviewed the questionnaire. Experts from the Child Survival Technical Support (CSTS) team and MACRO International also provided input. Sample Size We used n=300 in each of our two district-wide surveys (n=600 in total), as generally recommended, and including the guidance for cluster surveys in the KPC 2000+ manual. We planned for 30 clusters of 10 respondents each, in each of the two districts (60 clusters of 10 in total). This sample size is adequate to estimate many indicators with satisfactory precision in a KPC survey. Sampling Strategy We used a two-stage random cluster sampling technique in each district. The first level of sampling was a random selection of 30 clusters, with a probability proportional CS-23 Ethiopia, Final Evaluation, Save the Children, December 2012 68 to sample size (PPS). The sampling unit was the district’s communities (called Peasant Associations or kebeles). DHOs provided the names and populations of each. We listed all communities with their populations. We identified the sampling interval by dividing the total population by the required number of clusters (30). We randomly selected the first kebele within this interval, and we selected subsequent kebeles by adding the sampling interval to the cumulative population of the previously selected kebeles until all clusters were selected. The second level of sampling was a selection of ten households from each cluster. We identified eligible households and collected data on the same day. If the mother was not at home, but was expected, data collectors waited 30 minutes before proceeding to the next eligible house. We made return visits to unoccupied homes, or homes with an eligible infant but whose mother was not at home during our first visit. We also returned to homes with an eligible mother whose child had not been at home during our first stop. The kebele chairman identified the center of his kebele. The first household for interviews was selected by first going to the center of the community. Since each kebele has sub-villages (ketenas), we started at the sub-village located at the center of the kebele, normally a community of several groups of households. A household was defined as people living together and sharing the same cooking pot. Each data collection team had a team leader responsible for sampling. We saw households with children <24 months of age, their mother present, and with a length of residence of at least three months (i.e., not visiting). The kebele chairman spun a bottle to determine the direction for seeking households and the team supervisor counted households in the selected direction and randomly selected one household as the first in that cluster. The team registered all households along the chosen ray to the edge of the ketena, assigning each one a serial number. The starting household was then chosen by lot. Regardless of whether the household was eligible (mother with child < 24 months of age) or not, the team continued to the third house to the right of the main entrance until the required sample size in that cluster was achieved. In cases where no eligible child was found in a selected household, the next nearest household with an eligible child was selected. The data collectors started near the center of the village and moved outwards, remaining in the same kebele. The selection of the third (rather than the adjacent) household was intended to reduce homogeneity (thereby the “design effect”) within a cluster, increasing the precision of findings. In eligible households, data collectors interviewed the mother of the youngest child <24 months of age, asked about that child, and weighed that child. Selection and Training of Data Collectors We wanted Ministry of Health partners to act as supervisors and data collectors in order to increase their capacity in survey methodology, to create ownership of results, to create awareness about the survey content and its role in planning and implementation, and to strengthen the Save the Children-MOH partnership. Therefore, DHOs selected ten data collectors and five supervisors with survey experience and health backgrounds (in each district). Each district had five teams of three members each (one supervisor and two data collectors). The teams included staff from DHOs, Health Workers, and individuals who had completed 12th grade and were trained in interview techniques and data collection. CS-23 Ethiopia, Final Evaluation, Save the Children, December 2012 69 Save the Children/US Child Survival staff (M&E Coordinator, Health Program Coordinator, and District Program Officers) trained the supervisors and data collectors for three days (in Shebedino [March 12-14, 2012] and in Lanfero [March 26-28, 2012]), including field practice. The training covered survey objectives, sampling, data collection and interviewing techniques to maintain data quality, consider the local events calendar, ethics, questionnaire review, weighing, and exercises. We used active learning during training, including: 1) two-way communication; 2) group-work with presentations; 3) general discussion; 4) exercises; and 5) evaluation of trainees’ experiences. The field practice in both districts was in kebeles which were not selected as clusters. Measurements The birth date of the child was recorded (day, month, and year) from the EPI card when available. If unavailable, the mother was helped to recall with a local calendar of events. The age of children was recorded in months. The child’s weight was recorded in kilograms. Children were weighed with a 25 kg Salter scale, precise to 100 grams. The accuracy of the scale was checked daily, and the scale was zeroed with the weighing pants prior to every measurement. Depending on the child’s age, we used pants or a triangulated cloth (for young infants). We insisted on weighing children who were completely undressed, but if their mother resisted this, we weighed the child with very light underwear. Field Methods The surveys were conducted consecutively, in Shebedino from March 15-20, 2012, and in Lanfero from March 29-April 3, 2012. Each team completed one cluster per day, i.e. each team completed ten questionnaires daily. The time to complete one questionnaire ranged from 30 to 40 minutes, depending on the understanding of the respondent. The data collection in each district took six days (50 questionnaires daily). We allocated clusters to teams by daily lottery to avoid the complaints from uneven workload distribution due to obvious differences in travel, time, and distance to reach various clusters. All members of the team were responsible for ensuring that data collected at each household was accurate and complete. Data collectors checked their questionnaires to see if they were clearly filled out before leaving the household, ensuring that all answers were clear and reasonable, and that their handwriting was legible. At the end of each day, data collectors again checked all completed questionnaires with their supervisor to ensure that all items were completed and skip patterns adhered to. The team supervisor re-checked questionnaires, and discrepancies were referred to the data collector for correction. Each evening all teams met to review problems and identify solutions. Supervision Each team’s supervisor ensured adherence to protocol and collection of quality data. In turn, Save the Children survey managers supervised the team supervisors and solved problems which arose during data collection. The survey managers randomly selected one or two data collection teams (its supervisor and two data collectors) to supervise daily. During the first three days of each survey, they reviewed every completed questionnaire for completeness. Data Handling, Entry, and Checking After assuring the completeness of all cluster questionnaires, teams forwarded them to the survey managers for safe-keeping until the survey was completed. Most responses were pre-coded. Responses to open-ended questions were CS-23 Ethiopia, Final Evaluation, Save the Children, December 2012 70 categorized and coded. A trained data-entry clerk double-entered the data into SPSS (version 17.0) using pre-designed templates. We used Epi-Nut to calculate anthropometric indicators. Ethics Mothers had the right to refuse to participate. We told them the purpose of the survey and assured them of the confidentiality of their responses. They were interviewed in private to assure frank discussion. Analytic Approach We analyzed each survey separately because each district’s team needed to know its district’s profile. We computed the weighted average of both surveys to yield project￾specific summary indicators. We displayed data in modules, showing numerators, denominators and percents for each district and for the project (weighted percents). Table A: Indicator Name and Definition Name Definition Underweight % of children 0-23 months who are underweight (-2 SD for the median weight for age, according to WHO/NCHS reference population) Appropriate hand washing practices % of mothers of children 0-23 months who live in a household with soap or a locally appropriate cleanser at the place for hand washing and who washed their hands with soap at least 2 of the appropriate times during the day or night before the interview Increased food intake during diarrheal episode % of children aged 0-23 months with diarrhea in the last two weeks who were offered the same amount or more food during the illness. (The indicator name seems a misnomer, but we kept it.) Increased fluid intake during diarrheal episode % of children 0-23 months with diarrhea in the last two weeks who were offered more fluids during the illness ORT use % of children age 0-23 months with diarrhea in the last two weeks who received ORS and/or recommended home fluids. Zinc % of children 0-23 months with diarrhea in the last two weeks who were treated with zinc supplements ITN used by mother during pregnancy % of mothers of children 0-23 months who slept under an insecticide treated bed net during their pregnancy with the youngest child (in malaria risk areas, where bed net use is effective) (DID NOT ASK) ITN use by child % of children age 0-23 months who slept under an insecticide-treated bed net (in malaria risk areas, where bed net use is effective) the previous night Intermittent presumptive treatment NOTE: IPT not recommended in Ethiopia. DID NOT ASK Skilled delivery assistance % of children age 0-23 months whose births were attended by skilled personnel Postpartum visit for mother % of mothers of children age 0-23 who received a postpartum visit from an appropriate trained health worker within three days after the birth of the youngest child Post-natal visit to check on newborn within first 3 days after birth % of children age 0-23 who received a post-natal visit from an appropriate trained health worker within three days after the birth of the youngest child Immediate breastfeeding of newborns % of newborns who were put to the breast within one hour of delivery (We did not measure prelacteal feeding practice, therefore cannot say “immediate, exclusive…”) CS-23 Ethiopia, Final Evaluation, Save the Children, December 2012 71 Name Definition Exclusive breastfeeding % of children age 0-5 months who were exclusively breastfed during the last 24 hours Vitamin A supplementation in the last 6 months % of children age 6-23 months who received a dose of Vitamin A in the last 6 months (Mother’s recall) Measles vaccination % of children age 12-23 months who received a measles vaccination, regardless of age Adequate child spacing % of children age 0-23 months who were born at least 24 months after the previous surviving child (DID NOT ASK) Maternal TT vaccination % of mothers with children age 0-23 months who received at least two Tetanus toxoid vaccinations before the birth of the youngest child Ownership of insecticide-treated bed net % of households of children 0-23 months that own at least one insecticide￾treated bed net (in malaria risk areas, where bed net use is effective). Access to immunization services % of children age 12-23 months who received a DPT1 vaccination before they reached 12 months Health system performance regarding immunization services % of children age12-23 months who received a DPT3 vaccination before they reached 12 months Use of medicine during diarrhea % of children 0-23 months with diarrhea in the last two weeks who were not treated with antidiarrheals or antibiotics. Child with fever receives appropriate antimalarial treatment % of children age 0-23 months with a febrile episode during the last two weeks who were treated with an effective anti-malarial drug within 24 hours after the fever began Point of Use % of households of children age 0-23 months that treat water effectively Appropriate treatment for fever % of children aged 0-23 months with a febrile episode during the last two weeks who were treated with an effective antimalarial drug within 24 hours of fever onset Appropriate care seeking for pneumonia % of children age 0-23 months with cough and trouble breathing or breathe faster than usual with short rapid or difficult breathing in the last two weeks who were taken to an appropriate health provider Pneumonia danger signs % of mothers report either fast or difficult breathing as a sign of child illness needing treatment Neonatal danger signs % of mothers report knowledge of at least two neonatal danger signs needing treatment Postpartum danger signs % of mothers who know at least two maternal danger sign during postpartum period Child danger signs % of mothers who know at least three signs of illness in children needing treatment D. Results The Sample The total sample (n=600) includes more boys than girls (52.8% vs. 47.2% overall) in each district (Table 1). Overall, half of the children (50%) were between the ages of 12-23 months, one third (32%) were between the age of 6-11 months and 18% were below the age of six months. There were more young infants (<6 months) in Shebedino than Lanfero (21% vs. 12% respectively) while there were more older children (12-23 months) in Lanfero than CS-23 Ethiopia, Final Evaluation, Save the Children, December 2012 72 Shebedino (57% vs. 46%, respectively). Maternal schooling was below half (44%), and was especially low in Lanfero. The grade achievement among those attending school was: 48% from grade 1-4; 35% from grade 5-8; and 17% from grade 9-12. Mothers’ education levels were higher in Shebedino than in Lanfero (grade 9-12: 23% vs. 7%, respectively). Access to HFs is uniformly good, especially considering that mothers almost always travel on foot (96.3% overall) and usually the facility is a HP which provides limited services during limited hours of operation. To determine year-round access, we used 5 km radius or ≤ 1 hour walk (accessibility standard for Ethiopia) to get IMNCI/ICCM case management services. Currently, IMNCI/ICCM service is available at health centers (HC) and HPs; MoH policy allowed HEWs to treat pneumonia cases at the HP level. Table 1: Sample Description Indicator Shebedino Lanfero Project N D % N D % N D W% Male Child 161 300 53.7 153 300 51.0 314 600 52.8 Female Child 139 300 46.3 147 300 49.0 286 600 47.2 Age of Child (in months) ● <6 63 300 21.0 37 300 12.3 100 600 18.1 ● 6-11 98 300 32.7 93 300 31.0 191 600 32.1 ● 12-23 139 300 46.3 170 300 56.7 309 600 49.8 Maternal Schooling (any) 164 300 54.7 71 300 23.7 235 600 44.1 Highest Grade Achieved ● Grade 1-4 66 164 40.2 45 71 63.4 111 235 48.1 ● Grade 5-8 61 164 37.2 22 71 31.0 83 235 35.1 ● Grade 9-12 37 164 22.6 4 71 5.6 41 235 16.8 Access to Health Facility ( <=1 hour) 292 300 97.3 283 300 94.3 575 600 96.3 Most families (93.6%) owned agricultural land, but the size of land holding was small (Table 2). Some families (6.4%) do not have agricultural land at all. Lanfero land ownership (very rich: ≥ 5 timad/hectar [local unit]) is better than Shebedino (69.3% vs. 12.7%, respectively). More than three-fourths of mothers (76.7%) reported owning livestock, more commonly in Lanfero than Shebedino District: (84.0% vs. 73.0%). Ownership of household appliances was low: radio (32.2%); tape recorder (23.5%); television (6%). Family ownership of a means of transportation was very poor, but if they want to travel to HFs or elsewhere they have to walk; ownership of a bicycle was 5.8%; ownership of a moped was 2.9%; and ownership of an animal drawn cart was 5%. Ownership of a car or truck is very low at 0.7%. CS-23 Ethiopia, Final Evaluation, Save the Children, December 2012 73 Table 2: Household Wealth Indicator Shebedino Lanfero Project N D % N D % N D W% Agricultural land 289 300 96.3 265 300 88.3 554 600 93.6 Poor (0-1/4 timad/hectare*) 39 300 13.0 2 300 0.7 41 600 8.8 Medium 1/4-2timad/hectare) 137 300 45.7 15 300 5.0 152 600 31.9 Rich (2-4 timad/hectare) 75 300 25.0 40 300 13.3 115 600 21.0 Very rich (>=5 timad/hectare) 38 300 12.7 208 300 69.3 246 600 31.9 Livestock 219 300 73.0 252 300 84.0 471 600 76.7 Household Appliance Radio 94 300 31.3 102 300 34.0 196 600 32.2 Tape recorder 80 300 26.7 52 300 17.3 132 600 23.5 Television 24 300 8.0 6 300 2.0 30 600 6.0 Means of transport Bicycle 15 300 5.0 22 300 7.3 37 600 5.8 Moped 11 300 3.7 4 300 1.3 15 600 2.9 Animal drawn cart 12 300 4.0 21 300 7.0 33 600 5.0 Car or Truck 3 300 1.0 0 300 0.0 3 600 0.7 * Source: Shebedino District Rural Development and Agricultural Office Maternal and Newborn Care Overall, the following have improved since baseline: TT coverage by card or maternal recall (baseline: 91.7% vs. 93.5%); delivery at a HF (baseline: 2% vs. 26%); delivery by skilled attendant (baseline: 1% vs. 14.8%); postpartum visits for mother (baseline: 2% vs. 27.2%) and postnatal visits for newborn (baseline: 4% vs. 14.4%). Mothers’ knowledge of two or more maternal danger signs has also increased, from 8% at baseline to 17%. Maternal knowledge of two or more newborn danger signs (baseline: 29% vs. 27.8%) did not change significantly. (Please see Table 3 below). Maternal knowledge of two or more newborn danger signs declined in Shebedino (baseline: 35% vs. 24.7%) while it increased in Lanfero (baseline: 17% vs. 34%). Mothers’ knowledge of two or more maternal danger signs was better in Lanfero (baseline: 8% vs. 28%) than Shebedino (baseline: 8% vs. 11.3%). CS-23 Ethiopia, Final Evaluation, Save the Children, December 2012 74 Table 3: Maternal and Newborn Care by District Indicator Shebedino (%) Lanfero (%) CS-23 (%) BL1 EL2 BL EL BL EL TT2 89.7 91.3 96.3 97.7 91.7 93.5 Delivery at Health Facility 1 26.7 3 24.7 2 26.0 Skilled Attendant 1 14.3 2 15.7 1 14.8 PP Maternal Visit 3 30.3 1 21.0 2 27.2 Know 2+ Mat. Danger Signs 8 11.3 8 28.0 8 17.0 Postnatal Visit (within three days) 4 15.7 2 12.0 4 14.4 Know 2+ NB Danger Signs 35 24.7 17 34.0 29 27.8 Fig 1.Maternal and Newborn Care by District 0 20 40 60 80 100 Delivery at HF Skilled Attendant PP Maternal Visit Know 2+ Maternal DS Postnatal Newborn Visit Know 2+ Newborn DS % Lanfero Baseline Lanfero Endline Shebedino Baseline Shebedino Endline Essential Newborn Care Three-quarters (74%) of deliveries still take place at home, though this is an improvement from the baseline result, which showed that nearly all (98%) mothers delivered at home. More than two-thirds of birth attendants were either relatives, , trained TBAs, untrained TBAs, trained and untrained CHWs or mothers delivered their own babies (Table 4). Deliveries by relatives were more common in Shebedino than Lanfero, while deliveries by untrained TBAs were more frequent in Lanfero. Among the 291 infants (0-11 months of age) born at home in the two districts, the practices of clean cord cutting, drying and wrapping, and placement with the mother before the placenta was delivered, appear to have declined since baseline, while immediate breastfeeding and feeding of colostrum has improved. The following are the rates of clean cord cut practice among home￾delivered infants in Shebedino (baseline: 97.1% vs. 69.6%) and Lanfero (baseline: 98.3% vs. 1 Baseline 2 Endline CS-23 Ethiopia, Final Evaluation, Save the Children, December 2012 75 68.5%); drying and wrapping in Shebedino (baseline: 74.9% vs. 67.1%) and Lanfero (baseline: 99.4% vs. 65.4%); and infant placement with mother before the placenta was delivered in Shebedino (baseline: 72.9% vs. 42.9%) and Lanfero (baseline: 35.7% vs. 41.5%, better than Shebedino). On the positive side, immediate breastfeeding (baseline: 61.7% vs. 92.9% overall) and colostrum given to the infant (baseline: 81.6% vs. 93.8% overall) have increased from the baseline result. This was demonstrated in both districts as follows; immediate breastfeeding in Shebedino (baseline: 76.7% vs. 94.4%) and Lanfero (baseline: 27.4% vs. 90.0%); colostrum given to infants in Shebedino (baseline: 79.8% vs. 95.0%) and Lanfero (85.7% vs. 91.5%). Government policy and strategy, currently discourages home deliveries by TBAs and traditional practitioners. Consequently, the project did not train TBAs in clean delivery and encouraged them to promote facility delivery. However, as we identified through household surveys, there remains a sizable proportion of mothers who are using TBAs and traditional practitioners for delivery services in their home. Table 4: Newborn Care for Home Delivery (among infants 0-11 months of age) by District Indicator Shebedino Lanfero CS-23 BL EL BL EL BL EL Delivery attendant  Relative/Friends 48.2 55.3 31.5 38.5 43.1 49.6  No One (i.e., Self-Delivery) 33.5 1.2 1.2 4.6 23.7 2.4  TBA (Trained) 5.9 7.5 50.0 2.3 19.3 5.7  TBA (Untrained) 11.8 1.9 7.7 23.8 10.6 9.3  Community Health Worker (Trained) 0.6 0.6 9.5 1.5 3.3 0.9  Community Health Worker (Untrained) 0.6 0.0 1.2 0.8 0.8 0.3 Clean cord cutting 97.1 69.6 98.3 68.5 97.5 69.2 Immediate drying and wrapping 74.9 67.1 99.4 65.4 82.4 66.5 Placed with mother before placenta delivered 72.9 42.9 35.7 41.5 61.6 42.4 Immediate breastfeeding 76.7 94.4 27.4 90.0 61.7 92.9 Colostrum given 79.8 95.0 85.7 91.5 81.6 93.8 CS-23 Ethiopia, Final Evaluation, Save the Children, December 2012 76 Fig.2 Essential Newborn Care among Infants Delivered at Home (0-11 months of age) by District Nutrition Exclusive breastfeeding among infants <6 months and <4 months, vitamin A supplementation in the last six months, and complementary feeding among children 6-8 months old and 9-23 months old, have improved from baseline in both districts. Exclusive breastfeeding among infants < 6 months (baseline: 3.0% vs. 28.8% overall, DHS 2011 was 52%); exclusive breastfeeding among infants <4 months (baseline: 11.0% vs. 34.6% overall); vitamin A supplementation (baseline: 60.0% vs. 91.1% overall); complementary feeding among infants 6-8 months old (baseline: 30.0% vs. 57.1% overall) and among infants 9-23 months old (baseline: 42.0% vs. 64.1%overall) (Table 5). Infant and Young Child Feeding (IYCF) practice was better in both districts compared to the national IYCF practice among 6-23 months of age (4% vs. 2.5% for SNNPR DHS 2011). Vitamin A supplementation was higher in Shebedino (baseline: 49.0% vs. 94.9%) than Lanfero (baseline: 66.0% vs. 83.7%) and complementary feeding practice among infants 6-8 months of age was better in Shebedino than Lanfero (baseline: 51.0% vs. 64.8% and 20.0% vs. 42.2% respectively). Similarly, complementary feeding among infants 9-23 months of age was better in Shebedino than Lanfero (baseline: 63.0% vs. 71.0% and 32.0% vs. 50.5%, respectively). CS-23 Ethiopia, Final Evaluation, Save the Children, December 2012 77 Table 5: Nutrition; Breastfeeding and Complementary Feeding Practices, and Vitamin A Supplementation Status by District Indicator Shebedino Lanfero CS-23 BL EL BL EL BL EL Exclusive breastfeeding  <6months 2 27.0 4 32.4 3 28.8  <4months 10 30.2 13 43.2 11 34.6 Vitamin A supplementation (among those 6-23 months) 49 94.9 66 83.7 60 91.1 Appropriate Complementary Feeding  6-8months 51 64.8 20 42.2 30 57.1  9-23months 63 71.0 32 50.5 42 64.1 Fig.3 Nutrition breastfeeding and complementary feeding practices and vit A supplementation status by district Immunization Ethiopia introduced Pentavalent (DPT, plus Hepatitis B and Haemophilus influenzae, type B) in 2007. “DPT1” and “DPT3” includes Pentavalent 1 and Pentavalent 3, respectively. Most immunization information was reported by mothers, as many lacked cards for all immunizations, including measles and DPT or Pentavalent. Access to EPI services was good (Penta 1: baseline 79.9% vs. 96.9% overall), and performance or coverage has improved (Penta 3: baseline 46.7% vs. 71.4% overall) (Table 6). Indeed, immunization coverage improved in both districts; Shebedino District (Penta 1: baseline 76.2% vs. 98.6%; Penta 3: baseline 49.2% vs. 71.2%; Measles: baseline 57.1% vs. 86.3%) and Lanfero District (Penta 1: baseline 88.5% vs. 93.5%; Penta 3: baseline 41.0% vs. 71.8%; Measles: (baseline 66.0% vs. 78.2%). The percentage of infants who received both measles and Penta 3 increased from the baseline, but low in both districts; Shebedino (baseline: 39.7% vs. 59.7%) and Lanfero (baseline: 39.3% vs. 54.1%). CS-23 Ethiopia, Final Evaluation, Save the Children, December 2012 78 Table 6: Immunizations (among infants 12-23 months old) Indicator Shebedino Lanfero CS-23 BL EL BL EL BL EL Measles 57.1 86.3 66.0 78.2 59.7 83.6 Penta 1(“Access”) 76.2 98.6 88.5 93.5 79.9 96.9 Penta 3 (“System performance”) 49.2 71.2 41.0 71.8 46.7 71.4 Measles & Penta 3 39.7 59.7 39.3 54.1 39.6 57.8 Fig.4: Immunization (infants 12-23 months) by District 0 20 40 60 80 100 Measles Penta 1 Penta 3 Measles & Penta 3 % Lanfero Baseline Lanfero Endline Shebedino Baseline Shebedino Endline Morbidity Prevalence of fever and cough and fast or difficult breathing decreased from baseline, but the rate of diarrhea increased from baseline (Table 7). Overall, children with fever were baseline: 34.8% vs. 23.9%; with diarrhea baseline: 28.8% vs. 31.4%; and with cough and fast or difficult breathing baseline: 24.7% vs. 12.4%. Compared to the baseline, the prevalence of all illnesses two weeks prior to the survey, declined in Shebedino, while in Lanfero they increased. Lanfero District is a malarious and food insecure area which significantly contributes to this fact. In Shebedino illness prevalences were: fever (baseline: 38.0% vs. 22.0%); diarrhea (baseline: 32.3% vs. 31.7%); and cough and fast or difficult breathing (baseline: 28.0% vs. 7.3%). The prevalence of all increased in Lanfero as follows: fever (baseline: 27.4% vs. 27.7%); diarrhea (baseline: 20.7% vs. 31.0%); and cough and fast or difficult breathing (baseline: 17.0% vs. 22.3%). CS-23 Ethiopia, Final Evaluation, Save the Children, December 2012 79 Table 7: Morbidity Prevalence Two Weeks Prior to the KPC Survey in Both Districts Indicator Shebedino Lanfero CS-23 BL EL BL EL BL EL Prevalence of fever 38 22.0 27.4 27.7 34.8 23.9 Prevalence of diarrhea 32.3 31.7 20.7 31.0 28.8 31.4 Prevalence of cough and fast or difficult breathing 28 7.3 17 22.3 24.7 12.4 Any illness 54.6 34.0 48.3 60.0 52.7 42.8 Fig.5: Morbidity Prevalence by District Malaria Ownership of insecticide-treated bednets (ITN) was below half (baseline: 55.1% vs. 46.6% overall), it was better in Lanfero than Shebedino. Actual use of owned bednets by children improved from baseline (baseline: 73.1% vs. 84.7% overall). Overall, ownership of ITNs at endline was below the baseline in both districts, but more than three-fourths of the children in families that owned ITNs slept under the ITN Table 8). Overall, ITN use by children was very low (39.3% endline result). Only one-fourth of children (23.9% [149/600 weighted]) reportedly had fever during the prior fortnight compare with (34.8% [196/600]) baseline result. Careseeking behavior improved from baseline (baseline: 46.9% vs. 71.2% overall), careseeking within 24 hours of the onset of illness was better in Lanfero than Shebedino. Sources of care were mainly HCs or clinics (baseline: 60.5% vs. 74.7% overall) or HPs (baseline: 27.0% vs. 17.5% overall). Some careseekers (0.5%) in Lanfero used the hospital; however none did during the baseline survey in either district. Fewer (7.3%) went to pharmacies at endline than at baseline (11.0%). As reported by mothers, CS-23 Ethiopia, Final Evaluation, Save the Children, December 2012 80 careseeking behavior to HCs was better in Lanfero than Shebedino (85.9% vs. 68.9%, respectively), while no mothers sought care for their ill child from the pharmacy in Lanfero, some (11.1%) did in Shebedino. Of those seeking care, treatment with an appropriate antimalarial improved from baseline (baseline: 33.2% vs. 65.2% overall). In Lanfero, the rate of those seeking care who actually received an appropriate antimalarial was better than in Shebedino (79.7% vs. 57.8%, respectively). One-fifth (20.4% overall) of the treated children received coartem while 14.9% were treated with chloroquine. More children were treated with coartem in Lanfero (23/51 [45.1%]) than in Shebedino (2/26 [7.7%]); no children were treated with quinine in either district. All HPs or HEWs perform rapid diagnostic test for both malaria species of vivax and falciparum (they use multi species RDT) and treat accordingly. All kebeles in Lanfero District are malarious while few kebeles in Shebedino are malarious. Table 8: Malaria Prevention Practices and Care Seeking for Treatment by District Indicator Shebedino Lanfero CS-23 BL EL BL EL BL EL Ownership of ITN 48 34.3 71.3 70.3 55.1 46.6 ITN use by child (of those who own) 75.7 85.4 67.3 83.4 73.1 84.7 ITN use by child (overall) 36 29.3 48 58.7 40 39.3 Prevalence of fever 38 22.0 27.4 27.7 34.8 23.9 Careseeking 41.2 68.2 59.8 77.1 46.9 71.2 Treatment with appropriate antimalarial (of those seeking care) 19.1 57.8 65.3 79.7 33.2 65.2 Treatment with appropriate antimalarial (of those sick) 7.9 39.4 39.4 61.4 17.4 46.9 Fig.6: Malaria Care seeking and Antimalarial by District CS-23 Ethiopia, Final Evaluation, Save the Children, December 2012 81 Acute Respiratory Infection (ARI) Prevalence of cough and difficult or fast breathing has declined from the baseline; one in eight mothers (89/600 [12.4%]) reported that their children had cough and fast or difficult breathing in the prior fortnight versus one in four mothers (24.7%) from the baseline (Table 9). Overall care seeking behavior for ARI improved from the baseline (baseline: 31.9% vs. 44.8%); better in Lanfero than its counter Shebedino. Overall, children treated with an antibiotic for ARI decreased from the baseline; among children who sought care, only 44.2% were treated with antibiotic versus 88.5% at baseline. Caretakers’ knowledge of pneumonia danger signs and child danger signs has improved from the baseline as follows: the knowledge of pneumonia danger signs (baseline: 48.7% vs. 76.4% overall); two or more child danger signs (baseline: 50.9% vs. 74.3% overall); and three or more child danger signs (baseline: 15.2% vs. 51.6% overall). Table 9: ARI Prevalence, Care Seeking, Treatment and Danger Signs Knowledge by District Indicator Shebedino Lanfero CS-23 BL EL BL EL BL EL Prevalence of cough and difficult or fast breathing 28 7.3 17 22.3 24.7 12.4 Appropriate care seeking 32.1 40.9 31.4 52.2 31.9 44.8 Treated with antibiotic (of those seeking care) 88.9 40.9 87.5 50.7 88.5 44.2 Treated with antibiotic (of those sick) 28.6 40.9 27.5 50.7 28.3 44.2 Knowledge of pneumonia danger signs 59.7 77.7 23.7 74.0 48.7 76.4 Knowledge of 2+ child danger signs 46.3 72.3 61.3 78.0 50.9 74.3 Knowledge of 3+ child danger signs 14.3 54.3 17.3 46.3 15.2 51.6 CS-23 Ethiopia, Final Evaluation, Save the Children, December 2012 82 Fig.7 Care Seeking and Antibiotic Treatment of ARI by District Care seeking for ARI vs. Fever Sources of care seeking varied by district (Table 10). Care seeking due to fever at HC or clinic improved from the baseline (baseline: 60.5% vs. 74.7% overall); while careseeking to HP for fever declined from the baseline (baseline: 27.0% vs. 17.5% overall) as HPs are more accessible to families than HCs. Careseeking to community volunteers for fever decreased from baseline (baseline: 1.5% vs. 0.0% overall); though families still use private pharmacies to get treatment (only in Shebedino; none in Lanfero). Families using pharmacies for care seeking declined from the baseline (baseline: 11.0% vs. 7.5% overall); while other families have started seeking care from the hospital for treatment of fever (baseline: 0.0% vs. 0.5 overall). Care seeking for ARI at the HC or clinic increased from the baseline (baseline: 75.5% vs. 76.9% overall); but care seeking to HP for ARI was lower than baseline (baseline: 14.8% vs. 0.0% overall). Care seeking to community volunteers for ARI did not occur (baseline: 9.6% vs. 0.0% overall); but some care givers visited private pharmacies in Shebedino to get care for their ill child (baseline: 0.0% vs. 7.3% overall). There are more private pharmacies in Shebedino than Lanfero which may encourage families to go to pharmacies to get treatment. Of course, sometimes there are drug shortages in both districts which may motivate families to go to pharmacies to save the life of their children. Families have started to seek care from hospitals for ARI treatment (baseline: 0.0% vs. 0.9 overall). CS-23 Ethiopia, Final Evaluation, Save the Children, December 2012 83 Table 10: Source of Care Seeking for Fever and ARI by District Indicator Shebedino Lanfero CS23 (Weighted) Fever (N=45 ) ARI (N=13) Fever (N=64 ) ARI (N=37 ) Fever (N= 109) ARI (N=50) Health Facility of which: 88.9 69.2 98.4 94.6 92.7 95.8 HC, private or public clinic 68.9 69.2 85.9 91.9 74.7 76.9 HP 20.0 0 12.5 5.4 17.5 18.0 Community Volunteer 0 0 0 0 0 0.0 Pharmacy 11.1 30.8 0.0 0 7.3 4.2 Hospital 0 0 1.6 2.7 0.5 0.9 Fig. 8 Care Seeking for Fever and ARI Diarrhea Appropriate hand washing, point of water treatment, zinc treatment and increased fluid intake during diarrhea has improved. During the project implementation period there had been an outbreak of “acute watery diarrahea” in Lanfero District where the opportunity was used to strongly promote diarrhea prevention. This likely influenced the diarrhea prevention and treatment practices in Lanfero. The following practices improved at endline: appropriate hand washing (baseline: 28.2% vs. 60.0% overall); point of use water treatment (baseline: 13.0% vs. 31.6% overall); zinc treatment (baseline: 7.0% vs. 34.2% overall) and increased fluid intake (baseline: 19.8% vs. 59.3% overall). Prevalence of diarrhea increased in Lanfero but decreased in Shebedino (baseline: 28.8% vs. 31.4% overall). Children treated with antibiotics for diarrhea, ORT use and increased food intake during diarrhea have declined from baseline (Table 11) as follows: ORT use (baseline: 56.5% vs. 54.8% overall); increased food intake (baseline: 28.5% vs. 25.3% overall); and children treated with antibiotics (baseline: 41.0% vs. 3.5% overall). CS-23 Ethiopia, Final Evaluation, Save the Children, December 2012 84 ORT use was low in Shebedino and better in Lanfero (42.1% vs. 79.6%, respectively). The percentage of families who gave increased food during diarrhea declined more in Shebedino than Lanfero (20.0% vs. 35.5%, respectively), and the number of children treated with an antibiotic has decreased both in Shebedino and Lanfero (3.2% and 4.3%) Table 11: Diarrhea Prevention and Treatment by District Indicator Shebedino Lanfero CS-23 BL EL BL EL BL EL Appropriate hand washing 33.3 54.3 16.7 71.0 28.2 60.0 Point of use water treatment 16.3 28.3 5.3 38.0 13 31.6 Prevalence of diarrhea 32.3 31.7 20.7 31.0 28.8 31.4 ORT use 62.9 42.1 41.9 79.6 56.5 54.8 Zinc treatment 9.3 35.8 1.6 31.2 7.0 34.2 Increased food intake 26.8 20.0 32.3 35.5 28.5 25.3 Increased fluid intake 14.4 62.1 22.6 53.8 19.8 59.3 Children treated with Anti-biotic, anti-motility drugs 45 3.2 32.0 4.3 41 3.5 Fig. 9 Diarrhea Management by District Anthropometry Moderate underweight rate (<-2 Z-score) is lower than at baseline (18.8% vs. 7.7% overall) (Table 12A) while severe underweight rate (<-3 Z-score) has increased (3.1% vs. 7.7% overall) (Table 12B). However, this is better than the DHS 2011 finding (moderate: [29%]; and severe: [9%]). There were slightly more moderately underweight children in Shebedino than Lanfero (11.5% vs. 10.0%, respectively), but more severely underweight children in Lanfero (10% vs. 5.3%, respectively). Lanfero is a food insecure district. In both districts more boys were moderately and severely malnourished than girls. Mean weight-for-age z-score was worse for older children CS-23 Ethiopia, Final Evaluation, Save the Children, December 2012 85 (children 0-5 and 18-23 months old: -0.05 and -1.1 in Shebedino, and -0.33 and -1.77 in Lanfero). Nutritional status deteriorated as the age of children increased; young infants who were exclusively fed breastmilk were not underweight (none in either district). Table 12A: Moderate Underweight (Weight-for-Age Z-score <-2) Characteristic Shebedino (N=300) Lanfero (N=300) Total (N=600) N D % N D % N D W% Age in Months 0-5 0 63 0 0 37 0 0 100 0 6-11 4 98 4.1 3 93 3.2 7 191 3.7 12-17 7 79 8.9 16 102 15.7 23 181 12.7 18-23 5 60 8.3 11 68 16.2 16 128 12.5 Sex Male 11 139 7.9 17 153 11.1 28 314 8.9 Female 5 161 3.6 13 147 8.8 18 286 6.3 Total 16 300 11.5 30 300 10.0 46 600 7.7 Fig.10 Nutritional Status (WFA Z-score <-2) by Age and District There were more severely underweight children in Lanefro than their counter parts in Shebedino; among children 12-17 months old (15.7% vs. 8.9%, respectively); and 18-23 months old (16.2% vs. 8.3%, respectively). The mean weight-for- age was slightly worse than the region (-1.13 Z￾score) versus SNNPR (-1.2 Z-score) (DHS 2011). CS-23 Ethiopia, Final Evaluation, Save the Children, December 2012 86 Table 12B: Severe Underweight (Weight-for Age Z-score <-3) Characteristic Shebedino (N=300) Lanfero (N=300) Total (N=600) N D % N D % N D W% Age in Months 0-5 0 63 0 0 37 0 0 100 0 6-11 4 98 4.1 3 93 3.2 7 191 3.7 12-17 7 79 8.9 16 102 15.7 23 181 12.7 18-23 5 60 8.3 11 68 16.2 16 128 12.5 Sex Male 11 139 7.9 17 153 11.1 28 314 8.9 Female 5 161 3.1 13 147 8.8 18 286 6.3 Total 16 300 5.3 30 300 10.0 46 600 7.7 E. Discussion Validity These surveys were implemented according to protocol; (1) Data collection teams enumerated households, starting in the center of the ketana/sub-village. Although they were instructed to pick a starting house (determined by random selection of a number) along a ray in the direction of the spun bottle, one can imagine the ease of starting close to the central cluster of houses. If this occurred, it would systematically exclude more remote, impoverished, and marginalized families. On the other hand, there is no evidence that this occurred, since the supervisors of data collection teams were vigilant to see that sampling occurred according to protocol. (2) All anthropometrical measurements were taken for all children (n=600 overall); there was no rejection due to extreme values. All eligible children were weighed according to measurement protocol. Main Findings To compare the endline results with the baseline, we highlighted the main findings module by module and district by district. We used same baseline qualifiers to enable us to compare the endline result versus the baseline; the qualifiers were defined as follows: very low (<10%), low (10-39%), fair (40-59%), good (60-79%), high (80-89%), and very high (>90%). With regard to the defined qualifiers, most indicators have improved from the baseline results (Table 13A) CS-23 Ethiopia, Final Evaluation, Save the Children, December 2012 87 Table 13A: Summary Indicators: Scored by District and Project Compared at Baseline and Endline Results Module Indicator Shebedino Lanfero Project BL EL BL EL BL EL MNC TT2 Very High Very High Very High Very High Very High Very High Delivery at HF Very Low Low Very Low Low Very Low Low Skilled birth attendant Very Low Low Very Low Low Very Low Low PP maternal visit Very Low Low Very Low Low Very Low Low PN newborn visit Very Low Low Very Low Low Very Low Low PP maternal DS 2+ Very Low Low Very Low Low Very Low Low Newborn DS 2+ Low Low Low Low Low Low Clean cord cut Very High Good Very High Good Very High Good Dry and wrap Good Good Fair Good Good Good Placed with mother Good Fair Fair Fair Good Fair Immediate BF Good Very High Low Very High Good Very High Colostrum Good Very High High Very High High Very High Nutrition Exclusive BF Very Low Low Very Low Low Very Low Low Vitamin A Fair Very High Good High Good Very High CF (6-8/9-23) Fair/Good Good/Good Low/Low Fair/Fair Low/ Fair Fair/Good Immunizati ons Measles Fair High Good Good Fair High DPT1/Penta 1 Good Very High High Very High Good Very High DPT3/Penta 3 Low Good Low Good Low Good Malaria Own ITN Fair Low Good Good Fair Fair Use ITN Low High Fair High Low High Careseeking Fair Good Fair Good Fair Good Antimalarial Very Low Fair Low Good Very Low Good ARI Careseeking Low Fair Low Fair Low Fair Antibiotic Low Fair Low Fair Low Fair Child DS 2+ Fair Good Good Good Fair Good Diarrhea Hand washing Low Fair Low Good Low Good Water treatment Low Low Very Low Low Low Low ORT Good Good Fair fair Fair Fair Zinc Very Low Very Low Very Low Very Low Very Low Very Low Food increased Low Low Low Low Low Low Fluids increased Low Low Low Low Low Low CS-23 Ethiopia, Final Evaluation, Save the Children, December 2012 88 Next we transformed the qualifiers into ordinal “grades” familiar to most (American) school￾children. The districts’ “Report Cards” were not strong. Shebedino had (baseline: [2 A+ vs. 5 A+]; [baseline: 0A vs. 2 A]; [baseline: 7 B vs. 8 B]; [baseline: 6 C vs. 5 C]; [9 D vs. 11 D]; [baseline: 8 F vs. 1 F]). Lanfero had (baseline: [2 A+ vs. 4 A+]; [ 2 A vs. 4 A]; [ 4 B vs. 7 B]; [ 5 C vs. 6 C]; [11 D vs. 10 D]; [ 7 F vs. 1 F]). There was improvement in both districts compared to baseline “grades”. The Project has (baseline: [2 A+ vs. 5 A+]; [1 A vs. 2 A]; [5 B vs. 8 B]; [6 C vs. 6 C]; [10 D vs. 10 D]; [8 F vs. 1 F]), again there was improvement from the baseline “grades”. Although there was improvement from the baseline “grades”, the districts had different strengths and weaknesses. The light trellis shading shows the improved indicators from the baseline with one “grade” and the light down diagonal shading shows indicators that improved by more than one “grade.” (Please see the below legend.) Overall, the majority of indicators have improved from the baseline result in both districts that is great success for the project. Table 13B: Summary Indicators: Scored (“Transformed”) by District and Project Comparing Baseline and Endline Results Module Indicator Shebedino Lanfero Project BL EL BL EL BL EL Maternal and Newborn Care TT2 A+ A+ A+ A+ A+ A+ Maternal and Newborn Care Delivery at HF F D F D F D BL EL BL EL BL EL Maternal and Newborn Care Skilled birth attendant F D F D F D Maternal and Newborn Care PP maternal visit F D F D F D Maternal and Newborn Care PN newborn visit F D F D F D Maternal and Newborn Care PP maternal DS 2+ F D F D F D Maternal and Newborn Care Newborn DS 2+ D D D D D D Maternal and Newborn Care Clean cord cut A+ B A+ B A+ B Maternal and Newborn Care Dry and wrap B B C B B B Maternal and Newborn Care Placed with mother B C C C B C Maternal and Newborn Care Immediate BF B A+ D A+ B A+ Maternal and Newborn Care Colostrum B A+ A A+ A A+ Nutrition Exclusive BF F D F D F D CS-23 Ethiopia, Final Evaluation, Save the Children, December 2012 89 Module Indicator Shebedino Lanfero Project Nutrition Vitamin A C A+ B A B A+ Nutrition CF (6-8/9-23) C/B B/B D/D C/C D/C C/B Immunizations Measles C A B B C A Immunizations DPT1 B A+ A A+ B A+ Immunizations DPT3 D B D B D B Malaria Own ITN C D B B C C Malaria Use ITN D A C A D A Malaria Careseeking C B C A C B Malaria Antimalarial F C D A F B ARI Careseeking D C D C D C ARI Antibiotic D C D C D C ARI Child DS 2+ C B B B C B Diarrhea Hand-washing D C D B D B Diarrhea Water treatment D D F D D D Diarrhea ORT B B C C C C Diarrhea Zinc F F F F F F Diarrhea Food continued D D D D D D Diarrhea Fluids increased D D D D D D Legend: Light Trellis=Improved indicators from baseline with one “grade”. Light Down Diagonal= Indicators that improved by more than one “grade.” Result indicators sorted for the project: ITN use by child of those who own were improved from “low” to “high” in both districts, Measles immunization improved from “fair” to “high” in Shebedino, knowledge of two or more child danger signs on pneumonia was improved from “fair” to “good” in Shebedino, careseeking to HF was improved from “fair” to “”high” in Lanfero, maternal and newborn care was improved from “good” to “very high” in both districts. In general, 11 indicators have improved from the baseline results (two from “very low” to “low”; seven indicators from “low” to “fair “ and “ above”; five indicators were same “grade” with the baseline. CS-23 Ethiopia, Final Evaluation, Save the Children, December 2012 90 Table 13C: Results Indicators: Scored and Sorted for Project Comparing Baseline with Endline Results Module Indicator Shebedino Lanfero Project BL EL BL EL BL EL Maternal and Newborn Care PN newborn visit F D F D F D Nutrition Exclusive BF F D F D F D Diarrhea Zinc F F F F F F Maternal and Newborn Care Newborn DS 2+ D D D D D D Immunizations DPT3 D B D B D B Malaria Use ITN D A C A D A ARI Careseeking D C D C D C ARI Antibiotic D C D C D C Diarrhea Hand-washing D C D B D B Diarrhea Food increased D D D D D D Diarrhea Fluids increased D D D D D D Immunizations Measles C A B B C A Malaria Own ITN C D B B C C Malaria Careseeking C B C A C B ARI Child DS 2+ C B B B C B Diarrhea ORT B B C C C C Maternal and Newborn Care Immediate BF B A+ D A+ B A+ As can be seen from Table 14 below, the project has shown improvement in most of the indicators from the baseline though it may not have reached all of its targets. The overall target and target by district was reached or met beyond target for immediate breastfeeding (69% vs. 92.9%), exclusive breastfeeding for infants under six months old (25% vs. 28.8%), treatment with antibiotic for ARI (50% vs. 53.3%), appropriate hand washing practice (455 vs. 65%), zinc treatment for diarrhea (255 vs. 34.2%), increased fluid intake during diarrhea illness (36% vs. 59.3%) and use of medicine for diarrhea (22% vs. 3.5%). In addition, Lanfero District has met or passed its set targets for; care seeking for fever (77.1% vs. 75%), knowledge of 2+ child danger signs (78% vs. 75%), and ORT use for diarrhea (79.6% vs. 65%). All of the indicators have improved from the baseline, except knowledge of 2+ neonatal danger signs (29.3% vs. 27.8%), overall ITN use by child (39.9% vs. 39.3%), ORT use (56.5% vs. 54.8%, but this has improved in Lanfero from 41.9% to 79.9%) and increased fluid intake for diarrhea (28.5% vs. 25.3%) CS-23 Ethiopia, Final Evaluation, Save the Children, December 2012 91 Table 14: Results Indicators Determined by Household Survey by Endline Values, Targets, and District (April 2012) Indicator Shebedino Lanfero CS-23 BL EL TAR3 BL EL TAR BL EL TAR Postnatal Newborn Visit 4.0 15.7 30 2.3 12.0 30 3.5 14.4 30 Know 2+ Neonatal Danger Signs 34.7 24.7 50 17.1 33.7 40 29.3 27.8 60 IBF 76.7 94.4 77 27.4 90.0 50 61.7 92.9 69 EBF (<6m) 1.8 27.0 25 4.0 32.4 25 2.5 28.8 25 Measles 57.1 86.3 75 66.0 78.2 75 59.7 83.6 75 Penta 3 49.2 71.2 75 41.0 71.8 75 46.7 71.4 75 ITN Use by Child (overall) 36.3 29.3 65 48.2 58.7 65 39.9 39.3 65 Care seeking for fever 41.2 68.2 75 59.8 77.1 75 46.9 71.2 75 Appropriate Care Seeking for ARI 32.1 40.9 60 31.4 52.2 60 31.9 44.8 60 Treated with anti-biotic (of those sick with ARI) 28.6 54.5 50 27.5 50.7 50 28.3 53.3 50 Knowledge of 2+ Child Danger Signs 46.3 72.3 75 61.3 78.0 75 50.9 74.3 75 Know 2+ postpartum Danger Signs 8.0 11.3 30 8 28.0 30 8.0 17.0 30 Appropriate hand washing 33.3 54.3 50 16.7 71.0 35 28.2 60.0 45 ORT use for diarrhea 62.9 42.1 75 41.9 79.6 65 56.5 54.8 72 Zinc treatment for diarrhea 9.3 35.8 25 1.6 31.2 25 7.0 34.2 25 Increased food intake 26.8 20.0 40 32.3 35.5 50 28.5 25.3 43 Increased fluid intake 14.4 62.1 30 22.6 53.8 50 19.8 59.3 36 Use of medicine for diarrhea 45 3.2 25 32.0 4.3 15 41.0 3.5 22 3 Target CS-23 Ethiopia, Final Evaluation, Save the Children, December 2012 92 ANNEXES CS-23 Ethiopia, Final Evaluation, Save the Children, December 2012 93 Annex 1: Questionnaire Ethiopia Child Survival Program USAID/CSHGP Grant Number M/OAA/GH-07-003 Final Knowledge, Practice, and Coverage (KPC) Survey Questionnaire (Including Revised Rapid CATCH) Shebedino & Lanfero Districts, SNNPR March/April 2012 Conducted by Save the Children/USA CS-23 Ethiopia, Final Evaluation, Save the Children, December 2012 94 INTERVIEWER: Please Write Down the Responses in CAPITAL LETTERS Ask the mother if she has a child under 24 months who lives with her and she is not a visitor (stayed in household for less than 3 months from outside the Woreda). If confirmed that she has a child and she is not a visitor, write the name of the child and proceed with interview, if no thank the mother and end the interview. Name of the Child____________________________________ Identification Cluster Number PA/Kebele Name Interview Record Number Name of Mother Name of Supervisor Interview start time Data Entered by Date: ___/___/____ day/month/year 1 2 3 Final Visit Interview date ___/___/____ day/month/year ___/___/____ day/month/year ___/___/____ day/month/year For Supervisor Day Name of Interviewer Month Year Result Code* Result Code *Result Codes: 1. Completed 2. Respondent not at home 3. Postponed 4. Refused 5. Other______________________________________ Specify CS-23 Ethiopia, Final Evaluation, Save the Children, December 2012 95 Consent Page INFORMED CONSENT Hello. My name is ______________________________, and I am working with Save the Children. We are conducting a survey and would appreciate your participation. I would like to ask you about your health and the health of your youngest child under the age of two. This information will help Save the Children to plan health services and assess whether it is meeting its goals to improve children’s health. The survey usually takes _______ minutes to complete. Whatever information you provide will be kept strictly confidential and will not be shown to other persons. Participation in this survey is voluntary and you can choose not to answer any individual question or all of the questions. However, we hope that you will participate in this survey since your views are important. Will you participate in this survey? At this time, do you want to ask me anything about the survey? Signature of interviewer: ____________________ Date: ____________________ RESPONDENT AGREES TO BE INTERVIEWED…………1 RESPONDENT DOES NOT AGREE TO BE INTERVIEWED…………2 END CS-23 Ethiopia, Final Evaluation, Save the Children, December 2012 96 ALL QUESTIONS ARE TO BE ADDRESSED TO MOTHERS WITH A CHILD LESS THAN 24 MONTHS OF AGE Note: 1. Ask information for the youngest child in the household if there is more than one with less than 24 months of age. 2. Use local events to probe on age if not known I. INTRODUCTION/ CHILD SPACING No. Questions and Filters Coding Categories Skip 1 Have you ever attended school? Yes………………….………………….1 No…………………………….………..2  3 2 What is the highest grade you completed? Grade……………. 3 What is the name, sex, date of birth of your youngest child that you gave birth to and that is still alive? USE LOCAL EVENTS TO PROBE ON DATE OF BIRTH IF NOT KNOWN Youngest Child Name ________________________ Sex Male……………………………....1 Female…………..……...………....2 Date of Birth Day Month Year All subsequent questions pertain to the youngest child under two. II. MATERNAL AND NEWBORN CARE No. Questions and Filters Coding Categories Skip 4 During your pregnancy with (Name) did you receive an injection in the arm to prevent the baby from getting tetanus that is convulsions after birth? Yes………………………………..………1 No……………………..……………….....2 Don’t know………………...………….. ...9  6  6 5 While pregnant with (name), how many times did you receive such an injection? One………………………..…….……….1 Two………………...……………………2 Three or More…………………........……3 Don’t know………………….….………..9 CS-23 Ethiopia, Final Evaluation, Save the Children, December 2012 97 6 Did you receive any tetanus toxoid injection at any time before that pregnancy, including during a previous pregnancy or between pregnancies? Yes………………………………….……1 No……………………………………….2 Don’t know…………..………...…….…. 9  8  8 7 Before the pregnancy with (Name), how many times did you receive a tetanus injection? One……………………..………………..1 Two……………….…………………..…2. Three or More…………..………..………3 Don’t know………………............……....9 8 Have you ever given birth to a boy or girl who was born alive but later died? IF NO, PROBE: Any baby who cried or showed signs of life but did not survive? Yes . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . 1 No . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . 2  10 9 How many boys have died? And how many girls have died? IF NONE, RECORD '00'. None………………...……… Boys Dead . . . . . . ………. . . . . Girls Dead. .. ….. . . . ………… 10 From the children that you have given birth, is there a Child who is died before the age of 5 years? PROBE: for the child that she has given birth. Yes………………………..……………..1 No…………………..……………………2 Ser No The first child that died The second child that died The third child that died The fourth child that died A Child’s age B Sex C Date of birth (DD/mm/0000) D Date of death (DD/mm/0000) E Month of death Year of death F Possible cause of death as explained by the mother 1 2 3 11 How far are you from the nearest health facility? Distance in km…….... 12 How would you get there? Walk………………………………………1 Car ............................................................ 2 CS-23 Ethiopia, Final Evaluation, Save the Children, December 2012 98 RECORD ALL RESPONSES Motorcycle………………………………..3 Donkey/Horse Cart.. ................................. 4 Other………………...……………………5 ________________________________ (SPECIFY) 13 How long would it take you to get there? ______________hours 14 Who assisted with the delivery of (Name)? Anyone else? PROBE FOR THE TYPE(S) OF PERSON(S) AND RECORD ALL MENTIONED. IF RESPONDENT SAYS NO ONE ASSISTED, PROBE TO DETERMINE WHETHER ANY ADULTS WERE PRESENT AT THE DELIVERY. Doctor……………….………………..….1 Nurse…………………..............................2 Midwife………………..………..…….….3 Health Officer…..………………...……...4 Health Assistant……………………….....5 Health Extension Worker.………………..6 Trained Traditional Birth Attendant……..7 Trained Community Health Worker……...8 Traditional Birth Attendant……………....9 Community Health Worker…….……….10 Relative/Friend……………….…………11 No one……… …………………..............12  20  20  20  20  20  20 15 Was a clean birth kit used? Yes ......................................................... 1 No .......................................................... 2 Don’t Know ........................................... 9 16 What instrument was used to cut the cord? New Razor Blade .................................... 1 New And Boiled Razor Blade ................ 2 Used Razor Blade ................................... 3 Used And Boiled Razor Blade ................ 4 New Scissors........................................... 5 New And Boiled Scissors ....................... 6 Used Scissors .......................................... 7 Used And Boiled Scissors ...................... 8 CS-23 Ethiopia, Final Evaluation, Save the Children, December 2012 99 Knife ....................................................... 9 Other.........................................................10 _____________________________ (Specify) Don’t know...............................................9 17 Was (NAME) dried (wiped) before the placenta was delivered? Yes .......................................................... 1 No ........................................................... 2 Don’t know ............................................. 9 18 Was (NAME) wrapped in a clean, dry cloth or blanket before the placenta was delivered? Yes............................................................1 No..............................................................2 Don’t know................................................9 19 Where (NAME) put before the placenta was delivered? With Mother.............................................1 In Cot........................................................2 On Floor...................................................3 Bathed.......................................................4 Other.........................................................5 _____________________________ (SPECIFY) Don’t Know..............................................9 20 What did you do with (NAME) in the first hour or two after birth? MULTIPLE ANSWERS ARE POSSIBLE Breastfed...................................................1 Bathed…………………………………...2 Let Sleep...................................................3 Other.........................................................4 .____________________________ (SPECIFY) Don’t Know...............................................9 21 After (NAME) was born, did anyone check on your health? Yes ........................................................ 1 No...........................................................2  24 22 How many days or weeks after the delivery did the first check take place? Days After Delivery.............1 Weeks After Delivery...........2 Don’t Know...........................9 23 Who checked on your health at that time? PROBE FOR MOST QUALIFIED PERSON. HEALTH PROFESSIONAL Doctor…………………………………….1 Nurse……………………………...………2 CS-23 Ethiopia, Final Evaluation, Save the Children, December 2012 100 Midwife…………………………...………3 Health Officer…………….….……..…….4 Health Assistant………………….…….....5 Health Extension Worker…………....…...6 OTHER PERSON Traditional Birth Attendant.........................7 Other..........................................................8 ______________________________ (SPECIFY) 24 What are the signs of danger after giving birth indicating the need for you to seek health care for yourself? DO NOT PROBE. RECORD ALL MENTIONED. Fever .......................... ………………....1 Excessive Bleeding..................................2 Smelly Vaginal Discharge.......................3 Other.......................................................4 _____________________________ (SPECIFY) Don’t Know………………………........9 25 What are the signs to watch for that may indicate that a newborn baby < 2 months of age is ill? DO NOT PROBE. RECORD ALL MENTIONED. Unable to suck or sucking poorly……....1 Fast Breathing, grunting……….………..2 Severe Chest in-drawing……….……….3 Convulsions……………………..….…..4 Repeated Vomiting……………….….…5 Abdominal Distension……………….…6 No stool after 24 hours……………….…7 Red umbilicus or draining pus…........…..8 Multiple skin pustules……………….…..9 Red swollen eyes and pus discharges......10 Jaundice/yellow skin…………….…..…11 Pallor/Bleeding………………………...12 Fever/feels hot…………………………13 Low body temperature or feels cold.…..14 CS-23 Ethiopia, Final Evaluation, Save the Children, December 2012 101 Other___________________________15 (Specify) Don’t Know…………………….............16 26 During your postpartum check, were you counseled on the following?  Maternal Danger Signs  Family Planning  Maternal Vitamin A  Exclusive Breastfeeding  Newborn Danger Signs  Keeping the newborn warm  Child Immunization YES NO Maternal Danger Signs…...............1 2 Family Planning……………….....1 2 Maternal Vitamin A………….….1 2 Exclusive Breastfeeding..............1 2 Newborn Danger Signs................ 1 2 Keeping the Newborn Warm…….1 2 Child Immunization.......................1 2 27 Sometimes children >2 months of age get sick and need to receive care or treatment for illnesses. What are the signs of illness that would indicate your child above 2 months of age needs treatment? DO NOT PROBE. RECORD ALL MENTIONED. Looks Unwell or Not Playing Normally…1 Not Eating or Drinking…………….…......2 Lethargic or Difficult To Wake……..........3 High Fever………………………………..4 Fast or Difficult Breathing………...…......5 Vomits Everything………………..….…..6 Convulsions………………………..…..…7 Cough/Difficult Breathing………..….…..8 Diarrhea…………………………..…....…9 Don’t Know………………………...…...10 Other………………………………….…11 ________________________________ (SPECIFY) 28 In the first two months after delivery, did you receive a Vitamin A dose like this? SHOW AMPULE/CAPSULE/SYRUP Yes………………………………………1 No…………………………...…………..2 Questions 29-31 refer to the youngest child shortly after birth 29 After (Name) was born, did any health care provider or traditional birth attendant check on (Name’s) health? Yes………………………….…….……1 No……………………………………...2  32 CS-23 Ethiopia, Final Evaluation, Save the Children, December 2012 102 30 How many hours, days or weeks after the birth of (Name) did the first check take place? IF LESS THAN ONE DAY, CIRCLE 0 AND RECORD HOURS; IF ONE TO SIX DAYS CIRCLE 1 AND RECORD DAYS; IF MORE THAN 6 DAYS CIRCLE 2 AND RECORD WEEKS. Hours………………….….0 Days………………………1 Weeks……………………..2 Don’t Know…………….....9 31 Who checked on (Name’s) health at that time? Anyone else? PROBE FOR THE MOST QUALIFIED PERSON AND RECORD ALL MENTIONED. Doctor………………………..……..…..1 Nurse………………………....................2 Midwife…………..…………………….3 Health Officer….………….….………...4 Health Assistant……………………..…5 Health Extension worker………….……6 Trained Traditional Birth Attendant……7 Trained Community Health Worker……8 Traditional Birth Attendant………...…..9 Community Health Worker….……......10 Relative/Friend……..………………….11 No one…………… ……………...…...12 III. BREASTFEEDING/ INFANT AND YOUNG CHILD FEEDING 32 Did you ever breastfeed (NAME)? Yes……………………………….…..1 No…………………………….….…..2  40 33 How long after birth did you first put (NAME) to the breast? IF LESS THAN 1 HOUR, RECORD 0 HOURS, IF LESS THAN 24 HOURS RECORD THE HOURS, OTHERWISE RECORD DAYS Immediate.........................................0 Hours................................ Days........................................ Don’t Remember...............................9 34 During the first three or four days after delivery, before your regular milk began flowing, did you give (NAME) the liquid (Colostrum) that came from your breasts? Yes ......................................................1 No .......................................................2 Don’t Know .......................................9 35 Are you still breastfeeding (NAME)? Yes …………………………..…...…1 No …………………………………..2  38 CS-23 Ethiopia, Final Evaluation, Save the Children, December 2012 103 36 For how many months did you breastfeed (NAME)? IF LESS THAN ONE MONTH, RECORD “00” MONTHS. Months ................................ |___|___| 37 How many times did you breast feed yesterday during the day or at night? IF ANSWER IS NOT NUMERIC, PROBE FOR APPROPRIATE NUMBER No. of feeding within 24 hrs Can’t Remember………………..….9 38 Did (NAME) drink anything from a bottle with a nipple yesterday during the day or at night? Yes …………………………....….…1 No ………………...……….....……2 Don’t Know ………………….…....9 39 Did (NAME) start drinking/eating anything other than breast milk? Yes…………………..………….1 No……………………..………..2 Don’t Know…………………….9  43  43 40 Now I would like to ask you about liquids or foods (NAME) had yesterday during the day or at night. Did (NAME) drink/eat: READ THE LIST OF LIQUIDS (A THROUGH E, STARTING WITH “BREAST MILK”). YES NO DK A. Breast milk? ……………….…1 2 9 B. Plain water? …………....…......1 2 9 C. Commercially produced infant formula? ………………….1 2 9 D. Any fortified, commercially available infant and young child food” [e.g. Cerelac]? ……………….....1 2 9 E. Any (other) porridge or gruel? ……………..…...1 2 9 41 PLEASE FILL OUT THE FOLLOWING TABLE WITH THE ANSWERS TO THE QUESTIONS BELOW: Now I would like to ask you about (other) liquids or foods that (NAME) may have had yesterday during the day or at night. I am interested in whether your child had the item even if it was combined with other foods. Did (NAME) drink/eat: GROUP 1:DAIRY YES NO DK A. CHECK Q.40C – IF QUESTION 40C IS YES, CIRCLE YES FOR THIS QUESTION Commercially produced infant formula? ……………………1 2 9 B. Milk such as tinned, powdered, or fresh animal milk? ……………………1 2 9 C. Cheese, yogurt, or other milk products? ……………………1 2 9 GROUP 2: GRAIN YES NO DK D. CHECK Q.40D – IF QUESTION 40D IS YES, CIRCLE YES FOR THIS QUESTION CS-23 Ethiopia, Final Evaluation, Save the Children, December 2012 104 Any fortified, commercially available infant and young Child food (e.g. Cerelac)? ……………………1 2 9 E. CHECK Q.40E – IF QUESTION 40E IS YES, CIRCLE YES FOR THIS QUESTION Any (other) porridge or gruel? ……………………1 2 9 F. Bread, rice, noodles, or other foods made from grains? ……………………1 2 9 G. White potatoes, white yams, manioc, cassava, or any other foods made from roots? ……………………1 2 9 GROUP 3: VITAMIN A RICH VEGETABLES YES NO DK H. Pumpkin, carrots, squash, or sweet potatoes that are yellow or orange inside? …….…………..…1 2 9 I. Any dark green leafy vegetables? …….………….…1 2 9 J. Ripe mangoes, papayas or (INSERT ANY OTHER LOCALLY AVAILABLE VITAMIN A-RICH FRUITS)? …………..………1 2 9 K. Foods made with red palm oil, palm nut, palm nut pulp sauce? …….……………1 2 9 GROUP 4: OTHER FRUITS/VEGETABLES YES NO DK L. Any other fruits or vegetables like oranges, grapefruit or pineapple? ………...…….....1 2 9 GROUP 5: EGGS YES NO DK M. Eggs? …………..……1 2 9 GROUP 6: MEAT, POULTRY, FISH YES NO DK N. Liver, kidney, heart or other organ meats? ………………...1 2 9 O. Any meat, such as beef, pork, lamb, goat, chicken, or duck? …………...……1 2 9 P. Fresh or dried fish or shellfish? ……………...…1 2 9 Q. Grubs, snails, insects, and other small protein food? ……………...…1 2 9 GROUP 7: LEGUMES/NUTS YES NO DK R. Any foods made from beans, peas, lentils, or nuts? …………...……1 2 9 GROUP 8: OILS/FATS YES NO DK S. Any oils, fats, or butter, or foods made with any of these? …………........…1 2 9 T. CHECK 41A – 41S: HOW MANY FOOD GROUPS (GROUPS 1-8 IN ABOVE TABLE) HAVE AT LEAST 1 ‘YES’ CIRCLED? Number of Groups GROUP 9: OTHER FOODS YES NO DK U. Tea or coffee? ………………1 2 9 V. Any other liquids? ………………1 2 9 W. Any sugary foods, such as chocolates, candy, sweets, pastries, cakes, or biscuits? ………………1 2 9 CS-23 Ethiopia, Final Evaluation, Save the Children, December 2012 105 X. Any other solid or soft food? ………………1 2 9 42 How many times did (NAME) eat solid, semi-solid, or soft foods other than liquids yesterday during the day or at night? IF CAREGIVER ANSWERS SEVEN OR MORE TIMES, RECORD “7” WE WANT TO FIND OUT HOW MANY TIMES THE CHILD ATE ENOUGH TO BE FULL. SMALL SNACKS AND SMALL FEEDS SUCH AS ONE OR TWO BITES OF MOTHER’S OR SISTER’S FOOD SHOULD NOT BE COUNTED. LIQUIDS DO NOT COUNT FOR THIS QUESTION. DO NOT INCLUDE THIN SOUPS, WATERY GRUELS, OR ANY OTHER LIQUID. USE PROBING QUESTIONS TO HELP THE RESPONDENT REMEMBER ALL THE TIMES THE CHILD ATE YESTERDAY Number of Times Don’t Know…………………….9 IV. VITAMIN A SUPPLEMENTATION 43 Has (Name) ever received a Vitamin A dose (like this/any of these)? SHOW COMMON TYPES OF AMPULES/CAPSULES/SYRUPS Yes…………….………….…..…..…1 No…………………………….….….2 Don’t know…………….………...… 9  45  45 44 Did (Name) receive a Vitamin A dose within the last 6 months? Yes………...……..…………….……1 No………………………………..….2 Don’t know………..…………..…… 9 V. CHILD IMMUNIZATIONS 45 Do you have a card or child health booklet where (Name’s) vaccinations and Vitamin A (capsules) are written down? If yes: May I see it please? Yes………..........……………………1 No………..………………………….2 Don’t know……………..………….. 9 48  48 46 COPY VACCINATION DATE FOR DPT1, DPT3, DPT-hepB￾Hib1, DPT-hepB-Hib3, MEASLES AND VITAMIN A FROM THE CARD OR BOOKLET. IF VACCINES ARE NOT RECORDED IN CHILD HEALTH CARD OR BOOKLET, Day Month Year DPT1……|___||___|/|___||___|/|___||___|___||___| DPT3……|___||___|/|___||___|/|___||___|___||___| DPT-hepB￾Hib1……..|___||___|/|___||___|/|___||___|___||___| CS-23 Ethiopia, Final Evaluation, Save the Children, December 2012 106 FILL IN 99/99/9999. DPT-hepB￾Hib3….…..|___||___|/|___||___|/|___||___|___||___| Measles…...|___||___|/|___||___|/|___||___|___||___| Vitamin A…|___||___|/|___||___|/|___||___|___||___| 47 Has (NAME) received any vaccinations that are not recorded on this card, including vaccinations given during immunization campaigns? Yes………………….………………….……….……1 No………………………...………………….…...….2 Don’t know……………….…………………...……. 9  51  51 48 Has (NAME) received a DPT vaccination, that is, an injection given in the thigh, sometimes at the same time as polio drops? Yes………………………...……………….…..……1 No………………………………………………..….2 Don’t know……………………………………….... 9  50  50 49 How many times? NUMBER OF TIMES ...................... 50 Did (Name) ever receive an injection in the arm to prevent Measles? Yes…………………………………………..….....…1 No………………………..…………………….....….2 Don’t know…………………….……………….……9 VI. MALARIA - TREATMENT OF FEVER OF CHILD 51 Has (Name) been ill with fever at any time in the last 2 weeks? Yes……………………..…...………………………1 No………………………………………….…….….2 Don’t know…………………………..……………. 9  57  57 52 Did you seek advice or treatment for the fever? Yes…………………………..…………………..…1 . No………………………………………………….2  57 53 How many days after the fever began did you first seek treatment for (Name)? Same day………………………..……………...….0 Next day………………….………………………...1 Two or more days……………………………….…2 54 Where did you first go for advice or treatment? IF SOURCE IS HOSPITAL, HEALTH CENTER, OR CLINIC, WRITE THE NAME OF THE HEALTH FACILITY Hospital.....................................................................1 Health Center............................................................2 Health Post................................................................3 PVO Center…….......................................................4 Clinic..........................................................................5 CS-23 Ethiopia, Final Evaluation, Save the Children, December 2012 107 PLACE. _______________________________ (NAME OF PLACE) Field/Community Health Worker...............................6 Other Health Facility...................................................7 ________________________________________ (SPECIFY) OTHER SOURCE Traditional Practitioner...............................................8 Shop............................................................................9 Pharmacy...................................................................10 Community Distributors...........................................11 Friend/Relative.........................................................12 Other.........................................................................13 ___________________________________________ (SPECIFY) 55 At any time during the illness did (Name) take any drugs for the fever? Yes………………………..….………………..……1 No………………………………………….……….2 Don’t know……………………..……………....…. 9  57  57 CS-23 Ethiopia, Final Evaluation, Save the Children, December 2012 108 56 What drugs did (Name) take? Any other drugs? RECORD ALL MENTIONED. ASK TO SEE DRUG(S) IF TYPE OF DRUG IS NOT KNOWN. IF TYPE OF DRUG IS STILL NOT DETERMINED, SHOW TYPICAL ANTIMALARIAL DRUGS TO RESPONDENT FOR EACH ANTIMALARIAL MEDICINE ASK: How long after the fever started did (NAME) start taking the medicine? CIRCLE THE APPROPRIATE CODES: SAME DAY = 0 NEXT DAY AFTER THE FEVER = 1 TWO OR MORE DAYS AFTER THE FEVER = 2 DON’T KNOW = 9 ANTI-MALARIAL A. Fansidar……...... ….…0 1 2 9 B. Chloroquine………….0 1 2 9 C. Quinine………….…...0 1 2 9 D. ACT (Coartum)….…..0 1 2 9 OTHER DRUGS E. ASPRIN………….….0 1 2 9 F. PARACETAMOL…...0 1 2 9 X. Other………..…...….0 1 2 9 VII. MALARIA – ITN USE 57 Does your household have any mosquito nets that can be used while sleeping? Yes……………...……………………..……………1 No…………………….…………………………….2 62 58 Who slept under a bed net last night? If ANYONE OTHER THAN THE CHILD IS MENTIONED, RECORD OTHER. No One…………………….………..………..…….0 Child (Name)……………………………….…..…..1 Other ……………….................................................2  62  62 59 Which type of bed net did (Name) sleep under last night? PROBE: pre-treated net is soaked at household level by the family. Permanent is treated at factory level and supplied to the community. Permanent Net…………………………….…...…..1 Pretreated Net…………………….…………..……2 Other Net…………………………….………..…...3  62  62 60 Was the bed net that (Name) slept under last night ever soaked or dipped in a liquid treated to repel mosquitoes or bugs? Yes……………………………….….…..…………1 No…………………….…………………………….2 Don’t know………….……………………...……... 9 62 62 CS-23 Ethiopia, Final Evaluation, Save the Children, December 2012 109 61 How long ago was the net last soaked or dipped in a liquid treated to repel mosquitoes or bugs? IF LESS THAN 1 MONTH AGO, RECORD 00 MONTHS. IF LESS THAN 2 YEARS AGO, RECORD MONTHS AGO. IF 12 MONTHS AGO OR 1 YEAR AGO, PROBE FOR EXACT NUMBER OF MONTHS. Months………………………….. . More than 2 years ago…………………….….....2 Don’t Know………………………..……...…….9 VIII. CONTROL OF DIARRHEA 62 Has (Name) had diarrhea in the last two weeks? [DEFINITION OF DIARRHEA: 3 or more loose stools in 24 hours] Yes…………….……………………..…………1 No……………………………………………….2 Don’t know………………………….…………. 9  69  69 63 Was s/he given any of the following to drink at any time since s/he started having diarrhea: READ CHOICES ALOUD: a) A fluid made from a special packet called (local name for ORS packet)? b) A pre-packaged ORS liquid? c) A government-recommended homemade fluid? SHOW ORS PACKAGE. Yes No DK A. Fluid from ORS Packet……….…1 2 9 B. ORS Liquid…………………........1 2 9 C. Homemade fluid………................1 2 9 CS-23 Ethiopia, Final Evaluation, Save the Children, December 2012 110 64 What was given to treat the diarrhea? Anything else? If answer pill or syrup, show local packaging for zinc and ask if the child received this medicine RECORD ALL MENTIONED. Nothing........................................................................1 Fluid From Ors Packet.................................................2 Home-Made Fluid........................................................3 Pill Or Syrup, Zinc...……...........................….......…..4 Pill Or Syrup, Not Zinc...............................................5 Injection.......................................................................6 (Iv) Intravenous...........................................................7 Home Remedies/Herbal Medicines.............................8 Other……………………………………..……….….9 ____________________________________________ (SPECIFY) 65 When (NAME) had diarrhea, did you breastfeed him/her less than usual, about the same amount, or more than usual? Less..............................................................................1 Same.............................................................................2 More.............................................................................3 Child Not Breastfed.....................................................4 Don’t Know.................................................................9 66 When (NAME) had diarrhea, was he/she offered less than usual to drink, about the same amount, or more than usual to drink? Less..............................................................................1 Same............................................................................2 More............................................................................3 Nothing to Drink.........................................................4 Don’t Know.................................................................9 67 Was (NAME) offered less than usual to eat, about the same amount, or more than usual to eat? Less..............................................................................1 Same............................................................................2 More.............................................................................3 Nothing to Eat..............................................................4 Don’t Know..................................................................9 CS-23 Ethiopia, Final Evaluation, Save the Children, December 2012 111 68 During the period when (NAME) was recovering from diarrhea, did you give him/her less than usual to drink, about the same amount, or more than usual to drink? Less..............................................................................1 Same............................................................................2 More............................................................................3 Nothing to Drink.........................................................4 Don’t Know.................................................................9 IX. ARI/PNEUMONIA 69 Has (Name) had an illness with a cough that comes from the chest (cough or difficult Breathing) at any time in the last two weeks? PROBE THE MOTHER IF SHE DOES NOT UNDERSTAND COUGH THAT COMES FROM THE CHEST. Yes……………………………………………..……1 No……………………..…………………………….2 Don’t know……………..………………………….. 9  73  73 70 When (Name) had an illness with a cough, did he/she have trouble breathing or breathe faster than usual with short rapid or difficult breathing? Yes………………………………………….………1 No…………………………………..……………….2 Don’t know………………….……………….……. 9  73  73 71 Did you seek advice or treatment for the cough/fast breathing? Yes………………………..…...……………………1 No…………………………………………..……….2  73 72 Who gave you advice or treatment? Anyone else? RECORD ALL MENTIONED. Doctor………………………………………...…….1 Nurse……………………….....................................2 Health Officer …..………………………………….3 Health assistant……………………………………..4 Trained Community Health Worker………………..5 Other………………………………………………..6 72b) What treatment (NAME) got for cough or difficult breathing? PROBE TO DIFFERENTIATE SAFE REMEDY AND ANTI-BIOTIC Got only advice (to relief cough with safe remedy)….1 Got an anti-biotic treatment …………………………..2 Got treatment and referred to Hospital………………..3 Don’t know……………………………………………9 CS-23 Ethiopia, Final Evaluation, Save the Children, December 2012 112 X. WATER AND SANITATION 73 Do you treat your water in any way to make it safe for drinking? Yes……………………………...….…………..……1 No………………………………………………..….2  75 74 If yes, what do you usually do to the water to make it safer to drink? ONLY CHECK MORE THAN ONE RESPONSE IF SEVERAL METHODS ARE USUALLY USED TOGETHER, FOR EXAMPLE, CLOTH FILTRATION AND CHLORINE. Let it stand and settle/sedimentation……….....….….1 Strain it through cloth……………….…….….……...2 Boil……..……………………………….……......….3 Add bleach/Chlorine…………...……………..…......4 Water filter (Ceramic, sand, composite)………….....5 Solar Disinfection………….……………...………...6 Other...........................................................................7 Don’t Know…….………………………..…………..9 75 Can you show me where you usually wash your hands and what you use to wash hands? ASK TO SEE AND OBSERVE Inside/near toilet facility…………………………….1 Inside/near kitchen/cooking place…………………..2 Elsewhere in yard……………….…………………...3 Outside yard………………………………………....4 No specific place……….............................................5 No permission to see……………………….…..……9  77  77 76 OBSERVATION ONLY: IS THERE SOAP OR DETERGENT OR LOCALLY USED CLEANSING AGENT? THIS ITEM SHOULD BE EITHER IN PLACE OR BROUGHT BY THE INTERVIEWEE WITHIN ONE MINUTE. IF THE ITEM IS NOT PRESENT WITHIN ONE MINUTE CHECK NONE, EVEN IF BROUGHT OUT LATER. Soap……………………………………………..….1 Detergent……………………………………………2 Ash…………………………………..………….…...3 Mud/sand……………………………………............4 None…………………………….……………..…….5 Other ………………….………………………..….. 6 CS-23 Ethiopia, Final Evaluation, Save the Children, December 2012 113 XI. RESPONDENT’S HOUSEHOLD WEALTH No. Questions and Filters Coding Categories Skip 77 Does any member of this household own: A bicycle? A motorcycle? An animal-drawn cart? A car or truck? Television? Radio? Tape Recorder? YES NO Bicycle…..……. ………………………..1 2 Motorcycle………………………............1 2 Animal-Drawn Cart .......................……..1 2 Car/Truck . . . . …………....……….…...1 2 Television………………..……………...1 2 Radio………………….………………...1 2 Tape Recorder……………………….…..1 2 78 Does any member of this household own any land that can be used for agriculture? Yes……………………………….……….1 No………………………………………...2  80 79 How many (LOCAL UNITS) of agricultural land do members of this household own? IF MORE THAN 97, ENTER '97'. IF UNKNOWN, ENTER '99'. Local units…………………. _____________________________ Specify 80 Does this household own any livestock, herds, or farm animals? Yes……………………………..…..…….1 No…………………………..…………....2  82 81 How many of the following animals do this household own? Cattle? Milk cows, oxen, or bulls? Horses, donkeys, or mules? Camels? Goats? Sheep? Chickens? Cattle Cows/Oxen/Bulls Horses/Donkeys/Mules Camels Goats Sheep Chickens CS-23 Ethiopia, Final Evaluation, Save the Children, December 2012 114 IF NONE, ENTER '00'. IF MORE THAN 97, ENTER '97'. IF UNKNOWN, ENTER '99'. 82 Does any member of this household have an account with a bank/credit association/micro finance? If YES, How much is your estimated Annual income in Birr? WRITE THE AMOUNT OF ANNUAL INCOME IN THE SPACE PROVIDED. Yes………………………………………..….…1 No.……………………………………….….…..2 XII. ANTHROPOMETRICS 83 May I weigh (Name)? Yes……………………………..…………………1 No…………………………………………..…….2 |_____||_____| . |_____| Kilograms MUAC /_____//_____/. /_____/  end THANK THE MOTHER FOR THE INTERVIEW. Interview End Time__________________ CS-23 Ethiopia, Final Evaluation, Save the Children, December 2012 115 Annex 2: Tabulation Plan Ethiopia Child Survival Program Indicators and Tabulation Plan for Final Knowledge, Practice, and Coverage (KPC) Survey Questionnaire (Including Revised Rapid CATCH) Shebedino & Lanfero Districts, SNNPR March/April 2012 Conducted by Save the Children/USA USAID/CSHGP Grant Number M/OAA/GH-07-003 CS-23 Ethiopia, Final Evaluation, Save the Children, December 2012 116 PRIORITY CHILD HEALTH INDICATORS Maternal and Newborn Care 1. Percentage of mothers with children age 0-23 months who received at least two Tetanus toxoid before the birth of the youngest child 2. Percentage of children age 0-23 months whose births were attended by skilled personnel 3. Percentage of children age 0-23 months who received a post-natal visit from an appropriately trained health worker within three days after birth Breastfeeding and Infant and Young Child Feeding 4. Percentage of children age 0-5 months who were exclusively given breast milk the day prior to the interview 5. Percent of children age 6-23 months fed according to a minimum of appropriate feeding practices Vitamin A Supplementation 6. % of children age 6-23 months who received a dose of Vitamin A in the last 6 months: card verified or mother’s recall Immunization 7. Percent of children aged 12-23 months who received measles vaccine according to the vaccination card or mother’s recall by the time of the survey 8. Percent of children aged 12-23 months who received DTP1 according to the vaccination card or mother’s recall by the time of the survey 9. Percent of children age 12-23 months who received DTP3 according to the vaccination card or mother’s recall by the time of the survey Malaria 10. Percentage of children age 0-23 months with a febrile episode during the last two weeks who were treated with an effective anti-malarial drug within 24 hours after the fever began 11. Percentage of children age 0-23 months who slept under an insecticide-treated bed net the previous night Control of Diarrhea 12. Percentage of children age 0-23 months with diarrhea in the last two weeks who received oral rehydration solution (ORS) and/or recommended home fluids Acute Respiratory Infections 13. Percentage of children age 0-23 months with chest-related cough and fast and/or difficult breathing in the last two weeks who were taken to an appropriate health provider Water and Sanitation 14. Percentage of households of children age 0-23 months that treat water effectively 15. Percentage of mothers of children age 0-23 months who live in a household with soap at the place for hand washing Anthropometrics 16. Percentage of children age 0-23 months who are underweight (-2SD for the median weight for age, according to WHO/NCHS reference population) CS-23 Ethiopia, Final Evaluation, Save the Children, December 2012 117 Question Number Indicator How to Calculate the Indicator 4-7 Tetanus Toxoid % of mothers with children age 0-23 months who received at least 2 tetanus toxoid vaccinations before the birth of their youngest child #of mothers with children age 0-23 months who received at least 2 tetanus toxoid vaccinations before the birth of their youngest child (Q5 + Q7 >=2) AND (Q5 <> 9 AND Q7 <> 9) ____________________________________________ Total # of mothers of children age 0-23 months in the survey x 100 14 Skilled Delivery Assistance % of children age 0-23 months whose births were attended by skilled personnel # of children age 0-23 months whose birth was attended by a doctor, nurse, midwife or auxiliary midwife (Q14 = 1, 2, 3, 4, 5, 6) ____________________________________________ Total # of mothers of children age 0-23 months in the survey x 100 15-16 Clean Cord Care # of births using clean instrument (Q16 = 1 or 2 or 4 or 6 or 8) ___________________________________________ Number of mothers of children age 0-11 months in the survey x 100 17-18 Newborns Dried and Wrapped # of newborns who were dried and wrapped with a warm cloth or blanket immediately after birth( before placenta delivered) ( Q17 = 1 and Q18 = 1) ____________________________________________________ # of mothers of children age 0-11 months in the survey x 100 19 Placement at Birth Percent of children aged 0-23 months who were placed with the mother immediately after birth (Q19= 1) __________________________________________ Total no. of children aged 0-23 months x 100 CS-23 Ethiopia, Final Evaluation, Save the Children, December 2012 118 Question Number Indicator How to Calculate the Indicator 20 Immediate Breastfeeding Percent of children aged 0-23 months who were immediately breastfed at birth (Q20= 1) ___________________________________________ Total no. of children aged 0-23 months x 100 21-23 Postpartum Contact Percent of mothers who had at least one postpartum check-up (Q21= 1) _______________________________________________ Total no. of mothers with children less than 24 months x 100 24 Knowledge of Maternal Danger Signs Percent of mothers able to report at least two known maternal danger signs during the postpartum period (No. of mothers with at least two responses= 1 through 3) _______________________________________________ Total no. of mothers with children less than 24 months x 100 25 Knowledge of Neonatal Danger Signs Percent of mothers able to report at least two known neonatal danger signs (No. of mothers with at least two responses= 1 through 5) _______________________________________________ Total no. of mothers with children less than 24 months x 100 27 Maternal Knowledge of Child Danger Signs Percent of mothers of children aged 0-23 months who know at least two signs of childhood illness that indicate the need for treatment (No. of mothers who report at least two of the signs listed in responses 1 through 7) ____________________________________ Total no. of mothers of children aged 0-23 months x 100 CS-23 Ethiopia, Final Evaluation, Save the Children, December 2012 119 Question Number Indicator How to Calculate the Indicator 28 Maternal Vitamin A Supplementation Percent of mothers who received a Vitamin A dose during the first two months after delivery (Q28= 1) _______________________________________________ Total no. of mothers with children less than 24 months x 100 29-31 Post-Natal Visit to Check on the Newborn % of children age 0-23 months who received a post-natal visit from an appropriate trained health worker within three days after birth # of children age 0-23 months who received a post-natal visit (Q29=1) AND within three days after birth (Q30U = 0) or (Q30U= 1 and (Q30N <= 3)) AND by an appropriate health worker (Q31= 1, 2, 3, 4, 5, 6) ____________________________________________ Total # of children age 0-23 months in the survey U refers to the units of time (hours, days, weeks) and N refers to the corresponding number (Q30U=0 and Q30N=12 means 12 hours) x 100 32-33 Immediate Initiation of Breastfeeding Percent of infants less than 12 months of age who were put to the breast within one hour of delivery # of infants less than 12 months of age who were put to the breast within one hour of delivery (Q33= 00 (immediate) # of mothers of children age 0–11 months in the survey x 100 34-37 Colostrum % of mothers who gave colostrum Number of newborns who received colostrum (Q34= 1) ____________________________________________________ # of mothers of children age 0–11 months in the survey x 100 CS-23 Ethiopia, Final Evaluation, Save the Children, December 2012 120 Question Number Indicator How to Calculate the Indicator 38 Bottle Use 0-23 months Percent of children aged 0-23 months who had anything by bottle in the 24 hours preceding survey # children aged 0-23 months with response = 1 for Q.38 ______________________________________________ Total # children aged 0-23 months x 100 39-42 Exclusive Breastfeeding ** % of children age 0-5 months who were exclusively breastfed during the last 24 hours **NOTE: If any answers to Q40 or Q41 are coded as Don’t Know (9) or Missing (Blank) then the entire case should not be included in the numerator and denominator # of children age 0-5 months who drank breast milk in the previous 24 hours (Q40A= 1) AND Did not drink any other liquids in the previous 24 hours (Q40B<> 1 and Q40C<>1 and Q40D<>1, and Q40E<> 1) AND Was not given any other foods or liquids in the previous 24 hours (Q41T=0 AND Q41U=2 and Q41V=2 and Q41W=2 and Q41X=2) ____________________________________________ Total # of children age 0-5 months in the survey** x 100 39-42 Infant and Young Child Feeding Percent of children age 6-23 months fed according to a minimum of appropriate feeding practices See Below CS-23 Ethiopia, Final Evaluation, Save the Children, December 2012 121 Question Number Indicator How to Calculate the Indicator 43-46 Vitamin A Supplementation % of children age 6-23 months who received a dose of Vitamin A in the last 6 months: card verified or mother’s recall # of children age 6-23 months who received a dose of Vitamin A in the last 6 months [(Q43=1) AND (Q44=1)] OR [(Q45=1) AND (Q46Vitamin A Month <> 99 AND Q46Vitamin A Year <> 9999) AND (Date of Interview - Date of Vitamin A<=6 months)] ____________________________________________ x100 Total # of children age 6-23 months in the survey 45-46, 50 Measles Vaccination % of children aged 12-23 months who received measles vaccine according to the vaccination card or mother’s recall by the time of the survey # of children age 12-23 months who received a measles vaccination by the time of the interview as seen on the card (Q45=1) AND (Q46MM <> 99 AND Q46MY <> 9999) OR recalled by the mother (Q50 = 1) ____________________________________________ Total # of children age 12-23 months in the survey x 100 45-49 Access to Immunization Services % of children aged 12-23 months who received DTP1 according to the vaccination card or mother’s recall by the time of the survey # of children who received DTP1 at the time of the survey according to the vaccination card/child health booklet [(Q45=1) AND (Q46DTP1M <> 99 AND Q46DTP1Y <> 9999)] OR mother’s recall [(Q48=1) AND (Q49>=1)] __________________________________________ Total # of children age 12-23 months in the survey x 100 CS-23 Ethiopia, Final Evaluation, Save the Children, December 2012 122 Question Number Indicator How to Calculate the Indicator 45-49 Health Systems Performance Regarding Immunization Services % of children aged 12-23 months who received DTP3 according to the vaccination card or mother’s recall by the time of the survey #of children who received DTP3 at the time of the survey as verified by vaccination card or child health booklet (Q45=1) AND (Q46DTP3M <> 99 AND Q46DTP3Y <> 9999) OR Recalled by the mother [(Q48=1) AND (Q49>=3)] ____________________________________________ Total # of children age 12-23 months in the survey x 100 51-55 Treatment of Fever in Malarious Zones % of children age 0-23 months with a febrile episode during the last two weeks who were treated with an effective anti￾malarial drug within 24 hours after the fever began # of children age 0-23 months with a febrile episode during the last two weeks (Q51 = 1) AND who sought treatment within 24 hours (Q52=1) AND (Q53 = 0 OR Q53=1) AND Was treated with an appropriate anti-malarial drug (Q55 = 1) AND ( (Q56A <= 1or Q56B <= 1 or Q56C <= 1 or Q56D <=1 ___________________________________________ Total # of children age 0-23 months with a febrile episode in the last two weeks (Q51 = 1) Adjust tabulation plan according to what malaria treatment is appropriate based on national protocols. x 100 CS-23 Ethiopia, Final Evaluation, Save the Children, December 2012 123 Question Number Indicator How to Calculate the Indicator 57-59 ITN Use % of children age 0-23 months who slept under an insecticide-treated bed net the previous night # of children age 0-23 months who slept under an insecticide￾treated bed net the previous night ((Q57 =1) AND (Q58 =1)) AND (Q59 = 1) ___________________________________________ Total # of children age 0-23 months in the survey x 100 62 Careseeking for Diarrhea Percent of children aged 0-23 months with diarrhea in the last two weeks whose mothers sought outside advice or treatment for the illness No. of children with response= 1 for Q.62 ____________________________________________ No. of children with responses to Q.62 x 100 62-63 ORT Use % of children age 0-23 months with diarrhea in the last two weeks who received oral re-hydration solution and/or recommended home fluids # of children age 0-23 months with diarrhea in the last two weeks (Q62 = 1) AND who received oral rehydration solution (ORS) and/or recommended home fluids (Q63A = 1 or Q63B = 1 OR Q63C = 1) ____________________________________________ Total # of children age 0-23 months who had diarrhea in the last two weeks (Q62=1) x 100 64 Zinc Treatment for Diarrhea Proportion of children aged 0-23 months with diarrhea in the last two weeks who were treated with zinc supplements No. of children with response=4 for Q.64 ________________________________________________ No. of children aged 2-23 months with responses=1 for Q.62 x 100 CS-23 Ethiopia, Final Evaluation, Save the Children, December 2012 124 Question Number Indicator How to Calculate the Indicator 65-68 Increased Fluid Intake During a Diarrheal Episode Percent of children aged 0-23 months with diarrhea in the last two weeks who were offered more fluids during the illness No. of children with response= 3 for Q.65 or Q.66 or Q.68 ____________________________________________ No. of children with response=1 for Q.62 x 100 67 Increased Food Intake During a Diarrheal Episode Percent of children aged 0-23 months with diarrhea in the last two weeks who were offered the same amount or more food during the illness No. of children with response=2 or 3 for Q.67 _____________________________________ No. of children with responses=1 for Q.62 x 100 69-72 Appropriate Care Seeking for Pneumonia % of children age 0-23 months with chest-related cough and fast and/or difficult breathing in the last two weeks who were taken to an appropriate health provider # of children age 0-23 months with chest-related cough and difficult breathing in the last two weeks (Q69=1) AND (Q70= 1) AND who were taken to an appropriate health provider (Q71=1) AND (Q72 = 1, 2, 3 or 4) ____________________________________________ Total # of children age 0-23 months with chest-related cough in the last two weeks (Q69=1) AND (Q70= 1) x 100 73-74 Point of Use % of households of children age 0-23 months that treat water effectively # of households of mothers of children age 0-23 months that treat water effectively (Q73=1) AND (Q74= 3, 4, 5 or 6) ____________________________________________ Total # of mothers of children age 0-23 months in the survey x 100 CS-23 Ethiopia, Final Evaluation, Save the Children, December 2012 125 Question Number Indicator How to Calculate the Indicator 75-76 Appropriate Hand Washing Practices % of mothers of children age 0-23 months who live in households with soap at the place for hand washing # of mothers of children age 0-23 months who live in households with soap at the place for hand washing (Q75 <=4) AND (Q76<=2)) ____________________________________________ Total # of mothers of children age 0-23 months in the survey x 100 83 Underweight % of children age 0-23 months who are underweight (-SD for the median weight for age, according to WHO/NCHS reference population) # of children age 0-23 months with weight/age -2 SD for median weight for age, according to WHO/NCHS reference population (Q83= 1) ___________________________________________ Total # of children age 0-23 months in the survey x 100 CS-23 Ethiopia, Final Evaluation, Save the Children, December 2012 126 IYCF Calculations The summary IYCF indicator measures several IYCF practices among children age 6-23 months. Based on WHO guidelines for feeding breastfed (2003) and non-breastfed (2005) children, the IYCF practices indicator is comprised of the following three components: 1. Continued breastfeeding or feeding of milk or milk products 2. Feeding solid/semi-solid food the minimum number of times per day according to age and breastfeeding status 3. Feeding the minimum number of food groups per day according to breastfeeding status Feeding Practice Breastfeeding Status Breastfed Non-breastfed Breastfed or Fed milk or milk products Continued breastfeeding (A) Fed milk or milk products (i.e. milk, dairy products or infant formula) (B) Fed (solid/semi-solid foods) minimum number of times per day 6- 8 months 9-23 months Two (C) Three Four (D) Four Fed minimum number of food groups4 6-23 months Three (E) Four (F) X= age of child in months FOR THE BREASTFED CHILD In order to meet the minimum appropriate feeding practices, the breast fed child must meet ALL the following conditions: 1. The child must be between 6 and 23 months of age 2. Be fed breast milk in the previous 24 hours 3. If the child is between 6 and 8 months, be fed at least 2 times during the previous 24 hours. If the child is between 9 and 23 months, be fed at least 3 times during the previous 24 hours. 4. Be fed a minimum of 3 of the 8 food groups. (See the footnote 1 below for more information.) Syntax for these conditions: Q10A=1 AND [((x >= 6 AND x < = 8) AND (Q12 >=2 and Q12 <=7)) OR ((x >= 9 AND x < = 23) AND (Q12 >=3 and Q12<=7))] AND Q11T>=3 4 Based upon a 24 hour recall of food groups fed to the child age 6-23 months. The eight food groups are: 1. infant formula, milk other than breast milk, cheese or yogurt (Q.11A OR Q.11B OR Q.11C); 2. foods made from grains, roots, and tubers, including porridge, fortified baby food from grains (Q.11D OR Q.11E OR Q.11F OR Q.11G); 3. vitamin A-rich fruits and vegetables (and red palm oil) (Q.11H OR Q.11I OR Q.11J OR Q.11K); 4. other fruits and vegetables (Q.11L); 5. eggs (Q.11M); 6. meat, poultry, fish, and shellfish (and organ meats) (Q.11N OR Q.11O OR Q.11P OR Q.11Q); 7. legumes and nuts (Q.11R); 8. foods made with oil, fat, butter (Q.11S). CS-23 Ethiopia, Final Evaluation, Save the Children, December 2012 127 FOR THE NON-BREASTFED CHILD In order to meet the minimum appropriate feeding practices, the non-breast fed child must meet ALL the following conditions: 1. The child must be between 6 and 23 months of age 2. Not fed breast-milk in the previous 24 hours 3. Be fed milk or milk products 4. Be fed at least four times during the previous 24 hours 5. Be fed a minimum of 4 of the 8 food groups. (See the footnote 1 below for more information.) Syntax for these conditions: [(Q10A <> 1) AND (Q10C = 1 OR Q11B = 1 OR Q11C = 1)] AND (Q12>=4 and Q12 <=7) AND Q11T >=4 How to Calculate the Indicator Infant and Young Child Practice Indicator Percent of infants and young children aged 6-23 months fed according to a minimum of appropriate feeding practices # breastfed children aged 6-23 months fed according to a minimum of appropriate feeding practices (with the number of children who meet the criteria for all of the following three indicators: Continued Breastfeeding Indicator AND Minimum frequency of feeding for breastfed child AND Minimum dietary diversity for breastfed child) OR # non-breastfed children aged 6-23 months (with the number of children who meet the criteria for all of the following three indicators: Fed milk or milk products for non-breastfed children Indicator AND Minimum frequency of feeding for non-breastfed child AND Minimum dietary diversity for non-breastfed child ) Total # children aged 6-23 months in the survey x 100 CS-23 Ethiopia, Final Evaluation, Save the Children, December 2012 128 Annex 3: Schedule – Lanfero CS-23 final KPC Survey Field Schedule, Lanfero District, March 30-April 4, 2012 Cluster Number Sample PA/Kebele Number of Clusters Date Team No. Day Departure Time 1 Amchie 1 2/4/12 2 4 7:00AM 2 Wonte Sostoro 1 1/4/12 3 3 7:00AM 3 Luke Kudussa 1 1/4/12 2 3 7:00AM 4 Rephe 1 1/4/12 4 3 7:00AM 5 Girar 1 31/3/12 3 2 7:00AM 6 Rephe Chefuna 1 2/4/12 1 4 7:00AM 7 Meded Gagebo 2 1/4/12 5 &1 3 7:00AM 8 Meded Kussaya 1 31/3/12 4 2 7:00AM 9 Torra Qiqora 1 31/3/12 5 2 7:00AM 10 Meja Torra 1 3/4/12 3 5 7:00AM 11 Gebaba 1 2/4/12 5 4 7:00AM 12 Shanqa Tuffa 1 2/4/12 4 4 7:00AM 13 Warsha Shanqa 1 2/4/12 3 4 7:00AM 14 Wotambo Gobe 1 3/4/12 2 5 7:00AM 15 Wotambo Balchie 1 3/4/12 5 5 7:00AM 16 Archuma Wonte 1 30/3/12 1 1 7:00AM 17 Archuma Golla 1 30/3/12 2 1 7:00AM 18 Wonte Doye 1 3/4/12 4 5 7:00AM 19 Wonte Boditi 1 3/4/12 1 5 7:00AM 20 Shofode Debar 1 30/3/12 3 1 7:00AM 21 Grinzilla Gogillo 2 4/4/12 3 &2 6 7:00AM 22 Edeneba Agawe 1 4/4/12 1 6 7:00AM 23 Sesso 2 4/4/12 4 &5 6 7:00AM 24 Grinzilla Shofode 1 30/3/12 4 1 7:00AM 25 Tora 01 2 31/3/12 1&2 2 7:00AM 26 Mito 01 1 30/3/12 5 1 7:00AM 30 CS-23 Ethiopia, Final Evaluation, Save the Children, December 2012 129 Annex 4: Schedule – Shebedino CS-23 final KPC Survey Field Schedule, Shebedino District, March 15-20, 2012 Cluster Number Sample PA/Kebele Number of Clusters Date Team No. Day Departure Time 1 Abela Lida 1 17/3/12 1 3 7:00AM 2 Mu/Kutala 1 16/3/12 2 2 7:00AM 3 Mo/Shondolo 1 16/3/12 3 2 7:00AM 4 Remeda 1 16/3/12 4 2 7:00AM 5 Ga/Haro 1 18/3/12 5 4 7:00AM 6 Ga/Hireye 1 18/3/12 4 4 7:00AM 7 Nure Dulecha 1 19/3/12 3 5 7:00AM 8 Du/Teberako 1 19/3/12 2 5 7:00AM 9 Dobe Negasha 1 17/3/12 5 3 7:00AM 10 Bonoya Miride 1 18/3/12 1 4 7:00AM 11 Konsore Ano 1 17/3/12 4 3 7:00AM 12 Howolso 1 17/3/12 3 3 7:00AM 13 Dobe Toga 1 18/3/12 2 3 7:00AM 14 Go/ Hebisha 1 20/3/12 5 6 7:00AM 15 Telamo 2 19/3/12 1 & 5 4 7:00AM 16 Ha/Shisho 1 20/3/12 4 6 7:00AM 17 Ha/Hagawo 1 20/3/12 3 6 7:00AM 18 Asa/Mero 1 20/3/12 2 6 7:00AM 19 Arbe/Mero 1 20/3/12 1 6 7:00AM 20 Mi/Genet 1 15/3/12 2 1 7:00AM 21 Mo/Negasha 1 15/3/12 3 1 7:00AM 22 Dila Change 1 19/3/12 4 5 7:00AM 23 Dila Afarara 1 16/3/12 5 2 7:00AM 24 DiramoAfarara 1 17/3/12 2 3 7:00AM 25 Dila Gunbe 1 18/3/12 3 4 7:00AM 26 Fura 1 16/3/12 1 2 7:00AM 27 Taramessa 1 15/3/12 5 1 7:00AM 28 Leku 01 1 15/3/12 1 1 7:00AM 29 Leku 03 1 15/3/12 4 1 7:00AM 30 CS-23 Ethiopia, Final Evaluation, Save the Children, December 2012 130 Annex 7: CHW Training Matrix Project Area (Name of District or Community) Type of CHW/ Trainees Official Government CHW or Grantee￾Developed Cadre Paid or Volunteer Number Trained Over Life of Project Focus of Training/Major Topics Covered Shebedino District Volunteer Community Health Promoters (vCHPs) Government Volunteer 1171 Maternal and child health services such as antenatal care, postnatal care, immunization, family planning, disease prevention and control, sanitation and water treatment practices and Community-IMNCI Female vCHPs Grantee developed Volunteer 561 Prenatal and postnatal services to improve the health services utilization of mothers and children Elders, Fathers and Grandmothers Grantee developed Volunteer 120 MCH, immunization, family planning, prenatal and postnatal care, disease prevention and control Mother-to-Mother Care Groups (MMCGs) Grantee developed Volunteer 50 Prenatal and postnatal care, key messages on maternal and child health Health Extension Workers (HEWs) Government Paid 66 Integrated Management of Childhood Illness (IMNCI) Health Extension Workers (HEWs) Government Paid 62 Integrated Community Case Management (iCCM) Health Workers (HWs) Government Paid 70 Integrated Management of Childhood Illness (IMNCI) Lanfero District Volunteer Community Health Promoters (vCHPs) Government Volunteer 290 Maternal and child health services such as antenatal care, postnatal care, immunization, family planning, disease prevention and control, sanitation and water treatment practices and Community-IMNCI CS-23 Ethiopia, Final Evaluation, Save the Children, December 2012 131 Project Area (Name of District or Community) Type of CHW/ Trainees Official Government CHW or Grantee￾Developed Cadre Paid or Volunteer Number Trained Over Life of Project Focus of Training/Major Topics Covered Female vCHPs Grantee developed Volunteer 540 Prenatal and postnatal services to improve the health services utilization of mothers and children Elders, Fathers and Grandmothers Grantee developed Volunteer 120 MCH, immunization, family planning, prenatal and postnatal care, disease prevention and control Mother-to-Mother Care Groups (MMCGs) Grantee developed Volunteer 15 Prenatal and postnatal care, key messages on maternal and child health Health Extension Workers (HEWs) Government Paid 44 Integrated Management of Childhood Illness (IMNCI) Health Extension Workers (HEWs) Government Paid 39 Integrated Community Case Management (iCCM) Health Workers (HWs) Government Paid 34 Integrated Management of Childhood Illness (IMNCI) SC Hawassa Sub-office level* Health Workers Government Paid 26 Health Management Information System (HMIS), Integrated Supportive Supervision (ISS), M&E Community leaders, HEWs and HEWs supervisors Government Paid & Volunteer 379 Community-IMNCI HEWs, HEWs supervisors, District representatives Government Paid 67 Training of Trainers (TOT) on Community-IMNCI, 20 key messages on maternal and child health care Health Workers Government Paid 19 Zinc treatment and management * Capacity building for HWs, including SC staff. CS-23 Ethiopia, Final Evaluation, Save the Children, December 2012 132 Annex 8: Evaluation Team Members and Their Titles S/ N NAME ORGANIZATION POSITION I SHEBEDINO DISTRICT TEAM 1 One HEP Expert Regional Health Bureau (RHB) HEP Expert 2 One Expert Integrated Family Health Program (IFHP) 3 Sidama Zone HEP Expert Sidama Zone Health Desk (SZHD) HEP Expert 4 One Expert Shebedino District Health Office MNCH Expert 5 Getenet Kebede SC/US Health Program Coordinator 6 Karen Z. Waltensperger SC/US Senior Advisor, Health-Africa 7 David Marsh SC/US Senior Child Survival Advisor (Team Leader) II LANFERO DISTRICT TEAM 1 One Expert L10k ICCM Expert 2 Silite Zone HEP Expert Silite Zone Health Desk (SZHD) HEP Expert 3 One Expert Lanfero District Health Office MNCH Expert 4 Worku Tefera SC/US CS M&E Coordinator 5 Habtamu Tilahun SC/US KOICA/SCK Project Officer 6 Dr. Hailu Tesfaye SC/US Child Survival Advisor and Hwassa Sub-office Manager 7 Dr. Abeba Bekele SC/US SC/US EtCO Health and Nutrition Unit Head 8 Peter Waiswa Consultant External Evaluator (Team Leader) CS-23 Ethiopia, Final Evaluation, Save the Children, December 2012 133 Annex 9: Evaluation Assessment Methodology Scope of Work-Final Evaluation External Team Leader (Consultant) USAID/CSHGP CS-23 Project-Ethiopia Child Survival Program Cooperative Agreement Number: GHS-A-00-07-00023-00 Project Background Infant and child mortality rates are basic indicators of a country’s socioeconomic situation and quality of life (UNDPA, 2007). The recently reported infant mortality rate in Ethiopia (EDHS2010) is 59 deaths per 1000 live births. The estimate of child mortality is 31 deaths per 1000 children surviving to 12 months of age, while the overall under-5 mortality rate is 88 deaths per 1000 live births. Sixty-seven percent of all deaths to children under-five in Ethiopia take place before a child’s first birthday. The 2010 EDHS shows a rapid decrease in infant and under-five mortality compared to the period 5-9 years ago. The levels are also considerably lower than those reported in the 2005 EDHS. Infant mortality has decreased by 23 percent, from 77 to 59 deaths per 1000 births, while under-five mortality has decreased by 28 percent, from 123 to 88 per 1000 births. In the 2010 EDHS, in the fortnight before the survey, seven percent of children under age five showed symptoms of ARI, 17 percent exhibited fever, and 13 percent experienced diarrhea. Treatment from a health facility or provider was sought for only 27 percent of the children with ARI symptoms and 24 percent of the children with fever symptoms. Treatment was sought from a health facility or health provider for 31 percent of children with diarrhea, and only 31 percent of children with diarrhea received a rehydration solution from an ORS packet or a recommended fluid. Children of urban mothers were more likely than children of rural mothers to receive treatment from a health facility or health provider when they were sick with symptoms of ARI, fever, or diarrhea. Save the Children was awarded a five-year Standard USAID/CSHGP Child Survival Project (CS-23) - Innovation for Scale: Enhancing Ethiopia's Health Extension Package in the Southern Nations and Nationalities People's Region (SNNPR) - to address four main causes of child death: (1) pneumonia, (2) malaria, (3) diarrheal diseases (that together account for 68% of under-five mortality); and (4) neonatal infection, responsible for half of all neonatal mortality. This project is being implemented in the two SNNPR woredas (districts) of Shebedino (Sidama Zone) and Lanfero (Silti Zone) since September 2007. The project reaches a total population of 366,898 in Shebedino (255,209) and Lanfero (111,689) Districts, including 69,491 children 0-59 months old; and 87,496 women of reproductive age. The selected CSHGP interventions are: Pneumonia Case Management (35%); Immunization (5%); Control of Diarrheal Diseases (20%); Prevention and Treatment of Malaria (PTM) (20%); and Newborn Care (20%). The project’s key implementation strategy is Community Case Management (CCM), supported by behavior change at the household and community levels. CS-23 Ethiopia, Final Evaluation, Save the Children, December 2012 134 Purpose of the final evaluationa  To determine the extent to which the project accomplished the results that were outlined in the DIP and to present the evidence of these accomplishments.  To describe key factors that contributed to what worked or did not work regarding some or all aspects of the program and to inform future program actions.  To provide a record of how these results were obtained, so USAID can share these results with others outside of the CSHGP program – including U.S. Congress – so that in￾country partners and future PVO grantees understand what should be done if they want to reproduce these results.  To demonstrate how this project contributes to global learning about community-based health programming. The final evaluation provides an opportunity for all project stakeholders to take stock of accomplishments to date and to listen to the beneficiaries at all levels: including mothers and caregivers, other community members and opinion leaders, health workers, health system administrators, local partners, other organizations and donors. Contract days: twenty-five (25)  Review project documents and draft interview and group discussion data collection tools prior to travel to Ethiopia (4 days).  Travel to/from Ethiopia (2 days).  Team planning meetings with partner(s) (1 day).  Field work and data collection, including planning, collecting, analyzing, and synthesizing findings from the project site with the MTE team (10 days).  Findings review analysis (1 day).  In-country debriefing presentation (1 day).  Draft report writing (6 days). Final evaluation methodology The Team Leader (external consultant) will lead the final evaluation, which will focus on outcomes and results as measured against the goal, objective, and intermediate results outlined in the project’s detailed implementation plan (DIP). The core final evaluation team will include representatives from the MOH/RHB, ZHD, DHO, and IFHP. Save the Children, team members will include: Senior Child Survival Advisor/CS-23 Technical Backstop; Senior Advisor, Health-Africa; Health Unit Head in Ethiopia, Child Survival Program Manager, Child Survival Program Coordinator, District Program Officers; and Child Survival M&E Coordinator. Save the Children will provide the Team Leader and other team members with key project documents and both qualitative and quantitative data prior to arrival in-country. Hard copies of these and other documents will be available upon arrival. The Team Leader and Save the a From guidelines for final evaluation, Child Survival and Health Grants Program, USAID/GH/HIDN/NUT, July 2011. CS-23 Ethiopia, Final Evaluation, Save the Children, December 2012 135 Children will negotiate a timetable for documents to be sent out and details of fieldwork plans through email exchange through a point-person to be designated. Guidance for key informant interviews and focus group discussion will be drafted by the Team Leader and sent to Save the Children prior to arrival in-country. Save the Children will provide key documents to the Team Leader prior to the evaluation and is responsible for ensuring that hard and electronic copies of key program documents are available in the Awassa Sub-oOffice for review, including but not limited to:  Detailed Implementation Plan (DIP);  Baseline and endline KPC survey results;  Baseline and endline Health Facility Assessment (HFA) results;  Report of the midterm evaluation;  Annual Reports for Years 1, 2, and 4;  Action Research - service utilization report (Shebedino District); and  Performance progress monitoring survey report. The core evaluation team will meet in the Save the Children Awassa Sub-office for a day of discussion and logistics review, led by the Team Leader. This is an opportunity to review data collection tools, identify and resolve last-minute problems, divide tasks among participants, and take care of final logistic arrangements. The evaluation team will spend 4-5 days in the field in each district (Shebedino and Lanfero) to collect qualitative data using interviews, observations and focus group discussions. Key contacts will likely include:  Key informant interviews (KII) with Federal Ministry of Health, international partners (UNICEF, WHO, USAID), Regional Health Bureau (RHB), Zonal Health Departments (ZHDs), District Health Offices (DHOs) MNCH and Health Extension Program (HEP) experts;  KII with Health Center (HC) and Health Post (HP) staffs (HC Heads/Under-five Clinic Nurses/HEWs);  KII with other Child Survival (CS) partners like Integrated Family Health Program (IFHP/JSI) Regional Manager and IFHP;  Focus Group Discussion (FGD) with vCHWs/Community;  Observation of HPs;  Observation of HCs; and  Visit to households in selected communities. At the end of each day of fieldwork, the final evaluation team may spend up to 1-1½ hours reviewing that day’s findings, triangulating information, and highlighting strengths and gaps identified. At the end of fieldwork, the core final evaluation team will meet in Awassa to present and consolidate findings, identify lessons learnt, and draft key recommendations to inform a preliminary de-briefing with key partners. CS-23 Ethiopia, Final Evaluation, Save the Children, December 2012 136 Tentative Schedule for Save the Children Ethiopia Child Survival Program Final Evaluation August 18-September 1, 2012 Dates Activities Sat, 18 Aug  Arrivals in Addis Ababa Sun, 19 Aug  Arrivals in Awassa Mon, 20 Aug  FE team meets in Awassa Sub Office: Review of evaluation schedule, sites selected for field visits, tools, and logistics arrangements and finalization of task assignments Tue, 21 Aug  Interviews with RHB, SZHD, HEP and CS expert, IFHP  Interviews with SC project (CS Program Manager, Health Program Coordinator, CS M&E Coordinator)  Orientation of evaluation team and planning Wed, 22 Aug  Travel to Shebedino/Lanfero (two teams)  Evaluation team meet with DHO  Interview DHO head and HEW supervisor  Interview HC head & document review  Interview ZHD HEP expert (Shebedino) Thu, 23 Aug  Interview ZHD HEP expert (Lanfero)  Interview HC head and document review  Interview HEWs and document review  Interviews CS-23 field staff  Interview with vCHWs Fri, 24 Aug  Interview HC head and document review  Interview/FGD with community/HH  Interview HEWs and document review  Interview vCHWs Sat, 25 Aug  Interview/FGD with community/HH  Interview with HEWs and document review  Interview with vCHWs Sun, 26 Aug  Teams return to Awassa Mon, 27 Aug  Write up and follow-up interviews, as necessary Tue, 28 Aug  Consolidation of findings, lessons learnt Wed, 29 Aug  Group work to draft recommendations  De-briefing with partners (Awassa) – one hour Thu, 30 Aug  Travel back to Addis  Interviews with FMOH (Health Unit Head), international partners (TBD)  De-brief at EtCO (Country Director) Fri, 31 Aug  Interview with PSI marketing and communication unit  Review of FE key findings with USAID/Ethiopia Sat, 1 Sep  Departures 2 Sep-30 Sep (6 days)  Report writing (First draft of report due to SC by 15 Sep 2012) CS-23 Ethiopia, Final Evaluation, Save the Children, December 2012 137 Core final evaluation team members: 1. External Consultant/Team Leader 2. Senior Child Survival Technical Advisor/Technical Backstop---Dr. David Marsh 3. Senior Advisor, Health-Africa---Karen Z. Waltensperger 4. Health and Nutrition Unit Head---Dr. Abeba Bekele 5. CS-23 Program Manager--Dr. Hailu Tesfaye 6. CS-23 Lanfero District Program Officer---Habtamu Tilahun 7. CS-23 Shebedino District Program Officer---Yachiso Yamo 8. CS-23 Health Program Coordinator---Getenet Kebede 9. CS M&E Coordinator---Worku Tefera 10. CS-23 Community Mobilization Officers at Shebedino- Aschalew Alemu 11. CS-23 Community Mobilization Officers at Lanfero-Abdulmuhin Nuri 12. CS-23 MNCHN Officers for Shebedino-Mintesinot 13. CS-23 MNCHN Officers for Lanfero-Chikssa Sultan 14. Representative from IFHP (to be confirmed) 15. Representative from RHB 16. Representative from Siltie ZHD 17. Representative from Sidama ZHD 18. Representative from Shebedino DHO 19. Representative from Lanfero DHO 20. Joined for field visits and development of recommendations by Luwei Pearson, UNICEF Ethiopia Health Unit Head. Responsibilities of External Team Leader (Consultant)  Review key project documents and assessments.  Draft final evaluation interview and focus group discussion guides and observation tools.  Lead the final evaluation in-country.  Lead a sub-team and conduct interviews with key stakeholders and partners at regional/Awassa, woreda, and national levels.  Present final evaluation preliminary findings and recommendations at de-briefings with country office director and key partners.  Write/assemble and submit final draft of evaluation report, including annexes, per deliverable schedule. Team Leader Deliverables  Draft data collection tools by 5 August 2012.  Draft summary of final evaluation key findings and key recommendations before leaving Ethiopia.  First draft of the final evaluation report to be submitted by 15 September 2012.  Final draft of evaluation report submitted by 30 September 2012. Evaluation Guidelines  Current Final Evaluation Guidelines, Child Survival and Health Grants Program, USAID/GH/HIDN/NUT (July 2011). CS-23 Ethiopia, Final Evaluation, Save the Children, December 2012 138 Final Evaluation In-Country Schedule Shebedino and Lanfero District, SNNPR, August 19-29, 2012 DATE ACTIVITY TIME RESPONSIBLE Sun, 19 Aug  Arrival to Hwassa - CS-23 PROGRAM MANAGER Mon, 20 Aug BREAKFAST  FE team meets in Hwassa Sub-office  Review of evaluation schedule, sites selected for field visits, tools, and logistics arrangements and finalization of task assignments  Orientation of FE team and planning LUNCH  Teams are assigned to their tasks (logistics and transport arrangement) DINNER 7:00-8:00 8:00-8:30 8:30-9:30 9:30-12:00 12:00-1:30 1:30-4:30 7:00-8:30 HEALTH PROGRAM COORDINATOR Tue, 21 Aug BREAKFAST  Interview with RHB Head  Interview with Sidama ZHD Head  Interview with IFHP Head LUNCH  Interview with CS Program Manager  Interview with Health Program Coordinator  Interview with M & E Coordinator 7:00-8:00 8:45-9:45 10:00-11:00 11:15-12:00 12:00-1:30 1:30-2:30 2:35-3:35 3:40-4:40 CONSULTANT Wed, 22 Aug BREAKFAST TRAVEL TO SHEBEDINO/LANFERO  Evaluation team meet with DHO (Lanfero)  Interview DHO Head (Lanfero)  Interview MNCH Expert (Lanfero)  Evaluation team meet with DHO (Shebedino)  Interview DHO Head (Shebedino)  Interview MNCH Expert (Shebedino) LUNCH  Interview HC Head (Leku/Tora) and document review (both districts)  Reviewing day’s finding 7:00-8:00 8:00-10:30 10:30-10:45 10:45-11:45 11:50-12:50 8:45-9:45 9:45-10:45 10:50-11:50 12:50-2:20 2:20-3:20 4:15-5:15 7:00-8:30 CS M&E COORDINATOR (LANFERO) HEALTH PROGRAM COORDINATOR (SHEBEDINO) CS-23 Ethiopia, Final Evaluation, Save the Children, December 2012 139 DINNER Thu, 23 Aug BREAKFAST TRAVEL TO SILTIE ZONAL HEALTH DEPARTMENT/SHEBEDINO  Interview Siltie ZHD Head (Lanfero/Werabe Town)  Interview HC Head (Abela HC) and document review (Shebedino)  Interview HC Head (Mito HC) and document review (Lanfero)  Interview HEWs and document review (Shebedino) LUNCH  Interviews CS-23 senior field staff (both districts)  Interview with vCHWs (both districts)  Reviewing day’s finding DINNER 7:00-8:00 8:00-8:45 8:45-9:45 8:45-10:00 10:45-11:45 10:45-11:45 12:00-1:30 1:45-2:45 3:30-5:00 6:30-7:30 7:30-8:30 CS M&E COORDINATOR (LANFERO) CS M&E COORDINATOR (SHEBEDINO) Fr 24 Aug BREAKFAST TRAVEL TO SHEBEDINO AND LANFERO  Interview HC Head and document review (Archuma Wonte/Dobe Toga HCs [both districts])  Interview/FGD with community/HH (both districts)  Interview HEWs and document review (both districts) LUNCH  Interview vCHWs (both districts)  Reviewing day’s finding DINNER 7:00-8:00 8:00-9:00 9:00-10:00 10:00-11:30 11:30-12:30 12:30-2:00 2:00-3:30 6:00-7:00 7:00-8:00 HEALTH PROGRAM COORDINATOR (SHEBEDINO) CS M&E COORDINATOR (LANFERO) CS-23 Ethiopia, Final Evaluation, Save the Children, December 2012 140 Sat 25 Aug BREAKFAST TRAVEL TO SHEBEDINO AND LANFERO  Interview/FGD with community/HH (both districts)  Interview with HEWs and document review (both districts)  Interview with vCHWs (both districts) LUNCH  Reviewing day’s finding DINNER 7:00-8:00 8:00-9:00 9:00-10:30 10:30-11:30 11:30-1:00 1:00-2:30 6:00-7:00 7:00-8:00 HEALTH PROGRAM COORDINATOR (SHEBEDINO) AND CS M&E COORDINATOR (LANFERO) Sun, 26 Aug BREAKFAST  Teams return to Hwassa 7:00-8:30 8:30-11:00 Mon, 27 Aug  Write up and follow-up interviews, as necessary Tue, 28 Aug  Consolidation of findings, lessons learnt Wed, 29 Aug  Group work on recommendations  De-briefing with partners (Hawassa) 8:30-1:30  Lanfero team will spend overnight at Butajira Town throughout the FE field work (50km away from Lanfero/Tora Town [25km is asphalt road to Butajira Town]).  Shebedino team will spend overnight in Awassa throughout the FE fieldwork.  In each district: three KII with HEWs; three KII with HWs; three interviews with vCHWs; two FGDs with community/HH.  The estimated time for one KII is 1hr.  The estimated time for one FGD & interview with vCHWs is 1:30hrs. At both FE field work sites there are no standard hotels, so having carry-out food/biscuits is important. CS-23 Ethiopia, Final Evaluation, Save the Children, December 2012 141 Final Evaluation Report Methodology Innovation for Scale: Enhancing Ethiopia's Health Extension Package in the Southern Nations and Nationalities People's Region (SNNPR) Purpose of the final evaluation 1. To determine the extent to which the project accomplished the results that were outlined in the DIP and to present the evidence of these accomplishments. 2. To describe key factors that contributed to what worked or did not work regarding some or all aspects of the program and to inform future program actions. 3. To provide a record of how these results were obtained, so USAID can share these results with others outside of the CSHGP program – including U.S. Congress – so that in￾country partners and future PVO grantees understand what should be done if they want to reproduce these results. 4. To demonstrate how this project contributes to global learning about community-based health programming. Methods The methods have been suggested as per below:  The Team Leader (external consultant) will lead the final evaluation, which will focus on outcomes and results as measured against the goal, objective, and intermediate results outlined in the project’s detailed implementation plan (DIP).  The evaluation team will spend 4-5 days in the field in each district (Shebedino and Lanfero) to collect qualitative data using interviews, observations and focus group discussions. Key contacts will likely include:  Key informant interviews (KII) with Federal Ministry of Health, international partners (UNICEF, WHO, USAID), Regional Health Bureau (RHB), Zonal Health Departments (ZHDs), District Health Offices (DHOs) MNCH and Health Extension Program (HEP) experts  KII with Health Center (HC) and Health Post (HP) staffs (HC heads/Under five clinic nurses/HEWs)  KII with other Child Survival (CS) partners like Integrated Family Health Program (IFHP/JSI) regional manager and IFHP  Focus Group Discussion (FGD) with vCHWs/Community  Observation of HPs  Observation of HCs  Visit to households in selected communities. Just like in the MTE, we will select variables and components to examine based on the project’s results framework at each level, including: 1) Use of life-saving interventions; 2) Access to and availability of interventions; 3) Quality of intervention delivery; 4) Demand for interventions; 5) Policy environment; and 6) Inputs and Activities, Drugs, Equipment, Supplies, Training. We will use a convenient sample for key informant interviews and focus groups. Interviews, inventories and focus groups will be carried out after signed informed consent. Teams will take CS-23 Ethiopia, Final Evaluation, Save the Children, December 2012 142 hand-written notes in English and digital photographs of illustrative documents should be made. At the end of each day of fieldwork, the final evaluation sub-teams teams should spend approximately one hour reviewing the day’s findings, highlighting strengths and gaps identified, and planning the field work for the following day. Objective 1. To determine the extent to w hich the project accomplished the results that were outlined in the DIP and to present the evidence of these accomplishments. This is mainly a quantitative evaluation and is already answered through the quantitative final survey. Access to key project evaluation documents and reports will be important. Objective 2. To describe key factors that co ntributed to w hat worked or did not w ork regarding some or all aspects of the program and to inform future program actions. This will be the main focus of the current evaluation. Qualitative methods including KIIs and FGDs will be done with the respondents outlined above. Objective 3. To provide a record of how these results w ere obtained, so USAID can share these results w ith others outside of the CS HGP program – including U.S. Congress – so that in-country partners and future PVO grantees understand what should be done if they want to reproduce these results. To be answered through the report: Objective 4. To demonstrate how this project contributes to global learning about community-based health programming. This will also be answered though FGDs and IDI, and also document review and analysis. Sampling TYPE OF RESPONDENT NUMBER COMMENTS e.g. location National and district level stakeholders Key informant interviews (KII) with Federal Ministry of Health international partners (UNICEF, WHO, USAID) Regional Health Bureau (RHB), Zonal Health Departments (ZHDs) District Health Offices (DHOs) MNCH and Health Extension Program (HEP) experts Child Survival (CS) partners Integrated Family Health Program (IFHP/JSI) Regional Manager IFHP Other Health Center (HC) and Health Post (HP) staffs CS-23 Ethiopia, Final Evaluation, Save the Children, December 2012 143 KII with Health Center (HC) Health Post (HP) staffs (HC heads/Under five clinic nurses/HEWs) Focus Group Discussion (FGD) with vCHWs/Community vCHWs/Community Observation of HCs and visits to Households in selected communities Observation of HPs Observation of HCs Visit to households in selected communities CS-23 Ethiopia, Final Evaluation, Save the Children, December 2012 144 USAID/CSHGP CS-23 Project Ethiopia Child Survival Program: Innovation for Scale: Enhancing Ethiopia’s Health Extension Package in the Southern Nations and Nationalities People’s Region (SNNPR) Final Evaluation Cooperative Agreement Number: GHS-A-00-07-00023-00 FGD Guide with District Stakeholders and Project Staff Target respondents: Federal Ministry of Health, international partners (UNICEF, WHO, USAID), Regional Health Bureau (RHB), Zonal Health Departments (ZHDs), District Health Offices (DHOs) MNCH and Health Extension Program (HEP) experts and KII with other Child Survival (CS) partners like Integrated Family Health Program (IFHP/JSI) regional manager and IFHP. Introduction: Save the Children has been implementing a Child Survival Project - Innovation for Scale: Enhancing Ethiopia's Health Extension Package in the Southern Nations and Nationalities People's Region (SNNPR). The project aimed to address four main causes of child death: (1) pneumonia, (2) malaria, (3) diarrheal diseases and (4) neonatal infection. The project has been implemented in the two SNNPR woredas (districts) of Shebedino (Sidama Zone) and Lanfero (Silti Zone) since September 2007. Save the Children and partners are now conducting a final evaluation of the project with the aim of documenting accomplishments; and understanding the key factors that contributed to what worked or did not work. These results will be shared with the MOH, and with in-country and external partners so as to inform country and global learning about community-based health programming. You have been identified as one of the key respondents to inform this evaluation. We would like you to take a few minutes of your time so as to answer some questions. Questions: 1. Please summarize for me the project rationale and design including components. 2. Project implementation – Who did what? Role of SC, districts, partners? 3. What else was going on in terms of health system beyond the project: MOH and partners at district, health facility and community level? 4. For each IR, we will discuss if it was successful or not based on set targets, and then discuss the reasons for its being achieved or failure to achieve it. The project coordinator will present the final quantitative survey findings and then we discuss each in terms of level of achievement and the responsible factors. STRENGTHS WEAKNESS RECOMMENDATION IR-1: Access and availability of child health services and supplies increased Clinical IMNCI coverage Community IMNCI coverage IR-2: Quality of child health services improved CS-23 Ethiopia, Final Evaluation, Save the Children, December 2012 145 HEW performance Functional supervisory system Functional health system IR-3: Knowledge and acceptance of key child health services and practices improved Endline KPC/HH surveys IR-4: Policy and social environment enabled Has there been facilitation of policy dialogue, debate, technical updates; provision of evidence of feasibility? 5. Please describe for me the project as you understand it. 6. What do you consider to be the main successes of the project? a) for newborn babies; b) for older children; c) overall in terms of health systems support. Probe: implementation of all pillars of IMNCI, including; clinical IMNCI training of HC staff, and Health Extension Workers (HEWs) in HPs; provision of supervision and supplies for IMNCI; and training and support to volunteer community health workers (vCHWs) and others to improve family practices through community-IMNCI. 7. How relevant was the proposed project and its objectives in terms of the potential impact for alleviating pressing health or service-delivery problems? 8. How consistent is this innovation with existing policies, regulations, national health plans and priorities? 9. How is this innovation compared to alternative ones in terms of feasibility, equity, cultural appropriateness and community preferences? 10. In your opinion, what factors facilitated project success? Probe for health system, partnership, community, sociocultural and gender factors that might have supported implementation of the project. 11. What were the challenges or limitations of the project? Probe for health system (financing, human resources, drugs and other commodities, information systems, governance, and service delivery), partnership, community, sociocultural and gender factors that might have constrained implementation of the project. 12. What is your opinion about the scalability and sustainability of the project? Has the package of interventions been kept as simple as possible without jeopardizing outcomes? Probe for challenges, facilitators and opportunities. 13. What are your recommendations for the future implementation of this strategy? 14. Any other comments? CS-23 Ethiopia, Final Evaluation, Save the Children, December 2012 146 Consent Page INFORMED CONSENT Hello. My name is ______________________________, and I am working with Save the Children. We are conducting a survey and would appreciate your participation. I would like to ask you about your health and the health of your youngest child under the age of two. This information will help Save the Children to plan health services and assess whether it is meeting its goals to improve children’s health. The survey usually takes _______ minutes to complete. Whatever information you provide will be kept strictly confidential and will not be shown to other persons. Participation in this survey is voluntary and you can choose not to answer any individual question or all of the questions. However, we hope that you will participate in this survey since your views are important. Will you participate in this survey? At this time, do you want to ask me anything about the survey? Signature of interviewer: ____________________ Date: ____________________ RESPONDENT AGREES TO BE INTERVIEWED and signs …………1……………………………………………………  RESPONDENT DOES NOT AGREE TO BE INTERVIEWED…………2 END CS-23 Ethiopia, Final Evaluation, Save the Children, December 2012 147 USAID/CSHGP CS-23 Project Cooperative Agreement Number: GHS-A-00-07-00023-00 Ethiopia Child Survival Program: Innovation for Scale: Enhancing Ethiopia’s Health Extension Package in the Southern Nations and Nationalities People’s Region (SNNPR) Final Evaluation: Key Informant/FGD Guide for Health Workers Target respondents: HEWs and vCHWs Introduction: Save the Children has been implementing a Child Survival Project - Innovation for Scale: Enhancing Ethiopia's Health Extension Package in the Southern Nations and Nationalities People's Region (SNNPR). The project aimed to address four main causes of child death: (1) pneumonia, (2) malaria, (3) diarrheal diseases and (4) neonatal infection. The project has been implemented in the two SNNPR woredas (districts) of Shebedino (Sidama Zone) and Lanfero (Silti Zone) since September 2007. Save the Children and partners are now conducting a final evaluation of the project with the aim of documenting accomplishments; and understanding the key factors that contributed to what worked or did not work. These results will be shared with the MOH, and with in-country and external partners so as to inform country and global learning about community-based health programming. You have been identified as one of the key respondents to inform this evaluation. We would like you to take a few minutes of your time so as to answer some questions. Questions: 1. Please describe for me the project as you understand it? 2. What has been your role in the project? 3. How did you join the project and after recruitment into the project what was done to build your capacity? Probe: Training and type of training? Materials given? Community entry? 4. What do you consider to be the main successes of the project in terms of managing children with pneumonia, malaria, and diarrhea; and in terms of managing sick newborn babies? 5. Please comment on what you think have been the key changes in care for children and newborn babies in this area since the inception of this project. Probe: Changes in access to the poor or people in remote areas. 6. In your opinion, what factors facilitated project success. Probe for human resources (e.g. workload), drugs and other commodities availability/stock-outs, information systems (HMIS); linking with the community (including sociocultural and gender factors that might have constrained implementation of the project). 7. What were the challenges or limitations of the project? 8. What did you do to overcome the challenges? 9. How were you motivated in the project? Probe for sources of motivation (district, health workers, community and other). 10. What challenges did you have working with a) Health workers; b) other community health workers; c) Households? How did you try to overcome them? CS-23 Ethiopia, Final Evaluation, Save the Children, December 2012 148 11. In your opinion, was the package of interventions kept as simple as possible for you? Probe for challenges, facilitators and opportunities. 12. As the support from Save the Children and other partners comes to an end, what do you think about the sustainability of the project in this area? How about the scalability to other parts? Will you continue being part of this work? Explain why or why not? What will be the facilitators and barriers to continued participation in this work? 13. What are your recommendations for the future implementation of this strategy? Probe for a) at health facility level; b) at community level? What changes need to be made if any? 14. Any other comments? CS-23 Ethiopia, Final Evaluation, Save the Children, December 2012 149 USAID/CSHGP CS-23 PROJECT Cooperative Agreement Number: GHS-A-00-07-00023-00 Ethiopia Child Survival Program: Innovation for Scale: Enhancing Ethiopia’s Health Extension Package in the Southern Nations and nationalities People’s Region (SNNPR) Final Evaluation Key Informant/FGD Guide from HEWs and vCHWs Introduction: Save the Children has been implementing a Child Survival Project - Innovation for Scale: Enhancing Ethiopia's Health Extension Package in the Southern Nations and Nationalities People's Region (SNNPR). The project aimed to address four main causes of child death: (1) pneumonia, (2) malaria, (3) diarrheal diseases and (4) neonatal infection. The project has been implemented in the two SNNPR woredas (districts) of Shebedino (Sidama Zone) and Lanfero (Silti Zone) since September 2007. Save the Children and partners are now conducting a final evaluation of the project with the aim of documenting accomplishments; and understanding the key factors that contributed to what worked or did not work. These results will be shared with the MOH, and with in-country and external partners so as to inform country and global learning about community-based health programming. You have been identified as one of the key respondents to inform this evaluation. We would like you to take a few minutes of your time so as to answer some questions. Questions: 1. Please describe for me the project as you understand it. 2. What has been your role in the project? 3. After recruitment into the project what was done to build your capacity? Probe: Training and type of training? Materials given? Support for community entry? 4. What do you consider to be the main successes of the project in terms of managing children with pneumonia, malaria, and diarrhea; and in terms of managing well and sick newborn babies? 5. Please comment on what you think have been the key changes in care for children and newborn babies in this area since the inception of this project. Probe: Changes in access to the poor or people in remote areas. 6. The young infant (0-2 months) is many times difficult to care for. Please explain to me how the care was for the a) the well young infant in terms of home care practices; b) the sick young infant in terms of care seeking. What are the challenges in their care? What needs to be done differently to improve the care of the young infant? What strategies are used to visit mothers and babies within 3 days of birth? Do you think these are working? Why or why not? 7. We realized from the data we have collected that when children get cough and fast breathing/difficult breathing or pneumonia they are not taken to health posts but are taken to health centers? In your opinion, is this true? If so please explain why it is the case. Probe: How do communities perceive cough and fast breathing/difficult breathing or pneumonia in terms of severity and where it should be managed? 8. How does referral to HCs from HEWs happen? How do HCs refer to the next level? 9. What strategies are employed for promoting delivery at HCs? Do you think these are sufficient? CS-23 Ethiopia, Final Evaluation, Save the Children, December 2012 150 10. What are the main facilitating factors or obstacles in drug supply (ORS, ZN, AB, RDTs) at the HCs? 11. What is the system for support and supervision to HCs? How does Save support this system? What happens at the supervision visits? What are the strengths and weaknesses of this system? 12. What factors facilitated project success. Probe for human resources (e.g. workload), drugs and other commodities availability/stock-outs, information systems (HMIS); linking with the community (including sociocultural and gender factors that might have constrained implementation of the project). 13. What were the challenges or limitations of the project? Probe for human resources (e.g. workload), drugs and other commodities availability/stock-outs, information systems (HMIS); linking with the community (including sociocultural and gender factors that might have constrained implementation of the project). 14. What did you do to overcome the challenges? 15. We are aware that health posts are not optimally used for care seeking as more people tend to go to health centers. Why is this case? What drives the choice? 16. How were you motivated in the project? Probe for sources of motivation (district, health workers, community and other). 17. What challenges did you have working with a) Health workers; b) other community health workers; c) Households? How did you try to overcome them? 18. In your opinion, was the package of interventions kept as simple as possible for you? Probe for challenges, facilitators and opportunities. 19. As the support from Save the Children and other partners comes to an end, what do you think about the sustainability of the project in this area? Will you continue being part of this work? Explain why or why not? What will be the facilitators and barriers to continue this enhancement model of work? 20. What are your recommendations for the future implementation of this strategy? Probe for a) at health facility level; b) at community level? What changes need to be made if any? 21. Any other comments? CS-23 Ethiopia, Final Evaluation, Save the Children, December 2012 151 Annex 10: List of Persons Interviewed and Contacted During Final Evaluation-USAID/CSHGP - CS-23 Project Enhancing Ethiopia’s Health Extension Program in the Southern Nations and Nationalities People’s Region (SNNPR) Districts of Lanfero (Silte Zone) and Shebedino (Sidama Zone) List of key Informants Involved in the Child Survival Final Evaluation S/N Name Organization Position 1 Awol Badi MOH, Lanfro Woreda Health Office Lanfro Woreda Health Department Head 2 Asres Bedaso MOH, Lanfro Woreda Health Office Lanfro Woreda Maternal, Newborn Child Health Officer 3 Shikur Legaso MOH, Lanfro Woreda, Tora Tora Health Center Head 4 Be-ewketu Lakachew Silte Zone Health Department Health Development, Planning Program Coordinator 5 Abuselam Hasen Mito Health Center Mito Health Center Head 6 Yesuf Nesri Archuma Wonte Health Center Archuma Wonte Health Center Head 7 Elfite Alito Tora Kikora Health Post Tora Kikora Health Post HEW 8 Etenu Teketel Archuma Gola Health Post Archuma Golla Health Post HEW 9 Abaynesh Minda Shefode Debar Health Post Shefode Debar Health Post HEW 10 Abdulmuhin Nuri Save The Children Lanfro District, Community Mobilization Officer 11 Chiksa Sultan Save The Children Lanfro District, MNCH Officer 12 Aschenaki Zeryehun Sidama Zone Health Department Sidama Zone Health Department Head 13 Zergu Tafese IFHP Regional IFHP Manager 14 Elias Kayessa SCUS Former CS-23 Program Coordinator 15 Gizachew Kebede SNNPR Health Bureau Regional Health Bureau Deputy Head and HPDP Process Owner 16 Bedelu Badego MOH, Shebedino District Health Department Shebedino Woreda Health Department Head 17 Bekele Kamara MOH, Shebedino District Health Department Shebedino Woreda, MNCH Expert 18 Kedija Yimer MOH, Leku HC Leku HC OPD Team Leader 19 Yenenesh Teshome MOH, Holiso HP Holiso HP, HEW 20 Adane Utala MOH, Abela HC Abela HC Head 21 Yiftusera Senbeto MOH, Galuko Hireya HP Galuko Hireya HP, HEW 22 Mulunesh Kawiso MOH, Galuko Hireya Kebele Galuko Hireya Kebele Head 23 Zenash Ganebo MOH, Dilla Aferara HP Dilla Aferara HP, HEW 24 Yenenesh Fikadu MOH, Dobe Toga HC Dobe Toga HC, HEWs Supervisor 25 Mahatebe Fanta MOH, Gonowa Gabalo HP Gonowa Gabalo HP, HEW 26 Amsale Tadele MOH, Gonowa Gabalo HP Gonowa Gabalo HP, HEW 27 Fetelework Gizachew MOH, Ramada HP Ramada HP, HEW CS-23 Ethiopia, Final Evaluation, Save the Children, December 2012 152 List of Persons Interviewed and Contacted During the Child Survival Final Evaluation (TORA KIKORA Health Post), Lanfro District S/N Name Region District Role 1 Yasin Adem SNNPR Lanfro vCHW 2 Said Mosa SNNPR Lanfro vCHW 3 Hulumga Abdo SNNPR Lanfro vCHW 4 Jemal Ahmed SNNPR Lanfro vCHW 5 Zulfa Ajumamo SNNPR Lanfro vCHW 6 Memuna Ahmed SNNPR Lanfro vCHW 7 Bilcho Jemal SNNPR Lanfro vCHW 8 Kuraz Tirago SNNPR Lanfro vCHW List of Persons Interviewed and Contacted During the Child Survival Final Evaluation (MEDEB GAGABO Health Post), Lanfro District S/N Name Region District Role 1 Medina Shemolo SNNPR Lanfro Mother Group Member 2 Fetiya Mohamed SNNPR Lanfro Mother Group Member 3 Munaju Mude SNNPR Lanfro Mother Group Member 4 Fichase Yasin SNNPR Lanfro Mother Group Member 5 Rebiya Mohamed SNNPR Lanfro Mother Group Member 6 Sinchewa Jemal SNNPR Lanfro Mother Group Member 7 Ashereka Baleker SNNPR Lanfro Mother Group Member 8 Amina Nasir SNNPR Lanfro Mother Group Member 9 Fedila Abrar SNNPR Lanfro Mother Group Member 10 Nejaba Kemal SNNPR Lanfro Mother Group Member List of Persons Interviewed and Contacted During the Child Survival Final Evaluation (AMICH Health Post), Lanfero District S/N Name Region District Role 1 Roba Kinta SNNPR Lanfro vCHW 2 Awol Hutimo SNNPR Lanfro vCHW 3 Roba Muktar SNNPR Lanfro vCHW 4 Shemsu Hamza SNNPR Lanfro vCHW 5 Shirage Redi SNNPR Lanfro vCHW 6 Muntaha Genancho SNNPR Lanfro vCHW 7 Meboze Sirula SNNPR Lanfro vCHW 8 Dinkure Husen SNNPR Lanfro vCHW CS-23 Ethiopia, Final Evaluation, Save the Children, December 2012 153 List of Persons Interviewed and Contacted During the Child Survival Final Evaluation (AMICH Health Post), Lanfro District S/N Name Region District Role 1 Roba Kinta SNNPR Lanfro vCHW 2 Awol Hutimo SNNPR Lanfro vCHW 3 Roba Muktar SNNPR Lanfro vCHW 4 Shemsu Hamza SNNPR Lanfro vCHW 5 Shirage Redi SNNPR Lanfro vCHW 6 Muntaha Genancho SNNPR Lanfro vCHW 7 Meboze Sirula SNNPR Lanfro vCHW 8 Dinkure Husen SNNPR Lanfro vCHW List of Persons Interviewed and Contacted During the Child Survival Final Evaluation (SHEFODE DEBAR Health Post), Lanfro District S/N Name Region District Role 1 Alayka Bamude SNNPR Lanfro Mother Group Member 2 Layla Sakin SNNPR Lanfro Mother Group Member 3 Talima Gobena SNNPR Lanfro Mother Group Member 4 Sunemi Jemal SNNPR Lanfro Mother Group Member 5 Bamirka Hadera SNNPR Lanfro Mother Group Member 6 Shukurit Ayano SNNPR Lanfro Mother Group Member 7 Haisha Abdo SNNPR Lanfro Mother Group Member 8 Bedria Adem SNNPR Lanfro Mother Group Member 9 Rawda Temam SNNPR Lanfro Mother Group Member 10 Burtuge Yasin SNNPR Lanfro Mother Group Member 11 Shitu Misoro SNNPR Lanfro Mother Group Member 12 Nuria Osman SNNPR Lanfro Mother Group Member List of Persons Interviewed and Contacted During the Child Survival Final Evaluation (ARCHUMA GOLA Health Post), Lanfero District S/N Name Region District Role 1 Kemal Demolo SNNPR Lanfro vCHW 2 Gutago Doyoro SNNPR Lanfro vCHW 3 Radi Yunus SNNPR Lanfro vCHW 4 Safia Bumude SNNPR Lanfro vCHW 5 Etalu Hamid SNNPR Lanfro vCHW 6 Fedila Nursebo SNNPR Lanfro vCHW 7 Ilfe Saefa SNNPR Lanfro vCHW 8 Etala Dilsebo SNNPR Lanfro vCHW 9 Dema Dilsebo SNNPR Lanfro vCHW 10 Kalil Aliye SNNPR Lanfro vCHW CS-23 Ethiopia, Final Evaluation Report, Save the Children, December 2012 154 Annex 11: Final Operations Research Report Not Applicable. CS-23 Ethiopia, Final Evaluation Report, Save the Children, December 2012 155 Annex 12: Special Publications/Reports Annex 12A: Rapid Health Facility Assessment (R-HFA) Endline Report Child Survival Project Innovation for Scale: Enhancing Ethiopia’s Health Service Extension Program in the Southern Nations and Nationalities People’s Region (SNNPR) Implemented by: Save the Children/USA Lanfero and Shebedino Districts, SNNPR, Ethiopia August 2012 This report is made possible by the generous support of the American people through the United States Agency for International Development USAID). The contents are the responsibility of Save the Children and do not necessarily reflect the views of USAID or the United States Government. CS-23 Ethiopia, Final Evaluation Report, Save the Children, December 2012 156 Table of Contents Page I. INTRODUCTION ................................................................................................................. 158 II. METHODS ........................................................................................................................... 159 2.1 Core Indicators .................................................................................................................. 160 2.2 R-HFA Data Collection Tools .......................................................................................... 160 2.3 Data collectors training in R-HFA .................................................................................... 161 2.4 R-HFA Sampling Methodology ........................................................................................ 161 2.5 R-HFA Data Collection .................................................................................................... 162 2.6 Key Information collected in the R-HFA ......................................................................... 163 2.7 Data Entry and Analysis ................................................................................................... 163 III. RESULTS ............................................................................................................................ 163 3.1 Health Center .................................................................................................................... 163 3.2 Health Post ........................................................................................................................ 170 IV. DISCUSSION ...................................................................................................................... 173 ANNEXES ................................................................................................................................. 175 Annex 1: Data collectors Training Schedule .......................................................................... 175 Annex 2: Data Collection Schedule, Lanfero District, June 19-22, 2012 ............................... 176 Annex 3: Data Collection Schedule, Shebedino District, June 11-15, 2012 .......................... 177 Annex 4: Core Indicator Definitions ....................................................................................... 179 Annex 5: Optional Indicator Definitions ................................................................................ 183 Annex 6: Tabulation Plan (Core Indicators) ........................................................................... 184 Annex 7: Tabulation Plan (Optional Indicators) ..................................................................... 190 CS-23 Ethiopia, Final Evaluation Report, Save the Children, December 2012 157 Acronyms ANC Antinatal Care ARI Acute Respiratory Tract Infection CDD Control of Diarrheal Disease CHWs/TTBAs Community Health Workers/Trained Traditional Birth Attendants CCM Community Case Management CCM/P Community Case Management/Pneumonia CCM/NS Community Case Management/Neonatal Sepsis CSHGP Child Survival Health Grants Program CSTS Child Survival Technical Support D Denominator DPT Diphtheria, Pertussis, Tetanus EPI Expanded Program on Immunization GMP Growth Monitoring and Promotion HC Health Center HEP Health Extension Program HEWs Health Extension Workers HFs Health Facilities HMIS Health Management Information System HPs Health Posts HWs Health Workers ISS Integrated Supportive Supervision IMCI Integrated Management of Childhood Illness IMNCI Integrated Management of Newborn and Childhood Illness ITN/LLIN Insecticide Treated Net/Long Lasting Impregnated Net M&E Monitoring & Evaluation MNCH Maternal, Newborn and Child Health MOH Ministry of Health N Numerator NGOs Non-Government Organizations NS Neonatal Sepsis OPV Oral Polio Vaccine ORT Oral Rehydration Therapy P Present PCM Pneumonia Case Management R-HFA Rapid Health Facility Assessment RR Response Rate S Sanctioned SC Save the Children TT Tetanus Toxoid U5 Under 5 WHO Woreda Health Office CS-23 Ethiopia, Final Evaluation Report, Save the Children, December 2012 158 I. INTRODUCTION 1.1 Purpose of Rapid Health Facility Assessment The Rapid Health Facility Assessment (R-HFA) is one of several baseline and endline data studies to provide a quantitative assessment of child health services as well as identify opportunities and constraints to the program that the district health systems have. R-HFA results will be used to improve Maternal, Neonatal and Child Health (MNCH) services as well as the overall level of quality and access to primary health services. R-HFA evaluates first-level health facilities (HFs) for MNCH services. The objectives of the endline R-HFA were: 1. To assess endline levels of quality of and access to MNCH services at HFs and by Health Extension Workers (HEWs). 2. To assess key bottlenecks in MNCH service delivery at HFs and by HEWs and hence facilitate data use for action for decision makers, so that key actions can be implemented for service improvement. 3. To use the R-HFA results for decision making and to design future projects. 4. To compare baseline (BL) to endline (EL) values and to compare EL values to project targets. 1.2 Description of the Child Survival Project Save the Children (SC/USA) was awarded a five-year, Standard Child Survival Project, Innovation for Scale: Enhancing Ethiopia's Health Service Extension Program in the Southern Nations and Nationalities People's Region (SNNPR) in Lanfero and Shebedino Districts, to address four main causes of child death: pneumonia, diarrheal diseases, and malaria (that together account for 68% of under-five mortality); and neonatal infection, responsible for half of all neonatal mortality. The selected Child Survival Program interventions are: Pneumonia Case Management (PCM) 35%; Control of Diarrheal Diseases (CDD) 30%; Prevention and Treatment of Malaria 20%; Newborn Care 20%: and Immunization 5%. The project’s key implementation strategy is Community Case Management (CCM). Under this project, Save the Children has been demonstrating the feasibility of enhancing the current HEP for expanded impact by adding: (1) CCM of pneumonia (CCM/P) with antibiotics; (2) zinc treatment for diarrhea; (3) use of new formula ORS for dehydrating diarrhea; and (4) improved assessment/referral of neonatal sepsis (NS) with the possibility of CCM/NS to be added. 1.3 Health Service System The Federal Ministry of Health (FMOH) strives to bring evidence-based, life-saving interventions closer to the household level through the Health Extension Package (HEP). Delivered by trained community-based government-salaried HEWs, the HEP already incorporates CCM of diarrhea (CCM/D) with oral re-hydration solution (ORS) and CCM of malaria (CCM/M) with Artemisinin Combination Therapy (ACT). During project implementation, the FMOH changed the policy on pneumonia case management and the HEWs were allowed to treat pneumonia at the Health Post (HP) level. The current health care system provides services at the following levels: HP; health center (HC); district hospital; regional hospital and referral/specialized hospital. All health facilities (public and private) are required to CS-23 Ethiopia, Final Evaluation Report, Save the Children, December 2012 159 provide a minimum package of activities to cover basic health problems in an equitable, effective and efficient manner. Hospitals are required to provide a complementary package of curative care in an equitable, effective and efficient way using techniques unavailable at the primary level. In total the two project districts (Lanfero and Shebedino) have 11 standard HCs and 56 HPs. The HCs have standard staffing structure including Health Officer, Senior Nurse, Senior Midwife, Pharmacy Technician, Sanitarian, Laboratory Technician and other supportive administration staffs. The HPs are fully run by HEWs currently, with a minimum of two female HEWs in each HP. Neither district has a district hospital (one hospital is under construction in Shebedino District) designed to care for referrals from the HCs while HPs are referring to the HCs located within the service area (each HC has its own catchment area). The HPs are supervised by, and report to, the HCs and are overseen by the District Health Offices (DHOs). In Lanfero District there is an average of seven communities/kebeles under each HC while five communities/kebeles are under each HC in Shebedino District. Table 1: No. of Health Facilities in the project sites District Estimated pop. (2011) Health Center Health Post No. of Communities/kebeles Lanfero 134,243 4 25 27 Shebedino 261,128 7 31 35 395,371 11 56 62 In both districts, in each rural community/kebele there is one HP with two/three female HEWs. All supervision and policy oversight is the authority of FMOH who manage clinical and community services through a District Health Management System (DHMS). At the community level, the HEWs play an important role in the national strategy for primary health care promotion supported by trained, non-paid community volunteers (Health Development Army) in strengthening behavioral change communication among the community. The HEWs are trained by the government to treat basic illnesses like malaria, pneumonia and diarrhea and to provide health education and other preventive activities in their communities (“16 Health Extension Packages”). The HEWs are government employees who are paid monthly. Supervision of HEWs is generally the responsibility of the HCs and the overall management and monitoring the progress of HEP program is the responsibility of the DHOs. The HEWs report to and receive their drug/supply from HCs depending on the supply schedule of the DHO. Non-Government Organizations (NGOs) working in the project areas, work alongside this government health system. II. METHODS Endline R-HFA was conducted from June 11-15, 2012 in Shebedino and from June 19-22, 2012 in Lanfero, to measure access and quality of child health services in 11 Health Centers (Lanfaro 4; Shebedino 7) and 30 Health Posts (15 from each). It was conducted by Save the Children’s child survival project team in Hawassa and four health professionals from the DHOs (Shebedino 2; Lanfero 2). All 11 HCs in both districts were included in the R-HFA, since the HC number is low in both districts. According to R-HFA sampling procedures and guidelines, the HPs were CS-23 Ethiopia, Final Evaluation Report, Save the Children, December 2012 160 selected using simple random sampling (Random Number Table) from the list of all HPs (sampling frame/unit) obtained from both DHOs. The Regional Health Bureau (RHB) and the Zonal Health Department (ZHD) were briefed and consulted regarding the R-HFA. All data collectors were a team of Health Workers (HWs) trained in Integrated Management of Newborn and Childhood Illnesses (IMNCI) with ample experience in health service and survey techniques. Data collectors were trained on the assessment tool and data collection techniques for two days from June 9-10, 2012. The assessment tools were the same tools used for the baseline R-HFA. Considering the rural setting of HFs, the R-HFA was scheduled at three HFs per day in both districts. 2.1 Core Indicators Twelve core and seven optional indicators were included in the assessment. These 19 indicators were intended to rapidly and feasibly give a “balanced score card” for preparedness of a Health Center (HC) to deliver the three essential child health services: Growth Monitoring and Promotion (GMP); Expanded Program on Immunization (EPI); and Sick Child Care. This is a balanced score card in the sense that it examines indicators across a variety of domains, which all necessary for basic Health Facility (HF) are functioning: access; inputs; processes; and performance:  Access: Geographic access, service availability;  Inputs: Staffing, infrastructure, supplies, drugs, availability of immunizations, availability of guidelines, and infection control;  Processes: Information system, training, community coordination, community referral, laboratory, and supervision; and  Outputs: Correct assessment, correct treatment, and counseling. This same reasoning was used to construct a more simple, balanced score card for HPs/HEWs. Seven of the core and one of the optional indicators used for HCs were selected and adapted to measure the preparedness to deliver quality care among HPs/HEWs:  Inputs: Supplies, drugs;  Processes: Information system, training, and supervision; and  Outputs: Utilization and correct treatment (from register review). 2.2 R-HFA Data collection tools The survey instrument was based on the R-HFA tool that Child Survival Technical Support (CSTS+) has developed for use in CSHGP projects. Four main modules with a fifth module for HEWs were implemented in both project site districts. The R-HFA survey formats were adapted according to the project indicators and also some basic questions were added, i.e., category of health professionals, basic medical equipment and questions concerning the service of HEWs. Similarly, some questions that are not related to the project indicators were omitted from the survey format, i.e., Antenatal Care (ANC) service availability, ANC supplies, ANC drug, ANC information system, ANC services utilization, malaria drug (ACT), ITN/LLIN, and utilization of immunization services that was covered during Child Survival (CS) project endline KPC survey. The assessment formats were the following: CS-23 Ethiopia, Final Evaluation Report, Save the Children, December 2012 161  Observation Checklist for sick child care (Module 1): observation on the assessment, classification and treatment of six consecutive children with fever, diarrhea, or cough/difficult breathing.  Client Exit Interview (Module 2 ): To assess caretakers’ correct knowledge on how to administer drugs given for diarrhea, malaria, and/or pneumonia (used as a proxy for adequate counseling).  Health Facility Checklist (Module 3): To assess the presence of a minimal level of infrastructure, supplies, and medications.  HW Survey (Module 4): To assess the staffing, MNCH services offered, as well as frequency of training, supervision, and other key processes.  HEW Survey and Che cklist (Module 5): Collects data for HEW on inputs, processes, and service delivery (through register review). 2.3 Data collectors training in R-HFA Data collectors were trained for two days from June 9-10, 2012. As per the R-HFA guidelines, the length of the data collectors’ training would have been four days; however, considering the experience of the selected data collectors (most of them have been trained in IMNCI and have accumulated experience on the program), two days was sufficient. Right after the field practice, discussions were conducted among the teams about the lessons learned and the problems encountered during data collection. This helped data collectors to share experiences among themselves in order to better implement the actual assessment. During the training, the following points were emphasized: how and when to start at a HF, preparation for clinical session, selection of eligible children, completing the assessment questionnaire and giving immediate feedback to the HWs prior to leaving the HF on issues observed in order to help to improve the quality of clinical and management practices, and also encouraging the HFs to sustain good performance. 2.4 R-HFA Sampling Methodology The assessment involved sampling of HPs, and included all HCs available in both districts (4 HCs in Lanfaro; 7 HCs in Shebedino). Table2: Summary of Sample HFs assessed in both districts Unit Universe Sample Successful Assessments (Response Rate) HC 11 11 11 (100% RR) HP 56 30 30 (100% RR) Sampling of HPs and HCs: There are 56 HPs (25 in Lanfaro; 31 in Shebedino) in both districts. From the total 56 HPs, 30 were randomly selected from both districts; 15 HPs from each district and all HCs were included in the survey. To complete the questionnaire, surveyors interviewed the most experienced and senior HW in all HCs who were responsible for managing the treatment of children under five. Similarly, senior HEWs were interviewed in HPs (all HPs are CS-23 Ethiopia, Final Evaluation Report, Save the Children, December 2012 162 run by HEWs) where there are two or three HEWs assigned per HP. The detailed sampling procedures were as follows:  All HCs in both districts were automatically included in the R-HFA since only 11 HCs are available in both districts (4 in Lanfaro; 7 in Shebedino).  According to the R-HFA sampling procedures and guidelines, the HPs were selected using simple random sampling (using Random Number Table) from list of all HFs (sampling frame/unit) obtained from both DHOs. In Lanfaro District, 15 HPs were selected from the total 25 HPs. Similarly, 15 HPs were selected from 31 HPs in Shebedino District.  To assess the skills of HWs, sick child cases (fever/malaria, cough/difficult breathing or diarrhea) and linked caretaker exit interviews were included in the assessment (six consecutive cases).  The randomly selected HFs in both districts were given a consecutive unique code (Shebedino: 001 through 22; Lanfero: 23 through 41) since the data must be able to be associated with a specific HF. 2.5 R-HFA Data Collection The data collection took four days in each district except more days were needed for three HCs in Shebedino and one HC in Lanfero, since enough cases were not found on the scheduled survey date. To avoid bias due to inconsistent preparations and practices by the HFs, all HFs were not informed prior to the survey. Only the DHOs were informed in order to enable them to prepare themselves to support the implementation of the survey. Data were collected by R-HFA teams of two people each. Each team collected data from three facilities per day. The teams generally arrived at 8:30 a.m., when facilities opened. They completed the observations and exit interviews first, and then filled out all remaining forms in the HCs. Similarly, HEWs were interviewed at their respective HPs. In all HFs data collection was generally finished after mid￾day. Cases Observed and Caretakers Interviewed: In each HC visited, surveyors observed six consecutive eligible cases for care of sick children and interviewed their caretakers. Criteria for eligible cases included age (child 1-59 months), illness (malaria/fever, difficult breathing or diarrhea) and caretaker informed consent. Since 11 HCs were assessed, the total number of cases were 66 (11 x 6) observation and exit interviews. In most HCs, sufficient numbers of sick children were available on the survey day to fulfill the sample size expectation. However, there were three HCs in Shebedino that needed an additional three days, and one HC in Lanfero Districts that needed an additional two days to complete the survey due to a shortage of cases. HEWs Assessed: One HEW was included in the interview in each HP; however, where there were two assigned HEWs in a HP, the senior one was interviewed. A total of 30 HEWs were interviewed in both districts (15 in each district) including reviewing their records and treatment registration books according to the structured survey questionnaire. CS-23 Ethiopia, Final Evaluation Report, Save the Children, December 2012 163 2.6 Key information collected in the R-HFA Five components were covered: 1. MNCH services offered, staffing, staff qualification, frequency of training, supervision, and other key processes; 2. Availability of a minimal level of infrastructure, supplies, and medications; 3. Presence of quality processes – training, supervision, and infection control; 4. HW performance – adherence to standard protocols (based on IMNCI) for assessment, classification, management, and counseling for children under five with fever/malaria, ARI, and/or diarrhea; and 5. HEW performance assessed through coverage of services and maintenance of up-to-date registers; availability of basic resources (equipment, supplies); and key processes (supervision, training) that prepare them for community service provision. 2.7 Data entry and analysis The data were entered and analyzed using the pre-defined template (Excel data entry program) with automatic construction of disaggregated frequency tables, aggregate indicator score tables and bar charts for attainment of a set of 12 key indicators. The survey team checked each questionnaire daily for error and correction before leaving the respective HF. In addition, the Child Survival M&E Coordinator double-checked data at the field level and while entering the data. Moreover, for experience sharing among the survey teams and to keep maximal data reliability, there were daily evening sessions for early detection of problems and immediate action. Concerning the data analysis plan, the pre-defined template has automatically calculated tables and core indicators according to the R-HFA tabulation plan. III. RESULTS 3.1 Health Center Access to Health Services Population with year round geographic access (within 5km or ≤1 hour walk) to an authorized provider of curative child health services was 100% vs. baseline 57%. (Table 3). The government has constructed HPs in each kebele to enhance community health services and some basic clinical services. However, in each kebele the distance to access health services is not equal for all families. Each HC has a catchment area that includes an average of nine kebeles in Lanfero while an average of seven kebeles in Shebedino; however, the range is from seven to 11 kebeles in Lanfero versus two to 12 in Shebedino. Availability of child health services, i.e. growth monitoring through either HF or outreach, immunization through either HF or outreach, and sick child care through either HF or outreach was improved from the baseline result (baseline: 38% vs. 91% overall). The availability of child health services improved markedly in both districts over baseline: Lanfero (100% vs. baseline 33%) and Shebedino (86% vs. baseline 40%). HFs that offer delivery services on all days through either HF or outreach during the baseline were 72% overall, but the endline assessment result was 100% overall (Table 3). During CS-23 Ethiopia, Final Evaluation Report, Save the Children, December 2012 164 baseline, availability of delivery service on all days was (60%) in Shebedino vs. Lanfero (100%). All HCs have trained midwives and all HPs have HEWs trained on safe and clean delivery services, who provide delivery services in their respective HFs. The routine delivery services report from HCs and HPs indicate that the number of mothers using HFs for delivery services is increasing. As a result, the number of registered deliveries in HFs has improved in both districts (register review). Table 3: Geographic Access and Availability of Child Health and Delivery Services by District Indicator Shebedino Lanfero Project (W%) BL* EL** BL EL BL EL % of population with year round geographic access 62 100 45 100 57 100 Indicator #1: Availability of child health services 40 86 33 100 38 91 Indicator #1: Availability of delivery services 60 100 100 100 72 100 Figure 1: Geographic Access and Service Availability by Project Staffing, Infrastructure, Supplies and Drugs The number of HCs with the following in place on the day of survey, has improved from the baseline HFA (Table 4): 1)all clinical staff were present;2) all essential infrastructure was present and functioning; 3) all essential child supplies were present; 4)all basic neonatal and delivery services existed; and 5) all essential child, delivery, and neonatal drugs were in stock. HCs in which all clinical staff were present on the day of the survey (baseline: 69% vs. 73%); infrastructure (latrine, water, privacy) present and functioning (baseline: 38% vs. 89%); essential child health supplies (infant scale, timer/watch, spoon/cup/jug to administer ORS) (baseline: 42% vs. 100%); basic neonatal and delivery supplies (partographs, vacuum extractor, resuscitation equipment, infant scale and wraps) (baseline: 0% vs. 37%); availability of essential child drugs (baseline: 14% vs. 82%); availability of essential delivery and neonatal drugs (baseline: 10% vs. 73%). In both districts, inputs have improved from the baseline, but availability of maternal and newborn care supplies are still low. CS-23 Ethiopia, Final Evaluation Report, Save the Children, December 2012 165 Table 4: Staffing, Infrastructure, Supplies and Drugs by District Indicator Shebedino Lanfero Project (W%) BL EL BL EL BL EL Indicator #2: Staffing - % HC in which all clinical staff were present on the day of survey 71 76 67 69 69 73 Indicator #3: % HC with all essential infrastructure is present and functioning on the day of the survey 40 87 33 91 38 89 Indicator #4: Child - % HC with all essential supplies 60 100 0 100 42 100 Indicator #4: MNC - % HC with all basic neonatal and delivery supplies 0 43 0 25 0 37 Indicator #5: Child Drugs - % HC with all essential child drugs 20 71 0 100 14 82 Indicator #5: MNC Drugs - % HC with all essential delivery and neonatal drugs present on day of survey 0 86 33 50 10 73 Figure 2: Staff and Infrastructure by Project Figure 2: Staffing and Infrastructure by Project Figure 3: Availability of Supplies and Drugs by Project Figure 3: Availability of Supplies and Drugs by Project CS-23 Ethiopia, Final Evaluation Report, Save the Children, December 2012 166 Availability of Vaccines, Guidelines and Infection Control Availability of all vaccines (BCG, Penta, OPV, TT and Measles), all guidelines (sick child, immunization, delivery, maternal postpartum care and neonatal care) and all infection control (disinfectant, gloves, sharp container, soap, sharp disposal area, infectious waste disposal area, sharp disposal practices and infectious waste disposal practices) at HCs have improved from the baseline result (Table 5) – availability of vaccine (baseline: 38% vs. 100%); guidelines (baseline: 28% vs. 100%); infection control (baseline: 38% vs. 55%). All the three indicators have improved in both districts from the baseline results. In Shebedino and Lanfero the availability of vaccines increase as follows baseline: 40% vs. 100%, and 38% vs. 100%, respectively; guidelines from a baseline: 40% vs. 100%; 28% vs. 100%, respectively,; and for infection control from a baseline: 40% vs. 57%; 38% vs. 55%, respectively. Vaccine stock-out was reported as non-existent in the six months prior to the survey in both districts. Table 5: Availability of Vaccines, Guidelines and Infection Control Indicator Shebedino Lanfero Project (W%) BL EL BL EL BL EL Optional indicator #1: Vaccine Availability - % HC with all vaccines available in stock on day of survey 40 100 33 100 38 100 Optional indicator #2: Availability of Guidelines - % HC with all guidelines available and accessible on day of survey 40 100 0 100 28 100 Optional indicator #3: Infection Control - % HC with all infection control supplies and equipment on day of survey 40 57 33 50 38 55 Figure 4: Availability of Vaccines, Guidelines and Infection Control by Project Availability of Information System, Training, Supervision, Community Coordination and Laboratory Services The availability of information systems in place at HFs is reported in Table 6 below. This includes: all elements of the information system in place are; HFs/HWs receiving child health training in the 12 months prior to the survey; receiving supervision in the previous three months; the existence of at least one method for community coordination (participation in management committee meetings, participation through engagement of vCHWs CS-23 Ethiopia, Final Evaluation Report, Save the Children, December 2012 167 and discussion); and basic laboratory services increased from the baseline result. HCs that maintained up to-date records of sick U5 children (age, diagnosis and treatment), reported data which shows a dramatic increase of use from 0%at baseline to 100% overall in the three months prior to this survey. HWs reported receiving in-service or pre-service high levels of training in child health care in last 12 months (baseline: 90% vs. 100% overall) and reported receiving dramatically increased in-service or pre-service training in maternal neonatal care in last 12 months (baseline: 14% vs. 100% overall). HCs receiving external supervision at least once in the last three months (checked records, observed reports, provided feedback, praised good work, provided updates and discussed encountered problems) increased from 62% at baseline to 100% overall. The availability of mechanisms used to elicit community participation (participation on community management committee meetings and participation through engagement of CHWs) also increased (baseline: 62% vs. 100% overall). The availability of all basic essential maternal-child laboratory services (able to do or send out hemoglobin, smear or RDT, do or send out for glucose, HIV testing, do or send out sputum, do or send out syphilis testing) dramatically increased from 0% at baseline to 100% overall. In general, both districts achieved all indicators (100%) versus baseline results. Table 6: Availability of Information System, Training, Supervision, Community Coordination and Laboratory Services By District Indicator Shebedino Lanfero Project (W%) BL EL BL EL BL EL Indicator #6: Information System - % HC that maintain up-to-date records of sick U5 children 0 100 0 100 0 100 Indicator #7: Child Training - % HC in which interviewed HW reported receiving in-service or pre￾service training in child health in last 12 months 100 100 67 100 90 100 Indicator #7: MNC Training - % HF in which interviewed HW reported receiving in-service or pre￾service training in maternal neonatal care in last 12 months 20 100 0 100 14 100 Indicator #8: Supervision - % HC that received external supervision at least once in the last 3 months 60 100 67 100 62 100 Optional Indicator #4: Community Coordination - % HC with routine community participation in management meetings 67 100 60 100 62 100 Optional indicator #8: - Laboratory - % HC with adequate basic laboratory services in place or ability to send out 0 100 0 100 0 100 CS-23 Ethiopia, Final Evaluation Report, Save the Children, December 2012 168 Figure 5: Availability of Information System, Child & MNC Training by Project Figure 6: Availability of Supervision, Community Coordination and Laboratory Services by Project Health Workers’ Performance Though still low, the HWs’ performance in assessment, treatment and counseling has doubled over the baseline. Key assessment tasks, treatment appropriate to diagnosis/classification and counseling mothers of sick children improved from the baseline results (as per Table 7). Key assessment tasks routinely performed (checking general danger signs, assessing feeding practice, assessing nutritional status and vaccination status) increased modestly (baseline: 10% vs. 25.5% overall). Treatment appropriate to diagnosis/classification (fever, breathing problem or diarrhea) was nearly perfect (baseline: 76% vs. 95.3% overall). Caretakers of children prescribed an antibiotic, antimalarial, or ORS who correctly described how to administer all prescribed drugs increased modestly (baseline: 12% vs. 25.6% overall). Checking general danger sign, feeding practices, nutritional status and vaccination status are basic tasks to be performed for all sick children, but most HWs did not perform these during observation. Similarly, most mothers were not counseled by the HW on how to administer the prescribed drugs (exit interview). Rather, mothers were counseled by the Pharmacist (pharmacy section) when they collected the prescribed drugs. In principle, the HW would have counseled the caretakers before they leave the treatment room, but the system in Ethiopia is mainly designed to provide counseling on medication at the pharmacy. CS-23 Ethiopia, Final Evaluation Report, Save the Children, December 2012 169 Table 7: HWs Performance (assessment, treatment and counseling) by District Indicators Shebedino Lanfero Project (W%) BL EL BL EL BL EL Indicator #10: HW Performance (Assessment) - % HC where key assessment tasks are routinely performed 0 29 33 19 10 25.5 Indicator #11: HW Performance (Treatment)- % HC where treatment is routinely appropriate to diagnosis 80 100 67 87 76 95.3 Indicator #12: HW Performance (Counseling)- % HF where caretakers whose child was prescribed an antibiotic, antimalarial, or ORS, correctly describe how to administer all prescribed drugs 18 14 0 46 12 25.6 Figure 7: Health Workers Performance by District 0 20 40 60 80 100 Key assessment tasks routinely performed Treatment routinely appropriate to diagnosis Caretakers correctly described how to administer drugs % CS23 BL CS23 EL Utilization of Sick Child Services Annual numbers of clinical encounters for sick children and immunization visits improved from baseline, while Growth Monitoring (GM) showed no change (Table 8). This includes: use of curative services for children under five years old (baseline: 0.2 vs. 0.3 visits/year overall); use of immunization services for children under one year old (baseline: 0.1 vs. 0.2 visits/year overall); and use of GM services for children under three years old (baseline: 0.1 vs. 0.1 GM/year overall). Use of sick child services was better in Lanfero than Shebedino though there was improvement in both districts, but use of GM services showed no change from the baseline result in both districts. CS-23 Ethiopia, Final Evaluation Report, Save the Children, December 2012 170 Table 8: Utilization of Sick Child Services Performance/ output Indicator Shebedino Lanfero Project (W%) BL EL BL EL BL EL Utilization of curative services (U5 children) Indicator # 9 - Annualized number of clinical encounters for sick children per U5 population in catchment area 0.03 visits/ year 0.1 visits/ year 0.5 visits/ year 0.6 visits/ year 0.2 visits/ year 0.3 visits/ year Utilization of Immunization Services (U1 children) Optional indicator # 9a - Annualized number of immunization visits per U5 population in catchment area 0.1 child immuni￾zation/ year 0.13 immuni￾zation/ year 0.03 child immuni￾zation/ year 0.4 immuni￾zation/ year 0.1 child immuni zation/ year 0.2 immuni￾zation/ year Utilization of Growth Monitoring Services (U3 children) Optional indicator # 9b - Utilization of Growth Monitoring Services per U5 population in catchment area 0.1 GM/ year 0.1 GM/ year 0.33 GM/ year 0.1 GM /year 0.1 GM/ year 0.1 GM/ year 3.2 Health Post Availability of essential supplies and basic drugs HPs/HEWs with all essential supplies to support child health (infant scale, timer, spoon/cup/jug to administer ORS, and reference manuals) and with all first-line medications for child health on the day of the survey (ORS, antibiotic for pneumonia, first-line antimalarial and vitamin A) dramatically improved from baseline (Table 9). All HPs had all essential supplies and all first-line medications on the day of the survey (0% vs. 100%; 0% vs. 100% [overall] respectively). Both districts have equally improved from baseline (baseline: 0% vs. 100% overall). Table 9: Availability of Supplies and Drugs Indicator Shebedino Lanfero Project (W%) BL EL BL EL BL EL Indicator #4: Supplies - % HEWs with all essential supplies 0 100 0 100 0 100 CS-23 Ethiopia, Final Evaluation Report, Save the Children, December 2012 171 Indicator #5: Drugs - % HEWs with all first-line medications 0 100 0 100 0 100 Figure 8: Availability of Supplies and Drugs 0 20 40 60 80 100 All essential child supplies All essential first line medications % CS23 BL CS23 EL Availability of Information Sy stem, Training and Supervision Information system, training in Maternal and Newborn Care (MNC), supervision and treatment given appropriate to diagnosis/assessment have improved from the baseline results while training in child health showed no change from baseline result (Table 10). Though training in child health was similar to baseline, the content of training changed. At baseline HEWs were not trained to assess, classify and treat common childhood illnesses like pneumonia, malaria, diarrhea and acute malnutrition in children 2 to 59 months of age; and they were not trained to assess classify and refer sick young infants under 2 months of age (IMNCI/iCCM). Currently in both districts, all HEWs are trained in IMNCI/iCCM including pneumonia treatment. HPs/HEWs now maintain up-to-date records of sick children under five (age, diagnosis and treatment) (baseline: 0% vs. 100% overall); HEWs continue to receive in￾service or pre-service training in child health (immunization, ICCM, nutrition, breastfeeding, etc.) in last 12 months (baseline: 98% vs. 98% overall). HEWs received dramatically more in-service or pre￾service training in MNC in last 12 months (baseline: 17% vs. 98% overall). HEWs continued to receive adequate external supervision (check reports, observation, feedback, provide technical or managerial updates and discussions on problems encountered) at least once in the last three months (baseline: 98% vs. 100% overall). Clinical encounters with treatment appropriate to diagnosis/assessment for child with malaria, pneumonia, or diarrhea improved (baseline: 71% vs. 81% overall). In addition to Integrated Supportive Supervision (ISS) on a quarterly basis, HWs were assigned to HPs daily to give technical support and on-the-job training to update HEW knowledge and skills. Under each HC catchment area there was also weekly supervisions to HPs by HEW Supervisors to monitor progress of activities towards the overall goal and objectives. In both districts, we observed that all randomly selected HPs for the survey had wall charts, graphs, plan versus achievements, and other necessary information that helped them with decision making. CS-23 Ethiopia, Final Evaluation Report, Save the Children, December 2012 172 Table 10: Availability of Information System, Training and Supervision Indicator Shebedino Lanfero Project (W%) BL EL BL EL BL EL Indicator #6: Information System - % HEW that maintain up-to-date records of sick U5 children 56.3 100 50 100 0 100 Indicator #7: Training in Child Health - % HEW who received training in child health 100 100 94 95 98 98 Indicator #7: Training in MNC - % HEW who received training in MNC 25 100 0 95 17 98 Indicator #8: Supervision - % HEW that received external supervision at least once in the last three months 100 100 94 100 98 100 Indicator # 11 – HEW Performance (Treatment) - % HEWs in which at least 80% clinical encounters have treatment appropriate to diagnosis/assessment 69 77 75 87 71 81 Fig. 9: Availability of Information system, Training and Supervision by District 0 10 20 30 40 50 60 70 80 90 100 Information System Training in child health Training in MNC External supervision Appropriate treatment % CS23 BL CS23EL Utilization of Sick Child Services The use of curative services for children under five increased (baseline: 0% [average] vs. 0.5 visits/year overall) while the use of GM for children under three declined (baseline: 0.02 vs. 0% [average] GM/year overall) (Table 11). Utilization of sick child services was better in Lanfero than Shebedino (ICCM register review) though there was improvement in both districts. Utilization of GM services declined in both districts. CS-23 Ethiopia, Final Evaluation Report, Save the Children, December 2012 173 Table 11: Utilization of Sick Child Services Performan ce/output Indicator Shebedino Lanfero Project (W%) BL EL BL EL BL EL Utilization of curative services (U5 children) Indicator # 9 - Annualized number of HEW clinical encounters for sick children per U5 population in catchment area 0.03 visits/ year 0.4 visits/ year 0.07 visits/ year 0.8 visits/ year 0.0 visits/ year 0.5 visits/ year Utilization of Growth Monitoring Services (U3 children) Optional indicator # 9b -Annualized number of HEW encounters for growth monitoring per U5 population in catchment area 0.001 GM/year 0.0 GM/year 0.003 GM/year 0.0 GM/year 0.02 GM/year 0.0 GM/year IV. DISCUSSION Maternal and Child Health Delivery services in HFs is increasing in both districts as of routine delivery services report. However, there is lack of potable water, especially at HPs to manage delivery services and to assure the cleanliness of the delivery room (almost all HPs in both districts lack water source in their HF compound---they fetch water from surrounding water source by jericans [plastic water container] to manage delivery services in their respective HF). This problem needs a creative resolution and likely discourages mothers from choosing HF for delivery. Supplies and Drugs At the time of the survey, all HFs had essential supplies and drugs, but from the experience during the life of the project, the availability of supplies and drugs at HFs, especially drugs, is not sustainable in HPs, which may discourage mothers from using HPs. As we have identified from focus group discussions, key informant interviews and discussion with v-CHW, sometimes families prefer to go elsewhere due to a shortage of drugs. Solutions have to be found at all levels in order to better equip HFs with essential supplies and drugs in a sustainable manner. Availability of Information System, Training, Supervision, Community Coordination and Laboratory Services Both at the HC and HP levels, documentation of reports, records, and register quality has improved. HWs and HEWs are well trained in critical child health services CS-23 Ethiopia, Final Evaluation Report, Save the Children, December 2012 174 including IMNCI/ICCM. Though training in child health was similar to baseline, the content of training has changed. Currently in both districts, all HEWs are trained in IMNCI/iCCM including pneumonia treatment. Supervision is happening regularly and there is community participation. Infection Control The quality and existence of infection control practices in HCs has improved from baseline, but reliable sharps and infectious waste disposal needs strengthening in some HFs, especially HPs. Some HFs did not have protected an infectious waste disposal area, and sharps and waste disposal practices were identified as needing additional attention in order to improve the overall infection control system. Health Workers Performance Key assessment tasks, treatment appropriate to diagnosis/classification and practice of counseling mothers improved from baseline. However, checking general danger signs, assessing feeding practice, assessing nutritional status and vaccination status was overlooked by most HWs. In principle, checking general danger signs is one of the Integrated Community Case Management (i-CCM) treatment protocol to be fulfilled by all HWs to save the life of sick children under five. Table Although overall counseling skills have improved, during treatment ,most HWs did not counsel mothers on how to administer all prescribed drugs. However, mothers were later counseled by the pharmacist when they collected the prescribed drugs from the pharmacy; the protocol calls for the HW to counsel caretakers on how to administer the prescribed drugs. In order to address these problems and improve performance, it is crucial to devise mechanisms to monitor performance quality at all HFs. The current system of drug dispensation from pharmacies has impacted this by providing the opportunity for counseling caregiver to the pharmacy professional during drug dispensation rather than to the HW at the under-five clinic. Utilization of Sick Child Services Annual clinical encounters for sick children improved from the baseline result, but GM declined since baseline, especially at the HPs. The strategy of integrating GM with immunization is good in order to avoid missed opportunities and works well, however GM is conducted for children under three years old while immunization is conducted for children under one year of age. After mothers/caretakers have completed immunization for their children, there is little reason for them to return for GM. This is one of the biggest challenges for improving GM coverage, as most mothers do not bring their children to HFs only for GM service. Education needs to be conducted to convince mothers to use GM to tract their children’s growth status and enable them take immediate action before the occurrence of growth faltering. This is being addressed through promotion of periodic growth monitoring and nutritional assessment sessions offered quarterly at the HP level. CS-23 Ethiopia, Final Evaluation Report, Save the Children, December 2012 175 ANNEXES Annex 1: Data collectors Training Schedule June 9-10, 2012 Day Activities June 9-10, 2012  Introduction of participants  Administrative information General information  Purpose of the assessment  Training objectives  Assessment protocol and techniques Introduction to Module 1 and 2 - Clinical Observation & Exit Interview Clinical Observation - Sick Child  Review the instrument  Discussion Caretaker Exit Interview – Sick Child  Review the instrument  Discussion Introduction to Module 3 and 4 Health Worker Interview  Review the instrument  Discussion Health Facility Checklist  Review the instrument  Discussion Module 5  Review the instrument  Discussion  Reviewing sample HFs selected for the assessment General Discussion & Wrap-up  Informed Consent  Review any particular difficulties in application of the forms  Team identification for field assessment  Question and Answer  Materials collection CS-23 Ethiopia, Final Evaluation Report, Save the Children, December 2012 176 Annex 2: Data Collection Schedule, Lanfero District, June 19-22, 2012 HF Code Selected HF Name Type of HF Assigned Surveyors Date of Assessment Departure Time 01 Tora HC Getenet, Nasir Denur 20/6/2012 7:00am 02 Mito HC Worku, Abdimuhin 21/6/2012 7:00am 03 Gababa HC Merteyesus, Habtamu 21/6/2012 7:00am 04 Archuma Wonte HC Worku, Mirteyesus 20/6/2012 7:00am 05 Girinzila shofode HP Getenet, Nasir Denur 20/6/2012 7:00am 06 Shefode Debar HP Getenet, Nasir Denur 20/6/2012 7:00am 07 Amiche HP Getenet, Nasir Denur 20/6/2012 7:00am 08 Luke Kudusa HP Getenet, Nasir Denur 20/6/2012 7:00am 09 Wonte Sostoro HP Getenet, Nasir Denur 20/6/2012 7:00am 10 Archuma gola HP Worku, Abdimuhin 21/6/2012 7:00am 11 Wotambo gobe HP Worku, Abdimuhin 21/6/2012 7:00am 12 Meded gagabo HP Worku, Abdimuhin 21/6/2012 7:00am 13 Girinzila gogilo HP Worku, Abdimuhin 21/6/2012 7:00am 14 Wante Doye HP Worku, Abdimuhin 21/6/2012 7:00am 15 Wante lola HP Merteyesus, Habtamu 21/6/2012 7:00am 16 Meded kusaya HP Merteyesus, Habtamu 21/6/2012 7:00am 17 Edeneba agawo HP Merteyesus, Habtamu 21/6/2012 7:00am 18 Girar Waragise HP Merteyesus, Habtamu 21/6/2012 7:00am 19 Rape Chefuna HP Merteyesus, Habtamu 21/6/2012 7:00am Total 19 CS-23 Ethiopia, Final Evaluation Report, Save the Children, December 2012 177 Annex 3: Data Collection Schedule, Shebedino District, June 11-15, 2012 HF Code Selected HF Name Type of HF Assigned Surveyors Date of Survey Departure time 01 Abela-Lida HC Habtamu, Wondimu 13/6/2012 7:00am 02 Dobe-toga HC Habtamu, Wondimu 12/6/2012 7:00am 03 Telamo HC Worku, Tilahun 13/6/2012 7:00am 04 Leku HC Mintesinot, Saba, Hyimanot 12/6/2012 7:00am 05 Dulecha HC Worku, Tilahun 12/6/2012 7:00am 06 G/keristos HC Mintesinot, Saba, Hyimanot 12/6/2012 7:00am 07 Morocho HC Mintesinot, Saba, Hyimanot 13/6/2012 7:00am 08 Ramada HP Habtamu, Wondimu 14/6/2012 7:00am 09 Murancho-Kutala HP Habtamu, Wondimu 14/6/2012 7:00am 10 Morocho-Shondolo HP Mintesinot, Saba, Hyimanot 14/6/2012 7:00am 11 Fura HP Mintesinot, Saba, Hyimanot 14/6/2012 7:00am 12 Harbe-shisho HP Mintesinot, Saba, Hyimanot 14/6/2012 7:00am 13 Howolso HP Mintesinot, Saba, Hyimanot 15/6/2012 7:00am 14 Midre-genet HP Habtamu, Wondimu 15/6/2012 7:00am 15 Alawo-anno HP Worku, Tilahun 12/6/2012 7:00am 16 Bonoya-miride HP Worku, Tilahun 12/6/2012 7:00am 17 Dobe-negasha HP Worku, Tilahun 12/6/2012 7:00am 18 Morocho-negasha HP Mintesinot, Saba, Hyimanot 14/6/2012 7:00am 19 Nure-dullacha HP Habtamu, Wondimu 15/6/2012 7:00am CS-23 Ethiopia, Final Evaluation Report, Save the Children, December 2012 178 20 Diramo-aferara HP Worku, Tilahun 12/6/2012 7:00am 21 Dila-aferara HP Habtamu, Wondimu 15/6/2012 7:00am 22 Sadeka HP Worku, Tilahun 13/6/2012 7:00am Total Sample 22 CS-23 Ethiopia, Final Evaluation Report, Save the Children, December 2012 179 Annex 4: Core Indicator Definitions CORE INDICATOR DEFINITIONS - RAPID HEALTH FACILITY ASSESSMENT INDICATORS WITH AN ASTERISK (*) ARE STANDARD INTERNATIONAL HEALTH FACILITY ASSMENT NETWORK (IHFAN) INDICATORS Area of Analysis Indic. # Domain Indicator Instrument Denominator Reference ACCESS OVERALL Geographic Access to Curative Services % population with year-round geographic access (within 5 km. or one hour) to an authorized provider of curative child health services DHO Interview Target population JHU Pneumonia Care Assessment Tool (JHU tool outlines procedure for pneumonia care, but this procedure has been generalized to any curative child care) 1 CHILD Service Availability Child* % HF that offer three basic child health services (growth monitoring, immunization, sick child care) HW Interview All HF IHFAN Services Offered Indicator for child services only / Component of SPA Indicator 2.1 INPUTS 2 Staffing* % HF with all clinical staff presetn in surveyed HF on the day of the survey HW Interview All HF IHFAN Staffing Indicator 3 Infrastructure* % HF with all essentail infrastructure on day of the survey (power, improved water source, functional latrine for clients, communication equipment, emergency transport, overnight beds, setting allowing auditory and visual privacy) HF Checklist All HF IHFAN Infrastructure Indicator CS-23 Ethiopia, Final Evaluation Report, Save the Children, December 2012 180 4 CHILD Supplies Child* % HF with all 5 essential supplies to support child health in HF on day of the survey (accessible and working scale for child, accessible and working scale for infant, timing device for diagnosis of pneumonia, spoon/cup/jug to administer ORS, ITNs) HF Checklist All HF IHFAN Supplies Indicator - only for child health supplies 4 NEO Supplies Newborn* % HF All 4 essential supplies to support newborn health in HF on day of the survey (resuscitation device, weighing scale, antibiotics and baby wraps) HF Checklist All HF 5 CHILD Drugs Child* % HF with all 5 first-line medications for child health on day of the survey (ORS, oral antibiotic for pneumonia, first-line oral antibiotic for dysentery, first-line anti￾malarial, vitamin A) HF Checklist All HF IHFAN Treatments Indicator - only for treatments specific to health area 5 NEO Drugs Newborn* % HF with the first-line medication for newborn sepsis on day of the survey HF Checklist All HF PROCESSES 6 CHILD Information System (Child and Newborn)* % HF that maintain up-to- date records of sick U5 children (age, diagnosis, treatment) and have report in last 3 months and evidence of data use HW Interview (record review) All HF IHFAN Health Service Statistics Indicator CS-23 Ethiopia, Final Evaluation Report, Save the Children, December 2012 181 7 CHILD Training in Child Health % HF in which interviewed HW reported receiving in￾service or pre-service training in maternal, child or neonatal health in last 12 months HW interview All HF Component of SPA Indicator 1.4 7 NEO Training in Maternal Neonatal Care % HF in which interviewed HW reported receiving in￾service or pre-service training in child health in last 12 months HW Interview All HF 8 Supervision % HF that received external supervision at least once in the last 3 months (supervision included one or more of the following: checked records or reports, observed work, provided feedback, gave praise, provided updates, discussed problems, OR checked drug supply)) HW interview All HF Component of SPA Indicator 1.4 PERFORMANCE 9 CHILD Utilization of Curative Services % HF with > 1.0 sick child visits in the last 12 month per U5 childen in catchment area HW interview (record review) Number of U5 children in HF catchment area Based on WHO standard Utilization indicator 10 CHILD HW Performance (Assessment) % HF in which all 5 key assessment tasks made by HW (check presence of general danger signs, assess feeding practices, assess nutritional status, check vaccination status) (benchmark 5 of 6 clinical Clinical Observation All HF Several Components of SPA Indicator 2.5 CS-23 Ethiopia, Final Evaluation Report, Save the Children, December 2012 182 observations) 11 CHILD HW Performance (Treatment of Sick Child) % HF in which treatment is appropriate to diagnosis for child with malaria, pneumonia, or diarrhea (benchmark: 5 of the 6 clinical observations) Clinical Observation All HF WHO HFS Core Indicator #7 12 CHILD HW Performance (Counseling for Sick Child) % HF in which the caretaker whose child was prescribed an antibiotic, antimalarial, or ORS, can correctly describe how to administer all drugs (benchmark: 5 of the 6 caretakers interviewed) Exit interview - child All HF WHO HFS Core Indicator #11 CS-23 Ethiopia, Final Evaluation Report, Save the Children, December 2012 183 Annex 5: Optional Indicator Definitions OPTIONAL INDICATOR DEFINITIONS - RAPID HEALTH FACILITY ASSESSMENT Area of Analysis Indic. # Domain Indicator Instrument Denominator Reference INPUTS Opt2 Availability of Immunizations % HF with all nationally-mandated vaccines in stock on day of survey HF Checklist All HF WHO HFS / BASICS IHFA Opt3 Availability of Guidelines % HF with all nationally-mandated guidelines for care of children available and accessible on day of survey HF Checklist All HF IHFAN Indicator #4 Opt4 Infection Control % HF with all infection control supplies and equipment available on day of survey HF checklist All HF IHFAN Indicator #2 PROCESSES Opt5 HF-Community Coordination % HF that have routine community participation in management meetings (with evidence through notes) OR have a system for eliciting client opinion, and evidence that client feedback is reviewed HW interview (record review) All HF Component of SPA Indicator 1.5 Opt6 Community Referral % HF that received at least one referral from CHW in the last month HW interview (record review) All HF … Opt9 Laboratory Facilities % HF with adequate laboratory facilities on site or able to send out HW interview All HF IHFAN Indicator #7 PERFORMANCE Opt10 Utilization of Child Preventive Services # annualized encounters for children per U5 population in HF catchment area: a) for vaccination, b) for growth monitoring and promotion HW interview (record review) All HF … CS-23 Ethiopia, Final Evaluation Report, Save the Children, December 2012 184 Annex 6: Tabulation Plan (Core Indicators) TABULATION PLAN - CORE INDICATORS Area of Analysis Indic. # Domain Indicator Data Element(s) for Numerator Data Element(s) for Denominator Notes ACCESS OVERALL Geographic Access % population with year-round geographic access (within 5 km. or one hour) to an authorized provider of curative child health services Sum of populations of villages/neighborhoods that have year-round access to curative child health services (DHO Form, sum of column J figures where column K = YES) Total population in project area (DHO form - sum of all column J figures) 1 CHILD Service Availability* % HF that offer ALL three basic child health services (growth monitoring, immunization, sick child care) # HF with 3 basic child health services (Q.401/ 01A+B > 30 AND Q.401/02A+B > 4 AND Q.401/03A+B > 4) # HF surveyed INPUTS 2 Staffing* % staff in HF who provide clinical services were working (either in HF or in outreach activities) on the day of the survey # clinical care staff present on day of survey (Q.402A, boxes 01+02+03+04) # sanctioned staff working in HFsurveyed (Q.402B, boxes 01+02+03+04) this is the index value form of this indicator 3 Infrastructure* % essential infrastructure in HF on day of the survey (power; improved water source; functional latrine for clients; communication # Overnight beds (Q.301=1 AND 302=1); Communication (Q.303=1,2,or 3); Transport (Q.304=1 or 2); Electricity (Q.305 = 1 OR Q.306 = 1 or 3); Usable latrine (Q.307 = 1 AND Q.308 = 1,2,3, or 4 # HF surveyed x 7 infrastructure items this is the index value form of this indicator CS-23 Ethiopia, Final Evaluation Report, Save the Children, December 2012 185 equipment; emergency transport; overnight beds; setting allowing auditory and visual privacy) AND 309 = 1); Water from improved source (Q.310 =1 AND 311 = 1,2,3,4,5, or 6); Auditory and visual privacy (Q.312 = 1) 4 CHILD Supplies* % essential supplies to support child health in HF on day of the survey (accessible and working scale for child, accessible and working scale for infant, timing device for diagnosis of pneumonia, spoon/cup/jug to administer ORS) # essential supplies available in surveyed HF (Items 02-07 for which Q.313a = 1 or 2); items 02 - 05 are also functioning (Q.313b = 1 for all items 02 - 05) # HF surveyed this is the index value form of this indicator 4 NEO Supplies Neonatal % of 4 essential supplies to support newborn health in HF/CHW on day of the survey (resuscitation device, weighing scale, antibiotics and baby wraps) # essential neonatal supplies available in surveyed HF (Items 01-04 for which Q.313NEOa = 1 or 2); items 01 - 03 are also functioning (Q.313NEOb = 1 for all items 01 - 03) # HF surveyed x 4 essential items this is the index value form of this indicator 5 CHILD Drugs* % of first-line medications for child health in HF on day of the survey (ORS, oral antibiotic for pneumonia, first-line # first-line child drugs available and at least one valid (not expired)in HF on day of survey (Q.314, 01 - 05 all = 1 or 2) # HF surveyed x 5 essential drugs this is the index value form of this indicator CS-23 Ethiopia, Final Evaluation Report, Save the Children, December 2012 186 oral antibiotic for dysentery, first-line antimalarial, vitamin A) 5 NEO Drugs Neonatal % of first-line medications in HF on day of survey (for newborn sepsis and eye infections) # of first-line medications for newborns on day of the survey (Q.314NEO 01 and 02 = 1 or 2) # HF surveyed x 2 essential drugs this is the index value form of this indicator PROCESSES 6 CHILD Information System* % HF that maintain up-to-date records of sick U5 children (age, diagnosis, treatment) and for HF: have report in last 3 months and evidence of data use # HF in which all 3 data elements (age, diagnosis, treatment) are all complete in sick child register (Q.409 = A,B,C all circled) AND in which last entry is within last 7 days (Q.410 = 1) AND report written in last 3 months (Q.411 = 1 or 3) AND there is evidence of data use (Q.412 = A, B, or C) # HF surveyed 7 CHILD Training in Child Health % HF in which interviewed HW reported receiving in￾service or pre-service training in child health in last 12 months # HF in which interviewed HW received training in at least one child health topic in last 12 months (Q.404, 01 - 09 at least one response = 1) # HF surveyed SPA indicator is "at least 50% HW trained in…." but in order to make this more feasible, this instrument only asks about the interviewee. 7 NEO Training in Maternal- % HF in which interviewed HW # HF in which interviewed HW received training in at # HF surveyed SPA indicator is "at least 50% CS-23 Ethiopia, Final Evaluation Report, Save the Children, December 2012 187 Neonatal Care reported receiving in￾service or pre-service training in maternal neonatal care in last 12 months least one child health topic in last 12 months (Q.404, 10 - 12 at least one response = 1) HW trained in…." but in order to make this more feasible, this instrument only asks about the interviewee. 8 Supervision % HF that received external supervision at least once in the last 3 months (supervision included one or more of the following: checked records or reports, observed work, provided feedback, gave praise, provided updates, discussed problems)) # HF receiving supervision in last 6 months that included more than just delivering supplies (Q.406 = 1 and Q.407, 02 - 08, at least one response = 1) # HF surveyed CS-23 Ethiopia, Final Evaluation Report, Save the Children, December 2012 188 TABULATION PLAN - CORE INDICATORS (continued) Area of Analysis Indic. # Domain Indicator Data Element(s) for Numerator Data Element(s) for Denominator Notes PERFORMANCE 9 CHILD Utilizati on of Curative Services Annualized rate of clinical encounters for sick children per U5 population # clinical encounters with sick childen under 5 in last three complete calendar months (Q.413 x 4) # children in catchment area (Q.400B or form DHO or from Census data ) 1. Can get this data from routine HIS, but this is here as a validation check. 2. "Sick children" is chosen to give comparable data across different projects and health systems; however, the project may want to collect data on utilization of other types of services as well (e.g., growth monitoring, vaccination). These are included as optional indicators. 10 CHILD HW Perform ance (Assess ment) % of four key assessment tasks are made by HW (check presence of general danger signs, assess feeding practices, assess nutritional status, check vaccination status) # key assessment tasks complete (Number of answers = "Y" for Q.103A,B,C AND Q.104A,B,C) # sick child clinical encounters observed in all HF x 4 assessment tasks this is the index value form of this indicator 11 CHILD HW Perform ance (Treatm ent) % HF clinical encounters in which treatment is appropriate to diagnosis (for encounters in # sick child clinical encounters in which treatment was correct (Supervisor recode - Indicator HF: # sick child clinical encounters observed in all HF CS-23 Ethiopia, Final Evaluation Report, Save the Children, December 2012 189 which at least one of the presenting problems was fever, breathing problem, or diarrhea) (from record review for CHW / from Clinical Obs. for HF) #11 = 1 on Clinical Obs. Form) 12 CHILD HW Perform ance (Counse ling) % HF clinical encounters in which the caretaker whose child was prescribed an antibiotic, antimalarial, or ORS, can correctly describe how to administer all prescribed drugs # clinical encounters in which antibiotic, antimalarial, or ORS prescribed in which caretaker correctly describes the dose, frequency and duration of medication administration for ALL prescribed medications (Supervisor Recode - Indicator #12 = Y on Exit Interview Form ) # sick child clinical encounters observed in which antibiotic, antimalarial, or ORS prescribed (Q.201 = 1) CS-23 Ethiopia, Final Evaluation Report, Save the Children, December 2012 190 Annex 7: Tabulation Plan (Optional Indicators) TABULATION PLAN - OPTIONAL INDICATORS Area of Analysis Indic. # Domain Indicator Numerator Denominator Notes INPUTS Opt1 Availability of Immunizations % HF with all nationally-mandated vaccines in stock on day of survey # HF with all nationally￾mandated vaccines on day of survey (Q.314A, 01 - 04 all = 1 or 2) # HF surveyed Opt2 Availability of Guidelines % HF with all nationally-mandated guidelines for care of children available and accessible on day of survey # HF with all nationally￾mandated guidelines on day of survey (Q.314B, 01 - 05 all = 1) # HF surveyed Opt3 Infection Control % HF with all infection control supplies and equipment on day of survey # HF with all infection control supplies and equipment on day of survey, including all supplies available (Q.314c, all items 01 - 06 = 1 or 2) AND waste disposal for sharps and infectious waste is adequate (Q.314D-i = 1-6 AND Q314D-ii = 1-6 AND 314E-i = 1 or 2 AND 314E-ii = 1 or 2) # HF surveyed PROCESSES Opt4 HF-Community Coordination % HF with routine community participation in management meetings (with evidence through # HF that have routine community participation (Q.407B = A, B, or C) AND evidence that feedback is reviewed (Q.407C = A or B) # HF surveyed CS-23 Ethiopia, Final Evaluation Report, Save the Children, December 2012 191 notes) OR have a system for eliciting client opinion, and evidence that client feedback is reviewed Opt5 Community Referral % HF that received at least one referral from CHW in the last month # HF receiving at least one referral from a CHW in the last month (Q.407F = 1) # HF surveyed Opt8 Laboratory % HF with adequate basic laboratory facilities on site or ability to send out # HF with supplies to do eight basic laboratory tests or to send out (CBC, H/H, malaria RDT, urine glucose, urine proterin, HIV rapid test, AFB stain, syphilis testing) (Q417C 01-08 all = 1 or 2) # HF surveyed IHFAN laborato ry indicator PERFORMANCE Opt9 Utilization of Preventive Services Annualized rate of encounter for children for immunization / growth monitoring per U5 population in project area # encounters with children for immunization / growth monitoring in last 3 complete months (for immunization: Q.414A, sum answers for all HF / for growth monitoring: Q.414B, sum of answers for all HF) x 4 # U5 children in project area CS-23 Ethiopia, Final Evaluation, Save the Children, December 2012 192 Annex 12B: Supervising Health Extension Workers in Shebedino District in Ethiopia’s Southern Nations and Nationalities People’s Region: A Firm Foundation to Build Upon Hailu Tesfaye, Tedbabe Degefie, David Marsh Abstract Background: Ethiopia’s SNNPR has trained 638 health workers as Health Extension Worker (HEW) Supervisors to track and support Health Extension Program (HEP) implementation and HEW performance. Objectives: To describe current HEW supervision in SNNPR and inform the design of a feasible supervision tool and methodology. Methods: We conducted eight, in-depth interviews of all Supervisors in Shebedino District, plus their zonal and regional Supervisors, using semi-structured question guides to characterize the current plan, implementation, content, and skill of Supervisors and use of supervision findings. We used programmatically relevant criteria to categorize findings as high, medium, or low. Results: The HEW supervision occurred less frequently than planned, mainly due to transport shortages. Supervision coverage of HEWs was high. The frequency varied according to level: high at regional, medium at zonal and district levels, and low at some health centers. Supervision tools were mostly unavailable, and documentation of findings was low. The use of findings was medium at all levels. Most Supervisors were untrained, although training was available. Conclusions: Supportive supervision for HEP was in place, but there is a need for systematization through training and availability and use of supervision tools. Transport constraints demand fresh thinking, including less frequent, higher quality supervision, and engaging community in supervision. Introduction The Health Extension Program (HEP) is a pro-poor strategy in which trained, supervised Health Extension Workers (HEWs) deliver mainly preventive interventions to communities and households. HEWs are women who are well-educated (> grade 10) and well-trained (one year of theoretical and practical training). A successful HEP requires strong training and strong supervision. Assuring the quality of HEW performance, inseparably linked to supervision, remains a challenge. The Ministry of Health (MoH) assigns one HEW Supervisor to work full-time to supervise five HEWs at their respective Health Posts (HPs). The HEW Supervisors, based at Health Centers (HC), report to District Health Office (DHO). They are supposed to track HEP implementation and HEW performance, providing technical support to HEW and identifying areas for improvement. HEW Supervisors are either Environmental Health professionals or Public Nurses trained for one month in the HEP package and in supportive supervision. Supervisor training is competency based with 60% practical session and field practice. MoH plans to train 3200 HEW Supervisors in one year; SNNPR has trained its allotted 638 HEW Supervisors. CS-23 Ethiopia, Final Evaluation, Save the Children, December 2012 193 There are currently two supervision systems: the Integrated Supportive Supervision (ISS) introduced in 2006 and HEP Supervision introduced in 2007/8. The ISS Tool was developed before the supervision system for the HEP and lacks a section for HEP. The RHB, SZHD and Shebedino DHOs use the ISS tool and methods (frequency, sampling, etc); the HEP teams in the RHB and DHO use HP Supervision Checklist. Save the Children, USA (SC) initiated a five-year (2008-2012) health project in two districts in Southern Nations Nationalities People’s Region (SNNPR) to strengthen the delivery of child survival interventions through HEWs. During program design and planning the Save the Children team was reminded that HEW supervision remained a challenge. MoH partners prioritized it as the most important area for operations research. We aimed to describe and understand the current HEW supervision processes in SNNPR, thereby to inform the design of a feasible supervision tool and methodology. Methods Shebedino District has 35 rural kebeles, 30 Health Posts (HPs) and five HCs. The planned HEP supervision structure has: two experts at the Regional Health Bureau (RHB), one HEP Coordinator at both the Sidama Zone Health Department (SZHD) and at Shebedino District Health Offices (DHO), and one HEW Supervisor for every five HPs at each HC. We intended to interview all the Supervisors at each level. We conducted in-depth interviews over two weeks (June 1-15, 2009) in two sessions at each level with eight key informants: two HEP experts at RHB, one HEP coordinator at the SZHD, two HEP coordinators at Shebedino DHO, and three HEW Supervisors from three Shebedino HCs. Though Shebedino has five HCs, only three had HEW Supervisors in place. We used semi-structured question guides to characterize: (1) the current supervision plan; (2) its implementation; (3) explanations for variance, if any; (4) supervision content; (5) job aids; (6) use of findings; (7) Supervisor’s training; and (8) supervision of Supervisors. After obtaining respondents’ oral consent, we wrote detailed notes. We tabulated the responses thematically and categorized variables as high, medium and low, as follows: Frequency – The proportion of planned supervisions that actually occurred at each level in a given period (high: >75% planned supervision occurred; medium: 50-75%; low: only <50% planned supervision occurred). Coverage – The proportion of HPs actually supervised in a given period (high: >75%; medium: 50-75%; low: <50%). Checklist availability – High: checklist available, used and seen; medium: available but not used; low: no checklist Use of Data – Reported use of supervision findings for: (a) immediate feedback, (b) written feedback, (c) action planning based on the gaps, (d) informing the responsible actors and next level Supervisors, and (e) rewarding the best performances (high: 5/5; medium: 4/5; low: 3/5 or less). CS-23 Ethiopia, Final Evaluation, Save the Children, December 2012 194 Training in HEP Supervision – High: Master TOT; medium: 1 month; low: not trained or incomplete). Supervision of Supervisors – High: Supervisor was supervised every month and given feedback, documented; medium: Supervised every quarter and given feedback and documented; low: Supervision < quarterly. Results The RHB planned quarterly supervision of a sample of HPs (four HPs per district) using either the ISS Tool or the HEP Supervision Checklist. The ISS tool was used once, while the RHB HEP team completed the four scheduled supervision visits using the HEP Supervision Checklist. The HEP team visited all sampled HPs per plan. The planned content of the supervision addressed the HEP package (disease prevention and control, family and child health, hygiene and environmental health, health education and communication) and other health interventions. Actual supervision included reviewing the plan and performance for the HEP package, record review, on-site problem-solving, inquiring about but not inspecting supplies, reviewing monthly reports and occasionally liaising with community (by one Supervisor). Some RHB Supervisors were trained in supervision – or even as master trainers – while others had no training. RHB Supervisors were not supervised. The ZHD planned quarterly supervision of the DHO and HPs, but accomplished only three of the four visits (75%) due to transport, workload, and time constraints. The focus of the supervision was the HEP package, as assessed with the ISS Tool. ZHD used the findings to track activity and guide discussions with the decision-making ZHD management team. The HEP Supervisor had not been trained in supervision, nor had the RHB supervised the ZHD in last year. The DHO planned quarterly supervision and accomplished 75%-100% of its plan, less than complete due to shortages of transport, manpower and time. The focus of supervision was the HEP package and other health services, using the ISS Tool. Supervisors used findings for on-the￾spot feedback, and the district’s HMIS Committee used findings to revise plans in the case of low performance. This committee (composed of DHO, administration and other sector offices) also reviewed the content, completeness and use of the HMIS at the district level. The DHO Head and Deputy Head supervised the HEP Supervisors during the ISS of other health services. Neither district Supervisor was trained in ISS or HEP supervision, but they were supervised twice by the ZHD in last year at the DHO. The three HEW Supervisors1 at the Health Center level had different supervision plans. Some planned to supervise one HP per day while others planned two supervisions per week or even quarterly supervision. They accomplished only 40-60% of the plan due to lack of transport, budget for per diem, fuel, and supplies like stationery and due to the demands of work and meetings. The content of supervision included the HEP package, community sensitization, meeting volunteers and tracking achievements versus plan. They visited all planned HPs at each visit. They were not able to produce the HEP Supervision Checklist. The findings informed feedback, reporting and re-planning. Training in supervision varied: one received the full one 1 Two of the five (40%) HCs had no HEW Supervisor (one had resigned and had not been replaced; the other had never been deployed). CS-23 Ethiopia, Final Evaluation, Save the Children, December 2012 195 month; one did not complete it; the third had had none. They were supervised by DHO Supervisors two to three times in a quarter at the HC. “If I had all the necessary transport means and regular supply of job aids like stationery materials and checklists AND on top of this I get my per diem as is the case for EPI outreach work, I would have done a wonderful job and I am sure I and the HEWs would have implemented all the16 packages and I would have reached all the HPs I planned to reach and fulfilled all the supervision objectives for which I am trained.” WBD, HC, HEW Supervisor. HEP supervision varied by the level of supervision (Table 4). The frequency of supervision was high at regional level, medium at zonal and district levels, and low at some HCs. Coverage of supervision was high at all levels. Checklist availability was low at most levels, and the use of data was medium at all levels. Both training and supervision of Supervisors were medium to low, with low levels of training at zonal and district levels and low level of supervision of Supervisors at regional and zonal levels. Conclusion The beginnings of supportive supervision for the HEP were in place in Shebedino District. The DHO assigned Supervisors to three of the five HCs. These Supervisors reported to the DHO, tracked HEP implementation and HEW performance, provided technical support to HEW and identified areas of improvement. Even supervision of Supervisors was occurring. There were training manuals and supervision checklists were included in the training. Supervision training was competency-based and was sometimes used. There were also several areas for improvement. (1) Most Supervisors (from HC) were under￾trained. (2) While supervision training was competency-based, supervision itself was not fully competency-based because it was limited to DHO and HC levels, which precluded direct observation at HPs. (3) Supervision was not systematic, lacking checklists at all levels except for the HEP team at RHB level. (4) Supervision was under-implemented. (5) Supervision coverage was not readily obtainable or deemed important – i.e., it was not apparent if remote HPs were ever supervised. Only three of the five HCs had HEW Supervisors leaving 50% of the HPs without supervision. (5) HEW Supervisors lacked a uniform supervision plan. (6) HEW Supervisors at HCs were regularly supervised unlike Supervisors at regional and zonal levels. (7) Documentation of Supervisory findings was poor at all levels. (8) Alternative supervisory methods were lacking, i.e., HEWs never came to the HC Supervisors. (9) Community feedback was not part of supervision. (10) Transport challenges were real, but should have only constrained supervision to remote HPs. (11) No system identified, documented or rewarded good performances or exemplary supervision. (12) Typical with new programs, integration of pre- and post-HEP supervision was lacking. In conclusion, supervision is valued and in place and would benefit from fine-tuning. We recommend: 1. Supervision check-lists and job aids to: a. remind Supervisor of his/her competencies CS-23 Ethiopia, Final Evaluation, Save the Children, December 2012 196 b. assess his/her supervisees’ competencies. 2. Feasible supervision schedules. 3. Testing alternative supervision methods. 4. Competency-based supervision of Supervisors. 5. Continued competency based training for Supervisors 6. Transport/logistics support to HEW Supervisors 7. Recognition of exemplary Supervisory behavior. Table 1: Level, plan and performance of supervision for HEP at RHB, SZHD, Shebedino DHO and HCs, June 2009. Level ISS HEP Frequency Per Year Sample Size Frequency Per Year Sample Size Plan Actual Plan Actual Plan Actual Plan Actual RHB# 1 1 1-2HP per district 1-2HP per district** 4 4 2 HC and 4 HP per District 2 HC and 3-4 HP per district** ZHD 4 3 3 HP per district 3 HP per district DHO* 4 4 32 HP per month 32 HP per month by 2 Supervisors HC 4-52 4-24 5 HP per week 1-2 HP per week *Shebedino has 66 HEWs and 32 HPs ** Mostly achieve planned visits to HPs but in a few instances they may not visit a HP if the HEW is in a meeting, workshop or HP is inaccessible. # The SNNPR has 133 Districts (8 are Special Districts) and Sidama Zone has 19 Districts. CS-23 Ethiopia, Final Evaluation, Save the Children, December 2012 197 Table 2: Key informants’ educational back ground, knowledge and experience in supervision SN Current Position at Interview Age Sex Educational Status Trained in supervision (Y/N, for how long) Experience in Supervision (Yrs) 1 Family Health Department Head 52 M MD, MPH Yes in ISS for 5 days Supervisory position for 15 years 2 HEP expert 57 M BSc in Health Care Management Yes, 3 weeks, master trainer in HEP supervision 5 years in supervision and 30 yrs with MoH 3 SZHD HEP Coordinator 39 F Senior clinical Nurse, Diploma in Development Studies No 7 years in supervision and >20 yrs with MoH 4 DHO, HEP Coordinator 47 M Senior Clinical Nurse No 28 years MoH service and 1 year supervision experience 5 DHO, HEP Coordinator 34 M Senior Clinical Nurse No 13 yr in MoH and 7 yrs in supervision 6 Dulecha HC, HEW Supervisor 24 M Senior Clinical Nurse Yes 2 years MoH experience and 1 year supervision 7 Leku HC, HEW Supervisor 26 M Environmental Health Yes, but incomplete 6 years MoH and 2 years supervision 8 Telamo HC,HEW Supervisor 25 M Senior Clinical Nurse No 5 years MoH and 1 year supervision CS-23 Ethiopia, Final Evaluation, Save the Children, December 2012 198 Table 3: Summary result of key informant in depth interview for HEP Supervisors Questions RHB HEP Expert RE1. Getachew Assefa RE2. Dr. Sahile Sita ZHD HEP Co ZE1. Sr. Adanech Tadele Woreda HEP Supervisor (2) WS1.Tadesse Demisse WS2.Yona Hechera HEW Supervisor HS1.Wudeneh Beza-DulechaHC HS2.Kedilmeles Asfaw-Leku HC HS3. Alemu Bokolla-Telamo HC 1. What is the current supervision plan? RE1. Quarterly for sample HEP/Regional Q6mo RE2. Annually ZE1. The plan is on quarter basis to WHO WS1.Quarterly WS2.Same HS1. 1HP per day HS2. Two times per week for each kebele HS3. On quarterly basis 2. How Much of the plan is implemented? RE1. 4x at RHB level in 2000EC, we have monthly meeting as a team RE2. All the planned activities are implemented ZE1. Three times (75%) WS1.3x (75%) WS2.4x (100%) HS1. 2HP per week (40%) HS2. Six kebeles seen regularly out of ten (60%) HS3. implemented three times 3. What explains the Variance? RE1. No Variance RE2. No variance ZE1. Transport problem, workload and time constraint WS1. 1. Shortage of transport. 2. Shortage of manpower in the office. 3. Shortage of time due to different activity HS1. 1. No Transport 2. No Incentive/per diem 3. Supplies shortage: stationary HS2. 1. Shortage of transport 2. Fuel shortage for motor bike HS3. Transport problem, workload and overlapping with meetings 4 What is the content of supervision? RE1. 16 package of HEP, Kebele plan RE2. All health interventions ZE1. Health Extension packages (HEPs) (Intervention) WS1.All HEPs and integrated health activities at health center and HP level WS2. All HEP package and Health programs including environmental health, Surveillance, HIV etc. HS1. HEP 16 package, community support/sensitization/meet leaders and VCHW HS2. Activities mainly focused on Community compulsion. HS3. Documentation of plan and achievements 5 What job aids do RE1. RHB HEP team checklist, ZE1. Integrated supervision WS1.Check list HS1.Checklist (from training, Not seen CS-23 Ethiopia, Final Evaluation, Save the Children, December 2012 199 NB: RE=Regional Expert, ZE=Zonal Expert, WS=Woreda Supervisor/HEP Co, HS=HEW Supervisor, DHO=District Health Office, RHB=Regional Health Bureau, ZHD=Zonal Health Department, ISS=Integrated Supportive Supervision, HEP=Health Extension Program, HEW=Health Extension Worker. Supervisors use? ISS tool RE2. Integrated supportive supervision checklist checklist having all activities WS2.Same & was not available on interview) HS2. No chick list used during supervision HS3. There is no job aid . 6 How are data from supervision used? RE1. Spot feedback and support, Written feedback and action plan RE2. For re-planning, for discussion and monitoring the progress of activities ZE1. For analyzing activity progress and for discussing the progress with decision making body WS1. For feedback and for HMIS Committee meeting discussion and to re plan the low performance activities WS2. Same HS1. Feedback to HEWs HS2. For report and no written feedback given. HS3. For discussion, decision making and re-planning 7 What training do Supervisors received? RE1. Master TOT and HC & Woreda person trained for 3 months RE2. Some Supervisors took supervisory skill training while others not. ZE1. No specific training was given WS1.No formal training WS2. No Training HS1. 3 months HEW supervisory training HS2. Health extension supervision training but not completed HS3. No training was given .8 How are Supervisors supervised? RE1.RHB supervises DHO and DHO need to do monthly supervision to HC RE2. There was no integrated supervision from FMoH, but each department was supervising (separately) depending on the area of his/her responsibility. ZE1. There is no integrated supervision from RHB WS1.By District health office Head/Deputy Head and ZHD 2x in 2001EC FY ZHD gave feedback only once WS2. Same HS1. By DHO HEP Supervisors, had 1SS in 2 months, has not used checklist and no written feedback but verbal HS2.By Woreda Supervisor quarterly HS3. On quarterly basis by DHO using supportive supervision checklist CS-23 Ethiopia, Final Evaluation, Save the Children, December 2012 200 Table 4: Supervision of HEP/HEWs: Key characteristics by supervision level Characteristics RHB ZHD DHO HC Frequency High Medium Medium-High Low-Medium Coverage (HP) High High High High Checklist High Low Low Low Use of Data Medium Medium Medium Medium Training Medium Low Low Medium Supervision of Supervisors Low Low Medium Medium CS-23 Ethiopia, Final Evaluation, Save the Children, December 2012 201 Annex 12C: ICCM in Liben Ethiopia Ethiopian Journal of Health Development, Vol 23, No 2 (2009) CS-23 Ethiopia, Final Evaluation, Save the Children, December 2012 202 CS-23 Ethiopia, Final Evaluation, Save the Children, December 2012 203 CS-23 Ethiopia, Final Evaluation, Save the Children, December 2012 204 CS-23 Ethiopia, Final Evaluation, Save the Children, December 2012 205 CS-23 Ethiopia, Final Evaluation, Save the Children, December 2012 206 CS-23 Ethiopia, Final Evaluation, Save the Children, December 2012 207 CS-23 Ethiopia, Final Evaluation, Save the Children, December 2012 208 Annex 13: Project Data Form CS-23 Ethiopia, Final Evaluation, Save the Children, December 2012 209 CS-23 Ethiopia, Final Evaluation, Save the Children, December 2012 210 CS-23 Ethiopia, Final Evaluation, Save the Children, December 2012 211 CS-23 Ethiopia, Final Evaluation, Save the Children, December 2012 212 CS-23 Ethiopia, Final Evaluation, Save the Children, December 2012 213 CS-23 Ethiopia, Final Evaluation, Save the Children, December 2012 214 CS-23 Ethiopia, Final Evaluation, Save the Children, December 2012 215 CS-23 Ethiopia, Final Evaluation, Save the Children, December 2012 216 CS-23 Ethiopia, Final Evaluation, Save the Children, December 2012 217 CS-23 Ethiopia, Final Evaluation, Save the Children, December 2012 218 CS-23 Ethiopia, Final Evaluation, Save the Children, December 2012 219 CS-23 Ethiopia, Final Evaluation, Save the Children, December 2012 220 CS-23 Ethiopia, Final Evaluation, Save the Children, December 2012 221 CS-23 Ethiopia, Final Evaluation, Save the Children, December 2012 222 CS-23 Ethiopia, Final Evaluation, Save the Children, December 2012 223 Annex 14: Grantee Plans to Address Final Evaluation (FE) Findings Though the child survival project is ending as of September 29, 2012 Save the Children will take the actions detailed below to address final evaluation recommendations and findings.  Conduct FE preliminary findings debriefing with key stakeholders including the Regional Health Bureau (RHB), Sidama and Silite Zonal Health Department (ZHD) and District Health Offices (DHOs) in Shebedino and Lanfero.  Share key findings at the annual review meeting at the end of September.  Disseminate printed reports with key stakeholders like RHB, ZHD, DHOs, FMoH, UNICEF, JSI/IFHP and JSI/L10K.  Share report and findings with USAID Mission and MCHIP.  Share the KPC, HFA, FE findings and achievements with Regional, Zonal and District Health Office.  Use findings for future health planning and proposal development.  Scale-up lessons learned to other districts and in other CS projects, e.g., the pregnant mother forum. Save the Children has secured a two year grant from Save the Children/Korea and KOICA that began in August 2011 and will continue through August 2013. This support was designed to address many of the critical gaps in the CS-23 project and focus on addressing the MTE and FE recommendations, especially related to improving maternal and newborn health. We have also managed to maintain the services of one coordinator and two field project officers in each of the districts to follow up and address the recommendations of the FE at the district level. The iCCM project supported by UNICEF in both Shebedino (by Save the Children) and Lanfero District (by JSI/L10K), which runs until July 2013, will also complement and address the FE recommendations, particularly in relation to iCCM implementation quality and care-seeking for newborn care. CS-23 Ethiopia, Final Evaluation Report, Save the Children, December 2012 224 Annex 15: Grantee Response to Final Evaluation Findings Not Applicable.