FINAL EVALUATION REPORT: LIRA DISTRICT CHILD SURVIVAL PROJECT IN UGANDA December 2013 This publication was produced at the request of the United States Agency for International Development. It was prepared independently by Sue Leonard, Evaluation Team Leader, Mary Helen Carruth, MTI HQ Senior Advisor, Maternal and Child Heath, Todd Nitkin, MTI HQ Senior Advisor, Monitoring and Evaluation, and Lydia Akulo, Child Survival Project Manager Acknowledgements This final Evaluation is the result of the work of many people over the course of several months. Many thanks go to all of those who organized, implemented, and analyzed the surveys that provided the information base for the qualitative review. Thanks also go to the Evaluation Team who worked many long hours to gather and analyze the information from the field; and to the community volunteers, health facility staff and community and facility committee members who shared their wisdom with us. Most especially, thanks go to the CSP staff and management for implementing a successful program in collaboration with the District Health Office as well as the headquarters people who often work behind the scenes to help make such a program happen. Thank you to you all, Sue Leonard, Evaluator Final Evaluation Report: Lira District Child Survival Project in Uganda EDUCATING COMMUNITY HEALTH WORKERS, CAREGIVERS, AND HEALTH PROVIDERS REGARDING MATERNAL CHILD HEALTH AND EARLY CHILDHOOD DEVELOPMENT IN A POST-CONFLICT ENVIRONMENT December 1, 2013 GSH-A-00-09-00012-00 DISCLAIMER The author’s views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. CONTENTS Acronyms............................................................................................................................................................i Executive Summary ..........................................................................................................................................ii Evaluation Purpose and Evaluation Questions...........................................................................................ii Evaluation Purpose ....................................................................................................................................ii Evaluation Questions ................................................................................................................................ii Project Background.......................................................................................................................................... 2 Evaluation Methods and Limitations............................................................................................................. 6 Findings, Conclusions, and Recommendations........................................................................................... 7 Findings......................................................................................................................................................... 7 Conclusions.............................................................................................................................................. 20 Recommendations.................................................................................................................................. 21 Annexes Annex I. Program Learning Brief(s): Evidence Building Annex II. List of Publications and Presentations Related to the Project Annex III. Project Management Evaluation Annex IV. Work Plan Table Annex V. Rapid CATCH Table Annex VI. Final Knowledge, Practice, and Coverage Report Annex VII. Community Health Worker Training Matrix Annex VIII. Evaluation Scope of Work Annex IX. Evaluation Methods and Limitations Annex X. Data Collection Instruments A: Rapid Health Facility Assessment Report B: Data Collection Questionnaires Annex XI. Sources of Information Annex XII. Disclosure of Any Conflicts of Interest Annex XIII. Statement of Differences Annex XIV. Evaluation Team Members, Roles, and Their Titles Annex XV. Final Operations Research Report Annex XVI. Operations Research Brief Annex XVII. Stakeholder Debrief Powerpoint Presentation Annex XVIII. Project Data Form Annex XIX. Optional Annexes A: OCA Action Plan B: Project M and E Matrix C: Final Evaluation Results Meeting D: Response to midterm evaluation recommendations E: Project Checklists and Referral form ACRONYMS ANC Antenatal Care BL Baseline CSHGP Child Survival and Health Grants Program C-IMCI Community-based Integrated Management of Childhood Illness CSP Child Survival Program DHO District Health Office ECD Early Childhood Development ER Emergency EBF Exclusive Breastfeeding FE Final Evaluation HE Health Education HF Health Facility HH Household HHI Hands to Hearts International HP Health Provider HUMC Health Unit Management Committee iCCM Integrated Community Case Management IEC Information, Education and Communication IMCI Integrated Management of Childhood Illnesses IYCF Infant and Young Child Feeding KPC Knowledge, Practice and Coverage LC1 Local Council One (village level elected political official) MLG Mother Leader Groups ML Mother Leaders MNC Maternal Newborn Care MoH Ministry of Health MTE Mid-Term Evaluation MTI Medical Teams International NU-Heights Northern Uganda-Health Integration to Enhance Services PE Peer Educators RHFA Rapid Health Facility Assessment SBC Social and Behavior Change USAID U.S. Agency for International Development VHT Village Health Team (individual volunteers not teams) Final Evaluation Report: Lira District Child Survival Project in Uganda December 2013 Final Evaluation i Executive Summary Final Evaluation Report: Lira District Child Survival Project in Uganda This project was funded by the U.S. Agency for International Development through the Child Survival and Health Grants Program. December 2013 Evaluation, Purpose, and Evaluation Questions The FE provided an opportunity for all project stakeholders to take stock of accomplishments and lessons learned and to listen to beneficiaries at all levels. The FE Report will be used by the following audiences as a source of evidence to help inform decisions about future program designs and policies: • The Lira District Health Office, health facility staff, and Health Unit Management Committees (HUMC) • Uganda Ministry of Health • Village Health Teams (VHT) and Mothers Leader Groups volunteers • Hands to Hearts International (HHI) • The USAID CSHGP and the Uganda USAID Mission • Medical Teams International (MTI) HQ and field staff Key Evaluation Questions 1) To what extent did the project accomplish objectives stated in the DIP? Objective No.1: Communities assume responsibility for their own health through strengthened community capacity Objective No.2: Improved health (Community-based Integrated Management of Childhood Illness) and child care (Early Childhood Development) behaviors among mothers of children under five years of age. Objective No.3: Improved quality of health facility services through strengthened Integrated Management of Childhood Illness and Maternal and Newborn Care capacity. Objective No.4: Strengthened institutional capacity of MTI and the Lira District Health Office to implement effective child survival activities. 2) To what extent was the project implemented as planned? What changes were made to the planned implementation, and why those changes were made? 3) How were gender considerations addressed through the project? Are there any specific gender-related outcomes? 4) Which elements of the project have been or are likely to be sustained or expanded? Project Background h Growth Monitoring - Apuce Health Center, Aromo Key Findings: • 88.2% of children 0-5 mo. were EBF during the last 24 hours • 83.9% of children 0-23 were attended by skilled personnel at birth • In 100% of HF, care￾takers could correctly describe how to administer all prescribed drugs • Reports from all community groups indicated that health facility staff are “friendly and do not harass any more” • 83.6% of mothers of children aged 0-23 mo. report they talk/ sing to the child while feeding FINAL EVALUATION EXECUTIVE SUMMARY Final Evaluation Report: Lira District Child Survival Project in Uganda December 2013 Final Evaluation ii The Northern region of Uganda is one of the poorest parts of the country, (DHS 2011 found 40% of the population in the lowest wealth quintile) was marginalized from development benefits, and has suffered the effects of roughly 20 years of conflict and displacement. MTI collaborated on this four-year entry grant with the Lira District Health Office (DHO) team for operation in Erute North Sub-District (including Lira, Ogur and Aromo sub-counties) to revitalize the capacity for maternal and child primary health care. The project area had four health facilities (HF) at the onset of the project, three with delivery services; an additional four were reopened in April 2012, all four basic facilities are without delivery capacity. The 2011 DHS found a national infant mortality rate (IMR) of 54 per 1,000 live births while in Northern region the rate was 66, a child mortality rate (U5MR) of 90 per 1,000 live births with a rate of 105 in the North and a maternal mortality ratio (MMR) of 438 per 100,000 live births for the mid-2004 to mid-2011 period, higher than the MMR of 418 in the 2006 DHS. Evaluation Design, Methods, and Limitations MTI used a two-pronged strategy to improve the health and nutrition status of women and children under age five: a) to promote behavior change and community mobilization to take appropriate responsibility for health; and b) through training and mentoring, to build the DHO capacity to provide sustainable, quality service delivery at the facility and community levels. The primary project partner was the Lira DHO and all project interventions supported the district five-year development plan. Secondary partners were the sub-county governments and their extensions. MTI’s supporting partner was HHI, an Oregon-based NGO specializing in early childhood development (ECD) for infants and very young children. The Final Evaluation (FE) consisted of a combination of quantitative and qualitative methods using participatory techniques. The preparation began with alerting all partners to the future evaluation and gaining their commitment to be involved in the process. Both a 30 cluster Knowledge, Practice and Coverage survey (KPC) and a Rapid Health Facility Assessment (RHFA) were conducted and preliminary data reports were provided to the evaluation team. The consultant, Director of HHI, HQ Senior Advisor, Maternal and Child Health, and the Child Survival Project (CSP) Manager prepared qualitative interview guides and the evaluation process in accordance with USAID’s Evaluation Guidelines. A one-day orientation and training was conducted for the evaluation team to review focus group interview techniques and guides. Focus group discussions were held in selected communities in each of the sub-counties to gather qualitative information from representatives from each target population group. Those interviewed were: groups of mothers and, separately, fathers, VHTs and Mother Leader Groups (MLGs), HF staff, HUMCs, Parish Development Committees (PDC), and Sub-county chiefs. CSP staff participated in interviews in geographic areas where they were not working. Qualitative data collection followed the interview guides designed to triangulate results of the RHFA and KPC as well as to assess the effectiveness of project processes and ascertain projections concerning project sustainability. The external consultant led evaluation team members through a participatory analysis of the findings from focus group interviews, the KPC and RHFA. Because four new HF were opened in April 2012 there was little time for improvements to be made; to report on RHFA data in a meaningful way, only data from the original four clinics is included in this narrative report. Lira and Ogur communities were included in the ECD qualitative evaluation however did not receive services the final 18 months. Because the remaining project was implemented in all communities, non-ECD intervention communities were chosen for focus groups limiting the qualitative ECD information gained in Lira and Ogur. Findings and Conclusions Question 1: To what extent did the project accomplish the objectives stated in the DIP? Both the KPC and the RHFA showed improvements, many statistically significant, in all indicators from baseline and those improvements were born out by focus group discussions with all interviewed groups. Final Evaluation Report: Lira District Child Survival Project in Uganda December 2013 Final Evaluation iii Focus groups of parents, various community volunteers and health providers all reported project promoted changes in behaviors in households (HH) they thought impacted the health of the community. Finding/Behavior Change KPC RHFA Increase in immunization completion by one year of age Baseline (BL) 15.5% to 56% at final Exclusive Breastfeeding BL 73% to 88.2% at final Women are attending ANC for 4 visits, receiving TT, and delivering at the health facility ANC: BL 35% to 58%; TT: 76% to 86%; Deliver at HF: 35% to 84% Increased availability and use of latrines with hand washing resources BL 54% to 88% at final Mothers talking or singing to their child while feeding BL 23% to 64% at final Caretakers able to correctly describe how to administer all prescribed drugs BL 0%,100 at final Question 2: To what extent was the project implemented as planned? MTI implemented a solid child survival (CS) project essentially as planned. Question 3: How were gender considerations addressed through the project? MLGs were formed and trained to deliver health education (HE) to HHs as 75% of VHTs were men, and the ECD intervention actively included men in HE regarding options for interacting with children and child care. Question 4: Which elements of the project have been or are likely to be sustained or expanded? See conclusions. Conclusions: • MTI implemented a solid CS project with many statistically significant results both at the community and health facility levels. • District and Local Government are assuming responsibility for continuing interventions through government mandated, project strengthened, structures, health personnel and community volunteers. • Positive relationships were developed between the HFs and communities they serve, reducing animosity and increasing trust. • Strong community volunteer cadre was developed, trained and supported; support continues through government mandated structures and health personnel. MLGs and peer educators (PE) were added to existing volunteer cadre increasing HH coverage. • ECD intervention also had statistically significant results as well as enthusiastic community and HF staff acceptance with reported increases in male involvement in child care and decreases in family violence. More research would be required to determine the extent of the ECD effect on household violence. • Project management continued to evolve and improve over the life of the project. Recommendations: • Implement projects in close collaboration with governmental systems, especially in decentralized countries. • DHO and Sub counties continue to support HF staff, HUMCs, PDCs and community volunteer cadres. • Formally integrate MLGs and PEs into DHOs community volunteer cadre and replicate the model in other areas of the District. • MTI replicate HH volunteers in any new community behavior change project. • MTI include strategies for improving HF/community working relationships in future health project designs. • Collaborate with HHI to design an effective strategy for ECD implementation at scale. • MTI Africa Regional Team continue monitoring OCA progress and support MTI Uganda staff. The Lira District Child Survival Project in Uganda in Erute North Sub-District, Uganda is supported by the American people through the United States Agency for International Development (USAID) through its Child Survival and Health Grants Program. The Lira District Child Survival Project in Uganda is managed by Medical Teams International under Cooperative Agreement No. GHS-A-00-09-00012. The views expressed in this material do not necessarily reflect the views of USAID or the United States Government. For more information about Lira District Child Survival Project in Uganda, visit: www.medicalteams.org Final Evaluation Report: Lira District Child Survival Project in Uganda December 2013 Final Evaluation iv EVALUATION PURPOSE AND EVALUATION QUESTIONS EVALUATION PURPOSE The Final Evaluation (FE) provides an opportunity for all project stakeholders to take stock of accomplishments and lessons learned to date and to listen to beneficiaries at all levels. The FE Report will be used by the following audiences as a source of evidence to help inform decisions about future program designs and policies: • The Lira District Health Office (DHO), health facility (HF) staff, and Health Unit Management Committees (HUMC) • Uganda Ministry of Health • Village Health Teams (VHT) and Mother Leader Group (MLG) volunteers • Hands to Hearts International (HHI) • The USAID CSHGP and the Uganda USAID Mission • Medical Teams International (MTI) HQ and field staff The evaluation is the FE for this four-year entry grant child survival project. There will be no follow-on programming implemented by MTI, therefore the results of the FE will be used by the DHO, the Sub￾County leadership, HF staff and HUMCs, and village leadership and volunteers to develop strategies for continuing and/or adjusting the Child Survival Project (CSP) programming as appropriate. In addition: • MTI is considering integrating lessons learned into their other programming in Uganda, especially in the Northern region; • HHI will integrate lessons learned into their other world-wide programming; • USAID mission may consider replicating successful program approaches in other programs in Uganda. The evaluator was hired with project funds with specific approval from USAID. In addition, MTI worked with the Child Survival and Health Grants Program (CSHGP) to develop the Scope of Work (SOW), which received final approval from USAID. The draft and final reports are being submitted to CSHGP and MTI simultaneously. Because the evaluator is residing outside of the US, the final report will be submitted simultaneously electronically to both CSHGP and to MTI. MTI will complete publication of the document and submit the hard and CD copies to the various USAID agencies as described in the guidelines. EVALUATION QUESTIONS 1) To what extent did the project accomplish objectives stated in the DIP? Objective No.1: Communities assume responsibility for their own health through strengthened community capacity Objective No.2: Improved health (Community-based Integrated Management of Childhood Illness) and child care (Early Childhood Development) behaviors among mothers of children under five years of age. Objective No.3: Improved quality of health facility services through strengthened Integrated Management of Childhood Illness and Maternal and Newborn Care capacity. Objective No.4: Strengthened institutional capacity of MTI and the Lira District Health Office to implement effective and efficient child survival activities. Final Evaluation Report: Lira District Child Survival Project in Uganda December 2013 Final Evaluation 1 2) To what extent was the project implemented as planned? What changes were made to the planned implementation, and why those changes were made? 3) How were gender considerations addressed through the project? Are there any specific gender￾related outcomes? 4) Which elements of the project have been or are likely to be sustained or expanded? PROJECT BACKGROUND RESULTS FRAMEWORK GOAL Reduce child morbidity and mortality in Uganda. STRATEGIC OBJECTIVE Sustainable improvements in preventive maternal and child health behaviors and use of strengthened health services in Erute North Sub-District of Lira District. RESULTS / OBJECTIVES Communities assume responsibility for their own health through strengthened community capacity (Village Health Teams, Parish Development Councils, and Health Unit Management Committees). Improved health (C–IMCI) and child care (Early Childhood Development) behaviors among mothers of children under 5 years of age. Improved quality of Health Center services through strengthened capacity for Integrated Management of Childhood Illness and Maternal Newborn Care. Strengthened institutional capacity of Medical Teams International and the District Health Office to implement effective and efficient child survival activities. KEY ACTIVITIES Strengthen capacity of Village Health Teams in C-IMCI using a social and behavior change approach. Establish structured referral system between VHTs and local health facilities. Facilitate collaboration between VHTs and PDCs and HUMCs to benefit MCH. Village Health Teams promote social and behavior change through existing channels (women’s groups, church groups, community meetings) and through home visits. Support IMCI training for health facility staff. On-the-job mentoring of health facility staff. Joint supportive supervision with DHO and partners. HQ support for MTI Uganda capacity through structured plan based on Organizational Capacity Assessment and technical assistance in social and behavior change and monitoring and evaluation. MTI/DHO joint activities for planning, training, implementation, and evaluation. Final Evaluation Report: Lira District Child Survival Project in Uganda December 2013 Final Evaluation 2 HISTORY AND DEMOGRAPHICS Context: The Northern region of Uganda is one of the poorest parts of the country, (DHS 2011 found 40% of the population in the lowest wealth quintile) was marginalized from development benefits, and has suffered the effects of roughly 20 years of conflict and displacement brought about by the atrocities of the Lord’s Resistance Army. By the time of the onset of the project, the population was returning to their homes and the internally displaced persons camps closed in December 2010. The project was implemented in the three sub-counties of Aromo, Ogur, and Lira. Of the three, Aromo was the most affected by the conflict with a larger proportion of families displaced and returning to their homes through the first year of the project; its infrastructure was also destroyed to a greater degree. The prolonged conflict resulted in numerous orphans throughout the three sub-counties, many of whom have been absorbed into households, further stretching existing resources. The three sub-counties continued to progress toward peace and stability through the life of the project and no further episodes of conflict occurred. The 2011 DHS found a national infant mortality rate (IMR) of 54 per 1,000 live births while in Northern region the rate was 66, a child mortality rate (U5MR) of 90 per 1,000 live births with a rate of 105 in the North and a maternal mortality ratio (MMR) of 438 per 100,000 live births for the mid-2004 to mid￾2011 period. This is higher than the MMR of 418 in the 2006 DHS. MTI collaborated with the Lira DHO team for operation in Erute North Sub-District (Lira, Ogur and Aromo sub-counties) to revitalize the capacity for maternal, newborn and child primary health care. MTI’s Development Hypothesis and beneficiaries: VHTs were trained in integrated community case management (iCCM), however are not providing CCM services, as the drug supply in the district is insufficient for distribution at the community level. CSP directly benefited 25,498 children under age five (U5) and 25,498 women of reproductive age for a total of 50,996 direct beneficiaries, including: 5,348 infants 0-11 months, 5,400 children 12-23 months, and 14,750 children 24-59 months. The total population of the project area is 124,379. PROJECT DESIGN AND PARTNERS Project Goal and Strategy: The project overall goal was to reduce child morbidity and mortality in Erute North Sub-district of Lira District in northern Uganda. MTI used a two-pronged strategy to improve the health and nutrition status of women and children under age five: a) to promote behavior change and community mobilization to take appropriate responsibility for health; and b) through training and mentoring, to build the DHO capacity to provide sustainable, quality service delivery at the facility and community levels. Key Partners – Lira District Health Office and Government Structures: The primary project partner was the Lira DHO of the Ministry of Health and the HUMCs and clinics in the sub-counties the project served. All project interventions supported the District’s five-year development plan. Secondary, but very closely related, partners were the sub-county governments and their extensions, the Parish Development Committees (PDC) and Local Council Is (LC1). All programming was in collaboration with the DHO and the Sub-county chiefs of the three sub-counties. If we involve and educate community resource people/structures (VHTs, mothers groups, PDCs and HUMCs) to build their capacity to improve health at the household and community levels through promotion of key family health and child development practices, referrals, and community case management, as a result, key family health practices and referrals will improve. Final Evaluation Report: Lira District Child Survival Project in Uganda December 2013 Final Evaluation 3 HUMCs were reestablished according to government regulations at each of the four new health facilities and strengthened at the four original facilities. Because the Ugandan governmental system is decentralized, structures with authority were in place for the project to work with and through. Two VHTs, chosen by the community, were in place as the community link to the HF although their training was minimal; LCIs were politically elected in each village and mandated to oversee its wellbeing; PDCs, the next level up, include LC1s from each community, a VHT representative, and are responsible for developing plans incorporating community needs. These plans are sent on up to the Sub-county level. The Sub-county is responsible for all activities within their area and liaises between their HFs and the DHO to address health issues. Each HF has a HUMC, made up of “people of integrity” who are responsible for ongoing operations of the HF; the HF in-charges are appointed as secretaries to the committee. The DHO is primarily responsible for clinical issues. In the project areas, the PDCs and HUMCs worked together to address HF management issues (counting drugs on arrival, protecting the HF property, staff housing, advocating for improvements/ expansions of the physical plant and/or staff and services.) Supporting Partner – Hands to Hearts International (HHI): HHI, an Oregon-based NGO specializing in early childhood development (ECD) for infants and very young children, was contracted to provide a training of trainers for MTI and HF staff on ECD. HHI first adapted their model in another Ugandan population, using the resulting modifications and trainers to conduct the training in Lira. They also provided some monitoring/mentoring for the project intervention. The training consisted of eight modules, delivered approximately weekly to parents and caregivers by two community identified Peer Educators (PE), a man and a woman. The modules include information on: the role of parenting; physical, cognitive, language, and social and emotional development; baby cues and massage; and health, hygiene, nutrition and sanitation. The intervention was originally implemented in selected communities across the three sub-counties however, in response to the mid-term evaluation (MTE); the implementation strategy was altered to focus on all of the communities in Aromo sub-county with no further formal work being done in Ogur and Lira. Health Facilities in project area: By the end of the project there were eight HFs in the target area. One Level IV HF in Ogur functioning as a sub-county referral center for the three sub-counties, in addition Ogur has two HF IIs providing basic services, Lira has two HF IIIs, one providing delivery services and Aromo has one HF III with delivery services as well as two HF IIs. Since the MTE four HF IIs that had been closed during the conflict were reopened with staff housing, two in Aromo, and two in Ogur. Level of Effort (LOE:) Even though the project was entry level, MTI chose to focus on five technical areas in addition to ECD. They planned to devote 10% LOE to strengthen Immunization, 20% for Control of Diarrheal Disease, 25% for Pneumonia Case Management, 20% for Infant and Young Child Feeding (IYCF), and 25% for Maternal and Newborn Care (MNC). No LOE was devoted to the ECD activities funded out of the match. In actuality at midterm the project was devoting 55% of their effort to immunizations, with not enough attention to IYCF and MNC. The MTE recommended the project scale back on immunization activities and confine them to the original 10% LOE. Training: At the community level VHTs, 75% of whom are men, were trained in technical areas of the project intervention, iCCM and some in ECD. Mother Leader Groups (MLG) were formed with members chosen by the communities and expanded from 300 to 1428 in response to the MTE recommendation to improve household (HH) coverage. A group of 96 were trained as trainers and all members were trained on key maternal and newborn care (MNC) and nutrition issues. Both VHTs and MLG members received information, education, and communication (IEC) materials. The Training of Trainer Mother Leaders (ML), as the 96 were called, provided first line mentoring for MLGs in their areas. PEs, and Final Evaluation Report: Lira District Child Survival Project in Uganda December 2013 Final Evaluation 4 selected VHTs received ECD training. PEs and MLGs generally conducted HH and small group health education (HE) while VHTs generally provided back up support to both groups, were responsible for community mobilization and were the primary liaison between the community and the clinic on clinical matters. A structured referral system was developed and both VHTs and MLGs referred patients to the clinic. LC1s were not specifically trained, however were welcome and eager to attend VHT, MLG, and PE training and refreshers. Health providers (HP) in the original four HFs and in the four which opened in April 2012, received IMCI training and refresher training with guidelines and wall charts for the clinic. They also received on￾the-job mentoring and joint supportive supervision with the DHO, and training in ECD, MNC, supportive supervision and the referral system. HUMCs and PDCs were not directly trained in technical issues, however, were encouraged to join the VHT, MLG and PE training and refresher training. The PDCs were however trained in developing emergency (ER) transport systems in their communities. Quarterly meetings were also held for VHTs and MLGs to problem solve and upgrade their skills. MLs met bi-weekly with MLG members to identify and resolve any issues. Health Providers, HUMCs and PDCs were invited to the quarterly meetings to ensure all stakeholders heard consistent messages and to help resolve problems that arose between the community and the HF. Gender: Gender norms in the project area often preclude women from deciding on their own when to seek health services or participate in outreach or HE activities. Men often still make decisions about how women use their time and access family resources. This was addressed in a number of ways. Because 75% of the VHTs are men and the turnover was very low, a cadre of MLG members was added to the community volunteer resources to ensure young mothers had a comfortable resource for HE. VHT contacts provided men with more information on health concerns, especially on maternal health and danger signs, encouraging them to be more involved in the health of their families and attend antenatal care (ANC) with their wives. The village PE team consisted of a man and a woman to ensure both men and women would be reached with the ECD messages. Both the PEs and the MLGs addressed domestic violence in their health messages as well as male involvement in health issues in the family. The focus groups indicated both that domestic violence had decreased and men were attending more ANC visits with their wives. Technical Package: The technical package and activities were clear and, after modifying the implementation in accordance with the MTE recommendations, were well implemented and documented. ADDITIONAL PARTNERSHIPS Collaborations: The Malaria Communities Program, also implemented by MTI, served an overlapping geographic area and provided opportunities to collaborate on project surveys, community feedback sessions, staff training and, where the actual projects overlapped, on project activities. A Program Associate from MCHIP Washington provided CSP staff training on sustainability. UNICEF supports the Child Health Days as did the CSP. USAID Uganda Health Objectives: The project focused at a community and sub-county level on each of USAID Uganda’s health objectives. HF staff were trained in MNC; DHO and HF staff were trained in supportive supervision; and the new health facilities were provided with basic equipment. The primary project focus was at the community level where three cadres of volunteers were trained in basic health messages including the need for ANC visits and HF deliveries. Strong links were forged between the HFs and the communities with communication between the two groups improving as described by each group during focus group Final Evaluation Report: Lira District Child Survival Project in Uganda December 2013 Final Evaluation 5 discussions. PDCs and HUMCs honed their advocacy skills resulting in four HFs being refurbished, staffed and equipped with the basics. These groups are currently planning their next advocacy efforts for the health needs of their communities. MTI Relationship with USAID Uganda: MTI kept the USAID mission informed about activities and constraints throughout the project. The DIP and annual reports were shared with the Mission Health Team. Debriefings were held during DIP development and after the midterm and final evaluations. The Lira team also attended coordination meetings at the USAID Gulu office. Annex IV, with additional information regarding achievements, additions and changes is found in the Annex section of this submission. EVALUATION METHODS AND LIMITATIONS The Final Evaluation consisted of a combination of quantitative and qualitative methods using participatory techniques. The preparation began with alerting all partners to the future evaluation and gaining their commitment to be involved in the process. A 30 cluster Knowledge, Practice and Coverage survey (KPC) and a Rapid Health Facility Assessment (RHFA) was conducted in mid August, two weeks prior to the qualitative evaluation, guided by MTIs Senior Advisor, Monitoring and Evaluation. A 30-cluster stratified sampling design was utilized to select the mothers of children aged 0-23 months. A total of thirty clusters were sampled, with twelve households from each cluster being selected for interviews. With eight functioning HFs in the project area no sample size selection was necessary and all eight HFs were surveyed for the RHFA. Preliminary data reports were provided to the evaluation team for use in finalizing the interview guides. (Final KPC and RHFA reports are included in Annexes VI and X A) The consultant, Director of Hands to Hearts International, HQ Senior Advisor, Maternal and Child Health, and the CSP Project Manager prepared qualitative interview guides and planned the evaluation process in accordance with USAID’s Evaluation Guidelines. (see Annex XI for a list of the documents reviewed) A one-day orientation and training was conducted for evaluation team members to review focus group interview techniques and the interview guides, and make necessary revisions to the evaluation logistics. Focus group discussions were held in each of the sub-counties with representatives from each target population group to gather qualitative information. Comparatively low performing communities, as well as high performing communities were identified by project staff prior to the evaluation. Groups of mothers and, separately, groups of fathers were interviewed in those communities using identical questionnaires. In addition, groups of VHTs within a Parish were interviewed as were MLGs. HF staff, HUMCs, PDCs and Sub-county chiefs were interviewed to gain a non-community perspective on the community interventions and to gain a health system perspective on the project. To limit bias, project staff participated in interviews in geographic areas where they were not working. A list of all those interviewed are included in Annex XI. Qualitative data collection followed the interview guides that were designed to triangulate the results of the RHFA and the KPC as well as assess the effectiveness of project processes and ascertain projections concerning project sustainability. The complete packet of interview guides is included in Annex X B. Final Evaluation Report: Lira District Child Survival Project in Uganda December 2013 Final Evaluation 6 The External Consultant led evaluation team members through a participatory analysis of the findings from focus group interviews and of the KPC and RHFA findings. Results were triangulated using the following tools: • Quantitative 30 cluster KPC survey of mothers of children age 0-23 months, • Quantitative Rapid-Health Facility Services Assessment, • Comparison of data with available national and regional data from 2011 DHS, • Triangulation of perspectives of qualitative interviews with community volunteers, project beneficiaries, community leaders, health facility staff, sub-county leadership and district health staff. Evaluation Participants were: • External consultant, • MTI Uganda project staff, • Lira District Health Office and health facility staff, • Sub-County and Community leaders and volunteers, • MTI Senior Advisor, Monitoring and Evaluation and Senior Advisor, Maternal and Child Health. Finalization of Key Findings, Recommendations, and Action Planning • Debriefing on key findings and recommendations with representatives from community volunteers, health facility staff, sub-county leadership and district health staff. (see Annex XVI C for FE Results Meeting Agenda) • Debriefing with USAID Uganda Mission (see Annex XVII for debrief PowerPoint) • Debriefing with MTI Uganda and MTI Africa Regional staff Evaluation Limitations Because four new HF were opened in April 2012 just after the MTE, it left very little time for improvements to be made. This meant that in order to report on RHFA data in a meaningful way, only data from the original four clinics is included in the narrative of this report. The entire RHFA report is included in Annex X A. An issue for the ECD portion of the qualitative evaluation was that some communities from the comparison areas in Lira and Ogur were included in the ECD interviews, resulting in some of the communities chosen for interviews being ECD non-intervention communities. This limited the qualitative information available from intervention communities two years after the direct project support concluded. FINDINGS, CONCLUSIONS, AND RECOMMENDATIONS FINDINGS EVALUATION QUESTION #1: To what extent did the project accomplish objectives stated in the DIP? Summary Table of Inputs, Activities, and Outputs That Contributed to Key Outcomes Final Evaluation Report: Lira District Child Survival Project in Uganda December 2013 Final Evaluation 7 Project Objective No. 1: Communities assume responsibility for their own health through strengthened community capacity Project Inputs Activities Outputs Outcome Five year Lira District Development Plan Trainers MTI staff mentors Referral forms for VHTs and MLGs Strengthen capacity of VHTs and MLGs in C-IMCI with a social and behavior change (SBC) approach Establish a structured referral system for VHTs & MLGs to local HFs by providing training to HF staff, VHTs and MLGs and developing forms. Trained HF staff in supportive supervision for VHTs Quarterly Refresher meetings for VHTs and MLGs with help of HF staff, PDCs and HUMCs Trained PDCs in establishing ER transport systems in their parishes HF staff supervising VHTs in conjunction with PDCs and HUMCs VHTs and MLGs making appropriate referrals to HFs Problem solving during quarterly meetings with volunteers and HF staff concerning community and clinic issues 75% of the 560 VHTs received a supervisory visit during the last 6 months; 0% at baseline (BL). The majority of PDCs are initiating monthly meetings with VHTs, MLGs and HF staff in their parishes without MTI involvement to discuss clinic and referral issues 29 emergency transport systems established and are being used to transport community people to the HF; BL 0%. 100% of HFs received at least one referral from VHTs in the last month; BL 75%. (for complete list of indicators, see M&E Matrix, Annex XV1 B) Project Objective No. 2: Improved health (Community-based Integrated Management of Childhood Illness) and child care (Early Childhood Development) behaviors among mothers of children under five years of age. Project Inputs Activities Outputs Outcomes C-IMCI Baseline and final KPC Trainers MTI staff mentors SBC strategy iCCM curriculum, handouts and VHT manuals IEC and SBC materials Monitoring and Together with HF staff, train VHTs in iCCM Identify and train MLG members in IYCF and MNC messages Strengthen capacity of VHTs and MLGs to use SBC approach through existing channels Supportive supervision and mentoring for community volunteers Quarterly refresher 560 VHTs trained and supported in iCCM 1428 MLG members trained and supported in IYCF and MNC VHTs and MLGs delivering health messages and promoting SBC in communities (home visits, community meetings, church groups, immunization 88.2% of children 0-5 months were exclusively breastfed during the last 24 hours; BL 73.6%. DHS: for Uganda, 41% of children are EBF. 79.8% of children 0-23 months are not under weight; BL 72.3%. DHS: 87.7% are not underweight in Northern Uganda. 56.0% of children aged 12- 23 months were fully vaccinated (BCG, DPT3, Final Evaluation Report: Lira District Child Survival Project in Uganda December 2013 Final Evaluation 8 evaluation tools Non-monetary incentives for VHTs and MLGs (t-shirts, name tags, umbrellas) meetings for VHTs and MLs on technical issues provided by HF staff; PDCs and HUMCs included in meetings Organized drama group largely of MLG members, in project area sessions) VHTs case finding and referring patients to clinics Drama performances throughout the 3 sub￾counties with MNC and child health messages OPV3 and measles) by 12 months of age, card verified; BL 15.5%. DHS: 49% fully immunized by 23 months in Northern Uganda. 83.9% of children aged 0-23 months were attended by skilled personnel at birth; BL 35.3%. DHS: 51.9% attended by skilled personnel in Northern Uganda. 69% of children with chest￾related cough were treated with an antibiotic; BL 34.7%. DHS: 43.6% received antibiotic in Northern Uganda 88% of mothers report washing hands with soap at least 2 times in 24 hours; BL 54%. ECD HHI trainers HHI ECD Training Materials MTI staff mentors Monitoring and evaluation tools Non-monetary incentives for PEs (t-shirts, nametags) Train PEs, VHTs and HF staff in ECD messages and techniques Supportive supervision and mentoring for ECD PEs 452 PEs trained and supported in ECD messaging 19 HF staff trained as trainers in ECD 76.3% of mothers of children aged 0-23 months provide cognitive stimulation to their child in the form of games; BL 38%. 83.6% of mothers of children aged 0-23 months report they talk or sing to the child while feeding the child; BL 57.7%. Project Objective No. 3: Improved quality of health facility services through strengthened Integrated Management of Childhood Illnesses and Maternal and Newborn Care capacity. Project Inputs Activities Outputs Outcome Baseline and Final Rapid Health Facility Assessment Ugandan IMCI materials Ministry of Health (MoH) IMCI trainers Home-based Support IMCI and MNC training for HF staff On-the-Job mentoring of HF staff Involved HF staff, HUMCs and PDCs in meetings and training for community volunteers Encouraged HUMCs and IMCI training for 34 health workers in HFs MNC training for 46 health workers at HF level Refresher and supportive supervision training for 40 health workers In 100% of HFs, treatment was routinely appropriate to the diagnosis; BL 25%. In 100% of HFs, caretakers could correctly describe how to administer all prescribed drugs; BL 0%. All of the health workers except for one, completed all assessment tasks at least Final Evaluation Report: Lira District Child Survival Project in Uganda December 2013 Final Evaluation 9 maternal and newborn care materials IMCI Job Aids (wall charts, guidelines) PDCs to perform their mandated activities Conduct Joint supportive supervision visits with DHO and partners HUMCs and PDCs count drugs with HF staff upon delivery to HF; communicate results to communities HUMC and PDCs spot check for appropriate drug use & instructions at HH level 80% of the time; BL 21%. Reduction in complaints from community that HF staff are stealing drugs Reports from all community groups interviewed indicated that HF staff are “friendly and do not harass anymore.” Project Objective No. 4: Strengthened institutional capacity of MTI and the Lira District Health Office to implement effective and efficient child survival activities. Project inputs Activities Outputs Outcome Supportive Supervision check lists from Ministry of Health Organizational Capacity Assessment (OCA) Action plans for 6 MTI Uganda priority areas MTI/DHO joint activities for planning, training, implementation and evaluation HQ technical assistance in social and behavior change, and monitoring and evaluation HQ and Africa Regional team support for MTI Uganda capacity through structured plan based on OCA Joint supervision for HFs and VHTs together with DHO and health staff Support HF staff to arrange and facilitate coordination meetings with VHTs and MLGs Africa Regional team facilitated OCA update workshop in Nov 2012 to update action plans for priority areas All 4 of the original HFs and 3 of the new ones received joint DHO/MTI supervision visits in the previous 3 months; BL 1 of original 4. Progress has been made in implementing action plans for all six priority areas with strategic planning, communication & decision making being completed. CONTRIBUTIONS TO KEY OUTCOMES EVALUATION QUESTION #1: To what extent did the project accomplish objectives stated in the DIP? Objective 1: Communities assume responsibility for their own health through strengthened community capacity Training, support, and supervision for VHTs and MLGs: As documented in the Training Matrix (Annex VI), both VHTs and MLGs received training and refresher training in IYCF and MNC and the VHTs received iCCM training in addition to other topics throughout the project. CSP staff closely monitored the quality of home visits and community education sessions; supervision/ mentoring systems were established and monitored. (see Annex XIX E for project documents) VHTs supported MLGs in the community technically and helped them mobilize other necessary resources e.g. LC1 to encourage a family to comply with community by-laws or ER transport to the HF. VHTs were supervised technically by HF staff in the community and in the HF when a patient was referred. HF staff participated in the quarterly meetings organized by CSP, usually providing the technical refresher training for both the VHTs and the MLGs. LC1s, PDCs, HUMCs and HF staff all spoke of the critical work of the community volunteers and of the positive behavior changes being made Final Evaluation Report: Lira District Child Survival Project in Uganda December 2013 Final Evaluation 10 in the communities as a result of their work. Tangible support for about half of the VHTs came as bicycles from the government; these were given to the women VHTs and to some of the men. Community volunteers’ perception of project benefits to them and impact of their work: Both VHTs and MLGs described significant changes in behaviors of community members they attribute to their work. They see the increase in their knowledge and skills regarding Maternal and Child health issues as a direct result of the project activities. The changes they describe are confirmed by the KPC results, and by interviews with beneficiaries and HF staff. A few of their perceptions are: Change Observations and Triangulation Decrease to near extinction the extraction of “false teeth.” “False teeth” are actually a baby or small child’s permanent teeth made visible by dehydration and traditionally thought to be the cause of illness VHTs, MLGs and HF staff report seeing cases of extraction only rarely now Dramatic increase in immunization completion by 1 year KPC documented an increase in immunization completion by 12 months from 15.5 to 56.0% “No more polio” Polio was prevalent in the area only 10 years ago and now few cases are reported More women are attending ANC for 4 visits, receiving TT, men are going with them to the HF and women are delivering at the HF KPC documented 57.8% of women attend 4 ANC visits, up from 35.3%, 89.4% receive TT and 83.9% are delivering at the HF up from 35.3% at baseline. Increase in understanding “balanced diet” for adults and children The KPC indicates that more children from 6 months receive a minimum appropriate feeding now at 48%, up from 23.1%. Caregivers are providing proper care for the cord of the newborn Previously there were frequent cord infections, now HF staff report they are uncommon Increased availability and use of latrines with hand washing resources Local Councils are passing by-laws requiring improved health practices especially digging latrines. The KPC showed good hand-washing practices increased from 54% to 87.8%. Now mothers give colostrum to their babies and exclusively breast feed (EBF) At BL, colostrum was seen as bad and dirty so was discarded. Final KPC results for EBF were 88.2%; Immediate BF was 78.9% and 87.2% received no pre-lacteal feeds. Massage for babies not walking on time It is unclear why so many babies are not walking on time – perhaps because they are carried on the mother’s back for extended periods especially during farming seasons. Baby massage was taught in the ECD classes and parents report it eliminates this problem. Use of the HF has increased, parents are taking children in for care; women are delivering there. Community volunteers are more comfortable with the HF and think the community is also. They describe an improved relationship between the community and HF staff. Final Evaluation Report: Lira District Child Survival Project in Uganda December 2013 Final Evaluation 11 Community and HF staff perception of impact of volunteers: Community people described the volunteers as their source of quality health information, and as the first person they go to if there is a health problem. HF staff were pleased with the referrals they received; saw more families seeking timely care for children; felt there were positive changes occurring in family health practices due to the work of volunteers, decreasing the incidence of diarrhea and malnutrition. Sub-county chiefs described the volunteers as the first line health services in their areas. Sustainability: The VHTs and MLGs both see community norms are changing and feel the above changes will continue. They are saying: “we can’t go back to the old behaviors”; “we will keep making home visits and talk with families.” The indicator concerning ER transport was achieved with all 29 PDCs establishing transport systems in their parishes with contacts to help find drivers during emergencies, directing them to the best facilities to handle the emergency and mobilizing and encouraging the community to save money for use during emergencies. However, those systems generally involve the patient sitting on the back of a motorcycle or bicycle, adequate most of the time. If the patient is acutely ill or has complications of labor, it is a less effective method of transport. Because there are very limited community resources, developing a way to transport these patients probably will require financial assistance from the sub￾county or DHO level. Community volunteers are concerned that referrals will be difficult as the forms were developed by MTI. In addition they know there are few transportation resources at the sub-county level and almost none at the lower level HFs perhaps impacting supervision visits and transport of patients. Objective No.2: Improved health (Community-based Integrated Management of Childhood Illness) and child care (Early Childhood Development) behaviors among mothers of children under five years of age. Improvements in KPC data and strategies: All indicators showed improvement. (for complete list of indicators see M&E Matrix, Annex XIX B) The results were discussed during the FE Results Meeting. DHO and HF staff, and community volunteers are aware however that there is more work to do, as several indicators are lower than targeted. A variety of strategies seemed to contribute to the improvements: 1. Project interventions supported the Five-Year District Development Plan, 2. Improvement in HH coverage through development of the mothers group cadre of community volunteers and the MLs trainers/mentors, 3. Well-trained and supervised community volunteers with ongoing refresher training and review meetings, 4. Continuous mentoring by CSP staff, of MLs and VHTs and HF staff; collaboration with community leaders for supportive supervision and mentoring, 5. Strong partnerships with government structures and strong links between HFs and communities, 6. Motivational incentives and job aids: IEC materials, nametags, t/shirts, umbrellas and torches and acknowledgement from the HF staff and community leaders. Key practices with lesser improvement on KPC and lessons learned: Again, all indicators improved from baseline and all but one improved from mid-term. That indicator was: Children with chest–related cough taken to HP went from 86.1% at mid-term to 74.2% at final. The incidence of acute respiratory illness was down therefore the sample sizes were smaller. Four, although improving from mid-term, did not reach the target set by the project. Final Evaluation Report: Lira District Child Survival Project in Uganda December 2013 Final Evaluation 12 Indicator BL MTE FE Target Considerations/Lessons Learned EBF 73.6% 67.7% 88.2% 95% Mothers work in the fields and caregivers may begin early weaning to soothe the child. Breast milk expression is not a common practice. It was covered in the IYCF cards but without emphasis or details. Post￾partum Visit 16.3% 22.2% 40.8% 50% The increase in attendance at HFs increases the waiting times and may discourage mothers from going. VHTs are able to provide post-partum checks for uncomplicated deliveries and refer women with danger signs – refresher may be needed. More women VHTs would be an advantage. Child with pneumonia danger signs to HF 57.8% 86.1% 74.2% 80% Stock outs of medication means mothers often go directly to the drug shop for meds when children are sick. Zinc for diarrhea 0.9% 2.6% 17.6% 30% The project requested this indicator be dropped at mid-term, as there were no consistent supplies. The results were significantly increased at final. HF staff report they received zinc in one drug shipment, used it for diarrheal cases, saw improvements and now recommend mothers buy it from the drug shop, and some do. Strategies for Lira DHO, and Lira, Ogur and Aromo Sub-County Chiefs for continuing progress: 1. Support the technical team to do continuous support supervision and mentoring of HF staff and community volunteers; 2. Ensure HF staff are able to stay current with the latest health interventions and refresher education; 3. Support/encourage HUMCs and PDCs to continue supporting the HFs and communities including continuing to develop methods of ER transport for acutely ill people; 4. Mobilize the resources of other partners, such as Northern Uganda–Health Integration to Enhance Services (NU-Heights), to focus on those geographic and programmatic areas needing additional support that are consistent with their plans; 5. Ensure VHTs, PEs and MLGs members are supported with attention, technical updates, refresher training, job aids, referral forms, modest in-kind incentives, and continued respect. Target refresher training especially to those indicators needing more improvement such as post-partum visits; 6. Integrate Mother’s Group and Peer Education volunteers into the community health structure. Because they are not officially part of the health system, they could easily be lost even though they were key to the community gains made by the project. 7. Consider linking village volunteers to other government support programs to provide some “payment” for their services to the community. ECD implementation and impact: The project originally planned to conduct operations research to measure the impact of ECD training on health indicators. Intervention and comparison areas were mapped with the plan to compare the results in implementation communities with comparison communities. At midterm, however, it was Final Evaluation Report: Lira District Child Survival Project in Uganda December 2013 Final Evaluation 13 discovered that PEs, health staff and parents in the intervention area were so enthusiastic about the ECD skills they learned that they shared them with caregivers in the comparison areas. This sharing of information made it impossible to compare the impact of ECD training between the two areas. To have a greater impact on a smaller area, formal ECD activities were targeted to Aroma sub-district for the last 18 months of the project. Aromo was selected for special attention as this sub-district was more affected by the conflict than either Lira or Ogur sub-districts, had the largest portion of displaced families returning at the beginning of project and had greater infrastructure damage. One hundred and fifty additional PEs were trained. During this phase, PEs delivered ECD lessons directly to caregivers in their homes. Because training was aimed at the HH, they were able to reach both parents and other caregivers. The widespread enthusiasm from Phase 1 undoubtedly contributed, however, to the remarkable FE results seen in non-intervention sub-counties in the KPC after two years of no project support in this population with a recent history of experiencing prolonged extreme violence. Indicator Lira Ogur Aromo Cognitive Stimulation: % of mothers of children 6-23 months who report playing a game like “where is your nose.” Project wide BL data: 38% 83.3% 72.1% 76.1% Linguistic Learning: % of mothers of children 6-23 months who told a story, sang a song or named objects. Project wide BL data: 22.7% 63.9% 63.0% 67.0% % of mothers of children 6-23 months who report talking or singing to their child while feeding them. Project wide BL data: 57.7% 83.5% 83.8% 83.5% The focus group results from VHTs and from parents and caregivers indicate some interesting findings in addition to the above project indicators. In addition to child growth and development messages, PEs reinforced health messages including: immunizations, breastfeeding, nutrition, hygiene, sanitation, and safety while learning to use the toilet. Change Description Relationship with the child • Parents are playing with the child, singing and dancing with them, teaching them things, teaching them how to talk, making them toys, spending more time with them; • “My child is not afraid of me anymore; he is happy to see me,” “I have permission to play with my child”; • Children no longer fear their parents; parents now love their children more; • Children learn faster, are more interested, understand instructions better; • “Discipline is talking and encouraging but not harassing like before”; • Children are attending school. Relationship between parents • Less violence in the home because have knowledge of social and emotional development; • “My wife likes me better”; wives love husbands more because they help with the children; • Men involved with family health care, going to clinic with pregnant wives, with mothers taking children in for care; • Parents feel accountable to neighbors. Health behaviors • Baby massage: helps babies walk on time, sleep, and not be so fussy; • Using nappies for babies, keeping them clean; • Toilet training easier; proper disposal of child feces. Final Evaluation Report: Lira District Child Survival Project in Uganda December 2013 Final Evaluation 14 Decreased family violence is an unexpected finding in this FE, and community members and volunteers attributed it to the ECD interventions. Conversations with parents revealed that the ECD program gave parents specific behaviors they could do to understand and correct a child’s behavior without violence; and these behaviors worked in a population emerging from 20 years of sustained violent conflict. Objective No.3: Improved quality of health facility services through strengthened Integrated Management of Childhood Illness and Maternal and Newborn Care capacity. Impact evident from RHFA data and strategies: A major CSP strategy was to support the HUMCs and PDCs in their efforts to refurbish and reopen the HFs closed during the conflict. All four of the new facilities are level IIs and therefore do not offer delivery services. They were reopened after the MTE, and each is working to achieve the level of quality care as outlined in the Ministry of Health Guidelines for Designation, Establishment and Upgrading of Health Units. New HFs were provided with adult and child weighing scales and weighing pants, exam tables and screens, blood pressure machines, stethoscopes, fetoscopes, thermometers and gas cylinders, IMCI wall charts and guidelines. CSP staff trained staff in IMCI and mentored them to assist them in providing quality MNC and IMCI care. The following chart is limited to the findings in the four original HFs to better capture the results of project interventions as the four new facilities have been in operation for a year and a half. Data on all eight HF can be found in the RHFA. (Annex X A) Indicator BL Index Value FE Index Value Strategy/Activity % HF maintaining up-to-date records of sick U5 children & HF has report in last 3 months and evidence of data use 25% 80% IMCI training, mentoring and supportive supervision for HF staff % HF maintaining up-to-date records of ANC and deliveries 25% 100% MNC training, mentoring and supportive supervision for HF staff % U5 patients for whom all 5 assessment tasks were done at least 80% of the time 21% 98% IMCI training, mentoring and supportive supervision for HF staff % HF: treatment for U5 children is routinely appropriate to diagnosis 25% 100% IMCI training, mentoring and supportive supervision for HF staff % HF: caretakers correctly describe how to administer all prescribed drugs 0% 100% IMCI training, mentoring and supportive supervision for HF staff % HF receiving external supervision at least once in last 3 months 25% 100% Supportive supervision training and mentoring for DHO and higher level HF staff HF staff perceived benefits of CSP: At each HF, staff first mentioned the community level assistance they are getting from VHTs and MLGs when asked about the benefits of CSP. Change Description Quality of Care • VHTs promptly referring children and mothers with danger signs into HF • Improved skills through training and mentoring - see above RHFA data • Equipment made work easier. • Records at HF are better - see above RHFA data • Improvement in immunization, IMCI and ANC services – see above RHFA data • Growth Monitoring • Quality of monthly meetings at HF Final Evaluation Report: Lira District Child Survival Project in Uganda December 2013 Final Evaluation 15 Role of HUMCs and PDCs with HFs: HUMCs were reestablished according to government regulations at each of the HFs. They are working with the PDCs, who represent the communities the HF serves. HF staff report they care for the physical plant, protecting it from intruders, supervising construction of improvements; advocate on behalf of the HF to the sub-county and DHO for equipment and supplies, additional staff, upgrades to the facility and staff housing, approval to drill bore holes; and they assist with staff management focusing on punctuality, attendance and efficiency of work. The HUMCs and PDCs also count the drugs with the HF staff as they come into the facility and report their status to the communities. They mobilize community members for EPI and other programs and generally keep the lines of communication open between communities and the HF. HUMCs and PDCs have clear plans for the upcoming year with strategies for how they will continue with their advocacy to improve both services e.g. immunization points, and resources e.g. bore holes. Community perception of HF services: Focus group discussions in communities with mothers, fathers, and community volunteers gave the following descriptions of the changes in HF services. Some were issues addressed by the RHFA and are consistent with the results. Change Description Quality of Care • Children tested before treatment – RHFA confirmed, finding 99% of patients were seen in a facility completing all 5 assessment tasks at least 80% of the time. • Fewer maternal deaths, fewer baby deaths Medication • Better drug control; LCI now participating in counting drugs • Right prescriptions with clear instructions – RHFA confirmed, found 100% of the time caregivers correctly described how to administer all prescribed drugs Health Education • Health talks at the HF before treatment • Good explanation of how to prepare ORS • Support birth plans • Explain growth monitoring Relationship with HF staff • HF staff honor referrals from VHTs, good coordination with VHTs • No more harassment, staff are friendly • Staff are encouraging, spend more time with parents • Staff are at the HF Services • Family Planning services available • Better Immunization coverage • Enough beds in the ward • Sometimes provide mama kits for delivery • Good hygiene and sanitation at HF – confirmed by RHFA which found that 100% of HF had essential infrastructure present and functioning Sustainability: HF staff, HUMCs and PDCs all thought the improvements in the HF would continue for two reasons 1) staff and community volunteers had been trained and mentored, and are using those skills in their work; and 2) the lines of communication are strong between the community and the HF. They think problems especially around transportation, and drug and equipment supply issues will probably continue. Both the HUMCs and PDCs had well defined strategies in place for advocating with the sub-county and DHO to accomplish their goals. Objective No.4: Strengthened institutional capacity of MTI and the Lira District Health Office to implement effective and efficient child survival activities. Impact of DHO/MTI joint supervision to HFs and communities: Final Evaluation Report: Lira District Child Survival Project in Uganda December 2013 Final Evaluation 16 The KPC and RHFA results both demonstrate the improvement in care provided and community level behaviors changed. Facility and community level providers attribute the improvements in their practice to the quality of supportive supervision they received from MTI staff and through the DHO. Supervision is decentralized so that although there is supervision from the DHOs office quarterly, mentoring and supportive supervision is also the responsibility of the higher level HFs for the lower level ones; helping to decrease the transportation challenges. There was an additional level of supervision encouraged by CSP. HUMCs and PCDs, often involving HF staff, traveled to communities to provide supervision to community volunteers. Occasionally the Sub￾county chief would also be involved. That level of attention and oversight for volunteers was described as essential for them to function well in their jobs. The HF staff and the community volunteers spoke of how important it is for someone to come to supervise their work and of how they hope this will continue now that CSP is finished. MTI Uganda progress on organizational assessment low-scoring areas: MTI originally used JSI developed materials to assess the capacity of MTI Uganda to provide support to the project. After the MTE, it was decided that a newly published Catholic Relief Services (CRS) Organizational Capacity Assessment (OCA) materials were more appropriate for MTI Uganda’s situation. (see Annex XIX A for the complete end of project OCA matrix) In the baseline OCA there were six areas prioritized for improvement; two of those, Communication and Decision-making were combined in the final OCA. The areas of concern are: Succession Planning, Staff Salary and Benefits Policy, Strategic Planning, Communication and Decision-making, and Monitoring and Evaluation. The MTI Africa Region team facilitated an OCA and planning process in November of 2012 and reported on the implementation results as of August 2013. MTI Uganda had made considerable progress in all five areas of concern. They completed implementation of the action plans for Strategic Planning and Communication and Decision-making and are continuing to identify next steps in those areas; in addition they are making significant progress toward completing the remaining three. The Africa Regional team is monitoring and facilitating further progress toward continued improvement in the original weak areas. The Program Manager, who joined the team in November 2011, has been receiving project financial information on a monthly basis to assist with project planning. She has proved to be a strong force in assisting the project to make the adjustments recommended in the MTE and bring it to a successful conclusion. EVALUATION QUESTION #2: To what extent was the project implemented as planned? What changes were made to the planned implementation, and why those changes were made? Essentially the project was implemented as planned, however changes were made in response to the MTE recommendations. Immunization: By the time of the MTE, staff were spending an estimated 55% of their time assisting HF personnel in immunization implementation although the CSP LOE was planned at 10%. After the MTE that LOE was scaled back to the original 10% with the project only continuing to support the Child Health Days. Mothers Groups: Originally 300 Mothers Group members were trained. During the MTE it became clear that they were an excellent compliment to the VHTs, who are 75% men, making it possible for the project to reach women in the HH. After the MTE, their number was increased to 1428; they were organized by PDC; and they visited women in their HHs, easing the workload for the VHTs and ensuring pregnant women and mothers of children under two received timely MNC and IYCF information. In the last year of the program a drama group was organized by CSP made up largely of Final Evaluation Report: Lira District Child Survival Project in Uganda December 2013 Final Evaluation 17 MLG members who performed dramas on maternal and child health issues throughout the three sub￾counties in order to reach a broader and larger portion of the population. ECD: The ECD intervention was originally implemented in selected communities in the three sub￾counties. The MTE recommended scarce resources be concentrated in Aromo, the most severely conflict affected of the three, to serve all Aromo communities. 152 additional PEs were trained and delivered ECD training to HHs, instead of through group settings. (see Annex I – Program Learning Brief for more information) PDC: PDCs became more involved in monitoring and supervising project activities to strengthen community ownership and involvement in project management, and to increase the possibility that project activities would be sustainable after the end of CSP. Nutrition: VHTs and MLGs were trained on nutrition using the WHO/UNICEF curriculum for East Africa and supplied with MoH counseling materials as recommended in the MTE. EVALUATION QUESTION #3: How were gender considerations addressed through the project? Are there any specific gender-related outcomes? Existing gender norms in the project area often preclude women from deciding on their own when to seek health services or participate in outreach or health education activities. Men often still make decisions about how women use their time or access family resources. This was addressed in a number of ways. Because 75% of the VHTs are men and the turnover was very low, which prevented adding more female VHTs as originally planned, a cadre of MLGs were added to the community volunteer resources ensuring young mothers had a comfortable resource for HE. VHT contacts also provided men with more information on health concerns, especially information on maternal health and danger signs encouraging them to be more involved in the health of their families and attend ANC with their wives. The village PE team consisted of a man and a woman, again chosen by the community, to ensure both men and women would be reached with the ECD messages, generally together. Both the PEs and the MLGs addressed domestic violence in their health messages as well as male involvement in health issues in the family. The focus groups indicated both that domestic violence had decreased and men were attending more ANC visits with their wives and even assisting with immunizations for their children. Many of the members of the PDCs and HUMCs were men and by involving them in the supervision of community volunteers, CSP encouraged men in leadership positions to also communicate the project’s basic MNC and child health messages. EVALUATION QUESTION #4: which elements of the project have been or are likely to be sustained or expanded? The program was built on the Lira District Five Year Plan and worked in close collaboration with the District Health Office and all of the DHO and governmental structures in the three sub-counties where it was implemented. Both the health personnel and the community volunteers see the health messages as part of their lives and cannot imagine “going back.” The governmental structures from the sub-county chiefs through to the LC1s are engaged in making the systems work. Community level focus groups identified a “better relationship with the HF” as a positive outcome of the program; HF staff discussed how much they appreciated the work of the VHTs and MLGs. Improving the relationships was a significant contribution of the program and one that CSP deliberately addressed. Communication mechanisms were in place with the refresher trainings/quarterly meetings for VHTs. CSP recognized that community people only were disturbed about one or two HP behaviors. The project worked with HF staff towards clearly identifying those negative behaviors and making them feel able and confident they could change them. CSP remained neutral focused only on behaviors. The project chose the time and place for giving feedback to HPs carefully and involved only those from the Final Evaluation Report: Lira District Child Survival Project in Uganda December 2013 Final Evaluation 18 district in whose presence HPs would feel comfortable discussing issues. The HF staff also had realistic concerns about community members and those concerns were taken back to communities by the VHTs and also resulted in positive change. Survey findings indicate more women are delivering at the HF and attendance at ANC is also up, reinforcing comments in the focus groups about the improved attitudes of HPs. Continue Mechanism Needs Community Volunteers Promoting SBC DHO recommends absorbing MLG members and PEs into the Village Health Team. Current oversight mechanisms intend to continue. DHO indicates current referral form will be provided until the DHO completes development of an alternative. Transport resources HUMCs and PDCs Both groups see their roles as vital to the functioning of the HFs and both are government mandated. Sub-county requires PDC level plans for their planning exercise. Transport resources Health Worker Performance Health workers are proud of what they have accomplished and are asking for continued education; DHO/MoH have a continuing education mechanisms in place. Both DHO and Health Providers see from survey results there is more work to be done. Continued Support and Supervision ECD Behaviors The KPC demonstrates that ECD behaviors are continuing in Sub-counties with no further project inputs for past two years. View PEs as volunteers NEW PARTNER GRANTEES: CHANGES IN GRANTEE ORGANIZATION CAPACITY MTI HQ used the CSHGP grant as an opportunity to strengthen MTI Uganda institutionally. Prior to this grant, MTI Uganda was primarily implementing relief programming. At the onset of the grant HQ facilitated an organizational capacity assessment that identified six priority areas needing further work. Before the MTE, a minimum of attention was paid to the assessment results, however, a review of the OCA status was conducted immediately prior to the MTE by the Africa Regional team. The Africa Regional Director position was moved from HQ to Kampala in January 2012 and since then the Regional team has monitored and encouraged progress on OCA issues. An in-depth reassessment was conducted in November 2012 using the CRS assessment tool and combining two of the six areas. It was felt this tool was more appropriate for MTI to identify those areas requiring further interventions. A follow-up review was done in August 2013. MTI Uganda had made considerable progress in all five areas of concern. They completed implementation of the action plans for Strategic Planning, and Communication and Decision-making and are continuing to identify next steps in those areas; in addition they are making significant progress toward completing the remaining three areas: Succession Planning, Staff Salary and Benefits Policy, and Monitoring and Evaluation. The Africa Regional team is monitoring and facilitating further progress toward continued improvement in the original weak areas. Programmatically, MTI has made extensive use of and contributed to CORE technical resources, and built relationships with CORE members to benefit from their greater experience in implementing child survival programming. The USAID mandated MTE proved invaluable in providing very necessary guidance for project improvement. MTI incorporated the MTE recommendations throughout the project. The successful project conclusion is a result of the lessons learned from the MTE, HQ technical assistance to implement the MTE recommendations, and the leadership provided by the Project Manager hired shortly before the MTE. Final Evaluation Report: Lira District Child Survival Project in Uganda December 2013 Final Evaluation 19 CONCLUSIONS MTI implemented a solid child survival program with many statistically significant results: HH/community level indicators improved significantly as did HP quality of care indicators. Delivery mechanisms and management at HF and community levels also dramatically improved. District and Local Government are assuming responsibility for continuing interventions: Because the program chose to work through existing, though minimally trained, government structures and to contribute to the District five-year plan, the various government entities were picking up responsibility for program interventions before the project closed, improving the chances for sustaining the improved behaviors in HHs and HFs. CSP reconstituted or reactivated mandated structures, as with the HUMCs and PDCs and, with the MLGs and PEs, they added in a level keeping the structures and players connected, and continually strengthening their communications. This enabled existing bodies to provide support to HFs and community volunteers in the last months of the project. One of the program strengths was to ensure consistent messages were delivered through all interventions and at all levels. The project sponsored FE Results Meeting for the Lira DHO engaged representatives from all levels – volunteers through the District Chief Administrative Officer in a dialogue regarding the project outcomes and programmatic sustainability. (see Annex XIX C for agenda) Positive relationship between HFs and communities: The often contentious relationship changed to collaboration by the end of the project. This relationship, working with the strong commitment of community volunteers and HPs, increases the possibilities for sustained program interventions and continued progress toward improving community health outcomes. Community Volunteers: VHTs, MLG members and PEs all are described by the community and the HF staff as vital to changing behaviors in HHs. There is a level of respect for them not often seen in programs. The KPC survey results also point to significant improvements in HH behaviors. The MLG members and PEs are not currently a part of the official health structure and may be lost if not incorporated in some way even though they were key to changing behaviors at the HH level. ECD: The ECD program produced more impact that originally envisioned. HPs and parents enthusiastically implemented it, and community members and volunteers attributed decreased family violence and improved relationships between parents and children to the interventions. This was in addition to the significant improvements to the measured indicators. Project Management and OCA: As mentioned in the MTE, this was an ambitious project for a new agency. The country office had no experience with child survival programs and there were gaps in management oversight and support. This was overcome with the arrival of a new Project Manager shortly before the MTE. The project responded rigorously to the MTE recommendations and produced the above noteworthy results. (see Annex XIX D for MTE Recommendation Response) It was originally planned that MTI Uganda leadership would provide management oversight to the CSP. After the MTE, when the gap in management support became apparent, the Africa Regional team became more involved in providing the support needed. They also assumed responsibility for overseeing institutional strengthening for the MTI Uganda office. RECOMMENDATIONS This project always intended to operate for the duration of the grant and then close with the interventions continuing under the existing government structures. That has now happened and the DHO and Sub-county chiefs are assuming responsibility for project interventions and for continuing to strengthen the mandated health structures. The following are strategies suggested to the DHO and Sub-county chiefs to assist with continued progress toward district health objectives: Final Evaluation Report: Lira District Child Survival Project in Uganda December 2013 Final Evaluation 20 1. Support the technical team to do continuous supportive supervision and mentoring of HF staff and community volunteers; 2. Ensure HF staff are able to stay current with the latest health interventions and refresher education; 3. Support/encourage HUMCs and PDCs to continue supporting the HFs and communities including continuing to develop methods of ER transport for acutely ill people; 4. Mobilize the resources of other partners, such as Northern Uganda–Health Integration to Enhance Services (NU-Heights), to focus on those geographic and programmatic areas needing additional support that are consistent with their plans; 5. Ensure VHTs, MLs and PEs are supported with attention, technical updates, refresher training, job aids, referral forms, modest in-kind incentives, and continued respect. Target refresher training especially to those indicators needing more improvement such as post-partum visits; 6. Integrate Mother’s Group volunteers into the community health structure. Because they are not officially part of the health system, they could easily be lost even though they were key to the community gains made by the project. 7. Consider linking village volunteers to other government support programs to provide some “payment” for their services to the community A number of project interventions are remarkable and worthy of replication in future projects while the larger MTI Uganda management issues will need continued attention. It is recommended that close government collaboration be replicated whenever possible, along with strategies to build positive relationships between health workers and the communities they serve. As many other programs across the world have discovered, volunteer cadres of mothers delivering health messages to HHs is an extremely effective way to effect behavior change; this needs to be replicated both in future programming by MTI and also within the DHO mechanism in Northern Uganda whenever possible. The success of the ECD programming was remarkable and the cost was minimal. The intervention changed its form over the life of the project in response to challenges and now needs to develop a strategy for replication at scale. The direct and indirect results are also worthy of considerable attention. The possible ECD effect on reducing family violence needs to be examined more closely to determine the extent to which it influenced that reduction. TABLE OF RECOMMENDATIONS Finding Conclusion Recommenda￾tion Action Who Is Respon￾sible Collaboration with DHO and Governmental system strong; Government assuming responsibility Project strategy to work closely with DHO and government structures effective and increases sustainability Implement projects in close collaboration with governmental systems, especially in decentralized countries Assess country’s commitment to decentralization and design project around that commitment MTI HQ Final Evaluation Report: Lira District Child Survival Project in Uganda December 2013 Final Evaluation 21 Mandated governmental structures are in place, trained and functioning Current functioning is admirable, continued improvement will require sustained attention Continue to support HPs, HUMCs, PDCs and community volunteer cadres Provide technical updates & supportive supervision for HPs & community volunteers; continue engaging PDC & HUMCs and in local health management DHO and Sub County Administra -tion MLGs and PEs are providing quality HE at HH level MLGs and PEs are an effective mechanism for communicating behavior change information to HHs Formally integrate MLGs & PEs into DHOs community volunteer cadre & replicate model in other areas of the District Officially consider MLG members and PEs as part of the Village Health Team DHO HH behaviors changed significantly after MTE & the expansion of MLGs MLGs were effective communicators of MNC and IYCF messages to HHs Replicate the strategy in all HH behavior change projects Include HH level mother volunteers in future behavior change project designs MTI HQ/ Technical team Relationships between health workers and communities improved Project strategy for addressing health provider/ community relationships effective Replicate the strategy in all health projects Include strategies for improving working relationships in future health project designs MTI HQ/ Technical team ECD behaviors continue without program inputs for 18 months ECD messages effective and behaviors sustainable Include ECD programming in future child survival projects Collaborate with HHI to design effective strategy for implementation at scale MTI HQ and HHI OCA findings were not acted upon until after the MTE Country office was not able to address issues until leadership was provided by Africa Regional team Continue to address outstanding OCA issues and maintain successes achieved Africa Regional Team continue to monitor OCA progress and support MTI Uganda staff MTI HQ/ Africa Regional team Final Evaluation Report: Lira District Child Survival Project in Uganda December 2013 Final Evaluation 22 Integrating Early Childhood Development Activities with Child Survival Interventions A child’s early years are a critical period of development during which the foundations for children's emotional, social, intellectual and physical wellbeing are developed. The Lira Child Survival Project educated health workers and caregivers about early childhood development and simple actions they can take daily to support their infants was integrated as a key strategy. This project was funded by the U.S. Agency for International Development through the Child Survival and Health Grants Program. December 2013 Background Uganda’s Early Childhood Development (ECD) policy was established in 2007. While the Government of Uganda recognizes that ECD is important and considers it as the first level of education (Education Act 2008), investment in ECD is low. ECD services are primarily sector based. Health, education and social services are provided in different settings. Pre-school services in urban areas are provided by private providers and by NGO’s in rural areas. The Early Childhood Learning Framework was developed for ages 3-6 years and is a curriculum reference for implementers of ECD but is not mandated. As of yet, there is no framework for the ages of 0-3 years. Project Design The post-conflict period is often characterized by food insecurity, unreliable infrastructure, and a fragile family unit. It is an important time to rebuild community resilience by strengthening the parent-child bond, developing and reinforcing positive caregiving practices to improve long term child health and well-being. This can ameliorate the negative effects of the compromised environment, and build the foundation for physical and emotional health. In the Northern Uganda, post￾conflict context, service provision is continually improving as the Ministry of Health and NGOs re-construct health facilities and provide health services at the community level. The Lira Child Survival Project implemented by Medical Teams International and the Lira District Health Office served a population of 124,379 and operated in Lira, Aromo and Ogur sub-counties of Lira District in Northern Uganda. The project used a two-pronged approach to improve the health and nutrition status of women and children under age five: a) promoting behavior change and community mobilization to take appropriate responsibility for health; and b) building District Health Office (DHO) capacity to provide sustainable, quality service delivery at the facility and community levels. Educating health workers and caregivers about ECD and simple actions they can take daily to support their infants was integrated as a key strategy. ECD Peer Educator Tonny Anyang of Apua Parish teaches a mother about baby cues Key Findings: After training in early childhood development • 76% of mothers played games that provided cognitive stimulation for their child increased from 38% at baseline. • 78% of mothers told a story, sang a song or named objects for their child to encourage linguistic learning increased from 23% at baseline. • 84% of mothers helped their child walk, massaged them, or played a game to encourage physical development increased from 63% at baseline. PROGRAM LEARNING BRIEF December 2013 Methodology Hands to Hearts International (HHI), an international NGO based in Portland, Oregon, provided technical support for the ECD component of the project. The HHI’s A Curriculum for the Study of Early Childhood Development Trainer’s Manual was adapted to the local context and integrated available Uganda resources in the training design. The training was composed of eight modules: 1) Importance of the Role of Parents, 2)Physical Development, 3) Cognitive Development, 4) Language Development, 5)Social and Emotional Development, 6) Baby Cues, 7) Baby Massage and 8) Basic Health, Hygiene & Nutrition and Sanitation. HHI master trainers provided training of trainers in ECD for health facility staff and MTI Child Survival Project staff. These staff in turn recruited and trained 300 volunteer Peer Educators to provide education on both ECD and health issues to families in their communities. Peer Educators working in female/male pairs led group trainings in their communities to reach both men and women caregivers. The project originally planned to conduct operations research to measure the impact of ECD training on health indicators. Intervention and comparison areas were mapped with the plan to compare the results in implementation communities with comparison communities. At midterm, however, it was discovered that Peer Educators, health staff and parents in the intervention area were so enthusiastic about the ECD skills they learned that they shared them with caregivers in the comparison areas. This sharing of information made it impossible to compare the impact of ECD training between the two areas. To have a greater impact on a smaller area, formal ECD activities were targeted to Aroma sub-district for the last 18 months of the project. Aromo was selected for special attention as this sub-district was more affected by the conflict than either Lira or Ogur sub-districts, had the largest portion of displaced families returning at the beginning of project and had greater infrastructure damage. One hundred fifty two additional Peer Educators were trained. During this phase, Peer Educators delivered ECD lessons directly to caregivers in their homes. Findings To measure impact of the ECD intervention, questions were included in the baseline, midterm and final Knowledge, Practice and Coverage surveys to measure household practices encouraging cognitive, linguistic and physical development. Focus group interviews were also conducted with parents, community volunteers and health staff to provide information on project participants’ perspective of their participation in the ECD component and its impact. Quantitative data collected during the final survey showed a remarkable increase in key ECD indicators. After training in early childhood development, the percentage of mothers who played games which provided cognitive stimulation increased from 38% at baseline to 76% at final. The percentage of mothers who told a story, sang a song, or named objects for their child to encourage linguistic learning increased from 23% to 78% and the percentage of mothers who helped their child walk, massaged them, or played a game to challenge physical development increased from 63% to 84%. When broken down by sub-district, the data indicated that families in Lira and Ogur continued ECD practices after 18 months of no active ECD interventions, demonstrating impressive sustainability. Indicator Lira Ogur Aromo Linguistic Learning: % of mothers of children 6-23 months who told a story, sang a song or named objects for their child (linguistic learning) at least 2 times per week 83.3 72.1 76.1 Active Feeding: % of mothers of children 6-23 months who report talking or singing to their child while feeding them 63.9 63.0 67.0 Physical Development: % of mothers of children aged 0-23 months who report helping their child walk, playing a lap game with their child, massaging or rubbing them gently, or by giving them objects with which to play 83.5 83.8 83.5 Qualitative interviews with parents and Peer Educators conducted during the project final evaluation indicated a widespread enthusiasm for ECD promotion and an understanding of the importance of supporting a young December 2013 child’s development. Parents reported spending more time with their children playing, teaching them how to talk, and making toys for them. They also felt their relationships with their children improved and the parent – child bond was stronger. One parent comment “My child is not afraid of me anymore; he is happy to see me,” and another “I have permission to play with my child.” Discipline was achieved by taking and encouragement rather than physical punishment which had previously been the norm. Parents felt baby massage helps babies walk on time, sleep better, and not be so fussy. Mothers, fathers and Peer Educators reported a decrease in domestic violence. It is difficult to pinpoint the elements of ECD or other outside efforts that might have contributed to these observations. Community members reported hearing radio messages about family violence and mentioned interventions by community leaders and churches to mitigate domestic violence in their communities. The change was most likely due to the impact of improving household ECD skills coupled with government and church and community leader efforts to decrease levels of violence in post-conflict Lira District. Conclusions and Lessons Learned Post conflict is an important time to rebuild community resilience by strengthening the parent-child bond, developing and reinforcing positive caregiving practices to improve long term child health and well-being. Integrating early childhood development with child survival interventions in Lira District has led to improvements in parent-child relationships and has encouraged caregivers to spend greater and more meaningful time with children reading, singing, and playing games that encourage physical, cognitive and language development. Community members also report it has facilitated better relationships between spouses. By helping parents better understand and interact with their children, it also seems to be a contributing factor to reducing household violence. Because of the spillover of ECD messages between intervention and comparison communities, it is not possible to attribute gains the project made in health practices to the integration of early childhood development messages. Recommendations and Use of Findings The findings of this project are extremely encouraging and demonstrate the ability of HHI’s ECD model to be replicated at scale. As evidenced by qualitative and quantitative data, the approach promotes a wide variety of positive behavior changes. The fact that families in Lira and Ogur are practicing the ECD skills they learned 18 months after activities ceased suggests the approach is sustainable as well.. The greatest learning for HHI was the model MTI used of training a male and female Peer Educator in each village of more than 220 villages, this encouraged the local communities to feel ownership of the information and created lasting local resources in the PEs who remain long after the end of this project, additionally it encouraged men to participate and thereby increase their involvement in caregiving and likely was a factor in decreasing domestic violence. Given the significant and positive outcomes in child-caregiver relations, the decrease in domestic violence and the overall implications for child health and development, it would be wise for the Lira District Health Office to continue to support the existing PEs with once a year refreshers and small incentives to carry on the ECD messages. Lira has shown a simple, impactful and successful model to promote ECD, it could act as a model for the country and not allow the power of this intervention to fade and be forgotten. The Lira Child Survival Project in Erute North Sub-District, Northern Uganda is supported by the American people through the United States Agency for International Development (USAID) through its Child Survival and Health Grants Program. The Lira Child Survival Project is managed by Medical Teams International under Cooperative Agreement No. GSH-A-00- 09-00012. The views expressed in this material do not necessarily reflect the views of USAID or the United States Government. For more information about The Lira Child Survival Project, visit: www.medicalteams.org December 2013 ANNEX II. LIST OF PUBLICATIONS AND PRESENTATIONS RELATED TO THE PROJECT A presentation on the Lira Child Survival Project final evaluation and lessons learned was facilitated for Medical Teams International headquarters technical and management staff on November 19th, 2013. Final Evaluation Report: Lira District Child Survival Project in Uganda December 2013 Final Evaluation - Annexes ANNEX III. PROJECT MANAGEMENT EVALUATION Prior to this grant, MTI Uganda was primarily implementing relief programming. Simultaneously with the award of the CSP, MTI was awarded a grant for a Malaria Communities Project also implemented in Northern Uganda. MTI HQ used the CSHGP grant as an opportunity to strengthen MTI Uganda institutionally. At the onset of the work, HQ facilitated an organizational capacity assessment that identified six priority areas needing further attention. They originally used JSI developed materials to assess the capacity of MTI Uganda to provide support to the project. Prior to the MTE, a minimum of attention was paid to the assessment results, however, a review of the OCA status was conducted immediately prior to the MTE by the Africa Regional team. Since the Africa Regional Director position was moved from HQ to Kampala in January 2012, the Regional team has monitored and encouraged progress on the OCA issues. After the MTE, it was decided that the newly published Catholic Relief Services OCA materials were more appropriate for MTI Uganda’s situation. In the baseline OCA there were six areas prioritized for improvement. Two of those, Communication and Decision-making were combined in the final OCA. The areas of concern were: Succession Planning, Staff Salary and Benefits Policy, Strategic Planning, Communication and Decision-making, and Monitoring and Evaluation. The complete end of project OCA matrix follows this narrative. The MTI Africa Region team facilitated an OCA and planning process in November of 2012, and reported on the implementation results as of August 2013. (see Annex X1X A) MTI Uganda had made considerable progress in all five areas of concern. They completed implementation of the action plans for Strategic Planning and Communication and Decision-making and are continuing to identify next steps in those areas; in addition they are making significant progress toward completing action plans in the remaining three areas. The Africa Regional team is monitoring and facilitating further progress toward continued improvement in the original weak areas. It was originally planned that MTI Uganda leadership would provide management oversight to the CSP. After the MTE, when the gap in management support became apparent, the Africa Regional team became more involved in providing the support needed. The HQ technical team provided rigorous oversight and technical support to the project. The new Program Manager, who joined the team in November 2011, has proved to be a strong force in assisting the project in making the adjustments recommended in the MTE and bringing it to a successful conclusion. She provided a vision of the direction the program needed to go that, with the backstopping provided by the HQ technical advisors and Africa Regional team, enabled the project to produce KPC and RHFA results frequently significantly higher than the MTE results. The problem of no financial information being available to the Program Manager was resolved after the MTE; the last year of the project she received program financial information monthly. Lessons Learned 1. As an organization new to Child Survival programming, the mid-term evaluation was invaluable to provide an outside assessment of program progress and allow sufficient time for needed course corrections. 2. HQ technical advisors and senior management need to very closely monitor program management in country offices unfamiliar with Child Survival programming to ensure recommendations are followed. 3. Opportunities for strengthening an organization and country office are rare and need to be acted upon promptly by those with authority to make changes. Final Evaluation Report: Lira District Child Survival Project in Uganda December 2013 Final Evaluation - Annexes ANNEX IV. WORK PLAN TABLE Activity Activities Achieved, Dropped, Changed, or Added Start Up and Planning Recruit project staff All start up and planning activities were achieved Orientation training for staff Introductory visits and discussions with DHO at the district, sub-county and health facility level Planning Meeting with DHO Develop BCC strategy  focus groups  community feedback  doer/ non-doer surveys Preparation of DIP with consultant and DIP workshop Meeting with USAID in Kampala DIP meeting with DHO Cross visit to HealthPartners Child Survival and MCP projects in Bushenyi District Revise budget to be submitted with DIP Mapping health facilities and VHTs Meet USAID in Gulu – James Okello IMCI/ Support Identify local/country materials 34 health staff from four facilities were trained in IMCI in May 2010. Four new health Centre IIs opened in Ogur and Aromo sub-counties in April 2012. Supportive supervision and mentoring was expanded to cover all eight facilities. The project supported the newly opened facilities to establish HUMCs and take responsibly for supporting the CHWs in their catchment areas. Forty health workers at the 8 facilities were provided with IMCI refresher training in June 2013. Mentoring for health facility staff was provided on a monthly basis. DHO and MTI staff provided joint supportive supervision on a quarterly basis. Plan IMCI training with DHO Provide IMCI training Conduct clinical IMCI mentoring Conduct joint supervision of IMCI services C-IMCI and Community Case Management Plan C-IMCI training with DHO 560 VHTs were trained and provided community mobilization, community education, household visits, and referrals with support from project mobilizers. Identify local/country materials Train VHTs on C-IMCI VHTs provide community education with support from mobilizers Strengthen referral system from community to health facility by providing training to VHTs and providing referral forms Final Evaluation Report: Lira District Child Survival Project in Uganda December 2013 Final Evaluation - Annexes Activity Activities Achieved, Dropped, Changed, or Added Maternal and Newborn Care Identify local materials and tools Originally 300 Mothers Group members were trained. During the MTE it became clear that they were an excellent complement to the VHTs, who are 75% men, making it possible for the project to reach women in the HH. After the MTE their number was increased to 1148; they were organized by PDC; and they visited women in their HHs, easing the workload for the VHTs and ensuring pregnant women and mothers of children under two received timely MCN and IYCF information. During Year 3, the MLGs and VHTs were trained using the WHO/UNICEF Community Based Infant and Young Child Feeding Counseling Package. . MoH IYCF National Counseling Cards for Community Volunteers and Safe Motherhood Flip Charts for both facility staff and community volunteers were reproduced to support them during the health education In the last year of the program a drama group was organized by CSP made up largely of MG members who performed dramas on maternal and child health issues throughout the three sub-counties in order to reach a broader and larger portion of the population. Facility staff was provided with refresher training in MNC and follow up supportive supervision by the project MNH Mentor. Train VHTs on MNC VHTs provide community education with support from mobilizers Conduct MNC refresher training at HF level Supportive Supervision of perinatal services EPI Train VHTs on EPI By the time of the MTE, staff were spending an estimated 55% of their time assisting HF personnel in immunization implementation although the CSP LOE was planned at 10%. After the MTE that LOE was scaled back to the original 10% with the project only continuing to support the Child Health Days. VHTs provide community education with support from mobilizers Support Child Health Days / national immunization days. Early Childhood Development Hands to Hearts hires local trainers The ECD intervention was originally implemented in selected communities in the three sub-counties. The MTE recommended scarce resources be concentrated in Aromo, the most severely conflict affected of the three, to serve all Aromo communities. 152 additional PEs were trained and delivered ECD training to HHs, instead of through group settings. See Annex I – Program Learning Brief for more information. Training of Trainers in ECD ECD Trainers train Peer Educators in ECD in intervention areas ECD Trainers train VHTs in ECD in control areas Peer Educators train community members in intervention areas VHT Trainers train community members in control areas Follow up & Refresher TOT; gather lessons learned Coordinate, support and supervise Peer Final Evaluation Report: Lira District Child Survival Project in Uganda December 2013 Final Evaluation - Annexes Activity Activities Achieved, Dropped, Changed, or Added Educators Conduct ECD Review Meeting Conduct ECD Forum VHT Support Collect information on VHT membership and status During the first two year of project implementation MTI and health facility staff met quarterly with the VHTs at the sub￾county level for an all-day session which included reviewing reports, discussing issues faced by the VHTs, coordinating work, and imparting some training. To provide more effective support, after midterm sessions were held monthly at the parish rather than sub￾county level with smaller groups. PDCs became more involved in monitoring and supervising project activities to strengthen community ownership and involvement in project management, and to increase the possibility that project activities would be sustainable after the end of CSP. Provide joint supportive supervision Support health facility staff to arrange and facilitate coordination meetings with VHTs Health Unit Management Committee members attend VHT – health facility coordination meetings on a quarterly basis M&E Conduct baseline and final KPC Surveys M and E activities took place as planned. Solicit community feedback of KPC results Conduct focus groups discussions, and doer/ non-doer surveys LQAS Surveys Conduct HFA Midterm and Final Evaluations Final Evaluation Report: Lira District Child Survival Project in Uganda December 2013 Final Evaluation - Annexes ANNEX V. RAPID CATCH TABLE Rapid CATCH Table Indicator Baseline Estimate (%) MTE Estimate (%) Final Estimate (%) Percentage of children 0-23 months who are underweight (-2 SD for the median weight for age, according to WHO/HCHS reference population) 27.7% 16.2% 20.2% Percent of infants and young children aged 6-23 months fed according to a minimum of appropriate feeding practices. 23.1% 41.9% 48.9%* Percentage of mothers of children age 0-23 months who had four or more antenatal visits when they were pregnant with the youngest child 35.3% 50.0% 57.8%* Percentage of mothers with children age 0-23 months who were protected against Tetanus before the birth of the youngest child. 75.7% 73.7% 89.4%* Percentage of children age 0-23 months whose births were attended by skilled personnel 35.3% 50.5% 83.9%* Percent of children 0-23 months who received a post - partum visit by an appropriate trained health worker within three days after birth 16.3% 22.2% 40.8%* Percentage of children 0-5 months who were exclusively breastfed during the last 24 hours 73.6% 66.2% 88.2% Percentage of children age 6-23 months who received a dose of Vitamin A in the last 6 months (Mother’s recall). 70.1% 57.6% 85.3%* Percentage of children 12-23 months who received DPT1 according to the vaccination card or mother’s recall by the time of the survey 87.0% 89.4% 97.0%* Percentage of children 12-23 months who received DPT3 according to the vaccination card or mother’s recall by the time of the survey 85.1% 72.7% 91.7% Percentage of children 12-23 months who received a measles vaccination 77.0% 80.8% 82.1% Percentage of children 0-23 months who slept under an insecticide-treated bed net (in malaria risk areas, where bed net use is effective) the previous night. 51.3% 44.4% 58.3% Percentage of children 0-23 months with a febrile episode that ended during the last two weeks who were treated with an effective anti-malarial drug within 24 hours after the fever began. 25.0% 68.7% 51.5%* Final Evaluation Report: Lira District Child Survival Project in Uganda December 2013 Final Evaluation - Annexes Percentage of children 0-23 months with diarrhea in the last two weeks who received Oral Rehydration Solution (ORS) and/or recommended home fluids. 47.2% 52.5% 61.2% Percentage of children age 0-23 months with chest-related cough and fast/difficult breathing in the last two weeks who were taken to an appropriate health provider. 57.8% 84.8% 74.2% Percentage of households of children 0-23 months that treat water effectively. 11.3% 10.1% 41.4%* Percentage of mothers of children 0-23 months who live in households with soap at the place for hand washing 85.0% 87.4% 96.4%* Percentage of mothers of children age 0-23 months who are using a modern contraceptive method 33.3% 30.3% 28.1% * - Statistically Significant Final Evaluation Report: Lira District Child Survival Project in Uganda December 2013 Final Evaluation - Annexes Annex VI: Lira District Child Survival Project in Uganda Child Health and Development in a Transitional Region Erute North Sub-District, Uganda October 2009 – September 2013 In Partnership with Uganda Ministry of Health Lira District Health Office Final Knowledge, Practice, and Coverage Report September 2013 Acknowledgements_______________________________________________________ The FE survey of Lira District, Uganda is the result of a cooperative effort between many dedicated people living in Uganda, Medical Teams International Uganda, the Lira DHO, and MTI HQ. I am thankful to the staff of Medical Teams International Uganda for providing the opportunity to produce the FE KPC survey in Lira District. Thanks go to the Uganda Country Director Felix Omodi for oversight of the evaluation., Also, special thanks go to MTI Uganda Child Survival Program Manager Lydia Akulo, Africa Health Program Advisor Michael O’Brien, and MTI Monitoring and Evaluation Officer George Aguze, for their support, assistance, and collaboration in every aspect of the evaluation, including the design and development of the survey tools, hiring and training the survey staff, and managing every aspect of the data collection. Finally, I would like to express my gratitude and respect to each of the team members of the survey. Their tireless work and wonderful attitude made this survey possible. The Supervisors, staff members, and enumerators were a collaborative group consisting of members of MTI Uganda and the DHO and were essential in the completion of the survey. The supervisors and staff members of Medical Teams International Uganda will be essential in continuing the programs to assist the people of Lira district. Dr. Todd A. Nitkin, DPM, FACFAS, MPH Sr. Advisor in Monitoring and Evaluation Representative, Washington, DC Medical Teams International Medical Teams International Lira Child Survival Project Final KPC 2 Table of Contents Acknowledgements 2 Table of Contents 3 Acronyms 4 Executive Summary 5 Chapter 1 Background, Process and Partnership Building, and Methods 10 1.1 Background 10 1.2 Process and Partnership Building 11 1.3 Methods 12 Chapter 2 Maternal and Newborn Care 18 Chapter 3 Anthropometry and Infant and Young Child Feeding 20 4.1 Anthropometry-Nutritional Status of Children aged 0-23 months 20 4.2 IYCF-Nutrition 21 Chapter 4 Pneumonia Case Management 23 Chapter 5 Control of diarrhea 24 Chapter 6 Expanded Program of Immunization 27 Chapter 7 Early Childhood Development 29 Chapter 8 Malaria and the Management of Febrile Illness 33 Chapter 9 Health Contacts 35 Chapter 10 Summary 39 Attachments: A. Complete List of Indicators, including Project Indicators, Rapid CATCH indicators, and indicators of interest, with Comments Medical Teams International Lira Child Survival Project Final KPC 3 ACRONYMS ACT Artemisinin-based Combination Therapies ANC Antenatal Care ARI Acute Respiratory Infection BCC Behavior Change Communication BL Baseline Assessment CATCH Core Assessment Tool on Child Health CDD Control of Diarrheal Diseases CHW Community Health Workers C-HIS Community Health Information System C-IMCI Community IMCI CI Confidence Interval CL Confidence Limits CMR Crude Mortality Rate CS Child Survival CORE Collaborations and Resources Group CSHGP Child Survival and Health Grant Program CSP Child Survival Project D Precision DHO District Health Office DHS Demographic and Health Survey EBF Exclusive Breastfeeding EPI Expanded Program of Immunizations FE Final Evaluation HHI Hands to Hearts International HC Health Center HF Health Facility HIV/AIDS Human Immune Deficiency Virus/ Acquired Immune Deficiency Syndrome HQ Headquarters of MTI located in Portland, Oregon IDP Internally Displaced Person IMCI Integrated Management of Childhood Illnesses IMR Infant Mortality Rate IPTp Intermittent Preventive Treatment during pregnancy ITN Insecticide Treated Net IYCF Infant and Young Child Feeding KPC Knowledge, Practice, and Coverage Survey LLITN Long Lasting Insecticide Treated Net M&E Monitoring and Evaluation MCH Maternal and Child Health MICS Multiple Indicator Cluster Survey MNC Maternal Newborn Care MOH Ministry of Health MT Midterm MTE Midterm Evaluation MTI Medical Teams International MTI Uganda Medical Teams International Uganda N Sample size NGO Non-Governmental Organization NMCP National Malaria Control Program NUMAT Northern Uganda Malaria AIDS & Tuberculosis ORS Oral Rehydration Salts PCM Pneumonia Case Management PDC Parish Development Committee PHC Primary Health Care POU Point Of Use Medical Teams International Lira Child Survival Project Final KPC 4 Rapid CATCH Core Assessment Tool on Child Health R-HFA Rapid Health Facility Assessment SP Sulfadoxine-Pyrimethamine SS Statistically Significant SSD Statistically Significant Decline SSI Statistically significant Improvement SO Strategic Objective TBA Traditional Birth Attendant TOT Training of Trainers TT Tetanus Toxoid U5MR Under 5 Mortality Rate USAID United States Agency for International Development VHT Village Health Team WFA Weight for Age Medical Teams International Lira Child Survival Project Final KPC 5 Executive Summary At the outset of the project Uganda was a priority country for child survival efforts, with an IMR estimated in 2006 of 78/1,000 live births, U5MR of 134/1,000 live births, and an MMR of 550/100,000 which had not declined during the past ten years. The leading causes of child morbidity in Lira District were (in rank order): malaria, anemia, diarrhea, respiratory infections, and pneumonia. Causes of child mortality were: pneumonia, anemia, malaria, diarrhea, and respiratory infection. Malnutrition was an important contributing factor to infant and child deaths. The targeted location was Erute North Sub-district in Lira District in Northern Uganda. Direct beneficiaries were 25,498 children <5 and 25,498 WRA for a total of 50,996 direct beneficiaries. Capacity building activities with the DHO were aimed to improve the quality of health care for three sub-districts of Lira, Ogur and Aromo with a total population of 124,379. The project goal was to reduce child morbidity and mortality in Uganda. Objectives were: 1) Communities assume responsibility for their own health through strengthening community capacity (Village Health Team volunteers (VHTs), Parish Development Councils, and Health Sub-districts); 2) Improved health (C-IMCI) and child care (ECD) behaviors among mothers of children <5 years; 3) Improved quality of health facility services through strengthened IMCI and MNC capacity; 4) Strengthened institutional capacity of MTI and DHO to implement effective and efficient child survival activities. These objectives support MoH goals and strategies as well as those of USAID Uganda. MTI utilized a two-pronged strategy: a) promoting behavior change and community mobilization to take appropriate responsibility for health: and b) building DHO capacity to provide sustainable, quality service delivery at the facility and community levels. The levels of effort by intervention for this Child Survival Project (CSP) were: 25% MNC, 25% PCM, 20% IYCF, 20% CDD, and 10% EPI. MTI incorporated Early Childhood Development (ECD) activities into its CS project to enhance the impact and sustainability of technical interventions. Past research confirmed that child survival is indivisible from ECD – including health, physical, social/emotional, and language/cognitive domains. The CSP, therefore, integrated ECD into community health activities in order to improve feeding, care giving, and care-seeking behaviors by increasing women’s participation in CHW (VHT) structures and integrating ECD into C-IMCI and ANC/EPI outreaches. The primary implementing partner for this project was the Lira DHO, which regularly met with the design team and committed human resources to the implementation of the proposal, in order to harmonize approaches and plan for sustainability. Hands to Hearts International (HHI) was a collaborate partner, providing ECD TOT trainings for Peer Educators, and HF and MTI staff, and working with MTI and DHO to adapt the HHI curriculum to the local context. The project also coordinated with other CS stakeholders in country. This FE Knowledge, Practices, and Coverage (KPC) survey was performed in August, 2013. The overall objective of this FE survey was to estimate the current level of chosen indicators as per the monitoring and evaluation (M&E) Matrix of specific objectives and intervention logic in the areas of maternal newborn care (MNC), nutrition and Infant and Young Child Feeding (IYCF), Medical Teams International Lira Child Survival Project Final KPC 6 control of diarrheal disease (CDD), pneumonia case management (PCM), and Expanded Program of Immunization (EPI), along with the indicators of the Rapid Core Assessment Tool on Child Health (CATCH). A 30-cluster stratified sampling design was utilized to select the mothers of children aged 0-23 months. A total of thirty clusters were sampled in Lira District, with twelve households from each cluster being selected for interviews. The KPC FE survey was designed utilizing participatory principles of evaluation in the spirit of partnership and capacity building.1 The main findings in the areas of MNC, nutrition and IYCF, control of diarrheal disease, pneumonia case management, EPI, and ECD are as follows: MNC Eleven of the 12 MNC indicators, including all 6 of the indicators in the project Logframe and all the Rapid CATCH indicators for MNC, were statistically significantly improved (SSI) through the life of the project. In fact, the only MNC indicator not SSI is the rate of use of modern contraception. • The rate of mothers receiving at least 2 TT immunizations was already relatively high at baseline (BL) (73%), but still the project was able to sow a SSI in this indicator to 89%. • Additionally the rate of mothers taking proper IPT medication while pregnant was SSI from only 35% at BL to 68% at FE, and the rate of mothers who received iron while pregnant was SSI from 59% at BL to 91% at FE. • Also, regarding health messaging, there was a statistically significant improvement in the rate of mothers who are able to correctly identify at least 2 neonatal danger signs for which they should seek immediate care, from 2% at BL to 51% at FE. • At FE, 84% of all childbirths occur under the supervision of a skilled birth attendant, which is SSI from a BL rate of 35%. • There was a SSI in this the rates of both mothers and children who received a postnatal visit from a trained HW within 2 days after birth. Mothers received a visit at a rate of 50% at FE (up from only 17% at BL), and children received a postnatal visit at a rate of 41% at FE (up from 16% at BL). • The number of mothers with children aged 0 – 23 months who use a modern method of contraception was 33% at BL and 28% at FE. Anthropometry AND IYCF • At FE, The percentage of children with a Z-score of <-2SD at FE was 20%, not significantly decreased from the BL rate of 28%. However, the IYCF indicator relating to the proper feeding frequency and food diversity for children aged 6-23m was SSI from only 23% at BL to 49% at FE. • The % of children aged 0-5m who were exclusively breastfed during the 24 hours prior to the survey, while already high at 74% at BL, was SSI to 88% at FE. • Immediate BF practices for newborns was dramatically and SSI from 29% at BL to 79% at FE. There was also a SSI in the rate of mothers who did not give their child prelacteal feeds (from 47% at BL to 87% at FE), and therefore the summary indicator of immediate and exclusive breastfeeding was SSI from only 16% at BL to 71% at FE. 1 KPC 2000+ Field Guide, The Child Survival Technical Support Project and CORE, http://www.childsurvival.com/kpc2000/kpc2000.cfm, August 2001. Medical Teams International Lira Child Survival Project Final KPC 7 • There has been a SSI in the composite IYCF indicator (the rate at which children aged 6- 23m are fed the proper frequency and food group variety in their diet dependant on their age and breastfeeding status) from 23% at BL to 49% at FE. There was a SSI in proper feeding frequency in breastfed children (31% at BL to 66% at FE) which was the driving factor behind this improvement. • The % of children receiving Vitamin A in the past 6m at FE (85%) was SSI from BL (70%). Pneumonia Case Management • The % of children age 0-23 months with ARI who were treated with an antibiotic increased significantly form BL to FE (35% at BL to 69% at FE). • The % of children with ARI who were taken to an appropriate health provider increased significantly over the first half of the project (58% at BL to 86% at MT, but the FE result of 74% was not a SSI from BL. Control of Diarrheal Disease • There was a SSD in the prevalence in the rate of diarrhea at FE (24.3%) from BL (36%), which highlights the gains made in hygiene and water and sanitation (see below in this paragraph). • There has been a SSI in the % of children who received ORS when suffering from diarrhea since baseline (47% at BL to 62% at FE). • Zinc usage in children to treat diarrhea, which was almost non-existent at BL (2.6%), was still low but SSI to 18% by FE. • Hygiene was improved, with the % of mothers who have soap at the place for hand washing and reported washing their hands with soap at least 2 of the appropriate times during a 24 hour recall period (after defecation and at one of the following: before food preparation, before feeding children, or after attending to a child who has defecated) SSI from 54% at BL to 88% at FE. • The rate of safe feces disposal was SSI from 13% at BL to 61% at FE. • The % of households of children 0-23m that treat water effectively was SSI from28% to 38% from BL to FE. Expanded Program of Immunization The project drastically improved immunization coverage in the project area, with 9 of the 11 measured EPI indicators statistically significantly improved. These are excellent achievements that verify excellent gains in immunization at the HFs and through outreach, as well as IMCI, HF supervision, and drug chain management at the HF level. • Full EPI Coverage (% of children aged 12-23 months who received a BCG, DPT3, OPV3, and measles vaccination before the age of 12 months, card verified) showed a SSI from16% at BL to 56% at FE. • Card verified DPT1 was raised from 47% at BL to 91% at FE, and card verified DPT3 was raised from 35% at BL to 85% at FE. • Card verified measles vaccination was raised from 11% at BL to 66% at FE. Early Childhood Development Medical Teams International Lira Child Survival Project Final KPC 8 MTI incorporated ECD activities into its CS project to enhance the impact and sustainability of technical interventions. Research confirms that child survival is positively linked to ECD – including health, physical, social/emotional, and language/cognitive domains. • Six of the 7 ECD indicators that pertain to ECD behaviors at least 2 times per week were SSI over the course of the project. o There has been a SSI from BL of mothers of children aged 0-23 months who report playing games with their child in which they have their child identify their body parts, imitate actions, pretend play, or name objects (38% at BL to 76% at FE). o There has been a significant increase from BL of mothers report engaging their children in linguistic learning activities such as telling their child stories, singing them songs, or naming objects for them at least twice weekly (23% at BL to 64% at FE). • Beliefs regarding the importance of several ECD components were also SSI. All of the indicators measuring beliefs regarding the fundamental principles behind ECD were SSI, except for one which was already above 90% before ECD messaging began. • There has been a SSI from BL in the rate of mothers who report that they talk or sing to the child while feeding the child (58% at BL to 84% at FE). Malaria The project did not have a malaria component but rates of malaria and malaria prevention and care seeking practices improved. • Over the course of the project, the rate of malaria in children under 2 was SSD from 75% at BL to 58% at FE. • There has been a SSI in the use of bed nets in mothers of children under 2 since BL (41% at BL to 57% at FE), which is commendable considering there is a decrease in the availability of ITNs. However, the rate of children under 2 sleeping under a bed net was not statistically changed from 51% at BL to 58% at FE. • The rate of households that contained at least 1 ITN was SSD from 82% at BL to 58% at FE. This correlates with the findings of the health facility assessment at final that should a lack of available nets for distribution in the entire project area. • There has been a SSI in the rate of children treated with an antimalarial drug within 24 hours of the start of the fever, from 25% at BL to 52% at FE. The driving factor behind this increase is the fact that the rate of care seeking (mothers who sought care for their child with fever within24 hours) was SSI from only 22% at BL to 83% at FE. Health Contacts A great success of this project was the very significant increase in contact rates between mothers in the project area and qualified health workers orsources of accurate health information regarding MNC and mother and child health. • Due to the projects strengthening of community capacity through VHTs, Parish Development Committees (PDCs), Mother Leader Groups, and Health sub-districts, the rate of mothers having contact with trained health personnel in the previous month was SSI from only 29% at BL to 92% at FE. • Contact with VHTs led the SSI, with a rate of 81% by FE, up from only 10% at BL. Medical Teams International Lira Child Survival Project Final KPC 9 • Mothers now (at FE) attend a monthly meeting of some type in order to discuss their health or the health of their baby at a rate of 85%, a very large SSI from the baseline rate of 23%. • There is a high saturation of health messaging in all villages from the VHTs, at 80% at FE, and also Health Educators, at 61% at FE. • There is also high saturation of health messaging through the radio, which is shown by the high rate of mothers who report hearing health messaging from the radio, at 82% at FE. • There are high rates of mothers hearing each important health topic (hand washing, treating water, danger signs/care seeking, diarrhea care, and SBA), with all rates above 75% (see the table for Health Contacts in the Results section). Working in partnership with the DHO, MTI Uganda completed a project that improved the health of village communities in Lira District through building DHO capacity to provide sustainable, quality service delivery at the facility and community levels, and through promoting behavior change and community mobilization to take appropriate responsibility for health. This was accomplished through a combination of interventions, including providing supplementary training, supervision, and follow-up coaching of VHTs. The results of this FE KPC Survey reveal the many sustainable successes were accomplished in Lira District, as well as some continuing challenges of this project area that may be used in future projects as lessons learned. Medical Teams International Lira Child Survival Project Final KPC 10 CHAPTER 1 BACKGROUND, PROCESS AND PARTNERSHIP BUILDING, AND METHODS 1.1 Background Program Strategy and Interventions Goal and Objectives The project goal was to reduce child morbidity and mortality in Uganda. These objectives supported MoH goals and strategies.  Objective 1: Communities assume responsibility for their own health through strengthening community capacity (VHTs, Parish Development Councils, and Health Sub-districts).  Objective 2: Improved health (C-IMCI) and child care (ECD) behaviors among mothers of children <5 years.  Objective 3: Improved quality of HF services through strengthened IMCI and MNC capacity.  Objective 4: Strengthened institutional capacity of MTI and DHO to implement effective and efficient child survival activities. Strategic Approaches: MTI utilized a two-pronged strategy: a) promoting behavior change and community mobilization to take appropriate responsibility for health: and b) building DHO capacity to provide sustainable, quality service delivery at the facility and community levels. The CSP implemented activities that strengthen community volunteer capacity to improve maternal and child health, based on the MoH policy prioritizing Village Health Teams (VHTs). Through the CSP, supplementary training, supervision, and follow-up coaching of VHTs was provided. The CSP worked hand-in-hand with DHO and HF staff that functions as support for VHTs. At present, the Senior Health Educator of the DHO oversees VHT activities. At their request, the CSP assisted the DHO to build on previous training and strengthen the health facility staff skills through refresher trainings focused on selected topics within IMCI, such as CDD management with ORS+zinc and PCM. Technical Interventions: The level of effort by intervention for this CSP was: 25% Maternal and Newborn Care, 25% Pneumonia Case Management, 20% Infant and Young Child Feeding, 20% Control of Diarrheal Disease, and 10% Immunization. The CSP coordinated with the DHO and other actors in support of programs directed towards malaria control and HIV/AIDS. ECD supported the proposed technical interventions through cross-sectoral collaboration, focused on reinforcing positive early child care practices and interaction. Objectives of the KPC Survey The FE KPC survey was conducted in September of 2013. The objectives of the KPC survey were as follows:  Appropriately collect data on the major areas of child and maternal health, including: maternal and newborn care, infant and young child feeding (IYCF), anthropometry, immunization coverage, diarrhea, acute respiratory infection (ARI), fever and malaria, water and sanitation, and hygiene. The survey collected the appropriate data by interviewing 33 mothers of children aged from 0-23m of each age and previous illness category needed to obtain 33 answers to each question in each of 6 SAs. Therefore, this produced a sample size of 198 for each question/indicator. This data will be used to determine progress of all indicators in the Project Design and, combined with qualitative studies, will determine areas of success and challenges, and direct the project in its second half.  Promote capacity building: staff members were trained in the use of survey training methodology in order to facilitate future monitoring and evaluation.  Train staff members in data analysis through the use of LQAS hand tabulation and training sessions following data collection.  Train staff members in making changes to the monitoring and evaluation plan and health information system by relating the indicators used to the projects objectives, outputs, and activities during the data analysis training.  Promote community awareness and acceptance through follow up Community Feedback Sessions.  Partnership building: The survey utilized the concept of partnership building in all phases of training, data collection, and data analysis by involving all key stakeholders in all phases of the survey process. 1.2 Process and Partnership Building MTI had two principal implementing partners in the project, the DHO and Hands to Hearts International. The project was coordinated with other CS stakeholders in country as well. The DHO is the local representative of the MOH, and therefore was a critical partner for long-term strengthening of county health services. An inclusive process was followed to involve all stakeholders in the design, training, implementation, and analysis of the KPC survey. The DHO, HHI, and USAID were all invited to participate in all aspects of the survey via letters and personal contact that outlined the exact process and dates of the training and survey. The DHO was interested all phases of the survey, and helped in areas where they were able, including, getting the word out to all villages via radio and personal visits, and reviewing and discussing results. They were also helpful during the baseline in adapting the survey to the local context and ensuring that the survey was performed in a culturally appropriate manner. Permission to conduct the survey was obtained from the MOH. The survey team was comprised to be as inclusive as possible in order to foster partnership. . Six supervisors and twenty four enumerators were chosen from members of MTI Uganda and the communities of Lira District. Medical Teams International Lira Child Survival Project Final KPC 12 1.3 Methods The overall objective of this FE survey was to estimate the current level of chosen indicators as per the M&E Matrix of specific objectives and intervention logic in the areas of nutrition, immunization, pneumonia case management, and control of diarrheal disease, along with the indicators of the Rapid CATCH. In addition to this, questions and indicators were chosen to measure areas of Early Childhood Development. The survey was designed using LQAS methodology so that the project area may be divided into 6 management areas (Supervision Areas), and each of these areas monitored along with the project area as a whole. This provides the project with MT results for the entire project as well as a breakdown by SA so that actions based on the results may be targeted not only by indicator but also by management area. The KPC FE survey was designed utilizing participatory principles of evaluation in the spirit of partnership and capacity building.2 The Core Team consisted of members of MTI Uganda, with backstopping by the Sr. Advisor in M&E from HQ. The survey team was comprised of 6 supervisors, who were members of MTI Uganda, and the communities of Lira. The enumerators were devised of members of MTI Uganda, , and local survey takers (enumerators) chosen from Lira District. Selection of the team members was based on their skills and their future role in the project and thus provided ownership of the survey and the project itself. The Sr. Advisor in M&E HQ, Africa Health Program Advisor, MTI Uganda Child Survival Program Manager, MTI Monitoring and Evaluation Officer George Aguze, , and MTI Uganda country office staff were involved in the evaluation planning process, including the development of the questionnaires and the recruitment of various team members. The trainings for the KPC Survey, data entry, and data analysis were directed by the Sr. Advisor in M&E HQ. Data analysis was performed utilizing Epi Info by the Sr. Advisor in M&E HQ and shared back with the MTI Uganda Core staff for discussions so that the results could be shared with the country staff and then the communities of Lira District through Community Feedback sessions. Final data analysis was performed in MTI HQ. a. Development of the Questionnaire The evaluation team reviewed the project documents including the detailed M&E Matrix with the project’s goal, objectives, and activities. Key indicators were then chosen based on these parameters in conjunction with the newly revised Rapid CATCH (2008) and KPC 2000+ modules.3 The initial draft questionnaire was developed and shared with the MTI Uganda staff for comments, suggestions, and feedback. Local and regional translators then translated the finalized version of the questionnaire into Luo. Separate translators then translated the questionnaires back into English to ensure that the wording of the questions and answer choices were accurate. Any changes necessary were made at that time. Additional changes important to the local context were made throughout the 2 KPC 2000+ Field Guide, The Child Survival Technical Support Project and CORE, http://www.childsurvival.com/kpc2000/kpc2000.cfm, August 2001. 3 ibid. Medical Teams International Lira Child Survival Project Final KPC 13 training, and the final questionnaire was completed following the fourth day of the training which included a field test of the questionnaire. In addition to the questionnaires, anthropometric measurements consisting of age, gender, and weight of the eligible children aged 0-23 months were taken at the time of the survey from those children randomly chosen for the main questionnaire. Salter hanging scales were used for the weight measurement, which were calibrated prior to each weighing to ensure accuracy. The measurements were taken in order to calculate the child’s weight for age and corresponding Z-scores. b. Sampling design A 30-cluster stratified sampling design was utilized to select the mothers of children aged 0-23 months. A total of thirty clusters were sampled in Lira district, with twelve households with children under the age of 24 months from each cluster being selected for interviews. The overall sample size of 360 households was derived from modifying the simple random sampling design. In a simple random sampling design, a sample size (n) of 96 is derived based on the formula: n=z2 (pq)/d2 With a desired precision of d=0 .5: The estimated proportion p=0.5 (chosen as it requires the largest sample size, thus ensuring that an adequate sample size is chosen), and q=1-p. The desired Confidence Interval=95% leads to a z2 = 1.96 The required sample size (n) = 96. A cluster sample introduces bias in the form of the design effect into the sampling frame, meaning that households in close proximity have more in common than households that are from different areas of the same community, which decreases their possible variation. Therefore, the number of households should be doubled to at least 192. Taking into account that KPC surveys are used to estimate coverage for many different interventions at the same time, which requires looking at sub-samples (such as children less than 6 months old in order to estimate rates of exclusive breastfeeding), it is recommended that the sample size be increased even further, and therefore a sample size of 360 was chosen. Consequently, a 30-cluster stratified sampling design was adapted for the catchment area, with ten households interviewed in each cluster. The true estimate of the survey results include a margin of error as derived using the formula of 95% confidence limits: P= p + z √ (pq/n) P = the actual rate or proportion Medical Teams International Lira Child Survival Project Final KPC 14 p = the survey estimate p = prevalence q = 1-p z = the confidence level (1.96 for a confidence level of 95%) n = sample size Choosing 30 clusters of 12 mothers, for a total sample size of 360, was done to ensure that the 95% confidence interval of each estimate would be narrow enough to distinguish differences in indicators over time in a meaningful way. The number of mothers per cluster was raised slightly, from the average KPC Survey that utilizes 10 mothers, to 12 mothers, so that a certain number of ECD related indicators would have a sample size large enough to be able to effectively compare certain health related outcomes between Aromo sub county, which continued to implement ECD in the second half of the project, and Ogur and Lira sub counties, which did not continue to implement ECD in the second half of the project. c. The Selection Process: The MTI Africa Health Program Advisor, MTI Uganda Child Survival Program Manager, MTI Monitoring and Evaluation Officer, along with other country office staff, met with the DHO and various local leaders and community personnel to determine the respective populations of each of the villages included in the survey sampling frame. Population figures obtained from MOH Lira district statistics for 2011 were used in conjunction with mapping techniques and visualization of the areas by MTI Uganda staff. Each village in the project area was listed randomly, with its population beside it. When the list was complete, the cumulative population of each village was determined by summing the total population of that village with the combined population of all the preceding villages on the list. The total cumulative population of the villages in the catchment area was then divided by 30 (corresponding to the 30 clusters needed) to obtain the sampling interval for that region. A random number was then chosen, with the stipulation being that the number had to be less than or equal to the sampling interval. The cumulative population of each village was then consulted, and the village containing the random number (the village whose cumulative population is equal to or larger than the random number, and whose preceding village had a cumulative population less than the random number) was chosen as cluster number 1. The second cluster was then identified by adding the sampling interval to the random number. The village whose cumulative population contained this number was chosen as the location of cluster number 2. The remaining clusters were then identified by continuing to add the sampling interval to the number that identified the previous cluster. In this way, each cluster was randomly chosen, with proper weight assigned to each village based on its population size. The larger the size of a population of a village, the greater the chance of having one or more clusters assigned to it. Medical Teams International Lira Child Survival Project Final KPC 15 The center of each cluster was determined by allowing the supervisors and enumerators local to these villages enlist the help of the Village Chief or elders to determine the spot where they felt that an equal number of households were on each side. The survey team then chose a random starting direction by spinning a bottle in the physical center of the cluster. The team would then walk in the direction the bottle pointed, and count the number of households in that direction until they reached the end of the households in that cluster. The survey team would return to the center and then choose a random number from a random number table, with the requirement that it had to be less than the number of homes in that direction. They then counted the doorways in the direction the bottle was pointing until they reached the doorway that corresponded to the random number chosen. This was deemed the first house. A protocol was established and written during the training sessions, prior to the survey, that determined which households, children, and thus mothers would be eligible for the survey. If the chosen household contained a child aged 0-23 months that was present and sleeps in the house at night, and a mother that was present and sleeps in the house at night, the survey would be taken at this household. If there was no child under two present, or the mother was not available for the interview, the survey team would move to the household that had the closest door relative to the doorway of the household just eliminated. This procedure was then repeated until 12 successful interviews with eligible mothers were conducted in that cluster. d. Training of Supervisors and Enumerators The training of supervisors and enumerators required 4 days in total. The training curriculum was adapted from the CORE Group’s Knowledge, Practice, and Coverage (KPC) Survey Training Curriculum. The training curriculum was shared with the Core team prior to the training, and the Core team was fully involved with all aspects of the training in order to strengthen the local capacity to conduct future small-sample surveys. Six supervisors were chosen for the FE survey from MTI Uganda staff. The training regimen of the supervisors included: the objective of the evaluation, the sampling process of a 30-cluster sampling frame, proper selection of the clusters, households, children, and mothers, accepted technique and protocol regarding data collection, and an in-depth review of the questionnaires to be used. Measurement of weight was first demonstrated, and then performed, to ensure proper technique. Training methods used included several days of mini lecture followed by discussions, demonstrations, role-play, group work, and pre-testing of the questionnaires. Their responsibilities included supervising twenty-four local enumerators, taking part in every aspect of the data collection, and taking the lead in choosing each cluster’s center, the household chosen, the eligible infant, and then the eligible mother. The training of the enumerators took place with the training of the supervisors and consisted of a several day process that was similar in nature to the training regimen of the supervisors. It included the same several days of mini lecture followed by discussions, demonstrations, role-play, and group work including the measurement of weight, using several children under two for practice. Repeated practice administering the questionnaires and completing each set of 8 questionnaires properly was performed on local volunteer mothers who were not eligible for inclusion in the Medical Teams International Lira Child Survival Project Final KPC 16 actual survey. The survey teams then performed a field test of the questionnaire in nearby Lira district villages that had not been randomly selected for the survey, under the watchful eyes of the Core team. Following the field test the training concluded with a meeting to discuss any issues that arose during the field test and make final changes or adjustments of the questionnaire so that it would be as accurate and context appropriate as possible for the survey. These changes were then made to the questionnaire prior to making copies for the survey. e. Data Collection Each supervisor was assigned four enumerators for a total of six groups, each consisting of four enumerators and one supervisor. A supervisor went to the first chosen house, with two of the enumerators in his/her team, to determine the eligibility of that household, choose the eligible child aged 0-23 months, choose the eligible mother, and weigh the child chosen, using the protocol developed previously. The supervisor would then help determine which questionnaires in the set of 8 questionnaires could be asked at that household (always beginning with the main questionnaire). As the 2 enumerators conducted the rest of the interview with the appropriate questionnaires, the supervisor would then take the other two enumerators in the team to the next eligible household and repeat this process, and would therefore alternate between the two groups of two enumerators. The supervisor would then help each group find the remaining eligible children to complete the set of questionnaires. This allowed the supervisor to take the lead role in determining the eligibility of the household, the weighing process, and immediately checking and correcting any problems with each finished questionnaire while the mother was still available. Each evening the supervisors and Core team met and discussed any issues that arose during the day in order to ensure consistency in the data collection process. The data collection process required 5 days in total, with an average length of interview of approximately 35 minutes. f. Data Analysis A preliminary analysis of the data was performed by calculating frequency distributions of major indicators were prepared using the Epi-info 3.5.3 database, so that MTI Uganda staff had some immediate results to guide programmatic decisions and guide community feedback sessions . These results were then used in discussing the Project design in detail with the MTI Uganda staff in order to increase their capacity in developing project designs and formulating monitoring plans from the objectives, outputs, activities, and indicators chosen. The final analysis was then performed, also using the Epi-info 3.5.3 database. All Rapid CATCH indicators, indicators from the M&E Matrix, several indicators chosen from the KPC 2000+ modules, and indicators dealing with ECD and Health Contacts were presented in the analysis. A 95% confidence interval and a precision of 0.5 were used for each indicator, and 95% confidence limits were calculated for each. Medical Teams International Lira Child Survival Project Final KPC 17 g. Results and Discussion The results are organized into sections that represent each area of the different study indicators. The following chapters represent the program intervention areas, as per the M&E Matrix. Each table contains the indicators measured for that intervention area, with the M&E Matrix indicators denoted with a red box. Also, all Rapid CATCH Indicators are highlighted with orange font. All indicators that have shown a statistically significant improvement at FE have the FE data in red font, and any indicators showing a statistically significant decline are highlighted with a yellow font. Medical Teams International Lira Child Survival Project Final KPC 18 CHAPTER 2 MATERNAL AND NEWBORN CARE Project Matrix Indicators Are Denoted with Red Boxes Rapid CATCH Indicators Highlighted in Orange Font Data at FE that show a SSI are indicated with a red font Data at FE that show a SSD are indicated with a yellow font Every MNC indicator measured, except for modern contraception, was statistically significantly improved (SSI) over the life of this project. All project indicators were SSI, including the following: • Tetanus Toxoid: The percentage of mothers receiving at least two tetanus toxoid immunizations in Lira District was SSI from BL (73%) to FE (89%), despite the fact that there was no SSI at MTE. This increase indicates excellent improvement of antenatal care services and tetanus immunizations being performed by antenatal health care staff, as well as increased utilization of these services. • IPT: At baseline only 35% of mothers took proper anti-malarial medication at least 2 times during pregnancy with their youngest child. By FE this was SSI to 68%, which speaks to the utilization of ANC services as well as improved HF Supervision and drug chain improvements. • Iron While Pregnant: The percentage of mothers who received iron while pregnant was SSI from 59% at BL to 91% at FE, which again shows the excellent increase in utilization of ANC services, supportive supervision at the HF level, and drug chain improvements. • Skilled Birth Attendant: At FE 84% of all childbirths now occur under the supervision of a skilled birth attendant, which is SSI from BL (35%). MTI has accomplished this through focus on increasing the use of health facilities and skilled birth attendants through VHTs, and mothers groups who are being trained in health promotion. This training also included the importance of ensuring that both mother and child receive follow up care including a post partum visit with a trained health professional within 3 days of birth. • Danger Signs: There was a SSI in the rate of mothers who are able to report at least 2 maternal danger signs from 2% at BL to 51% at FE. This is due to extensive health messaging performed throughout the project. • Mother Postnatal Visit: There was a SSI in this indicator, from 16.7% at BL to 28% at FE, also due to increased activity of VHTs and mothers groups. Medical Teams International Lira Child Survival Project Final KPC 20 CHAPTER 3 ANTHROPOMETRY AND INFANT AND YOUNG CHILD NUTRITION 4.1 Anthropometry-Nutritional status of children aged 0-23 months Malnutrition and under-nutrition are major determinants in the increased vulnerability of children to many infectious diseases, including diarrhea, ARI, and febrile illness. Inversely, many infectious diseases may be the cause under-nutrition in children. In addition, the nutritional status of children indirectly reflects the health and nutrition status of mothers. Therefore, the nutritional status of children aged 0-23 months is an important indicator in relation to child survival and community health programs. In this survey, nutritional status was assessed through the anthropometric measurement of weight-for-age in children aged 0-23 months. The weight of each child was taken and combined with the age and gender of the child to calculate the weight-for-age indicator. The indicator is expressed in standard deviations (Z￾score) from the median values of weight-for-age of the CDC reference population from the year 2000. ANTHROPOMETRY Project Matrix Indicators Are Denoted with Red Boxes Rapid CATCH Indicators Highlighted in Orange Font Data at FE that show a SSI are indicated with a red font Data at FE that show a SSD are indicated with a yellow font • Undernutrition: The % of children with a Z-score of <-2SD at FE is 20%, not significantly decreased (SSI) from the BL rate of 28%. However, the IYCF indicator (see the next section on IYCF) was SSI from only 23% at BL to 49% at FE. 4.2 Infant and Young Child Feeding (IYCF)-Nutrition Health promotion and education regarding nutrition and breastfeeding was one of the interventions of MTI Uganda. The project worked to improve the nutritional status of children through the promotion of correct breastfeeding and complimentary feeding practices, including immediate breastfeeding following childbirth, exclusive breastfeeding of children under 6 months of age, and the introduction of digestible and nutritional complimentary foods in children 6 months and greater. BREAST FEEDING AND INFANT AND YOUNG CHILD FEEDING (IYCF) Project Matrix Indicators Are Denoted with Red Boxes Rapid CATCH Indicators Highlighted in Orange Font Data at FE that show a SSI are indicated with a red font Data at FE that show a SSD are indicated with a yellow font • Exclusive Breastfeeding: The % of children aged 0-5m who were exclusively breastfed during the 24 hours prior to the survey was SSI at FE (88%) from MTE (68%), but not from BL (73.6). This could be due to the fact that the Confidence Intervals (CIs) are larger due to the lower sample size of mothers with children aged 0-5m. • Immediate Breastfeeding: There was a very large and SSI of Immediate BF practices for newborns, which was only 29% at BL and 23% at MTE. By FE this was increased to 79%. • No Prelacteal Feeding: there was also a very large and SSI in the % of mothers who do not give their infant prelacteal feeds, rising from 47% at BL to 87% at FE. • Complimentary feeding: Complimentary feeding was relatively high at BL, at 70%, and was also high but not SSI at FE (86%). This is not significantly higher than at BL because the sample size of mothers of children aged 6-9m was small (n=53) due to 30 cluster methodology. However, a complimentary feeding rate of 86% is quite high. • Continued BF in children to 23m: Continued BF is statistically unchanged in all categories, which were already high Medical Teams International Lira Child Survival Project Final KPC 22 IYCF INDICATORS Project Matrix Indicators Are Denoted with Red Boxes Rapid CATCH Indicators Highlighted in Orange Font Data at FE that show a SSI are indicated with a red font Data at FE that show a SSD are indicated with a yellow font • IYCF Indicator: There has been a significant improvement in this indicator, increasing from 23% at BL to 49% at FE. There was a SSI in proper feeding frequency in breastfed children (31% at BL to 66% at FE) which was the driving factor in this improvement. . The results reveal that feeding practices for children aged 6-23m have improved significantly, particularly with the immediate and exclusive BF of newborns and children aged 0-5m, respectively, as has proper infant and young child feeding. • Vitamin A: The % of children receiving Vitamin A in the past 6m at FE (85%) was SSI from BL (70%). Medical Teams International Lira Child Survival Project Final KPC 23 CHAPTER 4 PNEUMONIA CASE MANAGEMENT Project Matrix Indicators Are Denoted with Red Boxes Rapid CATCH Indicators Highlighted in Orange Font Data at FE that show a SSI are indicated with a red font Data at FE that show a SSD are indicated with a yellow font Acute Respiratory Infection is recognized as one of the major public health problems in Uganda. Most children were given symptom relieving not curative medicines in the form of cough syrups or country medicines. Objectives of the project included the recognition of the danger signs of pneumonia, improved access to quality care, and the promotion of optimal and timely health seeking behaviors among mothers/caretakers. The present evaluation estimates the prevalence of ARI among children aged 0-23 months, the mother’s knowledge concerning ARI, the mother’s management of ARI, and timely health seeking behaviors in relation to ARI. • Rate of ARI: The rate of ARI was SSD from 49% at BL to 25% at FE, indicating improved hygiene (see the hygiene section, below). • Health Seeking for ARI: The % of children with ARI who were taken to an appropriate health provider increased significantly over the first half of the project (58% t BL to 86% at MT), but there was not a SSI from BL to FE (74%). The sample sizes were smaller at FE (n=89) due to lower rates of pneumonia, which is good, but this increased the CI and therefore the result at FE was not SSI. • Antibiotics for ARI: The % of children aged 0-23 months with ARI who were treated with an antibiotic also increased significantly form BL to MT (35% to 64%), and this remained SSI at FE (69%). • Timely care seeking for ARI: There was a SSI in timely care seeking behaviors for ARI, rising from 42% at BL to 62% at FE. This corresponds with a SSI in timely care seeking for fever/malaria as well (see the Malaria section for details). CHAPTER 5 CONTROL OF DIARRHEA Project Matrix Indicators Are Denoted with Red Boxes Rapid CATCH Indicators Highlighted in Orange Font Data at FE that show a SSI are indicated with a red font Data at FE that show a SSD are indicated with a yellow font Diarrhea was a common cause of childhood morbidity and mortality in Lira District and, and Uganda as a whole. It is well proven that diarrhea is one of the major contributors to malnutrition in children. MTI Uganda made a high priority the reduction of childhood diarrhea prevalence and morbidity through preventive and curative measures. This diarrhea management initiative was an intervention aimed at raising the awareness of mothers/caretakers about the necessary steps required to both prevent and treat diarrheal disease. Interventions aimed at the prevention of diarrhea included instruction in the importance of appropriate hand washing behaviors as well as the promotion of the use of latrines. Diarrhea case management at the household level included proper feeding and fluid management during diarrhea episodes, including the proper preparation and use of Oral Rehydration Salts (ORS). • Incidence of Diarrhea: The rate of diarrhea was SSD from 38% at BL to 24% at FE, which highlights the improvements in hygiene and the treatment of water in the household (see these sections for greater detail). • ORS Use: At MTE there was no SSI (47% at BL to 54% at MTE), but by FE a SSI was shown, with a rate of 61%. This SSI from BL is due to health messaging and availability and proper treatment at the HFs. • Zinc: Zinc usage was SSI from only 1% at BL to 18% at FE, when there was no change at MTE, so this shows increased effect of health messaging, IMCI, HF supervision, and drug chain management in the last half of the project. • Increased Fluids with Diarrhea: The rate of mothers who offer their child more to drink with diarrhea was SSI from 9% at BL to 27% at FE. However, because the amount of food offered was not SSI, the overall indicator of increased fluids and the same or More Food offered was not SSI (2% at BL and 5% at FE). HYGIENE Project Matrix Indicators Are Denoted with Red Boxes Rapid CATCH Indicators Highlighted in Orange Font Data at FE that show a SSI are indicated with a red font Data at FE that show a SSD are indicated with a yellow font • Soap at the Place for Hand washing: This indicator, despite being high at BL (85%), was still SSI through the project’s health messaging, to 96% at FE. • Appropriate hand washing with soap: There has been a significant increase from 45% at BL to 88% at FE, which effectively contributed to a lower rate of diarrhea at FE (see the diarrhea section, above). Medical Teams International Lira Child Survival Project Final KPC 26 WATER AND SANITATION Project Matrix Indicators Are Denoted with Red Boxes Rapid CATCH Indicators Highlighted in Orange Font Data at FE that show a SSI are indicated with a red font Data at FE that show a SSD are indicated with a yellow font • Point of Use water treatment: There had been no significant change in the rate at BL (11%) to MTE (11%). Investigation following the midterm revealed that most households did not feel that they needed to treat water because they had access to an improved water source (91.0% of households in Lira District have access to an improved water source). However, health messaging around the importance of treating water was increased and at FE there was a SSI increase at FE to 41%. Medical Teams International Lira Child Survival Project Final KPC 27 CHAPTER 6 IMMUNIZATION Project Matrix Indicators Are Denoted with Red Boxes Rapid CATCH Indicators Highlighted in Orange Font Data at FE that show a SSI are indicated with a red font Data at FE that show a SSD are indicated with a yellow font In this survey Expanded Program of Immunization (EPI) Access is measured by the percentage of children aged 12-23 months who received a DPT1 vaccination before the age of 12 months as verified by a vaccination card, and EPI Coverage is measured by the percentage of children aged 12-23 months who received a BCG, DPT3, OPV3, and measles vaccination before the age of 12 months verified by an immunization card, meaning that they received full vaccination coverage. Added to these measurements is the drop-out rate which measures the number of children aged 12-23 months who received a DPT1 vaccination by card verification or mothers recall but who were not continued in a vaccination program and therefore did not receive a DPT3 vaccination. These indicators provide an excellent picture of immunization services with regard to access, coverage, and completion of immunizations. Since MTE four additional health facilities were opened in the project area and immunization outreach was expanded. As a result immunization services access, performance and coverage has improved. • Vaccination Card: The % of children with a vaccination card was SSI from only 51%at BL to 85% at FE due to health messaging and IMCI in the HFs • Vaccinations, Card Verified, by 12m: Every indicator regarding the immunization of children by 12m of age was SSI, because all health units were supplied with cards, fridges and vaccines, and mothers now have cards. Mother leader groups were trained to teach mothers not to lose cards, and were going house to house and bringing children to HFs for immunization. Also, because more cards were present at survey, a higher % of children could be verified as having been vaccinated. This resulted in a DPT1 vaccination rate of 88%, a DPT3 vaccination rate of 79%, and a measles immunization rate of 62% at FE. This measles rate, which was 62% at FE, was the limiting factor, which culminated in a full vaccination rate of 56% at FE. • Again, all card verified vaccination rates were SSI by FE, with a DPT1 rate of 91% and a DPT3 rate of 85% at FE. The limiting factor was measles vaccination, which was SSI from 34% at BL to 73% at FE. This excellent SSI in immunization rates shows an increased availability of vaccines and IMCI at the HFs as well as outreach and access to immunization days. • Vitamin A: The Vitamin A rate was SSI from 11% at BL to 66% at FE, also showing increased drug availability, IMCI, immunization outreach and immunization access through immunization days. . • DPT1: There was a SSI in DPT1 immunization, both card verified and card or mother’s recall. DPT1 immunization by card or mother recall was SSI from 87% at BL, which was already quite high, to 97%at FE, which is almost universal coverage. • Measles vaccination (card verified or mother’s recall): There has been no significant change in measles vaccination coverage (82% at FE), as coverage was already quite high (77% at BL). Medical Teams International Lira Child Survival Project Final KPC 29 CHAPTER 7 EARLY CHILDHOOD DEVELOPMENT Project Matrix Indicators Are Denoted with Red Boxes Rapid CATCH Indicators Highlighted in Orange Font Data at FE that show a SSI are indicated with a red font Data at FE that show a SSD are indicated with a yellow font • All ECD indicators that pertain to ECD behaviors at least 2 times per week, except for 1, were SSI over the course of the project, including: physical interaction (playing, massaging, etc.) cognitive stimulation, teaching, linguistic learning, and singing/music. The indicator “The percentage of mothers aged children 0 – 23 months who massaged, hugged, or cuddled with their child at least 2 times per week” increased but not significantly. This indicates excellent saturation of ECD messaging to mothers as well as excellent adaption of ECD behaviors by the mothers in the project area. • Cognitive Stimulation: There was a significant increase from 38% at BL to 76% at FE, with all SAs meeting the DR. • Linguistic development: There has been a significant increase from 23% at BL to 64% at FE, indicating a substantial increase in mothers teaching children linguistics. BELIEFS REGARDING ECD • Likewise, all beliefs regarding the efficacy of the fundamental principles behind ECD were SSI, except for one which was already above 90% before ECD messaging began. This again relates to excellent saturation and uptake of ECD messaging. • The % mothers who believe that they should provide things for their child with which to play was SSI from 72% at BL to 98% at FE, and those who feel that playing helps the child learn was SSI from 78% at BL to 98% at FE. Also, the rate of mothers who believe playing with toys helps the child prepare for school was SSI from 88% at BL to 96% at FE. Medical Teams International Lira Child Survival Project Final KPC 31 ECD ACTIVITIES • Increased verbal stimulation: Verbal Stimulation through singing was SSI from 35% at BL to 79% at FE. • Stimulation while feeding: There has been a SSI in all of the ECD feeding indicators, except for looking into the child’s eyes while feeding, which was already at 87%. Talking to the child while feeding was SSI from 58% at BL to 84% at FE. Medical Teams International Lira Child Survival Project Final KPC 32 ECD INDICATORS BY SUBCOUNTY • There are no statistically significant differences in the uptake of ECD between the counties, and therefore, likewise, there are no statistically significant differences in health outcomes measured due to ECD. Medical Teams International Lira Child Survival Project Final KPC 33 CHAPTER 8 MALARIA AND THE MANAGEMENT OF FEBRILE ILLNESS Lira District is a malaria endemic area and the incidence of fever in children less than 24 months is quite high in these areas. Malaria is highly endemic in Uganda (90-98% P. falciparum) and is the leading cause of morbidity and mortality nearly country-wide.4 In Lira District, malaria accounts for 34% of the disease burden. The present evaluation estimates the prevalence of febrile illness (a proxy for malaria) among children aged 0-23 months, the mother’s knowledge concerning malaria and health seeking behaviors in relation to malaria. Project Matrix Indicators Are Denoted with Red Boxes Rapid CATCH Indicators Highlighted in Orange Font Data at FE that show a SSI are indicated with a red font Data at FE that show a SSD are indicated with a yellow font • ITN in home: There has been a SSD in the rate of households that have an ITN, from 82% at BL to on 58% at FE. This correlates with the findings of the health facility assessment at final that show a lack of available nets for distribution in the entire project area. • ITN use (mothers and Children under 2: There has been a SSI in the use of bed nets in mothers of children under 2 since BL (41% at BL to 57% at FE), which is commendable considering there is a decrease in the availability of ITNs. However, the rate of children under 2 sleeping under a bed net was not statistically changed from 51% at BL to 58% at FE. • Treatment of malaria: There has been a significant increase in the proper treatment of malaria (25% at BL to 52% at FE). This was not an intervention area of this project, so this SSI is 4 Uganda Malaria Control Strategic Plan: 2005-6 – 2009-10. indicative of the increased awareness of mothers of the importance of immediate care seeking through our other health messaging, and our efforts at the HF level that have caused an increased effectiveness of the HFs in IMCI, supervision, and increased availability of medications at the HF. Medical Teams International Lira Child Survival Project Final KPC 35 CHAPTER 9 HEALTH CONTACTS Project Matrix Indicators Are Denoted with Red Boxes Rapid CATCH Indicators Highlighted in Orange Font Data at FE that show a SSI are indicated with a red font Data at FE that show a SSD are indicated with a yellow font CONTACT WITH TRAINED HEALTH PERSONNEL • Due to the projects ability to strengthen community capacity through VHTs, Peer Educators, PDCs, Mother Leader Groups, and Health sub-districts, the rate of mothers having contact with trained health personnel in the previous month was SSI from only 29% at BL to 92% at FE. This is a noteworthy achievement, and the breakdown of this contact may be seen in the table, above. Contact with VHTs was SSI the greatest, with a rate of 81% by FE, up from only 10% at BL. HEALTH MEETNG ATTENDANCE • At FE mothers were attending a monthly meeting of some type on order to discuss their health or the health of their baby at a rate of 85%. This is a very large SSI from the baseline rate of 23%. Meeting facilitated by VHTs, Mother Group Leaders and Peer Educators allows this accessibility and created the demand for these groups, and this is another sustainable change in the community. The rate of attendance of each type of group is indicated in the table, above, with the largest SSI noted in Mother’s Groups (63% at FE), which is excellent. There is a SSI in the rate of attendance in each type of specific group. Medical Teams International Lira Child Survival Project Final KPC 37 HEALTH MESSAGING BY SOURCE • There is high saturation of health messaging from the VHTs, 80% at FE, and also Health Educators, 61% at FE. This excellent saturation of the community with health messaging from the VHTs and health educators was a major factor in the many health indicators that showed a SSI over the course of the project. • There is a very high saturation of health messaging through the radio in all villages of the project area, which is shown by the high rate of mothers who report hearing health messaging from the radio, 82% at FE. Medical Teams International Lira Child Survival Project Final KPC 38 TOPICS OF HEALTH MESSAGING • The high rate of mothers hearing each messages on key health topic again reveals the excellent saturation of this project area with health messages through community volunteers and health staff. Improvement in many key health practices can be attributed to the solid coverage of health messages. Each topic is broken down in the table, above, and all rates are quite high. These were not measured at BL, so no determination of SSI was possible. Medical Teams International Lira Child Survival Project Final KPC 39 CHAPTER 10 SUMMARY The goal of the Lira District Child Survival Project was to reduce child morbidity and mortality in Uganda. Objectives were: 1) Communities assume responsibility for their own health through strengthening community capacity (VHTs, Parish Development Councils, and Health Sub-districts); 2) Improved health (C-IMCI) and child care (ECD) behaviors among mothers of children <5 years; 3) Improved quality of health facility services through strengthened IMCI and MNC capacity; and 4) Strengthened institutional capacity of MTI and DHO to implement effective and efficient child survival activities. These objectives support MoH goals and strategies as well as those of USAID Uganda. MTI used a two-pronged strategy that included promoting behavior change and community mobilization to take appropriate responsibility for health and building DHO capacity to provide sustainable, quality service delivery at the facility and community levels. The level of effort by intervention for this Child Survival Project (CSP) was as follows: 1. 25% MNC 2. 25% PCM 3. 20% IYCF 4. 20% CDD 5. 10% EPI. MNC 11 of the 12 MNC indicators, including all 6 of the indicators in the project Logframe and all the Rapid CATCH indicators for MNC, were SSI through the life of the project. In fact, the only MNC indicator not SSI is the rate of use of modern contraception. The rate of mothers receiving at least 2 TT immunizations was already relatively high at BL (73%), but still the project was able to sow a SSI in this indicator to 89% at FE. Additionally the rate of mothers taking proper IPT medication while pregnant was SSI from only 35% at BL to 68% at FE, and the rate of mothers who received iron while pregnant was SSI from 59% at BL to 91% at FE. These statistically significant increases highlight the gains made in health messaging regarding the importance of ANC, HF support and supervision, and drug chain improvements. Also, regarding health messaging, there was a statistically significant improvement in the rate of mothers who are able to correctly identify at least 2 neonatal danger signs for which they should seek immediate care, from 2% at BL to 51% at FE. This is due to extensive health messaging performed throughout the project. Also, at FE 84% of all childbirths now occur under the supervision of a skilled birth attendant, which is SSI from BL (35%). MTI accomplished this through focus on increasing the use of health facilities and skilled birth attendants through VHTs, and mothers groups who trained in health promotion. This training also included the importance of ensuring that both mother and child receive follow up care including a post partum visit with a trained health professional within 3 days of birth, which is why there was a SSI in this the rates of both mothers and children who received a postnatal visit from a trained HW within 2 days after birth. Mothers received a visit at a rate of 50% at FE (up from only 17% at BL), and children received a postnatal visit at a rate of 41% at FE (up from 16% at BL). Anthropometry AND IYCF The nutritional status of children aged 0-23 months was of major concern in Lira District. The overall rate of under-nutrition at BL was 27.7%, with 17.0% moderately underweight and 10.7% severely underweight. This is similar to the results found by a DHS study in 2006 which revealed the overall rate of under-nutrition to be 21.8% in the Northern region of Uganda. At FE, The percentage of children with a Z-score of <-2SD at FE is 20%, not significantly decreased from the BL rate of 28%. However, the IYCF indicator relating to the proper feeding frequency and food diversity for children aged 6-23m was SSI from only 23% at BL to 49% at FE The percentage of children aged 0-5m who were exclusively breastfed during the 24 hours prior to the survey, while already at 74% at BL, was SSI to 88% at FE. Immediate BF practices for newborns was dramatically and SSI from 29% at BL to 79% at FE. There was also a SSI in the rate of mothers who did not give their child prelacteal feeds (from 47% at BL to 87% at FE), so the summary indicator of immediate and exclusive breastfeeding was SSI from only 16% at BL to 71% at FE. The complementary feeding rate for 6-9m old children was already high at BL (70%), and this was not SSI, but was 88% at FE. This is not significantly higher than at BL because the sample size of mothers of children aged 6-9m was small (n=53) due to 30 cluster methodology. However, a complimentary feeding rate of 88% is quite high. There has been a significant improvement in the composite IYCF indicator, which determines the rate at which children aged 6-23m are fed the proper frequency and food group variety in their diet dependant on their age and breastfeeding status. There was a SSI in this indicator from 23% at BL to 49% at FE. There was a SSI in proper feeding frequency in breastfed children (31% at BL to 66% at FE) which was the driving factor behind this improvement, and was stressed in health messaging throughout the project. Also, the % of children receiving Vitamin A in the past 6m at FE (85%) was SSI from BL (70%). This is excellent, particularly since the indicator was SSD at midterm. Efforts were made by the project to work with the facilities and mothers to ensure both a high supply and demand of Vitamin A. Pneumonia Case Management Pneumonia is the fifth-highest contributor to Lira District’s burden of disease, but is the primary cause of child mortality.5 Objectives of the project were the recognition of the danger signs of pneumonia, improving the access to quality care, and promoting optimal and timely health seeking behavior among mothers/caretakers. The % of children age 0- 23 months with ARI who were treated with an antibiotic increased significantly form BL to FE (35% at BL to 69% at FE).This again speaks to the project interventions of improving care seeking behaviors, IMCI and supervision at the HFs, and drug supply chain management. The % of children with ARI who were taken to an appropriate health provider increased significantly over the first half of the project (58% at BL to 86% at 5 Lira District Health Office Annual Report, July 2008 Medical Teams International Lira Child Survival Project Final KPC 41 MT, but the FE result of 74% was not a SSI from BL. Control of Diarrheal Disease MTI Uganda made a high priority the reduction of childhood diarrhea prevalence and morbidity through preventive and curative measures. This diarrhea management initiative was an intervention aimed at raising the awareness of mothers/caretakers about the necessary steps required to both prevent and treat diarrheal disease. Interventions aimed at the prevention of diarrhea included instruction in the importance of appropriate hand washing behaviors as well as the promotion of the use of sanitary latrines. Diarrhea case management at the household level includes proper feeding and fluid management during diarrhea episodes, including the proper preparation and use of Oral Rehydration Salts (ORS). There was a SSD in the prevalence in the rate of diarrhea at FE (24.3%) from BL (36%), which highlights the gains made in hygiene and water and sanitation (see the hygiene and water and sanitation paragraphs below.) There has been a SSI in the % of children who received ORS when suffering from diarrhea since baseline (47% at BL to 62% at FE), which is impressive as there was no SSI at MT (54%). This reveals that continuing efforts in health messaging created results over time. In addition, zinc usage in children to treat diarrhea, which was almost non-existent at BL (2.6%), was SSI to 18% by FE. This again highlights gains in both drug chain management and supervision at the HFs as well as IMCI. Hygiene was improved, with the percentage of mothers who have soap at the place for hand washing that washed their hands with soap at least 2 of the appropriate times during a 24 hour recall period (after defecation and at one of the following: before food preparation, before feeding children, or after attending to a child who has defecated) SSI from 54% at BL to 88% at FE. Also, the % of households with soap present at the place for washing was SSI from an already high 85% at BL to 96% at FE. Likewise, the rate of safe feces disposal was SSI from 13% at BL to 61% at FE. These results are due to excellent saturation of health messaging by the project, which resulted in a SS lower rate of diarrhea and better treatment of diarrhea cases that did occur. With regards to Water and Sanitation, the % of households of children 0-23m that treat water effectively was SSI from28% to 38% from BL to FE. Expanded Program of Immunization Immunization coverage improved in the project area, with 9 of the 11 measured EPI indicators statistically significantly improved. This can be attributed to the opening of four additional health facilities, expansion of immunization outreach and the CSP’s efforts to support immunization services. EPI Coverage is measured by the percentage of children aged 12-23 months who received a BCG, DPT3, OPV3, and measles vaccination before the age of 12 months verified by an immunization card, meaning that they received full vaccination coverage. The project showed a significant increase in EPI coverage from BL (16% at BL to 56% at FE). In addition, there has been a SSI in the coverage of DPT1, DPT3, and measles immunization, both card verified and card Medical Teams International Lira Child Survival Project Final KPC 42 verified to be by 12m of age. Card verified DPT1 was raised from 47% at BL to 91% at FE, and card verified DPT3 was raised from 35% at BL to 85% at FE. Also, card verified measles vaccination was raised from 11% at BL to 66% at FE. These are excellent achievements that verify excellent gains in immunization at the HFs and through outreach, as well as IMCI, HF supervision, and drug chain management at the HF level. While measles coverage was SSI by both card verified and card verified by 12m, it was not SSI by card or mothers recall (82% at FE) as coverage was already quite high (77% at BL). Early Childhood Development MTI incorporated ECD activities into its CS project to enhance the impact and sustainability of technical interventions. Research confirms that child survival is positively linked to ECD – including health, physical, social/emotional, and language/cognitive domains. Six of the 7 ECD indicators that pertain to ECD behaviors at least 2 times per week were SSI over the course of the project, including: physical interaction (playing, massaging) cognitive stimulation, teaching, linguistic learning, and singing/music. This indicates excellent saturation of ECD messaging to mothers as well as excellent adoption of ECD behaviors by the mothers in the project area. There has been a SSI from BL of mothers of children aged 0-23 months who report playing games with their child in which they have their child identify their body parts, imitate actions, pretend play, or name objects (38% at BL to 76% at FE). These are important builders of cognitive, motor, or linguistic functions. In addition, there has been a significant increase from BL of mothers report engaging their children in linguistic learning activities such as telling their child stories, singing them songs, or naming objects for them at least twice weekly (23% at BL to 64% at FE). Beliefs regarding the importance and efficacy of several ECD components were also SSI. All of the indicators measuring beliefs regarding the fundamental principles behind ECD were SSI, except for one which was already above 90% before ECD messaging began. The % mothers who believe that they should provide things for their child with which to play was raised from 72% at BL to 98% at FE, and those who feel that playing helps the child learn was SSI from 78% at BL to 98% at FE. Also, the % of mothers who believe playing with toys helps the child prepare for school was SSI from 88% at BL to 96% at FE. This again relates to excellent saturation and uptake of ECD messaging. All 3 indicators measuring interaction with their child during feeding were SSI. There has been a SSI from BL in the rate of mothers who provide report that they talk or sing to the child while feeding the child (58% at BL to 84% at FE). Malaria The prevalence of fever in children aged 0-23 months in Lira District was found to be quite high in the baseline KPC Survey at 74.8%. Malaria is highly endemic in Uganda (90-98% P. falciparum) and is the leading cause of morbidity and mortality nearly country-wide.6 In Lira District, malaria accounted for 34% of the disease burden at the start of the project. Over the course of the project, the rate of malaria in children under 2 6 Uganda Malaria Control Strategic Plan: 2005-6 – 2009-10. Medical Teams International Lira Child Survival Project Final KPC 43 was SSD from 75% at BL to 58% at FE. There has been a SSI in the use of bed nets in mothers of children under 2 since BL (41% at BL to 57% at FE), which is commendable considering there is a decrease in the availability of ITNs. However, the rate of children under 2 sleeping under a bed net was not statistically changed from 51% at BL to 58% at FE. The rate of households that contained at least 1 ITN was SSD from 82% at BL to 58% at FE. This correlates with the findings of the health facility assessment at final that should a lack of available nets for distribution in the entire project area. Of the children who had a fever in the 2 weeks prior to the survey, at BL only 25% were treated with an antimalarial drug within 24 hours of the start of the fever. There has been a significant increase in the proper treatment of malaria to 52% at FE. The driving factor behind this increase is the fact that the rate of care seeking (mothers who sought care for their child with fever within24 hours) was SSI from only 22% at BL to 83% at FE. Health Contacts One of the greatest successes of this project was the very significant increase in contact rates between mothers in the project area and qualified health workers or accurate health information regarding MNC and mother and child health. Due to the project’s strengthening of community capacity through VHTs, PDCs, Peer Educators, Mother Leader Groups, and Health sub-districts, the rate of mothers having contact with trained health personnel in the previous month was SSI from only 29% at BL to 92% at FE. Contact with VHTs led the SSI, with a rate of 81% by FE, up from only 10% at BL. Also, mothers now (at FE) attend a monthly meeting of some type on order to discuss their health or the health of their baby at a rate of 85%. This is a very large SSI from the baseline rate of 23%. The structure and function of these groups, supported by the CSP, allows this accessibility and created the demand for these groups, and this is another sustainable change in the community. The rate of attendance of each type of group in indicated in the table, above, with the largest rates noted in Mother’s Groups (63%), which is excellent. In addition, the saturation of the project area with health messaging on important MNC topics was excellent throughout the project. There is a high saturation of health messaging in all villages from the VHTs, with a rate of 80% at FE, and also Health Educators, with a rate of 61% at FE, due to the project initiatives utilizing VHTs and Health educators to reach mothers. Good saturation of the community with VHTs and health educators is evidenced by the many improved health indicators indicating changes in health behaviors in this project. There is a very high saturation of health messaging through the radio in all villages of the project area, which is shown by the high rate of mothers who report hearing health messaging from the radio, with a rate of 82% at FE. Finally, there is a very high rate of mothers hearing messages on each key health topic (hand washing, treating water, danger signs/care seeking, diarrhea care, and SBA) again reveals the excellent saturation of this project area with health messages through our project sources, which related to the high SSI achieved with health outcomes. Medical Teams International Lira Child Survival Project Final KPC 44 In conclusion, by working in partnership with the DHO, MTI Uganda implemented a project that improved the health of village communities in Lira District through building DHO capacity to provide sustainable, quality service delivery at the facility and community levels, and through promoting behavior change and community mobilization to take appropriate responsibility for health. This was accomplished through a combination of interventions, including providing supplementary training, supervision, and follow-up coaching of VHTs. The results of this FE KPC Survey reveal the many sustainable successes that were accomplished in Lira District, as well as a few continuing challenges of this project area that may be used in future projects as lessons learned. Medical Teams International Lira Child Survival Project Final KPC 45 ANNEX VII. COMMUNITY HEALTH WORKER TRAINING MATRIX Project Area (Name of District Or Community) Type of CHW Official Government CHW or Grantee￾Developed Cadre Paid or Volunteer Number Trained Over Life of Project Focus of Training Number of Days Male Female Health Centres Health Workers Government staff Paid 8 26 IMCI 11 days Health Centres Health workers Government Paid 10 30 Refresher on IMCI 3 days Health Centres Health Workers Government staff Paid 3 16 ToT in ECD 5 days Health Centres Health Workers Government staff Paid 8 38 Maternal Newborn Care 3 days Health Centre Health workers Government Paid 16 29 Refresher on MNC 3 day Community (Lira, Aromo and Ogur sub￾counties) and Health Centres Health Workers and Community Leaders Government staff Paid/ Volunteer 5 Community Leaders 21 Health Workers 7 Community Leaders 7 Health Workers Referral Systems 2 days Health Centres Health Workers Government staff Paid/ Volunteer 17 23 Joint Support Supervision 2 days Health workers/commu nity vaccinators Health workers/communi ty vaccinators Government staff Paid/ Volunteer 14 Community Vaccinators 11 Health Workers 10 Community Vaccinators 42 Health Workers Refreshed on EPI 1 Day Community (Lira, Aromo and Ogur sub￾counties) VHTs Government Volunteer 420 140 ICCM 5 days Community (Lira, Aromo and Ogur sub￾counties) VHTs Government Volunteers 420 140 Refresher on ICCM 2 days Community (Lira, Aromo and Ogur sub￾counties) VHTs and Peer Educators Government CHWs and Grantee developed Cadre(CSP) Volunteer 150 150 ECD 3 days Community Peer educators Grantee developed volunteers 64 88 ECD 5 Days Final Evaluation Report: Lira District Child Survival Project in Uganda December 2013 Final Evaluation - Annexes (Aromo sub￾county) Cadre(CSP) Community (Aromo sub￾county) Peer educators Grantee developed Cadre(CSP) volunteers 114 138 Refresher training on ECD 2 Days Community (Lira, Aromo and Ogur sub￾counties) Mothers Group Volunteers Grantee developed Cadre(CSP) Volunteer 00 1428 MNC and IYCF 5 days Community (Lira, Aromo and Ogur sub￾counties) VHTs Government CHWs Volunteer 420 140 • Referral Systems Reporting and Record Keeping • Maternal danger signs • Early health care seeking behavior • Immunizatio n • Health Hygiene and sanitation • The importance of home visits • Emergency Transport • Nutrition Half-day trainings provided during quarterly meetings Community (Lira, Aromo and Ogur sub￾counties) Mothers Group Volunteers Grantee developed Cadre(CSP) Volunteer 00 1248 MNC and IYCF Half-day trainings provided during quarterly meetings Final Evaluation Report: Lira District Child Survival Project in Uganda December 2013 Final Evaluation - Annexes ANNEX VIII. EVALUATION SCOPE OF WORK Terms of Reference for Final Evaluator External Consultant for the Lira Child Survival Project in Lira, Uganda September 4 – December 19, 2013 I. Introduction Medical Teams International will hire Sue Leonard as an independent consultant to conduct a final performance evaluation (FE) for the Lira Child Survival Project funded by USAID’s Child Survival and Health Grants Program (CSHGP) Cooperative Agreement No. GSH-A-00-09-00012, September 30, 2009 – September 30, 2013. USAID’s CSHGP supports community-oriented projects implemented by U.S. private voluntary organizations (PVOs) and nongovernmental organizations (NGOs) and their local partners. The purpose of this program is to contribute to sustained improvements in child survival and health outcomes by supporting the innovations of PVOs/NGOs and their in-country partners in reaching vulnerable populations. This document describes the Final Evaluator’s SOW for the Lira Child Survival Project FE. II. Background The project overall goal is to reduce child morbidity and mortality in Uganda, in support of Uganda Ministry of Health goals, objectives and strategies. The project has four key results/objectives: Objective No.1: Communities assume responsibility for their own health through strengthened community capacity (Village Health Teams, Parish Development Councils, and Health Sub-districts). Objective No.2: Improved health (Community-based Integrated Management of Childhood Illness) and child care (Early Childhood Development) behaviors among mothers of children under five years of age. Objective No.3: Improved quality of health facility services through strengthened Integrated Management of Childhood Illness and Maternal and Newborn Care capacity. Objective No.4: Strengthened institutional capacity of MTI and the Lira District Health Office to implement effective and efficient child survival activities. The Lira CSP has five child survival interventions (pneumonia, diarrhea, nutrition, immunizations and maternal newborn care) along with a sixth intervention Early Childhood Development (ECD) which has been implemented with match funding. The Final Evaluation Report: Lira District Child Survival Project in Uganda December 2013 Final Evaluation - Annexes CSP postulated that the addition of ECD would enhance child survival impact as documented in the Lancet “Child Development in Developing Countries” meta-analysis (2007). III. Project Population Beneficiaries* Total Total Population 125,000 Total Neonates 5625 live births per year Infants aged 0–11 Months 5,375 Children aged <5 Years 25,000 Women of Reproductive Age (15–49 years) 25,250 Total Beneficiaries 50,250 Expected Pregnancies 6468 expected pregnancies per year Community Health Workers or Volunteers (CHWs), Disaggregated by Sex 140 women and 420 men Village Health Team volunteers 150 female Peer Educators for Early Childhood Development 1428 Mother Group volunteers Health Facilities (Hospital to Sub Health Post) 4 Health Center II 3 Health Center III 1 Health Center IX Community-Based Structures (e.g., Village Development Committees [VDCs]) 8 Health Unit Management Committees and 29 Community Emergency Transport Committees *Source: National census in 2002, projections for 2013, 3.4% growth rate. IV. Partners The key partner is the Lira District Health Office (DHO) team and Ministry of Health (MoH) staff at eight health facilities. MTI has worked with the MoH to train Maternal Child Health staff, Village Health Teams, and Mother Leader Groups and provide joint follow-up supportive supervision. MTI has supported outreach and immunization activities. Hands to Hearts International (HHI), an international NGO based in Portland, Oregon, has been a consulting partner for the ECD component of the project. HHI worked with MTI and the DHO to adapt existing curriculum to the local context and provided ECD TOT trainings for MTI and health facility staff. On-going technical assistance focused on monitoring and evaluation. V. Key Activities Key activities for Objective No. 1 include training and follow-up to strengthen the capacity of Village Health Teams in integrated Community Case Management (iCCM); establishing a structured referral system between Village Health Teams (VHTs) and local health facilities; and facilitating collaboration between VHTs and Parish Development Committees and Health Unit Management Committees to benefit maternal and child health. Final Evaluation Report: Lira District Child Survival Project in Uganda December 2013 Final Evaluation - Annexes To achieve Objective No. 2, Village Health Teams and Mother Groups promote social and behavior change through community meetings and home visits. Under Objective 2, MTI also introduced early child development (ECD) for children under two with an operations research plan to assess whether the ECD training would enhance child survival outcomes in intervention versus control areas. Since this is a CSHGP entry grant, the innovation was not required. To have a greater impact on a small area, after the Midterm evaluation, ECD activities were confined to Aromo sub-county. For Objectives Nos. 3 and 4, MTI and the Lira DHO conduct joint activities for planning, training, implementation and evaluation. The project supported training in Integrated Management of Childhood Illness (IMCI) for health facility staff, followed by on-the-job mentoring of health facility staff and joint supportive supervision with the District Health Office partners. For Objective 4, as part of entry grant requirements, MTI Uganda receives support from MTI head office to sustainably build MTI Uganda capacity through a structured Action Plan based on results from the use of an Organizational Capacity Assessment tool adapted from the Organizational Capacity Tool developed by John Snow, Inc. VI. Purpose of the Final Evaluation The FE provides an opportunity for all project stakeholders to take stock of accomplishments and lessons learned to date and to listen to beneficiaries at all levels. The FE Report will be used by the following audiences as a source of evidence to help inform decisions about future program designs and policies: • The Lira District Health Office, health facility staff, and Health Unit Management Committees • Uganda Ministry of Health • Village Health Team and Mothers Group volunteers • Hands to Hearts International • The USAID CSHGP and the Uganda USAID Mission • Medical Teams International HQ and field staff VII. Methodology The evaluation approach comprises both a desk review of secondary data sources and collection of qualitative quantitative data to complement existing data. Medical Teams International will facilitate this sharing and feedback. Secondary Data: The final evaluator will review project reports including the Detailed Implementation Plan, annual reports and midterm evaluation report to assess the quality of quantitative and qualitative data and make assessments of project results in relation to the project design and targets set. The final evaluator will also review the USAID Global Health Initiative Uganda A Strategy for Accelerating Reductions in Maternal and Final Evaluation Report: Lira District Child Survival Project in Uganda December 2013 Final Evaluation - Annexes Neonatal Mortality, 2011, the Uganda Ministry of Health Road Map to accelerate Reduction of Maternal and Neonatal Morbidity and Mortality (2007), Operational Framework for Nutrition in the National Child Survival Strategy, IYCF National Counseling Cards for Community Volunteers and Safe Motherhood Flip Charts and Village Health Team Strategy and Operational Guidelines and training materials. Qualitative Data: Interviews with community volunteers, project beneficiaries, health staff, and District Health Team members will provide information on project participants’ perspective of their participation in the project and its impact. The external consultant Team Leader, Hands to Hearts consultant and MTI’s Senior Advisor, Maternal and Child Health will develop the interview guides with input from the Lira team. A sample of high performing and low performing communities will be selected. Quantitative Data: MTI’s Senior Advisor, Monitoring and Evaluation, will travel to Lira in August to support the team to implement a Health Facility Assessment and household Knowledge, Practice and Coverage (KPC) survey. Data Analysis: Quantitative and qualitative findings will be analyzed jointly by the external consultant, MTI field and HQ staff, District Health Team members, and health facility staff. Facilitation of the review workshops will be shared by the External Consultant and MTI HQ and project staff. Limitations: The evaluation report will include a discussion of the methodological limitations of the evaluation. VIII. Evaluation Questions 1) To what extent did the project accomplish objectives stated in the DIP? Objective No.1: Communities assume responsibility for their own health through strengthened community capacity a) What training, support and supervision were provided by the project to Village Health Teams and Mothers Groups? b) What do VHTs and Mothers Groups feel they have gained from the CSP’s support? What do they feel has been the impact of their work? c) What do beneficiaries and health staff feel has been the impact of the work of community volunteers? d) How will the VHTs and Mothers Groups continue to work after the project closes? What support will they need? Objective No.2: Improved health (Community-based Integrated Management of Childhood Illness) and child care (Early Childhood Development) behaviors among mothers of children under five years of age. a) As evidenced by KPC data, what key maternal, newborn and child health practices have improved? What strategies and activities contributed to the improvement? Final Evaluation Report: Lira District Child Survival Project in Uganda December 2013 Final Evaluation - Annexes b) What key maternal, newborn and child health practices have not improved? What are lessons lesson learned from indicators that did not improved? c) What can the Lira, Ogur and Aromo DHOs do to address the gaps that still exist? d) What are parents and VHT perspectives on the implementation of early childhood development and its impact on families? Objective No.3: Improved quality of health facility services through strengthened Integrated Management of Childhood Illness and Maternal and Newborn Care capacity. a) As evidenced by HFA findings, what has been the impact of project support on implementation of IMCI and MNC services at health facilities? What strategies and activities contributed to the improvement? b) How do health staff feel they have benefited from the support the project has provided? c) How have HUMCs supported the health facilities? d) Do families feel health facility services have improved? e) In what ways will improvements in services continue? Objective No.4: Strengthened institutional capacity of MTI and the Lira District Health Office to implement effective and efficient child survival activities. a) What has been the impact of DHO/MTI joint supervision visits to health facilities and communities? b) What progress has MTI Uganda made in improving low-scoring areas identified during the baseline organizational assessment? o Strategic Planning o Staff Salaries and Benefits policies o Communication o Decision-making o Succession Planning o Monitoring and Evaluation 2) To what extent was the project implemented as planned? What changes were made to the planned implementation, and why those changes were made? 3) How were gender considerations addressed through the project? Are there any specific gender-related outcomes? 4) Which elements of the project have been or are likely to be sustained or expanded? IX. Final Evaluator Characteristics and Expected Timeline The consultant will serve as the evaluation team leader and is welcome to propose additional evaluation team members to round out the evaluation team’s skill set in order to ensure adequate representation of evaluation, technical, geographic, cultural and language skills. Team members, their affiliations, and disclosure of conflicts of interest Final Evaluation Report: Lira District Child Survival Project in Uganda December 2013 Final Evaluation - Annexes will be listed in an annex to the evaluation report. The consultant will coordinate closely with the Medical Teams International team regarding tool finalization, evaluation methodology, timeline, and draft report finalization. Requirements: The consultant must be approved by USAID CSHGP and should meet the following minimum requirements: • Proven expertise and leadership in − integrated community-oriented reproductive, maternal, newborn, and child health projects − conduct of evaluations (baseline, endline) using mixed methods • Experience with design, collection, and analysis using applied research methods in a program implementation context • Familiarity with public health system in Uganda • Demonstrated ability to communicate with and lead a team of stakeholders, staff, and national experts in participatory evaluation • Familiarity with USAID programming • Skill or familiarity with cost analysis methods for program assessments • Excellent analytical and writing skills (English) • Signed statement explaining any conflict of interest Key Tasks of the Evaluation Team Leader: • Review project documents and resources to understand the project • Refine the evaluation objectives and key questions based on the CSHGP guidelines in coordination with Medical Teams International Teams and its partners • Provide input to developing the field evaluation schedule and assessment tools • Lead the team to complete the collection, analysis, and synthesis of supplemental information regarding the program performance • With the team, interpret both quantitative and qualitative results and draw conclusions, lessons learned, and recommendations regarding project outcome • Lead an in-country debriefing meeting with key stakeholders, with a PowerPoint slideshow deliverable, no longer than 20 slides (with USAID/Washington, DC, participation remotely, as able) • Prepare draft report in line with the CSHGP guidelines and submit to CSHGP and Medical Teams International simultaneously on or before October 16th. • Prepare and submit the final report, which is due at the USAID CSHGP GH/HIDN/NUT office on or before 90 days after the end of the project Timeline: August 18 - 31 September 4 -5 September 7 - 8 September 9 - 20 September 21 - 22 October 16 December 19 KPC survey and HFA implemented with support Preparation to travel Travel to Uganda Field Work Travel back to the US Draft report submitted to CSHGP and MTI by Report finalized based on feedback from CSHGP Final Evaluation Report: Lira District Child Survival Project in Uganda December 2013 Final Evaluation - Annexes from M and Advisor external consultant and MTI. Soft version submitted by external consultant and hard copies posted by MTI HQ. X. Final Evaluation Report The FE report should follow the outline in USAID CSHGP’s Guidelines for Final Evaluations. A draft and final report, written by the final evaluator, must be submitted directly to the CSHGP. Draft and final reports should be submitted according to the submission instructions as indicated in the guidelines. XI. Estimated Budget Unit cost Number of Units Total External Consultant’s time (Team Leader) Preparation - 2 Travel days – 2 Days in country - 12 Writing FE report - 4 Editing FE report - 1 $450 21 $9,450 Hands to Hearts Consultant time 2 days $500 2 $1,000 KPC Survey $4,100 1 $4,100 Health Facility Assessment $4,000 1 $4,000 Review meeting $3,500 1 $3,500 Focus group interviews $1,400 1 $1,400 Total $23,450 XII. Deliverables At the conclusion of the consultancy period, the consultant is expected to complete the following deliverables: • Lead an in-country debriefing meeting with key stakeholders (and remote participation by USAID/Washington, DC) with a PowerPoint presentation no longer than 20 slides for distribution • Prepare a draft report in line with the CSHGP guidelines and submit to CSHGP and Medical Teams International simultaneously on or before October 16 Final Evaluation Report: Lira District Child Survival Project in Uganda December 2013 Final Evaluation - Annexes • Prepare and submit the final report, which is due at the USAID CSHGP GH/HIDN/NUT office on or before 90 days after the end of the project. Final Evaluation Report: Lira District Child Survival Project in Uganda December 2013 Final Evaluation - Annexes ANNEX IX. EVALUATION METHODS AND LIMITATIONS The Final Evaluation consisted of a combination of quantitative and qualitative methods using participatory techniques. Preparation The preparation began with alerting all partners to the future evaluation and gaining their commitment to be involved in the process. Joint planning then began with the external consultant, Director of Hands to Hearts International, MTI HQ, MTI Uganda, and the Lira DHO in accordance with USAID’s Evaluation Guidelines. Documents Review 1. Project DIP 2. Annual Reports from Years 1 and 3 3. Mid Term Evaluation Report 4. USAID Global Health Initiative Uganda A Strategy for Accelerating Reductions in Maternal and Neonatal Mortality, 2011 5. Uganda Ministry of Health Road Map to accelerate reduction of Maternal and Neonatal Morbidity and Mortality (2007) 6. Operational Framework for Nutrition in the National Child Survival Strategy 7. IYCF National Counseling Cards for Community Volunteers and Safe Motherhood flip Charts 8. Village Health Team Strategy and Operational Guidelines and training materials Surveys and Interview Guides A 30 cluster KPC and a R-HFA was conducted in mid August, two weeks prior to the qualitative evaluation, guided by MTIs Senior Advisor, Monitoring and Evaluation. A 30-cluster stratified sampling design was utilized to select the mothers of children aged 0-23 months. A total of thirty clusters were sampled in Lira District, with twelve households from each cluster being selected for interviews. There are only eight functioning HFs in the project area therefore no sample size selection was necessary as all eight HFs were surveyed for the RHFA. Preliminary data reports were provided to the evaluation team for use in preparing the interview guides. Final KPC and RHFA reports are included in Annexes VI and X A. The consultant, Director of Hands to Hearts International, HQ Senior Advisor, Maternal and Child Health and the CSP Project Manager, prepared qualitative interview guides. Criteria were established for selection of the communities for qualitative final evaluation field visits. In Country A one-day orientation and training was conducted for evaluation team members to review focus group interview techniques, to review the interview guides, and make necessary revisions to the evaluation logistics. Focus group discussions were held in each of the sub-counties with representatives from each target population group to gather qualitative information. Comparatively low performing communities, as well as high performing communities were identified by project staff prior to the evaluation. Groups of mothers and, separately, groups of fathers were interviewed in those communities using identical questionnaires. In addition, groups of VHTs within a Parish were interviewed and Mother Groups were also interviewed. Health facility staff, HUMCs, PDCs and Sub-county chiefs were interviewed to gain a non-community perspective on the community interventions and to gain a health system perspective on the project. Project staff participated in interviews in geographic areas where they were not working. A list of all those interviewed are included in Annex XIV. Qualitative data collection followed the interview guides that were designed to triangulate the results of the RHFA and the KPC as well as to assess the effectiveness of project processes and ascertain projections Final Evaluation Report: Lira District Child Survival Project in Uganda December 2013 Final Evaluation - Annexes concerning project sustainability. The complete packet of interview guides is included in Annex X B. Evaluation Question Information Source Survey Data 1) To what extent did the project accomplish objectives stated in the DIP? 4) Which elements of the project have been or are likely to be sustained or expanded? • Mothers with children 0-23 months in high performing communities • Fathers with children 0-23 months in high performing communities • Mothers with children 0-23 months in low performing communities • Fathers with children 0-23 months in low performing communities • Village Health Teams • Peer Educators • Mother Groups • Health Unit Management Committees • Parish Development Committees • Health Facility Staff • Senior Assistant Secretary of Sub￾Counties (Sub-County Chiefs) • MTI Senior Advisor, M&E • Quantitative 30 cluster KPC survey of mothers of children age 0- 23 months • Quantitative Rapid-Health Facility Services Assessment 2) To what extent was the project implemented as planned? What changes were made to the planned implementation, and why those changes were made? • CSP Program Manager • Project Staff • MTI Senior Advisor Maternal Child Health • MTE Report • Project Annual Reports 3) How were gender considerations addressed through the project? Are there any specific gender￾related outcomes? • CSP Program Manager • MTI Senior Advisor Maternal Child Health • Program Staff Question1) Objective 4, b) What progress has MTI Uganda made in improving low-scoring areas identified during the baseline organizational assessment? • MTI Regional Director Africa • MTI Africa Health Programs Advisor • MTI Uganda, Human Resources • Organizational Capacity Assessment The External Consultant led evaluation team members through a participatory analysis of the findings from focus group interviews and of the KPC and RHFA findings. Results were triangulated using the following tools: • Quantitative 30 cluster KPC survey of mothers of children age 0-23 months Final Evaluation Report: Lira District Child Survival Project in Uganda December 2013 Final Evaluation - Annexes • Quantitative Rapid-Health Facility Services Assessment • Comparison of data with available national and regional data from 2011 DHS • Triangulation of perspectives of qualitative interviews with community volunteers, project beneficiaries, community leaders, HF staff, sub-county leadership and district health staff. Participation by: • External consultant • MTI Uganda project staff • Lira District Health Office and health facility staff • Sub-County and Community leaders and volunteers • MTI Senior Advisor, Monitoring and Evaluation and Senior Advisor, Maternal and Child Health Finalization of Key Findings, Recommendations, and Action Planning • FE Results Meeting on key findings and recommendations with representatives of DHO, community volunteers, health facility staff, and sub-county leadership. (see Annex XIX C for FE Results Meeting Agenda) • Debriefing with USAID Uganda Mission (see Annex XVII for debrief PowerPoint) • Debriefing with MTI Uganda and MTI Africa Regional staff Evaluation Limitations Because four new HF were opened in April 2012 just after the MTE, it left very little time for improvements to be made especially in the physical plants. This meant that in order to report on RHFA data in a meaningful way, only data from the original four clinics is included in the narrative of this report. The entire report is included in Annex X A. An issue for the ECD portion of the qualitative evaluation was that the decision was made also to include communities in Lira and Ogur in the ECD interviews. Except for the ECD interventions, all project interventions were implemented across all communities in the three sub-counties, resulting in some of the communities chosen for interviews being ECD non-intervention communities. This limited the qualitative information available from intervention communities two years after the direct project support concluded. Final Evaluation Report: Lira District Child Survival Project in Uganda December 2013 Final Evaluation - Annexes Annex X A Lira District Child Survival Project in Uganda Child Health and Development in a Transitional Region Erute North Sub-District, Uganda October 2009 – September 2013 In Partnership with Uganda Ministry of Health Lira District Health Office Final Evaluation Rapid Health Facility Assessment Erute North Sub-District Uganda September 2013 Table of Contents Acronyms 3 Executive Summary 4 Chapter 1 Program Overview 5 Chapter 2 Purpose of the Health Facility Assessment and Methodology 7 Chapter 3 Main Findings: Accomplishments and Priorities 12 Chapter 4 Conclusions/Summary 33 2 ACRONYMS ANC Antenatal Care ARI Acute Respiratory Infection CHP Community Health Promoter CHW Community Health Workers C-IMCI Community IMCI CS Child Survival CORE Collaborations and Resources Group CSHGP Child Survival and Health Grant Program CSP Child Survival Project DHO District Health Office EPI Expanded Program of Immunizations HF Health Facility HFA Health Facility Assessment HS Health services HW Health Worker IMCI Integrated Management of Childhood Illnesses M&E Monitoring and Evaluation MNC Maternal Newborn Care MOH Ministry of Health MTI Medical Teams International MTI/Uganda Medical Teams International/Uganda N Sample size ORS Oral Rehydration Salts PHC Primary Health Care RHFA Rapid Health Facility Assessment USAID United States Agency for International Development VHT Village Health Team 3 I. Executive Summary In October 2009, Medical Teams International began the Lira District Child Survival Project in Uganda: Child Health and Development in a Transitional Region aimed at reducing child and maternal mortality and morbidity in Erute North Sub-District, Uganda. In September 2013, the Child Survival team carried out a Rapid Health Facility Assessment (R￾HFA) as part of the Final Evaluation process to assess the capacity of the 8 health facilities presently functioning in Erute North Sub-District. The assessment measured capacity in the areas of access, inputs, processes, and performance. The team utilized 4 teams of three individuals, consisting of one supervisor and two enumerators, which included members of the same staff that performed the baseline and midterm KPC survey and RHFA. This staff was comprised of MTI Supervisors and DHO members. Each team contained at least two members with extensive experience in health. All 8 HFs were scheduled to be surveyed in 4 days. At BL and MTE there were 4 HF that were assessed: Aromo (HF level 3), Barapwo (HF level 3), Amuca (HF level 3), and Ogur (HF level 4). This evaluation has also added Abala (HF level 2), Akangi (HF level 2), Apuce (HF level 2), and Walela (HF level 2). Therefore, the results are presented in 2 ways, first a comparison of the rates of the 4 HF that were surveyed at BL and MT, as well as FE, and then the rate of all 8 HFs. At baseline (BL) and midterm evaluation (MTE) the R-HFA brought to light several areas of needed improvement at the HFs in the project area. At BL the HFs did not perform well in any of the major areas of analysis, including: access (service availability), inputs (staffing and supplies), processes (training and supervision, and Information systems) and HW performance. Some areas continued to struggle even at the MTE, but the second half of the project focused on these challenges and by this FE there were major improvements in several areas since BL, most notably: • Service availability-Growth Monitoring: 2/4 HF at BL to 8/8 HF at FE • Infrastructure: 1/4 HF at BL to 8/8 HF at FE • Supplies for child services:1/4 HF at BL to 3/4 original and 3/4 new HF (6/8 total) at FE • Drugs for Child health: 1/4 HF at BL to 8/8 HF at MT • Training: 1/4 HF at BL to 8/8 HF at FE • Supervision: 1/4 HF at BL to 8/8 HF at FE • HW Performance (Assessment): 0/4 HF at BL to 7/8 HF at FE • HW Performance (treatment): 1/4 HF at BL to 8/8 HF at FE • HW Performance (counseling): 0/4 HF at BL to 8/8 HF at FE • Optional indicator for availability of immunizations: 1/4 HF at BL to 8/8 HF at FE • Optional indicator for HF community participation: 3/4 HF at BL to 8/8 HF at FE • Optional indicator for community referral: 3/4 HF at BL to 8/8 HF at FE A complete discussion of the 12 Core Indicators and optional indicators measured in the RHFA is available in the Summary Section (Chapter 4). 4 CHAPTER 1 Program Overview Project Area and Description: Medical Teams International implemented the Lira District Child Survival Project (LDCSP) in the Erute North Sub-District of Northern Uganda. The goal of the project was to reduce morbidity and mortality of children under five and women of reproductive age in Erute North Sub-District, Uganda. Direct beneficiaries were 25,498 children <5 25,498 WRA for a total of 50,996 direct beneficiaries. The level of effort by intervention for this Child Survival Project (CSP) was: 1/4 MNC, 1/4 PCM, 20% IYCF, 20% CDD, and 10% EPI. Capacity building activities with the DHO included improving the quality of health care for the sub-district population of 113,400. The primary implementing partner for this project was the DHO. Hands to Hearts International (HHI) was a collaborative partner, providing Early Childhood Development TOT trainings for VHTs, Peer Educators, and HF staff, and working with MTI and DHO to adapt the HHI ECD curriculum to the local context. Health Care Services: The Ugandan MoH is decentralized with district teams responsible for planning, budgeting, and monitoring performance. Each district is divided into Health Sub￾Districts which are responsible for delivering a basic package of health services, including control of communicable disease, Integrated Management of Childhood Illness (IMCI), reproductive health, immunization, environmental health, health education and promotion, epidemics, and nutrition.1 Successful implementation largely reflects district management priorities, capacities, and resources for training and supervision. A recent follow-up study of 10 districts in 2009 (none in the North) had mixed results. Staff turnover following the training was low, but only about half of clinic visits incorporated IMCI. At baseline there were a total of 4 functioning government health facilities in the project area. During 2007-08, DHO reported 93% of HCs had no stock out for first-line malaria, 78% quinine, 90% Fansidar, 97% measles vaccine, 93% Oral Rehydration Salts (ORS) sachets, and 91% Cotrimoxizole tablets. According to DHS 2006, 56% of children with diarrhea in Northern Region were treated with ORS packets and there was negligible access to zinc. Utilization of health services is low, according to MTI’s Knowledge, Practices & Coverage (KPC) survey (conducted in Aug. 2007): 30% of mothers took their child to an appropriate provider for fever, and 58% for pneumonia symptoms. When asked about their impression of services at Health Facilities (HFs), focus group participants expressed dissatisfaction, citing inadequate manpower, unqualified personnel rendering services, harsh treatment from some HF personnel, late arrival or absenteeism 1 Uganda MoH: National Health Policy, 1999. of HF personnel causing patients to wait, and overwhelming numbers of patients at the HF. According to the district Biostatistician, approximately 54% of the Sub-district population lived within 5 km of a HF. The farthest distance from a sub-district referral center (HC IV) was 25 km, and 45 km to the regional referral center in Lira Town. Project Goals, Objectives and Strategy The project goal was to reduce child morbidity and mortality in Uganda. Objectives were: 1) Communities assume responsibility for their own health through strengthening community capacity (VHTs, Parish Development Councils, and Health Sub-districts); 2) Improved health (C-IMCI) and child care (ECD) behaviors among mothers of children <5 years; 3) Improved quality of health facility services through strengthened IMCI and MNC capacity; 4) Strengthened institutional capacity of MTI and DHO to implement effective and efficient child survival activities. These objectives supported MoH goals and strategies as well as those of USAID Uganda. MTI used a two-pronged strategy: a) promoting behavior change and community mobilization to take appropriate responsibility for health: and b) building DHO capacity to provide sustainable, quality service delivery at the facility and community levels. The level of effort by intervention for this Child Survival Project (CSP) was: 1/4 MNC, 1/4 PCM, 20% IYCF, 20% CDD, and 10% EPI. 6 CHAPTER 2 Purpose of the Rapid Health Facility Assessment and Methodology The objective of this Health Facility Assessment was to collect quantitative data regarding the health system in the project area, particularly the primary health care (PHC) facilities providing maternal, neonatal, and child health (MNCH) services. The new Rapid Health Facility Assessment (RHFA) used to collect this data had been designed and recently upgraded by CSTS, and is now called RHFA Version 2.1. 2 The CSTS RHFA tool is designed to be rapid and cost effective, and is designed to be used at the local level to devise strategies, with the MOH entity present in the project area, and to improve the delivery of integrated child health services.3 The BL assessment was conducted before IMCI training had begun and prior to any interventions aimed at health facilities so that an integrated strategy to improve the quality of health care could be implemented. This FE RHFA measured the progress made in the 12 core areas of HF capacity. The R-HFA version 2.1 tool utilizes as its subjects the first six children, under the age of five, entering the health facility on the day of the survey with diarrhea, fever, or cough and examines three major areas of health care delivery4 :  Case management: Does the healthcare worker (HW) assess, diagnose, and treat children with diarrhea, fever (malaria), and ARI properly? Does the HW explain follow up care to the caretaker well?  Health facility infrastructure: Does the health facility have the necessary equipment, supplies, medications, and privacy to perform adequate MCH services?  Management (Processes): Are the proper management processes being followed in the health facility (supervision, record keeping, and continuation of training)? This R-HFA tool focuses on the delivery of care for the most important causes of infant and child morbidity and mortality, which include: diarrhea, acute respiratory infections (ARI), malaria, measles, and malnutrition. The purpose of collecting this data is to allow the project, in conjunction with the MOH, DHO, and other health workers in the project area, to determine gaps in service and prioritize their response in order to provide essential, integrated health services. There are four main modules in the R-HFA, with a fifth optional module. All are formatted in Excel for ease of use5 : 2 R-HFA Version 2.1; CSTS 2007; http://www.childsurvival.com/rhfa_1.cfm 3 BASICS II Health Facility Assessment; BASICS 1999; http://www.basics.org/Publications/pubs/hfa/hfa_apdxc.htm 4 Ibid. 5 R-HFA Version 2.1; CSTS 2007; http://www.childsurvival.com/rhfa_1.cfm. a. Observation Checklist for sick child care: To observe the HW in the assessment, diagnosis, and treatment of six consecutive cases of care of children under the age of five with fever, diarrhea, or breathing difficulty. The HW is assessed for adherence to the national (IMCI) protocol for assessment, classification, and treatment of childhood illness. b. Client (Caretaker) Exit Interview: To assess whether the caretaker has the correct knowledge of how to administer drugs given for diarrhea, malaria, and/or breathing difficulty (used a proxy for adequate counseling), and whether the caretaker knows under what circumstances the child is to return to the clinic. c. Health Facility Checklist: To assess the presence of a minimal level of infrastructure, equipment, supplies, and medications. d. Health Worker Interview and Record Review: To assess the staffing, MNCH services offered, and also assess the frequency of training, supervision, and other key processes. e. CHW Survey and Checklist (optional): To collect data on CHWs regarding six of the twelve health facility core indicators (through examination of registers). Selecting the Sampling Frame The sampling methodology was revised by CSTS to use a quality assurance type approach similar, but not identical, to that used in LQAS. The new methodology dictates that at least 80% of the health facilities perform adequately, according to the indicators chosen in the HFA, for the project area to pass. This is called the performance benchmark. Also, an unacceptable level of 4/6 is chosen as the level that should not go undetected in determining that the health facilities are not performing adequately in regard to a given indicator6 . The alpha and beta errors have been placed at 10%. The new R-HFA software automatically calculates the sample size needed, and calculates the results following the survey. Because there are only 8 functioning HFs in the project area, no sample size selection was necessary as all 8 HFs were surveyed. Selecting the Survey Teams Four teams of three individuals, one supervisor and two enumerators were formed using the same staff that participated in the baseline KPC Survey and community Feedback sessions. This staff was comprised of MTI Supervisors and DHO members. Each team contained at least two members with experience in health. Therefore, all 8 HFs were scheduled to be surveyed in 4 days. Each element of the R-HFA was administered by the following members of the R-HFA survey team: a. Observation Checklist for sick child care: Enumerator with experience in health b. Client (Caretaker) Exit Interview: Enumerator, with guidance from Enumerator with experience in health c. Health Facility Checklist: Supervisor 6 R-HFA Version 2.1; CSTS 2007; http://www.childsurvival.com/rhfa_1.cfm 8 d. Health Worker Survey: Supervisor of Enumerator with experience in health; should be performed by whichever team member has completed their other duties. e. CHW Survey and Checklist (optional): Enumerator with guidance from Supervisor or Enumerator with experience Training the Survey Team The refreshment training of the survey team required two days and was facilitated by the MTI HQ Sr. Advisor in M&E who directed the FE survey. The main objectives of the training were to discuss the purpose of the survey and the resulting information; discuss the logistics of the survey; review and practice each of the forms; and practice administering these forms in the facility setting. A health facility that was near the training site but not part of the project in Erute North Sub-District was chosen for the HFA was used for the field test. The Survey Process Observation of Clinical Care and Caretaker Exit Interview The first six children under the age of five presenting to the facility during the survey period whose caretakers describe them as having diarrhea/vomiting, fever/malaria, or cough/difficulty breathing/pneumonia were included in the sample. The caretakers were met as they entered the clinic, and if they agreed to take part in the survey they were followed throughout the facility. If the caretaker brought more than one sick child under the age of five, one child was randomly chosen to be the index child. The enumerator with experience in health observed the clinical encounter between the HW and the caretaker and child. The second enumerator conducted the Exit Interview with the caretakers of sick children outside of the facility as they exited, following receiving the child’s medications. Health Facility Checklist After ensuring that these interviews were proceeding well, the supervisor completed the Health Facility Checklist with an available HW at the facility. A HW was present because determining the conditions in the consultation room and of some of the equipment required some discussion with the HW. Health Worker Survey Following completion of the observation of six consultations between the HW and the caretaker/child, the enumerator with experience or the supervisor, whichever was available, performed the HW Survey. 9 Providing Feedback to the Staff Surveyors were instructed to provide some feedback to staff on the day of the assessment. The feedback on positive points was provided to alleviate any anxiety the staff may have felt due to the survey, but also included constructive comments to improve clinical treatment and management techniques. It was recommended during the training that feedback regarding the following items be given7 :  Strengths and problems in case management, particularly in the assessment and treatment of sick children  Quality of home-care advice and communication between health workers and caretakers  Inappropriate use of medications  Problems in record keeping  Ways to improve clinic organization  Major barriers to effective practice Checking the Completed Questionnaires Completed questionnaires were checked by the supervisor and enumerators administering the questionnaire immediately at the conclusion of the interview so that any discrepancies or missed questions could be discussed with the person being interviewed. At the end of each facility session, the Supervisor reviewed all forms with the enumerators before leaving the facility. The completed forms were then brought to the central point chosen for data entry and given to the data entry staff. This was done nightly so that data entry could be performed during the data collection period. The data entry staff reviewed the completed questionnaires for accuracy while the survey team was there, in order to clarify or correct any unclear or incorrect items noticed in the forms. Data Entry During the data collection phase of the survey, data was entered into the R-HFA Excel program provided in the R-HFA zip file available on the CSTS website8 . Data was entered daily throughout the survey so that any discrepancies could be discussed with the supervisors as soon as possible. Cleaning of the survey data was accomplished by the data entry staff as the data was presented. Following data entry for the final assessment, all data was then combined in the single Excel file, provided by CSTS, and analyzed by the HQ Senior Advisor in M&E and was shared immediately with the team in Uganda. 7 R-HFA short instruction 12-09-07; R-HFA Version 2.1; CSTS 2007; http://www.childsurvival.com/rhfa_1.cfm 8 R-HFA Version 2.1; CSTS 2007; http://www.childsurvival.com/rhfa_1.cfm 10 CHAPTER 3 Main Findings ACCESS (INPUTS) 1. Service Availability COMPARISON OF THE 4 HF AT BL TO MTE TO FE Indic. # Domain Indicator % HF with all elements BL MTE FE 1 CHILD Service Availability - Child % HF that offer all three basic child health services (growth monitoring, immunization, sick child care) 0/4 1/4 3/4 1 ANC Service Availability - ANC % HF that offer ANC at least once a week 4/4 4/4 4/4 1 NEO Service Availability - Delivery % HF that offer delivery services on all days 2/4 1/4 3/4 FE OF ALL 8 HFS Indic. # Domain Indicator % HF with all elements 4 NEW HF ALL 8 HF 1 CHILD Service Availability - Child % HF that offer all three basic child health services (growth monitoring, immunization, sick child care) 2/4 5/8 1 ANC Service Availability - ANC % HF that offer ANC at least once a week 1/4 5/8 1 NEO Service Availability - Delivery % HF that offer delivery services on all days 0/4 3/8 • At baseline, 0/4 of the HFs offered all 3 basic child health services o This was due to the fact that no facilities saw patients 30 days per month (including outreach). However, all of the facilities provided sick child services at a minimum of Monday-Friday, for a total of 20 hours per week o At FE, 3/4 of the original HFs, all except Barapwo, offered child health services (HS) 30 days per month (including outreach), and 5/8 of the total HFs provided HS 30 days per month. In addition to Barapwo, Akangi and Walela did not provide HS 30 days per month. o Growth Monitoring:  At BL only 2 of the 4 HFs (Ogur and Amuca) offered growth monitoring.  At FE, all 8 HFs offered GM. At MTE it was discovered that mothers did not bring their health cards to the other facilities so they were not able to be checked. Activities were put in place at all HFs to encourage all mothers to bring their children’s health card with them to every visit. As part of the IMCI protocol, growth monitoring was introduced to all HFs so that all HFs provided growth monitoring services in the facility and/or through outreach. o Immunization:  At baseline all 4 HFs offered immunization services through the facility and/or outreach.  At FE all 8 HFs offered immunization services o ANC Services:  At baseline and MTE all HFs in Lira District offered ANC services and at final all the original HFs still offered ANC. Of the 4 new HFs, only Abala offered ANC. o Delivery Services:  At FE, Barapwo, one of the original HFs, and each of the new HFs did not provide delivery services. In 2008, Barapwo was upgraded from a level 2 facility to a level 3 facility and therefore had no maternity unit, and the new facilities are all level 2 and therefore had no maternity unit. 2. Staffing COMPARISON OF THE 4 HF AT BL TO MTE TO FE Indic. # Domain Indicator % HF with all elements Index Value (% avg. HF attainment BL MT E FE BL MTE FE 2 Staffing % HF with all staff who provide clinical services working on the day of survey 0/4 1/4 1/4 48% 68% 66% FE RESULTS OF ALL 8 HFS Indic. # Domain Indicator % HF with all elements Index Value (% avg. HF attainment 4 new HF ALL 8 HF 4 new HF ALL 8 HF 2 Staffing % HF with all staff who provide clinical services working on the day of survey 4/4 5/8 100% 83% This indicator has been changed from the original RHFA staffing indicator, which was: “% of HFs with at least one provider meeting the country definition as qualified to provide curative care for children is present on day of survey”. All HFs would have met the old indicator, both at BL and MTE, because at least one qualified 12 provider was present at each facility. However, the new indicator determines the number of each type of staff, and determines if they are all present on the day of the survey. o Therefore, at BL, 0/4 of the HFs met this requirement, with the average HF attainment at 48% (meaning the average HF had 48% of its qualified staff present). o At FE 1/4 (Amuca) of the original HFs had all staff present, and all of the new facilities had all staff present, for a total of 5/8 HFs. The project worked with the DHO to continue to improve staff attendance since MTE so that treatment of patients could be as effective as possible and refresher training in IMCI was as effective as possible by ensuring that all HW treating patients were present to benefit from project follow-up mentoring and supportive supervision activities after IMCI topic training is provided. 3. Infrastructure COMPARISON OF THE 4 HF AT BL TO MTE TO FE Ind # Domain Indicator % HF with all elements Index Value (% avg. HF attainment BL MT E FE BL MTE FE 3 Infrastructure % HF in which all essential infrastructure is present and functioning on day of the survey (improved water source; functional latrine for clients; setting allowing auditory and visual privacy) 1/4 4/4 4/4 63% 72% 100% FACTORS BL MTE FE Has at least one bed 3/4 3/4 3/4 Has 24 hour staff coverage 3/4 4/4 3/4 Has functioning emergency communication 3/4 3/4 0/4 Has emergency transportation usable today 0/4 0/4 0/4 Has electricity from the grid or a generator with fuel 3/4 1/4 1/4 Has a usable client latrine 4/4 4/4 4/4 Has water from protected water source on or near grounds 2/4 4/4 4/4 Has auditory and visual privacy 2/4 4/4 4/4 13 FE RESULTS OF ALL 8 HFS Indi c. # Domain Indicator % HF with all elements Index Value (% avg. HF attainment 4 New HF ALL 8 HF 4 New HF ALL 8 HF 3 Infrastructur e % HF in which all essential infrastructure is present and functioning on day of the survey (improved water source; functional latrine for clients; setting allowing auditory and visual privacy) 4/4 8/8 100% 100% FACTORS 4 NEW HF ALL 8 HF Has at least one bed 0/4 3/8 Has 24 hour staff coverage 1/4 4/8 Has functioning emergency communication 2/4 4/8 Has emergency transportation usable today 0/4 0/8 Has electricity from the grid or a generator with fuel 0/4 1/8 Has a usable client latrine 4/4 8/8 Has water from protected water source on or near grounds 4/4 8/8 Has auditory and visual privacy 4/4 8/8 • At BL, all essential health infrastructure was present on the day of the survey in only 1/4 of the HFs (Ogur), with the average HF attainment at 63%. o The limiting factors were: emergency transportation, which was not available in any of the HFs, an improved, protected water source, which was only available in 2/4 of the HFs (Ogur and Amuca), and auditory and visual privacy, which was also only available in 2/4 of the HFs (Ogur and Aromo). o However, there was 24 hour staff coverage in 3/4 of the original HFs (Aromo, Barapwo, and Ogur). o However, 3/4 of the HFs had electricity from the grid or a generator with fuel and functioning emergency communication (Aromo, Barapwo, and Ogur). o All of the HFs had a useable client latrine. • At FE, all essential health infrastructure was present on the day of the survey in all 4 of the original HFs and all 4 of the new HFs as well. o All 8 facilities had water from a protected water source and auditory and visual privacy (both up from 2/4 at BL). o Also, 24 hour staff coverage was available in 3/4 of the original HFs and only 1 of the new facilities (Walela) o However, electricity from the grid or a generator with fuel was down from 3/4 at BL (all HFs except Aromo), to only 1/8 at FE (Amuca). 4. Supplies – Child 14 COMPARISON OF THE 4 HF AT BL TO MTE TO FE Indic. # Domain Indicator % HF with all elements Index Value (% avg. HF attainment BL MTE FE BL MTE FE 4 CHIL D Supplies - Child % HF with all essential supplies to support child health on day of the survey (accessible and working scale for child, accessible and working scale for infant, timing device for diagnosis of pneumonia, spoon/cup/jug to administer ORS) 1/4 3/4 3/4 40% 95% 95% FACTORS BL MTE FE Has functioning and accessible infant scale 2/4 4/4 4/4 Has functioning and accessible scale for children/adults 2/4 3/4 4/4 Has functioning timer or watch 2/4 4/4 3/4 Has pitcher for ORS 1/4 4/4 4/4 Has cup or spoon for ORS 1/4 4/4 4/4 FE RESULTS OF ALL 8 HFS Indic. # Domain Indicator % HF with all elements Index Value (% avg. HF attainment 4 New HF ALL 8 HF 4 New HF ALL 8 HF 4 CHIL D Supplies - Child % HF with all essential supplies to support child health on day of the survey (accessible and working scale for child, accessible and working scale for infant, timing device for diagnosis of pneumonia, spoon/cup/jug to administer ORS) 3/4 6/8 85% 90% FACTORS 4 NEW HF ALL 8 HF Has functioning and accessible infant scale 3/4 7/8 Has functioning and accessible scale for children/adults 4/4 8/8 Has functioning timer or watch 4/4 7/8 Has pitcher for ORS 3/4 7/8 Has cup or spoon for ORS 3/4 7/8 15 • At BL only 1/4 of HFs (Barapwo) had all of the essential supplies to support child health, and the average HF attainment was 40% of these essential supplies.  ORS pitchers and spoons were only available in Barapwo, and scales and timers were available in only 2/4 (Barapwo and Amuca) of the facilities. • At FE 3/4 of the original HFs (Aromo, Barapwo, and Ogur) and 3/4 new HFs (all but Abala) had all of the essential supplies to support child health, and the average HF attainment was 90% of these essential supplies. Supplies – MNC COMPARISON OF THE 4 HF AT BL TO MTE TO FE Indic. # Domain Indicator % HF with all elements Index Value (% avg. HF attainment BL MT E FE BL MTE FE 4 Neo Supplies – HF % HF with all essential supplies to support maternal￾newborn health present on day of the survey (partograph, vacuum extractor, resuscitation device, weighing scale, antibiotics and baby wraps) 0/4 0/4 1/4 25% 30% 65% FACTORS BL MTE FE Has functioning neonatal resuscitation equipment 0/4 3/4 3/4 Has functioning and accessible infant scale 3/4 3/4 4/4 Has functioning vacuum extractor 0/4 0/4 1/4 Has neonatal wraps for warming 0/4 0/4 1/4 Has partographs 2/4 0/4 4/4 16 Supplies – MNC FE RESULTS OF ALL 8 HFS Indic. # Domain Indicator % HF with all elements Index Value (% avg. HF attainment 4 New HF ALL 8 HF 4 New HF ALL 8 HF 4 Neo Supplies – HF % HF with all essential supplies to support maternal￾newborn health present on day of the survey (partograph, vacuum extractor, resuscitation device, weighing scale, antibiotics and baby wraps) 0/4 1/8 0% 13% FACTORS 4 New HF ALL 8 HF Has functioning neonatal resuscitation equipment 0/4 3/8 Has functioning and accessible infant scale 0/4 4/8 Has functioning vacuum extractor 0/4 1/8 Has neonatal wraps for warming 0/4 1/8 Has partographs 0/4 4/8 • At BL, none (0/4) of the HFs had all of the essential supplies to support maternal￾newborn health available and the average HF attainment was only 25% of these essential supplies. o None of the facilities had neonatal resuscitation equipment, functioning vacuum extractors, or neonatal wraps for warming. o Only 2/4 of facilities (Aromo and Amuca) had partographs. • At FE, only Amuca (1/4) of the original HFs had all of the essential supplies to support maternal-newborn health available and the average HF attainment was at only 65% for these essential supplies. Also, none of the new HFs had all essential MNC supplies, for a total of only 1/8 HFs and only a 13% average HF attainment. • Aromo, Amuca, and Ogur had neonatal resuscitation equipment and infant scales. • Only Amuca had functioning vacuum extractors or neonatal wraps for warming. • All 4 of the original HFs had functioning partographs, and none of the new HFs had functioning partographs, for a total of 4/8. 17 Supplies – ANC COMPARISON OF THE 4 HF AT BL TO MTE TO FE Indic. # Domain Indicator % HF with all elements Index Value (% avg. HF attainment BL MT E FE BL MTE FE 4 ANC Supplies - ANC % HF with all essential supplies to support antenatal care present on day of survey (blood pressure machine, tetanus toxoid vaccine, hemoglobin reagents, syphilis testing kit, and albastix for protein) 0/4 0/4 0/4 42% 54% 75% FACTORS BL MTE FE Has functioning blood pressure equipment 2/4 4/4 3/4 Has hemoglobin testing reagents 1/4 3/4 2/4 Has syphilis testing kits 2/4 1/4 3/4 Has malaria test kits 2/4 2/4 4/4 Has urine albumin test strips 0/4 1/4 2/4 Has tetanus toxoid 3/4 2/4 4/4 Supplies – ANC FE RESULTS OF ALL 8 HFS Indic. # Domain Indicator % HF with all elements Index Value (% avg. HF attainment 4 New HF ALL 8 HF 4 New HF ALL 8 HF 4 ANC Supplies - ANC % HF with all essential supplies to support antenatal care present on day of survey (blood pressure machine, tetanus toxoid vaccine, hemoglobin reagents, syphilis testing kit, and albastix for protein) 0/4 0/4 13% 44% FACTORS 4 New HF ALL 8 HF Has functioning blood pressure equipment 1/4 4/8 Has hemoglobin testing reagents 0/4 2/8 Has syphilis testing kits 0/4 3/8 Has malaria test kits 1/4 3/8 Has urine albumin test strips 0/4 2/8 Has tetanus toxoid 3/4 7/8 • At BL 0/4 of the HFs had all of the essential supplies to support antenatal care. 18 • No facilities had albumin test strips, and only 1/4 of the HFs (Aromo) had syphilis testing kits and hemoglobin testing kits available. • Also, malaria test kits (Aromo and Amuca), blood pressure equipment (Ogur and Aromo), and syphilis testing kits were available in only 2/4 (Aromo and Amuca) of the facilities. • All HFs other than Barapwo had tetanus toxoid available. • At FE none of the HFs had all of the essential supplies to support antenatal care, with a 75% attainment at the original HFs and only a 44% attainment at all 8 HFs. • Only Aromo and Ogur had hemoglobin testing reagents or malaria test kits. • Ogur had all equipment except functional BP equipment. 5. a. Drugs - CHILD COMPARISON OF THE 4 HF AT BL TO MTE TO FE Indic. # Domain Indicator % HF with all elements Index Value (% avg. HF attainment BL MTE FE BL MTE FE 5 CHILD Drugs - CHILD % HF with all first line medications for child health present on day of the survey (ORS, oral antibiotic for pneumonia, first line oral antibiotic for dysentery, first line antimalarial, vitamin A) 1/4 1/4 4/4 40% 75% 100% FACTORS BL MTE FE Has ORS packets 3/4 4/4 4/4 Has first line child pneumonia drug 1/4 1/4 4/4 Has first line dysentery drug 1/4 2/4 4/4 Has first line antimalarial 1/4 4/4 4/4 Has vitamin A 2/4 4/4 4/4 FE RESULTS OF ALL 8 HFS Indic. # Domain Indicator % HF with all elements Index Value (% avg. HF attainment 4 NEW HF ALL 8 HF 4 New HF ALL 8 HF 5 MNC Drugs - MNC % HF with all essential delivery & neonatal drugs present on day of survey (i.e., Oxytocin, antibiotics for newborn sepsis and eye infections) 4/4 8/8 100% 100% FACTORS 4 NEW HF ALL 8 HF Has ORS packets 4/4 8/8 Has first line child pneumonia drug 4/4 8/8 Has first line dysentery drug 4/4 8/8 Has first line antimalarial 4/4 8/8 Has vitamin A 4/4 8/8 19 • At baseline, 1/4 of HFs (Amuca) had all first line medications available, which is defined as: ORS, a first line oral antibiotic for pneumonia, a first line drug for dysentery, a first line antimalarial, and Vitamin A. o This is due to the fact that while 1/4 of facilities had first line drugs for pneumonia (Barapwo) and 50 % of HFs had Vitamin A (Aromo and Amuca), only 1/4 of HFs (Amuca) had first line medications also available for malaria and dysentery. • At FE, all HFs had all first line medications available, the average HF attainment rose from 40% to 100% 5. b. Drugs – MNC COMPARISON OF THE 4 HF AT BL TO MTE TO FE Indic. # Domain Indicator % HF with all elements Index Value (% avg. HF attainment BL MT E FE BL MTE FE 5 MNC Drugs - MNC % HF with all essential delivery & neonatal drugs present on day of survey (i.e., Oxytocin, antibiotics for newborn sepsis and eye infections) 0/4 0/4 3/4 25% 42% 92% FACTORS BL MTE FE Has antibiotics for newborn sepsis/pneumonia 1/4 1/4 4/4 Has neonatal eye ointment 1/4 2/4 3/4 Has Oxytocin 1/4 2/4 4/4 Has Nevirapine (in high HIV prevalence areas only)- OPTIONAL 2/4 2/4 3/4 FE RESULTS OF ALL 8 HFS Indic. # Domain Indicator % HF with all elements Index Value (% avg. HF attainment 4 NEW HF ALL 8 HF 4 New HF ALL 8 HF 5 MNC Drugs - MNC % HF with all essential delivery & neonatal drugs present on day of survey (i.e., Oxytocin, antibiotics for newborn sepsis and eye infections) 0/4 3/8 34% 63% FACTORS 4 NEW HF ALL 8 HF Has antibiotics for newborn sepsis/pneumonia 2/4 6/8 Has neonatal eye ointment 2/4 5/8 Has Oxytocin 0/4 4/8 Has Nevirapine (in high HIV prevalence areas only) 0/4 3/8 20 • At BL, none of the HFs had all of the essential delivery and neonatal drugs present on the day of the survey. In fact, all 3 essential MNC drugs were available in only 1 out of the four HFs each (antibiotics-Amuca; neonatal eye ointment-Ogur; and Oxytocin-Ogur). • At FE 3/4 of the HFs had all of the essential delivery and neonatal drugs present on the day of the survey, with a 92% attainment rate. Only Ogur did not have eye ointment. None of the new HFs had MNC drugs as they are not performing deliveries and are level 2 HFs. 5. c. Drugs – ANC COMPARISON OF THE 4 HF AT BL TO MTE TO FE Indic. # Domain Indicator % HF with all elements Index Value (% avg. HF attainment BL MT E FE BL MTE FE 5 ANC Drugs – ANC % HF with all essential ANC drugs (Tetanus toxoid, iron/folate, IPT, ITNs) 1/4 4/4 0/4 69% 100% 69% FACTORS BL MTE FE Has tetanus toxoid 3/4 4/4 4/4 Has iron / folate 3/4 4/4 3/4 Has antimalarial IPT 3/4 4/4 4/4 Has ITNs 2/4 4/4 0/4 Drugs – ANC FE RESULTS OF ALL 8 HFS Indic. # Domain Indicator % HF with all elements Index Value (% avg. HF attainment 4 NEW HF ALL 8 HF 4 New HF ALL 8 HF 5 ANC Drugs – ANC % HF with all essential ANC drugs (Tetanus toxoid, iron/folate, IPT, ITNs) 0/4 0/8 56% 63% FACTORS 4 NEW HF ALL 8 HF Has tetanus toxoid 3/4 7/8 Has iron / folate 3/4 6/8 Has antimalarial IPT 3/4 7/8 Has ITNs 0/4 0/8 • At BL, only 2/4 HFs had all of the ANC drugs present on the day of the survey. Barapwo was missing 3 of the 4 drugs (Tetanus toxoid, Iron, and ITNs). Amuca was missing antimalarials. Only half of the HFs had ITNs (Aromo and Amuca). • At FE, none of the HFs had all the essential ANC drugs, because there were no ITNs available at any HF. Three out of 4 of the original HFs had all other ANC drugs, with Aromo only missing Iron. Three out of 4 of the new HFs had all other drugs, with Apuce missing Tetanus toxoid, Iron, and Fansidar for IPT. 21 PROCESSES 6. Information Systems Information System – Child COMPARISON OF THE 4 HF AT BL TO MTE TO FE Indic. # Domain Indicator % HF with all elements Index Value (% avg. HF attainment BL MT E FE BL MT E FE 6 CHIL D Informatio n System - Child % HF that maintain up-to-date records of sick U5 children (age, diagnosis, treatment) and for HF: have report in last 3 months and evidence of data use 0/4 0/4 0/4 25% 80% 80% Information System – Child FE RESULTS OF ALL 8 HFS Indic. # Domain Indicator % HF with all elements Index Value (% avg. HF attainment 4 NEW HF ALL 8 HF 4 New HF ALL 8 HF 6 CHIL D Informatio n System - Child % HF that maintain up-to-date records of sick U5 children (age, diagnosis, treatment) and for HF: have report in last 3 months and evidence of data use 0/4 0/8 80% 80% • At BL, none of the HFs maintained up to date records on sick children under 5 years of age, or up-to-date monthly service data. o The average facility had only 25% of the elements required for having up￾to-date records and monthly service data. o Ogur performed best with 80% of the elements in place. o Aromo and Barapwo HFs did not have any of the elements in place, while Amuca had only 20% of the elements in place. o Barapwo and Amuca used the data to create a wall chart, and Barapwo used the data to create a graph and in discussions, but the other HFs did not use the data at all. • At FE, still 0/4 of the original HFs and also 0/4 of the new HFs maintained up to date records on sick children under 5 years of age, or up-to-date monthly service data. o However, at FE each facility had 80% of the elements required for having up-to-date records and monthly service data, up from 25% at BL. o All facilities (8/8) had complete information in the sick child register, but no HF was using the monthly service data (MSR) which, therefore, is the limiting factor. 22 Information System – ANC COMPARISON OF THE 4 HF AT BL TO MTE TO FE Indic . # Domain Indicator % HF with all elements Index Value (% avg. HF attainment BL MTE FE BL MTE FE 6 MNC Informatio n System - ANC % HF that maintain up-to-date records of antenatal care (TT, blood pressure, expected date of delivery) & deliveries (present & up to date) 2/4 1/4 2/4 63% 75% 75% FACTORS BL MTE FE An ANC register was observed 3/4 4/4 4/4 ANC register with complete delivery information, last 3 months 2/4 3/4 4/4 ANC register with complete TT information, last 3 months 2/4 3/4 4/4 ANC register with complete BP information, last 3 months 2/4 0/4 3/4 ANC register with entry in last 7 days 3/4 4/4 4/4 Delivery register was observed 3/4 4/4 3/4 Delivery register was up to date (entry in last 30 days) 3/4 4/4 3/4 FE RESULTS OF ALL 8 HFS Indic . # Domain Indicator % HF with all elements Index Value (% avg. HF attainment 4 NEW HF ALL 8 HF 4 New HF ALL 8 HF 6 MNC Informatio n System - ANC % HF that maintain up-to-date records of antenatal care (TT, blood pressure, expected date of delivery) & deliveries (present & up to date) 0/4 2/8 14% 45% FACTORS 4 NEW HF ALL 8 HF An ANC register was observed 1/4 5/8 ANC register with complete delivery information, last 3 months 1/4 5/8 ANC register with complete TT information, last 3 months 1/4 5/8 ANC register with complete BP information, last 3 months 1/4 4/8 ANC register with entry in last 7 days 1/4 5/8 Delivery register was observed 0/4 3/8 Delivery register was up to date (entry in last 30 days) 0/4 3/8 • At BL, 2/4 HFs (Aromo and Amuca) had up to date records on antenatal care. Aromo and Amuca had all of the elements in place, and Ogur had 2/4 of the elements in place, while Barapwo had none of the elements in place. • At FE, all original HFs had up to date records on antenatal care. Amuca was missing BP information, and Barapwo did not have the delivery register up to 23 date. Of the new HFs, none had delivery information as they do not perform deliveries, and only Abala had an ANC register with complete information. 7. Training COMPARISON OF THE 4 HF AT BL TO MTE TO FE Indic. # Domain Indicator % HF with all elements BL MTE FE 7 CHIL D Training - Child Health % HF in which interviewed HW reported receiving in-service or pre-service training in child health in last 12 months 1/4 3/4 4/4 7 MNC Training - Maternal￾Neonatal Care % HF in which interviewed HW reported receiving in-service or pre-service training in maternal neonatal care in last 12 months 1/4 3/4 4/4 FACTORS CHILD HEALTH BL MTE FE Immunization training 0/4 2/4 4/4 Pneumonia case management training 0/4 3/4 4/4 Diarrhea case management training 1/4 3/4 4/4 Malaria case management training 0/4 3/4 4/4 ACT use training 0/4 3/4 4/4 ITN use training 0/4 3/4 4/4 Nutrition training 1/4 3/4 4/4 Breastfeeding promotion training 0/4 3/4 4/4 IMCI training 0/4 3/4 4/4 MNC BL MTE FE IPT use training 1/4 3/4 4/4 Newborn care training 0/4 2/4 4/4 Post-partum care training 0/4 1/4 4/4 ANC training 0/4 2/4 4/4 Infection control training 0/4 1/4 4/4 AMTSL training 0/4 1/4 4/4 Ob / neonatal emergencies referral training 0/4 1/4 4/4 Training FE RESULTS OF ALL 8 HFS Indic. # Domain Indicator % HF with all elements 4 NEW HF ALL 8 HF 7 CHIL D Training - Child Health % HF in which interviewed HW reported receiving in-service or pre-service training in child health in last 12 months 4/4 8/8 24 7 MNC Training - Maternal￾Neonatal Care % HF in which interviewed HW reported receiving in-service or pre-service training in maternal neonatal care in last 12 months 4/4 8/8 FACTORS CHILD HEALTH 4 NEW HF ALL 8 HF Immunization training 4/4 8/8 Pneumonia case management training 3/4 7/8 Diarrhea case management training 3/4 7/8 Malaria case management training 3/4 7/8 ACT use training 3/4 7/8 ITN use training 3/4 7/8 Nutrition training 2/4 6/8 Breastfeeding promotion training 3/4 7/8 IMCI training 3/4 7/8 MNC IPT use training 2/4 6/8 Newborn care training 4/4 8/8 Post-partum care training 3/4 7/8 ANC training 3/4 7/8 Infection control training 3/4 7/8 AMTSL training 3/4 7/8 Ob / neonatal emergencies referral training 3/4 7/8 • At BL. only1/4 of the HFs (Ogur) had HWs who reported receiving in-service or pre-service training in both maternal neonatal care and child health in the last 12 months. Training in MNC consists of: training in MNC and at least one other of the trainings listed above in the MNC section. Training in child health consists of: training in Child Health and at least one other of the trainings listed above in the child health section. Ogur received training in diarrhea case management, nutrition, and IPT use. • At FE, all of the original and new HFs had HWs who reported receiving in￾service or pre-service training in both maternal neonatal care and child health within the last 12 months. The various trainings are broken down in the above table. The child health trainings in pneumonia, diarrhea, malaria, ACT, ITN, BF, and IMCI were not received in Abala HF, and Abala and Apuce did not receive nutrition training. 25 8. Supervision COMPARISON OF THE 4 HF AT BL TO MTE TO FE Indic. # Domain Indicator % HF with all elements BL MTE FE 8 Supervision % HF that received external supervision at least once in the last 3 months (supervision included one or more of the following: checked records or reports, observed work, provided feedback, gave praise, provided updates, discussed problems)) 1/4 3/4 4/4 FE RESULTS OF ALL 8 HFS Indic. # Domain Indicator % HF with all elements 4 NEW HF ALL 8 HF 8 Supervision % HF that received external supervision at least once in the last 3 months (supervision included one or more of the following: checked records or reports, observed work, provided feedback, gave praise, provided updates, discussed problems)) 3/4 7/8 • At BL, only 1/4 of the HFs (Ogur) received any type of external supervision at least once in the 3 months prior to the survey. During this visit, the supervisor observed work in the HF, discussed problems, and checked drug supplies. • At FE, all 4 original HFs and 3/4 new HFs received supervision (Abala did not). 26 PERFORMANCE 9. Utilization of Curative services The percentage of HF with > 1 sick child encounters per child under the age of 5 in Lira District was not able to be calculated, because no facilities had complete sick child registries available at the time of survey. 10. HW Performance (Assessment) COMPARISON OF THE 4 HF AT BL TO MTE TO FE Indic. # Domain Indicator % HF with all elements Index Value (% avg. HF attainment BL MTE FE BL MT E FE 10 CHIL D HW Performance (Assessment) % patients in a facility for whom all 5 assessment tasks were done at least 80% of the time (check presence of general danger signs (including both vomiting and convulsions), assess feeding practices, assess nutritional status, check vaccination status) 0/4 0/4 3/4 21% 68% 98% FE RESULTS OF ALL 8 HFS Indic. # Domain Indicator % HF with all elements Index Value (% avg. HF attainment 4 NEW HF ALL 8 HF 4 NEW HF ALL 8 HF 10 CHIL D HW Performance (Assessment) % patients in a facility for whom all 5 assessment tasks were done at least 80% of the time (check presence of general danger signs (including both vomiting and convulsions), assess feeding practices, assess nutritional status, check vaccination status) 4/4 7/8 100% 99% In order for a HF to be considered as having key assessment tasks routinely performed, in greater than 80% (5 or 6 out of the 6 cases observed) of the encounters the HW must perform all of the key assessment tasks. • The baseline found that no facilities (0/4) contained HWs who were routinely performing all 5 key assessment tasks. o Furthermore, the average facility was performing only 21% of the key assessment tasks. o Ogur performed best by performing 63% of the assessment tasks, followed by Barapwo, which performed 21% of their assessment tasks. 27 Aromo and Amuca performed the worst, performing only 4% and 8% of their assessment tasks, respectively. o Assessment tasks which were assessed most frequently were: the ability to feed or breastfeed, asking if the child had been vomiting everything, and checking for the presence of convulsions, which were each asked 29% of the time. o Assessment tasks which were assessed least frequently were: checking the nutritional status and checking for immunizations on the child health card, which were each performed only 4% of the time. • At FE, 3/4 original facilities and all new HFs contained HWs who were routinely performing all 5 key assessment tasks. o Furthermore, at FE the average facility performed 98% of the key assessment tasks, with only Ogur not passing, only because 1 HW did not ask about convulsions in 2/6 children observed. All other performance was acceptable, thus the 98% attainment level. 10. HW Performance (Treatment) COMPARISON OF THE 4 HF AT BL TO MTE TO FE Indic. # Domain Indicator % HF with all elements BL MTE FE 11 CHILD HW Performance (Treatment) % HF where treatment is routinely appropriate(at least 80% of the time) to the diagnosis (for encounters in which at least one of the presenting problems was fever, breathing problem, or diarrhea) 1/4 4/4 4/4 HW Performance (Treatment) FE RESULTS OF ALL 8 HFS Indic. # Domain Indicator % HF with all elements 4 NEW HF ALL 8 HF 11 CHILD HW Performance (Treatment) % HF where treatment is routinely appropriate(at least 80% of the time) to the diagnosis (for encounters in which at least one of the presenting problems was fever, breathing problem, or diarrhea) 4/4 8/8 • At BL, only 1/4 of the HFs (Aromo) provided treatment that was routinely appropriate to the diagnosis. A HW must diagnose and treat the sick child correctly in greater than 80% (5 or 6 out of the 6 cases observed) of the encounters observed in order for a HF to be considered as providing correct treatment. 28 • At FE, all 8 facilities provided correct treatment appropriate to the diagnosis. 10. HW Performance (Counseling) COMPARISON OF THE 4 HF AT BL TO MTE TO FE Indic. # Domain Indicator % HF with all elements BL MTE FE 12 CHILD HW Performance (Counseling) % HF where caretakers whose child was prescribed an antibiotic, antimalarial, or ORS, correctly describe how to administer all prescribed drugs 0/4 0/4 4/4 FE RESULTS OF ALL 8 HFS Indic. # Domain Indicator % HF with all elements 4 NEW HF ALL 8 HF 12 CHILD HW Performance (Counseling) % HF where caretakers whose child was prescribed an antibiotic, antimalarial, or ORS, correctly describe how to administer all prescribed drugs 4/4 8/8 • AT BL, a very weak area in regards to HW performance was in the area of counseling the caretaker regarding the proper method of administering the medicines prescribed. None (0/4) of the HFs were routinely properly instructing caretakers in how to correctly administer drugs prescribed for their child. • At MTE, despite training in IMCI, this continued to be a challenge. It was determined that the HW was instructing the caretakers too quickly, and the dispensers were not aiding the caretakers by reiterating the instructions to them. In addition, many mothers were illiterate, and the dispensers were writing the instructions on the medication bag instead of drawing pictures to illustrate the instructions. The project worked on these 3 areas to improve the communication between the HWs and the caretakers, and at FE all 8 HFs were counseling caretakers properly regarding medication directions. 29 OPTIONAL INDICATORS COMPARISON OF THE 4 HF AT BL TO MTE TO FE Area of Analysis Indic. # Domain Indicator % HF with all elements Index Value (% avg. HF attainment) BL MTE FE BL MTE FE INPUTS Opt1 Availability of Immunizations % HF with all nationally￾mandated immunizations in stock on day of survey 1/4 4/4 4/4 44% 100% 100% Opt2 Availability of Guidelines % HF with all nationally￾mandated guidelines for care of children available and accessible on day of survey 1/4 2/4 4/4 46% 83% 100% Opt3 Infection Control % HF with all infection control supplies and equipment on day of survey 0/4 0/4 1/4 68% 65% 85% PROCESSES Opt4 HF-Community Participation % HF with at least one method for community participation and at least one way to incorporate information 3/4 4/4 4/4 Opt5 Community Referral % HF that received at least one referral from CHW (VHT) in the last month 3/4 4/4 4/4 FE RESULTS OF ALL 8 HFS Area of Analysi s Indic. # Domain Indicator % HF with all elements Index Value (% avg. HF attainment) 4 new HF ALL 8 HF 4 new HF ALL 8 HF INPUTS Opt1 Availability of Immunizations % HF with all nationally￾mandated immunizations in stock on day of survey 4/4 8/8 100% 100% Opt2 Availability of Guidelines % HF with all nationally￾mandated guidelines for care of children available and accessible on day of survey 1/4 5/8 54% 77% Opt3 Infection Control % HF with all infection control supplies and equipment on day of survey 0/4 1/8 83% 84% PROCESSES Opt4 HF-Community Participation % HF with at least one method for community participation and at least one way to incorporate information 4/4 8/8 Opt5 Community Referral % HF that received at least one referral from CHW (VHT) in the last month 4/4 8/8 30 • At BL, Only 1/4 of the HFs (Amuca) had all of the required immunizations in stock (BCG, OPV, DPT, and MMR), and had all of the guidelines regarding the care of children available and accessible on the day of the survey. At FE all 8 HFs had all required immunizations and all required guidelines present on the day of the survey. • At BL, no facilities had all of the requisite infection control supplies. This was due to the fact that no facility had an adequate infectious waste disposal area. Also, chlorine or other disinfectants were present in only 2/4 of the HFs (Ogur and Barapwo), and only 2/4 of the facilities had evidence of adequate sharps disposal and infectious waste disposal practices (Aromo and Amuca). In addition, Ogur did not have gloves or needles present, and Aromo did not have syringes available. At FE this continued, with only 1 original HF (Aromo) having the requisite infection control supplies because it was the only facility having an adequate infectious waste disposal areas and evidence of adequate infectious waste disposal practices. • At baseline, 3/4 of the facilities (Ogur, Aromo, and Amuca) had the combination of at least 1 method for community participation and at least one way to incorporate community information. At FE this had improved to all 8 facilities. • At BL, 3/4 (Ogur, Aromo, and Barapwo) of the HFs reported that they had received at least one referral from VHTs in the last month. At FE this had improved to all 8 of the HFs. 31 CHAPTER 4 Conclusions/Summary BASELINE FE 0% 20% 40% 60% 80% 100% 120% 140% 160% Geographic… Service… Staffing Infrastructure Supplies - Child Drugs - Child Availability of… Availability of… Infection Control Information… Training - Child… Supervision HF-Community… Community… Malaria Drug… ITN/LLIN Logistics Laboratory Assessment Treatment Counseling ACCESS INPUTS PROCESSES PERFORMANCE HF Child Health Balanced Score Card BASELINE FE ________________________________________________________________________ 0% 20% 40% 60% 80% 100% 120% 140% 160% Geographic Access Service Availability… Service Availability… Staffing Infrastructure Supplies - Neonatal Supplies - ANC Drugs - Neonatal Drugs - ANC Availability of… Infection Control Information… Training -… Supervision HF-Community… Malaria Drug… ITN/LLIN Logistics Laboratory HF MNC Balanced Score Card 33 The following is a summary and discussion of each of the 12 core indicators, along with a comparison to BL. The graphs above provide a visual summary of each of these areas, with a comparison of BL to MT. ACCESS (INPUTS) 1. Service Availability: At baseline, 0/4 of the HFs offered all 3 basic child health services o This was due to the fact that no facilities saw patients 30 days per month (including outreach). However, all of the facilities provided sick child services at a minimum of Monday-Friday, for a total of 20 hours per week o At FE, 3/4 of the original HFs, all except Barapwo, offered child health services (HS) 30 days per month (including outreach), and 5/8 of the total HFs provided HS 30 days per month. In addition to Barapwo, Akangi and Walela did not provide HS 30 days per month. o Growth Monitoring:  At BL only 2 of the 4 HFs (Ogur and Amuca) offered growth monitoring.  At FE, all 8 HFs offered GM. At MTE it was discovered that mothers did not bring their health cards to the other facilities so they were not able to be checked. Activities were put in place at all HFs to encourage all mothers to bring their children’s health card with them to every visit. As part of the IMCI protocol, growth monitoring was introduced to all HFs so that all HFs provided growth monitoring services in the facility and/or through outreach. o Immunization:  At baseline all 4 HFs offered immunization services through the facility and/or outreach.  At FE all 8 HFs offer immunization services o ANC Services:  At baseline and MTE all HFs in Lira District offered ANC services, and at final all the original HFs still offered ANC. Of the 4 new HFs, only Abala offered ANC. o Delivery Services:  At FE, Barapwo, of the original HFs, and each of the new HFs did not provide delivery services. In 2008, Barapwo was upgraded from a level 2 facility to a level 3 facility and therefore had no maternity unit, and the new facilities are all level 2 and therefore had no maternity unit. 2. Staffing: This indicator has been changed from the original RHFA staffing indicator, which was: “% of HFs with at least one provider meeting the country definition as qualified to provide curative care for children is present on day of survey”. All HFs would have met the old indicator, both at BL and MTE, because at least one qualified 34 provider was present at each facility. However, the new indicator determines the number of each type of staff, and determines if they are all present on the day of the survey. o Therefore, at BL, 0/4 of the HFs met this requirement, with the average HF attainment at 48% (meaning the average HF had 48% of its qualified staff present). o At FE 1/4 (Amuca) of the original HFs had all staff present, and all of the new facilities had all staff present, for a total of 5/8 HFs. The project worked with the DHO to continue to improve staff attendance since MTE so that treatment of patients could be as effective as possible and refresher training in IMCI was as effective as possible by ensuring that all HW treating patients were present to benefit from project follow-up mentoring and supportive supervision activities after IMCI topic training is provided. 3. Infrastructure: • At BL, all essential health infrastructure was present on the day of the survey in only 1/4 of the HFs (Ogur), with the average HF attainment at 63%. o The limiting factors were: emergency transportation, which was not available in any of the HFs, an improved, protected water source, which was only available in 2/4 of the HFs (Ogur and Amuca), and auditory and visual privacy, which was also only available in 2/4 of the HFs (Ogur and Aromo). o However, there was 24 hour staff coverage in 3/4 of the original HFs (Aromo, Barapwo, and Ogur). o However, 3/4 of the HFs had electricity from the grid or a generator with fuel and functioning emergency communication (Aromo, Barapwo, and Ogur). o All of the HFs had a useable client latrine. • At FE, all essential health infrastructure was present on the day of the survey in all 4 of the original HFs and all 4 of the new HFs as well. o All 8 facilities had water from a protected water source and auditory and visual privacy (both up from 2/4 at BL). o Also, 24 hour staff coverage was available in 3/4 of the original HFs and only 1 of the new facilities (Walela) o However, electricity from the grid or a generator with fuel was down from 3/4 at BL (all HFs except Aromo), to only 1/8 at FE (Amuca). 4. Supplies: a. Supplies – Child: • At BL only 1/4 of HFs (Barapwo) had all of the essential supplies to support child health, and the average HF attainment was 40% of these essential supplies. 35  ORS pitchers and spoons were only available in Barapwo, and scales and timers were available in only 2/4 (Barapwo and Amuca) of the facilities. • At FE 3/4 of the original HFs (Aromo, Barapwo, and Ogur) and 3/4 new HFs (all but Abala) had all of the essential supplies to support child health, and the average HF attainment was 90% of these essential supplies. b. Supplies-MNC: • At BL, none (0/4) of the HFs had all of the essential supplies to support maternal-newborn health available and the average HF attainment was only 25% of these essential supplies. o None of the facilities had neonatal resuscitation equipment, functioning vacuum extractors, or neonatal wraps for warming. o Only 2/4 of facilities (Aromo and Amuca) had partographs. • At FE, only Amuca (1/4) of the original HFs had all of the essential supplies to support maternal-newborn health available and the average HF attainment was at only 65% for these essential supplies. Also, none of the new HFs had all essential MNC supplies, for a total of only 1/8 HFs and only a 13% average HF attainment. • Aromo, Amuca, and Ogur had neonatal resuscitation equipment and infant scales. • Only Amuca had functioning vacuum extractors or neonatal wraps for warming. • All 4 of the original HFs had functioning partographs, and none of the new HFs had functioning partographs, for a total of 4/8. c. Supplies-ANC: • At BL 0/4 of the HFs had all of the essential supplies to support antenatal care.  No facilities had albumin test strips, and only 1/4 of the HFs (Aromo) had syphilis testing kits and hemoglobin testing kits available.  Also, malaria test kits (Aromo and Amuca), blood pressure equipment (Ogur and Aromo), and syphilis testing kits were available in only 2/4 (Aromo and Amuca) of the facilities.  All HFs other than Barapwo had tetanus toxoid available. • At FE none of the HFs had all of the essential supplies to support antenatal care, with a 75% attainment at the original HFs and only a 44% attainment at all 8 HFs.  Only Aromo and Ogur had hemoglobin testing reagents or malaria test kits.  Ogur had all equipment except functional BP equipment. 36 5. Drugs: a. Child: • At baseline, 1/4 of HFs (Amuca) had all first line medications available, which is defined as: ORS, a first line oral antibiotic for pneumonia, a first line drug for dysentery, a first line antimalarial, and Vitamin A. o This is due to the fact that while 1/4 of facilities had first line drugs for pneumonia (Barapwo) and 50 % of HFs had Vitamin A (Aromo and Amuca), only 1/4 of HFs (Amuca) had first line medications also available for malaria and dysentery. • At FE, all HFs had all first line medications available, the average HF attainment rose from 40% to 100% b. MNC: • At BL, none of the HFs had all of the essential delivery and neonatal drugs present on the day of the survey. In fact, all 3 essential MNC drugs were available in only 1 out of the four HFs each (antibiotics-Amuca; neonatal eye ointment-Ogur; and Oxytocin-Ogur). • At FE 3/4 of the original HFs had all of the essential delivery and neonatal drugs present on the day of the survey, with a 92% attainment rate. Only Ogur did not have eye ointment. None of the new HFs had MNC drugs as they are not performing deliveries and are level 2 HFs. c. ANC: • At BL, only 2/4 HFs had all of the ANC drugs present on the day of the survey. Barapwo was missing 3 of the 4 drugs (Tetanus toxoid, Iron, and ITNs). Amuca was missing antimalarials. Only half of the HFs had ITNs (Aromo and Amuca). • At FE, none of the HFs had all the essential ANC drugs, because there were no ITNs available at any HF. Three out of 4 of the original HFs had all other ANC drugs, with Aromo only missing Iron. Three out of 4 of the new HFs had all other drugs, with Apuce missing Tetanus toxoid, Iron, and Fansidar for IPT. PROCESSES 6. Information Systems a. Child: • At BL, none of the HFs maintained up to date records on sick children under 5 years of age, or up-to-date monthly service data. 37 o The average facility had only 25% of the elements required for having up￾to-date records and monthly service data. o Ogur performed best with 80% of the elements in place. o Aromo and Barapwo HFs did not have any of the elements in place, while Amuca had only 20% of the elements in place. o Barapwo and Amuca used the data to create a wall chart, and Barapwo used the data to create a graph and in discussions, but the other HFs did not use the data at all. • At FE, still 0/4 of the original HFs and also 0/4 of the new HFs maintained up to date records on sick children under 5 years of age, or up-to-date monthly service data. o However, at FE each facility had 80% of the elements required for having up-to-date records and monthly service data, up from 25% at BL. o All facilities (8/8) had complete information in the sick child register, but no HF was using the monthly service data (MSR) which, therefore, is the limiting factor. b. ANC: • At BL, 2/4 HFs (Aromo and Amuca) had up to date records on antenatal care. Aromo and Amuca had all of the elements in place, and Ogur had 2/4 of the elements in place, while Barapwo had none of the elements in place. • At FE, all original HFs had up to date records on antenatal care. Amuca was missing BP information, and Barapwo did not have the delivery register up to date. Of the new HFs, none had delivery information as they do not perform deliveries, and only Abala had an ANC register with complete information. 7. Training: • At BL. only1/4 of the HFs (Ogur) had HWs who reported receiving in-service or pre-service training in both maternal neonatal care and child health in the last 12 months. Training in MNC consists of: training in MNC and at least one other of the trainings listed above in the MNC section. Training in child health consists of: training in Child Health and at least one other of the trainings listed above in the child health section. Ogur received training in diarrhea case management, nutrition, and IPT use. • At FE, all of the original and new HFs had HWs who reported receiving in￾service or pre-service training in both maternal neonatal care and child health within the last 12 months. The various trainings are broken down in the above table. The child health trainings in pneumonia, diarrhea, malaria, ACT, ITN, BF, and IMCI were not received in Abala HF, and Abala and Apuce did not receive nutrition training. 38 8. Supervision: • At BL, only 1/4 of the HFs (Ogur) received any type of external supervision at least once in the 3 months prior to the survey. During this visit, the supervisor observed work in the HF, discussed problems, and checked drug supplies. • At FE, all 4 original HFs and 3/4 new HFs received supervision (Abala did not). PERFORMANCE 9. Utilization of Curative services: The percentage of HF with > 1 sick child encounters per child under the age of 5 in Lira District was not able to be calculated, because no facilities had complete sick child registries available at the time of survey. 10. HW Performance (Assessment): In order for a HF to be considered as having key assessment tasks routinely performed, in greater than 80% (5 or 6 out of the 6 cases observed) of the encounters the HW must perform all of the key assessment tasks. • The baseline found that no facilities (0/4) contained HWs who were routinely performing all 5 key assessment tasks. o Furthermore, the average facility was performing only 21% of the key assessment tasks. o Ogur performed best by performing 63% of the assessment tasks, followed by Barapwo, which performed 21% of their assessment tasks. Aromo and Amuca performed the worst, performing only 4% and 8% of their assessment tasks, respectively. o Assessment tasks which were assessed most frequently were: the ability to feed or breastfeed, asking if the child had been vomiting everything, and checking for the presence of convulsions, which were each asked 29% of the time. o Assessment tasks which were assessed least frequently were: checking the nutritional status and checking for immunizations on the child health card, which were each performed only 4% of the time. • At FE, 3/4 original facilities and all new HFs contained HWs who were routinely performing all 5 key assessment tasks. o Furthermore, at FE the average facility performed 98% of the key assessment tasks, with only Ogur not passing, only because 1 HW did not ask about convulsions in 2/6 children observed. All other performance was acceptable, thus the 98% attainment level. 11. HW Performance (Treatment): 12. At BL, only 1/4 of the HFs (Aromo) provided treatment that was routinely appropriate to the diagnosis. A HW must diagnose and treat the sick child correctly in greater than 80% (5 or 6 out of the 6 cases observed) of the encounters observed in order for a HF to be considered as providing correct treatment. 39 13. At FE, all 8 facilities provided correct treatment appropriate to the diagnosis. 14. HW Performance (Counseling): 15. AT BL, a very weak area in regards to HW performance was in the area of counseling the caretaker regarding the proper method of administering the medicines prescribed. None (0/4) of the HFs were routinely properly instructing caretakers in how to correctly administer drugs prescribed for their child. 16. At MTE, despite training in IMCI, this continued to be a challenge. It was determined that the HW was instructing the caretakers too quickly, and the dispensers were not aiding the caretakers by reiterating the instructions to them. In addition, many mothers were illiterate, and the dispensers were writing the instructions on the medication bag instead of drawing pictures to illustrate the instructions. The project worked on these 3 areas to improve the communication between the HWs and the caretakers, and at FE all 8 HFs were counseling caretakers properly regarding medication directions. 40 ANNEX X B DATA COLLECTION INSTRUMENTS FOCUS GROUP QUESTIONNAIRES Guide for Interviews with Beneficiaries (Mothers and fathers with children younger than 5 years) Date: __________ Interviewer: _____________________________________________ Nino dwe Apeny apeny Community: ______________________ Sub-County: ______________________________ Wang tic Gomola Number of Women/Men: W____/ M___ Wel mon/ coo M C 1. What activities have you participated in with VHTs and mothers groups? Kodi tic ango ame ibedo wunu iye karacel kede VHT kede gurup a toto? Receive home visits Attended community education sessions by VHT Pwonyere me yot kom me kin paco ame VHT oyubu Referrals to health facility by VHT Nwongo balo me wot idakatal ibot VHT Any other contact with VHT Kit rwate okene kede VHT? 2. Does anyone have an example of how the VHT and mothers groups helped this community? Nyo ngatoro tye kede apor ikom kit ame VHT kede group a toto okonyo kede jo me kan? ___________________________________________________________________________ ___________________________________________________________________________ 3. What have you learned from the VHTs or mothers group in the community about how to have a healthy pregnancy? Have you tried any of the practices discussed? Ngo ame VHT kede opwonye amon me kin paco opwonyo wu kede akwako kite me bedo kede yotkom ikare me yac?Kong itemo keto itic ngo ame opwonyo? ___________________________________________________________________________ ___________________________________________________________________________ 4. What have you learned about taking care of newborns? Have you tried any of the practices discussed? Ngo ame opwonyowu kede ite nyango akwako gwoko otino ame pwod onywalo anywala?Kong itemo keto itic ngo ame opwonyo? ___________________________________________________________________________ ___________________________________________________________________________ Final Evaluation Project: Lira District Child Survival Project in Uganda December 2013 Final Evaluation - Annexes 5. What new information about feeding infants and young children have you learned? Have you tried any of the practices discussed? Ngec ango anyen akwako pito otino ame inwongo?Kong itemo keto moro kiken itic kit ame opwonyi kede? ___________________________________________________________________________ ___________________________________________________________________________ 6. How do you know if a child is not growing well? Ingeo ningo ka atin pet ye adongo aber? ___________________________________________________________________________ ___________________________________________________________________________ 7. Why do some children not grow well? What do children who are not growing well need? Pingo otino okene pe dongo aber? Ngo ame otino ame pet ye adongo aber mito? ___________________________________________________________________________ ___________________________________________________________________________ 8. If your child has diarrhea what do you do? What signs do you look for to know you need to go the health center? Anyutene ame ineno me miyi ngeyo ni myero iter atin idakatal? Continue breast feeding, and maintain or increase feeding. Mede kede doto atin, kede mede kede mie cem aromaroma onyo adit akato Treat with ORS/ increase fluids. Mie yat kadukadu ame orubu kede pii/mie gin amata adit Other (En okene?) 9. How can you prevent diarrhea? Yore ango ame itwero gengo kede two cado? ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ 10. If your child has a cough or difficulty breathing what do you do? Ka atini tye kede aola nyo peko iweo mere,myero itim ngo? Early recognition of danger signs Neno con anyut areco ame aola kelo ikom atin Appropriate careseeking Moyo kony me dakatal con Other En okene 11. How can you prevent coughs and difficulty breathing? Yore ango ame iromo gengo kede two aola kede peko me weo atin? ___________________________________________________________________________ ___________________________________________________________________________ 12. Think of the way the community was four years ago, are there any health practices that are different now? Final Evaluation Project: Lira District Child Survival Project in Uganda December 2013 Final Evaluation - Annexes Nen kong imwaki angwen okato angec,nyo tye alokaloka amako yotkom ikin paci ___________________________________________________________________________ ___________________________________________________________________________ 13. Over the past four years, have there been changes in the quality of health care at the Health Centers --------------------------------------------------------------------------------------------------------------------------------------------- ----------------------------------------------------------------------------------------------------------------- If there were improvements, what were they? If things are worse, why is that so? Why do you think the changes happened? Ikare me mwaki angwen ame okato angec,kite me miyo kony me yot kom omede malo idakatal? Ka ee, ngo aber ame rwomere omede? Ka pe,pingo? Itamo ni alokaloka ni otimere pingo? ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ 14. In this community, do you think families are caring for their children differently than they were 4 years ago? In what ways? Why? Iwangtici, wun itamo ni onywal dong tye agwoko otinogi aber ikato imwaki angwen ame okato angec?Iyore mene? Kede pingo? ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ 15. Do you think those changes will continue after the Project ends? Itamo ni alokaloka ni bino mede ameda kadi bedi project ogik? Why? Pingo? ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ------------------------------------------------------------------------------------------------------------------------------- 16. Now that the Child Survival Project is ending, do you think the changes in the community and at the health facility will continue? Aman ame dong project tye akik oko ni, itamo ni alokaloka aber ni bino mede ameda i kin paci kede i dakatal? How will they continue?Iyore mene ame gin bino mede kede? ___________________________________________________________________________ ___________________________________________________________________________ What problems will make that difficult? Kit peki ango ame twero mio mede kede tic bedo atek? ___________________________________________________________________________ ___________________________________________________________________________ How could those problems be resolved? Kite ango ame kodi peki no twero cobere kede? ___________________________________________________________________________ ___________________________________________________________________________ Final Evaluation Project: Lira District Child Survival Project in Uganda December 2013 Final Evaluation - Annexes Guide for Interviews with Beneficiaries who received lessons on ECD (Mothers and fathers with children younger than 5 years) Date: __________ Interviewer: _____________________________________________ Nino dwe Apeny apeny Community: ______________________ Sub-County: ______________________________ Wangtic Gomola Number of Women/Men: W____/ M___ Wel mon/coo M C Remind the parents of the meaning of “ECD” – ECD stands for early childhood development, today we are referring to it as a set of caregiver actions and behaviors that support a child’s first years of development, including feeding, clothing, helping the child learn to speak and learn about the world, seeking health care, providing love, among many other actions. For and Ogur, do you remember attending ECD lessions? Questions for Focus Groups 1. What was the most useful thing you learned from the ECD lessons? Ngo apire tek ame inwongo i pwonyere anyen ikite me gwoko otino? ___________________________________________________________________________ ___________________________________________________________________________ 2. Is there a difference in how you spend time with your child now compared to how you did before you attended ECD lessons? If yes, explain/describe. Nyo tye alokaloka moro ikite ame ibedo kede atini ikareni ka iporo kede ikare ame pwod pe opwonyi kede yore anyen me gwoko otino. Ka ee kong imi apor. ___________________________________________________________________________ ___________________________________________________________________________ 3. Are the children responding differently to you now, e.g. after you attended the ECD lessons? If yes, explain/describe. Nyo itamo ni Otino ni dong aman winyi, ka iporo kede kare ame pwod pe inwongo kede pwony anyen ikite me gwoko gi?Ka ee, kop kit ame gin winyi kede? ___________________________________________________________________________ ___________________________________________________________________________ 4. Have you implemented any changes in hygiene, nutrition or sanitation after you attended the ECD lessons? If yes, explain/describe. Nyo Iketo itic pwony anyen ame ibin inwongo akwako cilo me kom,kede kite me gwoko ka konyere aber iyonge nwongo pwonyere me yore anyen ikite me gwoko otino?Ka ee kobi ___________________________________________________________________________ ___________________________________________________________________________ 5. How do you discipline your child ? Has this always been how you have disciplined? If it changed, why? Kite ango ame ipwonyo kede atin ka otimo bal ?Kit ame didik onwongo ipwonyo kede eno? Ka alokaloka tye iye ,pingo? ___________________________________________________________________________ ___________________________________________________________________________ 6. Is there ever violence between you and your husband, such as hitting, smacking or pushing? Has this always been how you have interacted? If it changed, why? Anaka ni kong gero moro obedo ikin yin kede cwari,calo goyi, dongi, onyo cori? Kit ame didik wun ubedo kede eno?Ka alokaloka tye iye pingo? ___________________________________________________________________________ Final Evaluation Project: Lira District Child Survival Project in Uganda December 2013 Final Evaluation - Annexes ___________________________________________________________________________ 7. Have you noticed any changes in your child’s health, growth or development after you attended the ECD lessons? If yes, what changes? Nyo ineno alokaloka ikom atini iyore me dong, nyo kite me nyang piny iyonge yin inwongo pwonyere anyen ikite me gwoke? Ka ee,alokaloka ango? ___________________________________________________________________________ ___________________________________________________________________________ 8. Is there anything else you wish to share? Onyo tyen cadenoro ame imito miyo wunu? __________________________________________________________________________ __________________________________________________________________________ Final Evaluation Project: Lira District Child Survival Project in Uganda December 2013 Final Evaluation - Annexes Guide for Interviews with Village Health Teams Date: ____________ Interviewer:_______________________________ Nino dwe Apeny apeny Community: ___________________ Sub-County: _____________________ Wangtic Gomola Number of VHTs: Men ___ Women ___ Wel a VHT Coo Mon 1. Intro: Why did you volunteer to work as a VHT? What have been the challenges? Acaki me apeny; Pingo idyere me tic acalo atic me yot kom me kin paco? Peki mene ame inwongo wunu alubere kede ticwu? ___________________________________________________________________________ ___________________________________________________________________________ 2. What topics did the trainings cover? Did you receive any refresher training? What topics did you received? Ikin kare mene ame inwongo wunu kede pwonyere? Kit pwonyere mene me yot ame inwongo wunu? Topics: Wipwony _________ How often: Kare ame powonyo kede Who provided the training____________________ Nga ame opwonyowu? 3. have you ever received any support from community leaders? Kodi kony ango ame inwongo ibot otela wu? If yes, what support have you received? ______________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ 4. What support have you received through visits from MTI staff? From the health staff or health assistants? Kodi kony ango ame inwongo wunu ka otic me medical team olimo wu? Aya ibot dakatal nyo dakatal me cil paco? Frequency Tyen adii What is done Ngo ame otimere? MTI staff Otic me medical team Health facility staff Otic me dakatal 5. Have you seen changes in the communtiy as a result of your work? Final Evaluation Project: Lira District Child Survival Project in Uganda December 2013 Final Evaluation - Annexes Nyo ineno alokaloka iwangtici kan, alubere kede tici? Care practices during pregnancy Kite me gwokere ikare me yac Care of newborns Gwoko atin ame pwod onywalo anywala Feeding infants and young children Doto kede pito otino Preventing diarrhea and respiratory infections Gengo cado kede aola Caring for children with diarrhea and respiratory infections Gwoko atin atye acado kede two aola 6. What have been the most difficult family health practices to change? Kodi tim me yot kom mene ame inwongo tek me loko ikin paci? ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ 7. Do you have any success stories about improving the health care or nutrition of children in your community? Nyo iye kede cadennoro aber akwako kite me rwom me yot kom kede pito otino oyito kede malo iwang tic wu ni? ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ 8. Now that the Child Survival Project is ending, will you continue to work in your community? What activities will you continue to carry out? Who will support you? Aman adong kony me project otum oko ni, pwod ibino mede kede tic iwang tici? Kodi ticene ame ibino mede kede? Nga ame bino konyo kuru wu iye? ___________________________________________________________________________ ___________________________________________________________________________ Are there some activitries you will not be able to continue? Kodi ticene ame itamu ni pe ibno twero mede kede? ___________________________________________________________________________ ___________________________________________________________________________ Which ones? Why? Ticene? Pingo? ___________________________________________________________________________ ___________________________________________________________________________ Final Evaluation Project: Lira District Child Survival Project in Uganda December 2013 Final Evaluation - Annexes Guide for Interviews with Village Health Teams & Peer Educators Who facilitated ECD lessons Date: ____________ Interviewer:_______________________________ Nino dwe Apeny apeny Community: ___________________ Sub-County: _____________________ Wang tic Gomola Number of VHTs: Men ___ Women ___ Wel VHT Coo Mon Number of PEs: Men ___ Women ___ Wel PEs Coo Mon Questions: Apeny 1. Did you notice changes in caregiver’s knowledge, attitudes or behaviors during the course of ECD lessons? If yes, please describe with some examples. Nyo Ineno alokaloka ingec,tam kede kwo a jo a gwoko otino ikare ame ipwonyo gi kede yore anyen me gwoko otino? Ka ee, kong ikobiwa kede apor. ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ Have you noticed changes in caregiver’s attitudes or behaviors that have persisted since the ECD trainings? If yes, please describe some examples. Nyo ineno alokloka me cuny kede kite me kwo ogwok otino ame pe olokere cakere ame opwonyogi?Ka eek kob kede apor ___________________________________________________________________________ ___________________________________________________________________________ 2. Are there unhealthy attitudes or practices that have not changed? If so, what are these attitudes and practices and why do you think this is? Nyo tye kit cuny nyo tim areco akwako yotkom ame pe olokere? Ka tye kodi cuny kede tic areco mene ame pe olokere kede itamo ni pingo pe olokere? ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ - 3. Have you personally made changes in your attitudes or behaviors because of your involvement with the ECD lessons? If yes, please describe. Yin dang ikelo alokaloka itami kede ikwo ni pi bedoacalo apwony me yore anyen me gwoko otino? Ka ee kobi --------------------------------------------------------------------------------------------------------------------------------------- ------------------------------------------------------------------------------------------------------------- 4. In regards to seeking health care when they notice IMCI danger signs, are caregivers now: Akwako kite me yenyo kony me yot kom ka onywal oneno anyut arecu ikareni: Onywal bunyu tero atin, onyo ri kede tero atin, onyo pe tero atin. • more likely _____(# who agree with this) • less likely _____(# who agree with this) • or have not changed _____(# who agree with this) in their practices of seeking health care for their child? Final Evaluation Project: Lira District Child Survival Project in Uganda December 2013 Final Evaluation - Annexes ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ 5. If they are now “more likely” or “less likely” to seek health care for their child, what do you attribute the changes to? Ka gin dong aman otwero bunyere me moyo kony me yotkom otino gi idakatal, ngo ame itamo ni mio gin obunyere me moyo kony pi otino gi ? ___________________________________________________________________________ ___________________________________________________________________________ 7. Is there anything else you would like to share? Nyo itye kede cadenoro onyo tamoro ame itwero nywako kedwa? ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ Final Evaluation Project: Lira District Child Survival Project in Uganda December 2013 Final Evaluation - Annexes Guide for Interviews with Mother Group Leaders Date: __________ Interviewer: _____________________________________________ Nino dwe Apeny apeny Community: ______________________ Sub-County: ______________________________ Wangtic Gombolola Number of Women Interviewed: ________________ Wel mon ame openyo 1. Why did you choose to work as a Mothers Group Leader? What have been the challenges? Pingo idyere me bedo acalo atela atoto me pwonyo kop me yot kom ikin paco? Peki mene ame inwongo alubere kede dyere wu? ___________________________________________________________________________ ___________________________________________________________________________ 2. What kind of health training did you receive over the past year as a Mothers Group Leader? Kodi pwonyere mene me yot kom ame ibin inwongo wunu i mwaka okato angec acalo toto odyere me pwony ikin pacu? ___________________________________________________________________________ ___________________________________________________________________________ 3. Did you receive any refresher training? If yes what topics? --------------------------------------------------------------------------------------------------------------------------------------------- ---------------------------------------------------------------------------------------------------------------- 4. Have you ever received support from community leaders? If so what support? --------------------------------------------------------------------------------------------------------------------------------------------- --------------------------------------------------------------------------------------------------------------- 5. How have you shared health messages with the community? Kit yore ango ame wun olalo kede ngec me yot kom ikin paci? --------------------------------------------------------------------------------------------------------------------------------------------- -----------------------------------------------------------------------------------------------------------------Have you seen changes in the communtiy over the past year and a half? Nyo ineno wunu kit alokaloka moro aber akwako gwok me yotkom i akina pacu iyi mwaka acel kede nucu ame okato angec? Care practices during pregnancy Kite me gwokere ikare me yac Care of newborns Gwoko atin ame pwod onywalo anywala Feeding infants and young children Doto kede pito atin Preventing diarrhea and respiratory infections Final Evaluation Project: Lira District Child Survival Project in Uganda December 2013 Final Evaluation - Annexes Kite me gengo cado kede two aola Caring for children with diarrhea and respiratory infections Kite me gwoko otino ame tye kede two cado kede aola 6. What have been the most difficult family health practices to change? Kodi tim me yotkom ango ame dong inwongo wunu loko obedo atek totwal ikin paco? ___________________________________________________________________________ ___________________________________________________________________________ 7. Do you have any success stories about improving the health care or nutrition of children in your community? Nyo itye kede cadenoro aber ame iromo mio akwako yore me dongo rwom me yotkom kede pit aber pi otino ikin pacu? ___________________________________________________________________________ ___________________________________________________________________________ 8. Now that the Child Survival Project is ending, will you continue to work in your community? What activities will you continue to carry out? Who will support you? Aman a dong projek agik oko ni,ibino wunu mede kede tic wu Iyi akina paco? Kodi ticene ame ibino mede kede tiyo? Itamo ni nga ame bino konyo korwu iye? Are there some activitries you will not be able to continue? Nyo tye tic mogo ame pe ibino romo tiyo wunu? _______________________ ___________________________________________________________________________ Which ones? Why? Ticene kede pingo? ___________________________________________________________________________ Final Evaluation Project: Lira District Child Survival Project in Uganda December 2013 Final Evaluation - Annexes Guide for Interviews with Health Unit Staff Health Facility: Sub-County: Date: __________ Interviewer: Staff Name: Position: Male (#) ____Female (#)_____Time in position at this health facility: 1. What was your greatest success at the health facility this last year? ___________________________________________________________________________ 2. Pease tell us about the training you received through MTI CSP or any other source. What was good about the training and what could be improved? Type of Training Date of Training Training Leader Strengths of Training Suggested Areas for Improvement Have you received any refresher training? If yes what topics? --------------------------------------------------------------------------------------------------------------------------------------------- --------------------------------------------------------------------------------------------------------------------------------------------- ----------------------------------------------------------------------------------- 3. Are there positive changes in your work at this health unit due to MTI CSP project activities? --------------------------------------------------------------------------------------------------------------------------------------------- ------------------------------------------------------------------------------------------------------------------- 4. What have been your greatest challenges at the health facility this last year? --------------------------------------------------------------------------------------------------------------------------------------------- ----------------------------------------------------------------------------------------------------------------- 5. What are additional skills and capacities the facility still needs to implement quality maternal, newborn and child health services? Do you work with VHTs or MLGs? If yes, in what way? ___________________________________________________________________________ ___________________________________________________________________________ 6. What is your opinion of the work done by the VHTs and Mothers Groups in their communities? ----------------------------------------------------------------------------------------------------------------------------- ___________________________________________________________________________ 7. Have you seen any positive changes in health and nutrition practices by families that you think are due to the work of the VHTs and Mothers Groups? ___________________________________________________________________________ ___________________________________________________________________________ Final Evaluation Project: Lira District Child Survival Project in Uganda December 2013 Final Evaluation - Annexes 8. Do VHTs refer patients to your health unit? Are these referrals appropriate? Can you describe a specific instance? ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ 9. Has your health facility received a supportive supervision visit during the past three months? If yes, by whom? ___________________________________________________________________________ ___________________________________________________________________________ 10. Has the HUMC supported your facility within the past year? Can you give specific instances of how they have or have not supported you? --------------------------------------------------------------------------------------------------------------------------------------------- ------------------------------------------------------------------------------------------------------------------ 11. Now that the Child Survival Project is ending, will the improvements continue? How? ---------------------------------------------------------------------------------------------------------------------------- ___________________________________________________________________________ Final Evaluation Project: Lira District Child Survival Project in Uganda December 2013 Final Evaluation - Annexes Guide for Interviews with PDCS Health Facility: Sub-County: Dakatal Gombolola Date: __________ Interviewer: Nino dwe Apeny apeny Number of members interviewed and their positions: Wel jo ame openyo kede rwom gi --------------------------------------------------------------------------------------------------------------------------------------------- --------------------------------------------------------------------------------------------------------------------------------------------- ---------------------------------------------------------- 1. What have you accomplished as the PDCs during the past four years? Ngo ame ityeko tiyo wunu aber a calo iryonget ame loo dakatal iyi mwaki angwen okato angec? ___________________________________________________________________________ ___________________________________________________________________________ 2. Have you received support from MTI staff? If so, what kind of support? Nyo Ibin inwongo wunu kony i bot otic me medical teams? Ka Ee, kodi konyene? --------------------------------------------------------------------------------------------------------------------------------------------- --------------------------------------------------------------------- Is there support you did not receive that would have been helpful? If so, what kind of support? Nyo tye kit konyoro ame pe ibin inwongo wunu ibot medical team ame itamuwunu ni onwongo romo bedo me kony totwal botwu? Ka ee, kodi konyene 3. Has the PDCs collaborated with the HUMCs to improve services? In what ways? Nyo iryonget wu ni obin okubere kede dakatal pi medo rwom me kony me yot kom idakatal kane? Iyore mene? ___________________________________________________________________________ ___________________________________________________________________________ 4. What is your opinion of the work done by the VHTs in their communities? Tamwu tye ni ngo alubere kede tica VHT ikin paci? --------------------------------------------------------------------------------------------------------------------------------------------- --------------------------------------------------------------------------------------------------------------------------------------------- 5. Do you support the VHTs in their work? (Yes or No) ______________________________ Ikonyo wunu kor VHT me tiyo tic gi? If yes, how do you support them? ____________________________________________ Ka ee ,I ikonyo kor gi iyore mene? If no, how could you support them? Ka pe, yore mene ame onwonongo myero ikony kor gi kede? __________________________________________ 6. Have you seen any changes in health and nutrition practices by families that you think are due to the work of the VHTs? If so, what changes? Nyo ineno wunu alokaloka moro itim me yotkom kede kite me pit aber Iyi akina paci, ame itamo wunu ni obedo adwogi me tic aber a VHT? Ka ee, alokaloka mene --------------------------------------------------------------------------------------------------------------------------------------------- --------------------------------------------------------------------------------------------------------------------------------------------- 7. What is your opinion of the work of Mothers Groups? Wun Itamo ningo ikom tic a toto ame odyere me pwony ikin paci? Final Evaluation Project: Lira District Child Survival Project in Uganda December 2013 Final Evaluation - Annexes --------------------------------------------------------------------------------------------------------------------------------------------- --------------------------------------------------------------------------------------------------------------------------------------------- 8. Do you support the Mothers Groups in their work? (Yes or No) Nyo ikonyo wunu kor toto ame odyere me pwony I akina pacu? (ee nyo pe) If yes, how do you support them? Ka ee iyore mene ame ikonyo kor gi kede? If no, how could you support them? Ka pe, yore mene ame iromo konyo kor gi kede? --------------------------------------------------------------------------------------------------------------------------------------------- --------------------------------------------------------------------------------------------------------------------------------------------- Have the health services in parish changed in the last four years? Onyo ineno alokaloka aber me yot kom I parish wu ni iyi akina mwaki angwen okato ni? --------------------------------------------------------------------------------------------------------------------------------------------- --------------------------------------------------------------------------------------------------------------------------------------------- 9. What is your PDC planning to do in this next year? Ngo ame iryonget wuni tye ayikere me timo imwaka abinoni? --------------------------------------------------------------------------------------------------------------------------------------------- --------------------------------------------------------------------------------------------------------------------------------------------- How will you make those things happen? Ibino keto wunu yika wu ni itic ningo? --------------------------------------------------------------------------------------------------------------------------------------------- --------------------------------------------------------------------------------------------------------------------------------------------- What obstacles will you face? Peki mene ame itamo wunu ni ibino rwate wunu kede?_ --------------------------------------------------------------------------------------------------------------------------------------------- --------------------------------------------------------------------------------------------------------------------------------------------- How will you solve those problems? Kite ango ame ibino cobo wunu ked peki no? --------------------------------------------------------------------------------------------------------------------------------------------- --------------------------------------------------------------------------------------------------------------------------------------------- How do you see the PDC working with the VHTs and Mother lead groups now that the project is ending? --------------------------------------------------------------------------------------------------------------------------------------------- --------------------------------------------------------------------------------------------------------------------------------------------- ------------------------------------------------------------------------------------------------------------------ Final Evaluation Project: Lira District Child Survival Project in Uganda December 2013 Final Evaluation - Annexes Guide for Interviews with Health Unit Management Committees Health Facility: Sub-County: Dakatal Gomola Date: __________ Interviewer: Nino dwe Apeny apeny Number of members interviewed and their positions: Wel jo ame openyo kede rwom gi 1. What have you accomplished as the HUMC during the past four years? Ngo ame ityeko tiyo wunu aber a calo iryonget ame loo dakatal iyi mwaki angwen okato angec? ___________________________________________________________________________ ___________________________________________________________________________ 2. Have you received support from MTI staff? If so, what kind of support? Nyo Ibin inwongo wunu kony i bot otic me medical teams? Ka Ee, kodi konyene? ___________________________________________________________________________ ------------------------------------------------------------------------------------------------------------------------------- Is there support you did not receive that would have been helpful? If so, what kind of support? Nyo tye kit konyoro ame pe ibin inwongo wunu ibot medical team ame itamuwunu ni onwongo romo bedo me kony totwal botwu? Ka ee, kodi konyene ___________________________________________________________________________ ------------------------------------------------------------------------------------------------------------------------------- 3. Has the HUMC collaborated with the health facility to improve services? In what ways? Nyo iryonget wu ni obin okubere kede dakatal pi medo rwom me kony me yot kom idakatal kane? Iyore mene? ___________________________________________________________________________ ___________________________________________________________________________ 4. What is your opinion of the work done by the VHTs in their communities? Tamwu tye ni ngo alubere kede tica VHT ikin paci? ___________________________________________________________________________ ------------------------------------------------------------------------------------------------------------------------------- 5. Do you support the VHTs in their work? (Yes or No) ______________________________ Ikonyo wunu kor VHT me tiyo tic gi? If yes, how do you support them? ____________________________________________ Ka ee ,I ikonyo kor gi iyore mene? If no, how could you support them? Ka pe, yore mene ame onwonongo myero ikony kor gi kede? ___________________________________________________________________________ ------------------------------------------------------------------------------------------------------------------------------- 6. Have you seen any changes in health and nutrition practices by families that you think are due to the work of the VHTs? If so, what changes? Nyo ineno wunu alokaloka moro itim me yotkom kede kite me pit aber Iyi akina paci, ame itamo wunu ni obedo adwogi me tic aber a VHT? Ka ee, alokaloka mene ___________________________________________________________________________ ------------------------------------------------------------------------------------------------------------------------------- 7. What is your opinion of the work of Mothers Groups? Wun Itamo ningo ikom tic a toto ame odyere me pwony ikin paci? ___________________________________________________________________________ ------------------------------------------------------------------------------------------------------------------------------- Final Evaluation Project: Lira District Child Survival Project in Uganda December 2013 Final Evaluation - Annexes 8. Do you support the Mothers Groups in their work? (Yes or No) Nyo ikonyo wunu kor toto ame odyere me pwony I akina pacu? (ee nyo pe) If yes, how do you support them? Ka ee iyore mene ame ikonyo kor gi kede? If no, how could you support them? Ka pe, yore mene ame iromo konyo kor gi kede? _________________________________________________________________________---------------------- --------------------------------------------------------------------------------------------------------------------------------------------- --------------------------------------------------------------------------------------------------- 9. Have the services at this Health Center changed in the past 4 years? Nyo alokaloka moro me kony me yotkom onen I dakatal Ikine me mwaki mwaki angwen okato angec In what ways? Iyore mene? ___________________________________________________________ Why do you think they changed? Itamo ni pingo olokere? ___________________________________________________________________________ ------------------------------------------------------------------------------------------------------------------------------- 10. Do you think those changes will continue after the project ends? Itamo ni alokaloka ni bino mede ka projek ogik? ___________________ How could they continue? Otwero mede iyore mene _________________________________________________ Or Why won’t they continue? Nyo pe otwero mede pingo? ___________________________________________________________________________ ------------------------------------------------------------------------------------------------------------------------------- 11. What is your HUMC planning to do in this next year? Ngo ame iryonget wuni tye ayikere me timo imwaka abinoni? ___________________________________________________________________________ ------------------------------------------------------------------------------------------------------------------------------- How will you make those things happen? Ibino keto wunu yika wu ni itic ningo? ___________________________________________________________________________ ------------------------------------------------------------------------------------------------------------------------------- What obstacles will you face? Peki mene ame itamo wunu ni ibino rwate wunu kede?_ _____________________________________________________________________________ ------------------------------------------------------------------------------------------------------------------------------- How will you solve those problems? Kite ango ame ibino cobo wunu ked peki no? ___________________________________________________________________________ ------------------------------------------------------------------------------------------------------------------------------- Final Evaluation Project: Lira District Child Survival Project in Uganda December 2013 Final Evaluation - Annexes Guide for Interviews with Sub-County Chief Sub District: Sub-County: Date: __________ Interviewer: Staff Name: Position: Male (#) ____Female (#)_____ 1. What was the greatest success in your sub-district this last year? ___________________________________________________________________________ 2. Please tell us about the training staff received through MTI CSP or any other source. What was good about the training and what could be improved? Type of Training Strengths of Training Suggested Areas for Improvement 3. Are there positive changes in your work in this sub district due to MTI CSP project activities? ______________________________________________________________________________________ ________________________________________________________________ 4. What have been your greatest challenges in the sub district this last year? ___________________________________________________________________________ 5. What are additional skills and capacities the sub district still needs to implement quality maternal, newborn and child health services? ___________________________________________________________________________ ___________________________________________________________________________ 6. What is your opinion of the work done by the VHTs and Mothers Groups in their communities? ___________________________________________________________________________ ___________________________________________________________________________ 7. Have you seen any positive changes in health and nutrition practices by families that you think are due to the work of the VHTs and Mothers Groups? ___________________________________________________________________________ ___________________________________________________________________________ 8. Do VHTs refer patients to the health units? Are these referrals appropriate? Can you describe a specific instance? (use if seems appropriate) ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ Final Evaluation Project: Lira District Child Survival Project in Uganda December 2013 Final Evaluation - Annexes 9. Are you, or others in your office, involved in conducting Supportive Supervision visits to health facility staff? If yes, how many visits were conducted in this last quarter? ___________________________________________________________________________ Do you think these visits will continue after the project is complete? How can that happen?_________________________________________________________________ ___________________________________________________________________________ 10. What do you see as the role of the HUMCs in relation to the health facilities? What are their advantages and disadvantages? Can you give specific instances of how they have or have not supported health facilities? ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ 10. Now that the Child Survival Project is ending, will the improvements continue? How? ___________________________________________________________________________ ___________________________________________________________________________ __________________________________________________________________________ Final Evaluation Project: Lira District Child Survival Project in Uganda December 2013 Final Evaluation - Annexes ANNEX XI. SOURCES OF INFORMATION List of Documents Reviewed 1) ◦ Project Detailed Implementation Plan 2) ◦ Project annual and midterm evaluation reports 3) ◦ USAID and Ministry of Health background documents  USAID Global Health Initiative Uganda A Strategy for Accelerating Reductions in Maternal and Neonatal Mortality, 2011  the Uganda Ministry of Health Road Map to accelerate Reduction of Maternal and Neonatal Morbidity and Mortality (2007)  Operational Framework for Nutrition in the National Child Survival Strategy  Uganda Demographic and Health Survey 2011 4) Social and behavior change materials  IYCF National Counseling Cards for Community Volunteers  Safe Motherhood Flip Charts  Village Health Team Strategy and Operational Guidelines and training materials. 5) Final Rapid Health Facility Assessment 6) Final Household Knowledge, Practice and Coverage (KPC) Survey Report. Final Evaluation Report: Lira District Child Survival Project in Uganda December 2013 Final Evaluation - Annexes List of Persons Interviewed and Contacted during the Final Evaluation Interviews with Mothers of Children 0 – 23 Months Sub-County Community Participant’s Name Aromo High performing Community (Atapara dam-walela) Alum Harriet Amongi Betty Auma Susan Mary Rose Joan Opio Adong Dokas Betty Eling Adong Evaline Betty Akello Aciro Scovia Low performing Community ( Okio walela- teabolo ) Betresy Ipali Jesca Stella Dockas Sikobiya Susan Dorcus Agea Acceng Mirriam Akello Janet Apio Molly Ogur High performing Community (Baradanga Adwoa-coo rom ps) Amoni Harriet Juspin Omara Adong Caroline Apio Teddy Aciro Christine Agness Ogwang Akello Sarah Esther ogwang Akite Betty Adong Susan Low performing Community (Angolocom- Ogur) Atim Everly Adong Stella Aguti Semmy Acio Sarah Apio Eunice Hope Akullo Lillian Acheng Florence Agaba Faith Auma Joy Akello Parah Lira High performing Community (Telela A￾Omito) Akello Lucy Atala Susan Phoebe Anyii Susan Arwata Final Evaluation Report: Lira District Child Survival Project in Uganda December 2013 Final Evaluation - Annexes Jenet Abeja Kilara Okila Agnes Akello Santa Opio Sarah Edonga Doris Oyuku Low performing Community (Olengobir -Amuca) Ongom Francis Florence Adol Atim Jannet Claire Odongo Betty Arot Santa Ongola Joyce Orima Flow Adol Jenet Ojok Teddy Ebonga Christine Obot Interviews with Fathers of Children 0 – 23 Months Sub-County Community Participant’s Name Aromo High performing Community (Wigweng –Odoro ) Munu Denish Alobo Tom Oroc Dickens Odongo Bonny Okidi James Okello Geoffrey Oton Geoffrey Ogwang Jimmy Omara Jaspher Opio Jaspher Low performing Community (Ajuri – Apua) Ongom Francis Ocen Bosco Omara Aloysious Odyek Alex Ojuka Bonny Obong Samson Opio James Olwa Tonny Otyang Jimmy Obol James Ogur High performing Community (Baradanga Adwoa￾coorom ps) Opio Peter Okullu Patrick Okello Sammuel Otyang Anjilo Obua Joel Omara Ambrose Final Evaluation Report: Lira District Child Survival Project in Uganda December 2013 Final Evaluation - Annexes Atum Yubentino Ogwang Yubentino Adi Albino Atyang Martin Low performing Community (Angolocom) Atim Kizito Omara Francis Anyima Felix Omara Alfred Apita Denish Ocen Alfred Opio Innocent Ekia Tom Okello Tom Lwak Otoro Nelson Lira High Peforming Community (Telela A) Ongom Francis Oluk James Anena Sam Opio Alex Anyonga Vincent Ouni Emmanuel Okello Joel Elem Peter Olet Tonny Low performing men (Olengobir Amuca) Orima Santo Opong George Opio Patrick Otim Fedinan Alobo Ceaser Emuna Maxuel Ogwang Tonny Okello Mike Etuke Ronny Ocim Jimmy Final Evaluation Report: Lira District Child Survival Project in Uganda December 2013 Final Evaluation - Annexes Interviews with Mother Group Leaders Sub-County Community Participant’s Name Aromo Otara Parish Sophia Okello Betty obonyo Amongi Christine Jenifer Omara Dorine Otim Amony Brenda Mary Ogwal Ketty Anam Acio Anna Colline Todo Acen Polly Ogur Acelela , Angolocom and Orit Parish Judith Awio Dorine cong Costa oyom Betty Todi Apio Shara Evaline opeto Akullo Scovia Akello Esther Angom Grace Akullo Eunice Lira Barapwo Parish Molly Omara Eunice Ocen Harriet Okaka Enesta Acili Kevin Ogwang Sarah Odur Monika Olwa Agness Okello Alit Betty Jane Ote Interviews with VHTs/PEs Sub-County Community Participant’s Name Aromo Owene Omara Jimmy Aron Odic Tom Ojok Jaspher Agonga Charles Juspanti Omara Owot Moses Esther Aoka Celestiono Okodi Aleny Jimmy Opido Richard Tonny Ogur Orit Acai Silvesto Todi Louis Final Evaluation Report: Lira District Child Survival Project in Uganda December 2013 Final Evaluation - Annexes Egonga Geoffrey Ameny Linous Okello Alex Odongo Bosco Onyona C.P Ojok Joel Jenet Ocen Auma Flow Odongo Joel Bob Susan Okabo Hellen Omara Akullo Scovia Otim Lawrence Okello Francis Angom Eunice Atim Gloria Oluk George Opio Jaspher Lira Barapwo Mido Aguma Esther oming Ogwok Alfred Lucy Awidi Collin Otim Grace Dengo Otit Charles Ongol Tonny Omondi Jimmy Ocen James Conny Onapa Apio Dorcus Jenet Onyinge Vicky Okello Sarah Odur Akello Betty Obua Amos Interviews with HUMCs Sub-County Community Participant’s Name Aromo Walela Olinga Paskol Atala Rose Atula C.V Atim Christine Odic Tom Esther Aoko Okello Augustine Ogur Akangi H/C11 Okello Joe Toli A Eunice Adiro Judith Opio Tonny Okello Mary Francis R F Olello Final Evaluation Report: Lira District Child Survival Project in Uganda December 2013 Final Evaluation - Annexes Egwali Fred Ogwang Patrick Obura Desmond Jacob Oweta Opio Patrick Amule Christine Okello Geoffrey Opio Richard Omara Bob Ojok Pious Odongo Moses Lira Barapwo Otto Click Okany George Okello George Ochen James Ogwal Moses Ebila Jino Sophia Atyeno Interviews with Health Facility Staff Sub-County Community Participant’s Name Aromo Walela Olinga Pascol Olwoc Jimmy Atim Christine Odich Tom Atworo James Esther Aoko Akullo Ketty Ogweng George Abonyo Dorothy Tino Beatrice Ogur Ogur Okullo Geoffrey Agoro Acio Charity Nandudu Babra Amongi Susan Akello Dorcus Middy Sophia Akello Shphie Ogwang James Doris Amodo Anying Betty Lira Barapwo Acio Catherine Ogang Moses Adong Agness Otyek Godtry Akullo Caroline Ejeng Sarah Atwongo Harriet Final Evaluation Report: Lira District Child Survival Project in Uganda December 2013 Final Evaluation - Annexes Ojok Johnson Okello George Akun Beatrice Interviews with Parish Development Council Members Sub-County Community Participant’s Name Aromo Walela Munu Denis Alele Filbert Odongo Gilbert Odongo Bonny Opio Lawrence Ajal Nelson Opio Jaspher Okello James Odoc Joe Ogwang Dicken Ogur Ogur Obong Moris Okello Joseph Awio Thomas Opio Tonny Akedo Christine Oloo Willy Okello Moses Ojok Tonny Interviews with Senior Assistant Secretaries of Sub-Counties (Sub-County Chief) Sub-County Name Aromo Emmanuel Komakech Ogur Eunice A. Toli Lira George Okany (Acting Sub-County Chief) Key Informant Interviews Name Title Agency Laura Peterson Executive Director Hands to Hearts International Lydia Akulo Project Manager Child Survival Program, MTI George Aguze Project M & E Officer Child Survival Program, MTI Dan Ward Regional Director, Africa MTI, Africa Region Mike O’brien Africa Health Programs Advisor MTI, Africa Region Allan Obore Human Resources/ Administration MTI, Uganda Final Evaluation Report: Lira District Child Survival Project in Uganda December 2013 Final Evaluation - Annexes ANNEX XIII. STATEMENT OF DIFFERENCES Medical Teams International does not wish to state any differences with the final evaluation report. We find that the report is comprehensive and constructive. Final Evaluation Report: Lira District Child Survival Project in Uganda December 2013 Final Evaluation - Annexes ANNEX XIV. EVALUATION TEAM MEMBERS, ROLES, AND THEIR TITLES Names of People who Participated in Qualitative Assessments September 9th -16th, 2013 No Name Title Role in the Survey 1. Lydia Akulo CSP Program Manager Planning and Coordinating evaluation activities 2. George Aguze Monitoring and Evaluation Officer Review of tools, data entry, technical support to team 3. Arach Marry CSP Community Outreach Coordinator Data collector 4. Odur Joel CSP Maternal Child Health Coordinator Data collector 5. Apiyo Christine CSP Maternal Child Health Mentor Data Collector 6. Achan Doreen CSP Village Health Team Mobilizer Data Collector 7. Omodo James CSP Village Health Team Mobilizer Data Collector 8. Apili Harriet CSP Village Health Team Mobilizer Data Collector 9. Ojungu Tobias CSP Village Health Team Mobilizer Data Collector 10. Ojok Nixson CSP Village Health Team Mobilizer Data Collector 11. Ajik Susan CSP Village Health Team Mobilizer Data Collector 12. Olobo Denis Enrolled Comprehensive Nurse (MTI Pader) Data Collector 13. Apenyo Kenedy Medical Clinical Officer (MTI Pader) Data Collector 14. Okwir Jasper Enrolled Comprehensive Nurse (MTI Ogur) Data Collector 15. Acheng Teddy Intern Data Collector 16. Awal Molly Logistic Assistant Data Collector 17. Opio Peter CSP Driver Driver 18. Okello Peter MTI Driver Driver 19. Angulu Tonny Hired Driver Driver 20. Mary Helen Carruth MTI Senior Advisor Maternal Child Health Evaluation Co-facilitator 21. Sue Leonard Independent Consultant External Consultant Final Evaluation Report: Lira District Child Survival Project in Uganda December 2013 Final Evaluation - Annexes ANNEX XV. FINAL OPERATIONS RESEARCH REPORT No Operations Research was conducted within the Lira Child Survival Project Final Evaluation Report: Lira District Child Survival Project in Uganda December 2013 Final Evaluation - Annexes ANNEX XVI. OPERATIONS RESEARCH BRIEF No operations research was conducted within the Lira Child Survival Project Final Evaluation Report: Lira District Child Survival Project in Uganda December 2013 Final Evaluation - Annexes Final Evaluation Stakeholder Debrief Medical Teams International The FE provides an opportunity for all project stakeholders to take stock of accomplishments and lessons learned to date and to listen to beneficiaries at all levels. The FE Report will be used by the following audiences as a source of evidence to help inform decisions about future program designs and policies:  Lira DHO, health facility staff and Health Unit Management Committees  Uganda Ministry of Health  Village Health Teams and Mothers Group Volunteers  Hands to Hearts International  USAID CSHGP and Uganda USAID Mission  Medical Teams International HQ and field staff 1. To what extent did the project accomplish objectives stated in the DIP? 2. To what extent was the project implemented as planned? What changes were made to the planned implementation, and why were those changes made? 3. How were gender considerations addressed through the project? Are there any specific gender-related outcomes? 4. Which elements of the project have been or are likely to be sustained or expanded?  Four year Child Survival Project  Implemented in Aromo, Ogur and Lira sub-counties of Lira District, Erute North Sub District  Implemented in collaboration with Ministry of Health, Lira District Health Office; and Hands to Hearts International  Objective 1: Communities assume responsibility for their own health through strengthened community capacity.  Objective 2: Improved health (C-IMCI) and child care (Early Childhood Development) behaviors among mothers of children under five years of age.  Objective 3: Improved quality of health facility services through strengthened IMCI and MNC capacity.  Objective 4: Strengthened institutional capacity of MTI and the Lira District Health Office to implement effective and efficient child survival activities.  Desk Review of Secondary Data: ◦ Detailed Implementation Plan ◦ Program Reports ◦ USAID and Ministry of Health background documents ◦ Project IEC materials  Qualitative Data: ◦ Interviews with: community volunteers and project beneficiaries, ◦ Health Unit Management Committees, Parish Development Committees, health staff, and District Health Team members and ◦ Director of Hands to Hearts International  Quantitative Data:  Health Facility Assessment and  Household Knowledge, Practice and Coverage (KPC) survey.  Data Analysis:  Quantitative and qualitative findings analyzed by MTI field and HQ staff, District Health Team members, and health facility staff  Objective 1: Strengthened community capacity – significant improvement in indicators  Supported the 5 year District Development Plan  Worked through the established governmental system: Local Council (LC1), Parish Development Council (PDC), Sub￾County, Health Unit Management Committees (HUMC), Health Facilities II, III, and IV and the District Health Office (DHO)  VHTs – trained, before had limited skills. Communities have two VHTs; are the primary health person in the community; Mother Leader Groups (MLG) support them.  Objective 1: Mother Leader Groups – Key to success of program  Originally 300 were trained; after mid-term added 1128 more, total of 1428 in 282 villages  96 Mother Leaders received a TOT, then trained the remaining mothers  Function like care groups – do HH level health education, group sessions, work closely with VHTs, HFs  Developed Community Theater Groups to dramatize health messages throughout the project area  Objective 1: Health Facility  Routine meetings with VHTs, MLGs to discuss health technical issues e.g. appropriateness of referrals; communication and work flow problems e.g. mothers not bringing cards to clinic, health staff being rude to patients  HUMCs closely monitor work of clinic (counting drugs when arrive, care for physical facility)  PDCs may be 2-3 in HF area. Also participate in stock taking of incoming drugs; maintain communication between HF and community.  Objective 2: Improved health (C-IMCI) Indicator Baseline Final DHS (North) Underweight 27.7 20.0 12.3 EBF 73.6 88.2 Uganda 41.0 IBF – within 1 hour 29 78.9 38.4 Minimum Appropriate Feeding 23.1 48.9 38.4 ANC four or more visits 35.3 57.8 Birth attended by skilled personnel 35.3 83.9 51.9 Maternal danger signs post natal 2.0 51.1 Immunizations complete by 12 mo. 15.5 56.0 (23 m) 49.0 Chest related cough to health clinic 57.8 74.2 80.5  Objective 2: Early Childhood Development (ECD)  Peer Educator (PE) volunteers, 2 per village, provided basic child development information at HH level along with key health messages Indicator Baseline Final Mothers providing cognitive stimulation 38.0 76.3 Mothers telling a story, naming objects, singing a song 22.7 64.4 Mothers singing while feeding 57.7 83.6  Gave parents “permission” to play with and relate to their children  First half of the project, implementation was scattered; last half, it was more intense and concentrated in one sub￾county;  Results were impressive across all three sub-counties with 25-40% gains in specific EDC indicators;  In addition to indicator changes, all community level focus groups reported: o Better relationship between spouses, o More love for the child, o Child responds more positively to parent, o Less domestic violence.  Changes in implementation ◦ Number of MLG members increased from 300 to 1148 after the MTE to better reach HHs with health education ◦ PDCs began monitoring and supervision of project activities to increase community involvement in management ◦ ECD interventions focus on one sub-county only after MTE ◦ Immunization LOE returned to 10% from 55% before MTE  Gender Issues Addressed ◦ Man and woman team as peer educators for ECD to be sure to reach whole population with messages; ◦ MLGs formed to reach mothers as peers as most VHTs are men; ◦ Encouraging male involvement in ANC and PMTCT  Integration of political and technical systems ◦ LCs and PDCs attended VHT and MGL training sessions on their own initiative; ◦ Representatives from each LC are on PDC; ◦ PDCs work with HUMCs to monitor and support the health facility; ◦ HF staff are on the HUMC; ◦ PDCs and HUMCs lobby, plan and report to the Sub￾County.  LCs are now enacting “bylaws” in communities requiring improved health behaviors (digging pit latrines, attending immunization clinic)  Parish Development Committees meet monthly with VHTs and also with MLGs along with HF staff to discuss issues/ problems and emerging health concerns;  Sub-County Chiefs go to health facilities sometimes weekly and to communities with HF In-Charge or staff;  Health planning starts with communities, through the LC1, contributing their issues/concerns to the PDCs who develop their priorities for the year and submit to the Sub￾County who submit their plans to the District. All activities must fit into the plan;  LC1s are involved in health issues in the community, working closely with VHTs and MLGs and serve on the PDC;  Relationships between communities and HF staff have improved and routine PDC meetings help resolve new/continuing issues – HF staff are friendly and do not harass  Indicators are improved from baseline and generally have met targets but need to be better according to HFs  Volunteers are committed – concerned that may not have referral forms, worry they will not be “kept in the loop”  District Health Officer assures they will continue to use the referral forms, that communication with them will continue, that they are the first line of health intervention and are critical to improvement of health in village.  Project Effect ◦ Strengthened the existing system, ◦ Provided training where needed (HF staff, VHTs, MLGs); Welcomed other leaders to trainings ◦ Developed and institutionalized lines of communication and monitoring systems from community to sub-county ◦ Closely monitored and corrected technical competence from community to the HF  Use ECD in future programming  Use MLGs, or HH level health education and mentoring, in future programming  Phase out of community level work at least in last 6 months of project. Project was not strong at mid-term therefore was still heavy into implementation toward the end of the project and could not phase out of communities earlier. Child Survival and Health Grants Program Project Summary Oct-15-2013 Medical Teams International (Uganda) General Project Information Cooperative Agreement Number: GHS-A-00-09-00012 MTI Headquarters Technical Backstop: Mary Helen Carruth MTI Headquarters Technical Backstop Backup: Janis Lindsteadt Field Program Manager: Lydia Akulo Midterm Evaluator: Judiann McNulty Final Evaluator: Headquarter Financial Contact: Pamela Blikstad Project Dates: 9/30/2009 - 9/29/2013 (FY2009) Project Type: New Partner USAID Mission Contact: Janex Kabarangira Project Web Site: www.medicalteams.org Field Program Manager Name: Lydia Akulo (Project Manager) Address: PO Box 1095 Lira Uganda Phone: 011-256-0772-696665 Fax: E-mail: lakulo@medicalteams.org Skype Name: Alternate Field Contact Name: Micheal O'brien (Africa Health Advisor) Address: PO Box 26073 Kampala Uganda Phone: 011-256-0789-817876 Fax: E-mail: mobrien@medicalteams.org Skype Name: Grant Funding Information USAID Funding: $1,499,646 PVO Match: $511,133 General Project Description Medical Teams International (MTI), a 2009 New Partner category grantee, is implementing the Lira District Child Survival Project in northern Uganda. The project goal is to reduce child morbidity and mortality through sustainable improvements in preventive maternal and child health behaviors and utilization of strengthened health services in Erute North Sub-District of Lira District. The capacity of Village Health Teams (VHTs) in community integrated management of childhood illness (C-IMCI) will be strengthened using a social and behavior change approach, a structured referral system between VHTs and local health facilities will be established, and facility staff will be trained in IMCI and also receive on-the-job mentoring. The project will incorporate Early Childhood Development (ECD) activities to enhance the impact and sustainability of project interventions. Integration of ECD will further improve health status by promoting positive caregiving practices in the earliest years of life to affect all domains (cognitive, physical, language, and social/emotional). Project Location Latitude: 2.16 Longitude: 32.93 Project Location Types: Rural Levels of Intervention: Health Center Health Post Level Community Province(s): Lango Sub-Region, Northern Region District(s): Lira District Sub-District(s): Lira, Aromo, and Ogur Sub-counties of North Erute County Operations Research Information There is no Operations Research (OR) component for this Project. Partners Lira District Health Office (Collaborating Partner) $0 Project communities (Collaborating Partner) $0 Strategies Social and Behavioral Change Strategies: Community Mobilization Interpersonal Communication Health Services Access Strategies: Implementation in a geographic area that the government has identified as poor and underserved Health Systems Strengthening: Supportive Supervision Referral-counterreferral system development for CHWs Review of clinical records (for quality assessment/feedback) Strategies for Enabling Environment: Building capacity of communities/CBOs to advocate to leaders for health Tools/Methodologies: BEHAVE Framework Rapid Health Facility Assessment LQAS Capacity Building Local Partners: Dist. Health System Health Facility Staff Interventions & Components Control of Diarrheal Diseases (20%) - Hand Washing - ORS/Home Fluids - Feeding/Breastfeeding - Care Seeking - Case Management/Counseling IMCI Integration CHW Training HF Training Immunizations (10%) - Classic 6 Vaccines - Mobilization IMCI Integration CHW Training HF Training Infant & Young Child Feeding (20%) - ENA - Comp. Feed. from 6 mos. - Cont. BF up to 24 mos. - Maternal Nutrition - Promote Excl. BF to 6 Months IMCI Integration CHW Training HF Training Maternal & Newborn Care (25%) - Recognition of Danger signs - Newborn Care - Post partum Care - Integation. with Iron & Folic Acid - Normal Delivery Care - Birth Plans - Emergency Transport - Neonatal Vitamin A - Kangaroo Mother Care (skin to skin care) - AMTSL IMCI Integration CHW Training HF Training Pneumonia Case Management (25%) - Case Management Counseling - Recognition of Pneumonia Danger Signs IMCI Integration CHW Training HF Training Operational Plan Indicators Number of People Trained in Maternal/Newborn Health Gender Year Target Actual Female 2010 0 Female 2010 148 Male 2010 478 Male 2010 0 Female 2011 150 Female 2011 474 Male 2011 462 Male 2011 480 Female 2012 474 Female 2012 1684 Male 2012 532 Male 2012 462 Female 2013 1694 Female 2013 1752 Male 2013 612 Male 2013 538 Number of People Trained in Child Health & Nutrition Gender Year Target Actual Female 2010 0 Female 2010 162 Male 2010 183 Male 2010 0 Female 2011 160 Female 2011 474 Male 2011 462 Male 2011 195 Female 2012 474 Female 2012 1684 Male 2012 532 Male 2012 462 Female 2013 1694 Female 2013 1752 Male 2013 612 Male 2013 538 Number of People Trained in Malaria Treatment or Prevention Gender Year Target Actual Female 2010 134 Female 2010 0 Male 2010 426 Male 2010 0 Female 2011 0 Female 2011 0 Male 2011 0 Male 2011 0 Female 2012 0 Female 2012 0 Male 2012 0 Male 2012 0 Female 2013 0 Female 2013 0 Male 2013 0 Male 2013 0 Locations & Sub-Areas Total Population: 124,379 Target Beneficiaries Uganda - MTI - FY2009 Children 0-59 months 25,498 Women 15-49 years 25,498 Beneficiaries Total 50,996 Rapid Catch Indicators: DIP Submission Sample Type: 30 Cluster Indicator Numerator Denominator Percentage Confidence Interval Percentage of mothers with children age 0-23 months who received at least two Tetanus toxoid vaccinations before the birth of their youngest child 227 300 75.7% 11.0 Percentage of children age 0-23 months whose births were attended by skilled personnel 106 300 35.3% 8.6 Percentage of children age 0-5 months who were exclusively breastfed during the last 24 hours 39 53 73.6% 26.0 Percentage of children age 6-23 months who received a dose of Vitamin A in the last 6 months: card verified or mother’s recall 141 201 70.1% 13.2 Percentage of children age 12-23 months who received a measles vaccination 124 161 77.0% 15.0 Percentage of children age 12-23 months who received DTP1 according to the vaccination card or mother’s recall by the time of the survey 140 161 87.0% 15.3 Percentage of children age 12-23 months who received DTP3 according to the vaccination card or mother’s recall by the time of the survey 137 161 85.1% 15.3 Percentage of children age 0-23 months with a febrile episode during the last two weeks who were treated with an effective anti-malarial drug within 24 hours after the fever began 55 220 25.0% 8.7 Percentage of children age 0-23 months with diarrhea in the last two weeks who received oral rehydration solution (ORS) and/or recommended home fluids 51 108 47.2% 16.0 Percentage of children age 0-23 months with chest-related cough and fast and/or difficult breathing in the last two weeks who were taken to an appropriate health provider 85 147 57.8% 14.7 Percentage of households of children age 0-23 months that treat water effectively 34 300 11.3% 5.2 Percentage of mothers of children age 0-23 months who live in households with soap at the place for hand washing 255 300 85.0% 11.2 Percentage of children age 0-23 months who slept under an insecticide-treated bednet (in malaria risk areas, where bednet use is effective) the previous night 154 300 51.3% 9.9 Percentage of children 0-23 months who are underweight (-2 SD for the median weight for age, according to the WHO/NCHS reference population) 83 300 27.7% 7.8 Percentage of infants and young children age 6-23 months fed according to a minimum of appropriate feeding practices 56 242 23.1% 8.1 Percentage of mothers of children age 0-23 months who had four or more antenatal visits when they were pregnant with the youngest child 106 300 35.3% 8.6 Percentage of mothers of children age 0-23 months who are using a modern contraceptive method 100 300 33.3% 8.4 Percentage of children age 0-23 months who received a post-natal visit from an appropriately trained health worker within two days after birth 49 300 16.3% 6.2 Rapid Catch Indicators: Mid-term Sample Type: LQAS Indicator Numerator Denominator Percentage Confidence Interval Percentage of mothers with children age 0-23 months who received at least two Tetanus toxoid vaccinations before the birth of their youngest child 146 198 73.7% 6.1 Percentage of children age 0-23 months whose births were attended by skilled personnel 100 198 50.5% 7.0 Percentage of children age 0-5 months who were exclusively breastfed during the last 24 hours 131 198 66.2% 6.6 Percentage of children age 6-23 months who received a dose of Vitamin A in the last 6 months: card verified or mother’s recall 110 198 55.6% 6.9 Percentage of children age 12-23 months who received a measles vaccination 160 198 80.8% 5.5 Percentage of children age 12-23 months who received DTP1 according to the vaccination card or mother’s recall by the time of the survey 177 198 89.4% 4.3 Percentage of children age 12-23 months who received DTP3 according to the vaccination card or mother’s recall by the time of the survey 144 198 72.7% 6.2 Percentage of children age 0-23 months with a febrile episode during the last two weeks who were treated with an effective anti-malarial drug within 24 hours after the fever began 136 198 68.7% 6.5 Percentage of children age 0-23 months with diarrhea in the last two weeks who received oral rehydration solution (ORS) and/or recommended home fluids 104 198 52.5% 7.0 Percentage of children age 0-23 months with chest-related cough and fast and/or difficult breathing in the last two weeks who were taken to an appropriate health provider 168 198 84.8% 5.0 Percentage of households of children age 0-23 months that treat water effectively 20 198 10.1% 4.2 Percentage of mothers of children age 0-23 months who live in households with soap at the place for hand washing 173 198 87.4% 4.6 Percentage of children age 0-23 months who slept under an insecticide-treated bednet (in malaria risk areas, where bednet use is effective) the previous night 88 198 44.4% 6.9 Percentage of children 0-23 months who are underweight (-2 SD for the median weight for age, according to the WHO/NCHS reference population) 32 198 16.2% 5.1 Percentage of infants and young children age 6-23 months fed according to a minimum of appropriate feeding practices 83 198 41.9% 6.9 Percentage of mothers of children age 0-23 months who had four or more antenatal visits when they were pregnant with the youngest child 99 198 50.0% 7.0 Percentage of mothers of children age 0-23 months who are using a modern contraceptive method 60 198 30.3% 6.4 Percentage of children age 0-23 months who received a post-natal visit from an appropriately trained health worker within two days after birth 44 198 22.2% 5.8 Rapid Catch Indicators: Final Evaluation Sample Type: 30 Cluster Indicator Numerator Denominator Percentage Confidence Interval Percentage of mothers with children age 0-23 months who received at least two Tetanus toxoid vaccinations before the birth of their youngest child 322 360 89.4% 4.5 Percentage of children age 0-23 months whose births were attended by skilled personnel 302 360 83.9% 5.4 Percentage of children age 0-5 months who were exclusively breastfed during the last 24 hours 82 93 88.2% 9.3 Percentage of children age 6-23 months who received a dose of Vitamin A in the last 6 months: card verified or mother’s recall 203 238 85.3% 6.4 Percentage of children age 12-23 months who received a measles vaccination 138 168 82.1% 8.2 Percentage of children age 12-23 months who received DTP1 according to the vaccination card or mother’s recall by the time of the survey 163 168 97.0% 3.6 Percentage of children age 12-23 months who received DTP3 according to the vaccination card or mother’s recall by the time of the survey 154 168 91.7% 5.9 Percentage of children age 0-23 months with a febrile episode during the last two weeks who were treated with an effective anti-malarial drug within 24 hours after the fever began 106 206 51.5% 9.7 Percentage of children age 0-23 months with diarrhea in the last two weeks who received oral rehydration solution (ORS) and/or recommended home fluids 52 85 61.2% 14.7 Percentage of children age 0-23 months with chest-related cough and fast and/or difficult breathing in the last two weeks who were taken to an appropriate health provider 66 89 74.2% 12.9 Percentage of households of children age 0-23 months that treat water effectively 149 360 41.4% 7.2 Percentage of mothers of children age 0-23 months who live in households with soap at the place for hand washing 347 360 96.4% 2.7 Percentage of children age 0-23 months who slept under an insecticide-treated bednet (in malaria risk areas, where bednet use is effective) the previous night 210 360 58.3% 7.2 Percentage of children 0-23 months who are underweight (-2 SD for the median weight for age, according to the WHO/NCHS reference population) 72 356 20.2% 5.9 Percentage of infants and young children age 6-23 months fed according to a minimum of appropriate feeding practices 129 264 48.9% 8.5 Percentage of mothers of children age 0-23 months who had four or more antenatal visits when they were pregnant with the youngest child 208 360 57.8% 7.2 Percentage of mothers of children age 0-23 months who are using a modern contraceptive method 101 360 28.1% 6.6 Percentage of children age 0-23 months who received a post-natal visit from an appropriately trained health worker within two days after birth 147 360 40.8% 7.2 Rapid Catch Indicator Comments ANNEX XIX A. MTI UGANDA OCA ACTION PLAN Issues Action Status at Mid-Term Status at Final Succession Planning 1. CD to bring member of management team to lobby and advocate for funding 2. Establish delegation policy 3. Provide equal opportunity for managers to serve as designated officer in charge 4. Create awareness among staff and ensure delegation is communicated to staff 5. Build capacity of management team; invite/hire reputable consultants to present to and/or train the management team; managers attend short courses e.g. U.M.I. and L.D.C. While delegation is practiced and an OIC is appointed as the situation arises, an overall plan and policy is yet to be developed. Progress has been made in addressing three of the five actions planned for succession planning • Member of management team is in place to lobby and advocate for funding. • Delegation policy is now in place. Delegation was practiced for 5 months in early 2013 when the MTI Country Director was out on sick leave. • Delegation plan communicated to all MTI staff. Final Evaluation Report: Lira District Child Survival Project in Uganda December 2013 Final Evaluation - Annexes Staff Salary & Benefits Policy 1. Adopt a salary and benefits policy 2. Revise the annual salary increase policy 3. Make uniform health insurance policies in all project sites 4. Provide medical insurance for immediate family members 5. Streamline/ clarify sick and maternity leave policies 6. Set standards on bereavement support 7. Review and disseminate end of project or end of contract policy 8. Conduct assessments of staff capacity to identify gaps 9. Approve funding for staff development; utilize consultants for professional development activities. MTI U drafted new policies that clarified benefits but due to a delay in a field visit by MTI HQ’s HR manager, policies will be finalized during that visit in May 2012. An assessment of staff capacity by the MTI U HR manager is currently underway. Progress has been made in addressing seven of the nine staff salary and benefits policy action items: • A new salary and benefits policy was reviewed and adopted by MTI HR and MTI HQ team in May 2012. • The annual salary increase policy has been revised, but not yet implemented due to budget constraints. • A uniform health insurance policy is now in place for all MTI staff providing insurance for family members. • In October 2012 OCA, MTI Uganda identified additional policies to be reviewed and updated, including procurement, security, and HR. These policies were updated and implemented in August 2013. This included a revision of sick and maternity leave policy and bereavement. • A full assessment of staff capacity and plan for funding for staff development has not yet been accomplished. Final Evaluation Report: Lira District Child Survival Project in Uganda December 2013 Final Evaluation - Annexes Strategic Planning 1. Request guidelines from HQ for developing SP 2. Develop a plan to develop a SP; identify external support needed from HQ or local consultant; gather information on funding sources and regional needs; meet with potential FBO partners; and conduct a SWOT analysis 3. Complete an MTI-Uganda SP (an off-site workshop) MTI U engaged in a strategic planning process facilitated by the Africa Region Deputy Director and led by the Country Director. The MTI U Strategic Plan, based on MTI’s agency plan, was finalized in March 2011. The plan will be reviewed annually and used to guide program and organizational growth. • Action items for developing a strategic plan were all completed: • MTI U engaged in a strategic planning process facilitated by the Africa Region Deputy Director and led by the Country Director. The MTI U Strategic Plan, based on MTI’s agency plan, was finalized in March 2011. The plan is reviewed annually and used to guide program and organizational growth. Final Evaluation Report: Lira District Child Survival Project in Uganda December 2013 Final Evaluation - Annexes Communication & Decision￾making 1. Finalize organizational structure with clearly defined channels of supervision and communication. 2. Organize regular staff meetings or other fora for staff to contribute ideas and improvement recommendations 3. Ensure communication is copied to all pertinent staff In July 2011 MTI U decentralized their structure, creating regional teams of projects in Northern Uganda and SW Uganda. This streamlined communication and reporting within MTI U. Project managers meet with their teams weekly, followed with monthly regional meetings and quarterly management team meetings which facilitate project reporting, sharing of upcoming key activities, and any constraints or obstacles are addressed. With the placement of the Africa Regional Health Advisor in Uganda, a regular system of communication and technical advising with project teams was developed. Progress has been made in all three action items regarding communication and decision making: • In July 2011 MTI U decentralized the organizational structure, creating regional teams of projects in Northern Uganda and SW Uganda. This streamlined supervision, communication, and reporting within MTI U. • Project managers meet with their teams weekly, followed with monthly regional meetings and quarterly management team meetings which facilitate project reporting, sharing of upcoming key activities, and any constraints or obstacles are addressed. • With the placement of the Africa Regional Health Advisor in Uganda, a regular system of communication and technical advising with project teams was developed. • In 2013, MTI Uganda is working to create a knowledge management system to more readily share information between staff. In the spring of 2013, MTI Uganda staff members were linked with the MTI HQ data system. Additionally, an information resource sharing system is now being developed. Final Evaluation Report: Lira District Child Survival Project in Uganda December 2013 Final Evaluation - Annexes Monitoring and Evaluation 1. Conduct a Readiness Assessment for M&E (capacity, resources, tools etc) 2. Develop a comprehensive M&E plan 3. Adapt relevant M&E tools from MTI HQ and other partners 4. Provide M&E support to all MTI Uganda projects 5. All M&E staff and PMs go through PCM All MTI U projects carry out M&E, with new tools being adapted. A comprehensive M&E plan is to be developed in FY 13. All key management staff have completed MTI Project Cycle Management (PCM) training with the exception of new staff in SW Uganda. It is planned to roll out PCM training in the next year to non-management staff who demonstrate a readiness to learn and apply it in their work. Two of the five action priorities for M&E were accomplished: • A readiness assessment and a comprehensive M&E plan were not performed due to timing and budgetary constraints. • However, all MTI U projects carry out M&E, with new tools being adapted and with support from MTI HQ. • All key management staff have completed MTI Project Cycle Management (PCM) training. Final Evaluation Report: Lira District Child Survival Project in Uganda December 2013 Final Evaluation - Annexes ANNEX XIX B. PROJECT M & E MATRIX The Project Monitoring and Evaluation Matrix Objectives Indicators Baseline Values EOP Target Progress as of MTE Progress as of FE Comments Objective/ Result 1: Communities assume responsibility for their own health through the strengthening community capacity (Village Health Teams, Parish Development Councils, and Health Sub￾districts). % of VHTs who are women 5% 25% 24% 24% There has been no turn-over of VHTs so no new women VHTs have been trained. All 1,428 MLGs are women and support the VHTs by providing information to the community. These women are not VHTs and therefore not factored into the calculation of this indicator. However, strengthening the MLG network is a notable step towards including women in the management of community health. % of VHTs who received a supervisory visit during the last 6 months 0% 50% 49.3% 75% The CSP has intensified VHT supervision using the newly developed activity tracking tool. Mothers groups also received quarterly supportive supervision. % of communities with an emergency/referral transportation system 0% 70% Data not available 29 Emergency transport committee formed and functional Emergency transport committees have been established in each of the 29 parishes through the PDCs. Final Evaluation Report: Lira District Child Survival Project in Uganda December 2013 Final Evaluation - Annexes Objectives Indicators Baseline Values EOP Target Progress as of MTE Progress as of FE Comments % of PDCs and HUMCs using information from CHIS for decision making in the last year, with at least 1 concrete example of action taken 0% 70% 0% 90% Project staff have mentored PDCs and HUMCs in using community level data from VHTs to make decisions. PDCs and HUMCs used information from VHT, MLG and PEs for planning both at the sub county and district, for example, to expand EPI outreaches, establish outreaches for Nodding Syndrome in Aromo, and the reopening of health units in Ogur and Aromo. Objective/ Result 2 Improved health (C-IMCI and child care (ECD) behaviors among mothers of children <5 years % of children 0-5 months who were exclusively breastfed during the last 24 hours 73.6% 95% 66.2% 88.2% % of children aged 0-23 months who were put to the breast within one hour of delivery 29.0% 60% 22.7% 78.9% % of children aged 0-23 months who did not receive prelacteal feeds during the first 3 days after delivery 46.6% 75% 55.0% 87.2% IYCF: % of children 6-23 months fed according to a minimum of appropriate feeding practices 23.1% 50% 41.9% 48.9% % of children 0-23 months with diarrhea in the last two weeks who received Oral Rehydration solution (ORS) and/or recommended home fluids. 47.2% 70% 52.5% 61.2% A challenge to meeting this indicator is frequent stock out of ORS. Findings from focus group interviews indicated that knowledge about home fluids, with the exception of home fluids was limited. Final Evaluation Report: Lira District Child Survival Project in Uganda December 2013 Final Evaluation - Annexes Objectives Indicators Baseline Values EOP Target Progress as of MTE Progress as of FE Comments % of children 0-23 months with diarrhea in the last two weeks who were treated with Zinc. 0.9% 30% 2.6% 17.6% The MTE recommended that because of unreliable supplies of zinc, this indicator has been dropped. Since MTE health facility staff have been encouraging parents to purchase zinc for their children with diarrhea from private pharmacies. % of mothers of children aged 0-23 months who live in households with soap at the place for hand washing and who washed their hands with soap at least 2 of the appropriate times during a 24 hour recall period 54.0% 80% 75.3% 87.8% % of children aged 0-23 months with chest-related cough and fast/difficult breathing in the last two weeks who were taken to an appropriate health provider. 57.8% 80% 84.8% 74.2% % of children aged 0-23 months with chest-related cough and fast/difficult breathing in the last two weeks who were treated with an antibiotic 34.7% 70% 64.4% 68.5% ANTHROPOMETRICS % of children 0-23 months who are not underweight (-2 SD for the median weight for age, according to WHO/NCHS reference population) 72.3% 88% 82.36% 79.8% MNC % of mothers with children aged 0-23 75.7% 90% 73.7% 89.4% Final Evaluation Report: Lira District Child Survival Project in Uganda December 2013 Final Evaluation - Annexes Objectives Indicators Baseline Values EOP Target Progress as of MTE Progress as of FE Comments months who received at least two Tetanus Toxoid vaccinations before the birth of their youngest child. % of mothers with children aged 0-23 months who received at least 2 doses of IPT during the pregnancy with this youngest child. 35.0% 60% 59.1% 68.3% % of children age 0-23 months whose births were attended by skilled personnel 35.3% 50% 50.5% 83.9% % of mothers of children 0-23 months who received a post-partum visit by an appropriate trained health worker within two days after the birth of the youngest child. 16.33% 50% 22.2% 40.8% % of mothers of children 0-23 m are able to report at least two known maternal danger signs during the postpartum period 2.0% 80% 20.0% 51.1% IMMUNIZATION % of children aged 12-23 months who received measles vaccine according to the vaccination card or mother’s recall 77.0% 90% 80.0% 82.1% % of children aged 12-23 months who are fully vaccinated (received BCG, DPT3, OPV3, and measles vaccines) by 12 months of age, card verified 15.5% 50% 37.9% 56.0% The VHT and MLG are tracking children with incomplete vaccinations and referring them. The indicator is expected to improve by the EOP. ECD 38.0% 80% 68.5% 76.3% An ECD forum was held in Aromo in Final Evaluation Report: Lira District Child Survival Project in Uganda December 2013 Final Evaluation - Annexes Objectives Indicators Baseline Values EOP Target Progress as of MTE Progress as of FE Comments % of mothers of children aged 0-23 months who provide cognitive stimulation to their child in the form of games such as “where are your eyes”, etc. March 2013. This was meant to further engage the community including sub county Health Facility in charges and the district leaders (health sub district in charge, DHO, RDC). This forum further created awareness about ECD. % of mothers of children aged 0-23 months who told their child a story, sang a song, or spent time naming objects for child at least 2 times in the past week 22.7% 75% 40.1% 64.4% % of mothers of children aged 0-23 months who report that they talk or sing to the child while feeding the child 57.7% 80% 65.4% 83.6% Objective/ Result 3 Improved quality of care in health facilities through strengthened capacity in IMCI and MNC % of HC have a passing score with regard to the assessment of sick children (> 80% of patients observed in each facility have all 5 assessment tasks performed on them by the HW) 0/4 3/4 0/4 7/8 All health workers except for one completed all assessment tasks at least 80% of the time. % of HC in which > 80% sick children treated according to protocol 1/4 3/4 4/4 8/8 % of HC staff received a supervisory visit within 3 months 1/4 3/4 3/4 7/8 Objective/ Result 4: Strengthened institutional capacity of MTI Demonstrate improvement in 6 low￾scoring priority areas identified during the Organizational Capacity Assessment 0 6 2 6 MTI Uganda is now using the Holistic Organizational Capacity Assessment Instrument developed by Catholic Relief Services so has not updated the JSI Score Card used at baseline. The updated action plan Final Evaluation Report: Lira District Child Survival Project in Uganda December 2013 Final Evaluation - Annexes Objectives Indicators Baseline Values EOP Target Progress as of MTE Progress as of FE Comments and DHO to implement effective and efficient child survival activities indicates progress made in all six priority areas identified as low scoring at baseline. Action plans for 6 priority areas implemented and scores improved 0 6 2 6 Progress has been made in implementing action plans for all six priority areas. The action plans for addressing strategic planning, communication and decision making have been completed. Progress has been made in addressing succession planning, staff salary and benefits policy, and monitoring and evaluation. Lessons learned and best practices are disseminated utilizing at least three different media (program manual, presentations, web site, program guidance and meetings with stakeholders) No Yes Yes MTI has shared successes and lessons learned with the DHO and NGOs working in Lira during quarterly district level health coordination meetings and during meeting with PDCs and USAID in Kampala and Gulu. MTI shared lessons learned and best practices with USAID-funded SCHIPHA project in northern Uganda through a joint training on performance improvement as well as a meeting with the MCHIP Ethiopia focal person. The Africa Health Advisor made a presentation on midterm survey results at headquarters in Portland, Final Evaluation Report: Lira District Child Survival Project in Uganda December 2013 Final Evaluation - Annexes Objectives Indicators Baseline Values EOP Target Progress as of MTE Progress as of FE Comments Oregon as well in March, 2012. % of health facilities received joint DHO/MTI supervision visits once per quarter 0 75% 100% 87% All eight health facilities received quarterly supportive supervision services. Final Evaluation Report: Lira District Child Survival Project in Uganda December 2013 Final Evaluation - Annexes ANNEX XIX C. FINAL EVALUATION RESULTS MEETING - LIRA Final Child Survival Programme Evaluation Result Meeting -18/ 09/2013 Time Activity Responsible Person 9.00 AM-9.05 Registration, Prayer, Introduction Agenda CSP staff 9.05-9.15 am Welcome Remarks Regional Program Manager 9.15.am -9.30 am Opening Remarks District Chief Administrative Officer 9.30. -9.45 am Programme Highlights CSP Program Manager 9. 45-10.00 am Break tea 10. 00 -11.00 am Project Achievement District Planner/MTI M & E 11. 00 am – 11.30 am Live Story VHT & MLG 11.30-11:45 Preliminary Results Evaluation Consultant 11:45-12:15 pm Open Discussion Led by Ogur Sub-County Chief -All Participants 12.15-12.30 Pm Remarks from MTI Country Director Country Director 12.30-1.00 pm Handover of Certificates Closing Remarks District Health Officer 1.00 pm-2:00pm Lunch and Departure All Participants Village Health Teams Mother Leader Groups Peer Educators PCD Members HUMC Members Health Facility Staff Sub County Chiefs District Planner District Health Officer District Chief Administrative Officer CSP Project Manager and Staff FE Team Final Evaluation Report: Lira District Child Survival Project in Uganda December 2013 Final Evaluation - Annexes ANNEX XIX D. RESPONSE TO MIDTERM RECOMMENDATIONS Midterm Evaluation Recommendations Progress as of September 2013 Recommendation 1: Triple the number of Mother Leader Groups to reach every household with a pregnant woman or children under two with key messages on maternal newborn care, complementary feeding, and optimal breastfeeding. While supporting behavior change in these interventions, the MLGs can also promote sanitation and monitor child health cards to minimize immunization drop-outs. The number of Mother Leader Groups has been tripled to 1428. The number of MLGs recruited was based on population numbers, ensuring adequate coverage. The MLGs have been provided with t-shirts, name tags and MoH counseling materials and support the VHTs to promote health messages through household visits and community dramas on IYCF/MNC, and sanitation. They monitor Child health cards and refer patients to the health facilities. Recommendation 2: Train VHTs and MLGs in nutrition using the WHO/UNICEF curriculum for East Africa which has been left with the project manager. This can be done for MLGs in two hour segments each month alternating with MNC training for a total of four hours of training per month. This may be supplemented with the Freedom from Hunger breastfeeding and complementary feeding modules that were previously sent to the project. The new CORE Group material on newborn care at home can serve as a basis for training in MNC. VHTs are trained during quarterly review meetings at the parish levels for about four hours, using the recommended WHO/UNICEF curriculum for East Africa. The MLGs were also trained using the same curriculum. MoH counseling materials for both facility staff and community volunteers were reproduced to support them during the health education VHTs and MLGs are following a community education plan to ensure that nutrition and MNC messages are covered comprehensively and thoroughly. Recommendation 3: Withdraw support to routine and outreach immunizations in order to free up staff time to focus on the other technical interventions, particularly maternal newborn care and nutrition, including breastfeeding, complementary feeding, and maternal nutrition. To improve complete coverage of immunizations, focus on identification of DPT and OPV drop-outs at the household level. Meet with District Health Office as soon as possible to share MTE results, review the MOU and explain the project’s decision to reduce support for vaccine and staff transport for outreach and assistance with immunizations to a level consistent with the MOU (only for Child Health Days). After the MTE, CSP staff time was refocused on supervision, monitoring and mentoring of both health workers and community volunteers to improve MNC/ IYCF and maternal nutrition services at the facility and community. Project support for EPI was limited to Child Health Days during which the MLG and VHTs go home to home to identify those children and pregnant mothers defaulters and give vitamin A and de￾worming tablets Final Evaluation Report: Lira District Child Survival Project in Uganda December 2013 Final Evaluation - Annexes Midterm Evaluation Recommendations Progress as of September 2013 Recommendation 4: Confine continuation of ECD to the available budget and to having a greater impact on a small area. The VHTs and PEs need more comprehensive training including the TOT aspects in order to better transmit learning to community members. The community groups must be kept to no more than 10-12 to ensure optimal learning. The sessions may be repeated twice for reinforcement. ECD was confined in Aromo Sub county with a total 252 Peer Educators promoting ECD practices in the community. Each PE visits 20 to 26 households per month. The MTI ECD Focal Person provides the topic to be discussed uniformly for each month Recommendation 5: Realign remaining budget (within USAID rules) to cover as many of the following priorities as possible: an additional staff member to have one person available to focus on ECD (match funding), reproduction of the quality IEC materials available from the MOH nutrition cluster, incentives for VHTs and MLGs, and training for VHTs and MLs. One staff member, Harriet Apili, focuses mainly on ECD activities. She was not newly hired, but her job description was changed so that she can dedicate more effort to ECD. High quality bound IEC materials on IYCF and MNC were acquired from the MOH and produced for the Lead Mothers. Copies for the VHTs and were printed, laminated, and distributed to the VHTs and MLGs. A third realignment was was approved by USAID to cover printing of additional counseling material, incentives for Mothers Leaders and refresher training for the VHTs on ICCM. Recommendation 6: Continue with plans for the equity study and for complete mapping of VHT coverage. The results may implicate coordination with the health centers to increase the number of VHTs and for CSP to train any new ones. The MTE KPC revealed that Aromo sub-county performed much lower in reaching project indicators as compared to Lira and Ogur sub-counties. This raised concerned about equity and household coverage. New VHTs were not trained because two VHTs were already assigned per village by the MOH and had already been trained in a one-time CSP activity. However, CSP did map the population of the project area, which informed the process of selection and placement of MLGs to improve household coverage. To provide more intensive support to families, Aromo sub-county has also been targeted for ECD activities. Equity is also being reached in other ways. Due to advocacy from the HUMC of Aromo, a midwife was transferred to the health facility to address the needs of this area. The District provided maintenance to the refrigerator at Aromo HCIII which Final Evaluation Report: Lira District Child Survival Project in Uganda December 2013 Final Evaluation - Annexes Midterm Evaluation Recommendations Progress as of September 2013 has enabled the newly opened health centers to carry out EPI outreaches. There are now 11 EPI outreaches in Aromo as compared with 8 before the opening of the new facilities. In addition, the four newly-opened health facilities are improving access to health services for the respective communities by reducing distances to the service points. Recommendation 7: Conduct quarterly VHT meetings and MLG bi-weekly meetings at the parish level to reduce group size for effective learning. VHTs and MLGs meet quarterly as planned with MTI Supervisors. The TOT/Mother Leaders meet bi-weekly with the trained mothers. All meetings are now conducted at the parish levels as recommended Recommendation 8: For M&E, the indicator for zinc treatment needs to be dropped. There are no zinc supplies and UNICEF is still working with the MOH at central level on a zinc protocol. The indicator for emergency transportation plans would be more useful for measuring project effort if the last phrase “with at least one use within the past three months” is dropped. MTI could report in the final survey the total number of transport plans, then, separately as an addendum, report whether any had been used in the last three months. We would prefer that communities not suffer emergencies requiring frequent use of the emergency transport system. Zinc is not supplied in the health facilities, and the related project indicator has been dropped. There are committees set at the parishes whose roles include having an emergency plan or the community Recommendation 9: MTI management should provide the project manager adequate financial information on a quarterly basis to enable her to adjust project activities appropriately. Management provides financial information to the project Manager on a monthly basis. Final Evaluation Report: Lira District Child Survival Project in Uganda December 2013 Final Evaluation - Annexes ANNEX XIX. PROJECT CHECKLISTS AND REFERRAL FORMS Final Evaluation Report: Lira District Child Survival Project in Uganda December 2013 Final Evaluation - Annexes Selection Criteria for Community Volunteers MTI would like to implement Early Childhood Development activities in Aromo Sub County through a network of community volunteers (Peer Educators). Step I: MTI staff explains the role of Peer educators to the community leaders 1. The MTI Village Health Team Mobilisers visit the Parish Development Committee and the Chairperson Local Council One to explain the ECD activity and request him to organize community for a meeting. 2. At the end of this meeting, a date should be set for the meeting Step II: MTI visits the community to explain the ECD in details and to ask them to nominate appropriate candidates. 1. MTI VHT Mobilisers accompanied by health facility staff and VHT Parish Mobilisers for that particular parish) visit the community on the agreed date. 2. The health facility staff and MTI ECD trainer explain to the community what ECD entails e.g. ECD involves a new way of caring for children less than 5 years by improving on the quality of their lives through providing support to their physical, mental, emotional/psychological development and proper nutrition, sanitation and a clean environment. Volunteers would need to be able to volunteer 6-8 hours a week .Volunteer must not be a VHT member. Priority is given to women with children less than 2 years of age. The incentive which will be provided by MTI in conjunction with the DHO will be training, technical support, and materials for training care givers and tracking activities in the community. Volunteers are not paid for their voluntary work and this role will not lead to a paid job A community volunteer is a person trusted and selected by the community and must currently be above 18 years, able to read and write at least in Luo, live in this community and is not local council chairperson I or Local Chairperson II (Check and balances) General qualities of good volunteers are: Kind and compassionate, trustworthy and respectful, good communication skills, Non-judgmental, live by example, have good support from their spouses, and wants to serve their community. Final Evaluation Report: Lira District Child Survival Project in Uganda December 2013 Final Evaluation - Annexes Step 3: the community to nominate 6 female and 6 male volunteers out of which 4 will be selected.( Nomination of three volunteers to every one needed is to avoid biasness and make the process more representative). Step 4: Administer ECD volunteer questionnaire to all those nominated Name----------------------------------------- Date parish---------------------------------- Name of interviewer---------------------- No Question Answer Answer given (Tick) Rating rationale 1 Where do you stay Does not live within the parish Discontinue the interview PE must come from the parish Lives within the parish 20 Is essential 2 How many hours can you decide to do ECD Work in a week ? 6-8 hours 20 Should be able to volunteer at least 6 hours a week 0-3 hours 0 Will not have time to volunteer 4-5 hours 10 More than 8 hours 20 Will be able to volunteer at least 6 hours a week 3 How many days do you do your own work a week? (either paid or unpaid) I do not work 20 Those who do not work may have more time to volunteer but may also have unpaid commitments 7 days a week Discontinue interview Will not have time to volunteer 6 full days a week Discontinue the interview Will not have time to volunteer 5 full days a week 5 May not have enough time to volunteer Less than 5 full days a week 10 More likely to have time to volunteer 4 Why would you like to serve as a community volunteer(can give more than one response) I am a mother/Father myself so I want to help my fellow mothers/Fathers 20 Able to identify with other parents/ care takers I want to gain experience in working in health activities and ECD work 10 Motivation may be good Final Evaluation Report: Lira District Child Survival Project in Uganda December 2013 Final Evaluation - Annexes I enjoy voluntarism since God rewards me 20 Motivation Is good I am jobless and I want a job Discontinue the interview Inappropriate motivation Only because I have been selected by the community Discontinue the interview Inappropriate motivation I want to serve my people by involving in ECD WORK 10 Motivation may be good For political interest Discontinue the interview Inappropriate motivation To be recognized by the community Discontinue the interview Inappropriate motivation To be given a bicycle Discontinue the interview Inappropriate motivation Signature of nominee: -------------------------------------------------- Signature of interviewer: ------------------------------------------------ Date: -------------------------------------------------------------------------- Step 5: Analysis is made and the best candidates are chosen and informed to come for the training. Final Evaluation Report: Lira District Child Survival Project in Uganda December 2013 Final Evaluation - Annexes Community Volunteer Activity Tracking Tool Name:_____________________ Check one: PE______ VHT_____ HF Staff____ MLG____ CSP staff___ Sub-County:_________________ Parish:_____________________ Village:____________________ ECD Lesson Numbers: Enter the lesson number(s) provided to the participant in form below. Lesson name Date(s) of Lesson Lesson name Date(s) of Lesson 1= Introduction to Early Childhood Development 5 = Social, Emotional Development 2= Physical Development 6 = Baby Cues 3= Cognitive Development 7 = Baby Massage 4= Language Development 8 = Health, Hygiene, Nutrition & Sanitation # Name of Caregiver Male (M) Female (F) # of children under 3 years old Relationship of the caregiver to the child Lesson #s given 1 2 3 4 5 6 7 8 1) 2) 3) 4) 5) 6) Final Evaluation Report: Lira District Child Survival Project in Uganda December 2013 Final Evaluation - Annexes Community Learning Session: Community Volunteer Quality Improvement Checklist Specify if Static Health Talk or Community Outreach: ____________________ Date:_______________ Audience: ________________________________ Venue: ____________ Community: ____________ Technical Content a) Presents technical information accurately Yes No b) Answered participants’ questions accurately or offered to find out answer if question was beyond technical knowledge Yes No c) Corrected misinformation Yes No Comments: Following the Education Plan a) Followed VHT guidelines Yes No b) Had all necessary materials ready and organized Yes No Comments: Facilitation Skills a) Spoke clearly and loudly Yes No b) Praised and encouraged participants Yes No c) Demonstrated respect to participants Yes No d) Created a dialogue and limited lecture style Yes No e) Used open questions to encourage discussion Yes No f) Visual Aids (if applicable) Yes No Final Evaluation Report: Lira District Child Survival Project in Uganda December 2013 Final Evaluation - Annexes  Showed visual aids Yes No  Displayed visual aids so that all participants could see Yes No g) Used other teaching methods and facilitation skills (demonstrations, role plays, stories, games) If other methods used, indicate what was done here: Yes No Comments and Follow up Needed : Record Keeping and Reporting Yes or No a) The register is updated if a family moves away, if a baby is born or a family member dies Yes No b) Referrals are correctly documented Yes No c) The register covers all the families in the VHT's sector Yes No d) Monthly reports are complete Yes No e) Monthly reports are submitted on time Yes No Actions to be Taken and Comments: Final Evaluation Report: Lira District Child Survival Project in Uganda December 2013 Final Evaluation - Annexes CSP PROJECT VHT REFERRAL FORM Nying ..…………………………………..….Mwaka……………………………………. Awobi/Nyako, Nying Totere………………………………...………… Nino dwe me tin……… ………Nino dwe atwo ocakere iye………………………..… Cawa otunu kede bot VHT………………………………………………………….…… Tye akoko/Tye kede………………………………………………………………….…… Danger signs……………………………………………………………………………… Acwalo idakatal anyinge ……………………………………………………………….... Nying VHT ocwalo…………………………………………………………………….… Nino Dwe kede Cawa Amwe Atwo Otunu Kede Idakatal ame ocwale iye………………………………………… --------------------------------------------------------------------------------------------------------- Final Evaluation Report: Lira District Child Survival Project in Uganda December 2013 Final Evaluation - Annexes