September 2011 Child Survival and Health Grants Program Cooperative Agreement No. GSH-A-00-09-00012-00. September 30, 2009 – September 30, 2013. Mary Helen Carruth, Child Survival Advisor, Medical Teams International. Evaluation Consultant, Judiann McNulty, DrPH. EVALUATION Mid-Term Evaluation Report: Lira District Child Survival Project in Uganda Julie Larsen Maher, Wildlife Conservation Society Mid-Term Evaluation Report Lira District Child Survival Project in Uganda Medical Teams International In Partnership with the Lira District Health Office Child Survival and Health Grants Program Cooperative Agreement No. GSH-A-00-09-00012-00 September 30, 2009 – September 30, 2013 Contact Person at MTI HQ Mary Helen Carruth, Child Survival Advisor, Medical Teams International, mhcarruth@medicalteams.org, 14150 SW Milton Ct. Tigard, OR 97224, Mailing address: P.O. Box 10, Portland OR 97207 Evaluation Consultant Judiann McNulty, DrPH List of Acronyms ANC Antenatal Care ARI Acute Respiratory Infection BC Behavior Change CDD Control of Diarrheal Disease DIP Detailed Implementation Plan DPT Diphtheria, Pertussis, Tetanus ECD Early Childhood Development ENC Essential Newborn Care EPI Expanded Program of Immunization FE Final Evaluation GFATM Global Fund for AIDS, Tuberculosis and Malaria HC Health Center HHI Hands to Hearts International HIS Health Information System HQ Headquarters HUMC Health Unit Management Committees IEC Information, Education, Communication iCCM Integrated Community Case Management IMCI Integrated Management of Childhood Illness IPT Intermittent Preventive Treatment IYCF Infant and Young Child Feeding KPC Knowledge, Practice, Coverage LQAS Lot Quality Assurance Sampling M&E Monitoring and Evaluation MCH Maternal Child Health MCHIP Maternal and Child Health Integrated Program MOH Ministry of Health MTE Midterm Evaluation MTI Medical Teams International NUMAT Northern Uganda Malaria AIDS & Tuberculosis Program OCA Organizational Capacity Assessment OPV Oral Polio Vaccine ORS Oral Rehydration Salts PE Peer Educators PLW Pregnant and Lactating Women PMTCT Prevention of Mother-to-Child Transmission R-HFA Rapid Health Facility Assessment SBC Social and Behavioral Change TT Tetanus Toxoid UNICEF United Nations International Children’s Emergency Fund USAID United States Agency for International Development WHO World Health Organization VHT Village Health Team WRA Women of Reproductive Age Table of Contents A. Preliminary Information Executive Summary Summary Table of Major Project Accomplishments 1 4 B. Overview of the Project Structure and Implementation 6 C. Evaluation Methodology and Limitations 8 D. Data Quality and Use 8 E. Presentation of Progress towards Achieving Results 10 F. Discussion of the Progress towards Achieving Results 1. Contribution toward Objectives/Results 2. Contextual Factors 3. Role of Key Partners 4. Overall Design Factors that are Influencing Progress towards Results 14 18 19 20 G. Discussion of Potential for Sustained Outcomes, Equity, and Community Health Worker Models 1. Progress towards Sustained Outcomes 2. Attention to Equity 3. Role of the Community health Worker 4. Contribution to Global Learning 22 22 23 24 H. Conclusions and Recommendations 24 I. Action Plan for Responding to Evaluator Recommendations 26 List of Annexes Annex 1: Results Highlight Annex 2: Technical Support Provided during Years One and Two Annex 3: Workplan Table Annex 4: Evaluation Assessment Methodology A. Evaluation team scope of work, including timeline of activities B. Brief descriptions of information gathering methodologies and documents reviewed C. Data collection instruments, including interview guides D. Calendar of evaluation activities E. Persons Interviewed and Contacted during the MTE F. MTE Mid-Term KPC Report G. R-HFA Report Annex 5: Evaluation Team Members and their Titles Annex 6: Evaluation of Early Childhood Development Component Annex 7: Organization Capacity Action Plan update Annex 8: Training Matrix Annex 9: Project Management Evaluation Annex 10: Sustainability Plan Up-Date Annex 11: Rapid CATCH Table Annex 12: Project Data Form 1 A. Preliminary Information Executive Summary Medical Teams International is in the third year of implementing a four-year entry grant Child Survival Project (CSP) in Lira District in northern Uganda, funded by the USAID Child Survival and Health Grants Program (CSHGP). The project targets the 3 sub-counties that comprise the Erute North Health Sub-District and is implemented in partnership with the Lira District Health Office (DHO) and the four health centers in the target area. The project overall goal is to reduce child morbidity and mortality in Erute North Sub-district of Lira District in northern Uganda. The project has four key results/objectives: 1. Communities assume responsibility for their own health through strengthened community capacity (Village Health Teams, Parish Development Councils, and Health Sub-districts). 2. Improved health (Community-based Integrated Management of Childhood Illness) and child care (Early Childhood Development) behaviors among mothers of children under five years of age. 3. Improved quality of health facility services through strengthened Integrated Management of Childhood Illness and Maternal and Newborn Care capacity. 4. Strengthened institutional capacity of MTI and the Lira District Health Office to implement effective and efficient child survival activities. CSP has five child survival interventions (pneumonia, diarrhea, nutrition, immunizations and maternal newborn care) along with a sixth intervention Early Childhood Development (ECD) which has been implemented with match funding. CSP postulated that the addition of ECD would enhance child survival impact as documented in the Lancet “Child Development in Developing Countries” meta-analysis (2007). CSP has trained 34 personnel from the health centers in IMCI and supplied the health centers (HCs) with essential equipment. The Mid-Term Rapid Health Facility Assessment (R-HFA) shows that personnel are now providing correct diagnosis and treatment. CSP has given additional training to health facility staff in maternal and newborn health and provides on-going mentoring and supportive supervision to the health workers, collaborating with the DHO to provide regular joint supervision. MTI has provided significant support to the DHO and health facilities for immunizations by transporting vaccine for routine and outreach clinics and personnel to outreach points with MTI staff actively participating in the clinics. On the community side, CSP trained 560 Village Health Team (VHT) volunteers in integrated Community Case Management (iCCM). CSP holds quarterly meetings with the VHTs for supportive supervision and to impart additional short training topics. The community strategy was for VHTs to educate their communities about the six intervention areas, using pre-existing groups at churches, mosques, etc. VHTs have been successful, according to Mid –Term KPC, in promoting appropriate care-seeking for child illness and making referrals. They have also started 2 making post-partum home visits, which has significantly increased post-partum care as seen in the Mid-Term KPC results. Using a cascaded training model, CSP rolled out ECD training to the village level through health personnel who trained VHTs and Peer Educators (PE) to lead ECD sessions during an intensive time period in the selected intervention communities. The training was very well received and the Mid-Term KPC shows significant gains in two of the three ECD indicators. (See Table 2 on page 15. The major constraints facing CSP are in the health centers with their high staff turn-over and frequent stock-outs of essential medicines. These issues go beyond the level of the DHO to the central level. Another constraint is that there are only two instead of the mandated five VHTs per village and there are many demands on their time by government programs and other NGOs. The CSP budget did not have resources to train and support additional VHTs. This constraint has affected promotion of behavior change, as has the reliance on iCCM training to provide sufficient knowledge across all five CS interventions. The MTE revealed knowledge deficits among VHTs in nutrition and maternal newborn care with requests from them and the target population for education on these topics. Summary of conclusions: 1) The CSP got off to a relatively fast start with excellent support from MTI HQ and consultants for the surveys, and DIP. They also accomplished the major trainings in IMCI and iCCM during the first year. MTI HQ and regional technical support for the project has been exemplary as shown in the table in Annex 2. 2) The R-HFA shows there has been some improvement in certain areas of service delivery, but most are stagnant or have gotten worse. Pharmaceuticals are in short supply or not available due to supply chain issues far beyond the scope of the CSP. On the other hand, health workers have improved assessment capacity and are prescribing appropriate treatments. 3) The MTE KPC shows significant improvements in some indicators while others have not improved. The indicators which improved most are largely related to the iCCM or post￾natal visit training for VHTs. The government promotion of institutional deliveries has been a factor in improving thet indicator for births attended by skilled health personnel. 4) The VHTs are quite committed and the iCCM training prepared them well to identify illnesses and newborn complications and to make referrals. They need more training to enable them to promote maternal health practices including recognition of danger signs by family members, and to promote optimal breastfeeding and complementary feeding. 5) Expansion of the Mother Leader Groups will provide needed support to VHTs in reaching every household to promote behavior change, monitor appropriate care-seeking, and mobilize the community around issues such as lack of sanitation. 6) During the first two years, the implementing staff suffered from inadequate guidance on organizing their time, balancing level of effort with requests from health center staff, and 3 planning in accordance with the DIP. The current project manager shows great potential for providing needed leadership, given adequate support from the country office with planning, tracking progress of implementation of the project work plan and conducting staff performance reviews according to MTI protocol. 7) The actual vs. planned level of effort for technical interventions has been unbalanced from the beginning with a disproportionate amount of effort on immunizations. This has affected progress towards achieving other indicators. Key recommendations 1) Triple the number of Mother Leader Groups to reach every household with a pregnant woman or children under two with key messages on maternal newborn care, complementary feeding, and optimal breastfeeding. While supporting behavior change in these interventions, the MLGs can also promote sanitation and monitor child health cards to minimize immunization drop-outs. 2) Withdraw support to routine and outreach immunizations in order to free up staff time to focus on the other technical interventions, particularly maternal newborn care and nutrition. Meet with District Health Office as soon as possible to share MTE results, review the MOU and explain the project’s decision to reduce support for vaccine and staff transport for outreach and assistance with immunizations to a level consistent with the MOU (only for Child Health Days). 3) Confine continuation of ECD to the available budget and to having a greater impact on a small area. 4) Realign remaining budget (within USAID rules) to cover as many of the following priorities as possible: an additional staff member to have one person available to focus on ECD (match funding), reproduction of the quality IEC materials available from the MOH nutrition cluster, incentives for VHTs and MLGs, and training for VHTs and MLs. 5) For M&E, the indicator for zinc treatment needs to be dropped. There are no zinc supplies and UNICEF is still working with the MOH at central level on a zinc protocol. The indicator for emergency transportation plans would be more useful for measuring project effort if the last phrase “with at least one use within the past three months” is dropped. 6) MTI management should provide the project manager adequate financial information on a quarterly basis to enable her to adjust project activities appropriately. 4 Summary Table of Major Project Accomplishments Project Objective 1: Communities assume responsibility for their own health through strengthened community capacity (Village Health Teams, Parish Development Councils, and Health Sub-districts). Project Inputs Activities Outputs Outcome  Supportive supervision and mentoring for Village Health Teams  Support and mentoring for Health Unit Management Committees  Referral forms for Village Health Teams Strengthen referral system from community to health facility by providing training to VHTs and developing pictorial referral cards Support health facility staff to arrange and facilitate coordination meetings with VHTs Facilitate collaboration between VHTs and PDCs and HUMCs to benefit MCH. HUMCs reactivated at all four health facilities in the project area. 24 meetings have been held since project inception. Quarterly VHT meetings in coordination with health facility staff take place quarterly in all three sub￾counties New EPI outreach in Otara Parish, Aromo created in October, 2011 due to PDC/HUMC collaboration and influence 24% of VHTs are women 49.3% of the 560 VHTs who were trained in iCCM received a supervisory visit in the last 6 months Project Objective 2: Improved health (Integrated Community Case Management) and child care (Early Childhood Development) behaviors among mothers of children <5  Social and Behavior Change Strategy  ICCM curriculum  ICCM handouts and VHT manuals for VHTs  Hands to Hearts Early Childhood Development Curriculum  Monitoring and evaluation tools  Non-monetary incentives to VHTs (t￾shirts for VHT parish leaders only, soap , certificates) Village Health Teams promote social and behavior change through existing channels (women’s groups, church groups, community meetings) and through home visits. Strengthen capacity of Village Health Teams in ICCM using a social and behavior change approach. Train VHTs in ICCM Train Peer Educators, VHTs and health facility staff in Early Childhood development 560 VHTs trained in ICCM 300 peer educators trained in Early Childhood Development 21 Trainer of Trainers(2 MTI staff and 19 health facility staff) trained in ECD 560 VHTs providing home visits, conducting community outreach, supporting static immunization days and making referrals Some household health behaviors improved in midterm evaluation LQAS KPC results, such as:  75.35% of mothers of children 0-23 months live in households with soap or ash at the place for hand washing and that washed their hands with soap or ash at least 2 of the appropriate times during a 24 hour recall period as compared with 54.0% at baseline  86.11% of children age 0-23 months with chest-related cough and fast/difficult breathing in the last two weeks who were taken to an appropriate health provider as compared with 57.8% at baseline 5 Project Objective 3: Improved quality of health facility services through strengthened Integrated Management of Childhood Illness and Maternal and Newborn Care capacity. Project Inputs Activities Outputs Outcome  Rapid Health Facility Assessment  Ugandan IMCI materials  Home-based maternal and newborn care materials Support IMCI training for health facility staff. Support MNC training for health facility staff On-the-job mentoring of health facility staff. Joint supportive supervision with DHO and partners MNC refresher training for 46 health workers at HF level IMCI training for 34 health workers at four health facilities Supportive supervision training for 40 health workers and community leaders 100% of health centers treated over 80% of sick children according to protocol. 75% of health center staff received a supervisory visit within three months 53.3% of children age 0 -23 months whose births were attended by skilled personnel as compared with 35.3% at baseline Project Objective 4: Strengthened institutional capacity of MTI and the Lira District Health Office to implement effective and efficient child survival activities. Project Inputs Activities Outputs Outcome  Organizational Capacity Assessment  Action plans for 6 MTI Uganda priority areas HQ support for MTI Uganda capacity through structured plan based on Organizational Capacity Assessment and technical assistance in social and behavior change and monitoring and evaluation. MTI/DHO joint activities for planning, training, implementation and evaluation. Joint supervision to VHTs together with the DHO and health facility staff Support health facility staff to arrange and facilitate coordination meetings with VHTs 100% of health facilities received joint DHO/MTI supervision visits once per quarter 2 out of 6 action plans completed: Strategic Planning and Communication and Decision Making. The rest are in progress and ongoing 6 B. Overview of the Project Structure and Implementation The project overall goal is to reduce child morbidity and mortality in Erute North Sub-district of Lira District in northern Uganda. The project has four key results/objectives: Objective No.1: Communities assume responsibility for their own health through strengthened community capacity (Village Health Teams, Parish Development Councils, and Health Sub￾districts). Objective No.2: Improved health (Community-based Integrated Management of Childhood Illness) and child care (Early Childhood Development) behaviors among mothers of children under five years of age. Objective No.3: Improved quality of health facility services through strengthened Integrated Management of Childhood Illness and Maternal and Newborn Care capacity. Objective No.4: Strengthened institutional capacity of MTI and the Lira District Health Office to implement effective and efficient child survival activities. The CSP benefits approximately 22,457 children under age five (U5) and 22,907 women of reproductive age (WRA) for a total of 45,364 direct beneficiaries. Target Population Population data Erute North Sub-District by target sub-county AROMO OGUR LIRA Sub-Total Total Population *** 34,800 53,200 25,400 113,400 Children 0-11 months (Calculated as 4.3% of total population) 1,496 2,288 1,092 4,876 Children 12-23 months (Calculated as 4.3% of total population) 1,496 2,288 1,092 4,876 Children 24-59 months (Calculated as 12.9% of total population) 4,038 6,170 2,497 12,705 Subtotal Under Age Five (Calculated as 20.2% of total population) 7,030 10,746 4,681 22,457 Women of Reproductive Age (WRA 15-49 yr) (Calculated as 20.2% of total population) 7,030 10,746 5,131 22,907 Total Target Population (Under Five + WRA) 14,060 21,492 9,812 45,364 *** National census in 2002, projections for 2010, 3.4% growth rate. Key activities for Objective No. 1 include training and follow-up to strengthen the capacity of Village Health Teams in integrated Community Case Management (iCCM); establishing a structured referral system between Village Health Teams (VHTs) and local health facilities; and facilitating collaboration between VHTs and Parish Development Committees and Health Unit Management Committees to benefit maternal and child health. For Objective No. 2, the plan was for Village Health Teams to promote social and behavior change through existing channels (women’s groups, church groups, community meetings) and through home visits. Under Objective 2, MTI also introduced early child development (ECD) 7 for children under two with an operations research plan to assess whether the ECD training would enhance child survival outcomes in intervention versus control areas. Since this is a CSHGP entry grant, the innovation was not required and MTI received no assistance from MCHIP with planning the operations research. For Objectives Nos. 3 and 4, MTI and the Lira District Health Office (DHO) conduct joint activities for planning, training, implementation and evaluation. The project supported IMCI training for health facility staff, followed by on-the-job mentoring of health facility staff and joint supportive supervision with the District Health Office partners. For Objective 4, as part of entry grant requirements, MTI Uganda receives support from MTI head office to sustainably build MTI Uganda capacity through a structured Action Plan based on results from the use of an Organizational Capacity Assessment tool adapted from the Organizational Capacity Tool developed by John Snow, Inc. The project planned to devote 10% level of effort to strengthen Immunization, 20% for Control of Diarrheal Disease, 25% for Pneumonia Case Management, 20% for Infant and Young Child Feeding, and 25% for Maternal and Newborn Care. No level of effort was allowed for the innovation for Early Childhood Development activities, which are funded out of MTI match. The key partner is the Lira District Health Office team and MoH staff at the four existing health facilities. At the time the DIP was written, it appeared that two additional health facilities would become operational, however that has not happened and is highly unlikely before the project ends. MTI has conducted many activities jointly with MoH staff, including training of Maternal Child Health staff, Village Health Teams, and Mother Leader Groups with some joint follow-up supportive supervision. MTI has participated heavily in routine health center outreach and all immunization activities. A supporting partner, Hands to Hearts International (HHI), an Oregon-based NGO specializing in early childhood development for infants and very young children, was contracted to provide the training of trainers for MTI staff and health center staff on ECD. HHI first adapted their model in another Uganda population, using those modifications and trainers from that population to conduct the training in Lira. MTI has kept the USAID mission apprised of the CSP since its inception. The DIP and first annual report were shared and a de-briefing held after the MTE to share results and discuss constraints. MTI looks forward to collaboration with the forth-coming mission bi-lateral project Northern Uganda– Health Integration to Enhance Services (NU-HITES) to improve health service delivery in the North. While there will not be direct overlap, there is potential for synergy and possibly sharing some training activities. In Annex 3, the work plan presented in the DIP has been modified with additional columns to explain status of activities. No major changes have been made which would require approval from USAID. 8 C. Evaluation Methodology and Limitations The mid-term evaluation consisted of a combination of quantitative and qualitative methods. In December 2011, the MTI Senior Advisor in M&E assisted with conducting a Rapid-Health Facility Assessment and with preparations for the Knowledge Practices and Coverage (KPC) survey. The KPC survey using Lot Quality Assurance Sampling (LQAS) was conducted in January 2012. Detailed reports of both surveys are attached in Annex 4 to this report and the results were used to plan the qualitative assessments and formulate the recommendations and conclusions. The participatory qualitative evaluation took part in two phases. During the first phase, the CSP team worked with the ECD consultant to conduct interviews and focus groups in the ECD intervention areas and to analyze the findings in a half-day workshop. Key results are included in this narrative and a more detailed report is found in Annex 6. The second phase of qualitative evaluation, led by the external evaluation consultant, focused on the standard child survival interventions across all project objectives. Three teams conducted focus groups with health workers, community volunteers and target population in all three sub￾counties and individual interviews with project and DHO staff. A calendar for the qualitative phase and lists of all those interviewed are included in the annexes 4d and e. Qualitative data collection followed interview guides that were designed to triangulate results of the R-HFA and KPC as well as to assess the effectiveness of project processes and ascertain the major challenges and constraints. Instruments are included in Annex 4c. One limitation to the qualitative evaluation was the selection of focus groups sites by the field staff. They were asked to select one strong and one weak site, but criteria were not well-defined and there was little apparent difference. Another minor limitation is that interviewing VHTs as a group from multiple villages may not have allowed for a very accurate assessment of their knowledge, but rather, only gave a general idea of composite knowledge of key messages. Collaboration of MTI staff from the field, region and HQ was exemplary. This facilitated additional data analysis and in-depth discussion for analysis of findings. D. Data Quality and Use Quantitative data sources for the project include the baseline KPC survey, annual KPC surveys using LQAS, and the baseline and mid-term Rapid Health Facility Assessments. CSP used qualitative methods for this mid-term evaluation along with the results of the mid-term KPC and R-HFA. Results, in terms of indicators, are measured by the quantitative methods. The quantitative data collection is of very high quality. There are no discrepancies between baseline, annual, and MTE surveys. Please see Annex 4f and Annex 4g for the complete reports including the methodology. The only suggestion would be to show the components of the composite indicator on IYCF separately in the report to enable staff to see where more effort may be needed. 9 Following the first annual report, CSP staff took an in-depth look at results by supervision area (SA) and determined that different SAs needed differing emphasis on particular behaviors. For example, due to low ORS use identified in the survey, Supervision Areas 2, 4 and 5 were targeted for more ORS promotion via the VHTs Project staff members collect information quarterly from the VHTs, who use standard report forms of the MOH. The project provided the VHTs training on how to improve their recording on these forms to capture their activities. Results are used in the quarterly meeting with the VHTs to discuss their progress and make plans. To enable the health centers to better track referrals from VHTs, the CSP provided each health center with a file box in which to keep the referral forms. Project staff review this file regularly and give feedback to the VHTs to improve appropriateness of the referrals. CSP has recently developed a spreadsheet to track supervision of the VHTs. Project effort related to improving the HIS involves routine review of the patient register for completeness and the training that was provided as part of IMCI on record keeping. CSP staff have also been directly assisting with the immunization registers at static and outreach clinics. CSP conducted qualitative studies as part of developing the behavior change strategy. They have also used semi-quantitative methods to assess the situation of emergency transport and are in the process of mapping coverage of VHTs and access to health services to assess equity. The project also used the Year 1 and midterm LQAS data to identify underperforming supervision areas and respond. Project staff who provide joint supervision of health centers with the DHO staff model correct use of the MOH tools, keeping a separate copy in order to follow-up with HC staff. As of yet, the Health Unit Management Committees (HUMC) are not using MOH data to make plans and take decisions. One HUMC did use population data to determine the need for establishment of additional points for immunization outreach. Having health center staff share some basic data every month on coverage and illnesses would help the HUMCs focus on making action plans beyond the obvious infrastructure needs. 10 E. Presentation of Progress towards Achieving Results Table 2. Monitoring and Evaluation Matrix – Progress at Mid Term Objective/ Result Indicators Baseline Value EOP Target LQAS Results 2011 Comments Objective/ Result 1: Communities assume responsibility for their own health through the strengthening community capacity (Village Health Teams, Parish Development Councils, and Health Sub-districts). % of VHTs who are women 5% 25% 24% More female VHTs were recruited during initial district mapping of VHTs % of VHTs who received a supervisory visit during the last 6 months 0% 50% 49.3% Trained and supervised 560 VHTs in Erute North in iCCM/C-IMCI The percentage of communities with an emergency/referral transportation system with at least one use within the past three months. 0% 70% Data not available Informal survey completed of emergency transport needs. Community mobilization and data collection planned for Years 3 and 4. % of PDCs and HUMCs that are using information from community HIS for decision making in the last year, with at least 1 concrete example of action taken 0% 70% 0% Reactivated HUMCs at the four health facilities the project supports. HUMCs are not using information from HIS for decision-making. HUMC membership includes PDC representatives. Objective/ Result 2 Improved health (C￾IMCI and child care (ECD) behaviors among mothers of children <5 years % of children 0-5 months who were exclusively breastfed during the last 24 hours 73.6% 95% 67.73% No improvement. Confidence intervals overlap. CSP has not yet heavily promoted EBF. % of children aged 0-23 months who were put to the breast within one hour of delivery 29.0% 60% 22.73% No change. CSP has not yet heavily promoted optimal BF, but has done formative research. % of children aged 0-23 months who did not receive prelacteal feeds during the first 3 days after delivery 46.6% 75% 55.03% No change. CSP has not yet heavily promoted optimal BF, but has done formative research. IYCF: % of children aged 6-23 months who are fed according to a minimum of appropriate feeding practices 23.14% 50% 42.25% Some change in both dietary diversity and feeding frequency but both could be affected by seasonal differences between BL and MTE. % of children 0-23 months with diarrhea in the last two weeks who received Oral Rehydration solution (ORS) and/or recommended home fluids. 47.2% 70% 53.51% No change. VHTs are not regularly supplied with ORS and are teaching homemade sugar and salt solution instead of promoting use of usual home fluids. % of children 0-23 months with diarrhea in the last two weeks who were treated with Zinc. 0.9% 30% 2.6% No change. Lack of zinc at HCs and not part of MOH supply kit. Indicator should be dropped. 11 % of mothers of children aged 0-23 months who live in households with soap at the place for hand washing and who washed their hands with soap at least 2 of the appropriate times during a 24 hour recall period 54.0% 80% 75.34% Significant increase. VHTs have been promoting hygiene and sanitation with encouragement from CSP and health centers. % of children aged 0-23 months with chest￾related cough and fast/difficult breathing in the last two weeks who were taken to an appropriate health provider. 57.8% 80% 86.11% Significant increase. This may be attributable to CSP training of VHTs in iCCM and referrals. % of children aged 0-23 months with chest￾related cough and fast/difficult breathing in the last two weeks who were treated with an antibiotic 34.7% 70% 64.36% Significant increase. This is a combination of improved care-seeking through VHT training in iCCM and improved treatment due to the CSP supported IMCI training. ANTHROPOMETRICS % of children 0-23 months who are not underweight (-2 SD for the median weight for age, according to WHO/NCHS reference population) 72.3% 88% 82.36% No change. The project area is quite food insecure and there has been no real promotion yet by CSP on optimal breastfeeding and complementary feeding. This should improve by the final survey. MNC % of mothers with children aged 0-23 months who received at least two Tetanus Toxoid vaccinations before the birth of their youngest child. 75.7% 90% 73.49% No change. This is somewhat surprising considering that CSP has heavily engaged in all immunizations and attendance at ANC is much increased.. However, there are occasional shortages of TT vaccine for outreach clinics. % of mothers with children aged 0-23 months who received at least 2 doses of IPT during the pregnancy with this youngest child. 35.0% 60% 59.13% Significant increase. The number of women having four antenatal visits also increased. VHTs urged to promote ANC by CSP and DHO. % of children age 0-23 months whose births were attended by skilled personnel 35.3% 50% 53.3% Significant increase. There is major MOH promotion for institutional deliveries and education and referral provided by VHTs. . % of mothers of children 0-23 months who received a post-partum visit by an appropriate trained health worker within three days after the birth of the youngest child. 16.33% 50% 30.04% Significant increase which can be attributed to CSP which trained VHTs to conduct post￾partum visits. This will increase more when VHTs also visit those who delivered at HC. % of mothers of children 0-23 m are able to report at least two known maternal danger signs during the postpartum period 2.0% 80% 19.98% Improved but still a long ways to go. This was not a focus of CSP until now but will be emphasized through Mother Leaders. 12 IMMUNIZATION Percent of children aged 12-23 months who received measles vaccine according to the vaccination card or mother’s recall by the time of the survey 77.0% 90% 79.73% No change in spite of major CSP inputs into immunization activities to assure delivery of services and to mobilize participation through VHTs. % of children aged 12-23 months who are fully vaccinated (received BCG, DPT3, OPV3, and measles vaccines) by 12 months of age, card verified 15.5% 50% 37.8 Improved, but still a high drop-out rate. There has not been concerted house-to-house tracking of defaulters by the VHTs who are spread too thin. ECD % of mothers of children aged 0-23 months who provide cognitive stimulation to their child in the form of games such as “where are your eyes”, etc. 38.0% 80% 68.74% Significant increase. This reflects diffusion far beyond the groups of parents in intervention areas who participated in ECD sessions. % of mothers of children aged 0-23 months who told their child a story, sang a song, or spent time naming objects for child at least 2 times in the past week 22.7% 75% 40.06% Significant improvement. This reflects diffusion far beyond the groups of parents in intervention areas who participated in ECD sessions. % of mothers of children aged 0-23 months who report that they talk or sing to the child while feeding the child 57.7% 80% 65.36% No change. The question in the survey may not have been well-understood or well-worded. Objective/ Result 3 Improved quality of care in health facilities through strengthened capacity in IMCI and MNC % of HC have a passing score with regard to the assessment of sick children (> 80% of patients observed in each facility have all 5 assessment tasks performed on them by the HW) 0% 75% 0% No change in % with passing score; however, the % average HF attainment (the average percentage of the 5 tasks that were performed at least 80% of the time in each HF) went from 21% to 68% % of HC in which > 80% sick children treated according to protocol 25% 75% 100% Great improvement. All four health facilities prescribed correct treatment. Unfortunately, they could not always provide the drugs. % of HC staff received a supervisory visit within 3 months 25% 75% 75% Increased from one to three due to CSP support for joint supervision. Objective/ Result 4: Strengthened institutional capacity of MTI and DHO to implement effective and efficient child Demonstrate improvement in 6 low-scoring priority areas identified during the Organizational Capacity Assessment The action plans for 2 t of the 6 priority areas, Strategic Planning and Communication and Decision Making, have been completed. The remaining 4 are in progress and ongoing Action plans for 6 priority areas implemented and scores improved Action plans developed and updated in Annex 7 13 survival activities Lessons learned and best practices are disseminated utilizing at least three different media (program manual, presentations, web site, program guidance and meetings with stakeholders) MTI CSP staff visited the Health Partners CSP to learn about their project implementation strategies, means of relating with local government systems and communities, reporting formats and how they work with the VHTs. MTI’s has shared successes and lessons learned with the DHO and NGOs working in Lira during quarterly district level health coordination meetings and during meeting with PDCs and USAID in Kampala and Gulu. % of health facilities received joint DHO/MTI supervision visits once per quarter 0 75% 100% All four health facilities receive quarterly joint supervision visits. 14 F. Discussion of the Progress towards Achieving Results 1. Contribution toward Objectives/Results Objective No.1: Communities assume responsibility for their own health through strengthened community capacity (Village Health Teams, Parish Development Councils, and Health Sub-districts). CSP is largely responsible for the current capacity of the Village Health Teams. While working in close concert with health center staff, it is CSP that made possible the five-day iCCM training and which instituted the quarterly meetings to support and reinforce the VHTs in their community work. (See training matrix in Annex 8.) CSP staff members have frequent contact with the VHTs in their communities and participate in quarterly meetings with VHTs at the health centers. Since the health centers are unable to supply them with drugs for malaria and pneumonia treatment, VHTs are using the iCCM training to create awareness of danger signs that require care-seeking and to make referrals. A review of the referral forms kept at the health centers shows that individual VHTs are making up to 20 referrals per month. The beneficiaries emphasized to the MTE teams that these referrals enable them to get timely attention at the health center. Health center staff are also pleased with the referrals which are encouraging families to seek care before it is too late. As will be discussed below, there is simply not adequate coverage of VHTs for the population to allow them to effectively engage in promoting other health behaviors at the household level. Community members express appreciation to the VHTs for their work, and in some cases, support them through loan of a bicycle, but there is a general misperception by community members that VHTs are being paid. This actually leads to some jealousy of VHTs in some communities. VHTs do receive support from community leaders who allow them to use meeting time for presentations and help them convene special meetings. Community leaders also help the VHTs mobilize community members for events such as Child Health Days. The Parish Development Councils have been engaged in the initial CSP work in the communities and are very aware of the need to improve health. Together with the Health Unit Management Committees (HUMC), the PDCs are contributing to improving health facility infrastructure. The HUMCs were re-activated by the CSP. Made up of representatives of the communities in the catchment area and health center staff, the HUMC of each sub-county meets quarterly with a goal of improving service delivery. To date, most of their efforts have focused around improving health facility infrastructure, mostly through advocacy to local government to provide funds for certain projects. Examples of other HUMC efforts: The Bar Apwo HUMC pointed out to the health center that the new buildings needed handicap access, which was then added. One of the HUMCs feels they have been able to reduce drug theft among staff, and reduce rudeness of staff towards patients. The Aromo HUMC used population data to decide another outreach point was needed in the sub-county. As yet, there has not been routine sharing of HIS data with the HUMCs. CSP is now facilitating this to enable the HUMCs to pay attention to needs other than infrastructure. HUMCs are linked to the PDCs through the many individuals who participate in both HUMC and PDC. 15 Objective No.2: Improved health (Community-based Integrated Management of Childhood Illness) and child care (Early Childhood Development) behaviors among mothers of children under five years of age. As seen in the chart above, considerable progress has been made on improving care-seeking for children with signs of respiratory infection which is likely due to the VHT explanations to parents and the referral system. This is attributable to the CSP training of VHTs in iCCM and referrals. The health centers have always supported VHTs to promote hygiene and the recent survey shows improvements related to hand-washing. During the MTE focus groups, beneficiaries reported gradual adoption of other behaviors, particularly in sanitation. In Section F.4 below, there is detailed discussion of overall design issues which has limited promotion of other health behaviors. The baseline was already relatively high for some of the ECD behaviors. The OR strategy for ECD called for intervention and control parishes within each sub-county. In the intervention areas, the ECD training was provided to large groups of parents in a series of five afternoon sessions. This was hardly the in-depth ECD training visualized in the project design. Participants, from nurses and VHTs to mothers and fathers, were enchanted with the ECD concepts and skills and quickly shared them with others. For example, nurses started teaching the skills to other mothers from control areas bringing children to the health center and parents told their relatives in the control areas while VHTs shared the training with colleagues in control areas. The qualitative results and high level of interest are positive (See Annex 6). The OR that was completed in Lira for the MTE was quite successful and statistically accurate, showing the reality of the situation regarding the hypothesis that the addition of ECD to child health programming leads to better health outcomes when all other health interventions are the same. Prior to the survey there was a valid concern that ECD messaging had been inadvertently given to mothers in control group areas, particularly at the HF based programs highlighting ECD. Because this was a real and valid concern, we went to great lengths to determine if contamination (in this case ECD messaging being given to mothers in control areas) had occurred, and if so, where. We took the following steps prior to determining if the study could continue: 1. The Africa Health Advisor and Monitoring and Evaluation Officer completed a mapping exercise in each SA to determine if contamination had occurred, and if so , in which areas. 2. When contamination was discovered, we carefully determined the areas of possible contamination and labeled the entire area as contaminated to ensure the validity of our results. A map detailing the areas of contamination (denoted in blue) was constructed by the Monitoring and Evaluation Officer. 3. The HQ Sr. Advisor in M&E flew to Lira to review the results and determine the OR plan. It was determined that because there were 3 relatively small areas of contamination, the survey could continue with some modifications. The control areas that had contamination were not used for this portion of the survey and the questionnaire in 16 the control SAs was changed slightly (see 4.) to act as further security from contamination. 4. The questionnaire in the control SAs had an extra question added at the beginning, in the protocol to determine if the mother is eligible to answer the survey: a. ASK THE MOTHER ONLY: In the last year did anyone teach you any new ways (baby cues, baby massage, physical development, cognitive development, love and affection, any other) to interact with your children in your own home which would help your children grow healthier? Onyo ngatoro obin opwonyi kede yore anyen (anyut me wang atin ka kono yie yom onyo pe, rwayo otino, yore ame otino dongo kede,yore ame otino cako niango kede ping, mara kede anyut me yom yic bot otino, en okene ame pe oketo piny kan) me leleo tam kede otinoni paconi ame twero konyo otinoni me dongo aber iyi mwaka acel ame okato angec? i. If No, continue with interview ii. If Yes ( probe further and if mother mentions one of the domains or new ways above, go to next nearest doorway) 5. SAMPLE SIZE: a sample size calculation was performed to ensure the correct sample size of both groups. A sample size of 95 would be sufficient, but we used a sample size of 99 so that each SA would have a sample size of 33. An email was sent to Dr. Bill Weiss at JHSPHJ to ensure the sample size calculation and thinking behind it was correct. The study clearly indicates that the addition of ECD to health programming in Lira did not improve health outcomes. When looking at the data, reasons for the results could be attributed to: 1. While ECD is a positive intervention, there are other factors more directly important in determining the effect of health interventions a. The data could indicate that SAs further from the available HFs did not perform as well as those closer to the HFs. i. SAs 2 and 6 did not meet the DR only 3 times (counting indicators with benchmarks) and are both control areas that are close to HFs, ii. The SA that did not meet the DRs most often (SA3 did to meet the DR 7 times) is an intervention SA but is a far distance from the HF. iii. SA 4 is a control SA that did not meet the DR 8 times, and it is also far from any HFs. b. Further study with this theory in mind would have to be done to statistically prove this 2. This project is working extremely hard and struggling to complete the many interventions required of a CSP. It could be that taking time, energy, staff, and resources away from interventions that DIRECTLY affect health outcomes (CSP Interventions) to put them towards interventions that only INDIRECTLY improve health outcomes (ECD Interventions) could cause the CSP interventions to be less complete and successful. This would explain the fact that 7 health indicators were significantly higher (>10%) in the Control areas, while only 3 of the 26 health indicators (not including the 3 actual ECD indicators) measured showed a significantly (>10%) higher result for the Intervention area. 3. It could be argued that the uptake of ECD was not complete, but this would not explain why the Control SAs performed better overall than the Intervention SAs, while the above 2 theories explain this. 17 Objective No.3: Improved quality of health facility services through strengthened Integrated Management of Childhood Illness and Maternal and Newborn Care capacity. The major input to improving quality of services was the IMCI training provided to personnel from all four health facilities. Both the MTE survey and the R-HFA show that children are now receiving antibiotics and being treated according to protocol. This is a major step forward, notwithstanding the major contextual constraints around health services described below. Designated health personnel also received training on maternal newborn care as shown in the Training Matrix in Annex 8. While the MOH is pushing institutional delivery, they are at the same time creating a barrier by requiring expecting women to bring all the supplies for the delivery including gloves, plastic sheeting, razor blades, etc. This is simply beyond the means of most families. Until very recently, another NGO working in the area was providing the “delivery kits” as an incentive for women to deliver at the Aromo health center. With this support now gone, it is uncertain whether the level of institutional deliveries will be sustained. The mentoring and joint supervision from CSP staff have been helpful and appreciated by health center staff. CSP has implemented good supervision tools, particularly for IMCI. Joint supervisions with DHO staff are held on schedule, but sustainability is an issue since CSP is providing transportation for DHO staff to reach the health centers. VHTs were given a short training on making home visits to post-partum women and newborns. This clearly shows up in the recent LQAS survey where there is a significant increase in the number of women who received a post-partum visit. To date, the VHTs have concentrated on visiting women who deliver at home, but will now be encouraged to visit women who deliver at the health centers because those women are often sent home within a few hours after delivery. VHTs may need a refresher training on newborn danger signs which were covered briefly among the many other topics in iCCM training. Objective No.4: Strengthened institutional capacity of MTI and the Lira District Health Office to implement effective and efficient child survival activities. (This section provides information for the New Partner Grantee requirement to report on changes in organizational capacity.) In reality, this objective focuses on improving institutional capacity of MTI Uganda. The institutional strengthening of the Lira DHO is occurring through activities under Objective 3, primarily through improvements in supervision and staff capacity. The institutional strengthening of MTI Uganda is led by the MTI regional staff for Africa who are providing follow-up on areas of concern identified in the Organizational Capacity Assessment conducted at the beginning of the CSP. The CSHGP grant and the Malaria Communities grant are the first large USAID funding that MTI Uganda has managed. Inherent with this management function is learning to adhere to USAID rules and regulations for financial management and reporting. In this, MTI Uganda has been challenged but receives assistance from MTI headquarters finance officers. The Management Evaluation in included as Annex 18 The opportunity to conduct an Organizational Capacity Assessment (OCA) afforded MTI U an avenue for organizational growth and capacity building. Following are key areas and actions, identified in the OCA for organizational capacity building, that were undertaken during this period (see Annex 7). Monitoring and Evaluation: All MTI U projects carry out M&E, with new tools being adapted. A comprehensive M&E plan is to be developed in FY 13. All key management staff have completed MTI Project Cycle Management (PCM) training with the exception of new staff in SW Uganda. It is planned to roll out PCM training in the next year to non-management staff who demonstrate a readiness to learn and apply it in their work. Staff Salary and Benefits Policy: MTI U drafted new policies that clarified benefits but due to a delay in a field visit by MTI HQ’s HR manager, policies will be finalized during that visit in May 2012. An assessment of staff capacity by the MTI U HR manager is currently underway. Communication and Decision Making: In July 2011 MTI U decentralized their structure, creating regional teams of projects in Northern Uganda and SW Uganda. This streamlined communication and reporting within MTI U. Project managers meet with their teams weekly, followed with monthly regional meetings and quarterly management team meetings which facilitate project reporting, sharing of upcoming key activities, and any constraints or obstacles are addressed. With the placement of the Africa Regional Health Advisor in Uganda, a regular system of communication and technical advising with project teams was developed. Succession Planning: While delegation is practiced and an OIC is appointed as the situation arises, an overall plan and policy is yet to be developed. Strategic Planning: MTI U engaged in a strategic planning process facilitated by the Africa Region Deputy Director and led by the Country Director. The MTI U Strategic Plan, based on MTI’s agency plan, was finalized in March 2011. The plan will be reviewed annually and used to guide program and organizational growth. 2. Contextual Factors There are several contextual factors affecting progress towards objectives. The first affects the care-seeking indicators and the second objective on improving service delivery. The MOH in Uganda is under-funded and suffers a crisis in management and personnel. As can be seen in the R-HFA results (Annex 4g), the health centers in the target area lack many critical pharmaceuticals and supplies including the most basic equipment. The qualitative evaluation confirmed shortages of the essential medicines for IMCI and prenatal care in all four health centers. This not only discourages people from seeking care, but also undermines the morale of the health workers who feel handicapped by their inability to provide the necessary medicines. They write prescriptions which people can take to local drug sellers. During the focus group discussions, beneficiaries say they may as will skip the step of going to the health center and just 19 go directly to the drug seller. Of course, drug sellers are not trained to diagnose and often sell adulterated drugs, but the population does not understand these concerns. Few health professionals want to be posted in rural areas, even though most health centers provide some type of housing for them. They request transfers as soon as possible to centers on main roads or in urban areas. Of the 34 doctors and nurses that CSP trained in IMCI, eleven are no longer working in the target area. While not a loss to Uganda, this is a detriment to the project meeting its indicators for appropriate assessment and treatment of illnesses and to the population receiving quality services. Another contextual factor that may be affecting results in Aromo Sub-County, which shows consistently lower indicators, is lingering psycho-social impact from the years of violence and life in camps for internally displaced persons (IDP). The majority of this population returned to their homes just prior to the design of CSP and others are still returning. While they are happy to be back on their land, recent life has been very hard due to the need to start from scratch with no resources to rebuild their homes and clear badly overgrown fields. There is still a very high level of food insecurity in the target area, particularly during the dry season (www.fews.net/ug) which affects the ability of families to attain optimal practices for maternal nutrition and complementary feeding. In context of MTI Uganda, this CSHGP grant, together with the current Malaria Communities Project, is their entrée into development rather than relief programming and their first large USAID funding. The grant requirements and USAID rules are more complex, require greater technical rigor and have a demanding implementation schedule, and have posed a learning curve for the country office staff. 3. Role of Key Partners Partners Role in Project Result of Collaboration/ Suggestions for improvements Hands to Hearts International Technical assistance for the ECD component. HHI was responsible for adapting their model to Uganda, preparing Ugandan trainers, developing the TOT curriculum and conducting the initial TOT for project and health center staff. HHI has provided excellent TA on the content of ECD for infants and young children. More importantly, the cascade training curricula should have been reviewed by the ECD consultant before they were rolled out. A detailed plan of action for each step of the ECD process, aligned with the budget, would have facilitated a smoother roll-out. On the other side, this collaboration provided HHI their first experience with quantitative measurement of results and selection of indicators. DHO Lira District and health center staff Service delivery, managing their own procurement and personnel, HMIS, and for supervision. District leadership is involved to Collaboration on planning was particularly close during the initial phases of the project when the major trainings were being rolled out. The 20 4. Overall Design Factors that are Influencing Progress towards Results There are several design factors that have influenced progress. These include the number of technical interventions, the allocation of level of effort, reliance on VHTs and existing community groups as channels for promoting behavior change, and dependence on iCCM training to give VHTs the necessary knowledge to promote key practices of the CSP. Number of technical interventions: Although the project is an entry grant, the proposal authors selected five technical intervention areas. This number of interventions would stretch even seasoned project staff. In addition, the project has, in essence, a sixth intervention with the ECD component. It is unfortunate that proposal reviewers did not suggest that MTI focus on fewer technical interventions. Three technical interventions with ECD would have been more manageable. Level of effort: Neither the proposal nor the DIP assigned a level of effort to the ECD component. Furthermore, the first project manager did not share the level of effort (LOE) plan with his staff nor refer back to the Detailed implementation plan himself when monitoring project implementation. Seeing the health center outreach for monthly immunization services and the scheduled immunization days at health centers as opportunities for direct contact with the target population, project staff began participating in all the immunization events to present health talks or supervise the VHTs presenting health talks. They agreed to bring the vaccine from the DHO and also transporting the vaccinators to the outreach sites, which meant staying there all day to transport them back and supervise the process. To fill that time, they began helping fill out the registers and more. As a result, instead of being just 10% LOE, immunization has become the dominant technical intervention. The following chart depicts the planned LOE, along with very subjective estimate of the actual LOE, and the projected LOE for the remainder of the project based on the revised work plan and action plan. Due to a clause in the MOU with the DHO stipulating that MTI will support the bi￾annual Child Health Days, some continued level of effort has been retained for immunizations. Level of Effort for Technical Interventions Intervention Planned % LOE Estimated % LOE first 29 months Projected % remaining 19 months Diarrhea 20 10 10 Pneumonia 25 10 10 some degree in project planning, monitoring, and evaluation. joint supervision seems to be going well and the DHO appreciates the support. The DHO and health centers have taken advantage of the CSP for transportation of vaccine, cold chain supplies, and vaccinators. This has obviated the DHO responsibility in EPI which is counter-productive to sustainability. 21 Immunizations 10 55 5 Nutrition (IYCF) 20 5 35 Maternal Newborn Care 25 10 30 Early Childhood Dev. 0 10 10 Reliance on VHTs and existing community groups as channels for promoting behavior change: According to the national plan, there should be five VHTs in every village, the number somewhat variable based on population. In the target sub-counties, there are only two VHTs for most villages. As more IDPs return home, there are additional villages which do not yet have VHTs. Other organizations working in the target area on reproductive health, HIV/AIDS, water & sanitation and malaria are also engaging the VHTs, therefore, there are multiple demands on their time. The initial orientation the VHTs received from health center staff when they were selected did not include an expectation to convene educational sessions explicitly for pregnant women and those with children under two. The VHTs sometimes disseminate messages through existing community meetings such as those called by community leaders for another purpose, or church services. These are ad hoc and not focused on the CSP priority audience, with no way of knowing how many are being reached. VHTs report that they use occasional home visits to promote sanitation and hygiene or when called upon in case of an illness in the household, and very recently, with CSP training have started making postnatal home visits. Initially, the project did not assist the VHTs to select and focus on any particular topics in their contacts with community members. Based on first annual survey data, the HQ backstop helped staff institute this change late in the second year, coming up with a list of sequenced topics for each month. Again, there is no defined avenue for the VHTs to reach the CSP population with the topics. Dependence on C-IMCI training to give VHTs the necessary knowledge to promote all key practices: When the proposal and DIP were written, C-IMCI was being implemented in some regions of Uganda. MTI assumed that the C-IMCI training was missing only the newborn care component and, once that was added, would serve to give the VHTs sufficient knowledge to promote all key practices of the CSP. Immediately after the DIP, the MOH adopted integrated Community Case Management (iCCM) with newborn care included. CSP used this government module to train 560 VHTs. The module, in fact, focuses on identification, treatment and/or referral of sick children, including those with acute malnutrition. There is little content on prevention, in general, and none on maternal care or infant and young child feeding. CSP staff did not recognize these deficiencies which have seriously limited the VHTs. Although the VHTs are not supplied with drugs needed to fully implement case management, the iCCM training has been very effective in giving them capacity to recognize illnesses or newborn complications and make appropriate referrals. The suggested changes to address the issues of design described above are included in the overall conclusions and recommendations below. These were the principal findings of the MTE. 22 G. Discussion of Potential for Sustained Outcomes, Equity, and Community Health Worker Models 1. Progress towards sustained outcomes MTI used the Child Survival Sustainability Assessment (CSSA) framework to organize and explain the sustainability strategy during a workshop in June, 2011, which was attended by staff from the CSP, Malaria Communities Project, and the DHO. The sustainability strategy with the up-dated progress towards indicators is presented in Annex 10. Focus of the sustainability strategy is on behavior change and improved service delivery by the health services. Since the VHTs are a government institution which was in place before the CSP and is mandated by the national policy, health centers will have to sustain them, although they will not be able to continue the quarterly meetings, though they meet monthly with the VHTs at the parish level. CSP will work with the HCs to plan for some continued level of supervision and in-service training for VHTs. The two health centers Aromo and Ogur that had started the Mother Leader Groups initially may sustain those original groups, but the CSP certainly does not expect the health centers to assume all the new MLG volunteers. Rather, CSP is considering the expanded MLGs as a short-term SBC strategy to intensely promote behavior changes. The MLGs will remain in their communities where they can continue to serve as resources but are not expected to continue their activities past the end of CSP. With this MTE, the CSP has begun to plan for phase-out, starting with rather immediate phase￾out of many immunization-related activities. This will give the health centers the responsibility for implementing alternate plans for transportation of vaccine and vaccinators. 2. Attention to equity Using results of the LQAS surveys, R-HFAs, and knowledge of the target area, CSP has done some analysis of equity. The proposed mapping of VHT coverage, and distances from villages to health centers will contribute greatly to this assessment of equity. The major issues seem to be geographic access to basic services, quality of services, gender considerations in terms of women having the ability to decide when to seek services and, lack of discretionary funds to pay for transportation or medicines. Geographic access is affected by the size of the HC catchment areas, lack of roads, and lack of any kind of transportation other than walking or bicycles in most communities. The outreach clinics do bring some basic services such as immunizations closer to the people. The Aromo HUMC was successful in getting an additional outreach point established after analysis of population data for the sub-county. The CSP plans to mobilize the VHTs and communities to develop emergency transport plans will increase access to services in an emergency, but will not increase access to routine care. Quality of services has been previously discussed in this report in relation to stock-outs, lack of functioning equipment, etc. This situation is compounded by the work ethic of many health 23 personnel who report for work very late or not at all. One of the chief complaints of all people interviewed during the MTE was the long waiting times at health centers. Gender norms often preclude women to deciding on their own when to seek health services or participate in outreach or health education activities. This is already being addressed in many communities through VHT contacts with men and may be further ameliorated when men are provided more information on health concerns, specifically the information on maternal health and danger signs, which they requested during MTE focus groups with males. Women’s multiple roles in farming, food processing, housework and child care make it difficult for them to seek services, particularly when the travel time and waiting times at the health center may take them away for an entire day. Alcoholism is highly prevalent throughout Uganda and complicates gender issues when men no longer play a productive role in the household but still make decisions about how women use their time or access family resources. Financial constraints are severe among the target population, many of whom have no cash income. To pay for transportation or medicine, they may have to sell off food stocks or productive assets. This highlights the need to stress preventive health behaviors such as hygiene, sanitation, exclusive breastfeeding, use of LLITNs, immunizations, etc. 3. Role of Community Health Workers As previously described, the community health workers or VHTs are a government structure which the MOH sees as key to their outreach. While the vision of the MOH is not exactly the same as the CSP in terms of promoting behavior change, the health centers appreciate the role of the VHT as channels of communication with the communities for events such as immunization days, for promoting use of services such as ANC and deliveries, and for their referrals. MTI has given the VHTs more training and supervision than the health centers, which face serious budget and transportation constraints. The health center staff can continue to use the monthly meetings as a means of supervision and to provide some training. Currently, the CSP staff meet quarterly with the VHTs for an all-day session which includes reviewing reports, discussing issues faced by the VHTs, coordinating work with other events and entities, and imparting some training. These sessions will become much more effective when held at the parish rather than sub-county level due to much smaller groups, but the health centers will not be able to convene multiple groups, hence, will have to revert to the large sub-county meeting. Before the CSP started, these sub-county level meetings seemed to be sufficient to motivate the VHTs to work to meet the expectations of the health center staff. Since the official VHTs have limitation in achieving coverage needed to promote substantial behavior change, some health centers were already working with an adjunct group called Mother’s Groups comprised of women volunteers. The CSP picked up on this and expanded to include 300 women in “Mother Leader” Groups. Parish leaders helped recruit energetic women with some literacy to whom MTI or HC staff gave a one-day training on some key messages – a very abbreviated C-IMCI. During the past six months, these motivated volunteers have started promoting behavior change in casual conversations, small groups convened after church, etc. As a result of the MTE, MTI will greatly expand this cadre in the coming months, using anapproach similar to the Care Group model to reach every household with a pregnant woman or child under two. The Mother Leaders will continue to coordinate closely with the VHT with an end to 24 serving as support to the VHT rather than a replacement. MTI sees the MLGs as a BCC strategy, not as another cadre for the MOH to have to supervise and support after the project ends. If the DHO decides to expand to 5 VHTs per village, these women would be obvious candidates for filling the additional slots. 4. Contribution to Global Learning At this point, there are no important lessons learned or promising practices from the project that would necessarily be considered for replication on a broader scale. The CSP is simply making a solid contribution to improving maternal child health in Lira District using standard approaches and following existing Government of Uganda programs and policies. H. Conclusions and Recommendations Conclusions:  The CSP got off to a relatively fast start with excellent support from MTI HQ and consultants for the surveys, and DIP. They also accomplished the major trainings in IMCI and iCCM during the first year. MTI HQ and regional technical support for the project exceeds expectations.  The R-HFA shows there has been some improvement in certain areas of service delivery, but most are stagnant or have gotten worse. In spite of an MTI donation of essential equipment, there is still a shortage or the equipment is not functioning. Pharmaceuticals are in short supply or not available due to supply chain issues far beyond the scope of the CSP. On the other hand, health workers have improved assessment capacity and are prescribing appropriate treatments.  The MTE KPC shows significant improvements in some indicators while others have not improved. The indicators which improved most are largely related to the iCCM or post-natal visit training for VHTs. The government promotion of institutional deliveries has been a factor in improving the indicator for births attended by skilled health personnel.  The VHTs are quite committed and the iCCM training prepared them well to identify illnesses and newborn complications and to make referrals. They need more training to enable them to promote maternal health practices including recognition of danger signs by family members, and to promote optimal breastfeeding and complementary feeding. Beneficiary families expressed keen interest in learning more about these topics which are critical for child health in the target area.  The women trained so far as Mother Leaders are quite motivated and willing to make home visits. Expansion of the Mother Leader Groups will provide needed support to VHTs in reaching every household to promote behavior change, monitor appropriate care-seeking, and mobilize the community around issues such as lack of sanitation. 25  The KPC shows much higher levels of knowledge for two ECD practices. Parents who participated in the training were enthusiastic about learning about ECD and about possible impact on broader family life issues. The information has spread beyond the intervention areas through motivated nurses and VHTs.  During the first two years, the implementing staff suffered from inadequate guidance on organizing their time, balancing level of effort with requests from health center staff, and planning in accordance with the DIP. The current project manager shows great potential for providing needed leadership, given adequate support from the country office, with planning, tracking progress of implementation of the project workplan and conducting staff performance reviews according to MTI protocol.  The actual vs. planned level of effort for technical interventions has been unbalanced from the beginning with a disproportionate amount of effort on immunizations. This has affected progress towards achieving other indicators and has not resulted in increased coverage of immunizations, which requires a strategy different from assuring delivery of vaccines and vaccinators to outreach points or general promotion of immunizations. Recommendations: 1) Triple the number of Mother Leader Groups to reach every household with a pregnant woman or children under two with key messages on maternal newborn care, complementary feeding, and optimal breastfeeding. While supporting behavior change in these interventions, the MLGs can also promote sanitation and monitor child health cards to minimize immunization drop-outs. 2) Train VHTs and MLGs in nutrition using the WHO/UNICEF curriculum for East Africa which has been left with the project manager. This can be done in two hour segments once a month alternating with MNC training for a total of four hours of training per month. This may be supplemented with the Freedom from Hunger breastfeeding and complementary feeding modules that were previously sent to the project. The new Core Group material on newborn care at home can serve as a basis for training Mother Leaders and VHTs in newborn care. 3) Withdraw support to routine and outreach immunizations in order to free up staff time to focus on the other technical interventions, particularly maternal newborn care and nutrition, including breastfeeding, complementary feeding, and maternal nutrition. To improve complete coverage of immunizations, focus on identification of DPT and OPV drop-outs at the household level. Meet with District Health Office as soon as possible to share MTE results, review the MOU and explain the project’s decision to reduce support for vaccine and staff transport for outreach and assistance with immunizations to a level consistent with the MOU (only for Child Health Days). 26 4) Confine continuation of ECD to the available budget and to having a greater impact on a small area. The VHTs and PEs need more comprehensive training including the TOT aspects in order to better transmit learning to community members. The community groups must be kept to no more than 10-12 to ensure optimal learning. The sessions may be repeated twice for reinforcement. 5) Realign remaining budget (within USAID rules) to cover as many of the following priorities as possible: an additional staff member to have one person available to focus on ECD (match funding), reproduction of the quality IEC materials available from the MOH nutrition cluster, incentives for VHTs and MLGs, and training for VHTs and MLs. 6) Continue with plans for the equity study and for complete mapping of VHT coverage. The results may implicate coordination with the health centers to increase the number of VHTs and for CSP to train any new ones. 7) Conduct quarterly VHT meetings and MLG bi-weekly meetings at the parish level to reduce group size for effective learning. 8) For M&E, the indicator for zinc treatment needs to be dropped. There are no zinc supplies and UNICEF is still working with the MOH at central level on a zinc protocol. The indicator for emergency transportation plans would be more useful for measuring project effort if the last phrase “with at least one use within the past three months” is dropped. MTI could report in the final survey the total number of transport plans, then, separately as an addendum, report whether any had been used in the last three months. We would prefer that communities not suffer emergencies requiring frequent use of the emergency transport system. 9) MTI management should provide the project manager adequate financial information on a quarterly basis to enable her to adjust project activities appropriately. I. Action Plan for Responding to Evaluator Recommendations Recommendation 1: Triple the number of Mother Leader Groups to reach every household with a pregnant woman or children under two with key messages on maternal newborn care, complementary feeding, and optimal breastfeeding. While supporting behavior change in these interventions, the MLGs can also promote sanitation and monitor child health cards to minimize immunization drop-outs. MTI Response: Ninety-six Lead Mothers will be identified and trained by MTI staff and VHT Parish Leaders in a five-day training on key MNC and nutrition messages. Next Lead Mothers, MTI staff and Parish VHT leaders will identify and train 43 mothers in each of the 24 parishes as Mother Group Leaders. The role of the Lead Mothers and Mother Group Leaders will be expanded to visit 10 -15 families with pregnant women or children under five years of age in their communities. Expansion of mothers groups will thus cover the entire program area. 27 Recommendation 2: Train VHTs and MLGs in nutrition using the WHO/UNICEF curriculum for East Africa which has been left with the project manager. This can be done for MLGs in two hour segments each month alternating with MNC training for a total of four hours of training per month. This may be supplemented with the Freedom from Hunger breastfeeding and complementary feeding modules that were previously sent to the project. The new CORE Group material on newborn care at home can serve as a basis for training in MNC. MTI Response: VHT meetings are only able to be held quarterly, due to their commitments with other stakeholders. Thus, having two-hour trainings once a month with VHTs is not feasible. VHTs, however, will be trained quarterly during their regular parish-level meetings focusing on MNC, IYCF and breastfeeding. Mother Leader Groups, however, will be able to meet with greater frequency and will receive concentrated trainings on the topics mentioned above, utilizing a combination of Freedom from Hunger curriculum, Core Group Taking Care of a Baby at Home After Birth: What Families Need to Do, and MOH nutrition cluster flip-charts developed with assistance from FANTA. Recommendation 3: Withdraw support to routine and outreach immunizations in order to free up staff time to focus on the other technical interventions, particularly maternal newborn care and nutrition, including breastfeeding, complementary feeding, and maternal nutrition. To improve complete coverage of immunizations, focus on identification of DPT and OPV drop￾outs at the household level. Meet with District Health Office as soon as possible to share MTE results, review the MOU and explain the project’s decision to reduce support for vaccine and staff transport for outreach and assistance with immunizations to a level consistent with the MOU (only for Child Health Days). MTI Response: The CSP Program Manager and the M&E Coordinator met with the DHO on Tuesday, March 27 to plan phase out of transportation for vaccines and staff for routine outreach and static sessions following the April, 2012, Child Health Days. Support for bi-annual Child Health Days will continue. Recommendation 4: Confine continuation of ECD to the available budget and to having a greater impact on a small area. The VHTs and PEs need more comprehensive training including the TOT aspects in order to better transmit learning to community members. The community groups must be kept to no more than 10-12 to ensure optimal learning. The sessions may be repeated twice for reinforcement. MTI Response: For the remaining months of the project, ECD activities will be focused in Aromo sub-county. The VHTs and Peer Educators in the intervention area who were previously trained will be retrained with the five day curriculum. VHTs and Peer Educators will be identified and trained in the comparison area. Training for parents will be rolled out in small groups and repeated twice. Recommendation 5: Realign remaining budget (within USAID rules) to cover as many of the following priorities as possible: an additional staff member to have one person available to focus 28 on ECD (match funding), reproduction of the quality IEC materials available from the MOH nutrition cluster, incentives for VHTs and MLGs, and training for VHTs and MLs. MTI Response: The budget will be realigned and submitted to USAID for approval by April 16, 2012. This budget will not include the hiring of an additional staff member, but it will account for the reproduction of IEC materials, incentives for VHTs and MLGs and training for VHTs and MLs accordingly. Recommendation 6: Continue with plans for the equity study and for complete mapping of VHT coverage. The results may implicate coordination with the health centers to increase the number of VHTs and for CSP to train any new ones. MTI Response: The mapping of VHTs and mothers groups will be completed by Friday, March 23, 2012. The information will be used to train and support VHTs, in coordination with the DHO, for communities which presently do not have access to the home visits, community education and referrals they provide. The results of this mapping will also inform the MLG selection and formation process to cover gaps in coverage. Recommendation 7: Conduct quarterly VHT meetings and MLG bi-weekly meetings at the parish level to reduce group size for effective learning. MTI Response: Both VHTs and MLG meetings will be conducted at the parish level starting in March, 2012. Recommendation 8: For M&E, the indicator for zinc treatment needs to be dropped. There are no zinc supplies and UNICEF is still working with the MOH at central level on a zinc protocol. The indicator for emergency transportation plans would be more useful for measuring project effort if the last phrase “with at least one use within the past three months” is dropped. MTI could report in the final survey the total number of transport plans, then, separately as an addendum, report whether any had been used in the last three months. We would prefer that communities not suffer emergencies requiring frequent use of the emergency transport system. MTI Response: MTI accepts the recommendation to omit the indicator for zinc treatment from the M and E matrix and requests approval from the CSHGP to do so. This was discussed during the February 29th debriefing with the Kampala USAID Mission. MTI also accepts the recommendation to drop the phrase “with at least one within the past three months” from the indicator for community emergency transportation plans. Recommendation 9: MTI management should provide the project manager adequate financial information on a quarterly basis to enable her to adjust project activities appropriately. MTI Response: MTI U has an established process the project manager follows for budget review and fund request. The project manager receives a copy of the project budget at the beginning of the fiscal year that indicates monthly and quarterly amounts per line item. Project funds are then requested based on the work plan and budget in consultation with senior 29 management and the finance department. Project managers also meet together quarterly to review fund balances, discuss findings, and agree on any adjustments. To protect both the organization and the individual, as a matter of policy, MTI-U does not distribute hard copies of quarterly financial reports to staff. Annex 1: Results Highlight A Promising Practice for Piquing Community Interest in Learning about Health and Child Care In Uganda, as in most countries, young families have many competing demands on their time. In a subsistence economy, like northern Uganda, the first priority is food production or income to assure the family will eat and meet the most basic needs. Many activities in that realm, particularly those of women, are very time consuming, such as processing cassava and maize, cultivating fields by hand, or fetching water. Taking time out to attend learning sessions about child care and health is either a luxury or low priority. Furthermore, people’s prior experience with educational sessions which were top-down, incomprehensible, disorganized or just plain boring often leaves them disinterested in attending future learning opportunities, which may be perceived as irrelevant and a waste of precious time. NGOs and government health workers alike often struggle to get young parents, particularly men, to attend educational activities. When rolling out the Early Child Development (ECD) component of the Child Survival Project (CSP) in Lira District, Medical Teams International ( MTI) staff members faced budget constraints in conducting general community sessions due to a decision to offer refreshments to participants. Therefore, staff decided to invite only fifteen couples to each village session. (They offered five sessions per community each lasting about four hours.) The decision to limit participation was predicated on the hope that the couples invited would share the learning with their friends and neighbors. MTI staff worked with local leaders to select the participating couples based on their level of community involvement, willingness to help neighbors, and openness to learning. The whole community quickly became aware that certain couples were being selected to participate is something special. Much to the surprise of MTI staff and volunteers conducting the EDC learning sessions, not only the fifteen invited participant couples showed up to learn, but also twenty to fifty other people! They came out of curiosity or simply a desire to be included in something selective. This happened in most of the 145 target villages. All of the participants, invited and uninvited, became quickly engrossed in learning about ECD and continued to attend all five sessions, even though there were not enough refreshments (the original incentive) to go around. Their continued participation was due not only to a topic novel to them, but also the interactive and dynamic nature of the learning sessions. This approach of creating an aura of a learning event being selective and special appears to be much more effective in motivating participation than a general invitation or urging all members of the target population to attend a meeting. The event must also be well-planned and engaging in methodology in order to maintain interest and assure optimal learning. 1 Annex 2 Technical Support Provided during Years One and Two Technical Support Provided Consultant/ Staff providing technical support Month Assistance to finalize Job Descriptions for CSP positions MTI HQ Child Survival Advisor September 2009 Refresher training for MTI Uganda Director of Operations who was responsible for leading the baseline KPC and Health Facility Assessment, and the Administrator, who was in responsible for data entry. MTI HQ Senior Advisor in Monitoring and Evaluation September 2009 Organizational Capacity Assessment MTI’s Director of Technical Services and Director of Regional Programs January 2010 Technical support to develop early childhood development component of project and pilot ECD curriculum. Early Childhood Development Consultant from Hands to Hearts International February 2010 Training in Social and Behavior Change and the use of the Doer /Non-Doer technique and development of social and behavior change strategy MTI HQ Child Survival Advisor February 2010 Technical support to develop Detailed Implementation Plan External consultant and HQ Child Survival Advisor February 2010 Technical support to pilot TOT for the ECD training component Early Childhood Development Consultant from Hands to Hearts International & Kampala based Hands to Hearts trainer May 2010 Technical support to lead TOT for Health Center workers on ECD training component Kampala based Hands to Hearts trainer July 2011 Technical Support to conduct Year 1 KPC LQAS survey MTI Africa Health Advisor with support from HQ Senior Advisor in M and E January 2011 Technical support to develop focus group discussion guides to identify barriers and facilitators for priority practices identified through Year 1 KPC. Social and behavior change strategy revised accordingly. HQ Child Survival Advisor February 2011 Introduction to Essential Nutrition Actions and Three Delays for accessing maternal care and technical support to develop sustainability plan HQ Child Survival Advisor June 2011 Technical Support to conduct Health Facility Assessment MTI Africa Health Advisor and HQ Senior Advisor in M and E December 2011 2 Technical Support to conduct mid-term KPC LQAS survey MTI Africa Health Advisor with support from HQ Senior Advisor in M and E January 2012 Technical Support during Mid-Term Evaluation External consultant, ECD consultant from Hands to Hearts International, MTI Africa Health Advisor, and MTI HQ Child Survival Advisor February 2012 1 Annex 3: Work Plan Table Activity Status of Activities IMCI Provide IMCI training 45 health facility and MTI staff trained in IMCI including 14 nurses 17 midwives, 4 clinical officers, a medical doctor, 8 nursing assistants, and 11 MTI staff. Conduct clinical IMCI mentoring IMCI mentoring is conducted monthly by the CSP MNC Mentor Conduct joint supervision of IMCI services Joint supportive supervision conducted quarterly, by DHOs, and Health Sub District and MTI staff ICCM/C-IMCI and Community Case Management Identify local/country materials MOH VHT materials procured for iCCM Train VHTs on ICCM/C-IMCI 560 VHTs provided with 5 day training in ICCM and other topics during quarterly and monthly meetings. VHTs provide community education with support from mobilizers 560 VHTs providing community education and home visits. Supportive supervision carried out quarterly. Strengthen referral system from community to health facility by providing training to VHTs and developing pictoral referral cards 560 VHTs referring community members to health services using the MOH referral form. Newborn Care Develop home based newborn care messages and maternal care messages if needed NC messages included in updated MOH VHT training materials for iCCM Identify local materials and tools NC messages included in updated MOH VHT training materials for iCCM Train VHTs on MNC NC included in 5 day ICCM training and maternal care discussed during quarterly meetings. VHTs provide community education with support from mobilizers 560 VHTs providing some community education and home visits Conduct MNC refresher training at HF level 46 health facility staff provided with a 3 day training in MNC by MTI staff. Supportive Supervision of perinatal services Supportive supervision provided by CSP MNC coordinator monthly EPI Train VHTs on EPI EPI included in 5 day ICCM training and reinforced during quarterly meetings. VHTs provide community education with support from mobilizers 560 VHTs providing some community education and home visits Support Child Health Days / national immunization days. CSP supports Child Health Days twice a year by mobilizing communities to support immunization sessions, providing education on importance of attending outreach services, and providing health staff with assistance with transportation and printing immunization cards Early Childhood Development Hands to Hearts hires local trainers 2 short term trainers were hired and provided training for MTI staff Training of Trainers in ECD 19 health facility staff trained as trainers in ECD ECD Trainers train VHTs in ECD in intervention areas 300 VHTs and Peer Educators trained in ECD in intervention areas ECD Trainers train VHTs in ECD in control areas Planned for second half of project. This is being re￾programmed only for Aromo Sub-County. VHT Trainers train community members in intervention areas 300 VHTs and Peer Educators provided ECD training for parents in intervention area VHT Trainers train community members in control areas Planned for second half of project. Now programmed for control area of Aromo sub-county. 2 Follow up & Refresher TOT; gather lessons learned Lessons learned gathered during MTE. Refresher training will be provided for health staff VHTs in control area in Aromo sub-county. Conduct ECD Review Meeting ECD review meeting was conducted during MTE Conduct ECD Forum ECD forum planned for Year 4 as part of final evaluation. Activity may be dropped. VHT Support Collect information on VHT membership and status Completed Provide joint supportive supervision Supportive supervision provided quarterly Support health facility staff to arrange and facilitate coordination meetings with VHTs Coordination meetings with VHTs occur quarterly Health Unit Management Committee members attend VHT – health facility coordination meetings on a quarterly basis Members of HUMC attend VHT quarterly coordination meetings M&E Condult baseline and final KPC Surveys Baseline 30 cluster KPCs conducted. Final 30 cluster KPC planned for final quarter of project. Solicit community feedback of KPC results Took place following baseline survey and Year 1 LQAS KPC through focus group discussions with various stakeholders and beneficiaries Conduct focus groups discussions, and doer/ non-doer surveys Focus group discussions and doer/non-doer surveys conducted during DIP development process and midterm evaluation. Doer/non-doer surveys planned for Year 3 to assess breastfeeding and IYCF practices. LQAS KPC Surveys LQAS KPC survey conducted at end of Year 1 and at midterm. Conduct R-HFA R-HFA conducted at baseline and midterm. Midterm and Final Evaluations Midterm evaluation conducted in February 2012 Objective Objective Met Activity Status Project Objective 1: Communities assume responsibility for their own health through strengthened community capacity (Village Health Teams, Parish Development Councils, and Health Sub-districts). Not Yet Trained and provided support for 560 VHTs in ICCM and other short topics. VHTs providing community education, home visits and referrals. Trained and supported 300 VHTs and Peer educators to promote early child development practices Facilitate collaboration between VHTs and PDCs and HUMCs to benefit MCH. This takes place primarily through HUMCs, as some HUMC members are also members of PDCs. Project Objective 2: Improved health (Community-based Integrated Management of Childhood Illness) and child care (Early Childhood Development) behaviors among mothers of children under five years of age. Not Yet 560 Village Health Team volunteers promote social and behavior change through women’s groups, church groups, community meetings and home visits. 300 Mother Leaders have started sharing health information with their peers. Project Objective 3: Improved quality of health facility services through strengthened Integrated Management of Childhood Illness and Maternal and Newborn Care capacity. Not Yet 34 health facility staff trained in IMCI and provided with follow up on-the-job mentoring and supportive supervision Project Objective 4: Strengthened institutional capacity of MTI and the Lira District Health Office to implement effective and efficient child survival activities. Not Yet Organizational Capacity Assessment (OCA) was conducted with MTI Uganda as part of the DIP preparation process. Summary of progress made is included in Annex 7. MTI/DHO engage in joint planning, training, implementation and evaluation. Annex 4a: Scope of Work – Midterm Evaluation Medical Teams International Lira Child Survival Project Lira, Uganda The Lira Child Survival Project in Uganda is a four-year project covering three sub-counties of Lira District (Lira, Ogur and Aromo). The key implementing partner is the Lira District Health Office which is charged with the responsibility of coordinating the delivery of health services throughout the district as required by the Ministry of Health. Hands to Hearts International (HHI), an international NGO based in Portland, Oregon, is a consulting partner for the early childhood development component of the project and has provided early childhood development training of trainers for MTI and health facility staff and Village Health Teams (VHTs). The northern region of Uganda, the poorest part of the country (DHS 2006 found 58% of the population in the lowest wealth quintile), has traditionally been marginalized from development benefits and suffered the effects of roughly 20 years of conflict and displacement brought about by the atrocities of the Lord’s Resistance Army. The overall goal of the project is to reduce child morbidity and mortality in Uganda, in support of Uganda Ministry of Health goals, objectives and strategies. The strategic objective is: sustainable improvements in preventive maternal and child health behaviors and use of strengthened health services in Erute North Sub-District of Lira District. The project has four key results/objectives: Objective No.1: Communities assume responsibility for their own health through strengthened community capacity (Village Health Teams, Parish Development Councils, and Health Sub-districts). Objective No.2: Improved health (Community-based Integrated Management of Childhood Illness) and child care (Early Childhood Development) behaviors among mothers of children under five years of age. Objective No.3: Improved quality of health facility services through strengthened Integrated Management of Childhood Illness and Maternal and Newborn Care capacity. Objective No.4: Strengthened institutional capacity of MTI and the Lira District Health Office to implement effective and efficient child survival activities. MTI is utilizing four primary strategies in order to reach the strategic objective of the project: 1. Targeted behavior change at the household level. Key family practices, and subsequently health status, will be improved at the household level by building the capacity of Village Health Teams in Community-based Integrated Management of Childhood Illness (C-IMCI) using a social and behavior change approach. 2. Community mobilization through capacity building of community organizations and leaders including planning and evaluation, and activities for sustainable positive health outcomes. 3. Quality of care and access at the clinic level by implementing IMCI through training, mentoring, supportive supervision for health facility staff and strengthening of referral and logistics systems. 4. Institutional capacity building for MTI and partners focusing on improved program quality and technical skills, strengthened project monitoring and evaluation, and institutionalization and dissemination of lessons learned. 2 The project devotes 10% level of effort to strengthen Immunization, 20% for Control of Diarrheal Disease, 25% for Pneumonia Case Management, 20% for Infant and Young Child Feeding, and 25% for Maternal and Newborn Care. The project also includes an innovation for Early Childhood Development activities that will contribute to increased attention placed by mothers on infant and young child development, child health status, and active feeding practices. Evaluation Audience: The audience for the findings from the final evaluation includes the Lira District Health Office, health unit management committees at all four health facilities in the project area, health facility staff, VHTs, project beneficiaries, the USAID CSHGP and Uganda Mission, and Medical Teams International HQ and field staff. Objectives of the Evaluation: 1. Assess progress in implementing the early phases of the DIP including preparation and structuring tasks (i.e. capacity building); 2. Assess progress towards achievement of objectives or yearly benchmarks; 3. Assess if interventions are sufficient to reach desired outcomes; 4. Assess progress in implementing the integrated early childhood development component; 5. Identify barriers/challenges to achievement of objectives; 6. Identify key factors that contributed to what seems to be working well or to what is not working well regarding some or all aspects of the program to inform project actions; and 7. Provide recommended actions to guide the project staff through the last half of the project. Methodology Medical Teams International will conduct a participatory evaluation involving representatives from Lira District Health Office, health unit management committees, health facility staff, VHTs, and project beneficiaries. Overall leadership will be provided by an external evaluator and an early childhood development consultant. Data will be gathered using a variety of qualitative and quantitative tools. 1. Quantitative data collection: The project will conduct a Health Facility Assessment during December, 2011 and KPC survey during January, 2012. Data will be available by February 27. 2. Qualitative data collection: The External Evaluator and Early Childhood Development Consultant will work with MTI HQ CS Advisor to define the qualitative information that needs to be collected and develop a plan for gathering this information. Qualitative data will be collected through interviews and/or focus groups with representatives from village health teams, health facility staff and project beneficiaries. 3. Data analysis: Quantitative and qualitative findings will be analyzed jointly with community members, project staff and DHO staff so that all stakeholders are involved in learning from the findings and understand the project’s impact. Facilitation of these workshops will be shared by the external evaluator and project staff. 4. Report writing: An evaluation report, following the USAID guidelines for Midterm Evaluations of USAID Child Survival grants, will be written by External Evaluator with input from the Early Childhood Development Consultant and MTI staff. The report will be considered final once it has been thoroughly reviewed and approved by MTI Uganda and headquarters staff. 3 Annex A: Report Writing Responsibilities Report Section Responsible A. Executive Summary External Evaluator B. Overview of the Project Structure and Implementation External Evaluator C. Evaluation Methodology and Limitations External Evaluator D.Data Quality and Use External Evaluator E. Presentation of Progress toward the Achievement of Project Results External Evaluator and Early Childhood Development Consultant F. Discussion of the Progress toward Achieving Results External Evaluator G.Discussion of Potential for Sustained Outcomes, Contributing to Scale, Equity, Community Health Worker Models, and Global Learning External Evaluator (Scale, Community Health Worker Models, and Global Learning) Africa Health Advisor (Equity) H.Conclusions and Recommendations External Evaluator and Early Childhood Development Consultant I. Action Plan for Responding to Evaluator Recommendations Africa Health Advisor/ Project Manager J. Change in Grantee Organization Capacity Africa Deputy Regional Director with support from Africa Regional Director and HQ Child Survival Advisor Annexes Responsible 1. List of Publications and Presentations Related to the Project 2. Results Highlight External Evaluator/ HQ Child Survival Advisor 3. Project Management Evaluation External Evaluator 4. Work Plan Table Project Manager 5. Community Education Plan Africa Health Advisor and Project Manager 6. Sustainability Plan HQ Child Survival Advisor 7. Rapid Catch Table HQ Child Survival Advisor 8. Midterm KPC Report MTI HQ M and E Advisor 9. CHW Training Matrix Africa Health Advisor 10. Evaluation Team Members and Titles Africa Health Advisor 4 11. Evaluation Assessment Methodology External Evaluator 12. List of Persons Interviewed and Contacted External Evaluator 13. Project Data Form HQ Child Survival Advisor Annex 4b: Evaluation Assessment Methodology Mid-term evaluation methodology The mid-term evaluation consisted of a combination of quantitative and qualitative methods. In December 2011, the MTI Senior Advisor in M&E assisted with conducting a Rapid-Health Facility Assessment and with preparations for the Knowledge Practices and Coverage (KPC) survey. The KPC survey using Lot Quality Assurance Sampling (LQAS) was conducted in January 2012. Detailed reports of both surveys are attached as Annexes 6 and 6A to this report and the results were used to plan the qualitative assessments and formulate the recommendations and conclusions. The participatory qualitative evaluation took part in two phases. During the first phase, the CSP team worked with the ECD consultant to conduct interviews and focus groups in the ECD intervention areas and to analyze the findings in a half-day workshop. Key results are included in this narrative and a more detailed report is found in Annex 2. The list of people interviewed either individually or in focus groups, along with locations, is found in Annex 10. The second phase of qualitative evaluation, led by the external evaluation consultant, focused on the standard child survival interventions across all project objectives. Three teams comprised of all project staff, MTI HQ and regional technical representatives and the consultan conducted focus groups with health workers, community volunteers and target population in all three sub￾counties. The consultant conducted informal individual interviews with project staff and with the MTI technical staff. Lists of all people interviewed are included in Annex 10 along with names of communities and health centers visited. Qualitative data collection followed interview guides that were designed to triangulate results of the R-HFA and KPC as well as to assess the effectiveness of project processes and ascertain the major challenges and constraints. Instruments are included in Annex 9. The external consultant read the project proposal, the final version of the Detailed Implementation Plan, the Baseline KPC report, the Behavior Change Strategy, the Sustainability Plan, and the First Annual Report to USAID. She also reviewed the findings of the first year LQAS survey, and both the mid-term KPC and R-HFA. Annex 4c1: Focus Group Discussion Guides for Early Childhood Development Focus Group Discussion For MTI / HHI Child Survival Project, Lira, Uganda Mid-term evaluation Feb. 2012 Survey team members: ________________________________________________ Location: __________________________________ Target Group - Mothers, Fathers or other primary caregivers who received formal ECD lessons Objectives for Focus Group Survey 1. Learn about care practices in given community 2. Identify what ECD messages caregivers learned and if/how this has changed attitudes or behaviors 3. Discover if new attitudes or behaviors are leading to new outcomes, particularly changes in health, for caregiver, child, or family 4. Identify if there are attitudes or behaviors that are unhealthy, but remain the same after ECD lessons. Discover why these persist. Focus Group Mothers, Fathers or other caregivers  Role - Primary caregiver is the person who is mostly responsible for a child’s day to day care – feeding, bathing, putting to sleep, etc…. Secondary caregiver is someone who helps, such as a supportive family member, neighbor or health worker, but is not responsible for day to day care of the child.  Title - Note of title of caregiver, such as: mother, father, aunt, grandmother, older sibling… Gender Role & Title Caregiver Number of Children in Household Age of Youngest Child Marital Status Attended formal ECD training? 1. 2. 2 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13. 14. 15. 16. 17. 18. 19. 20. 3 Explain the meaning of “ECD” – ECD stands for early childhood development, today we are referring to it as a set of caregiver actions and behaviors that support a child’s first years of development, including feeding, clothing, helping the child learn to speak and learn about the world, seeking health care, providing love, among many other actions. Questions for Focus Groups 1. Did you participate in formal ECD workshop(s) in your village? If yes, which ones? # yes ECD Lessons ________ Importance of the role of parents – a basic overview of ECD ________ Physical Development ________ Cognitive Development ________ Language Development ________ Social and Emotional Development ________ Baby Cues ________ Baby Massage ________ Basic Health, Hygiene & Nutrition 2. Additionally, did you hear messages on these subjects from another source? If yes, what was the source? (neighbour, nurse at health clinic, other…) # yes Source of info. ECD Lessons ________ __________________ Importance of the role of parents – a basic . overview of ECD ________ __________________ Physical Development ________ __________________ Cognitive Development ________ __________________ Language Development ________ __________________ Social and Emotional Development ________ __________________ Baby Cues ________ __________________ Baby Massage ________ __________________ Basic Health, Hygiene & Nutrition 3. What was the most useful thing you learned from the ECD lessons or messages? 4 4. Is there a difference in how you spend time with your child now? If yes, explain/describe how and why 5. Are the children responding differently to you now? If yes, explain/describe how and why . 6. Have you implemented any changes in hygiene, nutrition or sanitation? If yes, explain/describe how and why. 7. Have others noticed changes in your relationship with your child, your actions, confidence, or in other ways? If yes, explain/describe how and why. 8. Have you noticed any changes in your child’s health, growth or development? If yes, what changes? 9. How do you know if your child is developing well? 5 10. Would you like to learn other things about how children grow and develop? 11. Is there anything else you wish to share? Do you do any of these things with your child? Yes or no? 1. Talk with the child 2. Respond when child is asking 3. Read with the child 4. Tell stories to the child 5. Say prayers to or with the child 6. Use other techniques to help child talk If you do any of these things, why do you do them? How does a child learn to talk? How could you help them? Linguistic Record “Yes” with the # of times/week, or “No” 1. 2. 3. 4. 5. 6. Do you do any of the following while feeding ________ (name of the child)? (Ask only to 0-1 yr olds) -: 1. Look into the eyes of child 2. Talk to or sing to child 3. Show affection in some other way (specify how) 4. Hold her/him close to my body Cognitive Read answers, record # of “Yes”, # of “No” 1. 2. 3. 4. 6 If you do these things, why? *Do babies communicate before they can speak? If yes, how do they communicate? Linguistic Does your child show affection or love towards you? If yes, describe how. Social/emotional What are the ways you show affection or love towards your child? Do you think this is important for the child? If yes, why? Social/emotional 7 Focus Group Discussion For MTI / HHI Child Survival Project, Lira, Uganda Mid-term evaluation Feb. 2012 Survey team members: ______________________________________________ Location: __________________________________ Target Group - Mothers, Fathers or other primary caregivers who did NOT receive ECD lessons or messages Objectives for Focus Group Survey 5. Learn about care practices in given community 6. Encourage participation in HHI training 7. Encourage interest and empowerment in ECD among caregivers Focus Group Mothers, Fathers or other caregivers  Role - Primary caregiver is the person who is mostly responsible for a child’s day to day care – feeding, bathing, putting to sleep, etc…. Secondary caregiver is someone who helps, such as a supportive family member, neighbor or health worker, but is not responsible for day to day care of the child.  Title - Note of title of caregiver, such as: mother, father, aunt, grandmother, older sibling… Gender Role & Title Caregiver Number of Children in Household Age of Youngest Child Marital Status 21. 22. 23. 24. 25. 8 26. 27. 28. 29. 30. 31. 32. 33. 34. 35. 36. 37. 38. 39. 40. 9 Questions for Focus Groups 12. How would you describe a good parent? What kinds of things do good parents do with their young children? Why do they do these things? 13. What kinds of things does a bad parent do? Why are these behaviors bad? 14. How would you describe a “good baby”? Why do some babies behave the ways you describe? 15. How would you describe a “bad baby”? Why do some babies behave the ways you describe? 16. How do you show love and affection to your babies and young children? 17. Do mothers show this love and affection differently than fathers? 18. What resources exist in the community to support you in caring for your young child? 10 Cognitive 1. Do babies think before age one? # Yes ______ # No______ 2. Some children seem to be smarter (use local term) than others. Do you think this is determined when they are born, or are there things that you can do to make a child smarter? If so, what are these things? 3. How would you know if your child was developing too slowly? If your child was developing too slowly, what would you do? Linguistic 1. Do babies communicate before they can speak? # Yes ______ # No______ If yes, how do they communicate? 2. How does your baby learn to speak? 3. Do you sing to your children, or tell proverbs, stories or nursery rhymes to your children? Why? 11 Cognitive/Motor/Linguistic 1. Is it important for children to play? # Yes ______ # No______ If yes, why is it important? 2. How do you play with your child? Social/Emotional 1. Do babies have emotions? # Yes ______ # No______ 2. How can you tell if your baby is upset? 3. How do you soothe your baby? 12 Focus Group Discussion For MTI / HHI Child Survival Project, Lira, Uganda Mid-term evaluation Feb. 2012 Survey team members: ________________________________________________ Location: __________________________________ Target Group – professionals, community leaders, VHTs and Peer Educators from intervention areas Objectives for Focus Group Survey 8. Learn about child care practices in given community 9. Learn about their community’s response to ECD lessons 10. Identify if they have observed changes in caregiving practices 11. Identify any new attitudes or behaviors within communities 12. Learn if there are new outcomes, particularly changes in health, for caregiver, child, or family 13. Discover any other unintended outcomes of the ECD lessons Title Gender Children in Home? 1. 2. 3. 4. 5. 6. 7. 8. 13 Title Gender Children in Home? 9. 10. 11. 12. 13. 14. 15. Questions: 1. How many caregivers in your community participated in the formal 5 week set of ECD lessons? <30 30-40 41-60 61-80 81-100 >100 #____ #____ #____ #____ #____ #____ How many additional caregivers have you given ECD messages or lessons to? <30 30-40 41-60 61-80 81-100 >100 #____ #____ #____ #____ #____ #____ 2. Did you notice changes in caregiver’s knowledge, attitudes or behaviors during the five-weeks of ECD lessons? If yes, please describe some examples. 14 3. Have you noticed changes in caregiver’s attitudes or behaviors that have persisted over the year since the ECD trainings? If yes, please describe some examples. 4. Are there attitudes or practices that have not changed? If so, why do you think this is? 5. Have you personally made changes in your attitudes or behaviors because of your involvement with the ECD lessons? If yes, please describe. 6. How would you rate the level of knowledge about ECD among caregivers who participated in the ECD lessons?  Not very knowledgeable _____(# who agree with this)  Somewhat knowledgeable _____(# who agree with this)  Extremely knowledgeable _____(# who agree with this) 7. Have you noticed changes in the health of young children in your community in the last year? If you noted changes, please describe them and what you attribute these to. 8. In regards to seeking health care when they notice IMCI danger signs, are caregivers now: 15  more likely _____(# who agree with this)  less likely _____(# who agree with this)  or have not changed _____(# who agree with this) in their practices of seeking health care for their child? 9. If they are now “more likely” or “less likely” to seek health care for their child, what do you attribute the changes to? 10. What risk factors/dangers/challenges still exist for the young children of your community? 11. What are the general needs as defined and felt by the community, in regards to supporting healthy early childhood development? 12. What kinds of support would the community like to have for caring for their youngest children? 13. Is there anything else you would like to share? 16 Focus Group Discussion For MTI / HHI Child Survival Project, Lira, Uganda Mid-term evaluation Feb. 2012 Survey team members: ________________________________________________ Location: __________________________________ Target Group – health clinic workers who attended ECD TOT trainings and give ECD messages to patients Objectives for Focus Group Survey 14. Learn about their use of ECD messages at their health centers 15. Learn about their community’s response to ECD lessons 16. Identify if they have observed changes in caregiving practices 17. Identify any new attitudes or behaviors they have observed 18. Learn if there are new outcomes, particularly changes in health, for caregiver, child, or family 19. Discover any other unintended outcomes of the ECD lessons Title Gender Children in Home? 1. 2. 3. 4. 5. 17 Questions: 14. Did you give ECD messages to caregivers when they visited the clinic or during outreach? _____ # yes _____# no a) If yes, what message(s) did you share and why? b) How did you share them? _____ # During outreach _____ # During consultations _____ # During education sessions in the waiting area 15. How many caregivers did you give ECD messages to? <30 30-40 41-60 61-80 81-100 >100 #____ #____ #____ #____ #____ #____ Please estimate what percentage of patients that are caregivers did you give ECD messages to? _____0%, None _____25% _____33% _____50% _____75% _____100% 16. Did you notice changes in caregiver’s knowledge, attitudes or behaviors that you attribute to the ECD messages? If yes, please describe some examples. 18 17. Have you noticed changes in caregiver’s attitudes or behaviors that have persisted over the year since the ECD trainings? If yes, please describe some examples. 18. Are there attitudes or practices that have not changed? If so, why are do you think this is? 19. Have you personally made changes in your attitudes or behaviors because of your involvement with the ECD lessons? If yes, please describe. 20. Have you noticed changes in the health or development of young children in your Health Center in the last year? If you noted changes, please describe them and what you attribute these to. 21. In regards to seeking health care when they notice IMCI danger signs, are caregivers now: 19  more likely _____(# who agree with this)  less likely _____(# who agree with this)  or have not changed _____(# who agree with this) in their practices of seeking health care for their child? 22. If they are now “more likely” or “less likely” to seek health care for their child, what do you attribute the changes to? 23. What risk factors/dangers/challenges still exist for the young children of your Health Center? 24. Do you think the relationships between health workers and caregivers has changed because of the ECD messages? If yes, please describe some examples. 25. Is there anything else you would like to share? MTI Uganda Child Survival Midterm Evaluation 2012 Annex 4c2:   Focus Group Discussion Guides to Assess Technical Components of the Project   Guide for Interviews with Beneficiaries (Mothers and fathers with children younger than 5 years) Date: __________       Interviewer: _____________________________________________ Community: ______________________ Sub‐County: ______________________________ Number of Women/Men:  W____/ M___     1. What activities have you participated in with the VHTs? Receive home visits (probe for reasons and note) Attended community education sessions by VHT (probe for topics and note) Referrals to health facility by VHT (probe for reasons and note examples) Any other contact with VHT (note examples of other contacts) 2. Would you say the work of the VHTs is very important to your community, somewhat important or not important?  Why? # Who indicated “Very important” and why. _______________________________________ # Who indicated “Somewhat important” and why.__________________________________ # Who indicated “Not important” and why.________________________________________ 3. Does this community support the work of the VHTs?  How? ___________________________________________________________________________ ___________________________________________________________________________ 4. Does anyone have an example of how the VHT helped this community? ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ MTI Uganda Child Survival Midterm Evaluation 2012 2 5. How do you like to learn new information about keeping yourself and your family healthy?   ___________________________________________________________________________ ___________________________________________________________________________ 6. What messages have you heard from the VHTs or someone else in the community about how to have a healthy pregnancy?  Why are these practices important? Increase food consumption   Take iron tablets     Go to health center for ante‐natal care Get TT vaccine   Get more rest and decrease workload Importance of post‐natal care Other   7. What new information about breastfeeding have you learned from VHTs or someone else in the community? Benefits of breastfeeding. Why?     Immediate breastfeeding. Why is it important?   Exclusive breastfeeding. How long? Why? Frequency/on demand. How often? Continue breast feeding until at least 2 yrs. Other 8. What new information about feeding children have you learned from the VHT or someone in the community? Introduce solid foods at six months Frequency of feeding Consistency (thickness) of food Quantity   Variety (food groups)   Enriched local recipe Sources of Vitamin A Use separate plate MTI Uganda Child Survival Midterm Evaluation 2012 3 Active /Responsive feeding Other   9. If your child has diarrhea what do you do? Go to the health center. What signs do you look for to know you need to go the health center? Continue breast feeding, and maintain or increase feeding. Treat with ORS/ increase fluids. Other 10. What have you learned about cough and difficulty breathing in small children? Prevention   Early recognition of  danger signs   Appropriate careseeking Other 11. Over the past six years what health practices are different now (improved) in the community? Why?    ___________________________________________________________________________ ___________________________________________________________________________ 12. Of all the health topics you have learned, was there one that was so interesting you immediately went and told someone else?  Which topic? ___________________________________________________________________________ ___________________________________________________________________________ 13. Has a VHT ever shown you a picture with a health message? Do you remember any of them?  What do you remember about them? ___________________________________________________________________________ ___________________________________________________________________________ 14. What health and nutrition advice do you think some mothers find difficult to follow?   Why? Message Why?       MTI Uganda Child Survival Midterm Evaluation 2012 4 15. Can you tell me three things that you could do to help your baby’s cognitive development? ____________________________________________________________ ________________________________________________________________________ ________________________________________________________________________ 16. Has anyone taken his/her child to the nearest Health Center recently?  Where did s/he take his/her child?  What was wrong with the child?  What was your experience like there?___________________________________________________________________ ________________________________________________________________________ ________________________________________________________________________ 17. Have there been any recent improvements in the quality of health care at the Health Centers?  If yes, what were the improvements?  If no, why not? ___________________________________________________________________________ ___________________________________________________________________________ 18. Would you like to learn more about health and the care of your children? What specifically would you like to learn? How would you like to learn this?    ___________________________________________________________________________ ___________________________________________________________________________ 19. What will families do to continue the new health practices you mentioned that have been taken up in the last six years? (Refer back to question number 11.) ___________________________________________________________________________ ___________________________________________________________________________ MTI Uganda Child Survival Midterm Evaluation 2012 5 Guide for Interviews with Village Health Teams Date: ____________ Interviewer:_______________________________ Community: ___________________ Sub‐County:  _____________________ Number of VHTs:   Men ___ Women  ___ 1. Intro:  How long have you been VHTs, what do you like about being VHT? ___________________________________________________________________________ ___________________________________________________________________________ 2. In the last month, what activities have you done as a VHT in your community? Description of activities:_______________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ (If not mentioned probe about these activities) Community education sessions Home visits Referrals   3. What other NGOs or government programs do you work with? ________________________________________________________________________ ________________________________________________________________________ 4. How often do you receive training?  What topics did the trainings cover? Topics:                      _________ How often:                    _________ Who provided the training? _________________________________________________ 5. What was good (useful, appropriate) about the trainings or what could be improved? Good:      ___________               ___ Could be improved:                _______________ What is the most interesting thing/topic you have learned in the training?  ___________ ________________________________________________________________________ MTI Uganda Child Survival Midterm Evaluation 2012 6 6. What other health and nutrition trainings would you like to receive?  Why? ___________________________________________________________________________ 7. What support do you receive from community leaders? ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ 8. What additional support do you need from community leaders? ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ 9. a.   How often do you have contact with health facility staff? _____________________ b. What do you discuss when you meet with them? ___________________________ c. What support do they provide for your work?   _____________________________ d. What more would you like them to do to help you with your work as a VHT? ___________________________________________________________________________ ___________________________________________________________________________ Technical Information 1. What messages do you share with your community about nutrition for pregnant and breastfeeding mothers? Increase food consumption Take iron tablets   Go to health center for ante‐natal care Get TT vaccine Get more rest and decrease workload Importance of post‐natal care   Other   2. What messages do you share with your community about breastfeeding? Benefits of breast‐feeding     Immediate breast feeding     Exclusive breastfeeding   Frequency/on demand   Continue breast feeding until at least 2 yrs Other MTI Uganda Child Survival Midterm Evaluation 2012 7 3. What messages do you share with your community about appropriate complementary feeding? Introduce solid foods at six months Frequency of feeding Consistency (thickness) of food Quantity   Variety (food groups) Enriched local recipe Sources of Vitamin A Use separate plate Active /Responsive feeding Other 4. What messages do you share about diarrhea? Prevention by washing hands Identify and respond to danger signs Continue breast feeding, increase fluids and maintain or increase feeding   Other 5. What messages do you share about coughs and difficulty breathing?   Prevention   Early recognition of  danger signs   Appropriate careseeking   Other 6. What messages do you share about immunization? Pregnant women should receive TT vaccine   Children need to be fully vaccinated against 6 childhood diseases Other MTI Uganda Child Survival Midterm Evaluation 2012 8 7. How do promote these health messages? __________________________________________________________________________ ___________________________________________________________________________ 8. A.   Who do you reach during home visits? _____________________________________ B. Who do you reach during community education sessions? _____________________________________________________________________ 9. Do you speak about one topic more than the others?  If so, why? __________________________________________________________________________ ___________________________________________________________________________ 10. When you are advising people to _____________, what prevents some people from doing this behavior? How do you help families overcome this difficulty to practicing the advice you provide? ____________________________________________________ ________________________________________________________________________    11. Which recommended health and nutrition practices do you find the most difficult to practice in your home?    ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ Opinions 1. What motivates you to work as a VHT? ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ 2. A. What do you think your community thinks about your work as a VHT?____________ ________________________________________________________________________   B. Do they support you?  __________________________________________________ C. Do they follow your advice?  Why or why not? ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ 3. A. What do traditional healers and TBA's think about your work? __________________ B. Do they support your work?  _____________________________________________ C. Do they follow your advice? Why or why not? ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ MTI Uganda Child Survival Midterm Evaluation 2012 9 4. A. What support do you receive through visits from MTI staff?_________________ _____________________________________________________________________    B. From the health facility in‐charge or health assistants? _____________________ _____________________________________________________________________   C. What is done during their visit? _______________________________________ _____________________________________________________________________ Frequency What is done MTI staff    Health facility staff 5. What is your opinion about the relationship of VHTs with the health facilities? ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ 6. What suggestions do you have to improve this project? ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ 7. Do you have any success stories about improving the health care or nutrition of children in your community?   ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ MTI Uganda Child Survival Midterm Evaluation 2012 10 Guide for Interviews with Mothers Group Members Date: __________       Interviewer: _____________________________________________ Community: ______________________ Sub‐County: ______________________________ Number of Women Interviewed: ________________   1. When did you become a member of the Mothers Group?   2. What is the purpose of the mothers group?  What do you do? 3. What kind of health training did you receive as a member of the Mothers Group? 4. Who provided the training you received in the Mothers Group? 5. How do you use the knowledge you received from the training? 6. Which topics are most interesting to you? 7. Do you share these health messages? With whom? MTI Uganda Child Survival Midterm Evaluation 2012 11 8. How many other women have you talked to about health messages?  How many men? _____________________________________________________________________ _____________________________________________________________________ 9. Would you like to learn about other topics?  What other topics would you like to know more about? _____________________________________________________________________ _____________________________________________________________________ 10. If you learned about other topics, would you be willing to take those topics to ten houses in your community every other week? _____________________________________________________________________ _____________________________________________________________________ 11. Are you having contact with MTI CSP staff? How often does this happen? In what way? _____________________________________________________________________ _____________________________________________________________________ 12. Do you ever work alongside and coordinate with VHTs? How do you do this? 13. Do you ever work directly with a health facility?  How do you do this?   MTI Uganda Child Survival Midterm Evaluation 2012 12 Guide for Interviews with Health Unit Management Committees Health Facility:         Sub‐County:          Date: __________ Interviewer:               Number of members interviewed and their positions:                            1. Could you please explain the role of the HUMC? ___________________________________________________________________________ ___________________________________________________________________________ 2. What have you accomplished as the HUMC? ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ 3. What would the HUMC like to accomplish in the next two years? ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ 4. Are there positive changes in the services at this health center due to the project activities? ___________________________________________________________________________ ___________________________________________________________________________ 5. How will improvements at your health facility continue once MTI CSP ends? ___________________________________________________________________________ ___________________________________________________________________________ 6. What is your opinion of the work done by the VHTs in their communities? ___________________________________________________________________________ ___________________________________________________________________________ 7. Do you support the VHTs in their work? (Yes or No) ______________________________ If yes, how do you support them? ____________________________________________ If no, how could you support them? __________________________________________ 8. Have you seen any positive changes in health and nutrition practices by families that you think are due to the work of the VHTs? ___________________________________________________________________________ ___________________________________________________________________________ MTI Uganda Child Survival Midterm Evaluation 2012 13 9. What are the greatest challenges and successes of the health facility?    Challenge: __________________________________________________________________ Success: ____________________________________________________________________ 10. What is the greatest challenge and greatest success of the HUMC?    Challenge: __________________________________________________________________ Success: ____________________________________________________________________ 11. What do you think MTI CSP should do in the next year and a half? ___________________________________________________________________________ ___________________________________________________________________________ MTI Uganda Child Survival Midterm Evaluation 2012 14 Guide for Interviews with Health Unit Staff Health Facility:         Sub‐County:          Date: __________ Interviewer:               Staff Name:            Position:             Male (#) ____Female (#)_____Time in position at this health facility:       12. What has been your greatest success at the health facility this last year?    ___________________________________________________________________________ 13. Pease tell us about the training you received through MTI CSP or any other source.   What was good about the training and what could be improved? Type of Training Date of Training Training Leader Strengths of Training Suggested Areas for Improvement                  14. Are there positive changes in your work at this health unit due to MTI CSP project activities? ___________________________________________________________________________ ___________________________________________________________________________ 15. What have been your greatest challenges at the health facility this last year? ___________________________________________________________________________ 16. Do you have the equipment and supplies you need to provide quality care to women and children?    Equipment Supplies Available?  Y/N Functioning?  Y/N Clinical thermometers     Blood pressure machines     Curtains for privacy     Benches/chairs in waiting area     Benches/chairs in examination rooms     Child health cards     Children’s vitamins Lactation vitamins IMCI wall charts/guides MTI Uganda Child Survival Midterm Evaluation 2012 15 Are there other items not listed above that are needed at your facility? 17. What is your opinion of the work done by the VHTs in their communities? ___________________________________________________________________________ ___________________________________________________________________________ 18. Have you seen any positive changes in health and nutrition practices by families that you think are due to the work of the VHTs? ___________________________________________________________________________ ___________________________________________________________________________ 19. Do VHTs refer patients to your health unit? Are these referrals appropriate?  Can you describe a specific instance? ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ 20. Has your health facility received a supportive supervision visit during the past three months?  If yes, by whom? ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ 21. Could you describe the role of the HUMC in supporting your facility?  Can you give specific instances of how they have or have not supported you? ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ 22. What do you think MTI CSP should do in the next year and a half? ___________________________________________________________________________ ___________________________________________________________________________ 23. How will improvements at your health facility continue once the project ends? ___________________________________________________________________________ ___________________________________________________________________________ MTI Uganda Child Survival Midterm Evaluation 2012 16 Health Facility Supplementary Form Health Facility:         Sub‐County:          Date: __________ Interviewer:               13. Please review the IMCI ledger and note the number and type of referrals in the last 3 months: Referral Number Name of most distant community Child diarrhea     Child respiratory infection     Child malaria     Child immunization     Child malnutrition     Women’s ante‐natal care     Tetanus toxoid vaccine Childbirth Post‐natal care Adult malaria Other 1 Annex 4d: Child Survival Midterm Evaluation Schedule Day 1. Wednesday, February 22. Location: Aromo Sub-county  Group 1: Aromo Health Center Focus Groups: 1. Health facility workers 2. VHTs 3. HUMC Team Members: Harriet, Tobias, Mary Helen, Joel Okello, Joel Odur  Group 2: Strong village- Obalo-dak, Otara Parish Focus Groups: 1. Men beneficiaries 2. Women beneficiaries Team Members: Susan, James, Lydia, Judiann  Group 3: Weak village- Angole A, Otara Parish Focus Groups: 1. Men beneficiaries 2. Women beneficiaries Team Members: Nickson, Christine, Doreen, Anna Day 2. Thursday, February 23. Location: Ogur Sub-county  Group 1: Ogur Health Center Focus Groups: 1. Health facility workers 2. VHTs 3. HUMC Team Members: Nickson, Christine, Doreen, Anna, Joel Odu r  Group 2: Strong village- Akangi Parish, Akangi Village Focus Groups: 1. Men beneficiaries 2. Women beneficiaries Team Members: Harriet, Mary Helen, Tobias, Joel Okello  Group 3: Weak village- Gulwoo, Akangi Parish Focus Groups: 1. Men beneficiaries 2. Women beneficiaries Team Members: Susan, James, Lydia, Judiann 2 Day 3. Friday, February 24. Location: Lira Sub-county  Group 1: Location: Bar Apwo Health Facility Focus Groups: 1. Bar Apwo health facility workers 2. HUMC 3. VHTs Team Members: Susan, James, Lydia, Judiann  Group 2: Location: Bung A Village, Omito Parish (strong village) Focus Groups: 1. Women beneficiaries 2. Men beneficiaries 3. Mother’s Groups Team Members: Harriet, Tobias, Mary Helen, Joel Okello, Joel Odur  Group 3: Location: Bar Odyek Village, Omito Parish (weak village) and Amuca Health Facility Focus Groups: 1. Men beneficiaries 2. Women beneficiaries 3. Amuca health facility 4. HUMC Team Members: Nickson, Christine, Doreen, Anna 3 Annex 10: List of Persons Interviewed and Contacted during the MTE Child Survival Project Mid-term Evaluation FGD Participants s/n Sub County/HC Village Name of participant HEALTH WORKER FGD 01 Aromo HCIII _ Epilla Lillian 02 ,, _ Obok Patrick 03 Ogur HCIV _ Amodo Doris 04 ,, _ Akello Sophia 05 ,, _ Akullo Josephine 06 Barapwo HCIII _ Achola Hellen 07 ,, _ Harriet Atwongo 08 ‘’ _ Okello George 09 Amuca HCIII _ Acen Colline 10 ‘’ _ Amolo Lillian 11 ,, _ Atim Celia HEALTH UNIT MANAGEMENT COMMITTEE FGD 12 Aromo HCIII Barogin Okello Troy 13 ,, Akaidebe Odongo Robert 14 ,, Omolo Okor David 15 ,, Lelaodwe Ajok S Grace 16 ,, Onyapoyere Silvia Omara 17 ,, Akaidebe Obok Patrick 18 ,, Akaidebe Okello Julius Peter 19 ,, Akaidebe Epilla Lillian 20 Barapwo HCIII Aminyang Ebilla Jino 21 ,, Areco Dophia Atyeno 22 ,, Tedam Otto Click 23 ,, Anyalo Ogwal Moses 24 ,, Tedam Ochen James 25 ,, Tedam Adongo Agnes 26 Ogur HCIV Ogur Corner Opio Tonny 27 ,, Okwaloamara Adiro Judith 28 ,, Akano Akello Sophia 29 ,, Onywaloyale Grace Odongo 30 Amuca HCIII Okecoyere Odida Morris 31 ,, Okecoyere Opinya Moses 32 ,, Okecoyere Abwot Teddy 33 ,, Okecoyere Olung Tom 34 ,, Okecoyere Elizabeth Acheng 35 ,, Okecoyere Mary Oyena 36 ,, Okecoyere Omara Peter 37 ,, Okecoyere Omara Sam VHT FGD 38 Aromo S/ C Ayami Okot James 4 39 ,, Apua Ejang Christine 40 ,, Opokmidila Amule Santa 41 ,, Beo Lilly Angena 42 ,, Acholidumu Owot Moses 43 ,, Ataparadam Okidi Ambrose 44 ,, Alok Grace Owiny 45 ,, Ayitunga Adilo Fred 46 ,, Arwotolaro Apio Christine 47 ,, Adaganii Agonga Charles 48 ,, Ajiagood Juspanti Omara 49 ,, Tegweng Ongom Francis 50 ,, Aloc A Awor Molly 51 ,, Lelakworo Omara Jimmy Aaron 52 ,, Obalodak Ojok Jasper 53 ,, Aloc A Alele Filbert 54 Ogur S/C Acetlela Ameny Linous 55 ,, Timber Opio Mark 56 ,, Adilebi Ogwal Lawrence 57 ,, Awelo Atyang James 58 ,, Apuroimon Ojok Joel 59 ,, Diicunyi Akii Tobby 60 ,, Nginginya Komakech James 61 ,, Aulaimalo Anyima Felix 62 ,, Diicunyi Betty Okwir 63 ,, Nangabir Acen Rose 64 ,, Teakang Betty Owiny 65 ,, Atongimoco Mirriam Otim 66 ,, Onywaloyere Grace Odongo 67 ,, Agak Ketty Omara 68 ,, Bungmiciri Semmy Ekaro 69 ,, Baralegi Christine Okello 70 Lira S/C Areco Otim Sam 71 ,, Telela A Anena Sam 72 ,, Ongica B Omondi Jimmy 73 ,, Tedam Otiti Charles 74 ,, Anyalo Tom Parakak 75 ,, Akolodong Atim Julius 76 ,, Adelokok C P Obong 77 ,, Teokole Opio Morris 78 ,, Bung B Anyonga Vincent 79 ,, Bung A Agang John 80 ,, Ayere Esther Oming 81 ,, Tedam Mido Aguma 82 ,, Areco Hellen Ogweng 83 ,, Adelokok Phoebe Anyii 84 ,, Awita Apita Janet 5 85 ,, Telela A Atala Susan 86 ,, Telela B Akello Lucy 87 ,, Atodi Enin Epila MOTHER’S GROUP FGD 88 Aromo S/C Aloc A Keller Ongom 89 ,, Aloc A Alobo Jane 90 ,, Wigweng Atala Rose 91 ,, Aloc B Awino Grace 92 Ogur S/C Auleipiny Jennifer Anyima 93 ,, Akadocwat Amongi Jennifer 94 ,, Alenga Adong Ketty 95 Lira S/C Telela B Janet Abeja 96 ,, Telela A Harriet Obong 97 ,, Telela Betty Okello 98 ,, Adelokok Florence Oming 99 ,, Bung Dorcus Ogwang FATHER’S FGD 100 Aromo Angole B Ayo Jimmy 101 ,, ,, Ocen Bonny 102 ,, ,, Olot Samuel 103 ,, ,, Olal Geoffrey 104 ,, ,, Odong Bonny 105 ,, ,, Opito Bonny 106 ,, ,, Bodo Denish 107 ,, ,, Okite Denish 108 ,, Obalodak Okello Santo 109 ,, ,, Okello Moses D 110 ,, ,, Olobo Lujino 111 ,, ,, Okello Tonny 112 ,, ,, Okello Cosmas 113 ,, ,, Okello Richard 114 ,, ,, Olal Denis 115 ,, ,, Olaa Ambrose 116 Ogur Sub County Odokokome Omara G 117 ,, ,, Opio Bonny 118 ,, ,, Oroch Tonny 119 ,, ,, Lingo Emmanuel 120 ,, ,, Orwaa 121 ,, ,, Oroc H 122 ,, ,, Opio Bob 123 ,, ,, Omara Denish 124 ,, Gulwoo Ogweng Gerald 125 ,, ,, Ongom Patrick 126 ,, ,, Odyek Sam 127 ,, ,, Ogweng Bosco 6 128 ,, ,, Odongo Denish 129 ,, ,, Omara Micheal 130 ,, ,, Ocak Alfred 131 ,, ,, Okello David 132 Lira S/C Barodyek Nam Denish 133 ,, ,, Odongo Geoffrey 134 ,, ,, Obote Tom 135 ,, ,, Olet Denish 136 ,, ,, Aporo Geoffrey 137 ,, ,, Opio Tom 138 ,, ,, Ocen Peter 139 ,, ,, Odyek Bonny 140 Bung A ,, Omara Innocent 141 ,, ,, Anyanga Geoffrey 142 ,, ,, Ogwal Simon 143 ,, ,, Lee Denish 144 ,, ,, Ajal Seles 145 ,, ,, Oyom Richard 146 ,, ,, Ajal David 147 ,, ,, Obong Denish MOTHER’S FGD 148 Aromo S/C Angole B Eunice Akun 149 ,, ,, Adong Jenifer 150 ,, ,, Amoly Rose 151 ,, ,, Akello Juspine 152 ,, ,, Atim Anna 153 ,, ,, Aguti Anna 154 ,, ,, Akec Esther 155 ,, ,, Akullu Christine 156 ,, Obalodak Ajok Molly 157 ,, ,, Tabitha Akello 158 ,, ,, Abonyo Dorcus 159 ,, ,, Apio Jennifer 160 ,, ,, Acio Juspine 161 ,, ,, Akao Judith 162 ,, ,, Apica Anai 163 ,, ,, Ajok Brenda 164 ,, ,, Akao Sophia 165 ,, ,, Abonyo Grace 166 Ogur S/C Odokokome Caroline Olet 167 ,, ,, Jacinta Ogang 168 ,, ,, Sydonia Tia 169 ,, ,, Evaster Opio 170 ,, ,, Ejang Roseline 171 ,, ,, Acen Susan 172 ,, ,, Grace Ojok 7 173 ,, ,, Dilis Oroch 174 ,, Gulwoo Alek Scovia 175 ,, ,, Akiu Joan 176 ,, ,, Aceng Dillis 177 ,, ,, Akullu Jane 178 ,, ,, Akello Silvia 179 ,, ,, Auma Agnes 180 ,, ,, Ejang Betty 181 ,, ,, Hellen Okao 182 Lira S/C Bung A Betty Onweng 183 ,, ,, Susan Auma 184 ,, ,, Colline Okullo 185 ,, ,, Santa Opio 186 ,, ,, Evaline Ajal 187 ,, ,, Santa Okullu 188 ,, ,, Jacinta Ocen 189 ,, ,, Harriet 190 ,, Barodyek Hadoline Bua 191 ,, ,, Agnes Ojok 192 ,, ,, Janet Olet 193 ,, ,, Lillian Ogwang 194 ,, ,, Judith Okello 195 ,, ,, Christine Okello 196 ,, ,, Grace Odok 1 Annex 4e: List of Persons Interviewed and Contacted during the MTE A. Persons Contacted for the Evaluation of Early Childhood Development Component Lira Sub-County Village Comparison or Intervention Interview Type Names of participants Wigweng Comparison Fathers Otim Isaac Wigweng Comparison Fathers Olum Patrick Wigweng Comparison Fathers Okot Denis Wigweng Comparison Fathers Onyinge George Wigweng Comparison Fathers Okwir Emmanuel Wigweng Comparison Mothers Onyinge Bonny Wigweng Comparison Mothers Lillian Ego Wigweng Comparison Mothers Karoline Okello Wigweng Comparison Mothers Grace Awino Wigweng Comparison Mothers Grace Otim Anai Agali Comparison Fathers Ebonga Richard Anai Agali Comparison Fathers Ocip Tonny Anai Agali Comparison Fathers Olungo Moses Anai Agali Comparison Mother Colline Otim Anai Agali Comparison Mother Shilla Ocen Anai Agali Comparison Mother Harriet Otim Anai Agali Comparison Mother Aboke Eunice Anai Agali Comparison Fathers Obongo Godfrey Anai Agali Comparison Fathers Odongo Ivan Akolodong Intervention Fathers Adol William Akolodong Intervention Fathers Odongo Micheal Akolodong Intervention Fathers Opio Mark Akolodong Intervention Mothers Ajok Christine Akolodong Intervention Mothers Betty Okello Akolodong Intervention Mothers Lilly Opio Akolodong Intervention Mothers Anna Todi Akolodong Intervention Mothers Sarah Owac Akolodong Intervention Mothers Molly Lem Teokole Intervention Father Opio Patrick Teokole Intervention Mother Harriet Ogwang Teokole Intervention Mother Adong Jusphine Teokole Intervention Mother Adol Flore Teokole Intervention Mother Rema Okori Teokole Intervention Father Oloa Fred Teokole Intervention Father Ayo Selestio Teokole Intervention Father Akona Bosco Teokole Intervention Father Ogwang Patrick 2 Teokole Intervention Mother Santa Ongola Amuca Health Center Health Center Health Worker Amolo Lillian Amuca Health Center Health Center Health Worker Akullo Molly Barapwo Health Center Health Center Health Worker Akullu Caroline Barapwo Health Center Health Center Health Worker Achola Hellen Alolodong Intervention VHT/PE Atim Julius Olengobir Intervention VHT/PE Opio Patrick Olago Intervention VHT/PE Jannet Opio Areco Intervention VHT/PE Hellen Ogweng Awita Intervention VHTs/PEs Ongom Moses Teokole Intervention VHTs/PEs Opio Morris Olago Intervention VHTs/PEs Ogwal James Ongica Intervention VHTs/PEs Odwar Richard Areco Intervention VHTs/PEs Otim Sam Tedam Intervention VHTs/PEs Otiti Charles Akolodong Intervention VHTs/PEs Oucu richard Ayere Intervention VHTs/PEs Ogwok Alfred Tedam Intervention VHTs/PEs Mido Aguma Amin-nyanga Intervention VHTs/PEs Molly owani Awita Intervention VHTs/PEs Jannet Apita Ogur Sub-County Village Comparison or Intervention Interview Type Names of participants Okano idero Comparison Father Apita Bonny Okano Comparison Father Eles Obong Okano idero Comparison Father Apita Samuel Okano idero Comparison Mother Akello Agnes Okano idero Comparison Mother Mila Okeng Okano idero Comparison Father Opido Jimmy Okano idero Comparison Father Omara Augustine Karedonglac Intervention Father Ameny Linus Karedonglac Intervention Father Okello Tonny Karedonglac Intervention Father Ojok Geoffrey Karedonglac Intervention Father Adoli Quinto Karedonglac Intervention Father Odongo justine Karedonglac Intervention Mother Teddy Ojok Karedonglac Intervention Mother Evaline Ojok Karedonglac Intervention Mother Jenifer Okello Karedonglac Intervention Mother Jackline Agali Karedonglac Intervention Mother Polly Ocuna Te imat Comparison Fathers Ocen Tonny Te imat Comparison Mother Grace Ocen Te imat Comparison Mother Grace Olal Te imat Comparison Mother Silbina Oceng 3 Te imat Comparison Mother Helen Omara Te imat Comparison Fathers Olugu Alfred Te imat Comparison Mother Betty Odyek Te imat Comparison Fathers Oceng David Te imat Comparison Fathers Omara Alfred Te imat Comparison Fathers Omara Joel Ogur Health Center Health Center Health Worker Amodo Doris Ogur Health Center Health Center Health Worker Amono Josephine Ogeo B Comparison Fathers Okello Felix Orit Comparison VHT/PEs Opio Alfred Galilwite Intervention VHT/PEs Ojok Moris Gulwo Intervention VHT/PEs Awino Janet Bungmiciri Intervention VHT/PEs Susan Apio Nangabir Intervention VHT/PEs Apio Colline Oponi I Intervention VHT/PEs Apio Agness Apurugali Intervention VHT/PEs Akello Molly Acoro Intervention VHT/PEs Atim Jane Kardong lac Intervention VHT/PE Janet Ocen Nyinyinga Intervention VHT/PE Akello Esther Imakiokwoto Intervention VHT/PE Adong Harriet Bar opok Intervention VHT/PE Dilla Job Opio Akangi Intervention VHT/PE Apio susan Aromo Sub-County Village Comparison or Intervention Interview Type Names of participants Aloc A Intervention Fathers Ojok Tonny Aloc A Intervention Fathers Egwali Denis Aloc A Intervention Fathers Omara Nelson Aloc A Intervention Fathers Munu Denish Aloc A Intervention Fathers Alele Filbert Aloc A Intervention Mothers Awilo Grace Aloc A Intervention Mothers Alobo Jane Aloc A Intervention Mothers Ongola Kelar Aloc A Intervention Mothers Awor Molly Aloc A Intervention Fathers Opio Jaspher Ogeo B Comparison Fathers Okello Aron Ogeo B Comparison Fathers Anena Francis Ogeo B Comparison Fathers Ojok Martin Ogeo B Comparison Mothers Ewai Anna Ogeo B Comparison Fathers Ogweng Richard Ogeo B Comparison Fathers Opio Jimmy Ogeo B Comparison Mothers Kolin Opio Ogeo B Comparison Mothers Acen Mercy 4 Ogeo B Comparison Mothers Agweng Agnes Ogeo B Comparison Mothers Akullu Polly Wigweng Intervention Fathers Omeny Tonny Wigweng Intervention Fathers Ayali Silgard Wigweng Intervention Fathers Odongo Filbert Wigweng Intervention Mothers Lilly Oroc Wigweng Intervention Mothers Atala Rose Wigweng Intervention Fathers Opio Lawrence Wigweng Intervention Mothers Acen Felly Wigweng Intervention Fathers Ogwang Jimmy Wigweng Intervention Mothers Acio Dilish Wigweng Intervention Mothers Akao Lydia Tegweng Comparison Mothers Auma Esther Tegweng Comparison Mothers Adong Dokas Tegweng Comparison Mothers Adong Juliet Tegweng Comparison Mothers Akello Paska Tegweng Comparison Mothers Achola Joy Tegweng Comparison Fathers Ongom Francis Tegweng Comparison Fathers Adar George Bob Tegweng Comparison Fathers Okello Justine Tegweng Comparison Fathers Onyong Aron Tegweng Comparison Fathers Okello Justine Anyang Aromo Health Center Health Center Health Worker EPila Lillian Aromo Heatlh Center Health Center Health Worker Obok Patrick Adagani Control VHT/PEs Agonga Charles Acholidam Intervention VHT/PEs Owot moses Opokmidila Intervention VHT/PEs Amule Santa Ajia Intervention VHT/PEs Juspanti Omara Aromo sub county- Intervention VHT/PEs Abua Nighty Aromo Sub county- Intervention VHT/PEs Atim Dorcus Aromo sub county- Intervention VHT/PEs Alum susan Damodoca Intervention VHT/PEs Odongo Alex Damodoca Intervention VHT/PEs Akelo Agnes Tetido Intervention VHT/PEs Opio Tom Matalala Intervention VHT/PEs Egole Benzamin Barpii Intervention VHT/PEs Omeri Tommy Acholi dum Intervention VHT/PEs Akello Anna Telela Intervention VHT/PEs Atoke Hellen Barpii Intervention VHT/PEs Alyek Betty Ayile A Intervention VHT/PEs Ocet Bosco 5 B. Persons Contacted for the Evaluation of the Standard Child Survival Interventions Child Survival Project Mid-term Evaluation FGD Participants s/n Sub County/HC Village Name of participant HEALTH WORKER FGD 01 Aromo HCIII _ Epilla Lillian 02 ,, _ Obok Patrick 03 Ogur HCIV _ Amodo Doris 04 ,, _ Akello Sophia 05 ,, _ Akullo Josephine 06 Barapwo HCIII _ Achola Hellen 07 ,, _ Harriet Atwongo 08 ‘’ _ Okello George 09 Amuca HCIII _ Acen Colline 10 ‘’ _ Amolo Lillian 11 ,, _ Atim Celia HEALTH UNIT MANAGEMENT COMMITTEE FGD 12 Aromo HCIII Barogin Okello Troy 13 ,, Akaidebe Odongo Robert 14 ,, Omolo Okor David 15 ,, Lelaodwe Ajok S Grace 16 ,, Onyapoyere Silvia Omara 17 ,, Akaidebe Obok Patrick 18 ,, Akaidebe Okello Julius Peter 19 ,, Akaidebe Epilla Lillian 20 Barapwo HCIII Aminyang Ebilla Jino 21 ,, Areco Dophia Atyeno 22 ,, Tedam Otto Click 23 ,, Anyalo Ogwal Moses 24 ,, Tedam Ochen James 25 ,, Tedam Adongo Agnes 26 Ogur HCIV Ogur Corner Opio Tonny 27 ,, Okwaloamara Adiro Judith 28 ,, Akano Akello Sophia 29 ,, Onywaloyale Grace Odongo 30 Amuca HCIII Okecoyere Odida Morris 31 ,, Okecoyere Opinya Moses 32 ,, Okecoyere Abwot Teddy 33 ,, Okecoyere Olung Tom 34 ,, Okecoyere Elizabeth Acheng 35 ,, Okecoyere Mary Oyena 36 ,, Okecoyere Omara Peter 37 ,, Okecoyere Omara Sam VHT FGD 38 Aromo S/ C Ayami Okot James 6 39 ,, Apua Ejang Christine 40 ,, Opokmidila Amule Santa 41 ,, Beo Lilly Angena 42 ,, Acholidumu Owot Moses 43 ,, Ataparadam Okidi Ambrose 44 ,, Alok Grace Owiny 45 ,, Ayitunga Adilo Fred 46 ,, Arwotolaro Apio Christine 47 ,, Adaganii Agonga Charles 48 ,, Ajiagood Juspanti Omara 49 ,, Tegweng Ongom Francis 50 ,, Aloc A Awor Molly 51 ,, Lelakworo Omara Jimmy Aaron 52 ,, Obalodak Ojok Jasper 53 ,, Aloc A Alele Filbert 54 Ogur S/C Acetlela Ameny Linous 55 ,, Timber Opio Mark 56 ,, Adilebi Ogwal Lawrence 57 ,, Awelo Atyang James 58 ,, Apuroimon Ojok Joel 59 ,, Diicunyi Akii Tobby 60 ,, Nginginya Komakech James 61 ,, Aulaimalo Anyima Felix 62 ,, Diicunyi Betty Okwir 63 ,, Nangabir Acen Rose 64 ,, Teakang Betty Owiny 65 ,, Atongimoco Mirriam Otim 66 ,, Onywaloyere Grace Odongo 67 ,, Agak Ketty Omara 68 ,, Bungmiciri Semmy Ekaro 69 ,, Baralegi Christine Okello 70 Lira S/C Areco Otim Sam 71 ,, Telela A Anena Sam 72 ,, Ongica B Omondi Jimmy 73 ,, Tedam Otiti Charles 74 ,, Anyalo Tom Parakak 75 ,, Akolodong Atim Julius 76 ,, Adelokok C P Obong 77 ,, Teokole Opio Morris 78 ,, Bung B Anyonga Vincent 79 ,, Bung A Agang John 80 ,, Ayere Esther Oming 81 ,, Tedam Mido Aguma 82 ,, Areco Hellen Ogweng 83 ,, Adelokok Phoebe Anyii 84 ,, Awita Apita Janet 7 85 ,, Telela A Atala Susan 86 ,, Telela B Akello Lucy 87 ,, Atodi Enin Epila MOTHER’S GROUP FGD 88 Aromo S/C Aloc A Keller Ongom 89 ,, Aloc A Alobo Jane 90 ,, Wigweng Atala Rose 91 ,, Aloc B Awino Grace 92 Ogur S/C Auleipiny Jennifer Anyima 93 ,, Akadocwat Amongi Jennifer 94 ,, Alenga Adong Ketty 95 Lira S/C Telela B Janet Abeja 96 ,, Telela A Harriet Obong 97 ,, Telela Betty Okello 98 ,, Adelokok Florence Oming 99 ,, Bung Dorcus Ogwang FATHER’S FGD 100 Aromo Angole B Ayo Jimmy 101 ,, ,, Ocen Bonny 102 ,, ,, Olot Samuel 103 ,, ,, Olal Geoffrey 104 ,, ,, Odong Bonny 105 ,, ,, Opito Bonny 106 ,, ,, Bodo Denish 107 ,, ,, Okite Denish 108 ,, Obalodak Okello Santo 109 ,, ,, Okello Moses D 110 ,, ,, Olobo Lujino 111 ,, ,, Okello Tonny 112 ,, ,, Okello Cosmas 113 ,, ,, Okello Richard 114 ,, ,, Olal Denis 115 ,, ,, Olaa Ambrose 116 Ogur Sub County Odokokome Omara G 117 ,, ,, Opio Bonny 118 ,, ,, Oroch Tonny 119 ,, ,, Lingo Emmanuel 120 ,, ,, Orwaa 121 ,, ,, Oroc H 122 ,, ,, Opio Bob 123 ,, ,, Omara Denish 124 ,, Gulwoo Ogweng Gerald 125 ,, ,, Ongom Patrick 126 ,, ,, Odyek Sam 127 ,, ,, Ogweng Bosco 8 128 ,, ,, Odongo Denish 129 ,, ,, Omara Micheal 130 ,, ,, Ocak Alfred 131 ,, ,, Okello David 132 Lira S/C Barodyek Nam Denish 133 ,, ,, Odongo Geoffrey 134 ,, ,, Obote Tom 135 ,, ,, Olet Denish 136 ,, ,, Aporo Geoffrey 137 ,, ,, Opio Tom 138 ,, ,, Ocen Peter 139 ,, ,, Odyek Bonny 140 Bung A ,, Omara Innocent 141 ,, ,, Anyanga Geoffrey 142 ,, ,, Ogwal Simon 143 ,, ,, Lee Denish 144 ,, ,, Ajal Seles 145 ,, ,, Oyom Richard 146 ,, ,, Ajal David 147 ,, ,, Obong Denish MOTHER’S FGD 148 Aromo S/C Angole B Eunice Akun 149 ,, ,, Adong Jenifer 150 ,, ,, Amoly Rose 151 ,, ,, Akello Juspine 152 ,, ,, Atim Anna 153 ,, ,, Aguti Anna 154 ,, ,, Akec Esther 155 ,, ,, Akullu Christine 156 ,, Obalodak Ajok Molly 157 ,, ,, Tabitha Akello 158 ,, ,, Abonyo Dorcus 159 ,, ,, Apio Jennifer 160 ,, ,, Acio Juspine 161 ,, ,, Akao Judith 162 ,, ,, Apica Anai 163 ,, ,, Ajok Brenda 164 ,, ,, Akao Sophia 165 ,, ,, Abonyo Grace 166 Ogur S/C Odokokome Caroline Olet 167 ,, ,, Jacinta Ogang 168 ,, ,, Sydonia Tia 169 ,, ,, Evaster Opio 170 ,, ,, Ejang Roseline 171 ,, ,, Acen Susan 172 ,, ,, Grace Ojok 9 173 ,, ,, Dilis Oroch 174 ,, Gulwoo Alek Scovia 175 ,, ,, Akiu Joan 176 ,, ,, Aceng Dillis 177 ,, ,, Akullu Jane 178 ,, ,, Akello Silvia 179 ,, ,, Auma Agnes 180 ,, ,, Ejang Betty 181 ,, ,, Hellen Okao 182 Lira S/C Bung A Betty Onweng 183 ,, ,, Susan Auma 184 ,, ,, Colline Okullo 185 ,, ,, Santa Opio 186 ,, ,, Evaline Ajal 187 ,, ,, Santa Okullu 188 ,, ,, Jacinta Ocen 189 ,, ,, Harriet 190 ,, Barodyek Hadoline Bua 191 ,, ,, Agnes Ojok 192 ,, ,, Janet Olet 193 ,, ,, Lillian Ogwang 194 ,, ,, Judith Okello 195 ,, ,, Christine Okello 196 ,, ,, Grace Odok Annex 4f Lira District Child Survival Project in Uganda Child Health and Development in a Transitional Region Erute North Sub-District, Uganda October 2009 – September 2013 In Partnership with Uganda Ministry of Health Lira District Health Office Knowledge, Practice, and Coverage MTE LQAS Survey December 2011 2 Acknowledgements_______________________________________________________ The MTE survey of Lira District, Uganda is the result of a cooperative effort between many dedicated people living in Uganda, Medical Teams International Uganda, the Lira DHO, and MTI HQ. I am thankful to the staff of Medical Teams International Uganda for providing the opportunity to produce the MTE KPC survey in Lira District. Special thanks go to the Uganda Country Director Felix Omodi, Africa Health Advisor Anna Summer, MTI Uganda Child Survival Program Manager Lydia Akulo, MTI Monitoring and Evaluation Officer Joel Okello, Monitoring and Coordination Officer for MCP Ronald Apunyu for their support, assistance, and collaboration in every aspect of the evaluation, including the design and development of the survey tools, hiring and training the survey staff, and managing every aspect of the data collection Finally, I would like to express my gratitude and respect to each of the team members of the survey. Their tireless work and wonderful attitude made this survey possible. The Supervisors, staff members, and enumerators were a collaborative group consisting of members of MTI Uganda and the DHO and were essential in the completion of the survey. The supervisors and staff members of Medical Teams International Uganda will be essential in continuing the programs to assist the people of Lira district. Dr. Todd A. Nitkin, DPM, FACFAS, MPH Sr. Advisor in Monitoring and Evaluation Representative, Washington, DC Medical Teams International 3 Table of Contents Acknowledgements 2 Table of Contents 3 Acronyms 4 Executive Summary 5 Chapter 1 Background, Process and Partnership Building, and Methods 9 1.1 Background 9 1.2 Process and Partnership Building 17 1.3 Methods 18 Chapter 2 Maternal and Newborn Care 24 Chapter 3 Anthropometry and Infant and Young Child Feeding 26 4.1 Anthropometry-Nutritional Status of Children aged 0-23 months 26 4.2 IYCF-Nutrition 27 Chapter 4 Pneumonia Case Management 29 Chapter 5 Control of diarrhea 30 Chapter 6 Expanded Program of Immunization 32 Chapter 7 Early Childhood Development 34 Chapter 8 Malaria and the Management of Febrile Illness 35 Chapter 12 Summary 36 Attachments: A. Project Matrix Indicators B. Revised Rapid CATCH Indicators 4 ACRONYMS ACF Action Contre la Faim ACT Artemisinin-based Combination Therapies ANC Antenatal Care ARI Acute Respiratory Infection BCC Behavior Change Communication BCG Bacille Calmette-Guerin vaccine BL Assessment CATCH Core Assessment Tool on Child Health CDD Control of Diarrheal Diseases CHW Community Health Workers C-HIS Community Health Information System C-IMCI Community IMCI CI Confidence Interval CL Confidence Limits CMR Crude Mortality Rate CS Child Survival CORE Collaborations and Resources Group CSHGP Child Survival and Health Grant Program CSP Child Survival Project CSTS Child Survival Technical Support D Precision DHO District Health Office DHS Demographic and Health Survey EBF Exclusive Breastfeeding EPI Expanded Program of Immunizations HHI Hands to Hearts International HC Health Center HF Health Facility HIV/AIDS Human Immune Deficiency Virus/ Acquired Immune Deficiency Syndrome HQ Headquarters of MTI located in Portland, Oregon IDP Internally Displaced Person IMCI Integrated Management of Childhood Illnesses IMR Infant Mortality Rate IPTp Intermittent Preventive Treatment during pregnancy ITN Insecticide Treated Net IYCF Infant and Young Child Feeding KPC Knowledge, Practice, and Coverage Survey LLITN Long Lasting Insecticide Treated Net LQAS Lot Quality Assurance Sampling M&E Monitoring and Evaluation MCH Maternal and Child Health MICS Multiple Indicator Cluster Survey MNC Maternal Newborn Care MOH Ministry of Health MT Midterm MTE Midterm Evaluation MTI Medical Teams International MTI Uganda Medical Teams International Uganda N Sample size NGO Non-Governmental Organization NMCP National Malaria Control Program NUMAT Northern Uganda Malaria AIDS & Tuberculosis ORS Oral Rehydration Salts PCM Pneumonia Case Management PDC Parish Development Committee PHC Primary Health Care POU Point Of Use Rapid CATCH Core Assessment Tool on Child Health R-HFA Rapid Health Facility Assessment SP Sulfadoxine-Pyrimethamine SO Strategic Objective TBA Traditional Birth Attendant TOT Training of Trainers TT Tetanus Toxoid U5MR Under 5 Mortality Rate USAID United States Agency for International Development VHT Village Health Team WFA Weight for Age 5 Executive Summary Uganda is a priority country for child survival efforts, with an IMR estimated in 2006 of 78/1,000 live births, U5MR of 134/1,000 live births, and an MMR of 550/100,000 which has not declined during the past ten years. The leading causes of child morbidity in Lira District are (in rank order): malaria, anemia, diarrhea, respiratory infections, and pneumonia. Causes of child mortality are: pneumonia, anemia, malaria, diarrhea, and respiratory infection. Malnutrition is an important contributing factor to infant and child deaths. The targeted location is Erute North Sub-district in Lira District in Northern Uganda. Direct beneficiaries will be 21,948 children <5 and 24,624 WRA for a total of 46,572 direct beneficiaries. Capacity building activities with the DHO will improve the quality of health care for the sub-district population of 107,061. The project goal is to reduce child morbidity and mortality in Uganda. Objectives are: 1) Communities assume responsibility for their own health through strengthening community capacity (VHTs, Parish Development Councils, and Health Sub-districts); 2) Improved health (C-IMCI) and child care (ECD) behaviors among mothers of children <5 years; 3) Improved quality of health facility services through strengthened IMCI and MNC capacity; 4) Strengthened institutional capacity of MTI and DHO to implement effective and efficient child survival activities. These objectives support MoH goals and strategies as well as those of USAID Uganda. MTI is using a two-pronged strategy: a) promoting behavior change and community mobilization to take appropriate responsibility for health: and b) building DHO capacity to provide sustainable, quality service delivery at the facility and community levels. The level of effort by intervention for this Child Survival Project (CSP) is: 25% MNC, 25% PCM, 20% IYCF, 20% CDD, and 10% EPI. MTI is incorporating Early Childhood Development (ECD) activities into its CS project to enhance the impact and sustainability of technical interventions. Research confirms that child survival is indivisible from ECD – including health, physical, social/emotional, and language/cognitive domains. The CSP is integrating ECD into community health activities in order to improve feeding, care giving, and care-seeking behaviors by increasing women’s participation in CHW structures and integrating ECD into C-IMCI and ANC/EPI outreaches. The primary implementing partner for this project is the Lira DHO, which regularly met with the design team and committed human resources to the implementation of the proposal, in order to harmonize approaches and plan for sustainability. Hands to Hearts International (HHI) is a collaborate partner, providing ECD TOT trainings for VHTs, women leaders, and HF staff, and is working with MTI and DHO to adapt existing curriculum to the local context. The project is also coordinating with other CS stakeholders in country. This MTE Knowledge, Practices, and Coverage (KPC) survey was performed in January, 2012. The overall objective of this MTE survey was to estimate the current level of chosen indicators as per the monitoring and evaluation (M&E) Matrix of specific 6 objectives and intervention logic in the areas of maternal newborn care (MNC), nutrition and Infant and Young Child Feeding (IYCF), control of diarrheal disease (CDD), pneumonia case management (PCM), and Expanded Program of Immunization (EPI), along with the indicators of the Rapid Core Assessment Tool on Child Health (CATCH). A Lot Quality Assessment Survey design was utilized to, with the project area divided into 6 Supervision Areas (SAs) through which the project is managed. The SAs chosen for this project are: Ogur comprises SA1 and SA2, Aromo comprises SA3, and SA4, and Aromo comprises SA5 and SA6. Stratified random sampling of 33 eligible households (households that had both an eligible child and the child’s mother present at the time of survey) in each SA was utilized to select the mothers of children aged 0-23 months. The KPC MTE survey was designed utilizing participatory principles of evaluation in the spirit of partnership and capacity building.1 The main findings in the areas of MNC, nutrition and IYCF, control of diarrheal disease, pneumonia case management, EPI, and ECD are as follows: MNC Three of the five project indicators for MNC, IPT, skilled birth attendants, maternal knowledge of postpartum danger signs, and postnatal visit for the mother within 3 days after birth, significantly improved from BL to MT. However, only 2 of the five (IPT use and the use of skilled birth attendants) met the benchmarks set for them at MT. Only the % of mothers receiving at least two tetanus toxoid immunizations in Lira District remained the same through MT (73%). This indicator did not meet the benchmark of 83%, and SAs 1 and 3 did not meet the DR, indicating that they were statistically significantly below the benchmark. This will be investigated as portions of SAs 1 and 3 are furthest from the HFs. Regarding IPT, at baseline only 35% of mothers took proper anti-malarial medication at least 2 times during pregnancy with their youngest child. By midterm this was significantly increased to 59%, meeting the benchmark for MT. However, SA4 did not meet the DR. This will be investigated, as a portion of SA4 is also far from the HF. Also, At MTE, 53% of all childbirths occur under the supervision of a skilled birth attendant, which is statistically significantly improved from BL (35%). In addition, the project met the benchmark for the midterm and all 6 SAs met the decision rule for skilled birth attendance. Anthropometry AND IYCF The overall rate of under-nutrition at baseline was 27.7%, with 17.0% moderately underweight and 10.7% severely underweight. At MT, The % of children with a Z-score of <-2SD at MT is 17.6%, not significantly decreased from BL. However, the benchmark for the MT was met, and all SAs met the DR. Of the 4 IYCF indicators in the project matrix, only 1, the IYCF aggregated indicator, significantly increased at MT, increasing from 23% at BL to 42% at MT. Also, all SAs 1 KPC 2000+ Field Guide, The Child Survival Technical Support Project and CORE, http://www.childsurvival.com/kpc2000/kpc2000.cfm, August 2001. 7 met the DR set for MT. The % of children aged 0-5m who were exclusively breastfed during the last 24 hours did not increase at MT. Additionally, 4 of the 6 SAs did not meet the DR set by the MT benchmark (SAs 2, 3, 4, and 5). Immediate BF practices for newborns remained quite low, at 23%, which does not meet the MT BM. Also, 4 of the 6 SAs did not meet the DR (SAs1, 3, 5, and 6). Similarly, at 55% there was no significant improvement in the % of mothers who do not practice prelacteal feeding, and SAs 3, 4, and 5 did not meet the DR. Pneumonia Case Management Both ARI indicators in the project matrix met the benchmark for MT. The % of children with ARI who were taken to an appropriate health provider increased significantly over the first half of the project (58% to 86%). This increase not only met the BM but also met the Target for the project. All SAs met the DR. The % of children age 0-23 months with ARI who were treated with an antibiotic also increased significantly form BL to MT (35% to 64%). This increase also met the BM and also met the Target for the project. All SAs met the DR. Control of Diarrheal Disease There is a statistically significant increase in the % of mothers of children 0-23m who practice appropriate hand washing. The rate rose from 54% at BL to 75% at MT, with the project meeting the BM for the MT. However, SA 3 did not meet the DR for the benchmark set for MT. There has been no significant improvement in the % of children who received ORS when suffering from diarrhea since baseline (47% to 54%). Also, SAs 2 and 4 did not meet the DR for the BM set for the MT. In addition, zinc usage in children to treat diarrhea remains almost non-existent at 2.6%, due to the fact that the HFs do not have Zinc available. The project did not meet the BM for the MT, and SAs 1, 3, 4, and 5 did not meet the DR for the MT BM. The project will need to investigate the barriers to ORS and zinc use, and also investigate why SA4 did not meet the DR for any of the diarrhea indicators in the project matrix. The % of households of children age 0-23 months that treat water effectively (a Rapid CATCH indicator but not in the project matrix) remained very low at 11%, unchanged from BL. The likely reason for the low water treatment rates is that 91.0% of households in Lira District have access to an improved water source. Expanded Program of Immunization The project showed a significant increase in EPI coverage (BCG, DPT3, OPV3, and measles vaccination before the age of 12 months) from 16% at BL to 38% at MT, with all SAs meeting the DR. In addition, the BM for MT was met. However, there has been no significant change in measles vaccination coverage (80% at MT), as coverage was already quite high (77% at BL). SA1 did not meet the DR for the benchmark set for MT. With regard to Rapid CATCH indicators not in the project matrix, there has been no significant change in access to immunization, as measured by DPT1 immunization rates, card or mother verified (87% at BL to 89% at MT), because this indicator was already quite high. All SAs met the average of the project area as a whole. There also has been no significant change in Health system performance as to immunization, measured by DPT3, 8 card or mother verified (85% at BL to 73% at MT), because this indicator was already quite high. All SAs met the average of the project area as a whole. Early Childhood Development There has been a significant increase from BL (38% to 68%) of mothers of children aged 0-23 months who report playing games with their child in which they have their child identify their body parts, imitate actions, pretend play, or name objects, with all SAs meeting the DR.. There has also been a significant increase from BL of mothers reporting engaging their children in linguistic learning activities (23% at BL to 40% at MT), such as telling their child stories, singing them songs, or naming objects for them at least twice weekly, but this also represents a significant decrease from year 1 (62% at YR 1). SAs 3 and 4 did not meet the DR set for the MT for this indicator. In addition, there has been no significant change from BL in the % of mothers who report that they talk or sing to their child while feeding the child (58% at BL to 65% at MT). SAs 1 and 5 did not meet the DR for this indicator. Malaria At baseline only 51.3% of children less than 24 months presently slept under an insecticide treated bed net. There has been no significant increase in the use of bed nets since BL (51% to 43%), which can be attributed to the fact that there remains a lack of ITNs available in the program area . Also, SA 3 did not meet the average of the combined SAs (the average for the project area). Of the children who had a fever in the 2 weeks prior to the survey, at BL only 22.3% were brought to a qualified health facility within 48 hours of the start of the fever. There has been a significant increase in the proper treatment of malaria (69% at MT). However, SA3 did not meet the average of the combined SAs (project area). Investigation into why SA3 is lagging behind in both the prevention and treatment of malaria will be investigated. Working in partnership with the DHO, MTI Uganda is implementing a project that aims to improve the health of village communities in Lira District through building DHO capacity to provide sustainable, quality service delivery at the facility and community levels, and through promoting behavior change and community mobilization to take appropriate responsibility for health. This is being accomplished through a combination of interventions, including providing supplementary training, supervision, and follow-up coaching of VHTs. The results of this MT KPC Survey using LQAS methods will allow the project to continue in areas with success while developing strategies to investigate and improve both those interventions and SAs that have not met the desired benchmarks for the mid point of the project. 9 CHAPTER 1 BACKGROUND, PROCESS AND PARTNERSHIP BUILDING, AND METHODS 1.1 Background Project Location and Uganda Overview After almost 20 years of protracted insecurity and internal displacement due to attacks from the “Lord’s Resistance Army” (LRA), Northern Uganda has entered a welcome phase of peace. A peace agreement was signed in August 2006, followed by a detailed truce in October 2006. All Internally Displaced Person (IDP) camps have been disbanded in Lira District. This proposed project will operate in Erute North Sub-district of Lira District, part of the Northern Region. The North has traditionally been marginalized, receiving less assistance in comparison to the South. According to the 2006 Demographic and Health Survey (DHS), the Northern region is Uganda’s poorest, with 58% of the region’s population in the country’s lowest wealth quintile. DHS reports that the Northern Region has the fewest available Ante￾Natal Care (ANC) clinics and delivery services, and the North Central Region (where Lira is located) reports 76% of facilities have no source of external funding (including MoH, insurance schemes, and donor agencies), the lowest proportion in Uganda.2 The population is comprised of the Lango and Acholi tribes, with 99% Christians and approximately 1% Muslim. Uganda is a priority country for child survival efforts, with an IMR estimated in 2006 of 78/1,000 live births (neonatal mortality accounts for 41% of this), U5MR of 134/1,000 live births, and an MMR of 550/100,000 which has not declined during the past ten years.3 The leading direct causes of child morbidity in Lira District are (in rank order): malaria, anemia, diarrhea, respiratory infections, and pneumonia.4 Causes of child mortality are: pneumonia, anemia, malaria, diarrhea, and respiratory infection. Malnutrition is an important contributing factor to infant and child deaths. Causes of MMR include hemorrhage (26%), sepsis (22%), obstructed labor (13%), and other (25%, includes malaria and HIV).5 Child mortality values show improvement from 6-10 years ago, when IMR was 98 and U5MR 1626 , but in 2008 Uganda was rated 23trd internationally for high U5MR. In the 2006 DHS, IMR is highest in the Southwest (109) and Northern Regions (106). Total Fertility Rate (TFR) for Uganda remains high at 6.7 2 Uganda DHS, Service Provision Assessment Survey, 2007. 3 Statistics in this paragraph are from the 2006 Uganda Demographic Health Survey; MACRO International; USAID MCH Initiative; and the State of the World’s Children 2008, UNICEF. 4 Lira District Health Office, Annual Report FY2007-008. 5 MoH: Roadmap for Accelerating the Reduction of Maternal and Neonatal Mortality and Morbidity in Uganda 2007-15. 6 The MMR historical trend has variations in methodology and sampling that do not permit for precise comparison. 10 and higher in the North at 7.5. Life expectancy in Uganda is low for males and females (49.1/50.2). Twenty-five percent of households in the North are headed by women, and nearly 50% of female-headed households have virtually no income.7 The national literacy rate is 72.6%8 ; in the North it is only 56% (72% male, 42% female).9 Health Care Services : The Ugandan MoH is decentralized with district teams responsible for planning, budgeting, and monitoring performance. Each district is divided into Health Sub-Districts which are responsible for delivering a basic package of health services, including control of communicable disease, Integrated Management of Childhood Illness (IMCI), reproductive health, immunization, environmental health, health education and promotion, epidemics, and nutrition.10 Uganda was among the first countries to implement the IMCI approach on a national scale, beginning in 1996. By 2000, IMCI had been introduced to 55 of the 56 districts as a MoH priority. A follow-up study of 10 districts (none in the North) had mixed results. Staff turnover following the training was low, but only about half of clinic visits incorporated IMCI. Formation of Village Health Teams (VHTs) is a strategy of the National Health Policy, a focus of the first and second Health Sector Strategic Plan (HSSP) for the MoH. The VHT is planned to be the equivalent of a Health Center (HC) I level, which covers a population of 1,000. However, in reality there are only 2 VHTs per village versus the 5 recommended, and therefore each VHT often convers more than the 1,000 recommended. VHT responsibilities include identification of community health needs, mobilization and monitoring of resources (including HC performance), oversight of specific support activities by trained Community Health Workers (CHWs), and maintenance of registers of population and health status. Erute North has 4 health facilities: 3 level III and 1 level IV.11 One of the HFs (Aromo) isprivately funded. Action Contre le Faim (ACF) operated 5 Supplementary Feeding Centers and 5 Outpatient Therapeutic Programs in Lira District from 2004, but handed over these programs to DHO in 2009. Drug supply in Lira District is relatively stable. Utilization of health services is low, according to MTI’s Knowledge, Practices & Coverage (KPC) baseline survey (conducted in 2009): 22% of mothers took their child to an appropriate provider for fever, and 58% for pneumonia symptoms. Maternal & Newborn Care (MNC)12: On a national level, DHS 2006 found more than 90% of women went at least once for ante-natal care from a skilled provider, but the median length of pregnancy at first visit was 5.5 months and only 47% had 4 ANCs as recommended by national policy. Similarly, pregnant women in Lira District tend to 7 UNDP/GoU survey: Return Livelihood Trends in Northern Uganda. 8 UNDP Human Development Report, 2008. 9 Government of Uganda: Peace, Development & Reconciliation Plan for Northern Uganda. 2007. 10 Uganda MoH: National Health Policy, 1999. 11 Lira District Health Office Annual Work Plan, FY2008-09 12 If not otherwise noted, information comes from the Roadmap for Accelerating the Reduction of Maternal and Neonatal Mortality and Morbidity in Uganda 2007-2015; Ministry of Health, Republic of Uganda, 2007. 11 seek little and late antenatal care. Lira District HMIS also shows a higher dropout rate from 1st to 4th visit than the national level with only 26% of women attending ANC clinic 4 times.13 The DHS showed only 61.8% of mothers in rural areas have taken iron supplements during pregnancy, while only 32.9% were counseled about the signs of pregnancy complications. DHS found 50.8% of women have received 2 or more TT doses during last pregnancy. This was similar in rural areas and in the North. With malaria endemic in most of Uganda, including Lira District, Intermittent Presumptive Treatment during pregnancy (IPTp) is part of the ANC package, with at least 2 doses of Sulfadoxine-Pyrimethamine (SP) recommended to be given through ANC. DHS found 16% of women in rural areas have received at least 2 doses of SP during their last pregnancy. The Lira DHO reports 43% receiving 2 or more.14 In Lira District, during ANC visits at any HF, mothers should receive a “Mama Kit” free, containing 2 sheets of plastic, prophylactic eye ointment, soap, cord ligature, gloves, and 2 syringes. Supplies of Mama Kits from UNICEF were sporadic, and are no longer available at the time of the MT. Nationally, for rural areas, DHS 2006 showed 36% of childbirth to occur in a HF while MTI’s baseline KPC in 2009 found only 35% had a skilled health provider at childbirth. Focus groups with Parish Development Councils (PDCs), VHTs, and mothers revealed a poor opinion of the quality of labor and delivery services in HFs. Trained midwives are said to be “rude”, especially to young girls and older women, and “harsh” during labor. MTI’s KPC Baseline Survey in 2009 showed post-partum care visits for mothers and newborns both low at 16%, similar to DHS results for urban or rural areas. Post-partum care services for child spacing have limitations. The in-charge DHO of maternal health reports only condoms are typically sent and that other supplies (such as injectables, a popular method) are usually low. Infant and Young Child Feeding (IYCF): For the Northern Region, DHS 2006 found 40% stunting (ht/age <-2 SD) and 21.8% low weight for age (<-2 SD), higher than the national averages for rural areas (39.5%/16.5%). MTI’s KPC Survey in 2009 in Lira District found 27.7% with weight for age <-2 SD. Uganda has tended to focus on food security through food availability. Use of pre-lacteal feeds is common (about half of mothers) and early initiation at childbirth is low (Oct. 09 KPC 29%). Although mothers are embracing the message of exclusive breastfeeding, strengthening of recommended infant and young child feeding practices is still strongly needed to ensure optimal health and growth for infants and young children. ACF’s anthropometric survey reports that dietary diversity and frequency is low, with 97% consuming 1-3 food group sources, and 98.9% consuming 1-3 meals per day. Foods include cereals (usually rice, consumed by 42.3% of children under 5), vegetables (45%), 13 Lira District Health Office Annual Report, July 2008. 14 Lira District Health Office HMIS: District Quarterly Assessment Report, September 2008. 12 and legumes (38.8%). Food security is low in the district, and ACF’s survey indicated low food stocks and high price of food. Control of Diarrheal Diseases (CDD): Uganda follows the iCCM approach and CDD as a part of maternal-child health activities at the district and village levels. ORS is a part of the medicine kit irregularly provided to VHTs; in focus groups, VHTs stated they receive sporadic and insufficient ORS. According to DHS 2006, of children with diarrhea, 56% in Northern Region were treated with ORS packets and there was negligible access to zinc. The MTI KPC baseline survey in 2009 found that 47% of mothers to have given ORS during their child’s recent bout of diarrhea. In Lira District, health staff occasionally continue to recommend use of homemade sugar-salt solution, but also recommend increased fluids (breastmilk for infants 0-6 months) and feeding with porridge or other grain-based soft foods during diarrhea for children >6 months. Pneumonia Case Management (PCM): The low coverage of complete childhood immunization and high rates of undernutrition in Lira District most likely contributes to pneumonia as the second cause of morbidity. Pneumonia is the fifth-highest contributor to the district’s burden of disease, but is the primary cause of child mortality.15 DHS 2006 found 73% of children with signs of acute respiratory infection were taken to a skilled health provider; but only 47% received antibiotics (similar in Northern Region). MTI’s KPC Survey in 2009 found 58% of children with signs of pneumonia were taken to a skilled provider. Immunization: DHS 2006 found national complete immunization coverage for children 12-23 months to be 46% (documented or mother’s recall). A May 2008 ACF survey in Lira District found 42% of children received (documented) the measles vaccine. According to the MTI baseline KPC survey, EPI coverage, as measured by BCG, DPT3, OPV3, and measles vaccination before the age of 12 months verified by an immunization card (16%), was exceptionally low. Transport of vaccines from HFs to villages has proven in the past to be the greatest limitation. Malaria: The prevalence of fever in children aged 0-23 months in Lira District was found to be quite high in the baseline KPC Survey at 74.8%. Malaria is highly endemic in Uganda (90-98% P. falciparum) and is the leading cause of morbidity and mortality nearly country-wide.16 In Lira District, malaria accounts for 34% of the disease burden. Part of the reason for this can be explained by the fact that at baseline only 51.3% of children less than 24 months presently slept under an insecticide treated bed net. Of the children who had a fever in the 2 weeks prior to the survey, at BL only 22.3% were brought to a qualified health facility within 48 hours of the start of the fever. Early Childhood Development (ECD): Uganda’s ECD policy was established in 2007, and the sector is still in its infancy. The government ECD policy defines ECD as “a set of actions and behaviors that support a child’s development in a holistic manner including: feeding; providing clothing; shelter and supervision; preventing and attending 15 Lira District Health Office Annual Report, July 2008 16 Uganda Malaria Control Strategic Plan: 2005-6 – 2009-10. 13 to illnesses; engaging the child in interaction; providing stimulation and safe environment for play and exploration; providing love; affection and security; and enabling the development of self-esteem and self confidence.”17 The policy targets children below 8 years of age, with the first subset consisting of children 0-3 years. MTI plans to incorporate Early Childhood Development (ECD) activities into its CS project to enhance the impact and sustainability of interventions. The post-conflict period is often characterized by food insecurity, unreliable infrastructure, and a fragile family unit, and is an important time to rebuild community resilience by strengthening the mother-child bond, developing and reinforcing positive caregiving practices, and investing in quality of care, stimulation, and promoting optimal nutrition to improve long term child health and well being. The CSP has integrated ECD as a means to effectively respond to the unique needs of mothers and children within the post-conflict setting and to enhance health improvements. This innovation complements the third element of the C-IMCI Platform: integrated promotion of key family practices critical for health and promotion. Two primary mechanisms have been identified for ECD integration: women’s groups and ANC/EPI clinics (which occur simultaneously). The CSP utilizes women to improve health and nutrition to a mothers’ group in selected villages. These women leaders provide education on health and nutrition and ECD messages and practices to mothers’ groups they form in their own village and in one other nearby village each, forming a new pair with a woman leader from that village. ANC/EPI clinics have also been integrating ECD in order to increase attendance at the clinics and because ANCs provide an ideal opportunity for waiting pregnant women and their husbands to receive health/EDC messages through “parent chats” and practice on infants. Program Strategy and Interventions Goal and Objectives The project goal is to reduce child morbidity and mortality in Uganda. These objectives support MoH goals and strategies.  Objective 1: Communities assume responsibility for their own health through strengthening community capacity (VHTs, Parish Development Councils, and Health Sub-districts).  Objective 2 : Improved health (C-IMCI) and child care (ECD) behaviors among mothers of children <5 years.  Objective 3: Improved quality of HF services through strengthened IMCI and MNC capacity.  Objective 4: Strengthened institutional capacity of MTI and DHO to implement effective and efficient child survival activities. Strategic Approaches: MTI is utilizing a two-pronged strategy: a) promoting behavior change and community mobilization to take appropriate responsibility for health: and b) building DHO capacity to provide sustainable, quality service delivery at the facility and 17 Government of Uganda: The Early Childhood Development (ECD) Policy. October 2007. 14 community levels. The CSP is implementing activities that strengthen community volunteer capacity to improve maternal and child health, based on the MoH policy prioritizing Village Health Teams (VHTs). Through the CSP, supplementary training, supervision, and follow-up coaching of VHTs is being provided. The proposed CSP is working hand-in-hand with DHO and HF staff that functions as the link for VHTs. At present, the Senior Health Educator of the DHO oversees VHT activities. At the HF level, the Enrolled Nurse or Enrolled Midwife is the contact person for VHTs. The overall plan in Lira District is for iCCM-trained community volunteers to form community-based teams with TB-DOTS monitors. The CSP is assisting with this strategy, guiding communities to form VHTs that include the C-IMCI trained members along with other key community health volunteers With a dual purpose of (a) reaching mothers to improve IYCF practices and MNC care practices, and (b) building on the expertise and capacity of respected women in the community, the CSP is providing training to motivated Women Pairs to facilitate women’s groups as a strategy for community mobilization. Pairs of women, called Child Health Promoters (CHPs), are being selected, with one member selected to influence local health behaviors, such as TBAs or other older women. The CHPs (approx. 8 per parish, for a total of 150) are providing health and nutrition education to mothers’ groups in their own village and in one other nearby village each, forming a new pair with a woman leader from that village. In this way the groups are self-replicating and will reach coverage in an efficient manner. The groups are beginning to go through a series of topics, including ECD; new groups can be formed once the topics have been covered to participant’s satisfaction. The groups are working in tandem with VHTs, as an activity in support of VHTs to reach community goals for health. MTI is assisting the VHTs to meet with Parish Development Committees (PDCs) on a quarterly basis and use structured processes for action-oriented meetings that are based on analysis of the Community Health Information System (C-HIS) information and link needs to available resources. At their request, the CSP is assisting the DHO to build on previous training and strengthen the health facility staff skills through refresher trainings focused on selected topics within IMCI, such as CDD management with ORS+zinc and PCM. Technical Interventions: The level of effort by intervention for this CSP is: 25% Maternal and Newborn Care, 25% Pneumonia Case Management, 20% Infant and Young Child Feeding, 20% Control of Diarrheal Disease, and 10% Immunization. The CSP is coordinating with the DHO and other actors in support of programs directed towards malaria control and HIV/AIDS. ECD is supporting the proposed technical interventions through cross-sectoral collaboration, focused on reinforcing positive early child care practices and interaction. Maternal and Newborn Care (25%): The design of this MNC intervention is based on the Minimum Activities for Mothers and Newborns (MAMAN) framework. Community action is focused on home visits to encourage birth preparedness. Behavior Change Communication (BCC) strategies focus on dissemination of the benefits of early and 15 frequent ANC and complete TT immunization, the danger signs of pregnancy, the dangers inherent in labor and delivery that do not permit for early identification, and post-partum danger signs for mother and newborn. Post-partum home visits by VHTs (especially as women’s involvement in VHT is improved) provide education on cord care and thermal care to mothers. A focus on maternal nutrition, noting sources of micronutrients, emphasizes the preventive aspects of good nutrition before pregnancy, during pregnancy and after pregnancy. IYCF promotion of immediate and exclusive breastfeeding for newborns is linked to MNC activities. Adding ECD training at ANC visits for both parents is helping to draw greater attendance. TBAs that accept the present MoH policy restricting support for labor and delivery are being drawn in as persons of influence and/or as local women leaders and trained in key messages. The project has been working to improve HF staff attitude and approach to client care if improvements in the use of MNC services are to be achieved. Midwives are being encouraged to embrace their role as parent-educator and support person. Refresher training in safe labor and delivery for HF staff is being supported (by an MNC specialist from Kampala and an experienced American medical volunteer), with emphasis on use of the partograph, infection prevention, active management of the 3rd stage of labor, and newborn resuscitation. A Referral System workshop was held with key sub-county and VHT leaders in order to problem-solve barriers to timely referrals. Cord care, thermal care, and promotion of immediate and exclusive breastfeeding are elements of post￾partum care training. Pneumonia Case Management (25%): Improvements in HF and community-based management of respiratory infections is a critical element of the CSP. Pneumonia and diarrhea are the key causes of child morbidity and mortality and at baseline under￾detection of pneumonia was likely, with HF staff said to use only methods of observation to identify pneumonia and a tendency to categorize any fever as malaria. MTI is providing refresher training for HF staff, within the IMCI approach emphasizing diagnosis and treatment of the pneumonia-malaria complex. In coordination with the DHO, follow-up supportive supervision is including an emphasis on PCM skills. MTI is also coordinating with the DHO to provide administrative assistance for improving planning, procurement, and logistics, to ensure that supplies of essential medicines, including antibiotics for pneumonia, are available at HCs. Training of VHTs is teaching caretakers to recognize the danger signs of pneumonia and of the very sick child in general, to emphasize prompt care seeking. A focus on teaching “rational use of medicines” at the community level is included, along with promotion of understanding the difference between a cold or upper respiratory infection and pneumonia, so that caretakers will make wise decisions about the opportunity costs of seeking care. Links between immunization, Vitamin A status, exclusive breastfeeding, and the prevention of pneumonia or reduction of severity are being stressed. Infant and Young Child Feeding (20%): Key messages are based on the Essential Nutrition Actions (ENA), including training on management of common breastfeeding problems and cooking demonstrations based on locally available foods for a diversified 16 diet for children 6-23 months of age. Because KPC baseline data shows significant growth faltering after five months, appropriate complementary feeding is being emphasized. IYCF is enhanced by highlighting the importance of interaction and engagement while eating, making eye contact, and talking to the child to enhance good nutrition and care giving (ECD principles). The greatest level of IYCF efforts are with CHPs and women’s groups, with the rationale that peer-to-peer nutritional counseling and support will improve anthropometric status. The ENA package is also being used to strengthen the IMCI/C-IMCI training curricula for HF staff and VHTs. Control of Diarrheal Disease (20%): USAID funding is being used to improve VHT community-based management of diarrhea and recognition of danger signs of dehydration, blood in stool, or persistent diarrhea that requires seeking care at a HF. The project is working to increase the acceptance of ORS in the target area and improve management with zinc in order to create an opportunity for the CSP to contribute to national scale-up. VHTs, mothers’ groups, and PDCs are excellent media for increasing household awareness and use of this state-of-the-art recommended treatment. MTI is assisting the DHO to include details on ORS + zinc treatment into CDD training through the IMCI approach for HF staff and the C-IMCI approach for VHT/CHWs. At the same time, BCC strategies discourage the previous MoH promotion of home preparation of sugar and salt mixtures, and promote the use of recommended home fluids and nutritional management during illness, including increased breastfeeding (0-23m) and increased feeding (6-23m) up to 8 times/day in reduced amounts with locally used starch and legume staple food (bean and millet porridge). VHTs teach child caretakers to recognize the danger signs of dehydration, bloody diarrhea, and persistent diarrhea that require referral to an HF. MTI’s ECD component reinforces the importance of bonding and understanding infant cues that indicate need for seeking appropriate health care. BCC strategies emphasize prevention, including proper hand washing with soap, safe disposal of infant feces, and safe storage of water in the home. Integrated practices recommended for prevention of diarrhea is also emphasized, including promotion of exclusive breastfeeding, hygienic preparation of complementary foods, and complete immunization, particularly measles. Immunization (10%): The CSP is increasing EPI/ANC attendance and assisting the DHO to expand routine immunization. Integrating ECD in ANC clinics is a key strategy for improving attendance (see “MNC” intervention area above). The DHO has been rolling out efforts to revitalize immunization services with support from multi-lateral and bilateral partners, and this CSP is using Lot Quality Assurance Sampling (LQAS) as a methodology to efficiently target needs for additional outreach activities by HFs to increase immunization coverage and other critical services, such as ANC. MTI promotes community use of existing static health services for immunization. Community vaccinators are being trained by UNICEF and DHO; however, implementing an effective cold chain from the HF to the village level has been a challenge for DHO. Instead, DHO is emphasizing increased immunization coverage through the EPI strategy, which is integrated with ANC clinics. EPI includes immunizations, (DPT, Hib3+ Hep), 17 deworming, Vitamin A, and growth monitoring. This CSP is reinforcing the MoH policy of static EPI/ANC. Objectives of the KPC Survey The MTE KPC survey was conducted in January of 2012. The objectives of the KPC survey were as follows:  Appropriately collect data on the major areas of child and maternal health, including: maternal and newborn care, infant and young child feeding (IYCF), anthropometry, immunization coverage, diarrhea, acute respiratory infection (ARI), fever and malaria, water and sanitation, and hygiene. The survey collected the appropriate data by interviewing 33 mothers of children aged from 0-23m of each age and previous illness category needed to obtain 33 answers to each question in each of 6 SAs. Therefore, this produced a sample size of 198 for each question/indicator. This data will be used to determine progress of all indicators in the Project Design and, combined with qualitative studies, will determine areas of success and challenges, and direct the project in its second half.  Promote capacity building: staff members were trained in the use of survey training methodology in order to facilitate future monitoring and evaluation.  Train staff members in data analysis through the use of LQAS hand tabulation and training sessions following data collection.  Train staff members in making changes to the monitoring and evaluation plan and health information system by relating the indicators used to the projects objectives, outputs, and activities during the data analysis training.  Promote community awareness and acceptance through follow up Community Feedback Sessions.  Partnership building: The survey utilized the concept of partnership building in all phases of training, data collection, and data analysis by involving all key stakeholders in all phases of the survey process. 1.2 Process and Partnership Building MTI has two principal implementing partners in the project, the DHO and Hands to Hearts International. Hands to Hearts International (HHI) is a collaborate partner which has and will continue to provide ECD TOT trainings for VHTs, women leaders, and HF staff, and will work with MTI and DHO to adapt existing curriculum to the local context. The project is coordinated with other CS stakeholders in country as well. The DHO is the local representative of the MOH, and therefore a critical partner for long-term strengthening of county health services. An inclusive process was followed to involve all stakeholders in the design, training, implementation, and analysis of the KPC survey. The DHO, HHI, and USAID were all invited to participate in all aspects of the survey via letters and personal contact that outlined the exact process and dates of the training and survey. The DHO was interested all phases of the survey, and helped in areas where they were able, including, getting the word out to all villages via radio and personal visits, and reviewing and discussing results. They were also helpful during the baseline in adapting the survey to the local context and ensuring that the survey was performed in a culturally appropriate manner. Permission to conduct the survey was obtained from the MOH. The 18 survey team was comprised to be as inclusive as possible in order to foster partnership. . Six supervisors and twenty four enumerators were chosen from members of MTI Uganda and the communities of Lira District. 1.3 Methods The overall objective of this MTE survey is to estimate the current level of chosen indicators as per the M&E Matrix of specific objectives and intervention logic in the areas of nutrition, immunization, pneumonia case management, and control of diarrheal disease, along with the indicators of the Rapid CATCH. In addition to this, questions and indicators were chosen to measure areas of Early Childhood Development. The survey was designed using LQAS methodology so that the project area may be divided into 6 management areas (Supervision Areas), and each of these areas monitored along with the project area as a whole. This provides the project with MT results for the entire project as well as a breakdown by SA so that actions based on the results may be targeted not only by indicator but also by management area. The KPC MTE survey was designed utilizing participatory principles of evaluation in the spirit of partnership and capacity building.18 The Core Team consisted of members of MTI Uganda, with backstopping by the Sr. Advisor in M&E from HQ. The survey team was comprised of 6 supervisors, who were members of MTI Uganda, and the communities of Lira. The enumerators were devised of members of MTI Uganda, , and local survey takers (enumerators) chosen from Lira District. Selection of the team members was based on their skills and their future role in the project and thus provided ownership of the survey and the project itself. The Sr. Advisor in M&E HQ, Africa Health Advisor, MTI Uganda Child Survival Program Manager, MTI Monitoring and Evaluation Officer Joel Okello, , and MTI Uganda country office staff were involved in the evaluation planning process, including the development of the questionnaires and the recruitment of various team members. The trainings for the KPC Survey, data entry, and data analysis were directed by the Director of Operations, the Administrator, and the Sr. Advisor in M&E HQ. Data analysis was performed utilizing Epi Info by the Sr. Advisor in M&E HQ and shared back with the MTI Uganda Core staff for discussions so that the results could be shared with the country staff and then the communities of Lira District through Community Feedback sessions. Final data analysis was performed in MTI HQ. a. Development of the Questionnaire The evaluation team reviewed the project documents including the detailed M&E Matrix with the project’s goal, objectives, and activities. Key indicators were then chosen based on these parameters in conjunction with the newly revised Rapid CATCH (2008) and KPC 2000+ modules.19 The initial draft questionnaire was developed and shared with the MTI Uganda staff for comments, suggestions, and feedback. Local and regional 18 KPC 2000+ Field Guide, The Child Survival Technical Support Project and CORE, http://www.childsurvival.com/kpc2000/kpc2000.cfm, August 2001. 19 ibid. 19 translators then translated the finalized version of the questionnaire into Luo. Separate translators then translated the questionnaires back into English to ensure that the wording of the questions and answer choices were accurate. Any changes necessary were made at that time. Additional changes important to the local context would be made throughout the training, and the final questionnaire was completed following the fourth day of the training which included a field test of the questionnaire. Due to the use of LQAS methodology, in which it is necessary to have each question of every survey answered by the mother (unlike 30 cluster where some questions may go unanswered), there were 8 final questionnaires, broken down by the subgroups necessary to ensure that all questions could be answered by the mother randomly chosen to answer the survey questionnaire. They were: 1. Main Questionnaire-the first questionnaire, asked of the mother of any child 0-23m. Contains questions on the following: a. General information b. Maternal and Newborn Care including antenatal, birth, and postnatal care c. Hygiene d. Water and sanitation, including access to clean water and hygienic sanitation facilities e. Anthropometry f. Early Childhood Development 2. Questionnaire for 0-5m a. Exclusive BF 3. Questionnaire for 6-9m a. Complimentary BF 4. Questionnaire for 6-23m a. Infant and Young Child Feeding 5. Questionnaire for 12-23m a. Immunization coverage 6. Questionnaire for children 0-23m with diarrhea in the 2 weeks prior to the survey 7. Questionnaire for children 0-23m with ARI in the 2 weeks prior to the survey 8. Questionnaire for children 0-23m with malaria in the 2 weeks prior to the survey In addition to the questionnaires, anthropometric measurements consisting of age, gender, and weight of the eligible children aged 0-23 months were taken at the time of the survey from those children randomly chosen for the main questionnaire. Salter hanging scales were used for the weight measurement, which were calibrated prior to each weighing to ensure accuracy. The measurements were taken in order to calculate the child’s weight for age and corresponding Z-scores. 20 b. Sampling design A LQAS sampling design was utilized to add the benefit of managing the project by management areas (Supervision Areas). This methodology allowed the project to obtain rates on all indicators in the project design for the entire project area to compare with baseline, and also determine if each SA in the project met the Decision Rule set for each indicator based on the benchmark set by the project for MT. This allows the project to determine, with a low sample size in each SA, if a SA did not meet the DR for an indicator, indicating that it was statistically significantly below the benchmark set for that indicator. This alerts management that investigation into the causes, and an action plan to overcome the barriers, is required. c. The Selection Process: The MTI Africa Health Advisor, MTI Uganda Child Survival Program Manager, MTI Monitoring and Evaluation Officer Joel Okello, and Monitoring and Coordination Officer for MCP Ronald Apunyu, along with other country office staff met with the DHO and various local leaders and community personnel to determine the respective populations of each of the villages included in the survey sampling frame. Population figures obtained from MOH Lira district statistics for 2011 were used in conjunction with mapping techniques and visualization of the areas by MTI Uganda staff. The project had been divided into SAs at the outset of the project, for management purposes. Survey clusters for each SA were chosen separately. In each SA, each village in that SA was listed randomly, with its population beside it. When the list was complete, the cumulative population of each village was determined by summing the total population of that village with the combined population of all the preceding villages on the list. The total cumulative population of the villages in the SA was then divided by 33 (because 33 sets of the 8 questionnaires were to be answered in each SA) to obtain the sampling interval for that region. A random number was then chosen, with the stipulation being that the number had to be less than or equal to the sampling interval. The cumulative population of each village was then consulted, and the village containing the random number (the village whose cumulative population is equal to or larger than the random number, and whose preceding village had a cumulative population less than the random number) was chosen as cluster number 1. The second cluster was then identified by adding the sampling interval to the random number. The village whose cumulative population contained this number was chosen as the location of cluster number 2. The remaining clusters were then identified by continuing to add the sampling interval to the number that identified the previous cluster. In this way, each cluster was randomly chosen, with proper weight assigned to each village based on its population size. The larger the size of a population of a village, the greater the chance of having one or more clusters assigned to it. This was repeated for each of the 6 SAs, so that 33 sets of questionnaires would be asked in each SA. The center of each cluster was determined by allowing the supervisors and enumerators local to these villages enlist the help of the Village Chief or elders to determine the spot where they felt that an equal number of households were on each side. The survey team 21 then chose a random starting direction by spinning a bottle in the physical center of the cluster. The team would then walk in the direction the bottle pointed, and count the number of households in that direction until they reached the end of the households in that cluster. The survey team would return to the center and then choose a random number from a random number table, with the requirement that it had to be less than the number of homes in that direction. They then counted the doorways in the direction the bottle was pointing until they reached the doorway that corresponded to the random number chosen. This was deemed the first house. A protocol was established and written during the training sessions, prior to the survey, that determined which households, children, and thus mothers would be eligible for the survey. If the chosen household contained a child aged 0-23 months that was present and sleeps in the house at night, and a mother that was present and sleeps in the house at night, the survey would be taken at this household, starting with the Main questionnaire. Then, based on the child’s age and sicknesses suffered in the last 2 weeks, other questionnaires in the set of 8 questionnaires were asked. The survey team then would move to the household that had the closest door relative to the doorway of the household just eliminated to answer any questionnaires in the set of 8 questionnaires that the mother of a child in that house would be eligible to answer. This procedure was then repeated until all 8 questionnaires in the set of questionnaires for that cluster was completed. If a additional clusters were chosen in that same village, the process would then begin again for each new cluster with spinning of the bottle and randomly selecting the first house, until each set of 8 questionnaires were completed. 33 clusters in each SA were used to complete the survey of each SA. d. Training of Supervisors and Enumerators The training of supervisors and enumerators required 4 days in total. The training curriculum was adapted from the CORE Group’s Knowledge, Practice, and Coverage (KPC) Survey Training Curriculum and LQAS curriculum. The training curriculum was shared with the Core team prior to the training, and the Core team was fully involved with all aspects of the training in order to strengthen the local capacity to conduct future small-sample surveys. Six supervisors were chosen for the MTE survey from MTI Uganda staff. The training regimen of the supervisors included: the objective of the evaluation, the sampling process of a 30-cluster sampling frame, proper selection of the clusters, households, children, and mothers, accepted technique and protocol regarding data collection, and an in-depth review of the questionnaires to be used. Measurement of weight was first demonstrated, and then performed, to ensure proper technique. Training methods used included several days of mini lecture followed by discussions, demonstrations, role-play, group work, and pre-testing of the questionnaires. Their responsibilities included supervising twenty-four local enumerators, taking part in every aspect of the data collection, and taking the lead in choosing each cluster’s center, the household chosen, the eligible infant, and then the eligible mother. The training of the enumerators took place with the training of the supervisors and consisted of a several day process that was similar in nature to the training regimen of the supervisors. It included the same several days of mini lecture 22 followed by discussions, demonstrations, role-play, and group work including the measurement of weight, using several children under two for practice. Repeated practice administering the questionnaires and completing each set of 8 questionnaires properly was performed on local volunteer mothers who were not eligible for inclusion in the actual survey. The survey teams then performed a field test of the questionnaire in nearby Lira district villages that had not been randomly selected for the survey, under the watchful eyes of the Core team. Following the field test the training concluded with a meeting to discuss any issues that arose during the field test and make final changes or adjustments of the questionnaire so that it would be as accurate and context appropriate as possible for the survey. These changes were then made to the questionnaire prior to making copies for the survey. e. Data Collection Each supervisor was assigned four enumerators for a total of six groups, each consisting of four enumerators and one supervisor. A supervisor went to the first chosen house, with two of the enumerators in his/her team, to determine the eligibility of that household, choose the eligible child aged 0-23 months, choose the eligible mother, and weigh the child chosen, using the protocol developed previously. The supervisor would then help determine which questionnaires in the set of 8 questionnaires could be asked at that household (always beginning with the main questionnaire). As the 2 enumerators conducted the rest of the interview with the appropriate questionnaires, the supervisor would then take the other two enumerators in the team to the next eligible household and repeat this process, and would therefore alternate between the two groups of two enumerators. The supervisor would then help each group find the remaining eligible children to complete the set of questionnaires. This allowed the supervisor to take the lead role in determining the eligibility of the household, the weighing process, and immediately checking and correcting any problems with each finished questionnaire while the mother was still available. bbbbbbEach evening the supervisors and Core team met and discussed any issues that arose during the day in order to ensure consistency in the data collection process. The data collection process required 6 days in total, with an average length of interview of approximately 35 minutes. f. Data Analysis A preliminary analysis of the data was performed by calculating frequency distributions of major indicators were prepared using the Epi-info 3.5.1 database, so that MTI Uganda staff had some immediate results to guide programmatic decisions and guide community feedback sessions . These results were then used in discussing the Project design in detail with the MTI Uganda staff in order to increase their capacity in developing project designs and formulating monitoring plans from the objectives, outputs, activities, and indicators chosen. 23 The final analysis was then performed, also using the Epi-info 3.5.1 database. All Rapid CATCH indicators, indicators from the M&E Matrix, several indicators chosen from the KPC 2000+ modules, and indicators dealing with ECD and Health Contacts were presented in the analysis. A 95% confidence interval and a precision of 0.5 were used for each indicator, and 95% confidence limits were calculated for each. The results for each SA were calculated separately, and then entered into an Excel Spreadsheet created by the Sr. Advisor for M&E at HQ. This spreadsheet was designed to calculate the DRs for each SA for each indicator, and whether each SA met that DR for that indicator. It also calculated the rate for each indicator using the population of each SA to obtain a weighted average, which is more accurate as it takes each SA’s population into account in the calculation. g. Results and Discussion The results are organized into sections that represent each area of the different study indicators. The following chapters represent the program intervention areas, as per the M&E Matrix. The table located at the beginning of each section specifies the M&E Matrix indicators, with those that are also Rapid CATCH indicators in red font, their definitions, which SAs met or did not meet the DR, and the weighted combined rates for those indicators. The following table then contains any additional indicators that were not in the project matrix but were collected because they are Rapid CATCH indicators. Because they do not have MT benchmarks set for the project (because they are not in the project matrix) the average of results for all 6 SAs for that indicator was used to determine the DR. For these indicators, if a SA did not meet the DR it means that it is statistically significantly below the average of all 6 SAs combined; it does not mean that it did not meet a BM, because no BM was set. For those intervention areas that are not in the project matrix only contain the Rapid CATCH indicators for that area. 24 CHAPTER 2 MATERNAL AND NEWBORN CARE PROJECT MATRIX INDICATORS *Rapid CATCH Indicators highlighted in red  Tetanus Toxoid: The percentage of mothers receiving at least two tetanus toxoid immunizations in Lira District remained the same through MT (73%), indicating relatively good utilization of antenatal care services and tetanus immunizations being performed by antenatal health care staff when these services are utilized. This indicator did not meet the benchmark of 83%, and SAs 1 and 3 did not meet the DR, indicating that they were statistically significantly below the benchmark. This will be investigated as portions of SAs 1 and 3 are furthest from the HFs.  IPT: At baseline only 35% of mothers took proper anti-malarial medication at least 2 times during pregnancy with their youngest child. By midterm this was significantly increased to 59%, meeting the benchmark for MT. However, SA4 did not meet the DR. This will be investigated, as a portion of SA4 are far from the HF  Skilled Birth Attendant: At MTE, 53% of all childbirths occur under the supervision of a skilled birth attendant, which is statistically significantly improved from BL (35%). In addition, the project met the benchmark for the midterm and all 6 SAs met the decision rule. MTI has accomplished this through focus on increasing the use of health facilities and skilled birth attendants through VHTs, and CHPs who are being trained in health promotion. This training also includes the importance of ensuring that Bench-mark for this LQAS Weighted combined Prequency (average coverage) 1 2 3 4 56 Or Average Coverage 1 2 3 45 6 FOR MTE Numerator (yes answers) 24 27 Denominator (Total)-Sample size 33 33 33 33 33 33 Numerator (yes answers) 11 18 Denominator (Total)-Sample size 33 33 33 33 33 33 % of children age 0-23 months whose births were attended by skilled personnel Numerator (yes answers) 14 19 16 12 14 25 35.3% (29.9-41.0) 50.0% 43% 53.30% 46.35% 60.24% Denominator (Total)-Sample size 33 33 33 33 33 33 Numerator (yes answers) 7 17 Denominator (Total)-Sample size 33 33 33 33 33 33 Numerator (yes answers) 3 10 Denominator (Total)-Sample size 33 33 33 33 33 33 N Y Y Indicator Supervision Areas Base-line Final Target Decision Rule (Yor N) for each Supervision Area LL CI UL CI 17 27 22 29 75.7% (70.4-80.4) 90.0% 83% N Y N 73.49% Y Y Y 59.13% 52.28% 65.97% 67.34% 79.64% 18 26 14 24 35.0% (29.6-40.7) 60.0% 48% Y Y Y 9 19 6 4 16.7% (12.5-20.9) 50.0% 33% Y Y Y Y Y N 30.04% 23.65% 36.42% % of mothers with children age 0-23 months who were protected against Tetanus before the birth of the youngest child. (Tetanusgood) % of mothers with children aged 0- 23 months who received at least 2 doses of IPT during the pregnancy with this youngest child. % of mothers of children 0-23m who received a post-natal visit from an appropriately trained health worker within three days after birth Y N % of mothers of children 0-23m able to report at least two known maternal danger signs during the postpartum period indicating the need to seek health care? 14 4 5 4 2.3% (0.9-4.7) 80.0% 41% Y N N N 19.98% 14.41% 25.55% Y Y Y Y Y Y 25 both mother and child receive follow up care including a post partum visit with a trained health professional within 3 days of birth  Danger Signs: There was a statistically significant improvement from BL (2% at BL to 20% at MT); however, the project did not meet the benchmark set for MT, and SAs 2, 3, 4, and 6 did not meet the DR. The benchmark was set quite high at 41% considering the BL was only 2.3%, which explains the significant improvement while not meeting the BM.  Mother Postnatal Visit: There was significant improvement in this indicator, from 16.7% at BL to 30% at MT, and the benchmark (33%) was met due to confidence interval considerations. All SAs met the DR except SA6. RAPID CATCH INDICATORS NOT INCLUDED IN THE PROJECT MATRIX *All indicators that are not in the project matrix do not have a benchmark or target for the project. Therefore the average coverage (the MT combined frequency) of all 6 SAs is utilized to determine the DRs.  ANC: Antenatal coverage (4 or more antenatal visits) made a significant increase from BL (35%) to MT (50%), with all SAs meeting the DR.  Child Postnatal Visit: While postnatal visits to mothers increased from 16.7% to 30%, it is interesting to find that postnatal visits to children, while starting at the same point at BL (16%) did not significantly increase by MT (19%). No Supervision area was significantly below the average of all the SAs combined (average coverage) for this indicator. Reasons for this discrepancy in postnatal visits between mothers and infants will be investigated.  Contraceptive Use: Remains unchanged from BL. SA 3 is significantly below the average. Bench-mark for this LQAS Weighted combined Prequency (average coverage) **Combined Frequency for Project Area 1 2 3 4 56 Or Average Coverage 1 2 3 45 6 FOR MTE FOR YEAR 1 Numerator (yes answers) 13 19 15 12 22 18 39.8 (31.0-48.6) Denominator (Total)-Sample size 33 33 33 33 33 33 Numerator (yes answers) 2 7 10 8 15 2 NA Denominator (Total)-Sample size 33 33 33 33 33 33 Numerator (yes answers) 12 13 3 10 13 9 NA Denominator (Total)-Sample size 33 33 33 33 33 33 Y Y Y Indicator Supervision Areas Base-line Final Target Decision Rule (Yor N) for each Supervision Area LL CI UL CI 35.3% (29.9-41.0) NA 50% Y Y Y Y 49.57% 42.61% 56.54% 33.3% (28.0-39) NA 30% Y Y N Y 29.97% 23.59% 36.35% % of mothers of children age 0-23m who had four or more antenatal visits when they were pregnant with the youngest child Percentage of mothers of children age 0-23 months who are using a modern contraceptive method Y Y Y Y Percentage of children age 0-23 who received a post-natal visit from an appropriate trained health worker within two days after the birth of the youngest child 16.33% (12.2-20.5) NA 19% Y Y Y 18.77% 13.34% 24.21% 26 CHAPTER 3 ANTHROPOMETRY AND INFANT AND YOUNG CHILD NUTRITION 4.1 Anthropometry-Nutritional status of children aged 0-23 months Malnutrition and under-nutrition are major determinants in the increased vulnerability of children to many infectious diseases, including diarrhea, ARI, and febrile illness. Inversely, many infectious diseases may be the cause under-nutrition in children. In addition, the nutritional status of children indirectly reflects the health and nutrition status of mothers. Therefore, the nutritional status of children aged 0-23 months is an important indicator in relation to child survival and community health programs. In this survey, assessment of nutritional status was done through the anthropometric measurement of weight-for-age in children aged 0-23 months. The weight of each child was taken and combined with the age and gender of the child to calculate the weight-for-age indicator. The indicator is expressed in standard deviations (Z-score) from the median values of weight-for-age of the CDC reference population from the year 2000. ANTHROPOMETRY PROJECT MATRIX INDICATORS *Rapid CATCH Indicators are highlighted in red *this indicator had to be reversed in this table (to % NOT underweight) in order to calculate the DR  Undernutrition: The % of children with a Z-score of <-2SD at MT is 17.6%, not significantly decreased from BL. However, the benchmark for the MT was met, and all SAs met the DR. Bench-mark for this LQAS Weighted combined Prequency (average coverage) **Combined Frequency for Project Area 1 2 3 4 56 Or Average Coverage 1 2 3 45 6 FOR MTE FOR YEAR 1 Percentage of children age 0-23 months who are NOT underweight (<-2SD for the median weight for age, according to WHO/NCHS reference population) (WFA) Numerator (yes answers) 24 25 28 29 31 29 Denominator (Total)-Sample size 33 33 33 33 33 33 Y Y Y Indicator Supervision Areas Base-line Final Target Decision Rule (Yor N) for each Supervision Area LL CI UL CI 72.3% (66.9-77.3) 88.0% 80% Y Y Y 82.36% 77.05% 87.67% 27 4.2 Infant and Young Child Feeding (IYCF)-Nutrition Health promotion and education regarding nutrition and breastfeeding is one of the interventions of MTI Uganda. The project is improving the nutritional status of children through the promotion of correct breastfeeding and complimentary feeding practices, including immediate breastfeeding following childbirth, exclusive breastfeeding of children under 6 months of age, and the introduction of digestible and nutritional complimentary foods in children 6 months and greater. PROJECT MATRIX INDICATORS *Rapid CATCH Indicators are highlighted in red  Exclusive Breastfeeding: The % of children aged 0-5m who were exclusively breastfed during the 24 hours prior to the survey did not increase at MT. Additionally, 4 of the 6 SAs did not meet the DR set by the MT benchmark (SAs 2, 3, 4, and 5). The second half of the project will be concentrating on improving nutrition in the project area.  Immediate Breastfeeding: Immediate BF practices for newborns remained quite low, at 23%, which does not meet the MT BM. Also, 4 of the 6 SAs did not meet the DR (SAs1, 3, 5, and 6).  No Prelacteal Feeding: Again, at 55% there was no significant improvement for this indicator, and SAs 3, 4, and 5 did not meet the DR.  IYCF: There has been a significant improvement in this indicator, increasing from 23% at BL to 42% at MT. Also, all SAs met the DR set for MT. The results reveal that while feeding practices for children aged 6-23m have improved significantly, there remains challenges with immediate and exclusive BF of newborns and children aged 0-5m, respectively. Bench-mark for this LQAS Weighted combined Prequency (average coverage) **Combined Frequency for Project Area 1 2 3 4 56 Or Average Coverage 1 23 456 FOR MTE FOR YEAR 1 % of children 0-5 months who were exclusively breastfed during the last 24 hours (ExclBF) Numerator (yes answers) 24 21 19 23 18 26 Denominator (Total)-Sample size 33 33 33 33 33 33 % of children aged 0-23m who were immediately breastfed (put to the breast within 1 hour of birth) (ImmedBF) Numerator (yes answers) 7 11 5 16 3 4 Denominator (Total)-Sample size 33 33 33 33 33 33 % of children aged 0-23m who did not receive prelacteal feeding during the first 3 days of life (NoPrelact) Numerator (yes answers) 21 25 10 16 22 15 Denominator (Total)-Sample size 33 33 33 33 33 33 IYCF: % of children aged 6-23m who are fed according to a minimum of appropriate feeding practices) (IYCF) Numerator (yes answers) 11 16 14 14 13 15 Denominator (Total)-Sample size 33 33 33 33 33 33 46.6% (37.5- 59.2) 75.0% 61% 23.1% (18.4-27.9) 50.0% 37% 48.10% 35.37% 61.95% 42.25% N Y Y Y Y Y Y Indicator Supervision Areas Base-line Final Target Y Y N Decision Rule (Yor N) for each Supervision Area 73.6% (59.7-84.7) 95.0% 84% Y N Y N N N N Y 61.22% 74.24% Y N N 22.73% 29.0% (23.9-34.5) 60.0% 45% N Y N 36.4% (27.5-46.3) 26.3% (18.4-34.2) 55.5% (46.2-64.8) 79.7% (72.4-87.0) LL CI UL CI 55.03% 16.89% 28.57% 49.13% 67.73% 28 Programming in the 2nd half of the project will focus on determining the barriers to success and improving upon the immediate BF of newborns and the exclusive BF of children aged 0-5m, while continuing to improve feeding of children 6m of age and above. This will be accomplished through continued education of mothers through VHTs, CHPs, and HF staff about the importance of immediate breastfeeding in order to break the culturally and socially driven norms causing women to wait hours or even longer before breastfeeding their child. RAPID CATCH INDICATORS NOT INCLUDED IN THE PROJECT MATRIX *All indicators that are not in the project matrix do not have a benchmark or target for the project. Therefore the average coverage (the MT combined frequency) of all 6 SAs is utilized to determine the DRs.  Complimentary feeding: Complimentary feeding is quite high at MT, at 84.13% (79.1-89.2). This is not significantly higher than at BL (69.8% (55.7-81.7) because the sample size of mothers of children aged 6-9m at BL was small (n=53) due to 30 cluster methodology. However, a complimentary feeding rate of 84% is quite high. It is important to note that SA 2 did not meet the DR for the average of all SAs at MT combined.  Vitamin A: The % of children receiving Vitamin A in the past 6m at MT (55%) was significantly decreased from BL (70%). Reasons for this should be investigated to determine the barriers to receiving and taking Vitamin A. Bench-mark for this LQAS Weighted combined Prequency (average coverage) **Combined Frequency for Project Area 1 2 3 4 56 Or Average Coverage 1 2 3 45 6 FOR MTE FOR YEAR 1 % of infants aged 6-9 months receiving breast milk and complementary foods (ComplFeed) Numerator (yes answers) 31 23 28 27 31 29 NA Denominator (Total)-Sample size 34 33 33 33 33 33 % of children aged 6-23m who received Vitamin A in the past 6 months (Vitamin A) Numerator (yes answers) 18 15 16 17 21 23 Denominator (Total)-Sample size 33 33 33 33 33 33 YY Y 55.49% 48.57% 62.41% NA 70.1 (63.3-76.4) NA 55% Y Y Y Y YY Y Indicator Supervision Areas Base-line Final Target Decision Rule (Yor N) for each Supervision Area LL CI UL CI 79.05% 89.21% 69.8% (55.7-81.7) NA 84% Y N 84.13% 29 CHAPTER 4 PNEUMONIA CASE MANAGEMENT PROJECT MATRIX INDICATORS *Rapid CATCH Indicators are highlighted in red Acute Respiratory Infection is recognized as one of the major public health problems in Uganda. Most children were given symptom relieving not curative medicines in the form of cough syrups or country medicines. Objectives of the project are the recognition of the danger signs of pneumonia, improving the access to quality care, and promoting optimal and timely health seeking behavior among mothers/caretakers. The present evaluation estimates the prevalence of ARI among children aged 0-23 months, the mother’s knowledge concerning ARI, the mother’s management of ARI, and timely health seeking behaviors in relation to ARI.  Health Seeking for ARI: The % of children with ARI who were taken to an appropriate health provider increased significantly over the first half of the project (58% to 86%). This increase not only met the BM but also met the target for the project. All SAs met the DR.  Antibiotics f or ARI: The % of children age 0-23 months with ARI who were treated with an antibiotic also increased significantly form BL to MT (35% to 64%). This increase met both the BM for MT and also the target for the project. All SAs met the DR. Bench-mark for this LQAS Weighted combined Prequency (average coverage) **Combined Frequency for Project Area 1 2 3 4 56 Or Average Coverage 1 2 3 45 6 FOR MTE FOR YEAR 1 % of children age 0-23 months with chest-related cough and fast/difficult breathing in the last two weeks who were taken to an appropriate health provider. (ARIApprCareSeeking) Numerator (yes answers) 26 30 26 29 25 32 82.6 (73.6-91.6) Denominator (Total)-Sample size 33 33 33 33 33 33 % of children age 0-23 months with chest-related cough and fast/difficult breathing in the last two weeks who were treated with an antibiotic (AbxCough) Numerator (yes answers) 15 28 16 23 28 21 60,0 (51,7-68.3) Denominator (Total)-Sample size 33 33 33 33 33 33 Y Y Y Indicator Supervision Areas Base-line Final Target Decision Rule (Yor N) for each Supervision Area LL CI UL CI 57.8% (49.4-65.9) 80.0% 69% Y Y Y Y Y Y 34.7% (27.0-43.0) 70.0% 52% Y Y Y 57.69% 81.30% 90.93% 71.03% 86.11% 64.36% 30 CHAPTER 5 CONTROL OF DIARRHEA PROJECT MATRIX INDICATORS *Rapid CATCH Indicators are highlighted in red Diarrhea is a common cause of childhood morbidity and mortality in Lira District and, and Uganda as a whole. It is well proven that diarrhea is one of the major contributors to malnutrition in children. MTI Uganda has made a high priority the reduction of childhood diarrhea prevalence and morbidity through preventive and curative measures. This diarrhea management initiative is an intervention aimed at raising the awareness of mothers/caretakers about the necessary steps required to both prevent and treat diarrheal disease. Interventions aimed at the prevention of diarrhea include instruction in the importance of appropriate hand washing behaviors as well as the promotion of the use of sanitary latrines. Diarrhea case management at the household level is to include proper feeding and fluid management during diarrhea episodes, including the proper preparation and use of Oral Rehydration Salts (ORS).  ORS Use: There has been no significant improvement since baseline (47% to 54%). Also, SAs 2 and 4 did not meet the DR for the BM set for the MT.  Zinc: Zinc usage remains almost non-existent at 2.6%. The project did not meet the BM for the MT, and SAs 1, 3, 4, and 5 did not meet the DR for the MT BM. The reasons for this should be investigated and a decision as to an Action Plan determined to increase both demand and supply. Bench-mark for this LQAS Weighted combined Prequency (average coverage) **Combined Frequency for Project Area 1 2 3 4 56 Or Average Coverage 1 23456 FOR MTE FOR YEAR 1 % of children 0-23 months with diarrhea in the last two weeks who received Oral Rehydration solution (ORS) and/or recommended home fluids. (ORTUse) Numerator (yes answers) 22 14 19 11 19 19 46.5 (37.3-55.7) Denominator (Total)-Sample size 33 33 33 33 33 33 Numerator (yes answers) 0 2 0 0 02 1.8 (0.0-4.2) Denominator (Total)-Sample size 33 33 33 33 33 33 % of mothers of children 0-23m who live in households with soap or ash at the place for hand washing and that washed their hands with soap or ash at least 2 of the appropriate times during a 24 hour recall period. (approphandwashing) Numerator (yes answers) 22 32 21 17 32 25 54.2 (45.2-63.2) Denominator (Total)-Sample size 33 33 33 33 33 33 47.2% (37.5-57.1) 70.0% 59% 0.9% (0.0-5.1) 30.0% 15% 54.0% (48.2-59.7) 80.0% 67% 46.56% 0.38% 75.34% 81.35% 2.60% Y N Y N Y N Y Indicator Supervision Areas Base-line Final Target Decision Rule (Yor N) for each Supervision Area Y N N N Y % of children 0-23 months with diarrhea in the last two weeks who were treated with Zinc (Zinc) LL CI UL CI 69.34% 53.51% 60.46% 4.81% Y Y Y N Y Y 31  Appropriate hand washing with soap: There has been a significant increase from BL, with the project meeting the BM for the MT. However, SA 3 did not meet the DR for the benchmark set for MT. RAPID CATCH INDICATORS NOT INCLUDED IN THE PROJECT MATRIX *All indicators that are not in the project matrix do not have a benchmark or target for the project. Therefore the average coverage (the MT combined frequency) of all 6 SAs is utilized to determine the DRs.  Point of Use w ater treatment: There has been no significant change from BL, and the % is too low to determine a DR for each SA. The likely reason for the low water treatment rates is that 91.0% of households in Lira District have access to an improved water source Bench-mark for this LQAS Weighted combined Prequency (average coverage) **Combined Frequency for Project Area 1 2 3 4 56 Or Average Coverage 1 2 3 45 6 FOR MTE FOR YEAR 1 % of households of children age 0- 23 months that treat water effectively. Numerator (yes answers) 2 3 1 1 58 NA Denominator (Total)-Sample size 33 33 33 33 33 33 10.68% Indicator Supervision Areas Base-line Final Target Decision Rule (Yor N) for each Supervision Area LL CI UL CI 11.3 % (8.0-15.5) NA 11% 6.38% 14.98% 32 CHAPTER 6 IMMUNIZATION In this survey Expanded Program of Immunization (EPI) Access is measured by the percentage of children aged 12-23 months who received a DPT1 vaccination before the age of 12 months as verified by a vaccination card, and EPI Coverage is measured by the percentage of children aged 12-23 months who received a BCG, DPT3, OPV3, and measles vaccination before the age of 12 months verified by an immunization card, meaning that they received full vaccination coverage. Added to these measurements is the drop-out rate which measures the number of children aged 12-23 months who received a DPT1 vaccination by card verification or mothers recall but who were not continued in a vaccination program and therefore did not receive a DPT3 vaccination. These indicators provide an excellent picture of immunization services with regard to access, coverage, and completion of immunizations. PROJECT MATRIX INDICATORS *Rapid CATCH Indicators are highlighted in red  Measles vaccination (card verified or mother’s recall): There has been no significant change in measles vaccination coverage (80% at MT), as coverage was already quite high (77% at BL). Also, SA1 did not meet the DR for the benchmark set for MT. This should be investigated and in part may be due to the distance of some villages in sa1 from the HFs.  Full EPI coverage (card verified by 12m): There has been a significant increase in EPI coverage, from 16% at BL to 38% at final, with all SAs meeting the DR. The BM for MT was met. Bench-mark for this LQAS Weighted combined Prequency (average coverage) **Combined Frequency for Project Area 1 2 3 4 56 Or Average Coverage 1 2 3 45 6 FOR MTE FOR YEAR 1 Numerator (yes answers) 22 27 26 27 30 28 78.4 (70.9-85.9) Denominator (Total)-Sample size 33 33 33 33 33 33 Numerator (yes answers) 11 19 14 10 10 9 28.4 (20.2-36.6) Denominator (Total)-Sample size 33 33 33 33 33 33 Indicator Supervision Areas Base-line Final Target Decision Rule (Yor N) for each Supervision Area LL CI UL CI 79.73% Percent of children aged 12-23 months who received measles vaccine according to the vaccination card or mother’s recall by the time of the survey 77.0% (69.7-83.3) 90.0% 84% N Y Y YY Y YY Y % of children aged 12-23 months who are fully vaccinated (received BCG, DPT3, OPV3, and measles vaccines) by 12 months of age, card verified 15.5% (10.3-22.1) 50.0% 33% Y Y Y 37.88% 31.12% 44.63% 74.13% 85.33% 33 RAPID CATCH INDICATORS NOT INCLUDED IN THE PROJECT MATRIX *All indicators that are not in the project matrix do not have a benchmark or target for the project. Therefore the average coverage (the MT combined frequency) of all 6 SAs is utilized to determine the DRs.  Access to Immunization (DPT1): There has been no significant change in this indicator (87% at BL to 89% at MT) because this indicator was already quite high. All SAs met the average of the project area as a whole.  Health sys tem perfor mance as to i mmunization ( DPT3): There also has been no significant increase in this indicator (85% at BL to 73% at MT) because this indicator was already quite high. All SAs met the average of the project area as a whole. Bench-mark for this LQAS Weighted combined Prequency (average coverage) **Combined Frequency for Project Area 1 2 3 4 56 Or Average Coverage 1 2 3 45 6 FOR MTE FOR YEAR 1 Numerator (yes answers) 30 29 26 31 31 30 NA Denominator (Total)-Sample size 33 33 33 33 33 33 Numerator (yes answers) 24 24 22 26 22 26 NA Denominator (Total)-Sample size 33 33 33 33 33 33 Indicator Supervision Areas Base-line Final Target Decision Rule (Yor N) for each Supervision Area LL CI UL CI 87.0% (80.8-91.7) NA 89% Y Y Y YY Y 88.87% 84.49% 93.25% 73.23% 67.06% 79.40% Percent of children aged 12-23 months who received DPT1 according to the vaccination card or mother’s recall by the time of the survey Percent of children aged 12-23 months who received DPT3 according to the vaccination card or mother’s recall by the time of the survey 85.1% (78.6-90.2) NA 73% Y Y Y YY Y 34 CHAPTER 7 EARLY CHILDHOOD DEVELOPMENT PROJECT MATRIX INDICATORS *There are no Rapid CATCH Indicators in ECD  Cognitive development: There has been a significant increase from BL (38% at BL to 68% at MT), with all SAs meeting the DR.  Linguistic development: There has been a significant increase from BL (23% at BL to 40% at MT), but this also represents a significant decrease from year 1 (62% at YR1 to 40% at MT). SAs 3 and 4 did not meet the DR set for the MT for this indicator.  Stimulation while feeding: There has been no significant increase from BL for this indicator (58% at BL to 65% at MT). SAs 1 and 5 did not meet the DR for this indicator. Bench-mark for this LQAS Weighted combined Prequency (average coverage) **Combined Frequency for Project Area 1 2 3 4 56 Or Average Coverage 1 2 3 45 6 FOR MTE FOR YEAR 1 Numerator (yes answers) 21 29 19 20 23 22 68.4 (59.9-76.9) Denominator (Total)-Sample size 33 33 33 33 33 33 Numerator (yes answers) 13 21 10 3 13 14 61.9 (53.1-70.7) Denominator (Total)-Sample size 33 33 33 33 33 33 Numerator (yes answers) 18 24 20 21 14 27 64.4 (55.8-73.0) Denominator (Total)-Sample size 33 33 33 33 33 33 Indicator Supervision Areas Base-line Final Target Decision Rule (Yor N) for each Supervision Area Y % of mothers of children aged 0-23 months who told their child a story, sang a song, or spent time naming objects for (CHILD) at least 2 times in the past week % of mothers of children aged 0-23 months who report that they talk or sing to the child while feeding the child LL CI UL CI 38.0% (29.3-40.3) 80.0% 59% Y Y YY 22.7% (18.1-27.8) 75.0% 49% YY NN YY 68.47% 62.00% 74.94% 40.06% 33.23% 46.89% 57.7% (51.9-63.3) 80.0% 69% N Y Y N Y 65.36% 58.74% 71.99% Y % of mothers of children aged 0-23 months who provide cognitive stimulation to their child in the form of games such as “where are your eyes”, etc. Y 35 CHAPTER 8 MALARIA AND THE MANAGEMENT OF FEBRILE ILLNESS Lira District is a malaria endemic area and the incidence of fever in children less than 24 months is quite high in these areas. Malaria is highly endemic in Uganda (90-98% P. falciparum) and is the leading cause of morbidity and mortality nearly country-wide.20 In Lira District, malaria accounts for 34% of the disease burden. The present evaluation estimates the prevalence of febrile illness among children aged 0-23 months, the mother’s knowledge concerning malaria and health seeking behaviors in relation to malaria. RAPID CATCH INDICATORS NOT INCLUDED IN THE PROJECT MATRIX *All indicators that are not in the project matrix do not have a benchmark or target for the project.  ITN use (child): There has been no significant increase in the use of bed nets since BL (51% at BL to 43% at MT). SA 3 did not meet the average of the combined SAs (the average for the project area).  Treatment of malari a: There has been a significant increase in the proper treatment of malaria (25% to 69%). However, again sa3 did not meet the average of the combined SAs ( the average for the project area). 20 Uganda Malaria Control Strategic Plan: 2005-6 – 2009-10. Bench-mark for this LQAS Weighted combined Prequency (average coverage) **Combined Frequency for Project Area 1 2 3 4 56 Or Average Coverage 1 2 3456 FOR MTE FOR YEAR 1 % of children age 0-23 months who slept under an insecticide-treated bed net the previous night. Numerator (yes answers) 12 12 8 16 20 20 NA Denominator (Total)-Sample size 33 33 33 33 33 33 Numerator (yes answers) 23 27 18 21 24 23 NA Denominator (Total)-Sample size 33 33 33 33 33 33 36.36% Indicator Supervision Areas Base-line Final Target Decision Rule (Yor N) for each Supervision Area LL CI UL CI 43.26% 50.16% 51.3% (45.5-57.1) NA 43% Y Y N % of children age 0-23 months with a febrile episode during the last two weeks who were treated with an effective anti-malarial drug within 24 hours after the fever began 25.0% (19.4-31.3) NA 69% Y Y N YY Y YY Y 69.23% 62.80% 75.66% 36 CHAPTER 9 SUMMARY The goal of the Lira District Child Survival Project is to reduce child morbidity and mortality in Uganda. Objectives are: 1) Communities assume responsibility for their own health through strengthening community capacity (VHTs, Parish Development Councils, and Health Sub-districts); 2) Improved health (C-IMCI) and child care (ECD) behaviors among mothers of children <5 years; 3) Improved quality of health facility services through strengthened IMCI and MNC capacity; and 4) Strengthened institutional capacity of MTI and DHO to implement effective and efficient child survival activities. These objectives support MoH goals and strategies as well as those of USAID Uganda. MTI is using a two-pronged strategy that includes promoting behavior change and community mobilization to take appropriate responsibility for health and building DHO capacity to provide sustainable, quality service delivery at the facility and community levels. The level of effort by intervention for this Child Survival Project (CSP) is as follows: 1. 25% MNC 2. 25% PCM 3. 20% IYCF 4. 20% CDD 5. 10% EPI. MNC Three of the four project indicators for MNC, IPT, skilled birth attendants, maternal knowledge of postpartum danger signs, and postnatal visit for the mother within 3 days after birth, significantly improved from BL to MT. However, only 2 of the four (IPT use and the use of skilled birth attendants) met the benchmarks set for them at MT. Only the percentage of mothers receiving at least two tetanus toxoid immunizations in Lira District remained the same through midterm (73%), indicating relatively good utilization of antenatal care services and tetanus immunizations being performed by antenatal health care staff when these services are utilized. However, this indicator did not meet the benchmark of 83%, and SAs 1 and 3 did not meet the DR, indicating that they were statistically significantly below the benchmark. This will be investigated as portions of SAs 1 and 3 are furthest from the HFs. Regarding IPT, at baseline only 35% of mothers took proper anti-malarial medication at least 2 times during pregnancy with their youngest child. By midterm this was significantly increased to 59%, meeting the benchmark for MT. However, SA4 did not meet the DR. This will be investigated, as a portion of SA4 is far from the HF. Also, At MTE, 53% of all childbirths occur under the supervision of a skilled birth attendant, which is statistically significantly improved from BL (35%). In addition, the project met the benchmark for the midterm and all 6 SAs met the decision rule. MTI has accomplished this through focus on increasing the use of health facilities and skilled birth attendants through VHTs, and CHPs who are being trained in health promotion. This training also includes the importance of ensuring that both mother and child receive follow up care including a post partum visit with a trained 37 health professional within 3 days of birth. There was significant improvement from BL in the % of mothers who received postnatal visits, from 16.7% to 30%, and the benchmark (33%) was met due to confidence interval considerations. All SAs met the DR except SA6. While there was a statistically significant improvement from BL in the % of mothers who knew at least 2 postnatal danger signs (2% at BL to 20% at MT); however, the project did not meet the benchmark set for MT, and SAs 2, 3, 4, and 6 did not meet the DR. The benchmark was set quite high at 41% considering the BL was only 2.3%, which explains the significant improvement while not meeting the BM. Other MNC Rapid CATCH indicators (that were not in the project matrix) were measured at MT. Antenatal coverage (4 or more antenatal visits) made a significant increase from BL 35% at BL to 50% at MT, with all SAs meeting the DR. While postnatal visits to mothers increased from 16.7% to 30%, it is interesting to find that postnatal visits to children, while also starting at 16% at BL did not significantly increase by MT (19% at MT). This will be investigated to determine the proper course of action for the 3rd year of programming. All SAs met the average of the SAs combined (average coverage) for this indicator. Contraceptive use Remains unchanged from BL. SA 3 is significantly below the average. Anthropometry AND IYCF The nutritional status of children aged 0-23 months is of major concern in Lira District. The overall rate of under-nutrition at baseline was 27.7%, with 17.0% moderately underweight and 10.7% severely underweight. This is similar to the results found by a DHS study in 2006 which revealed the overall rate of under-nutrition to be 21.8% in the Northern region of Uganda. At MT, The percentage of children with a Z-score of <-2SD at MT is 17.6%, not significantly decreased from BL. However, the benchmark for the MT was met, and all SAs met the DR. Of the 4 IYCF indicators in the project matrix, only 1, the IYCF aggregated indicator, was statistically significantly increased at MT, increasing from 23% at BL to 42% at MT. Also, all SAs met the DR set for the MT BM for this indicator. The % of children aged 0- 5m who were exclusively breastfed during the 24 hours prior to the survey did not increase at MT. Additionally, 4 of the 6 SAs did not meet the DR set by the MT benchmark (SAs 2, 3, 4, and 5). The second half of the project will be concentrating on improving nutrition interventions. Immediate BF practices for newborns remained quite low, at 23%, which does not meet the MT BM. Also, 4 of the 6 SAs did not meet the DR (SAs1, 3, 5, and 6). Similarly, at 55% there was no significant improvement in the % of mothers who do not practice prelacteal feeding, and SAs 3, 4, and 5 did not meet the DR. Other nutrition Rapid CATCH indicators that were not in the project matrix were measured at MT. The complimentary feeding rate is quite high at MT, at 84.1% (79.1- 89.2). This is not statistically significantly higher than at BL (69.8% (55.7-81.7) because the sample size of mothers of children aged 6-9m at BL was small (n=53) due to utilizing 30 cluster methodology. However, a complimentary feeding rate of 84% is quite high. It is important to note that SA 2 did not meet the DR for the average of all SAs at MT. 38 Also, the % of children receiving Vitamin A in the past 6m at MT (55%) was significantly decreased from BL (70%). Reasons for this should be investigated to determine the barriers to receiving and taking Vitamin A. Pneumonia Case Management Pneumonia is the fifth-highest contributor to Lira District’s burden of disease, but is the primary cause of child mortality.21 Objectives of the project are the recognition of the danger signs of pneumonia, improving the access to quality care, and promoting optimal and timely health seeking behavior among mothers/caretakers. Both ARI indicators in the project matrix met the benchmark for MT. The % of children with ARI who were taken to an appropriate health provider increased significantly over the first half of the project (58% at BL to 86% at MT). This increase not only met the BM but also met the Target for the project. All SAs met the DR. The % of children age 0-23 months with ARI who were treated with an antibiotic also increased significantly form BL to MT (35% at BL to 64% at MT). This increase met the BM and also met the Target for the project. All SAs met the DR. Control of Diarrheal Disease MTI Uganda has made a high priority the reduction of childhood diarrhea prevalence and morbidity through preventive and curative measures. This diarrhea management initiative is an intervention aimed at raising the awareness of mothers/caretakers about the necessary steps required to both prevent and treat diarrheal disease. Interventions aimed at the prevention of diarrhea include instruction in the importance of appropriate hand washing behaviors as well as the promotion of the use of sanitary latrines. Diarrhea case management at the household level is to include proper feeding and fluid management during diarrhea episodes, including the proper preparation and use of Oral Rehydration Salts (ORS). There has been a statistically significant improvement in the % of mothers of children 0- 23m who practice appropriate hand washing. The rate rose from 54% at BL to 75% at MT, with the project meeting the BM for the MT. However, SA 3 did not meet the DR for the benchmark set for MT. There has been no significant improvement in the % of children who received ORS when suffering from diarrhea since baseline (47% at BL to 54% at MT). Also, SAs 2 and 4 did not meet the DR for the BM set for the MT. In addition, zinc usage in children to treat diarrhea remains almost non-existent at 2.6%. The project did not meet the BM for the MT, and SAs 1, 3, 4, and 5 did not meet the DR for the MT BM. The project will need to investigate the barriers to ORS and zinc use, and also investigate why SA4 did not meet the DR for any of the diarrhea indicators in the project matrix. A Rapid CATCH indicator for diarrhea (that was not in the project design matrix) is point of use water treatment. The % of households of children age 0-23 months that treat water effectively remained very low at 11%, unchanged from BL. The likely reason for the low water treatment rates is that 91.0% of households in Lira District have access to an improved water source 21 Lira District Health Office Annual Report, July 2008 39 Expanded Program of Immunization EPI Coverage is measured by the percentage of children aged 12-23 months who received a BCG, DPT3, OPV3, and measles vaccination before the age of 12 months verified by an immunization card, meaning that they received full vaccination coverage. The project showed a significant increase in EPI coverage from BL (16% at BL to 38% at MT), with all SAs meeting the DR. In addition, the BM for MT was met. However, there has been no significant change in measles vaccination coverage (80% at MT), as coverage was already quite high (77% at BL). SA1 did not meet the DR for the benchmark set for MT. This should be investigated and in part may be due to the distance of some villages in sa1 from the HFs. Rapid CATCH indicators for EPI coverage that were not in the project matrix were measured at MT. There has been no significant change in access to immunization, as measured by DPT1 immunization rates, card or mother verified (87% at BL to 89% at MT), because this indicator was already quite high. All SAs met the average of the project area as a whole. There also has been no significant change in Health system performance as to immunization, as measured by DPT3, card or mother verified (85% at BL to 73% at MT) because this indicator was already quite high. All SAs met the average of the project area as a whole. Early Childhood Development MTI is incorporating ECD activities into its CS project to enhance the impact and sustainability of technical interventions. Research confirms that child survival is positively linked to ECD – including health, physical, social/emotional, and language/cognitive domains. There has been a significant increase from BL of mothers of children aged 0-23 months who report playing games with their child in which they have their child identify their body parts, imitate actions, pretend play, or name objects (38% at BL to 68% at MT), with all SAs meeting the DR.. These are important builders of cognitive, motor, or linguistic functions. In addition, There has been a significant increase from BL of mothers report engaging their children in linguistic learning activities such as telling their child stories, singing them songs, or naming objects for them at least twice weekly(23% at BL to 40% at MT), but this also represents a significant decrease from year 1 (62% at YR1 to 40% at MT). SAs 3 and 4 did not meet the DR set for the MT for this indicator. In addition, there has been no significant change from BL in the % of mothers who provide report that they talk or sing to the child while feeding the child (58% at BL to 65% at MT). SAs 1 and 5 did not meet the DR for this indicator. The CSP is integrating ECD into community health activities in order to improve feeding, care giving, and care-seeking behaviors, by increasing women’s participation in CHW structures and integrating ECD into C-IMCI and ANC/EPI outreaches. Two primary mechanisms have been identified for ECD integration: women’s groups and ANC/EPI clinics (which occur simultaneously). The CSP is utilizing women pairs to improve health and nutrition to a mothers’ group in selected villages. These women leaders provide education on health and nutrition and ECD messages and practices to mothers’ groups they form in their own village and in one other nearby village each, forming a new pair with a woman leader from that village. 40 ANC/EPI clinics have also been selected for integration of ECD in order to increase attendance at the clinics and because ANCs provide an ideal opportunity for waiting pregnant women and their husbands to receive health/ECD messages through “parent chats” and practice on infants. Malaria The prevalence of fever in children aged 0-23 months in Lira District was found to be quite high in the baseline KPC Survey at 74.8%. Malaria is highly endemic in Uganda (90-98% P. falciparum) and is the leading cause of morbidity and mortality nearly country-wide.22 In Lira District, malaria accounts for 34% of the disease burden. Part of the reason for this can be explained by the fact that at baseline only 51.3% of children less than 24 months presently slept under an insecticide treated bed net. There has been no significant increase in the use of bed nets, at only 51% at MT. Barriers to distribution and use of ITNs will be investigated and an AP to increase demand and supply developed. Also, SA 3 did not meet the average of the combined SAs (average for the project area). Of the children who had a fever in the 2 weeks prior to the survey, at BL only 22.3% were brought to a qualified health facility within 48 hours of the start of the fever. There has been a significant increase in the proper treatment of malaria to 69% at MT. However, again SA3 did not meet the average of the combined SAs (project area). Investigation into why SA3 is lagging behind in both the prevention and treatment of malaria will be investigated. In conclusion, by working in partnership with the DHO, MTI Uganda is implementing a project that aims to improve the health of village communities in Lira District through building DHO capacity to provide sustainable, quality service delivery at the facility and community levels, and through promoting behavior change and community mobilization to take appropriate responsibility for health. This is being accomplished through a combination of interventions, including providing supplementary training, supervision, and follow-up coaching of VHTs. The results of this MT KPC Survey using LQAS methods will allow the project to continue to improve in areas where they have had success while developing strategies to investigate and improve those interventions and those SAs that have not met the desired benchmarks for the mid point of the project. 22 Uganda Malaria Control Strategic Plan: 2005-6 – 2009-10. Uganda CSP Project Matrix Indicators CSHGP Intervention Area Project Matrix Indicator Baseline MTE Nutrition Exclusive breastfeeding: Percentage of children 0-5 months who were exclusively breastfed during the last 24 hours 73.6% (59.7-84.7) 67.7% (61.2-74.2) Immediate breastfeeding of newborns: Percent of newborns who were put to the breast within one hour of delivery 29.0% (23.9-34.5) 22.7% (16.9-28.6) Exclusive breastfeeding of newborns: Percent of newborns who did not receive prelactal feeds during the first 3 days after delivery 46.6% (40.8-52.5) 55.0% (48.1-62.0) IYCF: Percent of children age 6-23 months fed according to a minimum of appropriate feeding practices 23.14% (18.4-27.9) 42.3 (35.4-49.1) Immunization Measles vaccination: Percent of children aged 12-23 months who received measles vaccine according to the vaccination card or mother’s recall by the time of the survey 77.0% (69.7-83.3) 78.4% (70.9-85.9) EPI Coverage: Percentage of children aged 12-23 months who are fully vaccinated (received BCG, DPT3, OPV3, and measles vaccines) by 12 months of age, card verified 15.5% (10.3-22.1) 37.9% (31.1-44.6) Control of Diarrhea ORT use: Percentage of children 0-23 months with diarrhea in the last two weeks who received Oral Rehydration solution (ORS) and/or recommended home fluids. 47.2% (37.5-57.1) 53.5% (46.6-60.5) Zinc: Percentage of children 0-23 months with diarrhea in the last two weeks who were treated with Zinc. 0.9% (0.0-5.1) 2.6% (0.4-4.8) Appropriate Hand washing Practices: Percentage of mothers of children 0-23 months who live in households with soap at the place for hand washing that washed their hands with soap at least 2 of the appropriate times during a 24 hour recall period. 54.0% (48.2-59.7) 75.3% (69.3-81.4) ARI/Pneumonia Appropriate Care Seeking for Pneumonia: Percentage of children age 0-23 months with chest-related cough and fast/difficult breathing in the last two weeks who were taken to an appropriate health provider. 57.8% (49.4-65.9) 86.1% (81.3-90.9) Treated with Antibiotic: Percentage of children age 0-23 months with chest-related cough and fast/difficult breathing in the last two weeks who were treated with an antibiotic 34.7% (27.0-43.0) 64.4% (57.7-71.0) MNC Maternal TT Vaccination: Percentage of mothers with children age 0-23 months who were protected against Tetanus before the birth of the youngest child. (Protected refers to receiving at least 2 TT or Td injections before the birth of the youngest child sufficiently close to that birth to provide protection.) 75.7% (70.4-80.4) 73.5% (67.3-79.6) IPT: Percentage of mothers with children age 0-23 months who received at least 2 doses of IPT during the pregnancy with this youngest child. 35.0% (29.6-40.7) 59.1% (52.3-66.0) Skilled Delivery Assistance: Percentage of children age 0-23 months whose births were attended by skilled personnel 35.3% (29.9-41.0) 53.3% (46.4-60.2) Post Partum visit to check on child within the first 3 days after birth: Percentage of mothers of children 0-23 months who received a post-partum visit by an appropriate trained health worker within three days after the birth of the youngest child. 16.33% (12.2-20.5) 18.8% (13.3-24.2) Post-partum Danger Signs:% of mothers of children 0-23 m are able to report at least two known maternal danger signs during the postpartum period 2.0% (0.7-4.3) 20.0% (14.4-25.6) Anthropometrics Underweight: Percentage of children 0-23 months who are underweight (-2 SD for the median weight for age, according to 27.7% (22.7-33.1) 17.6% (12.3-23.0) WHO/HCHS reference population) Early Childhood Development Cognitive Stimulation: Percentage of mothers of children aged 0-23 months who provide cognitive stimulation to their child in the form of games such as “where are your eyes, etc. 38.0% (29.3-40.3) 68.5% (62.0-74.9) Linguistic Stimulation: Percentage of mothers of children aged 0-23 months who told their child a story, sang a song, or spent time naming objects for (CHILD) at least 2 times in the past week 22.7% (18.1-27.8) 40.1% (33.2-46.9) Stimulation While Feeding: Percentage of mothers of children aged 0-23 months who report that they talk or sing to the child while feeding the child 57.7% (51.9-63.3) 65.4% (58.7-72.0) Uganda CSP December 2011 Revised Rapid CATCH indicators CSHGP Intervention Area Rapid CATCH Indicator BASELINE MTE Maternal Newborn Care (1)Antenatal Care: Percentage of mothers of children age 0- 23 months who had four or more antenatal visits when they were pregnant with the youngest child 35.3% (29.9-41.0) 49.6% (42.6-56.6) (2) Maternal TT Vaccination: Percentage of mothers with children age 0-23 months who were protected against Tetanus before the birth of the youngest child. (Protected refers to receiving at least 2 TT or Td injections before the birth of the youngest child sufficiently close to that birth to provide protection.) 75.7% (70.4-80.4) 73.5% (67.3-79.6) (3) Skilled Delivery Assistance: Percentage of children age 0- 23 months whose births were attended by skilled personnel 35.3% (29.9-41.0) 59.1% (52.3-66.0) (4) Post Partum visit to check on mother within the first 3 days after birth: Percent of mothers of children 0-23 months who received a post-partum visit by an appropriate trained health worker within three days after the birth of the youngest child. 16.7% (12.5-20.9) 30.0% (23.7-36.4) (5) Modern Contraception: Percentage of mothers of children age 0-23 months who are using a modern contraceptive method 33.3% (28.0-39) 30.0% (23.6-36.4) Breastfeeding (6) Exclusive breastfeeding: Percentage of children 0-5 months who were exclusively breastfed during the last 24 hours 73.6%(59.7-84.7) 67.7% (61.2-74.2) (7) IYCF: Percent of children age 6-23 months fed according to a minimum of appropriate feeding practices 23.1% (18.4-27.9) 42.6% (35.4-49.1) Vitamin A Supplementation (8) Vitamin A Supplementation in the last 6 months: Percentage of children age 6-23 months who received a dose of Vitamin A in the last 6 months (Mother’s recall). 70.1% (63.3-76.4) 55.5% (48.6-62.4) Immunization (9) Access to immunization services: Percent of children aged 12-23 months who received DTP1 according to the vaccination card or mother’s recall by the time of the survey 87.0% (80.8-91.7) 88.9% (84.5-93.3) (10) Health System Performance regarding Immunization services: Percent of children age 12-23 months who received DTP3 according to the vaccination card or mother’s recall by the time of the survey 85.1% (78.6-90.2) 73.2% (67.1-79.4) (11) Measles vaccination: Percent of children aged 12-23 months who received measles vaccine according to the vaccination card or mother’s recall by the time of the survey 77.0% (69.7-83.3) 78.4% (70.9-85.9) Malaria (12) Child sleeps under an insecticide-treated bednet: Percentage of children 0-23 months who slept under an insecticide-treated bed net (in malaria risk areas, where bed net use is effective) the previous night. 51.3% (45.5-57.1) 43.3% (36.4-50.2) (13) Child with fever receives appropriate antimalarial treatment: Percentage of children 0-23 months with a febrile episode that ended during the last two weeks who were treated with an effective anti-malarial drug within 24 hours after the fever began. 25.0% (19.4-31.3) 69.2% (62.8-75.7) Control of Diarrhea (14) ORT use: Percentage of children 0-23 months with diarrhea in the last two weeks who received Oral Rehydration Solution (ORS) and/or recommended home fluids. 47.2% (37.5-57.1) 53.5% (46.6-60.5) ARI/Pneumonia (15) Appropriate Care Seeking for Pneumonia: Percentage of children age 0-23 months with chest-related cough and fast/difficult breathing in the last two weeks who were taken to an appropriate health provider. 57.8% (49.4-65.9) 86.1% (81.3-90.9) Water and Sanitation (16) Point of Use (POU): Percentage of households of children 0-23 months that treat water effectively. 11.3% (8.0-15.5) 10.7% (6.4-15.0) (17) Soap at the Place for washing: Percentage of mothers of children age 0-23 months who live in a household with soap at the place for hand washing 85.0% (80.4-88.8) 88.6% (84.2-93.0) Anthropometrics (18) Underweight: Percentage of children 0-23 months who are underweight (-2 SD for the median weight for age, according to WHO/HCHS reference population) 27.7% (22.7-33.1) 17.6% (12.3-23.0) Ann nex 4g L Child M ira Distri d Health an Eru Oct U L Midterm Ev ict Child S nd Develo ute North S tober 2009 In Part Uganda M Lira Distr aluation Ra Erute No Dec Survival P pment in Sub-Distric 9 – Septem tnership w Ministry of rict Health apid Health orth Sub-Di Uganda cember 2011 Project in U a Transiti ct, Uganda mber 2013 with f Health h Office Facility As istrict Uganda ional Regi a 3 ssessment ion 2 Table of Contents Acronyms 3 Executive Summary 4 Chapter 1 Program Overview 8 Chapter 2 Purpose of the Health Facility Assessment and Methodology 10 Chapter 3 Main Findings: Accomplishments and Priorities 15 Chapter 4 Action Plan/Conclusions/Recommendations 36 Annex 1: Training of Survey Team 45 3 ACRONYMS ANC Antenatal Care ARI Acute Respiratory Infection CHP Community Health Promoter CHW Community Health Workers C-IMCI Community IMCI CS Child Survival CORE Collaborations and Resources Group CSHGP Child Survival and Health Grant Program CSP Child Survival Project DHO District Health Office EPI Expanded Program of Immunizations HF Health Facility HFA Health Facility Assessment HW Health Worker IMCI Integrated Management of Childhood Illnesses M&E Monitoring and Evaluation MNC Maternal Newborn Care MOH Ministry of Health MTI Medical Teams International MTI/Uganda Medical Teams International/Liberia N Sample size ORS Oral Rehydration Salts PHC Primary Health Care RHFA Rapid Health Facility Assessment USAID United States Agency for International Development VHT Village Health Team 4 I. Executive Summary In October 2009, Medical Teams International began the Lira District Child Survival Project in Uganda: Child Health and Development in a Transitional Region aimed at reducing child and maternal mortality and morbidity in Erute North Sub-District, Uganda. In December 2011, the Child Survival team carried out a Rapid Health Facility Assessment (R-HFA) as part of the Midterm Evaluation process to assess the capacity of the 4 health facilities presently functioning in Erute North Sub-District. The assessment measures capacity in the areas of access, inputs, processes, and performance. The team utilized 3 teams of three individuals, consisting of one supervisor and two enumerators, which included members of the same staff that performed the baseline KPC. This staff was comprised of MTI Supervisors and DHO members. Each team contained at least two members with extensive experience in health. All 4 HFs were scheduled to be surveyed in 2 days. It was necessary to return to 2 HFs to obtain the needed 6 children, as this facility was originally visited on a slow day due to it being market day in this area. Therefore, the entire HFA was completed in 3 consecutive days. At baseline the R-HFA brought to light several areas of needed improvement at the HFs in this Sub-District. The HFs did not perform well in any of the major areas of analysis, including: access (service availability), inputs (staffing and supplies), processes (training and supervision, and Information systems) and HW performance. There have been major improvements in several areas since BL, most notably:  Infrastructure: 1 out of 4 HF at BL to 4 out of 4 HF at MT  Supplies for child services:1 out of 4 HF at BL to 3 out of 4 HF at MT  Drugs for ANC: 1 out of 4 HF at BL to 4 out of 4 HF at MT  training in Child health: 1 out of 4 HF at BL to 3 out of 4 HF at MT  Supervision: 1/3 HF at BL to 3 out of 4 HF at MT  HW Performance (treatment): 1 out of 4 HF at BL to 4 out of 4 HF at MT  Optional indicator for availability of immunizations: 1 out of 4 HF at BL to 4 out of 4 HF at MT  Optional indicator for HF community participation: 3 out of 4 HF at BL to 4 out of 4 HF at MT  Optional indicator for community referral: 3 out of 4 HF at BL to 4 out of 4 HF at MT 5 A summary of the 12 core indicators measured in the RHFA follows: ACCESS (INPUTS) 1. Service Availability:  At baseline, 0/4 of the HFs offered all 3 basic child health services o No facilities saw patients 30 days per month (including outreach).  At MTE, 1/4 of the HFs, Amuca, now offer child health services 30 days per month (including outreach). Aromo and Ogur offer services 28 days per week. Barapwo offers services only M-F (20 hours per week) o Growth Monitoring:  At BL only 2 of the 4 HFs (Ogur and Amuca) offered growth monitoring.  MTE, only Amuca offered GM as mothers did not bring their health cards to the other facilities so they were not able to be checked. o Immunization:  At baseline all 4 HFs offered immunization services through the facility and/or outreach.  At MTE all 4 continue to offer immunization services o ANC Services:  At baseline and MTE all HFs in Lira District offer ANC services, o Delivery Services:  Barapwo still does not provide delivery services as it has just recently (2008) been upgraded from a level 2 facility to a level 3 facility and has no maternity unit. However, Barapwo is presently being moved to a new, adjacent site and will have a functional maternity unit for delivery services. 2. Staffing: o At BL, 0/4 of the HFs met this requirement, with the average HF attainment at 48% (meaning the average HF had 48% of its qualified staff present). o At MTE 1/4 (Amuca) had all staff present. This indicator has been changed from the original RHFA staffing indicator, which was: “% of HFs with at least one provider meeting the country definition as qualified to provide curative care for children is present on day of survey”. All HFs would have met the old indicator. 3. Infrastructure:  At BL, all essential health infrastructure was present on the day of the survey in only 1/4 of the HFs (Ogur), with the average HF attainment at 63%.  At MTE, all essential health infrastructure was present on the day of the survey in all 4 of the HFs 4. Supplies: a. Supplies – Child:  At BL only 1/4 of HFs (Barapwo) had all of the essential supplies to support child health, and the average HF attainment was 40% of these essential supplies.  At MTE 3/4 of HFs (Aromo, Barapwo, and Ogur) had all of the essential supplies to support child health, and the average HF attainment was 95% of these essential supplies. 6 b. Supplies-MNC:  At BL, none (0/4) of the HFs had all of the essential supplies to support maternal￾newborn health available and the average HF attainment was only 25% of these essential supplies.  At MTE, results were essentially the same, with none (0/4) of the HFs having all of the essential supplies to support maternal-newborn health available and the average HF attainment at only 30% of these essential supplies.  None have functioning vacuum extractors or neonatal wraps for warming. c. Supplies-ANC:  At BL 0/4 of the HFs had all of the essential supplies to support antenatal care.  No facilities have albumin test strips, and only 1/4 of the HFs (Aromo) had syphilis testing kits and hemoglobin testing kits available.  At MTE, the categories for ANC Supplies have changed slightly (see Results section, 4. Supplies-ANC for the new category list). However, none of the HFs have all of the essential supplies to support antenatal care.  Only Ogur has hemoglobin testing reagents or malaria test kits.  Ogur had all equipment except functional BP equipment. 5. Drugs: a. Child:  At baseline, 1/4 of HFs (Amuca) had all first line medications available.  At MTE, while still only Amuca had all first line medications available, the average HF attainment rose from 40% to 70% o All facilities now have ORS, first line Antimalarials, and Vitamin A o Only Amuca has 1st line pneumonia drugs o Only Amuca and Ogur have 1st line dysentery drugs b. MNC:  At BL, none of the HFs had all of the essential delivery and neonatal drugs present on the day of the survey. In fact, all 3 essential MNC drugs were available in only 1 out of the four HFs each (antibiotics-Amuca; neonatal eye ointment-Ogur; and Oxytocin-Ogur).  At MTE, still none of the HFs had all of the essential delivery and neonatal drugs present on the day of the survey. The drugs were available at the following locations-Amuca; neonatal eye ointment-Amuca and Ogur; and Oxytocin-Barapwo and Amuca. c. ANC:  At BL, only 2/4 HFs had all of the ANC drugs present on the day of the survey. At MTE, all HFs have all the essential ANC drugs. PROCESSES 6. Information Systems a. Child:  At BL, none of the HFs maintained up to date records on sick children under 5 years of age, or up-to-date monthly service data. o The average facility had only 25% of the elements required for having up￾to-date records and monthly service data.  At MTE, still 0/4 of the HFs maintained up to date records on sick children under 5 years of age, or up-to-date monthly service data. o However, each facility now has 80% of the elements required for having up-to-date records and monthly service data. o No facility has complete information in the sick child register, which was the limiting factor. 7 b. ANC:  At BL, 2/4 HFs (Aromo and Amuca) had up to date records on antenatal care.  At MTE, only Amuca (1/4) had up to date records on antenatal care. Aromo did not have up to date TT information as it did at BL. 7. Training:  At BL. only1/4 of the HFs (Ogur) had HWs who reported receiving in-service or pre￾service training in both maternal neonatal care and child health in last 12 months.  At MTE, 3/4 of the HF (all except Barapwo) had HWs who reported receiving in￾service or pre-service training in both maternal neonatal care and child health in last 12 months. 8. Supervision:  At BL, only 1/4 of the HFs (Ogur) received any type of external supervision at least once in the 3 months prior to the survey.  At MTE, 3/4 HFs received supervision (Barapwo did not). PERFORMANCE 9. Utilization of Cura tive services: The percentage of HF with > 1 sick child encounters per child under the age of 5 in Lira District was not able to be calculated, because no facilities had complete sick child registries available at the time of survey. 10. HW Performance (Assessment):  The baseline found that no facilities (0/4) contained HWs who were routinely performing all 5 key assessment tasks required for this indicator. o Furthermore, the average facility was performing only 21% of the key assessment tasks.  At MTE, no facilities (0/4) contained HWs who were routinely performing all 5 key assessment tasks. o However, the average facility is now performing 68% of the key assessment tasks. o Only Amuca was able to check child heath cards, which they did 100% of the time, because mothers attending the other facilities did not bring child health cards with them to their visit. The HW asked for the HC at the other 3 HFs, but the mothers did not have them. This was the limiting factor with this indicator: 11. HW Performance (Treatment):  At BL, only 1/4 of the HFs (Aromo) provided treatment that is routinely appropriate to the diagnosis.  At MTE, all 4 facilities provided correct treatment appropriate to the diagnosis. 12. HW Performance (Counseling):  AT BL, none (0/4) of the HFs were routinely properly instructing caretakers in how to correctly administer drugs prescribed for their child.  At MTE, despite training in IMCI, this continues to be a challenge as still no HF routinely had caretakers on a routine basis that could describe how to administer drugs prescribed for their child properly. 8 CHAPTER 1 Program Overview Project Area and Description: Medical Teams International is implementing the Lira District Child Survival Project (LDCSP) in the Erute North Sub-District of Northern Uganda. The goal of the project is to reduce morbidity and mortality of children under five and women of reproductive age in Erute North Sub-District, Uganda. Direct beneficiaries are 22,907 children <5 Aromo2, 907 WRA for a total of 45,814,572 direct beneficiaries. The level of effort by intervention for this Child Survival Project (CSP) is 1/4 MNC, 1/4 PCM, 20% IYCF, 20% CDD, and 10% EPI. Capacity building activities with the DHO are improving the quality of health care for the sub-district population of 113,400. The primary implementing partner for this project is DHO. Hands to Hearts International (HHI) is a collaborative partner, providing Early Childhood Development TOT trainings for VHTs, women leaders, and HF staff, and working with MTI and DHO to adapt existing curriculum to the local context. Health Care Services: The Ugandan MoH is decentralized with district teams responsible for planning, budgeting, and monitoring performance. Each district is divided into Health Sub￾Districts which are responsible for delivering a basic package of health services, including control of communicable disease, Integrated Management of Childhood Illness (IMCI), reproductive health, immunization, environmental health, health education and promotion, epidemics, and nutrition.1 Successful implementation largely reflects district management priorities, capacities, and resources for training and supervision. A recent follow-up study of 10 districts (none in the North) had mixed results. Staff turnover following the training was low, but only about half of clinic visits incorporated IMCI. At baseline there were a total of 4 functioning government health facilities in Erute North Sub-District, Uganda. During 2007-08, DHO reported 93% of HCs had no stock out for first-line malaria, 78% quinine, 90% Fansidar, 97% measles vaccine, 93% Oral Rehydration Salts (ORS) sachets, and 91% Cotrimoxizole tablets. According to DHS 2006, 56% of children with diarrhea in Northern Region were treated with ORS packets and there was negligible access to zinc. Utilization of health services is low, according to MTI’s Knowledge, Practices & Coverage (KPC) survey (conducted in Aug. 2007): 30% of mothers took their child to an appropriate provider for fever, and 58% for pneumonia symptoms. When asked about their impression of services at Health Facilities (HFs), focus group participants expressed dissatisfaction, citing inadequate manpower, unqualified personnel rendering services, harsh treatment from some HF personnel, late arrival or absenteeism of HF personnel causing patients to wait, and overwhelming 1 Uganda MoH: National Health Policy, 1999. 9 numbers of patients at the HF. According to the district Biostatistician, approximately 54% of the Sub-district population lives within 5 km of a HF. The farthest distance from a sub-district referral center (HC IV) is 25 km, and 45 km to the regional referral center in Lira Town. Project Goals, Objectives and Strategy The project goal is to reduce child morbidity and mortality in Uganda. Objectives are: 1) Communities assume responsibility for their own health through strengthening community capacity (VHTs, Parish Development Councils, and Health Sub-districts); 2) Improved health (C-IMCI) and child care (ECD) behaviors among mothers of children <5 years; 3) Improved quality of health facility services through strengthened IMCI and MNC capacity; 4) Strengthened institutional capacity of MTI and DHO to implement effective and efficient child survival activities. These objectives support MoH goals and strategies as well as those of USAID Uganda. MTI is using a two-pronged strategy: a) promoting behavior change and community mobilization to take appropriate responsibility for health: and b) building DHO capacity to provide sustainable, quality service delivery at the facility and community levels. The level of effort by intervention for this Child Survival Project (CSP) is: 1/4 MNC, 1/4 PCM, 20% IYCF, 20% CDD, and 10% EPI. 10 CHAPTER 2 Purpose of the Rapid Health Facility Assessment and Methodology The objective of this Health Facility Assessment is to collect quantitative data regarding the health system in the project area, particularly the primary health care (PHC) facilities (level 1) providing maternal, neonatal, and child health (MNCH) services. The new Rapid Health Facility Assessment (RHFA) used to collect this data has been designed and recently upgraded by CSTS, and is now called RHFA Version 2.1. 2 The RHFA is designed to be rapid and cost effective, and is designed to be used at the local level to devise strategies, with the MOH entity present in the project area, and to improve the delivery of integrated child health services.3 This baseline assessment was conducted before IMCI training had begun, and prior to any interventions aimed at health facilities so that an integrated strategy to improve the quality of heath care could be implemented. This baseline RHFA will measure progress in the 12 core areas of HF capacity so that areas of needed improvement may be identified. The R-HFA version 2.1 utilizes as its subjects the first six children, under the age of five, entering the health facility on the day of the survey with diarrhea, fever, or coughs and examines three major areas of health care delivery4 :  Case management: Does the healthcare worker (HW) assess, diagnose, and treat children with diarrhea, fever (malaria), and ARI properly? Does the HW explain follow up care to the caretaker well?  Health facility infrastructure: Does the health facility have the necessary equipment, supplies, medications, and privacy to perform adequate MCH services?  Management (Processes): Are the proper management processes being followed in the health facility (supervision, record keeping, and continuation of training)? The R-HFA focuses on the delivery of care for the most important causes of infant and child morbidity and mortality, which include: diarrhea, acute respiratory infections (ARI), malaria, measles, and malnutrition. The purpose of collecting this data is to allow the project, in conjunction with the MOH, DHO, and other health workers in the project area, to determine gaps in service and prioritize their response in order to provide 2 R-HFA Version 2.1; CSTS 2007; http://www.childsurvival.com/rhfa_1.cfm 3 BASICS II Health Facility Assessment; BASICS 1999; http://www.basics.org/Publications/pubs/hfa/hfa_apdxc.htm 4 Ibid. 11 essential, integrated health services. There are four main modules in the R-HFA, with a fifth optional module. All are formatted in Excel for ease of use5 : a. Observation Checklist for sick child care: To observe the HW in the assessment, diagnosis, and treatment of six consecutive cases of care of children under the age of five with fever, diarrhea, or breathing difficulty. The HW is assessed for adherence to the national (IMCI) protocol for assessment, classification, and treatment of childhood illness. b. Client (Caretaker) Exit Interview: To assess whether the caretaker has the correct knowledge of how to administer drugs given for diarrhea, malaria, and/or breathing difficulty (used a proxy for adequate counseling), and whether the caretaker knows under what circumstances the child is to return to the clinic. c. Health Facility Checklist: To assess the presence of a minimal level of infrastructure, equipment, supplies, and medications. d. Health Worker Interview and Record Review: To assess the staffing, MNCH services offered, and also assess the frequency of training, supervision, and other key processes. e. CHW Survey and Checklist (optional): To collect data on CHWs regarding six of the twelve health facility core indicators (through examination of registers). Selecting the Sampling Frame The sampling methodology has been revised by CSTS to use a quality assurance type approach similar, but not identical, to that used in LQAS. The new methodology dictates that at least 80% of the health facilities perform adequately, according to the indicators chosen in the HFA, for the project area to pass. This is called the performance benchmark. Also, an unacceptable level of 2/4 is chosen as the level that should not go undetected in determining that the health facilities are not performing adequately in regard to a given indicator6 . The alpha and beta errors have been placed at 10%. The new R-HFA software automatically calculates the sample size needed, and calculates the results following the survey. Because there are only 4 functioning HFs in the project area, no sample size selection was necessary as all 4 HFs were surveyed. Selecting the Survey Teams Three teams of three individuals, one supervisor and two enumerators were formed using the same staff that participated in the baseline KPC Survey and community Feedback sessions. This staff was comprised of MTI Supervisors and DHO members. Each team contained at least two members with experience in health. Therefore, all 4 HFs were scheduled to be surveyed in 2 days. It was necessary to return to 2 HFs to include the needed 6 children, as these facilities were originally visited on a slow days due to a 5 R-HFA Version 2.1; CSTS 2007; http://www.childsurvival.com/rhfa_1.cfm. 6 R-HFA Version 2.1; CSTS 2007; http://www.childsurvival.com/rhfa_1.cfm 12 market day and other factors. Therefore, the entire HFA was completed in 3 consecutive weekdays. Each element of the R-HFA was administered by the following members of the R-HFA survey team: a. Observation Checklist for sick child care: Enumerator with experience in health b. Client (Caretaker) Exit Interview: Enumerator, with guidance from Enumerator with experience in health c. Health Facility Checklist: Supervisor d. Health Worker Survey: Supervisor of Enumerator with experience in health; should be performed by whichever team member has completed their other duties. e. CHW Survey and Checklist (optional): Enumerator with guidance from Supervisor or Enumerator with experience Training the Survey Team The training of the survey team required four days and was facilitated by the HQ Sr. Advisor in M&E who directed the MT survey. The Training Curriculum devised for this training is available in Appendix 1. The main objectives of the training were to discuss the purpose of the survey and the resulting information; discuss the logistics of the survey; review and practice each of the forms; and practice administering these forms in the facility setting. A health facility that was near the training site but not part of the project in Erute North Sub-District was chosen for the HFA was used for the field test. The Survey Process Observation of Clinical Care and Caretaker Exit Interview The first six children under the age of five presenting to the facility during the survey period whose caretakers describe them as having diarrhea/vomiting, fever/malaria, or cough/difficulty breathing/pneumonia were included in the sample. The caretakers were met as they entered the clinic, and if they agreed to take part in the survey they were followed throughout the facility. If the caretaker brought more than one sick child under the age of five, one child was randomly chosen to be the index child. The enumerator with experience in health observed the clinical encounter between the HW and the caretaker and child. The second enumerator conducted the Exit Interview with the caretakers of sick children outside of the facility as they exited, following receiving the child’s medications. Health Facility Checklist After ensuring that these interviews were proceeding well, the supervisor completed the Health Facility Checklist with an available HW at the facility. A HW was present because determining the conditions in the consultation room and of some of the equipment required some discussion with the HW. 13 Health Worker Survey Following completion of the observation of six consultations between the HW and the caretaker/child, the enumerator with experience or the supervisor, whichever was available, performed the HW Survey. Providing Feedback to the Staff Surveyors were instructed to provide some feedback to staff on the day of the assessment. The feedback on positive points was provided to alleviate any anxiety the staff may have felt due to the survey, but also included any comments necessary to improve clinical treatment and management techniques. It was recommended during the training that feedback regarding the following items be given7 :  Strengths and problems in case management, particularly in the assessment and treatment of sick children  Quality of home-care advice and communication between health workers and caretakers  Inappropriate use of medications  Problems in record keeping  Ways to improve clinic organization  Major barriers to effective practice Checking the Completed Questionnaires Completed questionnaires were checked by the supervisor and enumerators administering the questionnaire immediately at the conclusion of the interview so that any discrepancies or missed questions could be discussed with the person being interviewed. At the end of each facility session, the Supervisor reviewed all forms with the enumerators before leaving the facility. The completed forms were then brought to the central point chosen for data entry and given to the data entry staff. This was done nightly so that data entry could be performed during the data collection period. The data entry staff reviewed the completed questionnaires for accuracy while the survey team was there, in order to clarify or correct any unclear or incorrect items noticed in the forms. Data Entry During the data collection phase of the survey, data was entered into the R-HFA Excel program provided in the R-HFA zip file available on the CSTS website8 . Data was entered daily throughout the survey so that any discrepancies could be discussed with the supervisors as soon as possible. Cleaning of the survey data was accomplished by the 7 R-HFA short instruction 12-09-07;R-HFA Version 2.1; CSTS 2007; http://www.childsurvival.com/rhfa_1.cfm 8 R-HFA Version 2.1; CSTS 2007; http://www.childsurvival.com/rhfa_1.cfm 14 data entry staff as the data was presented. Following data entry for the final assessment, all data was then combined in the single Excel file, provided by CSTS, and analyzed by the HQ Senior Advisor in M&E and was shared immediately with the team in Uganda. The R-HFA survey forms file has a tabulation plan for hand tabulating the disaggregated indicators (each indicator alone), and the aggregated indicators that comprise the 12 core indicators9 . This includes bar graphs and tables that will provide useful reporting tools to provide in the HFA Report. Constraints/Difficulties: Two of the HFs did not have the required number of children present with illness, due to environmental factors such as a market day in the village. Therefore, the same interview teams returned to these facilities the following day to complete the survey. This did not affect the results. 9 Ibid. CHAPTER 3 Main Findings ACCESS (INPUTS) 1. Service Availability BASELINE Indic. # Domain Indicator % HF with all elements 1 CHILD Service Availability - Child % HF that offer all three basic child health services (growth monitoring, immunization, sick child care) 0/4 1 ANC Service Availability - ANC % HF that offer ANC at least once a week 4/4 1 NEO Service Availability - Delivery % HF that offer delivery services on all days 2/4 MTE Indic. # Domain Indicator % HF with all elements 1 CHILD Service Availability - Child % HF that offer all three basic child health services (growth monitoring, immunization, sick child care) 1/4 1 ANC Service Availability - ANC % HF that offer ANC at least once a week 4/4 1 NEO Service Availability - Delivery % HF that offer delivery services on all days 1/4  At baseline, 0/4 of the HFs offered all 3 basic child health services o This is due to the fact that no facilities saw patients 30 days per month (including outreach). However, all of the facilities provided sick child services at a minimum of Monday-Friday, for a total of 20 hours per week  At MTE, 1/4 of the HFs, Amuca, now offer child health services 30 days per month (including outreach). Aromo and Ogur offer services 28 days per week. Barapwo offers services only M-F (20 hours per week) o Growth Monitoring:  At BL only 2 of the 4 HFs (Ogur and Amuca) offered growth monitoring.  At MTE, only Amuca offered GM as mothers did not bring their health cards to the other facilities so they were not able to be checked. Activities have been put in place at all HFs to encourage all mothers to bring their children’s health card with them to every visit. As part of the IMCI protocol, growth monitoring will continue to be introduced to the HFs so that all 16 HFs will provide growth monitoring services in the facility and/or through outreach. o Immunization:  At baseline all 4 HFs offered immunization services through the facility and/or outreach.  At MTE all 4 continue to offer immunization services o ANC Services:  At baseline and MTE all HFs in Lira District offer ANC services, o Delivery Services:  Barapwo still does not provide delivery services as it has just recently (2008) been upgraded from a level 2 facility to a level 3 facility and has no maternity unit. However, Barapwo is presently being moved to a new, adjacent site and will have a functional maternity unit for delivery services. 2. Staffing BASELINE Indic. # Domain Indicator % HF with all elements Index Value (% avg. HF attainment) 2 Staffing % HF with all staff who provide clinical services working on the day of survey 0/4 48% MTE Indic. # Domain Indicator % HF with all elements Index Value (% avg. HF attainment) 2 Staffing % HF with all staff who provide clinical services working on the day of survey 1/4 68% This indicator has been changed from the original RHFA staffing indicator, which was: “% of HFs with at least one provider meeting the country definition as qualified to provide curative care for children is present on day of survey”. All HFs would have met the old indicator, both at BL and MTE, because at least one qualified provider was present at each facility. However, the new indicator determines the number of each type of staff, and determines if they are all present on the day of the survey. o Therefore, at BL, 0/4 of the HFs met this requirement, with the average HF attainment at 48% (meaning the average HF had 48% of its qualified staff present). o At MTE 1/4 (Amuca) had all staff present. The project is working with the DHO to continue to improve staff attendance so that treatment of patients is as effective as possible and refresher training in IMCI is as 17 effective as possible by ensuring that all HW treating patients are present to benefit from project follow-up mentoring and supportive supervision activities after IMCI topic training is provided. 3. Infrastructure BASELINE Indic. # Domain Indicator % HF with all elements Index Value (% avg. HF attainment) 3 Infrastructure % HF in which all essential infrastructure is present and functioning on day of the survey (improved water source; functional latrine for clients; setting allowing auditory and visual privacy) 1/4 63% Has at least one bed 3/4 Has 24 hour staff coverage 3/4 Has functioning emergency communication 3/4 Has emergency transportation usable today 0/4 Has electricity from the grid or a generator with fuel 3/4 Has a usable client latrine 4/4 Has water from protected water source on or near grounds 2/4 Has auditory and visual privacy 2/4 MTE Indic. # Domain Indicator % HF with all elements Index Value (% avg. HF attainment) 3 Infrastructure % HF in which all essential infrastructure is present and functioning on day of the survey (improved water source; functional latrine for clients; setting allowing auditory and visual privacy) 4/4 72% Has at least one bed 3/4 Has 24 hour staff coverage 4/4 Has functioning emergency communication 3/4 Has emergency transportation usable today 0/4 Has electricity from the grid or a generator with fuel 1/4 Has a usable client latrine 4/4 Has water from protected water source on or near grounds 4/4 Has auditory and visual privacy 4/4  At BL, all essential health infrastructure was present on the day of the survey in only 1/4 of the HFs (Ogur), with the average HF attainment at 63%. o The limiting factors were: emergency transportation, which was not available in any of the HFs, an improved, protected water source, which was only available in 2/4 of the HFs (Ogur and Amuca), and auditory and visual privacy, which was also only available in 2/4 of the HFs (Ogur and Aromo). 18 o However, there is 24 hour staff coverage in 3/4 of the HFs (Aromo, Barapwo, and Ogur). o Also, 3/4 of the HFs had electricity from the grid or a generator with fuel and functioning emergency communication (Aromo, Barapwo, and Ogur). o All of the HFs had a useable client latrine.  At MTE, all essential health infrastructure was present on the day of the survey in all 4 of the HFs o All 4 facilities now have water from a protected water source and auditory and visual privacy (both up from 2/4 at BL). o Also, 24 hour staff coverage is now available in all 4 HFs (up from 3/4 at BL) o However, electricity from the grid or a generator with fuel is down from 3/4 at BL (all HFs except Aromo) to only 1/4 at MTE (Ogur). 19 4. Supplies Supplies – Child BASELINE Indic. # Domain Indicator % HF with all elements Index Value (% avg. HF attainment) 4 CHILD Supplies - Child % HF with all essential supplies to support child health on day of the survey (accessible and working scale for child, accessible and working scale for infant, timing device for diagnosis of pneumonia, spoon/cup/jug to administer ORS) 1/4 40% Has functioning and accessible infant scale 2/4 Has functioning and accessible scale for children/adults 2/4 Has functioning timer or watch 2/4 Has pitcher for ORS 1/4 Has cup or spoon for ORS 1/4 MIDTERM Indic. # Domain Indicator % HF with all elements Index Value (% avg. HF attainment) 4 CHILD Supplies - Child % HF with all essential supplies to support child health on day of the survey (accessible and working scale for child, accessible and working scale for infant, timing device for diagnosis of pneumonia, spoon/cup/jug to administer ORS) 3/4 95% Has functioning and accessible infant scale 4/4 Has functioning and accessible scale for children/adults 3/4 Has functioning timer or watch 4/4 Has pitcher for ORS 4/4 Has cup or spoon for ORS 4/4  At BL only 1/4 of HFs (Barapwo) had all of the essential supplies to support child health, and the average HF attainment was 40% of these essential supplies.  ORS pitchers and spoons were only available in Barapwo, and scales and timers were available in only 2/4 (Barapwo and Amuca) of the facilities. 20  At MTE 3/4 of HFs (Aromo, Barapwo, and Ogur) had all of the essential supplies to support child health, and the average HF attainment was 95% of these essential supplies.  Amuca did not have a functional scale for children/adults Supplies – MNC BASELINE Indic. # Domain Indicator % HF with all elements Index Value (% avg. HF attainment) 4 Neo Supplies – HF % HF with all essential supplies to support maternal-newborn health present on day of the survey (partograph, vacuum extractor, resuscitation device, weighing scale, antibiotics and baby wraps) 0/4 25% Has functioning neonatal resuscitation equipment 0/4 Has functioning and accessible infant scale 3/4 Has functioning vacuum extractor 0/4 Has neonatal wraps for warming 0/4 Has partographs 2/4 MTE Indic. # Domain Indicator % HF with all elements Index Value (% avg. HF attainment) 4 Neo Supplies – HF % HF with all essential supplies to support maternal-newborn health present on day of the survey (partograph, vacuum extractor, resuscitation device, weighing scale, antibiotics and baby wraps) 0/4 30% Has functioning neonatal resuscitation equipment 3/4 Has functioning and accessible infant scale 3/4 Has functioning vacuum extractor 0/4 Has neonatal wraps for warming 0/4 Has partographs 0/4  At BL, none (0/4) of the HFs had all of the essential supplies to support maternal-newborn health available and the average HF attainment was only 25% of these essential supplies. o None of the facilities have neonatal resuscitation equipment, functioning vacuum extractors, or neonatal wraps for warming. o Only 2/4 of facilities (Aromo and Amuca) have partographs.  At MTE, results were essentially the same, with none (0/4) of the HFs having all of the essential supplies to support maternal-newborn 21 health available and the average HF attainment at only 30% of these essential supplies.  All except Barapwo now have neonatal resuscitation equipment and infant scales  None have functioning vacuum extractors or neonatal wraps for warming.  Aromo and Amuca no longer have functioning partographs, so no HFs now have functioning partographs Supplies – ANC BASELINE Indic. # Domain Indicator % HF with all elements Index Value (% avg. HF attainment) 4 ANC Supplies - ANC % HF with all essential supplies to support antenatal care present on day of survey (blood pressure machine, tetanus toxoid vaccine, hemoglobin reagents, syphilis testing kit, and albastix for protein) 0/4 42% Has functioning blood pressure equipment 2/4 Has hemoglobin testing reagents 1/4 Has syphilis testing kits 2/4 Has malaria test kits 2/4 Has urine albumin test strips 0/4 Has tetanus toxoid 3/4 FINAL Indic. # Domain Indicator % HF with all elements Index Value (% avg. HF attainment) 4 ANC Supplies - ANC % HF with all essential supplies to support antenatal care present on day of survey (blood pressure machine, tetanus toxoid vaccine, hemoglobin reagents, syphilis testing kit, and albastix for protein) 0/4 54% Has functioning refrigerator 4/4 Has functioning blood pressure equipment 3/4 Has hemoglobin testing reagents 1/4 Has syphilis testing kits 2/4 Has malaria test kits 1/4 Has urine albumin test strips 2/4 Has tetanus toxoid 4/4  At BL 0/4 of the HFs had all of the essential supplies to support antenatal care.  No facilities have albumin test strips, and only 1/4 of the HFs (Aromo) had syphilis testing kits and hemoglobin testing kits available. 22  Also, malaria test kits (Aromo and Amuca), blood pressure equipment (Ogur and Aromo), and syphilis testing kits are available in only 2/4 (Aromo and Amuca) of the facilities.  All HFs other than Barapwo had tetanus toxoid available.  At MTE, the categories for ANC Supplies have changed slightly (see above). However, none of the HFs have all of the essential supplies to support antenatal care.  Only Ogur has hemoglobin testing reagents or malaria test kits.  Ogur had all equipment except functional BP equipment. 5. Drugs Drugs – Child BASELINE Indic. # Domain Indicator % HF with all elements Index Value (% avg. HF attainment) 5 CHILD Drugs - Child % HF with all first line medications for child health present on day of the survey (ORS, oral antibiotic for pneumonia, first line oral antibiotic for dysentery, first line antiamalarial, vitamin A) 1/4 40% Has ORS packets 3/4 Has first line child pneumonia drug 1/4 Has first line dysentery drug 1/4 Has first line antimalarial 1/4 Has vitamin A 2/4 MTE Indic. # Domain Indicator % HF with all elements Index Value (% avg. HF attainment) 5 CHILD Drugs - Child % HF with all first line medications for child health present on day of the survey (ORS, oral antibiotic for pneumonia, first line oral antibiotic for dysentery, first line antiamalarial, vitamin A) 1/4 75% Has ORS packets 4/4 Has first line child pneumonia drug 1/4 Has first line dysentery drug 2/4 Has first line antimalarial 4/4 Has vitamin A 4/4  At baseline, 1/4 of HFs (Amuca) had all first line medications available, which is defined as ORS, a first line oral antibiotic for pneumonia, a first line drug for dysentery, a first line antimalarial, and Vitamin A. o This is due to the fact that while 1/4 of facilities had first line drugs for pneumonia (Barapwo) and 50 % of HFs had Vitamin A (Aromo and 23 Amuca), only 1/4 of HFs (Amuca) had first line medications also available for malaria and dysentery.  At MTE, while still only Amuca had all first line medications available, the average HF attainment rose from 40% to 70% o All facilities now have ORS, first line Antimalarials, and Vitamin A o Only Amuca has 1st line pneumonia drugs o Only Amuca and Ogur have 1st line dysentery drugs Drugs – MNC BASELINE Indic. # Domain Indicator % HF with all elements Index Value (% avg. HF attainment) 5 MNC Drugs - MNC % HF with all essential delivery & neonatal drugs present on day of survey (i.e., Oxytocin, antibiotics for newborn sepsis and eye infections) 0/4 25% Has antibiotics for newborn sepsis/pneumonia 1/4 Has neonatal eye ointment 1/4 Has Oxytocin 1/4 Has Nevirapine (in high HIV prevalence areas only) 2/4 MTE Indic. # Domain Indicator % HIV with all elements Index Value (% avg. HIV attainment) 5 MNC Drugs – HIV % HIV with all essential delivery & neonatal drugs present on day of survey (i.e., Oxytocin, antibiotics for newborn sepsis and eye infections) 0/4 42% Has antibiotics for newborn sepsis/pneumonia 1/4 Has neonatal eye ointment 2/4 Has Oxytocin 2/4 Has Nevirapine (in high HIV prevalence areas only) 2/4  At BL, none of the HFs had all of the essential delivery and neonatal drugs present on the day of the survey. In fact, all 3 essential MNC drugs were available in only 1 out of the four HFs each (antibiotics-Amuca; neonatal eye ointment-Ogur; and Oxytocin-Ogur).  At MTE, still none of the HFs had all of the essential delivery and neonatal drugs present on the day of the survey. The drugs were available at the following 24 locations-Amuca; neonatal eye ointment-Amuca and Ogur; and Oxytocin￾Barapwo and Amuca. Drugs – ANC BASELINE Indic. # Domain Indicator % HIV with all elements Index Value (% avg. HIV attainment) 5 ANC Drugs – ANC % HF with all essential ANC drugs (Tetanus toxoid, iron/folate, IPT, ITNs) 1/4 69% Has tetanus toxoid 3/4 Has iron / folate 3/4 Has antimalarial IPT 3/4 Has ITNs 2/4 MTE Indic. # Domain Indicator % HIV with all elements Index Value (% avg. HIV attainment) 5 ANC Drugs – ANC % HF with all essential ANC drugs (Tetanus toxoid, iron/folate, IPT, ITNs) 4/4 100% Has tetanus toxoid 4/4 Has iron / folate 4/4 Has antimalarial IPT 4/4 Has ITNs 4/4  At BL, only 2/4 HFs had all of the ANC drugs present on the day of the survey. Barapwo was missing 3 of the 4 drugs (Tetanus toxoid, Iron, and ITNs). Amuca was missing antimalarials. Only half of the HFs had ITNs (Aromo and Amuca).  At MTE, all HFs have all the essential ANC drugs. 25 . PROCESSES 6. Information Systems Information System – Child Baseline Indic. # Domain Indicator % HF with all elements Index Value (% avg. HF attainment) 6 CHILD Information System - Child % HF that maintain up-to-date records of sick U5 children (age, diagnosis, treatment) and for HF: have report in last 3 months and evidence of data use 0/4 25% MTE Indic. # Domain Indicator % HF with all elements Index Value (% avg. HF attainment) 6 CHILD Information System - Child % HF that maintain up-to-date records of sick U5 children (age, diagnosis, treatment) and for HF: have report in last 3 months and evidence of data use 0/4 80%  At BL, none of the HFs maintained up to date records on sick children under 5 years of age, or up-to-date monthly service data. 26 o The average facility had only 25% of the elements required for having up￾to-date records and monthly service data. o Ogur performed best with 80% of the elements in place. o HFs Aromo and Barapwo did not have any of the elements in place, while Amuca had only 20% of the elements in place. o Barapwo and Amuca used the data to create a wall chart, and additionally Barapwo used the data to create a graph and in discussions, but the other HFs did not use the data at all.  At MTE, still 0/4 of the HFs maintained up to date records on sick children under 5 years of age, or up-to-date monthly service data. o However, each facility now has 80% of the elements required for having up-to-date records and monthly service data. o No facility has complete information in the sick child register, which was the limiting factor. Information System – ANC BASELINE Indic. # Domain Indicator % HF with all elements Index Value (% avg. HF attainment) 6 MNC Information System - ANC % HF that maintain up-to-date records of antenatal care (TT, blood pressure, expected date of delivery) & deliveries (present & up to date) 2/4 63% An ANC register was observed 3/4 ANC register with complete delivery information, last 3 months 2/4 ANC register with complete TT information, last 3 months 2/4 ANC register with complete BP information, last 3 months 2/4 ANC register with entry in last 7 days 3/4 Delivery register was observed 3/4 Delivery register was up to date (entry in last 30 days) 3/4 MTE Indic. # Domain Indicator % HF with all elements Index Value (% avg. HF attainment) 6 MNC Information System - ANC % HF that maintain up-to-date records of antenatal care (TT, blood pressure, expected date of delivery) & deliveries (present & up to date) 1/4 75% An ANC register was observed 4/4 27 ANC register with complete delivery information, last 3 months 3/4 ANC register with complete TT information, last 3 months 3/4 ANC register with complete BP information, last 3 months 0/4 ANC register with entry in last 7 days 4/4 Delivery register was observed 4/4 Delivery register was up to date (entry in last 30 days) 4/4  At BL, 2/4 HFs (Aromo and Amuca) had up to date records on antenatal care. Aromo and Amuca had all of the elements in place, and Ogur had 2/4 of the elements in place, while Barapwo had none of the elements in place.  At MTE, only Amuca had up to date records on antenatal care. Aromo did not have up to date TT information as it did at BL. 7. Training BASELINE Indic. # Domain Indicator % HF with all elements 7 CHILD Training - Child Health % HF in which interviewed HW reported receiving in-service or pre-service training in child health in last 12 months 1/4 7 MNC Training - Maternal￾Neonatal Care % HF in which interviewed HW reported receiving in-service or pre-service training in maternal neonatal care in last 12 months 1/4 Interviewed HW received any MNC training in last 12 mo. 1/4 CHILD HEALTH Immunization training 0/4 Pneumonia case management training 0/4 Diarrhea case management training 1/4 Malaria case management training 0/4 ACT use training 0/4 ITN use training 0/4 Nutrition training 1/4 Breastfeeding promotion training 0/4 IMCI training 0/4 MNC IPT use training 1/4 Newborn care training 0/4 Post-partum care training 0/4 ANC training 0/4 Infection control training 0/4 28 AMTSL training 0/4 Ob / neonatal emergencies referral training 0/4 MTE Indic. # Domain Indicator % HF with all elements 7 CHILD Training - Child Health % HF in which interviewed HW reported receiving in-service or pre-service training in child health in last 12 months 3/4 7 MNC Training - Maternal￾Neonatal Care % HF in which interviewed HW reported receiving in-service or pre-service training in maternal neonatal care in last 12 months 3/4 Interviewed HW received any MNC training in last 12 mo. 4/4 CHILD HEALTH Immunization training 2/4 Pneumonia case management training 3/4 Diarrhea case management training 3/4 Malaria case management training 3/4 ACT use training 3/4 ITN use training 3/4 Nutrition training 3/4 Breastfeeding promotion training 3/4 IMCI training 3/4 MNC IPT use training 3/4 Newborn care training 2/4 Post-partum care training 1/4 ANC training 2/4 Infection control training 1/4 AMTSL training 1/4 Ob / neonatal emergencies referral training 1/4 29  At BL. only1/4 of the HFs (Ogur) had HWs who reported receiving in-service or pre-service training in both maternal neonatal care and child health in last 12 months. Training in MNC consists of training in MNC and at least one other of the trainings listed above in the MNC section. Training in child health consists of training in Child Health and at least one other of the trainings listed above in the child health section. Ogur received training in Diarrhea case management, nutrition, and IPT use.  At MTE, 3/4 of the HF (all except Barapwo) had HWs who reported receiving in￾service or pre-service training in both maternal neonatal care and child health in last 12 months. The various trainings are broken down in the above table. 8. Supervision BASELINE Indic. # Domain Indicator % HF with all elements 8 Supervision % HF that received external supervision at least once in the last 3 months (supervision included one or more of the following: checked records or reports, observed work, provided feedback, gave praise, provided updates, discussed problems)) 1/4 MTE Indic. # Domain Indicator % HF with all elements 8 Supervision % HF that received external supervision at least once in the last 3 months (supervision included one or more of the following: checked records or reports, observed work, provided feedback, gave praise, provided updates, discussed problems)) 3/4  At BL, only 1/4 of the HFs (Ogur) received any type of external supervision at least once in the 3 months prior to the survey. During this visit, the supervisor observed work in the HF, discussed problems, and checked drug supplies.  At MTE, 3/4 HFs received supervision (Barapwo did not). 30 PERFORMANCE 9. Utilization of Curative services The percentage of HF with > 1 sick child encounters per child under the age of 5 in Lira District was not able to be calculated, because no facilities had complete sick child registries available at the time of survey. 10. HW Performance (Assessment) BASELINE Indic. # Domain Indicator % HF with passing score Index Value (% avg. HF attainment) 10 CHILD HW Performance (Assessment) % patients in a facility for whom all 5 assessment tasks were done at least 80% of the time (check presence of general danger signs (including both vomiting and convulsions), assess feeding practices, assess nutritional status, check vaccination status) 0/4 21% MTE Indic. # Domain Indicator % HF with passing score Index Value (% avg. HF attainment) 10 CHILD HW Performance (Assessment) % patients in a facility for whom all 5 assessment tasks were done at least 80% of the time (check presence of general danger signs (including both vomiting and convulsions), assess feeding practices, assess nutritional status, check vaccination status) 0/4 68% In order for a HF to be considered as having key assessment tasks routinely performed, in greater than 80% (5 or 6 out of the 6 cases observed) of the encounters the HW must perform all of the key assessment tasks.  The baseline found that no facilities (0/4) contained HWs who were routinely performing all 5 key assessment tasks. o Furthermore, the average facility was performing only 21% of the key assessment tasks. o Ogur performed best by performing 63% of the assessment tasks, followed by Barapwo which performed 21% of their assessment tasks. Aromo and Amuca performed the worst, performing only 4% and 8% of their assessment tasks, respectively. o Assessment tasks which were assessed most frequently were the ability to feed or breastfeed, asking if the child has been vomiting everything, and checking for the presence of convulsions which were each asked 29% of the time. 31 o Assessment tasks which were assessed least frequently were checking the nutritional status and checking for immunizations on the child health card, which were each performed only 4% of the time.  At MTE, no facilities (0/4) contained HWs who were routinely performing all 5 key assessment tasks. o However, the average facility is now performing 68% of the key assessment tasks. o Only Amuca was able to check child heath cards, which they did 100% of the time, because mothers attending the other facilities did not bring child health cards with them to their visit. The HW asked for the HC at the other 3 HFs, but the mothers did not have them. This was the limiting factor with this indicator:  Aromo and Ogur completed 100% of the other tasks other than checking the HC for nutritional status and vaccinations so would have passed this indicator had health cards been available. Barapwo, in addition to the HC issue, did not ask about breast feeding on 2 occasions, so would have still failed.  Amuca, while able to check HCs, still failed because the HW forgot to ask about convulsions on 2 occasions. o A plan has been implemented at the HFs in which all mothers are to be told at each visit that they must bring their Child Health Cards to every visit, on every occasion. 10. HW Performance (Treatment) BASELINE Indic. # Domain Indicator % HF with all elements 11 CHILD HW Performance (Treatment) % HF where treatment is routinely appropriate(at least 80% of the time) to the diagnosis (for encounters in which at least one of the presenting problems was fever, breathing problem, or diarrhea) 1/4 MTE Indic. # Domain Indicator % HF with all elements 11 CHILD HW Performance (Treatment) % HF where treatment is routinely appropriate(at least 80% of the time) to the diagnosis (for encounters in which at least one of the presenting problems was fever, breathing problem, or diarrhea) 4/4  At BL, only 1/4 of the HFs (Aromo) provided treatment that is routinely appropriate to the diagnosis. The HW must diagnose and treat the sick child 32 correctly in greater than 80% (5 or 6 out of the 6 cases observed) of the encounters observed in order for a HF to be considered as providing correct treatment.  At MTE, all 4 facilities provided correct treatment appropriate to the diagnosis. 11. HW Performance (Counseling) BASELINE Indic. # Domain Indicator % HF with all elements 12 CHILD HW Performance (Counseling) % HF where caretakers whose child was prescribed an antibiotic, antimalarial, or ORS, correctly describe how to administer all prescribed drugs 0/4 MTE Indic. # Domain Indicator % HF with all elements 12 CHILD HW Performance (Counseling) % HF where caretakers whose child was prescribed an antibiotic, antimalarial, or ORS, correctly describe how to administer all prescribed drugs 0/4  AT BL, a very weak area in regards to HW performance was in the area of counseling the caretaker regarding the proper method of administering the medicines prescribed. None (0/4) of the HFs were routinely properly instructing caretakers in how to correctly administer drugs prescribed for their child.  At MTE, despite training in IMCI, this continues to be a challenge as still no HF routinely had caretakers on a routine basis that could describe how to administer drugs prescribed for their child properly.  It was determined that the HW is instructing the mothers too quickly, and the dispensers are not aiding by reiterating the instructions to the mothers. In addition, many mothers are illiterate, and the dispensers are writing the instructions on the medication bag instead of drawing pictures to illustrate the instructions. The project will work on these 3 areas to improve the communication between the HWs and the caretakers. 33 OPTIONAL INDICATORS 12. Optional Indicators BASELINE Area of Analysis Indic. # Domain Indicator % HF with all elements Index Value (% avg. HF attainment) INPUTS Opt1 Availability of Immunizations % HF with all nationally-mandated immunizations in stock on day of survey 1/4 44% Opt2 Availability of Guidelines % HF with all nationally-mandated guidelines for care of children available and accessible on day of survey 1/4 46% Opt3 Infection Control % HF with all infection control supplies and equipment on day of survey 0/4 68% PROCESSES Opt4 HF-Community Participation % HF with at least one method for community participation and at least one way to incorporate information 3/4 Opt5 Community Referral % HF that received at least one referral from CHW (VHT) in the last month 3/4 MTE Area of Analysis Indic. # Domain Indicator % HF with all elements Index Value (% avg. HF attainment) INPUTS Opt1 Availability of Immunizations % HF with all nationally-mandated immunizations in stock on day of survey 4/4 100% Opt2 Availability of Guidelines % HF with all nationally-mandated guidelines for care of children available and accessible on day of survey 2/4 83% Opt3 Infection Control % HF with all infection control supplies and equipment on day of survey 0/4 65% PROCESSES Opt4 HF-Community Participation % HF with at least one method for community participation and at least one way to incorporate information 4/4 Opt5 Community Referral % HF that received at least one referral from CHW (VHT) in the last month 4/4 34  At BL, Only 1/4 of the HFs (Amuca) had all of the required immunizations in stock (BCG, OPV, DPT, and MMR), and had all of the guidelines regarding the care of children available and accessible on the day of the survey. At midterm, all 4 HFs had all required immunizations  At BL, no facilities had all of the requisite infection control supplies. This was due to the fact that no facility had an adequate infectious waste disposal area. Also, chlorine or other disinfectants were present in only 2/4 of the HFs (Ogur and Barapwo), and only 2/4 of the facilities had evidence of adequate sharps disposal and infectious waste disposal practices (Aromo and Amuca). In addition, Ogur did not have gloves or needles present, and Aromo did to have syringes available. At MTE, this continues with no HFs having the requisite infection control supplies and no facility having an adequate infectious waste disposal areas or evidence of adequate infectious waste disposal practices. While Chlorine, gloves, soap, and sharps containers are available at all 4 HFs at MTE, only Barapwo and Ogur have needles and Aromo did not have syringes.  At baseline 3/4 of the facilities (Ogur, Aromo, and Amuca) had the combination of at least 1 method for community participation and at least one way to incorporate community information. At MTE this has improved to all 4 facilities.  At baseline all four facilities had at least one mechanism to elicit community participation, and at MTE this continues: Ogur, Aromo, and Amuca had community participation in HF management committee meetings, and Barapwo elicited community participation through the engagement of VHTs.  At BL, 3/4 (Ogur, Aromo, and Barapwo) of the HFs report that they had received at least one referral from VHTs in the last month. At MTE this has improved to all 4 of the HFs. CHAPTER 4 Action Plan/Conclusions/Recommendations BASELINE MIDTERM 0% 20% 40% 60% 80% 100% 120% 140% 160% Geographic Access Service… Staffing Infrastructure Supplies - Child Drugs - Child Availability of… Availability of… Infection Control Information… Training - Child… Supervision HF-Community… Community Referral Malaria Drug… ITN/LLIN Logistics Laboratory Assessment Treatment Counseling ACCESS INPUTS PROCESSES PERFORMANCE HF Child Health Balanced Score Card 0% 20% 40% 60% 80% 100% 120% Geographic Access Service Availability -… Staffing Infrastructure Supplies - Child Drugs - Child Availability of… Availability of… Infection Control Information System -… Training - Child Health Supervision HF-Community… Community Referral Malaria Drug (ACT)… ITN/LLIN Logistics Laboratory Assessment Treatment Counseling ACCESS INPUTS PROCESSES PERFORMAN HF Child Health Balanced Score Card 36 BASELINE MIDTERM ________________________________________________________________________ 0% 20% 40% 60% 80% 100% 120% 140% 160% Geographic Access Service Availability… Service Availability… Staffing Infrastructure Supplies - Neonatal Supplies - ANC Drugs - Neonatal Drugs - ANC Availability of… Infection Control Information… Training -… Supervision HF-Community… Malaria Drug… ITN/LLIN Logistics Laboratory HF MNC Balanced Score Card 0% 20% 40% 60% 80% 100% 120% Geographic Access Service Availability -… Service Availability -… Staffing Infrastructure Supplies - Neonatal Supplies - ANC Drugs - Neonatal Drugs - ANC Availability of… Infection Control Information System -… Training - Maternal-… Supervision HF-Community… Malaria Drug (ACT)… ITN/LLIN Logistics Laboratory ACCESS INPUTS PROCESSES HF MNC Balanced Score Card 37 The following is a summary and discussion of each of the 12 core indicators, along with a comparison to BL. These items have been discussed with the DHO and interventions aimed at improving performance in those areas that remain a challenge are being implemented at the HFs. The graphs above provide a visual summary of each of these areas, with a comparison of BL to MT. ACCESS (INPUTS) 1. Service Availability:  At baseline, 0/4 of the HFs offered all 3 basic child health services o No facilities saw patients 30 days per month (including outreach). However, all of the facilities provided sick child services at a minimum of Monday-Friday, for a total of 20 hours per week  At MTE, 1/4 of the HFs, Amuca, now offer child health services 30 days per month (including outreach). Aromo and Ogur offer services 28 days per week. Barapwo offers services only M-F (20 hours per week) o Growth Monitoring:  At BL only 2 of the 4 HFs (Ogur and Amuca) offered growth monitoring.  At MTE, only Amuca offered GM as mothers did not bring their health cards to the other facilities so they were not able to be checked. Activities have been put in place at all HFs to encourage all mothers to bring their children’s health card with them to every visit. o Immunization:  At baseline all 4 HFs offered immunization services through the facility and/or outreach.  At MTE all 4 continue to offer immunization services o ANC Services:  At baseline and MTE all HFs in Lira District offer ANC services, o Delivery Services:  Barapwo still does not provide delivery services as it has just recently (2008) been upgraded from a level 2 facility to a level 3 facility and has no maternity unit. However, Barapwo is presently being moved to a new, adjacent site and will have a functional maternity unit for delivery services. 2. Staffing: This indicator has been changed from the original RHFA staffing indicator, which was: “% of HFs with at least one provider meeting the country definition as qualified to provide curative care for children is present on day of survey”. All HFs would have met the old indicator, both at BL and MTE, because at least one qualified provider was present at each facility. However, the new indicator determines the number of each 38 type of staff, and determines if they are all present on the day of the survey. o Therefore, at BL, 0/4 of the HFs met this requirement, with the average HF attainment at 48% (meaning the average HF had 48% of its qualified staff present). o At MTE 1/4 (Amuca) had all staff present. o Attendance has been an issue in Lira. The project is meeting with the DHO to discuss the RHFA results, and the project will work with the DHO to put more strict attendance policies in place. 3. Infrastructure:  At BL, all essential health infrastructure was present on the day of the survey in only 1/4 of the HFs (Ogur), with the average HF attainment at 63%. o The limiting factors were: emergency transportation, which was not available in any of the HFs, an improved, protected water source, which was only available in 2/4 of the HFs (Ogur and Amuca), and auditory and visual privacy, which was also only available in 2/4 of the HFs (Ogur and Aromo). o However, there is 24 hour staff coverage in 3/4 of the HFs (Aromo, Barapwo, and Ogur). o Also, 3/4 of the HFs had electricity from the grid or a generator with fuel and functioning emergency communication (Aromo, Barapwo, and Ogur). o All of the HFs had a useable client latrine.  At MTE, all essential health infrastructure was present on the day of the survey in all 4 of the HFs o All 4 facilities now have water from a protected water source and auditory and visual privacy (both up from 2/4 at BL). o Also, 24 hour staff coverage is now available in all 4 HFs (up from 3/4 at BL) o However, electricity from the grid or a generator with fuel is down from 3/4 at BL (all HFs except Aromo) to only 1/4 at MTE (Ogur). 4. Supplies: a. Supplies – Child:  At BL only 1/4 of HFs (Barapwo) had all of the essential supplies to support child health, and the average HF attainment was 40% of these essential supplies.  ORS pitchers and spoons were only available in Barapwo, and scales and timers were available in only 2/4 (Barapwo and Amuca) of the facilities. 39  At MTE 3/4 of HFs (Aromo, Barapwo, and Ogur) had all of the essential supplies to support child health, and the average HF attainment was 95% of these essential supplies.  Amuca did not have a functional scale for children/adults b. Supplies-MNC:  At BL, none (0/4) of the HFs had all of the essential supplies to support maternal-newborn health available and the average HF attainment was only 25% of these essential supplies. o None of the facilities have neonatal resuscitation equipment, functioning vacuum extractors, or neonatal wraps for warming. o Only 2/4 of facilities (Aromo and Amuca) have partographs.  At MTE, results were essentially the same, with none (0/4) of the HFs having all of the essential supplies to support maternal-newborn health available and the average HF attainment at only 30% of these essential supplies.  All except Barapwo now have neonatal resuscitation equipment and infant scales  None have functioning vacuum extractors or neonatal wraps for warming.  Aromo and Amuca no longer have functioning partographs, so no HFs now have functioning partographs  The project will discuss the lack of vacuum extractors with the DHO when we present the results of the MT HFA. c. Supplies-ANC:  At BL 0/4 of the HFs had all of the essential supplies to support antenatal care.  No facilities have albumin test strips, and only 1/4 of the HFs (Aromo) had syphilis testing kits and hemoglobin testing kits available.  Also, malaria test kits (Aromo and Amuca), blood pressure equipment (Ogur and Aromo), and syphilis testing kits are available in only 2/4 (Aromo and Amuca) of the facilities.  All HFs other than Barapwo had tetanus toxoid available.  At MTE, the categories for ANC Supplies have changed slightly (see Results section, 4. Supplies-ANC for the new category list). However, none of the HFs have all of the essential supplies to support antenatal care.  Only Ogur has hemoglobin testing reagents or malaria test kits.  Ogur had all equipment except functional BP equipment.  Hemoglobin testing reagents and malaria test kits are in the budget of the DHO. It will be discussed with the DHO the need to have a reliable flow of these items. The malfunctioning BP equipment has been replaced. 40 5. Drugs: a. Child:  At baseline, 1/4 of HFs (Amuca) had all first line medications available, which is defined as ORS, a first line oral antibiotic for pneumonia, a first line drug for dysentery, a first line antimalarial, and Vitamin A. o 1/4 of facilities had first line drugs for pneumonia (Barapwo) and 50 % of HFs had Vitamin A (Aromo and Amuca), but only 1/4 of HFs (Amuca) had first line medications also available for malaria and dysentery.  At MTE, while still only Amuca had all first line medications available, the average HF attainment rose from 40% to 70% o All facilities now have ORS, first line Antimalarials, and Vitamin A o Only Amuca has 1st line pneumonia drugs o Only Amuca and Ogur have 1st line dysentery drugs b. MNC:  At BL, none of the HFs had all of the essential delivery and neonatal drugs present on the day of the survey. In fact, all 3 essential MNC drugs were available in only 1 out of the four HFs each (antibiotics￾Amuca; neonatal eye ointment-Ogur; and Oxytocin-Ogur).  At MTE, still none of the HFs had all of the essential delivery and neonatal drugs present on the day of the survey. The drugs were available at the following locations: antibiotics for newborn sepsis￾Amuca; neonatal eye ointment-Amuca and Ogur; and Oxytocin￾Barapwo and Amuca. c. ANC:  At BL, only 2/4 HFs had all of the ANC drugs present on the day of the survey. Barapwo was missing 3 of the 4 drugs (Tetanus toxoid, Iron, and ITNs). Amuca was missing antimalarials. Only half of the HFs had ITNs (Aromo and Amuca).  At MTE, all HFs have all the essential ANC drugs. The project is working with the DHO as they have instituted a new procurement system for drugs that is not based on forecasting. The project is attempting to ensure that a reliable flow of 1st line medications for child, MNC, and ANC care is available to each HF. One possibility we will offer to the DHO is that they develop a system to monitor stocks in each of the 4 HFs and transfer overstocks of drugs or equipment that occur in any 1 facility to the other HFs that have a need for these items. 41 PROCESSES 6. Information Systems a. Child:  At BL, none of the HFs maintained up to date records on sick children under 5 years of age, or up-to-date monthly service data. o The average facility had only 25% of the elements required for having up-to-date records and monthly service data.  At MTE, still 0/4 of the HFs maintained up to date records on sick children under 5 years of age, or up-to-date monthly service data. o However, each facility now has 80% of the elements required for having up-to-date records and monthly service data. o No facility has complete information in the sick child register, which was the limiting factor. b. ANC:  At BL, 2/4 HFs (Aromo and Amuca) had up to date records on antenatal care. Aromo and Amuca had all of the elements in place, and Ogur had 2/4 of the elements in place, while Barapwo had none of the elements in place.  At MTE, only Amuca had up to date records on antenatal care. Aromo did not have up to date TT information as it did at BL. During IMCI supportive supervision and mentoring MTI staff will provide refresher training in properly completing the registers. 7. Training:  At BL. only1/4 of the HFs (Ogur) had HWs who reported receiving in￾service or pre-service training in both maternal neonatal care and child health in last 12 months. Ogur received training in Diarrhea case management, nutrition, and IPT use.  At MTE, 3/4 of the HF (all except Barapwo) had HWs who reported receiving in-service or pre-service training in both maternal neonatal care and child health in last 12 months. 8. Supervision:  At BL, only 1/4 of the HFs (Ogur) received any type of external supervision at least once in the 3 months prior to the survey. During this visit, the supervisor observed work in the HF, discussed problems, and checked drug supplies.  At MTE, 3/4 HFs received supervision (Barapwo did not). 42 All HF staff have received training, but there has been a turnover in staff at the Barapwo HF. MTI will train and supervise this new staff. PERFORMANCE 9. Utilization of Curative services: The percentage of HF with > 1 sick child encounters per child under the age of 5 in Lira District was not able to be calculated, because no facilities had complete sick child registries available at the time of survey. 10. HW Performance (Assessment):  The baseline found that no facilities (0/4) contained HWs who were routinely performing all 5 key assessment tasks required for this indicator. o Furthermore, the average facility was performing only 21% of the key assessment tasks. o Ogur performed best by performing 63% of the assessment tasks, followed by Barapwo which performed 21% of their assessment tasks. Aromo and Amuca performed the worst, performing only 4% and 8% of their assessment tasks, respectively. o Assessment tasks which were assessed most frequently were the ability to feed or breastfeed, asking if the child has been vomiting everything, and checking for the presence of convulsions which were each asked 29% of the time. o Assessment tasks which were assessed least frequently were checking the nutritional status and checking for immunizations on the child health card, which were each performed only 4% of the time.  At MTE, no facilities (0/4) contained HWs who were routinely performing all 5 key assessment tasks. o However, the average facility is now performing 68% of the key assessment tasks. o Only Amuca was able to check child heath cards, which they did 100% of the time, because mothers attending the other facilities did not bring child health cards with them to their visit. The HW asked for the HC at the other 3 HFs, but the mothers did not have them. This was the limiting factor with this indicator:  Aromo and Ogur completed 100% of the other tasks other than checking the HC for nutritional status and vaccinations so would have passed this indicator had health cards been available. Barapwo, in addition to the HC issue, did not ask about breast feeding on 2 occasions, so would have still failed.  Amuca, while able to check HCs, still failed because the HW forgot to ask about convulsions on 2 occasions. 43 11. HW Performance (Treatment):  At BL, only 1/4 of the HFs (Aromo) provided treatment that is routinely appropriate to the diagnosis.  At MTE, all 4 facilities provided correct treatment appropriate to the diagnosis. 12. HW Performance (Counseling):  AT BL, none (0/4) of the HFs were routinely properly instructing caretakers in how to correctly administer drugs prescribed for their child.  At MTE, despite training in IMCI, this continues to be a challenge as still no HF routinely had caretakers on a routine basis that could describe how to administer drugs prescribed for their child properly.  It was determined that the HW is instructing the mothers too quickly, and the dispensers are not aiding by reiterating the instructions to the mothers. In addition, many mothers are illiterate, and the dispensers are writing the instructions on the medication bag instead of drawing pictures to illustrate the instructions. The project will work on these 3 areas to improve the communication between the HWs and the caretakers. During Supportive Supervision and Mentoring visits the staff will provide, using the WHO IMCI monitoring checklist, refresher trainings for each staff in the specific components of IMCI that are required to properly do their jobs. CONCLUSIONS The Lira district CSP will continue to improve the lives of children and families in Lira District. The Midterm RHFA identified several areas that have improved since the BL assessment and others which continue to need improvement. The project will create an Action Plan to address those areas in need of improvement to ensure that these plans are included in 3rd and 4th year planning for the project. 44 Annex 1 Training Of the Survey Team The training of the survey team requires four days. The main objectives of the training are to discuss the purpose of the survey and the resulting information; discuss the logistics of the survey; review and practice each of the forms; and practice administering these forms in the facility setting. A health facility that is near the training site and which was not randomly chosen for the HFA will be used for the field test. The schedule10 below combines a discussion of each form with actual facility based practice in collecting the information needed to complete the form. Day Activities 1 AM: Opening & General Information Opening  Introduction of the participants  Administrative information General information  Purpose of the survey  Training objectives  Survey protocol and techniques  Introduction of Participant Guidelines  Clarification of participant expectations or concerns PM: Introduction to first two forms: Clinical Observation & Sick Child Clinical Observation - Sick Child  Review the instrument  Role play Caretaker Exit Interview – Sick Child  Review the instrument  Role play 2 AM: Health facility visit for Clinical Observation and Caretaker Exit Interview  Visit to health facility for practice of Clinical Observations and Exit Interviews  Debriefing of the health facility visit PM: Intro to Health Worker Interview and Health Facility Checklist Health Worker Interview  Review the instrument  Role play Health Facility Checklist  Review the instrument  Role play 3 AM: Health facility visit for HW Interview and Health Facility Checklist 10 Training schedule provided by CSTS; R-HFA short instruction 12-09-07; R-HFA Version 2.1; CSTS 2007; http://www.childsurvival.com/rhfa_1.cfm 45  Visit to health facility to practice the HW interview and using the Health Facility Checklist  Debriefing of health facility visit PM: Sampling Health Facility and Data Analysis Sampling health facilities in districts  Explanation of how health facilities were sampled in each district  Reviewing list of health facilities sampled and to be visited during the assessment Analysis of R-HFA data  Analysis of data at the health facility level. Identifying strengths and areas of needed improvement  Analysis of data at the district level. Identifying areas of needed improvement 1 Annex 5: Evaluation Team Members and their Titles External Independent Consultant: Judiann McNulty, DrPH MTI Headquarters, Portland, Oregon: Mary Helen Carruth, Child Survival Health Specialist Africa Regional Staff, Kampala, Uganda: Anna Summer, Africa Health Advisor MTI Uganda, Kampala Office: Alula David, Quality Assurance and Resource Development Director Awio Peter, Driver MTI Uganda, Lira Office: Akulo Lydia, Program Manager Odur Joel, Maternal & Child Health Coordinator Okello Joel, Monitoring & Evaluation Officer Apiyo Christine, Maternal & Child Health Mentor Apili Harriet, VHT Mobilizer Achan Doreen Susan, VHT Mobilizer Ajok Susan, VHT Mobilizer Omodo James, VHT Mobilizer Ojungu Tobias, VHT Mobilizer Ojok Nixon VHT Mobilizer Opio Peter Ssematimba, Driver Olyet Moses, Driver Annex 6 Mid-Term Evaluation Early Childhood Development Component Lira Child Survival Project in Uganda Conducted by Hands to Hearts International in collaboration with MTI in Lira, Uganda Report by: Laura Peterson, MA, Executive Director, Hands to Hearts International Introduction & Summary Early Childhood Development (ECD) training was included in the Lira Child Survival Project in Uganda to raise parents/caregiver’s awareness of their child’s health and development and instruct them how to improve these with their day-to-day interactions with their children and by better accessing health services. The ECD component also targeted health workers to enable them to be more supportive (less harsh) with parents, and build their overall base of knowledge. Hands to Hearts International’s (HHI) ECD training was piloted and implemented in Kampala for five months prior to formally launching efforts with MTI in Lira. This was done independently by HHI in order to learn the local customs, attitudes, beliefs and practices and to train a national HHI staff team to support this CSP. HHI’s national trainer was a native of Lira, spoke the local language and had a solid background in both ECD and community trainings with international NGOs. During the Detailed Implementation Plan development process in early 2010, HHI worked closely with MTI conducting focus groups with caregivers, village health teams (VHTs), health center staff and DOH, to understand the variety of factors and many challenges impacting caregivers and their young children in the Lira area. By late 2010 the ECD program was fully rolled out through-out a defined intervention area. This report summarizes findings from a qualitative evaluation of the ECD program, conducted in February 2012. The evaluation team conducted 21 Focus Groups with project beneficiaries. Participants included: six Health Center (HC) staff who led the ECD training of trainers (TOT) and who regularly gave ECD messages at their clinics; 40 VHTs and Peer Educators (PEs) who led the ECD trainings in their communities and continued to offer ECD information and support; 58 parents who attended formal ECD trainings; and 58 parents who did not receive ECD training. A quantitative survey was conducted in January 2012 to access progress in reaching the project indicators for cognitive, linguistic and social stimulation. Implementation In 2010 MTI CSP staff was trained to lead HHI’s full 16-hour training on early childhood development (ECD). This occurred over 3 separate visits from HHI staff and included the basic 2 2-day HHI technical training, a 3-day training of trainers (TOT) and a final TOT (expanded to 5 days) for MTI and 19 HC staff. Next MTI staff worked with the HC staff to modify and simplify HHI’s curriculum, they all took notes on this independently and agreed to lead the HHI training from this format. No new manual was created. This group then led a modified, simplified 3-day TOT trainings for 300 VHTs and PEs. These abbreviated TOTs were 15-18 hours and focused on teaching the technical content of ECD, less so on the facilitation skills to be effective ECD trainers. The shortened trainings were an outcome of budget constraints and overly ambitious program reach. This level of training was inadequate for participants with low levels of education with no ECD background. The VHTs/PEs then led a 5-week (one afternoon per week) ECD series of trainings in their 145 villages in late 2010. These were coordinated by Parish leaders who invited 30 participants per village. The trainings proved very popular and it’s estimated that an average of 60+ people participated in each training. Each ECD lesson lasted about 4 hours and included additional information on ICCM, and other messages promoted by local projects the VHTs and PEs were involved in, such as the creation of pit latrines and hand washing stations. The ECD trainings were led one time. MTI reported that 4,507 people participated, 37% were male. ECD messages continued to be widely distributed at the HCs and that 32,767 mothers heard ECD messages at immunization outreach during 2011. Strengths MTI’s CSP team had tremendous enthusiasm and commitment to the ECD initiative. Their determination and hard work led to the impressive reach of the ECD trainings, rolled out in 145 villages for 4,507 caregivers. Likewise, the HC staff, the VHTs and PEs also demonstrated great enthusiasm for the ECD education, sharing it far beyond the formal ECD trainings that they gave in their villages. HC staff in Bar Apwo, Ogur and Aromo estimated that they had given ECD messages to 60-85% of patients and that for each of them, this totaled anywhere from 500 - 1,000 caregivers per health worker. Of the 40 VHTs/PEs interviewed, they estimated that they each had an average of 62 participants (more than twice the number invited to attend) show up for every ECD lesson, and additionally that beyond the formal trainings, they each shared ECD lessons with another 88 caregivers. All shared great pride in now being seen as a supportive resource to caregivers and many stated that community members acknowledged their contributions. Nurse Apila at Aromo HC3 reported, “Now parents bring me gifts of sugar and grains to thank me for saving their baby’s life. I am very proud.” HC staff at Bar Apwo and Aromo both report that prior to ECD lessons, there were 10-12 children per month brought in in dire condition after traditional healers treated them by extracting teeth, or cutting the uvula. Since the ECD trainings over one year ago, they have not seen a single case of this in their clinics. 3 Challenges The ECD outreach plan was commendable, but overly ambitious in its scale. While the ECD component was very popular with all involved, and resulted in impressive outcomes, the program suffered from inadequate follow through and overambitious plan with budget constraints. This led to unfortunate outcomes: insufficient training for VHTs/PEs; limited ECD to only one-round of trainings in villages; invitees were chosen by Parish leaders and may have targeted married parents thereby missing the most vulnerable caregivers; ECD trainings were only held once, and there was no follow-up planned. Oddly, ECD also suffered from popularity (too many people voluntarily attended). This hindered the transfer of knowledge, as trainings overflowed with 60+ participants. Though the knowledge did have far reaching ripples, as many of those trained went on to share ECD lessons with others. Both ECD trained parents and HC staff disseminated ECD information into the control area, thus disrupting clear boundaries for evaluation. The participant numbers reported for ECD trainings and messages were not tracked accurately. At trainings only invitees were tracked for attendance, while many more participated. Thus the numbers were underreported. The number of mothers who received ECD messages was the number who participated in immunization clinics, this may double or triple count some, while the mothers who come after or left before the ECD messages should not be counted at all. A major challenge faced in this initiative was the very limited involvement and a lack of hands￾on technical support due to budgetary constraints and planning gaps. This contributed to the problem with poor communication between HHI and the CSP team, with changes being made in Lira that HHI was unaware of, and not able to provide support for, or guidance to. The ECD messages were intended to be integrated into the overall project and the ECD trainings were a mixture of HHI’s ECD curriculum, ICCM messages and other community priorities. This makes it difficult to determine the impact of ECD in isolation. Quantitative Findings Objective/ Result 2: Improved health (C-IMCI) and child care (ECD) behaviors among mothers of children <5 years Indicator Baseline Value EOP Target LQAS 2011 Comments of MTE evaluator ECD % of mothers of children aged 0-23 months who provide cognitive stimulation to their child in the form of games such as “where are your eyes”, etc. 38.0% 80% 68.74% Significant increase. This reflects diffusion far beyond the groups of parents in intervention areas who participated in ECD sessions. % of mothers of children aged 0-23 months who told their child a story, sang a song, or 22.7% 75% 40.06% Significant improvement. This reflects diffusion far 4 spent time naming objects for child at least 2 times in the past week beyond the groups of parents in intervention areas who participated in ECD sessions. % of mothers of children aged 0-23 months who report that they talk or sing to the child while feeding the child 57.7% 80% 65.36% No change. The question in the survey may not have been well￾understood or well￾worded. Qualitative Findings Health Clinic Staff (6) HC staff were enthusiastic promoters of ECD messages. They regularly shared messages with their clients about baby massage, breastfeeding, nutrition, baby cues, language and cognitive development and the importance of love and affection. They credited the ECD trainings with improved relations between staff and parents, increased use of HC services, and a decrease in using traditional healers. They also attributed outcomes such as: improved child and hygiene; greater affection between parent and child, with parents being more communicative and gentle with their child. VHTs and PEs (40) VHTs and PEs provided the front line of training on ECD in their villages. They were the cause of great changes in the attitudes and behaviors of those they trained. They built trust and confidence with community members and often provided counsel which led to parents taking their children to the HC for treatment. VHTs/PEs reported noticeable improvements were made in hygiene, nutrition, willingness to seek health care and immunizations, and a decrease in corporeal punishment. Mothers demonstrated improved breastfeeding practices, not discarding colostrum and providing more frequent and responsive feeding. “Malnutrition is down because of ECD messages on nutrition and hygiene” (Amuca). Greater love/bonding between parents and children was reported, as was improved overall health, cognitive and physical development. Reports on increased health seeking behaviors were unanimous. VHTs/PEs noted improved cognitive development after application of ECD lessons were applied on youngest children, parents report these children are “brighter” in comparison to older siblings. Improved physical development and lower cases of rickets (Aromo) were credited to One success story is that a mother in Ogur who learned the baby massage practiced on her neighbor’s son who has a disability that kept the knee pulled up to his chest and he couldn’t crawl. By giving massage, the child can now crawl. Now the father of child has learned and is continuing the massage. 5 baby massage. PEs have themselves made changes, spending more time with their children and giving them more love and affection. In Ogur the ECD education has rippled with ECD trained mothers teaching others. ECD Trained Caregivers (58 mothers/fathers) The basic themes from ECD trained parents were: greater awareness of health and development; more love/improved relationships; greater comfort with health services; greater intention and action to improve child health/development; fathers increased their role in caregiving; less alcoholism; and decreased family violence. Typical comments from ECD trained parents were: “because of the good relationship I have established between me and my child she now understands me and listens to me and above all loves me more than before”, “children do not fear us anymore”, and “whenever my child hears the sound of my bicycle while arriving home, she runs out to greet me.” One father indicated that the ECD trainings showed him that he had true value as a caregiver, something he had not seen before. This realization led him to come home at nights and not stay out drinking. At home he enjoyed the affection his child had for him. A significant finding was that both fathers and mothers reported that more of the family income is now spent on children, such as sending them to school, buying books/pens and new clothes. Historically this money was spent by the fathers on alcohol and/or cigarettes. Non-ECD Trained Caregivers (58 mothers/fathers) Parents in control groups reported doing many of the same things intervention parents do, such as singing to children, playing with them, etc. However, they seemed to be doing so out of habit, without intention or thinking about the bigger, longer-term effects of those actions on their children’s health or development. A surprising finding was the level of similarities found between the control and intervention parents, though there were distinct differences in two arenas. First the ECD trained parents seemed to be intentional and active in taking up more positive practices, such as investing more time with their children, stopping drinking, stopping violence (parent/child and parent/parent), digging latrines, and providing more varied nutrition. Second, the quality of relationship An ECD trained father taught his son some simple sign language. The baby can now communicate when he needs to go toilet, making him easier to care for. This baby is now comfortable with the father, freely approaches him and likes spending time with him. Father reported that with his first two children he would turn them away, send them to the mother, and he beat them. He does not beat this youngest child and is so proud of the relationship he has with him, now the older children are now growing closer to him too. Three of the four interview groups heard that fathers have decreased their drinking and this has led to less violence to entire family. 6 between parent/child was noted repeatedly by ECD parents as well as the PEs/VHTs and the HC staff all remarking on this as a significant. A few parents identified meeting a child’s emotional needs as important, such as “embracing child, asking about school.” Fathers are reported to have minimal involvement in children’s direct care, as is common in traditional gender roles and as was found in the baseline surveys. Fathers did not see their relationship with the child as central, rather providing for financial needs to care for child. Conclusions & Recommendations The ECD innovation was a valuable addition and contributed to the success of this CSP. It contributed to the improved relations between caregivers and health workers, an increased use of HCs, among a host of other outcomes already detailed. However there are a variety of ways to strengthen the impact of this intervention. Recommendation 1: The CSP team received inadequate hands on support at the time of the first community based training. The execution of the ECD training began to deviate from the original plan from this point on, and led the initiative to overreach its capacity, under train the local leaders and only provide the ECD trainings once. A detailed plan for roll-out of ECD trainings is needed to achieve the ECD objectives. This would include:  A plan for complete TOT training (minimum 5-days) for all HCs and new PEs/VHTs o 2 days to participate in regular and complete HHI training o 3 days TOT to learn how to be an ECD Trainer o Returning VHTs/PEs need the 3 day TOT as a refresher  ECD roll-out plan – where, when, by who, and how? o The next training roll-out should be at parish level  Tracking tools for attendance and individual completion of training as well as on-going ECD messages given at HCs  Plan for supervision and on-going support, with realistic budget estimates  Plan for on-going support and supervision to HC staff and VHTs/PEs and tracking tools to monitor these. Recommendation 2: An adequate budget must be obtained, applied strategically and monitored. While the ECD initiative was not funded by USAID, it was funded by required matching funds from MTI. The ECD component either needs greater funding, or to scale back the outreach to a smaller population, providing greater information and support to all involved. The CSP team agreed to focus the intervention on Aromo. Recommendation 3: HHI’s ECD lessons were modified and simplified by MTI. The abbreviated training needs to be formally documented and submitted to HHI for review. The ECD trainings included a mix of IMMC messages and a variety of other messages. The IMMC messages may correlate well with specific ECD messages, particularly HHI’s lesson on Health, Hygiene, Nutrition and Safety, which incorporates and reinforces many of the overarching CSP 7 messages. If ECD and IMMC are to be given at the same training, MTI must plan and prepare trainers for this. Recommendation 4: One factor that may have skewed this qualitative data positively is that every parent interviewed (control and intervention) was married, which is a protective factor and is not representative of general family composition. This does not seem representative of the general population. “Young mothers” were often referred to when talking about who continues in negative caregiving practices. Invitations to ECD trainings should target the most vulnerable caregivers. Recommendation 5: ECD trainings suffered from the wonderful problem of popularity, with 60+ participants per training. The massive group size undermines participant comprehension. Ideally trainings would have less than 20 participants. ECD trainings will be given twice in each community during the second half of this CSP. Recommendation 6: Health Centers can be asked to support, and regularly supervise PEs/VHTs, but there must be a plan for this, they must know what the expectations are for what they should do, how often, how to track it and who to report to. The HCs must be supported by MTI field staff. Recommendation 7: HHI is ready and interested to be of greater support to the success of this CSP. It is recommended that the CSP team identify how they wish to be supported and submit a request for appropriate funding to contract this. Also, the CSP team is encouraged to expand their communication, informing MTI HQ and HHI of their progress, challenges, adjustments, to share stories from the field and to ask for greater support as needed. 8 Annex: Focus Group Participation Group Location # People % Formal ECD lessons attended? % rec’d informal ECD lessons? Linguistic actions? Cog/Soc/ Emo Feeding? Health Clinic Bar Apwo 2 Health Clinic Aromo 1 Health Clinic Amucha 2 Health Clinic Ogur 1 VHTs/PEs Ogur HC4 7 VHTs/PEs Aromo HC3 9 VHTs/PEs Amuca 11 VHTs/PEs Aromo HC3 7 VHTs/PEs Ogur HC4 6 ECD Caregivers Acetlela, Ogur 10 83% 73% 98% 100% ECD Caregivers Karedonglac, Ogur 10 59% 14% 98% 98% ECD Caregivers Aloc, Aromo 9 82% 67% 90% 100% ECD Caregivers Akolodong, Amuca 9 69% 51% N/A 86% ECD Caregivers Wigweng, Aromo 10 83% 66% 100% 70% ECD Caregivers Te-Okole, Lira 10 74% 63% 83% 100% Non-ECD Caregivers Tegweng, Aromo 10 40% 50% Non-ECD Caregivers Ogeo B, Aromo 11 90% 90% Non-ECD Caregivers Okano-Ipero, Ogur 7 100% 30% Non-ECD Caregivers Te-Imat Ogur 10 100% 100% Non-ECD Caregivers Anai, Wigweng 10 100% 0% Non-ECD Caregivers Olaka Anex, Ogica B, Lira 10 100% 10% Total = 172 Annex 7: Overall Score Sheet and Action Plan Organizational Capacity Assessment MTI Uganda January 2010 Overall Score Sheet Section Sub-section Score Section Tally Average Score USG Score Governance Vision/Mission 3.27 2.9 Organizational Structure 2.6 Board Composition and Responsibility 2.37 Legal Status 3.9 Succession Planning 2.44 Administration Office Policie, Procedures and Systems 3.18 2.9 Travel Policies and Procedures 2.4 2.4 Procurement 2.88 2.88 Fixed Assets Control 3.33 3.33 Human Resources Management Job Descriptions 3.4 2.7 Recruitment and Retention 2.7 Staffing Levels 2.9 Personnel Policies 2.81 2.81 Staff Time Management 2.2 2.2 Staff Salaries and Benefits Policy 2.5 Staff Performance Management 2.9 Volunteers/Interns 2.3 Financial Management Financial Systems 3.4 3.5 Financial Controls 3.3 Financial Documentation 3.54 Audits 3.9 3.9 Financial Reporting 3.3 3.3 Organizational Management Strategic Planning 1.87 Workplan Development 3.6 3.6 Change Management 2.54 3 Knowledge Management 3.36 Stakeholder Involvement 3.4 New Opportunity Development 2.9 Leadership and Team Dynamics Communication 3 Decision Making 3.18 3.09 Program Management Donor Compliance 4 3.0 4 Sub-grant Management Technical Reporting 3.14 Transformational Development 2.72 Community Involvement 3.9 Culture and Gender 3.18 Project Performance Management NPI Program Implementation Status 3.1 Field Oversight 3 Standards 3.1 Supervision 3.2 Monitoring and Evaluation 2.9 2.9 Quality Assurance 3.3 AVERAGE Organizational Capacity Score 3.0 Average USG Grant Implementation Capacity Score 3.13 Average Organizational Capacity Score without USG Sub￾sections 3.0 MTI Uganda OCA ACTION PLAN Issues Action Time frame UPDATE February 2012 Succession Planning 1. CD to bring member of management team to lobby and advocate for funding 2. Establish delegation policy 3. Provide equal opportunity for managers to serve as designated officer in charge 4. Create awareness among staff and ensure delegation is communicated to staff 5. Build capacity of management team; invite/hire reputable consultants to present to and/or train the management team; managers attend short courses e.g. U.M.I. and L.D.C. When opportunity permits Ongoing Ongoing Ongoing While delegation is practiced and an OIC is appointed as the situation arises, an overall plan and policy is yet to be developed. Staff Salary & Benefits Policy 1. Adopt a salary and benefits policy 2. Revise the annual salary increase policy 3. Make uniform health insurance policies in all project sites. 4. Provide medical insurance for immediate family members 5. Streamline/clarify sick and maternity leave policies 6. Set standards on bereavement support 7. Review and disseminate end of project or end of contract policy 8. Conduct assessments of staff capacity to identify gaps 9. Approve funding for staff development; utilize consultants for professional development activities April 2010 & ongoing MTI U drafted new policies that clarified benefits but due to a delay in a field visit by MTI HQ’s HR manager, policies will be finalized during that visit in May 2012. An assessment of staff capacity by the MTI U HR manager is currently underway. Strategic Planning 1. Request guidelines from HQ for developing SP 2. Develop a plan to develop a SP; identify external support needed from HQ or local consultant; gather information on funding sources and regional needs; meet with potential FBO partners; and conduct a SWOT analysis 3. Complete an MTI-Uganda SP (an off-site workshop) Feb 2010 June 2010 October 2010 MTI U engaged in a strategic planning process facilitated by the Africa Region Deputy Director and led by the Country Director. The MTI U Strategic Plan, based on MTI’s agency plan, was finalized in March 2011. The plan will be reviewed annually and used to guide program and organizational growth. Communication & Decision￾making 1. Finalize organizational structure with clearly defined channels of supervision and communication. 2. Organize regular staff meetings or other fora for staff to contribute ideas and improvement recommendations 3. Ensure communication is copied to all pertinent staff March 2010 and ongoing In July 2011 MTI U decentralized their structure, creating regional teams of projects in Northern Uganda and SW Uganda. This streamlined communication and reporting within MTI U. Project managers meet with their teams weekly, followed with monthly regional meetings and quarterly management team meetings which facilitate project reporting, sharing of upcoming key activities, and any constraints or obstacles are addressed. With the placement of the Africa Regional Health Advisor in Uganda, a regular system of communication and technical advising with project teams was developed. Monitoring and Evaluation 1. Conduct a Readiness Assessment for M&E (capacity, resources, tools etc) 2. Develop a comprehensive M&E plan 3. Adapt relevant M&E tools from MTI HQ and other partners 4. Provide M&E support to all MTI Uganda projects 5. All M&E staff and PMs go through PCM Oct – Dec 2010 All MTI U projects carry out M&E, with new tools being adapted. A comprehensive M&E plan is to be developed in FY 13. All key management staff have completed MTI Project Cycle Management (PCM) training with the exception of new staff in SW Uganda. It is planned to roll out PCM training in the next year to non-management staff who demonstrate a readiness to learn and apply it in their work. 1 Annex 8: Training Matrix Training Matrix Project Level (HC or Community) Type of personnel Official government CHW or Grantee developed cadre Paid or Volunteer Number Trained Through March 2012 Focus of Training Number of Days Health Centres Health Workers Government staff Paid 34 IMCI 11 days Health Centres Health Workers Government staff Paid 19 ToT in ECD 5 days Health Centres Health Workers Government staff Paid 46 Maternal Newborn Care 3 days Community and Health Centres Health Workers and Community Leaders Government staff Paid/Volunteer 40 Referral Systems 2 days Community and Health Centres Health Workers and community Leaders Government staff Paid/Volunteer 40 Joint Support Supervision 2 days Community and Health Centres VHTs Government Volunteer 560 ICCM 5 days Community VHTs and Peer Educators Government CHWs, Grantee developed Cadre(CSP) Volunteer 300 ECD 3 days Community Mothers Group Grantee developed Cadre(CSP) Volunteer 300 ICCM 1 day Community VHTs Government CHWs Volunteer 560  Referral Systems Reporting and Record Keeping  Maternal danger signs  Early health care seeking behavior  Immunization  Health Hygiene and sanitation  The importance of home visits  Emergency Transport  Nutrition Half-day Training provided during quarterly meetings 1 Annex 9: Project Management Evaluation Planning Both project staff and DHO were involved in creating the DIP and work plan. Some project activities, such as IMCI training, had actually advanced significantly before the DIP was approved. Project staff revisited the plan and updated it for the first annual report. The issue that has arisen in regard to level of effort for the technical interventions was not due to the work plan itself, but rather, to adding the additional effort and time dedicated to supporting for DHO immunization activities on top of what was already planned. The one gap in the DIP was related to Early Childhood Development. No level of effort was assigned to ECD nor was there a detailed plan of how ECD would be rolled out to the community level. The DIP described the process as far as the training of trainers. From that point on, staff made plans stage by stage. This affected budget as well as timing and intensity of the ECD activities. For the remainder of the project, the original plan for ECD has been modified to focus ECD activities within one sub-county with a timeline established in the revised work plan. The evaluator is recommending that one staff be designated for ECD. Supervision of Project Staff The internal CSP supervisory system appears to be adequate. Current project leadership will provide the needed guidance for staff in planning effective use of their time and following the work plan. There is no plan for MTI to maintain this project beyond CSHGP funding, hence, no need for an institutionalized supervision system. Apart from the internal CSP supervision, one of the major capacity-building efforts of the project has been to strengthen DHO support and supervision of health centers and health center support and supervision of VHTs, primarily through mentoring and joint supervision activities. This was described earlier in this report. Human Resources and Staff Management No project operations are intended to be sustainable, other than the VHTs which are a government institution with clear policy guidelines from the national level. Morale of project staff appears high and there is good interaction between team members. The new project manager has been well-accepted and has assumed a positive leadership style. Other than the departure of the previous project manager, who left for another MTI posting in another region, there has been no staff turnover. This has been a real advantage for the CSP. Financial Management MTI U has an established process the project manager follows for budget review and fund request. The project manager receives a copy of the project budget at the beginning of the fiscal year that indicates monthly and quarterly amounts per line item. Project funds are then requested based on the work plan and budget in consultation with senior management and the finance department. Project managers also meet together quarterly to review fund balances, discuss 2 findings, and agree on any adjustments. To protect both the organization and the individual, as a matter of policy, MTI-U does not distribute hard copies of quarterly financial reports to staff. When a report for the quarter ending in December 2011 was sent to the MTE team from HQ upon request, it appears the budget may be under spent. This will enable the project to implement some recommendations that require funds. Logistics Apart from the initial investments in office equipment, vehicles, and equipment for the health centers, there has been little procurement. There were no complaints about procurement. With the experience of emergencies and short-term projects, rapid procurement may well be one of MTI Uganda’s strengths. Information Management This was described under Section D. The project M&E coordinator is very competent and is receiving excellent mentoring from MTI headquarters. Technical and Administrative Support This was the first USAID and CSHGP grant received and implemented by MTI-U. It coincided and was simultaneously implement along with their first MCP grant. Initially, it was very challenging undertaking and managing the complexities of GSHGP while also managing the MCP. This stretched MTI-U as an organization to grow and learn and adapt to these new opportunities. Strong technical and administrative support and guidance from MTI HQ was essential. Incorporating HQ technical assistance within the reporting structure of MTI-U along with the support of the Regional Deputy Regional Director and Regional Health Advisor has been an ongoing learning process. Through the opportunity of the CSP, MTI-U management have had the opportunity to learn and apply technical and administrative guidance which has strengthened them as an organization. Annex 7. The OCA Scoresheet and Action Plan indicate the progress made by MTI-U during this period of the CSP. In undertaking CSP, MTI-U cultivated and developed strong linkages with the DHO, local authorities, and other partners, mobilized and oriented staff to the context and technical requirements of the project, such as the DIP, reporting, and MTE. The Operations Manager together with the Project Manager reviewed and approved the CSP workplan. The Country Director meets at least bi-weekly with the project team, DHO, HFs and other district officials to receive feedback. Findings are reported quarterly at the MTI-U management meetings, which include project managers, and recorded in meeting minutes. In the first two years of implementation, MTI Uganda senior management did not closely monitor project staff time allocated for supporting the DHO in immunization activities against the workplan. While the workplan allocated only 10% level of effort to immunizations, project staff spent approximately 55% of their time on this activity which reduced time available to implement the education plan, negatively impacting the progress of the project in this key project area. 3 Feedback regarding this oversight provided by the HQ technical support group and regional management, as well as findings of the MTE, have reinforced the importance of MTI U senior management monitoring activities against the workplan, ensuring application of technical recommendations and ensuring timely communication of any variance to HQ and the regional health advisor. In addition, the weekly update of project activities provided by the CSP Project Manager to the Northern Uganda Regional Manager should be reviewed and compared to the workplan and any discrepancies noted and a follow up plan developed. Management Lessons Learned 1. For an entry grant project, in a country office with no experience in child survival, HQ technical staff should be involved in assessing the technical performance of the child survival project staff, particularly the manager. 2. Country and regional management should oversee implementation of detailed workplan and provide guidance and ideas in technical and management recommendations. They need to encourage project statt to adopt technical recommendations. 3. MTI management should provide the project manager adequate financial information on a quarterly basis to enable her to adjust project activities appropriately. Annex 10: Child Survival Sustainability Framework– up-dated MTI Lira Child Survival Project in Uganda Sustainability Framework The MTI Lira Child Survival Project in Uganda used the Child Survival Sustainability Assessment (CSSA) framework to organize and explain its sustainability strategy. A workshop was held on June, 22 to: 1) Define the project system, its actors and its vision 2) Define goals of partners for contributing to the shared vision; barriers/challenges 3) Identify elements of each dimension, responsibilities and corresponding indicators The workshop was facilitated by the MTI Africa Health Advisor and HQ Child Survival Advisor and was attended by 16 MTI Child Survival and Malaria Communities Project staff, the MTI Uganda Country Director, and the Lira District Health Biostatician and Assistant District Health Officer In - Charge Environmental health. Project Sustainability Vision: All children under 5 years and their caretakers live healthy and productive lives through communities taking responsibility for their own health and provision of quality health services. The Project System The project system was defined as the circle of stakeholders directly involved in the local system, in this case the district health system, community structures, volunteers, leaders and families. Outside of this circle are external stakeholders which also influence certain processes within the system. An issue discussed during the workshop was how sustainability depends on stakeholders within the system collaborating with both internal and external stakeholders. The defined system is shown in the following graphic: Lira District  DHO and health facility staff  Religious leaders  Community members (households)  VHTs  Local Government Authorities  Parish level leaders INGOs Ministry of Health Other National Ministries e.g MOES/ECD policy Private Sector Donors (USAID, MTI HQ) Security Local NGOs and 2 Health Outcomes: The outcome of the sustainability process - the population health. The project uses data from KPC household surveys at baseline, midterm and final evaluation to monitor these indicators. Health Service Provision: How well the local health providers—both facility and community￾based—deliver services and products to the beneficiary population. It is focused on access and quality. The project uses information from the Rapid Health Facility Assessment to track changes in this component. Organizational Capacity and Viability: This refers to a range of functions that are necessary to the life of a local organization, its administration, and its ability to perform its mission. These include leadership and governance, financial management, human resources, and organizational performance. The CSP indicators most relevant for this component are those related to supportive supervision “Percentage of HC staff received a supervisory visit within 3 months” and “Percentage of HUMCs that are using information from community HIS for decision making in the last year with at least one concrete example of action taken” . Community Capacity: This refers to community members who are active in promoting and mobilizing the community for maternal and child health, such as Village Health Teams, Peer Educators and Mothers Group Leaders, To assess community competence, the CSP will track two indicators: “Percentage of communities with an emergency/referral transportation system with at least one use within the past 3 months” and “Number of VHT members providing home visits, community education and referrals in the community for early access to health services.” Enabling Environment: An enabling environment is a set of conditions that can either support or weaken gains in health. These conditions include legal and policy frameworks and political, social and cultural, and economic factors. Some environmental factors may be within the ability of a project or local system stakeholders to influence, whereas others will not be. The project has been advocating for reliable supplies of drugs at health facilities and will track “Percentage of HC reporting no stockouts of ORS and zinc during the past quarter”. 3 Tracking Sustainability: The following matrix of indictors was developed to track sustainability. Data from the 2009 baseline KPC survey and Health Facility Assessments was used to establish baseline indicator levels and the data from the 2011 Health Facility Assessment and 2012 KPC survey and project monitoring reports were used to establish midterm results. Dimension I: Health Outcomes and Services Indicator Baseline Targets at final Progress at midterm Ia. Percentage of mothers of children 0 - 23 years whose births were attended by skilled personnel 35% 50% 43% Ib. Percentage of mothers with children 0 -23 months who received at least two doses of IPT during pregnancy with their youngest child 35% 60% 59% Ic. Percentage of mothers of children 0 - 23 months who received a post partum visit by a trained health worker within 3 days after the birth of the youngest child 16.33% 50% 30% Id. Percentage of children aged 0 – 23 months who were put to the breast within one hour of delivery 29% 60% 22.7% Ie. Percentage of mothers of children aged 0 – 23 months who report that they talk or sing to the child while feeding their child 57% 80% 65% 1f. Percentage of HC in which > 80% sick children are treated according to protocol 25% 75% 100% Five of the six indicators for health outcomes and health services improved since baseline based on KPC and HFA results. The uptake of immediate breastfeeding did not improve. Dimension II: Organizational Capacity and Viability Indicator Baseline Targets at final Progress at midterm IIa. Percentage of HC staff received a supervisory visit within 3 months 25% 75% 75% IIb. Percentage of HUMCs that are using information from community HIS for decision making in the last year with at least one concrete example of action taken 0% 70% 25% 4 As of mid-term, seventy five percent of health facility staff receive supportive supervision visits. The HUMC at the Aromo health facility is the only HUMC which presently uses data for decision making. Dimension III: Community Competence and Political Environment Indicator Baseline Targets at final Progress at midterm IIIa. Percentage of communities with an emergency/referral transportation system with at least one use within the past 3 months 0% 70% Little progress has been made IIIb. Percentage of HC reporting no stockouts of ORS during the past quarter 0% 75% 75% IIIc. Number of VHT members providing home visits, community education and referrals in the community for early access to health services. 560 560 560 The project has not collected data from all communities but a survey of 73 communities indicates that 20% of the communities have established and emergency transport system with at least one use within the previous three months. The project has trained and supported 560 VHTs who are providing home visits, some community education and referrals Annex 11: Rapid CATCH Table, Midterm Evaluation Results, March 2012 CSHGP Intervention Area Rapid CATCH Indicator Baseline Midterm Percentage Confidence Interval Percentage Confidence Interval Infant and Young Child Feeding and Anthropometrics Underweight: Percentage of children 0-23 months who are underweight (-2 SD for the median weight for age, according to WHO/HCHS reference population) 27.7% 7.8 16.2% 5.1 Infant and Young Child Feeding: Percent of infants and young children aged 6-23 months fed according to a minimum of appropriate feeding practices. 23.1% 8.1 41.9% 6.9 Maternal Newborn Care Ante-Natal Care: Percentage of mothers of children age 0- 23 months who had four or more antenatal visits when they were pregnant with the youngest child 35.3% 8.6 50.0% 7.0 Maternal TT Vaccination: Percentage of mothers with children age 0-23 months who were protected against Tetanus before the birth of the youngest child. 75.7% 11.0 73.7% 6.1 Skilled Delivery Assistance: Percentage of children age 0-23 months whose births were attended by skilled personnel 35.3% 8.6 50.5% 7.0 Post-Partum visit to check on newborn within the first 3 days after birth: Percent of children 0-23 months who received a post -partum visit by an appropriate trained health worker within three days after birth 16.3% 6.2 22.2% 5.8 Exclusive breastfeeding: Percentage of children 0-5 months who were exclusively breastfed during the last 24 hours 73.6% 26.0 66.2% 6.6 Vitamin A Supplementation Vitamin A Supplementation in the last 6 months: Percentage of children age 6-23 months who received a dose of Vitamin A in the last 6 months (Mother’s recall). 70.1% 13.2 57.6% 6.9 Immunization Access to immunization services: Percentage of children 12- 23 months who received DPT1 before they reached g12 months (card verified 87.0% 15.3 89.4% 4.3 Health System Performance regarding Immunization services: Percentage of children12-23 months who received DPT3 before they reached 12 months (card verified). 85.1% 15.3 72.7% 6.2 Measles vaccination: Percentage of children 12-23 months who received a measles vaccination by the time of the survey (card verified) 85.1% 15.3 80.8% 5.5 Malaria Child sleeps under an insecticide-treated bednet: Percentage of children 0-23 months who slept under an insecticide￾treated bed net (in malaria risk areas, where bed net use is effective) the previous night. 51.3% 9.9 44.4% 6.9 Child with fever receives appropriate antimalarial treatment: Percentage of children 0-23 months with a febrile episode that ended during the last two weeks who were treated with an effective anti-malarial drug within 24 hours after the fever began. 25.0% 8.7 68.7% 6.5 Control of Diarrhea ORT use: Percentage of children 0-23 months with diarrhea in the last two weeks who received Oral Rehydration Solution (ORS) and/or recommended home fluids. 47.2% 16.0 52.5% 7.0 ARI/Pneumonia Appropriate Care Seeking for Pneumonia: Percentage of children age 0-23 months with chest-related cough and fast/difficult breathing in the last two weeks who were taken to an appropriate health provider. 57.8% 14.7 84.8% 5.0 Water and Sanitation Point of Use (POU): Percentage of households of children 0-23 months that treat water effectively. 11.3% 5.2 10.1% 4.2 Appropriate Hand washing Practices: Percentage of mothers of children 0-23 months who live in households with soap at the place for hand washing 85.0% 11.2 87.4% 4.6 Child Spacing Birth Spacing: Percentage of children age 0-23 months who were born at least 24 months after the previous surviving child. 33.3% 8.4 30.3% 6.4 Child Survival and Health Grants Program Project Summary Mar-01-2012 Medical Teams International (Uganda) General Project Information Cooperative Agreement Number: GHS-A-00-09-00012 MTI Headquarters Technical Backstop: Mary Helen Carruth MTI Headquarters Technical Backstop Backup: Janis Lindsteadt Field Program Manager: Lydia Akulo Midterm Evaluator: Judiann McNulty Final Evaluator: Headquarter Financial Contact: Pamela Blikstad Project Dates: 9/30/2009 - 9/29/2013 (FY2009) Project Type: New Partner USAID Mission Contact: Janex Kabarangira Project Web Site: www.medicalteams.org Field Program Manager Name: Lydia Akulo (Project Manager) Address: PO Box 1095 Lira Uganda Phone: 011-256-0772-696665 Fax: E-mail: lakulo@medicalteams.org Skype Name: Alternate Field Contact Name: Anna Summer (Africa Health Advisor) Address: PO Box 26073 Kampala Uganda Phone: 011-256-0789-817876 Fax: E-mail: asummer@medicalteams.org Skype Name: Grant Funding Information USAID Funding: $1,499,646 PVO Match: $511,133 General Project Description Medical Teams International (MTI), a 2009 New Partner category grantee, is implementing the Lira District Child Survival Project in northern Uganda. The project goal is to reduce child morbidity and mortality through sustainable improvements in preventive maternal and child health behaviors and utilization of strengthened health services in Erute North Sub-District of Lira District. The capacity of Village Health Teams (VHTs) in community integrated management of childhood illness (C-IMCI) will be strengthened using a social and behavior change approach, a structured referral system between VHTs and local health facilities will be established, and facility staff will be trained in IMCI and also receive on-the-job mentoring. The project will incorporate Early Childhood Development (ECD) activities to enhance the impact and sustainability of project interventions. Integration of ECD will further improve health status by promoting positive caregiving practices in the earliest years of life to affect all domains (cognitive, physical, language, and social/emotional). Project Location Latitude: 2.16 Longitude: 32.93 Project Location Types: Rural Levels of Intervention: Health Center Health Post Level Community Province(s): Lango Sub-Region, Northern Region District(s): Lira District Sub-District(s): Lira, Aromo, and Ogur Sub-counties of North Erute County Operations Research Information There is no Operations Research (OR) component for this Project. Partners Lira District Health Office (Collaborating Partner) $0 Project communities (Collaborating Partner) $0 Strategies Social and Behavioral Change Strategies: Community Mobilization Interpersonal Communication Health Services Access Strategies: Implementation in a geographic area that the government has identified as poor and underserved Health Systems Strengthening: Supportive Supervision Referral-counterreferral system development for CHWs Review of clinical records (for quality assessment/feedback) Strategies for Enabling Environment: Building capacity of communities/CBOs to advocate to leaders for health Tools/Methodologies: BEHAVE Framework Rapid Health Facility Assessment LQAS Capacity Building Local Partners: Dist. Health System Health Facility Staff Interventions & Components Control of Diarrheal Diseases (20%) - Hand Washing - ORS/Home Fluids - Feeding/Breastfeeding - Care Seeking - Case Management/Counseling IMCI Integration CHW Training HF Training Immunizations (10%) - Classic 6 Vaccines - Mobilization IMCI Integration CHW Training HF Training Infant & Young Child Feeding (20%) - ENA - Comp. Feed. from 6 mos. - Cont. BF up to 24 mos. - Maternal Nutrition - Promote Excl. BF to 6 Months IMCI Integration CHW Training HF Training Maternal & Newborn Care (25%) - Recognition of Danger signs - Newborn Care - Post partum Care - Integation. with Iron & Folic Acid - Normal Delivery Care - Birth Plans - Emergency Transport - Neonatal Vitamin A - Kangaroo Mother Care (skin to skin care) - AMTSL IMCI Integration CHW Training HF Training Pneumonia Case Management (25%) - Case Management Counseling - Recognition of Pneumonia Danger Signs IMCI Integration CHW Training HF Training Operational Plan Indicators Number of People Trained in Maternal/Newborn Health Gender Year Target Actual Female 2010 0 Female 2010 148 Male 2010 478 Male 2010 0 Female 2011 150 Female 2011 474 Male 2011 462 Male 2011 480 Female 2012 474 Male 2012 462 Female 2013 0 Male 2013 0 Number of People Trained in Child Health & Nutrition Gender Year Target Actual Female 2010 0 Female 2010 162 Male 2010 183 Male 2010 0 Female 2011 160 Female 2011 474 Male 2011 462 Male 2011 195 Female 2012 474 Male 2012 462 Female 2013 0 Male 2013 0 Number of People Trained in Malaria Treatment or Prevention Gender Year Target Actual Female 2010 134 Female 2010 0 Male 2010 426 Male 2010 0 Female 2011 0 Female 2011 0 Male 2011 0 Male 2011 0 Female 2012 0 Male 2012 0 Female 2013 0 Male 2013 0 Locations & Sub-Areas Total Population: 113,400 Target Beneficiaries Uganda - MTI - FY2009 Children 0-59 months 22,457 Women 15-49 years 22,907 Beneficiaries Total 45,364 Rapid Catch Indicators: DIP Submission Sample Type: 30 Cluster Indicator Numerator Denominator Percentage Confidence Interval Percentage of mothers with children age 0-23 months who received at least two Tetanus toxoid vaccinations before the birth of their youngest child 227 300 75.7% 11.0 Percentage of children age 0-23 months whose births were attended by skilled personnel 106 300 35.3% 8.6 Percentage of children age 0-5 months who were exclusively breastfed during the last 24 hours 39 53 73.6% 26.0 Percentage of children age 6-23 months who received a dose of Vitamin A in the last 6 months: card verified or mother’s recall 141 201 70.1% 13.2 Percentage of children age 12-23 months who received a measles vaccination 124 161 77.0% 15.0 Percentage of children age 12-23 months who received DTP1 according to the vaccination card or mother’s recall by the time of the survey 140 161 87.0% 15.3 Percentage of children age 12-23 months who received DTP3 according to the vaccination card or mother’s recall by the time of the survey 137 161 85.1% 15.3 Percentage of children age 0-23 months with a febrile episode during the last two weeks who were treated with an effective anti-malarial drug within 24 hours after the fever began 55 220 25.0% 8.7 Percentage of children age 0-23 months with diarrhea in the last two weeks who received oral rehydration solution (ORS) and/or recommended home fluids 51 108 47.2% 16.0 Percentage of children age 0-23 months with chest-related cough and fast and/or difficult breathing in the last two weeks who were taken to an appropriate health provider 85 147 57.8% 14.7 Percentage of households of children age 0-23 months that treat water effectively 34 300 11.3% 5.2 Percentage of mothers of children age 0-23 months who live in households with soap at the place for hand washing 255 300 85.0% 11.2 Percentage of children age 0-23 months who slept under an insecticide-treated bednet (in malaria risk areas, where bednet use is effective) the previous night 154 300 51.3% 9.9 Percentage of children 0-23 months who are underweight (-2 SD for the median weight for age, according to the WHO/NCHS reference population) 83 300 27.7% 7.8 Percentage of infants and young children age 6-23 months fed according to a minimum of appropriate feeding practices 56 242 23.1% 8.1 Percentage of mothers of children age 0-23 months who had four or more antenatal visits when they were pregnant with the youngest child 106 300 35.3% 8.6 Percentage of mothers of children age 0-23 months who are using a modern contraceptive method 100 300 33.3% 8.4 Percentage of children age 0-23 months who received a post-natal visit from an appropriately trained health worker within two days after birth 49 300 16.3% 6.2 Rapid Catch Indicators: Mid-term Sample Type: LQAS Indicator Numerator Denominator Percentage Confidence Interval Percentage of mothers with children age 0-23 months who received at least two Tetanus toxoid vaccinations before the birth of their youngest child 146 198 73.7% 6.1 Percentage of children age 0-23 months whose births were attended by skilled personnel 100 198 50.5% 7.0 Percentage of children age 0-5 months who were exclusively breastfed during the last 24 hours 131 198 66.2% 6.6 Percentage of children age 6-23 months who received a dose of Vitamin A in the last 6 months: card verified or mother’s recall 114 198 57.6% 6.9 Percentage of children age 12-23 months who received a measles vaccination 160 198 80.8% 5.5 Percentage of children age 12-23 months who received DTP1 according to the vaccination card or mother’s recall by the time of the survey 177 198 89.4% 4.3 Percentage of children age 12-23 months who received DTP3 according to the vaccination card or mother’s recall by the time of the survey 144 198 72.7% 6.2 Percentage of children age 0-23 months with a febrile episode during the last two weeks who were treated with an effective anti-malarial drug within 24 hours after the fever began 136 198 68.7% 6.5 Percentage of children age 0-23 months with diarrhea in the last two weeks who received oral rehydration solution (ORS) and/or recommended home fluids 104 198 52.5% 7.0 Percentage of children age 0-23 months with chest-related cough and fast and/or difficult breathing in the last two weeks who were taken to an appropriate health provider 168 198 84.8% 5.0 Percentage of households of children age 0-23 months that treat water effectively 20 198 10.1% 4.2 Percentage of mothers of children age 0-23 months who live in households with soap at the place for hand washing 173 198 87.4% 4.6 Percentage of children age 0-23 months who slept under an insecticide-treated bednet (in malaria risk areas, where bednet use is effective) the previous night 88 198 44.4% 6.9 Percentage of children 0-23 months who are underweight (-2 SD for the median weight for age, according to the WHO/NCHS reference population) 32 198 16.2% 5.1 Percentage of infants and young children age 6-23 months fed according to a minimum of appropriate feeding practices 83 198 41.9% 6.9 Percentage of mothers of children age 0-23 months who had four or more antenatal visits when they were pregnant with the youngest child 99 198 50.0% 7.0 Percentage of mothers of children age 0-23 months who are using a modern contraceptive method 60 198 30.3% 6.4 Percentage of children age 0-23 months who received a post-natal visit from an appropriately trained health worker within two days after birth 44 198 22.2% 5.8 Rapid Catch Indicators: Final Evaluation Rapid Catch Indicator Comments