Tube Poka Child Survival Project FINAL EVALUATION REPORT Submitted December 31, 2009 Chitipa District, Northern Malawi Richard Crespo, PhD, External Evaluation Consultant & Team Leader Victor Kabaghe, Child Survival Project Director Melanie Morrow, Director of Maternal & Child Health Programs Sarah Borger, Maternal & Child Health Specialist Cooperative Agreement No. GHS-A-00-05-00032-00 Program Dates: October 1, 2005 to September 30, 2009 (Intentionally blank) i Table of Contents List of Acronyms ........................................................................................................................... iii A. Preliminary Information: Executive Summary ......................................................................... 1 B. Overview of the Project.............................................................................................................. 4 Project location ........................................................................................................................... 4 Population estimates ................................................................................................................... 4 Description of Technical and Cross Cutting Interventions ......................................................... 4 Project Design ............................................................................................................................. 6 Partnerships ................................................................................................................................. 7 Mission Collaboration ................................................................................................................. 8 C. Data Quality ............................................................................................................................... 8 D. Project Results.......................................................................................................................... 10 E. Discussion of the Results.......................................................................................................... 12 Objective 1 ................................................................................................................................ 12 Objective 2 ................................................................................................................................ 15 Objective 3 ................................................................................................................................ 19 Overall Factors Affecting All Objectives ................................................................................. 26 F. Potential for Sustainability and Replication ............................................................................ 26 Progress Towards Sustained Outcomes .................................................................................... 27 Contribution to Replication or Scale-up ................................................................................... 29 Attention to Equity .................................................................................................................... 30 Role of Community Health Workers ........................................................................................ 30 Contribution to Global Learning ............................................................................................... 31 G. Conclusions and Recommendations ....................................................................................... 31 Recommendations ..................................................................................................................... 32 Annexes Annex 1: Results Highlight .......................................................................................................... A1 Annex 2: List of Publications and Presentations Related to the Project ...................................... A2 Annex 3: Project Management Evaluation .................................................................................. A3 Annex 4: Workplan Table ............................................................................................................ A7 Annex 5: Rapid CATCH Table.................................................................................................. A10 Annex 6: CHW Training Matrix ............................................................................................... A11 Annex 7: Evaluation Team Members and their Titles ............................................................... A12 Annex 8: Evaluation Assessment Methodology ........................................................................ A13 Annex 9: List of persons interviewed and contacted during Final Evaluation .......................... A14 Annex 10: Project Data Form .................................................................................................... A15 Annex 11: Grantee Plans to Address Final Evaluation Findings .............................................. A24 Annex 12: Final KPC Report ..................................................................................................... A25 ii Table of Tables Table 1: Summary of Main Accomplishments ............................................................................... 2 Table 2: Monitoring and Evaluation Table ................................................................................... 10 Table 3: Specific Occasions When Respondents Washed Hands ................................................. 23 Table 4: Tube Poka Sustainability Framework ............................................................................. 27 Table 5: Workplan Table ............................................................................................................. A7 Table 6: Rapid CATCH Indicators ............................................................................................ A10 Table 7: Community Health Worker Training Matrix ............................................................... A11 Table of Charts Chart 1: KPC Findings for Malaria and Pneumonia Control ........................................................ 20 Chart 2: Signs of Childhood Illness Identified by Mothers during the Final KPC ....................... 21 Chart 3: Diarrhea Control Indictors .............................................................................................. 22 Chart 4: KPC Nutrition Findings .................................................................................................. 23 Chart 5: Indicators for Managing the Sick Child and Childhood Immunization .......................... 25 Chart 6: HIV/AIDS Indicators ...................................................................................................... 26 iii List of Acronyms ANC Antenatal Care ARI Acute Respiratory Infection CCAP Church of Central Africa Presbyterian C-HIS Community Health Information System CG Care Groups C-IMCI Community Integrated Management of Childhood Illness CS Child Survival DEHO District Environmental Health Officer DHMT District Health Management Team DHO District Health Officer DIP Detailed Implementation Plan EBF Exclusive Breastfeeding HC Health Center HF Health Facility HFA Health Facility Assessment HIS Health Information System HP Health Post HSA Health Surveillance Assistant IMCI Integrated Management of Childhood Illness ITN Insecticide Treated Net KPC Knowledge, Practices, and Coverage LRA Local Rapid Assessment M&E Monitoring and Evaluation MOH Ministry of Health ORT Oral Rehydration Therapy OVC Orphans and Vulnerable Children PCM Pneumonia Case Management PMTCT Prevention of Mother to Child Transmission of HIV/AIDS TPCSP Tube Poka Child Survival Program SP Sulfadoxine-Pyrimethamine VCT Voluntary Counseling and Testing for HIV/AIDS VHC Village Health Committee WRM World Relief Malawi 1 A. Preliminary Information: Executive Summary World Relief Malawi’s Tube Poka Child Survival Program aimed to reduce the disease burden and mortality in children under five and women of reproductive age by establishing a community-based system for implementing C-IMCI using Care Groups of local volunteers. The Program was implemented primarily at the community level and was designed to build the capacity of local partners and empower community institutions to sustain child survival interventions. The project had three program objectives: 1. Strengthening the capacity of Chitipa District’s health system to implement child survival and health interventions according to Integrated Management of Childhood Illness (IMCI) protocols. 2. Developing sustainable community-based mechanisms to improve prevention and care￾seeking practices for childhood illnesses at the household and community level. 3. Promotion of Key Family Practices for Child Health. The Tube Poka Child Survival Project operated within the Chitipa District of Malawi. Chitipa District is the northern most district and is further from the capital city than any other district in Malawi. The topography is largely mountainous, with Chitipa’s estimated 174,7861 people scattered over the mountains in 475 villages. The road to the district capital is not paved, making it is among the most isolated districts in the country. The people of Chitipa speak 17 dialects, complicating communications. The project interventions with corresponding percent effort were: malaria prevention and case management (30%), nutrition, including exclusive breast feeding (EBF), micronutrients and the Hearth program (20%), immunization (20%), control of diarrheal diseases (CDD) (15%), pneumonia case management (10%), and HIV/AIDS prevention (5%). Main Accomplishments 1. Tube Poka formed several partnerships that expanded the coverage and effectiveness of the project. World Relief Malawi (WRM) signed agreements with UNICEF on Accelerated Child Survival and Development (ACSD), and with PSI/Malawi to implement the Integrated Diarrhea Prevention Project (IDPP). These programs were implemented rapidly and effectively because of the Care Group structure set up by the project. This partnership contributed to high levels of achievement in household water purification and increased use of ORS. By the end of the project 70% of the households were treating their water, and 94% of these were using WaterGuard or chlorine. 2. Tube Poka’s relationship with the District Assembly and the Chitipa District Health Management Team was very strong. The project staff attended regular District Executive Committee meetings (the technical arm of the District Development Secretariat) and presented reports following each monitoring survey. Through these experiences the Tube Poka staff modeled how to present data using PowerPoint and a LCD projector. Ministry of Health (MOH) and District staff followed their examples and over time learned how to make similar professional presentations. 3. An unplanned development in community organization was the creation of zonal committees based on the geographical boundaries of the traditional authorities. Such 2 groups were formed in the first Malawi CSP and based on their success, the project leadership decided to replicate them in Chitipa. The purpose was to create an organization that could advocate for health care before political authorities, represent themselves before community leaders and Traditional Authorities, and ensure that Care Groups function as planned. 4. The measurable indicator with largest increase in percentage points was home treatment of diarrhea with ORT. It increased 56 points, from 8% to 64%. 5. In the nutrition intervention, the indicators of EBF and complementary breastfeeding during six to nine months of age, both doubled over the life of the project. EBF increased from 40% to 80%. Complimentary feeding increased from 40% to 85%. 6. The percent of caregivers who knew two or more ways of reducing the risk of HIV infection increased from 68% to 86%. 7. By the end of the project, nearly all caregivers in the district knew two or more danger signs for childhood illnesses: from a baseline of 71% to 96% at the end of the project. Table 1: Summary of Main Accomplishments Project Objective #1: Strengthen the capacity of Chitipa District’s health system to improve the quality and Coverage of IMCI services Project Inputs Activities Output Outcomes Quantitative survey and analysis methods Invited MOH and DHMT staff to participate in Baseline, Midterm and Final Evaluations 2 MCH Coordinators, 2 DEHOs, 4Assistant EHOs, 1Malaria Control Manager, HIV/AIDS Coordinator and a IMCI Coordinator participated in the surveys •All survey results were shared with the MOH/DHMT •District staff reporting utilizing these techniques to collect district Invited MOH/DHMT staff data to participate in the seven monitoring surveys Identified a need for data entry and analysis software Trained DHMT staff on EpiInfo software and analysis capabilities MOH IMCI curriculum on approved program interventions Cross trained select HSAs on the one of seven intervention areas Trained 2 HSAs in each intervention, for a total of 12 HSAs Care Group Focus Groups reported feeling connected to the MOH and supported by their local HSA Routine visitation and supervision conducted jointly with CSP and HSA staff 30% HSAs visited CG in last month based on qualitative survey results Train HSAs in nutrition interventions Trained 30 HSAs in P/D Hearth UNICEF Accelerated CS Grant Organized training for new HSAs in Care Groups and village clinics Trained 25 new HSAs •42% of children with suspected malaria taken for treatment within 24 hrs. •44% of children with suspected pneumonia taken for treatment within 24 hrs. TP data and project reports TP attendance at quarterly Technical Review meetings Routine sharing of project data with the district •DHMT had representative data on CS measures from the HSAs. •Use of project data to compare to district data 3 Project Objective #2: Develop sustainable community-based mechanisms to improve prevention and care-seeking practices for childhood illnesses at household and community level Trained 34 promoters to train Care Groups Promoters train and supervise Care Groups in the community through monthly training and supervision visits. Over 3,000 CGV trained. At the Final KPC, 17% of households were visited twice in last mo; 42% visited once in last mo. Organized community meetings Held meetings with Pastors and Village Leaders related to community mobilization •Monthly visits by promoters and supervisors for training and supportive supervision of pastors 362 Churches involved in supervising and supporting Care Groups According to the supervisor review: •74% of CG supervised by a pastor in last month. •84% of pastors report conducting CGV replacement. •71% of pastors report communicating health messages in last month. •75% of pastors attended CG meeting in last month. Encouraged communities to look at health issues beyond the village level Promoters attend community meetings on a regular basis 10 Regional Zonal Committees were organized based on district authority geographical areas According to the supervisor review: •80% of zonal committees surveyed had met in the past month •Average attendance at their last meeting was reported at 85% VHCs and pastors report discussing health issues at community meetings, including problems and concerns brought by CGVs. According to the supervisor review: •39% of VHC met in the last 2 months. •79% of VDC discussed health issues in the last month •83% CGV focus group participants said village headmen invited them to speak at local meetings Project Objective #3: Promotion of Key Family Practices for Child Health 34 Health Promoters and 7 Supervisors were trained Training camps held before each of the seven intervention areas Monthly visits to Care Groups for training and supervision Train volunteers and caretakers to recognize signs and symptoms of common childhood diseases Over 3,000 CGV trained in seven key intervention areas • Increased knowledge of care seeking signs from 71% to 96% Train volunteers and caretakers to practice disease prevention activities and conduct appropriate home care • The percentage of children 0-6 months exclusively breastfed increased from 40% to 80% • The percentage of children with diarrhea treated with ORT increased from 8% to 64% Train volunteers and caretakers to seek medical care when necessary • The percentage of children whose birth was attended by a skilled provider increased from 55% to 73% •The percentage of children treated for suspected malaria at the HF within 24 hours increased from 18% to 42% 4 B. Overview of the Project World Relief Malawi Tube Poka Child Survival Program aimed to reduce the disease burden and mortality in children under five and women of reproductive age by establishing an effective community-based system for implementing C-IMCI using Care Groups of local volunteers. The Program was implemented primarily at the community level and was designed to build the capacity of local partners and empower community institutions to sustain child survival interventions. The project had three specific objectives: 1. Strengthening the capacity of Chitipa District’s health system to implement child survival and health interventions according to Integrated Management of Childhood Illness (IMCI) protocols. 2. Developing sustainable community-based mechanisms to improve prevention and care￾seeking practices for childhood illnesses at the household and community level. 3. Promotion of Key Family Practices for Child Health. Project location World Relief’s Tube Poka Child Survival Project operated within the Chitipa District of Malawi. The Chitipa District is the northern most district and is further from the capital city than any other district in Malawi. Chitipa has two international borders – Tanzania to the north and Zambia to the west. It also shares boundaries with the Karonga District in the northeast and Rumphi District in the south. The topography of the district is largely mountainous, with Chitipa’s estimated 174,7861 people scattered over the mountains in 475 villages. The road to the district capital is not paved, making it is among the most isolated districts in the country. Cell phone coverage is available throughout much of the district, but internet access is limited. The people of Chitipa speak 17 dialects, which further complicated communications. Population estimates The beneficiary population totals 72,226 comprised of 32,025 children under five (9,413 children less than 12 months, 8,335 children 12-23 months, and 14,277 children 24-59 months) and 40,201 women ages 15-491 . Description of Technical and Cross Cutting Interventions The project interventions with corresponding percent effort were as follows:  Malaria prevention and case management 30%,  Nutrition, including exclusive breast feeding (EBF), micronutrients and the Hearth program 20%,  Immunization 20%,  Control of diarrheal diseases (CDD) 15%,  Pneumonia case management (PCM) 10%,  HIV/AIDS prevention 5%. During the course of the project, volunteers and MOH staff were trained to deliver the same essential messages consistent with IMCI protocols. Informal training also occurred between TPCSP staff and pastors, traditional healers, and VHC members. The dominant, cross-cutting 5 message of this project was that mothers (or caregivers) should seek care from a trained provider when their child is sick. A variety of methods and channels of communication were used to convey all of the project’s messages. Volunteers, pastors, MOH staff, and village leaders were trained (formally and informally) to deliver the same messages, so that community members heard them from a variety of credible sources. Communities received messages through multiple delivery systems: at home, church, village meetings, and community celebrations. The messages were communicated through songs, drama, role play, dances, clinic encounters, IEC materials in health facilities, and through announcements in church. The project gave a priority to malaria control and prevention at community and household levels, while promoting prompt care seeking and optimal care provision by health care providers. Mothers/caregivers received messages during home visits on home management of fever, complications of malaria and danger signs that warranted immediate treatment at a health facility. Children with suspected malaria who received treatment at HP/HC were encouraged to be followed up at home by CGVs. Pregnant women were encouraged to register early for ANC, take two doses of SP and iron supplements (for at least 60 days) in pregnancy. The project encouraged and facilitated the distribution of nets through the MOH and communicated messages communicated messages on the importance of ITNs, how to use them, the signs of malaria, and to seek immediate treatment in suspected cases. The nutrition intervention involved, 1) encouraging mothers to initiate BF immediately postpartum and within the first hour of birth; 2) discouraging and eliminating harmful BF practices and encouraging BF on demand; 3) maintaining and supporting EBF for the first six months of life; 4) introducing appropriate complementary food from six months of life on along with BF through at least two years of age; 5) encouraging appropriate dietary management of the sick child according to IMCI guidelines; 6) promoting vitamin A supplements every six months for children 6-59 months old; 7) encouraging regular attendance of the caretakers and children at GMC; 8) encouraging pregnant women to consume more food during pregnancy and to take iron supplements; and 9) encouraging women to receive Vitamin A supplements immediately after delivery. Growth Monitoring sessions were conducted monthly by MOH staff during EPI clinics, and CSP staff and volunteers supported weighing, recording, checking of immunization status, delivery of vaccines, and nutrition counseling for mothers of children under five years of age. In addition to the activities highlighted above, the project used the Hearth approach in two communities. Hearth is a community-based program for rehabilitation of mildly and moderately malnourished children ages 12-36 months. Hearth sessions provide an opportunity for mothers and caregivers to practice preparing locally available nutrient-dense foods, using active feeding techniques, and practicing good hygiene and sanitation related to food handling and storage. A Hearth cycle consists of 12 days of intensive group activity followed by two weeks of practice and follow-up in the home. 6 Diarrhea prevention messages focused on stopping disease transmission through: 1) promoting basic hygiene practices (use of latrines, washing hands before food preparation, before feeding child, after defecation, after cleaning child’s feces and before eating; use of dish racks, and protection of household water sources) and, 2) promoting exclusive breastfeeding for the first six months of life, then introducing appropriate complementary feeding after six months of age. Case Management Strategies for acute diarrhea and dehydration included: 1) promoting appropriate dietary management during diarrhea by continuing breast feeding, giving small frequent feeds to children six months and older, offering increased fluid and increased or continued food intake during and up to two weeks following diarrhea episode, 2) promoting use of ORT, 3) recognizing danger signs requiring prompt referral to the health facility, and 4) discouraging antibiotics or anti-diarrheal use unless prescribed by a trained provider. For pneumonia case management, the key interventions involved teaching mothers 1) to recognize symptoms and danger signs of pneumonia, 2) to seek prompt treatment from a trained health provider within 24 hours, 3) to comply with treatment instructions and to offer increased fluids and continued or increased foods to children following episode of pneumonia, including continued breastfeeding, and 4) to obtain vitamin A supplements for children aged six months to five years every six months. Mothers and caregivers learned that pneumonia is treatable when diagnosed early, certain healthy behaviors may reduce the risk of contracting pneumonia, and that good nutrition including breastfeeding and vitamin A supplementation along with complete immunization reduces the incidence of severe pneumonia and the risk of pneumonia-related mortality. The project encouraged care seeking behavior and strengthened available services by training 25 new HSAs on optimal pneumonia case management according to IMCI protocols by partnering with UNICEF. The interventions for immunization involved encouraging caretakers to obtain vaccinations for their children on schedule, and encouraging pregnant women to receive at least two doses of tetanus toxoid (TT). Project staff strengthened MOH EPI activities by providing social mobilization and occasional transportation during times of budget shortfalls. HIV/AIDS interventions focused on promoting delivery with a trained provider and prevention of HIV/AIDS. It is hoped that the continued emphasis on delivering with a trained provider will set the stage for increased participation in PMTCT programs as they become available at the health center level. Health education messages included, 1) delivery by a trained health provider, 2) recognizing symptoms of STDs or AIDS, and 3) knowledge of ways to prevent STDs and HIV infections. Project Design The Tuba Poka project used a modified version of the Care Group model and partnering with the District MOH, the Department of Social Welfare, and 362 local churches. The Care Group model uses a census based approach to reach every beneficiary household through the use of local volunteers called Care Group Volunteers (CGV) to create positive behavior change. CGVs conduct home visits, promote behavior change communication (BCC) for key household 7 practices and prompt care seeking according to C-IMCI guidelines. These volunteers are mentored by regional project staff (Health Promoters) and supervised by project Supervisors. In Chitipa, one volunteer assumed responsibility for his/her assigned ten to fifteen households, training mothers and caretakers on key messages related to specific interventions at least once every two weeks. Ten to fifteen volunteers in the same geographic area come together to form a Care Group which met every two weeks to exchange ideas and to receive peer support and training from paid WR Promoters. Each Promoter provides training and supervision to 8 -12 Care Groups depending on geographic area and distance between Care Groups and households. World Relief developed the Care Group model in Mozambique, but this is the first Child Survival project to involve local churches as community-based organizations in direct implementation and supervision of Care Groups. The Chitipa District is considered to be 96% Christian and nearly every village has a church, if not several. Like other Child Survival projects using CGVs, all households with children under five and women of reproductive age were to be covered by volunteers; the unique element is that the Care Groups are anchored by local churches through supervision and CGV selection in hope of increasing sustainability beyond previous Care Group project successes. The project identified one church within each area, and the church leaders were oriented to the project and invited to participate. Three hundred sixty two churches participated in hosting Care Groups. Church leaders participated in selecting CGVs and in organizing care groups. The church and other community leaders who were interviewed during the MTE said that the project had broken down church and community barriers and opened the door for cooperation in areas outside of child survival such as community development. Partnerships Several formal partnerships were formed that assisted and complimented CSP activities. World Relief Malawi (WRM) signed agreements with UNICEF on Accelerated Child Survival and Development (ACSD), and with PSI/Malawi to implement Integrated Diarrhea Prevention Project (IDPP). PSI’s IDPP is funded by USAID to promote hygiene and sanitation including the use of a chlorine-based point-of-use water treatment called WaterGuard. This partnership contributed to high levels of achievement in household water purification. As part of this project, PSI provided project staff with push bikes and materials for demonstrations of ORS use. The IDDP activities complement the hygiene and health messages the CSP promoted by supplying the essential commodities needed for the desired behavior changes, by creating a synergistic effect in the community, and by including school teachers to show children how to purify and serve clean water in school and at home. UNICEF complemented the IDPP project by donating additional WaterGuard stock (37,000 units), which was distributed by the CGVs. UNICEF staff commented on how rapidly the whole stock was distributed to households through the CG network. Evidence of the UNICEF, PSI, and WRM collaboration is that at the final KPC survey 70% of the households responded that they were treating their water, and 94% of these reported using WaterGuard or chlorine. ITN utilization efforts were bolstered by multiple net distribution campaigns by the MOH which provided 14,000 free nets. Additionally, the Church of Central Africa Presbyterian (CCAP) 8 provided 1,000 ITNs and Willow Creek Community Church donated 500 ITNs, along with ten protected waters sources and HIV/AIDS activities in selected areas. Access to care was improved during the life of the project due to MOH construction of 60 new village clinics across the district, reducing travel time from an average of four hours to an average of 30 to 60 minutes. Sixty Health Surveillance Assistants (HSA) were also funded at this time. TPCSP, in collaboration with UNICEF’s ACSD project, was able to train 25 of new HSAs in village health. In addition, the District Assembly, MOH, District Social Welfare, District Community Development, and civil society organizations supported the project and recognized the role Care Groups play in transformational development. Mission Collaboration The program objectives conform to USAID Mission SO8 and the 2008/2009 MOH National health plan which is in line with Program of Works 2004. Program objectives contribute to accomplishing the USAID Mission’s CSHGP Strategic Objectives (SO) and Intermediate Results (IR). The USAID mission priorities focus on increasing the quality of maternal and child health care services at health facilities and in communities. Tube Poka staff participated in the quarterly Synergy meetings sponsored by USAID. These provided a forum for discussing pertinent programming issues with USAID local Mission teams particularly in the health and population, nutrition and HIV/AIDS sections. The presentation of Midterm KPC findings with USAID gave the project a chance to share progress and discuss future project plans. More importantly, the Synergy meetings gave WRM an opportunity to share the Care Group model with NGO partners. In 2007, the I-LIFE consortium (including CARE, CRS, Save the Children, Africare, Emmanuel International, Salvation Army and World Vision) began using Care Groups to serve over 60,000 households. C. Data Quality Tube Poka’s monitoring and evaluation system was comprised of various types of monitoring activities. A KPC 2000+ survey, including Rapid Catch Indicators, was conducted at the baseline and final evaluation. In January 2008, a KPC survey was also conducted to identify the status of the measurable indicators at midterm. The project conducted Local Rapid Assessments to gauge the status of selected KPC indicators following the completion of each intervention. CGVs record monthly mortality statistics including the cause and incidence of deaths for children under five years of age (based on verbal autopsy). Qualitative data were collected in the form of focus group discussions and key informant interviews. These were conducted at the baseline, midterm evaluation, in the development of educational materials for the CGVs, and during the final evaluation. For example, focus groups were conducted in July 2008 that explored mothers’ knowledge, beliefs and practices for giving fluids and foods with childhood diarrhea. A common belief was that sweet foods, including gruel, a grain-based home fluid, would choke a baby, so it was not likely to be given to a child with diarrhea. The information collected in the focus groups was used to develop key messages specific to this situation, and then shared with the Care Group 9 Volunteers during training sessions. The project staff then encouraged Care Group Volunteers to monitor giving fluids and food in diarrhea cases in order to observe change in mothers’ practice. The staff had difficulty with managing quantitative data. The mid-term evaluation report recommended that the project establish a system for monitoring implementation of the Care Group model and implementing a system for data tracking at the community level. Neither of these was done. Another mid-term recommendation was to use LRA methodology to monitor the sustainability data indicators. The staff did collect some of these data, but did so from promoter reports. The data from the reports however, were not consolidated into an aggregate report form and tracked over time. The limited data collected were not used to monitor progress in sustainability. Similarly CGVs had registers and recorded data monthly but the project did not have a system for aggregating and monitoring registry data. For example, the registers recorded the number of households served by the CGVs, but there was no record of the total number of households served by the project or how many CGV were active. The project managers estimated that 85% of the registries were up to date, but this was based on an estimate. They did not have any data to verify their estimate. One datum from the registries that was used by the project was records of deaths. The Director of Social Welfare reported that the project manager or appropriate supervisor would assist in investigating the reasons for childhood deaths. Collaboratively they would work with the family, community leaders, and the Area Development Council to discuss ways to prevent such deaths in the future. A mid-term recommendation was that the project employ a full time M&E manager in order to manage the quantitative data systems. WRM did reassign the Deputy Manager to this position, but a data management system was never implemented. The most serious consequence is that WRM did not have a way to know if the Care Group model was implemented with fidelity. Consequently there was no reliable information on the number of CGVs or when there was turnover of CGVs, making it difficult for the DHMT to form adequate plans for taking over the CGs. The information that was used most often was from the narrative reports. Supervisors, promoters and managers all produced monthly reports. The managers and supervisors identified issues and problems in their reports and used them in planning for the subsequent month. Additionally the supervisors and promoters occasionally used the household spot-check technique and included their findings in their monthly reports. Data from the project’s baseline, mid-term, final and monitoring surveys were shared with the MOH, the Care Group Zonal Committees and the Area Development Committees, fostering a common understanding of priorities and progress. The MOH used Tube Poka’s baseline data in their reports, and used its data for making grant applications until they felt comfortable conducting their own surveys. In regards to MOH’s HIS system, Tube Poka promoters collaborated by collecting HIS data in areas where there were no HSAs or where HSAs needed assistance. Nevertheless, the reliability of the MOH data was not consistent. One of the main reasons was due to logistical problems. 10 During the rainy season data were not obtainable from mountainous, remote areas. Periodic fuel shortages would cause HSAs to miss reporting deadlines. At one point in time the HIS manager passed away and the position was not filled for many months, resulting in data not being collected and processed. All of these factors were influenced by the fact that Chitipa is one of the most remote and poorest districts in Malawi. D. Project Results Table 2: Monitoring and Evaluation Table 1. Strengthen the capacity of Chitipa District’s health system to improve the quality and Coverage of IMCI services Objectives/ Intermediate Results Indicators Data Source Baseline Value Final Value EOP Target Increase knowledge and compliance of health providers in C￾IMCI services /protocols Percentage of providers who have IMCI/C-IMCI training HFA HSAs are trained by the district MCH Coordinator using the mandated MOH curriculum. A second HFA was not conducted. Percentage of HSAs and HC staff who comply with standard case management protocols Supervision of HSAs by MOH and HFA Improve access to preventive health services Percentage of target population with access to preventive health services (<5km) HFA, Project or District Records MOH created 60 new village clinics to increase access in remote villages. Improve the C-HIS Monitoring ring Percentage of HSAs who provide monthly CHIS data to HC Monthly CG stats and HP records HSAs delivery of monthly reports is very inconsistent during rainy season and times of fuel shortages. 2. Develop sustainable community-based mechanisms to improve prevention and care-seeking practices for childhood illnesses at household and community level Objectives Indicators Data Source Baseline Value Final Value EOP Target Build capacity for improved support systems Percentage of HSAs who receive C-IMCI training Project and District Records TP trained 25, all others trained by MOH using MOH curriculum. 80% Percentage of pastors who receive C-IMCI training Project Records No formal training, learned as they participated in CGs 80% Percentage of traditional healers who receive C-IMCI training Project Records Project did not train THs, although staff reported they began to drop their practice as mothers increased attendance at HF 60% Develop a sustainable community-based mechanism Percentage of households visited by their volunteer in the previous two weeks Final KPC NA 17.3% 70% Percentage of Care Groups with volunteer attendance of at least 70% Supervisory checklists (Supervisor’s report) NA 62% 70% Percentage of VHC/VDC who met in the last 2months Community data (Supervisor’s report) NA 79% * 60% Percentage of CGs supervised by trained HSAs Project records NA 30% 70% 11 3. Promotion of Key Family Practices for Child Health Objectives Indicators Data Source Baseline Value Final Value EOP Target Increase knowledge of care seeking signs Percentage of caretakers of children 0- 23m who know at least 2 childhood illness danger signs for seeking care immediately (RC) KPC 71.1% 95.7% 80% Improve home management of sick children Percentage of sick children age 0-23m who received increased fluids and continued feeding during an illness in the past two weeks (RC) KPC 3.9% 22.6% 60% Increase utilization of immunization services Percentage of all children 12-23m fully immunized (BCG, Polio3, DPT3, and Measles) before 24 months as verified by card. * Baseline includes children 12-23m who had a BCG scar and children 9- 11m who had a DPT3, Polio3 and measles vaccine KPC 68.8% 73.4% 80% Increase utilization of ITNs and health/referral services for prevention and treatment of malaria Percentage of children age 0-23m who slept under an insecticide-treated net the previous night (RC) *Baseline includes all nets regardless of insecticide treatment KPC 41.0% 43.7% 60% Percentage of children 0-23m with suspected malaria in the previous 2 weeks who sought treatment from a trained provider within 24 hours of illness onset KPC 17.5% 41.7% 50% Increase utilization of health/referral services for treatment of pneumonia Percentage of children 0-23m with rapid/difficult breathing (suspected pneumonia) in the previous 2 weeks who sought treatment from a trained provider within 24 hours KPC 20.9% 44.0% 50% Increase prevention and home care practices for diarrheal disease Percentage of children 0-23m with diarrhea in the previous 2 weeks who received ORT (home available fluids, breastfeeding or ORS) KPC 8% 64.4% 60% Percentage of mothers with children age 0-23m who report that they wash their hands with soap/ash before food preparation, before feeding children, after defecation, and after attending to a child who has defecated (RC) *Baseline from LRA1 KPC 3.9% 19.3% 60% Improve the nutritional status of children through improve feeding/ supplementation practices Percentage of children 0-5m who were exclusively breastfed during the last 24 hours (RC) KPC 40.0% 79.7% 60% Percentage of children 6-9m who received breastmilk and complementary foods during the last 24 hours (RC) KPC 39.8% 85.2% 70% 12 * 79% refers to the percent of VHC/VDCs that discussed health issues in the last month, from the July 2009 Supervisors’ report. Refer to the Final KPC in Annex X for complete information regarding project and Rapid CATCH indicators. E. Discussion of the Results Objective 1 Objective number 1 was to strengthen the capacity of Chitipa District’s health system to improve the quality and Coverage of IMCI services. This was to be accomplished by increasing knowledge and compliance of health providers in C-IMCI services, improving access to preventive health services, and improving the C-HIS monitoring system. Tube Poka’s strategy for accomplishing this objective was to engage MOH staff in project activities and learn through on the job experience, and in turn to participate in the relevant District sponsored organizations. As a result Tube Poka’s relationship with the District Percentage of children 6-11m who received at least 1 dose of Vitamin A and children 12-23m who received at least 2 doses of Vitamin A in the previous 12 months, as evidenced by card *Baseline measurement is from MT in January 2007 KPC 11.1% 6.5% 60% Percentage of malnourished children who completed 12 days of Hearth who have achieve adequate (200g) or catch￾up (400g) growth for at least 2 months after Hearth Hearth program register NA 81% (22 of 27 children) 70% Increase HIV/AIDS prevention through knowledge and behavior change Percentage of mothers with children age 0-23m who cite at least two known ways of reducing the risk of HIV infection (RC) KPC 67.5% 85.7% 80% Percentage of children age 0-23m whose births were attended by skilled health personnel (RC) KPC 55.2% 72.7% 70% Rapid CATCH Indicators Not Included as Project Indicators Indicators Data Source Baseline Value Final Value EOP Target 1 Percentage of children age 0-23m who are underweight (- 2SD from the median weight-for-age, according to the WHO/NCHS reference population) KPC 29.9% 13.1% NA 2 Percentage of children age 0-23months who were born at least 24 months after the previous surviving child *BL is from 2006 monitoring survey KPC 39.5% 86.7% NA 4 Percentage of mothers with children age 0-23m who received at least two tetanus toxoid injections before the birth of their youngest child *Includes TT verified by card KPC 64% 59.3% NA 8 Percentage of children age 12-23m who received a measles vaccine *BL includes measles verified by card for children 9-23 months KPC 55.2% 91.6% NA 13 Assembly and the Chitipa District Health Management Team was very strong. The project staff presented reports at the regular District Executive Committee meetings (the technical arm of the District Development Secretariat). The Project Manager and Deputy Project Manager used these opportunities to share project impact stories, data and critical issues requiring stakeholder input. Through these experiences the Tube Poka staff modeled how to present data using PowerPoint and a LCD projector. MOH and District staff followed their examples and over time learned how to make similar professional presentations. An additional skill shared by project staff was training the MOH statistician and 15 other MOH staff in how to use EpiInfo. Staff at the health facilities gave credit to Tube Poka for increased care seeking and demand for services which contributed to the MOH obtaining an additional 14,000 ITNs for the district. MOH also appreciated the project’s participation during Child Health and Sanitation days, Child Health Week and World AIDS Day. By the second half of the project the MOH District Health Plan had incorporated into its interventions support for Tube Poka’s child survival activities. MOH staff participated in all of Tube Poka’s major activities. It began with their participation in the DIP planning process, including the DIP defense in the United States. One important learning experience was participating in the baseline health facilities assessment and reviewing the findings. Understanding the gaps in knowledge and services through this experience was much more effective than reading a report. As a result there was greater impetus on the part of MOH leaders to have the HSAs participate in the technical training for the Tube Poka staff. A total of 50 HSAs joined in the technical and P/D Hearth training sessions. The MOH’s IMCI Coordinator commented that the Hearth training was especially beneficial in his understanding of how to address malnutrition. Additionally, Tube Poka staff trained 25 new HSAs who were hired by the MOH in the third year of the project. Thirty HSAs participated in the PSI training on malaria and water treatment that was organized in collaboration with Tube Poka. Moreover, 94 HSAs participated in IEC training conducted by PSI. MOH staff actively participated in the mid-term and final evaluations. They learned how to plan, conduct, and analyze focus groups on the job with the project staff as well as learning how to implement quantitative surveys. During both the midterm and final evaluations MOH staff, such as the MCH Coordinator, had leadership roles, which increased the depth of the learning experience. An important capacity building component was strengthening MOH staff’s understanding of the Care Group model. One of the important learning experiences was the participation of key district level staff in doing the mapping for organizing Care Groups. They commented that the structured organizational process of organizing volunteers based on units of ten households was a new experience for them. They had only considered coverage in terms of attendance at outreach sessions and health facilities, not in terms of community demographics. The project’s promoters encouraged the HSAs to join them as they worked with the pastors and village leaders in organizing the volunteers into Care Groups, and later when the promoters conducted supervision visits. It is important to note that the selection of CGVs in the TPCSP was a variation of the original Care Group model with volunteers selected and then signed to a 10 HH group rather than selecting a CGV from each household block. 14 An unanticipated capacity building relationship was with the District’s Social Welfare Department. One of its major responsibilities is child welfare, including OVCs. One of the department’s problems was collecting reliable data on the number of orphans. Initially they relied upon TP baseline data, but later the TPCSP taught the department leaders how to organize surveys and tabulate data. Consequently the department was able to produce it own reports with reliable information, and obtain grants to expand its work. Another capacity building skill developed by Tube Poka was in community organization. At the community level the Social Welfare Department works with child protection committees. Because of the Care Group network Tube Poka had access to all the communities in the district and relationships with the respective community leaders. Tube Poka managers began collaborating with the department in planning teacher education and child protection interventions. As the organizations worked together the social welfare field workers improved their skills in organizing and training child protection committees. Additionally, the department was able to extend its coverage well beyond its budget by working with Tube Poka Promoters and the Care Groups. One tangible benefit for the department was when Action Aid did an assessment of child protection needs in the district they found that the communities were already organized. Action Aid obtained the data they needed and subsequently funded a project through the Social Welfare Department. Constraints While Tube Poka collaborated extensively with the MOH and other district organizations, some key elements of objective one were not accomplished. The program was able to train 25 new HSAs with UNICEF funding, but they did not complete the health and supervisory trainings as outlined in the DIP such as essential drug supply monitoring, establishing effective surveillance systems, basic problem solving approaches, supportive supervision and maintenance of good referral systems. In addition, the project was unable to conduct a second HFA at the end of the project to monitor progress of objective one. There were also some constraints at the district level that affected the full accomplishment of this objective. The MOH had no system for documenting adherence to IMCI and case management protocols, thus further complicating the tracking of these indicators. Another factor was under￾staffing at the district level to the extent that there was insufficient staff to supervise and process the little documentation that was available. The health budget for Chitipa was insufficient to meet the needs of the already limited staff, including funding the logistics for supervision and data management. The quality, and the extent, of coverage IMCI services were substantially affected by the district’s inability to train HSAs on the new protocol following a national decision to change the anti-malaria protocols. This resulted in a lack of malaria treatment at the community level. The expectation on the part of the MOH was that care seekers would go to the health centers where they would receive better quality care. However, the additional travel may have discouraged care seeking practices. Without drugs, community members stated that they saw little reason for going to the health posts, and were further discouraged from going to the health centers because of distance and costs. The lack of planning and budgeting for the full scope of this policy decision may have had an unintended consequence of reducing access to preventive services. 15 District budget constraints also affected immunization coverage. There were times when the HSAs did not have gas money for EPI outreach. Occasionally in such cases Tube Poka staff provided transportation, but they were not able to in all cases. The budget constraints also inhibited the hiring of a sufficient number of HSAs, so there were areas that were not consistently covered with immunization services. The lack of continuity in the position of District Health Officer (DHO) constrained Tube Poka’s progress in leadership development. Over the life of the project there were four different DHOs, with time gaps in filling the position. No DHO stayed for a whole year. Consequently every time a new DHO came in issues of policy, coverage, and training would have to be discussed all over again, which would disrupt Tube Poka’s collaboration with the staff. In summary, a strength of this project was its partnerships with UNICEF, PSI, and the District MOH. Their collaboration facilitated new grants and strengthened the MOH staff’s ability to manage development projects. Additionally MOH staff acquired skills in child survival planning and in qualitative research by participating in these activities with Tube Poka staff. MOH staff saw the value of a census based approach when working with communities, and came to appreciate the Care Groups as effective resources for promoting child survival. Staffing and budgeting constraints in the MOH limited the project’s ability to fully achieve success in Objective 1. Objective 2 The second objective was to develop sustainable community-based mechanisms to improve prevention and care-seek practices for childhood illnesses at household and community levels. The plan was to accomplish this through the creation, training and support of Care Groups, as well as training and support for pastors, Village Health Committees, and traditional healers. These community structures were to be linked to the HSAs with gradual co-management that would result in the HSAs taking over technical support for the Care Groups after the project’s end. The foundational strategy for achieving this object was establishing Care Groups that were integrated within existing institutions, including VDCs, HSAs, pastors, traditional healers, and the District MOH to create a sustainable support structure for the volunteers and further build the capacity of communities and the health system. The original project plan included a traditional Care Group structure with supervision conducted by the village leadership or Village Health Committee (if functional), with volunteers selected after the population census with one CGV per 10 HH block and included the creation of Pastoral/ Traditional Healer Care Groups to receive leadership and health messages. This model was altered to implement, recruit and supervise Care Groups through local churches as described earlier in Section B. The rational for involving churches is that they are community-based organizations that have well established ties to the community. The expectation was that Care Groups based out of sustainable local institutions would provide even greater sustainability beyond the proven Care Group model. In reference to involving community leaders, this was one of the strengths of this project. Pastors, and other community leaders, became involved through Tube Poka staff taking time 16 early on to explain the project, seek their counsel, and provide information to them on a regular basis. The project staff involved community leaders at all levels; pastors, village headmen, village development committees, and the traditional authorities (or chiefs). During the final field evaluation the evaluation team conducted 15 focus groups with village headmen. When asked if they received reports from Care Groups in their areas, most responded that they had. In a follow up question they were asked if they had been involved in Care Group meeting with pastors and the majority stated they had. In turn they were asked if the pastors reported on CGV activities and brought up health issues; they overwhelmingly said the pastors had done so. The evaluation team also conducted 15 focus groups with pastors. They were asked if they collaborated with village headmen to solve health issues. All of the pastors interviewed stated that they had done so. When asked which types of topics the pastors discussed with the village headmen the most common answer was health issues, followed by community development and food security. This provides an indication that pastors and village headmen were discussing health issues and even expanding beyond that to discuss broader issues of community development. It is likely that this interaction was influenced by the Tube Poka project, as pastors stated that prior to the project they rarely discussed community issues with the village headmen. An additional indication of community support was the village headmen’s statements regarding continuing to utilize the Care Group structure. They were asked who will support the care groups when the project ends and the vast response was both themselves and the pastors, while a majority also mentioned the MOH. It is interesting to note that the village headmen identified the pastors along with themselves as the ones who would support the Care Groups. This finding suggests that there is a partnership that exists between these two leadership groups in the community. While it is difficult to quantify focus group data, two overarching trends can be seen. First, is willingness on behalf of the village leaders and pastors to voice their support for CGVs and pledge to continue offering encouragement. However, it also appears that there may not be a depth of understanding of their roles and the roles of the CGVs that is necessary to truly sustain such a community structure. When speaking with the village leadership regarding the role of the pastors they overwhelmingly stated that encouragement is their primary role. Less than half responded that the pastors were to mobilize their church or help solve problems. Moreover, very few listed activities such as replace volunteers, receive reports, or provide discipline and none of them stated that it was the role of the pastor to supervise the Care Groups. This seems to be collaborated by the CGVs who when asked how they manage challenges responded almost unanimously to continue providing encouragement. When the village leadership was asked for the main challenges they encounter with Care Groups the most common answer was transportation, however other common answers included bad collaboration between the village leadership and the CGVs, CGV laziness, CGV dropout, and that the CGV did not have supplies such as WaterGuard. The pastors listed similar challenges 17 including transport, dropouts and CGV apathy. When the CGVs were asked what their main challenges the most common response was transportation, but close behind that was the need for incentives or allowances, followed by beneficiary demands and poor welcome in the households. Other statements of support, collaboration and mutual reporting with village leadership appear ideal. However, reports from staff are more tempered. While there is a desire for churches and village leadership to hold synergistic roles human frailties often prevent real change from occurring past the initial vote of support. Specific to the Malawi variation of the Care Group model was the selection of individual churches within a community by WRM that were given responsibility over the entire village. In areas with a strong central church that had good relationships with the village leadership the project seemed to thrive. However, if there were divisions or rivalries within the church, between other churches or between the village leadership the project faced difficulties. When these barriers could be broken down the transformational power of community cohesion was remarkable and these individual stories are inspiring. A development outside of the scope of the DIP was the organization of zonal committees based on the geographical boundaries of the traditional authorities. This was a successful component of the first Malawi CSP and the TP staff were committed to expanding this useful community mechanism. The purpose was to create an organization that could advocate for health care before political authorities, represent the CGVs before community leaders and Traditional Authorities, and ensure that Care Groups function as planned. The evaluation team conducted ten focus groups with representatives of ten different zonal committees. The committees met regularly; almost all had met in the last month a couple met twice. When asked if they received information on Care Groups the majority responded that they had. The zonal committees selected stated that they felt their role was to monitor and supervise Care Groups, problem solve, and encourage CGV. Their responses indicate a high level of involvement in Care Group matters, providing evidence of their commitment to supporting the Care Groups. In terms of plans for supporting the care groups after the project ends, all stated they had plans. The following is a list of the plans they identified:  Establishing a fund to support CGVs;  Continuing with encouragement;  Writing funding proposals;  Sponsor planning meetings;  Creating a cassava garden and donating maize;  Plan to become a formal community based organization (that could directly receive funding). While it is a positive sign that the zonal committees were discussing the issue of support for CGVs, there was the expectation on the part of some that outside funding would be needed. Another concern was that none of the focus groups referred to collaboration with the MOH as part of their sustainability. Nevertheless, the creation of these local committees with a focus of bettering life in their community is commendable and likely to make a contribution to supporting volunteers. 18 Another strategy for accomplishing Objective 2 was to train village health committees that would take responsibility for identifying and solving health issues. After the project began it became apparent that there already existed a community structure that could address health issues, and that was village development committees (VDC). Consequently, Tube Poka staff shifted their focus to working with these committees. They are organized by the village headmen and the traditional authorities. They meet almost every month and are held as open forums with community participation. The Tube Poka staff members were invited and the promoters regularly attended the meetings. In sustainability indicators collected for the final evaluation (July 2009), 70% of the promoters reported attending at least one VDC meeting in the last month. Initially the VDCs were not accustomed to address health issues. As the promoters, then the pastors and the zonal committees, began to bring up health issues, health began to be a regular topic. As of July, project supervisors reported that79% of the VDCs had discussed health issues in the last quarter. While the project worked with these local committees they did not provide any health or leadership trainings as planned for in the DIP. The traditional authorities were another level of community support for the Care Groups and for health development. These men are traditional chiefs who govern large areas. They deal with civil issues such as land ownership, civil disputes, and community development. Chitipa District has five of them and the evaluation team interviewed four. All of them knew the Tube Poka promoters and supervisors who worked in their area. They stated that the project staff reported to them on a regular basis. They stated that health issues most frequently discussed were malaria prevention and sanitation/hygiene. WRM obtained funding from a US donor to install eight boreholes and the pertinent traditional authorities expressed a high level of appreciation for this resource. When asked what changes they have observed in their area of authority, most the TAs that were interviewed replied that they felt there were fewer deaths, increased care seeking and some felt that there was more deliveries at the health centers and less malaria. One traditional authority commented to an evaluator that he attended many fewer funerals and as a result had more time to devote to development issues. They all stated that the project made life better for their people. The one community leadership group whose participating diminished over time was that of the traditional healers. While Tube Poka promoters tried to engage them, they were resistant. Over time the promoters reported that many of them stopped practicing because people were relying more on the CGVs for advice, and were seeking treatment at the health facilities. However, when CGVs were asked why caretakers would delay treatment for pneumonia the overwhelming answer throughout the district was because the caretakers go to the traditional healer. HSA training and participation with the care groups was another community-based mechanism for accomplishing Objective 2. They were considered community-based because the staffed health posts and lived in the community. The district MCH Coordinator was responsible for training them in C-IMCI according to the authorized MOH curriculum. Tube Poka did take responsibility for training 25 newly hired HSAs in how to run village clinics, as part of a UNICEF Accelerated Child Survival funding, while the district MOH trained another 35. Part of the plan was for the HSAs to be trained on the job in the Care Group model. They were to accompany promoters on training and supervision sessions with the Care Groups in their 19 catchment area. Over time the expectation was that they would take over supervising the Care Groups by the end of the project. While the Tube Poka promoters invited HSAs to accompany them, there was no explicit and deliberate process for turning over Care Group supervision to the HSAs. In July 2009, supervisors reported that only 30% of the Care Groups had been visited by an HSA during the previous two months. Additionally, only 31% of the HSAs conducted household spot checks with a promoter in the last month, and only 25% attended a VDC meeting with a promoter in the last two months. This relatively low level of involvement of HSAs at the end of project indicates that there may be little supervision by the HSAs when the project ends. In addition to on site interaction with Care Groups, the plan was to provide HSAs with leadership and technical trainings. New HSAs were trained by TPCSP with UNICEF funding and TP also trained two HSAs at each intervention training (14 HSAs were trained on one topic each); however, a more wide reaching skills and leadership training was not conducted. It is important to note that Tube Poka and MOH leaders met in May 2009 to draft a policy statement regarding HSA supervision of Care Groups; however, the HSAs missed the opportunity for in depth training and relationship building with the Care Groups. It is possible that by adopting a formal policy the level of HSAs’ supervision will increase, but unfortunately it would have been more effective if the promoters could have done on-the-job training with them. However, this policy is further complicated by the project’s lack of data regarding the specific number of CGVs and their rate of turnover, both critical in helping the MOH determine the level of support and continued training that would be required. As of now, the policy statement was still in draft form. Objective 3 Tube Poka’s third objective was to promote key family practices for child health. This was to be accomplished by increased knowledge of danger signs, improved home management of sick children, increased immunization, malaria prevention and treatment, increased care seeking, improved childhood nutrition, and increased knowledge about HIV/AIDS prevention. The strategy for communicating the health messages was for the CGVs to conduct home visits twice a month in to every household in the project area. Additionally, the training provided to pastors and village leaders would result in a wide dissemination of health messages across sectors. The primary source for analysis of this objective was the KPC findings. The findings and corresponding analysis are presented in this report by intervention areas. In the areas of malaria and pneumonia control, the KPC findings indicate that the project improved from the baseline to the final but it did not meet the EOP targets. The following graph presents the final KPC findings. 20 Chart 1: KPC Findings for Malaria and Pneumonia Control The above chart indicates that 43.7% of the children 0-23 months old slept under a net. There is no statistical difference between the baseline, midterm and final and it is lower than the EOP target of 60%. One potential factor that could have influenced this finding is the seasonal variation in using ITNs. The baseline was conducted in May, the mid-term was in January at the height of the hot season and the final was conducted in July, the coldest part of the year when mosquitoes are less bothersome. In discussions with the Tube Poka staff they stated that use of ITNs does vary by seasons. They stated that people, including themselves, tended to not sleep under an ITN during the cold season because the population of mosquitoes was substantially lower. Of the households who reported owning nets, the percentage of children sleeping under an ITN the previous night was 71.3% at Midterm and 49.6% at the Final. In addition, the project does have some monitoring data that indicated increased usage during March 2009, perhaps as high as 70%. In reference to ITN ownership, Tube Poka did not purchase nets for distribution, but it did collaborate extensively with the MOH in distributing their nets. ITN utilization efforts were bolstered by multiple net distribution campaigns by the MOH which provided 14,000 free nets. The Church of Central Africa Presbyterian (CCAP) provided 1,000 nets and World Relief Malawi distributed 500 free nets. Net ownership increased dramatically from 44.2% at baseline to 88.0% of household reporting that they owned a net during the final KPC. Chart 3 above shows an increase in care seeking behavior for suspected cases of malaria and pneumonia from the baseline through the final, but the project did not reach its EOP targets. In an effort to understand factors that may have affected care seeking, the evaluation team reviewed the mother’s answers to danger signs during the Final KPC. While 93% recognized fever as a danger sign requiring treatment, their ability to recognize other signs was at a moderate to low level. Of particular concern is that only 21% identified rapid breathing as a danger sign. If they would have been prompted perhaps more would have identified this sign, nevertheless ARIs are the second leading cause of death according to project records. Thus it is important that mothers 0 10 20 30 40 50 60 70 80 90 100 Slept under ITN Malaria Tx at HC within 24H Pneumonia Tx at HC within 24H Baseline Midterm Final Target 21 are acutely aware of this sign and the importance of immediate treatment. These findings were supported by information collected during 15 focus groups with mothers where most of the focus groups listed fever, but other important signs were less listed far less often. Chart 2: Signs of Childhood Illness Identified by Mothers during the Final KPC To further explore possible reasons for the relatively low care seeking, the evaluation team asked the mothers about the things that they needed to consider before taking their child to a HF for medical care. They had a wide range of responses, but many of the mothers responded that they had to consider food preparation for their family while they will be away and if home care is a possibility. Other common responses were concerns for transport/distance, having clean clothes, if they have the child’s “Health Passport”, and getting permission or money from their husbands. Mothers have to take a lot of things into consideration when deciding to take a sick child to a HF; it is not a simple matter of just taking off for the HF when a child gets sick. A mother has to balance a lot of factors which may influence her decision whether or not to take her child to the HF. In the case of signs of pneumonia, there are additional factors which may influence care seeking. When the CGVs were asked to state the reasons for low care seeking for pneumonia, they overwhelmingly stated that mothers first went to traditional healers. This was an unexpected response, given that staff indicated that traditional healer were practicing less because of the child survival project. Upon analysis by the Tube Poka staff, they identified some cultural factors that may have influenced mothers’ behavior. In their experience they confirmed the CGVs observation that many mothers still go secretly to the traditional healers and there is a tradition of using specific herbs for respiratory illness and mothers may have first tried these before going to the HF. However, only 56% of children with rapid or difficult breathing were ever taken to a HF, regardless of timing. There may have been a language barrier as staff stated that pneumonia was a difficult term to translate in many of the local languages. In some cases the word that was used for pneumonia could refer to anything with a sharp pain. In other 93% 60% 53% 51% 45% 21% 17% 11% 9% 2% 0% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% 22 languages the term was vague and nonspecific because there was no equivalent word for pneumonia. Another important factor is that the MOH in Chitipa restricted the use of pneumonia treatment (sulfa drugs) by HSAs. This prevented treatment for pneumonia at the community level and requires that mothers go straight to the health center and bypass the local health post. While the children may receive better care at the health center, the mothers interviewed reported that this additional distance was a barrier to prompt care seeking. In discussions about this with the UNICEF representative, and later the USAID Mission, it became clear that Chitipa’s policy was not the national policy. UNICEF anticipated implementing a program of training the HSAs to manage sulfa drugs at the health posts. In the intervention of diarrhea control, the KPC findings are presented in the following chart. Chart 3: Diarrhea Control Indictors The project accomplished its EOP target for controlling diarrhea with ORT. As observed in the chart above there was a substantial increase in using ORT from the baseline to the midterm. An important contribution came from the partnerships with PSI and UNICEF. These agencies contributed a stock of free ORS packets that were distributed by the CGVs. Additionally, PSI contributed social marketing materials about using ORS packets. Chart 3 above indicates that hand washing at the four crucial times (after defecation, after helping a child who has defecated, before food preparation, and before feeding a child) was not done. The Malawi MOH policy is that hand washing should occur at five times; after using the toilet, after changing a child’s nappy, before feeding a child or breastfeeding, before preparing food, and before eating. Only 15.0% of caretakers reported washing at all five of these times. Analyzing specific responses to questions about when household members washed hands indicated that while 86% of the caretaker’s reported hand washing after defecation, the 0 10 20 30 40 50 60 70 80 90 100 Diarrhea tx with ORT Hand washing at 4 critical times Baseline Midterm Final Target 23 percentage reported for the other three all fell below the project target of 60%. The following table presents these findings of those who washed, using soap. Table 3: Specific Occasions When Respondents Washed Hands Occasion MTE Final After defecation 75% 86% Before eating 64% 71% After helping a child who has defecated 52% 68% Before food preparation 43% 56% Before feeding a child 21% 38% Percent who responded 4/5 times from the above list NA 55% These data indicate that a high proportion of respondents washed their hands after defecating and before eating. Additional data in support of hand washing are that 88% of respondents stated they used soap and the interviewers observed soap in 72% of the households. These findings are supported by the cultural tradition of washing hands before eating. It is customary in Malawian society bring a basin and a cup to the table before beginning to eat. Water is poured from the cup on to hands above the basin. The person washing makes hand washing movements as the water is being poured. The change that Tube Poka introduced was using soap during hand washing and the inclusion of additional times when it is critical to practice hand washing. The project’s nutrition indicators measurable by the KPC surveys were EBF, continued breastfeeding during the ages of six to nine months and vitamin A dosages. The KPC findings for these indicators are found in the following chart. Chart 4: KPC Nutrition Findings The high increase in EBF is a notable achievement for the Tube Poka project. The rate increased 100% by the mid-term and was sustained to the end of the project. The CGVs taught mothers about EBF and complementary feeding during their household visits as well as explaining the value of vitamin A and supporting the MOH in its campaigns for dispensing capsules. 0 10 20 30 40 50 60 70 80 90 100 0-5m EBF 6-9m Continued BF VitA by card VitA by recall Baseline Midterm Final Target 24 Breastfeeding through at least two years of age is also strongly supported by the District MOH. The District Hospital has multiple signs reading “Chitipa District Hospital is a Baby Friendly Hospital. We support, promote and protect breastfeeding” thus providing a consistent message from the village to the district level. In Chitipa, vitamin A distribution is conducted mainly through health campaigns held every six months. The DHMT purposefully does not ask mothers to bring their child’s health card so they do not discourage mothers who do not have cards. It is very plausible that this documentation issue has greatly influenced the program indicator for vitamin A since the vaccination had to be verified by documentation on the child’s health card. In order to obtain an indication of dosing, the KPC interviewers asked the respondents to recall if the child under two years of age had received a vitamin A capsule in the last six months. As reported in the above chart 89% stated in the affirmative. This high percentage indicates that caregivers knew about vitamin A and that it was distributed during vaccination outreach sessions. Another nutrition component was implementation of the Hearth program. Hearth is a community based program that teaches mothers how to prepare nutritious meals with locally available foods for children who are mild and moderately malnourished. Mothers meet in a convenient location daily with their children and each brings local food. They learn about food preparation, cook a nutritionally balanced meal together and feed their children. As project staff and CGVs, along with the participating mothers, track the weight gain of the children during the two weeks of the Hearth program, they also discuss feeding practices, give instruction on nutrition, and help the mothers understand that the improvement in their child’s health is clearly tied to nutritional changes that they are able to make. All of the staff were trained in Hearth, however it was completed in only two villages. The promoters however, incorporated the Hearth concept of using a variety of locally available foods for good nutrition into the CGVs’ training. In a summary meeting during the final evaluation with the staff they commented that this nutrition message was one of the most valuable things that was communicate to mothers. The tradition in the culture is to prepare meals that are predominately based on a corn meal mush or porridge with little variety. While seasonal fruits and vegetables are available, traditionally people do not eat a lot of these foods. Some of the staff observed that mothers were introducing more fruits and vegetables into their children’s meals because of the Hearth concepts. Despite the lack of Hearth implementation, the findings from the KPC survey indicate that the percent of under-weight children (-2SD from the median weight-for-age) decreased from 30% at the baseline to 15% at the final survey. Two of the project’s indicators that addressed the care of the sick child were knowledge of care seeking signs and home management of the sick child. On the preventive side, the project tracked childhood immunization. The data from the KPC for these indicators is presented in the following chart. 25 Chart 5: Indicators for Managing the Sick Child and Childhood Immunization As shown above, the project surpassed its target for mothers knowing two or more danger signs. The target was surpassed at the mid-term and the rate continued to increase to the end of the project. The CGVs were the principle channel for communicating the messages about danger signs, which they did when they visited the households. It is remarkable that they communicated the messages without any IEC aids. Tube Poka intended to produce picture-based educational material for use by the CGVs but these materials were not produced until the end of the project. The high rate of knowledge is testament of the CGVs ability to communicate verbally and through songs and drama. The indicator for giving increased fluids and continued feeding to a child during episodes of illness missed the EOP target by 27 percentage points, reaching only to 23% by the end of the project from the baseline rate of 4%. When discussing this issue with mothers they stated that when sick, children refused to eat or drink most of the time. Children would fuss and resist and mothers felt that they did not want to badger them. They were asked the reasons for giving more food and fluids and they had no trouble articulating the correct responses. They felt that insisting made the child feel worse so they resisted doing so. This dichotomy between knowledge and actual behavior change is likely a result of the low household visitation rate as indicated in the final KPC. The final immunization rate of 73% did not meet the EOP target, as shown in the preceding chart. Documentation for this indictor could have been affected in part by the lack of immunization cards, although the percentages of children with cards increased from the midterm to final. Another possible factor that may have affected immunization coverage was understaffing of HSAs at the health posts. Their responsibility is to conduct monthly vaccinations at their assigned Health Posts. As discussed in more detail in the following section on sustainability, there is a gap of approximately 59 HSAs in the district. This means that numerous villages do not have consistent vaccination services. The health center staff members try to cover the health posts without HSAs but at times they are not able to serve all these 0 10 20 30 40 50 60 70 80 90 100 Knows 2+ Danger Signs Increased Fluids and Feeding Fully Immunized by 1 year Measles by recall Baseline Midterm Final Target 26 villages during the rainy season or because of lack of petrol for their motorcycles. The Tube Poka staff provided transportation when they were able to do so, but now that the project is ended this assistance will no longer be available. Nevertheless, a when caregivers were asked to recall if their child received a measles vaccine, 92% responded that they had, indicating possible documentation issues. The final intervention area for this project was HIV/AIDS. The focus was on mothers knowing ways to reduce the risk of HIV infection and on increasing births attended by trained personnel. The KPC findings are presented in the following chart. Chart 6: HIV/AIDS Indicators The project accomplished both of its indicators for this intervention. The rate of mothers’ knowledge of two or more ways to reduce HIV infection increased from 68% to 86%. This represents a substantial proportion of all the caregivers in the district, and indicates that caregivers in the district have a base of knowledge about HIV transmission. While this gradual increase in HIV knowledge is to be celebrated, the project did not implement the HIV lessons until the last year of the project. Therefore, the plausibility of project impact on the improvement, especially from baseline to midterm is uncertain. The percent of births attended by trained personnel also increased significantly from 55% at baseline to 73% at the final evaluation. Overall Factors Affecting All Objectives The project had difficulty with financial management. There were challenges in getting the needed resources to the project site inhibiting project activities including training and supervision. Moreover, funds were not spent in accordance with the DIP budget. Specifically concerning is an under spending in fiscal years 2006-2008, with a dramatic spike in spending in 2009. For more information, please refer to Annex 3. F. Potential for Sustainability and Replication 0 10 20 30 40 50 60 70 80 90 Know 2+ ways of reducing HIV risk % Births attended by skilled personnel Baseline Midterm Final Target 27 Progress Towards Sustained Outcomes This section begins with a summary of the project’s sustainability framework followed by a discussion of the potential for, and constraints to, sustainability. Table 4: Tube Poka Sustainability Framework Project Objectives Planned Key Activities Status of Activities Strengthen the capacity of Chitipa District’s health system to improve the quality and coverage of IMCI services a) Improved knowledge of providers b) Improved access to preventative health services c) Community-Health Information System (C-HIS) Monitoring •Review existing facility-based C￾HIS. •Train and supportively supervise HSAs and HC staff on a C-HIS that reaches the household level. •Joint M&E planning and implementation with MOH. •Support outreach activities, HSA training and supervision by CSP and MOH staff. •Train and support VHCs to monitor HSAs and CG volunteers, and interpret HIS results. •Train 25 HSAs to institute Village Clinics in hard to reach places. •2 HSAs were trained with project staff per health intervention, totaling 12 HSAs trained on one intervention each. •Meetings with health center staff revealed that health centers have noted increased care seeking in the project area. •While portions of the C-HIS has been on-going, data collection and quality have been challenges. •Through UNICEF funding, 25 new HSAs were trained. Develop sustainable community-based mechanisms to improve prevention and care-seeking practices for childhood illnesses at household and community level •Establish Care Groups for training and sustaining community health volunteers & involve HSAs from the beginning. •Strengthen (with leadership and skills building) community support systems: village health committee members, pastors, traditional healers, Village Clinics and other community-based groups. •Care Groups have been established and have been trained. •Community leaders received health knowledge through their interaction with the project. Promotion of key family health practices for child health •Establish Care Groups. •Utilize multiple methods and channels for BCC. •Update training materials and train care group volunteers, community leaders and MOH staff the same key messages using non￾formal techniques •Care group volunteers were established and trained on all interventions •Training curriculum and picture cards were printed at the end of the project for continued use by Care Group volunteers. 80% HSAs receive C-IMCI training •Meet with MOH to establish partnership. •Train HSAs with staff. •A limited number of HSAs were trained along with the staff •Through UNICEF funding, the project trained 25 new HSAs in village clinic implementation 80% pastors receive training in C-IMCI •Meet with pastors to establish relationship. •Train pastors on C-IMCI. •Many meetings were held with pastors •Informal health training occurred as they interacted with project staff 60% traditional healers receive training in C-IMCI •Meet with traditional healers to establish relationships. •Meetings with traditional healers declined •Project staff report less and less of them continued practicing. 28 CGs with volunteer attendance of at least 70% •Train CG volunteers in their role in the community and the importance of regular attendance at CG meetings. •Monitor with supervisory checklists. As of July, 2009, promoters report that 62% of CGs have volunteer attendance of at least 70% 60% VHC met in the last 2 months •Meet with village health committees to provide leadership training and support. [The project transitioned to working with the indigenous structure of Village Development Committees (VDC)]. • VDCs meet monthly and TP promoter regularly attends and reports, along with pastors, zonal committee members and CGVs. According to focus group data, only a few of the VHCs had met in the last 2 months. 70% of CGs supervised by trained HSAs •Partner HSAs and promoters to establish relationship with CG. •Promoters mentor HSAs and HSAs help train and supervise CGs for eventual phase out of the promoters. As of July 2009, promoters reported that 30% of the care groups were supervised and supported by an HSA. 70% of households visited by their volunteer in the last 2 weeks •Train CG volunteers on the importance of their role in the community and regular visits to their neighbors. Final KPC: 17.3% of mothers with children U-2 visited by CGV in the last two weeks. Another source of ongoing community support was the buy-in to heath promotion by the school teachers. They participated in the PSI and UNICEF promotion of water purification and the use of WaterGuard. UNICEF gave them WaterGuard units and they showed the children how to purify water and serve potable water each day. Teachers also modeled how to collect and store rain water. This type of involvement was an indication of the extent to which community leaders became involved in health care. The pastors, village headmen, traditional leaders and teachers are independent from Tube Poka and the MOH. One of the project strengths is their strong relationship with the District MOH. The MOH was supportive of the Care Group model as evidenced by encouraging HSAs to work closely with project promoters with the goal of transitioning the training and support of CGs to the HSAs at project end. In April 2009, the DHMT drafted a policy statement clarifying the role of HSAs to include supervising and maintaining the Care Groups. The policy statement was to have been presented to the whole District Council for approval in September. A key policy in the document is that HSAs have been designated as the supervisors of the Care Groups in their area. A number of constraints however, will affect the MOH’s ability to support the Care Groups. One is that few of the HSAs have experience working with Care Groups. The Tube Poka supervisors’ monitoring report for July 2009 indicated that only 30% of the Care Groups had been visited by a HSA in the last quarter. Only 28% of the HSAs attended a VDC meeting with a promoter in the previous two months. The original plan in the DIP was that all the HSAs were to accompany promoters on training and supervision visits and after the mid-term the HSAs were to conduct the primary supervision of the Care Groups. The above data indicate that this did not 29 occur. Consequently the HSAs do not know the Care Group model well and what is involved in supervising the Care Groups. A second constraint is that the HSAs have not been involved in training CGVs, so they will be ill equipped to train replacements. Thus retraining volunteers who drop out will be an ongoing challenge. One of the complications is that some CGVs periodically leave their communities for extended periods of time. For example the Care Groups near the international borders with Tanzania have members with dual citizenship as a result of intermarriage. At times they leave home to stay in the neighboring country, leaving volunteer responsibilities behind. Agricultural practices also contribute to migration; in Chitipa Northwest, Nthalire and Wenya supervision areas, some people leave their homes during dry season to do winter cropping in Zambia. Because of the HSAs’ lack of experience in working with Care Groups, it will be a challenge for them to address these gaps in CGVs participation or more importantly, to know how to advocate for the appropriate selection of volunteers to avoid these issues in the future. A third constraint is under-staffing at the health posts. The MOH uses the formula of one HSA per one thousand people to determine the number needed. Based on this formula there should be 180 HSAs working in health posts throughout the district. At the time of the final evaluation the district had 148 HSAs; however 27 of them were specialized, working at health centers or the district hospital, resulting in 121 HSAs working in health posts. The result is a gap of 59 HSAs. The current number of HSAs is not sufficient to provide coverage for the whole district, much less to supervise all of the Care Groups. A fourth constraint is that Tube Poka staff did not monitor coverage of all the households in the district, thus there is no way to reliably know the number of CGVs, identify gaps in household coverage by the CGVs, and to monitor volunteer attrition. Without this monitoring the potential exists for the system to gradually not provide coverage for large areas of the district. It is also problematic as the DHMT does not have a reasonable estimate of the amount of resources that will be required to support and train CGVs. It is unfortunate that Tube Poka will turn over the Care Group system without adequate monitoring tools. Contribution to Replication or Scale-up Two of Tube Poka’s key partners gained valuable experience in working with this project. In the north of Chitipa District, Care Groups were used by PSI/Malawi to implement its Integrated Diarrhea Prevention Program (IDPP) which promoted Water Guard and Thanzi ORS. UNICEF gained a positive experience with Care Groups through implementing its Accelerated Child Survival and Development activities. Both organizations commented on the efficiency of the Care Group model for rapidly disseminating innovations such as WaterGuard. The potential exists for these agencies to promote the use of Care Groups in other projects in Malawi. The wide dissemination of the Care Group approach at national level has attracted a high degree of interest from NGOs working in Malawi. Care Groups have been used since 2007 in the I￾LIFE consortium to reach over 60,000 families with maternal and child health and nutrition messages. Members of this consortium include CARE, CADECOM/CRS, Save the Children, Africare, Emmanuel International, Salvation Army and World Vision. The director of I-LIFE learned about care groups from WR’s child survival project in Mozambique. Additionally, key 30 members of I-LIFE visited Chitipa at the beginning stages of their CG implementation and initial results are very positive. Attention to Equity Participating as CGVs gave women in Chitipa opportunities that they otherwise would not have had. Over the life of the project nearly 4,000 CGVs were trained, and approximately half of them were women. They acquired knowledge and skills about health care that gave them a position of influence in their communities. The women volunteers gained credibility as CGVs such that they were invited to give reports at VDC meetings and participated in discussions about health issues Partner agencies donated literature and the pastors used this as an opportunity to teach women how to read. During the community focus groups a few volunteers shared with the lead evaluator that Care Groups would meet weekly instead of bi-weekly. Every other week they would conduct Care Group business as planned, but in the intervening weeks they would have literacy classes. The classes were the initiative of the pastors and volunteers; Tube Poka staff were not involved. This is another example of people in Chitipa District taking ownership of their own development. At the professional level, Tube Poka provided opportunities for a few women to obtain jobs, and have positions of leadership as health promoters and supervisors. In these positions they taught and supervised men, who were either volunteers or health promoters, being supervised by women supervisors. The women staff even participated in motorcycle driving classes, which was something thought of as only for men. The women staff members gained skills that made them marketable for other development agencies or for positions in government. It is important to note that all of the staff members were from Chitipa District, giving employment and leadership positions to people from one of the poorest districts in the country. Role of Community Health Workers The CHW model used in this project was based on Care Groups. As describe earlier, the CGVs were organized into groups of ten, with each CGV assigned to visit and counsel 10 households. The care groups were to meet twice a month. One meeting at the end of the month was for reporting and completing their registries. The second meeting was for training. Health promoters trained the CGVs in their Care Groups. When they completed the full cycle of training in the CS interventions, the promoters continued the pattern of training visits and use them to refresh content or address problems and issues. Prior to the CS project, there were no health volunteers; through the Care Group system the project organized and trained over 3,600 CGVs. Chitipa District is highly Christianized and every village has at least one church. World Relief Malawi decided to use the churches as a community resource and engaged the pastors, along with the village headmen, in organizing the Care Groups. The identity of the volunteers is with their Care Groups with leadership from the pastors. The Tube Poka health promoters’ role was to supervise and train the CGVs. As discussed earlier, the plan was to gradually turn over the supervision to the MOH and HSAs. Unfortunately this did not occur for the most part. Organizing the volunteers into Care Groups had a number of advantages. The group provided social support for the volunteers, thus making them accountable to the group, not some agent 31 external to the community. Another advantage is the group dynamics make it easy for volunteers to help each other, especially those who have difficulty reading. Volunteers also covered for each other when for example, one was pregnant and could not make all her household visits. Additionally, they have been known to cover for each other when a volunteer had to leave on business, was sick, or had to care for sick family members. The Care Group model is an efficient way to diffuse innovations such as WaterGuard. The CGVs can be reached in a relatively short period of time because they meet biweekly. The system of assigning a volunteer to every household with a child under five or woman of reproductive age means that it is possible to contact virtually all households within a month or two. PSI and UNICEF staff commented that they had never seen WaterGuard diffused as rapidly as it was in Chitipa. Training is facilitated using the Care Group model. Care Groups are trained in their community, which contributes to making the training more relevant. It is also cost efficient because volunteers do not need to be transported and given meals, and there is no rent for training centers. The training process is slower this way because of the short time for training, and the promoters only train one of their groups at a time. For volunteers who have limited schooling experience however, the learning process is enhanced by introducing content a little at a time. The second type of CHW was the HSAs. They are trained and paid by the MOH, and are part of the national system for community health care. The MOH has a standard training curriculum and policies that define their role and responsibilities. Tube Poka had a supportive relationship with the HSAs but did not supervise them, as they were directly responsible to MOH. The Tube Poka project intended to engage the HSAs in supervising the Care Groups, but there was never a policy in place until the end of the project for HSAs to do so. Tube Poka directors invited a few HSAs to participate in the technical training for the project staff, and a couple also participated in the health facility assessment, and the mid-term and final evaluations. Contribution to Global Learning The contribution made by this project is the Care Group model and its contribution to replicating Care Groups in Malawi. The benefits of the model are that all households in a catchment area are visited regularly by volunteers, the volunteers are supported by Care Groups, and the process of organizing and maintaining the groups generates a high degree of participant ownership. Engaging community leaders, pastors in this case, to help organize and supervise the care groups was a critical feature of this project. While few project areas will have the level of church support that exists in Chitipa, the component of community ownership of the Care Groups is still possible using community structures such as village headmen, and community councils. G. Conclusions and Recommendations The primary conclusions regarding this project are as follows. 1. The TPCSP Care Group model was effective in communicating health messages to a society with low literacy levels, and effective for disseminating innovations such as ORT and WaterGuard. High percentages of mothers and caregivers knew signs of illness that warranted taking a child to the health center, they knew the methods of transmitting HIV, 32 and they adopted EBF. In a matter of a couple of months the care groups distributed 37,000 units of WaterGuard throughout the district. 2. The Care Group model generated a high level of interest among other NGOs in Malawi. At least three implemented care groups and a number of others were in the planning stage. The NGO Synergy consortium adopted a resolution endorsing Care Groups. The Tube Poka project served as model for other organizations in the country. 3. The Tube Poka project never did implement a monitoring system for tracking the fidelity of the Care Group model. Consequently when it turned over the project to the MOH it was not able to report gaps in Care Group coverage of households and identify where CGV positions needed to be filled with new volunteers. Without a monitoring system it is possible that Care Groups could dissolve without anyone’s knowledge. 4. The HSAs were never fully integrated into supervising the care groups. It is unrealistic to expect that after Tube Poka staff member leave, the HSAs will be able to automatically pick up supervision skills and responsibilities. A danger exists that supervision will be haphazard and that Care Groups will not regard themselves as being linked to the MOH health care system. While the community vocalized ongoing support for the Care Groups, they are not able to train the volunteers and provide technical supervision. For the Care Group model to function most effectively, there should be a partnership between village leadership and the MOH with ongoing support provided by influential community-based organizations. 5. This project sought to implement Care Groups through individual community-based organizations with the support of village leadership rather than implementing through village leadership with the support of community-based organizations. However, this alteration of the Care Group model did not result in the level of impact hoped for, especially as compared to the first WR Malawi CSP. Nor does it appear to have improved likelihood for sustaining activity as a result of being based in a community￾based organization. 6. The project had difficulties managing funds which had a direct impact on project activities and subsequent performance indicators. Funds were not spent according to the DIP budget and multiple activities were not performed. Recommendations Following are recommendations for follow up after the project ends, given that WRM expects to continue working in Chitipa. 1. WRM should if possible, conduct training for HSAs on the model. HSAs should learn how to supervise Care Groups, how to document fidelity of the model, and how to incorporate new volunteers to fill the place of dropouts. 2. If WRM continues to work with pastors in Chitipa, it should include teaching and encouragement on maintaining Care Groups as originally planned in the DIP. 3. World Relief should investigate financial and programmatic lessons learned before investing in another long term project. 1 Tube Poka Project Census, Chitipa, Malawi 2006 Annexes~ Page 1 Annex 1: Results Highlight Strengthening the capacity of the ministry of health in child survival projects is a challenge because these projects target poor, underserved areas. One important implication is that the ministry of health in these areas is often underfunded and understaffed. This weak infrastructure makes it difficult for an NGO to sufficiently strengthen the MOH’s capacity to sustain child survival interventions. Chitipa District in Malawi is a case in point. One example is that during the rainy season data reports from MOH health posts were not obtainable from mountainous, remote areas, thus creating gaps in the monitoring system. Another factor was under-staffing at the district level to the extent that there was insufficient staff to supervise and process the little documentation that was available. Tube Poka’s strategy for helping the MOH in light of these constraints was based on intensive relationship building. MOH staff participated in all of Tube Poka’s major activities. It began with their participation in the DIP planning process, including the DIP defense in the United States. One important learning experience for MOH staff was participating in the baseline health facilities assessment and reviewing the findings. Understanding the gaps in knowledge and services through this experience was much more effective than reading a report. The staff was diligent in engaging MOH staff in project activities and helping them learn through on the job experience. MOH staff participated in all the project’s training programs; they joined in all research activities and both the mid-term and final evaluation. In turn the project managers participated in all the relevant District sponsored organizations. They attended every quarterly District Assembly meeting and reported on the project’s activities. They presented reports at the regular District Executive Committee meetings (the technical arm of the District Development Secretariat). The Project Manager and Deputy Project Manager used these opportunities to share project impact stories, data and critical issues requiring stakeholder input. Through these experiences the Tube Poka staff modeled how to present data using PowerPoint and a LCD projector. MOH and District staff followed their examples and over time learned how to make similar professional presentations. As a result Tube Poka’s relationship with the District Assembly and the Chitipa District Health Management Team was very strong. MOH staff at all levels came to trust Tube Poka staff and regarded them as genuine partners. MOH staff learned how to plan, conduct, and analyze focus groups on the job with the project staff as well as learned how to implement quantitative surveys. During both the midterm and final evaluations MOH staff, such as the MCH Coordinator, had leadership roles, which increased the depth of the learning experience. The staff at the health facilities gave credit to Tube Poka for increased care seeking and demand for services which contributed to the MOH obtaining an additional 14,000 ITNs for the district. Collaboration between the MOH and WRM resulted in additional funding from PSI and UNICEF. The MOH gave public recognition to Tube Poka during national holiday celebrations such as Child Health and Sanitation days, Child Health Week and World AIDS Day. By the second half of the project the MOH District Health Plan had incorporated into its interventions support for Tube Poka’s child survival activities. Annexes~ Page 2 Annex 2: List of Publications and Presentations Related to the Project Publications CORE Group, April 2009. Community Approaches to Child Health in Malawi: Community Integrated management of Childhood Illness (C-IMCI) Framework Presentations “A Snapshot of IDPP Monthly Report for July 2008.” Victor Kabaghe, World Relief Malawi Meeting with PSI, Chitipa, Malawi August 2008 “World Relief Malawi Tube Poka KPC Findings” Richard Thindwa, Deputy Director TPCSP USAID/Malawi Synergy Meeting February 2008 “The Care Group Model” Victor Kabaghe, Director TPCSP USAID/Malawi Synergy Meeting March 2008 “Community Approaches to Child Health in Malawi: Community Integrated management of Childhood Illness (C-IMCI) Framework” Melanie Morrow, World Relief CORE Spring Meeting, Atlanta, Georgia April 2008 “Control of Diarrheal Disease in the Tube Poka Child Survival Project” Cholera Postmortem Integrated Disease Surveillance and Response (IDSR) Meeting, Chitipa District Hospital June 2008 “Monitoring Survey Results” Victor Kabaghe, World Relief Malawi Chitipa District Health Management Team Meeting, Chitipa, Malawi May 2009 Annexes~ Page 3 Annex 3: Project Management Evaluation Planning The Tube Poka Child Survival Project actively engaged the Chitipa District assembly, health department and welfare department staff in the planning process throughout the life of the project. In the DIP planning process the above groups were joined in the planning process by village headmen, health committees, and clergy. An important issue brought up in this planning was that of incentive-based volunteerism compared to altruistic volunteerism. Initially there was the expectation that with foreign funding, volunteers would receive cash or material incentives. The distinction between these two was made clear, and the group came to the conclusion that incentive-based volunteerism limits the ability of programs to have a lasting impact in the community. While the issue continued to arise to some degree, the ample discussion of this issue during the DIP planning provided a foundation for maintaining true volunteerism. It was especially important that the political and health officials understood the issue and supported volunteerism. The planning process also engaged key strategic partners, including the National Malaria Control Program, the IMCI/Early Childhood Development office of the MOH, and UNICEF. The project managers were diligent in reporting to the district health partners, national agencies and UNICEF and obtaining their input in planning, throughout the life of the project. It was clear at the mid-term and final evaluation that officials from these agencies actively participated in the project. One constant link was the participation of the District Environmental Health Officer. He participated in the DIP planning, DIP defense in the US, and throughout the life of the project. He provided a vital link of continuity during district assembly and health officer leadership changes. Project Management The DIP work plan was practical; however it was not fully used to guide the implementation process. The project managers did not have a process for using the DIP to direct project planning. As a result some key interventions were not adequately implemented. For example PD/Hearth was only implemented as a pilot; there was no follow through to fully implement it in needy areas. Additionally the sustainability plan in the DIP was not followed. At the mid-term the plan was revised, however, the updated indicators were not monitored on a consistent basis. A critical consequence was that HSAs were not integrated into the Care Group training and supervision process. Supervision The supervisory system was adequate, but not well documented. Supervisors submitted their reports, but there was no system for synthesizing and summarizing data from the reports. Consequently there was no way to know if issues and problems were addressed. Additionally the data collected on sustainability indicators were not extracted and monitored. For the final evaluation the evaluator had to ask the supervisors to prepare reports from their last quarter as none were available. Up to the time of that reports the project managers did not know the status of these indicators. Annexes~ Page 4 Unfortunately the supervision system is not institutionalized and is not likely to be maintained. As described in the body of the final evaluation report, the HSAs were to take over supervising the Care Groups. They were not consistently integrated into supervising Care Groups during the projects’ implementation. It is unrealistic to assume that they can do so after the project is finished when the majority of them have not been trained. It is a positive step that the MOH has adopted the policy of HSAs supervising Care Groups, but its implementation will be weak without proper training. Human Resource and Staff Management WRM was not expecting to continue its involvement in child survival programming after the end of the project, thus there was no need for policies and procedures to continue operations within WRM. WRM did support the staff in finding new jobs. At the time of the final evaluation nearly all the staff had jobs waiting for them. A number of staff were able to obtain jobs with the MOH, and will be valuable resources for helping to continue child survival interventions. During the final evaluation staff morale was good. There was one incident among the staff, but the project manager and WRM Program Director dealt with the problem appropriately. It was apparent that the staff members were passionate about the project. They were clearly engaged in the final analysis and had an appropriate sense of ownership over the results. The project experienced some staff turnover in the third year. It was due to financial management issues that are addressed in the following section. The financial management problems cause uncertainty among some staff about job security so they resigned to take secure positions with the MOH. This problem affected the morale of the whole team at that time. By the time of the final evaluation, however, the team’s morale was positive. Financial Management The project had difficulty with financial management including cash flow and budgeting issues. World Relief is undergoing a detailed examination of the World Relief Malawi country office to fully explore the nature of these issues. World Relief has developed a new system for tracking expenses and budgets and this tool is already proving invaluable for better tracking expenditures relative to budget. Information Management As discussed in the final evaluation report a gap in information management was that the project did not implement the Care Group monitoring system that was recommended at the mid-term. At that time the evaluator stressed the importance of having a way to monitor the fidelity of the Care Group model and he engaged the staff in designing an indicator monitoring system. It was not implemented, and as a consequence key components of the model were adhered to. Two critical components that were not monitored were coverage of all households in the district and home visitation twice a month. Two other mid-term recommendations regarding information management were to implement a system for data tracking at the community level and to use LRA methodology to monitor the sustainability data indicators. Neither of these were done. The staff did collect some of these data, but did so from promoter reports. The data from the reports however, were not Annexes~ Page 5 consolidated into an aggregate report form and tracked over time. The limited data collected were not used to monitor progress in sustainability. One LRA was done, but it was not done until six months from the end of the project. In regards to strengthening MOH’s HIS system, Tube Poka promoters collaborated by collecting HIS data in areas where there were no HSAs or where HSAs needed assistance. Nevertheless, the reliability of the MOH data was not consistent. One of the main reasons was due to logistical problems. During the rainy season data were not obtainable from mountainous, remote areas. Periodic fuel shortages would cause HSAs to miss reporting deadlines. At one point in time the HIS manager passed away and the position was not filled for many months, resulting in data not being collected and processed. All of these factors were influenced by the fact that Chitipa is one of the most remote and poorest district in Malawi. In terms of communicating what the project has achieved, the project managers communicated consistently with District Assembly and health officials. The WRM Program Director visited the project once a quarter and kept WRM headquarters informed. Technical and Administrative Support The project received support both from the WRM Country office in Lilongwe and from WR headquarters. The Country Director provided training support to church leaders. The Director of Programs made quarterly supportive visits (as noted above) to discuss programmatic as well as administrative issues while WRM Finance and Administration monitored program expenses and provided advice on financial prudence. The IT technician also assisted in helping to keep the project IT systems operational. In reference to external technical assistance, South African-based consultants worked with the project team to develop improved training curriculum for the care group volunteers. WR/HQ was in constant communication with WRM and the Tube Poka Project Managers through email and telephone. Additionally, WR sent a Maternal and Child Health Specialist to lead the team in conducting a KPC survey (recommended by the Mid-Term Evaluation) and train staff in qualitative and quantitative research methods. Management Lessons Learned One management lesson learned was the importance of the Project Manager being based in the project site. Because of the remoteness of Chitipa District and the lack of internet communications, the Project Manager initially was based in Mzuzu, a four hour drive, over a very bad road, from Chitipa. In the third year, however conditions improved such that the Manager could be based in Chitipa. While the conditions were justifiable in the first two years, having the Manager full time in Chitipa for the last two years contributed to strengthening relationships with the MOH and NGO partners. Culturally it was important that the head of the project be present for reporting and planning meetings. Another management lesson was the value of hiring local people for the project staff. The fact that staff were local advanced the process of building trust with community leaders and overcame potential language barriers. Chitipa District has seven different languages and over a dozen dialects. Non-residents would have been greatly hampered without the local language. Annexes~ Page 6 A management lesson for opportunities in the future is to not underestimate the importance of information management and of designating a full time data manager. As discussed above, and in the body of the report, this project was adversely affect by not having such a staff person. Annexes~ Page 7 Annex 4: Workplan Table Table 5: Workplan Table 1. Strengthen the capacity of Chitipa District’s health system to improve the quality and Coverage of IMCI services Objectives/ Activities Objective Met Activity Status Increase knowledge and compliance of health providers in C-IMCI services and protocols Objective Partially met Activity 1: Training and supportive supervision of HSAs and HC staff Trained 15 HSAs in project interventions and survey implementation; Some HSAs accompanied TP promoters on supervision of CG Activity 2: Joint M&E planning and implementation with MOH TP staff regularly shared data with MOH and showed them how to use data for program planning. Improve access to preventive health services Objective met Activity 1: Support outreach activities TP promoters assisted HSAs with transportation and assisted during outreach Activity 2: HSA training and supervision by CSP and MOH staff CSP staff did not supervise HSAs, this was done through the MOH by the District MCH Coordinator. CSP trained 25 HSAs to run village clinics; Improve the C-HIS Monitoring Partially met Activity 1:Review existing CHIS TP staff helped MOH learn how to use data for program planning Activity 2: Train and support VHCs to monitor HSAs and CG volunteers, and interpret HIS results MTE recommended a community monitoring system but not implemented. TP staff regularly shared CG mortality data with VDC; 83% of CG focus group stated they reported to village headmen in last month. 2. Develop sustainable community-based mechanisms to improve prevention and care-seeking practices for childhood illnesses at household and community level Objectives/ Activities Objective Met Activity Status Build capacity for improved support systems Objective met Activity 1: Train and support HSAs TP trained 25 HSAs to run village clinics; all other HSAs trained by District MCH Coordinator. Activity 2: Provide training and supportive supervision of pastors and traditional healers No traditional healers were trained; trained pastors as they participated in CG training; 362 churches participated in CSP. Develop a sustainable community-based mechanism Objective partially met Activity 1: Train and supervise Care Groups CG organized into Zonal Committees; pastors regularly visit CG; CG provide feedback to VDCs Activity 2: Create links between CGs and HSAs, CGs and VHCs 30% of HSAs visited CG in July 09; 80% of village headmen in FE focus group received reports from CG; 83% of CG focus groups say they report to VDC. Activity 3: Train and provide supportive supervision for HSAs and VHC members TP did not train or supervise HSAs, the MOH did. Informal training of VHC/VDC trough TP promoters’ regular participation in VDC meetings and sharing data with them. Annexes~ Page 8 3. Promotion of Key Family Practices for Child Health Objectives/ Activities Objective Met Activity Status 50% of children with suspected malaria receive treatment by trained provider within 24h Final KPC: 41.7% Activity: Train volunteers and caretakers to recognize signs of fever/malaria and importance of immediate treatment (within 24h) All CGVs trained; 95% of mothers ID fever as sign of malaria in FE focus groups. 60% of children <2 sleep under ITN Final KPC: 43.7%. Monitoring survey during rainy season, 71% Activity: Train volunteers and caretakers in method of transmission and prevention with mosquito nets All CGVs trained; mothers’ focus groups say malaria was topic most frequently taught. 50% of children with rapid, difficult breathing (suspected pneumonia) treated at HF within 24h Final KPC: 44.0% Activity: Train volunteers and caretakers to recognize cough and rapid/difficult breathing as signs of pneumonia, to seek treatment in HF within 24h for cough and fast/difficult breathing All CGVs trained; 44% of mothers in KPC sought care for child within 24 hrs. 80% of children 12-23m fully immunized by card Final KPC: 74.8% Activity 1: Collaborate with MoH on EPI outreaches TP promoters regularly conducted outreach with HSAs. Activity 2: Volunteers to mobilize community for immunization services CGVs consistently organized their communities 50% of sick children offered increased fluids and food during illness Final KPC: 22.6% Activity: Train volunteers and caretakers to continue feeding the sick child to increase fluid intake All CGVs trained. 60% of caretakers wash hands before food preparation, before child feeding, after defecation Final KPC: 19.3% Food prep- 56% Child feeding￾38% After defecation￾86% Activity: Train volunteers and caretakers in proper hygiene and hand washing practices; promote hand washing stations at home All CGVs trained; final KPC 72% HH had soap for hand washing. 60% of children with diarrhea receive ORT Final KPC: 64.4% Activity: Train volunteers and caretakers to treat diarrhea with ORT (either ORS packets, breastmilk, and/or home solutions) All CGVs trained. 80% of caregivers know at least 2 danger signs for seeking care immediately Final KPC: 95.7% Activity: Train volunteers and caretakers to recognize danger signs and appropriate care All CGVs trained. Annexes~ Page 9 seeking practices 60% of children EBF for 0-5m Final KPC: 81.8% Activity: Train volunteers to counsel caretakers on importance of EBF and support EBF All CGVs trained. 70% of children 6-9m receive complementary feeding Final KPC: 82.5% Activity: Train volunteers and caretakers on importance of appropriate and adequate complementary feeding All CGVs trained. 60% of children 6-23 months receive appropriate number of dose(s) of Vitamin A per year Final KPC:6.0%; one dose per mother’s recall: 88.5% Activity 1: Train volunteers and pastors on importance of VAC supplementation All CGVs trained; pastors informally trained Activity 2: Mobilize community to access VAC from community drug kits, mobile clinics and national campaigns Community drug kits dropped; TP promoters assisted with outreach where VAC distributed. 70% of children who complete Hearth achieve and sustain adequate (200g) or catch-up (400g) growth per month for at least 2m after Hearth Objective not met Activity 1: Train volunteers in Hearth methodology CGVs trained. Activity 2: Conduct 2 Hearth cycles in first 2 years; repeat in 3rd and 4th year as required Hearth not implemented beyond pilot. Activity 3: Maintain Hearth registers Health registers not maintained. 70% of mothers to deliver by a trained health provider Final KPC: 72.7% Activity: Train volunteers and caretakers on importance of ANC and encourage delivery by trained health provider CGVs trained. 80% of caretakers will know >2 ways to prevent STD/HIV/AIDS Final KPC: 85.7% Activity 1: Train volunteers and caretakers on causes and prevention of STI/HIV/AIDS CGVs trained. Activity 2: Promote utilization of VCT services Collaborated with US church that donated VCT kits. Annexes~ Page 10 Annex 5: Rapid CATCH Table Table 6: Rapid CATCH Indicators Indicators Baseline Estimate Midterm Estimate Final Estimate 1 Percentage of children age 0-23m who are underweight (-2SD from the median weight-for-age, according to the WHO/NCHS reference population) 29.9% 10.4% 15.2%* 2 Percentage of children age 0-23months who were born at least 24 months after the previous surviving child ▪Baseline is from 2006 monitoring survey 39.5% 83.5% 87.3%* 3 Percentage of children age 0-23m whose births were attended by skilled health personnel 55.2% 61.0% 72.7%* 4 Percentage of mothers with children age 0-23m who received at least two tetanus toxoid injections before the birth of their youngest child ▪Includes TT verified by card 64.0% 53.0% 59.3% 5 Percentage of children 0-5m who were exclusively breastfed during the last 24 hours 40.0% 84.2% 79.7%* 6 Percentage of children 6-9m who received breastmilk and complementary foods during the last 24 hours 39.8% 80.5% 85.2%* 7 Percentage of children age 12-23m who are fully vaccinated (against the five vaccine-preventable diseases) before the first birthday ▪Baseline includes children 12-23 months who had a BCG scar and children 9-11 months who had a DPT3, Polio3 and measles vaccine NA 75.2% 71.7% 8 Percentage of children age 12-23m who received a measles vaccine ▪Baseline includes measles verified by card for children 9-23 months NA 94.8% 91.6% 9 Percentage of children age 0-23m who slept under an insecticide-treated net the previous night ▪Baseline includes all nets regardless of insecticide treatment 41.0% 39.7% 43.7% 10 Percentage of mothers with children age 0-23m who cite at least two known ways of reducing the risk of HIV infection 67.5% 76.5% 85.7% 11 Percentage of mothers with children age 0-23m who report that they wash their hands with soap/ash before food preparation, before feeding children, after defecation, and after attending to a child who has defecated 3.9% 12.3% 19.3%* 12 Percentage of mothers of children age 0-23m who know at least two signs of childhood illness that indicate the need for treatment 71.1% 88.7% 95.7%* 13 Percentage of sick children age 0-23m who received increased fluids and continued feeding during an illness in the past two weeks 3.9% 17.8% 22.6%* * Denotes statistical significance with 95% confidence Annexes~ Page 11 Annex 6: CHW Training Matrix Table 7: Community Health Worker Training Matrix Project Area (Supervision area or Village) Group Trained Organization Paid/Volunteer Number of People trained over the life of the project Focus of the training Health Post/Village Clinic HSAs MoH Paid 25 C-IMCI Health Post HSAs MoH Paid 12 CS; C-IMCI Health Post HSAs MoH Paid 30 P/D Hearth All Supervision Areas Health Promoters WRM Paid 34 CS All Supervision Areas Supervisors WRM Paid 7 CS All Supervision Areas CGVs Community￾based Volunteers Over 4,000 CS Chitipa Town Manager for Water and Sanitation, Maternal and Child Health Coordinator (2), DEHO (2), AEHO (4), Malaria control manager, HIV/AIDS & IMCI Coord MoH Paid 12 Survey Implementation and analysis Chitipa Town DHMT MoH Paid Approximately 10 EpiInfo software Annexes~ Page 12 Annex 7: Evaluation Team Members and their Titles Name Organization Position Richard Crespo Consultant Lead Evaluator Sarah Borger World Relief Headquarters Maternal and Child Health Specialist Gibson Nkanaunena World Relief Malawi Director of Programs Victor Kabaghe World Relief Malawi – TPCSP Project Manager Richard Thindwa World Relief Malawi – TPCSP Deputy Project Manager Joseph Simwaka Chitipa District Hospital Environmental Health Intern Thandie Msukwa World Relief Malawi – TPCSP Health Promoter Foreward Chilanga World Relief Malawi – TPCSP Health Education Supervisor Misheck Mdambo District Social Welfare Child Protection Officer Jill M Mtambo World Relief Malawi – TPCSP Health Education Supervisor George Nundwe Chitipa District Hospital Maternal and Child Health Coordinator Wongani Mulungu World Relief Malawi – TPCSP Health Education Supervisor Thomas A Nkhonjera World Relief Malawi -TPCSP Health Education Supervisor Paul L Ng’ambi World Relief Malawi -TPCSP Health Education Supervisor Mbasa Msiska World Relief Malawi - TPCSP Project Accounts Assistant Victor Kabaghe World Relief Malawi -TPCSP Project Director Andrew Kasache Banda World Relief Malawi -TPCSP Project Driver Isaac Munthali World Relief Malawi -TPCSP Project Driver Nelson Mwandwanga World Relief Malawi -TPCSP Project Driver Rachel Gondwe World Relief Malawi -TPCSP Project Secretary German Phikani World Relief Malawi -TPCSP Office Assistant Maureen Mtambo World Relief Malawi - TPCSP Health Promoter Alfred Munkhondya World Relief Malawi - TPCSP Health Education Supervisor George Mkandawire World Relief Malawi HIV/AIDS Care and Support Coord. Benjamin Nyondo World Relief Malawi – TPCSP Health Promoter Charles Kapira World Relief Malawi – TPCSP Health Promoter Maseso Mwiba World Relief Malawi - TPCSP Health Promoter Annexes~ Page 13 Annex 8: Evaluation Assessment Methodology The final evaluation used a participatory approach. The lead evaluator engaged the whole project team in the evaluation process. The first step was to involve the team in planning the evaluation. The lead evaluator began by reviewing the USAID final evaluation guidelines. Within the framework of the guidelines he asked the project team to identify what they had done up to this point, including the things that were still in process. As an individual identified an activity it was written on a flip chart. The whole group was then asked to list the indictors that could verify that activity. In a large group discussion format the lead evaluator went around the room asking project team members to identify an activity and list indicators. The result was a series of flip chart pages with activities and indicators that could verify each one. The flip chart pages were then organized into evaluation topics and placed together on a wall of the meeting room. The project team was divided into small groups and each group was assigned an evaluation topic. The task of the small groups was to plan how they would collect information about that topic. In preparation for this task the lead evaluator gave them a framework for their task. They were asked to identify who, or what group of people (i.e. mothers, health center staff, village leaders, etc) could contribute information about the topic. For each group they then made a list of questions about the evaluation topic. These questions in turn served as the basis for writing survey instruments. For this task the workshop participants were reorganized into new small groups that were charged with creating a survey for a specific population, e.g. mothers, CVCGs, village leaders, etc. The lead evaluator gave the groups guidelines and directions for writing a survey. He also made the rounds of the groups to give specific instructions. As a small group completed a draft of their survey it was reviewed by the lead evaluation and then revised by the group. For data collection the evaluation team was organized into groups of four and assigned districts for conducting interviews. Supervisors went to districts other than their own. Data analysis was also done in a participatory mode. Working in small groups the team entered data from the surveys on to data tables. Then they were shown how to organize data into tables and graphs. Their tables and graphs were placed on flip charts and pasted to the walls of the meeting room, organized by the survey questions for each target population (mothers, CGVs, village headmen, etc.). The lead evaluator then led the group through an exercise that resulted in the group drawing conclusions and making the recommendations. These served as the foundation for this report. Annexes~ Page 14 Annex 9: List of persons interviewed and contacted during Final Evaluation In addition to interviewing World Relief Malawi, Tube Poka, and World Relief Headquarters staff, the following people or groups were interviewed during the final evaluation process. Region/ Village Mothers CGVs Church Leaders Village Headmen HSA Zonal Committee Health Facility Traditional Authority Chitipa North    Masyesye      Mwenifuwy      Chitipa Northwest Zamamba I      Udonda      Chitipa Central Namatubi      Wenya    Nyimbili      Namukumtha      Mlembe      Nthalire    Kamphyongo      Thereve      Chitipa Northeast    Chindnkha      Chipakama      Misuku Mwandisi      Mbowe      Kasanbola      Annexes~ Page 15 Annex 10: Project Data Form Child Survival and Health Grants Program Project Summary Dec-30-2009 World Relief Corporation (Malawi) General Project Information Cooperative Agreement Number: GHS-A-00-05-00032 WRC Headquarters Technical Backstop: Rachel Hower WRC Headquarters Technical Backstop Backup: Field Program Manager: Victor Kabaghe Midterm Evaluator: Richard Crespo Final Evaluator: Richard Crespo Headquarter Financial Contact: Rachel Hower Project Dates: 9/30/2005 - 9/30/2009 (FY05) Project Type: Standard USAID Mission Contact: Catherine Chiphazi Project Web Site: Field Program Manager Name: Victor Kabaghe Address: Malawi Phone: (265) 8-505-589 Fax: E-mail: vkabaghe@wr.org Skype Name: Alternate Field Contact Name: Richard Thindwa (Deputy Program Director) Address: Malawi Phone: (265) 8-319-400 Fax: E-mail: rthindwa@wr.org Skype Name: Grant Funding Information USAID Funding: $1,500,000 PVO Match: $522,034 General Project Description Annexes~ Page 16 Program Goals: 1) Strengthen the capacity of the health district to implement Child Survival and Health interventions by improving the quality and coverage of C-IMCI services. 2) Develop sustainable community based mechanisms to improve preventive and care seeking practices for childhood illnesses at the household and community level. Interventions: - Malaria prevention and case management - Nutrition including exclusive breast-feeding and micronutrients - Control of diarrheal diseases - Pneumonia case management - HIV/AIDS prevention - Child Immunization Strategies: Community Integrated Management of Childhood Illnesses (C-IMCI) will be implemented using the Care Group strategy; other strategies will include PD/Hearth, HIV/AIDS prevention adapting World Relief’s Mobilizing for Life strategy. Project Location Latitude: -9.72 Longitude: 33.27 Project Location Types: (None Selected) Levels of Intervention: (None Selected) Province(s): -- District(s): Chitipa District, Northern Malawi Sub-District(s): -- Operations Research Information OR Project Title: -- Cost of OR Activities: -- Research Partner(s): -- OR Project Description: -- Partners Chitipa District Ministry of Health (Collaborating Partner) $0 Church of Central Africa Presbyterian (CCAP), Synod of Livingstonia (SOL) (Collaborating Partner) $0 Population Services International (PSI) Malawi (Collaborating Partner) $0 United Nations Children Fund (UNICEF) $0 Annexes~ Page 17 (Collaborating Partner) Strategies Social and Behavioral Change Strategies: Group interventions Interpersonal Communication Strategies for Enabling Environment: Advocacy for policy change or resource mobilization Tools/Methodologies: Rapid Health Facility Assessment Participatory Rapid/Rural Appraisal Capacity Building Local Partners: Pharmacists or Drug Vendors Traditional Healers Dist. Health System Health Facility Staff Health CBOs Faith-Based Organizations (FBOs) Interventions & Components Immunizations (20%) - Classic 6 Vaccines - Vitamin A - Surveillance - Mobilization IMCI Integration CHW Training HF Training Nutrition (20%) - Complementary Feeding from 6 months - Hearth - Continuous BF up to 24 months - Growth Monitoring IMCI Integration CHW Training HF Training Pneumonia Case Management (10%) - Case Management Counseling - Recognition of Pneumonia Danger Signs IMCI Integration CHW Training HF Training Control of Diarrheal Diseases (15%) - Hand Washing - ORS/Home Fluids - Feeding/Breastfeeding - Care Seeking - Case Management/Counseling IMCI Integration CHW Training HF Training Malaria (30%) - Training in Malaria CM - Access to providers and drugs - Antenatal Prevention Treatment - ITN (Bednets) - Care Seeking, Recog., Compliance - IPT IMCI Integration CHW Training HF Training HIV/AIDS (5%) - Behavior Change Strategy CHW Training HF Training Operational Plan Indicators Number of People Trained in Maternal/Newborn Health Annexes~ Page 18 There is no data for this project for this operational plan indicator. Number of People Trained in Child Health & Nutrition There is no data for this project for this operational plan indicator. Number of People Trained in Malaria Treatment or Prevention There is no data for this project for this operational plan indicator. Locations & Sub-Areas Total Population: 174,786 Target Beneficiaries Malawi - WRC - FY05 Infants < 12 months 9,413 Children 0-59 months 0 Women 15-49 years 40,201 Beneficiaries Total 49,614 Rapid Catch Indicators: DIP Submission Sample Type: 30 Cluster Indicator Numerator Denominator Percentage Confidence Interval Percentage of children age 0-23 months who are underweight (- 2 SD from the median weight￾for-age, according to the WHO/NCHS reference population) 108 361 29.9% 7.4 Percentage of children age 0-23 months who were born at least 24 months after the previous surviving child 130 329 39.5% 8.6 Percentage of children age 0-23 months whose births were attended by skilled health personnel 201 364 55.2% 9.2 Percentage of mothers of children age 0-23 months who received at least two tetanus toxoid injections before the 233 364 64.0% 9.6 Annexes~ Page 19 birth of their youngest child Percentage of infants age 0-5 months who were exclusively breastfed in the last 24 hours 20 50 40.0% 22.2 Percentage of infants age 6-9 months receiving breastmilk and complementary foods 78 196 39.8% 11.2 Percentage of children age 12- 23 months who are fully vaccinated (against the five vaccine-preventable diseases) before the first birthday 238 346 68.8% 10.0 Percentage of children age 12- 23 months who received a measles vaccine 191 346 55.2% 9.4 Percentage of children age 0-23 months who slept under an insecticide-treated bednet the previous night (in malaria-risk areas only) 66 161 41.0% 12.5 Percentage of mothers who know at least two signs of childhood illness that indicate the need for treatment 234 329 71.1% 10.3 Percentage of sick children age 0-23 months who received increased fluids and continued feeding during an illness in the past two weeks 12 311 3.9% 3.1 Percentage of mothers of children age 0-23 months who cite at least two known ways of reducing the risk of HIV infection 243 360 67.5% 9.8 Percentage of mothers of children age 0-23 months who wash their hands with soap/ash before food preparation, before feeding children, after defecation, and after attending to a child who has defecated 11 284 3.9% 3.2 Rapid Catch Indicators: Mid-term Annexes~ Page 20 Sample Type: 30 Cluster Indicator Numerator Denominator Percentage Confidence Interval Percentage of children age 0-23 months who are underweight (- 2 SD from the median weight￾for-age, according to the WHO/NCHS reference population) 13 289 4.5% 3.4 Percentage of children age 0-23 months who were born at least 24 months after the previous surviving child 27 300 9.0% 4.7 Percentage of children age 0-23 months whose births were attended by skilled health personnel 183 300 61.0% 10.4 Percentage of mothers of children age 0-23 months who received at least two tetanus toxoid injections before the birth of their youngest child 159 194 82.0% 13.8 Percentage of infants age 0-5 months who were exclusively breastfed in the last 24 hours 85 101 84.2% 19.3 Percentage of infants age 6-9 months receiving breastmilk and complementary foods 33 41 80.5% 30.0 Percentage of children age 12- 23 months who are fully vaccinated (against the five vaccine-preventable diseases) before the first birthday 69 103 67.0% 18.2 Percentage of children age 12- 23 months who received a measles vaccine 186 297 62.6% 10.5 Percentage of children age 0-23 months who slept under an insecticide-treated bednet the previous night (in malaria-risk areas only) 119 300 39.7% 9.0 Annexes~ Page 21 Percentage of mothers who know at least two signs of childhood illness that indicate the need for treatment 266 300 88.7% 11.2 Percentage of sick children age 0-23 months who received increased fluids and continued feeding during an illness in the past two weeks 32 180 17.8% 8.3 Percentage of mothers of children age 0-23 months who cite at least two known ways of reducing the risk of HIV infection 230 300 76.7% 11.0 Percentage of mothers of children age 0-23 months who wash their hands with soap/ash before food preparation, before feeding children, after defecation, and after attending to a child who has defecated 37 300 12.3% 5.4 Rapid Catch Indicators: Final Evaluation Sample Type: 30 Cluster Indicator Numerator Denominator Percentage Confidence Interval Percentage of children age 0-23 months who are underweight (- 2 SD from the median weight￾for-age, according to the WHO/NCHS reference population) 45 297 15.2% 6.0 Percentage of children age 0-23 months who were born at least 24 months after the previous surviving child 145 166 87.3% 15.1 Percentage of children age 0-23 months whose births were attended by skilled health personnel 218 300 72.7% 10.9 Percentage of mothers of children age 0-23 months who received at least two tetanus 178 300 59.3% 10.3 Annexes~ Page 22 toxoid injections before the birth of their youngest child Percentage of infants age 0-5 months who were exclusively breastfed in the last 24 hours 55 69 79.7% 23.1 Percentage of infants age 6-9 months receiving breastmilk and complementary foods 52 61 85.2% 24.8 Percentage of children age 12- 23 months who are fully vaccinated (against the five vaccine-preventable diseases) before the first birthday 99 138 71.7% 16.0 Percentage of children age 12- 23 months who received a measles vaccine 142 155 91.6% 15.7 Percentage of children age 0-23 months who slept under an insecticide-treated bednet the previous night (in malaria-risk areas only) 131 300 43.7% 9.3 Percentage of mothers who know at least two signs of childhood illness that indicate the need for treatment 287 300 95.7% 11.3 Percentage of sick children age 0-23 months who received increased fluids and continued feeding during an illness in the past two weeks 51 226 22.6% 8.2 Percentage of mothers of children age 0-23 months who cite at least two known ways of reducing the risk of HIV infection 257 300 85.7% 11.2 Percentage of mothers of children age 0-23 months who wash their hands with soap/ash before food preparation, before feeding children, after defecation, and after attending to a child who has defecated 58 300 19.3% 6.7 Annexes~ Page 23 Rapid Catch Indicator Comments Maternal tetanus includes TT verified by card divided by all caretakers in the survey. Annexes~ Page 24 Annex 11: Grantee Plans to Address Final Evaluation Findings World Relief sincerely appreciates the tireless efforts of Dr. Richard Crespo as he led this evaluation. His suggestions and observations were communicated to the CSP staff and as applicable are being incorporated into World Relief’s plans for future activities. The continued support and training of HSAs and local leaders in Chitipa is very important to World Relief. We are currently drafting a proposal to the USAID Mission focusing on nutrition. We envision that this proposal could provide the needed funding to expand training to these two important groups pending further negotiations with consortium members and the Mission. Annexes~ Page 25 Annex 12: Final KPC Report Annexes~ Page 26 TUBE POKA CHILD SURVIVAL PROJECT CHITIPA DISTRICT, MALAWI FINAL KPC SURVEY REPORT December 31, 2009 Victor Kabaghe, WR Malawi Child Survival Project Manager Richard Thindwa, WR Malawi Child Survival Project Deputy Manager Sarah Borger, WR Maternal and Child Health Specialist Melanie Morrow, WR Director of Maternal and Child Health Programs Cooperative Agreement No. GHS-A-00-05-00032-00 Program Dates: October 1, 2005 to September 30, 2009 Annexes~ Page 27 ACRONYMS ACT Artemisinin Combination Therapy (also referred to as LA in Malawi) AIDS Acquired Immune Deficiency Syndrome ANC Antenatal care ART Antiretroviral Therapy BCG Bacille Calmette-Guerin CATCH Core Assessment Tool on Child Health CG Care Group CGV Care Group Volunteer C-HIS Community Health Information System C-IMCI Community Integrated Management of Childhood Illness CS Child Survival CSP Child Survival Project DHMT District Health Management Team DHS Demographic Health Survey DPT Diphtheria, Pertussis, Tetanus EBF Exclusive Breast Feeding EHP Essential Health Care Package EOP End of Project EPI Expanded Program on Immunization GMC Growth Monitoring Counseling HC Health Center HF Health Facility HSA Health Surveillance Assistants IMCI Integrated Management of Childhood Illness IPTp Intermittent Presumptive Therapy during pregnancy ITN Insecticide Treated Net KPC Knowledge, Practice and Coverage HIV Human Immunodeficiency Virus MOH Ministry of Health ORT Oral Rehydration Therapy ORS Oral Rehydration Solution PMTCT Prevention of Mother to Child Transmission of HIV PSI Population Services International SP Sulfadoxine- Pyrimethamine STI Sexually Transmitted Infection TBA Traditional Birth Assistant TT Tetanus Toxoid UNICEF United Nations Infants and Children’s Fund USAID United States Agency for International Development VCT Voluntary Counseling and Testing WHO World Health Organization WR World Relief Annexes~ Page 28 TABLE OF CONTENTS I. EXECUTIVE SUMMARY ..................................................................................................... 1 II. BACKGROUND ..................................................................................................................... 1 National Standards and Policies .................................................................................................. 2 Overview of the Care Group Model ............................................................................................ 3 Program Goals and Objectives .................................................................................................... 3 Intervention Specific Objectives ................................................................................................. 4 III. PROCESS AND PARTNERSHIP BUILDING ...................................................................... 5 IV. METHODS .............................................................................................................................. 5 Sampling Design ......................................................................................................................... 7 Interviewer Recruitment .............................................................................................................. 7 Interviewer Training .................................................................................................................... 8 Data collection............................................................................................................................. 8 Data Entry and Analysis.............................................................................................................. 8 V. RESULTS................................................................................................................................ 9 VI. DISCUSSION ........................................................................................................................ 10 Discussion of Program Indicators ............................................................................................. 10 Discussion of Other Results ...................................................................................................... 14 External Comparisons ............................................................................................................... 15 Information Dissemination ........................................................................................................ 16 TABLES Table 1 Intervention Mix and Level of effort ................................................................................ 4 Table 2 Program and KPC Indicator Definitions ........................................................................... 6 Table 3 Project Indicator Results ................................................................................................... 9 Table 4 Comparison of Indicators with National Data Sources .................................................. 15 ANNEXES Annex A Final KPC Survey Questionnaire Annex B Sampling Framework Annex C Final KPC Survey- List of Persons and Roles Annex D Indicator Table Annex E Raw Data Tables Annex E Training Schedule and Required Resources Page | 1 I. EXECUTIVE SUMMARY In July 2009, The World Relief (WR) Tube Poka Child Survival Team conducted a Final KPC survey in the project area Chitipa District, Malawi. The survey was designed to assess the knowledge and practices of mothers of children 0-23 months in diarrheal disease control, malaria control, pneumonia, infant and young child feeding, immunization coverage, growth monitoring, birth spacing and HIV/AIDS. The Baseline questionnaire was modified at the time of the Midterm and this same modified questionnaire was repeated at the Final with the alteration of the sustainability question for great detail. A 30 cluster survey methodology used to select the respondents and the results analyzed using EpiInfo software. This project has met or exceeded the following indicator targets:  Percentage of caretakers of children 0-23m who know at least two childhood illness danger signs for seeking care immediately  Percentage of children 0-23m with diarrhea in the previous two weeks who received Oral Rehydration Therapy (ORT) (home available fluids, ORS, or breastfeeding)  Percentage of children 0-5m who were exclusively breastfed during the last 24 hours  Percentage of children 6-9m who received breastmilk and complementary foods during the last 24 hours  Percentage of mothers with children age 0-23m who cited at least two known ways of reducing the risk of HIV infection  Percentage of children age 0-23m whose births were attended by skilled health personnel II. BACKGROUND World Relief’s Tube Poka Child Survival Project operated within the Chitipa District of Malawi from 2006 to 2009. The population of Chitipa is primarily rural, with an estimated total population of 174,786 people scattered over the mountains in 475 villages. The beneficiary population of 89,974 is comprised of 49,773 children under five (9,413 children less than 12 months, 8,335 children 12-23 months, and 14,277 children 24-59 months) and 40,201 women ages 15-49 years2 . Chitipa’s residents are diverse, with approximately 15 ethnic groups speaking 17 different dialects. Chitipa is predominantly Christian (96%), with the highest illiteracy rate (33%) of the six northern districts3 . Differential household food distribution favors males, putting mothers and female children at increased risk of malnutrition. Malaria, pneumonia and diarrhea remain public health threats particularly to children in Chitipa. According to the 2004 Demographic Health Survey (DHS), disease burden attributable to malaria is over 36%, and case fatality rate is 50% higher than the national average4 . Page | 2 Chitipa District is further from the capital city than any other district in Malawi and is among the most isolated. Over 50% of households in Chitipa have to walk more than 30 minutes to a safe water source5 . The poor infrastructure hampers potential areas of development in the district; the road to the district capital is not paved, access to internet services is currently available but unreliable, power cuts for all of Chitipa Township are frequent, and the water supply system at the District Township has also had difficulties. However, installation of power lines has extended to southern corridor (Ntahlire and Wenya) and Misuku areas giving more opportunities for Information Technology installations. The District Assembly, World Relief and one recent Internet Cafe, have established satellite internet connections during the life of this project. National Standards and Policies Health Services: Malawi’s formal health system includes government run health centers which provide family planning, safe motherhood, environmental health, immunization, disease prevention, antenatal care (ANC), and growth monitoring and counseling (GMC) at no cost to users. Malawi’s MOH and UNICEF promote an essential health care package (EHP) focusing on common illnesses and equitable to the poor using a sector wide approach. It emphasizes insecticide treated net (ITN) distribution and use, increased access to prompt treatment within 24 hours for children with suspected malaria and pneumonia, and increased access to intermittent preventative treatment (IPTp) for pregnant women. In addition to regional health centers, there is one District Hospital that provides more advanced care including Antiretroviral Therapy (ART) and Prevention of Mother to Child Transmission of HIV (PMTCT), and 148 community based Health Surveillance Assistants (HSA) for health education and limited treatment. Health Information System: Surveillance data are reported monthly by the HSAs to health facilities which in turn submit monthly reports to the District Statistician for inclusion in the District database. Copies of the compiled data are sent monthly to a central database. Surveillance reports are collated quarterly and circulated to local, regional and national MOH authorities, as well as shared with HSAs, community leaders and Village Health Committees to inform decision-making, responses to outbreaks, etc. A gamut of health information is collected, including inpatient and outpatient services, community based data, personnel status and movement, and stock-outs for essential drugs. Reporting data includes delivery outcomes, expanded program on immunization (EPI), sexually transmitted infections (STI), cholera, and general morbidity and mortality data. However, areas not covered by HSAs do not have up to date information, and outreach immunization data is not accurate due to staff shortages. Moreover, household level data is largely unavailable. Immunizations: Malawi’s Expanded Program on Immunization follows child vaccination guidelines set forth by the World Health Organization (WHO). A child is considered fully vaccinated if she or he has received one BCG vaccine, three DPT and polio vaccines, and one measles vaccine. DPT and polio vaccines are given at approximately 6, 10, and 14 weeks of age. The measles vaccine should be given at or soon after the child reaches nine months of age. The Malawi EPI recommends that children the complete schedule of vaccinations before 12 months of age. Polio vaccine at or around birth is being promoted, although it is not yet widely practiced in Malawi. Vaccinations are provided at health centers and while there are frequent health outreach campaigns they rarely include vaccinations. Page | 3 Nutrition: The Malawi Ministry of Health is a strong promoter of exclusive breastfeeding (EBF) for the first six months of life and continued breastfeeding with appropriate complementary feeding up to two years or beyond. This policy applies to all children unless there are medical indications, in line with the UNICEF and WHO Global Strategy on Infant and Young Child Feeding.6 In addition, it is MOH policy to supplement children age 6-59 months with a Vitamin A capsule once every six months. Distribution of Vitamin A occurs most frequently at twice annual health outreach campaigns. During this time the District Health Management Team (DHMT) has specifically not requested mothers to bring vaccination cards as it may dissuade those who do not own cards. Policy on Malaria: In 2008, the MOH changed its policy on malaria treatment to combination therapy due to high resistance to anti-malarial drugs in the country. Following this change previous malaria medication was removed from the HSAs’ drug kits with plans for training and restocking with the new treatment. However, the training and subsequent restocking has yet to take place resulting in a lack of available treatment at the community level. The MOH recommends two doses of IPTp with SP during pregnancy and they provide ITNs for free, but this distribution is limited due to shortages in supplies. HIV/AIDS: Voluntary Counseling and Testing (VCT) sites are available at all health centers (HC) in Chitipa. ART and PMTCT services are available at the District Hospital. Diarrhea Prevention and Treatment: The Ministry of Health emphasizes on the promotion of safe water uptake and appropriate hygiene practices. Their policy recommends hand washing at five critical times using the model of the hand where the thumb represents washing hands after using the toilet, the index finger represents hand washing after changing baby’s nappies, the middle finger represents washing hands before handling food and before breast feeding, the ring finger represents washing hands before cooking, and the small finger before eating food. The policy also emphasizes on the construction and usage of a pit latrine and a hand washing facility. With regards to treatment of diarrhea, the policy recommends that care takers use ORT and or any available homemade fluids while taking the child to health facility. Overview of the Care Group Model The Tuba Poka project uses a modified version Care Group Model and partners with the District MOH, the Department of Social Welfare, and 362 local churches. Over 3,000 Care Group Volunteers (CGV) are organized into Care Groups (CG) of approximately ten volunteer members, providing the functional units for training, management and health information. Each CGV is given responsibility for ten households, until every house with a child under five years of age or women of reproductive age in the village is covered by a volunteer. The CGVs visit each of their ten households every two weeks to share health lessons, check on the health of their children and build relationships. World Relief developed the Care Group strategy in Mozambique, but this is the first child survival project to involve local churches as community￾based organizations in direct implementation of care groups in the hope of providing for even greater sustainability beyond previously published successes7,8 . Program Goals and Objectives There were three overall strategic objectives of the Tube Poka project: Page | 4  Strengthen the capacity of Chitipa District’s health system to implement Child Survival and Health interventions according to Integrated Management of Childhood Illness (IMCI) protocols.  Develop sustainable community-based mechanisms to improve prevention and care￾seeking practices for childhood illnesses at the household and community level.  Improve coverage and utilization rates of malaria control strategies according to Roll Back Malaria guidelines. Table 8 Intervention Mix and Level of effort Intervention Mix Level of Effort Malaria Prevention and Case Management 30% Nutrition 20% Control of Diarrheal Diseases 20% Immunization 15% Pneumonia Case Management 10% HIV/AIDS Prevention 5% All of the planned interventions have been implemented with exception of district wide implementation of Hearth as part of Nutrition. The project implemented Hearth in two villages quite successfully but was not able to scale up these activities throughout the District. Intervention Specific Objectives 1. Malaria Prevention and Case Management 50% of children with fever (suspected malaria) receive treatment by trained provider within 24h of onset of symptoms 60% of children sleep under ITN 2. Nutrition 60% of children exclusively breastfeed for 0-5 months 70% of children 6-9 months will receive complementary feeding 60% of children 6-23 months receive appropriate number of dose(s) of Vitamin A per year 70% of children who complete Hearth achieve and sustain adequate (200g) or catch-up (400g) growth per month for at least 2 months after Hearth (*can not be measured with the KPC Survey tool) 3. Control of Diarrheal Disease 80% of caretakers know at least 2 danger signs for seeking care immediately 60% of sick children offered increased fluids and food during illness 60% of caregivers wash hands before food preparation, before child feeding, after defecation and after cleaning child’s feces 60% of children with diarrhea receive oral rehydration therapy 4. Immunization Page | 5 80% of children 12-23m fully immunized by their first birthday (originally, fully immunized at the time of the survey) 5. Pneumonia Case Management 50% of children with rapid, difficult breathing (suspected pneumonia) treated at health facilities within 24 hours of the onset of symptoms 6. HIV/AIDS Prevention 70% of mothers will deliver by a trained health provider 80% of caretakers will know at least two known ways to reduce the risk of HIV/AIDS infection III. PROCESS AND PARTNERSHIP BUILDING The MOH is the project’s main partner as it provides all health services in the District. The project also partners with UNICEF to supplement C-IMCI activities, PSI to promote and distribute Thanzi ORS and Waterguard, and has collaborated with the District Social Welfare Office. Additionally, the project collaborations with PSI/Malawi on integrated diarrhea prevention through a marketing strategy for Oral Rehydration Solution (ORS) and pot chlorination products. The Midterm and Final KPC surveys provided an additional opportunity for partnership building with the District MOH and the Department of Social Welfare. Representatives participated in the survey training, data collection and debriefing discussions. This process provided the opportunity for project staff to better understand the activities and challenges of our partners in the communities. It also provided the opportunity for the project staff and partners to collaboratively develop potential solutions to issues identified in the communities through the survey process. IV. METHODS The purpose of the KPC survey is to establish estimates for the level of achievement of the end of project (EOP) targets and key preventive behaviors including, but not limited to increased utilization of preventive services, appropriate home care behaviors for sick children, and prompt care seeking. The survey has allowed program staff to assess progress on key indicators at Baseline, Midterm and Final. The Final Evaluation KPC questionnaire is designed for mothers/caretakers of children 0-23 months and was based on the questionnaire used for the Baseline KPC survey in April 2005 and Midterm KPC in January 2007. The original questionnaire drew from the Rapid CATCH indicators and the KPC 2000+ modules with adaptations to fit the local context. The questionnaire was pre-tested before it was used for data collection and contains 49 questions that cover the topics below. Please see Annex A for the full questionnaire. 1-4 Identification and ages 5-8 Breastfeeding and complementary feeding 9-12 Illness recognition and care seeking 13 Diarrhea Page | 6 14-16 Pneumonia 17-24 Malaria Control and Prevention 25-28 Growth Monitoring 29-32 Immunizations 33-37 Water and Sanitation 38-46 Maternal and Neonatal Health 47-48 HIV/AIDS 49 Sustainability Table 9 Program and KPC Indicator Definitions Intervention Area Indicator Definitions Numerator Denominator Care Seeking Caretakers of children 0-23 months who know at least 2 childhood illness danger signs for seeking care immediately [RC 12] All caregivers of children age 0-23 months Home Management Number of children 0-23 months who were offered increased fluids and continued or increased feeding during illness [RC 13] Number of children 0-23 months who were sick in the past two weeks Immunization Percentage of children 12-23 months fully immunized (BCG, Polio3, DPT3, and Measles) before 24 months as verified by card All children age 12-23 months Number of children age 12-23 months who are fully vaccinated before the first birthday (requires Polio 3, DPT 3 and Measles) [RC 7] Number of children age 12-23 months with vaccination cards Number of caretakers with children age 12-23 months who recalled that the child received a measles vaccine [RC 8] All caretakers of children age 12- 23 months Number of mothers with children 0-23 months who receiving at least two tetanus toxoid injections verified by card before the birth of their youngest child [RC 4*] All mothers with children 0-23 months Control of Malaria Number of children 0-23 months who received treatment for suspected malaria from a trained health provider within 24 hours Number of children age 0-23 months with suspected malaria (fever, convulsions or malaria) in the past two weeks Number of children 0-23 months who slept under an ITN (ever treated or long-lasting net) the previous night [RC 9] All children 0-23 months Pneumonia Control Management Number of children 0-23 who received treatment for suspected pneumonia from a trained provider within 24 hours Number of children age 0-23 months with suspected pneumonia (rapid/difficult breathing) in the past two weeks Control of Diarrheal Disease Number of children 0-23 months who received ORT/ORS/home available fluids/breastfeeding for diarrhea Number of children age 0-23 months with diarrhea in the past two weeks Number of caregivers of children 0-23 months who report washing their hands with soap/ash before food preparation, before child feeding, after defecation, and after attending to a child who has defecated [RC 11] All caregivers of children age 0-23 months Nutrition Number of children 0-5 months who were exclusively breastfed during the past 24 hours, based on dietary recall [RC 5] Number of children age 0-5 months Page | 7 Number of children 6-9 months who received breast milk and complementary foods during the last 24 hours, based on dietary recall [RC 6] Number of children age 6-9 months Number of children 6-11 months who received at least 1 dose of Vitamin A verified by card plus the number of children 12-23 months who received at least 2 doses of Vitamin A verified by card in the previous 12 months Number of children 6-23 months Number of children age 0-23m who are underweight (-2SD from the median weight-for-age, according to the 1978 WHO/NCHS reference population) [RC 1] Number of children 0-23 months HIV/AIDS Prevention Number of caretakers with children 0-23 months who cite at least two known ways of reducing the risk of HIV infection [RC 10] All caretakers with children 0-23 months Number of children age 0-23 months whose births were attended by skilled health personnel (Doctor, Nurse or Midwife) [RC 3] All children age 0-23 months Other Number of children age 0-23 months who were born at least 24 months after the previous surviving child [RC 2] Number of mothers with more than one biological child 0-59 months * RC 4 is intended to measure maternal recall, however the program measured verification by card Sampling Design The sample size was determined using a 2 stage 30x10 clusters sampling method for surveys as outlined by the CSTS+ KPC Guidance. This model uses the following formula to calculate the sample size: N= Z²(1-P)P E² N= Sample size; Z=1.96 (for a confidence interval of 95%); P= Known prevalence; E=% within=±0.05. Thirty clusters were randomly selected from a list of all the villages in Chitipa, taking into account the differences in population size of the villages (Proportional Population Cluster Sampling method). See Annex B for the sampling framework. For each cluster, interviews were conducted with 10 households having children under the age of two years. Upon arriving in a village, the village headman was asked to identify a place considered close to the central point of the village. At the central site, a member of the survey team spun a pen. The survey team started in the direction of the pen (pointed end) to the first house. If the object pointed in the direction where there were no houses, the procedure would be repeated until there were houses in that direction. The interview started at the nearest household and continued to the next one in the same direction until the required number of households per cluster was met. In cases where the chosen direction had less than the required sample the object would be re-spun to change direction at the farthest household and the team proceeded in that direction until the total required sample in that cluster was met. Interviewer Recruitment Interviewers for the Final KPC survey were recruited from among the project leadership and staff and local partner agencies. Interviewers consisted of the Child Survival Deputy Project Page | 8 Manager, Project Supervisors and Health Promoters along with representatives from the District Ministry of Health and the Department of Social Welfare. All interviewers were familiar with the local cultural context, proficient in the necessary languages spoken in the communities and had experience with conducting interviews. Additionally, several members of the project staff had also participated in the project’s Baseline and Midterm KPC surveys. Interviewer Training The interviewers completed three days of training to review the survey sampling methodology as well as how to read the survey questionnaire fluently and code responses accurately. This training was conducted by the Child Survival (CS) Project Manager and Deputy Manager. Interviewer training covered the objectives of the KPC survey, the process and rationale of 30 cluster methodology, basic survey interview techniques and an in-depth review of the survey content. The training sessions were interactive and built on the survey team members’ previous experience with conducting surveys. In pairs, surveyors practiced reading the questionnaire and coding responses accurately. In addition, interviewers received training on how to check for accurate coding on survey questionnaires. Pre-testing of the questionnaire took place in neighboring villages and provided interviewers with an additional opportunity to practice conducting the survey and coding responses on the survey form. It also allowed the survey team to take note of potential problems that may be encountered in the field and to strategize ways to overcome the identified challenges. An additional half day of training was conducted after pre-testing to review the revised questionnaire form and to discuss problems that were observed in the field or in the coding of the questionnaires. As a result of pretesting, additional possible responses were added to the sustainability question. See Annex C for a complete listing of person/roles involved with the surveying process. Data collection The survey staff were divided into three teams and were each assigned to ten villages that had been randomly selected proportional to size for inclusion in the survey. The fourth team was responsible for data entry and cleaning. The interviews were conducted over a period of ten days. The most significant constraint involved long distances to the villages furthest from the project office, which at times could not be covered by vehicle, because of lack of sufficient roads. The survey questionnaires were checked for accuracy in the field, so that any missing information could be gathered before the team left the village. The completed survey questionnaires were reviewed by the Project Manager, as they were received at the project office. Data Entry and Analysis Data entry began as the questionnaires were returned to the project office and was finished within ten days of field work. The data was entered by the Project Office Secretary, the Project Office Assistant, the Project Accountant and the Project Manager. Preliminary data analysis was conducted in EpiInfo by the Project Manager so that results could be immediately discussed with the survey team. Final data analysis and quality control checks were conducted by the WR HQ Maternal and Child Health Specialist. Frequencies, means, and cross tabulations of key variables were performed as appropriate. Page | 9 V. RESULTS The survey interviewed 300 mothers/caretakers, 44.3% (133/300) were 25 years of age or younger, while 55.7% (167/300) were 25 years old or older with an average age of 26. The children covered in the survey were 47.7% (143/300) female and 52.3% male. There were 69 children ages 0-5 months (23.0%), 77 children 6-11 months (25.7%) and 154 children 12-23 months (51.3%). The average child age was between 11 and 12 months of age. Of the mothers surveyed, 44.7% had one biological child less than 60 months old, 52.7% had two children and 2.7% had three with a mean of 1.6 children 0-59 months of age per respondent. Table 10 Project Indicator Results Indicator Baseline Percentage Midterm Percentage Final EOP Target Numerator Denominator Percent Confidence Interval CARE SEEKING Percentage of caretakers of children 0-23m who know at least 2 childhood illness danger signs for seeking care immediately (RC) 71.1% 88.7% 287/300 95.7% 93.3%- 98.0% 80% ▲ HOME MANAGEMENT Percentage of sick children age 0-23m who received increased fluids and continued feeding during an illness in the past two weeks (RC) 3.9% 17.8% 51/226 22.6% 16.7%- 28.4% 50% ▼ IMMUNIZATION Percentage of all children 12-23m fully immunized (BCG, Polio3, DPT3, and Measles) before 24 months as verified by card. *BL includes children 12-23 months who had a BCG scar and children 9-11 months who had a DPT3, Polio3 and measles vaccine 68.8% 63.7% 113/154 73.4% 65.5%- 81.3% 80% ▼ Percentage of children age 12-23m who are fully vaccinated (against the five vaccine￾preventable diseases) before the first birthday (RC) *BL includes children 12-23 months who had a BCG scar and children 9-11 months who had a DPT3, Polio3 and measles vaccine 68.8% 75.2% 99/138 71.7% 64.1%- 79.4% RC Percentage of children age 12-23m who received a measles vaccine **Includes only measles verified by card 55.2% 64.1% 133/179 74.3% 66.3%- 82.3% RC MALARIA Percentage of children age 0-23m who slept under an insecticide-treated net the previous night (RC) *BL includes all nets regardless of insecticide treatment 41.0% 39.7% 131/300 43.7% 36.9%- 50.4% 60% ▼ Percentage of children 0-23m with suspected malaria in the previous 2 weeks who sought treatment from a trained provider within 24 hours of illness onset 18% 41.4% 40/96 41.7% 29.6%- 53.7% 50% ▼ PNEUMONIA CASE MANAGEMENT Page | 10 Percentage of children 0-23m with rapid/difficult breathing (suspected pneumonia) in the previous 2 weeks who sought treatment from a trained provider within 24 hours 20.9% 31.0% 11/25 44.0% 24.9%- 63.1% 50% ▼ CONTROL OF DIARRHEAL DISEASE Percentage of children 0-23m with diarrhea in the previous 2 weeks who received ORT (home available fluids or ORS) 8% 58.3% 58/90 64.4% 52.8%- 76.1% 60% ▲ Percentage of mothers with children age 0-23m who report that they wash their hands with soap/ash before food preparation, before feeding children, after defecation, and after attending to a child who has defecated (RC) *BL from LRA1 3.9% 12.3% 58/300 19.3% 15.0%- 23.7% 60% ▼ NUTRITION Percentage of children 0-5m who were exclusively breastfed during the last 24 hours (RC) 40.0% 84.2% 55/69 79.7% 69.8%- 89.6% 60% ▲ Percentage of children 6-9m who received breastmilk and complementary foods during the last 24 hours (RC) 39.8% 80.5% 52/61 85.2% 76.0%- 94.5% 70% ▲ Percentage of children 6-11 months who received at least 1 dose of Vitamin A and children 12-23 months who received at least 2 doses of Vitamin A in the previous 12 months, as evidenced by card -- 11.1% 15/231 6.5% 2.0%- 11.0% 60% ▼ HIV/AIDS PREVENTION Percentage of mothers with children age 0-23m who cite at least two known ways of reducing the risk of HIV infection (RC) 67.5% 76.5% 257/300 85.7% 81.7%- 89.7% 80% ▲ Percentage of children age 0-23m whose births were attended by skilled health personnel (RC) 55.2% 61.0% 218/300 72.7% 64.9%- 80.4% 70% ▲ RAPID CATCH (NOT INCLUDED AS PROGRAM GOALS) Percentage of children age 0-23m who are underweight (-2SD from the median weight￾for-age, according to the 1978 WHO/NCHS reference population) 29.9% 10.4% 45/297 15.2% 11.2%- 19.1% RC Percentage of children age 0-23m who were born at least 24 months after the previous surviving child 39.5% 83.5% 145/166 87.3% 81.3%- 93.4% RC Percentage of mothers with children 0-23m who received at least two tetanus toxoid injections before the birth of their youngest child **Includes 2TT verified by card Of mothers with cards 64.0% 53.0% 82.4% 178/300 59.3% 92.5% 51.7%- 66.9% RC See Annex D for a detailed description of Baseline, Midterm and Final KPC results. VI. DISCUSSION Discussion of Program Indicators Community Integrated Management of Childhood Illness Page | 11 The number of caretakers who were able to cite two or more danger signs increased steadily from the Baseline of 77.1% to 88.7% at Midterm and 95.7% at Final, surpassing the EOP target of 80%. Moreover, the increases at both Midterm and Final are statistically significant based on a 95% confidence interval. Results were cross tabulated against maternal age (< 25 years and ≥ 25 years) and no significant difference was seen. When asked what contributed to the success of this indicator staff cited the strong relationships and consistent messaging at the health centers. A statistically significant increase was also seen in the percentage of children who were offered increased fluids and continued feeding during illness from the Baseline of 3.9% to the Final of 22.6%. This however, fell below the project target of 60%. Run independently, the final percentage of children offered increased fluids was 29.2% while those children receiving increased or continued foods was 49.6%. Cross tabulations of maternal age, child age of less than six months and incidence of diarrhea in the last two weeks showed no significant difference. Immunization The Baseline immunization rate included children 12 months and older who had a BCG scar and those children 9-11 months who had been vaccinated with DPT3, Polio3 and measles verified by card. The survey questionnaire was changed at the Midterm KPC to bring the project in line with current international standards thus making comparisons to the Baseline difficult. According to the Rapid Catch definition, the percentage of children with cards who were completely immunized (DPT3, Polio3 and measles) by the first birthday fell from 75.2% to 71.7% from Midterm to Final. This downward trend was identified following the March monitoring survey and the results were shared with the District Health Management Team. At that time, the District MOH was reporting a fully immunized rate of around 77%. The MOH and project staff related the dip in immunization levels to the lack of immunization cards due to printing contract issues. However, the percentage child vaccination cards seen by survey interviewers increased from 76.3% at Midterm to 88.3% at Final. The DHMT also stated that they have had some recent issues of stock outs with vaccinations that could have contributed to the drop in immunization rates. Keeping with the intent of the original indicator, Midterm and Final immunization rates were also calculated to include children, 12-23 months, fully immunized (BCG, DPT3, Polio3 and measles) at the time of the survey. At Midterm 63.7% of children less than 24 months were fully vaccinated with 73.4% at Final, below the EOP target of 80%. The Rapid CATCH indicators include the number of children who received a measles vaccine based upon maternal recall as an estimate of overall immunization rate regardless of card status. Using this definition, 94.8% children received a measles vaccine at Midterm and 91.6% at Final. Based on a 95% confidence interval, the differences seen in immunization rates from Midterm to Final are not statistically significant. Cross tabulations by maternal age and child gender were run for both immunization by 12 and 24 months and no difference in immunization coverage was observed. Malaria The number of children who slept under an insecticide treated net the night before was 43.7% at Final. This represents only a slight, and statistically insignificant, change from the 39.7% at Midterm or the 41% at Baseline, although the Baseline included all nets regardless of treatment. Page | 12 However, overall net ownership has increased dramatically from 44.2% in 2005, to 55.1% in 2007 to 88.0% in 2009. The percentage of those with nets that have been treated in the last six months or were long-lasting nets was 65.5% (173/264), up from 45% at Midterm. Of those who responded that they had nets, 59.6% (173/264) had the net hanging while 45.8% (121/264) of the households were able to produce the net, but it was not hanging and 4.5% (12/264) were not able to locate the net. There was a net dipping campaign conducted by the MOH about a week before the survey and staff reported that people fear hanging the nets the same day after dipping as it is believed the chemicals will cause skin irritation. Seasonality changes may have played a role in ITN usage as the Baseline was conducted in May, the Midterm was in January at the height of the hot season and the Final was conducted in July, the coldest part of the year when mosquitoes are less bothersome. The project does have some monitoring information that indicated increased usage during March of 2009. Cross tabulations showed insignificant differences when comparing maternal age (<25/≥25), child age (0-5 months, 6-11 months and 12-24 months) and child gender. The percentage of children with suspected malaria taken to the health center within 24 hours improved from Baseline (15.5%) to Midterm (41.4%). However, no additional improvement was achieved from Midterm to Final (41.7%) missing the project’s target of 50%. Despite 93.3% (290/300) of the caretakers listing fever as a danger sign requiring treatment at the health facility, the accompanying behavior was lagging, perhaps related to decreases in access to malaria treatment at community level. In October 2008 the Malawi MOH changed the first line treatment for malaria from Fansidar to Artemisinin Combination Therapy (ACT, commonly referred to as LA). The MOH has yet to train the HSAs how to prescribe the new drug; therefore, there is no available treatment for malaria at the community/health post level. The MOH has plans to train and stock the HSAs in the new drug, but the timing is yet unknown. In addition, the Promoter staff reported issues with the quality of services provided at specific health centers including lack of time spent examining the child, lack of explanation or instructions when drugs were prescribed or even a lack of weekend hours. These issues were raised with the DHMT in May 2009 and ongoing dialog between the program and MOH staff has continued to identify problematic health centers. However, this is inconsistent with the staff’s earlier support of the health centers and Final Evaluation interviews where all mothers stated that the last time they went to the health center they were assisted. Nevertheless, the lack of improvement in this indicator is especially concerning when compared to prevalence and severity of malaria in children under five. Based on the program’s Health Information System, malaria represented 38% of all deaths for children less than five years of age in the project area from August 2007 to July 2009. Malaria prevention and immediate care seeking remains a critical need for the children of Chitipa. Pneumonia Case Management The percentage of children with rapid or difficult breathing (suspected pneumonia) that were treated at the health facility had risen from the Baseline of 20.9% to 31.0% at Midterm to 44.0% at Final. The small number of cases (25) resulting in a wide confidence interval (24.9%-63.1% at Final) makes it difficult to determine if this increase is statistically significant. Nonetheless, this Page | 13 indicator fell below the project’s target of 50%. The staff cited difficulties with the quality of health services and unpredictable hours at the health center to be a contributing factor, as discussed with malaria. Control of Diarrheal Diseases Large improvements can be seen in the percentage of children with diarrhea in the past two weeks who were offered ORT (ORS, home available fluids or breastfeeding). At Baseline, only 8% of children were given ORT which increased at Midterm to 58.3% and 64.4% at the Final, surpassing the end of project target of 60%. If “tak(ing) the child to the health center” was included as a positive response, assuming that the child will receive ORS, the percentage rose to 64.6% at Midterm 82.2% at Final. These improvements can be seen without differentiation between maternal age (<25/ ≥25), child age (0-5 months, 6-11 months or 12-23 months) or gender of the child. Hand washing at all four critical times remained a challenge throughout the project. At Baseline 3.9% of caretakers reported washing hands with soap or ash after defecation, after helping a child who has defecated, before food preparation and before feeding a child. This improved to 12.3% in 2007 and to 19.3% in July of 2009; yet fell far below the target of 60%. Examining the four times individually: 85.7% of caretakers listed after defecation, 68.3% listed after helping a child who has defecated, 55.7% cited before food preparation and 38.0% listed before feeding a child. The Malawi MOH policy is that hand washing should occur at five times; after using the toilet, after changing a child’s nappy, before feeding a child or breastfeeding, before preparing food, and before eating. Only 15.0% of caretakers reported washing at all five of these times. As water availability is a concern in Chitipa the possibility of this hindering hand washing was investigated. However, staff felt that water scarcity had little impact on the caretaker’s behavior of hand washing and while not statistically valid, this was supported by a sub-analysis of the Final KPC results by supervision area. Nutrition The large improvements seen in exclusive breastfeeding and complementary feeding rates from the Baseline to Midterm were maintained through to the Final representing almost a two fold increase in both indicators. Exclusive breastfeeding in children 0-5 months increased from 40.0% at Baseline to 84.2% at Midterm and was maintained at 79.7% in the Final survey, easily surpassing the project target of 60%. These gains were seen uniformly regardless of maternal age or the number of children living in the household (biological or otherwise). Continued breastfeeding and complementary feeding in children 6-9 months rose from 39.8% to 80.5% to 82.5%, again well surpassing the target of 70%. Breastfeeding through two years of age is also strongly supported by the District MOH. The District Hospital has multiple signs reading “Chitipa District Hospital is a Baby Friendly Hospital. We support, promote and protect breastfeeding” thus providing a consistent message from the village to the District level. Written verification of Vitamin A has been an ongoing concern of the project. The project’s target was that 60% of children 6-11 months were to receive one dose of Vitamin A and children 12-23 months were to receive two doses of Vitamin A within the past twelve months as evidenced in their health card. This indicator was not measured at the Baseline survey, but had a statistically insignificant drop from 11.1% at Midterm to 6.0% at Final. The District MOH is Page | 14 committed to providing Vitamin A supplementation and does so through outreach campaigns held throughout the district every six months. However, the DHMT specifically avoids asking caretakers to bring their health cards in fear that those who do not have cards will not attend. When examining Vitamin A coverage by maternal recall rates from Midterm to Final, a similar statistically insignificant dip was seen from 95.0% to 89.2%. HIV Prevention There has been a steady increase in caretakers’ knowledge of methods to help prevent the spread of HIV. The percentage of mothers able to list two known methods of preventing HIV increased from 67.5% at Baseline to 76.5% at Midterm to 85.7% at Final, exceeding the program target of 80%. Numerator and denominator values are no longer available for the baseline data so we are unable to determine if this change is statistically significant. However, as the project implemented the HIV lessons in FY09 it is difficult to link this increase in knowledge to programmatic activities. Improvements have also been seen in the percentage of children 0-23 months whose births were attended by skilled personnel (a doctor or nurse) from 55.2% at Baseline to 61.0% at Midterm to 72.7% at the Final exceeding the target of 70%. Unfortunately, PMTCT services are only available at the District Hospital. However, it is hoped that increased health center births will facilitate this treatment as it becomes available at the health center level. Little difference was observed in skilled delivery based on maternal age. Discussion of Other Results The percentage of children age 0-23 months who were underweight as defined by the Rapid CATCH indicator (-2SD from the median weight for age, according to the 1978 WHO reference population) decreased from 29.9% at Baseline to 10.4% at Midterm to 15.2% at the time of the Final survey. Overall, this is a noteworthy change in the prevalence of underweight children from Baseline to Final, although there was a small and not statistically significant rebound from Midterm to Final. Cross tabulations revealed little difference between the percentages of male and female children who were underweight. It is encouraging to see movement on this indicator in light of the increased exclusive and continued breastfeeding practices. Birth Spacing In a 2006 monitoring survey, only 39.5% of children 0-23 months were born at least 24 months after a previous surviving child. At Midterm, this rose to 83.5% and at the Final 87.3% of the children surveyed had adequate birth spacing. While there are limitations in comparing results from surveys with varying sampling methodologies and rigor, the confidence intervals suggest that the change from Baseline to Final is significant. The program celebrates this positive behavior for maternal and child health, but timing significantly decreases the probability that this increase (especially between the Baseline and the Midterm) was a result of programmatic activities. Maternal Care According to the Final, 59.3% of mothers received two or more doses of tetanus toxoid (TT) vaccine representing an increase over the 53.0% at Midterm; though both are lower than the Page | 15 Baseline indication of 64.0%. Contributing to this may be difficulties with documentation and possible stock-outs as previously mentioned. It should be noted that the Rapid CATCH definition includes maternal recall for two TT doses; however, the program measured two TT doses verified by card. If we examine the percentage of mothers with two TT doses compared to those mothers that had health cards at the time of the survey the percentages rise to 83.8% at Baseline to 82.4% at Midterm and 92.5% at Final. Cross tabulations by maternal age showed little variance. Additionally, the number of mothers who had two verified doses of preventative malaria treatment (SP) rose from 58.2% at Baseline to 69.1% at Midterm and 71.7% at Final. While the percentage of mothers reporting the use of prenatal iron held steady at 85.2% to 86.3% to 88.3%. Sustainability The sustainability of the Care Group model using unpaid volunteers has been proven effective and sustainable in other WR child survival programs, particularly evidenced in Mozambique where the Care Group Model was first implemented9,10. In order to assess the fidelity of the World Relief Tube Poka Child Survival Project in Malawi, a question was added to the Midterm and Final surveys to evaluate respondents’ exposure to the program two and four years after its inception. At Midterm, 46.0% of households surveyed were visited by a World Relief/ Tube Poka volunteer in the last two weeks and 32.3% reported that they had not yet been visited in their home. At Final, the percentage of visitation in the last two weeks fell to 17.3%, 42.3% responded that they had been visited in the last month, 32.0% over a month ago and 25.7% responded never. Staff stated that within the Malawi context, the term “never” is different from “never ever”, but nonetheless this represents a critical area of ongoing concern. While the Final evaluation team will further explore this issue, changes to the Care Group model in supervision and volunteer selection coupled with implementation difficulties appear to be strong contributing factors. External Comparisons Table 11 Comparison of Indicators with National Data Sources Indicators DHS 200411 (U5 unless indicated) MICS 200612 (U5 unless indicated) Final KPC 2009 (U2 unless indicated) Percent children with diarrhea treated with ORT 69.0%* -- 64.4% Percent of mothers who offered more liquids when child was ill (DHS= ill with diarrhea) 35.9% -- 29.2% Percent of mothers who offered more or the same food when child was ill (DHS= ill with diarrhea) 55.1% -- 49.6% Percent mothers who sought treatment from HF for child with fast/difficult breathing 19.3%* -- 44.0% Percent of mothers who sought treatment from HF within 48 hours for child with fever 21.0%* -- 49.0% Percent of HH who own a bed net 39.1%* 35.4%** 88.0% Percent children who slept under bed net last night 15.0%* 4.6%** 43.7% Percent children underweight 22.8%* 19.5%** 15.2% Percent of children who were breastfed within hour of birth 68.6%* -- 79.3% Percent of children 0-5 months who are exclusively breastfed 53.3% 55.2%* 79.7% Percent children 12-23months fully immunized at time of survey 51.1%* 62.0%* 73.4% Percent with four or more ANC visits 55.7%* -- 46.3% Page | 16 Percent who took iron tablets during pregnancy -- 31.8%** 88.3% Percent delivery by a health professional 52.0%* 50.4%* 72.7% * Rural Rates **Rates for Chitipa District Information Dissemination Data collection and entry was occurred August 6th-13th, 2009. An initial snapshot of key indicators was run and discussed with the survey team as the very first level of sharing the survey outcomes. Results of this survey together with the results of the Final Evaluation were compiled and shared by the evaluation consultant with stakeholders in Chitipa and Lilongwe as follows: July 31st, 2009 Tube Poka Supervisors and select Health Promoters August 12th, 2009 Chitipa District Executive Committee (the technical arm of the District Development Secretariat), UNICEF and PSI August 14th, 2009 World Relief Malawi Leadership Team August 14th, 2009 USAID Mission After the departure of the evaluator and WR Headquarters staff, the project team continued to hold a series of dissemination meetings across the District with various partners at community level the week of August 17th-21st, 2009. 2 Tube Poka Project Census, Chitipa, Malawi 2006 3 Chitipa Socioeconomic Profile 2002 4 Demographic Health Survey. (2004). Malawi Demographic Health Survey Report. 5 Chitipa District Socio Economic Profile. Republic of Malawi October 2002 6 Global Strategy on Infant and Young Child Feeding, World Health Organization (2003) 7 A. Edward, P. Ernst, C. Taylor, S. Becker, E. Mazive, H. Perry . Examining the evidence of under-five mortality reduction in a community-based programme in Gaza, Mozambique. Transactions of the Royal Society of Tropical Medicine and Hygiene, Volume 101, Issue 8, Pages 814-82 8 WR Mozambique Final Evaluation Findings Report, Taylor, C. 2003 Page | 17 9 WR Mozambique Final Evaluation Findings Report, Taylor, C. 2003 10 A. Edward, P. Ernst, C. Taylor, S. Becker, E. Mazive, H. Perry . Examining the evidence of under-five mortality reduction in a community-based programme in Gaza, Mozambique. Transactions of the Royal Society of Tropical Medicine and Hygiene, Volume 101, Issue 8, Pages 814-82 11 Demographic Health Survey. (2004). Malawi Demographic Health Survey Report. 12 Monitoring the Situation of Children and Women. Malawi Multiple Indicator Cluster Survey. (2006). MICS Preliminary Report. Annex A | Page | 1 ANNEX A: FINAL KPC SURVEY QUESTIONNAIRE CHITIPA DISTRICT - TUBE POKA CHILD SURVIVAL PROGRAM IDENTIFICATION SECTION: ________ SUPERVISION AREA: __________ Village: _________________ Household No. __________ Checked by: _________________ (Name of Supervisor) INTERVIEWER: Begin by introducing yourself, for example, “We are from the Tube Poka Child Survival Program and we would like to ask you for some information that will help us improve the health of people in this area.” Ask the mother or caretaker of the child all the questions as clearly as possible in order to get appropriate responses. Probing may be required in certain instances to try and reach the desired response from the interviewee. Please keep in mind that this survey targets mothers of children less than 24 months of age. Date of interview: Day __ __ / Month __ __ /Year __ __ __ __ Name of interviewer: ____________________________________ Name of Mother: _______________________________________ BASIC DEMOGRAPHIC INFORMATION 1. How old are you? __ __ years 2. How many children living in this household are under the age of five? __ child/children 3. How many of those children are your biological children? __ child/children 4. READ ONE OF THE FOLLOWING QUESTIONS BASED UPON MOTHER’S RESPONSE TO Q.3 ONLY 1 CHILD UNDER FIVE: “What is the name, sex, and date of birth of that child?” Annex A | Page | 2 MORE THAN 1 CHILD UNDER FIVE: “What are the names, sexes, and dates of birth of your two youngest biological children?” NAME SEX DATE OF BIRTH AGE IN MONTHS 1 1. MALE 2. FEMALE __ __ / __ __ / __ __ DD MM YY 2 1. MALE 2. FEMALE __ __ / __ __ / __ __ DD MM YY IF THE YOUNGEST CHILD IS 24 MONTHS OR OLDER, STOP AND GO TO THE NEXT HOUSE. ALL SUBSEQUENT QUESTIONS PERTAIN TO THE YOUNGEST CHILD UNDER THE AGE OF TWO YEARS. BREASTFEEDING & COMPLEMENTARY FEEDING 5. Are you breastfeeding (name of child) now? A. Yes  Go to 7 B. No 6. Have you ever breastfeed (name of child)? A. Yes B. No  Go to 8 7. How long after birth did you first put (name of child) to the breast? A. Immediately/within first hour after delivery B. After the first hour after delivery C. After eight hours after delivery Annex A | Page | 3 D. Don’t know 8. What types of liquids and foods did (name of child) consume yesterday during the day or at night? PLACE A CHECK MARK IN THE BOX NEXT TO EACH ITEM CONSUMED. LIQUID/FOOD CONSUMED IN LAST 24 HOURS? A Breastmilk? B Plain water? C Other liquids? D Mashed, pureed, solid, or semi-solid foods? E Anything else? SPECIFY: ________________ ________________ ________________ _________ Illness Recognition and Care Seeking 9. Sometimes children get sick and need to receive care or treatment for illnesses. What are the signs of illness that would indicate your child needs treatment? DO NOT PROMPT. CIRCLE ALL MENTIONED. A. Don’t know B. Looks unwell or not playing normally C. Not eating, drinking, or breastfeeding D. Lethargic or difficult to wake E. High fever Annex A | Page | 4 F. Fast or difficult breathing G. Vomits everything H. Convulsions I. Gets worse despite home care J. Looks dehydrated (dry mouth or no tears) K. Other ________________________________ (SPECIFY) 10. Did (name of child) experience any of the following in the past two weeks? READ CHOICES ALOUD AND CIRCLE ALL MENTIONED BY RESPONDENT. A. Diarrhea A.1 How many times did the child have a loose stool? Less than 3 3 or more (CIRCLE ONE) B. Blood in stool C. Cough D. Rapid or difficult breathing E. Fever F. Malaria G. Convulsions H. Other_________________________ (SPECIFY) I. None of the above  GO TO Q-22 11. When (name of child) was sick, was he/she offered less than usual to drink, about the same amount, or more than usual to drink?” A. Less than usual B. Same amount Annex A | Page | 5 C. More than usual 12. When (name of child) was sick, was he/she offered less than usual to eat, about the same amount, or more than usual to eat? A. Less than usual B. Same amount C. More than usual Diarrhea THE FOLLOWING QUESTION PERTAINS ONLY TO THE CHILDREN WHO HAD DIARRHEA IN Question 10-A 13. When (name of child) had diarrhea, did you give the child anything? DO NOT PROMPT. MULTIPLE RESPONSES POSSIBLE. AFTER EACH RESPONSE, ASK: ANYTHING ELSE? A. Nothing B. ORS sachet C. Sugar-salt solution D. Cereal based ORT (rice water, maize water) E. Breastmilk F. Water G. Other available drinks H. Medication for diarrhea I. Take child to the hospital/clinic J. Other ____________________________________ SPECIFY PNEUMONIA Annex A | Page | 6 THE FOLLOWING QUESTIONS PERTAIN ONLY TO THE CHILDREN WHO HAD RAPID OR DIFFICULT BREATHING Question 10-D. 14. Did you seek treatment for your child? A. Yes B. No  GO TO Q-22 15. From whom did you seek treatment when (name of child) had difficulty in breathing? DO NOT PROMPT. MULTIPLE RESPONSES POSSIBLE. A. Hospital* B. Health Center/Health Post* C. Injectionist  GO TO Q-22 D. Traditional birth attendant  GO TO Q-22 E. Traditional healer  GO TO Q-22 F. Pharmacy/shop  GO TO Q-22 G. Relatives and friends  GO TO Q-22 H. Others (specify) ______________  GO TO Q-22 16. *IF THE MOTHER ANSWERED A or B ABOVE, ASK: How soon after the difficulty in breathing began did (name of child) receive treatment? DO NOT PROMPT, JUST ASK HOW LONG IT WAS UNTIL THE CHILD RECEIVED TREATMENT. A. Less than one day (within 24h) B. After one day (24h – 48h) C. Two days or more D. Don’t know MALARIA CONTROL Annex A | Page | 7 THE FOLLOWING QUESTIONS PERTAIN ONLY TO THE CHILDREN WHO HAD FEVER, MALARIA OR CONVULSIONS IN Question 10-E,F,G. 17. When (name of child) had fever, what treatment did you give? DO NOT PROMPT. MULTIPLE RESPONSES POSSIBLE. A. Take the child to the hospital or health center* B. Home treatment  GO TO Q-19 C. Wet the child to decrease fever  GO TO Q-19 D. Other (specify) ____________________  GO TO Q-19 18. *IF THE MOTHER ANSWERED A ABOVE, ASK: How soon after the fever started was (name of child) treated at the hospital or health center? DO NOT PROMPT, JUST ASK HOW LONG IT WAS UNTIL THE CHILD RECEIVED TREATMENT. A. Less than one day (within 24h) B. After one day (24h – 48h) C. Two days or more D. Did not receive treatment E. Don’t know 19. Did (name of child) take any drugs for the fever? A. Yes B. No  GO TO Q-22 C. Don’t know  GO TO Q-22 20. What drugs did (name of child) take? DO NOT PROMPT. A. SP/Fansidar Annex A | Page | 8 B. La C. Chloroquine D. Amodiaquine E. Quinine F. ACT G. Other _______________________ SPECIFY H. Don’t know 21. When did (name of child) take the drugs for the fever? DO NOT PROMPT, JUST ASK HOW LONG IT WAS UNTIL THE CHILD RECEIVED THE DRUGS. A. Less than one day (within 24h) B. After one day (24h – 48h) C. Two days or more D. Don’t know Malaria Prevention 22. Do you have any mosquito nets in your house? IF YES, ASK: Can I see it? A. Yes (Net seen hanging) B. Yes (Net seen, but not hanging) C. Yes (Net NOT seen) D. No  GO TO Q-25 E. Doesn’t know  GO TO Q-25 Annex A | Page | 9 23. Who slept under a mosquito net last night? DO NOT PROMPT. CIRCLE ALL THAT APPLY. A. Child (name of child) B. Respondent (mother) C. Other individual(s) ______________________________ (SPECIFY) D. Did not use a net 24. Was the mosquito net ever soaked or dipped in a liquid to repel mosquitoes or bugs? A. Yes, within the past 6 months. B. Yes, within the past 1 year. C. No D. Long-lasting net; does not need to be retreated E. Doesn’t know Growth Monitoring 25. Does (name of child) have a growth monitoring card? ASK THE MOTHER TO SEE THE CARD. a. Yes (Card is seen by interviewer) b. Yes, but card is missing or lost (Card is NOT seen by interviewer)  GO TO Q-27 c. No  GO TO Q-27 26. LOOK AT THE GROWTH MONITORING CARD OF THE CHILD AND RECORD THE FOLLOWING INFORMATION: Has the child been weighed in the last 4 months? A. Yes B. No 27. May I weigh (name of child)? Annex A | Page | 10 A. Yes B. No  Go to Q-29 28. IF MOTHER AGREES, WEIGH THE CHILD AND RECORD WEIGHT BELOW. RECORD TO THE NEAREST TENTH. ___ ___ . ___ KILOGRAMS Immunizations 29. Did (name of child) ever receive an injection in the leg to prevent measles? A. Yes B. No C. Don’t know 30. Did (name of child) receive a Vitamin A dose in the last 6 months? A. Yes B. No C. Don’t know 31. Do you have a card or health passbook where (name of child)’s vaccinations are written down? ASK TO SEE THE CARD/HEALTH PASSBOOK. A. Yes (Card is seen by interviewer) B. Yes, but card is missing or lost (Card is NOT seen by interviewer)GO TO Q-33 C. No (Never had a card) GO TO Q-33 D. Doesn’t know GO TO Q-33 32. RECORD INFORMATION EXACTLY AS IT APPEARS ON (NAME OF CHILD’S) VACCINATION CARD. Annex A | Page | 11 DAY MONTH YEAR BCG Polio 0 Polio 1 Polio 2 Polio 3 DPT 1+HepB+Hib DPT 2+ HepB+Hib DPT 3+ HepB+Hib Measles Vitamin A Water and Sanitation THE FOLLOWING QUESTIONS APPLY TO THE MOTHER 33. Do you treat your water in any way to make it safer for drinking? A. Yes B. No GO TO Q-35 C. Don’t know GO TO Q-35 34. If yes, what do you usually do to the water to make it safer to drink? DO NOT PROMPT. CIRCLE ALL MENTIONED. A. Let it stand and settle/sedimentation B. Strain it through cloth C. Boil D. Add bleach/chlorine/Waterguard E. Water filter (ceramic, sand, composite) F. Solar disinfection G. Other Annex A | Page | 12 H. Don’t know 35. When do you wash your hands? DO NOT PROMPT. CIRCLE ALL MENTIONED. A. Never B. Before food preparation C. Before feeding child D. After defecation/visiting toilet E. After attending to a child who has defecated/soiled F. Before eating G. Other ___________________________ (SPECIFY) 36. Do you use soap or ash when washing your hands? A. Yes B. No  GO TO Q-38 37. ASK TO SEE SOAP OR OTHER SUBSTANCE USED FOR HANDWASHING. A. Soap observed B. Soap substitute (e.g. ash) observed C. Soap NOT observed Maternal and Neonatal Health 38. When you were pregnant with (name of child) did you go to the health center for antenatal checkups? If yes, how many times? A. Yes, once B. Yes, twice C. Yes, three times Annex A | Page | 13 D. Yes, four or more times E. No F. Don’t know 39. While you were pregnant with (name of child) did you receive an antenatal health card? ASK TO SEE THE CARD. A. Yes (Card is seen by interviewer) B. Yes (Card is NOT seen by interviewer) GO TO Q-42 C. No GO TO Q-42 D. Doesn’t know GO TO Q-42 40. CHECK CARD TO SEE HOW MANY TETANUS INJECTIONS THE MOTHER RECEIVED: A. None B. One C. Two D. More than two 41. CHECK CARD TO SEE HOW MANY DOSES OF SP THE MOTHER RECEIVED: A. None B. One C. Two D. More than two 42. Did you take iron tablets daily during your last pregnancy? A. Yes Annex A | Page | 14 B. No 43. Now I would like to ask you about the time when you gave birth to (Name of child). Who assisted you with (name of child) delivery? (DO NOT PROMPT.) A. Doctor B. Nurse/midwife C. Traditional birth attendant ________________________ (NAME) D. Family member or friend________________________________________ (SPECIFY RELATIONSHIP TO RESPONDENT) E. Other ________________________________ (SPECIFY) F. No one 44. After (name of child) was born, did any health care provider or traditional birth attendant check on (name of child’s) health? A. Yes B. No GO TO Q-47 C. Don’t know GO TO Q-47 45. Who checked on (name of child’s) health at that time? (DO NOT PROMT. CIRCLE ALL MENTIONED.) A. Doctor B. Nurse/midwife C. Traditional birth attendant D. Family member or friend________________________________________ (SPECIFY RELATIONSHIP TO RESPONDENT) E. Other ________________________________ (SPECIFY) Annex A | Page | 15 F. No one 46. How long after the birth of (name of child) did the first check take place? A. Less than one day (within 24h) B. After one day (24h – 48h)) C. Two days or more D. Don’t know HIV/AIDS 47. Have you ever heard of HIV/AIDS? A. Yes B. No  Go to Q-49 48. What can a person do to avoid getting AIDS or the virus that causes AIDS? (DO NOT PROMPT. CIRCLE ALL MENTIONED.) A. Abstain from sex B. Use condoms C. Limit sex to one partner/Stay faithful to one partner D. HIV Testing and Counseling/VCT E. Limit number of sexual partners F. Avoid sex with prostitutes G. Avoid sex with persons who have many partners H. Avoid intercourse with persons of the same sex Annex A | Page | 16 I. Avoid sex with persons who inject drugs intravenously J. Avoid blood transfusions K. Avoid injections L. Avoid sharing razors, blades M. Avoid kissing N. Avoid mosquito bites O. Seek protection from traditional healer P. Nothing Q. Other________________________ (SPECIFY) R. Doesn’t know Sustainability 49. When did World Relief/Tube Poke volunteer visit you? A. Last week B. Past two weeks C. Past three weeks D. Past four weeks E. Never F. Other (Added after pre-testing) THE END Thank you for responding to all these questions. Annex B | Page | 1 ANNEX B: SAMPLING FRAMEWORK Annex B | Page | 2 Annex C | Page | 1 ANNEX C: FINAL KPC SURVEY TEAM- LIST OF PERSONS AND ROLES Malawi Child Survival Project KPC Survey Participants – July 1-14, 2009 No. Name Department Position 1 Richard Thindwa World Relief Malawi – TPCSP Deputy Project Director 2 Joseph Simwaka Chitipa District Hospital Environmental Health Intern 3 Thandie Msukwa World Relief Malawi – TPCSP Health Promoter 4 Foreward Chilanga World Relief Malawi – TPCSP Health Education Supervisor 5 Misheck Mdambo District Social Welfare Child Protection Officer 6 Jill M Mtambo World Relief Malawi – TPCSP Health Education Supervisor 7 George Nundwe Chitipa District Hospital Maternal and Child Health Coordinator 8 Wongani Mulungu World Relief Malawi – TPCSP Health Education Supervisor 9 Thomas A Nkhonjera World Relief Malawi -TPCSP Health Education Supervisor 10 Paul L Ng’ambi World Relief Malawi -TPCSP Health Education Supervisor 11 Mbasa Msiska World Relief Malawi - TPCSP Project Accounts Assistant 12 Victor Kabaghe World Relief Malawi -TPCSP Project Director 13 Andrew Kasache Banda World Relief Malawi -TPCSP Project Driver 14 Isaac Munthali World Relief Malawi -TPCSP Project Driver 15 Nelson Mwandwanga World Relief Malawi -TPCSP Project Driver 16 Rachel Gondwe World Relief Malawi -TPCSP Project Secretary 17 German Phikani World Relief Malawi -TPCSP Office Assistant 18 Maureen Mtambo World Relief Malawi - TPCSP Health Promoter 19 Alfred Munkhondya World Relief Malawi - TPCSP Health Education Supervisor 20 George Mkandawire World Relief Malawi – Country Office HIV/AIDS Care and Support Coordinator 21 Benjamin Nyondo World Relief Malawi – TPCSP Health Promoter 22 Charles Kapira World Relief Malawi – TPCSP Health Promoter 23 Maseso Mwiba World Relief Malawi - TPCSP Health Promoter Annex D | Page | 1 ANNEX D: INDICATOR TABLE BASELINE April 2005 MIDTERM January 2007 FINAL July 2009 EOP Numerator/ TARGET Denominator Percent Confidence Interval Numerator/ Denominator Percent Confidence Interval Numerator/ Denominator Percent Confidence Interval CARE SEEKING KNOWLEDGE Percentage of caretakers of children 0-23m who know at least 2 childhood illness danger signs for seeking care immediately (RC) 234/329 71.1% 60.8%-81.4% 266/300 88.7% 84.5%-92.0% 287/300 95.7% 93.3%-98.0% 80% HOME MANAGEMENT Percentage of sick children age 0-23m who received increased fluids and continued feeding during an illness in the past two weeks (RC) Increased fluids Increased or continued foods 12/311 3.9% 0.8%-7.0% 32/180 17.8% 12.5%-24.2% 51/226 66/266 112/266 22.6% 29.2% 49.6% 16.7%-28.4% 21.4%-37.0% 42.3%-56.9% 50% IMMUNIZATION Percentage of all children 12-23m fully immunized (BCG, Polio3, DPT3, and Measles) before 24 months as verified by card. * Baseline includes children 12-23m who had a BCG scar and children 9-11m who had a DPT3, Polio3 and measles vaccine 238/346 68.8% 58.8%-78.8% 86/135 63.7% 55.0%-71.8% 113/154 73.4% 65.5%-81.3% 80% Percentage of children age 12-23m who are fully vaccinated (against the five vaccine-preventable diseases) before the first birthday (RC) **Baseline includes children 12-23m who had a BCG scar and children 9-11m who had a DPT3, Polio3 and measles vaccine -- -- -- 76/101 75.2% 65.7%-83.3% 99/138 71.7% 64.1%-79.4% RC Percentage of children age 12-23m who received a measles vaccine per maternal recall NA NA NA 128/135 94.8% 89.6%-97.9% 142/155 91.6% 86.1%-95.5% RC Percentage of children age 9-23m who received a measles vaccine verified by card 191/346 55.2% 45.8%-64.6% 107/167 64.1% 57.6%-70.6% 133/147 74.3% 66.3%-82.3% NA MALARIA Percentage of children age 0-23m who slept under an insecticide-treated net the previous night (RC) **Baseline includes all nets regardless of insecticide treatment 66/161 41.0% 28.5%-53.5% 119/300 39.7% 34.1%-45.4% 131/300 43.7% 36.9%-50.4% 60% Percentage of net ownership -- 44.2% -- 167/300 55.1% 49.8%-61.4% 264/300 88.0% 83.5%-92.5% NA Annex D | Page | 2 Percentage of children 0 -23m who slept under an ITN the previous night, of those who own nets -- -- -- 119/167 71.3% 63.8% -78.0% 131/264 49.6% 42.0% -57.3% NA Percentage of children 0 -23m with suspected malaria in the previous 2 weeks who sought treatment from a trained provider within 24 hours of illness onset -- 17.5% -- 36/87 41.4% 30.9% -52.4% 40/96 41.7% 29.6% -53.7% 50% PNEUMONIA CASE MANAGEMENT Percentage of children 0 -23m with rapid/difficult breathing (suspected pneumonia) in the previous 2 weeks who sought treatment from a trained provider within 24 hours 39/187 20.9% 16.7% -25.1% 9/29 31.0% 15.3 -50.8% 11/25 44.0% 24.9% -63.1% 50% CONTROL OF DIARRHEAL DISEASE Percentage of children 0 -23m with diarrhea in the previous 2 weeks who received ORT (home available fluids, ORS, or breastfeeding ) Including treatment at H F -- 8% -- 28/48 31/48 58.3% 64.6% 43.2% -72.4% 49.5% -77.8% 58/90 74/90 64.4% 82.2% 52.8% -76.1% 73.9% -90.5% 60% Percentage of mothers with children age 0 -23m who report that they wash their hands with soap/ash before food preparation, before feeding children, after defecation, and after attending to a child who has defecated (RC) *Baseline from LRA1 11/284 3.9% 2.3% -5.5% 37/300 12.3% 8.8% -16.6% 58/300 19.3% 15.0% -23.7% 60% Percentage of caretakers with children 0 -23m who reported and washing with soap or ash After Defecation Before eating Before feeding a child After helping a child who has def. Food preparation 4/5 times listed above All 5 times -- -- -- 225/300 191/300 64/300 157/300 130/300 NA 26/300 75.0% 63.7% 21.3% 52.3% 43.3% NA 8.7% 69.7% -79.8% 57.9% -69.1% 16.8% -26.4% 46.5% -58.1% 37.6% -49.1% NA 5.7% -12.4% 257/300 213/300 114/300 205/300 167/300 165/300 45/300 85.7% 71.0% 38.0% 68.3% 55.7% 55.0% 15.0% 81.7% -89.7% 66.0% -89.7% 32.8% -43.2% 61.5% -75.2% 50.8% -60.5% 49.3% -60.7% 11.1% -18.9 % NA NUTRITION Percentage of children 0 -5m who were exclusively breastfed during the last 24 hours (RC) 20/50 40.0% 17.8% -17.8% 85/101 84.2% 75.6% -90.7% 55/69 79.7% 69.8% -89.6% 60% Percentage of children 6 -9m who received breastmilk and complementary foods during the last 24 hours (RC) 78/196 39.8% 28.6% -51.0% 33/41 80.5% 65.1% -91.2% 52/61 85.2% 76.0% -94.5% 70% Percentage of children 6 -11m who received at least 1 dose of Vitamin A and children 12 -23m who received at least 2 doses of Vitamin A in the previous 12 months, as evidenced by card -- -- -- 22/199 11.1% 4.9% -17.3 % 15/231 6.5% 2.0% -11.0% 60% Annex D | Page | 3 Percentage of children 6-23m who received VitA within the past 6 months per maternal recall -- -- -- 189/199 95.0% 91.0%-97.6% 206/231 89.2% 84.8%-93.6% NA Percentage of malnourished children who completed 12 days of Hearth achieve adequate (200g) or catch-up (400g) growth for at least 2 months after Hearth Can not be calculated from the KPC NA NA NA NA NA NA NA NA NA HIV/AIDS PREVENTION Percentage of mothers with children age 0-23m who cite at least two known ways of reducing the risk of HIV infection (RC) 243/360 67.5% 60.7%-74.3% 230/300 76.5% 71.5%-81.3% 257/300 85.7% 81.7%-89.7% 80% Percentage of children age 0-23m whose births were attended by skilled health personnel (RC) 201/364 55.2% 46.0%-64.4% 183/300 61.0% 55.2%-66.6% 218/300 72.7% 64.9%-80.4% 70% RAPID CATCH (NOT INCLUDED AS PROGRAM GOALS) Percentage of children age 0-23m who are underweight (-2SD from the median weight-for￾age, according to the WHO/NCHS reference population) 108/361 29.9% 46.0%-22.5% 31/298 10.4% 7.2%-14.4% 45/297 15.2% 11.2%-19.1% RC Percentage of children age 0-23m who were born at least 24 months after the previous surviving child **Baseline is from 2006 monitoring survey 130/329 39.5% 30.9%-48.1% 116/139 83.5% 76.2%-89.2% 145/166 87.3% 81.3%-93.4% RC Percentage of mothers with children 0-23m who received at least two tetanus toxoid injections before the birth of their youngest child **Includes 2TT verified by card Of those mothers with cards 233/364 233/278 64.0% 83.8% 54.4%-73.6% 77.7%-89.9% 155/300 155/188 53.0% 82.4% 47.2%-58.8% 76.2%-87.6% 178/300 173/187 59.3% 92.5% 51.7%-66.9% 88.2%-96.8% RC OTHER ANALYSIS Percentage of caretakers with children 0-23mo who were visited by a volunteer: Last two weeks Last month Over a month ago Never (MT “Not visited”) -- -- -- 138/300 -- -- 97/300 46.0% -- -- 32.3% 40.3-51.8% -- -- 27.1%-37.9% 52/300 75/300 96/300 77/300 17.3% 25.0% 32.0% 25.7% 13.2%-22.1% 18.7%-31.3% 26.8%-37.6% 20.8%-31.0% NA Percentage of caretakers with children 0-23m who reported treating their water in any way to make it safer to drink -- -- -- 150/300 50.0% 44.2%-55.8% 210/300 70.0% 60.7%-79.3% NA Annex D | Page | 4 Percentage of mothers with children 0-23m who recalled attending antenatal checkups at the Health Center At least once Three or more times Four or more times Never Don’t remember -- -- -- 284/300 226/300 133/300 0 3/300 94.7% 75.3% 44.3% 0% 1.0% 91.5%-96.9% 70.1%-80.1% 38.6%-50.2% 0% 0.2%-2.9% 297/300 241/300 139/300 3/300 0 99.0% 80.3% 46.3% 1.0% 0% 97.5%-100% 75.2%-85.5% 38.8%-53.8% 0%-2.5% -- NA Percentage of mothers with children 0-23m and with antenatal cards who had two or more verified doses of SP during her last pregnancy 162/278 58.2% 50.0%-66.4% 130/188 69.1% 62.0%-75.7% 134/187 71.7% 65.3%-78.1% NA Percentage of mothers with children 0-23m who recalled taking iron supplements during her last pregnancy 310/364 85.2% 80.0%-90.4% 259/300 86.3% 81.9%-90.0% 265/300 88.3% 84.8%-91.9% NA Age Distribution of mothers Under 25 years 25+ years 16-19 20-29 30-39 40-49 -- -- -- -- -- -- 133/300 167/300 30/300 169/300 93/300 8/300 44.3% 55.7% 10.0% 56.3% 31.0% 2.7% 38.4%-50.3% 49.7%-61.6% 6.6%-13.4% 49.8%-62.9% 24.4%-37.6% 1.0%-4.3% NA Age Distribution of children 0-5m 6-9m 6-11m 12-23m -- -- -- -- -- -- 69/300 61/300 77/300 154/300 23.0% 20.3% 25.7% 51.3% 17.4%-27.1% 16.5%-26.1% 21.4%-31.7% 45.9%-57.4% NA Gender of children Female Male -- -- -- -- -- -- 143/300 157/300 47.7% 52.3% 42.4%-52.9% 47.1%-57.6% NA Number of children 0-60m living in the same household One Two Three Four -- -- -- -- -- -- 132/300 155/300 10/300 3/300 44.0% 51.7% 3.3% 1.0% 37.4%-50.6% 45.8%-57.6% 1.5%-5.1% 0-2.1% NA Annex E | Page | 1 ANNEX E: RAW DATA TABLES DEMOGRAPHIC INFORMATION Maternal Age Frequency Percent 25+ years old 167 55.7% Under 25 years old 133 44.3% Total 300 100.0% 16 - 19 30 10.0% 20 - 29 169 56.3% 30 - 39 93 31.0% 40 - 49 8 2.7% Total 300 100.0% Number of children living in the same household Frequency Percent 1 132 44.0% 2 155 51.7% 3 10 3.3% 4 3 1.0% Total 300 100.0% How many biological children are living with you? Frequency Percent 1 134 44.7% 2 158 52.7% 3 8 2.7% Total 300 100.0% Child Age Distribution Frequency Percent >0 - 5 66 22.0% >10 - 11 16 5.3% >12 - 23 155 51.7% >6 - 9 63 21.0% Total 300 100.0% BREASTFEEDING AND COMPLEMENTARY FEEDING Are you breastfeeding (name of child) now? Frequency Percent Annex E | Page | 2 No 20 6.7% Yes 280 93.3% Total 300 100.0% Have you ever breastfeed (name of child)? Frequency Percent No 1 5.0% Yes 19 95.0% Total 20 100.0% How long after birth did you first put (name of child) to the breast? Frequency Percent E. Immediately/within first hour 237 79.3% F. After the first hour after delivery 46 15.4% G. After eight hours after delivery 14 4.7% H. Don’t know 2 0.7% Total 299 100% What types of liquids and foods did (name of child) consume yesterday during the day or at night? Yes Percent No Percent Breastmilk 276 92.0% 24 8.0% Plain water 166 55.3% 134 44.7% Other liquids 96 32.0% 204 68.0% Mashed, pureed, solid, or semi-solid foods 217 72.3% 83 27.7% Other 12 4.0% 288 96.0% ILLNESS RECOGNITION AND CARE SEEKING What are the signs of illness that would indicate your child needs treatment? Yes Percent No Percent Don’t know 0 0% 300 100% Looks unwell or not playing normally 158 52.7% 142 47.3% Not eating, drinking, or breastfeeding 181 60.3% 119 39.7% Lethargic or difficult to wake 51 17.0% 249 83.0% Fever 280 93.3% 20 6.7% Fast or difficult breathing 64 21.3% 236 78.7% Vomits everything 134 44.7% 166 55.3% Convulsions 27 9.0% 273 91.0% Gets worse despite home care 5 1.7% 295 98.3% Looks dehydrated (dry mouth or no tears) 33 11.0% 267 89.0% Other 154 51.3% 146 48.7% Annex E | Page | 3 Did (name of child) experience any of the following in the past two weeks? Yes Percent No Percent Diarrhea 90 30.0% 210 70.0% Diarrhea ≤ 3 times 32 35.6% 58 64.4% Diarrhea > 3 times 52 57.8% 38 42.2% Missing 6 6.7% 84 93.3% Blood in stool 2 0.7% 298 99.3% Cough 175 58.3% 125 41.7% Rapid or difficult breathing 25 8.3% 275 91.7% Fever 90 30.0% 210 70.0% Malaria 40 13.3% 260 86.7% Convulsions 9 3.0% 291 97.0% Other 8 2.7% 292 97.3% None 74 24.7% 226 75.3% When (name of child) was sick, was he/she offered less than usual to drink, about the same amount, or more than usual to drink?” Frequency Percent A. Less than usual 82 36.3% B. Same amount 78 345% C. More than usual 66 29.2% Total 226 100.0% When (name of child) was sick, was he/she offered less than usual to eat, about the same amount, or more than usual to eat? Frequency Percent D. Less than usual 114 50.4% E. Same amount 65 28.8% F. More than usual 47 20.8% Total 226 100.0% DIARRHEA When (name of child) had diarrhea, did you give the child anything? Yes Percent No Percent Nothing 8 8.9% 82 91.1% ORS Sachet 33 36.7% 57 63.3% Sugar-salt Solution 5 5.6% 85 94.4% Cereal based ORT (rice/maize water) 1 1.1% 89 98.9% Breastmilk 36 40.0% 54 60.0% Annex E | Page | 4 Water 10 11.1% 80 88.9% Other available drinks 7 7.8% 83 92.2% Medication for diarrhea 18 20.0% 72 80.0% Take child to the hospital/clinic 32 35.6% 58 64.4% Other 19 21.1% 71 78.9% PNEUMONIA Did you seek treatment for your child with rapid or difficult breathing? Frequency Percent No 7 29.2% Yes 17 70.8% Total 24 100.0% From whom did you seek treatment when (name of child) had difficulty in breathing? Yes Percent No Percent Hospital 1 4.0% 24 96.0% Health Center/Health Post 13 52.0% 12 48.0% Injectionist 0 0% 25 100% Traditional Birth Attendant 1 4.0% 24 96.0% Traditional Healer 0 0% 25 100% Pharmacy/Shop 1 4.0% 24 96.0% Relatives/Friends 0 0% 25 100% Others 1 4.0% 24 96.0% How soon after the difficulty in breathing began did (name of child) receive treatment? Frequency Percent Less than one day (within 24 hours) 11 78.6% After one day (24-28 hours) 2 14.3% Two days or more 1 7.1% Don’t know 0 0% Total 14 100% MALARIA CONTROL When (name of child) had fever, what treatment did you give? Frequency Percent Take the child to the hospital or HC 52 54.7% Home treatment 32 33.7% We the child to decrease fever 1 1.1% Other 10 10.5% Total 95 100.0% How soon after the fever started was (name of child) treated at the hospital or health center? Frequency Percent Annex E | Page | 5 Less than one day (within 24 hours) 40 76.9% After one day (24-28 hours) 7 13.5% Two days or more 5 9.6% Total 52 100.0% Did (name of child) take any drugs for the fever? Frequency Percent No 12 12.6% Yes 83 87.4% Total 95 100.0% What drugs did (name of child) take? Frequency Percent SP/Fansidar 10 15.4% LA 15 23.1% Amodiaquine 1 1.5% Quinine 1 1.5% Other 38 58.5% Total 65 100.0% When did (name of child) take the drugs for the fever? Frequency Percent Less than one day (within 24 hours) 69 83.1% After one day (24-28 hours) 7 8.4% Two days or more 7 8.4% Total 83 100.0% MALARIA PREVENTION Do you have any mosquito nets in your house? IF YES, ASK: Can I see it? Frequency Percent Yes (Net seen hanging) 131 43.7% Yes (Net seen, but not hanging) 121 40.3% Yes (Net NOT seen) 12 4.0% No 36 12.0% Total 300 100.0% Who slept under a mosquito net last night? Yes Percent No Percent Child 143 47.7% 157 52.3% Respondent 143 47.7% 157 52.3% Other(s) 63 21.0% 237 79.0% Did not use a net 120 40.0% 180 60.0% Annex E | Page | 6 Was the mosquito net ever soaked or dipped in a liquid to repel mosquitoes or bugs? Frequency Percent Yes, within the past 6 months 157 59.5% Yes, within the past 1 year 33 12.5% No 58 22.0% Long-lasting net 16 6.1% Total 264 100.0% GROWTH MONITORING Does (name of child) have a growth monitoring card? Frequency Percent Yes, card is seen by interviewer 264 88.3% Yes, but card is missing or lost 29 9.7% No 6 2.0% Total 299 100.0% Has the child been weighed in the last 4 months? Frequency Percent No 15 5.7% Yes 247 93.6% Missing 2 0.8% Total 264 100.0% May I weigh (name of child)? Frequency Percent Missing 1 0.3% No 2 0.7% Yes 297 99.0% Total 300 100.0% IMMUNIZATIONS Did (name of child) ever receive an injection in the leg to prevent measles? Frequency Percent No 13 8.4% Yes 142 91.6% Total 155 100.0% Did (name of child) receive a Vitamin A dose in the last 6 months? Frequency Percent Annex E | Page | 7 No 27 11.5% Yes 207 88.5% Total 234 100.0% Do you have a card or health passbook where (name of child)’s vaccinations are written down? Frequency Percent Yes, card seen by interviewer 265 88.3% Yes, but card is missing or lost 30 10.0% No, never had a card 5 1.7% Total 300 100.0% WATER AND SANITATION Do you treat your water in any way to make it safer for drinking? Frequency Percent No 90 30.0% Yes 210 70.0% Total 300 100.0% If yes, what do you usually do to the water to make it safer to drink? Yes Percent No Percent Let it stand and settle/ sedimentation 4 1.9% 206 98.1% Strain it through cloth 3 1.4% 207 98.6% Boil 22 10.5% 188 89.5% Add bleach/chorine/Waterguard 197 93.8% 13 6.2% Water filter (ceramic, sand, composite) 2 1.0% 208 99.0% Solar disinfection 0 0% 210 100% Other 3 1.4% 207 98.6% Don’t know 0 0% 210 100% When do you wash your hands? Yes Percent No Percent Never 0 0% 300 100% Before food preparation 183 61.0% 117 39.0% Before feeding a child 126 42.0% 174 58.0% After defecation/ visiting the toilet 290 96.7% 10 3.3% After attending a child who has defecated 224 74.7% 76 25.3% Before eating 243 81.0% 57 19.0% Other 53 17.7% 247 82.3% Do you use soap or ash when washing your hands? Frequency Percent No 36 12.0% Annex E | Page | 8 Yes 264 88.0% Total 300 100.0% Ask to see the soap or other substance used for handwashing. Frequency Percent Soap observed 217 72.3% Soap Substitute observed 5 1.7% Soap NOT observed 44 14.7% Missing 34 11.3% Total 300 100.0% MATERNAL AND NEONATAL HEALTH When you were pregnant with (name of child) did you go to the health center for antenatal checkups? If yes, how many times? Frequency Percent Yes, once 18 6.0% Yes, twice 38 12.7% Yes, three times 102 34.0% Yes, four or more times 139 46.3% No 3 1.0% Don’t know 300 100.0% While you were pregnant with (name of child) did you receive an antenatal health card? Frequency Percent Yes, card seen by interviewer 187 62.3% Yes, card is not seen by interviewer 88 29.3% No 23 7.7% Missing 2 0.7% Total 298 100.0% How many tetanus injections did the mother receive according to the card? Frequency Percent None 3 1.6% One 11 5.9% Two 33 17.6% More than two 140 74.9% Total 187 100.0% How many tetanus injections did the mother receive according to the card? Frequency Percent Missing 1 0.5% Annex E | Page | 9 None 4 2.1% One 48 25.7% Two 107 57.2% More than two 27 14.4% Total 187 100.0% Did you take iron tablets daily during your last pregnancy? Frequency Percent No 32 10.8% Yes 265 89.2% Total 297 100.0% Who assisted you with (name of child) delivery? Frequency Percent Doctor 14 4.7% Nurse/Midwife 204 68.0% Traditional birth attendant 29 9.7% Family member or friend 37 12.3% Other 15 5.0% No One 1 0.3% Total 300 100% After (name of child) was born, did any health care provider or traditional birth attendant check on (name of child’s) health? Frequency Percent Don't know 3 1.0% No 21 7.0% Yes 274 91.9% Total 298 100.0% Who checked on (name of child’s) health at that time? Frequency Percent Doctor 23 8.4% Nurse/Midwife 203 74.1% Traditional birth attendant 21 7.7% Family member or friend 14 5.1% Other 13 4.7% No One 0 0% Total 274 100% How long after the birth of (name of child) did the first check take place? Frequency Percent Annex E | Page | 10 Less than one day (within 24 hours) 242 88.3% After one day (24-48 hours) 19 6.9% Two days or more 10 3.6% Don’t know 3 1.1% Total 274 100.0% HIV/AIDS Have you ever heard of HIV/AIDS? Frequency Percent No 3 1.0% Yes 293 99.0% Total 296 100.0% What can a person do to avoid getting AIDS or the virus that causes AIDS? Yes Percent No Percent Abstain from sex 198 67.6% 95 32.4% Use condoms 199 67.9% 94 32.1% Limit sex to one partner/Stay faithful to one partner 183 62.5% 110 37.5% HIV Testing and Counseling/VCT 60 20.0% 240 80.0% Limit number of sexual partners 20 6.7% 280 93.3% Avoid sex with prostitutes 28 9.3% 272 90.7% Avoid sex with persons who have many partners 7 2.4% 286 97.6% Avoid intercourse with persons of the same sex 2 0.7% 298 99.3% Avoid sex with persons who inject drugs intravenously 0 0% 300 100% Avoid blood transfusions 17 5.8% 276 94.2% Avoid injections 63 21.0% 237 79.0% Avoid sharing razors, blades 161 53.7% 139 46.3% Avoid kissing 5 1.7% 295 98.3% Avoid mosquito bites 1 0.3% 299 99.7% Seek protection from traditional healer 0 0% 300 100% Nothing 1 0.3% 299 99.7% Other 23 7.7% 277 92.3% Doesn’t know 1 0.3% 292 99.7% SUSTAINABILITY When did World Relief/Tube Poke volunteer visit you? Frequency Percent Last week 30 10.0% Past two weeks 22 7.3% Past three weeks 19 6.3% Past four weeks 56 18.7% Never 77 25.7% Other 96 32.0% Annex E | Page | 11 Total 300 100.0% Annex E | Page | 12 ANNEX F: TRAINING SCHEDULE AND REQUIRED RESOURCES The training schedule for supervisors and interviewers DATE ACTIVITY RESPONSIBLE PERSONS 22 – 30 June 2009 Planning for the survey activity with Country office Project Management; Director of Programs; Country office Finance and Administration 01 July 2009 Health Promoters, and supervisors from Nthalire, Wenya and Misuku arrive at the project office Rachel Gondwe; Mbasa Msiska and German Phikani 02 -03 July 2009 Survey Team meet to plan and review the survey tool Project Directors; Survey Team Members 04 July 2009 Survey Team split into three and go for field pre-testing Project Directors 5 July 2009 Survey teams leave for data collection. ALL 6-11 July 2009  Data Collection  Setting up data entry equipment  Data entry also commences Survey teams; Data entry clerks; Project Director (supervising the process) 13 July 2009 Review meeting to share field experiences and very preliminary results Project Director, Survey Teams. 14-15 July 2009 Sharing the KPC Survey Files with MCH Specialist Project Director 16-30 July 2009 In-depth data analysis and report writing MCH Specialist; Project Directors Project resource requirements of the survey, namely monetary costs and amount of staff time devoted to KPC activities The total budget for the survey was MK 800.000 (US$6,430) Monetary costs include: stationary (printing survey questionnaires, purchasing note pads and pens) and three vehicles and all motor cycles which were used for community mobilization. Staff time requirements: During planning, the day commenced at 08.00 hours and ended at or after 17.30 hours while in the field the day was much longer up to 17.45 in most cases. This was so because of the difficulties in finding children 0-23 months of age. Annex E | Page | 13