African Medical and Research Foundation Final Evaluation Report Busia Child Survival Project (BCSP) Busia and Samia Districts, Kenya October 2005 – September 2010 USAID/HIDN/CSHGP Cooperative Agreement Number: GHS-A-00-05-00009-00 Date of Submission: December 15, 2010 Final Evaluation Consultant: Judy Lewis Contributors to Final Evaluation Report: Cudjoe Bennett, MPH (Technical Advisor – AMREF in USA) Jennifer Weiss, MPH (Technical Advisor￾AMREF in USA) David Wamalwa (Project Manager – BCSP) Gilbert Wangalwa (M&E Officer – BCSP) Judith Ngeresa (BCC Officer – BCSP) Everlyne Omunjalu (Training Officer – BCSP) Bibianne Situma (Administrative Assistant – BCSP) Bill Yaggy (Director of Institutional Giving – AMREF in USA) Acknowledgements Thank you to the AMREF staff in Busia, Nairobi and New York. Everyone was incredibly hospitable, supportive and well organized. The professional manner and efficient work of the team made the Final Evaluation a well organized, successful and participatory process. It was an excellent learning opportunity for everyone. Kenya and Western Province faced many challenges during the course of this project, but the Busia Child Survival Project (BCSP) persevered and was able to maximize opportunities for improving the health of children, mothers and communities. This evaluation was organized by the Busia AMREF Team in close collaboration with the DHMT. AMREF took the additional initiative of hiring Kenyan research assistants to assure a more objective and consistent level of field interviews. However, the BCSP and DHMT staff were intimately involved in planning, implementation, analysis and recommendations of the FE. Everyone was committed and enthusiastic about learning from BCSP. All members of the team are acknowledged in the annex. Thank you to the many mothers, leaders, community health workers and DHMT staff who welcomed us and responded openly about their opinions and expectations. I would also like to thank the AMREF staff for their understanding of delays in report completion as I dealt with major health issues with both of my parents this past fall. Except when specified otherwise, opinions expressed here are those of the consultant, informed by the participatory process of evaluation. Judy Lewis Evaluation Consultant December 15, 2010 Acronyms AIDS Acquired Immune Deficiency Syndrome AMPATH Academic Model Providing Access to Healthcare APHIA II AIDS, Population, and Health Integrated Assistance Phase II AMREF African Medical and Research Foundation ANC Antenatal Care ART Anti Retroviral Treatment ARV Anti Retroviral Drugs BCC Behavior Change and Communication BCSP Busia Child Survival Project CBHMIS Community Based Health Management Information System CHC Community Health Committee CHEW Community Health Extension Worker CHW Community Health Worker C-IMCI Community Integrated Management of Childhood Illnesses COE Center of Excellence CORP Community’s Own Resource Person CS Community Strategy CSP Child Survival Project CSSA Child Survival Sustainability Assessment CtC Child to Child CU Community Unit CU5 Children Under 5 (years of age) DHMT District Health Management Team DIP Detailed Implementation Plan EBF Exclusive Breast Feeding EmOC Emergency Obstetrical Care EOC Essential Obstetrical Care FANC Focused Antenatal Care FE Final Evaluation GLUK Great Lakes University of Kisumu HFA Health Facilities Assessment HIV Human Immune Deficiency Virus IEC Information, Education, Communication IMCI Integrated Management of Childhood Illnesses IPT Intermittent Preventive Treatment (malaria) KCO Kenya Country Office (AMREF) KPC Knowledge, Practices and Coverage LLIN Long Lasting Insecticide-treated Net LQAS Lot Quality Assurance M&E Monitoring and Evaluation MAMAN Minimum Activities for Mothers and Newborns MCH Maternal and Child Health MNC Maternal Newborn Care MOH Ministry of Health MOPHS Ministry of Public Health and Sanitation MSF Medicins San Frontieres MTE Mid-Term Evaluation MtM Mother to mother NGO Non-Governmental Organization PDQ Partnership Defined Quality PHC Primary Health Care PMTCT Prevention of Mother to Child Transmission (HIV) PPC Post Partum Care PSI Population Services International PVO Private Voluntary Organization SBA Skilled Birth Attendant/Attendance SMNA Safe Motherhood Needs Assessment SP Sulfadoxine Pyrimethamine STI Sexually Transmitted Infection TBA Traditional Birth Attendant TOT Training of Trainers UNICEF United Nations Childrens Fund USAID United States Agency for International Development VCT Voluntary Counseling and Testing (HIV) WHO World Health Organization 5x5x5 Five messages to five people to five other people/households Table of Contents A. The Executive Summary ............................................................................................................. 1 B. Overview of the Project: .............................................................................................................. 4 C. Data Quality: Strengths and Limitations .................................................................................... 8 D. Presentation of Project Results .................................................................................................. 9 E. Discussion of the Results ......................................................................................................... 13 F. Discussion of Potential for Sustained Outcomes, Contribution to Scale, Equity, Community Health Worker Models, and Global Learning .................................................................. 24 G. Conclusions and Recommendations ....................................................................................... 30 Annexes Annex 1: Results Highlight Annex 2. List of Publications Annex 3: Project Management Evaluation Annex 4: Workplan Table and Project Results Framework Annex 5. Rapid CATCH Table Annex 6: Final KPC Report Annex 7: CHW Training Matrix Annex 8: Evaluation Team Members and Titles Annex 9: Evaluation Assessment Methodology Annex 10: List of Persons Interviewed and Contacted During Final Evaluation Annex 11: Special Reports Annex 12: Project Data Form Annex 13: Grantee Plans to Address Final Evaluation Findings A. Executive Summary 1. Project goal and objectives. The goal of the program was a sustained reduction in child and maternal mortality in Funyula and Butula divisions, Busia District. The project objectives focused on Maternal Newborn Care (40%), Malaria (40%) and HIV/AIDS (20%). The specific objectives were: Maternal and Newborn Care 1.1.1 Increased proportion of pregnant women who attend antenatal clinic at least four times during pregnancy and postnatal clinic at least once. 1.1.2 Increased proportion of women attended by a skilled health professional during delivery. 1.1.3 Increased proportion of complicated deliveries referred to and managed at health facilities. 1.1.4 Improved quality of EmOC at health facilities. Malaria 2.1.1 Improved knowledge and practice of malaria prevention and treatment at household and community level. 2.1.2 Increased proportion of pregnant women and CU5 who sleep under insecticide-treated bed nets. 2.1.3 Improved case management of severe malaria among CU5 at health facilities. 2.1.4 Increased percentage of pregnant women who receive Intermittent Preventive Therapy (IPT). HIV/AIDS 3.1.1 Increased knowledge and understanding of PMTCT and ART among women of reproductive age (15-49 years). 3.1.2 Increased access to HIV counseling and testing among pregnant women at ANC. 3.1.3 Increased number of HIV+ pregnant women and newborns who receive PMTCT and AIDS care and treatment. 3.1.4 Improved feeding practices among caretakers of infants 0-5 months of age. 2. Main accomplishments of the project. Key Findings/Results The BCSP achieved most of its objectives (see Table 1 below for details). Among the most significant changes were more than doubling the proportion of women delivering at facilities and receiving postpartum care. Other results were increased use of LLINs, IPT, and HIV testing. At the community level the training of 910 CHWs and creation of CHCs for each local governmental area were also major accomplishments. The DHMT and health facility staff also had increased capacity through training and supervision. The BCSP was a model for the Kenya MOPHS’ Community Strategy and helped develop many of the training materials for the government, including MNC training. Community mobilization and engagement was another major accomplishment. 1 Table 1: Summary of Major Project Accomplishments by Intervention Area Maternal Newborn Care (40%) by Project Objectives with Successful Outcomes #1: Increased proportion of women who attend antenatal clinic at least four times and postnatal clinic at least once #2: Increased proportion of women who delivered under supervision of a skilled health attendant #3: Increased proportion of women who deliver at a health facility Project Inputs Activities Outputs Outcomes IEC and BCC materials Trainers Facility improvement, PDQ Development of MNC training manual in collaboration with Kenya MOPHS and USAID CHEW Training Formation of MtM Groups Training (together with staff at district health system) Monitoring and supervision Manual will be used by MOPHS for implementation throughout the country 14 CHEWs trained in Community MNC 51 mothers’ groups formed and educated 16 Health facilities staff trained in FANC and EOC 850 CHWs monitored and supervised by CHEWs, DHMT and BSCP staff Increased % of mothers of children 0-23 months who attend ANC at least four times during most recent pregnancy from 32% to 49% Increase % of mothers of infants 0-5 months who attend postnatal care within two days of delivery from 23% to 58% Increased % of children 0-23 months whose delivery was attended by a skilled health professional from 26% to 57% Increased % of mothers of children 0-23 months who deliver at health facility from 20% to 53% Malaria (40%) by Project Objectives with Successful Outcomes #5: Improved knowledge and practice of malaria prevention and treatment at household and community level #6: Increased proportion of WRA and CU5 who sleep under insecticide-treated nets #8: Increased proportion of pregnant women receiving IPT Project Inputs Activities Outputs Outcomes IEC and BCC materials Trainers LLINs CHWs training to educate about malaria and work with MtM, Child to Child, Child to Parent, 5x5x5, and Positive Deviance LLINs distributed Health Facilities staff training 51 mothers’ groups formed and educated 70 (4200) Child/child, school health clubs 10,000 nets to households 24 Health Facilities staff trained in Increased % of children 0-23 months taken to HF or Community Health Worker with 24 hours after onset of fever from 7% to 38% Increased % of households with at least one ITN from 77% to 90% Increased % of mothers of children 0-23 months who 2 3 Monitoring and Supervision IMCI 850 CHWs monitored and supervised by CHEWs, DHMT and BSCP staff slept under ITNs the previous night from 65% to 93% Increased % of children 0-23 months who slept under ITNs the previous night from 70% to 93% Increased % of mothers of children 0-23 months who received at least 2 doses of SP for IPT during ANC from 21% to 57% HIV/AIDS (20%) by Project Objectives with Successful Outcomes #9 Increased access to HIV counseling and testing among pregnant women at ANC #10 Increased knowledge of PMTCT and ART among WRA #12 Improved feeding practices among caretakers of infants 0-5 months of age Project Inputs Activities Outputs Outcomes IEC and BCC materials Trainers Community education Health Facilities staff training Monitoring and Supervision 23 Health Facilities staff trained in PMTCT 850 CHWs monitored and supervised by CHEWs, DHMT and BSCP staff Increased % of mothers tested and counseled for HIV at ANC during their last pregnancy from 53% to 85% Increased % of mothers of children 0-23 months who know their HIV status from 41% to 81% Increased % of mothers of children 0-23 months who cite at least two ways of PMTCT from 23% to 84% Increased % of children 0-5 months of age who were exclusively breast fed in the last 24 hours from 11% to 52% Main conclusions and recommendations. The BCSP is an excellent example of NGO and government collaboration to improve the health of mothers and newborns. Many of its strategies can be adapted for other low resource settings. Lessons learned from community mobilization, partnering for research and BCC for hard to change behaviors should be shared with the child survival community. This project demonstrated the importance of working with the continuum of care, starting with communities and working closely with facilities. B. Overview of the Project: The African Medical and Research Foundation (AMREF) in collaboration with the Ministry of Public Health and Sanitation (MOPHS) with a grant from USAID implemented a five year (October 2005 – September 2010) Child Survival Project in Butula and Samia districts in Western Kenya. Project intervention areas, goal and objectives: The project intervention areas were maternal and newborn care (40%), malaria (40%) and HIV/AIDS (20%). The goal was to contribute to “sustained reduction of child and maternal morbidity and mortality.” The specific objectives were: 1) increased access to and utilization of maternal and newborn care services; 2) reduced malaria incidence among pregnant women and children below five years, and 3.) reduced HIV infections among the newborns. See Project Results Framework in Annex 4. The project sites and population: The Busia Child Survival Project (BCSP) was implemented in Butula and Funyula Divisions in Busia and Samia Districts respectively. Both constituencies were formerly in Busia District, but government reorganization during the course of the project created more districts. As a result of this, Funyula was placed into the new Samia District. The two divisions cover an area of 1,261 sq. kilometers with a total population of 215,384 persons. The beneficiary population was 49,858 women of reproductive age (WRA) and 31,644 children under five (CU5). The area is located in the Western Province in the southwestern part of Kenya. It borders Uganda to the west and Lake Victoria to the south. See Figure 1. (next page) for map of project location and sites. Gaps in Maternal and child health indicators that necessitated the implementation of BCSP: At the time of the project’s inception in 2005, the maternal and child health indicators in Butula and Funyula were very poor overall and in comparison to national indicators (source KDHS, 2003). Indicator BCSP Area Kenya Maternal Mortality Rate 680/100,000 414/100,000 Neonatal Mortality Rate 40/1000 30/1000 Infant Mortality Rate 80/1000 70/1000 Child (<5) Mortality Rate 144/1000 115/1000 Project approaches/strategies: The BCSP Project adopted the Ministry of Public Health and Sanitation (MOPHS) Community Strategy for program implementation. The Community Strategy directs support to promote health and to prevent disease in the communities. The strategy proposes empowerment of communities to adopt healthy life styles and strengthened linkages with the formal health sector through community health workers which are supervised and supported by community health extension workers. The MOPHS and MOH categorize the health care system into six levels: • Level 1: Community: Village/Household/Family/Individuals • Level 2: Dispensaries/Clinics • Level 3: Health Centers, Maternity and Nursing Home • Level 4: Primary Hospital • Level 5: Secondary Hospital • Level 6: Tertiary Hospital The BCSP focused on Levels 1,2, 3 and 4, with a strong emphasis on Level 1. 4 Figure 1. Map of Busia and Samia Districts 5 The key drivers of health care services at Level 1 are the Community Health Workers (CHWs), Community Health Extension Workers (CHEWs) and Community Health Committees (CHCs). CHWs are volunteers who are supervised by CHEWs. CHEWs are MOPHS employees who are trained as Public Health Officers/Technicians or Public Health Nurses. The CHCs are composed of community leaders and staffed by CHEWs. The Community Strategy is based on Community Units (CUs) of 5000, each with a CHC. It specifies a ratio of 1 CHW for 20 households and 1 CHEW supervising 25 CHWs. BCSP used the following specific approaches: Capacity Building, Behavior Change and Communication (BCC), Quality Control and Assurance, Health Systems Research and Advocacy. Each are described below: 1. Capacity Building: • The project trained 14 CHEWs in the Community Strategy (CS), Maternal Newborn Care (MNC), Community Based Health Management Information System (CBHMIS), Partnership Defined Quality (PDQ) and Strategic Management and Leadership. • A total of 23 facility based health workers were trained in Integrated Management of Child Illnesses (IMCI), Prevention of Maternal to Child Transmission of HIV (PMTCT), Essential Obstetric Care (EOC) and Focused Antenatal Care (FANC). • Nine hundred ten (910) CHWs were selected and trained in CS, Primary Health Care (PHC), MNC and CBHMIS. BSCP provided bicycles to CHW leaders to facilitate their ability to work with CHWs in their area. CHW leaders were master trainers and helped support the training and supervisory structures. • The project also trained the Busia and Samia District Health Management Teams (DHMT) in Leadership Management and Community Strategy, and supported the formation and training of 50 Community Health Committees. • To facilitate provision of quality health services, the BCSP provided renovation, equipment and training to create MNC Centers of Excellence (COE) at four health facilities which required renovation, equipment and training. These were two health centers that conduct deliveries: Bumala “B” in Butula and Nangina Dispensary in Samia; and two hospitals: Khunyangu in Butula and Sio￾Port in Samia. • ITN provision was another intervention to improve the capacity of beneficiaries to prevent malaria. A total of 10,000 ITNs were distributed during the project. 2. Quality Control and Assurance: To enhance the quality of MNC services in all health facilities within the project areas, the project supported the DHMTs in conducting facilitative supervision using a checklist that was developed by the Provincial Health Management Team under AMREF guidance. 3. Health Systems Research: To document best practices and lessons learned, BCSP collaborated with two universities – The Great Lakes University of Kisumu and Karolinska Institute in Sweden – to conduct studies that included: 1) Assessment of the most viable community transportation systems for mothers; 2) Assessment of factors contributing to motivation and retention of CHWs; 3) Assessment of factors contributing to male partner involvement in promoting skilled deliveries; and, 4) Assessment of 6 knowledge, attitudes and practices regarding PMTCT. The project also conducted operations research to assess the effectiveness of the MOPHS community strategy in enhancing the uptake of maternal and newborn care services: “Community-Based Maternal and Newborn Care: Effectiveness of Kenya’s community strategy for maternal and newborn health care in Busia and Samia counties, Kenya.” November 2010 4. Behavior Change and Communication (BCC): Three BCC approaches were used to reach out to the targeted groups. These were: 5*5*5, a process in which five messages are selected and given to five different households or people in a village. Each of the initial recipients of the five messages passes the same messages to five other households or people in the neighborhood. This process goes on until the whole community or geographical area is covered. Mother to Mother Support Groups were created by identifying pregnant women who are attending ANC in a specific area. These women served as positive deviance role models for other women who were not attending ANC. These mothers then met on a regular basis and invited women who were not attending ANC to come to their sessions. These sessions provided information on all matters pertaining to maternal and child health, including postpartum care and Exclusive Breast Feeding (EBF). Child to Child is school based health club where pupils are selected and brought together to form a club that discusses matters related to health and to promote the dissemination of health information to all students in the school as well as Child to Parent. School club members are involved in community sensitization starting with their fellow children, then to parents and the community as a whole. Some of the key messages were: “Always seek prompt treatment for fevers within 24 hours,” “Malaria is only caused by a mosquito bite of a female anopheles mosquito,” “Always adhere to treatment plans and finish prescribed drugs.” 5. Advocacy: AMREF worked with the MOPHS to accept and replicate the training manual for CHWs on MNC. The MOPHS utilized the draft training manual developed by AMREF to form a working group consisting of representatives of UNICEF, USAID Kenya Mission, and WHO to review and revise the materials. The revised manual was accepted by the MOPHS and used nationally to train CHWs on MNC. AMREF also intends to use the results of operations research and the end of project evaluation findings to lobby the MOPHS to adopt the Community Strategy in implementing these innovations. 6. Partnerships. BCSP key partners ranged from the national to the community levels. BCSP worked with the MOPHS to pilot the CS, develop the curriculum for MNH for CHW, and serve on the CS Taskforce). BCSP partnered with USAID Kenya Country Mission in terms of identifying priorities, learning from other grantees, and advocating at the policy level; Ministry of Education at the district level to create school based programs for health education; Provincial Administration leadership to implement services at the community level; academic institutions to do operations research; other NGOs (MSF Spain, AMPATH, APHIA II) to coordinate delivery of services at the local level; and local communities to create ongoing structures such as CHCs and CHWs for delivering and monitoring services. (Refer to Annex 4) 7 C. Data Quality: Strengths and Limitations BCSP collected an enormous amount of data in this project, using a good mix of quantitative and qualitative techniques. In addition to baseline, midterm and end of project KPCs, there was good tracking of program activities through process indicators, and collection of qualitative data through small studies conducted in collaboration with local and international universities and operations research conducted by the project. This data was used to improve program operations and was presented to all partners in the project in annual feedback meetings. One example of how data was used to make project management decisions came from the baseline data collection and DIP. Data indicated that a major reason that women did not use facilities was how they were treated by the staff. The project delayed implementation of increasing referrals from the community to facilities until this issue could be addressed through health facility assessment and staff training. There was strong collaboration with the DHMT in terms of M&E, which included implementation of the MTE and FE as well as joint M&E training for CHEWS and Health Facility staff and Community Based Health Information Systems (CBHMIS) for CHEWS, Health Facility In-Charges, and CHWs. However, the training did not seem to translate into exchange of information and utilization of DHMT data as part of program evaluation. Even at the Final Evaluation, there were no DHMT data available to actually document increases in utilization of Health Facilities for antenatal care, deliveries and postnatal care, or for malaria prevention, diagnosis and treatment, or HIV/AIDS. This would have been very useful data to complement the results of the KPC, which indicated large increases in utilization. This data could also provide information for MOPHS for planning facility expansion and human resources for health. This means that most of the evidence for the impact of the BCSP was based on data collected by the AMREF project staff. As a result of the training, information was collected from the CHWs and from them to the CHEWs, who presented some of it back to the CHCs. But most of the information was transferred upward through the facilities to the District and Provincial levels of the MOPHS. This means that the information was not available in a timely or useful way to local planning groups, which should be a critical element in the Community Strategy. This suggests that while skills of DHMT HIS staff may have improved over the course of the BCSP, communication and feedback did not. It is therefore difficult to assess the specific actions which CHWs, CHEWs, CHCs and Health Facilities took which resulted in the generally positive results in the program. Some decisions about which data to collect at the beginning of the project did limit analysis of progress. For example, even though MNC was a big component of the program, the Health Facilities Assessment (HFA) focused only on IMCI, so there was no data on MNC laboratory testing or EmOC supplies. The only part of the HFA that was used for assessing accomplishment of project objectives was malaria treatment. Another example is that the KPC did not collect data about knowledge of danger signs of pregnancy, delivery and postpartum. This is important since so many women still deliver at home and/or return home very shortly after delivery. By contrast, the KPC did assess knowledge for the components of malaria and HIV/AIDS. Other areas of data limitations derive from lack of documentation of where and when the post-partum visit took place. It is not clear if the large increase in postpartum visits within 2 days is primarily due to facility deliveries in which the postpartum visit occurred before the woman left the hospital (which is sometimes shortly after delivery). In addition, no data was collected from or about TBAs, who remain 8 9 active in these communities. While there were reports in some facilities of TBAs referring for delivery, it was not possible to document how or why this occurred in some places and not others. D. Presentation of Project Results Table 2 provides a listing of all the objectives, indicators, data sources, baseline measures, and targets. Table 2. Presentation of Progress Toward Objectives Objective/ Result Indicators (by technical intervention or cross- cutting) Source/ Measurement Method Baseline Value Final Value EOP Target Increased proportion of women who attend antenatal clinic at least 4 times and postnatal clinic at least once % of mothers of children 0 – 23 months who know 2 ways (ITN & IPT) to prevent malaria. KPC 17% 14% 62% % of mothers of children 0-23 months who attend ANC at least four times during most recent pregnancy KPC 32% 49% 50% % of mothers of infants 0-5 months who attend postnatal care within two days of delivery KPC 23% 58% 40% Increased proportion of women who delivered under supervision of a skilled health professional % of children 0-23 months whose delivery was attended by a skilled health professional (nurses with midwifery training, doctors, midwives) KPC 26% 56% 40% Increased proportion of women who deliver at a health facility % of mothers of children 0-23 months who deliver at health facility KPC 20% 53% 35% Increased access to basic Emergency Obstetric Care at health facilities No. of health facilities providing basic EmOC (administer antibiotics, oxytocic drugs, and anticonvulsants; perform manual removal of placenta, assisted vaginal delivery, and manual vacuum aspiration) SMNA 0 Not done 6 Improved knowledge and practice of malaria prevention and treatment at household and community level % of children 0-23 months taken to HF or Community Health Worker within 24 hours after onset of fever KPC 7% 38% 60% % of shopkeepers correctly dispensing anti-malarial drugs according to MOH protocol Shopkeepers Survey Shopkeeper records 15% Not done 80% 10 Table 2. Presentation of Progress Toward Objectives Objective/ Result Indicators (by technical intervention or cross- cutting) Source/ Measurement Method Baseline Value Final Value EOP Target Increased proportion of women and children under five who sleep under insecticide￾treated nets % of households with at least one ITN KPC 77% 95% 90% % of mothers of children 0-23 months who slept under ITNs the previous night KPC 65% 93% 80% % of children 0-23 months who slept under ITNs the previous night KPC 70% 93% 80% Improved case management of malaria/fever among CU5 at health facilities % of HF staff who assess, classify and treat malaria/fever according to MoH protocols R-HFA 0%* 0%* 40% Increased proportion of pregnant women receiving IPT % of mothers of children 0-23 months who received at least 2 doses of SP for IPT during ANC. KPC 21% 57% 60% Increased knowledge and understanding of PMTCT and ART among women of reproductive age (15-49 years) % of mothers of children 0 – 23 months who cite at least two ways of PMTCT KPC 23% 84% 59% % of mothers of children 0-23 months who know that risk of MTCT can be reduced by ART KPC 33% 24% 50% Increased access to HIV counseling and testing among pregnant women at ANC. % of mothers of children 0-23 months counseled and tested for HIV at ANC during their most recent pregnancy KPC 53% 85% 70% % of mothers of children 0-23 months who know their HIV status KPC 41% 81% 60% 11 12 Table 2. Presentation of Progress Toward Objectives Objective/ Result Indicators (by technical intervention or cross- cutting) Source/ Measurement Method Baseline Value Final Value EOP Target Improved feeding practices among caretakers of children 0-5 months % of children age 0-5 months who were exclusively breastfed during the last 24 hours KPC 11% 52% 40% Sustainability Indicators Dimension I: Health status and health services % of children 0 – 23 mothers whose delivery was attended by a skilled health professional KPC 26% 57% 40% % of children 0 – 5 months who were exclusively breastfed within last 24 hours KPC 11% 52% 40% % of facilities visited (using facilitative supervision checklists) at least 4 times a year Facilitative Supervision reports 0 100% 80% Dimension II: organizational capacity and viability Organizational capacity of AMREF and DHMT to steer the project towards achievement of its goal Organizational Capacity Assessment1 * * * Dimension III: Community competence and capacity % of CHCs who are active (by the time of survey, had had at least one meeting) ** CHW and CHC audit 0 35% 80% % of CHC who had submitted at least one monthly report. Facility records 0 35% 80% *This indicator was not measured ** Because the CHCs were formed and trained not long before the final evaluation, only 35% had held one confirmed meeting and submitted a report, so this indicator probably understates the level of activity of the CHCs. 1 See attached organizational capacity assessment report Annex 11 E. Discussion of the Results The BCSP met or exceeded almost all its project objectives. Several critical MNC objectives doubled (4 antenatal visits, skilled delivery, delivery at facility and post-partum visit within in two days). The most dramatic increases in objectives were seen in malaria, taking child to health facility within 24 hours of fever increase 5-fold and maternal coverage of IPT tripled. In terms of HIV/AIDS, there were also major improvements in testing during pregnancy, maternal awareness of status and EBF. These results are discussed by intervention and cross cutting area in the sections below, addressing successful objectives and those where the objective was not achieved. Qualitative data from the FE will be incorporated in each discussion. The sustainability objectives are addressed in section F.1. Maternal Newborn Objectives MNC objectives were to increase health facility utilization and skilled birth attendance and these are directly related to each other. The increases in ANC, PPC, SBA and facility deliveries were all of a high magnitude. Based on the FE, these increases were attributed to improvements in the quality of care provided at the facilities (both service availability and attitudes) and treatment by staff. CHW home visits, pregnancy registration and MtM support groups were also cited as sources of information and encouragement of ANC and facility delivery. Improvement of facilities, especially the creation of the four MNH Centers of Excellence clearly improved women and family satisfaction with facility delivery and increased positive perceptions of the importance of skilled birth attendance and facility births. Compared to most maternity wards in developing countries, the renovated hospitals provided privacy and more comfort as well as responsive and respectful staff. Nurses felt they could provide better care as a result of the upgrading well and this gave them higher job satisfaction. Mother and Newborn Premature Newborn in Maternity Unit 13 There have definitely been changes in community attitudes toward facility delivery and ANC. This was confirmed by the FE. One Assistant Chief was convinced of the importance of having his wife deliver in the hospital even though all previous births had been at home, and one of the reasons he made this decision was to be a role model for others in his community. He was also actively advocating for the upgrading of the local dispensary to be able to provide deliveries with a nurse. However, major barriers remain. These are primarily distance to facilities, costs and transportation (especially lack of emergency transport) which are inter-related. In terms of facility delivery costs, community members (women and men) talked about the high costs of facility delivery during the FE. These costs were further exacerbated by hospital policies which keep women and newborns in the hospital until their families pay for the delivery. They are charged for each additional day of hospitalization. This further increases the charges and imposes financial hardship on vulnerable families. In general, community leaders and even some DHMT staff did not know the actual cost of delivery, and when they gave an estimate, it was usually below the actual charges. This means they cannot adequately address this important community barrier. Distance to facilities is another problem. This includes facilities that provide skilled delivery which are often located far from some of the villages. It also includes health centers with functional laboratories and supplies. Both community members and health facility staff cited concerns about women who come for ANC at facilities without laboratory facilities. These women have to be referred to a more distant site for HIV testing and other laboratory tests—they often do not complete this referral and have inadequate ANC as a result. Transportation to facilities is a major problem, both for ANC and facility delivery as well as for transport in obstetrical emergencies. According to one of the research projects the BCSP conducted in collaboration with a graduate student at Great Lakes University of Kisumu (GLUK) in 2008, the majority of women who delivered in hospitals took bicycles (45%) or walked (22%), 22% took matatus and the rest used other means (taxi, private car, motorcycle, government or mission hospital ambulance). These results were consistent with findings of previous studies done by WHO, 2005 and MoH, 2002. The decision to spend money for this transport was rarely made by the woman alone (7%), the husband made the decision in 38% of the cases and some combination of husband, other family members and woman in the rest. Decisions to spend money for transport and facility delivery must be addressed in the family and community context if changes are to occur. The issue of emergency transportation is an even greater problem because of the lack of resources at the hospitals. There is little availability of ambulance transport (at one of the FE site visits, the hospital had an ambulance on the grounds but it was not working). Patients and families have to pay for ambulance transport which is quite expensive. In an emergency women have to be transferred a great distance for a c-section, even if they are having a facility delivery. Women with home deliveries often cannot afford any mode of transportation. The BCSP had planned to have income generating activities to help support community funds for emergency transport, but given the many other demands of this project this did not occur. A further complication is that during the rainy season, many roads are impassable. The pictures below illustrate two of these problems. 14 Road conditions after heavy rain during FE Non-Functioning Ambulance If the issues of cost, distance and transport cannot be addressed, the improvements in utilization of facility based MNC may not continue to improve or be sustained at the level achieved by the BCSP. Another resource for MNC is the TBAs who live in communities and are available 24/7. In addition to the group and individual interviews which were conducted for all intervention areas, TBAs were interviewed about maternal newborn care. Two groups were identified, one from an area where facility deliveries had increased and TBAs brought women to the facility to deliver and the other from an area with lower facility deliveries and TBAs who still delivered babies at home. The CHEWs identified these groups as “TBAs who refer” and “TBAs who don’t refer.” A group interview was conducted with 4 “TBAs who refer” at the health facility and another interview was conducted with 3 TBAs “who don’t refer” at a home near the health clinic. These interviews provided insight into the perspectives of women who are important community resources. In this part of Kenya, the role of TBAs is to provide comfort to women during pregnancy by giving massage and other support and to provide delivery and immediate postnatal support and monitoring to mothers and newborns. Based on the interviews, all TBAs refer women to hospitals when there is a complication or emergency, the difference is that the “TBAs who refer” do so for all the women they follow during pregnancy. They sometimes accompany them to antenatal care and often accompany them to deliver. Most in both groups reported being well treated by health facility staff. Sometimes they attend the delivery but mostly they are there during labor and postpartum. The “TBAs who refer” reported there were other TBAs in their area who continued to do deliveries. In regard to the cost issue, TBAs charge a lower amount for delivering babies, are available at a short distance, come to the home and can be paid on a barter basis if the family has no money. Women said they trusted TBAs because they are members of their community and they have personal relationships with them. In the original proposal, TBAs were to be trained as health educators and promoters. This project activity was eliminated because of the lack of funds and also because the MOPHS decided that TBAs were not a priority. There was no systematic effort to include TBAs in the community mobilization effort or to work with them to find mutually agreed roles for the benefit of women and newborns. This is something that should be considered in the continuation of the project by the community and DHMT. 15 TBAs This group refers all pregnant women they see to the hospital It is notable that in spite of all the obstacles, there was community receptivity that resulted in significant changes in where and how women deliver. This can be attributed to effective BCC, community mobilization and CHWs. The MNH training for facilities, CHEWs and CHWs was very important to this effort. The focus on improving the quality of care at the facilities PRIOR to initiating community mobilization and referrals was identified as a best practice (see Annex 1). BCSP was the primary NGO promoting MNC in the area, so the improvements in outcome indicators are due in large part to its efforts. Malaria Malaria objectives were related to knowledge, practices and health care utilization. The fact that knowledge of malaria prevention did not increase to meet the objective, while indicators of coverage increased demonstrates that knowledge is not always necessary to change behavior (although it often is a prerequisite). In this case, the criteria for knowledge were two specific indicators ITN and IPTs, and this resulted in the low knowledge level. Most women knew about ITNs (almost 85%) but not many were aware of IPT as a preventive measure. They were receiving this as part of routine ANC care but the reason for taking the medication to prevent malaria may not have received special emphasis. This is a reflection on the level of health education about ANC and should be addressed but it is probably not an accurate reflection of the use of IPT. Almost every home had a LLIN and almost all mothers and children under five were sleeping under a net. This is a remarkable accomplishment due to the efforts of AMREF and PSI. After the BSCP proposal was submitted, PSI led a national initiative to distribute ITNs, which meant that by the baseline KPC 77% of households with children 23 months or younger had an ITN. Due to the BCSP’s distribution of nets to remote areas that PSI did not reach, as well as its BCC, CHW training and community mobilization, the percent of households with ITNs increased by almost 20% as did the use of nets within these households. In conducting community interviews during the FE, a frequent comment was that people of all ages now want ITNs so the community clearly values this form of malaria prevention. The increase in the coverage of women receiving SP twice during their pregnancy can be attributed to the increase in women having 4 or more antenatal visits and the improvement in knowledge, skills and supplies at the health facilities. Identification and care seeking for children with one symptom of malaria (fever) also 16 improved from 7% to 38%, although this is an indicator which should be increased further. Another concern is that almost all cases of fever are treated as malaria (one site during the FE reported treating 700 cases of malaria/fever during the past month but also said they had a stock out of malaria medications during this same period). Due to the lack of lab facilities at most dispensaries, malaria diagnosis is based on symptoms resulting in overtreatment and ultimately may lead to drug resistance. The objective of increasing the percentage of children <5 who were treated according to MOPHS protocol for malaria was not measured. If it had been, the lack of proper testing would have been further documented. Malaria was almost always one of the first health problems identified by community members during the FE interviews. This indicates that the community is aware of the costs of malaria in terms of illness and the danger to pregnant women and young children. Most of those interviewed felt that the BCSP had reduced malaria in these groups. One objective that was not achieved was increasing the number of shopkeepers correctly dispensing anti-malarial drugs according to MOPHS protocol. This intervention (training and monitoring the shop keepers dispensing the drugs) was not sustained during the life of the project, primarily because there were so many other BCC, training and monitoring demands to meet other program objectives. While this was a laudable objective, it would have required a level of attention to private outlets which is difficult and time consuming. It would have been more effective to ensure that people were being accurately diagnosed through malaria testing and that the facilities had medications available to treat those properly diagnosed. Then, the project could have turned its attention to assessing, training and monitoring private drug dispensing outlets. HIV/AIDS HIV testing during ANC and knowledge of HIV status improved—84% of women reported being tested during their last pregnancy and 81% knew the results of the test. Women who were not tested may be the ones who had to go to another clinical site to receive the test or receive limited antenatal care. Knowledge about ways to PMTCT increased in terms of women identifying at least two ways but knowledge, specifically about ART as a means of preventing MTCT did not change. In the FE knowledge about HIV prevention was strong among community members and all interviewed mothers had been tested while they were pregnant. There were many HIV/AIDS programs going on at the same time as the BSCP including the large APHIA II and AMPATH programs. They were doing mass screenings and HIV prevention. APHIA II even did some door to door screening which was reported by one mother in the FE interviews. The women in the Mother to Mother groups interviewed had the most knowledge about HIV prevention compared to other community groups. They knew the importance of condoms, and that ART would prevent the baby from getting HIV from a positive mother. However, they were concerned about the lack of male use of condoms, the interference of alcohol consumption with condom use and the fact that because ARVs make HIV+ people look healthy that they may be engaging in more high risk sexual activity. All those interviewed from the community to facility level pointed out the important role that men play in decision-making about HIV testing for their wives, and their lack of participation in VCT. While most 17 women are tested during ANC, very few men respond to facility requests for VCT even when their wife is found to be HIV+ (one nurse said she had seen only three men for HIV testing in the past year). A couple of the new fathers who were interviewed in the FE had been tested, one when he wanted to marry a second wife and another because his pregnant wife took him with her to be tested. Other HIV/AIDS findings of the FE included the finding that health workers said that there is a greater acceptance of condoms. One CHEW reported he had just distributed 6000 to youth. Another CHEW had the perspective that HIV was not a major area of his work for BCSP. Another finding was that for women in remote areas, health workers recognize that HIV+ women may still deliver at home. They said that they often give these women ART to take at delivery as early as 28 weeks since they are not sure they will return for ANC. Most of the community members interviewed in the FE knew that an HIV+ woman should deliver at a facility. Community leaders and CHCs thought that stigma associated with HIV had been reduced and noted that there were some people who were letting others know their status. Exclusive breastfeeding was included as an objective for HIV/AIDS rather than MNH. The BCSP area has had a low rate of EBF, with early introduction of mixed feeding due to pressure from family members and beliefs that breast milk does not provide adequate nutrition, especially if the mother is not eating well. It is also customary to give newborns, water or other drinks before breast milk. While the project was successful in increasing EBF in the last 24 hours for infants 0-5 months from 11% to 52%, there seemed to be some confusion about the messages. While CHWs said that HIV+ mothers should EBF for 6 months, one of the new mothers said that “avoiding breastfeeding” was a way to prevent HIV transmission and another said that HIV+ mothers should not breastfeed at all. The EBF BCC materials show pictures of a woman eating a full plate of nutritious foods—meat, fruits, vegetables, ugali, and beans—a meal well beyond the means of most women in rural areas even on an occasional basis. If women think this is the diet that is necessary to be healthy for EBF, they may be getting the unintended message that their diet is not good enough for EBF. Contribution toward Objectives As can be seen from Table 2, most of the objectives of BCSP were achieved and described above. Some of the exceptions have been discussed above and others will be addressed in this section. In general the reason for success was BCSP’s close collaboration with the DHMT to mobilize communities to increase awareness about MNC, malaria and HIV/AIDS and to engage community resources in prioritizing health issues. This was done by using many of the strategies that AMREF has developed and is one of its strengths as an NGO. The approaches built upon existing community leadership structures by reaching out to Chiefs and Assistant Chiefs. They also required creating new community structures such as CHCs, CHWs, MtM Support Groups, and school based health education programs. Each of these groups will be presented in more detail. Community mobilization was a major key to the success of this project. The Provincial Administration functions through local elected leaders who are designated Chief (large divisions/locations) and Assistant Chiefs (sublocations or smaller units within divisions/locations). From the beginning of the BCSP, these leaders were engaged in planning and implementation. These were key individuals for reaching out to the communities and to create new structures to focus on the CS implementation including the MNH, Malaria and HIV/AIDS focal areas of the BCSP. Based on FE interviews, these leaders felt involved and respected. They had two primary concerns. The first was 18 the gap in communication between the first planning year of the project and the time they felt the project actually got started (about halfway through the grant). This was due to the decision to work on facility capacity and the political disruption. Certainly in terms of the former, better communication about the decisions to delay CHW training and work with the facilities would have addressed this concern. Without this, some leaders commented that the project “disappeared” for a couple of years and then things really started going. They felt that much more progress could have been made if this momentum had been in place throughout the project. The second was that some leaders expressed a desire to be included in the trainings for CHWs so that they could be better informed about activities and the response to health problems. Most of the chiefs felt that CHWs had made their jobs a lot easier because CHWs were intimately connected to what was happening in the community and brought the concerns of the people to them. Community Health Committees (CHCs) were an important new community structure for the implementation of the CS and the BCSP—they provided community oversight and feedback for the work of CHWs and CHEWs. These committees include chiefs and assistant chiefs, CHEWs, CHWs, women’s and men’s groups representatives and sometimes youth leaders. They met every couple of months or once a quarter to review progress and look at health indicators for their areas. They also helped spread health information in the community and provided outreach for health education sessions sponsored by the project. One of the innovative approaches to provide information for CHCs was the use of chalk boards that provided information on the sublocation and project indicators on a quarterly basis. These indicators were population, number of households, number of pregnant women, number of children under 2, number of women attending 4 visits for ANC, number of pregnant women receiving PMTCT, number of deliveries by skilled birth attendants, number of children under 2 fully immunized, number of pregnant women using ITNs, number of children on EBF, number of households using treated water, number of households using latrines, and number of dialogue days. See pictures below. Chalk boards for CHCs Board in use FE Board in storage at hospital FE The chalk boards were one way of facilitating community participation and most of the CHCs said they used them and found them valuable. There did seem to be a few limitations in the use of the chalk boards. One was their size, they are quite large and therefore not easy to transport. Another was the 19 lack of denominators for most of the numbers that were being posted, making them of limited value for documenting progress in terms of the whole population. A third was that some of the boards viewed during the FE were not up to date. In one hospital, the boards were all locked up in a room to “protect them,” making them less accessible for community viewing and use. CHWs will be more extensively discussed in the section on CHWs, Section F. 4. CHWs were a major new group in community mobilization. There were a few CHWs from previous NGO projects, but they were not a well established part of the community level of the health system. BCSP trained 910 CHWs and helped DHMT structure and provide supervision through expanded roles for CHEWs and Leader CHWs. This was a major initiative and the foundation of the project’s objectives to increase knowledge and service utilization for MNC, malaria and HIV/AIDS. Mother to Mother Support Groups were another cross cutting intervention. The idea of identifying women who were pregnant and utilizing antenatal care early in the project was a good one. Since most women did not receive 4 antenatal visits, this was a way of identifying positive deviants and working with them to identify ways to get other women into service, including having them reach out to other pregnant women. Fifty-one (51) mothers’ groups were formed and trained. Women from these groups were interviewed during the FE. They felt that the health education they had received helped them and made them better resource people for other pregnant women. These groups helped spread the word about the importance of ANC, testing for HIV, facility births and EBF. School based education programs were a way to broaden the dissemination of messages about malaria in particular. Seventy (70) school clubs were formed and 4200 children were involved in peer/community education. This approach helped children learn the importance of health and prevention from an early age and also helped them improve communication skills as well as knowledge. The two school groups interviewed during the FE were enthusiastic about the process and had expanded their health club work to sanitation and HIV prevention. Because this was an additional component of the project the only indicators for this activity were malaria related. The funding came from AMREF Netherlands and Spain. Unfortunately, there was not much process or direct outcome evaluation available to review. Fathers’ groups were also planned but not implemented due to the time frame and funding constraints. Engaging men, especially in the male dominated communities found in Busia and Samia, could address some of the barriers to health care seeking for women and children. Another community resource that was not fully engaged was TBAs. It is clear that the use of multiple levels of community mobilization was a major strength of this project and underlies the successful accomplishment of most program indicators. Another major cross-cutting approach was capacity building, not just at the community level as described above but also at the level of health services and facilities. A large component of this was training (CHWs, CHEWs, health facility staff, DHMT) as well as the improvement of MNH facilities. Eighteen major trainings were undertaken which ranged from 2 to 12 days in length and the number of people receiving training ranged from 13 to 910 (Annex 7). While the majority of this training was focused on CHWs, CHEWs and health facility staff, CHCs also received training in Primary Health Care and the Community Strategy. This extensive level of capacity building will have payoffs well past the grant funding period. Even if all of the CHWs and CHC members do not stay engaged, they will be 20 better informed citizens and able to share their knowledge and training in other ways. The communities will benefit and professionals who move into other geographic areas or organizational roles will also be better informed. This will improve the ability of MOPHS to implement the community strategy. Contextual Factors As described in many of the project documents, Western Province and the Busia and Samia Districts are among the most densely populated and poorest in the country. Poor health indicators for mothers and children are a reflection of high needs and low resources. Unemployment was always cited as one of the top problems by community leaders. Low levels of formal education, especially for women, as well as traditional male dominance in family decision-making also contribute to high levels of maternal and child mortality and morbidity. Costs for transportation and care were the major reasons for not using health facilities for antenatal, delivery and postpartum care. Some traditional practices and beliefs were noted as barriers in the original planning and barrier analysis. However, these were not frequently noted as barriers in the FE, even when the interviews specifically asked about them. One example is traditional practices around home delivery and the burial of the placenta—women, TBAs and health facility workers all said these were no longer done. As in many subsistence level agricultural communities around the world, economic factors are major influences on health decisions about care utilization. Economic factors also influence the effectiveness of new interventions such as the use of volunteer CHWs. The biggest impediment to sustainability is the lack of any financial support for their activities. This contributed to the attrition rates of CHWs which are likely to increase with the end of the project. There were several other major contextual factors which affected the project implementation. These included: • The need to delay CHW training while quality of health facilities was addressed • The request from the MOPHS to pilot the Community Strategy which meant waiting for some clarification on the strategy • Working with MOPHS to develop the MNC training manual required several levels of review and approval resulting in delays • The post-election violence in 2008, which affected Busia, and stopped activities for almost 5 months. This coincided with major staff turnover for BCSP • The reorganization of MOPHS Districts which caused disruption in District support to the two intervention areas. HIV/AIDS and malaria prevention (especially ITN distribution) was greatly aided by the contributions of other programs working in the area such as APHIA II, AMPATH, MSF Spain and PSI. Undoubtedly the work of these organizations contributed to the achievement of project indicators in these areas. AMREF and BSCP worked in partnership with these organizations to reduce duplication. As always, multiple organizations working at the community level is sometimes confusing for community members who have trouble distinguishing which NGO programs they participated in. One confounding factor is that different NGOs have different approaches to CHWs and whether they are paid, as well as in terms of the provision of “incentives” for service utilization. This makes it difficult for programs that don’t provide payment or incentives. 21 Role of key partners The table below shows organizations that AMREF partnered with and the roles that each partner played and the results of the collaboration: Partner Role in Project Results of Collaboration Ministry of Public Health and Sanitation (MOPHS) Provide technical support through: • Training of CHWs, CHCs and health facility workers • Provision of health facility supplies including drugs to support MNC services. • Conducting support supervision and carrying out on-job refresher trainings for health workers on MNC, malaria and HIV/AIDS • Post staff at the community level (CHEWs) to supervise and support the CHWs Through the partnership with the MOPHS the following were accomplished: • MNC training manuals for CHWs and TOTs were developed. • The ministry staff played a critical role in training and supporting the community structures-CHWs, CHCs, CHEWs. • Through the improved MNC, HIV/AIDS and malaria services at health facilities, the number of mothers seeking services significantly increased. Ministry of Education The Ministry of Education through the District Education Officers in Butula and Samia facilitated the formation of school health clubs and also selected some teachers who were then trained by the project to disseminate MNC, malaria and HIV/AIDS messages to pupils and communities 70 school health clubs were formed and through their participation in child to child and child to parents activities contributed to an increased number of community members reached with health messages on MNC, HIV/AIDS and malaria. Provincial Administration The officials of the Provincial Administration who include the Chiefs and Assistant Chiefs participated in the social mobilization of the communities, the selection of CHWs and CHCs, community dialogue meetings and also organized public meetings for the CHWs to provide community health education on MNC, HIV/AIDS and malaria issues Through the dialogue meetings and community public meetings, a high number of community members were reached with health messages. 22 Partner Role in Project Results of Collaboration Great Lakes University of Kisumu (GLUK) The collaboration between the project and GLUK enabled the GLUK MPH students to carry out research work in the BCSP sites covering areas relevant to the project Through the collaboration, three studies were carried by the masters students, namely: • A study on “Community Emergency Transportation Systems for mothers seeking obstetric services” • A study on “Factors contributing to CHW motivation and retention” • A study on “Factors influencing male partner involvement in promoting skilled deliveries” AMPATH AMPATH was a member of the BCSP implementation team and provided technical support in the area of PMTCT and also renovated some health facilities within the project sites to facilitate provision of quality MNC and HIV/AIDS services Regular supply of HIV test kits contributed to increased uptake of PMTCT services in health facilities in the project area. MSF Spain MSF Spain was a member of the BCSP implementation team and provided technical support in the area of PMTCT. Regular supply of HIV test kits contributed to increased uptake of PMTCT services in health facilities in the project area. USAID Kenya Mission Office USAID Mission in Kenya organized PVO and child survival stakeholders meetings that facilitated exchange of experiences. Mission Office was supportive of AMREF’s and BCSP’s role in working with the MOPHS in piloting the Community Strategy and MNH curriculum for CHWs. Enhanced capacity of the project team through stakeholders experience sharing meetings and exchange visits to child survival projects. APHIA II Western Covered other child survival interventions such as family planning, household hygiene, infant feeding, and also trained CHWs in HIVAIDS home based care and formed and supported some mother to mother support groups. Increased coverage of child survival interventions, more mother and care takes reached through the additional M-to-M support groups formed by APHIA II, reduced infant/childhood morbidity resulting from improved feeding practices supported by APHIA II. 23 Partner Role in Project Results of Collaboration Communities in Butula and Samia Through partnership with the communities, the project was able to establish the community structures, namely, the CHWs, CHCs, and PDQ teams that played a crucial role in the project implementation process. The community members also allocated some resources at the community level such as funds to support the project interventions. The community partnering processes contributed to increased community involvement in the project interventions and improvement in the MNC, malaria and PMTCT service uptake. The variety and extent of partner organizations in the BCSP illustrates the important role that collaboration played in the development and implementation of this project. AMREF has a history of working with other sectors and organizations which was an excellent way to optimize the resources available for BCSP as well as to affect national policy and local sustainability. Overall Design Factors that Influenced Results As presented in Annex 1. Results Highlight, one major design choice was to focus on improving facilities before conducting a massive community outreach to promote service utilization. This was based on information obtained during the DIP. The training strategy described above and in Annex 7 was another key design choice which was made more challenging when the MOPHS doubled the number of CHWs required for the population of the project. The project’s commitment to capacity building will continue to improve community health and services. Partnering with multiple levels of community and health facilities was another strength of the project and undoubtedly contributed to the overall success in achieving objectives. F. Discussion of Potential for Sustained Outcomes, Contribution to Scale, Equity, Community Health Worker Models, and Global Learning This section will only address issues that have not been covered previously. 1. Progress Toward Sustained Outcomes The project indicated that it would use the Sustainability Framework in the DIP and this was further defined in the 1st and 2nd year annual report. BCSP used all three dimensions of Sustainability and 5 of the 6 CSSA Framework indicators. The three dimensions and six indicators are: Dimension 1: health outcomes and health services characteristics; Dimension 2: local organizational capacity and viability; and Dimension 3: community capacity and social ecological context. The only indicator which was not included from the beginning was the social ecological context (the Year 1 Annual Report indicated this would be reported but not measured). The sustainability indicators related to specific project intervention areas have been discussed in the previous section. Sustainability was an issue of concern in the organizational capacity assessment which was conducted in the first year of the project—the consultant felt that the exit strategy should have been incorporated 24 and planned for since the project’s inception. The project made great progress in all areas but it is not clear how many will be sustained in the absence of additional funding either through the MOPHS CS or other NGOs. Each of the five indicators of CSSA will be briefly addressed although measurements were limited for several of them. Objective targets were exceeded in the health status indicators. It is expected that many of these will be sustained because of increased women’s and community receptivity to ANC, SBA and PPC. This assumes that the health facilities are properly staffed and continue to treat the women and their families with respect. As indicated previously, cost, distance and transport remain major barriers to SBA. It is not clear that the same will be true for EBF, especially because the EBF BCC was part of HIV messages and this resulted in some confusion about who should EBF. Since there remain many cultural and gender beliefs about infant feeding, it is not clear that this will be sustained. The final Health services capacity indicator was facilitative supervision checklists, although training in ANC, EOC and PMTCT, M&E, and PDQ as well as equipping and renovating MNH Centers of Excellence could certainly be considered important measures of health services capacity. At the end of the project all facilities were using the supervision checklists at least 4 times a year, Training had been completed in all areas except PDQ which only was completed at one health facility. The training of staff is an ongoing resource for the BCSP area, except in cases of staff transfer (which occurs frequently). It is not clear how new staff will be trained in these areas, either on the job or in separate training. For staff with the training, services should continue at high quality as long as supportive supervision continues. The DHMT has the skills to do this, but may not have the transport and/or time without the support of BCSP. The MNH Centers of Excellence will continue to be resources but will need maintenance and upgrading with time. Local organizational capacity was used to measure strength of the DHMT and AMREF/BCSP. An organizational capacity assessment was conducted by an outside organization in the first year of the project. This identified some of the areas that were addressed in the project. Measures of this included improving understanding of financial planning, management and budget (focused on AMREF) and communication between field staff in the DHMT. The former was addressed through staff training and the latter by the provision of cell phones. While measures of either capacity or viability were formally used, there are indications of sustainability through local organizational capacity. The BCSP no longer exists but many of the staff are being transferred to other AMREF projects where financial training will be useful; these are skills that will be valued in any organizational setting. Cell phones are here to stay so the DHMT communications should remain in place as long as there is minimal funding for phone cards. M&E training occurred but the M&E system at DHMT did not seem to be functioning well. No broader health system data on ANC, SBA, PPC, lab work, etc. was available from the DHMT and one of the complaints during the FE was that the flow of information was only upward with little feedback or data back to the facilities. The institutional weaknesses identified in the organizational capacity assessment were not addressed, although the government does seem to be committing resources to the CS which should improve the situation. Local organizational viability was also not measured but was addressed during the life of the project in terms of AMREF staff training in strategic leadership which was disseminated to CHEWs and CHW leaders. CHEWs and CHWs are the core for continuation of intervention activities at the completion of the grant. Other efforts in this area were collection of additional funds for project activities and collaboration with GLUK for key research issues. Sustainability of these activities is more questionable. Additional funds will be dependent on the availability of other NGO projects in the area 25 and/or the commitment of more Kenya government funds for health services. It would be possible and an excellent idea to continue the research activities with GLUK through the DHMT but that would require better data systems and a focus on research as well as service delivery. Community capacity indicators were unevenly achieved. CHCs and CHWs played a major role here and the major indicators measured how often CHCs submitted data. The former was well below the project target of 70% with only 35% of CHCs submitting data on a monthly basis. Some of this may be due to the lack of feedback on data from the DHMT described in the organizational viability description as well as the discussion about the use of chalk boards. In terms of sustainability of CHCs, most of the CHC’s interviewed in the FE said they would continue to meet but probably not as often. It is likely that they will continue to play an important role, especially as the CS rolls out nationally. The data review and submission issues will need to be separately addressed if this is going to work going forward. One factor will be the retention and work activities of the CHWs who collect the community data. Related to this is the issue of ongoing CHW training and support (ranging from transport and other expenses to stipends). In the FE, many community leaders and health facility workers hoped that another NGO would come into the area and be able to work with the CHWs trained by BCSP—this was their main vision of sustainability. In general, the CSSA Framework was not systematically incorporated and measured throughout the project. It was primarily addressed in Year 1 when the first measurements were made and the Sustainability Dashboard was created. However, discussion of CSSA and indications of measurements diminished with each year of the project and the final indicators were few compared to the original plan. This is no doubt due to the increasing demands of other activities such as training 910 CHWs and developing the national MNC training curriculum. Both of these were major contributors to sustainability but were not measured as part of the original CSSA indicators. The phase out plan is to turn responsibility for program activities over to the DHMT, CHCs, and the communities themselves. The sustainability of this approach is addressed above. The other main problem is that even though the need for a phase out plan was identified in the organizational capacity assessment in the first year of the project, it was never really a major part of the project. This is no doubt partially due to the many changes that took place during the project, and the need to respond to the requests of the MOPHS for increasing the number of CHWs to be trained. It is difficult to think about phase out when trying to keep up with changing demands. 2. Contribution to replication or scale-up BCSP was a tremendous success in terms of scaling up training of CHWs at the direct request of MOPHS. The project tripled the number of CHWs trained when the CS reduced the population ratio for each CHW. All communities in the target area had CHCs and outreach was directed at the entire population. More importantly, the MOPHS asked the BCSP to serve as a pilot area for the implementation of the CS. This means that the lessons learned will be used at a national level providing replication and scale-up to the Kenyan population. The KCO Program Director is a member of the national task force advising the MOPHS on the implementation of the CS and has already had significant input on community structures and incentives for CHWs. This is a major contribution of the BCSP and one which recognizes AMREFs role in Kenyan health development. In addition, when BCSP began writing the MNH Training Manual, the MOPHS asked them to work with the ministry to come up with a national training curriculum. While this delayed implementation of BCSP, it means that the project had national impact in MNH guidelines and training. These are significant accomplishments for 26 any CSP. AMREF is also implementing a new program on MNH in Lamu which will be using the lessons learned from BCSP. 3. Attention to Equity Several types of equity were addressed in the BCSP: gender, socioeconomic, and geographic. The government of Kenya is committed to equity in all its services including health, which has been a challenge due to ethnic strife in recent elections. Kenya is a country with over 40 indigenous tribal groups (70% are composed of five major groups Kikuyu, Luo, Luhya, Kamba and the Kalenjin) and more than 60 languages and dialects. Most (90%) of the population served by BCSP was Luhya with 5% Teso and 5% other (Luo, Kikuyu, and Somali). The project made every effort to reach all members of communities including minority ethnic groups although this was not specifically tracked. Most of the population is Christian although there was a small Muslim population, mostly in trade centers. Gender equity was a major issue addressed by the BCSP. The Luhya are a male dominant society – men decide family size and make all key family decisions, and wife inheritance and polygamy are common. Women have less education than men, are not allowed to own land, animals, or businesses and have little control over their own lives. Gender inequality is a major barrier to access to healthcare services for women and children. CHCs and CHWs were predominantly male, although women were represented. The predominance of men is partially due to women’s lack of education and literacy skills, this includes TBAs. However, involvement of men was essential to reaching out to other men in the community and educating them about the importance of maternal and child health issues. Community leaders were also predominantly male and they were able to serve as role models for other men in the community by having their wives deliver in facilities and go for ANC. Community dialogue meetings and home visits by CHWs also reached husbands/fathers. Unfortunately, the father to father groups were not implemented—this has been demonstrated elsewhere to be a very effective approach to engaging men in MCH. Women were engaged directly through MtM Support Groups, home visits, and community education. This was an empowering strategy and one that seemed to work in combination with the community mobilization approaches to men. This was evidenced by the objectives achieved in ANC, SBA, PPC, Malaria and HIV/AIDS. Greater effort should be devoted to increasing men’s awareness about the importance of women’s and children’s health and actively engaging them in the process. This should include an emphasis on men’s health as well since men don’t go for health care themselves, including HIV or other types of STI testing and treatment. The role of men in promoting SBA was another area of collaborative research with GLUK. This study was completed in the Spring of 2010 so the results were not incorporated in the program implementation. This work will be of value to future MNH projects in the area. Major findings were that higher levels of male partner education and employment were associated with SBA, although age was not. This suggests that men with lower education and no formal employment should be targeted in future projects. Cultural practices that were identified were the practice of keeping newborns inside for several days after delivery (4 days for boys and 3 for girls). Men were also concerned about how they would be treated at health facilities. Socioeconomic equity affects almost everyone. Samia and Busia are two of the poorest districts in Kenya. Agricultural income is limited and affected by weather and market prices and there are very few other means of employment. One of the original ideas of the BCSP was to introduce small income generating activities, especially for women. This would be a good idea moving forward since it was not possible to implement this among the increased demands of the project for the CS and CHW training. Fees at facilities also need to be considered as they often present a major barrier to care. 27 Geographic equity is still a problem, although the BCSP was able to get all communities involved in planning and advocating for services. New dispensaries are opening closer to remote areas and birthing facilities are also being expanded. One problem faced by the MOPHS is that even though facilities are expanding, there is limited manpower to provide services. More support must be given for trained health providers to staff these buildings as well as equipment to run them. There were many examples of this during the FE: a laboratory that was built but not fully equipped or staffed, a birthing facility but no one to deliver babies, etc. 4. Role of Community Health Workers As discussed previously, CHWs were the keystone of this project. Their roles included household registration of pregnant women, home visits during pregnancy and post-partum, community education, data collection for the CHCs and DHMT, and serving as health resource people for their communities. CHWs are an integral part of the MOPHS CS and will be scaled up nationally—although major issues of CHW to population ratio, exact role, and whether there will be any remuneration in the form of incentives or salary support are not clear. The CHWs in the BCSP were volunteers, selected by their own communities. CHWs indicated that they spent 1-3 days per week in this role. CHWs received a lot of community recognition and felt valued for their work, but often mentioned that they could use some support in terms of transport, expenses or stipends. It might be that CHWs volunteering 1 day a week is a sustainable role with minimal expense. However, when the time commitment becomes more extensive and starts to cut into opportunities for other income generating activities this may not be adequate. This is exactly the issue that MOPHS is struggling with in assessing population ratios, work responsibilities and the idea of payment. (Note: even if CHWs were paid as little as US$10 per month-- $20 is being considered—it amounts to a huge sum of money when it is considered at a national scale. One suggestion was that as funds are decentralized to Districts, local decision-makers would have more money to allocate, but this is not usually the way things work. When limited funds are “decentralized,” the problem of insufficiency remains.) In BCSP CHWs valued the training they received, recognizing that this would make them more valuable as volunteers or employees on other projects. They also benefited from supervision from BCSP staff who trained CHEWs and CHW Leaders to do field supervision. This was an important component that ensured CHWs were doing their work well and gave them the support they needed. It is not clear how or if this supervision will continue once the grant ends. The DHMT and CHEWs said that it would continue, but CHEWs have many other responsibilities and once the focus of the grant is over, other activities may take priority. Again, this will be dependent on the implementation of the CS and the resources that are available. Attrition rate of CHWs will also have to be addressed. A research project on CHW motivation conducted with a graduate student from GLUK, found a retention level for previous CHWs was 30%, although it was higher than this for those trained in BCSP—closer to 80% over two years). Almost all (84%) of the CHWs were serving more than the recommended 20 households (even with the increased numbers of CHWs trained). While only 10% said they expected monetary gain upon recruitment, 65% said that reimbursements had motivated them to continue and almost 40% said that material incentives like bicycles, t-shirts and ITNs had contributed to their motivation. Almost 100% said refresher courses were important for continued motivation. Some of the community leaders encouraged the community to “reward” the CHWs for their work by assisting them with their farm workload, but this was not common in the BCSP. One problem with the research is that there are many CHWs and some work for 28 other NGOs and receive more in the way of remuneration. This will be another factor for the MOPHS to consider when implementing the CS. Certainly, Busia and Samia Districts will be in an excellent position to implement the CS—they have a large supply of well trained CHWs, a structure of supervision and leadership within the CHW cadre as well as the CHEWs. Annex 7 provides the Training Matrix, much of which was focused on CHWs, their supervisors and the CHCs which provided oversight for their work. 5. Contribution to Global Learning The BCSP resulted in many important lessons and promising practices. In fact, it was hard to select one for the Results Highlight because there were so many from which to choose. • Community Health Workers have already been discussed in great detail, but BCSP was able to provide extensive training in a short period of time to enable CHWs to be up and running in the field once this part of the implementation began. This demonstrates that rapid implementation is possible with appropriate training tools and skilled field staff. AMREF was also able to enlist a graduate student from GLUK to help identify motivators for CHWs. This information will be of value to the Kenya MOPHS as it moves forward with the Community Strategy. • Engaging community leaders and decision makers from the planning stage of the program is a critical lesson for other projects. Often this is not done until a program is well underway. It was clear from the FE and the results of this project that communities felt this was their project as well as AMREF’s and that they are committed to finding ways of sustaining it. While there were some communication difficulties, in general community leaders felt they had a say in the decision-making and if anything, wanted more training and involvement. • Linking the community and the facility was another important accomplishment. BSCP identified health service providers as one of the barriers to facility care early in the project. They focused on fixing the problem before promoting facility use at the community level (Annex 1. Results Highlight). The use of PDQ, while not fully implemented at all facilities, helped the community understand that they could advocate to improve the services that were available. Working at the community and facility level at the same time is an important principle for all projects that try to increase facility utilization. • Developing research partnerships with a local university provided resources for learning more about project area needs and resources. This is a practice which should be encouraged in all CSP. The linkages should be established early on because of the time delay in getting the projects started and receiving the results. Projects should also make sure there is joint access to the data that is collected. • Making program results visible to the community to facilitate local decision-making was another innovation in this project. While the use of the chalk boards had some limitations, the overall purpose of being able to show the community their own information about project objectives was a valuable one. It also makes data collection a more important activity for the CHWs. • Organizing MtM and CtC were excellent approaches to extend outreach to the community and to empower women and future community leaders. Mothers who were engaged in good practices and utilizing health services were the most effective people to reach other mothers. 29 Children learned health messages and how to improve their communities. As mentioned previously, it is unfortunate that this approach was not able to be used with fathers as well. G. Conclusions and Recommendations 1. Conclusions. BCSP was able to achieve most of its objectives focused on MNH, Malaria and HIV/AIDS. These were challenging targets in a resource poor setting and during a tumultuous time politically. Another challenge was meeting the MOPHS’ rising expectations about training and curriculum development. BCSP made remarkable progress on some hard to change behaviors, including facility deliveries (which more than doubled from 20% to 53%) in a setting where cultural norms support home births; and increasing 4 ANC visits and post partum care. These in turn made possible the increase in IPT, testing for HIV/AIDS and PMTCT. Knowledge levels also increased although not to the same high levels. This demonstrates that improving the system of care and creating a cadre of trained CHWs can in itself improve access and utilization of health services. Community mobilization was one of AMREF’s most important skills in making this project a success. Other contributing factors were staff development, M&E, and strong linkages with local and national decision-makers. AMREF was able to raise additional funds to conduct activities related to the project and worked in close collaboration with other NGOs in the project area. Flexibility in responding to MOPHS requests and adaptability in terms of timing, training numbers and curriculum development were other assets that made this project a success. 2. Recommendations. Increasing access to skilled delivery and EMOC has been demonstrated to be the most effective way to save the lives of women and newborns. The methods used by BCSP to achieve these difficult to change behaviors and resources should be used as a model for other projects. AMREF’s rapid scale-up of CHW training and supervision should be shared. AMREF’s community mobilization process should be further documented and written up as an example for other programs. The emphasis should be on the steps and approach to working in communities before, during and after a CSP. Models for research collaboration with academic institutions on topics related to CSP interventions should be developed perhaps including a sample memorandum of understanding which addresses research topic selection, timeliness of data collection and analysis, and data ownership. The results of the CHW motivation study and the MNC operations research project should be shared with Kenya MOPHS and the global community. This project accomplished so many of its objectives in spite of major delays due to political instability, and delays related to improving the quality of the project: working to improve 30 31 facilities before engaging communities, collaboration with government on training materials and approaches, and scaling up CHW training. With continuation funding for another 2-3 years, much more would have been learned. The CSHGP should consider short term continuation grants for programs that can demonstrate impact and the need for more time to solidify implementation and results. Program intervention sustainability is something that can only be determined after the grant funding ends—the CSHGP should conduct separate sustainability studies by returning to project areas 3-5 years after a grant ends to determine factors that contributed to sustainability as well as which interventions or community structures are most likely to survive. Annexes African Medical and Research Foundation Final Evaluation Report Busia Child Survival Project (BCSP) Butula and Samia Constituencies of Busia County, Kenya October 2005 – September 2010 USAID/HIDN/CSHGP Cooperative Agreement Number: GHS-A-00-05-00009-00 Date of Submission: December 15, 2010 Annex 1: Results Highlight The major highlight of this program was a best practice—improving the quality of antenatal and delivery services BEFORE initiating a community strategy to increase referrals and utilization of health facilities. The AMREF-BCSP recognized the importance of Minimum Activities for Mothers and Newborns (MAMAN)and the community to hospital continuum of care. The project based program implementation on the needs assessment conducted as part of the DIP process. The program also conducted innovative strategies to make this happen, building on the KPC and Facilities Assessment, by conducting qualitative research, Partner-Defined Quality (PDQ), developing Maternal and Newborn Centers of Excellence, and effectively engaging the community at all levels. The AMREF-BCSP project identified the need for improved facility-level care, both in terms of services and provider relationships with patients, early in the baseline assessment. It was clear from the data in the KPC and community interviews that facility problems needed to be addressed. To do this, the project initiated a PDQ process at several facilities and also found funding for 4 MNH Centers of Excellence. While the PDQ process was only completed at 2 facilities, the program was able to increase awareness and improve health service quality at most of the hospitals and clinics associated with the program. AMREF-BSCP developed a close working relationship with the DHMT which enabled the “spread effect” as did the joint training programs that were developed for health providers. In addition, AMREF was able to use additional funding to remodel and equip the MNH Centers of Excellence. These Centers of Excellence provided privacy, more comfort, patient centered care, and better delivery equipment. They also allowed for the integration of newborn care in the same area as maternity. Women and their family members felt welcomed and respected at these maternity units, as did the TBAs, who often accompanied women to the hospital. Attention to improving care at facilities actually slowed the implementation of training CHWs and community engagement, but the approach proved effective in the long run. After care at the facilities had improved, AMREF￾BCSP invested heavily in training CHWs in health education, outreach, and referral. The CHWs were directly linked to health facilities and community health committees. Community leadership and health capacity were enhanced through the AMREF-BCSP. People at the community level felt that health services had improved and that their own knowledge about what to do had increased due to education by the CHWs. In addition, AMREF-BCSP was a key partner with the Ministry of Health in developing the national curriculum for community based maternal and newborn care—this curriculum and community strategy is now being adopted in all regions of Kenya. The success of this coordinated and phased-in process of improving MNH services prior to increasing community awareness and referrals to facilities was demonstrated through an increase in mothers with at least 4 antenatal visits from 30% to 50%; increase in mothers delivering at facilities from 20% to 53%; and an increase in postnatal visits within two days from 23% to 58%. These are remarkable increases in a short period of time. In addition, women and community leaders expressed general satisfaction with care at the facilities. Traditional birth attendants also reported that they were well received when they accompanied women to the hospital. Interviews with health facility staff also confirmed that husbands, family members and TBAs were welcome to be with women in the hospital. Staff at the MNH Centers of Excellence were proud of the improvements made in their facilities and the fact that they were able to provide better care to mothers and newborns. AMREF’s approach to improving the continuum of care to mothers and newborns should be used as a model for other programs to promote the importance of working at all levels from community to hospital. Annex 2. List of Publications S/No. Title of paper/presentation Title of meeting/conference/ Committee Date paper presented 1 Improving quality of maternal and newborn care through enhanced interpersonal communication Kenya Paediatrics Association held in Mombasa, Kenya April, 2009 2 Increasing access and utilization of maternal and newborn care services : The case of Busia Child Survival project AMREF KCO Annual Program Meeting March, 2009 3 Improving quality of maternal and newborn care through enhanced interpersonal communication Child Survival PVO meeting held in Mombasa, Kenya August, 2009 4 Effectiveness of community health workers in influencing maternal and child health practices in Busia and Samia AMREF KCO 2009 Annual Technical Meeting held in Nairobi, Kenya Nov., 20010 5 Supporting the Kenya Government’s Community Strategy: Successes & Challenges Child Survival Stakeholders’ meeting held in Nairobi, Kenya May, 2009 6 Busia Child Survival Project: Strengthening collaboration between AMREF and APHIA II Western USAID/APHIA II /AMREF (BCSP) collaboration meeting held at PATH offices, Nairobi, Kenya April, 2009 7 Effectiveness and cost summary of the Ministry of Public Health and Sanitation’s Community strategy for community-based maternal and newborn health service delivery in Busia and Samia districts, Kenya An Operations Research proposal presented to AMREF KCO Ethics & Scientific Research Committee June, 2009 8 Effectiveness and Cost Summary of Ministry of Public Health and Sanitation’s Strategy for Community-Based Maternal and Newborn Health Service Delivery in Funyula and Butula Constituencies, Busia County, Kenya.” To be submitted for publication. November 2010 Annex 3: Project Management Evaluation Major areas of project management which affected program implementation and outcomes are presented below. Planning The BCSP had the benefit (and the burden) of submitting the proposal 2 times prior to being funded. This allowed for an inclusive planning process and helped create commitment to the project by AMREF and the DHMT. Several of the key people from AMREF and DHMT had been involved since the first proposal. The DIP plan was practical given the situation at the time it was prepared. It had to be modified over the life of the project due to changing circumstances described in the narrative of the FE report. Because of delays in implementation and increases in training expectations by tripling the number of CHWs as requested by the MOPHS, several activities were reduced or eliminated due to budgetary constraints. These included TBA training, Father to Father Support Groups, working with private pharmacies about malaria medication distribution, and PDQ activities at the health facilities. There seem to be a few organizational issues between AMREF USA and KCO in terms of who makes decisions about some of these changes in project and financial requirements, KCO made the commitments to increase CHW training without the participation of AMREF USA, which then had the responsibility of finding the additional funds required. This is something that AMREF should probably address for future projects. However, in general, AMREF was able to manage these changes effectively and should be noted for its organization flexibility. Supervision of Project Staff, Human Resource and Staff Management Project staff supervision was in place for the duration of the project, although there seemed to be some problems midway through with some irregularities which were addressed by AMREF KCO and US. Figure 1 shows the organizational structure with identified personnel at the end of the project. A supervision checklist was developed with the DHMT for health facility staff. It is not clear if this will continue after the project ends. The DHMT staff reported it was very useful and improved quality, but DHMT staff and transport shortages will affect the ability to continue this activity. The organizational structure for the DHMT that should support ongoing project activities is presented in Figure 2. Morale of project personnel was strong at the end of the project. Most of the BCSP team had been together for the past two and half years and had good working relationships with each other and the DHMT. One BCSP, the M&E Supervisor, had been involved throughout the project. He indicated that there were some problems around the MTE which were resolved by the new management team. One strong point is that the Training Officer for BCSP was seconded by the DHMT, further strengthening relationships and project sustainability. This DHMT staff person started a doctoral program in public health in the fall, so will continue to use the BCSP experience in her professional work. Several DHMT staff were involved in the project since the beginning, including the new Samia DHMT Director. This contributed to the impact of the program and the commitment to continuation of the project activities. BSCP no doubt helped the leadership transition for the new district through training and support. Annex 3, Figure 1: BCSP Organization Structure (July, 2010) USA Nairobi, Kenya AMREF KCO Western Province Busia\Samia Districts AMREF USA Bill Yaggy Director of Institutional Giving USAID, Washington CSHGP Staff AMREF HQ Program leaders (Malaria, HIV/AIDS, and Training) Community Structures Societal Institutions, VHCs, CORPs, Households Gilbert Wangalwa Project Officer (M/E) Judith Ngeresa Project Officer (BCC) Everlyne Omunjalu Senior Project Officer (Training) Health Facility staff and HEWs in Butula & Funyula DHMT/DHMB Alan Kadenge Deputy Finance Manager Local Partners (MSF Spain/AMPATH, PSI) David Wamalwa CS Project Manager PHMT Western Province Timothy Munyao Finance Manager Dr Festus Ilako Head of Programs Mette Kjaer Country Director USAID Mission Dr Sheila Macharia MoH, HQ DCH, DRH, DOMC, NASCOP Cudjoe Bennett Technical Advisor Health Extension Workers (LQAS Lot Supervisors) Peter Ofware Program Manager, Child and Reproductive Health Annex 3, Figure 2: Project Management Structure One of the biggest personnel challenges was staff turnover. As mentioned above, one BCSP staff person was in the project for all 5 years, the M&E Supervisor. Every other position experienced staff turnover, sometimes more than once. BCSP over the life of the grant is shown in Table 1 below. This turnover was cited by several DHMT staff as a problem in program implementation since activities were often delayed in the transition and new people had to learn about the program and develop relationships with the community and DHMT. This was not a criticism of the new staff, whom everyone respected and felt had done a good job, but a reflection of the transition process. Another level of staff turnover was at AMREF USA—there were 4 Technical Backstops during the project, with the longest duration being two and half years with the most recent person in that position. KCO leadership was more stable, the Head of Programs was involved with BCSP since its inception and provided program management support until the new Program Manager was hired, and has continued to be interested and provide support. Both of these KCO staff are heavily involved in Kenya government taskforces and advisory activities. BUSIA CHILD SURVIVAL PROJECT – Project Personnel, current and those that have left Name of Officer Title Date of employ ment Date left the project Reason for leaving the project David Wamalwa Project Manager June 2008 Still with the project Not applicable Gilbert Wangalwa Monitoring & Evaluation Officer 2006 Still with the project Not applicable Judith Ngeresa BCC Officer August 2009 Still with the project Not applicable Everlyne Omunjalu Training Officer (On secondment from the MOPHS) March 2010 Still with the project Not applicable Bibianne Situma Admin. Assistant June 2008 Still with the project Not applicable Goeffrey S. Kathukia Driver June 2008 Still with the project Not applicable George Ong’ayo Office Assistant March 2005 Still with the project Not applicable Meshack Ndirangu Project Manager 2005 2007 Was re-deployed to another project within AMREF at a senior level Judith Raburu Training Officer 2006 Jan. 2008 Took up a job with another Organization Julius Onyango BCC Officer 2006 June 2009 Tendered resignation on personal grounds George Oele Training Officer June 2008 March 2010 Was re-located to another project within AMREF Humphrey Wakoli Driver 2007 April 2010 The vehicle he was driving was sold because it was old and its maintenance and running costs were very high, and so the driver’s position was declared redundant. Emily Masese Admin. Assistant March 2006 2007 Took up a job with another organization Francis Kikanga Driver 2005 June 2008 Was moved to another project within AMREF near his home John Mwinami Assistant Accountant 2006 Still with the project, based at KCO N/A The majority of remaining staff at the end of project expect to be re-deployed to other AMREF health projects within Kenya. Specifically, David Wamalwa, Project Manager, will move to the AphiaPlus project in Central and Eastern, and will be based in Embu in a similar capacity, while Gilbert Wangalwa, M&E Officer, will serve in the same role for a new Global Fund TB project. Financial Management and Logistics In general the financial management and accountability was adequate. The only irregularity was identified and corrected. The budget for the project was supplemented by other AMREF donors when shortfalls occurred for key program activities. As discussed earlier, budget shortfalls were primarily due to increasing training requirements resulting from the BCSP serving as a pilot for the Kenya government’s Community Strategy implementation. While this reduced the scope of some other project activities, it also is likely to ensure that BCSP interventions will be maintained at the community level through the next phase of the CS implementation. The details of the structure of the CS are still being negotiated so it is not clear if the full scale of operation developed in BCSP can be maintained. Similarly, it is not clear that HIV tests, ITNs, other supplies and BCC materials will continue to be available now that the BCSP has ended. This will depend on government funding and other NGOs who may be working in the area. In terms of project implementation, logistics functioned well and supported program activities in a timely way. Information Management Most of the issues about information management were addressed in the main body of the FE. The M&E data have been used beyond the project, BCSP has been used to provide input to national planning and the HIS have been used by other AMREF projects with training provided by the BCSP M&E Supervisor. DHMT staff reported using the training to improve data collection but also noted staff shortages as being a major problem in developing efficient and effective systems and providing timely feedback to facilities and communities. All BCSP partners including the community demonstrated a clear understanding of what the project achieved. The most common response was that it could have achieved even more with 1-2 additional years of funding. Technical and Administrative Support The main sources of external technical assistance were provided by the AMREF USA Technical Backstop and the KCO program manager. This assistance seemed to be adequate to project needs in terms of conducting required CSHGP activities. When additional staff training was required, such as for M&E activities, this was provided. The success of this approach is demonstrated by the M&E Supervisor then becoming a trainer for other projects. It would seem that the major need is for consistency in technical assistance from the beginning of the project, but this is not always practical or possible. In terms of headquarters and regional support to the field project, the KCO Program Manager indicated that he visited the project twice a year, met with the project officer once a month by phone, and had quarterly program reviews in Nairobi. In addition he was available as needed for specific issues and concerns. The Technical Backstop at AMREF USA played a key role in the proposal writing, DIP, MTE and operations research; was available by phone and email as needed; and, visited the project one to two times a year. Management Lessons Learned The primary management lesson was how to be flexible to changing needs, especially when engaged in significant collaboration with government. Since the latter plays a major role in sustainability of any project, it is critical to be responsive. AMREF demonstrated this ability and was still able to meet the expectations of the CSHGP as well as most project objectives. The internal learning that should take place is how to engage in this level of flexibility and decision-making with the full participation of all AMREF organizational units in deciding how to reallocate and/or identify new resources for project activities. Annex 4: Workplan Table and Project Results Framework BSCP Workplan Table (DIP planned activities and achievements by Year Oct 2005 – Sept 2010) Planned Activities Activity carried out Reasons why the activity was not Yes No carried out (X) (X) Planned activities and achievements for the financial year 1 (Oct 2005 – Sept 2006) 1. Hire project staff X 2. Set up project field office in Busia X 3. Hold inception meeting with the DHMT X 4. Hold inception meeting with the child survival PVOs X 5. Inception meetings with project beneficiaries (community) X 6. Establishment of the project management structures (PIT, TAG) X 7. Develop memorandum of understanding (MOU) with the Busia DHMT X 8. Conduct the baseline survey through KPC survey X 9. Conduct a safe motherhood need assessment survey X 10. Conduct an Integrated Health Facility Assessment (IHFA) X 11. Conduct a qualitative research X 12. Conduct an anti-malarial drug vendor survey X 13. Develop the project detailed implementation plan (DIP) X 14. Present the DIP in a mini-university conference at Baltimore in the US X 15. Identification/selection of the project CORPs X 16. Develop a Behavior Change and Communication (BCC) Strategy X 17. Conduct a workshop on BEHAVE X 18. Carry out cross-visits to other Child Survival projects by the project staff X 19. Conduct the mapping of the community resources through Geographic Information System (GIS) X Planned Activities Activity carried out Reasons why the activity was not Yes No carried out (X) (X) 20. The Project Manager attends a Qualitative Research workshop to build his research capacity X 21. The project M&E Officer attends a workshop in Rockville, Maryland on designing and conducting Operations Research X 22. C-IMCI workshop for the project BCC Officer and the Busia Health Education Officer X 23. The Lot supervisors, two DHMT members and the project (AMREF) staff attend a TOF training at the AMREF International Training Centre X 24. Hold an orientation workshop for the health workers on Partner-Defined Quality (PDQ) X 25. Four health facilities (Khunyangu hospital, Bumala “B” health center, Sio￾Port hospital and Nangina dispensary) identified for upgrading to child survival centers of excellence X 26. Develop a research proposal to test the impact of PDQ X 27. The project team work with the Kenya Pediatric Association to develop a research proposal on MAMAN X 28. Conduct a Training Needs Assessment (TNA) for health workers in the health facilities within the project sites X 29. Design the Community implementation and CBHMIS frameworks X Planned activities and achievements for the financial year 2 (Oct 2006 – Sept 2007) 1. Procure ITNs, HIV test kits and EmOC equipments and distribute to site X 2. Conduct training needs assessment X 3. Further review curricula and prepare training materials on PMTCT, EOC, FANC, M&E X Planned Activities Activity carried out Reasons why the activity was not Yes No carried out (X) (X) 4. Conduct joint capacity assessment & capacity building action planning for DHMT &AMREF X 5. Implement MAMAN operations research X The MAMAN (CBMNC) OR study was moved to the 4th year to be implemented in line with the MOPHS community strategy. 6. Distribute subsidized ITNs to targeted groups X 7. Renovate and equip four (4) Centers of Excellence X 8. Roll-out community implementation framework in 360 villages X 9. M&E/HIS training for HF, DHMT and CHEWs X    10. PDQ sessions implemented by CHEWs, CORPs, HF staff and health facility committees X Due to inadequate funding, all the PDQ sessions covered only one (1) health facility (Bumala “B” Health Centre) 11. Quality of care/IPCC workshop for DHMT/HF staff X X 12. Advocacy to influence practice and policy within AMREF and at district, province and national forums X 13. Review and develop/adopt BCC materials X    14. Orient the CHEWs to the Child-to-child, child-to-parent, positive deviant and 5x5x5 approaches X 15. Make contacts and begin initial planning with existing school health clubs in 33 in the project area X 16. Organize project slogan/rallying call contest & launch of child-to-child approach X 17. Continue with the c-to-c activities in schools; monitor them with teachers X Planned Activities Activity carried out Reasons why the activity was not Yes No carried out (X) (X) 18. Conduct and support health days – malaria field days, World AIDS day X 19. Orientation of CORPs to the Parent-to￾Parent BCC approach X Due to inadequate funding, the project did not manage to support the father-to-father groups but only supported the MtM groups 20. CHEWs orient CORPs on the PD approach X 21. CORPs formulate/identify father-to￾father clubs & mother-to-mother clubs and identify positive deviants X Only MtM clubs formed, not able to form father to father clubs due to inadequate funding. 22. Revise existing guides and checklist X 23. Follow up training on facilitative supervision X X 24. Explore gaps and opportunities within the existing Community HIS X 25. Develop/adopt community HIS X 26. Procure materials for HIS X 27. Train CORPs on CBHMIS X 28. Pilot CBHMIS data collection X 29. Roll-out community HIS data collection X    30. Train TOT in anti-malarial treatment X    31. Conduct Focused ANC training for DHMTs, HWs, and project staff X    32. PMTCT training for project staff, DHMT and HF workers X    33. Train 40 CORPs in supporting PMTCT X    34. Cascade training of CORPs on supporting PMTCT X 35. Quality assurance workshop for project staff, DHMT and HF workers X 36. Work with the KPA and the CSHGP to design MAMAN X The MAMAN OR study was changed to the CBMN OR protocol for the latter was developed with the technical inputs of the AMREF Ethics and Scientific Research Committee and not the KPA Planned Activities Activity carried out Reasons why the activity was not Yes No carried out (X) (X) 37. Implement MAMAN X The implementation of MAMAN which was named CBMNC OR was started in the 4th financial year. The design of MAMAN was changed to be in line with the MOPHS community strategy. 38. Identify domiciliary midwives X 39. Train DHMTs, FH staff project staff and domiciliary mid-wives on EOC X Due to inadequate budget, the training of domiciliary mid-wives was not undertaken by the project, hence selection not done. 40. Train CORPs in EOC X Not done because of inadequate budget. 41. Cross-visits between health facilities X 42. Train 250 shopkeepers in malaria, home treatment and appropriate drug use X Malaria treatment protocol was changed by the MOH, and “at the counter drugs” that were managed by the shopkeepers removed. So training of shopkeepers not necessary. 43. Conduct refresher training to shopkeepers X Reason as above 44. Conduct an IMCI Trainers training for DHMTs and HF workers X    45. IMCI case management course for HF workers X    46. Train 40 CORPs in malaria prevention, home treatment, drug use and ITN treatment X 47. Conduct training of trainers CIMCI dialogue approach X The project budget could not adequately support the C-IMCI component, so training of trainers on C-IMCI was not done. 48. CIMCI training on dialogue approach for 240 CORPs X Reason as above 49. Prepare and submit annual reports X 50. Hold annual project review with partners X 51. Conduct annual KPC survey, IHFA & Qualitative research X Planned Activities Activity carried out Reasons why the activity was not Yes No carried out (X) (X) 52. Hold planning meeting with HF staff and health committees X X 53. Hold planning and review meetings with the PIC X    54. Technical Advisory Group (TAG) meetings X    55. Facilitative supervision visits X    56. Monitoring and evaluation X    57. Regular updates to the DDC by AMREF X    58. Documentation and dissemination of lessons learnt and best practices X    Planned activities and achievements for the financial year 3 (Oct 2007 – Sept 2008) 1. Procure and distribute ITNs and HIV test kits X X 2. Design prioritized OR protocols X 3. Implement OR studies X 4. Distribute ITNs/LLITNs X 5. Launch four (4) COE X 6. Implement stages 4 and 5 of PDQ X Four stages of PDQ only fully implemented In one (1) facility – Bumala “B”. Funds not enough to allow completion of the four stages in all facilities 7. Undertake quality of care workshop for DHMT and HF staff X 8. Carry out advocacy to influence police and practice X 9. Produce branded T-shirts with BCC messages for CHWs X 10. Carry out the C-to-C, C-to-P, 5x5x5 and positive deviant activities X 11. Undertake contest of the project slogan and rallying call X 12. Support DHMT to undertake the national health days X 13. CHWs formulate/identify m-to-m and father-to-father groups X No father to father groups Planned Activities Activity carried out Reasons why the activity was not Yes No carried out (X) (X) 14. CHWs oriented on the 5x5x5 approaches X 15. Disseminate health messages via radio X This was not done because buying airtime was too expensive for the project to afford. 16. CHWs implement the 5x5x5 on key messages X 17. Procure materials for C-HMIS (registers and chalk boards) X 18. Train CHWs on CBHMIS X 19. Roll-out CBHMIS X 20. Conduct cascade training of CHWs on MNC X    21. Train CHWs on PMTCT X 22. Orientation of TBAs of their new roles on MNC X Project budget was inadequate and could not allow the training of TBAs. 23. Train 120 CHWs on EOC X X EOC training for CHWs was undertaken during the MNC trainings and not done separately. 24. Prepare annual and progress reports X 25. Hold an annual planning and review meeting with partners X 26. Conduct midterm review X 27. Refine the DIP based on the MTE findings X 28. Project Implementation Committee (PIC) meetings X 29. Conduct routine monitoring X 30. Regular update of the DDC on the project progress by AMREF X    31. Organize and hold Technical Advisory Group (TAG) meetings X 32. Develop a phase out plan X    Planned activities and achievements for the financial year 4 (Oct 2008 – Sept 2009) 1. Purchase and distribute HIV test kits to Health Facilities X 2. Conduct c-to-c activities X 3. Procure and distribute ITNs to targeted groups X Planned Activities Activity carried out Reasons why the activity was not Yes No carried out (X) (X) 4. Support DHMTs to in conducting national health days-malaria day, World AIDS day etc X 5. Carry out dissemination of health messages via radio X The dissemination of messages via radio was very expensive and hence unaffordable by the project. 6. Under carry the 5x5x5 activities X 7. Produce relevant IEC/ Health Learning materials X Apart from the T-shirts which had health messages and also served as IEC materials the project adopted other health learning materials developed by the MOH 8. Train CHEWs and CHWs on MNC X 9. Train CHWs on C-IMCI X As explained elsewhere in this report, C-IMCI as not carried out by the project due to inadequate budget 10. Follow up the CHWs on CIMCI X Not done because C-IMCI was not covered. 11. Train shopkeepers on ration drug use X Explained elsewhere in this report 12. EOC training for domiciliary mid-wives, and refresher for HF workers X Community or domiciliary were not trained by the project due to inadequate funding 13. Retraining of HWs on FANC X    14. Facilitative supervisory visits X    15. Annual Health Facility Assessment X    16. Conduct the CBMNC Operations Research X 17. Support CBHMIS X 18. HIS data collection X    19. Conduct continuous M&E X 20. Prepare annual progress reports X 21. Hold project review meetings X 22. Submit annual reports to the CSHGP X 23. Conduct KPC and IHFA surveys X 24. Hold Project Implementation Committee meetings X 25. Hold Technical Advisory Group (TAG) meeting X Planned Activities Activity carried out Reasons why the activity was not Yes No carried out (X) (X) Planned activities and achievements for the financial year 5 (Oct 2009 – Sept 2010) 1. Carry out BCC campaigns -5x5x5, M-to￾M, child to child X 2. Purchase “T” shirts (Incentives) for CHWs X 3. Hold monthly planning and review meetings with CHWs & CHCs X 4. Hold quarterly community dialogue meetings X 5. Household visitations by CHWs X 6. Purchase and distribute HIV test kits to health facilities in project sites X 7. Support DHMTs to conduct facilitative supervision to health facilities X 8. Re-train CHWs on Health Management Information System (HMIS) X 9. Conduct a KPC survey for the OR and end of project evaluation X 10. Conduct continuous monitoring and evaluation of the project activities X 11. Conduct end of project evaluation X 12. Hold a stakeholders annual review meeting X 13. Planning and review meetings with the project implementation team X 14. Participate in the child survival stakeholders meeting X 15. Participate in the child survival PVO meeting X 16. Conduct a stakeholders’ meeting to develop a phase-out and sustainability plan X 17. Disseminate EOP evaluation findings to partners/stakeholders X Dissemination of the EOP evaluation report not yet done as the report is still being finalized. 18. Disseminate OR findings to partners/stakeholders X The MNC OR report is still being compiled and so dissemination is yet to be done. 19. Hand over the project activities to the MOPHS X Project Results Framework IR1 Mothers’ knowledge on the importance of delivery by skilled health worker increased IR4 ANC services usage by mothers increased Strategic Objective 3 Increased proportion of women who deliver at a health facility IR1 Mothers’ knowledge on the importance of delivery at a health facility increased IR3 Provider/ client relationship improved IR4 CORPs with a functioning emergency transport system increased IR2 Health workers’ delivery skills increased Strategic Objective 4 Improved quality of and access to basic Emergency Obstetric Care at health facilities IR3 Health facilities offering basic EmOC increased IR1 Health workers skills in EmOC Increased MATERNAL & NEWBORN CARE Strategic Objective 1 Increased proportion of women who attend antenatal clinic at least 4 times and postnatal IR1 Mothers’ knowledge on the importance of ANC and PPC increased IR3 Improved quality of ANC and PPC services at health facility IR2 Mothers’ access to ANC & PPC services increased IR 2 Community access to basic EmOC increased Strategic Objective 2 Increased proportion of women who delivered under supervision of a skilled health professional MALARIA Strategic Objective 8 Increased proportion of pregnant women receiving IPT Strategic Objective 7 Improved case management of malaria among CU5 at health facilities IR1 Health workers skills in IMCI improved Strategic Objective 5 Improved knowledge and practice of malaria prevention and treatment at household and community level IR1 Community’s access to information on malaria prevention and treatment increased IR2 Skills on management of malaria at the community increased IR4 Shopkeepers’ skills on dispensing of anti-malarials improved IR3 Community’s access to prevention and treatment supplies increased Strategic Objective 6 Increased proportion of women and children under five who sleep under insecticide￾treated nets IR1 Mothers’ access to ANC services increased IR1 Access to ITNs by women and children increased IR2 Mothers’ knowledge on importance of IPT is increased IR2 Access to re￾treatment kits by households owning nets is increased IR3 Community members’ knowledge on importance of ITNs is increased HIV/AIDS Strategic Objective 9 Increased access to HIV counseling and testing among pregnant women at ANC. IR2 Health facilities offering HIV counseling and testing increased IR1 Health workers skills on HIV counseling and testing IR3 Mothers’ knowledge on importance of knowing HIV status is increased Strategic Objective 10 Increased knowledge and understandin g of PMTCT and ART among women of rep roductive IR1 Mothers’ attendance at ANC is increas e d IR2 Skills in PMTCT support increased IR3 Increased community education on PMTCT Strategic Objective 11 Increased # of HIV+ pregnant women who receive PMTCT IR1 Health facilities offering PMTCT increased IR2 Health workers’ skills in PMTCT increased IR3 Mothers’ knowledge on importance of PMTCT increased IR1 Mothers/fathers knowledge on importance of exclusive breastfeeding is increased IR2 Fathers’ knowledge on nutrition of pregnant women and infants is increased Mothers/ Strategic Objective 12 Improved feeding practices among caretakers of children 0-5 months Annex 5. Rapid CATCH Table Sample Type: LQAS Indicator Numerator Denominator Percentage Confidence Interval Percentage of children age 0-23 months who are underweight (-2 SD from the median weight-for￾age, according to the WHO/NCHS reference population) 0 0 0.0% 0.0 Percentage of children age 0-23 months who were born at least 24 months after the previous surviving child 73 133 54.9% 8.5 Percentage of children age 0-23 months whose births were attended by skilled health personnel 75 134 56.0% 8.4 Percentage of mothers of children age 0-23 months who received at least two tetanus toxoid injections before the birth of their youngest child 57 106 53.8% 9.5 Percentage of infants age 0-5 months who were exclusively breastfed in the last 24 hours 70 127 55.1% 8.7 Percentage of infants age 6-9 months receiving breastmilk and complementary foods 0 0 0.0% 0.0 Percentage of children age 12-23 months who are fully vaccinated (against the five vaccine￾preventable diseases) before the first birthday 0 0 0.0% 0.0 Percentage of children age 12-23 months who received a measles vaccine 0 0 0.0% 0.0 Percentage of children age 0-23 months who slept under an insecticide-treated bednet the previous night (in malaria-risk areas only) 124 133 93.2% 4.3 Percentage of mothers who know at least two signs of childhood illness that indicate the need for treatment 122 133 91.7% 4.7 Percentage of sick children age 0-23 months who received increased fluids and continued feeding during an illness in the past two weeks 0 0 0.0% 0.0 Percentage of mothers of children age 0-23 months who cite at least two known ways of reducing the risk of HIV infection 73 133 54.9% 8.5 Percentage of mothers of children age 0-23 months who wash their hands with soap/ash before food preparation, before feeding children, after defecation, and after attending to a child who has defecated 0 0 0.0% 0.0 Annex 6: Final KPC Report Report on the Final Knowledge, Practice, and Coverage Survey of the Busia Child Survival Project (BCSP): Annex 6 Busia and Samia Districts, Kenya October 2005 – September 2010 USAID/HIDN/CSHGP Cooperative Agreement Number: GHS-A-00-05-00009-00 Date of Submission: July 28, 2010 Table of Contents ACRONYMS ................................................................................................................................................................ 2 1 EXECUTIVE SUMMARY ................................................................................................................................ 3 2 BACKGROUND ................................................................................................................................................ 4 3 METHODS ......................................................................................................................................................... 8 4 RESULTS ......................................................................................................................................................... 15 5 DISCUSSION ................................................................................................................................................... 24 REPORT ON THE FINAL EVALUATION KNOWLEDGE, PRACTICES AND COVERAGE SURVEY July 2010 ACKNOWLEDGMENTS The KPC Survey was carried out by AMREF Kenya in collaboration with DHMTs Busia, Butula and Samia. We sincerely thank all who contributed to the successful completion of the survey. The quality of the results of the survey was due to the dedicated work of the Core team members. AMREF Kenya gratefully acknowledges the diverse roles that the following members of the team played: Mr James Kuya (MoH-Busia) and Mr. Emmanuel Luvai (MoH-Samia). We further acknowledge the roles played by the field and data entry staff during the KPC survey. The field supervisors were: John Mwengeh (MoH-Butula), Gladys Mumia (MoH-Samia), Everlyne Walela (MoH-Butula), Job Ambale (MoH-Butula), Rebacca Matalanga (MoH-Butula), Maurice Makokha (MoH-Samia) and Betty Sawenja (MoH-Samia). The interviewers were: Adolf Musindalo, Judith Adhiambo, Georgina Odanga, Winfred Halonyere, Oscar Muganda, Aisha Ali, Leah Oboya, Fidel Bwire, Winnie Obanda, Kennedy Alwanga, Regina Onguba, Phanice Mangeni, Isaac Olunga, Isaiah Akuku. The data entry clerks were Isaiah Osotsi and Dorothy Achieng. The diversity of roles played by AMREF-BCSP project staff and AMREF KCO staff is highly appreciated. 1 ACRONYMS AIDS Acquired Immune Deficiency Syndrome AMREF African Medical and Research Foundation ANC Antenatal Care ART Anti Retroviral Therapy BCSP Busia Child Survival Project CATCH Core Assessment Tool for Child Health CSTS+ Child Survival Technical Support Plus DHMT District Health Management Team DHRIO District Health Records and Information Officer DMOH District Medical Officer of Health DIP Detailed Implementation Plan EmOC Emergency Obstetric Care EOP End of Project EPI Expanded Programme on Immunization HIV Human Immune Deficiency Virus IMCI Integrated Management of Child Illnesses IPT Intermittent Preventive Treatment ITN Insecticide Treated Nets KDHS Kenya Demographic and Health Survey KNBS Kenya National Bureau of Statistics KPC Knowledge, Practice and Coverage LLIN Long Lasting Insecticidal Nets LQAS Lot Quality Assurance Sampling MDG Millenium Development Goal MoH Ministry of Health MoMS Ministry of Medical Services MoPHS Ministry of Public Health and sanitation MSF Medecins Sans Frontieres (Spain) MTCT Mother to Child Transmission NGO Non Governmental Organization PMTCT Prevention of Mother to Child Transmission PPC Post Partum Care RH Reproductive Health SP Sulfadoxine-Pyrimethamine TBA Traditional Birth Attendants USAID United States Agency for International Development WHO World Health Organization WRA Women of Reproductive Age 2 1 EXECUTIVE SUMMARY Introduction: The KPC Final Evaluation survey was conducted in Butula and Funyula divisions of Butula and Samia districts respectively in Western Kenya. Funyula and Butula divisions form the catchment area for the 5-year (October 2005 to September 2010) USAID funded Busia Child Survival Project. The project is aligned with Millennium Development Goals (MDGs) 4 and 5. It targets approximately 38,000 infants and children 0-59 months (CU5) and 42,000 women of reproductive age (WRA). Under five, infant and neonatal mortality rates for western province are 121/1000, 65/1000 and 4/1000 respectively, while the maternal mortality ratio is 488/100,000 live births. The project’s intention was to sustainably reduce child and maternal mortality by focusing on maternal and newborn care, malaria control, and HIV/AIDS. Project objectives under Maternal and Newborn Care have been to: increase proportion of pregnant women who attend antenatal clinic at least four times during pregnancy and postnatal clinic at least once within two days of delivery; increased proportion of women attended by a skilled health professional during delivery; increased proportion of complicated deliveries managed at health facilities; and improved quality of Emergency Obstetric Care (EmOC) at health facilities. Project objectives under Malaria are: improve knowledge and practice of malaria prevention and treatment at household and community level; increase proportion of pregnant women and CU5 who sleep under insecticide-treated nets; improve case management of fever and presumed malaria among CU5 at health facilities; and increase percentage of pregnant women who receive Intermittent Preventive Therapy (IPT). Project objectives under HIV/AIDS are: increase knowledge and understanding of PMTCT and ART among women of reproductive age (15-49 years); increase access to HIV counseling and testing among pregnant women at ANC; increase number of HIV+ pregnant women and newborns who receive PMTCT and HIV/AIDS care and treatment; improved feeding practices among caretakers of infants 0-5 months of age. Objectives: The objectives of the KPC survey were: to estimate progress in achieving end of project achievements on critical child health indicators in Butula and Funyula divisions. Methods: The KPC questionnaire was adapted from the KPC baseline survey questionnaires. The survey targeted mothers with children 0 – 23 months. A sampling frame was developed based on 7 project supervision areas, and list of villages and households maintained by the Ministry of Planning from the 1999 population census. Supervisors and interviewers were trained on LQAS methodology. This methodology was used to identify at least 19 interview locations in each lot (supervision area), select first households, and select respondents within households. Parallel-sampling was used to ensure adequate sub-sample sizes. An overall sample size of 134 was achieved for all project indicators. Elaborate quality control procedures were applied during data collection and entry. Data was analyzed using SPSS version 16 for windows after completion of data entry and cleaning. Results: Key findings from this survey are: Maternal and Newborn Care: about one-half (49%) of mothers attend ANC at least 4 times in line with focused antenatal care approach, and 58% utilize post-natal care within 2 days in line with targeted post partum care approach; similarly, slightly more than one-half (56%) of the 3 mothers are delivered by skilled health professionals in line with the current safe motherhood policy, and almost the same proportion (53%) deliver in a health facility. Malaria: Although most (95%) of the mothers know that ITNs are a method of preventing malaria, only 14% know at least 2 ways of malaria prevention; that is ITN and IPT. Notably, a about one-third (38%) of children are referred to a health facility within 24 hours after onset of fever. Almost all households (95%) have at least one insecticide treated net, and 93% of the mothers and children reported having utilized the net the previous night. Regarding prevention of malaria during pregnancy, slightly more than one-half (57%) of the mothers receive at least 2 doses of SP. Further, (92%) of mothers know at least 2 signs of childhood illness that indicate the need for treatment. However, 38% of the children were taken to health facility or community health worker within 24 hours after onset of fever. HIV/AIDS: majority (84%) of the mothers cited at least 2 ways of preventing MTCT, and only one-quarter (24%) know that the risk of MTCT can be reduced by use of ARV drugs. Almost all (85%) of the mothers are counseled and tested for HIV at ANC know their HIV status. Breastfeeding: almost one-half (52%) of the mothers breastfeed their babies exclusively for six months. 2 BACKGROUND A. Project Location and Background of the Area Location, administrative divisions, area and population Butula and Samia are two of the twenty-eight districts in the Western Province of Kenya bordering Uganda. The population of Butula and Samia is estimated at 214,754. The districts have two administrative divisions: Butula and Funyula which form the project catchment area (KNBS, 2007). The two divisions together contain 13 locations, 49 sub- locations and 312 villages. They occupy an area of 245.2 and 281.2 Sq Km for Butula and Funyula respectively (Ministry of Finance and Planning, 2002). Other key information Crude birth rate for the two districts is 47/1000, while crude death rate is 20/1000 (Kenya National Bureau of Statistics, 2009). The population is growing at 3.40% per annum (ibid). Life expectancy for females is 46 years, and for males 40 years (ibid). Under five, infant and neonatal mortality rates are 121/1000, 65/1000 and 24/1000 respectively (KDHS, 2008). The maternal mortality ratio is 370/100,000 live births, lower than the national average of 488/100,000 (ibid). B. Characteristics of the Target Beneficiary Population The direct beneficiaries of the project include approximately 38,000 infants and children 0-59 months (CU5) and 42,000 women of reproductive age (WRA). Luhya comprises the main ethnic group in the project catchment area, with few Luo. 4 C. Social, Economic and Health Conditions within the Project Area Literacy The literacy level for the district by sex is 76% for males and 55.3% for females. The primary school enrolment rates are 92% for boys and 91% for girls respectively. Drop out rates are 10% for boys and 12% for girls (Ministry of Finance and Planning, 2002). Economic conditions Busia is the 67th poorest of Kenya’s 71 districts (original district before the subdivision in 2007). Sixty six percent of the population earns less than US $1 per day (compared to 56% nationally). On average, households earn Ksh 5,149 per month. The main economic activity is trading with neighbouring Uganda, with Busia town – being a cross border centre. Away from busia town, the rural economy is heavily reliant on fishing, and agriculture, with cassava, millet, sweet potatoes, beans, maize and sugar cane being the principal cash crops. The main causes of poverty include: lack of markets for farm produce (mainly sugar cane) and fish, and poor communication and transport infrastructure. Butula and Samia have only one major road, a section of the Trans-African Highway connecting Mombasa and Lagos. This road is a factor in HIV transmission because of the long-haul truckers and the commercial sex workers they patronize. Poor health is itself a major contributing factor to poverty in the district. High rates of malaria, TB, HIV/AIDS and childhood illnesses cause people to lose many workdays, and the cost of treatment has a significant impact on already meager family incomes. Causes of under- five and Maternal Mortality The main causes of mortality in Busia district hospital which is the main referral facility in the project area are malaria (28%), HIV/AIDS (12%), anemia (14%), pneumonia (7%), Protein￾Energy Malnutrition (5%), TB (3.3%). They are followed by injuries, RTAs, gastroenteritis and neonatal sepsis (Busia District Health Report, 2009). In 2008, the HIV prevalence among ANC clients was 6%. Health Care Delivery in the area There are 26 health facilities in the project catchment area that are either private, mission or GOK facilities. In Butula, there is 1 district hospital (Khunyangu), 3 health centres, 3 private clinics, and 8 dispensaries. In Funyula, there are 2 hospitals; a district hospital (sio-port) and a mission hospital (Nangina), 5 private clinics and 7 dispensaries. Average distance to the nearest health facility is 4km (Ministry of Finance and Planning, 2002) and the doctor – patient ratio is 1:41,200. D. National Standards/Policies Regarding Maternal and Child Health Maternal and childcare services are delivered in line with Ministry of Public Health and Sanitation (MOPHS) standards and policies. Key existing national standards are: FP (Family Planning); FANC (Focused Antenatal Care) and Malaria in Pregnancy; PMTCT (Prevention of Mother to Child Transmission of HIV); EOC (essential obstetric care); targeted post-partum care; IMCI (Integrated Management of Child hood Illnesses); and EPI (Expanded Programme on Immunization); newborn care, pediatric psychosocial counseling. 5 E. Overview of the Project: Partners, goals, objectives, and strategies The primary implementing partners for this project are AMREF Kenya and District Health Management Teams (DHMT) for Busia, Butula and Samia. History of AMREF in Kenya AMREF’s vision is better health for Africa. Founded in 1957, it is now Africa’s largest health development NGO. AMREF comprises a headquarters in Nairobi, Kenya; country program offices in Kenya, Ethiopia, Uganda, Tanzania, and South Africa; additional program sites in Southern Sudan and Somalia; and 12 national offices in Western Europe, the United States, and Canada. AMREF has more than 40 years’ experience in community-based health care, and currently implements more than 100 projects in seven countries in sub-Saharan Africa. The cross-cutting themes in all these projects are: capacity strengthening, community partnering and health system research. AMREF has experience in planning and implementation of child survival programs. Between 1987 and 1995, AMREF, in partnership with the Nairobi City Council, successfully implemented a USAID-funded child survival program in the slums of Nairobi. Between 1992 and 1999, AMREF in collaboration with MoH Uganda successfully implemented a child survival program in Luweero and Nakasongola Districts, Uganda. Butula and Samia DHMTs Originally the project worked under one DHMT before the larger Busia district was split into four districts (Budalangi, Samia, Butula and Busia) by late 2008. The project is currently working with the three DHMTs of Busia, Butula and Samia districts. The tthree DHMTs plan, implement and monitor the delivery of healthcare in their respective districts, and directly oversees management of most of the health facilities in Butula and Funyula divisions. The DHMTs are therefore strategically positioned to influence the health of women and children in the project area. The teams were selected as the partners to maximize the gains made by the project, and importantly, to ensure sustainability. To achieve these, the project has strengthened DHMT’s leadership and management, supervision, facilitation and M&E skills, enabling them to better fulfill their responsibilities and to make better use of outside assistance. Project Goal, Objectives and Strategies The project was destined to run for five years (October 2005 to September 2010) and it is in the last quarter of the fifth year (2010). It focuses on addressing Millennium Development Goals (MDGs) 4 and 5 of reducing child mortality and improving maternal health respectively. Through evidence-based approaches, the project strives to significantly lower child and maternal mortality by 2010, thus setting the stage for the achievement of the MDG targets of 2015. The project goal is a sustained reduction in child and maternal mortality in the two divisions. The project has a focus on maternal and newborn care, malaria control, and HIV/AIDS, at 40%, 40%, and 20% levels of effort respectively. 6 The project applies 3 strategic approaches towards the achievement of these goals: (i) Capacity Building (ii) Quality Assurance and (iii) Behaviour Change and Communication. F. Objectives of the KPC Survey KPC survey main objectives were as follows: ‚ Assess progress towards achievement of project objectives (9) and end of project targets ‚ To obtain information on caretaker knowledge and practices with regard to maternal and newborn care, breastfeeding and nutrition, malaria prevention, integrated management of childhood illnesses and PMTCT. ‚ To build the capacity of field supervisors and interviewers to collect KPC data using LQAS (Lot Quality Assurance Sampling) and interpret the findings 7 3. METHODS A. Questionnaire Development KPC 2000+ The project adopted the KPC questionnaires used during the baseline survey; questions on the anthropometric measures and water and sanitation were excluded from the baseline questionnaire. Because of parallel sampling between the categories of mothers of children 0-23 months, mothers of children 0-5 months and children in whom fever/malaria manifested in the past 2 weeks, three sets of questionnaires namely; tool A, tool B and tool C were developed to be administered to each of the three categories. Tool A included 41 questions and was used on the initial qualifying respondent in each selected village. Questionnaire B had questions from Questionnaire A on respondent background information, breastfeeding, and postpartum care. It was only to be used with a mother of an infant under six months of age if the initial qualifying respondent’s child selected was between the ages of 6 and 23 months. Questionnaire C had questions from Questionnaire A on respondent background information and integrated management of childhood illnesses. It was only used on mothers of children 0-23 months if the initial qualifying respondent’s child in the selected village had not experienced fever or malaria in the past 2 weeks preceding the interview. The Rapid CATCH questions were used as the foundation of the KPC questionnaire. Questions from KPC 2000+ modules, from the Child Survival Technical Support Plus (CSTS+) Unit at MACRO were included to suit the specific project objectives and indicators: Respondent Background Information (Survey questions 1-8); Maternal Newborn Care (Survey questions 9- 24); Breastfeeding and Nutrition (Survey questions 25-27); Malaria prevention (Survey questions 28-30); Integrated Management of Childhood Illness (IMCI) (Survey questions 31-34) and PMTCT (Survey questions 35-41). B. KPC Indicators The objectives and indicators for the project measured in this KPC survey are outlined below (Table 1.3.1). Table 1.3.1: Objectives and indicators Objectives Indicator Definition of Indicator Intervention 1: Maternal and Newborn Care 1.1 Increased proportion of women who attend antenatal clinic at least 4 times and postnatal clinic at least once % of mothers of children 0- 23 months who attend ANC at least four times during most recent pregnancy Mothers who mentioned attending ANC at least 4 times for Question # 10 and Question # 11(Tool A)/Total mothers answering # 10 and # 11. % of mothers of infants 0-5 months who attend postnatal care within two days of delivery Mothers who responded to 1 or 2 for question #22 tool A and Question # 10 tool B/Total mothers answering # 22 tool A and # 10 tool B 8 Objectives Indicator Definition of Indicator 1.2 Increased proportion of women who delivered under supervision of a skilled health professional % of children 0-23 months whose delivery was attended by skilled health professional (nurses with midwifery training, doctors, midwives) Mothers who mentioned A, B or C for question # 18 tool A/Total mothers answering # 18 tool A 1.3 Increased proportion of women who deliver at a health facility % of mothers of children 0- 23 months who deliver at health facility Mothers who responded to 3,4,5,6,7,8 or 9 for question # 17 tool A/Total mothers answering # 17 tool A Intervention 2: Malaria 2.1 Improved knowledge and practice of malaria prevention and treatment at household and community level % of mothers of children 0- 23 months who know at least 2 ways to prevent malaria Mothers who know at least two (2) correct responses (Correct = I, 2 or 3) for Question #28/Total mothers answering #28. % of children 0-23 months referred to HF within 24 hours after onset of fever Mothers who mentioned 1 (same day) to Question # 34 tool A and had mentioned 3 (seen by a worker at health facility to question # 33 plus those who mentioned 1 (same day) to Question # 12 tool C and had mentioned 3 (seen by a worker at health facility to question # 11 tool C/Total mothers answering 6 (fever) or 7 (malaria) to Question #32 tool A and Question # 10 tool C 2.2 Increased proportion of women and children under five who sleep under insecticide-treated nets % of households with at least one ITN Mothers who responded to 1 for Question # 29 (a) in tool A/Total mothers answering # 29 (a) % of mothers of children 0- 23 months who slept under ITNs the previous night Mothers who responded to B for Question # 30 in tool A/Total mothers answering # 29 (i) in tool A % of children 0-23 months who slept under ITNs the previous night Mothers who responded to A for Question # 30 in tool A/Total mothers answering # 29 (i) (interviewed) in tool A 2.4 Increased proportion of pregnant women receiving IPT % of mothers of children 0- 23 months who received at least 2 doses of SP for IPT during ANC. Mothers who mentioned receiving at least 2 doses of SP for # 16 (c) tool A/Total mothers answering # 16a (interviewed) in tool A. Intervention 3: HIV/AIDS 9 Objectives Indicator Definition of Indicator 3.1 Increased knowledge and understanding of PMTCT and ART among women of reproductive age (15-49 years) % of mothers of children 0- 23 months who cite at least two ways of preventing MTCT Mothers who mentioned at least 2 of 1,3 and 6 for question # 37 tool A/Total mothers answering # 37 tool A % of mothers of children 0- 23 months who know that risk of MTCT can be reduced by ART Mothers who mentioned 1 (take antiretroviral drugs –ARV) for question # 37 tool A/Total mothers answering # 37 tool A 3.2 Increased access to HIV counseling and testing among pregnant women at ANC. % of mothers of children 0- 23 months counseled and tested for HIV at ANC during their most recent pregnancy Mothers who responded to 1 for Question # 40 in tool A/Total mothers interviewed using tool A % of mothers of children 0- 23 months who know their HIV status Mothers who responded to 1 for Question # 41 in tool A/Total mothers interviewed using tool A 3.4 Improved feeding practices among caretakers of children 0-5 months % of children age 0-5 months who were exclusively breastfed during the last 24 hours Mothers of children 0-5 months who do not state responses B, C, D or E and state response A for question #27 tool A and Question # 15 tool B/Total mothers answering # 27 tool A and # 15 tool B C. Sampling Design Target population and choice of sampling method The survey targeted mothers of children 0 – 23 months. The LQAS (Lot Quality Assurance Sampling) methodology used during the baseline was similarly used during the final evaluation for ease of comparison of the study results. Sampling frame The detailed sampling frame organized by division, location, sub-location and village was prepared from the list of households maintained by the Ministry of Planning and based on the 1999 population census. The list contains the villages with the respective numbers of households in the project area. The distribution of villages per supervision area and the respective number of households is shown in table 1.3.2 below. 10 Table 1.3.2: Number of villages and households Supervision Area Number of Villages Number of households 1 Supervision Area 1:Bujumba/Bumala 44 5,466 2 Supervision Area 2: Marachi Central 47 5,347 3 Supervision Area 3: Elugulu/Elukhari 45 5,534 4 Supervision Area 4: Marachi East 50 5,266 5 Supervision Area 5: Namboboto/Nambuku 55 4,498 6 Supervision Area 6: Nangosia/Odiado/Agenga 62 5,929 7 Supervision Area 7: Nanguba/Bwiri 57 5,198 Identifying interview locations The locations of interviews (villages) in each lot were selected based on the relative sizes (i.e. probability proportion to size of households in each village as per the 1999 census) of the villages. The sampling interval for each lot was determined by dividing the cumulative household population for the lot with 19. The location of the first interview in the lot was determined using a random number. Selecting households and respondents Survey supervisors were trained on how to select first households in a selected village. In each selected village, the supervisor with the assistance of the CHW determined the boundaries of the village and after agreeing with him/her regarding the boundaries of the village, compiled a list of all the households in that village. From this list, a household was randomly selected. The interviewer went to this house to determine if there was a respondent eligible for the study, if present, tool A was administered. If there was not, then the interviewer proceeded to the next closest household whose front door faces the one she/he had just left and repeated this process until she/he would get an eligible respondent. Use of parallel sampling technique If the respondent’s youngest child was under six months of age and had been sick with a fever within the preceding two weeks, then the interviewer had completed his/her work in that village and could proceed to the next selected village. If the child had not been sick or was not 0-5 months of age, the technique of parallel sampling was used to ensure enough children aged 0-5 months and children with fever/malaria in the preceding 2 weeks were enrolled. In order to achieve this, the interviewer would proceed to the next nearest household until an eligible interviewee could be found for tool B or tool C. In cases where the initial respondent’s child was not 0-5 months and had not experienced fever or malaria in the preceding 2 weeks, then the interviewer had to keep moving to the next nearest household until both tools B and C were separately completed. A total of 231 KPC questionnaires were administered, of which 134 were Questionnaire A, 58 were Questionnaire B, and 39 were Questionnaire C. 11 Sample size A minimum sample size of 19 per lot per indicator was used. With a sample size of 19 per lot (and therefore 133 for the project area), it was possible to calculate coverage in the project area with a precision of 10% at 95% confidence level. The project monitoring and evaluation officer determined the estimates of the confidence limits for the survey results manually. The technique of parallel sampling was used to ensure a denominator of 19 was achieved for all project indicators. D. KPC TRAINING KPC training curriculum The materials used to train the supervisors and interviewers included: KPC 2000+ - Field Guide; and KPC training Module 2 (Training Supervisors and Interviewers). Selection and orientation of Supervisors and interviewers Selection of supervisors was based on experience acquired either during the baseline, Rapid Catch indicators, MAMAN, second annual LQAS surveys or midterm evaluation. The supervisors and interviewers were trained on the questionnaires and survey methodology for 3 days and facilitated by BCSP M&E officer, the District Health Records and Information Officer (DHRIO-Busia) and District Public Health Officer (DPHO-Samia) who were also Core Team member. E. LOGISTICAL PREPARATIONS Scheduling, Supplies, printing, and copying Stationery for the entire final evaluation process were estimated and procured in April 2010 and the printing and photocopying of tools was done at the BCSP office. Transportation Plan Vehicles used were project vehicle (1) and hired taxis (7). A total of 8 vehicles were used. Each team (supervisor and 2 interviewers) had a vehicle. During the training, each of the survey teams prepared a survey itinerary indicating which villages they planned to go on each day. The itineraries were prepared such that after finishing the interview in one village, the team proceeded to the next nearest selected village. The itineraries were revised every evening during the debriefing sessions with the supervisors. F. DATA COLLECTION AND QUALITY CONTROL PROCEDURES Duration of survey/interview and challenges Data collection took 6 days, from 7th June to 14th June 2010. On average, tool A took half an hour to complete, while tools B and C took much less time. A number of challenges were 12 encountered during the KPC survey. These included cases where villages identified during the 1999 census had been subdivided and renamed. Quality control procedures in the field Every supervisor observed at least one interview per interviewer each day and completed the KPC Quality Control Checklist and gave feedback to the interviewer immediately after the interview. The supervisor also checked each completed questionnaire for errors in the field and followed up with the interviewer for correction of any identified errors. Quality Control during Data Entry Data was entered by two data entry clerks who had been recruited based on their previous experience with project KPC surveys using SPSS for data entry and analysis. They also underwent the supervisors/interviewers training to familiarize with the tools and the entire survey process. The M&E officer supervised data entry. He ensured that the data entry clerks had minimum distraction during the data entry period and the working environment was conducive for their work. He was consulted for errors encountered in the process of data entry and acted as a link between the field personnel and data entry clerks. He ensured that feedback was given to the field personnel (during evening debriefing sessions) whenever errors were identified during data entry. Quality of the data entry process was monitored by randomly checking a sample of entered records to ensure that the data was entered accurately. Double entry was used to ensure correct entry of data. Frequencies of all the fields were run to look for outliers (results that are very high or very low or unexpected that might indicate a possible error) and also to ensure that the correct denominators for all indicators were captured. He ensured that the data entry clerk adhered to data back-up protocols. The analysis program was tested using the questionnaires completed on the first day. This was to check if the validation rules and the CHK programs were properly working and the questions in the questionnaire were consistent with the designed questionnaire structure. Data was cleaned by running frequencies to identify any inconsistencies and missing entries. Records that had incorrect entries were confirmed from the relevant questionnaire and corrections made to reflect questionnaire entries. Questionnaires that had no responses checked (entered) were taken back to the supervisors for completion. Data cleaning also involved running frequencies to establish if the denominator for each question was attained. If not attained related questions were sorted and their entries compared to identify the inconsistency. Visual scanning of all records in the database was done to identify errors. The common problems encountered during data cleaning included missing values, incorrect denominators for respective fields and incorrect entries. Data Analysis Data was analyzed using SPSS version 16 for Windows. Based on each indicator definition (the specific questions in the tools representing various fields), frequency and cross tabulations were 13 14 run to get both the numerator and denominator values of each indicators. The results were the displayed in a tabular format to compare with the baseline and end of project targets. Dissemination The KPC preliminary results were represented to the final evaluation team in preparation for the qualitative assessment and summary of the qualitative findings on the 12th July and 21st July respectively. The findings will be disseminated to stakeholders that include community members from Butula and Samia Districts, Ministry of Public Health and Sanitation (MoPHS) and Ministry of Medical Services (MoMS) officials at National, provincial and district levels; Project Implementation team members (DHMT Busia and Samia, MSF Spain and AMPATH), Child Survival PVOs (APHIA II Western and Healthright International), AMREF Kenya Country Office and Project staff. 4 RESULTS A. Indicators Project Indicators A denominator of 134 for each project indicator was achieved (Table 1.4.1). Table 1.4.1: Coverage levels and corresponding confidence intervals for project indicators Maternal and Newborn Care Indicators Numerator Denominator Proportion estimate 95% CI % of mothers of children 0-23 months who attend ANC at least four times during most recent pregnancy 65 134 48.9% 40.9% - 56.9% (±8%) % of mothers of infants 0-5 months who attend postnatal care within two days of delivery 79 134 58.1% 50.1% - 66.1% (±8%) % of children 0-23 months whose delivery was attended by a skilled health professional (nurses with midwifery training, doctors, midwives) 75 134 56.0% 48.0% - 64.0% (±8%) % of mothers of children 0-23 months who deliver at health facility 71 134 53.0% 45.0% - 61.0% (±8%) Malaria Numerator Denominator Proportion estimate 95% CI % of mothers of children 0 – 23 months who know 2 ways (ITN & IPT) to prevent malaria 19 134 14.2% 8.2% - 20.2% (±6%) % of children 0-23 months taken to HF or Community Health Worker within 24 hours after onset of fever 50 133 37.6% 29.6% - 45.6 (±8%) % of households with at least one ITN 127 134 94.8% 89.8% - 99.87% (±5%) % of mothers of children 0-23 months who slept under ITNs the previous night 124 133 93.2% 86.2% - 100.2% (±7%) % of children 0-23 months who slept under ITNs the previous night 123 133 92.5% 86.5% - 98.5% (±6%) 15 16 % of mothers of children 0-23 months who received at least 2 doses of SP for IPT during ANC. 76 133 57.1% 50.1% - 64.1% (±7%) HIV/AIDS Numerator Denominator Proportion estimate 95% CI % of mothers of children 0 – 23 months who cite at least two ways of preventing MTCT 112 133 84.2% 76.2% - 92.2% (±8%) % of mothers of children 0-23 months who know that risk of MTCT can be reduced by ART 32 133 24.1% 16.1% - 32.1% (±8%) % of mothers of children 0-23 months counseled and tested for HIV at ANC during their most recent pregnancy 113 133 85.0% 79.0% - 91.0%) (±6%) % of mothers of children 0-23 months who know their HIV status 108 134 80.6% 72.6% - 88.6% (±8%) % of children age 0-5 months who were exclusively breastfed during the last 24 hours 70 127 51.5% 44.5% - 58.5% (±7%) Rapid Catch Indicators Rapid Catch indicators that were not project indicators are included in table 1.4.2 below. Table 1.4.2: Coverage levels and corresponding confidence intervals for CATCH indicators that were not project indicators Other Rapid CATCH Indicator Table Numerator Denominator Proportion Sentinel Measure of Child Health and Well-being Percentage of children age 0-23 months who are underweight (-2 SD from the median weight-for age, according to the WHO/NCHS reference population) - - - Prevention of Illness/Death Percentage of children age 0-23 months who were born at least 24 months after the previous surviving child 73 133 54.9% Percentage of mothers with children age 0-23 months who received at least 2 tetanus toxoid injections before the birth of their youngest child 57 106 53.8% Percentage of mothers with children age 0-23 months who cite at least 2 known ways of reducing the risk of HIV infection 73 133 54.9% Percentage of mothers with children age 0-23 months who report that they wash their hands with soap/ash before food preparation, before feeding children, after defecation, and after attending to a child who has defecated - - - Management/Treatment of Illness Percentage of mothers of children age 0-23 months who know at least 2 signs of childhood illness that indicate the need for treatment 122 133 91.7% Percentage of sick children age 0-23 months who received increased fluids and continued feeding during an illness in the past two weeks - - - G. Social and Demographic Characteristics Age and sex distribution of children The distribution of the children by sex revealed that 50.7% (68/134) of the children were female and 49.3% (66/134) were male. The information collected during the survey pertained to the youngest child under the age of two years (Table 1.4.3). Table 1.4.3: Age distribution of respondents (Tool A) Age of the Youngest Child (Months) Frequency (N=134) Percent 0 – 5 80 59.1 6 - 11 26 19.4 12 – 23 28 20.9 Respondent characteristics The mean age of the mothers interviewed (Tool A) was 26 years. 95.5% (128/134) of mothers reported having attended school at some point. Of these, a majority had only attended primary school (76% or 102/134) and only a few had reached secondary school (18.7% or 25/134); only 1 (0.7%) had gone on to higher education. A clear majority of respondents were of the Luhya Tribe (82.2% or 111/134) and the rest were of the Luo Tribe (17.2% or 23/134). Table 1.4.4: Mothers’ School attendance Level of schooling reached Frequency Percent Never attended School 6 4.5 Reached Primary level 102 76.1 Reached Secondary level and above 26 19.4 Total 134 100.0 The proportion of women who have no outside work (78.4%) is greater than those who work outside the home to earn money (21.6%). Almost one-quarter (27.6%) of the mothers are in agriculture, 24.1% are selling foods and the rest are selling handicrafts, shopkeepers/tailors/hair dressers or salaried workers. Most mothers (85.1% or 114/134) are married, 12.7% or 17/134 are single and 2.2% (3/134) are either separated or widowed. For those who have spouses 85.8% said their husbands work while the rest their husbands do not work. C. Child spacing Mothers reported having an average of 1.77 children under the age of five. Sixty four percent (86/134) of the mothers reported having two or more children under the age of five. The distribution of the number of children per mother is as indicated in table 1.4.5 below. Table 1.4.5: Distribution of Children Under 5 Yrs per mother Number of Children Frequency Percent One Child 48 35.8 Two Children 69 51.5 Three Children 17 12.7 Total 134 100.0 18 Antenatal care Slightly more than half of mothers (57.5% or 77/134) had a maternal health card, and about one￾third (38.8%), had the card but was not available. Only 3% (4/134) indicated that they never had a card. 27.8% (37/133) had made at least four ANC visits during their most recent pregnancy (had mother child health card/book). An additional 28 mothers who did not have cards reported from memory having attended four or more ANC sessions, bringing the total by card and memory to 48.9% (65/133). By card and memory, 95.5% (127/133) had made at least one ANC visit. Eighty-two percent (111/134) of mothers said they had received at least one dose of tetanus toxoid during their most recent pregnancy and 36.6% (49/134) said they had received two or more doses. Delivery care Almost one-half of mothers delivered outside a health facility (47% or 63/134), with most of the deliveries (43.3% or 58/134) occurring at home. About two-thirds of health facility-based deliveries were done at MoH facilities (64.6% or 42/65) and the remainder were done at mission or other facilities. About one-quarter (27.6% or 37/134) of the deliveries were attended by traditional birth attendants (TBA) (Figure 1.4.1). About a third (56% or 75/134) of deliveries were attended by a trained health professional (doctor, nurse, midwife or community midwife). Figure 1.4.1: Proportions of respondents attended by various attendants during delivery Traditional Birth Attendant, 27.6% Skilled Birth Attendant, 56.0% Other, 11.9% No One, 0.7% Postpartum Care Slightly more than one-half of the mothers (58.1%, or 79/134) of infants under six months of age reported having had their health checked by a trained health professional (doctor, nurse/midwife, community midwife) after the delivery. A similar proportion of the mothers said their child’s health was checked as well. 19 E. Breastfeeding and Nutrition, and Anthropometry Breastfeeding and Nutrition Nearly all of the mothers of infants under six months (98.5% or 135/137) reported having breastfed at some time. Sixty one (84/137) initiated breastfeeding within the first hour after delivery, and only 51.8% (71/137) exclusively breastfed their infants under six months of age. G. Malaria Prevention Forty percent (54/134) of mothers knew two or more ways to prevent malaria. Awareness of the importance of bed nets is especially high with 95% (127/134) of mothers responding positively. Ninety two percent (123/134) of households have at least one bed net. Fifty-five percent (73/134) of mothers report they have at least one long lasting insecticide treated net (LLIN, which lasts up to five years without requiring retreatment) and 40% (54/134) have at least one insecticide￾treated bed net (ITN, which requires retreatment every six months.) Fifty six percent (30/54) of the ITNs were reported to have been retreated within the previous six months. Ninety three percent of mothers (93% or 124/134) and (92% or 124134) of children slept under an ITN or LLIN the previous night. H. Integrated Management of Childhood Illness Knowledge of danger sings While the percentage of caregivers who knew two or more danger signs associated with childhood illness and the need for treatment was relatively high at 91% (122/134), awareness of some key danger signs, particularly those most closely associated with acute respiratory illnesses (ARI), was low, as noted in the table 1.4.6 below: Table 1.4.6: Proportions of mothers who knew danger signs associated with childhood illness Knowledge of Danger Signs Signaling the Need for Treatment Frequency (N=133) Percent High Fever 121 90% Looks Unwell or Not Playing Normally 63 47% Not Eating or Drinking 83 62% Vomits Everything 62 46% Abnormally sleepy 14 10% Diarrhea 40 30% Fast or Difficult Breathing 16 12% Convulsions 4 3% Do Not Know 1 1% Care Seeking for the Sick Child 20 21 Sixty-seven percent (90/134) of mothers reported that their youngest child had had a fever or presumed malaria within the previous two weeks. Slightly more than one-half (58% or 75/129) sought care at a health facility within 24 hours. I. HIV/AIDS Awareness of HIV/AIDS is high across the project area, with 99% (132/134) of mothers saying they had heard of AIDS; 71% (95/134) saying that it could be transferred during delivery and pregnancy; and 66% (89/134) saying that it could be transferred through breastfeeding. Sixty-one percent (82/134) of mothers knew at least one way to prevent the transfer of HIV to their child, and one-quarter (25% or 34/134) were able to list at least two ways to prevent MTCT. About one-quarter (24% or 32/134) of the mothers knew that the risk of MTCT can be reduced by ART. The proportions of the mothers who cited the various methods of preventing MTCT are outlined in table 1.4.7 below. Table 1.4.7: Proportions of mothers who mentioned the various methods of preventing MTCT Knowledge of Ways to Prevent MTCT Frequency Percent Be delivered by a skilled birth attendant. 61 46% Take ARV drugs. 32 24% Continue proper breastfeeding. 27 20% Maintain a healthy diet. 7 5% More than three quarters (85% or 114/134) of the mothers said they had been offered an HIV test as part of their most recent ANC. Similar number of mothers (84% or 112/133) was tested. Ninety-six percent (96/112) of the mothers tested were informed of the results of the test. L. Performance at End of Project Table 1.4.13: Significance Tests Indicator Baseline End of Project Is the change from Baseline to EOP statistically significant Maternal and Newborn Care Indicators Proportion estimate 95% CI Proportion estimate 95% CI % of mothers of children 0-23 months who attend ANC at least four times during most recent pregnancy 32% 24% - 40% (+ 8%) 49% 41% - 57% (+8%) Yes % of mothers of infants 0-5 months who attend postnatal care within two days of delivery 23% 16% - 30% (+ 7%) 58% 50% - 66% (+8%) Yes % of children 0-23 months whose delivery was attended by a skilled health professional (nurses with midwifery training, doctors, midwives) 26% 19% - 33% (+ 7%) 56% 48% - 64% (+8%) Yes % of mothers of children 0-23 months who deliver at health facility 20% 13% - 27% (+ 7%) 53% 45% - 61% (+8%) Yes Malaria Proportion estimate 95% CI Proportion estimate 95% CI Is the change from Baseline to EOP statistically significant I % of mothers of children 0 – 23 months who know 2 ways (ITN & IPT) to prevent malaria 17% 9% - 25% (+ 7%) 14% 8% - 20% (+6%) No % of children 0-23 months taken to HF or Community Health Worker within 24 hours after onset of fever 7% 3% - 11% (+ 4%) 38% 30% - 46% (+8%) Yes % of households with at least one ITN 77% 70% - 84% (+ 7%) 95% 90% - 100% (+5%) Yes % of mothers of children 0-23 months who slept under ITNs the previous night 65% 57% - 73% (+ 8%) 93% 86% - 100% (+7%) Yes % of children 0-23 months who slept under ITNs the previous night 70% 62% - 78% (+ 8%) 93% 87% - 99% (+6%) Yes 22 23 % of mothers of children 0-23 months who received at least 2 doses of SP for IPT during ANC. 21% 14% - 28% (+7%) 57% 50% - 64% (+7%) Yes HIV/AIDS Proportion estimate 95% CI Proportion estimate 95% CI Is the change from Baseline to EOP statistically significant % of mothers of children 0 – 23 months who cite at least two ways of preventing MTCT 23% 16% -30% (+ 7%) 84% 76% - 92% (+8%) Yes % of mothers of children 0-23 months who know that risk of MTCT can be reduced by ART 33% 25% - 41% (+8%) 24% 16% - 32% (+8%) No % of mothers of children 0-23 months counseled and tested for HIV at ANC during their most recent pregnancy 53% 45% - 61% (+ 8%) 85% 79% - 91%) (+6%) Yes % of mothers of children 0-23 months who know their HIV status 41% 32% - 49% (+ 8%) 81% 73% - 89% (+8%) Yes % of children age 0-5 months who were exclusively breastfed during the last 24 hours 11% 6% - 16% (+ 5%) 52% 45% - 49% (+7%) Yes 5 DISCUSSION A. Maternal Newborn Care Antenatal care The four ANC attendances increased among mothers of children 0 - 23 months from 32% at baseline to 49% at end of project. The change in the ANC attendance in the project area is statistically significant and surpassed the national ANC coverage of 47% (KDHS, 2008) though still falls below the national target of 80%. The end of project achievement reached the end of project target of 50%. Delivery Care The health facility deliveries at 53% are above the national average of 43% and double the average for western province at 25% (KDHS 2008). The change in the delivery in health facility from the baseline values of 26% is statistically significant and more so performed beyond the end term target of 35%. Postpartum Care The change in postpartum care within 2 days after delivery from 23% at baseline to 58% at end of project is statistically significant. The EOP achievement is 18% higher than the planned target of 40%. The postpartum attendance within 2 days of delivery is higher than the national and western province averages of 6.9% and 6.8% respectively (KDHS, 2008). B. Malaria Availability of ITNs Availability of at least any type of mosquito net in households is quite high (95%) in the project area, well beyond the national and western province averages of 61% and 74% respectively (KDHS, 2008). This is also beyond the national target of 60% (National Malaria Strategy: 2001- 2010). The 18% increase from the baseline value of 77% is statistically significant. Utilization of ITNs The project has surpassed the national target of the National Malaria Strategy 2001 – 2010 to have 60% of all babies sleeping under nets by 2010. The project coverage (93%) of children 0 – 23 months sleeping under LLIN/ITN is statistically significant and well above the end of year target of 80%. It is almost double the national average of 51% for children aged 2 years sleeping under ITN (KDHS, 2008) Intermittent preventive treatment of malaria 24 25 The increase in receiving two doses of IPT among pregnant women by 36% from the baseline figure of 21% is statistically significant despite being below the EOP target of 60%. The IPT coverage (2+ doses) in the project area is higher than the national coverage of 15% (KDHS, 2008). The knowledge of IPT as a malaria prevention method in pregnancy by mothers has never changed from the baseline value of 17%. Integrated Management of Childhood Illness There was a great leap in the proportion of children who sought health facility care within 24 hours when fever manifested. Though still below the EOP target of 60% the change from 7% at baseline to 38% at EOP was statistically significant. C. HIV/AIDS Awareness about PMTCT Knowledge on at least two ways of HIV transmission from mother to baby increased significantly. The 84% achievement at EOP is above the planned target of 59%. Utilization of PMTCT services KDHS (2008) findings indicate that 75% of the ANC mothers were tested for HIV. The national target is 90% (ibid). At 85%, the coverage in the project area is above the national coverage and the EOP of 70%. Feeding practices for children under 6 months The increased coverage to 52% of exclusive breastfeeding in the project area is above the national coverage of 32% (KDHS, 2009). The increase is statistically significant and falls above the EOP target of 40%. KPC Results (Baseline, Midterm and Final) Objective/Result Indicators (by technical or cross-cutting intervention) Baseline Value MIDTERM EOP Target Actual Target Actual 1. Increased proportion of women who attend antenatal clinic at least four times and postnatal clinic at least once % of mothers of children 0-23 months who attend ANC at least four times during most recent pregnancy 32% 46% 50.38% 50% 49% % of mothers of infants 0-5 months who attend postnatal care within two days of delivery 23% 33% 27.40% 40% 58% 2. Increased proportion of women who delivered under supervision of a skilled health professional % of children 0-23 months whose delivery was attended by a skilled health professional (nurses with midwifery training, doctors, midwives) 26% 37% 30.83% 40% 56% 3. Increased proportion of women who deliver at a health facility % of mothers of children 0-23 months who deliver at health facility 20% 31% 30.08% 35% 53% 5. Improved knowledge and practice of malaria prevention and treatment at household and community level % of mothers of children 0 – 23 months who know 2 ways (ITN & IPT) to prevent malaria 17% 51% 15.80% 62% 14% % of children 0-23 months taken to HF or Community Health Worker within 24 hours after onset of fever 7% 46% 34.60% 60% 38% 6. Increased proportion of WRA and CU5 who sleep under insecticide-treated nets % of households with at least one ITN 77% 87% 91.70% 90% 95% % of mothers of children 0-23 months who slept under ITNs the previous night 65% 76.3 76.70% 80% 93% % of children 0-23 months who slept under ITNs the previous night 70% 78% 87.20% 80% 93% 8. Increased proportion of pregnant women receiving IPT % of mothers of children 0-23 months who received at least 2 doses of SP for IPT during ANC. 21% 50% 26.32% 60% 57% 9. Increased knowledge and understanding of PMTCT and ART among women of reproductive age (15-49 years) % of mothers of children 0 – 23 months who cite at least two ways of preventing MTCT 23% 50% 33.10% 59% 84% % of mothers of children 0-23 months who know that risk of MTCT can be reduced by ART 33% 46% 32.30% 50% 24% 10. Increased access to HIV counseling and testing among pregnant women at ANC. % of mothers of children 0-23 months counseled and tested for HIV at ANC during their most recent pregnancy 53% 66% 83.46% 70% 85% 26 27 Objective/Result Indicators (by technical or cross-cutting intervention) Baseline Value MIDTERM EOP Target Actual Target Actual % of mothers of children 0-23 months who know their HIV status 41% 55% 72.90% 60% 81% 12. Improved feeding practices among caretakers of children 0-5 months % of children age 0-5 months who were exclusively breastfed during the last 24 hours 11% 34% 21.90% 40% 52% Annex 7: CHW Training Matrix S/NO Title of training Cadres of personnel trained No. Trained Duration of training Year training undertaken 1 Training of Facilitators (TOF) DHMT members, Lot supervisors & AMREF staff 13 12 days 2006 2 Partnership defined quality (PDQ) Health facility staff & CHEWs 23 5 days 2006 3 Focused antenatal care (FANC) Health facility staff 16 5 days 2007 4 Essential obstetric and newborn care (EONC) Health facility staff 16 10 days 2007 5 Prevention of mother to child transmission (PMTCT) Health facility staff 23 10 days 2007 6 Monitoring and Evaluation Health facility staff CHEWs 22 5 days 2007 7 Integrated Management of Childhood Illnesses (IMCI) Health facility staff 24 12 days 2007 8 Primary Health care (PHC) and Community Strategy CHEWs 14 5 days 2007 9 Strategic leadership in health and development CHEWs 14 5 days 2006 10 Community maternal and newborn care management (CMNC) CHEWs 14 3 days 2008 11 Community Based Health Management Information System (CBHIMS) CHEWs and health facility in-charges 25 3 days 2008 12 Conducting Knowledge Practice & Coverage Survey CHEWs 14 5 days 2007 13 Trainer of trainers (TOT) for CHW leaders CHWs 40 5 days 2007 14 Training of CHWs in leadership CHWs 101 2 days 2008 15 PHC and Community Strategy CHWs 910 5 days 2007 16 CBHIMS training for CHWs CHWs 763 2 days 2008 17 CMNC CHWs 850 7 days 2008 18 PHC and Community Strategy CHC members 600 3 days 2009 Annex 8: Evaluation Team Members and Titles Judy Lewis, Final Evaluation Team Consultant Cudjoe Bennett, MPH (Technical Advisor – AMREF in USA) David Wamalwa (Project Manager – BCSP) Gilbert Wangalwa (M&E Officer – BCSP) Judith Ngeresa (BCC Officer – BCSP) Everlyne Omunjalu (Training Officer – BCSP) Everline Wanga (Deputy DHRIO-Busia) Phelgona Otieno (DPHN-Samia) Rosebella Odindo (DMOH-Butula) Michael Obiero (Deputy DHRIO, Busia) Christopher Sivo (DPHN- Butula) Charles Ngira (DPHO-Butula) Emmanuel Luvai (DPHO – Samia) James Simiyu Ojwanga (Deputy DPHO- Busia) Research Assistants Eunice Lubale Rafael Okech Felix Makasanda Ofware Hillary Okwara Millicent Kasaya Roseline Musila Kenneth Okumu Violet Juma Annex 9: Evaluation Assessment Methodology Following the CSHGP guidelines, the methodology for the Final Evaluation was participatory and focused on qualitative information to complement the quantitative data collected throughout the BCSP. The FE Team was comprised of the consultant, the AMREF USA Technical Backstop, 5 BCSP staff, 9 members of the DHMT and 8 research assistants who were hired to do field interviews. The RAs were college educated and most had prior experience doing research interviews. The decision to use RAs to do the interviews was made by the BCSP and AMREF USA. They had successfully worked with RAs previously and felt that hiring people from outside the district to do the interviews would lead to less bias in collecting the information. The BCSP and the consultant agreed prior to the FE on 15 groups to be interviewed (individual mothers with new babies, MtM groups, individual fathers with new babies, CHWs, CHW Leaders, CHEWs, Health Facility Staff, CHCs, Chiefs, School Teacher/Club Leaders, School Club Members, School pupils who were not members of Clubs, TBAs, DHMT, AMREF/BCSP Team). A few of the interview schedules were developed by the consultant in advance and translated, the rest were translated during the first three days of the evaluation. RAs, the AMREF Technical Backstop and the consultant conducted the field interviews. RAs worked in teams of two with one RA conducting the interview and the other writing the responses in a notebook provided by the BCSP. The AMREF Technical Backstop and the consultant directly recorded their interviews on laptop computers. The team debriefed at the end of each day, and the RAs continued to transcribe their interviews in the evenings. The BCSP was involved in planning the evaluation schedule and format prior to the arrival of the consultant and this smaller team met for one day before the full evaluation team was assembled for preparation for the FE. The BCSP planning day resulted in the identification of 4 health facility areas from 4 different LOTS. They were selected based on performance on the project objectives (2 high performing areas and 2 low performing areas) and other factors such as distance from services. This was done to provide greater opportunity for variation of responses to FE interviews. The first preparatory day was followed by two days of training and preparation with the full FE Team including BCSP, DHMT and Research Assistants. The agenda for the three preparatory days of field work was scheduled as follows: . Activity Monday 12‐ Jul Team Discussion about Agenda and Work Plan Review expectations and objectives Review Site Visits and logistics KPC Preliminary Results Review expectations and objectives Develop and revise interviews Translate interview schedules Assignments for report writing Tuesday Morning 13‐ Jul Introductions Welcome Expectations Overview of Evaluation and Field Visits Presentation of Preliminary KPC Why do qualitative assessments? What will we do with the data? LUNCH Tuesday Afternoon Role Play presentation Practice interviewing and notetaking in small groups Large group debriefing Modify interviews as required Wednesday Morning 14‐ Jul Complete practice interviews Debrief Modify interviews as required LUNCH Wednesday Afternoon Review final interviews Discuss sites and teams, prepare for field work Plan time to review field work each day As can be seen from the agenda, there was an overview of the project (for the RAs), a discussion of what was already known, the purpose of qualitative research, and a major emphasis on skills practice and revision of interview guides to improve the quality of the data. Role Playing Group Interview—Training Session Data collection took place over 4 days, with the entire data collection team conducting interviews with all 10-12 groups in an area in one day. This facilitated travel arrangements and allowed for team debriefing at the end of the day. BCSP FE Schedule July 2010 Sunday Monday Tuesday Wednesday Thursday Friday Saturday 11 Consultant/ Technical Backstop Meeting 12 BCSP Team Meeting:   FE Preparation 13 Day 1 FE Team Preparation/ Training 14 Day 2 FE Team Preparation/ Training 15 Field Inter‐ views: Nambuku 16 Field Interviews:   Sio‐Port 17 Field Interviews: Khunyango 18 Consultant/ Technical Backstop Meeting 19 Field Interviews: Bwaliro 20 Interview Write‐ups: Research Assistants 21 Summary of Field Interviews: FE Team 22 Summary of Field Interviews: FE Team Recommenda‐ tions 23 FE Wrap Up Discussion Points Writing Assignments 24 Each day of the field visits, the 4 RA teams and the AMREF USA Technical Backstop and the FE consultant conducted interviews. They were accompanied to the field by BCSP staff who met with the CHEWs based in the facilities to arrange for the interviews. Individual mothers and fathers were interviewed at their homes, so the two teams doing these interviews were guided by local CHWs. MtM Groups and CHCs were interviewed at their usual meeting sites. Chiefs, CHEWs and health facility staff were interviewed in their offices or nearby. CHWs and CHW Leaders were usually interviewed at the health facilities. School interviews were conducted on school grounds. One group of TBAs was interviewed on the grounds of the health facility and one at a neighboring house. DHMT and AMREF/BCSP interviews were conducted by the FE consultant at sites of mutual convenience, usually in the field or at the FE hotel site. Required interviews for each location were: individual mothers, individual fathers, CHWs, CHEWs, Health facility in-charge, CHCs and chiefs. School and TBA interviews were scheduled at two of the four locations. CHW leaders were interviewed as possible. An example of how the location interviews were conducted by type of interview appears in the chart below. Interviews July 15 Nambuku Team 4 Team 1 Team 3 Team 2 JL/CB   Mothers (individual) XXXXX Mothers’ groups X Fathers (Individual) XXXXX CHWs X CHEWS XX Health Facility Staff X CHC Committees X Chiefs School  teacher/patron   X School  club members       X School  non‐club students X CHW Leaders X TBAs   DHMT (individual JL) AMREF Team (Individual  JL) Field Pictures: Interviews with CHCs, Chiefs, Health Facilities At the conclusion of the field work, the RAs spent one day summarizing their interviews to be used by the larger evaluation team.  BCSP staff also worked evenings to pull all the information together from the qualitative interviews by group interviewed utilizing information from all 4 locations. These summaries were then used by the DHMT and BCSP staff.  They worked in teams focusing on each of the intervention areas, summarizing salient points from each of the groups of people interviewed.   The whole FE Team then met to discuss findings and make recommendations.  Individual interview data and summaries were used to write the FE Report. BCSP Final Evaluation Team, July 20, 2010 Annex 10: List of Persons Interviewed and Contacted During Final Evaluation CHC BWALIRO 1. Rose Musimbi 2. Nelson Okumu 3. Nicodemus Ochanda 4. Fredrick Odhiambo 5. Dorosila Wanga 6. Amina Nekesa 7. Jonai Makana 8. Florence WAmbongo 9. Doroth Adhiambo 10. Cicilia Abonyo 11. Risper Narotso 12. Fcedes okhala 13. PratricK Owanga CHC MANGO 1. Fredrick E Owino 2. Johnstone WAfula 3. Nancy Opondo 4. Alex Barasa 5. Risper Agolla 6. Elisapham Mangeni 7. Lilian Wesonga 8. Jenifer Swali 9. Barsa Joseph 10. Everlyne Narotso 11. George BWire HAKATI CHC 1. Catechist Joab Bwire Wafula 2. Wilfrida Nabwire Wanyama 3. Jacinta Wandera 4. Caleb Moya 5. Richard Nabongo 6. James Nabegesa 7. Sammy W. Omanyo 8. Michael Wanyama 9. Josephat khaede 10. Sylvester K Ouma 11. Joseph Oketchi BUKLHALARIRE CHC 1. Hellen Nabwire 2. Zachary Obiero 3. Francis Miena 4. Rev. Emily Odhiambo 5. Gregory Mukolwere 6. Justus Ambani 7. Rutimem Makokha 8. Samuel Onyango 9. Francis Asoma 10. Marygoretti Mwene 11. Patrick Oboya 12. Okoti Wabudi 13. Silas Ndon’ga TBAs BWALIRO 1. Caroline Ajiambo 2. Marceline Eliathe 3. Jesca Adhiambo 4. Everline obuora TBAs KHUNYANGU 1. Rosemary A Odingo 2. Mariamu Namukuru 3. Martina Arani 4. Phoebe Agola MOTHER TO MOTHER SIO PORT 1. Ruth Alungat 2. Josper Akoth 3. Caroline Awino 4. Henrieta Ogai 5. Judith Nangira 6. Seraphine Atieno 7. Damaris Abayo 8. Victorine Ajiambo 9. Penpola Nagula 10. Alice Bwire 11. Hawa Hamisi 12. Juliet Barasa 13. Everlyne Awino 14. Eunice Akinyi 15. Anjeline Oundo 16. Asha Aruna 17. Azida Halima 18. Rebecca Ambasu 19. Praxidis Obaya 20. Uanina Nekesa 21. Caroline Akello CHEWS LOT 7 1. Lylian Wekongo 2. Maurice Makokha INDIVIDUAL MOTHERS/FATHERS SIO – PORT 1. Josephine Ajiambo 2. Lilian Opondo 3. Rebecca Adongo 4. Hellen Juma 5. Mariam Ali 6. Beatrice Ali 7. Fredrick Sanya 8. Godfrey Ouma 9. Peter Osinya 10. Benard Ogutu 11. Rashid ukidadi 12. Rose Atieno 13. Sheila Auma 14. Anne Apondi CHWs SIO PORT 1. Hellen Atulo 2. Godfrey Musungu 3. I Bahati Ndanda 4. Raymond Meshack Ndanda 5. Mathews Appolen 6. Mildred Oduory 7. Judith Adhiambo 8. David N Ojiambo 9. Alice a Oundo 10. Christine Agutu 11. Aaron Ndako 12. Paul Wandera INDIVIDUAL FATHERS LOT 5 1. Nicholas wandera 2. Sigune Fredrick 3. Kenedy Omondi 4. Shadrack Okeda 5. Victor obadia 6. Sospeter Okwaba 7. Chrispine Mugeni CHEWS LOT 5 1. Gladys Mumia 2. Lena Mulumia M 2 M LOT 5 1. Beldine Adhiambo 2. Colleta Anyango 3. Seraphine Nabwire 4. Joseline Wafula 5. Pamela Auma 6. Rebecca Ongia 7. Esther Ouma 8. Lucy Apondi 9. Christine Mirriana 10. Jonas Sikudi 11. Sara Nabwire 12. Dibora Juma 13. Babura Juma 14. Dorica Akinyi 15. Jackline Oduor INDIVIDUAL MOTHERS LOT 5 1. Ann victorine 2. Faith Osta 3. Agripina Bukhalaka 4. Rose Atieno 5. Taaka Josephine 6. Morine L Auma 7. Trizzer Mwangagi 8. Milka Otieno 9. Trufosa Were CHWS BWALIRO LOT 3 1. Mildred Namusenda 2. Patrick Odwori 3. Everline Maudo 4. Mildred Makana 5. Benedict P Okumu 6. Gerald Omondi 7. Nikanun Ogutu 8. Patrick Wabuko 9. Ignatius O Wanga 10. Patrick M Juma 11. Bonface Owino 12. Jane opiyo 13. Evalyne Namenge 14. Edward Wanga 15. Mildred Akoth INDIVIDUAL FATHERS AND MOTHERS BWALIRO 1. Aquinaza Amollo 2. Maximilla Buyalla 3. Jacinta Mukok 4. Christine Achieng 5. Violet Nabwire 6. Hesbon Wangaywa 7. Stephen Baanga 8. Benard Obiero 9. Benjamin Ogutu 10. Godfrey Ojame M2 M BWALIRO 1. Pamela Odhiambo 2. Syphrosa Lusweiti 3. Eunice Anyango 4. Lilian Ngesa 5. Eunice Otieno 6. Christine Atsieno 7. Lilian Akinyi 8. Jackline Odhiambo 9. Emily Night 10. Lydia Awino 11. Cecilia Were 12. Nancy Awino 13. Eunice Akinyi 14. Evaline Achieng 15. Roda Apondi 16. Wilkista Auma 17. Elizabeth Akinyi 18. Philis Musa 19. Jane Anyango 20. Mary Otieno CHEWs LOT 3 1. Rebecca Matalanga 2. Godfrey Makio CHEWs LOT 2 1. Penina Khayumbi 2. Everlyne Walela CHWS KHUNYANGU 1. Mildred Auma 2. Beatrice Auma 3. Dismas Oduor Ojwang 4. Stephen Owino 5. Judith Anyango 6. Jane Okumu 7. Mary Awino 8. Leonard Amati 9. Mary Barasa 10. Lawrence wafula 11. Justine Akinyi 12. Leonard Z. Nzuia 13. Beatrice Narotso 14. Henry Yonga Yonga 15. Ann A Ndalira 16. Anne A ongero M2M KHUNYANGU 1. Celestine Adioo 2. Duscan Linda 3. Seline Makungu 4. Mercy Anyango 5. Triza Apondi 6. Susan Barasa 7. Lilian Akinyi 8. Agneta Achieng 9. Rosemary Adiambo 10. Alice Ouma 11. Inviolata Night 12. Susan Atieno 13. Ruth Akinyi 14. Catharine Akinyi 15. Jenifer Adhiambo 16. Sylivia Mukanga 17. Wilybroda Awino 18. Metrine Aori 19. Rose Auma 20. Morine Adwori 21. Everline Wekesa DHMT 1. Emmanuel Luvai – DPHO Samia 2. Rosebella Odindo – DPHN/AG. MOH Butula 3. Roseline Oboya – DPHN Busia 4. James Kuya – DHRIO Busia AMREF Busia 1. David Wamalwa – Project Manager 2. Gilbert Wangalwa—Monitoring and Evaluation Officer 3. Judith Ngeresa—BCC Officer 4. Everlyne Omunjalu—Training Officer 5. Bibianne Situma—Administrative Assistant Kenya Country Office 1. Peter Ofware 2. Festus Illako US Office 1. Bill Yaggy 2. Cudjoe Bennett Annex 11: Special Reports Abstracts or Executive Summaries are provided for the following. For full reports/papers, please contact AMREF Technical Consultant Jennifer Weiss, weiss@amrefusa.org • Organizational Capacity Assessment • Improving Means of Community Transportation for Women Seeking Obstetric Health Care • Motivational factors that determine retention of community health workers in Busia District, Kenya • Determinants of Male Partner Involvement in Promoting Deliveries by Health Professionals in Busia Kenya • Effectiveness and Cost Summary of Ministry of Public Health and Sanitation’s Strategy for Community-Based Maternal and Newborn Health Service Delivery in Funyula and Butula Constituencies, Busia County, Kenya BCSP Organizational Capacity Assessment Final Report Executive Summary Impact Centre, Nairobi, Kenya The Busia Child Survival Project (BCSP) covers Butula and Funyula Divisions of Busia District. A baseline study carried out by AMREF identified inadequate maternal and new born care, Malaria and HIV/AIDS as the leading causes of newborn, child and maternal morbidity and mortality in the two divisions. Accordingly, the project focus is on Maternal and newborn care (40%); Malaria control (40%) and; HIV/AIDS (20%). This is a five year project (October 2005-September 2010) funded by United States Agency for International Development (USAID). The project’s intended beneficiaries include 49,858 women of reproductive age and 31,664 children under 5 years in Funyula and Butula divisions1. The project is a partnership between African Medical and Research Foundation (AMREF) and Busia District Health Management Team (DHMT). Project Goal and Objectives Goal The project aims at reducing child and maternal morbidity and mortality in Funyula and Butula divisions within Busia, and creating sustainable health structures. Objectives: Maternal and Newborn Care 1. Increased proportion of pregnant women who attend antenatal clinic at least four times during pregnancy from 32% to 50% and postnatal clinic at least once from 23% to 40%. 2. Increased proportion of women attended by a skilled health professional during delivery from 26% to 40%. 3. Increased proportion of women who deliver at a health facility from 20% to 35% 1 AMREF, Busia Child Survival Project Proposal 40 4. Improved quality of and access to basic EmOC at health facilities. Malaria Control 1. Improved knowledge and practice of malaria prevention and treatment at household and community level. 2. Increased proportion of pregnant women and Children under 5 years (CU5) who sleep under insecticide-treated bed nets from 65% to 80%. 3. Improved case management of malaria/fever among CU5 at health facilities. 4. Increased proportion of pregnant women who receive at least two doses of SP for Intermittent Preventive Therapy (IPT) from 21% to 60%. HIV/AIDS 1) Increased access to HIV counselling and testing among pregnant women at Ante Natal clinic (ANC) from 53% to 68% 2) Increased number of HIV infected pregnant women and newborns who receive Prevention of Mother To Child Transmission (PMTCT) 3) Increased knowledge and understanding of PMTCT and ART among women of reproductive age from 33% to 48% 4) Improved feeding practices among caretakers of children 0-5 months from 11% to 36%. Project Strategies In order to achieve its goals, the project has adopted three strategic approaches: 1. Building capacity of the DHMT, health facility staff, and Community Own Resource Persons (CORPs) to effectively address maternal and new born care issues; 2. Quality assurance and improvement approaches to improve quality of care at health facilities and in the community; 3. Behavior change and communication strategies at the household and community level addressing cultural and societal barriers to disease prevention. 41 Key Findings The Organization Capacity Assessment was focused on AMREF and the Busia District Health Management Team (DHMT) as implementing partners of the BCSP. The assessment sought to establish AMREF’s and DHMT capacity along seven capacity areas: • Management practices and governance •Administrative Infrastructure and Procedures • Financial Resources Management • Human Resource Management • Technical knowledge and skills •Programme Management, monitoring, evaluation and reporting • Organizational Learning The Organization Capacity Assessment was conducted through interviews, focus group discussions, document review and a self administered questionnaire in AMREF KCO, BCSP offices and at the DHMT. The BCSP design is consistent with governmental policy. The BCSP is relevant and appropriate in that its design meets the priority needs of the target group. The first year of the project (October 2005- September 2006) was a preparatory period. All the necessary infrastructure has been put in place in terms of physical facilities, office equipment, baseline assessments, formation of teams and committees. Partnership with DHMT has also been concretized into a solid working relationship. The activities that were identified in the DIP to be undertaken in the first year have all been undertaken. The project manager and the DHMO are confident that project interventions can now be scaled up. The organizational strength scores for each of the assessment variables was as follows: • Management practices and governance (74%) •Administrative Infrastructure and Procedures (61%) • Financial Resources Management (50%) 42 • Human Resource Management (62%) • Technical knowledge and skills (84%) •Programme Management, monitoring, evaluation and reporting (82+86+70). Average= 79% • Organizational Learning (66%) Overall, AMREF and the Busia DHMT registered an organizational average capacity score of 68%. The capacity gaps identified in AMREF are more issues of information, Communication and Practice than institutional weaknesses. Otherwise the overall rating would have been much higher. The capacity gaps in DHMT have elements of institutional weaknesses. In the consultant’s view, this is not a reflection of the Busia DHMT. It is a factor of it being a government organ. The qualitative strengths in AMREF include experience in Child Survival projects, donor confidence as demonstrated by the funding levels for AMREF projects, fund management responsibilities for regional health interventions, international recognition and leadership roles on health issues at the national level and programme presence in the Lake Victoria Region. The DHMT has strengths in having a health infrastructure and personnel in Funyula and Butula and also at the Busia District headquarters. Nearly all of the information that led to a low score is available in the project documents. This suggests that there is a strong need for the project staff, PIT and members of the DHMT to adequately acquaint themselves with the contents of the project documentation. The overall baseline organizational capacity assessment of AMREF and DHMT is that the partnership has the capacity to implement BCSP as shown by both the qualitative and quantitative aspects of the assessment. In spite of the high organizational capacity, the assessment established a couple of gaps in certain critical areas of organizational capacity. These include: 43 Management Practices and Governance Information is not being shared amongst partners in such a way that everyone involved has sufficient understanding of the project implementation parameters. For instance, induction of PIT members should be such that they have adequate knowledge of the project by the time they attend their first meeting. Members of the PIT and the DHMT should have a general understanding of the administrative and financial systems and procedures under which the project will be implemented While employment practices may not be well known by non-employees, partner representatives and members of the PIT should generally be aware of what the project requirements are and as part of the their project management roles, endeavour to ensure that these are in place. Programme Management, Monitoring and Evaluation Besides the lack of technical capacity identified at the beginning of the project, the DHMT is faced with inadequate administrative resources and communication facilities Sustainability AMREF’s exit strategy is not well articulated. 44 Improving Means of Community Transportation for Women Seeking Obstetric Health Care D.M. Mogere, MCHD Thesis, Great Lakes University of Kisumu, February 2009 ABSTRACT Background This exploratory descriptive study was carried out in four Health Facilities within Busia and Samia districts of Western Kenya. The region has one of the highest maternal mortality rates in the country and majority of the women deliver at home. Objective The primary objective of the study was to establish available means of community transportation that are used to enable women reach health facilities while seeking obstetric health care with the aim of identifying appropriate and feasible means of transportation that may be tested and implemented in Busia . Methodology The study adopted a cross sectional descriptive exploratory study design using both quantitative and qualitative data collection techniques. One hundred and fifty two (151) patients who were admitted in maternity wards every day for 8 weeks in four hospitals were interviewed using structured questionnaires. Five FGDs were conducted with ANC women, village elders and women leaders. Key Informant Interviews were held with health facility managers in the four hospitals. Desk Review of Ministry of Health policy documents, CDF act (2007),LATF Guidelines and Kenya Gazette Notices was also done. Results The findings showed that seven means of transportation are used by women to travel to the hospital .Forty five percent of the women used bicycles, 22% walked, 21.7 % matatu and the rest used taxi, Government ambulance, mission hospital ambulance, private car and motor cycle. Only 4% of the women used hospital ambulance to travel to the hospital. These results were consistent with findings of previous studies done by WHO, 2005 and MoH, 2002. 45 Motivational factors that determine retention of community health workers in Busia District, Kenya Collins James Owek, MCHD Thesis, Great Lakes University of Kisumu, August, 2009 ABSTRACT This study was done in two divisions of the greater Busia district (Butula and Funyula) in September, 2008 in order to assess the motivational factors that determine the retention of community health workers. This followed a concern by dismal performance on health indicators by (DHMT) District Health Management Team and AMREF Busia. Both quantitative and qualitative approaches were used to collect data from the community health workers and other stakeholders. A total of 300 questionnaires were administered to the CHWs in the quantitative approach while for the qualitative aspect, six key informants were interviewed and seven focus group discussions were held. The study revealed that 73% of the CHWs consider that they are currently recognized by the community, while 40% perceived recognition by the community as a determinant that would retain them. There was overwhelming demand for the services of the CHWs whereby almost 84.6% of the CHWs serve more than the recommended 20 households; despite the workload 61.8% of them felt they care for the right number of clients. About 10% of the CHWs said they expected monetary gain on recruitment and currently 65% of the CHWs acknowledge reimbursements as having motivated them to continue serving. Currently, 38.5% of the CHWs said material incentives like bicycles, T-shirts and bed-nets have contributed to motivate them and 76% perceived that this would improve on their motivation. Among the CHWs who had served for at least 3 years, 96% had attended refresher courses which have contributed to their motivation. It was found that the retention level for CHWs was 30%. The current motivational determinants are recognition by the community members, demand for the CHWs services by the community, skill development, provision incentives and inclusive supervision schemes. The perceptions of the CHWs on retention include; family support, community support and health care system support. Provision of the working materials for the CHWs like bags, CHWs kit, and reporting materials, harmonize the workload for the CHWs in order to improve on quality of care, encourage team work and peer-support within the CHWs work and also for income generating activities. There should be inclusion of the community leaders in the supervision scheme for the CHWs and empowerment of the CHWs and other community structures through continuous refresher courses by the health care system. 46 Determinants of Male Partner Involvement in Promoting Deliveries by Health Professionals in Busia Kenya Mildred Nanjala Wamalwa, MCHD Thesis, Great Lakes University of Kisumu, March, 2010 EXECUTIVE SUMMARY Busia and Samia districts, where this study was carried have a maternal mortality ratio of 680/100,000 live births and an infant mortality rate of 80/1.000 which far surpass the national figures of 414/100,000 and 77/1,000 respectively (KDHS, 2003). The low rate of deliveries by health professionals in the two districts at only 26% (AMREF, 2006) compared to a national coverage of 44% (KDHS, 2003) is cited as one the contributing factors to high maternal and infant mortality rates. A survey in the two districts by AMREF in 2006 established that low coverage of deliveries by health professionals was partially attributed to low involvement of male partners in matters of pregnancy and child birth. (AMREF, 2006) The study aimed at establishing factors that affect male partner involvement in supporting their spouses to access delivery services from skilled professionals. Specifically, the study looked at the effects of the level of education, age and occupation of male partners in supporting their spouses to access skilled delivery services. The study was undertaken in Butula and Funyula divisions in Busia and Samia districts respectively, and it utilized cross-sectional and retrospective study designs and covered a sample size of 385 houses that were randomly sampled using a simple random method. The study respondents included male partners (385) and their spouses (380) with children aged 0-12 months at the time of the study. Two focus group discussions (one each in Butula and Funyula) were also held with the community health workers and key informant interviews were held with the in-charges of maternity ward in the four health facilities (Sio Port Hospital, Nangina Dispensery, Bumala B and Khunyangu Hospital) that are used by mothers in the study area for delivery services. The study established no relationship between the age of the male partner and the type of delivery service (skilled or unskilled) that spouses accesses. The study, however, established that male partners with a high level of basic education, salaried employment were more likely to support their spouses to seek the service of a health professionals than those with low education or none and low income. Majority of the male partners also regarded pregnancy and child issues as a woman’s matter. Negative cultural practices by the local community were also identified as a factor contributing to low male involvement. For instance it is believed that “to avoid the bad omen striking the home and killing the newborn, a baby boy should be kept indoors for four days and girl for three days after birth”. Delivering from hospital therefore will mean that the babies will be taken out of the house before the culturally prescribed period. The study also established negative attitudes of the health workers as a hindrance to male participation. To enhance male involvement in the promotion of skilled delivery services in the study area, it is recommended that men be assisted to start income generating projects to increase their levels of income, extensive health education to men regarding negative cultural practices and importance of skilled deliveries be done, a customer care training be conducted for health workers to improve client￾health provider relationship. The education levels of the men should also be enhanced through the adult literacy programme while at the same time promoting boy child education. 47 Effectiveness and Cost Summary of Ministry of Public Health and Sanitation’s Strategy for Community-Based Maternal and Newborn Health Service Delivery in Funyula and Butula Constituencies, Busia County, Kenya Executive Summary Background: In 2008, the global mortality rate in children younger than five years was estimated to be 8.8 million, of which neonatal deaths accounted for 41%. This mortality is linked closely to slow progress in reduction of maternal mortality, estimated at 342,900 maternal deaths worldwide in 2008. In Kenya, maternal mortality ratio (MMR) and neonatal mortality ratio are estimated at 414 women per 100,000 live births and 31 deaths per 1,000 live births, respectively. To reverse the maternal and newborn health outcomes and contribute to MDGs 4 and 5, Kenya’s National Health Sector Strategic Plan II (NHSSP II) emphasizes the promotion of individual and community health through a number of strategies, one of them being the community health strategy. This is to be achieved by providing health care services for all cohorts and socio-economic groups at household and community levels; building the capacity of community health extension workers and community health workers to provide community level services; strengthening health facility–community linkages through effective decentralization and partnership; raising the community’s awareness on their rights to health services; and use of results of needs assessments to identify major gaps and themes for community action. The African Medical and Research Foundation, a key partner with the Ministry of Public Health and Sanitation, through funding from United States Agency for International Development (USAID), designed a Community-based Maternal and Newborn Care (CBMNC) intervention and implemented it within the framework of the community health strategy in Butula and Funyula Constituencies at an approximate cost of US $50,000 (Kshs. 4,213,486). The interventions aimed at empowering women, men, families and communities to stay healthy, make healthy decisions and respond to maternal and neonatal needs and emergencies; strengthen linkages between the different levels of service delivery; and strengthen community action in addressing their health problems. Methods: The objective of the study was to evaluate the effectiveness of the intervention by assessing change in knowledge in a sample of 1092 pregnant women and utilization of the essential maternal and newborn care practices among 532 women with children aged 0 – 23 months over a period of two years. We used Lot Quality Assurance Sampling to draw the first sample from a population of 15,000 pregnant women, and Systematic Sampling to draw the second sample from a population of 11,800 women with children aged 0 – 23 months. Following a pre-intervention baseline survey, the study used a time series design with a total of three post-intervention observations at intervals of seven months over a period of 22 months. Results: Fisher’s exact test comparing knowledge of individual birth planning and preparedness showed that at first observation, 89.8% (n = 353) of pregnant women knew between 0-3 (out of 7) components of the individual birth plan and 8.2% (n = 353) knew 4-6 components. At the third observation, there was a statistically significant (p < 0.001) reduction in the percentage (63.9%, n = 352) of women knowing 0-3 components and a statistically significant (p < 0.001) increase in the percentage (32.7%, n = 352) of women knowing 4-6 components. There was a statistically significant 23.3% increase (p < 0.001) in the numbers of women of children aged 0-23 months who reported ANC attendance at least four times during their most recent pregnancy 48 49 from the pre-intervention observation to the second post-intervention observation. Additionally, children aged 0-23 months whose births were attended by skilled health personnel showed a statistically significant (p < 0.001) increase from 30.8% at pre-intervention observation to 57.1% at the third post￾intervention observation. In terms of infants 0-5 months who exclusively breastfed, there was a statistically significant (p < 0.001) increase in the percentage of women adopting this practice from 19.5% at pre-intervention observation to 51.5% at the third post-intervention observation. Discussion: Developing community health strategy structures and using them to deliver community based maternal and newborn care translated into improved health knowledge and practices in Busia and Samia Constituencies of Busia County, Kenya. The study showed that the community’s knowledge of essential maternal and newborn care (EMNC) was positively influenced by interactions with CHWs. The increase in adoption of EMNC practices demonstrates that, with active support from community structures, pregnant women will not only know more about the importance of their health during pregnancy, but will also act upon that knowledge. Conclusion: In this programme, establishment of community-based structures focusing on maternal and newborn care increased information to community members and encouraged and facilitated important behaviour changes. Annex 12: Project Data Form General Project Information Cooperative Agreement Number: GHS-A-00-05-00009 AMREF Headquarters Technical Backstop: Cudjoe Bennett AMREF Headquarters Technical Backstop Back-Up: Field Program Manager: David Wamalwa Midterm Evaluator: Larry Casazza Final Evaluator: Judy Lewis Headquarters Financial Contact: Project Dates: 9/30/2005 – 9/30/2010 (FY05) USAID Mission Contact Dr. Sheila Macharia Project Web Site: Field Program Manager Name: David Wamalwa (Project Manager) Address: AMREF Busia Busia Kenya Phone: 254 0733229992 Fax: E-mail: david_wamalwa@yahoo.com Skype Name: Alternate Field Contact Name: Festus Illako (Project Manager) Address: Nairobi Kenya Phone: 254 20 699 4000 Fax: 254 20 606 340 E-mail: Festus.ilako@amref.org Skype Name: Grant Funding Information USAID Funding: $1,305,312 PVO Match: $834,000 General Project Description Program Goal: A sustained reduction in child and maternal mortality in Funyula division of Samia District and Butula division of Busia District, Kenya. Interventions: -Maternal and Newborn Care –Malaria Control – HIV/AIDS Strategies: (i) Building capacity of the DHMTs, health facility and community level health workers aimed at improving access to quality health services; (ii) Quality assurance approaches to improve quality of care at health facilities through Partnership Defined Quality (PDQ) and facilitative supervision; by alerting TBA roles in the communities and strengthening partnerships between CORPs and health facility workers; and via the procurement of essential 50 supplies; and (iii) Behavior change and communication strategies at the household and community level directly addressing cultural and societal barriers to disease prevention Project Location Latitude: 0/47 Longitude: 34.10 Project Location Types: Rural Levels of Intervention: District Hospital Health Center Health Post Level Community Province(s): -- District(s): Butula Division of Busia District and Funyula Divisions of Samia District in the Western Province of Kenya Sub-district(s): -- Operations Research Information OR Project Title: Effectiveness and Cost Summary of the Ministry of Public Health and Sanitation’s Strategy for Community-Based Maternal and Newborn Health Service Delivery in Busia and Samia Districts, Kenya Cost of OR Activities: $106,729 Research Partner(s): -- OR Project Description The purpose of the study is to test the effectiveness of the MOPHS strategy for community-based health service delivery in reducing deaths of children less than 28 days old In Busia and Samia districts, maternal mortality, HIV prevalence among pregnant women, and infant and child mortality all exceed their respective national averages. Poor child health appears to be inextricably linked to poor maternal health, and both are notably associated with interrelationships between malaria, HIV, and the quality of and access to maternal and newborn care (MNC). Approximately 80% of women in Busia and Samia districts deliver without a skilled attendant and hardly make a post-partum visit to a health facility. Many deaths can be prevented if women use skilled MNC practices during pregnancy, delivery, and postpartum period. Poverty and the subordinate social status of women exacerbate the situation. 51 The quality of care in the Busia and Samia district health facilities has suffered due to low numbers of health care workers. The attitude of fatigued health workers towards care seekers has been subpar in relation to patients’ cultural and socio-economic needs. Consequently, mothers have resorted to alternative MNC services from unqualified personnel with limited knowledge and skills on essential maternal and newborn care (EMNC). AMREF Busia Child Survival Project has intervened to address the problem though community strategy for community based maternal and newborn care. The objective of the study is to test the effectiveness and summarize the cost of the Ministry of Public Health and Sanitation strategy for community-based health service delivery to mothers and newborns. Effectiveness will be measured in terms of: (i) increased awareness of EMNC practices among pregnant women and mothers/caretakers of children less than 23 months; (ii) increased adoption of EMNC practices among pregnant women and mothers/caretakers of children less than 23 months; and (iii) reduced deaths among children less than 28 days. Partners Busia District Health Management Team (Collaborating Partner) $0 Samia District Health Management Team (Collaborating Partner) $0 PSI (Collaborating Partner) $0 MSF-Spain (Collaborating Partner) $0 AMPATH (Subgrantee) $0 APHIA II (Collaborating Partner) $0 Strategies Social and Behavioral Change Strategies: Group interventions Interpersonal Communication Social Marketing Mass media and small media 52 Health Systems Strengthening: Supportive Supervision Developing/Helping to develop job aids Monitoring health facility worker adherence with evidence-based guidelines Providing feedback on health worker performance Community role in supervision of CHWs Community role in recruitment of CHWs Coordinating existing HMIS with community level data Community input on quality improvement Strategies for Enabling Environment: Advocacy for policy change or resource mobilization Building capacity of communities/CBOs to advocate to leaders for health Tools/Methodologies: BEHAVE Framework Sustainability Framework (CSSA) Rapid Health Facility Assessment LQAS Participatory Rapid/Rural Appraisal MAMAN Framework Capacity Building Local Partners: Pharmacists or Drug Vendors Traditional Healers Dist. Health System Health Facility Staff Health CBOs Government sanctioned CHWs Interventions and Components Immunizations IMCI Integration CHW Training HF Training Nutrition IMCI Integration CHW Training HF Training Vitamin A IMCI Integration CHW Training HF Training Micronutrients CHW Training HF Training Pneumonia Case Management IMCI Integration CHW Training HF Training 53 Control of Diarrheal Diseases IMCI Integration CHW Training HF Training Malaria (40%) - Training in Malaria CM - Adequate Supply of Malarial Drug - Access to providers and drugs - Antenatal Prevention Treatment - ITN (Bednets) - Care Seeking, Recog., Compliance - IPT IMCI Integration CHW Training HF Training Maternal and Newborn Care (40%) - Emergency Obstetric Care - Neonatal Tetanus - Recognition of Danger signs - Newborn Care - Post partum Care - Integration with Iron and Folic Acid - Normal Delivery Care - Birth Plans - STI Treat. With Antenat. Visit - PMTCT of HIV - Emergency Transport IMCI Integration CHW Training HF Training Healthy Timing/Spacing of Pregnancy IMCI Integration CHW Training HF Training Breastfeeding IMCI Integration CHW Training HF Training HIV/AIDS (20%) - PMTCT IMCI Integration CHW Training HF Training Family Planning IMCI Integration CHW Training HF Training Tuberculosis IMCI Integration CHW Training HF Training Location and Sub-Areas Total Population: 202,348 Target Beneficiaries Kenya-AMREF-FY05 Children 0-59 months 31,664 Women 15-49 years 49,858 Beneficiaries Total 81,522 54 Rapid Catch Indicators: DIP Submission Sample Type: LQAS Indicator Numerator Denominator Percentage Confidence Interval Percentage of children age 0-23 months who are underweight (-2 SD from the median weight-for￾age, according to the WHO/NCHS reference population) 22 133 16.5% 6.3 Percentage of children age 0-23 months who were born at least 24 months after the previous surviving child 50 86 58.1% 10.4 Percentage of children age 0-23 months whose births were attended by skilled health personnel 34 133 25.6% 7.4 Percentage of mothers of children age 0-23 months who received at least two tetanus toxoid injections before the birth of their youngest child 67 133 50.4% 8.5 Percentage of infants age 0-5 months who were exclusively breastfed in the last 24 hours 15 133 11.3% 5.4 Percentage of infants age 6-9 months receiving breastmilk and complementary foods 133 133 100.0% 0.0 Percentage of children age 12-23 months who are fully vaccinated (against the five vaccine￾preventable diseases) before the first birthday 40 133 30.1% 7.8 Percentage of children age 12-23 months who received a measles vaccine 86 133 64.7% 8.1 Percentage of children age 0-23 months who slept under an insecticide-treated bednet the previous night (in malaria-risk areas only) 93 133 69.9% 7.8 Percentage of mothers who know at least two signs of childhood illness that indicate the need for treatment 116 133 87.2% 5.7 Percentage of sick children age 0-23 months who received increased fluids and continued feeding during an illness in the past two weeks 1 133 0.8% 1.5 Percentage of mothers of children age 0-23 months who cite at least two known ways of reducing the risk of HIV infection 86 133 64.7% 8.1 Percentage of mothers of children age 0-23 months who wash their hands with soap/ash before food preparation, before feeding children, after defecation, and after attending to a child who has defecated 1 133 0.8% 1.5 55 Rapid Catch Indicators: Mid-term Sample Type: LQAS Indicator Numerator Denominator Percentage Confidence Interval Percentage of children age 0-23 months who are underweight (-2 SD from the median weight-for￾age, according to the WHO/NCHS reference population) 0 0 0.0% 0.0 Percentage of children age 0-23 months who were born at least 24 months after the previous surviving child 0 0 0.0% 0.0 Percentage of children age 0-23 months whose births were attended by skilled health personnel 41 133 30.8% 7.8 Percentage of mothers of children age 0-23 months who received at least two tetanus toxoid injections before the birth of their youngest child 69 133 51.9% 8.5 Percentage of infants age 0-5 months who were exclusively breastfed in the last 24 hours 29 113 25.7% 8.1 Percentage of infants age 6-9 months receiving breastmilk and complementary foods 0 0 0.0% 0.0 Percentage of children age 12-23 months who are fully vaccinated (against the five vaccine￾preventable diseases) before the first birthday 0 0 0.0% 0.0 Percentage of children age 12-23 months who received a measles vaccine 0 0 0.0% 0.0 Percentage of children age 0-23 months who slept under an insecticide-treated bednet the previous night (in malaria-risk areas only) 116 133 87.2% 5.7 Percentage of mothers who know at least two signs of childhood illness that indicate the need for treatment 118 133 88.7% 5.4 Percentage of sick children age 0-23 months who received increased fluids and continued feeding during an illness in the past two weeks 13 133 9.8% 5.0 Percentage of mothers of children age 0-23 months who cite at least two known ways of reducing the risk of HIV infection 93 133 69.9% 7.8 Percentage of mothers of children age 0-23 months who wash their hands with soap/ash before food preparation, before feeding children, after defecation, and after attending to a child who has defecated 0 0 0.0% 0.0 56 57 Annex 13: Grantee Plans to Address Final Evaluation Findings In accordance with AMREF’s Knowledge Management Strategy (2010-2014), AMREF is committed to applying the lessons learned from the BCSP final evaluation to ensure better health for Africa. Within AMREF, findings from the final evaluation will be shared at the Annual Technical Meeting of all program staff. AMREF has already begun to apply specific methodologies used during the program: The BCSP M&E Officer traveled to South Africa to train AMREF program managers in LQAS for their own child survival program. In addition, several AMREF staff members have expressed interested in implementing the PDQ methodology in future programs. In addition to applying these lessons within the organization, AMREF will also share findings with the broader child survival community, though a variety of forums: • Organize presentation of findings and lessons learned at Spring 2011 CORE Meeting • Submit abstracts on lessons learned to Global Health Council and other conferences, as appropriate • Upload final evaluation document on to Development Clearinghouse (DEC) and other web sites • Circulate final evaluation document on appropriate technical list serves • Publish MNC OR report in a peer reviewed journal