RWANDA “UMUCYO” (ILLUMINATION) CHILD SURVIVAL PROJECT FINAL EVALUATION REPORT Former Kibogora Health District Dates: 12 - 29 September 2006 Jean Meyer Capps Evaluation Team Leader and External Consultant This final evaluation was completed in compliance with Cooperative Agreement HFP-A-00-01-00029-00 for World Relief, 7 East Baltimore Street, Baltimore, MD 21202 1 World Relief/Rwanda Child Survival Final Evaluation December 2006 TABLE OF CONTENTS ACRONYMS AND DEFINITIONS ACKNOWLEDGEMENTS A. EXECUTIVE SUMMARY........................................................................................... 7 B. ASSESSMENT OF RESULTS AND IMPACT OF THE PROJECT .......................... 9 B.1 Results of Program Efforts compared to Baseline and Targets......................10 B.2 Results: Technical Approach..........................................................................13 B.2.a HIV/AIDS............................................................................................. 13 Summary of CSP Syphilis Activities for Since the MTE Report....................... 14 B.2.b Immunization ........................................................................................ 16 B.2.c Reproductive Health/Maternal and Newborn Care............................... 17 B.2.d Diarrhea................................................................................................. 18 B.2.e Nutrition and Breastfeeding.................................................................. 20 B.2.f Malaria .................................................................................................. 22 B.2.f.i Prevention: Increasing access to ITNs.............................................. 22 B.2.f.ii Home Based Management of Malaria (HBM).................................. 23 B.2.f.iii Intermittent Presumptive Treatment (IPT) ...................................... 28 B.3 Cross Cutting Approaches..............................................................................29 B.3.a Community Mobilization...................................................................... 29 B.3.b Behavior Change Communication........................................................ 30 B.3.c Capacity Building ................................................................................. 31 B.3.c.i Strengthening the PVO organization................................................ 31 B.3.c.ii Approach to Strengthening Local Partners...................................... 31 B.3.c.iii Health Facilities Strengthening....................................................... 33 B.3.c.iv Strengthening Health Worker Performance .................................... 33 B.3.c.v Training............................................................................................. 34 B.4 Sustainability Strategy....................................................................................34 C. PROJECT MANAGEMENT...................................................................................... 35 C.1 Planning ..........................................................................................................35 C.2 Staff Training..................................................................................................36 C.3 Supervision of Project Staff............................................................................37 C.4 Human Resources and Staff Management......................................................37 C.5 Financial Management....................................................................................38 C.6 Logistics..........................................................................................................38 C.7 Information Management ...............................................................................39 C.8 Technical and Administrative Support ...........................................................39 C.9 Mission Collaboration ....................................................................................40 C.10 Management Lessons Learned .......................................................................40 D. OTHER ISSUES IDENTIFIED BY THE TEAM...................................................... 40 E. CONCLUSIONS AND RECOMMENDATIONS...................................................... 41 E.1 Comments to USAID......................................................................................42 E.2 World Relief’s Plans for Using Best Practices and Lessons Learned ............44 E.3 Potential for scale-up and expansion of the project........................................45 2 World Relief/Rwanda Child Survival Final Evaluation December 2006 F. RESULTS HIGHLIGHT: INCREASING WOMEN’S EMPOWERMENT IN THE FAMILY AND CIVIL SOCIETY.................................................................................... 46 G. ATTACHMENTS....................................................................................................... 48 ANNEX A.1 EVALUATION TEAM MEMBERS...................................................48 ANNEX A.2 LOCAL DISSEMINATION MEETING ATTENDANCE LIST ......49 ANNEX B. FINAL KPC REPORT............................................................................. i ANNEX C. EVALUATION ASSESSMENT METHODOLOGY AND PERSONS INTERVIEWED AND/OR CONTACTED ...............................................................53 ANNEX E. CD-ROM WITH ELECTRONIC COPY OF THE REPORT................55 ANNEX F. SPECIAL REPORTS .............................................................................56 ANNEX F.1 SUSTAINABILITY PLAN............................................................ 56 ANNEX F.2 SUMMARY REPORT ON HBM EVALUATION CONDUCTED IN NYAMASHEKE DISTRICT .......................................................................... 59 ANNEX G. PUBLICATIONS AND PRESENTATIONS........................................63 ANNEX H. UPDATED PROJECT DATA FORM ..................................................64 3 World Relief/Rwanda Child Survival Final Evaluation December 2006 ACRONYMS AND DEFINITIONS ACT Artemisinine-based Combination Therapy ART Antiretroviral Therapy ANC Antenatal care AQ/SP Combination therapy with Amodioquine and Sulfadoxine/Pyrimethamine ARI Acute respiratory infection BCC Behavior change communication Concern Concern Worldwide (another US based PVO) Cell Geographic unit with approximately 100 households CCM Community Case Management C-IMCI Community Integrated Management of Child Illness CSP Child survival project DHS Demographic health survey DHT District Health Team DIP Detailed implementation plan DPT Diphtheria, pertussis and tetanus (immunization) EOP End of project EBF Exclusive breastfeeding EIP Expanded Impact Program EPI Expanded Program on Immunization GAVI Global Alliance for Vaccines and Immunization GFATM Global Fund to Fight AIDS, Tuberculosis and Malaria GNS Good nutritional status (“green” zone on Road to Health cards) HIV Human immunodeficiency virus HBC Home Based Care HBM Home Based Management (Malaria) HIS Health Information System IMCI Integrated Management of Childhood Illness IRC International Rescue Committee ITN Insecticide-treated bed net KHD Kibogora Health District KPC Knowledge, practice and coverage LLIN Long Life Insecticide Impregnated Net LRA Local rapid assessment ORS Oral rehydration solution ORT Oral rehydration therapy OVC Orphans and Vulnerable Children PEPFAR President’s Emergency Plan for AIDS Relief PLWHA Persons Living with HIV/AIDS PMI President’s Malaria Initiative PNLP National Malaria Control Program PMTCT Prevention of mother-to-child transmission (of HIV infection) POU Point of Use (water treatment) PVO Private Voluntary Organization (A US-based international NGO) PSI Population Services International 4 World Relief/Rwanda Child Survival Final Evaluation December 2006 RBM Roll Back Malaria Sector Geographic unit with approximately five cells STI Sexually transmitted infection TBA Traditional birth attendant TOT Training of trainers TT Tetanus toxoid (immunization) UNICEF United Nations Children's Fund USAID United States Agency for International Development WHO World Health Organization VCT Voluntary counseling and testing (for HIV infection) Sûr'Eau A solution of 0.5% sodium hypochlorite that disinfects contaminated water and is promoted by PSI for point-of-use water treatment 5 World Relief/Rwanda Child Survival Final Evaluation December 2006 ACKNOWLEDGEMENTS The evaluation team would like to thank all of the partners and stakeholders, especially members of the national Ministry of Health and the former Kibogora Health District and District Administration (now part of Nyamasheke District) for their enthusiastic participation in the final evaluation fieldwork and dissemination. We would also like to thank the community volunteers whose hard work and dedication are an inspiration to all of us who work in trying to improve the health status of families and communities worldwide. 6 World Relief/Rwanda Child Survival Final Evaluation December 2006 A. Executive Summary The World Relief Rwanda “Umucyo” Child Survival Program targeted Kibogora Health District in Cyangugu Province, located on the western border of Rwanda on the shores of Lake Kivu. The Umucyo CSP partnered with the health district of Kibogora in targeting 57,505 direct beneficiaries (33,484 women aged 15-49, and 24,021 children under five years old). Overall program objectives are: (1) Reduce morbidity and mortality among children 0-5years and women of childbearing age; (2) Strengthen the capacity of the Kibogora Health District (KHD) to implement and sustain Child Survival interventions, and (3) Empower communities to make decisions to improve their health status. At the end of the project, Umucyo had met, and in some cases significantly exceeded all targets. In addition, the strong health monitoring and information systems enabled the project to track progress towards objectives on a regular basis and demonstrate the impact of the program in national level meetings. Mortality and morbidity data tracked in this system helped to verify the findings in the final KPC that demonstrated significant impact. Spot checks of immunization cards in families living far away from health centers also helped to verify the findings. In addition to introducing the Care Group model to Rwanda, the CSP also introduced two innovative CS methodologies: the PD/Hearth approach to community based nutrition rehabilitation for malnourished children and, along with Concern Worldwide and the International Rescue Committee (IRC), Home Based Management (HBM) of fever (suspected malaria). World Relief expanded on the initial objective of PD/Hearth and it became a means of forming women’s groups centered on child nutrition and health that will continue after the program. HBM proved to be very effective and was credited with significantly decreasing malaria morbidity and mortality by the District Hospital and an assessment conducted by BASICS and PNLP, the National Malaria Control Program. The effectiveness of the Care Group approach for community mobilization has now been verified and documented in multiple settings including Mozambique, Malawi, Cambodia, and now Rwanda. There is no question that this approach not only has positive effects on maternal and child health incomes, but also serves to empower women to participate in civil society and improve the overall household and community quality of life. Focus group discussions with community members, Care Group volunteers, pastors and community leaders confirm that they believe that the household health and hygiene behaviors will be maintained after the end of the program. Health facility utilization, encouraged by significant mutuelle1 membership, is also likely to remain significantly above baseline levels. However, the overall poverty of the area, accompanied by significant food insecurity, threatens the maintenance of improved health status of mothers and children at the community level. 1 National health insurance program 7 World Relief/Rwanda Child Survival Final Evaluation December 2006 Health systems and health worker performance were significantly strengthened by the project. Turnover of continuing supervision for community health activities will continue after CSP funding ends, supported with a cost-share between World Relief and the Health Center Management Committees (COSAs). A small number of the CSP staff members will be employed by the health centers. While Kibogora District is one of six districts included in the new Expanded Impact Program that will be implemented along with Concern Worldwide and the IRC, the amount of support to the project area will decrease significantly. World Relief has established an extremely effective community-based framework for decreasing maternal and child mortality and increasing the quality of life overall in their project area. WHO has already recognized the District as the first in Rwanda to reach Abuja Roll Back Malaria targets for ITN coverage and timely care-seeking for fever. The Kibogora District Health Team gives the credit for these achievements to World Relief’s program. Other organizations such as the Salvation Army World Service Organization, Food for the Hungry, and the American Red Cross are replicating the Care Group model in other countries, and World Relief has plan for scaling up the approach in other programs. World Relief has demonstrated that valuable data collection can be done by community members without undue burdens being placed on these volunteers. The methodology of developing a community-based HIS that complements the MOH so that achievements are captured and shared should also be documented and disseminated. Lessons learned in the CSP will contribute substantially to the design of the new Expanded Impact Program. For the Care Group model to impact future child survival and other health programs, World Relief would need to seek assistance to develop cost estimates of the essential components of the model, including initiation and eventual scale-up. This information could be disseminated to other PVOs, but also shared with international programs and Ministries of Health who are currently struggling to develop strategies to meet health targets in the Millennium Development Goals. 8 World Relief/Rwanda Child Survival Final Evaluation December 2006 B. Assessment of Results and Impact of the Project Overview The Umucyo Child Survival Project's goals were: • To reduce morbidity and mortality in children under 5 years of age and in women 15-49 years of age; • To strengthen the capacity of the Kibogora Health District (KHD) to implement and sustain child survival (CS) interventions; and, • To empower communities to make decisions to improve their health. The overall under-five mortality rate for Rwanda at the beginning of the program was 196 deaths per 1,000 live births (and 216 for rural Rwanda), and the overall maternal mortality ratio was 787-1,300 per 100,000 births, giving the country some of the highest rates of child and maternal mortality in the world.2 The rates within the KHD were presumed to have been similar to the national rates at the outset of the Project. The national HIV seroprevalence rate was estimated to be 11%. Malaria is the leading cause of under-five mortality, and diarrhea is the third leading cause. Childhood malnutrition was common, with 42% of children under-five nationally moderately or severely stunted. The baseline knowledge, practice and coverage (KPC) survey for the Project demonstrated that only 47% of children 12-23 months of age were fully vaccinated by their first birthday (according to their immunization card), and only 43% of women with a child 0-23 months of age had received two doses of TT before the child's birth (according to their immunization card). The project’s major strategies and End of Project (EOP) objectives were: ♦ HIV/AIDS (20% effort): Community-wide education in HIV/STI prevention, promotion of voluntary counseling and testing, and home care. ♦ 80% of women will know at least two common symptoms of STIs (other than HIV/AIDS.) ♦ 80% of women willing to care for a relative with AIDS in their own household ♦ Malaria (20% effort): Community-wide education in malaria prevention and treatment seeking behaviors; improved access to Insecticide Treated Nets (ITNs) and re-treatment. ♦ 50% of children with fever (suspected malaria) will be treated within 24 hours at health facility. ♦ 40% of children < age 2 and pregnant women will be sleeping under an ITN. ♦ Nutrition and Breastfeeding (BF) Promotion (15% and 5% effort respectively). Community-wide education to promote improved infant and child feeding, community-based rehabilitation of malnourished children through Hearth, and Vitamin A Capsule (VAC) distribution at EPI clinics. 2 Rwanda Enquête Démographique et de Santé 2000 (ESDR-II). Office National de la Population, Kigali, Rwanda; ORC Macro, Calverton, MD, USA, 2001. 9 World Relief/Rwanda Child Survival Final Evaluation December 2006 ♦ 50% of mothers will give appropriate weaning foods (enriched porridge) at least once/day. ♦ 60% of mothers will offer same amount or more food to child during illness. ♦ 80% of children who completed the Hearth program achieve and sustain adequate or catch-up growth for at least 2 months after Hearth. ♦ 80% of children 6-59 mo. will receive 1 dose of Vitamin A capsules per year. ♦ 40% of children 6-59 mo. will receive 2 doses of Vitamin A capsules per year. ♦ Diarrhea (15% effort): Education to improve hygiene and home treatment of diarrhea using Oral Rehydration Therapy (ORT), improved access to Oral Rehydration Solution (ORS), and training of drug sellers to improve rational drug use. ♦ 50% of children with diarrhea will be treated with more fluids than usual. ♦ 75% of mothers will know at least three danger signs of diarrhea requiring medical treatment. ♦ Immunization (15% effort): Community-wide education and expansion of mobile EPI clinics to improve access to services. ♦ 75% of children will be completely immunized by 1 year of age for polio, DPT, Tetanus Toxoid (TT), and measles. ♦ 50% of pregnant women in project area will receive at least 2 doses of TT before the birth of a child. ♦ Maternal and Newborn Care (10% effort): Promotion of safe delivery via Traditional Birth Attendant (TBA) training, improvements in quality of care, and assisting communities to plan for obstetric emergencies. ♦ 50% of women will give birth at a health facility or with a trained TBA. ♦ 70% of pregnant women will have emergency plan in place before delivery. B.1 Results of Program Efforts compared to Baseline and Targets IMCI Sick Child Baseline KPC% Midterm KPC% Final KPC% EOP Target 1. Percentage of mothers who know at least three signs indicating a sick child needs treatment at health facility. 92.3% 98.0% 99.3% N/A 2. Proportion of mothers who give increased liquid to sick child. N/A 56.8% 90.0% 60% 3. Proportion of mothers who give increased feeding to sick children. 3.7% 67.4% 90.0% 60% Malaria & Care Seeking Baseline KPC% Midterm KPC% Final KPC% EOP Target 1. Children age 0-23 months who slept under insecticide treated mosquito netting within last 24 hours. 3.0% 66.0% 69.7% 40% 2. Pregnant women who slept under mosquito net within last 24 hours. 3.5% 64.5% 77.8% 40% 3. Mothers who take child age 0-23 months 3.7% 31.2% 80.4% 50% 10 World Relief/Rwanda Child Survival Final Evaluation December 2006 with suspected malaria for treatment within 24 hours of fever either at a health facility OR a distributor (83% of treated children went to a health facility; 17% went to distributor) (health facility only) (health facility only) 4. Fever: 2-week prevalence among children under five. 74.8% 28.7% 17.3% N/A 5. Percentage of women who subscribe to community health insurance N/A N/A 86.7% N/A Diarrhea & Hygiene Baseline KPC% Midterm KPC% Final KPC% EOP Target 1. Proportion of mothers who wash hands with soap or ash before preparing food, before feeding child, after defecation and after attending to a child who has defecated. 0.0% 39.3% (3 occasion s) 94.0% (4 occasio ns) N/A 2. Proportion of mothers with children from 0-23 months who increase fluids for a child with diarrhea 31.0% 66.7% 87.2% 50% 3. Proportion of mothers who know the danger signs of diarrhea. 83% (2 signs) 77.7% (3 signs) 97.7% (3 signs) 75% (3 signs) 4. Diarrhea: 2-week prevalence among children under five. 66.4% 26.7% 13.0% N/A 5. % of mothers of children aged 0-23 months who make available soap for washing hands (seen by interviewer) N/A N/A 97.0% N/A 6. % of mothers of children aged 0-23 months who have a latrine in good condition N/A N/A 84.0% N/A 7. % of mothers of children aged 0-23 months who have rubbish pit in the household (seen by interviewer) N/A N/A 94.0% N/A 8. % of mothers of children aged 0-23 months with a dish rack in their household (seen by interviewer) N/A N/A 89.0% N/A Nutrition Baseline KPC% Midterm KPC% Final KPC% EOP Target 1. Children 6-23 months who have received at least 1 dose of Vitamin A per year (evidenced by card) 33.4% 15.4% 83.4% 80% 2. Nutritional status of children 0-23 months: Children with adequate weight for age ( >-2 SD). 84.1% 71.7% 85.3% N/A 3. Nutritional status of children 0-23 months: Children who are underweight (not maintaining adequate weight for age -<2 SD) 15.9% 28.3% 14.7% N/A 11 World Relief/Rwanda Child Survival Final Evaluation December 2006 4. Mothers who have prepared an appropriate weaning meal for a child 0-23 months (as reported within last 24h). 19.5% 54.2% 87.9% (child 6-23 months receivin g breast￾milk and comple mentary foods in last 24 hours) 50% 4. Exclusive breastfeeding for children 0-5 months (as reported for last 24h) 60.3% 77.0% 99.0% 75% 5. Mothers who initiate breastfeeding within first hour after delivery. 37.7% 71.3% 90.0% N/A 6. From monitoring data (not KPC): Children who maintain adequate or catch up growth after 1st Hearth session. (requires growth monitoring of children who have completed Hearth rehabilitation sessions. Midterm KPC data provides first measurement) N/A 53.3% 67.8% 80% Immunization Baseline KPC% Midterm KPC% Final KPC% EOP Target 1. % of children age 12-23 months who are fully vaccinated (against the five vaccine￾preventable diseases) before the first birthday. 47.1% 85.5% 96.6% 75.0% 2. Children age 9–23 months who received a measles vaccine 76.6% 87.1% 97.0% N/A 2. Maternal TT: mothers who received at least 2 doses before birth of last child as evidenced by card. 43.8% 27.7% 51.3% 50.0% (2d) 3. Maternal TT: mothers who received at least 2 doses as reported by mother missing her card N/A 54.3% 41.7% N/A Reproductive Health Baseline KPC% Midterm KPC% Final KPC% EOP Target 1. Families with an emergency transport plan in place prior to last delivery (ambulance/ hammock). 24% 46.7% 79.0% 70% 2. Women with permission to execute emergency transport plan prior to last 16.4% 42.7% 95.7% 70% 12 World Relief/Rwanda Child Survival Final Evaluation December 2006 delivery. 3. Women who report completing at least 3 prenatal check-ups during last pregnancy. N/A 45.0% 91.0% N/A 4. Women who delivered last child at health facility w/ doctor, nurse, auxiliary nurse. 23.1% 35% 72.3% 50% 5. Women who have given birth to a second child more than 24 months after the birth of the previous child 73.1% 77.% 85.7% N/A HIV/AIDS & STI Baseline KPC% Midterm KPC% Final KPC% EOP Target 1. Women who know at least 2 methods of HIV/AIDS prevention. 80% 91.3% 99.3% (3 method s) N/A 2. Women who know at least 2 symptoms of STIs 47.0% 92.3% 93.3% 80% 3. Women who report willingness to care for a person with AIDS in their own home. N/A 96.7% 99.3% 80% B.2 Results: Technical Approach The project strategy, intervention mix and location were all well described in the 2004 MTE report prepared by Dr. Henry Perry. By the time of the MTE, the program approach had already proven to be very effective. The findings of the Final Evaluation concur with those findings. This report will focus on describing the changes, improvements, challenges that have emerged since the Midterm Evaluation, or that reflect overall on the project from the beginning. B.2.a HIV/AIDS Knowledge about prevention for HIV/AIDS and STIs is now high. As in most African countries, many programs have focused on raising awareness of the causes and prevention of HIV/AIDS. World Relief contributed to significant increases in access to VCT services in the district and secured separate funding for additional HIV/AIDS programs in the project area, such as an adult Hearth group for PLWHA. Stigma has decreased; most people are now willing to relate to and even care for people with HIV in their homes. Mothers responded well to VCT and PMTCT. People living with HIV/AIDS (PLWHA) now know about treatment with ARVs and also know where to go for testing and treatment. (Four health centers now provide VCT, PMTCT and ARVs). A small number of PLWHA from each health center catchment area participated in an adult Hearth. Rwanda is now receiving assistance through PEPFAR and other HIV/AIDS treatment programs. World Relief is not directly involved in treatment programs at this time. 13 World Relief/Rwanda Child Survival Final Evaluation December 2006 VCT utilization at Kibogora Hospital 0 2,000 4,000 6,000 8,000 10,000 2001 2002 2003 2004 2005 2006 (Jan-Jun) Number of people tested Summary of CSP Syphilis Activities for Since the MTE Report by Dr. Henry Perry Prior to the mid-term evaluation, syphilis testing and treatment were available at only one site in the project area (Kibogora District Hospital). CSP staff acted on Dr. Perry’s recommendation to increase access to these services and successfully advocated with district leaders for additional sites. In addition to the hospital, four health centers now offer syphilis testing and treatment. Messages on syphilis testing and treatment were also reinforced through family education on VCT and PMTCT. The CSP began collecting monthly service statistics from health centers and the hospital to track the number of people accessing testing services, and to examine the relationship between HIV and syphilis in the population. The following graphs show: (a) the increase in the number of pregnant women tested during ANC with the additional testing sites; and (b) trends in HIV and RPR serostatus over five years of VCT and syphilis testing in the project area. Although VCT is now free, when fees were still charged for testing the project paid for 1500 malnourished individuals and all care group volunteers to be tested. The project also trained volunteers in HBC. They also trained teachers to share HIV prevention messages with school children (age-appropriate education focusing primarily on abstinence), and formed 18 anti-AIDS clubs, which promote positive living and risk reduction behaviors for everyone, regardless of HIV status. 14 World Relief/Rwanda Child Survival Final Evaluation December 2006 Pregnant Women Tested for Syphilis in CSP Project Area (2002-2006) 0 500 1000 1500 2000 2500 3000 3500 4000 2002 (Mar-Dec) 2003 2004 2005 2006 (Jan-Jun) Year Number of pregnant women tested HIV and Syphilis among Pregnant Women in Project Area (2002-2006) 0% 1% 2% 3% 4% 5% 6% 7% 2002 (Mar-Dec) 2003 2004 2005 2006 (Jan-Jun) Year Percent testing positive RPR+ HIV+ HIV+ and RPR+ Challenges and Threats There are gender differences in the uptake of VCT. Community members said that although VCT has increased among men, they are less likely to go for VCT than their 15 World Relief/Rwanda Child Survival Final Evaluation December 2006 wives. Some men reportedly send their wives for testing and assume their situation is the same as hers. Some community health workers reported that a small number of HIV+ men who know their status continue to engage in unsafe behaviors. Child-headed households were not specifically a focus of the CSP, though Care Group volunteers say that they know these households and try to reach out specifically to these children. This is an area that should receive specific attention by district health services after the program ends. B.2.b Immunization The CSP assisted outreach programs by transporting staff and materials and mobilizing the community to access immunization services. Program partners reached more than 370 vaccination sites in one year. This support also enabled the Health Centers (HC) to start eight new immunization outreach sites. Project staff also helped to improve MOH immunization data recording, which had traditionally been very weak. Additional CS activities, such as deworming, growth monitoring, Vitamin A distribution, antenatal check-ups, ITNs, ORS and Sûr’Eau sales were integrated into immunization outreach, thereby expanding the coverage for several CS interventions. The KPC results showed almost universal immunization coverage in the project area and the majority of children are fully immunized on schedule. Spot checks of immunization cards for families living a long distance from health centers confirmed that the coverage is actually extremely high. Factors that contributed to the high coverage Volunteers worked with every household in the project area and reminded families when to take their children to the outreach sessions for services. Volunteers also helped to organize the outreach sessions in the communities. When the HC saw that certain children had not come for immunizations they provided a list of these children to the volunteer in the community who would find them and bring them for their vaccinations. In addition, when the volunteer made home visits they checked the immunization status of each child in the household. Care Group volunteers also tracked all births in the community. Coordination between facilities and volunteers was excellent and staff stated that because of this they were able to plan and achieve better coverage. The CSP (and the District with GAVI funds) supported transport to increase frequency of sessions. Mothers interviewed during the FE said, “Now that the children are immunized there is less measles.” The community has also created demand for immunization and, together with the COSAs, they can help to hold the health centers accountable to provide quality services. The project has also enabled community members to voice complaints related to health services, when they have them, at regular community meetings. At the time of the FE, however, community members were very satisfied with immunization services. 16 World Relief/Rwanda Child Survival Final Evaluation December 2006 Early in the program, mothers used to be afraid of the side effects of immunization. The evaluation team found that mothers now understand that a fever can be normal and will go away in one day. The team concluded that increased education through the volunteers has helped mothers to overcome their fears, and increased demand for immunization. Motivation for TT and ANC in general was enhanced by the distribution of ITNs as an incentive to attend ANC. Mothers not only had received TT, but many also knew that five TT shots would confer lifetime protection. Lack of maternal health cards was the major reason that TT coverage was difficult to document. Some, but not all of the mothers, knew the benefits of TT for the mother and baby. The exact percentage was not measured in the surveys. When they did not know, mothers said that the health center staff did not explain it to them. This would be an area for improvement for the District Health Services after the program ends. Documenting maternal TT coverage remains the major area of improvement needed in immunization services in the district. The performance contracting approach introduced by CordAid/Memisa, and other organizations in Rwanda and supported by USAID has been adopted by the District Health Services. It provides incentives to HC staff to keep EPI coverage high. This will reward health centers for maintaining coverage, though it will be more challenging once the project ends. Lack of transport post-project will be a threat to sustainability. GAVI support is expected to continue after the CSP ends, but the details were not available at the district level at the time of the FE. In response to the direct question about whether coverage might decrease if volunteers stop reminding and accompanying children to outreach, respondents reported that there is no reason why the volunteers could not continue doing this because demand is so high and the benefits are now so obvious. B.2.c Reproductive Health/Maternal and Newborn Care To discourage home births, all community health workers (CHWs) were mobilized to follow and accompany pregnant women to give birth in the health center. A network of maternal health animators was developed. They consisted of health animators, traditional birth attendants (TBAs), and the 404 CSP volunteers chosen by association (2 per association) to follow pregnant women within the ‘action zone’ of their respective associations. CHWs, including former TBAs who accompany pregnant women to the health center for delivery receive an incentive of about $1. Another strategy adopted by the KHD is to make HC delivery free of cost to those women who attend a designated minimum number of ANC visits. A significant increase in the percent of mothers delivering at a health facility was documented in monitoring and evaluation surveys. Decreasing the extremely high Maternal Mortality in African countries is a Millennium Development Goal that has been targeted for specific attention by the United Nations. Although the project population was small, documenting decreases in maternal deaths would have been desirable. The WR 17 World Relief/Rwanda Child Survival Final Evaluation December 2006 CSP, however, did not specifically measure MMR but captured mortality in the target population of WRA. The data did show a downward trend in the mortality rate among this beneficiary population, but it was not cause-specific. There is general consensus among reproductive health experts, however, that deaths among pregnant women during delivery would be a direct result of the observed increased numbers of skilled deliveries at health centers, which was an indicator that was measured by the program. Challenges Prior to the CSP, the MOH trained and equipped TBAs with delivery kits so that they could assist with some emergency deliveries in the community. The national policy changed during the life of the project and the only role that TBAs are now allowed to play is the same role as other health animators: to accompany pregnant women to HF for delivery and follow them up when they return home. The project helped to facilitate this shift in the project area by training TBAs in their new role and involving them in other project activities. In Rwandan culture, women are not usually comfortable revealing their pregnancy in the early stage. Prior to the program, most women would delay attending ANC until later in pregnancy, thereby keeping ANC coverage rates in the first trimester low. The CSP responded to this by making mosquito nets (ITNs) available at ANC to motivate women to attend in the first trimester of pregnancy. KHD’s strategy has also improved to encourage early and regular ANC attendance to qualify for free delivery care in health facilities. There are still challenges for health centers to support referrals of pregnant mothers who are not mutuelle members and need to go to the hospital from some health centers. There were discrepancies in prices quoted for ambulance transport and whether credit would or would not be extended to a woman who could not pay. Even mutuelle members have to pay for the ambulance, but the cost is lower. Cost for non-mutuelle members to use the ambulance is 20,000 francs (vs. 4,000 francs for member). It is not the responsibility of the CSP to solve this problem, but the cost may hinder the ability of a beneficiary or her family to follow through with the birth plans promoted by the program. Supposedly, if the mother does not pay back the costs, it is a burden on the HC. Mutuelle membership is a particular challenge for those poor families who do not qualify for government-sponsored membership yet are not wealthy enough to buy their own insurance. B.2.d Diarrhea The project made major contributions to reductions in diarrhea disease mortality and morbidity through (1) major promotion of household and personal hygiene improvements; and (2) promoting household point-of-use-water treatment using Sûr’Eau (a chlorine-based product distributed by PSI) and/or boiling water. 18 World Relief/Rwanda Child Survival Final Evaluation December 2006 PSI's grant for Sûr’Eau ended a few months prior to the end of the CSP, but they have received additional funds from PSI-Washington to produce and distribute 25,000 bottles of Sûr’Eau, which they hope will cover the period until another grant is secured. There was a gap in availability of Sûr’Eau while this was worked out. But as of a meeting between World Relief and PSI in September 2006, it was understood that Sûr’Eau would soon be available, at least in limited quantities. But changes in the source of funding mean that the subsidy that can be provided will be reduced and the price to the consumer will increase. Though small, the price increase is not ideal, particularly since it comes along with repackaging the product in a smaller container. One bottle will treat the same amount of water, but the cognitive shift required of consumers (paying more for what would be appear to be less product) is unfortunate. PSI has assured World Relief that based on WR’s existing MOU with PSI, once Sûr’Eau becomes available again World Relief will be able to order Sûr’Eau at a special distributor price. World Relief also discussed their vast distribution network and the possibility of ordering Sûr’Eau (or other products) for distribution to health centers. The staff in the health centers that are part of the sustainability transition would deliver the products to the established CSP Umucyo distributors who could continue selling them in the community and sustain the CSP impact. To enhance communities’ capacity to manage diarrhea episodes at home, Umucyo collaborated with UNICEF Rwanda and made 10,000 packages of ORS available for distribution to households and also promoted the use of home-prepared fluids (such as rice-based gruel) when ORS packets were not available. Results Diarrhea Prevention and Management 0% 20% 40% 60% 80% 100% Handwashing (3+) Increased fluids for child with diarrhea 2 Week Prevalence of Diarrhea in U5s Baseline Midterm Final 19 World Relief/Rwanda Child Survival Final Evaluation December 2006 Although the two-week diarrhea prevalence decreased significantly, measurements at baseline and at the end of the program were done during different seasons and therefore are not entirely comparable. However, important preventive behaviors increased significantly. Appropriate hand washing behaviors increased from 39% at midterm to 94% in the final survey. Data collection for this indicator included a visual check on the presence of soap in the household. B.2.e Nutrition and Breastfeeding Two major strategies helped to change nutrition behaviors for young children and pregnant women. Information on diet diversification and the suitability of locally available foods was spread through volunteers and health workers. PD/Hearth identified foods that were fed to healthy children in the same environment and demonstrated these through the PD/Hearth group rehabilitation sessions conducted in each community. Very significantly, strong emphasis was placed on feeding children more food when sick and this alone would account for improving the nutritional status of children in the project area. Mothers of children <2 yrs who increase feeding for sick children 0% 20% 40% 60% 80% 100% Baseline KPC Midterm KPC Final KPC Rwanda DHS 2001 2004 2006 2005 PD/Hearth was initially used to rehabilitate children in the “red” or “yellow” zones on the Road to Health card as determined during growth monitoring sessions. But mothers and children in the sessions often continued to meet and cook together after their children were sufficiently rehabilitated to no longer “need” to be in the program. Some of the PD foods identified were fish, cassava leaves, guinea pigs and eggs. Mothers who were interviewed stated that prior to the program they really didn’t pay much attention to what they fed their children, they rarely mixed foods, and they usually fed children only once a day. 20 World Relief/Rwanda Child Survival Final Evaluation December 2006 Results of the Hearth program varied in different areas of the project. The following chart shows preliminary analysis from data from Hearth sessions in various places. Kibogora Karambi Ruheru Gatare Rangiro Nyamasheke Yove Hanika Total Number of children who began program 755 618 613 648 433 619 557 649 4892 Number of children who died during the program 3 2 3 4 3 2 5 2 24 Percentage of children who died 0.41% 0.33% 0.51% 0.63% 0.72% 0.35% 1.05% 0.32% 0.51% Number of children who dropped out or were absent 15 5 26 7 11 48 74 18 204 Percentage of children who dropped out or were absent 1.99% 0.81% 4.2% 1.08% 2.54% 7.75% 13.3% 2.77% 4.17% Total number of children who completed a Hearth session 737 611 584 637 419 569 478 629 4664 Number of children in good nutritional status following Hearth session 484 407 412 367 340 428 238 488 3164 Percentage of children rehabilitated 65.7% 66.6% 70.6% 57.6% 81.2% 75.2% 49.8% 77.6% 67.8% Source: Christine Brackett Over 67% of children participating in Hearth gained at least 200 grams and more significantly became more active and playful. The latter changes were more obvious to mothers than weight gain and motivated them to continue bringing their children. The Mama Lumieres, or Hearth session leader mothers, as well as Hearth mothers have formed associations and have started income-generating activities with or without assistance from World Relief. These activities have included providing small numbers of breeding animals such as guinea pigs, rabbits and goats. Christine Brackett (cbrackett@exchange.fullerton.edu), a graduate student at California State University￾Fullerton, is writing a more detailed report analyzing the Hearth program results in each area of the project for her master’s thesis. This report will be available in June 2007. Results The percentage of infants age 0-6 months who were exclusively breastfed in the last 24 hours rose from 60% to 99% and 90% of mothers breastfed their babies during the first 21 World Relief/Rwanda Child Survival Final Evaluation December 2006 hour after delivery, up from 38% at baseline. Children 6-24 month of age who received at least one dose of Vitamin A rose from 33% to 83%. Challenges Food insecurity problems in some areas remain a threat and may undermine the gains from the nutrition behavior changes that were achieved by the program. Restrictions on fishing in Lake Kivu and on access to lands now part of Nyungwe National Forest keep some communities away from their usual food sources and the effected families will need alternative sources of income and food to avoid increases in malnutrition in those areas. There also are some orphans in the area who are being cared for by grandmothers and they are also at higher risk for becoming malnourished. B.2.f Malaria B.2.f.i Prevention: Increasing access to ITNs The project significantly increased access to ITNs, through distribution at ANC, community sales, and mobilization for participation in national campaigns. All mothers interviewed have treated mosquito nets and interviewers observed the nets hanging in homes. Many households have three or more treated mosquito nets. The project also provided nets to the hospital and all health centers, as they did not have nets at the beginning of the project. Results Children <24 months who slept under ITN last night 0% 10% 20% 30% 40% 50% 60% 70% 80% Baseline KPC Midterm KPC Final KPC Rwanda DHS (under fives) 2001 2004 2006 2005 22 World Relief/Rwanda Child Survival Final Evaluation December 2006 Pregnant Women Who Slept Under an ITN Last Night 0% 20% 40% 60% 80% 100% Baseline KPC Midterm KPC Final KPC Rwanda DHS 2001 2004 2006 2005 Challenges There is a need to plan for the transition to long-life nets, which were provided in a national campaign in September 2006 (after the final KPC survey). CSP staff who will now be employed by health centers for sustained community mobilization should encourage people to take down their retreatable nets and replace it with the long-life nets (LLINs) that were distributed in the campaign. Supplies of retreatment kits are dwindling at the national level, but WR was able to order 1000 units of the retreatment product, Karishya Force, from PSI before the end of the project to assist families in retreating remaining ITNs. The long life nets will be effective for five years, however, and are the preferred option, as retreatment kits will be phased out in the near future. B.2.f.ii Home Based Management of Malaria (HBM) Background Based on programs they had seen in West Africa, especially the experiences in Senegal, discussion on community-based treatment of malaria began among the three CSHGP PVO headquarters (Concern, the IRC and World Relief) backstops in 2000. Each of the three PVO programs had allocated significant program effort to malaria interventions targeting children under 5 and women of child-bearing age. USAID Washington staff at the time was encouraging joint activity planning within various countries. During the same period, PVOs had already started discussions about creating a common model for community-based treatment of malaria. At the same time the CORE Group expressed interest in supporting collaboration among its PVO members within countries. At the spring 2003 CORE meeting, the availability of seed money was announced to members who were interested in pursuing joint child 23 World Relief/Rwanda Child Survival Final Evaluation December 2006 survival activities. This offer provided Concern Worldwide, the IRC, and World Relief with the impetus to discuss a potential partnership focused on malaria. World Relief and two other PVOS collaborated and developed an initial proposal for CORE, which was vetted and refined with input from CORE’s Malaria Working Group. Initially, the MOH was reluctant to support community-based case management models. In 2003, however, the government showed interest in developing a national malaria strategy that included community case management. This change in attitude was aided by: • Advocacy and support from the USAID Mission for the NGOs’ approach and involvement in implementation; • Development and release of a national malaria strategic plan; and • Advocacy from other organizations and donors, especially UNICEF, GFATM and Roll Back Malaria for more effort in the fight against malaria, and to consider piloting community-based programming. World Relief shared technical assistance consultations with the other PVOs at the country level even before CORE’s involvement began. World Relief brought Robb Davis of Freedom for Hunger to provide training on participatory adult learning and health education facilitation (all three agencies contributed to the consultant fees). At the USAID mission, Jules Mihigo strongly supported increased malaria programming and a using community-based treatment model. He and other Mission staff helped the NGOs to gain access to meetings with the PNLP, lent legitimacy to the proposed program, and contributed an additional $200,000 for implementation. World Relief took on the additional management burden and responsibility for channeling the funds to all three organizations. This is significant because World Relief does not have large amounts of private funds to easily accommodate the extra administrative costs. This demonstrates the value World Relief placed on supporting the start up of the HBM initiative. The Rwanda PNLP was the overall program leader. The CORE-funded pilot program was initially limited to the three NGOs and the area of the three health districts where they were working. The PNLP was sufficiently convinced of the value of the approach that they wanted to expand to additional areas within the health districts, and added three more districts to be managed by UNICEF. Since 2005, the PNLP has introduced HBM into another nine districts (totaling 12 of 30 nationwide). More details about the PVO HBM partnership are documented in a CORE paper released in July 2006, downloadable from CORE Group website (www.coregroup.org). Throughout the program, World Relief provided strong technical support to their field sites, via e-mail, phone, and in-person communication as needs arose, and communicated with the other two PVOs via email and phone conversations. But it was the face-to-face meetings, especially during concurrent HQ field visits, that were the most productive 24 World Relief/Rwanda Child Survival Final Evaluation December 2006 opportunities for coordinating with the two other PVOs. On the ground, the field staff actively participated in partner meetings and took the lead in fine tuning implementation of HBM to the specific context in Kibogora. All three of the PVOs were part of the National Technical Committee that was tasked with oversight of the HBM program. This Committee included PNLP, USAID, the Quality Assurance Project, UNICEF, PSI, and representatives from each of the districts. Meetings were held monthly, and committee members visited the HBM distributors periodically during the pilot study to observe case management, follow-up home visits and review of drug availability and conditions. USAID frequently facilitated the meetings between the partners. Activities started in 2004 with two health center staff per facility and five World Relief CSP staff. Distributors in their respective zones were selected by their communities. In the World Relief areas, some but not all of the distributors selected were pre-existing Care Group volunteers. Distributors were trained on various topics about managing simple cases of suspected malaria and recognizing and referring children with danger signs of malaria to health centers. They also followed-up children who were treated by them and by the health center. The distributors received yearly refresher training conducted by the District Health Management Team and health center staff. Coordination meetings were held three times a year where data were monitored for progress towards targets. Health center staff and CSP staff held monthly meetings with distributors to distribute supplies, collect and analyze data and solve problems. DHMT, health center staff and distributor representatives participated in exchange visits to Kibungo and Kirehe Districts to compare how the program was being implemented in different areas. These exchange visits demonstrated that distributors could probably expand to additional community-based activities, such as distributing zinc or ORS. This evidence will be used to support the C-IMCI design for the EIP. By September of 2005, 496 malaria drug distributors elected by the community were trained and equipped with kits that include medicine registers and referral cards. They distribute anti-malarial drugs to children with fever ages six to fifty-nine months. In the areas where the HBM strategy was operational, 85% of children under five are treated within 24 hours, and no deaths occurred after treatment by community drug distributors. The EIP will include a national scale-up and sustainability plan for home based treatment for malaria. BASICS and the PNLP conducted an assessment of the pilot phase of HBM, which will be used to inform the design of the next phase. The PNLP and MSH’s RPM+ will work to identify longer term funding mechanisms, including the Global Fund, and will advocate for including HBM services in the national social health insurance exemption programs (e.g. mutuelles) or direct cost recovery for longer term supply. In addition, the EIP proposal included some money for drugs (including malaria) in the budget. USAID’s new Presidential Malaria Initiative (PMI) has agreed to provide ACT drugs for facility level malaria treatment. One EIP district (Kirehe) will be covered by ACTs for HBM during PMI Year One (2007) and all four malaria endemic EIP districts 25 World Relief/Rwanda Child Survival Final Evaluation December 2006 (Kirehe, Ngoma, Nyamasheke, and Gisagara) will receive ACTs for HBM through PMI by the end of next year. Both community groups and sector/cellule leaders were universally enthusiastic about the services provided by the HBM distributors and stated that they feel that the quality of the drugs and the services provided were very good. Project data confirm that care seeking for a child within 24 hours of the onset of fever increased at both the community and health center level. This was confirmed in the HBM assessment done by the PNLP and BASICS in November 2006. Even prior to the introduction of HBM, the CSP had done significant community education on early care-seeking. Care-seeking within 24 hours of fever rose from 4% at baseline to 32% at midterm, and reached 80% at end of project with the assistance of the HBM program. Since children under the age of 6 months are not eligible for HBM, increases in care seeking at the health center reportedly increased from referrals by the HBM distributors as well as other community groups (such as TBAs and leaders) who were trained by the CSP. Focus groups conducted during the FE fieldwork revealed that the community recognizes the need to provide incentives (not necessarily monetary) to motivate distributors to continue their work at the same level of quality. Over the course of the project, community beliefs about the causes of malaria changed from attributing the symptoms of malaria to poisoning or curse to the understanding that mosquitoes were the cause. Places where families sought care shifted from to traditional healers, to drug treatment by distributors and health facilities. Even when much of the care seeking was at health centers, care was sought earlier in the course of the illness when the symptoms were less severe. The project and DHMT HMIS and distributor data were used to determine the numbers and ages of children who were treated and to order supplies. In addition, data about the percentage of children who recovered and/or were referred per month were collected. This information was used to assess the quality of their work and to provide data for the PNLP. Distributors were also expected to follow up their clients who had been referred to the health centers Distributors are now very knowledgeable about the malaria situation in their communities and are able to disseminate information with confidence to both the community and to the health centers. Health center staff can now analyze trends in malaria in their respective catchment areas from the reports provided by distributors. They also use the information collected to determine the strengths, weaknesses, and opportunities in the HBM intervention and decide what to do to improve services. Information from data collection is shared with the general population to make them aware of the situation and is credited with encouraging community members to promptly seek treatment and also to use ITNs to prevent malaria. 26 World Relief/Rwanda Child Survival Final Evaluation December 2006 Results Mothers who take child < 2 yrs to trained provider within 24 hrs of fever 0% 20% 40% 60% 80% 100% Baseline KPC Midterm KPC Final KPC Rwanda DHS West Province (<5s, <48 hrs) 2001 2004 2006 2005 A focus group with 15 beneficiary mothers showed that 13of them were members of a mutuelle. They usually get treatment at health centers (14) or distributors (33 ). They all know the danger signs of malaria. Eight of the mothers had consulted a distributor (a sign of confidence), even when the child was ultimately treated at the health center. All mothers know the danger signs of malaria. Distributors treat the right category of children (6-59 months) and the mothers say that they appreciate HBM especially because they are near to them. In every case of fever they have sought treatment, either at a health center or from a distributor. An important shift in understanding the cause of malaria has occurred. Many people used to think that convulsions were caused by demons. Now, as a result of the education received from the project, even grandmothers know that malaria can cause convulsions in children. Distributors say that they appreciate their work for these reasons: o They have gained more knowledge about malaria from the project. o They are happy that deaths and malaria cases have decreased. o The program has discouraged charlatans (those who sell medicines illegally). o Mothers seek treatment promptly when children have fever. o Children are treated by distributors near their homes. o HBM helps with other illnesses as well because distributors are able to refer sick children to health center for other illnesses. Distributors are committed to continuing their work even after the program ends, if they continue having some refresher training and the drugs are available. Distributors formed associations (separate from Care Group associations). Illegal drug sellers, sources of ineffective or counterfeit drugs in the past, were displaced by HBM distributors. HBM 3 Totals can be more than 15 because of multiple episodes or different children. 27 World Relief/Rwanda Child Survival Final Evaluation December 2006 has increased equity of access to effective treatment: families can purchase effective anti￾malarial drugs for a small fee, and most distributors reported that they would provide drugs on credit to families who could not pay. Religious leaders who were interviewed reported that people appreciate the home based management because there are no stockouts and the distributors are available in the community. When families approach them for help, pastors pray with the family and then refer the mother and sick child to a distributor or the health center. Distributors referred severe malaria cases to the health centers, ensuring appropriate care and strengthening the program’s impact on morbidity and mortality. HC staff report an increase in early care-seeking, which has decreased the number of severe malaria cases that resulted from delays in the past. The DHT credited early recognition and referral by the CSP as one of the reasons that the WR CSP receives a lot of the credit for the decreased malaria mortality at the hospital and health centers. Challenges Some mothers said that they are too poor to subscribe to a mutuelle. The price of the mutuelle has increased and they fear that enrollment might decline. Project staffs are concerned about the impact this may have on care-seeking at facilities. This increase in prompt treatment-seeking is attributed both to volunteer mobilization and education in the community, and to mutuelle membership, as enrolled families are no longer afraid that they will not be able to afford treatment. Distributors say they need batteries for flashlights to treat children at night. They also need raincoats, boots and/or umbrella. (Some of the distributors are also Care Group volunteers and they received umbrellas last year. The remaining 2/3 of 500 distributors did not get one. This is a threat to distributor activity during the rainy season when malaria increases, as distributors may be less willing to venture out to provide treatment. B.2.f.iii Intermittent Presumptive Treatment (IPT) IPT was not part of the original CSP as there was no national policy in support of it at the time the project was designed. Rwanda changed the policy 2005. Umucyo participated in pre-implementation workshops, in preparation for launching the Intermittent Presumptive Treatment (IPT) in the country. The high final ANC coverage bodes well for this intervention as long as drug supplies remain reliable. 28 World Relief/Rwanda Child Survival Final Evaluation December 2006 B.3 Cross Cutting Approaches B.3.a Community Mobilization By the end of the project, 233 Care Groups consisting of over 2800 committed volunteers formed the core of the project. In 2005, the average attendance rate for all care groups was 85 % and the volunteer default rate was less than 4%. Such high rates of participation and commitment are attributable to the supportive supervision the volunteers receive, as well as the positive reinforcement given by the community and district team for the volunteers’ contribution to maternal and child health. World Relief has given public recognition of the importance of the contribution that volunteers are making to improving health status in their communities. Additionally, WR recognizes the role that participation in the program has played in the volunteers’ own personal development. By helping volunteers at various levels to form associations, a government prerequisite for receiving development assistance from many programs, World Relief has taken important steps in fostering the sustainability of the volunteers in their communities. The Midterm Evaluation Report looked into the strengths and challenges of care groups in the CSP, verifying that they are appropriate and effective in the Rwandan setting. This is especially true in an area where the churches were engaged and active in the program. Clergy from 11 denominations participated in Pastoral Care Groups, reinforced key messages with their congregations, and lent additional credibility to care group volunteers in their communities through their support for the Project’s activities. Data collected through the well-organized community-based information system (C-HIS) every two weeks complemented the monitoring and evaluation surveys and were instrumental in convincing stakeholders that the CSP’s mobilization efforts were effective. Data collection, analysis and sharing reinforced mobilization efforts. Now that World Relief has documented the effectiveness of Care Groups, the questions remains if Care Groups, or the essential components of them can be replicated on a much larger scale through partnerships between PVOs, governments, and major donors. To answer this question, World Relief, and/or the other groups using this approach will need to analyze and cost-out the essential components of the methodology. Dr. William Reinke, a Johns Hopkins University professor of international health systems planning and financing says that scaling-up a methodology that has been proven effective in pilot, or small-scale projects such as the CSP, is less costly than the initial start-up costs.4 Advocates for the Care Group approach could be much more effective in approaching donors if they had this information. There is much still to be learned about the overall impact of organizing communities along the Care Group model. Clearly there are transformative development processes in 4 Personal communication, November 2006. 29 World Relief/Rwanda Child Survival Final Evaluation December 2006 progress, such as women’s empowerment, local problem-solving, and the growth of civil society, that the CSP evaluation cannot fully measure or document. The Rwanda CSP Care Group framework could certainly be used as the template for a number of development strategies. It remains for World Relief to articulate these possibilities to get the support needed to build on these valuable foundations and profit from the momentum and good will already established by the CSP. World Relief is not a research organization, and does not have the financial means to conduct research, either alone or with contracts. However, while the community mobilization results of the program are still intact, it would benefit everyone in the development community to learn as much as possible about the impact this approach has had in the project area. These lessons can be applied to the wider field of development and do not necessarily need to be restricted to child survival, HIV/AIDS or other health programs. B.3.b Behavior Change Communication The dramatic increases in project indicators from the baseline resulted from the reinforcing nature of the programs BCC approach. At first glance, the project would have appeared to have too many interventions. On closer examination, the interventions were largely integrated in two places: at the community level through Care Group volunteers, and at the Health Center working with health facility workers. Furthermore, the other players in the program all served to reinforce the same messages. While skeptical of the approaches at first, District health officials became convinced as they saw substantial decreases in morbidity and mortality and improvements in overall health status of the communities. Message reinforcement by pastors from the pulpit and through church-based Care Groups, as well as support from local leaders helped to provide the message from multiple sources. Some BCC messages were also supported through national government programs and implementation of performance-based reimbursement that rewards both providers (such as TBAs) and beneficiaries for participating in preventive care and/or early treatment. Adjustments of MOH policies toward technically strong approaches (ACT, ITNs, IPT, etc) served to remove barriers to compliance with the recommended behaviors. The challenge that remains will be to reinforce and refresh the volunteers over time. World Relief has negotiated with the MOH and the District HC COSAs to institutionalize periodic support to the Care Group volunteers to keep them up to date as best practices continue to evolve for key behaviors. 30 World Relief/Rwanda Child Survival Final Evaluation December 2006 B.3.c Capacity Building B.3.c.i Strengthening the PVO organization World Relief was one of the first US PVOs to participate in the PVO Child Survival and Health Grants Program that started in 1985. WR is a founding member of CORE, and health staff from World Relief’s headquarters have been active participants in CORE group activities. In addition, the Care Group experience from Mozambique has been published and shared with other PVOs. World Relief relocated their headquarters from Wheaton, Illinois to Baltimore. One of the reasons for this move was to be closer to the Baltimore-Washington area and make it easier to participate in the organizational capacity building opportunities that would have been difficult to attend regularly. The headquarters backstop changed a few times during the life of the project. At the end of the project, the health advisor who backstopped the program at the time of the midterm evaluation was no longer working at World Relief. This is not uncommon for US PVOs. Melanie Morrow, who had participated in designing the original program, returned to backstop the project and participated in the final evaluation. One important capacity element included supporting a strong and talented Program Manager. As a technically strong, competent and compassionate leader, she was recognized by many on the evaluation team as a valuable role model for demonstrating the capacity of women as leaders in development. World Relief helped her to enroll in the Future Generations Master of Applied Community Change and Conservation degree program. This program has the advantage of allowing students to remain in their jobs, while traveling to the US and field sites periodically throughout the year. Projects and assignments can be completed online through extensive email consultation with professors and classmates. B.3.c.ii Approach to Strengthening Local Partners The MTE documented many of the supportive measures that World Relief provided to District Health Offices and to building capacity of District Health personnel. These measures, coupled with documenting results and supporting and supplementing the District HIS, helped to convince District Health officers and the District Hospital personnel of the value of a community-based approach when partnered with the formal health sector. Assistance with using data for decision-making was especially important as the Districts have begun to assume more responsibility for management of health programming. Performance-based reimbursement has been introduced into the Government of Rwanda health system and the ability to demonstrate results is essential for the District Health Management Team. 31 World Relief/Rwanda Child Survival Final Evaluation December 2006 Health workers, local leaders, church representatives, Care Group members and World Relief staff all benefited from several professional training opportunities provided by the project. Most notably, World Relief hosted Gretchen and Warren Berggren, some of the developers of the PD/Hearth Methodology for child nutrition rehabilitation to come to Rwanda and provide training for World Relief, as well as other organizations such as Concern Worldwide. The Care Group approach has also been adopted by several other PVOs in other countries. The Rwanda grant experience has both contributed to and benefited from this collaboration. A network of community health workers, involving Health Animators, Maternal Health Animators, and CG volunteers was created by the health district. Some of the Care Group volunteers were elected as representatives at HC level. Umucyo volunteers assisted health centers by informing mothers about scheduled outreach sessions and mobilizing them for immunization. At these outreach sites, they organize the mothers and take advantage of the occasion to carry out some Care Group activities such as weighing children, selling ITNs, ORS and Sûr’Eau. They communicated with the HCs through the referral/counter-referral system of immunization defaulters and through other events that required community mobilization such as vitamin A supplementation campaigns. CG leaders received technical support and supervision in participatory adult education techniques, activity planning and coordination, and reporting guidelines. This empowered the CG leaders to run volunteer meetings and trainings in the community, and to produce reports for Community Development Committees (CDCs) and health centers. Their leadership skills have improved as a result of these trainings and they are sufficiently prepared to replace the World Relief-hired promoters, who have been gradually reducing their presence in the communities as they turned over responsibilities. The volunteers were trained in managing funds and assisted in forming an association management committee. More than 95% of volunteers are now working in associations. The project also provided regular training, supervision, and management tools (registers, store cards, registers for inputs and outputs, etc.) to the management committees of 202 volunteer associations. Umucyo assisted these associations in assigning tasks to select volunteers or groups of volunteers. For example, ‘Light Mothers’ (Mama Lumieres) were assigned to lead Hearth programs; some volunteers took responsibility for growth monitoring, some were maternal health animators, and others were anti-malarial drug distributors. The project also helped the volunteers to get formal registration and approval from the administrative (as opposed to the health) district authorities, and to open local bank accounts. The CSP supported these associations by providing them health promotion products to sell in the community. A portion of the proceeds from these sales are retained by the associations and deposited in savings accounts. Similarly, every family is sensitized and encouraged to give to the volunteer association 50 Rwandan francs (about $0.10) per child every quarter as a contribution for the community-based nutrition program. By the 32 World Relief/Rwanda Child Survival Final Evaluation December 2006 end of 2005, the assets of all the volunteer associations totaled 5,326,787 francs ($9,426). Records of purchases and sales are well kept, and an association management committee was established to ensure accountability and sustainability. Furthermore, World Relief provided 836 sheep/goats to these associations, which have multiplied to 1,098 and are providing a source of income, manure, animal proteins, etc, to support the volunteers. The CSP initiated five volunteer associations of PLWHAs that currently include 54 members. These groups help build social support for the PLWHAs, and also serve as a resource for delivering specific messages on HIV/AIDS prevention and care for other PLWHAs. In response to the advocacy efforts of the CSP, World Relief Rwanda Micro Enterprise Development Program awarded a grant of 705000 francs ($1,250) to support small business/ income generating initiatives developed by the associations. B.3.c.iii Health Facilities Strengthening The WR CSP was first, and foremost, a community-based project. In June 2004, the Project supported a health facilities assessment of MOH facilities, including the Kibogora District Hospital. A modification of the approach developed by the BASICS Project was employed. This was carried out by Dr. Jean Kagubare, who is the former National Director of Health Services for the Ministry of Health and recently received a PhD at the Johns Hopkins School of Public Health. Strengthening the day to day functioning of health facilities was not a major focus of the program. Yet, on the other hand, introducing community-based child survival programming, and developing and strengthening collaboration with health facilities to strengthen the community-health center-hospital continuum, was an essential part of the program. In this function the project excelled. The CSP did not focus very much on improving the clinical skills of health center providers, especially related to the quality of labor, delivery, postpartum and newborn care. A health facilities survey was not conducted at baseline, as it was not a requirement of the CSHGP at the time. Future MNC strengthening programs in the district should address these issues. B.3.c.iv Strengthening Health Worker Performance As mentioned, formal health facility assessment was not done at the end of the project, so it was difficult to quantify the impact the program had on increasing the quality of care at facilities. Focus group discussions with health facility workers revealed that they felt their knowledge of child survival interventions significantly increased as a result of the training and support that they received in the program. Their skills in interacting with community-based health workers and in using the community HIS to help them target activities and increase coverage in key interventions were considerably strengthened by the program. Health workers are also residents of the communities targeted by the program, so their training was reinforced by the information that they received in their communities. 33 World Relief/Rwanda Child Survival Final Evaluation December 2006 Feedback from community members and Care Group volunteers attributed improved quality of health worker performance to the strategies the project used to build capacity at multiple levels. Encouragement that the project gave to increase mutuelle membership also decreased barriers to care. Health workers are also more receptive to referrals from community based health workers, especially TBAs and HBM distributors. Training conducted by the program at both levels has improved referral systems and documentation. Lessons learned Supervision and support of volunteers is much easier in associations than individually and the Government of Rwanda now wants assistance to communities to be directed through associations. All communities understand this and a variety of associations are being formed at the sector level. The cellule teams of 10 people have each designated one person to be in charge of health who can supervise the care group associations. Pastors and health centers can also be very helpful in supporting and encouraging volunteers in their communities. B.3.c.v Training The effectiveness of the cascade and modular training approaches used by the program were well documented in the MTE. For Care Group volunteers, the latter half of the program consisted of refreshers and updates, rather than introduction of large amounts of new material, although there was additional emphasis on MNC content. The major new training undertaken during the second half of the program was for selection and orientation of HBM distributors. A separate report on the BASICS/PNLP HBM assessment is under development, and training is included in this assessment. The National Malaria Control Program (PNLP) was heavily involved in this training, as were the two other PVOs, Concern Worldwide and the International Rescue Committee. Care Group volunteers have job aides in the form of booklets and pictures that they receive in their training. Volunteers report that the job aids help them to remember the material that they have learned and are very useful to them overall. The high literacy levels in the country mean that print materials are appropriate for both the volunteers and even the communities. B.4 Sustainability Strategy The lasting effects of the community behavior change strategies implemented by program have been mentioned. The sustainability plan called for health center staff to take over Care Group Leader training after the end of the project. A systematic turnover of 34 World Relief/Rwanda Child Survival Final Evaluation December 2006 responsibility has already taken place over the last two years of the project (See Annex F.1). Transition funding has since been assured through a cost-sharing arrangement between WR and the COSAs. WR Rwanda will pay salaries and costs for Oct-Dec 2006 at 100% because District and COSA budgets for 2006 were already set. World Relief’s percentage contribution decreases every 6 months as the COSAs take full responsibility for the continued staffing over the next year(s). Ten former CSP promoters will now be employed by HCs for continued outreach and supervision, and one former CSP coordinator has been placed at Kibogora Hospital to manage their activities. The sustainability goals and objectives that were articulated in the DIP evolved during the life of the program to keep up with both the technical progress in the intervention areas and the changing configuration of Rwandan health services. It is clear, however, that the partnership developed in the early years of the program established the trust that was necessary to work out an arrangement to sustain services that were highly valued by the community and the District. If the district health services representatives, especially the Head of the Kibogora District Hospital, had not supported this transition as strongly as they did, such an arrangement would have been extremely difficult for World Relief. The COSAs’ willingness to take on these additional salaries is also a strong reflection of the program’s perceived value. This transitioning cost-sharing arrangement may represent one of the first instances in which a CSP has handed over a functioning system, complete with transitioning the financial support. It would be worthwhile to monitor the process over the next several years and learn how much of the planned services are able to be locally sustained. Undoubtedly the availability and acceptability of mutuelles have provided funds at the local level to make all of this possible. There are still challenges to reach all of the poorest of the poor with mutuelle membership and the district government is still trying to find ways of addressing this problem. C. Project Management C.1 Planning As documented in the MTE Report, the DIP was very practical and represented a good blueprint for the program. Of course, over five years the technical guidelines in several areas evolved, but World Relief adapted well to these changes. The Kibogora area is a very close-knit community and World Relief’s office is located on the same compound as the District Hospital, so it was relatively easy for the health system partners to communicate. The evaluation team found that joint planning was very collaborative and all partners were involved in the program implementation. 35 World Relief/Rwanda Child Survival Final Evaluation December 2006 C.2 Staff Training The training process was continual, with training for each new technical intervention taking place on a quarterly basis, and performance assessments to monitor use of new skills and knowledge. During focus group interviews with staff, they were very enthusiastic about the training they had received and felt that the project had provided them with excellent skills to perform their duties. The lessons learned about staff capacity building included providing regular refreshers as well as strong supervisory support. According to them, the major motivational reinforcement for the skills they acquired during the program came when they were able to see the positive effects on the health status of women and children and the obvious decreases in mortality. Staff members were initially skeptical that the project would be successful because there were no plans to give out food or money. They were convinced when they saw the community changing their behaviors and the positive impact that it had. In addition, learning that the resources for these positive changes could be found locally and did not need to come from “outside” or from “living like foreigners” was extremely empowering. They also learned that working with volunteers could be as effective as it turned out to be. They learned from the volunteers themselves that when they saw the changes in the communities and their behaviors, volunteers’ motivation was also reinforced. Humility was another lesson learned by the staff. Since they did not act superior to the volunteers that they taught, they were accepted, trusted and respected within the communities. Because they were trusted, they were able to dispel many of the superstitions that had negatively impacted on health behaviors in the past. Staff stated that the training prepared them well for the work that they needed to do. The manuals and course syllabi they were given, as well as instruction in training methodologies helped the training to be high quality. Some of the specific skills they acquired included: o Conducting group work o Techniques for approaching people in communities o Techniques for conducting a home visit o Clarifying CSP objectives and ways of working toward those objectives In the beginning, many of the promoters thought the job looked “medical”. This intimidated a few of them because many had originally held jobs as teachers, or other non-medical type of work. They appreciated that everyone was treated equally. They learned how to talk to people from different strata in the community. Men had to learn how to talk to women about things that men usually didn’t know about. The project brought in additional local trainers in subjects such as malaria, nutrition for PLWHAs, 36 World Relief/Rwanda Child Survival Final Evaluation December 2006 microenterprise development programs (MED), field supervision techniques, human resource management, and survey techniques. The Head of the District Hospital credited almost all improvements in the quality of health services at facilities (especially health centers) to the training received through the CSP. One reason he gave was that clients were referred to health centers and to the hospital in less serious condition and were able to be treated more successfully. This motivated the health center staff to work with the community-based CSP workers to mobilize clients to come in earlier when danger signs were detected. Thus the improvements reinforced each other. At the time of the FE, the DHT was able to provide excellent corroborating data to support decreases in morbidity and mortality and other impacts achieved in the program that were measured in surveys. While cascade training can have limitations in terms of quality if not closely supervised, it appeared that consistency in content, along with significant reinforcement through several channels (refreshers, supervision and exposure), contributed to the effectiveness of the training. C.3 Supervision of Project Staff Supervisory check lists and formal assessment of knowledge were made on a regular basis. For the Promoters, a salary bonus was given quarterly to the best performers. The sustainability plan includes provision of one promoter at each health center, with one overall coordinator to be placed at the hospital (district health headquarters). The CSP demonstrated the empowering effects of regular and supportive supervision. Staff stated that they were well supported by the Kibogora WR office. The only complaint the staff had was that they only had 20 motorbikes, whereas more motorbikes would have helped more of them to cover long distances within the project area. Some promoters had to walk two hours to reach some of their communities. C.4 Human Resources and Staff Management Essential personnel policies are in place and turnover was extremely low throughout the project. There was very high morale and good working relationships within the World Relief staff and between World Relief and the partners. The same Project Manager and Deputy Project Manager stayed the entire program. All staff members were given long advance notice prior to the end of the program. Some will transition to working in the health centers, to be initially paid with a cost-share between World Relief’s private funds and the COSAs funds they receive from cost-recovery. Others will be employed as part of World Relief’s contribution to the new Expanded Impact Program CSP along with staff from Concern and the International Rescue Committee. 37 World Relief/Rwanda Child Survival Final Evaluation December 2006 The female Project Manager served as a good role model for the staff. Several of the male staff members mentioned that having a female Project Coordinator had made them realize that women could fill many more roles outside of the home than they had originally believed. She also had strong ties to the project area and turned down multiple opportunities for jobs with international organizations in Kigali. Undoubtedly project staff benefited from the reinforcement provided by the involvement and support of the churches, including the pastors. Staff members were part of the communities that benefited from the project, and their motivation was reinforced by the respect that communities showed for the impact of the project. The WR project design appears on the surface to be fairly labor intensive. A cost￾effectiveness analysis was not conducted as part of the FE. Nevertheless, the adequacy of the staff configuration for government health services to absorb the project infrastructure and sustain the project’s achievements should be monitored during the transition phase. C.5 Financial Management Financial systems at project, country office and headquarters levels have supported implementation of a highly successful project. Additional private World Relief funds that were initially set aside to transition some staff members to work in health centers had to be allocated to the new EIP. After the FE field work demonstrated such remarkable results, World Relief, the District Health Team and the Ministry of Health developed a detailed strategy and budget to transition some staff to health center employees who would eventually be paid completely by the COSAs. Rwandan government health services were still in the process of undergoing significant decentralization at the time the CSP ended. While all partners, and the USAID TA provided through IntraHealth are optimistic that changes are moving in the direction of significantly more local resources for health, it is too early to say for certain that this is the case. C.6 Logistics As the CSP was community-based, it used very few commodities, aside from those that were sold by the volunteers, such as Sûr’Eau and ORT. The only concern expressed by staff and partners had to do with having fewer motorbikes than desirable. The project area is mountainous and the roads are poor, especially in the rainy season. Health Center staff cited lack of transport as the major constraint on sustaining work with communities. During the evaluation, it was mentioned that GAVI was providing support to the District for immunizations and this would include transportation. The evaluation team encouraged the DHT to coordinate community support work with immunization outreach supported by GAVI. 38 World Relief/Rwanda Child Survival Final Evaluation December 2006 At the end of the project, when the effectiveness of Sûr’Eau was well established, there was concern that future supplies were in doubt. Since point-of-use household water treatment was so successful in the program, a continuing supply is very important to sustaining the CSP impact in this area. World Relief followed up with PSI in Kigali and learned the product will continue to be available, albeit at a higher price and in slightly altered packaging (as described earlier in this report). C.7 Information Management The project’s system of collecting data from each Care Group volunteer who visits ten households every two weeks was simple, effective, and sustainable, provided that the supervision strategy for after the project is maintained. The project HIS was triangulated with District Hospital Data and graphs highlighting results from the final KPC survey were presented at the Final Evaluation Results Dissemination meetings in Kibogora and Kigali. Several relevant graphs are included in this report. Mortality and morbidity data from the district were sufficiently accurate that they were also used by the BASICS/PNLP HBM assessment team in their national presentations and will be included in the final report. World Relief has demonstrated that a community-based HIS is feasible and affordable. Results were collected and analyzed both locally and by outside evaluators. The supervision of data collection, analysis and dissemination is done by the same person who also manages the program. Even though there is nothing to suggest that this has biased any findings, World Relief should make sure that that other people are involved who can verify the findings in order to maintain the high credibility the organization currently enjoys. C.8 Technical and Administrative Support The types of outside TA provided to the project have already been discussed in other sections of this report. World Relief provided consistent support from headquarters, although the individual backstops changed during the lifetime of the project. Strong local leadership by the Project Manager coupled with a strong work plan in the DIP made it possible to implement the project without a lot of direction from headquarters. Sharing technical assistance consultants with other PVOs decreased cost and contributed to scaling up impact throughout Rwanda. Implementing the PD/Hearth program could have benefited from on-going technical assistance specifically addressing that component of the nutrition intervention, especially as the focus of the activity changed from rehabilitating moderately malnourished children to assisting mildly and moderately malnourished children to become “normal” weight (which is interpreted as being in the “green” zone on the Road to Health card). There is a need for further dialogue between all of the PVOs implementing Hearth and the technical 39 World Relief/Rwanda Child Survival Final Evaluation December 2006 experts who promote this methodology to refine the methodology as well as identify its other potential development benefits for communities. C.9 Mission Collaboration The project collaborated with the USAID mission in many areas. The most significant of these collaborations involved the HBM program where the three PVOs advocated for support from the MOH (PNLP) and the USAID mission. When the PVO groups secured a small seed grant from the CORE Group, the mission made a significant financial match and also facilitated coordination meetings in Kigali. This “vote of confidence” helped to solidify the PNLP’s collaboration for developing the HBM. By the end of the project, the USAID mission was already discussing how the new PMI support coming to Rwanda could help to provide ACT drugs to the new Expanded Impact Program. C.10 Management Lessons Learned The value of the continuity of management leadership afforded by having the same Project Managers throughout the entire program was the major lesson learned. In this case, World Relief Rwanda knows that it has been very fortunate, as other PVO/NGO programs have experienced turnover when talented managers were lured away with higher salaried jobs in Kigali. The results may have been quite different if the managers had left during the program. Future programs might extend management responsibilities to a slightly larger number of people just in case the top managers leave the program. Another important lesson was the value of perseverance in the face of initial opposition by the District Health Team leadership. This is a common experience in countries that do not have a long experience with Child Survival projects. This is especially true in a post￾relief environment where NGOs are largely viewed as “mini-donors” and their capacity building functions have not yet shown their value. By the end of the CSP, the initial DHT skeptics were the biggest supporters of the community-based approach. D. Other Issues Identified by the Team Some development policies in other sectors, such as the environment, appear to negatively impact the sustainability of health and nutrition interventions undertaken by both the World Relief Child Survival program and the District Health team. Access to fishing in Lake Kivu is limited to members of fishing associations, and this has driven the price of a kilo of fish from 1000 francs to 3000 francs in a very short time. The higher cost has made fish inaccessible to communities that, in the past, have relied on fish as a key protein source for rehabilitation of children in PD/Hearth. In Yove, families used to plant crops on land belonging to the Nyungwe National Forest. Community representatives said that they were promised jobs planting trees if they stopped using the 40 World Relief/Rwanda Child Survival Final Evaluation December 2006 park land for crops. Now many are cultivating less than a half hectare of hillside, which does not provide enough food for the family, much less any surplus to sell. The promised jobs have not yet materialized. Not surprisingly, Yove made the least progress in Hearth sessions due to food shortages at the household and community levels. The evaluation team identified the need for additional income-generating activities, including microfinance. World Relief began small scale income generation activities with groups that were part of the CSP. Groups who received this assistance recognize the benefit and expressed strong desires to continue diversifying sources of family income. The incomes are so tenuous, however, and the food security problems so great, that World Relief should assess ways of developing larger-scale micro-enterprise and small scale food security projects using the already formed Care Groups and Hearth volunteers. The District agronomist is a valuable resource and has counterparts in each Cellule. Other NGOs in the country, such as the Heifer Project, have specialized expertise in the area of small animal husbandry with women’s groups and they could be tapped for expertise, or facilitated to begin this work in the District. It is recommended that World Relief assess whether they can initiate microcredit or other income generation activities in the area. If this is not in World Relief’s managerial interests, World Relief should invite other organizations to visit the area and consider working with the local government to start projects to improve the economic situation in these communities. An upcoming visit from the One Acre Fund is described in the next section. On the positive side, new government policies mandating use of fuel-efficient stoves to conserve firewood seem to be quite effective and families report that they use very little additional fuel to boil drinking water. This policy has a positive effect on health behaviors that were promoted in the community and is likely to be sustainable. E. Conclusions and Recommendations World Relief’s CSP met, and most cases far exceeded project targets and goals. In addition, there is abundant evidence that major improvements in household behaviors, especially in hygiene, water treatment, and child feeding can be sustained without depending on government health services. The roles of women have been expanded in relation to health, but also in civil society overall. Some female CSP volunteers now hold elected office in their communities. The C-HIS demonstrated a major decrease in child mortality and a marked downward trend in deaths among women of childbearing age. The project demonstrated that this model, involving an integrated community-based program implemented by large numbers of volunteers within a well-defined catchment area and using specific behaviorally-focused communications, not only improves the health status of women and children, but actually raises the quality of life for entire 41 World Relief/Rwanda Child Survival Final Evaluation December 2006 communities as well. The supervisory support systems achieved and maintained consistent messages across many different communication channels and also ensured that monitoring systems demonstrated impact. The additional resources available to the local health system through the mutuelles have removed some of the usual barriers to improving health services and allowing beneficiaries to follow though on the care-seeking actions promoted in the programs. In this case, government policies have enhanced the effectiveness of the child survival approach. Project staff aided the introduction of mutuelles into the project area. World Relief’s willingness to collaborate with two other PVOs that are both secular and use other methodologies in their programs demonstrates a flexibility and adaptability that bodes well for inclusion in additional programs. The nutritional aspect of the health impact, however, is threatened by factors outside of the control of the health program. Overall poverty and food insecurity mean that the gains seen in changing household behaviors may be outweighed by the overall lack of food. The answer to this problem lies in broadening assistance from other development programs that will strengthen the overall food security situation in these vulnerable communities. In January 2007, former CSP staff in Kibogora will host a visit from the One Acre Fund, which is interested in (a) learning about care groups for possible application to its existing projects in Kenya and (b) exploring the potential introduction of its model for increasing agricultural output and market linkages for small farmers into existing care groups in the Umucyo project area. While the managers of the program are highly-dedicated and competent in their implementation of the program, World Relief would have been very vulnerable if they had left during the life of the program. It is recommended that additional leadership positions be included in future programs to build capacity within the Rwandan health workforce and also help to mitigate against major changes in the consistency of program approaches that might occur if the program managers leave. E.1 Comments to USAID USAID CSHGP has long encouraged PVOs to collaborate within countries to implement programs for larger impact at scale. The small USAID grant that was channeled through the CORE Group was enough “seed” money to enable cooperation that had already started between three PVOs to begin to bear fruit. Once the process was started, the USAID mission and other support followed. This also allowed the innovative community-based case management approach of the HBM to receive much wider attention nationally and internationally than any single CSP could have accomplished. As a result, the unique contributions of World Relief and partner PVOs to scaling up effective CS approaches are more likely to get the attention from international programs, such as RBM and PMI that they richly deserve. 42 World Relief/Rwanda Child Survival Final Evaluation December 2006 The major impediment to scaling up the lessons learned from the Umucyo program is the funding trend away from integrated multi-intervention programs back towards disease￾specific vertical programming. While the comprehensive and deep penetration of impact into the very way families and communities view their own power to promote health and prevent disease is undeniable, support for these types of programs will continue to decline without champions within the donor community. PVOs have had great difficulty gaining audiences to present the potential value-added of approaches such as Care Groups for the vertical programs. Although the HBM program, for example, increased access to early malaria treatment across Rwanda, the trend and accompanying decreases in malaria morbidity and mortality was already occurring in the World Relief areas. This indicates that there are other significant positive factors involved in supporting the trend. Most likely it was the intensive community mobilization to recognize and take action on a variety of health threats that played a significant role. Additional comments are included in the scaling-up recommendations below. Severe malaria cases in Kibogora Hospital 0 100 200 300 400 500 600 2001 2002 2003 2004 2005 2006 (Jan-Jun) Year Cases HBM Introduced The effectiveness of the Care Group approach for community mobilization, health education and health status monitoring has been well documented in multiple projects in several countries. Further evidence of mortality reduction has been documented in this project. Other PVOs are now implementing the approach, including some secular organizations that do not work through religious organizations in project communities. Future inquiries into how governments can best support PVOs and CBOs to implement care groups are needed, as are investigations to identify critical elements of the approach that are necessary for replication into larger scale programs such as a national program. Research is needed to understand the key factors in this type of program that make it work. PVOs have neither the money nor the personnel to conduct this kind of research. USAID should support this research through some other mechanisms so that these effective strategies can be scaled up. 43 World Relief/Rwanda Child Survival Final Evaluation December 2006 E.2 World Relief’s Plans for Using Best Practices and Lessons Learned from the Program Three best practices were established or elaborated in CSP Umucyo and offer lessons learned for future WR CSPs and for other organizations utilizing care groups in their community mobilization activities. These include: (1) home-based management of fever (suspected malaria) using community-based distributors; (2) the partnership with PSI, through which volunteers sold ITNs and Sur’Eau for point-of-use water treatment, expanding access to these products and providing a small financial incentive to volunteers; and (3) registration of care groups as formal associations, enabling volunteers to establish group savings accounts, revolving loan funds, and/or to invest jointly in small livestock. In addition, WR has designated Kibogora as a Center of Excellence for training in care group methodology for new WR child survival staff as well as visitors from peer organizations, government agencies, and technical partners. Sharing Lessons Learned WR staff have shared strategies and results from CSP Umucyo with health officials and peer organizations in Rwanda and elsewhere. For example, when Kibogora became the first district in Rwanda to reach Abuja targets for ITN coverage, the CSP Program Manager gave an invited talk describing the Project’s ITN strategy at a regional Roll Back Malaria meeting in Kigali in November 2004. The Director of Maternal and Child Health at WR HQ shared the care group methodology and results from the CSP Umucyo mid-term evaluation with her colleagues in the Rainer Arnhold Fellows Program for young leaders in international development. As a result of these conversations, SurfAid International has adopted care groups; Child Family Welfare Shops in Kenya has expanded their program strategy and introduced care groups for community outreach; and the One Acre Fund is exploring opportunities to incorporate care groups into its community-level programs in health and agriculture. One Acre’s founder is scheduled to visit Kibogora in January 2007 to learn more about care groups and their potential application to One Acre’s work with rural women subsistence farmers in Kenya. Additional dissemination of best practices to peer organizations and the child survival community will be achieved through presentations at professional conferences; WR plans to submit abstracts for the November 2007 APHA and June 2008 Global Health Council annual conferences. Expanded Impact Project As the partner responsible for community mobilization in the Expanded Impact Project (2007-2012), WR is working with the IRC and Concern Worldwide to scale up home￾based management of malaria and control of diarrheal disease, and to introduce community-case management of pneumonia in Rwanda. Care groups will be 44 World Relief/Rwanda Child Survival Final Evaluation December 2006 implemented in two health center catchment areas within each of the six EIP districts. IRC’s senior child survival technical advisor traveled to Kibogora in November 2006 and met with care group volunteers and promoters to learn more about WR’s approach. Expansion into Burundi WR is currently developing a proposal for a child survival program in Gitega Province, in central Burundi. The program design for the new project draws substantially on best practices and lessons learned in CSP Umucyo, including all three best practices outlined above. With technical and advocacy support from WHO, CSP staff and care group volunteers will partner with MOH-trained community health workers to introduce HBM in Burundi. WR will partner with PSI to bring POU water treatment to isolated rural communities in the project area. As in Rwanda, formal registration of community-level associations is possible; WR is exploring the feasibility of this mechanism to enable group savings and income-generating activities. During project start-up, WR will sponsor relevant staff from the MOH and key technical partners to visit with Kibogora district health staff, project staff, and care group volunteers to learn about the program methodology, results of the child survival interventions, and the success of the HBM pilot. Technical Assistance to CRS WR is a sub-grantee in the IMA-directed AXxes Project in DR Congo. WR is helping AXxes partner Catholic Relief Services (CRS) to pilot care groups, beginning with one health zone in South Kivu. TA activities will include an upcoming visit by CRS to Kibogora to learn about the program strategy first hand. E.3 Potential for scale-up and expansion of the project World Relief and IRC have partnered with Concern Worldwide for an Expanded Impact Child Survival Project in six districts that began October 1, 2006. The three districts that were included in each organization’s CSP will also be included in the new program and additional districts will be added. The recent HBM assessment that was done by BASICS and the PNLP has shown impressive results wherever the approach was piloted. There is a consensus that the PNLP is happy with the HBM results and will scale it up with other donors. UNICEF was already added in the current pilot with encouragement of the PNLP and donors. The Care Group model for effective Child Survival impact now needs to move to the next phase to be adopted beyond what is considered the “pilot” stage. To do this, a critical path, including resource allocation, needs to be identified and the investments that would be required need estimation. It is not reasonable to expect a PVO, with its limited staff and financial capacity to do the work of a research university for this purpose. For this reason, it is recommended that USAID include Care Group model scale-up in its health systems management research portfolio in the near future. 45 World Relief/Rwanda Child Survival Final Evaluation December 2006 F. Results Highlight: Increasing Women’s Empowerment in the Family and Civil Society Poor status and low empowerment of women in poor Rwandan communities is often cited as one of the major barriers to changing household and community behaviors and increasing access to health care for women and young children. In World Relief’s CSP the roles of both genders changed significantly and contributed to the significant positive impacts of the program. Seven men and two women on the evaluation team volunteered to participate in a focus group discussion tasked with describing what, if any, impacts on health-related gender roles that the CSP achieved. Several key points emerged from this discussion. Several points below come directly from statements made by group members. Of the 2,961 volunteers that the CSP has trained, 95% are women. The CSP manager was a Rwandese woman and 48% of the staff were women. The participants cited the female Program Manager as an example that shows a woman can be a decision-maker. While most of the interventions targeted women, they included men in the trainings to “sensitize” them to the issues. It was felt that after training the women, the CSP trained the community, and the men started to see the value of the volunteers’ work. “Before, men wouldn’t believe that a woman could change something in society. But now, they have seen that women got the community to change behavior and get immunizations and antenatal care. Before they thought that only educated women could change things. But now they know that all women can take care of their children and make nutritious food, so women can change things. When men saw that women volunteers were changing things, they started valuing women more”. The role of women in community decision-making also changed. Before the CSP, there were no women in the local administration. The training the women received from the project helped them to gain respect in their communities and then they were invited to join the local administration. At the beginning of the project there were no female cell coordinators; now nine female CSP volunteers have become cell coordinators. In addition, four female staff members have been elected to the local (cell) administration committee. Men’s roles within the family regarding mothers’ and children’s health and nutrition also changed. By the end of the project, men were providing more food for their families. Even though they had always provided for their families before the CSP, they are now planning together with their wives rather than just giving some money to their wives and expecting her to do everything. Men are even buying fish and fruit for their children, whereas they would not have done so in the past. Men were not originally very interested in mutuelles because they don’t get sick very often. This attitude was an obstacle to mutuelle enrollment because it is the men who usually pay the membership fees. After the project spent time discussing the importance 46 World Relief/Rwanda Child Survival Final Evaluation December 2006 of having this insurance, they understood the value of membership for their families and agreed to enroll. Traditionally there has been a division of labor for men and women in the family. Kitchen work was for women only. Women were the ones who were expected to carry the hoes back home in the evening after both men and women worked in the fields (men were not supposed to carry them). Now, men will carry their own, sometimes the women’s hoes too, and they are helping out by going for firewood or fetching water. Men are participating in hygiene: cleaning the house, building latrines, providing soap, etc. Men are also now more willing to wash their children. Men will now also help take care of their wives when they have a baby. In the past, a man would find another woman in the neighborhood to take care of her. Men now realize that they can take care of babies when their wives go out and they are actually doing so, rather than getting another woman to care for the children. Husbands now make an emergency plan for a safe delivery together with their wives when she is pregnant. The percent of women who had permission to execute an emergency transport plan before her last delivery rose from 16% at baseline to 96% in the final KPC; those who actually had a plan rose from 25% to 79%. Perhaps the most inspiring findings included improvements in relationships within the family. “Husbands love their children and their wives more than they used to. He helps out and takes care of the baby and provides for them more. There is evidence that he loves his wife more. Husbands loved their wives before too, but they didn’t know how to do these things. Now husbands say that they wish they had known that they could do these things for their wives. Men are also now aware of how much work their wives are doing. They didn’t realize how hard it was to take care of the house and children. Now they know how much work it is.” Men have now allowed educational training to take place in their homes (when the volunteers go to visit the households), whereas in the past they would not allow anyone to come into their homes. The discussion group acknowledged that not all men in the project area are doing these things, but felt that those who aren’t doing it know they should be doing them. World Relief staff members admitted that their own attitudes towards gender had also changed, saying, “We are now aware that maternal and child health concerns us all, not just women. It’s not a woman’s issue or a man’s issue—it’s for everyone. We try our best to promote maternal and child health.” 47 World Relief/Rwanda Child Survival Final Evaluation December 2006 G. ATTACHMENTS ANNEX A.1 EVALUATION TEAM MEMBERS Jean Capps External Evaluator and Team Leader Kabadege Melene CSP Manager Kwizera Maurice CSP Assistant Manager Dr Mukarugwiro Beatha Kibogora Hospital Dr Nsabimana Damien Director of Kibogora Hospital Tuyishime Martin Kiboroga Hospital Mugeni Murasa Cathy MOH Niyitegeka Francois MOH / PNLP Melanie Morrow World Relief HQ Rachel Hower World Relief HQ Anna West World Relief HQ Vincent Oloo World Relief Sudan Rose Eio World Relief Sudan Bishyizehagari Thomas World Relief Rwanda Niyonkuru Mathieu Titulaire, Kibogora Health Center Shumbusho Albert Titulaire, Nyamasheke Health Center Pastor Kakira Jeremie Free Methodist Church Kibogora Pastor Nsengiyumva Casmir ADEPR Tyazo Bankundiye Etienne Representative of the Mayor of the District Ndundayino Jean Claude Executive Secretary Cyato Sector Ngiriyambonye Elie Charges Affaires Sociales, Bushekeri Ndayisenga Félicien CSP Ntawukuriryayo Fidèle CSP Mukantagara Xaverine CSP Nsabimana Marcel CSP Uwamahoro Chantal CSP Ngiruwonsanga Narcisse CSP Ngirabakunzi Nathanael CSP Nyiransabimana Jeanette CSP Munanira Daniel CSP Muteteri Beatrice CSP Nsengimana Martin CSP Nyiranzeyimana Beatrice CSP 48 World Relief/Rwanda Child Survival Final Evaluation December 2006 ANNEX A.2 LOCAL DISSEMINATION MEETING ATTENDANCE LIST WR CSP Final Evaluation Local Dissemination Meeting Free Methodist Church School Conference Room, Kibogora Sept 23, 2006 Bankundiye Etienne Representative of the Mayor of the District Melanie Morrow WRHQ Oloo Vincent World Relief Sudan Eio Rose World Relief Sudan Thomas World Relief Rwanda Tugirumuremyi Charles World Relief Rwanda Ndundayino Jean Claude Executive Secretary Cyato Sector Karangwa Anaclet Representant of the Executive Secretary Sector Kanjongo Nyiraneza Constance Representant of Executive Secretary Kagano Sector Ngiriyambonye Elie Social Affairs Sector Bushekeri Dr Nsabimana Damien Director of Kibogora Hospital Dr Mukarugwiro Beatha Kibogora Hospital Dr Birindwa Kibogora Hospital Nsengimana Emmanuel Kibogora Hospital Shumbusho Albert Titulaire Nyamasheke HC Nyirabazungu Xavera Titulaire Rangiro HC Nkomeje Naphtal Titulaire Ruheru HC Ndayisaba Leonard TitulaireYove HC Iyamuremye Marcel Titulaire Karambi HC Basabose Eustache Titulaire Gatare HC Kwitonda Félicien Titulaire Hanika HC Niyonkuru Mathieu Titulaire Kibogora HC Ngirumpatse Bernard President COSA Rangiro HC Nkezabera Ephrem President COSA Yove HC Sinzinkayo Etienne President COSA Gatare HC Nshimyumukiza Emmanuel President COSA Ruheru HC Uwiragiye Suzanne President COSA Nyamasheke HC Irambona Gabriel Sosthene Pastor Free Methodist Church Nsengiyumva Casimir Pastor Pentecostal Church Nyirahabineza Mathilde Delegate of Roman Catholic Chuch Nsabimana Thertullien Pastor Friends Church Nzeyimana Gaspard Pastor Free Methodist Church Kakira Jérémie Pastor Free Methodist Church Kabadege Melene CSP Manager Kwizera Maurice CSP Assistant Manager Ayirwanda Jean Baptiste CSP Admin Assistant Byiringiro Simeon CSP Ntawukuriryayo Fidele CSP 49 World Relief/Rwanda Child Survival Final Evaluation December 2006 Kaberuka Eugene CSP Uzarama Jean de Dieu CSP Ndayisenga Félicien CSP Ndikumana Fidèle CSP Simbarikure Theogene CSP Mayira Gaspard CSP Uwurukundo Roda CSP Nsabiman Marcel CSP Nyirahitimana Immaculee CSP Bienvenu Leon CSP Ngiruwonsanga Narcisse CSP Harerimana Fiacre CSP Yankurije Marcelline CSP Uwamahoro Chantal CSP Ngiruwonsanga Narcisse CSP Ngirabakunzi Nathanael CSP Nsabimana Marcel CSP Barandemera Aphrodis CSP Kayigema François CSP Munanira Daniel CSP Nyiransabimana Jeannette CSP Sikuye Theoneste CSP Siborurrema Esron CSP Mukandahayo Marie Claire CSP Simbarikure Theogene CSP Nyirantezimana Laurentine CSP Ntawukuriryayo Fidele CSP Muteteri Beatrice CSP Nsengimana Martin CSP Muhaweniamana Dancille CSP Mutako Liberé CSP Sinarinziryo Josephine CSP Uwimana Sarah CSP Mukashema Placidie CSP Nyiranzeyimana Beatrice CSP Bavugirije Oreste CSP Mukantagara Xaverine CSP Yandagiye Marie Chantal CSP Rachel Hower WRHQ Anna West WRHQ 50 World Relief/Rwanda Child Survival Final Evaluation December 2006 ANNEX A.3 KIGALI DISSEMINATION MEETING ATTENDANCE LIST WR CSP Final Evaluation National Dissemination meeting Novotel/Kigali Sept 25, 2006 Dr Ilibagiza Denise MOH Dr Mukundwa Aline MOH Mugeni Murasa Cathy MOH Niyitegeka Francois MOH (PNLP) Kagame Eric USAID Safari Venant USAID Gasherebuka Jean Bosco OMS Cyaka Yves PSI Rwanda Achille Karore TWUBAKANE Habimana Marc Nyamagabe District Ndizigiye Dieudonne Nyamasheke District (Dir Health) Dr Nsabimana Damien Director of Kibogora Hospital Melanie Morrow WRHQ Anna West WRHQ Rachel Hower WRHQ Frank Marnkoen KMMT Eudi Rogeo CWW Rose Luz CWW Delphine Pinault CARE Eric Rwagasore Christ Church Kakira Jeremie Free Methodist Church Eio Rose WR Sudan Oloo Vincent WR Sudan Stephan Bauman WRR Jean Paul Ndagijimana WRR Bishyizehagari Thomas WRR Busingye Rose WRR Erisa Mutabazi WRR Senyoni Christian WRR Ngoga Emmanuel WRR Karekezi Charles WRR Uwineza Marie Louise WRR/Child Development Project Melene Kabadege CSP Umucyo Kwizera Maurice CSP Umucyo Mayira Gaspard CSP Umucyo Munanira Daniel CSP Umucyo Muhawenimana Dancille CSP Umucyo Nsengimana Martin CSP Umucyo Nyirantezimana Laurentine CSP Umucyo Nyiranzeyimana Beatrice CSP Umucyo 51 World Relief/Rwanda Child Survival Final Evaluation December 2006 Ntawukuriryayo Fidele CSP Umucyo Muteteri Beatrice CSP Umucyo Ndayisenga Felicien CSP Umucyo Ngiruwonsanga Narcisse CSP Umucyo 52 World Relief/Rwanda Child Survival Final Evaluation December 2006 ANNEX B. FINAL KPC REPORT RWANDA CHILD SURVIVAL PROJECT Rapid Knowledge, Practice and Coverage (KPC) Survey Report For Final Evaluation Nyamasheke District West Province, Rwanda Dates: 15 June - 21 June 2006 Authors: Melene Kabadege Maurice Kwizera Christine Brackett i World Relief/Rwanda Child Survival Final Evaluation December 2006 ACKNOWLEDGEMENTS World Relief Rwanda Child Survival Program wishes to acknowledge and thank the following: Our most sincere thanks are directed to United States Agency for International Development for their fiscal and programmatic support. We thank the Rwanda National Ministry of Health for collaboration in the final half of the program. Warm thanks to the Kibogora Hospital’s administrative board and the chiefs of the Health Centers for their active participation in the KPC survey activities at all levels. Our thanks to World Relief Headquarters and World Relief Rwanda for their continued support, particularly in the KPC report development. Particular acknowledgement goes to the mayor of Nyamasheke District for ongoing support and encouragement. Our thanks and appreciation to the volunteer leaders of the care groups who have kindly guided and introduced the survey team to the local community households. In addition, thank you to the survey team for their diligence and precision in data collection (see Annex 2 for a full list of surveyors and supervisors). Special thanks go to Sheila Etherington and the Free Methodist Mission for ongoing friendship and support. Finally, most sincere thanks go to the mothers of the Nyamasheke District who willingly participated in our survey, and to countless others within the community of Nyamasheke who have supported the Umucyo Child Survival project. ii World Relief/Rwanda Child Survival Final Evaluation December 2006 ACRONYMS ARI Acute Respiratory Infection CSP Child Survival Project DHS Demographic Health Survey EPI Expanded Program for Immunization HBM Home-Based Management of Malaria ITN Insecticide Treated Net KPC Knowledge Practice and Coverage KHD Kibogora Health District ND Nyamasheke District PLWA People Living With AIDS PMTCT Prevention of Mother To Child Transmission PNLP National Malaria Control Program VCT Voluntary Counseling and Testing iii World Relief/Rwanda Child Survival Final Evaluation December 2006 TABLE OF CONTENTS ACKNOWLEDGEMENTS Acronyms EXECUTIVE SUMMARY 1. INTRODUCTION ............................................................................................... vii 1.1 Purpose of the Study...................................................................................... vii 1.2 Purpose of “Umucyo” Child Survival Program............................................. vii 1.3 Program Goals............................................................................................... vii 1.4 Organizational Structure of Umucyo............................................................ viii 1.5 Beneficiary Population: location and size ...................................................... ix 1.6 Situational Analysis........................................................................................ ix 1.7 Summary Health Profile at Baseline............................................................... xi 1.8 Survey Schedule: Dates and activities........................................................... xii 2. METHODOLGY XIII 2.1 Questionnaire: Review and adaptation ............................................................... xiii 2.2 Sampling Design.......................................................................................... xiv 2.3 Data Collection ...............................................................................................60 2.4 Data Analysis..................................................................................................60 3. RESULTS AND DISCUSSION XV 3.1 Characteristics of the Survey Population..............................................................xv THE FOLLOWING TABLES, COMPILED FROM SURVEY DATA, PROVIDE AN OVERVIEW OF THE RESPONDENT POPULATION: XV 3.2 Results for Intervention Indicators ..................................................................... xix TABLE 1: IMCI SICK CHILD: XIX Table 2: Malaria Prevention and rapid care seeking.......................................... xix Table 3: Diarrhea and Hygiene:..............................................................................xx Table 4: Nutrition ................................................................................................... xxi Table 5: Immunization......................................................................................... xxiii Table 6: Reproductive Health and HIV/AIDS................................................... xxiv 4.0 CONCLUSION AND RECOMMENDATIONS XXV iv EXECUTIVE SUMMARY Background: The Knowledge/Practice/Capacity (KPC) survey was conducted from 15 – 21 June, 2006, as part of the final evaluation for the “Umucyo” Child Survival Project (CSP) sponsored by USAID/World Relief. This five-year project (2001-2006) has been implemented in what was formerly known as the Kibogora Health District (KHD), in the former Cyangugu Province, Rwanda. In January 2006, the Rwandan government restructured the health district and government district demarcations as part of national decentralization; the project area has now been absorbed into a larger region that is known as the Nyamasheke District in the new West Province of Rwanda (health districts and government districts have been merged). While our physical project area itself has remained the same, the title of Kibogora Health District (KHD) no longer exists as such; in the rest of this report, the project area will be referred to as a part of Nyamasheke District (ND). Overall project goals of the Child Survival Program (CSP) are to: reduce morbidity and mortality in children 0-5 years and among women of child bearing age (15-49); strengthen the capacity in the project area of the Nyamasheke District (ND) to implement and sustain Child Survival (CS) interventions; and to empower communities to improve their health. When the project began in 2001, the under-five mortality rate in Rwanda was estimated at 196 per 1,000 live births and the maternal mortality rate at 1,300 per 100,000 live births5 . Within the project area in ND, malaria was the primary cause of morbidity and mortality among children under five6 . Other leading causes of childhood death included acute respiratory infection (ARI), diarrhea disease, malnutrition, and HIV/AIDS. Purpose: The purpose of this report is to share project achievements and goals by comparing results from this final survey with data collected in 2001 at the baseline and in 2004 at the midterm evaluation. Survey data reports knowledge, practice and coverage (KPC) within the project area in ND related to the following standard indicators: a) Mothers’ educational background b) Household sanitation knowledge and practices c) Breastfeeding and child nutrition d) Diarrhea case management e) Growth monitoring f) Child immunizations g) Malaria case management h) Maternal care i) HIV/AIDS and other sexually transmitted infections (STI) Setting: The project area in Nyamasheke District is located in the southwestern region of Rwanda, along the shores of Lake Kivu. The total target population is 152,981 people (2006 ND 5 Demographic Health Survey for Rwanda 2000 6 Demographic Health Survey for Rwanda 2000, KPC Baseline Survey World Relief Child Survival Program – Umucyo - July 2006 World Relief/Rwanda Child Survival Final Evaluation December 2006 records). Direct beneficiaries include 35,798 women of child-bearing age and 25,241 children under five years old. Results: The Umucyo CSP is pleased to report achievement in nearly all intervention objectives. Significant rates of achievement over the past five years have occurred in prevention of malaria and diarrhea, immunization, and reproductive health. ITN use among children under five years and pregnant women increased from 3% to 69% and 3.5% to 77%, respectively and care seeking within 24 hours for fever (suspected malaria) in children increased from 4% to 80%. For diarrhea, hand washing in conjunction with 4 key behaviors increased from 0% to 94% among mothers. 96% of children under one year old in the project area have been fully immunized. The number of women who delivered their last child at a health facility increased from 23% to 72%, and women with permission to execute emergency transport plans prior to last delivery increased from 16% to 95%. Data indicates increased knowledge and compliance with nutritional practices, including early initiation of breastfeeding increasing from 38% to 99% since the beginning of the project. At project’s end, 87% of mothers knew how to prepare appropriate weaning food for their children. In addition, the project assisted in rehabilitating 3164 malnourished children (68% of PD/Hearth program participants). Conclusion: Umucyo has achieved results in a wide range of child survival and community health activities. Under the future USAID grant (Expanded Impact), the project area of ND may be a center for training for other child survival programs in Rwanda and surrounding countries. vi vii 1. Introduction 1.1 Purpose of the study Comparison of this final KPC data with that measured at the baseline in 2001 and at the midterm KPC in 2004 will indicate project achievements and impact during the five-year implementation of the CSP. This information will be used by both beneficiary communities and future community stakeholders, including the USAID Expanded Impact grant (“Umusanzu for the Children”) to be implemented in Nyamasheke District and surrounding areas. KPC survey data reports knowledge and practices within the project area in ND related to the following standard indicators: a) mothers’ educational background; b) household sanitation knowledge and practices; c) breast feeding and child nutrition; d) diarrhea case management; e) growth monitoring; f) child immunizations; g) malaria case management; h) maternal care; i) HIV/AIDS and other sexually transmitted infections (STI). 1.2 Purpose of “Umucyo” child survival program USAID and World Relief Corporation have funded this five-year CSP (September 2001- September 2006). World Relief is a faith-based, non-profit organization working to build capacity within communities and promote long-term stability in post-conflict societies. World Relief child survival programs partner with local communities and institutions, seeking to strengthen community ownership and sustainability of development efforts. The WR Rwanda CSP project was named “Umucyo” by the Rwandese project staff. Umucyo means “to illuminate” in the local language of Kinyanrwanda and was deemed an appropriate metaphor to encompass the project’s mission, to educate communities and build their capacity to improve their health status. Specific program goals follow. 1.3 Program goals 1) Reduce morbidity and mortality among children 0-5 and among women of child-bearing age, defined as ages 15-49. 2) Strengthen the capacity of the project area in Nyamasheke District (ND) to implement and sustain Child Survival interventions. 3) Empower communities to make decisions to improve their health status. For complete description of project goals and end of project objectives (EOP), see Annex 1. viii 1.4 Organizational structure of Umucyo and Intervention Activities As a World Relief child survival program, Umucyo is a community-based project that uses the Care Group Model strategy to train and mobilize a vast network of volunteers. Each volunteer is responsible for training and monitoring 10-15 households within their own neighborhood in preventive and care-seeking health practices. Volunteers are then organized into groups of 10 – 15 people, called “care groups”, which meet twice per month. The designated leader of the care group conducts the care group meetings with the assistance of a trained World Relief staff member, called a “promoter”. The promoter shares a health message, and care group members practice training one another in the information being presented, using non-formal techniques such as songs and stories. In the following month, each care group volunteer will visit the 10-15 households for which he/she is responsible and train the mother or caretaker in the same health message. Overseeing the promoters are the project coordinators; each is responsible for the promoters and care group volunteers within the project area. Umucyo has added one area coordinator since the mid-term evaluation, for a total of nine. The area coordinators report to the program manager and assistant manager, who are responsible for all training, technical guidance, logistics and overall function of the Umucyo program. The program manager and all paid staff are Rwandese nationals; most are from the Nyamasheke region and all live within the areas where they work. In addition, as care group meetings are also a time for sharing concerns from the communities and for reporting vital data, such as births, deaths and illnesses, the care group leader has been responsible for reporting data to his/her project area coordinator. Such data will now be reported directly to the local health center. Since the mid-term evaluation, care group leaders have been trained and are prepared to lead care groups and train volunteers without the presence of the promoter. Project interventions mobilized the community in the following topics: diarrhea and hygiene, immunization, HIV/AIDS & STI, Malaria, Nutrition (including Hearth), and Reproductive Health. In partnership with the MOH (PNLP) and the Health District Team, the project’s malaria intervention expanded after the mid-term evaluation to include Home Based Management of Malaria using community distributors of anti-malarial drugs for children age 6- 59 months. The MOH policy on malaria and HBM is currently under review/being developed (see Annex 5 for other MOH standards guiding health workers to improve maternal health, child health, and family planning services). ix Organizational Structure of Umucyo 1.5 Beneficiary population: Location and size The Umucyo CSP covers a significant portion of the Nyamsheke District, encompassing nine health centers and the Kibogora Hospital. The total project area is approximately 648,244 square kilometers. Since the project began in 2001, additional communities from the regions of Kibuye and Gisakura have been added in response to requests from administrative chiefs; KPC survey results do not include these regions. Direct beneficiaries include 35,798 women of child-bearing age (age 15-49) and 25,241 children under five years old. 1.6 Situational analysis The project area in Nyamasheke is entirely rural. The terrain is mountainous and bordered by Lake Kivu to the west, and the Nyungwe Forrest to the east. The humid climate, particularly in Mutovu, Karundura and Kamiranzovu valleys, provide an ideal context for malaria, causing mosquitoes. Road and phone access throughout Nyamasheke remains extremely limited. The region receives adequate rainfall and most inhabitants are subsistence farmers. However, Program Manager Assistant Area Coordinator #5 Promoters Care Group: 10-15 Each Volunteer within Care Group responsible for 10-15 Households 9 Area Coordinators 35 Promoters 286 Care Groups 2,864 Volunteers 29,166 Households AC #1 AC #8 AC#7 AC #9 AC #3 AC #2 AC #6 AC#4 2 1 9 1 8 5 4 3 7 6 x Rwanda is the most densely populated nation in Africa and the average family farms ¼ hectare of land or less, which is inadequate for meeting dietary needs. While the markets are stocked with a variety of fruits and vegetables, malnutrition (low weight for age) was identified at the baseline to be widespread within the region. The primary employers within the area are the mission hospital and a tea plantation in Nyamasheke administrative district, 25 kilometers from the hospital. Those in close proximity to Lake Kivu may also make their living fishing, operating taxi boats, or by trading with Congo. Local markets and small shops support a marginal degree of commerce. Rwanda still suffers from the devastation of the 1994 genocide, referred to locally as “the war”. As of 2001, there were an estimated 210,000 orphaned children, due to the dual devastation of genocide and HIV/AIDS7 . National infrastructure, social fabric and human resources were severely impacted by the massacres. Rwanda lost many qualified teachers and as of 2000, 27.1%8 of women in Cyangugu province were illiterate. The Rwandese family structure was particularly devastated. As of 2000, 36.9% of households were female headed9 . Many women were widowed or their husbands have been imprisoned; over 100,000 people remain in prison awaiting trial for genocide-related crimes. Within the project area in Nyamasheke District, socio-economic conditions worsened between the baseline and mid-term KPC surveys. The economy appears to have stabilized, however, since the mid-term KPC. Although inflation has continued, local government projects have provided jobs in the community (road work, conservation work at Lake Kivu, etc.). The national government has also introduced affordable health insurance; according to the 2006 KPC survey, 86.7% of the community has subscribed, making health care more affordable. Community members have also been encouraged by local authorities to join income-generating associations, and local banks have made small loans available to facilitate local development. Because Rwandan society was impacted so greatly by the genocide, even child-rearing practices have been affected by the lasting effects of trauma experienced by the majority of families. It is sometimes difficult for caretakers to give the same care to a child after the experience of loss of other children and family members. In addition, there are many households in the community headed by women or children struggling to provide for extended families and orphans. In the same way, the stigma attached to HIV/AIDS can impact how children are cared for; children whose parents have HIV/AIDS or who have HIV/AIDS themselves may not be regarded as worthy of care by parents or caretakers. The project has encouraged HIV/AIDS (PMTCT) testing, community-wide, to combat the problem of stigma, and has encouraged care for children regardless of morbidity. 7 HIV/AIDS Information in Africa, 2006, http://www.avert.org/subaadults.htm 8 Demographic Health Survey for Rwanda 2000 9 Demographic Health Survey for Rwanda 2000 xi Life expectancy averages 49 years for women of the West Province10, and at baseline, Rwanda was one of nine African countries most affected by the AIDS virus11. As of 2000, sero￾prevalence rates had reached 10.8% among men and 13.5% among women12. Poor socioeconomic conditions, lack of health infrastructure and education due to the disruption caused by the genocide combine within a context of social and sexual practices were contributing to rapid rates of HIV transmission. However, much progress has been made during the course of the project period in reduction of HIV/AIDS sero-prevalence. Preliminary data from the DHS 2005 survey of Rwanda indicated that sero-prevalence has been reduced to 2.4% among men 15- 49 and to 3.7% among women 15-49 in the West Province. At the same time, data from May 2006 from the Kibogora Hospital indicated that 6% of mothers willing to participate in PMTCT were HIV positive (47/801), indicating there is a continued need for ongoing efforts toward AIDS prevention and care for those affected by the disease. 1.7 Summary: Health profile at baseline At baseline, national infant mortality rates were recorded at 107 per 1,000 live births; the under￾five mortality rate was 196 per 1,000 live births13. Maternal mortality rates were reported by varying sources to range from 787 to 1300 per 100,000 live births14. Nationwide, malaria was found to be responsible for 34% of child morbidity; most children suffered 3 to 6 episodes per year at baseline 15. Kibogora Hospital reported 33% of pediatric deaths due to malaria16. Umucyo’s baseline data reports two-week prevalence for fever among children under five at 74.8%. Nationwide, ARI was the second major cause of under-five morbidity at 21%17. Diarrhea and malnutrition were other leading causes of mortality, especially considering that they exacerbate other childhood infections18. In the project area in Nyamasheke District, dehydration from diarrhea at baseline was determined to be the 3rd leading cause of mortality among children under five. Two-week prevalence for diarrhea at baseline was 66.4%. According to the Rwanda Demographic Health Survey (DHS) 2000, 40% of Rwandese children were moderately or severely stunted (-<2SD), 27% were underweight for age 10 World Relief Rwanda Detailed Implementation Plan, 2002. 11 UN AIDS Fact Sheet, www.unaids.org 12 Demographic Health Survey for Rwanda 2000 13 Demographic Health Survey for Rwanda 2000 14 Kibogora Health District Records, DHS 2000 15 Demographic Health Survey for Rwanda 2000 16 2003 Kibogora Hospital Annual Report 17 Demographic Health Survey for Rwanda 2000 18 Demographic Health Survey for Rwanda 2000 xii (<-SD), and 9% suffered from wasting (-/=25 years xvi Literacy: The situational analysis indicates that 85% of women who have children under two years old in the project area in Nyamasheke District are literate. The literacy rate has steadily improved since the baseline and midterm KPC surveys, most likely due to the increasing number of literacy centers in the area. Figure 3: Occupation Farmer 98.3% Other 0.3% Teacher 1.3% Occupation: As described in the situational analysis, the project area in Nyamasheke District is primarily agricultural. Most families farm crops for their own consumption and use any surplus to sell or trade at the market. Because most respondents farm in order to provide for their families, it is clear that child health and feeding practices are dependent on numerous agricultural conditions and factors. Breastfeeding and child-feeding practices may be affected by seasonal crop schedules, as mothers or primary caretakers may spend long hours outside the home tending to fields. Rapid referral to a health facility for sick children may also be influenced by such demands, or by crop yields, as they provide many families their sole source of income. Figure 4: Gestational Status 6.0% 94.0% Pregnant Not Pregnant xvii Gestational Status Data indicates that only 6.0% of women surveyed are pregnant in the project area. Umucyo has heavily emphasized reproductive health and child spacing in health education interventions in the second half of the project. Figure 5: Distribution of Children by Age 31.3% 20.3% 48.3% 0.0% 10.0% 20.0% 30.0% 40.0% 50.0% 60.0% 0-5 months 6-11 months 12-23 months Age distribution of children: The KPC survey questions targeted the youngest child of the children under two in each household. Almost half (48.3%) of the children were between 12-23 months. xviii Figure 6: Distribution of Children by Sex Female 52% Male 48% Figure 7: Number of Children per Household 32% 60% 8% 1 child 2 children 3 children Sex Distribution / Number of Children per Household: A slightly higher percentage of children are female (52%) than male (48%). It is significant that over half (68%) of the households have at least two children under five years old. The need for child survival activities in order to ensure the health and well being within these families is evident. xix 3.2 Results for intervention indicators For summary table of final KPC results with confidence intervals, see Annex 3. Table 1: IMCI Sick Child. Indicator Baseline KPC% Midterm KPC% Final KPC% EOP Target 1. Percentage of mothers who know at least three signs indicating a sick child needs treatment at health facility. 92.3% 98.0% 99.3% N/A 2. Proportion of mothers who give increased liquid to sick child. N/A 56.8% 90.0% 60% 3. Proportion of mothers who give increased feeding to sick children. 3.7% 67.4% 90.0% 60% Since the midterm KPC survey, Umucyo has focused on maintaining high levels of knowledge of IMCI Sick Child indicators, and has emphasized teaching the need for increased feeding and fluids during illness and recovery to prevent dehydration and malnutrition. Table 2: Malaria prevention and rapid care seeking Indicator Baseline KPC% Midterm KPC% Final KPC% EOP Target 1. Children age 0-23 months who slept under insecticide treated mosquito netting within last 24 hours. 3.0% 66.0% 69.7% 40% 2. Pregnant women who slept under mosquito net within last 24 hours. 3.5% 64.5% 77.8% 40% 3. Mothers who take child age 0-23 months with suspected malaria for treatment within 24 hours of fever either at a health facility OR a distributor (83% of treated children went to a health facility; 17% went to distributor) 3.7% (health facility only) 31.2% (health facility only) 80.4% 50% 4. Fever: 2-week prevalence among children under five. 74.8% 28.7% 17.3% N/A 5. Percentage of women who subscribe to community health insurance N/A N/A 86.7% N/A At the time of the baseline survey, malaria was determined to be the primary cause of morbidity and mortality for children under five years old by all sources consulted. The 2002 Kibogora Hospital annual report reported 33% of child illnesses to be confirmed malaria. As shown in Table 2, prevention efforts revealed by the baseline survey were seriously inadequate. While xx respondents reported knowledge of fever as a danger sign for a sick child (Table 1), rates for timely care-seeking indicated either delay or the complete absence of treatment for children with suspected malaria. Survey results from the final KPC (Table 2) indicate considerable increase in knowledge and behavior regarding malaria prevention (use of ITN nets) and timeliness of care for suspected malaria. In December 2004, the CSP began to partner with the Ministry of Health in a home-based management malaria program. Distributors of malaria medication record how quickly a mother brings her child to be evaluated and given medicine. Data collected by distributors in the home￾based malaria program in May 2006 indicate that 95% of mothers who seek medication from a distributor do so within 24 hours of fever. KPC final survey results show that of the mothers who sought care within 24 hours, 83% went to a health facility and 17% went to distributor. Because the KPC targeted children under 2 years, and distributors cannot treat children under 6 months (they treat children aged 6-59 months), many of those targeted in the survey were not eligible to be treated by a distributor, so it’s not clear what proportion of mothers of children aged 6-59 months choose to take their children to the health center rather than a distributor. It may be that the number of mothers who seek treatment from distributors is decreasing because of the growing number of subscribers to health insurance (86.7%, Table 2). Malaria treatment and additional services are provided at health centers at a relatively low cost (100 Rf) to those who have subscribed to health insurance, compared to the lower cost (50 Rf) for simply receiving medicine from a distributor, without additional services. Table 3: Diarrhea and Hygiene Indicator Baselin e KPC% Midterm KPC% Final KPC% EOP Target 1. Proportion of mothers who wash hands with soap or ash before preparing food, before feeding child, after defecation and after attending to a child who has defecated. 0.0% 39.3% (3 occasions) 94.0% (4 occasions) N/A 2. Proportion of mothers with children from 0- 23 months who increase fluids for a child with diarrhea 31.0% 66.7% 87.2% 50% 3. Proportion of mothers who know the danger signs of diarrhea. 83% (2 signs) 77.7% (3 signs) 97.7% (3 signs) 75% (3 signs) 4. Diarrhea: 2-week prevalence among children under five. 66.4% 26.7% 13.0% N/A 5. % of mothers of children aged 0-23 months who make available soap for washing hands (seen by interviewer) NA NA 97.0% NA xxi 6. % of mothers of children aged 0-23 months who have a latrine in good condition NA NA 84.0% NA 7. % of mothers of children aged 0-23 months who have rubbish pit in the household (seen by interviewer) NA NA 94.0% NA 8. % of mothers of children aged 0-23 months with a dish rack in their household (seen by interviewer) NA NA 89.0% NA Since baseline levels surpassed the original target of 75% for caretakers reporting knowledge of danger signs indicating dehydration, Umucyo increased the indicator from “knows 2 signs” to “knows 3 signs”. Midterm and final data reveal that the new target was also surpassed. The project promoted a wide variety of behaviors to help prevent diarrhea, including hand washing with soap/ash in conjunction with key activities (94% by the final KPC), as well as use of sanitary latrines, rubbish pits, and dish racks. The behaviors were well accepted by the community, and the majority of surveyed households had adopted them. The project also promoted the use of Sur’Eau, a point of use water treatment. Table 4: Nutrition Indicator Baseline KPC% Midterm KPC% Final KPC% EOP Target 1. Children 6-23 months who have received at least 1 dose of Vitamin A per year (evidenced by card) 33.4% 15.4% * 83.4% 80% 2. Nutritional status of children 0-23 months: Children with adequate weight for age ( >-2 SD). 84.1% 71.7% 85.3% NA 3. Nutritional status of children 0-23 months: Children not maintaining adequate weight for age (<-2 SD) 15.9% 28.3% 14.7% NA 4. Mothers who have prepared an appropriate weaning meal for a child 0-23 months (as reported within last 24h). 19.5% 54.2% 87.9%* * 50% 5. Exclusive breastfeeding for children 0-5 months (as reported for last 24h) 60.3% 77.0% 99.0% 75% 6. Mothers who initiate breastfeeding within first hour after delivery. 37.7% 71.3% 90.0% NA *See discussion on recording coverage for immunization and Vitamin A, below (Table 5 comments) **Child 6-23 months who received breastmilk and complementary foods in the last 24 hours xxii Malnutrition prevention and rehabilitation within homes and communities for children who are not in a critical state is facilitated using the PD/Hearth methodology. PD/Hearth largely focuses on improving knowledge and feeding practices among caretakers. Other important components of the program include training mothers in growth monitoring and to recognize symptoms of malnutrition as a serious threat to health and child development. The project has demonstrated progress in reaching program goals, particularly in breastfeeding practices (99% of mothers reported exclusively breastfeeding children under 6 months, for example) and caretaker knowledge (i.e., 90% of mothers report giving their ill child the same or more food during illness; see Annex 3). Focus group responses from PD/Hearth participant mothers indicate that virtually 100% have learned how to prepare a balanced meal for children from locally available foods (16 focus groups in 8 project areas; 127 mothers). Drop-out rates of participant mothers are very low (4.17%), and mothers in 11 out of 14 focus groups said what they liked most about participating in PD/Hearth was that their children were rehabilitated from malnutrition. Mothers in 10 out of 14 groups said it was that they had learned to prepare nutritious meals with local foods. The project has shown success in rehabilitating nearly 68% of all children over time who participated in PD/Hearth sessions, according to project growth monitoring records. The percent of children attending PD/Hearth who achieved and sustained adequate (200 grams) or catch up (400 grams) growth for at least 2 months after Hearth increased from 53.3% at mid-term to 67.8% at the end of the project (12 points lower than the 80% end of project goal). The success rate is higher for PD/Hearth programs that have been continuing growth monitoring of children for over one year (70.7%). In addition, different regions within the project area have reported a range of rehabilitation rates, from 49.79% to 81.15%, depending on the region. Participants in regions with the lowest rates have reported in focus groups that low rehabilitation can be attributed to illness of children, poverty (i.e., not enough money to buy food), high number of children in the household, lack of time for the caretaker to prepare appropriate food, failure of husbands to provide food for the family, severity of children’s malnutrition levels (i.e., children are “low in the yellow part of the growth chart”), and the fact that some children are orphans and don’t have regular care. In addition, the region in the project area that reports the lowest rate of malnutrition rehabilitation is located next to the Nyungwe Forest, where the government has prevented families from farming land in areas surrounding the forest. As competition for land already exists in densely populated Rwanda, the combination of increasing costs and limited land availability forest impacts nutrition rates. Conditions within the target community that contribute to malnutrition are perhaps the most significant obstacle that Umucyo has faced. As reported in the midterm evaluation, between 2002 and 2004 there was a significant increase in cost of living. For instance, the cost of 1 kilo of beans had doubled from 100RF to 200RF, palm oil had more than doubled (200RF to 500RF) and soybeans had tripled from 100-300R. Since 2004, prices have stabilized, which perhaps has impacted malnutrition rates (low weight for age). xxiii Most encouraging is that according to growth monitoring in the community health information system, malnutrition rates in the project area have steadily decreased since the midterm KPC survey, from 34.6% in September 2004 to 20.5% in June 2006. Table 5: Immunization Indicator Baselin e KPC% Midterm KPC% Final KPC% EOP Target 1. % of children age 12-23 months who are fully vaccinated (against the five vaccine￾preventable diseases) before the first birthday. 47.1% 85.5% 96.6% 75.0% 2. Children age 9–23 months who received a measles vaccine 76.6% 87.1% 97.0% NA 2. Maternal TT: mothers who received at least 2 doses before birth of last child as evidenced by card. 43.8% 27.7% 51.3% 50.0% (2d) 3. Maternal TT: mothers who received at least 2 doses as reported by mother missing her card N/A 54.3% 41.7% N/A At baseline, immunization coverage reports varied widely depending on the source of data. However, the KPC baseline indicated only 47.1% of children 9-23 months were fully immunized (as referenced by their cards), compared to 76% nationwide21. Data also indicated at midterm that only 15.4% of children had received at least one dose of Vitamin A (see Table 4, above), an unacceptably low level. As previously mentioned, Expanded Program for Immunization (EPI) in 2002 was limited to a small number of communities, through what was formerly known as the Kibogora Health District, but in addition, inconsistent recording of vaccines on the immunization cards complicated precise measurements for coverage. The project is pleased to report that since the midterm survey, there has been a massive effort on the part of the health centers to update immunization cards and make them as precise as possible; in addition, Umucyo staff have been allowed to assist in updating cards at EPI sessions. Complete immunization coverage is reported at 96.6%, and 83.4% have received at least one dose of Vitamin A. 21 National Ministry of Health, WRR Detailed Implementation Plan xxiv Table 6: Reproductive Health and HIV/AIDS Reproductive Health Baseline KPC% Midterm KPC% Final KPC% EOP Target 1. Families with an emergency transport plan in place prior to last delivery (ambulance/ hammock). 24% 46.7% 79.0% 70% 2. Women with permission to execute emergency transport plan prior to last delivery. 16.4% 42.7% 95.7% 70% 3. Women who report completing at least 3 prenatal check-ups during last pregnancy. N/A 45.0% 91.0% N/A 4. Women who delivered last child at health facility w/ doctor, nurse, auxiliary nurse. 23.1% 35% 72.3% 50% 5. Women who have given birth to a second child more than 24 months after the birth of the previous child 73.1% 77.% 85.7% NA HIV/AIDS 1. Women who know at least 2 methods of HIV/AIDS prevention. 80% 91.3% 99.3% (3 method s) N/A 2. Women who know at least 2 symptoms of STIs 47.0% 92.3% 93.3% 80% 3. Women who report willingness to care for a person with AIDS in their own home. N/A 96.7% 99.3% 80% At the time of the midterm KPC survey, reproductive health indicators were a challenge, particularly deliveries performed by skilled health personnel. As a result, the project increased its efforts to educate and train the community through care groups. A new role in the care groups was created for “maternal health animators” who were trained to heighten awareness among mothers about the importance of delivering at the health centers. Animators also had the responsibility of accompanying mothers to health centers and to perform post-partum visitations. The health centers in the region have agreed to give 300 Rf to animators as an incentive, for every pregnant woman accompanied. Health insurance programs have also encouraged mothers to deliver at health centers because of reduced cost; in addition, the cost of delivery is free at health centers for those who have obtained ante-natal care. The addition of a new health center in the Karambi region of Gatare district enabled more women in the western area of the project to access care with a trained provider, both for antenatal care and delivery. EOP targets have been exceeded, most likely as a combination of all of these efforts. xxv Umucyo’s results regarding HIV/AIDS are encouraging even as the virus continues to threaten health, family structure and economic security in the project area in ND. Umucyo recognizes that the AIDS problem involves and affects all people within any given population; therefore, in addition to the primary intervention efforts that target women and children, the CSP has sought to expand educational efforts to include men and youth, and to promote compassion and home care for people living with AIDS (PLWA). Umucyo has formed three associations of Umucyo volunteers living with HIV/AIDS. The project also mobilized churches to implement associations committed to care with HIV/AIDS in the community; to date, twenty-three groups have been formed for this purpose, groups initiated by churches. Members of the PLWA support groups have been trained by Umucyo to speak about their experience at community gatherings and at Anti-AIDS club meetings for youth that the project also initiated and supports. The project initiated and organized training for twenty￾five anti-AIDS clubs meeting regularly, fifteen at the primary school level and ten at the secondary school level. Forty-three primary schools (over 20,000 children between six and thirteen years old) received HIV intervention training, including prevention of HIV/AIDS and giving care to PLWHA. In addition to prevention efforts through education and sensitivity training, Umucyo also supports and encourages Voluntary Counseling and Testing (VCT) and Prevention of Mother to Child Transmission (PMTCT). During the course of the project, Umucyo has at times matched contributions made for VCT, resulting in the testing and counseling of approximately 13,294 community members at Kibogora Hospital and 20,397 at local health centers. After the midterm KPC survey, three more health centers began to perform PMTCT testing; as a result 5,891 women and 1,767 men have been tested to date (ND health center and hospital records). 4.0 Conclusion and recommendations The Umucyo program has achieved significant results in a wide range of activities. Outstanding achievements beyond target objectives have been measured in the malaria intervention, in diarrhea and hygiene, in exclusive breastfeeding, HIV/AIDS awareness and community-based response, and in immunization coverage. Although the project faced challenges in reproductive health and malnutrition at the midterm KPC survey, remarkable improvement has also occurred in both arenas since that time, particularly in reproductive health. Umucyo volunteers have been very encouraged with these results and as a result have been further motivated to work together to continue impacting the health of the community. Challenges in the community still exist, particularly in regards to socioeconomic barriers that impact malnutrition; it is hoped that there will be new opportunities to assist the community to find creative ways to overcome these barriers. xxvi There is strong evidence that many aspects of the CSP will be sustained in the community, even as the project ends. Local authorities at sector and cell levels have begun to be involved in project activities that are done by volunteers, especially in immunization, diarrhea, hygiene, community-based nutrition programs, maternal health, and malaria, that are now included in the administrative district plans for the community. In fact, World Relief and the COSAs are sharing the costs to maintain one project staff person per health center catchment area, for continued supervision of Care Groups and compilation of data from the community health information system. World Relief’s contribution will gradually decrease until the activities are fully funded by the COSAs and owned by the community. The health centers are also committed to continue encouraging volunteer associations with income generating activities. Volunteers are also prepared to continue their work. In focus groups with PD/Hearth volunteer mothers in July 2006, 100% of volunteers (144) said that they would continue their work without supervision. Informal visitation in July 2006 to a local care group association indicated that the group was, above all, motivated to continue volunteer service because of the very evident results they saw in the community around them. As their friends and neighbors became healthier, even cleaner, the care group volunteers were encouraged to continue making such a strong impact in the lives of those around them. The importance of volunteers cannot be overemphasized. The project could not reach such results without its vast network of volunteers; they are perhaps the most valuable resource the community knows. The care group model and the associations of volunteers for income￾generating activities are extremely important and should be supported and encouraged whenever possible in future interventions in the project area in ND. Many of these volunteers are models in the community and would be excellent trainers or community mobilizers in future interventions. In conclusion, as the project comes to an end, Umucyo staff and volunteers wish to express tremendous gratitude and pride, pleased to have had the privilege to be involved in such a worthy endeavor. While it is tremendous that Umucyo has reached all of its project goals and objectives over the past five years, staff and volunteers recognize that the importance of impacting the community in such a positive way cannot be measured fully; the future potential of such work remains to be seen. xxvii ANNEX 1: PROGRAM OBJECTIVES FROM DETAILED IMPLEMENTATION PLAN Malaria (20% effort): Community-wide education in malaria prevention and treatment seeking behaviors; improved access to Insecticide Treated Nets (ITNs) and re-treatment. ¾ 50% of children with fever (suspected malaria) will be treated within 24 hours at health facility. ¾ 40% of children -2 SD) 256 300 85.3% 79.7% 91.0% 2. Percentage of children age 0-23 months who are underweight (Children not maintaining adequate weight for age -<2 SD) 44 300 14.7% 9.0% 20.3% 3. From monitoring data (not KPC): Children who maintain adequate or catch up growth after 1st Hearth session 3164 4664 67.8% 65.9% 69.7% 4. Percentage of infants age 0-5 months who were exclusively breastfed in the last 24 hours 98 99 99.0% 96.2% 100.0 % 5. Breastfeeding initiation within 1hour of delivery 270 300 90.0% 85.2% 94.8% 6. Percentage of infants age 6-23 months receiving breastmilk and complementary foods 175 199 87.9% 81.5% 94.3% 7. Children 6-23 months who have received at least 1 dose of Vitamin A per year (evidenced by card) 166 199 83.4% 76.1% 90.7% Reproductive Health xxxiv 1. Families with an emergency transport plan in place prior to last delivery (ambulance/hammock) 237 300 79.0% 72.5% 85.5% 2. Women with permission to execute emergency transport plan prior to last delivery 287 300 95.7% 92.4% 98.9% Percentage of children age 0-23 months who were born at least 24 months after the previous surviving child 156 182 85.7% 78.5% 92.9% 3. Women who report completing at least 3 prenatal check ups during pregnancy 273 300 91.0% 86.4% 95.6% 4. Percentage of children age 0-23 months whose births were attended by skilled health personnel (delivered at health facility with doctor, nurse, auxiliary nurse) 217 300 72.3% 65.2% 79.5% 5. Women who delivered last child with trained TBA 8 300 2.7% 0.1% 5.2% STIs and HIV/AIDS 1. Percentage of mothers of children age 0-23 months who cite at least three known ways of reducing the risk of HIV infection 298 300 99.3% 98.0% 100.0 % 2. Percentage of mothers with children age 0- 23 months who know at least two symptoms of STIs 280 300 93.3% 89.3% 97.3% 3. Women who report willingness to care for a person with AIDS in their own home 298 300 99.3% 98.0% 100.0 % xxxv ANNEX 4: KPC QUESTIONNAIRE World Relief Rwanda Umucyo CSP Final KPC Survey QUESTIONNAIRE NO|____| |____| SURVEYOR NAMES : ____________________________________________________ SUPERVISOR NAMES ____________________________________________________ PROVINCE : _________________________ DISTRICT : ________________________ HEALTH CENTER : __________________ AREA : ____________________________ SECTOR : __________________________ CELL:__________________________ AGREEMENT Hello ! My name is……I work with Umucyo project . We are conducting a survey and would appreciate your participation. I would like to ask you about your health and the health of your youngest child under the age of two. This information will help Umucyo to assess whether it is meeting its goals to improve children’s health. The survey usually takes 45 minutes to complete. Whatever information you provide will be kept confidential and will not be shown to other persons. Participation in this survey is voluntary and you can choose not to answer any individual question or all of the questions. However, we hope that you will participate in this survey since your views are important. At this time, do you want to ask me anything about the survey ? Are you willing for me to ask you questions ? Yes No Signature of interviewer :_______________________________ Date : _______________ xxxvi If the mother does not agree, leave the household. If she agrees, ask if she has a child of 0-23 months, then complete the identification and begin to adress the questionnaire. Remember to thank the mother for her responses before leaving her. H. IDENTIFICATION What is your name?………………………………………………………. What is the name of the head of this household…………………………………………….. Do you know how to read? 1. Yes 2. No What is you job ? 1.Farmer 2. Businesswoman 3. Teacher 4. Nurse 5. Other job, specify…………………………. This questionnaire targets mothers (or caretakers) of children less than 24 months of age. 1. (RC#1) RECORD INTERVIEW DATE DAY MONTH YEAR 2. (RC#2) “How old are you?” RECORD AGE OF RESPONDENT IN YEARS: ____ ____ 3. (CSP) “Are you pregnant now?” 1. YES 2. NO 4. (RC#3) “How many children living in this household are under age five?” ____ 5. (RC#4) “How many of those children are your biological children?” ____ 6a. (RC#5) READ ONE OF THE FOLLOWING QUESTIONS BASED ON MOTHER’S RESPONSE TO Q.4: IF ONLY 1 CHILD UNDER FIVE: “What is the name, sex, date of birth, and age of that child?” xxxvii IF MORE THAN 1 CHILD UNDER FIVE: “What are the names, sexes, dates of birth, and age of your two youngest children?” NAME SEX DATE OF BIRTH AGE IN MONTHS 1 1. MALE 2. FEMALE __ __ / __ __ / __ __ DD MM YY 2 1. MALE 2. FEMALE __ __ / __ __ / __ __ DD MM YY 3 1. MALE 2. FEMALE __ __ / __ __ / __ __ DD MM YY 4 1. MALE 2. FEMALE __ __ / __ __ / __ __ DD MM YY 6b. Do you have children aged between five and six years old? 1.YES 2. NO 6c. IF YES, “What are the names, sexes, dates of birth, and age of those children?” NAME SEX DATE OF BIRTH AGE IN MONTHS 1 1.MALE 2.FEMALE __ __ / __ __ / __ __ DD MM YY 2 1.MALE 2.FEMALE __ __ / __ __ / __ __ DD MM YY H.1 H.2 ALL SUBSEQUENT QUESTIONS PERTAIN TO THE YOUNGEST CHILD UNDER AGE TWO xxxviii H.3 Anthropometry 7. Did ( NAME) has been weighed in the last three months? Specify if a. By card b. Verbal report 1. YES 2. NO 8. (RC#6) “May I weigh (NAME)?” 2. YES 3. NOÆ SKIP TO Q.10 9. (RC#7) IF THE MOTHER AGREES, WEIGH THE CHILD AND RECORD WEIGHT BELOW. RECORD TO THE NEAREST TENTH. ___ ___. ___ KILOGRAMS H.4 Maternal and Newborn Care 10. (RC#8) “Before you gave birth to (NAME) did you receive an injection in the arm to prevent the baby from getting tetanus, that is, convulsions after birth?” 1. YES 2. NO Æ SKIP TO Q.12 8. DON’T KNOW Æ SKIP TO Q.12 11. (RC#9) “How many times did you receive such an injection?” (VERIFY WITH CARD) Specify if a. By card b. Verbal report 1. ONCE 2. TWICE xxxix 3. MORE THAN TWO TIMES 8. DON’T KNOW 12. (RC#10) “Now I would like to ask you about the time when you gave birth to (NAME). Who assisted you with (NAME’S) delivery?” A. DOCTOR B. NURSE (A-1 or A-2) (Assume ‘Nurse (A-1 or A-2) if in Hospital) C. AUXILIARY NURSE (A-3) (Assume ‘Auxiliary nurse’ if in health center) D. TRADITIONAL BIRTH ATTENDANT ______________________ (NAME) E. COMMUNITY HEALTH WORKER F. FAMILY MEMBER _______________________________________ (SPECIFY RELATIONSHIP TO RESPONDENT) G. OTHER (SPECIFY) ________________________________ Y. NO ONE 13. Before the birth of (NAME), did you go to Health Center for prenatal check-up? a. YES b. NO Æ SKIP TO Q.15 8. DON’T KNOW Æ SKIP TO Q.15 14. a) How many times did you go to Health Center for prenatal check-up? Specify if a. By card b. Verbal report a. ONE TIME b. TWO TIMES c. THREE TIMES d. FOUR TIMES 14. b) When did you start going to health center for prenatal check ups? 1.DURING THE FIRST TRIMESTER OF PREGNANCY 2.DURING THE SECOND TRIMESTER OF PREGNANCY 3.DURING THE THIRD TRIMESTER OF PREGNANCY xl BIRTH PLAN: 15. (CSP) 15a. “Before the birth of (NAME), did you have a plan in place to get to the health center in case you needed emergency medical care?” 1. YES 2. NO Æ SKIP TO Q.16 15b. “What was that plan?” SPECIFY:_______________________________________ ________________________________________________ 15c. “If an emergency had occurred, could you have followed the plan straight away, or would you have needed to wait for permission from your husband or mother-in-law (or anyone else)?” 1. Already had permission in advance 2. Would have had to wait for permission 16. (CSP) (Pregnant women are not concerned by this question) “Are you currently doing something or using any method to delay or ovoid getting pregnant?” 1. YES 2. NO →SKIP TO Q.18 17.(CSP) ) (Pregnant women are not concerned by this question) “What is the main method you or your husband/partner are you using now to avoid/postpone getting pregnant?” Specify if a. By card b. Verbal report A. BARRIER METHODS B. PILLS C. CONTRACEPTIVE INJECTIONS xli D. NORPLANT E. IUD F. TUBAL LIGATION G. VASECTOMY H. OTHER (SPECIFY) ________________________________ H.4.a Breastfeeding and Nutrition 18.(RC#11) “Did you ever breastfeed (NAME)?” 1. YES 2. NOÆSKIP TO Q.20 19. (RC#12 AND CSP) “How long after birth did you first put (NAME) to the breast?” 1. IMMEDIATELY/WITHIN FIRST HOUR AFTER DELIVERY 2. FROM 1-2 HOURS 3. AFTER TWO HOURS 3.a. (CSP) WITHIN FIRST DAY 3.b. (CSP) AFTER FIRST DAY 20. How many times each day does (NAME) eat ? INCLUDE SNACKS & BREASTMILK 1. ONE TIME EACH DAY 2. TWO TIMES EACH DAY 3. THREE TIMES EACH DAY 4. FOUR TIMES EACH DAY 5. FIVE OR MORE EACH DAY 21. (RC#13) “I would like to ask you about the types of liquids and foods that (NAME) consumed yesterday during the day or at night. Did (NAME) have. . . READ EACH OF THE FOLLOWING AND PLACE A CHECK MARK IN THE BOX NEXT TO EACH ITEM CONSUMED. xlii LIQUID/FOOD CONSUMED IN LAST 24 HOURS? A Breast milk? B Plain water? C Other liquids? (e.g. sorghum liquid) D Mashed, pureed, solid, or semi￾solid foods? SPECIFY RECIPE: ________________ ________________________ ________________________ ________________________ ________________________ ________________________ E Anything else? SPECIFY: ________________________ ________________________ ________________________ 22a. Do you feel comfortable with the amount of food and/or liquid that you have given to your Child in the past week? 1. YES 2. NO 22b. If NO, “Why not?” (Specify)…………………………………………………………. 23a. Did you participate in the Hearth program 1. YES 2.NO 23b. If YES, “Which of your children participated, and when?” Name(s) _____________________ (TIME PERIOD[s]) From...................To......................” Name(s) _____________________ (TIME PERIOD[s]) From...................To......................” Name(s) _____________________ (TIME PERIOD[s]) From...................To......................” xliii Child Immunization 24. (RC#14) “Do you have a card where (NAME’S) vaccinations are written down?” IF ‘YES’ASK “May I see it please?” 1. YES, SEEN BY INTERVIEWER 2. NOT AVAILABLE (lost/misplaced, not in home)ÆSKIP TO Q.26 3. NEVER HAD A CARDÆSKIP TO Q.26 8. DON’T KNOWÆSKIP TO Q.26 25. (RC#15) RECORD INFORMATION EXACTLY AS IT APPEARS ON (NAME’S) VACCINATION CARD. DAY MONTH YEAR BCG POLIO 0 POLIO 1 POLIO 2 POLIO 3 Pentavalent 1 Pentavalent 2 Pentavalent 3 MEASLES VITAMIN A xliv Malaria Prevention 26. (RC#17) 26.a) “Do you have any bed net in your house?” 1. YES 2. NOÆSKIP TO Q. 28 26.b) Can I see it? 1. YESÆ RECORD YOUR FINDINGS: A. MN available B. MN absent 2. NOÆSKIP TO Q. 28 26.c) (If available) When Was the bed net ever soaked or dipped in a liquid to repel Mosquitoes or bugs? 1. LESS THAN 6 MONTHS 2. MORE THAN 6 MONTHS 3. IT IS THE LONG LASTING NET (Tuzanet/Mamanet) 27. (RC#18) “Who slept under a bed net last night?” CIRCLE ALL THAT APPLY. A. CHILD (NAME) B1. RESPONDENT (IF PREGNANT) B2. RESPONDENT (IF NOT PREGNANT) C. OTHER INDIVIDUAL (S) __________________________________ (SPECIFY AND NOTE IF PREGNANT) D. NO ONE Integrated Management of Childhood Illnesses (IMCI) 28. (RC#20) “Sometimes children get sick and need to receive care or treatment for illnesses. What are the signs of illness that would indicate your child needs treatment?” DO NOT PROMPT. CIRCLE ALL MENTIONED. A. DON’T KNOW xlv B. LOOKS UNWELL OR NOT PLAYING NORMALLY C. NOT EATING OR DRINKING D. LETHARGIC OR DIFFICULT TO WAKE E. HIGH FEVER F. FAST OR DIFFICULT BREATHING G. VOMITS EVERYTHING H. CONVULSIONS I. OTHER ________________________________ (SPECIFY) J. OTHER ________________________________ (SPECIFY) K. OTHER ________________________________ (SPECIFY) xlvi 29. (CSP) “What are danger signs that a child with diarrhea needs medical treatment?” ASK MOTHER TO LIST AND CIRCLE ALL RESPONSES—DO NOT PROMPT A. FEVER B. EXTREME THIRST (DEHYDRATION) C. WON’T EAT OR DRINK NORMALLY D. VOMITS FREQUENTLY E. PASSES SEVERAL WATERY STOOLS IN 1-2 HOURS F. BLOOD IN STOOL G. PERSISTENT DIARRHEA (2 OR MORE WEEKS) H. OTHER (SPECIFY): ______________________________ I. OTHER (SPECIFY): ______________________________ 30. (RC#21) “Did (NAME) experience any of the following in the past two weeks?” READ CHOICES ALOUD AND CIRCLE ALL MENTIONED BY RESPONDENT. A. DIARRHEA B. BLOOD IN STOOL C. COUGH D. DIFFICULT BREATHING E. FAST BREATHING/SHORT, QUICK BREATHS F. FEVER G. MALARIA H. CONVULSIONS I. OTHER _________________________ (SPECIFY) J. OTHER _________________________ (SPECIFY) K. NONE OF THE ABOVEÆ SKIP TO Q.35 31. (RC#22) “When (NAME) was sick, was he/she offered less than usual to drink, about the same amount, or more than usual to drink?” 1. LESS THAN USUAL 2. SAME AMOUNT 3. MORE THAN USUAL 32. (RC#23) “When (NAME) was sick, was he/she offered less than usual to eat, about the same amount, or more than usual to eat?” 1. LESS THAN USUAL 2. SAME AMOUNT 3. MORE THAN USUAL ***IF CHILD DID NOT HAVE DIARRHEA, FEVER, OR MALARIA IN PAST TWO WEEKS, ÆSKIP TO Q.35 33. (CSP) IF CHILD HAD DIARRHEA ASK, “When (NAME) had diarrhea, was he/she offered less than usual to drink, about the same amount, or more than usual to drink?” OTHERWISE ÆSKIP TO NEXT Q. (IF CHILD ONLY HAD DIARRHEA IN LAST TWO WEEKS, INTERVIEWER CAN MARK ANSWER BASED ON RESPONSE TO Q.31 WITHOUT RE-ASKING ABOUT FLUID INTAKE.) A. LESS THAN USUAL B. SAME AMOUNT C. MORE THAN USUAL 34. (CSP) IF CHILD HAD FEVER OR MALARIA IN LAST TWO WEEKS ASK: 34a) “Did you seek treatment?” a. YES (ANSWER 34b AND 34c) IN BOX BELOW) b. NO ÆSKIP TO Q.35 34b) “Where did you seek treatment?” CIRCLE ALL THAT APPLY 34c) “How long after the illness began did you seek treatment from (SOURCE IDENTIFIED IN 35 b)?” 1. HEALTH CENTER/ DISPENSARY/ DISTRIBUTOR 1. SAME DAY 2. NEXT DAY 3. MORE THAN 24 HOURS LATER 4. N/A 2. PHARMACY 1. SAME DAY 2. NEXT DAY 3. MORE THAN 24 HOURS LATER 4. N/A 3. TRADITIONAL HEALER 1. SAME DAY 2. NEXT DAY 3. MORE THAN 24 HOURS LATER 4. N/A xlvii 4.OTHER (SPECIFY): 1. SAME DAY 2. NEXT DAY 3. MORE THAN 24 HOURS LATER 4. N/A H.5 HIV/AIDS 35. (RC#24) “Have you ever heard of an illness called AIDS?” 1. YES 2. NOÆ SKIP TO Q.42 36. (RC#25) “What can a person do to avoid getting AIDS or the virus that causes AIDS?” (CIRCLE ALL MENTIONED) A. NOTHING B. ABSTAIN FROM SEX C. USE CONDOMS D. LIMIT SEX TO ONE PARTNER/STAY FAITHFUL TO ONE PARTNER E. LIMIT NUMBER OF SEXUAL PARTNERS F. AVOID SEX WITH PROSTITUTES G. AVOID SEX WITH PERSONS WHO HAVE MANY PARTNERS H. AVOID INTERCOURSE WITH PERSONS OF THE SAME SEX I. AVOID SEX WITH PERSONS WHO INJECT DRUGS INTRAVENOUSLY J. AVOID BLOOD TRANSFUSIONS K. AVOID INJECTIONS L. AVOID KISSING M. AVOID MOSQUITO BITES N. SEEK PROTECTION FROM TRADITIONAL HEALER O. AVOID SHARING RAZORS, BLADES P. OTHER________________________ (SPECIFY) Q. OTHER________________________ (SPECIFY) R. DON’T KNOW xlviii 37. (CSP) Can the virus that causes AIDS be transmitted from a mother to a child? During pregnancy? 1. YES 2. NO 8. DON’T KNOW During delivery? 1. YES 2. NO 8. DON’T KNOW During breastfeeding ? 1. YES 2. NO 8. DON’T KNOW 38. a) CSP) If a mother is infected with the AIDS virus, is there any way to avoid transmission to the baby? 1. YES 2. NO 8. DON’T KNOW 38.b) (If yes) How? 1. PMTCT mentioned 2. PMTCT not mentioned 39. Have you ever been tested for HIV/AIDS? 1. YES 2. NO 8. DON’T KNOW 40. If a relative of yours became sick with the AIDS virus would you be willing to care for him or her in your own household? 1. YES 2. NO 8. DON’T KNOW 41. Would you allow your child to play with a child who has the AIDS virus? 1. YES 2. NO 8. DON’T KNOW 42. (KPC) Apart from AIDS, have you heard about other infections that can be transmitted through sexual contact? 1. YES 2. NOÆ SKIP TO Q.44 3. DON’T KNOW 43. KPC) “What signs and symptoms would lead you to think that a man or woman had such an infection?” RECORD ALL MENTIONED; PROBE: “Any others?” xlix A. ABDOMINAL PAIN B. GENITAL DISCHARGE/DRIPPING C. BURNING PAIN ON URINATION D. REDNESS/INFLAMMATION IN GENITAL AREA E. SWELLING IN GENITAL AREA F. GENITAL SORES/ULCERS G. GENITAL WARTS H. BLOOD IN URINE I. LOSS OF WEIGHT J. IMPOTENCE (IN MEN) K. INABILITY TO GIVE BIRTH (IN WOMEN) L. NO SYMPTOMS M. OTHER (SPECIFY): __________________________ N. OTHER (SPECIFY): __________________________ O. DO NOT KNOW H.6 Hand-washing Practices and household hygiene 44. Before we end, I’d like to ask some more questions. When do you wash your hands with soap or ash? DO NOT PROMPT. CIRCLE ALL MENTIONED. A. NEVER B. BEFORE FOOD PREPARATION C. BEFORE FEEDING CHILDREN D. AFTER DEFECATION E. AFTER ATTENDING TO A CHILD WHO HAS DEFECATED X. OTHER ___________________________ (SPECIFY) 45.Can I see the soap you use to wash hands? SOAP AVAILABLE SOAP NOT AVAILABLE 46.Can I see your latrine? GOOD LATRINE BAD LATRINE LATRINE ABSENT 47. Can I see your rubbish pit? AVAILABLE NOT AVAILABLE 48. Can I see the place you dry your dish after washing up PLACE AVAILABLE PLACE NOT AVAILABLE l H.6.a.i.1.1 Health Insurance 49. Is this household associated to the Health Insurance (MUTUELLE)? Specify if a. By card b. Verbal report 1. YES 2. NO Check if all questions have been asked, and then close the interview. Be sure to thank the respondent for their participation. ANNEX 5: Ministry of Health Standards The Ministry of Health established standards to guide health workers to improve maternal health, child health, and family planning services. These standards direct the Health Care Team to ensure equity of services at all levels and facilitate the training and supervision of staff providing these services. Standards address: - preventive measures for children (e.g., improving vaccination coverage) - treatment of malnutrition among infants - prenatal counseling - family planning - follow-up of labor and delivery (childbirth) 1.4.1.1 Goals for Improving Preventive Measures for Children - to prevent and screen for malnutrition - to prevent TB, polio, diphtheria, tetanus, whooping cough, and measles through vaccination coverage - to prevent vitamin A deficiency related illnesses by coordinating Vitamin A distribution - to screen children for illnesses during health center visits - to screen children for signs of mental retardation - to educate mothers about meeting the health and nutrition needs of their children 1.4.1.2 Goals for Treating Malnutrition Among Infants - to screen for illnesses which lead to malnutrition and treating these underlying illnesses - to prevent vitamin deficiencies and anemia li - to investigate the causes of malnutrition with parents/caretakers and recommend treatment - to educate women regarding the importance of maintaining good nutrition for their children and themselves - to treat children and mothers suffering from malnutrition 1.4.1.3 Goals for Providing Prenatal Counseling - to screen for pregnancies at risk and take appropriate interventions to reduce the risks - to screen for complications during pregnancy and intervene to reduce such complications - to decrease the incidence of tetanus in the mother and child through vaccination coverage - to educate mothers to take preventive measures to reduce complications during pregnancy, delivery, and care of the newborn - to raise the awareness of women regarding the importance of family planning 1.4.1.4 Goals for Family Planning - to provide the community with clear and complete information regarding available family planning methods - to recommend and prescribe family planning methods and interventions appropriate to the clients, considering any contraindications - to skillfully administer the appropriate family planning intervention - to encourage the follow-up of clients to seek care to determine the clients clinical history and laboratory test results if any complaints/complications arise - to ensure that adequate services are provided to infertile couples 1.4.1.5 Goals for Ensuring Adequate Follow-up of Labor and Delivery - to provide early screening for risky pregnancies and take appropriate interventions - to provide early screening for any problems encountered immediately post-partum - to ensure the best possible conditions and environment for delivery - to convey the importance of family planning and preventive services available for children (well child checks, vaccination, and nutrition) lii Annex 6: Cells and Selected Clusters Former Kibogora Health District Cells SECTOR S No CELL POPULATION BUHORO 1 WIMANA 630 2 KAYENZI 800 3 GASHARU 540 4 WINGABE 610 5 GASHWI 812 6 RYASAGAHARA 468 7 NYAKABINGO 732 8 GASEKE 535 9 MURAMA 685 KAGUNGA 10 KANYAMUGIRA 740 11 BUHA 584 12 WIMANA 441 13 KATABARO 464 14 SABURUME 773 15 RUNYINYA 654 16 KAMATARE 432 MACUBA 17 RUTOVU 650 18 MUTONGO 541 19 MBOGO 454 20 RUPANGO 480 21 KANYENKONDO 662 22 NYABIHANGA 478 MWASA 23 NYARUSANGE 1093 24 NYAKABINGO 1405 25 KANYEGE 1267 26 KAJUMIRO 922 27 RUMAMFU 763 28 RWANKUBA 684 RUGANO 29 RUHINGO 978 30 KABUGA 864 31 KANZU 948 32 KAZONGORERO 714 33 NYABITARE 559 34 BUGARAMA 473 35 NYAKIYAGA 724 36 GASAYO 660 BIREMBO 37 RUSOZI 674 38 GITWA 455 39 RWAMIKO 1070 40 KIREHE 836 41 BUTARE 1152 liii RUKANU 42 GATYAZO 902 43 KIJIMA 604 44 BUNYAMANZA 585 45 BITABA 1022 46 RWAMBOGO 326 RUMAMFU 47 NYAGAHINGA 436 48 BITEGA 223 49 KIRAMBIRA 803 50 KAGARAMA 463 51 KIGANDE 1078 GITONGO 52 KABUYAGA 624 53 RYAKAYUMBU 395 54 KABUKUNZI 310 55 GASHARU 316 56 MICYAMO 474 57 KIGUGU 430 KAGANO 58 MPOMBO 941 59 GAKO 1616 60 GITABA 928 61 BISORO 1042 62 BAGARAMA 1536 NYAKABING O 63 GITWA 1400 64 REMERA 893 65 RWISOVU 762 66 RUSHONDI 722 RAMBIRA 67 KADUHA 1047 68 BYAHI 1354 69 MUGOHE 1358 70 NTUMBA 857 TYAZO 71 BIZENGA 771 72 KIBOGORA 1082 73 MASEKA 854 74 KAGARAMA 656 75 RWINYANA 727 76 MUNINI 1085 77 NYAGACACA 823 MURAZA 78 KINYINYA 760 79 GASHASHI 987 80 RUBYINIRO 976 81 GITAMBI 451 82 KIGOTE 771 BUTAMBAR A 83 GASHARU 616 84 KIJIBAMBA 819 85 KAMASERA 517 86 RWESERO 374 87 MUTUSA 1039 MUKINJA 88 GIKOMERO 431 89 GAKOMEYE 622 90 MURAMBI 451 liv 91 KIBARE 706 92 KABUYEKERU 464 MUBUMBAN O 93 RWENGERO 900 94 GASAYO 510 95 MIKINGO 988 96 GACYAMO 845 97 NYAMARAGA 1109 NYAMASHE KE 98 NINZI 993 99 KAVUNE 937 100 MURWA 803 101 NYABAGENI 585 102 GIKUYU 484 NGOMA 103 CYESHERO 524 104 MASHUHIRO 746 105 KADASOMWA 855 106 RUGEREGERE 649 107 NYARUSANGE 474 CYATO 108 CYATO 1672 109 NYAKABINGO 1691 110 MURENGE 1216 111 KAREHE 1273 112 BIGEYO 982 KANJONGO 113 NYABUSAMA 2046 114 KABUYE 1307 115 KIVOGA 1244 116 NYARUHONDO 1446 117 NYAGISASA 1768 118 KORWE 1512 RUHERU 119 MUNYINYA 1651 120 KIBUYE 965 121 KAGANO 301 122 RUSAMBU 407 123 RUBONA 610 124 NYAMATEME 729 125 KANAZI 674 RWUMBA 126 RUHENGERI 1943 127 GASASA 683 128 RWUMBA 3854 129 RUGABE 1058 130 NTSINDUKA 463 131 GASHIHE 1213 132 MUTUNTU 702 YOVE 133 RUSHAHAGA 998 134 YOVE 511 135 GITUNTU 497 136 BWANAMA 907 137 KIZINGA 2895 GAHISI 138 GAKENKE 1944 139 GAHISI 1283 lv 140 RUGANZU 800 141 BANDA 1298 142 GASANANE 1304 MPABE 143 RUDEHERO 693 144 RUGOMERO 681 145 GATAGARA 599 146 GASEBEYA 629 147 BUNYENYEZI 480 148 MUNINI 570 RANGIRO 149 NYARWUNGO 561 150 KIBAVU 678 151 NYARUBASHA 442 152 KANEKE 695 153 NYAKABINGO 974 154 MURAMBI 737 KARAMBI 155 GASOVU 1152 156 KARAMBO 635 157 RUSHYARARA 929 158 RYANYAGAHANGA RA 577 159 MURAMBI 606 160 MISIRIMBO 994 MUGOMBA 161 RUDAGA 1636 162 BITARE 1298 163 KAGEYO 1018 164 RUSHYARARA 1298 165 RWUNAMUKA 1003 166 KARAMBO 1231 NGANGE 167 WIBUNGO 815 168 KAGARAMA 833 169 RUSHUBI 734 170 MWEYA 537 171 KAMONYI 492 172 KABAHIGA 519 173 NYABYUMBA 702 CYIYA 174 MUDUHA 783 175 MUHORORO 534 176 GITOVU 632 177 GITWE 1083 178 RUGABE 1113 Total Population 152981 SECTORS No CELLS CLUSTERS KANJONGO 117 NYAGISASA 1 RUHERU 123 RUBONA 2 RWUMBA 128 RWUMBA 3 RWUMBA 131 GASHIHE 4 YOVE 137 KIZINGA 5 GAHISI 139 GAHISI 6 lvi MPABE 145 GATAGARA 7 RANGIRO 153 NYAKABINGO 8 KARAMBI 159 MURAMBI 9 MUGOMBA 164 RUSHYARARA 10 NGANGE 169 RUSHUBI 11 CYIYA 177 GITWE 12 BUHORO 5 GASHWI 13 KAGUNGA 14 SABURUME 14 MWASA 23 NYARUSANGE 15 MWASA 27 RUMAMFU 16 RUGANO 34 BUGARAMA 17 BIREMBO 41 BUTARE 18 RUMAMFU 49 KIRAMBIRA 19 GITONGO 54 KABUKUNZI 20 KAGANO 62 BAGARAMA 21 RAMBIRA 67 KADUHA 22 TYAZO 72 KIBOGORA 23 MURAZA 78 KINYINYA 24 BUTAMBARA 85 KAMASERA 25 MUBUMBANO 93 RWENGERO 26 NYAMASHEKE 99 KAVUNE 27 NGOMA 106 RUGEREGERE 28 CYATO 110 MURENGE 29 KANJONGO 114 KABUYE 30 lvii ANNEX C. EVALUATION ASSESSMENT METHODOLOGY AND PERSONS INTERVIEWED AND/OR CONTACTED Project Documents and Presentations Reviewed Detailed Implementation Plan Mid-term Evaluation Report and Recommendations Annual Reports Special Reports DHS findings CORE Group HBM Paper Hearth review paper by Christine Brackett (preliminary draft) BASICS PNLP/HBM preliminary review presentation Review of Project Timeline Surveys and Qualitative Assessments: Knowledge, Practice and Coverage (KPC) survey HBM Distributors performance survey Monitoring surveys Key informant interviews with ___ Focus group discussions with: Mothers Care Group volunteers Health Animators and Maternal Health Animators Health center staff Local government leaders HBM distributors Pastoral Care groups PD/Hearth groups and Mama Lumieres Community Development Committee (CDC) members Health center management committee (COSA) members World Relief Rwanda Senior Managers Special gender and equity program impact exercise Review of program, community and DHT Health Information Systems Data comparison with other national statistics (e.g. 2005 DHS) Feedback presentations to stakeholders at district and national level 53 Expanded Impact Project (Concern, IRC and World Relief) proposal review and recommendations provided for applying lessons learned into the new project World Relief Rwanda debriefed at local and national level Written report for submission to USAID 54 ANNEX E. CD-ROM WITH ELECTRONIC COPY OF THE REPORT Enclosed 55 ANNEX F. SPECIAL REPORTS ANNEX F.1 SUSTAINABILITY PLAN Sustainability Plan for Community-Based Child Survival Achievements Nyamasheke District (Former Kibogora Health District) September 28, 2006 The Umucyo Child Survival Project (October 2001-September 2006) operated in the former Kibogora Health District with funding from USAID and World Relief from October 2001-September 2006. The project staff, in partnership with health district and facility personnel, community leaders, churches and 2800 community volunteers worked together to raise key community indicators in maternal-child health. The following are examples of changes in project indicators measured by Umucyo: Indicator November 2001 June 2006 Malaria: Insecticide Treated Net use by children under 2. 3% 70% Appropriate Treatment-seeking within 24 hours for children with suspected malaria. 4% 83% Nutrition/IMCI: Appropriate feeding of the sick child (more food when sick). 11% 90% Exclusive breastfeeding among children under 6 months. 60% 99% Diarrhea: Use of ORS or home available fluids during diarrhea. 31% 80% Hand washing with soap/ash after defecation, after attending to a child who has defecated, before food preparation, and before feeding a child. 0% 94% Maternal & Newborn Care: Deliveries at health facilities. 23% 72% Immunization: Complete immunization coverage by one year of age. 41% 97% 56 Deaths (all causes) of children U5 in health centers and Kibogora Hospital 0 25 50 75 100 125 2001 2002 2003 2004 2005 2006 Year Deaths * Note: Data for 2006 are for January through June only. These achievements were realized in collaboration with other actors in the district, yet the backbone of the community mobilization was the training of one volunteer for every nyumba kumi (10 households). The volunteer relayed intervention messages to her households via home visits twice per month. During the home visits, the volunteer also would make a mental note of vital events (births, deaths, pregnancies, illnesses of public health importance) that are reported back to form the basis for the community health information system. The volunteers were trained in groups called Care Groups, which with the help of World Relief, became over 200 formally recognized associations. Modest annual incentives were given to the volunteer associations, which also earned income from the sale of Sur’Eau, subsidized ITNs and ORS packets. The initial recruitment and training of the volunteers required intensive support from World Relief staff. However, for sustainability purposes, one lead volunteer from each care group was trained to be the intermediary between her group and the health center. Rather than needing many promoters per health center to reach each group individually, it is now possible for just one promoter per health center to follow up with the lead volunteers of each care group, providing periodic updates, supervision, and aggregation and analysis of data collected by the volunteers for the community health information system. 57 Resources and Cost Sharing: With decentralization, health center management committees (COSAS) now have resources to influence staffing at health centers. With the revenue from performance-based incentives (largely achieved through the community-based work of the Umucyo volunteers), it is now possible for the COSAs to contribute to the salary of one promoter per health center who would be retained for ongoing supportive supervision and management of the volunteer associations. Maintenance of health achievements will also require modest support for transportation, registers kept by volunteers, periodic incentives for the volunteer associations, and technical support from World Relief. The Kibogora District Hospital, the director of district health, and World Relief are seeking sources of support to assist these costs as they are gradually transitioned from donors to the health centers over a proposed three year period. WR Rwanda will pay salaries and costs for Oct-Dec 2006 at 100% because District and COSA budgets for 2006 were already set. World Relief’s percentage contribution decreases every 6 months as the COSAs take full responsibility for the continued staffing over the next year(s). Ten former CSP promoters will now be employed by HCs for continued outreach and supervision, and one former CSP coordinator has been placed at Kibogora Hospital to manage their activities. Human Resources: Human resource needs include one promoter-coordinator per health center (10) plus one coordinator at Kibogora Hospital to coordinate and supervise community level activity. At the health center level, the promoter-coordinator will have the following responsibilities: Continue with training, supervision, feedback and encouragement of CHWs; supervise and support volunteer associations; compile and analyze data from health information system; coordinate with HC for outreach and support activities; Coordinate with Expanded Impact project activities (HBM, diarrhea, pneumonia and M&E); Continue to relate to pastor care groups; Supervise growth monitoring and assist with EPI outreach; Relate to COSA, Sector and Cellule CDC and other administrative officials; Assist associations with the income-generation activities, especially the sale of health-related products. The promoter-coordinator will report to the Titulaire of the health center in addition to receiving training and supervision from the CS coordinator at hospital level. At the hospital level, the coordinator will: Compile, analyze and disseminate HIS results, ensure coordination, capacity building and supervision of promoters at each health center; Coordinate with Expanded Impact staff for Nyamasheke. The coordinator would report to the hospital in addition to providing reports to World Relief Kigali. 58 World Relief Rwanda Child Survival Project Final Evaluation December 28, 2006 ANNEX F.2 SUMMARY REPORT ON HBM EVALUATION CONDUCTED IN NYAMASHEKE DISTRICT Preliminary The strategy of Malaria-Home Based Management (HBM) in the former Kibogora Health District started in December 2004. In partnership with Umucyo CSP, the MOH (PNLP) and the Health District Team, the HBM was progressively implemented from one Health Center (Nyamasheke Health Center) and then expanded in all of the Health Centers of former Kibogora Health District. In November 2005, the entire areas of eight HC were covered. Purpose The purpose of the HBM evaluation was to find out the achievements, the community acceptability, the quality of care provided by distributors, the level of clients satisfaction, and the challenges faced by “distributors” and or health facilities staff, in order to make recommendations for improving the strategy in coming years. Targets The evaluation targeted these three categories of people most involved in the strategy: Caretakers (mothers) of children 6-59 months, anti-malaria drug distributors and health centers staff. Methodology In all health centers catchments areas, we conducted KPC Survey and Qualitative Assessment KPC Survey, June 06: Sampling Design A rapid 30-cluster (10 households per cluster) sample was used to sample the catchments area of the project area in Nyamasheke District (approximate number of households). A sampling interval of 5,099 was determined by using the following formula: Sampling interval (SI) = Total survey population (152,981) Total number of clusters (30) SI = 5099 Cluster Selection The starting cluster was selected using a random number table. The next cell cluster was selected by taking the sum of the random number and the sampling interval. Second cluster = Random Number (RN)+SI Identification of the remaining clusters was calculated by adding the sampling interval to the population number of the previous cluster. 59 World Relief Rwanda Child Survival Project Final Evaluation December 28, 2006 Clusters 3-30 = Population within previous cell + SI Household Selection From each cluster, ten households were selected. The first household was selected randomly from the centre of each cluster by spinning a bottle and walking in the direction indicated by the bottleneck. Surveyors moved clockwise from the first household to select the remaining 9 households. Data Collection Data collection occurred over a three-day period from 19-21 June, 2006. Data Analysis The analysis was completed using EPI INFO version 3.2.2. KPC Results Indicator Numerator Denominator Final KPC 1. Children <23m who slept under mosquito net within last 24h. 209 300 69.7% 2. Pregnant women who slept under mosquito net within last 24h. 14 18 77.8% 3. Mothers who take child < 24m with suspected malaria to Distributor within 24 hours of fever 7 51 13.7% 4. Mothers who take child < 24m with suspected malaria to health facility within 24 hours of fever 34 51 66.6% 5. Mothers who take child <23m with suspected malaria to health facility/Distributor within 24h of fever. 41 51 80.3% QUALITATIVE ASSESSMENT, SEPTEMBER 2006 The qualitative assessment was conducted through Focus Group Discussions, Interviews and Tools assessment as follows: ƒ 8 Focus Groups Discussion (FGD) of caretakers who sought treatment for a child with fever in the community in past 3 months, ƒ 61 Interviews and 8 Focus Groups Discussion (FGD) of distributors were organized. In the same way, at least 10 % of distributors in each HC area were chosen randomly (using 60 World Relief Rwanda Child Survival Project Final Evaluation December 28, 2006 the random numbers table). We assessed different tools used by distributors at community level through home visit. ƒ Interviews with Health Center staff in charge of HBM supervision and Assessment of tools used at HC level. MAIN FINDINGS Caretaker’s FGD o All of the mothers interviewed know well at least 3 signs of illness requiring the seek of treatment o All of the mothers interviewed who sought treatment to malaria drug distributors gave correctly the drugs to children and completed the dose on right times. Moreover all confirm that they received counseling on drug use from the distributor. o Almost all mothers stipulated that their children were cured after taking the medicine and they appreciate highly the quality of care received. Distributor’s FGD o All of the distributors who participated in FGD know the danger signs in children with fever requiring the referral to HC. o All of the distributors who participated in FGD know well the correct treatment (blister and dosage) to give to the child according to her/his age. o All of distributors who participated in FGD reported to attend monthly meeting of their association and quarterly meeting with Health center staff. o All of the distributors who participated in FGD affirmed to have been supervised at least once in past 3 months o All of the distributors who participated in FGD confirm that the main challenge they have is to strengthen their income generating associations. Distributor’s Interviews and tool assessment o The interview conducted with the distributor during home visit was combined with the assessment of the distributor tools such as stock card, registers, referral card, blisters, etc. o All of the distributors interviewed know well the correct treatment (blister and dosage) to give to the child according to her/his age. o All of the distributors visited had stock cards, client register, and referral forms well completed. The date of referral, the reason of referral and the treatment provided before the reference were filled in. o All of distributors visited kept blisters in appropriate places (in dry, close and clean box). o Only one distributor confirmed getting stock out in three past months. Health Center staff’s Interviews and tool assessment o Health Center staff confirmed that the strategy is benefic to the community and to the HC, because severe malaria cases had decreased and mother seek treatment before complications occurred. o All Health Center staff stipulated that they received almost all referral from the community distributors and the distributors completed correctly the referral form. 61 World Relief Rwanda Child Survival Project Final Evaluation December 28, 2006 o All of the health centers staff completed the date of the referral arriving and mentioned the diagnosis on the referral card. o All of the health centers had conducted the quarterly meeting of distributors in the last three months. Threats to sustainability o Maintaining the high levels of achievement will require support for supervision and periodic refresher training from the health centers to the Distributors. o HF staff is willing to provide this support but lack sufficient capacity and resources, esp. staff and transportation. Conclusion o All groups interviewed felt that Malaria HBM is effective and would continue, but distributors would need periodic support. o Health Facilities have allocated one staff to continue to support distributors. o Supporting the income generating activities will empower distributor associations to continue their voluntary work. 62 World Relief Rwanda Child Survival Project Final Evaluation December 28, 2006 ANNEX G. PUBLICATIONS AND PRESENTATIONS Roll Back Malaria Africa Regional Meeting November 18, 2004 in Kigali Rwanda “Umucyo Child Survival Program, World Relief” Melene Kabadege, Child Survival Project Director, World Relief Rwanda Christian Connections for International Health May 29, 2006 in Germantown, MD “Care Groups and Community-Based Behavior Change Communication” Melanie Morrow, Director of Maternal & Child Health Programs, World Relief USAID CSHGP Mini-University June 8, 2006 in Baltimore, Maryland “Care Groups − Innovative Strategy for Mobilizing and Sustaining Local Volunteers” Olubukola Ojuola, Child Survival Specialist, World Relief Meeting of the International Council, Medical Ambassadors International August 24, 2006 in Modesto, California “Care Groups and Behavior Change Communication”: Half-day presentation and discussion of lessons learned that MAI could apply to its own programming to increase the scale with which they use health volunteers. Melanie Morrow, Director of Maternal & Child Health Programs, World Relief Annual meeting of the Rainer Arnhold Fellows Program. September 3-10, 2006 in Northern California “Scaling Up Care Groups: Empowering Communities, Saving Lives” Melanie Morrow, Director of Maternal & Child Health Programs, World Relief 63 World Relief Rwanda Child Survival Project Final Evaluation December 28, 2006 ANNEX H. UPDATED PROJECT DATA FORM 64 World Relief Rwanda Child Survival Project Final Evaluation December 28, 2006 65 World Relief Rwanda Child Survival Project Final Evaluation December 28, 2006 66 World Relief Rwanda Child Survival Project Final Evaluation December 28, 2006 67 World Relief Rwanda Child Survival Project Final Evaluation December 28, 2006 68 World Relief Rwanda Child Survival Project Final Evaluation December 28, 2006 69