RWANDA “UMUCYO” (ILLUMINATION) CHILD SURVIVAL PROJECT MIDTERM EVALUATION REPORT Kibogora Health District Cyangugu Province, Rwanda Dates: 1 - 10 September 2004 Henry Perry, Evaluation Team Leader and External Consultant Ndizigiye Dieudonne, Chief of Kibogora Health District Anbrasi Edward Raj, Director of Maternal and Child Health, World Relief Melene Kabadege, Project Manager, Umucyo Child Survival Project, World Relief, Kibogora, Rwanda Maurice Kwizera, Assistant Project Manager and Director of Training, Umucyo Child Survival Project, Kibogora, Rwanda This mid-term evaluation was completed in compliance with Cooperative Agreement HFP-A-00-01-00029-00 for World Relief, 7 East Baltimore Street, Baltimore, MD 21202 World Relief/Rwanda Child Survival Mid-Term Evaluation September 2004 1 "God can use small people to do great things." Statement of Umucyo Project Promoter during the Mid-Term Evaluation World Relief/Rwanda Child Survival Mid-Term Evaluation September 2004 1 Major Project Achievements Diarrhea and Hygiene Intervention 0.0 31.0 39.3 66.7 50.0 0 10 20 30 40 50 60 70 80 90 100 Mothers wash hands before feeding child Mothers increase fluids for child with diarrhea Percentage Baseline Mid-Term EOP Goal STIs and HIV/AIDS Intervention 47.0 80.0 92.3 91.3 80.0 0 10 20 30 40 50 60 70 80 90 100 Women know 2 symptoms of STIs Women know 2 methods of HIV/AIDS prevention Percentage Baseline Mid-Term EOP Goal Nutrition Intervention 60.3 19.5 83.3 50.4 75.0 50.0 0 10 20 30 40 50 60 70 80 90 100 Exclusive breastfeeding in infants < 6 mo. in past 24 hours Appropriate weaning meal provided to children 6 months and older Percentage Baseline Mid-Term EOP Goal Immunization Intervention 47.1 43.8 85.5 54.3 85.0 50.0 0 10 20 30 40 50 60 70 80 90 100 Percentage of children completely vaccinated Mothers obtained 2 TTs during previous pregnancy Percentage Baseline Mid-Term EOP Goal Malaria Intervention 3.0 3.5 3.7 66.0 64.5 31.2 40.0 40.0 50.0 0 10 20 30 40 50 60 70 80 90 100 Children who slept under treated bednet Pregnant women who slept under treated bednet Children treated within 24 hrs of fever Percentage Baseline Mid-Term EOP Goal Reproductive Health Intervention 24.0 16.4 46.7 42.7 70.0 70.0 0 10 20 30 40 50 60 70 80 90 100 Families with an emergency transport plan in place for pregnant woman Women who had permission to execute emergency transport plan during last pregnancy Percentage Baseline Mid-Term EOP Goal Under-Five Mortality 196.0 216.2 78.6 0 50 100 150 200 250 Rwanda (national) Rwanda (rural) Projecf Area (July 2003-March 2004) Deaths per 1,000 live births World Relief/Rwanda Child Survival Mid-Term Evaluation September 2004 1 TABLE OF CONTENTS ACKNOWLEDGEMENTS ACRONYMS AND DEFINITIONS I. MAIN REPORT A. SUMMARY B. ASSESSMENT OF PROGRESS MADE TOWARD ACHIEVEMENT OF PROGRAM OBJECTIVES B.1. TECHNICAL APPROACH B.1.a. OVERVIEW OF PROJECT B.1.b. PROGRESS REPORT BY INTERVENTION AREA B.1.b.i. ACTIVITIES RELATED TO SPECIFIC INTERVENTIONS PROPOSED IN THE DETAILED IMPLEMENTATION PLAN (DIP) B.1.b.ii. PROGRESS TOWARD BENCHMARKS OR INTERMEDIATE OBJECTIVES B.1.b.iii. EFFECTIVENESS OF THE INTERVENTIONS B.1.b.iv. CHANGES IN THE TECHNICAL APPROACHES AND RATIONALE B.1.b.v. SPECIAL OUTCOMES, UNEXPECTED SUCCESSES OR CONSTRAINTS B.1.b.vi. FOLLOW-UP AND NEXT STEPS B.2. CROSS-CUTTING APPROACHES B.2.a. COMMUNITY MOBILIZATION B.2.a.i. COMMUNITY MOBILIZATION ACTIVITIES B.2.a.ii. RESPONSE OF THE COMMUNITY B.2.a.iii. HOW ACTIVITIES HAVE BEEN USED TO REFINE PROGRAM IMPLEMENTATION PLANS B.2.a.iv. BARRIERS PREVENTING COMMUNITY MEMBERS FROM PARTICIPATING IN PROGRAM B.2.a.v. FACTORS AFFECTING PROJECT'S ABILITY TO MOBILIZE THE COMMUNITY B.2.b. COMMUNICATION FOR BEHAVIOR CHANGE B.2.b.i. APPROPRIATENESS AND EFFECTIVENESS OF APPROACH B.2.b.ii.TECHNICAL SOUNDNESS OF MESSAGES B.2.b.iii. DEGREE TO WHICH MESSAGES ARE TEACHING SKILLS AND CHANGING BEHAVIORS AND SOCIAL NORMS B.2.b.iv. MEASUREMENT OF EFFECTS OF BEHAVIOR CHANGE COMMUNICATIONS B.2.b.v. METHODS FOR GATHERING DATA TO MEASURE BEHAVIOR CHANGE B.2.b.vi.INNOVATIVE APPROACHES TO BEHAVIOR CHANGE B.2.c. CAPACITY BUILDING APPROACH B.2.c.i. APPROACH TO STRENGTHENING THE PVO ORGANIZATION B.2.c.ii. APPROACH TO STRENGTHENING THE LOCAL PARTNER ORGANIZATION B.2.c.iii. APPROACH TO HEALTH FACILITIES STRENGTHENING B.2.c.iv. APPROACH TO STRENGTHENING HEALTH WORKER PERFORMANCE B.2.c.v. APPROACH TO TRAINING World Relief/Rwanda Child Survival Mid-Term Evaluation September 2004 2 B.2.d. SUSTAINABILITY STRATEGY B.2.d.i. PROGRESS IN MEETING SUSTAINABILITY OBJECTIVES B.2.d.ii. INITIAL ACTIVITIES IN BEGINNING PHASE OUT B.2.d.iii. APPROACHES TO FINANCIAL SUSTAINABILITY B.2.d.iv. OPINION OF BENEFICIARY COMMUNITY ABOUT ALTERNATIVE FUNDING AT THE COMPLETION OF THE PROJECT B.2.d.v. PROGRESS IN FORMAL SUSTAINABILITY DESIGN METHODOLOGY C. PROGRAM MANAGEMENT C.1. PLANNING C.2. STAFF TRAINING C.3. SUPERVISION OF PROGRAM STAFF C.4. HUMAN RESOURCES AND STAFF MANAGEMENT C.5. FINANCIAL MANAGEMENT C.6. LOGISTICS C.7. INFORMATION MANAGEMENT C.8. TECHNICAL AND ADMINISTRATIVE SUPPORT D. OTHER ISSUES IDENTIFIED BY THE TEAM E. CONCLUSIONS AND RECOMMENDATIONS F. RESULTS HIGHLIGHT II. THE ACTION PLAN III. ATTACHMENTS A. BASELINE INFORMATION FROM THE DIP B. EVALUATION TEAM MEMBERS C. EVALUATION ASSESSMENT METHODOLOGY D. LIST OF PERSONS CONTACTED AND/OR INTERVIEWED E. DISKETTE OR CD WITH ELECTRONIC COPY OF REPORT F. PROJECT DATA SHEET FORM - UPDATED VERSION G. SPECIAL REPORTS SPECIAL REPORT 1. THE ROLE OF SYPHILIS SCREENING AND TREATMENT FOR PREGNANT WOMEN IN CHILD SURVIVAL PROGRAMS IN DEVELOPING COUNTRIES SPECIAL REPORT 2. SUMMARY OF FOCUS GROUP DISCUSSIONS AND KEY INFORMANT INTERVIEWS SPECIAL REPORT 3. NOTES OF DISCUSSIONS WITH PROGRAM STAFF SPECIAL REPORT 4. PRELIMINARY ANALYSIS OF MORTALITY DATA SPECIAL REPORT 5. DESCRIPTION OF THE HEALTH INFORMATION SYSTEM SPECIAL REPORT 6. MIDTERM KPC REPORT World Relief/Rwanda Child Survival Mid-Term Evaluation September 2004 3 Acknowledgements I am grateful to World Relief, its headquarters office in Baltimore, the national headquarters staff in Kigali, and its Project staff in Kibogora for their hospitality and hard work. This was my first trip to Rwanda and only my second trip to Africa, so I had many wonderful new experiences. I am also grateful to Sheila Etherington and others with the Free Methodist Church in Kibogora for their hospitality, good food, and lodging while we were in Kibogora. And, not least, I am grateful to the people of the Kibogora Health District who have become true partners in this effort to improve child survival. World Relief/Rwanda Child Survival Mid-Term Evaluation September 2004 4 Acronyms and Definitions ANC Antenatal care Animators Community health workers that have been selected and supported by the Ministry of Health for many years before the Project began ARI Acute respiratory infection BCC Behavior change communication Cell Geographic unit with approximately 100 households CSP Child survival project DHS Demographic health survey DIP Detailed implementation plan DPT Diphtheria, pertussis and tetanus (immunization) EOP End of project EBF Exclusive breastfeeding EPI Expanded program for immunization HIV Human immuno-deficiency virus IMCI Integrated management of childhood illness IRC International Rescue Committee ITN Insecticide-treated bed net KHD Kibogora Health District KPC Knowledge, practice and coverage LRA Local rapid assessment Mutuelles Programs of group savings on a regular basis to pay for health care when a serious illness arises. This is administered in the community. ORS Oral rehydration solution ORT Oral rehydration therapy PLWA Persons living with AIDS PMTCT Prevention of mother-to-child transmission (of HIV infection) PSI Population Services International Sector Geographic unit with approximately 5 cells Sûr'Eau A solution of 0.5% sodium hypochlorite that disinfects contaminated water and is promoted by PSI STI Sexually transmitted infection TBA Traditional birth attendant TOT Training of trainers TT Tetanus toxoid (immunization) Umucyo Word in local language (Kinyarwanda) meaning "illumination," taken as the name of the World Relief Child Survival Project in Kibogora, Rwanda UNICEF United Nations Children's Fund USAID United States Agency for International Development VCT Voluntary counseling and testing (for HIV infection) World Relief/Rwanda Child Survival Mid-Term Evaluation September 2004 5 I. MAIN REPORT A. Executive SUMMARY The World Relief Rwanda “Umucyo” Child Survival Program targets Kibogora Health District in Cyangugu province, located on the western border of Rwanda on the shores of Lake Kivu. The Umucyo CSP partners with the Health district of Kibogora in targeting 57,505 direct beneficiaries (33,484 women of child bearing age, defined as women aged 15-49, and approximately 24,021 children under five years old). Overall program objectives are: 1) Reduce morbidity and mortality among children 0-5years and women of child bearing age; 2) Strengthen the capacity of the Kibogora Health District (KHD) to implement and sustain Child Survival interventions, and 3) Empower communities to make decisions to improve their health status. The Umucyo CSP has made steady progress in meeting intervention objectives. Outstanding achievements beyond target objectives have been measured within the malaria intervention, in diarrhea and hygiene, in exclusive breastfeeding, HIV/AIDS awareness and community based response, and in immunization coverage. Achievement for Selected Indicators : Indicator BL MT Increase: ITN use: children under 5 3% 66% Increase: ITN use pregnant women 3.5% 64.8% Decrease: 2 week prevalence for fever among children under 5 years. 74.8% 28.7% Decrease: 2 week prevalence for diarrhea among children under 5 years. 66.4% 26.7% Challenges to program goals and overall community health continue to exist, particularly in the area nutrition. Increases in staple food costs combine with limited land resources and income generation opportunities cause more families to suffer from food shortages and malnutrition. In order to address socioeconomic barriers, the project will explore options for income generation or agriculture programs. Umucyo will review all interventions during the second half of the program with intense focus on nutrition concerns for both pregnant women and young children, appropriate weaning practices, basic growth monitoring, and improved feeding practices. In addition, the project will focus on further strengthening the Hearth associations and involving Community Development Committees and Health Centers in order to further capacitate community based response to malnutrition. World Relief/Rwanda Child Survival Mid-Term Evaluation September 2004 6 B. ASSESSMENT OF PROGRESS MADE TOWARD ACHIEVEMENT OF PROGRAM OBJECTIVES This evaluation is truly extensive and in-depth as a result of the mid-term knowledge, practice and coverage (KPC) survey undertaken by the Project and the large number of focus group discussions carried out during the time of the evaluation. Consequently, this report contains a comprehensive assessment of progress made over the past 3 years in achieving Project objectives. B.1. TECHNICAL APPROACH The overall approach is to use simple, effective, community-based sustainable child survival interventions with a major focus on behavior change. Community empowerment, pursuit of equity by ensuring that basic services reach the entire target population, and collaboration with the Ministry of Health District activities and with the Kibogora District Hospital (operated by the Free Methodist Church of Rwanda) through partnership and strengthening activities also constitute fundamental elements of the technical approach. B.1.a. OVERVIEW OF PROJECT Objectives The Umucyo Child Survival Project's goals are: · To reduce morbidity and mortality in children under 5 years of age and in women 15-49 years of age; · To strengthen the capacity of the Kibogora Health District (KHD) to implement and sustain child survival (CS) interventions; and, · To empower communities to make decisions to improve their health. The overall under-five mortality rate for Rwanda is 196.0 deaths per 1,000 live births (and 216.2 for rural Rwanda), and the overall maternal mortality ratio is 787-1,300 per 100,000 births, giving the country some of the highest rates of child and maternal mortality in the world.1 The rates within the KHD are presumed to have been similar to the national rates at the outset of the Project. The national HIV seroprevalence rate is 11%. Malaria is the leading cause of under-five mortality, and diarrhea is the third leading cause. Childhood malnutrition is common, with 42% of children under-five nationally moderately or severely stunted. The baseline knowledge, practice and coverage (KPC) survey for the Project demonstrated that only 47% of children 12- 23 months of age were fully vaccinated by their first birthday (according to their immunization card), and only 43% of women with a child 0-23 months of age had received two doses of TT before the child's birth (according to their immunization card). 1 Rwanda Enquête Démographique et de Santé 2000 (ESDR-II). Office National de la Population, Kigali, Rwanda; ORC Macro, Calverton, MD, USA, 2001. World Relief/Rwanda Child Survival Mid-Term Evaluation September 2004 7 The Project objectives, as shown in Table 1, address these major health problems. Table 1. Project Interventions, Effort, and End-of-Project Objectives Area of intervention Percentage of project effort End-of-project objectives HIV/AIDS and STIs 20% 80% of women will know at least common symptoms of STIs (other than HIV/AIDS) 80% of women will be willing to care for a relative with AIDS in their household Malaria 20% 40% of children 0-23 months of age and 40% of pregnant women will be sleeping under an insecticide￾treated bednet (ITN) 50% of children with fever (suspected malaria) will be treated within 24 hours at a health facility Nutrition and breastfeeding promotion 15% nutrition; 5% breastfeeding promotion 50% of mothers will give appropriate weaning foods (enriched porridge) at least once/day 60% of mothers will offer same amount or more food to child during illness 80% of children who completed the Hearth program will achieve and sustain adequate or catch-up growth for at least 2 months after Hearth 80% of children 6-59 months of age will receive 1 dose of vitamin A capsules per year 40% of children 6-59 months of age will receive 2 doses of vitamin A per year Diarrhea 15% 50% of children with diarrhea will be treated with more fluids than usual 75% of mothers will know at least 3 danger signs of diarrhea requiring medical treatment Immunizations 15% 75% of children will be completely immunized by 1 year of age for polio, DPT (diphtheria, pertussis and tetanus), and measles 50% of pregnant women will receive at least 2 doses of tetanus toxoid (TT) before the birth of their child Reproductive health 10% 50% of women will give birth at a health facility or with a trained traditional birth attendant (TBA) 70% of women will have an emergency plan in place before delivery World Relief/Rwanda Child Survival Mid-Term Evaluation September 2004 8 Project Area The Project is located in southwestern Rwanda, bordering Lake Kivu and the Democratic Republic of Congo (see Figure 2). The land is hilly and mountainous, mostly at an altitude of approximately 5,000 feet. The area is a densely populated rural area consisting mainly of subsistence farmers and their families. Rwanda is the most densely populated country in Africa. Project Area Rwanda World Relief/Rwanda Child Survival Mid-Term Evaluation September 2004 9 The main crops are beans, cassava and sweet potato. Those living near the lakeshore are involved in fishing and in commerce with Congo. Cash crops in the area include tea and coffee. Most family farms have 1/4 hectare or less, which is insufficient to meet a family's nutritional needs. According to the mid-term KPC survey, 73% of women of childbearing age are literate. The area was heavily affected by the 1994 war and genocide, during which 500,000-1 million people (mostly of Tutsi ethnicity) were massacred throughout the country. Another 1.2 million people fled as refugees to nearby Goma, Zaire, where thousands more died. As a result, there are 87 men for every 100 women in Rwanda. The Free Methodist Mission has been working in this area for 60 years now and has an extensive network of churches and schools, and it also owns and operates the Kibogora District Hospital on behalf of the Ministry of Health. This facility has been in operation now for 40 years. The Mission is the largest private employer in the area. The Project area is relatively small, with the most distant point from Kibogora being 48 kilometers away. The roads, while unpaved, are in fairly good repair and readily passable except during the heaviest rainy periods (April and November). The Project area is contiguous with that of Kibogora Health District of the MOH, which contains two of the government's Administrative Districts (Nyamasheke and Gatare Districts). It contains 648,244 square kilometers. There is a population of approximately 9,000 people who reside in the Nyamasheke Administrative District who are not part of the Kibogora Health District. The Project area is composed of a total 145,583 persons, with 34,066 women of childbearing age and 24,021 children under five years of age. The leading causes of childhood death within the Project area are malaria, pneumonia, diarrheal disease, malnutrition and HIV/AIDS. Intervention Mix The major Project activities involve (1) training and support of community volunteers, who are responsible for peer-to-peer education, and (2) strengthening the capacity of the Kibogora MOH District to provide essential services. Because Integrated Management of Childhood Illness (IMCI) has not yet been introduced in Rwanda by the MOH, the decision was made to not include the diagnosis and treatment of childhood pneumonia as an intervention because it was deemed more feasible to postpone this until IMCI is introduced nationwide in the near future, presumably around the time of the completion of this Project. General Program Strategy The Project focuses its efforts on health education of mothers with young children. This education is focused on promoting household behaviors that will prevent disease and also on the utilization of health services for early disease treatment and disease prevention. A strategy of training and supervising one Volunteer for each 10 - 15 houses has been developed,2 and these Volunteers provided carefully targeted messages to each of the households. These Volunteers work 3-4 hours a week and never more than 1 hour on a given day. 2 These 10-household units are referred to a nyumba kumi. World Relief/Rwanda Child Survival Mid-Term Evaluation September 2004 10 The 10 or so women for whom the Volunteer is responsible do not meet as a group. Instead, the Volunteer visits their homes. On average, she visits 3 houses a week. At that time she shares the most recent set of messages she has learned, and she observes the progress each woman has made in implementing the previous health messages the Volunteer has given her. Ten adjacent Volunteers form a Care Group. The Care Groups select a coordinator, called a Chief. The Care Group meets every two weeks with their paid supervisor, called a Promoter. Together, the 10 Volunteers comprising a Care Group are responsible for an administrative area referred to by the government as a Cell, which has approximately 100 households. The Promoters, like the Volunteers, are long-term residents in the area where they work. Cells and Sectors are part of the nationwide program of decentralization. Sectors have 5-7 Cells. Both Cells and Sectors have Community Development Committees (CDCs) which coordinate development efforts in their jurisdiction as part of the national strategy for development. Each Promoter is responsible for 9 Care Groups. Promoters are supervised by paid staff called Area Coordinators. The Area Coordinators visit one Promoter every day, so that a Promoter is supervised in the field every two weeks. In all, there are 2,864 Volunteers (5% of whom are men), 31 Promoters, and 8 Area Coordinators. This structure has been developed by World Relief in its highly successful child survival project in Mozambique and now applied in Cambodia, Malawi, and Rwanda. During each meeting between a Promoter and the Care Group, the Promoter shares a health message and then the Care Group members practice training each other in sharing the information being presented. At the time of the Care Group meeting, births and deaths during the previous month are also reported. The Promoter consolidates this information and reports it to the Area Coordinator. During the next two weeks before the next Care Group meeting, each Volunteer is responsible to visit each of the households under her/his jurisdiction to teach the message. Each Promoter is responsible for 9 Care Groups and s/he visits each Care Group one day every two weeks. The 10th day is devoted to a meeting of the Promoters themselves. The Care Groups meet once every two weeks, and the meeting lasts usually an hour or so. The Care Groups determine the time of the meeting, and the Promoter is responsible to be present. Once a month a sub-group of Promoters meet among themselves in the area where they are working, and at the time of their other meeting each month all the Promoters meet together in Kibogora at the Umucyo office. The Project Manager and Assistant Manager, working with the Area Coordinators, Promoters, and MOH staff, develop a curriculum for each intervention and then Promoters and Volunteers receive training using the curriculum. Then, the Volunteers share this information with the 10 households they are responsible for. During the first 3-4 months of Project implementation, the focus was on developing and implementing the diarrhea and hygiene curriculum. Other interventions were introduced at 3-4 month intervals in the following order: immunizations, HIV/AIDS, malaria, nutrition, and finally reproductive health. World Relief/Rwanda Child Survival Mid-Term Evaluation September 2004 11 At the same time, efforts were underway to strengthen MOH capability in the District by providing training, supplies, logistical support. Insecticide-treated bednets (ITNs) are made available at Health Centers at a reduced cost (about US$ 3.60) to sell to mothers who come for a prenatal visit during the first trimester of pregnancy. The Project assists with the transport of MOH Health Center staff to outreach EPI sites and in the transport of vaccines from the capital (Kigali) to the MOH District Office and then to the outreach EPI sites. The Project also provides transport and food costs for some of the meetings that MOH staff members are required to attend outside of the District. B.1.b. PROGRESS REPORT BY INTERVENTION AREA The Project began in October 2001, with the official inauguration taking place in January 2002. Over the first 3-4 months of Project activity, staff members were recruited, and meetings were held with local authorities and with the MOH staff. Each Promoter made an inventory of all the key resources in his/her area. The Project established the following criteria for selecting Volunteers: · Must be literate · Must have a good reputation in the community and be respected by the community · Must be willing to work on a volunteer basis 3-4 hours a week · Must show maturity (young students were not accepted), preferably be married, and have children · Must have exemplary health behavior, with a clean house, latrine, no malnourished children, and so forth · Preferably have previous experience as a community health worker of some type3 · Preferably be female. Volunteers were selected for a given Cell by a group of local authorities, local Health Center staff, local pastors, the Area Coordinator, and the Promoters working in the Cell. The initial year of the Project was devoted to recruiting Project leadership staff, carrying out the baseline KPC survey, finalizing the DIP, training field staff, establishing working relationships with partners in the area including the MOH and the Kibogora District Hospital, the community, community leaders, and community-based organizations. Actual implementation of field activities did not begin until June 2002. The Project selected a "cascade approach" to the training of staff. Through this approach, the Project Manager and Assistant Manager develop the curriculum for each module (with the collaboration of experts). Then the higher-level staff members are first trained on a topic. They then train the lower-level paid staff members, who then train the Volunteers and other community collaborators. Interventions were introduced sequentially in the following order: 3 The MOH has had a program for many years of training volunteer Community Health Workers called Health Animators. Health Animators, as well as TBAs, were given preference for selection as a Volunteer. World Relief/Rwanda Child Survival Mid-Term Evaluation September 2004 12 1. Diarrhea and hygiene intervention (field activities with Volunteers began in June 2002) 2. EPI intervention (field activities with Volunteers began in October 2002) 3. HIV/AIDS & STI intervention (field activities with Volunteers began in February 2003) 4. Malaria intervention (field activities with Volunteers began in June 2003) 5. Nutrition and Hearth intervention (field activities with Volunteers began in October 2003) 6. Reproductive health intervention (field activities with Volunteers began in January 2004) The planned schedule of activities as outlined in the DIP was followed without any significant deviation except for some modest delays of 1-2 months. The biggest delay was related to difficulty in procuring motorcycles, which were tied up in customs for several months longer than anticipated. B.1.b.i. ACTIVITIES RELATED TO SPECIFIC INTERVENTIONS AS PROPOSED IN THE DETAILED IMPLEMENTATION PLAN (DIP) Diarrhea and Hygiene After the Area Coordinators carried out formative research to learn about diarrhea and hygiene issues and how they are perceived and handled in the Project area, formal discussions were held with the MOH and community leaders about what specific activities the Project might undertake. Then the Project leaders, including the Area Coordinators and World Relief HQ support staff, developed a curriculum that was pre-tested and then taught to the Promoters in a Training of Trainers (TOT) workshop. Then Volunteers and Pastoral Care groups were trained and began their community-based activities in June 2002. Training and Project activities included the following: · Education about diarrhea and its causes and effects, · Promotion of household cleanliness (including building latrines and crude wooden stands on which to set dishes after washing them), · Promotion of personal hygiene (handwashing after defecation and before eating, bathing, etc.), · Treatment of water before drinking with either Sûr'Eau or by boiling it, · Treatment of childhood diarrhea with home-made ORS, increased fluids, and increased food intake, · Recognition of danger signs indicating the need for referral. EPI After the Area Coordinators carried out formative research in the communities to learn about issues related to vaccine-preventable diseases and to the utilization of EPI services (including logistical issues), formal discussions were held with the MOH and community leaders about what specific activities the Project might undertake. Then the Project leaders, including the Area World Relief/Rwanda Child Survival Mid-Term Evaluation September 2004 13 Coordinators and World Relief HQ support staff, developed a curriculum that was pre-tested and then taught to the Promoters in a Training of Trainers (TOT) workshop. Then Volunteers and Pastoral Care Groups were trained and began their community-based activities in October 2002. MOH staffed were also given refresher training in EPI. The Project activities included the following: · Education of mothers about the importance of EPI, · Transport of vaccines from Kigali to the Kibogora District MOH Office and from the Health Centers to the EPI outreach sites, · Transport of Health Center staff to EPI outreach sites, · Notification of mothers when the EPI outreach session nearest their village were schedules to be held, · Follow-up of mothers whose child had fallen behind in his/her immunization schedule, and, · Assisting Health Center staff with the EPI outreach session. HIV/AIDS & STI After the Area Coordinators carried out formative research in the communities to learn about issues related to HIV/AIDS and STIs, formal discussions were held with the MOH and community leaders about what specific activities the Project might undertake. Then the Project leaders, including the Area Coordinators and World Relief HQ support staff (including the HQ HIV/AIDS Trainer), developed a curriculum that was pre-tested and then taught to the Promoters in a Training of Trainers (TOT) workshop. Then Volunteers and Pastoral Care Groups were trained and began their community-based activities in February 2003. Among the activities that the Project carried out were the following: · Education about HIV/AIDS and STIs, · Formation of 9 support groups for PLWAs, · Formation of 17 Anti-AIDS clubs, · Training of 89 AIDS facilitators and 229 school teachers and promotion of AIDS awareness in schools and in community groups, with 2,615 students receiving education about HIV/AIDS, and, · Promotion of VCT among Volunteers (80% of the 2,864 Volunteers and their partners obtained VCT, as did approximately 3,500 other community members). The number of persons obtaining VCT show a significant increase beginning at the time of the implementation of the HIV/AIDS and STI Intervention in March 2003. The number of persons tested in 2003 was more than twice the number tested in 2002, and, at the rate exhibited for the first 8 months of 2004, the total number tested in 2004 should exceed the number tested in 2003 by another 14%. World Relief/Rwanda Child Survival Mid-Term Evaluation September 2004 14 Number of Persons Screened for HIV Infection through Voluntary Counseling and Testing, January 2001-August 2004 0 100 200 300 400 500 600 700 800 2001 2002 2003 2004 HIV/AIDS and STI Intervention began Malaria After the Area Coordinators carried out formative research in the communities to learn about issues related to malaria, formal discussions were held with the MOH and community leaders about what specific activities the Project might undertake. Negotiations were undertaken with PSI regarding the distribution of insecticide treated bednets (ITNs). Then the Project leaders, including the Area Coordinators and World Relief HQ support staff, developed a curriculum which was pre-tested and then taught to the Promoters in a Training of Trainers (TOT) workshop. Then Volunteers and Pastoral Care Groups were trained and began their community￾based activities in June 2003. Local drug sellers received training in the signs of malaria and the proper treatment, and the MOH District staff also received refresher training in malaria diagnosis and treatment. Among the activities that the Project carried out were the following: · Education about malaria, its causes, symptoms and treatment, · Distribution of bed nets to homes with children aged less than 5 years or with pregnant women, and, · Provision of the opportunity to women who come to the Health Center in the first trimester of pregnancy to buy an ITN at a subsidized price (for 2,100 francs, about US$ 3.60). Nutrition and Hearth Program After the Area Coordinators carried out formative research in the communities to learn about issues related to childhood malnutrition, formal discussions were held with the MOH and community leaders about what specific activities the Project might undertake. Then the Project leaders, including the Area Coordinators and World Relief HQ support staff, developed a curriculum which was pre-tested and then taught to the Promoters in a Training of Trainers (TOT) workshop. Then Volunteers and Pastoral Care Groups were trained and began their community-based activities in October 2003. MOH District staff members were included in the training. Then, the MOH staff and Project staff carried out growth monitoring in the communities to determine which children would be eligible for enrollment in the Hearth Program. In October 2003, the Project hosted a regional workshop led by Drs. Warren and World Relief/Rwanda Child Survival Mid-Term Evaluation September 2004 15 Gretchen Berggren, the originators of the Hearth Model. Child survival program leaders from a number of African countries attended this as did Umucyo Project staff members. Reproductive Health After the Area Coordinators carried out formative research in the communities to learn about issues related to reproductive health, formal discussions were held with the MOH and community leaders about what specific activities the Project might undertake. Then the Project leaders, including the Area Coordinators and World Relief HQ support staff, developed a curriculum which was pre-tested and then taught to the Promoters in a Training of Trainers (TOT) workshop. Then Volunteers and Pastoral Care Groups were trained and began their community-based activities in January 2004. World Relief/Rwanda Child Survival Mid-Term Evaluation September 2004 16 B.1.b.ii. PROGRESS TOWARD BENCHMARKS OR INTERMEDIATE OBJECTIVES In early August 2004, the Project conducted a mid-term KPC survey, enabling an assessment of progress in reaching end-of-project objectives. The complete report is attached in Attachment F.1. The survey was carried out by MOH health staff, Project staff (interviewing in communities in which they don't normally work). The key findings are shown in Table 2. Table 2. Comparison of Monitoring Indicators from Baseline to Mid-term measurements to End-of-Project Targets 3. Women who report willingness to care for a person with AIDS in their own home DNC 96.7% 80% Malaria: Prevention and Rapid Treatment 1. Children aged 0-23 months who slept under mosquito netting within previous 24 hours 3.0% 66.0% 40% 2. Pregnant women who slept under mosquito net within previous 24hours 3.5% 64.5% 40% 3. Mothers who take child 0-23 months with suspected malaria to health facility within 24 hours of developing fever 3.7% 31.2% 50% 4. Prevalence of fever during the previous 2 weeks among children 0-59 months 74.8% 28.7% GNS DNC: data not collected GNS: goal not specified Indicator Baseline KPC% Midterm KPC% EOP Target Diarrhea and Hygiene 1. Mothers who wash hands with soap or ash before feeding child 0.0% 39.3% GNS 2. Mothers with children aged 0-23 months who increase fluids for a child with diarrhea 31.0% 66.7% 50% 3. Mothers who know danger signs of diarrhea 83.0% (2 signs) 77.7% (3 signs) 75% (3 signs) 4. Prevalence of diarrhea in previous 2 weeks among under-5s 66.4% 26.7% GNS Immunization 1. All immunizations obtained by first birthday among children 12-23 months of age 47.1% 85.5% 75% 2. Maternal TT: mothers who received at least 2 doses before birth of last child as evidenced by card 43.8% 27.7% 60% 3. Maternal TT: mothers who received at least 2 does as reported by the mother DNC 54.3% GNS HIV/AIDS & STIs 1. Women who know at least 2 methods of HIV/AIDS prevention 80.0% 91.3% GNS 2. Women who know at least 2 symptoms of STIs 47.0% 92.3% 80% World Relief/Rwanda Child Survival Mid-Term Evaluation September 2004 17 Nutrition 1. Breastfeeding initiation within 2 hours of delivery 37.7% 71.3% GNS 2. Exclusive breastfeeding for children 0-5 months (as reported for previous 24 hours) 60.3% 83.3% 75% 3. Mothers who have prepared an appropriate weaning meal for a child 0-23 months (as reported within previous 24 hours) 19.5% 50.4% 50% 4. Mothers feeding a child 0-23 months of age 5 or more times per day DNC 78.0% GNS 5. Children 6-23 months of age who have received at least 1 dose of vitamin A during previous year (evidenced by card) 33.4% 15.4% 80% 6. Hearth: children who maintain adequate or catch-up growth after first Hearth session DNC 53.3% 80% 7. Nutritional status of children 0-23 months: children with adequate weight-for-age ( weight￾for-age >-2 SD) 84.1% 71.7% GNS Reproductive Health 1. Women who report completing at least 3 prenatal check ups during last pregnancy DNC 45.0% GNS 2. Women who delivered previous child at health facility with doctor, nurse, or auxiliary nurse 23.1% 35.0% 50% 3. Women who delivered previous child with trained TBA 20.1% 13.3% GNS 4. Women with permission to execute emergency transport plan prior to last delivery 16.4% 42.7% 70% 5. Families with an emergency transport plan in place prior to previous delivery (ambulance or hammock) 24.0% 46.7% 70% 6. Percentage of children 0-23 months who were born at least 24 months after the previous surviving child 76.6 77.0 GNS IMCI 1. Mothers understanding at least two signs that would indicate the necessity for treatment of a sick child at a health facility 92.3 98.0% GNS 2. Mothers who give increased liquids when child sick DNC 56.8% 60% 3. Mothers who increase feeding or given the same amount of food when child sick 11.2% 67.4% 60% DNC: data not collected GNS: goal not specified The improvements as demonstrated by the indicators are nothing short of truly exceptional. For the Diarrhea and Hygiene Intervention, handwashing increased from a baseline of 0% to 39%, and the percentage of mothers giving increased fluids when their child had diarrhea doubled. The percentage of mothers who know at least three dangers signs exceeded the end-of-project objective. How much it increased from baseline is difficult to know since the baseline measure is knowledge of 2 rather than 3 indicators. World Relief/Rwanda Child Survival Mid-Term Evaluation September 2004 18 The percentage of children with diarrhea during the previous two weeks fell by more than half. However, some of this drop is most certainly a seasonal effect: the baseline data were collected during December 2001 during the rainy season, when rates of diarrhea are most certainly higher than in August 2004, when the mid-term data were collected. August is in the dry season. . Diarrhea and Hygiene Intervention 0.0 31.0 39.3 66.7 50.0 0 10 20 30 40 50 60 70 80 90 100 Mothers wash hands before feeding child Mothers increase fluids for child with diarrhea Percentage Baseline Mid-Term EOP Goal With respect to the Immunization Intervention, the improvements in EPI coverage are remarkable. The coverage among children nearly doubled and just exceeded the end-of-project objective. The data for maternal TT are difficult to determine. The coverage as determined by card review declined, but it was clear to everyone that this did not represent reality because the recording of maternal immunizations on health cards was not always carried out. If the verbal reports of mothers are accurate, then the maternal TT coverage has increased by approximately one-fourth and just exceeds the end-of-project objective. World Relief/Rwanda Child Survival Mid-Term Evaluation September 2004 19 Immunization Intervention 47.1 43.8 85.5 54.3 85.0 50.0 0 10 20 30 40 50 60 70 80 90 100 Percentage of children completely vaccinated Mothers obtained 2 TTs during previous pregnancy Percentage Baseline Mid-Term EOP Goal For the HIV/AIDS & STI Intervention, the percentage of women who know at least two symptoms of STIs almost doubled. A slight improvement was observed in the (already high) percentage of women who know at least two methods of HIV/AIDS prevention, starting from a high baseline level of 80% to 91%. Practically all the respondents stated that they would be willing to care for someone with AIDS in their home. Unfortunately, baseline data were not collected for this latter indicator. STIs and HIV/AIDS Intervention 47.0 80.0 92.3 91.3 80.0 0 10 20 30 40 50 60 70 80 90 100 Women know 2 symptoms of STIs Women know 2 methods of HIV/AIDS prevention Percentage Baseline Mid-Term EOP Goal World Relief/Rwanda Child Survival Mid-Term Evaluation September 2004 20 Concerning the Malaria Intervention, we observe a dramatic increase in the percentage of the target population using ITNs. For both under-2-year-old children and for pregnant women, the percentage using an ITN rose from 3% to around 65%, substantially exceeding the end-of-project target of 50% for both groups. The percentage of mothers taking their child to a Health Center within 24 hours of developing a high fever also showed a remarkable increase, although not quite as dramatic - from 4% to 31%. Finally, the prevalence of high fever during the previous two weeks among children aged less than 5 years shows a dramatic drop, from 75% to 29%. However, the seasonal effect for malaria prevalence is most certainly similar to that for diarrhea - higher during the rainy season (when the baseline data were collected) and lower in the dry season (when the mid-term data were collected). Malaria Intervention 3.0 3.5 3.7 66.0 64.5 31.2 40.0 40.0 50.0 0 10 20 30 40 50 60 70 80 90 100 Children who slept under treated bednet Pregnant women who slept under treated bednet Children treated within 24 hrs of fever Percentage Baseline Mid-Term EOP Goal The Nutrition Intervention shows marked improvements in nutrition-related behaviors. The percentage of mothers who initiated breastfeeding soon after birth almost doubled, and the prevalence of exclusive breastfeeding increased by 23%, exceeding the end-of-project objective. The percentage of mothers who prepared an appropriate weaning meal more than doubled and just exceeded the end-of-project objective. Almost 4 out of 5 mothers are feeding their child at least 5 times a day. The levels for the indicators concerning vitamin A distribution and nutritional status appear to have shown a decline from the baseline survey. The percentage of children who received a vitamin A capsule during the previous year as measured from the health cards fell from 33% to 15%. But this, like the maternal TT immunization coverage as measured from the health cards, can be explained by the inability of the staff to register this information at the time of distribution. The percentage of children who completed the Hearth sessions who did not exhibit adequate growth fell far short of the end-of-project objective (53% vs. 80%), but unfortunately there is no World Relief/Rwanda Child Survival Mid-Term Evaluation September 2004 21 baseline data for comparison. Finally, and perhaps most importantly, the percentage of children with adequate weight-for-age fell by 12 percentage points, from 84% to 72%. Seasonal effects possibly could account for the less-than-hoped for results concerning nutritional status. August of this year, when the mid-term data were collected, was a period at the end of an extended dry period and marked increase in food prices, and there were severe food shortages. When the baseline data were collected in December 2001, the harvest had been collected and food supplies were relatively abundant. However, there were numerous comments made in the focus group discussions that many families have become landless during the past 10 years because of population growth, leading to higher levels of childhood malnutrition. Nutrition Intervention 60.3 19.5 83.3 50.4 75.0 50.0 0 10 20 30 40 50 60 70 80 90 100 Exclusive breastfeeding in infants < 6 mo. in past 24 hours Appropriate weaning meal provided to children 6 months and older Percentage Baseline Mid-Term EOP Goal The Reproductive Health intervention was implemented only beginning in January 2004, only 6 months before the mid-term KPC survey. Even so, remarkable progress has been made. There is a fairly high level of utilization of prenatal care (45% obtained three or more visits during the last pregnancy), but, unfortunately, there is no baseline measure for comparison. There was a 52% increase in utilization of the Health Center for delivery (from 23 to 35%). This increase must have been related to some kind of "halo effect" associated with all the other activities carried out by the Project that were not directly the focused promotion of delivery at the Health Center since many, if not most, of the deliveries referred to in this indicator occurred before the Reproductive Health Intervention was introduced. World Relief/Rwanda Child Survival Mid-Term Evaluation September 2004 22 Reproductive Health Intervention 24.0 16.4 46.7 42.7 70.0 70.0 0 10 20 30 40 50 60 70 80 90 100 Families with an emergency transport plan in place for pregnant woman Women who had permission to execute emergency transport plan during last pregnancy Percentage Baseline Mid-Term EOP Goal The qualitative assessment (discussed later) provides suggestive evidence that the overall utilization of health services at the Health Centers and at the Kibogora District Hospital has increased as a result of Project activities in the villages, so the increase in utilization of Health Centers for delivery may be one aspect of this overall trend unrelated to the specific Reproductive Health Intervention per se. The decline in the utilization of trained TBAs may reflect an increase in utilization of the Health Center among those who in the past were using a trained TBA in the community. Dramatic progress has been made in the availability of an emergency transport plan in the community and in the capacity of women to execute that plan if they feel the need. Again, it appears that progress on these issues must have been made early in the Project, prior to the formal implementation of the Reproductive Health Intervention. While strong progress was made on these two indicators, with levels doubling at least, the end-of-project goals are still substantially higher than the levels observed at the mid-term. Although there is no specific IMCI intervention, the Project has made significant progress in promoting simple behaviors that can be effective in reducing mortality from childhood illness, namely early seeking of health care treatment when warning signs develop as well as providing increased fluids and food when the child is sick. Almost all of the mothers (98%) know at least two danger signs for which they should seek treatment for their child, and 56% of mothers say they now give increased liquids when their child is sick. Unfortunately, baseline measures of these indicators were not obtained. However, the percentage of mothers who say they give increased food to their child when it is sick increased from 4% at baseline to 42% at the time of the mid-term KPC. World Relief/Rwanda Child Survival Mid-Term Evaluation September 2004 23 Just as the actual decline in diarrhea and malaria prevalence is most certainly overstated because of the difference in seasons when the baseline and mid-term surveys were carried out, the lack of progress with reducing levels of malnutrition is most certainly explained by seasonal changes in levels of malnutrition rather than lack of effectiveness of the Nutrition Intervention. Food is scarce in August and much more abundant in December. Out of 18 indicators measured in the mid-term KPC for which EOP targets have been established, 9 of the indicators surpassed the EOP goals at the time of the mid-term KPC. B.1.b.iii. EFFECTIVENESS OF THE INTERVENTIONS The quantitative achievements of the Project are extraordinary, and reflect a remarkable change in behaviors in the communities during a relatively short period of time. These achievements reflect the technical quality of the interventions as well as the strength of the approach for reaching, mobilizing and, in fact, empowering the community. These findings are reinforced by the qualitative data collected from focus groups in the community and from key informant interviews. Forty-four different focus group discussions were held with mothers, Volunteers, TBAs, pastors, school teachers, Anti-AIDS Club members, Health Center staff, Cell and Sector Coordinators, and Community Development Committee members. Key informant interviews were held with the MOH District Chief and with the hospital leadership. The details of these discussions are reported in Attachment F.4. Here, I will give an overview of the focus group discussions and key informant interviews. Summary Conclusions of the Qualitative Assessment There is great enthusiasm within the community and among community leaders for the Project because: · The Project involves the entire community – working “within” the community, not “on top of” the community, · People and communities are feeling empowered, · The poorest are being reached and helped, · The Project has been effective in reducing illness (mainly diarrhea and malaria), and, · The number of deaths among children is declining. There is increased utilization of health services of all types, but particularly: · Immunization services, · Treatment of illnesses at the Health Centers · Deliveries at the Health Center, and, · Transport of pregnant women to the Kibogora District Hospital. There is a greater solidarity within the community as a result of the Project: · People are communicating among themselves better, · Barriers have been broken down, and, · People are starting to work together better for the common good. World Relief/Rwanda Child Survival Mid-Term Evaluation September 2004 24 There have been notable changes in behavior as a result of Project activities: · Handwashing has increased, · Hygiene and cleanliness have improved, · People are taking care to drink safe water by either boiling it or using Sûr'Eau, · People are building latrines and repairing those that have fallen into disrepair, · People are using insecticide-treated bednets, · The women report, “Our husbands are happier that their houses are cleaner, and they love us more,” · Men are now motivated to help – they are taking their children for health care, transporting women to the hospital, digging latrines and rubbish pits, and building tables to put dishes on, · The new knowledge and training people have received have become “light” for the community (they now understand that malaria and diarrhea not caused by demons or poisons), and, · There is now earlier utilization of health services before illnesses become severe – especially for malaria. Concerning the Diarrhea Intervention, the qualitative assessment confirmed the quantitative findings. Mothers now know how to prevent and how to manage diarrhea. There is better hygiene and the houses are cleaner. Many reports from the focus group discussions indicated that there is now less childhood diarrhea and fewer diarrhea deaths. With respect to the HIV/AIDS Intervention, the focus group discussions revealed that people are eager to have more ready access to VCT, and they appreciate the Project’s support of this activity. Furthermore, they are eager to learn more about how to support PLWAs and AIDS orphans, and there is less stigmatization. For example, focus group participants said that not only do "adulterers" and non-Christians get AIDS, but "even good Christian people get AIDS.” The focus group participants also reported that they have noticed that there are fewer cases of malaria now, and also fewer cases of severe malaria. Furthermore, the respondents attributed this to the distribution of ITNs by the Project. The respondents commented that they appreciated that the poorest household have been able to obtain bednets. The Health Center staff members participating in the focus group discussions were also reporting that they are seeing more cases of people coming to the Health Centers for early treatment of malaria. Concerning the Nutrition Intervention, the focus group respondents reported that they like the Hearth Program because it enables mothers to manage and monitor malnutrition themselves with their own food. There were also reports that the Kibogora Hospital was seeing more malnourished children being brought to the Kibogora District Hospital. (Whether this represents a higher level of malnutrition or a greater willingness of parents to seek treatment is not clear). There were fewer comments concerning the Reproductive Health Intervention, probably because it started so recently and also because it has been overshadowed by the other interventions which have demonstrated such remarkable progress. However, the focus group participants reported more pregnant women going to the Health Centers for ANC. The Project's policy of making ITNs available to pregnant women who go to the Health Center during their first trimester of World Relief/Rwanda Child Survival Mid-Term Evaluation September 2004 25 pregnancy has been effective. TBAs who were interviewed reported that more mothers are now going to the Health Center or the hospital for delivery. The hospital leaders who were interviewed reported that more women are now coming to the hospital for delivery, and they are beginning to see the emergency transport systems for pregnant patients begin to function. A number of respondents in the focus group discussions reported that they believed that the number of women dying from pregnancy-related causes had decreased since the Project began. Assessment of Mortality Data One of the unique features of the approach developed by World Relief in the Care Group model is the collection of vital events. Each month, Volunteers collect information on the number of births, the number of deaths in children under five years of age, and the number of deaths among women of reproductive age. This information is passed on to the Promoter who then consolidates the data for his/her geographic area of responsibility and passes it on to the Area Coordinator. Then, the data is consolidated for the entire Project area. At the time of the evaluation, vital events were analyzed for the period from July 2003 until March 2004. These data are not baseline data obviously, since the Project had moved into full-fledged field operations 18 months before the vital events registration system began. Overall, 195 deaths of children aged less than 5 years were recorded, and 2,481 live births. The overall under-5 mortality is, therefore 78.6 deaths per 1,000 live births, representing a 60-64% reduction compared to the under-5 mortality for Rwanda nationally (depending on whether the comparison group is the national rate or the rural rate for Rwanda. These data are presented and discussed more fully in Attachment F.5. Unfortunately, baseline data are not available, and the consolidation of this data is still at this point incomplete. Nevertheless, in concert with the strong improvements in the coverage of services known to reduce child mortality as demonstrated by the mid-term KPC survey and by the strong statements from many participants in the focus group discussions indicating that child mortality has declined, these data provide further supportive evidence supporting the conclusion that a significant impact on child mortality has already taken place within the Project area. Under-Five Mortality 196.0 216.2 78.6 0 50 100 150 200 250 Rwanda (national) Rwanda (rural) Projecf Area (July 2003-March 2004) Deaths per 1,000 live births World Relief/Rwanda Child Survival Mid-Term Evaluation September 2004 26 In our discussions with the MOH District Health Chief, he indicated that he had recorded fewer deaths from malaria that have taken place in MOH facilities (Health Centers and the District Hospital): 68 in 2001, 54 in 2002, 44 in 2003, and only 7 so far during the first 7 months of 2004. Similarly, he has recorded a drop in deaths from diarrhea that were reported to him: 10 in 2001, 9 in 2002, 6 in 2003, and none during the first 7 months of 2004. These data also suggest that mortality in children is dropping since most malaria and diarrhea deaths are among children. But, of course, there is a marked under-reporting of deaths in the MOH vital events registration system. Thus, the remarkable progress demonstrated by the mid-term KPC survey, by the qualitative assessments, and by this review of mortality data all supports the conclusion that the interventions of the Project have been effective. B.1.b.iv. CHANGES IN THE TECHNICAL APPROACHES AND RATIONALE This is an extremely well-designed Project, and represents an expansion of a model of community health work that World Relief first developed in Mozambique and is now being implemented not only in Rwanda but also in Cambodia and Malawi. No changes in the technical approach and rationale have been needed to date. B.1.b.v. SPECIAL OUTCOMES, UNEXPECTED SUCCESSES OR CONSTRAINTS The rapid uptake of interventions once they are developed and once the Project infrastructure is in place is truly impressive. A similar rapid uptake was also achieved in Mozambique, along with a marked reduction in under-five mortality. I am not aware of any other field projects in the world that have been able to achieve such rapid uptake of healthy behaviors in populations of this size with associated major declines in under-five mortality. One constraint that has arisen is the loss of staff to another PVO which has moved into the area. Soon after this PVO moved in, it offered higher salaries than the Project. During the past year or so, 2 Area Coordinators, 6 Promoters and 1 Accountant have left the Project. Approximately half of this turnover has been attributed to the higher salaries offered by the other PVO. There is concern that more will leave as well. This new PVO is beginning AIDS-related activities in the Kibogora Health District. B.1.b.vi. FOLLOW-UP AND NEXT STEPS The Project has only two more years before its completion in September 2006. The challenge during the forthcoming period will be to sustain the high-quality programming already achieved, maintain the high levels of coverage already achieved, and make further progress on those areas that have shown limited progress to date or that still have significant challenges in meeting the end-of-project objectives. In addition, the Project needs to, in my opinion, begin to take steps to seek funding for establishing its Project site in Kibogora as a model teaching and training area where practical operations research can be carried out, so that others can learn about the approach and apply it in other parts of Rwanda. I think the Project should expand its activities once the current cycle of World Relief/Rwanda Child Survival Mid-Term Evaluation September 2004 27 financial support finishes, and I also think that World Relief should play a leadership role in expanding this approach nationwide. B.2. CROSS-CUTTING APPROACHES The Care Group approach, using one Volunteer mother for every 10-15 households and linking them with a group support system of 9 other Volunteers is perhaps the most important cross￾cutting element of the Project. This makes peer-to-peer education and training possible, and it empowers the Volunteers. The second important cross-cutting approach is collaboration with the MOH District and strengthening its capacity. The third important cross-cutting approach is community empowerment. B.2.a. COMMUNITY MOBILIZATION The Care Group approach with Volunteers enables community mobilization to occur quickly and effectively. The engagement of the Project with other community-based organizations, particularly churches, and with the existing political structure of Cells and Sectors also enables community mobilization to occur. Without the engagement of the MOH District activities, such mobilization in many respects would not be effective. Thus, if the community were mobilized to attend EPI outreach sessions but the MOH District staff did not arrive in the community to provide immunizations at the scheduled time and place (as occurs so often around the world), the mobilization would have been self-defeating. The Project's efforts at strengthening the quality of MOH services through training and logistical support have been critical in enabling the mobilized community to obtain services that are effective and that the communities can have trust and confidence in. B.2.a.i. COMMUNITY MOBILIZATION ACTIVITIES The Project has mobilized the community to (1) clean its houses, to engage in better hygiene and sanitation, to consume safe water, to provide optimal treatment for childhood diarrhea, and to refer children when danger signs develop; (2) to attend EPI outreach sessions; (3) to understand the causes, transmission, and prevention of HIV/AIDS and STIs, to provide more support to PLWAs and AIDS orphans, and to obtain VCT; (4) to use ITNs and seek treatment early when signs and symptoms of malaria develop; (5) to provide appropriate infant and child feeding, with a focus on exclusive breastfeeding during the first 6 months of life and frequent feeding with appropriate weaning foods beginning at 6 months of age, to enable Volunteers working with mothers to monitor the growth of children, and to provide supplementary feeding and education in Hearth Programs for those children who have become malnourished; and (6) to promote the utilization of ANC at the Health Centers, the utilization of trained providers to attend the birth, the development of emergency transport plans for each community, and the ability of mothers to be able to execute the plan. All of these activities require strong community collaboration and mobilization. The findings from the mid-term KPC survey and from the qualitative assessments are testimony to the effectiveness of the Project's community mobilization activities. World Relief/Rwanda Child Survival Mid-Term Evaluation September 2004 28 B.2.a.ii. RESPONSE OF THE COMMUNITY The response of the community has been enthusiastic and supportive. As mentioned above, there were many examples mentioned in the focus group discussion regarding how social solidarity has increased as a result of the Project. Volunteers are becoming empowered leaders in the communities and extending their efforts to other areas beyond health. The neighbors that comprise the 10-household units (nyumba kumi) served by the Volunteers are now working more effectively together. The Care Group/Volunteer structure has proven to be an effective way for local leaders to get information out to everyone in the community and to collect information they need in their work. Churches are working better together. People are communicating among themselves better and addressing serious problems affecting their health using new knowledge and training that is empowering them, and the progress they are witnessing as a result is further fueling the empowerment process. B.2.a.iii. HOW ACTIVITIES HAVE BEEN USED TO REFINE PROGRAM IMPLEMENTATION PLANS There has been little in the way of changes that have had to be made since the preparation of the Detailed Implementation Plan. The Project is extremely well-planned and the previous experience in Mozambique using the Care Group approach has been informative. B.2.a.iv. BARRIERS PREVENTING COMMUNITY MEMBERS FROM PARTICIPATING IN PROGRAM ACTIVITIES Volunteers are expected to have exemplary behaviors. However, some of them are so poor that they are not able to do this. For example, they cannot afford to build a latrine or to provide adequate food to their children to prevent them from becoming malnourished. Other families have similar problems. There were many comments in the focus group discussions that some families are so poor that they cannot afford to build latrines or to provide the food necessary to rehabilitate their malnourished children in the Hearth Program. In addition, families have to pay for health services for treatment of illness. Even though the fees are low, many families don't have the money required. So, a significant barrier to the utilization of services exists for the poorest families. Many of these poorest families are also probably furthest away from the Health Center as well. This is one of the reasons that many requests were made during the focus group discussions for the Volunteers to be trained to have the capacity to provide basic medical care, including the treatment of malaria. The Project has plans to begin exploring approaches to increase the availability of food to the poorest households. Approaches under consideration are increasing local food production, expanding family incomes, and implementing food-for-work schemes. The Project also has plans to begin training Volunteers in treatment of malaria, based on an experience in Ethiopia in which child mortality was reduced by half using this approach. B.2.a.v. FACTORS AFFECTING PROJECT'S ABILITY TO MOBILIZE THE COMMUNITY The Kibogora Health District has a long history of relatively well-developed health services. The Kibogora District Hospital, operated by the Free Methodist Church of Rwanda, has been World Relief/Rwanda Child Survival Mid-Term Evaluation September 2004 29 functioning for 40 years now and is considered to be one of the best rural district hospitals in the country. A level of trust between the population and the health system has thus been built up over a long period of time. The Project's association with the Free Methodist Church as well as its partnership with the MOH has given credibility to its work, which has helped to mobilize the community. Trust between the community and the health system is essential for mobilizing the community. At times, during the rainy season, the roads are impassable and therefore it is impossible to reach some of the communities for field work. In addition, communication within the District is always difficult since there is no means of communication other than word of mouth. B.2.b. COMMUNICATION FOR BEHAVIOR CHANGE The Care Group/Volunteer model makes it possible for Volunteers to teach their neighbors and to promote healthy behaviors. The Volunteers are expected to "practice what they preach" as well, further reinforcing the messages they are sharing with their neighbors. This form of communication for behavior change is effective B.2.b.i. APPROPRIATENESS AND EFFECTIVENESS OF APPROACH The progress achieved by the Project in such a short period of time, as demonstrated by the KPC survey, the qualitative assessment, and the mortality impact analysis all provide convincing evidence that the approach to community mobilization is appropriate and effective. B.2.b.ii. TECHNICAL SOUNDNESS OF MESSAGES Due to the short time available in Rwanda and the focus on a strong qualitative evaluation, I did not have time to review the training materials. They are all written in Kinyarwanda, the local language, so it would have been difficult to assess them. However, I understand that all the messages and training materials were developed using local technical expertise, which is considerable (including MOH staff and health staff at the Kibogora District Hospital). The technical staff at World Relief HQ in Baltimore also reviewed the materials. B.2.b.iii. DEGREE TO WHICH MESSAGES ARE TEACHING SKILLS AND CHANGING BEHAVIORS AND SOCIAL NORMS The qualitative assessments, as described above in Section B.1.b.iii as well as in Attachment F. 2, all point to the remarkable degree to which the Care Group/Volunteer approach has been effective in teaching skills and in changing behaviors. The findings from the KPC survey, described in Section B.1.b.iii as well as in Attachment F.1., confirm this impression. It is too early to tell whether social norms have actually changed. B.2.b.iv. MEASUREMENT OF EFFECTS OF BEHAVIOR CHANGE COMMUNICATIONS Considerable evidence exists regarding the effects of the Project's behavior change communications (BCC) strategy, as shown in the mid-term KPC survey results, the qualitative assessments, and the mortality assessment. The Project's unique community- based health World Relief/Rwanda Child Survival Mid-Term Evaluation September 2004 30 information system and its system of local rapid assessments (LRAs), described further in Attachment F.6, make it possible to measure changes in behavior on a quarterly basis and to make rapid adjustments if appropriate progress is not being made. In brief, the Volunteers report on a monthly basis the types of activities in which they have been engaged. B.2.b.v. METHODS FOR GATHERING DATA TO MEASURE BEHAVIOR CHANGE In addition to the KPC surveys, the Project has developed a unique community-based information system. Each month, Volunteers provide to their supervising Promoter information about their work in the 10 households for which they are responsible. The Promoter consolidates this information and passes it on to the Promoter's supervisor, the Area Coordinator. The Area Coordinator then consolidates this information and passes it to the Assistant Project Manager, who consolidates this information for the entire Project area. In addition to this, a Local Rapid Assessment (LRA) is carried out every quarter to assess the competencies and practices of the Promoter, Volunteers and Mothers. At this time, key indicators are measured and progress is assessed. A more complete description of the HIS can be found in Attachment F.6. B.2.b.vi. INNOVATIVE APPROACHES TO BEHAVIOR CHANGE World Relief has implemented a number of innovations related to behavior change. The first innovation is the structure of Care Groups and the role of Volunteers. The use of peers to educate is both empowering for the Volunteers and persuasive for those with whom the Volunteer is working, who then immediately apply what they have learned. The second innovation is the "cascade approach" to training, whereby the senior Project staff members are trained, and they then train the junior Project staff members, who then train the Volunteers. The third innovation is simplicity of the messages, the availability of indicators to monitor progress in the implementation of the messages, and the monitoring and evaluation methods for monitoring progress. Finally, the fourth innovation is the strong presence of supervisory staff in the field to directly observe progress. All supervisory staff members spend most of their time in the field, and their presence is appreciated by those who report to them. B.2.c. CAPACITY BUILDING APPROACH Capacity building is one of the great strengths of the Project. As a result, many of the benefits provided by the Project will be sustained beyond the duration of Project funding, leading to long￾term benefits for World Relief itself, for the services of the MOH in the District, and for the community. B.2.c.i. APPROACH TO STRENGTHENING THE PVO ORGANIZATION This is World Relief's first child survival project in Rwanda. Ms. Melene Kabadege, a strong Rwandan who is a nurse-midwife with graduate training in public health, was selected as Project Manager. She traveled to the US to work with World Relief HQ staff there, and she also traveled to Mozambique to learn about the Care Group/Volunteer approach that World Relief has World Relief/Rwanda Child Survival Mid-Term Evaluation September 2004 31 developed there. The Project has provided an enormous amount of training to staff at all levels, but the senior Project staff indicated that they would still like more training in management and supervision. The Project Manager has participated in evaluations of other NGOs, including IRC and Concern, who also have USAID child survival grants in Rwanda. World Relief has joined with IRC and Concern to start a joint initiative to train community volunteers to provide treatment for malaria in the community. This project is just now in its early stages. World Relief Project staff members, together with a number of staff members from the Kibogora MOH District staff, have traveled to other parts of Rwanda to visit the child survival projects of IRC and Concern and to visit Mutuelles that are functioning in other areas. The Project staff have developed a remarkable capacity during a short period of time to develop training materials "from scratch," to carry out training, to develop a first-class monitoring and evaluation system, and to develop a first-class supportive supervisory system based on field visits and performance assessments. In October 2003, the Project hosted a regional workshop on the Hearth Model, attended by 70 program leaders from Africa and led by Drs. Warren and Gretchen Berggren, who developed this pioneering approach to community-based rehabilitation of malnourished children. At the World Relief HQ level in Baltimore, the child survival staff members there regularly attend USAID workshops related to management and technical support, CORE meetings, and meetings of the Global Health Council. Dr. Anbrasi Edward Raj, Director of Maternal and Child Health for World Relief, is actually a full-time faculty member in the Department of International Health at the Johns Hopkins University Bloomberg School of Public Health with a half-time responsibility to work for World Relief, so this gives her access to current developments in international public health as well. B.2.c.ii. APPROACH TO STRENGTHENING THE LOCAL PARTNER ORGANIZATION The Project has played an important role in strengthening the capacity of the Kibogora MOH District. In recent years, Memisa/CORDAID (a Dutch development agency) has played an important role with facilities strengthening and the African Development Bank has just completed construction of a very large health center for the Gatare Health Area, which serves 45,000 people inside the Project area. The Project was able to help with the final stages of the MOH District Health Office and with the installation and maintenance of office equipment. The Project has made possible water and electricity for the District Health Office as well. All District MOH staff received training concerning the Project's interventions, and the Project has assisted with the transport of vaccines and staff to the EPI outreach sites. The Project holds frequent meetings with MOH staff. Monthly meetings are held to plan activities and to discuss issues in program implementation. The Health Centers invite Project staff to their monthly meetings as well to its meetings with the Health Committee for each Health Center. These Health Committees are composed of community members who supervise the Health Center operations. World Relief/Rwanda Child Survival Mid-Term Evaluation September 2004 32 The Health Center gives the Project staff a schedule of the EPI and ANC sessions for the upcoming month, and the Project disseminates this information to the households via its Volunteer network. The Health Center staff also makes a list of EPI dropouts and shares this with the Project, which then transmits this to the Volunteers. The Project then transports the Health Center staff and vaccine to EPI outreach sites in the communities. By "building a bridge" between the MOH services and the community, the Project has strengthened the capacity of the MOH District staff to collaborate with community people and to be able to work effectively with them. For example, the Volunteers are now assisting the MOH staff in recording information about immunizations and vitamin A distribution that the MOH staff could not carry out because of the volume of services being carried out and the time limitations involved. In addition, the community people mentioned that they are being treated with increased respect by the MOH staff. This stronger collaboration will help to sustain Project impact after funding ends. In this Project, the community is a true partner. Capacity within the community is being strengthened through educating and training community people, working with indigenous private organizations (mainly churches), supporting the formation of new community organizations and efforts (Anti-AIDS Clubs, Volunteer Associations4 , Mutuelles, emergency transport systems), and training and supporting local development officials (particularly the Community Development Committees). B.2.c.iii. APPROACH TO HEALTH FACILITIES STRENGTHENING In June 2004, the Project supported a health facilities assessment of MOH facilities, including the Kibogora District Hospital. A modification of the approach developed by the BASICS Project was employed. This was carried out by Dr. Jean Kagubare, who is the former National Director of Health Services for the Ministry of Health and currently a Ph. D. student at the Johns Hopkins School of Public Health. This assessment is now being reviewed by the MOH District staff, and the Project will assist in addressing major issues that the MOH considers to be priorities. B.2.c.iv. APPROACH TO STRENGTHENING HEALTH WORKER PERFORMANCE The Project is continually assessing the quality of performance of its Promoters and Volunteers based on observations of their work in the field and competency tests. Each quarter, a pay incentive is offered to Promoters for outstanding performance, with a 4-10% salary increase given to those with superior performance. This is based on objective assessments of their performance in the field (as evaluated by Area Coordinators using a check list) and other formal assessments. Area Supervisors have supervisory checklists as do Promoters. In addition, weaker staff members are paired with stronger staff and are provided stronger supervision to improve performance. For instance, the Promoters give more support to the 4 Volunteer Associations are formal groups of volunteers. There is a national policy to encourage the formation of formal associations of all types. Most Volunteer Associations have only 10 members or so and make up a Care Group. World Relief/Rwanda Child Survival Mid-Term Evaluation September 2004 33 weaker Volunteers by helping them with their home visits. I spent a lot of time with all of the staff during my week in Kibogora, and I was impressed with the quality of the staff, their morale, their ability to work together, and their ability to accomplish tasks together. B.2.c.v. APPROACH TO TRAINING The "cascade approach" to training has been mentioned earlier. Training is modular. Interventions are implemented in a sequential fashion, and training workshops are held just before the interventions are to be implemented in the communities. Training materials are developed by the senior Project staff. The Project staff members feel that they have received high-quality training, but they say the need more refresher training to review what they had learned earlier. The curriculum for each module is developed by the Assistant Program Manager with the MOH and hospital staff. Then a workshop is held with the Area Coordinators. A booklet is developed to use as a "textbook," and this is sent to HQ in Baltimore (in English) for review. After translation into Kinyarwanda, the booklets are printed and a workshop is held with the MOH District staff to share the information with them. Following this, a 3-4 day workshop is held for training the Promoters. The curriculum for each intervention has 3-4 lessons, and the Promoters share one lesson every two weeks with each Care Group. Every month, the Promoters meet together to discuss the issues they face in teaching the Volunteers. B.2.d. SUSTAINABILITY STRATEGY Great care has been taken by the Project to ensure that the benefits provided by the Project will be sustained beyond the five years of Project funding. The focus on education and training of paid staff and Volunteers; involvement, education and empowerment of the community; and the use of low-cost approaches all support this strategy. For instance, the use of home-based ORT (using locally available ingredients) is promoted rather than oral rehydration solution (ORS) packets (which are manufactured) to avoid dependence on products which may or may not be available and which must be purchased. In addition, the Project has been very clear and transparent in specifying that funding may not be available after September 2006, so everyone involved is aware of this. The most important sustainability strategy is to ensure that the Volunteers continue to function if no further Project activities continue after the current funding cycle. This was a key point that was raised many times in the qualitative assessment, so further efforts to strengthen the Volunteer Associations and their ability to generate income will be critical during the next two years. Finally, the strengthening of MOH District activities is a part of the sustainability strategy of the Project as well since those benefits will continue long after the Project ends. Again, the training activities provided by the Project for the MOH staff are central to this. Part of the Project's sustainability strategy is to increase the role of the Community Development Committees (CDCs) in the supervision of the Volunteers. Since the CDCs are part of the World Relief/Rwanda Child Survival Mid-Term Evaluation September 2004 34 government's well-developed long-term strategy for national development, this approach seems appropriate and workable. The CDCs have attended many training sessions conducted by the Project and they are well aware that they will be taking over increasing responsibility from the Promoters for the work of the Volunteers. The CDCs were also invited by the Project to visit the Mutuelles in other parts of Rwanda along with Project and MOH staff. They did this, and now the CDCs are ready to provide a leadership role in developing these Mutuelles within the Kibogora Health District. B.2.d.i. PROGRESS IN MEETING SUSTAINABILITY OBJECTIVES The sustainability objectives of the Project focus on the following: · Building up the MOH capacity · Setting up a structure and incentives for Volunteers to continue after the Project ends · Focusing on behavior change. The MOH is now stronger in its capacity to provide ongoing child survival services as a result of the Project's contributions as a result of the training and logistical support that has been provided. The CDCs are now becoming stronger and more involved in the Project and are now on-target to take a supervisory role over the Volunteers should the Project cease at the end of current funding. The Volunteers are becoming stronger and are beginning to function as Volunteer Associations, which should provide ongoing incentives for their continuing functioning. Finally, the focus on behavior change activities should be sustained and, in fact, should grow as a result of an "epidemic" of health-promoting behaviors that is now beginning as a result of the empowerment of Volunteers and households they work with. B.2.d.ii. INITIAL ACTIVITIES IN BEGINNING PHASE OUT The Project has been forthright with its staff and partners that funding is not assured after the current funding ends. Although a strong hope exists that the Project will somehow be able to continue and expand, realism prevails. Perhaps the most important activity so far has been the involvement of the CDCs at the outset of Project functioning and the initiation of a process through which they will take over responsibility for the supervision of Volunteers at the time the Project ends. The focus on community initiative and the empowerment of local villagers is another part of the strategy of being able to phase out while at the same time sustaining the impact of the Project. And, finally, of course, the strategy of strengthening the MOH's activities in the District, which have been considerable so far, is part of the process of the Project's being able to phase out and sustain its benefits in the longer term. B.2.d.iii. APPROACHES TO FINANCIAL SUSTAINABILITY The Project has been designed in such a way that costs are minimal. Furthermore, reliance is placed on partners who do not depend on short-term external funding for their support, such as the MOH, the community, and community-based organizations. Thus, supporting these partners in the short run will lead to their enhanced capacity to continue activities initiated by the Project once Project funding ends in two more years. World Relief/Rwanda Child Survival Mid-Term Evaluation September 2004 35 B.2.d.iv. OPINION OF BENEFICIARY COMMUNITY ABOUT ALTERNATIVE FUNDING AT THE COMPLETION OF THE PROJECT Many comments were made during the focus group discussions regarding the interest of community people in the Project's support of income-generation projects and of the formation of associations who could together implement income-generating activities to make it possible for these groups to continue their health-related activities in the community on a longer-term basis. The establishment of Mutuelles, which are savings groups that members can draw on for health expenses, is well developed in other parts of Rwanda. The Project has provided training about how they function and provided visits to well-established programs in other parts of the country. B.2.d.v. PROGRESS IN FORMAL SUSTAINABILITY DESIGN METHODOLOGY No formal sustainability design methodology is being used. C. PROGRAM MANAGEMENT Overall, the Project is extremely well-managed. On the basis of my limited experience, I can say that World Relief/Rwanda and World Relief International appear to be well-run organizations with competent and committed leadership staff. C.1. PLANNING The DIP is a first-class document. It is well thought out, comprehensive and detailed, and authoritative. The fundamental strategies used are adapted from World Relief's successful project in Mozambique. The Project Manager and Assistant Manager have come to the conclusion that the DIP has too many activities in it. The Project was not able to complete all the activities that had been planned for the Nutrition and Reproductive Health Interventions. Planning of day-to-day activities in the field is a collaborative process which involves all of the Promoters as well as the MOH staff in regular planning meetings. C.2. STAFF TRAINING The Project has made a major investment in staff training. The Project's Director of Training (who is also the Assistant Program Manager) is very dynamic and capable. The training process is an ongoing one, with training for each new technical intervention taking place on a quarterly basis, and assessments of performance using the new skills and knowledge acquired occurs on an ongoing basis through supervisory check lists and formal assessment of knowledge. For the Promoters, a salary bonus is given quarterly to those who are the best performers. C.3. SUPERVISION OF PROGRAM STAFF Supervision is also strong within the Project. The Program Manager herself spends a considerable amount of time in the field observing activities. The Area Managers and the Promoters spend almost all of their time observing their staff in action, helping to address difficult issues, and supporting weaker staff members and weaker Volunteers. The structure of the staff, their numbers, roles and responsibilities seemed to be appropriate to the objectives of World Relief/Rwanda Child Survival Mid-Term Evaluation September 2004 36 the Project. There do not appear to be any major concerns among the staff except for the issue of salary (see below). There has been a minor change in the supervisory structure of the Project. Initially, the geographic areas for which the Area Coordinators were responsible did not coincide with the geographic areas for which the MOH Health Centers were responsible. However, this has been changed so that they now do coincide. One of the Health Center Areas is quite large and therefore it has two Area Coordinators. Many comments were made by lower level staff and Volunteers that indicate that they view field supervision as extremely important because it shows that the Project values the work of the lower levels staff and Volunteers and, from the perspective of the lower level staff and the community, it gives importance to their work. C.4. HUMAN RESOURCES AND STAFF MANAGEMENT I spent 6 days working with the entire field staff in Kibogora. I observed a group of dynamic and highly motivated people who were excited about their work and the progress they had made in a short period of time. Morale, cohesion, and working relationships all appear to be quite positive, and these attributes of the staff constitute an important reason for the remarkable progress made by the Project to date. Many Project staff members feel that they should receive a raise. The salary structure for the Project is based on that followed by all programs of World Relief/Rwanda, and apparently this structure is one that results in lower salaries compared to similar levels of work in other NGOs. The Promoters receive approximately US$ 100 per month, and the Program Manager feels that this should be raised. The Program Manager and the Assistant Program Manager are dynamic, committed, and well￾qualified for their positions. They are both eager to pursue additional training to become even more effective in their work. The Project staff appreciates the style of supervision that has been established. As one staff member said, "If we make a mistake, our supervisor corrects us kindly and respectfully." The Project pays for 80% of the medical expenses of its staff, and also tries to help out with personal and family issues. For example, the Project's vehicle is occasionally loaned out to use when a staff member has a family wedding. There are no vacant positions at present. However, as mentioned elsewhere, 8 employees have resigned. Four resignations were for personal reasons such as illness, but 4 employees left to join another NGO that has recently started working in the same area and these were able to obtain a higher salary. The loss of trained staff is a major concern of the current Project staff members and they worry that they may lose more staff if the salary levels are not increased. C.5. FINANCIAL MANAGEMENT The funds of the Project appear to be managed well. There have been no problems in the Project obtaining the funds to support its budgeted activities. Detailed justification is required for all World Relief/Rwanda Child Survival Mid-Term Evaluation September 2004 37 Project expenses. The budget is under-spent at present, primarily because not all funds budgeted for consultants have been used. C.6. LOGISTICS The Project has one vehicle and 18 motorcycles, so transportation has not been a major issue. There are relatively few materials and supplies needed by the Project, and these have been obtained without any major delays. The main issue, according to the Program Manager, is that sometimes funds are not sufficient to cover all the expenses involved in certain activities mandated in the DIP. This lack of funds is usually in the area of logistical support for Project activities. No major logistical challenges face the Project during its final two years. During discussions with the entire staff, they requested items that currently aren't available, including ORS packets, two-way radios, more scales to weigh children, and more and better audio-visual materials to use in educating Volunteers and communities (especially regarding HIV/AIDS). Generators are needed as well since there is no electricity in the villages. C.7. INFORMATION MANAGEMENT The health information system (HIS) has been described at various other places in this report and in Attachment F.6. A large amount of information is collected - perhaps too much - since it is not fully analyzed and used for program strengthening. However, this information is used to monitor staff performance and progress in field implementation of technical interventions. This information is obtained from: · Monthly information provided by Volunteers, Promoters, and Area Coordinators regarding their activities, · Supervisory checklists of staff performance in the field carried out at least monthly, and, · Local rapid assessments based on a random sample of households every quarter, measuring progress in reaching the objectives of the Project. This information is in fact used (if not fully) to establish bonuses for Promoters, to provide additional support for weaker staff, and to adjust field activities as needed. Monthly statistical reports are generated but these are currently incomplete and not fully analyzed. The data are currently located on Excel spreadsheets, making a comprehensive analysis more difficult than if the data were in a database such as EPI INFO. One of the important features of the Project's HIS is how the information gathered by the Volunteers is passed on directly to the Health Center for the area in which the Volunteer works. Thus, this information becomes incorporated into the Health Center's information, which is then passed on to the MOH District office and into the Moth's HIS. The Project has created many forms for recording information for the HIS. However, the Volunteers themselves do not have any specific forms to fill out. They keep their data in notebooks with no special format. The Project intends to develop forms for the Volunteers to use in the near future. World Relief/Rwanda Child Survival Mid-Term Evaluation September 2004 38 C.8. TECHNICAL AND ADMINISTRATIVE SUPPORT The Project has been able to save some budgeted funds by utilizing qualified local consultants instead of expatriates. This includes support for the baseline KPC survey, the mid-term KPC survey, the health facility assessment survey, and training in the Hearth Program. Dr. Anbrasi Edward Raj, Director of Maternal and Child Health for World Relief based in Baltimore, was in Rwanda after the completion of the DIP to assist the Project with its initial steps in implementation. Several other staff members from the HQ office in Baltimore have also visited the Project to assist with various tasks. For instance, Kathryn Bradbury, Program Assistant in Baltimore, was in Rwanda to facilitate the mid-term evaluation. The Project's Program Manager appears to have a strong working relationship with her supervisor, Rebecca Chandler, who is the National Program Director for World Relief/Rwanda. They are in frequent communication by telephone. There is good telephone communication between Kibogora and Kigali and, in fact, cell phone capability exists within the small town of Kibogora itself, so the Program Coordinator can communicate with her supervisor frequently. The Project has greatly benefited from the support and collaboration of Sheila Etherington, a nurse-midwife with the Free Methodist Church in Kibogora who has been working there for 22 years and now serves as the Administrator of the Kibogora District Hospital. She has a long￾standing professional relationship with the Project's Program Manager, who is also a nurse￾midwife who previously worked in the Kibogora District Hospital. Ms. Etherington is widely known and respected throughout Rwanda and serves as an unpaid Project advisor and source of considerable assistance. Technical assistance that is anticipated during the final two years of the Project includes strengthening the HIS, addressing sustainability issues, and implementation of a Quality Assurance (QA) program in the Health Centers. D. OTHER ISSUES IDENTIFIED BY THE TEAM The Mayor of the Nyamasheke Administrative District has been urging the Project to expand its activities for the remaining 9,000 persons in his area who do not live in the Kibogora Health District. The Project and the national leadership of World Relief are currently considering this. E. CONCLUSIONS AND RECOMMENDATIONS Conclusions The Umucyo Child Survival Project is one of the most, if not the most, promising health projects that I am aware of anywhere in the world. The potential for continued progress in reducing child and maternal mortality and the spread of HIV/AIDS is enormous because of the effectiveness of its community-based activities, its ability to reach all households, and the effectiveness of its monitoring and evaluation activities. Consequently, I consider that this Project should be considered as a model for Rwanda and the rest of the world as well as a model of equity and empowerment for development more generally. Why do I come to this conclusion? The Project is building local capacity that is sustainable, and maximizes existing resources, including the MOH, CDCs, Sectors, Cells, and churches. It is World Relief/Rwanda Child Survival Mid-Term Evaluation September 2004 39 addressing key problems underlying child and maternal mortality such as childhood malnutrition; lack of handwashing, sanitation and hygiene; lack of protection from mosquito vectors of malaria and lack of seeking early treatment for symptoms of malaria, resistance to seeking referral when obstetrical problems arise, and lack of awareness about HIV/AIDS and stigmatization of those who are affected. Furthermore, the methodology is flexible and adaptable to changing needs and circumstances, and it lends itself to operations research and quality improvement activities which can strengthen programming even further. Nutrition is a key component of the Project. Improvements in child nutrition are necessary if a long-term decline in child mortality is to be sustained. The Project's effectiveness is obvious to staff and community members, and progress is demonstrable objectively through the measurement of population-based indicators. The approach is inexpensive, affordable, and sustainable. And, last but not least, other interventions (such as the treatment of childhood pneumonia, treatment of AIDS with ARVs, and detection and treatment of tuberculosis) can be added relatively easily Recommendations Staff Recommendations After 5 days of qualitative assessments of the Project with the community and other partners, the Project staff developed the following list of recommendations for Project strengthening, listed in order of importance (based on number of votes from the entire Project staff): · Support Volunteer Associations (35 votes) · Strengthen the Hearth Program (30 votes) · Increase staff salaries (27 votes) · Lengthen the duration of the program (16 votes) · Train the community in income-generating activities ( 8 votes) · Continue to promote to pregnant mothers delivery in the Health Center (6 votes) · Distribute mosquito nets to all households (4 votes) They also proposed the following specific activities, again listed in order of importance (but this time, based on the number of discussion groups that recommended this). · Provide support to PLWAs (e.g., food, anti-retroviral medications) and to the Anti-AIDS Clubs (training, audio-visual aids) (5 groups recommended this) · Provide higher salaries for the staff and more motorcycles (4 groups mentioned this) · Strengthen the Hearth Program: expand it, provide food for those who can't obtain it, carry out better follow up, develop a Hearth Program for malnourished children who are hospitalized (3 groups recommended this) · Support the Volunteer Associations: provide them with money, domesticated animals, training for the Association (3 groups mentioned this) · Provide more support to the MOH District: more training for staff, more support for transportation (ambulance and motorcycles), medications, laptops, computers, radios (3 groups mentioned this) · Continue to empower communities (3 groups mentioned this] · Provide income-generating activities for the population, including training in agriculture and micro-finance (3 groups mentioned this) · Provide mosquito nets for everyone in the population (3 groups mentioned this) World Relief/Rwanda Child Survival Mid-Term Evaluation September 2004 40 · Lengthen the Project beyond its current end date (2 groups recommended this) · Extend VCT/PMTCT to all Health Centers (2 groups recommended this) · Provide more training to the Community Development Committees (1 group recommended this) · Provide more training to the Umucyo staff (1 group recommended this) · Provide more training to Pastoral Care Groups (1 group recommended this) · Integrate agricultural activities into the nutrition program (1 group recommended this) · Provide visual aids for the trainings on HIV/AIDS and STIs (1 group recommended this) · Rehabilitate one of the Health Centers which is in poor condition (1 group recommended this) · Maintain good relations with all the partners (MOH District, churches, community leaders, and so forth) (1 group mentioned this) · Improve the health information system (1 group mentioned this) Recommendations of the Evaluation Team Leader (Henry Perry) Recommendation 1: The Project should continue beyond two more years and become a center for teaching and training to enable expansion to other districts and eventually scale up nationwide. The rationale for this recommendation is as follows: · The Project is extremely well-designed. · The Project site is an ideal place to implement the Project (there is a great health need there, there is a strong MOH presence there, there is a referral hospital providing good care nearby, and there is a level of trust present between the community and the health system). · World Relief is a strong organization, the Project leadership and staff are strong, and the Project is well-supervised and well-run. · The monitoring and evaluation system is world-class. · The KPC survey results are very impressive (though the impressive declines recorded in malaria and diarrhea prevalence may be overstated and the progress in improving nutrition understated because of their confounding with seasonal effects). · The qualitative assessment results are very impressive. · The vital events registration system has the potential to become a pioneering breakthrough. Recommendation 2: The Project should explore the importance of syphilis in the Project area and consider adding screening pregnant women for syphilis at the time of the first ANC visit and provide treatment at the same visit. The rationale for this recommendation is as follows (and explained more fully in Attachment G. Special Report 1): World Relief/Rwanda Child Survival Mid-Term Evaluation September 2004 41 · Syphilis in pregnant women is associated with a 50-80% rate of adverse outcomes (spontaneous abortion, stillbirths, or early neonatal mortality). · Diagnosis and treatment are inexpensive and simple. · Syphilis rates are known to be high in much of Africa (the prevalence among pregnant women has been reported to be as high as 17.5%). · At the Kibogora District Hospital laboratory, 8% of patients who were tested in 2003 were RPR positive and in 2004, 40% of the tests are positive (based on discussion with head of the laboratory). · There is a high rate of spontaneous abortion in the Kibogora Health District (according to one well-informed local health professional). · Cases of congenital syphilis are being seen at the Kibogora District Hospital. · Control of STIs, particularly those that cause genital ulcers (like syphilis) is one of the most effective ways of reducing the transmission of HIV infection since persons with ulcerative genital lesions who are HIV-positive are much more likely to transmit or receive the infection during sexual intercourse. Recommendation 3: Syphilis testing at the time of ANC should be combined with the option of obtaining Voluntary Counseling and Testing (VCT) for HIV infection The rationale for this recommendation is based on the following: · There is a strong demand within the population (as was expressed in many focus group discussions) for VCT. · Antenatal screening for syphilis and VCT among pregnant women go "hand and hand" in the sense that the same blood specimen can be used to test for both types of infection, and the treatment for those who test positive for syphilis can be given at the same time as counseling regarding the results of the syphilis and HIV testing. Recommendation 4: Make the nutrition program more aggressive and comprehensive (including increasing the availability of food); Emphasis on the nutrition messages should begin during pregnancy, and messages regarding appropriate feeding behaviors should be given more emphasis (including greater attention to actual feeding practices as well). The rationale for this recommendation is based on the following: · Nutrition is the intervention that appears to present the most significant challenge, due to growing socioeconomic constraints within the target community. · Lack of food availability appears to be a serious issue because of landlessness and lack of money with which to buy food. · The Hearth Program appears to be working well and is well-accepted by the community. · Even though rates of exclusive breastfeeding (EBF) during the first 6 months of life appear to be high (77% according to the mid-term KPC survey), adhering to the policy of World Relief/Rwanda Child Survival Mid-Term Evaluation September 2004 42 EBF with no additional water or liquids during the first six months of life is often not followed 100% of the time even among mothers who are reportedly exclusively breastfeeding their child. In addition, once the idea of EBF is accepted, mothers often find it difficult to switch from EBP when their child is 6 months of age and begin frequent feeding with weaning foods. · Emphasis should be given to detecting malnutrition early - when it is not severe and when the child is still very young - to facilitate rehabilitation. Rehabilitation of severely malnourished children is much more difficult than rehabilitation of mildly or moderately malnourished children. · A comprehensive approach to increasing food availability for the poorest families with severely malnourished children (or for those families at high risk of having severely malnourished children) needs to be developed. Analysis of the health information system to determine risk factors for severe malnutrition would help to define program priorities. These families need help with direct food assistance, income (either through direct donation of money or, preferably, through giving the families opportunities for income generation), or opportunities to increase their food production. Terracing through a food￾for-work program appears to have special attraction because it would provide short-term assistance while at the same time increasing the availability of land for agricultural production. Recommendation 5: Increase emphasis on child spacing The rationale for this recommendation is based on the following: · The negative impact of frequent births (less than 36 months apart) on the nutritional status and mortality of both the previous as well as the subsequent child is well known. · The overall utilization of family planning in the Project area is low (approximately 12% of women of reproductive age based on the February 2004 local rapid assessment), and there is a high demand expressed during the qualitative assessment for access to family planning services. · The area is densely populated and land has become scarce during the last decade. · Increased child spacing will also reduce the number of maternal deaths in the Project area by lowering fertility. Recommendation 6: Efforts to assure sustainability of Volunteers is key because they will be needed to sustain progress and to expand into AIDS treatment with ARVs and TB detection and promotion The rationale for this recommendation is the following: · There is general agreement that the most important single component of the Project and its effectiveness are the Volunteers. · The Volunteers have been empowered and are highly motivated to continue their work after the completion of the Project. · Efforts are underway to transfer supervision of Volunteers to CDCs. World Relief/Rwanda Child Survival Mid-Term Evaluation September 2004 43 Recommendation 7: Repeat prevalence assessments of malnutrition, diarrhea and malaria in December of 2004 and again in August and December 2006 The rationale for the recommendation is based on the following: · The mid-term KPC showed a dramatic drop in the prevalence of diarrhea and malaria and a worsening in the level of childhood malnutrition. · The mid-term KPC was carried out during the dry season and before the harvest, when rates of diarrhea and malaria are typically at an ebb and childhood malnutrition is worse. The baseline KPC, on the other hand, was carried out in December, toward the end of the rainy season and after the harvest, when rates of diarrhea and malaria are typically higher and when rates of childhood malnutrition are typically less. · These seasonal effects make it hard to determine the impact of the Project on these illnesses and what trends are actually present. · Therefore, it seems reasonable to carry out a repeat assessment of the incidence of diarrhea and malaria and the prevalence of childhood malnutrition in December of 2004 and then compare these results with August 2004 and December 2002 to make a more informed judgment regarding whether the actual changes observed to date, and also the changes observed at the end of the Project in 2006 can be attributed to the Project's efforts. Recommendation 8: Strengthen the current vital events registration system and use this information for routine monitoring and evaluation; consider carrying out a birth history survey at the end of the Project. The rationale for this recommendation is based on the following (with additional detail located in Attachment F, Special Report 5: · The measurement of vital events, the calculation of under-five mortality rates and maternal mortality rates, and the assessment of a program's mortality impact have almost never been seriously addressed by health programs in developing countries. · The reasons for this are multiple, but include the following: -It is often assumed that such activities require large populations; -These activities are usually left to governments or to special research projects directed by universities; -The efforts at data collection and analysis appear to require a level of sophistication and quality not available to programs; -The attribution of mortality impact requires ideally a comparison group which is rarely available. · World Relief is one of the few PVOs willing to include the collection of vital events in its health information system. · The system developed by World Relief is simple and not cumbersome. · Attention has not yet been given to assuring and demonstrating a high level of quality of this data, and hence its accuracy is open to question. · Strengthening the vital events registration system would require only minimal additional effort. World Relief/Rwanda Child Survival Mid-Term Evaluation September 2004 44 · Strengthening efforts should include the addition of an independent and retrospective collection of vital events for the previous 6 months for a sample of households and then cross-correlating this with the data obtained by the existing system. · Attention should be given to training the staff and Volunteers regarding the differences between stillbirths and live births and to the classification of the outcomes of pregnancies that have been registered. · The determination of the age at child death and the cause of death (possibly using a computerized verbal autopsy protocol, at least for a representative sample of deaths) would give marked additional precision and quality to the data collected. · The determination of the age of death for women of reproductive age and the cause of death, for a least a representative sample of deaths, would give marked additional precision and quality to the data collected. · By carrying out a birth history questionnaire at the end of the Project, baseline under-5 mortality for the 5-year period before the Project began can be obtained, and the data from the ongoing vital events registration system can be validated. · Such refinements would give this World Relief Child Survival Project an opportunity to make a breakthrough in leading child survival programs and maternal health programs into a new paradigm of monitoring mortality impact and refining program activities on the basis of monitoring mortality impact. F. RESULTS HIGHLIGHT The Care Group approach developed by World Relief has proven itself in both Mozambique and now in Rwanda to be a highly effective method for changing household behaviors that are related to improving child survival and also for increasing health service utilization. Perhaps the most dramatic example of the effectiveness of the approach is shown in the figure below for the malaria intervention. Malaria is the leading cause of death among children aged less than 5 years of age. World Relief/Rwanda Child Survival Mid-Term Evaluation September 2004 45 Progress in Implementation of Malaria Intervention, Rwanda/World Relief Child Survival Project, 2001-2004 3.0 3.5 3.7 66.0 64.5 31.2 40.0 40.0 50.0 0 10 20 30 40 50 60 70 80 90 100 Children who slept under treated bednet Pregnant women who slept under treated bednet Children treated within 24 hrs of fever Percentage Baseline Mid-Term EOP* Goal *End of project Changes in Deaths From Malaria, Rwanda/World Relief Child Survival Project, 2001-2004 0 10 20 30 40 50 60 70 80 2001 2002 2003 2004* No. deaths among those 5 yrs. and older No. deaths among children *2004 data pro-rated based on reports through August. Data obtained from Ministry of Health reports. World Relief/Rwanda Child Survival Mid-Term Evaluation September 2004 46 The percentage of children and pregnant women sleeping under insecticide-treated bed nets increased by 20-fold, and the percentage of children treated for malaria within 24 hours of developing fever increased by 10-fold. Marked improvements were also observed in hand washing, use of oral rehydration fluid for diarrhea, obtaining immunizations, knowledge of symptoms of sexually transmitted diseases and methods of preventing AIDS transmission, appropriate infant feeding, and in the development of emergency transport plans for pregnant women. After only two years of functioning, the under￾five mortality in the Project area is only one-third that for the rest of Rwanda. II. THE ACTION PLAN In response to findings and lessons learned from this MTE exercise, and to recommendations made by the external evaluator, the Umucyo CSP staff has developed the following list of priorities in order to strengthen the program’s impact within the remaining 2 years. · Initiate additional Hearth program sessions, beyond the two originally scheduled. · Involve Community Development Committees in Hearth and train them to assist Umucyo in raising awareness regarding nutrition for mothers and young children. · Work with Health District to improve recording: -Care Groups to assist EPI staff to ensure recording during EPI sessions. -Collaborate with the KHD to improve recording of maternal TT during ANC visits. -Continue to pursue discussions with the KHD to involve Umucyo volunteers in Vitamin A distribution. · Pursue possible partners or strategy to integrate agricultural activities within the Nutrition Program. · Advocate for KHD to add screening pregnant women for syphilis at first ANC visit and treatment at HC. · Increase emphasis on Child Spacing during care group discussions. · Incorporate more teaching aids and visual teaching materials for HIV/AIDS training. · Repeat prevalence assessments of malnutrition, diarrhea and malaria in August and December 2004, 2005, 2006. · Continue to give initiate forums for discussion and feedback on M&E results to volunteers, local authorities and HC staff. · Provide more support, training and recognition to volunteers associations. · Provide assistance to volunteer income generating activities, through seeking training, access to credit, etc. World Relief/Rwanda Child Survival Mid-Term Evaluation September 2004 47 The following is the projected timeline for implementing these priorities. New priority activities are highlighted. Year 4 –5 Action Plan Year 4 Year 5 Planned Activities Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 Interventions a. Review Nutrition & 2nd Hearth Cycle Staff training on Hearth methodology X Extend Hearth Program X X Hearth Program Follow-up X X X X X X X X Train Community Development Committees to assist with Community Nutrition Program. X X X X X X X X Monthly Growth Monitoring X X X X X X X X Work with KHD to improve recording of Immunizations, Vit A, etc. X X X X X X X X Integrate agriculture activities with Nutrition Program X X X X X X X X b. Review Maternal Care (RH) Intervention TBA training X Strength existing TBA Associations and initiate new TBA Associations X X X X X X X Initiate Hamoc Associations X X X X X X X TBA Association Follow-up X X X X X X X Help KHD to incorporate syphilis screening to ANC visits and increase HC capacity to provide treatment. X X X X X X Increase emphasis on Child Spacing X X X X X X c. Review Diarrhea and Hygiene X X Continue to train families in diarrhea case management X X X X X X Train Primary school on D/H X X X X X X e. Review EPI Train Primary school on EPI X X X X X X Continue to support EPI Outreach sessions X X X X X X f. Review HIV/AIDS Strengthen CDLS X X X X X Support PLWA by creating Care Givers within the churches X X X X X Support Anti AIDS Clubs X X X X X Train Primary schools on HIV/AIDS X X X X X Develop more visual aids to assist with HIV/AIDS training. X X X X X g. Review Malaria X X Support HBM Program X X X X X X X X World Relief/Rwanda Child Survival Mid-Term Evaluation September 2004 48 Continue to make bednets available X X X X X X X X Maintain BN retreatment system in the community X X X X X X X X Monitoring and Evaluation Improve HIS @ MIS X X X X X X X X Bimonthly meeting to give feedback from field X X X X X X X X Monthly monitoring visits to field X X X X X X X X Monthly program narrative and financial report to WRHQ X X X X X X X X LRA covering phased-in interventions X X X X X X X X Annual Report X X Repeat Prevalence assessments of malnutrition, diarrhea and malaria in August and December 2004, 2005, 2006 X X X Continue to facilitate feedback exchange for M&E results between volunteers, local authorities and HC staff. X X X X X X X X Sustainability activities Maintain good partnership with KHD, Local Authorities, Churches, local NGOs. X X X X X X X X Provide more training and support to volunteers associations X X X X X X X X Provide more assistance with volunteers income generating activities X X X X X X X X Involve the HC staff, local authorities, churches in volunteers health activities X X X X X X X X Continue to involve volunteers in HC activities X X X X X X X X Support implementation and KHD staff Association which focus on community health activities initiated by the Prefect of Cyangugu Province. X X X X X X X World Relief/Rwanda Child Survival Mid-Term Evaluation September 2004 49 III. ATTACHMENTS A. BASELINE INFORMATION FROM THE DIP There are no significant changes to baseline information collection at the time of the DIP. B. EVALUATION TEAM MEMBERS Kathryn Bradbury, BA, Program Assistant, World Relief, Baltimore, MD, USA Ndizigiye Dieudonne, Diplomat in Public Health, Chief of Kibogora Health District, Ministry of Health, Government of Rwanda, Kibogora, Rwanda Melene Kabadege, Diplomat in Public Health, Project Manager, Umucyo Child Survival Program, World Relief, Kibogora, Rwanda Maurice Kwizer, Assistant Project Manager, Umucyo Child Survival Program, World Relief, Kibogora, Rwanda Henry B. Perry, MD, PhD, MPH, Carl Taylor Professor for Equity and Empowerment, Future Generations, Franklin, WV, USA (Evaluation Team Leader) Anbrasi Edward Raj, PhD, MPH, Director of Maternal and Child Health, World Relief, Baltimore, MD, USA C. EVALUATION ASSESSMENT METHODOLOGY The methodology for the mid-term KPC evaluation is described in that report in Attachment F.7, and the methodology for the qualitative assessment is described in Attachment F.3. I made special visits to one Care Group and to one group of mothers with their malnourished children who were holding a Hearth session. I also visited the Kibogora District Hospital and interviewed Key Informants, including the MOH District Chief and the World Relief/Rwanda Program Director. Extended discussions were held with the Project staff, with the Project Manager and Assistant Manager, and with the International Director of Maternal and Child Health. Timetable of Evaluation 2-6 August 2004 Date collection for rapid knowledge, practice and coverage (KPC) survey 31 August 2004 Completion of analysis and final report of KPC survey Arrival of Dr. Perry in Rwanda 1 September 2004 Meeting in Kigali with representatives of World Relief, Ministry of Health, USAID, UNICEF, and PSI. Those attending included: David Brose, Country Director, World Relief/Rwanda Rebecca Chandler, National Program Director, World Relief/Rwanda Melene Kabadege, Program Manager, Umucyo Child Survival Project, WR/Rwanda Anbrasi Edward Raj, Director, Maternal and Child Health, World Relief International Kathryn Bradbury, Program Assistant, World Relief International World Relief/Rwanda Child Survival Mid-Term Evaluation September 2004 50 Claude Rwagacondo Emile, Coordinator, Integrated National Malaria Control Program, Ministry of Health Dr. Mark Komyunbolizi, Chief, Immunization and IMCI, Ministry of Health Christophe Tocco, Program Officer, USAID Regan Whitworth, Senior Technical Officer, USAID Franci Kimomuko, UNICEF Yves Chaka, PSI Christophe Habiyanda, Assistant Program Manager, Concern 1 September 2004 Travel to Kibogora 2 September 2004 Meeting with program staff in Kibogora (Project leaders with Area Coordinators and Promoters) 3-4 September 2004 Interviews with key informants (Chief of Kibogora Health District, Ministry of Health staff in Kibogora District, and staff of the Kibogora Free Methodist District Hospital) and focus group discussions with key groups (selected representatives of Community Development Committees, Anti-AIDS Clubs, Sector and Cell Coordinators, Volunteers, Teachers, Pastors and Pastoral Care Volunteers, Mothers or Caretakers, and Health Center Staff). These were carried out by evaluation team and program staff. 6 September 2004 Review of findings from interviews and focus group discussions 7 September 2004 Consolidation of evaluation findings 8 September 2004 Meetings with Project leadership Return to Kigali 9 September 2004 Dissemination seminar with representatives of the Ministry of Health, USAID, WHO, PSI, IRC, and Concern 11 September 2004 Departure of Dr. Perry from Rwanda 30 September 2004 Submission of evaluation report to World Relief Headquarters World Relief/Rwanda Child Survival Mid-Term Evaluation September 2004 51 D. LIST OF PERSONS CONTACTED AND/OR INTERVIEWED Dan Brose, Country Director, World Relief, Rwanda Rebecca Chandler, Director of Programs, World Relief, Rwanda Care Group members, Kibogora Health District Community Development Committee members, Kibogora Health District Hearth Program members (mothers of malnourished children) Gatare Health Center staff, Kibogora Health District Ndizigiye Dieudonne, Chief of Kibogora Health District Pastoral Care Group members, Kibogora Health District Sheila Etherington, Administrator, Kibogora District Hospital Umucyo Child Survival Project staff members, Kibogora Health District E. DISKETTE OR CD WITH ELECTRONIC COPY OF REPORT Included in submission package PVO NAME: UMUCYO CSP DATE of INFO September 2004 LOCATION (COUNTRY): RWANDA PROJECT AREA Kibogora Health District, Cyangugu Province 52 F. PROJECT DATA SHEET FORM - UPDATED VERSION KPC2000+ RAPID CATCH INDICATORS INDICATOR DEFINITION NUMERATOR DENOMINATOR ESTIMATE CONFIDENCE LIMITS 1. Percentage of children age 0–23 months who are underweight (-2 SD from the median weight-for-age, according to the WHO/NCHS reference population) Numerator No. of children age 0–23 months whose weight (Rapid CATCH Question 7) is –2 SD from the median weight of the WHO/NCHS reference population for their age. Denominator Number of children age 0–23 months in the survey who were weighed (response=1 for Rapid CATCH Question 6) 85 300 28.3% 21.1% - 35.5% 2. Percentage of children age 0–23 months who were born at least 24 months after the previous surviving child Numerator Number of children age 0–23 months whose date of birth is at least 24 months after the previous sibling’s date of birth (Rapid CATCH Question 5) Denominator Number of children age 0–23 months in the survey who have an older sibling 137 178 77.0% 68.2% - 85.8% 3. Percentage of children age 0–23 months whose births were attended by skilled health personnel Numerator Number of children age 0–23 months with responses =A (‘doctor’), B (‘nurse/midwife’), or C (‘auxiliary midwife’) for Rapid CATCH Question 10 Denominator Number of children age 0–23 months in the survey 105 300 35.0% 27.4% - 42.6% PVO NAME: UMUCYO CSP DATE of INFO September 2004 LOCATION (COUNTRY): RWANDA PROJECT AREA Kibogora Health District, Cyangugu Province 53 KPC2000+ RAPID CATCH INDICATORS (continued) INDICATOR DEFINITION NUMERATOR DENOMINATOR ESTIMATE CONFIDENCE LIMITS 4. Percentage of mothers of children age 0–23 months who received at least two tetanus toxoid injections before the birth of their youngest child Numerator Number of mothers of children age 0–23 months with responses=2 (‘twice’) or 3 (‘more than two times’) for Rapid CATCH Question 9 Denominator Number of mothers of children age 0–23 months in the survey 163 300 54.3% 46.4% - 62.3% 5. Percentage of infants age 0– 5 months who were exclusively breastfed in the last 24 hours Numerator Number of infants age 0–5 months with only response=A (‘breastmilk’) for Rapid CATCH Question 13 Denominator Number of infants age 0–5 months in the survey 65 78 83.3% 71.6% - 95.0% 6. Percentage of infants age 6– 23 months receiving breastmilk and complementary foods Numerator Number of infants age 6–23 months with responses= A (‘breastmilk’) and D (‘mashed, pureed, solid, or semi-solid foods’) for Rapid CATCH Question 13 Denominator Number of infants age 6–9 months in the survey 112 222 50.4% 41.2% - 59.7% PVO NAME: UMUCYO CSP DATE of INFO September 2004 LOCATION (COUNTRY): RWANDA PROJECT AREA Kibogora Health District, Cyangugu Province 54 7. Percentage of children age 9–23 months who received a measles vaccine Numerator Number of children age 9–23 months with response=1 (‘yes’) for Rapid CATCH Question 16 Denominator Number of children age 12–23 months in the survey 156 179 87.1% 80.4% - 93.9% 8. THIS INDICATOR IS REQUESTED OF PROJECTS WORKING IN MALARIA ENDEMIC AREAS ONLY Percentage of children age 0–23 months who slept under an insecticide-treated bednet the previous night (in malaria￾risk areas only) Numerator Number of children age 0–23 months with ‘child’ (response=A) mentioned among responses to Rapid CATCH Question 18 AND response=1 (‘yes’) for Rapid CATCH Question 19 Denominator Number of children age 0–23 months in the survey 198 300 66.0% 58.4% - 73.6% KPC2000+ RAPID CATCH INDICATORS (continued) INDICATOR DEFINITION NUMERATOR DENOMINATOR ESTIMATE CONFIDENCE LIMITS 9. Percentage of mothers who know at least two signs of childhood illness that indicate the need for treatment Numerator Number of mothers of children age 0–23 months who report at least two of the signs listed in B through H of Rapid CATCHQuestion 20 Denominator Number of mothers of children age 0–23 months in the survey 294 300 98.0% 96.0% - 100.0% 10. Percentage of sick children age 0–23 months who received increased feeding during an illness in the past two weeks Numerator Number of children age 0–23 months with response=3 (‘more than usual’) for Rapid CATCH Question 22 AND response=2 (‘same amount’) or 3 (‘more than usual’) for Rapid CATCH Question 23 Denominator Number of children surveyed who were reportedly sick in the past two 97 144 67.4 56.6% - 78.1% PVO NAME: UMUCYO CSP DATE of INFO September 2004 LOCATION (COUNTRY): RWANDA PROJECT AREA Kibogora Health District, Cyangugu Province 55 INDICATOR DEFINITION NUMERATOR DENOMINATOR ESTIMATE CONFIDENCE LIMITS 13. Percentage of children age 12–23 months who are fully vaccinated (against the five vaccine￾preventable diseases) before the first birthday Numerator Number of children age 12–23 months who received Polio3 (OPV3), DPT3, and measles vaccines before the first birthday, according to the child’s vaccination card (as documented in Rapid CATCH Question 15) Denominator Number of children age 12–23 months in the survey who have a vaccination card that was seen by the interviewer (response=1 ‘yes, seen by interviewer’ for Rapid CATCH Question 14) 153 179 85.5% 78.1% - 92.8% If you defined an indicator differently in your KPC survey report than what is recommended by the Rapid CATCH, please describe below: 4. The percentage of mothers who received at last two doses of TT is measured through verbal reporting, as card records are often not maintained correctly. Consistent recording is an issue that the project is working with the health district staff to improve. 10. Percentage of mothers who know at least three signs of childhood illness that indicate the need for treatment. Objective changed from 2 signs to 3 after the baseline survey. 56 ATTACHMENT G. SPECIAL REPORTS SPECIAL REPORT 1. THE ROLE OF SYPHILIS SCREENING AND TREATMENT FOR PREGNANT WOMEN IN CHILD SURVIVAL PROGRAMS IN DEVELOPING COUNTRIES Introduction To date, many programs whose goal is to improve child survival have given limited emphasis on the detection and treatment of syphilis in pregnant women. The purpose of this brief communication is to give a justification for including this activity as part of the standard "repertoire" of child survival interventions in developing countries. Gloyd et al. (2001) state that "Although syphilis is an important cause of morbidity and mortality, and antenatal screening is relatively feasible and effective, this intervention has not been promoted to the extent of other, less efficacious interventions." McDermott et al. (1993) state that "… syphilis remains a largely ignored maternal and perinatal health problem in most sub-Saharan African countries." Globally, more than 500,000 fetal deaths are now attributable to syphilis, a level of infection that rivals the maternal-to-child transmission of HIV infection, but receives much less attention even though congenital syphilis is much easier and less costly to prevent (Schmid, 2004). The World Health Organization estimates that congenital syphilis is responsible for 460,000 abortions or stillbirths, 270,000 cases of congenital syphilis, and the birth of 270,000 low-birth-weight or premature babies (Finelli et al., 1998). Even though the burden of disease associated with congenital syphilis easily exceeds that for other neonatal infections, including HIV and neonatal tetanus, the detection and treatment of maternal syphilis does not receive the high priority status that it deserves (Saloojee et al., 2003). Characteristics of the Disease Syphilis is caused by a slowly replicating spirochete, Treponema pallidum, which is transmitted through sexual contact. The organism is highly sensitive to penicillin. To date, no resistance to the antibiotic has been observed. The disease is characterized by distinct clinical phases and a long latency period between the initial clinical manifestations (primarily a genital ulcer - chancre - and secondary syphilis) and the severe systemic complications of tertiary syphilis, which occur 5-20 years after initial infection. Those infected with syphilis rarely seek treatment. Most cases are asymptomatic or, if ulcers develop, women rarely seek treatment (Ronald and Plourde, 1998). Most pregnant women with syphilis are asymptomatic and can be identified only through serological screening (Peeling and Ye, 2004). Pregnant women with untreated syphilis who acquired the infection during the previous two years transmit the infection to the fetus in almost all cases. Fetal infection is the result of hematogenous spread from an infected mother, though transmission can occur at delivery from direct contact with infected genital lesions of the mother. Early syphilis is characterized by spirochetemia (presence of spirochetes in the bloodstream), so women who are recently infected have a nearly 100% transmission to the fetus. 57 With congenital syphilis, signs and symptoms often do not occur until 3 weeks after birth, but some are present at birth (such as hepatosplenomegaly, bloated abdomen, cutaneous lesions, and nasal discharge turning into purulent rhinitis). Other findings include generalized lymphadenopathy, and hepatitis. Skeletal findings include osteochondritis, periostitis, osteomyelitis, and osteitis. Nervous system involvement can occur along with nephrotic syndrome, acute glomerulonephritis, and ocular abnormalities (Boot et al., 1992). The frequency of clinical findings in cases of congenital syphilis is as follows: · prematurity and low-birth-weight (10-40% of cases) · hepatomegaly with or without splenomegaly (33-100% of cases) · a blistering skin rash (40% of cases) · bone changes on x-ray (75-100% of cases) (Saloojee et al., 2004). Congenital syphilis has been observed to cause mental retardation as well as other neurological problems (Over and Piot, 1993). Patients with syphilis who have genital ulcer disease are at much great risk of transmitting HIV infection with sexual intercourse than those without genital ulcer disease (Gelmon and Piot, 1996). Recommended Treatment of Pregnant Women The Centers for Disease Control and Prevention and the World Health Organization recommend a single dose of 2.4 million units of benzathine penicillin G for recently acquired syphilis and 7.2 million units over three weeks if the duration of syphilis is at least a year (CDC, 2002; WHO, 2001). The woman's sexual partner should also be treated to prevent reinfection. Ideally, women should be tested twice, once in early pregnancy and once during the final stages of the third trimester (Saloojee et al., 2004). What Is the Prevalence of Syphilis Infection in Pregnant Women in Developing Countries? Evidence from Africa accumulated in 1990 or before indicates that the prevalence of a positive syphilis test among pregnant women varied from 0.9% to 17.5%, with the median prevalence being 7.5% (Over and Piot, 1993). One study reported a prevalence in Rwanda of 4% in the late 1980s (Senyonyi, 1987). A more recent report estimates the prevalence to be 4-15% among pregnant women in Africa (Gloyd et al., 2001). One report has noted a decline in maternal syphilis in Nairobi, from 7.3% in 1995 to 3.8% in 1997 (Temmerman et al., 1999). Prevalence rates at present in Asia and Eastern Europe are estimated to be 8% and 14.5%, respectively (Gloyd et al., 2001). A recent study from rural Haiti reported a prevalence of 5.5% (Fitzgerald et al., 1998), and Jamaica has a reported prevalence of 2.2% (Smikle et al., 1990). What Are the Risks of Adverse Fetal and Infant Outcomes for Pregnant Women who Have Syphilis? Risk of Spontaneous Abortion and Intrauterine Fetal Death 58 In a study in Kenya, the risk of spontaneous abortion was three times greater among pregnant women with a positive test for syphilis compared to controls (Temmerman et al, 1992). In Zambia, this risk is 5.0 times greater (Hira et al., 1990). In Ethiopia, 5% of all fetuses are estimated to be lost through syphilis-induced abortion (Schulz et al., 1990). The risk of intrauterine fetal death in women who are sero-positive is increased 4.8 times (Temmerman et al., 1990). Risk of Stillbirth In an Zambian demonstration project to detect and treat syphilis during pregnancy (Hira et al., 1990), sero-positive mothers had a 3.6 times increased risk of stillbirth, 2.6 times increased risk of prematurity, and 7.8 times increased risk of low-birth-weight. In Zambia, 24% of stillbirths have been attributed to congenital syphilis (Schulz et al., 1987). There was a 20-25% rate of stillbirth among sero-positive mothers in Malawi group (McDermott et al., 1993). In areas of high prevalence, syphilis can be the cause of up to half of stillbirths (Goldenberg and Thompson, 2003). Risk of Perinatal Death In a study from Malawi, pregnant women with a positive serology had a risk of perinatal death that was 3.4 times greater than those with a negative serology, controlling for other risk factors (McDermott et al., 1996). In Zambia, 30% of perinatal mortality has been attributed to congenital syphilis (Schulz et al., 1987). In rural Malawi, with a prevalence of syphilis among pregnant women at only 3.5%, syphilis was judged to be the cause of 20-25% of perinatal deaths (McDermott et al., 1993 and 1996). In Zambia and in Zimbabwe, syphilis has been estimated to cause 20-30% of perinatal deaths (Hira, 1986; Aiken, 1992). Risk of Prematurity and Low-Birth-Weight In the same Zambian demonstration project (Hira et al., 1990), sero-positive mothers had a 2.6 times increased risk of prematurity and a 7.8 times increased risk of low-birth-weight. Risk of Neonatal Death Approximately 10-12% of infants born to mothers with a positive serology will die during the neonatal period if untreated (Over and Piot, 1993). Risk of Post-Neonatal Death Increased mortality during the post-neonatal period has been observed among infants born to sero-positive mothers (McDermott et al., 1993). Risk of Congenital Syphilis and Its Sequelae Congenital syphilis, which affects 25-75% of exposed infants, can result in severe physical and mental handicaps if the child survives (Over and Piot, 1993). Overall Risk of an Adverse Outcome In the same Zambian demonstration project (Hira et al., 1990), the overall increased risk of an adverse outcome due to syphilis was 8.3. Overall, active syphilis in pregnancy causes adverse outcomes in 50-80% of pregnancies surviving past 12 weeks gestation, primarily as spontaneous 59 abortions in the second and early third trimester, stillbirths, and congenital syphilis (Gloyd et al., 2001). Approximately 50% of pregnancies among mothers with primary or secondary syphilis result in abortion, stillbirth, perinatal death, or premature delivery. Irreversible sequelae and death due to syphilis occur in 50-75% of the infants who are born with infection (Over and Piot, 1993). Even after treatment, women who have syphilis during pregnancy still have a 2.5-fold increased risk of adverse outcomes compared to uninfected women (Lubiganon P et al., 2002). Current Status of Detection and Treatment of Maternal Syphilis A recent review of countries in sub-Saharan Africa estimates that 1.6 million pregnant women with syphilis are undetected each year, including 1.0 million who obtain antenatal care even though 73% of the population in the region live in countries in which antenatal screening for syphilis is part of the national health care policy (Gloyd et al., 2001). Practical Issues in Screening and Treatment Traditionally, screening tests have not been available at primary health care centers because the reagents require cold storage and a rotator is needed which requires electricity. However, new tests have become available using reagents that are stable at room temperature, battery-powered rotators are now available, and on-site confirmatory tests are also available (Peeling and Ye, 2004). Biological false positive tests for syphilis do occur as a result of cross-reactivity with yaws infection and rheumatologic diseases such as systemic lupus erythematosus. With the VDRL test, the percentage of false positives was 0.6% in one study in Jamaica (Smikle et al., 1990). A recent review reports that the sensitivity of the RPR test is 86-100% and the specificity is 93- 98% (Peeling and Ye, 2004). False positive tests can be obtained from as many as 28% of those testing positive, and confirmatory tests are available only in reference laboratories (Peeling and Ye, 2004). Tests are not generally available at low cost that can distinguish between current active infection and previously treated infections, but this is an area of intense and active investigation (Peeling and Ye, 2004). Cost-Effectiveness of the Screening and Treatment Intervention Cost of Screening The current cost of screening is US$ 0.20 - 0.60 per test (Gloyd et al., 2001; Hira et al., 1990; Peeling and Ye, 2004). Cost of Treatment The cost of treatment is approximately US$ 1.00 per dose, or a total of $3.00 if three doses are administered. In countries with a 1% prevalence, it is estimated that the cost of a screening and treatment program is US$ 0.42 per woman per pregnancy (Finelli et al., 1998). Over and Piot (1993) estimated the cost-effectiveness of syphilis treatment (in 1990 dollars) and came up with a total estimated cost, assuming a population prevalence of 5%, of $38 per case treated. For a population prevalence of 25%, the total cost would be $9 per case treated. 60 Cost to Prevent an Adverse Outcome of the Pregnancy In most of sub-Saharan Africa, US$ 12.00 would be necessary to prevent an adverse outcome associated with syphilis in pregnancy, taking in account the costs of screening and drug treatment (Gloyd et al., 2001). In Zambia, the cost of averting an adverse outcome was also US$ 12.00 (Hira et al., 1990). In countries with a 1% prevalence, it is estimated that the cost for each syphilis-associated death or adverse pregnancy outcome that is averted is US$ 70.00 (Finelli et al., 1998) comparing favorably to a cost of US$ 41.00 for each measles death averted through immunization (Gloyd et al., 2001). Over and Piot (1993) estimate that 396.3 disability-adjusted life years (DALYS) would be saved for each effectively treated case. Thus the cost per DALY saved is $ 0.02- $0.47, depending on the population prevalence (assuming that it is at least 5% and an HIV epidemic is also present). In one demonstration project (Hira et al., 1990), attempts to strengthen the screening and proper treatment of sero-positive pregnant women and their partners reduced adverse outcomes attributable to syphilis by almost two-thirds. Overall Cost-Effectiveness In one setting in rural Haiti, the decentralization of testing using a solar-generated electricity stored in 6-volt batteries and inverted to alternating current for a centrifuge and rotator, and reagents were stored in a propane-powered refrigerator. Health Center staff was trained to perform the test. The rate of congenital syphilis before the intervention was estimated to be 550 per 100,000 live births, and after the intervention it fell by 75% (Fitzgerald et al., 2003). Several studies have suggested that maternal syphilis control programs are among the most cost￾effective health interventions available, comparing favorably with childhood immunization (Behets et al., 1997; Schultz et al., 1992; Stray-Pederson, 1983). A recent review of the scientific basis for antenatal care noted that "serological screening and treatment of syphilis is inexpensive and cost-effective" (Villar and Bergsio, 1997). Program Implications In order to prevent congenital syphilis infection from maternal transmission, the following steps must occur (Schmid, 2004): · Women must access antenatal care, and access it early, · Antenatal care programs must provide syphilis testing, · Women must receive test results promptly, · Notification must result in appropriate treatment, and, · Women must remain uninfected during pregnancy. The opportunity exists to provide counseling, screening and surveillance for both HIV and syphilis together, and these should not be missed (Mullick et al., 2004). Thus, in the presence of an HIV epidemic, programs should link screening for maternal syphilis with voluntary counseling and testing (VCT) for HIV infection. 61 Conclusions The burden of disease associated with maternal syphilis has not been sufficiently appreciated by those involved in child survival program implementation, nor has the low cost and effectiveness of screening and treatment. With the expansion of VCT now in countries affected by HIV/AIDS, the opportunity exists to link this activity with screening and treatment of maternal syphilis, thereby having an impact on slowing the transmission of HIV infection and also on reducing the major burden of disease caused by syphilis. References Behets FM, Brathwaite A, Bennett L, et al. 1997. Decentralization of syphilis screening for improved case management in Jamaican public health clinics. American Journal of Public Health 87:1019-21. Boot JM, Oranje AP, de Groot R, et al. 1992. Congenital syphilis. International Journal of Sexually Transmitted Diseases and AIDS. 3:161-7. Centers for Disease Control and Prevention. 2002. Sexually transmitted disease treatment guidelines 2002. Morbidity and Mortality Weekly Report 51:1-78. Finelli L, Berman SM, Koumans EH, Levine WC. Congenital syphilis. Bulletin of the World Health Organization 76 Suppl 2:126-8. Fitzgerald DW, Behet FM, Lucet C, Roberfroid D. 1998. Prevalence, burden, and control of syphilis in Haiti's rural Artibonite region. International Journal of Infectious Diseases 2:127- 131. Fitzgerald DW, Behets F, Preval J, et al. 2003. Decreased congenital syphilis incidence in Haiti's rural Artibonite region following decentralized prenatal screening. American Journal of Public Health 93:444-6. Gelmon LJ, Piot P. 1996. The interactions between HIV and other sexually transmitted infections. In: Mann J, Tarantola D (eds). AIDS in the World II. New York: Oxford University Press, p. 99. Gloyd S, Chai S, Mercer MA. 2001. Antenatal syphilis in sub-Saharan Africa: missed opportunities for mortality reduction. Health Policy and Planning 16:29-34. Goldenberg RL, Thompson C. 2003. The infectious origins of stillbirth. American Journal of Obstetrics and Gynecology 189:861-73. 62 Hira SK. 1986. Sexually transmitted diseases, a menance to mothers and children. World Health Forum 7:243-47. Hira SK, Bhat GJ, Chikamata DM, et al., 1990. Syphilis intervention in pregnancy: Zambian demonstration project. Genitourinary Medicine 66:159-64. Lubiganon P, Piaggio G, Villar J, et al. 2002. The epidemiology of syphilis in pregnancy. International Journal of STDs and AIDS 13:486-94. McDermoot J, Steketee R, Wirima J. 1993. Syphilis-associated perinatal and infant mortality in rural Malawi. Bulletin of the World Health Organization 71:773-780. McDermoot J, Steketee R, Wirima J. 1996. Perinatal mortality in rural Malawi. Bulletin of the World Health Organization 74:165-71. Mullick S, Broutet N, Ye H, et al. Controlling congenital syphilis in the era of HIV/AIDS (Commentary). Bulletin of the World Health Organization 82:431-2. Over M, Piot P. 1993. HIV infection and sexually transmitted diseases. In: Jamison DT, Mosley WH, Meashan AR, Bobadilla JL (eds), Disease Control Priorities in Developing Countries. New York: Oxford University Press, pp. 455-527. Peeling RW, Ye H. 2004. Diagnostic tools for preventing and managing maternal and congenital syphilis: an overview. Bulletin of the World Health Organization 82:439-46. Ronald A, Plourde P. 1998. Why are syphilis programs failing? (editorial) International Journal of Infectious Diseases 2:121-2. Saloojee H, Sithembiso V, Goga Y, et al. 2004. The prevention and management of congenital syphilis: an overview and recommendations. Bulletin of the World Health Organization 82:424- 430. Schmid G. 2004. Economic and programmatic aspects of congenital syphilis prevention. Bulletin of the World Health Organization 82:402-409. Schulz KF, Cates W, O'Mara PR. 1987. Pregnancy loss, infant death, and suffering: legacy of syphilis and gonorrhea in Africa. Genitourinary Medicine 63:320-5. Schultz KF, Schulte JM, Berman SM. 1992. Maternal health and child survival: opportunities to protect both women and children from the adverse consequences of reproductive tract infections. In: Germain A et al. (eds.). Reproductive Tract Infections. New York: Plenum Press, pp. 821-42. Senyonyi F. 1987. Prévalence des infections gynécologiques à Chlamydia trachomatis et à Neisseria gonorrhoeae chez 199 femmes examinees en consultations prénatales ou en consultations pour planning familial à Kigali et à Kibgayi. Revue Médicale Rwandaise 19:85-89. 63 Smikle MF, James OB, Prabhakar P. 1990. Biological false positive serological tests for syphilis in the Jamaican population. Genitourinary Medicine 66:76-8. Stray-Pederson B. 1983. Economic evaluation of maternal screening to prevent congenital syphilis. Sexually Transmitted Diseases 10: 167-72. Temmerman M, Plummer FA, Mirza NB, et al. 1990. Infection with HIV as a risk factor for adverse obstetrical outcomes. AIDS 4:1087-93. Temmerman M, Lopita MI, Sanghvi HC, et al. 1992. The role of maternal syphilis, gonorrhea and HIV-1 infections in spontaneous abortion. International Journal of Sexually Transmitted Infections and AIDS. 3:418-22. Temmerman M, Fonck K, Bashir F, et al. 1999. Declining syphilis prevalence in pregnant women in Nairobi since 1995; another success story in the STD field? International Journal of Sexually Transmitted Diseases and AIDS 10:405-8. Villar J, Bergsjo P. 1997. Scientific basis for the content of routine antenatal care. I. Philosophy, recent studies, and power to eliminate or alleviate adverse maternal outcomes. Acta Obstetrica et Gynecologica Scandinavia 76:1-14. World Health Organization. 2001. Guidelines for the Management of Sexually Transmitted Infections. Geneva: World Health Organization (available from http://www.who.int/docstore/hiv/STIManagementguidelines/who_hiv_aids_2001.01). 64 ATTACHMENT G. SPECIAL REPORT 2. SUMMARY OF FOCUS GROUP DISCUSSIONS AND KEY INFORMANT INTERVIEWS The following is a summary of the reports of the focus group discussions that were held. Although a set of specific questions was developed for each type of focus group (as shown in Attachment G.3), we report here only a list of the strengths and weaknesses/ recommendations reported by those who carried out the focus group discussions. Altogether, 44 focus group discussions were held. For each, a group of 6-8 people were gathered for a discussion which was led by one of the Project staff members who were working in a different geographic area of the Project or by one of the external members of the evaluation team. There were also two Project staff members who recorded the comments made in the discussion group. The discussions lasted approximately 45 minutes - 1 hour. The schedule of focus group discussions, a list of who held them and who participated in them is also shown in Attachment G.3. Focus Group Discussions with Cell and Sector Coordinators (2 held) Strengths · The Project is well-known to us. · Volunteers have become local leaders. We now expect them to be present at all our community meetings, and we ask them to give health messages to those who are present. · We now have fewer deaths from diarrhea, and we no longer believe that those with diarrhea have been poisoned. We now have better hygiene and more latrines. · The Project transports MOH District staff and vaccines for EPI Outreach Sessions. · The Project is "within" the community, not outside of it. We like that. · The Sector Coordinators support the Project. When local resistance develops, they come to the Project's support. · The Volunteers promote Mutuelles and encourage community members to join them. · The Cell Coordinators are also reinforcing the health messages and verifying if the communities are implementing the messages. · The number of latrines has increased. · There are fewer deaths among children and among pregnant women. · We are glad the Project has decreased the cost of VCT. More will obtain this now. · We appreciate the availability of ITNs. Malaria has decreased as a result. · The Volunteers have been effective in promoting growth monitoring, EPI, and vitamin A campaigns. They call people together with drums. · The Volunteers are helpful in spreading messages to the community that they authorities want to transmit. · The Hearth Program is good for our mothers. We like that it is taught by neighbors. · Hygiene has improved. Our people are washing their hands, boiling water or using Sûr'Eau, using latrines, and keeping their dishes clean (by putting them on a special platform) after washing them. 65 Weaknesses/Recommendations · The Hearth Program needs to be able to give food to the poorest mothers - for those whose children are enrolled in the Hearth Program as well as for those who aren't. The Project should provide this. · There are not ITNs for all households. The price is not too high, and all households should be able to purchase them. · There are not enough community-wide meetings or meetings of communities from different Sectors coming together in which experiences can be exchanged (with Volunteers, Cell Coordinators, and Sector Coordinators present). We need to have more of these. · The Sector and Cell Coordinators have not received enough training. · The Volunteers need some incentives, such as a goat. The better Volunteers should receive more incentives. · The Volunteer Associations need support. They should receive domesticated animals (e.g. goats) to provide compost for their fields. · The Project should provide help with ARVs. · The Project should continue two years beyond the current end date. Focus Group Discussions with Volunteers (6 held) Strengths · The morbidity and mortality of children has decreased. · Household hygiene has improved. · Mothers know how to manage childhood diarrhea. · Project activities reach everyone in the village. · We thank the project for the knowledge it has given us, for helping the Volunteers to initiate Volunteer Associations, and for helping us obtain VCT. · We are thankful for the help that the Promoters have given us, especially in helping us give messages to resistant households. · We appreciate being able to do growth monitoring, and we appreciate the methods of teaching that we have learned (songs and pictures). · The bednets have reduced the incidence of malaria. · There is less dysentery and cholera now. · Community people now go to the Health Center instead of to charlatans and herbalists. · More mothers are going to the Health Center for ANC. · Mothers are now motivated to obtain immunizations, even immediately after their child is born. · Some of us (Volunteers) have been able to become Promoters, and sometimes we have been given additional responsibilities to lead the Hearth Program. · Our husbands are happier that their houses are cleaner, and they love us more. · All households now have small kitchen gardens. · Mothers know the danger signs of diarrhea and don't confuse diarrhea with poisoning. · Mothers don't confuse kwashiorkor with poisoning now. · The drinking water is now treated with Sûr'Eau or it is boiled. · Before, mosquito nets were only for the rich. Now they're for the poor as well. 66 · One Volunteer said, "If someone tries to stop me and my work, I will go to the justice authorities." · We appreciate that we are able to learn. It's like going to school without having to pay a fee. · We now feel free to go to the Health Center. The staff members there make us feel valued for our work. Weaknesses/Recommendations · We can't do anything to support complicated cases, such as PLWAs. · People come to us with many questions. They call us "people of word only." We want to be able to help in some practical and concrete way. We want training in first aid and basic simple medical care. · Some of us are too poor to implement some of the messages. For example, not all of us can afford to build a latrine or provide sufficient food for our children. · The training we have received regarding child spacing is not sufficient. We want information about specific family planning methods. · We need more training about STIs. · We need more training about VCT and about home care for PLWAs, and we need more training in how to counsel people who are HIV-positive. · Fathers need more training. Sometimes they are resistant. · We want training about nutrition in adults. Our husbands want this. · We need ORS packets. · We need incentives. The community laughs at us because we work for free. · We want to meet with all the Volunteers in our Sector to discuss our activities and share experiences. · We want domestic animals so we can be exemplary in nutrition. · We want the local authorities to continue to support us and to value our work. · When we become sick, we would like the Project to be able to help us obtain medical care. · We want financial support for our Volunteer Associations. · We want a certificate from the Project. · We want pants (for men) or skirts (for women) to go with our Project T-shirts so that we will have a "uniform." · We want training in income-generating activities. For example, we would like to be able to make soap or clothes. · We want hammocks to use to transport patients to the hospital. · We want more training in community mobilization methods to use during ceremonies and other important occasions. We want to learn games we can play, and we want to learn how to make presentations to the community. · We want to receive training like the TBAs received. · We want the Volunteer Associations to receive pay for vitamin A distribution. (The MOH has been paying Animators to assist with vitamin A distribution during national campaigns. Many animators are Volunteers, but the Volunteers provide a major support to the campaigns. Those Volunteers who are not Animators also want to be paid. About 60% of the MOH's Animators in the District are now Volunteers.) 67 · We want to visit other parts of the District to see what other Volunteers are doing. · We want to receive some kind of incentive from the Health Center when we refer a patient who is pregnant. (TBAs receive such an incentive now.) · Volunteers should not destroy bad latrines, but should let the local authorities do that. (In the past, some Volunteers destroyed these latrines on their own, and this has led to difficulties in the community.) · We want money to help us participate in Mutuelles. Focus Group Discussions with Traditional Birth Attendants (1 held) Strengths · We know the Project and what it is doing. · We have learned a lot about malaria and diarrheal diseases. · The Project is promoting mothers to give birth at the Health Centers and to obtain VCT. · The Project is teaching us to refer pregnant women to the Health Center when they have a problem. · Some diseases (like measles) have disappeared. · Growth monitoring is motivating to mothers. · Mothers now know how to make nutritious foods. · We are now able to do our work better because of the training provided by the Project. · Because of the training the Project has given us, we now give more value to our work and the importance of referring our patients to the Health Center. · We have received training in hygiene. · We appreciate the motivation that the Project has given to men. They are now readier to transport pregnant women to the hospital. · We appreciate the training we have received in how to complete reports about our activities. Weaknesses/Recommendations · We need flashlights to help us with our work when it is dark. · We need certain materials, such as gloves. · We need incentives. When we refer our patients to the Health Center, we no longer are paid by them. · We want more training about how to do our jobs better. Focus Group Discussions with Mothers/Caretakers (7 held) Strengths · The Project has done us a great service by making bed nets available. · The Hearth Program is very important. We have learned how to use local foods. · We are taught how to prepare drinking water. · We have learned how to care for PLWAs and now we are willing to care for them. · We now know how to treat diarrhea. Before, we thought giving more fluids would cause more diarrhea. · Our houses are cleaner and our hygiene is better. Light (umucyo) has really reached us. · Because we take our children earlier to the Health Center when they are not as sick, we don't have to pay as much. 68 · Our children are having fewer illnesses, so we have more money to spend on other needs. · The number of kwashiorkor cases has decreased. · We are grateful for VCT, for growth monitoring, and EPI. · Because of the Project's teachings, our husbands are now taking our children to the Health Center and to the growth monitoring sessions. · We don't confuse kwashiorkor with poisoning now. · We are grateful for the Hearth Program. Before, we thought that only food from other countries would cure kwashiorkor. · We now know the signs of malaria. · Mothers whose children have been in the Hearth Program now say, "Our children will never become malnourished again." · We appreciate the work of the Volunteers. They are always with us. They remind us when our children need to be vaccinated. · We now know the importance of colostrum and exclusive breastfeeding. · The Volunteers help us to learn many health messages. Weaknesses/Recommendations · We need more mosquito nets. · Those of us mothers with malnourished children want to form an association and receive training on income-generation. · The Hearth Program needs expansion. · The Volunteers should have basic medicines in order to treat certain illnesses. We think they should have anti-malarials, de-worming medicine, and ORS. · We need hammocks to transport sick patients to the hospital. · We need more information on child spacing. · We need training in agriculture and in carrying for domestic animals in order to reduce malnutrition. · During our growth monitoring sessions, we need to be able to give children who lose weight something like de-worming medicine to help them. · Illiteracy is a barrier for us. We want to be able to learn how to read and write. · We want to have VCT in every Health Center. · We want to have domestic animals for fertilizing our fields. · We want training in how to make milk from soya. Focus Group Discussions with Health Center Staff (7 held) Strengths · We are grateful for the collaboration for transporting vaccine and MOH District staff to EPI outreach sites. · We see fewer cases of malaria, and we believe this is due to the increased use of bednets. · We feel the collaboration of the Volunteers with growth monitoring is good because we in the Health Centers know we don't have the capacity to do this. · We have received many good training sessions. Ones in particular that were good include: immunizations, nutrition, child spacing, the Hearth Program, lot quality assurance sampling, KPC surveys, maternal health, Mutuelles, malaria management, and management of STIs. 69 · We have had many meetings with the Project staff. · Many more women are coming for ANC, people are coming earlier when sick, and there is an overall increase in Health Center utilization. · The Project has helped us develop a community-based information system. · We see fewer diseases now that are due to poor hygiene. · The Volunteers help to find children who haven't received all their immunizations and prevent dropouts. · We appreciate that the Project can get health information to everyone in the community. · The incentive to "reward" pregnant women who come to the Health Center for ANC during their first trimester with a bednet at a reduced price has been effective. · The Volunteers help us (Health Center staff) when we come to the community. They tell us which houses to go to when we come looking for patients to help. · The Project helps the Health Center staff get messages to the community, especially when there is a disease outbreak. · The Project transports medicines from the MOH District Office to the Health Centers. · The Project invites us to participate in the planning for Project activities, and we appreciate that. · We appreciate the Project's help with VCT. · The health data from the community that is provided by the Volunteers is integrated into our information system, which we pass on to the MOH District Chief. · There are good relationships between the Animators in the community and the Volunteers. · The surveys carried out by the Project have given us a clearer view of the Project's work and have helped us to have a clearer view of how the work in our District compares with that in other districts. · We are ready to continue working with the Volunteers after the Project ends. · Volunteers have become opinion leaders in the community. · When we lose an Animator, we chose a Volunteer as a replacement because the Volunteers are well-trained and highly motivated. · We appreciate the Volunteers being willing to work for no pay. Weaknesses/Recommendations · The Health Center is lacking some medicines and we need financial support from the Project in order to be able to get these. · We want to hold meetings with the Volunteers so we can get to know each other better and work better together. · We need computers for our health information system. · We would like for the Volunteers to give a message discouraging polygamy. This is a cause of malnutrition in children because the men who practice polygamy cannot support all their children adequately. · The Volunteers need to make more effort to motivate women to come to the Health Center for delivery. Providing soap, bednets, and/or Sûr'Eau would help a lot. · We need to be able to provide PMTCT at each of the Health Centers. At present, only one Health Center has this capability. · We need more meetings between the Health Center staff and the Project staff. 70 · We would like the Project to give us a salary supplement. · The Volunteers should receive more incentives. · The Kibogora Health Center staff members have a very negative attitude about the Project. · We want training in how to counsel PLWAs. · We want the Project to continue longer than for just two more years. Focus Group Discussions with Anti-AIDS Club Members (2 held) Strengths · The training we have received on HIV/AIDS has been good. It has enabled us to teach others. Now more people are going for VCT. · The training we received about how to support AIDS orphans and how not to stigmatize them has been good. · The Project has helped some of us to obtain VCT. · Our link with churches has been effective in addressing issues of AIDS and PLWAs. Weaknesses · We want uniforms from the Project. These will help us with our community mobilization activities. · We need more training on how to counsel, care for, and support PLWAs. We need to be able to give them food, blankets, and soap. · We want VCT. · We need audio-visual aids, such as videos. · We want training on income generation for associations, especially associations of PLWAs and their caregivers. · PLWAs need mosquito nets. · Some of our clubs have developed proposals for income generation. We want the Project to support us in this. Focus Group Discussions with Teachers (6 held) Strengths · We know the Project and its interventions. · We've received training on HIV/AIDS, and we have used this information to teach our students. · The methods the Project staff members have used to teach children in primary school have been effective. We would like to use this same approach in teaching our students about HIV/AIDS and other topics. · We see improvements in the community. There are now fewer cases of cholera and dysentery among school children. · Since we live in the community, we see many improvements in hygiene that have been made. Fewer children are missing school because of illness. · The messages we and the Project are giving school children are helping their parents to change. They are now boiling their water and changing other behaviors. · The way the Project disseminates messages is good, such as when there is an epidemic of typhoid among children. 71 · There are now fewer child deaths in the community. · The training provided by the Project is leading young people to try to decrease the spread of AIDS. · As a result of the training provided by the Project, mothers are now giving their children a snack to bring to school. · We now have booklets that the Project provided us and we are now using them to help us in our teaching. · We appreciate the growth monitoring activities that are taking place in the community. · We appreciate the support the Project has made to assist poor people in obtaining mosquito nets. · We appreciate how the Project has been able to motivate the Volunteers to work without pay. We're astonished to see them teaching for no pay when that is what we receive pay to do. Weaknesses/Recommendations · We want Sûr'Eau available for our children when that are at school. · We want booklets with health messages to put in the library. · We want other teaching aids and audio-visual aids to assist us in teaching the health messages. · We want the Project to assist us in establishing Anti-AIDS clubs in the primary schools. · We want a Volunteer to come to the primary school to teach the students. · One of the teachers in each primary school should become a Volunteer and teach the other teachers. · We want help in starting pre-schools and materials/training for teaching health messages to these young children. · We want the Project to help primary schools have domestic animals and to help the primary schools become a reference regarding domestic animals in the surrounding communities. · We want the Project to support AIDS orphans who are in primary school with clothes, notebooks, and so forth. Focus Group Discussions with Pastors and Pastoral Church Groups (5 held) Strengths · Malaria cases have decreased because of the bednet distribution program. · We now know not only to pray together when someone is sick but also to go to the Health Center. These two approaches are complementary, not competitive. · We are now able to distinguish malaria from illnesses caused by demons and poisons, and we know that prayer is not the proper treatment of malaria. · We now know that AIDS not only affects adulterers and those who don't believe, but good Christian people as well. · Since we are in contact with many people, the training provided by the Project has enabled us to teach many people. Our teaching about health complements our Bible teaching. · We are pleased that the Project recognizes us as community leaders. 72 · These groups are a good opportunity for us to share and work together among our different churches. · The Volunteers and the Promoters working with the pastors give added value to the messages the pastors are giving. · We appreciate that the Project is reaching the lowest level of the community. · The Project is helping church members to improve both spiritually as well as physically. · We appreciate the teaching methods used by the Project. We still remember the picture and story about malaria. · Now, many church members are giving testimony about the fact that they have HIV/AIDS. · We appreciate that the Project is helping everyone without differentiation. · We are no longer hesitant to share health messages. · The Project staff members sometime attend church services to share health messages. And sometimes they meet with groups within the church, such as youth groups or women's groups. · The Project has taught church members that it is not enough to be concerned with spiritual matters. They should also be concerned with health matters. Weaknesses/Recommendations · We would like to receive a certificate from the Project. · When make home visits to PLWAs, we would like to be able to offer them something. · We want to visit other parts of the District to see what pastoral groups are doing there. · We would like notebooks and pens. · We want the same incentives that Volunteers receive (T-shirts, and so forth). · We want training on how to counsel PLWAs. · We want training on child spacing, on STIs, on income-generating activities, management, agriculture and commerce. · We want training in how to prepare food for people in all age groups and in how to prepare weaning foods. · We want the Project to continue longer than just the next two years. · We want mosquito nets for all our church members. · We want the Project to support the formation of associations for church members who have AIDS. · We want training about tuberculosis and about adult education. · We are committed to continuing our activities after the Project ends. Focus Group Discussions with Community Development Committee Members (3 held) Strengths · We see many changes in our communities that represent for us community development: better hygiene, fewer malaria deaths, people using clean water for drinking, people making an effort to prevent AIDS. · We appreciate that the Project is assisting in the formation of Volunteer Associations. It is the national policy to form associations. · Volunteers are now exemplary community leaders in all areas, not just health. 73 · The Volunteers assist the CDCs in transmitting messages and in obtaining information the CDC needs. · We're happy because children are being vaccinated. · We appreciate the improved hygiene in the villages. · We appreciate the Hearth Program. It should continue. We appreciate that mothers can monitor the growth of their children in the village. · We appreciate the opportunity the Project gave us to visit Mutuelles in other districts, and we appreciate the support provided by the Project in helping us to establish Mutuelles. · We appreciate the collaboration which takes place between the CDC and the Volunteers, especially when an outbreak occurs. · We are pleased the Project is enabling the Volunteers to obtain VCT. This encourages other to get tested as well. Weaknesses/Recommendations · The Project should establish food-for-work projects. There is not enough food in the community. · The Project should add other interventions such as agriculture. · The Project should help us start pre-schools. · The Project should distribute bednets to the entire population. · We would like to meet with the Volunteers to share experiences. · We would like the Project to assist the Volunteer Associations with domesticated animals such as pigs and goats. · We would like more training on the health interventions so we can help more in the community. · We want help with terracing in our communities. Key Informant Interview with Kibogora District Hospital Leadership Strengths · There has been a decrease in the incidence of malaria, and we think that this has been due to the Project. Not only are there fewer cases, but the cases are less severe. · There has been an increased utilization of VCT by the Volunteers and their husbands. · More cases of childhood malnutrition are now being referred from the community. · There is now an increase in the utilization of hospital services. Patients are coming earlier for treatment. The number of deliveries in the hospital is increasing. · The emergency transport systems to bring pregnant women to the hospital are starting to work. · There are now fewer cases of diarrhea coming to the hospital. · The referral/counter-referral cards for severe malnutrition are helpful to us at the hospital and to the Volunteers, and they help to make the program more efficient. · People are now more aware of health issues and more motivated to utilize health services. · The hospital and Health Centers now have bednets for patients, thanks to the Project. · We appreciate the support the Project has given to the VCT program. 74 Weaknesses/Recommendations · We need help in getting our ambulance repaired or in obtaining a new one. · We need continued support for our VCT program. · We need help with internet access. · We want to develop a Hearth Program for our hospitalized malnourished children. · We need an additional computer. · We want to provide quality assurance training for all our staff. · We need case-management guidelines for tuberculosis and essential obstetric care. Key Informant Interview with Kibogora District Chief Strengths · The objectives of the Project are good. They address real community problems and community solutions to these problems. · We in the MOH District have developed good relations with the Project even though at first we were uneasy about the Project - particularly since the Project was proposing to "bypass" the District MOH staff and work directly with the community. · The good results achieved by the Project has motivated the MOH District staff and to collaborate with the Project. · As a result, our District has now achieved better results than other districts. · The system of Volunteers is very good and so are the other approaches to working with the community. In the past, projects have given money directly to the District. When the money stopped, the activities stopped. · The support and training given to the MOH District staff have been very good. · The Volunteers help the MOH District staff get messages out to the communities, especially when there are epidemic outbreaks. · We appreciate the support provided by the Project to help us finish our new office, to help us maintain office equipment, and to transport vaccines from Kigali, and to transport our staff and vaccines to the EPI outreach sessions. · The training provided to the MOH District staff on Mutuelles has been very good, as well as the opportunity to visit other child survival projects. · The KPC survey results are accurate - more accurate that the data at the Health Centers. Promoters and Volunteers are going to assist the Health Center staff in recording services provided (especially immunizations and vitamin A). · The mortality of children is falling because of Project activities, and the number of cases of malaria is also decreasing. · The Project's data collection system is integrated with the health information system of the MOH Health Centers, which pass their information to the MOH District Office. · The Project always has time to discuss issues raised by the MOH District staff related to the health information collected by the Project in the community. · The mid-term evaluation has been a good experience to see what has been accomplished and what remains to be done. 75 Weaknesses/Recommendations · Could the Project take over some of the activities that were previously supported by other donors, such as the Memisa/Cordaid activity or the Projet Sante Population? · We need help in transporting medicines from Kigali. · We need a formalized agreement for the Project to transport vaccines for us. · We need help in buying certain medicines. · We need help in repairing some of our Health Centers. · We need help in supplying some of our new Health Centers with equipment. · We need help in repairing our vehicle and also in obtaining motorbikes and radios. · The Volunteer Associations need help so that they will be sustainable. · The Volunteer Associations need training in management. · The Project should be more flexible in its management so that it can support some activities not mentioned in the DIP, such as those listed above. · The Project "promised" to help look for resources to obtain an ambulance, two motorbikes, and radios. · The Health Centers need help in forming and operating health committees. 76 ATTACHMENT G. SPECIAL REPORT 3. NOTES OF DISCUSSIONS WITH PROGRAM STAFF All of the 39 paid project staff (Project Administrator, Assistant Project Administrator, 4 Area Coordinators, and 33 Promoters) met with the evaluation team for extensive discussions. The entire days of Thursday, September 2, Monday September 6 and Tuesday September 7 were devoted to discussions with the entire Project staff regarding the Project's strengths and weaknesses and the findings from the focus group discussions. What follows is a summary of these discussions. Their Comments on the KPC Survey Results 1. The high level of malnutrition seen in the survey is a result of the fact that there has been a long dry season this summer and increased poverty, resulting in food shortages. The cost of food has doubled or tripled. Fishermen have not been able to obtain as much fish from the lake because of border security issues. Many families have no land or insufficient land for food production and therefore must obtain their food in the market with cash. In past, people could go to the forest to cultivate crops but the government does not allow this anymore. HIV-infected children are always malnourished, and the number of these children is increasing. 2. There has been poor documentation of doses of vitamin A and the doses of maternal TT immunizations that have been given, leading to a falsely low coverage of these activities as reported by the KPC. 3. We should have given more attention to promoting the utilization of ANC. Women are reluctant to go to a Health Center during the first trimester because they are afraid other women will laugh at them because they have a "small belly." Mothers are reluctant to give birth in the Health Center because it is usually far away from home and they can't pay the fee for this either. 4. There is only one site in the entire District where mothers can enroll in the PMTCT program. 5. There is more malnutrition and less malaria and diarrhea in August 2004 (when the mid-term KPC survey was carried out) than in December, 2001, when the baseline survey was carried out. Their Comments on Training The initial training was good, and there has been some continuing education, but we want more continuing education. At present, we receive continuing education once a month. The overall quality of our training is good, and our training materials have been adequate. Many of the training materials are in English and are difficult for us to understand. Many of the consultants who also come here do not speak French, making it difficult for us to learn. We received training from specialists initially, including experts from the MOH and the District Hospital. We took some very important educational trips. Four staff visited Mutuelles in other parts of Rwanda, 4 visited a child survival project of the International Rescue Committee, and 30 visited a CARE child survival project. 77 We need training in how to help the Volunteers continue their work after the Project is finished. We also want training in how to counsel Volunteers who underwent VCT and are HIV-positive. We need training in how to help the Volunteer Associations obtain skills in income generations. The Volunteers are very eager for this. Promoters are often asked by the community to help resolve conflicts. They need training in how to do this. We also need training in the data analysis of the surveys that we carry out. We are grateful for the opportunities that some of us have had to participate in surveys and to learn how to use EPI INFO. We want to continue our training so we can move to a higher level of work. Their Comments on Supervision The Promoters reported that their supervisor (Area Coordinator) comes to visit them in their communities once a week for an entire day. The day when the Area Coordinator arrives is usually unknown to the Promoter beforehand. Since we have a two-week workplan, the Area Coordinator knows where to find the Promoter. The Area Coordinator fills in for the Promoter if the Promoter is unable to be present for the meeting of a Care Group. The Promoters are pleased with the supervisory system and say that their supervisors come to help them, not police them. The Area Coordinators report that they have a checklist they follow when the go to the field for supervision. They spend almost every day in the field supervising. The Area Coordinators are pleased when the Promoters sometimes ask them to come and help with a difficult issue or problem. The Volunteers are also pleased when the Area Coordinator comes to a Care Group meeting to assist the Promoter. The Assistant Project Manager spends three days a week in the field supervising the Area Coordinators. Sometimes the visits are announced and sometimes they aren't. He takes a supervisory check list with him. While he is in the area, he also makes contact with the local authorities, talks with Promoters individually, and talks with the Volunteers. The Area Coordinators say that they appreciate the help and support of the Assistant Project Manager in the field and that they appreciate his help in dealing with community leaders and church leaders. They say that the Volunteers are encouraged when they see the Promoter supported by the Area Coordinator and the Area Coordinator supported by the Assistant Program Manager - it gives value to their work. When the supervisors are making home visits with the Volunteers, it demonstrates value of the work in they eyes of the community. Their Comments Regarding Efforts to Improve Health Worker Performance Trips to other project sites have been educational and very motivational. We have very good trainings. The ones they mentioned specifically were trainings on: · Adult education · Community mobilization · Positive deviance methodology · Communication techniques 78 · English · Computer skills · How to ride and take care of a motorcycle · Counseling · Data gathering techniques (KPC, local rapid assessment, lot quality assurance sampling) · How to do a home visit · Supervision · Resources management · Marketing and using Sûr'Eau (a solution of 0.5% sodium hypochlorite solution which disinfects contaminated water) · Special kits for water and food analysis. We are grateful that the Project has paid for us to take driving lessons (for motorcycles). New staff in the Project need a lot of special training in order to be able to catch up. Their Comments on Technical Support and Resources We need a few more motorbikes. We have 18 at present. We need better roads within the District and also a better way to communicate among ourselves. We need better visual aids (especially videos) to assist us in our education about HIV/AIDS and STIs. The printed materials on child survival in Rwanda are very limited. We need more. Their Comments of Information Management and the Health Information System We appreciate the information collected by the Volunteers in their Care Groups because we know it is accurate. Our community-based information system makes it possible to learn quickly if there is a health problem in the community. The Promoters can compare the results of the community-based health information system on a month-by-month basis. The data collected in the community-based health information system is useful for the Health Centers. They and the Promoters review this data and make decisions regarding their work based on this information. We have good forms and they are important. But we need to devised forms for the Volunteers to use. At present, they are keeping information in their own personal notebooks. We need to share the information from the community-based health information system with the Community Development Committees. We are starting to do this, but we need technical assistance. We also want to be able to help the Health Centers make better use of the data collected in the communities. We hope that the community-based health information system will be able to continue after the Project ends. Their Comments on Logistical Support We have a need for many things. We need: · ORS packets · More weighing scales for growth monitoring · Audio-visual aids and generators to use in teaching in the community - especially for AIDS and STIs. · Video recorder and digital camera to document our work · Laptop computer for the MOH District Chief 79 · Radio communications with the Health Centers and two-way radios for staff · Internet access Their Comments on Technical and Administrative Support We appreciate the specialists the Project has recruited to teach us. We appreciate the training workshops that have been given to the Promoters and we want them to continue. The staff meetings that we have are very helpful. We appreciate the supervision that we receive. We appreciate the support provided from the World Relief office in Kigali and from Headquarters in Baltimore. Their Comments on Project Planning We appreciate the way that the Project plans with all the staff together, and we appreciate the way that the Project also invites the MOH District staff to participate in the planning as well. We are glad that the Project uses national guidelines in developing its plans and activities. The DIP has too many activities in it, and we wish the DIP were more flexible. Their Comments on Personnel Management The staff has a good relationship with the Project administration. We're paid on time. When we make mistakes we are corrected kindly and respectfully, so we are grateful for that. We are grateful that the Project helps us with our medical expenses (it pays for 80% of staff medical expenses). If a family problem arises, the Project helps us out. The Project vehicle is available at times for important personal events, such as a wedding. We appreciate the support and guidance the Project is giving us spiritually. We need more salary, and when we take a trip for the Project, we need to have our expenses reimbursed. Their Comments on Financial Management The Project has good financial management. Every small expense has to be justified and accounted for. Sometimes, however, there is not enough money to cover necessary expenses. Their Summary of the Projects Weaknesses (from their perspective, provided at the end of the evaluation, on the basis of discussion in 4 groups) · We don't give enough attention to child spacing [4 groups mentioned this] · We haven't made enough progress in nutrition [3 groups mentioned this] · We are losing trained staff [2 groups mentioned this] · Too many women are still giving birth at home [2 groups mentioned this] · We have not had enough training in STIs [2 groups mentioned this] · Mosquito nets are not available to everyone [1 group mentioned this] · Too many families don't have adequate food for their needs [1 group mentioned this] · The number of meetings between Volunteers and local authorities is not enough [1 group mentioned this] · The Community Development Committees, primary school teachers, and Anti-AIDS Clubs are not receiving enough training 80 · We have not had oral rehydration solution (ORS) packets to give in the community · The paid staff and the Volunteers have not received enough training in HIV/AIDS. · We have not provided sufficient support to the Volunteer Associations · We have not provided any material support to PLWAs Their Recommendations for Project Strengthening (based on conclusions from 4 discussion groups and votes from all individual staff, with each staff member given 3 votes) · Support Volunteer Associations [35 votes, recommended by 4 discussion groups] · Strengthen the Hearth Program [30 votes, recommended by 3 discussion groups] · Increase staff salaries [27 votes, recommended by 4 discussion groups] · Lengthen the duration of the program [16 votes, recommended by 4 discussion groups] · Train the community in income-generating activities [8 votes, recommended by 1 discussion group] · Continue to promote delivery in the Health Center to pregnant mothers [6 votes, recommended by 1 discussion group] · Distribute mosquito nets to all households [4 votes, recommended by 1 discussion group] Their Initial Thoughts Regarding What Elements Should Be Included in the Proposed Action Plan for the Final Two Years of the Project (final discussion held with staff, listed in order of importance, with number of the four discussion groups recommending this as a priority) · Provide support to PLWAs (e.g., food, anti-retroviral medications) and to the Anti-AIDS Clubs (training, audio-visual aids) [5 groups recommended this] · Provide higher salaries for the staff and more motorcycles [4 groups mentioned this] · Strengthen the Hearth Program: expand it, provide food for those who can't obtain it, carry out better follow up, develop a Hearth Program for malnourished children who are hospitalized [3 groups recommended this] · Support the Volunteer Associations: provide them with money, domesticated animals, training for the Association [3 groups mentioned this] · Provide more support to the MOH District: more training for staff, more support for transportation (ambulance and motorcycles), medications, laptops, computers, radios [3 groups mentioned this] · Continue to empower communities (3 groups mentioned this] · Provide income-generating activities for the population, including agriculture training and support for micro-finance [3 groups mentioned this] · Provide mosquito nets for everyone in the population [3 groups mentioned this] · Lengthen the Project beyond its current end date [2 groups recommended this] · Extend VCT/PMTCT to all Health Centers [2 groups recommended this] · Provide more training to the Community Development Committees [1 group recommended this] · Provide more training to the Umucyo staff [1 group recommended this] · Provide more training to Pastoral Care Groups [1 group recommended this] · Integrate agricultural activities into the nutrition program [1 group recommended this] · Provide visual aids for the trainings on HIV/AIDS and STIs [1 group recommended this] 81 · Rehabilitate one of the Health Centers which is in poor condition [1 group recommended this] · Maintain good relations with all the partners (MOH District, churches, community leaders, and so forth) [1 group mentioned this] · Improve the health information system [1 group mentioned this] 82 Attachment G. Special Report 4. Preliminary Analysis of Mortality Data The data from the Health Information System (HIS) was reviewed to assess rates of under-five mortality. Each month, the Volunteers report the number of births, number of deaths of under￾five children, and number of deaths of women of reproductive age. These data began to be recorded in the HIS beginning in July 2003, approximately one year after beginning to implement the Project interventions. Table 1 summarizes the available information. Data are not available for more than 9 months in any of the Health Areas, and data were not available for October 2003. The Yore Health Area later merged into the Ruheru and Rangiro Health Areas. The rates observed here represent a 79% reduction compared to the overall under-five mortality rate observed for Rwanda in the Rwanda Demographic and Health Survey 2000 (196.0 vs. 78.6). This finding should be viewed as a tentative rather than a conclusive one. Higher-quality data are needed with convincing quality control before definitive conclusions can be drawn. Table 1. Summary Information regarding Births and Under-Five Deaths, Kibogora Health District, July 2003-August 2004 Health Area Number of births recorded Number of under-five deaths recorded Under-five mortality rate (deaths per 1,000 births) Comments Gatare 540 41 75.9 Based on 9 months of data (before July 2003, Oct 2003, and after April 2004 missing) Karambi 383 29 75.7 Based on 8 months of data (before July 2003, Oct 2003, and after April 2004 missing) Nyamasheke 485 36 74.2 Based on 8 months of data (before July 2003, August 2003 (births only),Oct 2003, and after March 2004 missing) Rangiro 410 36 87.8 Based on 8 months of data (before July 2003, Oct 2003, and after March 2004 missing) Ruheru 617 49 79.4 Based on 9 months of data (before July 2003, Oct 2003, and after April 2004 missing) Yore 46 4 87.0 Based on 1 month of data (July 2003) Total 2,481 195 78.6 An analysis of the under-5 mortality data by month is shown in the figure below. There is no clear monthly trend apparent here. Perhaps what is most striking is the marked variation in the number of births reported each month, raising the question of the quality of the data and reinforcing the need to implement quality control procedures. 83 Under-5 Deaths, Births, and Under-5 Mortality Rate by Month, Kibogora Health District, July 2003-April 2004* 0 50 100 150 200 250 300 350 400 450 500 Jul￾03 Aug￾03 Sep￾03 Oct￾03 Nov￾03 Dec￾03 Jan￾04 Feb￾04 Mar￾04 Apr￾04 IMR Births Deaths *October 2003 data missing Unfortunately, no baseline data are available for the period before the Project began or soon after the Project began. This problem can be overcome by carrying out a pregnancy history survey at the time the Project ends. This will provide retrospective data for the 5-year period before the Project began as well as for the 5-year period of Project implementation. 84 ATTACHMENT G. SPECIAL REPORT 5. DESCRIPTION OF THE HEALTH INFORMATION SYSTEM The Project has developed a remarkable health information system to monitor progress, strengthen program performance, and strengthen the performance of individual staff members. Community-based Routine Health Information System Each month, the Volunteers provide a report to the Promoter about his/her activities, and this information is consolidated into a monthly report for each Health Area and then for the entire District. The information collected depends on the stage of intervention implementation. Once a specific intervention is implemented, then specific indicators are added to assess the progress in implementing that particular intervention. Thus, with the implementation of the first intervention (diarrhea and hygiene), information was collected about the following: · How many homes have latrines, how many are in good condition, and how many are in poor condition · Whether a compost is present for storing biodegradable materials · Whether ORS is present or not in each home · Whether a stand for drying dishes is present Indicators monitored in this fashion for the EPI Intervention include: · Number of children up to date with their vaccination schedule, number who have completed their schedule, and number who have fallen behind · Number of mothers with up-to-date TT immunization and those that need additional doses Indicators for the HIV/AIDS Intervention include: · Number of persons who have obtained VCT Indicators for the Malaria Intervention include: · Number of households with ITNs and without ITNs · Number of retreatment kits sold for ITNs (sales of Karishya Force) Indicators for the Nutrition Intervention include: · Number of children with no malnutrition, with moderate malnutrition, and with severe malnutrition (based on growth monitoring) · Total number of children weighed · Number of children receiving vitamin A capsules Indicators for the Reproductive Health Intervention · Number of pregnant women obtaining their first ANC during the first trimester of pregnancy, during the second trimester, and during the third · Place of birth for previous pregnancy 85 The Volunteers also report the number of births, deaths of under-5 children and deaths of women of reproductive age. In addition, each month the Promoters provide a report based on their activities. Included in this report are: · Number of meetings with Volunteers · Number of EPI outreach sessions held · Number of field supervisions of Volunteers and, of that number, how many performed well and how many performed poorly · Number of home visits carried out · How many meetings with the community were held · Number of meetings with Kibogora Health District staff · Number of visits with church groups · Number of contacts with church pastors and other religious authorities · Number of supervisory visits carried out · Percentage of members of Care Group attending each meeting The compilation of this information on a monthly basis enables the Area Coordinators, the Assistant Project Manager and the Project Manager to monitor progress in field activities, to make corrections to strengthen Project activities, and to provide additional support for those Promoters and Volunteers who are lagging behind the others. Local Rapid Assessment (LRA) In addition to this information provided on a monthly basis by Volunteers and Promoters, a system of local rapid assessment (LRA) is carried out quarterly. This has been developed using the lot quality assessment survey approach. All of the indicators associated with Project objectives are assessed by interviewing a sample of mothers. The findings are then compared with the baseline KPC survey and with the EOP goals. Each quarter, with all of the Promoters present, the Project selects at random 9 Care Groups to be included in the LRA. Then, 2-3 Volunteers within each of the Care Groups are chosen at random, yielding a total sample size of 18-27 Volunteers. Once the data are collected, they are tabulated by hand by Health Area. Then the results are shared with the entire Project staff and discussed. Normally, 2 days are required to collect the data and 1 day to tabulate it. At the time of discussion of the LRA findings, MOH District staff members attend the discussion as do representatives from the Kibogora District Hospital. Then, feedback is provided to the Volunteers and to the community. The Program Administrator and Assistant Program Administrator decided what information will be collected, and the information that is collected varies from quarter to quarter. The MOH Health Center staff members serve as interviewers and go out to the community to visit the homes of the Volunteers selected to be in the LRA. The Chief of the Care Group assists then interviewer in locating the houses where the Volunteers to be interviewed are living. A summary of these findings for mothers can be found in Table 1. Similar information is collected from a sample of Volunteers. 86 A brief review of the LRA reports suggests that there is some "drift" in the definition of certain indicators from quarter to quarter, so they may not be measuring exactly the same behavior. Some indicators are, for instance, "at least the same or more" fluids or food when the child is sick and has diarrhea, and some are just "more." The Project's Program Manager and Assistant Manager say that they do not have enough time to manage the HIS and use it optimally. As a result, they are proposing that they Project hire an HIS manager. 87 Table 1. Findings from Quarterly Local Rapid Assessments (LRAs) Compared with KPC Survey Findings and End-of-Project Objectives Indicator Baseline KPC level LRA 1 (Oct 2002) LRA 2 (Mar 03) LRA 3 (Jun 03) LRA 4 (Sept 03) LRA 5 (Feb 04) Mid-term KPC level EOP objective Diarrhea/hygiene 1. % of women who now at least 3 dangers signs of diarrhea 83.0 (2 signs) 66.0 70.2 73.1 81.6 63.8 77.7 75.0 2. % of mothers who give more liquids when their child has diarrhea 31.0 71.4 80.0 90.4 85.5 46.4 66.7 50.0 3. 3. % of mothers of children age 0–23 months who wash their hands with soap/ash before food preparation, before feeding children, after defecation, and after attending to a child who has defecated 0.0 80.5 (3 occasions) 91.8 (3 occasions) 61.9 (3 occasions) 39.3 4. % of women who have a latrine in good condition 40.5 51.0 57.2 61.0 EPI 1. % of children who have completed all their immunizations prior to their 1st birthday 41.0 93.1 94.6 97.1 94.5 85.5 85.0 2. % of children age 9-12 months who received a measles vaccination 76.6 87.1 2. % of children who have received at least 1 dose of vitamin A 33.8 12.5 15.4 50.0 88 Indicator Baseline KPC level LRA 1 (Oct 2002) LRA 2 (Mar 03) LRA 3 (Jun 03) LRA 4 (Sept 03) LRA 5 (Feb 04) Mid-term KPC level EOP objective 3. % of women who received at least 2 doses of TT during the previous pregnancy 43.8 88.5 91.1 92.2 89.1 54.3 (according to verbal report, 22.7% according to card review) 60.0 HIV/AIDS & STI 1. % of women who know at least 2 symptoms of STIs or AIDS 47.0 65.8 50.6 92.3 80.0 2. % of women who know 3 ways to prevent HIV/AIDS 80.0 65.5 76.6 59.7 91.3 3. % of women who affirm that they would be willing to care for a person with AIDS 94.9 94.5 96.7 80.0 Malaria 1. Among those infants who had fever in the previous 2 weeks, the % who sought treatment 3.7 46.8 42.0 31.2 50.0 2. % of <2 children who sleep under ITN 3.0 17.2 64.0 66.0 40.0 3. % of pregnant women who sleep under ITN 3.5 6.5 39.1 76.2 64.5 40.0 Nutrition 1. % of infants age 0-5 months who were exclusively breastfed in the last 24 hours 60.3 59.4 66.8 83.3 75.0 89 Indicator Baseline KPC level LRA 1 (Oct 2002) LRA 2 (Mar 03) LRA 3 (Jun 03) LRA 4 (Sept 03) LRA 5 (Feb 04) Mid-term KPC level EOP objective 2. % of mothers who breastfeed their babies immediately after delivery during the first hour 37.7 71.3 3. % of mothers who are giving appropriate weaning food to the child (6-23 months of age) at least once a day 19.5 71.8 35.4 46.6 50.4 50.0 4. % of mothers who give same of more food when child sick 11.2 72.0 82.7 74.8 67.4 60.0 5. % of children < 2 years of age with weight-for-age >/= 80% 84.1 79.6 78.4 76.9 71.7 6. % of children < 2 yrs of age with weight-for-age <80>60% 17.0 22.2 7. % of children < 2 yrs of age with weight-for-age <0% 4.6 0.8 8. % of children 0-23 months who are underweight (-2 SD) 15.9 21.2 23.1 28.3 9. % of children who have completed Hearth Program who have gained 200-400 grams per month since the completion of the program 53.3% (August 2004 records) 80.0 90 Indicator Baseline KPC level LRA 1 (Oct 2002) LRA 2 (Mar 03) LRA 3 (Jun 03) LRA 4 (Sept 03) LRA 5 (Feb 04) Mid-term KPC level EOP objective Reproductive Health 1. % of women who obtained ANC during previous pregnancy 90.2 95.7 2. % of pregnant women who have emergency plan for transport in place before delivery 24.6 65.5 78.3 85.9 46.7 70.0 3. % of women who gave birth at a health facility 23.2 23.9 29.8 26.8 35.0 50.0 4. % of women using an injectable contraceptive 1.0 11.7 6.7 5. % of women using birth control pills 1.7 1.0 6.0 6. % of women who have given birth to a second child more than 24 months after the birth of the previous child 73.1 84.4 83.7 77.0 IMCI 1. % of mothers who know 3 danger signs of sick children for which treatment should be sought 92.3 81.8 92.1 74.8 98.0 (2 signs) 91 SPECIAL REPORT 6. RWANDA CHILD SURVIVAL PROJECT Rapid Knowledge, Practice and Coverage (KPC) Survey Report For Midterm Evaluation Kibogora Health District Cyangugu Province, Rwanda Dates: 2 August- 6 August 2004 Authors: Kabadege Melene Kwizera Maurice Bradbury Kathryn Chandler Rebecca 92 ACKNOWLEDGEMENTS World Relief Rwanda Child Survival Program wishes to acknowledge and thank the following: Our most sincere thanks are directed to United States Agency for International Development for their fiscal and programmatic support. The Rwanda National Ministry of Health for collaboration in the ending half of the program Warm thanks to Kibogora Health District’s administrative board and the chiefs of the Health Centers for their active participation in the KPC survey activities at all levels. Thanks to World Relief Headquarters for their great support, especially in the KPC report development. And to World Relief Rwanda, who sent two staff members, including the Director of the WRR programs, to assist with data analysis. Particular acknowledgement is directed to the Mayors of Gatare and Nyamasheke Districts who gave permission without delay for the Child Survival project to conduct the KPC survey in their areas of jurisdiction. Thanks also for their ongoing support and encouragement. Thanks and appreciation for the volunteer leaders of the care groups who have kindly guided and introduced the survey team to the community and households. In addition, thanks to the survey team for their diligence and precision in data collection. Thanks to Mr. Isaac Ntahobakurira, of the School of Public Health, Butare, Rwanda for providing guidance and support in data analysis. Special thanks to Sheila Etherington and the Free Methodist Mission for ongoing friendship and support. And most sincere thanks to the mothers of Kibogora Health District who willingly participated in our survey, and to countless others within the community of Kibogora who have supported the Umucyo Child Survival project. 93 ACRONYMS ARI Acute Respiratory Infection CDC Community Development Committee CSP Child Survival Project DHS Demographic Health Survey EPI Expanded Program for Immunization ITN Insecticide Treated Net KCP Knowledge Practice and Coverage KHD Kibogora Health District PLWA People Living With AIDS PMTCT Prevention of Mother To Child Transmission VCT Voluntary Counseling and Testing 94 TABLE OF CONTENTS ACKNOWLEDGEMENTS ACRONYMS EXECUTIVE SUMMARY 1. INTRODUCTION ...........................................................................................................................96 1.1 PURPOSE OF THE STUDY..................................................................................................96 1.2 PURPOSE OF “UMUCYO” CHILD SURVIVAL PROGRAM..................................................96 1.3 PROGRAM GOALS ............................................................................................................96 1.4 ORGANIZATIONAL STRUCTURE OF UMUCYO..................................................................97 1.4 BENEFICIARY POPULATION: LOCATION AND SIZE..........................................................98 1.5 SITUATIONAL ANALYSIS ..................................................................................................98 1.6 SUMMARY HEALTH PROFILE AT BASELINE....................................................................99 1.7 SURVEY SCHEDULE: DATES AND ACTIVITIES ...............................................................100 2. METHODOLGY.....................................................................................................................101 2.1 QUESTIONNAIRE: REVIEW AND ADAPTATION......................................................................101 2.2 SAMPLING DESIGN .........................................................................................................102 2.3 DATA COLLECTION........................................................................................................103 2.4 DATA ANALYSIS .............................................................................................................103 3. RESULTS AND DISCUSSION..............................................................................................103 3.1 CHARACTERISTICS OF THE SURVEY POPULATION...............................................................103 THE FOLLOWING TABLES, COMPILED FROM SURVEY DATA, PROVIDE AN OVERVIEW OF THE RESPONDENT POPULATION:...........................................................103 3.1 RESULTS FOR INTERVENTION INDICATORS..........................................................................106 TABLE 1: IMCI SICK CHILD:..................................................................................................107 TABLE 2: MALARIA PREVENTION AND RAPID CARE SEEKING........................................................107 TABLE 3: DIARRHEA AND HYGIENE:.............................................................................................108 TABLE 4: NUTRITION....................................................................................................................108 TABLE 5: IMMUNIZATION .............................................................................................................110 TABLE 6: REPRODUCTIVE HEALTH AND HIV/AIDS .....................................................................111 4.0 CONCLUSION AND RECOMMENDATIONS..................................................................112 95 EXECUTIVE SUMMARY Background: This survey was conducted during the period 2 August through 6 August 2004 as part of a midterm evaluation for the “Umucyo” Child Survival Project (CSP) sponsored by USAID/World Relief. This five year project (2001-2005) is being implemented within the Kibogora Health District (KHD), Cyangugu Province, Rwanda. Overall Project goals are to: 1) Reduce morbidity and mortality in children 0-5 years and among women of child bearing age (15-49), 2) Strengthen the capacity of the Kibogora Health District to implement and sustain Child Survival (CS) interventions, and 3) to empower communities to improve their health. When the project began in 2001, Rwanda’s under five mortality rate was estimated at 196 per 1,000 live births and the maternal mortality rate at 1,300 per 100,000 live birthsi . Within Kibogora Health District, malaria is the primary cause of morbidity and mortality among children under fiveii. Other leading causes of childhood death include Acute Respiratory Infection (ARI), diarrhea disease, malnutrition and HIV/AIDS. Purpose: To measure progress toward project goals by comparing results from this midterm survey with data collected in 2001 at the baseline. Survey data reports knowledge, practice and coverage (KPC) within KHD related to the following standard indicators: a) Mothers educational background, b) household sanitation knowledge and practices, c) breast feeding and child nutrition, d) diarrhea case management, e) growth monitoring, f) child immunizations, g) malaria case management, h) maternal care, i) HIV/AIDS and other sexually transmitted infections (STI). Setting: Kibogora Health District is located in the south western region of Rwanda, along the shores of Lake Kivu. The total population is 145,583 (KHD records). Direct beneficiaries include 34,066 women of child bearing age and 24,021 children under five years old (KPC Survey, August 2004). Results: The Umucyo CSP has recorded steady progress in most intervention objectives. Significant achievements were made in prevention of malaria and diarrhea, as net use among children under five years and pregnant women increased from 3% to 66% and 3.5% to 64.8% respectively. Incidence of fever within past 2 weeks for under fives decreased from 74.8% to 28.7%, For diarrhea, 2 week prevalence decreased from 66.4% of under fives to 26.7%. Worsening socioeconomic conditions threaten CSP efforts to combat malnutrition. While data indicate increased knowledge and compliance with nutritional practices (77% growth monitoring, early initiation of breastfeeding 37-71%), the percentage of malnourished children monitored by the project has increased (15.9% to 28.3%). Umucyo fully recognizes the threat posed by food security, and is already working to identify income generation or sustainable agriculture schemes. Plans to improve vaccine records by card are in effect between the CSP and Kibogora Health District. Conclusion: Umucyo has achieved results in a wide range of child survival and community health activities. Efforts to address the complex problem of malnutrition will require extensive efforts and possible linkage with another program initiative in order to address the growing problem of food security. 96 1. Introduction 1.1 Purpose of the Study Results from this midterm survey will be used by the CSP, the beneficiary communities, and other stakeholders to make informed decisions regarding public health within Kibogora Heath District. Comparison of this midterm data with that measured at the baseline in 2001 will indicate both areas of success and areas for improvement which, along with findings from the project’s midterm evaluation exercise, will guide the CSP strategy throughout the project’s remaining three years. KPC survey data reports knowledge and practices within KHD related to the following standard indicators: a) Mothers educational background, b) household sanitation knowledge and practices, c) breast feeding and child nutrition, d) diarrhea case management, e) growth monitoring, f) child immunizations, g) malaria case management, h) maternal care, i) HIV/AIDS and other sexually transmitted infections (STI). 1.2 Purpose of “Umucyo” Child Survival Program This five year CSP( September 2001- September 2006) is funded by USAID and World Relief Corporation. World Relief is faith based, private voluntary organization, working to build capacity within communities and promote long term stability in post conflict societies. World Relief Child Survival programs work in partnership with local communities and institutions, seeking to strengthen community ownership and sustainability of development efforts. The WR Rwanda CSP project was named “Umucyo” by the Rwandese project staff. Umucyo means “to illuminate” in Kinyarwanda and was deemed an appropriate metaphor to encompass the project’s mission, which is to educate communities and build their capacity to improve their health status. Specific project goals are as follows. 1.3 Program Goals 1) Reduce morbidity and mortality among children 0-5years and women of child bearing age, defined as 15-49. 2) Strengthen the capacity of the Kibogora Health District (KHD) to implement and sustain Child Survival interventions. 3) Empower communities to make decisions to improve their health status. For complete description of project goals and end of project objectives (EOP), see Annex 1. 97 1.4 Organizational Structure of Umucyo As with all World Relief Child Survival programs, Umucyo is a community based program that depends on a vast net work of volunteers to train households in preventive and care seeking practices. The Program Manager and all paid staff are Rwandese nationals; most are from the Kibogora region and all live within the areas were they work. The program is implemented using the Care Group Model strategy, which involves training and mobilizing a vast network of volunteers, who are each responsible for training and monitoring the health of 10-15 households within their own neighborhood. The volunteers are organized into units of 10-15 called “care groups”. There are usually 1 or 2 care groups per cell community. The volunteers come together twice per month at care group meetings were they are visited by a World Relief staff member called a health Promoter. Each care group has a designated leader or chief; together the chief and Promoter conduct the care group meetings. The Promoter shares a health message and then the care group practices training each other in the information being presented, in preparation to make home visits. During the months time before the next care group meeting, each volunteer must visit each of the 10-15 households for which they are responsible and share the training with mothers or caretakers. Care group meetings are also time for sharing concerns from the communities and for reporting vital data, such as births, deaths and illnesses. The promoter then submits the community data reported by each care group to their Area Coordinator. Umucyo has 8 area coordinators, each responsible for Promoters and volunteers within one of 33 sectors, 31 from Kibogora Health District and 2 from Rusenyi District in Kibuye Province. The Area Coordinators report to the Program Manager and Assistant Manager, who are responsible for all training, technical guidance, logistics and overall function of the Umucyo program. 98 Organizational Structure of Umucyo 1.4 Beneficiary Population: location and size The Umucyo CSP covers the entire Kibogora Health District, which includes the administrative district areas of Nyamasheke and Gatare. The total area is approximately 648,244 square kilometers. These districts are subdivided into 32 sectors, with 179 cell communities. Each cell typically includes 50-100 households, grouped into blocks of 10-19 households called nyumba kumi. Since the project began in 2001, additional communities from 1 sector of Kibuye Province have been added in response to requests from administrative chiefs, bringing the total number of sectors served to 33. Direct beneficiaries include 33,484 women of child bearing age (defined as women aged 15-49) and approximately 24,021 children under five years old. 1.5 Situational Analysis Kibogora Health District is entirely rural. The terrain is mountainous and bordered by Lake Kivu to the west, and the Nyungwe Forest to the east. The humid climate, particularly in Mutovu, Karundura and Kamiranzovu valleys, provide an ideal context for malaria causing mosquitoes. Road and phone access throughout Kibogora remains extremely limited. The region receives Program Manager Assistant Manager Area Coordinator #4 Promoters Care Group: 10-15 Volunteers 5 2 4 1 2 9 7 4 1 0 9 8 5 6 4 1 3 Each Volunteer within Care Group responsible for 8 Area Coordinators 31 Promoters 286 Care Groups 2,864 Volunteers 29,166 Households AC #7 AC#6 AC #8 AC #3 AC #5 AC #2 AC #1 99 adequate rainfall and most inhabitants are subsistence farmers. However, Rwanda is the most densely populated nation in Africa and the average family farms ¼ hectare of land or less, which is inadequate for meeting dietary needs. While the markets are stocked with a variety of fruits and vegetables, malnutrition was identified at the baseline to be widespread within the region. The primary employers within the area are the mission hospital and a tea plantation in Yove Administrative District, 25 kilometers from Kibogora. Those in close proximity to Lake Kivu may also make their living fishing, operating taxi boats, or by trading with Congo. Local markets and small shops support a marginal degree of commerce. Rwanda still suffers from the devastation of the 1994 genocide, referred to locally as “the war”. As of 2000 there were an estimated 85,000 orphaned children, though that number has surely increased due to the AIDS epidemiciii. National infrastructure, social fabric and human resources were severely impacted by the massacres. Rwanda lost many qualified teachers and as of 2000, 27.1%iv of women in Cyangugu province were illiterate. The Rwandese family structure was particularly devastated. As of 2000, 36.9% of households were female headedv . Many women were widowed or their husbands have been imprisoned; 130,000 people remain in prison awaiting trial for genocide-related crimes. Within Kibogora, already strained socioeconomic conditions have worsened in the past 3 years and many women or child headed households struggle to provide for extended families and a growing number of orphans. Cultural traditions such as “inkwano” (bride price) further burden families and pose barriers for women and orphans. In addition, lingering mistrust complicates reconciliation. Even child rearing practices have been affected by the lasting effects of trauma experienced by the majority of families. Life expectancy averages 49 years for women of Cyangugu provincevi. Rwanda is one of nine African countries most affected by the AIDS virusvii. As of 2000, sero-prevalence rates reached 10.8% among men and 13.5% among womenviii.Poor socioeconomic conditions, lack of health infrastructure and education due to the disruption caused by genocide combine within a context of social and sexual practices to ensure rapid rates of HIV transmission. 2002 Data from Kibogora Health Center, the only Health Center in Kibogora Health District that provides PMTCT testing and counseling, reported that 5.9% of mothers willing to participate in PMTCT were HIV-positive (67/1133). This rate only includes women willing to be tested within one of 7 Health Centers Areas. Knowing that approximately 30% to 40% of children born to HIV-positive mothers will also contract the AIDS virusix, there is an obvious need for ongoing efforts toward AIDS prevention, and care for those affected by the disease, both directly and indirectly. 1.6 Summary Health Profile at Baseline National infant mortality rates in 2000 were recorded at 107 per live births; the under five mortality rate was 196 per live birthsx . Maternal mortality rates were reported by varying sources to range from 787 to 1300 per 100,000 live birthsxi . Nationwide, malaria was found to be responsible for 34% of child morbidity; most children suffer 3 to 6 episodes per year xii. Kibogora hospital reported 33% of pediatric deaths due to malaria xiii.Umucyo’s baseline data reports 2 week prevalence for fever among under fives at 74.8%. Nationwide, ARI was the second major cause of under five morbidity at 21%xiv . 100 Diarrhea and malnutrition were other leading causes of mortality, especially considering that they exacerbate other childhood infectionsxv. In Kibogora, dehydration from diarrhea was determined to be the 3rd leading cause of mortality among children under five years. Two week prevalence for diarrhea at baseline was 66.4%. According to the Rwanda Demographic Health Survey (DHS), 40% of Rwandese children were moderately or severely stunted (-<2SD), 27% were underweight for age (<-SD), and 9% suffered from wasting (-/=25 years 104 Figure 2: Literacy 73% 27% Literate Illiterate Figure 3: Occupation Farmer 98.3% Potter 0.3% Teacher 1.3% Occupation: As described in the situational analysis, Kibogora region is primarily agricultural. Most families farm crops for their own consumption and use any surplus to sell or trade at the market. Because most respondents farm in order to provide for their families, it is clear that child health and feeding practices are dependent on numerous agricultural conditions and factors. Breastfeeding and child feeding practices may be affected by seasonal crop schedules, as mothers or primary care takers may spend long hours outside the home tending to fields. Rapid referral to a health facility for sick children may also be influenced by such demands, or by crop yields, as they provide many families their sole source of income. 105 Figure 4: Gestational Status 10.3% 89.7% Pregnant Not Pregnant Figure 5: Distribution of Children by Age 31.0% 29.6% 12.5% 16.8% 10.0% 0.0% 10.0% 20.0% 30.0% 40.0% 0-11 months 12-23 months 24-35 months 36-47 months 48-59 months 106 Figure 6: Distribution of Children by Sex 44.6% 55.4% 0.0% 10.0% 20.0% 30.0% 40.0% 50.0% 60.0% Male Female Number of Children per Household 2 22 157 119 1 child 39.7% 2 children 52.3% 3 children 7% 4 children 2% Number of Children per Household: It is significant that over half (52.3%) of the households have at least 2 children under five years old. The need for child survival activities in order to ensure the health and well being within these families is evident. 3.1 Results for Intervention Indicators For Summary Table of Results for all Indicators, See Annex 3. 107 Table 1: IMCI Sick Child: During a discussion of preliminary KPC data at the feed back meeting following the KPC exercise, the KHD committee and Umucyo staff agreed to focus on maintaining the high level of knowledge regarding knowledge of danger signs for the remainder of the project, and to intensify efforts emphasizing increased feeding and fluids during recovery to prevent dehydration and malnutrition. Table 2: Malaria Prevention and rapid care seeking Malaria was determined the primary cause of morbidity and mortality for children under 5 years old by all sources consulted at the time of the baseline survey. The 2003 Kibogora Hospital annual report reported 33% of child illnesses to be confirmed malaria. KPC baseline survey data indicated a two- week prevalence for fever of 74.8% among children under-2 years. As shown in Table 2, prevention efforts revealed by the baseline survey were seriously inadequate. While respondents reported knowledge of fever as a danger sign for sick child (Table 1), rates for timely care seeking indicated either delay or the complete absence of treatment for children with suspected malaria. Behavior change regarding both prevention and rapid treatment indicated by midterm survey results, including from less than 5% (all indicators) to 66% (net use) and 31.2% (referral to health facility within 24h), constitute considerable achievements made within the first half of the CSP. Two week prevalence for fever among under- fives has decreased significantly, from 74.8% in December 2002 to 28.7% in August 2004). Indicator Baseline KPC% Midterm KPC% EOP Target 1. Number of mothers understanding at least two signs indicating a sick child needs treatment at health facility. 92.3% 98.0% NA 2. Proportion of mothers who give increased liquid to sick child. NA 56.8% 60% 3. Proportion of mothers who give increased feeding to sick children. 3.7% 42.4% 60% Indicator Baseline KPC% Midterm KPC% EOP Target 1. Children <23m who slept under mosquito netting within last 24h. 3.0% 66.0% 50% 2. Pregnant women who slept under mosquito net within last 24h. 3.5% 64.5% 50% 3. Mothers who take child <23m with suspected malaria to health facility within 24h of fever. 3.7% 31.2% 50% 4. Fever: 2 week prevalence among under- fives. 74.8% 28.7% NA 108 Table 3: Diarrhea and Hygiene: Since baseline levels surpassed the original target of 75% for caretakers reporting knowledge of danger signs indicating dehydration, Umucyo increased the indicator from “know 2 signs” to “know 3 signs”. Midterm data reveals that the new target was also surpassed. In addition, 2 week prevalence for diarrhea among under-fives decreased from 66.4% at baseline to 26.7% (Midterm KPC). Umucyo and KHD staff recognize the importance of maintaining results achieved thus far, and intend to focus efforts on improving practices related to home care (increasing fluids, etc.). The project will also focus review of this intervention in areas were the project has identified need for continued support, such as the Hanika Health Center area. Table 4: Nutrition * See discussion on recording coverage for immunization and Vitamin A, p.17 ** Indicator for hearth requires growth monitoring of children who have completed Hearth rehabilitation sessions. Midterm KPC data provides first measurement. Indicator Baseline KPC% Midterm KPC% EOP Target 1. Proportion of mothers who wash hands with soap or ash before feeding child. 33.7% 83.75 NA 2. Proportion of mothers with children from 0-23 months who increase fluids for a child with diarrhea 31% 66.7% 50% 3 .Proportion of mothers who know the danger signs of diarrhea. 83% (2 signs) 77.7% (3 signs) 75% (3 signs) 4. Diarrhea: 2 week prevalence among under- fives. 66.4% 26.7% NA Indicator Baseline KPC% Midterm KPC% EOP Target 8. Children 6-23 who have received at least 1 dose of Vitamin A per year (evidenced by card) 33.4% 15.4% * 80% 1. Nutritional status of children 0-23 months: Children with adequate weight for age ( >-2 SD). 84.1% 71.7% NA 2. Nutritional status of children 0-23 months: Children not maintaining adequate weight for age (<-2 SD) 15.9% 28.3% NA 3. Hearth: Children who maintain adequate or catch up growth after 1st Hearth session. NA 53.3%** 80% 4. Exclusive breastfeeding for children 0-6 months (as reported for last 24h) 60.3% 77.0% 75% 5. Mothers who initiate breastfeeding within 2 hours after delivery. 37.7% 71.3% NA 109 Conditions within the target community that contribute to malnutrition are perhaps the most significant obstacle that the Umucyo project currently faces. While rates of malnutrition have always existed in KHD (15.9% at baseline), both KPC midterm results and data from District sources indicate a recent increase in inadequate weight for age (KHD records). Inquires within the communities reveal a significant increase in cost of living since 2002. For instance, the cost of 1 kilo of beans has doubled from 100RF to 200RF, palm oil has more than doubled (200RF to 500RF) and soya (butter product mixed with small fish) has tripled from 100-300R. In addition to increasing food costs, national efforts to preserve Nyungwe forest now prevent families within Gatare District from farming land in areas surrounding the forest. As competition for land already exists in densely populated Rwanda, the combination of increasing costs, limited land availability and the growing number of widows, orphans and families suffering the effects of AIDS, has resulted in deteriorating nutritional status within Kibogora. In addition, World Food Program support for nutritional rehabilitation within Health Centers ended in 2003, leaving Health Center staff without adequate resources to treat the increasing number of severely malnourished children. These conditions continued to intensify at the same time that Umucyo began to introduce a nutrition intervention, which largely focuses on improving knowledge and feeding practices among caretakers. Other important components of the program include training mothers in growth monitoring and to recognize symptoms of malnutrition as a serious threat to health and child development. Prevention and rehabilitation within homes and communities for children who are not in a critical state is facilitated using the Hearth methodology (see Annex 4 ). While results from these efforts regarding improved breastfeeding practices and increased awareness among caretakers are encouraging , adequate weight for age among children under five has actually decreased. As indicated in Table 4, just over half (53%) of children maintained weight two months after completing the first Hearth session. Mothers report valuing the hearth sessions, and rates for mothers and children who attended the entire first session remained above 89%. Some groups have actually initiated extra Hearth meetings in addition to those assisted by Umucyo. However, the economic constrains that communities in Kibogora are facing continue to prevent children and families from having enough to eat. While mothers can identify which nutrient rich foods to add to their child’s daily porridge and report knowing that they should increase the number of daily feedings, families without land to produce food or a crop to trade at the market, or any other source of income, find it increasingly difficult to meet their daily nutritional needs. Mother groups who continue Hearth meetings report it helpful and cost effective to regularly compile ingredients and cook an enriched meal for their children together. Such collaborative feedings allow each mother to be responsible for procuring only one ingredient to contribute to the group, were as if she were to cook alone she would need to obtain a variety. Some groups have even initiated a small fund to provide the group with nutrient rich supplements, such as palm oil and fish powder, and to contribute to the cost of Vermox for de-worming. Umucyo will continue to encourage these efforts by sending promoters to support the meetings and mothers’ associations that pool resources to pay for de-worming. Umucyo is currently exploring possible 110 ways to link with an income generation project or agriculture initiative in order to address these food security concerns that contribute to malnutrition. Table 5: Immunization Indicator Baseline KPC% Midterm KPC% EOP Target Immunization 1. Complete coverage among children <12 months. 47.1% 85.5% 75% 3. Maternal TT: mothers who received at least 2 doses before birth of last child as evidenced by card. 43.8% 27.7% 50%(2d) 4. Maternal TT: mothers who received at least 2 doses as reported by the mother. NA. 54.3% NA The KHD acknowledges inconsistent recording of vaccines or Vitamin A doses on the vaccination card. Often EPI and Vitamin A campaigns are understaffed and therefore personnel lack the time to add records to each person’s card, or have fallen out of the habit of doing so. This lack of consistent recording is evident in the discrepancies noted between District records and other data sources. For example, the Health Centers offer maternal TT as part of all prenatal visits. The number of women recorded for prenatal visits at each Health Center suggests maternal TT coverage rate to be range from 55.2 % to 94.7% (KHD Records). However, the percentage evidenced by card during the midterm KPC is only 27.7%, which is less than the baseline. Because such inconsistencies are known to exist, Umucyo added a question to the midterm survey asking respondents to recall immunizations received. While the project acknowledges problems with validity in self reporting, it remains telling that the level of immunization coverage reported by mothers (54.3%) corresponds more closely with District records for injections given as part of antenatal care (55.2-94%). In order to improve consistency in vaccine records, the KHD has agreed to allow an Umucyo staff member to assist in updating cards at the EPI sessions. Not only will this improve reliability of records, it will also provide another opportunity for Umucyo to encourage community mobilization for immunization compliance. In order to improve Vitamin A coverage and recording, Umucyo is currently negotiating with KHD to distribute the supplement at EPI sessions, assisted by CSP staff. The KHD has agreed to conduct training for CSP staff this fall in order to allow for Vitamin A distribution to occur at EPI sites, in addition to the national campaigns. 111 Table 6: Reproductive Health and HIV/AIDS Midterm results indicate steady progress toward end of project objectives for indicators concerning safe delivery, including emergency transport plans and delivery with a trained providers. The apparent shift from home delivery to delivery at health facility is also encouraging. The addition of a new Health Center to be completed in Karambi region of Gatare District this year should enable more women in western KHD to access care with a trained provider, both for antenatal care and delivery. Umucyo’s results regarding HIV/AIDS are encouraging even as the virus continues to threaten health, family structure and economic security in Kibogora region. Midterm measurement of increased knowledge and decreased stigma indicate that their strategies are effective. Umucyo recognizes that the AIDS problem involves and affects all people within any given population and so in addition to the primary intervention efforts which target women and children, the CSP has sought to expand educational efforts to include men and youth, and to promote compassion and home care for people living with AIDS (PLWA). To date, Umucyo has initiated 2 support groups for PLWA and 7 groups providing home care. Eighteen members of the PLWA support groups have been trained by Umucyo to speak about their experience at community gatherings, and at Anti AIDS club meetings for youth that the project also initiated and supports. So far there are 17 Anti AIDS clubs meeting regularly, and 89 facilitators trained by Umucyo. The project has also trained 30 facilitators and 229 teachers to educate and raise awareness within Kibogora’s schools. Umucyo provides the schools with a curriculum and the total number of students reached is approximately 2,615. In addition to prevention efforts through education and sensitivity training, Umucyo also supports and encourages Voluntary Counseling and Testing (VCT). Between July 2003 and July Reproductive Health Baseline KPC% Midterm KPC% EOP Target 1. Families with an emergency transport plan in place prior to last delivery (ambulance/ hammock). 24% 46.7% 70% 2. Women with permission to execute emergency transport plan prior to last delivery. 16.4% 42.7% 70% 3. Women who report completing at least 3 prenatal check ups during last pregnancy. NA 45.0% NA 4. Women who delivered last child at health facility w/ doctor, nurse, auxiliary nurse. 23.1% 35% 50% 5. Women who delivered last child w/ trained TBA 20.1% 13.3%* NA HIV/AIDS 6.Women who know at least 2 methods of HIV/AIDS prevention. 80% 91.3% NA 7. Women who know at least 2 symptoms of STIs 8. Women who report willingness to care for a person with AIDS in their own home. NA 96.7% 80% 112 2004, Umucyo matched contributions made for VCT, resulting in approximately 5696 community members tested and counseled. 4.0 Conclusion and Recommendations The Umucyo program has achieved significant results in a wide range of activities. Outstanding achievements beyond target objectives have been measured within the malaria intervention, in diarrhea and hygiene, in exclusive breastfeeding, HIV/AIDS awareness and community based response, and in immunization coverage. Obvious challenges exist in the area of malnutrition. In order to address socioeconomic barriers, the project will explore options for income generation or agriculture programs. Umucyo will review all interventions during the second half of the program with intense focus on further emphasizing nutrition knowledge, growth monitoring and improved feeding practices. In addition, the project will focus on further strengthening the hearth associations and involving CDCs and Health Centers in order to further capacitate community based response to malnutrition. Negotiations with the MOH District could result in CSP volunteer associations that are permitted to assist with growth monitoring, Vermox delivery and Vitamin A distribution at EPI sessions. Other areas of focus during the remainder of the CSP will be to organize TBA associations in order to streamline training and increase referral to Health Centers in the event of delivery complications. To further increase the availability of emergency transport, Umucyo intends to organize “hammock brigades” and encourage community members to collaborate in transporting pregnant women in time for safe delivery. Now that Umucyo has fully established operational procedures and revised the health training curriculum, the project looks forward to dedicating the remaining years to further strengthening community associates that will contribute to sustainability of Umucyo’s accomplishments post project. 113 ANNEX 1: PROGRAM OBJECTIVES FROM DETAILED IMPLEMENTATION PLAN Malaria (20% effort): Community-wide education in malaria prevention and treatment seeking behaviors; improved access to Insecticide Treated Nets (ITNs) and re-treatment. ÿ 50% of children with fever (suspected malaria) will be treated within 24 hours at health facility. ÿ 40% of children -2 SD). 84.1% 71.7% NA 2. Nutritional status of children 0-23 months: Children not maintaining adequate weight for age (<-2 SD). 15.9% 28.3% NA 3. Hearth: Children who maintain adequate or catch up growth after 1st Hearth session. NA 53.3%* 80%* 4. Exclusive breastfeeding for children 0-5 months (as reported for last 24h). 60.3% 83.3% 75% 5. Mothers who initiate breastfeeding within 1 hour after delivery. 37.7% 71.3% NA 6. Mothers who have prepared an appropriate weaning meal for a child 6-23 months (as reported within last 24h). 19.5% 50.4% 50% 7. Mothers feeding a child 0-23 months 5 or more times per day. NA 78.0% NA 8. Children 6-23 who have received at least 1 dose of Vitamin A per year (evidenced by card). 33.4% 15.4% 80% 117 ANNEX 3: KPC QUESTIONNAIRE World Relief Rwanda Umucyo CSP Midterm KPC Survey Questionnaire QUESTIONNAIRE NO |____| |____| NOM DE L’ENQUETEUR : ________________________________________ NOM DU SUPERVISEUR _________________________________ PROVINCE : ________________ DISTRICT ADMINISTRATIF : _____________ CENTRE DE SANTE : _________________ REGION : _________________ SECTEUR : _____________________ CELLULE :_________________ STATEMENT OF CONSENT Hello, my name is, I work with World Relief community health project……We are doing a study about the health of mothers and children.I would like to ask you some questions related to this subject. The information that we are collecting will help the Umucyo CSP and the Kibogora Health District better understand and plan for ways to address health challanges within this community. This interview will not take much time, approximately________minutes. All responses will be kept confidential. You have the right to terminate this interview at any time, or to choose not to answer any questions. However, we hope that you would be able to take just a short time to help in this effort. Do you have any questions ? (If not) May we begin ? Yes No If the person does not give permission to begin, thank them for their time and wish them a good day. If they agree to participate, ask if the respondant has a child 23 aged months or younger. If yes, proceed with questionaire. If no, explain that the questionairre is designed for mothers with children under 23 months, say thank you and wish them a good day. IDENTIFICATION What is your name?………………………………………………………. Who is the head of your household?………………………………………………….. Can you read? 1. Yes 2. No What is your occupation?…………………………………………………………... 118 This questionnaire targets mothers (or caretakers) of children less than 24 months of age. 1. (RC#1) RECORD INTERVIEW DATE DAY MONTH YEAR 2. (RC#2) “How old are you?” RECORD AGE OF RESPONDENT IN YEARS: ____ ____ 3. (CSP) “Are you pregnant now?” 1. YES 2. NO 4. (RC#3) “How many children living in this household are under age five?” ____ 5. (RC#4) “How many of those children are your biological children?” ____ 6. (RC#5) READ ONE OF THE FOLLOWING QUESTIONS BASED ON MOTHER’S RESPONSE TO Q.4: IF ONLY 1 CHILD UNDER FIVE: “What is the name, sex, and date of birth of that child?” IF MORE THAN 1 CHILD UNDER FIVE: “What are the names, sexes, and dates of birth of your two youngest children?” NAME SEX DATE OF BIRTH 1 1. MALE 2. FEMALE __ __ / __ __ / __ __ DD MM YY 2 1. MALE 2. FEMALE __ __ / __ __ / __ __ DD MM YY 3 1. MALE 2. FEMALE __ __ / __ __ / __ __ DD MM YY 4 1. MALE 2. FEMALE __ __ / __ __ / __ __ DD MM YY 119 Anthropometry 7. Has ( NAME) been weighed in the last four months? Specify: a. By card b. Verbal report 1. YES 2. NO 8. (RC#6) “May I weigh (NAME)?” 2. YES 3. NO‡ SKIP TO Q.10 9. (RC#7) IF MOTHER AGREES, WEIGH THE CHILD AND RECORD WEIGHT BELOW. RECORD TO THE NEAREST TENTH. ___ ___ . ___ KILOGRAMS Maternal and Newborn Care 10. (RC#8) “Before you gave birth to (NAME) did you receive an injection in the arm to prevent the baby from getting tetanus, that is, convulsions after birth?” 1. YES 2. NO ‡ SKIP TO Q.12 8. DON’T KNOW ‡ SKIP TO Q.12 11. (RC#9) “How many times did you receive such an injection?” (VERIFY WITH CARD AT HEALTH CENTER.) Specify: a. By card b. Verbal report 1. ONCE 2. TWICE 3. MORE THAN TWO TIMES 8. DON’T KNOW ALL SUBSEQUENT QUESTIONS PERTAIN TO THE YOUNGEST CHILD UNDER AGE TWO 120 12. (RC#10) “Now I would like to ask you about the time when you gave birth to (NAME). Who assisted you with (NAME’S) delivery?” A. DOCTOR B. NURSE (A-1 or A-2) (Assume ‘Nurse (A-1 or A-2) if in Hospital) C. AUXILIARY NURSE (A-3) (Assume ‘Auxiliary nurse’ if in Health Center) D. TRADITIONAL BIRTH ATTENDANT ______________________ (NAME) E. COMMUNITY HEALTH WORKER F. FAMILY MEMBER _______________________________________ (SPECIFY RELATIONSHIP TO RESPONDENT) G. OTHER (SPECIFY) ________________________________ Y. NO ONE 13. Before the birth of (NAME), did you go to Health Center for prenatal check-up? 1. YES 2. NO ‡ SKIP TO Q.15 8. DON’T KNOW ‡ SKIP TO Q.15 14. How many times did you go to Health Center for prenatal check-up? Specify: a. By card b. Verbal report 1. ONE TIME 2. TWO TIMES 3. THREE TIMES 4. FOUR TIMES BIRTH PLAN: 15. (CSP) 15a. “Before the birth of (NAME), did you have a plan in place to get to the Health Center in case you needed emergency medical care?” 1. YES 2. NO ‡ SKIP TO Q.16 121 15b. “What was that plan?” SPECIFY:_______________________________________ ________________________________________________ 15c. “If an emergency had occurred, could you have followed the plan straight away, or would you have needed to wait for permission from your husband or mother-in-law (or anyone else)?” 1. Already had permission in advance 2. Would have had to wait for permission 16. (CSP) (Not applicable for pregnant women) “What method are you currently using to delay or avoid getting pregnant ?” 1. YES 2. NO ®SKIP TO Q.18 17.(CSP) (Not applicable for pregnant women) “What is the main method you or your husband/partner are you using now to avoid/postpone getting pregnant?” A. BARRIER METHODS B. PILLS C. CONTRACEPTIVES INJECTIONS D. NORPLAN E. IUD F. TUBAL LIGATION G. VASECTOMY H. OTHER (SPECIFY) ________________________________ Breastfeeding and Nutrition 18.(RC#11) “Did you ever breastfeed (NAME)?” 1. YES 2. NO‡SKIP TO Q.20 122 19. (RC#12 AND CSP) “How long after birth did you first put (NAME) to the breast?” 1. IMMEDIATELY/WITHIN FIRST HOUR AFTER DELIVERY 2. FROM 1-2 HOURS 3. AFTER TWO HOURS 3.a. (CSP) WITHIN FIRST DAY 3.b. (CSP) AFTER FIRST DAY 20. How many times each day does (NAME) eat ? INCLUDE SNACKS & BREASTMILK 1. ONE TIME EACH DAY 2. TWO TIMES EACH DAY 3. THREE TIMES EACH DAY 4. FOUR TIMES EACH DAY 5. FIVE OR MORE EACH DAY 21.(RC#13) “I would like to ask you about the types of liquids and foods that (NAME) consumed yesterday during the day or at night. Did (NAME) have. . . READ EACH OF THE FOLLOWING AND PLACE A CHECK MARK IN THE BOX NEXT TO EACH ITEM CONSUMED. LIQUID/FOOD CONSUM ED IN LAST 24 HOURS? A Breastmilk? B Plain water? C Other liquids? (e.g. sorghum liquid) D Mashed, pureed, solid, or semi￾solid foods? SPECIFY RECIPE: ________________ ________________________ ________________________ ________________________ ________________________ ________________________ E Anything else? SPECIFY: ________________________ ________________________ ________________________ 123 Child Immunization CSP: Ask questions on vaccination both about card and verbal report. (The Rapid Catch only asks for one or the other but we want both.) 22. (RC#14) “Do you have a card where (NAME’S) vaccinations are written down?” IF ‘YES’ASK “May I see it please?” 1. YES, SEEN BY INTERVIEWER 2. NOT AVAILABLE (lost/misplaced, not in home)‡SKIP TO Q.24 3. NEVER HAD A CARD‡SKIP TO Q.24 8. DON’T KNOW‡SKIP TO Q.24 23. (RC#15) RECORD INFORMATION EXACTLY AS IT APPEARS ON (NAME’S) VACCINATION CARD. DAY MONTH YEAR BCG POLIO 0 POLIO 1 POLIO 2 POLIO 3 Pentavalent Pentavalent Pentavalent MEASLES VITAMIN A 124 24. (RC#16) 24. a) “Did (NAME) ever receive an injection to prevent measles?” (circle the number corresponding to the mother’s report) 1. YES 2. NO ‡SKIP TO Q.25 8. DON’T KNOW ‡SKIP TO Q.25 24. b) When did he/she receive the injection to prevent measles?” (circle the number corresponding to the mother’s report) 1. Before the first birthday 2. After the first birthday Malaria Prevention 25.(RC#17) 25.a) “Do you have any bed net in your house?” 1. YES 2. NO‡SKIP TO Q.28 25.b) Can I see it? 1. YES‡ RECORD YOUR FINDINGS: A. MN available B. MN absent 2. NO‡SKIP TO Q.28 26. (RC#18) “Who slept under a bednet last night?” CIRCLE ALL THAT APPLY. A. CHILD (NAME) B1. RESPONDENT (IF PREGNANT) B2. RESPONDENT (IF NOT PREGNANT) C. OTHER INDIVIDUAL(S) __________________________________ (SPECIFY AND NOTE IF PREGNANT) D. NO ONE 27. (RC#19) Was the bednet ever soaked or dipped in a liquid to repel mosquitoes or bugs in six last months ? 1. YES 2. NO 8. DON’T KNOW 125 Integrated Management of Childhood Illnesses (IMCI) 28. (RC#20) “Sometimes children get sick and need to receive care or treatment for illnesses. What are the signs of illness that would indicate your child needs treatment?” DO NOT PROMPT. CIRCLE ALL MENTIONED. A. DON’T KNOW B. LOOKS UNWELL OR NOT PLAYING NORMALLY C. NOT EATING OR DRINKING D. LETHARGIC OR DIFFICULT TO WAKE E. HIGH FEVER F. FAST OR DIFFICULT BREATHING G. VOMITS EVERYTHING H. CONVULSIONS I. OTHER ________________________________ (SPECIFY) J. OTHER ________________________________ (SPECIFY) K. OTHER ________________________________ (SPECIFY) 126 29. (CSP) “What are danger signs that a child with diarrhea needs medical treatment?” ASK MOTHER TO LIST AND CIRCLE ALL RESPONSES—DO NOT PROMPT a. FEVER b. EXTREME THIRST (DEHYDRATION) c. WON’T EAT OR DRINK NORMALLY d. VOMITS FREQUENTLY e. PASSES SEVERAL WATERY STOOLS IN 1-2 HOURS f. BLOOD IN STOOL g. PERSISTENT DIARRHEA (2 OR MORE WEEKS) h. OTHER (SPECIFY): ______________________________ i. OTHER (SPECIFY): ______________________________ 30. (RC#21) “Did (NAME) experience any of the following in the past two weeks?” READ CHOICES ALOUD AND CIRCLE ALL MENTIONED BY RESPONDENT. A. DIARRHEA B. BLOOD IN STOOL C. COUGH D. DIFFICULT BREATHING E. FAST BREATHING/SHORT, QUICK BREATHS F. FEVER G. MALARIA H. CONVULSIONS I. OTHER _________________________ (SPECIFY) J. OTHER _________________________ (SPECIFY) K. NONE OF THE ABOVE‡ SKIP TO Q.35 127 31. (RC#22) “When (NAME) was sick, was he/she offered less than usual to drink, about the same amount, or more than usual to drink?” 1. LESS THAN USUAL 2. SAME AMOUNT 3. MORE THAN USUAL 32. (RC#23) “When (NAME) was sick, was he/she offered less than usual to eat, about the same amount, or more than usual to eat?” 1. LESS THAN USUAL 2. SAME AMOUNT 3. MORE THAN USUAL ***IF CHILD DID NOT HAVE DIARRHEA, FEVER, OR MALARIA IN PAST TWO WEEKS, ‡SKIP TO Q.35 33. (CSP) IF CHILD HAD DIARRHEA ASK, “When (NAME) had diarrhea, was he/she offered less than usual to drink, about the same amount, or more than usual to drink?” OTHERWISE ‡SKIP TO NEXT Q. (IF CHILD ONLY HAD DIARRHEA IN LAST TWO WEEKS, INTERVIEWER CAN MARK ANSWER BASED ON RESPONSE TO Q.29 WITHOUT RE-ASKING ABOUT FLUID INTAKE.) a. LESS THAN USUAL b. SAME AMOUNT c. MORE THAN USUAL 34. (CSP) IF CHILD HAD FEVER OR MALARIA IN LAST TWO WEEKS ASK: 34a) “Did you seek treatment?” OTHERWISE ‡SKIP TO Q.35 a. YES (ANSWER 32b AND 32c) IN BOX BELOW) b. NO ‡SKIP TO Q.35 128 34b) “Where did you seek treatment?” CIRCLE ALL THAT APPLY 34c) “How long after the illness began did you seek treatment from (SOURCE IDENTIFIED IN 32b)?” 1. HEALTH CENTER OR DISPENSARY 1. SAME DAY 2. NEXT DAY 3. MORE THAN 24 HOURS LATER 4. N/A 2. PHARMACY 1. SAME DAY 2. NEXT DAY 3. MORE THAN 24 HOURS LATER 4. N/A 3. TRADITIONAL HEALER 1. SAME DAY 2. NEXT DAY 3. MORE THAN 24 HOURS LATER 4. N/A 4.OTHER (SPECIFY): 1. SAME DAY 2. NEXT DAY 3. MORE THAN 24 HOURS LATER 4. N/A HIV/AIDS 35. (RC#24) “Have you ever heard of an illness called AIDS?” 1. YES 2. NO‡ SKIP TO Q.42 36. (RC#25) “What can a person do to avoid getting AIDS or the virus that causes AIDS?” (CIRCLE ALL MENTIONED) A. NOTHING B. ABSTAIN FROM SEX C. USE CONDOMS D. LIMIT SEX TO ONE PARTNER/STAY FAITHFUL TO ONE PARTNER E. LIMIT NUMBER OF SEXUAL PARTNERS F. AVOID SEX WITH PROSTITUTES G. AVOID SEX WITH PERSONS WHO HAVE MANY PARTNERS H. AVOID INTERCOURSE WITH PERSONS OF THE SAME SEX I. AVOID SEX WITH PERSONS WHO INJECT DRUGS INTRAVENOUSLY J. AVOID BLOOD TRANSFUSIONS K. AVOID INJECTIONS 129 L. AVOID KISSING M. AVOID MOSQUITO BITES N. SEEK PROTECTION FROM TRADITIONAL HEALER O. AVOID SHARING RAZORS, BLADES W. OTHER________________________ (SPECIFY) X. OTHER________________________ (SPECIFY) Z. DON’T KNOW 37. Can the virus that causes AIDS be transmitted from a mother to a child? During pregnancy? 1. YES 2. NO 8. DON’T KNOW During delivery? 1. YES 2. NO 8. DON’T KNOW During breastfeeding ? 1. YES 2. NO 8. DON’T KNOW 38. If a mother is infected with the AIDS virus, is there any way to avoid transmission to the baby? 1. YES 2. NO 8. DON’T KNOW 39. Have you ever been tested for HIV/AIDS? 1. YES 2. NO 8. DON’T KNOW 40. If a relative of yours became sick with the AIDS virus would you be willing to care for him or her in your own household? 1. YES 2. NO 8. DON’T KNOW 41. Would you allow your child to play with a child who has the AIDS virus ? 1. YES 2. NO 8. DON’T KNOW 42. (KPC) Apart from AIDS, have you heard about other infections that can be transmitted through sexual contact? 1. YES 2. NO‡ SKIP TO Q.44 8. DON’T KNOW 130 43. KPC) “What signs and symptoms would lead you to think that a man or woman had such an infection?” RECORD ALL MENTIONED; PROBE: “Any others?” a. ABDOMINAL PAIN b. GENITAL DISCHARGE/DRIPPING c. BURNING PAIN ON URINATION d. REDNESS/INFLAMMATION IN GENITAL AREA e. SWELLING IN GENITAL AREA f. GENITAL SORES/ULCERS g. GENITAL WARTS h. BLOOD IN URINE i. LOSS OF WEIGHT j. IMPOTENCE (IN MEN) k. INABILITY TO GIVE BIRTH (IN WOMEN) l. NO SYMPTOMS m. OTHER (SPECIFY): __________________________ n. OTHER (SPECIFY): __________________________ Hand-washing Practices 44.Before we end, I’d like to ask two more questions. When do you wash your hands with soap/ash? DO NOT PROMPT. CIRCLE ALL MENTIONED. A. NEVER B. BEFORE FOOD PREPARATION C. BEFORE FEEDING CHILDREN D. AFTER DEFECATION E. AFTER ATTENDING TO A CHILD WHO HAS DEFECATED X. OTHER ___________________________ (SPECIFY) Source of Knowledge: 45.Did you listen to a health message in last months ? 1. YES 2. NO 8. DON’T KNOW 131 46. From where did you get the health message? A. HEALTH FACILITY B. CHURCH C. COMMUNITY VOLUNTEERS D. RADIO E. OTHER___________________________ (SPECIFY) Ask the mother if she has other comments or questions to ask you. If not, thank her for the time and responses and say goodbye. 132 ANNEX 4: HEARTH METHODOLOGY Overview of Hearth (excerpted from PVO CS Grants Program Technical Reference Materials, p. 12.) “One relatively new approach to nutrition rehabilitation which has been working very well and takes a holistic approach to nutrition is the Hearth nutrition model. The model involves mothers, families, and neighborhoods in rehabilitating their own malnourished children by using local food and know-how. The goal of this approach is to not only rehabilitate the participating children but also reduce the prevalence of childhood malnutrition in the community and to energize the mothers and community to take broader, sustained action against malnutrition and poor health. The Hearth intervention takes place in the context of growth monitoring and counseling and micronutrient supplementation. In the early 1990s, the Hearth approach was initiated in Bangladesh by World Relief Corporation and the Christian Service Society as part of a PVO child survival program….” World Relief’s approach to Hearth in Kibogora Health District, Rwanda: CSP Promoters and volunteers identify poor mothers who have well-nourished children, and analyze their feeding practices. The Project Manager then designs the menu for rehabilitation (a snack + enriched porridge, and designs the education program around these locally sustainable behaviors. Volunteers and project staff identify malnourished children (< 70% weight for age) 0- 3 years of age at an outreach GMC session specially designed to identify children in need, and invite mothers to participate in the month-long Hearth program. It takes 2 weeks of supplemental feeding for children to experience “brightening” that mothers can attribute to food, and according to behavior change theory, it takes about 21 days of practicing a new behavior to fully adopt it. Hearth has the twofold purpose of changing a mother’s behavior and rehabilitating her child. CSP promoters and volunteers conduct 2 weeks of supervised instruction/practice in the community at a volunteer’s home (6 days per week for 2 weeks). Then for 2 weeks afterwards, volunteers visit mothers feeding their own children at home. Mothers must participate actively in Hearth. They must bring palm oil, fish powder, tomatoes, or even water, firewood, or cooking pots if that is all they can afford. Children are weighed at the beginning and at the end of the month; that weight is recorded on the Hearth register with the date. If a child still does not achieve at least adequate growth at the end of the month-long Hearth cycle, they can participate in the next Hearth. The Promoters and lead volunteers (Mammas Lumieres) track children’s growth on Hearth registers at weighing/reinforcement sessions. Volunteers also follow-up with Hearth children after Hearth. Within the Umucyo CS program, many groups of mothers have continued meeting together after the official Hearth sessions, in order to support each other in growth monitoring activities, and to continue pooling resources to prepare fortified snacks for their children. Some groups have even begun small saving programs, were members contribute small amounts of money to the group “nutrition” fund, enabling the group to procure palm oil and fish supplements and to pay for deworming. This strategy allows even the poorest mothers to participate and feel supported by their neighbors, increasing the likelihood that new feeding practices and catch up growth will be sustained. 133 ANNEX 5: Ministry of Health Standards The Ministry of Health established standards to guide health workers to improve maternal health, child health, and family planning services. These standards direct the Health Care Team to ensure equity of services at all levels and facilitate the training and supervision of staff providing these services. Standards address: - preventive measures for children (e.g., improving vaccination coverage) - treatment of malnutrition among infants - prenatal counseling - family planning - follow-up of labor and delivery (childbirth) 1.4.1.1 Goals for Improving Preventive Measures for Children - to prevent and screen for malnutrition - to prevent TB, polio, diphtheria, tetanus, whooping cough, and measles through vaccination coverage - to prevent vitamin A deficiency related illnesses by coordinating Vitamin A distribution - to screen children for illnesses during Health Center visits - to screen children for signs of mental retardation - to educate mothers about meeting the health and nutrition needs of their children 1.4.1.2 Goals for Treating Malnutrition Among Infants - to screen for illnesses which lead to malnutrition and treating these underlying illnesses - to prevent vitamin deficiencies and anemia - to investigate the causes of malnutrition with parents/caretakers and recommend treatment - to educate women regarding the importance of maintaining good nutrition for their children and themselves - to treat children and mothers suffering from malnutrition 1.4.1.3 Goals for Providing Prenatal Counseling - to screen for pregnancies at risk and take appropriate interventions to reduce the risks - to screen for complications during pregnancy and intervene to reduce such complications - to decrease the incidence of tetanus in the mother and child through vaccination coverage - to educate mothers to take preventive measures to reduce complications during pregnancy, delivery, and care of the newborn - to raise the awareness of women regarding the importance of family planning 134 1.4.1.4 Goals for Family Planning - to provide the community with clear and complete information regarding available family planning methods - to recommend and prescribe family planning methods and interventions appropriate to the clients, considering any contraindications - to skillfully administer the appropriate family planning intervention - to encourage the follow-up of clients to seek care to determine the clients clinical history and laboratory test results if any complaints/complications arise - to ensure that adequate services are provided to infertile couples 1.4.1.5 Goals for Ensuring Adequate Follow-up of Labor and Delivery - to provide early screening for risky pregnancies and take appropriate interventions - to provide early screening for any problems encountered immediately post-partum - to ensure the best possible conditions and environment for delivery - to convey the importance of family planning and preventive services available for children (well child checks, vaccination, and nutrition) i Demographic Health Survey for Rwanda 2000 ii Demographic Health Survey for Rwanda 2000, KPC Baseline Survey iii Demographic Health Survey for Rwanda 2000 iv Demographic Health Survey for Rwanda 2000 v Demographic Health Survey for Rwanda 2000 vi World Relief Rwanda Detailed Implementation Plan, 2002. vii UN AIDS Fact Sheet, www.unaids.org viii Demographic Health Survey for Rwanda 2000 ix UN AIDS Fact Sheet, www.unaids.org x Demographic Health Survey for Rwanda 2000 xi Kibogora Health District Records, DHS 2000 xii Demographic Health Survey for Rwanda 2000 xiii 2003 Kibogora Hospital Annual Report xiv Demographic Health Survey for Rwanda 2000 xv Demographic Health Survey for Rwanda 2000 xvi KHD records, World Relief Rwanda Detailed Implementation Plan xvii National Ministry of Health, WRR Detailed Implementation Plan