Child Survival 18-Guinea Final Evaluation Report Cost Extension of Cooperative Agreement No. FAO-A-00-98-000024-00 Community Health Initiative for the Districts of Kouroussa and Mandiana Guinea 1 October 2002 - 30 September 2006 Dr. Eric Sarriot Evaluation Team Leader Contact Person: Eric Swedberg MPH Child Survival Advisor December 2006 Acknowledgements Thanks are due to the departmental and regional health teams in Kankan, Mandiana and Kouroussa for their active participation in the evaluation. All the members of the evaluation team, beginning with the Save the Children team and its partnering NGOs GAAPE and AJVDM, put in a great amount of work during the project and its evaluation. The CS-18 project is financed by the United States Agency for International Development under the reference: Cost Extension of Cooperative Agreement No. FAO-A-00-98-000024-00. CS-18 Guinea, Final Evaluation, Save the Children, December 2006 2 Table of Contents Page Table of Contents…………………………………………………………………… 3 Acronyms and Terms……………………………………………………………….. 4 A. Executive Summary…………………………………………………………….. 6 B. Assessment of the Results and Project Impact…………………………………... 9 1. Introduction…………………………………………………………………….. 9 2. Graphic Summary……………………………………………………………… 9 3. Results: Technical Approaches………………………………………………… 11 4. Results: Cross-Cutting Approaches……………………………………………. 36 C. Project Management…………………………………………………………….. 45 D. Other Topics Identified by the Evaluation Team……………………………….. 46 E. Dissemination of Lessons Learned and Results………………………………… 46 F. Results Highlights………………………………………………………………. 47 G. Conclusions and Recommendations……………………………………………. 47 1. General Recommendations Made by the Evaluation Team…………………… 47 2. Specific Recommendations Made by the Consultant………………………….. 48 H. List of Annexes…………………………………………………………………. 50 Annexes 1. Excerpts from the DIP with Baseline Results………………………………. 2. Evaluation Team Members…………………………………………………. 3. Final Evaluation Methodology……………………………………………… 4. List of People Contacted……………………………………………………. 5. List of Participants in the Final Evaluation Debriefing…………………….. 6. Training Plan for the Health District of Mandiana…………………………. 7. Training Plan for the Health District of Kouroussa………………………… 8. Analysis of Deaths Avoided…………………………….………………….. 9. Interventions with the Greatest Estimated Impact…………………………. 10. Final Evaluation Photographs………………………………………………. 11. Explanation of the Index “Percent of Deficit Filled”………………………. 12. Summary of Project Progress………………………………………………. 13. Results Highlight…………………………………………………………… 14. Updated CSHGP Data Form……………………………………………….. 53 60 61 62 63 64 66 68 89 70 73 74 76 77 CS-18 Guinea, Final Evaluation, Save the Children, December 2006 3 Glossary of Acronyms and Terms AJVDM Association des Jeunes Volontaires pour le Développement de Mandiana – the Association of Young Volunteers of Mandiana AVHC Association of Village Health Committees BCG Tuberculosis Vaccine C-IMCI Community-Integrated Management of Childhood Illness CBDA Community-Based Distribution Agent CHW Community Health Worker COGES Health Facility Management Committee CSHGP Child Survival and Health Grants Program CS Child Survival CSTS Child Survival Technical Support DHS Demographic and Health Survey DIP Detailed Implementation Plan DPS District Health Office DPT Diphtheria-Pertussis-Tetanus Immunization DRS MOH Regional Director EBF Exclusive Breastfeeding EPI Expanded Program on Immunization FE Final Evaluation FHI Family Health International FP Family Planning GAAPE Kouroussa NGO- Groupe d’Appui à l’Auto Promotion Paysanne et à Protection de l’Environnement – the Support Group for the Self Promotion of the Land and the Protection of the Environment HC Health Center HIS Health Information System HIV/AIDS Human Immuno-Deficiency Virus/Acquired Immune Deficiency Syndrome HW Health Worker IMCI Integrated Management of Child Illness IUD Intra-uterine Device JHU Johns Hopkins University KPC Knowledge, Practice, and Coverage Survey M&E Monitoring and Evaluation MCH Maternal and Child Health MMR Maternal Mortality Ratio MOH Ministry of Health MSH Management Sciences for Health MTE Midterm Evaluation MURIGA Community-funded revolving emergency health loan fund NGO Non-Governmental Organization NIDS National Immunization Days OR Operations Research CS-18 Guinea, Final Evaluation, Save the Children, December 2006 4 ORS Oral Rehydration Solution PLG Program Learning Group PRISM Pour Renforcer les Interventions en Sante Reproductive et MST/SIDA-MSH Regional Health Project-Bilateral USAID project led by MSH PSI Population Services International PVO Private Voluntary Organization RH Reproductive Health SBA Skilled Birth Attendant SC Save the Children Federation, Inc. (US) STI Sexually Transmitted Infection TA Technical Assistance TBA Traditional Birth Attendant TT Tetanus Toxoid UNICEF United Nations Children's Fund USAID United States Agency for International Development VCT Voluntary Counseling and Testing VHC Village Health Committee WRA Women of Reproductive Age CS-18 Guinea, Final Evaluation, Save the Children, December 2006 5 A. Executive Summary CS-18, ISCOM (The Community Health Initiative for the Districts of Kouroussa and Mandiana Guinea; “Initiative pour la Santé Communautaire”) was implemented by Save the Children (SC) through funding from USAID/Washington (Office of Global Health; Child Survival and Health Grants Program—CSHGP) from October 2002 to September 2006 (cycle CS-18 of CSHGP grants). This project followed two funding cycles: 1) From 1997 to 1998 (CS-12), SC performed baseline studies in the Mandiana District. 2) This was followed by a child survival project (CS-14) in the Mandiana District, from 1998 to 2002. CS-18 expanded its interventions to the entire district of Mandiana and, in the case of certain interventions, to the district of Kouroussa. The human geography of the two districts includes 527 villages, with a total population of 393,060 inhabitants; 85,402 of whom are children under the age of five and 106,753 are women of reproductive age (WRA). When the project was launched, the under-five mortality rate was estimated at 222 per 1,000 live births, and the maternal mortality rate was estimated at 528 per 1,000,000 live births. The project was designed to reduce the principle causes of these mortality rates, by addressing the following factors: 1) low utilization of health facilities by the population, 2) weak health information systems, and 3) great difficulties in the provision of quality health services necessary to improve the health indicators in both districts (and particularly in the district of Kouroussa, which had not benefited from the previous material and human investments that led to the success of “The Mandiana Model”1 ). CS-18 focused on improving these indicators through five principle technical interventions: Maternal and Newborn Health (commonly called Maternal and Child Health (MCH) in the project’s documentation); family planning; nutrition and micronutrients; immunization; and HIV/AIDS prevention. The project design was unique as it established the role of the SC teams as that of a catalyst and supervisor of technical interventions in health centers and at the community level. These interventions were carried out directly by the District Health Offices (DPS: “Directions Préfectorales de la Santé”) and two local NGOs which were just beginning to implement health services. These two NGOs were: 1) GAAPE (Groupe d’Appui à l’Auto Promotion Paysanne et à Protection de l’Environnement, The Support Group for the Self Promotion of the Land and the Protection of the Environment) in Kouroussa, and 2) AJVDM (Association des Jeunes Volontaires pour le Développement de Mandiana, The Association of Young Volunteers of Mandiana). These NGOs had no previous experience in health, but had a recognized presence and involvement in both districts at the community level. 1 See the 2002 final evaluation report for CS-14, and the investigation done by the MSH PRISM Project in 2003. CS-18 Guinea, Final Evaluation, Save the Children, December 2006 6 The project goals were to: ƒ Increase utilization of services by the communities through the development of community￾based services; ƒ Promote appropriate health practices for both mother and child at the household and community levels; ƒ Develop the capacity of partnering NGOs and further develop the capacity of the DPS. Two of the key channels identified at the start of the project to implement this strategy were the Village Health Committees (VHC), and the MURIGAs (mutual village funds for the treatment of obstetrical emergencies, and later on for other child health emergencies). The final evaluation took place in September 2006 and showed significant success in several areas, including: ƒ Perinatality: increase of prenatal consultations and of trained birth attendant (TBA) coverage, a doubling of awareness of danger signs during pregnancy and delivery, and of Vitamin A during postpartum (from 45% to 70%); ƒ The project had an undocumented impact on the Maternal Mortality Ratio (MMR), and clearly demonstrated an increase of deliveries assisted by a TBA and a skilled birth attendant (SBA) (from 64% to 99% for both categories combined); ƒ CS-18 established a well-functioning community health fund system (MURIGAs) where none previously existed, to manage obstetrical emergencies in the community; ƒ Nutritional behaviors (breastfeeding, complimentary feeding) improved significantly and under-nutrition indicators were reduced to close to 50% lower than the national average; ƒ Contraception utilization rates doubled over the course of the project, and unmet needs decreased by half, leading to 53% of women having their contraception needs satisfied; ƒ In terms of vaccination, the coverage rate for all antigens doubled, reaching 39%; ƒ Success was achieved on a smaller scale, mostly in terms of developing strategies such as the use of peer educators, rather than results, regarding the issue of HIV/AIDS, which was new for CS-18. ƒ Finally, further substantial benefits were achieved in non-targeted interventions, such as the treatment of respiratory infections through the Integrated Management of Childhood Illnesses (IMCI). An analysis of avoided mortality, based on the Lancet series, indicates that more than 600 child mortalities were averted in four years (Annex 8). This estimate does not take into account the deaths avoided by a regression in indicators that remained at the same level. This analysis does not demonstrate the significant development of capacities of individual organizations (such as the VHC, MURIGAs, local NGOs, Health Centers and DPS), or the functional link created between these institutions. CS-18 was designed to replicate the “model of Mandiana”. The final evaluation revealed that through the initial collaboration between the two districts, the original achievements in Mandiana were largely replicated and possibly even surpassed in Kouroussa. Over the course of the project, Mandiana, using about 1/3 of the initial human and material investment, faced CS-18 Guinea, Final Evaluation, Save the Children, December 2006 7 mitigated results: it improved the status of half of the targeted indicators, but saw a deterioration of the other half. CS-18 therefore provides elements of analysis, not only regarding the efficiency of a promising model for community health in Upper Guinea, but also regarding the dose-effect relationship between investments and technical assistance, and the sustainability of the achievements. As a result, it is clear that this model is highly effective and sustainable. Areas for further study should include exploration of potential levels of sustainability that can be achieved with reduced resources and for how long. Summary of Conclusions and Recommendations: Several recommendations can be found at the end of this report, however there is only one fundamental recommendation for all partners: CS-18 demonstrates a potentially viable model of an effective community health program in Upper Guinea. This model created a functional bond between the professionals and managers of health, and the community forces. In the middle of this model are the local NGOs whose goodwill was shown, and whose capacity was developed. All the partners, including the DPS and DRS, recognize the importance of these new partners for the achievement of project success. The five key parts of this recommendation include: 1) The MOH and district structures must quickly define a viable model to maintain a viable community health system, with or without external support. Of course the level of realization will vary depending on the type of support received, but the national MOH needs to define the community health model that Guinea wants to develop. 2) In order to advance in this direction, it is recommended that the DPS/DRS organize, without delay, a roundtable to formalize the first stages of this process. Within this framework, there should be a discussion of the transfer of the project motorcycles to the local NGOs in order to contribute to the continuation of activities and achievements. 3) SC should seek to develop a new project in Upper Guinea with the objectives of studying the costs of obtaining sustainable progress. 4) At the same time, USAID and/or other potential donors should consider the potential of the CS-18 project. The next phase, Phase 3, could test the minimum investment needed to achieve minimum progress in the districts of Mandiana and Kouroussa, while also allowing for the extension of activities to a new zone. Such an investment would include agreements and contracts with civil society, and detailing of expected results, and the necessary support of the model from administrative authorities (districts, decentralization, social affairs). This proposal is an opportunity to combine efforts of good governorship with concrete health benefits for children under five and WRA at the community level2 . 5) Finally, stakeholders in Guinea, and especially the MOH, must quickly find a way to ensure adequate levels of essential drugs and medical supplies. It will not be useful to increase the demand (which will be high) for services, and to increase access to care, if the essential vaccines, contraceptives and drugs are not available. Solutions exist, but they must be put into place if Guinea wants a true primary health care system. 2 Examples of concrete results for good governorship include: deaths avoided, improved health indicators, reinforcement of community organization and the potential of the communities to require better services; and balancing of the public-NGOs roles in the district development. CS-18 Guinea, Final Evaluation, Save the Children, December 2006 8 B. Assessment of the Results and Project Impact 1. Introduction SC hired an international consultant to lead this final evaluation. The terms of reference required a highly participative approach for this evaluation, and included the following objectives: ƒ To analyze the level of achievement of CS-18 project objectives; ƒ To assess the strengths, weaknesses, and lessons learned; ƒ To examine the potential sustainability of project achievements and health benefits; ƒ To make recommendations for the continuing role of Save the Children in the health of mothers and children in Upper Guinea. The evaluation methodology and the FE team members are listed in Annexes 2 and 3. This evaluation was conducted between September 11 - 22, 2006. The results are presented below according to the format required by the USAID/CSHGP. Recommendations are presented following the discussion of the results by intervention. 2. Graphic Summary As per the FE guidelines, the key project indicators with measurable result for both periods (2002 and 2006), are presented in a single graphic summary. The information is presented in Figure 1 and is detailed in the following section. CS-18 Guinea, Final Evaluation, Save the Children, December 2006 9 Figure 1: Graphic Summary: Progression of the key indicators from 2002-2006 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Two prenatal consultations Knowledge of danger signs Referals for obstetrical complications Deliveries by trained attendants Deliveries by skilled attendants Contraceptive prevalence Complete vaccination by 12 mo. BCG vaccination BCG-measles drop-out DTP3 Measles TT 2 doses DTP drop-out Early BF Exclusive BF Complementary Feeding 6-9 mo. Postpartum Vit. A Use of iodized salt Knowledge of two means of HIV/AIDS prevention STD in last 12 months Treatment of STDs 2002 2006 CS-18 Guinea, Final Evaluation, Save the Children, December 2006 10 3. Results: Technical Approaches For each intervention, a table will present the objectives established in the Detailed Implementation Plan (DIP) approved by USAID. A general table is available in Annex 1. Results and more detailed comparisons follow and are accompanied by the evaluation team analyses. The results are presented overall for the project. A separate analysis of the two districts is presented in a next section. The results presented are from the KPC study, unless otherwise noted. a. Global View of CS-18 Implementation Strategies CS-18 began with the development of an action plan and annual project plans, the identification of NGO partners and an evaluation of their capacity, and the conduction of baseline studies, and the dissemination of their results. Field activities began five months before the end of the first year. Principle project activities included: ƒ Capacity assessment of the Village Health Committees (VHC); ƒ Conduction of numerous trainings (see “Cross-cutting Interventions/Training”), and exchange visits abroad and between the two districts; ƒ Support of DPS activities (National Immunization Days (NIDs), World AIDS Day and others); ƒ Operations research (for example on the use of the intra-uterine devices for family planning, and the creation of village pharmacies in pilot zones at Mandiana); and ƒ Daily field work by the NGOs community animators, with motorcycles and stipends provided by SC. The essential responsibilities of these animators were: - Supervision of the community health agents (CHW) and traditional birth attendants (TBA); - The conduct or support of community education sessions; - The microplanning of the activities with the health centers (HC), the VHC, the peer educators and others working in HIV/AIDS prevention and the distribution of contraceptives; - The support for the activities of MURIGAs (Community-funded revolving emergency health loan fund); - Creation of the Associations of VHC (AVHC); and - Relaunching and expansion of the Youth Listening Centers. A team of four SC employees were incorporated with each of the two DPS offices, and worked in close collaboration to support the DPS. The project sought synergies of intervention with the other stakeholders in the area3 and took an active part in the coordination of activities at the district, regional, and national levels (District Technical Committees of Health, Regional Technical Committees of Health, National Reviews of the Expanded Immunization Program (EPI) and Review of Primary Health Care). 3 HKI, EH, FHI, PRISM, UNICEF. CS-18 Guinea, Final Evaluation, Save the Children, December 2006 11 The number of villages reached by the project in Kouroussa increased after the midterm evaluation (See Table 1). Additionally, several end of project studies were conducted before the final evaluation which enhanced the analysis included in this report. Table 1: CS-18 Geographic Coverage by District Mandiana Kouroussa Urban community (head office DPS) 1 1 Sub-districts 11 11 Total number of villages in the district 133 289 Number of villages targeted by CS-18 133 (100%) as of May 2003 80 (28%) 42 as of May 2003 + 38 after September 2004 Number of VHCs supported 133 80 Number of Assoc. of VHC 12 12 Number of MURIGAs supported 133 80 Number of villages targeted for vaccination outreach visits 107 67 b. Environmental Constraints A certain number of environmental constraints had an impact on project achievements and must be noted: ƒ Repeated increases in the cost of fuel and primary food products; ƒ Stock-outs of vaccines, drugs and management tools; ƒ Elections: presidential (2003), communal and community (2005); ƒ An epidemic of meningitis in the district of Mandiana (2006); and ƒ General Strike of workers (2006). c. Maternal and Newborn Health The CS-18 project has addressed its objectives and obtained substantial results for this component. Certain limitations in achievement were observed due to the insufficient measurement of indicators and the inability to overcome some constraints, previously mentioned. CS-18 Guinea, Final Evaluation, Save the Children, December 2006 12 Table 2: Achievements by Objective Performance Indicators Objective Final result * Three antenatal consultations, one of which is in the 9th month of their last pregnancy [two antenatal consultations] 80% Nm [+] Mothers know at least two danger signs during pregnancy and the postpartum period 60% √ Use of a chemoprophylaxis (iron folate and chloroquine) during the last pregnancy 80% √ Last birth assisted by trained personnel 80% √ At least two postnatal visits including one in the first week of birth [two postnatal visits] 60% Nm + Complications referred in a medical facility during the last pregnancy 80% X * Key: √ Achieved objective + Proven or probable progress X Objective not obtained Nm Not Measured d. Use of Pre and Postnatal Consultation Services The very specific indicators defined in the DIP, were unfortunately not measured (“80% of the pregnant women carry out three antenatal consultations, one of which was in the 9th month at the time of their last pregnancy”; and “60% of the women carry out at least two postnatal consultations including one during the first week”). The analysis is therefore performed on specific elements. The use of antenatal consultations (at least two consultations) nevertheless increased from 68%) to 91% overall (particularly remarkable in Kouroussa). The use of three antenatal consultations is around 70% overall. This is a positive result compared with the national level DHS data (2005), which indicates that 73% of women had two antenatal consultations or more . The PRISM Project measured the achievement of three antenatal consultations, one of which was in the 9th month of pregnancy in 2003, and showed higher levels in Mandiana and Kouroussa (two traditionally resource-poor districts) compared to the regional capital district of Kankan (Figure 2). The number of women having at least two postnatal consultations in 2006 is equivalent in the two districts (58% overall), but was not measured in 2002. CS-18 Guinea, Final Evaluation, Save the Children, December 2006 13 Figure 2: Use of Prenatal Consultation Services in Three Districts of Upper Guinea (PRISM 2003) 60% 56% 44% 0% 20% 40% 60% 80% 3 Prenatal Care Visits - one in 9th month Mandiana Kouroussa Kankan i) Quality of pre and postnatal care Several indicators are used to measure the quality of ante and postnatal consultations, as well as the effectiveness of CS-18 efforts to reach women with essential messages for maternal health. These results are mixed, but several are encouraging: ƒ Ninety-two percent (92%) of the women received a prophylactic antimalarial treatment during their last pregnancy. However, the treatment was most often chloroquine rather than Fansidar, which is the drug of choice according to the national policy which is more than one year old, because of drug resistance to chloroquine. (Only 8% of the women received Fansidar.) ƒ The coverage rate for two doses of TT for pregnant women remained stable (73% in 2002 and 78% at the endline KPC survey). ƒ On the other hand, knowledge of danger signs during pregnancy by women, doubled in four years in the two districts (Figure 3). ƒ The coverage of women receiving a megadose of Vitamin A in the postpartum period increased by 45% to 70% between 2002 and 2006. CS-18 Guinea, Final Evaluation, Save the Children, December 2006 14 Figure 3: Knowledge of at Least Two Danger Signs During Pregnancy and Childbirth by Mothers of Children Less Than 24 Months Old (KPC Survey 2006) 32% 68% 0% 10% 20% 30% 40% 50% 60% 70% 80% 2002 2006 ii) Management of Uncomplicated Childbirth The project strategy for improving childbirth was based primarily on the training of traditional birth attendants (TBA). Such a strategy can sometimes reduce the proportion of deliveries by skilled health personnel, however, this was not the case in CS-18 (Figure 4). During CS-18, not only does the number of births assisted by trained TBAs increase by almost a third (from 43% to 56%), but one also observes an increase in the proportion of childbirths in health facilities which leads to: ƒ A doubling of the proportion of births assisted by skilled health personnel (from 19% to 43%)4 ; and ƒ The reduction in the number of unassisted births at home. These results are remarkable, especially since the KPC survey fully covered each of the two districts, whereas the coverage of project community activities reached only half of the villages in Kouroussa (see Table 1). The project received reports of villages without VHCs (not covered by CS-18) which received services from neighboring TBAs, either in the assistance with uncomplicated births, or in organizing obstetrical referrals to a health facility. 4 This result is similar to the PRISM project survey in 2003. CS-18 Guinea, Final Evaluation, Save the Children, December 2006 15 Figure 4: Personnel Assisting with the Most Recent Birth According to KPCs in 2002 and 2006 45% 56% 19% 43% 0% 20% 40% 60% 80% 100% 2002 2006 Trained TBA Skilled Personnel iii) Obstetrical Referrals The project worked hard to increase the number of obstetric referrals by encouraging the creation of MURIGAs and providing support to VHCs, in addition to training TBAs and health workers in emergency obstetrical and neonatal health. The results obtained provide a mixed view: ƒ The KPC shows a reduction in the percentage of women reporting having been referred during a pregnancy or a complicated birth (from 69% to 49% between 2002 and 2006). However, this indicator has many limits since it asks mothers to recognize and remember danger signs. In addition, the number of births in health centers has increased, therefore resulting in fewer referrals from the community to the health centers. ƒ The number of referrals supported by the MURIGAs has nevertheless increased each year, from 33 per annum in 2004 to 59 per annum in 20065 . ƒ The percentage of obstetrical referrals compared to the births recorded in the health information system (HIS) has remained stable (around 1.5%), except for a peak in the last two quarters 2004, which was not maintained (Figure 5). Overall, during the three last years, the HIS reports the following data: ƒ 29,671 recorded births; ƒ 67 maternal deaths (or 225 deaths for every 100,000 live births); ƒ 12,344 births assisted by TBAs (41%); ƒ Only 1,604 births (5%) were assisted by untrained TBAs; ƒ 10,632 births were assisted by health personnel (36% of deliveries); ƒ 5,091 births assisted in a health facility (17%); ƒ 617 referrals to the health centers and 314 to a hospital (3.1% of the births having been referred). 5 See the section on MURIGAs CS-18 Guinea, Final Evaluation, Save the Children, December 2006 16 Figure 5: Percentage of Births Recorded by the HIS Referred to a Health Center or a Hospital per Quarter 1.6% 1.7% 1.4% 4.0% 15.4% 1.7% 3.0% 1.8% 1.1% 1.8% 1.5% 0.0% 2.0% 4.0% 6.0% 8.0% 10.0% 12.0% 14.0% 16.0% 18.0% Dec-03 Feb-04 Apr-04 Jun-04 Aug-04 Oct-04 Dec-04 Feb-05 Apr-05 Jun-05 Aug-05 Oct-05 Dec-05 Feb-06 Apr-06 Jun-06 iv) Maternal Deaths Figure 6 shows the total number of maternal deaths recorded by the HIS over the last ten quarters. These figures cannot be considered as absolute from a statistical point of view, as there may be under-reporting of births and deaths. They do, however, present a very encouraging trend which should be tracked and confirmed over time. v) Discussion: Maternal and Newborn Care Intervention The evaluation team identified the following elements of success: ƒ Identification, training and equipping of TBAs; (107 TBAs trained and equipped); ƒ Training of Health Agents in emergency obstetrical and neonatal health (59 health workers trained); ƒ Communication and continuing education of WRA by the health agents and in particular by the VHC supported by the NGO animators; ƒ Support for the TT vaccination campaigns of pregnant women; ƒ The installation of MURIGAs and the decentralization of their management; ƒ Training of VHCs on danger signs during pregnancy and childbirth; ƒ Intensive follow-up of activities by trained NGO animators and health workers; and ƒ The management of the HIS by the district health authorities. CS-18 Guinea, Final Evaluation, Save the Children, December 2006 17 Figure 6: Number of Maternal Deaths Recorded by the HIS for Mandiana and Kouroussa per Quarter 12 4 6 8 6 10 4 6 2 18 14 14 6 8 4 4 4 2 0 5 10 15 20 25 30 35 Dec-02 Mar-03 Jun-03 Sep-03 Dec-03 Mar-04 Jun-04 Sep-04 Dec-04 Mar-05 Jun-05 Sep-05 Dec-05 Mar-06 Jun-06 Sep-06 Kouroussa Mandiana Among these success factors, some interventions are particularly critical for the replication of this model, including: ƒ The identification, training, and equipping of TBAs; ƒ The regular follow-up of activities on all levels, especially through local NGO animators; and ƒ Training on different levels, and the active involvement of community structures (VHCs) in communications about, and mobilization and management around maternal health issues. Several constraints restricted the achievement of project objectives. ƒ The first constraints are related to the environment: - The mobility and seasonal movement of women to and from the mining zones; - Distance to the medical facilities; - The high workload of women in the households; and - The remoteness of certain villages. ƒ Structural weaknesses of the health system: - Limited outreach visits and active case finding of pregnant women or women recently delivered; - Frequent breakdowns of logistics and fuel stock-outs at the health centers; - Stock-outs of drugs and management tools; and - Insufficient number of health personnel; ƒ Finally, the strategy of creating MURIGAs only covered part of Kouroussa District. Problems of over charging for services in the hospital were not completely solved. e. Family Planning The CS-18 Project obtained remarkable results in its family planning (FP) intervention, exceeding the objectives set and making substantial progress in increasing the accessibility and CS-18 Guinea, Final Evaluation, Save the Children, December 2006 18 use of FP services. However, the health system remains fragile as demonstrated by frequent stock-outs, which threaten these gains. Table 3: Achievements of FP Objectives Performance Indicators Objective Final result * % of mothers which do not wish to have another child in the two next years and who use a modern contraceptive method 50% √ Knowledge of three FP methods -- √ % of women living in a 5km radius of FP services -- √ % of health centers offering at least three FP methods -- √ Absence of stock-outs in the last four months -- √ * Key: √ Achieved objective + Proven or probable progress X Objective not obtained Nm Not Measured i) Knowledge and Use of Contraceptive Methods The use of modern FP methods by mothers not wanting another child in the next two years, doubled overall (from 24% to 51% against an objective of 50% - Figure 7). This progress is largely due to tremendous gains made in Kouroussa which multiplied this indicator by 7 (from 9% to 66%). The rate in Mandiana remained stable or progressed slightly (from 37% to 43%)6 . The use of a contraceptive method by mothers in the KPC study was 50%, as compared to the 2005 DHS in which only 9.1% of the women in union currently used a method of contraception. Almost all women (96%) in the two districts know at least three methods of contraception. Figure 7: Use of Modern FP 24% 51% 0% 10% 20% 30% 40% 50% 60% 2002 2006 In terms of impact, 73% of the women of Mandiana and Kouroussa, spaced their last two pregnancies by at least 24 months. According to the DHS study, 86% of the women in the area of Kankan had at least 24 months between pregnancies. There was no comparable measurement in the 2002 KPC study. 6 The difference observed for Mandiana from 37% to 43% is not statistically significant, but the overall change for both districts and Kouroussa is. CS-18 Guinea, Final Evaluation, Save the Children, December 2006 19 Figure 8: Comparison of the Contraceptive Prevalence of Mothers in CS-18 Districts and the National Level. (Mothers of children less than two years old in the KPC 2006 and mothers in the DHS 2005 survey). 49% 9% 0% 10% 20% 30% 40% 50% 60% Mand / Kourou 06 DHS 2005 ii) Availability and Access to FP Services Access to FP services is high (76%) within a radius of less than 5 km, in particular due to the provision of contraceptive pills by peer educators and the community-based providers who are now authorized to provide this method. The 24 centers supported by the project (12 in each prefecture) offer three methods of FP, but had many stock-outs in the last four months (Figure 9). CS-18 also conducted operations research (OR) in nine villages of Mandiana to increase the use of intra-uterine devices (IUD), by a combination of training health workers, mobilization, and dissemination of information at the community level. This OR was evaluated by FHI and included the insertion of 73 IUDs between January and June 2005, compared with only 13 during the previous period. Figure 9: Stock-outs of Contraceptives in the Last Four Months (N= 12 Centers of Health in Kouroussa and 12 in Mandiana) 10 6 16 2 6 8 0 5 10 15 20 Mandiana Kouroussa Total Stockouts No Stockouts CS-18 Guinea, Final Evaluation, Save the Children, December 2006 20 iii) Discussion: Family Planning The following factors have enabled these remarkable results: ƒ Training of health workers, community-based agents, and peer educators; ƒ The availability of oral contraceptives at the community level; ƒ Follow-up and supervision by the NGO partners and the health agents supported by the NGO animators (in micro-planning and logistical support); ƒ Effective involvement of the VHCs in social mobilization; ƒ Orientation and involvement of the religious and community leaders; and ƒ Installation of a buffer stock of contraceptive supplies to prepare for stock-outs. The primary constraints faced were: ƒ The reservations of certain religious leaders, and cultural attitudes which were sometimes expressed in a violent way against FP clients. Despite the progress made, this progress needs to be sustained. ƒ Repetitive stock-outs are an overall threat to maintaining the communities’ health. iv) Flexible Fund Questions USAID provided Flexible Fund support for some of the FP activities in this project. Flexible Funding reporting guidelines require that the following sections be addressed. (This information is provided throughout this FE report, however a synthesis is presented here.) (a) The project worked very closely with the USAID Mission in Guinea and its PRISM Project. The PRISM Project provided a buffer stock of contraceptives, but did not take responsibility for long-term, logistical support. In fact, the community approaches of CS-18 highlighted a latent demand for contraception and the capacity to increase this demand to satisfy the unmet FP needs. This more clearly revealed the frailty of the health system, and its ongoing management and logistics challenges related to contraceptives and essential drugs. (b) The project increased the use of the contraceptive methods and engaged the community in the reduction of cultural obstacles, which are important and sometimes resulted in domestic violence. The involvement of the VHC and the religious leaders was significant. (c) The FE did not focus on the quality of FP services. This was addressed by both the PRISM Project (common tools) and MOH authorities. Access to services was increased and the provision of contraceptive methods closer to the communities, helped improve the quality of these services. (d) Access was improved primarily because of the provision of oral contraceptives at the community level. However, stock-outs are frequent in health centers and call into question the sustainability of this improved access. (e) CS-18 led a pilot effort to offer IUDs as a contraceptive method. This pilot was successful and could be expanded. CS-18 Guinea, Final Evaluation, Save the Children, December 2006 21 (f) There is no indication of a lack of compliance with the Mexico City Policy or the Thiart Amendment. (g) The project showed the validity and effectiveness of involvement of local NGOs to implement community health activities, including FP. This was deeply appreciated and recognized by the medical authorities. f. Vaccination CS-18 supported the improvement of vaccination coverage in the two districts, despite extremely difficult conditions, including lack of fuel and vaccine stock-outs. Important progress was achieved, even though the (overly) ambitious objectives were not fully realized. Table 4: Vaccination Achievements by Objective Indicators of performance Objective7 Final result * Children 12-23 months old completely vaccinated before one year of age 80% X + Coverage of BCG 90% X + DTC3 before one year of age 80% X + Measles before one year of age 80% X + Outreach visits increased 85% X TT Vaccination8 80% X Drop-out BCG – Measles9 -- + Drop-out DPT1-DPT3 -- + * Key: √ Achieved objective + Proven or probable progress X Objective not obtained Nm Not Measured i) Vaccine Coverage Rate Figure 10, based on the KPC, shows the progress made in vaccination coverage for each antigen. On the whole, complete vaccination coverage doubled, increasing from 17% to 39%, despite numerous stock-outs. These solid results are supported by the HIS EPI data analysis, shown in Figure 11. It is important to note that coverage in Kouroussa caught up with coverage in Mandiana. This will be discussed later in this report. 7 The objectives of vaccination coverage set in the DIP were based on the denominator of children less than two years having a vaccination chart. The cover rate for this sub-group will be increasingly higher than for all children less than two years old. Even by using this measurement, the objectives set were not achieved by the project. In this report, we use the most strict measurement of coverage by antigen, i.e. “number of children of less than two years old with a vaccination by the antigen confirmed by card/a total number children of less than two years of the sample.” A child vaccinated at the time of a campaign, but whose vaccination is not noted in the notebook, thus is regarded as not vaccinated. This very conservative definition, is recognized internationally. 8 See section on Maternal and Newborn Health for TT, whose indicator remained constant. 9 The indicator of drop-out used by the project is that of BCG-Measles to conform to the Guinea national policy. The drop-out rate for DPT1-DPT3 is more commonly used internationally and will also be shown. CS-18 Guinea, Final Evaluation, Save the Children, December 2006 22 Figure 10: Vaccination Coverage by Antigen (KPC 2002 and 2006) 32% 32% 19% 19% 17% 52% 51% 44% 42% 39% 0% 10% 20% 30% 40% 50% 60% BCG DTP1 DTP3 Measles Complete 2002 2006 Figure 11: Vaccination Coverage According to the HIS in the Districts of Mandiana and Kouroussa (six-month periods 1 (January-June 2002) through period 9 (January-June 2006) 23% 34% 46% 64% 63% 74% 81% 84% 90% 64% 81% 64% 80% 86% 82% 90% 85% 92% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% 123456789 Kouroussa Mandiana Prior to the project, in 1999 the district BCG coverage levels were lower (31%) than the national level. Now, through the project, they have caught up with the national levels (both 52%). CS-18 Guinea, Final Evaluation, Save the Children, December 2006 23 Figure 12: Comparison of the BCG Coverage Between the CS-18 KPC Studies (Mandiana￾Kouroussa) and DHS Studies. Two periods of comparison: before the current project (1999 for the DHS and 2002 for CS-18) and at the end of the project (2005 for the DHS and 2006 for CS￾18) 32% 52% 46% 52% 0% 10% 20% 30% 40% 50% 60% Before ISCOM End of ISCOM Mandiana&Kouroussa National Coverage Figure 13 represents very important progress made in the vaccination drop-out rate, which was reduced by half or more over the course of the project. It is also an indicator of the improvement of the quality of care, and improvement made in the recording of health data. Figure 13: Vaccination Drop-out Rates 43% 41% 20% 14% 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% Drop-out BCG-Measles Drop-out DTP1-DTP3 2002 2006 ii) Support for Vaccination Outreach Visits The project actively supported outreach visits by health center staff, by encouraging microplanning, through monitoring, and on occasion, by providing logistical support when the HC transportation failed (through motor bikes borrowed from the local NGO animators). Nevertheless, it is necessary to note that project objectives for immunization were not fully reached, with only 54% of the programmed vaccination outreach visits carried out. CS-18 Guinea, Final Evaluation, Save the Children, December 2006 24 Table 5 shows the number of outreach visits made to the number of villages targeted by this strategy for each year. The planned number was reduced in Mandiana after 2004, to allow for an intensification of the efforts in Kouroussa. This was an effective strategy since Kouroussa reached and then exceeded the number of outreach visits in Mandiana (reaching almost three visits per village per annum in 2006, incomplete year), whereas the level was maintained in the last two years in Mandiana. Table 5: Planning and Achievement of Outreach Visits by Target Village in the Two Districts by Year Activities/Village 2003 2004 2005 2006 Programmed 2.2 5.4 2.7 2.7 Mandiana Actual 1.8 2.4 1.4 1.7 Programmed 1.8 4.3 3.6 4.3 Kouroussa Actual 1.6 1.0 1.5 2.9 iii) Discussion: Vaccination The FE team identified the following successes: ƒ The involvement of VHCs in vaccination activities; ƒ Outreach visits and active case findings of defaulters (mothers and children who get the first immunization, but do not return for the second or third dose) were conducted; ƒ Production and distribution of management tools to the DPS and HCs by the project; ƒ Reach Every District Campaigns organized at the national level with the support of UNICEF; and ƒ Personal efforts, for example the Districts Director of Kouroussa transported the vaccine herself from Conakry to Kouroussa to prevent vaccine stock-outs. Important constraints however, limited the achievement of objectives: ƒ The project and its partners could not conduct all of the outreach visits and support all of the active case finding efforts for vaccination drop-outs; and ƒ In addition to the socio-cultural constraints (for example movement to the mining zones, and feared side-effects of vaccination), the weaknesses of the health system continue to be the principle limiting factor (frequent breakdowns of logistics; frequent fuel stock￾outs, stock-outs of vaccines and management tools; lack of adequate numbers of health personnel). g. Nutrition and Micronutrients The project achieved important results in its nutritional intervention. These results certainly contributed to lowering the prevalence of malnutrition compared to national levels. This component also had an important impact on improving health and the reduction of child mortality. However, this component is also one of the most problematic in terms of sustainability, due to the cost of the strategies and the limited success of the growth monitoring and promotion strategy. Finally, while there was progress in Vitamin A supplementation, the iodized salt strategy was not successful. CS-18 Guinea, Final Evaluation, Save the Children, December 2006 25 Table 6: Achievements by Objective in Nutrition Performance Indicators Objective Final result * Early breastfeeding (1st hour) 40% √ Exclusive breastfeeding (0-6 months) 60% √ Appropriate complementary feeding (6-9 months) 70% X + Moderate malnutrition level (weight/age <2DS and >3DS) 15% √ Severe malnutrition -- √ Vitamin A coverage (6-59 months) 90% Nm Post-partum Vitamin A 60%10 √ Use of iodized salt -- X * Key: √ Achieved objective + Proven or probable progress X Objective not obtained Nm Not Measured i) Nutrition of the Young Child The percentage of newborns benefiting from breastfeeding within the first hour, as well as the percentage of children benefiting from exclusive breastfeeding (EBF) for up to six months, doubled during the life of the project (Figure 14). An increase in the percentage of children from 6 to 9 months old benefiting from complementary feeding also increased (to 52%), but did not reach the project goal of 70%). The level of EBF achieved (70%) represents the principle intervention contribution to preventing deaths (see Annex 8) and compares very favorably with the regional figures (50% in the PRISM, 2003 Survey, and 27% in the national DHS 2005). 10 The objective set was for two doses of Vitamin A, while the final indicator was one megadose of Vitamin A. CS-18 Guinea, Final Evaluation, Save the Children, December 2006 26 Figure 14: Breastfeeding and Complementary Feeding of the Young Child (KPC Surveys) 23% 36% 33% 48% 70% 52% 0% 10% 20% 30% 40% 50% 60% 70% 80% Early BF Exclusive BF 0 - 6 mo. Complementary Feeding 6-9 mo. 2002 2006 ii) Nutritional Monitoring and Growth Promotion of Children Efforts to improve the monitoring of children’s nutritional status achieved modest results. According to the KPC survey in 2006, only 48% of the children have at least one weighing noted on their health record. The project monitoring system indicates that the community weighing and growth promotion sessions in Mandiana as well as in Kouroussa, do not include more than 17% of the expected number of children targeted in the growth monitoring sessions. In one quarter, this percentage reached 40%. The NGO partners reported that the organization of community growth promotion and monitoring sessions is very hard. Mothers find it difficult to regularly attend growth monitoring and promotion sessions to their other daily activities. During the community weighings, the percentage of children in the red zone (severe malnutrition wt/age) oscillates around 2% (1% to 5%), and for the yellow zone (moderate malnutrition wt/age) it oscillates around 14% (8% to 16%). This overall stable situation hides differences between the two districts. The figures for Kouroussa were more worrisome at the beginning of the project and have now been reduced to the Mandiana levels. There was a downward trend, particularly for moderate malnutrition in Kouroussa, as Figure 15 indicates. CS-18 Guinea, Final Evaluation, Save the Children, December 2006 27 Figure 15: Percentages of Children in the Red Zone (severe malnutrition) and in the Yellow Zone (moderate malnutrition) During Community Weighings in the District of Kouroussa 2% 4% 5% 4% 5% 7% 6% 3% 2% 5% 40% 28% 23% 16% 12% 17% 20% 16% 13% 15% 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% Dec-03 Feb-04 Apr-04 Jun-04 Aug-04 Oct-04 Dec-04 Feb-05 Apr-05 Jun-05 Aug-05 Oct-05 Dec-05 Feb-06 Apr-06 Jun-06 % in red zone % in yellow zone iii) Supplementation of Vitamin A and Use of Iodized Salt The majority of postpartum women (70% compared to 16% at baseline) received a megadose of vitamin A. The availability of the Vitamin A capsules and the number of TBAs trained in their distribution, largely contributed to the increase in the coverage in the project districts. The coverage of Vitamin A supplementation of children 6 to 59 months old, unfortunately was not measured. Vitamin A supplementation is primarily done during the National Immunization Days (NIDs), but is rarely done routinely during visits to the health centers. The proportion of mothers who add iodized salt to meals did not increase between 2002 and 2006 (18%). This result remains very weak compared with the results of the last DHS where it is 43.9%. This low level is explained by the lack of iodized salt in the market and the rupture of the salt testing kits which allow households to know the content of iodine in their salt. The project did not specifically seek to address this. iv) Nutritional Status of Children There are no clear comparative measurements to evaluate the impact of the project on the nutritional status of children. One can nevertheless carry out comparisons with the data from the latest DHS. Figure 16 shows that 16% of the children in the project zones suffer from moderate or severe malnutrition; this is in comparison with a national average of 29% according to 2005 DHS. The project has in fact reached a rate of moderate malnutrition lower than the objective of 15% which had been set. CS-18 Guinea, Final Evaluation, Save the Children, December 2006 28 Figure 16: Rates of Malnutrition (ponderal deficit) Compared Between the CS-18 Project Zone (CPC 2006 investigation) and the National Indicators (EDS 2005) 4% 7% 12% 22% 0% 5% 10% 15% 20% 25% 30% 35% Mand/Kour National Moderate Malnut Severe Malnut v) Discussion: Nutrition and Micronutrients The elements which contributed to the improvement of the nutritional status of children in the project zones include: ƒ Growth monitoring and promotion sessions, and culinary demonstrations, in addition to education and counseling sessions; ƒ Training of nutritionists within each VHC and the organization of hearths when more than eight cases of severe malnutrition are detected among the children during community weighings; and ƒ The constant involvement of the local VHCs and the support of NGOs. Constraints faced include: ƒ Efforts to organize the growth promotion and monitoring sessions, did not result in an increase in the proportion of children served, and is not likely to be sustainable without project support; ƒ Resources necessary for conducting hearths are limited; and ƒ Absence of iodized salt and kits to test salt, as well as the absence of a specific strategy to improve this situation. h. HIV/AIDS Prevention The CS-18 project refocused its work on malaria (CS-14) to begin a new intervention addressing HIV/AIDS and STIs in CS-18. Many efforts were made to develop the necessary strategies, and interesting results were observed in terms of process and inputs, but not yet in terms of behavior change. CS-18 Guinea, Final Evaluation, Save the Children, December 2006 29 Table 7: Achievements by Objective in the Nutrition Component Performance Indicators Objective Final result * Use of condoms with non-regular partners (by mothers) 80% X NM Adolescents 15-24 years old who know that condoms can protect against HIV/AIDS 80% X Mothers who know at least two means of HIV/AIDS prevention 80% X + WRA who know that the condom can protect against HIV/AIDS 80% X + Four leaders per village (700) trained on the prevention of HIV/AIDS 700 √ Reduction in STIs reported by mothers in the last 12 months Between 3% and 10% X Treatment for STIs sought by women -- + * Key: √ Achieved objective + Proven or probable progress X Objective not obtained Nm Not Measured i) Knowledge and Condom Use The goal of 80% use of condoms with “non-regular” partners was overly ambitious for the project, considering the time allowed and the number of other project interventions. However, there was progress in terms of information are shown by the KPC studies, where the level of knowledge doubled in four years: ƒ The percentage of women who know two means of protection against HIV/AIDS increased from 29% (2002) to 61% (2006); and ƒ The recognition of the condom as a means of protection for women, increased from 21% to 46%. The use of a condom by mothers at the time of a sexual encounter with a non-regular partner was not measured in 2002 and its estimate in the 2006 KPC had to be rejected after detailed review by the evaluation team. It seems that there was confusion within the project on the nature of this objective, which is recorded in contradictory ways in project documents. The project also carried out a study of young people with access to the Youth Listening Centers, mainly in urban/semi-urban environments. This survey was not randomized, so the results can not be extrapolated to the general population of young people in the two districts. Of 220 young people questioned, 43% reported always using a condom, and 30% said they used a condom sometimes. The evaluation team also met with the groups of peer educators. At least once, an evaluation team member observed a peer educator in a rural area with a display of various contraceptive CS-18 Guinea, Final Evaluation, Save the Children, December 2006 30 methods that did not include condoms. In group discussions however, the peer educators were motivated and mobilized to promote protective behaviors (including the use of condoms) during sexual relations. In an anonymous rapid survey conducted during the focus groups, the great majority reported having used condoms in their last sexual encounter. ii) Sexually Transmittable Infections (STIs) One of CS-18’s objectives was the reduction of STIs reported by mothers during the KPC study. In Figure 17, there is an increase of 8%, to 14%, several problems arise in how this objective was defined. ƒ On one hand, women are often asymptomatic and are not aware if they have an STI; and ƒ If an intervention is effective in the detection of STIs and their treatment, there will be an increase in the rates of infection observed, as they are more likely to be detected and reported. Therefore, there will appear to be an increase in the incidence before a reduction is noted. The rates of infection do not necessarily increase, but rather more infections (which would otherwise go unnoticed) are detected. While inconclusive, this possibility is reinforced by the increase in the number of women informed of an infection and who sought treatment. This last indicator increased from 51% to 80% (Figure 17) and constitutes a very positive result of the project. Figure 17: Sexually Transmittable Infections (STI) Reported and Treated (KPC 2002, 2006). 8% 51% 14% 80% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% STD Reported STD Treated 2002 2006 iii) Discussion: HIV/AIDS and STIs The progress observed in this project component is due to the diversity of approaches used including, varied channels of communication (peer educators, radio, and listening centers) to reach the young people. The cultural obstacles for the prevention of HIV/AIDS are important. The project reached 700 community leaders, the majority of whom were religious leaders, and conducted necessary discussions on the prevention of AIDS and religious standards. Additionally, HC staff were trained on the syndromic treatment of STIs, and community structures (youth listening centers and VHCs) were included in the communication and education efforts. CS-18 Guinea, Final Evaluation, Save the Children, December 2006 31 The strengths of this intervention identified by the evaluation team are: ƒ Identification and training of motivated peer educators; ƒ Decentralization of youth listening center activities; ƒ Communications targeting multiple channels; ƒ Advocacy meetings and orientation of key leaders; ƒ Training of the health workers on the syndromic management of STIs; and ƒ Parallel activities undertaken by PSI11. Despite these strengths, the goals for this intervention were not fully achieved. The FE team identified the following factors that limited the achievement of project objectives: ƒ Stock-outs of drugs for the syndromic management of STIs; ƒ The difficulty of conducting partner notification of patients presenting with STIs; ƒ Cultural barriers which were only begun to be addressed; ƒ The project duration (four years with major programmatic changes) was too short given the complexity of interventions (MCH, FP and HIV/AIDS). Based on the degree of the HIV/AIDS epidemic in Guinea, the results obtained by the project, and the time it takes to change sexual behaviors, the inclusion of this intervention in this project remains valid. However it is not surprising that the results on this new component are modest; ƒ The difficulty in addressing environmental constraints: - Displacements of populations to the mining zones; and - The absence of a strategy to reach specific target groups (“free women”, drivers, and military personnel). ƒ The broader absence of AIDS interventions (nonexistent or unknown sites for testing and antiretroviral treatment) which can have an effect of creating demand for prevention activities. iv) Disaggregation of the Mandiana and Kouroussa District Results CS-18 project activities were conducted in two distinct districts: 1) The District of Mandiana, where after six years of programming and positive results in maternal and child health the district became known as the “Mandiana Model”; and 2) The District of Kouroussa, a resource-poor district in a region which as a whole, is underserved. 11 The evaluation team could not meet with the PSI team in Kankan, which has a very small team involved in condom distribution in the two districts. For the period from January 2002 to July 2006, PSI reports the distribution of the following products: - Injectable contraceptives (Dépo-Provera): Kouroussa: 201 units; Mandiana: 161 units; Total Upper Guinea: 2,948 units. - Oral contraceptive “Planyl”: Kouroussa 435 cycles; Mandiana: 3 cycles; Total Upper Guinea:7,824 cycles. - Condoms “Prudence Plus”: Kouroussa: 161,160 condoms; Mandiana: 102,480 condoms; Total Upper Guinea: 3,680,380 condoms. CS-18 Guinea, Final Evaluation, Save the Children, December 2006 32 The two districts have different levels of health indicators related to the project. The comparison of progress made on improving indicators is difficult because the starting point is different for each district. A useful tool for comparison is a gap analysis of the percent difference, or the deficit. The deficit to be filled is defined by how much remains in order to reach an optimal level of the indicator. The percentage of deficit to be filled is an indication of project performance. This index is explained in Annex 11. The analysis of the 23 indicators for which we have pre and post measurements for the two districts is now possible. Table 8 gives the percentage of the deficit between 2002 and 2006 for the principle project indicators, as well as the four childhood illness or IMCI indicators not targeted by the project. Table 8 shows the following: 1) Overall, notable progress was made on the IMCI indicators even though they were not targeted by the project. This reflects the artificial aspect of identifying specific technical interventions for projects using an integrated approach. Progress on these indicators is the best evidence that CS-18 established and effective C-IMCI strategy. (Progress in the treatment of respiratory infections is the second greatest contributor to avoided deaths during the project, after breastfeeding.) 2) Kouroussa achieved a reduction in the deficit for all indicators. For half of them, this reduction is higher than 52%. 3) Mandiana achieved a reduction of the deficit for 12/23 indicators. For these 12 indicators, the median deficit filled is 30%. 4) For 11 indicators, Mandiana observed a regression (the median of the regression is 36% for these 11 indicators). 5) For all the indicators except one (drop-out DPT 1-3), Kouroussa had better results than Mandiana12. 12 Remember that this measurement is making up the difference between the baseline and the objective. In general, the indicators reached similar levels in the two districts in 2006. CS-18 Guinea, Final Evaluation, Save the Children, December 2006 33 Table 8: Analysis of the Deficit Made up by Indicator for the Two Districts (see Annex 11 for Explanation) Percentage of the Deficit Made up Between 2002 and 2006 Mandiana Kouroussa Maternal Health Recognition of danger signs +33% +69% Trained deliveries +16% +26% TT Vaccination - 36% +46% Referral for complicated births (reported by women) - 309% +20% Two antenatal care visits +57% +92% Family Planning Modern FP use +13% +61% Nutrition Early breastfeeding (1 hr.) +26% +36% Exclusive breastfeeding (0-6 mos.) +43% +60% Complementary feeding (6-9 mos.) - 21% +58% Postpartum Vitamin A - 60% +67% Vaccinations Drop-outs DPT1-3 +83% +56% BCG - 130% +35% DPT1 - 150% +52% DPT3 - 13% +51% Measles - 28% +49% Complete vaccination - 5% +41% HIV/AIDS Knowledge of condoms by women +3% +53% Treatment of STI +8% +79% Knowledge of two modes of HIV/AIDS prevention - 7% +85% Other Indicators Respiratory infection treatment +36% +45% Breastfeeding during diarrhea +2% +34% ORS use - 47% +3% Knowledge of insecticide treated mosquito nets +36% +77% Several explanations can be given for this: ƒ After the success of the preceding phases in Mandiana, the possibility of loss of motivation cannot be completely ignored. ƒ For Kouroussa, on the other hand, the opportunity to collaborate with a project like CS￾18 was a new source of energy, which has contributed to other positive elements: - New energy due to the nomination right before the beginning of CS-18 of the first woman DPS in Guinea; and - The neglected district also benefited from support from the World Bank (new offices), UNICEF (transportation) support, and a new paved road through the district. ƒ CS-18 staff and partners decided to intensify efforts in Kouroussa, where the delay on several indicators was larger, during the MTE. This choice was reasonable and suggests CS-18 Guinea, Final Evaluation, Save the Children, December 2006 34 several possible lessons. Figure 18 represents the intensity of the efforts of CS-18 in each district, according to three criteria: - In blue, the direct costs ascribed to activities in each district are divided by the number of targeted villages (133, that is to say the total number of villages in Mandiana; and 80 is one third of the villages in Kouroussa). According to this measurement, the efforts were 2.2 times greater in Kouroussa than in Mandiana. - In purple, the number of people trained during the four years of the project in relation to the number of targeted villages. According to this measurement, the efforts were 2.7 times greater in Kouroussa than in Mandiana. - In yellow, the number of NGO animators used to cover 100 villages. According to these measurements, the efforts were 2.5 times more intense in Kouroussa than in Mandiana. - These were coarse measurements of the intensity of efforts, but it is important to notice the similarity of the three comparisons. The differences observed apply to all of these factors, and it should be noted that the KPC included the entire district of Kouroussa, not only the 80 targeted villages. There are various considerations offered from this analysis: ƒ For the DPS of Mandiana: a phenomenon of the erosion of project benefits has started. How will the DPS react to consolidate the benefits with its partners? ƒ For the DPS of Kouroussa: Kouroussa was able to benefit from the lessons learned in Mandiana during the previous project cycles. How will the DPS avoid following the same path towards an erosion of the acquired benefits (of course with its partners)? ƒ For Save the Children: CS-18 has created an opportunity for serious operations research on sustainability. How will this be considered if there is an opportunity to obtain new funding from donors? Figure 18: Intensity of Project Efforts According to Three Measurements 0 5 10 15 20 Mandiana Kouroussa $100 per village Persons trained/village Anim/100 villages The relationship between the levels of effort in Kouroussa compared to Mandiana oscillates around 2.5 for the three measurements presented. ƒ For USAID: With funding and reduced intensity of efforts in general, Mandiana maintained the benefits acquired (it made some progress in some areas counterbalanced by an almost equivalent number of regressions). Can USAID draw some lessons in terms CS-18 Guinea, Final Evaluation, Save the Children, December 2006 35 of a progressive and realistic approach to the link between sustainability and scale of programming? What would have been the evolution in Mandiana without any support? How many lives were saved at lower cost, compared with a new zone which was just starting off? Is there an advantage to continuing investments where successes were obtained, making it possible to draw global lessons? Is it possible, after one or two phases of interventions, to target minimal investments for a durable and slower progress in the old zones, while extending the benefits to new zones? 4. Results: Cross-Cutting Approaches a. Capacity Development and Partnerships The CS-18 approach was based on the establishment of solid partnerships, and the development of the institutional and inter-organizational capacities. This strategy was shown in the DIP by a number of indicators known as sustainability. Separately, the indicator on the development of VHC capacities (achieved between 80% and 97%) and that on NGO and DPS capacity (judged more qualitatively), achieved 100% (Table 9). Table 9: Achievements of the Sustainability Objectives Performance Indicators Objective Final result * Villages will have volunteers providing community￾based services 80% √ Villages will have a system of funding obstetrical emergencies (MURIGAS) 80% √ VHCs proficient in planning and conducting community health activities 80% √ Villages which will have a health development plan 80% √ Sub-districts will have associations of VHCs 80% √ Reinforce the competences of NGOs and the DPS offices (HIS system) -- + * Key: √ Achieved objective + Proven or probable progress X Objective not obtained Nm Not Measured The following sections discuss the achievements for the principle partner organizations. i) Partnership with the DPS The two DPS are very satisfied with the CS-18 partnership. The SC teams were integrated in the DPS office space, and initially presented some logistical challenges in Mandiana (Kouroussa has new buildings built by the World Bank). Materially, CS-18 contributed a limited but important share in terms of electrification with solar panels, and some data-processing help in addition to the reproduction of management tools for the health centers and buffer stocks for the community initiatives. CS-18 Guinea, Final Evaluation, Save the Children, December 2006 36 This partnership is one of proximity and daily working together on the same issues. In Mandiana, a rotating “presidency” was organized to direct the team meetings. The “presidency” alternated between the DPS or his assistant, and the SC district coordinator. This approach was not used in Kouroussa. The evaluation team did not examine in detail the pros or cons of this team approach, but it is important to note that the district directors needed to understand that the field activities needed to be stimulated and orchestrated, more than controlled. The institutional responsibility for health services ultimately rests with the DPS, not with the international NGO partner. In addition to daily collaboration, exchange visits abroad were organized to benefit the DPS. These trainings and trips were directly related to the innovations and joint initiatives of the DPS and CS-18, however, other health system staff expressed jealousy at these perceived “perks”. There are intangible benefits of a close partnership and joint monitoring of activities, even when the joint supervisions were not carried out as often as planned (only once in 2005 in Mandiana and not since). The emphasis on joint supervision is perhaps inappropriate. Indeed the DPS conducts field supervisions with support (primarily vehicles, fuel, and too often per diem) by donors and partners including UNICEF, PRISM, and CS-18. This dependency required the DPS to align their plans with the calendars of these agencies (for example vaccination days for UNICEF). Consequently, it becomes difficult to respect a calendar of joint supervision with CS-18, in addition to the other calendars. Moreover the results were obtained without this joint supervision actually being conducted regularly13. The following collaborative activities were carried out: ƒ Quasi-regular (weekly) coordination meetings and review of monitoring data; ƒ reflection and resolution of problems; and ƒ launching and support of initiatives (operations research or pilots). Finally, the effective first level of supervision (community-based distributors, TBAs, health posts, VHCs, peer educators) was done thanks to the intense efforts of the NGOs animators. Their support and efforts supporting the health centers (which frequently lacked fuel or a means of transportation to carry out outreach visits and supervisions), were tremendous. The key role of the DPS on this level was to integrate the civil society – through both local NGOs – as active stakeholders in the health system of the districts. Instead of playing a role as competitor or hindrance (too often observed when the medical authorities feel threatened in their role), the two DPS (as well as the DRS and the higher levels of the MOH) had the wisdom to integrate the efforts of GAAPE and AJVDM through CS-18, into their own plans to improve pubic health. 13 Each DPS has 12 health centers to supervise. At this level, even two annual visits for tens and hundreds of health posts and community agents is enormous. It is not the supervision starting from the DPS which can make a difference, but the supervision by the health center staff. CS-18 Guinea, Final Evaluation, Save the Children, December 2006 37 This integration of community strategies was developed thanks to the support of CS-18 including the use of community HIS data. The semi-annual reports and plans of the DPS, now integrate the HC and HP data with the reports from the CBDAs, TBAs and even the VHCs and MURIGAs. In the sense that CS-18 facilitated this process, the project played an important part in the development of the capacity of the DPS beyond the health system. The DPS and CS-18 used two parallel information systems starting from only one data collection. It was not ideal, but the DPS produced semi-annual reports rich in information and will be able to continue to do so. The project was also in a position to produce the quarterly data required of it. This system thus seems to have functioned well. In the future, more efforts could be placed at the DPS level to use the routine data in a strategy for continuous quality improvement. ii) Partnership with NGOs: GAAPE and AJVDM Both local partner NGOs played an essential part in the success of the project. Since the start, SC began the work of rigorous diagnosis and development of the institutional capacity as well as the technical trainings necessary for project activities. Tables 10 and 11 show the results of the institutional capacity evaluations of the two partners. A detailed report is available in French. CS-18 used an approach of self-evaluation, where the facilitators led a critical self-analysis. The value and the limits of the analysis are summarized in this extract of the NGO evaluation report, referring to the improvement of the scores in the capacity: “The results of this evaluation are not universal and are also partly subjective although the evaluation was based on testimonies and document review. The scores obtained to determine the level of development of NGOs are a function of the criteria defined in the tool used. Admittedly, these scores can vary with another type of tool. Consequently, the scores with 100% of progress do mean that the maturity of these elements is complete. However, it clearly also notes real progress of these NGOs in their working environment with tangible results. This can be shown by certain elements which make the NGO more credible in the eyes of donors and governmental institutions, including inter alia: The segregation of the organization into separate bodies with transparent governorship, the development of strategic plans, handbooks of procedures and management, capacity to develop projects, the tendency of these NGOs to work with regional approval and the extension of their activities to other districts as well as taking into account gender in their internal policies. The general assembly meetings which are the opportunities to report progress and define the annual priorities, became regular with the invitation of the representatives of donors and the districts administrative authorities showed their transparency and efforts to obtain the support of all. Before the contractualization with Save the Children, the majority of these indicators of institutional and organizational development were not taken into account by AJVDM and GAAPE given that they were really Associative Groups.” -NGO Evaluation Report, July 2006 CS-18 Guinea, Final Evaluation, Save the Children, December 2006 38 Table 10: AJVDM’s (Mandiana) Capacity Assessment Background information AJVDM Mandiana Date of creation 15/11/98 Legal recognition agreement 146/P/97/Mna and O50/RAK/KK/CAB/SERACO 20/11/2005 Date of assessment 01/04/2003 25/07/2006 Capacity element evaluated Score Score 1. NGO Structure 7/12 10/12 2. Mission and strategic planning 4/10 8/10 3. Monitoring and evaluation 1/8 4/8 4. Programs and services 1/7 7/7 5. Financial resources 2/6 6/6 6. Human resources 1/14 9/14 7. Leadership/management 4/9 8/9 8. Financial system 5/13 12/13 9. External relations 2/3 3/3 10. Gender 1/3 2/3 TOTAL 28/85 69/85 Table 11: GAAPE’s (Kouroussa) Capacity Assessment Background information GAAPE (Kouroussa) Date of creation 17/07/97 Legal recognition agreement 23/P/97/Kssa and 06 RAK/KK/CAB/SERACO 31/08/2001 Date of assessment 31/03/2003 25/07/2006 Capacity element evaluated Score Score 1. Structure of NGO 7/12 10/12 2. Mission and strategic planning 2/10 9/10 3. Monitoring and evaluation 3/8 6/8 4. Programs and services 5/7 7/7 5. Financial resources 2/6 6/6 6. Human resources 2/14 10/14 7. Leadership/management 8/9 9/9 8. Financial system 5/13 11/13 9. External relations 2/3 3/3 10. Gender 1/3 3/3 TOTAL 37/85 74/85 CS-18 Guinea, Final Evaluation, Save the Children, December 2006 39 This institutional strengthening (development of strategic plans and administrative procedures, reorganization on the basis of evaluations, development of recruitment procedures, improvement of monitoring systems, and improved communication with donors) resulted in advancements in the development of technical knowledge. This leads to the NGOs developing an increased capacity to develop and manage contracts with other partners and to diversify their funding sources. In the sustainability analysis, this corresponds to an improvement of capacity as well as the viability of the organizations. These improvements will still need to be reinforced by obtaining recognition at the district, regional and national levels. However, as indicated in the preceding section, beyond the individual capacity development of the organization, the greatest contribution from CS-18 was in the effective and coordinated involvement of local NGOs. Additionally, the activities of the MOH helped to increase the visibility of NGOs vis-à-vis the MOH and donors. This involved expanding the NGOs’ activities and competences, ensuring their participation in planning, and monitoring and evaluation activities, and also a reciprocal transfer of knowledge by exchange visits between the two NGOs. Additional success factors included: training; ongoing “coaching”; material support; and the existence of detailed agreements between SC and GAAPE/AJVDM. b. Development of the VHCs and AVHC The VHCs and AVHCs drove the community level activities: ƒ 213 VHCs were created in the project intervention zone; and ƒ 24 AVHCs were created more recently. CS-18 used a tool similar to that used with local NGOs to assess the capacity and self-sufficiency of the 213 VHCs. From this analysis, the VHCs were divided into three categories according to their degree of organizational maturity. Table 12 shows the results of this analysis. Table 12: VHC Capacity in Self-Management, as per the CS-18 Institutional Diagnosis VHC Capacity in self-management “Advanced” “In progress” “Limited” Mandiana 95 33 5 Kouroussa 76 3 1 Total 171 36 6 According to this analysis, the project achieved its goal, with 80% of VHCs having an “advanced” capacity and 97% having an “advanced” capacity or being “in progress.” Figure 19 shows a modest but probable difference between the VHCs of Kouroussa and Mandiana which may be explained by the presence of the NGO animators. VHC capacity was 2.5 times weaker in Mandiana than in Kouroussa. CS-18 Guinea, Final Evaluation, Save the Children, December 2006 40 The VHC evaluation report is available in French. The FE did not allow for a detailed and critical study of these results. The study of the VHCs would certainly present questions related to methodologies, but the report shows a credible effort to obtain reliable, quality data. Figure 19: Distribution of VHC Capacity “Advanced”, “Average”, or “Low” for Mandiana and Kouroussa 71% 95% 25% 4% 4%1% 0% 20% 40% 60% 80% 100% Mandiana Kouroussa Weak VHC Average VHC Strong VHC The VHCs were an essential means of social mobilization and provided effective communication for promoting changes in health behaviors. The media were used to complement their efforts. The VHC collaborated with health centers and the DPS to: ƒ involve themselves in the NIDs, to conduct active case findings (for both vaccination drop-outs and for pre/postnatal consultations) and to investigate infectious disease monitoring; ƒ support outreach visits and conduct social mobilization; ƒ ensure the follow-up of pregnant women in certain cases, and ensure Vitamin A supplementation; and ƒ facilitate mediation between community members and health agents for the correct payment of consultation fees14. Additional efforts have been made through collaboration between VHCs and NGOs to obtain funding from other sources. In one instance, a contract was established between AJVDM and the VHC of Mandiana II. The role of the AVHC is greatly appreciated and recognized by all, but their existence is too recent (and rather fragile) to be assured in the future. Guinea also has a system of COGES (management boards for the HCs) which is largely non-functional. In the future, the AVHC could continue to support the VHCs and alleviate weaknesses with the COGES, which lack community representatives like the AVHCs. However, at this time, the existence of the AVHCs is too new for the partners to consider this option. Other factors, in addition to the frailty of the recent structures (AVHC) which restrict this approach, are fatigue and possible lack of motivation by community members, if they are not 14 The current prices are inconsistent and don’t make sense. As a result, the communities question the credibility of the health system when the official prices are circumvented. CS-18 Guinea, Final Evaluation, Save the Children, December 2006 41 supported by the local NGOs and the AVHCs. Many members of the community are still illiterate and need considerable organizational, coordination and monitoring skills. The project developed practical and visual tools which enable the accomplishment of this work without literacy skills, however, the potential danger exists for the loss of voluntary resources when the work is too professionalized. c. Creation of MURIGAs MURIGAs were created in response to credibility problems which arose when their management was centralized. They gradually supported a growing number of obstetric referrals, particularly in Mandiana, where they were established earlier (Figure 20). A review of the operations of MURIGAs was conducted with the partners and their coverage was extended to include emergency pediatric referrals. Figure 20: Obstetrical Referrals Supported by the MURIGAs - 2003 to 2006 00 2 3 33 37 56 0 10 20 30 40 50 60 70 2003 2004 2005 2006 Mandiana Kouroussa Between 2003 and 2006, the MURIGAs covered the cost of 131 of 931 obstetrical referrals recorded by the HIS (14%). The HIS data shows a progressive increase in the number of referrals managed by the MURIGAs between 2003 and 2006. The factors of successes identified by evaluation team are: ƒ Related to operations: - Decentralization of funds at the village level; - Establishment of a three-part agreement between MURIGAs, taxi unions, and hospitals; - Training of fund managers; - Contribution of members; and - Expanding the coverage to both obstetrical and pediatric emergencies. CS-18 Guinea, Final Evaluation, Save the Children, December 2006 42 ƒ Related to supports: - Subsidization of the MURIGA by certain districts; - Involvement of the local authorities. Support of the state and the partners for the reinforcement of institutional capacities and management of the MURIGA (ambulances, radio, motor bikes, management tools); - Existence of a system of management tools for monitoring; - Training of the TBAs/Health Workers/VHC/NGOs; - Collaboration with the MOH through the Division of Reproductive Health; - Monitoring of the MURIGAs’ activities in the DPS reports; - Organization of meetings to improve communication and diffusion of messages by the rural radio. The FE team identified the following limiting factors: ƒ Cases of non-observance of the tripartite convention and tariffs by people receiving benefits; ƒ The poor treatment of patients by referral facilities; ƒ The irregularity of repayments (and sometimes of the contributions); ƒ The limited coverage of the villages (some villages were not included in CS-18); and ƒ Cultural concerns like the fear of being in debt, which discourages certain families from using the MURIGAs. Repayment of borrowed funds does not occur, however this is not creating a deficit because of the high level of contributions. The still limited use of the funds and the absence of repayments nevertheless calls for monitoring of fund management. Continued support in their management is necessary. d. Training The project established and conducted an ambitious and multidimensional training scheme (technical, managerial, personal, and organizational). The training plan is available in Annex 12. e. Innovation and Sustainability: Research on the “Mandiana Model” Undoubtedly, the most important lesson of the CS-18 project is the potential of an effective model of community health in Upper Guinea. Three essential elements contributed to the improvements in MCH: ƒ The mobilization of community structures; VHC and MURIGAs; ƒ The expansion of health services from the HCs to the communities and the support of these services by the VHC; and ƒ The support provided by GAAPE and AJVDM, under DPS supervision, to ensure the micro–planning and monitoring of health activities. Figure 21 represents the essential institutions and their relationships which made the model a success. Figure 22 shows the institutional relationships whose existence is threatened by the end of the project, in dashed lines. The challenge for the MOH, the DRS, and the local actors will be to maintain this collaboration. Recommendations for this are made at the end of this report. CS-18 Guinea, Final Evaluation, Save the Children, December 2006 43 Figure 21: An Effective Model of Community Health DPS HC HP Village Drug Kit AVHC VHC Local NGO TBA CA MURIGA Rural Community Resources Support « Cement » Figure 22: Threatened Sustainability of the Community Health Model DPS HC HP Village Drug Kit AVHC VHC Local NGO TBA CA MURIGA Rural Community Key: - - - - - precarious bonds and institutions post-project In the analysis of progress made toward sustainability conducted by the project during the DIP15, important progress was ensured in components 1 (health results), 2 (quality of the care), 3 (organizational capacity) and 5 (community capacity). However, important threats remain in terms of the viability of NGOs in their current role, the financial viability of MURIGAs, the 15 The analysis framework on sustainability examines six essential components or directions needed to establish sustainable progress: 1) health indicators; 2) service indicators; 3) local organizational capacity; 4) local organizational viability; 5) community capacity; and 6) socio-ecological environment. See http://www.childsurvival.com/documents/CSTS/SustainabilityArticleIJHPM_2004_22.pdf. CS-18 Guinea, Final Evaluation, Save the Children, December 2006 44 viability of the support of health centers for community-based service (component 4), and in the essential functions normally provided by the MOH (component 6). C. Project Management The management of the project was examined in great detail during the MTE, and therefore it was not the subject of detailed study for this FE. The following observations were noted however, during the FE: ƒ CS-18 developed an effective system of planning and monitoring activities. This planning was conducted at all levels: at the Kankan office, with each DPS and NGO partner, and through follow-up of the micro-planning performed by NGO partners with the VHC and HCs. ƒ When there were delays related to planning (as is often the case with projects with training components16), project staff was able to identify them and make appropriate adjustments. ƒ The project personnel were obviously motivated, qualified and trained well. The doctors and technical staff of the project were led by a chief of project, whose administrative and personnel skills, attitudes, and intelligence were noted by all the partners, including USAID and the MOH. Various team members worked well together and provided mutual support. CS-18 staff and partners functioned as an effective and well-oiled team. It is unfortunate that the project approach creates a situation where this kind of capacity can not always be maintained, but must be rebuilt again elsewhere. ƒ Financial management was not examined, but detailed statistical data was provided on a timely basis during the evaluation, when requested by the evaluators. ƒ The project had little time to compensate for logistical failures in the health system. Its own logistics system was reliable. During the course of the project it would not have made sense to replace the logistics system which is the MOH’s responsibility. Serious and repeated deficiencies do remain in the health system’s logistical process which threatens the progress achieved by the project. ƒ The monitoring and evaluation system, and HIS function well and provide valuable information. The project team had prepared a detailed analysis for the FE. This shows a tremendous capacity to produce, analyze, and apply information. Certain errors were found in the analyses, but were corrected with the technical support of the CS Advisor based in Westport. Recently, the project’s HIS Officer left this position, however, this position was filled before the FE Team Leader arrived, and information was provided as requested. ƒ Collaboration with the USAID Mission was excellent. The USAID Mission staff expressed a high degree of appreciation and satisfaction for SC’s work, despite their absence during the final debriefing. 16 If one calculates the number of lost working days represented by training plans, it would become necessary to close the whole health system during the first year of the project to be able to conduct these trainings. Fortunately, the project managers had the good sense to delay some of the trainings. Projects designed to develop local capacity should operate with a longer calendar than projects developed to quickly address a deficiency in a health service. CS-18 Guinea, Final Evaluation, Save the Children, December 2006 45 D. Other Topics Identified by the Evaluation Team The FE team identified the following lessons learned: 1) Community involvement is necessary to improve health. ƒ VHCs can be effectively involved in basic health activities through: - Training/equipment; and - Support and follow-up by local NGOs (for success and sustainability); ƒ The support of leaders (community, religious) is essential for VHC activities. 2) Institutional resources and collaboration are necessary for success. ƒ The capacity of local NGOs can be developed and their involvement helps to improve health indicators. ƒ The involvement of national NGOs can be integrated into MOH activities to advance achievements of health indicators. ƒ The co-location of the DPS team with the project team in the same room, facilitates their integration and coordination. ƒ The AVHC can be involved in the resolution of long-term problems facing the VHC. 3) Lessons learned for increasing sustainability: ƒ Arrangements between partners (for example DPS-NGOs) need to be formalized in order to be sustained. ƒ The formalization of this collaboration facilitates motivation and fundraising. ƒ The early weaning of the AVHC and VHC from external/SC support (one year after establishing an approach) is a risk for the sustainability of gains made. 4) Lessons learned related to technical approaches: ƒ The following approaches showed very important results: - Training and monitoring of TBAs; - Advocacy with religious leaders; - Peer education; - Combination of tools and approaches for communication; and - Nutritional demonstrations, sensitizing, weighings combined with drug distribution. E. Dissemination of Lessons Learned and Results17 SC will disseminate these lessons learned to development professionals initially in Guinea, through quarterly meetings with MOH partners in Kankan. Several of the recommendations suggest a roundtable organized by the MOH authorities, in order to define a viable model of community health for Guinea. SC also hopes to transmit the lessons learned through participation in the new bilateral project which will emphasize good governorship. SC organizes an annual Program Learning Group (PLG) meeting for its health programs, where this experience will be 17 This section was written by Save the Children staff team. CS-18 Guinea, Final Evaluation, Save the Children, December 2006 46 shared extensively. SC recently disseminated some of the results during the most recent APHA meeting held in November 2006. Finally, the SC team and partners (the FE Team Leader and Dr. Peter Winch of Johns Hopkins) intend to publish an article on lessons learned from CS-18 with regard to progress on sustainability and implications for donors. F. Results Highlights Annex 13 presents a summary highlight page for use by USAID and underscores the results obtained in Maternal Health. G. Conclusions and Recommendations 1. General Recommendations Made by the Evaluation Team The evaluation team collectively developed the following recommendations shown in Table 13. Table 13: Recommendations of the Evaluation Team Recommendations for the DPS 1) Formalize bonds between DPS-HC-VHC 2) Formalize a partnership between the DPS and local NGOs Recommendations for SC 1) Subsidize NGOs to ensure follow-up 2) Equip NGOs with reliable transportation (motor bikes) Recommendations for NGOs 1) To continue to ensure the supervision and refresher training of VHC and AVHC 2) To work out the micro plans for community structures 3) Fundraising 4) To continue to support the DPS, the VHC, AVHC in the implementation of activities Recommendations for the VHC 1) To mobilize the communities 2) To hold growth monitoring sessions 3) To develop micro plans 4) To continue the activities after the project end date Technical Recommendations 1) To combine community weighings with the distribution of Mébendazole and Vitamin A 2) HIS: To take into account referrals carried out by the TBA towards the health centers (not only referrals to the hospitals) 3) To pay attention to, and follow the nutrition activities at the national level CS-18 Guinea, Final Evaluation, Save the Children, December 2006 47 2. Specific Recommendations Made by the Team Leader In addition to the recommendations presented below, several points can be underlined and some additional recommendations can be made. I. Technically, the project does not have interventions where a major readjustment is necessary, if the intervention were repeated or extended. However, if this should occur, the following modifications should be considered: ƒ As discussed in the MTE, a malaria intervention could be added since the project obtained convincing results in other IMCI activities. An extension could support these activities. ƒ The HIV/AIDS component would deserve to have more time to develop and improve its results, but the general direction taken was good. ƒ In the event of an extension, greater attention must be paid to the key indicators which were not measured in a suitable way (for example the quality of pre- and postnatal services, and birth). Progress in child nutrition should be confirmed and accelerated. ƒ Efforts must be continued to ensure the supervision of TBAs, and to prevent postpartum hemorrhage. Future capacity development should reinforce supervision and support from the community HC structures. II. The principle recommendations related to ensuring the continuity of progress are: ƒ As recommended in the preceding section, the relationships and essential roles of the “model” must be formalized. ƒ The financial, administrative, and governance risks must be taken into account, especially in relation to providing oversight and support to the MURIGAs. ƒ The benefits of improving access to care (supplies and drugs) will not be sustained unless the MOH sets up an essential drugs logistics system, or it finds a way to develop a public￾private partnership with wholesalers. A solution must be found quickly. ƒ Certain vulnerable areas were identified (AVHC, MURIGAs, outreach visits, institutional bonds). The existing institutions and possible future projects will have to consider them closely. ƒ The central question for Guinean institutions should be, “Did Guinea, which defines community health as a national priority, find in Mandiana and Kouroussa the elements of its model for community health?” The answer to this question are proposed below. III. Save the Children should seek to develop a new project in Upper Guinea designed to study the costs of obtaining sustainable progress. IV. Additionally, USAID and/or other donors should consider the potential value of CS-18 activities and accomplishments. This potential is illustrated in Figures 23a and 23b. CS-18 Guinea, Final Evaluation, Save the Children, December 2006 48 Phases 1 and 2 (Figure 23a and 23b) represent what was obtained during the CS-14 and CS-18 projects: ƒ In Phase 1, the investment was in Mandiana, with very positive results shown by the fast ascent of the blue curve in Figure 23b. ƒ In the second phase, Mandiana did not receive more than approximately a third of the initial investment, and obtained only very limited growth of the health indicators (in fact a reasonable increase for half of them, and a regression for another group of indicators). During this time, Kouroussa achieved a fast increase in its indicators (see Figure 23b) with an investment of less than half the initial investment for Mandiana. ƒ The following phase (Phase 3) could test the minimum investment required to obtain a minimum progress in the districts of Mandiana and Kouroussa, while allowing an expansion of the benefit to a new zone. Figure 23a: Distribution of Investments Between Mandiana, Kouroussa, and an Extension Zone of a Possible Third Phase 0 200 400 600 800 Phase 1 (CS12-14) Phase 2 (CS18) Phase 3? Intervention Phases Distribution des ressources Mandiana Kouroussa New Districts Figure 23b: Progress of Health Indicators in Mandiana, Kouroussa, and a New Zone in a Third Possible Phase 0% 10% 20% 30% 40% 50% 60% 70% 80% Phase 0 Phase 1 (CS12-14) Phase 2 (CS18) Phase 3? Indic Man Indic Kou Indic Exten1 Indic Exten2 CS-18 Guinea, Final Evaluation, Save the Children, December 2006 49 V. The district MOH structures must quickly define a viable model in order to maintain a viable model of community health. ƒ The structures of the MOH/DRS/DPS must accept and assume the leadership which falls to them. These structures can define the agenda for the country or wait for another model suggested by a donor. However, the leadership must take the initiative and engage their own resources. ƒ A negotiated role must be recognized for the local NGOs which carried out valuable work and whose presence on the ground cannot be replaced by more health structures. ƒ NGOs themselves will have to offer to the partners a reasonable cost structure. ƒ It is important to be realistic about the material constraints of the DPS and DRS, and it is also necessary to be determined to analyze them in a realistic and honest way. ƒ The prefect and the decentralization structures must be involved and helped to appreciate the social impact of an effective community health system. ƒ Various stakeholders must have a cohesive strategy with respect to donors, but must initially develop a cohesive in-house strategy on the model which they wish to develop. ƒ The problem of the unreliability of the essential drug supply must be solved quickly. ƒ A study on the models and their viability would readily support USAID’s interest in questions of governance. There is perhaps a convergence of interests between Guinea and its donors. VI. In order to advance concretely towards a viable institutional model, the DRS/DPS must urgently organize a roundtable to: ƒ Establish agreements and principles of operation beyond the involvement of SC; ƒ Outline a common vision for the long term roles of each in community health; ƒ Define the first realistic and realizable mechanisms to institutionalize relationships and to continue essential activities; ƒ Optimize the distribution of resources based on (including means of transportation used by GAAPE and AJVDM for the project period). ƒ In addition to the organizers, the participants in this roundtable should be: - GAAPE, AJVDM; - Save the Children; - Districts, structures of decentralization and social affairs; and - Leaders selected among the persons in charge for HCs and community organizations (AVHC). H. List of Annexes 1. Excerpts from the DIP with Baseline Results 2. Evaluation Team Members 3. Final Evaluation methodology 4. List of People Contacted 5. List of Participants in the Final Evaluation Debriefing 6. Training Plan for the Health District of Mandiana CS-18 Guinea, Final Evaluation, Save the Children, December 2006 50 7. Training Plan for the Health District of Kouroussa 8. Analysis of Deaths Avoided 9. Interventions with the Greatest Estimated Impact 10. Final Evaluation Photographs 11. Explanation of the Index “Percent of Deficit Filled” 12. Summary of Project Progress 13. Results Highlight 14. Updated CSHGP Data Form CS-18 Guinea, Final Evaluation, Save the Children, December 2006 51 ANNEXES Annex 1. Excerpts from the DIP with Baseline Results Annex 2. Evaluation Team Members Annex 3. Final Evaluation methodology Annex 4. List of People Contacted Annex 5. List of Participants in the Final Evaluation Debriefing Annex 6. Training Plan for the Health District of Mandiana Annex 7. Training Plan for the Health District of Kouroussa Annex 8. Analysis of Deaths Avoided Annex 9. Interventions with the Greatest Estimated Impact Annex 10. Final Evaluation Photographs Annex 11. Explanation of the Index “Percent of Deficit Filled” Annex 12. Summary of Project Progress Annex 13. Results Highlight Annex 14. Updated CSHGP Data Form Annex 1 Excerpts from the DIP with Baseline Results A. Executive Summary This four-year Child Survival-18 Project, The Community Health Initiative for the Districts of Mandiana and Kouroussa, is a partnership between Save the Children (SC), the two District Health Offices (DPS) and two local non-governmental organizations (NGOs). This partnership builds on the extensive programming experience of SC in Guinea to address the primary causes of child and maternal mortality in an underserved, poor, and remote area of Guinea. SC and its partners will combine their resources and experience, and apply these towards improving child survival in Guinea by: (1) Increasing the use of key health services and improved maternal and child health practices at the household level; and (2) Increasing the capacity of local entities (the regional health office, the two district health offices, the two local NGOs, and community organizations) to assume responsibility for health activities and adopt innovative CS-18 approaches. SC/Sahel’s extensive experience, capability, and credibility in community mobilization, maternal and child health interventions, and capacity building of partners in Guinea, position this partnership well to play an important role in this most needy region of the country. The report of the CS-14 final evaluation, conducted in Mandiana in August 2002, lists the following priority conclusions: The project has met or exceeded most of its objectives for the technical interventions (Nutrition/Vitamin A, Maternal and Newborn Care, Immunization, Malaria Control, Family Planning) and sustainability. The project has had a positive impact on national health policy in: 1) the development and acceptance of a revised TBA training curriculum and manual; and 2) integrating Vitamin A distribution into NID. Prospects for institutional sustainability are strong due to a commitment to building capacity of partners, as well as communication and collaboration among the various stakeholders, including the DPS, VHC, and COGES. The project’s recognized success has leveraged additional resources from other international organizations and donors. Engaging senior women and traditional practitioners into the communication strategy has brought about changes in social norms that enhance the potential for behavioral sustainability. The project has built financial sustainability at the community level through emergency revolving funds, transport plans, membership cards guaranteeing payment, cereal banks, etc. The project has been exemplary in its synergy and collaboration with other USAID￾funded projects and technical resources such as the American Collage of Nurse Midwives (ACNM), PRIME, CSTS, BASICS II, CHANGE, NGO Networks for Health, PRISM, etc. It has taken advantage of every opportunity to learn and apply state-of-the-art programming, and to promote best practices and quality of care. The project has been recognized as highly replicable, with potential for broader applications beyond Mandiana. CS-18 Guinea, Final Evaluation, Save the Children, December 2006 53 The project has empowered communities through capacity-building, literacy training, and the tools to collect and manage relevant health data for decision-making. The project has created a culture of communication, collaboration, innovation, monitoring, and quality of care that can be sustained and transmitted to NGO sub-grantees and other partners under CS-18. The CS-18 project site will scale up the successful CS-14 strategies beyond the CS-14 original project site of the district of Mandiana and expand services to the district of Kouroussa, both in Upper Guinea. The two districts with 527 villages have a total population of 393,060 including 85,402 children under five years old and 106,753 women of reproductive age. The CS-18 Extension Project is designed to address high under-five and maternal mortality in these sites, estimated at over 222 deaths/1,000 live births and 528/100,000 live births, respectively. The populations of these two districts, especially that of Kouroussa, have low use of health services and health information, and the current health services do not meet their preventive and curative health needs. SC will work with two CS-18 partners, the two Health Districts, and two Guinean NGOs, to implement four child survival interventions, specifically: • Immunization (10% of intervention effort), • Nutrition and Micronutrients (30% of intervention effort), • Maternal and Newborn Care (40% intervention effort), and • HIV/AIDs (20% of intervention effort). CS-18 interventions will be implemented through the following strategies: • Joint DRS, DPS and SC design, implementation, and evaluation of approaches to maternal and child health in the districts that inform strategies of other PVOs and the MOH. • Improving the technical, financial, and institutional capacity of two local NGOs. • Mobilizing communities through Village Health Committees and Health Center Management Committees to improve essential health services and conduct focused education activities. • Improving the quality of health services provided in the community and the health facilities. SC will document the feasibility and results of implementing these four interventions through five innovative strategies for child survival in Guinea: (1) Training of traditional birth attendants (TBAs) using a revised and updated community-life savings skills curriculum; (2) Behavior Change interventions to improve utilization of early postpartum care; (3) The Positive Deviance (PD) Approach for sustainable community-based rehabilitation and prevention of malnutrition; (4) The Child-to-Child approach to increasing vaccination coverage, and; (5) Centres d’Ecoute (listening centers), which will provide safe venues for peer education, informal discussion groups, and structured socialization. The CS-18 design reflects SC experience and expertise with the community-level implementation of all four CS-18 interventions, and builds on current innovative SC/Sahel work with PD and community mobilization. The Project builds on the existing SC/Sahel current local CS-18 Guinea, Final Evaluation, Save the Children, December 2006 54 partnerships with one of the MOH Health Districts and with one of the local NGOs, and reflects extensive discussions with local NGOs, health districts, other regional NGOs, and the regional MOH. CS-18 Goals are: • A sustained reduction in under-five and maternal mortality in the two health districts; and • A sustained improvement in the nutritional status of 0 to 36 month-old children. The CS-18 Results are: • Increased use of key health services and improved Maternal and Child Health practices at the household level in two districts; • Local entities (MOH, local NGO, communities) will be able to assume responsibility for activities and adopt innovative CS-18 approaches. These Results will be achieved through the seven Intermediate Results: • Increased availability of selected MCH services in the two health districts; • Improved quality of selected MCH services in the two districts; • Increased caretaker knowledge and awareness of selected child survival issues; • Documented feasibility and results of implementing innovative CS-18 approaches; • Improved capacity of two district health offices to effectively support community health services and activities; • Demonstrated SC/Sahel capacity in capacity building of two local NGOs and two district health offices; and • Improved capacity of two local NGOs and communities to effectively address priority health needs of mothers and children under 5. Under the “Cost Extension” grant category, SC has received $1,250,000 of USAID funding over a four-year period, which started on September 30, 2002. This USAID funding is matched by $416,625 from SC (25% match). Principal authors of this draft DIP are: Dunni Goodman, Mamadou Balde, Carmen Weder, and Eric Swedberg although all the Guinea staff were involved in writing various sections. The USAID/Guinea Office Reproductive Health Advisor Kathy Jacquart participated in the DIP workshop in April 2003. The contact people at SC to discuss this application are, Carmen Weder Manager, Office of Health and Eric Swedberg, Child Survival Specialist. Summary Findings of Baseline Assessments: 1. KPC Survey: The following table summarizes the KPC findings from the three surveys. Overall the KPC surveys suggest that in Mandiana the greatest progress has been in the 73 district villages which were the major focus on CS-14 efforts. In the last two years of the project progress has also been made in the secteurs through the creation and training of VHCs in the 60 secteurs. Indicators are still low and justify the choice of interventions of CS-18. In Kouroussa the low level of the indicators also suggest the CS-18 project interventions are merited. CS-18 Guinea, Final Evaluation, Save the Children, December 2006 55 Table 6: Selected Indicators of the KPC Studies Indicators Mandiana district 1997 2002 Mandiana sectors 2002 Kouroussa 2002 EDS 99 Proportion of children less than 6 months exclusively breast fed 6/55 (10.9 %) (4 mo. age) 35/90 (38.9%) 39/89 (43.8%) 17/58 (29.3%) 12% Proportion of mothers having started to breast feed within one hour after birth 62/293 (21.2%) 130/300 (43%) 75/293 (26%) 19/289 (6%) 29.7% Proportion of mothers who add iodized salt to child’s meal 36/293 (12,3 %) 54/192 (28,0 %) 8/119 (6.7%) 13/136 (10%) 5% Proportion of children having their weight on a growth monitoring chart at least once 5/293 (1,7 %) 79/117 (67,5 %) 15/46 (32.6%) 5/39 (12.8%) Proportion of children aged 12 to 23 months completely vaccinated by first birthday 21/139 (15.1 %) 54/77 (70,1 %) 16/42 (38.11%) 20/136 (15%) 30% Indicators Mandian a districts 1997 Mandiana sectors 2002 Kouroussa 2002 EDS 99 Indicat ors Proportion of mothers who know at what age a child must be vaccinated against the measles 12/293 (4.1 %) 200/300 (66,7 %) 58/293 (19.8%) 35/289 (12.1%) Proportion of children suspected of being infected by pneumonia treated with an antibiotic 5/25 (20.0%) 15/35 (42.8%) 11/63 (17.5%) 12/39 (30.8%) Proportion of children with diarrhea given ORS 2/59 (3.4%) 35/83 (42.2%) 19/75 (42.2%) 9/89 (1%) 32.5% Proportion of breast fed children who had more breast feeding than usual during their last diarrhea episode 6/52 (11.5%) 16/79 (20.3%) 8/75 (10.7%) 7/86 (8.1%) Proportion of children not exclusively breast fed whose mother gave them liquid other than the maternal milk more than usual when they had diarrhea 3/44 (6.8 %) 11/83 (13.3 %) 2/75 (2.7%) 5/85 (5.9%) 53% Proportion of children 6 to 9 months given additional food (meat, fish) 15/34 (44%) 30/45 (67%) 24/62 (39%) 7/71 (10%) Proportion of suspected cases of malaria treated with chloroquine 52/121 (43.0%) 92/159 (57.8 %) 72/160 (45%) 53/76 (69.7%) - CS-18 Guinea, Final Evaluation, Save the Children, December 2006 56 Proportion of suspected cases treated with chloroquine, correct posology 23/121 (19.0 %) 69/92 (75 %) 31/72 (43%) 15/76 (19.7%) - Proportion of mothers who recognize mosquito nets as a method to avoid malaria 45/293 (15.4 %) 193/291 (66.3 %) 60/293 (20.5%) 31/289 (10.7%) - Mothers having received at least two doses of TT 70/293 (23.9 %) 154/177 (87%) 91/124 (73.4%) 80/109 (73.4%) 43.1% Proportion of mothers having at least two antenatal visits during the last pregnancy 133/293 (45.4 %) 259/300 (86.3 %) 169/293 (57.7%) 192/289 (66.4%) 66.6% Proportion of women of reproductive age who know at least 2 of the 5 signs of danger 38/293 (13.0 %) 134/300 (44.7 %) 95/293 (32.4%) 73/289 (25.3%) Proportion of women who do not wish a baby within next two years who use modern contraception method 15/200 (7.5 %) 99/200 (49.5 %) 47/196 (24%) 18/186 (9.7%) 6.8% Proportion of mothers using a condom with irregular partner 34/44 (72.7%) 20/37 (54.1%) 8/17 (47.1%) 20.1% Proportion of mothers having had the STI during the last 12 months and sought care at a health center 15/23 (65.2%) 7/15 (46.7%) 22/54 40.7% 86.1% Proportion of mothers naming at least two ways of prevention against HIV 143/298 (48%) 77/288 (26.7%) 62/275 (22.5%) Nutrition: The KPC indicates that exclusive breastfeeding up to six months of age is higher in Mandiana (39% Districts and 44% Secteurs) than in Kouroussa (29%). In Mandiana districts 43% of infants are breastfed within the first hour after birth. In the Mandiana secteurs (26%) and in Kouroussa (6%) this is much lower. Less than one third of mothers in Mandiana districts report using iodized salt in children’s meals. This practice is much lower in the Mandiana secteurs and Kouroussa and merits more emphasis in CS-18. The majority of children received vitamin A during the last six months through the national days of vaccination (NDV). Post￾partum Vitamin A supplementation is not the norm in Guinea and was not surprising to be much lower in Kouroussa (9%). Vaccination: The vaccination campaign for children from 12-23 months is 70% in the district of Mandiana, 38% in the sectors against only 15% in the districts of Kouroussa. Maternal and Newborn Care: More than the half of mothers did at least two antenatal consultation within the districts and sectors of Mandiana and Kouroussa (Respectively 86.3% ; 57.7% and 66.4%. The survey revealed that the majority of mothers have received the TT CS-18 Guinea, Final Evaluation, Save the Children, December 2006 57 vaccination at least twice( 87% in the districts of Mandiana and 73.4% in the sectors of Mandiana and Kouroussa.) The number of mothers assisted by a health professional (nurses, doctors) during childbirth is 23.3% in the districts of Mandiana, 14% in the sectors and 31.5% in Kouroussa but more than one third of childbirth is assisted by the TBA in Mandiana against 31% in Kouroussa. In the districts of Kouroussa around a quarter of mothers (25.3%) know at least two signs of danger during childbirth (or pregnancy). That proportion is about 44.7% in the districts of Mandiana and 32.4% in its sectors. HIV/AIDS/STD: The majority of the population interviewed (97.7 in the district of Mandiana ; 98.3% in sectors and 94% in the district of Kouroussa) have heard of HIV/AIDS. Though, 48% of them know at least two ways of avoiding HIV/AIDS. That proportion is around 26.7% in the sectors of Mandiana and 22.5% for the districts of Kouroussa. Around 72 % of husbands who have had occasional sexual acts outside in the district of Mandiana have used a condom during the act. That proportion is around 54.1% for the sectors of Mandiana and 47.1% for the districts of Kouroussa. More than three fourths (87.4%) of mothers in the sectors of Mandiana have heard of STI. That proportion is 72.3% for the districts of Mandiana, only 25% for the districts of Kouroussa. But 7.7% of mothers in the district of Mandiana have got the STI during the last 12 months. That proposition is about 5.1% for the sectors of Mandiana and 18.7% for the district of Kouroussa. among them around a third went to hospital for treatment. That proportion about 60.9% for the district of Mandiana. 2. Village Health Committee Assessment: This assessment confirmed that the majority of VHCs in Mandiana continue to conduct health activities as well as other development activities in their localities. 73 of 133 VHCs have received official government legal recognition. There are not large differences between the district and secteur VHCs although the district VHCs which were created two years earlier are stronger. 124 or 93% of all the VHCs have overall strong functioning, while 9 VHCs (7%) need greater attention. 3. NGO Capacity Assessments: Overall GAAPE is at 47 % of it’s institutional and organizational capacity and AJVDM is at 35 % of institutional and organizational capacity (Mandiana). The table below summarizes the results for the 10 capacity areas: Table 7: Institutional Capacity Assessment of Local NGOs C. Elements of the Evaluation D. NGO GAAPE AJVDM E. Date of creation 15 September 1997 15 November 1998 F. Government agreement 23 / P / 97 / Kssa 146 / P / 99 / Mna 1. Structure of the NGO : - status and policies and regulations 3/3 3/3 - limitation of the mandates ½ ½ - functioning 3/7 3/7 2. Mission and strategic planning : - mission ¼ 3/ 4 - strategic plan 1/6 1/6 CS-18 Guinea, Final Evaluation, Save the Children, December 2006 58 3. Monitoring and evaluation : 3/8 1/8 4. Programs and services 7/7 1/7 5. Financial resources 2/6 2/6 6. Human resources : - recruitment 1/5 Voluntary integration policy (0) - Evaluation of personnel 1/8 1/8 7. Leadership/Management : - leadership 4/4 3/4 - administrative system 4/5 1/5 8. Financial system : - financial management 3/4 2/4 - accounting management No accountant (0) 1/5 - internal control 2/4 2/4 9. External relationships 2/3 3/3 10. Gender 1/3 1/3 TOTAL 39 / 83(47%) 29 /83(35%) 4. Organizational Life Assessment in Kouroussa: All villages have agricultural associations, however none of the villages have community organizations concerned specifically with health issues. Village leaders have been involved in working with the MOH to inform and mobilize the population and they expressed the desire to form village health committees. The focus group participants identified the major diseases in children as: malaria, diarrhea, measles, tetanus, ARIs, tuberculosis, and intestinal parasites. Illnesses in women of reproductive health were identified to be: MSTs, malaria, anemia, malnutrition, eclampsia, intestinal parasites, and stomach illness. Adolescents mentioned: malaria, MST, parasites, anemia, hernias, migraines, malnutrition, diarrhea, hemorroids, and onchocerosis. Other general problems mentioned discussed were the high cost of medications, accessibility to health centers, and the lack of personnel. Ninety TBAs were identified in the prefecture 53% are trained, 33% are equipped, and 27% are literate. Eighty-one traditional healers were identified, 9% who have worked with health workers in the past. The majority have extensive experience; 60% with over 20 years of work experience. CS-18 Guinea, Final Evaluation, Save the Children, December 2006 59 Annex 2 Evaluation Team Members N° Name Organization Position Location 1 Dr Richard James DRS DRS Interim Director Kankan 2 Adama Diop SC Advisor SR Mandiana 3 Dr Abdoulaye Oumar Diallo SC District Coordinator Mandiana 4 Adama Doumbouya SC NGO Advisor Mandiana 5 Dr Mamadou Aliou Barry SC HIS Manager Kankan 6 Mahamadou Sylla SC District Assistant Kouroussa 7 Dr Mory Touré SC NGO Advisor Kouroussa 8 Facely Kourouma SC District Coordinator Kouroussa 9 Fatoumata Diakité SC RH Advisor Kouroussa 10 Dr Aissata Condé DPS Director Kouroussa 11 Ansoumane Doumessy NGO GAAPE Supervisor Kouroussa 12 Adama Camara NGO AJVDM Supervisor Mandiana 13 Eric Swedberg SC CS Advisor Westport 14 Fatoumata Binta Barry SC Executive Assistant Kankan 15 Dr Amara Traoré DPS Director Mandiana 16 Eric Sarriot Consultant USA 17 Damou Rahim Keita SC Coordinator of Health Program Kankan 18 Dr Mamadou Oury Diallo SC M&E Coordinator Kankan CS-18 Guinea, Final Evaluation, Save the Children, December 2006 60 Annex 3 Final Evaluation Methodology The final evaluation of CS-18 was conducted according to the usual participative evaluation methodology of USAID/CSHGP. The project collected a significant amounts of data in the months prior to the evaluation. These studies informed the final analysis and are as follows: 1. Knowledge, Practices, and Coverage Surveys in Mandiana and Kouroussa (KPC 2006); 2. Evaluation of NGO partner capacity; 3. Evaluation of VHC capacity; 4. Qualitative Survey of the Youth in the Listening Centers; and 5. Review of the MURIGA (community funds for obstetrical emergencies). Other secondary data was also available: 6. Health Information System Monitoring Data of CS-18; 7. Demographic Health Survey of Guinea 2005; and 8. MOH Health information system/DPS. After the members of the evaluation team met with USAID in Conakry, the FE team, including the principle local partners (see Appendix 2), were invited to Kankan on September 12, 2006. During first days, the CS-18 team presented the results of the preliminary studies. This work allowed the joint development and prioritization of evaluation questions that needed to be addressed. The evaluation team then developed simple study tools, to guide individual interviews and group discussions. Two teams were sent to each district and led the following activities: - Discussions with VHC and AVHC members; - Discussions with health workers; - Discussions with mothers’ groups; - Discussions with TBAs; - Focus group discussions with peer educators; and - Discussions with the VHC nutritionists. In addition to these talks, the FE Team Leader independently led a visit and discussions with the directors of each DPS, and a focus group discussion with the NGO animators. The FE Team Leader also met with executives from the local NGOs. Upon return from the field visits, the evaluation team divided into subgroups to analyze information obtained and to present lessons learned and recommendations to the larger evaluation team. A first debriefing presentation was conducted with local partners on September 20, followed by a presentation in Conakry, with the Ministry of Health on September 22, 2006. CS-18 Guinea, Final Evaluation, Save the Children, December 2006 61 Annex 4 List of People Contacted 1 - Prefectoral Level Authorities: - Mr. Ibrahima KOITA, Secretary-General in charge of Administrative Affairs; - Mr. Mory KOMA, Director of Micro Realization; - Mr. Fankalo NANAMOU, Secretary-general of the Urban District; - Mr. Moussa DIAWARA, Manager of Community Organizations; - Mrs. Saran CAMARA, Secretary of the Prefect. 2 - Prefectoral Level Direction of Health - Dr. Aissata COP, Prefectoral Director of Health; - Dr. Odilon HABA, In charge of the Community Base Service; - Dr. Ibrahima Kalil KEITA, Doctor In charge of the Diseases; - Mr. Amara KOUROUMA, In charge of School Health; - Mrs. Damba MARA, Radio Operator. 3 - NGO Level – GAAPE - Mr. Ansoumane DIOUMESSY, Executive Director of NGO and Supervisor of the CS-18 project activities; - Mr. Famany TRAORE, Animator; - Miss Saran CAMARA, Animator; - Mr. Sayon KEITA, Animator; - Mr. Moro DIALLO, Administrative and Financial Director of NGO; - Miss Kany DIOUMESSY, Animator; - Mr. Laye Kouroussa CAMARA, Animator; - Mrs. Sabouyan CAMARA, Animator. 4 - Health Center Level - Kouroussa - Mrs. Marie TRAORE, Head of the Health Center; - Mr. Maxim KOLIE, Vaccinator. 5 - CVS Level - Koumana - Mr. Aly KEITA, Secretary and Community Agent; - Miss Nagnouma KEITA, Peer Educator; - Mr. Kéballa KOUROUMA, President VHC; - Mrs. Hawa KEITA, Nutritionist; - Mrs. Mariame KEITA, TBA - Mr. Kébama KOUROUMA, Treasurer COGES. 6 - USAID Level - Mrs. Elizabeth Kibour; Senior Health Sector Advisor and HIV TEAM Leader; USAID/Guinea - Mrs. Traoré Soukeinatou; Public Health Specialist CS-18 Guinea, Final Evaluation, Save the Children, December 2006 62 Annex 5 List of Participants in the Final Evaluation Debriefing NR NAME DEPARTMENT FUNCTION 1 Dr. Pogba GBANACE Ministry of Public Health Principle Private Secretary to the Minister 2 Dr. Mahi BARRY Ministry of Public Health National Director, Public Health 3 Dr. Sidaty KEITA Ministry of Public Health Head of the Division of Reproductive Health 4 Dr. Boubacar SALL Ministry of Public Health National Coordinator, Monitoring/Evaluation 5 Dr. Séré KABA Ministry of Public Health National Coordinator, Maternity Without Risk Program 6 Dr. Hawa TOURE Ministry of Public Health National Assistant Coordinator, Maternity Without Risk Program 7 Dr. Camille Tafsir SOUMAH Ministry of Public Health National EPI Coordinator 8 Dr. Did not gum ONOVOGUI Ministry of Public Health Head of the MCH Section 9 Mr. Damou Rahim KEITA Save the Children Coordinator, Health Programs 10 Mr. Eric SWEDBERG Save the Children CS Advisor 11 Mr. Eric SARRIOT Independent Consultant 12 Dr. Aissata COP Ministry of Public Health Prefectoral Director of Health Kouroussa 13 Dr. Amara TRAORE Ministry of Public Health Prefectoral Director of Health Mandiana 14 Dr. Richard JAMES Ministry of Public Health Regional Direction of Health Kankan 15 Dr. Delphine Nagnouma TOUNKARA Ministry of Public Health Division Promotion of Health 16 Dr. Faroumata TOURE Ministry of Public Health Responsible for Family Planning CS-18 Guinea, Final Evaluation, Save the Children, December 2006 63 Annex 6 Training Plan for the Health District of Mandiana THEME Prévus Atteints REFERENCE DUREE Formation SONU PERIODE PLAN DE FORMATION DISTRICT SANITAIRE DE MANDIANA (2002 - 2004) CIBLES SMI PARTICIPANTS VOLETS Personnel CS18/DPS 4 4 1 mois T1/A1 Document du participant/Fiche d'apprentissage en santé maternelle et néonatale/Guides du participant AS 33 32 12 jours T3/A1 Document du participant/Fiche d'apprentissage en santé maternelle et néonatale/Guides du participant Formation sur facteurs de risque au cours de la grossesse Animateurs 6 6 T1/A2 Les soins prénatals 5 jours Muriga/caisse communautaire Formation SONU Guide de formation des animateurs et superviseur ONG Guide de formation des animateurs et superviseur ONG NUTRITION SMI Alimentation de l'enfant de 0-35 mois La malnutrition protéino-énergétique Animateurs 6 6 T1/A2 Les micro nutriments et leur carence 5 jours Le FARN Alimentation de l'enfant de 0-35 mois 20 16 May-04 La surveillance nutritionnelle La malnutrition protéino-énergétique AS 20 16 5 jours T3/A2 Les micro nutriments et leur carence Le FARN Surveillance nutritionnelle des enfants de 0-35 mois Secrétaire des CVS 60 60 3 jours T2/A2 Guide de formation des AC sur la surveillance nutritionnelle des enfants de 0-36 mois Module de formation des AS en nutrition Guide de formation des animateurs et superviseur ONG Guide de formation des animateurs et superviseur ONG IMMUNISATION NUTRITION La vaccination des enfants Le calendrier vaccinal Animateurs 6 6 2 jours T1/A2 La vaccination des femmes enceintes Animateurs 6 Guide de formation des animateurs et superviseur ONG Guide de formation des animateurs et superviseur ONG 6 T1/A2 PF IMMUNISATION Importance de la planification familiale Les méthodes de PF disponibles en Guinée 5 jours Animateurs 6 Guide de formation des animateurs et superviseur ONG Guide de formation des animateurs et superviseur ONG 6 T1/A2 PF IST/VIH/SIDA Les infections sexuellement transmissibles Animateurs 6 6 Jun-04 Paires éducateurs 120 120 5 jours T2/A2 Curriculum de formation des pairs éducateurs/boîte à image santé jeune IST/SIDA Le VIH/SIDA Guide de formation des animateurs et superviseur ONG Curriculum de formation des AC en PCIME-C AUTRES IST/VIH/SIDA PCIME-C Animateurs 6 6 3 jours T3/A2 Nutritionnistes et AV des CVS 120 110 3 jours T3/A2 Curriculum de formation des AC en PCIME-C Personnel CS18 1 1 T4/A1 Animateurs 6 6 3 jours TA/A1 Secrétaire des CVS 60 60 3 jours T1/A2 Guide de formation des CVS en système de collecte des données démographiques et sanitaires au niveau village Gestion des caisses Trésorier des CVS 72 72 3 jours T2/A2 Technique de Supervision Animateurs 6 6 5 jours Guide de formation des animateurs et superviseur ONG Andragogie et Mobilisation sociale Animateurs 6 6 Guide de formation des animateurs et superviseur ONG T1/A2 SIS Guide d'utilisation des informations démographiques et sanitaires au niveau communautaire Curriculum de formation des AC en PCIME-C AUTRES PCIME-C CS-18 Guinea, Final Evaluation, Save the Children, December 2006 64 Prévus Atteints 10 jours SMI Formation en SONU AV 25 25 T2/A4 Curriculum AV PF Recyclage formel en distribution directe AC 266 266 3 jours T3/A4 Curriculum AC des contraceptifs oraux Planification Gestion 4jours/session T3,T4/A3 Modules Plaidoyer et mobilisation des ressources ACVS 12 12 en 4 sessions T1,T2/A4 RENFORCEMENT Gouvernance interne DE CAPACITES Formation en Word 5 jours en Modules Formation en Excel Equipe 2 sessions Formation en SIS Cadre 1 1 3 jours T2/A3 Modules Formation sur Outil Enquete Fait/Non fait District 4 4 5 jours T2/A4 ONG 1 1 10 jours en T2/A3 Modules Planification Strategique 2 sessions VOLETS THEME CIBLES DUREE REFERENCE 2 2 T1/A2 PLAN DE FORMATION DISTRICT SANITAIRE DE MANDIANA (2004 - 2006) PARTICIPANTS PERIODE CS-18 Guinea, Final Evaluation, Save the Children, December 2006 65 Annex 7 Training Plan for the Health District of Kouroussa PLAN DE FORMATION DISTRICT SANITAIRE KOUROUSSA (2002 - 2004) VOLETS TYPES DE FORMATION CIBLES Prevu Atteint DUREE PERIODE REFERENCE Orientation des formateurs AV Medecin chef maternite 1 1 3 jours T4/A1 Guide des Formateur en SONU et sages femmes 3 3 SMI Formation des superviseurs AV Agents PEV,CPS et 10 10 2 jours T4/A1 Guide de formation agent GAAPE 6 6 Fiche Nationale de supervision Formation des superviseurs SBC Agents PEV,CPS 10 10 5 jours T1/A2 agent GAAPE et 6 6 Curriculum Formateurs/SBC Conseiller d'ONG 1 1 PF Formation des CVS en PF Hommes 42 42 5 jours T1/A2 Femmes 42 42 Curriculum AC/SBC SMI Formation en SONU AV 42 42 10 jours T2/A2 Curriculum national AV Formation en collecte des donnees demo sanitaire au niveau communautaire Vice Presidents 42 42 3 jours SIS Presidents 42 42 T2/A2 curriculum SIS communautaire Formation des CVS Nutritionnistes 42 42 Curriculum nutrition Assainissements 42 42 5 jours T2/A2 Curriculum PCIME-C NUTRITION Formation des agents de Agents PEV 6 6 Curriculum nutrition Sante en nutrition Chefs de Postes 4 4 4 jours T2/A2 Curriculum PCIME-C IST/VIH/SIDA Formation Pairs educateurs Jeunes leaders 84 81 T3/A2 Curriculum Pairs educateurs Formation des Gestionnaires Presidents CVS 84 84 3 jours SMI MURIGA Tresorier CVS 84 84 T4/A2 Curriculum Gerant Muriga VACCINATION L' importance vaccination VA CVS 42 42 Les maladies cibles du PEV Nutritionniste CVS 42 42 2 jours T2/A2 Curriculum AC/Poster PEV Le calendrier vaccinal Agents assainissement 42 37 Les effets secondaires CS-18 Guinea, Final Evaluation, Save the Children, December 2006 66 PLAN DE FORMATION DISTRICT SANITAIRE KOUROUSSA (2004 - 2006) VOLETS TYPES DE FORMATION CIBLES Prevu Atteint DUREE PERIODE REFERENC E Formation des superviseurs AV 3 jours T1/A3 Guide de formation SMI Agents PEV,CPS et 6 6 Fiche Nationale de supervision Formation des superviseurs SBC Agents PEV,CPS 10 10 5 jours T1/A3 Curriculum SBC PF Formation des CVS en PF Hommes 38 38 5 jours T1/A3 Femmes 38 38 Curriculum AC/SBC Formation en collecte des donnees demo sanitaire au niveau communautaire Vice Presidents 38 38 3 jours SIS Presidents 38 34 T2/A3 curriculum SIS communautaire Formation en nutrition et PCIME-C Nutritionnistes 38 38 Curriculum nutrition Assainissements 38 33 5 jours T2/A3 Curriculum PCIME-C NUTRITION Formation des agents de Agents PEV 6 6 Curriculum nutrition Sante en nutrition Chefs de Postes 2 2 4 jours T2/A2 Curriculum PCIME-C IST/VIH/SID A Formation Pairs educateurs Jeunes leaders 76 73 T3/A3 Curriculum Pairs educateurs Formation des Gestionnaires Presidents CVS 38 34 SMI MURIGA Tresorier CVS 38 38 3 jours T4/A3 Curriculum Gerant Muriga Formation en SONU AV 38 40 10 jours T2/A3 Curriculum national AV VACCINATION L' importance vaccination AV CVS 38 38 Les maladies cibles du PEV Nutritionniste CVS 38 38 2 jours T2/A3 Curriculum AC/Poster PEV Le calendrier vaccinal Agents assainissement 38 33 Les effets secondaires RECYCLAGE Distribution des FORMEL EN PF contraceptifs oraux en Agents communautaires 160 158 3jours T2/A4 Curriculum AC premiere intension Formation en Word 10 jours Formation en Excel Equipe cadre District 3 3 en 2 sessions T2/A3 Modules outil informatique SIS 3 jours T1/3 Module SIS Techniqu Enquete Fait/ 5 jours T2/A4 Outil Fait/Non fait Non fait RENFORCEMENT DE Planification et gestion CAPACITES Paldoyer et mobilisation ACVS 12 12 5 jours/session T2,T3/A4 Modules des ressources en 3 sessions Gouvernance interne Planification strategique ONG 1 1 5jours/session T2/A3 Modules en 2 sessions CS-18 Guinea, Final Evaluation, Save the Children, December 2006 67 Annex 8 Analysis of Death Avoided (According to the Lancet and thanks to Jim Ricca) CS-18 Guinea, Final Evaluation, Save the Children, December 2006 68 Annex 9 Interventions with the Greatest Estimated Impact CS-18 Guinea, Final Evaluation, Save the Children, December 2006 69 Annex 10 Final Evaluation Photographs A project leader encouraged by the results presented and analyzed by the partners and members of the project team. Mandiana Reststop (to recover from the bumps on the way) and the beautiful road of Kouroussa. A success for health. A team strategy: Health/DPS, health agents, community agents, VHC members VHC, peer-educators, NGO animators, VHC nutritionist, and district coordinator of Save the Children… In Kouroussa as in Mandiana, the Save the Children team works within the offices of the DPS. A support for stock-outs of management tools was extremely appreciated. CS-18 Guinea, Final Evaluation, Save the Children, December 2006 70 Equipment for the health centers: scales and suggestion boxes for quality improvement. Posting of unfortunately irrational costs of services: cost of purchase of the sachets SRO = 450 FG; posted cost of sale = 200 FG! With that, stock-outs will continue. Local NGOs, GAAPE and AJVDM, organize, develop their capacity and project towards the future. Irreplaceable resources: executives and animators of the local NGOs, and the famous means of transport. Analysis and compilation of various sources of information by the teams. CS-18 Guinea, Final Evaluation, Save the Children, December 2006 71 Debriefing and development of the final recommendations: in Kankan, then in Conakry. The debriefing for the MOH was organized by the DRS, with the two DPS for Mandiana and Kouroussa. CS-18 Guinea, Final Evaluation, Save the Children, December 2006 72 Annex 11 Explanation of the Index “Percent of Deficit Filled” Figure A illustrates the principle of construction of this index: • Indicator X progresses from Phase 1 to Phase 2, from 20% to 40%. The deficit to be filled was of: ƒ (100% - 20%) = 80%. The percentage of this made up deficit is of: ƒ (40% - 20%)/80% = 25% • The indicator Y begins on a level much higher (80%). As it is frequently the case, it is more difficult to advance an indicator already on this level. In this example indicator Y is measured at 85% in Phase 2. The deficit to be filled was of: ƒ (100% - 80%) = 20% The percentage of this made up deficit is of: ƒ (85% - 80%)/20% = 25%. • The last example is the indicator Z, which increased by 10% to 55% and thus filled 50% of the deficit which was (100% - 10%) = 90%. CSTS frequently uses this index in its analyses for USAID. Unpublished work analyzing the results of many projects shows that the percentage of recovered deficits progresses independently of the initial level of the indicators. One commonly observes in projects financed by USAID/CSHGP, a percentage reduction of the deficit by 30%, regardless of the starting level of the indicators. • In the example of Figure A, progress carried out between the various indicators is compared. Progress for the first two indicators is equal, with a filling of 25% of the deficit in both cases. The performance for the last indicator is much better, since one observes a reduction of 50% of the deficit to be addressed. Figure A: Illustration of the principle of the percentage of deficit to address 20% 40% 25% 80% 85% 25% 10% 55% 50% 0% 20% 40% 60% 80% 100% Level Phase 1 Level Phase 2 % to be filled between Phase 1 & 2 Indicator Indicator Indicator 1 See reviews of CSHGP progam. 2 Performance Index analysis. Bill Weiss. CSSP. Unpublished. 1999. CS-18 Guinea, Final Evaluation, Save the Children, December 2006 73 Annex 12 Summary of Project Progress CS-18 Guinea, Final Evaluation, Save the Children, December 2006 74 CS-18 Guinea, Final Evaluation, Save the Children, December 2006 75 Annex 13 Results Highlight Maternal mortality in Guinea, which was 528 in 1999, has jumped to 980 per 100,000 live births in the 2005 DHS. This figure reflects an overall increase in adult mortality and is also attributable to the failure of the health system to provide adequate prenatal, emergency obstetrical care, and post-partum care coupled with poor nutritional status of women in their reproductive years. Indications are that causes of maternal mortality are similar to other developing countries, including hemorrhage, abortion, and sepsis. SC has worked in two districts of Upper Guinea over the last ten years to developed effective interventions for addressing several of the major causes of maternal and newborn mortality. The recent USAID-funded CS-18 child survival project included maternal and newborn care interventions that have been universally accepted by community residents, religious leaders, village authorities, health providers and traditional practitioners. Although demand for antenatal care was already relatively high (68%), the project increased demand through educational activities, so that now 91% of women benefited from at least two antenatal consultations. Rapid recognition of danger signs is important in order to get a mother to a higher level of care as needed. Educational activities sought to increase the proportion of women able to recognize at least two danger signs and this indicator more than doubled from 32% to 68%, over the life of the project. At the start of project activities, only about a fifth (19%) of births were attended by a skilled health care provider, and 45% were attended by trained TBAs, while about a third delivered at home without any trained assistance. The CS18 project taught obstetrical skills to 59 health workers as well as the basics of clean delivery and community life-saving skills to 107 TBAs. Now, 43% of births are attended by skilled providers, 56% by trained TBAs, and less than 1% lack any trained assistance. At the beginning of the project, postpartum care was almost unheard of in this region of Guinea. CS￾18 developed an early postpartum care program (EPP) that includes the use of trained TBAs as links who are closely associated with the village health committee and skilled providers. The EPP care includes: (1) recognizing and referring for danger signs of hemorrhage and infection; (2) supporting exclusive breastfeeding; (3) recognizing and treating common breast conditions, such as engorgement and mastitis; and 4) support of and referral to the MOH for family planning. At the end of the project, 58% of women received two postnatal consultations. SC and its partners established a health insurance scheme to support both the transportation and health care costs of obstetrical emergencies. A total of 171 schemes called “MURIGAs” were established and have been used to cover the costs of 131 of a total of 931 obstetrical emergencies. The insurance schemes are managed by the local village health committees and provide timely financial support to families. These women would most likely not have received referral care without this support. The village health information system established by the project indicates a steady and dramatic decrease in the number of maternal deaths from 82 in the first year to about 22 in the last year. Although incomplete, this data does suggest the project interventions are effectively addressing the maternal mortality crisis in this area of Guinea. CS-18 Guinea, Final Evaluation, Save the Children, December 2006 76 Annex 14 Updated CSHGP Data Form CS-18 Guinea, Final Evaluation, Save the Children, December 2006 77 CS-18 Guinea, Final Evaluation, Save the Children, December 2006 78 CS-18 Guinea, Final Evaluation, Save the Children, December 2006 79 CS-18 Guinea, Final Evaluation, Save the Children, December 2006 80 Child Survival and Health Grants Program Project Summary Dec-06-2006 Save the Children (Guinea) General Project Information: Cooperative Agreement Number: FAO-A-00-98-00024-00 Project Grant Cycle: 18 Project Dates: (9/30/2002 - 9/29/2006) Project Type: Cost XT SC Headquarters Technical Backstop: Eric Swedberg Field Program Manager: Dunni Goodman Midterm Evaluator: Final Evaluator: Eric Sarriot USAID Mission Contact: Field Program Manager Information: Name: Dunni Goodman Address: Phone: Fax: E-mail: dgood1980@yahoo.com  Alternate Field Contact: Name: Damou Kieta Address: Kankan  Phone: 224 710725  E-mail: savegnea@sotelgui.net.gn  Funding Information: USAID Funding:(US $): $1,400,000  PVO match:(US $) $416,675  Project Information: Description: This cost extention program will scale up the program’s goals of: 1) a sustained reduction in under-five and maternal mortality in the two health districts; and 2) a sustained improvement in the nutritional status of 0 to 36 month-old children.  It proposes to continue the efforts in the following intervention areas; immunization, nutrition and micronutrients, maternal and newborn Care, and HIV/AIDS.  The programs interventions will be implemented through the following strategies:  1) Joint DRS, DPS and SC design, implementation, and evaluation of approaches to maternal and child health in the districts that inform strategies of other PVOs and the MOH.  2) Improving the technical, financial, and institutional capacity of two local NGOs.  3) Mobilizing communities through Village Health Committees and Health Center Management Committees to improve essential health services and conduct focused education activities.  4)Improving the quality of health services provided in the community and the health facilities. Location: Districts of Kouroussa(expanded area for CS XVIII) and Mandiana(original project site in CS XIV project)  Project Partners Partner Type Subgrant Amount Support Group for the Self Promotion of the Land and the Protection of the Environment Subgrantee $40,000.00  The Association of Young Volunteers of Mandiana Subgrantee $40,000.00  Subgrant Total $80,000.00 General Strategies Planned: Private Sector Involvement Advocacy on Health Policy Strengthen Decentralized Health System Information System Technologies  M&E Assessment Strategies: KPC Survey Organizational Capacity Assessment with Local Partners Participatory Rapid Appraisal Community-based Monitoring Techniques Participatory Evaluation Techniques (for mid-term or final evaluation)  Behavior Change & Communication (BCC) Strategies: Mass Media Interpersonal Communication Peer Communication Support Groups  Groups targeted for Capacity Building: PVO Non-Govt Partners Other Private Sector Govt Community CS Project Team  Local NGO Traditional Healers Dist. Health System Health Facility Staff Health CBOs CHWs  Interventions/Program Components: Immunizations (10 %)  (IMCI Integration)  (CHW Training)   - Polio   - Classic 6 Vaccines   - Vitamin A   - Surveillance   - Mobilization   - Community Registers Nutrition (15 %)  (IMCI Integration)  (CHW Training)   - ENA   - Comp. Feed. from 6 mos.   - Hearth   - Cont. BF up to 24 mos.   - Growth Monitoring   - Maternal Nutrition Vitamin A (5 %)  (IMCI Integration)  (CHW Training)   - Supplementation   - Post Partum   - Integrated with EPI Micronutrients (5 %)  (CHW Training)   - Iron Folate in Pregnancy Maternal & Newborn Care (35 %)  (IMCI Integration)  (CHW Training)  (HF Training)   - Emerg. Obstet. Care   - Neonatal Tetanus   - Recog. of Danger signs   - Post partum Care   - Delay 1st preg Child Spacing   - Integr. with Iron & Folate   - Normal Delivery Care   - Birth Plans   - Home Based LSS   - Control of post-partum bleeding   - Emergency Transport Child Spacing (10 %)  (CHW Training)   - Child Spacing Promotion   - Pre/Post Natal Serv. Integration HIV/AIDS (20 %)  (CHW Training)   - Behavior Change Strategy   - Access/Use of Condoms   - ABC Target Beneficiaries: Infants < 12 months: 17,080 Children 12-23 months: 14,689 Children 0-23 months: 32,299 Children 24-59 months: 51,242 Children 0-59 months: 85,402 Women 15-49 years: 106,753 Population of Target Area: 393,060 Rapid Catch Indicators: Indicator Numerator Denominator Percentage Confidence Interval Percentage of children age 0-23 months who are underweight (-2 SD from the median weight-for-age, according to the WHO/NCHS reference population) 89 563 15.8% 4.5 Percentage of children age 0-23 months who were born at least 24 months after the previous surviving child 290 397 73.0% 9.5 Percentage of children age 0-23 months whose births were attended by skilled health personnel 334 597 55.9% 7.2 Percentage of mothers of children age 0-23 months who received at least two tetanus toxoid injections before the birth of their youngest child 311 402 77.4% 9.5 Percentage of infants age 0-5 months who were exclusively breastfed in the last 24 hours 144 206 69.9% 13.0 Percentage of infants age 6-9 months receiving breastmilk and complementary foods 67 128 52.3% 15.2 Percentage of children age 12-23 months who are fully vaccinated (against the five vaccine-preventable diseases) before the first birthday 91 162 56.2% 13.8 Percentage of children age 12-23 months who received a measles vaccine 88 162 54.3% 13.7 Percentage of children age 0-23 months who slept under an insecticide-treated bednet the previous night (in malaria-risk areas only) 376 597 63.0% 7.5 Percentage of mothers who know at least two signs of childhood illness that indicate the need for treatment 346 597 58.0% 7.3 Percentage of sick children age 0-23 months who received increased fluids and continued feeding during an illness in the past two weeks 296 335 88.4% 10.6 Percentage of mothers of children age 0-23 months who cite at least two known ways of reducing the risk of HIV infection 357 585 61.0% 7.5 Percentage of mothers of children age 0-23 months who wash their hands with soap/ash before food preparation, before feeding children, after defecation, and after attending to a child who has defecated 21 597 3.5 % 2.1 Comments for Rapid Catch Indicators