1 Plan USA, INC. D/b/a CHILDREACH Plan MALI: CHILD SURVIVAL XVII PROJECT Cooperative Agreement No: HFP-A-00-01-00046-00 FINAL EVALUATION REPORT - YEAR 5 ® IMPLEMENTING AGENCY: Plan Mali In partnership with Ministry of Health Mali, UNICEF and Local NGOs ® LOCATION: Mali Kita District in Kayes Region ® CONTACT PERSON: National Executive Director, Childreach Plan USA INC. 155, Plan Way, Warwick, Rhode Island 02886 ® Project Beginning Date: September 30th 2001 Ending Date: September 29, 2006 Submission Date; December 31st, 2006 Prepared by: Marguerite Joseph, Consultant Reviewed and Edited by: Pierre-Marie Metangmo, Laban Tsuma and Plan Mali Team 2 ACKNOWLEDGEMENTS The principal author of this report would like to express her appreciation to the Plan staff who gave so generously of their time and effort to improve the situation of the women and children of Kita District in Mali. The author is especially grateful to those who supported the FE team during the evaluation process and made the work possible. The leadership and support of Supriyanto, Mali Country Director, Casimir Youmbi, Program Support Manager, Cheik Magasouba, Program Unit Manager, Dr. Souleymane Bagayoko, Plan Mali National Health Advisor, and Dr. Sita Sidibe, CS Project Coordinator facilitated this work tremendously. Additional thanks goes to those who conducted the final surveys and quantitative tools found in the FE: Suleiman Bagayoko, Sita Sidibe and the entire CS team. Thanks also go to Dr. Pierre Marie Metangmo, Plan’s Health Programs Coordinator and Dr. Laban Tsuma, Plan’s Health Associate who greatly enhanced the Final Evaluation process. 3 LIST OF ACRONYMS ACSD Accelerated Child Survival and Development AMACO Local NGO ANICT National Agency of Investment in the Collectivity of the Territory ARAFD/C Local NGO ASACO Health Facility Management Committee/Community Health Associations ARI Acute Respiratory Infection ANC Antenatal Care BASICS Basic Support for Institutionalizing Child Survival BCC Behavior Change Communication BCG Bacille Camlette Guerin CBIS Community Based Information Systems CDD Control of Diarrheal Disease CHW Community Health Worker C-IMCI Community and Household Integrated Management of Childhood Illness CM Community Mobilization CMR Child Mortality Rate CS Child Survival CSComs Primary Health Facility CSHGP Child Survival Health Grants Program CSP Child Survival Project CSSA Child Survival Sustainability Assessment CSTS Child Survival Technical Support CSV/VHC Committee de Sante Villageous/Village Health Committee DDC District Development Committee DHO District Health Office DHS Demographic and Health Survey DIP Detailed Implementation Plan DPT Diphtheria Pertusis Tetanus EPI Expanded Program on Immunization GSM Grants Support Manager HF/HC Health Facility/Health Center HFA Health Facility Assessment HIS Health Information System HKI Helen Keller Foundation HW Health Worker IEC Information, Education, Communication 4 IMCI Integrated Management of Childhood Illness IMR Infant Mortality Rate INGO International Non-Government Organization ITN Insecticide Treated Net KPC Knowledge Practices and Coverage LQAS Lot Quality Assurance Sampling MAP Minimum Activity Package M&E Monitoring and Evaluation MOH Ministry of Health MTE Mid Term Evaluation NGO Non Governmental Organization OR Operations Research ORS/ORT Oral Rehydration Solution/Therapy PC Project Coordinator PCM Pneumonia Case Management PDRIK Kita Integrated Rural Development Project PHC Primary Health Care PLA Participatory Learning and Action PSM Program Support Manager PUM Program Unit Manager PVO Private Voluntary Organization RBM Roll Back Malaria RHF Recommended Home Fluid SCM Standard Case Management SP Sulphadoxime Pyrimethamine TA Technical Assistance TBA Traditional Birth Attendant TOT Training of Trainers TT Tetanus Toxoid UNICEF United Nations Children’s Fund USAID United States Agency for International Development WHO World Health Organization 5 TABLE OF CONTENTS Page A. EXECUTIVE SUMMARY.................................................................................6 B. ASSESSMENT OF RESULTS AND IMPACT OF THE PROGRAM ...............9 1. RESULTS: SUMMARY CHART.................................................................9 2. RESULTS: TECHNICAL APPROACH ....................................................12 3. RESULTS: CROSS-CUTTING APPROACHES…………………………….28 a. Community Mobilization………………………………………………… 28 b. Communication for Behavior Change………………………………….33 c. Capacity Building Approach…………………………………………… 35 d. Sustainability Strategy……………………………………………………46 C. PROGRAM MANAGEMENT.........................................................................49 1. PLANNING…………….............................................................49 2. STAFF TRAINING....................................................................51 3. SUPERVISION OF PROGRAM STAFF...................................52 4. HUMAN RESOURCES AND STAFF MANAGEMENT ............53 5. FINANCIAL MANAGEMENT ...................................................55 6. LOGISTICS……………. ...........................................................56 7. INFORMATION MANAGEMENT .............................................56 8. TECHNICAL AND ADMINISTRATIVE SUPPORT ..................59 9. MISSION COLLABORATION…………………………………… 60 9. MANAGEMENT LESSONS LEARNED...................................60 D. OTHER ISSUES IDENTIFIED BY TEAM ......................................................61 E. CONCLUSIONS AND RECOMMENDATIONS .............................................61 F. RESULTS HIGHLIGHT..................................................................................66 ATTACHMENTS.................................................................................................67 A. Evaluation Team Members and their titles B. Evaluation Team Members and their titles C. List of persons interviewed and contacted D. Project Data Sheet form – updated version 6 A. Summary Plan International implemented a USAID funded Child Survival project in partnership with the Mali Ministry of Health, and local municipalities in the project area during the last five years. The CS project was located in Kita District, in the southeastern part of Kayes Region, in the republic of Mali with a total population of 315,5201 people. The project focused of 23 of 26 health areas within the catchment area of functioning community health centers. The project interventions were: 1) Malaria (35%) 2) Immunization (35%) 3) Diarrhea (20%) 4) Pneumonia (10%) The program approach used was to help strengthen the existing health care system. This approach comprised of two strategies: 1. Training and building the capacity of health personnel and supporting community health Associations (ASACOs) in order to increase utilization of health care services; 2. Supporting IMCI implementation in health facilities (CSComs). As per the findings of the final evaluation activity, the main accomplishments of the program include the following: 1. Significant reduction in child morbidity and mortality due to malaria, pneumonia and diarrhea. (Per qualitative interviews, CHWs and KPC data). 2. Strengthening of EPI ‘Strategy Avance’ outreach activities and integration of prenatal care as part of the outreach service. 3. Increased use of Insecticide Treated Nets and household coverage by ITNs in project area. 4. Marked improvement in hygiene and sanitation conditions in project households and communities. 5. Improved quality of health services. 6. Establishment of a functioning community based health information system. 7. Strengthened linkages between communities and health facilities. 8. Increased use of health services by project populations. 9. Community mobilization and participation for maternal and child health. 10. Capacity building of community based organizations and resource persons. 11. Increased knowledge and improved health practices of mothers in project communities. 12. Strengthened technical capacity of local NGOs to implement health and children survival projects. 13. Strengthened technical capacity of all local partners in monitoring and evaluation. 1 RACE: Recencement Administrativ á Caractére Electoral, Kita District, 2001, Ministère de l’administration territoriale et des collectivités. 7 14. The CSP was able to demonstrate the vision of partnership dynamic whereby stakeholders at every level played an important role in the achievement of a common goal and objectives. Highlights from the comparison of the baseline and final evaluation surveys: In Malaria, two of the most critical indicators achieved results that were above project targets: 1) Children 0-23 months who slept under an impregnated bednet the night prior to the survey went from an extremely low 3.9% at the KPC baseline to an incredibly desirable coverage of 98.1% at the final; and 2) care-seeking by mothers to health facilities which was 59.1% at baseline and went up to 83.7% at the final. The proportion of <5 year olds who completed a full treatment for malaria also saw very good results at 98.1%, although there is no baseline or target to compare it with. The proportion of community health workers (relais) who know when to refer children with fever went from 0% at baseline to 72.5% at the final. The proportion of CSCOM (health facility) staff trained was 95.7%. Two indicators targeting mothers for malaria also saw big improvements: the proportion of mothers attending ANC who received iron/folate supplementation during the last pregnancy went from 65% at baseline to 73.5% at the final; and the mothers of children 0-23 months who reported taking chloroquine/SP during the last pregnancy also went from 55% to 76.8%. Under Immunization and Vitamin A, except for one indicator: the proportion of children 6-59 months who received Vitamin A twice in the past 12 months – noted at only 50% at the final (as compared to the 60% target), all others surpassed the objectives set by the project. For, percentage of children 12-23 months completely vaccinated by the first birthday, the project saw a change from 51% at the baseline to 77.2% at the final. Likewise the project saw a great reduction in the drop out rate, with the proportion of children 12-23 months with DPT 1 minus those with DPT 3 going from 49% at baseline to 18.9% at the final. The proportion of children 12-23 months with measles vaccine also did very well, beginning with 51% at baseline and ending with 83.5% at final, surpassing the 80% target. For Diarrheal Disease Control, when it came to the proportion of children 0-5 months who received only breast milk in the 24 hours before the survey, there was a big improvement from the baseline 11.6% to 62.4% at the final. Although there was no baseline data for the proportion of mothers able to site at least 2 methods of preventing diarrhea, the final was 67.6%, almost reaching the project target of 70. But the results for diarrhea treatment with ORS or home-based fluids, seem to indicate that there has been no change (25.2% at baseline and 25.5% at final). This was due to a supply issue. Also noted in the KPC was tremendous success in related IMCI indicators, although not included in the DIP objectives. In Pneumonia Case Management the project achieved two out of three objectives. When it came to the proportion of children 0-23 months with fast/difficult breathing taken to a health facility, the project contributed to an enormous improvement, as it was 20% at the baseline and jumped to 60% at the final. The proportion of fever cases referred to the health facility (overlapping symptom with malaria) was 59.1% at baseline and jumped to 83.7% at final. 8 In conclusion, the final KPC survey and other data show that the Kita CSP achieved the majority of its objectives. It is important to note that in terms of achievement of objectives, behavior change was not a problem. Most objectives not achieved, were related to coverage. i.e service and supply challenges that were outside the direct control of the project. The result of project assessments in terms of capacity of partners shows that a lot of progress was made. Health committees are functioning well and achieved promising scores in the Child Survival Sustainability Assessment). Qualitative interviews with community members and partners at all levels revealed strong and positive sentiments from all parties about the project. Per the latter, the project contributed to overall community health and hygiene; increased access and coverage; increased knowledge and care-seeking; community ownership and participation; technical and managerial capacity in health services; technical capacity for the development and maintenance of a community based information system, data analysis, planning, decision-making, monitoring and evaluation. The free net policy in the second half of the project was a major boost to Kita. This collaboration effort of the MOH, UNICEF and the CSP resulted in a unique opportunity for supply to meet demand that had already been created through community education sessions, home visits and interpersonal contact with relais, VHCs and project animators. 9 B1. Assessment of Results and Impact of the Program 1. Results: Summary Chart Key Indicators Baseline KPC 2001 LQAS 2004 Final KPC, service records, reports 2006 Target/ Objective Malaria Case Management (35%) Children 0-23 months who slept under an impregnated Bednet the night prior to the survey 3.9 22.5 98.1 60 Proportion of fever cases referred to the CSComs (i.e. mothers who sought care at health facility) 59.1 83.7 75 Proportion of <5 year olds who completed a full treatment for malaria ---- ---- 98.1 Proportion of relays who (know when to) refer children with fever 0% --- 72.5 80 Proportion of HC staff trained ---- --- 95.7 100 Proportion of mothers attending ANC who received iron/folic acid supplementation during last pregnancy 65 79.6 73.5 80 Mothers of children 0-23 months who reported taking Chloroquine/SP during the last pregnancy 55 82.2 76.8 80 Immunization/Disease Surveillance (35%) Percentage of children 12-23 months completely vaccinated by the first birthday 51 69.5 77.2 71 Proportion of children 12-23 months with DPT 1 minus those with DPT 3 (drop-out) 49 18.9 20 Proportion of children 12-23 months with measles vaccination 51 83.5 80 Proportion of children 6-59 months who received Vitamin A twice in the past 12 month 50 60 Mothers of children 0-23 months who reported receiving at least 2 doses of TT during their last pregnancy 24.1 83.9 70 Proportion of women delivering at the CSComs who receive one dose of Vitamin A (200,000 IU) before leaving the center 97.2 60 Proportion of cases of measles polio, neonatal tetanus reported to the CSCOMs 100% 100% Response of the EPI CSCom to reported data 100% 100% 10 Key Indicators Baseline KPC 2001 LQAS 2004 Final KPC, service records, reports 2006 Target/ Objective Control of Diarrheal Disease (20%) Proportion of children 0-5 months who received only breast milk in the 24 hours before the survey 11.6 62.4 60 Proportion of mothers able to site at least 2 methods of preventing diarrhea 67.6 70 Proportion of cases of diarrhea in children who were treated with ORS or home-based fluids 25.2 25.5 70 Proportion of HC staff, relays and ASACOs trained (same as above) 100% Relais; 95% HW; 0% ASACOS 100 Pneumonia (10%) Proportion of children 0-23 months with fast/difficult breathing who are taken to a health facility 20 60 40 Proportion of fever cases referred to the CSCom (same as above – presumption of malaria) 59.1 83.7 75 Proportion of CSCom staff trained in PCM (same as above) 95.7 100% Proportion of CSComs with 10 essential drugs in stock during the last six months (including antibiotics) 0 (range 3 -9) 33.3% (range 7 to 10 drugs available) ---- Additional Indicators not in Detailed Implementation Plan (IMCI, Maternal & Neonatal Health) Mothers of children 0-23 months who know 2 signs indicating that a child needs care 35.3 90.6 87.8 60 Children 0-23 months who were sick who received at least the same amount, or more than usual, to drink 26.6 68.5 85.8 60 Children 0-23 months who were sick who received at least the same amount, or more than usual, to eat 21 50.3 86.3 60 Children 0-23 months who have an Immunization Card 49 90.7 94.6 80 Mothers of children 0-23 months with a prenatal card 14 84.9 70.3% 60 Mothers of children 0-23 months who reported attending at least three ante-natal consultations during their last pregnancy 14 55.9 70.3 60 Mothers of children 0-23 months who reported having been attended by a trained health worker/TBA during the last childbirth 68 65.1 73 80 Percent of children who received breast milk within the first hour of birth 79.7 51.5 90 Mothers of children 0-23 months who delivered at the CSCOM reported receiving Vitamin A 40.1 97.2 60 11 Key Indicators Baseline KPC 2001 LQAS 2004 Final KPC, service records, reports 2006 Target/ Objective Capacity Building Number of ASACOs that held their statutory general assemblies during the past year 62.5 75 75 Number of statutory general assemblies held during the past year ---- 78.3 --- Number of ASACOs that held their statutory office meetings during the past year 12.5 54.6 75 Number of statutory office meetings that were held during the past year --- 85.4 70 Number of ASACOs that have financial reports showing debits and credits 0 100 75 Number of ASACOs that have women in high positions 0 30.4 ---- Number of women holding high positions 0 8 46 Number of CSComs that bought drugs with funds from the system of cost recovery from the past year 100 75 Percentage of users that say they are in general satisfied with the health services received from the structures 95.5 94.4 40% increase Number of NGO partners that have a document in which is consigned vision, mission and plan of action 100 100% Number of NGO partners with training programs for their personnel 0 Number of NGO partners with assessment results on their organization capacity 100 100 100 Number of NGO partners with complete financial reports: all the receipts and payments recorded 100 100 Number of NGO partners with management plans for the next year (provisional budget) 100 100 Existence of a training plan for CS personnel executed through regular training of personnel during the life of the project Yes Operations research will be designed for at least two CSP intervention areas Yes Note: Although there are only a total of 8 women with high posts in the ASACOs, other women do make up part of the membership and participate in activities. Challenges include workload at home; husbands consent and low literacy levels. 12 B2. Results: Technical Approach a. Overview of the project Plan International implemented a Child Survival project in partnership with the Mali Ministry of Health, and local municipalities in the project area during the last five years. The CS project is located in Kita District, in the southeastern part of Kayes Region, in the republic of Mali with a total population of 315,5202 people. During the life of the project, it phased a total of 23 of 26 functional health areas within the district. The project interventions fell under the following categories: 5) Malaria (35%): Reduction of mortality and morbidity associated with malaria in children and pregnant women, through prevention education, promotion of use of impregnated bed nets, improved treatment of malaria, and prenatal chemo-prophylaxis. 6) Diarrhea (20%): Reduction of diarrhea-associated mortality and morbidity through a) teaching and promoting prevention measures in the home, b) strengthening mothers’ capacity to recognize and provide home treatment for mild diarrhea with fluid and dietary management, and c) strengthening mother’s capacity to identify signs of moderate and severe diarrhea, know sources of care, take the child for care, and comply with health provider recommendations. 7) Immunizations (35%): Increased immunization coverage in the program area for all infants by the end of the first year of life, and tetanus toxoid (second dose) immunization for pregnant women, increased measles vaccination (including twice yearly vitamin A) and prevention of measles-caused diarrhea and death. 8) Pneumonia (10%): Reduction of pneumonia-associated mortality through prompt, appropriate and standardized pneumonia case management, a sustainable supply of antibiotics at the health centers (CSComs), prompt recognition by relays of pneumonia signs (fever, fast breathing) and referral of suspected cases to the nearest health center, and appropriate mothers’ care-seeking behavior. The program approach used was to help strengthen the existing health care system. This comprised of two strategies: 1. Training and building the capacity of health personnel and supporting community health Associations (ASACOs) in order to increase utilization of health care services; 2. Supporting IMCI implementation in health facilities (CSComs). b. Progress by Intervention area Control of Malaria i. Results and outcomes 2 RACE: Recencement Administrativ á Caractére Electoral, Kita District, 2001, Ministère de l’administration territoriale et des collectivités. 13 Per the above Summary table, it is clear that the project achieved improvements in every indicator for this component, even though it did not achieve all of its objectives. Two of the most critical indicators achieved results that were above project targets: 1) Children 0-23 months who slept under an impregnated bednet the night prior to the survey went from an extremely low 3.9% at the KPC baseline to an incredibly desirable coverage of 98.1% at the final, surpassing the 60% target; and 2) care-seeking by mothers to health facilities which was 59.1% at baseline and went up to 83.7% at the final, surpassing the project target of 75%. Undeniably, a third indicator: Proportion of <5 year olds who completed a full treatment for malaria also saw very good results at 98.1%, although there is no baseline or target to compare it with. Another area of great success for this component relates to knowledge and capacity. i.e. the proportion of community health workers (relais) who know when to refer children with fever went from 0% at baseline to 72.5% at the final (although slightly below the target 80%); and the proportion of CSCom (health facility) staff trained was 95.7%, although again, slightly below the project target of 100%. Two indicators targeting mothers for malaria also saw big improvements: the proportion of mothers attending ANC who received iron/folate supplementation during the last pregnancy went from 65% at baseline to 73.5% at the final, fairly close to achieving the 80% target; and the mothers of children 0-23 months who reported taking chloroquine/SP during the last pregnancy also went from 55% to 76.8%. This was only slightly short of the project target of 80%. Related maternal health indicators also monitored by the project although not included in the DIP were Mothers of children 0-23 months who reported attending at least 3 antenatal consultations during their last pregnancy went from 14% at baseline to 70.3% at the final, surpassing the 60% target; and mothers who reported having been attended by a trained assistant during the last childbirth went up slightly from 68% at baseline to 73% at final, not quite meeting the project objective of 80%. ii. Factors affecting achievement of program objectives and outcomes. The improvement seen in all of the malaria indicators is a result of several factors, including the community mobilization (CM) strategy used by the CSP. Establishing community entities such as the village health committees (locally known as Committee de Sante Villageous), and supporting the identification and training of community health workers (known as Relais), was key to bringing health messages down to community members. Project Animators from the local partner NGOs working in target communities played a key role in supporting community mobilization and the technical aspects of activities. The mobilization of the above community resource persons and subsequent training in malaria and other intervention areas enabled the project to not only reach mothers and promote behavior change, but increase the populations’ access to advice and care-seeking. Thus the project’s CM strategy and capacity building of community resource persons, culminating in IEC/BCC at the community level, contributed to improved malaria case management. Directly reflected in the results above, project messages being promoted by community volunteers who had continuous contact with community mothers, made it difficult for a mother not to know what the treatment protocol for malaria is, and difficult for a mother to delay care-seeking when she was always being advised not to. The 14 presence of community health workers allowed for the establishment of a referral system. According to the FGDs with women and the community volunteers themselves, Relais are consulted in cases of illness, and thus are in a position to refer cases that need to be referred. And to further ensure completion of treatment, they actually take the initiative to follow up the cases that they have referred. (Often the health worker will write a note back to the Relais and instruct the client to pass it on to him/her). As was discussed in the project DIP, myths around malaria were rampant, and use of traditional treatment for symptoms of malaria was common practice. Thus, it is clear from the results that the above-mentioned project strategies helped to address this issue. With regard to the success in the area of ITN coverage, this was in large part due to the additional injection of approximately 45,000 Roll Back Malaria (RBM)-funded ITNs brought into the Kita district during the second half of the project. This intervention was facilitated by the Ministry of Health National Malaria Control Program and UNICEF ACSD program as well. The CSP had only been able to procure 1000 nets up until that point. Subsidizing nets at half price at the beginning of the project did show an enormous success, and the stock was depleted quite quickly. But unfortunately, as the cost recovery factor was a challenge, the CSP could not continue with this support. With the arrival of the RBM nets available for free to the target groups, access and availability of nets was no longer an issue. Incentives and criteria for this free distribution were established: e.g. upon the completion of immunizations for a child within the first 12 months, a mother would receive a free net. Upon the completion of 3 antenatal visits, she could also receive a free net; if a woman delivered her baby at the health facility, she could receive a free net; upon coming in with her infant to receive his/her first vaccination, she could also receive a free net. As was the intention of this policy, this strategy enabled the project to make gains in the above health indicators as well. i.e. vaccination and antenatal care. Relais were also able to get nets for personal use as a reward for their work in ITN promotion and re-impregnation. Thus the collaborative effort of the MOH, UNICEF and the CSP resulted in a unique opportunity for supply to meet the demand had already been created (via community education sessions, home visits and interpersonal contact with Relais, VHCs, and project Animators). The CSP effort in creating the demand and desire to sleep under ITNs in the first place, was tremendous. It should be noted that as a follow-up to the initial distribution of ITNs sold at a subsidized cost, the project conducted a small operations research activity. This study actually uncovered the fact that some of ITNs sold were not being used, but rather saved for a special occasion such as the future marriage of a child. As a result of this qualitative research, the project identified the need for improved BCC on the importance of actually using the nets. And via LQAS monitoring, they were also able to target specific health catchment areas that had low indicators in this regard. One factor mentioned that did also help in the increased utilization of nets was satisfied user promotion of the product - as it was noted that the use of nets addressed the ‘nuisance’ aspect of mosquitoes as well. And although not a specific project indicator, the KPC did calculate approximately 39% re-impregnation of nets. 15 iii. For objectives not fully achieved Considering the fact that Relais were either not very active or did not exist in the Kita project communities at all prior to the start-up of the CSP, the increase from 0% to 72.5% in referral of cases seen at the community level is a great achievement. The project goal of 80% was perhaps a little ambitious under the circumstances, but nevertheless they were not far from achieving it. This data was derived from the referral register kept by the Relais in the 12 communities that were included in the random sample for the survey. This included old and new project communities and could have been influenced by either of the following factors: 1) the greatly reduced technical support and supervision of Relais from areas phased out after the first two years of the project and 2) the continued need for capacity building of new area Relais who are still putting their brand new skills into practice. The notable increase in IPT coverage for pregnant women is due in large part to the strategy of combining antenatal clinics with immunization outreach in communities every month. (Discussed below under Immunization). But as good a strategy as this was, coverage at the final KPC survey actually fell slightly from where it had been at the midterm, and did not quite reach the project objective. The latter is due to other constraints – namely the stock out of Sulfadoxine￾Pyrimethamine (SP), at a time when the MOH policy was requiring service providers to no longer give out Chloroquine, and the fact that some health workers, according to interviews with the MOH, were selling the SP rather than providing it for free as part of antenatal services, or holding on to it for actual cases of malaria, since their stock was limited. The objective for iron/folic acid coverage of pregnant women also showed an increase from the baseline, but again a slight decrease from the midterm LQAS, also due to stock out problems. Although benefiting from being a UNICEF-target district under the ACSD program, the supply of such supplements, including that of Vitamin A, has not always adequately covered the needs of the target population. According to interviews during the final evaluation activity, this is also partly due to the inaccuracy of population census data. Although the indicator for health care staff trained falls short, this is not a reflection on the project, but rather of staff turnover. The project trained all of the health facility doctors functioning in the project area, (as per training section below), and even managed to train five of the six new doctors at the time of staff changes. The last individual came in the last months of the project, and at that late point, no more trainings were scheduled or budgeted for. Please note that the above applies to staff training for diarrhea and pneumonia as well as it all fell within the context of training for IMCI. iv. Successes and lessons learned The below revolve around insecticide treated nets. 16 Successes · It was reportedly through the influence of the project staff in a Bamako malaria stakeholder meeting, that the MOH eventually established the policy that ITNs should be free for the specific target groups. This push and influence by the project was later substantiated by the fact that the RBM program also promoted this strategy to ensure better coverage of pregnant woman and under fives in malaria endemic areas. · One successful strategy that contributed to improving several child survival indicators was the use of incentives like receiving a net; incentives which encouraged improved health practices such as completion of childhood vaccination by the age of 12 months, attendance to antenatal care services during pregnancy, and delivery at the health facility. · In interviews with village chiefs, mayors, as well as volunteers it was clearly the unanimous opinion of all that the community-wide coverage by ITNs has greatly reduced the number of cases of malaria as well as child deaths due to malaria. People have a very visual recollection of how many young children used to fall sick every rainy season and the deaths that they used to have to deal with. Thus the project seems to be credited with having very much alleviated this problem. When one asks about project achievements, the issue of ITN coverage is always mentioned. · Needless to say, in order to have had the above coverage and use of ITNs, community and household demand for this product had to be created. Thus, as mentioned above, the project success in this area must be noted as well. Lessons Learned · In the first half of the project when the CSP began to provide communities with ITNs at a subsidized cost, apparently the MOH had been sitting on a large stock of ITNs for quite a while because people were not buying them. Once the MOH followed the CSP example and reduced this cost to 1000 francs, (which made them affordable to almost everyone) their remaining stock was sold off very quickly and thus contributed to the increased coverage of households by ITNs. This is thus an example of the fact that the ‘supply component’ i.e. service or product, has to be affordable and accessible in order for the demand created to be demonstrated, and thus have an impact. v. Special outcomes, unexpected successes or constraints. · Complementarity with UNICEF: The coverage of 98% of households by ITNs was indeed an unexpected level of success, and clearly due to the availability of free nets. Thus, as mentioned above, the participation of UNICEF and the MOH in this contributed to great synergy and this enormous success. 17 · It is important to reiterate the constraint encountered with regard to the supply of ITNs in the first part of the project. The CSP could not obtain more nets for the community, even once they had created demand and the first supply had been completely depleted because of a cost-recovery problem. Thus many households did not have access to nets until the RBM supply became available. Also to be noted is that this new supply of nets has also run out - reportedly 6 months to a year ago, depending on the catchment area. Thus a few of the newer project communities have complained because they have not been able to benefit from nets at all. The MOH states that they put in the order a long time ago and expect another 15,000 + nets to be arriving soon. vi. Potential for scale-up or expanding impact The potential for scale up of these interventions is tremendous and has already been demonstrated by Plan Mali as some of the other program units have come to learn from what was being done in Kita already and benefited from training given by the Kita CSP staff. The National Health Advisor and office in Bamako are also very supportive of scaling up the C-IMCI strategy to all four of their program units and have been seeking funding for the same. In addition to this, the district, region and its partners, notably UNICEF, have also seen the merits of scaling up C￾IMCI based on the project experience. They have thus allocated some funding to begin this process in 2007, although it must be noted that this may only be limited to the Kita district, including expansion to cover untouched catchment areas in the district. This activity is not expected to include the full scale field presence and support for community mobilization that was a part of the Plan CSP effort, but apparently the provision of bicycles for the CHW has been included in the budget. Immunization (Vitamin A) i. Results and outcomes The Kita CSP did extremely well in this component with only one out of eight indicators falling short of the project target. i.e the proportion of children 6-59 months who received Vitamin A twice in the past 12 months was noted at 50% at the final, as compared to the 60% target. Except for the above indicator, all others for Immunization and Vitamin A surpassed the objectives set by the project. The success in the area of coverage is particularly notable because of the challenges faced. For, percentage of children 12-23 months completely vaccinated by the first birthday, the project saw a change from 51% at the baseline to 77.2% at the final. The target was 71%. Likewise the project saw a great reduction in the drop out rate, with the proportion of children 12-23 months with DPT 1 minus those with DPT 3 going from 49% at baseline to 18.9% at the final, more than meeting the 20% objective. The proportion of children 12-23 months with measles vaccine also did very well, beginning with 51% at baseline and ending with 83.5% at final, surpassing the 80% target. Mothers of children 0-23 months who delivered at health facilities reporting that they received at least 2 doses of Vitamin A (200,000 IU) before 18 leaving the center was 97.2% at the final compared to the target 60%. No baseline data is available. In terms of disease surveillance: the proportion of cases of measles, polio, neonatal tetanus reported to the health facilities was 100%, and the response by the health facility on reported data was 100%. Success in related indicators also monitored by the project but not in the DIP included: Children 0-23 months who have an immunization card (49% at baseline and 94.6% at the final – target was 80%); Mothers of children 0-23 months who delivered at the health facility reported receiving Vitamin A was 40.1% at the midterm LQAS, and 97.2% at the final KPC, more than achieving the 60% target; and Mothers of children 0-23 months with a prenatal card was 70% at the final, again bypassing the 60% target. ii. Factors affecting achievement of program objectives and outcomes. The primary factor contributing to the big improvement in immunization coverage in the project area was the use of the “Strategy Avance.” This strategy is essentially a community outreach service for EPI whereby a vaccinator from the health facility goes out to communities every month to provide this service. The projects’ support for the inclusion of antenatal clinics in this service as part of the national policy but not always implemented, also served to greatly increase the tetanus toxoid (TT) vaccination coverage of pregnant women. The community mobilization element of this activity (i.e. trained Relais and VHCs carry out a census of children under five and pregnant women in the community, track completion of vaccination schedule, and follow-up absentees), was critical to the success of this intervention. The hard work and commitment of community Relais in particular, including promotion and education, individual counseling at the household level, particularly in the case of children or pregnant women who had missed appointments, was key to getting the coverage desired and reducing the percentage of dropouts between those receiving first and last doses of DPT and Polio. In most communities, the vaccination and antenatal care ‘strategie avance’ activities also had the backing of the village chiefs, who were very supportive and encouraged community members to be responsible and respect the childhood immunization calendar. One chief was even reported to severely reprimand and threaten mothers who the VHC mobilizer or relais had had to go find, or who were late coming to the monthly activity. In addition to this, the participation of the traditional birth attendant (TBA) as a member of the village health committee also facilitated the identification and enrollment of pregnant women in the above mentioned activity. And as mentioned, the VHC mobilizer also helped ensure the attendance of children and women scheduled to receive services. All of the above contributed to the big success of this component. A special project effort that also helped increase vaccination coverage in the entire district of Kita (beyond the project areas), was the project’s response to the Regional MOH’s plea for help to assist with the problem of low vaccination coverage. As the Kita district and Kayes region had amongst the worst coverage in the country, the CSP came up with the idea to use project 19 NICRA to support this additional work in immunization. In collaboration with the district, the CSP identified 13 areas for the development of additional vaccination posts (1 within the project area where distance from the health facility was an issue, and 12 outside of the project area). Based on an agreement with the respective communities that they would take responsibility for the costs of these activities after 6 months, the project initiated vaccination activities by supporting the salary and training of vaccinators. The MOH provided refrigerators for the cold chain and vaccines, and in a couple of cases the community mayor supported the provision of fuel for a motorcycle, and the provision of a motorcycle. The MOH subsequently opened two additional vaccination posts for a total of 15 posts bringing services closer to communities. The availability of Vitamin A supplementation from UNICEF, combined with the training of health facility workers in perinatal health by the project, resulted in the excellent coverage in Vitamin A of mothers delivering at the facility. Needless to say, many mothers in the project area still deliver at home, and this supplementation was unfortunately not available at the community level. Disease surveillance (according to WHO protocol and following the MOH policy) was initiated in Kita through the support of the CSP as well. For this component, the community Relais was trained in the identification of possible cases and to immediately notify health facilities. During the life of the project two cases of meningitis were identified, and reported per the protocol. iii. For objectives not fully achieved The only DIP objective under this component that was not achieved was that of the proportion of children 6-59 months who received Vitamin A twice in the past 12 month (50% verses the 60% target). Although the project, through its partners, would have wished to ensure the continuous supply of Vitamin A, this was limited by the campaign approach chosen, and evidently the amount of Vitamin A designated for the district. The MOH, in collaboration with UNICEF, implemented a Vitamin A campaign to cover under 5 years olds every six months, but a stock of Vitamin A for this purpose was not available at the health facilities (let alone the community level with community Relais who could have played a very big role in ensuring both the coverage of children and postnatal mothers within 40 days of delivery). Thus the group of children who are close to, but not yet 6 months of age at the time of a campaign, are obliged to wait for another six months before they can get their first dose. This of course has an impact on the Vitamin A coverage of the 6 – 11 month old children who upon reaching the age of eligibility don’t have access to Vitamin A right away. Before the UNICEF support of this activity, Helen Keller International was implementing Vitamin A campaigns as well. But as the project did not collect baseline data for this indicator, it is difficult to know how much progress has been made in this area. iv. Successes and lessons learned Successes · Under this component, the project staff again found that the presence of the community Relais really made a difference. The latter maintains a census of all eligible children in 20 the community and thus the target group is clearly identified. The commitment of this individual, with the support of the village health committee included follow-up at the household level. · The 2-3 day quarterly coordination meeting established in each health catchment area was an important element of the success of the immunization activities. The meetings included the participation of actors from every village in the catchment area including the health facility (CSCom), the ASACO, the mayor, the two Relais and the President of the VHC. The meetings were forums for assessment of performance in the previous quarter, reporting on health facility income and finances, information-sharing, problem solving and planning of upcoming quarterly activities and financial expenditures – using the annual work plan as a reference. (Planning for the strategie avance activities entailed looking at the distance between posts for fuel cost estimates; looking at the closeness of villages for grouping, although experience showed that some villages were difficult and did not want to go to neighboring villages for their service; choosing dates that were appropriate and did not interfere with things like market days etc). The added value of these meetings was that reports on vaccination coverage per village created a spirit of competition and renewed commitment, as villages compared results and wanted to do just as well or better than their neighbors. Thus this quarterly coordination meeting ended up serving as a tool that fostered community mobilization, collaboration, learning, monitoring and evaluation, good governance and transparency. · One example of a successful undertaking that came out of the learning and problem￾solving done in the above-mentioned quarterly coordination meeting was the development of a new strategy to increase vaccination coverage in Kita District. This was the development of vaccination posts in large catchment areas, and in areas with no health facility at all. The latter were areas that were thus not covered by the project either, as the project only worked in areas that were served by a community health center. A vaccinator was hired and trained to work in these vaccination posts for the monthly strategy avance; supplementing the existing services at health facilities. This strategy enabled EPI services to be more accessible to the population, and more consistent, as the health worker in the health facility had other job responsibilities and was not always able to keep to the different village schedules. The success of this initiative taken by the Kita district partners has influenced the MOH in the Region to promote this strategy elsewhere in the region. The Kayes Region has even suggested that other districts visit Kita to learn from them. Lessons Learned · Despite the success of the vaccination post initiative in increasing vaccination coverage in the district as a whole, many of the vaccination posts taken over by the community after the requisite 6 month period are no longer really working. According to the MOH, there are ten or eleven of the fifteen that were established that are not up and running like they were when the CSP supported them. As all but one of these were in communities 21 where the project was not present to really foster and facilitate community ownership, this experience shows that developing community ownership and responsibility does require strategic effort, and perhaps more than the six months that the project allocated. It is important to note that the one vaccination post established in the project area itself and supported by the project for only a six month period, is working very well under the management of the community and the support of the health facility and ASACO. v. Special outcomes, unexpected successes or constraints · One unexpected influence of the quarterly coordination meetings on the Relais representing their communities was that they established their own meeting as well. The purpose of this meeting was for sharing of experience, exchange of information, continued learning and sharing of data. Unfortunately due to the cost of these meetings to participants, including the potential need for lodging and food due to distances, these meetings have not really continued. · A constraint to good vaccination coverage is the fact that the district still has communities that are not covered by health services. There are a reported 44 catchment areas eligible for a health facility in the Kita district, whereas only 26 actually have a community health center established. Thus although the vaccination post strategy helped to address this gap, it has clearly proven to be very difficult to ensure the sustainability of these without the active support of a health facility (whose income can potentially cover the cost of a vaccinator, a motorbike and fuel). · Inaccessibility of certain areas during rainy season because of rivers and flooding. In some areas this can be for up to six months of the year. · Periodic stock outs of antigens such as BCG, measles, yellow fever and even supplements like Vitamin A, and iron folic acid for pregnant women, also constraints to attaining and sustaining very good coverage. The MOH feels that much of this problem is due to poor census data on the target population. (Census data serves as the basis for the stock that the national store has available for each region). The Kita district has even been accused of using more than its allocation. This has also led to health workers being asked not to open vials for BCG for example, until they have had a chance to pull together a large number of children who are eligible for this vaccine - as they cannot afford wastage. Needless to say this contributes to decreased coverage. · Another issue mentioned is that sometimes the child has actually died, and yet this is not recorded or reflected in the district calculation of immunization coverage. (Having a community maintained HIS that addresses issues like this one and enables the district to come out with accurate statistics is one of the reasons why the district is very much in support of scaling up C-IMCI). · Other constraints to good coverage raised by mothers during FGDs seem to be primarily related to carelessness and negligence which can continue to be addressed through 22 IEC/BCC. e.g. sometimes the family migrates elsewhere for an extended period of time (seasonal migration for work), but without the immunization card it is difficult to follow up with the vaccine schedule; some mothers don’t know the vaccination calendar and miss appointments; field work sometimes takes mothers away from their parental duties; distance from some vaccination sites or reluctance to join other villages for vaccination sessions when they are asked to group themselves. vi. Potential for scale-up or expanding the impact Like the malaria intervention (within the context of C-IMCI), the above mentioned EPI strategies that proved to be successful, have a lot of potential for scale-up, and are ones that both Plan Mali, the district of Kita and the Kayes Region want to support and replicate. Diarrhea i. Results and outcomes The results for the diarrhea component were mixed. When it came to the proportion of children 0-5 months who received only breast milk in the 24 hours before the survey (exclusive breastfeeding), there was a big improvement from the baseline 11.6% to 62.4% at the final, achieving the 60% target. Although there was no baseline data for the proportion of mothers able to site at least 2 methods of preventing diarrhea, the final was 67.6%, almost reaching the project target of 70. But the results for diarrhea treatment with ORS or home-based fluids seem to indicate that there has been no change (25.2% at baseline and 25.5% at final). FGD discussions with mothers seem to indicate that the above question was understood to mean use of ORS packets only because in all group interviews it was clear that the preparation of sugar, salt, solution at home was common knowledge. Mothers also spoke about giving other fluids such a liquid porridge and breast milk; all of the above they said, as substitutes for ORS packets when they can’t get them. The results for the proportion of HC staff, relais, and ASACOs trained for this component was 100%, 95.7% and 0% respectively. Thus with respect to the first two cadre, the project target was achieved, and close to being achieved. Tremendous success in related IMCI indicators that were monitored by the project, although not included in the DIP objectives, were the following: Mothers of children 0-23 months who know 2 signs indicating that a child needs care went from 35.3% at baseline to 87.8% at the final, above and beyond the 60% target; Children 0-23 months who were sick who received at least the same amount or more than usual to drink went from 26.6% at baseline to 85.8% at the final, again more than the 60% target; and Children 0-23 months who were sick who received at least the same amount, or more than usual to eat started at 21% at baseline and went to 86.3% at final, again surpassing the 60% target. 23 Although the project also followed the indicator for immediate breastfeeding: percent of children who received breast milk within the first hour of birth, the results seem to indicate a difference in how the question was posed at the baseline and final (79.7% and 51.5% respectively) because the a big reduction in this indicator does not make sense. At the baseline, the enumerators probably focused on whether or not the child received colostrom (as this was emphasized to be the rationale behind the question). While at the final KPC, the enumerators must have focused on the actual hour after child birth that the child received the breast – which gave us a much smaller percentage of people. Thus it is difficult to compare these results. Suffice it to say that FGDs with mothers did show that they understand the importance of colostrum and do give it to their infants. It was noted though, that many mothers do not start breast feeding within the first hour after birth because according to them, sometimes the baby is very tired and falls asleep right away, or the mother is tired and wants to clean up first. But the child does, nevertheless receive the colostrom. It was also revealed that a few mothers are still influenced by mothers-in-law or TBAs to give other products to clear the baby’s throat after childbirth. This could be lukewarm water or a traditional infusion. And as it relates to continued child health, according to the KPC results only 24% of 6-9 months old babies were receiving breast milk in addition to solid or semi-solid foods. None in the sample were receiving breast milk only. Results for hand washing with soap – although this was not a project indicator, the KPC did reveal that there is much work still to be done for this aspect of hygiene. The CS indicator for hand washing with soap before meal preparation and feeding and after toileting is 30%. Although one of the four groups of women interviewed during the final evaluation said that the cost of regular soap is an obstacle; some use ash or a locally made type of soap (which were not responses included in the survey questionnaire). But most women (particularly in the urban area) felt that lack of hand washing with soap is based on habits (i.e. lack of proper education of the girl child in the development of good hygiene practices), and a lack of information. Some mentioned laziness or being too busy to think about it as being reasons as well. ii. Factors affecting achievement of program objectives and outcomes. As with the other components, the success in the area of exclusive breastfeeding is due to the mobilization of community resource persons and the IEC/BCC strategy. A full fledged effort went into the promotion of exclusive breastfeeding, as is demonstrated by the big achievement noted in the survey. In addition to health education sessions either done while mothers waited for the vaccination activities to begin or scheduled as a separate activity, targeted individual counseling and home visits were also done. The project came up with innovative activities such as competitions for the most beautiful baby (healthiest baby), which required that the child be exclusively breastfed, fully vaccinated, that the mother have followed antenatal care, and that the child be weighed to check on its nutritional status as well. Prizes for these competitions were such things as diapers, T-shirts, toys, baby baths, ORS packets and ITNs). All mothers who participated in the contests received gifts of encouragement as well. 24 An innovative strategy adopted in the latter part of the project to help pull up the health indicators in general, and certainly used for exclusive breastfeeding as well, was identifying problem indicators and problem communities by way of the LQAS survey done twice a year. During the actual LQAS activity, when a mother’s response was not correct or ideal, education on the particular issue would be done right then and there. As the tool was being used for monitoring rather than evaluation purposes, this was a luxury that the program could afford to take. Likewise, when all the survey data had been entered and analyzed, program staff would study the results for each individual sample community, and thus be able to identify specific catchment areas for reinforced IEC/BCC on specific indicators (based on how they had done in the survey). For each LQAS done, Animators would be encouraged to choose new communities so that as much of the health catchment areas could be sampled during these surveys. Thus a strategy of targeted community households and target indicators contributed to the tremendous progress in the area of exclusive breastfeeding among other indicators. Successful community mobilization really paid off for the CDD component, as reported by project Animators and communities alike during the final evaluation interviews. And one can see this reflected in mothers’ knowledge of ways to prevent diarrhea (68%). Many communities organized regular ‘Health Days’ where primarily the young people in the community would be tasked with going around and ensuring that the community was cleaned up. i.e. brush would be cut down, stagnant waters removed, properties inspected for trash and so forth. Some community chiefs even established a fine for community members who did not abide by the established consensus for maintaining a healthy and sanitary environment. And as was reported in all FGDs, although the project did not specifically finance water and sanitation activities, a lot of improvements in this area took place during the life of the project. i.e. repair of water pumps on artesian wells so as to avoid drinking regular well water, building of barriers to keep animals away from water points, in the case of wells, construction of well pail holders to keep the bucket from being left on the ground to get dirty, disinfecting drinking water with Clorox tablets or use of filters, and the construction of latrines in households. Some families even pooled resources to construct a latrine that they could share. Much of this community initiated and supported improvement is credited to education and mobilization done by community Relais and the project Animators. Some of these communities were fortunate enough to benefit from Kita Integrated Rural Development Project (PDRIK) government contributions to their efforts in the establishment of new water points and rehabilitation of old ones. Other project achievements related to the sick child and care during illness, again reflect the level of IEC/BCC that took place during the life of the program, the dedication of community resource persons, and the respect that mothers accorded these individuals. Project Animators supporting and participating in activities was also critical for the validation of the community volunteers trained and reinforcement of messages. And the development of relations between community people (Relais and VHC) and the health facility has also facilitated referral and the increased use of services. It should also be noted that the training of health providers in IMCI has contributed to a consistent reduction in client expenditures in drugs for the sick child, which should be encouraging to the clients. This is also considered an indication that the health providers are in fact using the IMCI methodology to diagnose and manage the sick child. 25 iii. For objectives not fully achieved - contributing factors. The one project-specific indicator under this component that did not show progress was use of Oral Rehydration Solution packets. Health facilities, the District and project staff all mentioned that there had been a problem with the supply of ORS throughout the life of the project; starting at the national level. As previously mentioned though, FGD discussions with women in the project area clearly revealed that just about all mothers know about ORS packets, are also able to describe how to make the sugar, salt, solution at home, and give other fluids during diarrhea episodes as well. And again please note that the project did train all of the health facility doctors functioning in the project area, per the objective. But due to staff turn over, it was obliged to make an effort to redo trainings. One of six new comers arrived too late in the project to receive this training. iv. Successes and lessons learned BCC for hygiene and sanitation had an impact evidenced by the great reduction in incidence of diarrhea cases occurring in the two weeks prior to the final KPC – conducted at a time of year (rainy season) when there are often more cases of diarrhea than at the time the baseline KPC. The success of the BCC efforts, as previously mentioned, was also evident by the degree of community mobilization for the improvement of water points, community cleanup and construction of latrines. v. Special outcomes, unexpected successes or constraints. Constraints · Availability of ORS packets was a problem. Simultaneously, the MOH policy was against the promotion of making sugar salt solution at home; thus causing a dilemma for project in terms of IEC/BCC for rehydration therapy. · There is a traditional belief that washing hands will reduce your chances of making money that day. This has been mentioned as something that may also contribute to poor hand washing habits. · Due to the fact that ‘old habits die hard’ achieving significant behavior change in hand washing seems to be a real challenge. This is likely to require some qualitative research first, and subsequent development of strategic marketing and communications methodologies. vi. Potential for scale-up or expanding impact 26 Like the other components undertaken in the Kita CSP, the IEC/BCC undertaken for Diarrhea control as part of C-IMCI, have a lot of potential for scale up. These clearly had an impact on community and household resource mobilization even when there was limited financial support for water and sanitation. Pneumonia i. Results and outcomes The project achieved two out of three objectives set for the pneumonia case management component. When it came to the proportion of children 0-23 months with fast/difficult breathing taken to a health facility, the project contributed to an enormous improvement, as it was 20% at the baseline and jumped to 60% at the final. This achievement was far beyond the 40% objective set. The proportion of fever cases referred to the health facility (overlapping symptom with malaria) was 59.1% at baseline and jumped to 83.7% at final (as mentioned under malaria results). This also surpassed the target of 75%. As has already been mentioned under the other components, the proportion of CSCom (health facility) staff trained was 95.7%, close to the 100% target. And lastly, although no precise objective was set for the following related capacity building indicator, the proportion of CSComs with 10 essential drugs in stock during the last six months (including antibiotics) went from none (zero) to a third (33.3%) in the sample of 12 CSComs surveyed. ii. Factors affecting achievement of program objectives and outcomes Again, as mentioned in the intervention areas discussed above, the factors above all else that contributed to behavior change and improved care-seeking was the presence of community resource persons doing community based IEC/BCC, including periodic and targeted home visits. The fact that mothers had easy access to these individuals for advice during episodes of illness also helped tremendously with referral to the health facility. It is important to note that unlike other illnesses, according to mothers interviewed in FGDs done during the life of the project, there are no traditional treatments for pneumonia. This clearly facilitates care-seeking to health facilities as well. The improvement in availability of essential drugs at the community health centers is a direct result of the capacity building given to Community Health Associations (ASACOs), who help with the management and functioning of the health facility. According to the FGDs with the ASACOs, the training in financial and administrative management including drug inventory and procurement gave them new skills and really helped them with the management of the latter and other tasks related to the running of the facility. iii. For objectives not fully achieved -contributing factors. 27 Please refer to previous comments on staff turnover. iv. Successes and lessons learned Successes · It is evident that the community mobilization and IEC/BCC for pneumonia prevention had a dramatic impact when one compares the KPC final incidence of pneumonia prior to the survey with the baseline KPC (61% at baseline and only 9% at final). Mothers practice keeping the child warm and also using a thick, body-warming oil (beurre de calite). Care seeking for pneumonia and ARI also improved dramatically and can be attributed to the same. · This component also benefited from the innovative data based targeted education that proved successful in improving all of the indicators. i.e use of the LQAS to identify problem areas and focus education and counseling in communities that did not have good results. · Training of designated pharmacy person(s) from the ASACO did have an impact in increasing the number of facilities with essential drugs. i.e. a lot of effort and focus during the training was on keeping track of the drug inventory and the practice of ordering drugs in advance so as not to run out. v. Special outcomes, unexpected successes or constraints Constraint · One constraint to access is distance to services. Providing the Village Health Committee with a stock of simple medicines (via the ASACO) for the community health volunteer to use is in fact part of the MOH policy. But nevertheless there is resistance to this policy both at the HF and district level because of past problems with ‘village doctors’ treating cases beyond their capacity. Thus this kind of strategy would require real district commitment, support, and supervision. One concern of health facilities (CSComs) is that they will lose precious income which contributes to the functioning of the facility. Although the purchase of drugs at the community level would go to the ASACO/CSCom who provide the drugs, the facility would not get the visit fee. The latter helps to cover staff salaries. vi. Potential for scale up or expanding the impact. 28 Please refer to previous comments on project component scale up as part of the C-IMCI strategy. This is already being adopted in other Plan Mali projects and fund-raising efforts for replication are undergoing. c. New tools or approaches that the program developed/used; operations research or special studies Plan was the first organization to use the LQAS methodology in Mali. Kita CSP team members helped with a regional training workshop to encourage the use of this methodology across the West Africa Region. The research-education approach to targeting IEC/BCC and specific communities is an innovation that really helped to improve project indicators. One operations research activity that the project undertook before the influx of free ITNs was on the community management of ITNs for sale. i.e. management by ASACOs. This study was done in four project villages and consisted of a focus group interview with a group of men, a group of women and the general assembly (population). Although the study did look at aspects of prevention, the main focus was on the sustainability of this activity at the community level in terms of management mechanisms, transparency, procurement and distribution. All participants knew methods of malaria prevention and were ready to purchase nets for their young children. They felt that it was important for the ITN distribution activity to be managed by a committee with well defined members and roles for each member. The women felt that it should be a woman’s group or committee handling the activity, while the men felt that it should be the chief, village health committee or relais – with even the mayor’s involvement. (Note that although the actual activity was indeed successful and increased the coverage of the target population, the challenge of having enough funds to renew the stock did remain). The project also undertook two sustainability studies using the CSTS tool. The first one was in 2005 after the headquarters staff came and trained the project staff on this methodology. The second one was at the final in 2006. The scores for health and health services in 2005 showed an emerging to intermediate result (36.5 and 38.7 respectively). In comparison, the 2006 scores showed that a lot of improvements had occurred as they went up to 68 and 66 respectively for those two indexes. Scores for several of the other sustainability indexes were also good at the final. i.e. Organizational capacity of ASACOs got a score of 65; Organizational viability of ASACOs scored 70; and the Community capacity index score was 75. The lowest area of achievement was in the Environmental index (53) which reflects the fact that the project was not really intervening in these areas. B3. Results: Cross-cutting approaches a. Community Mobilization 29 i. Effectiveness of approach The community mobilization approach used has been mentioned in the discussions above as a key factor in the CSP achieving its objectives. This approach proved to be tremendously effective and community leadership and members were very responsive to it. To facilitate the process of community participation and responsibility for health, the project proposed that communities 1) develop village health committees; and 2) identify two community health volunteers (relais), one man and one woman; and 3) strengthen the ASACO (community health Associations) that manage health services. Each village health committee newly developed in the project area was made up of strategic members including the most popular traditional birth attendant in the village, a community mobilizer, the relais themselves, and a president. Via the above development and strengthening of community stakeholders, the community mobilization strategy included bringing health information and education to the community level, developing community based health activities in collaboration with the health facilities (strategy avance’s immunization and prenatal clinics), and the development of a health information system that allows for accurate monitoring and follow-up. It is important to note that much of the impact of the project can be directly attributed to the relais who was the recipient of much more training, tended to be the most able in terms of literacy skills, and whose role was more clearly defined than the others. The TBA also tended to be quite active as she had a specific target group and indicators that she was dealing with. i.e. pregnant women. She would keep a register of these woman and get assistance from a more literate person if need be. But unfortunately there were cases where the VHC as a whole lost some of their dynamism. This was due to the fact that much of the focus and attention tended to be on the relais. The relais was the person who maintained all of the community health information registers (specifics discussed in the health information system section of the report). S/he was the one who submitted reports to the health facility and project staff, and played a leadership role in the IEC/BCC activities. Thus supervision and monitoring visits by the project staff and health facilities tended to target this individual. As effective and strategic as the above approach was, it was not without problems. The motivation of the relais became an issue because to ensure community ownership and sustainability, the project was not encouraging these community volunteers with a stipend. Communities were urged to find ways to encourage them, such as helping them with their farms or giving them gifts in kind. The health facility also gave them per diem when they helped with vaccination and Vitamin A campaigns, and they received per diem during trainings. In an effort to increase volunteer motivation the project and its partners came up with a couple of other strategies: giving relais the annual health membership card to sell to community households so that they could get a percentage of the sales (these cards allow people to access health services at a cheaper cost through the year); and trying to develop the common practice of giving gifts of money to the relais during the child baptism ceremonies, where gifts of money to the infant and family is a custom. But many relais got de-motivated prior to some of these initiatives, and certainly most would have liked regular remuneration. Project staff raised this as a challenge during the final 30 evaluation interviews, including the fact that many of the relais eventually dropped out. This figure is in fact not alarmingly high though. Of a total of 404 volunteers, 20 or so dropped out in the rural areas. Reasons for the latter were primarily migration and de-motivation. Migration is a particular problem in isolated areas such as the Kaarta area (northern part of district), where younger men search for greener pastures elsewhere. There was also a death and the issue of husbands not always liking that their wives were out working when they had household responsibilities to take care of. In the urban area the problem was more critical though, with as many as 8 out of 18 volunteers dropping out due to schooling, work opportunities and changing neighborhoods. Another challenge encountered was that of literacy levels. Literacy levels in Kita district are very low, confirmed by the KPC final survey showing that 59% of the mothers had received no education at all, and only 3 of 392 mothers interviewed had been to secondary school. Community health workers needed a minimum level of literacy in order to maintain the different registers that formed part of the community based health information system. Finding a woman volunteer who could maintain this information was not always possible, even though it was found that women relais tended to be more reliable and willing. The number of men who were qualified to do this work was also limited. But working as a pair, the man relais was the one who had to be charged with managing the CBIS. ii. Community Mobilization Objectives The community mobilization objectives involved identifying community resource persons for health, and helping to establish community structures which would enable the community to become partners for health. Thus this mobilization of communities and strengthening and reinforcing of their capacity fell within the following objectives articulated in the DIP: 1. Communities will learn to voluntarily take responsibility for implementation of development activities. As discussed above, the project proposed the creation of a village health committee that could be a major player for health at the community level. Once this committee was established (one per each project village), its members met about once a month to plan and organize for health activities in their communities. As mentioned before, representatives of these committees also attended the quarterly coordination meetings of the health catchment area to plan and coordinate activities. Thus the project did meet this objective; the results of which is seen in the achievement of many of the knowledge and behavior change indicators. Again it is important to note that over time, the relais, who was a part of the VHC, did end up carrying more of the burden and playing a more important role in the above than the committee as a whole. The project also undertook the strengthening of the ASACOs who were community representatives in the management of the community health center. These local organizations really did become much more active, and contributed to planning of health and development activities. 31 2. Community will develop the capacity to define and implement development interventions. This second community mobilization objective was also achieved as witnessed during the quarterly coordination meetings where village representatives discussed achievements, problems and made decisions around those. As a result of the project, communities began to working in collaboration with the health facility providers, were able to have their voices heard, and participate in problem-solving and planning. 3. Increased revenue for poor households through the development of community micro-credit projects. Micro-credit projects are part of Plan’s core program funded through private funds. Several of the Plan Kita Program Unit areas overlapped with the Kita CSP communities. Thus some CSP communities benefited from the program’s micro-credit, while others did not. 4. Promotion of social mobilization. This objective was also achieved in that community resource persons really did get community participation in education and strategy avance activities. Community chiefs also contributed to social mobilization; very much vested and interested in seeing an improvement in the health conditions and indicators in their communities. Social mobilization resulted in the participation of youth in community hygiene and sanitation. Social mobilization also resulted in mothers who were very responsive; attending health talks and receptive to advise and information. iii. Lessons Learned for future community mobilization efforts · One lesson that the project learned was that it is important to provide adequate training for all VHC members, acknowledge and place a lot of importance on the role that each member plays in community mobilization, organizing and planning of activities. This motivates people and helps to keep them active. In the Kita CSP, much of the training, subsequent support, follow-up, monitoring and supervision was focused on the relais. As a result, although other VHC members had a designated role, over time many got de-motivated and perhaps did not feel that they had much of a contribution to make. The project, for example, was unable to continue the training of TBAs after the Phase I communities (8 of 23 total catchment areas), thus the TBA VHC member in more than half of the catchment area did not benefit from technical training. The VHC community mobilizers did not really receive training either. To help address the problem of VHC dynamism, the project proposed that the VHCs have monthly meetings. These meetings encouraged information-sharing between members, and was an opportunity to discuss data and activities so that all the members could feel engaged in work. 32 But staff did comment that per their experience, a concerted effort needs to be made to do the above, as a monthly meeting is not the full remedy. iv. Community Demand for program activities to continue As can be seen in the LQAS data which enabled the project to look at indicators for each of the catchment areas individually, it is clear that community demand has been established. The first 8 project areas were phased out after two years of concentrated project staff efforts. At the time of the last LQAS, more than two years after the withdrawal of consistent support, these areas did maintain a lot of their health indicators, even though there was a few went down. This is an indication that community demand was well developed. During FGDs women spoke about the health activities including vaccination sessions, prenatal care services, follow-up, and registration of newborns etc. When asked whether or not they were acquainted with the relais, all said yes and said that they had regular contact with this individual for health activities or advice. When asked about any changes during the life of the project, all mentioned a big change in terms of water, sanitation and hygiene, the prevalence of common childhood diseases and infant mortality. Everyone, including mayors and chiefs, as well as the different committees established (VHCs and ASACOs) showed a great appreciation for the changes and improvements in health. There is a huge increase in general awareness and understanding of elements of basic health at the community level, which has led to improved care-seeking and use of health services. The participation in health activities has become common practice and community members are very happy to have these services. It should be mentioned that community cohesiveness, demand and enthusiasm was more notable in the rural areas than in the urban. v. Plans for Sustaining activities at program close-out The community mobilization and capacity building approach of the project was developed with sustainability in mind. As discussed already, community actors were encouraged to participate in the development and implementation of health activities, problem identification, assessment, planning and decision making. Encouraging the involvement of village chiefs, local mayors, community counselors and strong linkages with the health facilities were all part of the plans for sustaining activities at program close-out. All of these strategies were successfully put in place as planned. And in support of the above, when it came to catchment areas that were completely isolated from Kita town for up to six months during the rainy season, the project realized the importance of placing their strongest Animators in these areas. In addition these areas received the best motorcycles and a lot of support from the NGO and Plan CSP supervisors during the dry season. vi. Are the sustainability plans realistic? The phased approach used by the Kita CSP was an opportunity to observe whether or not these strategies were effective over time. As mentioned above, 8 catchment areas made up Phase I 33 project communities. These communities were phased out in 2004, and have only been periodically supervised by project staff in the past two years of the project. Community data and reports from the health facilities in those catchment areas demonstrate that activities initiated by the CSP have continued during this period and that the health facilities, ASACOs and relais, in particular, are still working closely together. Thus in the above example communities, sustainability has indeed been demonstrated. It must be noted though, that not all the CSP communities benefited from the level of effort given to the first half or so. i.e. the first set of 8 and the second set of 6 health catchment areas. Due to delays in the continued development of community health facilities in the district, the project was not able to expand into new catchment areas as early as it had planned to. Thus the last 9 catchment areas that were phased in had a project effort of only 20, 16 and 11 months respectively. These communities, particularly the last set, are in need of much more support to ensure sustainability, as they are barely getting started in this regard. Under the circumstances, it would be very unfortunate if Plan could not continue with the work that it has already begun. b. Communication for Behavior Change i. Effectiveness of approach The IEC/BCC approach was primarily that of passing on health messages and advice through a trained community health worker (relais) who were given materials based on the C-IMCI model. Their main activities were health education sessions and interpersonal advice and counseling during home visits. IEC/BCC was supported by an enabling environment created by the community mobilization discussed above. i.e. community leadership, entities and resource persons were all in support of these messages and behavior change. They were outspoken, many organized community health days, promoted improvements in water points and the construction of latrines. Thus mothers and caretakers were encouraged on all fronts. The validation that the relais received in this regard, in addition to the regular affirmation of these health messages from project Animators, contributed to mothers adopting improved health behaviors, and greater awareness of when to seek services and the availability of health services. Additional creative activities such as the ‘Most Beautiful Baby’ competitions gave mothers interest and an incentive to adopt new behaviors. The ITN distribution incentives were also strategies that promoted healthy behaviors i.e. giving a free net to a mother when her child has completed his/her vaccinations before the age of 12 months, or when she has completed a certain number of antenatal care visits. Targeting BCC based on individual community LQAS results, as was discussed already, was another strategy that really seemed to help in improving health indicators as well. In the last year of the project, the project Animators were even asked to do mini surveys of 19 households every month to continue pin-pointing needs in IEC/BCC, and respond to them. In addition to the above, the project used the local radio station to share CS messages, ii. Behavior Change objectives 34 Please refer to the specific intervention areas above for results in behavior change indicators. But in a review of those, it should be noted that the project achieved the large majority of its behavior change objectives. The very few objectives that were not achieved were in fact due to supply issues and not due to a lack of care-seeking on the part of the project population. i.e. use of ORS packets, coverage of pregnant women in iron/folic acid and anti-malarials – as previously discussed. iii. Lessons Learned · The project experience showed that it was useful to begin with community-wide BCC activities in order to reach everyone with the basic health messages. But subsequent to this, it worked really well to identify target individuals or groups of households for BCC. i.e. periodically go through identifying problem indicators and corresponding individuals or clusters of households that need the intervention; focus on the latter, and then beginning the process all over again. Mini surveys across a project area or an LQAS, in this case, was a great tool for this. iv. Sustaining behaviors at program close-out As mentioned under the ‘Community demand for activities to continue’ section above, two years after the phase out, the initial project communities continued to retain a lot of the project gains. And this is despite the fact that project field presence in those communities had been for a period of two year only (2002-2004). Targeting mothers with children under 12 months of age, the LQAS at the baseline followed 14 indicators across the 8 catchment areas. The results showed a range of 1-6 out of 14 indicators showing up as problematic (i.e. an average of 3.6 problem indicators across the catchment area). A couple of the catchment areas had as many as 5 and 6 problem indicators out of 14, several had 3 and 4 problem indicators, one had 2 and another had 1 problem indicator out of the 14. At the time that the project was leaving in 2004, the range was 0-2 problem indicators out of 14 indicators (an average of 0.75, or less than one problem indicator across the catchment area). In 2006, two years after the phase out, the range is 0-4, with almost an even number of the 8 catchment areas having either no problem indicators, 1 problem indicator or two problem indicators out of 14; except for one community who went from only having one problem indicator at the time of the phase out, to having 4 two years later. The latter in fact pulls up the average to 1.5 problem indicators out of 14 across the catchment area. Thus although project achievements were not sustained at 100%, one can see that communities have maintained many of the improved behaviors. v. Are the sustainability plans realistic? As evident from the LQAS results which give catchment area specific data, in addition to project and district staff assessments based on supervision visits, the facility health worker plays a big role in assuring the achievement of health indicators, and sustaining them. Areas that have had dynamic health providers who have bought into the project strategies and objectives have done 35 very well. Those where the provider is less dynamic, or there has been a temporary absence, tend to be the ones that show up with a higher number of problem indicators. Thus project strategies to help strengthen health services, as well as establish and strengthen linkages between communities and health facilities were right on target. The project supported the training of all health providers in IMCI. It supported the implementation of the outreach strategie avance immunization and prenatal care services. And in collaboration with the district team, it has tried to instill in health providers, the fact that community relais are part of their responsibility. i.e. to ensure that the health providers schedule supervision visits to the community health workers as part of their normal activities. There has been a bit of a challenge with the latter because many health facilities got used to the fact the project Animators were on the ground, supervising and supporting community based activities on a regular basis. Per the midterm recommendation, this mentality has slowly been changing over the last two years. But there is still a lot of room for improvement as health providers don’t always schedule designated time for this. Many though, at least touch base with the relais during the outreach activities. As the district is going ahead with the C-IMCI strategy, and trying to expand to new catchment areas with the potential support of UNICEF, it is expected that the community based work initiated by the project will continue to be seen as important. Certainly, health facility workers do report that the presence of these community resource persons has really been helpful. The continued importance placed on community based information derived from the work of the relais, is also expected to be an incentive for health workers and the district to maintain contact with community structures and entities and encourage them in their work. vi. How was the impact of BCC interventions measured/evaluated? The impact of the BCC interventions were evaluated through the bi-annual LQAS studies and the baseline and final KPC studies. But in addition to these quantitative surveys, the project conducted a qualitative FGD studies during the life of the program and at the final evaluation which revealed that mothers participated in health activities, were familiar with health messages and had been influenced accordingly. c. Capacity Building Approach i. Strengthening the PVO Organization Capacity building effects on the overall organization The experience and capacity of Plan Headquarters staff is already well established. Per the Detailed Implementation Plan, the focus of capacity building was to be at the Mali country level. Above and beyond the actual Kita Child Survival Project team, the Mali country office benefited tremendously from exposure to this program, starting with the project start up workshop conducted by the Senior Health Project Coordinator based in Washington. The Bamako based 36 Health Advisor also conducted a training on the development of Detailed Implementation Plans for the benefit of Plan program staff. There are four geographical areas of focus for Plan in the country. These Program Units all had the opportunity to learn from the Kita CSP, and increase their understanding of child survival programming. i.e. child sponsorship staff from all four program units have received training in the C-IMCI strategy, and several staff from two other program units participated in a training for the LQAS methodology. Plan Kangaba and Plan Kati learned about the usefulness of the quarterly coordination meetings at the health facility catchment area and want to replicate that activity. As a result of the Kita CSP, the Plan health portfolio in these Program Units will be expanding. The new country project outline for the next 5 years of health programming is actually a replication of CS project with the inclusion of nutrition, HIV/AIDS prevention, and primary school health. The Kita CSP staff also gave technical assistance to Plan countries in the region. They facilitated an LQAS training workshop which was hosted in Kita. Among the non Mali participants were Plan Guinea, Plan Senegal and Plan Benin. Capacity of the PVO to design, implement and evaluate effective child survival programs Two indicators were identified in the DIP to ensure the increased capacity of the Plan Mali team to do the above: 1. Existence of a training plan for CS personnel executed through regular training of personnel during the life of the project; 2. Operations research will be designed for at least two CSP intervention areas. Plan Mali invested a lot of time and money on capacity building of the new CSP staff, including training opportunities in the capital of Bamako and overseas (details discussed under the training section of this report). This was a worthwhile investment as it really prepared this team of professionals to undertake every aspect of the project, and be in a position to build the capacity of the local partner NGOs, and government and community partners as well. A lot that came with the CSP was new to the project staff. This included DIP development, LQAS monitoring, the Child Survival Sustainability Assessment, the KPC methodology and some of its indicators. Influence on other programs operated by the PVO (and in Mali) The influence of the Kita CSP on other Plan Mali programs operated by the PVO was discussed above. Most of Plan Mali staff are permanent staff and those transferred between different Program Units and the country headquarters in Bamako. This has the great advantage that capacity built stays in house and can be transferred to other geographical areas and staff through internal training activities and exchange visits; both things that have already been undertaken during the life of the project. 37 The project has recently become more visible and thus in a position to influence the Malian health community. A visit from USAID Washington Child Survival headquarters helped to stimulate the Mali USAID mission’s interest in the project, and Plan Mali was subsequently invited to do a presentation during a partners meeting. This was thus an opportunity for the project staff to share some of the project approaches, experience and results. And as a result, other NGO staff has become interested in visiting Kita as a learning opportunity. i.e. HKI who are just starting out with a new CSP in Mali. The Health Advisor also participated in a Roll Back Malaria forum where he did a presentation that included aspects of antenatal care and ITN distribution based on the CSP experience. And the Kita CSP and others such as Save the Children’s influenced the MOH at the national level to develop report forms for community health information system. As a result of these projects, soon after the MTE, the relais was officially recognized by the MOH as part of the health system. ii. Strengthening Local Partner Organizations Assessments on organizational capacity of local partners. As mentioned in the Kita CSP DIP, organizational assessments formed Phase I of the Kita project’s capacity building strategy. A formal assessment of the local NGO partner ARAFD/C and of the first 8 catchment area ASACOs was done in year 2 of the project. This focused general on the level of functionality of these institutions, as represented by the project indicators in the summary table starting on page 9 of this report. According to the assessment report, the ASACOs had gaps in information related to legalizing their entities, lack of knowledge in the area of rules and regulations, roles and responsibility of ASACO and its members. ASACO members had not received training for skills in financial and administrative management, and in drug inventory and procurement. Many of the groups were not very cohesive, there was a lot of absenteeism and they did not hold regular meetings. In addition, women were not permitted to hold positions within the organizations. ASACOs did not have Memorandums of Understanding with the MOH, and did not usually have formal work contracts with the doctors that they employed to run the community health center. There was no equity in payment of health workers or standards, or pay increases based on the number of years worked etc. This resulted in de-motivated health providers. The local partner NGO ARAFD/C is a consortium of NGOs. This partner was an already established Plan Mali partner in another part of the country prior to this project. Six of their Animators were transferred to Kita for the CSP, and two new ones were hired. The capacity assessment done on this organization was very general and overlooked the specific project indicators. But overall ARAFD/C was not found to be a partner that demonstrated the capacity to expand with the expansion needs of the CSP i.e. phasing in of new areas over the life of the project. Three of their Animators had to be replaced because they were non-performing, and 38 another local NGO by the name of AMACO, was brought on to be the co-implementing partner for new project areas. Less formal assessments were also done for the integration of non-formal community health centers falling under Phase II of the project expansion. These were done during a joint supervision visit with the district MOH. Among the needs identified was support in material and furniture, and strengthening of management and services. MOH district training needs were also identified during the DIP development, revealing that none of the clinical staff in the functioning community health centers had received training in IMCI. And in line with the partnership approach of the project, this partner was also included in the capacity building for all major project activities including TOT for IMCI, KPC survey and LQAS. Changes in organizational capacity of the local partner and contributing factors Changes in capacity of partners came under Phase II (implementing capacity building) of the strategy. These changes were monitored via a set of indicators included in the Results Summary table. NGO partners In this area, the project achieved four out of five of its indicators: · Both of the NGO partners have a document in which is consigned vision, mission and plan of action (100%); · Both of the NGO partners have the assessment results on their organizational capacity (100%); · Both of the NGO partners have complete financial reports: all the receipts and payments recorded (100%); · Both of the NGO partners have a management plan for the next year (provisional budget) (100%); · Neither of the NGO partners have a training program for their personnel (0%). Although baseline data on these specific indicators was not collected during the initial assessment, interviews with the local NGO staff revealed that they had accrued a lot of capacity as a result of being part of this project. This includes the following: · Technical capacity of field staff in child survival intervention areas; · Strengthened community mobilization and BCC skills; · Strengthened skills in training, community capacity building and participation; · Strengthened analytical, supervision and monitoring skills; · Development of expertise in C-IMCI and C-HIS; · Development of data survey skills – KPC and LQAS; · Strengthened partnership building and coordination skills. 39 Being part of the Kita CSP has also increased the two local NGOs profile and visibility in the health community. This is an opportunity to be able to forge other partnerships and implement new projects. At this point both ARAFD/C and AMACO feel very capable of replicating this type of project elsewhere (and making modifications based on their experience. i.e. they highlight the importance of having a nutrition component; of training field Animators in all related areas such as family planning, in view of their role as advisors and facilitators for health in geographical areas where people do not have easy access to information). Both institutions are in the process of developing proposals for potential donors. Community partners (ASACOs) The above mentioned assessments measured success in achieving project targets in four of the seven objectives. Among the objectives achieved was: · ASACOs that held their statutory general assemblies during the past year went from 62.5% at baseline to meeting the project target of 75%; · 78.3% of the statutory general assemblies meetings were held in the past year - obviously very good results although no baseline is available, and no specific target was set; · 85.4% of statutory office meetings were held during the past year, meeting the target 70% - although no baseline is available for that indicator either; · 100% of ASACOs have financial reports showing debits and credits, surpassing the target 75%. The response for this indicator at the Midterm LQAS was zero. Where the Kita CSP did slightly less well included: · 51.5% of the ASACOs held their statutory office meetings during the past year, compared to the 75% objective. But it must be noted that this is still a substantial increase from what was seen at the baseline, 12.5%. . · 8 women are holding high positions in ASACOs, compared to zero at the time of the baseline. This unfortunately was still below the project objective of 46 (2 for each of 23 ASACOs). · 30.4% of ASACOs have women in high positions – although no specific project target was set, and the starting point was also zero, with respect to the related prior indicator, and discussions with project staff, this result is not considered to be as good as hoped. It is though, a big step forward as traditionally, no women were allowed to participate prior to the start up of the project. Another change that came out during the final evaluation interviews was that as a result of having a better understanding of their roles, ASACOs in general are more in touch with 40 community needs and can better represent communities when making decisions about health services and outreach. They also have increased knowledge about health. ASACOs are generally more active than they used to be. They communicate quite frequently with the relais and ensure that people at the community level receive important information or announcements. And they also help with mobilization for key activities. The relationship with the relais is mainly related to follow-up and identification of needs, (in ITN for example). It should be noted that some of the ASACOs and ASACO members are not very involved in community mobilization and active support of the relais. As these are elected positions of great responsibility (including financial), many of these individuals tend to be older, trusted and respected community members who don’t necessarily have the youthfulness and energy to be out supervising and supporting the community health work being undertaken. Another issue raised is that the literacy levels of ASACO members are a constraint. As previously mentioned, the overall literacy level of the population is fairly low. Thus close supervision and support of these entities is important, particularly with regard to the reporting that is part of ASACO monthly activities. Best practices and lessons learned in capacity building of local partners Lessons Learned · The project experience highlighted the importance of project partners understanding of proposed strategies, agreement and clarity with regard to each institution’s roles and responsibilities. Regular communication and forums for discussion and planning between partners greatly facilitated this process and reduced initial misunderstandings and delays. · The partnership modality used in Plan Mali of annual contracts proved to be a problem for the CSP because there would always be a delay of a month or two before the renewal of the contract every year (caused by annual vacation leave of all Plan Mali staff in August). This delay extended to 4 months one year. As these interruptions interfered with program implementation, project staff proposed a longer contract with respect to multi￾year projects. (It was even suggested that hiring Animators directly would have saved the project a lot of time and effort). iii. Health Facilities Strengthening Approach for improved management and services at health facilities A key project indicator used to measure success in this area was the following: Number of CSComs that bought drugs with funds from the system of cost recovery from the past year. The result showed that all (100%) of the CSComs (health facilities) were able to accomplish this, achieving the project target of 75%. 41 Health facility strengthening by the project in collaboration with its district MOH partner, focused on training of health providers in IMCI and C-IMCI. In addition to improved case management, health facility staff participated in the KPC surveys and the LQAS surveys as well. The latter in particular was a useful tool for them to assess the health status in their own catchment areas, as feedback to the health facility and ASACO after the analysis was always a part of the process. Many health providers became more dynamic as a result, visiting communities regularly and supporting the strategie avance activities. Plan Mali was also able to equip health facilities through the use of matching funds. And the ASACO training (the Mayor and health provider also participated) in administration, finance, drug management and the decentralized health system, also contributed to strengthen health facilities capacity to meet the needs of the population. Improved and more consistent supervision of health facilities by the district health team in Kita and the project staff was also part of the plan for strengthening the management and services of the health facilities. Lessons Learned · Per the project experience, training and capacity building contributed to better understanding of roles and responsibilities and motivation and ability to do the job. Supervision and follow up support thereafter was also important to help clarify problems coming up. · All activities related to strengthening of the government partner tended to depend on the availability of per diem. When there was no per diem involved, getting government participation seemed a little slower. Plans for sustaining health facility strengthening The key factor for sustaining health facility strengthening is the ownership of the MOH in this aspect of the program, and follow-though in terms of regular supportive supervision, continued training and training of new staff when there is turnover. Certainly during the life of the program, the Kita district team members were very active partners, and very interested in participating in all the trainings and CSP activities. The issue of sustainability though, has always been uncertain because of district resources and priorities. Supervision has not always been consistent or followed the IMCI protocol. District Health Team members are constantly on travel attending meetings or trainings at the regional level or in the capital. Thus there is always a challenge in this regard. But as clearly seen during the final evaluation interviews, the district MOH has been very happy to be able to benefit from the support of the Plan CSP, and has completely bought in to all of the project strategies. As a result of the project, the district is undertaking a lot of things that they were not able to do before – such as outreach activities. And they are the only district in Region 42 with IMCI trained staff. The Kita CSP results have certainly helped put Kita ‘on the map’ in terms of being an example district with regard to community based activities, and improved health indicators. As a demonstration of their interest and commitment, the district has pursued negotiations with their UNICEF partner to continue some of these activities at the end of the project. Support for supervision visits are included as part of the budget. Linkages between facilities and the communities. As mentioned before, the project was quite successful in their endeavor to link communities and health facilities. Village Health Committee representatives participate in the quarterly coordination meeting organized at the health facility level for the health catchment area. And community relais are in constant contact with the health facility to submit their monthly reports or plan and coordinate activities such as the ITN re-impregnation campaigns, follow up of clients or the strategie avance. Thus the relationship between project communities and health facilities are well established, although not entirely perfect as some health providers and ASACOs are not really getting around to supervising and supporting the relais at the community level. iv. Strengthening Health Worker Performance Approach for strengthening health worker performance Please refer to the discussion under health facility strengthening above. Performance objectives Under the capacity building objectives listed in the Summary Table on page 11, the performance of health workers were measured through the following indicator: Percentage of users that say they are in general satisfied with the health services received from the structures. The result of the above shows no change (95.5% at baseline and 94.4% at final). But as discussed with project staff, it is clear that this is a problem related to the methodology of the data collection. Client exit interviews apparently was not the ideal method to get at this information, because culturally, it would be very difficult for a client being interviewed right at the doors of the health center to say anything negative about the service. Thus the results do not reflect the efforts and improvements noted during interviews with health facility staff and the district partner, and evident by the increased use of services as well. Plans for sustaining health worker performance During the final evaluation interviews, the district representative admitted that currently a few of the trained health providers don’t practice IMCI exactly as prescribed. They attribute this to the fact that supervision of this personnel has not been adequate. As previously mentioned, this is due to competing priorities, shortage of available staff, logistics and budget. But the MOH feels that health workers still subscribe to IMCI clinical practice, and assimilate new things faster now 43 because health facility financial reports show that the cost of prescriptions to the clients have consistently gone down due to reduction of polypharmacy. This is an indication that they are using the skills learned during the IMCI training. As was discussed with the district MOH, they are also expecting to be able to support refresher training for health workers through UNICEF funding next year. Tools used to assess the results of improving health worker performance The supervision tool that the district has to assess health worker performance is the from the standard IMCI guidelines. But again, because of limited time for supervision or combining different supervision activities, they do not often have time to use those actual tools. The change that they have noted over the life of the program is the one mentioned above – there is an overall reduction in the number of drugs being prescribed as seen through the reduced cost of treatment to the client. Training Effectiveness of training strategy Much of this has been discussed under the capacity building section. But it should be emphasized that training and capacity building of partners and stakeholders at every level was a big part of the CSP strategy to improve the health of women and children in the intervention area. The positive results of this strategy is demonstrated through several things, amongst which are: 1) the level of community participation and volunteer initiative in undertaking community based health activities; 2) the fact that community health Associations (ASACOs) are managing health facility finances, have greatly improved availability of drugs in the facility, meet regularly, plan quarterly activities and expenditures, and according to the project sustainability study, fall under the category of ‘dynamic.’ In addition to this, health providers trained in IMCI and C-IMCI have stronger links with the community. They are working in close collaboration with the ASACOs, collaborated with the CHW for outreach activities, campaigns and follow-up of clients. These are all new activities resulting from the training and promotion of program approaches. Thus in conclusion, the training has created a cadre of skilled and knowledgeable community members and health providers who are now in a good position to contribute to any number of things. Among them are health programming, assessments, problem-identification, planning, decision-making, financial management, and drug procurement. Training objectives The training objectives as outlined in the DIP included the following: 1). Training of Plan Mali Staff: (5 staff): KPC survey methodology; 44 Health Facility Assessment; Lot Quality Assessment Survey; Participatory Rapid Appraisal. These training objectives were met and surpassed as project staff in fact received a lot more training and preparation than outlined in the above. Among other things this included extended training and orientation for the Health Information Manager in Bamako, a month long training in Dakar for the Assistant Project Coordinator for M&E, an Epidemiology course and IMCI TOT for the Assistant Project Coordinator for Training, training in social communications in Belgium for two of the staff, and training in journalism techniques and publication of the project newsletter. The latter enabled the project to gain more visibility as project activities and results were shared within the district, region and at the national level as well. 2). Training of MOH health personnel: (6 TOT; 52 health workers, 30 matrons/midwives) TOT in IMCI; IMCI; Behavior change communication; Technical training (matrons and midwives). The TOT for IMCI was done as the above. Two of the six participants came from the Kita district team. Others were from Plan and the national MOH. The health provider at all 23 health facilities were trained in IMCI and C-IMCI, and 5 additional ones were trained to replace the ones who had transferred out. 3 doctors from the district level and a nurse also received this training. Facility health workers and district staff were also trained in KPC survey and LQAS survey methodology, perinatal health, disease surveillance, and community based information systems. The facility staff also participated in the ASACO financial management and administration training. 17 assistant midwives and 3 obstetrical nurses were trained in the management of women during labor, and newborn care. In addition to the above, the project also supported the training of 8 vaccinators for the strategie avance activities. 3). Training at the community level: (520 Relais, 26 ASACOs, 260 VHCs, 52 TBAs). Relais in child survival; TBAs technical training; ASACOs in financial and administrative management, community participation, disease surveillance; Relais and ASACOs in community health information system and surveillance. At the time of the DIP, the project overestimated the number of villages they would be working in. The fact that the government funds were not available for the development of health facilities in un-served catchment areas of the district, further reduced the estimated 520 relais to be trained. Thus 404 relais were trained for the 202 project villages (2 per village). Twenty-five traditional birth attendants were trained in the Phase I communities. It was not possible to do this 45 training in the second and third set of communities because of funding shortages. Twenty-three ASACOs, corresponding to the 23 catchment areas, were trained in the above in addition to decentralization and governance. And corresponding to the number of project villages, 202 VHCs also received training as planned. 4). Local NGO: (12 Animators). BCC strategy; C-IMCI; HIS/surveillance. These objectives were accomplished and more as the project decided to take on a second NGO partner and thus a total of 16 Animators and 2 Coordinators received the above training. In addition, this team also participated in the ASACO training on decentralization and governance, PLA methodology, health policy in Mali, and nutrition. The two Coordinators also benefited from a Water, Hygiene and Sanitation training workshop. Evidence that the training resulted in new ways of doing things, increased knowledge and skills Certainly the big investment that the project made in training and capacity building of partners at every level resulted in new ways of doing things, and increased knowledge and skills. Prior to the CSP, there were no community based health activities being implemented in the Kita district. i.e no monthly immunization outreach, prenatal care services or health education. There were no community resource persons or village based structures focusing on health issues, no community based information system, no communication between villages and health facilities, and no referrals and follow-up of clients. As mentioned under the assessment of capacity section, the existing ASACOs (only 10 of 46 potential catchment areas), were not really cohesive. They were not legally registered entities and did not really have the skill set to manage the financial and administrative aspects of the community health facility. Thus with the existence of all of these new activities that project partners have been undertaking, the coordination, joint planning and improved management of health programming, it is very evident that all of the training increased capacity and brought new skills that are being utilized. Plans for sustaining training activities In terms of training for the ASACOs, the Ministry of Social Development does have a small budget for the training of these entities – albeit limited. Thus the basic training of new ASACO members (membership is renewed every three years), and entirely new ASACO entities being developed with the development of new health catchment facilities, is more or less assured. What has been noted though is that a more intensive training program is needed for ASACOs to develop the capacity to function at the optimal level. Getting additional support for this in the coming years is not clear at this point, although there is a chance that ASACOs or the district can submit applications for ANICT government funding (National Agency of Investment in the Collectivity of the Territory) to support health activities. These funds are now available as part of 46 the International Monetary Fund loan forgiveness program. i.e. loan repayment moneys are being re-invested in health, education and social programs. As far as the training and/or refresher training for community health volunteers, the project partners feel that this can be assured by the facility health provider himself. And depending on the modality of the training, (i.e. if it is on-the-job), and availability of a budget for this from the health facility or district, this training does not necessarily need to have a lot of costs attached to it. But again the district hopes to leverage funding from other partners such as UNICEF for continued support of training, supervision and other activities. This also goes for the continued training of facility health providers as well. Are the sustainability plans for training realistic? The above-mentioned possibilities for training are realistic enough in that the Ministry of Social Development has already been able to undertake the training of ASACOs, the ANICT government funds are available every year for those who want to compete for it and the district does have the advantage of having UNICEF as a partner. In addition to the above, the Plan Program Unit is well established in Kita and both the Unit staff and Plan Mali are interested in leveraging additional funding to be able to continue with some of the work that the CSP started. d. Sustainability Strategy i. Achievement of Sustainability goals and objectives and evolution of sustainability plans through project implementation The Kita CSP sustainability objectives were the following: 1). Stimulate community participation and ownership; 2). Increase the capacity of the ASACOs in management, advocacy, and as community representatives, and 3). Ensure health services are financially sustained through a cost-recovery system in which the community participates through the use of the services. With reference to the first objective, this was certainly accomplished, as even while the community relais, may wish for remuneration or be de-motivated at times, according to final evaluation FGDs, s/he continues to feel a sense of responsibility for helping community members with health issues. They report that what keeps them going is the satisfaction that comes with seeing improvements and better health in their communities. As witnessed by project staff, village chiefs and counsels have also demonstrated this ownership for the health and well being of their communities. In final evaluation FGDs, chiefs mentioned having pulled together the relais candidates and village health committee members; being in 47 touch with them as it relates to community awareness and activities; and stimulating the population when the need arises. Village chiefs also help the ASACOs when they have problems, and assure village quota on the annual collection that supports the community health facility. Some chiefs have even developed a committee to survey the cleanliness of the community and fine people who do not respect this. Sustainability objective number two was also achieved. Please refer to the discussions under capacity building. With respect to the last objective, the health facility assessment did show that health facilities (33.3%) were able to purchase and provide the full package of essential drugs needed for clients during the six months prior to the assessment. Thus at least a third of the facilities (as opposed to none at the start-up of the project), seem to be doing okay in this area. Qualitative interviews with the ASACO during the evaluation did reveal though that many of their budgets are very tight. They said that sometimes they have to dip into the income from the sale of medicines, to cover other costs (such as salaries), which they have been advised not to do because they need those funds to renew the stock of medicines. Although clinic visits, the sale of the annual health cards to households and village annual quotas should be enough to cover the running cost of the health facility, this is sometimes a challenge because: 1) not everyone has caught on to the importance of buying the health card (health coverage) which reduces the families health care costs during the year; and 2) some health facilities such as urban ones, have not been able to benefit from the annual village quota system of support for the community health center. But ASACOs are aware that they need to make more efforts in the above for the health centers to become more financially secure. One strategy that many have already undertaken is to give the cards to the relais who are well known to community members, and who are able to go house to house to educate those who don’t know about the importance of health coverage. Thus they are well positioned to help increase the sale of these cards. As an incentive and reward for this work, relais receive a percentage of the sale of each card. Some of the project Phase II ASACOs who had not yet been weaned by the project, were in fact quite aware that things would not be easy after the project’s withdrawal. In preparation for this they requested an increase of village contributions and had been saving up money during the last six months of the project. Per the DIP document, the project expected to leave key advancements in place: · Strengthened community capacity to organize health actions and management systems to increase coverage of maternal and child interventions; · CSComs who are responsive to the health needs of the community, involved with the community, and who provide quality services and a Minimum Activity Package (MAP); · ASACOs who are community representatives advocating for the community health needs; · Community and the health facilities linked through outreach activities, functioning health committees; 48 · An improved mechanism for the financing of recurrent costs and costing of CSCom fees. This initial sustainability plan did not change or evolve through the implementation of the project, and all of the above were key areas that improved over the life of the project. ii. Status of the phase-over plan and continuing technical and management assistance As was planned in the DIP devolution strategy, by the end of the program, much of the responsibility for supervision, monitoring and training had been handed over to the ASACOs, the communities and the MOH. Again, 8 catchment areas of the 23 had been phased out two years prior to the projects’ end, and activities and reporting in those areas were still ongoing at the time of the final. All training activities for the project were completed early in the last year, and the MOH was in discussions with UNICEF about support for ongoing training, months before the close out of this project. As previously mentioned, although the project encouraged the district MOH on the issue of supervision, this was one area that has faced a lot of challenges. As also mentioned in the DIP, Plan’s country-wide sponsorship program that includes health will continue to be present in the Kita district. Kita, as well as Plan’s other program units in the country intends on expanding their health component to include C-IMCI and thus with adequate resource mobilization, can feasibly continue to give the district of Kita some continuing technical and management assistance. iii. Have the approaches to building financial sustainability-- (e.g., local level financing, cost recovery, resource diversification, corporate sponsorships) been successful? Plan as an organization has done well with resource mobilization. They were able to get an additional $40,000 from the French for malaria control after participating in a documentary about the problem in Mali and speaking at a press conference. They were also able to use $4,000 from the project NICRA to support additional activities in immunization. And they were able to get funds to purchase some materials and equipment for new health structures lacking support. With their sponsorship funds, Plan Mali is also able to envisage expansion of the health portfolio to increase their impact on the child survival indicators in their respective sponsorship communities. In terms of the project partners on the ground, as previously mentioned, the district MOH has been able to negotiate additional support from UNICEF; whereas other districts in the region were also competing for a chance at getting this funding. As the district has more visibility now in the health community, there is a chance that they will be able to leverage more funding to expand and cover catchment areas that are just opening up. i.e. 10 new ones in the Kita district are now ready for health interventions. But this cannot be done until the new health structures get materials, furniture and start-up supplies. And there are a potential 10 or 11 more that need to be developed in the future to cover the entire district population. 49 Work with the ASACOs and community mobilization has certainly increased the populations awareness of their roles and responsibility with regard to community health centers (CSComs). The training and support to build this awareness, ability to mobilize community resources and increase the use of services has resulted in much progress in this area. It is important to mention though that due to the delay in the development and subsequent integration of a number of catchment areas into the project, some communities (ASACOs, VHCs, relais) are not in the least bit prepared for the phase out or sustaining the investments already made. This is particularly the case for the 4 catchment areas that were integrated as late as May and November 2005 (11 and 16 months prior to the end of the project). Needless to say, much of those months have been dedicated to just getting initial understanding, buy-in and awareness. After which there was the training and capacity-building of these cadres. It is thus unrealistic to think that they can now carry on with the vision and aspirations of the project, as have the communities who had two, or close to three years of intensive support and subsequent follow-up. There are an additional 5 catchment areas that were integrated barely 20 months prior to the end of the project. They are in fact better off than the last set. And some of the ones that are the most challenged geographically (in the Kaarta), are much mobilized and have dynamic health workers. But with only 20 months of support, the phase-out may be a little premature to expect a good level of financial sustainability. iv. Community demand for services, and engagement/influence Please refer to discussions under the BCC section on how the program built demand for services. And the previous sections on capacity and sustainability on village chief and ASACOs efforts in increasing community awareness of health services, importance of health coverage (annual household health card membership) and importance of community support for sustaining the health center. The VHC/relais who are represented at the quarterly coordination meetings and the ASACOs management bodies are entities that give the community sufficient opportunities to influence how services are delivered. Their engagement and influence with regard to community outreach, negotiating and participating in planning and allocation of health facility resources has already been demonstrated. Again, this is more pertaining to communities who have been with the project for longer. C. Program Management C1. Planning a. How inclusive was the program planning process and what effect did this have on the implementation process? The MOH government partner at the district level was involved in every step of the initial planning, and continued to be throughout the life of the program. This included the Chief Medical officer and his Assistant Chief Medical Officer team. The first NGO partner, on the 50 other hand, was not involved in the DIP development. But all Animators received a copy of the DIP subsequent to its development and had a chance to review this together. There are monthly and quarterly meetings between the Plan Kita CSP staff and the partner NGO staff. And there are quarterly coordination meetings held with the district MOH. On a bi-annual basis, the district MOH leads a Monitoring meeting with the ASACOs, facility health workers, president of the communes, Ministry of Social Services, NGOs, and PDRIK. These meetings are an opportunity for all stakeholders to coordinate their efforts and do joint planning based on the review of progress and challenges. A Steering Committee for the Kita CSP was developed with representation from the Kita District, Kayes Region, National Federation of Community health Associations in Mali, the partner NGO representative, Plan Mali representatives, President of the District Council and the CSP Project Manager. Meetings are also held twice a year; once in Bamako and once in Kita for review, analysis and decision-making. Every year there is a team evaluation of project activities, review of the LQAS results, and review of progress towards attaining the annual objectives. Both NGO partners participate in this evaluation and in annual program planning. An annual meeting is also held with the district MOH to review achievements and contribute to the Kita district annual planning process. All of the above demonstrates just how inclusive the planning process for this entire project has been. And this has certainly had a positive impact on the implementation process. The need for regular meetings and the involvement of all stakeholders in planning and decision making was seen early on and this fostered good partnerships and a rather smooth implementation process by all standards. b. DIP work plan and what could be added to DIP preparation and review process that would have strengthened implementation The above was done in a very thorough manner and implementation was smooth as already mentioned. c. Gaps in the DIP and how they addressed by the program staff The major issue that the project ended up dealing with, was that their extension plan (i.e. phasing in of new catchment areas over the life of the project), was entirely dependant on PDRIK government plans to support the development of new health facilities. The project premise was that community health interventions could only be implemented in areas that already had a referral community health center to work with. Unfortunately many areas forming part of the Kita district actually do not have access to health services at all. These areas, outside of developed health catchment areas, were thus not considered as eligible for participation in the CSP. 51 As such, when PDRIK did not come through as quickly as planned, there was no backup plan in place. And the project had to delay the integration of new catchment areas to the point where the last ones only received support for 11 and 16 months as previously mentioned. In addition, there was also the withdrawal of Belgium funding in Kita. They also supported new health structures with equipment, and supported nutrition activities as well. Project Animators interviewed during the final evaluation process all agreed that nutrition was a component that was notably absent considering the needs (18% of children 0-23 months were underweight at the time of the baseline). But again this was a result of the assumption that either the Belgium’s or Helen Keller International (whose program was in fact relatively limited) would be covering this component. It is to Plan Mali’s credit that they were able to mobilize a lot of private funding to help the district prepare some of the new health structures. Thanks to the additional support in equipment and medical supplies to these facilities, the project was able to push ahead to eventually include 23 of the 26 health catchment areas planned. The project should also be commended for the extra effort that they put into helping assure EPI coverage in non-project areas. As the project interventions in year 1 and 2 only covered 8 of a potential 46 catchment areas, the Kayes region and Kita district were very happy that the project agreed to extend the immunization component to cover the whole district. C2. Staff Training a. Change in knowledge, skills and competencies of the program and partner's staff, and evidence that the staff has applied these skills both within the program and in another context The Plan Kita CSP staff was all new to the USAID BHR PVC Child Survival Grants Program. As such much of what they were doing was new to them as individual professionals and new to Plan Mali as well. All project staff interviewed spoke about what a tremendous learning experience it had been to participate in this project. They were very appreciative of the technical support received from their headquarters staff, and the new skills that they had developed and used. i.e in the development of detailed implementation plans, community health information systems, KPC and LQAS survey methodologies, capacity and sustainability assessments, training, supervision, M&E and working in partnerships. Program staff has indeed had opportunities to apply these new skills for the benefit of the project, as well as outside the project for the benefit of other program units and in the Bamako office. As mentioned before, Plan staff from other countries in the region also benefited from LQAS training done by the Kita CSP staff. NGO partner staff has also developed a lot of new skills, as mentioned earlier in the report under changes in capacity. During the final evaluation interviews they spoke about how they used their new skills in BCC to support CHWs in their work; how their strengthened competency in community mobilization and capacity building helped them to be creative as they dealt with the challenges of dealing with communities who are inactive or lack incentive. And the supervisors also talked about how they had learned more about supervision as a result of their supervision activities in the project. They also said that they are in a better position to contribute to proposal developments being undertaken by their organizations. They feel particularly competent in the area of community based child survival programming and C-IMCI. The latter have been 52 applying these skills as such in an effort to pursue new partnerships at this close-out point of the CSP. District MOH partners also acquired new skills as discussed in the training section of the report. As the district and health facility staff participated in the KPC and LQAS surveys, they had an opportunity to use these new competencies as well. New skills in case management were also put to use. Although with the coming of IMCI, the district staff had access to those tools for supervision, they unfortunately have not used them much because of time constraints. Also new and a result of program support, is the implementation of disease surveillance. This remains apparently theoretical in much of Mali, but has now been able to be implemented in Kita. All of this data, including the data derived from the CHIS, will start to be entered into the reporting system in 2007, and thus forwarded to Bamako at that point. b. Resources dedicated to staff training As was discussed in corresponding sections, a lot of resources were dedicated to staff training. Plan did not cut corners in this regard and it really paid off, as evident in the number of assessment activities that the team was able to undertake, the number of quantitative surveys, the close supervision and monitoring of the CHIS, detailed planning and good level of documentation on project experiences and activities. c. Lessons Learned about building the capacity of program staff Although not really a lesson learned, as Plan did in fact dedicate a lot of time on staff capacity building right from the start, as mentioned above. But Plan has really seen the return on this investment as: 1) it contributed to the success of the program; and 2) it has contributed to overall Plan Mali capacity in health and child survival programming. Thus their experience serves as an example for other programs. C3. Supervision of Program Staff a. Supervisory system The supervisory system within the CSP was more than adequate. Technical backstops from Washington visited the program regularly and were able to give timely support and monitor project progress. Technical supervision of the project also came from the Bamako, Mali National Health Advisor who was involved in all key aspects of the program. Supervisors for the partner NGO staff in Kita came from their Bamako offices on a monthly basis. The NGO project Coordinators on the ground supervised Animators every month for a period of several days each. After the midterm evaluation, the CSP staff decided that it would be a good idea to strengthen supervision in order to increase project impact. Thus the M&E Officer and the HIS Officer each took responsibility to supervise the work of the NGO Animators in the 53 field. They spent several days out of every month in project communities supporting activities and reporting in particular. Unfortunately one gap in supervision was that of the health facility workers themselves as the project trainer and manager were dependent on district MOH staff availability for joint supervision. Again one challenge was that the district team of doctors was out for different activities and priorities on a regular basis. So it was soon realized that it was difficult to follow the recommended guidelines for supervision. The solution was to do combined supervision of different programs, and touch upon all the different things necessary while out on a visit. But this was also sporadic. Apparently there were specific UNICEF supported health facilities and these received more attention than the others. In talking to the district MOH during the final evaluation activity, which they participated in, it was very evident though that they are well acquainted with each health facility, what is happening there and what the issues are. The bi-annual Monitoring meetings are very useful in that regard and they too have regular contact with health providers through training activities organized at the regional level, meetings and so forth. b. Institutionalization of supervisory system Again, although the supervisory system is in place, this is the biggest area of weakness. The challenges are not only at the district level, but also at the health facility and ASACO levels, where there is minimal supervision and support of relais working in communities. Many health workers supervise the CHWs when they come out for vaccination activities. But needless to say, as the visit is not dedicated to supervision, it tends to be inadequate. c. Evidence that the program’s approach to strengthening supervisory systems has been adopted beyond the program The CSP proposed some good strategies for the strengthening of supervision systems. For example in the training of ASACOs, which included the health provider, they encouraged them to include a supervision budget so that relais could be supervised. But as per the above, for various reasons including lack of motivation, this has not happened in most places. At the district level the CSP encouraged the team to develop a supervision package, rather than go out only for one thing. This would allow the district team to maximize on every opportunity. Although this has been adopted to a certain degree, it has not been consistent. These strategies related to IMCI and C-IMCI programming have been shared with other program units who are interested in replicating this experience, but they have not yet been implemented. C4. Human Resources and Staff Management 54 a. Personnel policies and procedures of the grantee and partner organizations to continue program operations that are intended to be sustainable As mentioned in the sustainability section, the Plan Kita Program Unit is established in Kita and will continue to support health interventions to the degree that they can, albeit potentially at a reduced effort than what the CSP had. Plan Mali is trying to mobilize resources to support child survival interventions in all their program units. The CSP staff are also permanent Plan Mali staff and will thus continue to be resources for the organization whether they continue to be based out of Kita or elsewhere. The government partner policies and procedures are in place to continue program operations that are intended to be sustainable. The MOH has plans to train new health worker personnel, and the training of ASACOs is part of Ministry of Development responsibility. Again the MOH has bought into all activities that were introduced by the project and as they have recognized the great value of community relais they plan on continuing to support this cadre as part of the health system. They give them per diems whenever there is a special activity that they can be engaged in. b. Morale, cohesion and working relationships of program personnel and how this affected program implementation. The CSP was a project that was separate from the rest of the Plan Mali programming, which was all within the context of sponsorship communities. Because of this, it fell a little bit outside of the program unit cohesion. Thus at the beginning of the project, there were some initial adjustments that had to be made. But since then project staff morale has been good. Cohesion and working relationships have been excellent, and this has contributed to a positive working atmosphere and camaraderie amongst the team members that has had a positive affect on program implementation. c. Level of staff turnover throughout the life of the program, and the impact it has had on program implementation. The only change within the Plan CSP staff was the transfer of the first Project Coordinator to Bamako headquarters to move into the position of National Health Advisor. As the team was very seasoned by then, the transition of the Assistant Project Coordinator for Training into the Project Coordinator position was fairly seamless. The fact that the first Project Coordinator was still in a position to continue supporting the program in such things as key studies (Sustainability Assessment, KPC etc) was helpful. Three Animators from the first partner NGO, ARAFD/C were relieved of their duties and replaced in year two of the project. This was in fact for the better because the new staff were very capable and hard working. d. Plans to facilitate staff transition to other paying jobs at the end of the program 55 As mentioned before, all of the Plan CSP staff fell in the category of permanent staff who was working with Plan in other program units for many years before the start up of the CSP. The only new staff was the first Project Coordinator who was subsequently transferred to Plan Bamako and thus is now permanent staff as well. All CSP staff will be re-integrated where needs are identified, and will be resources to the organization as they move towards replicating the CSP experience in all of their program units. C5. Financial Management b. Adequacy of the PVO’s and partners’ financial management and accountability for program finances and budgeting and program implementers budgeting skills The Kita CSP worked within the Plan Mali financial management and accounting system. They received adequate support as there was a project accountant based in Bamako who was designated at 100% level of effort. For routine expenses, the CSP team had a petty cash of 100,000 CFA (approximately $200) which was renewed as it was being spent down. Money for special activities and trainings would be requested and sent separately. Working in collaboration with the project accountant in Bamako, the first Project Manager actually developed a tool to monitor project expenditures. As this was during the first half of the project, it was very helpful for everyone to keep track of expenditures and manage unforeseen expenses. c. Resources in place to finance operations and activities that are intended to be sustained beyond this cooperative agreement Plan Mali has been in the process of mobilizing funding to continue supporting the district of Kita in child survival, in addition to positioning itself to replicate this experience in other program units. During a meeting with the Program Support Manager and Country Management Unit in Bamako at the end of the evaluation, they expressed the fact that as a child-centered community development organization they are very much in support of the CSP interventions. As such, they are committed to ensuring one year of bridge funding from Plan to consolidate the achievements in Kita, and are particularly interested in seeing continued support to those catchment areas that are newer and not ready to be phased out. The difficult situation that the CSP found itself in due to the lack of PDRICK funds for community health center construction, equipment and materials, has pushed Plan Mali to focus on supporting the whole package, as one element cannot be implemented without the other. As a result, they have sought support from Plan Canada to be able to provide assistance with facility construction and equipping, as they used to in the past. Using the community mobilization and partnership strategy that was shown to be successful in the CSP, Plan feels assured that their investment now is worthwhile, as the strategy supports the sustainability of these health structures. 56 Again it is feasible that ASACOs tap into the aforementioned ANICT government resources for support with health activities that they need to undertake as well. d. Outside technical assistance available to assist the grantee and its partners to develop financial plans for sustainability In house technical capacity was adequate to meet this need. C6. Logistics a. Impact of logistics (procurement and distribution of equipment, supplies, vehicles, etc.) on the implementation of the program All procurement was done within the Plan Mali system. This included one project vehicle and a motorcycle to support every project Animator, Coordinator and the Plan CSP staff as well. Motorcycles were also used for supervision when necessary. The only problem encountered is that the difficult terrain in Kita sped up the wear and tear on motorcycles. By the time of the midterm (by year 3 of the project), the pool of motorcycles was in very bad condition. So until it was replaced soon thereafter, this did become an issue that field staff had to deal with. b. Logistics system and support of operations and activities that are intended to be sustained Again Plan Mali and the Kita Program Unit are very interested in continuing to support child survival in Kita – at least to the degree possible. And for the most part, both the district and health facilities have the logistical capacity to undertake activities such as outreach and supervision. The constraints tend to be related to staff time and conflicting priorities. C7. Information Management a. Effectiveness of the system to measure progress towards program objectives The management of health information and monitoring and evaluation was among the projects strongest areas. The priority given to this aspect of the project is easily demonstrated by the fact that Plan dedicated two staff persons to this area: a Health Information Manager and an Assistant Project Coordinator for M& E. This was an excellent strategy to ensure that project progress could be measured every step of the way, and they have to be commended for this. The system established was very effective because not was monthly activity data available to everyone and an opportunity to identify areas of weakness, but progress towards achieving health indicators was also effectively measured through the bi-annual LQAS. The Kita project also went so far as to develop annual objectives on project indicators, as another strategy to increase 57 their chances of achieving the end of project objectives. As previously discussed, annual achievements and challenges were reviewed at project partners meetings every year, and all project staff, including Animators, would plan and strategize on how to achieve continued improvements in the following year. Each Annual work plan developed, and planning for specific program activities was based on the achievements and challenges encountered in the previous year. b. System of collecting, reporting and using data at all program levels and how program data was used to make management or technical decisions. There was a lot of community based data collected every month. And this was monitored both on a monthly and quarterly basis. The reports coming from the relais were discussed and analyzed at the health facility level by the health provider and the ASACO who were able to make decisions based on this data. i.e. around outreach, follow-up, needs in ITN etc. The CHIS data from relais was compiled by Animators in each catchment area and submitted to their supervisors. Animators also wrote their own monthly narrative reports, which included project activities undertaken, supervision, achievements and problems encountered. Each of the NGO Coordinators supervising the work of their Animators subsequently compiled all of this information into one monthly report and submitted it to the Plan CSP staff. The Health Information Manager and the Assistant Project Coordinator for M&E were each responsible for overseeing the work of one NGO team, and they would thus have their respective monthly meeting to discuss this data and report content. The Plan staff also reviewed individual Animator reports in order to be able to assess how things were going in each catchment area. And the field supervision of Animators in their communities also gave the Plan supervisors a chance to look at the community registers, check on the completeness and accuracy of the data being reported and give additional technical support where needed. And lastly, every Monday the 4 key Plan Kita CSP team members would meet to discuss the previous week, issues and plan their activities for the week accordingly. Thus the review of data, analysis and decision making was also being done at the project level. Partner NGO reports were submitted to the Plan CSP supervisors on the 25th of every month, and once it was reviewed, analyzed, and compiled, a report would be sent to the district MOH partner by the 5th of every month. The CHIS that was adapted by the project consisted of 7 registers. These registers consisted of the following: 1) Households 2) Births; 3) Deaths; 4) Children under 5 years Vaccination and Vitamin A; 5) Pregnant women Antenatal Care, Delivery and Postnatal care follow-up; 6) Disease Surveillance; 7) Referrals. Prior to the development of the above registers, the project did a complete census of the population in order to identify the target groups. As household compounds tend to consist of an ensemble of households, they registered the whole group under the name of the absolute head of 58 the family. i.e. the old man or father of the younger fathers. The number on the household registration card was also used as an identification mark, after which they wrote up the names of each individual family, including all the children under 5 and pregnant women. This census data is updated every month as newborns and pregnant women are identified. c. Program staff skills to continue collecting program data/information and to use it for program revisions or strengthening Certainly the program staff, including the NGO partner staff and community volunteers, is sufficiently skilled to continue collecting program data/information and to use it for program revisions or strengthening in the future. The Plan Mali national office is very interested in being able to use LQAS in the other program units, and the MOH at the regional level in Kayes is particularly interested in the community based information system. It has been encouraging other districts to come and learn from Kita. d. Did the program conduct or use special assessments, mini survey focus groups, etc. to solve problems or test new approaches? Give examples of the research, use of data, and outcomes. Participatory Rural Appraisal was used to understand certain behaviors. e.g. although there was an increase in prenatal coverage and visits to the health facilities, delivery at the health facilities was still low. The results of this activity confirmed that the main problem revolves around transportation to the facilities. As previously mentioned, one study undertaken through the use of Focus Group Discussions was on Malaria control insecticide treated net management and use by communities. The incentive behind doing this study was to assess this activity so as to be able to justify the additional procurement of nets by Plan Mali for the project. (As mentioned before, this was prior to the arrival of RBM nets that were distributed for free to target groups meeting the established criteria). The study looked at knowledge and attitude on the prevention of malaria, community management strategy and what they proposed for sustainability. The results of the study showed that people do know the methods of prevention, and were very willing to purchase nets to protect their young children. In terms of management of the nets, women tended to feel that a woman’s management committee or group would be best for both the management and sustainability of the activity. Men, on the other hand, tended to think that the relais, VHC, village chief or the mayor should manage the nets. Overall, community members were willing to pay from 1000 to 1500 CFA for a net. Also already mentioned were two Sustainability Studies undertaken by the project in 2005 and again in 2006. The 2006 scores showed a lot of improvements, and a generally very optimistic picture for sustainability with regard to at least 5 of the 6 different sustainability indexes. i.e. scores were between 65 and 75. Still weak though was the score for the Environmental index (53), reflecting limited CSP support in this area. 59 e. Strengthening other existing data collection systems (i.e. government) This was not a need expressed by the MOH. f. Program staff, headquarters staff, local level partners, and the community understanding of what the program has achieved As one of the strong points of this project was the health information system and M&E, all program staff, headquarters staff, local partners and the community had access to how the project was doing in terms of progress and achievement of program indicators. As previously mentioned this was shared in quarterly meetings at every level and discussed on an annual basis. Program partners even participated in the KPC and regular LQAS. During the final evaluation FGDs and interviews with program partners at the community, NGO and MOH level, all spoke about the achievements of the program. At the community level, there was particular appreciation for the Malaria control activities and the ITN distribution as something that had really contributed to reduced infant/child deaths. Community people also talked about how the project had brought them vaccination, prenatal care, improved sanitation and hygiene and education on health. At the partner level comments reflected the prior mentioned, as well as achievements in capacity, skills￾building, and technical knowledge. At this end point in the program, the final KPC and assessment results will also be shared with program partners and the final evaluation report will be translated into French. g. Use of program’s monitoring and impact data beyond this child survival program Program monitoring and impact data and project experience has been shared with the USAID mission and other NGOs and stakeholders working in Mali via presentations and meetings. This was also done through the project newsletter which was distributed in Kita, Kayes and Bamako. Program results and activities were also shared with visitors such as the French press mobilized by Plan France during the World Malaria Day. C8. Technical and Administrative Support a. Types, sources, timeliness, and utility of external technical assistance the program has received to date. From the US headquarters office, the CSP received support and assistance in training and implementation of the initial KPC baseline survey, including the Rapid Catch data. A Detailed Implementation Plan Workshop was also conducted which helped Plan Mali, the project staff and partners understand the process that needed to be undertaken. They were also assisted with the development of the latter. Through the life of the program, visits were made every year, particularly supporting planning, monitoring and documentation. Headquarters staff also helped 60 gave them training on the Child Survival Sustainability Assessment framework which they were able to use twice during the life of the project. Overall, the CSP staff is very happy with the support that they received. It was deemed to have been very timely and useful to them, greatly facilitating the successful implementation of this project. The CSP team also received technical assistance from the Plan Mali office in Bamako. The Health Coordinator gave them support with the LQAS, strategy development, and linking with the Plan Regional Health Advisor in Accra. The Plan Mali Country Director and Program Support Manager also visited the project and provided support. b. Assistance need that was not available and how PVO headquarters and/or USAID could better plan for the technical assistance needs of PVO programs No technical assistance needs were identified whereby assistance was not available. c. PVO headquarters and regional technical and managerial support of the field program and time devoted to supporting this program. The Senior Health Program Manager at Plan headquarters in Washington devotes 10% of his time, and the Health Associate devotes 25% of his time to backstop this project. The Program Unit Manager in Kita dedicates 25% of his time to give management support, and 5% level of effort comes from the Country Director, Program Support Manager and Grants Support Manager. C9. Mission Collaboration Plan Mali has a good relationship with the USAID mission. This relationship was primarily fostered by the visits from their Child Survival technical support staff in Washington, who made a point to visit the mission during visits in-country. The relationship with the mission became closer in the last year or so of the project, as a result of a visit from USAID Washington. Plan has been participating in USAID partner meetings and has presented the project at this forum as well. The mission and partners have expressed a lot of interest in the program approaches, experience and results. And Plan staff will be sharing some of the tools such as LQAS methodology and the Child Survival Sustainability Assessment framework. At the time of the development of the proposal, the USAID mission was very much in support of the interventions that Plan was proposing. The Mission continues to be in support of the national policy priorities, including malaria, immunization and IMCI. C10. Management Lessons Learned 61 According to project staff, the key management lessons learned are: The monthly review meetings with project staff have been a key management strategy. The project directors learned early on that in addition to the technical planning and evaluation tasks at the monthly meetings, it was key to spend time interacting with staff to learn their concerns and discuss the difficulties they were dealing with. The monthly meetings became a forum for sharing concerns, problem solving and strengthening the bonds between staff members. It seems to have worked well as there has been very little turnover among staff in the five years of project. The management structure of the project did not change during the project LOP. Because of the well experienced staff and stable structure and procedures, the project was able to maintain its operations without interruption during the departure of the first project director. D. Other Issues Identified by the Team There were no other issues identified by the team. E. Conclusions and Recommendations In conclusion, the final KPC survey and other data show that the Kita CSP achieved the majority of its objectives. It is important to note that in terms of achievement of objectives, behavior change was not a problem. Most objectives not achieved, were related to coverage. i.e service and supply challenges that were outside the direct control of the project. The result of project assessments in terms of capacity of partners shows that a lot of progress was made. Health committees are functioning well and achieved promising scores in the Child Survival Sustainability Assessment). Qualitative interviews with community members and partners at all levels revealed strong and positive sentiments from all parties about the project. Per the latter, the project contributed to overall community health and hygiene; increased access and coverage; increased knowledge and care-seeking; community ownership and participation; technical and managerial capacity in health services; technical capacity for the development and maintenance of a community based information system, data analysis, planning, decision-making, monitoring and evaluation. Most important achievements 1. Significant reduction in child morbidity and mortality due to malaria, pneumonia and diarrhea (per qualitative interviews, CHWs and KPC data). 2. Strengthening of EPI ‘Strategy Avance’ outreach activities and integration of prenatal care as part of the outreach service. 3. Increased use of Insecticide Treated Nets and household coverage by ITNs in project area. 4. Marked improvement in hygiene and sanitation conditions in project households and communities. 5. Improved quality of health services. 6. Establishment of a functioning community based health information system. 62 7. Strengthened linkages between communities and health facilities. 8. Increased use of health services by project populations. 9. Community mobilization and participation for maternal and child health. 10. Capacity building of community based organizations and resource persons. 11. Increased knowledge and improved health practices of mothers in project communities. 12. Strengthened technical capacity of local NGOs to implement health and child survival projects. 13. Strengthened technical capacity of all local partners in monitoring and evaluation. 14. The CSP was able to demonstrate the vision of partnership dynamic whereby stakeholders at every level played an important role in the achievement of a common goal and objectives. Constraints and other factors affecting program performance · The CSP was dependant on the Kita Integrated Rural Development Project (PDRICK) for expansion and equipping of health facilities, and thus when there were problems with the latter, the project experienced delays in integrating new catchment areas. Between a possible 46 catchment areas in the district, there were only 10 functional centers at the beginning of the project, and a subsequent 13. The project had hoped to work in a total of 26 during the life of the project, and had not expected to be in a situation where they would only have a limited amount of time in the last areas integrated. · Inaccessibility of certain communities during the rainy season which meant that for close to six months out of the year, no outreach services or supervision activities could be done in those areas. Project Animators and community resource persons working in those areas continued to work without additional support or community health services. · Kita is the largest district in the region, with the second largest population. And yet up to 50% of the district has no access to services at all. · Mobilize funds to cover community health center costs and outreach is a challenge. Although resources coming from the village quotas, sale of household annual health insurance card, and the MOH helped to cover some of the basic expenses like salaries, ASACOs did report that they can barely cover expenses and sometimes run into a shortage of funds. Although the Mayor’s offices all consider health a priority and are very much in support of the project activities, for the most part, they do not have enough of a budget to contribute to covering health costs in their catchment areas. i.e. salary for vaccinators, the purchase of motorcycles for outreach activities or fuel. (Sometimes they can hardly pay their own personnel). Lessons learned 63 1. The amount of time needed to achieve project objectives is not adequate to assure the sustainability of those achievements. i.e. projects need to allocate additional time for continued capacity building, ownership and systems strengthening. 2. Involvement of the Community Health Associations (ASACOs) and Community Health Workers (Relais) in quarterly planning through coordination meetings is a forum for community mobilization, good governance and ownership of communities in health activities. 3. To ensure the success of project activities it is very important to establish a regular forum for communication, exchange, planning and coordination of all partners. 4. (Relais) are indispensable for the achievement of reduced maternal and child morbidity and mortality. 5. The motivation/compensation of Relais by the ASACOs is more sustainable than compensation from the community itself. 6. The presence of project personnel for technical support in project communities is critical to the achievement of project objectives and the transfer of capacities. 7. To obtain adequate coverage of household by ITNs, it is necessary to make them available at no cost. Without the availability of free nets through the RBM strategy, achieving desirable coverage is very difficult, if not impossible. 8. Periodic mini-surveys of community households is a tool that is very useful in helping to identify problem areas and targeting for IEC/BCC. 9. LQAS monitoring enables the project to clearly identify problem areas by community. 10. In areas where the health service provider is very dynamic, an improvement in health indicators can be achieved relatively quickly. 11. Using Relais to sell the health card (carte d’adhesion) is a strategy that increases the sale of these cards by bringing them to people’s door step. The small sum that the Relais earn off of the sale of each card contributes to motivating them in their work, and the increased number of sales increases the income for the health facility. Promising Practice Integration of prenatal care outreach with community EPI services, as a strategy to increase coverage for pregnant women. Recommendations 1. Document and replicate the project experience, interventions and strategies to the other Plan Program Units. 2. Submit a proposal for funding to support the extension of program activities to cover the remaining areas of the district. 3. Ensure the continued support and monitoring of project areas and interventions by the Kita Program Unit for a minimum of two more years. 4. It is recommended that the district and the Ministry of Social Development encourage and participate in the quarterly health catchment area coordination meetings initiated by the CSP as a tool for capacity building, planning and monitoring of Area Community Health Associations (ASACOs) and community indicators. 64 5. It is recommended that the district continue to undertake the bi-annual LQAS surveys for monitoring of key indicators, the identification of health problems by catchment area, and need for operations research. 6. Based on the results of the health facility assessment, it is recommended that the district identify strategies to ensure the regular supervision of health facilities. 7. It is important for the Ministry of Social Development to ensure quality continuing education and training of ASACOs, especially in view of the changing membership every three years. 8. Health facilities (CSComs) need to maintain the follow up and support of village health committees (CSVs) introduced by the project, otherwise the sustainability of these entities is questionable. 9. To sustain community mobilization and education activities as well as community health worker (Relais) motivation, ASACOs need to support the continuation of the quarterly coordination meetings in each health catchment areas. 10. It is recommended that communes (Mayor’s office) support communities in the implementation of outreach EPI and Antenatal care services (‘Strategie Avance’). 11. The CSP local partner NGOs should capitalize on the experience attained by using the community health project approach when implementing new health projects. 12. It is advisable for ASACOs to try and contribute to and maintain the motivation of the health service provider because as per the CSP experience, areas with dynamic health providers appear to have the best health indicators. 13. In health systems where communities support health facility income through the purchase of health cards, it is recommended that the CHWs or ‘Relais’ strategy be used to increase health card sales. By giving Relais health cards to sell door to door, it increases access, availability and opportunity. This approach also has the added benefit of motivating the Relais because he/she can earn a small sum off of the sale of each health card. Potential for Sustainability Please refer to previous sections How PVO headquarters intends to use the best practices and lessons learned and communicate this information to the broader development community. The final evaluation results of the Mali CSP are very encouraging to Plan given the difficulties encountered early on after the project start-up with the inability of a key partner, PDRIK, to fund the construction of new CSComs as scheduled. The community organizational structure of ASACOs, CSVs and Relais has enabled the effective delivery of health to households within the project area. This structure has been an effective vehicle for delivering health products like free ITNs made available through RBM and UNICEF’s ACSD program. Regular monitoring of project objectives including the application of LQAS, followed by periodic stakeholder’s discussions on the implication of this data on program implementation at several levels presented a powerful approach to focus on problem areas and to initiate corrective action in a timely fashion. 65 Plan intends to utilize lessons learned to strengthen her country programs health worldwide. Presently plan is in the process of commissioning a global thematic evaluation for child survival. This Mali FE will contribute significantly to this thematic evaluation by highlighting those variables that influence relevance, impact, effectiveness, efficiency and sustainability of child survival initiatives. Potential for scale-up and expansion of the program. Please refer to previous comments under intervention sections. 66 F. Results Highlight Plan Mali attempted to calculate the number of lives saved between 2001 and 2006 (Lives Saved Analysis) in the Kita project area by use of the Bellagio calculator. The Bellagio Group on Child Survival represents many technical agencies. It published the 2003 Child Survival and 2005 Neonatal Survival articles in the Lancet. The Bellagio group estimated the percentage of deaths attributable to the six most common causes of child mortality (diarrhea, pneumonia, measles, malaria, HIV/AIDS and neonatal causes) in 42 high mortality countries including Mali. It reviewed the literature for interventions supported by the evidence as effective against these six main causes and estimated the effect size of these interventions from the literature. The calculator estimates the number of deaths attributable to the six main causes and accounts for both the effects of malnutrition on mortality and the effect of interventions on multiple conditions (e.g. Vitamin A reduces measles, diarrhea and malaria deaths). It also avoids double counts. Lives saved for each intervention is given by the product of baseline number of deaths for that cause, intervention effectiveness and change of coverage. Given the dire baseline situation in Kita, the relative contribution of malaria to child mortality in the Kayes Region of Mali and the widespread availability and use of ITNs through the RBM, UNICEF’s ACSD and Kita CSP programs, the results of lives saved are very impressive. The project was able to save 2879 lives in the course of its 5 years which approximates to under $1000 per life saved. In the last year of the project alone it is estimated that it saved 514 lives from malaria, 304 lives from pneumonia, 205 lives from diarrhea, 93 lives from neonatal causes, 34 lives from measles and zero from HIV/AIDS for a total of 1151 lives saved. This makes the Kita CSP a highly efficient program worth emulation in the region. 67 I. ATTACHMENTS: A. Evaluation Team Members and their titles Marguerite Joseph, External Consultant/Team Leader Dr. Simaga Ismaile, MOH District management team representative Mr. Kalifa Diarra, Social Development Services Dr. Aliou Kayo, Groupe Pivot representative (Umbrella organization for local health NGOs) Dr. Samakou Keita, Private clinician/public health specialist, Kita Dr. Souleymane Bagayoko, National Health Advisor/Plan Mali Dr. Sita Sidibe, Project Manager Mamadou Seck, HIS Officer Elie Coulibaly, M&E Officer B. Evaluation Assessment methodology The Plan International Health advisor and project staff were able to mobilize a few people to join the evaluation team (as per the list of participants below). Thus with a total of nine evaluation team members (5 outsiders and 4 Plan staff), we were able to pull together four teams of two persons (each team consisting of one outsider and one staff person); in addition to the team leader as an observer of field data collection. Note that the implementing NGO Animators and Coordinators who are based in the field and have a close relationship to the communities, did not make up part of the field data collection teams for the final evaluation activity. The evaluation team was limited in the choice of locations to visit due to inaccessibility of a number of locations because of the rainy season, and inaccessibility to others due to the great distances from Kita town. Among those sites that were accessible for this activity, four locations which had not been part of the final KPC survey sample were chosen. Each of the four team was able to go to a designated location (two locations from Phase I, and two locations from Phase II of the project), for the period of two days to conduct FGDs and key informant interviews with project participants. The outsider (non-Plan International staff person) conducted all of the interviews, and the Plan staff person took notes. Project sites: First Phase: Fladougou-Marena and Makamdiabougou; Last Phase: Sagabari and Sirakoro. FGDs: 16 total (4 community women; 4 community men; 4 VHC/Relais; 4 ASACOs) Key Informant Interviews: 11 total (4 village chiefs and counsel; 3 Mayors; 4 health facility workers-CSCom) Teams for data collection: 4 teams of 2 (note taker and facilitator) MOH, NGO Partner and Staff interviews: Conducted separately by Evaluation Team Leader. 68 C. List of persons interviewed and contacted Plan Staff Dr. Souleymane Bagayoko, National Health Advisor/Plan Mali (Bamako based) Cheik Magasouba, Kita Program Unit Manager CSP Project Staff Dr. Sita Sidibe, Project Manager Mamadou Seck, HIS Officer Elie Coulibaly, M&E Officer Implementing NGO Partner, ARAFD/C Mariam Ture, President Ibrahim Diakite, Kita Project Coordinator Yacouba Sacko, Animator Jean Sidibe, Animator Badara Aliou Bare, Animator Mady Dembele, Animator Moussa Kone, Animator Mamadou Sidibe, Animator Implementing NGO Partner, AMACO Allaye Yalcoue, Project Supervisor (Bamako) Diougodie Dolo, Kita Project Coordinator Coulibaly Boubacar, Animator Abdoulaye Coulibaly, Animator Alhassane Bine Guire, Animator Mamadou Dambou Kante, Animator Malick Yaleouye, Animator MOH partner Dr. Ismaile Simaga, Assistant Chief Medical Officer (Kita CSP point person for District Health Management Team) D. Project Data Sheet form – updated version