Child Survival 20 – Mali Scaling – Up Community – Based Services In the Sikasso Region of Mali Cooperative Agreement No.: GHA-A-00-0003-00 Midterm Evaluation Report 30 September 2004-30 September 2009 Submitted by: Save the Children Federation, Inc. 54 Wilton Road Westport, CT 06880 Telephone: 203/221-4000 Fax: 203/221-4056 Contact Person: Eric Swedberg, MPH, Child Health and Nutrition Advisor Prepared by: Marcie Rubardt, Consultant Submitted to: USAID/GH/HIDN/NUT/CSHGP November 30, 2007 This report is made possible by the generous support of the American people through the United States Agency for International Development (USAID). The contents are the responsibility of Save the Children and do not necessarily reflect the views of USAID or the United States Government. Acknowledgements I would like to acknowledge the efforts of the primary evaluation team in reaching all five districts and making this evaluation work. In addition, we certainly appreciated the support and enthusiasm of the District Advisors in each district. I would also like to thank Dr. Boubacar Sidibe and Dr. Male Aissata Diarra for their help in making sure all the logistics came together. Finally, I would like to thank the government counterparts who made the effort to join us in the synthesis workshop and who contributed in good faith that working together would make this project stronger. CS-20 Sikasso, Mali, Midterm Evaluation, November 2007 Save the Children 2 List of Acronyms ACT Artemisinin Combination Therapy ARI Acute Respiratory Infection ASACO Management committees for health centers in Mali’s system ATN Bilateral project for national level health programming in Mali BCC Behavior Change Strategy CPM Chefs de Postes Médicaux/Health Center Heads CPR Contraceptive Prevalence Rate CS-20 “Scaling-Up Community-Based Services in the Sikasso Region of Mali” project funded in large part through the 20th cycle of the PVO CSH Grants Program, which began in September 2004. CYP Couple Years of Protection DHO District Health Officer DIP Detailed Implementation Plan DK Drug Kit FP Family Planning IPT Intermittent Presumptive Treatment (for malaria in pregnant women) ITN Insecticide Treated Net M&E Monitoring and Evaluation MOH Ministry of Health ORS Oral Rehydration Solution PDSEC District Development Plan PVO Private Voluntary Organization Relais Community Health Volunteers in the Mali system – Drug Kit Managers for the project SC Save the Children Federation, Inc. SP Sulfadoxine-Pyrimethamine USAID United States Agency for International Development CS-20 Sikasso, Mali, Midterm Evaluation, November 2007 Save the Children 3 Table of Contents Acronyms………………………………………………………………………………... 3 A. Executive Summary…………………………………………………………………... 5 B. Assessment of Progress Towards Objectives…………………………………………. 7 1. Technical Approach…………………………………………………………………. 7 a. Overview…………………………………………………………………………… 7 b. Progress Report by Intervention……………………………………………………. 9 2. Cross-Cutting Approaches…………………………………………………………… 16 a. Community Mobilization-Community Management of Drug Kits……………….… 17 b. BCC Strategy………………………………………………………………………. 19 c. Partnership and Capacity Building………………………………………….……… 21 d. Sustainability Strategy……………………………………………………………… 24 C. Project Management…………………….……………………………………………. 25 1. Planning…………………………….……………………………………………….. 25 2. Staff Training……………………….……………………………………………….. 25 3. Supervision of Project Staff………………………………………………………….. 26 4. Human Resource Management………………………………………………………. 26 5. Financial and Logistics Management…………………………………………………. 26 6. Information System Management……………………………………………………. 27 7. Technical Support……………………………………………………………………. 28 8. Mission Collaboration…………………………..…………………………………… 28 D. Other Issues Identified by the Team ………………………………………………… 28 E. Conclusions and Recommendations………………………………………………….. 28 F. Results Highlights……………………………………………………………………... 29 ANNEXES ANNEX 1 List of Project Objectives with Evaluator Comments ANNEX 2 Evaluation Team Members ANNEX 3 Evaluation Methodology ANNEX 4 People Contacted ANNEX 5 Compiled List of Recommendations ANNEX 6 Target Populations by District-2007 Estimates ANNEX 7 Training Summary ANNEX 8 Annual Project Plan CS-20 Sikasso, Mali, Midterm Evaluation, November 2007 Save the Children 4 A. Executive Summary The Child Survival Project, “Scaling-Up Community-Based Services in the Sikasso Region of Mali” (CS-20, Sikasso) reinforces community level services for malaria, diarrhea, family planning and pneumonia in five districts in the Sikasso Region of Mali. It was designed as an “Expanded Impact Project”, lasting five years, (September 30, 2004-September 30, 2009)with $2.5 million of USAID funding matched by $833,330 from Save the Children. It is targeting a total population of 988,265. As an Expanded Impact Project, this project aims to impact children’s health on a large scale and in a sustainable way. By extending the availability of health education and basic treatment to the community level through the implementation of community-based Drug Kits, the expected health impact will be achieved by decreasing and treating malaria, diarrhea, and acute respiratory illness (ARI) and increasing the use of family planning. By working through the Ministry of Health (MOH) as the primary implementing partner at the regional, district, and health center levels, the project aims to maximize both scale and sustainability. A detailed list of the project objectives is included in Annex 1. The most significant accomplishment of this project is the realization of community activities in five districts with a total population of nearly one million. CS-20 has significantly emphasized capacity building through training, supervision, establishment of monthly meetings, and advising of its MOH and community partners at all levels. The partners are the primary implementers and their capacity is the primary sustainability strategy. To date, the project has established a total 478 community Drug Kits, covering all of the villages in the five districts that meet the criteria. These are managed by a total of 956 community health workers (Relais) and a total of 1,434 members of Oversight Committees. A health team has also been established in each village to extend community involvement. At the time of this evaluation, all the essential activities for establishing these structures had been completed (training, supervision, and development of management and information systems) such that the remainder of the project will be focusing on reinforcement and sustainability, as well as on achieving the impact these structures have the potential to yield. Highlights from the project accomplishments to date include: • 60,800 children treated for malaria with more than 33,336 of those being treated at the community level through the Drug Kits. • An estimated coverage of 20% of diarrhea cases during the past year (based on an estimated number of cases per child per year) treated with oral rehydration solution (ORS), with the great majority of these treated at the community level through the Drug Kits. • Mosquito nets distributed to cover over 30% of estimated pregnant women, not including recent facilitated distribution of an additional 30,600 nets which were targeting both women and children. • Contribution of 631 Couple Years of Protection (CYP) by the Drug Kits in one year (June, 2005 – June, 2006), despite the stated preference for Depo Provera by most women and referrals to the health centers. This represents 20% of all CYP distributed by government systems. CS-20 Sikasso, Mali, Midterm Evaluation, November 2007 Save the Children 5 The project has not yet achieved as much progress as was expected related to child health. This is because the Drug Kits have been functioning for just over one year, and reinforcement of the effectiveness of the community activities still needs to be completed. This project has been flexible in its response to constraints that have presented themselves over the course of implementation: • Recent retirement of chloroquine from the national treatment protocol for malaria has meant that the core activity for Drug Kits is no longer possible. It is hoped that recent project operations research results will convince the national malaria program to allow introduction of Artemisinin Combination Therapy (ACT) through the Drug Kits at the community level. • The national policy for the distribution of free mosquito nets for pregnant women and children who completed their immunizations meant that the original strategy of distributing nets through the cotton associations, which was meant to provide a mechanism for credit, is no longer necessary. Their involvement was dropped. • The MOH was concerned that inclusion of iron and sulfadoxine-pyrimethamine (SP) in the Drug Kits would remove an incentive for women to go to prenatal care at the health centers, so these products are not included in the Drug Kits. • ARI messages tend to get lost due to the lack of a concrete product/treatment associated with them in the Drug Kits and the perception that these messages are less important. • Women are reporting their husbands have significantly less awareness of, or interest in family planning than they do. Because of the emphasis on scale, partnership, and placing the MOH first in implementation at all levels, prospects for significant sustainability of the community activities are high. While described in more detail in the report, the main recommendations focus on this sustainability. Annex 5 includes a compiled list of recommendations, some of which are mentioned below. 1. Facilitate ongoing discussions at each level among the different entities associated with supporting the community activities to clarify roles and responsibilities, identify ways to increase links between them, and to develop a sustainability plan. 2. Specifically focus on mobilizing sources for long term motivation of the Relais, including reimbursement for costs associated with their activities, and reinforcement of the monthly meetings as a strategy for their supervision and training. 3. Specific technical recommendations include making every effort to facilitate the introduction of ACT and zinc into the Drug Kits, providing timers to the Relais as a way to raise the profile of ARI, and developing a male motivation strategy for family planning. CS-20 Sikasso, Mali, Midterm Evaluation, November 2007 Save the Children 6 B. Assessment of Progress Made Towards Achievement of Project Objectives Overall Project Accomplishments While ultimately this project is about health impact on children under five, the adaptation of the proven community-level strategies in Mali of Drug Kits and BCC to implementation on a large scale and with MOH as the primary implementer is the innovation. The project also hopes to build on results from operations research in implementing both Artemisinin Combination Therapy and zinc treatment for diarrhea at the community level to incorporate these drugs into the community Drug Kits. As of the midterm evaluation, the project had 478 functioning Drug Kits, managed by 956 Relais and 1434 Surveillance Committee members, serving 79 different health center areas. The majority of these Drug Kits are managed effectively and are increasing their capital. The project trained and was facilitating the function of independent MOH structures at all levels in order to support and maintain the community level activities. The establishment of monthly Relais meetings at their respective health centers is a key intervention in developing this support. Community Team with their Drug Kit 1. Technical Approach a. Overview CS-20 is implemented in five districts in the Sikasso Region of Mali. The total estimated target population is 988,265, with an estimated 232,242 children under five and an estimated 232,242 women of childbearing age. This project covers interventions in malaria (35% level of effort), CS-20 Sikasso, Mali, Midterm Evaluation, November 2007 Save the Children 7 diarrhea case management (20% level of effort), Pneumonia case management (20% level of effort) and family planning (25% level of effort). SC zones in Sikasso Region While a complete table of specific objectives with recommendations for adjustment is included in Annex 1, key results objectives include: • Increased use of impregnated mosquito nets by pregnant women and children under five; • Improved treatment for malaria and diarrhea through community Drug Kits and referral for treatment of ARI; • Increased use of family planning; and • Increased Intermittent Preventive Treatment (IPT) with SP as part of prenatal care. Intermediate objectives include: • Increased access to services through installation of community Drug Kits that do not experience stock-outs; • Improved quality of services as indicated by FP client satisfaction and appropriate referrals; • Improved demand through increased mothers’ knowledge of danger signs and the importance of sleeping under mosquito nets; and • Improved capacity of districts and communities to manage Drug Kits and other community health activities. The primary project strategy is the establishment of village Drug Kits as a means for increasing access to proven treatments (malaria, ORS, contraceptives), referral (for ARI, and severe malaria and diarrhea), and health education. It is working in a total of 79 health center areas, with 231 old and 247 new Drug Kits revitalized and initiated. These are in villages which cover approximately 47% of CS-20 Sikasso, Mali, Midterm Evaluation, November 2007 Save the Children 8 the population and are in addition to the population covered in the villages where the health centers themselves are located. See table in Annex 6. All of the villages in the five districts which meet the criteria of being at least five kilometers from a health center and with a population of at least 500 people, now have Drug Kits. The primary strategy for achieving implementation at scale is partnership with the MOH. With one to three project staff functioning in an advisory role in each district, it falls on the MOH to be the primary implementers: assuring the training, provision, and supervision of the community activities. In their advisory role, project staff serve as a catalyst to keep things moving at the district and health center levels, while also supporting independent capacity building at the community level to prepare them for functioning with the more limited supervision the district will be able to provide. b. Progress Report by Intervention Malaria Objectives and Targets: 1. 30% of children <5 slept under an Insecticide Treated Net (ITN) the previous night; 2. 50% of pregnant women slept under an ITN the previous night; 3. 60% of children <5 with fever in the last two weeks treated through Drug Kits or health centers; and 4. 70% of pregnant women will receive intermittent preventive treatment with Fanisdar. The main activities for addressing malaria include: • The establishment of community Drug Kits for early treatment of simple malaria and referral of complicated malaria after providing the first dose of treatment. • Operations research documenting the implementation of ACT treatment through Drug Kits at the community level. • Facilitation of the availability of mosquito nets at the health center and community levels, consistent with the government’s recent policy of making mosquito nets available to pregnant women in prenatal care and children completing their immunizations. The project facilitated transport of 30,600 mosquito nets during the last quarter, and is working with two districts to pilot distribution systems through the Relais at the community level. • BCC activities through group education, counseling associated with treatment, and follow￾up home visits. Results from the project monitoring system from July 2006 – June 2007 indicate that significant numbers of children were treated, particularly through the Drug Kits for malaria. Using population estimates for pregnant women and children under five, it also looks like coverage for mosquito nets per population may be improving, although these data do not reflect utilization. Qualitatively, mothers indicated a change in their attitude towards recognizing the importance of sleeping under mosquito nets for themselves and their children. They emphatically expressed appreciation for the availability of malaria treatment at the community level, and are concerned that it is currently not available (see below challenges). CS-20 Sikasso, Mali, Midterm Evaluation, November 2007 Save the Children 9 Table 1. Number of Children < 5 Completely Treated for Malaria Drug Kit Treatments Health Center Treatments Total Population under 5 Bougouni 8980 6874 15854 74021 Kolondieba 4677 1834 6511 34529 Yanfolila 2828 1149 3977 25214 Selengue 1722 1958 3680 15832 Yorosso 15129 15649 30778 36324 Total 33336 27464 60800 185921 These numbers were not converted to percentages because there is significant variation in the number of estimated cases per year. However, it is particularly significant to note that more than half of all treatment occurred from the Drug Kits. Referred mother getting malaria treatment for her son at the health center. CS-20 Sikasso, Mali, Midterm Evaluation, November 2007 Save the Children 10 Figure 1. Proportion of ITNs Distributed-Pregnant Women 32.70 31.75 27.18 28.06 22.89 0.00 5.00 10.00 15.00 20.00 25.00 30.00 35.00 Proportion of ITNs distributed relative to estimated number of pregnant women over 18 mois BOUGOUNI KOLONDIEBA YANFOLILA SELINGUE YOROSSO Figure 2. Proportion of ITNs Distributed-Children 12.03 11.85 10.06 10.41 8.45 10.89 0.00 2.00 4.00 6.00 8.00 10.00 12.00 14.00 % BOUGOUNI KOLONDIEBA YANFOLILA SELINGUE YOROSSO Total Districts Proportion of Mosquito Nets Distributed Relative to the Number of Children <5 BOUGOUNI KOLONDIEBA YANFOLILA SELINGUE YOROSSO Total In the above graphs, the numerator is the number of mosquito nets reported by the heath centers as distributed, while the denominator is the estimated number of pregnant women and children under five. These numbers do NOT reflect the 30,600 nets which were distributed in the last quarter with project facilitation. In addition, a campaign for net distribution is planned for December 2007 which should help with coverage. Meanwhile, although there are allegedly enough nets to meet the projected demand associated with the new policy at the national level, they have not reached the regional level and below, such that availability is currently the primary challenge. Low coverage of SP for pregnant women: Monitoring data seem to indicate that there is a problem with women not receiving SP during their prenatal consultations. Unfortunately, this information was not available prior to the qualitative work such that the evaluation did not follow up on reasons for this. CS-20 Sikasso, Mali, Midterm Evaluation, November 2007 Save the Children 11 There have been two changes in policy which have significantly impacted the project’s strategies: 1) Withdrawal of Cholorquine from the national essential drug list: While ACT is currently available in the health centers for malaria treatment, chloroquine was withdrawn from the Drug Kits in June 2007 and has not yet been replaced with ACT. The project is currently preparing to present its operations research results showing that ACT CAN be effectively and safely provided by community health volunteers and it is hoping to at least be able to include ACT in the Drug Kits that are part of the project. Meanwhile, people are unhappy with the loss of malaria treatment in their communities and they report bypassing the Drug Kits because treatment is no longer available. The credibility of the Relais is reduced and both utilization and receipts have decreased for the Drug Kits. There is also some residual confusion over whether mothers are waiting two days after receiving paracetamol to seek treatment, confusing it with the message that was associated with chloroquine treatment. If there is a delay or problem with providing ACT at the community level, both the potential impact of the project on mortality through early treatment of malaria, and the potential for the sustainability of the Drug Kits will be threatened. 2) Free distribution of mosquito nets for pregnant women and fully vaccinated children: As part of the Detailed Implementation Plan (DIP), the project developed the strategy of distributing ITNs through cotton associations in order to facilitate credit for people who might otherwise not afford them. However, early in the project the MOH decided to make ITNs free for pregnant women through prenatal care, and for children under five who complete their immunizations – thus removing the financial barrier for the project’s target population. As a result, the project dropped the partnership with the cotton associations and decided instead to focus on the MOH distribution systems. In two districts, the project has piloted ITN distribution through the Relais. Benefits for Relais distribution include added credibility, the ability to follow up on actual utilization, and assurance of ITN availability associated with outreach strategy. While it means an additional step for women who get prenatal care at the health center to then return to the village with a “prescription” for their net, and the Relais are then required to complete the ITN tracking documentation, it still seems to be beneficial. Finally, with the policy focus on immunization completion, children under one become the primary target, which may negatively affect the specific project objective. Recommendations: • Facilitate the essential steps for introduction of ACT through the Drug Kits as quickly as possible. • Building on the experience with assuring supply and on documentation developed in the two districts where ITNs are available through the Relais, encourage the other districts to consider distributing ITNs at the community level. Continue to facilitate the supply of ITNs at the district level when necessary. • Reinforce promotion of SP as part of prenatal care with both prenatal care providers and pregnant women. CS-20 Sikasso, Mali, Midterm Evaluation, November 2007 Save the Children 12 Diarrhea Objectives and Targets: • 30% of children under two years old with watery diarrhea in the last two weeks were treated with ORS, an appropriate home-based solution, and/or increased fluids. Key activities: • Provision of ORS through the community Drug Kits; • Health education and home visits promoting continued feeding and fluids; • Operations research piloting distribution of zinc at the community level as part of diarrhea treatment; and • Supplemental funding from American Idol has been obtained to facilitate the introduction of zinc at the community level. Figure 3. Percentage of Estimated Diarrhea Cases Treated with ORS 0.00 5.00 10.00 15.00 20.00 25.00 30.00 35.00 40.00 BOUGOUNI KOLONDIEBA YANFOLILA SELINGUE YOROSSO Total % Estimated Diarrhea Cases Treated with ORS % treated drug kits Drug Kits AND Health Center In this case, the denominator uses an estimated number of 2.4 diarrhea cases per child per year while the numerator is the number of ORS packets distributed. The most significant observation is that almost ALL of the ORS is distributed through the Drug Kits, indicating the mothers do not go to the health center for diarrhea treatment. Recommendation • Introduce zinc in the Drug Kits. CS-20 Sikasso, Mali, Midterm Evaluation, November 2007 Save the Children 13 Acute Respiratory Illness Objective and Targets: • 60% of children <5 with difficult or rapid breathing who visited Drug Kits are appropriately referred to a health center. Key Activities: • The only activity for ARI is health education and counseling, with referral when a child presents with difficult breathing. The monitoring system does not track the number of children presenting at the Drug Kits with difficult breathing, so it is not possible to track the number who are appropriately referred. However, the qualitative assessment identified significant weakness in the awareness of danger signs for ARI or understanding of the importance of referral. People indicated that it is difficult to focus on ARI health education and messages because there is no associated “product” in the Drug Kits so people do not see it as important. Recommendation: • Reconsider the ARI strategy in order to raise its profile in the project. The inclusion of timers as part of the Drug Kits would allow the Relais to have a concrete action with which to associate ARI messages, and would provide an incentive for mothers to visit the Relais for a child with pneumonia to determine their need for referral, despite the lack of medical treatment at the community level. Family Planning Objectives and Targets: • 8% of all women of reproductive age using a family planning method; and • 50% of FP clients report they are satisfied with the services they received. Key Activities: • Community based distribution of Lo-Femenal, Ovrette, and condoms through the Drug Kits, with referral to the health centers for Depo-Provera or to the district for other methods. • Health education and home visits promoting family planning – largely targeting women. With family planning integrated as one part of a four-intervention project with a primary strategy of community access and distribution, there has not been any emphasis on improving quality of family planning services at the health center level. Similarly, there have been some problems with availability at the district and Regional levels but the project has not been involved with the supply chain except as part of the overall re-supply system for the Drug Kits. The project has not intervened with method mix. Depo-provera, which is largely available through the health centers, is still the most popular method. The project is in full compliance with both the Mexico City Policy since abortion is not discussed at all, and with the Tiahrt Amendment since all family planning use is completely voluntary and all available methods are discussed. The graph below indicates the number of Couple Years of Protection (CYPs) by method distributed in each district. When the CYP is converted to Contraceptive Prevalence Rate (based upon the CS-20 Sikasso, Mali, Midterm Evaluation, November 2007 Save the Children 14 estimated women of reproductive age population) the project has a CPR of 5.6 for the five districts. This, however, is a significant underestimate because there is only 72% of the data from the health centers and there are other private providers that are not captured in this data. Figure 4. Couple Years of Protection 0.00 100.00 200.00 300.00 400.00 500.00 600.00 CAP BOUGOUNI KOLONDIEBA YANFOLILA SELINGUE YOROSSO TOTAL Dist r i ct s Couples Annees Protections - CP (07/2006 - 06/2007) Pillule Condoms The table below highlights that the Drug Kits are actually contributing 20% of the total CYP that are distributed through government systems. Table 2. Couple Years of Protection (July 2006 – June 2007) Drug Kit CYP % of total CYP by DK Health Center CYP Total CYP Bougouni 181 15 989.3 1170 Kolondieba 122 25 365.7 488 Yanfolila 77 32 162 239 Selengue 103 24 324.8 428 Yorosso 148 19 626.5 774 Total 631 20 2468.3 3099 The biggest challenge with the family planning strategy is that it has not particularly targeted men. Women seem to be better informed and more interested in family planning than their husbands, and they express frustration that their husbands refuse to allow their wives to use family planning. The qualitative baseline data indicated that men are concerned about their wives using family planning because it might encourage promiscuity, feel it is against the religion, and feel it is not natural. The project is moving towards introduction of the Standard Days Method, feeling it might be a natural method that would involve men in family planning. CS-20 Sikasso, Mali, Midterm Evaluation, November 2007 Save the Children 15 Generally, Ovrette is not being used in the Drug Kits and so is expiring. The Drug Kits will need to dispose of the expired Ovrette and should consider keeping only a small amount in the future. Recommendation: • Taking advantage of the men already involved with the project (Relais, Oversight Committee and health team) develop a strategy for targeting husbands of women who are interested in FP through a man to man (pairs-educateurs) approach. • Involve religious leaders in FP promotion based on the experience from Ségou. 2. Cross-Cutting Approaches General Findings The emphasis on increased access at scale through MOH partnership provides the framework for this high impact project. By placing Drug Kits in all villages meeting the criteria, and by linking Drug Kits with health areas, the project is effectively extending access to service through the five districts. By working through the MOH partners and structures, project activities are fully integrated and therefore seen as an extension of MOH activities and goals. All activities have also been undertaken while keeping in mind the limitations of the MOH in providing long term support for community activities. Cross-cutting observations regarding the implementation process include: • In all five districts, the MOH is the primary implementer with project staff clear about their function as advisors. The close coordination among project staff has allowed for consistency in implementation across districts, while the differing strengths and weaknesses in the Advisors and MOH partners has led to minor variations in implementation. However, despite the varying approaches and strengths, the conclusions for this project are quite uniform. The Drug Kits are clearly highly appreciated by all concerned. Mothers appreciate the increased proximity of health services, the decreased cost associated with going to the health center, and the assured quality of the drugs relative to those they might buy in the market. Implementation objectives: • 95% of 244 new Drug Kits planned to be created are installed; • 80% of Drug Kits had no stock-out of ORS in the last three months; • 50% of District Health Officers (DHOs) complete two integrated supervisory visits of health centers in the last year; • 80% of DHOs will conduct monthly meetings to monitor health center activities; • 70% of Oversight Committees will be supervised at least once/month by the health center; • 60% of management committees participating in the monthly meetings of the Drug Kit managers at the health center; and • 70% of the membership of Oversight Committees are women. CS-20 Sikasso, Mali, Midterm Evaluation, November 2007 Save the Children 16 The project is making good progress in achieving these objectives. More than 244 new Drug Kits have been established, reliable supplies of ORS are being maintained, and the MOH at both the district and health center levels is carrying out supervision activities. While some of this supervision has been subsidized by the project, particularly at the health center level, more sustainable alternatives, such as integration of supervision with outreach activities, are being explored. The progress towards establishing monthly meetings at the heath centers for the Relais, in a way that can be sustained, is one such strategy. At the moment, a sustainable way to support Relais participation in these meetings is still being considered, and the participation of the Oversight Committees, most of whom are men, has not been emphasized. a. Community Mobilization - Community Management of Drug Kits This intervention is the core successful intervention which this project takes to scale for extended impact. While a lot of the impact remains to be reaped, the intermediate steps of training and implementation have essentially been completed. Community Support Structures Activities: • Two Relais for each Drug Kit (one man and one woman) have been trained, participate in monthly meetings at the health center, manage the Drug Kits, and provide health education and counseling. For each Drug Kit, an Oversight Committee of three people has been established. While one of the objectives suggests that the majority of the members should be women, communities selected a great majority of men for this responsibility. In order to extend community participation and responsibility beyond the five people immediately designated, the project also facilitated the formation of a health team in each community. This includes the Village Headman, other elders, traditional birth attendants, the President of the Women’s Association, and the President of the Youth Association. The MOH decided not to include iron folate and SP in the Drug Kits because it might provide an excuse for women not to come to prenatal care. The project may want to revisit this in the face of the apparently low SP coverage. Management of Drug Kits Activities: • Establishment and use of an information system for managing drug and cash flow, tracking community activities, and assuring referral completion and follow up. The information collected can be easily integrated with the health center statistics to enhance the MOH data collection, and ensure that the control systems are consistent with MOH norms. Drugs are purchased from the health centers, but at the same price as the health center gets them from the district pharmacy. The Drug Kits then apply the same mark up as the health centers such that the price for drugs is the same whether people get them from the Drug Kit or the health center. By giving the community the mark up margin, the Drug Kits are covered for inflation and waste. At this point, it seems most of the Drug Kits are CS-20 Sikasso, Mali, Midterm Evaluation, November 2007 Save the Children 17 gaining capital, although this will likely be in question if malaria treatment continues to be unavailable. The Relais have been trained in signs for referral for severe malaria, diarrhea, and respiratory infection. Patients are given the referral notebook to take to the health center when they go. In most cases, referred patients receive priority treatment, and occasionally are not even charged the consultation fee. Challenges: While the challenges for such an intervention are many, the evaluation focused on some of the most significant: Relais motivation and support: Most of the Relais acknowledged they had agreed to work voluntarily, and most cited reasons for their interest which included appreciation for the trust the community had placed in them, the respect of the elders, helping out the health of their community, and closer links with the health center. This said, they frequently mentioned concerns about the expenses they incur for monthly meetings and restocking the Drug Kits and were concerned about support. Role of the Surveillance Committee and Health Team too narrow: While most of the Oversight Committees acknowledged regularly checking the medicine stock, money, and notebooks; they didn’t seem to understand how their role goes beyond this control. Many of the Drug Kits are building up capital that is not being managed, and findings are not regularly reported to the health team and/or other community members in order to extend the transparency of and involvement with the Drug Kits. The project had originally planned for some of the gains to be distributed to the Relais as an incentive, but this is not generally being done. Finally, almost no women are involved with these committees, although staff acknowledged that the tendency for women to be more stable and to be better money managers would make it an advantage. The health teams also tend to have limited involvement, although some of the Village Headmen themselves participate in planning activities and overseeing Drug Kits. Other members occasionally help with mobilizing community members for health education. Second Relais tends to be underutilized: The second Relais, often a woman, tends to be less involved with managing the Drug Kits or carrying out activities at the community level. While the specific relationship between the two Relais is unique to each village, the second may be able to take more responsibility with some of the record keeping notebooks (such as the births and deaths, BCC/community activities, and supervision) and would likely also be more involved if (s)he were to participate regularly in the monthly meetings. Balancing work load with adequate documentation in the information system: While the project is very aware of this problem and has made a significant effort to reduce and simplify the community level reporting system by recently decreasing the notebooks from nine to six; the work load is still significant - especially for those Relais who may be less literate. However, particularly if the project is going to extend distribution of ACT and free mosquito nets to the Drug Kits, documentation according to MOH norms will be essential. CS-20 Sikasso, Mali, Midterm Evaluation, November 2007 Save the Children 18 Recommendations: • Develop a strategy for motivating the Relais as well as for supporting the expenses associated with their activities. (Note: coordinating these discussions with the introduction of ACT in the Drug Kits will likely facilitate solutions) o Mobilize communities to show their appreciation for the Relais activities. For example: helping the Relais with cultivating their fields, and supporting the expenses associated with the monthly meetings with profits from the Drug Kits, collecting money, and/or providing money through the community associations. o Encourage recognition from the district and health centers through personal appreciation and encouragement, priority treatment, and possibly subsidized care. o Support the participation for both Relais in the monthly meetings. o Consider provision of small “appreciation gifts” such as T-shirts for the Relais on an intermittent basis. o Involve the ASACOS in providing encouragement and additional support for monthly meeting expenses as needed. • Facilitate discussion meetings among the Relais, Oversight Committee and Health Team (possibly as part of evaluation feedback) to review their roles and responsibilities, and to plan for the sustainability of the Drug Kits: o Broaden transparency and community involvement in the Drug Kits through regular feedback to the village chief and leaders from the Oversight Committee on their activities and findings. o Expand the role of the Oversight Committee to include active management of the money generated by the Drug Kits. Ideas include distribution of money as motivation for the Relais, use of the money to support the monthly meetings, and/or to support other health activities in the village. o Reinforce the involvement of the second Relais by delegating responsibility for some of the record books (e.g. births/deaths, health education activities, and village visits), and for strengthening the health education activities. o Plan for the long-term reimbursement of expenses for monthly meetings and medicine re￾supply. o Reconsider ways to increase women’s involvement in the Oversight Committees. b. BCC Strategy Objective and Targets: • 80% of mothers know three or more childhood danger signs of illness; • 60% of mothers know where a Drug Kits is located.; and • 80% of women state that ITNs are very important for children under five. Key Activities: • A qualitative Doer/Non-Doer study was carried out and used as the basis for a workshop using the BEHAVE approach to develop the priority behaviors and messages for the BCC strategy. Partners at the district and regional levels participated in this workshop, although many of the participants have since moved on and the information from the baseline work has not been shared with replacement personnel. CS-20 Sikasso, Mali, Midterm Evaluation, November 2007 Save the Children 19 • Clear and simple “Cartes de Conseil” visual aids were developed with pictures designed to stimulate questions and discussion, and with reminders of the key messages on the back for the Relais. Some of the Relais had the idea of focusing their health education by using only one or two cards and allowing for significant questions and answers, while others were still trying to cover too much information in one session. Contracts with local radio stations, using short messages which are repeated frequently throughout the day, seem to contribute significantly to the coverage of essential messages. Mothers were able to identify the key messages, and Relais indicated the radio gives them both credibility and ideas for what to say during health education. Despite the considerable effort this project put into formalizing and focusing both the messages and the strategies, the implementation of the BCC strategy on the ground tends to be inconsistent, and often “timid” as one District Medical Officer said. While the better Relais are able to focus their activities and messages, and health education plans seem to be discussed between different community players, there still seem to be too many messages, and it is unclear whether the effort is systematically reaching all women with a focus on the essential behaviors. The Radio Team in Sikasso Recommendations • Strengthen family planning promotion–see Family Planning section above. • Reinforce health education activities by: o Focusing on one message (one advisory card) at a time and developing it in a participatory way – whether through group education or individual counseling. o Reinforce the promotion related to weaker objectives such as ARI. CS-20 Sikasso, Mali, Midterm Evaluation, November 2007 Save the Children 20 c. Partnership and Capacity Building This project has worked at the health center, district, and regional levels to assure the involvement of all of the essential players, using the authority of the higher levels to facilitate implementation at the lower levels. Similarly, training was completed using a cascade strategy starting with a training team at the regional level in order to maximize both commitment and consistency. Health Center Level Key Activities: • Supervision of field activities is carried out by the Head of the health center. Including supervision of the community Drug Kits as part of the activities associated with the outreach strategy is increasingly being emphasized since funding for outreach is more reliable, often coming from the ASACO. The limitations with this approach are that it is often someone other than the Head who is doing the outreach, and it is done irregularly. • Monthly Relais Meetings organized by the Head of the health center are the key intervention for assuring ongoing linkages between the health center and their Relais/ Drug Kits. When supervision in the field is limited, these meetings serve as a substitute. They are also used to discuss problems, to re-supply and oversee the Drug Kits, and for collecting utilization data. It appears that these meetings might have had a tendency to get over-focused on the data collection and record keeping elements. A few of these meetings, particularly in the more urban areas where health centers are closer together, are carried out jointly across more than one health center area. While this has advantages for exchanging ideas and reinforcing health center staff, it has added challenges in terms of transport and support since both Relais and health center staff are then expected to travel. In a couple of cases, Relais have also joined together to form their own association, but the evaluation was unable to assess whether this is beneficial or an added level of bureaucracy. • The Head of the health center is the primary link between the health center and the community activities. He was responsible for training the Relais and Oversight Committees, and is responsible for their supervision. An incidental benefit to using these Heads as the trainers for the Relais, was that their case management skills were also strengthened. Other staff in the health centers were not trained, and many have not been particularly involved with the community activities. • Re-supplying the community Drug Kits is part of the health center responsibility. While some districts and health centers are organized to request medicines for the Drug Kits based on predicted needs, most are ordering the drugs from the District Pharmacy when the requests (and money) come in from the communities. The latter system leads to unnecessary delays in refilling supplies and sometimes stock-outs. The ability to order in advance depends on orientation towards a prediction system, as well as on the health center having enough capital to pay before it receives the money from the communities. CS-20 Sikasso, Mali, Midterm Evaluation, November 2007 Save the Children 21 • The role of the ASACO is to manage and support the health center activities, including those at the community level. Many ASACO members mentioned involvement in community activities including providing encouragement, supervising Drug Kits, and helping to resolve problems. However, particularly due to their focus on financial management and revenue generation, many ASACOs are not clearly involved with the community activities. • A protocol for participation by the mayors was developed and signed by the project with each of the mayors. Its purpose was to increase the mayors’ level of involvement with community health activities since, with decentralization, the Commun/local government is responsible for health activities in its area. While some mayors have been involved with the monthly meetings, assisted with mobilizing the Village Headmen, and/or helped resolve problems with Drug Kits, the involvement seems to have been limited and generally has not extended to financial support. There may have been more mayoral involvement in Yorosso and Yanfolila where the Prefet and/or Sous-Prefets were also involved. The participation of their advisors as members of the ASACO, also facilitated the mayors’ participation in health activities. In summary, some of the challenges at the health center level, which have been mentioned in the discussion on activities, include: • Support for the community activities at the health center level tends to be centralized around the Head of the health center and not involve other health center staff; • Integration of supervision with outreach activities as a solution for sustainability is constrained by the irregularity of these activities and the fact that it often is not the Head of the health center who is doing the outreach; • While the data are collected during the monthly meetings, they tend to not be analyzed or considered very carefully and limited feedback to the Relais/communities is provided; and • It is difficult to get consistent involvement and support for community activities from mayors and ASACO members. Recommendations: • Reinforce the effectiveness of the monthly meetings as the central element of supervision and capacity building at the community level. o Work with health center staff to identify discussion priorities, plan the agenda, and to review progress while progressively decreasing active advisor participation in the meetings themselves. o Develop a series of mini-modules on a variety of topics which can be used for providing refresher training as part of monthly meetings. o Assure support for participation of both Relais from each community in the monthly meeting. • Facilitate discussion meeting, (possibly as part of evaluation feedback) among the health center staff, mayors’ office, and ASACOs to review their roles and responsibilities, and to plan for the sustainability of the community activities. o Encourage increased ASACO involvement in community activities by: ƒ Encouraging the village ASACO representative to participate in the health team in villages where there are Drug Kits; and ƒ Encouraging ASACO participation in and support for the monthly Relais meetings. CS-20 Sikasso, Mali, Midterm Evaluation, November 2007 Save the Children 22 o Encourage mayors’ offices to include health activities, including participation in monthly meetings and community supervision, in their annual work plans (PDSEC). o Expand the involvement of health center staff beyond that of the Head by orienting them to the community activities, involving them in the monthly Relais meetings, and encouraging their supervision of Drug Kits during their village outreach activities. • In order to assure timely re-supply at the community level, health centers should make an effort to include the predicted drug needs for the village Drug Kits in their regular requests to the District Pharmacy, rather than waiting until specific requests come in. Use option to move stock between villages, particularly in the case of expiring drugs. District Level At the district level, the main project involvement has been the daily integration of the project District Advisors. They have an office in the District Health Office, and regularly participate in district meetings and activities, particularly as they pertain to the community level. The district has a designated “Point Person” for project activities which has significantly facilitated implementation. All of the project activities are included in the district’s Operational Plan, and the district training team, with financial and technical support from the project, took primary responsibility for training all of the Heads of the health centers. District staff indicate the Advisors are sometimes mistaken for MOH employees. In the case of Yanfolila, the Prefet also facilitated implementation activities through the distribution of a letter requesting the support and involvement of all of the mayors in the community activities. At the district level, the biggest challenges come from the turnover and lack of availability of key personnel. While the District Advisor is focused only on the project, district staff have competing priorities among all the national health programs. In most of the districts, key staff have left their positions with limited to no formal handover of project activities, leaving the project continually trying to catch up with new staff. Both of these challenges, along with shortage of resources, negatively impact the ability of district staff to consistently guide and supervise the community activities. Recommendations: • Facilitate the availability of mosquito nets at district and health center levels (see malaria section); • Develop a handover “package” (or check list) of essential information related to the community activities to be used when staff turn over at the district level; and • Continue to balance the division of responsibilities between project and MOH staff in a way that assures progress while not undermining MOH ownership and responsibility. Regional Level The region has been very supportive of the project’s interventions and strategies, and its role has been to provide the guidance and authority in promoting the project to their districts. They also serve as an intermediary on national policy issues such as the use of ACT and zinc at the community level. Their training team was involved with developing all of the training materials, and they trained the District Training Teams on project financial and technical mangaement. CS-20 Sikasso, Mali, Midterm Evaluation, November 2007 Save the Children 23 Training Training has been an integral part of capacity building at all levels of the regional health system. With each level responsible for training the level beneath it, skills were effectively transferred from the regional level through the districts to the health centers and community Relais. This strategy allowed the project to train a large number of people in a relatively short time, although it meant that some of the training occurred simultaneously, and the District Training Team and Advisors were unable to supervise all of it. In all cases, the money and logistics for the training, as well as the content, were managed by the district and health center teams. This strategy has also allowed the project to already carry out a refresher training for all of the Relais, including preliminary orientation on ACT in preparation for the hoped-for inclusion in the Drug Kits. As with the first training, the district training teams oriented the health center Heads for the refresher training who then took responsibility for training their Relais. As part of the preparation for the refresher training, the project also carried out a special training for any new health center Heads, recognizing that attrition would have occurred. This special training is also planned for each of the remaining two years of the project, although it is unlikely that funding will be available to continue such a practice after the end of the project. The biggest challenge with a cascade training approach, and this project is no exception, is the assurance of training quality at the lowest levels. While the project put a significant emphasis on adult education methodologies and orientation towards participatory health education as part of their training of trainers approach at the district level, the training at the health center level had to include more technical training. This compromised the time available to also orient the health center Heads as trainers. As a result, while the district and regional training teams identified themselves as trainers, the health center Heads were not able to clearly describe how they approached training their Relais. It seems that most of the Relais training was actually done by lecture even though the training materials suggested other approaches. A summary of the training activities is included in Annex 7. Recommendation: • Model and practice participatory adult education methods as part of the health center training, reinforcing their role as trainers in addition to that of transferring information. d. Sustainability Strategy In summary, all the pieces are (almost) in place for the community level activities in this project to have a reasonable chance for sustainability. The emphasis on placing the MOH in the lead for implementation, balanced with project support and facilitation to move things forward has been successful in allowing the district and health centers to develop a sense of involvement with these activities. In addition, all of the strategies and activities have been rigorously “screened” to be sure they are realistic on a large scale and feasible for the eventual assumption of full responsibility by the districts and region. This said, there is still a need to further work with the entities at the health center and community levels to increase their understanding of the needs for sustainability and to involve them in developing ways to meet those needs. This should be part of the activities for the remaining two years as the project develops its phase-out plan. CS-20 Sikasso, Mali, Midterm Evaluation, November 2007 Save the Children 24 Recommendation: • Using the discussion meetings mentioned previously, develop a plan for Save the Children (SC) to gradually reduce its responsibilities as the different institutions at each level identify and address gaps in support for community activities. C. Project Management This project seems to have been extremely well-managed, despite its having three Project Coordinators (Regional Advisors) in three years. This seems to have largely been due to the effective delegation of day to day oversight and management responsibilities to the Training and M&E Coordinators. They were able to assure significant continuity, and also had enough authority to be sure staff in the field got their needs met in terms of financial, logistical and program support. One of the results of this situation was that the overall project management structure was more horizontal than is often the case, and all of the staff felt they had both the latitude and the responsibility to make the project work. This project managed to develop a real sense of teamwork among all of the District Advisors early in the project, which also significantly contributed to the success of this horizontal structure. 1. Planning The MOH partners at the district and regional levels were involved with planning the project initially, using the baseline data that had been collected. Beyond that, specific activities (training, supervision, evaluation, etc.) are planned with whomever is involved. For each activity, the advisors work with their point people to develop the plan, budget, and eventually to implement the activity. It was quite clear when talking to all of the partners that they understood the project objectives and were involved with facilitating their achievement. Project activities have been integrated with the district Operational Plan from the beginning. The Detailed Implementation Plan was translated into French and distributed, but is not referred to. The partners, in collaboration with the project staff, are responsible for collecting project monitoring data, and for integrating community data with their regular health center reporting system. The activity reports and Drug Kit management information are monitored by the District Advisors along with the Head of the health center and the district Point Person. They make an effort to target supervision when problems are identified. The project M&E Officer is responsible for generating the monitoring reports for tracking progress on the specific project indicators. While everything is in place for these reports, he has been involved with other work and has not yet begun systematically generating these reports for feedback. 2. Staff Training In this project, staff capacity is being built through: • Participation as trainees and then trainers in the project training system; • Participation in district, regional and national meetings as members of the district teams; • Experience with guided innovations such as use of the BEHAVE approach to developing behavior change strategies, development of ACT protocols, and operations research for ACT and zinc implementation; and • Supervised latitude in developing their own approaches at the district level. CS-20 Sikasso, Mali, Midterm Evaluation, November 2007 Save the Children 25 In addition, the Project Coordinator(s) participated in a variety of regional meetings and activities associated with other Save the Children health activities. 3. Supervision of Project Staff The horizontal management structure is a key to the effective supervision and positive morale of project staff. District staff appreciate the sense of responsibility and freedom they have to function independently in their relationships with their districts, yet they receive advice and support from the two intermediate level staff, as well as from each other, as needed. The two intermediate staff also clearly have a similar, mutually supportive relationship. The past two Project Coordinators (at least) had positive relationships with the staff, providing leadership and support without depending on authority to get the job done. Monthly meetings are held with the staff, bringing them in from their respective districts to file reports, share accomplishments and challenges, and plan the next steps. The organizational structure and workload are completely appropriate for the technical and managerial needs of the project. 4. Human Resource Management Staffing has remained as planned in the DIP with the exception of the turnover in Project Coordinators. Two of the Project Coordinators left for further studies and one was promoted internally. While these are all actions to be encouraged they make project management challenging. As indicated above, the project structure, with its delegation of responsibility to the intermediate level, has coped extremely well with this turnover, and it does not seem to have significantly impacted implementation at the field level. Meanwhile, the project has moved quickly to find a replacement of the most recent Coordinator. Key personnel policies and job descriptions are in place. Careful agreements were signed at each level with the MOH partners and even the mayors. These agreements outlined the roles and responsibilities of each partner. The agreements with the MOH have been well respected, and those with the mayors have yielded less collaboration than hoped. The morale and cohesion of the whole project team are remarkable. They describe themselves as a “family”, they look after each other, and they share their lives as well as their work - particularly since many of them have families living elsewhere. They suggest that this cohesion began during the early phases of the project with training for and execution of the baseline survey, but it is unclear what, if anything, was specifically done to cultivate it. Save the Children encourages staff to apply for positions in other projects as projects transition. However, there are no specific plans for staff transition at this time, when CS-20 ends. 5. Financial and Logistics Management Internally, this project has maintained an effective balance between financial and logistics control, and facilitation of project activities. With spending authorization delegated to the intermediate staff in the absence of the Project Coordinator, day-to-day activities have continued despite the turnover of Coordinators. The project also managed to put systems in place to allow the MOH to manage the budgets for training and supervision. While it took orienting by project staff to clarify that the money was considered an advance that needed to be accounted for rather than an expense that was CS-20 Sikasso, Mali, Midterm Evaluation, November 2007 Save the Children 26 theirs regardless of how many people actually participated, staff felt the benefit in terms of MOH involvement and ownership for the activities was worth the effort. The overall budget is managed in Bamako by Save the Children’s Financial Manager and the Health Program Manager. The project spending is on track with approximately 53% of the total direct cost field budget and 57% of the headquarters direct cost budget spent by the end of August. Given the plan to introduce zinc and ACT during the coming year, and the need to train and support Relais in their use, this is appropriate. With respect to financial sustainability, the viability of the Drug Kits benefits from the long time government experience with the Bamako Initiative. The standard mark up of 20% is adequate to cover for both inflation and loss. The willingness of the health centers to pass the drugs on to the community level without their normal 20% mark up from the district pharmacy means the drugs are the same price at either the health center or the community, while still giving communities the benefit of mark-up to cover costs. The essential procurement for this project was limited to the BCC materials and the initial drug stock for the Drug Kits. This was effectively managed from the Bamako office. Both zinc and ACT will need to be procured during the coming year. With respect to ACT, the supply for children under five will be available nationally as determined by the new national policy. The project will still need to procure the initial stock for people over five since it will be part of the Drug Kit revolving funds. If needed, funds are available for this and the drugs are available nationally. Zinc is not yet available in the national pharmacy despite the recent policy for using it in routine diarrhea treatment. While the American Idol funding will allow for purchase of the initial stock, the project is working on the policy and supply systems which will assure the ability of the Drug Kits to continue to re￾supply themselves once the initial stock is sold. 6. Information System Management The information system, designed to fully integrate with that of the MOH, collects essential utilization, drug management, and vital statistic information at the community level, and involves both the Relais and the health center staff in its use. In order to avoid expecting community volunteers to fill out excessive reports, the project developed a notebook system where the Relais record their daily transactions. These are then brought to the monthly meetings where the health center staff extract the essential information and add it to their monthly reports. One of the limitations of this system is that while the project staff make sure the community data are collected, some of the health center data are missing. An analysis of monitoring data over the past year indicated that only 72% of health center reports had been turned in. While the data collection system is pretty well in place, there is still work to be done to effectively use the data generated. The data are not analyzed at the point of collection, and neither the project nor the MOH have effectively implemented a feedback system. The project has been reviewing the data to assess change over time, but the data had not been used to assess progress against objectives until this evaluation. The monitoring system provided the data used for the graphs included earlier in this report. CS-20 Sikasso, Mali, Midterm Evaluation, November 2007 Save the Children 27 Finally, the project did some nice qualitative work to inform the behavior change strategy and the capacity building approach at the health center and community levels. Project staff were quite aware of the results from these surveys and referred to them during discussion. However, while the MOH counterparts who participated in project start-up may have been aware of these surveys, the newer MOH staff were not familiar with the studies. Recommendation: • Assure the integration of community data with that of the health center in their monthly report. Adapt an analysis table for community data and orient the health center to its use – both for their own planning and for feedback to the Relais. 7. Technical Support This project has had the technical support it needed to function effectively. Most immediately, this included four support visits from home office. In addition to participating in the DIP workshop and the midterm evaluation, the technical support person facilitated the BEHAVE workshop for development of the BCC strategy, and assist staff with preparations for the midterm evaluation. This project has also benefited significantly from operations research which was separately funded and received significant technical support from Johns Hopkins. It is the synergy between this project and the operations research that leaves this project in a position to now promote (and hopefully to implement) distribution of ACT and zinc at the community level. (See Highlight below) 8. Mission Collaboration The local USAID Mission provided start-up funding for two months in advance of the actual CS-20 in order to facilitate timely start-up given the receipt of two, simultaneous child survival grants. They have continued to work with the project in an advisory role, and have actively supported the move towards ACT distribution at the community level. They are also prepared to help facilitate the supply of zinc by working through ATN, the national level bilateral. The Mission facilitates quarterly meetings of PVOs and contractors implementing health programs, to which Save the Children actively contributes. D. Other Issues Identified by the Team N/A E. Conclusions and Recommendations In conclusion, this project is implementing effective interventions on a large scale, through successful partnerships with the MOH at the health center, district and regional levels. While the full impact of these interventions is yet to be realized, the essential elements are in place, functioning well, and in a position to enhance impact in the remaining two years. As a result, the primary task for the project now is to reinforce the effectiveness and independent function of the structures it has put in place. This said, there are a couple of project objectives which are no longer pertinent due to changes in policy, and a couple of other ones which may be unrealistic. Comments from the evaluator on their relevance and likelihood of achievement is included with the list of objectives in Annex 1. Finally, while the recommendations for the remaining time have been included in the sections where they were discussed, a compiled list of these recommendations is also in Annex 5. CS-20 Sikasso, Mali, Midterm Evaluation, November 2007 Save the Children 28 F. Results Highlight HIGHLIGHT – Synergy Between Drug Kits And Operations Research For Implementation Of Artemisinin Combination Therapy at the Community Level The CS-20 project in Sikasso Region of Mali is increasing community level access to treatment for diarrhea, malaria, and family planning through large-scale implementation of community-managed Drug Kits in five districts, covering a total population of nearly one million. By partnering with the Ministry of Health at the health center, district, and regional levels, it has maximized both the scale and the potential for sustainability of this intervention. It has strengthened government capacity at the health center level to train and support community level activities, and has put management systems in place at the community level to assure honest and effective management of the Drug Kits in a way that will not depend on project support. The potential significance of this intervention is now greatly enhanced by its synergy with operations research on community distribution of Artemisinin Combination Therapy (ACT). With the increasing ineffectiveness of chloroquine-based, and even SP, therapies for malaria, the international community is seeking effective treatment without losing the access and potential for early treatment that has been possible to date. In the case of the CS-20 project, the completion of an operations research study documenting the effective distribution and use of ACT through community-based Drug Kits coincides with the national policy retiring chloroquine from use and the project’s establishment of support structures for community treatment. The project is thus uniquely situated to implement ACT distribution at the community level on a large scale. At the time of the evaluation, the project was demonstrating effectiveness in implementing community-based Drug Kits at scale. 478 Drug Kits, in 79 functioning health center areas were functioning with community management and oversight, and with health center support including re-supply of the drugs and carrying out monthly meetings for the community volunteers in their areas. In the five districts, all communities meeting the project criteria of being more than five kilometers from a health center and a population greater than 500 had Drug Kits. The majority of Drug Kits were increasing their capital, and they were providing almost all of the ORS treatment and more than half of the malaria treatment in the five districts. The region has reviewed and accepted the operations research results, and the national policy allowing for community distribution is pending the immediate presentation and review of the results at the national level. The availability of these results, showing that community volunteers can safely and effectively distribute ACT, should make it possible for the government to accept community distribution and to maintain the improved access for early malaria treatment that the Drug Kits initially provided when chloroquine was the first line treatment. CS-20 Sikasso, Mali, Midterm Evaluation, November 2007 Save the Children 29 ANNEXES ANNEX 1 List of Project Objectives with Evaluator Comments ANNEX 2 Evaluation Team Members ANNEX 3 Evaluation Methodology ANNEX 4 People Contacted ANNEX 5 Compiled List of Recommendations ANNEX 6 Target Populations by District-2007 Estimates ANNEX 7 Completed Training Activities ANNEX 8 Annual Project Plan CS-20 Sikasso, Mali, Midterm Evaluation, November 2007 Save the Children 30 ANNEX 1 – List of Project Objectives with Evaluator Comments Result/ Intermediate Result # End of Program Target/indicator Method Baseline Interv. 1 30% of children <5 slept under an ITN the previous night. R-1: Increased practice KPC 8.4% Malaria of key household behaviors. 2 50% of pregnant women slept under an ITN the previous night. KPC 25.9% Malaria 3 60% of children <5 with fever in the last two weeks used Drug Kits (DKs) or health centers. This will only be realistic if ACT is introduced through the Drug Kits. KPC 25.8% Malaria 4 30% of children under 2 years with watery diarrhea in the last two weeks were treated with ORS, an appropriate home-based solution, and/or increased fluids. Might be interesting to add zinc treatment to the objectives. KPC 17.5% DCM 5 70% of pregnant women will receive malaria intermittent preventive treatment (SP). This is a worthwhile objective, but the project seems to be lagging on this due to both lack of availability at the health center and apparent provider reticence to encourage it. KPC 7.4% Malaria R-2: Increased use of services at health centers and in the community (village Drug Kits). 6 8% of all women use a family planning method. KPC 5.1% FP 7 95% of 244 new Drug Kits planned to be created are installed. MOH Monitoring system ALL 8 50% of villages have associations selling ITNs. Associations are no longer part of the strategy due to free distribution. It seems unlikely that all districts will adopt community-level distribution as a strategy given the need for control systems for free distribution plus the MOH desire to use net distribution as an incentive for increasing health center utilization. MOH Monitoring system Malaria 9 80% of Drug Kits had not stock-out of chloroquine in the last three months. No longer applicable. If ACT is introduced at the community level, the objective could be used for ACT. MOH Monitoring system Malaria 10 80% of Drug Kits had not stock-out of FP methods in last three months. The priority methods are Lo-Femenol and condoms. Demand for Ovrette is low enough that its availability at the community level probably is not a high priority. MOH Monitoring system Malaria 11 80% of Drug Kits had no stock-out of ORS in last three months. MOH Monitoring system FP IR-1: Increased availability of select MCH services in the community. 12 80% of Drug Kits had no stock-out of iron folate in last three months. Not applicable. Only available through prenatal visits. MOH Monitoring system DCM CS-20 Sikasso, Mali, Midterm Evaluation, November 2007 Save the Children 31 Result/ Intermediate Result # End of Program Target/indicator Method Baseline Interv. 13 50% of FP clients report they are satisfied with the services they receive. KPC Not Measured IR-2: Improved DCM quality of select MCH services. 14 60% of children<5 with difficult or rapid breathing who visited Drug Kits are appropriately referred to a health center. KPC 25% Malaria 15 80% of mothers know three or more childhood danger sings of illness. KPC 60.7% ALL 16 60% of mothers know where a Drug Kit is located. KPC 30.5% ALL IR-3: Increase demand of appropriate health services (using knowledge, attitude and access as proxies for demand) 17 80% of women state that ITNs are very important for children <5. KPC 32.0% ALL 18 50% of DHOs complete two integrated supervisory visits of health centers in the last year. MOH 20% ALL 19 80% of DHOs will conduct monthly meetings to monitor the health center activities. MOH 40% ALL 20 Health centers, District Medical Directors, MOH and SC collaborate in writing two articles on DK and ITN promotion approaches. The project needs to decide if this is still a priority. Final eval. None ALL IR-4: Increased capacity of 5 District Health offices and regional MOH to effectively support CS￾20 services and activities. 21 MOH and partners adapt and distribute an Operations Manual for DKs and ITNs. The pieces for this are in place, and it is worth the effort to compile it in order to facilitate replication by others. Final eval. Draft manual ALL 22 70% of Oversight Committees will be supervised at least once/month by the health centers. The focus so far has been on the Relais-the project may want to reconsider whether the objective should remain with the focus on the Oversight Committees. OCAT 20% ALL 23 60% of Management Committees participate in the monthly meetings of the DK Managers at the health centers. OCAT 0% ALL IR-5: Increased capacity of community (village & HC Board) to effectively address health needs of mothers and children. 24 70% of the membership of the Oversight Committees are women. This is an unrealistic objective, even if increased women’s participation is worthwhile as staff indicate. OCAT 37% ALL CS-20 Sikasso, Mali, Midterm Evaluation, November 2007 Save the Children 32 ANNEX 2 – Evaluation Team Members Core Evaluation Team Members 1. Dr. Sidibe Halidou, PNLP, Bamako 2. Dr. Dembele Karim, Point Person for CS-20, Sikasso Regional Health Office 3. Modibo Bamadio, SC M&E Officer, Bamako 4. Soufiana Kaba, CS-20 Training Officer 5. Zana Daou, CS-20 M&E Officer 6. Dr. Male Aissata Diarra, Malaria Operations Research Coordinator 7. Dr. Bogoba Diarra, National Consultant 8. Eric Swedberg, Technical Support, Save the Children Headquarters 9. Marcie Rubardt, Consultant and Team Leader CS-20 Sikasso, Mali, Midterm Evaluation, November 2007 Save the Children 33 ANNEX 3 - Evaluation Methodology Evaluation Methodology This evaluation was a participatory evaluation, involving external consultants, project staff, and MOH partners. The core evaluation team was divided into three sub-teams to facilitate visiting of all five districts within the allocated time. For data collection in each district, the primary evaluation team was joined by the project’s District Advisors for that district. Each sub-team spent one to two days in each district: meeting with the district Health staff, visiting two health centers, and visiting one community/Drug Kit for each of the health centers. An effort was made to select a stronger and weaker health center for each district, and to select a stronger and weaker Drug Kit, in order to offer a range of the project experience to date. One of the three teams only visited one district (Yorosso) and also met with staff at the Regional level. At each level, key informant interviews and focus group discussions were used to gather information. These were supplemented by observations of the Drug Kits and review of the record books at the community level. The evaluation team also reviewed the project documents, including: • The baseline KPC survey; • The baseline qualitative Doer-Non-Doer Analysis; • The Detailed Implementation Plan; and • The three Annual Reports (the Third Year Report was still in draft). Once the data were collected, each team synthesized its observations, entering them electronically into a matrix organized by theme, informant and site. The matrices from all three teams were then merged and sorted by theme in order to organize the data collected across the different districts and informants. Once the data collection was complete, an expanded group of stakeholders came together for two days of synthesis and analysis to develop conclusions and recommendations from the evaluation. Participants included the primary evaluation team, District Advisors, and representatives from the MOH in the five districts, most often the project point person and the District Health Officer. The conclusions and recommendations in this report reflect the results of these discussions. CS-20 Sikasso, Mali, Midterm Evaluation, November 2007 Save the Children 34 ANNEX 4 - People Contacted Sikasso Region 1. Dr. Daouda Konate, Head of Health Programs, Sikasso Region/Point Person for CS-20 2. Dr. Kone Abdullaye, Responsible for Health Programs 3. Sarah Dolo, Midwife/Training Team Yorosso District 1. Dr Ibrahima Diarra, District Health Officer 2. Souleymane Traoré, Project Point Person 3. Yaporo Traoré, District Pharmacy 4. Central Health Center a. Nanpena Village 5. Koury Health Center a. Tandio Village 6. Yaya Dao, Mayor, Koury Bougouni District 1. Dr YATTARA Hamadoun, District Health Officer 2. Karoga KEITA, M&E 3. Oumou COULIBALY, Midwife 4. Koumantou Health Center a. Guérékélé Village 5. Keleya Health Center a. Soulouba Village Kolondieba District 1. Dr DIARRA Alkadri, Assistant District Health Officer 2. TRAORE Amadou, District Pharmacy Manager 3. SANOGO Tiémoko, Environmental Health Officer 4. Moumine Sanoga, Immunizations 5. KONE Bocary, M&E 6. Kebila Health Center a. Bafaga Village 7. Bougoula Health Center a. Zantoumala Village Yanfolila District 1. Dr. Alou Coulibaly, Assistant District Health Officer, CS-20 Point Person 2. Yorobougoula Health Center a. Bereko Village 3. Badogo Health Center a. Karatu Village Selengue District 1. Tagan Health Center a. Fingoana Village 2. Binko Health Center CS-20 Sikasso, Mali, Midterm Evaluation, November 2007 Save the Children 35 a. Gueleba Village 3. Dr. Coulibaly Check Oumar, District Health Officer 4. Kane Haoua Traoré, Midwife/Training Team 5. Issoumaïla Doumbia, District Pharmacist In each health center, the teams met with the ASACO management committee, and health center staff including the head of the health center, the responsible person for the pharmacy, and other staff as available. In each community, the teams met with the two Relais who were responsible for the Drug Kits, the Oversight Committee/Health Team members, and a group of mothers. CS-20 Sikasso, Mali, Midterm Evaluation, November 2007 Save the Children 36 ANNEX 5 – Compiled List of Recommendations Technical Recommendations Malaria • Facilitate the essential steps for introduction of ACT through the Drug Kits as quickly as possible. • Building on the experience with assuring supply and on documentation developed in the two districts where ITNs are available through the Relais, encourage the other districts to consider distributing ITNs at the community level. Continue to facilitate the supply of ITNs at the district level when necessary. Diarrhea • Introduce zinc in the Drug Kits ARI • Reconsider the ARI strategy in order to raise its profile in the project. The inclusion of timers as part of the Drug Kits would allow the Relais to have a concrete action with which to associate ARI messages, and would provide an incentive for mothers to visit the Relais for a child with pneumonia to determine their need for referral despite the lack of medical treatment at the community level. FP • Taking advantage of the men already involved with the project (Relais, Oversight Committee and health team) develop a strategy for targeting husbands of women who are interested in FP through a man to man (pairs-educateurs) approach. • Involve religious leaders in FP promotion based on the experience from Ségou. Community Management of Drug Kits • Develop a strategy for motivating the Relais as well as for supporting the expenses associated with their activities. (Note: coordinating these discussions with the introduction of ACT in the Drug Kits will likely facilitate solutions) o Mobilize communities to show their appreciation for the Relais activities. For example: helping the Relais with cultivating their fields, and supporting the expenses associated with the monthly meetings with profits from the Drug Kits, collecting money, and/or providing money through the community associations. o Encourage recognition from the district and CSCOMs through personal appreciation and encouragement, priority treatment, and possibly subsidized care. o Support the participation for both Relais in the monthly meetings. o Consider provision of small “appreciation gifts” such as T-shirts for the Relais on an intermittent basis. o Involve the ASACOS in providing encouragement and additional support for monthly meeting expenses as needed. • Facilitate discussion meetings among the Relais, Oversight Committee and Health Team (possibly as part of evaluation feedback) to review their roles and responsibilities, and to plan for the sustainability of the Drug Kits: CS-20 Sikasso, Mali, Midterm Evaluation, November 2007 Save the Children 37 o Broaden transparency and community involvement in the Drug Kits through regular feedback to the village chief and leaders from the Oversight Committee on their activities and findings. o Expand the role of the Oversight Committee to include active management of the money generated by the Drug Kits. Ideas include distribution of money as motivation for the Relais, use of the money to support the monthly meetings, and/or to support other health activities in the village. o Reinforce the involvement of the second Relais by delegating responsibility for some of the record books (e.g. births/deaths, health education activities, and village visits), and for strengthening the health education activities. o Plan for the long term reimbursement of expenses for monthly meetings and medicine re-supply. o Reconsider ways to increase women’s involvement in the Oversight Committees. BCC Strategy • Strengthen family planning promotion–see above • Reinforce health education activities by: o Focusing on one message (one advisory card) at a time and developing it in a participatory way – whether through group education or individual counseling. o Reinforce the promotion related to weaker objectives such as ARI. Capacity Building and Partnership – Health Center Level • Reinforce the effectiveness of the monthly meetings as the central element of supervision and capacity building at the community level. o Work with health center staff to identify discussion priorities, plan the agenda, and to review progress while progressively decreasing active advisor participation in the meetings themselves. o Develop a series of mini-modules on a variety of topics which can be used for providing refresher training as part of monthly meetings. o Assure support for participation of both Relais from each community in the monthly meeting. • Facilitate discussion meeting, (possibly as part of evaluation feedback) among the health center staff, mayors’ office, and ASACOs to review their roles and responsibilities, and to plan for the sustainability of the community activities: o Encourage increased ASACO involvement in community activities by: ƒ Encouraging the village ASACO representative to participate in the health team in villages where there are Drug Kits. ƒ Encouraging ASACO participation in and support for the monthly Relais meetings. o Encourage mayors’ offices to include health activities, including participation in monthly meetings and community supervision, in their annual work plans (PDSEC). o Expand the involvement of health center staff beyond that of the Head by orienting them to the community activities, involving them in the monthly Relais meetings, and encouraging their supervision of Drug Kits during their village outreach activities. CS-20 Sikasso, Mali, Midterm Evaluation, November 2007 Save the Children 38 • Assure the integration of community data with that of the health center in their monthly report. Adapt an analysis table for community data and orient the health center to its use – both for their own planning and for feedback to the Relais. • In order to assure timely re-supply at the community level, health centers should make an effort to include the predicted drug needs for the village Drug Kits in their regular requests to the District Pharmacy, rather than waiting until specific requests come in. Use option to move stock between villages, particularly in the case of expiring drugs. District Partnership • Facilitate the availability of mosquito nets at district and health center levels (see above). • Develop a handover “package” (or check list) of essential information related to the community activities to be used when staff turn over at the district level. • Continue to balance the division of responsibilities between project and MOH staff in a way that assures progress while not undermining MOH ownership and responsibility. Training • Model and practice participatory adult education methods as part of the CPM training, reinforcing their role as trainers in addition to that of transferring information. Sustainability • Using the discussion meetings mentioned above, develop a plan for SC to gradually diminish its responsibilities as the different institutions at each level identify and address gaps in support for community activities. CS-20 Sikasso, Mali, Midterm Evaluation, November 2007 Save the Children 39 ANNEX 6 – Target Populations by District – 2007 Estimates DISTRICT/ ZONE SANITAIRE TOTAL POPULATION POPULATION OF VILLAGES WITH DRUG KITS TOTAL NUMBER OF VILLAGES NUMBER OF VILLAGES WITH DRUG KITS # FUNCIONAL HEALTH CENTERSl BOUGOUNI 385,720 131,818 477 182 32 KOLONDIEBA 176,122 96,000 206 109 16 YANFOLILA 130,126 78,871 145 103 15 SELINGUE 81,520 32,893 46 28 5 YOROSSO 188,194 110,156 93 56 11 TOTAL 961,682 449,738 967 478 79 CS-20 Sikasso, Mali, Midterm Evaluation, November 2007 Save the Children 40 ANNEX 7 – Training Summary Situation Des Formations Du Projet Survie De L’enfant Cs20/Sikasso I. Formation des formateurs des districts sanitaires – Training of Trainers – Health Districts Thème Année/Période Lieu # Jours # Participants Cibles Formation des formateurs sur l’offre des services à base communautaire (paludisme, diarrhée, IRA, PF, CCC, gestion des CP) Training trainers in community level services 17 au 22 Octobre 2005 Bougouni 6 jours 6 days 25 ־ 4 médecins doctors ־ 4 sages femmes midwives ־ 5 techniciens supérieurs de l’action sociale social workers ־ 2 assistants médicaux medical assistants ־ 10 conseillers CS-20 districts Project District Advisors II. Formation des formateurs des relais communautaires (Chefs de postes médicaux) Training of Relais by Health Post In-charges. Thème District sanitaire Année/Période Lieu # Jours # Participants Cibles Bougouni 30 Janvier au 03 Février 2006 Bougouni 6 27 Chefs de postes médicaux Health Center Heads Kolondièba 17 au 22/11/05 Kolondièba 6 16 Yanfolila 15 au 21/11/2005 Yanfolila 6 13 Yorosso 18 au 23/11/2005 Yorosso 6 11 ־ 10 CPM sage 1 ־ femme Sélingué 21 au 26/11/2006 Sélingué 6 5 Formation des formateurs sur l’offre des services à base communautaire (paludisme, diarrhée, IRA, PF, CCC, gestion des CP) Training Health Center Heads in community level interventions /////////////////////////////////// /////////////////////////////// Total 72 III. Formation des membres des comités de surveillance des caisses pharmaceutiques villageoises Training of Oversight Committee Members Thème District sanitaire Année/Période Lieu # Jours # Participants Cibles Bougouni 2006 Aires de santé 3 days 354 ־ secrétaires Kolondièba 2006 Aires de santé 3 176 Yanfolila 2006 Aires de santé 3 204 Yorosso 2006 Aires de santé 3 112 Sélingué 2006 Aires de santé 3 52 Formation des membres des comités de surveillance des CP (système de gestion des CP) Training Oversight Committees on management ־ trésoriers et/ou commissaires aux comptes formés ,secretaries ־ treasurers and //////////////////////////////////// //////////////////////////////// Total 898 commissioners IV. Formation des relais (gérants des CP) Training of Drug Kit Managers Thème District sanitaire Année/Période Lieu # Jours # Participants Cibles Bougouni 2006 Aires de santé 6 days 358 ־ gérants principaux Kolondièba 2006 Aires de santé 6 201 Yanfolila 2006 Aires de santé 6 206 Yorosso 2006 Aires de santé 6 112 Sélingué 2006 Aires de santé 6 56 Formation des gérants des CP sur l’offre des services à base communautaire (paludisme, diarrhée, IRA, PF, CCC, gestion des CP) ־ gérants suppléants ־ 2 Relais per community Training Drug Kit managers on community level services //////////////////////////////////// //////////////////////////////// Total 933 2 3 V. Formation des nouveaux chefs de postes médicaux (CPM) Training of New Health Center Heads Thème District sanitaire Année/Période Lieu # Jours # Participants Cibles Bougouni 10 au 13/05/2007 Aires de santé 4 6 Kolondièba 2007 Aires de santé 4 5 Yanfolila 2007 Aires de santé 4 5 Yorosso 14 au 17/05/2007 Aires de santé 4 4 Sélingué Aires de santé 0 0 Formation des gérants des CP sur les services à base communautaire (paludisme, diarrhée, IRA, PF, CCC, gestion des CP) Training trainers of Drug Kit Managers on community services //////////////////////////////////// //////////////////////////////// Total 20 Nouveaux chefs de postes médicaux New Heads of health Centers VI. Recyclage des relais (gérants des CP) Refresher for Drug Kit Managers Thème District sanitaire Année/Période Lieu # Jours # Participants Cibles Bougouni 2007 Aires de santé 4 274 Kolondièba 2007 Aires de santé 4 207 Yanfolila 2007 Aires de santé 4 206 Yorosso 2007 Aires de santé 4 112 Sélingué 2007 Aires de santé 4 56 Recyclage des gérants des CP en techniques de CCC et sur paludisme, diarrhée, IRA & PF Refresher training of Drug Kit Managers on community interventions community services //////////////////////////////////// //////////////////////////////// Total 855 ־ gérants principaux ־ gérants suppléants ־ 2 Relais per community T1 T2 T3 T4 2009 DK Population Expected Total Population Expected Reached 30% of the children less than 5 years old slept under an ITN the previous night Children 0 to 5 years old BCC for the promotion of ITN use by the DK Managers in the villages and radio broadcasts XXXX 22,937 49,046 % of children less than 5 years old having slept under ITNs the preceding night District Advisors, Health Center personnel, DK Managers Final KPC Study 50% of pregnant women slept under an ITN the previous night Pregnant women BCC for the promotion of ITN use by the DK Managers in the villages and radio broadcasts XXXX 11,244 24,042 % of pregnant women having slept under ITNs the preceding night District Advisors, Health Center personnel, DK Managers Final KPC Study 60% of the children less than 5 years old who had a fever in the 2 last weeks used the village drug kits or Health Centers Children 0 to 5 years old Promotion of the use of the DK and Health Center by the DK Managers, radio broadcasts, the District Advisors and the Health Center personnel XXXX 45,874 98,092 % of children less than 5 years old with fever having used the DK and/or Health Center for treatment District Advisors, Health Center personnel, DK Managers Final KPC Study 70% of pregnant women will receive IPT for malaria Pregnant women BCC for the promotion of the importance of two doses of SP during pregnancy by the DK Managers, the Health Center personnel and radio broadcasts XXXX 15,741 33,659 % of pregnant women having taken 2 doses of SP Health Center personnel, DK Managers, District Advisors Reports, Final KPC Study 100% of the DK mangers will be trained in case management of simple malaria with ACT Drug Kit Managers DK Manager training on case management of simple malaria with ACTs and the use of counseling cards X 956 % of DK Managers trained in case management of simple malaria using ACTs Training Coordinator, Head Nurses, District Advisors Training reports 100% of the DKs will receive ACTs Villages DKs Estimate of the need, orders, purchase of ACT, and distribution XX 449,738 961,682 % of DKs stocked with ACTs CS-20 Leadership, Administration, District Delivery receipts 100% of the DKs will be equipped with counseling cards for ACTs Villages DKs Design, printing, and distribution of counseling cards XX 1,063 counseling cards Number of counseling cards made CS-20 Leadership, Administration, District Health Office, Services contract and Delivery receipts 100% of messages will be created, recorded and broadcast on radios Radios Design of messages, development of contract with the radio stations, and message dissemination XX 5 Radio Stations Numbers of contracts Training Coordinator, Administration, District Health Office, Services contract 80% of the DKs will not have stock-outs of ACTs in the previous 3 months Children 0 to 5 years old Advocacy for the introduction of ACT into the DK by Regional Health Office and Save the Children staff X 382 % of DK not having stock-outs of ACTs Coordination health, DK Managers, HC Staff (Head Nurses/GDV), District Advisors Monthly HIS reports, Final KPC Study 80% of the mothers know at least 3 danger signs for childhood diseases Mothers of children 0 - 5 years old BCC on the danger signs of childhood diseases by the DK Managers and the radio stations X X X X NA NA % of mothers who know at least three danger signs in children DK Managers, HC Staff, District Advisors Final KPC Study 60% of the mothers know where a DK is located Mothers of children 0 - 5 years old BCC on the location of DKs in the villages with CHWs X X X X NA NA % of mothers who know where the DK is DK Managers Final KPC Study 80% of the women declare that ITNs are very important for children less than 5 years old Mothers of children 0 - 5 years old BCC on the importance of ITNs for children under 5 by the DK managers and through radio broadcasts X X X X NA NA % of mothers who declare that the ITNs are important for children 0 to 5 years old DK Managers Final KPC Study Information Source ANNEX 8 - Annual Project Plan CS-20 Sikasso (Fiscal Year: October 2007 - September 2008) OBJECTIVES Target Population Activities Period Results Indicators Persons Responsible I - MALARIA CS-20 Sikassso, Mali, Midterm Evaluation, November 2007 Save the Children 44 T1 T2 T3 T4 2009 DK Population Expected Total Population Expected Reached 60% of the children less than 5 years old with respiratory difficulty or fast breathing seen by a DK manager are referred in the Health Centers Children 0 to 5 years old Correct referral of children under 5 to the Health Center XXXX 45,874 98,092 % of children 0 to 5 years old with pneumonia who visit a DK and are referred to a Health Center DK Managers Monthly HIS reports, Final KPC Study 80% of the mothers know at least 3 danger signs of childhood diseases Mothers of children 0 - 5 years old BCC on the danger signs of childhood diseases by the DK Managers and radio stations X X X X NA NA % of mothers who know at least three danger signs in children DK Managers, HC Staff, District Advisors Final KPC Study 30% of the children less than 5 years old having diarrhea in the 2 last weeks received ORS, an appropriate home￾based solution and/or increased fluids Children 0 to 5 years old BCC on the importance of ORS in the treatment of diarrhea by the DK Managers and radio stations, BCC on feeding and increasing fluids for children with diarrhea XXXX 22,937 49,046 Number of children < 5 years old with diarrhea who receive ORS DK Managers, HC Staff, District Advisors Monthly HIS reports, Final KPC Study 80% of the DK did not have stock-outs of ORS in the last 3 months Villages DKs Follow-up of the stock of the DK, regular restocking of the DK by the HC XXXX 382 DK A number of DKs stocked with ORS Head Nurses, District Advisors Supervision Reports, book of stock 80% of the mothers know at least 3 danger signs of childhood diseases Mothers of children 0 - 5 years old BCC on the danger signs of childhood diseases by the DK Managers and radio stations X X X X NA NA % of mothers who know at least three danger signs in children DK Managers, HC Staff, District Advisors Final KPC Study 8% of the women of reproductive age use a modern method of family planning Women of Reproductive Age BCC for the promotion of FP products targeting men by radio broadcasts and the DK Managers XXXX 17,963 8,275 % of women of reproductive age who use a modern method of FP DK Managers, HC Staff Reports LOCATED 80% of the DKs will not have stock-outs of FP products in the last 3 months Villages with DKs Follow-up of the DK stock, regular restocking of the DK by the FD and DRC XXXX 498 DK A number of DKs stocked with FP methods Head Nurses, District Advisors Supervision Reports, stock book 50% of FP customers report that they are satisfied with the services received Customers (men + women) BCC for the promotion of FP products (men + women) by the radio stations and the DK Managers, referral to Health Center and management of side effects X X X X NA NA NA % of FP clients who report they are satisfied with the services they received DK Managers, HC Staff KPC Study Objectives Target Population Activities Period Results Indicators Persons Responsible Information Source IV - FAMILY PLANNING III - DIARRHEA II - PNEUMONIA CS-20 Sikassso, Mali, Midterm Evaluation, November 2007 Save the Children 45 T1 T2 T3 T4 2009 DK Population Expected Total Population Expected Reached 60% of the Oversight Committees take part in the monthly meetings of the DK managers in the Health Centers Oversight Committees Updating the Oversight Committees by reminding them of their roles and responsibilities for improved involvement in DK activities X X X X 478 Number of active Oversight Committees District Advisors, HC Staff Annual Reports, Monthly Advisor Reports ASACO (Health Center Management Board) Initiation of meetings with the ASACO to encourage them to support the monthly meetings between the DK Managers and Health Centers XXXX 961,682 Number of meetings held with a report of the proceedings HC Staff, District Advisors Annual report, monthly Advisor Management Report, meeting reports Village Health Team Update the village health teams on their roles and responsibilities concerning the DKs XXXX 449,738 Number of active village health teams HC Staff, District Advisors Annual report, monthly Advisor Management Report, meeting reports Mayors Follow the protocol for collaboration with the mayoral offices XXXX Number of contracts with mayors fulfilled District Advisors Annual report, monthly District Council Management Report Village Health Team Establishment of formal, periodic meetings between the village health team members XXXX 449,738 Number of meetings held with a report of the proceedings HC Staff, District Advisors Annual report, monthly Advisor Management Report Village Health Team Institutionalize formal quarterly feedback meetings with the community leaders XXXX 449,738 Number of feedback meetings held with meeting reports HC Staff, District Advisors Annual report, monthly Advisor Management Report Village Health Team Update the Health Center teams on their roles and responsibilities related to community activities XXXX 449,738 Number of active Health Center teams HC Staff, District Advisors Annual report, monthly Advisor Management Report, Meeting Report Village Health Team, ASACO, DK Managers, Oversight Committees, Health Center staff Discuss the activities, collect information, restock the DK, prepare mini orientation handbooks for DK Managers, update DK Managers on technical interventions, and plan outreach visits XXXX 449,738 Number of monthly meetings held with list of participants & meeting reports HC Staff, ASACO Annual report, monthly Advisor Management Report Assistant DK Manager Engage the second DK managers in more activities (transfer of certain data collection tools such as: death and birth records, talks… and household visits by patients to the DK) XXXX 449,738 Number of secondary DK managers active in the activities District Advisors, HC Staff Annual report, monthly Advisor Management Report 70% of the members of the Oversight Committees are women Oversight Committees Revise the composition of Oversight Committees with increased emphasis on the need for women representatives X X 1,004 Number of women who are members of the Oversight Committees HC Staff, District Advisors Annual report, monthly Advisor Management Report, Meeting Report 60% of the Oversight Committees are supervised at least 1 time per month by the Health Center Oversight Committees Revise the supervision guidelines for the Health Centers, orient other HC personnel on project activities, organize supervision, design and provide supervision reports, provide feedback on the supervision visits during the monthly meetings in the Health Center X X X X 287 Oversight Committee numbers CS-20 Leadership, Head Nurses, District Advisors Supervision Reports OBJECTIVES Target Population ACTIVITIES PERIOD RESULTS Indicators V - COMMUNITY MANAGEMENT OF DRUG KITS Persons Responsible Information Source CS-20 Sikassso, Mali, Midterm Evaluation, November 2007 Save the Children 46 T1 T2 T3 T4 2009 DK Population Expected Total Population Expected Reached 50% of District Health Office carried out 2 integrated supervisions the previous year Health Center staff Revise the supervision guides so that they include CS-20 activities in the integrated supervisory visits of the Health Centers X X 39 Health Center Number of HCs supervised District Health Team, CS-20 Leadership Supervision Reports 80% of District Health Offices will hold monthly meetings to monitor Health Center progress Health Center staff Present CS-20 activities during monthly meetings in order to discuss challenges and propose solutions XXXX 5 District Health Office Number of monthly meetings held District Health Team Monthly District Advisor Activity Reports The Health Centers, the District Health Officers, Ministry of Health and SC will collaborate to write 2 articles on the DK approach DK and promotion of the ITNs District Health Team, CS-20 Staff, Regional Health Office, National Health Office (DNS) Establish a team for writing articles, organize a meeting for team members, produce a draft of the articles, validate the drafts and disseminate the articles written X Number of articles written CS-20 Leadership, District Health Team, Regional Health Team, National MOH Articles written Regional supervision of the districts CS-20 Leadership, Regional Health Office Establish the protocol and make the budget available, organize supervision activities and produce the reports X X 5 districts 449,738 961,682 Number of districts supervised CS-20 Leadership & Regional MOH Supervision Reports Supervision by Health Centers of the villages DK Establish the protocol, establish and make available the budget, organize the supervision, produce the reports X X X X 78 Health Centers 449,738 961,682 Number of DKs supervised CS-20 Leadership & Head Nurses Supervision Reports PERIOD RESULTS Indicators Persons Responsible Information Source VI - CAPACITY BUILDING OF DISTRICTS AND Health Centers OBJECTIVES Target Population ACTIVITIES CS-20 Sikassso, Mali, Midterm Evaluation, November 2007 Save the Children 47 T1 T2 T3 T4 2009 DK Population Expected Total Population Expected Reached DK Managers, Health Center Staff Prepare the data collection forms for each level; provide the District Advisors with data collection forms, receive the data collection reports, check the data collected and enter the data into the HIS database; and present the data during quarterly staff meetings XXXX Data collection reports received M&E Coordinator HIS Reports CS-20 Leadership Participate in the regional CROCEP meetings X Number of Regional CROCEP meetings attended CS-20 Regional Advisor Participation Reports CS-20 Leadership Participate in the quarterly Regional MOH meetings XXXX Number of Quarterly meetings attended CS-20 Regional Advisor Participation Reports District Advisors, Head Nurses, DK managers Supervise project activities on the ground XXXX Number of supervision visits conducted CS-20 Regional Advisor Supervision Reports Health Center Integrate the data from the DK Managers in the monthly Health Center reports XXXX 478 DK Number of Health Center having integrated DK data in their monthly Head Nurses, District Advisors Monthly District Advisor Activity Reports ASACO Members Ensure that ASACO supports the monthly DK Manager and Health Center meetings X X X X 78 ASACO Number of ASACOs which support the monthly meetings of the DK Managers with Health Center District Advisors, District Health Office, Head Nurses Meeting Reports with statement Villages (Community leaders) Ensure that villages support the participation of the DK Managers in the monthly meetings with Health Centers and the restocking of the DK (traveling expense) XXXX 478 Village DKs Number of village DKs which support the travel of the DK Managers for the monthly meetings & restocking District Advisors, Head Nurses, ASACO Meeting Reports Oversight Committees Ensure that the Oversight Committees share the quarterly profits generated by the DK sales with the two DK Managers; Organize BCC sessions with the villages to plan other strategies to support the DK Managers' efforts XXXX 478 Village DKs Number of Oversight Committees which share the DK profits with the managers; number of BCC sessions conducted in the villages District Advisors, Head Nurses Monthly District Advisor Activity Reports Health Center personnel, ASACO, Mayor, District Governor, District Health Teams Organize forums at the district levels to plan strategies for the continuity of activities after the project ends X 5 Health Districts Number of forums held CS-20 Leadership, District Advisors, District Health Office Forum Reports VIII - PHASE OUT PLAN Ensure the sustainability of CS-20 activities OBJECTIVES Target Population ACTIVITIES PERIOD RESULTS Indicators VII - MONITORING AND EVALUATION Persons Responsible Information Source CS-20 Sikassso, Mali, Midterm Evaluation, November 2007 Save the Children 48