1 Mid-Term Evaluation Report Of Child Survival Project Kita District, Mali For Plan International USA, Inc. Childreach Cooperative Agreement No: HFP-A-00-01-00046-00 September 30, 2001, - September 29, 2006 Report Prepared By Aguibou Coulibaly, Plan - Mali Souleymane Bagayoko, Plan - Mali Pierre-Marie Metangmo, Plan – USA Laban Tsuma, Plan - USA Sandra Wilcox, Consultant December 2004 – January 2005 2 ACKNOWLEDGEMENTS The principal author of this report would like to thank the staff of the Kita Program Unit for its support, hard work and patience during the period of the mid-term evaluation. They gave freely of their time despite their busy schedules and responsibilities. It is also important to acknowledge the high degree of dedication and professionalism that they bring to their work. It is evident in the quality of the programs that the Program Unit manages under very difficult circumstances. It is also important to acknowledge the hard work and dedication of the Kita District health team who despite their pressing commitments, made significant contributions as members of the evaluation team. The useful insights and contributions of Plan/ Mali’s National Child Survival Coordinator, and its Health Coordinators for Banamba and Kangaba, were also appreciated, as were those of the representatives from the FELASCOM, the Kayes Regional Health Office, and the Kita Social Service Office. The efforts and support of Plan International USA and its Health Program Coordinator must also be acknowledged for their guidance before, during and after the evaluation. Despite the best efforts of both the Kita PU and the headquarters staff, factual errors may persist in the report. They must be considered the responsibility of the principal author who tried to grasp the complexity of the environment in a short period of time. She hopes that she was successful. 3 ACRONYMS ANC Antenatal Care ARAFD/C Consortium of local NGOs (ACD, ARAFD and ODILE) ASACO CSCom Health Board/ Community Health Association BCC Behavior Change Communication CBOs Community-based Organizations CDC Community Development Coordinator CPO Country Programme Outline CPM Chief of Medical Post CSCom Community Health Center CSP Child Survival Program CSRef Referral/ District Health Center CSV(VHC) Village Health Committee DIP Detailed Implementation Plan DNS National Health Directorate DRDS-ES Regional Directorate of Social Development and Economic Solidarity DRS Regional Health Directorate EVP/EPI Extended Vaccination Program FELASCOM National Association of Community Health Associations HC Health Center HIS Health Information System IEC Information, Education, Communication IMCI/PCIME Integrated Management of Childhood Illness ICPM Chief Nurse of medical Post KPC Knowledge, Practice, Coverage LQAS Lot Quality Assurance Sample MOH Ministry of Health NGO Non-Government Organization NICRA Negotiated Indirect Cost Rate ORS Oral Rehydration Solution PDRIK Kita Integrated Rural Development Project (Debt Relief ) PRA Participatory Rural Appraisal PU Plan Program Unit (Regional Office) PUM Plan Program Unit Manager SE Child Survival SSSC District Health and Social Development Service TBA Traditional Birth Attendant TT Tetanus Toxoid UNICEF United Nations International Children’s Emergency Fund USAID United States Agency for International Development USNO Plan United States National Office WHO World Health Organization WRA Women of Reproductive Age (15-49) 4 Table of Contents Acknowledgements Acronyms A. Executive Summary 5 B. Assessment of Progress toward objectives 13 B.1. Technical Approach 13 B.2. Cross-Cutting Approaches 19 a) Community Mobilization 19 b) Communication for Behavior Change 21 c) Capacity Building 22 i) Strengthening the PVO Organization 22 ii) Strengthening the Local Partner Organizations 22 iii) Health Facilities Strengthening 22 iv) Strengthening Health Worker Performance 25 v) Training 25 d) Sustainability Strategy 27 C. Program Management 28 C.1. Planning 28 C.2. Staff Training 29 C.3. Supervision of Program Staff 30 C.4. Human Resources and Staff Management 30 C.5. Financial Management 31 C.6. Logistics 31 C.7. Information Management 32 C.8. Technical and Administrative Support 33 E. Conclusions and Recommendations 33 F. Results Highlight 37 III Action Plan 38 IV. Attachments 42 5 A. Summary The Plan- Kita Child Survival Project is located in the southeastern section of Kita district in southern Mali. The project targets 52,091 women of reproductive age, 10,418 infants 0-11 months and 36,464 children ages 1-4. Plan has been working in the region since 1994 in areas of integrated health and development. It has been working with the same project NGO partner, ARAFD/C, during this time. According to the 1996 DHS, infant mortality is 135 per 1000 live births and child mortality is 259 per 1000. Principal causes of death are malaria, diarrhea and measles with underlying complications attributed to malnutrition. Utilization of health services was extremely low, (8%). The program approach is to strengthen and improve the quality of the existing health care system. This approach comprises two strategies: (1) Training and building the capacity of health personnel and supporting community health committees (ASACOs) in order to increase utilization of health care services, and (2) Supporting implementation of IMCI skills/ training in the CSComs. The project’s four intervention areas are reduction of morbidity and mortality associated with Malaria (35%), reduction of morbidity and mortality through control of Diarrheal Disease (20%), increasing immunization coverage (35%) in the district for children and pregnant women , and reduction of morbidity and mortality associated with Pneumonia (10%). As can be seen from the LQAS results achieved in year 3, the project is making substantial progress towards achievement of its objectives. The project has either surpassed or achieved most of its year 3 objectives. It has been particularly successful in increasing immunization coverage. The percentage of fully vaccinated children in Kita district increased from 30 percent at the baseline to nearly 70 percent at the November LQAS. Other notable achievements include the strengthening of the CSCom health committee or ASACOs management capabilities. For the first time, project CSComs are now able to fully support the health center staffs and maintain regular supplies of essential medicines. These achievements are due to improved ASACO management skills, improved quality of care through IMCI training and increased community mobilization activities through village relays and Village health Committees. The main concerns (constraints) revolve around whether the communities and CSComs will be able to maintain the level of health achievements after the project phases out of their areas. All actors believe that the phase-in/ phase-out period (2 years) is too short. To address this, the project is proposing to conduct an evaluation of the CSComs that have been phased out of and make recommendations for follow-up strategies. Other constraints concern the lack of anticipated support from PIDRIK for building and equipping of 10 CSComs in the district, which was to complement CS project support. This was to have occurred at the beginning of the project and although the support is still promised, so far it has not arrived. Plan and the MOH are addressing the shortfall by contributing extra resources, but there is a question as to whether the project will be able to reach and strengthen all 26 CSComs as planned. The CS Project has done an impressive job of strengthening the skills and capacities of district health staff through training. The IMCI training has been particularly well 6 received and has greatly improved the quality of care at the health centers. The project has also supported immunization efforts through transportation assistance and fuel. The project has also provided equipment for all project CSComs. The CSCom health committee (ASACO) management skills have also been strengthened through the project, as have the village health committees. The project has also greatly improved the skills of the NGO partner staff in the areas of community mobilization. The conclusions, challenges and recommendations are discussed below: Conclusions Successes · As noted in the LQAS results, the project is making good progress in achieving its projected results. It has already either met or surpassed most of its year 3 objectives. Perhaps the biggest success is the achievement of almost 70 percent of children who are fully vaccinated. The recognition of this achievement on the part of the MOH is a huge indication of the project’s success. Prior to project initiation, the MOH had targeted Kita District as having the worst immunization levels in the country. Now they are sending a team to verify the results and are using it as a model for other districts. · The project has also achieved tremendous success in mobilizing communities to use health services. The relays, animators and VHCs have been very effective in mobilizing their communities to seek care and preventive services, especially immunizations and prenatal care. The number of referrals by village relays increased from 27 in 2003 to 118 in 2004. The project staff have also been innovative in helping health center staff bring services to the villages. This has been by supporting the work of the CSCom vaccination staff and by encouraging the matrons to accompany them and offer prenatal consultations in the villages at the same time. · Another contribution has been the strengthening of the management skills of the ASACOs. Every ASACO visited had its accounting and bank books in order. Also, every ASACO was able to meet its financial obligations of supporting the salaries of the CSCom staffs. Given the poverty of the region, this is a commendable achievement and may be the only country/ region where local government health services are completely supported by community constituents. · The project has done a good job of developing a community-based health information system. There is good evidence that the information is being used by the communities and CSComs for planning and for making health care decisions. · The IMCI training that was instigated by the project, has greatly improved the quality of care at the CScoms. The health staff are now able to perform broad diagnosis of child illness and provide counseling for caregivers. This improved quality of care seems to be partially responsible for the increase in visits for services. · Plan has done a good job of strengthening local partner technical capacities. The local NGO staff now possess health and mobilization skills that they are able to use in other areas. 7 · Despite the unavailability of the originally planned support, the project has been able to meet all the designated equipment needs for the project CSComs and provide material support for outreach activities. Challenges · One of the biggest issues is the sustainability of program inputs and whether the phase in/phase out timeline is too short. The project has achieved tremendous success in getting people to use preventive and curative services. All the project inputs such as IMCI training of providers, formation of relays and VHCs and corresponding community mobilization activities, and strengthening of the ASACOs so they can fulfill their management mandate, have contributed to the increase use of services and the self-sufficiency of the CSComs. The question is whether the ASACOs, VHCs and relays are sufficiently capable at this point to continue CSCom support activities on their own. There is concern that the organizational support structures are not in place yet for them to do this. · It is clear that the relays need a lot of motivation to continue their activities. They are asked to maintain 7 registries, organize meetings, follow-up on referrals, mobilize villages for immunizations, and generally address health issues in the villages. The project animators who live in the area are doing a good job of training and motivating them but once the project phases out, they will no longer do this. · The project has had to work around the fact that PIDRIK has not come through with the promised constructing and equipping of 10 CSComs for the district health system. Plan and the CSRef have been able to come up with some extra resources but it is unclear whether they will be able to fully equip and construct all 26 CSComs as planned. · The project has done a good job of working with community HC and relays and training them in areas of community mobilization and in the health intervention areas. However, there is still work to be done in the area of IEC/ BCC. Many villagers are still unaware of proper care related to CDD, such as exclusive breastfeeding and handwashing. Likewise some villages are very aware of malaria prevention and use of bednets while others are not as attuned. The project lacks an overarching BCC strategy that targets specific behaviors and includes specific communication interventions. The project is also lacking in IEC materials that the relays and animators could use to reinforce health messages. In the DIP the project had planned to develop materials but was unable to with the budget adjustments. 8 · Given the difficult roads and the large distances to be traveled, particularly with the 5 new CSComs that are located at the other end of the district, the project staff will need significant travel time and vehicle support for supervision and support activities. This will infringe on the vehicle support already required for on-going monitoring activities in the current project areas. In addition, the motorbikes that the animators use are almost worn out and require significant repairs. Recommendations · The project should conduct an evaluation of the CSCom areas that it has phased out of to see how well the health agents, ASACOs, VHCs and relays are doing in managing the health activities without project support. If there are problems then the project should take steps to address them in a timely manner. · In order to strengthen the sustainability of the relays, the evaluation team recommended that the Chef du Post become the supervisor of the relays and their activities. This could be done at the time of regular outreach visits to the villages and/or through monthly meetings of all the relays at the CSCom. There are some administrative arrangements that need to be made in order for the CDP to supervise the village relays but once this is done there would be a system in place for on-going supervision of village health activities. This could start by having the CSCom team and the animators conduct joint team supervisions in the surrounding villages. Then after the animators leave, the supervision system would be in place. This would strengthen the sustainability of relay and village health committee activities. The on-going supervision of the relays would also strengthen their motivation, particularly if they were seen as working with the chef du Post who is very respected in the area. The evaluation team also recommended that the project staff and district health staff attend quarterly supervision meetings done at the CSComs. Their presence would encourage the other actors (ASACOs, CSCom staff, VHCs and relays) to attend and get involved in the process. They also pointed out that if the district staff were there, there would be more likelihood that the mayors would participate in the relay supervision meetings. This is important given that the mayors have access to their own source of funds and would be more likely to spend them on relay activities if they were involved in their management. · If PIDRIK comes through with funding this year for the construction of the 10 CSComs that they are promising, then the project needs to work with them and the MOH to see how to equip the facilities and carry out the corresponding community activities with the limited project resources. If PIDRIK does not come through with the anticipated funding, then the project needs to decide how many more CSComs it can realistically work with by the end of the project. Once the CS project has the results of the CSCom phase-out evaluation discussed above, it will be able to decide whether it should put more resources into new CSCom 9 areas or if it should devote them to strengthening the sustainability of the old ones. · Recommend that the project contract a short term BCC consultant to help them design a strategy that includes behavior change targets and strategies as well as awareness raising communications activities. It would be useful to employ more entertaining cultural formats that could be used to create interest in the villages such as stories, songs, street theater, and other methods. · The CS project needs to follow through on its original plan to acquire and/or develop IEC materials that are appropriate for the target audiences. The health messages need to be reinforced through support materials. Also the materials would help the relays with their educational tasks. · If resources permit, the project should consider acquiring another vehicle for the work in the new CSComs in the far northern section of the district. It should also consider replacing some of the worn out motorbikes being used by the animators. · In order to strengthen some of the intervention activities, the project should consider providing breastfeeding technical assistance to improve technique and promote exclusive breastfeeding in the communities. Action Plan III. Action Plan Recommendation Steps that need to be taken to effectively address recommendation Actor who will be responsible for each step Timeline. By when each step shall have been accomplished The project should conduct an evaluation of the CSCom areas that it has phased out of to see how well the health agents, ASACOs, VHCs and relays are doing in managing the health activities without project support. If there are problems then the project should take steps to address them in a timely manner. 1) Develop TOR of the survey 2) Identify the institution, which will carry out the survey and negotiate a contract for carrying out the survey 3) Organize a workshop for the validation of the protocol before the starting of the survey. 1) Dr Bagayoko PSE Coordinator 2) Dr Aguibou Coulibaly 3) Dr January 05 March 05 April 05 10 Bagayoko 2. In order to strengthen the sustainability of the relays, the evaluation team recommended that the Chef du Post become the supervisor of the relays and their activities. This could be done at the time of regular outreach visits to the villages and/or through monthly meetings of all the relays at the CSCom. There are some administrative arrangements that need to be made in order for the CDP to supervise the village relays but once this is done there would be a system in place for on-going supervision of village health activities. This could start by having the CSCom team and the animators conduct joint team supervisions in the surrounding villages. Then after the animators leave, the supervision system would be in place. This would strengthen the sustainability of relay and village health committee activities. The on￾going supervision of the relays would also strengthen their motivation, particularly if they were seen as working with the chef du Post who is very respected in the area. The evaluation team also recommended that the project staff and district health staff attend quarterly supervision meetings done at the CSComs. Their presence would encourage the other actors (ASACOs, CSCom staff, VHCs and relays) to attend and get involved in the process. They also pointed out that if the district staff were there, there would be more likelihood that the mayors would participate in the relay supervision meetings. This is important given that the mayors have access to their own source of funds and would be more likely to spend 1) Make administrative arrangements (making memorandum of understanding, entering of the supervision responsibility in the job description of heads of post) 2)Define and negotiate the implementation of a mechanism of joint supervision of health activities in villages with a schedule of implementation and budget 3) Develop and negotiate the implementation of a plan of organization of meetings with actors (mayor, CSREF, PSE team) 1) Head of district health center 2) Elie Coulibaly Responsible for SE Monitoring and Evaluation 3) Dr Souleymane Bagayoko January 05 January 05 February 05 11 them on relay activities if they were involved in their management. 3. If PIDRIK comes through with funding this year for the construction of the 10 CSComs that they are promising, then the project needs to work with them and the MOH to see how to equip the facilities and carry out the corresponding community activities with the limited project resources. If PIDRIK does not come through with the anticipated funding, then the project needs to decide how many more CSComs it can realistically work with by the end of the project. Once the CS project has the results of the CSCom phase-out evaluation discussed above, it will be able to decide whether it should put more resources into new CSCom areas or if it should devote them to strengthening the sustainability of the old zones. (We purpose to reformulate this recommendation as below due to the fact that the extension of the project is no longer linked to PDRIK) On the basis of the results of the CSCom Phase out study as recommended above, determine with the ministry of health and the other partners if more resources must be put into new health zones or if these must be devoted to reinforce the sustainability of the old zones. See Action plan Recommendation 1 12 4. Recommend that the project contract a short term BCC consultant to help them design a strategy that includes behavior change targets and strategies as well as awareness raising communications activities. It would be useful to employ more entertaining cultural formats that could be used to create interest in the villages such as stories, songs, street theater, and other methods. 1) Develop terms of reference of the consultation and plan financial resources for that purpose 2) Identify the consultant and get the protocol of agreement signed 3) Designing strategy and train actors of the implementation Dr Bagayoko Dr Bagayoko Consultant (under the supervision of Dr Bagayoko March 05 March 05 May 05 5) The CS project needs to follow through on its original plan to acquire and/or develop IEC materials that are appropriate for the target audiences. The health messages need to be reinforced through support materials. Also the materials would help the relays with their educational tasks. 1) Plan financial resources for the acquisition of adequate supports 2) Make census of existing IEC materials at the level of CNIECS and ensure the obtaining of more. 3) Put IEC materials at the disposal of actors (animators and Relays) Dr Bagayoko Dr Sita Sidibe Dr Sita Sidibe May 05 Jun 05 Jun 05 6) If resources permit, the project should consider acquiring another vehicle for the work in the new CSComs in the far northern section of the district. It should also consider replacing some of the worn out motorbikes being used by the animators 1) Plan for the purchasing of a vehicle and motorbikes for the project PUM Kita February 05 7) In order to strengthen some of the intervention activities, the project should consider providing breastfeeding technical assistance to improve technique and promote exclusive breastfeeding in the 1) On the basis of the results of LQAS survey, identify communities where there is less breast-feeding and work with partners (NGOs) to Dr Bagayoko January 05 13 communities. develop a special action plan to promote exclusive breast-feeding in these communities 2) Conduct a communication campaign targeting behavior change on breast-feeding in these communities 3) Evaluate action of the campaign NGOs (under the supervision of Dr Bagayoko) Elie Coulibaly February 05 August 05 B. Assessment of Progress toward Program Objectives B.1. Technical Approach Overview Plan International is implementing a five-year Child Survival project in partnership with the Mali Ministry of Health, ARAFD/C – a local consortium of NGOs, and local municipalities in Kita District, located in the southeastern part of Kayes Region, in the republic of Mali. Kita’s total population in 2001 was estimated at 315,520 people of whom the project will cover 260,458 people by its fifth year. Estimated beneficiaries include: 52,091 women of childbearing age, 10,418 infants ages 0-11 months, and 36,464 children ages 1-4 years. Plan has been working in Kita District since October 1994, carrying out integrated health and hygiene, reproductive health, education, water and sanitation, environment, and micro-credit activities. It has been working with the local NGO partner, which has extensive experience in training and delivery of health messages. Health indicators for Kayes Region and Kita District lag behind those of Mali as a whole. According to the 1996 DHS, infant mortality in Kayes was 135/1000 and under-five mortality was 259/1000. The major causes of death of children in Kayes region are malaria, ARI, diarrhea, malnutrition and measles. Challenges to health of women and children include: poor road conditions and long distances to health centers, the insufficient number of first level health centers (only ten of the currently proposed 46 health centers are functional), inadequate quality of health services provided due to insufficient training of personnel, and low income and purchasing power of the population, combined with high user fees of health centers. Utilization of health centers is very low (around 8% of the eligible population) and traditional healers are often sought for medical advice before health care providers. 14 The program approach is to help strengthen the existing health care system and improve quality of services. This approach comprises two strategies: (1) Training and building the capacity of health personnel and supporting community health committees (ASACOs) in order to increase utilization of health care services, and (2) Supporting IMCI implementation in CSComs. The four interventions are: i) Malaria (35%): Reduction of mortality and morbidity associated with malaria in children and pregnant women, through prevention education, promotion of use of impregnated bed nets, improved treatment of malaria, and prenatal chemo-prophylaxis. ii) Diarrhea (20%): Reduction of diarrhea￾associated mortality and morbidity through a) teaching and promoting prevention measures in the home, b) strengthening mothers’ capacity to recognize and provide home treatment for mild diarrhea with fluid and dietarymanagement, and c) strengthening mother’s capacity to identify signs of moderate and severe diarrhea, know sources of care, take the child for care, and comply with health provider recommendations. iii) Immunizations (35%): Increased immunization coverage in the program area for all infants by the end of the first year of life, and tetanus toxoid (second dose) immunization for pregnant women, increased measles vaccination (including twice yearly vitamin A) and prevention of measles-caused diarrhea and death. And iv) Pneumonia (10%): Reduction of pneumonia-associated mortality through prompt, appropriate and standardized pneumonia case management, a sustainable supply of antibiotics at the health centers (CSComs), prompt recognition by relays of pneumonia signs (fever, fast breathing) and referral of suspected cases to the nearest health center, and appropriate mothers’ care-seeking behavior. B. 1. b. Progress by Intervention Area As can be seen from the table below recording key LQAS results achieved in year 3, the project is making substantial progress towards achievement of its objectives. As demonstrated, the project has either surpassed or achieved most of its year 3 objectives. Tab l e 1 : Evolution of indicators in the Kita project zone, June 2004 I ndicators Yr 3 % Ob jective Resu lts Mars 03 Resu lts Nov. 03 Resu lts June 04 Ch il dren that have an i mmun ization card 65 68 .42 86 . 84 90.73 Ch il dren fully vaccinated 65 33 . 55 50. 00 69.54 Women with a consultation card/bookl et 40 59 . 21 83 . 55 84.87 Women havi ng attended at l east three ante -natal consultations 40 29 . 61 53 . 02 55.92 Women havi ng taken iron folic acid during last pregnancy 75 58 . 55 76 . 32 79.61 Women havi ng taken ch l oroqu i ne during last pregnancy 70 65 81 . 58 82.23 15 I ndicators Yr 3 % Ob jective Resu lts Mars 03 Resu lts Nov. 03 Resu lts June 04 Women havi ng received at least two doses of ATV 40 35 32 . 89 78.95 Women havi ng been attended by a trained wo rker during childbirth 75 75 54. 61 65.13 Women havi ng taken Vit. A within 40 days after chil dbirth 35 - 8 . 55 40.13 Ch il d havi ng s l ept under an i mp regnated bednet 30 20 6 . 25 22.45 Women knowi ng 2 s i gns i ndicating a child needs care 50 45 67 . 10 90. 59 Ch il dren who when s ick received at least the same amount , or mo re, to drink as usual 50 40 82 . 95 68 . 53 Ch il dren who when s ick received at least the same amount , or mo re, to eat as usual 50 45 43 . 72 50. 34 Women knowi ng how to descr i be ORS preparati on 50 40 35 . 09 77 . 14 Immunization (35%): Goal: To increase immunization coverage in the program area for all infants by the end of the first year of life, and tetanus toxoid (second dose) immunization for pregnant women, increased measles vaccination (including twice yearly Vitamin A) and prevention of measles-related diarrhea and death. Perhaps the biggest achievement of the project to date has been its high level of vaccination coverage. With almost 70 percent of the children in the district completely immunized, the project is close to achieving its overall objective of 80 percent. The Kita district was selected for this project because it had the lowest immunization levels in Mali and was of great concern to the MOH. If they do achieve 80 percent coverage, Kita will be the only district in Mali to do so. To achieve this record level, the project has gone beyond working in the designated 13 CSComs (health center areas). Because the District Health System does not have CSComs in 30 areas, the project agreed to support additional vaccination posts and support for mobile teams and CBOs in these 30 areas. The project covered all costs related to continuous operation of the cold-chain system and supported maintenance and repair of cold-chain logistics. The MOH is sending a team to validate the LQAS findings in early 2005. If their results concur with the LQAS findings, Kita will be the district with the highest immunization rates in Mali and the MOH will be using it as a model. According to the DIP, the MOH rate for fully vaccinated children in the Kayes region in 2001 was 13.5 percent. Also only 40 percent of women had received two doses of anti￾tetanus vaccine during their last pregnancy. As we can see, the rate is now 78 percent in the Kita district. Issues also revolved around availability of vaccination cards, now close to 91 percent of children have them. 16 The project followed the strategies planned in the DIP in order to improve immunization coverage. Working with the MOH and ASACOS, they established the monthly outreach services with associated communities, supported bi-annual measles campaigns, ensured that quality vaccines were administered and provided training to CSCom health workers, particularly the vaccination staff. Malaria (35%): GOAL: To reduce mortality and morbidity associated with malaria in children and pregnant women, through prevention education, promotion of use of impregnated bed nets, improved treatment of malaria, and prenatal chemo-prophylaxis. Malaria is the primary cause for medical visits and causes 13 percent of deaths in Mali. One child in five dies of malaria before the age of five and has an average of 2 episodes per year. According to community-based project data, malaria continues to account for the bulk of childhood deaths. 226 out of 360 childhood deaths (63%) in the project area were attributed to malaria. The bulk of childhood deaths (237) 65% occurred in children between 0-11 months of age. In depth investigation by the project revealed that most malaria related deaths occurred during the rainy seasons. This finding was validated by the evaluation team when they visited local sites and reviewed census data kept by the relays and local health committees. The project staff have used these findings to show the local health committees the importance of using impregnated bed-nets during the rainy seasons. Although there is good evidence that the message about using bed-nets is reaching the communities, there are still a number of traditional beliefs that influence behavior in this regard. Evaluation team member were told that malaria was not found in rural areas, only in cities. Others stated that the nets were inconvenient to use, among other things. The numbers of referrals have increased. There is documentation that in the second year, 27 referrals were made from communities to CSComs and in year 3 this increased to 118. The evaluators noted that many referrals were made by the relays for malaria treatment at the CSComs. This is an indication that the relays and animators are getting the message out to mothers to send children with fever to the health centers, and that this is being done at early stages of disease. There is also evidence that the IMCI training provided to the CSCom staffs has dramatically improved the quality of care offered at the centers and thus improved access. Although the project has not developed an on going cost recovery program for bed-nets, they have developed an innovative strategy with the CSRef to offer free bed-nets, made available through UNICEF and Plan/ France, to fully immunized children and pregnant mothers who attend prenatal care. This strategy plus the relay mobilization activities are contributing to increased use of nets. According to the November LQAS the use of nets (34.4) is a little above the objective for year 3 (30%) but it may be difficult to reach the final program objective of 60 percent. During year 2 of the project, Plan distributed 1000 treated bed-nets to the initial 8 CSCom villages. The nets were sold at low cost and the funds used to support the local health committees. 17 During year two the project partner NGO, ARAFD/C trained community health workers (Relays) and produced a module adapted to their educational levels. The training included information on the four intervention areas including malaria prevention, recognition, referral and treatment follow-up. The relays also promote presumptive treatment for pregnant women. Diarrhea (20%): Goal: To reduce diarrhea associated mortality and morbidity through a) teaching and promoting prevention measures in the home, b) strengthening mothers capacity to recognize and provide home treatment for mild diarrhea with fluid and dietary management, and c) strengthening mother’s capacity to identify signs of moderate and severe diarrhea, know sources of care, take child to care, and comply with health provider recommendations. To date, the project has supported training for CSCom health workers and community health workers in diarrhea management. These courses have been part of the general training given to community health workers (relays), community health associations (CBOs) and to government health center staffs. The training includes the four intervention areas plus clinical and community IMCI training. As a result of the training, the government health workers, community health association workers, and NGO staff organized and conducted education sessions for mothers and community health workers (relays). The training enables these groups to recognize uncomplicated cases and treat appropriately at home with fluids and ORS, and to refer children presenting danger signs to the appropriate health centers. All of the government health staff have been trained in IMCI which includes diarrhea case management and treatment. According to the LQAS data, the project is on track with some of the diarrhea related indicators and not yet so with others. According to the November LQAS, 26 percent of mothers (target 40%) were exclusively feeding their 0 to five month old babies. Given the difficulties of convincing mothers to exclusively breastfeed, it may be difficult to achieve the 60 % end of project goal without additional training and motivation. The evaluation team noted that while the majority of mothers breastfeed, they are not using appropriate latching-on techniques and did not seem to understand the importance of exclusive breastfeeding. The women interviewed during the team evaluation in the villages appear to have a good understanding of methods for preventing diarrhea. They know that hand washing is important and when to do it but according to the LQAS, a very small percentage of women (.8%) are actually washing their hands before food preparation. The project staff indicated that the reason this was so low was because many women do not wash with water but use ash or other local products to clean their hands. According to the June O4 LQAS, 68.5 percent (target 50%) of children are receiving the same amount or more fluids when they are sick and 50 (target 50%) percent received the same amount or more food than usual. It was noted during the evaluation team field interviews that although many women understood the importance of giving fluids during 18 diarrhea and how to prepare ORS or home fluids, most preferred to treat their child with antibiotics because it stops the diarrhea. So although the project is making progress in diarrhea home management, it seems there is more to be done in the education of community members about exclusive breastfeeding, hand washing and the importance of fluids and food during diarrhea episodes. Pneumonia Case Management (10%): Goal: To reduce pneumonia associated mortality through prompt, appropriate and standardized pneumonia case management (IMCI), a sustainable supply of antibiotics at the CSComs, prompt recognition by relays of pneumonia signs (fever, fast breathing) and referral of suspected cases to the nearest health center and appropriate care-seeking behavior. As part of the project’s training program, all health facility staff have been trained in pneumonia case management through the IMCI program. In addition, community IMCI training has been beneficial in educating mothers and relays on recognition of danger signs. The mothers knowledge and recognition of fever and fast breathing and health care seeking behavior have significantly improved (68%) over baseline (30%) indicating that the training and education activities are effective. As a result the numbers of referrals to CSComs for fever are steadily increasing. Approximately 60 percent of health center staff have been trained to recognize and treat respiratory problems and know when to refer severe cases to the district hospital. As a result of project work with the CSComs and ASACOs, the evaluation team noted that all the pharmacies at the CSComs were stocked with the essential medicines and were able to appropriately account for and order medicines as required. There were no complaints about shortages. This is a definite improvement as the DIP noted that most of the CSComs in the district did not have adequate supplies of medicines to treat pneumonia or other diseases. Generally, the CS project has followed the work plan outlined in the DIP. There have been very few changes in the implementation. Perhaps the one problem that was not anticipated in the DIP concerns the anticipated support expected from PIDRIK (Projet de Developpement Rural Integre de Kita). PIDRIK is a coordinating entity for a five-year rural development project that originated out of the cotton industry and the needs of local cotton producers, but is now backed by the government of Mali. It is funded by the government of Mali, OPEC, Islamic Development Bank of Mali and CMDT, in the amount of 19 billion CFA. In the DIP it was stated that PIDRIK would be working in several development projects in the Kita area which would complement and support the CS project activities. Areas that PIDRIK would work in included literacy programs, agricultural development of lowland floodplains, road and bridge development, 50 new wells, and infrastructure. The CS project and CSRef expected them to build and equip 10 CSComs in accordance with the district health and development plan. Unfortunately, this has not occurred. Each year 19 PIDRIK has promised to build the 10 CSComs and they are saying they will do this in 2005 but to date this has not occurred. For this reason, the project had to increase Plan’s counterpart funding of the project and pull funds from some of the project areas in order to work with the first 8 CSComs. In 2005 they will begin working in 5 additional CSComs. At this point the project is not sure if it will be able to reach all 26 CSComs. If PIDRIK comes through in 2005 with the construction of 10 CSComs there is still a question as to whether they will also cover the costs for equipping the facilities. B.2. Cross-cutting Approaches B.2.a Community Mobilization In general the community mobilization activities are carried out by the ARAFD/C NGO animators. One animator is assigned to each project CSCom. Originally there were 13 animators but currently there are 5 who work with the current 5 project CSComs. An additional 5 have been hired to work in the new CSComs beginning in 2005. These animators work with village relays and village committees to conduct mobilization and education activities. Two relays are selected by each village to work with the project and they receive on-the-job training from the animators. Each CSCom is responsible for several villages of varying sizes. Each animator received training on the project intervention areas: malaria, diarrhea, pneumonia, immunization, community IMCI and MARP (survey methods). The animators also received finance and administration training to facilitate work with the ASACOs and help them organize and manage their budgets for the CSComs. They believe that the work they do with the ASACOs has really made a difference. During the field visits it was evident that the ASACOs were able to collect enough fees from clinic visits to cover the salaries of the CSCom personnel. This has been a major achievement in the region. In fact 2004 was the first year that many of the ASACOs were able to meet the salary requirements. They were also able to maintain supplies of essential medicines in all the CSComs, which had not been the case at the time of the DIP. The ASACOs will continue to be responsible for managing the CSComs after the project phases out. During the evaluation team visits, they seemed confident that they would be able to do this. They demonstrated a good level of understanding of the health problems facing the communities. The issue with the ASACOs is their own sustainability, they are sustained by the fees paid by the village members and the villages don’t always participate. Also the ASACO board changes every three years, so there is a need for re-training. The animators and relays have been very involved in mobilization of community groups, particularly for immunizations. In fact, the district’s dramatic increase in vaccination levels is partially credited to their mobilization efforts. They have formed and work with women’s groups and educate them about the intervention areas. Some of the animators have recognized that men are the principal decision makers and have begun to work with the men in the villages in order to address health issues. This has happened when they are working with the women and discover that they can’t afford to pay for the immunization or maternity cards. In this case the animators and relays arrange meetings with the men 20 in order to address this. They also invite the mayor to these meetings. In addition they have done general activities such as community diagnosis and problem identification and facilitated group consensus on solutions. In general the animators felt that the training had helped them become better mobilizers. They had learned new skills and techniques for working with the villages. The evaluation team found that the communities visited really liked the mobilization work of the relays and the ASACOs. Also the relays have done a good job of keeping track of village health status through the census/ registry activities, and informing the village committees about problems that need to be addressed, such as high numbers of malaria cases etc. A major issue is that there are not always enough relays particularly in the larger villages where two volunteers are not enough to cover all the inhabitants and the village chief may not wield enough influence to get the community to fully participate in health activities. There is a consensus among the animators and the evaluation team that it is too early to phase out of the community activities. There has been a delay in the phase out of about 6 months but many feel that it is still too early to phase out. They feel that the communities need more education inputs and follow-up activities. There have been some issues regarding selection of relays. Although the communities select the relays according to the needs of the project, some of the relays have not demonstrated the commitment necessary to carry out expected activities. In these cases, the animators have had to bring the problems to the attention of the VHCs or village chief and these then decide on replacements. The animators interviewed also think there may be problem in keeping the relays motivated. There is a lot of work expected of the volunteer relays who have to fill out 7 registries and keep them updated on a regular basis. This relay motivation issue is a major challenge for the animators. The relays do use the registries and reports and do follow-up on the vaccinations and pre-natal visits with the families. However, when the animator contracts weren’t renewed for 4 months between June- September of 2004, and the animators were not active in the villages, many relays stopped working. So the animators do not think they will continue once the project phases out of the CSComs. However, some villages have taken the work of the relays seriously and in order to support them have formed work groups to tend the relay’s crops while the relay works on health education in the village. Other CSComs are offering free health care for the relays and their families. The relays interviewed stated that they have received regular supervision and support from the animators (at least 2 visits during the past 3 months). They were pleased with the quality of supervision received. The major issue is that the animators, who are supervised by the project, are not sufficiently integrated with the CSCom staff who are supervised by the MOH. The evaluation team believes that for continuity, the relays need 21 to be supervised by the CSCom staff. Then once the project phases out, the CSCom staff will continue monitoring the relays. However, because the animators and relays have been organized by the project, they do not fall in the CSCom system of supervision and so far there has not been joint supervision. B.2.b. Communication for Behavior Change The key behavior change strategies employed by this project revolve around community organization. When the project begins in an area, ARAFD/C begins working with the villages to put a 5-member Village Health Committee (VHC) in place. The committee then works with them to select 2 people as relays. The VHCs and relays are then trained in all the project intervention areas by the animators. This on-the-job training prepares these two entities to do mobilization and IEC activities, home visits, and community education sessions with the video projectors. Then the second part of the strategy is that through the trained relays and VHCs, skills are transferred to the communities through women’s groups and other community forums. Unfortunately the planned development and purchase of IEC materials has been delayed. The funds were used on other project activities. At the beginning of the project ARAFD/C and the training coordinator were told that materials would be provided by the project but unanticipated expenses caused the project to divert funding to other areas. To date the project has provided video players and televisions to each of the CSComs but they lack videos for them. Also the ASACOs have to provide fuel for the generator in order to run the TV/video. The NGO has asked the project for picture boxes, flip charts and posters. The methods employed for BCC include group discussion, videos, home visits and meetings/ assemblies. So far the project has not developed more entertaining formats such as skits, role plays, songs or story telling. For the most part the messages that are delivered are technically accurate. The problem is that the messages may not be reaching everyone. To address this, the project began signing agreements with local radio stations in 2004 and the animators have been invited to speak about health issues, though this is not done on a regular basis. In some instances ARAFD/C has purchased time on local radio or TV stations. In order to influence behavior change, the project decided to locate the animators in the CSCom village. They felt that the animator was likely to be more influential if they were living in the areas. Behavior change is measured generally through the LQAS monitoring system used by the project. They have been able to measure how many children are vaccinated, how many pregnant mothers are seeking prenatal care, use of home-based fluids for diarrhea etc. After the LQAS, all the animators are convened by the project as well as the village chiefs and ASACO members, and informed of the results. In addition, the registries maintained by the relays keep current information about health problems of children under five and women of fertile ages. They also record information about referrals to the 22 HCs for care and follow-up activities back in the communities. The follow-up notes give information about how well the mother is following the doctor’s recommendations. The registries kept by the relays have proven to be a useful means for motivating communities to address health problems. For example, last year the project became aware of a diarrhea problem in one of the CSComs through the relays who informed the animators of diarrhea related deaths. When the animator realized what had happened, he decided to educate all the relays in the CSCom about recognition of diarrhea danger signs, home management and referral. The evaluation team found that there is evidence of increased attendance at the CSCom clinics by the project villages. There is a shared common understanding of local health problems and increased awareness on the part of the villagers about these problems. There has been an increase in the numbers of women attending prenatal clinics and being vaccinated for TT. There is increased use of impregnated mosquito nets, particularly by young children and pregnant women. The main weakness identified by the team was that there was not enough involvement of men or other decision makers (grandmothers or mothers-in-law) in the management of women’s and children’s health. B.2.c. Capacity Building Approach B.2.c.i. Strengthening the PVO Organization The organizational capacity building indicators stated in the DIP are directed at partner organizations and not Plan. However, the CS project has had a beneficial effect on Plan/ Mali. For example, the CS project is being used as a model for Plan’s program Unit Offices throughout Mali which all have health officers who are implementing child health activities. In fact three of the health officers were members of the CS evaluation team. According to these officers, they are focusing on the same interventions as the CS project. For the Kita Program Unit area, the director commented that the CS project has allowed him to expand beyond the Plan villages where they have foster children. He sees that the project has allowed them to better organize and strengthen these additional communities. He stated that the originality of the project approach is helping Plan to strengthen the Mali decentralization system in his region. He sees the community based organizations taking on their roles in health care. No longer is it just the health worker who deals with community health but now it is the whole community. So when the health worker leaves, the system will continue. He also sees great value in the use of community based data and he is encouraging the DHO to consider extending this practice through out Kita district as well as other areas where Plan works. Plan International has been strengthened by this CS grant in several ways. The headquarters has just gone through a change in technical backstopping staff, and this evaluation provides a training ground for the new staff member. Plan HQ will be able to transfer technical and managerial skills to other child survival projects that are currently 23 implemented by Plan globally as well as provide technical guidance and comments on development of Plan private funded CS proposals. B.2.c.ii Strengthening Local Partner Organizations B.2.iii Health Facilities Strengthening A major focus of the project’s capacity building efforts have been on the CSComs and district health system. The project conducted a Health Facilities Assessment of the initial 8 CSComs during the first year. This was coordinated by the DHO. Each year the project receives a list from the CSComs regarding materials and equipment needed. The project staff then visits the CSComs and identifies the needed items that are within the capacity of the project. During this initial assessment, they also evaluated the training needs of the Chef du Post, the Matrons and other staff. After this first experience, the project realized that all staff needed upgraded training and that all the CSComs had the same staff training needs. The IMCI training met the needs. The project has received a request and bought new equipment for the 5 new CSComs that it will be entering in 2005. The CSRef will decide how the equipment is to be distributed among these centers. The project believes that in order to build MOH capacity, it is necessary to go one step at a time. Each partner has a copy of the DIP that has been translated into French and it outlines each partner’s role and responsibility. Each partner has received training in the health sector policies, good governance and decentralization. The district MOH requests training from the project on a yearly basis. The project also has on-going training activities with the Ngo staff (animators). The Plan staff works closely with the NGO animators to solve problems when they arise. The project also organizes quarterly partner coordination meetings. These include the CSRef, the NGO (ARAFD/C), Plan and the ASACOs. For each ASACO, the 2 relays and the VHC are also invited to attend. In addition there are meetings held at the health area chaired by the ASACOs who make their reports discussing activities, problems, and solutions. They also review problems from the previous quarter and plan what is to be addressed during the next quarter. The project is usually invited to these. The DHO commented that he would like to see more coordination between project staff and district health staff at the CSCom/ area level. The evaluation team also identified a need to have the community relays who are currently supervised by the project animators, also be supervised by the MOH Chef du Post at the CSComs. Currently the MOH does not have authority over the relays. However, the proposed arrangement, which would require an agreement between the CSRef and the project, would assure that the relay work would continue after the animators leave the CSCom. Everyone agreed that it would be good to have the project staff, including the animators, attend the quarterly CSRef meetings. That way everyone would be on the same page when planning health activities in the district and in the specific health areas. The DHO felt that it was important for the CDPs and the animators from the same areas to be working together. He also felt that this was a way to involve the relays (through the animators who supervise them) and bring them into the CSCom supervision system. 24 In general, the project has assured that the CSComs have functioning stocks of medicines that are replenished on a regular basis. So far the project has assured this in the 8 CSComs that they are working in. In addition, it is obvious that there are increasing numbers of satisfied users because the number of users of services are increasing at the CSCom health centers. Regarding the ASACOs, the evaluation team found that they had a good understanding of their roles and responsibilities. They assured that the health centers had and maintained appropriate supplies and equipment, such as cold-chain. They assured good vaccination levels in their areas. There were good management systems in place at the centers that showed well organized accounting documents, account balances and that financial reports are made at the general assemblies. There was evidence of regular meetings with area partners (CSRef, NGO and Plan). There was also evidence of statutes and by-laws. On a less positive note, the ASACOs visited still do not have their legal agreements with the Government. There was also limited representation of women on the committees. Some of the committees were also having difficulty collecting the financial fees that are to be paid by the villages for the support of the ASACOs. To the project staff, one of the biggest challenges is getting the ASACOs to the point where they will take on the total management of the CSCom health system. They would like to see the ASACOs take on a stronger leadership role in making decisions about how to improve clinic functioning and see each ASACO solve its own problems. The evaluation team thought it was a positive sign that when asked about sustainability, one of the ASACOs (Sangdianbo) asked Plan to help them identify a new partner to help them continue after the CS project phases out. According to project staff, before the CS project the ASACOs would not have been interested in financial planning or technical assistance. The project has also worked to strengthen the NGO consortium ARAFD/C. An assessment was done of the lead NGO, ARAFD, by Plan when they first started working with them in the 1990s. Since they are the only agency working in public health Kita, Plan asked them to work in consortium with another NGO on this project. Initially, 13 animators were hired. In July of 2004, 8 were phased out and 5 remained. There have been problems in the contracting of the animators. The project had under budgeted the amount of funds needed to subcontract ARAFD/C to take care of all the IEC and community mobilization activities, including hire the animators. One of the problems was that Plan only gave ARAFD/C yearly contracts. As a result, ARAFD/C only hired the animators for the length of time they had in their contracts. Since the NGO is completely dependent on Plan or other donors to pay staff, they could not maintain the staff if the contracts were not renewed immediately. This occurred in 2004 when the project had exhausted the sub-contract line item and it took several months for Plan to help them modify the budget and add funds for the subcontract. As a result, all the animators were laid off for 4 months. This created a huge lapse in field activities. Data wasn’t collected from the villages, people were not educated, mobilizations were delayed etc. According to the PUM, this problem has been resolved and the new animators will be given 2 year contracts that will last them through the project. They will also be paid for any time 25 worked when the contract was not viable. The project will be working with another NGO, AMOCO, in the 5 new health areas that it will phase into in 2005. According to the NGO coordinator, the organization and staff have benefited greatly from all the training and support that they’ve received from the project. The NGO received management training in Bamako and technical training in Kita. The coordinator says that this has really enhanced the skills of the staff and that after the project ends, they will be able to market themselves as skilled BCC and mobilization experts who know how to work with communities in the region. B.2.iv. Health Worker Performance As mentioned in the above section, when the project conducted its health facility assessment (HFA), they also assessed the skills of the health workers in the various health areas where the project planned to work. From the HFA results, they knew that there was a big need for skills and found it necessary to train all the staff at the CSComs. The project staff realized that only providing training on the four health interventions would not be enough. They realized that the IMCI training was more comprehensive and would be the most practical, given the needs identified. Because the IMCI approach trains clinicians to conduct a thorough diagnostic when treating a child for any ailment, there was a better chance of catching any additional health problems or needs (such as immunizations or nutrition counseling) than would occur if the clinician only treated the most immediate problem. Also the IMCI training curriculum and support materials are so user-friendly that the project felt this would be the most helpful training for the health facility staffs. Following this decision, the project training coordinator as well as a doctor from the CSRef were selected to be trained as IMCI trainers. Once trained, these two trainers then proceeded to train the CSCom health agents in Kita district. The staff at the CSComs and the project staff all commented that the IMCI training has been the most useful for them. They believe they are making fewer errors in diagnosis because the materials are so helpful in leading them through the diagnostic process. After training has been completed, the CSRef staff visits the sites and follows-up with the health agents on site. Also if project staff notice any problems or lack of skills at the field site, they discuss it with the CSRef who then follows up in the field. In fact this occurred at one site (Sanguana). The project had provided materials and support for vaccination activities in the area but the CDP was not doing the immunizations in the villages. When the project staff complained to the DHO about this, he followed up and replaced the health agent with someone who was more enthusiastic about visiting the villages. During the team evaluation of the CSCom sites, it was noted that the health agents had received a lot of training by the project but the training that they appreciated the most was the IMCI training. The agents were functioning well, had a good understanding of their mission, were able to use the SIS effectively and were well integrated with the communities around them. As noted in other sections of this report, the only weaknesses identified were that the referrals from the village relays were not recorded in the CSCom information registries and the relays were not receiving enough supervision from the CDPs. 26 B.2.c.v. Training As can be seen in the training plan located in the annex, the project has conducted several training programs at all different levels, for Plan project staff, NGO animators, district and area health staffs, vaccinators, ASACO members, relays, and village health committee members. However, the key training intervention has been the clinical IMCI training described above. The project is now organizing clinical IMCI training for the 5 new health areas that they will be phasing into in 2005. In addition to clinical IMCI, the project has also introduced community IMCI. The community training module was developed by UNICEF for Mali. Again the CS project training coordinator was trained as a trainer for community IMCI along with Plan Mali’s National Coordinator for Child Survival. In addition to Kita, they have collaborated in providing this training in Plan’s other PUs. The staff believe that the community IMCI has been critical in preventing childhood mortality. Through the training given to animators and relays, mothers in the villages are able to start treating sick children before going to the health centers. They are also made aware of the need to go to the center and because the quality of care has been improved, receive adequate treatment. Although communications training was provided to Plan and Ngo staff in 2004, there is still a need to provide more BCC training to the animators and relays. They need more skills and techniques for communicating with the communities. This training is needed for the new animators who will be working in the new Health Areas and refresher training is needed for the existing animators. The training coordinator also believes that the project needs more time to bring about behavior change in these villages. He thinks it is too early to phase out of the villages and that the relays and VHCs aren’t yet ready to manage the process on their own. Two staff were also given communications training in Belgium. However, these were not the staff directly responsible for communications activities. The training Coordinator also recognizes the need for IEC materials. He conducted an assessment of the kinds of materials needed and then went to Bamako and met with the government agency in charge of IEC materials development. He identified what was needed and made arrangements to buy them. However when he returned to Kita, he learned that the funds that were to be used for purchasing materials had been diverted to other priorities. Despite the problems, he does believe that the video players that he was able to buy for 6 CSComs and videos have been very useful as discussion tools. As evidence that the training has been useful, the training coordinator notes that the ASACOs now have clear financial accounting books and bank books. This was not the case at the start of the project. He also noted a definite change since the baseline assessments of the CSComs. The registries and support materials are clearer. They are able to conduct social mobilization activities, they can now manage the CSCom funds, and there are much better relationships between the ASACOs and the health staffs. 27 Other evidence of training impact is found in the numerous well-organized record books kept by the village relays. In addition, the community mobilization training has resulted in increased attendance at clinics. At the time of the baseline assessment, only 20 percent of prenatal patients visited the health centers for care, now 60 percent are being seen at the facilities or in outreach visits. The same is true for the number of children vaccinated. At the time of the baseline only 25-30% of children were fully vaccinated, now that is closer to 70 percent. The training coordinator noted that he is getting numerous requests for training from villages and the government health offices. He and the project cannot provide all the training requested but he thinks that some things should be prioritized. For example, there is a need to train the village health committees and relays on how to reimpregnate their mosquito nets. This will require additional resources from donors but he believes it is critical if the project does not want to lose the edge it has gained in combating childhood malaria. B.2.d. Sustainability Strategy Progress towards objectives: 1) stimulate community participation and ownership: There appears to be good progress in the area of participation. Each of the CSCom villages have established village health committees and have relays working on the health issues and documenting community health status. Although the LQAS did not measure sustainability indicators, the mid-term evaluation team found that the villages visited had a very positive impression of the CS activities and were participating in education sessions and village health committee events. The mothers group participants stated that the activities had increased their knowledge about child health, particularly immunizations. Although the VHCs have become increasingly active in health promotion, there is still a sense that it is too early to phase out and that more training and support is needed in the communities. 2) increase the capacity of the ASACOs in management, advocacy and as community representatives: 3) ensure health services are financially sustained through a cost-recovery system in which the community participates through the use of the services. As noted above, there has been a tremendous increase in the ASACO management skills. They are keeping organized accounting books and bank books, they are able to mobilize the communities for health and they manage the CSCom funds. They have a good understanding of their roles and responsibilities and have their internal statutes and regulations. Other evidence of their effectiveness is that the in all the CSComs visited, the staff are fully supported through the funds brought in for services. This had not been the case before the CS project. Obviously this increased use of services can be attributed to improved quality of care helped by the CS project supported training, and by the increased community mobilization activities that are encouraged by the ASACOs, VHCs and relays. Also the clinics all have functioning essential drug supply systems. None of the pharmacies had had stockouts during the past year. 28 To date, the project has phased out of 5 CSComs and is planning to phase out of more during the next year. There is concern that the VHCs and relays are not yet ready to take over health promotion and mobilization activities without the support of the animators. As evidence of this, the animators noted that when the NGO contract was not renewed on time and the animators were laid off, the relays stopped working. To address this, the CS project is planning to conduct an OR study in the communities that have been phased out to see the status of planned health activities. The mid-term evaluation team believes that in order for activities to be sustained after phase out, the Chefs du Post at the CSComs need to take over supervision of the village relays. This has been a problem in the past because the animators, who supervise the relays, work for the CS project and the CDP works for the MOH. These two do not participate nor are they supervised through the same health system. In some instances the animators have a good working relationship with the CDP but that is not always the case and because the two work with different systems, there has been no pressure for them to work together. The DHO would like the animators to be part of the district health team and have them attend quarterly district health meetings. These issues were discussed during the team evaluation and it was decided that the project and the district health office (CSRef) will sign a memorandum of understanding in which the CDP will be designated as the CSCom supervisor for the village relays. The animators will still provide technical supervision as long as the project is in the area but the long term supervisor will be the CDP. The DHO would like to see quarterly supervision sessions at the CSComs wherein the CSRef and CS project staff from Kita would also attend. He thought that if there were enough agencies involved then they could also invite the mayor and try to get his involvement in the work of the VHCs and relays. Since the mayors also have funds for community activities this was seen as a strategy to garner more support for the relays. Despite the concerns about the sustainability of activities after phasing out, the ASACOs interviewed all stated that they felt confident in their abilities to maintain social mobilization activities and continue managing the CSComs successfully. There is also an issue in that the fees that are supposed to be paid by the area community members are not being paid regularly. Another problem that has affected project viability is that when the project was originally planned, it was assumed that PIDRIK would cover the cost of building and equipping 10 CSComs in Kita district. As discussed in section B.1. of this report, so far this has not happened. Plan and the CSRef have had to work around this problem and it has reduced their resources that were meant for other activities. Fortunately Plan has been able to meet some of the requirements with private funds. 2.C. Program Management C.1 Planning Project planning occurs at quarterly Coordinating Committee meetings in which all partners participate, the ASACO, CSRef, Plan, NGOs, and the VHCs. The project has 29 generally followed the DIP workplan, of which all partners have copies. The work plan is reviewed each trimester by the partners and used as part of the quarterly planning activities. Some items in the workplan have not been completed such as the development of IEC materials. Also after the first year, the HFAs have not been done at all the CSComs because the project realizes that the health staffs at these facilities all need IMCI training and other assessments are used to project equipment needs. But generally the work plan is used as a guide for project planning. Given that all partners have copies of the DIP including the monitoring and evaluation plan, and discuss the activities at regular planning meetings, it is clear that they have a good understanding of project goals and objectives. It was clear from the LQAS that most of the year 3 objectives have been reached or surpassed. Likewise from interviews conducted with field staff, headquarters staff, and the communities it is clear that they also understand the purpose of the project and appreciate the benefits they are receiving from it. Each month there is a regular meeting of project staff to evaluate progress and plan activities for the next month. Whatever planned activities have not been completed during the month are then programmed for the next month. The project makes regular use of monitoring data. The LQAS is conducted twice a year and allows the project to determine the level of attainment of objectives. It also permits the project to see if work is reaching its objectives in the different zones. The results are compared with the SIS data collected at the CSComs in order to get a complete picture and validate findings. If they find, for example, that vaccination rates are low in a particular zone, then the staff visit the communities and talk about it at village meetings. C.2. Staff Training The project staff participated in two types of training. When the project began, the project director asked each staff member to identify what weaknesses they had that needed to be strengthened in order to perform their jobs. For example the data manager came to the project from an environmental health background. In order to strengthen his data management skills, the project sent him to the MOH for a month to receive training from the HIS department. Likewise the monitoring and evaluation coordinator did not have experience with the LQAS methodology. So the project director arranged for him to attend a regional LQAS training program in Rwanda. The training coordinator has a general medicine background and does not have public health training. So the project arranged for him to attend a four month epidemiology course offered at the university in Bamako during the rainy season. The staff’s work is monitored by the project director. Because the project director is a physician who completed an MPH with a specialty in monitoring and evaluation, he has been able to follow-up with the training and assist with the required activities. For example, the monitoring and evaluation coordinator leads the bi-annual LQAS surveys but the project coordinator assists him with the analysis by using SPSS (a tool that the 30 M&E coordinator is still learning). In addition, each staff member writes a report of any training they receive. It seems that the staff training has been a worthwhile investment as it has allowed the staff to acquire pertinent skills for their jobs. In addition, the training was done locally or regionally and was not overly expensive. C.3. Supervision of Program Staff The staff activities and performance are monitored by the coordinator during their regular project planning meetings which occur weekly, monthly and quarterly. Because the staff is small it is easy to keep track of personnel and activities. Supervision of the animators is the responsibility of the NGO staff but the NGO sends the monthly plan of activities to the CS project. The CS staff then coordinates their field visits with those of the animators and are able to provide additional supervision of relays or ASACO members as required. C.4. Human Resources and Staff Management The Project Coordinator works closely with two Assistant Program Coordinators and the HIS Manager to provide direction to the CS team. He also receives input from Plan’s National Child Survival Coordinator and the Plan/Mali country office. The project team is composed of the Project Coordinator, the Assistant Program Coordinator for Training, who oversees the CSCom health personnel and partner NGO staff; the Assistant Program Coordinator for Monitoring and Evaluation, who oversees M&E for the project; and the Health Information Systems Manager, who is responsible for the design and implementation of the HIS (SIS) and basic surveillance. The Project Coordinator oversees all field staff. The two assistant project coordinators and the HIS manager report to the project coordinator. In addition the project has administrative support staff and a driver. All the positions are filled and there have not been changes in the personnel plan since the DIP was written. The coordinator was the only staff who was hired from outside Plan. All positions have Plan job descriptions and are managed as core Plan staff members. Each staff member is evaluated at the end of the year in accordance with the requirements of their job descriptions. The NGO animators are hired for specific community mobilization tasks and also have specific job descriptions by which they are evaluated. In general the morale among the Plan staff is high. It is a small group of staff who work together with good coordination on a regular basis. There has been more of a problem with the NGO staff morale. This is because the Ngo has been given yearly contracts by Plan that need to be re negotiated every year and there have been delays in re-contracting of staff. There were also budgetary issues for funding of NGO activities that had to be resolved. This also took several months and resulted in laying off the animators for 4 months. The lack of animator continuity has created problems in the stability of village relays and community mobilization activities. The Plan PU manager stated that this 31 problem is being resolved. The project is going to sign a two year contract with the two NGO partners so the animators will have stability. Also the animators will be paid for time worked without a contract. In general, among the Plan staff there has been no staff turnover. However because of the budget problems and animator lay-offs, there has been turnover among the animators. Though they have been able to hire some of them back for the new contracts. Except for the project coordinator all the Plan staff are permanent Plan staff who were brought into the project from other Plan projects. Therefore they are guaranteed other jobs after the project ends either in the Kita area or in another PU in Mali. The PU manager stated that even though the Project Coordinator was not permanent Plan staff that there would be a place for him at Kita if he wanted to stay. Since the CS project has started Plan has installed Health Coordinator positions in each of the country PUs. Kita does not have one so when the project ends, he would be in a position to assume this role. C.5. Financial Management The project coordinator and the accountant reviewed the financial reports and then submitted monthly financial reports to the Operations Support Manager in the national office via the Program Unit Manager. The program tracked costs incurred using the already established financial system of Plan/Mali. Expenditures were broken down using specific codes for labor, equipment, supplies, and facilities. Project expenditure reports were sent directly from Plan/Mali to the US office of Plan where they were reviewed for USAID compliance and submitted to USAID. Plan’s corporate general ledger system assigns a project ID number to enable accurate tracking of project expenditures related to each project. Plan’s organizational finance system also enables Plan/USA to separately track donations from USAID and non-USAID sources. After the US office submitted reports to USAID, and received the re-imbursement, funding transfers were made on a monthly basis from the US office to the Child Survival project to reimburse project expenditures. C. 6 Logistics There is one project vehicle but if necessary the project makes use of other vehicles in the PU. The project has not had problems with supplies and equipment. The project prepared their budgets for purchase of major equipment items. These have been delivered to the existing CSComs. The MOH and ASACOs are supplying the smaller supply items and instruments. The biggest logistics challenges will be supplying equipment for the new CSComs. It will require a budget modification and funds in order to cover the costs. Originally PIDRIK and Plan were going to construct and equip 10 CSComs but now the money that Plan had was spent to construct and equip the project’s original 8 CSComs and funding is no longer available to equip more CSComs. Another issue concerns whether there will be 32 sufficient vehicles for the work in the 5 new CSComs which are located in the northern part of the region and will require extensive travel. Because the existing vehicle will still be needed for project work in the southern part of the district, there may be a need for another vehicle to handle all the supervisory and other project demands in the north. C.7. Information Management The program’s community-based information management system (SIS) has been in place since February 2003. The data is recorded by the village relays in 7 record books including family census, vaccinations, prenatal care, childhood illnesses, deaths and epidemics. This information is collected by the animators on a monthly basis and given to the project Information Systems manager. There have been some gaps in the information due to the laying off of the animators. Also the animators for the new CSComs have been recruited but are being trained and are not yet functioning at their new posts. At the time of the mid-term evaluation the animators were busy updating records from the period when they had been laid off. Once the Information Manager receives these reports, he checks the information with the data collected by the CSRef (district) from their health centers. If there is a discrepancy in the figures, he then visits the particular CSCom or village to check the relay reports and registries and make appropriate changes. The project data is also validated against regional data which is reported every quarter. This community-based data has been very useful for informing communities about their own health issues. The project staff review health indicators on a village-by-village basis. If there are problems, such as a drop in vaccination rates, the project then brings it to the VHC’s attention and works with them to discover what the problem is and correct it. For example in the Sanjangbougu CScom which includes 13 villages, the information manager noted that there were 7 child deaths in May/ June 2004. So they visited the area to see if vaccinations were up to date or what the cause was. However when they visited, the parents told them that the children had died from malaria. The children had been vaccinated and the parents said they died of fever. The staff wondered about this because this time period is early for malaria but since the children were not taken to the health center, they could not verify the cause. However, the staff used this example as an opportunity to educate the villagers about the need to keep the area clean and free of standing water even if it is not malaria season. They also instructed them to use bed nets. In another area, Kassaro, the project staff found that the TT vaccination rate of WRA dropped from 60% to 40%. When the staff followed up to find out what had happened, it turned out that rumors had been spread that TT caused sterility. Fortunately, the staff were able to call a village meeting before the problem became more pronounced and correct the misunderstandings. The information manager also works with the LQAS surveys, which are done twice a year and he compares these results with the SIS findings. He felt confident about the project data. He indicated that the LQAS is 90 percent accurate and the SIS is cross 33 checked with the district data. Also the LQAS is compared with the monthly SIS data. The project did conduct a KPC survey at the beginning of the project and they did a Rapid Catch in November 2004. C.8. Technical and Administrative Support The project has received regular assistance from the Plan headquarters staff. The Child Survival Program Coordinator has worked with them during the proposal process, the launch of the project, the DIP process and the mid-term evaluation. He also reviews and backstops the KPC and LQAS survey process. The previous HQ Health Assistant visited in 2003 to help iron out some of the programming and budget issues. The new CS Health Assistant has been coordinating with the project regarding mid-term evaluation and LQAS results. The HQ contracts specialist also visited in 2003 and 2004 in order to help the project figure out budget modifications to cover the NGO partner expenses and address other unanticipated expenses. In addition Plan Mali has exhibited a lot of interest in the project. The Country Director has visited twice a year along with the National CS Advisor. In addition the Plan Executive Director visited the project in 2003. To date the project has not received other outside technical assistance. However, the evaluation interviews indicated that the project might benefit from technical assistance in the development of a project wide BCC strategy and follow-up as well as appropriate use of IEC materials. The Plan Mali Director also advocated technical assistance for participatory IEC materials development. Also, although breastfeeding techniques and counseling are not a specific project intervention area, it is included as part of the diarrhea control activities and IMCI. Given the low exclusive breastfeeding rates and in consideration of the CDD objectives, the project may want to consider some additional technical assistance and community training in this area. As mentioned, the project is also planning an evaluation of the CSComs that the project has phased out of, in order to see how sustainable the interventions have been and if the phase out is as premature as many of the staff and community actors have indicated. The project coordinator indicated that technical assistance would be helpful for the design of this study. E. Conclusions and Recommendations Conclusions Successes · As noted in the LQAS results, the project is making good progress in achieving its projected results. It has already either met or surpassed most of its year 3 objectives. Perhaps the biggest success is the achievement of almost 70 percent of children who are fully vaccinated. The recognition of this achievement on the part of the MOH is a huge indication of the project’s success. Prior to project initiation, the MOH had targeted Kita District as having the worst immunization levels in the country. Now they are sending a team to verify the results and are using it as a model for other districts. 34 · The project has also achieved tremendous success in mobilizing communities to use health services. The relays, animators and VHCs have been very effective in mobilizing their communities to seek care and preventive services, especially immunizations and prenatal care. The number of referrals by village relays increased from 27 in 2003 to 118 in 2004. The project staff have also been innovative in helping health center staff bring services to the villages. This has been by supporting the work of the CSCom vaccination staff and by encouraging the matrons to accompany them and offer prenatal consultations in the villages at the same time. · Another contribution has been the strengthening of the management skills of the ASACOs. Every ASACO visited had its accounting and bank books in order. Also, every ASACO was able to meet its financial obligations of supporting the salaries of the CSCom staffs. Given the poverty of the region, this is a commendable achievement and may be the only country/ region where local government health services are completely supported by community constituents. · The project has done a good job of developing a community-based health information system. There is good evidence that the information is being used by the communities and CSComs for planning and for making health care decisions. · The IMCI training that was instigated by the project, has greatly improved the quality of care at the CScoms. The health staff are now able to perform broad diagnosis of child illness and provide counseling for caregivers. This improved quality of care seems to be partially responsible for the increase in visits for services. · Plan has done a good job of strengthening local partner technical capacities. The local NGO staff now possess health and mobilization skills that they are able to use in other areas. · Despite the unavailability of the originally planned support, the project has been able to meet all the designated equipment needs for the project CSComs and provide material support for outreach activities. Challenges · One of the biggest issues is the sustainability of program inputs and whether the phase in/phase out timeline is too short. The project has achieved tremendous success in getting people to use preventive and curative services. All the project inputs such as IMCI training of providers, formation of relays and VHCs and corresponding community mobilization activities, and strengthening of the ASACOs so they can fulfill their management mandate, have contributed to the increase use of services and the self-sufficiency of the CSComs. The question is whether the ASACOs, VHCs and relays are sufficiently capable at this point to continue CSCom support activities on their own. There is concern that the organizational support structures are not in place yet for them to do this. 35 · It is clear that the relays need a lot of motivation to continue their activities. They are asked to maintain 7 registries, organize meetings, follow-up on referrals, mobilize villages for immunizations, and generally address health issues in the villages. The project animators who live in the area are doing a good job of training and motivating them but once the project phases out, they will no longer do this. · The project has had to work around the fact that PIDRIK has not come through with the promised constructing and equipping of 10 CSComs for the district health system. Plan and the CSRef have been able to come up with some extra resources but it is unclear whether they will be able to fully equip and construct all 26 CSComs as planned. · The project has done a good job of working with community HC and relays and training them in areas of community mobilization and in the health intervention areas. However, there is still work to be done in the area of IEC/ BCC. Many villagers are still unaware of proper care related to CDD, such as exclusive breastfeeding and handwashing. Likewise some villages are very aware of malaria prevention and use of bednets while others are not as attuned. The project lacks an overarching BCC strategy that targets specific behaviors and includes specific communication interventions. The project is also lacking in IEC materials that the relays and animators could use to reinforce health messages. In the DIP the project had planned to develop materials but was unable to with the budget adjustments. · Given the difficult roads and the large distances to be traveled, particularly with the 5 new CSComs that are located at the other end of the district, the project staff will need significant travel time and vehicle support for supervision and support activities. This will infringe on the vehicle support already required for on-going monitoring activities in the current project areas. In addition, the motorbikes that the animators use are almost worn out and require significant repairs. Recommendations · The project should conduct an evaluation of the CSCom areas that it has phased out of to see how well the health agents, ASACOs, VHCs and relays are doing in managing the health activities without project support. If there are problems then the project should take steps to address them in a timely manner. · In order to strengthen the sustainability of the relays, the evaluation team recommended that the Chef du Post become the supervisor of the relays and their activities. This could be done at the time of regular outreach visits to the villages and/or through monthly meetings of all the relays at the CSCom. There are some administrative arrangements that need to be made in order for the CDP to supervise the village relays but once this is done there would be a system in place 36 for on-going supervision of village health activities. This could start by having the CSCom team and the animators conduct joint team supervisions in the surrounding villages. Then after the animators leave, the supervision system would be in place. This would strengthen the sustainability of relay and village health committee activities. The on-going supervision of the relays would also strengthen their motivation, particularly if they were seen as working with the chef du Post who is very respected in the area. The evaluation team also recommended that the project staff and district health staff attend quarterly supervision meetings done at the CSComs. Their presence would encourage the other actors (ASACOs, CSCom staff, VHCs and relays) to attend and get involved in the process. They also pointed out that if the district staff were there, there would be more likelihood that the mayors would participate in the relay supervision meetings. This is important given that the mayors have access to their own source of funds and would be more likely to spend them on relay activities if they were involved in their management. · If PIDRIK comes through with funding this year for the construction of the 10 CSComs that they are promising, then the project needs to work with them and the MOH to see how to equip the facilities and carry out the corresponding community activities with the limited project resources. If PIDRIK does not come through with the anticipated funding, then the project needs to decide how many more CSComs it can realistically work with by the end of the project. Once the CS project has the results of the CSCom phase-out evaluation discussed above, it will be able to decide whether it should put more resources into new CSCom areas or if it should devote them to strengthening the sustainability of the old ones. · Recommend that the project contract a short term BCC consultant to help them design a strategy that includes behavior change targets and strategies as well as awareness raising communications activities. It would be useful to employ more entertaining cultural formats that could be used to create interest in the villages such as stories, songs, street theater, and other methods. · The CS project needs to follow through on its original plan to acquire and/or develop IEC materials that are appropriate for the target audiences. The health messages need to be reinforced through support materials. Also the materials would help the relays with their educational tasks. · If resources permit, the project should consider acquiring another vehicle for the work in the new CSComs in the far northern section of the district. It should also consider replacing some of the worn out motorbikes being used by the animators. · In order to strengthen some of the intervention activities, the project should consider providing breastfeeding technical assistance to improve technique and promote exclusive breastfeeding in the communities. 37 F. Results Highlight Perhaps the biggest achievement of the project to date has been its high level of vaccination coverage. With almost 70 percent of the children in the district completely immunized, the project is close to achieving its overall objective of 80 percent. The Kita district was selected for this project because it had the lowest immunization levels in Mali and was of great concern to the MOH. If they do achieve 80 percent coverage, Kita will be the only district in Mali to do so. To achieve this record level, the project has gone beyond working in the designated 13 CSComs (health center areas). Because the District Health System does not have CSComs in 30 areas, the project agreed to support additional vaccination posts and support for mobile teams and CBOs in these 30 areas. The project covered all costs related to continuous operation of the cold-chain system and supported maintenance and repair of cold-chain logistics. The MOH is sending a team to validate the LQAS findings in early 2005. If their results concur with the LQAS findings, Kita will be the district with the highest immunization rates in Mali and the MOH will be using it as a model. I ndicators Project Year 3 % Ob jective LQAS Resu lts March 2003 LQAS Resu lts November 2003 LQAS Resu lts June 2004 LQAS Resu lts Novembe r 2004 Fu ll I mmun ization Coverage (F I C) 65 33 . 55 50. 00 69.54 70.00 According to the DIP, the MOH rate for fully vaccinated children in the Kayes region in 2001 was 13.5 percent. Also only 40 percent of women had received two doses of anti￾tetanus vaccine during their last pregnancy. As we can see, the rate is now 78 percent in the Kita district. Issues also revolved around availability of vaccination cards, now close to 91 percent of children have them. The project followed the strategies planned in the DIP in order to improve immunization coverage. Working with the MOH and ASACOS, they established the monthly outreach services with associated communities, supported bi-annual measles campaigns, ensured that quality vaccines were administered and provided training to CSCom health workers, particularly the vaccination staff. This support given to District MOH to achieve greater immunization coverage contributes directly to CSHGP Intermediate Result 1 and more indirectly to Intermediate Results 2 and 3. 38 III. Action Plan Recommendation Steps that need to be taken to effectively address recommendation Actor who will be responsible for each step Timeline. By when each step shall have been accomplished The project should conduct an evaluation of the CSCom areas that it has phased out of to see how well the health agents, ASACOs, VHCs and relays are doing in managing the health activities without project support. If there are problems then the project should take steps to address them in a timely manner. 1) Develop TOR of the survey 2) Identify the institution, which will carry out the survey and negotiate a contract for carrying out the survey 3) Organize a workshop for the validation of the protocol before the starting of the survey. 1) Dr Bagayoko PSE Coordinator 2) Dr Aguibou Coulibaly 3) Dr Bagayoko January 05 March 05 April 05 2. In order to strengthen the sustainability of the relays, the evaluation team recommended that the Chef du Post become the supervisor of the relays and their activities. This could be done at the time of regular outreach visits to the villages and/or through monthly meetings of all the relays at the CSCom. There are some administrative arrangements that need to be made in order for the CDP to supervise the village relays but once this is done there would be a system in place for on-going supervision of village health activities. This could start by having the CSCom team and the animators conduct joint team 1) Make administrative arrangements (making memorandum of understanding, entering of the supervision responsibility in the job description of heads of post) 2)Define and negotiate the implementation of a mechanism of joint supervision of health activities in villages with a schedule of implementation and budget 1) Head of district health center 2) Elie Coulibaly Responsible for SE Monitoring and Evaluation January 05 January 05 February 05 39 supervisions in the surrounding villages. Then after the animators leave, the supervision system would be in place. This would strengthen the sustainability of relay and village health committee activities. The on￾going supervision of the relays would also strengthen their motivation, particularly if they were seen as working with the chef du Post who is very respected in the area. The evaluation team also recommended that the project staff and district health staff attend quarterly supervision meetings done at the CSComs. Their presence would encourage the other actors (ASACOs, CSCom staff, VHCs and relays) to attend and get involved in the process. They also pointed out that if the district staff were there, there would be more likelihood that the mayors would participate in the relay supervision meetings. This is important given that the mayors have access to their own source of funds and would be more likely to spend them on relay activities if they were involved in their management. 3) Develop and negotiate the implementation of a plan of organization of meetings with actors (mayor, CSREF, PSE team) 3) Dr Souleymane Bagayoko 3. If PIDRIK comes through with funding this year for the construction of the 10 CSComs that they are promising, then the project needs to work with them and the MOH to see how to equip the facilities and carry out the corresponding community activities with the limited project resources. If PIDRIK does not come through with the anticipated funding, then the project needs to decide how many more CSComs it can realistically work with by the end of the project. Once the CS project has the results of the CSCom phase-out evaluation discussed above, it will be See Action plan Recommendation 1 40 able to decide whether it should put more resources into new CSCom areas or if it should devote them to strengthening the sustainability of the old zones. (We purpose to reformulate this recommendation as below due to the fact that the extension of the project is no longer linked to PDRIK) On the basis of the results of the CSCom Phase out study as recommended above, determine with the ministry of health and the other partners if more resources must be put into new health zones or if these must be devoted to reinforce the sustainability of the old zones. 4. Recommend that the project contract a short term BCC consultant to help them design a strategy that includes behavior change targets and strategies as well as awareness raising communications activities. It would be useful to employ more entertaining cultural formats that could be used to create interest in the villages such as stories, songs, street theater, and other methods. 1) Develop terms of reference of the consultation and plan financial resources for that purpose 2) Identify the consultant and get the protocol of agreement signed 3) Designing strategy and train actors of the implementation Dr Bagayoko Dr Bagayoko Consultant (under the supervision of Dr Bagayoko March 05 March 05 May 05 5) The CS project needs to follow through on its original plan to acquire and/or develop IEC materials that are appropriate for the target audiences. The health messages need to be reinforced through support materials. Also the materials would help the relays with their educational tasks. 1) Plan financial resources for the acquisition of adequate supports 2) Make census of existing IEC materials at the level of CNIECS and ensure the obtaining Dr Bagayoko Dr Sita Sidibe May 05 Jun 05 41 of more. 3) Put IEC materials at the disposal of actors (animators and Relays) Dr Sita Sidibe Jun 05 6) If resources permit, the project should consider acquiring another vehicle for the work in the new CSComs in the far northern section of the district. It should also consider replacing some of the worn out motorbikes being used by the animators 1) Plan for the purchasing of a vehicle and motorbikes for the project PUM Kita February 05 7) In order to strengthen some of the intervention activities, the project should consider providing breastfeeding technical assistance to improve technique and promote exclusive breastfeeding in the communities. 1) On the basis of the results of LQAS survey, identify communities where there is less breast-feeding and work with partners (NGOs) to develop a special action plan to promote exclusive breast-feeding in these communities 2) Conduct a communication campaign targeting behavior change on breast-feeding in these communities 3) Evaluate action of the campaign Dr Bagayoko NGOs (under the supervision of Dr Bagayoko) Elie Coulibaly January 05 February 05 August 05 42 IV. ATTACHMENTS A. Baseline Assessment from the DIP 43 B. Evaluation Team Members and Titles 47 C. Evaluation Assessment Methodology 48 D. List of Contacts 49 F. 3 rd LQAS Report 50 G. Project Data Sheet Form- Updated Version 71 43 Attachment A. Baseline Information from DIP SUMMARY OF BASELINE ASSESSMENTS Two baseline studies were undertaken prior to the development of the DIP. The researchers were staff members of the local NGO (ACD) that will be doing the training for the project. Two district health officers and two Plan Kita staff members supervised them. Dr. Pierre-Marie Metangmo, Plan International Health Program Coordinator trained the whole team in the use of the Rapid CATCH (Core Assessment Tool on Child Health) questions. Complete report in French can be found as Annex II and I. It is being translated into English. E.1 Evaluation of CSComs in Kita District The objective of this study was to have baseline information about the availability of and access to maternal and child health services in the district, as well as the quality of those services, with emphasis on the following: o Initial and continuing training on health protocols, IMCI and nutrition o Availability of ten essential medicines o Capacity of health workers to communicate health messages to users o Degree of user satisfaction with services received at the health centers. The study was carried out during the period from January 18 to 28, 2002. Eight CSComs were visited and information was collected through observations, interviews and inventories. Each of these CSComs is considered “functional” according the MOH’s definition. In each center, the sample constituted a minimum of ten children who presented with diarrhea (15), ARI or cough (16), fever/malaria (22), or malnutrition (5), and their mothers. All staff present were observed or interviewed. These included one doctor, 8 nurses, 7 midwives, one obstetric nurse and three nurses aides. Forty-two mothers were interviewed as they left the center. The following services were available CSComs: examination of sick children, CPN, family planning, birthing/deliveries, and vaccination. Centers were officially open from 7:30 am to 4:00 pm. Travel time to reach the center averaged more than one hour, and women had arrived by foot, bicycle, motor scooter or donkey cart. Of the 19 staff interviewed, nine had training in treatment procedures and protocols, one had IMCI training, six had nutrition training, and 10 in midwifery. A center is considered to be adequately equipped if it has the following: medical stethoscope, thermometer, blood pressure, otoscope, tongue depressors, scales (regular and for babies), refrigerator and tools for measuring and mixing. Using this standard, four centers ere not adequately equipped. The centers had anywhere from 30% to 90% of the ten essential medicines. No center had all ten essential medicines. 44 Staff were observed during examinations of sick children in order to determine their training needs, with the following results: o Diarrhea cases correctly diagnosed: 60% o Dehydration as a result of diarrhea correctly diagnosed: 40% o Diarrhea cases appropriately treated: 25% o Dysentery cases appropriately treated: 67% o Cough/breathing difficulties correctly diagnosed: 6.7% o Cough/breathing difficulties appropriately treated: 27.3% o Fever/malaria cases correctly diagnosed: 27.3% o Fever/malaria cases appropriately treated: 75% o No cases of malnutrition were diagnosed and therefore none was treated. o Verification of possession of vaccination cards took place for 10 of 42 children or 23.8%. o None of the mothers was asked whether she had a vaccination card or health book. o 70% of staff did not know the vaccination schedule. o 65% of staff did not know at least three signs of serious illness that would indicate the need for referral. o In 34.9% of cases observed, at least three messages about child health care were transmitted. Exit interviews with mothers: o 54.8% of mothers said they had learned at least two types of home care for their sick children after the examination. o 59.5% said the health care provider had told them about signs that would indicate the need to bring the child to the health center. o 84% were able to tell the interviewer at least two of the signs they had learned during the examination. Supervision: o 89.5% of staff said they got supervision once a trimester. o Only 17.6% reported getting any feedback from their supervision. User satisfaction: 55% of mother said they had difficulties in using the CSCom health services, including the long distance which takes a long time (54.5%), child care (9.1%) and the hours during which the CSCom is open. While 40 mothers interviewed (95.5%) said they were satisfied with the medical attention their children had received, what is interesting is the reasons they gave for being satisfied: o The health care provider really looked at the child carefully. o The health care provider really welcomed me/the health care provider was friendly o The health care provider reassured me/told me it was nothing serious. o The health care provider told me what was wrong with my child. o The health care provider gave me a lot of explanation. 45 The researchers’ general impression was that women were timid about questioning the health workers. Recurring costs: · There was a lack of ledger books for keeping accounts. · Supporting records, such as the treasurer’s records, were badly filled out or not filled out at all. · Absence of justifying document for expenses declared. · At Kassaro CSCom salaries hadn’t been paid for a year, and in Sandjanbougou there was no financial information at all. E.2 Baseline Evaluation of Knowledge, Attitudes and Practices This study was undertaken by the same researchers from January 20 to February 5, 2002. The objective was to determine knowledge, beliefs and behaviors regarding malaria, diarrhea, immunization, ARI and nutrition. Women between the ages of 15-49 and their children ages 0-23 months were randomly chosen. The total number of mother/child pairs was 430. Findings: o 18.4% of children were two standard deviations below average in weight for age. o 68.6% of infants had an assisted birth (from a health worker). o 24.1% of mothers had received two TT during their last pregnancy. o 51.0% of children ages 12-23 months had completed their vaccines before their first birthday. o 61.5% of children ages 12-23 months had been vaccinated against measles. o 11.6% of infants ages 0-5 months had been exclusively breastfed during the 24 hours preceding the interview. o 79.7% of children ages 0-23 months were breastfed immediately within the first hour after birth. o 9.8% of infants ages 6-9 months had received breast milk and complementary food during the 24 hours preceding the interview. o 3.9% of children ages 0-23 months had slept under an insecticide treated bed net the night before the interview. o 39.5% of mothers of children ages 0-23 months could cite at least two methods of preventing HIV/AIDS, including abstinence, having only one partner or using condoms. o 27.9% of mothers said they washed their hands with soap before preparing food or giving food to their children. Health care behaviors: ß 26.6% of children ages 0-23 months had received increased amounts of liquid and continued eating while sick during the preceding two weeks. ß 25.2% of children with diarrhea during the preceding two weeks had received ORT or liquid foods made at home. 46 ß 35.3% of mothers of children ages 0-23 months knew at least two signs of serious illnesses that would necessitate taking their children for treatment, including high fever, not eating, listlessness and vomiting everything that had been eaten. 47 Attachment B. Evaluation Team Members and their Titles Names Titles Elie Coulibaly CSP M&E Assistant Alassane Cissé Program Unit Manager Plan Kita Issa Traoré District Chief Medical Officer, Kita Souleymane Bagayoko Coordinator of CSP Abdoulaye Diakité Chief of the Social Development Ministry for Kita Soriba Diarra FELASCOM President Sita Sidibé CSP Training Assistant Aguibou Coulibaly National Coordinator for Child Survival in Plan Mali Mamadou Seck Assistant Information Systems manager for CSP Dian Namory Sidibé Medical Coordinator, Plan Banamba Bourema Yanogué Zone Chief, Plan Kita Bemba Doumbia Zone Chief, Plan Kita Seydou Diabaté Medical Coordinator, Plan Kangaba Ibrahima Diakité Coordinator of ARAFD/C Kita Mamadou Keita Kayes Regional Health Officer for Child Survival Pierre Marie Metangmo Health Programs Coordinator Plan (IH) USA Sandra Wilcox Child Survival Project Evaluation Consultant Ibrahim Simaga Assistant Chief Medical officer, Kita 48 Attachment C. Evaluation Assessment Methodology and Preliminary Results The mid-term field evaluation took place between December 12th and 22nd 2004. The consultant and Plan Child Survival Coordinator met with the Plan Director of Programs and the national Coordinator for Child Survival in Bamako on the 13th. They also met with the USAID health officer very briefly and traveled to Kita that day. The field evaluation took place between the 14th – 21st of December at the project sites in the Kita district. On December 14th the consultant and CS Coordinator met with the rest of the evaluation team. The team, as noted in Attachment B, was composed of a broad array of stakeholders including individuals from the project team (Plan and local NGO (ARAFD/C) staff, Plan regional, national and international offices, the Kayes Regional and Kita District MOH, the FELASCOM (ASACO national organization), and the Kita Ministry of Social Development. The team met between December 14th – 20th at Plan’s regional Program Unit office in Kita. The first two days were spent listening to a briefing by project staff of the project’s achievements and issues, discussing key project questions and concerns, small group preparation of field visit questionnaires, organizing the group into teams and selecting sites to be visited. On the third day, the team finalized the questionnaires and field tested them at a non-project site. On December 16th, 17th and 18th, the evaluation team was split into three groups and each visited one CSCom and at least one associated village each day. A total of 6 health Areas were visited. The teams met with and interviewed: the staff of the CSCom, the ASACO members, the village Health Committee members, women’s groups, the Relays and Animators. Afterwards the visiting teams conducted home visits and interviewed some of the village members. After each visit the individual teams met and outlined strengths, weaknesses and preliminary recommendations regarding the sites they had visited. On December 19th the evaluation team met and reviewed the preliminary findings from each group. These were discussed and then compiled into a summary of results document. During this period and on December 18th and 19th the Consultant conducted individual interviews with project staff, partners, the animators, the MOH representatives, Felascom director and Chief of the Social Development Ministry. The consultant conducted a preliminary debriefing with the CS Project staff on the morning of the 20th and then traveled to Bamako in the afternoon. She met with and debriefed the Plan Country Director on December 21st and returned to the US on the 22nd . 49 Attachment D. List of Contacts Kasimir Youmbi Plan Program Support Manager Aguibou Coulibaly Plan national Child Survival Coordinator Joachim Segurado Plan Country Director Alassane Cisse Plan Program Unit Manager, Kita Elie Coulibaly CSP M&E Assistant Issa Traore District Chief Medical Officer, Kita Soulemane Bagayoko Coordinator of CSP Abdoulaya Diakite Chief of Ministry of Social Development Soriba Diarra President of the FELASCOM Sita Sidibe CSP Training Assistant Mamadou Seck Information Systems manager Dian Namory Sidibe Health Coordinator, Plan Banamba Bourema Yanogue Zone Chief, Plan Kita Bemba Doumbia Zone Chief, Plan Kita Seydou Diabate Health Coordinator, Plan Kangaba Ibrahima Diakite NGO Coordinator ARAFD-C Kita Mamadou Keita Kayes Regional Health Officer for Child Survival Ibrahim Simaga Assistant Chief Medical officer, Kita 50 Attachm ent F Resu lts of the third LQAS su rvey (JUNE 2004) E li e COUL I BALY, CSP Mon itoring and Eva l uat i on Ass istant CSP D r Sita SIDIBE, CSP Training Assistant MON I TOR I NG THE KI TA CH I LD SURVI VAL PROJECT. JUNE 25 – 30 , 2004 51 Septembe r 2004 SUMMARY Page I - I NTRODUCTION 2 II - RESULTS 4 A. Current level of indicators 4 B. Evol ution of indicators 6 C. Graphs 7 D. I dentifi cati on of pr i orities by health area 11 III - DI SCUSSION 14 IV- CONCLUS ION and RECOMMENDATIONS 16 APPENDI CES 18 52 I - I NTRODUCTIO N The third Pl an Kita-Koura Ch ild Survi val Project LQAS survey was conducted from June 20 through 25 2004, and health and soci al services personnel and ASACOs we re closely involved in this survey technique. Through this exercise, the project staff begins to undertake the process of transferring competenc i es with the active participation of the first benef i ci aries and stakehol ders of the project. The survey was carr i ed out i n five (5) days. Resu lts obtained from this survey provi ded us with quite revealing i ndi cati ons that we will now ill ustrate in this report. The results will certainly be useful for us as indicators mon itoring the project for this year, and they will also enabl e us to draw up an acti on pl an taki ng i nto account the areas that are consi dered to show weaknesses . The survey showed us that mo thers’ behavi or in the management of ch il dhood ill ness had changed s i gn ifi cantl y. Resu lts for this survey have proved that the rate of coverage of i mmun i zati on has cl early progressed (69.54% o f fully vaccinated children as oppose d to 50% i n the Novembe r 2003 su rvey.) Concerni ng care for mo thers, there has been quite interesting progress (during the last pregnancy, 79.61 % of women had taken iron and foli c aci d i n June 2004, as agai nst 76. 32% i n Novembe r 2003) Women ’s behavi our relating to ANC (Ante -Natal Consu ltation) i mp roved, showi ng a rate of 55.92 % i n June 2004, as compa red with a yearly objecti ve of 40%. The rate of women’s i mmun i zati on coverage has i mp roved by 8. 90 % between Novembe r 2003 and June 2004, With relation to wom en taki ng vitami n A i n the 40 days i mmedi atel y following childbirth, the result of 40.13% has exceeded the annual objecti ve of 35%. 53 The results of this June 2004 LQAS survey enabl ed us to i dentify the zones with problems, or the priority zones for wh i ch we are mak i ng suggesti ons and recommendati ons. - Ma i ntai n and rei nforce vacci nati on at a month l y rate in advanced strategy i n all health areas and outreach posts, with i mp li cati on of traditional birth attendants in this outreach strategy for ANC and PNC and /or mon itoring healthy children. - Decentrali ze the strategy for distri buti ng vitami n A i n all health area vill ages with close involvement of commun ity health relay wo rkers i n the process. - Enter all the ti mes vitami n A is taken in a CSCom register, and i n the notebooks of the commun ity level relay wo rkers. - Emphas i ze commun i cati on for behavi or change, targeti ng mo thers so that they will adopt behavi ors favorabl e to their health and that of their children. - Foll ow up i mp l ementati on of mi cro -p l ans and HI S i n the chil d survival project zones. - Emphas i ze support for vill age hea lth committees and commun ity health relay wo rkers in analysis and interpretation of data collected each month . A. OBJECTI VES OF THE SURVEY 1. Compa re the level of evolution of indic ators to the results of the Novembe r 2003 su rvey and to the project’s third year objectives. 2. List the priority zones by health area in order to undertake necessary corrective action. B - METHODOLOGY 1. Sa mp li ng : The survey target concerned mo thers with ch il dren aged from 0 through 23 months , who li ved i n the project i nterventi on zones for at l east 6 months . Th is survey popul ation was sel ected through two parall el samp l es: mo thers with children aged from 12 through 23 months , and those with children aged 0 through 11 months We put var i ous questi ons to the mo thers concerni ng the mai n areas of the chil d survi val project. 54 For each of the groups of women, a samp l e of a maxi mum o f 19 subjects was sel ected at random i n each of the 8 health areas in the zone covered by the project, giving a total of 303 women. The numbe r of subjects for the survey i n each vill age was chosen as follows: - Ca l cu l ate the accumu l ated popu l ation of the vill ages i n the supervised area (health zone) - A list of all the househol ds was drawn up by the project team. - The f irst househol d to be surveyed was sel ected at random, using a tabl e of random numbe rs, from the list of househol ds. - From this househol d , women to be quest i oned were sought out i n the househol ds at a rate of one woman per househol d , and sel ecti ng the nearest house. - If there is mo re than one woman i n the househol d who fits the target criteria, a draw is made among the names of all of those i n the househol d , and the sel ected name is retained for the samp l e. The s i ze of the samp l e var i es accordi ng to the numbe r of avail abl e responses from 303 to 143 and accordi ng to whether or not the two groups of women are part of the denomi nator. 2. Ana l ys is (or tabulation) of data The tabl e of anal ysis for each health area was used by the survey team to make anal ysis of the data: the who l e operati on was conducted manuall y. 3 . Data ana l ys is. We conducted data anal ysis from the summary tabl e and the tabl e of the « rule of decisions ». II - RESULTS OF THE SURVEY - The current l eve l o f i nd i cators m easured. : Average cover of indicators in Kita project intervent i on zones, June 04. 55 Level of the indicator in each group of women Size of samp le Average Cover , by Perc entage 0-11 12 -23 Together 0- 11 12 - 23 Total 0-11 12 -23 Total Ch ildren having a vaccination card. - 137 137 - 151 151 - 90.72 90.7 2 Fully vaccinated children - 105 105 - 151 151 69.53 69.5 3 Women having a consultation card/booklet 129 - 129 15 2 0 152 84.86 - 84.8 6 Women having comp leted at least three ante - natal visits. 85 - 85 15 2 0 152 55.92 55.9 2 - Women having taken iron folic acid during the last pregnancy 121 - 121 15 2 0 152 79.60 - 79.6 0 Women having taken chloroquine during the last pregnancy 125 0 125 15 2 0 152 82.23 - 82.2 3 Women having received two doses of more of ATV 120 0 120 15 2 0 152 78.94 - 78.9 4 Women having given birth i n a health center 80 0 80 15 2 0 152 52.63 - 52.6 3 Women having been assisted by a trained health worker in childbirth 99 0 99 15 2 0 152 65.13 - 65.1 3 Women having taken Vitam in A in the 40 days following childbirth 61 0 61 15 2 0 152 40.13 - 40.1 3 Wom en possessing a bednet 34 58 92 15 2 151 303 22.36 38.41 30.3 6 Ch ild having slept under an i mpregnated bednet. 30 38 68 15 2 151 303 19.73 25.16 22.4 4 Women knowing 2 signs indicating that the child needs care 45 138 183 50 152 202 90 90.78 90.5 9 Sick children having received at least the same quant ity or more than usual to drink 98 98 143 143 68,53 - 68,5 3 Sick children having received at least the same quant ity or more than usual to eat - 72 72 - 143 143 50.34 - 50.3 4 Women knowing how to describ e the preparation of ORS. - 54 54 - 70 70 77.14 - 77,1 4 Tabl e no. 1 denotes the current situation of indicators in the two target group sel ected for the survey: ( Women with children aged 0 -11 months and women with children aged 12-23 months) Ana l ysis o f this table provides interesting infor mation on: - i mmun i zati on coverage of chil dren under the age of 24 months - use of health services by mo thers to preserve their health status and that of their children. - the i mpact of commun i cati on acti vities for behavi or change (BCC ) on mo thers’ attitudes and behavi or. 56 - Use of i mp regnated bednets i n our commun ities, particularly for ch il dren under the age of 5 years and for pregnant women. Resu lts in this table lead to the followi ng deduct i ons: - use by ch il dre n of i mp regnated bednets appears to be the i ndi cator with the weakest response l evel i n this survey (22,45% o f children slept under an i nsecti ci de-treated bednet the previ ous n i ght). - Women ’s conservati on of i mmun i zati on cards and their knowledge of ch il dhood ill nesses seem to be the i ndi cators with the highest scores: 90. 72 % for the first, and 90.59% for the second B - Evo l ution o f the i nd i cators i n rel ati on to the baseli ne study and the Nove m be r 2003 LQAS survey Tab l e N°2 : Level of evol ution of the i nd i cators i n the e i ght (8) f irst project i ntervent i on zones i n Kita, June 2004 I ndi cators Baseli ne l evel - % 3 year l evel - % 3 year l evel - % Novembe r 2003 3 year l evel - % June 2004 Ch il dren havi ng a i mmun i zati on card 49 65 86 . 84 90.73 Fu ll y v acci nated chil dren 51 65 50. 00 69.54 Women with an ANC card/book l et 14 40 83 . 55 84.87 Women havi ng comp l eted at l east 3 ANC visits 14 40 53 . 02 55.92 Women havi ng taken iron+foli c aci d dur i ng the l ast pregnancy 65 75 76 . 32 79.61 Women havi ng taken 55 70 83 . 78 82.23 1 Basic KPC %age is fixed at 51% of fully vaccinated children from 12 through 23 months (53 children vaccinated out of 104 holding a card, as compared with a total of 212 surveyed children). This calculation excluded children not holding a vaccination card from the sample. We have therefore adjusted the indicator by relating it to the overall number of surveyed children (53/212 =25%) in accordance with general practice and methodology applied in the LQAS framework). 57 ch l oroqu i ne dur i ng the l ast pregnancy Women havi ng recei ved at l east 2 doses of ATV 24.1 40 71 . 05 78.95 Women havi ng been ass isted by a trai ned hea lth wo rker in ch il dbirth 68 .6 75 68 . 24 65.13 Women havi ng taken Vitami n A i n the 40 days following ch il dbirth - 35 40. 56 40.13 Ch il d havi ng s l ept under an i nsecti ci de-treated bednet. 3 .9 30 17 . 67 22.45 Women knowi ng 2 s i gns i ndi cati ng that the chil d needs care 35 .3 50 87 . 75 90. 59 Si ck ch il dren havi ng recei ved at l east the same quantity or mo re than usual to dr i nk 26 .6 50 66 . 52 68 . 53 Si ck ch il dren havi ng recei ved at l east the same quantity or mo re than usual to eat 21 50 44. 04 50. 34 Women knowi ng how to descr i be the preparati on of ORS. 49 50 58 . 75 77 . 14 C. G raphs ill ustrati ng the re su lts in com par i son with the m on ito ri ng i nd i cators 58 0 10 20 30 40 50 60 70 80 90 100 Existence child. vacc card Fully vaccinated children Women with consultation card Women 3 ANCs objective Yr3 result Nov 03 result June 04 59 0 10 20 30 40 50 60 70 80 90 3 ANC visits Iron + folic acid taken Chloroquine taken Two doses of ATV Objective Yr 3 Result Nov 03 Result June 04 60 0 10 20 30 40 50 60 70 80 90 100 Vit A 40 days after giving birth Children having slept under ITM bednet. Women knowing 2 signs of illness requiring treatment Objective YR 3 Result Nov 03 Result June 04 0 10 20 30 40 50 60 70 80 Sick child +drink Sick Child + eat Women knowing SRO description Objective Yr 3 Result Nov 03 Result June 04 61 Tabl e no. 2 h i gh li ghts the extent of evol ution of i ndi cators between the surveys of Novembe r 2003 , June 2004 and the baseli ne study, and these results are compa red to the objecti ve set for the third year of the project. We note a s i gn ifi cant i mp rovement i n the l evel of indicators compa red with the result s of the Novembe r 2003 LQAS su rvey. - The evol ution is very significant for indicators of behavior and practi ces ( Women havi ng had at l east three postnatal consu ltations, women havi ng recei ved at l east two doses of ATV, and women havi ng taken vitami n A) - The rate of possessi on of vacci nati on cards by chil dren aged from 12 through 23 months is an indicator with a good score. The objecti ve set for year three of the project has been decisively exceeded. - For the i ndi cator measur i ng the behavi or of a mo ther faced with a ch il d suffering from d i arrhea, the objecti ve set for year III of the project has been cl early attained. - The objecti ve set has been attai ned or exceeded i n 11 cases out of 14 - For three sentry i ndi cators, the objecti ves set have not been attained; results of these indicators are characterized by pockets where they have become a pr i ority and they call for particular attention: ( Women havi ng been ass isted during childbirth, children havi ng s l ept under i mp regnated bednets and s i ck ch il dren havi ng been gi ven at l east as much as usual to eat.) - For five (5) indicators ( women havi ng taken chl oroqu i ne dur i ng their l ast pregnancy, women havi ng been ass isted in childbirth by a health wo rker, women havi ng taken vitami n A i n the 40 days following childbirt h, sick children having received at least as much as usual to eat, sick children having received at least as much as usual to dr i nk) the l evel of indicators has fallen between Novembe r 2003 and June 2004. C- I dent ification of priorities - 1 -Pr i or iti es not ed for women with children aged from 0 through 11 m onths 62 Tab l e N°3 : I dent ifi cat i on of pr i orities accordi ng to results of the LQAS3 survey i n the i ntervent i on zones of the P l an Ma li, K ita PU CSP i n June 2004 Indicators BEND BOUG DIDJAN FLD MAK KAS SEB SAND Women, having a consultation card/booklet No N N N N N N N Women having received two or more doses of ATV N N N N N N N N Women, having completed at least three ante-natal consultations N N N N N N N N Women having taken iron + folic acid during the last pregnancy N N N N N N N N Women having taken chloroquine during the last pregnancy N N N N N N N N Women having given birth in a health center NNN Yes Y N N N N N Women having been assisted by a trained health worker during childbirth Y Y N N N N N N Women having taken vitamin A in the 40 days after childbirth N N Y N N N N N Women possessing a bednet N N N N N N N N Children 0-11 months, having slept under an impregnated bednet N N N N N N N N Women knowing two signs that the child needs care at home or elsewhere N N N N N N N N Children having presented symptoms of illness during the previous two weeks N N N N N N N N Sick children having received at least the same amount as usual, or more, to drink. N N N N/A N N N N Sick children, having received at least the same amount as usual, or more to eat. N N N/A Y N N N N Women, having heard of ORS N N N N N N N N Women, knowing how to describe ORS preparation Y Key: no priority N/A : Not App li cable 63 priority Tabl eau No. 3 be l ow p resents the success registered in each health area i n the project i nterventi on zone. We note here that there has been cl ear progress at the l evel of indicators i n each health area. All the areas surveyed have made real progress, the pockets showi ng priority needs or the areas of concern have been consi derabl y reduced i n the i ntervals between the three surveys: i n Ma rch 2003 , there we re 29 priority pockets, in Novembe r 2003 : 12 pockets , and the current survey i n June 2004 shows 6 pockets . We note that every health area has made a great l eap forward between the Novembe r 2003 su rvey and the June 2004 one (for examp l e the Dji d i an area had 4 p ri ority zones i n Novembe r 2004, agai nst onl y 1 i n the June 2004 survey, and Sandi anbougou registered 4 priorities compa red with 0 th is ti me.) The areas of Makandi anbougou , Sébékoro, Sandi anbougou , mai ntai ned their l evel of perfor mance (no red square either in the Novembe r 2003 survey or that of June 2004). Out of a total of sixteen (16) sentry i ndi cators targeted by the survey, four (4) present pr i ority areas. 2 Priorities noted among women with children aged from 12 through 23 months Table N°4: Identification of priorities in accordance with results of the Plan Mali, Kita PU third LQAS survey, in June 2004 I nd icators BEND BOUG D I DJAN FLD MAK KAS SEB SAND Ch il dren havi ng a immun ization card N N N N N N N N Fu lly vaccinated children N N N N N N N Y Women possess i ng a bednet N N N N N N N N Ch il dren from 12 through 23 months hav i ng s lept under an i mp regnated bednet N N N N N N N N Women knowi ng two s i gns that a ch il d needs treat m ent at home or e lsewhere. N N N N N N N N 64 Ch il dren of 12 - 23 months , havi ng presented symptoms of ill ness dur i ng the previ ous two weeks N N N N N N N N S ick children from 12 - 23 months hav ing received at least the sa m e a mount as usua l, or mo re, to drink. N N N N N N N N S ick children from 12 - 23 mon ths having received at least the sa m e a mount as usua l, or mo re, to eat N N N N Y N N N Women hav i ng heard of ORS N N N N N N N/ A N Women , knowi ng how to describe ORS preparation N/ A N/ A N N/ A N N N/ A N/A Key: Good Bad N/A: Not App li cable From the contents of this table we note that there has been a cl ear evol ution of the targeted sentry i ndi cators between the LQAS survey of Novembe r 2003 and that of June 2004 (on l y I pocket of pr i ority need i n June 2004 as compa red with 12 i n the November 2003 su rvey). I n additi on , the table shows that onl y the Sandi anbougou hea lth area presents a pr i ority zone and for onl y one i ndi cator « fully vaccinated ch il dren » wh il e the areas of Bougar i baya , Bendougouba , Sebekoro , Dji d i an, Sandi anbougou , all have at l east one i ndi vi dua l priority during the Novembe r 2003 LQAS su rvey. Concerni ng the possess i on of chil dren’s vacci nati on cards, whil e just one (1) area, Bougar i baya , presented a pr i ority need i n Novembe r 2003 , dur i ng the present survey, no health area has presented a pr i ority pocket for this indicator. We note that all the areas have made a real effort. However, we still have to note that there are differences between the l evels of perfor mance of the areas (see the summary tabl e i n appendix): different l evels of priority for the various efforts to be made are observed i n the health areas. The Sandi anbougou hea lth area stands out among the other hea lth areas because of its weak l evel of perfor mance and its score is in comp l ete contrast with the tendency elsewhere. 65 Th is fact calls for an effort on our part and should make us parti cu l arly attentive: we must gi ve it our supervisory attention to bring this zone to the same l evel of perfor mance as the others. To sustai n the success we have noted previ ous l y, we must be vi g il ant and put forth the maxi mum effort not onl y to mai ntai n this perfor mance, but also to i mp rove them. III. DISCUSSION I n this chapter, we will attemp t to gi ve el ements of answers to some of our preoccupati ons, whi ch we cons i der pertinent in order to attain the objecti ves that will l ead the chil d survi val to full success. 1 . i mm un i zat i on : The percentage of full y vacci nated chil dren is of 69.54 % compa red with an objecti ve of 65%, i n our opi n i on represents a strong perfor mance. The results for this indicator are striking: (33.55% i n Ma rch 2003 , 50 % i n Novembe r 2003 , and 69. 54% i n June 2004) . Th is success is certainly remarkabl e, but there are certain difficulties that shou l d be poi nted out – among others the distance of the popu l ation from vacci nati on centers, the proliferation of agricultural haml ets, irregular conduct of vacci nati on i n outreach strategy, del ays i n payment of vacci nators’ salaries: the examp l e bei ng that of the vacci nator for Sandi anbougou , l ack of foll ow-up of i mp l ementati on of the ASACO membe rs’ mi cro -p l ans i n the var i ous health areas surveyed – all of these are major constrai nts wh i ch must be countered and wh i ch call for remova l as they represent real obstacl es to the Chil d Survi val Project. 2 . The use of i mp regnated bednets The i ndi cator concerni ng usage of i mp regnated bednets has made c l ear progress. It has gone from 17 . 6% i n Novembe r 2003 to 22 . 45% i n June 2004. Th is can be expl ai ned i n part by Pl an’s support for free suppl y of i nsecti ci de tre ated nets to the ASACOs (March 2003) and a lso by the promo ti ona l sales of i mp regnated bednets by the M i n istry of Health (June 2003) . The l evel of progressi on rel ating to this indicator is acceptable and encouragi ng with a success rate of 22.45% towards an objecti ve of 30%. 66 I n additi on to the comb i ned efforts concerni ng these two acti ons, the recent arrival of UNICEF in the circle ( Ap ril 2004) p l aci ng at the d isposition of the Health referral Center a stock of 30 000 i mp regnated bednets to be provi ded fr ee of charge to women havi ng attended a postnatal consu ltation and to chil dren who have comp l eted i mmun i zati on before their first birthday, will certainly contribute to i mp rove responses to this indicator in the comi ng months. I n coll aborati on with its partners, the Chil d Survi val project will p l ace particular emphas is on treating bednets and other supports i n the next months to i mp rove this activity, wh i ch until now has remai ned a weak li nk i n the poli cy of preventi ve mal aria control. 1 - Commun i cat i on a ctivities to change mo thers’ behavi or: With regard to mo thers’ attitudes to giving mo re food than usual to a s i ck ch il d , the indicator has shown a z i gzag devel opment : (43 . 72% i n Ma rch 2003 ; 44. 04% Novembe r 2003 , and 42. 17% i n June 2004), Th is indicator s hou l d attract our attenti on most of a ll, because not on l y the object i ve for the th ird year (50%) has not been attai ned, but also, and i n part i cu l ar, the level of perfor m ance for th is indicator has fallen between Nove mbe r 2003 and June 2004. We are encouragi ng the organi zers and chi efs of medi cal posts to pl ace a particular emphas is on commun i cati on acti vities for Behavior Change in this respect, and also call on greater ASACO i nvol vement i n soci al mob ili zati on acti vities in their respective health areas. 4 . Concern i ng ad m i n istration of Vitam i n A: From Novembe r 2003 to June 2004, the level of indicator relating to admi n istration of Vitami n A to women i n the 40 days foll owi ng ch il dbirth has fall en – the l evel pass i ng from 40 . 56% i n Novembe r 2003 to 40. 13% i n June 2004. To i mp rove the tendency, we must have the unwaver i ng support of the ch i efs of medi cal centers and all the CSCom staff on one hand, and on the other hand that of the Chil d Survi val project through the organi zers representi ng it for this a ctivity at commun ity level. I mp li cati on of the commun ity health relay wo rkers is very i mpo rtant in the strategy for Vitami n A d istri buti on to post-partum women and to ch il dren aged from 6 through 59 months 67 We are not l oos i ng s i ght of the efforts made by those i n charge of the Referral Health Center in bringing the rate of average coverage for this i ndi cator from 8 . 55% i n Ma rch 2003 to 40. 56% i n Novembe r 2003 I n additi on , although the l evel of perfor mance certai n l y fell between Novembe r 2003 and June 2004, it has to be recogni zed that the objecti ve for year three (35%) has cl early been attai ned. I V. CONCLUS I ON AND RECOMMENDATI ONS On p resenting the results of this third LQAS su rvey, we note c l ear i mp rovement i n the mon itoring indictors, each health area havi ng shown real i mp rovement. The objecti ves set for year three (3) have been met for 12 i ndi cators out of 14 with significant increases in maternal protecti on (ante -natal consu ltations, admi n istration of iron and folic acid) and for the protection of i nfants (vacci nati on , usage of i nsecti ci de treated bednets). The results of this survey give essential infor mation. The questi onna ire admi n istered to mo thers of chil dren aged from 12 through 23 months shows that out of the ei ght areas surveyed, onl y the health area of Sandi anbougou has a pr i ority need pocket and that is for on l y one i ndi cator “fully vaccinated children wh il e five (5) areas presented at least one pr i ority pocket: Sandi anbougou , Dji d i an, Bougar i baya , Sebekoro, Bendougouba i n Novembe r 2003 I n additi on , it appears from anal ysis of the questionnaire addressed to mo thers with children aged from 0 through 11 months that the areas of Sandi anbougou , Sebekoro, Makandi anbougou mai ntai ned the same perfor mance, from the Novembe r 2003 su rvey through that of June 2004 and that they have presented no def i cit for the 16 i ndi cators. Dur i ng the same per i od , priority zones have been i ndi cated i n the other: F l adougou-Ma réna 1, Dji d i an 1 , Kassaro 1 as compa red with 3 i n Novembe r 2003 , Bendougouba 1 compa red with 3 , Bougar i baya 2 compa red with 4 With regard to chil d i mmun i zati on , a clear i mp rovement is noted i n the situation, and the numbe r of fully vaccinated children has passed from 50% to 69. 54% from Novembe r 2003 through June 2004 We have obtai ned this result thanks to the joint efforts of the CSREF and Pl an, wh i ch have set up outreach vacci nati on posts i n zones whose access 68 poses a probl em, and throughout the circle. There too, it is i mpo rtant to recogni ze the efforts of the decentralised organizations ( mayors’ offices, ASACOs) , wh i ch have supported this initiative on a voluntary basis. There has obvi ous l y been cons i derabl e progress, but we must not forget the mai n ai m o f the project, namel y: Sustai nabl e i mp rovement i n the health status of children and wom en of ch il dbear i ng age; this is why we must take i nto account the pr i orities i dentifi ed i n anal ysis of the results of this third survey to ensure nor mal conti nuati on of project acti vities. We make the foll owi ng recommendati ons: 1 - Add ressed to the ASACOs - Greater involvement by ASACOS i n drawi ng up, i mp l ementi ng, mon itoring and eval uati ng CSCom acti vities (taken into account by the mi cro -p l an for i mp l ementati on of i ndi cators recognised during this survey as bei ng pr i orities) - Support the rel ay workers’ acti vities in certain areas (such as d istri buti on of vitami n A and soc i al mob ilisati on .) 2 - Add ressed to the ch i efs of hea lth centers - Ma i ntai n month l y vacci nati ons as an outreach strategy i n all the health areas and outreach posts, and addi ng ANC, PNC and mon itoring healthy children - Devel op a strategy for distri buti on of Vitami n A (to ch il dren and post partum women ) in CSComs and i n outreach strategy, i nvol vi ng relay wo rkers at the commun ity level - Undertake routi ne registration of Vitami n A d istrib ution in CSCom support for ms and those at the disposal of commun ity relay wo rkers. 3 -Add ressed to the comm un ity organizers - Rei nforce commun i cati on acti vities for behavior change by mo thers and by men for the care of si ck ch il dren. - Support the CSV and rel ay workers i n anal ysis and interpretation of the health data coll ected on a month l y bas is. - Gi ve strong support to vill age hea lth committees in setting up a vill age hea lth devel opment pl an. 69 4 - Add ressed to the supervisory te ams (Plan, ARAFD/C, CSREF , SDSES) : - Undertake regul ar integrated supervisi on i n the chil d survi val project and vacci nati on zones and outreach posts. - Mon itor i mp l ementati on of mi cro -p l ans and the HI S (Hea lth I nfor mation System) in Child Survi val Project zones. We beli eve that LQAS shou l d be mo re wi del y used by hea lth and soci al devel opment servi ces because it is not only quick and easy but also i nexpens i ve i n compa rison with other survey techni ques such as KPC (knowl edge, practi ce, coverage.) It also allows for identification of zones with strong perfor mance and those with weak perfor mance, whi ch enabl es us to be mo re effective and efficient in i mp l ementi ng our acti ons. We have therefore taken the decisi on to transfer the next LQAS surveys (the project’s fourth year surveys) to the Health Referral Center in order to prepare it for conti nu ity of this activity after withdrawa l of the project. The results obtained during this survey coul d be used to ass ist the mid - ter m eval uati on , wh i ch i s programmed for Decembe r 2004. 70 Append ix 1 : TABLE SUMMARIZING INDICATORS BY HEALTH AREA IN KITA (LQAS 12-23 MONTHS) LQAS 3 JUNE 04 ES 71 N 0 Indicators BEND BOUG DIDJAN FLD MAK KAS SEB SAND Total Average coverage Objectives Yr 3 13 Children, having a vaccination card Number of correct replies 18 19 19 14 11 18 19 19 137 Size 19 19 19 18 19 19 19 19 151 Decision rule 16 16 16 16 16 16 16 16 90.73 95.00 13* * Fully vaccinated children Number of correct replies 16 17 16 13 9 12 18 4 105 Size 19 19 19 18 19 19 19 19 151 Decision rule 9 9 9 9 9 9 9 9 69.54 60.00 15Women possessing a bednet Number of correct replies 5 11 10 12 9 3 4 4 58 Size 19 19 19 18 19 19 19 19 151 Decision rule 2 2 2 2 2 2 2 2 38.41 25.00 15* Child aged 12-23 months, having slept under an impregnated bednet Number of correct replies 4 6 9 10 2 3 2 2 38 Size 19 19 19 18 19 19 19 19 151 Decision rule 1 1 1 1 1 1 1 1 25.17 20.00 18Women knowing at least two signs that child needs care at home or elsewhere Number of correct replies 15 19 18 13 19 19 19 16 138 Size 19 19 19 18 19 19 19 19 151 Decision rule 6 6 6 6 6 6 6 6 91.39 45.00 19 Children 12-23 having presented symptoms of illness during previous two weeks 72 Number of correct replies 17 13 10 13 18 15 15 13 114 Size 19 19 19 18 19 19 19 19 151 Decision rule 75.50 - 73 TABLE SUMMARIZING INDICATORS BY HEALTH AREA IN KITA (LQAS 12-23 MONTHS) Continued LQAS June 2004 N 0 Indicators BEND BOUG DIDJAN FLD MAK KAS SEB SAND Total Average Coverage Project Objectives Yr 3 20 Sick children from 12-23 months having received at least the same amount as usual, or more, to drink Number of correct replies 15 13 11 14 6 14 14 11 98 Size 17 13 19 18 19 19 19 19 143 Decision rule 4 3 5 5 5 5 5 5 68.53 40.00 21 Sick children from 12-23 months having received at least the same amount as usual, or more, to eat Number of correct replies 10 7 11 12 3 8 15 6 72 Size 17 13 19 18 19 19 19 19 143 Decision rule 5 4 6 6 6 6 6 6 50.35 45.00 22Women having heard of ORS Number of correct replies 19 19 15 18 15 19 15 N/A 120 Size 19 19 19 18 17 19 15 126 Decision rule 5 5 5 5 4 5 4 95.24 40.00 23Women knowing how to describe ORS preparation Number of correct replies N/A N/A 15 N/A 14 11 14 N/A 54 Size 19 N/A 17 19 15 70 Decision rule 5 4 5 4 77.14 40.00 24Well-nourished children Number of correct replies N/A N/A N/A N/A N/A N/A Size Decision rule 85.00 74 Attachment G CSHGP Project Data Sheet- updated version Child Survival Grants Program Project Summary Mid Term Submission: Jan-31-2005 PLAN Mali Field Contact Information: First Name: Last Name: Address: City: State/Province: Zip/Postal Code: Country Mali Telephone: Fax: (223) 2228143 E-mail: @plan-international.org Project Web Site: Project Information: Project Description: Plan International is implementing a new five-year Child Survival project in partnership with the Mali Ministry of Health, PDRIK (Projet de Développement Rural Intégré de Kita), Action Couverture Développement (ACD), and local municipalities in Kita District, located in the southeastern part of Kayes Region, in the republic of Mali. Kita’s total population in 2001 was estimated at 315,520 people of whom the project will cover 260,458 people by its fifth year. Estimated beneficiaries include: 52,091 women of childbearing age, 10,418 infants ages 0-11 months, and 36,464 children ages 1-4 years. Plan has been working in Kita District since October 1994, carrying out integrated health and hygiene, reproductive health, education, water and sanitation, environment, and micro-credit activities. It has been working with the local NGO partner, which has extensive experience in training and delivery of health messages. Health indicators for Kayes Region and Kita District lag behind those of Mali as a whole. According to the 1996 DHS, infant mortality in Kayes was 135/1000 and under-five mortality was 259/1000. The major causes of death of children in Kayes region are 75 malaria, ARI, diarrhea, malnutrition and measles. Challenges to health of women and children include: poor road conditions and long distances to health centers, the insufficient number of first level health centers (only ten of the currently proposed 46 health centers are functional), inadequate quality of health services provided due to insufficient training of personnel, and low income and purchasing power of the population, combined with high user fees of health centers. Utilization of health centers is very low (around 8% of the eligible population) and traditional healers are often sought for medical advice before health care providers. The program approach is to help strengthen the existing health care system. This approach comprises two strategies: (1) Training and building the capacity of health personnel and supporting community health committees (ASACOs) in order to increase utilization of health care services, and (2) Supporting IMCI implementation in CSComs. The four interventions are: i) Malaria (35%): Reduction of mortality and morbidity associated with malaria in children and pregnant women, through prevention education, promotion of use of impregnated bed nets, improved treatment of malaria, and prenatal chemo-prophylaxis. ii) Diarrhea (20%): Reduction of diarrhea-associated mortality and morbidity through a) teaching and promoting prevention measures in the home, b) strengthening mothers’ capacity to recognize and provide home treatment for mild diarrhea with fluid and dietary management, and c) strengthening mother’s capacity to identify signs of moderate and severe diarrhea, know sources of care, take the child for care, and comply with health provider recommendations. iii) Immunizations (35%): Increased immunization coverage in the program area for all infants by the end of the first year of life, and tetanus toxoid (second dose) immunization for pregnant women, increased measles vaccination (including twice yearly vitamin A) and prevention of measles-caused diarrhea and death. And iv) Pneumonia (10%): Reduction of pneumonia-associated mortality through prompt, appropriate and standardized pneumonia case management, a sustainable supply of antibiotics at the health centers (CSComs), prompt recognition by relays of pneumonia signs (fever, fast breathing) and referral of suspected cases to the nearest health center, and appropriate mothers’ care-seeking behavior. Partners: Mali Ministry of Health and Action Couverture Developpement (ACD) Project Location: Kita District, in Kayes Region, Mali Grant Funding Information: USAID Funding:(US $) $1,300,000 PVO match:(US $) $433,333 Target Beneficiaries: Type Number 76 infants (0-11 months): 10,418 12-23 month old children: 10,157 0-59 month old children: 46,882 Women 15-49: 52,091 Estimated Number of Births: 63,105 Beneficiary Residence: Urban/Peri-Urban % Rural % 10% 90% General Strategies Planned: Advocacy on Health Policy Strengthen Decentralized Health System Information System Technologies M&E Assessment Strategies: KPC Survey Health Facility Assessment Organizational Capacity Assessment with Local Partners Participatory Rapid Appraisal Lot Quality Assurance Sampling Community-based Monitoring Techniques Participatory Evaluation Techniques (for mid-term or final evaluation) Behavior Change & Communication (BCC) Strategies: Mass Media Interpersonal Communication Support Groups Capacity Building Targets Planned: PVO Non-Govt Partners Other Private Sector Govt Community US HQ (CS unit) CS Project Team Local NGO Networked Group Traditional Healers Dist. Health System Health Facility Staff Health CBOs Other CBOs CHWs Interventions: Immunizations 35 % ** IMCI Integration ** CHW Training ** HF Training *** Polio *** Vitamin A *** Surveillance *** Cold Chain Strengthening 77 *** Mobilization Acute Respiratory Infection 10 % ** IMCI Integration ** CHW Training ** HF Training *** Pneum. Case Mngmnt. *** Recognition of ARI Danger Signs Control of Diarrheal Diseases 20 % ** IMCI Integration ** CHW Training ** HF Training *** Hand Washing *** ORS/Home Fluids *** Feeding/Breastfeeding *** Care Seeking Malaria 35 % ** IMCI Integration ** CHW Training ** HF Training *** Training in Malaria CM *** Antenatal Prevention Treatment *** ITN (Bednets) *** IPT *** Community Treatment of Malaria 78 Indicator Numerator Denominator Estimated Percentage Confidence line Percentage of children age 0-23 months who are underweight (-2 SD from the median weight-for-age, according to the WHO/NCHS reference population) 79 430 18.4 5.5 Percentage of children age 0-23 months who were born at least 24 months after the previous surviving child 0 0 0.0 0.0 Percentage of children age 0-23 months whose births were attended by skilled health personnel 99 152 65.1 11.0 Percentage of mothers of children age 0-23 months who received at least two tetanus toxoid injections before the birth of their youngest child 120 152 78.9 9.5 Percentage of infants age 0-5 months who were exclusively breastfed in the last 24 hours 10 86 11.6 9.9 Percentage of infants age 6-9 months receiving breastmilk and complementary foods 4 41 9.8 13.1 Percentage of children age 12-23 months who are fully vaccinated (against the five vaccine-preventable diseases) before the first birthday 105 151 69.5 10.7 Percentage of children age 12-23 months who received a measles vaccine 64 104 61.5 13.5 Percentage of children age 0-23 months who slept under an insecticide-treated bednet the previous night (in malaria-risk areas only) 68 303 22.4 6.9 Percentage of mothers who know at least two signs of childhood illness that indicate the need for treatment 183 202 90.6 6.0 Percentage of sick children age 0-23 months who received increased fluids and continued feeding during an illness in the past two weeks 72 143 50.4 11.9 Percentage of mothers of children age 170 430 39.5 6.8 79 0-23 months who cite at least two known ways of reducing the risk of HIV infection Percentage of mothers of children age 0-23 months who wash their hands with soap/ash before food preparation, before feeding children, after defecation, and after attending to a child who has defecated 120 430 27.9 6.3 Comments TB Indicator Indicator Numerator Denominator Estimated Percentage % of new smear positive cases who were successfully treated 0 0 0.0