Mali Child Survival Project (CS-XV) Midterm Evaluation Helen Keller International/Mali Submitted to USAID/BHR/PVC February 14, 2002 contacts: Zeina Sifri Diakalia Kone Deputy Director for Child Survival Child Survival Project Coordinator Helen Keller Worldwide Headquarters Helen Keller International/Mali 352 Park Avenue South, suite 1200 Quinzambougou Rue 555 New York, NY 10010 Porte 27 USA Bamako, Mali 212 532 0544 ext. 810 223 21 5293 fax: 212 532 6014 fax: 223 21 5294 zsifri@hki.org hkimali@spider.toolnet.org 1 HKI Mali/Mid-Term Evaluation TABLE OF CONTENTS Abbreviations page 3 Executive Summary 5 I. Technical Approach A. Overview of the project 7 B. Objectives and Progress by Intervention Areas 7 i. progress toward accomplishment of DIP intervention objectives ii. effectiveness of CS interventions iii. changes in technical approach C. New Approaches and Operational Studies 14 II. Cross-Cutting Approaches A. Capacity Building Approach 16 i. national nutrition/CS policies and programs ii. regional and district nutrition/CS programs iii. support for regional N/CS program development iv. support to district level management teams and district programs v. CSComs and rural dispensaries vi. community health associations vii. NGO capacity viii. community capacity ix. PVO organization B. Progress Related to Sustainability Strategy 27 III. Program support and Management A. Planning and Coordination of Project Activities 28 B. Training Strategy for MOH, NGOs, and community partners 29 C. Staff Management and supervision 31 D. Human Resources 31 E. Financial Management 32 F. Logistics 32 G. Information system, monitoring, and evaluation 32 H. Administrative Support 33 I. Technical Assistance 33 IV. Conclusions and recommendations A. Main achievements 34 B. Main constraints 35 C. Main recommendations 36 D. Conclusions regarding participatory methodology 37 V. The Action Plan 39 Attachments: A. Field Program Summary B. Program Goals and Objectives C. Program Location D. Program Design 2 E. Project Partners and their Roles F. Health Information System G. Members of Mid-term Evaluation Team H. Assessment Methodology I. List of Persons Interviewed and Contacted 3 Abbreviations ACOD Association Conseil pour le Développement AI Appreciative Inquiry ASACO Association de Santé Communautaire CBD Community-Based Distribution CHV Community Health Volunteers CS Child Survival CSCom Centre de Santé Communautaire CSP Child Survival Project DHS Demographic and Health Survey DHT District Health Team DIP Detailed Implemenation Plan DMT District Management Teams (Equipe Socio-Sanitaire de Cercle) DTC District Technical Committee ESSC Equipe Socio-Sanitaire de Cercle FFM Food Frequency Method GMs Grandmothers JRM Journées Régionales de Micronutriments MCH Maternal / Child Health MI Micronutrient Initiative project MIS Management Information System MOH Ministry of Health MPA Minimum Packet of Activities MSL Monitoring, Sharing, and Learning MSD Ministry of Social Development MTE Mid-Term Evaluation MS Ministère de la Santé N/CS Nutrition/Child Survival 4 ORS Oral Rehydration Solution PEM Protein-Energy Malnutrition PNC Prenatal Clinics SSS Sugar Salt Solution TA Technical Assistance TAC Traditional Actors and Communicators TBA Traditional Birth Attendant TC Traditional Communicators VAC Vitamin A Capsules WRA Women of reproductive age 5 Executive Summary The overall goal of the CS-15 project implemented by HKI in Mali is to assist the Government of Mali and partner NGOs to develop programs that promote good nutrition-related practices at the health facility, community and household levels. Ninety percent of project effort focuses on maternal and child nutrition interventions which, in addition to a limited focus on management of diarrheal disease are expected to contribute to improved health and survival of both women and children. The project is characterized by a capacity-building orientation and aims to strengthen capacity within the MOH, NGOs, communities and households in the Koulikoro Region in three health districts. The main accomplishments of the project include: increased commitment on the part of the MOH to maternal and child nutrition, tangibly demonstrated from the national to health facility level; greatly strengthened nutrition supplementation and counseling activities within the community health centers/CSComs; greatly increased coverage of pregnant women with iron-folate supplementation, and of children 6-59 months with VAC supplementation through mass campaigns; and ongoing radio broadcasting on priority project nutrition/child survival topics though the network of local, private radio stations. Project objectives address health facility level services, community-based services and household/individual nutrition-related behaviors. Mid-term evaluation (MTE) data suggest that considerable progress has been made toward accomplishment of the health facility level objectives related to improving nutrition/child survival services. Regarding the DIP objectives dealing with the provision of community-based services though NGO-supported community￾based distribution (CBD) agents, these activities have not yet been initiated but there are plans to do so on a limited scale in Phase II of the project. Regarding behavior change objectives at the household level, positive changes in individual practices appear to be limited but this can be partially explained by delays in implementing NGO-supported community activities. Major constraints encountered in implementing project activities include: the discrepancy between a considerable number of DIP population-based behavior change objectives and the project design, which does not provide for intensified behavior change strategies in all project areas and therefore limits coverage; delays in implementation of NGO-coordinated behavior change activities at the community level; the absence of a clear and effective system of monitoring and documenting project-supported activities, which would contribute to ongoing program learning and decision-making; and inadequate skills on the part of project staff to coordinate community mobilization approaches to behavior change. All these elements will be addressed in Phase II of the project. The strong commitment to capacity-building which all project/HKI staff have is consistently observed through the approach adopted by CSP staff which consists of “providing support” but not “taking over.” In the first two years of the project considerable progress has been made toward meeting sustainability objectives at the regional, district, CSCom and NGO levels. At the community level, anticipated activities with community health associations/ASACOs, NGOs and community volunteers are just being initiated so it is too early to assess their effectiveness or impact. Priority recommendations for Phase II of the CS-15 project include the need: to identify specialized technical assistance in community mobilization for behavior change; to finalize agreements with NGO partners and ensure regular follow-up of their nutrition/CS promotion activities; to systematically involve grandmothers (muso koroba) in all N/CS activities given their role as special advisors and supervisors at the household level in all matters related to women’s and children’s health; to provide training and regular follow up to ASACO members on both management and N/CS promotion responsibilities; to develop the use of simple, participatory behavior change activities in order to optimize learning and changes in household/women’s N/CS 6 practices; to develop a comprehensive but simple system for monitoring, sharing and learning from project implementation; and to provide gender training to all project staff and NGO partners. The MTE was carried out using a participatory stakeholder-driven methodology in which 26 project collaborators were involved. As contrasted with traditional approaches to evaluation, HKI committed greater time and resources to the participatory approach in anticipation of benefits in terms of both program and organizational learning. Project staff and other collaborators who were part of the evaluation team concluded that the participatory approach was very advantageous. According to MTE team members, the active involvement of project stakeholders at all steps in the process contributed to: increased understanding on the part of stakeholders of the project strategies and their respective roles; broad consensus regarding the strengths, weaknesses and priority actions to be taken in Phase II to strengthen project implementation, i.e. ownership of evaluation results; strengthened interpersonal relationships between project actors which will facilitate future collaboration; reinforced knowledge of the technical aspects of N/CS addressed in the project; and knowledge of basic concepts/approach to participatory evaluation which can be applied in other work-related activities. Dr. Diakalia Koné, the CSP Coordinator stated, “I think this process should be called ‘participatory evaluation and training.’” He concluded that the MTE was cost-effective in so far as it not only satisfied a donor requirement but also had a very strong formative value for project staff and their collaborators. 7 I. TECHNICAL APPROACH A. Overview of the Project Location and target population The project is implemented in the Koulikoro Region in central west Mali in three of the seven health districts (cercles): Kati, Kolokani and Koulikoro. Due to the large population in Kati District, it is divided into two zones, Kati and Ouelessebougou. (In this report reference is made both to the “four zones” and the “three districts/cercles.”) The health and nutrition problems in Koulikoro Region are similar to those elsewhere in rural Mali. Likewise, the status of the health facilities is similar. Out of 77 health centers (CSOM) in the four project zones, only 30 are functional, which represents less than 40% coverage. The CSComs are entirely community managed (by the ASACOs), although they are supervised every three months by the district health teams. The total beneficiary population for the project is 175,500 children 0 –59 months of age and 210,600 women of reproductive age (WRA). In the DIP three target groups are identified: 1) Children between 0 and 5 years of age 2) Pregnant and lactating women 3) Other groups of people in the community who influence the behavior of the targeted children and women (men, traditional birth attendants (TBAs), and influential women). While the DIP identifies all three of these groups as “target populations”, the project objectives only define anticipated changes in knowledge/attitudes/practices at the level of women and children. Overall program strategy The overall goal of the project is to assist the Government of Mali and partner NGOs to develop programs that promote good nutrition-related practices at the health facility, community and household levels. Ultimately the objective is to improve the nutritional status of women and children and thereby improve their health and survival. The project is characterized by a capacity-building orientation at all levels and in this spirit all project activities explicitly aim to strengthen MOH, NGO, community and household capacity . Development of the strategy adopted by HKI in Mali to promote CS builds on many of the lessons learned in HKI’s CS XI project that was implemented in Niger from 1995 to 1999. The comprehensive final evaluation of the Niger project generated lessons related to a variety of components and activities, particularly at the district and community levels. HKI’s commitment to building on past experiences and lessons is clearly demonstrated by the fact that the lessons from the Niger CS XI project were systematically used to develop the Mali CS XV project. The project aims to improve nutrition/child survival programs and practices from the national to the community level. More specifically, it seeks to bring about sustained changes in skills and programs implemented by the MOH and the Ministry of Social Development (MSD), communities themselves (CSComs/ASACO), local NGOs, private local radios and traditional communicators. The table below summarizes the objectives and activities supported by the project at each level. B. Objectives and Progress by Intervention Areas In the technical approach adopted in the project, 90% of the effort focuses on nutrition and nutrition-related activities and 10% addresses diarrheal disease management. The interventions are being implemented in two phases. In the first two years of the project activities related to vitamin A, iron-folate, iodized salt and diarrheal disease management were introduced. In the 8 second phase of the project the breastfeeding and complementary feeding activities will be developed. Estimated Program Effort by Intervention General nutrition (including complementary feeding) 20% Vitamin A 30% Iron-folate and iodized salt 30% Breastfeeding promotion 10% Diarrheal disease management 10% i. Progress toward accomplishment of DIP intervention objectives During the MTE, in-depth/qualitative interviews were conducted not only with WRA but also with older women/grandmothers, community leaders, ASACO members, various levels of health workers, project staff and other partners. In addition, we interviewed a convenience sample of 56 men and 67women (intercept interviews) on a few key knowledge and practice parameters. Based on the information collected from these various sources, it is possible to formulate some conclusions regarding progress toward the accomplishment of DIP objectives and on the effectiveness of the interventions being implemented. ii. Effectiveness of CS interventions a. Vitamin A DIP objectives 80% of children 12-59 months old received a VAC in the last 6 months 50% of mothers who attended prenatal services at CSComs with minimum packet of activities (MPA) receive a VAC within 40 days of delivery 30% of mothers who delivered in communities with CS CBD programs receive a VAC within 40 days of delivery 95% VAC coverage among children diagnosed with chronic diarrhea, measles, severe PEM, and xeropthalmia at CSComs with MPA Combined HKI Food Frequency scores for consumption of vitamin A rich foods among preschool children 12-71 months old increases by 1.5 days Combined HKI Food Frequency scores for consumption of vitamin A rich foods among women of reproductive age increases by 1.5 days Baseline results Measurement <15 km from functional health center >15 km from functional health center % of children 12-59 months old who received a VAC in the last 6 months 79.1% 88.5% % of mothers who received a VAC within 40 days of delivery 7.5% 5.4% HKI/FFM combined score for vitamin A rich foods: children 12-71 months old 3.4 days/week 2.9 days/week HKI/FFM combined score for vitamin A rich foods: women of reproductive age 3.35 days/week 2.85 days/week 9 · Supplementation of children and women post-partum The project has supported three strategies to increase distribution of vitamin A to children 6-59 months of age: 1) Regional Micronutrient Days (JRM); 2) systematic prescription of vitamin A to both sick and healthy children during consultations at health facilities; 3) recording of all prescriptions of vitamin A by health facility staff. At the same time, the project has increased public awareness of the importance of vitamin A supplementation through several types of communication and education activities: 1) radio programs; 2) presentations by groups of traditional communicators; 3) health education sessions at health facilities; and 4) improved counseling of women during health facility visits. · Increased supplementation of children 12 months to 5 years of age Regional MOH officials state that the JRM’s have significantly contributed to increasing VAC distribution to children in this age group. According to MOH statistics, in the first JRM 107% of children in the target age group (6-59 months) received vitamin A supplementation whereas in the third JRM 110% received the supplement. MOH officials state that the “inflated” percentage of children who received vitamin A capsules is probably due to: a) an under-estimation of the population of children in the target group (due to margin of error or out-of-date census data used for the denominator); and/or b) administration of the supplement to some children who were either under 12 months or over 5 years of age. In regard to supplementation of children at health facilities, according to both District Management Teams (DMT) and health facility staff, the supplementation of both well and sick children has become a routine activity. They state that both the training supported by HKI and supervision of health facilities have contributed to the systematic prescription of vitamin A that, in the past, was inconsistently administered. A simple system for monitoring the number of children seen in health facilities who receive VAC has recently been established, however, this information is not yet being tabulated on a regular basis. · Increased supplementation of women post-partum According to the baseline data, at the outset of the project very few women were receiving VAC after delivery (approximately 7%). The main strategies adopted by the project to increase supplementation of women post-partum are: 1) support for the JRM; and 2) training and supervision of health facility staff, mainly nurses and trained midwives. The JRM have greatly contributed to increased VAC supplementation of post-partum women. According to MOH statistics, in the first JRM 139% of breastfeeding women received vitamin A capsules whereas in the third JRM 40% received them. According to project staff the data from the first JRM are inflated due to under-estimation of the number of post-partum women (denominator). Regardless of the problems with these figures, regional MOH officials are confident that the JRMs have made a significant contribution to increasing VAC coverage of post￾partum women. Regarding the administration of VAC after delivery, according to regional, district and CSCom level staff, post-partum supplementation was not systematically done prior to the HKI-supported training on nutrition/micronutrients. In the project baseline survey less than 10% of the women interviewed reported having received VAC within 40 days after delivery. Health workers at district and facility levels state that VAC distribution has become a routine procedure with all women who deliver in the CSComs. These improvements in health workers’ practices appear to be attributable to the HKI-supported training, the new and simple recording system on micronutrient administration put in place with HKI support, and the follow-up provided to health workers both by DMTs and CS project staff. 10 Unfortunately, according to the recently published DHS-III (2001) data1 , less than one third (29%) of women in rural Mali give birth in formal health facilities where VACs are available. In the DIP it is stated that community volunteers will be involved in distributing vitamin A capsules to women who give birth outside of the formal health structures. For Phase II of the project there are plans to pilot community-based distribution of VAC in a limited number of NGO-supported communities through community volunteers or TBAs. The results of the MTE intercept interviews show that about one quarter of WRA (27%) and 39% of men know that VAC supplementation is important for post-partum women. The local, project￾supported radio stations are broadcasting frequent messages regarding the importance of VAC for women after delivery. The difference in the levels of knowledge of women and men can probably be explained by the fact that men listen to the radio much more frequently than women. · Consumption of vitamin A-rich foods by children and pregnant women While the results of MTE focus group interviews suggest that there has been some improvement in women’s knowledge of Vitamin-A rich foods, it appears that the increase is limited. The importance of consuming these foods has been disseminated through health workers, radio messages, traditional communicators and NGO-supported community volunteers in a few areas. While it appears that health workers are providing more advice to women on micronutrient-rich foods than was the case before project activities began, it does not appear that all of them are providing this advice. In addition, in many cases, it seems that they inform women of the list of foods that they and their children should consume but do not discuss with them if and how they might access these foods. Many women report that they face economic constraints in this regard. In addition, in many places the recommended micronutrient-rich foods are not available all year round. b. Iron-Folate Supplementation DIP Objectives 67% of pregnant women received 90 iron-folate tablets 85% iron supplement adherence by women who received supplements from CSComs with MPA 85% iron supplement adherence by women who received supplements from CBD agent 95% iron supplement coverage among children diagnosed as severely anemic by pallor at CSComs with MPA Baseline results Measurement <15 km from functional health center >15 km from functional health center % of pregnant women who attend prenatal services and receive iron-folate tablets 39.6% 31.9% DIP objectives address both increased prescription of iron folate to pregnant women and compliance by women who receive tablets from a CSCom or a CBD agent. Project activities to promote increased iron supplementation have attempted to impact both the supply and demand for this activity. On the one hand knowledge and skills of health workers related to micronutrients/nutrition have been strengthened both through training and through regular supervision visits by DMTs and project staff. On the other hand, to increase community 1 Preliminary results of the DHS-III were released in Sept. 2001. Final results will be published in Dec. 2001. 11 awareness of the importance of iron supplementation, information has been disseminated through radio messages, presentations by traditional communicators (TC), and NGO-supported community health volunteers. At baseline approximately one third of women interviewed stated that they received iron tablets during their pregnancy. The MTE intercept interviews suggest that almost three quarters of women (70%) are now aware of the importance of iron supplementation. Men’s knowledge of the importance of iron supplementation for pregnant women also appears to have greatly increased from approximately one quarter at baseline to more than one half in the MTE intercept interviews. This and other qualitative MTE data suggest that awareness of the importance of iron-folate supplementation has increased. This is probably due to increased counseling and prescription by health workers, to frequent radio messages, and to a lesser extent to the performances by traditional communicators. Prior to the CS project activities, iron tablets were prescribed to some extent to women during prenatal consultations. However, according to DMT and CSCom staff the prescription of iron has become much more systematic, and apparently all of the functioning CSComs now have a regular supply of the tablets. CSCom records of micronutrient prescription, initiated with project support, suggest that iron is being systematically prescribed. However, micronutrient activities are not yet systematically analyzed by the DMT during their supervision visits to each CSCom every three months. Women’s attendance at prenatal clinics (PNC) is relatively limited. According to the preliminary DHS-III data, in rural areas only about half of all pregnant women (53%) participate in any PNC. Given the relatively limited attendance at PNCs, in the DIP it is anticipated that CBD agents will distribute iron at the community level. CBD activities are to be initiated in the second phase of the CSP project, although they will only be carried out in about one quarter of the project area as a pilot approach. Regarding the DIP objectives related to compliance, it is considered difficult to assess and no attempt was made during the MTE to evaluate changes in compliance. HKI/Mali is currently conducting a study to determine factors which contribute to non-compliance by pregnant women and study results will be used to improve counseling and messages provided to pregnant women/households. For the final project survey, adequate preparation will be given to the evaluation team so that the two compliance objectives are assessed appropriately in the final evaluation. c. Iodized salt DIP Objectives 95% of salt samples from market tested for iodate and iodide every six months by agents in each CSComs with MPA 80% of households use iodized salt Baseline results Measurement <15 km from functional health center >15 km from functional health center % of households using iodized salt 56.8% 56.8% The CSP aims to increase both the supply of iodized salt on the market and the demand for it by households. To increase the supply CSP activities have included testing salt sold in local markets and raising awareness among salt sellers. In order to encourage market salt sellers to only sell iodized salt, local MOH and MSD workers along with CSP supervisors are expected to periodically test the quality of salt being sold. Unfortunately, the first set of salt testing kits provided by UNICEF 12 expired only a few months after distribution, and since February 2001 salt testing has been suspended. The CSP is waiting for UNICEF to replace the expired kits in order to ensure bi￾annual testing of salt being sold. There has been a significant increase in salt sellers’ knowledge of the nutritional value of iodized salt. At present the vast majority are aware of the nutritional value of iodized salt. Most say they learned about it from the radio. However, many salt merchants are still not selling it primarily because much of the salt sold by wholesalers in Bamako is not iodized. Regarding household level salt consumption, in the baseline survey in which women were interviewed, approximately 60% of households were reported to be already using iodized salt. Based on MTE intercept and focus group interviews it is estimated that at least half of the households are currently using iodized salt, which suggests that consumption has not significantly increased. In most families men are responsible for purchasing salt for their families and the MTE intercept interviews revealed that almost three quarters of the men (72%) and almost half of the women (47%) know that it is important to consume iodized salt. In most households men are responsible for purchasing salt for domestic use so it is beneficial that their level of awareness is quite high. It appears that men’s greater knowledge of iodized salt is attributed primarily to the fact that they have had greater exposure to the local radio broadcasts promoting its consumption. While it appears that knowledge of iodized salt is increasing, particularly among men and salt sellers, it is not clear how accessible it is to households in local markets, due to the testing constraints identified above. d. Management of diarrheal disease DIP Objectives (for children) 50% of cases of diarrhea presenting at CSComs with MPA receive ORS 40% of cases of diarrhea seen by CS CBD agents receive ORS and are referred to local CSComs 25% of cases of diarrhea receive ORS 75% of cases of diarrhea given as much or more food than usual Baseline results Measurement <15 km from functional health center >15 km from functional health center % of children with diarrhea who receive ORS: 12-71 months old 8.2% 9.3% % of children with diarrhea who receive as much or more food than usual 65.6% 66.7% For improved management of diarrheal disease, DIP objectives focus primarily on increased use of ORS. In the baseline study, very few women reported giving ORS during a previous bout of diarrhea to their infant (approximately 5%) or to their preschool child (approximately 10%). In the MTE several constraints were identified with the focus on ORS use. ORS packets are rarely available outside of the health facilities, and it is clear that most cases of diarrhea are treated at home and not seen in health facilities. MOH policy for early home treatment of diarrhea recommends that home fluids be given if ORS is not available. It appears that the knowledge of both health workers and community members has increased regarding the importance of giving ORS to children with diarrhea. However, constraints are identified with this knowledge/advice. Health workers and community members give insufficient attention to the importance of fluid intake during diarrhea. In both the MTE focus groups and intercept interviews the majority of both men and women state that “when a child first has diarrhea he/she should be given ORS.” However, in most of these sites ORS packets are not 13 available. Even for the 45% of the population that lives within 15 km of a health facility/CSCom, in the case of diarrhea the first recourse at the family level should be home treatment focusing on giving increased/large quantities of home fluids. ORS is usually not necessary at the outset of diarrhea. Only about 10% of both men and women stated that “large quantities of fluids” should be given. The MTE team concluded that in both health facility and community level CSP activities insufficient emphasis has been put on promoting the importance of giving increased/large quantities of home fluids at the outset of diarrhea and prior to signs of dehydration. e. Promotion of breastfeeding and complementary Feeding DIP Objectives 80% of mothers offer the breast to their infant within the first four hours after delivery 20% of infants 0-4 months old are exclusively breastfed 60% of infants 6-9 months old are introduced to complementary foods Baseline results Measurement <15 km from functional health center >15 km from functional health center % of newborns who are put to the breast within the first four hours after birth 67.7% 69.5% % of infants exclusively breastfed for the first four months 13.5% 10.8% % of infants 6-9 months old who receive complementary foods 52.8% 50.7% During the DIP review it was recommended to HKI that a phased approach be used to introducing/developing the various nutrition/CS topics. For this reason the technical interventions are being developed in two phases, corresponding to the first and second halves of the project implementation period. It was decided to deal with breastfeeding and complementary feeding in the upcoming second phase of the project. For this reason, no activities have yet been developed related to these two interventions. iii. Changes in technical approach No substantive changes have been made in the technical approach. However, the MTE did reveal the need for a shift in focus related to the advice given regarding management of diarrhea at home. It was also agreed during the MTE that the malaria component which was being considered would not be developed. · Home management of diarrheal disease DIP objectives deal with increasing ORS use and ensuring feeding during diarrhea. In terms of fluids, while in the DIP the focus is on promoting use of ORS, given MOH policy and the constraints associated with this approach (discussed above), the MTE conclusion was that in the second phase of the project messages and counseling need to focus on “increasing intake of home fluids” during early stages of diarrhea, rather than on ORS. It is alarming that only 10% of household members presently know that the first priority is to give a child with diarrhea large quantities of fluids to avoid dehydration. Unfortunately, the DIP does not include an objective on increased fluid intake but it seems important to assess this parameter in some way, both in periodic monitoring activities and in the final evaluation. 14 · Malaria In the DIP it is stated that the CSP should consider developing a malaria intervention, given the prevalence of malaria in the Koulikoro region and the significant impact of malaria on anemia. The DIP stipulates that additional resources would need to be identified in order to develop this component. The MTE team concluded that, particularly in light of the time required to develop community-level activities with NGOs, it is not advisable for the malaria intervention to be developed in the second phase of this project. Should the CSP be accorded an extension it might be appropriate to envision incorporating this intervention at that time. C. New Approaches and Operational Studies Regional Micronutrient Days (Journées Régionales de Micronutriments/JRM) Based on HKI’s experience organizing mass micronutrient distribution activities, particularly in Niger, a pioneer experience for Mali was carried out in the Koulikoro region with support from both the CSP and the MI project. Since 1998 vitamin A has been distributed during national immunization days (NIDs) to children 6-59 months of age. HKI has provided support to the MOH to organize JRM six months after the NIDs in order to provide complete VAC coverage to children as well as to increase VAC coverage of post-partum women and iron-folate coverage of pregnant women. An important characteristic of HKI’s approach to development of this activity is the fact that it is organized at the regional, rather than national level, and that the ownership and organizational capacity have been developed at that level. From the national to CSCom levels, health sector staff express a strong sense of satisfaction with this activity in so far as it facilitates distribution of vitamin A and iron supplements to a large number of women and children who do not regularly participate in activities at health facilities. At the central level, MOH officials state that based on the positive experience in Koulikoro they hope to use this approach throughout the country. According to national level MOH officials, the MOH is currently trying to identify international partners who will provide support for this activity to be organized in other regions. A valuable complementary activity to the June 2000 JRM was a follow-up study to evaluate compliance of women who received iron supplements and obtain their feedback on the organization of the JRM. Conducted three months after the JRM, the study aimed to determine the percentage of women who had taken at least half of the 90 iron tablets received during the JRM. The results of the study showed that of the 56% of the target population of pregnant women who received the iron supplements during the JRM, 69% of them took at least half of tablets. This is very encouraging when compared with pre-JRM levels of coverage of approximately 30%. Partnerships with ten local NGOs for community health promotion As a technical assistance organization, HKI does not aim to provide services directly at the community level. Given the proliferation of local NGOs in Mali, the project has pursued a systematic process to identify a number of NGOs which have some previous experience in health/nutrition and who are already working in one of the four project zones. NGOs were asked to submit proposals to the CSP for funding to allow them to reinforce and expand their nutrition/CS activities. For this reason, the project budget has allotted $60,000 for sub-grants to local NGOs. The process of identifying potential NGOs, studying their proposals and developing formal agreements with each of them has taken longer than expected. No contracts have been signed as of yet. A recommendation of the MTE is that the CSP should aim to finalize agreements with the ten NGOs as soon as possible 15 The NGO partnerships will enable the project to implement ongoing nutrition/CS promotion activities in approximately 25% of the project villages. It is important to note that most of the NGOs are quite young, i.e. have limited experience implementing community health/nutrition programs. The MTE revealed that there is a need to strengthen the community level programs of most, if not all, of the NGOs, not only in terms of technical N/CS competencies (as anticipated in the DIP) but also in terms of the approach they use to working with communities. In order to provide this assistance to the NGOs the CSP will need to access specialized technical assistance, as the necessary skills required for this task do not exist within the CSP team. Based on this need identified in the MTE, HKI is considering two possible strategies for accessing specialized expertise in this area: either hire an additional staff person with these skills, or obtain assistance from the Malian Groupe Pivot/Santé Population, a well-respected umbrella organization of PVOs. Organizational assessment and strengthening of community health associations (ASACO) The MOH in Mali has adopted a strategy that involves the decentralization of responsibility to the ASACOs for the management and financing of health services at the local level in each aire de santé. Each of the CSComs is managed by an ASACO. Unfortunately, many of the CSComs are not functional, often due to poor management by the ASACOs. An important component of CSP efforts to strengthen CSCom N/CS programs is the organizational assessment and training of the ASACOs. The CSP identified a local NGO, ACOD (Association Conseil pour le Développement), which is well-known in Mali for its work in capacity strengthening of community organizations, to conduct an initial assessment of the ASACOs and secondly, to organize training activities for them. The study has just been completed and clear recommendations have been made regarding the proposed training. The CSP should arrange for the ACOD-supported training of ASACOs to take place as soon as possible. It is also critical that CSCom, DHT and project staff provide ongoing follow-up and support to the ASACOs once they are trained. Involvement of traditional communicators Innovative and culturally-based communication activities to promote priority N/CS topics have been carried out by traditional communicators. In many villages groups of actors, singers and musicians already existed and they were identified as local resource persons who could be involved in disseminating the N/CS messages. Approximately half of the project-supported villages have already benefited from community presentations by the local groups. These events have been very much appreciated by local community members however the large number of people who attend and the comical nature of their performances appear to detract from the impact they have on people’s knowledge and practices related to N/CS. The MTE identified a number of ways in which the educational value of these performances can be increased. Collaboration with grandmothers (muso koroba) One of the community strategies described in the DIP for promoting changes in N/CS norms and practices is the training of “influential women.” This category of women includes both “experienced grandmothers” (muso koroba) and “dynamic women’s group leaders” (muso nyamogo). While the importance of involving women’s group leaders was obvious to CSP staff and their partner health/social development workers, through the interviews conducted during the MTE the importance of involving the grandmothers/muso koroba became much clearer. Results of interviews with health workers, WRA, older women and community leaders clearly showed that these older women play a leading role in all household decision-making regarding health/nutrition. According to cultural norms, they are responsible for advising and supervising women during pregnancy, before and after childbirth and for providing care and advice regarding care of the newborn and infant in the first months of life. The MTE team concluded that for all CSP 16 interventions related to vitamin A, iron, management of diarrheal disease, and the diets of infants and pregnant and breastfeeding women, both younger women and husbands seek their advice and assistance. In addition, all of the grandmothers interviewed stated that they are very interested in learning more about the “modern ideas” regarding Nutrition/Survival of women and children. For these reasons it was concluded that in the second phase of the project it is essential that considerable importance be given to involving them in community N/CS activities. Collaboration with private/local radio stations In recent years there has been a proliferation of private, local radio stations in Mali. Currently there are more than 200 of them in the country. In the project zones approximately 75% of the villages have access to the broadcasts of the local stations. The project has taken advantage of this excellent, and relatively low cost, means of communication and has signed memoranda of understanding with seven of the nine stations. Training and follow-up of the stations has lead to accurate broadcasting on all of the priority CSP topics/messages. The involvement of the local radio stations in promotion of the CS/N topics has been very beneficial, particularly in the large number of communities in the project zone (approximately 62%), which do not have access to functional health facilities. Educating salt merchants An important component of the strategy to promote increased consumption of iodized salt involves dissemination of radio messages specifically targeting salt merchants to educate them on the nutritional advantages of iodized salt and to encourage them to sell it. The MTE showed that this has been an effective strategy that, along with efforts to increase public awareness, has contributed to increased availability of iodized salt. II. CROSS-CUTTING PROGRAM APPROACHES AND LESSONS LEARNED The overall project strategy aims to strengthen capacity to develop and implement nutrition/CS programs at several levels, namely, of the MOH at the national level; of the Regional Health Directorate in Koulikoro; of the district management teams in the three targeted districts/cercles; of health facility staff (CSCom and rural dispensaries); of community health associations (ASACO) that manage the CSComs; and of other community structures and actors. In addition, at the family level the project aims to strengthen the knowledge and practices of household actors related to nutrition/CS. In this section project-supported strategies and activities at each of these levels are discussed, accomplishments and constraints related to each are described, and based on the MTE findings, lessons for future programming are reported (in the boxes). A. Capacity Building Approach i. Strengthening national nutrition/CS policies and programs While the focus of CSP interventions is at the community, district and regional levels, the project also aims to advocate for appropriate nutrition/CS policies and programs at the national level. According to central level MOH officials, the CSP coordinator has made a significant contribution to the definition of national policies and priorities related to N/CS through his regular and consistent participation in national level planning meetings, and through informal advice he provides to MOH officials on an ongoing basis. The effectiveness of this strategy stems to a great extent from the fact that the CSP coordinator is a very knowledgeable and well-respected figure in pubic health in the country. His unimposing, collaborative style has undoubtedly contributed to his effectiveness as an advocate for N/CS and advisor to MOH officials at the national level. 17 Lesson learned It is important for CS projects that focus their activities at the district, service delivery and community levels to also attempt to influence national level policy-making and program development through participation in national level planning forums and other advocacy efforts. This can best be done through the establishment of a close working relationship between the project coordinator/s and national level MOH officials ii. Strengthening regional and district nutrition/CS programs At the regional and district levels, CSP activities aim to strengthen N/CS strategies within regional level plans, and strengthen the ability of district level teams to plan, implement, and supervise the N/CS components of their own programs. · Involvement of regional MOH/MSD staff in development of the CS project As a means of developing a sense of ownership of the CSP project from the outset, a positive initial step was the involvement of regional MOH/MSD officials in a series of consultative meetings. Regional officials identify this as a significant step in establishing a collaborative working relationship with HKI. Lesson learned In order to establish a collaborative working relationship and to develop a sense of ownership on the part of regional and district officials, in the initial stages of CS project development it is essential that substantive discussions/planning meetings be held with them to elicit their priorities and suggestions regarding project targets and strategies. · Heightened commitment to nutrition issues: Prior to the CSP, the level of awareness of and commitment to addressing nutritional issues related to women’s and children’s health was relatively low among MOH and MSD officials. There is now a consensus among them that their understanding of these issues has greatly increased, as has their commitment to addressing them. Project-supported activities that have contributed to heightened commitment to maternal and child nutrition include: the baseline survey and several other project-supported studies which have clearly documented nutrition-related problems; involvement of MOH and MSD staff in conducting both the baseline survey and several small studies on other nutrition topics; training of most MOH and MSD staff on nutrition/CS; and ongoing involvement of CSP staff in regional and district planning and coordination meetings. Lesson learned In order to increase the commitment of regional and district level officials to new N/CS issues and concepts, ongoing and various efforts need to be made to share information with them and to help them incorporate the new concepts into the context of their ongoing programs. · Non-directive, partnership approach adopted by the project: “Projet d’appui” The DIP clearly states that a “partnering” and “capacity building” approach will be adopted in working with regional and district level health/social development structures. MTE interviews revealed that regional MOH and MSD officials have a very favorable impression of the approach used by project staff to provide collaborative support to the region, and they describe the project as a veritable “projet d’appui”. They stated, “As compared with other donor-funded projects/programs the CSP has been carried out in a much more collaborative and less top-down fashion.” The collaborative, non-directive fashion in which project support has been provided to the regional and district level structures is identified as a very positive aspect of project implementation which has contributed to ownership and capacity building on the part of MOH/MSD staff. 18 Lesson learned Most donor-funded development programs/projects state that they collaborate with and support government structures and staff. However, in many cases the degree of collaboration is limited and programs/projects often operate quite independently and/or outside of government plans and programs. If externally-funded projects sincerely aim to support the development of government strategies and capacity, all project staff must be committed to these concepts, must invest considerable effort in developing close relationships with government partners based on frequent contact and communication, and must be skilled at providing non-directive support for decision￾making rather than initiating and managing activities independently. · Access to the project document/DIP Regional MOH and MSD officials identified a constraint regarding access to the DIP document. Although they received copies of the DIP in English shortly after it was approved, they did not receive the French translation until much later. Furthermore, there is a consensus among both government and CSP staff that the French translation is extremely difficult to understand. This has meant that project partners have not had access to a detailed description of project objectives and strategies. Lesson learned In all CSP projects it is important that the DIP be made available to project partners as soon as possible, and in their working language in order to facilitate their understanding of project targets and strategies and to strengthen collaboration with project staff. iii. Support for regional N/CS program development Support for regional level planning has taken place primarily through the ongoing involvement and contribution by the project coordinator to virtually all planning and coordination activities at that level. The coordinator is, in fact, a special advisor to the regional MOH team in all matters related to N/CS. Clearly, this special status has enabled him to have considerable influence, both formally and informally, on regional level planning and coordination efforts. · Increased coordination between MOH/MSD staff and partner organizations Although there is a regional coordinator for NGO-MOH/MSD affairs, in the past there has never been an occasion for NGOs to meet and share their experiences related to community health promotion. A very positive contribution made by the CSP at the regional level has been its efforts to promote sharing and coordination between the various local NGOs, international organizations working in the area, and MOH/MSD staff. The CSP helped to organize the first meeting between these various organizations, which was found to be very useful to NGO and government agents alike. · Stipends for regional counterparts A decision was made by HKI during the project development phase to provide two key regional counterparts from the MOH and MSD with a regular stipend (i.e., topping up their government salaries) for coordination of project-supported activities. This practice goes against attempts being made by the CSP/HKI to develop ownership for the N/CS activities on the part of regional officials. This has created a precedent, which is difficult to change at this point, however, future HKI programs will avoid adopting such an approach which can be counterproductive in terms of sustainability and promoting organizational commitment to project-supported strategies. 19 Lesson learned Given HKI’s institutional-strengthening philosophy that aims to strengthen existing government programs and staff, future HKI projects/programs should not commit themselves to providing stipends to government officials. This practice may create an inappropriate and unsustainable source of motivation for government partners. Other types of motivation should be used which do not create a dependency on the resources of external organizations. iv. Support to district level management teams (DMT/ESSC) and district programs Project efforts to strengthen district level N/CS programs have consisted mainly of: staff training on N/CS; technical inputs into district program planning; support for the JRMs; and support for supervision activities. At the district level, the four project supervisors have primary responsibility for coordinating CSP activities with the DMT. · Training in nutrition/child survival All of the DMT members state that the N/CS training organized with CSP support was beneficial in so far as most of their staff members had very limited previous training in nutrition. Apparently the curriculum of the national health/social development training schools deals with nutrition in only a very superficial way. Although the training appears to have greatly increased DMT knowledge of nutrition, MTE findings suggest that there is a need to continue to reinforce and expand their knowledge. There is a tendency to assume that formal training workshops are required to reinforce their learning, and other simpler strategies have not been explored, such as periodically providing them with short and simple documents like those given to MOH/MSD staff on both technical and programmatic topics. Lesson learned In Mali, as in many countries, health and social development staff’s knowledge of nutrition is generally limited, and structured training is necessary to provide them with the basic concepts. In order to continue to reinforce and expand their understanding of nutrition and commitment to developing it in their programs, it is important to develop a mechanism for providing short and simple nutrition-related information to DMT staff on an ongoing basis. This should include information on community nutrition strategies developed both in Mali and elsewhere. · Inputs into district program planning The participation of CSP staff in regularly scheduled DMT meetings contributes to the development of good communication and coordination of DMT activities related to N/CS, and encourages the integration of CSP-supported activities into DMT planning. Lesson learned Reinforcement of N/CS activities in district programs requires involvement of CSP staff in all DMT planning exercises. · Support for JRMs Overall, the DMTs are very satisfied with the support received for the organization of the JRMs (as discussed above.) In some cases resources for this activity were received quite late at the district level, which was quite problematic for both district and regional staff. Lesson learned In the organization of future JRMs it is important that all allotted resources arrive at the district level at least 15 days ahead of time in order to enable DMTs to dispatch and coordinate use of those resources in an organized and timely fashion. 20 · Support for supervision of CSComs In Mali’s decentralized community health services scheme, CSComs are financially and managerially autonomous, however, they are supposed to be supervised every three months by the DMT. These are “integrated supervision” visits and the project is working to ensure that N/CS activities are systematically addressed during these routine visits. CSP support for supervision of N/CS activities has consisted of: participation by CSP staff in integrated supervision visits; additional supervision visits carried out by CSP staff; and support for one “N/CS-specific supervision.” Lesson learned CSPs that aim to strengthen N/CS activities at the health facility level should provide support to existing mechanisms for supervision of health facilities so that routine supervision progressively addresses N/CS components. Such efforts should contribute to sustaining organizational commitment to supervision of those activities. v. Strengthening Community health centers (CSComs) and rural dispensaries The CSP aims to strengthen provision of N/CS services in the community-managed CSComs, which are officially recognized by the MOH, and in rural dispensaries which are beyond the purview of the MOH. (At the present time there are 33 CSComs in the project area which are operating at an acceptable level of functionality. Others are at various stages of development). All CSP efforts to strengthen CSCom activities are coordinated with DMT activities. In an attempt to strengthen service delivery in areas beyond the CSCom catchment areas (two thirds of the villages in the project zone), CSP staff also provide some support to the dispensaries. CSP support to both CSComs and rural dispensaries consists primarily of training and follow-up supervision visits. · Training Project-supported training sessions on N/CS have been organized in all four project zones and staff in all CSComs (doctors, nurses and midwives) have participated in the two to three-day workshop. In addition, staff from some rural dispensaries have also participated in these sessions. Participants from both types of facilities state that the training activities were well organized and very useful. All CSCom and dispensary level interviewees state that since the training their supplementation and nutrition counseling/promotion activities have increased. While all available evidence suggests that this is true, it is important that ongoing follow-up be provided to those trained in order to reinforce what they were taught and to identify gaps in their knowledge and skills. A specific challenge in this regard relates to the fact that the DMTs are not responsible for supervising dispensary staff. Although CSP staff are carrying out periodic visits to these facilities, no mechanism exists to ensure ongoing supervision after the project ends. Lesson learned The training of both CSCom and rural dispensary staff in N/CS was an important initial step in strengthening facility-based activities related to N/CS. Ongoing supervision visits are required to consolidate the knowledge and skills acquired. DMTs need to try to identify a mechanism to ensure periodic supervision of the dispensaries in addition to the CSComs that are visited every three months. Another mechanism that should be used to provide ongoing support to both CSCom and dispensary staff is to provide them with simple and short documentation on both technical nutrition issues and community nutrition program strategies. · Micronutrient supplies Although vitamin A, iron and ORS are all on the list of essential drugs in Mali, before 2000 vitamin A was not available in most CSComs in the project zone and iron tablets were frequently 21 available but not systematically prescribed to pregnant women. At present all health facilities visited had vitamin A, iron capsules, and ORS packets in stock. CSP activities have contributed to ensuring regular availability of both micronutrients and ORS through the training and follow-up of CSCom staff to increase their awareness of the importance of micronutrients and ORS and of good stock management practices. Lesson learned An essential component of any strategy to increase micronutrient supplementation is to strengthen the ability of the existing health system/facilities to effectively control the stock management and supply of these products. · Monitoring of micronutrient distribution The CSP is advocating for inclusion of indicators on the prescription/administration of iron, vitamin A, and ORS in the national MOH health information system. This concept has been accepted at the national level but has not yet been operationalized in Koulikoro region. In the meantime, the CSP has helped the DMTs put in place a system to record these activities using small notebooks (cahiers). In all CSComs visited during the MTE these activities were being systematically documented and health workers were proud to show that they are regularly providing supplementation to target group women and children. This represents a significant improvement at the health facility level. Lesson learned Strategies to increase vitamin A and iron supplementation should be accompanied by record￾keeping and periodic reporting on these activities. Projects should advocate for inclusion of indicators related to VAC and iron supplementation into existing MOH information systems. vi. Strengthening Community Health Associations (ASACO) At present there are 33 ASACOs which are responsible for managing the 33 CSComs currently operational in the three districts in the project area. The DIP states that the CSP will strengthen the ASACOs in several ways by developing their ability to: advocate for appropriate/ improved nutrition services; identify training needs of CSCom staff; and build links between CSComs and CBD activities. It is important to note that the ASACOs are generally very weak organizations and DIP objectives may be unrealistic in terms of the extent to which CSP support can strengthen their effectiveness during the four-year life of the project. Activities which the CSP has planned to carry out with the ASACOs include a diagnosis of their organizational capacity, and the training of ASACO members in management, planning, N/CS, advocacy and health promotion. To date only the first activity has been realized. · Diagnosis of organizational capacity The CSP contracted a small study of the ASACOs to a local NGO, ACOD, which specializes in organizational analysis and strengthening. The study revealed multiple weaknesses in the structure and functioning of these associations, including: their lack of knowledge of the rules and regulations defining the organization and management of both ASACOs and CSComs; lack of understanding by ASACO members of their roles in health promotion, in addition to their roles in managing CSCom drugs and finances; infrequent association meetings; and the extremely limited representation of women. Based on the weaknesses identified in the study, ACOD is being contracted by the CSP to carry out a series of training activities with the ASACOs. 22 Lesson learned All CSPs that seek to increase community capacity to manage health/ nutrition services should develop strategies to provide incremental and ongoing support to community organizations. Such efforts should include initial analysis of organizational capacity, training, and follow-up support. If this component is to be addressed in a serious way, specialized expertise is usually required given that more clinically-oriented project staff rarely have the specialized skills required for this task. vii. Strengthening NGO capacity In order to bring about changes in nutrition/CS practices at the household and community levels, a major project component involves partnering with local NGOs in order to strengthen and expand their N/CS activities with communities, including CBD of micronutrients & ORS. Planned CSP activities include: identification of viable NGO partners; training of NGO program coordinators in N/CS; regular communication with NGO partners; and periodic follow-up visits to NGO-supported communities. · Identification of NGO partners An initial and time-consuming step involved the identification of viable NGO partners. Although there are numerous NGOs in Mali, the quality of their work varies considerably, partly as a function of the length of time that they have existed as organizations. This process has taken somewhat longer than expected in part due to the fact that apparently it was not made sufficiently clear to the NGOs what type of activities/expenses could and could not be funded in the context of the CSP. However, according to the CSP coordinator, in the near future agreements/contracts will be signed with several of the identified partner organizations so that their community level N/CS activities can be reinforced/expanded as planned. . Lesson learned The process of identifying viable local NGO partners can be time-consuming. In eliciting proposals from local NGOs for collaborative activities it is important to be as clear as possible regarding the types of activities/equipment which the project is prepared to support so that NGO proposals are in keeping with project funding and programmatic parameters. · Training of NGO program staff in nutrition/CS Although written agreements have not yet been drawn up with each of the partner NGOs, all key NGO program staff have participated in CSP-supported training sessions on N/CS. In MTE interviews conducted with staff members of all NGO partners, they stated that the training on N/CS has been of great value to them in so far as it has provided them with current and in-depth technical information on major key maternal and child health nutrition issues. Lesson learned In establishing partnerships with local NGOs it is essential to provide them with up-to-date technical information on the maternal and child nutrition/ health issues and interventions supported by the project. This should be done through formal training sessions and reinforced through dissemination of short and simple documents on both technical aspects of N/CS and community experiences from other places in Mali as well as from elsewhere in Africa. · Mechanisms for sharing and learning with NGOs NGO partners suggest that HKI organize periodic meetings with representatives from each organization to allow them to share their experiences implementing community level health/nutrition programs and to learn from each other. They suggest that such sessions could 23 provide a forum for discussion of issues of common and critical interest to them, such as “sustainable motivational strategies for community volunteers.” Lesson learned The project can play an important role in encouraging sharing and learning between its NGO partners by organizing periodic “exchange and learning” gatherings involving CSP staff and all ten partner organizations. · Coverage by local NGOs As stated above, NGOs will play a major role in implementing community-level behavior change activities. However, the MTE revealed a constraint associated with NGO coverage and community level behavior change activities. NGO activities will be carried out in only about one third of the total number of villages in the project zone. Furthermore, most of the partner NGOs are working in areas that have access to functioning health centers. This means that in close to two thirds of the project villages intensive nutrition/CS promotion activities, including CBD, will not be carried out. This is problematic in so far as some of the DIP objectives are population based, i.e. refer to all villages in the project zone. Given the fact that community level activities will be much more intensive in NGO areas, it is suggested that for the final evaluation the sample be stratified to include villages with and without NGO activities. Lesson learned In the design of CS projects it is important to differentiate between zones/ sub-populations for which the intensity of the CS interventions will differ and to formulate different objectives accordingly. viii. Strengthening Community capacity In addition to support for N/CS activities carried out in health facilities, the CSP is investing considerable effort and resources into developing the capacity of community groups and individuals to promote N/CS through collaboration with local private radio stations, traditional communicators, salt sellers (discussed above), community health volunteers, and influential community women. Based on MTE results, activities are now also envisioned with community leaders and grandmothers (muso koroba). · Local private radio stations The project has taken advantage of the network of small, private radio stations that exist in Mali by helping the MOH/MSD to develop partnerships with nine of them. Radio broadcasters have been trained, they participated in developing radio messages and received taped messages on all key N/CS topics, contracts have been established with them based on a pre-arranged broadcasting schedule, and all of the radios are regularly visited by MOH/MSD and CSP staff to monitor their N/CS programs. Overall, the technical N/CS content of their programs is accurate, although a few of their messages will need to be modified based on MTE decisions to re-orient certain messages based on community feedback. All private radios in Mali are required to broadcast health and development messages, and radio staff interviewed all stated that the training and materials they have received from the CSP enable them to broadcast accurate N/CS messages. Furthermore, they stated that when the project comes to an end they will continue to use this information and materials in their broadcasts. Feedback from community members shows that radio messages have been heard and have contributed to increased knowledge of N/CS topics. Unfortunately, women report that they have much less access to the radio and less time for listening. 24 Lesson learned Collaboration with local radio stations for broadcasting N/CS messages can be useful in increasing knowledge of key health/nutrition information. It is important to determine the times when women are more likely to listen to the radio and to use songs containing health messages in order to catch their attention and increase the impact of this media on them. · Traditional communicators The project is working with both existing and newly-formed groups of traditional actors & communicators (TAC) to disseminate N/CS information through community level animation sessions. This approach builds on a traditional form of communication that is very much enjoyed by community members. However, the MTE showed that retention of the messages presented in these sessions is generally limited. MTE team members formulated a number of suggestions for modifying the presentation of the TAC sessions in order to increase their educational impact. Lesson learned Efforts should be made to increase the educational impact of the community sessions organized by traditional actors and communicators by limiting the number of topics addressed in each performance, by limiting the size of the audiences, and by eliciting discussion of the topics presented during a break in the middle of the program. · Community health volunteers and CBD of micronutrients and ORS Partner NGOs will be responsible for developing the N/CS promotion activities at the community level using either existing or newly-chosen community health volunteers (CHV). After training, the CHVs will be expected to carry out both educational/promotional activities for women and children related to N/CS and to distribute/sell VAC, iron-folate, and ORS in a cost-recovery system. The MTE identified several existing or anticipated difficulties/ constraints related to the activities of the CHVs, such as: insufficient motivation to carry out their community tasks; absence of a sense of community ownership and responsibility for the CHVs; the decision to select both male and female CHVs to promote maternal and child health without clearly defining the role that male CHVs will play in these activities; the use of traditional, directive health/nutrition education methods by some of the NGOs; and lack of support for CBD on the part of some DMT members. It is hoped that CHV income from CBD cost-recovery activities will be an ongoing source of motivation for them, however, in past experiences in Mali the limited income volunteers receive is often not enough encouragement for them to continue this activity. In order to increase the overall effectiveness of CHV activities, project staff need to carefully examine these constraints with partners and identify ways to try to overcome them Lesson learned Community health volunteers can potentially play an important role in carrying out health/nutrition promotion activities including CBD. However, they must be trained, regularly supervised and be motivated, particularly by factors stemming from within their own communities. If CHV activities are to be sustained, it is essential that the NGOs develop strategies to promote ownership of the CHVs on the part of community leaders and groups. · Influential community women: leaders of women’s groups (muso nyamogo) and grandmothers (muso koroba) In the DIP it is stated that community level N/CS activities will involve “influential women,” however, in the past there was not a consensus on the meaning of this concept. Some project actors defined it as the “women’s group leaders” while others viewed it as the “grandmothers/older, experienced women.” Most of the activities already initiated at the 25 community level focus on the former group of women, or muso nyamogo in Bambara. While it is very important that these women be involved in the N/CS activities, MTE interviews clarified the fact that the grandmothers, or muso koroba, play a special role as preferred advisors on all health/nutrition matters at the household and community levels and that it is extremely important that they be systematically involved in all such activities. Role and Influence of Grandmothers/muso koroba on Maternal and Child Health/Nutrition Summary of findings from Mid-Term Evaluation interviews with grandmothers, younger women, community leaders and health post nurses · Grandmothers (GM) are “special advisors” on family health/nutrition: In Malian society GMs have always played, and continue to play, a leading role in maternal/child health (MCH) matters. Both men and women in the family acknowledge their experience in MCH and refer to them as “special advisors” on all MCH matters. · Advice and supervision to pregnant women: GMs are expected to advise pregnant women on their diet, work, and prenatal consultations. In most cases husbands expect the GMs (their mothers) to supervise the pregnancies and inform them accordingly. Regarding their diet, almost all GMs advise them not to eat too much and to avoid certain foods, “so that they don’t gain too much weight which will make the foetus to big and make the delivery difficult.” Concerning pregnant women’s work, most GMs advise that they avoid the heavy jobs like carrying wood and other heavy loads, however, GMs and younger women agree that this advice is rarely put into practice. · Advice and supervision of the newborn: After delivery GMs advise younger women on whether or not to give the colostrum, when to put the baby to the breast and later on, when and what complementary foods to give the child. · Direct and indirect Influence of GMs on child health: GMs play a two-pronged role in child health. First, they influence the practices of younger women by advising them and supervising what they do with their young children. Secondly, they take special and direct responsibility for young children at the household level, bathing, clothing, caring for them and providing them with food and drink, when their mothers are either busy with other tasks or absent from the courtyard. · Increased role of GMs with the sick child: The role/involvement of GMs with young children in the family is accentuated when the child is sick. Due to GMs experience diagnosing and treating childhood illnesses, when a child is sick he/she spends more time with the GM. In many cases, the sick child sleeps with the GM at night so that she can observe the evolution of sickness and provide timely and appropriate treatment. In most cases initial treatment is provided/supervised by the GM at the household level. If the illness becomes worse, GMs play a key role in deciding to refer the child to a specialist, either a traditional healer and/or a formal sector health worker. · Network of GM health/nutrition advisors: GMs say that when they feel that their treatment approach is not adequate, they consult with other women in their GM network who have more experience than they do with the particular illness. Lesson learned Past MCH programs have tended to focus almost exclusively on women of reproductive age. The MTE clearly showed that the practices of these women are significantly influenced by the opinions and advice of older, experienced women in the community. In all nutrition/child survival activities it is important to involve both the muso koroba, who are recognized as “special advisors” on all health/nutrition matters, and the muso nyamogo, who are the articulate, often younger women who are able to mobilize other women in the community for health/ nutrition related activities. 26 · Traditional male village leaders In the initial qualitative research and behavior change strategy development, the traditional community leaders were not identified as community resource persons who can play a determining role in the success or failure of efforts to promote community health/nutrition at that level. There was a consensus among MTE team members that in the first phase of the project￾supported N/CS activities insufficient effort was invested in establishing rapport with community leaders, educating them on project objectives and strategies, and eliciting their active involvement. Lesson learned In all community level efforts to promote maternal and child health/ nutrition, traditional community leaders should not only be “informed” of program activities, but should be viewed as resource persons and their “active participation” should be sought in informing and mobilizing other community members. Project partners should elicit their suggestions and advice regarding how to organize community level activities in order to increase both community ownership and the chances of sustainability of those activities. · Participatory tools for nutrition/health education As planned, a set of counseling cards on the key N/CS topics has been developed, though not yet printed and distributed, for use both at the community and health facility levels. These will undoubtedly be useful tools for both group and individual counseling activities. In addition, the MTE team concluded that the use of simple and participatory materials, specifically songs, stories and group discussions should be encouraged. These methods are all based on traditional and culturally-adapted forms of communication and, furthermore, can all be used by communities themselves to promote the priority nutrition/CS topics. Apparently the partner NGOs have only limited experience using these types of learning tools. Lesson learned In the development of community-level activities by the NGOs priority should be given to the use of stories, songs, and facilitated group discussions in order to elicit the active participation of community members and increase learning and chances of behavior change. ix. Strengthening the PVO organization In the DIP, one facet of capacity-building involves increasing the capacity of HKI to leverage additional funds to expand nutrition activities. HKI/Mali was able to access significant Canadian MI funds, which are being used as a match for USAID CS funds. HKI-Mali is also currently in discussions with Save the Children regarding a possible new project on Saving Newborn Lives. Another aspect of the capacity building activities undertaken for the PVO were done through the Appreciative Inquiry (AI) method. Such an exercise was undertaken to assess the strengths and weaknesses of the PVO as a whole. HKI headquarters organized an AI workshop in January 2001 in which all HKI headquarters staff participated. The Regional HKI office for Africa also undertook such an exercise and the implementation is now at the country level. In addition to undertaking an AI exercise for its office in Mali, HKI is also planning on undertaking a team￾building exercise for the staff, that will be based on the results of the AI. The HKI headquarters office has also benefited in many ways from its participation in the CORE group activities and this has lead to the development of new partnerships with other PVOs to benefit from their experience in the filed of Child Survival. HKI’s participation in the CORE group has also improved, along with its participation in the Working Groups. The direct implications for the HKI-Mali office have been an increase in communication and technical transfer of information between HKI headquarters and the Mali office. 27 B. Progress Related to Sustainability Strategy Based on the Capacity Building and Sustainability table included in the DIP, the table below indicates the significant progress made toward accomplishment of the sustainability objectives. In the first two years of project implementation progress has been made toward the sustainability objectives at the regional, district, CSCom, ASACO and NGO levels. It is only at the community level that CBD activities have not yet been initiated. Table: Progress Toward Capacity Building and Sustainability Objectives Objectives Progress toward objectives Regional/district level Key personnel in health districts and region are able to plan and implement quality nutrition programs · All key regional & district personnel trained in N/CS · Regional staff assumed primary responsibility for planning and implementing 3 JRMs · Nutrition activities included in annual regional health plan Health region and districts maintain formal coordination mechanisms · Frequent informal meetings between regional staff and CSP staff · CSP staff participate in regularly scheduled district coordination meetings · The regional steering committee has had one annual meeting. HKI/MOH/NGOs will define further funding needs and leverage required resources · For JRM regional health directorate accessed UNICEF resources. District level Districts maintain adequate supervision of CSComs · CSComs regularly supervised by DHT and CSP staff CSCom level CSCom health agents deliver nutrition and diarrhea services at acceptable level of quality · All CSCom workers trained on technical and counseling aspects of nutrition and diarrhea services · Systematic administration of VAC to women immediately after delivery in CSCom · Routine prescription of iron-folate to pregnant women · Systematic distribution of VAC to sick children and well children 12-59 months seen at CSCom CSComs renew stocks of supplements and ORS packets from cost recovery · All CSComs have regular supplies of micronutrients and SRO from MOH cost recovery system ASACO level ASACOs ensure that nutrition and diarrhea services are maintained in the MPA at CSComs · Organizational analysis of ASACOs completed and training planned on both nutrition/CS and organizational management topics ASCACOs can assess adequacy of nutrition and diarrhea services being provided · Planned for Phase II of CSP Local NGO partners Key NGO personnel able to plan and implement nutrition programs · All key NGO personnel trained in nutrition/CS · Agreements with NGOs to develop community nutrition activities to be signed very soon · NGO community nutrition activities will be 28 implemented in Phase II of CSP NGOs maintain adequate supervision of CBD workers · CBD system with NGO volunteer network to be developed in Phase II NGOs leverage additional funding to expand their nutrition interventions · To be explored in Phase II Community Based Distributors NGO CBD agents deliver nutrition and diarrhea services at acceptable level of quality · CBD activities to be developed in Phase II CBD agents remuneration will increase through sales of different products · Remuneration system to be developed in Phase II HKI HKI will define further funding needs and leverage required resources · Addition resources have been leveraged from UNICEF, PLAN and MI for expanded nutrition activities III. PROGRAM SUPPORT AND MANAGEMENT A. Planning and coordination of project activities Common understanding of project objectives/strategies MTE interviews suggest that project staff and local partners have a basic understanding of project objectives and strategies. However, the CSP has not provided all project actors and partners with documentation on project objectives and strategies adapted to both the linguistic and educational capabilities of each category of collaborators. For example, regional MOH/MSD staff have not had access to a clear French translation of the DIP, and CSCom workers have not received a summary of the project. Many MTE team members stated that they would have liked to have had a copy of the "visual framework” of the project, used in the MTE, to help them better understand the project objectives and components. As stated earlier, HKI recognizes the need to provide all project partners with a copy of the DIP in their working language. Regional Steering Committee (Comité de Pilotage) According to the DIP, the regional steering committee is supposed to meet twice a year to oversee all project activities. Since project implementation began in late 1999, the committee has met once. In the future it is planned that these meetings will take place at least twice a year to ensure that key regional actors are informed of project progress and to consolidate their support for the project. District Technical Committee (Comité Technique de District) The DIP envisions quarterly meetings of the District Technical Committee (DTC) to provide technical oversight to the project and ensure its coordination with district programs and staff responsibilities. Since the project began only one DTC meeting has been held in each district. In interviews with regional and district MOH/MSD staff they stated that it is important for regular coordinating meetings to be held. The CSP team will discuss with District Health Team (DHT) partners to determine the most appropriate frequency of these meetings. Coordination with District Health Teams In most cases project-supported activities are planned and coordinated with the DHTs through the regular DHT coordination meetings and through ongoing informal contact/communication 29 between CSP staff and DHT members. However, according to regional MOH/MSD officials, in some cases the project “parachutes” activities that have not been anticipated in district plans. In one of the districts, DHT officials complained that the CS supervisor sometimes makes decisions and organizes activities without consulting them. In this case planning/coordination of project￾supported activities with the DHTs needs to be strengthened. Accomplishments related to Year 1 and 2 work-plan Approximately three-quarters of the activities planned for years one and two of the project have either been fully implemented or are currently being implemented. Planned activities which have not been carried out include: 1) training of the CSP Coordinator in IMCI; 2) Training of ASACO members; 3) assessment of partner capacity to develop action plans; 4) organization of a social mobilization campaign to promote micronutrient supplementation; 5) workshop to identify weaknesses in the Management Information System (MIS) and development of a plan to address those weaknesses; and 6) pre-test the check-list being developed to assess functionality of CSCom. During the action planning exercise following the MTE the CSP team is going to decide when and if each of these activities will be carried out. Please refer to the Action Plan included on page 38 for a detailed view of progress on implementation of workplan activities. B. Training strategy for MOH, NGOs and community partners Development and implementation of the training strategy is a key component in the project. It has involved the development of training manuals, training of trainers, and training of health/social workers at several levels. Development of training modules and training of trainers According to MOH officials, an important contribution made by HKI was the development of two training modules on nutrition developed in collaboration with the newly-created Nutrition Division in the MOH. In the past, in the basic health/social development training schools around the country the development of curriculum on nutrition has been very limited. The modules developed with the support of both the CS and MI projects are the first extensive training modules on maternal and child nutrition that have been developed in Mali. A “training of trainers module” was developed along with a second module for use in “training health and social development workers.” From a pedagogical standpoint the modules are quite innovative in so far as the learning activities proposed for each of the sessions are based on active learning principles and involve the use of a variety of participatory learning methods. In each of the project zones, members of the DHT were selected to constitute a team of nutrition trainers. A five-day workshop was organized to introduce them to the technical N/CS content as well as to basic adult education principles and techniques. The inclusion of the sessions on adult education was a very positive facet of the training in so far as it prepared the team members to use active learning techniques during the subsequent training workshops. In the third quarter of 2001 workshops were organized in all project zones for district, CSCom, and rural dispensary staff. The total number of health sector staff trained in the CSP by category is: 16 DHT trainers, 15 doctors, 90 nurses, 33 nurses assistants, 86 trained midwives, and 31 NGO staff members. The training of ASACO members and, in most cases, of CBD agents has not taken place as scheduled, due to delays in prerequisite activities with both of these categories of collaborators. One of the partner NGOs has begun training community volunteers and 90 people have been trained. Content of training modules The technical content of the nutrition training modules is consistent with the MOH priorities as spelled out in nutrition policy/program documents. However, the MTE team concluded that the content is too narrowly focused on the technical aspects of maternal and child nutrition and does 30 not address a number of broader issues, which would provide trainees with a more comprehensive vision of the relationship between nutrition and health among Malian women and children, and would help them to more effectively promote nutrition at the household and family levels. These insufficiencies in the training modules are reflected in weaknesses in the work of health/social development agents at the facility and community levels. Concepts/information which need to be incorporated into the modules include: 1) Basic notions of women’s and children’s nutrition: The influence of Malian women’s work and nutrition on their health status and on the development of their fetus/newborn. 2) Importance of weight gain by pregnant women: This concept is not covered in the training modules and the MTE revealed that health workers do not advise pregnant women that it is important for them to progressively gain weight during pregnancy. Furthermore, the traditional belief, which is also communicated by many health workers, is that “limited weight should be gained to limit the growth of the foetus and to facilitate delivery.” It seems that health workers understand the importance of micronutrient supplementation of pregnant women but do not understand women’s broader nutritional needs during pregnancy. 3) Socio-cultural and economic factors which influence nutrition practices: There are a variety of socio-cultural beliefs and practices related to maternal and child nutrition (some of which were documented in the CSP qualitative research), which health/social workers need to take into consideration in promoting alternative nutrition concepts with women and communities. Trainees also need to be sensitive to the economic factors that influence nutritional options and practices. 4) Household strategies for diarrheal disease management: The training manual focuses on treatment plans for diarrhea to be followed in clinical settings. Importance should also be given to household management strategies at the outset of diarrhea and emphasize the role that early home intervention can play in preventing dehydration. 5) Administration of home fluids: In the section on management of diarrheal disease the focus is on the use of ORS and there is little discussion of the use of home fluids in large quantities at the early stages of diarrhea. Given that ORS is not readily available to most households, in terms of home treatment at the outset of diarrhea the module should focus on developing the reflex of giving large quantities of available fluids. 6) Community mobilization for promoting N/CS: The module should include some discussion of how health/social workers can communicate and collaborate with local leaders, groups, and influential people, including traditional practitioners, to promote good N/CS practices outside of health facilities. While it should not be expected that health/social workers will develop elaborate community mobilization strategies, they should be encouraged to communicate priority nutritional information to community leaders and groups that they come in contact with, for example, regarding iodized salt and weight gain during pregnancy, and encourage them to promote these important concepts. 7) Simplified module/technical reference sheets: MTE team members felt that the content of the training module is in many cases too complicated for ASACO members and lower level health workers (health assistants, trained midwives) to easily comprehend. It was proposed that a summary, or a series of technical information cards, should be prepared to use during training and to distribute to these groups for later reference. 31 C. Staff management and supervision Personnel management system HKI Mali has not had a personnel performance assessment system in place and, therefore, the project coordinator has not had clearly defined procedures to follow in dealing with personnel performance issues. The new HKI Mali Country Representative is in the process of developing a performance appraisal system for all HKI/Mali employees as part of a broader effort to reorganize the administrative portion of the office. An evaluation of CSP staff is planned for the end of 2001, early 2002. Coordination/monitoring of supervisors’ work Feedback and observations from the four supervisors posted in the four project zones are important mechanisms for monitoring project implementation and for generating lessons to strengthen project strategies. In this regard two weaknesses have been identified related to reporting by the supervisors, and the coordination of meetings with CSP staff. First, supervisors have been expected to prepare written reports on activities carried out in their respective zones only every three months. In addition, the content of their reports is almost exclusively descriptive, in other words, activities reported on are not systematically analyzed in terms of strengths, weaknesses, and lessons learned. The project coordinator has agreed to require monthly activity reports from each supervisor. It is recommended that the prescribed reporting format include both descriptive and analytical information on all activities carried out. Secondly, although the project coordinator has frequent contact with each of the supervisors on an individual basis, regular staff meetings with all supervisors are not held. In Phase II of the project it is recommended that regular meetings of this sort be held to allow greater sharing between project staff, and improved monitoring and coordination of their activities. The supervisors feel strongly that a regular staff meeting will be beneficial, in so far as it will allow them to share experiences among themselves and better coordinate their activities. Staff training and development The four project field supervisors participated in an introductory course on public health and on the project objectives, strategies, activities, and roles. Other support for staff development has come primarily from one-to-one contact between the project coordinator and the supervisors. D. Human resources: · Project coordinator: The project coordinator is a very experienced and highly motivated medical doctor involved with public health, who is highly respected by the MOH. He has very good technical knowledge related to all of the N/CS policy issues, program interventions, and health facility components of the project. He does not, however, have in-depth knowledge and experience related to community level behavior change strategies. This in part explains the need to access specialized technical assistance related to community mobilization and empowerment approaches in order to promote improved community practices related to N/CS. · Zonal supervisors: The project recruited two female and two male field supervisors, posting one in each of the four project zones. They all appear to be very motivated in their jobs, and in most cases they maintain close relationships with the district and CSCom level health/social development workers in their respective zones. Two have clinical backgrounds while the other two have backgrounds in education/psychology. While they appear to have quite good knowledge of technical N/CS issues, their knowledge and skills related to community dynamics and behavior change are generally weaker. This is identified as a 32 limiting factor in their ability to work effectively with the NGOs and directly at the community level. · Project administrative assistant: An administrative assistant was recruited at the outset of the project to provide assistance to the coordinator. Due to financial irregularities in his work, his contract was terminated in July 2001. HKI has not yet recruited another person to take up these duties. E. Financial Management At the country office level there have been problems with the project administrator/accountant. The former country representative had asked the administrator to report directly to him rather than to the Chief Accountant/Administrator. This procedure was changed at the end of 2000. In July 2001, due to the irregularities in his financial procedures, he was discharged from his responsibilities. The new country representative is reviewing all financial management systems. HKI Mali has been operating for three years and an audit has not yet been carried out. This activity will be conducted before the end of this year. Regarding financial management at headquarters level, due to the loss of all electronic and hard copies of financial records in the Sept 11th disaster, HKI HQ is in the process of reconstructing its entire financial system. At mid-point in the project implementation, project grant spending seems a little under the planned disbursement calendar and this will be taken into consideration in the second part of the project. F. Logistics Logistical support planned for in the project has been organized and available. No particular problems have been encountered at this level. G. Information system, monitoring and evaluation Although the DIP describes several mechanisms to be put in place in order to monitor project implementation and generate lessons on an ongoing basis, the MTE reveals that most of these mechanisms are not yet operational. At the CSComs, the CSP has introduced a system of notebooks for recording micronutrient and ORS distribution as a temporary measure while working to incorporate these indicators into the MOH MIS system. Overall, as compared with other project components the information/ monitoring/evaluation component appears to be the weakest. · Comprehensive inventory of project-supported communities and interventions: A basic tool required for monitoring activities in the project zone is a comprehensive list of the target group population (women and children) by village. At the beginning of the MTE, project staff had not yet compiled the list. Such a list is the basis for developing a matrix that indicates which activities are being carried out in which villages. By the end of the MTE project staff had completed the preparation of lists by supervisor-area. All CSP staff agree that this tool will be of great help in planning and monitoring their activities. · Quantitative and qualitative information collection: CSP staff have established several mechanisms for collecting quantitative information regarding the implementation of project￾supported activities. For example, notebooks have been introduced into health facilities to record micronutrient supplementation activities, and checklists have been given to most radio stations to record programs broadcast in relation to programs scheduled. However, in these and other cases the information being recorded has not been collected and analyzed. In 33 addition, each of these mechanisms involves collection of quantitative information. No mechanisms have been established to collect, share and synthesize qualitative information between project collaborators. Qualitative information is necessary in order to identify strengths, weaknesses, and constraints encountered in either the structure or implementation of planned activities, and it is also needed in the formulation of lessons learned. · Small complementary studies: In the context of the project a number of small studies have been carried out which have provided very useful information on different nutrition-related issues/strategies. In addition to the baseline survey, other studies include a qualitative community study, a study on beliefs related to micronutrients, a study on availability of micronutrient-rich foods, and most recently an analysis of ASACO organizational capacity. Another study has just been launched (supported by the MI project) on iron supplementation compliance that will also be of relevance to CSP partners. All studies have been carried out in close collaboration with health/social workers. Results of the various studies, however, have not been widely circulated. It is recommended that the results of all such studies be systematically shared with project collaborators as a way of reinforcing their knowledge of and commitment to N/CS issues. It is preferable that study results be disseminated in a simplified and summarized form in order to be of greater interest to MOH/MSD field staff. · Monitoring trends in behavior change: Prior to the MTE no mechanism was in place for monitoring behavior change at the community level. Based on the MTE experience of using intercept interviews to collect information on key parameters of community members’ knowledge and behavior related to N/CS, plans have been made to periodically use this tool to monitor trends in behavior change at that level. · Project evaluation: HKI views evaluation not only in terms of its importance for accountability purposes but also in terms of its role in program and organizational learning. For this reason, the DIP clearly states that both the mid-term and final project evaluations will be conducted using a participatory, stakeholder-driven methodology. Twenty-five project collaborators and staff members were involved in this MTE. Key benefits of the participatory methodology identified by MTE team members include: increased understanding of project objectives, strategies, and roles among project stakeholders; accurate assessment of both strengths and weaknesses of project implementation; consensus on the conclusions and recommendations for implementation of Phase II activities; and increased commitment to project goals on the part of all stakeholders. H. Administrative support Due to the loss of the project administrative/financial assistant a few months ago and to the current organization of the Bamako office, the CSP coordinator presently receives almost no administrative assistance. This is a constraint in so far as he spends considerable amounts of his valuable time dealing with administrative/logistical issues. The new country representative is aware of this situation and plans to identify a strategy to provide him with basic administrative support. I. Technical assistance · Technical assistance provided by HKI Project staff are very appreciative of the technical assistance (TA) they received from the former Africa Regional Nutrition Advisor, Dr. Ag Bendech (now the country representative for Burkina Faso), and HKI/HQ. Particularly during project start-up, the assistance provided by Dr. Ag Bendech, who was based in Bamako until 9 months ago, was invaluable. For several key 34 activities TA was provided by the NY office, namely for several behavior change-related activities and for DIP development. For qualitative research and for the workshop on the development of a behavior change strategy, both HQ and HKI/Niger provided TA. The external evaluator concurs with project staff that there were weaknesses in the quality of this TA. The limitations of the methodology proposed for the qualitative research included inadequacies in interview guides, and inappropriate procedures adopted for data analysis, which resulted in superficial and poorly structured reporting of the results. Regarding the communication strategy development workshop, the major constraint was that the exercise focused on mass media and more formal communication channels, namely traditional communicators. The workshop included almost no discussion of “informal communicators,” namely community leaders, women’s group leaders and grandmothers. The MTE identified this oversight in the behavior change strategy. Lastly, a month-long “strategy testing” phase was included in the communication strategy development process. Based on interviews with project staff, the usefulness of this costly process is unclear. For future communication/ behavior-change strategy development is it recommended that more qualified TA be provided to field teams. · Technical assistance needs: Given the absence of strong community development/mobilization skills on the CSP team and the importance of the behavior change objectives and the community component of the project, in Phase II technical assistance in this area will be required if this component is to be adequately developed and staff skills reinforced. HKI is looking at several options related to either creating a position for a person with expertise in social mobilization or seeking short-term TA for this purpose. The new Country Representative comes to HKI with a long history of community-based work with Save the Children and will strengthen the orientation and quality of the community approach. CSP staff are particularly interested in incorporating grandmothers/muso koroba into N/CS activities and if funds are available additional TA may be sought to develop an adapted strategy for working with them. · Results highlights: Refer to the information sheet on the “Role and Influence of Grandmothers / muso koroba on Maternal and Child Health / Nutrition” on page 25. IV. CONCLUSIONS AND RECOMMENDATIONS A. Main achievements 1. Increased MOH commitment to maternal and child nutrition MOH officials’ understanding of nutritional dimensions of maternal and child health and their commitment to incorporating nutritional indicators and components into public health programs appears to have considerably increased in the past two years. While these changes cannot be attributed solely to the CSP, the close advisory and working relationship which has been established between the CSP coordinator and MOH officials from the national to the district levels has contributed to the following: establishment of a Division of Nutrition in the MOH; development of a national nutrition plan; development of the first MOH training manuals on maternal and child nutrition; inclusion of nutrition-related activities in the action plans of each of the districts in the project area; and greatly increased distribution of micronutrients to pregnant women and children, both through mass campaigns and at health facilities. These accomplishments have been the results of the institution-building strategies adopted in the projectm, which have been operationalized through the development of frequent contact, communication, and support by 35 project staff to MOH/MSD staff. Regional officials report that none of the other externally-funded projects supporting regional programs have worked in such a close and collaborative fashion. 2. Nutrition activities strengthened at the health center/CSCom level Due to greater awareness of local nutrition-related problems, training of health workers in all CSComs, and regular follow-up/supervision visits, both supplementation and nutritional counseling activities have increased considerably. VAC and iron-folate tablets, which are included in the national cost-recovery system, are more consistently available at the CSCom level and more systematically prescribed to pregnant women and children 6-59 months of age. MTE interviews suggest that both men and women’s knowledge of the importance of iron and VAC supplementation for women and children has increased. 3. Increased coverage of iron-folate and vitamin A supplementation through mass distribution For the first time in Mali, with support from the CSP, three mass micronutrient distribution activities/Journées Régionales de Micronutriments (JRM) were organized in the Koulikoro region. Regional MOH/MSD officials took primary responsibility for organizing this activity, which greatly contributed to increased awareness of the importance of micronutrients and to increased coverage. This HKI-initiated experience in Koulikoro has been greatly appreciated by the MOH, and the JRM model has already been replicated in four other regions, with plans to expand it to the rest of the country pending donor support. 4. Network of local radio stations are promoting nutrition/child survival topics In collaboration with district level MOH/MSD structures, the CSP has established collaborative relationship with 9 local radio stations that are part of the extensive network of small, private stations in Mali. Through training and regular follow-up of radio broadcasters, all stations are regularly disseminating N/CS messages. For several reasons, it appears that men listen to the radio much more frequently than women. However, men’s support for improved maternal and child N/CS practices is believed to be a positive enabling factor for behavior change in women. There appear to be good prospects that the local stations will continue broadcasting N/CS messages after the project ends. B. Main constraints 1. Delays in implementing behavior change activities beyond health facilities While progress has been made in improving health worker practices addressed in facility level DIP objectives, it appears that there has been limited progress related to DIP behavior change objectives due to constraints with the community-level activities being coordinated by ten local NGO partners. These constraints relate in part to delays in finalizing agreements with the local NGOs and also to the fact that NGO skills in community mobilization/development do not appear to be as strong as was indicated in the project design. MTE recommendations address the need to finalize the sub-agreements with partner NGOs and to strengthen NGO skills in community mobilizing and behavior change. 2. Discrepancy between DIP behavior change objectives and project design Many of the DIP objectives deal with individual behavior change within the entire target population, however, in the project design the more intensive behavior change activities outside of health facilities do not affect the entire target population. Unfortunately, some of the results￾based objectives included in the proposal were changed to population-based objectives during the DIP preparation. Several project-supported activities are contributing to strengthening health 36 facility services, however, only 45% of the target CSP population has access to a functioning health center (i.e., are less than 15 kms from a functioning facility). NGO-supported volunteers at the community level will carry out the most intensive education/communication activities. However, these activities will be carried out with only a maximum of 25% of the total beneficiary population Due to this discrepancy between DIP behavior change objectives and project design/strategies, by the end of the project it is unlikely that significant progress will be made toward those objectives. It is recommended that for the final evaluation a stratified sample of villages in the project area be used, wherein communities with and without NGO support be analyzed separately. At least this will make it possible to assess the impact of the variable intensity of interventions. 3. Insufficient monitoring and documentation In the DIP there is discussion of the importance of monitoring and evaluation as tools for learning and ongoing program improvement. However, a major weakness identified in project implementation is the relatively limited attention given to monitoring and documenting accomplishments, constraints, and lessons learned both in written form and through collective information-sharing and analysis. While some efforts were made to establish information collection mechanisms for monitoring purposes, none of the recorded information has been analyzed. Comprehensive documentation and follow-up of project staff activities has not taken place due to the absence of regular staff meetings, and to reporting procedures which are too infrequent and insufficiently analytical. 4. Inadequate project staff skills in community mobilizing approaches to behavior change Project staff appear to have adequate skills related to the technical aspects of N/CS that are required for working with both the DHT and CSCom agents. However, their orientation and skills related to community mobilization for health-related behavior change are relatively weak. This is a definite constraint given the importance of the community component in reinforcing ASACO organizational and health promotion skills, and in promoting changes in household/individual behavior in collaboration with the local NGOs. Several complementary strategies were identified with project staff during the MTE for redressing this weakness. C. Main recommendations 1. Identifying specialized technical assistance in community mobilization for behavior change Community level activities need to be based on the concept of communication and mobilization for empowerment of community, individuals, and groups, to strengthen their ability to make decisions and take action related to health/nutrition themselves. In order to strengthen the skills of CSP staff and harmonize the community mobilization strategies used by NGO partners, the project needs to identify specialized technical assistance and/or additional in-house capacity on either a part-time or full-time basis. The HKI Country Representative is actively looking at alternative ways of responding to this need, including the possibility of hiring an additional staff member specialized in these areas and/or accessing short-term TA either from within or outside of HKI. 2. Development and follow-up of NGO community mobilizing activities Several immediate priorities are identified related to collaborative efforts with the ten local NGOs: finalizing agreements with them; strengthening and harmonizing NGO strategies related to community mobilization for behavior change; developing a common training module for use with community volunteers, grandmothers, women’s group leaders, and community leaders; and regular follow-up/sharing with NGO partners to reinforce their strategies and formulate lessons learned. 37 3. Involvement of grandmothers (muso koroba) in nutrition/CS activities MTE interviews with grandmothers (GMs) and other family members clearly revealed the prominent role they play in dealing with all health promotion and illness management issues related to women and children. These findings were strongly confirmed by the traditional male community leaders. It was concluded that GMs have a strong influence in defining and communicating community norms related to health/nutrition that in turn significantly determine individual practices. There was a broad consensus among MTE team members that in all project zones efforts should be made to involve GMs in N/CS activities in order to dialogue with them and increase their support for the “improved” N/CS practices promoted by the project. 4. Training and follow-up of local community health associations/ ASACOs In the DIP it is expected that the ASACOs in the project zones will play an important role, both in managing community health center/CSCom activities related to N/CS, and in promoting good health/nutrition practices among community members. Based on weaknesses in their knowledge and skills identified in the recently completed organizational assessment, they need to be trained as soon as possible and supervised on a regular basis. The project has already taken steps to strengthen management capacity of ASACO by enlisting the collaboration of ACOD, a Malian NGO that works in this domain. 5. Simple, participatory behavior change activities Community level nutrition/health education activities should be based on principles of participatory adult learning in order to maximize the impact they have on learning and behavior change. It is also important to develop activities that can be used/sustained by community groups themselves rather than those which require the presence and skills of a trained community health/development worker. Emphasis should be put on the use of stories, songs, and group discussions rather than on traditional health talks (les causeries classiques), which are based on a directive, top-down approach. 6. System for monitoring, sharing and learning (MSL) from project implementation In order to ensure ongoing monitoring of project activities a comprehensive but simple MSL strategy should be developed by the project team. The system should include the collection of both key quantitative and qualitative information at each level of project activities and should include mechanisms for periodic sharing of experiences/observations/information between project partners and the formulation of lessons learned on an ongoing basis. The system itself should be assessed on an ongoing basis in order to modify it, if necessary, to make it as useful as possible for project decision-making and documentation for future programs. 7. Gender training for project staff and NGO partners Orientation to gender and training in gender analysis emerged during the MTE as both an organizational and programmatic need. Gender training should be organized for all project staff in order to increase their sensitivity to gender dynamics that shape the situation and constraints faced by women in project-supported communities, and to strengthen the project’s ability to effectively promote women’s involvement and empowerment. D. Conclusions regarding the participatory evaluation methodology As planned in the DIP, the MTE was carried out using a participatory, stakeholder-driven methodology. HKI invested considerable resources in order to involve a large number of project stakeholders, and allotted a five-week period for the MTE (4 weeks of team work plus one week for report writing) in light of anticipated advantages in terms of both program and organizational 38 learning. (The choice of the participatory methodology was strongly influenced by the very positive experience HKI had using this participatory methodology in the evaluation of its CS project in Niger in 1999.) The CSP Coordinator In Mali participated in a CSTS-sponsored workshop on this participatory methodology in 1999 and, therefore, was well prepared to co￾facilitate the entire process. Twenty-six project stakeholders were involved in carrying out the evaluation, which was coordinated by the external evaluator, Dr. Judi Aubel. Through a very participatory but structured process, the evaluation team was involved in developing the evaluation questions, collecting and analyzing both primary and secondary information, and formulating lessons learned (See Attachment H: Assessment Methodology). Immediately after the evaluation, project staff developed a revised action plan for Phase II of the project (see Action Plan below), based on the evaluation findings and recommendations developed by the team. Feedback on the participatory methodology was collected through a group exercise and through short individual interviews with evaluation team members. There was a broad consensus among team members that the participatory approach had multiple advantages as compared with the traditional, expert-driven evaluation. Key benefits of the approach identified by team members include: active involvement of project stakeholders at all steps in the planning and implementation of the evaluation; increased understanding on the part of stakeholders of the project objectives, strategies, and roles of different actors at different levels; project actors themselves identified strengths and weaknesses in project implementation; there was a broad consensus among project actors on the strengths, weaknesses and priority actions to be taken in Phase II to strengthen program implementation, i.e., there was ownership of evaluation results; valuable sharing and learning between project implementers and the outside evaluator; strengthened interpersonal relationships between project actors, which will facilitate future collaboration between them; reinforced knowledge of the technical aspects of N/CS addressed in the project; and knowledge of basic concepts/approaches to participatory evaluation that can be applied in other work-related activities. Disadvantages of the approach listed by team members include: considerable human, financial and logistical resources required; excessively long work days; and the fatigue of team members associated with the volume of work required. Overall, team members concluded that compared to traditional evaluation methods the participatory approach was very beneficial both for the project and for their personal development. Dr.Diakalia Koné, the CSP Coordinator concluded that from his perspective the MTE was cost-effective in so far as it not only satisfied a donor requirement but also had a very strong formative value for project staff and their collaborators. He repeatedly stated, “I think this process should be called ‘participatory evaluation and training.’ There was agreement among all MTE team members that the evaluation was a valuable learning/training experience for all those involved. 39 V. ACTION PLAN PROJECT WORKPLAN The activities highlighted in green are those that have already taken place, those in yellow are those that are being implemented, and those in blue have not been implemented yet at the time of the MTE of the CSP. Year 1: 2000 Year 2: 2001 Activities 3 rd trim. 4 th trim. 1 st trim. 2 nd trim. 3 rd trim. 4 th trim. A M J J A S O N D J F M A M J J A S PROJECT COORDINATION Finalization of plan x x x x Recruitment of 4 facilitators or field animators x x x x x Purchase 4 motorcycles x Purchase a car x Elaborate operational management procedures, procedures for monitoring/evaluation and for project annual review x x x Organize meetings of the Regional Steering Committee x x Organize meetings of the Comité de Cercle x x x x Organize monthly meetings of the ESSC x x x x x x x x x Organize the Regional Micronutrient Days (to administer the 2nd dose of vitamin A) x x x Finalize the procedures, protocols and agreements for the partnership and collaboration with local NGOs x x x X TRAINING Determine the training needs of CSCom agents, CBD agents, and ASACOs and NGOs x x x Elaborate training modules / curricula x x x Finalize the les advice cards (cartes-conseil) x x Train the field animators in nutrition (interventions) x x Train the CSP Coordinator on IMCI x x Train the ESSC in nutrition x Train the CSComs, ASACO and NGO agents in nutrition x x 40 Year 1: 2000 Year 2: 2001 Activities 3 rd trim. 4 th trim. 1 st trim. 2 nd trim. 3 rd trim. 4 th trim. A M J J A S O N D J F M A M J J A S Train the CBD agents and recycled midwives x x Organize a training workshop for the radio animators on micronutrients and IEC messages x x Plan study visits between Cercles and between NGOs (if possible) x x Train ASACO members in order to increase their participation in the ASACO life « la vie associative » x x x Train the CSCom agents and ASACO members in management of stocks (of iron/folate, VAC, and ORS) x x RESEARCH Qualitative research on the nutritional behaviors of the communities in nutrition x x x x x Determine the partners’ capacities and elaborate and action plan x x Undertake operational research on the participation and capacities of communities in the management of ASACO x I.E.C. Organize a social mobilization campaign on micronutrient and micronutrient-rich foods, similar to those done during the JRM x x x Organize a workshop for the elaboration of the IEC strategy and tools x Create, elaborate and implement an advocacy plan on nutrition x x x x x x x x x x x x x x x x x x Inform all partners and communities on the nutrition program x x x x x x x x x Organize the launching of the IEC nutrition campaign by the local radios and other communication channels x x x x x 41 Year 1: 2000 Year 2: 2001 Activities 3 rd trim. 4 th trim. 1 st trim. 2 nd trim. 3 rd trim. 4 th trim. A M J J A S O N D J F M A M J J A S MONITORING AND EVALUATION Organize a workshop to identify the strengths and weaknesses in the MIS and elaborate an action plan x x Elaborate tools for the suprevision and evaluation of functional health Cercles x x x Test, multiply and disseminate the collected information x x Supervision of activities - Of the CBD by the CSCom once a trimester - Of the CSCom by the Cercle once a trimester - Of the Cercle by the Region once a semester - Of the Region by the Central level once a semester x x x x x x 42 Attachment A : Field Program Summary Estimated Program Effort and USAID Funding by Intervention and Program Site Population : Children and Women 43 44 Attachment B : Program Goals and Objectives 45 46 47 48 49 50 Attachment C : Program Location 51 52 53 54 Attachment D : Program Design 55 56 Attachment E : Project Partners and their Roles 57 58 59 60 61 Attachment F : Health Information System 62 63 Attachment G: Members of Mid-term Evaluation Team Mid-Term Evaluation of the Child Survival Project NAME POSITION Mamady Sogoré TSAS Ouélessébougou Daouda Samaké Supervisor C.S.HKI Nialen Kaba Supervisor C.S.HKI Moriba Camara Administrator DRAS Koulikokoro Fama Kondo Supervisor C.S.HKI Baba Diallo Koulikoro Doctor Mme Faye Djélika Sidibé Regional Health Directorate/Koulikoro Mme Sangaré Kamissa Supervisor C.S.HKI Namaké Bouaré Health Center/ Kati Oumar Mallé Animator BN/NGO N'Tji Tiémoko Coulibaly Trainer CAED/NGO, Kolokani Mady Makanéra TDC Kolokani Konimba Ballo Head Nurse Koula (Koulikoro) El Moctar Maiga Head Nurse Ouélessébougou Mamadou Doumbia Head Nurse Sébékoro II Mme Sy Sirandou Midwife CSC Kati Lamoussa Traoré Regional Health Directorate/ Koulikoro Boubacar Diarra DSSAN/ CPS Health Sériba Traoré Central Head Nurse Kolokani Adama Kané Nutrition Division /MOH Modibo Diarra DSSAN/ CPS Santé MamadouSIDIBE Regional Health Directorate/Koulikoro Diakalia KONE CSP Coordinator Koulikoro HKI Victor AGUAYO Regional Advisor N/CS HKI Adama SANGARE Onchocerciasis Coordinator HKI Kourounkou KEITA Trachoma School Health Coordinator HKI Judi AUBEL MTE Coordinator Karen Waltensperger Country Director HKI/Mali 64 Attachment H: Assessment Methodology Steps in the Participatory Evaluation Mid-Term Evaluation : HKI Mali / Child Survival Project Phase I: Preparatory Meetings (Coordination Group of the Evaluation) August 20 - 22 Step 1: Define the global goal and objectives of the evaluation Step 2: Identify the evaluation team member Step 3 : Identify the data collection sites and the secondary information to analyze Step 4: Prepare the administrative and logistics aspects Step 5 : Elaborate the visual framework of the project Phase II : Evaluation planning workshop (Evaluation Team) August 23 - 25 Step 6 : Develop a team spirit/vision for the evaluation team Step 7 : Define the evaluation questions Step 8 : Identify the data collection sources and techniques Phase III : Preparation for the data collection (Coordination Group) August 27-29 (Field Teams) August 30, 31 – Sept 1. Step 9 : Elaborate the tools / guides for data collection Step 10 : Orient the Field Teams on the collection and analysis of qualitative information Phase IV : Data collection in the field (Field teams) September 3 - 11 Step 11 : Undertake the interviews and observations Step 12 : Analyze the collected information Phase V : Sharing of the collected information (Coordination Group) September 12 -14 Step 13 : Summarize the findings/results of the data collection at the level of the four zones Phase VI : Workshop for the preparation of lessons learned (Evaluation Team) September 17 au 19 Step 14 : Formulate the lessons learned from the findings/results of the field work Step 15 : Collect feedback of the Evaluation Team members on the methodology used in the evaluation Phase VII : Summarize the c conclusions of the evaluation (Coordination Group) September 20 Step 16 : Summarize the lessons learned and the main conclusion of the project actors and HKI Phase VIII : Prepare a revised action plan for the project (Project Team) Later date Step 17 : Elaborate a revised action plan for the project from the lessons learned from the evaluation Phase VIII : Dissemination and discussion of the conclusions of the evaluation (Evaluation Coordinator) August 24 - 27 (HKI Team) August 21 (Project Team) Later date Step 18 : Write the evaluation report Step 19 : Restitution with the MOH authorities Step 20 : Dissemination and discussion of the evaluation report with the other project partners 65 Attachment I: List of Persons Interviewed and Contacted National Level MOH: Dr. Fodi Bondi, Assistant Director of Health Dr. Kanyasi. Dr. Modibo Diarra, Director of Nutrition Division Kamara Sonogo. Diarra Kane Regional Level Koulikoro, Regional MOH/SW Office: Dr. Sidibe, Deputy Regional Director of Health Dr. Diarra Bogoba, Planning Officer Mme. Ndiaye, Regional Director of Social Welfare La Moussa Traore, Deputy Regional Director of Social Welfare Koulikoro District: A total of 8-10 District staff were interviewed. Kati District: A total of 12-14 District staff were interviewed 10 NGOs were involved, with 1-2 representatives interviewed from each NGO, including: CAEB: Mr. Doukoure A representative from ACOD APDEV: Mr. Dembele Community Level Village locations included: Ngofala Zana Koula Siraboulou Sirakirola Siby Niame Bancoumana Numbers of People Interviewed: 16 Traditional theatre group members 26 CSCOM/ASACO members 80 Women 40 Grandmothers 26 Community leaders 50 Men