Child Survival 20 – Mali Partnership to Maximize Access and Quality of Family Planning Services in Ségou, Mali 30 September 2004 – 30 September 2009 Cooperative Agreement No.: GHA-A-00-00003 Midterm Evaluation Report Prepared by: Marguerite Joseph, Consultant Submitted by: Save the Children Federation, Inc. 54 Wilton Road Westport, CT 06880 Telephone: (203) 221-4000 Fax: (203) 221-4059 Contact Person: Eric Swedberg, MPH, Child Survival Advisor Submitted to: USAID/GH/HIDN/NUT/CSHGP October 31, 2007 TABLE OF CONTENTS A. Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . ………… 5 B. Assessment of progress made in achievement of program objectives 1. Technical Approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 a. General Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 b. Progress report by intervention area . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 c. New tools or approaches . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17 2. Cross-cutting approaches . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19 a. Community Mobilization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20 b. Communication for Behavior Change . . . . . . . . . . . . . . . . . . . . . . . . . . 21 c. Capacity Building Approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23 i. Strengthening the PVO Organization …... . . . . . . . . . . . . . . . . 23 ii. Strengthening Local Partner Organizations . . . . . . . . . . . . . . . 24 iii. Health Facilities/Health Worker Strengthening . . . . . . . . . . . . 29 iv. Training . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31 d. Sustainability Strategy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . … 33 C. Program Management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37 1. Planning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37 2. Staff Training . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38 3. Supervision of Program Staff . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . 38 4. Human Resources and Staff Management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40 5. Financial Management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41 6. Logistics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42 7. Information Management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43 8. Technical and Administrative Support . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ... 43 9. Mission Collaboration………………………………………………. ……. 45 D. Other Issues Identified by the Team………………………………………. 45 E. Conclusions and Recommendations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45 F. Results Highlight . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50 ANNEXES A. Baseline Information from the DIP B. KPC Report (translated summary report) C. Evaluation Team Members and Their Titles D. Evaluation Assessment Methodology E. List of People Interviewed and Contacted F. Attached diskette or DC with electric copy of report G. Cost-Effectiveness Draft Report H. NGO Organizational Capacity Assessment Report I. Project Data Sheet form – updated version J. Phase Two Action Plan K. Annual Indicators Reporting Table CS-20 Mali, Ségou, October 2007 2 Save the Children Acronyms AMPRODE/SAHEL/JIGI Association Malienne pour la Protection et le Développement de l’Environnement au Sahel ASACO Association de Santé Communautaire ASDAP Association de Soutien au Développement des Activités de Population/Association for the Support of Population Development Activities BCC Behavior Change Communication C-IMCI Community-based Integrated Management of Childhood Illnesses CA Community Agents CBD Community-Based Distribution CFA Malian Currency CHW Community Health Worker CPM Health facility in-charge CPR Contraceptive Prevalence Rate CS Child Survival CSCom Community Health Center CSHGP USAID’s Child Survival and Health Grants Program CSTS Child Survival Technical Services CYP Couple Year of Protection DIP Detailed Implementation Plan DRC District Drug Depot DPM National Pharmacy and Medicines Department DRS Direction Regional de la Sante￾ELCO Eligible Couple FGD Focus Group Discussions FHI Family Health International FP Family Planning GP/SP Groupe Pivot/Santé Population HC Health Center HF High Islamic Council HFA Health Facility Assessment HIC High Islamic Council HIS Health Information System HIV/AIDS Human Immune-deficiency Syndrome/Acquired Immune Deficiency Syndrome HSS Health System strengthening IEC Information, Education and Communication IR Intermediate Result KPC Knowledge, Practices and Coverage KR Key Result LAM Lactational Amenorrhea Method M&E Monitoring and Evaluation MEC Medical Eligibility Criteria MOH Ministry of Health MTE Mid Term Evaluation CS-20 Mali, Ségou, October 2007 3 Save the Children NGO Non-Governmental Organization OC Oversight Committee OCA Organizational Capacity Assessment ORS Oral Rehydration Solution PA Project Administrator PDME Program Design Monitoring and Evaluation PDQ Partner Defined Quality PE Peer Educator PPM Regional Pharmacy of Mali PVO Private Voluntary Organization QA Quality Assurance RAPID Resources for, and Analysis of the population and its Impact on Development RH Reproductive Health RHO Regional Health Office SASDE Accelerated Strategy for Child Survival Project of UNICEF SC Save the Children Federation, Inc. SO Strategic Objective STI Sexually Transmitted Infections TOT Training of Trainers UNFPA United Nations Population Fund UNICEF United Nations Children’s Fund USAID United States Agency for International Development VOC Village Oversight Committee WRA Women of Reproductive Age (15 to 49 years of age) CS-20 Mali, Ségou, October 2007 4 Save the Children A. Executive Summary Save the Children (SC) is implementing an expanded impact, five-year, four-district Child Survival Project, CS-20,Partnership to Maximize Access and Quality of Family Planning Services in Ségou, Mali. The project is targeting an estimated 184,940 women and 170,275 men of reproductive age. CS-20 covers four of Ségou’s seven districts: Macina, Niono, Tominian, and Baraouéli. SC and Groupe Pivot/Santé Population (GP/SP) have teamed up with two proven national NGOs: ASDAP and the Consortium: AMPRODE-SAHEL& JIGI and the regional, district, and local level MOH structures to build a network of well-supported, community-based distribution agents. CS-20 key strategies have been divided into two interlinked approaches during the first half of the project: community-based services and health services strengthening. Community-based services include: marketing and sales of contraceptives by peer educators (PE) and community-based distribution (CBD) agents in villages not served by a health center (HC); increasing effective behavior change communication (BCC) through interpersonal and popular village channels; assuring contraceptive availability, and; counseling and referral to the HCs. Health services strengthening includes: strengthening of contraceptive logistics management; strengthening of provider skills; and capacity building of service providers and HC Boards. A cost effectiveness study conducted prior to the midterm evaluation (MTE) has demonstrated that the cost of the combined approach involving the PVO and NGO partners is higher than the cost of health service strengthening alone. However, due to the impact on CPR (much higher in health zones reached by the combine approach), Phase II of the project will be implementing the combined approach in all the health zones in the new districts. This project’s goal is that: 17% and 24% of women of reproductive age (WRA) (15-49 years) use modern contraception in Barouéli/Macina and Niono/Tomininian, respectively. This will be achieved through the following results (and selected indicators): (1) increased access to family planning (FP) services (2) improved quality of FP service delivery (3) increased FP knowledge and interest, and (4) improved technical and institutional capacity in health services and communities to provide quality FP. Key achievements to date: 1. Revitalization of FP in the region of Ségou and repositioning of FP in the minimum package of services. 2. Mobilization of Muslim religious leaders for FP, establishment of Islamic Committees and clarification of the Koran’s position of FP through advocacy. 3. Strengthening of FP services with the training of service providers and ASACO members in the project area on micro-planning and, contraceptive logistics and management. 4. Training and provision of contraceptives to 234 community agents in 198 villages. 5. The training and integration of 10 private pharmacists in the health information system and establishment of a procurement scheme for them through HFs. 6. Implementation of operations research on cost effectiveness of community/HF strategy compared to HF strengthening alone. Conclusions and recommendations: The capacity-building efforts of the project have resulted in the revitalization of FP in the target districts. FP has become a part of routine MCH activities at the HFs as well as at the community CS-20 Mali, Ségou, October 2007 5 Save the Children level through community outreach activities and the establishment of CBD agents in communities located 5 km or more from a HF. Prospects for sustainability are good as there has been a good level of buy-in from the MOH district partners. FP will be integrated in other supervision and monitoring activities which are currently supported by UNICEF in project districts. Regional, district and HF level Drug Depot Managers have been trained on contraceptive logistics and the demand for contraceptives at the community level will continue to exert pressure on the national level to ensure contraceptive security. (Please refer to Annex J for a copy of the Phase II Action Plan reflecting sustainability in Phase I districts and how Save the Children will address MTE recommendations for Phase II districts.) The main constraints currently faced, include contraceptive security as the government has not been able to assure the constant supply of contraceptives. The mobility of the population and their absence during many months of the year makes it challenging to ensure that clients are continuing to use their contraceptive methods. Access to communities from HFs during the rainy season is also difficult. Save the Children needs to consider modifying their budget to ensure success in the second phase of the project, i.e. reducing to one implementing NGO based on the organizational assessment and MTE results. This will allow for easier management and technical oversight of activities and logistical support within the constraints of a limited program budget and ambitious results set out by the program. MTE Recommendations 1. Modify the project in accordance with available resources and the needs identified during the MTE, i.e. consider cutting back from two to only one implementing NGO working across the two Phase II districts based on a follow-up organizational assessment of the two partner NGOs and the MTE results in the Phase I districts; consider reducing the field staff to one Coordinator, an Assistant Coordinator, two supervisors and a maximum of 15 Animators to support the 36 health zones in the Phase II areas; submit a request for a budget modification to USAID BHR/PVC to allow for the purchase of field staff motorcycles to ensure staff transportation across a large and difficult program intervention area. 2. For Phase II of the Ségou FP project, staff needs to revisit the PE strategy in view of the constraints encountered with this cadre during Phase I. 3. It would be useful to explore an adaptation of the PDQ process to match available resources, or select only one or two of the less popular HFs per district to undertake the full process and inform other health zone areas as well. 4. The project should support the systematic integration of FP into community outreach activities being undertaken by HFs in order to increase access to methods that are not provided by the CBDs, i.e. injectables, and provide counseling and adequate care to clients experiencing side effects. 5. Technical project staff needs to dedicate more time and resources for supervision of all levels to: strengthen the capacity of field staff, HF staff and community volunteers; ensure the quality of work; and ensure adequate communication between the different levels. CS-20 Mali, Ségou, October 2007 6 6. Project management should do their utmost to ensure the rapid start-up of activities in Phase II districts in view of the limited time left before the end of the project, i.e. timely signing of agreement(s) with the local NGO(s) so that project staff can receive their salaries without a long delay; ensure the provision of transportation and necessary materials for staff, ensure the availability of contraceptives, etc. Save the Children B. Assessment of the progress made toward achievement of project objectives 1. Technical Approach a. Project Overview Save the Children (SC) is implementing an expanded impact, five-year, four-district Child Survival Project, Partnership to Maximize Access and Quality of Family Planning Services in Ségou, Mali. Responding to communities’ needs, MOH priorities, and SC national and global strategy, the project seeks a sustainable increase in the voluntary use of modern methods of contraceptives among women and men between the ages of 15 and 49. Mali has one of the lowest overall contraceptive prevalence rates (CPR) in sub-Saharan Africa (5.7%) and thus one of the highest fertility rates: 6.8. These rates vary substantially by place of residence with 14.7% CPR in urban areas and 2.8% in the rural areas that will be targeted by CS-20. The project is targeting an estimated 184,940 women and 170,275 men of reproductive age. CS-20 will cover four of Ségou’s seven districts: Macina, Niono, Tominian, and Baraouéli. SC and GP/SP have teamed up with two proven national NGOs: ASDAP and the Consortium: AMPRODE￾SAHEL& JIGI and the regional, district level, and local MOH structures to build a network of well supported community-based distribution agents. The project relies on existing health personnel, structures, relationships, and plans, including where possible, an existing network of community volunteers established through the UNICEF SASDE (Accelerated Strategy for Child Survival) project. CS-20 key strategies will be divided into two interlinked approaches during the first half of the project: community-based services and health services strengthening. Community-based services include: marketing and sales of contraceptives by PEs and community-based distribution (CBD) agents in villages not served by a health center (HC); increasing effective behavior change communication (BCC) through interpersonal and popular village channels; assuring contraceptive availability, and; counseling and referral to the HCs. Health services strengthening includes: strengthening of contraceptive logistics management; strengthening of provider skills; and capacity building of service providers and HC Boards. A cost effectiveness study conducted prior to the MTE has demonstrated that the cost of the combined approach involving the PVO and NGO partners is not higher than the cost of health service strengthening alone. Thus Phase II of the project will only be implementing the combined approach in the new districts. This project goal is that: 17% and 24% of women of reproductive age (WRA) (15-49 years) use modern contraception in Barouéli/Macina and Niono/Tomininian, respectively. This will be achieved through the following results (and selected indicators): (1) increased access to FP services (2) improved quality of FP service delivery (3) increased FP knowledge and interest, and (4) improved technical and institutional capacity in health services and communities to provide quality FP. CS-20 Mali, Ségou, October 2007 7 Save the Children b. Progress made in Phase I of the project i. Activities as proposed in DIP The Ségou program DIP proposed two strategies: community-based services and health service strengthening. Part of the program area (Community Zones) was to benefit from both, and the other part of the program area were to benefit from the latter only so as to subsequently be able to conduct a study comparing the cost-effectiveness of the two strategies. Community-based services were implemented in target villages only. Identified villages were supposed to meet the following guidelines: > 5 km from a HF and population > 500 persons would have a CBD agent, and those > 5 km from a HF with a population > 750 would also have PEs. Both of these community volunteers were to be community-based distribution agents providing FP services. It should be noted that due to some delays in start-up and logistics, and the rush to get activities started, the project leadership was not able to give enough oversight in this selection process. The selection criteria led by Phase I districts (Macina and Baroueili) was not entirely respected. In the district of Macina, for example, many of the villages identified for the community￾based approach were those who had not previously benefited from a CARE project that closed out several years prior to the start-up of this project. ƒ Marketing and sales of contraceptives by a well-supported network of adult CBD agents and youth PEs in villages not served by a HC Once communities for the above approach were identified, project and HF staff partners worked in collaboration with community leaders to select community health volunteers (locally known as relais) to work as Community Based Distributor agents and PEs. In total 234 received training (114 in Macina and 126 in Baraouéli). Of these 88 were PEs (44 in each of the two districts). Note that there were more PEs established than initially planned in the DIP (88 verses 70) due to the number of communities and population size that ended up becoming a part of the Community Zone interventions. Nevertheless it was noted that due to the fact that most girls are already married by their mid teens, and tend to be married to older men, it is not acceptable for non-married youth to provide them with these services. The other disadvantage of working with youth is that they tend to be less stable in these rural communities as they are constantly on the move seeking employment opportunities. The project had also planned on establishing approximately 138 CBD agents, but ended up with 152 (26 more in Baraouéli than envisaged, and 12 less in Macina). In reality, both cadres were FP distributors so this was a big advantage. The distinction between the two was more in terms of age and target clients. Per the DIP, the project sought out the assistance of DELIVER for the provision of contraceptives. Unfortunately there was an unforeseeable delay and commodities did not arrive in the project communities until April 2006, approximately one year after community agents were already in place. Nevertheless, the trained volunteers are equipped with contraceptive supplies (combined oral pill, progesterone only pills, condoms, and vaginal foam tablets). Clients requesting other methods, or those experiencing side effects are referred to the HF. ƒ Increasing effective BCC through interpersonal and popular village channels CS-20 Mali, Ségou, October 2007 8 In Community Zone areas, the CBDs and PE strategies have facilitated information, education and communication, and behavior change. PAs also trained in adult education methodologies and FP Save the Children scheduled supportive supervision to CBDs/Peer educators once or twice a month and would help them with group education sessions as well. Per project supervisors, education sessions were organized around specific themes such as the advantages of FP and side effects of certain methods. Key messages included: the importance of rest between childbirths; improved child health and reduction in infant mortality; reduced burden on the family; reduced stress on the couple; more time for economic activities; and reduced unplanned pregnancies in young people. Interpersonal communication and counseling was mainly done via home visits which are a part of CBD job responsibilities. Per reports during the MTE activities, these consisted of visiting households where there was a clear need for this service (such as those with many children), and performing follow-up of clients who had missed their appointments. Targeting women’s groups or associations, and youth groups was another BCC strategy. Education sessions were sometimes conducted in the fields where members of women’s groups and the population in general spend most of their time during planting and harvesting season. Use of village ‘criers’ to announce and encourage participation in community education sessions, was reported by program staff as something that was common practice. A project supervisor who was interviewed reported that some Animators have been able to take advantage of village cultural activities to disseminate FP messages. During the first national campaign for FP, small funds permitted PAs to work with HF and community partners to organize folkloric festivities and such things as soccer matches where FP messages could be disseminated. ƒ Assuring contraceptive availability The project efforts for this component involved partnership with the DELIVER Project which closed in October 2006. In 2005, DELIVER was able to provide support with training in contraceptive logistics and management. With the co-facilitation of the Directorate of Pharmacy and Drugs (DPM) and Regional Health Division, DELIVER trained 44 Drug Depot Managers and pharmacists at regional and district levels (10 private). (Please refer to the training section for more detail.) A supervision visit was also undertaken by DELIVER in 2006 to monitor progress and provide follow-up support to the project. During this activity (DELIVER staff visited a total of six HFs; three per district as well as the District Drug Depot (DRC) in each district. The recommendations that came out of this visit were for the project to: ensure the continued capacity building of Drug Depot Managers during supervision visits; ensure follow up of contraceptive logistics at all levels: ensure compliance with guidelines and procedures for contraceptive logistics; and involve the ASACOs more actively in the procurement of contraceptives. ƒ Counseling and referral to the HF for FP The community-based strategy relied on the presence of CBDs and PEs to increase counseling and referral opportunities for women and potential clients. Client counseling and referral took place either when volunteers were sought out by community members or during home visits. Health education sessions were occasions when people could ask a lot of questions around FP, and some people would come up to the volunteer or PA after sessions with personal questions that they had CS-20 Mali, Ségou, October 2007 9 Save the Children not wanted to voice in the larger group discussion. These were used as counseling and referral opportunities as well. Health services strengthening activities covered all health zones in the two target districts of Macina and Baroueili. Each health zone has a HF covering a catchment area consisting of 5-15 villages with varying population sizes. The project assigned an NGO Animator to each HF whereby they would support health service strengthening only (Health Zones), or both health service strengthening and community-based services (Community Zones). ƒ Strengthening HC provider skills This component started with the training of a total of 58 In-Charges and Matrons (29 each representing 29 HFs) in FP, counseling and service provision. Support from PAs based at the HFs was a key strategy for health services strengthening as well, in addition to supervision from the district and project supervisors, coordinators and managers. Please refer to the section of strengthening health worker performance for more details. ƒ Strengthening contraceptive logistics management Training was a big part of strengthening contraceptive logistics management. Again this was conducted with the support of the DELIVER project who worked in collaboration with DPM and DRS. Thirty-four Drug Depot Managers and 10 private pharmacies from the two districts benefited from this training. ƒ Capacity building of service providers and ASACOs (Health Facility Committees) for sustainable and quality FP services. Capacity building efforts included training, support from project staff and the district health team, and periodic supervision from the regional office as well. A total of 59 ASACO members participated in project-supported training, which specifically targeted the president and treasurer of these HF committees. With the support of PAs, HFs have all a developed a FP education program. Education on FP is now integrated into health education sessions that are conducted two to three days a week in most HFs. These sessions are usually implemented during the days allocated for vaccination and pre-natal care when there is a large clientele of WRA. To encourage more consisted outreach to communities, the project also offered financial support to the ‘Strategy Avance’ (outreach visits/services) monthly community-based outreach program. During these activities basic services such as vaccination, prenatal care and health education sessions are provided to communities located at a pre-determined distance from the HF. The integration of FP services in this program has also contributed to increased access as this is an opportunity for women using the injectable, depo-provera, to receive this service in the community rather than having to travel to the HF. FP clients experiencing side￾effects also have the opportunity to consult a service provider during these visits as well. CS-20 Mali, Ségou, October 2007 10 Save the Children ƒ Partner Defined Quality CS-20, Ségou had planned to implement but was not able to complete all the phases of the Partner Defined Quality (PDQ) process. Twenty-five project and MOH facility staff received an orientation on this methodology in Bamako, and subsequently implemented Phase I of the process which entailed holding meetings in project communities, conducting community mapping exercises and inviting representatives to participate in this activity. The project conducted Phase 1 (building support) and Phase II (exploring quality with providers and community members separately). At this point they encountered problems when they realized that due to the tight budget, they could not invite community members to come to the health zone level for the Phase III activity (Bridging the Gap), due to cost implications such as refreshment and transportation expenses. Thus although a few of the PAs were able to complete the phases of PDQ, and deal with these issues, many were not able to complete the PDQ activity per the guidelines, which was a big disappointment for project staff who had initiated the quality improvement process. ii. Progress toward benchmarks or intermediate objectives Table 1. Ségou FP Program Indicators # Indicators Data source Baseline (2005) Midterm- (2007) Midterm Objective SO Key Result 1 Couple-Years of Protection (per year) HIS 3247 ----- SO Key Result 2 17%/24% of WRA (15-49 years) use modern FP method KPC 8,2% 10.6% 17% SO Key Result 3 65%/70% of WRA report being a ‘new user’ of a modern method of FP KPC 59,0% 65.6% 65% SO Key Result 3 Number of users new to contraception (per year) 9486 ------ SO Key Result 4 60%/65% Continuation rates KPC 67.6% 57.3% 60% SO Key Result 7 Adequate birth spacing KPC 82,1% 69.3% ----- SO Key Result 9 Unmet need for FP KPC 88,0% 78.9% ----- IR3.1 70%/90% of women (15-49) live within 5 km of a FP service delivery point KPC 56,4% 47.4% 70% IR3.1 50% of villages >5km from HC and >750 persons have PEs providing FP services to adolescents HIS 46% 50% IR3.1 50% of villages >5km from HC and >500 persons have CBD agents providing FP services to adults HIS 78% 50% IR3.2 40%/70% of WRA report discussing FP with a health or FP worker or promoter in the past 12 months KPC 25,1% 35.6% 40% IR3.5 90% of CBD agents and PEs had no stock-outs of oral contraceptives or condoms in the last three months CBD Survey 9% 26.2% 90% IR3.6 50% of the community outreach clinics include the provision of Depo-Provera HIS 0% 35% 50% CS-20 Mali, Ségou, October 2007 11 Save the Children CS-20 Mali, Ségou, October 2007 12 # Indicators Data source Baseline (2005) Midterm- (2007) Midterm Objective IR2.1 50%/65% of FP clients receive adequate counseling KPC 12,8% 34.2% 50% IR2.3 90% of HCs that report no stockouts of Depo-Provera in the last 12 months (6 months) HIS 56.25% 100% IR2.4 90% of CBD agents have received 3 supervision visits in the last quarter KPC 0,0% 70.2% 90% IR2.4 80% of CSComs that received a supervisory visit in the past 3 (6) months (routine supervision) HIS 20% IR2.7 Number of FP trainees by type of personnel and topic of training (training volume) HIS/Project reports IR1.1 40/60% of sexually active men and women report having discussed FP with their spouse or (cohabiting) partner in the last 12 months KPC 8,0% 10.8% 40% IR1.3 90% of respondents report having heard about at least three methods of FP KPC 83,8% 96.6% 90% IR1.4 75%/90% of women (15-49) know where to obtain FP services KPC 59,8% 93.3% 75% IR1.7 50% of mothers with children <12 months who received counseling about birth spacing KPC 23,5% 35.5% 50% IR4.1 Program sustainability plan in place Project reports Present ------ IR4.6 60%/80% of planned HF (CSComs and CSrefs) supervisory visits by District Medical Team are conducted HFA 25% 55% 60% IR4.7 60%/80% of HFs have undertaken a quality improvement process for FP and are measuring progress HFA 80% (Monitoring/ Assessment process) 60% IR4.8 Project partners document and disseminate the cost-effectiveness of community-based distribution in Mali Project Reports Documentation is underway National dissemination planned for February 2008 ------- IR4.9 50%/60% of HC Boards supervise CBD agents on a quarterly basis Organizational Capacity Assessment 0% 35% 50% IR4.10 50%/60% of villages with CBD/PEs have an effective OC Organizational Capacity Assessment 0% 30% 50% IR4.11 30%/30% of the membership of OCs will be women Organizational Capacity Assessment 30% 30% * Refer to Training section of report Save the Children The above table shows the project progress for the established indicators. Overall, the project did not do as well as was originally hoped. Of the 11 KPC indicators with a specific MTE objective, only three were achieved. This is explained by delays related to administration, finances and logistics, and also largely due to the very late delivery of contraceptive commodities to project communities (as previously mentioned). Since most projects need about a year to complete start-up activities such as developing MOUs with partners, training and setting up logistics etc, the MTE objectives were in fact quite ambitious. It must be noted that the ambitious nature of the indicators was due to the planned phase out of the first districts at the mid-point of the project. This phase-out has put tremendous pressure on the project to achieve results in the first districts. Under the Strategic Objective: Increased voluntary use of FP and improved FP practices, the results were actually quite good. In terms of use of modern FP, the project reached 10.6%, which was an improvement from the 8.2% baseline, but did fall short of the 17% MTE objective. Again, this is explained by the fact that contraceptive supplies were only available at the community level for 11 months. The project met its objective of percent of new users in the last 12 months (66%). With regard to continued use of FP the project was also very close to achieving its objective (57% verses 60%). Although there was no specific objective outlined for Key Results 7, the percent of mothers with adequate birth spacing showed an unexplained decrease from 82% to 69%. Key Results 4 appears to show a decrease rather than an increase in continuation rates. On the positive side, in Key Result 9 there was a notable decrease in the percentage of women with an unmet need for FP (from 88% at baseline to 79% at MTE). The IR results below are presented in the order that they appear in the DIP Table: Under the Intermediate Result 3: Increased access of communities to FP services, the results were mixed. The indicator measuring increased access (within 5 km of supply point), shows an unexpected reduction from 56% at the time of the baseline, to 47% at the MTE. This is hard to explain even though as mentioned previously, the selection criteria for target villages were not always upheld. However, the project did well on the related indicators: percent of villages meeting the criteria who have a PE or CBD agent, (46% and 78% respectively), almost meeting the 50% project objective in the first case and surpassing it by far in the second case. The project saw quite a bit of improvement in the percent of women who report discussing FP with a health worker or promoter in the past 12 months (from 25% at baseline to 34% at MTE). This fell a little short of the MTE objective set at 40%. In the area of stock-outs only 26% of agents and PEs did not experience stock-out of some commodities, verses the desired 90% MTE objective (It is important to note that the question in the survey mistakenly focused on progesterone only pills rather than pills in general. Thus this is what the data reflects. There has been no stock-out of the combined oral pill). The stock-out of vaginal foam tablets was commonly reported by both community volunteers and health providers (up to the district level). It appears that the popularity of this method was underestimated, in comparison with the large stock of the combined pill. Another indicator reflecting positively on service provision is that 35% of HFs are providing the injectable Depo Provera during monthly community outreach activities (HIS reports). Although this did not quite meet the MTE objective of 50%, it was still a big improvement from 0% at baseline. Under the Intermediate Result 2: Improved quality of FP service delivery by providers in facilities and in the community results were good for the most part. Adequate counseling saw a big increase from the baseline (13% to 34%), although it did not quite meet the 50% objective; 100% of HFs reported no stock-outs of depo-provera in the last 12 months; 70% of CBD agents received three supervision visits in the last quarter verses the 90% MTE objective; and 20% of HFs received a CS-20 Mali, Ségou, October 2007 13 Save the Children supervisory visit in the past three months. (Please refer to the Training Table for the number of FP trainees by type of personnel and topic of training.) Under the Intermediate Result 1: Increased Knowledge and interest in FP services through PVO/NGO involvement the project saw some very good results. Communication about FP with spouses was only 11% compared to the 40% MTE objective. With regard to the percent of women who had heard of at least three FP methods (97%), and the percent of women who knew where to obtain FP services (93%). In both cases, the MTE objectives were surpassed (90% and 75% respectively). The project also saw an improvement with the percent of women with children under 12 months receiving counseling about birth spacing (36%), compared to 24% at baseline. Under Intermediate Result 4: Improved social and policy environment for FP services and behaviors a program sustainability plan was put in place (Please see sustainability section – Table 6 and Detailed Action Plan Annex J); 55% of planned supervisory visits by the District Management Team were conducted. A quality improvement process has been undertaken by HFs during the two day monitoring visits with 80% having undertaken this activity (MTE objective was 60%). None have undertaken a quality assurance (QA) process outside of this bi-annual monitoring activity. Thirty-five percent of ASACOs supervised CBD agents on a quarterly basis which is a significant improvement from the 0% at baseline, although not quite meeting the 50% MTE objective. With regard to the FP Oversight Committee, 30% of villages report that they have an effective committee, verses the 50% MTE objective. Of these committees, only 30% have female membership, however this is an improvement as there was no such structure at the beginning of the project and women generally were not part of community leadership. iii. Effectiveness of the interventions. The CBD approach has clearly been effective when one compares the data between the Community Zones and Health Systems Strengthening only Zones, as highlighted in the table below. Table 2. Effectiveness of Interventions Indicators Community Zone (Combined approach) Health System Strengthening only Zone (HSS only) Contraceptive prevalence 15% 7.9% Resides within 5 km. from FP services 78.6% 26.9% Received appropriate counseling 39.5% 28.6% Communicated with spouse in last 12 months 12.8% 9.6% Knows a source of FP contraceptives 93.2% 86.1% Mothers with child under 12 months are informed about child spacing 36.5% 27.9% As per community interviews with women during the MTE, the CBD agents are known in the community and women are aware of their services. CBD agents were also cited as a source of information and services for FP in addition to HF staff (25%). The project did not disaggregate the CS-20 Mali, Ségou, October 2007 14 Save the Children response from the areas with CBD agents versus those without them. But nevertheless, it is clear that the presence of a CBD agent in villages has increased access to information, interpersonal communication and counseling. According to program staff, the PE intervention did not seem to have as much impact as hoped in this context. Peer educators are generally youth under the age of 25 and unmarried. As previously mentioned, most girls in the Ségou region, as well as most rural areas of Mali, marry at a very young age (usually by mid teens). They are thus no longer considered youth, but rather young married women who can more easily be reached by fellow married women. It was also mentioned that once married, it is not socially acceptable for young girls to continue participating in youth group activities. Young married men, on the other hand, often continue participating in youth groups without any social bias. The usefulness of the Oversight Committees was also questioned. Although the idea was to develop a non-technical support system for CBDs, there is a question as to whether or not these structures are also susceptible to issues related to motivation as well. Some project staff reported that these entities become yet another community structure that needed to be provided with an orientation on FP and supportive supervision. Some Animators reportedly found it difficult to find enough time to support the Oversight Committees in addition to the CBDs and PEs. Field staff report that BCC activities such as organizing soccer matches and folkloric activities around the national FP campaign, as well as village cultural events have been effective strategies to increase the visibility of the project and for FP promotion and information. Program staff and partners interviewed highlighted advocacy with the religious leaders as having been very effective. This was an intervention that exceeded everyone’s expectations because as previously mentioned, it not only resulted in the development of FP/Health Committees both in Macina and Baroueli, but these highly respected religious leaders now openly support FP and child spacing for the health of both mother and child. In view of the influence that these religious leaders have on social cultural norms and behaviors, this was a big accomplishment for the project, and to the degree that participating health zone level Imams have reached people with a positive message about FP, it certainly has been effective. This has been a great beginning. During the MTE interviews, the religious leaders suggested that next steps should include building their capacity in communication to facilitate community mobilization activities; creating forums for exchange between religious leaders; providing them with tools and materials for BCC; strengthening their capacity in key areas of health; and involving them in all community mobilization activities. With regard to the radio broadcast success in Macina, this has been quite effective in stimulating community interest. Discussions during the MTE centered on the added value of this effort because of the survey results showing that only 23% of women had heard FP messages from the radio. However, the project has not verified whether or not the source for these messages was the local radio station that they worked with or whether it was the national radio stations that also reach this population. According to program partners, in rural areas men tend to be the ones with radios (small portable radios that they carry). Although men were not surveyed, one can assume that a bigger percentage of them would have heard these messages since they are the primary listeners. This is one clear way to reach men with promotional messages and an opportunity to correct misinformation about FP. Per the MOH district and HF partners interviewed, project health service strengthening activities have made a difference in FP service provision. Additionally, education sessions consistently being CS-20 Mali, Ségou, October 2007 15 Save the Children conducted in the HFs and during community outreach (Strategie Avance) are also effective since HFs are still cited as a main source of FP information and services in the survey (75%). When asked about the privacy issue with outreach services and concerns from clients who may not want their husbands to know that they are using FP, one provider interviewed said that they provide services in a space that allows for privacy and so are able to ensure confidentiality during these outreach visits as well. It must be noted though that results from the service provider MTE interviews show that there is still a lot of work to be done - particularly in the area of medical eligibility criteria and attitudes. (Please refer to Health Facility Strengthening section). iv. Changes in the technical approaches outlined in the DIP and rationale There have been no changes since the DIP. v. Special outcomes, unexpected successes or constraints The advocacy initiative with the religious leaders under this project was a great success. Workshop activities focused on highlighting Koranic verses demonstrating that Islam is not against FP. Key religious leaders in the intervention areas who were initially unsupportive, have now changed their position on FP and have promoted it in sermons and in their interpersonal communication with their membership. This is discussed in more detail under the Results Highlight section at the end of the report. Constraints 1. Delay in provision of contraceptive commodities. 2. Due to limited funds, the project could not implement more than one or two organized BCC events per health zone. vi. Follow-up and next steps As per the results of the cost-effectiveness study discussed below, Phase II will scale up the community strategy to the two new districts of Niono and Tominian. Because the PE strategy was not as effective as originally expected during the MTE, the project team decided that they would not invest in this intervention for the Phase II scale-up. With regard to the creation of Oversight Committees, the project staff will revisit this issue through a closer review of the effectiveness of the existing structures. Another strategy under review will also be that of the radio program. The issue of limited project funds and the need to prioritize what works and has an impact, is the main impetus for this. Program staff will be assessing access to radio programming at the village level, which radio programs are most listened to, and the degree to which they are heard. In order to build upon the success of advocacy with the religious leaders, the project will expand efforts to try and reach as many of the religious leaders at the village level as possible. In the MTE sample communities visited, it did not appear that women in villages had heard FP messages from a religious leader. Thus having garnered the support of important religious leaders at the district headquarters and health zone levels, the project can now work to reach more of the lower level religious leaders located in villages. CS-20 Mali, Ségou, October 2007 16 Save the Children The project will also be increasing efforts to sensitize men and engage them in dialogue on FP. Staff and partners will also explore developing specific strategies to encourage communication between couples. (Please refer to attached Action Plan for more details.) c. New tools or approaches, operations research/special studies The Ségou project adapted existing MOH tools for community and HF supervision and reporting. Although the project did not implement new approaches, as mentioned before, they split up the intervention area into community zones and health zones to compare the cost-effectiveness of the two strategies. CS-20 wanted to investigate whether CBD of FP services is a cost-effective mode of service delivery by testing the hypothesis that the impact of combining community-based services with health services strengthening is worth the additional cost, compared to simply strengthening health services. The complete report is included in Annex G. The study objectives included: ƒ The total costs of providing contraceptive services through CBD programs in selected health zones in Baroueli and Macina; ƒ The total costs of providing contraceptive services within HFs including HCs and district hospitals; ƒ The effectiveness (measured by CYP) for each mode of service delivery; and ƒ The cost effectiveness (measured by CYP) for each mode of service delivery. The baseline study was conducted in all intervention areas to allow for the desegregation of results by study group. The MTE assessed factors such as CPR and a separate cost-effectiveness study was also undertaken. Results have shown that when excluding the additional cost of a participating PVO/NGO, the community intervention does not cost more. Thus with an initial investment in training of MOH partners and service providers, this strategy can be scaled up without costing more to the existing health system. The study applied a randomized experimental-control group design to health zones in Macina and Baraouéli districts. Fourteen health zones in the control group received a “health systems strengthening approach” (local health centers are provided training and resources to reinforce the provision of FP services) while fifteen health zones in the experimental group received the “community-based distribution intervention” (CBD agents generate interest in use of FP through administering contraceptives or refer clients to the health center for FP and health centers.) The two strategies both use BCC efforts to increase use of FP services. One strategy will be selected for Save the Children’s second phase project work and for the MOH’s efforts toward supporting FP in Mali. Methods In order to assess the cost-effectiveness of the two alternative models, staff from the Mali MOH, Save the Children, and Family Health International (FHI) first developed a base case cost analysis by outlining the direct financial and non-financial resources used in delivering the two approaches as they were implemented from March 2006-March 2007. The team created and organized an Excel￾based spreadsheet workbook to track all intervention-related costs associated with inputs such as project planning time and trainings, transportation, supervision, equipment, contraceptive supplies CS-20 Mali, Ségou, October 2007 17 Save the Children and other consumables needed for delivering the interventions such as t-shirts, sacks, visual aids, and printed materials. In order to measure effectiveness, contraceptive utilization or uptake data were collected from each of the sites as attributed to each of the interventions. Conversion factors were applied to determine couple years of protection (CYP) for each of the contraceptive methods and overall CYP based on methods developed by Stover et al. 1 The analysis was conducted from three cost perspectives. In the first scenario, project staff examined the cost per CYP using direct costs only (as described above) in the numerator. Furthermore, in a second cost-effectiveness scenario the numerator includes additional data pertaining to MOH personnel time and costs. In a third scenario, staff include NGO costs such as the personnel costs of Save the Children in the numerator. Results The total direct (base case) costs amounted to 10,586,607 CFA for the HSS intervention and totaled 18,918,235 CFA for the CBD plus HSS intervention. Applying a conversion factor of 500 CFA = $1US, this amounts to $21,173 and $37,836 respectively. Under scenario 1, for HSS, the annual cost per CYP is 7,581 CFA or $15.16. For CBD plus HSS, the annual cost per CYP is 16,702 CFA or $33.40. Under scenario 2, the annual cost per CYP for HSS is 10,254 CFA or $20.51. For CBD plus HSS, the annual cost per CYP is 20,308 CFA or $40.62. Under scenario 3, the annual cost per CYP for HSS is 27,008 CFA or $54.02. For CBD plus HSS, the annual cost per CYP is 40,963 CFA or $81.93. Discussion Although the incremental cost per CYP is lower under the HHS intervention, the additional cost of $18.24 per CYP under the first cost-effectiveness scenario (direct costs only) represents the cost of “accessibility” or reaching segments of the population that experience significant barriers to accessing FP services in the zones receiving the community-based intervention. Possibilities for increasing the cost-effectiveness of CBD exist and should be further examined in Phase II of the project. Sensitivity analyses indicate a strong possibility of increasing CYP (and thus decreasing the cost per CYP) under the CBD approach by adding long-term FP approaches to the mix of methods provided to the population under this strategy. The MTE team recommends piloting this type of strategy in one or two health zones in Phase II of the project. Furthermore, training costs (particularly curriculum development) will most likely be reduced in the second phase while contraceptive uptake could be increased through enhanced BCC efforts, community sensitization, and expansion beyond the initial pilot phase. Given the potential change in these factors, the MTE team recommends further investigation into the cost-effectiveness of scaling up CBD in Mali in the second phase of the CS-20 Ségou. CS-20 Mali, Ségou, October 2007 18 1 Stover, J., Bertrand, J., Shelton, J., Empirically Based Conversion Factors for Calculating Couple-Years of Protection, Evaluation Review, Vol. 24, No. 1, 3-46 (2000) Save the Children 2. Cross-cutting approaches Cross-Cutting Activities (Community Mobilization and BCC): ƒ Radio broadcasts A contract was signed with local radio stations in both Macina and Baroueli. These radio programs seem to have had different degrees of success. The radio program was particularly innovative in Macina, where program staff report that they organized and participated in roundtable discussions and interview sessions with the radio announcers around the various FP issues, i.e. advantages of FP, contraceptive methods available at different levels, the management of side effects and male involvement. These radio programs were on the air several days a week, and generated a lot of interest, questions and comments from women as well as men. One PA was even able to take the radio announcer with him to visit the community during cultural events. In Baroueli, each PA was responsible for supporting the radio program once a month. ƒ Advocacy workshops with Imams The Ségou project collaborated with the POLICY Project for this intervention. The advocacy activity planned was in fact an opportunity for the Resources for, and Analysis of the Population and it’s Impact on Development (RAPID) model developed by POLICY for religious leaders to be piloted in the Ségou region. The High Islamic Council (HIC), which has been working in collaboration with POLICY were the lead facilitators of these workshops. The workshops discussed population issues and their impacts on health, education and the economy. The overall aim of the workshops was to refute the myth that the Koran is against FP, by highlighting verses specifically in support of child spacing for maternal health. The district of Macina organized one workshop which had excellent participation from religious leaders and representatives from the district level as well as the health zone levels. As was expected, participants’ initial response was skepticism; and for this reason the presence of the HIC expert Imams was extremely critical. The workshop turned out to be a tremendous success, culminating with the group decision to form a committee to highlight the fact that Islam is not against FP and actually encourages child spacing for the health of the mother and child. In the district of Baroueli, they were able to take the advocacy a step further by organizing workshops in every health zone as well. This allowed for greater participation from the religious community and closer access to the village level. Baroueli leaders formed a committee as well. Imams who have participated in these workshops have been reported to have disseminated positive messages about FP during sermons. Religious leaders have also participated in the radio programs on FP; they have participated in community activities at the district level; and they have encouraged support from religious leaders who did not participate in the workshops. ƒ Development of Oversight Committees This community mobilization activity proposed the identification of approximately five village leaders who would provide a support system for CBDs. Typically Oversight Committee, helped CBDs to mobilize people for education sessions, and assisted them with issues or problems. CS-20 Mali, Ségou, October 2007 19 Save the Children a. Community Mobilization i. Activities undertaken by the project As discussed previously, once target villages were identified, the project linked with their respective community leaders to identify CBD agents and PEs. Criteria included that these individuals be stable residents of these communities. Each PA and collaborating HF In-charge (Chef de Poste Medical) took the responsibility of training the group of volunteers identified in their catchment area villages. The project also worked to mobilize community leaders to form Oversight Committees. This proved to be a little difficult for some Animators as they were already busy supporting newly functional CBD agents, PEs and health providers. ii. Community responsiveness, social cohesion and community capacity Targeted communities have been very responsive to the project. Volunteers identified are, for the most part, active and recognized by community members. The project has generated a good level of support for FP. The village chiefs and their advisors have been supportive and facilitated the community mobilization process. Some advisors were even chosen to be the CBD agent and community capacity through CBD agents and PEs has been developed. The latter have knowledge of FP methods and potential side effects, manage and keep records of their stock of contraceptives, maintain records on FP clients and utilization, and educate and counsel clients. Where Oversight Committees exist, these leaders often help the volunteers by notifying community members about education sessions or community outreach activities. Although women in MTE interviews said that their spouses do not support them using FP, one project staff pointed out the fact that the community has accepted project activities and that women are not being prevented from attending education sessions. Thus it is clear that there has been a lot of progress towards acceptance of FP by the male leadership in the community. Since only 10% of women interviewed during the MTE quantitative survey had actually discussed FP with their partners/spouses, it is very difficult to know whether or not they would meet with opposition on the subject. One might argue that women assume that their spouses will be unsupportive because of the historical context/cultural and social norms, and this may not always be the case. iii. How activities have been used to refine project implementation plans Please refer to comments in the Follow-up and Next Steps Sections, i.e. PE, Oversight Committees and expanding advocacy with religious leaders. The project experience in this first phase has also helped staff realize that there needs to be closer participation in the selection of villages meeting the criteria for a CBD agent, and more care with the geographical assignments of Animators to ensure equal work load. iv. Barriers to prevent members of the community from benefiting from the project It should be mentioned that the district of Baroueli was particularly known for being extremely religious and even hostile to FP prior to the arrival of the project. FP activities had only really been undertaken in one health zone. It is thus to the credit of the project that at this point women and community volunteers alike, interviewed during the MTE activity do not feel that there are any barriers that prevent them from benefiting from the project. This is a little contradictory to the general sense that women feel that their spouses are against them using FP. However, it is clear from several sources, including women and health providers, that lack of spousal support does not prevent women from using FP services as many do so in secret, hiding the fact that they are FP clients from their husbands. Other barriers mentioned include poverty and the cost of methods – CS-20 Mali, Ségou, October 2007 20 Save the Children many women cannot afford to pay for these on their own, and household finances are still controlled by men. v. Impact of community factors on project implementation There are a couple of factors that do have an impact on project implementation: one is the difficult terrain especially during the rainy season when some HFs are unable to conduct outreach activities because the roads are impassable. As the rainy season can last for a couple of months (July/August) accessibility to services during these months is limited. A second factor affecting community participation and use of services is the mobility of the population. Project staff and health providers discussed the fact that a good portion of the population in the region moves away from January–July to look for employment and additional income. Thus it is often difficult to find enough community members around for IEC/BCC activities, and difficult to know if an FP client has dropped out when she doesn’t return for services during these months. It must be noted that CBD agents and PEs are also amongst those who leave periodically even though one of the criteria for selection of community volunteers was that they be individuals who do not move. The reality is that migration for economic survival is a part of life in this region and this is not something that the project is able to resolve. For example, during the MTE surveys of community agents in May 2007, the interviewers were only able to reach a sample of 37 of these volunteers in Macina (total 114), compared to a sample of 85 in Baroueli (total 126). Project staff has also mentioned that the low level of education and literacy in the population does have an impact on how dynamic community volunteers are. b. Communication for Behavior Change i. Behavior Change strategy The project has put in place a comprehensive behavior change strategy. As outlined in the Activities section, one of the main activities is providing community-based services via the establishment of a CBD and peer education program to increase access to information, education, counseling and FP services. In addition, the project conducted a very successful advocacy effort with the Muslim leadership in the two districts; implemented a radio program; and has occasionally promoted FP through cultural and sports events in the communities. ii. Appropriateness and effectiveness of project’s approach to behavior change and how barriers are being addressed As per interviews with project partners at different levels during the MTE activity, all are very much in support of the project approach to behavior change. Health providers across the board say that they have seen an increase in the number of FP clients as a result of the presence of CBDs in targeted villages, as well as the fact that the HFs are actively providing FP education, counseling and services now. Everyone is very enthusiastic and excited about the impact of the advocacy workshops as well, and program staff in Macina was very happy with the success of the radio program there. To address the biggest barrier to FP (religious and cultural norms as well as men), as previously discussed, the project dedicated a lot of effort towards advocacy with religious leaders. In addition to this, by working with the community leadership at the village level, and in some cases, successfully establishing Oversight Committees, the program has begun the process of addressing the above mentioned barriers. During the MTE, it was noted that more effort needs to go into sensitizing men CS-20 Mali, Ségou, October 2007 21 Save the Children on FP. Program staff and MOH partners mention evidence of progress in this area though as some men have been seen escorting their wives to HFs for FP services. Although it is often difficult to ensure active male participation in education sessions, a few men in communities have also sat in on some of the education sessions facilitated by CBDs and PAs. iii. FP Messages Messages used by the project are consistent with what is being used at the national level. Project staff participates in the national FP working group and also participated in a workshop to review existing IEC/BCC materials. One FP message that the project has not been able to promote is that FP enables women and families to limit the number of births. Although permanent methods (tubal ligations) are generally available at the district level, because of the country’s socio-cultural norms, the MOH has agreed to promote ‘birth spacing’ but does not want to focus too much attention on limiting family size. Thus efforts to promote or provide access to longer term methods are limited. The fact that Mali has been identified as one of the countries for ‘Repositioning Family Planning’ is encouraging, and with so many key partners in-country, not to mention the high maternal and infant mortality and food insecurity, it is likely that more efforts in this direction will be forthcoming in the future. iv. Changes due to BCC messages Messages highlighted during advocacy workshops which successfully clarify that the Koran and Islam are not against FP (by identifying hadices that support birth spacing), are an example of how the project has been able to negotiate changes and have an influence on social norms. Religious leaders are amongst the most respected individuals in the Malian society, and having them support FP has begun, and is expected to continue to influence changes. Messages disseminated during education sessions in communities have also elicited a lot of interest and discussion. The increased number of women seeking CBDs and health providers for services is evidence of the influence that these messages have had on traditional behaviors and norms. During interviews in communities and with the ASACOs (HF committees), several people mentioned that in the past birth spacing was practiced via traditional methods that are now no longer being practiced as a result of the project. v. Measuring effects of behavior change activities and tools The effects of the behavior change activities are being measured mainly through routine monthly data reported by CBDs and health providers. This monitoring is quite effective. At the community level, volunteers report on new and old FP clients and utilization of the stock of contraceptives which lets them know whether BCC activities are having a positive influence. Volunteers use simple record books to document this information as well as information on client evaluation and diagnosis, and monthly BCC activities, follow-up and referral. PAs collect and compile this data, and analyze this with the HF; including the HF data as well. In addition to this, the project has supported the development of ‘Tableau de Bords’ or data charts posted in the HF. Using annual objectives, the In-Charges are able to calculate the number of clients that they need to see per month in order to achieve their objective. Based on the number of clients seen during the month, the chart is updated and allows them to measure their progress towards achieving this objective. Thus the tools being used do seem to be very effective and appropriate. The MTE quantitative survey conducted was also an opportunity to measure the impact that the project has had since the baseline in early 2005. As discussed in the Results Section above, the MTE results are an indication that BCC messages have had an influence on the use of modern FP, an CS-20 Mali, Ségou, October 2007 22 Save the Children increase in knowledge of methods, and dramatic increase in knowledge of where services can be accessed. vi. Use of data As per questions asked during the MTE, the data gathered on a monthly basis is used primarily by the project and collaborating MOH staff at the HF, district and regional levels. CBD agents and PEs at the community level do understand their data and can see when they have sold a lot of contraceptives verses when they have not. Unfortunately, not a lot of discussion about this data seems to occur at this level or to be used by the communities. PAs and the HF In-charges, on the other hand, gather this information, compiling the CBD and HF data and analyze progress during the month in terms of clients, counseling sessions, and use of stock. This data is also analyzed by project supervisors who compile data for their area before submitting it to the project coordinators. This data simultaneously goes to the district office which subsequently submits reports to the region. Data is discussed during monthly meetings which are joint meetings with all project field staff, In￾charges and the district team. In Macina, the In-Charge (CPMs) attended monthly meetings. In Baroueli they did so periodically in a larger partners meeting held in Bamako quarterly. vii. Successful innovative approaches Although advocacy with religious leaders is not new, the experience is indeed notable, as mentioned above. c. Capacity Building Approach i. Strengthening the Grantee Organization • Capacity building objectives, indicators and targets. There were no specific capacity building objectives for Save the Children staff. • Approaches and tools used to assess capacity Not applicable to Save the Children under this grant. • Organizational capacity building of grantee There were no activities related to organizational capacity building of the grantee. • Indications of increased organizational capacity? Although there were no specific organizational development objectives, it is clear that the organizational capacity of Save the Children/Mali staff has been increased as a result of implementing this project. Staff in-country was able to participate in a CSTS Project Design Monitoring and Evaluation workshop. Staff benefited from an orientation on the KPC survey and they have also learned new skills in the areas of advocacy and contraceptive logistics through partners such as POLICY and DELIVER. The biggest area of learning reported is that of partnership and collaboration. SC also participated in the PDQ workshop, and the methodology is now being used by other organizations as a result of the training. CS-20 Mali, Ségou, October 2007 23 Save the Children ii. Strengthening Local Partner Organizations • Organizational capacity building efforts with the local partners In addition to the MOH, the main SC partner for the Ségou FP project is Group Pivot/Sante Population. As mentioned in the project DIP, GP/SP is an umbrella organization that has many years of experience providing management and technical assistance to NGOs implementing health programs. Areas for institutional strengthening identified and progress or lack thereof are included in the table below. As discussed in the DIP, ASDAP and the Consortium AMPRODE SAHEL/JIGI were the two local NGOs identified as the implementing agencies for the first phase of this project in Macina and Baroueli. Several areas for improvement were discussed in the DIP and are included in the table below along with progress made to date. Table. 3. Partner Progress CS-20 Mali, Ségou, October 2007 24 Group Pivot needs identified Effort/Progress made The need to put in place a strategy for sustainability of its activities and those of NGOs; Not done. The development of a policy to sustain funding; Not done The need for representation in the regions. Not done ASDAP and Consortium AMPRODE Sahel improvements At the NGO headquarters in Bamako: Provide orientation to staff on SC and USAID procedures This was done at the very beginning of the project Funding: Diversify donor base and increase resource mobilization ASDAP was able to get an FP flex-fund project funded through World Learning as a result of their experience with CS-20 Ségou. Communication: Learn English in order to facilitate communications with partner organizations and access to working documents. This was not done because after project start-up, it was not something that was perceived as a priority. Budget: Need to improve accounting system to enable analysis of variations in expenditures by pipeline analysis. There has been no progress in this area. In fact there is a need to address delays in financial reporting resulting in delays in getting money to the field for activities and salaries. Equipment: Need to set up an accounting software system Not done. Each NGO uses its own software system.. Provide NGOs with a computerized database to maximize the impact of program information and other interventions. Not done. Field staff: Provide training and supervision in order to build the capacity of NGO field staff in FP, BCC and CBD. Training was provided to all project coordinators, supervisors and animators in the field. Those interviewed felt that their skills in these areas and Save the Children programming in general had been strengthened as a result of the training, supervision and technical support provided by the project. Establish a process for information sharing and regular reviews. The field level staff hold monthly staff meetings that the Bamako staff participate in from time to time. Data is reviewed during these meetings among staff and MOH partners in the HFs and districts. The Ségou FP project has also instituted quarterly meetings in Bamako that the field coordinators participate in. The purpose of these meetings is for sharing, review, problem solving and planning. Note that although no effort was put into the GP identified areas of need mentioned in the DIP, they did strengthen their capacity in other ways: GP Ségou project staff and 23 member NGOs benefited from training in PDQ; capacity of Training Specialist in use of computers was strengthened; project accountant received training in grants management and SC accounting system; and GP staff received training in PDME. In the CS-20 project DIP, other areas identified for improvement included: training in management skills, and planning and use of HIS tools for improved data collection. As mentioned in the above table, quarterly meetings helped to strengthen skills in planning. GP/SP staff seconded Save the Children for this project (specifically the Training Specialist) participated in the adaptation of MOH tools for this project. The implementing NGOs using these tools have also strengthened their skills as they use these tools for data collection and planning on a monthly basis. Improvements in management skills include program planning, development of proposals, partnership and communication. With regard to the MOH, the main capacity building activities were related to revitalizing FP in the region. Several training activities were undertaken in conjunction with the region and district. The cascade training approach was used, and a team of master trainers was established at the regional level. All health providers were targeted (please see the Training section), and areas of improvement included quality of care, BCC and outreach, and access to FP services including counseling and contraceptive method choice. With regard to the partners at the community level, these were identified as the Health Center Committees (ASACOs), and the Village Oversight Committees. The prior received training from the project. Specific capacity building objectives are included in the Results Summary Table and discussed under IR4. Table 4. ASACO Functional Elements that Need Improvement Strategy Activity Effort/Progress made Improve management skills Improve procurement of essential supplies Train and supervise ASACOs in financial management This was done within the training highlighted in the training table below. Build capacity in planning health activities. Training ASACOs in management and monitoring of activities. Same as above. CS-20 Mali, Ségou, October 2007 25 Save the Children Support VOC and CBD agents Orient ASACO members on the roles and responsibilities of CA. Train ASACO members in supervision techniques Develop supervision plan and supervision guide. Involve ASACO members in the supervision of community activities ASACOs members are community delegates and were given an orientation on the community strategy. They were part of the planning for CBDs and PEs, roles and responsibilities. Supervision conducted by the ASACOs of CBDs was done by the delegate from the respective villages. Per the MTE results 35% did this. • Roles and responsibilities of each of the local partners GP’s responsibility is for the overall management of the two local NGOs while Save the Children has an agreement with GP, who in turn has a agreements with the two NGOs. All funding to the NGOs goes through GP. ASDAP and Consortium AMPRODE SAHEL/JIGI were responsible for direct implementation and employed field staff to be able to carry this out. All of this was detailed in the DIP and there haven not been any changes since the DIP. • Outcomes of any assessment At the beginning of the project organizational assessments were conducted with the above￾mentioned local NGOs as well as all the others under consideration. This assessment included visits to their field offices and activities. The tools used to evaluate organizational capacity were an institutional questionnaire and a capacity assessment which mainly focused on ten major elements including area of health expertise, governance, human and material resources, and financial administration. Overall the best NGOs which were selected to implement activities were ASDAP (88%) and AMPRODE – SAHEL/JIGI (83%). Health Facility Assessments were also conducted. Overall, it was clear that FP was seen as a secondary activity as many facilities did not have contraceptives available because they did not have many clients. There was no effort to create demand, very few community outreach services taking place, and no counseling of WRAs during visits to the clinic. CPMs tended to see this as the matrons’ work and were not very involved. The district teams also saw FP as something secondary as they were not receiving funding to support this activity. They also perceived FP as somewhat of a challenge in this environment and so FP was not part of monitoring or planning activities undertaken by the districts and regions prior to the start of the project. • Changes in organizational capacities of the local partner As these NGOs were contracted for Phase I of the project, a follow up assessment is currently being conducted before decisions are made for Phase II implementing partner(s). NGO staff interviewed during the MTE did feel that they had developed their skills in FP as a result of the project; increased their capacity in the area of training of community volunteers, supervision, analysis, planning and management skills. Staff report that project training activities and technical support from the project and external partners are the reasons for their increased skills. CS-20 Mali, Ségou, October 2007 26 Save the Children A follow-up assessment was conducted by the team as part of the midterm evaluation. The assessment included a review of management documents and resources, and interviews with executive directors, administrative staff, accounting managers and program officers The table below summarizes findings from the assessment FINDINGS DESIGNATION CONSORTIUM AMPRODE SAHEL /JIGI ASDAP 1. Analysis of structural authorities At this level, we noticed the existence of a general organizational plan, clear and updated for Jigi and in process of being updated for AMPRODE. In addition to that, we were able to verify the existence of regulations and status, the legal authorization of settlement; a manual of procedure updated in 2004 for Jigi and in process of being updated for AMPRODE. Insufficiency in the structuring of the organization chart. Organization chart not displayed. Existence of a general organizational plan, clear and updated. We were also able to verify the existence of regulations and status, the legal authorization settlement ; a manual of procedure in process of being updating Organization chart not displayed 2. Analysis of the internal control system We noticed an internal control system done by an advisory cabinet led by a team of accounting commissioners. People in charge of internal control promptly proceed to the control of the balance, reports and other. Existence of an internal control system conducted systematically and annually by an advisory cabinet 3. Analysis of finance and accounting system Compliance with the account management cycle by the specification of accounting documents, use of Excel Spreadsheets. Existence of petty cash ; the recording operations are done daily and the different supporting documents are classified and archived Agents are paid in cash and not by bank transfer and the salary notes are not systematically given to them. The accountants and field agents don’t have a good mastery of GP/SP and Save the Children administrative and financial procedures. Compliance of the account management cycle by the specification of accounting documents, use of Excel Spreadsheets. Existence of petty cash ; the recording operations are done daily and the different supporting documents are classified and archived Agents are paid in cash and not by bank transfer. The accountants and field agents don’t have a good mastery of GP/SP/Save the Children administrative and financials procedures. 4. Analysis of procedures manual Existence of a manual of procedure but which needs to be updated. Existence of procedures manual that is the process of being updated. 5. Analysis of annual financial reports The reports are regularly produced according to the guidance on grants’ contract. The reports are regularly produced according to the guidance on grants’ contract. CS-20 Mali, Ségou, October 2007 27 Save the Children 6. Analysis of Records system In general, the recording system is good In general, the recording system is good 7. Budget follow-up The System of budget follow-up is compliant with the guidance with grants’ contract The System of budget follow-up is compliant with the guidance with grants’ contract 8. Bank reconciliation status Statements are regularly produced and they are concordant Statements are regularly produced and they are concordant 9. Analysis procurement system The system puts in competition the potential suppliers and respects the procurement procedure. The system puts in competition the potential suppliers and respects the procurement procedure. 10. Analysis of stocks’ management Stock non existent Stock non-existent 11. Analysis of fixed assets Existence of a register of fixed assets which precises the serial number, the cost of the procurement, the purchase date and the funding sources. The consortium conducts an annual physical inventory of its properties, by précising the different transfers. Existence of a register of fixed assets which precises the serial number, the cost of the procurement, the purchase date and the funding sources. Conducts an annual physical inventory of its properties but doesn’t precise the transfers conducted during the year. The organizational capacity in the 2 NGOs has notably improved though despite room for improvement in certain aspects. Points were assigned during the assessment and conducted out of a total of 31 points: • The Consortium AMPRODE SAHEL /JIGUI received a total of 26 point (20 points at baseline in 2005) • ASDAP received a total of 28. (21 at baseline in 2005) These results illustrate the relative good management of the two organisms, which proves that they are able to continue even though a number of improvements need to be made. The following recommendations were made based on the results and we believe that if implemented will lead to an improvement in the quality of their respective management system. A- NGO: CONSORTIUM AMPRODE SAHEL/JIGI: 1. Salaries should be paid by bank transfer, 2. Monthly salary statements should systematically be given to all the agents, 3. Proceed to update the organizational structure chart by précising the positions, 4. Display the organization chart in the offices, 5. Promptly review the procedures manual, CS-20 Mali, Ségou, October 2007 28 Save the Children 6. Insure real estate properties against natural disasters, 7. Organize an orientation session for accountants and field agents on GP/SP & Save the Children administrative and financial procedures in collaboration with different departments. B- NGO: ASDAP: 1- Salaries should be paid by bank transfer, 2- Insure real estate properties against natural disasters, 3- Display the organization chart within all the offices and on the display board, 4-Improve the inventory system of properties at the end of year by précising the different movements, 5- Promptly review the procedures manual 6- Organize an orientation session for accountants and field agents on GP/SP & Save the Children administrative and financial procedures in collaboration with different departments It is important to note that after undertaking the Organizational Capacity Assessment of the two NGOs, ASDAP made a decision to withdraw from the project. Consortium Amprode-Jigi Sahel will therefore be the only implementing local NGO in Phase II. • Challenges in further building the capacity of project partners Primary challenges are mainly related to the limited funding available to the project. Because this project involves a partnership of four major institutions, in addition to MOH and other partners, the budget is spread quite thinly. iii. Health Facilities Strengthening • Appropriateness and effectiveness of activities HF strengthening activities have mainly focused on training the In-Charge (CPM), matrons and the drug depot managers (Please refer to the Training Table below). In addition, this training is accompanied by the placement of a PA in every health center. These Animators ensure that the necessary follow-up support is available to the staff, including: modeling education session methodologies; counseling for FP; trying to address missed opportunities; FP data collection; general support given to the drug depot managers and ASACOs during routine activities; and monthly and quarterly meetings where data is discussed and micro-planning is done. Placing an Animator within each health center was a very effective strategy (per MTE interviews with HF staff). Overall, the CPMs who had previously designated FP as a responsibility of the midwives, have since increased their ownership of this as part of their overall responsibility. As per the reports, they monitor activities, see clients when the Midwives are busy, are involved with planning for outreach activities that include FP, and analyze the data with the Animator at the end of every month. Although placing a PA in each HF (health zone) is an appropriate strategy, it does come with a high CS-20 Mali, Ségou, October 2007 29 Save the Children cost. Since the new districts have a total of 36 health centers, allocating at least two health centers to one Animator in the upcoming Phase II will likely be more financially feasible. The HFA conducted at this midpoint in the project gives a good idea of how effective the above mentioned activities have been. (Please refer to the attached report). A total of 52 health providers were interviewed at 27 of the 29 HFs being supported by the project, and it was found that 92% are involved in FP service provision and had provided services during the three months preceding the survey. • Tools for health facility assessments The project used the standard HFA tool to conduct a baseline assessment. This tool was appropriate and effective because it allowed the project to identify challenge areas such as: most HFs did not have sufficient contraceptive supplies; most had low FP use; as a rule, HFs did not provide counseling to clients nor did clients have a choice of methods available to them; there was a lack of basic equipment to provide quality services, including infection prevention equipment and tools; and there were limited FP outreach services due to a lack of staffing and financial resources. The health system has instituted a big monitoring activity every six months. This consists of a visit to all HFs by members of the district health team. The project supported FP monitoring and a tool was also developed for this purpose. During this monitoring assessment, the team looks at the management of stock of contraceptives, clients, FP activities undertaken including counseling services, education sessions, community outreach etc. This activity is scheduled over a two- day period at the end of which the team discusses results and recommendations for the health service providers and ASACOs to address. These recommendations are kept on file both at the HF and district to be referred to during the next monitoring visit. According to the district representatives, these tools have been very effective, enabling them to see improvements in such things as welcoming clients; HFs have prepared a space for FP counseling; and the method of prescribing contraceptives has greatly improved (before clients could go straight to the Drug Depot Managers for re-supply without a prescription. Now there is a follow-up counseling and all clients have a card for documentation prior to re-supply). The project conducted a follow up HFA at the mid-term as well. • Linkages between these facilities and the communities In the community zones, CS-20 established a CBD and PE presence, gave them an initial stock of contraceptives and built their capacity with the collaboration of the catchment area HFs. As a result, all volunteers and health providers are well acquainted with each other. At this point, CBDs can go to HFs and report on clients, contraceptive use, and receive a new stock as needed. This was happening in various places at the time of the Animator withdrawal in May. Community volunteers also have contact with the HF during the monthly community outreach activities (Strategy Avance) which were supported by the project for a six-month period of time in order to encourage HFs to take this up on a routine basis. The community agents and Oversight Committees notify community members about the outreach activity and they help to identify children for vaccination, women who need pre and postnatal care services, and clients wanting FP services for that day (mainly depo provera injectables). CS-20 Mali, Ségou, October 2007 30 Save the Children iv. Strengthening Health Worker Performance • Approach to strengthening health worker performance Please refer to above discussion under HF strengthening. • Effectiveness of approach The various approaches used to strengthening health worker performance have shown some results (as discussed above). This has included training, supervision and monitoring, and ensuring regular supplies. It is unfortunate that the project was not able to complete the PDQ process because this clearly would have contributed to health provider learning and increased collaboration and sensitivity towards community/client needs. Although the improvements are notable, per staff and district MOH reports during the MTE, more progress is still needed. The service provider (HFA) survey also shows that there are areas that still need improvement, i.e. eligibility criteria, discussed below. Providers still have some of their old biases related to method eligibility for clients. In the HFA, 71% of providers interviewed said that they would require that a woman had at least four children before agreeing to provide her with Depo Provera; almost 8% said that they would not provide a non-married woman with the oral pill, and 16% wouldn’t provide her with condoms. With regard to client visits for FP, a disappointing 58% said that if a client wanted a hormonal method but did not have her period at the time of the visit to the HF, they would tell her to come back when she has her period. In a situation where client using an oral contraceptive presents with either HIV or an STI, almost half (48%) of the providers said that they would switch the individual from the oral pill to condoms. • Performance assessment tools Supervisory checklists have been developed by the project. Per the district supervisors, these are appropriate and effective for measuring change that occurred during project activities. • Use of assessment results to improve the quality of services Results from both regular supervision (quarterly by the districts) and the more complex monitoring assessments (bi-annual) are discussed and recommendations for improvement have been made for the HFs. Project staff is involved in these activities and help to provide follow up on this. One HF visited during the MTE for example, mentioned that they had increased the frequency of FP education sessions as a result of the recommendation that came out of a supervision visit. • Addressing gaps between performance standards and actual performance To help address gaps in performance, the project planned and carried out the following: Developed Standards of training aligned with MOH norms and procedures; Procured and distributed revised FP/RH policies, norms and procedures for all health providers; Trained health providers in the use of recently revised FP/RH policies, norms and procedures; Developed and implemented the use of referral protocols for FP services; Set performance standards for all providers including CBD agents CS-20 Mali, Ségou, October 2007 31 Save the Children and PEs; Put in place an integrated supervision system, including the development of a supervision guide, and trained supervisors in its use; and provided on-the-job training during supervisory visits. In addition to this, the project provided both the regional and district MOH offices with financial support to be able to carry out these activities during the last year. v. Training • Training strategy and its effectiveness. Project training started with a Training of Trainers (TOT) and subsequently targeted the training of all NGO, MOH and community partners and key players. The table below highlights training activities that took place during Phase I of the project. Table 5. Training Activities Training Participants Number Trained Training Content Regional Health team; Macina and Baroueli District Health teams; Other district and Project coordinators and supervisors. 16 Adult Education IEC/BCC FP Contraceptive Logistics and Data/Report Management PAs and Health Center In-Charge 58 Adult Education IEC/BCC Anatomy/physiology Contraceptive Methods Contraceptive Logistics and Data/Report Management Health Center Depot Managers (including DRC) 34 Contraceptive Logistics (including ensuring adequate supplies for community-based volunteers based on utilization rates) Private pharmacists 10 Contraceptive Logistics Health Center Midwives 29 IEC/BCC Anatomy/physiology Contraceptive Methods Community Based Distributors and PEs 236 IEC/BCC Anatomy/physiology Contraceptive Methods Contraceptive Logistics and Data/Report Management PAs and Health Center In-Charge 32 Stages of Partner Defined Quality process and applicability in the project Health Center Committees (ASACOS) 59 members Management of the Health Center Micro-Planning (quarterly) CS-20 Mali, Ségou, October 2007 32 Save the Children The number of individuals trained tended to be more than what was planned in the DIP for Phase I: 29 midwives trained verses 15 in DIP; 236 CBDs and PEs verses 208 in DIP; but only 58 ASACO members were trained verses 124 in DIP (trained two rather than three members per ASACO). • Progress made towards objectives The project set out to train all health providers working in the two Phase I target districts, as well as the district and regional teams, depot managers, private sector pharmacists and community-based volunteers. They were able to meet all of these training objectives. • Evidence that training resulted in new ways of doing things/increased knowledge and skills As mentioned under the HF improvement section, district supervisors interviewed noted improvements in several areas including. counseling and dispensing of contraceptives to clients. Health facilities also report increased FP education sessions during the week, and increased outreach activities where there has been a systematic effort to include FP. Unlike before when this service was not a part of the Strategy Avance, 35% of providers are now providing Depo Provera injectables during outreach. During the MTE women were also asked if they had chosen the FP method that they are using. In all cases, women said that they had been the ones who chose the method; it had not been chosen for them. This is a good indication that providers are respecting the clients’ ability to choose a method themselves. d. Sustainability Strategy i. Progress on sustainability objectives and monitoring indicators The sustainability objectives in the DIP revolved around sustaining program interventions, i.e. behavior change, quality of care and contraceptive supply. ƒ Strengthening the management skills of ASACOs: This was accomplished and they are maintaining their books and are able to maintain the costs of running the HFs. ƒ Enabling Community Agents to purchase contraceptive products at a fee that makes it possible for them to make a profit: This was done. They are purchasing at the same price as the HFs. ƒ Ensuring that HF staff continue to supervise community volunteers on a regular basis: The plan is for this to become a part of the Strategy Avance outreach visit. All HF staff interviewed discussed this. One good indication is that outreach services have become part of their routine. ƒ Strengthening the contraceptive supply system through training logistics management: This was done. ƒ Sustained use of contraceptive methods so that drug depot managers are not tempted to reduce orders in order to prevent the accumulation of expired products: The project provided training in contraceptive logistics and included forecasting to enable them to regulate their supply. CS-20 Mali, Ségou, October 2007 33 Save the Children ƒ Advocacy efforts with influential members such as religious and community leaders and men, and IEC activities at community and facility levels to create a sustainable and supportive environment for FP: This has been done quite successfully so far. The project will be strengthening activities which target men in order to continue to reduce that barrier to FP. During the MTE, the project collected data on the related IR4 indicators, and level of achievement. These include: supervisory visits by district implemented verses planned (55%); Supervision of community agents by ASACOs (35%); communities with effective Oversight Committees (30%), and female membership in these committees (30%). ii. Groundwork for the phase-out strategy From the beginning of the project, the district partners were informed that project support would be for a two-year period before the scale-up to two new districts. In the months preceding the MTE, PAs were scheduled to conduct meetings with the HF and community partners to prepare for Phase out. This included ensuring that everyone understood how the activities were going to continue without the presence of the Animator who had been facilitating a great deal such as the provision of contraceptive supplies to CBDs, supervision, and collection of community data. Per MTE interviews, it seems that the phase-out preparation activities were not as rigorously and consistently implemented as planned. A few Animators seemed to have been scheduling meetings for the last minute and then got pulled out a few weeks earlier than they expected. Nevertheless, all providers interviewed said that they were ready to continue with what the project had put in place. They were committed to supervising CBDs during the outreach activities and continuing to ensure that enough contraceptive supplies would be available. iii. Approaches to build financial sustainability The districts have determined that removing all fees for FP services really helps promote the use of these services, and has been especially effective in the district of Barouéli where all fees for FP were eliminated. Clients do have to pay a small fee for contraceptive supplies though; the same fee at the HF as at the CBD level. The small profit that CBDs are able to make from the sale of these are in fact very minimal but does allow him/her to earn a small salary for their effort. Unfortunately, this is reportedly not enough to enssure volunteer motivation. iv. Community sustaining project services The project was careful not to provide services that would subsequently require alternative funding to be continued. Animators did support community mobilization, education and the supervision of CBDs, but a lot of their effort was focused on the technical capacity building of the community volunteers. As a result, activities can continue once these community agents are trained and well motivated. Even though there will be a reduction in the number of education sessions taking place in Phase I districts after the project ends community demand for FP services is well established. This is good motivation for the CBDs to continue renewing their stock and providing this service. In addition to the above, CBDs have also been recruited to work as CHWs for child survival through UNICEF-funded support and recently received bicycles, which is a nice incentive. The regional and district offices do emphasize the integration of health services, so these bicycles are expected to help the volunteer with every aspect of his work, including travel to the HF for meetings, to submit reports and to pick up new supplies. Community-based distribution will likely include such things as oral rehydration solution (ORS) packets and other Community-based CS-20 Mali, Ségou, October 2007 34 Save the Children Integration of Childhood Illnesses (C-IMCI) related provisions that can be sold for profit in the future. v. Progress/Evolution on sustainability plans The project did not use a formal sustainability methodology. Please refer to the Capacity Building section and above sections for discussion on partners and evolution of strategy. One insight gained through the experience in this first phase of the project was just how much district leadership can determine the success of a project. This was evident in the district of Baroueli where the initiative and the strong involvement of the District Medical Officer in Charge advanced the FP agenda. This leadership influenced the HF and community levels, including advocacy and service provision whereby results are notably better than those in the district of Macina. This serves as an example of the strategic advantage gained by having a strong MOH leadership alliance and buy-in from the beginning. This will be useful for the project in the upcoming scale up in the second half. At the end of the MTE , which benefitted from the active participation of the region and two districts, a sustainability plan for the two districts was discussed based on the MTE recommendations. Table 6. Sustainability Strategy Sustainability Strategy for Phase I Districts (Barouéli and Macina) MTE Recommendation Action to be taken Remove FP fees in all HFs of Macina. Negotiate removal of FP service fees with the HF Committees (ASACO). Ensure follow-up of contraceptive logistics at every level. Put in place a monitoring system for the management of contraceptives. Apply PEPS norms. Strengthen the supervision of community agents by the HF teams during community outreach activities. Strengthen the relationship between staff in the HF. Strengthen supervision at every level (Region, District, HF) Make the standardized tools available to the regions and districts. Continue advocating for financial support for supervision activities. Take advantage of all opportunities, i.e. other activities, to undertake supervision. In charge (CPM) monitoring of monthly data from community agents (sale of contraceptives, BCC activities). Put in place a tool for data collection of CBD data during the community outreach conducted by the HF. The districts and HFs need to involve religious leaders in national FP campaign activities. Target influential leaders who are in favor of FP for campaign activities. 3. Family Planning a. Contraceptive supplies There has been an issue with the stock-out of certain contraceptives both the at the HF and community levels, primarily with the foam tablets and progesterone-only oral pills. According to CS-20 Mali, Ségou, October 2007 35 Save the Children reports from the MOH during the MTE, stock-out at this time is at the national level (Directorate for Pharmacy and Drugs or DPM), resulting in stock-outs at all the other levels. Project staff has already participated in discussions around this issue, but say that it is out of their hands (although it should be noted that the Group Pivot leadership believed that the project could perhaps advocate for a solution with the support of USAID and UNFPA). In the past though, at times when there were complaints of no stocks in the region, the national levels were saying that there was not a problem. Project partners and staff say that there is clearly a communication problem between the regional Pharmacy of Mali (PPM) which provides drugs to the region and districts, and the national PPM which is a part of the DPM. Everyone recognizes that this is something that needs to be strengthened. b. Knowledge and interest in family planning Knowledge and interest for FP have definitely increased as a result of the project. Knowledge increase can be noted in the KPC results (from 84% at baseline to 97% at the MTE). Field staff report a huge interest in FP stimulated by the project and demonstrated by the number of people, both men and women who come up to them with questions. As mentioned before, the project activities for IEC/BCC included advocacy with the religious leaders whereby they have given messages of support after sermons. IEC/BCC included the use of radio programming and community events for FP information dissemination; education sessions with women and men at both the HFs and the communities; and also counseling and FP services. It is important to note that a national FP campaign implemented yearly has also contributed to increased knowledge and interest in FP. Also mentioned before is that identified barriers are being reduced as people are talking openly about FP and project education sessions are not being blocked. Some men are even escorting their wives for services. The advocacy with religious leaders was a big step in this direction and the sensitization of village leadership about the importance of maternal health has also helped. Evidence of increased use of FP is seen in the results section (from 8.6% at baseline to 10.6% at the mid￾term), and it is important to note that contraceptive supplies have been available at the community level for less than a year. Prior to the project, there were virtually no supplies to be found even at the HF level. c. Quality of family planning services Please refer to previous discussions under HF strengthening and health provider performance. d. Access to family planning services CS-20’s community-based service strategy dramatically increased access to FP services. This is particularly critical because target communities were defined as those located at 5 km and further from the HF. Time taken for travel, cost of transportation and rains making roads impassible are all barriers to use of services. As such, this strategy really succeeded in increasing access. The additional provision of depo provera during community outreach activities also helps. e. Range of methods, client choice and method mix The project has succeeded in increasing the range of methods available in the region. At the start up, many HFs did not even have a stock of contraceptives, and some had expired contraceptives. With the assistance of the DELIVER Project, CS-20 was able to bring in vaginal foam tablets, condoms and progesterone-only oral pills which were not available before. Although there is a problem with contraceptive stocks right now, the system that has been put in place with the support of DELIVER, should ensure that the range of methods available stays constant. CS-20 Mali, Ségou, October 2007 36 Save the Children f. Compliance with prohibitions and restrictions All project partners have signed documents attesting to compliance related to USAID population funds, Mexico City Policy and the Tiahrt Amendment. g. OR/Case study As mentioned before, the project undertook a study to compare the cost-effectiveness of the two different strategies which will certainly be used to inform the PVO community and those implementing similar activities. Please refer to the attached draft report in Annex G. C. Project Management 1. Planning a. Groups involved in project planning Group Pivot and the MOH partners were involved in all initial proposal design and in the DIP workshops conducted by the project at the start up. As the local NGO partners, ASDAP and AMPROD-SAHEL/JIGI came on board a few months later and were not involved in this initial planning. Needless to say, as implementing partners, these two NGOs did participate in orientation and planning workshops once they were selected to participate in the project. Both partner NGOs participated in the DIP development workshops. b. DIP work plan and schedule The project has done a good job of staying on track in terms of the work plan submitted in the DIP, except for the very long delay in the procurement of contraceptives for the start-up of CBD activities. As mentioned before, they were depending on the DELIVER Project for this component. Issues of poor financial reporting on the part of the two local NGOs apparently resulted in a few delays in the disbursement of funds on a regular basis. One of the main things affected by this in terms of program activities was that the field team did not always have enough money to purchase fuel to be able to carry out their activities within the program area. As a result, they would frequently incur small delays in conducting routine activities. c. Understanding of project’s objectives All staff, MOH and community partners interviewed said that they were aware of and understood the project’s objectives. d. Copy of objectives, monitoring and evaluation plan All partners at the regional and district levels, and NGO partners as well reported that they have copies of the objectives and monitoring and evaluation plan. e. Use of monitoring data for planning Project monitoring data is reviewed every month starting at the HF level, districts, regions and Bamako staff in the various NGOs as well. Data is discussed monthly and quarterly planning is an opportunity used by the project to adjust activities based on the routine data collected. CS-20 Mali, Ségou, October 2007 37 Save the Children 2. Staff Training a. Process for continual improvement of project staff Although the project had initially planned on conducting refresher training for field staff, this was not done due to limited funds, however since they had received training within two years of the midterm (close out of Phase I), this was not necessarily something that needed to be done (although one local NGO was able to undertake this activity). Project Coordinators and supervisors provided on-the-job training, follow-up and support to new Animators who replaced ones that left the project. This was also the case with new MOH staff replacements. It should also be noted that the project and district system for monitoring (bi-annually) supervision and support is designed to encourage continual improvement in knowledge, skills and competencies of staff. This has been relatively effective although there is a need to increase efforts in the area of quality of care to support improved performance and reduce provider biases. (Please refer to previous discussion.). As mentioned before, the project was not able to complete the PDQ process as they were trying to do it on a larger scale and project resources were unable to support that. Doing this in a sample health zone in each of the new districts may be a way to be able to gather qualitative information that will be useful for quality improvement purposes district-wide. b. Monitoring of trainee performance Trainee performance in new skill areas is monitored through supervision visits. Tools for data collection and monitoring were developed by the project. PAs received an orientation on the use of these tools. Support for monitoring has been provided by project coordinators and supervisors in addition to being a routine activity for the district team. Supervision checklists developed by the project assisted with this task. c. Resources dedicated to staff training Resources dedicated to staff training are limited. This is compounded by the fact that this project involves a lot of different partners. All staff interviewed nevertheless felt that they had learned a lot from the project training and technical support provided, and most of the staff hired by the project already had some experience working in FP and community health. 3. Supervision of Project Staff a. System for effective staff performance Staff is located at three primary levels: Bamako has the senior project staff from Save the Children and partner Group Pivot (staff seconded from the latter work out of the Save the Children office); and senior staff from ASDAP and AMPROD-SAHEL/JIGI. The two districts have the two respective Project Coordinators with their team of two Supervisors. The Health Facilities across each of the two districts have an Animator based on site. The initial plan was for the Save the Children and Group Pivot project staff to be based at the regional level in Ségou so that they could cover the work being done in the districts. This office had to be closed soon after project start-up due to the realization that the project did not have adequate funds to cover this expense. This entire staff was relocated to the Bamako office, which has allowed for increased project staff participation at the national level. However, as a result, support for Project Coordinators and their staff in the field is now provided “long distance”. It was expected CS-20 Mali, Ségou, October 2007 38 Save the Children that this staff would spend a lot of time in the districts providing technical assistance and support, and would work with the regional and district MOH, for various reasons it is not always feasible for staff to travel there monthly due to travel costs and their work at the national level on such things as adapting training materials, adapting IEC/BCC materials, and liaising and working with the different partners (DELIVER, POLICY Projects etc). It is also important to note that although the project staff in Bamako comprise the project leadership and provide all technical guidance for activities, they do not have management/administrative responsibility over the NGO project teams implementing activities in the districts. Thus the process of directing and supporting staff for effective job performance sometimes has to go through the NGO Bamako level first. The distance, along with the latter, do have an impact on communication between the different project team members and effectiveness and timeliness of supervision, guidance and technical support. The project has strong leadership and dedicated project coordinators (although one of the coordinators resigned a few months before the MTE). Supervision at these levels includes meetings and debriefing which often focus on indicators and quarterly plans. For supervision at the Animator/HF and community levels, supervisory checklists are used in addition, and NGO managers sometimes quiz their staff on their technical knowledge. Relationships between project team members from the different levels is reportedly very good and collaborative. On-the-job training in this project is part of the project strategy for local NGO strengthening and mentoring. Field staff report that supervisory visits are very useful and have helped them increase their capacity in FP programming. Work planning is adequate as it is undertaken every three months. Monthly meetings at the field level serve to review and refine those plans based on data and issues. Problem￾solving is a big part of supervision visits and takes place at the field level with the district partners, and in consultation with the Bamako staff. b. Appropriateness of personnel and supervisory visits The numbers, roles and workload of personnel appear to be appropriate except for the fact that in the division and allocation of Animator work by health zones (each Animator was designated to one health zone), some Animators ended up with a much bigger catchment area than others. This is because some health zones have as few as three villages, while others have three or four times as many. One interviewee mentioned that the project should also think about allocating more difficult terrain to the male Animators rather than the female Animators because they tend to be stronger and are more skilled with the motorcycles. These issues will be carefully reviewed in Phase II. The frequency of supervisory visits was less often than planned especially for the Bamako staff as they were no longer based in the region, and from at least one of the two local NGOs. Supervisory visits from the local NGO managers are supposed to take place once every quarter (they spend 7-10 days in the field during these visits). Save the Children and GP project staff, on the other hand, was supposed to have quite a constant presence in the field (per the DIP, 50% of Project Advisor’s time and 70% of training specialist’s time), however as per staff interviews, this was not always the case. After the training of partners at the HF and community levels, for example, it would have been helpful if the project Training Specialist could have spent more time in the field ensuring that trainees were adequately prepared and able to carry out new tasks related to the enhanced FP services. Logistics have been mentioned as an area of challenge for the project, and more effort needs to be made in order to maximize the input of the technical project staff, who are not spending the designated time in the field. One technical staff person should not be tied to another’s schedule CS-20 Mali, Ségou, October 2007 39 Save the Children either; for example if the project manager needs to be in Bamako for meetings and planning that the training specialist does not need to attend, the training specialist should not be ‘stuck’ in Bamako when his time would be much better spent in the field supporting staff and partners. 4. Human Resources and Staff Management a. Management system As this is a partnership project, each organization manages its own personnel. Save the Children has a Country Director and Deputy for Administration and Finance. Each provides approximately 10% of their time to CS-20. A Health Coordinator gives direct support to all health programs in country (25% level of effort to this project). There have been no changes to this system since the start up of the project, and all positions are currently filled. b. Personnel policies and procedures Key personnel policies and procedures are in place within all of the partner organizations. All project personnel report having a job description. c. Morale, cohesion of project personnel As mentioned before, the working relationship between the different project team members, across the four organizations is very good. Both of the local NGO leaders in Bamako said that the team was like a family. Unfortunately, a couple of things did affect field staff morale including the regular tardiness of salaries due to poor financial reporting from NGOs resulting in delays in the quarterly budget disbursement to the NGOs, and the progressively problematic condition of the motorcycles purchased as part of the Save the Children match for this project. The latter did prove to be a problem because per staff interviews, by year two of the project they were never sure when they might break down on the way to an activity, and not be able to carry it out, or be stuck spending the night on the road. As it turns out, the match commitment made during the design of CS-20 Ségou and CS-20 Sikasso proved to be an unnecessarily big burden on Save the Children Mali logistically when both proposals were funded. Project vehicles are expensive items (approximately US $45,000 for each of the two vehicles purchased for the two CS-20 programs), and the added commitment to purchase motorcycles for field staff covering both projects (at least 35 for Ségou and an unknown number for Sikasso) was in fact very difficult to implement. As a result, fund raising efforts only allowed for the purchase of motorcycles of poor quality and unfortunately not the most durable or suitable for the difficult terrain of the Ségou project. At this point, the motorcycles are in very bad condition and this will pose a problem for the upcoming Phase II of the project which is located in districts of Ségou that are even further out than the previous ones. SC Senior management in Mali is working to resolve the issue that might include purchasing new motorcycles for Phase II. Although the working relationships between the different organizations is good, it should be noted also that the local NGOs (particularly the Consortium AMPRODE) would have liked to see more project support at the administrative and management level, both financially (cost-sharing rather than all efforts being considered match) and in the area of capacity building as well. In MTE interviews, they reported that they had not participated in the trainings that field staff received. Both CS-20 Mali, Ségou, October 2007 40 Save the Children local NGOs struggle at times with the match contributions (logistics) as vehicles are not in the best condition or are being shared with other projects. d. Staff turnover Save the Children did have a change in Project Advisor about six months prior to the MTE. Fortunately, the new Advisor was already an SC staff person and has had some involvement with the project since the beginning. The Health Coordinator position has had three different staff persons since the beginning of the project. Fortunately, the second person in that position did stay on for almost two years, and thus provided the project with some continuity in terms of technical support from the country office. The current Health Coordinator has only been in place for a few months and is still in the process of getting acquainted with the project reporting requirements and M&E. There has been no staff turnover with the Group Pivot secondees to the project. There was quite a high level of staff turnover between the two local NGOs. AMPROD￾SAHEL/JIGI lost four out of 15 field staff before the midterm; one of whom was the Project Coordinator. This did leave a void in leadership in the district of Barouéli as he was replaced by a Project Supervisor with less experience and capacity. ASDAP lost three staff out of 14 (one was pregnant when her first year contract ended so it was not renewed in view of the nature of the job), and one got a job with the MOH). Fortunately this does not appear to have had an impact on the work in Macina as the leadership is strong and dedicated. e. Staff transitions Although this is a midterm point in the project, this issue has come up because the project activities in the districts of Baroueli and Macina have come to a close. Thus PAs were in fact no longer employed and available during the MTE in June, as their contracts had ended in May. A few of the Animators may have already found other jobs, and ASDAP reported that they may be able to integrate one or two of their staff into other projects. Save the Children and district partners all agree that since this staff is already trained and experienced, it would be in everyone’s best interest for staff who performed well to be contracted again for the new districts. However, as was outlined in the DIP, the project will be conducting an organizational assessment with these two NGOs to assess progress and performance. This process will also be informed by the various studies done for the midterm evaluation (KPC, HFA, CBD) in the two target districts. Because of the financial and management oversight level of effort, and increased financial resources needed to work with two local NGOs, it may make more sense for the project to limit itself to only one implementing NGO covering the two new districts for the Phase II scale up. 5. Financial Management a. Management and accountability for project finances As mentioned before, Save the Children has a contract agreement with Group Pivot. Group Pivot, in turn, has sub-contract agreements with ASDAP and the Consortium AMPROD-SAHEL/JIGI. Each of the local organizations involved in the project prepares and submits a quarterly budget which is consolidated by GP, and then submits a request to Save the Children. Thus financial reporting requires that reports from all three institutions be ready for one submission (including the appropriate receipts and documentation for expenses incurred), before they are reviewed by Save the Children, accepted and reimbursement for approved expenses made. Any delay on the part of one of the three organizations, delays the submission of the report and affects all three organizations. CS-20 Mali, Ségou, October 2007 41 Save the Children Save the Children administrative and financial staff is doing its best to process these reports as soon as they are received (although one NGO said that staff turnover in SC affected the smoothness of the transactions and delays in the reconciliation of financial reports). In addition, the grantee disbursement system and policy requires that they only provide a second disbursement based on disbursements accounted for in the previous financial report, rather than what may be requested for upcoming activities. This poses a problem because it is not always possible to provide receipts accounting for the full quarterly disbursement given if certain Animators have had delays and do not have receipts to submit because they have not yet completed the designated activities. Unfortunately, such a situation disrupts the full funding for upcoming quarterly activities and was a common occurrence during Phase I. The project Accountant (GP secondee to Save the Children), and Grants Manager (in place for the first year of the project) gave the grantees an orientation on Save the Children financial procedures and documentation requirements. The Accountant has also worked with the field staff on these issues so that they would be better able to provide the correct financial documentation and reduce delays. Unfortunately delays in reimbursement due to other reasons (such as the prior example) are not always things that are under the control of the field staff as they are working in collaboration with MOH and community partners. The local NGO partners are not in a position to continue activities in the two Phase I districts unless they get support from another donor. Although ASDAP has been able to secure funding from World Learning, this is for another project in another geographical area of the country. At the moment, there are no plans for activities to continue in Macina and Barouéli outside of the MOH and the ASACOs. The ASACOs have, for the most part, already been able to financially support outreach activities implemented by health providers (usually the Matron and a vaccinator). The project gave them support in the way of a fuel budget to be able to initiate these activities for a six￾month period. Many of the ASACOs used this period as a savings opportunity and are in a better position to support these outreach activities now. Others will be trying to include this expense into their budgets; many of the HFs were not previously undertaking outreach activities). Regional and District teams who need to continue undertaking FP supervision (quarterly) and monitoring (bi-annual) activities expect to be able to integrate them into supervision and monitoring activities that are supported by UNICEF. During interviews with both levels, the MOH was very candid about the fact that when they do not have donor support for supervision and monitoring activities, their own limited budgets do not enable them to undertake those activities. As the region is working with UNICEF to support child survival, they plan to integrate FP into this program and continue to sustain the Ségou CS-20 achievements. 6. Logistics a. Impact of logistics Please refer to the above discussion on challenges related to project motorcycles, and location of senior technical staff in Bamabko, resulting in reduced levels of technical and supervisory support envisaged in the DIP. The big disappointment during this phase of the project was the year-long delay in the procurement of contraceptive commodities for the newly trained community-based distributors. This has CS-20 Mali, Ségou, October 2007 42 Save the Children compromised the CYP and use of FP results seen at the midterm as the implementation period was only one year, rather than two. b. Logistics challenges project will facing The logistics challenges that the project will continue to face include not having motorcycles in good condition for project implementation in the new districts. In an ideal situation, the project would have purchased suitable motorcycles with a project life of about four years in these areas. Since Save the Children/Mali does not expect to be able to mobilize new resources to purchase new motorcycles at this time (only two years after having purchased the first set), it will be necessary for them to look at making a budget amendment and getting authorization from USAID in order to rectify this situation. Without this, it will be almost impossible to carry out scale-up activities during Phase II. It is important to note that Save the Children has already contributed above the match agreement on this project. The Mali Country leadership reports having contributed 33% of the project budget during this past fiscal year. 7. Information Management a. System to measure progress towards project objectives Save the Children/Mali has an M&E person who assisted the project with the development of a database. This enables staff to enter the data obtained in the NGO quarterly reports and monitor progress towards achieving program HIS-based objectives. Most of the impact objectives are measured through the KPC survey - conducted at the midterm by an external institution hired by the project. This institution also conducted the HFA and the survey with the CBDs and PEs. These MTE studies serve as final data for the districts of Baroueli and Macina. The system is effective in measuring progress towards project objectives. As previously mentioned, staff has used data to identify problem areas and to make decisions on areas of effort needed. The MTE provided a great opportunity to have a detailed review of project accomplishments and challenges, and this data was used for decision-making and action planning for the second half of the project. b. Collecting, reporting and use of data Data coming from the community levels is collected by the PAs. All CHW data is kept in small exercise books which are easy to replace. CHWs keep information on client evaluation and diagnosis, use of contraceptives and re-supply of contraceptives, monthly activities including management of contraceptive supply, BCC activities, follow-up, number of clients having received three provisions or more of contraceptives during the year, and referrals. At the HF level, they also keep similar data including inventory managed by the Drug Depot Manager. All data is then analyzed by the PAs and the HF In-charge that they work with. The In-charge and Animators compile this data and it is sent to the project Supervisors. Supervisors analyze and compile data coming from all their health zones, and this is submitted to the project Coordinators. The MOH district health teams also receive data from the HFs and this is all reviewed and discussed during monthly meetings where they make decisions based on progress or issues identified. Project staff coming from Bamako (from the NGOs, Group Pivot and SC), also review this data during supervision visits and when quarterly reports are submitted by the NGOs. The regional MOH office in Ségou also receives monthly reports from the district and quarterly meetings with all the district representatives are an opportunity to discuss and analyze progress and issues. CS-20 Mali, Ségou, October 2007 43 Save the Children c. Data and method(s) of data analysis Please refer to the previous discussions on data generated at the community and HF levels, and analysis of this data. d. Use and support of existing systems The project worked with the MOH to adapt and expand existing data collection tools for FP. These are now being used by MOH staff at the HF level. Tools for CBDs were also adapted from previous projects and will continue to be used by the MOH in the old and new districts. During the MTE, the MOH Division of Reproductive Health in Bamako scheduled a meeting with FP partners in-country to discuss and share tools and work together to standardize what is being used in different projects. e. Use of data for management decision-making Data and issues are discussed and decisions are made based on this data both in the field (during supervision) and during the quarterly project partner meetings in Bamako. f. Assessments The project undertook one important study on the cost effectiveness of combining the community￾based strategy with HF based strategy (discussed previously). In addition, assessments of the local partner NGOs were also conducted. 8. Technical and Administrative Support a. External technical assistance As previously mentioned, the project staff were able to benefit from a PDME workshop which was conducted in Mali right at the beginning of this project. The project was able to get external assistance from two major players: The POLICY project in collaboration with the High Islamic Council were able to collaborate with the project and give them technical assistance with the advocacy activity targeting religious leaders; and the DELIVER Project provided the project with assistance for the training and capacity-building of staff and partners in contraceptive logistics management. In addition, DELIVER was able to provide an initial supply of contraceptives to the two districts and the target communities (albeit much later than planned). The project also received assistance from a GP/SP staff person attached to the CARE Kenya Ciwara Project for the training of ASACOs. Other than the late supply of contraceptives, this external assistance was invaluable to the project. b. Anticipated technical assistance needs In the upcoming second phase of the project, project partners will be taking the lessons learned from Phase I to scale up. As all partners have already received training, it is not anticipated that there will be much need for technical assistance outside of regular support and visits from the SC home office backstop. c. Grantee headquarters and regional support The FP/RH Advisor based in Save the Children’s Washington office (Dr. Winifred Mwebesa) is the technical backstop person for this project. Since the start-up of this project, she has made a trip to Mali every year, helped with the elaboration of the DIP, provides technical guidance and monitors activities via e-mail and telephone communications, reviews quarterly reports, and assists with the preparation and translation of annual reports. Eric Swedberg is the Child Survival Advisor in Westport who liaises with USAID. He assisted with the elaboration of the project DIP and manages CS-20 Mali, Ségou, October 2007 44 Save the Children correspondence with BHR/PVC. Both participate in CORE activities and working groups that contribute to their capacity to provide technical support to the project. 9. Mission Collaboration The project has an excellent collaboration with the USAID Mission. The mission’s support of FP is evident by the fact that they are supporting the repositioning of FP, and encouraging and supporting country partners in the implementation of their strategies. CS-20 Ségou responds and contributes to the Mission’s overall health objectives. As mentioned earlier in the report, Save the Children staff participate in the FP working group at the national level, working with both the MOH and the mission bilateral programs. There has also been sharing and collaboration with the mission bilateral project CARE Keneya Ciwara, as Group Pivot is also a partner under that project. Meetings between all FP partners in country are regular and the level of coordination is good. Interactions and sharing with the USAID Mission are regular. D. Other Issues Identified by the Team None. E. Conclusions and Recommendations The project has made a lot of headway with respect to results, i.e. use of FP, community knowledge, the establishment of systems, and capacity building. The capacity-building achievements of the project include the revitalization of FP in the target districts, and these services have become a part of routine MCH activities at the HFs, as well as at the community level through community outreach activities and the establishment of CBD agents in communities located a fair distance from an HF. Prospects for sustainability are good as there has been a high level of buy-in from the MOH district partners. FP will be integrated in other supervision and monitoring activities which are currently supported by UNICEF in project districts. Regional, district and HF level Drug Depot Managers have been trained on contraceptive logistics and the demand for contraceptives at these and community levels is expected to exert pressure on the national level to assure contraceptive security. (Please refer to Annex J for a copy of Phase II Action Plan reflecting sustainability in Phase I districts and how Save the Children will address MTE recommendations for Phase II districts.) The main constraints include contraceptive security as the government has not been able to assure the constant supply of contraceptives. The mobility of the population and their absence during many months of the year makes it challenging to ensure that clients are continuing to use modern methods, and access to communities from HFs during the rainy season is also difficult. To ensure impact in the second phase of the project, Save the Children needs to consider modifying their budget (possibly reducing to one implementing NGO) based on the organizational assessment and MTE results. This will allow for easier management and technical oversight of activities and logistical support, and is due to the limited program budget and ambitious program results. CS-20 Mali, Ségou, October 2007 45 Save the Children Key Achievements 1. Involvement, buy-in and collaboration of the Ségou Region, the target districts and the HFs in the FP project intervention area. 2. Revitalization of FP in the region of Ségou and inclusion of FP in the minimum packet of services. 3. Strengthening of FP services with the training of 29, 29, 34 service providers (In Charge, Matron and Drug Depot Manager, respectively) in the project area. 4. Training of 59 ASACO members in the project area on micro-planning and, contraceptive logistics and management. 5. Training and provision of contraceptives to 234 community agents in 198 villages. 6. The training and integration of 10 private pharmacists in the health information system and establishment of procurement scheme for them through HFs. 7. Strengthening supervision and monitoring for FP, including the use of data for decision making. 8. Mobilization of Muslim religious leaders for FP, establishment of Islamic Committees and clarification of the Koran’s position of FP through advocacy. 9. Initiation of routine education and communication activities on FP at the HF and community levels (including making available FP IEC materials available). 10. Removal of taboo around the subject of FP. 11. Participation and visibility of the project at the national level. 12. Development of FP training manuals. 13. Implementation of operations research on cost effectiveness of community/HF strategy compared to HF strengthening alone. Challenges 1. Delay in the delivery of contraceptives for CBDs and HFs for approximately one year post training. 2. Periodic stock-out of contraceptives (vaginal foam and progesterone only pill) at all levels, including nationally. 3. Lack of resources to complete a proper PDQ process across project area. 4. Men at the household level continue to be a barrier for FP. 5. Availability of contraceptives through mobile salesmen with no training and unmonitored products. 6. Low education and literacy levels of women and the population in general, is a barrier to the use of FP. 7. Low literacy levels of community agents makes capacity building a challenge. 8. Migration of population every year makes it difficult to conduct IEC/BCC activities and follow up on FP clients. Implementing IEC/BCC activities is also difficult when population is present because their days are occupied with planting and harvesting. 9. The terrain is difficult but especially so during the rainy season where some routes become impossible to traverse. During this season, most of the community outreach activities are suspended. 10. The margin of profit from the sale of contraceptives is minimal (40 -60 CFA or between 8 and 11 cents per product). This, by itself, is not adequate to serve as CBD motivation. CS-20 Mali, Ségou, October 2007 46 Save the Children 11. Per the HFA results, traditional attitudes of providers and lack of knowledge on medical criteria for FP contraceptives clients constitute a barrier to FP. 12. The fact that the SC and GP project was based in Bamako (rather than Ségou) did not allow for the close monitoring and support of project implementation, i.e. giving timely guidance to the local NGO field staff for new activities and communication with the MOH regional and district partners. 13. Poor financial management on the part of the local partner NGOs caused regular delays in the implementation of certain activities and payment of field staff salaries. 14. All project partners found that their budget was very tight and it was a challenge to implement and manage their responsibilities under the project. 15. Matching fund contribution commitments for each of the project partners (SC and others) ended up being a big burden on those offices. The terrain is challenging and it was difficult to put the entire cost of logistics as match (two vehicles and over 40 motorcycles for two CS projects funded simultaneously). As a result, SC was forced to purchase cheaper motorcycles that are no longer able to support Phase II activities. 16. Because of the delay in the arrival of contraceptive supplies, the project CYP at the midterm was only for a one year implementation period. This was a disadvantage when calculating the cost￾effectiveness of the program. 17. It is still early for project phase out in the first two districts. It would have been ideal for the project to have had at least another six-month phase out period to implement hand-over to HFs and CBDs-HF supervisory, reporting and procurement systems. Lessons Learned 1. When looking at issues of sustainability, the cost effectiveness study undertaken by the project has shown that having CBD agents in communities combined with the HF FP services, produces results that make it cost-effective. 2. The project experience with advocacy among religious leaders demonstrated that religion is not a barrier to FP, and the use of these leaders has contributed to developing an enabling environment for FP. 3. The removal of service fees for FP has greatly encouraged the use of these services. 4. The use of good radio programming stimulated the population’s interest in FP demonstrated by the questions and discussions directed to program staff after these events. 5. The national campaigns for FP encouraged local campaigns and activities because communities felt that they were participating in a national activity. 6. Scheduled meetings (jours de concertation) between the district and the HF In-Charge were very important forums and supported collaboration and strengthening of the partnership. 7. It is important to take the community context into consideration when identifying community volunteers, i.e. many of the project populations are mobile, and most especially young people. Thus there has to be some flexibility with regard to selection criteria for community agents and PEs. 8. Considering time limitations when dealing with a relatively short term project, it is important to prioritize where efforts are best spent. For example, the effort needed to establish an FP committee in each village proved to be a challenge due to time limitation. It was more important to focus on the capacity building of community agents and the Health Facility Committee, which already included a representative from each village. CS-20 Mali, Ségou, October 2007 47 Save the Children 9. In order to maximize results in the area of capacity building, the project learned that it was important to use every opportunity and contact for continuing education of HF and community partners. 10. It is important to involve the HF In–Charge in all project planning events (not just at the district level). This helps to ensure buy-in, quality of services, the support of CBDs and sustainability. 11. It is important for all project logistics, equipment and contraceptives to be in place before the start-up of project activities. Without designated transportation it was difficult for project field staff to travel out to communities and begin the community mobilization process. Likewise, the one-year delay in contraceptives was detrimental because trained CBDs were not able to start their work immediately. 12. Project staff (Animators) should receive adequate training prior to their placement in health zones in order to ensure that they are comfortable with their job description and position, to be an asset to the HF and community, even when planning for them to participate in upcoming trainings with health workers. Recommendations 1. Modify the project in accordance with available resources and the needs identified at the MTE, i.e. consider cutting back from two to only one implementing NGO working across the two Phase II districts based on a follow-up organizational assessment of the two partner NGOs and the MTE results in the Phase I districts. Consider reducing the field staff to one Coordinator, an Assistant Coordinator, two supervisors and a maximum of 15 Animators to support the 36 health zones in the Phase II areas. Submit a request for budget modifications to USAID BHR/PVC to allow for the purchase of field staff motorcycles to ensure staff transportation across a large and difficult program intervention area. 2. Review the lessons learned from Phase I to accelerate project implementation in Phase II and avoid the different delays that the project experienced during Phase I. 3. Project managers should schedule debriefing meeting with NGO partner field and headquarter staff to share surveys, lessons learned and recommendations. 4. For Phase II, staff needs to revisit the PE strategy in view of the constraints encountered with this cadre during Phase I. 5. It would be useful for the project to explore using the RAPID model for advocacy with service providers in order to address traditional attitudes towards FP and eligibility criteria. 6. It would be useful to explore an adaptation of the PDQ process to match available resources, or select only one or two of the less popular HFs per district to undertake the full process and inform other health zone areas as well. 7. The project should support the systematic integration of FP into community outreach activities being undertaken by HFs to increase access to methods that are not provided by the CBDs, i.e. injectables, and provide counseling and adequate care to clients experiencing side effects. 8. The project should involve the HF In-Charge in the new Phase II areas in the project by organizing an orientation on the project, the role of the Animators that will be assigned to their areas, and involve them in all project planning including the choice of target villages for the establishment of CBD activities. 9. The project should continue to develop motivations for community volunteers, i.e. (congratulatory messages on the radio, assistance from the community in planting and harvesting of their gardens CS-20 Mali, Ségou, October 2007 48 Save the Children etc) and encourage model CBDs by offering them opportunities such as participation in meetings and training activities. 10. To achieve program objectives, the project should put more of its effort in areas that were not as successful, i.e. IR 2 and 3, rather than those that have shown very good results, i.e. IR 1. 11. Depending on the availability of funding, the project should allocate a small budget for national FP campaign activities every year in project health zones. This helps increase visibility and attention for FP, i.e. organize cultural events, soccer matches etc. 12. Technical project staff needs to dedicate more time and resources for supervision of all levels to: strengthen the capacity of field staff, HF staff and community volunteers; ensure the quality of work; and ensure adequate communication between the different levels. 13. In Phase II of the project, SC and GP staff needs to be involved in the decision making on target villages and in the designation of working areas for PAs so that there is a good balance of workload between PAs. Some health zones are three to four times larger than others. Thus the number of villages within a given health zone should be taken into consideration, and the work divided around number of villages to cover, rather than be health zone specific. 14. Project management should do their utmost to ensure the rapid start-up of activities in Phase II districts in view of the limited time left before the end of the project. This would include timely signing of agreement(s) with the local NGO(s) so that project staff can receive their salaries without a long delay, ensuring the provision of transportation and necessary materials for staff, and ensuring the availability of contraceptives. 15. Partner NGOs should submit timely financial reports per administrative requirements in order to reduce delays in the disbursement of funds, payment of field staff salaries, and the implementation of activities. GP should give closer oversight to the NGO(s) to ensure that this happens. 16. Save the Children/Mali should identify adequate support for the translation of project documents from French to English, so that the technical advisor in the Washington office can more appropriately focus her support to the project on technical issues. IR 1 : – Increased Access 1. Per the result of the cost-effectiveness study, extend the community strategy to all program areas fitting the criteria for CBD. 2. Encourage new project districts to remove service fees for FP. 3. Strengthen supervision and follow-up of CBDs during community outreach activities. 4. Use every opportunity/contact with community agents for continued capacity building. 5. Create opportunities for exchange between CBDs in the same health zone. 6. Integrate the Eligible Couple (ELCO) strategy into the training of CBD agents. 7. Develop an appropriate strategy with women’s groups/associations. 8. Re-visit the estimates on the initial stock of contraceptives given to CBDs. IR2: Improved Quality 1. Lactational Amenorrhea Method (LAM) should be integrated in the choice of methods promoted at the HF and community levels (included in IEC and HIS). 2. Develop a strategy to manage contraceptive stock-outs at the regional and district levels. 3. Develop a report format to monitor the quality of FP services. IR3: – Increased FP knowledge and interest of communities CS-20 Mali, Ségou, October 2007 49 Save the Children 1. Supervision at all levels needs to be regular and efficient to ensure quality of services. 2. Undertake advocacy with religious leaders at the village level. 3. Strengthen FP sensitization targeting men. 4. Adopt a couples counseling strategy at the household level to encourage couples communication. 5. Involve women religious leaders in the religious leader advocacy workshops. 6. Evaluate the feasibility of continuing with the radio program per availability of funds. IR4: Increased technical and institutional capacity of health services and communities 1. Create a forum for information exchange between the ASACOs and the HF staff (including the PA) to support decision-making and linkages with the CBDs. F. Results Highlight Advocacy with religious leaders: In partnership with Health Policy Project and the High Islamic Council (HIC), The Ségou FP project organized advocacy workshops targeting religious leaders from the districts of Macina and Baraouéli. The Resources for, and Analysis of the Population and its Impact on Development, “RAPID” model advocacy tools on FP, are geared towards improving the understanding of FP concepts and discussing issues of population and its impact on health, education and the economy. The workshop particularly focused on the Islamic position around FP. The workshops brought together approximately 192 Imams and Koranic schoolmasters of various Islamic denominations to discuss and show evidence through the Koranic verses that Islam is in fact not against FP. A key outcome of the workshop that was initiated by the group in Macina, was the establishment of a local Islamic committee for the support of FP. This was then created in both districts. This is a major achievement given the strong opposition to traditional Islamic values including the negative attitude and high level of resistance for FP use, and taboo around dialogue regarding FP. These committees are a communication channel for reaching religious leaders at the village level, as well as community members with a positive message about FP. A notable new activity undertaken by these committees has been the organization of sermons clarifying Islam’s view on FP. This has been done in many of the mosques in Macina and Baraouéli. Other activities that religious leaders have been involved in include broadcasts on local radio stations with the support of field staff, and sensitizing leaders known to oppose FP within the community. Advocacy with the religious leaders is considered to have been a success because as a result of their activities, these FP committees have made a major contribution to improving perceptions about FP in 29 Health Zones (15 in Baraouéli and 14 in Macina), with a population of 194,531 in Baraouéli and 213,027 in Macina. Before the advocacy workshop: Almamy TRAORE, Imam at the village of BERTHA, Macina We are religious leaders, Imams and representatives of the Islamic community in the district of Macina; for this reason we demand that this workshop be canceled/interrupted as we consider it useless to conduct it as we are all opposed to family planning. CS-20 Mali, Ségou, October 2007 50 Save the Children After the workshop Dr. Cheick Ahmed Tidiane Traore, Head Doctor CSRéf - Baraouéli: In 2005, during the first national FP campaign, we organized an advocacy meeting with religious leaders. During the debates, I was always weakened by participants who reminded me each time about the contents of the Koran that I do not master. At the end of the meeting, I noticed that participants had not changed their opinions and opposition to FP. Now that they have understood and that I have a committee that is empowered and is composed of eminent experts on the Koran, I am no longer apprehensive of having to approach religious leaders on FP. Almamy TRAORE, Imam at the village of BERTHA, Macina (member of the committee) The faith of a man is only confirmed when the happiness that one seeks for oneself, is sought for others too. From now on, I am convinced that FP is a way of reducing mortality and helping families to improve their welfare as the prophets did. Now, I am convinced and I adhere to your opinion. CS-20 Mali, Ségou, October 2007 51 Save the Children IV. ATTACHMENTS A. Baseline information from the DIP Program M&E Plan: Results, Indicators, Targets (key results also included in the report). Result # End of Program Target/ Indicator Method Baseline Number or Survey Denominator Baseline Percent Confidence Interval KR1 Couple-Years of Protection (per year) MOH SS Baseline N/A WRA: 489 8.2% 5.8%, 11.5% KR2 17%/24% of WRA (15- 49 years) use modern FP method Pop. Survey Adol 401 6.5% 4.2%, 10.0% 65%/70% of WRA report WRA: 39 59.0% 43.5%, 72.9% being a ‘new user’ of a modern method of FP Pop. Survey Adol: 26 65.4% 47.1%, 80.0% KR3 Number of users new to contraception (per year) MOH SS Baseline N/A WRA: 37 67.6% 51.9%, 80.1% KR4 60%/65% Continuation rates Pop. Survey Adol: 26 61.5% 40.2%, 79.2% WRA: 95 82.1% 71.5%, 89.4% KR7 Adequate birth spacing Pop. Survey Adol: 92 78.3% 67.8%, 86.0% WRA: 351 88.0% 83.5%, 91.4% Key end result /strategic objective: Increased voluntary use of FP and improved FP practices KR9 Unmet need for FP Pop. Survey Adol: 285 88.1% 82.5%, 92.0% 70%/90% of women (15- 49) live within 5 km of a FP service delivery point Pop. Survey 392 56.4% 45.9%, 66.4% 50% of villages >5km from HC and >750 persons have PEs providing FP services to adolescents MOH SS Baseline N/A N/A R3.1 50% of villages >5km from HC and >500 persons have CBD agents providing FP services to adults MOH SS Baseline N/A N/A WRA: 662 25.1% 20.9%, 29.8% IR 3.2 40%/70% of WRA report discussing FP with a health or FP worker or promoter in the past 12 months Pop. Survey Adol: 585 23.1% 18.9%, 27.9% Result 3: Increased access of communities to FP services IR 3.5 90% of CBD agents and PEs had no stock-outs of oral contraceptives or condoms in the last three months CBD/PE survey 11 9% CS-20 Mali, Ségou, October 2007 52 Save the Children CS-20 Mali, Ségou, October 2007 53 Result # End of Program Target/ Indicator Method Baseline Number or Survey Denominator Baseline Percent Confidence Interval IR 3.6 50% of the community outreach clinics include the provision of Depo￾Provera MOH SS Baseline N/A 0% IR2.1 50%/65% of FP clients receive adequate counseling Pop. Survey 39 12.8% 5.8%, 26.1% IR2.3 90% of HCs that report no stockouts of Depo￾Provera in the last 12 months (6 months) HFA MOH SS 16 56.25% 90% of CBD agents have received 3 supervision visits in the last quarter CBD/PE Survey 11 0% IR 2.4 80% of CSComs that received a supervisory visit in the past 3 (6) months (routine supervision) MOH SS 10 40% Result 2: Improved quality of FP service delivery by providers in facilities and in the community IR 2.7 Number of FP trainees by type of personnel and topic of training (training volume) MOH SS Baseline N/A N/A WRA: 577 MRA: 455 8.0% 16.9% 5.6%, 11.2% 13.2%, 21.4% IR 1.1 40/60% of sexually active men and women report having discussed FP with their spouse or (cohabiting) partner in the last 12 months Pop. Survey Female Adol: 504 Male Adol: 258 7.5% 20.2% 5.0%, 11.2% 15.7%, 25.5% WRA: 662 MRA: 673 83.8% 65.8% 79.9%, 87.1% 61.2%, 702% IR 1.3 90% of respondents report having heard about at least three methods of FP Pop. Survey Female Adol: 585 Male Adol: 584 82.2% 57.2% 78.2%, 85.7% 51.9%, 62.3% WRA: 662 59.8% 54.1%, 65.3% IR 1.4 75%/90% of women (15- 49) know where to obtain FP services Pop. Survey Adol: 585 60.3% 55.0%, 65.5% IR 1.5 50%/80% of men report they are favorable towards the use of modern contraception Pop. Survey *Not measured in baseline survey IR1.6 70%/80% of men can cite at least two benefits of modern contraception Pop. Survey MRA: 673 Adol: 584 53.8% 40.6% 48.8%, 58.7% 35.8%, 45.6% WRA: 174 23.6% 16.7%, 32.1% Result 1: Increased knowledge and interest in FP services through PVO/NGO involvement IR1.7 50% of mothers with children <12 months who received counseling about birth spacing Pop. Survey Adol: 171 20.5% 14.7%, 27.9% IR4.1 Program sustainability plan in place Project Reports Save the Children CS-20 Mali, Ségou, October 2007 54 Result # End of Program Target/ Indicator Method Baseline Number or Survey Denominator Baseline Percent Confidence Interval IR4.6 60%/80% of planned HF (CSComs and CSrefs) supervisory visits by District Medical Team are conducted HFA 12 25% IR4.7 60%/80% of HFs have undertaken a quality improvement process for FP and are measuring progress HFA Baseline N/A N/A IR 4.8 Project partners document and disseminate the cost￾effectiveness of community-based distribution in Mali Project Reports IR 4.9 50%/60% of HC Boards supervise CBD agents on a quarterly basis OCAT Baseline N/A 0% IR4.10 50%/60% of villages with CBD/PEs have an effective OC OCAT Baseline N/A 0% Result 4: Improved social and policy environment for FP services and behaviors IR4.11 30%/30% of the membership of OCs will be women OCAT Baseline N/A N/A Indicator sources: Pop. Sur. = Population-based Survey, MOH SS = Ministry of Health Service Statistics, OCAT = Organizational Capacity Assessment Tool, CBD/PE Survey = Community-based Agent and PE Survey Save the Children B. KPC Report Midterm Survey 2007– Translated Summary The following document is a translated summary of the midterm survey conducted in Phase I districts (Macina and Baroueli) in 2007. In addition, a baseline survey has recently been conducted in phase 2 districts (Niono and Tominian) and a report will be submitted once the analysis is completed. The summary describes the methodology, data collection and management as well as main findings and conclusions from the survey. The full document is available in French. During the first phase (2004 – 2007) of the project, interventions were implemented in the districts of Barouéli and Macina. In the second phase (2007 – 2009), activities will begin in two other localities (Niono and Tominian) of the area of Ségou is envisaged during 2007. At the beginning of the project, a baseline survey was conducted in the first 2 districts (March 2005). At midterm during the project, a new survey has been conducted with three major objectives: 1. To assess the situation at midterm (this will be the final evaluation for the first 2 districts) 2. To compare indicators at midterm with results from the baseline survey, in order to track progress achieved since the beginning of the project 3. To make recommendations or future regarding future activities, in order to help achieve project objectives as well as support the selection of the strategy to be adopted in the 2 new districts. This document course includes three parts. The first describes methodology. The results themselves are then presented in the second part. The third part draws the conclusions and the recommendations, in particular underlining the evolutions between the baseline situation and midterm. I - Methodology The study is based on the information collected according to terms defined in the scope of work. Interviews were conducted with 3 target groups: women of reproductive age (WRA) aged 15 - 49 years, Community based distribution agents (adult CBD agents and peer educators) and health providers. In addition to the interviews with target groups, information was collected at the CSCom level, regarding the equipment, supplies and staff available, as well as other conditions in connection with FP service delivery. Data collected from WRA was conducted according to a sampling design described hereafter. 1.1 Sampling of the women of reproductive age (WRA) In each of the two districts (Barouéli and Macina), the investigation included 300 women who were interviewed in 30 villages. This sample used a cluster sample design conducted in 3 stages: Stage 1: Selection of clusters. The project provided the list of the villages covered by the intervention, updated with current population figures. For each district, 30 clusters were selected by using a proportional probability to population technique Stage 2: Selection of households CS-20 Mali, Ségou, October 2007 55 Save the Children In each cluster selected, the investigation was to cover 10 households which were randomly selected. From a reference point in the cluster (which corresponds to the village), the method of the bottle was used to identify a route. While following this route, field agents went from concession in concession, until they had achieved the number of households planned for the cluster. The interviewers always turned right if they found themselves at crossroads. In concessions where there were several households, a drawing was done to retain one of them. Stage 3: Selection of women of reproductive age In each household drawn, a WRA was interviewed. If several were available, a drawing enabled to select one of them. If the woman identified was not immediately available, the interviewer would come back to the household once or twice, before replacing her by another also drawn from the list. 1.2 Sampling of community agents and health providers and data collection at CSCom level The initial plan was to interview all the providers at community and facility level who had been trained by the project (93 health providers, 88 peer educators and 250 CBD agents). Due to a decision to reduce the number of providers interviewed, a total of 52 health providers (including 27 in Barouéli and 25 in Macina) and 59 CBD agents (44 in Barouéli, 15 in Macina) and 63 peer educators (41 and 22 for Barouéli and Macina, respectively) were surveyed. These individuals were identified using the list of trained agents, which was provided by the project. At the CSCom level, data was collected from a total of 27 facilities (13 in Barouéli and 14 in Macina). 1.3 Data collection tools The interviews were conducted using 3 different questionnaires, according to the target group: women of reproductive age, CBD agents and health providers. A fourth questionnaire was used for collecting data at the CSCom level. Data was collected using the same questionnaires as those used at baseline in 2005. 1.4 Data collection in the field Field activities were carried out by a team of 16 agents (including 8 men and 8 women). These agents benefitted from a 4- day training. Facilitated by the Principal Researcher, this training was dedicated to thorough discussions of the questionnaires, the translation of questions into Bamanankan, as well as interview techniques and selection of the women in the villages. Project staff provided assistance including clarifications about contraceptive methods, IEC materials and the suitable terminologies used in the local language. The data collection in the field was carried out from the 4th – 16th May, 2007. This was done by teams of four including 1 supervisor, 2 female interviewers and 1 male investigator. Two teams were assigned to each of the two districts. The interviews of women of reproductive age were carried out by female interviewers, while those of the CBD agents and health providers were done by male interviewers. Data collection at the CSCom level was done by supervisors. The main challenge to getting the data collected was the absence of some CBD agents or health providers at the time this was done. CS-20 Mali, Ségou, October 2007 56 Save the Children 1.5 Data management and analysis Data was processed using computers The first stage of this management was data entry using EPI INFO. Data entry forms were developed and tested, before using them for the entry itself. These forms were designed so as to minimize the risks of data entry errors, using control files (*.chk). Once data entry was complete, the data were subjected to control and validation tests, in order to detect and to correct possible anomalies (inconsistencies, omissions, repetitions, etc). The data thus cleaned were used for the tabulation using STATA. In this report, the results from the interviews are described separately by target group. For the WRA and the CBD agents, in addition to overall performance (both districts merged), we also provided indicators by approach (combined versus HSS alone). It should however be specified that the overall performances among women were obtained after weighting based on total population per district. This was all the more necessary as the survey included the same sample size (300) in the two districts, despite different population size (87 974 for Barouéli and 68 760 for Macina, according to the provided brought up to date data the project). To compare baseline and the midterm data, we established a table that compares key indicators from the two periods. We did not carry out Chi 2 tests, as we did not have baseline database. Main findings, conclusions and recommendations The essential question was whether progress had been made compared to the baseline situation. To answer this question, we looked at some key indicators resulting from the baseline and midterm evaluations. The following section summarizes major lessons learned from these results. WRA have a high level of knowledge of contraception. In 2007, practically all these women know at least 3 methods (96,6%); the most popular being the pill, injectables and the condom. Despite high levels of knowledge, in 2007, only 10,6% of married women are using a modern contraceptive method. However, unmet need remains rather high: 78,9% of the women in union who are not pregnant, and do not want any more children (or not immediately) are not using any method. The current level of contraceptive prevalence, although weak- if one takes into account unmet need and the levels of knowledge - seems nevertheless to suggest an upward trend compared to the baseline situation, since CPR was 8,2% in 2005. In 2007, most of the current users (65,6%) had started use during the last 12 months. Such an observation is an indicator of the upward trend of the contraceptive prevalence. Moreover, the proportion of new users in 2007 is slightly higher than that of 2005 (59%). While continuation rates seem to be regressing, since the rate evolved from 67,5% at baseline to 57,3% at midterm, it is important to note that this was based on a smaller number of women using the method. CS-20 Mali, Ségou, October 2007 57 Save the Children Table 26: Summary of key indicators, in 2005 and 2007. Indicators Baseline (2005) Midterm (2007) Midterm Objective Married WRA who are using a modern contraceptive method (CPR) 8,2% 10,6% 17% Began use of a contraceptive method within the last 12 months (new users) 59,0% 65,6% 65% Did not use another method before the current method (continuation rate) 67,6% 57,3% 60% Births spaced at least 24 months 82,1% 69,3% ----- Unmet Need2 88,0% 78,9% ----- WRA residing within 5 km from a FP service delivery point 56,4% 47,4% 70% Discussed FP with a health or FP provider during the last 12 months 25,1% 35,6% 40% CBD agents who did not experience stock-outs in oral contraceptives or condom, within the last 3 months 9% 26,2% 90% Adequate counseling 12,8% 34,2% 50% CBD agents who received at least 3 supervision visits during the last 3 months 0,0% 70,2% 90% Discussed FP with spouse or partner during the last 12 months (sexually active women) 8,0%3 10,8% 40% Has heard about at least 3 FP method 83,8% 96,6% 90% Knows where to access FP services 59,8% 93,3% 75% Mother of a child less than 12 months who received counseling about birth spacing 23,5% 35,5% 50% In 2007, 34,2% of modern contraception users received adequate counseling before receiving the method. This result translates a significant progress compared to the baseline situation where the rate was 12,8%. The midterm evaluation however revealed certain aspects of counseling that are less impressive. At the beginning of the project, only 59,8% of women knew at least one source for FP methods. At midterm, the situation has significantly evolved to reach 93,3%; with CScoms (75,3%), and, to a lesser extent, CBD agents (26%), as well as CSRef (19,4%) being the most known sources. The geographical accessibility for FP does not seem however not generalized in 2007, since only 47,4% of the women reside within 5 km of a service delivery point. Surprisingly, this indicator of accessibility had revealed a level apparently higher in 2005 (56,4%) In regards to IEC, during the last 12 months before the survey in 2007, 35,6% of the women spoke about PF with a health / FP provider. During the last 30 days, 49,7% saw or heard messages about FP, but a rather limited proportion of women (13,9%) had contacts with the animators or CBD 2 The question was asked of women who were not pregnant, and do not want any more children or children within the next 2 years and are not currently using a contraceptive method CS-20 Mali, Ségou, October 2007 58 3 Men were asked this question during baseline – The current indicator is based on the same question being asked of women Save the Children agents. In addition, during the pre or postnatal consultations, the women do not always advice about family planning. These various indicators suggest a weak frequency of IEC about FP. However, the situation in 2007 seems better compared to that of 2005. Indeed, discussions with health providers are more frequent at midterm (35,6%) in comparison to baseline (25,1%). And the mothers of child aged less than 12 months who received a counseling about spacing births was 23,5% in 2005, compared to 35,5% in 2007. Beyond the overall situation, this study revealed important differences between the women in the two districts, and also according to the type of villages (CBD or HSS). Results show that: - Modern contraceptive prevalence is higher in Barouéli (11,3%) that in Macina (9,7%). - The frequency of counseling of users is a little higher in Barouéli than in Macina. However, the quality of the counseling is much better in Macina, with in particular a higher rate of use of samples or images to provide information about the methods available: 70,5% against 28% in Barouéli. - The rates of knowledge about contraceptive methods are in general lower among women in Macina. In addition, fewer of them mentioned living within 5 km of a service delivery point. On the other hand, the contacts with animators and CBD agents were less frequent in Barouéli, with a rate of 9,1%, against 20% for Macina. - The comparisons between the Community and the health systems strengthening villages often revealed differences favorable to the first. Indeed, in regards to knowledge, access, IEC or counseling about FP, results are often better in the Community zone. The modern contraceptive prevalence is approximately twice higher in Community villages: 15,0% compared to 7,9% in the HSS zone. The interviews conducted with CBD agents revealed that practically most of them (98,3%) benefited from basic training which covered FP. However, only 8,1% of CBD agents in Macina received refresher training. During the last 3 months, 93,4% of CBD agents provided FP services; spermicides, the male condom and the combined pill were the methods cited as the most popular methods dispensed. For the period however, only 26,2% of the relays did not experience stock-outs in condoms or progestative pills. This indicator relating to stock-outs needs to be interpreted in context as compared to the situation at baseline which had a stock-out rate of 9% (there were very few active CBD agents then and despite methods being available, given the low levels of use, most of them might have expired). Progress with CBD agents, during these two last years, can also be noted by the frequency of supervisory visits. At baseline, none of the CBD agents had received a supervisory visit during the previous quarter, compared with 70% mentioning at least 3 supervisory visits in the last quarter at midterm. The inventory of IEC materials conducted with CBD agents showed that the great majority do not have IEC materials relating to risk factors or complications of pregnancy, or reproductive health. It is also important to note that these specific activities mentioned above have not been the focus of this program. Fortunately, Family planning IEC materials are less of an issue, as 91% mentioned having flip charts. Regarding health providers at CSCom level, the great majority received basic training or additional training which covered FP. During the previous 3 months before the evaluation, 92,3% of these CS-20 Mali, Ségou, October 2007 59 Save the Children providers had delivered FP services to clients; methods most frequently delivered were injectables one, the combined pill, LAM and the condom. Practically all health providers would require a minimum age or maximum to recommend the pill (COC) or injectables. In addition, the large majority of health providers (70,6%) would require a minimum number of children (in average 4) in order to recommend injectables. These findings provide direction to future orientation of the program. It is up to the project team to deduct the necessary programmatic implications. CS-20 Mali, Ségou, October 2007 60 Save the Children C. Evaluation Team Members and their titles Team A 1. Dr Gassim Cisse – Project Manager CS Project in Sikasso 2. Oumou Kéita- Focal point person MOH National level Bamako 3. Dr Bagayoko Thierno – Chief Medical Officer District of Baroueli 4. Dr. Winnie Mwebesa – Headquarters Technical Advisor Team B 1. Aminata Kayo – Project Manager 2. Amadou Diarra – Project Trainer 3. Dr Traoré Modibo –Focal point person, MOH Regional level Ségou 4. Marguerite Joseph – External Evaluator 5. Dr Kanté Moussa- Focal point person/Assistant Chief Medical Officer District of Macina CS-20 Mali, Ségou, October 2007 61 Save the Children D. Evaluation Assessment Methodology The evaluation was led by the outside team leader with the assistance of the headquarters technical backstop. Also participating in the teams, per the above list, were key MOH program partners at every level, and program staff. Individual and group interviews were conducted with the MOH central Division of Reproductive Health, with the Regional and District partners as well, with Save the Children, Groupe Pivot and the two local (implementing) NGO partner staff in Bamako and the field levels. The external evaluator and headquarters backstop person conducted all of the above-mentioned interviews. FGDs and interviews were organized at the HF and community levels, targeting health service providers (In-charge, Midwives, depot manager), and ASACOs (HF Management committee), women, men, and relais (CHWs). In addition, the teams also interviewed religious leaders who had been involved in program advocacy. A total of 8 health zones across the two program districts were selected by the external evaluator for these activities. The sample was biased towards the larger health zones and villages with larger populations than others. Final Evaluation Calendar of Activities June 18th – July 2nd DATES ACTIVITES FACILITATEURS Lundi 18/06/07 o Introduction de l’équipe d’évaluation auprès du Staff du Mali o Aperçu sur le projet o Présentation des termes de référence par les consultants o Sélection des sites de l’enquête o Répartition des groupes/équipes o Développement des outils de collectes des données Dr Sidibé, A .Kayo, A. Diarra, F. Tony Winnie, Marguerite Mardi 19/06/07 o Interviews des représentants des ONG (2) et GP/SP o Entretien avec le staff de Save (coordinateur des programmes, conseillère régionale et staff GP/SP) o Traduction des outils de collectes A .Kayo, A. Diarra, F .Tony Winnie, Marguerite Mercredi 20/06/07 o Finalisation collectes des données et préparation des équipes d’évaluateurs o Discussion sur les résultats de l’enquête quantitative A .Kayo, A. Diarra, F. Tony, Winnie, Marguerite Jeudi 21/06/07 o Départ sur Ségou o Introduction des équipes au niveau de la DRS o Interviews au niveau de Ségou (Directeur régional, point focal) o Départ sur Macina o Interview avec le personnel du terrain du Ministère de la Santé A. Kayo, Dr Traoré, O. Keita, A. Diarra CS-20 Mali, Ségou, October 2007 62 Save the Children CS-20 Mali, Ségou, October 2007 63 DATES ACTIVITES FACILITATEURS Vendredi 22/06/07 o Interview du personnel du terrain de ASDAP o Interview des cibles (Prestataires de services, ASACO, Relais Communautaires, FAP, Hommes, Leaders communautaires) A. Kayo , Dr Traoré, O. Keita Samedi 23/06/07 o Suite interview des cibles o Retour à Ségou A. Kayo, Dr Traoré, O. Keita Dimanche 24/06/07 o Repos : Visite touristique de Ségou par les consultants, ébauche des rapports d’évaluation (OFF) A. Kayo, Dr Traoré, O. Keita Lundi 25/06/07 o Interview du personnel du terrain de AMPROD o Interview des cibles (Prestataires de services, ASACO, Relais Communautaires, FAP, Hommes, Leaders communautaires A. Kayo, Dr Traoré, O. Keita Mardi 26/06/07 o Suite interviews des cibles de Baraouéli o Retour à Ségou A. Kayo, Dr Traoré, O. Keita Mercredi 27/06/07 o Rapports des différentes équipes o Formuler les leçons apprises A. Kayo, Dr Traoré, O. Keita Jeudi 28/06/07 o Développement du plan d’action Winnie, Marguerite Vendredi 29/06/07 o Débriefing avec la DRS à Ségou o Retour à Bamako Winnie, Marguerite Samedi 30/06/07 o Finalisation du plan d’action avec l’équipe de Bamako o Préparation du débriefing Winnie, Marguerite Dimanche 1/07/07 o Ebauche du rapport final Marguerite Lundi 2/07/07 o Débriefing avec les partenaires/USAID o Prise en compte des recommandations Dr Sidibé, A .Kayo Winnie, Marguerite Mardi 3/07/07 o Ebauche du rapport final Marguerite Save the Children Midterm Evaluation sites District Health Zone (CSCom personnel; ASACO) Villages visited (Relais/CBDs women, men) Ngolokouna Goro Tongué Toumou Monimpé Nanabougou Macina Kermetogo Payakan Konobougou Kodougouni Dougoufè Fasongo Nianzana Sié Baraouéli Tamani Koniwèrè CS-20 Mali, Ségou, October 2007 64 Save the Children Focus Group Interviews – Women Knowledge of CBDs/PEs and participation in project activities a. Are there relais in your village? If yes, do you know them? b. Do women seek assistance from the relais? If yes, which ones (male/female)? How frequently? If no, why not? c. What kind of services do the relais provide in your village? (What kind of activities do they undertake)? d. Have you participated in any FP education/promotion activities? If yes, describe. e. What other types of health/FP activities would you have liked to see in your community during the last years of the project? f. Have you heard messages or received information about FP from other sources? If yes, which? Practices/Behavior Change g. For women in this village who are using FP? What influenced them in making this decision? h. For women in the village not using FP, what has influenced their decision? i. Are there socio-cultural attitudes and practices that restrict women’s decisions around family planning? If yes, what are they? And how can these be reduced or eliminated? j. Are women in this community able to communicate with their husbands about reproductive health and family planning? Explain. k. Do women in your village practice child spacing? If yes, how? What length are children spaced and why? If no, why not? l. What would be a reason for a woman to drop out of FP? Services m. What do you think of the FP services in your HF? n. When you received FP services in the village, did you receive information about the different choice of FP methods? Were you the one who chose the method that you are using? o. In your opinion, in what ways could services at the HF be improved? How could they be improved in the community with the relais? Community Mobilization 14. What kind of barriers exist to prevent members of the community from benefiting from the project? Focus Group Interviews – Married Men or men partners Knowledge of CBDs/PEs and participation in project activities a. Are there relais in your village? If yes, do you know them? b. What kind of services do the relais provide in your village? (What kind of activities do they undertake)? c. Have you participated in any FP education/promotion activities? If yes, describe. d. Have you heard messages or received information about FP from religious leaders? Other sources? If yes, which? CS-20 Mali, Ségou, October 2007 65 Save the Children Practices/Behavior Change e. For men in this village who support the use of FP? What influenced them in making this decision? f. For men in the community who do not support FP, what has influenced their decision? g. Are there socio-cultural attitudes and practices that restrict women’s decisions around family planning? If yes, what are they? Can these be reduced or eliminated? Explain. h. As husbands and fathers, what is your role in FP? i. Do men in this village communicate with their wives about reproductive health and family planning? Explain. j. Do couples in this village try to space children? How? Community Mobilization 11. What kind of barriers exist to prevent members of the community from benefiting from the project? CS-20 Mali, Ségou, October 2007 66 Save the Children E. List of Persons Interviewed and Contacted Institution Name MOH - Reproductive Health Division Dr Keita Binta Mme Oumou Keita Save the Children Sahel Country Office Mme Dunny Goodman Mr Falilou Diouf Dr Sidibé Dr Gassim Cissé Mme Aminata Kayo Groupe pivot Santé population Mr Souleymane Dolo Amadou Diarra (Groupe pívot secundee) Paul S Dao (Groupe pívot secundee) Asdap Dr Sylla Aby Doucouré Dr Simbara Fatalmoudou National level Consortium Amprode Sahel/jigui Mr Badian Kanitao Mama Sibini Sekou Sanogo Regional level Regional Health Division Dr Dicko Allasane Dr Traoré Modibo Macina Dr Moussa Kanté Dr Fadouba Sidibé Dr Sidi Mohamed Harby Dr Thierno Bagayoko Dr Ngolo Bagayoko District level Baraouéli CS-20 Mali, Ségou, October 2007 67 Save the Children F. CD with electronic copy of the report in MS WORD 2000 Attached CS-20 Mali, Ségou, October 2007 68 Save the Children G. Cost- Effectiveness Draft Report Draft Report Cost-Effectiveness Analysis of Two Approaches to Increasing the Use of Contraceptive Methods in the Health Districts of Macina and Baraoueli, Ségou Region of Mali Executive Summary Purpose Despite government and USAID efforts, modern contraceptive prevalence in Mali remains one of the lowest in any USAID-supported country and the contraceptive prevalence rate (CPR) increased from only 4.5% in 1995-1996 to 5.7% in 2001. CPR in the Ségou Region of Mali is comparatively lower at 2.8%. The purpose of this report is to describe the costs and cost-effectiveness of a community-based distribution (CBD) and a health systems strengthening (HSS) approach conducted by the Mali Ministry of Health and Save the Children, with technical assistance from Family Health International, for increasing the use of contraceptive methods in the Ségou Region of Mali beginning in March 2006. The study applied a randomized experimental-control group design to health zones in Macina and Baraoueli districts. Fourteen health zones in the control group received a “health systems strengthening approach” (local health centers are provided training and resources to reinforce the provision of family planning services) while fifteen health zones in the experimental group received the “community-based distribution intervention” (CBD agents generate interest in use of family planning through administering contraceptives or refer clients to the health center for family planning and health centers.) The two strategies both use behavior change communication BCC efforts to increase use of family planning. One strategy will be selected for Save the Children’s second phase project work and for the Ministry of Health’s efforts toward supporting family planning in Mali. Methods In order to assess the cost-effectiveness of the two alternative models, staff from the Mali Ministry of Health, Save the Children, and Family Health International first developed a base case cost analysis by outlining the direct financial and non-financial resources used in delivering the two approaches as they were implemented from March-2006-March 2007. The team created and organized an Excel-based spreadsheet workbook to track all intervention-related costs associated with inputs such as project planning time and trainings, transportation, supervision, equipment, contraceptive supplies and other consumables needed for delivering the interventions such as t-shirts, sacks, visual aids, and printed materials. In order to measure effectiveness, contraceptive utilization or uptake data were collected from each of the sites as attributed to each of the interventions. Conversion factors were applied to determine couple years of protection (CYP) for each of the contraceptive methods and overall CYP based on methods developed by Stover et al. xxiv The analysis was conducted from three cost perspectives. In the first scenario, we examine the cost per CYP using direct costs only (as described above) in the numerator. Furthermore, in a second cost￾effectiveness scenario the numerator includes additional data pertaining to Ministry of Health personnel time and costs. In a third scenario we include NGO costs such as the personnel costs of Save the Children in the numerator. CS-20 Mali, Ségou, October 2007 69 Save the Children Results The total direct (base case) costs amounted to 10,586,607 CFA for the HSS intervention and totaled 18,918,235 CFA for the CBD plus HSS intervention. Applying a conversion factor of 500 CFA = $1US, this amounts to $21,173 and $37,836 respectively. Under scenario 1, For HSS, the annual cost per CYP is 7,581 CFA or $15.16. For CBD plus HSS, the annual cost per CYP is 16,702 CFA or $33.40. Under scenario 2, the annual cost per CYP for HSS is 10,254 CFA or $20.51. For CBD plus HSS, the annual cost per CYP is 20,308 CFA or $40.62. Under scenario 3, the annual cost per CYP for HSS is 27,008 CFA or $54.02. For CBD plus HSS, the annual cost per CYP is 40,963 CFA or $81.93. Discussion Although the incremental cost per CYP is lower under the HHS intervention, the additional cost of $18.24 per CYP under the first cost-effectiveness scenario (direct costs only) represents the cost of “accessibility” or reaching segments of the population that experience significant barriers to accessing family planning services in the zones receiving the community-based intervention. Possibilities for increasing the cost-effectiveness of CBD exist and should be further examined in Phase II of the Save the Children project. Sensitivity analyses indicate a strong possibility of increasing CYP (and thus decreasing the cost per CYP) under the CBD approach by adding long-term family planning approaches to the mix of methods provided to the population under this strategy. We recommend piloting this type of strategy in one or two health zones in phase II of the Save the Children project. Furthermore, training costs (particularly curriculum development) will most likely be reduced in the second phase while contraceptive uptake could be increased through enhanced BCC efforts, community sensitization, and expansion beyond the initial pilot phase. Given the potential change in these factors, we recommend further investigation into the cost-effectiveness of scaling up CBD in Mali in the second phase of the Save the Children project. I. Introduction Mali’s population, currently estimated at 10.5 million, will double in just 22 years due to a high annual growth rate of 2.2%4. The country is sparsely populated with 7.9 inhabitants per square kilometer; although that average belies a much more densely populated South and an extremely sparse population in the North, which consists mostly of desert. Mali’s economy is one of the poorest in the world, and over 90% of the population lives on less than $2 a day5. Infrastructure is weak; only 45% of the rural population lives in communities with a year-round passable road6. Formal schooling is low among the population (66% of men and 77% of women have had no schooling). Moreover, within this context, reproductive health in Mali is challenged on several fronts. CS-20 Mali, Ségou, October 2007 70 4 Resources for Analysis of Population and its Impact on Development (RAPID) presentation by the Policy Project, Bamako, Mali April 2003. 5 UNDP. Human development reports, Human development indicators 2002. Mali. From the website: http://hdr.undp.org/reports/view_reports.cfm 6 Ministry of Health, Macro International. Enquete Demographic et de Santé 2001. (DHS III). 2002. From the website: http://www.measuredhs.com/data/indicators/ Save the Children Reproductive Health in Mali Mali has one of the lowest overall contraceptive prevalence rates (CPR) in sub-Saharan Africa (5.7%) and subsequently one of the highest fertility rates – 6.8%. These rates vary substantially by place of residence with 14.7% CPR in urban areas and 2.8% in the rural areas targeted by CS-207. Progress to improve CPR in the past decade has been modest, with only a slight increase from 4.5% in 1995-1996 to 5.7% in 2001. Yet, unmet need for family planning is substantial. One in five women of reproductive age do not want more children and two in five women want to wait at least 2 years to have another child (DHS citation). Moreover, there exists a large gap between women who have ever used modern contraceptives (16%) versus those who currently use them (6%). The DHS III results do provide some indication that younger women are choosing to delay having their first child and/or are spacing their children earlier in their reproductive lives. Overall, despite low contraceptive use rates, basic knowledge of at least one modern contraceptive method is high (78% for both married and unmarried women)8. The most widely known methods reported by married women were oral contraceptives (68%); condoms (62%); and injections (57%). Other methods are less well known (including longer term methods) such as female sterilization (33%); vaginal methods (27%); implants (Norplant) (26%); IUDs (21%); and sterilization of men (13%). The proportion of married women reporting knowledge of at least one modern method of contraception increased moderately from 67% in 1996 to 78% in 2001. Family planning client knowledge of the range of methods available and side effects is relatively low. Additional data suggests communication between partners about family planning is also low; 64% of all women who had heard of a contraceptive method had never discussed FP with their partner9. The situation in the rural Ségou Region, where the first phase CS-20, Save the Children project has been carried out, is similar to the national situation. The CPR in this region is 3.9% and knowledge of at least one method is 71%. Nearly one-quarter of all women (23%) reported using a contraceptive method at one time. Male use-rates were even higher, with 31% of all men reporting having even used a contraceptive method. The two modern methods most used by married women in Ségou are pills (2.8%), and injections (2.1%). For other methods, use is very low - 0.1% for implants and 0.3% for either sterilization and condoms. IUD use in Ségou is non-existent. There were 11,220 new FP users in MOH health facilities in Ségou in 2002, which represents a CPR of 2.5%10. Cultural and Religious Context Mali is a largely Islamic society and attitudes towards family planning are strongly influenced by religion. There is a widespread and erroneous belief that the Koran condemns contraception and that seeking to avert childbirth is sinful. Such attitudes contribute enormously to the low use of family planning11. There remain many religious leaders, especially those with little formal education in the rural areas, who are against FP and hesitate to debate it because doing so would even be considered sacrilegious12. Some leaders believe FP could be used to justify prostitution and cause the deterioration of the family and social fabric or the loss CS-20 Mali, Ségou, October 2007 71 7 DHS III 8 DHS III 9 DHS III 10 Ségou MOH 2002 statistical report. 11 Kane, T. et al. The Impact of Family Planning Multimedia Campaign in Bamako, Mali, Studies in Family Planning, 29 (3): 309-323. 1998. 12 Guitteye, A.M. Draft Report Studying the Key Actors in Family Planning in Mali, Policy Project, December 2002. Save the Children of cultural and traditional values. Conversely, there are a growing number of religious leaders, including the leadership of the Islamic High Council of Mali, who are favorable to FP and who argue its Koranic justification. These leaders support modern methods including oral contraceptives, injections, condoms, and IUDs. Save the Children / GP/SP have initiated with the Policy Project and technical assistance of the Islamic High Council of Mali, a discussion with other religious leaders in Baraoueli and Macina about the Koran and FP. They used advocacy tools including a video and discussion guide. The Islamic High Council of Mali participated in the recent Maximizing Access and Quality Exchange (MAQ) workshop in Bamako13. According to the Council, circumstances legally permitting FP are: (1) when there is a fear of inability to materially support a large family; (2) when there is a fear that the health of the child is deteriorating or their education is suffering due to the work of the father (for example the father is often traveling); (3) when the wife wishes to avoid the pain of delivery or the burden of breastfeeding; and (4) when a woman wishes to protect her beauty and youth14. Lastly, men’s opposition to family in Mali has been an important consideration in the design of CS￾20 and was also highlighted at the recent MAQ Exchange. In West Africa, husbands generally want to have more children and want to have them sooner than do their wives15. Moreover, family planning has also been perceived to encourage infidelity among wives and conflicts with men’s interest in having children to assist with farming. Community Based-Distribution of Family Planning Based upon the Bamako Initiative, the Malian health system is driven by local communities through associations of village representatives (HC Boards) which develop, put in place, and manage HCs, the first level of care for the population16. Ideally, a HC Board covers a health zone with a population of 10,000 residents, and the farthest point from a HC to any community member is 15 kilometers. Distances, sparse populations, and lack of infrastructure mean that this ideal is not always the reality. The next level of care is the district hospital, managed by the MOH. A district hospital is meant to serve the populations of all the HCs in its district, regardless of population or distance. The district hospital also provides technical monitoring and support to the HCs. Thirty-seven percent of the population of Mali lives more than 15 kilometers from a HC offering the minimum package of activities. For 68% of the rural population, the time needed to get to a HC is greater than 30 minutes by walking. Mali’s low national CPR in rural areas (2.8%) may be partially attributable to limitations in access to FP services. CS-20 Mali, Ségou, October 2007 72 13 El Hadj Dkady Drame, High Islamic Council of Mali, Presentation to the MAQ Exchange, September 2003, Bamako, Mali. 14 Islamic leaders also promote natural methods in Mali including: withdrawal, the separation of the couple, breastfeeding, the introduction of honey in the vagina before intercourse, the spider web method, the tied knot (tafo), and belts (baga). 15 Bankole, A. Desired Fertility and Fertility Behavior Among the Yoruba of Nigeria: A Study of Couple Preferences and Subsequent Fertility, Population Studies 49 (2): 317-328. 16 Although in principle health center boards receive training and support from the MOH, in reality they are often virtually untrained and neither skilled in health center management nor fund raising and financial management. Save the Children The combination of difficult terrain, insufficient health infrastructure, long distances to reach service delivery points, the frequent unavailability of a particular product or service, and insufficient and under-qualified staff severely limit utilization (only 0.17 new visits/resident/year)17. The public sector is important in the distribution and sale of various contraceptive methods. Fifty-two percent of women obtain contraception in the medical sector, largely in HCs (35%). In an effort to improve access to contraceptives, the first community based distribution (CBD) pilot program was initiated in the Katibougou district 1986. More recently, CBD projects have been implemented at large-scale in Mali. In 1994, the MOH, with the technical assistance of the Population Council, implemented a five year CBD project in five regions of the country titled Integrated Youth Health Program (IYHP). Despite these efforts, a cost-effectiveness study of CBD of family planning has not been conducted to-date. Furthermore, recent studies have documented success with the CBD approach. The majority of health care providers with experience in supporting CBD programs expressed a positive appreciation for the work of the CBD agents. Healthcare providers agreed that CBD agents generally gave good information to clients and that their activities also led to increases in FP use in the HCs18. An evaluation of IYHP showed that CPR among women 15-49 years of age in the program area increased from 28.6% in 1998 to 68.5% in 2002.19 Several successful factors of CBD programming were identified. The programs were able to effectively increase demand for FP by: (1) providing frequent and personalized information, education and communication (IEC) activities, (2) encouraging HC personnel to be proactive about discussing FP with patients; and (3) organizing school and community-based educational activities. However, despite the positive results indicated by CBD programs, no study has been conducted to￾date on the effectiveness and cost-effectiveness of the approach in Mali. A similar cost-effectiveness study20 was conducted by FHI, Population Council and the MOH in Burkina Faso in 1999, to assess the costs and cost-effectiveness of CBD interventions that were implemented in an operations research project. The project was design to test the impact of community based and clinic interventions on reproductive health knowledge, attitudes and practices as well as overall contraceptive prevalence. Services included family planning; malaria treatment and pneumonia, STI/HIV treatment and prevention; diarrhea treatment and prevention or prevention of female genital cutting. Results showed that the combination of clinic strengthening and CBD agents was effective in increasing the number of new users of modern methods. The clinic strengthening plus CBD program intervention was also considered cost-effective with an incremental cost of $13.50 per new use for the CBD program. The relative high costs of the CBD program were attributed to associated clinician training. Costs CS-20 Mali, Ségou, October 2007 73 17 MOH Report, 2001. 18 Ajaji, A. et al., Evaluation of Family Planning Programs, Population Council, 2002. 19 Diawara, A. Integrated Youth Health Program 1997 – 2002: Final Evaluation Groupe Pivot 2002. 20 Rich Homan et al., Cost and Cost-Effectiveness Analysis of Bazega Phase 1. Population Council, 2000. xxiv Stover, J., Bertrand, J., Shelton, J., Empirically Based Conversion Factors for Calculating Couple-Years of Protection, Evaluation Review, Vol. 24, No. 1, 3-46 (2000) Save the Children were driven by a large number of factors that included the number, duration and cost of trainers for the training sessions. Recommendations from the study included: • Reducing the amount and duration of clinician training. • Reducing the number of trainers used to train CBD agents and exploring the possibility of on-the￾job training and • Increasing the level of payments for the CBD agents In December 2002, a meeting in Washington, DC where best practices in CBD programs in Sub￾Saharan Africa was discussed, highlighted the role of CBD programs in bringing family planning services to underserved or remote areas. Recommendations included the need to focus on areas where health care infrastructure is weak and contraceptive use is low and also the need for improved documentation of CBD costs and best practices. Subsequently, Save the Children decided to conduct one such study in the Ségou region of Mali that will inform the MOH of Mali on the cost and cost-effectiveness of adding CBD programming to a health system strengthening of family planning services. CS-20 Study Design Currently, Save the Children (SC) and its partner Groupe Pivot/ Sante Population are implementing an expanded impact, five-year, four-district Child Survival Project, Partnership to Maximize Access and Quality of Family Planning Services in Ségou, Mali. Two local NGOs were selected to implement project activities in the districts. The project site includes a highly dispersed population of 814,712 in a vast geographical area (46,423 km2) with an estimated 184, 940 women and 170, 275 men of reproductive age. CS-20 covers four of Ségou’s seven districts: Macina, Niono, Tominian, and Baraouéli. The operations research was conducted in the districts of Macina and Baraouéli during the first phase of the project which includes the first three years. The study used a randomized experimental-control group design. A total of 29 health zones in Macina (14) and Baraoueli (15) were eligible to participate in the study. Half of the health zones in each district were randomly assigned to either the experimental or the control group. In September 2006, Save the Children and Family Health International began collaborating on the technical aspects of the research study to examine the cost-effectiveness of both the experimental and control approaches for increasing the uptake of contraceptive methods in the two health districts. The purpose of the study is to provide information regarding the incremental cost (change in cost) and cost-effectiveness (change in cost/change in effectiveness) of the following two approaches: 1) Community-based distribution (CBD plus HSS) – CBD agents generate interest in use of family planning through behavior change communication (BCC) activities and administer contraceptives (pills, condoms, and spermicides) or refer clients to the health center for other available family planning methods and services. CBD agents are provided training and resources to reinforce the provision of family planning services. 2) Health systems strengthening (HSS) – Local health centers are provided training and resources to reinforce the provision of family planning services through behavior change communication (BCC) and administering of contraceptives (pills, condoms, spermicides, injectables). More specifically, the two groups are defined as follows: CS-20 Mali, Ségou, October 2007 74 Save the Children Experimental Group CBD services were established in target villages as defined below: • Target villages for adult CBD services were defined as villages in the health zones with a population of over 500 and over 5 kilometers from a health center to be served by two CBD agents, one male and one female. • Target villages for CBD services by peer educators were defined as villages in the health zones selected for CBD with a population of over 750 and over 7.5 kilometers from a health center to be served by two peer educators, one male and one female. Community agents were recruited and selected by their communities based on criteria established with the District Health Offices (DHO). Community agents were trained and supported to: ƒ Conduct market and sales of contraceptive methods that include pills, condoms, and spermicide; ƒ Provide counseling and referral to the health centers for FP ƒ Increase effective BCC ƒ Assure contraceptive availability. In addition, in all the health zones, the project strengthened health services by: ƒ Strengthening HC provider skills; ƒ Strengthening contraceptive logistics management; and ƒ Capacity building of service providers and HC Boards for sustainable and quality FP services. Control group Health services were strengthened in all the health zones included in this group. The basis of comparison was the cost and FP uptake in zones with a CBD component, compared to those without. In the control group health zones, the project strengthened health services by: ƒ Strengthening HC provider skills; ƒ Strengthening contraceptive logistics management; and ƒ Capacity building of service providers and HC Boards for sustainable and quality FP services. III. Cost-Effectiveness Study Objective With regard to costs, the purpose of this research is to calculate the additional direct cost of resources including preparation and training, personnel time and costs, supervision time and costs, transportation / fuel costs, equipment, contraceptive supplies, and other consumables needed for delivering the interventions such as t-shirts, sacks, visual aids and printed materials of the two models (separately). With regard to effectiveness, the purpose of this research is to calculate the value of introducing these two alternative approaches into the health system by tracking the distribution of contraceptives to clients (i.e., FP uptake) within each of the two alternative approaches (separately). Ultimately, this distribution of contraceptives are translated into Couple Years Protection (CYP) for each of the methods and then summed to obtain a “total CYP” for each approach. This technique is applied as it is a commonly used as a measure of outcome in cost-effectiveness analysis of family planning programs and interventions. More specifically, the primary aims of the cost-effectiveness analysis include: • Calculate the total resource costs of providing contraceptives services through (CBD plus HHS) programs in targeted villages in the districts of Macina and Baroueli. • Calculate the total resource costs of HHS alone in the districts of Macina and Baroueli CS-20 Mali, Ségou, October 2007 75 Save the Children • Measure the effectiveness, expressed in the utilization of contraceptives and translated in “couple years of protection” (CYP) for each model of service delivery • Measure the cost-effectiveness for each model of service delivery, expressed in terms of incremental cost per CYP (change in cost/change in CYP). Secondary objectives include: • Conducting sensitivity analyses to examine cost-effectiveness under two alternative scenarios of costs: o Direct costs plus Ministry of Health Personnel costs o Direct costs plus Ministry of Health Personnel costs and NGO costs (including Save the Children costs). • Formulating policy recommendations based on the findings IV. Activities and Methods Since September 2006, Save the Children (Aminata Kayo, Tony Magassouba, Boubacar Sidibe, Dunni Goodman, Falilou Diouf, and Winifride Mwebesa), with technical assistance from Family Health International (Dr. Aaron Beaston-Blaakman and Erin McGinn) to develop tools for collecting both cost and contraceptive uptake data for the cost-effectiveness analysis. Base Case Cost Analysis In order to assess the cost-effectiveness of the two alternative models, the teams first developed a base case cost analysis by outlining both financial and non-financial resources used in delivering both approaches as they were implemented from March 2006-March 2007. This time period represents the period during which the project was executed. Subsequently, the team created and organized an Excel-based spreadsheet workbook to track all direct costs associated with these resources including planning time and all trainings, transportation, supervision, all equipment and visual aids and supplies, all contraceptive supplies purchased for the project and other consumables needed for delivering the interventions such as t-shirts and sacks. Capital costs (including some training costs and purchased motorbike vehicles) were amortized for an estimated resource use period of 5 years at a rate of 3%. More specifically, the following is a list of resource categories that were tracked among the two groups based on available data from the project for the base case cost analysis: 1. Training costs – This includes training costs and associated resources such as curriculum development, facilitator time and costs, venue costs, per diems for participants, transportation to and from the trainings, curriculum development and supplies for eight trainings (including training of trainers, training of community health workers (CPM / Matrones), training “PDQ,” training “gerants des DV / DRC Privee,” training ASACO, training for community distribution, and peer-educator training). 2. Supervision – This includes the cost of any supervision resources including supervisor time and transportation. 3. Transportation – This includes any transportation / fuel costs that are expended during the one￾year project period, including the cost of vehicles. 4. Equipment and operating costs – This includes any printing, copying, internet, telephone, utilities costs, etc. 5. Commodities (Contraceptive Supplies) – This includes all commodities distributed in the field plus any shipping and transportation costs and were estimated based on USAID international price list. CS-20 Mali, Ségou, October 2007 76 Save the Children 6. Other consumables – This includes t-shirts and sacks purchased as an incentive for CBD workers along with visual aids and printed materials. Additional Cost Data The cost-effectiveness of the intervention and control conditions can be looked at from various perspectives since scale-up can be conducted from a variety of resource bases and policy interests. As outlined above under secondary study objectives, two additional cost-effectiveness scenarios were examined after collecting data pertaining to Ministry of Health Personnel time and costs and allocating additional NGO costs. To obtain this information, personnel cost data were gathered from existing study records and average personnel salary data were collected from the Ministry of Health and participating NGOs. Personnel time allocation reports were collected and health center personnel and CBD workers were interviewed to gather data about the proportion of their time spent on family planning activities prior to the alternative interventions and subsequent to the alternative interventions. The difference in time was valued and added as the incremental time cost of providing family planning services. Lastly, additional data pertaining to indirect costs (i.e. building space and utilities) were gathered to fully allocate these costs to the two interventions. Measurement of Impact of Intervention In order to measure effectiveness or impact of the intervention, contraceptive utilization or uptake data were collected from each of the sites as attributed to each of the interventions. Data were obtained pertaining to the follow contraceptive methods: Injectables, Oral Contraceptives, Male and Female Condoms, and Spermicide. These data were captured over a 12-month period (July 2006-June 2007). Conversion factors were applied to determine couple years of protection (CYP) for each of the contraceptive methods and overall CYP based on methods developed by Stover et alxxiv. Conversion factors are as follows on a ‘per year’ basis: Injectables (4), Oral Contraceptives (15), Male Condoms (120) and Female Condoms (120), and Spermicide (120). V. General Cost and Effectiveness Results The total direct (base case) costs amounted to 10,586,607 CFA for the HSS intervention and totaled 18,918,235 CFA for the CBD plus HSS intervention. Applying a conversion factor of 500 CFA = $1US, this amounts to $21,173 and $37,836 respectively. Table 1 shows the cost distribution proportions of these costs among the primary resource categories for HSS and CDB, and HSS only. For both interventions, training and equipment costs represent the highest proportion of costs, while additional costs were spread more evenly among categories of supervision and transportation. Table 1 – Distribution of Costs By Intervention and Resource Category Resource HSS CDB Plus HSS Training 29% 39% Supervision 15% 13% Transportation 12% 11% Equipment 25% 21% Contraceptive Supplies 8% 7% CS-20 Mali, Ségou, October 2007 77 Save the Children Other Consumables (including t-shirts, visual aids, printed materials) 11% 9% Table 2 shows the one-year contraceptive uptake data counts and Table 3 represents the conversion to couple years protection (CYP). The data indicate the heavily reliance of CBD workers on the use of pills, condoms, and spermicides in the field and the high amounts of injectibles delivered at the health centers. Despite the fact that CBD workers refer to health centers for injectibles and do not provided them at this point in time, the referral process did not yield the same amount of injectibles as provided directly within the control group. In Macina alone, almost 4 times the amount of injectibles are offered and provided by the control group when compared to CBD. Alternatively, in Baraoueli, 4 times the amount of pills were provided in the experimental group when compared to the control. Table 2 - One Year Contraceptive Uptake Data Counts HSS CBD Plus HSS Method Macina Baraoueli Macina Baraoueli Injectables 2746 1832 762 976 Ovrette (pilule) 165 726 142 275 Pilplan/ lo-femenal (pilule) 1723 890 4188 3729 Male Condoms 1366 909 1689 2328 Female Condoms 0 0 0 0 Spermicide 2906 3560 5433 10,140 These outlined differences are also reflected in the conversion to CYP as injectibles yield a higher CYP than pills, condoms, or spermicides. In total, HHS yielded 1,396 CYPs and CBD plus HHS yielded 1,184 CYPs. Table 3 - One Year Data Couple Years Protection (CYP) Conversion HSS CBD Plus HSS Methods Macina Baraoueli Macina Baraoueli Injectables 654 436 181 232 Ovrette (pilule) 11 48 9 18 Pilplan/ lo-femenal (pilule) 115 59 279 249 Male Condoms 11 8 14 19 Female Condoms 0 0 0 0 Spermicide 24 30 45 85 Total 815 581 529 603 CS-20 Mali, Ségou, October 2007 78 Save the Children As it can be observed from these data, contraceptive uptake varied in each district and raises some additional questions as to whether this was due to contraceptive stock-outs, the effectiveness of messages from CBD agents, or other factors; scale-up of the project may benefit from further investigation into these issues. VI. Cost-Effectiveness Analyses The cost of the intervention and control conditions can be looked at from three perspectives as scale-up or adaptation of either approach can be conducted from a variety of resource bases and policy initiatives. In the first scenario (Table 4) only direct costs only associated with project planning time and trainings, transportation, supervision, equipment, contraceptive supplies and other consumables needed for delivering the interventions such as t-shirts, sacks, visual aids, and printed materials are considered. These would be the costs required to replicate this effort in another district without considering existing MOH staff or additional NGO costs. Results from this perspective are outlined in Table 4, and show the annual cost per CYP for HSS as 7,551 CFA or $15.16. For CBD plus HSS, the annual cost per CYP is 16,702 CFA or $33.40. Table 4 – Base Case Cost-Effectiveness Analysis HSS CDB Plus HSS Total Direct Costs (CFA) 10,586,607 18,918,235 Total Direct Costs ($) $ 21,173 $ 37,836 CYP (1 Year) 1,396 1,133 Annual Cost Per CYP (CFA) 7,581 16,702 Annual Direct Cost Per CYP $ $ 15.16 $ 33.40 The second analysis took into consideration all direct costs, plus the additional Ministry of Health personnel costs required for delivering added family planning services. As MOH staff must share these added responsibilities with other services, understanding these costs are important. In this second scenario, the annual cost per CYP for HSS is 10,254 CFA or $20.51. For CBD plus HSS, the annual cost per CYP is 20,308 CFA or $40.62. Table 5 – Cost-Effectiveness Analysis Scenario 2 – Direct Costs Plus MOH Personnel Costs HSS CDB Plus HSS Added MOH Personnel Costs (CFA) 3,732,861 4,084,506 Direct Costs (CFA) 10,586,607 18,918,235 Total Annual Costs (CFA) 14,319,468 23,002,741 Total Annual Costs ($) $ 28,639 $ 46,005 CYP (1 Year) 1,396 1,133 Annual Cost Per CYP (CFA) 10,254 20,308 Annual Direct Plus MOH Cost Per CYP ($) $ 20.51 $ 40.62 CS-20 Mali, Ségou, October 2007 79 Save the Children Finally, the third analysis took into consideration all direct costs, Ministry of Health personnel costs, and personnel and other costs incurred by NGO partners. In this third scenario, the annual cost per CYP for HSS totals 27,008 CFA or $54.02. For CBD plus HSS, the annual cost per CYP totals 40,963 CFA or $81.93. Table 6 – Cost-Effectiveness Analysis Scenario 3 - Direct Costs Plus MOH Personnel Costs and NGO Costs (including Save the Children) HSS CDB Plus HSS Added NGO Costs (CFA) 23,394,916 23,394,916 Added MOH Personnel Costs (CFA) 3,732,861 4,084,506 Direct Costs (CFA) 10,586,607 18,918,235 Total Annual Costs (CFA) 37,714,384 46,397,657 Total Annual Costs ($) $ 75,429 $ 92,795 CYP (1 Year) 1,396 1,133 Annual Cost Per CYP (CFA) 27,008 40,963 Annual Direct Plus MOH and NGO Cost Per CYP ($) $ 54.02 $ 81.93 I. Discussion / Recommendations As discussed earlier, Mali is faced with a significant rural population and poor infrastructure, making access to health care services, including family planning a challenge for the majority of the population. CBD services are one mechanism by which the MOH can address these needs. Understanding the costs and impacts of such a strategy are important within the context of the existing health care infrastructure as well as for improving upon the approach in the future. Our results indicate that in the pilot phase, adding a CBD component doubled the direct costs. However, if all costs associated with the intervention are taken into consideration, such as MOH personnel costs and the inputs provided by NGOs, CBD is 60% higher relative to HSS alone. If the only objective of this study was to maximize CYP at the lowest cost, one would undoubtedly choose the HSS approach, as it is lower in cost and maximizes CYP when compared to CBD. On the other hand, if one considers several important social, geographic, and cultural factors associated with or not accounted for in this economic analysis, the development and application of CBD within Mali begins to make more sense. First, given the objective of accessibility (in addition to increased CYP) the added cost of CBD can be interpreted as the “cost of access” to family planning services for underserved segments of the population. Secondly, this study presents results based upon a pilot analysis where processes were still largely in a development phase. The true “cost of access” can be determined over a longer time period of implementation. It is reasonable to say that over time with ongoing BCC, contraceptive uptake would increase, thus increasing the CYP per dollar spent. Lastly, HSS can serve as a framework or reference case against which the cost and impact of CBD programs can be measured while moving forward in the second phase if Save the Children’s project in the Ségou region. CS-20 Mali, Ségou, October 2007 80 Save the Children The majority of the methods provided in Macina and Baraoueli had very low CYP value (pills, condoms, spermicides). The following example demonstrates a way of lowering the cost per CYP under the CBD approach by introducing longer-acting or permanent methods to the mix available at the health center while the CBD agents were trained to promote, counsel, and refer for these methods. A relatively small uptake of LAPMs could have a significant impact on the cost-effectiveness of the intervention. Consider if either implants or IUDs were offered by the health centers, and economies were made to add this method within the existing direct costs. If only 5% of injectable users switched to either implants or IUDs, CYP would increase under CBD by approximately 37%. With these estimates, direct costs for CBD would fall from $33.40 per CYP to $24.47 per CYP. Further use of LAPMs would decrease the cost per CYP even more significantly. Furthermore, it should be noted that costs to the client were not accounted for in the cost￾effectiveness analysis. If these costs were added to the model, they would mostly likely increase the cost of the HSS approach since they would include time and transportation costs to the client. Such costs would not be reflected under CBD. Recommendations With the objective of advancing family planning services in Mali, increasing CYPs and improving access to family planning methods, we recommend that the Ministry of Health and Save the Children continue to expand their efforts in the Ségou region by developing and evaluating a more cost-effective approach to scaling-up CBD services. There are several possible means for achieving the goals of increased CYPs, lower costs, and increased accessibility of family planning services among the population. These ways are described as follows: • Seek ways to reduce the costs of training – As training of CBD workers continues, facilitators will most likely be able to reduce preparation costs (including curriculum development costs). Moreover, during Phase II of the Save the Children project, experienced CBD workers will be trained to provide family planning specific services. • Consider piloting one or two health centers to provide an LAPM – As described above in the sensitivity analysis, adding an LAPM to the method mix will greatly contribute a rise in CYPs and a subsequent decrease in the cost per CYP. • Continue with rigorous BCC efforts – Sustained BCC efforts will most likely have greater effects over time (resulting in family planning sensitization in the community and thus, increased CYP). • Conduct cost-effectiveness analyses during the scale-up phase and over a longer period of contraceptive uptake to gather a more complete perspective of the cost-effectiveness of CBD compared to other approaches for improving the use of family planning services in Mali. These data will be extremely useful for the Ministry of Health as it considers the use of CBD workers for family planning work throughout the country. CS-20 Mali, Ségou, October 2007 81 Save the Children H. NGO Organizational Capacity Assessment Report ORGANIZATIONAL CAPACITY ASSESSMENT OF NGOs HAVING IMPLEMENTED ACTIVITIES DURING THE FIRST PHASE OF CS 20 SEGOU INTRODUCTION Save the Children (SC) in partnership with the Ministry of Health, Groupe Pivot Santé Population (GP /SP) and two national NGOs have implementing since October 2004, a five year expanded impact program within four districts of the region of Segou. This project named « A Partnership to Maximize Access and Quality of Family Planning Services in Ségou Region, Mali ». This project, responding to the communities’ needs that are a priority for the Ministry of Health and Save the Children national and international strategies, will increase sustainable and voluntary use of modern methods of contraception among women and men of 15 and 49 years old. The first phase of the project ended in September 2007. The midterm evaluation has enabled the review of intervention strategies based on recommendations from the evaluation in order to achieve program objectives during the second phase. During the first phase, the two selected NGOs (Consortium AMPRODE SAHEL/JIGI, ASDAP) have demonstrated their capacity to manage the delivery of FP programming and services at community level. In order to ascertain their capacity to continue implementing activities in the 2 new districts, the project team (SC and GP/SP) have conducted an organizational capacity assessment of the two NGOs. The present report describes the steps followed during the process of the evaluation of the two NGOs. This report describes three components: the Methodology used, the activities implemented and recommendations. I- METHODOLOGY The NGOs were evaluated in accordance with norms and procedures in use in the two organizations. The terms of reference that were established for this exercise defined the context of the evaluation, objectives, strategies and the mechanism of the procedures to use. The team that conducted the evaluation included: • CS 20 Regional Advisor for the Child Survival project in Segou • Groupe Pivot Santé Population’s internal auditor • Groupe Pivot Santé Population’s assistant internal auditor The evaluation was conducted in 2 stages: • Phase I: Document review under the control management tool • Phase II- : Individual Interviews with executive directors, administration, accounting managers and program officers. II- ACTIVITIES The implemented activities during the evaluation process are the followings: A- Development of terms of reference and review of data collection tools CS-20 Mali, Ségou, October 2007 82 Save the Children This stage allowed the team to focus on the data to collect, how to classify them according to their utility for the project and to develop the matrix to integrate the information. B –Results Analysis / Findings The table below the findings based on the framework used for the assessment. Table 1: Analysis of information collected: CONSORTIUM AMPRODE SAHEL /JIGI DESIGNATION FINDINGS FINDINGS CONSORTIUM AMPRODE SAHEL /JIGI ASDAP 12. Analysis of structural authorities At this level, we noticed the existence of a general organizational plan, clear and updated for Jigi and in process of being updated for AMPRODE. In addition to that, we were able to verify the existence of regulations and status, the legal authorization of settlement; a manual of procedure updated in 2004 for Jigi and in process of being updated for AMPRODE. Insufficiency in the structuring of the organization chart. Organization chart not displayed. Existence of a general organizational plan, clear and updated. We were also able to verify the existence of regulations and status, the legal authorization settlement ; a manual of procedure in process of being updating Organization chart not displayed 13. Analysis of the internal control system We noticed an internal control system done by an advisory cabinet led by a team of accounting commissioners. People in charge of internal control promptly proceed to the control of the balance, reports and other. Existence of an internal control system conducted systematically and annually by an advisory cabinet 14. Analysis of finance and accounting system Compliance with the account management cycle by the specification of accounting documents, use of Excel Spreadsheets. Existence of petty cash ; the recording operations are done daily and the different supporting documents are Compliance of the account management cycle by the specification of accounting documents, use of Excel Spreadsheets. Existence of petty cash ; the recording operations are done daily and the different supporting documents are classified and archived Agents are paid in cash and not by bank CS-20 Mali, Ségou, October 2007 83 Save the Children CS-20 Mali, Ségou, October 2007 84 DESIGNATION FINDINGS FINDINGS CONSORTIUM AMPRODE SAHEL /JIGI ASDAP classified and archived Agents are paid in cash and not by bank transfer and the salary notes are not systematically given to them. The accountants and field agents don’t have a good mastery of GP/SP and Save the Children administrative and financial procedures. transfer. The accountants and field agents don’t have a good mastery of GP/SP/Save the Children administrative and financials procedures. 15. Analysis of procedures manual Existence of a manual of procedure but which needs to be updated. Existence of procedures manual that is the process of being updated. 16. Analysis of annual financial reports The reports are regularly produced according to the guidance on grants’ contract. The reports are regularly produced according to the guidance on grants’ contract. 17. Analysis of Records system In general, the recording system is good In general, the recording system is good 18. Budget follow-up The System of budget follow-up is compliant with the guidance with grants’ contract The System of budget follow-up is compliant with the guidance with grants’ contract 19. Bank reconciliation status Statements are regularly produced and they are concordant Statements are regularly produced and they are concordant 20. Analysis procurement system The system puts in competition the potential suppliers and respects the procurement procedure. The system puts in competition the potential suppliers and respects the procurement procedure. 21. Analysis of stocks’ management Stock non existent Stock non-existent 22. Analysis of fixed assets Existence of a register of fixed assets which describes the serial number, the cost of the procurement, the purchase date and the funding sources. The consortium conducts an annual physical inventory of its properties, by précising the different transfers. Existence of a register of fixed assets which describes the serial number, the cost of the procurement, the purchase date and the funding sources. Conducts an annual physical inventory of its properties but doesn’t precise the transfers conducted during the year. Save the Children Table 2 : Structural Analysis of management capacity of the two NGOs NGO Operational NGO 5 years and more of experience in FP Management capacity of health programs Have an administrati on and finance management manual Have human resources Last annual activity report Copy of last annual finance report Minutes of the last GA Three last minutes of staff meeting or executive committee Internal control analysis TOTAL CONSORTI UM AMPRODE SAHEL/JI GUI 1 2 5 2 5 1 1 1 1 7 26 ASDAP 1 2 6 2 5 1 1 1 1 8 28 The management analysis of the two partners NGOs has allowed us to assess their operational status. The evaluation was based on the following criteria (see table) which show strengths and weaknesses. The results summarized in the table above were done out of a total of 31 points: • The Consortium AMPRODE SAHEL /JIGUI received a total of 26 point • ASDAP received a total of 28. In comparison to the baseline assessment in 2005 the two NGOs had received: • Consortium AMPRODE SAHEL /JIGI :20, • ASDAP 21 , These results illustrate the relative good management of the two organisms, which proves that they are able to continue even though a number of improvements need to be made. CS-20 Mali, Ségou, October 2007 Save the Children 85 III- RECOMMENDATIONS Based on these results, we make the following recommendations which should lead to an improvement in the quality of their respective management system. A- NGO: CONSORTIUM AMPRODE SAHEL/JIGI: Salaries should be paid by bank transfer, Monthly salary statements should systematically be given to all the agents, Proceed to update the organizational structure chart by précising the positions, Display the organization chart in the offices, Promptly review the procedures manual, Insure real estate properties against natural disasters, Organize an orientation session for accountants and field agents on GP/SP & Save the Children administrative and financial procedures in collaboration with different departments. B- NGO: ASDAP: 1- Salaries should be paid by bank transfer, 2- Insure real estate properties against natural disasters, 3- Display the organization chart within all the offices and on the display board, 4-Improve the inventory system of properties at the end of year by précising the different movements, 5- Promptly review the procedures manual 6- Organize an orientation session for accountants and field agents on GP/SP & Save the Children administrative and financial procedures in collaboration with different departments Note: It is important to note that after undertaking the Organizational Capacity Assessment of the two NGOs, ASDAP made a decision to withdraw from the project. CS-20 Mali, Ségou, October 2007 86 Save the Children I. Project Data Sheet SCF (Mali) General Information Funding Mechanism: CSHGP / USAID USAID Washington Funding: $1,700,000 USAID Mission Funding: $800,000 SCF Match Funding: $558,210 Cooperative Agreement No: GHA-A-00-04-0003-00" Project Start/End Dates: (Sep 30, 2004 - Sep 30, 2009) Project Name: Partnership to Maximize Access and Quality of Family Planning Services Project Description: SC is implementing an expanded impact 5-year project in the Segou Region of Mali in the following districts: Macina,Baraoueli (Phase I: 2004 - 2007, Niono and Touminian (Phase II: 2007 - 2009) . Mali has one of the lowest overall CPR in sub-Saharan Africa (5.7%) and thus one of the highest fertility rates: 6.8. There remains a large unmet need with the 2001 DHS reporting significant proportions of WRA who either wanted no more children (21%) or wanted to wait at least two years to have another child (39%). Mali’s population currently estimated at 10.5 million will double in just 22 years due to a high annual growth rate of 2.2%. The project goal is for 25% of WRA (15-49 years) to use modern contraception. This will be achieved through the following results: 1) increased access to FP services (50% of villages > 5 km from health center and > 500 persons have community-based distribution agents providing FP services - 50% of villages > 5 km from health center and > 750 persons have peer educators providing FP services) 2) improved quality of FP service delivery - 80% community-based distribution agents provide adequate counseling (disaggregate by accurate information on choice, correct use, common side effects and when to return) 3) increased FP knowledge and interest (60% of men and women will report ever discussing FP with their spouse or partner, 80% of women and men can cite at least three benefits of modern contraception) 4) improved technical and institutional capacity health services and communities to provide quality FP (80% of health centers have undertaken a quality improvement process and are measuring progress, 80% of villages with community-based distribution agents have an effective Oversight Committee) The key strategies may be broken down into two categories: community-based services and health services. The community-based services include: marketing and sales of contraceptives by peer educators and community-based distributors in villages where there is not a health center; increasing effective BCC through interpersonal and popular village channels; assuring contraceptive availability; and counseling and referral to health centers. The health services strengthening includes: strengthening of contraceptive logistics management; strengthening of provider skills; and capacity building of service providers and health center boards. Group Pivot and two local NGOs will be sub-grantees on the project. The project is also collaborating with CARE, GP/SP HIV partners, ASDAP and the UNICEF SASDE project in the Region. The CBDs trained CS-20 Mali, Ségou, October 2007 87 Save the Children by the SASDE project will be trained to add FP services to their other child survival activities. In addition, they will collaborate with the DELIVER project, the national Abt Associates project and CARE’s Kenya Ciwara Region: Segou District: Macina and Baraoueli,Niono and Touminian Districts Project Coordinates: Latitude: 11.3166667 Longitude: -5.6666667 Geographic Subareas (Does this project collect, monitor and report on Flex Fund indicators for different geographic project subareas ?) If this is true, click Yes and enter each distinct subarea name: If this is false, click No. Yes No Subarea Name Listing: Click Box Next to Name to Remove Subarea Macina & Baraoueli Districts - Phase one (2004 - 2007) Niono & Touminian Districts - Phase tw o (2007 - 2009) CS-20 Mali, Ségou, October 2007 88 Save the Children Grantee Project List Sub Form 2: Project Contacts Flex Fund Home Key Stakeholder Contacts Grantee HQ Technical Backstop: Winnie Mw ebesa Grantee HQ Financial Backstop: Carmen Weder Grantee Regional Contact: Grantee Regional Contact List Funding Mechanism Contact: Nazo Kureshy USAID Mission Contact: Christine Kolars Sow (Tech. Advisor, AIDS & Child Survival) USAID Washington Representative: Jenny Troung Primary Field Contact First name: Aminata Last name: Kayo Title: Project Coordinator Telephone: 223-229-613 Fax: Email: akayo@saveusamali.org Alternate Field Contact First name: Sidibe Last name: Boubacar Title: Health Coordinator CS-20 Mali, Ségou, October 2007 89 Save the Children Telephone: 223-229-6135 Fax: Email: bsidibe@saveusamali.org Partner Information: Name Type $ Allocated Remove Groupe Pivot/Sante Population Subgrantee $1213219 Regional health authorities Collaborating $0.00 District health authorities Collaborating $0.00 DELIVER Project Collaborating $0.00 ASDAP Subgrantee $157030 APRODE jigi SAHEL Subgrantee $166742 Health center boards Collaborating $0.00 Oversight committees in villages w ith Collaborating $0.00 FHI Collaborating $0.00 CS-20 Mali, Ségou, October 2007 90 Save the Children Sub Form 3: Project Beneficiaries Grantee Project List Flex Fund Home Programmatic Area A: Adults Type Age Range Macina & Baraoueli Districts - Phase one (2004 - 2007) Niono & Touminian Districts - Phase two (2007 - 2009) WRA 15-49 89,663 Married WRA 15-49 Men 15-59 Programmatic Area B: Youth Type Age Range Macina & Baraoueli Districts - Phase one (2004 - 2007) Niono & Touminian Districts - Phase two (2007 - 2009) Female Youth 10-14 Female Youth 15-19 Female Youth 20-24 Male Youth 10-14 Male Youth 15-19 Male Youth 20-24 Population of Target Area Macina & Baraoueli Districts - Phase one (2004 - 2007) Niono & Touminian Districts - Phase two (2007 - 2009) Population of Target Area: 407,558 465,032 CS-20 Mali, Ségou, October 2007 91 Save the Children Comments WRA and Pop of target area temporarily split (adjust w ith actual ry data) beneficia Sub Form 4: Project Focus Areas Grantee Project List Flex Fund Home CS-20 Mali, Ségou, October 2007 92 Key Technical Focus Areas Youth Behavior Change Communication (BCC) Community-based distribution (CBD) Health Facilities Integration Integration HIV/AIDS Contraceptive logistics Cost Recovery Social Marketing Female Genital Cutting Post Abortion Care Gender Contraceptive Methods Distributed Female Sterilization Male Sterilization Pills IUD Injectables Implants Male Condom Female Condom Diaphragm Foam/Jelly Lactational Amenorrhea Standard Days Method Fertility Awareness Methods (Non SDM) emergency contraception Save the Children Sub Form 5: Data Entry of Core Indicators Project Data Phase: Annual Report 3 Core Indicators (2007) More Information LEVEL:1 Indicator Required Core Indicator Geographic Sub Area Number Numerator Denominator Percent (Auto- Calc) Confidence Interval (Auto-Calc) Yes/No Macina & Baraoueli Districts - Phase one (2004 - 2007) 3247 Couple Years of Protection (CYPs) Niono & Touminian Districts - Phase two (2007 - 2009) Macina & Baraoueli Districts - Phase one (2004 - 2007) 0 38 0 0 Percent of Facilities Reporting No Stockouts in the last quarter Niono & Touminian Districts - Phase two (2007 - 2009) CS-20 Mali, Ségou, October 2007 Save the Children 93 Macina & Baraoueli Districts - Phase one (2004 - 2007) 22 64 34.4 18.5 Percent of Clients who Receive Adequate Counseling Niono & Touminian Districts - Phase two (2007 - 2009) Macina & Baraoueli Districts - Phase one (2004 - Percent of Facilities 2007) Offering 3 or More Modern FP Methods Niono & Touminian Districts - Phase two (2007 - 2009) Macina & Baraoueli Districts - Phase one (2004 - 2007) 183 387 47.3 8.5 Percent of Population Living Within 5 km of a FP Service Delivery Point Niono & Touminian Districts - Phase two (2007 - 2009) CS-20 Mali, Ségou, October 2007 Save the Children 94 Macina & Baraoueli Districts - Phase one (2004 - 2007) 9486 Number of Acceptors New to Contraception Niono & Touminian Districts - Phase two (2007 - 2009) Macina & Baraoueli Districts - Phase one (2004 - 2007) Program Sustainability Plan in Place Niono & Touminian Districts - Phase two (2007 - 2009) Comments The project is tracking additional indicators that are reported in the MTE report CS-20 Mali, Ségou, October 2007 Save the Children 95 CS-20 Mali, Ségou, October 2007 Save the Children 96 J. Phase Two Action Plan CS-20 Mali, Ségou, October 2007 97 SUMMARY FINDINGS and CONCLUSIONS RECOMMENDATIONS INITIAL ACTIONS PERSON(S) RESPONSIBLE TIME FRAME IR1: Increased access of communities to FP services The analysis of CPR by strategy (CBD versus CBD and HSS) shows that the CBD approach was more effective in reaching women with services Expand community based distribution in all the health zones in the 2 new districts Identify, train and equip all the CBD agents selected (based on agreed upon criteria of village population and distance to a CSCom) Project staff with authorities in new districts and regional point person July – Dec 2007 FP consultation free significantly influenced the use of FP services at CSCom level in Barouéli Encourage free FP consultation in new districts Negotiate with district authorities the establishment of free FP consultations CS20 project staff with authorities in new districts and regional point person Oct – Dec 2007 Limited identification of eligible couples to be targeted with FP messages and services CBD agents should be trained in ELCO method Review training curricula to ensure ELCO method is included Train & supervise CBD agents in use of ELCO method Project staff November –Dec 07 Limited contact with women’s groups and associations where women can be encouraged to discuss the benefits of birth spacing and use of contraception Intensify activities with women’s groups & women’s associations Systematically identify women’s groups and women’s associations by health zone and by village Project staff Coordinators/ Supervisors/ Animators November 07 Most of the CBD agents did not use up the initial stock of contraceptive supplies they received Review initial supplies given to CBD agents Initial supplies should be based on village population, level of contraceptive use Project staff November 07 Peer educators are less successful in providing FP services compared to adult CBD agents The project should review the role of PEs in the second phase in order to maximize resources Consider use of PEs in health zones that are predominantly urban Coordinators/ Supervisors/ Animators November 07 Women who access ANC, Postnatal care and immunization services are not receiving FP services Need to minimize missed opportunities to reach WRA with FP services Integrate FP messages and services during ANC, postnatal care and immunization sessions Train CSCom providers to target women during these critical periods Track number of women Project staff/ animators/ CSCom staff Nov Save the Children CS-20 Mali, Ségou, October 2007 98 SUMMARY FINDINGS and CONCLUSIONS RECOMMENDATIONS INITIAL ACTIONS PERSON(S) RESPONSIBLE TIME FRAME receiving care during these periods who are also receiving FP messages and services IR2: Improved quality of service delivery by health providers at the facility and community level Low use of methods such as 1) LAM 2) longer acting and permanent methods (LAPMs) The method mix should be expanded to include methods such as 1) LAM and SDN 2) Longer acting and permanent methods which might increase the cost￾effectiveness of the program (by increasing CYP) Develop an appropriate strategy to effectively integrate lopper LAM and SDM in the Malian context. Work with IRH/DSR to develop/ adapt support materials to operationalize the methods Promote the 2 methods in the new districts (ie: Exclusive breastfeeding support groups) Work with regional and district health authorities to make LAPMs available by training providers and providing equipment and supplies Project staff/ DSR/IRH Project staff/ DSR / IRH Coordinators/ Supervisors/ Animators Project staff/ DSR/ November 07-Jan 08 November 07-Jan 08 December 07- June 09 Nov 07 – Sept 09 There are frequent stock-outs of certain products (spermicides, injectables) at facility and regional levels despite the fact that providers have been trained Need for a strategy that prevents stock-outs at regional and district level Track stock levels at all levels (Regular review of monthly inventories) Advocate for improved supplies at regional level Project staff/ Coordinators/ Supervisors Project staff/ DSR December 07- Sept 09 Providers have biases about providing FP services Need to implement a quality improvement system for FP services Develop a plan for QI and share with partners Build capacity of implementing NGOs in QI Project staff Jan 08 The performance of providers/ CBD agents varies in each district Need to strengthen supervision at all levels provided on a regular and effective manner Review current supervision system to identify gaps Identify opportunities for integrated supportive supervision Project staff Coordinators/ Supervisors/ NGO point person Jan 08- June 09 Supportive supervision and regular follow-up of CBD agents contributed to the successful implementation of Strengthen supportive supervision of CBD agents and capacity building opportunities Review health zone planning for outreach activities and ensure supervision of CBD agents is included Ensure capacity building Project staff Coordinators/ Animators, CPM et ASACO Project staff December 07- June 09 Save the Children CS-20 Mali, Ségou, October 2007 99 SUMMARY FINDINGS and CONCLUSIONS RECOMMENDATIONS INITIAL ACTIONS PERSON(S) RESPONSIBLE TIME FRAME program activities at community levels Capacity building of CBD agents contributed to improved quality of information with target groups for CBD agents at each contact Coordinators/ Supervisors/ Animators & district point person CBD agents can mentor their peers and this leads to improved data collection and quality of services at community level Create opportunities to enable information sharing between CBD agents of the same health zone Organize regular for a (every six months) at the health zone level to allow information sharing between CBD agents Project staff Coordinators/ Animators & CPM Mar 08- June 09 Limited resources and contributions from the community let to most of the health zones being unable to complete PDQ process in their communities PDQ methodology to be replaced by a methodology that requires less resources for implementation IR3: Increased FP knowledge and interest of communities Religious leaders have been influential in changing attitudes about FP especially at regional and in large cities and among some men Decentralize involvement of religious leaders up to village level Organize decentralized advocacy sessions within the health zones Trained religious leaders should replicate activities at village level Involve Imams from principal mosques to facilitate activities at village level (quarterly meetings) Project staff Coordinators/ Supervisors/ Animators Coordinators/ Supervisors/ Animators Coordinators/ Supervisors/ Animators February 08- June 09 February 08- June 09 February 08- June 09 8.9% mentioned spousal opposition to FP as the reason for not using FP Strengthen FP sensitization activities targeting men Develop a strategy that engages men in FP (regular meetings with men, ASACO members at village level, meetings between religious leaders and men., use of traditional channels for disseminating messages) Project staff Coordinators/ Supervisors/ Animators December 07-June 09 Only 10% of women interviewed mentioned Implement a strategy that targets couples in order to promote couple Implement a strategy that promotes couple communication Project staff Coordinators/ Supervisors/ December 07-June 09 Save the Children CS-20 Mali, Ségou, October 2007 100 SUMMARY FINDINGS and CONCLUSIONS RECOMMENDATIONS INITIAL ACTIONS PERSON(S) RESPONSIBLE TIME FRAME discussing FP with their partners communication Train CBD agents in use of ELCO method to direct home visits for couple counseling Animators Religious leaders have been influential in changing attitudes about FP but this has not reached women Involve female religious leaders in advocacy sessions with religious leaders Organize advocacy sessions in villages and include meetings with female religious leaders Coordinators/ Supervisors/ Animators December 07-June 09 There still are taboos regarding FP despite the progress made so far Strengthen communication activities at all levels with greater involvement of community leaders to allow better understanding and acceptance of FP Develop a BCC strategy for areas where Muslim is not the predominant religion (animists, Christians) Project staff/ Coordinators/ Supervisors/ Animators /district point person Jan 08- June 09 Radios were not an effective way of reaching communities with FP message in phase 1 Identify local radios with high coverage and evaluate feasibility based on available funding resources Review data from population based survey to identify high coverage radios Review budget to assess whether radios contracts can be issued If funding allows, sign negotiate contracts with local radios Develop a media plan that incorporates FP messages Provide orientation to radio promoters Project staff/ Coordinators December 07- September 09 Tominian has a different religious profile ( predominantly Christian) Implement an advocacy strategy with religious leaders that is appropriate for the new districts (Tominian) Work with POLICY to develop an appropriate advocacy strategy for religious leaders in Tominian Project staff / POLICY Jan 08 IR4: Increased technical and institutional capacity of health services and communities for FP/RH services and behaviors Limited involvement of ASACO members in the implementation of FP within their health zones Lack of integration of some CBD agents in the health system within certain health zones Need for more involvement of ASACO members Establish a framework for information exchange between ASACOs and providers at CSCom level including the animator - regarding FP technical information to guide decision making and support of CD agents Organize regular meetings between ASACO s and technical staff at CsComs Coordinators/ Supervisors/ Animators December 07-June 09 Save the Children CS-20 Mali, Ségou, October 2007 101 SUMMARY FINDINGS and CONCLUSIONS RECOMMENDATIONS INITIAL ACTIONS PERSON(S) RESPONSIBLE TIME FRAME Cost-effectiveness study The analysis of CPR by strategy (CBD and HSS versus and HSS alone) shows that the combined approach though more expensive was more effective in reaching women with services Implement the combined approach (CBD + HSS) in the new districts Use agreed upon criteria to select villages that are eligible to have both approaches within Project staff July – November 07 CYP was mainly limited to the use of pills, condoms and injectables Continue collaboration with FHI and MOH to refine costing information Identify strategies that will make FP services more cost effective - increase CYP - decrease cost Project staff with support from FHI and FP & RH Advisor (DC) November 07 – June 09 Capacity building of local NGOs The accountants and field agents don’t have a good mastery of GP/SP/Save the Children administrative and financials procedures. Need to provide more orientation of NGO accountants and coordinators regarding SC and GP/SP’s administrative and financial management procedures Integrate administrative and financial procedures into training curricula targeting accountants/coordinators Project staff November 07 Data collection needs to be standardized at all levels especially in the field Need to train coordinators and supervisors in use of Excel to standardize data at all levels Plan training session on use of Excel for coordinators/ supervisors Project staff November 07 Financial reports are not always submitted in a timely manner Need to strengthen supervision and regular follow-up with NGO accountants Plan to conduct quarterly supervision of NGO accountants Project staff November 07- June 09 Need to ensure all staff are oriented regarding USAID FP policies Need to train NGO staff regarding USAID FP policies Train and monitor NGO staff regarding compliance with USAID FP policies Project staff November 07 Logistics Limited resources led to some activities not being implemented in the first phase Need to ensure resources are available to enable implementation of activities at field level Review current budget to ensure to sufficient funds are allocated for on the ground supervisory visits Project staff October 07 Training The method mix is limited to just a few methods and needs to be expanded Review curricula to include LAM, SDM and LAPMs Finalize MOU with IRH that outlines activities to introduce LAM and SDM Project staff November 07 Save the Children CS-20 Mali, Ségou, October 2007 102 SUMMARY FINDINGS and CONCLUSIONS RECOMMENDATIONS INITIAL ACTIONS PERSON(S) RESPONSIBLE TIME FRAME Providers have biases about providing FP methods Review curricula to ensure MEC are included Train and supervise providers to apply MEC when counseling clients about FP Project staff November 07 – Sept 09 Sustainability Strategy FP consultation free significantly influenced the use of FP services at CSCom level in Barouéli Encourage free FP consultation in all the health zones in Baroueli and Macina Negotiate with the ASACOs and district health authorities to make FP consultations free . Project staff & Coordinators November 07 – Jan 08 Stock-out of FP methods will threaten the sustainability of FP service delivery at the district level Ensure follow-up of contraceptive logistics at all levels Establish a follow-up system for the management of contraceptive methods. Apply first in first out standard Project staff/ Coordinators/ Supervisors/ Animators /district point person December07- September 09 CBD agents need to be integrated into the health system in order to ensure reports are submitted and CBD agents continue to be supervised Strengthen supervision of CBD agents by CSCom during outreach Strengthen collaboration between staff at CSCom level Coordinators/ Supervisors/ Animators December 07- June 09 Without regular supervision, FP services will not be sustained at the health zone level Strengthen supervision at all levels (CSCom, CSRef, DRS). Ensure harmonized supervision tools are made available to project staff and used Continue to advocate for funding for supervisory activities Ensure integrated supervision during other activities Project staff Project staff November 07 – Jan 08 November 07 – Jan 08 Without the presence of animators- data might not be collected from CBD agents CPM should ensure monthly data is collected and submitted by CBD agents Establish a data collection process for community based activities during Project staff Coordinators/ Supervisors/ December 07 The influence of religious leaders should be further tapped CSom and CSRef staff should involve religious leaders during the national FP campaign Target influential leaders in favor of FP to involve them in FP campaigns Coord/Supervis/ Animators/district point person/ CPM Feb – March 08 & Feb – March 09 Supervision of project activities by SC/GPSP and local NGO Save the Children CS-20 Mali, Ségou, October 2007 Save the Children 103 K. Annual Indicators Reporting Table Annual Indicators Reporting Table for Annual Reports Service Statistics Indicator Number Dates Covered Numerator Denominator Percent Confidence Interval Yes/ No Data Source/ Time Covered Total number of beneficiaries program 89663 WRA MOH KR1. Couple-years of protection (CYPs) (per year) 3247 Oct 1, 2006– Sept 30, 2007 MOH R3.1 % of facilities reporting no stockouts in injectables in the last quarter21 0 38 0% Project statistics (Oct 1, 1006 – June 30, 2007 Optional indicators Number of users new to contraception 9486 Oct 1, 2006– June 30, 2007 Population-Based Survey Indicators (core indicators in bold) – If required, to be completed for IP / first annual report, at final and each time a population-based survey is conducted. 21This information will be updated once results for the last quarter have been finalized Indicator Numerator Denominator Percent Confidence Limits (95%) Contraceptive use among WRA 48 459 10.6% 7,7% - 13,4% Unmet need for family planning 249 315 78.9% 74,4% - 83,4% Adequate birth spacing 208 300 69.3% 64,2% - 74,6% % respondents who know about at least three methods of family planning 579 600 96.6% 95,1% - 98,0% % mothers with children < 12 months who received counseling about birth spacing 73 206 35.5% 29,1% - 42,2% % of respondents of reproductive age who report discussing family planning with a health or family planning workers or promoter in the past 12 months 214 600 35.6% 31,8% - 39,4% Optional indicators Please include other indicators measured by the population-based survey that are to be used for M&E CS-20 Mali, Ségou, October 2007 Save the Children 104