Medical Care Development International Final Evaluation of the Toliara Province Child Survival Project October 1st, 2002 − September 29th, 2006 Cooperative Agreement Number: FAO-A-00-98-00027-00 Submission Date: April 23, 2007 Submitted to: Jill Boezwinkle Child Survival and Health Grants Program USAID/GH/HIDN/NUT/CSHGP 1300 Pennsylvania Avenue NW Room 3.7-74 Washington D.C. 20523-3700 Prepared by: Marc Debay, MD, MPH, PhD With other members of the Final Evaluation Team PVO Headquarters Contact: Luis Benavente Senior Health Project Officer Medical Care Development International 8401 Colesville Rd Suite 425, Silver Spring MD 20910 Phone (301) 562-1920 lbenavente@mcd.org; mcdi@mcd.org Table of Contents A. SUMMARY .............................................................................................................. 1 B. ASSESSMENT OF RESULTS AND IMPACT OF THE PROGRAM ....................... 7 1. Results: Summary Chart.................................................................................................................................... 7 2. Results: Technical Approach ........................................................................................................................... 10 a. Project Overview............................................................................................................................................. 10 b. Progress by intervention area in Toliara II...................................................................................................... 13 i. Pneumonia Case Management ..................................................................................................................... 14 ii. Malaria Control........................................................................................................................................... 15 iii. Diarrhea Diseases Control ......................................................................................................................... 17 iv. Immunization ............................................................................................................................................. 20 v. Breastfeeding and nutrition ......................................................................................................................... 23 vi. Child spacing and reproductive health ....................................................................................................... 24 c. Progress by intervention area in Betioky District............................................................................................ 27 d. New tools or approaches ................................................................................................................................. 32 i. The VISA Approach .................................................................................................................................... 32 ii. Kôminina Mendrika.................................................................................................................................... 33 iii. Health Care Financing Insurance............................................................................................................... 33 3. Results: Cross-cutting Approaches.................................................................................................................. 34 a. Community Mobilization ................................................................................................................................ 34 b. Communication for Behavior Change............................................................................................................. 40 c. Capacity Building............................................................................................................................................ 41 i. Strengthening the Grantee Organization ...................................................................................................... 41 ii. Strengthening Local Partners Organizations............................................................................................... 42 iii. Health Facilities Strengthening.................................................................................................................. 45 iv. Strengthening Health Worker Performance ............................................................................................... 49 v. Training....................................................................................................................................................... 54 d. Sustainability................................................................................................................................................... 58 4. Program Management...................................................................................................................................... 60 a. Planning........................................................................................................................................................... 60 b. MCDI Staff Training....................................................................................................................................... 61 c. MCDI Staff Supervision.................................................................................................................................. 62 d. Human Resources Management...................................................................................................................... 62 e. Financial Management .................................................................................................................................... 63 f. Logistics........................................................................................................................................................... 64 g. Information Management................................................................................................................................ 65 h. Technical and Administrative Support............................................................................................................ 68 i. Mission Collaboration...................................................................................................................................... 69 j. Management Lessons Learned......................................................................................................................... 70 C. CONCLUSIONS AND RECOMMENDATIONS ..................................................... 71 a. Success in meeting objectives ......................................................................................................................... 71 b. Main Achievements and Factors Affecting Performance................................................................................ 74 c. Best practices and lessons learned................................................................................................................... 75 d. Final Evaluation Recommendations................................................................................................................ 76 e. Dissemination of Best Practices and Lessons Learned.................................................................................... 79 f. Potential for Scale-Up...................................................................................................................................... 79 Medical Care Development International i TPCSP Final Evaluation ANNEXES..................................................................................................................... 80 Appendix 1. CSHGP Project Data Form..................................................................................................... 81 Appendix 2. Maps ....................................................................................................................................... 86 Appendix 3. TPCSP Organigram and partners............................................................................................ 87 Appendix 4. Timeline of Overall Project Implementation, October 2002 – December 2006 ..................... 88 Appendix 5. TPCSP 2006 Benchmarks ...................................................................................................... 90 Appendix 6. FE Assessment Methodology ............................................................................................... 100 Appendix 7. FE Field Work Schedule....................................................................................................... 104 Appendix 8. FE Workshops Participants................................................................................................... 105 Appendix 9. Baseline and End-of-Project Results in Betioky................................................................... 107 Appendix 10. Main BCC materials used under TPCSP ............................................................................ 108 Appendix 11. Training Events Attended by MCDI Staff.......................................................................... 109 Appendix 12. TPCSP Human Resources .................................................................................................. 111 LIST OF FIGURES Figure 1. FE Results for Pneumonia Case Management............................................................................................ 15 Figure 2. FE Results for Malaria Control................................................................................................................... 16 Figure 3. FE Results for Control of Diarrheal Diseases............................................................................................. 18 Figure 4. FE Results for Immunization...................................................................................................................... 21 Figure 5. FE Results for Breastfeeding...................................................................................................................... 23 Figure 6. FE Results for Child Spacing ..................................................................................................................... 25 Figure 7 Selected KPC survey results for CDD in Betioky and Toliara II, 1998-2006 .............................................. 29 Figure 8 Selected KPC survey results for IMM in Betioky and Toliara II, 1998-2006 .............................................. 30 Figure 9 Selected KPC survey results for BRE in Betioky and Toliara II, 1998-2006............................................... 31 Figure 10 Selected KPC survey results for CS in Betioky and Toliara II, 1998-2006................................................ 31 Figure 11 Numbers of ITNs provided by MCDI in Toliara II and Betioky in 2005 and 2006 ................................... 36 LIST OF TABLES Table 1 CSHGP grants to MCDI in Madagascar, 1996-2006..................................................................................... 10 Table 2 Demographic and health services data for Toliara II and Betioky................................................................. 11 Table 3 Timing of KPC and HFA surveys in Betioky and Toliara, 1998-2006.......................................................... 12 Table 4 Number of specific TPCSP objectives met in Toliara II and Betioky ........................................................... 13 Table 5 FE Results for TPCSP in Betioky.................................................................................................................. 28 Table 6 Health products provided by MCDI and availability at the CBD level in Toliara II and Betioky ................. 35 Table 7 Community radio products developed under TPCSP .................................................................................... 37 Table 8 Samples of BHC, HW and children in TPCSP HFAs.................................................................................... 47 Table 9 Selected indicators of services availability in Toliara and Betioky, 2006...................................................... 48 Table 10 Number of CHVs identified and supervised in FY02, FY03, and FY04 ..................................................... 51 Table 11 Number of trained CHVs supervised by quarter in Toliara II, FY 2006...................................................... 51 Table 12 Distribution of health sectors by number of quarterly supervisory visits received, Toliara II, 2006 ........... 51 Table 13 Change in indicators of health worker performance in IMCI, Toliara II and Betioky, 2002 and 2006 ....... 53 Table 14 Main training activities under TPCSP ......................................................................................................... 55 Table 15 Number of trained and functional CHVs in Toliara II in FY04................................................................... 56 Table 16 Trained and retrained CHVs by topic in Toliara II ...................................................................................... 56 Table 17 Summary of Achievements of TPCSP Objectives in Toliara and Betioky .................................................. 71 Medical Care Development International ii TPCSP Final Evaluation Acronyms ARI Acute Respiratory Infections BCC Behavior Change Communication BF Breastfeeding BFHI Baby Friendly Hospital Initiative BHC Basic Health Center BSCSP Betioky Sud Child Survival Project CBD Community Based Distributor CHV Community Health Volunteer CSTS Child Survival Technical Support DHMT District Health Management Team DHS Demographic Health Survey DIP Detailed Implementation Plan DOHFP District Office of Health and Family Planning EBF Exclusive Beast Feeding ENA Essential Nutrition Action EPI Expanded Program of Immunization FA Field Agent FP Family Planning HFA Health Facility Assessment HIS Health Information System HW Health Worker IEC Information Education Communication IMCI Integrated Management of Childhood Illness ITN Insecticide Treated Net KPC Knowledge, Practice and Coverage LOE Level of Effort LQAS Lot Quality Assurance Sampling MCDI Medical Care Development International MOHFO Ministry of Health and Family Planning MTE Midterm Evaluation NGO Non-Governmental Organization ORT Oral Rehydration Therapy PCM Pneumonia Case Management PSI Population Service International PVO Private Voluntary Organization RAD Rural Animation Department/Delegate ROHFP Regional Office of Health and Family Planning TBA Traditional Birth Attendant TOT Training of Trainers TPCSP Toliara Province Child Survival Project TT2 Tetanus Toxoid 2 VISA Visit Identify Sensitize Accompany WRA Women of Reproductive Age Medical Care Development International iii TPCSP Final Evaluation A. Summary The Toliara Province Child Survival Project (TPCSP, 2003-2006) is a cost-extension of the Betioky Child Survival Project (BSCSP, 1998-2002), which itself built on a Planning Grant project (1996-1998) awarded to MCDI to prepare its application for a first CSHGP grant. In 2006, USAID awarded MCDI an Expanded Impact grant for the Toliara Region Expanded Impact Project (TREIP, 2007-2011), which will extend the successful results achieved in Betioky and Toliara II to the nine districts of the South West Region. The Final Evaluation (FE) of TPCSP is conducted with this 10-year perspective to contribute to the planning and evaluation of TREIP. The goal of TPCSP is: “To reduce morbidity and mortality among children under 5 and to improve the health status of women of reproductive age (WRA) in the Betioky-Sud and Toliara II Districts.” Its three main strategies are to (1) strengthen capacity at the community level, (2) strengthen case management, BCC, and management/supervision at the facility and district levels; and (3) facilitate synergies with other donor partners.” TPCSP focuses on five technical interventions with similar level of effort. Four of these interventions--Control of Diarrheal Diseases, Immunization, Breastfeeding, and Child spacing-- were already implemented during BSCSP, although complementary feeding and vitamin A supplementation were added to Breastfeeding, and HIV prevention was added to Child Spacing during TPCSP. In addition, two new interventions were added during TPCSP: Pneumonia Case Management and Malaria Control. TPCSP is implemented in the two districts of Toliara II and Betioky, with a total beneficiary population 63,791 under fives and 81,510 women of reproductive age. In Betioky, TPCSP continued supporting activities implemented under BSCSP through the local NGO VEMIMA, with the primary goal of building local capacity to maintain or improve the results previously achieved and achieving the same results as in Toliara II for the new interventions. In Toliara II, TPCSP implemented the same approaches and activities as in the previous project in Betioky, with the primary goal of achieving the same or better results for the BSCSP interventions and specific new objectives for the two new interventions. The TPCSP primary partners are the Ministry of Health and Family Planning (MOHFP) at the Region (Regional Office of Health and Family Planning-ROHFP), District (District Office of Health and Family Planning-DOHFP) and Commune (Basic Health Center-BHC) levels, the Ministry of Communication and Youth, and three local NGOS (VEMIMA, Mampifoha and Miainga). Overall, TPCSP main activities were implemented according to plans. The baseline KPC and HFA surveys and the DIP workshop were completed in December 2002 and in March 2003, respectively, and the midterm KPC and HFA surveys and evaluation were conducted in February and June 2005, respectively. Most of the training of health workers (HW) was completed by the end of FY03. All new Community Health Volunteers (CHVs) in Toliara II and Betioky were trained by the end of FY05. A package of five social marketing products was launched mid￾FY05 in Toliara II and Betioky. The final KPC and HFA surveys were conducted in November and December 2006 and the final evaluation in February 2007; these activities were purposely delayed by MCDI, with USAID approval, to avoid potential interruptions by the presidential election that took place in December 2006. Medical Care Development International 1 TPCSP Final Evaluation The main TPCSP accomplishments are: Behavioral change and community health • 548 CHVs trained in communication skills, VISA approach, and TPCSP interventions • 6 Field Agents trained as CHV trainers • Modules for training CHVs in TPCSP interventions developed • BCC tools for TPCSP interventions developed and used • Implementation of the VISA approach regarding TPCSP interventions • Implementation of community-based distribution of key social marketing health products • Development and broadcasting of radio programs on TPCSP interventions • Relay of project activities to NGOs VEMIMA Mampifoha et Miainga • Support to BHC for supervision of CHVs • Support to implementation of health campaigns • Support to implementation of Kôminina Mendrika in 8 communes Quality of Care in Basic Health Centers • 11 Trainers trained in clinical IMCI • 55 HWs trained in IMCI, TPCSP interventions, supervision and training • Support to DOHFP for supervision of BHCs and HWs Technical interventions As all the TPCSP quantitative objectives in the DIP pertain to Toliara II, the criteria for meeting an objective in Toliara II is that the objective is included in the confidence interval of the 2006 KPC estimate. The criteria for meeting an objective in Betioky is that the 2006 KPC result for the related indicator in Betioky is the same or better as in Toliara II (that is, the two confidence intervals overlap or that of Betioky exceeds that of Toliara II). According to these criteria, TPCSP achieved its objectives for two thirds of project indicators in Toliara II and three quarters in Betioky. Pneumonia case management: The objectives are met for early treatment seeking in both districts, and in Toliara for knowledge of danger signs. The indicator for the IMCI clinical skills of HWs showed the same progress in both districts but did not reach the end-of-project objective. Malaria control: The use of ITNs by children under two increased in both districts, but only reached the objective in Toliara II. The correct treatment of children with fever and the preventive treatment during pregnancy increased significantly, but did not reach the objectives in Toliara. In Betioky, the former reached the same level as in Toliara (objective met) while the latter lagged behind (objective not met). The objective of the availability of antimalarials at the community level is met in Toliara and Betioky. Control Diarrheal Diseases: Almost all five indicators for this intervention increased and reached the objectives in both districts. Only the percentage of children with diarrhea given more fluids increased but did not reach the objective in Toliara. Immunization: Childhood immunization showed remarkable progress in Toliara, both in terms of increase in percentage of completely immunized children by age 1 and of decrease in dropout rates (objectives met). In Betioky, the percentage of completely immunized children remained stable but significantly lower than in Toliara, and the dropout rate increased (objectives not met). Tetanus immunization also increased in both districts, but the coverage indicator based on Medical Care Development International 2 TPCSP Final Evaluation immunization card did not reach the objective in Toliara (the rate based on mother’s report did). In Betioky, the tetanus immunization coverage reached similar levels to that in Toliara (objective met). Breastfeeding and nutrition: The three indicators of breastfeeding and nutrition reached the objectives in Toliara and in Betioky. Child spacing: The contraceptive prevalence among mothers of children under two reached the objective in both districts. The knowledge of HIV prevention and that of child spacing as a contraceptive method did not reach the objectives in Toliara. These two indicators reached the objectives in Betioky in the sense that they achieved are the same low levels as in Toliara. Community mobilization and BCC Local NGO staff and MCDI Field Agents trained, followed up and supported networks of CHVs in Toliara and Betioky, as planned. CHVs successfully implemented the new approaches developed under BSCSP, which primarily rely on interpersonal communication during home visits and the involvement of VISA mothers. TPCSP also successfully sponsored a variety of community mass mobilization activities. Capacity building Communities: TPCSP provided support to create network of CHVs effectively working with other community members, local authorities and HWs on improving health behaviors. Local NGOs: TPCSP effectively transferred implementation responsibility for BCC activities to VEMIMA in Betioky, and began working with other NGOs in Toliara II. BHCs: HWs increased HWs’ performance in areas of project interventions, in training of trainers, and in community mobilization and supervision of CHVs DHMT: The DHMTs in Toliara and Betioky increased their capacity for managing community health and mobilization, supervision of HWs and other TPCSP activities ROHFP: ROHFP staff increased their capacity in community health and mobilization and in supervision of HWs; they are well prepared and motivated for increased leadership under TREIP Sustainability TPCSP achieved its implicit sustainability objectives of: • Behavior changes at the family level • Establishment of community structures to support these changes • Availability of appropriate community health approaches and tools • Strengthened health facilities, including health workers performance • Strengthened management capacity of ROHFP and of DOHFP in Toliara and Betioky • Network of actors working synergistically in health and child survival in Toliara and Betioky Program management Successful aspects of MCDI/Madagascar management can be considered as objectives met: • Strong collaboration with the USAID mission and other partners • Inclusion of all partners in the planning and evaluation process Medical Care Development International 3 TPCSP Final Evaluation • Provision of valuable training and professional experience to project staff and good preparation for effective contribution to TREIP • Acquisition of advanced skills in the development and implementation of population- and facility-based surveys and in the management and analysis of related data • Strong human and financial resources management preparing well for the implementation of TREIP and other future health programs In conclusion: TPCSP achieved most of its objectives of improved behavior and coverage of interventions, and improved capacity to improve child health and survival of communities, local NGOs, BHC, and DHMT. In Toliara II: • All but one project indicators of behaviors and coverage of interventions significantly increased, and two thirds of the interventions achieved the project objectives • Communities have improved their capacity to promote and support healthy behaviors among mother of children under two • BHCs and the DHMT have improved their capacity to provide quality maternal and child health services and address the need of the communities they serve In Betioky: • TPCSP was able to successfully transfer most of the implementation responsibilities to the local NGO VEMIMA • In this context, most behaviors and coverage of interventions that had been improved at the end of BSCSP in 2002 were maintained or further increased; behaviors and coverage of new interventions increased to levels similar to those achieved in Toliara II The FE Team identified several factors that contributed to this success: • Excellent collaboration among partners and working atmosphere • Effective training and supervision of CHVs, HW, and DHMT members • Effective community mobilization approaches based on CHVs and VISA mothers • Effective mass mobilization through community radio programs and health campaigns • Improved availability of essential drugs, vaccines and medical equipment The FE Team also identified several constraints to better results or sustainability: • Small number and frequent replacement of health workers • Low performance of some CHVs • Enormous logistic and accessibility problems • Persistent drugs and vaccines stock outs • Low literacy of the population Medical Care Development International 4 TPCSP Final Evaluation Summary of Recommendations The following main recommendations primarily apply to TREIP: Basic Health Centers • Develop realistic plans and budget allocations to continue support to CHVs • Respect instructions regarding CHV reporting system • Strengthen active follow up of immunization dropouts ROHFP • Ensure sound distribution of health workers among BHCs • Develop strategies to motivate and retain HWs assigned to BHCs • Avoid overlap between activities and commitments with various partners • Continue institutionalization of formative integrated supervision and related tools MCDI/Madagascar -Community mobilization and behavior change communication • Evaluate the effectiveness of the training, follow-up and supervision system of CHVs • Review the community heath information system and develop clear reporting instructions • Complement interpersonal BCC approaches with mass communication approaches -Quality of care • Ensure active participation of DHMTs and BHC HWs in planning activities • Include a community component and involve all partners in the elaboration of plans for quality improvement in health services -Program management • Ensure active involvement of BHC and local authorities in planning and implementation • Adopt and implement explicit organizational development plans and partnership agreements for each local NGO under TREIP. Develop the necessary expertise within MCDI. • Conduct training needs assessment before developing new training plans • Prepare summaries and translation of project documents, including the TREIP DIP, to ensure easy reading and use by all MCDI and partners’ staff • Strengthen communication of results through preparation of final technical reports and publications, and their dissemination to all partners • Compile all valuable training, BCC and HIS materials successfully used during TPCSP, and prepare technical descriptions of their purpose and recommendations for further use • Undertake a study of the actual CHV training, incentives, and all other costs and of the related sustainability issues before scaling-up such program • Prepare and disseminate technical documentation of success stories, best practices, lessons learned and technical materials For MCDI US: • Provide sustained and documented technical assistance for strategic planning and specific areas of identified needs to maximize project results and build capacity of Field Office Medical Care Development International 5 TPCSP Final Evaluation Medical Care Development International 6 TPCSP Final Evaluation • Plan staff training and development on the basis of needs assessments; include technical follow up to ensure the transfer of learning B. Assessment of Results and Impact of the Program 1. Results: Summary Chart DIP Objective / Indicator Data FE Source BL MTE Est. UCL LCL TPCSP Objective Objective met PNEUMONIA CASE MANAGEMENT (LOE 15%) PCM1: Increase from 15% to 45% the percentage of mothers with children 0- 23 months with fast/difficult breathing during the last two weeks who sought treatment from a health facility by the end of the day KPC 15 19 56 65 47 45 YES PCM2: Increase from 25% to 65% the percentage of mothers with children 0- 23 months who can cite at least two danger signs of pneumonia as a reason to seek treatment KPC 25 45 63 69 58 65 YES PCM3: Increase from 0% to 60% the percentage of clinical staff who use correctly IMCI protocols HFA 0 27 31 N/A N/A 60 NO MALARIA (LOE 15%) MAL1: Increase from 3% to 20% the percentage of children 0-23 months who slept under an insecticide-treated bed net the previous night KPC 3 7 58 64 52 20 YES MAL2: Increase from 13% to 50% the percentage of mothers of children 0-23 months with a febrile episode ending during the last two weeks who give correct treatment at home KPC 13 25 36 47 24 50 NO MAL3: Increase from 31% to 80% the percentage of mothers who took anti￾malarial medicine to prevent malaria during pregnancy KPC 31 21 72 77 67 80 NO MAL4: Increase from 0% to 75% the percentage of CBDs with no stock-outs of anti-malarial medicines during the preceding 6 months FA 0 50 92 N/A N/A 75 YES Medical Care Development International 7 TPCSP Final Evaluation DIP Objective / Indicator Data FE Source BL MTE Est. UCL LCL TPCSP Objective Objective met CONTROL OF DIARRHEAL DISEASE (LOE 15%) CDD1: Increase from 34% to 65% the percentage of children 0-23 months who had diarrhea in the past two weeks who were given more than the usual amount of fluids during a diarrheal episode KPC 34 58 52 59 45 65 NO CDD2: Increase from 24% to 55% the percentage of children 0-23 months who had diarrhea in the past two weeks who were given the same or more than the usual amount of foods during the diarrheal episode KPC 24 64 54 61 47 55 YES CDD3: Increase from 36% to 65% the percentage of children 0-23 months who had diarrhea in the past two weeks who were given the same or more than the usual amount of breast milk during a diarrheal episode KPC 36 60 66 72 60 65 YES CDD4: Increase from 52% to 80% the percentage of children 0-23 months who had diarrhea in the past two weeks and whose mothers sought outside advice or treatment for the illness within 24 hours of the first sign of danger KPC 52 55 74 84 65 80 YES CDD5: Increase from 33% to 65% the percentage of mothers of children 0-23 months who can cite at least 2 danger signs for diarrhea as a reason to seek advice or treatment at a health facility KPC 33 45 64 70 59 65 YES IMMUNIZATION (LOE 20%) IMM1: Increase from 30% to 60% the percentage of children 12-23 months who are fully immunized per the vaccination card KPC 30 52 71 76 66 60 YES IMM2: Increase from 34% to 65% the percentage of mothers who received at least two tetanus toxoid (TT) injections before the birth of their youngest child [per the vaccination card] KPC 34 20 50 58 42 65 NO IMM3: Reduce from 21% to 10% or less the percentage of children 12-23 months who default between the DPT1 and DPT3 doses KPC 21 N/A 10 15 5 10 YES BREASTFEEDING AND NUTRITION (LOE 15%) BRE1: Increase from 2% to 35% the percentage of children age 0-5 months who are exclusively breastfed KPC 2 34 41 46 35 35 YES BRE2: Increase from 24% to 55% the percentage of mothers who initiate breastfeeding within one hour after giving birth KPC 24 42 54 60 49 55 YES BRE3: Increase from 19% to 50% the percentage of children 12-23 months who receive 5 or more feeds per day (meals and snacks) in addition to breastfeeding KPC 19 36 48 53 42 50 YES Medical Care Development International 8 TPCSP Final Evaluation Medical Care Development International 9 TPCSP Final Evaluation DIP Objective / Indicator Data FE Source BL MTE Est. UCL LCL TPCSP Objective Objective met CHILD SPACING AND REPRODUCTIVE HEALTH (LOE 20%) CS1: Increase from 9% to 25% the percentage of mothers who are not pregnant, do not want another child in the next two years or are not sure, and are using a modern method of contraception KPC 9 22 24 29 19 25 YES CS2: Increase from 21% to 60% the percentage of women [mothers of 0-23 months children] who can cite at least two ways to reduce the risk of Human Immunodeficiency Virus (HIV) infection KPC 21 22 21 26 16 60 NO CS3: Increase from 0% to 75% the percentage CBDs with no stock-outs of condoms FA 0 36 88 N/A N/A 75 YES CS4: Increase from 1%to 40% the percentage of mothers who know of exclusive breastfeeding as a child spacing method KPC 1 4 4 6 2 40 NO TOTAL PROJECT OBJECTIVES MET: 15 TOTAL PROJECT OBJECTIVES: 22 % 68% Source: BL MTE FE KPC Report on KPC Baseline Survey 2002 February 2003 Evaluation à mi-parcours. Rapport de l’Enquête de Suivi CAP, Novembre 2004 Résumé des résultats de l’enquête finale 2006 – SSD Toliara II, Février 2007 HFA MTE Report, September 2005 MTE Report, September 2005 MCDI HIS Manager, February 2007 FA MTE Report, September 2005 MTE Report, September 2005 MCDI BCC Manager, February 2007 Note: 1. BL: Baseline; MTE: Midterm evaluation; FE: Final evaluation: Est.: Survey estimate; UCL: Upper Confidence Limit; LCL: Lower Confidence Limit; KPC: Knowledge, Practice and Coverage survey; HFA: Health Facility Assessment; FA: Field Agent supervision report. 2. MTE KPC survey based on LQAS methodology. 3. Objective is met when included in the reported confidence interval of the 2006 KPC survey estimate. 2. Results: Technical Approach a. Project Overview The Toliara Province Child Survival Project (TPCSP) is a cost-extension of the Betioky Child Survival Project (1998-2002), which itself built on a Planning Grant project (1996-1998) awarded to MCDI to prepare its application for a first CSHGP grant. Table 1 shows the main features of these three successive grants and that of the Expanded Impact grant awarded to MCDI in 2006 to extend the successful results achieved in Betioky and Toliara II to the nine districts of the South West Region (Toliara Region Expanded Impact Project—TREIP). The Final Evaluation (FE) of the TPCSP is conducted with this 10-year perspective to contribute to the planning and evaluation of TREIP. Appendix 2 shows the map of Madagascar with the in two districts of Betioky and Toliara II and the South West Region. Table 1 CSHGP grants to MCDI in Madagascar, 1996-2006 Phase 1 Phase 2 Phase 3 Phase 4 Period 1996-1997 1998-2002 2003-2006 2007-2011 Type of Grant Planning Grant Standard Grant Cost Extension Expanded Impact Amount: USAID: MCDI: - - $ 996.739 - $1,229,843 - $2,499,986 - Name - Betioky Child Survival Project (BCSP) Toliara Province Child Survival Project (TPCSP) Toliara Region Expanded Impact Project (TREIP) Beneficiary population - 33,768 under fives 39,397 WRA 63,791 under fives 81,510 WRA 178,400 under fives 256,450 WRA Location Betioky District Betioky District Betioky and Toliara II Districts South West Region (9 districts) Interventions and level of effort -- -- CDD IMM BRE CS PCM (15%) MAL (15%) CDD (15%) IMM (15%) BRE/NUT (15%) CS/RH (20%) -- MAL (30%) CDD (30%) IMM (20%) BRE (20%) -- Note: BRE: Breastfeeding; CDD: Control of Diarrheal Diseases; CS: Child Spacing; IMM: Immunizations; MAL: Malaria; NUT: Nutrition; PCM: Pneumonia Case Management; RH: Reproductive Health Source: Phase 2: BSCSP FE report; Phase 3: DIP, except for beneficiary population as revised in TPCSP Annual Report 2004; Phase 4: TREIP Proposal, 2005 The goal of the TPCSP is: “To reduce morbidity and mortality among children under 5 and to improve the health status of women of reproductive age (WRA) in the Betioky-Sud and Toliara II Districts.” The three main strategies of the TPCSP as defined in the DIP are: (i) “Strengthening capacity at the community level through the development of sustainable community institutions, networks, and community-based personnel; (ii) Institutional strengthening of MOH by improving the case management, BCC, and management/supervision skills of facility and district health personnel; and (iii) Promotion and facilitation of synergies with other donor partners on training modules, equipment and materials, and the exchange of technical knowledge and lessons learned.” Medical Care Development International 10 TPCSP Final Evaluation TPCSP focuses on five technical interventions with similar level of effort (see Table 1). Four of these interventions--Control of Diarrheal Diseases (CDD), Immunization (IMM), Breastfeeding (BRE), and Child spacing (CS)--were already implemented during BCSP, although nutrition (complementary feeding and vitamin A supplementation) was added to Breastfeeding, and HIV prevention was added to Child Spacing during TPCSP. Two new interventions, Pneumonia Case Management (PCM) and Malaria Control (MAL), were added during TPCSP. The TPCSP primary partners are the Ministry of Health and Family Planning (MOHFP) at the Region (Regional Office of Health and Family Planning-ROHFP), District (District Office of Health and Family Planning-DOHFP) and Commune (Basic Health Center-BHC) levels, the Ministry of Communication and Youth, and three local NGOS (VEMIMA, Mampifoha and Miainga). Other TPCSP implementation partners include SantéNet, PSI, WHO, UNICEF, UNFPA and the USAID mission in Madagascar. Appendix 2 shows the TPCSP organigram including MDCI staff and the relationships with all the institutional and community partners. TPCSP is implemented in the two districts of Toliara II and Betioky. In Betioky, TPCSP continued supporting activities implemented under BSCSP through the local NGO VEMIMA, with the primary goal of building local capacity to maintain the results. In Toliara II, TPCSP implemented the same approaches and activities as under BSCSP with the goal of achieving the same results as in Betioky. In addition, in both districts TPCSP introduced interventions that had not been implemented in the previous project, and ensured that new health workers and community health workers were trained and included in project activities. Table 2 shows the key demographic and health services data of the two project districts. Table 2 Demographic and health services data for Toliara II and Betioky Characteristics Toliara II Betioky Total Total population 207,459 188,084 395,543 Population < 5km 61,465 81497 142,962 Fokontanys 267 289 556 Communes 24 27 51 Health Sectors 33 26 59 BHC 33 26 59 Physicians 13 11 26 Midwives 6 5 11 Nurses 10 4 14 Health aids 10 9 19 Source: BHC data by district, ROHFP, 2006 Note: The number of BHC and health workers in Toliara and Betioky varies in the various TPCSP documents. The numbers above are used throughout this report unless indicated otherwise Appendix 4 presents a timeline of the TPCSP implementation. The baseline KPC and HFA surveys and the DIP workshop were completed in December 2002 and in March 2003, respectively, and the midterm KPC and HFA surveys and evaluation were conducted in February and June 2005, respectively. Most of the training of the District Health Management Team (DHMT) and Health Workers (HW) of the Basic Health Centers (BHC) was completed by the end of FY03. This included Training of Trainers (ToT), Essential Nutrition Actions, Expanded Program on Immunization, Family Planning and IMCI. All new CHV in Toliara II and Betioky received their first round of training during FY04 (ENA/EPI/FP/IMCI/Visa approach) and their Medical Care Development International 11 TPCSP Final Evaluation second round of training in FY05 (Malaria and HIV/AIDS). The complete package of five social marketing products was launched mid-FY05. The main TPCSP accomplishments are: Behavioral change and community health • 548 CHVs trained in communication skills, VISA approach, and TPCSP interventions • 6 Field Agent trained as CHV trainers • Modules for training CHVs in TPCSP interventions developed • BCC tools for TPCSP interventions developed and used • Implementation of the VISA approach regarding TPCSP interventions • Implementation of community-based distribution of key social marketing health products • Development and broadcasting of radio programs on TPCSP interventions • Relay of project activities to NGOs VEMIMA Mampifoha et Miainga • Support to BHC for supervision of CHVs • Support to implementation of health campaigns • Support to implementation of Kôminina Mendrika in 8 communes Quality of Care in Basic Health Centers • 11 Trainers trained in clinical IMCI • 55 HWs trained in IMCI, TPCSP interventions, supervision and training • Support to DOHFP for supervision of BHCs and HWs MCDI conducted a regular series of KPC and HFA surveys since the beginning of the CSHGP funding of projects in Betioky. Table 3 shows the timing of each type of survey conducted in Betioky and Toliara II between 1998 and 2006. All KPC surveys followed the KPC+ methodology and used a sample size of about 300 mothers of children under two, except for the 2004 MTE survey in Toliara II, which used Lot Quality Assurance Sampling with total sample of size of 133 (19 mothers in 7 health sectors). Focus Group Discussions were conducted with mothers in each cluster of the KPC surveys starting in 2004. Although many indicators changed from BSCSP to TPCSP, a series of indicators are available in Betioky from 1998 to 2006, and are used in section B.2.c on TPCSP Progress in Betioky. The HFA surveys primarily consisted in IMCI clinical skills of HW and in availability of services. More details on the changes that occurred in this series of HFAs and on the progression to the Integrated Supervision Tool in 2006 are provided in section B.3.c.iii. Table 3 Timing of KPC and HFA surveys in Betioky and Toliara, 1998-2006 1998 2000 2002 2004 2006 Betioky KPC x x x x HFA x x Integrated supervision x Toliara KPC x x x HFA x x x Integrated supervision x The FE KPC and HFA surveys were conducted in November and December 2006 and the final evaluation in February 2007; these activities were purposely delayed by MCDI, with USAID Medical Care Development International 12 TPCSP Final Evaluation approval, to avoid potential interruptions by the presidential election that took place in December 2006. The KPC and HFA results for the key project indicators in Toliara II from 2002 to 2006 are presented in section B.1 (Results: Summary Chart), and discussed for each intervention in section B.2.b (Progress by intervention area in Toliara II). Data from surveys in Betioky from 1998 to 2006, and progress in comparison to that in Toliara II, are presented and discussed section B.2.c (Progress by intervention area in Betioky). Table 4 presents a quick overview of TPCSP achievements in terms of number of objectives met per interventions. The criteria to determine whether a TPCSP has been met in Toliara II or in Betioky are different, and are explained in the respective sections on Progress in Toliara II (section B.2.b) and in Betioky (section B.2.c). Table 4 Number of specific TPCSP objectives met in Toliara II and Betioky Interventions Project Objective Toliara II Betioky PCM 3 2 2 MAL 4 2 2 DIAR 5 4 5 IMM 3 2 1 BRE 3 3 3 CS 4 2 4 Total : 22 15 17 % 100% 68% 77% The fieldwork of TPCSP FE took place between February 11 and 26, 2007 and involved a total of 40 participants. The methodology, schedule and lists of participants of the FE are in Appendix 6, Appendix 7 and Appendix 8, respectively. b. Progress by intervention area in Toliara II The five sections below discuss the TPCSP technical interventions starting with a brief description of the main strategies followed with charts and a discussion of the KPC or HFA surveys results. Each section then continues with a discussion of the factors of success and constraints, the lessons learned and special outcomes, and the plans for scaling up. All conclusions and recommendations specific to the technical interventions are in the Conclusions and Recommendations section (see section C). The series of charts below represent the results of the three KPC surveys or other sources of data in 2002 (baseline), 2004 (midterm) and 2006 (end-of-project). The 2006 KPC results on these charts include the confidence intervals of the survey estimate1 and the end-of-project objective. For the purpose of this report, an objective is considered achieved if it is included in the confidence interval of the survey estimate. 1 All estimates and confidence interval are from the “Résumé des résultats de l’enquête finale 2006 – SSD Toliara II,” February 2006, a preliminary document made available to the FE Team. Confidence intervals in this document do not take into account the cluster sampling design of the 2006 KPC survey and therefore are underestimated. Medical Care Development International 13 TPCSP Final Evaluation i. Pneumonia Case Management Approach summary (as per the DIP): • Improving family and community practices • Improving skills of health facility workers • Improving health system support Achievement of objectives The two PCM behavioral and knowledge indicators have increased significantly. The percentage of mothers with children under two who could mention at least two danger signs of pneumonia steadily increased from 25% in the baseline to 63% in the FE KPC surveys, achieving the 65% target for this indicator. The percentage of mothers with children under two who had symptoms of pneumonia (cough and rapid or difficult breathing) who sought treatment for their child in a health facility within the next day increased from 15% in the baseline to 56% in the FE KPC surveys, thereby achieving the 45% target for this indicator. The “percentage of clinical staff who uses correctly IMCI protocols” increased from 0% to 27% at midterm and 31% at the end of the project, but did not reach the 60% objective. This indicator is a complex construction based on the satisfaction of several criteria of correct evaluation, classification and treatment of the three main symptoms of child illnesses (fever, cough and diarrhea).2 Given the high number of criteria used for this indicator, the 60% objective may be ambitious. Nevertheless, the increase of this indicator shortly after the IMCI training of all HW and its maintenance at the same level until the end of the project probably reflect true and sustained improvements in health worker performance in IMCI. 2 Results for this indicator are not available in the draft HFA reports 2004 and 2006, but the results for a series of indicators used to calculate this indicator are available. These are discussed in section B.3.c.iv. Medical Care Development International 14 TPCSP Final Evaluation Figure 1. FE Results for Pneumonia Case Management Early treatment seeking for rapid/difficult breathing Knowledge of at least two danger signs 11% 19% 56% 45% 0% 20% 40% 60% 80% 100% 2002 2004 2006 25% 45% 63% 65% 0% 20% 40% 60% 80% 100% 2002 2004 2006 Objective: PCM1: Increase from 11% to 45% the percentage of mothers with children 0-23 months with fast/difficult breathing during the last two weeks who sought treatment from a health facility by the end of the day Objective: PCM2: Increase from 25% to 65% the percentage of mothers with children 0-23 months who can cite at least two danger signs of pneumonia as a reason to seek treatment IMCI clinical skills of health workers 0% 27% 31% 60% 0% 20% 40% 60% 80% 100% 2002 2004 2006 Objective: PCM3: Increase from 0% to 60% the percentage of clinical staff who use correctly IMCI protocols The factors of success, Constraints and Lessons learned for the three TPCSP Childhood Illness interventions (PCM, CDD and MAL) are discussed in section B.2.b.iii. ii. Malaria Control Approach summary (as per the DIP): • Improving access to quality care • Improving care seeking behavior • Promoting malaria prevention among children and pregnant women Achievement of objectives Although the three population-level indicators for Malaria Control increased, only one of the corresponding objectives is met. The percentage of children under two who slept under an ITN the previous night increased from 3% at baseline to 58% at the end of the project, thereby Medical Care Development International 15 TPCSP Final Evaluation overcoming by far the 20% project objective. Most of this increase in ITN use happened in the last two years of the project. The percentage of households that own an ITN reached 74% at the end of the project. TPCSP’s support to the distribution on ITNs in Toliara II is discussed in section B.3.a. The two objectives for malaria case management and antenatal care are not met. The percentage of mothers of children under two with a febrile episode ending in the last two weeks who gave a correct treatment at home nearly tripled from 13% at baseline to 36% at the end of the project, but this remains short of the 50% objective. The percentage of mothers who took anti-malarial medicines during pregnancy more than doubled from 31% at baseline to 72% at the end of the project, but this remains short of the 80% project objective. In 2006, the MCDI Field Agents (FA) reported that 92% of CBDs had stocks of antimalarials during their supervision visits in October or November 2006 (see section B.3.a). Although this indicator is based on one observation at one point in time instead of during a six-month period, this objective is considered as met. Figure 2. FE Results for Malaria Control Children sleeping under an ITN Mothers who give correct treatment for fever 3% 7% 58% 20% 0% 20% 40% 60% 80% 100% 2002 2004 2006 13% 25% 36% 50% 0% 20% 40% 60% 80% 100% 2002 2004 2006 Objective: MAL1: Increase from 3% to 20% the percentage of children 0-23 months who slept under an insecticide￾treated bed net the previous night Objective: MAL2: Increase from 13% to 50% the percentage of mothers of children 0-23 months with a febrile episode ending during the last two weeks who give correct treatment at home Medical Care Development International 16 TPCSP Final Evaluation Mothers who took antimalarials during pregnancy CBDs with no stock-outs of antimalarials 31% 21% 72% 80% 0% 20% 40% 60% 80% 100% 2002 2004 2006 0% 50% 92% 75% 0% 20% 40% 60% 80% 100% 2002 2004 2006 Objective: MAL3: Increase from 31% to 80% the percentage of mothers who took anti-malarial medicine to prevent malaria during pregnancy Objective: MAL4: Increase from 0% to 75% the percentage of CBDs with no stock-outs of anti-malarial medicines during the preceding 6 months The factors of success, Constraints and Lessons learned for the three TPCSP Childhood Illness interventions (PCM, CDD and MAL) are discussed in section B.2.b.iii. iii. Diarrhea Diseases Control Approach summary (as per the DIP): • Improving skills of health facility workers • Improving health system support • Improving family and community practices Achievement of objectives Four of the five project objectives for CDD are met. The two other project objectives for home management of diarrhea are met. The percentage of children under two who had diarrhea in the last two weeks and who were given the same or more than usual amount of food increased from 24% at baseline to 54% at the end of the project, thereby meeting the 55% objective. The percentage of children under two who had diarrhea in the last two weeks and who were given the same or more than usual amount of breast milk increased from 36% at baseline to 66% at the end of the project, thereby also meeting the 65% objective. However, the percentage of children under two who had diarrhea in the last two weeks and who were given more than the usual amount of fluids increased from 34% at baseline to 52% at the end of the project, but remained short of the 65% project objective. The two objectives for early treatment seeking for diarrhea also are met. The percentage of children under two who had diarrhea in the last two weeks and whose mothers sought outside help within 24 hours of the first danger sign increased from 52% at baseline to 74% at the end of the project, thereby meeting the 80% project objective. The percentage of mothers under two who can cite at least two danger signs for diarrhea increased from 33% at baseline to 64% at the end of the project, thereby meeting the 65% project objective. Medical Care Development International 17 TPCSP Final Evaluation Figure 3. FE Results for Control of Diarrheal Diseases Children with diarrhea who are given more fluids Children with diarrhea who are given same or more food 34% 58% 52% 65% 0% 20% 40% 60% 80% 100% 2002 2004 2006 24% 64% 54% 55% 0% 20% 40% 60% 80% 100% 2002 2004 2006 Objective: CDD1: Increase from 34% to 65% the percentage of children 0-23 months who had diarrhea in the past two weeks who were given more than the usual amount of fluids during a diarrheal episode Objective: CDD2: Increase from 24% to 55% the percentage of children 0-23 months who had diarrhea in the past two weeks who were given the same or more than the usual amount of foods during the diarrheal episode Children with diarrhea who are given same or more breast milk Early treatment seeking for children with diarrhea 36% 60% 66% 65% 0% 20% 40% 60% 80% 100% 2002 2004 2006 52% 55% 74% 80% 0% 20% 40% 60% 80% 100% 2002 2004 2006 Objective: CDD3: Increase from 36% to 65% the percentage of children 0-23 months who had diarrhea in the past two weeks who were given the same or more than the usual amount of breast milk during a diarrheal episode Objective: CDD4: Increase from 52% to 80% the percentage of children 0-23 months who had diarrhea in the past two weeks and whose mothers sought outside advice or treatment for the illness within 24 hours of the first sign of danger Medical Care Development International 18 TPCSP Final Evaluation Mothers who can cite at least 2 danger signs of diarrhea 33% 45% 64% 65% 0% 20% 40% 60% 80% 100% 2002 2004 2006 Objective: CDD5: Increase from 33% to 65% the percentage of mothers of children 0-23 months who can cite at least 2 danger signs for diarrhea as a reason to seek advice or treatment at a health facility The FE Team identified the following factors of success, constraints and lessons learned for the TPCSP Childhood Illness interventions (PCM, CDD and MAL) in Toliara: Factors of success • There is a serene, convivial and warm working atmosphere among all TPCSP partners • Collaboration among partners is effective • Training of the following agents took place in a timely manner: - 9 DHMT members and 2 IMCI officers of the ROHFP were trained in IMCI management, and in community health and mobilization, integrated supervision, and development and production of BCC materials - 31 HW were trained in IMCI - 6 Field Agents were trained in IMCI and in monitoring/supervision of CHVs - 441 CHVs were trained in case management of childhood illnesses (PCM/MAL/CDD) according to the national IMCI protocol • Tools for supervision of HW performance and BHC are available, and HW are supervised • The VISA approach is implemented by 402 CHV • 36 radio products on IMCI developed and broadcasted • Most BHCs have adequate supply of essential drugs for IMCI • 185 CBDs are trained and functional, and have social marketing products available • TPCSP increased social marketing of ITNs after MTE Constraints • Most BHCs only have one health care provider, and therefore are closed whenever that provider travels • Some CHVs do not have satisfactory performance levels • The logistical support is very limited given the challenges of poor geographical accessibility • Essential drugs stock outs still exist Medical Care Development International 19 TPCSP Final Evaluation • ROHFP staff members are not always available for project activities because of conflicting commitments Lessons learned • MOHFP, NGOs and MCDI staff now know how to implement the VISA approach • BCC approaches such as Home Visits, General Assemblies, and FGDs are effective • Contribution of local NGOs is very important for the implementation and sustainability of BCC activities • Collaboration of partners’ (decentralized government services, NGOs, HWs, CHVs, and communities) is very important • Capacity building activities of partners (NGO, HWs, CHVs) has positive effects iv. Immunization Approach summary (as per the DIP): • Improving the availability of immunization services • Improving the quality of immunization services • Improving people’s behavior to seek immunization services • Strengthening the immunization monitoring system Achievement of objectives Two of three immunization objectives are met in Toliara II. The coverage of childhood immunization show impressive increases over the four-year project implementation period. The percentage of children 12-23 months of age who are fully immunized (as verified by examination of the immunization card at the time of the survey) doubled from 30% at baseline to 71% at the end of the project, which is far higher than the project objective. The percentage of mothers with a child less than 2 years of age who could show a record of TT2 immunization at the time of the survey increased from 34% to 50% during the same period, thereby remaining short of the project objective. The DPT1/DT3 dropout rate declined from 21% at baseline to 10% at the end of the project, which was the project objective. Medical Care Development International 20 TPCSP Final Evaluation Figure 4. FE Results for Immunization Children 12-23 months completely immunized Mothers who received at least two doses of TT during last pregnancy 30% 52% 71% 60% 0% 20% 40% 60% 80% 100% 2002 2004 2006 34% 20% 50% 65% 0% 20% 40% 60% 80% 100% 2002 2004 2006 Objective: IMM1: Increase from 30% to 60% the percentage of children 12-23 months who are fully immunized per the vaccination card Objective: IMM2: Increase from 34% to 65% the percentage of mothers who received at least two tetanus toxoid (TT) injections before the birth of their youngest child [per the vaccination card] Immunization drop out rates 21% 0% 10% 10% 0% 20% 40% 60% 80% 100% 2002 2004 2006 Objective: IMM3: Reduce from 21% to 10% or less the percentage of children 12-23 months who default between the DPT1 and DPT3 doses The FE Team identified the following factors of success, constraints and lessons learned for the IMM intervention in Toliara: Factors of success • The cold chain is functional and vaccines and management tools are available in most BHCs: o In Toliara II, 61% (14/23) of supervised BHCs have a functional refrigerator (2006 Integrated Supervision) • HW in BHCs have improved their capacity to deliver and manage immunization services: o 60 HW are trained in EPI and 60 HW are updated on management of vaccines • The DHMT has conducted outreach immunization activities in selected areas • Local NGOs and CHVs have improved their capacity to promote and mobilize community for immunization: Medical Care Development International 21 TPCSP Final Evaluation o In Toliara II, 458 CHV were trained in EPI, and 65% of CHVs practice the VISA approach to promote EPI (BCC Manager, February 2006) o CHVs regularly mobilize communities through home visits, interpersonal communication, local radio and listening groups, Focus Group Discussion, Community General Assemblies. o CHVs collaborate with BHCs staff to identify mothers who are not up-to-date with their child’s immunization according to the immunization registers, and then make home visits to encourage them to bring their child to the next immunization sessions. • The MOHPF, with support from MCDI, conducted three mass immunization campaigns since October 2005 and launched the first national “Semaine de Santé pour la Mère et l’Enfant” (SSME) in October 2006 (SSME will be held biannually in the future). • The Kôminina Mendrika approach was launched in 4 communes in Toliara II Constraints • Children and pregnant women usually do not receive immunization cards during mass campaigns. o The TT2 immunization status according to the declaration of mothers of children under two in Toliara II shows an increase in coverage from 50% at baseline to 82% at the end of the project, that is, a coverage estimate largely higher than that based on immunization cards and than the TPCSP objective for that intervention • There has been frequent stock outs of immunization cards until 2006 • There are only 1 or 2 HWs per BHC, and only a few BHCs with a physician • There still are problems with the cold chain and occasional vaccines stock outs o Only 52% of BHC have a stock of all antigens (2006 Integrated Supervision) • Mothers often lose their child’s immunization card and their own • Measles immunization is often delayed or missed because of the minimum number of children who have to be present before a vial is open • Mothers still have low literacy levels and poor knowledge about immunization, which makes it difficult for them to come to immunization services and follow the immunization schedule • Pregnant women and mothers of children under two still have problems to access BHCs from remote villages • Some BHCs have major accessibility problem to implement outreach and mobile immunization strategies during the rainy season (December to March) • Part of the population is migrant and CHVs cannot follow them up • There are security problems in some areas that make it difficult to implement outreach and mobile immunization strategies Lessons learned • To reach high immunization coverage, it is necessary to: o Actively search for dropouts o Implement the outreach strategy o Supervise HW in BHC (by DHMT) and supervise DHMT (by ROHFP) o Ensure close collaboration between BHC and CHV o Use immunization cards o Take into account literacy level when communicating messages or immunization schedule or appointment Medical Care Development International 22 TPCSP Final Evaluation • The implementation of Kôminina Mendrika is a promising strategy to increase immunization coverage • Mass immunization campaigns and SSME are effective strategies to increase immunization coverage v. Breastfeeding and nutrition Approach summary (as per the DIP): • Improving family and community practices • Improving skills of health workers Achievement of objectives The three project objectives are met in Toliara II. The percentage of children 0-5 months of age who are exclusively breastfed increased from 2% at baseline to 41% at the end of the project. The percentage of mothers who initiate breastfeeding within one hour after giving birth increased from 24% at baseline to 54% at the end of the project. The practice of complementary feeding also improved, with the percentage of children 12-23 months of age who receive 5 or more feed per day in addition to breastfeeding increased from 19% at baseline to 48% at the end of the project. Figure 5. FE Results for Breastfeeding Exclusive breastfeeding Early initiation of breastfeeding 2% 34% 41% 35% 0% 20% 40% 60% 80% 100% 2002 2004 2006 24% 42% 54% 55% 0% 20% 40% 60% 80% 100% 2002 2004 2006 Objective: BRE1: Increase from 2% to 35% the percentage of children age 0-5 months who are exclusively breastfed Objective: BRE2: Increase from 24% to 55% the percentage of mothers who initiate breastfeeding within one hour after giving birth Medical Care Development International 23 TPCSP Final Evaluation Complementary feeding 19% 36% 48% 50% 0% 20% 40% 60% 80% 100% 2002 2004 2006 Objective: BRE3: Increase from 19% to 50% the percentage of children 12-23 months who receive 5 or more feeds per day (meals and snacks) in addition to breastfeeding The FE Team identified the following factors of success, constraints and lessons learned for the BRE intervention in Toliara: Factors of success • 472 CHV trained in ENA and 86% retrained • Regular community mobilization and education by CHVs and FA in all villages • Regular educational radio spots followed by facilitation of listening groups • Implementation of the VISA approach, which gives direct example of successful applications of recommended breastfeeding and nutrition practices. • Capacity building of NGOs on breastfeeding and nutrition • 60 HW trained and retrained in ENA • Combination of breastfeeding education conducted in BHCs, followed by home visits by CHVs to reinforce the messages and practices Constraints • Cultural habits and traditions that do not favor EBF and proper infant and child nutrition, often emphasized by grandmothers • Poor understanding of the key BCC message by mothers Lessons learned • The role of grandmothers needs to be taken into account in the BCC approaches to breastfeeding promotion • SSME and health festivals are effective • Home visits and witnessing are effective vi. Child spacing and reproductive health Approach summary (as per the DIP): • Better access to quality family planning • Improving the use of child spacing services Medical Care Development International 24 TPCSP Final Evaluation Achievement of objectives Only two of the four project objectives for child spacing and reproductive health are met. Contraceptive prevalence among mothers of children under two who are not pregnant and do not want another child in the next two years or are not sure increased from 9% at baseline to 24% at the end of the project, thereby meeting the 25% objective. The percentage mothers of children under two who can cite at least two ways to prevent HIV infection had not increased at midterm, and still remain at 21% at the end of the project. This far below the 60% project objective. Although breastfeeding practices improved during the project period, only 4% of mothers of children under two know of exclusive breastfeeding as a child spacing method; the objective to increase this percentage to 40% is not met. In 2006, the FA reported that 88% of CBDs trained and supervised had stocks of condoms in the last 6 months. Although this indicator is slightly different from the one in the DIP, the corresponding 75% objective can be considered as met Figure 6. FE Results for Child Spacing Contraceptive prevalence among mothers of children under two Knowledge of HIV prevention 9% 22% 24% 25% 0% 20% 40% 60% 80% 100% 2002 2004 2006 21% 22% 21% 60% 0% 20% 40% 60% 80% 100% 2002 2004 2006 Objective: CS1: Increase from 9% to 25% the percentage of mothers who are not pregnant, do not want another child in the next two years or are not sure, and are using a modern method of contraception Objective: CS2: Increase from 21% to 60% the percentage of women [mothers of 0-23 months children] who can cite at least two ways to reduce the risk of Human Immunodeficiency Virus (HIV) infection Medical Care Development International 25 TPCSP Final Evaluation CBDs with no condom stock-outs Knowledge of exclusive breastfeeding as child spacing method 0% 36% 78% 75% 0% 20% 40% 60% 80% 100% 2002 2004 2006 1% 4% 4% 40% 0% 20% 40% 60% 80% 100% 2002 2004 2006 Objective: CS3: Increase from 0% to 75% the percentage CBDs with no stock-outs of condoms Objective: CS4: Increase from 1% to 40% the percentage of mothers who know of exclusive breastfeeding as a child spacing method The FE Team identified the following factors of success, constraints and lessons learned for the CS intervention in Toliara: Factors of success • 503 CHVs are trained in FP promotion • 100% of HW are trained in FP • 100% of BHC are adequately equipped for FP services • The VISA approach is effective in introducing and supporting mothers of children under two in the use of contraception • The various health festivals and the October 2006 SSME have helped in the promotion and adoption of family planning • Contraceptives are availability in most BHC Constraints CS: • Some cultural traditions and habits do not favor Child Spacing, and are often reinforced by grandmothers (for instance, delay of 3 months or more for first postpartum visit) • Poor understanding of the key BCC message by mothers • There is only one song on Lactational Amenorrhea Method among the 123 radio products developed by MCDI HIV/AIDS: • Poor understanding of the key BCC message by mothers • Insufficient training and activities of CHV in HIV/AIDS Lessons learned • Community mobilization and quality services are effective in increasing contraceptive prevalence • Grandmothers need to be taken into account in the BCC approaches to child spacing • Intensification of HIV/AIDS program is necessary Medical Care Development International 26 TPCSP Final Evaluation c. Progress by intervention area in Betioky District Table 5 presents the results of the KPC surveys for Betioky at TPCSP baseline (when indicator available) and end of project. Although no specific objective was specified in the TPCSP DIP for Betioky, the project design assumes that (1) the results obtained at the end of the BSCSP in 2002 would at least be maintained until 2006 for the interventions that were continued (new interventions are those for which no data are available at baseline-“N/A”), or, that (2) the objectives set for Toliara would also be achieved in Betioky, or, that (3) the same or better results would be achieved for the new interventions in Betioky as in Toliara. The three last columns of Table 5 provide an appreciation of whether the project objective for an indicator is met (“YES”) or not (“NO”) according to these three criteria. Criteria (1) is clearly the most relaxed one since it only requires maintaining activities that were implemented at the end-of-project in 2002. Nine of eleven (91%) available indicators suggest that these objectives are met (an objective is met if the estimate’s confidence intervals of the baseline and the end-of-project surveys overlap). Criteria (2) is the most ambitious one since it assumes that the same level of activities would be implemented in Betioky as in Toliara, even for the new interventions. Twelve of the twenty-two (55%) indicators suggest that the objectives are met (an objective is met if it is included in survey estimate’s confidence interval). Most unmet objectives are those for the new interventions (PCM, MAL and HIV prevention), but also those for IMM. Criteria (3) provides an intermediate perspective that assumes that results in Betioky should be just as good as in Toliara, but not better. Therefore an objective that is not met in Toliara can be met in Betioky if the results are equivalent to those achieved in Toliara. Seventeen of the twenty-two (74%) indicators suggest that the objectives are met (an objective is met if the estimate’s confidence intervals from the two FE surveys overlap). Unmet objectives are those of the two new interventions PCM and MAL, and those for immunization. More detailed presentation of these results is discussed below, along with graphs of results in Betioky and Toliara from 1998 to 2006, when available. Medical Care Development International 27 TPCSP Final Evaluation Table 5 FE Results for TPCSP in Betioky # Indicator BL FE Versus BSCSP FE (1) Versus Toliara Object. (2) Versus Toliara Results (3) PCM1 Early treatment seeking for rapid/difficult breathing N/A 44 N/A YES YES PCM2 Knowledge of at least two danger signs N/A 46 N/A NO NO PCM3 Clinical staff who use correctly IMCI protocols N/A 27 N/A NO YES MAL1 Children sleeping under an ITN N/A 43 N/A YES NO MAL2 Mothers who give correct treatment for fever N/A 31 N/A NO YES MAL3 Mothers who took antimalarials during pregnancy N/A 50 N/A NO NO MAL4 CBDs with no stock-outs of antimalarials N/A N/A N/A YES YES CDD1 Children with diarrhea who are given more fluids 68 65 YES YES YES CDD2 Children with diarrhea who are given same or more food 86 76 YES YES YES CDD3 Children with diarrhea who are given same or more breast milk 93 90 YES YES YES CDD4 Early treatment seeking for children with diarrhea N/A 52 N/A NO YES CDD5 Mothers who can cite at least 2 danger signs of diarrhea 65 66 YES YES YES IMM1 Children 12-23 months completely immunized 44 46 YES NO NO IMM2 Mothers who received two doses of TT during last pregnancy 28 37 YES NO YES IMM3 Immunization drop out rates 15 23 YES NO NO BRE1 Exclusive breastfeeding 62 42 NO YES YES BRE2 Early initiation of breastfeeding 55 53 YES YES YES BRE3 Complementary feeding N/A 80 N/A YES YES CS1 Contraceptive prevalence among mothers of children under 2 17 28 YES YES YES CS2 Knowledge of HIV prevention N/A 36 N/A NO YES CS3 CBDs with no stock-outs of condoms N/A 90 N/A YES YES CS4 Knowledge of exclusive breastfeeding as child spacing method N/A 3 N/A NO YES Total YES: 10 12 17 Total available indicators: 11 22 22 % 91% 55% 77% Although both PCM and Malaria interventions were introduced under TPCSP and therefore new for Betioky, the FE results suggest that all the related indicators have increased if one assumes a baseline in Betioky similar to that found in Toliara (see Results Chart in section B.1). Two indicators (PCM1 and MAL1) even have reached the objective set for Toliara. The percentage of mothers who seek early treatment for their child with rapid/difficult breathing reached the same level as in Toliara. The percentage of children under two who sleep under an ITN also reached the Toliara objective, but did not increase as much as in Toliara, where this indicator increased well beyond the objective. Two other indicators of new interventions achieved the same levels in Betioky as in Toliara: the percentage of clinical staff who use correctly IMCI protocols, and the percentage of CBDs with no stock out of anti-malarial drugs.3 The three last indicators of new interventions did not reach the Toliara objectives. The mothers’ knowledge of least two danger signs increased but not as much as in Toliara, where the objective was achieved. The percentage of mother who correctly treat their children under two with fever increased as much as in Toliara, but did not reach the Toliara objective. Finally, the percentage of mothers who took antimalarials during pregnancy only reached a lower level than in Toliara. 3 The final value of this indicator was not available when finalizing this report but was reported as equivalent to that for Toliara by MCDI/Madagascar staff. Medical Care Development International 28 TPCSP Final Evaluation Figure 7 shows the KPC results for CDD in Betioky and Toliara from 1998 to 2006. The four indicators shown in the graph were maintained from the TPCSP baseline to the end of the project. The 1998-2006 perspective shows that two indicators of home-management of diarrhea (give more fluids—CDD1, and give same or more food—CDD2) have in fact remained at those levels since 1998. The two other indicators, appropriate breastfeeding practice during diarrhea—CDD3 and knowledge of 2 danger signs—CDD5, show an impressive increase during BSCSP, which is maintained throughout TPCSP. The only new indicator, early treatment seeking for diarrhea—CDD4, may have remained at the same level during TPCSP, if one assumes the same baseline as in Toliara, and may have reached the same level as in Toliara4 at the end of the project, but it did not reach the Toliara objective. Figure 7 Selected KPC survey results for CDD in Betioky and Toliara II, 1998-2006 CDD1: Children with diarrhea who are given more fluids CDD2: Children with diarrhea who are given same or more food 34% 58% 52% 53% 65% 68% 0% 20% 40% 60% 80% 100% 1998 2000 2002 2004 2006 Toliara II Betioky Sud 24% 64% 54% 66% 62% 76% 86% 0% 20% 40% 60% 80% 100% 1998 2000 2002 2004 2006 Toliara II Betioky Sud CDD3: Children with diarrhea who are given same or more breast milk CDD5: Mothers who can cite at least 2 danger signs of diarrhea 36% 60% 66% 61% 65% 90% 93% 0% 20% 40% 60% 80% 100% 1998 2000 2002 2004 2006 Toliara II Betioky Sud 6% 33% 45% 64% 65% 66% 0% 20% 40% 60% 80% 100% 1998 2000 2002 2004 2006 Toliara II Betioky Sud Figure 8 shows the KPC results for IMM in Betioky and Toliara from 1998 to 2006. The percentage of children completely immunized by age 1 (IMM1) was maintained from baseline to the end of the project, which is a success given the impressive increase of this indicator during BSCSP. The level achieved at the end of TPCSP, however, is far lower than that achieved in Toliara, and the Toliara objective is not reach. The RHFPO, which overall reports higher levels 4 This indicator estimate is based on a very small sample size (63 cases of diarrhea in the 2006 Betioky KPC) and the two confidence intervals barely overlap. Medical Care Development International 29 TPCSP Final Evaluation of completely immunized children based on the measles immunization coverage, also finds lower coverage for this indicator in Betioky (76%) than in Toliara (97%). At least one reason for this difference probably is related to the KPC finding that the indicator of immunization drop-out (IMM3) increased in Betioky during the TPCSP period, although this indicator can still be considered as maintained throughout TPCSP since the confidence intervals for these two indicators overlap. The Betioky EPI officer also reported during the second FE workshop that the district still faces issues of kerosene supply, which is confirmed by the results of the Integrated Supervision (see section B.3.c.iii). Although the tetanus immunization coverage of pregnant women (IMM2) increased in Betioky and reached the level achieved in Toliara, it remained short of the Toliara objective, just as it did in Toliara. As discussed above for the TPCSP Progress in Toliara (see section B.2.b.iv), this indicator, based on the immunization status as observed in the mother’s immunization card at the time of the survey, does not reflect the true tetanus immunization coverage among pregnant women. The tetanus immunization coverage as reported by mothers in Betioky is 37% in the baseline and 77% in end-of-project surveys. Figure 8 Selected KPC survey results for IMM in Betioky and Toliara II, 1998-2006 Children 12-23 months completely immunized Mothers who received at least two doses of TT during last pregnancy 30% 52% 71% 9% 44% 46% 0% 20% 40% 60% 80% 100% 1998 2000 2002 2004 2006 Toliara II Betioky 34% 20% 50% 21% 37% 28% 0% 20% 40% 60% 80% 100% 1998 2000 2002 2004 2006 Toliara II Betioky Sud Figure 9 shows selected KPC survey results for BRE in Betioky and Toliara II from 1998 to 2006. The percentage of children 0-5 months who are exclusively breastfed (BRE1) decreased from baseline to the end of the project in Betioky, but reached the same level as that achieved in Toliara. The FE Team could not find a definitive explanation for this decrease. This is the only indicator that decreased in Betioky during TPCSP. Although the simple overestimation of this indicator at the end of BSCSP is a possibility to consider, other explanations related to the decreased direct support from MCDI are worth investigating, particularly in the perspective of the scale-up of this and other interventions at the regional level under TREIP. The early initiation of breastfeeding (BRE2) was maintained at the same level during the project implementation, which also is the same as that achieved in Toliara. The Toliara objective was achieved both in Betioky and Toliara. Although there is no baseline for the complementary feeding indicator (BRE3) in Betioky, the level achieved at the end of the project is markedly higher than in Toliara, and therefore the project objective for this indicator can be considered as met. Medical Care Development International 30 TPCSP Final Evaluation Figure 9 Selected KPC survey results for BRE in Betioky and Toliara II, 1998-2006 BRE1: Exclusive breastfeeding BRE2: Early initiation of breastfeeding 22% 34% 41% 11% 42% 42% 64% 0% 20% 40% 60% 80% 100% 1998 2000 2002 2004 2006 Toliara II Betioky Sud 24% 42% 54% 29% 35% 55% 53% 0% 20% 40% 60% 80% 100% 1998 2000 2002 2004 2006 Toliara II Betioky Sud Note: EBF in 1998 and 2000 is for 0-4 month old, which is expected to be slightly higher than for 0-5 month olds. EBF for 0-4 months in 2002 in Betioky is 62%. Figure 10 shows the KPC results for CS in Betioky and Toliara II from 1998 to 2006. The contraceptive prevalence among mothers of children under two (CS1) increased from baseline to the end of the project in Betioky, and even reached the Toliara objective. Figure 10 also shows, for reference, the progress in the contraceptive prevalence among women of reproductive age in Betioky and Toliara under TPCSP. This indicator, which more widely used outside of child spacing programs, is judiciously reported in various MCDI KPC surveys reports. However, the selection of the sample of WRA, as opposed to mothers of children under two, is not explicit, and therefore the comparison of the results this indicator with those from other sources, such as the Demographic and Health Surveys, remains hazardous. Nevertheless, the contraceptive prevalence among WRAs was successfully maintained in Betioky during TPCSP, while it increased in parallel to the contraceptive prevalence among mothers of children under two in Toliara. The interpretation of these last findings is beyond the scope of this report. Figure 10 Selected KPC survey results for CS in Betioky and Toliara II, 1998-2006 CS1: Contraceptive prevalence among mothers of children under two Contraceptive prevalence among women of reproductive age—NOT a project indicator 9% 22% 24% 9% 15% 28% 17% 0% 20% 40% 60% 80% 100% 1998 2000 2002 2004 2006 Toliara II Betioky Sud 12% 25% 29% 32% 29% 0% 20% 40% 60% 80% 100% 1998 2000 2002 2004 2006 Toliara II Betioky Sud Medical Care Development International 31 TPCSP Final Evaluation The knowledge of HIV prevention (CS2) reached significantly higher levels in Betioky than in Toliara, which is a success, but did not reach the objective set for Toliara. The FE Team did not find any explanation for the relative success of this intervention on Betioky. Just as in Toliara, the knowledge of exclusive breastfeeding as a child spacing method (CS4) did not increase in Betioky. In addition to the comments on the progress for each intervention presented above, the FE Team identified the following general factors of success, constraints and lessons learned for the TPCSP interventions in Betioky. Factors of success • VISA approach and child survival interventions well established and maintained in the community since BSCSP • Dynamism and good acceptance of NGO partner VEMIMA Constraints • Absenteeism of HW seems to be particularly problematic in Betioky • The DHMT members in Betioky were only trained in management Lessons Learned • Well-trained and motivated NGO staff can effectively implement the community health and mobilization approaches developed under BSCSP and TPCSP d. New tools or approaches This section highlights the TPCSP achievements in terms of: • VISA approach • Health Care Financing Insurance • Kôminina Mendrika i. The VISA Approach The VISA Approach is the unique aspect of the community health volunteers program of TPCSP. The approach was further developed based on the experience gained under the previous child survival project in Betioky. It provides a way to sustain the number of motivated CHVs’ by continuously recruiting new CHVs among mothers, called VISA mothers or Remy Limy in Malagasy, who have successfully adopted healthy behaviors for their own child. VISA refers to the main role played by the mothers enrolled in the program and stands for Visit, Identify, Sensitize and Accompany: visit new mothers, identify with them inadequate behaviors, sensitize them on how to adopt optimal behaviors, and accompany them during the process of behavioral change. CHVs trained in the VISA approach are asked to identify 5 new mothers in their communities who showed particular interest in the health of their child and have successfully adopted healthy behaviors. These mothers are then encouraged to share their experience with others and those who get involved and motivated by this volunteer work are invited to become CHV and receive the appropriate training. This continuing enrollment of new and motivated mothers allows for the replacement of CHVs who lose interest in or stop this work for whatever reason. Medical Care Development International 32 TPCSP Final Evaluation ii. Kôminina Mendrika MCDI is one among numerous partners in Madagascar to implement the nationally-adopted community health and mobilization strategy Kôminina Mendrika, or “Commune Champion.” It is implemented primarily by a consortium of organizations including MCDI within the USAID￾funded project SantéNet. The strategy aims at empowerment and good governance of communities as well as at specific health improvements and behavior changes. A typical cycle of implementation in one Commune lasts 9 to 12 months and includes 10 well-defined steps. Communes that implement the strategy and achieve their self-defined objectives are awarded the status and certification of Kôminina Mendrika. The Commune can then receive support for one or two other cycles. In FY05, MCDI submitted a proposal to SantéNet to implement this strategy in 10 communes including 4 in Toliara II and 4 in Betioky. MCDI proposed to strengthen the interventions found as the weakest during in the 2004 KPC (malaria prevention among pregnant women, knowledge of HIV prevention, use of ITN by children under 5). Use of modern contraceptives and childhood immunization were also included in the proposal as USAID and MOH priorities. By the end of the project in September 2006, the 10 communes were selected, 10 Technical Animators were recruited and trained, Kôminina Mendrika Committees were created in each communes, CBDs were trained and received their initial stocks of social marketing products, and baseline data collection was completed. Although these activities are primarily implemented through another project and funding source, the Kôminina Mendrika strategy complements well the CHV program and other activities implemented under TPCSP. The strategy involves new community partners in addition to the BHC already involved in TPCSP, primarily the Commune administration but also schools and others. The Kôminina Mendrika strategy will be expanded under TREIP and therefore the experience gained and lessons learned under TPCSP are highly relevant to the FE of TPCSP. iii. Health Care Financing Insurance TPCSP continued the support to the community-based Health Care Financing Insurance in the Commune of Ankazomanga (population 3,200) developed under BSCSP. As a result of a sudden change in the national cost-recovery policy in favor of free health services for all, the pilot project in Ankazomanga was suspended at the end of 2002. With the progressive return of a cost-recovery policy in 2004, MCDI reestablished the pilot Health Care Financing Insurance in Ankazomanga in 2005. At the same time, MCDI participated in the preparation of the national guide for establishing community-based health insurance, in collaboration with SantéNet. The MTE recommended expanding the Ankazomanga pilot project, and MCDI identified and launched a community-based health insurance in three new communes in January 2006. Medical Care Development International 33 TPCSP Final Evaluation 3. Results: Cross-cutting Approaches a. Community Mobilization The Community Mobilization strategy of TPCSP is based on the creation and support of a network of community members who reinforce each other in the promotion of key messages and healthy behaviors. These community members include community health volunteers (CHV), VISA mothers (see section B.2.d.i), traditional birth attendants (TBA), and various local leaders. Some CHVs are also trained to become community-based distributors (CBD). MCDI recruited 6 Field Agents to lead the TPCSP community mobilization activities in Toliara II. All originally from the project area, Field Agents were each assigned one defined area of Toliara II. After their training in Q3 and Q4 of FY03, they have been actively identifying, training and supervising CHVs, and mobilizing local authorities and community members for health activities such as festivals, mass campaigns and others. They also play a key role in linking BHCs and the DHMT with the various community health agents and activities. The FAs are supervised by the MCDI BCC Manager and submit regular reports of their activities. TPCSP also partnered with local NGOs to relay project support to the community. In Betioky, building on achievements from the previous project, TPCSP handed over direct responsibility for project-sponsored activities with communities to VEMIMA, a local women’s association that was already a partner of BSCSP. In Toliara, TPCSP has started working with the local NGO Mampifoha, and has supported the creation and the implementation of limited activities by Miainga. TPCSP strategies, activities and achievements to build local NGOs’ capacity for community health is further described in section B.3.c.ii. The role and function of CHV and CBDs in the project are described below, and their training is discussed in section B.3.c.v. CHVs TPCSP identified and trained a total of 548 CHVs in Toliara II, among whom 496 (91%) are still functional at the end of the project. The follow up and supervision of CHVs is described in section B.3.c.iv, and the type and timing of their training is discussed in section B.3.c.v. CHVs are mainly involved in conducting various BCC activities, described in section B.3.b, and in identifying VISA mothers (see section B.2.d.i). They also link communities to BHC, where they often conduct health education activities and assist with immunization activities and follow￾up on dropouts. Many CHV also are CBDs. CBDs TPCSP trained a total of 185 CBDs in Toliara II, most of them at the end of FY05. In Betioky, TPCSP CBDs that had been trained under the BSCSP and that were still active at mid-term, and also trained new CBD. The training of CBDs included home-based management of malaria with chloroquine (PaluStop) the use of ITNs (Supermoustiquaire), counseling on oral contraceptives (PilPlan) and condoms (Protector), the use of safe water (Sur'Eau), and use of the various tools for the management of the social marketing of these products. Medical Care Development International 34 TPCSP Final Evaluation Although the national malaria policy changed in October 2005 from a fee-based distribution of ITN and chloroquine to a system of free distribution of these products in health services, CBDs still manage social marketing products. PSI provides an initial stock of each product to each trained CBD, then CBDs buy new products to MCDI with the money from the sale of their products, and MCDI gets new supplies from PSI. This system frees MCDI from any financial control of CBDs. As the sale of the products is lucrative, some CBDs even use their own money to buy products from MCDI. Other forms of chloroquine (''Ody Tazomoka'') are available for free in the BHC, which receive them from the MOHFP. Although condoms and contraceptive pills also are distributed for free in health centers, CBD remain involved in the distribution of Protector and PilPlan for a fee. The national policy on CDD also is the use of home-based fluids or ORS, which is distributed for free in health centers. CBDs are not involved in the distribution of ORS but promote the use of Sur'Eau. Table 6 shows the total number of units of the five social marketing products that MCDI provided to CBD and others distributors in 2005 and 2006. It also shows the total number of units available at the CBD level during a supervision visit conducted in September and October 2006, and the percentage of CBD who had these products in stock during those visits. These data come from the reports of Field Agents, and are based on 177 CBDs out of the 185 functional CBDs in the 22 communes of Toliara II in which MCDI supports CBDs (2 other communes are supported by ASSOS within the Kôminina Mendrika initiative). According to this data source, 92% and 88% of CBDs had stocks of antimalarials and condoms, respectively; these two results are used for the TPCSP indicators for the Malaria Control (MAL4) and Child Spacing (CS3) interventions, even though they represent availability at a particular point rather than during a whole 6-month period. Table 6 Health products provided by MCDI and availability at the CBD level in Toliara II and Betioky Units provided by MCDI in Toliara II and Betioky (1) Availability at CBD level in Toliara II (Sept/Oct 2006) (2) CBD with stock Product Units 2005 2006 Total Total Units # % Supermoustiquaire ITN 18,598 19,479 38,077 11,954 153 86% PaluStop (infants) Packets of 3*75mg CQ 15,430 21,330 36,760 3030 165 93% PaluStop (child 12-59) Packets of 3*150mg CQ 15,610 54,230 69,840 4210 163 92% Sur’Eau Bottle -- 412 412 65 46 26% PilPlan Packets of 21 pills -- 56,940 56,940 16140 170 96% Protector Condoms 24,480 192 24672 4416 155 88% Source: (1) MCDI stock cards; (2) MCDI Field Agent Reports, Sept/Oct 2006 Medical Care Development International 35 TPCSP Final Evaluation In accordance with a MTE recommendation, TPCSP increased its social marketing activities for the ITN Supermoustiquaire in 2005. Figure 11 shows the progression of the provision of ITNs by MCDI to CBD, local NGOs and other distributors in 2005 and 2006. This rapid progression and the total number provided is consistent with the increase in the project indicator on use of ITNs by children under two in Toliara II from 7% at midterm (November 2004) to 58% at the end of the project (December 2006). Although MCDI provides the majority of ITNs in Toliara, other partners have distributed ITN during the same period: the MOHFP provided small quantities for free distribution in BHC; PSI provides ITNs to regular shopkeepers in villages that are accessible, SALFA/Global Fund sells ITNs through associations related to the Lutheran churches that exist in certain villages, and ASSOS sells ITNs within the Kôminina Mendrika initiative that they implement in two villages. Figure 11 Numbers of ITNs provided by MCDI in Toliara II and Betioky in 2005 and 2006 0 10,000 20,000 30,000 40,000 Jan-Mar Apr-Jun Jul-Sep Oct-Dec Jan-Mar Apr-Jun Jul-Sep Oct-Dec 2005 2006 # ITNs provided in quarter Cumulative # ITNs provided In addition to creating a network of community members capable of promoting healthy behaviors, TPCSP sponsored various types of mass mobilization events. Community Radio At the beginning of the project, TPCSP sponsored training of MCDI staff (all FA), other partners’ staff (4) and 11 radio communicators from local radio stations in Toliara II and Betioky. This training was conducted by ALT-Radio, a project based in Toliara that provides technical support for educational radio programs in local community radios. The training included the preparation of various radio products such as shows, songs, poems, sketches and spots focusing on specific health and educational messages or issues. Trainees were also taught Medical Care Development International 36 TPCSP Final Evaluation how to create and animate community listening groups in relation with educational radio programs. MCDI contributed 2 or 3 radio products a month to local community radios through ALT-Radio. In January 2007, for instance, MCDI contributed one product on colostrum and one on Health Care Financing Insurance. These products are broadcasted several times a months by several local radios: RCA and Mazava in Toliara II, and RFA in Betioky. Table 7 shows the number of the different types of community radio products developed under TPCSP by health intervention. The most frequently developed topic is IMCI, which may include a variety of child health and survival themes such as PCM, MAL and CDD. Table 7 Community radio products developed under TPCSP Radio products Health intervention Show Spot Sketch Poem Song Total IMCI 20 9 6 1 0 36 TB 0 17 1 4 0 22 EBF 8 5 0 3 3 19 FP 2 4 3 4 0 13 ANC 3 2 5 0 0 10 EPI 3 2 0 0 1 6 Complementary feeding 3 1 1 0 0 5 TBA 3 0 0 0 0 3 AIDS 1 0 0 0 2 3 VISA 2 0 0 0 0 2 Health Care Financing Insurance 1 0 0 1 0 2 LAM 0 0 0 0 1 1 HYGIENE 0 0 1 0 0 1 TOTAL 46 40 17 13 7 123 Source: BCC Manager, MCDI, February 2006 Health Festivals TPCSP sponsored health festivals in 28 of the 29 health sectors in Toliara II and in all 21 health sectors Betioky in Q4-FY04 and Q3-FY06 (see MTE report for a description of what this activity involves). This activity has now ended because the MOHFP began a series of mass campaigns, including the SSME described below. Mass campaigns TPCSP also contributed to various health campaigns sponsored by the MOHFP, such as immunization campaigns SSME In October 2006, the MOHFP launched its first national “Semaine de Santé pour la Mère et l’Enfant” (SSME—Health Week for Mother and Child). Several MOHFP officials (MOHFP/FHD, ROHFP, DOHFP in Toliara II) acknowledged MCDI’s support for this highly publicized and successful event. SSME will be held biannually in the future. Medical Care Development International 37 TPCSP Final Evaluation Kôminina Mendrika TPCSP contributed to the implementation of the KM approach in 4 communes in Toliara and in 4 communes in Betioky. The KM approach is described as a new and promising approach in section B.2.d.ii. Below are the answers to FE Guidelines questions developed by the FE Team. Were the objectives met for community mobilization? TPCSP community mobilization activities are praised by MOHFP officials, and most knowledge and behavioral project objectives are achieved. Most planned community mobilization activities are implemented in terms of training and follow-up of CHVs by local NGO staff and Field Agents, and support to various mass mobilization activities. However, shortcomings in the achievements of some of the knowledge and behavioral project objectives suggest that community mobilization efforts may have not entirely been as effective as expected. How effective was the approach for community mobilization? Local leaders interviewed in the four communities visited by the FE Team listed among activities of CHVs the promotion of health in general (antenatal care, nutritional surveillance, breastfeeding, family planning, malaria and ITN, utilization of BHC, immunization and tuberculosis), the distribution of health booklets and mass campaigns. Local Leaders found that TPCSP community mobilization approaches were effective because: • CHVs are motivated and determined to conduct their activities, they know their subjects, and they are skilled in communication techniques • Mothers who have successfully practiced key behaviors can effectively encourage other mothers in adopting these practices • Key messages are well received and practiced in the communities • Local authorities, HWs and Field Agents participate in community health and mobilization, and often directly assist CHVs and other community members involved in these activities Local leaders also pointed out that CHVs would benefit from some transportation means like bikes and even some compensation to improve their motivation and effectiveness. Mothers interviewed in the four communities visited by the FE Team know the various activities of the CHVs, including their transfer of knowledge to VISA mothers. They acknowledge that CHVs have helped them change their life and ensure that their children are healthy, and that practices of exclusive breastfeeding and family planning have increased in the their communities. Although CHVs can account for a large part of the successes of TPCSP, it appears that there are still limitations to the current program: • Some CHV do not have the capacity or even willingness to conduct the activities expected from them. This observation was made to the FE Team during their field visits by at least one BHC physician as well as by some mothers, and the FE team confirmed it during the workshop. Medical Care Development International 38 TPCSP Final Evaluation • FA and CHVs respond more to MCDI staff than to HWs, and there is not enough direct collaboration between CHVs and HWs. Various MOHFP officials raised this concern on several occasions. • CHVs seem to receive enough in-kind compensation to suggest that they may not remain as active and motivated when the project stops. MOHFP officials also raised this concern. The various health festivals, campaigns and the SSME generate a lot of satisfaction among MOHFP and local authorities because the immediate and large impact on the interventions on which they focus. The Kôminina Mendrika approach, although broader in its scope and its duration, has a similar effect. Although these approaches certainly have their values and effectively complement other approaches implemented on a continuous basis, their sustainability and long-term cost-effectiveness remains an issue that do not seem to be fully addressed. Lessons learned for future community mobilization efforts? The FE Team identified the following lessons learned from the TPCSP mobilization efforts: • Participation of local authorities is necessary for the success of community health and mobilization activities • The VISA approach is effective and sustainable • Educational community radio and listening groups are effective • Health festivals and campaigns, including the SSME, are effective Is there demand in the community for project activities to continue? How was this measured? The increase, sometime large, in many indicators measured by the KPC surveys suggests that the demand for the child survival interventions promoted by TPCSP is strong in the project area. In addition, community members and local leaders interviewed in the four communities visited by the FE Team clearly report that most communities are committed to continue supporting CHVs and the VISA approach. All mothers interviewed wish that CHVs continue their activities because they see the benefits for their children’s health. Local authorities in the four communities visited by the FE Team all wish that CHVs, the VISA approach, the CBDs and the radio programs continue to ensure that communities take charge of their own health. They recognize the positive results achieved in terms of the use of ITNs, utilization of health services, including immunization and family planning, and accessibility to social marketing products. What are the plans for sustaining these activities once the project closes? All four BHCs visited by the FE Team have included community mobilization activities in their Annual Work Plan (Plan de Travail Annuel—PTA). One BHC has included update of their CHVs’ skills. The current DOHFP Annual Work Plan in Toliara includes various types of community mobilization activities such as Health Festivals, Kôminina Mendrika, follow-up of CHVs, and training of TBAs. Local authorities of three of the four Community Development Committees interviewed by the FE Team reported that their Community Development Plan did not include community mobilization for health, but that they could indeed include such activities if discussed and adopted in a timely manner by the CDC. They acknowledged that CHVs and VISA mothers are trained and motivated, and that they will continue their activities at the end of the project. Medical Care Development International 39 TPCSP Final Evaluation The two local NGOs VEMIMA and Mampifoha regularly include community mobilization activities in their work plan and expect to continue receiving support for such activities under TREIP. Finally, TREIP primarily focuses on scaling up the community mobilization approaches developed and implemented under BSCSP and TPCSP to the nine districts of the South West Region. This will include training and support of CHVs, VISA mothers, Kôminina Mendrika, support to community educational radio programs, and various other mass mobilization activities. Are the sustainability plans realistic? Community mobilization included in the current work plans of the CDC, BHC, DOHFP and local NGO probably are realistic and funded, but there no information on the possibilities, financial or other, that these TPCSP partners will be able to continue implementing these community mobilization activities. Several MOHFP officials expressed their concern that there was no MOHFP budget for such activities so far. The overall strategies of TREIP are realistic and ambitious; detailed planning of activities to be implemented is under way at the time of writing this TPCSP FE report. Among other specific input under TREIP, 4 or 5 Field Agents and 1 supervisor will be recruited in each District of the South West region including Toliara II and Betioky b. Communication for Behavior Change One of the key functions of the CHVs is to provide health education to mothers and support behavior change. TPCSP trained all CHVs in the content of the key messages to deliver to mothers and taught them communication skills, and Field Agents and BHC HWs reinforce these knowledge and skills during their regular meetings with and supervision visits of CHVs. BCC approaches implemented by CHVs are the VISA approach (see section B.2.d.i), home visits, interpersonal communication, support to local community radio and listening groups, sketches, Focus Group Discussions, Community General Assemblies, health education in BHC, and advocacy with local leaders. One major activity of TPCSP has been the development, production and distribution of various materials to support CHVs’ and HWs’ behavior change activities. Appendix 10 presents the type, content, use and the total number of most BCC materials distributed in the project area over the life of the project. In addition, MCDI adopted and used without further testing a series of posters and generic BCC supports produced at the national level by the MOHFP and its partners (PSI, UNFPA, UNICEF). The MOHFP authorized MCDI’s use of these materials and provided samples for the project area. Were the behavior change objectives met? The progress achieved for most TPCSP indicators for the five technical interventions demonstrate significant behavior change in the community (see Progress by interventions in section B.2.b). Most key TPCSP indicators show an increase (or an expected appropriate decrease) in Toliara, and most indicators in Betioky also showed an increase or a status quo since the end of BSCSP (see section B.2.c). Medical Care Development International 40 TPCSP Final Evaluation How effective was the approach for communication and behavior change? CHVs in the four communities visited by the FE Team found that home visits and interpersonal communication are the most effective approaches because of the confidentiality that they ensure. In Betioky, CHV also mentioned community radio, FGD, and General Assemblies as effective. Staff from the 2 NGOs VEMIMA and Mampifoha also found that home visits are most effective because of the confidentiality that they ensure and because CHVs can see directly the level of understanding of the mothers and their reactions to the messages. According to them, health education and counseling in BHC is also effective because mothers respect the opinion of the HW and are particularly attentive to health issues when they come to the health center. CHVs can effectively complement health education received in health centers through home visits. Other BCC methods, by order of perceived effectiveness, are sketches, FGD, CIP and General Assemblies. What were the lessons learned? The FE Team identified the same lessons learned from the TPCSP BCC as for community mobilization efforts (see section B.3.a): • Participation of local authorities is necessary for the success of community health and mobilization activities • The VISA approach is effective and sustainable • Educational community radio and listening groups are effective • Health festivals and campaigns, including the SSME, are effective How will these behaviors be sustained once the project closes? Are the sustainability plans realistic? The discussion on how the community mobilization activities will be sustained is entirely relevant to the BCC activities (see section B.3.a). How was the impact of BCC interventions measured/evaluated? TPCSP primarily measured the impact of interventions through the series of KPC surveys conducted in 2002, 2004 and 2006 in Toliara, and in 1998, 2000, 2002 and 2006 in Betioky. FA also conducted grouped supervision meetings with CHVs during which they discussed progresses in the community. FA supervision reports are analyzed on a monthly basis by MCDI staff, but no overall summary report was available to the FE Team. c. Capacity Building The DIP does not specifically outline a capacity building strategy for TPCSP. However, TPCSP achieved significant organizational development changes of the following partners. i. Strengthening the Grantee Organization How has this grant improved the capacity of the grantee to design, implement and evaluate effective child survival projects? TPCSP has given the Home Office and Field Office staff the opportunity to interact with cooperating agencies, peers, and consultants and learn more about tools and methods to continue improving child survival programs. This has been a two-way road in the sense that MCDI staff Medical Care Development International 41 TPCSP Final Evaluation is not just learning in this process but also contributing with innovative approaches and tools to other PVOs doing CS projects in similar areas. Both Home Office and Field Office staff regularly attend the Mini-University and Technical Updates organized by USAID, CSTS, and CORE, in person and through Elluminate. During TPCSP, MCDI/Madagascar staff benefited from various national and international training opportunities (see section B.4.b), but particularly from developing experience and commitment to work in child survival and for the poorest communities. TPCSP was also an opportunity for MCDI/Madagascar to build strong networks with governmental and non￾governmental organizations involved in child survival in the project area and at the national level. This constitutes a very valuable investment for MCDI. TPCSP strengthened the capacities of MCDI/Madagascar in child health and survival and related areas to the point that it won a subcontract with Chemonics on the USAID-funded SantéNet project, a Flex Fund Grant to implement a project in a neighboring region, and more recently a CSHGP Grant for TREIP (2007-2011) in the highly competitive Expanded Impact category. MCDI/Madagascar staff is often requested by other organizations in Madagascar (ADRA, SF FJKM, Voahary Salama) to provide technical assistance and training in planning and evaluation of child survival projects. MCDI National Coordinator is also part of the Senior Technical Committee for Child Survival of the MOHFP, along with other representatives of WHO, UNICEF, BASICS, USAID and other organizations. In this role, he is able to share MCDI’s experience for the elaboration of various national policy documents such as the National Policy on Child Survival, the national strategy for child survival, the national guide for the introduction of the community-based case management of diarrhea, acute respiratory infections and malaria (including the introduction of zinc, cotrimoxazole, hypo-osmolar ORS, and Artemisin-based Combined Therapy). The former MOHFP Director of Family Health, now Vice-Minister for Health, reported to the FE Team that MCDI has become the main reference organization for community health, mobilization and C-IMCI in Madagascar. How have effects of this grant influenced other projects operated by the grantee? MCDI Madagascar’s success in securing additional funding for child survival projects is mentioned above. Finally, the experience gained in Madagascar through BSCSP and TPCSP helped MCDI HO and FO in the design, planning and evaluation of the MCDI CSHGP project in Benin (2003-2007) and South Africa (2004-2008). ii. Strengthening Local Partners Organizations The various activities of TPCSP to strengthen Health Facilities are discussed in section B.3.c.iii. TPCSP regularly assisted the DHMT in Toliara II and in Betioky in overall reviews and planning of project activities, in training and supervision of HW and CHVs, and in the development of tools for data collection and BCC. The DHMT members received training in the various project activities they were involved in, including training of trainers in community health and mobilization and in IMCI, integrated supervision, and development of IEC/BCC materials (see section B.3.c.v). TPCSP also provided technical assistance and computers for the HIS service of the DOHFP, and provided logistic support for the DOHFP training and supervision activities. The TPCSP DIP did not include any other specific capacity building activities, and there has not been any Organizational Capacity Assessment of the DHFPO. Although most TPCSP efforts in capacity building were aimed at the district level, TPCSP also collaborated with the ROHFP in Toliara in the area of training and supervision, and for the Medical Care Development International 42 TPCSP Final Evaluation development of the Integrated Supervision tool and method (see section B.3.c.iii). This collaboration has increased after the MTE and with the announcement of the CSHGP Grant for TREIP. The ROHFP staff acknowledged to the FE Team that under TPCSP, they have learned much about community health and mobilization, and about coordination between HW, CHV and DOHFP. The ROHFP reports occasional disagreement with MCDI when TPCSP introduced new tools in the MOHFP services without being instructed or obtaining final approval from the regional or central level, even though these initiatives turned out to be very relevant and successful. Examples are the integrated supervision tool, the community health HIS, and the adaptation of IMCI training from 12 to 6 days. TPCSP has worked with three local NGOs, which have distinct geographical areas of interventions: • VEMIMA is a Women’s Association for Development created in 1995, and which played a significant role in the Betioky Child Survival Project. It was included as a full partner in the TPCSP proposal and DIP. By the end of FY03, TPCSP transferred all responsibilities for community activities in Betioky to VEMIMA, which was awarded a formal role in the DHMT of Betioky. Since then, VEMIMA has regularly and successfully prepared work plans according to the DIP and the DOHFP annual work plan, managed funds made available for these activities, and submitted activity and financial reports. Beside MCDI, VEMIMA also works with other partners such as Aide et Action, Seecaline, and the World Food Program. Medical Care Development International 43 TPCSP Final Evaluation • MAMPIFOHA is an NGO created in 2003 by rural animation agents from the Ministry of Communication and Youth, and has the entire Toliara province as zone of intervention. In Toliara II, TPCSP has gradually involved Mampifoha in the implementation of various community health activities with CHVs and CBDs. As they have been trained in all TPCSP interventions and have worked with MCDI FAs, they feel that they now are ready to continue activities implemented by the FAs so far. • MIAINGA In Toliara II, the local MCDI staff (all the FA and some others staff members) created the local NGO Miainga in 2004, in preparation for the end of TPCSP. Although this NGO has only limited involvement in project activities so far (contribution to the social marketing program), the members are all very qualified and motivated to pursue the organization’s purpose. • ZATOVO : ZATOVO is an NGO created in 2004 with the entire province as zone of intervention. In Toliara II, TPCSP has also gradually involved ZATOVO in the implementation of community health activities. MCDI staff regularly meet with representatives of the TPCSP’s local NGO partners in MCDI’s office to discuss their overall vision and project-supported activities. The NGO representatives interviewed by the FE Team reported a very good collaboration and a very cordial and warm working atmosphere with MCDI. However, they mentioned that they had not had any systematic or on-site supervision of their organization and activities from MCDI. Describe the outcomes of any assessment, formal or informal, conducted at the outset and conclusion of the project to determine the organizational capacities of local partners. The TPCSP DIP reports on the main findings of an Institutional Strengths Assessment of MCDI conducted with support of CSTS in March 2002, and mentions the intention of replicating this assessment in the future and adapting it for use with partner organizations. In 2006, TPCSP commissioned an Organizational Assessment of VEMIMA that was conducted in August 2006 by a local consultant. The objective of the assessment was to provide recommendations and an action plan to MCDI on how to continue strengthening VEMIMA’s capacity to implement child survival activities. The assessment was conducted in 30 days through a participatory approach involving the MCDI coordinator and a one-week workshop with 22 VEMIMA members in Betioky. The collection of electronic draft documents available to the FE Team in February 2007 provides insufficient information to summarize the findings. They include a list of recommendation but no work plan, objective or indicator of progress. In January 2007, Mampifoha conducted an internal evaluation, which was reported as very positive and motivating; no results were available at the time of the FE. MCDI has planned to commission an Organizational Assessment of Mampifhoa similar to that completed for VEMIMA. MCDI has also planned to commission such Organizational Assessment for Miainga and Zatovo. How have the organizational capacities of the local partner changed since the beginning of the project? What factors/interventions have most contributed to those changes? Only one organizational assessment of one local NGO partner was conducted at the end of the project, and this assessment does not provide data on changes in the capacity of the NGO Medical Care Development International 44 TPCSP Final Evaluation assessed. Nevertheless, the three main TPCSP local NGO partner have strengthened their capacities as follows: VEMIMA staff benefited from TPCSP training in nutrition, FP, EPI, SM, IMCI, communication skills, training skills, and from the various opportunities to implement community health activities. VEMIMA also received training in project management that will be used for developing new projects. VEMIMA staff now feels that they have the capacity to successfully implement project activities without external technical and administrative support. VEMIMA will be one of the NGO partners that will implement TREIP in several districts, and will receive its first grant within this project. MAMPIFOHA also benefited from training under TPCSP and improved its capacity to implement various project activities, mainly community-based distribution of health products. Mampifoha reported that this combination of training and opportunities to implement activities and see results was most beneficial. Mampifoha will be one of the NGO partners that will implement TREIP in several districts. MIAINGA has been able to develop its vision and organizational structure and to register as a local NGO. All its members benefited from various training, implementation and management opportunities as MCDI staff members under TPCSP, and they feel they are ready to accept more implementation responsibilities. In the meantime, all the Miainga members will remain MCDI staff under TREIP and continue to build their capacity. What are the best practices and lessons learned in capacity building of local partners? Local NGO staff reported to the FE Team that they acquired technical and management competence and confidence under TPCSP, and that they are now better prepared for independent operations. MCDI/Madagascar staff realized that thorough assessments of partner organizational capacity, and the development of related appropriate plans, are necessary to maximize capacity building efforts. iii. Health Facilities Strengthening TPCSP used various approaches to strengthen health facilities’ capacity: • Training (see section B.3.c.v) • Supervision (see section B.3.c.iv) • Quality improvement (see section B.3.c.iii) • Supply and equipment: o IMCI kits o BCC materials • Management assistance, including on HIS • Logistics support, including to outreach strategy What tools did the project use for health facility assessments? TPCSP used the following tools to monitor BHC’s capacities: • MOH Monthly activity reports • MOH Audit tools, for the financial management and cost-recovery system in BHCs Medical Care Development International 45 TPCSP Final Evaluation • MOH Monitoring tools used during the monthly and quarterly planning meetings • Specific MOH program monitoring tools • Integrated supervision tools: one developed by the MOH at the national level and one developed at the regional level by the RHO and MCDI The information from these tools is primarily used during the monthly and quarterly review and planning meetings, and during formative supervision visits. In the 8 communes where Kôminina Mendrika was launched, performance tables are used to show health activities conducted by the BHC and the communities, and the impact of these activities on selected indicators. These tables are posted in the Commune Offices. TPCSP conducted systematic Health Facility Assessments at baseline, midterm and end of the project. The data from these assessments were used in planning project activities in the DIP, and for monitoring progress during the midterm and final evaluations. The three HFA together measure changes in HW performance regarding IMCI clinical skills and in availability of services, but changed in content. The IMCI assessment includes a services provider clinical observation checklist, a provider interview and a mother’s interview. The IMCI assessment slightly changed from 2002 to 2006 to adapt to changes in the national IMCI algorithm. In 2006, the HFA did not include interviews of providers and mothers, and used a new and larger services availability assessment developed and adopted at the regional level as part of the Integrated Supervision approach level (see section B.3.c.iv). The new component of availability of services comprises far more items than the previous HFAs since it indeed integrates all the services expected in a BHC. The 2006 HFA is also called the “2006 Integrated Supervision.” Table 8 shows that the three HFAs were conducted in between 55% and 69% of the BHCs. The 2004 HFA draft report suggests that the most functional and accessible BHCs were selected, which makes sense to maximize the chances to complete full assessments, but such a selection probably biases the results towards better performance. On the other hand, the selection of BHC for the Integrated Supervision in 2006 may have purposely favored the most remote BHCs (no narrative report was available at the time of the FE). In any case, any interpretation of the results should take into account the sample selection criteria. Only one health worker is assessed in each BHC; this also raises issues of representativeness of the sample. The number of child consultations observed was reduced from 3 or 4 per HW in the first HFAs to only one per HW during the 2006 HFA/Integrated Supervision, raising issues of inference of the results to the population of children. Medical Care Development International 46 TPCSP Final Evaluation Table 8 Samples of BHC, HW and children in TPCSP HFAs BHC HW Children # % # % # HFA 2006 Clinical IMCI assessment Toliara II 18 55% 19 66% 21 Betioky 18 69% 20 100% 22 Total 36 61% 39 80% 43 Availability of services Toliara II 23 70% Betioky 17 65% Total 40 68% HFA 2004 Toliara II 22 67% 22 76% 103 Betioky N/A -- HFA 2002 Toliara II 18 6 Betioky 15 6 Note: % BHC assumes 33 BHC in Toliara and 26 BHC in Betioky and % HW assume 29 HW in Toliara and 20 in Betioky, as per ROHFP data in see Table 2 Selected results from the various HFAs are presented in Table 9 and Table 13, and the HFA and Integrated Supervision tools are further discussed in section B.4.g. How effective was the approach for improved management and services at health facilities? The results of the availability of services components of the 2002 and 2004 HFA could not be used to assess progress in the capacity of health facilities for the FE report because only draft reports are available that do not allow for reliable quotation of data. Table 9 shows selected results of the 2006 HFA/Integrated Supervision in Toliara and Betioky. A significant percentage of BHCs selected in both districts do not have key elements necessary for the implementation of IMCI such as at least one HW trained in IMCI, at least one CHV trained, IMCI cards or IMCI kit, an ORT corner, or at least one functional refrigerator and stock of kerosene. The 2006 HFA/Integrated Supervision also shows that a significant percentage of BHCs do not have adequate supply of essential drugs for the case management of childhood illnesses such as ORS, chloroquine, sulfadoxine pyrimethamine, vitamin A or antibiotics for more advanced cases. Similarly, a significant percentage of BHCs do not have adequate supply of contraceptive. These relatively poor results low-level results may be due to the selection of the sample of BHC, as mentioned above. Medical Care Development International 47 TPCSP Final Evaluation Table 9 Selected indicators of services availability in Toliara and Betioky, 2006 BHC with: TOLIARA II n=23 BETIOKY n=17 IMCI staff and equipment At least one HW trained in IMCI 91% 77% At least one CHV trained in IMCI 100% 82% Stocks of IMCI cards (2-48 months) 65% 77% Complete IMCI kit 70% 53% ORT corner 52% 47% At least one refrigerator 83% 100% At least one functional refrigerator 61% 88% Stock of kerosene 22% 12% Essential drugs Acide nalidixique 5% 0% Ampicilline (injectable) 46% 29% Chloroquine 77% 65% Cotrimoxazole 77% 77% Gentamycine (injectable) 36% 18% Metronidazole 77% 59% ORS 55% 65% Quinine (injectable) 78% 65% Sulfadoxine pyrimethamine 77% 53% Vitamine A 36% 41% Contraceptives/condoms Oral contraceptive 83% 76% Injectable contraceptive 96% 65% Condom 78% 24% Source: 2006 HFA/Integrated Supervision HW interviewed by the FE Team reported that in general the capacity building approach of the TPCSP has been effective, but they pointed out a few aspect to improve such as the selection of HW to be trained and the lack of tools and supports after the training. A persistent limitation to health facilities strengthening is the limited number and high turnover of health workers in BHCs. There is only one HW in many BHC, and therefore the BHC is closed as soon as he or she leaves for whatever reason. Departures and new assignments of HW may also disrupt implementation of activities if the new staff has not had the necessary training or is not properly briefed when starting on the job. What were the lessons learned? HW interviewed by the FE Team cited two lessons learned from the capacity building approach of TPCSP: • Collaboration between BHC and CHV and other community health agents is important • Involvement of local authorities and local leaders in the implementation of community health activities is necessary Medical Care Development International 48 TPCSP Final Evaluation What are the plans for sustaining these activities once the project closes? Are the sustainability plans realistic? The ROHFP and the DHMTs in Toliara and Betioky will continue to use some of the TPCSP approaches and tools for BHC strengthening such as training, supervision and quality improvement. TREIP will support these approaches at the regional level in all Districts (the MOHFP and MCDI were in the process of developing the Detailed Implementation Plan of this project at the time of the FE). These approaches and tools will also be developed and supported by other partners, such as SantéNet in the area of Quality Improvement. Discuss linkages between these facilities and the communities. TPCSP fostered multiple opportunities for linking communities to BHCs through: • Active participation of CHVs and other community members in various health festivals and campaigns sponsored by the MOHFP, including the SSME • Involvement of HWs in field activities during mass campaigns • Involvement of CHVs in BHC health services such as immunization (through home visits to dropouts) or referral of children with severe illnesses • Support to the implementation of ENA and C-IMCI through home visits by CHVs and the VISA approach • Community participation in the Quality Improvement initiatives in selected BHCs • Community contribution to the Health Care Financing Insurance system in the commune of Ankazomanga since 1999 and in three new communes since 2006 • Communities are contributing to the financing and management of BHC through the national cost-recovery system. • Launch and successful implementation of the KM initiative in 4 communes in October 2005 During the FE Field Visits, HW reported instances where the communities or local authorities had complained when no HW were present in the BHC, thereby demonstrating their concern and demand for accessible health services. In one BHC, HW reported that they regularly listen to feedback from the community about health services. iv. Strengthening Health Worker Performance How effective was the approach for strengthening health worker performance? Three types of health workers are considered in this section: health workers (HW) in BHCs, community health volunteers (CHV), and community-based distributors (CBD), who are CHVs with additional training and responsibilities. HWs TPCSP provided various training opportunities to health workers in BHC and in the DOHFP. These are presented and discussed in section B.3.c.v. In addition, TPCSP strengthened health workers performance by supporting district-level review and planning meetings, during which HWs are informed of and advised on how to improve the status of their BHC. TPCSP also strengthened the facilities in which HW work, including their community outreach activities, thereby providing HWs with an environment more conducive to improved performance. Medical Care Development International 49 TPCSP Final Evaluation TPCSP supported the supervision of HW since the beginning of the project, and progressively improved the system by developing an Integrated Supervision tool, and by introducing a formative approach to supervision. TPCSP developed a first version supervision tool in FY03, which was then used on an annual basis. In 2006, a large team of representatives from the ROHFP, the Regional Hospital, the DOHFP in Toliara and Betioky, several BHCs and MCDI began the development of an Integrated Supervision tool. This tool now includes a limited number of supervision items essential for quality services in BHCs and that can be completed in one day by a team of two supervisors.5 This tool was first used in October and November 2006 in 40 BHCs in Toliara and Betioky (see section B.3.c.iii) by a team of 6 senior physicians or other health professional working in pairs. Each pair included at least one member of the DOHFP in Toliara or Betioky, and one member from the ROHFP or MCDI. In addition to the supervision tool, TPCSP helped the introduction and adoption of the approach of formative supervision, through which individual advice and on-the-spot training are provided to supervised HWs. As per a MTE recommendation, the integrated and formative supervision approach was adopted at the regional level mid-2006. HWs interviewed by the FE Team, and staff from the two DHMT and the RHFPO, all found that the following TPCSP approaches to strengthening health workers performance have been effective: • Training and refresher training • Integrated supervision of BHC HW by DHMT and RHFPO • Support to regular technical reviews and exchanges • Regular supply in essential drugs, vaccine and medical supply, management tools • Quality improvement approaches (introduced more recently, in collaboration with SantéNet) The DHMT in Toliara reported to the FE Team that they receive about two supervision visits from the MOHFP central level, and sometime combined with international reviews of specific technical programs. CHVs TPCSP identified and trained 548 CHVs in key project interventions in Toliara II (see section B.3.c.v). TPCSP also provided CHV with BCC materials and supervised their activities. TPCSP began supervising CHVs at the beginning of the project in FY02 and continued throughout FY06. The three tables below present summaries of quantitative data from the MCDI tracking system on the supervision of CHVs by FAs (see section B.4.g). Table 10 shows the number of CHVs identified in 24 communes in Toliara, and the number of CHVs supervised in FY02, FY03, and FY04, the year of their training. According to these data, the percentage of CHVs supervised at least once increased from 71% to 91% during that period. 5 The MOHFP also had developed an Integrated Supervision tool, but that was widely considered too complex for effective use in the South West region because it included all the MOHFP programs, quantitative and qualitative components, and required 3 days per BHC to be completed. Medical Care Development International 50 TPCSP Final Evaluation Table 10 Number of CHVs identified and supervised in FY02, FY03, and FY04 FY02 FY03 FY04 Identified CHV 460 484 548 Supervised CHV 325 433 496 71% 89% 91% Source: MCDI BCC Manager, February 2007 After their training, all CHVs are supposed to receive regular follow-up and supervisory visits by the FAs. The FAs were trained to use the supervision methodology developed under BCSP and that comprises grouped and individual level components. Table 11 shows the number of CHVs reached in grouped visits each quarter of FY06. Assuming that different CHVs are supervised at least two successive quarters, and that the number of functional CHV remained the same as in FY04 (that is, 496), about 80% of CHVs might have been supervised every 6 months, which is short of the TPCSP DIP target of more than 80% of quarterly supervision visits. Table 11 Number of trained CHVs supervised by quarter in Toliara II, FY 2006 Number supervised Supervision units Q1 Q2 Q3 Q4 Total Total in area Trained CHVs 232 249 246 42 769 496 Health sectors 17 14 14 4 27 33 Source: MCDI BCC Manager, February 2007 Table 12 uses the same data source to show the absolute and cumulative distribution of the 33 health sectors in Toliara by the number of quarterly visits made by FAs in FY06. Only 3 % of health sectors received 4 quarterly visits by a FA, which would be necessary to supervise CHVs on a quarterly basis. Only 52% of health sectors received 4 , 3 or 2 quarterly visits, that is, 48% percent of health sectors received only 1 or no supervisory visit in 2006. The 6 health sectors that did not receive any supervisory visit by FA in FY06 might be health sectors with no CHV, or health sectors where CHVs are supervised by another organization. Table 12 Distribution of health sectors by number of quarterly supervisory visits received, Toliara II, 2006 Number of quarterly visits 4 3 2 1 0 Absolute frequency 1 3 13 10 6 % 3% 9% 39% 30% 18% Cumulative frequency 1 4 17 27 33 % 3% 12% 52% 82% 100% Source: MCDI BCC Manager, February 2007 In conclusion, the supervision of CHVs by FA has been far less frequent than expected in the DIP, which states that 80% of CHVs would be supervised on a quarterly basis. While it is possible that HW in BHC supervise CHV, it is not reported in the MCDI supervision records. CBDs The number and type of training received by CHVs to become CBD is discussed in section B.3.c.v. The volume of social marketing products provided by MCDI is described in section B.3.a. Medical Care Development International 51 TPCSP Final Evaluation After their training and their first donation of social marketing products, CBDs, are regularly visited by FA. During these visits, FA primarily check their stock of products but they do not formally supervise their activities. Table 6 shows that about 90% of CBDs had stock of the various social marketing products in Toliara II in September and October 2006, except for stock of Sur’Eau. This reflects the quality of management of the program by CBDs, MCDI and PSI. Were the performance objectives met? TPCSP did not set specific objectives in terms of performance of health workers in the BHC or of CHVs. The training and supervision provided to HW, and the improved environment in which they work as a results of strengthening health facilities (see section B.3.c.iii) certainly contributed to improving HW performances and achieving project objectives. Similarly, training and supervision of CHVs, and their support in conducting various community health activities, certainly contributed to improving their performance and achieving project objectives. What were the best practices and lessons learned? The FE Team found that HW performance benefits most from: • Continuing training by a pool of trainers from various levels of the health system • Regular follow-up and supervision What are the plans for sustaining health worker performance once the project closes? Are the sustainability plans realistic? The ROHFP and DOHFP in Toliara and Betioky will continue to use some of the TPCSP approaches to strengthening health workers performance: • The ROHFP adopted the Integrated Supervision tool and the formative approach developed, and to use it at the regional level • In Toliara II, supervision of CHVs is now included in the tasks and responsibilities of HW in BHC In addition, TREIP will also support these approaches, including training and integrated supervision, in all nine districts of the South West region. The MOHFP and MCDI are in the process of developing the Detailed Implementation Plan of TREIP. Were the tools used to assess the results of improving health worker performance sensitive enough to measure change over the life of the project? The tools that could be used to measure changes in health workers performance are the Monthly Activity Reports, the monthly EPI Reports, the CHV reports to the BHC, the IEC/BCC Reports of the DHO, the IMCI follow-up reports, the FA supervision reports. Several tools are used under TPCSP to assess HW performance and provide feedback or take corrective actions. For instance, the Monthly Activity Report of the BHC is used during the monthly planning meetings at the DOHFP level to review accomplishments and provide guidance on how to improve the services in the BHC. The Integrated Supervision tool allows direct observation and to provide feedback and on-the-spot training. So far only the IMCI assessment components of the series of HFA conducted in Betioky and Toliara II provide data to measure change in HW performance over the duration of TPCSP. Table 13 presents selected indicators from the IMCI assessment components conducted at baseline (2002 HFA) and at the end of project (Integrated Supervision) in Toliara and Betioky. Medical Care Development International 52 TPCSP Final Evaluation The size of the samples on which these data are based is given in Table 8. Overall, and ignoring sample size issues, most indicators presented, although not all, show a clear increase in performance. These IMCI assessments have therefore been sensitive enough to measure those changes in health workers performance. Table 13 Change in indicators of health worker performance in IMCI, Toliara II and Betioky, 2002 and 2006 Toliara II Betioky Health workers who: BL FE BL FE Looked for at least 3 danger signs 13% 81% 11% 86% Looked for symptoms of cough, diarrhea and fever 59% 57% 57% 86% Evaluated immunization status 50% 95% N/A 91% Correctly classified: Cough 21% 87% n/a 58% Diarrhea 53%* 73% 36% 71% Fever 13% 73% 20% 82% Correctly evaluated: Cough 6%* 67% 19% 42% Diarrhea 10%* 45% 27% 29% Fever 8%* 47% 0% 59% Use correctly IMCI protocols (PCM3) 0% 31% N/A N/A * Data from MTE HFA, 2004 Supervision visits allow the identification of changes at the individual level but so far no organized data for measurement of changes in health worker performance at the individual, district or BHC levels is available. The first round of supervision visits conducted in October and November 2006 yielded data of a quality that will allow trends analysis if further rounds are completed and generate the same data and indicators. TPCSP did not conduct regular assessment of the performance of trained CHVs using explicit standard of performance. Overall, the various tools to monitor CHV activities do not provide direct measures of the improvement of their performance over the life of the project.6 Just as the IMCI assessments discussed above, such measures require explicit definitions of performance standards and objectives, the development of tools to measure these standards, and the collection of the related data before the performance improvement intervention. Then comparable data need to be collected one or several times during the implementation of the intervention. Ideally, the sensitivity of these measures should be evaluated before attempting to assess health worker performance improvement to avoid inconclusive results. How did the project address the gaps between performance standards and actual performance? The MTE recommended defining quality of care standards. TPCSP did not undertake this task as it is part of the work under way within SantéNet. The main standards used in the HFA or the Integrated Supervision tools are those of the national IMCI strategy. Poor performance of health workers in respect to such standards is addressed during micro-planning meetings at the District level, when specific training, supervision and 6 Indirect measures such as the increase in the coverage of related activities are discussed in other sections of the report. Medical Care Development International 53 TPCSP Final Evaluation supply needs can be identified. The needs are also addressed at the individual level during formative supervision visits. v. Training How effective was the training strategy? The TCSP training strategy can be considered effective in so far as most project objectives were met. TPCSP did not conduct formal evaluation of the training strategy and activities. Were the training objectives met? The DIP specifies training objectives in terms of types and number of trainees but not in terms of knowledge or skills to acquire or change, or in terms of impact on trainees’ performance. TPCSP did not conduct a training needs assessment or develop a training plan that specified such training objectives. Appendix 2 shows that most of the training of trainers and of HWs was completed by the beginning of the second year of the project. Training of CHV and CBD began in the second year of the project continued until the end of FY05. Most of the information on the training activities of TPCSP in this section comes from the 2006 Benchmarks (see Appendix 5) and the MTE report. MCDI staff also confirmed or provided additional information during their review of this report. Although this information is available in the MCDI tracking system, there is no compilation and analytical report of all the TPCSP training activities. The compilation gathered here after several iterations of verifications and complement of information still contains inconsistencies, but serves its purpose of a general review of the training activities conducted under TPCSP. Table 14 summarizes the number of trainees by topic and category under the TPCSP. Medical Care Development International 54 TPCSP Final Evaluation Table 14 Main training activities under TPCSP Trainees Theme/topic Toliara II Betioky Total # Date # Date CHV Communication/ENA 472 FY04 432 FY04 904 IMCI 505 FY04 465 FY04 970 EPI 458 FY04 412 FY04 870 CS 503 FY05 465 FY04 968 Visa 466 FY04 345 FY04 811 AIDS 486 FY05 153 FY04 639 CBDA 5 social marketing products 20 165 Q4-FY04 Q3-FY05 50 FY04 235 TBA Safe motherhood and child health 53 FY04 57 FY04 110 ToT CHV 60 Q4-FY03 60 ENA 60 Q3-FY03 26 Q1-FY03 86 EPI 60 Q4-FY03 00 60 FP 32 Q4-FY03 00 32 EPI epidemiologic surveillance 60 Q3-FY03 00 60 IMCI 31 Q1-FY04 24 Q4-FY03 55 Malaria 60 Q1-FY04 19 Q1-FY05 79 HW EPI management of vaccines 60 Q1-FY04 00 60 ToT CHV 0 Q2-FY03 0 0 ToT IMCI (with 7 ITHCP trainers) 11 Q1-FY04 3 Q1-FY05 14 IMCI supervision 11 Q4-FY03 3 Q1-FY05 14 DHMT Malaria 11 Q4-FY03 3 Q1-FY05 14 NGO/RAD TOT 2 Q1-FY03 1 FY03 3 Malaria 2 Q1-FY03 2 FY03 3 Source: 2006 Benchmarks, MCDI, February 2006; MTE report, MCDI, 2005; HIS Manager, February 2007. Specific features and achievements of the training provided to each type of trainees is provided below. CHVs TPCSP trained 6 FA and 6 DHMT or RAD staff as trainers in Q3 and Q4 of FY03, who then conducted the CHV training by team of 2, at least one of which being a FA. TPCSP identified and trained a total number of 548 community members as CHVs in Toliara II and 623 in Betioky. Most of this training was conducted in the second and third years of the project. In Betioky, most CHVs had been trained under BSCSP and therefore only received refresher training under TPCSP. Table 15 shows the initial number of CHVs identified and trained in Toliara II at the launch of the project and the number of CHVs trained afterwards to replace those who resigned or died after their training. At the end of FY04, TPCSP had trained a total of 548 CHVs and there were still 496 functional CHVs. Medical Care Development International 55 TPCSP Final Evaluation Table 15 Number of trained and functional CHVs in Toliara II in FY04 Initially Trained Deceased Resigned Newly Trained Total Trained FY04 Functional CHV FY04 Number 484 5 47 64 548 496 % Total trained 88% 1% 9% 12% 100% 91% Source: MCDI BCC Manager, February 2007 The CHV training typically consisted of a three-day curriculum. The various topics of these training workshops and the number of CHV trained and retrained in Toliara are presented in Table 16. The TPCSP also organized refresher-training events for more than 80% of the trained CHVs in each topic in Toliara. Table 16 Trained and retrained CHVs by topic in Toliara II Total TOPIC BCS/ ENA IMCI VISA EPI FP AIDS Number Trained 548 472 505 466 458 503 486 % Total Trained 100% 86% 92% 85% 84% 92% 89% Number Retrained n/a 408 466 395 370 0 0 % Trained n/a 86% 92% 85% 81% 0% 0% Note: BCS: Basic communication skills; ENA: Essential Nutrition Actions, including exclusive breastfeeding; IMCI: includes ARI, Diarrhea, Malaria; VISA: VISA Approach; EPI: Expanded Program in immunization; FP: Family Planning CBDs TPCSP trained a total of 185 CBDs in Toliara II, as proposed in the DIP. A first group of 20 CBDs were identified and trained in Q4 FY04 and the others in Q3 and Q4 of FY05. The training included home-based management of malaria with chloroquine (PaluStop) the use of ITNs (Supermoustiquaire), counseling on oral contraceptives (PilPlan) and condoms (Protector), the use of safe water (Sur’Eau), and use of the various tools for the management of the social marketing of these products. PSI trained MCDI trainers on all social marketing health products and MCDI trained the CBDs. TPCSP also trained 50 CBDs in Betioky. TBAs MCDI staff and the DHMT in Toliara and Betioky introduced 28 and 21 HW, respectively, from these two districts to the MOHFP training module for TBAs on safe motherhood and child health. Then MCDI staff, DHMT members and HWs trained a total of 53 and 57 TBAs in Toliara and Betioky, respectively, in FY 2004. TPCSP also introduced a new type of report, specifically designed for illiterate TBA, to be used instead of the existing monthly report. Follow-up of trained TBAs is under the responsibility of the HWs of the BHCs where they live. Health workers The TPCSP provided various training opportunities to HWs from the BHCs in Toliara and Betioky. Most of this training was completed by December 2003, except for the malaria training conducted in February 2005 in Betioky only. Training of HWs included: • Training of Trainers, with a focus on training of CHV Medical Care Development International 56 TPCSP Final Evaluation • IMCI: In Toliara II, TPCSP trained 31 HW in clinical IMCI. This training was conducted in 6-day sessions including half-time theory and half-time practice in the regional hospital. The training included clinical competencies, and organization of health services to allow effective IMCI and C-IMCI. The training included a follow-up visit, during which IMCI kits were provided, and was followed by a series of supervision visits on a quarterly basis. In Betioky, TPCSP provided quarterly group follow-up of HW previously trained in IMCI, and 2 to 3 day refresher training focusing on gaps identified during follow up sessions to 24 HW. The total number of HW trained or retrained in IMCI in the two districts is therefore 55. • Essential Nutrition Action: TPCSP trained 26 HW in ENA in Betioky, in collaboration with the Linkages Project. • Expanded Program on Immunization: TPCSP provided 3-day refresher training to a total 60 HWs in Toliara II and Betioky during the monthly review meetings of the DOHFP. • EPI epidemiologic surveillance • Family Planning: In Toliara, TPCSP retrained 32 HW in FP promotion and services. This five-day training was based on a module of the MOHFP and included FP HIS and CBD. • Malaria: training of 19 HW and 7 DHMT members in Betioky and 60 HW in Toliara in malaria case management and prevention, including intermittent preventive therapy in pregnancy and ITNs. This training was conducted in Betioky in response to a need identified during supervision visits. TPCSP adapted and made available for use modules for training HW in C-IMCI in 6 days. These modules were adapted from the national modules used at that time and that included an 11-day program. The TPCSP used HFA as the main tools to assess the performance of the BHC HW (see section B.4.g). The HFAs have measured a series in IMCI practice indicators of health workers performance, which, overall, show definite improvements (see section B.3.c.iv). The composite indicator of IMCI competence of HW chosen in the DIP as project indicator (PCM3) shows improvement at midterm but then remains stable until the end of the project. DHMT members In Q2-FY03, 6 DHMT members were trained as trainers of CHVs, at the same time as MCDI FA. In Q1-FY04, MCDI, Linkages and ROHFP staff trained 11 trainers in clinical IMCI (4 DHMT members and 7 trainers from the Training Institute for Health Care Providers in Toliara). This training also included ENA, EPI, and CS. DHMT members reported to the FE Team that overall, each program manager received some training from TPCSP, primarily in IMCI. For EPI, training consisted mainly in technical support. Medical Care Development International 57 TPCSP Final Evaluation NGOs Various members of the TPCSP’s local NGO partners received training in the key interventions (EBF, IMCI, ENA EPI, FP and HIV/AIDS), in basic communication skills, and in training of trainers of CHVs. Members of VEMIMA also received training in budget preparation. Members of local NGOs (VEMIMA and Mampifoha) interviewed by the FE Team expressed their confidence that they now have the competence to continue implementing these activities. What evidence is there that suggests that the training implemented has resulted in new ways of doing things, or increased knowledge and skills of the participants? The success of TPCSP in achieving most of its objectives suggests that most of the training was effective. TPCSP did not conduct formal assessment of CHV performance. The FE Team conducted group interviews with CHVs in two communities in the two project districts and found that they had the competencies expected in the area of CS and HIV/AIDS. The series of HFA surveys conducted at baseline, midterm and at the end of the project show clear improvements in IMCI clinical skills of HWs (see section B.3.c.iv). What were the best practices and lessons learned? The FE Team found the following lessons learned in terms of training: • The development of a supervision tool is necessary before any training • Formative follow-up and supervision in necessary after any training • The evaluation of training programs is difficult What are the plans for sustaining these training activities once the project closes? Are the sustainability plans for training realistic? TREIP will continue supporting some of the TPCSP training and supervision activities at the regional level in all districts; the MOH and MCDI are in the process of developing their Detailed Implementation Plan. DHMT and BHC staff all were trained as trainers and able to conduct some training when needed. The MTE recommended the adoption of formative supervision, and this approach has been adopted during the last round of supervision visits of xx BHC in Toliara and Betioky. d. Sustainability Were the sustainability goals and objectives that were articulated in the DIP met? How did the initial sustainability plan (if there was one) evolve through the implementation of the project? The TPCSP DIP does not include a sustainability plan other than the table in an Annex to the DIP, which comes from the original TPCSP proposal and is revised according to the proposal reviewers’ comments. The three sustainability objectives formulated in that table can be considered as achieved, although only none of the five results-based objectives is clearly achieved. The results-based sustainability objective # 1.1 related to expansion of the accessibility to health credit was unrealistic and is not achieved. The formulation of the results￾Medical Care Development International 58 TPCSP Final Evaluation based sustainability objectives # 2.1 is unclear; the results-based sustainability objective # 2.2 is not achieved according to the 2006 Integrated Supervision data. The results-based sustainability objectives # 3.1 and # 3.2 have not been pursued as such: The enrollment of CHVs in officially recognized associations has been considered a barrier rather than an incentive to CHV recruitment and sustainability, and the involvement of school teachers in child survival programs is a project component that has been transferred to another NGO Aide et Action. The TPCSP also includes an implicit sustainability strategy, and the achievements with this respect can be summarized as follow: • Families have adopted new healthy behaviors • Community structures are in place to promote and support these behaviors • HW in BHCs and the DHMT in Toliara II and Betioky have improved their capacity to manage childhood illnesses, conduct BCC activities, manage health programs and conduct supervision • The ROHFP staff and the DHMT have improved their capacity in supervision and in the provision of IMCI kits, tools, essential drugs, vaccines, contraceptives, and kerosene • Appropriate approaches and tools are available (VISA approach, BCC materials, training modules, supervision checklist) • A network of actors, including local NGOs, works synergistically on child survival in Toliara II and Betioky What is the status of the phase-over plan, and is it on schedule? After the project, will there be any continuing technical and management assistance? MCDI did not develop an Exit Plan because TPCSP activities and technical assistance will be continued under TREIP in the nine districts of the South West Region. Have the approaches to building financial sustainability-- (e.g., local level financing, cost recovery, resource diversification, corporate sponsorships) been successful? TPCSP did not develop a specific financial sustainability plan. The two following activities have financial sustainability objectives: • TPCSP has assisted VEMIMA in the development of income generating activities and a membership system. VEMIMA already has opened a bank account for these funds. • TPCSP’s support to the community-based Health Care Financing Insurance in the Commune of Ankazomanga (population 3,200) is briefly described in section B.2.d.iii. The extension of this pilot project to three new Communes was delayed until to January 2006 because of changes in the national cost-recovery policy. Although the TPCSP approach to community health and mobilization is widely recognized by the MOHFP and its partners as successful and replicable, the public funding of this type of activities remains insufficient. TPCSP has not fully detailed the contributions of TPCSP to the CHVs program, which will need continued support to ensure sustainability. The same holds true for in-service training and supervision of HWs. How has the project built demand for services, and is the community sufficiently engaged to influence how services are delivered? The increase in several TPCSP indicators (PCM1, MAL3, CDD4, IMM1, IMM2, CS1) supports the increased demand for health services in Toliara II and Betioky. Some BHCs have noticed an Medical Care Development International 59 TPCSP Final Evaluation increase in the use of their services, like in Miary and Ambohimahavelona where the average number of outpatient visits more than doubled between 2001 and 2006. Section B.3.c.iii presents various ways TPCSP fostered links between communities to BHCs, and thereby community demand for health services. Communities otherwise are not fully engaged in the management and quality improvement of health services. 4. Program Management a. Planning How inclusive was the project planning process and what effect did this have on the implementation process? The TPCSP DIP preparation took about 2 and a half months and included a series of meetings and workshops. The DIP lists all the participants to this planning exercise, which include staff from the ROHFP, the two DHMT from Toliara II and Betioky, the Ministry of Communication and Youth, local NGOs, representatives of the community and local authorities, staff from MCDI Madagascar, and the USAID Madagascar Child Survival Manager. All these categories of participants, and often the same individuals, have then been involved in the implementation of the project and in the midterm and final evaluations. With changes in staff, however, the FE Team found that only one current DHMT member in Toliara had participated in the DIP workshop in 2003. The FE Team found that the DIP document was available in English at the DOHFP level in Toliara II and at the ROHFP level, but not actually used for planning purposes. During TPCSP implementation, however, all activities involving the DHMT and HWs in BHCs were included in the DOHFP annual action plans in Toliara II and in Betioky through the active participation of MCDI staff in their annual and monthly planning meetings. DOHFP staff members acknowledged to the FE Team that their participation to the TPCSP planning process gave them a common vision of the project and developed a good team spirit and collaboration. Although not directly involved in most of TPCSP activities, the RHFPO staff also acknowledged the importance of this participation. Some VEMIMA staff members participated in the development of the DIP in 2003. VEMIMA does not have a copy of the DIP and reported that it would have been useful to better understand and contribute to the project. No member of Mampifoha participated in the development of the DIP. To what extent was the DIP work plan practical? MCDI staff reported to the FE Team that although the DIP was sometimes ambitious, they found that about 85% of activities were implemented as planned. One example of an ambitious activity was the plan to supervise 85% of CHVs on a quarterly basis. As the DIP was not translated, it is likely that it has not been directly used by most MCDI and partners’ staff. Medical Care Development International 60 TPCSP Final Evaluation Based on the grantee’s and its partner’s experience with this project, what could be added to the DIP preparation and review process that would have strengthened implementation? The DIP, overall, served its purpose although it did not include specific objectives, indicators or strategies for sustainability, capacity building, or training. The FE Team also found that the DIP did not give enough emphasis on supporting mass campaigns. Based on MCDI/Madagascar’s experience with TPCSP, the DIP development process could have been improved by involving the central level of the MOHFP. It is also clear that a translation and maybe be a practical summary of the final document would have been useful to all partners. b. MCDI Staff Training What change is there in the knowledge, skills and competencies of the project and partner's staff? All MCDI staff acknowledged that they increased their knowledge and skills for child survival interventions and programs though their work on TPCSP. The list of specific in-service training each professional staff member received between October 2003 and September 2006 is presented in Appendix 10. All this training was highly relevant to their positions and provided in a timely manner. The Project Administrator, on staff since 1999, did not receive any specific training beside USAID grant management, and may benefit from advanced training in project management. The training of MOHFP and NGO staff is discussed in section B.3.c.v. Is there evidence that the staff has applied these skills both within the project and in another context? The effective and successful implementation of project activities suggests that all MCDI staff used the training that they received. All MCDI staff is well appreciated by their partners for their technical skills and their collaborative and supportive approach. All MCDI professional staff acknowledged that the training received under TPCSP has been very useful in their work and implementation of the project activities, and this experience has been invaluable in their professional development so far. Several key MCDI professional staff members were promoted within MCDI Madagascar or recruited by other organizations to work on similar projects. Their up-to-date and successful experience on the TPCSP certainly contributed to their professional development. Most of the MCDI staff hired in Toliara decided to build on the skills and experience gained while working on TPCSP to create a NGO (Miainga) and prepare to apply for grant and implement similar projects (see section B.3.c.ii). Were adequate resources dedicated to staff training? The DIP did not specify any training plan, nor did it include a specific budget line for MCDI staff training. All the training that the staff received was funded under the general training component and provided when the suitable opportunity occurred. MCDI also allowed some of the project staff to participate in outside training programs that were relevant to their responsibility on the project and to their professional development. Medical Care Development International 61 TPCSP Final Evaluation What are the overall lessons learned about building the capacity of project staff? MCDI/Madagascar staff found that they could have more systematically shared among themselves the information and experience gained through their respective training opportunities. The review of the MCDI staff training and development suggests that effective capacity building requires a staff training needs assessment and plan tailored to the specific objectives of the project; a supervision system to ensure the transfer of learning and identify related constraints; and the provision of technical assistance to complement and reinforce the application of the new knowledge and skills. Such staff development plan should take into account the inevitable staff turnover. Lessons learned on building capacity of the MOHFP and local NGO is discussed in section B.3.c.ii. c. MCDI Staff Supervision Was the supervisory system adequate? The supervision with the health workers in BHCs in Toliara and Betioky is discussed in section B.3.c.iii and B.3.c.iv and the supervision of local NGO partners is discussed in section B.3.c.ii. MCDI/Madagascar does not follow a formal system of supervision and evaluation of its staff. Rather, the technical managers submit monthly reports to the Project Manager, and all senior staff meet monthly for several days in Toliara to review progress and adapt or develop new action plans together. All TPCSP staff also meets on a quarterly basis. Is the supervisory system fully institutionalized and can it be maintained? This MCDI/Madagascar supervision system and working style has satisfactorily met the needs of the TPCSP management and the needs of the technical managers in terms of technical and professional guidance. Is there evidence that the project’s approach to strengthening supervisory systems has been adopted beyond the project? TPCSP strengthened the supervision system in Toliara and Betioky, and the approaches and tools developed and used under TPCSP are adopted at the regional level and will be implemented under TREIP. d. Human Resources Management The MCDI professional staff comprised 1 Project Manager, 1 BCC Manager, 1 QOC Manager, 1 HIS Manager, 1 Administrator, 6 Field Agents and support staff. In addition, the Ministry of Communication and Youth assigns two Rural Animation Delegates (RAD) to the project. Appendix 2 shows the TPCSP organigram including MDCI staff and the relationships with all the institutional and community partners. Appendix 12 lists the individual professional and support staff of the TPCSP from the beginning to the end of TPCSP, with the start and end date of their employment, as appropriate. Medical Care Development International 62 TPCSP Final Evaluation Are essential personnel policies and procedures of the grantee and partner organizations in place, to continue project operations that are intended to be sustainable? The FE Team found that MCDI Madagascar has clear and sound personnel policies in place. The Human Resources Policy is accordance to national laws and MCDI’s policies. Describe the morale, cohesion and working relationships of project personnel and how this affected project implementation. The FE Team confirmed the finding of the MTE that the TPCSP enjoyed a very motivated and competent technical and managerial staff. Working relationships and communication between the Project Manager and the technical staff are open and supportive, and individual professional development is consistently valued and encouraged. Teamwork has in general been constructive, without major conflict, and follows the leitmotiv that there is “no success without social life!” Representatives of the local NGOs, the two DOHFP and the ROHFP interviewed by the FE Team all reported a satisfactory working relationship and atmosphere with MCDI staff, and that this contributed to a successful implementation of the project. Describe the level of staff turnover throughout the life of the project, and the impact it has had on project implementation. Twelve technical and managerial staff members from the 1998-2002 Betioky CSP remained employed on the TPCSP. Only two key technical staff members changed during TPCSP implementation: the HIS Manager in 2005 and the Quality of Care Manager in 2006. These changes corresponded to career advancements for the staff members involved, and suggest that their work experience under TPCSP was valuable. Their departure and replacement, however, required additional training and staff development investments from the MCDI Madagascar, and have sometimes led to gaps in services. Have plans been developed to facilitate staff transition to other paying jobs at the end of the project? MCDI did not develop staff transition plan because all TPCSP staff will be maintained under TREIP. e. Financial Management Discuss the adequacy of the grantee’s and partners’ financial management and accountability for project finances and budgeting. MCDI financial management has been excellent and transparent. No partner expressed any significant complaints to the FE Team. MCDI technical managers were all trained in budgeting. They submit their plans and budget for approval by the Project Manager and Project Administrator and then manage entirely their own activities. MCDI financial staff acknowledged that on 2 or 3 instances they lacked the funds to implement scheduled activities, and that these activities had to be delayed. The reason for this delayed availability of funds is that MCDI financial staff always face serious difficulties in submitting all the justification for the funds spent according to the deadline, and that the funds for a new period is only released by MCDI headquarters when all funds from the previous period are justified. Medical Care Development International 63 TPCSP Final Evaluation Several TPCSP partners expressed their concerns with specific aspects of the MCDI policy on per-diem, such the amount provided (which has remained the same throughout the duration of the project and sometimes is three times lower that that provided by other donors in Madagascar) and the requirement to provide a receipt for lodging expenses in areas where there is no hotel (this last requirement means no compensation for agents assigned to work in these areas, which is perceived as a disincentive). If the project budget was adjusted, explain why. TPCSP project has respected its budget so far and there has not been any change in the budget during the duration of the project. Do the project implementers have adequate budgeting skills to be able to accurately estimate costs and elaborate on budgets for future programming? The FE Team agreed that all the technical and management staff from MCDI have the skills to estimate costs and prepare budgets for project activities. This is also true for TPCSP partners (RHFPO, DHMT, NGO), for whom this is a routine activity under TPCSP with all donors. Are adequate resources in place to finance operations and activities that are intended to be sustained beyond this cooperative agreement? Beside the MOHFP and its partner’s resources, the main source of external funding to continue TPCSP activities will be TREIP. Was there sufficient outside technical assistance available to assist the grantee and its partners to develop financial plans for sustainability? MCDI Madagascar staff did not receive any technical assistance in the development of financial plans for sustainability. f. Logistics What impact has logistics (procurement and distribution of equipment, supplies, vehicles, etc.) had on the implementation of the project? TPCSP faced enormous logistical constraints due to the distance between sites and the poor conditions of the roads, particularly when heavy rains make them inaccessible to any vehicles. However, the FE Team found that overall the logistic support available to the TPCSP has been adequate at the MCDI level, where two vehicles have been available from the previous project (1 is three-year old and the other 7-year old). Inevitably, there have been instances where more than two vehicles were needed at the same time, and other solutions were found to implement the activities as planned (vehicles from the ROHFP or DOHFP, taxis, oxcarts or feet). The project did not provide motorbikes to the Field Agents, who traveled to their respective site with the project vehicles or by buses, oxcarts or feet. Local NGOs also reported that they do not have any transportation means and that they spend a lot of time waiting (sometimes days) or traveling in local taxis or oxcarts or feet. TPCSP did not provide vehicles to the DOHFP or the ROHFP but was regularly able provide transportation for project related activities, which is recognized to have been very helpful by the DHMT and the ROHFP staff. TPCSP rehabilitated one office room in the ROHFP. Medical Care Development International 64 TPCSP Final Evaluation TPCSP uses a total of 14 computers: 5 individual computers for the professional staff and the project secretary, 2 for Field Agents, 2 for the DOHFP in Betioky, 2 for the DOHFP in Toliara II, and 3 in the Antananarivo office. These computers are often out of service for technical problems, viruses, or power shortages. Internet access remains difficult in Toliara and Betioky. TPCSP uses 5 different offices in Toliara (3), Betioky (1) and Antananarivo (1). The DHMT in Toliara reported to the FE Team that the logistic support from TPCSP (vehicles, fuel, meeting rooms, training materials, televisions and videocassette player, computers) has been very useful and contributed to achieving project objectives. Is the logistics system sufficiently strong to support operations and activities that are intended to be sustained? Both DHMT and the RHFPO staff report the need for increased resources for transportation and logistical needs to continue activities implemented under TPCSP. TREIP will provide new logistical resources at the regional level. g. Information Management The TPCSP monitoring and evaluation system includes: 1. The routine MOH Health Information System: primarily the monthly activity reports of Basic Health Centers and their monthly and annual DHO compilation; more recently, the Integrated Supervision report; 2. The project Community Health Information System: monthly activity reports of CHVs, submitted to the BHC and forwarded to the DHO. 3. The project monitoring tools: KPC and HFA surveys, reports of CHVs and CBDs supervision by FAs; senior staff and FA activity reports; and annual reports; evaluation reports. How effective was the system to measure progress towards project objectives? The TPCSP HIS, especially the KPC and HFA surveys, have been very useful in planning activities at the beginning of the project and in adjusting strategies and activities during project implementation. At midterm, for instance, the results of the KPC using the LQAS methods allowed the identification of the project implementation areas and the interventions that were lagging behind. For instance, the use of ITNs and PaluStop and the knowledge of HIV prevention were the indicators of interventions found to need more efforts, and the project areas along the coast were identified as those where the use of ITN was the lowest. The ROHFP staff also found that recommendations from the MTE were useful in making adjustment to their programs. It seems that the usefulness of the indicator PCM3 should now be evaluated on the basis of the experience of the several surveys measuring it. This could be done at the beginning of TREIP in preparation of the future surveys. Was there a systematic way of collecting, reporting and using data at all project levels? Cite examples of how project data was used to make management or technical decisions. The TPCSP measured progress towards the technical interventions’ objectives through a series of three KPC and HFA surveys. These surveys were successfully completed at baseline, midterm Medical Care Development International 65 TPCSP Final Evaluation and the end of the project. The results of the surveys were available and used for the DIP workshop, the MTE and the FE. At midterm, findings that several key indicators seemed to lag behind urged project managers to reorient the efforts and related strategies of the project (See above). TPCSP used data collection and reporting at the community, BHC and DHO levels. At the community levels, CHVs submit report of their activities to the BHCs, giving the opportunity to the HWs to follow and provide feedback and reorient their activities as needed. BHCs submit their monthly reports to the DHO, where the DHMT prepares monthly and annual district level reports. This monthly report includes data on CHVs activities. All this information is available to MCDI staff as needed. One key event during which this information is used is the monthly review and planning meetings, when all HW from the BHC come to present and discuss their past and future activities. MCDI has been instrumental in the preparation and facilitation of these meeting in Toliara II and in Betioky. MCDI Field Agents report their CHV supervision activities in a Field Visit report (“rapport de descente”) that includes quantitative and qualitative data. These data are entered and analyzed by the HIS Manager, who then prepares reports of this information to share with the BCC Manager, and other MCDI staff as needed. All the MCDI Managers prepare a report of their activities that are available to the other Managers and the Project Manager. The TPCSP HIS Managers have developed a series of databases to manage and retrieve project activity tracking data such as training, supervision, social marketing products movements. Although these databases are typically relevant and well conceived, they often exist as occasional tools and work without a complete design, documentation, validation and systematic purpose and use. Several of these databases were made available to the FE Team as answers to specific questions, and their analyses and interpretation required many assumptions regarding their data source, completion, quality or even date. At the DOHFP level in Toliara, several DHMT members have direct responsibilities for data collection, management and reporting, but the FE Team could not find any consolidated report including critical analyses and interpretation of these data. Trends analyses seem especially lacking. Is the project staff sufficiently skilled to continue collecting project data/information and to use it for project revisions or strengthening? Several MCDI staff have advanced skills in collection, analysis and utilization of data such as those from KPC, LQAS, HFA, and other sources. Activities of data collection, analysis and use require substantial amounts of time from these qualified TPCSP staff, from the FA collecting the data to the HIS Manager entering and analyzing them and to the other Managers interpreting and making the appropriate decisions. No specific provision or plans to ensure the successful transfer and continuation of the necessary information system had been made at the time of the FE, except for TREIP, which will continue support for such activities at the district and regional levels. VEMIMA reports being sufficiently competent in collecting and utilizing data to continue implementing project activities but still needs support, particularly financial. The FE Team noted, however, that local NGO partners typically do not systematically collect or use data on their activities but tend to rely instead on oral reports from CHVs. Medical Care Development International 66 TPCSP Final Evaluation Did the project conduct or use special assessments, mini survey focus groups, etc. to solve problems or test new approaches? Give examples of the research, use of data, and outcomes. TPCSP conducted a series of KPC surveys and HFA, as described and used in several instances in this FE report. The TPCSP DIP proposes to carry out four operations research activities: • Testing a health information tool to enable health facilities to track and report the number of completely immunized children. • Development of a supervisory tool that focuses on quality of care and IMCI protocols and study of the impact of this tool on quality of care and adherence to IMCI protocols • Health care financing insurance • Census-based tracking of individuals involving CHVs TPCSP did not develop formal protocols to conduct these research projects. TPCSP successfully developed and used an Integrated Supervision tool in 2006, but its impact has not been tested. The health care financing insurance pilot project started in BSCSP in Ankazomanga was interrupted in 2002 because a change in the related national policy (see section B.2.d.iii). It gradually resumed in 2005 and was replicated in three new communes in 2006. MCDI has also been involved in two data collection and research efforts with INSPC: • Analysis of service delivery for children under 5 • Evaluation of the TBA activities To what extent did the project strengthen other existing data collection systems (i.e. government)? MCDI helped analyze and use the data collected by the BHCs and transmitted to the DHO by preparing and participating in the monthly review and planning meetings of the DHO. TPCSP also helped developing and using the Integrated Supervision tool in Toliara and Betioky. TPCSP has established and strengthened the data collection systems at the community level (CHV reports to the BHCs). However, MOHFP staff often report that TPCSP used a community health HIS that is different from the one proposed by the MOHFP, that often CHV report directly to MCDI rather than to their BHC, and that this is a source of confusion for many. Do the project staff, headquarters staff, local level partners, and the community have a clear understanding of what the project has achieved? MCDI staff in Madagascar and in the headquarters have a good knowledge of the TPCSP achievements. The TPCSP staff acquired a clear understanding of what the project achieved in terms of health objectives through the series of KPC and other surveys and through the project tracking system. MCDI staff also has a good understanding of the overall progress and achievements and the constraints faced by the project within its national and international context—these were clearly outlined n the Expanded Impact Grant application proposal submitted to USAID in November 2005. Medical Care Development International 67 TPCSP Final Evaluation Local NGO staff, however, do not have a good understanding of the project’s progress towards its objectives. VEMIMA reports that they do not receive regular feedback on TPCSP progress but that they obtain such information from the BHCs. For instance, they obtain information on immunization coverage from the BHC, and then decide on appropriate actions in terms of tracking dropouts. How have the project’s monitoring and impact data been used beyond this child survival project? Data from the monitoring and evaluation system have been used outside of the project as follows: • Presentation of project achievements at the CSTS Mini-University • Presentation to USAID Madagascar during midterm and final evaluations • Presentation of the health care financing insurance pilot project during “Journées Nationales de Santé” (National Health Days) • Preparation of the application to USAID for the funding of TREIP h. Technical and Administrative Support Discuss types and sources, timeliness, and utility of external technical assistance the project has received to date. Appendix 2 shows the timing of the in-country technical and administrative assistance provided to TPCSP. At the beginning of the project in 2003, TPCSP obtained local technical assistance to develop BCC messages and tools for the new child interventions included in the project. In 2005, three MCDI Headquarters staff visited the project: the Director, the administrative officer, and the health economist. No technical backstop CS officer travel to Madagascar during the TPCSP duration. In 2006, TPCSP obtained local technical assistance for the documentation of the VISA approach (see section B.2.d.i) and to conduct the organizational assessment of VEMIMA (see section B.3.c.ii). The FE Team did not review any finalized report of the technical or administrative assistance received. In addition, MCDI/Madagascar receives regular technical and administrative assistance from the MCDI/HO though email, phone calls, during their regular visits to the US and the HO. What assistance did the project need that was not available? The FE Team identified the following areas for which technical assistance would have been beneficial to the project and beyond: • Documentation: TPCSP developed and implemented several innovative and successful approaches such as setting up of a network of CHV and the VISA approach, the health care financing insurance, health festivals, and the adaptation and implementation of C-IMCI. Technical documentation of the implementation, tools, impact, factors of success and cost of these approaches would be beneficial to all partners considering their replication and scale-up in Madagascar and elsewhere. • Training needs assessment and planning: Training was a major activity proposed in the DIP. The development of a training plan based on a complete training needs assessment and including specific performance objectives and specifically linked to project objectives helps program managers in the implementation, monitoring and evaluation of large-scale training Medical Care Development International 68 TPCSP Final Evaluation program. Such technical assistance should include a thorough review of the training tools and programs developed to draw lessons learned so far. • Organizational Development: Capacity building of project partners was a key strategy of TPCSP but no formal assessment was conducted and no organizational development plan was developed under TPCSP. These assessments are potentially very useful if followed up with action plans and repeated after a year or two to assess improvement in organizational capacity, draw conclusions on the effectiveness of the efforts made so far and develop new objectives and action plans. • Health information management, including documentation of the various components of the HIS, validation of the quality and completeness of the data, consolidation of the available data, and reporting and publication of validated, analyzed and interpreted data. The local NGOs did not report any unmet need for TA to the FE Team. How could grantee headquarters and/or USAID better plan for the technical assistance needs of grantee? MCDI could improve planning for technical assistance by conducting regular needs assessment, and ensuring full benefits of the assistance by producing and validating the related final reports and by following up the implementation of recommendations. Discuss grantee headquarters and regional technical and managerial support of the field project. Approximately how much time has been devoted to supporting this project? The number of field visits by MCDI HO is described at the beginning of this section. MCDI Director typically directly communicates with the Project Manager for most key project management issues. The CS Specialist, the International Health Program Assistant, and the Financial Officer assist him. The FE Team was not able to estimate the time that MCDI HO devoted to TPCSP. MCDI/Madagascar staff acknowledged to the FE Team that they received adequate technical and administrative support from the MCDI Headquarters. i. Mission Collaboration The FY 2003 – 2008 Strategic Plan is composed of four strategic objectives (SO) and TPCSP fully supports the Health, Population and Nutrition SO2 “Use of Selected Health Services and Products Increased, and Practices Improved” by increasing the demand for selected maternal and child health care services and products and promoting behavioral changes. MCDI is a partner of the SantéNet consortium implementing the main bilateral project, and supports 10 “Kôminina Mendrika / Communes Champions” in the South West Region including 4 in Toliara and 4 in Betioky. MCDI also contributes to the social marketing program implemented by PSI by training and supporting CBDs in the distribution of 5 products in the TPCSP area. MCDI also contribute to Performance and Quality Improvement components of SantéNet in 8 BHCs in Toliara and Betioky. USAID/Madagascar has been closely involved in and supportive of the MCDI program in the Toliara Province since the first CSHGP grant. MCDI staff regularly met with USAID staff during TPCSP (see Appendix 4) and the USAID Madagascar Child Survival Manager often participates in project meetings in Antananarivo and Toliara. USAID/Madagascar often quotes Medical Care Development International 69 TPCSP Final Evaluation results from surveys and the various innovative approaches of TPCSP as successes of its programs j. Management Lessons Learned • TPCSP experienced delays in receipts of quarterly funds owing to delays in the field and tight deadline for justification of funds. Explore alternatives to ensure timely funding Medical Care Development International 70 TPCSP Final Evaluation C. Conclusions and recommendations a. Success in meeting objectives Table 17 summarizes the achievements in meeting TPCSP objectives in Toliara and in Betioky for the 22 key project indicators. The values of these indicators in Toliara are presented in section B.1 and discussed in section B.2.b. The values in Betioky are presented and discussed in section B.2.c. Table 17 Summary of Achievements of TPCSP Objectives in Toliara and Betioky Toliara Betioky # Indicator1 Versus TPCSP Objectives2 Versus Toliara II Results3 PCM1 Early treatment seeking for rapid/difficult breathing YES YES PCM2 Knowledge of at least two danger signs YES NO PCM3 Clinical staff who use correctly IMCI protocols NO YES MAL1 Children sleeping under an ITN YES NO MAL2 Mothers who give correct treatment for fever NO YES MAL3 Mothers who took antimalarials during pregnancy NO NO MAL4 CBDs with no stock-outs of antimalarials YES YES CDD1 Children with diarrhea who are given more fluids NO YES CDD2 Children with diarrhea who are given same or more food YES YES CDD3 Children with diarrhea who are given same or more breast milk YES YES CDD4 Early treatment seeking for children with diarrhea YES YES CDD5 Mothers who can cite at least 2 danger signs of diarrhea YES YES IMM1 Children 12-23 months completely immunized YES NO IMM2 Mothers who received two doses of TT during last pregnancy NO YES IMM3 Immunization drop out rates YES NO BRE1 Exclusive breastfeeding YES YES BRE2 Early initiation of breastfeeding YES YES BRE3 Complementary feeding YES YES CS1 Contraceptive prevalence among mothers of children under 2 YES YES CS2 Knowledge of HIV prevention NO YES CS3 CBDs with no stock-outs of condoms YES YES CS4 Knowledge of exclusive breastfeeding as child spacing method NO YES Total YES 15 17 Total indicators 22 22 % 68% 77% 1 See full definition of indicators in Results: Summary Chart, section B.1 2 Objective met if included in confidence interval of 2006 KPC estimate 3 Objective met if confidence interval of 2006 KPC estimate for Betioky overlaps with that for Toliara II Overall, TPCSP achieved the objectives set for two thirds of the project indicators in Toliara and three quarters in Betioky. Criteria for meeting objectives in Betioky used in this assessment are less rigorous than in Toliara to compensate for the additional challenge faced in Betioky to implement most project activities through a local NGO, with limited direct support from MCDI (see section B.2.c). According to these criteria, half the objectives were met in both Toliara and Betioky, and the other half were met in one district but not in the other. One objective was not met in either district. Medical Care Development International 71 TPCSP Final Evaluation Pneumonia case management TPCSP in both districts reached the objective for early treatment seeking, and in Toliara for the percentage of mothers of children under two who know at least two danger signs. The indicator for the IMCI clinical skills of HWs showed the same progress in both districts but did not reach the end-of-project objective. Malaria control The use of ITN by children under two increased in both districts, primarily during the second half of the project when ITN became widely available and an intense community mobilization was undertaken for this intervention. The increase was significantly lower in Betioky than in Toliara, and therefore the objective for this intervention is not met in Betioky according to the criteria used for this analysis. None of the two other indicators for malaria control (correct treatment of children with fever and preventive treatment during pregnancy) are met in Toliara. In Betioky, the percentage of mothers under two who gave correct treatment to their children with fever reached the same level as in Toliara, and therefore the objective is met. The percentage mothers of children under two who took antimalarials during their last pregnancy did not reach the same level in Betioky as in Toliara (which is high but lower than the objective), and therefore the objective is not met. The percentage of CBDs with antimalarials stock during the last supervision visits in Toliara was 92% in Toliara, well above the TPCSP objective. This percentage was equivalent in Betioky. Diarrheal diseases control Almost all five indicators for this intervention increased in both districts. Only the percentage of children with diarrhea given more fluids increased but did not reach the objective in Toliara. In Betioky, the levels of this indicator and that of appropriate nutrition for children with diarrhea remained at those levels since the first KPC in 1998. Immunization Childhood immunization showed remarkable progress in Toliara, both in terms of increase in percentage of completely immunized children by age 1 and in terms of decrease in dropout rates. This was not the case in Betioky, however, where the percentage on completely immunized remained stable but significantly lower than in Toliara, and the dropout rate increased. Tetanus immunization also increased in both districts. However, the coverage indicator based on immunization card, chosen as TPCSP indicator, did not reach the objective in Toliara. The rate based on mother’s report did. In Betioky, the tetanus immunization reached similar level to that in Toliara. Breastfeeding and nutrition The three indicators of breastfeeding and nutrition reached the objectives in Toliara and in Betioky. Child spacing The contraceptive prevalence among mothers of children under two increased and reached the objective in both districts. Medical Care Development International 72 TPCSP Final Evaluation The knowledge of HIV prevention and that of exclusive breastfeeding as a contraceptive method did not reach the objectives in Toliara. The objectives are met in Betioky in the sense that the results achieved are the same, although low, as in Toliara. Community mobilization and behavior change communication Most behavior change key project indicators reached project objectives. Local NGO staff and Field Agents trained, followed up and supported networks of CHVs in Toliara and Betioky, as planned. CHVs successfully implemented the new approaches developed under BSCSP, which primarily rely on interpersonal communication during home visits and the involvement of VISA mothers. TPCSP also successfully sponsored a variety of community mass mobilization activities. Capacity building Overall, TPCSP training objectives in terms of number of HW trained and of topics were met in the first half of the project. In addition, HWs were involved in most of the training and supervision of CHVs, and this has increased their commitment to the project and its activities. TPCSP built capacity at several levels: Communities • TPCSP provided support to create network of CHVs working with other community members, local authorities and HWs on improving health behaviors and practices • Community mobilization strategies implemented by these CHVs (VISA approach, community radios, health festivals, advocacy with local leaders, links with BHCs) are appropriate and effective Local NGOs • TPCSP effectively transferred all implementation responsibility for project BCC activities to VEMIMA in Betioky, and began working with Mampifoha and Miainga for the distribution of ITNs in Toliara BHCs • HWs in BHCs increased their performance in the areas of project interventions (IMCI, malaria control and ITN, BF and nutrition, CS and HIV prevention), in training of trainers, and in community mobilization and supervision of CHVs DHMT • The two DHMT in Toliara and Betioky increased their capacity for managing community health and mobilization, supervision of HWs in BHC and other TPCSP activities ROHFP • ROHFP staff increased their capacity for managing community health and mobilization, supervision of HWs in BHC and are prepared and motivated for increased responsibility and leadership under TREIP Sustainability TPCSP achieved its implicit sustainability objectives of: Medical Care Development International 73 TPCSP Final Evaluation • Behavior changes at the family level • Community structures to support these changes • Strengthened health facilities, including health workers performance • Strengthened management capacity of ROHFP and DOHFP in Toliara and Betioky • Availability of appropriate community health approaches and tools • Network of actors working synergistically in health and child survival in Toliara and Betioky Program management Successful aspects of TPCSP management can be considered as objectives met: • MCDI/Madagascar has developed and maintained a strong collaborative relationship with the USAID mission and other partners • TPCSP included all partners in the planning and evaluation process. • MCDI/Madagascar staff received appropriate training and gained valuable professional experience that prepares them well for effectively contributing to TREIP or other child survival programs • MCDI/Madagascar has acquired advanced skills in the development and implementation of population- and facility-based surveys and in the management and analysis of related data • MCDI/Madagascar built strong human and financial resources management skills and is well prepared for the implementation of TREIP and other future health programs b. Main Achievements and Factors Affecting Performance TPCSP achieved most of its objectives of improved behavior and coverage of interventions, and improved the capacity to improve child health and survival of communities, local NGOs, BHC, and DHMT. In Toliara II: • All but one project indicators of behaviors and coverage of interventions significantly increased, and two thirds of them achieved the project objectives • Communities have improved their capacity to promote and support healthy behaviors among mother of children under two • BHC and DHMT have improved their capacity to provide quality maternal and child health services and address the need of the communities they serve In Betioky: • TPCSP was able to successfully transfer most of the implementation responsibilities to the local NGO VEMIMA and reduced its direct involvement in project management • In this context, most behaviors and coverage of interventions that had been improved at the end of BSCSP in 2002 were maintained or further increased; behaviors and coverage of new interventions increased to levels similar to those achieved in Toliara with full support of MCDI Medical Care Development International 74 TPCSP Final Evaluation The FE Team identified several factors that have increased program performance: • Excellent collaboration among partners and working atmosphere • Effective training and supervision of CHVs, HW, and DHMT members • Effective community mobilization approaches based on CHVs and VISA mothers • Effective mass mobilization activities such as community radio programs and health festivals and campaigns • Improved availability of essential drugs, vaccines and medical equipment The FE Team also identified several constraints to better results or sustainability: • Small number and frequent replacement of health workers • Low performance of some CHVs • Enormous logistic and accessibility problems • Persistent drugs and vaccines stock outs • Low literacy of the population c. Best practices and lessons learned Best practices • VISA Approach: The VISA Approach is the unique aspect of the community health volunteers program of TPCSP. The approach was further developed based on the experience gained under the previous project in Betioky (see section B.2.d.i), and recognized by most MOHFP and partners as a successful model for community health in Madagascar. • Health care financing insurance: TPCSP continued the support and development to Health Care Financing Insurance system in the Commune of Ankazomanga (B.2.d.iii). This pilot project is also recognized by most MOHFP as a successful model of community health financing. • Community Health Information System: The CHV monthly reports and the FA supervision reports are of interest for drawing lessons on their potential use by BHC, the DHO and the RHO, and at the national level. This system is worth a systematic review in the perspective of its adoption beyond the TPCSP areas. • KPC and HFA surveys. The relatively large sample size and good quality of the series of baseline, midterm and final KPC and HFA surveys in Toliara and Betioky allow for more advanced analyses of the determinants of coverage and behavior change than those typically expected from a KPC survey. Finding of such analyses, if successful, would be of interest to the ROHFP and MCDI for planning activities under TREIP and for the documentation and evaluation of this scale up initiative. Medical Care Development International 75 TPCSP Final Evaluation Lessons learned The main lessons learned identified during the TPCSP FE follow. Technical • High immunization coverage requires to actively look for dropouts, implement outreach strategies, supervise HW in BHC, ensure close collaboration between BHC and CHV, use of immunization cards; consideration of literacy level when communicating with mothers and care takers • Mass immunization campaigns and SSME are effective strategies to increase immunization coverage • Community mobilization and quality services are effective in increasing contraceptive prevalence • Intensification of HIV/AIDS program is necessary Crosscutting • The VISA approach is effective and sustainable • The implementation of Kôminina Mendrika is a promising community mobilization strategy • Educational community radio with listening groups is an effective behavior change strategy • Grandmothers needs to be taken into account in BCC approaches to breastfeeding promotion and child spacing • Collaboration with partners is very important for the success of community health and mobilization activities • Organizational development assessments and plans are necessary to maximize capacity building of partners • HW performance benefits most from continuing training by a pool of trainers from various levels of the health system and regular formative follow-up and supervision Program Management • Well-trained and motivated NGO staff can effectively implement TPCSP community health and mobilization approaches • Effective capacity building of MCDI staff requires a training needs assessment and plan; a supervision system and technical assistance to ensure the transfer of learning; and consideration of the inevitable staff turnover. d. Final Evaluation Recommendations The FE Team developed the following recommendations to the Basic Health Centers, the ROHFP, MCDI/Madagascar, and MCDI/Home Office, most of which applying primarily to TREIP implementation. Basic Health Centers • Develop realistic plans and budget allocations to continue support to CHVs Medical Care Development International 76 TPCSP Final Evaluation • Include community mobilization in Annual Work Plan and Community Development Plan • Respect instructions regarding CHV reporting system • Strengthen active follow up of immunization dropouts ROHFP • Ensure sound distribution of health workers among BHCs • Develop strategies to motivate and retain HWs assigned to BHCs • Ensure complete transfer of information and responsibilities when health workers are replaced • Avoid overlap between activities and commitments with various partners • Ensure that ROHFP staff and DHMT members are polyvalent to better integrate programs • Continue institutionalization of formative integrated supervision and related tools. • Extend implementation of Kôminina Mendrika to all communes • Reinforce collaboration between all partners to share costs of related activities MCDI/Madagascar Community mobilization and behavior change communication • Evaluate the effectiveness of the training, follow-up and supervision system of CHVs, and apply lessons learned • Develop methods and tools, including the appropriate standards, to monitor CHV performance • Develop a process of ongoing evaluation of CHVs, and of their de-selection if needed • Update BCC materials (counseling cards) and emphasize messages on danger signs and early treatment seeking • Conduct comprehensive review and data analysis of the community heath information system implemented under BSCSP and TPCSP in collaboration with the two DOHFPs and the ROHFP, and develop clear instructions for CHV reporting system (CHVÆBHCÆDOHFPÆROHFPÆMOHFP) • Complement interpersonal BCC approaches (CHV/VISA) with mass communication approaches Quality of care • Ensure active participation of DHMTs and BHC HWs in planning activities • Provide technical assistance to all staff in charge of program at the regional and district levels, as opposed to only those in charge of a particular activities, to ensure staff polyvalence and better integration of programs • Include a community component and involve all partners in the elaboration of plans for quality improvement in health services Medical Care Development International 77 TPCSP Final Evaluation Program management • Ensure active involvement of BHC and local authorities in planning and implementation of activities • Build upon lessons learned from TPCSP’s work with local NGOs when planning and implementing TREIP, which will be mostly implemented by local NGOs. • Adopt and implement explicit organizational development plans and partnership agreements for each local NGO under TREIP. Develop the necessary methodology, tools, and expertise within MCDI. • Inform and train NGO in approaches to supervision, and adopt systematic supervision for local NGOs • Conduct training needs assessment before developing new training plans • Prepare summaries and translation of project documents, including the TREIP DIP, to ensure easy reading and use by all MCDI and partners’ staff • Strengthen communication of results through preparation of final technical reports and publications and their dissemination to all partners • Compile all valuable training, BCC and HIS materials developed and successfully used during TPCSP, and prepare technical descriptions of their purpose, lessons learned and recommendations for further use and adaptation. • Undertake a study of the actual CHV training, incentives, and all other costs and of the related sustainability issues before scaling-up such program • Prepare and disseminate technical documentation of success stories, best practices, lessons learned and technical materials that could be used within MCDI and other partner organizations including the MOHFP. • Review MCDI procedures in terms of per diem and align them with those of other donors or at least other USAID grantees • Ensure a separate budget line for MCDI staff training • Explore alternatives to current field funding mechanisms to ensure timely funding of planned activities in case of delays in justification of funds • Ensure that each staff member systematically share the knowledge and skills gained during training opportunities For MCDI US: • Provide sustained and documented technical assistance for overall strategic planning and for specific areas of identified needs to maximize results of projects and build capacity of Field Office • Plan staff training and development on the basis of needs assessments and the specific objectives of projects; include a follow up and technical assistance to ensure the transfer of learning. Medical Care Development International 78 TPCSP Final Evaluation e. Dissemination of Best Practices and Lessons Learned Best practices and lessons learned from TPCSP will primarily be applied in the planning and implementing of TREIP, which will also support further effort of documentation of these best practices. f. Potential for Scale-Up MCDI recognized early the potential for scale-up of BSCSP and TPCSP, and developed an Expanded Impact CSHGP proposal that was successfully awarded a grant for the period 2007- 2011. Medical Care Development International 79 TPCSP Final Evaluation ANNEXES Medical Care Development International 80 TPCSP Final Evaluation Appendix 1. CSHGP Project Data Form Medical Care Development International 81 TPCSP Final Evaluation Medical Care Development International 82 TPCSP Final Evaluation Medical Care Development International 83 TPCSP Final Evaluation Medical Care Development International 84 TPCSP Final Evaluation Medical Care Development International 85 TPCSP Final Evaluation Medical Care Developmen Appendix 2. Maps t International 86 TPCSP Final Evaluation Appendix 3. TPCSP Organigram and partners HQ CS Coordinator CS Support Team Project Manager and Field Staff NMTS * UNFPA Local NGOs UNICEF HE/IEC Coordinator Field Administrator HIS Specialist QOC Advisor National MINSAN Ministry of Education MICC District MINSAN Provincial MINSAN Teachers Animators Health Facilities Field Agents Community Health Committees Community Health Volunteers Community Based Distributors Regional Credit Scheme Advisor Credit Insurance Fund Mgt Committees Schools MpanentanÕI Appel Vemima Toliara Ankililoaka Miray Yale Med Scl PSI Cooperation Hierarchy *Nurse and Midwifery Training School Fanjiry COMMUNITY LEVEL Source: TPCSP DIP Medical Care Development International 87 TPCSP Final Evaluation Appendix 4. Timeline of Overall Project Implementation, October 2002 – December 2006 Management BL / Midterm / Final KPC x x x DIP / MTE / FE x x Visits to USAID Mission x x x x x x x Tech/Adm Assistance MCDI Intl. Div. Director field visit x x MCDI Admin. Officer field visit x MCDI Health Economist field visit x BCC (local TA) x Documentation VISA xx Organisational Assessment xxx BCC/SM Recruitment and training FA x x x Training CHV Toliara 2 x x x xxxxxxxx xxxxxx Training new CHV Betioky xxx Training CBDA x x xxxxx Health Festivals x x x x x x Cty Radio launched in Toliara 2 x Complete SM Package promoted xxxxxxxxxxxxxxxxxxxx National campaign SSME/Hiaka x FY 2005 FY 2006 FY07 Key activities/events FY 2004 Q1 Q2 Q3 Q4 Q1 Q2 Q1 Q2 Q3 Q4 FY 2003 Q3 Q4 Q1 Q2 Q3 Q4 Q1 Medical Care Development International 88 TPCSP Final Evaluation Continued. QOC ToT DHMT Toliara 2 x x ToT FA and HW x ToT NGO x x ENA HW Toliara 2 x x x EPI HW Toliara 2 x FP HW Toliara 2 x IMCI HW Toliara 2 x IMCI New HW Betioky x ENA New HW Betioky x EPI Epidemiol Surveillance x Malaria HW x Supervision DHMT Toliara 2 x Integrated supervision CSB, Tol/Btky xx Others VEMIMA in charge in Betioky xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx Cost recovery policy reestablished x xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx Mutuelle A/ga reactivated xxxxxxxxxxxxxxxxxxxxxxxx Mutuelles Extension xxxxxxxxxxxx KM implementation x FY07 Key activities/events FY 2003 FY 2004 FY 2005 Q2 Q3 Q4 FY 2006 Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 Q1 Q1 Q2 Q3 Q4 Q1 Medical Care Development International 89 TPCSP Final Evaluation Appendix 5. TPCSP 2006 Benchmarks Table prepared by MCDI Madagascar staff, and translated and edited for clarity and consistency by the author of this report. Most numbers refer to TPCSP implementation in Toliara II, except when indicated otherwise. Benchmarks, as per DIP Achievements, as per MCDI Madagascar tracking system PCM BCC 06 FA recruited and trained in ToT and in supervision of CHVs including management of ARI/pneumonia -6 FA trained on ToT and all project interventions except FP -6 DHMT and RAD staff trained on ToT and all project interventions except FP ARI/pneumonia curriculum for CHVs developed Curriculum and training modules for CHVs developed and available 420 CHVs in place and trained in ARI 505 trained in ARI/Pneumonia according to the C-IMCI protocol (BCC Manager data) 80% of CHVs retrained in ARI/pneumonia on quarterly basis More than 80% CHVs retrained in IRA/Pneumonia Source: BCC Manager MCDI/2006 Supervision tools for C-IMCI adapted and available at all levels (CHVs, HW, etc.) Done for CHVs (this is the MCDI grouped/individual supervision tool) and for HW (Integrated Supervision tool) 60 HW trained in ToT and in supervision of CHVs in relation to management of ARI/pneumonia according to IMCI protocol 60 HW trained in ToT and in supervision of CHVs in relation to management of ARI/pneumonia according to IMCI protocol (MCDI: Training report) 80% of CHVs supervised on quarterly basis More than 80% CHVs supervised on a semiannual basis (FA supervision reports) Education messages on ARI/pneumonia adapted Done BCC support of ARI/pneumonia adapted, developed, and made available at all levels (CHVs, HW, CIS, Schools, NGOs, RAD) Yes (except for schools) Festivals held annually in each health sector Festivals held in 28/29 health sectors in Tuléar II and 21/21 in Betioky Medical Care Development International 90 TPCSP Final Evaluation Benchmarks, Achievements, as per DIP as per MCDI Madagascar tracking system QOC A team of 6 trainers (DPS, MCDI, SSD) retrained in C-IMCI 11 IMCI Trainers identified and trained C-IMCI training modules for HW adapted and made available for use Done (this is 6-day IMCI training module) 60 HW trained in C-IMCI in accordance with the C-IMCI protocol -55 HWs trained in C-IMCI in accordance with the C-IMCI protocol in Toliara -13 New HWs trained in Q4-FY03 in Betioky C-IMCI flowchart made available to all the BHC Completed —SOURCE: MOHF Consultation forms for C-IMCI implementation regularly made available at the level of the 28 BHC Completed—SOURCE: MOHF C-IMCI supervision tools for BHCs adapted -C-IMCI supervision tools for BHCs adapted. -Integrated supervision tool developed and used in 23 BHC in Toliara and 17 BHC in Betioky in December 2006 (2006 Integrated Supervision) Members of the DHMT trained on the C-IMCI supervision tool, and on planning and supervision technique 11 DHMT members trained in IMCI supervision 80% of the BHC are regularly supervised in accordance to supervision guidelines 80% of the BHC are regularly supervised 70% BHC in Toliara and 65% in Betioky received an IMCI integrated supervision visit (2006 Integrated Supervision) The quality of ARI/pneumonia case management evaluated annually by the HFA IMCI services delivery survey done in 2002, in October 2004 et December 2006 28 BHCs equipped with C-IMCI kit including (01 watch to check on the frequency of respiration, ORT kit to control dehydration, infant scalc, thermometer, and case management card) 28 BHCs equipped with C-IMCI kit 70% of supervised BHC in Toliara and 53% in Betioky are equipped in C-IMCI kits (2006 Integrated Supervision) Procurement of antibiotics for the BHC carried out every quarter Procurement done every quarter Toliara: 77% of BHCs have adequate stock of cotrimoxazole and 46% have adequate stock of injectable Ampicilline; Betioky: 77% and 29%, respectively (2006 Integrated Supervision) Medical Care Development International 91 TPCSP Final Evaluation Benchmarks, Achievements, as per DIP as per MCDI Madagascar tracking system Malaria BCC 184 CBD trained on the community-based sales of ITNs, re￾impregnation, and social marketing of ITNs 185 CBD trained on community-based sales of ITN (BCC Manager data). 184 CBD regularly supplied with ITNs, and the re￾impregnation kit through PSI private channels 38,077 ITNs provided to CBD and other distributors in Toliara II and Betioky in 2005 and 2006 (MCDI stock management system) 50% of CSB regularly supplied with ITNs in accordance with the PFU channel A change in national policy favored free ITN distribution at the CSB level. CBDs received ITNs from the PSI social marketing program Social groups formed in 25% of the health sector to facilitate the acquisition of lTNs by the rural populations Not done, because demand for ITNs has been very high CHV, CBD, NGO, & BHC equipped in BCC support to promote malaria prevention through the use of ITNs Done Radio programs on IEC/BCC for malaria prepared & broadcasted regularly Educational radio programs broadcasted regularly and community listening groups set up in collaboration with ALT Radio. 8 radio communication agents trained in the fight against malaria, and on the development of educational programs on malaria control 11 radio communication agents and 6 FA trained on development of radio programs on malaria 60 spots & 60 radio programs broadcasted on malaria control and the promotion of ITNs More than 60 radio spots and programs on malaria broadcasted (integrated in IMCI spots) 420 CHVs, 184 CBDs, 4 NGOs, RAD & FA trained on the prevention & on home-based case management of malaria 505 CHVs, 185 CBDA, 2 NGO, 02 RAD et 6 FA trained on prevention and case management of malaria according to C￾IMCI protocol 60 HW, 06 FA, RAD, 4 NGO trained in ToT on the prevention and home-based case management of malaria, supervision of malaria case management and utilization of ITNs 60 HW, 06 FA, 02 ONG, 02 RAD trained in Tot (MCDI: Compilation Training report) Malaria supervision tool adapted, developed, and accessible at all levels (CHV, CBDA, HW etc.) Developed by WHO 80% of CHVs & CBDs supervised quarterly -More than 80% CHVs supervised on a semiannual basis (FA supervision reports) Medical Care Development International 92 TPCSP Final Evaluation Benchmarks, Achievements, as per DIP as per MCDI Madagascar tracking system 80% of CHVs & CBDs retrained on the prevention & malaria case management 100% of trained CHVs received refresher training on preventions and case management of malaria (FA supervision reports) Messages on malaria control and promotion of the ITNs adapted Done BCC support for malaria control & promotion of ITNs adapted, developed, and accessible at all levels (CHV, HW, CIS, Schools,NGOs, RAD) Done (except in schools) Festivals held annually in each health sector See PCM1/BCC above Contests for social mobilization to promote the use of ITNs launched every six months in all the health sectors. Not done 60 HW retrained every six months on case management and prevention of malaria in pregnant women 60 HW trained on case management and prevention of malaria in pregnant women A team of 6 trainers (DPS, MCDI, SSD) retrained on the malaria treatment protocol for pregnant women 11 IMCI retrained on the malaria treatment protocol for pregnant women Training module of malaria in pregnant women adapted Not done. Waiting for change in malaria national policy on prevention and treatment of malaria (IPT, artemisinine) Procurement of antimalarial drugs for the BHC carried out every quarter Chloroquine available in 77% of BHC, SP in 77% and injectable quinine in 78% of BHC in Toliara, and 65% and 65% in Betioky, respectively (2006 Integrated Supervision data) 60 HW retrained on the PFU Done by DHO during monthly review and planning meetings CBD module on chloroquine adapted Done 184 CBDs trained in management of chloroquine, the direction for use & the algorithm for malaria treatment for the CBD 185 CBDs trained in management and use of chloroquine 80% of CBDs supervised quarterly on community distribution Completed Management tools for the CBD developed Completed Procurement of chloroquine for the DBC carried out every quarter Completed. 36,760 PaluStop infants and 69,840 PaluStop child provided to CBD and other distributors in Toliara II and Betioky in 2005 and 2006 (MCDI stock management system) Medical Care Development International 93 TPCSP Final Evaluation Benchmarks, Achievements, as per DIP as per MCDI Madagascar tracking system QOC A team of 6 trainers (DPS, MCDI, SSD) retrained on the malaria case management 11 IMCI Trainers identified and trained in Q2-FY03 and Q1- FY04? 60 HW trained in Malaria Case Management in accordance with the C-IMCI -55 HW trained in C-IMCI in accordance with the C-IMCI protocol in Q1-FY04 in Toliara -13 New HWs trained in Q4-FY03 in Betioky Training modules for Malaria Case Management for HW adapted & made available for use Done (6-day C-IMCI training module) Diarrhea BCC Training module in prevention and home-based care of diarrhea adapted Done (C-IMCI module) 420 CHV, 06 FAs, 04 NGOs, RAD trained in diarrheal diseases prevention, including hygiene and home-based management of diarrhea 505 CHV, 6 FA, 2 NGO, and 02 RAD trained in diarrheal diseases prevention, including hygiene and home-based management of diarrhea in accordance with the C-IMCI (BCC Manager data; FA report) 184 CBDs trained in preparation of ORS, community-based sale of ORS, the diagnosis of dehydration, & oral rehydration Not done. MOH policy is promotion of home-based fluids. 420 CHV & 184 CBDs retrained in home-based case management and prevention of diarrhea 402 CHV and 185 CBDs retrained (BCC Manager data) BCC materials and messages on diarrheal control and prevention adapted & made available for use at all levels (CHV, CBDs, HW, NGO, ISC:Inter Sectoriel Committees) Done 06 FA recruited and trained in diarrheal diseases control and management Done 06 FA, HW, RAD, NGO trained in ToT for diarrheal diseases control 06 FA and 60 HW trained in Q4-FY03; RAD and NGO trained 80% of CHV and CDBA are supervised quarterly on diarrheal diseases control and management of ORS -More than 80% CHVs and CBDs supervised on a semiannual basis (FA supervision reports) -N/A for CBD Number of ORS sold per year N/A Medical Care Development International 94 TPCSP Final Evaluation Benchmarks, Achievements, as per DIP as per MCDI Madagascar tracking system 28 support groups established to promote the fight against diarrhea, promote breastfeeding, and promotion of vaccination. 5 support groups in place in one health sector 80% of villages have a local committee on Cholera to avert cholera epidemics N/A. No activities in this area since last cholera epidemics occurred in 2000. 4 NGOs trained in the mentoring approach, and planning of child survival interventions, 03 ONG trained (VEMIMA, Mampifoha, Miainga) in the mentoring approach, and planning of child survival interventions, Festivals held annually in each health sector SEE PCM/BCC 60 spots et 60 radio programs on diarrhea prepared and disseminated regularly More than 60 radio spots and programs on IMCI broadcasted SEE IMCI ACHIEVEMENTS UNDER PCM/QOC Immunization BCC 420 CHV trained in the promotion of EPI, BCC circuits, & VISA approach 458 CHV trained in promotion of EPI in Toliara II (BCC Manager data) 60% of CHV apply the VISA approach to vaccination 65% of CHV apply the VISA approach to vaccination— FA reports EPI training module for CHV developed Done Messages & BCC materials on vaccination adapted & made available at all levels of the health system. Done Mass media used in the promotion of EPI Done. Educational radio program on EPI broadcasted and community listening groups set up in collaboration with ALT Radio. 6 programs/spots on EPI developed. Recording system to follow up on the drop-outs of children to be immunized is elaborated Done. CHV have system to register women and children to be immunized 420 CHV trained in tracking of drop-out cases in the community and 80% of CHV are supervised quarterly on EPI related activities Done for 458 CHV trained in EPI 80% of CHVs are supervised quarterly on EPI related activities More than 80% CHVs supervised on a semiannual basis (FA supervision reports) Medical Care Development International 95 TPCSP Final Evaluation Benchmarks, Achievements, as per DIP as per MCDI Madagascar tracking system QOC A team of 6 trainers (RHO, MCDI, DHO) retrained on EPI 11 trainers retrained on EPI and EPI guidelines in Q2-FY03 and Q1-FY04 60 HW trained in the new EPI strategies that include vaccination within the context of C-IMCI, utilization of multiple doses vaccine vials, and Vitamin A supplementation 60 HW trained in EPI in Q4-FY03 60 HW retrained on the rationale of the EPI national policy 60 HW retrained by DHO and MCDI during monthly review and planning meetings 12 refrigerators assigned to the population N/A 28 BHC equipped with refrigerators are functional 61% of supervised BHCs in Toliara and 88% in Betioky are equipped with functioning refrigerators (2006 Integrated Supervision) 28 BHC regularly supplied with kerosene, and spare parts 28 BHC regularly supplied with kerosene, and spare parts. -22% of supervised BHC in Toliara and 12% in Betioky have kerosene available in December 2006 (2006 Integrated Supervision) Security stock of spare parts (wicks, & chimney for the refrigerators available in the SSD warehouses) No BHC has adequate stock of spare parts for refrigerators (2006 Integrated Supervision) Regular re-supply in kerosene for refrigerator established with EMAD -22% of supervised BHC in Toliara and 12% in Betioky have kerosene available in December 2006 (2006 Integrated Supervision) DHO EPI manager, and 60 HW trained in maintenance of cold chain Done, during the supervision by DHO, and during EPI training DHO warehouse with refrigerators is functional and is maintained -Toliara II: not done: refrigerator kept in MCDI office for security reasons -Betioky: done HFA realized annually to evaluate the quality of service delivery and the need for ID-service training under the EPI for HW IMCI services delivery survey done in October 2004 et December 2006 includes questions EPI and ID-service indicators 80% of BHC, including the cold chain system, are supervised -EPI supervision of all BHC done twice a year -70% BHC in Toliara and 65% in Betioky received an IMCI integrated supervision visit in Dec 2006 (2006 Integrated Supervision) Medical Care Development International 96 TPCSP Final Evaluation Benchmarks, Achievements, as per DIP as per MCDI Madagascar tracking system 60 HW retrained in management of vaccines 60 HW retrained by the team of DHO and MCDI during monthly review and planning meetings Vaccines are supplied to the 28 BHCs every quarter -Quarterly supply of all BHCs. -Less than 70% and less than 88% of supervised BHC in Toliara II and Betioky, respectively, have each vaccines in stock in December 2006 (2006 Integrated Supervision) SEE OTHER IMCI ACHIEVEMENTS UNDER PCM/QOC Breastfeeding BCC 420 CHV trained in the promotion of breastfeeding 472 CHV trained in promotion of BF and ENA (BCC Manager data) 60% of CHV apply the VISA approach on breastfeeding and VISA approach 402/445 CHV (90%) trained on VISA apply it on BF — SOURCE: FA reports Training modules on BF and AEN for the CHV developed Done Messages & BCC materials on BF adapted and available at all levels Done Mass media used in the promotion of BF Educational radio program on BF broadcasted and community listening groups set up in collaboration with ALT Radio. 19 radio products on EBF and complementary feeding developed 80% of CHV are quarterly supervised on BF related activities More than 80% CHVs supervised on a semiannual basis (FA supervision reports) QOC A team of 6 trainers (RHO, MCDI, DHO) retrained on the ENA & utilization of the modules 11 trainers retrained on AEN in Q2-FY03 and Q1-FY04 60 HW trained in the ENA 60 HW trained on AEN in Q2- and Q3-FY03 in Toliara 60 HW retrained in BFHI 7 HW trained in BFHI, in collaboration with WHO and Linkages Baby friendly hospital initiative implemented in the 28 BHCs 1/28 BHC in Toliara, 2/2 District Hospital in Betioky, 3/21 BHC in Betioky implements BFHI–SOURCE? WHOM Channels to re-supply BHC with Vitamin A established project with the EMAD and with the PFU arrangement Done Medical Care Development International 97 TPCSP Final Evaluation Medical Care Development International 98 TPCSP Final Evaluation Benchmarks, as per DIP Achievements, as per MCDI Madagascar tracking system Procurement & re-supply of Vitamin A for the 28 BHC carried out on a quarterly basis -Done on a quarterly basis -36% and 41% of supervised BHCs in Toliara and Betioky, respectively, have stock of Vitamin A in December 2006 (2006 Integrated Supervision) HFA realized annually to evaluate the quality of service delivery (counseling in AEN and breastfeeding by the HW) IMCI services delivery survey done in October 2004 et December 2006 includes questions nutrition indicators Supervision tool for BHC on BF, including supervision of the BFHI Done 28 HW trained in the promotion of breastfeeding (messages, benefits, IHAB, follow-up) and surveillance and growth promotion 7 HWS, 1 BHC of Toliara, 2 CHD of Betioky Sud and 3 BHCs of Betioky Sud trained in BFHI approach. Supervision tools include qualitative analysis of EBF developed Completed (Integrated Supervision tool) 28 BHC practice surveillance for growth monitoring of sick and healthy children Completed (2006 Integrated Supervision) SEE OTHER IMCI ACHIEVEMENTS UNDER PCM/QOC Child spacing BCC 420 CHV trained in promotion of FP, BCC techniques, VISA approach 503 CHV trained on promotion of FP; 472 trained in basic communication skills; 466 trained in VISA approach (BCC Manager data) 60% of CHV apply the approach VISA on FP Done Training module in FP for the CHV developed Module available Messages & BCC materials on FP adapted and made available at all levels of the system Done Mass media used to promote FP Educational radio program on FP broadcasted and community listening groups set up in collaboration with ALT Radio. 13 radio products on FP and 1 on LAM developed Recording system that is developed for community-level activities is established to track dropouts from the FP program Done 420 CHV trained on follow up of drop-outs at the community level Done 80% of the CHV are supervised quarterly in FP related activities Done Medical Care Development International 99 TPCSP Final Evaluation Benchmarks, as per DIP Achievements, as per MCDI Madagascar tracking system QOC 28 BHC adequately equipped with medical equipment and supplies for Family Planning 28 BHC adequately equipped with medical equipment and supplies for Family Planning 28 BHC re-supplied quarterly with contraceptives in accordance with the established PFU system -Done on a quarterly basis -83%, 96%, and 78% with adequate stocks in CO, CI and condoms in Toliara, and 76%, 65%, and 24% in Betioky (2006 Integrated Supervision) 28 BHC providing birth spacing services to the target population 33 BHCs of Toliara II and 21 CSB of Betioky Sud equipped and supplied to provide FP services Procurement channels for contraceptives established Cancelled District Pharmacies have appropriate stock of contraceptives Phagedis in Toliara and Betioky without stock outs in the last 6 months of project A team of 6 trainers (RHO, MCDI, DHO) retrained on birth spacing 11 trainers retrained on FP in Q2-FY03 and Q1-FY04 60 HW trained & retrained on the four family planning methods according to national policy for FP 100% HW trained in FP in Toliara II and 88% in Betioky (2006 Integrated Supervision) 60 HW retrained on PFU, supply and drug management Cancelled HFA realized annually to evaluate the quality of FP services being provided by the HW IMCI services delivery survey done in October 2004 et December 2006 includes questions FP and FP-service indicators 80% of BHC are supervised 70% BHC in Toliara and 65% in Betioky received an integrated supervision visit in Dec 2006 (2006 Integrated Supervision) Supervision tools for the BHC on FP are developed Integrated Supervision tool developed and includes FP Activities and Benchmarks listed in the DIP but DROPPED: 50 Schools implement the child-to-child approach 50 teachers of elementary school trained in child-to-child approach and in management of ARI 80% of schools implementing CTC program supervised on quarterly basis by the CISCO on the implementation of the AEPE Intersectorial Committees (CIS) working in collaboration with the community for the implementation of diarrhea prevention activities that include hygiene, water and sanitation, including the building and utilization of latrines. Activity transferred to NGO Aide et Action Toliara 420 CHV trained, equipped with Salter scales for surveillance and growth monitoring Cancelled Appendix 6. FE Assessment Methodology The TPCSP FE fieldwork was conducted in two weeks from February 12 through 26, 2007 according to the schedule in Appendix 7. Prior to this period, the TPCSP staff had conducted a Knowledge, Practice and Coverage (KPC) survey of 300 mothers of children under two in each project district (data collection was done in December 2006) and a Health Facility Assessment of 23 and 17 Basic Health Centers in Toliara and Betioky, respectively (data collection was done between October and December 2006). The project staff also prepared summary tables with the results of these surveys, including results for the primary project indicators, and a summary of the main project accomplishments referring to the activities and benchmarks listed in the DIP work plan (see Appendix 5). The tentative schedule for the fieldwork of the FE was prepared and agreed upon in advance by MCDI/Madagascar staff and the Final Evaluation Coordinator. The first day of fieldwork was spent in Antananarivo to meet with representatives of USAID/Madagascar and the Division of Family Health of the Ministry of Health and Family Planning (MOHFP). During the meeting at the Division of Family Health, the Director assigned one of the staff of the Division to join the FE Team. The next day in Toliara was spent with representatives of the Regional Office of Heath and Family Planning (ROHFP) and of the District Office of Health and family Planning (DOHFP) of Toliara II. All these initial meetings had a double purpose of (1) briefing the persons met on the goal and process of the TPCSP FE, and (2) interviewing them on general project issues such as collaboration between their institutions and MCDI, sustainability and continuation of TPCSP into TREIP, and expectations from the FE. A Large FE Team of about 40 individuals and a Small FE Team of about 12 individuals, also members of the Large FE Team, conducted the FE fieldwork. The Large FE Team comprised the MCDI staff, representatives of the MOHFP, ROHFP, DOHFP and BHCs, and local NGOs, community members (CHVs and mothers), representatives of other partner organizations, and the Final Evaluation Coordinator. The list of the Large FE Team members who participated in the first or in the second FE workshops is presented in Appendix 7. This list shows that a total of 40 individuals participated in either one of the workshops, but only 22 of them participated in both. The main group of participants in the first workshop who did not attend the second workshop are the MCDI Field Agents (6) and local NGO members (2) who were committed to data collection for the baseline surveys for TREIP. Unfortunately, the FE Team could not interview these key actors in the implementation of the project. Two other observations in the participation in the FE process are noteworthy: only 1 representative from the DHO of Toliara II (the DHO chief) only came to the second workshop, and three senior MCDI technical staff involved in the project since its conception were only present at one workshop and not in the Field Visits. The absence of full participation of these key TPCSP actors has limited the value of the participatory approach of the FE. The Large FE Team first met during a one-day workshop at the beginning of the FE fieldwork to define the FE questions, and to propose first answers to these questions or identify potential Medical Care Development International 100 TPCSP Final Evaluation sources of data that would help answer them. The participants were divided into five groups to work on one of the following FE topics: 1. Technical Interventions PCM, MAL, CDD 2. Technical Interventions IMM, BRE, CD 3. Cross-cutting interventions: Community mobilization, BCC, Sustainability 4. Cross-cutting interventions: Capacity building 5. Project Management Each group received a table like the one below with the FE Questions column already filled in with questions adapted from the USAID/CSHGP guidelines related to the topic assigned to the group. FE Question Data Source Answer Question 1 Question 2 … Each group attempted to answer the questions with the data available, or identify data sources that would help answer the questions. The groups were encouraged to use as much as possible the available data before suggesting any new data collection effort. The results of each group were presented, discussed and amended as needed in a Large FE Team plenary session. The Small FE Team of about 12 members then met during a second day to further specify the data needs and the possible methods and tools to collect and analyze these data within the timeframe of the FE fieldwork. Using the tables prepared by the Large FE Team the day before, they listed all the data needs by data source as follows. Data Source Data need Results Source 1 Data 1 Data 2 Data 3 Source 2 Data 2 Data 3 … … This list was then used to develop the appropriate methods and develop the related tools for data collection or analysis with respect to each source. The Smaller FE Team also collected and analyzed the data that required field visits. The data needs required visits and interviews with MCDI and ROHFP staff, with DHMT members in Toliara and Betioky, with HWs from BHC, with local authorities and local leaders, and with community members such as CHVs and mothers of children under two. Such data from both Toliara II and Betioky appeared necessary to allow comparison between the project implementation approaches in these two districts. Medical Care Development International 101 TPCSP Final Evaluation To ensure maximum representativeness with a small sample, two Communes were chosen randomly in each district. One Commune was chosen among those where a Quality Improvement initiative was ongoing in the BHC, as this suggests that the BHC has a higher performance level. The other Commune was chosen among the remaining ones. In each Commune, the FE Team members interviewed the HW available and gathered local authorities, local leaders, CHVs and mothers to conduct group interviews. The composition of the two FE field visit teams and the number of individuals interviewed in each site or organization visited are presented in the two tables below. The two teams were balanced in terms of institutional affiliation and experience with the project and the field. Team 1 had an easier time traveling to their selected BHCs, and also conducted the interview in Toliara with the RHFPO, MCDI staff and ALT-radio. Team 2 had many misfortunes on the roads to Betioky, made more difficult by the heavy rain, and could not meet with as many HWs and DHMT members as expected. TEAM 1 TEAM 2 Toliara II Betioky Dr PHILIPPE (RHFPO), Chief Dr NORBERT (RHFPO), Chief Dr NELSON (MCDI) Dr JERRY (MCDI) Dr HERY (DHO BETIOKY) Dr MANOU (DHO TOLIARA II) Mr SAMUEL (DHO BETIOKY) Dr FELICIE (MOHFP/FH/SSEA) Mme JEANNE (MCDI AT) Mme MOMA (VEMIMA) Dr MARC, FE Coordinator Data Source Number of persons interviewed Total Regional and Project level RHO 4 4 MCDI 6 6 District level Toliara II Betioky DHMT 9 2 11 BHC and communities Ambohi￾mahavelon Miary Belamoty Manasoa￾Fanjahira HW 1 1 0 1 3 Local authorities 2 2 2 3 9 Local leaders 2 2 3 2 9 CHV 3 5 4 2 14 Mothers 2 2 4 3 11 After the field visits, the Smaller FE Team analyzed the data collected and presented their results in the Tool used to list the data need by source, producing a one large 25-page document. As time did not allow them to use this information to answer the FE questions themselves, they simply shared this 25-page document with the Large FE Team during the second one-day workshop at the end of the second week of the FE. The Larger FE Team reviewed these results in the same small working groups as during the first workshop. Each working group formulated the answers to the FE questions that they has used during the first workshop, and also formulated the general conclusions and recommendations of the FE. The results of the small working groups were presented, discussed and adopted in a plenary session, although with not enough time as most participants wanted. Medical Care Development International 102 TPCSP Final Evaluation Medical Care Development International 103 TPCSP Final Evaluation The products of the two workshops and field visits are available at MCDI/Madagascar. These products served as main materials for the FE report, although these data, analyses and opinions were further developed and completed as needed to meet the USAID/CSHGP guidelines for FE reports. After the workshop, MCDI/staff and the FE Coordinator spent on day at the TPCSP office to gather additional data and information, and discuss outstanding issues, relevant to the FE report. The last day of the FE fieldwork was used to present preliminary results, conclusions and recommendations to USAID/Madagascar in Antananarivo. A PowerPoint presentation was prepared with graphs of the main results of the KPC surveys, and a summary of the results of the FE fieldwork. The FE Coordinator prepared this FE report with email inputs from the TPCSP National Coordinator and HIS Manager to complete the missing information. A near-final draft of the report was submitted to MCDI/HO and MCDI/Madagascar, which they reviewed before sending comments and last inputs. The FE Coordinator then finalized the report. Medical Care Development International 104 TPCSP Final Evaluation Appendix 7. FE Field Work Schedule Date (February) Activities 12 Planning meeting with MCDI and consultant Meeting with USAID Madagascar (Mr Benjamin Andriamitantsoa) Meeting with MOHFP, Division of family Health 13 Travel to Toliara Meeting with RHO Meeting with DHO Toliara Meeting with MCDI staff 14 Preparation FE Workshop 1 --Review preliminary survey data --Adoption of workshop objectives and schedule --Development of presentation, methodology and tools 15 Participatory Final Evaluation Workshop-Large Group: --Presentation of project achievements and results --Presentation of Final Evaluation methodology and schedule --Development and adoption of Final Evaluation questions --Confirmation of Field Visits teams 16 Participatory Final Evaluation Workshop-Small Group (Field Visits teams): --Definition of specific FE questions and data sources --Preparation of Field Visits teams 17 Development of field visits data collection tools 18 Finalization of field visits data collection tools 19 Finalization of field visits data collection tools Preparation of field visits teams Travel to Betioky (Team 2) and Field visits (Team 1) 20 Field visits 21 Field visits 22 Field visits reports and preparation of workshop 2 23 Participatory Final Evaluation Workshop-Large Group 24 Collection of data/documents with MCDI Team; travel to Antananarivo 25 Preparation presentation to USAID 26 Presentation to USAID Debriefing of MCDI and consultant Appendix 8. FE Workshops Participants Last and First Name Institution Position Workshop #1 #2 Both 1 RAVELOMANANTSOA Félicie MOHFP/FH/CAH, Tana Clinical IMCI / IEC Officer + + + 2 RATOLOJANAHARY Veloson Philippe MOHFP/RHO, Toliara Program Officer + + + 3 RASOLOFOMANANA Romuald MOHFP/RHO, Toliara EPI Officer + + + 4 RAKOTOARISOA Norbert MOHFP/RHO, Toliara Focal Point IMCI + + + 5 RAZAFINIMPIASA Lalatiana MOHFP/RHO, Toliara Deputy Director + + + 6 RAZANAKOLONA MOHFP/DHO, Betioky IMCI Officer + + + 7 TALIASOA Rigoberthe MOHFP/DHO, Betioky Reproductive Health Officer + + + 8 RAKOTOMALALA Jeannot MOHFP/DHO, Betioky EPI Officer + + + 9 RANDRIAMANAMPISOA MOHFP/DHO, Betioky Focal Point IMCI + 10 RAMANANTENAHARISOA Claudia MOHFP/DHO, Toliara Chief + 11 RAZAFINDRAKOTOHASINA Manohisoa MOHFP/BHC, Ambohimahavelona, Toliara II Chief + 12 RASOLONDRAIBE Iarilalao MOHFP/BHC, Miary, Toliara II Chief + + + 13 RANDRIANARISOA Yvon DIRDAT, Toliara Chief + 14 MOMA Communication Ministry, Betioky Rural Animation Delegate + + + 15 RAMBELOSON Mamy Community, Ambohimahavelona CHV + + + 16 SOLONDRAZANY Justin Community, Maromiandra, Toliara II CHV + + + 17 LAMOZY Soavila Community, Saririaka, Toliara II CHV + + + 18 ROBELSON Jean Community, Beroroha Marofoty, Toliara II CHV/CBDA + + + 19 RAZAFINIRINA Sylvienne Community, Maromiandra, Toliara II Mother + + + 20 NIRINA Community, Saririaka, Toliara II Mother + 21 BOTOSON René NGO Mampifoha, Toliara Board member + 22 IBRAMDJEE Georges NGO Mampifoha, Toliara President + 23 SOANDRIAKA Christophère NGO Mampifoha, Toliara Board member + 24 KOTIPOKE Véronique NGO VEMIMA, Betioky Board member + + + 25 RAZANAMAVO Eugénie NGO VEMIMA, Betioky Board member + + + Medical Care Development International 105 TPCSP Final Evaluation Last and First Name Institution Position Workshop #1 #2 Both 26 RAHARIJIMY Patrick MCDI, Toliara Administrative Assistant + + + 27 ANDRIAMAHEFA Oliva MCDI, Toliara BCC Manager + 28 IADANAKO MCDI, Toliara Supervisor + 29 RAKOTOBE Jeanne MCDI, Toliara Field Agent + + + 30 RANDRIANTSOA Edmée MCDI, Toliara Field Agent + 31 RAZAFINATOANY Dany MCDI, Toliara Field Agent + 32 RAZAKASON Oly MCDI, Toliara Field Agent + 33 VELONARISOA MCDI, Toliara Field Agent + 34 RAKOTOZAFY Jerry MCDI, Toliara HIS Manager + + + 35 FANOMEZA Rija MCDI, Toliara Former HIS Manager + 36 RAZAFIMANANTSOA Nelson MCDI, Toliara QOC Manager + + + 37 RATSIRARSON Joséa MCDI, Tana National Coord/Project Manag. + 38 FANOMEZANTSOA Benja SanteNet, Toliara Regional Coordinator + 39 THOMPSON Crystal Peace Corp, Toliara Volunteer, SantéNet + 40 DEBAY Marc Independent Final Evaluation Coordinator + + + Number of participants: 34 28 22 Medical Care Development International 106 TPCSP Final Evaluation Appendix 9. Baseline and End-of-Project Results in Betioky Betioky Toliara # Baseline FE FE Indicator Est. UCL LCL Est. UCL LCL Est. UCL LCL TPCSP Object. PCM1 Early treatment seeking for rapid/difficult breathing N/A N/A N/A 44 52 36 56 65 47 45 PCM2 Knowledge of at least two danger signs N/A N/A N/A 46 52 40 63 69 58 65 PCM3 Clinical staff who use correctly IMCI protocols N/A N/A N/A 27 N/A N/A 31 N/A N/A 60 MAL1 Children sleeping under an ITN N/A N/A N/A 43 49 38 58 64 52 20 MAL2 Mothers who give correct treatment for fever N/A N/A N/A 31 37 24 36 47 24 50 MAL3 Mothers who took antimalarials during pregnancy N/A N/A N/A 50 56 44 72 77 67 80 MAL4 CBDs with no stock-outs of antimalarials N/A N/A N/A N/A N/A N/A 92 N/A N/A 0 CDD1 Children with diarrhea who are given more fluids 68 74 61 65 72 58 52 59 45 65 CDD2 Children with diarrhea who are given same or more food 86 92 80 76 82 70 54 61 47 55 CDD3 Children with diarrhea who are given same or more breast milk 93 97 89 90 94 86 66 72 60 65 CDD4 Early treatment seeking for children with diarrhea N/A N/A N/A 52 65 40 74 84 65 80 CDD5 Mothers who can cite at least 2 danger signs of diarrhea 65 70 60 66 71 60 64 70 59 65 IMM1 Children 12-23 months completely immunized 44 50 38 46 51 40 71 76 66 60 IMM2 Mothers who received at least two doses of TT during last pregnancy 28 33 22 37 43 32 50 58 42 65 IMM3 Immunization drop out rates 15 20 9 23 29 17 10 15 5 10 BRE1 Exclusive breastfeeding 64 70 58 42 48 37 41 46 35 35 BRE2 Early initiation of breastfeeding 55 60 49 53 59 48 54 60 49 55 BRE3 Complementary feeding N/A N/A N/A 80 85 75 48 53 42 50 CS1 Contraceptive prevalence among mothers of children under two 17 23 12 28 33 23 24 29 19 25 CS2 Knowledge of HIV prevention N/A N/A N/A 36 42 31 21 26 16 60 CS3 CBDs with no stock-outs of condoms N/A N/A N/A 90 N/A N/A 88 N/A N/A 75 CS4 Knowledge of exclusive breastfeeding as child spacing method N/A N/A N/A 3 5 1 4 6 2 40 Medical Care Development International 107 TPCSP Final Evaluation Appendix 10. Main BCC materials used under TPCSP TPCSP Intervention Support Content Use/Users Observation Quantity produced/provided/distributed by MCDI One counseling card per intervention Key practices Interpersonal communication; for use by CHW, HW, CDB, TBA Tested and adopted without change All 1,500 per topic Gazety (written guide for each intervention) Key practices Mothers read the Gazety pass it on to others Tested and adopted without change 25,000 per topic Gazety Home-based case management Mothers read the Gazety pass it on to others Newly developed by MCDI IMCI 15,000 MOH Child Health Booklet All key health message and practices Given to all children in health centers Tested and adopted without minor change (introduction of message on depressed fontanel) 20,000 (MOHFP adopted a policy of sale of the Child Health Booklets but MCDI made this initial donation) IRA Poster Counting respiration rate and looking for danger signs of pneumonia Public display BHCs and other public locations Newly developed by MCDI 1,500 IMM Immunization certification for mothers Individual ownership or display Newly developed but not adopted by MOH yet 2,500 during pretest BRE Banner (2x4 meters) Exclusive breastfeeding until 6 months Public display in all BHC Adapted and adopted 100 CDD Counseling card Messages on danger signs (depressed fontanel) Interpersonal communication; for use by CHW, HW, CDB, TBA Newly developed and adopted by MOH 1500 MCDI also developed one banner for each key TPCSP message for use by BHCs and Communes during various mass events (festivals, campaigns) or for permanent display. Medical Care Development International 108 TPCSP Final Evaluation Appendix 11. Training Events Attended by MCDI Staff Trainee Topic Duration Year Location Sponsoring organization CSTS Mini-University 5 2003 Baltimore, USA CSTS Technical update in IMCI, FP, ENA, Community case management 15 2004 Tana MINSANPF WHO/UNICEF/WB CSTS Mini-University 5 2004 Baltimore, USA CSTS Project Manager Technical update in IMCI, FP, ENA, Community case management 15 2006 Tana MINSANPF WHO/UNICEF/WB Project Administrator Procedures de Gestion des Fonds USAID 5 2005 Tana USAID Washington Training of trainers 15 2003 Tana INSPC Quality improvement in Health Systems Management 30 2004 Toliara MINSANPF INSPC/ITEM/CESAG Training of trainers on promotion of health products though community-based sales 30 2004 Toliara MINSANPF INSPC/ITEM/CESAG Training of trainers on promotion of health products though community-based sales 5 2005 Tana PSI Madagascar Essential Nutrition Action 3 2005 Tana MINSANPF/Nutrition Linkages Health Worker Performance Improvement 4 2005 Tana SanteNet Training of trainers (advanced) 15 2006 Antsirabe - Tana SanteNet / Training Resources Group, Inc BCC Manager CSTS Mini-University 5 2006 Baltimore, USA CSTS QOC Manager, former Quality improvement in Health Systems Management 30 2004 Toliara MINSANPF INSPC/ITEM/CESAG QOC Manager IMCI 6 2006 Toliara MCDI/MSPF Medical Care Development International 109 TPCSP Final Evaluation Trainee Topic Duration Year Location Sponsoring organization HIS Manager, former CSTS Mini-University 5 2003 Baltimore, USA CSTS Program Design, Monitoring and Evaluation 10 2005 Tana CSTS/Flex Fund Training of trainers (advanced) 15 2006 Antsirabe - Tana SanteNet / Training Resources Group, Inc HIS Manager SPSS 5 2006 Tana Training of trainers 10 2003 Toliara MCDI/MINSANPF C-IMCI, ENA, FP, HIV/AIDS, communication skills, negotiation skills, Malaria, Diarrhea, ARI, EPI, Safe Motherhood (TBA), supervision techniques 3-5 each 2003 to 2004 Toliara MCDI/MINSANPF French 30 2004 Toliara Computer skills 20 2004 Toliara Eurelec Training of trainers on promotion of health products though community-based sales 5 2005 Tana PSI Madagascar Training of trainers on promotion of health products though community-based sales 5 2005 Tana PSI Madagascar Training of trainers on Kôminina Mendrika 10 2005 Tana/Toliara SanteNet/USAID Field Agents Message production for radio communication 7 2005 Toliara ALT-radio Source: MCDI Madagascar, February 2007. Medical Care Development International 110 TPCSP Final Evaluation t International 111 TPCSP Final Evaluation Appendix 12. TPCSP Human Resources Name Position Start End Comments RATSIRARSON Joséa Project Manager 1997 Also MCDI National Coordinator RAKOTOARISOA Rosa QOC Manager 1997 2006 Accepted position as BASICS representative RAZAFIMAHEFA Mialy Administrator, associate 1997 RAZAFIHARISON Andrianasolo Driver 1997 RAKOTONANDRASANA Nary Guard 1997 2006 Deceased ALIJIMY Fabien Administrator 1999 TSABOTOVELOMANDROSO Guard 1999 FANOMEZA Rija HIS Manager 1999 2005 Promoted Project Manager, Flex Fund/MCDI (Ihosy) MODESTE Victor Guard 2000 RAKOTONANDRASANA Emile Driver 2001 2004 Dismissed (DUI) RAKOTOBE Jeanne Field Agent 2003 RAKOTONDRAMAMY Andry Field Agent 2003 ANDRIAMAHEFA Oliva BCC Manager 2003 RASOANIRINA Elysée Secretary 2003 2005 Accepted position at ROHFP RANDRIANTSOA Edmée Field Agent 2003 RAZAFINATOANY Dany Field Agent 2003 RABEKOTO Heritiana Field Agent 2003 RAHERIMANDIMBIRAY Samueline Field Agent 2003 2004 Accepted position at DOHFP Toliara II RAZAKASON Oly Field Agent 2004 MAGISTRE Robert Guard 2004 ANDRIANANDRINTANY Rivo Driver 2004 Medical Care Developmen ELAHIKE Jean Guard 2004 2005 Dismissed ANDRY Bienvenu Guard 2005 2005 Dismissed RAKOTOZAFY Jerry HIS Manager 2006 RANDRIANARISOA Carène Secrétaire 2006 RAHARIJIMY Patrick Assistant Administrator 2006 VELONARISOA Mutuelle technician 2006 RAHARINIAINA Régis Guard 2006 2006 Accepted position at SantéNet RAZAFIMANANTSOA Nelson QOC Manager 2006 Medical Care Development International 112 TPCSP Final Evaluation Name Position Start End Comments Staff detached from Ministry of Communication MOMA Rural Animation Delegate, Betioky Sud 1997 IBRAMDJEE Georges Chief of Regional Service of Rural, Toliara 2003 2005 Retired CHARLES Rural Animation Delegate, Toliara 2003 2004 Accepted position as Assistant to the Mayor, Toliara 113 Medical Care Development International Toliara Province Child Survival Project (TPCSP) Madagascar Report on KPC Final Survey 2006 March 2007 114 Table of Contents ACKNOWLEDGMENTS ..................................................................115 EXECUTIVE SUMMARY ................................................................116 INTRODUCTION ..............................................................................120 METHODOLOGY..............................................................................121 FINDINGS AND DISCUSSION......................................................125 Annexes Annex 1. Questionnaires in French.....................................................................130 Annex 2: TP Child Survival Indicators by stratum ......................................149 Annex 3: Clusters..........................................................................................................153 115 Acknowledgments This Report is based on the findings from TPCSP’s Final KPC Survey carried out by Medical Care Development International. The survey was carried out between November and December of 2006 by a team led by Dr Josea Ratsirarson, Survey Field Coordinator and Dr Jérémia Rakotozafy, HIS Manager with assistance from the staff of the Health Regional Direction. Special thanks go to the village leaders and communities and the Health Regional Direction Staff of all health sector visited who allowed MCDI to conduct the survey. 116 Executive summary During 2002-2006, MCDI carried out the Child Survival Project in District Health of Toliara II including District Health Betioky. The project was financed by USAID-BHR/BVC. The goal of the Project was to reduce morbidity and mortality among children less than 5 years of age and to improve the health status of women of reproductive age (WRA) in the Toliara II District Health, in the South of Madagascar. Intervention included Breastfeeding, Diarrheal Disease, Immunization, IMCI and Child spacing. MCDI opted strategy based on community activities (BCC) for the promotion of household behavior keys, institutional capacity building and synergy between all different partners and promoters of health intervention through a multisectoral platform to reinforce Child Survival activities. The Knowledge, Practices and Coverage (KPC) FINAL survey for TPCSP was carried out between November and December of 2006 in 30 clusters in Toliara II and 30 clusters of Betioky. The survey was conducted by MCDI with assistance from Regional MOH staff. The survey served as an instrument for designing and monitoring/evaluating the project objectives. A detailed discussion of targets achieved in included elsewhere (Final Evaluation Report). A summary of descriptive findings: Breastfeeding and Nutrition ƒ In Toliara, 54% of the mothers said they began breastfeeding within the first hour after delivery, in Betioly the corresponding percentage was 53%. ƒ In Toliara 78% of the mothers said they gave colostrum to their last child, the corresponding proportion for Betioky was 79%. ƒ In Toliara 41% of infants under the age of 6 months were being exclusively breastfed, the corresponding proportion in Betioky was 42%. ƒ In Toliara 28 % of the mothers could mention at least 3 benefits of colostrum. The corresponding proportion in Betioky was 5%. ƒ In Toliara 10, 23 and 66% of the mothers could name at least three, two and one benefits of exclusive breastfeeding, respectively. The corresponding proportions in Betioky were 12%, 36% and 98%. ƒ In Toliara 48% of children of 12-23 months received 5 or more meals per day (meals and snacks) in addition to breast milk. The corresponding proportion in Betioky was 80%. ƒ Both in Toliara and Betioky 100% of children 6-9 months old received breastmilk and complementary foods in the 24h before the survey. ƒ In Toliara 18% of children 0-23 months were underweight, in Betioky 21% were. 117 ƒ In Toliara 59% of mothers stated that they received iron folic acid during the last pregnancy. The corresponding proportion in Betioky was 47%. ƒ In Toliara 79% reported that their children 6 to 23 months received a dose of vitamin A during the last 6 months. According to child health card, this percentage is only 21%. In Betioky the corresponding proportions were 76% and 26%, respectively. Diarrheal Disease ƒ In Toliara, 39% of sampled children had diarrhea in the two week period before the survey. The corresponding proportion in Betioky was 47%. ƒ In Toliara during the last diarrheal episode, 66% of children were given more breast milk than usual or the same amount, 52% received more fluids and 54% were given more or the same amount of solid and semi-solid foods. The corresponding proportions in Betioky were 90%, 65% and 77% respectively. ƒ In Toliara 43% of children recovering from diarrhea received more food than usual in the week following the illness. The corresponding proportion in Betioky was 48%. ƒ In Toliara 74% of mothers sought treatment at the health center in 24 hours of the first danger sign for diarrheal disease in the child. The corresponding proportion in Betioky was 52%. ƒ In Toliara 64% of the mothers knew at least two danger signs for diarrhea. The corresponding proportion in Betioky was 66%. ƒ In Toliara 5% of mothers of children 0-23 months report washing their hands with soap/detergent/ash before preparing food, before feeding the children, after defecating and after having changed a child who has defecated. The corresponding proportion in Betioky was 6%. Respiratory Illness ƒ In Toliara 68% of children in the sample had suffered from ARI with cough or difficult breathing in the two weeks preceding the survey. The corresponding proportion in Betioky was 64%. ƒ In Toliara 63% of mothers know at least 2 danger signs during an ARI, the corresponding proportion in Betioky was 46%. ƒ In Toliara, during the respiratory illness episode, 70% of children were given more breast milk than usual or the same amount, 38% received more fluids and 57% were given more or the same amount of solid and semi-solid foods. The corresponding proportions in Betioky were 82%, 51% and 73% respectively. 118 ƒ In Toliara 56% of mothers sought treatment at the health center in 24 hours of the first danger sign for ARI in the child. The corresponding proportion in Betioky was 44%. Malaria In Toliara, 49% of children under two years had had fever/malaria in the two weeks preceding the survey. The corresponding proportion in Betioky was 50%. ƒ In Toliara, during the febrile/malarial episode, 43% of children received more fluids. The corresponding proportion in Betioky was 80%. ƒ In Toliara, 18% of mothers whose child suffered from malaria sought for treatment at the health center at the end of the day of the first sign of malaria, 51% of mothers whose child suffered from malaria sought for treatment at the health center within the 48 hours of the first sign of malaria and 34% of febrile children received an antimalarial. The corresponding proportions in Betioky were 8%, 38% and 31% respectively. ƒ In Toliara 72% of mothers recalled having received malarial chemoprophylaxis during the pregnancy of their last child. The corresponding proportion in Betioky was 50%. ƒ In Toliara 74% of households possessed at least one bednet, and 58% of children between 0-23 months slept under an insecticide treated bed net during the last night. The corresponding proportions in Betioky were 58% and 43%, respectively. IMCI: ƒ In Toliara, 77% of mothers could name at least 2 danger signs on child illnesses. The corresponding proportion in Betioky was 73%. Immunization ƒ In Toliara 71% of children 12-23 months were fully immunized (against the five vaccine preventable illnesses) per vaccination card, 57% of children of 12-23 months possessed a notebook of immunization according to the mother’s report. The corresponding proportions in Betioky were 46% and 67% respectively. ƒ In Toliara DPT default rate was 10% of children 12-23 months. The corresponding proportion in Betioky was 23%. ƒ In Toliara 30% of mothers could name one benefit of vaccinating children, and 12% could name at least three. The corresponding proportions in Betioky were 52% and 11%, respectively. ƒ In Toliara 50% of mothers had received 2 or more doses of Tetanus Toxoid vaccine before the birth of their youngest child per vaccination card. The corresponding proportion in Betioky was 37% 119 ƒ In Toliara 38% of mothers knew the benefits of being vaccinated against tetanus during pregnancy. The corresponding proportion in Betioky was 56%. Maternal and Prenatal Care ƒ In Toliara 41% of mothers had at least two prenatal visits during the last pregnancy according by maternal health card. The corresponding proportion in Betioky was 42%. ƒ In Toliara 41% of births of children 0-23 months were assisted by a health professional and 43% were in a health center. The corresponding proportions in Betioky were 34% and 26%, respectively. Child Spacing ƒ In Toliara 87% of the last children 0-23 months were born at least 24 months after the preceding living child. The corresponding proportion in Betioky was 56%. ƒ In Toliara 24% of mothers of children 0-23 months who were not pregnant, did not want another child in the next two years or were not sure were using a modern method of contraception. The corresponding proportion in Betioky was 28%. ƒ In Toliara 4% of mothers knew that exclusive breastfeeding is a mean for birth spacing, and 59% know at least one benefit of birth spacing. The corresponding proportion in Betioky was 3%. HIV/Aids ƒ In Toliara 48% of mothers of children 0-23 months could name at least two ways to reduce the risk of HIV infection. The corresponding proportion in Betioky was 21%. 120 Introduction Background Medical Care Development (MCD) is a health planning, management, research, and training organization established in Augusta, Maine in 1966. The International Division of MCD, Medical Care Development International (MCDI), is a PVO based out of Washington D.C. MCDI has over 25 years of experience in carrying out health projects in 30 countries in Africa, Latin American and the Middle East. From 2002 to 2006 MCDI executed a child survival project in the District of Betioky and Toliara, which is located in the Toliara Province of Madagascar. Health Conditions With an estimated population size of 16 million and a GDP of US$250 per capita, Madagascar is among the poorest countries in the world. The Malagasy population is Madagascar’s youngest; over half are less than 20 years old (median age of the population is 16.3 years old) and 18.3% of the population is less than 5 years old. The majority of households do not have toilets and three quarters of the Malagasy people do not have access to potable water. Malagasy life expectancy is 57 years, and 31 Malagasy in 100 die before the age of forty. 58 children per 1000 live births die before their first birthday, and for every 1000 children born, 94 die before reaching the age of 5. Mortality is lower in urban areas than in rural areas, and is especially low in the capital. Malaria, respiratory infections, bilharziasis and Tuberculosis are endemic. 48% of children aged less than 5 have impaired growth and 40% are underweight. TPCSP Final KPC Survey The primary objective of the survey was to compare initial baseline indicators with those achieved at the end of the project. This report focuses on the descriptive findings since the Final Evaluation Report includes a detailed analysis comparing baseline and final values to assess if the targets included in the DIP were reached. 121 Methodology The Questionnaire The generic questionnaire of the KPC 2000+ was used to produce 5 kinds of questionnaires: ‰ 1 questionnaire, enclosing the majority of Rapid CATCH indicators, was for mothers of children between 0-23 months, ‰ 1 questionnaire, engendering the information on exclusive breastfeeding practice, intended for mothers of children under 6 months, ‰ 1 questionnaire, for mothers of children between 6-9 months, to assess the introduction of complementary food ‰ 1 questionnaire, for mothers of children 12-23 months, to assess the child immunization status and complementary feeding over 12 months ‰ and 1 questionnaire, for women between 15-45 years, to assess the use of modern contraceptive methods Final questionnaires were translated into the local dialect. Before being finalized, they were pre-tested by the survey supervisors as part of their training. A final refinement was carried out during the supervisors’ training. Questionnaires included questions relevant to the interventions in TCSP’s interventions (Toliara and Betioky), plus Rapid CATCH indicators. Since this final KPC was planned in August 2006, Rapid CATCH indicators introduced after August 2006 could not be included. Selection of the Sample Within each stratum (Toliara II and Betioky), each questionnaire was administrated to 300 mothers of children between 0-23 months from households selected at random. The selection process of households was that described in KPC 2000+ Field Guide and instructions and guidelines for supervisors were elaborated before their training. To reduce the homogeneity of samples (the sampling effect dEf) the selection process of households was bound by the two following points: 9 selection of the first household: from a principal household of each of selected village quadrant; 9 selection of the next household: the third household the door of which is the nearest to the preceding household. Sampling in parallel was applied to the following specific groups: 9 mothers of children between 0 and 23 months; 122 9 mothers of children under 6 months, to analyze breastfeeding in children under 4 and 6 months; 9 mothers of children between 6 and 9 months, to assess the introduction of complementary feedings; 9 children between 12 and 23 months, to assess immunization and the continuation of breastfeeding/alimentation over 12 months, combined with an “Over Sampling” to be applied for the groups of 12 to 23 months of age who have received DPT1 to analyze the loss rate between DPT1 and DPT3; 9 women of 15-45 years of age, to assess modern contraceptive use and HIV. The “Over Sampling” method was applied to the following specific groups: 9 children of 0-23 months of age breastfed and/or are being breastfed, to assess the breastfeeding-initiation; 9 children of 6-23 months of age, to analyze nutritional supplementation; 9 mothers of children between 0-23 months who are not pregnant and do not wish to have more children in the next 2 years; 9 children of 0-23 months of age who were sick within the 2 weeks preceding the survey. It is to be noted that for each specific age group over-sampled, questionnaires were administrated to at least 7 mothers in order to assure a reasonable minimum number of answers in each of those age groups. As suggested by TPCSP’s mid-term evaluation, data on family planning, child spacing and maternal care was collected from women of reproductive age (15 to 45 years). Training of Supervisors and Interviewers The training of supervisors and interviewers was done in Toliara country with the local technical staff of MOH. The MCDI survey team selected supervisors and interviewers from the Southwest area. The interviewers were selected based on basic criteria such as literacy, availability and gender. Several of the survey supervisors were identified through their involvement with MCDI's local partner organizations and the Regional MOH staff. All supervisors were health service providers. The training of supervisors and interviewers took place over four days, including two days for field testing the survey instrument, and actual interviewing practice. Prior to the first day of training, the MCDI team reviewed the training design and delegated responsibility for various sessions to the three survey trainers. The first training day was dedicated to an introduction (who is USAID, who is MCDI), group discussions of important child survival health issues in Betioky and Toliara II, administration, and survey methodology. The MCDI team instructed the trainees in the following: (a) purpose and objectives of the survey, (b) selection of the sample size, (c) 123 selection of the 30 clusters/villages in each stratum, (d) random selection of starting household and next household selection, and (e) roles of supervisors and interviewers. The next day of training consisted of the following: (a) reviewing of the survey (question by question) in order to familiarize participants with each module and understand the intention of each question, (b) focusing on interviewing techniques, (d) determining age of child, (e) confidentiality, (e) over sampling techniques and (f) practicing interviewing techniques and the roles of the supervisor and interviewers. The third day of training commenced with a field test of the survey questionnaire. The survey team selected the villages of Betsinjaka, located 15 minutes from the training site. Each supervisor and surveyor interviewed from two to three mothers of children 0-23 months old. The final day of training involved instruction on data entry and manual tabulation. The participants entered data from questionnaires filled out during the field exercise. The practice data forms were double checked by supervisors and the core survey team. Conducting the Interviews The survey was conducted in November and December 2006 in Toliara II and Betioky. The core survey team and supervisors went out to the selected clusters prior to visit appropriate officials for consent to carry out the survey and to notify them of the survey teams' imminent arrival. The supervisors of each team were responsible for the selection of the starting household and survey direction. The supervisors observed at least two complete interviews by each interviewer each day. Each questionnaire was checked for completeness before the survey team left the survey area, so that in the event of missing or contradictory information, the mother could be visited again the same day. In order to ensure consent, interviewers verbally asked the mothers for their consent to carry out the interview and provided assurance of confidentiality. The consent form advised the potential respondent that she was not forced to participate in the survey. The interviewers were required to sign each form verifying that it had been read to the mother, and that she had consented to participate. The household surveys were done in parallel with health facility assessment in the same clusters (the facility nearest to the cluster), and followed by Focus Groups to explain findings. The detailed HFA and the findings of the Focus Groups have not been included in this report. Data Analysis The data was tabulated and analyzed using Microsoft Excel and exported to SPSS (Statistical Package for the Social Sciences) available from SPSS, Inc. SPSS was initially chosen for the 124 survey analysis since it is a more advanced software program than EPI-INFO, providing project staff with the flexibility to conduct more rigorous analyses. The bulk of the data analysis consisted of frequency distributions for each of the questions and key cross with age of infants for questions related to breastfeeding/nutrition and vaccinations. 125 Findings and Discussion Breastfeeding and Nutrition Breastfeeding initiation: In Toliara, 54% of the mothers said they began breastfeeding within the first hour after delivery; in Betioky the corresponding percentage was 53%. Exclusive Breastfeeding: In Toliara 41% of infants under the age of 6 months were being exclusively breastfed; the corresponding proportion in Betioky was 42%. According to DHS 03-04, in Madagascar 67.1% (weighted average) of children younger than 6 months had received exclusive breastfeeding. Both in Toliara and Betioky, the proportion with EBF seems to have declined between 2002 and 2006. As discussed in the Final Evaluation report, this deserves further investigation. Hypotheses to test may include a) EBF rate was higher in 2002 because social instability at that time reduced commerce of breast milk substitutes, as described for extreme upheaval, and b) EBF rate in 2006 decreased because an increased participation of women in the work force. Focus groups have confirmed that women are not taking infants to their workplaces. Feeding times: In Toliara 48% of children of 12-23 months received 5 or more meals per day (meals and snacks) in addition to breast milk. The corresponding proportion in Betioky was 80%. Both in Toliara and Betioky 100% of children 6-9 months old received breastmilk and complementary foods in the 24h before the survey. Anthropometry: In Toliara 18% of children 0-23 months were underweight, in Betioky 21% were. The corresponding prevalence in Madagascar (DHS 03-04) for children below 2 years of age was 36.6%. Micronutrient supplementation: The supplementation in micronutrients was in general low: In Toliara 59% of mothers stated that they received iron and folic acid during the last pregnancy. The corresponding proportion in Betioky was 47%. In Toliara 79% reported that their children 6 to 23 months received a dose of vitamin A during the last 6 months. According to child health card, this percentage is only 21%. In Betioky the corresponding proportions were 76% and 26%, respectively. According to DHS 03-04, in Madagascar 69.2% (weighted average) of children 6-23mo had received Vitamin A supplements. Diarrheal Disease Diarrhea Period Prevalence: At the time of our survey, Toliara, 39% of sampled children had diarrhea in the two week period before the survey. The corresponding 126 proportion in Betioky was 47%. According to DHS 03-04, the weighted period prevalence rate of diarrhea for children younger than 24 months was 14.3%. Dietary management of Diarrhea: During the diarrheal episode, in Toliara during the last diarrheal episode, 66% of children were given more breast milk than usual or the same amount, 52% received more fluids and 54% were given more or the same amount of solid and semi-solid foods. The corresponding proportions in Betioky were 90%, 65% and 77% respectively. In Toliara 43% of children recovering from diarrhea received more food than usual in the week following the illness. The corresponding proportion in Betioky was 48%. DHS 03-04 does not report this by age group. For children under 5 years of age with diarrhea, 29.4% continued feeding as usual, and 9.1% received more food. 34.9% had more than the usual amount of fluids during the episode. Care Seeking: In Toliara 74% of mothers sought treatment at the health center in 24 hours of the first danger sign for diarrheal disease in the child. The corresponding proportion in Betioky was 52%. According to DHS 03-04 data, overall proportion of children younger than 24 months with diarrhea (not only cases with dehydration or other danger signs) taken to a health facility was 32.1%. Handwashing: Of the all households in the sample, in Toliara 5% of mothers of children 0-23 months report washing their hands with soap/detergent/ash before preparing food, before feeding the children, after defecating and after having changed a child who has defecated. The corresponding proportion in Betioky was 6%. According to DHS 03-04, 51% of homes had some sort of facility for handwashing. Respiratory Illness ARI Period Prevalence: In Toliara 68% of children in the sample had suffered from ARI with coughing or difficult breathing in the two weeks preceding the survey. The corresponding proportion in Betioky was 64%. According to DHS 03-04, the weighted period prevalence rate of ARI for children younger than 24 months was 11%. Dietary Management of ARI: In Toliara, during the respiratory illness episode 70% of children were given more breast milk than usual or the same amount, 38% received more fluids and 57% were given more or the same amount of solid and semi-solid foods. The corresponding proportions in Betioky were 82%, 51% and 73% respectively. DHS does not provide information on dietary management of respiratory infections. Care Seeking: In Toliara 56% of mothers sought treatment at the health center in 24 hours of the first danger sign for ARI in the child. The corresponding proportion in Betioky was 44%. According to DHS 03-04, the weighted proportion of ARI cases younger than 24 months that were treated at a health facility was 43.6%. 127 Knowledge of Danger Signs: In Toliara 63% of mothers know at least 2 danger signs during an ARI, the corresponding proportion in Betioky was 46%. Malaria Fever Period Prevalence: In Toliara, 49% of children under two years had had fever/malaria in the two weeks preceding the survey. The corresponding proportion in Betioky was 50%. According to DHS 03-04, the weighted period prevalence rate of fever for children younger than 24 months was 25.2%. Fluids: During the malaria episode, in Toliara 43% of children received more fluids. The corresponding proportion in Betioky was 80%. Care Seeking: In Toliara, 18% of mothers whose child suffered from malaria sought for treatment at the health center by the end of the day of the first sign of malaria. 51% of mothers whose child suffered from malaria sought for treatment at the health center within 48 hours of the first sign of malaria. 34% of febrile children received an antimalarial. The corresponding proportions in Betioky were 8%, 38% and 31% respectively. Chemoprophylaxis/IPT: In Toliara 72% of mothers recalled having received malarial chemoprophylaxis during the pregnancy of their last child. The corresponding proportion in Betioky was 50%. According to DHS 03-04, in Madagascar 58% of women had antimalarial chemoprophylaxis in her last pregnancy. Bednet ownership and use: In Toliara 74% of households possessed at least one bednet, and 58% of children between 0-23 months slept under an insecticide treated bed net during the last night. The corresponding proportions in Betioky were 58% and 43%, respectively. According to DHS 03-04, 38.9% of households with children under 5 years of age owned at least a bednet, while 36% of children under 5 years of age slept under a bednet the night before the survey. IMCI: Danger signs: In Toliara, 77% of mothers could name at least 2 danger signs on child illnesses. The corresponding proportion in Betioky was 73%. Immunization Vaccination card and full coverage: In Toliara 71% of children 12-23 months were fully immunized (against the five vaccine-preventable illnesses) per vaccination card, 57% of children of 12-23 months possessed a notebook of immunization according to the mother’s report. The corresponding proportions in Betioky were 46% and 67% respectively 128 According to DHS 03-04, 50.2% of children 12-23 months of age had a vaccination notebook, while 52.9% of children 12-23 months of age were fully immunized. DTP default rate: In Toliara DPT default rate was 10% of children 12-23 months. The corresponding proportion in Betioky was 23%. Knowledge: In Toliara 30% of mothers could name one benefit of vaccinating children, and 12% could name at least three. The corresponding proportions in Betioky were 11% and 52%, respectively. In Toliara 38% of mothers knew the benefits of being vaccinated against tetanus during pregnancy. The corresponding proportion in Betioky was 56%. Tetanus: In Toliara 50% of mothers had received 2 or more doses of Tetanus Toxoid vaccine before the birth of their youngest child per vaccination card. The corresponding proportion in Betioky was 37% Maternal and Prenatal Care Antenatal care: In Toliara 41% of mothers had at least two prenatal visits during the last pregnancy according by maternal health card. The corresponding proportion in Betioky was 42%. According to DHS 03-04, 80% of mothers had at least one prenatal control. Birth care: In Toliara 41% of births of children 0-23 months were assisted by a health professional and 43% were in a health center. The corresponding proportions in Betioky were 34% and 26%, respectively. According to DHS 03-04, 51% of births were assisted by a health professional. Child Spacing Birth interval: In Toliara 87% of the last children 0-23 months were born at least 24 months after the preceding living child. The corresponding proportion in Betioky was 56%. According to DHS 03-04, 33% of children had an interval of 24-35 months with the preceding sibling. Contraceptive use: In Toliara, 24% of non-pregnant mothers of children 0-23 months, who did not want another child in the next two years, were not using a modern method of contraception. The corresponding proportion in Betioky was 28%. According to DHS 03-04, only 18.3% of women with children younger than 5 years use any modern contraceptive method. 129 LAM: In Toliara 4% of mothers knew that exclusive breastfeeding is a mean for birth spacing, while 59% know at least one benefit of birth spacing. The corresponding proportion in Betioky was 3%. According to DHS 03-04, 37.7% of all women of reproductive age mentioned LAM (the Lactation Amenorrhea Method). HIV/Aids Prevention: In Toliara 48% of mothers of children 0-23 could name at least two ways to reduce the risk of HIV infection. The corresponding proportion in Betioky was 21%. According to DHS 03-04, 50.8% name condoms as a way to reduce risk of getting HIV, 60.4% named reducing the number of sexual partners. 130 Annex 1. Questionnaires in French Questionnaire #1. A poser aux mères d’enfant de 0 à 23 mois CARACTÉRISTIQUES GÉNÉRALES DE L’ENQUÊTÉE No . QUESTIONS ET FILTRES CODES PASSER À ANNÉES À L’ÉCOLE................... A Pendant combien d’années avez-vous fréquenté l’école1 ? SI JAMAIS, ENREGISTREZ >00'. Adapter selon : sait lire/écrire/pas du tout ??? B Travaillez-vous à l’extérieur de la maison pour gagner de l’argent ? SI NON, ENCERCLEZ AA@ (PAS DE TRAVAIL À L’EXTÉRIEUR) SI OUI, quel genre de travail faites-vous ? PAS DE TRAVAIL À L’EXTÉRIEUR......... A ARTISANAT.............................................. B RÉCOLTE................................................. C VENTE DE NOURRITURE....................... D COMMERCE/VENDEUSE DE RUE......... E SERVANTE/EMPLOYÉE DOMESTIQUE .F TRAVAIL SALARIÉ................................... G AUTRE__________________________ X (PRÉCISEZ) Î D C Qui s’occupe de (NOM) quand vous êtes absente de la maison ? MÈRE (ENQUÊTÉE) ................................ A MARI/PARTENAIRE................................. B ENFANTS PLUS ÂGÉS............................ C AUTRES PARENTS ______________ .... D (PRÉCISEZ VOISINS/AMIS ......................................... E BONNE ......................................................F JARDIN D’ENFANT.................................. G AUTRE__________________________ X (PRÉCISEZ) HYGIENE No QUESTIONS ET FILTRES CODES PASSER À D Quel type de toilettes les membres du ménage utilisent-ils? TOILETTES AVEC CHASSE D’EAU.......11 FOSSES/LATRINES FOSSES TRADITIONNELLES...........21 FOSSES AMÉLIORÉES VENTILÉES 22 PAS DE TOILETTES/NATURE ...............31 AUTRE ________________________ 96 (PRÉCISEZ) ÎF E Utilisez-vous ces toilettes en commun avec d’autres ménages ? OUI ............................................................1 NON...........................................................2 F Dans le ménage, que faites-vous des selles des bébés et des jeunes enfants qui n’utilisent pas les toilettes ? JETÉES DANS TOILETTES/LATRINES...1 ENTERRÉES DANS LA COUR 2 PAS JETÉES/LAISSÉES PAR TERRE 3 AUTRE_________________________ 96 (PRÉCISEZ) 131 ALLAITEMENT MATERNEL No QUESTIONS ET FILTRES CODES PASSER À 1 Avez-vous allaité (NOM)? OUI........................................................... 1 NON ......................................................... 2 Î3 2 Combien de temps après la naissance avez-vous commencé à allaiter (NOM)?1 IMMÉDIATEMENT/PENDANT LA PREMIÈRE HEURE SUIVANT L’ACCOUCHEMENT.. 1 APRÈS LA PREMIÈRE HEURE .............. 2 2 bis Donnez vous actuellement le lait maternel à (NOM) OUI........................................................... 1 NON ......................................................... 2 Î 3 2 ter Si non, vous avez allaité pendant combien de mois ? [ ] Mois 3 Est-ce que (NOM) a reçu une dose de Vitamine A comme celle-ci au cours des six derniers mois ?7 MONTREZ L’AMPOULE/CAPSULE/SIROP . A POSER SEULEMENT AUX MERES D’ENFANT DE 6 à 23 MOIS OUI 1 NON 2 NE SAIT PAS ………………………..3 NON APPLICABLE ………………………..4 PRISE EN CHARGE DE LA DIARRHEE NO . QUESTIONS ET FILTRES CODES PASSE R À 4 Habituellement, quand vous lavez-vous les mains avec du savon/cendre ? ENREGISTREZ TOUT CE QUI EST MENTIONNÉ. JAMAIS .................................................... A AVANT DE PRÉPARER LA NOURRITURE ................................. B AVANT DE NOURRIR LES ENFANTS.... C APRÈS ÊTRE ALLÉ À LA SELLE............ D APRÈS AVOIR NETTOYÉ UN ENFANT QUI EST ALLÉ À LA SELLE E AUTRE__________________________ X (PRÉCISEZ) 5 Est-ce que (NOM) a eu la diarrhée au cours des deux dernières semaines ? 1 OUI ............................................................1 NON ..........................................................2 NE SAIT PAS ............................................8 Î 15 Î 15 6 Qu’avez vous donné pour traiter la diarrhée de (NOM) ? Rien d’autre ? ENREGISTREZ TOUT CE QUI EST MENTIONNÉ. RIEN......................................................... A SOLUTION SACHET SRO....................... B PRÉPARATION MAISON ........................ C PILULE OU SIROP .................................. D INJECTION .............................................. E (IV) INTRAVEINEUSE.............................. F REMÈDES MAISON/ MÉDICAMENTS TRADITIONNELS.........G ANTIDIARRHEIQUE ………………………H AUTRE__________________________ X (PRÉCISEZ) 7 Quand (NOM) a eu la diarrhée, l’avez-vous allaité moins que d’habitude, environ la même quantité ou plus que d’habitude ? MOINS.......................................................1 MÊME........................................................2 PLUS .........................................................3 ARRET DE L’ALLAITEMENT....................4 132 NO . QUESTIONS ET FILTRES CODES PASSE R À A NE PAS POSER SI L’ENFANT NE S’ALLAITE PLUS (Q 2 bis NON) NON APPLICABLE…………………………5 NE SAIT PAS ............................................8 7 bis Est ce que (NOM) est actuellement sous Allaitement Maternel Exclusif ? A POSER SEULEMENT AUX ENFANTS DE MOINS DE 06 MOIS OUI ........................................................... 1 NON ......................................................... 2 NE SAIT PAS……………………………….3 Î9 8 Quand (NOM) a eu la diarrhée, lui avez-vous donné moins à boire que d’habitude, environ la même quantité ou plus que d’habitude ? A NE PAS POSER SI L’ENFANT EST SOUS A.M.E MOINS.......................................................1 MÊME........................................................2 PLUS .........................................................3 ARRET DES BOISSON ............................4 NON APPLICABLE ………………………..5 NE SAIT PAS ............................................8 9 Lui avez-vous donné moins à manger que d’habitude, environ la même quantité ou plus que d’habitude ? A NE PAS POSER SI L’ENFANT EST SOUS A.M.E MOINS.......................................................1 MÊME........................................................2 PLUS .........................................................3 ARRET DE L’ALIMENTATION..................4 NON APPLICABLE ………………………..5 NE SAIT PAS ............................................8 10 Avez-vous recherché des conseils ou un traitement auprès de quelqu’un à l’extérieur de la maison pour la diarrhée de (NOM) ? OUI ............................................................1 NON ..........................................................2 Î15 11 Quels signes vous ont motivé pour chercher des conseils ou traitement de la diarrhée de (NOM) ? NQ NE SEMBLE PAS BIEN OU NE JOUE PAS NORMALEMENT …………………. 1 NE MANGE PAS OU NE BOIT PAS……2 APATHIQUE SE RÉVEILLE DIFFICILEMENT ………………………... 3 FIÈVRE ÉLEVÉE …………………….…..4 VOMIT TOUT……………………………...5 Diarrhée prolongée de plus de 14 j …….6 Sang ou glaire dans les selles ………….7 Fontanelle déprimée (HEVO) …………..8 Froideur des extrémités …………………9 Globe oculaire enfoncé ………….……10 Perte de connaissance ………………..11 AUTRE I (PRÉCISEZ) AUTRE J (PRÉCISEZ) 12 Où êtes-vous allé pour rechercher un conseil un traitement ? 3 SI LA SOURCE EST L’HÔPITAL, LE CENTRE DE SANTÉ OU LA CLINIQUE, INSCRIVEZ LE NOM DE L’ENDROIT _______________________________________________________ (NOM DE L’ENDROIT) ÉTABLISSEMENT DE SANTÉ HÔPITAL..............................................01 CENTRE DE SANTÉ ...........................02 POSTE DE SANTÉ..............................03 CENTRE ONG .....................................04 CLINIQUE ............................................05 AGENT DE SANTÉ DE TERRAIN/COMMUNAUTAIRE 06 AUTRE ÉTABLISSEMENT DE SANTÉ ____________________07 (PRÉCISEZ) AUTRE SOURCE GUÉRISSEUR TRADITIONNEL ..........08 BOUTIQUE...........................................09 PHARMACIE........................................10 DISTRIBUTEURS COMMUNAUTAIRES.....................11 AMIS/PARENTS ..................................12 AUTRE_________________________ 88 (PRÉCISEZ) 13 Pendant la diarrhée de (NOM), est ce qu’il/elle a présenté : 1. une diarrhée prolongée de plus de 14 jours Diarrhée prolongée de plus de 14 jours….1 Sang ou glaire dans les selles…………….2 133 NO . QUESTIONS ET FILTRES CODES PASSE R À 2. du sang ou glaire dans les selles 3. aucune des deux 4. ne sait pas Aucune des deux…………………………...3 Ne sait pas…………………………………..4 Î15 Î15 14 Avez vous cherché traitement pour ces manifestations OUI…………………………………………..1 NON…………………………………………2 NE SAIT PAS………………………………3 15 A votre connaissance, quels sont les signes qui doivent nécessiter la recherche de traitement en cas de diarrhée de l’enfant ? NQ NE SEMBLE PAS BIEN OU NE JOUE PAS NORMALEMENT …………………. 1 NE MANGE PAS OU NE BOIT PAS……2 APATHIQUE SE RÉVEILLE DIFFICILEMENT ………………………... 3 FIÈVRE ÉLEVÉE …………………….…..4 VOMIT TOUT……………………………...5 Diarrhée prolongée de plus de 14 j …….6 Sang ou glaire dans les selles ………….7 Fontanelle déprimée (HEVO) …………..8 Froideur des extrémités …………………9 Globe oculaire enfoncé ………….…..…10 Perte de connaissance ……………..…..11 AUTRE I (PRÉCISEZ) AUTRE J (PRÉCISEZ) 134 Appendix 13. IMMUNIZATION NO . QUESTIONS ET FILTRES CODES PASSE R À 16 Avez-vous un carnet de santé maternel pour la grossesse de (NOM) ? OUI, VU .....................................................1 PAS DISPONIBLE.....................................2 N’A JAMAIS EU DE CARNET...................3 Î18 Î18 17 REGARDEZ LE CARNET ET ENREGISTREZ LES DATES POUR CHAQUE INJECTION DE VAT INSCRITE SUR LE CARNET JOUR MOIS ANNÉE ┌──┬──╥──┬──╥──┬──┬──┬──┐ 1er ......... │░░│░░║░░│░░║░░│░░│░░│░░│ ├──┼──╫──┼──╫──┼──┼──┼──┤ 2 eme ...... │░░│░░║░░│░░║░░│░░│░░│░░│ ├──┼──╫──┼──╫──┼──┼──┼──┤ 3 eme ...... │░░│░░║░░│░░║░░│░░│░░│░░│ ├──┼──╫──┼──╫──┼──┼──┼──┤ 4 eme ...... │░░│░░║░░│░░║░░│░░│░░│░░│ ├──┼──╫──┼──╫──┼──┼──┼──┤ 5 eme ...... │░░│░░║░░│░░║░░│░░│░░│░░│ ├──┼──╫──┼──╫──┼──┼──┼──┤ 6eme....... │░░│░░║░░│░░║░░│░░│░░│░░│ └──┴──╨──┴──╨──┴──┴──┴──┘ 18 Avant d’accoucher de (Nom), avez-vous reçu une injection au bras2 pour empêcher le bébé d’avoir le tétanos, c’est à dire des convulsions après la naissance ? QM SI OUI, MENTIONNER COMBIEN : ┌───┬───┐ NOMBRE D’INJECTION.......│░░░│░░░│ └───┴───┘ OUI ............................................................1 NON...........................................................2 NE SAIT PAS ............................................8 135 Appendix 14. 136 Appendix 15. ESPACEMENT DE NAISSANCE NO . QUESTIONS ET FILTRES CODES PASSE R À 19 Etes-vous actuellement enceinte ? OUI ....................................................... 1 NON ..................................................... 2 PAS SÛRE ........................................... 8 Î23 20 Voulez-vous un autre enfant ? OUI ....................................................... 1 NON ..................................................... 2 NE SAIT PAS ....................................... 8 Î22 Î22 21 Quand souhaiteriez vous avoir votre prochain enfant ? AU COURS DES 2 PROCHAINES ANNÉES........................................... 1 DANS PLUS DE 2 ANS........................ 2 PAS SÛRE QUAND ............................. 8 Î23 22 Actuellement, faites-vous quelque chose ou utilisez-vous une méthode pour retarder ou éviter de tomber enceinte ? SI NON, ENCERCLEZ >01' [PAS DE MÉTHODE] SI OUI, DEMANDEZ : « Quelle est la principale méthode que vous ou votre mari/partenaire utilisez pour retarder/éviter de tomber enceinte ? » ENCERCLEZ LE CODE APPROPRIÉ. PAS DE MÉTHODE ........................... 01 NORPLANT........................................ 02 INJECTIONS ...................................... 03 PILULE ............................................... 04 DIU ..................................................... 05 MÉTHODE DE BARRIÈRE/ DIAPHRAGME ............................... 06 CONDOM ........................................... 07 MOUSSE/GELÉE............................... 08 LIGATURE DES TROMPES .............. 09 VASECTOMIE.................................... 10 ALLAITE. MATERN. AMÉNORRHÉE (ALLAIT. EXCLUSIF) ..................... 11 RYTHME ............................................ 12 ABSTINENCE .................................... 13 RETRAIT ............................................ 14 AUTRE________________________ 96 (PRÉCISER) 23 Maintenant je voudrais vous poser une question concernant les services d’espacement des naissances/planification familiale dans votre communauté. Connaissez-vous un endroit où vous pourriez obtenir une méthode d’espacement des naissances/planification familiale ? SI NON, ENCERCLEZ AZ@ [NE SAIT PAS] SI OUI, DEMANDEZ : « Où est-ce ? » 1 ENREGISTREZ TOUT CE QUI EST MENTIONNÉ. SI LA SOURCE EST UN HÔPITAL, UN CENTRE DE SANTÉ OU UNE CLINIQUE, ÉCRIVEZ LE NOM DE L’ENDROIT. _______________________________________________________ (NOM DE L’ENDROIT) ÉTABLISSEMENT DE SANTÉ HÔPITAL...........................................A CENTRE DE SANTÉ ........................B CENTRE ONG ..................................C POSTE DE SANTÉ ...........................D CLINIQUE PLANNING FAMILIAL.....E AGENT DE SANTÉ DE TERRAIN/ COMMUNAUTAIRE...................... F PHARMACIE.....................................G AUTRE ÉTABLISSEMENT DE SANTÉ____________________ H (PRÉCISER) AUTRE SOURCE BOUTIQUE ........................................ I ÉGLISE ..............................................J AMIS/PARENTS ...............................K AUTRE ________________________ X (PRÉCISER) NE SAIT PAS ....................................... Z 24 Combien d’enfants qui vivent dans ce ménage ont moins de cinq ans ? UN ENFANT......................................... 1 DEUX ENFANTS.................................. 2 TROIS ENFANTS OU PLUS................ 3 Î27 25 Combien de ces enfants sont vos enfants biologiques ? UN ENFANT......................................... 1 Î27 137 NO . QUESTIONS ET FILTRES CODES PASSE R À DEUX ENFANTS.................................. 2 TROIS ENFANTS OU PLUS................ 3 ENFANT #1 SEXE MASCULIN .........1 FÉMININ..............2 DATE DE NAISS. ENFANT #2 SEXE MASCULIN ......... 1 FÉMININ ............. 2 DATE DE NAISS. JOUR...... JOUR ..... MOIS ...... MOIS...... ANNÉE ... ANNÉE... 26 Quel est le sexe et la date de naissance de vos deux plus jeunes enfants ? 138 139 Appendix 16. QUESTIONS COMPLEMENTAIRES MALADIES DE L’ENFANT : NO. QUESTIONS ET FILTRES CODES Aller à 27 Il arrive parfois que les enfants tombent malades et qu’il aient besoin de soins ou de traitements pour la maladie. Quels sont les signes de maladie qui montreraient que votre enfant a besoin d’un traitement? ENREGISTREZ TOUT CE QUI EST MENTIONNÉ. NE SAIT PAS ..................................................... A NE SEMBLE PAS BIEN OU NE JOUE PAS NORMALEMENT ............................................... B NE MANGE PAS OU NE BOIT PAS.................. C APATHIQUE SE RÉVEILLE DIFFICILEMENT D FIÈVRE ÉLEVÉE ............................................... E RESPIRATION RAPIDE OU BRUYANTE.......... F VOMIT TOUT .....................................................G CONVULSIONS ................................................. H AUTRE I (PRÉCISEZ) AUTRE J (PRÉCISEZ) AUTRE K (PRÉCISEZ) 28 EST-CE QUE (NOM) a souffert de l’un des problèmes suivants au cours des deux dernières semaines? ENREGISTREZ TOUT CE QUI EST MENTIONNÉ Diarrhée ? Sang dans les selles ? Toux ? Difficultés pour respirer ? Respiration rapide ou courte, souffle court ? Fièvre ? Paludisme ? Convulsions ? DIARRHÉE......................................................... A SANG DANS LES SELLES................................ B TOUX ................................................................. C DIFFICULTÉS RESPIRATOIRES...................... D RESPIRATION RAPIDE /COURTE SOUFFLE COURT ............................................. E FIÈVRE .............................................................. F PALUDISME.......................................................G CONVULSIONS ................................................. H N’ETAIT PAS MALADE…………………………...I Î 33 29 Est-ce que (NOM) a souffert d’un AUTRE problème au cours des deux dernières semaines ? PRÉCISEZ: ________________________________________ 30 Quand (NOM) a été malade, l’avez-vous allaité moins que d’habitude, environ la même quantité ou plus que d’habitude ? A NE PAS POSER SI L’ENFANT NE S’ALLAITE PLUS (C’est à dire si Q2 bis : NON) MOINS.......................................................1 MÊME........................................................2 PLUS .........................................................3 ARRET DE L’ALLAITEMENT....................4 NON APPLICABLE…………………………5 NE SAIT PAS ............................................8 31 Quand (NOM) a été malade, lui avez-vous donné moins à boire que d’habitude, environ la même quantité ou plus que d’habitude ? A NE PAS POSER SI L’ENFANT EST SOUS A.M.E (C’est à dire si Q7bis : NON ou NE SAIT PAS) MOINS.......................................................1 MÊME........................................................2 PLUS .........................................................3 ARRET DE BOIRE ....................................4 NON APPLICABLE…………………………5 NE SAIT PAS ............................................8 32 Quand (NOM) a été malade, lui avez-vous donné moins à manger que d’habitude, environ la même quantité ou plus que d’habitude ? A NE PAS POSER SI L’ENFANT EST SOUS A.M.E (C’est à dire si MOINS.......................................................1 MÊME........................................................2 PLUS .........................................................3 ARRET DE L’ALIMENTATION..................4 NON APPLICABLE…………………………5 140 Q7bis : NON ou NE SAIT PAS) NE SAIT PAS ............................................8 141 PREVENTION DU PALUDISME : NO. QUESTIONS ET FILTRES CODES Aller à 33 Avez-vous des moustiquaires dans votre maison ? OUI ............................................................. 1 NON............................................................ 2 NE SAIT PAS ............................................. 8 Î 36 Î 36 34 Qui a dormi sous une moustiquaire la nuit dernière ? ENREGISTREZ TOUTES LES PERSONNES MENTIONNÉES. ENFANT (NOM) ......................................... A MOI-MÊME................................................. B MARI/PARTENAIRE...................................C AUTRE _________________________ X (PRÉCISEZ) 35 Est-ce que la moustiquaire a déjà été trempée ou imprégnée avec un insecticide ? OUI ............................................................. 1 NON............................................................ 2 NE SAIT PAS ............................................. 8 142 PRENATAL CARE AND DELIVERY No QUESTIONS ET FILTRES CODES PASSE R À 36 Avez-vous fait des visites prénatales pendant que vous étiez enceinte de (NOM) ?1 OUI ............................................................. 1 NON............................................................ 2 NE SAIT PAS ........................................…8 Î38 Î38 37 Combien de fois avez-vous fait ces VPN durant votre grossesse ? ┌───┬───┐ NOMBRE DE FOIS ...............│░░░│░░░│ └───┴───┘ 38 Avez-vous un carnet de santé maternel pour la grossesse de (NOM) ? OUI, VU .....................................................1 PAS DISPONIBLE.....................................2 N’A JAMAIS EU DE CARNET...................3 Î40 Î40 39 REGARDEZ LE CARNET ET NOTEZ LE NOMBRE DE VISITES PRENATALES EFFECTUÉES PENDANT QUE LA MÈRE ETAIT ENCEINTE DE (NOM). ┌───┬───┐ NOMBRE DE VISITES..........│░░░│░░░│ └───┴───┘ 40 Où avez-vous accouché de (NOM) ? 1 SI LA SOURCE EST UN HÔPITAL, UN CENTRE DE SANTÉ OU UNE CLINIQUE, ÉCRIVEZ LE NOM DE L’ENDROIT. _______________________________________________________ (NOM DE L’ENDROIT) À LA MAISON VOTRE DOMICILE..............................11 AUTRE DOMICILE..............................12 ÉTABLISSEMENT SANITAIRE HÔPITAL .............................................21 CLINIQUE ...........................................22 CENTRE DE SANTÉ...........................23 CENTRE ONG ....................................24 POSTE DE SANTÉ .............................25 AUTRE ÉTABLISSEMENT DE SANTÉ _______________ 26 (PRÉCISEZ) AUTRE_______________ 96 (PRÉCISEZ) 41 Qui vous a assisté pendant l’accouchement de (NOM)? 1 ENREGISTREZ TOUTES LES PERSONNES MENTIONNÉES PROFESSIONNEL DE LA SANTE MÉDECIN.............................................. A INFIRMIERE/SAGE-FEMME ................ B SAGE-FEMME AUXILIAIRE ................C AUTRE PERSONNE ACCOUCHEUSE TRADITIONNELLE D AGENT DE SANTÉ COMMUNAUTAIRE E MEMBRE FAMILLE _____________ F (PRÉCISEZ) AUTRE_______________________ G (PRÉCISEZ) PERSONNE ............................................. Y 143 ANTHROPOMETRIE : No QUESTIONS ET FILTRES CODES PASSE R A 42 Est-ce que (NOM) a un carnet de suivi de croissance ? SI OUI : Est-ce que je peux le voir ? OUI, VU.................................................... 1 PAS DISPONIBLE/PERDU/ÉGARÉ ........ 2 N’A JAMAIS EU DE CARNET.................. 3 NE SAIT PAS ........................................... 8 Î45 Î45 Î45 43 VÉRIFIEZ LE CARNET DE (NOM) POUR VOIR S’IL A ÉTÉ PESÉ AU COURS DES QUATRE DERNIERS MOIS OUI........................................................... 1 NON ......................................................... 2 NE SAIT PAS ........................................... 8 45. DEMANDEZ À LA MÈRE LA PERMISSION DE PESER ET DE MESURER (NOM). SI ELLE ACCEPTE DE VOUS LAISSER PRENDRE LES MENSURATIONS DE (NOM), ENREGISTREZ LES INFORMATIONS NÉCESSAIRES DANS L’ESPACE CI-DESSOUS. SI LA MERE REFUSE QUE L’ON PRENNE LES MENSURATIONS DE (NOM), LAISSEZ LES COLONNES 1-4 EN BLANC ET ENREGISTREZ >3' [REFUSÈ] A LA COLONNE 5. DEMANDEZ DE PRENDRE LES MENSURATIONS DE CHAQUE FRÈRE ET SŒUR DE (NOMS) ÂGÉS DE MOINS DE CINQ ANS. ENREGISTREZ LES MENSURATIONS DE (NOM) SUR LA PREMIÈRE LIGNE. 1 NOM DE L’ENFANT PRENEZ LES MENSURATIONS DE (NOM) EN PREMIER, PUIS CELLES DES FRÈRES ET SŒURS ÂGES DE MOINS DE CINQ ANS. 2 Quelle est sa date de naissance ? RECOPIEZ LA DATE DE NAISSANCE À PARTIR DU CARNET S’IL EST DISPONIBLE. SI LE CARNET N’EST PAS DISPONIBLE, ENREGISTREZ LA DATE FOURNIE PAR LA MÈRE. 3 POIDS (KILOGRAMMES) 4 RÉSULTAT 1 MESURÉ 2 PAS RÉSENT 3 REFUSÉ 6 AUTRE JJ MM ANNÉE 0 . (1) _________________________ 0 . (2) _________________________ 0 . (3) _________________________ 0 . (4) _________________________ 0 . (5) _________________________ Les questions méritant une attention particulière pour l’over sampling (au moins 7 réponses par grappe) : • Q1 : Réponse 1 • Q3 : Réponse 1 ou 2 ou 3 • Q5 : Réponse 1 • Q7 : Réponse 1 ou 2 ou 3 ou 4 ou 8 • Q7 bis : Réponse 2 ou 3 • Q10 : Réponse 1 • Q13 : Réponse 2 ou 3 144 • Q16 : Réponse 1 • Q22 : une réponse cochée • Q30 : Réponse 1 ou 2 ou 3 ou 4 ou 8 • Q31 : Réponse 1 ou 2 ou 3 ou 4 ou 8 • Q42 : Réponse 1 • Q44 : Réponse 1 ou 2 ou 8 • Q45 : avec au moins une réponse V.6.2. Questionnaire #2. A poser aux femmes de 15 à 45 ans (Echantillonnage parallèle) 2. NO . QUESTIONS ET FILTRES CODES PASSE R À 1 Etes-vous actuellement enceinte ? OUI ....................................................... 1 NON ..................................................... 2 PAS SÛRE ........................................... 8 Î5 2 Voulez-vous un autre enfant ? OUI ....................................................... 1 NON ..................................................... 2 NE SAIT PAS ....................................... 8 Î4 Î4 3 Quand souhaiteriez vous avoir votre prochain enfant ? AU COURS DES 2 PROCHAINES ANNÉES........................................... 1 DANS PLUS DE 2 ANS........................ 2 PAS SÛRE QUAND ............................. 8 Î5 4 Actuellement, faites-vous quelque chose ou utilisez-vous une méthode pour retarder ou éviter de tomber enceinte ? SI NON, ENCERCLEZ >01' [PAS DE MÉTHODE] SI OUI, DEMANDEZ : « Quelle est la principale méthode que vous ou votre mari/partenaire utilisez pour retarder/éviter de tomber enceinte ? » ENCERCLEZ LE CODE APPROPRIÉ. PAS DE MÉTHODE ........................... 01 NORPLANT........................................ 02 INJECTIONS ...................................... 03 PILULE ............................................... 04 DIU ..................................................... 05 MÉTHODE DE BARRIÈRE/ DIAPHRAGME ............................... 06 CONDOM ........................................... 07 MOUSSE/GELÉE............................... 08 LIGATURE DES TROMPES .............. 09 VASECTOMIE.................................... 10 ALLAITE. MATERN. AMÉNORRHÉE (ALLAIT. EXCLUSIF) ..................... 11 RYTHME ............................................ 12 ABSTINENCE .................................... 13 RETRAIT ............................................ 14 AUTRE________________________ 96 (PRÉCISER) 5 Avez-vous entendu parler d’une maladie appelée sida ? OUI ..........................................................1 NON.........................................................2 ÎFIN 6 Que peut-on faire pour éviter d’avoir le sida ? Autre chose ? ENREGISTREZ TOUT CE QUI EST MENTIONNÉ. S'ABSTENIR DE RAPPORTS SEX. ....A UTILISER DES CONDOMS .................B LIMITER RAPPORTS SEX. À UN PARTENAIRE /RESTER FIDÉLE À UN SEUL PARTENAIRE .......................C LIMITER LE NOMBRE DE PARTE-NAIRES SEXUELS ........................................D ÉVITER RAP. SEX. PROSTITUÉES....E ÉVITER RAP. SEX. AVEC PERSONNES AYANT BEAUCOUP PARTENAIRESF ÉVITER RAP. SEX. AVEC PERSONNES DU MÊME SEXE ....G 145 NO . QUESTIONS ET FILTRES CODES PASSE R À ÉVITER RAP. SEX. AVEC PERSONNES QUI SE FONT DES INJECTIONS INTRAVEINEUSES DE DROGUES H ÉVITER TRANSFUSIONS SANG ......... I ÉVITER INJECTIONS ...........................J ÉVITER D'EMBRASSER......................K ÉVITER PIQÛRES MOUSTIQUES ...... L CHERCHER PROTECTION DES GUÉRISSEURS TRADITIONNELS M ÉVITER PARTAGER RASOIRS/LAMESN AUTRE ............................................... W (PRÉCISER) AUTRE ............................................... X (PRÉCISER) NE SAIT PAS ....................................... Z Les questions méritant une attention particulière pour l’over sampling (au moins 7 réponses par grappe) : • Q4 : avec une réponse V.6.3. Questionnaire #3. A poser aux mères d’enfant de 12 à 23 mois (Pour la vaccination, l’alimentation complémentaire et la continuation de l’allaitement au delà de 12 mois) No QUESTIONS ET FILTRES CODES PASSER À 1 Avez-vous un carnet où les vaccinations (NOM) sont inscrites ?2 SI OUI : Puis-je le voir ? OUI, VU PAR ENQUÊTEUR.....................1 PAS DISPONIBLE/PERDU/ÉGARÉ.........2 JAMAIS EU DE CARNET.........................3 NE SAIT PAS............................................8 Î 3 Î 3 Î 3 (1) COPIER LES DATES DE VACCINATION POUR CHAQUE VACCIN À PARTIR DU CARNET.2 NOTER >44' DANS LA COLONNE >JOUR= SI LE CARNET INDIQUE QU’UN VACCIN A ÉTÉ DONNÉ, MAIS QUE LA DATE N’EST PAS REPORTÉE. JOUR MOIS ANNÉE BCG BCG........ POLIO 0 (POLIO À LA NAISSANCE) P0 ........... POLIO 1 P1 ........... POLIO 2 P2 ........... POLIO 3 P3 ........... DTCoq 1 DTC 1 ..... DTCoq 2 DTC 2 ..... DTCoq 3 DTC 3 ..... ROUGEOLE ROUG..... VITAMINE A (LA PLUS RÉCENTE) VIT. A .... 2 12 Avez vous allaiter (NOM) auparavant ? OUI ……………………………………….1 NON………………………………………2 NE SAIT PAS……………………………8 Î14 Î14 146 No QUESTIONS ET FILTRES CODES PASSER À 13 Actuellement est-ce que vous allaitez (NOM)? OUI........................................................... 1 NON ......................................................... 2 14 Combien de fois (NOM) a t-il mangé un aliment semi-solide (broyés ou en purée) au cours de la journée ou de la nuit d’hier ? SI 7 FOIS OU PLUS, ENREGISTREZ 7. ┌───┐ NOMBRE DE FOIS ........................ │░░░│ └───┘ NE SAIT PAS ........................................... 8 Les questions méritant une attention particulière pour l’over sampling (au moins 7 réponses par grappe) : • Q1 : Réponse 1 • Q12 : Réponse 1 V.6.4. Questionnaire #4. A poser aux mères d’enfant de 0 à 6 mois (Pour l’évaluation de l’allaitement maternel exclusif) No QUESTIONS ET FILTRES CODES PASSER À A B C D E F J’aimerais vous poser des questions sur les types de liquides que (NOM) a bu au cours de la journée et de la nuit. Est-ce que (NOM) a bu les liquides suivants hier au cours de la journée ou de la nuit? COCHEZ DANS LA CASE SI L’ENFANT A BU LE LIQUIDE EN QUESTION. Lait maternel ? Eau Plate ? Lait en poudre vendu en commerce ? Tout autre lait tel qu’en boîte, en poudre, ou du lait frais d’animal ? Jus de fruit ? Autres liquides tels que de l’eau sucrée, du thé, du café, des boissons gazeuses ou du bouillon ? ┌───┐ A..................................................... │░░░│ └───┘ ┌───┐ B..................................................... │░░░│ └───┘ ┌───┐ C..................................................... │░░░│ └───┘ ┌───┐ D..................................................... │░░░│ └───┘ ┌───┐ E..................................................... │░░░│ └───┘ ┌───┐ F ..................................................... │░░░│ └───┘ G H I J K L Maintenant je voudrais vous poser des questions sur les types d’aliments4 que [NOM] a mangés hier au cours de la journée ou de la nuit. Est-ce que [NOM] a mangé les aliments suivants au cours de la journée ou de la nuit d’hier ? COCHEZ LA CASE SI L’ENFANT A MANGÉ L’ALIMENT EN QUESTION. Des aliments à base de graines [par exemple, mil, sorgho, maïs, riz, blé, bouillie, ou d’autres graines locales] ? Citrouille, ignames rouges ou jaunes ou de la courge, des carottes ou des patates douces rouges ? Tout autre aliment à base de racines ou de tubercules [par exemple, les pommes de terre, les ignames blanches, le manioc ou d’autres racines/tubercules locaux] ?5 Des légumes à feuilles vertes ? Mangue, papaye [ou d’autres fruits locaux riches en Vitamine A] ? Autres fruits et légumes [exemple, les bananes, pommes/compote, avocats, tomates] ? ┌───┐ G..................................................... │░░░│ └───┘ ┌───┐ H..................................................... │░░░│ └───┘ ┌───┐ I ...................................................... │░░░│ └───┘ ┌───┐ J...................................................... │░░░│ └───┘ ┌───┐ K..................................................... │░░░│ └───┘ ┌───┐ L ..................................................... │░░░│ └───┘ ┌───┐ M .................................................... │░░░│ 147 No QUESTIONS ET FILTRES CODES PASSER À M N O P Viande, volaille, poisson, fruits de mer ou des œufs ? Aliments préparés à partir de légumes [exemple, les lentilles, haricots, soja, légumes à gousses, ou arachides] ? Fromage ou yaourt ? Aliments à base d’huile, graisse ou de beurre ? └───┘ ┌───┐ N..................................................... │░░░│ └───┘ ┌───┐ O..................................................... │░░░│ └───┘ ┌───┐ P..................................................... │░░░│ └───┘ Les questions méritant une attention particulière pour l’over sampling (au moins 7 réponses par grappe) : • Cette question doit être posée à au moins 7 mères d’enfant de 0-4 mois par grappe V.6.5. Questionnaire #5. A poser aux mères d’enfant de 6 à 9 mois (Pour l’évaluation de l’introduction de l’alimentation complémentaire) No QUESTIONS ET FILTRES CODES PASSER À G H I J K L M N O P Je voudrais vous poser des questions sur les types d’aliments4 que [NOM] a mangés hier au cours de la journée ou de la nuit. Est-ce que [NOM] a mangé les aliments suivants au cours de la journée ou de la nuit d’hier ? COCHEZ LA CASE SI L’ENFANT A MANGÉ L’ALIMENT EN QUESTION. Des aliments à base de graines [par exemple, mil, sorgho, maïs, riz, blé, bouillie, ou d’autres graines locales] ? Citrouille, ignames rouges ou jaunes ou de la courge, des carottes ou des patates douces rouges ? Tout autre aliment à base de racines ou de tubercules [par exemple, les pommes de terre, les ignames blanches, le manioc ou d’autres racines/tubercules locaux] ?5 Des légumes à feuilles vertes ? Mangue, papaye [ou d’autres fruits locaux riches en Vitamine A] ? Autres fruits et légumes [exemple, les bananes, pommes/compote, avocats, tomates] ? Viande, volaille, poisson, fruits de mer ou des œufs ? Aliments préparés à partir de légumes [exemple, les lentilles, haricots, soja, légumes à gousses, ou arachides] ? Fromage ou yaourt ? Aliments à base d’huile, graisse ou de beurre ? ┌───┐ G..................................................... │░░░│ └───┘ ┌───┐ H..................................................... │░░░│ └───┘ ┌───┐ I ...................................................... │░░░│ └───┘ ┌───┐ J...................................................... │░░░│ └───┘ ┌───┐ K..................................................... │░░░│ └───┘ ┌───┐ L ..................................................... │░░░│ └───┘ ┌───┐ M .................................................... │░░░│ └───┘ ┌───┐ N..................................................... │░░░│ └───┘ ┌───┐ O..................................................... │░░░│ └───┘ ┌───┐ P..................................................... │░░░│ └───┘ 148 Annex 2: TP Child Survival Indicators by stratum Pneumonia Case Management Indicators Toliara II Betioky 1. % of children 0-23 months with IRA in the two weeks preceding the survey 204/300= 68% 194/300=64% 2. % of mothers who sought treatment when their child had rapid or difficult breathing in the two weeks preceding the survey. 65/117= 56% 70/161= 44% 3. % of mothers who know at least two signs of danger for ARI which necessitate seeking treatment 189/300=64% 138/300=46% 4. % of mothers who seek treatment from health centers when their child has an ARI 99/204=49% 100/226=44% 5. % of mothers who seek treatment for their child within the 48 hours of the first sign of ARI. 73/204=36% 72/226=31% 6. % of children who had ARI in the two weeks preceding the survey and who received more fluids than usual during the illness 77/204=38% 116/226=51% 7. % of children who had ARI in the two weeks preceding the survey and who received the same quantity or more breast milk than usual during the illness 144/204=70% 185/226=82% 8. % of children who had ARI in the two weeks preceding the survey and who received the same quantity or more food than usual during the illness 117/204=57% 164/226=73% Diarrheal Disease Indicators Toliara II Betioky 1. % of children 0-23 months with diarrhea in the two weeks preceding the survey. 117/300=39% 141/300=47% 2. * % children 0-23 months with diarrhea in the past two weeks who were given more than usual or the same amount of breast milk during the episode. 137/209=66% 183/203=90% 3. % children 0-23 months who had diarrhea in the past two weeks, who were given more than the usual amount of fluids during the episode. 109/209=52% 132/203=76% 4. % children 0-23 months who had diarrhea in the past two weeks, who were given same or more than the usual amount of food during the episode. 113/209=54% 154/203=76% 5. % of children healed from diarrhea who received more food than usual in the week following the illness. 43/100=43% 68/141=48% 6. * % mothers of children 0-23 months who can cite at least 2 danger signs of diarrhea as a reason to seek advice or treatment at the health facility. 193/300=64% 197/300=66% 7. % mothers who sought treatment at health center in the 24 hours of the first danger sign for diarrhea. 80/108=74% 33/63=52% 8. % of mothers of children 0-23 months who report washing their hands with soap/detergent/ash before preparing food, before feeding the children, after having gone to the bathroom and after having changed a child who has defecated. 16/300=5% 18/300=6% 9. % mothers who sought treatment at a health center in the 48 hours of the first sign of diarrhea. 122/209=58% 71/203=37% 10. % mothers who sought treatment at a health center 144/209=69% 74/203=37% 150 Malaria Indicators Toliara II Betioky 1. % of children 0-23 months with fever/ malaria in the two weeks preceding the survey. 147/300=49% 150/300=50% 2. % of children 0-23 months who slept under an insecticide treated bed net the night preceding the survey. 174/300=58% 130/300=43% 3. % of mothers who received a malarial prophylaxis during the pregnancy of their last child 209/300=72% 149/300=50% 4. % of children with fever who received household treatment appropriate for malarial syndrome (denominator being revised) 24/67=34% 63/206=31% 5. % of homes that own at least one mosquito net 225/300=74% 173/300=58% 6. % of mothers who sought treatment within the 48 hours of the first sign of malaria 104/204=51% 78/206=38% 7. % of mothers who sought treatment at the health center at the end of the day of the first sign of malaria. 37/204=18% 17/206=8% 8. % of children who received more fluids than usual during a febrile illness. 91/204=43% 165/206=80% IMCI Indicators Toliara II Betioky 1. % of mothers of children 0-23 months who know at least two signs of danger of childhood illnesses which necessitate treatment. 230/300=77% 220/300=73% Vaccination Indicators Toliara II Betioky 1. % children 12-23 mo that own an immunization card 170/300=57% 200/300=67% 2. % children 12-23 mo that own an immunization certificate (diploma) 35/300=12% 10/300=3% 3. % of children 12-23 months who are fully immunized (against the five vaccine preventable illnesses) per vaccination card 213/300=12% 137/300=46% 4. % of mothers who have received 2 or more doses of Tetanus Toxoid vaccine before the birth of their youngest child per vaccination card Recall =125/300 Card=88/300 Recall =120/300 Card=113/300 5. % of children 12-23 months who defaulted between DPT1 and DPT3 doses 15/155=10% 50/215=23% 6. *% of mothers who know the benefits of being vaccinated against tetanus during pregnancy 114/300=38% 169/300=56% 7. * % of mothers who know at least three benefits of vaccinating children 35/300=12% 34/300=11% 8. * % of mothers who can name 1+ benefits of vaccinating children 90/300=30% 155/300=52% 151 Breastfeeding and nutrition Indicators Toliara II Betioky 1. % of mothers of children 0-6 who received breast milk exclusively during the previous 24 hours. 118/300=41% 127/300=42% 2. % of mothers who initiate breastfeeding within an hour after giving birth. 163/300=54% 160/300=53% 3. % of children 0-23 months who are underweight (-2 standard deviation of the weight for age median, according to WHO/NCHS population reference). 59/231=18% 45/215=21% 4. * % of children 12-23 months who receive 5 of more meals per day (meals and snacks) in addition to breast milk. 147/300=48% 227/285=80% 5. % of children 0-23 months who received colostrum. 233/300=78% 238/300=79% 6. % of children 6-9 months who received breast milk and complementary foods during the past 24 hours. 300/300=100% 296/296=100% 7. % of mothers who know at least three benefits of colostrum 88/300=28% 14/300=5% 8. * % mothers who know at least three benefits of exclusive breastfeeding 30/300=10% 36/300=12% 9. * % mothers who know one benefit of exclusive breastfeeding 195/300=65% 293/300=98% 10. * % mothers who know two benefit of exclusive breastfeeding 70/300=23% 109/300=36% 11. % of children 6 to 23 months who received a dose of vitamin A during the last 6 months, according to their health card. Recall=237/300 card: 64/300 Recall=229/300 card: 78/300 12. % of mothers who received iron folic acid during the last pregnancy Recall: 174/300=59% Recall : 141/300=47% Child Spacing and Reproductive health Indicators Toliara II Betioky 1. % of mothers who are not pregnant, do not want another child in the next two years or are not sure, and are using a modern method of contraception 40/166=24% 53/188=28% 2. % of mothers who know that exclusive breastfeeding is a means for birth spacing. 11/300=4% 61/300=3% 3. % of mothers of 0-23 months who can cite 02 ways to reduce the risk of HIV infection 62/300= 21% 109/300=36% 4. % of children 0-23 months who were born at least 24 months after the preceding living child. 126/145=87% 95/169=56% 5. % of mothers who know at least three benefits of birth spacing 25/300=8% 29/299=10% 6. % of mothers who know the benefits of birth spacing 108/300=36% 297/299=99% Prenatal Care Indicators Toliara II Betioky 1. % des mères qui possèdent un carnet de CPN 157/300= 52% 151/300=50% 152 2. % of mothers who had at least two prenatal visits during their last pregnancy, by maternal health card Recall: 218/300 card: 122/300 Recall : 213/300 card : 127/300 3. % of children 0-23 months whose birth was assisted by a health professional/ trained health worker 123/300=41% 101/300=34% 4. % of women whose last child was born in a health center 128/300=43% 79/300=26% Annex 3: Clusters Clusters in Toliara II CLUSTER # COMMUNE FOKONTANY 1 Ankilibory 2 Beheloka Vatolalaky 3 Soalara Sud Soalara Bas 4 St Augustin 5 St Augustin Sarodrano 6 Ankaray Sud 7 Analamisampy Ampasikibo 8 Ankorodamoty 9 Antseva 10 Ankililoaka Tanambao Am/sy 11 Ankililoaka Ankililoaka 1 12 Mileneky 13 Milenaky Ankarabato 14 Tsianisiha 15 Tsianisiha Beravy Haut 16 Marofoty Beroroha 17 Saririaka 18 Ankilimaliniky Sakabera sikily 19 Madiorano 20 Manombo Fiherene masay 21 Andranovory 22 Andranovory Anjabaky 23 Ambohimahavelo Ambohimahavelo 24 Ambolofoty Manatsofy 25 Manorofify Andranovaky 26 Belalanda Bekoaky 27 Maromiandra Antsary 28 Miary Agnolaky 29 Behompy Maroala 30 Andranohinaly Befoly 154 Clusters in . CLUSTER # COMMUNE FOKONTANY 1 Ambatry Ambatry Mitsinjo 2 Ambatokapike Nord 3 Masiaboay Andamilamy I 4 Maroarivo Maroaraivo Ankazomb 5 Ankazomanga Ouest Bevala 6 Betioky-Sud Antsakoamasy 7 Beantake Ampasindava Ouest 8 Antohabato Ankazoabokely 9 Beavoha Mahazoarivo 10 Tameantsoa Laniry 11 Andranomangatsiaka Bevaro Bas 12 Ankilivalo Bevaro Haut 13 Manalobe Ankiliabo 14 Bezaha Iii 15 Bezaha Saloavaratsy 16 Tameantsoa Eboro 17 Ihotry 18 Tongobory Tamia 19 Vatolatsaka 20 Vatolatsaka Mavozaza 21 Lazarivo 22 Andranomavo 23 Lazarivo Vohitsevo I 24 Manintsy I 25 Marosavoa Ankotika 26 Salobe Salobe 27 Savazy I 28 Behisatry 29 Belamoty Mihaiky 30 Tanambao Haut Andromasy I 155