1 Plan USA, INC. D/b/a CHILDREACH Plan CAMEROON : CHILD SURVIVAL XVI PROJECT Cooperative Agreement No: FAO-A-00-00-00035-00 FINAL EVALUATION REPORT - YEAR 4 ® IMPLEMENTING AGENCY: Plan Cameroon In partnership with Ministry of Health Cameroon AAPPEC, FESADE Catholic and Presbyterian Mission Health Services ® LOCATION: Cameroon 3 Health Districts: Bertoua, Doumé and Nguelemendouka in the East Province ® CONTACT PERSON: Mr. Sam Worthington National Executive Director, Childreach Plan USA INC. 155, Plan Way, Warwick, Rhode Island 02886 ® Project Beginning Date: September 30th 2000 Ending Date: September 29, 2004 Prepared by: Della Dash, Consultant Reviewed and Edited by: Pierre-Marie Metangmo, Laban Tsuma and Plan Cameroon Team September, 2004 2 LIST OF ACRONYMS AAPPEC Association for the Self – Promotion of the East Province Population ACHRP Association of Community Health Resource Persons AIDS Acquired Immunodeficiency Syndrome BCC Behavior Change Communication CAPP Center for supply of drugs and health materials CBO Community Based Organizations CHW Community Health Worker COGE Health Area Management Committee COSA Health Area Committee members CS Child Survival CS XVI Child Survival XVI DHC District Health Committee DHMT District Health Management Teams DMO District Medical Officer DIP Detailed Implementation Plan DS Development Structure EPI Expanded Program of Immunization FESADE Women’s Health Development in Sub-Saharan Africa FGD Focus Group Discussion GAVI Global Alliance for Vaccines and Immunization HIPCI Highly Indebted and Poor Countries Initiative HCN Health Center Nurse HFA Health Facility Assessment HKI Helen Keller International HIS Health Information System HIV Human Immunodeficiency Virus ICC Nurse in charge of Health Centers IEC Information, Education, and Communication IHC Integrated Health Centers IMCI Integrated Management of Childhood Illness ITN Insecticide Treated Net KPC Knowledge, Practice, and Coverage Survey LQAS Lot Quality Assurance Sampling MOH Ministry of Health MTCT Mother-To-Child Transmission MTE Mid-Term Evaluation NGO Non Governmental Organization NID National Immunization Day ORS Oral Rehydration Salts ORT Oral Rehydration Therapy OVC Orphans and Vulnerable Children PD Positive Deviance PLA Participatory Learning and Action PSI Population Services International 3 PVO Private Voluntary Organization SSS Salt and sugar solution TBA Traditional Birth Attendant TOT Trainer of Trainers TT Tetanus Toxoid U5 Under five years USAID United States Agency for International Development VHC Village Health Committee WCBA Women of Child Bearing Age 4 TABLE OF CONTENTS Page A. EXECUTIVE SUMMARY............................................................................................5 B. ASSESSMENT OF RESULTS AND IMPACT OF THE PROGRAM ........................7 1. RESULTS: SUMMARY CHART...........................................................................8 2. RESULTS: TECHNICAL APPROACH...............................................................17 3. RESULTS: CROSS-CUTTING APPROACHES………………………………..35 a. Community Mobilization………………………………………………….…35 b. Communication for Behavior Change………………………………………..37 c. Capacity Building Approach…………………………………………………39 d. Sustainability Strategy……………………………………………………….48 C. PROGRAM MANAGEMENT ...................................................................................50 1. PLANING…………….. ......................................................................50 2. STAFF TRAINING .............................................................................54 3. SUPERVISION OF PROGRAM STAFF............................................56 4. HUMAN RESOURCES AND STAFF MANAGEMENT..................57 5. FINANCIAL MANAGEMENT..........................................................59 6. LOGISTICS……………. ....................................................................62 7. INFORMATION MANAGEMENT....................................................63 8. TECHNICAL AND ADMINISTRATIVE SUPPORT .......................65 9. MANAGEMENT LESSONS LEARNED...........................................66 D. OTHER ISSUES IDENTIFIED BY TEAM................................................................67 E. CONCLUSIONS AND RECOMMENDATIONS.......................................................68 F. RESULTS HIGHLIGHT ..............................................................................................79 ATTACHMENTS ..............................................................................................................80 Plan Cameroon Child Survival Project Final Evaluation Report August 23-September 1, 2004 5 A. Executive Summary The goal of the Plan CS XVI project was to create a sustained reduction in infant and child mortality through increased access to community-based health care and improved quality of health services in the project area. The project supported the MOH community outreach policy to empower communities to improve local health resources and increase access to health services. This project was located in three Health Districts of the western part of Cameroon’s heavily forested Eastern Province. The province is Cameroon’s poorest, and Plan has been working there since 1996. The project benefited inhabitants of 267 villages (a total population of 211,2641 ) located in 27 Health Areas of Bertoua, Doume and Nguelemendouka health districts. The project targeted approximately 95,024 beneficiaries with 38,009 children under five, 8,447 infants 0 to 11 months and 48,568 women of reproductive age (15 to 49). Sustainability has been ensured by strengthening the capacity of community resource persons (Health Area Committee Representatives, TBAs), community-based organizations (women’s groups), and the MOH staff (HCNs, community pharmacy workers) to provide quality services in the project area. Project interventions were strengthened using BCC activities designed to raise awareness about health issues and promote health-seeking behaviors. At the same time that communities were being empowered to take charge of their health needs, the project worked through its MOH partnership to strengthen the services of the health facilities by providing needed supplies and equipment and re-training staff. In addition, the project set up systems to assure delivery of outreach services to affiliated communities. It also worked to improve the functioning of the Health Area Committees. The project goal was achieved through the successful implementation of a pilot project using the IMCI approach, which improved local access to preventive and curative services through capacity building interventions that targeted households, communities and health facilities. As this was Plan Cameroon’s first CS grant, they had a tremendous learning curve to overcome, which they successfully demonstrated with significant results and tremendous potential for nation-wide scale-up. Plan achieved or exceed nearly every project indicator during the life of this project. This project has tremendous potential for scale up to the entire country. Plan piloted IMCI in one province in Cameroon as part of a national pilot program, with the intention of using the experiences from the pilot for national scale up. Plan is now in the position of assisting the Government of Cameroon in expanding IMCI to the entire country based on the lessons learned from this pilot project. 1 2000 MOH Eastern Health Delegation Report. Plan Cameroon Child Survival Project Final Evaluation Report August 23-September 1, 2004 6 Significant results were demonstrated using the IMCI approach, as shown in the table below. IMCI Indicator Baseline Final Trained Final Untrained The number of children 0-23 months seen at health facilities that were assessed for all danger signs 10.5% 72% 42.3% The number of caretakers who were correctly counseled about their sick child 36.3% 97% 65.4% The number of caretakers who were given oral medications, who know how to correctly administer the treatment 44.8% 84.6% 34.6% Conclusions from the final evaluation include the need for an MOH policy to address staff turnover which are adversely affecting IMCI implementation by resulting in a loss of trained health facility staff. Also raised was the need for a participatory process with all involved project partners, as well as within the CS project management. Lastly, the need for much closer supervision at all levels became a major theme during the final evaluation, and all project partners agreed that supportive supervision needed to be greatly increased for IMCI scale-up throughout the country. The Plan project staff is creative, hardworking, very loyal, and smart. In spite of any constraints experienced, they all worked long hours in a difficult environment, and managed to demonstrate very good results. They were also a wonderful team to work with throughout the final evaluation. I applaud each and every one of them! Plan Cameroon Child Survival Project Final Evaluation Report August 23-September 1, 2004 7 B. Assessment of Results and Impact of the Program The final evaluation was conducted using a participatory approach, and the information and recommendations that are detailed in this report are a result of the KPC, HFA, FGD, home visits, the project monitoring system, and data reviews. The goal of this project is the sustained reduction in infant and child mortality by fostering a partnership between communities and health facilities to address the major causes of morbidity and mortality in children less than 5 years of age. Plan Cameroon successfully piloted IMCI in one district of Cameroon, leading the effort to introduce IMCI to the country, with the intention by Plan and the MOH (and UNICEF, WHO, HKI, PSI and others) to scale up IMCI throughout the entire country based on the results of this project (as well as the other two pilot districts carried out by UNICEF and WHO). Plan intends to submit another CS proposal, this time under the expanded impact category to scale up IMCI in four provinces in the country as part of a nation-wide scale up campaign being co-funded by several donors. This report is written with the intention of providing a road map for scale up of IMCI in Cameroon. The successes realized during this project will be capitalized on, while the weaknesses and constraints encountered are being thoroughly discussed and analyzed using a transparent approach. Recommendations have been drafted for all areas of the project with a focus on improving problematic issues for successful scale up. Everyone who participated in this project, and all those involved in the final evaluation collectively felt that the project had gotten off to a slow start. The first year and a half were taken up by preparing the foundation for the project, including hiring staff, positioning supplies and logistics, establishing community structures, and training Plan staff, AAPPEC staff and health promoters, health facility and other MOH staff, and all the community structures that would become the backbone for the implementation of the project. For this reason, everyone felt that the project had only really been producing results for a short period of time (barely over two years), and that in reality the project should continue for another two years in order to really demonstrate impact. They thought that the gains in the first two years (baseline to midterm) would not be as great as the gains in the final two years (midterm to final). However, a review of the data shows that in spite of the time taken for the project to establish itself, tremendous gains were achieved throughout the life of the project, and while there is still room for continued growth, everyone was surprised at how far they had actually come. They reported that when the project first started, villages had a huge lack of health information and general management. They were disorganized, and lacked the skills and knowledge necessary to make decisions, to plan and to follow up on their own development. By the end of the project, most villages were becoming well organized with at least one functioning community structure that was actively engaged in educating the Plan Cameroon Child Survival Project Final Evaluation Report August 23-September 1, 2004 8 community, constructing water and sanitation schemes, and changing the behavior of mothers and other caretakers with regards to maternal and child health. Likewise, they stated that when the project started, the health centers were a mess, there was garbage everywhere, and the staff didn’t come to work very often. Due to the training and equipping of the health center staff, coupled with their enhanced collaboration with health committee and CBO members and TBAs trained in the project, the health centers are now cleaner and the staff comes to work regularly, and there is increased utilization of health centers by the population. The dialogue structure is really working well now in most places, and the number of outreach stations and activities has increased more than 5-fold since 2000. The project staff witnessed a dramatic shift in the thinking and behavior of mothers /community members and health facility staff as they became more open to participation and learning. This motivated everyone to work even harder to amplify the incredible changes they knew the project was facilitating in an inhospitable environment plagued with poverty and disease, and in the face of tremendous hardship. They stated that the project has been very challenging, but also very rewarding! 1. Results: Summary Charts The following tables provide specific project objectives, indicators, method of measurement, end of project target, and baseline, midterm and final evaluation values. Tables are included for all project intervention areas including malaria, diarrhea, malnutrition, immunization, IMCI, and capacity building and sustainability for both the facility and community levels. Plan Cameroon Child Survival Project Final Evaluation Report August 23-September 1, 2004 9 IMCI goal: To improve the quality of care of children both in health facilities and at home by ensuring (1) that health facility personnel and caretakers follow a holistic approach to treatment of illnesses, (2) that caretakers understand proper home care for illnesses and (3) that there is good communication between health facility personnel and caretakers. Objectives Measuremen t Method End of Project Target Baseline value End Point Value Level of achievement Increase from 58.5% to 80% the # of mothers who know at least 2 danger signs that children need to be seen at a health facility KPC 80% 58.5 % 65.0 % 81 % Increase from 11% to 50% the number of children, 0- 23 months, seen at health facilities that were assessed for all danger signs. HFA (Observation) 50% 10.5 % 33.9 % 68 % Increase from 36% to 70% the # of caretakers who were correctly counseled about their sick child HFA 70% 36.3 % 69.4 % 99.1 % Increase from 45% to 70% the # of caretakers who were given oral medications, who know how to correctly administer the treatment HFA (exit interview with caretaker) 70% 44.8 % 61.7 % 88.1% Increase from 2 to 6 per year the number of supervision visits to the IHC IHFA Project monitoring system 57.1 82.6 % Plan Cameroon Child Survival Project Final Evaluation Report August 23-September 1, 2004 10 Malaria Goal: To reduce malaria associated mortality and morbidity in children and pregnant women through improved malaria case management, malaria treatment and antenatal chemoprophylaxis as well as the promotion of the use of bed nets and their re-treatment among the project population. Objective Measurement Method End of project Target Base line Value End Point Value Level of achievement Increase from 6% to 20% the # of households that have an ITN KPC 20% 6 % 33.6 % 168% Increase from 4% to 20% the # of children, 0-23 months, and their mothers who sleep under an ITN KPC 20% 0.4 % 33.6 % 168% Increase from 28% to 50% of mothers of children 0- 23 months who received iron/folic acid supplementation during the last pregnancy KPC 50% 28.4 % 51.4 % 103% All women’s groups trained on home management of malaria Project monitoring system 100% 0 100% (120 CBOs) 100% All health facility staff trained in MCM Project monitoring system 100% 0 46 personnel, =100% 100% Plan Cameroon Child Survival Project Final Evaluation Report August 23-September 1, 2004 11 Diarrhea Goal: To reduce diarrhea associated mortality and morbidity through prompt and appropriate case management of (a) all episodes at home with fluid and dietary management and (b) recognition of danger signs with appropriate referral. Objectives Measurement Method End of Project Target Base line Value End Point Value Level of achievement Increase from 42.3% to 70% the number of children who received more fluids than usual during the last diarrhea episode KPC 70% 42.3 % 56.2 80% Increase from 62.8 % to 88.4% the # children who were breastfed the same or more during the last diarrhea episode KPC 88.4% 62.8 % 65.7 % 74% Increase from 51% to 70% the # of children who were given the same or more food during the last diarrhea episode KPC 70% 51.3 % 76.6 % 109% Increase from 23% to 70% the # children with simple diarrhea, seen in health facilities, that were correctly treated HFA (Observation) 70% 23 % 66.7 % 95 % Hand washing with soap or ash by mothers of children 0-23 months before food preparation, feeding the child, on return for defecating and after changing the nappies of a child who has defecated KPC 2.6 % 20.6 % Hand washing with soap or ash by mothers of children 0-23 months before food preparation for the child KPC 42.2 % 49.4 % Hand washing with soap or ash by mothers of children 0-23 months before feeding the child KPC 35.4 % 62.5 % Hand washing with soap or ash by mothers of children 0-23 months after defecating KPC 31.1 % 36.1 % Hand washing with soap or ash by mothers of children 0-23 months after cleaning a child who has defecated KPC 18.7 % 51.1 % Plan Cameroon Child Survival Project Final Evaluation Report August 23-September 1, 2004 12 Immunization goal: To increase immunization coverage in the program area for all infants by the end of their first year of life, and provide tetanus toxoid (second dose) immunization for pregnant women. Objectives Measurement Method End of Project Target Baseline value End Point Value Level of achievement Increase from 24.7% to 60% the # children 12-23 months who were fully immunized before their first birthday KPC 60% 24.7 % 54.9 % 91.5 % Reduce the DPT dropout rate from 37.5% to 15% KPC 15% 37.5 % 13.1 % 115% Increase from 60.7% to 80% the # of children, 12-23 months, vaccinated with measles KPC 80% 60.7 % 62.1 % 78 % Increase from 47% to 70% the # of mothers of children 0-23 months who had TT2 during their last pregnancy KPC 70% 46.5 % 55.3 % 79% Increase from 9.4% to 40% the # of children who received Vitamin A supplementation during the last 6 months (This objective will be valid if the MOH implements regular Vitamin A supplementation in the East Province) KPC 40% 9.4 % 26.0 % 65% Increase from 30% to 80% the # of children seen in health facilities that had immunization status checked HFA (Observation) 80% 29.5 % 78.7 % 98.4 % Plan Cameroon Child Survival Project Final Evaluation Report August 23-September 1, 2004 13 Malnutrition goal: To reduce malnutrition associated morbidity and mortality through monthly growth monitoring and follow-up, household feeding programs, nutrition education, breastfeeding promotion, and vitamin A supplementation. Objectives Measurement Method End of Project Target Base line Value End Point Value Level of achievement Increase from 11% to 40% the # of children, 0-23 months, who attended a growth monitoring session during the last 4 months and whose weight was plotted on the road to health growth monitoring card KPC 40% 11 % 44.4 % 111% Decrease from 19.1% to 10% the # of moderate and severely malnourished children KPC 10% 19.1 % 16.4 % 70% Increase from 0% to 40% # of mothers of children (0- 23 months) with mild to moderate malnourished children participate in community based rehabilitation program Project monitoring system 40% 0 62% 155% Increase from 3% to 50% the # of children seen at health facilities who had nutrition status plotted on the road to health chart of the child health card. HFA (Observation) 50% 3.2 % 54.3% 109% Increase from 29% to 35% the # of mothers of children 0-6 months of age who exclusively breastfed the child KPC 35% 29 % 56.9 % 163% Increase from 66% to 80% the # of mothers of children 6-9 months who are giving the child complementary food KPC 80% 65..8 % 76.4% 95.5% Plan Cameroon Child Survival Project Final Evaluation Report August 23-September 1, 2004 14 Capacity Building and Sustainability Goal: To strengthen both the MOH and community health organizations so that they can work together to improve the health situation of children and families. Objectives Indicators Measurement Method End of Project Target Baseline value End Point Value Level of achievement MOH: Improved personal mastery and change of mental models of facility staff through acquiring IMCI skills. 1. % of cases of children 0-23 months who were seen at health facilities who were assessed for all danger signs. 2. % of cases seen at health facilities where the health facility staff completed correctly all the counseling tasks 3. % of cases seen at health facilities where caretakers who were prescribed oral medications for their sick child, knew how to correctly administer treatment - HFA - Interviews FGD 50% 70% 70% 10.5 % 36.3 % 44.8 33.9 % 69.4 % 61.7 % 67.8 % 99.1 % 88.1 % 1. New supervisory instruments are developed that include a section for joint problem solving. 2. Description of problems solved together during supervisory visits. 3. Description of problems solved jointly between health facility staff and communities through discussions at health area committee meetings. - Project monitoring system - Interviews FGD 0% 100% 4. % of IHCs who receive regular supervisory visits by DHT. 0% 32.1% Improved team learning and systems thinking through adjustments made to the supervisory system in order to emphasize supportive supervision and joint problem solving and through regular meetings between health facility staff and community representatives. 5. % of VHCs and community resource persons who receive regular supervisory visits from IHC. 0% 15% Plan Cameroon Child Survival Project Final Evaluation Report August 23-September 1, 2004 15 Communities: 1. % of COSA that have a written vision statement. - Project monitoring system Health area committees 100% 100% (COSA) develop clear visions about changes in the health situation that they want to see for each health area and about the role of the COGE. ACHRP develop clear visions about their role in improving health in communities. 2. % of ACHRP that can articulate a vision statement. - Interviews FGD 100% 100% 1. % of health area committees with action plans. - Project monitoring system 20% 100%* Health area committees develop abilities for systems thinking though making action plans to improve the health situation in each health area. ACHRP develop system thinking abilities directed toward improving health at the local level. 2. % of ACHRP with action plans. - Interviews FGD 0 75% Health area committees change mental models for handling health issues through combined work of the different villages in each health area. % of health areas with joint village plans for addressing health issues. - Project monitoring system - Interviews FGD 0 100%* Plan Cameroon Child Survival Project Final Evaluation Report August 23-September 1, 2004 16 1. % COGE trained in financial record keeping. - Project monitoring system 50% 100% Health area committee members and community health resource persons improve personal mastery of managing the community health system. 2. % of COGE who have participated in management training. 3. % of COSA who have participated in training in the health topics of this project (Immunization, Diarrhea, Malaria, Nutrition, IMCI) - Interviews FGD 50% 0 100% 100% 1. % of COGE with financial records up to date. 2. % of COSA that hold regular meetings with health facility staff. - Project monitoring system - Interviews FGD 20% 20% 80% 60% Health area committees work with both communities and health facilities in order to improve the health situation of children and families in the health area. 3. % of COSA with action plans. 0% 100%* Association of Community Health Resource Persons (ACHRP) works with individual communities in order to improve health situation for community members. 1. % of ACHRPs that hold regular meetings. 2. % of ACHRPs that have action plans. - Project monitoring system - Interviews FGD 0% 0% 80% 75% Health facilities and Community Health Resource persons are linked in efforts to improve community health % of ACHRPs who receive regular supervisory visits from health facility staff. - Project monitoring system - Interviews FGD 0% 15% *100% for some specific activities such as outreach and vaccination Plan Cameroon Child Survival Project Final Evaluation Report August 23-September 1, 2004 17 2. Results: Technical Approach a. Introduction Plan International has just completed implementation of a four-year Child Survival XVI project in partnership with the Ministry of Health (MOH) and the Association for the Self-Promotion of the Eastern Province Population (AAPPEC). Partners also included the Catholic Mission, the Presbyterian Mission, Population Services International (PSI), Helen Keller International (HKI), and the Women’s Health Development in Sub-Saharan Africa (FESADE). The CS XVI project was located in three Health Districts of the western part of Cameroon’s heavily forested Eastern Province. The province is Cameroon’s poorest and borders the Democratic Republic of Congo. Plan has been working in this large, sparsely populated province since 1996. The project benefited inhabitants of 267 villages (a total population of 211,2642 ) located in 27 Health Areas of Bertoua, Doume and Nguelemendouka health districts. The project targeted approximately 95,024 beneficiaries with 38,009 children under five, 8,447 infants 0 to 11 months and 48,568 women of reproductive age (15 to 49). The goal of the Plan CS XVI project was to create a sustained reduction in infant and child mortality through increased access to community-based health care and improved quality of health services in the project area. This goal was achieved through the implementation of IMCI, which improved local access to preventive and curative services through capacity building interventions that targeted households, communities and health facilities. The project supported the MOH community outreach policy to empower communities to improve local health resources and increase access to health services. Sustainability has been ensured by strengthening the capacity of community resource persons (Health Area Committee Representatives, TBAs), community-based organizations (women’s groups), and the MOH staff (HCNs, community pharmacy workers) to provide quality services in the project area. Project interventions were strengthened using BCC activities designed to raise awareness about health issues and promote health-seeking behaviors. At the same time that communities were being empowered to take charge of their health needs, the project worked through its MOH partnership to strengthen the services of the health facilities by providing needed supplies and equipment and re-training staff. In addition, the project set up systems to assure delivery of outreach services to affiliated communities. It also worked to improve the functioning of the Health Area Committees. The project intervened in four main areas using IMCI as the approach: Malaria (35%), by improving Malaria Case Management, antenatal malaria prevention and treatment, and promoting increased use and re-treatment of insecticide treated mosquito nets; Malnutrition (25%) including mother managed growth monitoring and follow-up, 2 2000 MOH Eastern Health Delegation Report. Plan Cameroon Child Survival Project Final Evaluation Report August 23-September 1, 2004 18 household feeding programs, nutrition education, breastfeeding promotion, and proper weaning practices; Diarrhea (20%) involving prompt and appropriate Diarrhea Case Management, including use of ORS, proper hygiene, recognition of danger signs with appropriate referral and promotion of preventive measures; Immunization (20%) with increased immunization coverage in the program area for all infants by the end of the first year, vitamin A supplementation for children 6 months to 5 years, and tetanus toxoid (second dose) for pregnant women. Although funded from a different source, additional interventions were integrated into the CS project including pneumonia and HIV/AIDS and the Positive Deviance /Hearth Approach in rehabilitating malnourished children. The sustainability strategy stimulated community participation and ownership of health activities, improved the quality of MOH health facility and outreach services, and ensured financial sustainability through cost-recovery schemes. Community participation and ownership were encouraged at all phases from Health Area Committee actions, through monthly supervisory visits, through work with local committees and through all training and education activities. MOH policy was followed for all interventions, while Plan also helped the government to draft new policy on IMCI. Financial sustainability was attained through cost recovery and community income-generating activities supported by Plan as part of its ongoing development work in the region. Plan Cameroon Child Survival Project Final Evaluation Report August 23-September 1, 2004 19 b. Progress Report by Interventions IMCI Plan’s IMCI intervention in this child survival project, which began in 2000, preceded the government roll-out of IMCI in Cameroon by two years. Initially, Plan trained all of the 46 health facility staff working in the three project districts using a 6-day IMCI training, which was later to be followed up by the standard 11-day IMCI training recommended by WHO. The project then began working with the health facility staff to support them in fully integrating IMCI into their daily practice at the health centers in all three districts. Two years later, in 2002 the MOH initiated the pilot IMCI program in Cameroon, selecting three districts to begin with – Doume District in the East Province with Plan, Ngaoundere District in the Adamaoua Province with UNICEF, and Eseka District in the Centre Province with UNFPA (who eventually dropped out due to funding constraints, and WHO has assumed responsibility for this district). Thus, half way into this child survival project, Plan was required to redesign its IMCI intervention and scale it back to only one district (Doume District), which was selected by the MOH to participate in the IMCI pilot program for Cameroon. Only 30% (6 out of 20) of the health facility staff in Doume District was then provided with the standard 11-day IMCI training that the MOH had adapted for Cameroon. However, during the life of the project there was a high turnover of the few trained staff, causing a significant negative impact on health center activities (including outreach) and management. Furthermore, the WHO/UNICEF recommended assessment necessary for the implementation of the C-IMCI component still needs to be completed, while another batch of 6 staff from Doume health district is currently undergoing the 11-day training, which began September 6th . Given this history, it was decided that during the final evaluation of this child survival project, only the data for Doume District would be used to measure the impact of the IMCI intervention. It was further decided that the IMCI data would be presented for both trained (11-day) and untrained health facility staff. Doume District IMCI Indicator Baseline Final Trained Final Untrained The number of children 0-23 months seen at health facilities that were assessed for all danger signs 10.5% 72% 42.3% The number of caretakers who were correctly counseled about their sick child 36.3% 97% 65.4% The number of caretakers who were given oral medications, who know how to correctly administer the treatment 44.8% 84.6% 34.6% Plan Cameroon Child Survival Project Final Evaluation Report August 23-September 1, 2004 20 The following analysis is based on the data for Doume District for both trained and untrained health facility staff, and target beneficiaries. The number of mothers who know at least 2 danger signs that children need to be seen at a health facility increased from 61.6% baseline to 73.3% final. The increase is modest and the targeted 80% was not achieved. Even though the KPC showed this low increase, FGD during the final evaluation showed that many mothers do know the danger signs that are being taught through this project. This was further explored through the focus group discussions with the community health workers who were providing health education to the mothers, and through a review of the health education curriculum being used by the project to train these same community health workers. Upon further examination, it was learned that there is considerable mobility of mothers and children to/from the cities and villages. While the project is working through community health structures (which most mothers who live in the villages participate in) to disseminate messages, these more transient mothers do not participate in the activities of these community health structures, and they therefore may not receive the messages and information the project is disseminating. However, the KPC may have included a number of these more transient mothers, which could partially explain the minimal increase in this indicator. The delay in implementing IMCI in this district also played a role in the minimal increase. Through the FGD with mothers it was learned that there are considerable cultural barriers (grandmothers pressuring them into continuing their traditional practices) that are creating a situation where young women are feeling “confused” by the conflicting information they are receiving from grandmothers vs. the project, and that this is causing them to feel like they need to “choose” between traditional/cultural values and modern practices. One woman expressed feeling “guilty and conflicted” when her mother insisted on (scarification) making small cuts and then putting a traditional ointment on the soft spot on her newborns head because the spot was “too deep”, thus causing an open, oozing wound that had become infected. Another woman was unable to prevent her mother from feeding her newborn water just after birth, despite what she had learned to the contrary, and stated feeling “scared” that her baby might die from diarrhea, but at the same time she also felt too powerless to confront her mother. Thus, many young women are struggling to participate in project activities and follow through with the new information they are learning due to cultural practices and beliefs. Conversely, grandmothers who are participating as members of the community health structures are having a very positive effect on younger mothers. For example, several young mothers who had not been participating in health education activities testified during FGD that they had learned many positive attitudes and behaviors from their trained mothers including exclusive breastfeeding for the first six months and vaccinating their children following the immunization schedule. The number of children 0-23 months seen at health facilities that were assessed for all danger signs increased from 11% baseline to 72% (trained) and 42.3% (untrained), Plan Cameroon Child Survival Project Final Evaluation Report August 23-September 1, 2004 21 demonstrating the importance of the 11-day IMCI training. Further information from the HFA revealed that in almost all cases trained health workers were more motivated than their untrained colleagues. It was reported during the final evaluation that at the beginning of the project the public health centers were very dirty with garbage and animal droppings littering the environment both internally and externally, and that health facility staff did not report to work on any type of regular schedule. However, at the final evaluation the health centers were clean and many of the staff both trained and untrained were reporting for duty regularly. The final evaluation team discussed the need for the MOH to provide training for a targeted 100% of the MOH staff, using a phasing process to reduce the workload, perhaps over a 2-year period of time. Donors and agencies involved in IMCI should assist with the training process (both funding and technical support). A suggestion was made to use the health facility staff/sites that are performing excellently as model universities for IMCI – having staff from around that area visit the facility and learn from the model staff. This will not only provide a resource center for IMCI in the area, but will further reinforce the quality and pride of the facility staff who are excelling. The number of caretakers who were correctly counseled about their sick child increased from 36% baseline to 97% (trained) and 65.4% (untrained), further confirming the impact of IMCI training. This is often one of the most difficult components of facility IMCI and this project has mastered it with their trained staff. The final KPC and IHFA results confirms that mothers followed counseling instructions because the proportion of mothers who know at least 2 danger signs increased from 58.8 % at baseline to 73.3 %. Unfortunately, there is currently no referral system between the health facilities and the community health structures, so follow up at the community level by the health workers is rarely done. While quarterly meetings are supposed to be taking place on a regular basis to coordinate activities at both the facility and community levels, these meetings do not often happen as scheduled. The number of caretakers who were given oral medications, who know how to correctly administer the treatment increased from 45% to 84.6% (trained) and 34.6% (untrained). Unfortunately, due to the extreme poverty of this area, many families are unable to pay for services at the health facilities and therefore do not take their children in when they are sick. Furthermore, when they do take them to the health facility, many are unable to purchase the prescribed medication to complete the recommended treatment. During one home visit a father stated that the day before we arrived, his child had a high fever and malaria, and since he had so little money he could only go to the local kiosk Plan Cameroon Child Survival Project Final Evaluation Report August 23-September 1, 2004 22 and purchase two tablets of tetracycline, which he then gave to his child. He stated that the child was better today and the fever was gone. The number of health centers with supervisory visits that increased from 2 to 6 per year increased from 0% at baseline to 48% at final. The issue of supervision was discussed in great detail throughout the final evaluation due to the fact that this activity was considered to be pivotal for IMCI success. It was felt that the current level of supervision of the health facility staff by the district and provincial staff and by the project staff was inadequate. While the initial 11-day IMCI training is very comprehensive and includes several days of hands-on practical training, it should be followed up with regular supportive supervision and routine evaluation to ensure that the information learned is appropriately applied in the health facility setting, and that the required systems are in place and working well. This includes the tools needed for assessment, classification, counseling and treatment, the cold chain, the pharmacy and drug supply, transportation and logistics (ensuring that they are available, operating well, and maintained), and the management issues with regards to staff motivation and human resources. It was also felt that district level staff need to provide regular, continuous, quality supportive supervision to health facility workers during regular outreach, and also to periodically monitor and support community health workers and community structures. FGD with health facility staff revealed that supervision was inconsistent, inadequate in depth and breadth, and left the staff feeling unsupported. The staff also stated that when supervision is done properly it serves as a refresher course to keep them alert and performing well, and also helps them solve problems and improve the quality of services that the health facilities offer. At the provincial and central levels, the MOH needs to coordinate IMCI efforts throughout the country, ensuring collaboration and consistency as well as sharing lessons learned and scaling up successful approaches and activities. The number of supervisory visits to VHC/ACHRPs that increased to 4 per annum increased from 0% at baseline to 70% at final. Both health promoters and health facility staff conducted these supervisory visits. Further to the discussion above, supervision of community health structures and workers by project and health facility staff was felt to be inadequate for this project. These first line community health workers are the mainstay of this project, ensuring that health education messages are delivered and taken up for behavior change by the community. However, with only a few days of initial training and minimal supervision received by these community health workers, the expectation that they will demonstrate quality performance into the future is unrealistic. The training these workers initially received was not adequate to ensure the quality of their productivity and needs to be Plan Cameroon Child Survival Project Final Evaluation Report August 23-September 1, 2004 23 followed up on a regular basis (ex. monthly until they demonstrate high performance, and then quarterly), which was not done sufficiently. Likewise, it was suggested that pre-positioning of supplies is necessary for ensuring that they are in place and accessible at the COMMUNITY level. This means putting commodities in villages (ORS, condoms, blank immunization cards, malaria kits, bed nets, etc.). Finally, Plan, AAPPEC and other agencies involved in IMCI in Cameroon need to provide regular, continuous (for the first few years), quality supportive supervision to community health workers and community structures, health facility staff, district staff and the provincial and central policy-making bodies of the MOH. During the final evaluation the team discussed in detail the issue of utilization rates. It was determined that while utilization of health centers (including outreach) has increased, it is still not high enough to provide coverage for the target population. It was felt that utilization needs to be increased to around 60% to be truly effective. For IMCI scale up, this is a huge issue and will have a big impact on the success of any future project. The team agreed that there would need to be further discussion as to what strategies should be adopted, but that far more outreach activities will be required in any case. Special emphasis needs to be placed on exploring how to: · Measure utilization against population size to better ensure that coverage figures are accurate. · Increase utilization by marketing of health facilities and their services · Ensure quality services on a continued basis. And that the following needs should be addressed: · The need for either more project staff or an increase in the number of local partners to work with on child health issues (other Plan partners working in HACI should be included in child health projects so that when projects end, other partners can continue with the activities in the field). · The need to provide close supportive supervision on a regular and consistent basis at all levels: Plan staff, MOH staff, AAPPEC staff, community health workers, community structures and households (caregivers). · The need to provide routine maintenance for all equipment and supplies to keep activities operational. · The need for a fully functioning referral (and counter-referral) system to be put into place and maintained at all levels, with defined follow up actions. Plan Cameroon Child Survival Project Final Evaluation Report August 23-September 1, 2004 24 IMCI Recommendations · The 11-day IMCI training is needed for all staff (doctors and nurses) and supervisors at all health facilities. · IMCI-trained (health facility and supervisory) staff should be required to remain in their positions (MOH policy change) for a given period of time (ex. 2 years), and should not be moved within that time period. · Health districts need to take ownership of IMCI in order for it to be successfully implemented. · Intensive, integrated, facilitative supervision for IMCI needs to be carried out at all levels including project partners, health facilities, community health workers and structures, and households. · Routine evaluation (ex. Quarterly) should be done alternating between quantitative (ex. LQAS) and qualitative tools (ex. Focus group discussion) to spot check the impact of health education activities and behavior change interventions. · Preposition supplies and commodities at the village level. · The 16 key messages for IMCI should be further disseminated to schools (to affect children and the teenage pregnancy cohort) and through mass media (using local languages). · Develop a strategy for accommodating mobility of mothers and children, and their vaccination cards. · Grandmothers should be included in the target population for health education, and the CBO and COSA should recruit grandmothers as members wherever possible. · Carry out planned operations research on cultural barriers to healthy behaviors. · Provide routine maintenance for all equipment and supplies to keep activities operational. · Consider developing a multimedia marketing strategy to increase utilization of health services and staff. · Develop a fully functioning referral (and counter-referral) system to be put into place and maintained at all levels, with defined follow up actions. · Review the current fee system at health centers, and ensure that treatment and drugs are available for those unable to pay. · Consider incorporating traditional healers, kiosks and local pharmacies into project activities regarding appropriate drug management and dispensing. Plan Cameroon Child Survival Project Final Evaluation Report August 23-September 1, 2004 25 Malaria Malaria is highly endemic throughout the project area, and the malaria intervention was well received by all project partners. However, there has been some conflict around whether to sell bed nets at a subsidized price, or to distribute them free of charge. Initially the project was in favor of creating an income generating project from bed net sales. Unfortunately, many community members stated they were still unable to afford the nets. Plan project staff felt that this was untrue, and that subsidized nets were affordable. Plan distributed 6,000 free ITNs to sponsored children during the second year of the project. The project further distributed 6,250 ITNs through trained CBOs and more than 90 % have been sold. Local production of nets was evaluated, but found to be too difficult. Collaboration with PSI eventually brought greatly reduced prices and increased supplies of bed nets to the project area, but this didn’t happen until the final year of the project, so the impact has not yet been felt. During the life of the project the MOH rolled out its Roll Back Malaria program and started distributing ITNs free of charge to pregnant women attending prenatal clinics at health facilities. The Chad-Cameroon oil pipeline transportation company COTCO also distributed ITNs to the population along the pipeline in Belabo health area in the project zone. The CBO trained by the project in this town was hired to impregnate the nets. PSI has also contributed in promoting ITN use and now is providing K-O tabs for re￾treatment. The number of households that have an (initially treated) ITN increased from 6% baseline to 33.6% at final. However, this data includes bed nets that were initially impregnated as well as those that were AND were not retreated in the last six months. During one home visit a father stated that the day before we arrived, his child had a high fever and malaria. The team observed one mosquito net (in fair condition – dirty but intact) in this house, and the father reported that he and his wife and new baby slept under it, while the other 8 children, including the one who had malaria the day before slept without a mosquito net. The father was able to correctly demonstrate how to tuck the bottom of the net under the mattress when in use. When asked why he did not purchase a second net, he explained that he had no resources with which to purchase one, and that the one he did have was given to him by Plan two years previous. When asked if it had been retreated, he answered negatively, stating that he was unaware of re-treatment. During FGD with mothers, some of them stated that they wanted to retreat their nets, but they didn’t know where to purchase the K-O Tabs, while others in the group stated not having the financial means to obtain them. The number of children, 0-23 months, and their mothers who sleep under an ITN dramatically increased from 0.4 to 33.6%, with only 15.6% re-impregnated during the last six months. Plan Cameroon Child Survival Project Final Evaluation Report August 23-September 1, 2004 26 Re-impregnation remains problematic, and these activities fell far short of the target for a variety of reasons. The insecticide (K-O Tabs) used to retreat the nets was not available in the market for the first three years of the project, and attempts to have them supplied by PSI were delayed for over two years. The PSI Project has now made the treatment kits widely available at a subsidized rate in the local markets. However, there is still a need to organize bi-annual community re-impregnation campaigns. . The health personnel, CBOs and health committees will facilitate the purchase and re-treatment process for ITN users. Trained CBOs involved in ITN promotion have been recognized by the MOH and collaborate directly with MOH personnel at the district and health center levels. The number of mothers of children 0-23 months who received iron/folic acid supplementation during the last pregnancy as recorded on the maternal health card increased from 28.4% to 51.4%, reaching the target of 50% coverage. This is a very positive result, especially because anemia in pregnancy is very common in the project area. During antenatal care provided at the health facilities, iron/folic acid supplements are provided at a highly subsidized rate. However, during the final evaluation discussions it was felt that this activity needed to be significantly increased, particularly during outreach in the next scale up project. The number of women’s groups trained on home management of malaria during the life of the project grew from 0% at baseline to 100% coverage by 120 CBOs at final. FGD showed that messages were given by the CBOs to mothers, especially concerning the use of ITNs to prevent malaria. Regarding malaria treatment, mothers responded that they tepid sponged their child and then consulted the nearest health facility. The MOH is currently in the process of putting in place home based care of malaria with treatment kits that are to be distributed shortly at the community level. The number of health facility staff trained in malaria case management increased from 0% at baseline to 100% at final. Health facility staff all received training on MCM, and their performance was dramatically improved, as captured in the HFA. The Cameroon MOH policy has just been revised for treatment of malaria. The first line treatment is Artesunate and Amodiaquine given as combination therapy. The health staff has been instructed to complete their stock of chloroquine while the country awaits the arrival of new treatment kits containing the combination therapy. Sulfadoxine-pyremethamine (Fansidar) has been adopted for intermittent preventive treatment (IPT) for pregnant women and it has to be administered under direct observation (DOT) at prenatal clinics. Plan Cameroon Child Survival Project Final Evaluation Report August 23-September 1, 2004 27 Malaria Recommendations · Significant emphasis should be placed on re-treatment activities for future projects. · Project partners need to make a consistent and unified decision regarding the sale vs. free distribution of bed nets. · Project indicators need to be expanded for bed nets and re-impregnation to include issues of knowledge, access and usage. · A focus on intermittent presumptive treatment of pregnant women should be included in future projects. · A focus on iron/folic acid supplementation during pregnancy should be included in future projects. · Knowledge (dosage, frequency) and access to drug treatment should be expanded and include street vendors, kiosks, traditional healers and community health workers for expanded home-based treatment. Plan Cameroon Child Survival Project Final Evaluation Report August 23-September 1, 2004 28 Diarrhea Diarrhea remains a daunting problem in the project area, and while many caretakers are improving their responses to sick children, there is still need for improvement. Plan is actively engaged in water supply and protection schemes and latrine construction with many of the communities in the project area. The project has worked closely with PSI to make ORS more readily available at the village level, and CBOs are routinely training mothers to prevent diarrhea, to prepare and administer ORS, appropriate home made fluids, increase breastfeeding, hand washing with soap or wood ash, and to recognize the danger signs and seek treatment. The project is encouraging CBOs and kiosk owners to purchase and distribute ORS sachets at the community level. During a home visit, the mother of a one year old showed the team her family pharmacy which contained a package of ORS, cotrimoxazole, and paracetamol. When questioned about how to use these supplies, she correctly explained and demonstrated. The KPC 2004 showed an increase from 42.3% at baseline to 56.2% at the final, missing the target of 70% of children who receive more fluids than usual during the last diarrhea episode. This was further elaborated on during the focus group discussion with mothers, some of whom stated that giving more fluid would aggravate diarrhea, and instead most mothers focused on providing more food, as demonstrated by the indicator for increasing food during diarrhea from 51.3% at baseline to 76.6% at final, exceeding the 70% target. While nearly all women breastfeed their children regularly, there was not a significant increase (2.9%) of women who provided additional breastfeeding during episodes of diarrhea. This can partially be explained from information on cultural beliefs obtained during the focus group discussions which separated types of diarrhea, some of which are caused by the mother having sexual relations with her husband while breastfeeding. The health facility assessment shows a baseline value of 23% and a final value of 67.4% of children seen at the health facility that were correctly treated for diarrhea, thus not reaching the target of 70%. When this data is disaggregated for trained and untrained staff in Doume District, where IMCI is being piloting, they are both exactly the same (85.7% trained and untrained). There are many cultural beliefs that are preventing caretakers from changing their behavior and integrating these behavior changes into daily practice. Plan Cameroon Child Survival Project Final Evaluation Report August 23-September 1, 2004 29 Diarrhea Recommendations · Additional focus on hand washing should be included for scale up activities, specifically on increasing access, reorienting belief systems, and improving habits. · Explore possible solutions for soap access including micro-enterprise for home￾based production and community sales. · Reinforce and strengthen BCC messages for refuse disposal, latrine construction and usage, water treatment, and food storage. · Expand ORS access and usage at the household level. Plan Cameroon Child Survival Project Final Evaluation Report August 23-September 1, 2004 30 Immunization When the child survival project began in 2000, the MOH in the East Province did not have sufficient cold chain material, means of transportation for outreach, or a sufficient and regular stock of vaccines. There was also a very low utilization rate of health facilities. This project greatly strengthened the immunization infrastructure by providing extensive training for health facility staff along with refrigerators and other vaccine equipment and supplies. Vaccines, needles and syringes are provided by the MOH, with assistance from UNICEF. The MOH, with support from the GAVI & HIPC initiatives has also reinforced the EPI nationwide. Staff motivation posed a challenge during the first year, however this improved somewhat with training and increased means for service delivery to target populations. The CS project measured immunization status through the KPC by using immunization cards. While this is an acceptable method of measurement, the realities in the East Province are such that most women don’t have cards for themselves and their children, and often those that do have cards don’t bring them to outreach and health facility activities. Also, during NIDS, vaccines are not recorded on the cards by the health facility staff. Therefore, it is believed that the KPC greatly underestimated the true immunization coverage rate. This was further explored during the FGD with the health facility staff during the final evaluation, at which time the staff stated that they questioned the validity of the results, and they also discussed the high level of mobility of the target population. Children aged 12 to 23 months who are fully immunized increased from 24.7% baseline to 54.9% end point value. However the targeted 60% was not attained. This is partially explained by the fact that some mothers migrate from district to district. Also FGD showed that some mothers don’t have their vaccination cards all the time and at times some mothers do not have the vaccination cards of their children and the nurse administers the vaccine without recording it on the card. In spite of not attaining the targeted immunization completion rate, the dropout rate for DPT decreased from 37.5% baseline to 13.1% end point value, exceeding the 15% target. Children aged 12 – 23 months vaccinated with measles increased from 60.7 to 62.1% not attaining the 80% target. This is due mostly to poor record keeping and communication. It is thought that the coverage rate is probably higher due to the reasons mentioned above. Mothers of children aged 0 – 23 months who had TT2 during their last pregnancy increased from 47% to 55.3% not achieving the targeted 70%. This could also be due to a lack of cards kept by mothers as well as health facility staff not diligently recording vaccinations when administered. Vitamin A supplementation for children aged 6 months to 5 years during the last 6 months increased from 9.4% to 58.9% exceeding the 40% target. However there is still the second dose problem which could be addressed during opportunities like growth Plan Cameroon Child Survival Project Final Evaluation Report August 23-September 1, 2004 31 monitoring sessions by CBO members. Currently, negotiations are going on between Plan and HKI to seek ways of integrating the second dose of Vitamin A with ivermectin. Children visiting health facilities that had their immunization status checked increased from 30% baseline to 78.7% end point value. While the target of 80% was just missed, it is likely that the HFA figure is low due to the fact that not all mothers bring the immunization cards of their children to the health facility. Immunization Recommendations · Increase outreach activities for vitamin A for children (6 months to 5 years) and TT for pregnant women. · Improve immunization record keeping (cards, registers, etc) and communication. · Revisit how to measure coverage rates (other tools, records, etc). · NIDS should be recorded on immunization cards and health center registers. · Refresher courses on immunization should be implemented for health facility staff. · Mothers should be taught to bring immunization cards to the health center when coming for consultation or treatment. Plan Cameroon Child Survival Project Final Evaluation Report August 23-September 1, 2004 32 Malnutrition Malnutrition has been and remains a constant problem throughout Cameroon, especially in the rural areas. It is primarily due to inappropriate or non-use of locally available foods rich in vitamins, minerals and protein. Results of a recent study by Plan revealed up to 30% malnutrition among children under three years old. Growth monitoring and community-based nutrition rehabilitation, nutrition education and IEC are the primary components of this intervention. Scales and other equipment were supplied by the project, as was refresher training of staff. However, in spite of these efforts, this intervention has not demonstrated significant results. For this reason, the project decided to switch to using the PD/Hearth approach instead. This change has only recently begun and is not yet being fully implemented. In the next scale up project, it will be a major focus of the nutrition component. Children who attended growth-monitoring sessions during the last 4 months and whose weights were plotted on the road to health growth monitoring card increased from 11% to 44.4%, which exceeded the targeted 40%. Since this indicator was measured using children’s cards we believe that the actual value is higher because many children did not have their cards with them for various reasons, e.g. mothers relocate and the cards are left behind. CBO members in their respective communities conduct growth monitoring and nutrition rehabilitation sessions. During these monthly sessions children are weighed and the information is recorded both in the community register as well as plotted on the child’s card. This is followed by a nutrition demonstration session where all the mothers of U-5 children participate. For children who were identified with malnutrition, the CBO members follow them through home visits where they ensure that mothers are putting into practice what they learned at the demonstration sessions. Emphasis is placed on using locally available foods to prepare balanced meals for the children. In spite of this regular exercise conducted by the community members, the malnutrition rate decrease is not significant, from 19.1% baseline value to 16.4%, missing the targeted 10%. Even though this is partially explained by the fact that nutrition rehabilitation did not start until after the MTE, it also confirms that there is need to adopt innovative approaches like PD/Hearth, which has shown its worth in a pilot project carried out by Plan Cameroon with funding from the Japan National Plan office, where some of the children participating in the first round of hearths had weight gains of up to 1200 g. CBO members underwent a 6-day training on child health issues covering all the 6 intervention areas. Some members were further trained on nutrition rehabilitation using locally available foods. CBO members were also trained on management and community ITN treatment and re-impregnation. The CBO members were not helped by the project to organize themselves into sub-committees responsible for taking care of the various activities e.g. nutrition, ITN treatment and re-impregnation. Supervision was a major issue as indicated earlier, however, all 120 CBOs were not trained at the same time. Plan Cameroon Child Survival Project Final Evaluation Report August 23-September 1, 2004 33 Phase-in was progressive and indirectly CBO members were trained on planning, and job segregation. CBOs were encouraged to do it by themselves. Work was organized based on the Health Center community outreach sessions. During such community health days they carried out nutrition demonstrations, health talks, etc. Meanwhile, in between, there were weekly sessions planned but which were postponed occasionally because of other community events. Follow-up of efficacy of CBO work was by testimonies from mothers and health staff, monthly reports of the CBOs, and field verification of cases of malnutrition. Data is validated using the supervision checklist in the field. Direct Observation of CBOs by the project staff (health supervisors) was also done. Only 62.5% of children identified with mild to moderate malnutrition participated in the community-based rehabilitation program. About 1/3rd of these children dropped out from nutrition rehabilitation sessions because their expectation for free food was not met. The number of children seen at the health facilities who had their nutrition status plotted on the road to health chart of the child health card increased considerably from 3.2% to 54.3, exceeding the targeted 50%. For health staff who were trained on IMCI it increased to 84%. Mothers of children 0 – 6 months of age who exclusively breastfed their children increased from 29 to 56.9%. However, there are still problems with breastfeeding due to the negative influence of grandmothers and grandfathers regarding this activity and the provision of water for newborns and infants. Mothers of children 6 – 9 months who are giving the child complementary food had an insignificant increase from 66% to 67.3%. The issue of supervision of CBOs through periodic observation and support was problematic, as mentioned under IMCI, and needs to be improved considerably. Plan Cameroon Child Survival Project Final Evaluation Report August 23-September 1, 2004 34 Malnutrition Recommendations · Ensure that CBO coverage is provided for at least 80% of communities. · Health cards should be freely distributed by CBO/TBA/COSA to all children 0 – 5 yrs without a card · Intensify supervision and mobilization of CBOs engaged in rehabilitating malnourished children (and record keeping). · Change the malnutrition strategy from nutrition demonstrations to PD/Hearth and scale up to all project areas. · Address cultural issues around food/feeding, breastfeeding and fluids. · Integrate vitamin A distribution with growth monitoring sessions. · Increase training of health center staff, community structures and community health workers on nutrition rehabilitation. · Develop a marketing strategy for PD Hearth, and ensure that community members understand that free food will NOT be distributed. Plan Cameroon Child Survival Project Final Evaluation Report August 23-September 1, 2004 35 3. Results: Cross-cutting Approaches and Issues Encountered a. Community Mobilization Health centre personnel, Project staff, and the health promoters were the front line staff of the project, and they were in charge of social mobilization, sensitization, and training of CBOs. They were pivotal to reaching the communities. The work they have done with the community has built tremendous trust between them and the community. They made sure that each CBO had a work plan, that it was being followed, and that the CBO was implementing their work and doing it well. They were responsible for backstopping, monitoring and supervising all the work of the CBOs, TBAs and COSAs at the community level. The health promoters stated that if they, the project field staff and health centre personnel were not diligently following up the CBOs, then the quality of work dropped very quickly. The health promoters collected project data on a monthly basis and forwarded it to the health districts and Plan. The promoters participated (as student trainers in the 2nd and 3rd year of the project) in a majority of the training sessions for the CBOs which were conducted by the project and the district and health center staff. Once they acquired the relevant knowledge and mastered the training techniques, they then carried out the training of 21 CBOs in the final year of the project, with project management playing a supervisory role. The health promoters worked with the CBOs on a daily basis, and in collaboration with the MOH staff when they provided outreach. For example, when the MOH staff finalized their outreach schedule, then the promoters would organize people for outreach immunization and other activities. During outreach, the health promoters, health committee and CBO members and TBAs were part of the team carrying out activities and worked closely with the health center staff in charge. Usually when the health staff was doing technical work, the health promoters would provide health education, demonstrations and other activities with mothers. A topic would be selected and they would work with mothers on it. They would also take full advantage of all opportunities to provide teaching and education in follow up to consultations carried out between mothers and health center staff. For instance, when a patient was diagnosed with malaria by the health center staff, then the promoter would work with the patient and/or caregiver to teach them about malaria prevention and treatment. They also assisted in the training of TBAs conducted by the project staff and followed up trainees after the training to ensure they conducted deliveries respecting basic hygienic conditions and wearing gloves. Bi-annual reviews and synthesis of work plans with the CBOs and COSAs included looking back at the last six months and discussing problems encountered, solutions, what was achieved and then making work plans for the next six months. This was helpful, but needed to be reinforced in order to be sustainable. Participants in these meetings included COSA, CBO, TBA, health center staff, project staff, and other project partners. Plan Cameroon Child Survival Project Final Evaluation Report August 23-September 1, 2004 36 Each promoter was assigned 2-3 areas made up of about 25-35 villages. They worked exclusively in the field at the community level, and did not have an office anywhere. The promoters felt that they had greatly increased their ability to function in this role due to the numerous and wide range of trainings provided to them by Plan and AAPPEC. Many tools and resources were given to the health promoters to help them do their jobs. They received teaching aids, demonstration supplies, management tools, and many other materials, all of which they said were very helpful. There were several problems that occurred at the community structure level. In reality, only one CBO per community was being trained, and this was not sufficient to ensure sustainable dissemination of health messages to all community members, so there existed a need to train more people (recruiting as many grandmothers to participate as possible) and also to provide better supervision. The partnership between the community and the health facility also needed to be strengthened. Incentives presented a problem as PLAN began providing incentives early on in the project, but this was not sustainable and it raised expectations of community members when other project partners came to work with them, including AAPPEC and the health promoters as well as the MOH staff. Because people moved around a lot and would leave a village, turnover of trained people became a problem. One possible solution that was discussed to increase morale was to have a ceremony and provide a badge to the community health worker, and then have an annual renewal system so that their participation was reinforced. Another idea was to have peer reviews in the community, to further support the work of the community health worker. Other suggestions included reorganizing and strengthening the relationship and coordination between the COSA and the CBOs, TBAs and other community partners, ensuring representation of the CBOs and other community partners in the COSA, and strengthening the capacity of the COSA to operate effectively. Community Mobilization Recommendations · Train more than one CBO per community, and include as many grandmothers as possible. · Increase supportive supervision of community structures for community mobilization activities. · Strengthen the relationship between the community structures and the health facility staff. · Provide a badge and perhaps a ceremony to recognize and motivate the community health workers and structures. · Introduce peer reviews of community health workers in the villages. · Reorganize and strengthen the relationship between all the community-based structures and partners. Plan Cameroon Child Survival Project Final Evaluation Report August 23-September 1, 2004 37 b. Communication for Behavior Change One of the major activities of the project was awareness raising on child health issues. This was intended to provide mothers and caretakers of children pertinent information on the six top killer diseases of under fives. It was expected that this increased awareness would result in a positive change in attitudes and practices in the individuals and subsequently in the communities in favor of behaviors that promote child and maternal health and enhance their survival. Building on existing IEC materials of the MOH and the 16 key household/community practices that promote child health and survival developed by WHO/UNICEF, the project, the MOH and AAPPEC developed a set of IEC materials in the six domains of the project (flyers, posters, audio messages) and used them to facilitate behavior change communication between service providers and the caretakers of children. The project trained frontline health staff, CBO and health committee members, TBAs, the health promoters and project field staff on how to use these materials. The audio messages were broadcast by two rural radio stations in the project zone. The methods used to disseminate the messages included group health education talks, interpersonal communication, sketches, songs, and radio broadcasts. The FGD and KPC results demonstrate that there has been a tremendous increase in the knowledge level on child health issues among caretakers, and a significant change in behavior. Some behavior change objectives were attained while others were not. The overall trend was in favor of positive behavior change. However, there is still much to be done to sustain the trend. ITN use by under fives and mothers increased from 0.4 % at baseline to 33.6 % at the final evaluation, which exceeded the objective of 20%. The proportion of mothers who wash their hands with soap before feeding the child increased from 35.4 % at baseline to 62.5 % at the final evaluation. Those who wash hands with soap on return from defecating increased from 2.6 % to 20.6 %. The proportion of mothers of children aged 0-6 months who practiced exclusive breastfeeding increased from 29 % at base line to 57 %. The proportion of mothers who know at least 2 danger signs that a child needs to be taken to a health facility increased from 58.5 % to 73.3% at the final evaluation. Participation of mothers with malnourished children in community nutrition rehabilitation sessions increased from 0% at baseline to 62.5 % at end of the project, far exceeding the 40 % target. Plan Cameroon Child Survival Project Final Evaluation Report August 23-September 1, 2004 38 The level of increase in knowledge of caretakers is now much higher but it is not matched by a corresponding increase in behavior change. This gap between knowledge and practice/behavior is accounted for partly by cultural beliefs, taboos and misconceptions. Increase in awareness and knowledge on some health issues does not necessarily lead to behavior change. For example mothers know that it is good to increase fluid intake and breast feeding for children having diarrhea, but many do not practice it because of cultural beliefs and misconceptions (conflict between tradition and knew knowledge). Frontline health personnel, health committee and CBO members, TBAs, and health promoters will need to be assisted/supported to continue to carry out activities that enhance mothers/caretakers ability to adopt new attitudes and practices which promote behavior change in favor of child health. Grandmothers will need to be brought into the change process. The participatory learning and action (PLA) approach is being applied, as is the BEHAVE model to facilitate behavior change. The health promoters learned that communication for behavior change is very important, and it takes a lot of patience and time. Behavior Change Communication Recommendations · Grand mothers should in the future be targeted in BCC training. · From time to time conduct surveys (FGD, LQAS, Rapid Catch) to ascertain if IEC messages are being effectively put into practice. · Health districts should avail themselves of existing rural radio stations to disseminate key messages on child and maternal health. · Continue to multiply and use the IEC materials developed during the life of the project. · Reinforce the application of innovative approaches such as child-to-child, child-to￾parents, and peer-to-peer. Plan Cameroon Child Survival Project Final Evaluation Report August 23-September 1, 2004 39 c. Capacity Building Approach All of the indicators on capacity building relating to assessing danger signs, health staff correctly completing counseling tasks, and caretakers knowing how to correctly administer treatment have been reported in the above sections. Plan International has been strengthened by this CS grant in several ways. The headquarters has just gone through a change in technical backstopping staff, and this final evaluation provides a training ground for the new staff member to help build an expanded impact project for scaling up IMCI in an entire country. Plan HQ will be able to transfer technical and managerial skills to other child survival projects that are currently implemented by Plan globally as well as provide technical guidance and comments on development of Plan private funded CS proposals. This was the first time that Plan Cameroon has received a CS grant, and the first time it has also tried to partner with a local NGO in the East province. Several field staff described the following situation: When the project began, it was with Plan staff. The MOH health care providers and partner NGOs came on board a year later. Plan really had ownership and power over the project decisions, budget and activities. Both Plan and AAPPEC held regular monthly project meetings, however only the AAPPEC supervisor attended both meetings. Initially, there was very little coordination or communication, and everyone was struggling very hard in the field to implement the activities. Plan CS project staff did the initial training of CBOs, TBAs, and health committees with AAPPEC health promoters observing and learning how to do it, and toward the end of the third year when they had mastered the methodology, the promoters then began to do the training on their own while Plan played a supervisory role. Several field staff members stated that the Plan project atmosphere was very top down, and that they did not feel included in project management. The work plans were designed and given to the staff with a time frame, sending people off to execute the plans and then come back, write a report and then the staff member was given a new set of instructions. It was felt that a more participatory approach to project management would have helped to better facilitate the project activities and the field work. The CS project has reinforced Plan’s capacity to deliver outputs within the health domain in Cameroon. Plan needs to strengthen its relationships with other PVOs working in Cameroon, and to share lessons learned. It was reported that the CS project had tremendous impact on the other domains Plan works on in the East: · Learning Domain - because children are healthier, school attendance has increased (especially in the Pygmy population) by around 50% for both male and female children · Livelihood – the Pygmy population is now healthier and is actively engaged in family-owned farming activities, which previously was not the case · Habitat – Improvements and building of homes in concert with the child survival project activities has facilitated healthier environments for families Plan Cameroon Child Survival Project Final Evaluation Report August 23-September 1, 2004 40 · Rights of the Child – The project has facilitated parents to uphold their commitment to provide health for their children · Sponsorship – The rate of loss of sponsored children from death by curable diseases has decreased by 10% AAPPEC has grown tremendously as an organization by being involved in this CS project. Plan reported that the head of the organization had a very heavy workload, and he was not present at some important meetings so decisions could not be taken. This slowed down some of the management decisions at times. It was stated that they really need to have someone with health expertise above the level of the health promoters. AAPPEC reported the following strengths and weaknesses based on their experience over the last four years working with Plan on this CS project. · We are a local NGO and we work and provide a voice for the Pygmy population. · Plan’s philosophy was to have local partnership, and this partnership is the first one for Plan Cameroon in the East province, so AAPPEC is the training ground for this experience. Plan can learn a lot from this partnership. · AAPPEC has learned a lot about child survival, about the strategy of working with mothers in health projects, and also with CBO, COSA, DS – this is all new for our NGO. · AAPPEC has gained tremendous knowledge and improved their approach toward working with communities, especially with the Pygmies. · This partnership has made the NGO become “known” with the MOH at the central level, which is very important for our organization. We have signed a local agreement with the local health authority in the east province. Because of our work with Plan, we have written another proposal and received funding from the European Union for a health project called PASS – training of traditional birth attendants in the villages in three districts in the East of Cameroon. · The strategy of working with CBOs was very positive, and AAPPEC learned a lot about how to work with community structures through this experience. · The CS project goal was also very helpful for AAPPEC, and the organization has since integrated some CS activities into its other programs, and is looking for other funding to sustain them. · Previously AAPPEC only worked with the Baka people, but this project helped the organization expand and work also with the Bantu people. This was very valuable and important for AAPPEC’s growth. However, now the project is ending with really only two years of implementation – and AAPPEC needs support to continue with the activities in order to finish what was started with this project, and possibly expand further into the entire East Province. Weaknesses: · Plan’s financial regulations are strict. · While on paper the project is four years, in reality the project has only been implementing activities for a little over two years because of all the preparation activities that took a lot of time. This is too short a duration to change behavior at the community level · Plan Cameroon Child Survival Project Final Evaluation Report August 23-September 1, 2004 41 · This is the first partnership for Plan in East Cameroon with a local NGO, and they don’t have any previous experience to draw from, and so they were learning by doing, and many mistakes were made with all the partners, such as Plan trying to maintain power and at the same time trying to share it. · The lack of adequate human resources at the health facilities, and their de￾motivation caused tremendous stress to the project. · When a larger NGO works with a smaller local NGO, it is difficult for communities to understand the difference between the two, and this affected AAPPEC’s image in the field. · All three partners (Plan, AAPPEC and MOH) were in the field together and were supposed to bring the same message. However, in reality each partner often came with a different message and this caused great difficulty at the community level in terms of motivation, funding, and their comprehension of the project. · The financial and material support was only for CS activities in the field, and thus the infrastructure of AAPPEC was not strengthened. The organization was not provided with a computer, a printer or any other resources that will remain with and help continue to strengthen our office. · Plan promised to help AAPPEC with capacity building of its staff during the project. However, in reality it was mainly the capacity of the health promoters that was increased. · The management of the project activities in the field was confusing earlier on in the project. The AAPPEC health promoters received instructions from AAPPEC, from Plan and from the MOH. This caused real difficulties for the staff and the project activities. An Organizational Capacity Assessment of AAPPEC was done at the beginning of the project and repeated at the end of the project. Results of the study included increased visibility and a wider geographic area, new and better approaches with respect to sensitization and training, improvement in conditions and climate of work with government and other organizations, increased number of domains of intervention, acquisition of equipment and improved methods of work. Some of the conclusions from the final report of the assessment include: · Plan’s objective of improving the capacity of AAPPEC by involving it as a partner organization in the implementation of the CS Project has been attained up to a certain extent at both the institutional and manpower levels in terms of lessons learned and gaps filled. · The degree of attainment of this objective would have been much higher and durable had more permanent AAPPEC technical staff been directly involved in the implementation process. · AAPPEC has definitely benefited from its involvement in the CS Project in the sense that most of the previous gaps have been filled but other important ones still need to be filled through training workshops and coaching. These include: ß Basic management skills ß Dealing with resistance to change in behavior ß Report writing skills ß Skills in conflict resolution Plan Cameroon Child Survival Project Final Evaluation Report August 23-September 1, 2004 42 ß Participatory action training methods The local community structures have been improved by enriching their understanding, knowledge and experience through participation in the CS project. Before the project started, villages had a huge lack of general management. With the training and support the health promoters have provided to the CBOs, those communities can now plan, make decisions, and follow up on those decisions. One of their main focuses in backstopping or supporting these CBOs is capacity building and sustainability since they knew that this project would come to an end and these people needed to have skills to organize themselves. When the communities were informed that the project was ending, they expressed a lot of concern that the health promoters would be leaving. They really had a lot of involvement in conflict management in the CBOs – they worked as brokers to resolve conflict. For this to happen, they needed to build long term trust. When the project started there were no supervisory tools available, so the project and partners developed new supervisory instruments that included a section for joint problem solving. This indicator can therefore be evaluated as having undergone a 100% increase from 0 at the baseline. The CBO supervisory checklist was developed and used to supervise, identify and solve problems at the community level. Problem solving between health facility staff and communities was done through discussions at health area committee meetings. The District health team supervised IHC more than they did prior to the onset of the project where 0% supervision was noted at baseline and 32.1% end point value. During the life of the project, health staff were encouraged to supervise VHC and community resource persons. The number of VHCs who receive regular supervisory visits from IHF increased from 0 % at baseline to 15% end point value. The health staff have been slowly increasing these activities, which have always been assumed in their normal work, but which were never implemented prior to the onset of the project. COSAs have a written vision statement which was developed in the MOH policy that re￾orientated primary health care in Cameroon. With support from the project, the COSA members lead the ACHRP to hold regular monthly meetings and the vision statement of COSA was applied at this level as well, and in the context of that particular community. The health staff supported the Health area committees to develop action plans especially in line with the outreach activities. This gave a rise from 0% at the baseline to 100% end point value. Most of the associations of community resource persons (ACHRP) developed action plans as can be seen with the dramatic increase from 0% baseline to 75% end point value. Plan Cameroon Child Survival Project Final Evaluation Report August 23-September 1, 2004 43 It was reported that “the good parts about the project were really the change in the thinking of the mothers, as they became more open to participation. The problem remained the cultural limitations and constraints, particularly the grandmothers who did not want to change the old ways. For example, when the project began working with the mothers, none of them would stand up and introduce themselves. By the end of the project, the meetings were very dynamic and the women not only got up to introduce themselves, but participated freely speaking openly and giving health talks in front of both men and women. The mothers are also participating in outreach now, and the health centers are cleaner and utilization has increased a great deal, which has caused the staff to behave differently, and improve their conduct.” · Mother’s of children are more aware that they have to recognize that their children need to be in good health, and know what to do in order to keep them healthy, and how to respond when they are sick · In these communities, mothers have been able to realize that they can use the food within their environment to keep their children well nourished and healthy, and that they don’t need to purchase imported products · The role and existence of the CBOs should be maintained and not lost after the project ends. They also need a lot of supervision and follow up to be productive. · Define the minimum and maximum number of CBOs needed per health area who can work effectively and efficiently to implement the project in order to achieve sustainability. The MOH has grown tremendously by being involved in this CS project. Plan reported that the district level should effectively play its leading role in ensuring that things are working well, supervision is happening on the front line, data comes on time and is analyzed, feedback is provided, etc. There’s a limit to the assistance that Plan can give, and the MOH needs to play their role. They are implementers and we only provide technical backstopping and logistics, so the districts need to play a leadership role and take ownership of the project and its activities. The MOH at the central level stated the following: · There is a general problem with supervision as far as the final evaluation is concerned. At the beginning of the project, they should have fixed the amount of supervision that would be planned during each year. · We would like Plan to continue to help the MOH to scale up and implement IMCI all over Cameroon, applying the lessons learned from this initial 4-year pilot project. · The MOH, with Plan support, should train many more health facility staff all over the country in order to address the issue of recidivism and reorganization of staff. Plan Cameroon Child Survival Project Final Evaluation Report August 23-September 1, 2004 44 The MOH district medical officers reported the following strengths and weaknesses based on their experience over the last four years working with Plan on this CS project. Successes: · Plan’s overall image with the government has been improved through the provision of this support to the MOH · Health centers have become functional, and the MOH has been provided with training, materials, medication, motorcycles, refrigerators, skills, bed nets for the community, logistics, weighing scales, solar panels for energy, training (ANC, management, how to reinforce the capacity of the community structures and health workers, maintenance of cold chain, community pharmacies, reinforcement of sensibilization through mass media, etc.) · Management – how to manage a program from start to finish, reinforced our capacity to make work plans and action plans · Skills – the management skills learned and tools developed from this project we are able to use to help us manage our districts – all the other projects and activities · Outreach – activities, supervision, fuel, support, NIDS · Behavior change - for communities · Support – technical, managerial and financial · Ownership – a sense of pride that our district indicators are better than other districts in the East Province who are not involved in the CS project. · Partnership – contacts have helped us study through the internet and to have access to information and technical support and knowledge from the outside, how to integrate the action plans of three different organizations, and the three different districts, a sense of pride to belong to the first district to be chosen to demonstrate IMCI, how to conduct evaluations, how to analyze data, how to do focus group discussions, quarterly evaluation meetings have helped us all monitor the activities being done and plan for the future quarter, and to know if we are achieving our objectives. Motivated us and encouraged us to make more effort and to increase our activities and our level of performance quality. Weaknesses: · There was a problem in the beginning – it was not well explained to us how the project would be conducted and what our roles were, the objectives and how we should be involved. Around the MTE, things changed and we became more involved. We participated in the MTE, and subsequently in the planning activities to follow up on the recommendations of the MTE for the final two years of the project. From that time on, we were more involved in the objectives and activities of the project. · “Who was doing what” was confusing at times. There was a problem of collecting data in the field because the data was going from the community directly to the Plan staff, and it wasn’t passing through the MOH system. This left us out of the loop, and so communication was a problem. Information and data were also going directly from Plan to the community, bypassing the MOH again, and this caused problems. · Financial support from Plan often came late, after project activities had been started many times. We often had to almost write an application to get the funds. Plan Cameroon Child Survival Project Final Evaluation Report August 23-September 1, 2004 45 · After the MOH integrated their action plans with Plan, these integrated action plans were not respected or followed. · Though there was support from Plan for supervision of project activities in the field by the MOH, there should have been more support from Plan and from the MOH provincial level for these activities, such as transportation, human resources, incentives (to dialogue structures). · Plan sometimes takes decisions without consulting the MOH, and then we find out later that various activities are occurring in the field. Also, sometimes there are too many activities being executed simultaneously and it is difficult to coordinate everything. Strengthening Health Worker Performance Three main approaches were used to strengthen health worker performance; IMCI training, supportive supervision, and health system improvement through supply of basic equipment and materials. The IHFA, KPC, FGD and the personal testimonies of MOH members of the evaluation team showed that the above approaches recorded significant improvements in health worker performance. The proportion of children consulted at the health facility whose vaccination status was checked increased from 29.5 % at baseline to 96 % at final evaluation for those who underwent the 11-day IMCI training, and to 78.7 % for the untrained. The objective was to attain 80 %. The proportion of children consulted at the health facility who were assessed for all danger signs increased from 11 % at baseline to 72 % and 42.3 % at final evaluation respectively for trained and untrained staff. The objective was 50 %. The number of outreach posts in the project zone increased from 35 at baseline to 175 at final evaluation (5 fold increase) and were visited once every month by the health center nurse of the health area. Overall, a majority of the performance objectives were met and more specifically for the staff who underwent IMCI training for 11 days. This is demonstrated by the performance indicators already cited above. The proportion of sick children whose caretakers were correctly counseled about the sick child increased from 36 % at baseline to 97 % and 65.4 % at final evaluation respectively for trained and untrained staff. The objective was 70 %. Some of the best practices identified in the process of enhancing the performance of health workers included adoption of the IMCI strategy for training of personnel. The standardization of the clinical consultation of the sick child was done by using standardized consultation forms and a diagnostic and treatment guide. Counseling of the caretaker was provided for home treatment and when to return for routine follow up or in the case where the situation does not improve. To ensure health worker performance is sustained after the project is over, it will be necessary to: · Continue IMCI training for the untrained frontline staff · Continue to support facilitative supervision of health personnel after the training Plan Cameroon Child Survival Project Final Evaluation Report August 23-September 1, 2004 46 · Continue to ensure availability of essential drugs in the health facility · Render functional the referral system in order to receive and take care of cases referred by the first level effectively, and build confidence in the health system · Continue to provide assistance for initial training in IMCI, supervision, and refresher courses when necessary. · Continue to assist by providing essential equipment and materials AAPPEC through the health promoters will continue to assist the CBOs and other community based health resource persons to mobilize mothers and caretakers to carry on with health activities at the community level, as well as, reinforce the link between frontline health personnel and the community. The MOH will continue to reinforce the availability of essential drugs and other supplies. IHFA, KPC and FGD, interviews, and data review were the tools used to assess health worker performance, and they are sensitive enough to measure change over time. The gaps were assessed through the baseline studies (IHFA, KPC, FGD, document review, and interview of key informants). Activities planned to fill the gaps included training in IMCI, facilitative supervision, and provision of essential equipment and supplies. Training The project used the participatory learning and action (PLA) methodology in training community based health resource persons. The IMCI training for health personnel used lectures, role play, case studies, video show, and clinical practice at health facilities. The methods used were effective because more than 90 % of the participants improved their knowledge, skills, and practices as shown to be the pre- and post tests. All participants at the 1st 11-day national IMCI training workshop organized by MOH/WHO with funding from Plan Cameroon, UNICEF and UNFPA were successful and were awarded end of course certificates. Post training follow up supervision has also found them performing effectively in the field. The training of community health resource persons (CHRPs) was 100 % attained. The objective for IMCI was 100 % attained with respect to course curriculum and only 30 % attained with respect to the number of persons trained against the number planned. The training was effective because CBOs now carry out community growth monitoring and promotion, nutrition rehabilitation, bed net promotion and impregnation, and counseling of pregnant women to attend the prenatal clinic. Health personnel trained on IMCI are effectively practicing the approach, carrying out more outreach work, and working in collaboration with CBOs, COSA and TBAs. Their relationship with their clients is improving significantly, as demonstrated through FGD. The use of the problem tree for problem analysis, priority setting, clinical practice in IMCI training, preparation of balanced meals and weaning diets from local foods, the Plan Cameroon Child Survival Project Final Evaluation Report August 23-September 1, 2004 47 shift from the nutrition demonstration to the PD/Hearth approach in community nutrition rehabilitation, and the development of training modules and IEC materials that were adapted to the local realities are some of the best practices and lessons learned in the project. In terms of sustainability, the training modules were developed for each subject and are available and can be consulted at any moment by those who want to conduct such training. TOT was done including MOH, AAPPEC and CBO partners who can continue with training after the project. Actually, some of the CBOs are already being solicited by other NGOs and organizations to partner with them to implement projects in the health and other domains. Plan is still working in the project zone and will continue to assist the MOH and other partners based on need and available means. Trained community health resource persons were equipped and are continuing with some of their key activities. AAPPEC had started training independently before the end of the project. Now that Plan has decided to continue to partner with her for the one year bridge period, they will continue to train together with the MOH personnel and some CBO leaders. The trained CBOs are effectively training mothers and other CBOs and some of them are being hired to do mass impregnation of nets by the MOH and some local business corporations. Capacity Building Recommendations · Continue training health personnel on IMCI with preference for more nurses. · Provide training for CBOs on resource mobilization skills. · Continue to support facilitative supervision at all levels. · Provide a comprehensive orientation to the new DMO when that position is changed during the project, so the new person can join the team and come up to speed with everyone else and participate fully. · We would like Plan to teach us how to do evaluations for all of our activities so we can ensure that our indicators are the correct indictors to measure what we are doing, and how we are doing at it (where we are periodically). LQAS, and possibly other tools as well. · There needs to be better communication, collaboration and integration with MOH at the district and health facility levels. · Financial planning for the project needs to be done jointly with all project partners. Plan Cameroon Child Survival Project Final Evaluation Report August 23-September 1, 2004 48 d. Sustainability Strategy Plan Cameroon’s Child Survival Project defined sustainability as the ability of communities to sustain health improvements and enhance the health system in the absence of a project team or external funding. The project is to be judged sustainable if: 1) the Community Resource Persons are responsive to the health needs of the community and provide quality services accordingly; and 2) a strong link is maintained between the community and health facilities through monthly outreach activities, quality health services, and functioning health committees. The sustainability plan envisaged the stimulation of community participation and ownership of preventive promotional health activities, improved quality of MOH health facilities and services and the putting in place of a viable financial system through cost recovery schemes. The first two components of the strategy have been attained to a large extent while financial sustainability is still rather weak. Community participation and ownership were strongly reinforced through empowerment by training of health committees and CBO members, TBAs and the health promoters who worked in close collaboration with frontline health staff to provide essential health services to the beneficiaries though outreach services. On their own, the community based resource persons (health committees, CBOs and TBAs) are continuing to raise awareness among community members on the key household and community practices that promote and enhance child health and survival. The CBOs active involvement in maternal and child health activities is an innovation that has been initiated by the CS project and it is showing promising signs of a lot of potential for success. The project trained 120 CBOs i.e. 3,600 members, more than 90 % of whom are women (mothers of under –fives inclusive) who are directly involved in child care. This explains their interest in child health issues. Plan has already obtained funding from Plan US national office to continue with CS activities for the next one year. Apart from this, Plan will continue to provide assistance to the MOH and other organized groups intervening in the health domain in the project zone even with core funds. Plan is also submitting a new proposal to USAID under the expanded impact category, which if granted, will enable scale up of IMCI in 11 health districts where Plan is undertaking integrated community development work in three provinces of the country. Plan is assisting CBOs to engage in income generating activities such as bed net treatment and selling, increased crop production, small livestock production etc. These activities will increase members incomes and permit them to pay for health services for their children. Core funds will continue to be used to provide support for health interventions that benefit children and their mothers. A course on fund raising and project proposal development is being planned for leaders of some of the CBOs the CS project trained. The course should empower participants to design projects and leverage resources from other donors. Plan Cameroon Child Survival Project Final Evaluation Report August 23-September 1, 2004 49 The project has greatly empowered beneficiary populations on child health issues in particular, and other sectors of overall development. The CBOs, health committees TBAs, frontline health staff, health promoters and project staff had their capacity enhanced on awareness raising of health issues, and they used this knowledge to create an increased demand for health services among the population. Health services were brought nearer to the population through a reinforcement of the outreach strategy with the active involvement of health committees, CBOs and TBAs. IMCI training for some frontline health staff has improved the technical quality and human relation aspects of their services to the client, thus reinforcing confidence of the population in their health facilities and its staff. Supply of basic equipment and materials to health facilities and to trained community health resource persons has increased the demand for their services. Communities are starting to hold health personnel accountable, when they fail to respect a rendezvous for outreach activities. This was unheard of before the project started. · It was reported that some Plan program staff continue to feel that they should have been more closely involved in CS activities because once the CS project ends, they will remain in the field working with communities who hold them accountable for continuing CS activities, and this may lead to disillusionment of communities and thereby have the unexpected deleterious effect of sabotaging other Plan program activities in the field with the same beneficiary population. · The project activities should continue in order to reap the benefits of all the hard work spent over the last four years to build a foundation for better health in these communities. · The lessons learned from this project should be shared and scaled up with other projects in Cameroon. Sustainability Recommendations · The project should consider reassessing income generation activities in general, and possibly developing a business plan specifically. · Continued strengthening of supportive supervision skills by MOH staff is required. · Continued strengthening of community structures and health workers is needed. Plan Cameroon Child Survival Project Final Evaluation Report August 23-September 1, 2004 50 C. Program Management The following section of this report provides a discussion of the management issues (strengths and weaknesses) encountered during the project, within Plan among the staff, and with the project partners. The team viewed this as an opportunity to not only flush out the problems and constraints experienced in this project, but to elaborate on why they occurred and then identify plausible solutions and recommendations that could be applied to a scale-up project to ensure improved management success and increased technical impact. The Plan project staff is creative, hardworking, very loyal, and smart. In spite of any constraints experienced, they all worked long hours in a difficult environment, and managed to demonstrate very good results. They were also a wonderful team to work with throughout the final evaluation. I applaud each and every one of them! 1. Planning Planning was done at various levels throughout the project, as described below. The planning design is well thought out and realistic. At the national level, there was a project steering committee that was scheduled to meet twice a year. This committee met only twice during the life of the project because it was convened and chaired by the minister of health whose time was limited. However, Plan felt that it was a terrific advocacy tool to have the minister of health involved in and knowledgeable about the CS project, and also engaged and participating with the project partners when time permitted. Members included the minister of health and his close collaborators, Plan (country director, domain coordinators, members of the country management team, CS project staff and Plan program area manager), AAPPEC, the 3 DMOs, the provincial delegate of health for the East Province, WHO, and leaders of a few CBOs. This committee was supposed to provide institutional support for the project, and to work on any policy issues that needed to be addressed. In reality, this committee allowed for horizontal communication which helped facilitate the project, but as it only met twice, it didn’t serve the original purpose it was designed for. · Since Plan is demonstrating IMCI in Cameroon for the first time, it’s a bit of a missed opportunity to not have had more consistent collaboration at the policy level through these meetings. During the project, the MOH was in the process of adapting IMCI to the country, and this caused considerable delays in project implementation both in depth and breadth. However, Plan was very flexible and worked with the MOH on IMCI adaptation (materials and training), and implementation (at the facility level). In the future, it might be more effective to have this meeting convened by a delegate of the Minister of Health, if possible. At the provincial level, members included the DMOs (of the three project areas), provincial delegate of health, the project coordinator and team, program area manager Plan Cameroon Child Survival Project Final Evaluation Report August 23-September 1, 2004 51 and national health advisor from Plan, the head of AAPPEC, and the heads of the Presbyterian and Catholic health services, and a representative from the social insurance dispensary in Bertoua, and the presidents of the 3 district health committees. Quarterly reports were presented by all participants to share project activities and what has been done, what difficulties were encountered, and planning for the next three months. In the life of the project, 12 out of 16 meetings took place. This meeting was convened and chaired by the provincial delegate of health. This meeting was useful for drawing the attention of the DMOs to the project activities in their districts, and for appropriate management of project supplies and equipment in the field. Joint supervision by all partners was weak during this project, and these meetings were an attempt to help rectify the situation. · For future meetings, it might be more helpful to focus on joint supervision in the field, as well as implementation of activities, since this was an issue that was raised during the final evaluation. At the district level, there was a monthly coordination meeting between the DMO, the health center head nurses, Plan (health supervisors and assistants, and the field coordinators), AAPPEC (technical supervisor of the health promoters and the health promoters themselves), and other project staff. During this meeting, project objectives were addressed and discussion ensued on which activities had been achieved and what still needed to be done. Planning was also done for the following month. Subsequently, all of these individual monthly plans were then integrated into their individual organization monthly work plans. Within Plan, the health supervisors then followed up on the monthly work plan. Initially these meetings did not occur regularly, but during the third and fourth years of the project these meetings took place at least 8 out of 12 months. These meetings were effective in ensuring that duplication of efforts was avoided. There was also consensus on outreach collaboration by project partners. Health messages for each month were discussed and decided jointly. Resources and logistics were also coordinated to ensure smooth operation of field activities. Feedback from these meetings reinforced the CBO, TBA and COSA members. At the health facility level, no formal regular coordination meetings were scheduled. The CS project had mandatory monthly coordination meetings for all project staff. It was reported that activity reports were presented and discussed, and proposals were made on how to move forward. Monthly action plans for the coming months were also presented and validated. Following the meeting, a synthesis was done by the project coordinator and this was forwarded to the country office. The field coordinators were invited on a regular basis, and one member of AAPPEC also attended these meetings. · The project coordinator has excellent planning skills, and used them to the best of his ability to ensure that all project activities occurred as planned, on time, and demonstrated impact. Using this management style, the project demonstrated significant impact for nearly all indicators. Plan Cameroon Child Survival Project Final Evaluation Report August 23-September 1, 2004 52 However, project staff interviewed during the final evaluation reported that the top-down management style of the project leadership made them feel that they were not really included in project planning or project management, but rather were given assignments and then told to execute them and report on the outcome. After providing their reports, they were given another assignment. It was reported that staff meetings were mostly conducted using a lecture style, and that very little discussion took place. They didn’t feel as if they had any decision-making authority. The health promoters (hired by AAPPEC) were represented in these planning meetings by their supervisor, although they were the front-line staff implementing all activities at the community level. · For future project management, it would be more helpful to use a participatory methodology that really included all the project staff in the planning and management process. This would increase morale, help coordinate activities in the field, decentralize the decision-making authority, make the staff more efficient, and build their capacity at the same time. It was reported that AAPPEC also held monthly project meetings, but that Plan was represented by the health promoter supervisor, who is an AAPPEC employee, in those meetings. · For future project planning, it is important to have a regular project meeting that includes all of the project partners. Having separate meetings led to miscommunication and confusion in the field. These issues negatively impacted the project. The overall Plan Program held monthly and/or quarterly meetings, during which the CS project staff participated. · It was reported that this activity was very helpful for both the Plan program and the CS project staff. After the CS MTE, all Plan staff began working more closely together and sharing information on a more regular basis, which “improved things considerably”. Collaboration between the CS project coordinator and the Plan program area manager occurred on a regular basis. Key issues were addressed by both, as they were jointly members of the country management team. A direct link existed between the CS project coordinator and the country director, and a positive working relationship was maintained throughout the life of the project. The DIP for this project was pretty realistic, and most activities were carried out according to the annual plans. The DIP preparation activities were adequate for this project, and no further suggestions are being made to amend it. The gaps in the DIP included two technical areas, pneumonia and HIV/AIDS, that were not originally included in the DIP but were integrated with this child survival project and funded from other sources. Hand washing was also added during the DIP review. Some Plan Cameroon Child Survival Project Final Evaluation Report August 23-September 1, 2004 53 of the indicator targets were also raised after the DIP review process. Logistics were increased after the MTE review, as well as the number of health promoters. Planning Recommendations · If possible, the project steering committee at the national level should be convened by MOH HQ but not the person of the Minister of health. · The agenda for provincial level project management meetings needs to include the issue of supervision. · The project should consider using a participatory method that includes all the project staff in the planning and management processes. · Regularly scheduled (weekly/monthly) project planning meetings need to include all project partners. Plan Cameroon Child Survival Project Final Evaluation Report August 23-September 1, 2004 54 2. Staff Training All project staff have increased their capacity during this project, specifically on the technical areas of malaria, particularly having to do with ITNs, PD hearth for malnutrition, the entire strategy of IMCI, HIV/AIDS prevention of MTCT, and work on OVC. The team learned to use many evaluation tools including the KPC, LQAS, HFA, FGD and home visits, and PLAN management tools. PLA methodology was also taught, along with TOT and behavior change communication techniques. The staff is fully engaged in using these tools in their daily work. Adequate resources were dedicated to staff training. Building the capacity of project staff permitted them to all be more functional and productive in their roles. This also increased the motivation of the staff, as they are more confident doing their jobs, and thus this contributed to the project achieving its targets. However, the staff felt that they were not able to increase their capacity as much as they would have liked to, and had the project management been more decentralized they could have learned and grown much more. The MOH called upon Plan to support and provide assistance on staff training at various levels, and the contacts that the MOH gained through this partnership allowed them to have internet access to study online, and to access information, knowledge and technical support from the outside. However, the DMOs requested that Plan continue to provide training and support by teaching them how to do evaluations for all of their activities, and to help them ensure that their indicators are appropriate for measuring what they are doing. They also wanted to learn how to use LQAS as well as other monitoring and evaluation tools. AAPPEC had a mixed experience with staff training through this partnership. While the nine health promoters (hired for the CS project exclusively) gained tremendous capacity through frequent trainings of all kinds, the regular AAPPEC staff did not benefit as profoundly, and thus it was felt that much of the gains made during this project partnership would be lost if AAPPEC failed to retain the nine promoters. There was a lot of pressure from community groups requesting training from Plan, even in geographic areas that were not included in the project, as Plan had increasing capability and demonstrated increasing effectiveness in the field. Plan Cameroon Child Survival Project Final Evaluation Report August 23-September 1, 2004 55 Staff Training Recommendations · The project should consider a decentralized management system in the future to amplify staff growth through hands-on training and experience. · In future partnerships, Plan should consider training regular partner staff to help build the capacity of its partner organizations and ensure sustainability after project completion. · Plan could provide additional training for MOH leadership staff on monitoring and evaluation tools. Plan Cameroon Child Survival Project Final Evaluation Report August 23-September 1, 2004 56 3. Supervision of Program Staff The health supervisor supervised the assistant health supervisors. The assistant project coordinators supervised the health supervisors. The project coordinator supervised the assistant project coordinators. The national health supervisor and the program support manager supervised the project coordinator. The technical backstop from Plan USA provided ongoing technical support throughout the project. The supervisory system was logical given the structure of the project however it was not always responsive to the needs of the project staff. Many of the field staff stated that they were not routinely supervised in the field and that they would have felt much more supported had there been routine supportive supervisory visits for the community level work. While Plan has an institutionalized supervisory system that is fully functional, with checks and balances at all levels, the CS project staff did not feel that they had much recourse to effect change/decentralization of the project management structure. The supervisory system was felt to be inadequate for project management at all levels and for all project partners including the MOH and AAPPEC. Project Staff Supervision Recommendations · Plan may wish to consider implementing routine organizational assessments at the program level to ensure that management systems are functional, effective and progressively addressing staff needs. · Capacity building for supportive supervision should be built into each new project. Plan Cameroon Child Survival Project Final Evaluation Report August 23-September 1, 2004 57 4. Human Resources and Staff Management All Plan projects in the East program area work as a program team with communities and other partners. Plan Program activities in the East program area include HACI, Child Survival, WATSAN, Rights of the Child, Sponsorship, Learning, Habitat, and Livelihood. These activities are coordinated by Plan staff in various areas, with Domain Specialists coming from Yaounde Country Office to provide specific training for technical support to the field program staff, and training for CBO leadership, who then do TOT for their CBO members. Supervision of CBOs is done by Plan field staff periodically as needed depending upon the specific activities being done in the field. Plan ensures that all essential personnel policies and procedures are in place for every provincial office and in all projects. While the CS project will end imminently, Plan will remain in the project area as they currently have several other programs that will continue into the future. Therefore, in future projects, all Plan field staff should be actively aware of all Plan projects and their activities in order to be able to effectively follow up with communities when they are in the field, and particularly when one project ends and the community raises questions about continuing project activities. This has become an issue with the current CS project. At the same time, staff workload should be periodically reviewed to ensure that there is equity, and that staff do not become overwhelmed with too much work. For example, the M & E staff person for the CS project was taken away from the data system part time to do training of the COSA and CBO in the field while he remained accountable for a full time M & E position. Also, the nine health promoters covered 35 villages each on a monthly basis, ensuring supervision of COSAs, CBOs, and home visits, facilitating outreach activities, health education sessions, and meetings, and participating in trainings and data collection activities, which they admitted was humanly impossible to do. Many of the other project staff also felt overwhelmed with their workload, and that they could not do it all well, so they did all of it at less quality because they felt spread too thin on the ground. The project coordinator stated he had an open door policy throughout the life of the project however, his “top-down management style” prevented many staff members from feeling comfortable approaching him. The staff reported that they helped each other in the field in order to get the work done, which they were all very committed to doing, and that they liked each other and worked well together. They did state that it was difficult to work in an environment where they did not feel empowered. The staff are all extremely hard working, loyal and dedicated to helping the people of the East Province in Cameroon, and it was therefore felt that the difficulties they experienced in their work environment did not significantly effect project implementation. Both the MOH and AAPPEC have gained considerable ability to manage their organizations effectively and efficiently. However, they both reported that they continue to require further assistance in order to further improve their management systems. Plan Cameroon Child Survival Project Final Evaluation Report August 23-September 1, 2004 58 Regarding staff turnover, the project had some initial instability for the M & E position, however this was stabilized in the second year of the project. No other significant staff turnover occurred, and thus the project ran smoothly with the project team remaining together from the beginning to the end of the project. The CS project has obtained additional funding to continue activities for one more year. Plan is also submitting a CS proposal in the expanded impact category and should they be awarded another grant in Cameroon, then the project staff would be retained to continue working on this new project. However, after the project ends, the staff will be able to transition to other employment due to the local demand for training, and the capacity built by this project, which has left most project staff in high demand for other jobs both within Plan and with other local organizations. While the health promoters stated feeling uncertain about their future once the CS project ends, one has already been hired by Plan as the HACI supervisor, and AAPPEC will retain the rest of them to continue with Plan during the one-year bridge period. HR and Staff Management Recommendations · For future partnerships, significant emphasis needs to be placed on building the management capacity of the partner organizations. · Plan may wish to consider adopting a more participatory management method that is responsive to staff needs, and respects a more integrated planning process. · Project management should be decentralized and democratic with all partners contributing their ideas, experience and solutions for problems encountered in the field. · Project staff should have ownership over their roles and responsibilities, and should be given the authority to do their jobs effectively, respecting the organogram. · Staff should be replaced when positions become vacant. · Increased transparency is needed to ensure that all staff are part of the phase-out, transition, and sustainability discussion. · All Plan field staff need to be remain actively aware of all Plan projects and their activities. · Workload should be assessed periodically, and adjusted to the realities of each position and the field work required. Plan Cameroon Child Survival Project Final Evaluation Report August 23-September 1, 2004 59 5. Financial Management The CS project used their funds adequately, and expenditures matched closely with the project budget. All funds will be exhausted by the close of the project. A problem was encountered when the value of the US dollar fell in 2003, resulting in increased costs and lower returns in the exchange rate fluctuation, but Plan absorbed the extra costs with no additional expenses charged to the CS project. The assistant field supervisor sent monthly financial reports to the field supervisor who then sent them to the Project Coordinator and the project accountant. The project coordinator and the accountant reviewed the financial reports and then submitted monthly financial reports to the Operations Support Manager in the national office. The program tracked costs incurred using the already established financial system of Plan/Cameroon. Expenditures were broken down using specific codes for labor, equipment, supplies, and facilities. Project expenditure reports were sent directly from Plan/Cameroon to the US office of Plan where they were reviewed for USAID compliance and submitted to USAID. Plan’s corporate general ledger system assigns a project ID number to enable accurate tracking of project expenditures related to each project. Plan’s organizational finance system also enables Plan/USA to separately track donations from USAID and non-USAID sources. After the US office submitted reports to USAID, and received the re-imbursement, funding transfers were made on a monthly basis from the US office to the Child Survival project to reimburse project expenditures. It was reported that funding between the CS project and the East Province Plan program seemed to get mixed up and caused confusion in financial management. The Plan program sometimes spent CS project funds thinking they were program funds, leaving the CS project short on cash until additional funding arrived from the Plan country office. Plan should consider establishing a separate bank account for CS project funds in order to maintain good accounting balances. The project worked with the partner organizations to ensure monthly financial reporting on project funds spent through these organizations. Plan transferred funds to AAPPEC’s bank account based on a collaboration agreement signed together. Plan audited AAPPEC CS project accounts on a semester basis before disbursing more funds. Funds provided to MOH partners were justified after the activities had been implemented and an activity report submitted along with the justification documents. The CS project has obtained additional funding to continue CS activities for one more year. Plan also has separate funding to continue the other programs that were being implemented in the same province simultaneously. Within Plan and outside of Plan, technical support was provided for financial sustainability, planning and audits. The CS project had four audits during the life of the project, and all were successful with only minor issues raised and resolved. There were no major findings. Plan Cameroon Child Survival Project Final Evaluation Report August 23-September 1, 2004 60 AAPPEC Through the capacity built by this project, AAPPEC was able to leverage additional funding from the European Union to continue some project activities. However, AAPPEC also stated that they were not included in budget discussions to help determine the overall level of funding they would require to implement the agreed upon activities during the life of the project, but rather were told how much they would receive They also felt excluded from budgetary discussions around specific training and field activities. AAPPEC felt that they were not provided with assistance to help them strengthen their organization overall. For instance, the organization was not provided with a computer, a printer or any other resources that would remain with the organization to help strengthen their office capacity after the end of the CS project. It was reported that only three motorcycles (3 years old each) which were used by the health promoters were given to AAPPEC at the end of the project, and that these motorcycles will cost AAPPEC more money in repairs and maintenance than they will help alleviate transportation constraints. The health promoters stated that they received a lot of training, tools and resources that they benefited from, and that this support helped them go out and do their jobs better. They learned how to work with communities, about gender equity, management, supervision, social mobilization, capacity building, they built their knowledge and skills in maternal and child health, and in general increased their confidence. They learned to trust and respect their community counterparts, even if they are poor, and to help them problem-solve and not just hand out medicine. They all felt that they were much more open-minded now, and that there is no way they could pass by a child in need and not help. They gained a tremendous amount of awareness through this project, and they will continue to use it and build on it in the future. They were the front line field operators doing much of the work for the project, and whatever gains the project made at the community level were due to the work of these health promoters. The MOH wanted to take this opportunity to thank Plan for providing technical, financial and material support to begin the implementation of IMCI in Cameroon. In particular, they acquired motorcycles, refrigerators, bed nets for the community, logistics, weighing scales, solar pans for energy, and training (ANC, management, how to reinforce the capacity of the community structures and health workers, maintenance of cold chain, community pharmacies, reinforcement of sensibilization through mass media, and other issues) . The DMOs have begun working with the MOH to try and facilitate more consistent funding and distribution of supplies, equipment and drugs for health facilities and for outreach activities. However, they too felt that they were not included in budget discussions with Plan regarding the activities that they were involved in. They stated that financial support from Plan often came late, many times after project activities had been initiated, that they “often had to almost write an application to get (their) funds from Plan”. Plan did support the MOH significantly but many more needs within the MOH became apparent. Plan Cameroon Child Survival Project Final Evaluation Report August 23-September 1, 2004 61 Financial Management Recommendations · Additional focus on financial management is needed with project partners. · Establish a separate bank account for future CS projects. · Issues of equity between project field staff should be discussed and resolved jointly. · All project partners need to be involved in budget discussions and decisions. · All projects and activities within Plan should harmonize all incentives to communities in order to minimize conflict. Plan Cameroon Child Survival Project Final Evaluation Report August 23-September 1, 2004 62 6. Logistics There has been good collaboration on logistics between the Plan CS project and the other Plan program activities. This includes vehicles, as the CS project had 2 cars which they shared with the other program activities. The CS project did not have a photocopier, so the Plan program shared theirs with the child survival project. Printers were also shared, as well as other equipment and supplies. Transportation was not an issue for project activities in general. We have not had a problem with the means to purchase the supplies and equipment needed for project activities, but there were very long delays in acquisitions due to buying from manufacturers abroad, delays at the seaport, and clearance by customs and other ministries (Ministry of Environment needs to authorize purchases of refrigerators, etc.). Logistics Recommendations · Logistics should be pre-positioned for project staff and partners in order to begin using them at the beginning of the project, to not lose time, and to have a greater impact. · All project staff should be involved in the discussion of logistics requirements and processes. · The project needs to budget for very essential equipment and spare parts and have them pre-positioned so that when things break they can easily be fixed and not negatively impact project activities (ex. wicks and globes for refrigerators). Plan Cameroon Child Survival Project Final Evaluation Report August 23-September 1, 2004 63 7. Information Management The project had a very comprehensive monitoring system that kept track of all indicators using both quantitative and qualitative means. The data was in both EXCEL (for project monitoring) and in EPI Info (for HFA and KPC baseline, MTE and FE). Data collection, compilation, analysis and feedback was done at all project levels. Community data was collected using forms for CBOs, COSAs and TBAs on a monthly basis, and one form from the MOH health facilities was compiled monthly, containing data on all the technical intervention domains of the project except HIV/AIDS. Plan collected copies of all these monthly forms and entered the data into the project monitoring system, which was then analyzed and used for project monitoring and activity adjustments as required. It was also discussed at monthly and quarterly project meetings with project partners, and then used to make decisions on outreach activities, food demonstrations, etc., by the health center staff in coordination with the community workers during outreach activities. Several issues arose: 1) only 40-50% of the monthly community forms were being returned from the field, and so the data set was never complete, making it difficult to rely on the analysis with a strong degree of certainty, 2) while the system and forms were developed jointly by all project partners, with everyone receiving training on their use, the majority of the field staff and communities were not included in the use of the data system from the start of the project, but rather phased in over time, making it more difficult for everyone to have ownership over the data, the results, and the need for periodic changes in the project approach and activities. 3) the project experienced a lot of problems with computer maintenance, without proper repairs, thus slowing down the work considerably, 4) the health promoters who were the front line workers in this project did all the data collection for the midterm LQAS and final KPC, but they were not invited to participate in the data analysis or subsequent discussions Previously, all the data was coming to Plan and bypassing the MOH, but in the final year of the project the system was revised so that the forms were compiled at the health center level and then sent to the district for further compilation, and then shared with Plan. Plan has been working with the MOH to strengthen their ability to acquire data, compile results and then analyze them for project monitoring. However, the health facility staff remains weak in this area in terms of skills, transportation, motivation, etc. There will need to be a tremendous focus on this if it is to become truly sustained by the MOH staff. The project staff and partners have been effectively using their data to some extent, and there has been a real effort to train communities to also use their data. However, throughout the project much of the data analysis and use had been somewhat superficial. It was clear that by the end of the final evaluation, all involved partners clearly understood the gains the project had achieved. Several staff and partners admitted that they now had a much deeper understanding of how to probe into the data for analysis, and to ask probing questions to ascertain a much better understanding of how things are really working (or not), and how to make realistic decisions to correct them. This will be extremely important for scale up activities for IMCI throughout the country. The gains and lessons learned from this final evaluation have already begun to be shared with the Plan Cameroon Child Survival Project Final Evaluation Report August 23-September 1, 2004 64 MOH IMCI task force at the central level, UNICEF, WHO, the US Embassy, and several other organizations who are all involved in IMCI implementation and scale up in Cameroon. The program used the following evaluative tools to conduct special assessments: · LQAS was used during the MTE to assess vaccination coverage · A mini-survey was done to select CBOs for training on household ITN possession and use · Rapid catch was used for development of the proposal for a cost extension · PDI was employed to assess the prevalence of malnutrition in some villages, as a prerequisite for P/D Hearth training PLA was used with women’s groups to analyze behavior change interventions for malaria, malnutrition, diarrhea and vaccinations. LQAS was used to assess vaccination coverage at the MTE. An organizational assessment was conducted (baseline and final) for AAPPEC by an external consultant. PDI was used to assess the prevalence of malnutrition in order to implement P/D hearth sessions in two villages. Mini-surveys were employed to select CBOs to be trained on household possession and use of ITN. To assess the level of attainment of rapid catch indicators, a survey was done in order to develop the cost extension proposal, which subsequently was not funded. Information Management Recommendations · Further strengthen the MOH to collect, compile, analyze and use their data. · All project partners should be involved in data analysis and use from the inception of the project. · Revise the project monitoring system, collect only the data that can and will be used on a monthly or quarterly basis, and then phase it into the MOH system. · Revise the monitoring system tools and forms to follow the MOH system and create ONLY those forms necessary to fill the data gaps. · Data should be deeply analyzed and used at each level on a monthly basis. · Provide computer software training for all data tools and systems. · Assist the MOH in developing a community based monitoring system for C-IMCI. · The COSA should be trained to ensure that all forms are completed and compiled on a monthly basis, and the data sent to the health facility. · Provide sufficient numbers of computers with regular maintenance and repair support. Plan Cameroon Child Survival Project Final Evaluation Report August 23-September 1, 2004 65 8. Technical and Administrative Support The Plan regional health advisor and the regional program support manager both came to visit the project twice and offered technical support and assistance. The first visits of the two officers were contact visits to Plan Cameroon when they were newly hired at WARO. The CS project being the major grant funded project that Plan Cameroon was implementing, they both paid a working visit to the East program area and visited some of the project activities in the field. The regional health advisor joined the reproductive health advisor from the USNO. They visited trained TBAs in Mpelepanga in Diang. He also paid a working visit to a Baka women’s CBO promoting ITN use as well as participated in outreach activities conducted by the nurse from the Petit Pol Health centre and the AAPPEC health Centre in Mayos village. He visited Grand Mboulaye health centre and had discussions with the health personnel proposing how they could improve on service delivery for children and mothers and the population as a whole. His second visit was during the MTE during which he backstopped the strategy for social promotion of ITNs and also held discussions with Plan Cameroon management on the hosting of the Regional health advisors meeting in Bamenda, Cameroon. The regional program advisor on his first working visit to the East Program Area visited GUH activities including CS field activities. He visited the pygmy housing project in Mayos and Nkolbikon and the Baka women, CBO, ITN and growth monitoring activities. He also visited Moundi health centre and had discussions with the staff on problems they were facing, the project assistance they were receiving, and made proposals on how to ameliorate the working conditions to enable field staff to provide quality health services for children and their mothers. Assistance that the project felt was needed but not available and could be better planned for by USAID or Plan HQ included providing additional funding for staff to attend international meetings and to participate in distance learning activities for staff development and career enhancement. The project coordinator was flooded with email information on a regular basis from many technical advisors within Plan, and so access to information at that level did not seem to be a problem. The technical backstop visited the project bi-annually throughout the life of the project, and provided email support and technical resources continuously and upon special requests. Technical and Administrative Support Recommendations · The technical backstop and the technical advisor should both continually assess management issues in the field during routine monitoring visits. Plan Cameroon Child Survival Project Final Evaluation Report August 23-September 1, 2004 66 9. Management Lessons Learned This was the first ever grant that Plan Cameroon received, and it was managed vertically. It was a challenge to constantly dialogue and negotiate with all Plan Programs and staff, as well as project partners. Being open and in constant communication helped the project become successful and respond to various issues, situations, challenges and successes. A more participatory approach toward management would have helped the project achieve even higher results and more impact as staff would have been more organized and the work more coordinated. Project partners also would have benefited more directly had there been a more integrated approach toward project management. Plan Cameroon Child Survival Project Final Evaluation Report August 23-September 1, 2004 67 D. Other Issues Identified By The Team All of the issues identified during the final evaluation have been thoroughly discussed throughout this report. No additional issues were raised or identified. Plan Cameroon Child Survival Project Final Evaluation Report August 23-September 1, 2004 68 Conclusions and Recommendations Objects, Outcomes and Success Everyone who participated in this project, and all those involved in the final evaluation collectively felt that the project had gotten off to a slow start. However, a review of the data shows that in spite of the time taken for the project to establish itself, tremendous gains were achieved throughout the life of the project, and while there is still room for continued growth, everyone was surprised at how far they had actually come. Plan managed to meet or exceed nearly every project indicator target by the final evaluation of this project, as demonstrated by a review of the data tables contained in the body of this report. Plan Cameroon successfully piloted IMCI in one district of Cameroon, leading the effort to introduce IMCI to the country, with the intention by Plan and the MOH (and UNICEF, WHO, HKI, PSI and others) to scale up IMCI throughout the entire country based on the results of this project (as well as the other two pilot districts carried out by UNICEF and WHO). Plan intends to submit another CS proposal, this time under the expanded impact category to scale up IMCI in four provinces in the country as part of a nation-wide scale up campaign being co-funded by several donors. The project staff witnessed a dramatic shift in the thinking and behavior of mothers /community members and health facility staff as they became more open to participation and learning. This motivated everyone to work even harder to amplify the incredible changes they knew the project was facilitating in an inhospitable environment plagued with poverty and disease, and in the face of tremendous hardship. They stated that the project has been very challenging, but also very rewarding! Achievements, Constraints and Other Factors Plan’s IMCI intervention in this child survival project, which began in 2000, preceded the government roll-out of IMCI in Cameroon by two years. Initially, Plan trained all of the 46 health facility staff working in the three project districts using a 6-day IMCI training, which was later to be followed up by the standard 11-day IMCI training recommended by WHO. The project then began working with the health facility staff to support them in fully integrating IMCI into their daily practice at the health centers in all three districts. Two years later, in 2002 the MOH initiated the pilot IMCI program in Cameroon, selecting three districts to begin with – Doume District in the East Province with Plan, Ngaoundere District in the Adamaoua Province with UNICEF, and Eseka District in the Centre Province with UNFPA (who eventually dropped out due to funding constraints, and WHO has assumed responsibility for this district). Plan Cameroon Child Survival Project Final Evaluation Report August 23-September 1, 2004 69 Thus, half way into this child survival project, Plan was required to redesign its IMCI intervention and scale it back to only one district (Doume District), which was selected by the MOH to participate in the IMCI pilot program for Cameroon. This situation greatly impacted Plan’s ability to demonstrate substantial success in IMCI implementation in the East Province. However, in spite of the situation, Plan did manage to demonstrate significant impact, and with a bridge-year ensuing upon the end of this project, they are certain to achieve additional results which will greatly impact the sustainability of CS project activities. Best Practices and Lessons Learned As this was Plan Cameroon’s first CS grant, they had a tremendous learning curve to overcome, which they successfully demonstrated with significant results and tremendous potential for nation-wide scale-up. Most importantly, these include the need for both a participatory process with all involved parties, and within the CS project management. The use of data during the final evaluation provided a more in-depth exploration than the project had previously accomplished using their data set. They increased their capacity to understand how to deeply explore the activities, accomplishments and constraints using the information that was routinely being collected by their HIS, and this will help them to make significant changes and improvements for future projects. The need for policy reform at the MOH level became glaringly obvious during the final evaluation, and it was very exciting to have all project partners commit to assisting the MOH to rectify this problem and move forward quickly to improve the situation and position Cameroon for successful IMCI scale-up. Lastly, the need for much closer supervision at all levels became a major theme during the final evaluation, and all project partners agreed that supportive supervision needed to be greatly increased for IMCI scale-up throughout the country. PVO HQ Use of Best Practices With learning out of this project Plan HQ has continued to build a fund of knowledge and expertise that it is sharing across Plan countries and regions through existing technical networks and the official Plan health newsletter-Sante Salud. Plan HQ has acquired a wealth of technical and grant management skills and will be able to transfer these skills to other child survival projects that are currently implemented by Plan globally as well as provide technical guidance and comments on development of Plan private funded CS proposals. In order to fully realize child centered community development HQ will expand and work beyond central USAID funding. Opportunities exist to market and liaise with USAID mission in-country to access donor funds. Plan Cameroon Child Survival Project Final Evaluation Report August 23-September 1, 2004 70 Potential for Scale-Up This project has tremendous potential for scale up to the entire country. Plan piloted IMCI in one province in Cameroon as part of a national pilot program, with the intention of using the experiences from the pilot for national scale up. Plan is now in the position of assisting the Government of Cameroon in expanding IMCI to the entire country based on the lessons learned from this pilot project. Additional donors (WHO, UNICEF, and others) have already pledged their support, both financial and technical, for the scale up of IMCI in Cameroon, and with Plan playing a pivotal role in leading this expansion, based on their experience and expertise, Cameroon is sure to benefit in the long run. This approach will surely be successful if all partners continue to collaborate closely on the issues raised from this pilot project, addressing the policy constraints (turnover of facility staff), and emphasizing a strong focus on community involvement. Plan Cameroon Child Survival Project Final Evaluation Report August 23-September 1, 2004 71 Recommendations IMCI 1. The 11-day IMCI training is needed for all staff (doctors and nurses) and supervisors at all health facilities. 2. IMCI-trained (health facility and supervisory) staff should be required to remain in their positions (MOH policy change) for a given period of time (ex. 2 years), and should not be moved within that time period. 3. Health districts need to take ownership of IMCI in order for it to be successfully implemented. 4. Intensive, integrated, facilitative supervision for IMCI needs to be carried out at all levels including project partners, health facilities, community health workers and structures, and households. 5. Routine evaluation (ex. Quarterly) should be done alternating between quantitative (ex. LQAS) and qualitative tools (ex. Focus group discussion) to spot check the impact of health education activities and behavior change interventions. 6. Preposition supplies and commodities at the village level. 7. The 16 key messages for IMCI should be further disseminated to schools (to affect children and the teenage pregnancy cohort) and through mass media (using local languages). 8. Develop a strategy for accommodating mobility of mothers and children, and their vaccination cards. 9. Grandmothers should be included in the target population for health education, and the CBO and COSA should recruit grandmothers as members wherever possible. 10. Carry out planned operations research on cultural barriers to healthy behaviors. 11. Provide routine maintenance for all equipment and supplies to keep activities operational. 12. Consider developing a multimedia marketing strategy to increase utilization of health services and staff. 13. Develop a fully functioning referral (and counter-referral) system to be put into place and maintained at all levels, with defined follow up actions. Plan Cameroon Child Survival Project Final Evaluation Report August 23-September 1, 2004 72 14. Review the current fee system at health centers, and ensure that treatment and drugs are available for those unable to pay. 15. Consider incorporating traditional healers, kiosks and local pharmacies into project activities regarding appropriate drug management and dispensing. Malaria 1. Significant emphasis should be placed on re-treatment activities for future projects. 2. Project partners need to make a consistent and unified decision regarding the sale vs. free distribution of bed nets. 3. Project indicators need to be expanded for bed nets and re-impregnation to include issues of knowledge, access and usage. 4. A focus on intermittent presumptive treatment of pregnant women should be included in future projects. 5. A focus on iron/folic acid supplementation during pregnancy should be included in future projects. 6. Knowledge (dosage, frequency) and access to drug treatment should be expanded and include street vendors, kiosks, traditional healers and community health workers for expanded home-based treatment. Diarrhea 1. Additional focus on hand washing should be included for scale up activities, specifically on increasing access, reorienting belief systems, and improving habits. 2. Explore possible solutions for soap access including micro-enterprise for home￾based production and community sales. 3. Reinforce and strengthen BCC messages for refuse disposal, latrine construction and usage, water treatment, and food storage. 4. Expand ORS access and usage at the household level. Plan Cameroon Child Survival Project Final Evaluation Report August 23-September 1, 2004 73 Immunization 1. Increase outreach activities for vitamin A for children (6 months to 5 years) and TT for pregnant women. 2. Improve immunization record keeping (cards, registers, etc) and communication. 3. Revisit how to measure coverage rates (other tools, records, etc). 4. NIDS should be recorded on immunization cards and health center registers. 5. Refresher courses on immunization should be implemented for health facility staff. 6. Mothers should be taught to bring immunization cards to the health center when coming for consultation or treatment. Malnutrition 1. Ensure that CBO coverage is provided for at least 80% of communities. 2. Health cards should be freely distributed by CBO/TBA/COSA to all children 0 – 5 yrs without a card 3. Intensify supervision and mobilization of CBOs engaged in rehabilitating malnourished children (and record keeping). 4. Change the malnutrition strategy from nutrition demonstrations to PD/Hearth and scale up to all project areas. 5. Address cultural issues around food/feeding, breastfeeding and fluids. 6. Integrate vitamin A distribution with growth monitoring sessions. 7. Increase training of health center staff, community structures and community health workers on nutrition rehabilitation. 8. Develop a marketing strategy for PD Hearth, and ensure that community members understand that free food will NOT be distributed. Community Mobilization 1. Train more than one CBO per community, and include as many grandmothers as possible. Plan Cameroon Child Survival Project Final Evaluation Report August 23-September 1, 2004 74 2. Increase supportive supervision of community structures for community mobilization activities. 3. Strengthen the relationship between the community structures and the health facility staff. 4. Provide a badge and perhaps a ceremony to recognize and motivate the community health workers and structures. 5. Introduce peer reviews of community health workers in the villages. 6. Reorganize and strengthen the relationship between all the community-based structures and partners. Communication for Behavior Change 1. Grand mothers should in the future be targeted in BCC training. 2. From time to time conduct surveys (FGD, LQAS, Rapid Catch) to ascertain if IEC messages are being effectively put into practice. 3. Health districts should avail themselves of existing rural radio stations to disseminate key messages on child and maternal health. 4. Continue to multiply and use the IEC materials developed during the life of the project. 5. Reinforce the application of innovative approaches such as child-to-child, child￾to-parents, and peer-to-peer. Capacity Building 1. Continue training health personnel on IMCI with preference for more nurses. 2. Provide training for CBOs on resource mobilization skills. 3. Continue to support facilitative supervision at all levels. 4. Provide a comprehensive orientation to the new DMO when that position is changed during the project, so the new person can join the team and come up to speed with everyone else and participate fully. Plan Cameroon Child Survival Project Final Evaluation Report August 23-September 1, 2004 75 5. We would like Plan to teach us how to do evaluations for all of our activities so we can ensure that our indicators are the correct indicators to measure what we are doing, and how we are doing at it (where we are periodically). LQAS, and possibly other tools as well. 6. There needs to be better communication, collaboration and integration with MOH at the district and health facility levels. 7. Financial planning for the project needs to be done jointly with all project partners. Sustainability 1. The project should consider reassessing income generation activities in general, and possibly developing a business plan specifically. 2. Continued strengthening of supportive supervision skills by MOH staff is required. 3. Continued strengthening of community structures and health workers is needed. Program Planning 1. If possible, the project steering committee at the national level should be convened by MOH HQ and not the Minister of Health. 2. The agenda for provincial level project management meetings needs to include the issue of supervision. 3. The project should consider using a participatory method that includes all the project staff in the planning and management processes. 4. Regularly scheduled (weekly/monthly) project planning meetings need to include all project partners. Staff Training 1. The project should consider a decentralized management system in the future to amplify staff growth through hands-on training and experience. 2. In future partnerships, Plan should consider training regular partner staff to help build the capacity of its partner organizations and ensure sustainability after project completion. Plan Cameroon Child Survival Project Final Evaluation Report August 23-September 1, 2004 76 3. Plan could provide additional training for MOH leadership staff on monitoring and evaluation tools. Project Staff Supervision 1. Plan may wish to consider implementing routine organizational assessments at the program level to ensure that management systems are functional, effective and progressively addressing staff needs. 2. Capacity building for supportive supervision should be built into each new project. Human Resources and Staff Management 1. For future partnerships, significant emphasis needs to be placed on building the management capacity of the partner organizations. 2. Plan may wish to consider adopting a more participatory management method that is responsive to staff needs, and respects a more integrated planning process. 3. Project management should be decentralized and democratic with all partners contributing their ideas, experience and solutions for problems encountered in the field. 4. Project staff should have ownership over their roles and responsibilities, and should be given the authority to do their jobs effectively, respecting the organogram. 5. Staff should be replaced when positions become vacant. 6. Increased transparency is needed to ensure that all staff are part of the phase-out, transition, and sustainability discussion. 7. All Plan field staff need to remain actively aware of all Plan projects and their activities. 8. Workload should be assessed periodically, and adjusted to the realities of each position and the field work required. Plan Cameroon Child Survival Project Final Evaluation Report August 23-September 1, 2004 77 Financial Management 1. Additional focus on financial management is needed with project partners. 2. Establish a separate bank account for future CS projects. 3. Issues of equity between project field staff should be discussed and resolved jointly. 4. All project partners need to be involved in budget discussions and decisions. 5. All projects and activities within Plan should harmonize all incentives to communities in order to minimize conflict. Logistics 1. Logistics should be pre-positioned for project staff and partners in order to begin using them at the beginning of the project, to not lose time, and to have a greater impact. 2. All project staff should be involved in the discussion of logistics requirements and processes. 3. The project needs to budget for very essential equipment and spare parts and have them pre-positioned so that when things break they can easily be fixed and not negatively impact project activities (ex. wicks and globes for refrigerators). Information Management 1. Further strengthen the MOH to collect, compile, analyze and use their data. 2. All project partners should be involved in data analysis and use from the inception of the project. 3. Revise the project monitoring system, collect only the data that can and will be used on a monthly or quarterly basis, and then phase it into the MOH system. 4. Revise the monitoring system tools and forms to follow the MOH system and create ONLY those forms necessary to fill the data gaps. 5. Data should be deeply analyzed and used at each level on a monthly basis. Plan Cameroon Child Survival Project Final Evaluation Report August 23-September 1, 2004 78 6. Provide computer software training for all data tools and systems. 7. Assist the MOH in developing a community based monitoring system for C￾IMCI. 8. The COSA should be trained to ensure that all forms are completed and compiled on a monthly basis, and the data sent to the health facility. 9. Provide sufficient numbers of computers with regular maintenance and repair support. Technical and Administrative Support 1. The technical backstop and the technical advisor should both continually assess management issues in the field during routine monitoring visits. Plan Cameroon Child Survival Project Final Evaluation Report August 23-September 1, 2004 79 Results Highlight The goal of the Plan CS XVI project was to create a sustained reduction in infant and child mortality through increased access to community-based health care and improved quality of health services in the project area. This goal was achieved through the implementation of a pilot project using the IMCI approach, which improved local access to preventive and curative services through capacity building interventions that targeted households, communities and health facilities. The project supported the MOH community outreach policy to empower communities to improve local health resources and increase access to health services. Sustainability has been ensured by strengthening the capacity of community resource persons (Health Area Committee Representatives, TBAs), community-based organizations (women’s groups), and the MOH staff (HCNs, community pharmacy workers) to provide quality services in the project area. Project interventions were strengthened using BCC activities designed to raise awareness about health issues and promote health-seeking behaviors. At the same time that communities were being empowered to take charge of their health needs, the project worked through its MOH partnership to strengthen the services of the health facilities by providing needed supplies and equipment and re-training staff. In addition, the project set up systems to assure delivery of outreach services to affiliated communities. It also worked to improve the functioning of the Health Area Committees. The CS XVI project was located in three Health Districts of the western part of Cameroon’s heavily forested Eastern Province. The province is Cameroon’s poorest and borders the Democratic Republic of Congo. Plan has been working in this large, sparsely populated province since 1996. The project benefited inhabitants of 267 villages (a total population of 211,2643 ) located in 27 Health Areas of Bertoua, Doume and Nguelemendouka health districts. The project targeted approximately 95,024 beneficiaries with 38,009 children under five, 8,447 infants 0 to 11 months and 48,568 women of reproductive age (15 to 49). Significant results were demonstrated using the IMCI approach, as shown in the table below. IMCI Indicator Baseline Final Trained Final Untrained The number of children 0-23 months seen at health facilities that were assessed for all danger signs 10.5% 72% 42.3% The number of caretakers who were correctly counseled about their sick child 36.3% 97% 65.4% The number of caretakers who were given oral medications, who know how to correctly administer the treatment 44.8% 84.6% 34.6% 3 2000 MOH Eastern Health Delegation Report. Plan Cameroon Child Survival Project Final Evaluation Report August 23-September 1, 2004 80 Attachments 1. Final Evaluation Team Members 2. Evaluation Assessment Methodology, Process and Logistics 3. Final Evaluation - Focus Group Discussion and Home Visit Results 4. Diskette/CD - Electronic Copy of FE Report (in WORD 2000) 5. Special Reports 6. Project Data Sheet Form Plan Cameroon Child Survival Project Final Evaluation Report August 23-September 1, 2004 81 1. Final Evaluation Team Members # NAME Plan NGO PARTNERS MOH TITLE 1. DELLA DASH EXTERNAL CONSULTANT 2. PIERRE-MARIE METANGMO X HEALTH PROGRAM COORDINATOR (IH) 3. ESTHER TALLAH X HEALTH COORDINATOR 4. LUCAS GANYE X PROGRAM AREA MANAGER 5. MFORNYAM C.N X PROJECT COORDINATOR 6. MVONGO MBANA X APCME 7. NDJI YVE X CPME ASSISTANT 8. FANNY AFANA X HEALTH SUPERVISOR 9. ANGOA MIMBANG X HEALTH SUPERVISOR 10. TCHUISSEU KEMENI VIVIANE X AHS/HIS CSP 11. FRANCO EKEME X FIELD COORDINATOR 12. NGORAN ROLAND X FIELD COORDINATOR 13. JULIE AKAME HKI PROGRAM OFFICER 14. DJOCK SALOMON PSI PROMOTOR 15. MPIANG MPIANG JACQUES AAPPEC HEALTH PROMOTERS’ SUPERVISOR 16. JEROME NGUELE AAPPEC COORDINATOR 17. NGBWAME ELLE X PSF MINSANTE 18. AMESSE FRANCOIS X DMO DOUME 19. NGOUADJIO PATRICE X DMO NGUELEMENDOUKA 20. TEDJOUKA ETIENNE X DMO BERTOUA Additional people consulted included: · Mr. Prem Shukla, Country Director, Plan Cameroon · Dr. Blanche Anya, WHO · Ms. Natalie Lell, SSH/DHR Coordinator, American Embassy, Cameroon Plan Cameroon Child Survival Project Final Evaluation Report August 23-September 1, 2004 82 2. Evaluation Assessment Methodology, Process and Logistics The team used a participatory process to facilitate the final evaluation. All members of the 21 person team participated in every activity to the best of their ability, learning and growing throughout the process. A strong focus was placed on teamwork, cooperation, communication, having fun and enjoying the work, trust, decision-making, capacity building, creative problem solving, and finding solutions and recommendations to improve future project management and implementation. Real success comes when the group becomes a team and the team pushes beyond narrow individual roles to create original products. The group develops a new, special energy which stimulates cohesive work. The team members were taken out of their standard frame of reference and asked to engage in things new and different…perhaps out of their comfort zone, to join in with a free spirit and an open mind, and to tap into their creativity, their knowledge, their experience, and their hearts… We used a number of different icebreakers, exercises and games to ensure a strong team spirit prevailed, while focusing on the goal of the evaluation. We created a safe environment in which all team members were free to participate freely without fear of reprisal or ridicule, but rather encouragement to think things through and question/probe deeply as to the reasons why the project had both the strengths and the weaknesses it did. We learned a lot from each other by building something valuable together. We began with a presentation of the entire life of the project using a format that listed every project indicator, along with their baseline, midterm and final results for comparison and discussion. The team reviewed each indicator, discussing the project activities designed to impact that indicator, and the tools used to measure it. Throughout this discussion, the data gaps were identified and listed. The team then discussed the types of tools needed to fill the data gaps, followed by an exercise done in small groups where each group developed one of the necessary tools. Finally, the groups went out to the field to pretest their tools, and then returned to make the necessary changes and share the finalized tools with the entire group. Two days were then spent in the field collecting the data needed to fill the gaps previously identified, followed by two days back in the office collating the data and translating it. Finally, the entire team came back together to analyze the data, using the same data table from the first day, discussing each indicator again to see if the information we had received during the field work further explained the results. Throughout this discussion, several themes emerged which are described in detail in this report. Finally, the team was led through an exercise that helped them formulated lessons learned, both positive and negative, to share with and inform the wider child survival community, further reinforcing the fact that what they accomplished through this project matters not only in the context of this country, but contributes to the global impact on child survival. Some of the resources used include the following: 1. Quicksilver: Adventure Games, Initiative Problems, Trust Activities and a Guide to Effective Leadership by Karl Rohnke 2. Group Process by Corey and Corey 3. Technical Reference Materials by CSTS 4. Request for Applications (RFA) by USAID 5. Final Evaluation Guidelines, August 2004, by USAID 6. The New Games Book by The New Games Foundation 7. More New Games Book by The New Games Foundation The supply list for the evaluation included the following items: Flip chart paper Colored markers White board and markers Computers and printers (ink and paper) Music box and music (CDs or tapes) Candy, treats, balloons, toys, games, etc. Plan Cameroon Child Survival Project Final Evaluation Report August 23-September 1, 2004 83 Final Evaluation Agenda (Tuesday, August 24 – Wednesday, September 1, 2004) Tuesday, August 24, 2004 8:00 AM Introduction Exercises 8:25 AM Expectations and Agenda Exercise 8:40 AM Ground Rules and Consensus Exercise 9:00 AM Trust Exercise 9:30 AM Inspiration and Challenge 9:45 AM Break 10:00 AM Presentation of Child Survival Program 1:00 PM Lunch 2:00 PM Discussion on Program and Presentation 2:30 PM Identify Data Gaps for FE 3:30 PM Identify Tools Needed to Close Gaps 4:00 PM Small Teams Development 4:30 PM Break & Games 5:00 PM Small Team Work – Tool Development 6:30 PM Plenary – Share Tools Developed 7:00 PM Closure for Day Wednesday, August 25, 2004 8:00 AM Recap from Tuesday and present agenda for today 8:15 AM Wake up game 8:30 AM Small Team Work – Finish Tools 9:45 AM Break 10:00 AM Site Selection and Logistics 12:00 PM Lunch 1:00 PM Pre-test Tools 4:00 PM Amend and Finalize Tools (and Logistics) 6:00 PM Plenary – Share Tools and Logistics 6:30 PM Closure for the Day Thursday, August 26, 2004 – Friday, August 27, 2004 7:00 AM – 6:00 PM Field Work and Data Collection Saturday, August 28, 2004 9:00 AM Present agenda for today 9:15 AM Wake up Game 9:30 AM Field Reports by Teams 10:45 AM Break and Game 11:00 AM Continue Field Reports 1:00 PM Lunch 2:00 PM Data Entry and Analysis (and recommendations list) (Della reviewing documents) 3:00 PM Break and Game 3:30 PM Continue Data Entry and Analysis (Della reviewing documents) 5:00 PM Closure for the Day Sunday, August 29, 2004 9:00 AM Present agenda for today 9:15 AM Wake up Game 9:30 AM Continue Data Entry and Analysis (Della reviewing documents and interviewing staff) 10:45 AM Break and Game 11:00 AM Continue Data Entry and Analysis (Della interviewing staff) 1:00 PM Lunch 2:00 PM Continue Data Entry and Analysis and Translations (Della interviewing staff) Plan Cameroon Child Survival Project Final Evaluation Report August 23-September 1, 2004 84 3:00 PM Break and Game 3:30 PM Continue Data Entry and Analysis and Translations (Della interviewing staff) 5:00 PM Closure for the Day Monday, August 30, 2004 9:00 AM Recap from Sunday and Present agenda for today 9:15 AM Wake up Game 9:30 AM Plenary Analysis of Data 10:45 AM Break and Game 11:00 AM Continue Plenary Analysis of Data – and develop recommendations 1:00 PM Lunch 2:00 PM Continue Plenary Analysis of Data – and develop recommendations 3:30 PM Break and Game 4:00 PM Continue Plenary Analysis of Data – and develop recommendations 5:00 PM Closure for the Day Tuesday August 31, 2004 9:00 AM Recap from Monday and Present agenda for today 9:15 AM Wake up Game 9:30 AM Plan for Feedback Workshop 10:45 AM Break and Game 11:00 AM Begin preparing presentations for Feedback Workshop (Della interviewing staff) 1:00 PM Lunch 2:00 PM Continue preparing presentations (Della interviewing staff) 3:00 PM Break and Game 3:30 PM Finalize presentations (Della interviewing staff) 5:00 PM Closure for the Day Wednesday, September 1, 2004 9:00 AM CS Project FE Restitution workshop 3:00 PM Drive to Yaounde Thursday, September 2, 2004 2:00 PM Meeting with MOH on IMCI 4:00 PM Meeting with American Embassy GROUND RULES · Punctuality · Mutual respect · Put off cell phone – must sing a song if your cell phone rings · Congratulate and appreciate people’s contribution · Team spirit · Not interrupt each other EXPECTATIONS · Have a winning proposal from this · Bring out success and challenges · Assess achievements · Recommendations for improvement · Clarify what should be improved · Indication on the real impact on children life · Full participation · Have fun · Identify all party contribution Plan Cameroon Child Survival Project Final Evaluation Report August 23-September 1, 2004 85 · Yield information for MOH to enhance and foster IMCI implementation (scaling) · Learning by using tools · Lean a lot from each other DATA GAPS 1. How are the people who received the IMCI 11 days training in Doume performing compared to the untrained in the same district in data source? 2. Recognition of danger signs for a child with diarrhea (despite the effort, why are women not changing their behavior? 3. Why the women gave more food to the children and less fluid? 4. why the simple diarrhea treatment target was not attained 5. explore more on hand washing with mothers 6. why the great differences on hand washing as concern after defecation 7. go back and check the data source on immunization coverage 8. focus group discussion with the health staff on immunization coverage 9. verify the authenticity of immunization data 10. on IMCI we need to work with mothers to know the way they are learning 11. capacity building and sustainability indicators 12. degree of ownership community/ health center 13. behavioral change TOOLS NEEDED 1. Review Doume IHFA –observation and exit interview data compare trained/untrained workers on IMCI 2. FGD – CBOs + TBA + dialogue structure ( group 4) FGD – mother (who are not CBO)/ caretakers/Grandmother (group 3) FGD - Health worker (group 1) 3. review KPC(final baseline data for LQAS 4. review the health facility data to 3 districts 5. review of monitoring system at the districts level and Plan 6. checklist - home visit (group 2) Plan Cameroon Child Survival Project Final Evaluation Report August 23-September 1, 2004 86 Pre-testing for FGD tools (Mother, CBO) is being done in Ndoumbi Village approximately 13 KM from Bertoua. SITE SELECTION Thursday, August 26, 2004 Bertoua Doume Nguelemendouka Schedule Community Health center Community Health center Community Health center Team 2 Kano, Mbombi (best) · Mother FGD · CBO/TBA/DS FGD Thursday AM Team 4 Diang, Yanda I (worst) · Mother FGD · CBO/TBA/DS Team 1 Dimako, Nkolbikon (worst) · Mother FGD · CBO/TBA/DS FGD Team 3 N’ka (best) · Mother FGD · CBO/TBA/DS FGD Team 2 Kano, Mbombi (best) · Home Visits Thursday PM Team 4 Diang, Yanda I (worst) · Home Visits Team 1 Dimako, Nkolbikon (worst) · Home Visits Team 3 N’ka (best) · Home Visits FIELD TEAMS 1- Doume 2- Bertoua 3- N’ka 4- Bertoua - Julie Akame (HKI) - Dr. Elle (MOH) - Nguele (AAPPEC) - Amesse (DMO Doume) - Della Dash (Consultant) - Mvongo (Plan) - Fanny (Plan) - Dr. Tedjouka (DMO Bertoua) - Angoa (Plan) - Patrice (Plan) - Mpiang (AAPPEC) - Dr. Tallah (Plan) - Pierre Marie (Plan) - Franco Ekeme (Plan) - Dr. Patrice (DMO N’ka) - Viviane (Plan) - Ngoran Roland (Plan) - Dr. MfornyamChristopher (Plan) - Djock Salomon (PSI) Plan Cameroon Child Survival Project Final Evaluation Report August 23-September 1, 2004 87 SITE SELECTION Friday, August 27, 2004 Bertoua Doume Nguelemendouka Schedule Community Health center Community Health center Community Health center Team 2 Essanjdane, Belabo (worst) · Mother FGD · CBO/TBA/DS FGD Friday AM Team 4 Nkolbikon , Bonis II (best) · Mother FGD CBO/TBA/DS Nkolbikon · FGD Team 1 Motcheboum (best) · Mother FGD · CBO/TBA/DS FGD Team 3 Ngoap, Ngoumou (worst) · Mother FGD · CBO/TBA/DS FGD Team 2 Essanjdane, Belabo (worst) · Home Visits Friday PM Team 4 Nkolbikon , Bonis II (best) · Home Visits Team 1 Motcheboum (best) · Home Visits Doume · FGD Team 3 Ngoap, Ngoumou (worst) · Home Visits N’KI · FGD 8 mother FGD 8 CBO FGD 3 Health center FGD 8 sets of home visits Plan Cameroon Child Survival Project Final Evaluation Report August 23-September 1, 2004 88 Final Evaluation - Focus Group Discussion and Home Visit Results COSA/OBC/TBA FGD Results Question 1 Messages about diarrhea: o Continue breastfeeding, give more water, or ORS, give rice water, homemade solution from guava leaves, SSS, give more food o In case of danger signs (tiredness, sunken eyes) advised to go to hospital o Use latrines, protect food, wash hands before cooking and eating o Environmental hygiene and sanitation o Go to farm with portable water o Avoid drinking contaminated water o Boil water from spring before drinking in the rainy season o Personal hygiene o Keep baby clean How messages are given: o Use IEC material on diarrhea o By demonstrating o Using posters o Explanation o Health education during meetings Means of verification: o Through observation o Home visits o Decrease in illness/death o Increased utilization of health services o Growth monitoring o Child’s health status o During demonstrations o Testimonies from parents Question 2 Efficiency of health messages: o Reduction in diseases (children are less ill now, and less child deaths) o Behavior change in women o Testimonies from mothers o Observations during home visits o Technique of hand washing has changed Why no change in behavior? o Resistance to change old habits die hard) o Negligence, ignorance for those who do not attend health education sessions o Forgetfulness o Too busy o Poverty o Religious and ancestral beliefs o Misconceptions o *** It’s not true, on the contrary many people are changing their behavior Question 3 Supervision: o Supervision is good because it ensures that we are on the right tract, if the work is well done and improvement and difficulties encountered. o We have received working material o During supervision they verify if bed nets have been sold and if we have a work plan Plan Cameroon Child Survival Project Final Evaluation Report August 23-September 1, 2004 89 o Supervision should be more regular and health personnel also have to supervise us o Supervision made me to start using material (TBA) o Correction of errors as a form of refresher course Question 4 o Acquired new knowledge, training and equipment of TBAs, CBOs, importance of vaccination, balanced diet, village cleanliness, ITNs, rehabilitation of malnourished children, clean delivery, knowledge on common childhood illnesses, VIP latrines, reduction in child illness and death o Reduction in health expenses leading to increased savings o Improved hand washing o Improvement in child caring habits o Harmony in the families o Behavior change o Pygmies integrated into the system o Collaboration with traditional healers o Improved working relationship and partnership Question 5 o Continue awareness raising on child health issues o Continue to train others, take over and continue with project activities, TBAs will continue to work because they had been trained and equipped, continue to teach by demonstration o Continue to use mosquito nets o Continue to weigh the children, advise women on the importance of vaccination o Pass the message down the family from generation to generation o Support the activities to ensure continuity (Essanjane) o Need to strengthen the partnership between communities and health facilities o Life of the project is too short o Need assistance o Material and equipment received will need maintenance Mothers FGD Results Question 1 Danger signs: o Tiredness o Weigh lost o Unable nor suck o Vomiting o Weigh lost o Frequent watery stool in a day o Unable to suck or eat o Blood in stool o Thirsty o Temperature rise (fever) o Suckle eyes o Dehydrated o Stomach ache o Cries a lot o Dry skin o Foamy stool What do you give/do: o Clean water frequently o Antibiotic o Traditional medicine o Hospital Plan Cameroon Child Survival Project Final Evaluation Report August 23-September 1, 2004 90 o SSS o Antibiotic o Traditional medicine o Water from boil rice o Capsule o Roasted plantain o ORS o Rice water o Carrot juice o Infusion o Vitamins, if no appetite o Guava leaves solution o More breast feeding o More food (than water to prevent tiredness) o Flagyl Why more food but not more fluid: o Regain lost weight o Provide energy to the tired child o Provide energy to the tired child o Solid food will make the stool harder while water will worsen the situation o One mother says she give more water than food Question 2 Hand washing when: o Before breastfeeding the child o Before and after eating or feeding the child o On return from toilet o Before preparing the child’s milk o Before cooking the child food o On return from the farm o Before and after eating o Before breastfeeding the child o On return from toilet o Before and after eating o After cleaning a child who has defecated o After sexual intercourse Why: o To avoid diseases o To avoid microbes Why few mothers mentioned washing hand after using the latrine (and more after cleaning child stool): o No habit to wash hands with soap o Difficult to wash hands with soap at all moments o Only God who protects the children o When you change the child who has defecated, your hand comes in contact with the stool. This is not the case when an adult defecates and cleans his/herself o Scarcity of water o Ignorance, negligence, some believe that dirt can’t kill Africans Question 3 Who gives information: o Nurses o Our mothers o Grandmothers o grandfathers o Plan workers o Health promoters Plan Cameroon Child Survival Project Final Evaluation Report August 23-September 1, 2004 91 o COSA (community health workers) What types of information: o Personal hygiene of children o Vaccination o Vitamin A at six months o Do not keep children naked o Latrine should be far from home o Environmental hygiene and sanitation o Balance diet o Winning diet o Malaria o Diarrhea o Cholera o Yellow fever o Measles o ITNs Advice vs. customs, culture: o Children hygiene o Environmental hygiene and sanitation o Vaccination o Malaria o Diarrhea o Balance diet/ weaning o Weighing of children o Hand washing o Take sick children to health centers and not to traditional practitioners o Buy drugs from health centre and not from street vendors o ITN o Cholera o Yellow fever o Measles Significance: o Babies healthier than before – reduction of mortality and morbidity o Illness has been reduced o They now spend less on health o More time to go about their activities How often: o Once a month during outreach o On demand when child is sick o Prenatal consultation Materials used: o Posters o Weighing scales o IEC material o Practical Demonstration o Nutrition demonstration o Songs Why not put into practice: o Forgetfulness o Negligence o Lack of means o Forgetfulness o Time consuming o Conflict with customs o Lack of initiative and love for the child o Resistance to change Plan Cameroon Child Survival Project Final Evaluation Report August 23-September 1, 2004 92 Question 4 Benefits o Know ledges and advises o ITNs o Outreach prenatal visit o Behavior change which has led to reduction of diseases and death Question 5 After the project: o Continue to put in practice the acquired knowledge o Share the acquired know ledges with other o Continue to put in practice the acquired knowledge o Continue with fight again diarrhoea, malaria, vaccination child care Questions asked by participants: o Why do we not reduce the price of ITNs o Why are ITNs only given to pregnant women Health Facility Staff FGD Results Question 1 Immunization results: o Health workers doubt the authenticity of survey statistics o Their statistics show that they are always above the survey results. Some centers even claim to have attained more than 100% o Negligence on the part of the mothers who can not conserve vaccination cards o Insufficient sensitization o Mobility of children of vaccinate age o Difficult to convince some mothers on the importance of vaccination o Misinterpretation of vaccination effects by some mothers o Some health staff don’t know and/or follow the vaccination schedule o Some health staff haven’t acquired enough training on vaccination o Demotivation (irregular incentives) o Insufficient collaboration between stakeholders o Population statistics don’t tie with field realities o Personnel transfer o Cold chain not functioning o State credits difficult to be unblocked (difficult to get funding from MOH for health center) Question 2 How do you treat simple diarrhea: o ORS o SSS o Advice (give more liquid, continue breastfeeding, in case of gravity bring the child back to the hospital, in case of artificial feeding ask mother to demonstrate how she cooks. If there is an error they are corrected) o Mother/environmental hygiene o Deworming o Homemade solution o Check immunization status Why antibiotics are being administered: o Done only by untrained health staff o Some have developed habits if giving the antibiotic o Instability of trained staff as a result of transfers o Some cases are associated with other infections that need antibiotics Plan Cameroon Child Survival Project Final Evaluation Report August 23-September 1, 2004 93 Question 3 What do you think about the supervision you had during the project period: o Supervision is normal o Serve as refresher course o Keep the health staff awake o Average, could be improved o A lot of improvement needed Question 4 How have you benefited from the project: o Behavior change in the communities (eg. Hand washing) o Importance of ITN o Improvement in the performances of health staff o Material and equipment (refrigerators, motor bikes, solar energy, weighing scales, registers, vaccination cards) o Infrastructures (health center latrines) o Motivation of health staff (per diem) o Training of CBOs o Increased knowledge on IMCI o More knowledge Question 5 What will you do when the project ends: o Continue with the activities o Continue to put in practice the acquired knowledge o Heritage that will continue o Improved skills and knowledge Have you any questions: o Will there be a team to continue supervision o Is it not possible to renew the project in order to attain the objective of 60% vaccination coverage Plan Cameroon Child Survival Project Final Evaluation Report August 23-September 1, 2004 94 Home Visit Checklist Intervention Yes No Comments 1. Malaria · Treatment o Medication (dosage, stock) 3/27 with drugs o IPT No pregnant women · Environment o Cleanliness 12/23 generally improved o Reservoirs 4/23 · Bed net o Number of nets 17 o Condition 11/17 in good condition o Age of net(s) 3 months to 3 years o Impregnated (if yes, when) 16 impregnated, 0 reimpregnated o Over child bed 5/17 o Over mother bed 14/17 (family bed) o Over other bed 1/17 grandmother o Used last night 15/17 – 1 not used/traveled o CBO to demonstrate re-treatment in one village 7/8 2. Diarrhea · Environmental Sanitation o Hand washing site 2/23 o Soap 21/23 o Latrine 16/23 open pits, unimproved o Cleanliness 4/16 o Refuse disposal 5/23, but too close to house o Food storage 12/23 o Waste water disposal 3/23 · Water o Source Wells, springs, boreholes o Storage Open and closed containers o Treatment 1 well treated every 3 months · Treatment o ORS 7/23 o Homemade solution 13/23 o Other 11/23 o Mother to demonstrate ORS preparation in one village 10/23 3. Immunization · Child card o Status (MMRDPT, Vitamin A, yellow fever, BCG) 15/23 · Pregnant woman card o Status (TT, Iron/Folic Acid) 2/23, 11/23 confirmed TT 4. IMCI · Mother to demonstrate assessment of child wellness 14/23 · Mother to verbalize follow up based on findings and explain 13/23 5. Nutrition · Food availability 23/23 · Food present (kinds) 8/23 season foods · Mother to demonstrate child feeding 7/23 · Check growth monitoring chart on vaccination card and follow up 13/23 Plan Cameroon Child Survival Project Final Evaluation Report August 23-September 1, 2004 95 Diskette or CD with electronic copy of the report in MS WORD 2000 Plan Cameroon Child Survival Project Final Evaluation Report August 23-September 1, 2004 96 Special reports 1) Baseline KPC Report 2) Baseline IHFA report 3) Baseline FGD Report. 4) The Project Detailed Implementation Plan (DIP) 5) The Start Up Workshop Report 6) First Year Annual Report 7) Base line OCA for AAPPEC 8) Base line OCA for FESADE 9) MTE Report 10) Third Year Annual Report 11) Final KPC Report 12) Final IHFA Report 13) Training module on child health issues 14) Trainers module on PLA methodology 15) TBA training module 16) IMCI module 17) Training Report on 6-day course on IMCI 18) Report on TOT on child health issues 19 ) Project quarterly reports from AAPPEC 20) Project provincial coordination committee reports 21) Project steering committee reports 22) Training Report on 11-day course on IMCI Plan Cameroon Child Survival Project Final Evaluation Report August 23-September 1, 2004 97 Project Data Sheet form – updated version CSGP Data Form Child Survival Grants Program Project Summary FE Submission: Oct-29-2004 Plan Cameroon Field Contact Information: First Name: Last Name: Address: PLAN International Cameroon Bastos, P.O. Box 25236 Messa City: Yaounde State/Province: Zip/Postal Code: Country Cameroon Telephone: Fax: +237-221-5457 E-mail: Project Web Site: - Project Information: Project Description: PLAN International is implementing a four-year Child Survival XVI project in three health districts of the western part of Cameroon’s heavily forested Eastern Province. The project is located in the health districts of Bertoua, Doume and Nguelemendouka. The project targets approximately 95,024 beneficiaries with 38,009 children under five, 8,447 infants 0-11 months and 48,568 women of reproductive age (15-49). The project is implemented in partnership with the Ministry of Health (MOH), the Women’s Health Development in Sub Saharan Africa (FESADE), and the Association for the Self-Promotion of the Eastern Province Population (AAPPEC). The project site was chosen because of its poor health indicators and inadequate access Plan Cameroon Child Survival Project Final Evaluation Report August 23-September 1, 2004 98 to health services. The project area has high rates of infant, under 5 and maternal mortality (77 per 1,000, 151 per 1,000 and 430 per 1000,000 live births respectively) Malaria, diarrhea, measles, respiratory infections and malnutrition are the main causes of under 5 mortality. Many villages in the project area are located far from health centers (over 5 km away). Health center staff cannot reach many of these communities for outreach services on a regular basis due to lack of transportation and supplies. Both these factors contribute to under-utilization of health services in the project area. The project was established in order to create a partnership between communities and the MOH in order to address the major health problems in the area. The project will intervene in four main areas: Malaria (35%), by improving Malaria Case management (home and facility treatment), antenatal malaria prevention and treatment and promoting increased use and re-treatment of insecticide treated mosquito nets; Malnutrition (25%) which includes mother managed growth monitoring and follow-up, household feeding programs, nutrition education, breastfeeding promotion and proper weaning practices; Diarrhea (20%) which involves proper hygiene, recognition of danger signs with appropriate referral and promotion of preventive measures; Immunization (20%) aimed at increased immunization coverage in the program area for all infants by the end of the first year, vitamin A supplementation for children 6 months to five years, and tetanus toxoid (second dose for pregnant women). In addition, the project will implement Cameroon’s national IMCI policy in the project area. The project will promote community empowerment of health actions through strengthening community-based groups to promote healthy behaviors in the community and develop strategies to utilize health services. At the same time, the project will improve the quality of health services offered and strengthen ties between communities and health services. Partners: Cameroon MOH; AAPPEC (Association for the Self-Promotion of the East Province Population); FESADE (Women's Health and Development in Sub-Saharan Africa); Catholic Mission; Presbyterian Mission Project Location: 3 districts: Bertoua, Doume and Nguelemendouka in the Eastern Province of Cameroon Grant Funding Information: USAID Funding:(US $) $1,000,000 PVO match:(US $) $ 366,778 Target Beneficiaries: Type Number Infants (0-11 months): 8,447 0-59 month old children: 38,009 Plan Cameroon Child Survival Project Final Evaluation Report August 23-September 1, 2004 99 Women 15-49: 48,568 Estimated Number of Births: 42,233 Beneficiary Residence: Urban/Peri-Urban % Rural % 10% 90% General Strategies Planned: Strengthen Decentralized Health System M&E Assessment Strategies: KPC Survey Health Facility Assessment Organizational Capacity Assessment with Local Partners Community-based Monitoring Techniques Behavior Change & Communication (BCC) Strategies: Interpersonal Communication Peer Communication Support Groups Mass Media Capacity Building Targets Planned: PVO Non-Govt Partners Other Private Sector Govt Community Field Office HQ CS Project Team Local NGO Pharmacists Traditional Healers Dist. Health System Health Facility Staff Health CBOs Interventions: Immunization 20 % ** IMCI Integration ** HF Training *** Polio *** Classic 6 Vaccines *** Vitamin A *** Surveillance Plan Cameroon Child Survival Project Final Evaluation Report August 23-September 1, 2004 100 *** Cold Chain Strengthening Nutrition 25 % ** IMCI Integration ** HF Training *** Comp. Feed. from 6 mos. *** Hearth *** Cont. BF up to 24 mos. *** Growth Monitoring Control of Diarrheal Diseases 20 % ** IMCI Integration ** HF Training *** Hand Washing *** ORS/Home Fluids *** Feeding/Breastfeeding *** Care Seeking *** Case Mngmnt./Counseling Malaria 35 % ** IMCI Integration ** HF Training *** Training in Malaria CM *** Adequate Supply of Malarial Drug *** Access to providers and drugs *** Antenatal Prevention/Treatment *** ITN (Bednets) *** Care Seeking, Recognition, Compliance Indicator Numerator Denominator Estimated Percentage Confidence limits Percentage of children age 0-23 months who are underweight (-2 SD from the median weight-for-age, according to the WHO/NCHS reference population) 59 360 16.4 4.2 Percentage of children age 0-23 months who were born at least 24 months after the previous surviving 120 164 73.2 6.0 Plan Cameroon Child Survival Project Final Evaluation Report August 23-September 1, 2004 101 Indicator Numerator Denominator Estimated Percentage Confidence limits child Percentage of children age 0-23 months whose births were attended by skilled health personnel 173 360 48.1 12.5 Percentage of mothers of children age 0-23 months who received at least two tetanus toxoid injections before the birth of their youngest child 199 360 55.3 6.0 Percentage of infants age 0-5 months who were exclusively breastfed in the last 24 hours 66 116 56.9 11.9 Percentage of infants age 6-9 months receiving breastmilk and complementary foods 37 55 67.3 9.9 Percentage of children age 12-23 months who are fully vaccinated (against the five vaccine-preventable diseases) before the first birthday 81 153 52.9 11.4 Percentage of children age 12-23 months who received a measles vaccine 95 153 62.1 9.9 Percentage of children age 0-23 months who slept under an insecticide-treated bednet the previous night (in malaria-risk areas only) 56 360 15.6 4.8 Percentage of mothers who know at least two signs of childhood illness that indicate the need for treatment 234 360 65.0 8.5 Percentage of sick children age 0-23 months who received increased fluids and continued feeding during an illness in the past two weeks 57 262 21.8 6.5 Percentage of mothers of children age 0-23 months who cite at least two known ways of reducing the risk of HIV infection 221 360 61.4 2.4 Percentage of mothers of children age 0-23 months who wash their hands with soap/ash before food preparation, before feeding children, after defecation, and after attending to a child who has defecated 74 360 20.6 9.7 Plan Cameroon Child Survival Project Final Evaluation Report August 23-September 1, 2004 102