FOR DI PIKIN DEM WEL BODI (THE HEALTH OF THE CHILD) Community-based health initiatives implemented through social cohesion strategies in Koinadugu District, Sierra Leone Midterm Evaluation Report Jean Meyer Capps, RN MPH Team Leader Date of Submission: Cooperative Agreement: GHS-A-00-03-00013-00 October 1, 2003-September 30, 2008 CARE Sierra Leone Child Survival XIX Project Midterm Evaluation Report June 4, 2006 2 Table of Contents Abbreviations and Acronyms…………………………………………………………………...4 A. Executive Summary………………………………………………………………..…….6 B. Assessment of the progress made toward achievement of program objectives………………………………………………………………..…..9 . 1. Technical Approach………………………………………………………….…….….9 a. Overview…………………………………………………………….……9 b. Interventions…………………………………………………………….10 i. Immunization…………………………………………………...10 ii. Nutrition………………………………………………………....12 iii. Malaria…………………………………………………….….…16 iv. Maternal and Newborn Care…………………………….….…19 c. New Tools and Approaches……………………………………….…....23 2. Cross-cutting Approaches..........................................................................................23 a. Community Mobilization………………………………………….…...24 b. Communication for Behavior Change………………………….….….29 c. Capacity Building Approach…………………………………………..31 i. Strengthening the PVO Organization…………………………31 ii. Strengthening Local Partner Organizations………………….31 iii. Health Facilities Strengthening………………………………..34 iv. Strengthening Health Worker Performance………………….35 v. Training………………………………………………...……….37 d. Sustainability Strategy…………………………………………………38 C. Program Management………………………………………………………………….40 1. Planning…………………………………………………………………………40 2. Staff Training…………………………………………………………………...40 3. Supervision of Program Staff………………………………………………….42 4. Human Resources and Staff Management……………………………………43 5. Financial Management…………………………………………………………44 6. Logistics…………………………………………………………………………44 7. Information Management……………………………………………………...45 8. Technical and Administrative Support………………………………………..49 9. Mission Collaboration………………………………………………………….49 D. Other Issues Identified by the Team…………………………………………………..50 E. Conclusions and Recommendations……………………………………………….….50 F. Results Highlight……………………………………………………………………….54 CARE Sierra Leone Child Survival XIX Project Midterm Evaluation Report June 4, 2006 3 III. Action Plan……………………………………………………………………….....55 IV. Attachments A. Baseline Information from the DIP B. Evaluation Team Members and their Titles C. Evaluation Assessment Methodology D. List of persons Interviewed and Contacted F. Special Report G. Project Data Sheet H. Post-Midterm Evaluation Review of DIP Workplan, for incorporation into post-MTE Action Plan I. Post MTE -- Action Plan Priorities for FY2006 J. "Communicating Health Communicating Rights" Addendum Sheet CARE Sierra Leone Child Survival XIX Project Midterm Evaluation Report June 4, 2006 4 ACT Artesunate Combination Therapy AIDS Acquired Immune Deficiency Syndrome ANC Antenatal Care APM Assistant Program Manager ARI Acute Respiratory Infection BCC Behavior Change Communication CARE-SL CARE Sierra Leone Mission CBGP Community Based Growth Promotion CBO Community-Based Organization CCF Christian Children’s Fund CES Christian Extension Services CHC Community Health Club Chiefdom Third level administrative unit in Sierra Leone, under the District. C-IMCI Community-Based Integrated Management of Childhood Illnesses CO Country Office (CARE Sierra Leone) COPE Client Oriented Provider Efficient CRS Catholic Relief Services CS Child Survival Project CSTS Child Survival Technical Services DC District Council DFID British Department for International Department DHMT District Health Management Team DIP Detailed Implementation Plan District Second level administrative unit in SL, under the Region and above the Chiefdom DMO District Medical Officer, in charge of DHO EmOC Emergency Obstetric Care ENA Essential Nutrition Actions EPI Expanded Program in Immunization GOSL Government of Sierra Leone HBLSS Home-Based Life Saving Skills HH Household HMIS Health Management Information Systems HIV Human Immune Deficiency Virus HQ Headquarters IEC Information Education and Communication IMCI Integrated Management of Childhood Illnesses IPT Intermittent Presumptive Therapy ISA Institutional Strengths Assessment ITN Insecticide Treated Mosquito Net KPC Knowledge, Practice and Coverage LLIN Long Life Insecticide Impregnated Net LNGO Local Non-Governmental Organization LQAS Lot Quality Assurance Sampling MCH Maternal and Child Health M&E Monitoring and Evaluation CARE Sierra Leone Child Survival XIX Project Midterm Evaluation Report June 4, 2006 5 MMR Maternal Mortality Rate MNC Maternal and Newborn Care MOHS Ministry of Health and Sanitation, Government of Sierra Leone NID National Immunization Day NGO Non-Governmental Organization ORS Oral Rehydration Solution PHU Peripheral Health Unit PM Program Manager PP Post Partum PRCA Participatory Rural Communication Appraisal PVO Private Voluntary Organization QA Quality Assurance QOC Quality of Care RBM Roll Back Malaria RDT Rapid Diagnostic Test Region The largest administrative unit at the sub-national level RH Reproductive health SCM Standard Case Management SL Sierra Leone SP Sulfadoxine-Pyrimethamine TA Technical Assistance TBA Traditional Birth Attendant TNA Training Needs Assessment TOT Training of Trainers TT Tetanus Toxoid UNICEF United Nations Children’s Fund USAID United States Agency for International Development VA Vitamin A WFP World Food Program WRA Women of Reproductive Age CARE Sierra Leone Child Survival XIX Project Midterm Evaluation Report June 4, 2006 6 A. Summary The CARE Child Survival Program (CSP) is a 5-year standard category USAID Cooperative Agreement. The project is implemented in partnership with the Sierra Leonean MOHS District Health Management Team in Koinadugu District in the northern region of the country. The program was one of the first two CSHGP grants awarded in Sierra Leone following the end of a tragic 15-year civil war and follows several years of direct humanitarian relief activities by international organizations and NGOs in the area. Direct beneficiaries include an estimated 39,838 children under age 5 and 56,240 Women of Reproductive Age (WRA). These were adjusted in the Second Annual Report from the DIP total of 48,630 under five and 51,491 WRA (a decrease of 4% in the total number of beneficiaries) after national census in 2004-2005; the census, however, is still under some review and discussion. The goal of the project is to improve the health status of children under five and women of reproductive age (WRA) in 54 communities in 5 chiefdoms of Koinadugu District. Principle objectives of the program are: 1) Strengthened family and household knowledge and decision￾making skills related to health of women and children resulting in the practice of positive behaviors to prevent, recognize and manage common diseases; 2) Enhanced community capacity to form groups and institutions that sustain health initiatives, demonstrate social cohesion, and promote good governance mechanisms; and, 3) Improved quality and accessibility of services provided by MOHS personnel and MOHS extension services. The project interventions, EPI (15%), nutrition (20%) malaria (35%) and MNC (30%) are designed to be implemented using the C-IMCI strategy1 . The GOSL has just begun to plan for training Koinadugu DHMT staff in IMCI, much later than was anticipated in the DIP. The CARE program technical interventions are still implemented along interventions in health facilities, while the program is much more integrated at the community level through the project’s major community mobilization mechanism: Community Health Clubs (CHCs). Community Health Clubs are organized and trained in all 56 communities. CARE’s adaptation of the CHC approach is now a model that is already being incorporated into other programs, both within CARE and in other organizations. Each community now has at least one Community Based Growth Promotion (CBGP) group. In partnership with the DHMT and UNICEF, the project helped distributed over 5,000 Long Life Insecticide Treated Nets (LLINs) and is poised to distribute another 11,000 that are already stored in the district. A COPE quality of care assessment was completed in the Peripheral Health Units (PHUs). During this Midterm Evaluation, the MTE team reviewed the status of recommendations and found the CSP, DHMT and District Council have made some progress; 5 actions were selected by the MTE team for prioritization during the remaining life of the project. Cross-project visits and consultations between CARE and the International Rescue Committee (IRC) project in Kenema have greatly benefited both projects. CARE’s strong contributions to community-based malaria programs have already attracted additional funding from DFID to support expansion of malaria activities. The project has already made significant progress in developing strengthened family and household knowledge and decision-making skills related to the health of women and children. 1CARE SL CS XIX DIP, 2004. CARE Sierra Leone Child Survival XIX Project Midterm Evaluation Report June 4, 2006 7 Formation and training of CHCs, Village Development Councils, and CBGPs has greatly enhanced community capacity to form groups and institutions to address problems in health and other areas, especially education. In many areas, quality and accessibility of services provided by MOHS personnel and MOHS extension services has been improved, especially in client satisfaction and extending services through outreach programs. The sustainability of the activities and achievements of the project at the community level will be enhanced by the changed community social norms promoted by the project; sustainability at the facility level is largely dependent on MOHS full staffing and outside of CARE's direct control. CARE has been implementing the CSP in the midst of major decentralization of health services where decision-making and resource management is shifting from the central government to the districts and with proposed changes in responsibility between District Health Office and district government council. While undergoing these major changes, the Koinadugu DHMT has tried to operate while constrained with only 60% of the staffing that is needed according to national standards. The current DMO is the third since the program began. At the same time, the CSP is one of the first true development programs to begin in Koinadugu since the war and follows a long period when the population and local government representatives became accustomed to NGO relief programs. CHCS and VDCs have been trained in child survival topics in a series of 25 sessions, with additional emphasis on child rights and gender issues. CBGP groups have been trained in conjunction with PHU staff. PHU staff has had technical update sessions on nutrition, malaria (provided by MOHS, UNICEF and CARE) and EPI. In addition, through joint planning and facilitation, CARE has strengthened the capacity of DHMT staff to utilize participatory facilitation techniques. A student intern from Emory University MPH program was placed with the DHMT HMIS staff and, along with CSP staff, provided training in data collection, management and interpretation. Ninety-seven Memoranda of Understanding (MOU) were developed between the project and each participating VDC to outline the parameters of the project and to detail specific roles and responsibilities of CARE-SL and each participating community. MOUs have fostered accountability and transparency for operations. The chronic understaffing of the DHMT facilities is the major threat to the sustainability of the continuation of project interventions after the end of the project, but this is not under CARE’s control. The community based, preventative and health care seeking (early treatment) emphasis of the program has high potential for sustainability if the District Council commits to the minimal inputs necessary to insure that periodic refresher trainings are provided over time. Focus group discussions in communities revealed an appreciation for the need to plan for sustainability. Frequent DMO turnover, however, has delayed expected progress in capacity-building. CARE has established a model (CHCs) for strong community based programming in Koinadugu District and, despite setbacks from high DHMT turnover, is now well on the way to establishing strong partnerships with the DHMT and local District Council. These are significant accomplishments. Community based CSP activities through the CHCs have already resulted in empirically noticeable reductions in mortality and morbidity. CARE Sierra Leone Child Survival XIX Project Midterm Evaluation Report June 4, 2006 8 Local PHU health functioning has been strengthened, and many quality elements have improved, but the differences between baseline, current and future staff clinical performance can not be objectively determined from current project documents and assessments. The CSP is encouraged to use more quantitative measures of clinical PHU capacity so that impact on improving the quality of clinical service care provision in malaria, malnutrition and maternal/newborn care will not be only assessed on the basis of client provider satisfaction methods (COPE). Project training materials and methodologies overall are appropriate and high quality, but in a few cases need to be updated to reflect current international standards. A plan for incorporating revised national policies into CARE’s training materials over time will need to be developed. CARE’s role in revising these policies, especially related to advocacy, will need to be a priority for the new full￾time Health Sector Coordinator for CARE Sierra Leone. The CSP is impacted by many factors that are not under CARE’s direct control, such as health policies and the understaffing at MOHS health facilities. The central MOHS is updating several policies and has welcomed CARE’s interest in providing input. The CARE Health Sector Coordinator position in Freetown will be vital for CARE’s to play a role in updating these important national policies as well as helping the DHMT to advocate for health needs in Koinadugu District. To strengthen this role, CARE should involve the CSP partners in identifying key issues that need attention at the national level. The CSP was designed using C-IMCI as a basis and included many roles for CARE to play in bringing IMCI to Koinadugu, while at the same time supporting quality in following existing standard case management policies. IMCI roll out in SL has been slower than anticipated and is yet to be introduced in 2006 (not 2004 as planned; discussion at national level continues with official publication of protocols pending). As of the MTE, C-IMCI, or IMCI does not appear in evidence as the guiding framework of the CSP, although many individual aspects are in place. Discussions with field staff revealed some confusion of the process involved in transitioning vertical disease-specific programming (e.g. malaria) to the integrated and more holistic approach of IMCI. CARE has demonstrated strong capacity to provide the sorely-needed community component to the national and global Roll Back Malaria effort. CARE should strongly consider building upon this experience to become a major player in RBM in Sierra Leone by sharing the lessons learned while developing their methodology with their other programs. CARE’s CSP design supports the new GOSL (2004) national malaria strategy. In practice, however, government and CARE workers are still promoting community cleanup campaigns as one element of malaria prevention activities, along with ITN/LLITN distribution and use. As desirable as environmental clean up campaigns can be, malaria components of the CSP should only promote the current, technically sound approaches that are found in the 2004 revised National Malaria Strategy. Maternal and Newborn Care is a significant component of the project that was planned for introduction in the 3rd year of the program. This introduction will coincide with several maternal care infrastructural changes in Koinadugu, especially the renovated District Hospital and new Maternity Center, as well as introduction of an updated national MCH policy. Some MNC training planned in the DIP, especially TBA HBLSS training, is probably overly ambitious and should be reconsidered. CARE plans to provide MNC technical assistance from an HQ specialist CARE Sierra Leone Child Survival XIX Project Midterm Evaluation Report June 4, 2006 9 shortly after the MTE to develop detailed plans for this component of the program. Close attention will need to be paid to the MNC DIP components and adjustments will be needed in the PVO Action Plan. As IMCI rolls out in Koinadugu District as expected during the remaining life of this project, CARE should reorient staff and partners to C-IMCI and the role that CARE will/should play in its implementation. Given that the community components are CARE’s strengths, the health facility and health worker IMCI strengthening plans in the DIP should be closely coordinated with DHMT and partners (especially UNICEF) to assign appropriate responsibilities within the partnership. CARE reviewed the evaluation team’s comments with field staff and management in Freetown and Koinadugu, and again as follow-up discussion with HQ. Their responses are incorporated into this report and elaborated in an overall Action Plan that is included in Annex XX. A specific follow-up plan for the COPE assessment is included in Annex XX. CARE now has a much better grasp on the implementation issues to be faced in the rapidly changing project operating environment that is now significantly different in some ways from when the program was originally designed. Based on the MTE findings, CARE has taken these changes into consideration, along with the MTE findings, and has used the opportunity offered in USAID’s MTE guidelines2 to review the work plan as part of the MTE Action Plan. Organization of the Midterm Evaluation Report Recommendations and suggestions related to specific interventions or elements are included in the sections of the report covering those areas. Overall conclusions and recommendations to both CARE and USAID are found in Conclusions and Recommendations at the end of the report. B. Assessment of the progress made toward achievement of program objectives 1. Technical Approach a. Overview The project goal is to improve the health status of children under five and women of reproductive age (WRA) in Koinadugu, the largest and one of the most remote districts in Sierra Leone. The project operates in fifty-four villages within five of the eleven chiefdoms of the district. Project activities are implemented in collaboration with the District Hospital and twenty-one peripheral health units (PHUs) in the five operational chiefdoms of Wara Wara Yagala, Sengbeh, Follosaba Dembelia, Dembelia Sinkunia and Neini. The population of these five chiefdoms comprises the primary beneficiaries of the project. Direct beneficiaries include an estimated 39,838 children under age 5 and 56,240 Women of Reproductive Age (WRA). These were adjusted in the Second Annual Report from the DIP total of 48,630 under five and 51,491 WRA (a decrease of 4% in the total number of beneficiaries) after national census in 2004-2005; the census, however, is still under some review and discussion. This population is served by 22 Peripheral Health Units (PHUs). 2 See Section III “Action Plan) USAID MTE guidelines, p. 17 CARE Sierra Leone Child Survival XIX Project Midterm Evaluation Report June 4, 2006 10 The epidemiological picture in Koinadugu is characterized by some of the highest infant and child rates and maternal mortality ratios in the world, caused by very high incidence of communicable diseases such as malaria, acute respiratory infection and diarrheal disease, with high underlying rates of malnutrition. Government-provided health services and facilities in Koinadugu District were devastated by the war and have not yet fully recovered. There was also severe damage to the social services infrastructure, reduced numbers of Ministry of Health and Sanitation staff, well as Health Managers to provide supervision, outreach and other forms of support to outlying health units and communities. According to GOSL national standards there are currently only 13% of the recommended 114 technical staff and 65% of the recommended 26 public health care delivery staff. Koinadugu district currently has a ratio of 5,450 persons per peripheral health unit and only one doctor for every 117,165 people. The CARE CSP is designed to use innovative techniques to build partnerships between communities and government with three principal objectives: 1. Strengthened family and household knowledge and decision-making skills related to the health of women and children resulting in the practice of positive behaviors to prevent, recognize and manage common diseases; 2. Enhanced community capacity to form groups and institutions that sustain health initiatives, demonstrate social cohesion, and promote good governance mechanisms; 3. Improved quality and accessibility of services provided by MOHS personnel and MOHS extension services b. Interventions The DIP design included the following four interventions that are supposed to be integrated through the C-IMCI strategy: The Expanded Program in Immunization (EPI) Intervention (15%) focuses on raising vaccination coverage of children and pregnant women from very low baseline levels. CARE is working with communities and MOHS to promote EPI outreach through the Community Health Clubs (CHCs). The second prong of the EPI strategy encourages CHCs to use appropriate BCC strategies to increase demand for and utilization of EPI services. A third prong consists of regular support to MOHS in organization, communication and logistic support to strengthen links with communities for outreach. The baseline for the project showed that 45.7% of children aged 12–23 months were fully vaccinated (against the five vaccine-preventable diseases) before their first birthday.3 In the first year, the project conducted participatory health education sessions with 56 CHCs on immunization. The primary objective at the time was to emphasize importance and benefits of immunization and to encourage communities to go to PHUs for immunizations. Early formative research showed that men make most of the health decisions in the household and that their understanding of the importance of immunizations would be key to achieving project targets. But women outnumber men at most CHC sessions by 2:1. 3 CARE Sierra Leone, Child Survival Baseline Survey Report, April 2004 CARE Sierra Leone Child Survival XIX Project Midterm Evaluation Report June 4, 2006 11 The project supported the supply of vaccines at the 21 PHUs and facilitated the organization of 56 of 84 PHU outreach sessions conducted from May to September 2005 by the district’s 21 PHUs. UNICEF is currently working with the MOHS to ensure a sustainable maintenance system for the cold chain equipment in the project’s operational locations and CARE provides logistic support as requested for additional assistance. The graph above depicts the number of children in the project area who were immunized for DPT3 within the period from January to July 2005. The greatest numbers of children were immunized after the 2005 LQAS was done. The project support to EPI through outreach, mobilization for national immunization days and supporting cold chain management has resulted in more than 200% increase in the number of children immunized through PHUs in the project area but without sufficient coverage in the entire district to insure against outbreaks. Source: Koinadugu MOHS monitoring data, 2005 Immunization statistics collected at the District Hospital in 2005 still reported coverage rates of beyond 100% based on 1985 national census population estimates. The CSP worked with the DHMT to calculate the catchment population more accurately and get better village-based population statistics. UNICEF also provided technical assistance to the DHMT. A measles immunization/mass ITN distribution campaign, funded by the Canadians, is scheduled in Koinadugu District for October 2006, but the details were not available at the time of the MTE. In spite of project efforts, and substantial increases in coverage of specific antigens (see DPT above), complete immunization coverage before age one year, as measured at baseline and the 2005 LQAS, have stayed about the same. The 2005 LQAS documented wide variations in coverage between chiefdoms. The DHMT, along with UNICEF, decided that routine immunization services, even with significant community mobilization, outreach and national immunization day campaigns would not be sufficient to raise complete immunization coverage to a level high enough to prevent outbreaks. In March 2006, at the same time the MTE was being conducted, DHMT and UNICEF were engaged in a door-to-door immunization mop-up with Children Immunised for DPT3 in the Project Area - 2005 0 100 200 300 400 500 600 700 800 900 Jan Feb March April May June July # of children immunised CARE Sierra Leone Child Survival XIX Project Midterm Evaluation Report June 4, 2006 12 immunizations and deworming medications. The planned 2006 LQAS should reflect the results of these efforts. Maternal Tetanus Toxoid Coverage The project’s M&E system showed a significant drop in maternal TT coverage from the 2004 baseline to the 2005 LQAS. The APM attributed the poor coverage to uneven performance between PHUs in different chiefdoms and poor ANC attendance. The LQAS showed wide variations (from 16% to 58%) in TT coverage between chiefdoms. Another possible explanation would be either poor ANC attendance or “stock outs” of TT at the PHUs. At the time, it was assumed these numbers were consistent with poor overall PHU worker performance in those chiefdoms. No ANC coverage statistics are available anywhere in Sierra Leone, so it is difficult to determine if this is the reason. In spite of these April 2005 findings, the reason for the drop was still unknown in March 2006. Planned MNC TA should include analysis of reasons for the low TT coverage. There are many factors at the PHUs that cause “missed opportunities” for children to be immunized, such as fixed immunization days and reluctance to open a vial of vaccine (particularly the expensive multi-antigens) to immunize one child. These have been standard barriers to coverage since the EPI program began in the 1980’s, and there are specific measures that can be taken to overcome them. Clearly, there are many players engaged in addressing the EPI problems in Koinadugu and CARE would not be able to implement any changes alone. Introduction of the IMCI algorithm to PHUs should introduce checking immunization status and vaccination at every contact with the child. But this will require specific emphasis on immunization as a quality element at the PHUs. This can present a chance for dialogue on addressing missed opportunities. These issues are best addressed at DHMT level meetings that CARE can assist to organize but will require collaboration with other partners, especially UNICEF. (See Health Systems strengthening section.) CARE’s added value in District EPI programs is in community mobilization (with BCC strategy) and continuing support for quality improvement at the PHUs. CHC sessions should emphasize not only that children should be immunized, but the timing as well. The BASICS publication “Immunization Essentials” can be an excellent technical resource for CARE staff and DHMT partner discussions about the way forward. Future population-based immunization surveys, conducted by CARE and others (e.g. KPC and LQAS) will reflect the actions of multiple players, some with CARE’s involvement, some without. CARE should not assume direct responsibility, including vaccine transportation, for assuring the sustainability of immunization outreach. In the remaining time, CARE should shift participation to assisting the DHMT in planning for financing, logistics and supervision, and quality control of immunization services after the CSP has ended. If the maternal TT coverage remains low after the upcoming LQAS (June 2006) CARE should place special emphasis on determining the reasons for the low coverage and addressing them with the DHMT and other partners. In any case, CARE should document what steps were taken after the poor findings in the lowest performing chiefdoms in the 2005 LQAS. EPI activities should be completely turned over to the DHMT by the last year of the CSP. CARE Sierra Leone Child Survival XIX Project Midterm Evaluation Report June 4, 2006 13 Nutrition In the first part of the program, CHC members received training on exclusive breastfeeding, importance of growth monitoring, identification of nutritional disorders including VA deficiency, techniques and recipes for complementary feeding based on locally available foods, balanced diet and diarrhoea management. On average, 38% of participants were men and 62% were women at sessions covering these topics. The lowest turnouts were for the sessions covering complementary feeding, balanced diet and diarrhoea management. After the earlier sessions, new BCC strategies were devised to find better information channels to effectively reinforce the same message about nutrition with men. CARE collaborated with MOHS and other PVOs (i.e. HKI) and mobilized communities for several Vitamin A distribution campaigns. According to MOHS, 74.4% of under-fives in the district received a vitamin A capsule and project LQAS in 2005 showed similar results (73.2%). However, MOHS data show only 7% of post-partum women were covered during the same time period, attributing this to poor postpartum care attendance. (There is no standard MOHS recommendation for postpartum care and Vitamin A statistics are included with EPI, not MNC) CSP data paint a different picture from the MOHS. The 2005 LQAS indicated that postpartum Vitamin A coverage in the project area rose to 52% from a baseline of 18% in the KPC. Documenting successful efforts to increase maternal Vitamin A supplementation is a global problem. Lack of maternal cards and Vitamin A statistics kept with child EPI data are cited as some of the reasons for low recorded coverage, even when high quality interventions are implemented. MOHS statistics for the district indicate that 8% of women in the five chiefdoms where CARE is working received iron/folate during pregnancy. This could be due to the low stocks, low antenatal clinic attendance or poor record keeping on the part of MOHS. Project data, thought to be more accurate, indicate that pregnant women who received/bought at least 90 iron/folate tablets rose from 60% in the KPC to 80% in the LQAS. The project is not tracking consumption, however, and that is a far more significant indicator. There were anecdotal reports that women are not taking iron/folate tablets due to side effects. The CSP catchment area has extremely high diarrhea prevalence, measured to be around 29-30% in multiple studies at different times of the year. The CSP included diarrhea disease control as part of the nutrition intervention. Undoubtedly, feeding practices and diarrhea both play significant roles in the very high malnutrition prevalence among young children in the District. The 2002 MICS attributed 46% of child mortality in Sierra Leone to malnutrition. In the Title II DRP Baseline, children <2SD underweight in Koinadugu were measured at 30.6%.4 Although feeding during illness is mentioned in project CHC training materials, the amount of emphasis given to this extremely important topic needs to be increased. Less than half of children with diarrhea were given increased food and fluids with diarrhea in the baseline KPC.5 LQAS findings a year later were less than half of the KPC findings. 4 CORAD DRP Baseline Study, 2004. p. 23. 5CARE SL CSP Baseline KPC, p. 14. CARE Sierra Leone Child Survival XIX Project Midterm Evaluation Report June 4, 2006 14 CARE has worked at all levels to promote breastfeeding and dietary diversification in complementary foods for children. CARE also provided PHU BF and growth monitoring training, but time did not allow for the MTE team to determine if CARE is implementing the “Ten Steps for Successful Breastfeeding” (DIP p. 33) and is including these elements on supervision checklists per type of facility as planned. CARE reviewed the DHMT Health MIS and found that while under-five growth monitoring information was being collected at PHU level, it was not being analyzed at either the PHU or District level. The project reviewed old nutrition data collected at the PHUs in an attempt to understand some of the nutritional trends at community and district level and used this information to develop plans for integrating the CSP Community Based Growth Promoter (CBGP) information into the MOHS MIS. CBGP training was completed just prior to the evaluation and CBGP groups will begin operation in communities in mid-2006. Even before officially “beginning” Growth Promotion activities, trained group members have helped PHU health workers weigh children and have begun sharing nutrition behavior change messages in the communities. BCC nutrition activities included discussions on using the produce from community and backyard gardens that have been promoted through the Title II and other programs in the communities. Community focus group discussions indicated that they now keep approximately 1/3 for seed, 1/3 for consumption and 1/3 for sale. The most frequently cited use of the proceeds from sale is to pay school fees. The project is tracking several indicators; the M&E matrix lists three indicators for nutrition: one for breastfeeding initiation, one for exclusive breastfeeding and one for introduction of complementary feeding and continued breastfeeding. In the comparison matrix presented at the MTE, an indicator for deworming was included. Discussions with CARE indicated that the project is also monitoring some indicators that may not be included in the formal CSP M&E plan, but are important for District level efforts. Focus group discussions in communities as well as the LQAS indicate that significant changes in breastfeeding and complementary feeding behaviors have taken place at the household level. Community members already detect improvements in their children’s health and attribute many of these changes to nutrition behaviors. Changes in the types of foods provided to young children appear to have been fostered by removing taboos associated with certain foods through the project BCC strategies. Responses to focus group questions about feeding frequency, however, did not consistently show clear understanding of the number of times per day small children need to be fed. This may be a reflection that the CBGP groups are not yet active in communities. This first LQAS, conducted approximately 1½ years after initiation of the project, showed that 62.5% of mothers initiated breastfeeding within 1 hour after delivery and 32.7% of babies 0-5 months were exclusively breastfed. Of 61 respondents in the LQAS, 96.7% indicated giving their youngest child 0-23 months hot water within the first three days after delivery, 37.8% gave milk CARE Sierra Leone Child Survival XIX Project Midterm Evaluation Report June 4, 2006 15 and 32.9% gave fruit juice indicating that the meaning of “exclusive” breastfeeding will need clarification for correct self-report and interpretation of survey results.6 Although the project has promoted “exclusive” breastfeeding from the moment of birth through CHCs and recent CGBP training, it is always worthwhile to revisit and re-emphasize this message throughout the life of the project as this type of traditional behavior is often more resistant to change. Linking diarrhea with nutrition is very appropriate and is complemented by the strong emphasis on environmental sanitation measures supported by the CHC training. In some communities this training was started by CARE’s Water and Sanitation programs. Point of Use (POU) water treatment strategies have not been incorporated into the program. The CHC training manual, however, contains outdated information promoting Sugar-Salt-Solution (SSS) for home base diarrhea treatment. This approach was discarded by the WHO over a decade ago when it was discovered that even when training programs were extremely high quality, caretakers tended to make mixtures dangerously high in sodium. The TRMs have counseled against promoting SSS for many years, instead advising the promotion of home-based fluids and ORS. This residual policy is probably a reflection of the information isolation in Sierra Leone due to 15 years of civil war. When this information was pointed out to CARE, they immediately agreed to address this in the curriculum and the training materials. It is recommended that CARE replace promoting SSS with ORS, where available, and/or home￾based fluids along with continued/increased breastfeeding. This information should be specifically shared with the DHMT and included in PHU personnel training. Depending on the quality of IMCI training curricula in SL, this information may already be included. The results of a recent national nutrition study showed alarmingly high nationwide rates of malnutrition, but the quality of the results have been questioned. Baseline Title II anthropometric findings showed that malnutrition in the CARE areas in Koinadugu exceed 30% (weight/age). Currently, Title II is distributing food in 21 out of the 56 communities where the CARE CSP is working and WFP is distributing food in other communities. The CARE DIP planned to introduce Hearth in Year 3, using CBGP data for selecting eligible communities. CRS is piloting Hearth in the area, but the results of the pilot have not yet been made available. Given that CBGP 6CARE CS XIX CSP Lot Quality Assurance Survey Koinadugu District Report April, 2005, p. 47. One female CHC member, a mother of two small children said during one of the MTE focus groups: “I gave my first child hot water and began feeding her solid food shortly after she was born. She was sick all of the time and didn’t grow well. With this child (sitting in her lap), I exclusively breastfed and she has been very healthy”. The younger child had bright eyes, shiny hair, was very active and giggled while another child played with her. CARE Sierra Leone Child Survival XIX Project Midterm Evaluation Report June 4, 2006 16 groups will not begin operating in communities until the second half of Year 3, it is likely that Hearth, if it is introduced at all, will not be introduced until Year 4. The CARE HQ backstop, who is a nutritionist, is collaborating with CRS’ Headquarters staff to refine the plan to use the lessons learned from the CRS Hearth pilot as the basis for introducing Hearth in the CSP communities. Experience in other CSPs has shown that without careful preparation, what is promoted to be the PD/Hearth approach reverts to cooking demonstrations, nutrition discussions and little more. Wisely, CARE is showing prudence by making sure that the project will be able to implement a technically strong Hearth approach before beginning. The evaluation team leader has found that some other PVOs who are using the PD/Hearth methodology in other countries, while impressed with how effectively it rehabilitates malnourished children using local resources, are also finding that implementing PD/Hearth effectively is relatively labor intensive and requires significant supervision. Although included in the DIP, at this point in the program, CARE has decided to consider Hearth and make the decision after learning from CRS' experience. CHC groups are already doing nutrition education, and these activities will be renewed, stressed again, and added to be combined with more ENA education. (Note that CARE Sierra Leone is also involved in discussions with the regional office of HKI to see if collaboration on a Hearth model for pregnant women would be useful to implement as part of the MNC strategy.) These proposed actions seem to be appropriate. It is recommended, however, that CARE revise the program description in the Action Plan to reflect these more tentative plans. If the decision is ultimately taken to not implement Hearth, alternative plans for referring children with severe malnutrition (e.g. to the District Hospital) should be promoted by the program. In addition, lessons learned and essential considerations regarding whether or not to pursue Hearth in this environment should be documented in Annual Reports. CARE is very fortunate to have an experienced Child Survival nutritionist as the CSP HQ backstop. With emphasis on ENA in this year's CARE Annual Child Survival Workshop and follow up by the CSP HQ backstop, the project should be able to build upon previous nutrition training to develop a full ENA strategy and increase areas that require more emphasis, such as sick child feeding. Possible additional approaches should be considered. During the MTE, CARE HQ agreed to review all project materials and training approaches to ensure that they are aligned with ENA. Training materials with regards to diarrhea case management should be updated to coincide with the TRMs. Extra feeding and fluids (including breast milk) during illness should receive special emphasis in project BCC efforts. CARE should engage the MOHS Planning and MNC offices in Freetown to discuss ways that maternal Vitamin A supplementation coverage can be more accurately and sustainably recorded in the MOHS HMIS. The malaria intervention (35%) sections of the DIP planned to train peripheral health unit staff to recognize malaria and provide standard case management, educate community members about malaria and its treatment, promoting intermittent presumptive treatment of malaria amongst pregnant women and promoting and distributing insecticide treated mosquito nets (ITNs) through social marketing. Social marketing was later changed in favor of free distribution when CARE Sierra Leone Child Survival XIX Project Midterm Evaluation Report June 4, 2006 17 supplies of LLIN became available from UNICEF in Koinadugu, and supplies for social marketing were difficult to obtain. Sierra Leone has removed tariffs on ITN, so the possibility that the private sector will be able to develop is better now. A new national MOHS malaria policy was developed in 2004 and closely follows international RBM protocols. This change impacted the CSP in several ways: 1) the first line malaria treatment was changed from chloroquine to artesunate-amodioquine combination therapy (ACT) 2) Intermittent Presumptive Therapy (IPT) using S/P was officially adopted and 3) increasing ITN coverage took on importance as the major malaria prevention strategy. Free distribution of ITNs through MCH programs is now promoted. Prevention The CSP is fortunate to have had access to Long Life Insecticide-Impregnated Nets (LLIN), which have effective life-spans of up to 5 years without the need to be retreated. This removes a major constraint in effective ITN coverage. UNICEF/DHMT/CARE partnered to distribute 5,490 LLINs, and is poised to assist in distribution of 11,000 more nets that are already available at the DHMT as soon as UNICEF releases data monitoring forms. (Hopefully this will happen before the height of the 2006 rainy season.) All partners in the ITN distribution agreed that CARE added specific value by collaborating with CHCs to promote and locally monitor the actual use of LLINs at the household level. This household level surveillance is an extremely important contribution that the worldwide RBM movement has had great difficulty achieving. This could be a methodology that CARE could bring to solve some of the constraints in this key RBM area. Measuring the full effectiveness of the CSP’s ITN efforts, however, will be difficult. Due to problems following the KPC skip pattern, only families with children that had fever were asked about ITN usage the night before in the baseline KPC, probably underestimating baseline ITN coverage. (Although national ITN coverage at the time was estimated to be less than 2% from other surveys.) The 2005 LQAS contained this same error (coverage at 18% among children with fever, a low denominator). This was measured prior to the mass distribution. A special malaria￾only LQAS was conducted after the late 2005 mass distribution, only in communities with distribution. This was done in particular to observe if nets were hung up and had signs of use where children or pregnant women would be sleeping, and the survey skip error was corrected. Results showed 85.3% of children under five to have slept under an ITN the night previously. The National Malaria Policy coverage target is 30% by 2008. It would greatly benefit the national RBM effort if Koinadugu District health partners collectively strived for as high ITN coverage as possible, while acknowledging this coverage will be limited by access to nets. An unfortunate holdover from old African malaria prevention policies, promoting brush clearing and burying, was found in the project and partner health promotion activities. This approach was mentioned in project messages and training materials, along with promotion of ITN use and early care-seeking for fever, and was the most frequent prevention measure mentioned in focus group interviews during the MTE. It was well established over 15 years ago that these measures may decrease A. aegypti, carriers of yellow fever and dengue (not major causes of child mortality in the area) and some nuisance mosquito species, but are totally ineffective against Anopheles spp the carriers of malaria. While environmental cleanliness is extremely desirable and has multiple health benefits, promoting it can give false assurance to communities that they will reduce CARE Sierra Leone Child Survival XIX Project Midterm Evaluation Report June 4, 2006 18 malaria. Malaria is the major cause of child mortality and the CSP should emphasize prevention measures that have been proven effective. While the source of this practice is likely to have been the GOSL MOHS malaria policies in the past, it is absent in the 2004 National Malaria Strategy. Case Management The change in malaria treatment protocols rendered the KPC baseline figure for children treated by an effective antimalarial drug meaningless. This is no reflection on the CARE CSP KPC which measured chloroquine use, the first line drug at the time. The project appropriately adopted a new baseline of zero just prior to the roll out of ACTs in Koinadugu. The availability of ACTs presents a wonderful opportunity to provide children with effective treatment for malaria. CARE facilitated distribution of UNICEF-donated ACTs on behalf of the DHMT, including the innovative approach of obtaining MOUs with VDCs to account for the drugs and developing an inventory data control system. This introduces community accountability for PHUs. These contributions are laudable, but CARE appears to be much more operational than is recommended for sustainability. Also, CARE risks being viewed as a type of drug inventory police if the DHMT is not seen as having the ultimate responsibility for drug use and stock maintenance by health facilities, with transparency through input from VDCs. The worldwide introduction of ACTs link sustainable use of them to improved diagnosis capacity because they are considerably more expensive than previous malaria treatment drugs. Since providing large-scale increases in laboratories isn’t practical, and health manpower limitations make slide-positive diagnosis unrealistic, Rapid Diagnostic Tests (RDTs) are being promoted internationally as appropriate to distinguish malarial fevers from other causes and were included in the DIP. CARE has not started introduction of RDTs as described in the DIP. They are not currently available in the country, and have not yet been introduced as part of the National Malaria Control Program. In any case, since RDTs are not 100% sensitive to all cases of malaria in children, febrile child CSP beneficiaries are treated symptomatically per international guidelines in MOHS facilities. Currently there are no serious consequences since adequate free supplies of ACTs are available. Given CARE’s current limited role in clinical malaria case management, it is appropriate not to pursue RDTs at this time, but this should be clarified in the Action Plan. On the other hand, treating fevers with ACTs raises serious sustainability concerns for the time when UNICEF’s funding for drugs runs out and Sierra Leone is required to pay for the drugs. The long-term prospects for continued access to ACTs probably lie with the Global Fund assistance to the MOHS and CARE should only play a facilitative role in District plans for the future. While correct treatment of children with a febrile episode is a project indicator, and health worker malaria case management training was part of the DIP,7 CARE’s role in improving the quality of malaria case management has been limited. UNICEF provided the training for updating malaria case management. CARE staff attended the training, but did not facilitate it. To date, baseline malaria clinical treatment quality assessments (health worker competency to diagnosis, treat, follow-up, and refer appropriately) have not been done—by CARE, UNICEF or DHMT. It remains a missing element in the ability to measure improved access to quality malaria case management care. It is also not clear if the overlapping presentation of pneumonia 7 DIP, p. 36. CARE Sierra Leone Child Survival XIX Project Midterm Evaluation Report June 4, 2006 19 and malaria was covered in any training that PHU health workers have received—either in the past, or recently. Project surveys place more emphasis on care-seeking for cough and difficulty breathing, rather than fever with difficult breathing that would be the more important consideration in a gravely ill child. At the time of the MTE, it appeared the DHMT was not clear yet of national plans for further training of staff, and was not seeking for CARE to provide PHU capacity building in this area. Good IMCI health worker training would address many of these concerns as the vertical malaria standard case management is replaced with an integrated approach that addresses the other factors in assessing and treating the ill child. IPT There is evidence that IPT with SP is now available at the CSP area PHUs. CARE helped to distribute the S/P to the health facilities and included the drug on the inventory lists. There is great confusion in communities, however, as to the number of ANC visits a pregnant woman should make. IPT is linked with ANC, (at least 2 visits after quickening are required at least one month apart) whether facility-based or in outreach sessions. IPT is one of several project interventions that will require increased ANC coverage to increase the coverage of the intervention. (See MNC intervention for more information.) Follow-up and next steps: ß CSP/DHMT/UNICEF collectively establish a Koinadugu District ITN target coverage and the CSP closely capture the CSP (and the new CARE MOSI malaria project) contributions to achieving these targets. The CSP should not view attainment of the original DIP ITN targets as justification for reducing level of effort in this part of the program as they fall far short of desired coverage. (Abuja targets are 60%) ß CARE should document the lessons learned in community-based ITN surveillance, especially where the CHC model can be shown to provide a unique contribution to attainment of national RBM goals. CARE should share these lessons learned within the PVO and RBM communities, recognizing that CARE may need to be proactive in obtaining opportunities to share these experiences. Following the next large distribution would be a good time to do this. ß CARE should turn over responsibility for monitoring ACT supplies to the DHMT as soon as possible. If not feasible, then a DHMT representative should accompany CARE on each monitoring visit and cosign any reports. Follow-up on any irregularities and accountability to UNICEF should primarily be the responsibility of the DHMT, but could also be discussed at CSP partners meetings. The Maternal and Newborn Care (MNC) intervention (30%) aims to improve access to information and basic maternal health care by providing educational sessions on danger signs recognition and birth preparedness at the community and household level, promoting Tetanus Toxoid (TT) vaccination and iron supplementation for pregnant women and VA supplementation for postpartum women, and training PHU staff in intermittent presumptive treatment of malaria in pregnant women. Major MNC efforts in the CSP were not planned until after the MTE. The CSP has trained CHC members in recognizing danger signs and encouraging skilled delivery at health centers. Consumption of iron/folate tablets is also promoted, as is postpartum CARE Sierra Leone Child Survival XIX Project Midterm Evaluation Report June 4, 2006 20 vitamin A. All indicators have shown increases, with the exception of maternal tetanus toxoid vaccination. Wide variations in vaccination coverage have been found in the chiefdoms and will be investigated to try to uncover the reasons for the poor coverage. Community and household awareness of maternal (with the exception of prolonged labor) and newborn danger signs is high, as is the appropriate action to take when these signs are noticed. Some preventive measures, such as iron/folate and malaria medication during pregnancy are known, but the appropriate number of antenatal visits is not. Community members’ responses to “How many times should a pregnant woman go for antenatal checkup” ranged from “daily” to “once” during pregnancy.8 The CARE “Communicating Health: Communicating Rights” CHC training manual stresses the importance and promotes antenatal care and specific antenatal actions, but does not quantify the number of visits or actions. CARE has (very wisely) deliberately targeted men, and to a lesser extent, grandmothers as decision-makers in developing their community-based MNC strategy. A separate chapter for men’s involvement in pregnancy is included in the CHC training curriculum. The curriculum does not cover the key messages for post-partum and newborn care (when over 40% of mortality occurs). The KPC identified that most frequent immediate action taken with the newborn after delivery is to place the child on the floor, causing significant risk for hypothermia and infection which are both major causes of newborn mortality. Additional training sessions for post-partum and newborn care should be developed as part of the MNH strategy, complementing the previous basic maternal health care training provided to CHCs. The DIP provides for training TBAs in HBLSS. At the time of the MTE, CSP managers stated there were no plans to train TBAs, outside of their involvement as members of CHCs or CBGP groups. One reason for this may be because other NGOs and UNICEF are providing funds and facilitation to the DHMT for this. CRS has trained TBAs in the program area and CRS-trained TBAs are now part of CHCs and are seeking support to replenish supplies for their TBA kits. CSP is not clear what the TBA training entailed and has not yet tried to directly coordinate with this activity; this will be investigated as part of developing the Maternal and Newborn Health strategy, to be initiated during the second half of the project. The World Bank provided the DHMT with two new ambulances for obstetric and other health emergencies. The District Hospital is undergoing major World Bank-financed renovations and District Hospital services are temporarily operated out of a community health center. UNICEF is building a brand-new Maternal Child Center adjacent to the new District Hospital which will have new delivery facilities. When the hospital renovations are completed, comprehensive EmOC services should be available there (including blood transfusion, cesarean section, intravenous antibiotics, etc.) and overall maternal care services should be much better quality than in the past. Until the renovated hospital opens, only blood transfusions and antibiotics are available in Kabala. Until then, women must be transported an additional 1½ hours away from Kabala for Caesarean Section. 8 Follow-up with the MOHS MNC section revealed that policy is now being written and no standard number of prenatal visits has been promoted up until now. CARE Sierra Leone Child Survival XIX Project Midterm Evaluation Report June 4, 2006 21 The World Bank, however, has not provided the means for the PHUs to communicate with the District Hospital to advise when the ambulance should be sent to pick up emergency cases. Cellular telephones will be soon available in the areas surrounding Kabala, but it is not known how far the service will extend into the rest of the District. Even then, electricity to charge them will not be available at most PHUs. Clear plans for paying for fuel for the ambulances are also not in place. The DHMT is looking to UNICEF, NGOs, communities, or families themselves to pay for two-way radios as well as for assistance to address the fueling issues. Prior to the war, the District had radios for communication and the system, according to the DHMT, worked well. Currently money for fuel for the ambulance is coming out of the pockets of DHMT and NGO staff. Loans to families for this purpose are generally not repaid. (It should be noted that during presentation of Key Findings of the MTE in Freetown, UNICEF mentioned that they have had significant difficulty in identifying a source for radio repair and maintenance in Sierra Leone or the Western region of Africa. So this obstacle to improved maternal health does not yet have an easy solution in sight.) In some areas, with the encouragement of CARE and other NGOs, communities have organized hammock brigades to transport women to PHUs, where very limited care is available for women experiencing obstetric complications. Transport to the District Hospital via hammock is not realistic for most communities due to very long distances and rough terrain. Experience in other CSPs has shown fueling ambulances to be the Achilles’ heel of emergency transport strategies. CSP partners, including the DHMT, District Council and UNICEF, will have to plan very carefully in developing protocols that will be sustainable without outside input after the current program ends. IRC pays for ambulance services in its CSP with project funds. CARE can learn the pros and cons of this approach in the upcoming IRC CSP MTE. CARE has already begun discussing birth planning and community-managed revolving funds for emergency transport to the District Hospital with the CHCs. It is not clear whether lessons learned from revolving-loan funds in other CSPs have already been shared with CARE and DHMT managers. CARE has significant experience with community-based revolving funds. Additional experience has been documented by Save the Children (Guinea) and other PVOs working with mutuelles in francophone African countries. The CSP relies heavily on MOH policies for messages and actions promoted in the program. As of the MTE, MOHS MNC policy is being written. This could be one reason for some confusion on what MNC messages the CSP should be promoting. MOHS has said that it welcomes CARE’s input as the policy rewriting process is underway. This presents an opportunity for CARE to have input and demonstrate leadership in bringing Sierra Leone’s policies and practices in line with current international standards. Lack of awareness and coordination between CRS, DHMT and CARE MNC activities is symptomatic of the overall current limited coordination among health and nutrition activities at the District level. CARE’s assistance with helping the DMO call together all health partners in the District for better planning and coordination should go a long way towards helping alleviate some of these problems. CARE has made multiple attempts to coordinate activities with the previous DMOs, but is more optimistic for success with the newest DMO. CARE Sierra Leone Child Survival XIX Project Midterm Evaluation Report June 4, 2006 22 Lack of reliable information about baseline MNC services, which would primarily be found in maternal cards and facility maternal registers, may be a major hindrance to assessing progress in improving maternal behaviors in the CSP. Lack of registers was identified in the COPE assessment and CARE has provided them since then. Many CSP indicators are linked to ANC attendance (TT, IPT, ITNs after the campaigns, etc). Raising ANC participation and quality of care is essential to achieving higher coverage of these indicators. In addition, current MOHS MNC data collection excludes information on post-partum and newborn checkups and maternal post-partum vitamin A supplementation in MNC documents. The lack of baseline MNC-specific health worker performance data (e.g. quality of labor and delivery and immediate newborn services for facility-based deliveries) will make assessing impact of CARE’s MNC capacity￾building activities on access to quality MNC clinic services difficult at the end of the program. In addition, identifying the best strategies (e.g. more outreach vs. promoting higher clinic attendance) will require better data for decision making. Helen Keller reported successful results in a Maternal Positive Deviance pilot study in nearby Guinea which has cultural and economic similarities to Sierra Leone, and has been interested in sharing this experience and contacting NGOs in Sierra Leone. CARE CSP managers have discussed integrating the lessons learned into their program, but no decision has been made. Follow-up and next steps At the time of the MTE, it was agreed that CARE HQ will arrange for MNC technical assistance to the CSP as soon as possible after the MTE is completed, preferably in July or August 2006. This technical assistance will be arranged by CARE Child Health cluster with the HQ Sexual and Reproductive Health Cluster, both within the Health Unit, and should include time for the HQ advisor to accompany the new CARE Sierra Leone Health Sector Coordinator and meet with the MOHS MNC and Planning offices. These meetings will be to discuss policy plans as well as appropriate roles for CARE working with the DHMT and other partners in Kabala. CARE should resist agreeing to unsustainable actions under the CSP and focus primarily on behavior change, improved clinical practice and realistic capacity building in the remaining time of the project. Either coinciding with, or following the MNC advisor’s visit, CARE should assist the DMO to call a meeting of all major MNC players in Koinadugu with the objective of coordinating roles and responsibilities into a District MNC plan of action. Appropriate parts of this plan should be added to CARE’s CSP work plans, including the COPE follow-up, for the duration of the project. Assessing the monitoring and evaluation aspects of the MNC intervention should be part of developing the MNC strategy. Maternal/newborn verbal autopsy skills training could be considered for CARE, DHMT, and other partners and integrated into the Monitoring and Evaluation plan. CARE could consider using the BASICS/Save the Children Social and Verbal Autopsy publication based on lessons learned in nearby Guinea, a country with many similarities to Sierra Leone. Information from the enhanced M&E should be fed back into CSP program planning. As mentioned, CARE should be engaged in the current MNC policy dialogue at the central level and advocate for including updated MNC areas of emphasis in official MOHS CARE Sierra Leone Child Survival XIX Project Midterm Evaluation Report June 4, 2006 23 documentation and data collection. Some of these elements are required for GOSL reporting on Millennium Development Goals (MDGs). Measuring health worker performance in conducting ANC, deliveries and postpartum care should be included in the M&E recommendations. CARE should not wait for the results of the MNC consultancy to promote placing newborns with their mother immediately after delivery (covering both mother and child to prevent hypothermia.) This can be the responsibility of the family or CHC member because the birth attendant often has to choose between helping the mother or the baby. These actions can be integrated with the already-introduced breastfeeding promotion and refined later, after the details of the MNC TA are decided. Since so many program activities in several interventions (nutrition, EPI and malaria) are linked to receiving services at ANC, promoting clear messages as to the number of appropriate visits and what clients should expect at these visits should be included in CHC and VDC training. CARE should be engaged in the dialogue with the central MOHS MNC offices in making sure that the new MNC policy includes specific recommendations about the recommended number of ANC and PP visits as well as standard services to be offered. CARE HQ could obtain and share documentation from other CSP revolving-fund and community based emergency transportation schemes with the CSP partnership to guide them with their planning. Save the Children’s Mandiana program in Guinea is only one of several that have relevant lessons to share. Plans for this activity should clearly be addressed as part of the MTE Action Plan. c. New Tools and Approaches CARE used the COPE for Child Health methodology, developed by EngenderHealth, to assess several quality health care elements in the DHMT PHUs. The method is highly participatory and produced an action plan with timeframes for follow-up. During the MTE, the action plan was reviewed to assess the status of follow-up actions and to select priorities among those actions which had not yet been completed. (See sections on health system and health worker strengthening. A summary of the COPE action plan follow-up is found in Attachment XX) 2. Cross-cutting Approaches (technical interventions) The CSP project design is based on integrating technical approaches within C-IMCI9 . IMCI within the MOHS health facilities is planned only now for introduction into the Koinadugu DHMT. Although many elements of C-IMCI are incorporated into the CSP, it is not yet the operating framework of the program. The DIP also described collaboration with MOHS in strengthening health worker clinical skills in health facilities as part of CARE’s role in improving quality of health services. At the time of the midterm, CARE staff felt that the DHMT, with MOHS and UNICEF support, had primary responsibility for clinical performance improvement at the PHUs, and CARE has sought a supportive role. While CARE has devoted considerable time and effort to improving many of the 9 CARE Sierra Leone CS XIX DIP, pp 1-3. CARE Sierra Leone Child Survival XIX Project Midterm Evaluation Report June 4, 2006 24 elements that are necessary for quality care at facilities, the exact nature of that supportive role, specifically related to health worker clinical performance (assessment, diagnosis, treatment, referral, follow-up, etc.) is not clear. Since no assessments of clinical case management performance have been conducted at health facilities in the project area, it is not possible to determine if any improvements in those specific quality elements have taken place. The perspective of CARE’s implementation role in meeting program objectives with regards to clinical performance (especially objective 3) as written in the DIP was based upon an assumption that the MOHS would roll out IMCI training in the district with support from actors such as UNICEF, while CARE would play a supportive role (as occurs in most CARE CSP). IMCI training has not proceeded as planned and how Objective 3 will be addressed will need to be clarified as part of the MTE Action Plan (Changes in specific intervention approaches have already been addressed in those sections of the report.) Cross-cutting approaches (program design) a. Community Mobilization Community Health Clubs (CHCs) The foundation of the CSP community mobilization (and much of the BCC strategy) is based in the Community Health Clubs (CHCs) in each of the project communities. [The following information is an in-depth look at CHCs, the primary community mobilization approach of the CARE CSP. This description includes factors beyond health effectiveness and includes many of the additional specific issues USAID would like the report to address including equity, contributions to scale-up and to developing civil society. In addition, CARE SL and CARE HQ asked specific questions about certain aspects of the CHCs while developing their most recent country strategic plan. (These elements are in italics.)] · The socio-economic implication of CHCs CARE’s CHC model scales up innovative approaches that strengthen civil society, and address equity (especially with regard to women’s participation in public meetings). Complementary activities undertaken by the Village Development Committees (VDCs), with encouragement from the CSP, include construction of schools and mobilizing support for girls to go to them. There is a perception that girls attendance in schools has significantly increased. CHCs empower community members by helping them to understand the extent that their health status is under their control. They also stimulate identification of local solutions to problems. In spite of the overall poverty in the program areas, CHCs assure that pooling resources and revolving loan funds can provide much better access to health services than were previously available. By emphasizing women’s involvement and the role that education plays in improving health status, CHCs mobilize community decision-making processes to overcome prevailing cultural norms and behaviors. Health and nutrition behavior change promoted through the CHCs (especially CBGP groups) are overcoming taboos, especially with regard to foods that are fed to young children and care seeking behaviors. Membership in CHCs is higher among women than men. This may be related to times when community members of specific genders are available but is also related to CARE's specific desire to ensure that women have a majority presence in CARE Sierra Leone Child Survival XIX Project Midterm Evaluation Report June 4, 2006 25 the groups, while men are also more than "tokenly" involved. CSP staff and CHCs are exploring ways to encourage higher male participation in areas such as Follosaba Dembelia Chiefdom, where male participation is only 26% (see below). Table Membership in Community Health Clubs 10 Chiefdom Male % Female % Total Dembelia Sinkunia 139 39% 219 61% 358 Wara Wara Yagala 193 40% 288 60% 481 Follosaba Dembelia 113 26% 326 74% 439 Sengbeh 178 36% 315 64% 493 Neini 57 51% 54 49% 111 TOTAL 680 36% 1,202 64% 1,882 Other programs worldwide have discovered that experience gained through community structures such as CHCs has spill-over effects into other civil society sectors and support greater participation in democratic processes. Education, women’s participation, and supporting access to health care by repairing roads are the most obvious activities occurring in CSP communities at this time. Creating work plans with communities has developed planning and implementation skills. Developing MOUs with communities and involving CHCS and VDCs in oversight of drug distribution to PHUs has helped improve linkages between health facilities and communities (part of C-IMCI) and increased the accountability of community and MOHS structures. Involving VDCs in monitoring ITN use after they were distributed helped assure the nets were used by the target population and not sold and has contributed to community-based health information systems. Health has proved to be a very good entry-point for community development, and CHCs are excellent community structures to serve as the foundation for a variety of development activities. · Challenges of the approach in general as well as of the process of translating it into action. The Child Survival Project completed training CHCs in 2005, just before the end of year 2 of the project. An additional Community Mobilizer was hired to maintain a consistent amount of contact with CHCs. Although CHCs are supposed to consist of no more than 30 members, some CHCs have up to 60 members each. After membership gets too large, the group is split in two. The project would be challenged to maintain open membership while at the same time limiting membership to 30 members. If limited to 30, either the number of meetings per community would increase considerably, as would the workload of the community mobilizer, or some current members would need to be excluded. Project staff state that measures have been taken to schedule meetings at more convenient times for men, seeking a 60/40 balance women: men. Project data will need to be monitored to see if the participation of men increases as a result of 10 First Annual Report October 2004. CARE Sierra Leone Child Survival XIX Project Midterm Evaluation Report June 4, 2006 26 these changes. Males are also reached through the VDCs which have a significantly lower percentage of female members. The sustainability of CHCs without CARE mobilizers is unknown at this point. The second half of the project will require development of an exit strategy where mobilizers gradually decrease the frequency of contact with each individual CHC and turn over the support to the MOHS/PHU staff and/or District Council. While interest and enthusiasm for CHCs is high, it is not known how long attendance will stay at current levels. It might be reasonable to wait for a period of time before determining the actual number of members who are likely to be sustained in the long term. Another major challenge will be to see how technical updates and MOHS policy updates can be integrated into the training at the CHC level. Since the approach uses cascade training strategies, these updates need to be planned at each level: project management staff, program staff and DHMT/PHU counterparts, field staff and finally CHCs. Updates will also need to be disseminated to other projects. Although the CHC is already being scaled-up by CARE and other organizations that are using CHC training materials, they are providing the training without the assistance of CSP staff. Updating the training materials might be accomplished with yearly, or at least periodic, updates on the CHC training materials. This could be in the form of a newsletter, or something similar. CARE should estimate the start-up and recurrent implementation costs of CHCs including training and supervision, if the approach is to be replicated in other programs. (Scale-up) CARE would also need to place the CHC implementation in context by assessing other activities taking place in the same area. For instance, are women really attending PHUs for ANC more frequently due to CHCs promoting ANC, or is it the WFP or Title II food that is attracting them and attendance will fall once the food stops? · The effects and relationship of the CHC structure on other village based institutions such as Village Development Committees (VDCs) The direct relationship of the CHC structure on VDCs has consisted of involving them in CSP interventions. Some of these include: collecting demographic statistics, ITN distribution and participating in mini-surveys on their use, distributing radios, participating in documenting receipt of antimalarials at the PHUs, restructuring the VDCs and participating in health orientations. Indirect effects include collecting funds for emergency transportation (including proceeds from community and backyard gardens), increased women’s participation in public discussions, and increased girls’ school attendance. Construction of community schools has also increased and VDCs have hired their own teachers. CHCs have been credited with motivating VDCs to do road and bridge repairs to provide access for PHU health outreach activities. Although difficult to directly attribute to the CHCs, it would seem that the approach used in the program has helped move away from the “hand-out” expectations that communities have in relation to NGO programs and towards plans for sustainability after the CSP ends. This assumption should be followed up in the program monitoring in the second half of the project. CARE Sierra Leone Child Survival XIX Project Midterm Evaluation Report June 4, 2006 27 ß Community participants (CHC members) – non-participants perceptions on the function of CHCs Because of time constraints and perceptions that the majority of community members were participants, non-participants were not interviewed during the MTE. It would be a good idea to include this in the follow-up monitoring of the CHCs in the second half of the program and as part of the Final Evaluation. · Field staff, Ministry of Health (and other stakeholders) perceptions/challenges with the implementation of CHCs DHMT representatives, District Council, Community and Religious Leaders all confirmed that the CHCs were the key factor in the community mobilization that has made the positive changes in maternal and child outcomes since the beginning of the CSP. CHCs/VDCs, field staff and stakeholders are all challenged to deal with improving communication between PHUs and the District Hospital in order to access emergency care. These improvements will be especially important to support the malaria and MNC components of the program. In addition, the extremely poor roads in many parts of the project area make access to emergency health services extremely difficult. Although he is not aware of CHCs specifically, the new District Medical Officer (DMO) views community mobilization and outreach as essential to increasing coverage of key child survival health services. · Missed opportunities – what could add value to the CHC approach? Linking CHCs to income generating activities (through additional projects) could help address the underlying poverty that leads to poor health. At the same time, CARE should follow-up on complementary activities (part of the C-IMCI “multi-sectoral platform” necessary for sustained health improvement). For example, where water and sanitation activities have been implemented, CARE should design follow-up monitoring (along with the District Council) to see that those interventions are sustained. New CARE activities (such as the Title II program renewal) should be designed to complement, reinforce and sustain CSP activities. · To what extent is the CHC communicating Rights? The CHC training program includes CARE’s Rights Based Approach as an integral component of the program. The effect is already evident in higher participation of women in public discussions and community structures (CHCs and CBGPs). Multilingual discussions reinforce the need for everyone to participate. The most noticeable effect has been the dramatic increase in women’s participation in community health activities, and an increase in the enrollment of girls in the community supported schools within the project coverage area (a Dimension III indicator in this project’s Child Survival Sustainability framework. Other community mobilization activities undertaken by the project (input from the MTE Team) CARE Sierra Leone Child Survival XIX Project Midterm Evaluation Report June 4, 2006 28 Mobilization activities with CHCs include health sessions along with certificates for participants, CHC monthly activity plans, distribution of radios to all communities through CHCs, community mapping exercises, participation in NIDs, “Sensitization” campaigns (e.g. World AIDS Day), CHC competitions and establishing community and backyard gardens Mobilization activities with VDCs include formation and/or restructuring of VDCS, ITN distribution along with mini-surveys on household ITN use, distribution small radios to CHCs, and collaboration on antimalarial drug distribution to PHUs, Both CHCs and VDCs contribute to community data collection on PHU outreach sessions, National Immunization Day Strategy (NIDS), ITN distribution. They also participated in CBGP volunteer training and certification. They also promote dialogue between PHU and communities by organizing meetings between them. In addition, both organize dialogues with communities on project activities and make community referrals to PHUs and construction of TBA delivery huts. In schools they have organized health education quiz competitions and constructed community schools. Basic hygiene facilities such as drying racks and latrines have been promoted. Community dramas and Radio Binumani have been used as major communication venues due to high rates of illiteracy. All CHC and VDC activities are contributing to community cohesion and refining program implementation plans through community meetings, CHC cross-information (outreach), construction of sanitary facilities, increased clinic attendance, CHC monthly activity planning, and identifying health problems and developing practical solutions to address them. In addition, updating demographic data, sharing health information, teaching conflict resolution and encouraging participation in MOHS-organized activities foster greater participation in civil society. Barriers that prevent community members from benefiting from the program include: cultural barriers (especially for women speaking in public), taboos and that prevent pregnant woman from eating meat and egg (making compliance with project teachings difficult), beliefs that prolonged labor is due to infidelity (which can only be “cured” through confession during labor), customs that keep postpartum women from leaving the house (“hibernation”), patients refusing to use the hammock for transport, the fact that women do not make decisions about health care for themselves or their children; “tribal sensitivity”, and (very) high illiteracy rates. Cultural taboos are addressed through community health education (e.g. through CHC sessions and CBGP training), outreaches, and repeatedly encouraging women to actively participate in decision making through sensitizing communities. Tribal sensitizations are addressed through community sensitizations that include all decision-makers. Illiteracy is addressed through encouraging girl’s education and community schools. Adult literacy programs are sorely needed in the area, but are not a focus of the Child Survival Program. The need to frequently reschedule planned activities while trying to keep to the CSP project timeline, trying to maintain gender balance for participatory decision making, and needing to consider taboos around discussing sensitive issues relating to HIV/AIDS, family planning etc, are all community factors that impact program implementation. Communities that are cut off CARE Sierra Leone Child Survival XIX Project Midterm Evaluation Report June 4, 2006 29 during the rainy season, wild life attacks on humans (leopard and buffalo) that have made people afraid to go out, lack of material support for activities as compared to other projects or organizations, are all socio-ecological environmental conditions that limit the project’s ability mobilize communities. But these factors have been addressed by readjusting plans in communities that get cut off due to the rains, integrating activities with other projects and organizations and conducting community sensitization activities on the importance of both men and women’s participation in development. b. Communication for Behavior Change CSP communities are expanding their activities beyond original program interventions and expanding into strengthening civil society to include collecting funds for emergency transportation, supporting schools, fixing road and bridges and teaching other communities what they have learned. Qualitative and quantitative assessments have determined that most targeted child survival behaviors have significantly improved, indicating that the CSP’s BCC strategy is effective. Activities have been well-accepted and new ways of transmitting information, especially via radio, which had been very low, was facilitated when CARE provided small, inexpensive radios to the CHCs. The BCC strategy addresses 1) the “culture of silence” barrier by actively encouraging women’s participation (along with asset ownership by women); 2) men as the major decision makers, 3) cultural beliefs, taboos and practices; 4) school attendance, especially by girls; 5) poor clinic attendance by improving the quality of care at health facilities 6) decreasing dependency by emphasizing sustainability; and 7) avoiding duplication of effort by partnering with other organizations. . The community mobilization strategy (see previous section) and the BCC strategy reinforce each other. Gender balanced community structures, emphasis on participatory approaches in community meetings, community-led drama groups that use community languages and group feedback discussions on health issues to identify taboos and traditions/cultural beliefs in collaboration with DHMT all contribute to a two-way dialogue and communication. Messages through the media (e.g. community radio) have been facilitated by the development of the community radio station and collaboration with the CSP in developing messages. In addition, CSP messages are similar to, and reinforced by, other projects in the District, such as Title II DRP and CARE’s previous water and sanitation program. The project provides logistical support to the DHMT to facilitate access to health services/facilities and builds MOHS capacity to conduct health talks that promote community awareness on the importance of improving physical access to health facilities (e.g. construction of roads and bridges). This has resulted in some communities repairing roads and bridges in their area to facilitate access for community health outreach sessions. Using CARE’s Rights Based Approach (RBA) has increased transparency and accountability and sets the example of values that the CSP promotes. The CSP has developed MOUs with communities and MOHS. CHCs are non-discriminatory and membership is free. To demonstrate the need for gender balance, women determine the site of wells in communities where water and sanitation activities are also located. The CSP collaborates with CARE’s Country Office projects CARE Sierra Leone Child Survival XIX Project Midterm Evaluation Report June 4, 2006 30 and other partners to improve the livelihood of beneficiaries. Coordination meetings with other partners, address duplication/over lapping of activities at all levels (district, chiefdom, community.) For the most part, project messages are technically correct, especially when MOHS policies and messages conform to international recommendations. [Exceptions have already been noted in the Technical Interventions section of this report.] The project is adhering to MOHS national policies with reference to other international health updates, which are discussed at coordination meetings for policy review, e.g. Malaria cases management, ITN targets, etc. Challenges to technical quality are in areas where the current MOHS policies and/or practices have not yet been updated to coincide with international standards. Advocacy for policy change is not a major focus of the CSP design, but CARE has valuable experience to contribute to the MOHS as policies are updated. The program has gone beyond increasing knowledge and has succeeded in changing social norms, especially related to women speaking in public and participating in public decisions and including removing food taboos for pregnant women and small children. Many of the positive behavioral changes have already been mentioned. Additional positive changes include: exclusive breastfeeding and hand washing have noticeably reduced diarrhea incidence. Some villages have started savings and loans to pay for emergency transportation and organized hammock systems to transport emergency cases. Children and pregnant women are now visiting/attending clinics more regularly rather than going to traditional healers/herbalists. Many complicated deliveries are now referred early, and not delayed due to the common belief that delayed labor is a result of infidelity during pregnancy which can be solved through confession. [However, it was noted during the MTE fieldwork that prolonged labor was the one danger sign that was not frequently mentioned by communities.] Communities are now taking their children to be immunized in much larger numbers. They state that they do it because they understand the importance as a result of CSP activities. Communities are now accustomed to listening to health media broadcasts using portable radios that were provided by CSP. The CHC members provide batteries and minor maintenance. VDCs monitor the use of the radios for intended purposes. CHC communities are spontaneously scaling-up CSP BCC by communicating CSP messages to non-CHC communities. Tools used in the BCC approach are appropriate and include: observation checklists, focus group discussion, key informant interviews, LQAS, PRCA, KPC, transect walk-observation, mapping, supervision checklist, community birth and deaths registers, program exchange visit with partners. Information is used to identify gaps and make adjustments to the program. Data analyzed by the CSP is shared with communities and stakeholders. Innovative BCC approaches used include organizing community drama competitions about prevalent health problems; establishing by-laws for proper use of facilities like wells, latrines etc; (Additional information on measuring the effectiveness of the BCC approach is included in the Information section of Program Management.) CARE Sierra Leone Child Survival XIX Project Midterm Evaluation Report June 4, 2006 31 VDCs now check on pregnant women and lactating mothers and encourage them to attend ANC and immunizations, monitor the use of ITNs at household level. Communities have put in place emergency preparedness schemes for obstetric evacuation. Health management committees have been established at community level and are responsible to resolve conflicts between PHU and community. They contribute to helping with the services and upkeep of PHUs. These are valuable skills that can transfer to other important civil society and development activities in project communities. c. Capacity Building Approach Strengthening the PVO Organization The 2004 DIP capacity building plans focused on local partner and community capacity building and did not specify plans for international headquarters capacity building. Nevertheless, continuous staff capacity building has been established in CARE’s organizational culture for many years. CARE headquarters staff participate in a wide-variety of capacity building activities, including participation in CORE group working groups, technical workshops, and internal CARE training activities. CSP managers have participated in several national, regional and international workshops and other staff have participated in programs inside Sierra Leone. The most significant CSP staff capacity building activities have included TA on the COPE methodology, CSSA (jointly with IRC) and cross-visits to IRC’s Child Survival project in Kono. Both IRC and CARE have provided participants in each other’s midterm evaluations and both organizations have adapted aspects of the other’s programs. Besides the technical exchanges, staff from both organizations state that they appreciate the moral support they get from their colleagues in the other projects. Training in both IMCI and Community IMCI for CARE SL staff is the major remaining staff capacity need. DHMT staff state that 8 DMOs from Sierra Leone MOHS were trained as master trainers in IMCI in Ghana in 2004. UNICEF facilitated facility-IMCI training in Koinadugu District in late 2004, but there was no follow up and there has been staff turnover since then. In verbal discussions with national MOHS staff involved in this area, it is said that IMCI will “soon” be adopted into MOHS policy and that training should roll-out from this; however, no dates have been set as yet. The CARE SL CSP DIP framework is based on C-IMCI and several individual components are in place. But there is insufficient demonstration that the C-IMCI approach is guiding framework of program implementation. In other countries, CARE HQ in Atlanta has been a leader in promoting C-IMCI as an approach to Child Survival Programs. CARE should be able to orient SL staff as to the differences between C-IMCI and vertical programs (even when they include community mobilization and involvement). Within this training, CARE HQ should help clarify CARE SL’s appropriate role in providing leadership in the DHMT’s health facility and health worker training in support of IMCI and C-IMCI. Strengthening Local Partner Organizations Koinadugu District Council On the 20th of February 2004, the president of Sierra Leone launched the re-birth of decentralization and devolution of central government functions. Local councils have since been established, including a local council in Koinadugu District. The central government is CARE Sierra Leone Child Survival XIX Project Midterm Evaluation Report June 4, 2006 32 transferring some of its functions and responsibilities to sub-national levels of government (districts) while still retaining centralized control of monitoring the functions of these councils. Koinadugu district council is at the sub-national level of government, which has jurisdiction over a limited range of state functions within a defined geographical area. Under the Local government Act, 2004 the following functions have been devolved from central MOHS for management at the District level: a. Registration of Births and Deaths b. Public health information and education c. Primary Health care d. Secondary Health care e. Maintenance of non-technical equipment f. Facilities management g. Procurement of equipment and medicines. CHC and VDC training has been the major emphasis of the CSP and has been quite successful. Due to decentralization of health services in Sierra Leone since the beginning of the program, partnership with the local District Council has become a much more important component of the CSP than was originally anticipated. The Council now plays a much more significant role in health programs, including resource allocation. The devolution process at the time of adoption was largely theoretical and did not address the day-to-day operations of both the council and the MOHS in Koinadugu. There were gaps in the definition of core functions. For example, it was not clear whether public health information and education involved development or also included dissemination. The implication of these new developments on project capacity building is expected to become clearer over time. The District Council Health Committee Chairman substantially contributed to the MTE team fieldwork and provided in-depth information in an interview conducted by CARE Atlanta’s HQ CSP backstop at that time. The DC is very supportive of the CARE CSP and the relationship between the two partners is strong. (See interview notes in Annex X) Christian Extension Services (CES) CES’ role in CARE’s CSP rests primarily in community mobilization, especially with CHCs. CES had already has received significant organizational capacity building from CRWRC and World Relief in previous programs when they were asked to join the CARE CSP. CARE has assisted CES with PRA training. In the second year of the project, CARE conducted an Institutional Strengths Assessment (ISA) of CES. The ISA provided a self-assessment of CES’s health program, technical experience and the quality of backstop support to field projects. The ISA helped CARE to determine which issues need to be reinforced between headquarters staff and field sites as well as what how data collection can be used to advocate for additional resources for health-related operations within the larger organization. CARE plans to train CES on the use of the new participatory health communication toolkit with new Community Health Clubs which will be released soon. The extent of CES’ involvement in the remainder of the CSP should be clarified in the revised work plan that results from CARE’s MTE Action Plan. DHMT Koinadugu District CARE Sierra Leone Child Survival XIX Project Midterm Evaluation Report June 4, 2006 33 DHMT capacity building, to date, has been uneven. Quality improvement efforts at the PHU level have been significant, with CARE playing the major role in bringing partners together and strengthening elements of quality of care, as well as relationships between communities and first level health facilities. Changes that have been noted since CS capacity-building began include: increased EPI coverage, improved PHU attendance, improved environmental sanitation (including areas around PHUs), improved data collection and access to information, “amazing” female participation in development activities, greater awareness about common illness and normal growth patterns in children. The project has assisted with the DHMT HMIS, helped improve quality through the COPE assessment and helped the DHMT to implement its own programs at community level. It is unclear, however, whether CARE feels it should play a leadership or mentoring role with the District Level DHMT, or wait for the DHMT to develop plans and assist with them. CARE is not always included in the initial planning of intervention-related DHMT activities, which cover both CSP and non-CSP communities. For example, in the past DHMT has sometimes worked only with UNICEF. This appears to have improved with the latest DMO who has invited CARE and other NGOs to meetings to plan district activities, for example, the recent Rapid Response Initiative (EPI mop-up). CARE Kabala staff responses to questions about CARE’s role with District-level intervention￾related programs seemed to be one of supporting DHMT plans, rather than being proactive in assisting DHMT to develop plans. This can seem to be a fine line. The explanation has been the need to follow “MOHS policy”, or “DHMT responsibility” even when policies or approaches are significantly out of date. This less proactive role could be partially attributed to frequent DMO turnover and limited DHMT human resource capacity in program planning. Relationships with the DHMT were very poor earlier in the program, apparently not due to anything that CARE had done. Each change in personnel required relationship-building between CARE and the DHMT to start over. At the time of the MTE, however, a new and very experienced DMO had come to Koinadugu. In spite of initial skepticism by CARE staff that have seen a new DMO every year, he has stated his intention to remain in the District longer than one year. He has pledged to improve quality from the DHMT perspective and is making unannounced visits to PHUs and replacing staff immediately when indicated. Simultaneously, he is interviewing students in national public health schools to encourage them to come to the District to work after graduation. Although currently overloaded (he is also the only doctor at the District Hospital) he has demonstrated a strong cooperative attitude towards CARE’s CSP staff (and this MTE) and supports the program’s work with communities and PHUs. CARE has, appropriately, placed the responsibility for calling CSP partner’s meetings to the DMO. Yet realistically, these meetings are taking lower priority than the huge staffing problems and direct health care service requirements of the District Hospital. But failure to clarify roles and responsibilities for all of the players (DHMT, NGOs, International Organizations, and District Council) could result in duplication or contradiction of effort, missed opportunities, and wasted resources. Thus, CARE’s (or another organization’s) role in facilitating these regular meetings could make the difference in whether or not coordination ever occurs. (It should be noted that role confusion between a PVO and the local MOH counterpart in the initial implementation of a CSP program, especially in an area where a CSP has never been CARE Sierra Leone Child Survival XIX Project Midterm Evaluation Report June 4, 2006 34 implemented, is not an uncommon finding at the time of a MTE). It is reasonable to expect that the rest of the project will focus on to scaling-up, assuring quality and sustaining intervention efforts. That is what appears to be happening in the CARE CSP, but this assessment is based on very recent changes. As results from increasing coverage of proven community-based CS interventions becomes more and more apparent, relationships within the partnership could be expected to continue to improve. That leaves the question of what CS leadership role that CARE will chose to play at the District Level, including the increasingly expanded importance of working with the District Council. DHMT organization capacity has changed, but attribution strictly to the CARE CSP at the central level is difficult. Policy changes (especially in malaria and IMCI and to a lesser extent EPI) have resulted in changed behaviors at the local DHMT. UNICEF has organized trainings (such as in malaria case management) where CARE were participants, but not organizers. Qualitative assessments conducted during the MTE indicate that CARE’s major contributions to DHMT capacity building are in the areas of improving quality of care at the first line health facilities and integrating community-level data into the District HMIS. This was facilitated by the COPE assessment which brought partners together and resulted in specific actions to be taken by each partner. CARE’s role could expand if CARE decided to devote more time on the technical aspects of the CSP interventions and supports the shift from vertical programming to C-IMCI as was planned in the DIP. Health Facilities Strengthening The CSP computerized the DHMT HIS (and provided an intern to work with the DHMT on-site), established the district medical stores inventory system and reviewed PHU data and printing services. Tools developed included PHU assessment checklists, focus group methods with stakeholders, rapid and in-depth assessments and the COPE quality assessment tool. Linkages between health facilities and communities were strengthened by training PHU staff as TOTs for communities. Community members are also now signatories for receipt of drugs at the PHUs (ACT and SP) and ITN distribution. From the final approved DIP (June 2004), p. 24: “The project had initially planned to conduct a health facility assessment to evaluate quality of care in the facilities. UNICEF is now planning to conduct the WHO facility assessment in the communing months in Koinadugu District and, therefore, it seems redundant for the project to do a separate one. CARE staff, along with the DHMT, will be partnering with UNICEF on this endeavor and will be involved in finalizing the tool and carrying out the assessment. . .” According to the First Annual Report, rapid in-depth Peripheral Health Unit (PHU) assessments were carried out in the operational chiefdoms in October 2003 and collected information on: 1) PHU accessibility to the various target communities: (2) population size and characteristics of each PHU catchment area, and; (3) the functional capabilities (i.e. staffing qualifications; equipment, availability of drugs etc.) of each PHU in the operational area. In total, 14 PHUs identified in the project area were adequately operating. The assessments carried out led to the selection of 97 villages in the five project chiefdoms. This was followed by the COPE Quality of Care in Health Facilities (COPE) assessment in May 2005 (timed closely with the LQAS). This qualitative assessment looked at several aspects: client perceptions, supplies and drugs, gaps in CARE Sierra Leone Child Survival XIX Project Midterm Evaluation Report June 4, 2006 35 perceptions between community and health providers and staffing. Health worker clinical performance (also related to next section of this report), patient flow analysis, and the COPE￾IMCI Record Review were not conducted.11 Discussions with CARE staff, both from headquarters and the field at the time of the MTE indicated that there was no alternate plan for assessing the health facility and health worker performance that was supposed to be covered in the UNICEF assessment. The major missing element not addressed in other assessments conducted by the CSP was health worker clinical treatment performance (see Health Worker Strengthening below.) The COPE follow-up Action Plan covered many areas with responsibility falling to many players, including DHMT, DC, UNICEF, CARE, CHCs, and VDCs. Many of these follow-up actions are not under direct CARE control. The Action Plan was reviewed at the time of the MTE and next steps were reviewed and clarified with partners (see Attachment XX) The ability of the PHUs, and additional referral points (such as the District Hospital) to provide quality clinical care for women and children experiencing danger signs of serious illness or complications of pregnancy (or even routine deliveries) are essential components of a program that is mobilizing families and communities to seek care, especially for emergencies, at health facilities. As mentioned earlier, at the time of the DIP, UNICEF was expected to conduct Health Facility assessments. It was assumed that these findings could be used as the baseline to be used for measuring project progress at the end of the CSP. But the anticipated content of these assessments were not available in project documents and the assessments never took place. This will make determining CARE’s contribution toward actual progress in providing quality clinical care from the health system and the health worker very difficult at this time as well as at the end of the program. This is in contrast to the considerable quality of care improvement efforts that the CSP has done in the overall health facility functions (e.g. infection control, interpersonal relationships, facility cleanliness and upkeep, record keeping, etc.). Strengthening Health Workers Performance The CSP has trained CBGPs and PHU staff in growth promotion, vaccination for NIDS, and UNICEF provided malaria case management training. The project has also conducted joint outreach activities for routine EPI, NIDs and ANC with PHU staff. DHMT staff were involved in project surveys/assessments, supervision and planning. CARE supported PHU staff in analyzing data collected at grass root level and displaying vital information at the PHUs. This has led to strengthening relationships between health facilities and communities (an important C￾IMCI component). Assessment results have been used to enable DHMT to identify lapses and take appropriate action, improve outreach activities and improve sanitation as a result of BCC activities. 11 At the time of the study, IMCI had not yet been adopted as a national protocol. CARE Sierra Leone Child Survival XIX Project Midterm Evaluation Report June 4, 2006 36 Challenges in the project’s capacity building approach include the shift in national policy towards IMCI which was anticipated in the DIP but has been rolled out more slowly than scheduled. The DIP clearly specifies that the APM position had been modified to include IMCI training12. At the time of the MTE, CARE had implemented some aspects of C-IMCI at the community level, but was still focused on supporting vertical standard case management at the health facilities. Even though the final approved 2004 DIP clearly identifies CARE as responsible for IMCI case management training13, as of the MTE CARE had not yet defined its specific role in facility-based IMCI roll-out in relation to the other partners, especially DHMT and UNICEF. On the other hand, the project’s DIP M&E matrix measures “Percent of PHU staff following standard case management protocols.”14 It is unclear how this measurement will be adjusted as the definition of “appropriate case management” changes with the introduction of IMCI. In addition, the DIP committed CARE to working with the World Bank in developing cost-recovery mechanisms for IMCI-related drugs,15 even though DHMT staff are not yet trained in IMCI as of March 2006 and CARE staff are not yet trained in IMCI. It appears that the CSP has relegated responsibility for IMCI training to the DHMT, but has not adjusted the CSP program description and work plan to reflect these changes. Gaps in information on case￾management practices both at baseline and since CSP implementation began and revision of standard case management, particularly with regards to malaria, will make determination of CARE’s impact in this area very difficult. A major shift in malaria treatment protocols has already occurred, and CARE played a major role in facilitating access to the new drugs (ACT) and is monitoring their distribution. But it is not possible to comment on appropriate clinical assessment skills prior to providing treatment, or now that malaria case-management training has been done by UNICEF. It is also not possible to measure performance in conducting routine and complicated deliveries and other aspects of maternal and newborn care. This should be addressed 12 CARE CS XIX DIP, June 2004, p. 5 13 Ibid, p. 26. 14 Ibid, p. 53 15 Ibid. p. 26. CSP CES DHMT CBOs PHUs CHCs CBGPs VDCs CARE Sierra Leone Child Survival XIX Project Midterm Evaluation Report June 4, 2006 37 in the upcoming MNC Technical Assistance that CARE intends to provide after the MTE. [See comments in the Technical Interventions for additional comments on health worker performance measurement challenges.] Challenges to the CSP's health facility strengthening efforts include: weak communication and transportation networks, (this is not primarily CARE’s responsibility) and poor DHMT staff retention, particularly at periphery. The DHMT DMO and Nurse-in-Charge turnover have hindered partnership definition including clear roles and responsibilities, in spite of multiple attempts by CARE to establish them. Access to facilities during the raining season is especially difficult and, in one remote area of the District, wild life attacks have decreased attendance in health programs. The COPE assessed interpersonal communications and some organizational skills. For support of basic prevention and early treatment actions, the training programs are certainly valuable and help the health workers to set priorities and promote effective CS caretaker actions. In many PHUs, however, the health care worker must function as a trained diagnostician and be able to treat basic conditions as well as organize appropriate referrals to the District Hospital in a timely manner. Without a baseline assessment of health worker clinical function at the beginning of the project, and a monitoring plan, neither the DHMT nor CARE can track progress in this area outside of a small sample of client satisfaction exit interviews and health worker self￾assessments. The project is not tracking individual health worker clinical performance against performance standards. It is unclear if the DHMT intends to undertake any performance measurements as IMCI is rolled out in the District. Training CARE has conducted refreshers in breastfeeding, nutrition, and other subjects for health workers, most of whom have basic training at the level of an auxiliary nurse or lower, but are required to provide the services of a much more highly trained health worker. CARE CSP staff provide a report of a four-day training of trainers (TOT) workshop in Bo on emergency preparedness, malaria prevention and case management that was organized by UNICEF for six DHMTs and five NGOs (CARE was one of the NGOs). Workshop objectives included emergency preparedness to enable districts to source and use supplies for emergencies, training of trainers workshop for new UNICEF districts on emergency preparedness, malaria prevention and case management and provided review for the old UNICEF-supported districts (Koinadugu is in the second group). Much of the content coincided with CARE’s Child Survival interventions. The malaria case-management with Artesunate￾Amodioquine Combination Therapy (ACT) update was given by Sister Pity Kanu, a representative of the Koinadugu DHMT. CARE staff were participants, not organizers or facilitators of this training. When asked about CARE’s role in improving health worker clinical treatment performance, staff replied that they see their role as supporting the DHMT training. Although included in program objective 3 and in the DIP project work plan, it is unclear from the descriptions of program activities in program documents and interviews with the CARE Kabala staff what extra value CARE brings to improving case management, aside from assisting with drug logistics (e.g. distributing and documenting the use of the UNICEF-donated ACTs). These roles raise sustainability questions and should be turned over to the DHMT as soon as possible. CARE Sierra Leone Child Survival XIX Project Midterm Evaluation Report June 4, 2006 38 CARE’s role in upcoming and routine health worker clinical skills training should be included in the Action Plan as part of the follow-up to the MTE. CARE has been very proactive in contributing to the development of the PHU supervision checklist – of special interest to the current DMO. It is hoped that direct observation of case treatment by PHU staff will eventually be included in this supervision system. The project will need to identify how the supervision (along with the checklists) will continue after the CSP ends. Training methods are highly participatory and success is reflected in the observed and measured behavior changes on the part of health worker staff and community members. Due to low literacy levels, most training is verbal and visual at the community level, with minimum written materials. CARE has adapted CHC training materials based on successful materials used in CARE Zimbabwe. These are accompanied with high-quality photographs that can be used to illustrate important teaching points in the curriculum. These materials will be put into use after a few technical corrections are made. d. Sustainability Strategy The CARE CSP, along with the IRC CSP in Kono district, were selected for Technical Assistance (TA) from the Child Survival Technical Support Project (CSTS+) to carry out the Child Survival Sustainability Assessment (CSSA). The assessment commenced with a visioning exercise that was conducted jointly for IRC and CARE in March 2004. Staff from the two projects, along with their respective MOHS district counterparts, came together for a three-day Working Group meeting held March 29-31in Freetown. The CARE project has since completed the mapping of its indicators of sustainability. Sustainability Dashboard (from the CSSA) A map of CARE’s CSP situation on six-component sustainability ‘Dashboard’ is depicted below. The Dashboard depicts the status of indicators of sustainability. Theoretically, a context where health outcomes are sustainable is one where all the indicators equal 100%. Clearly organizational viability and capacity are low and service quality is critically lagging in Koinadugu. CARE Sierra Leone Child Survival XIX Project Midterm Evaluation Report June 4, 2006 39 The second dashboard below depicts the project’s targets for the various indicators for sustainability. This is a reflection of how far the project implementing its current plan of activities will move towards its vision. These results are a combination of the project and the local system’s planned effort measured in the six components. Graph 4: CARE Child Survival Project in Sierra Leone Sustainability Dashboard (Target for 2008) 0 20 40 60 80 Health Status Health Services Organizational capacity Organizational viability Community capacity Political and policy environment Graph 3 : CARE Child Survival Project in Sierra Leone Sustainability Dashboard 2005 0 20 40 60 80 Health Status Health Services Organizational capacity Organizational viability Community capacity Political and policy environment CARE Sierra Leone Child Survival XIX Project Midterm Evaluation Report June 4, 2006 40 C. Program Management 1. Planning CARE’s primary planning partners include the DHMT, District Council, UNICEF, other CARE programs, CHCs and VDCs. In addition to the DHMT, UNICEF is implementing several child survival related programs in Koinadugu District, sometimes involving CARE and other times not. It was not clear at the time of the MTE to what extent UNICEF was working with CARE as a full partner in the planning of their programs, or if CARE was one of many organizations included in the implementation of plans that UNICEF developed independently with MOHS. It is clear that CARE’s community-based programs have led to significant success in UNICEF￾supported activities, including ITN and ACT distribution. CARE was significantly involved in an EPI mop-up campaign which was completed shortly before the MTE initiated, providing a great deal of the organization, communication and logistic support. With the exceptions noted in other sections of this report, plans are basically on schedule according to the DIP. To the extent that they are behind, the factors (such as IMCI roll out or DHMT staff changes) are largely outside of CARE’s control. Hopefully, this situation will improve if CARE helps to facilitate the DHMT and DMO to organize District-wide partner planning meetings as they have attempted to do in the past. Project objectives are well understood by field staff and headquarters staff. At the country level, the specific differences between the Child Survival implementation and other health and nutrition related programs (e.g. Title II, water/sanitation) are not as well understood. This should be helped when the CARE Freetown Health Coordinator position, vacant since July 2005, is filled. Each DMO has been given multiple copies of the project DIP. Turnover in the DMO position (3 times in 2½ years) has proven frustrating to the CSP staff. The new DMO has been given a copy and states his intention to become familiar with the details in it. Partner representatives on the MTE team stated that their participation in the evaluation has helped them to become more familiar with the details of the program including the M&E plan. LQAS, quarterly and annual reports are shared with partners and frequently discussed. LQAS data from 2005 helped focus attention to low performing chiefdoms and helped identify interventions that need more attention. CARE placed an intern at the DHMT to assist in updating the HMIS. The project assists the DHMT by compiling PHU and community-based data and giving reports. This includes efforts to monitor inventory of the UNICEF-donated ACTS. Efforts to integrate project data with DHMT are ongoing. 2. Staff Training Training needs are developed jointly with CARE HQ, CARE SL Freetown and Kabala management and the staff member. Opportunities largely depend on the availability of training courses, within Sierra Leone, internationally or regionally. On-line opportunities exist via the CARE Academy. International Program management staff are brought together by CARE/Atlanta once a year and provided training in topics that are applicable to many programs. CARE Sierra Leone Child Survival XIX Project Midterm Evaluation Report June 4, 2006 41 A wide variety of training methods are used, from presentations, group work and field visits. Pre and post tests are done during trainings, and observation/feedback/discussions are done during HQ visits and during the annual workshops previous years’ topics/methodologies are revisited during informal discussions, but there is no strategy for formalized training effectiveness assessments. Clinical skills sufficient to oversee DHMT capacity building in clinical diagnosis and treatment of malaria, severe malnutrition and obstetrics are not found in the full-time staff implementing the child survival program. It is well known that Sierra Leone is severely challenged in providing experienced public health professionals, especially in the clinical specialties. Competencies in these areas are not found in short term training and will need to be addressed with technical assistance with experts in the respective fields. CARE and CSP partner roles and responsibilities for the leadership of these areas will need to be clarified in the MTE Action Plan. Staff have been sent to multiple trainings, both child survival related (such as qualitative assessment methods, BCC skills) and general development topics, including human rights. The Monitoring and Evaluation Officer attended a training workshop on data collection, entry and analysis using EPI info data entry techniques in Uganda, organized by the CORE group. The M&E officer in turn facilitated training for other Country Office Monitoring and Evaluation Officers on the same. The Assistant Project Manager attended a workshop facilitated by the CORE group on qualitative research methods, in Nairobi in December 2005. The Project and Assistant Project Managers attended the 10th CARE Annual Child Survival Workshop, held in Malawi and organized by the CARE Health team in Atlanta. The workshop provides a unique opportunity for Child Survival program staff in other countries to share implementation experiences and technical information. Staff also participated in workshops that include the following: · Community mobilization for development · Positive Deviance (PD) using the PD Hearth model – organized by CRS · Development of key messages for specific targets on Vitamin A supplementation and intake of Iodized salt – organized by HKI · Ethical decision making in programming · Strategic planning for middle and senior level staff · Effective and supportive supervision CARE Sierra Leone organized workshops on CARE programming principles including the Rights Based Approaches (RBA), Household Livelihood Security (HLS) and Gender Equity and Development. The CARE Academy operated by CARE USA is web-based and provides an opportunity for staff to pursue academic courses related to humanitarian and development work. CSP staff have enrolled in the Academy to pursue courses ranging from project implementation standards to human resource management. Each staff member in Kabala is also expected to research a technical topic and present it at a meeting of all CS staff. The availability of electronic communication to staff at the base level has also provided a boost to staff capacity building efforts. Staff now has the opportunity to browse CARE Sierra Leone Child Survival XIX Project Midterm Evaluation Report June 4, 2006 42 technical sites through the Internet and download relevant materials from other health projects. Internet sources of health information, however, are relied upon more heavily than standardized Child Survival information sources, such as the Technical Reference Materials which were produced specifically for use in PVO Child Survival programs. CARE International devotes more time and resources for staff training than many other PVOs. In many cases, trainings and international meetings are funded from many sources; therefore the amount of money in the CSP budget reflects only a fraction of the training resources provided by CARE. At the time of the MTE, planned project management changes were in process, coinciding with the time in the project that Child Survival technical aspects will be significantly intensified. It is highly likely that project management staff will need additional capacity building in the individual technical components: malaria, nutrition, IMCI, and maternal care. Some of these needs will be addressed by CARE International’s Child Survival meetings and MNC Technical Assistance that was planned during the MTE. CARE HQ is currently assessing individual CSP staff capacity needs and will make a determination if additional resources will be required. 3. Supervision of Program Staff Due to CARE’s organizational structure, CSP staff are supervised by the CARE Freetown office, not CARE International. CARE Kabala staff are very positive in their assessment of supervisory support from Freetown, especially when there was a Freetown Health Sector staff person. CARE HQ staff serve largely in advisory and technical assistance roles, yet are the ones ultimately responsible to USAID for project outcomes and accountability. So far this has not been a problem. The project has been challenged with personnel management changes in both Freetown and Atlanta. With each new change, the institutional memory of why certain decisions were made when the project was designed becomes less clear. These are not insurmountable problems, but an overall reorientation to the specific commitments that CARE has made in the DIP would make it easier to made adjustments at this point. This will help to assure that the CSP does not come up short when compared to commitments at the time of the Final Evaluation. CARE should review all activities in the DIP, compare them to current plans for the remainder of the program and reconcile the work plan in the MTE Plan of Action. CARE should strongly consider responding to the opportunity offered in the MTE guidelines to provide a revised program description in the Action Plan since the DIP is considered the Cooperative Agreement Program Description that replaces the Program Description from the original project proposal. Since the beginning, the project has received technical backstopping from the CARE USA Health team based in Atlanta and the CARE Sierra Leone Country Office senior management team. The CARE SL Health Sector Coordinator, until her departure for another assignment in July 2005, was based in Freetown and paid frequent field visits to the project. CARE is in the process of hiring a replacement for her, indicating the priority that CARE Freetown places on health in its country programs. The number of visits from CARE HQ and CARE Freetown has been sufficient, and is comparable with other well-managed CSPs. CARE can overcome the lost institutional memory with time devoted to making sure everyone from Atlanta to Freetown to Kabala “are on the same CARE Sierra Leone Child Survival XIX Project Midterm Evaluation Report June 4, 2006 43 page” and maintaining the current commitment to regular monitoring visits for the remainder of the program. 4. Human Resources and Staff Management At the time the program was funded, it was well known that finding an experienced national child survival program manager would be difficult. At the same time, building national capacity is a priority of Child Survival programs. After the first program manager left abruptly, CARE wisely brought in an experienced expatriate Program Manager, Boiketho Matshalaga Murima, and planned for her to stay for only the first half of the project while training a Sierra Leonean replacement. Under her leadership, the CSP weathered many challenges that might have stymied other projects in similar conditions. The Assistant Program Manager, Vandy Kamara, is a national and has been groomed to take over after the MTE. He is extremely enthusiastic and welcomes any opportunity to learn. All positions have been filled. One additional Field Mobilizer, a female, was added because it wasn’t possible to cover all communities with the existing staff. CARE has written job descriptions at headquarters and in the Sierra Leone office. DHMT have job descriptions, but in many cases employees have had to take on additional responsibilities beyond their level of training. Staffing is currently only 60% of the national standard. The DMO and District Council are unhappy about these circumstances and are working extremely hard to bring DHMT to national staffing standards. The District Council has provided funding to enable two youth to enroll in training as MCH aides with the expectation that they will choose to return and work in Koinadugu District. The DMO has been to the national public health schools to try to hire more staff for the DHMT. Morale and teamwork among staff and between the CARE Freetown and CARE Kabala staff is very good and appears supportive. The relationship between CARE Kabala and the Koinadugu DHMT was rocky at first with the earlier DMOs, but is now strong. The District Council recognizes CARE’s significant contribution and suggests that CARE remain working in health programs in the District. The major turnover since the beginning of the program has been at the CARE Atlanta Headquarters, CARE Sierra Leone Freetown offices, and at the DHMT. The original CARE HQ backstop left within the first year to work with USAID’s CSHGP office in Washington, but other CARE HQ staff took over and visited the project. CARE Senior Child Survival program manager, Dr. Sanjay Sinho, was promoted to head CARE’s health office when Dr. Jim Sarn left and took a position with USAID Afghanistan. CARE’s Senior Technical Advisor for Child Health, Joan Jennings, joined CARE well after the DIP was approved. She is assisted by other program staff in Health Unit. The CARE Sierra Leone Health Sector Coordinator, until her departure in mid-2005, was based in Freetown and paid frequent field visits to the project. The CARE Sierra Leone CSP Program Manager arrived after the KPC, and therefore was not working at the time of the baseline studies. Staff turnover at project level is very low, with the exception of the planned departure of the Program Manager who is to be replaced by the Assistant Program Manager. CARE Sierra Leone Child Survival XIX Project Midterm Evaluation Report June 4, 2006 44 CARE has a well organized and transparent system for posting and selecting employees and staff have successfully transitioned to new programs when others have ended. CARE Sierra Leone is actively leveraging CSP funding and seeking support for additional programs, such as the recent DFID MOSI malaria program. CARE has been working in Sierra Leone since 1961 and has a long-term commitment to the country. But USAID Sierra Leone has dropped health from its draft Strategic Plan. This will make it difficult for CARE to seek complementary funding from the local mission for continued support for the initiatives developed in the CSP after the program ends. 5. Financial Management As primarily a capacity building and behavior change program, the CSP does not provide large amounts of physical inputs. Funding health sector activities is largely the DHMT responsibility, but the UNICEF-supplied drugs and commodities (ITNs) and World Bank supplied supports (ambulances) leave many questions about continued funding unanswered. Funding the expatriate program manager position was expensive relative to national staff, but was necessary due to the lack of qualified health managers in SL after the war. CARE has a good track record for USAID financial reporting and the project spend-down of both USAID funds and match is on target. CARE has recently been awarded $2 million from DFID to support malaria programming in several districts, including Koinadugu (the “MOSI” project). The portions that cover CSP communities could be considered additional match because the CSP and MOSI approaches are very closely aligned and designed to be synergistic. Hopefully, CARE SL and CARE HQ will continue to seek out complementary and follow-on funding to enhance and sustain the CSP impact. One opportunity that CARE SL might consider would be to develop a proposal for community level malaria surveillance and proactively meet with the country Global Fund for future proposals. This would face considerable challenges, given the current problems with the SL Global Fund Country Coordinating Mechanism but could result in considerable support for continued and scaled-up implementation of successful CSP activities. The lack of large amounts of inputs, including drugs, strong integration with the DHMT and District Council also bode well for sustainable financing, but CARE will need to advocate strongly for earmarked funding for transportation and logistical support for outreach and emergency transportation in the District Council and MOHS budgets. Financial sustainability for the program and partners are discussed elsewhere in this report. Spend down of both USAID and match funds in the CSP is on schedule. 6. Logistics CARE CSP has been very involved, along with UNICEF and the DHMT, in drug and commodity distribution, and involved more than is normally the case for a CARE project in the tracking of drug use and supplies. CARE will need to shift responsibility for this to ensure sustainability. The project helped develop accountability checklists and signed MOUs with local VDCs to oversee drug supply security. But drug supplies after the free supplies are exhausted have not been discussed and are a threat to the overall sustainability of program efforts. These CARE Sierra Leone Child Survival XIX Project Midterm Evaluation Report June 4, 2006 45 issues are most appropriately addressed at the national level as they require support from the Global Fund or other large donors. Routine equipment and drug supplies remain problems that CARE alone cannot address. The DIP mentions that CARE will provide medical equipment to PHUs in the third year of the project. At present, the only possible equipment under discussion is five VHF communication sets for PHUs to support emergency referral systems; however, UNICEF, who will supply the remaining PHUs in the district, has had difficulty in identifying a local or regional source for maintenance and repair. Equipment supplied by CARE and the process for turning over responsibility for drug and ITN logistics should be clarified in the Action Plan. The CARE commitment in the DIP to develop a cost-recovery mechanism for the IMCI essential drug list 16 has been superseded by UNICEF taking responsibility for developing these mechanisms with DHMT. CARE plans to add value to this plan by focusing on transparency and accountability at the PHU and VDC levels, along with district council and DHMT levels. There are plans underway for a massive measles-ITN campaign in October 2006, but details were not available at the time of the MTE. CARE should be proactive in becoming involved in this planning process in order to determine the impact it will have on CSP and MOSI programming. 7. Information Management CARE placed an intern in the DHMT to strengthen HMIS. Two disease-monitoring volunteers (a male and female per village) have been selected from each CHC during the integrated program activities in coordination with CARE SL’s water and sanitation project in the district. These volunteers participated in the training on disease surveillance led by CARE SL and MOHS staff. Topics included disease recognition, tracking, recording and reporting of common communicable diseases, such as diarrhea, malaria, ARI, etc. PHU staff now monitors the surveillance volunteers. CARE SL staff continues to work with PHU staff and community surveillance volunteers to build their capacity and facilitate the analysis and use of surveillance data to find viable solutions to pertinent health problems. Project monitoring data compared to the KPC baseline findings are found in Table I. Follow-up LQAS is scheduled for June 2006. Table 1. Comparison: Baseline, LQAS, DIP targets, as of March 2006 Indicators Baseline Percent (2004) LQAS Percent (2005) DIP Target (2007) Nutrition/Feeding practices Percent of children aged 0-23 months who were breastfed within the first hour after birth. 19.5% 55.3% 50% 16 CARE Child Survival XIX DIP, approved June 2004, p. 26. CARE Sierra Leone Child Survival XIX Project Midterm Evaluation Report June 4, 2006 46 % of children aged 0-5 months who were exclusively breastfed during the last 24 hours. 8.3% 32.4% 20% % of children aged 6-23 months who received a high dose of vitamin A supplement during the last six months. 68.2% 73.2% 80% % of children 6-59 months who received deworming medication during the last 6 months 15.9% 33.9% - Maternal and Newborn Care % of women aged 15-49 who know at least two symptoms that indicate the need to seek referral for emergency obstetric care 37.8% 57.8% 75% % of children aged 0-23 months whose births were attended by skilled health personnel.(Includes doctor, nurse, MCHA) TBAs were not considered skilled 15.1% 23.2% 30% % of mothers able to report at least two neonatal danger signs. 7.4% 23.2% 50% % of mothers who received/bought >= 90 iron supplements while pregnant with the youngest child less than 24 months. 60% 87% 80% % of mothers who received a vitamin A dose during the first two months after delivery 17.8% 52.6% 50% % of mothers who received deworming medication during the second or third trimester of a pregnancy within the last two years. 21.7% 33.7% - % of mothers who took anti-malarial medicine to prevent malaria during pregnancy 31.0% 58.1% 50% EPI % of mothers with children aged 0-23 months who received at least two tetanus toxoid injections before the birth of their youngest child less than 24 months of age. 47.2% 29.5% 70% % of children aged 12 – 23 months who are fully vaccinated (against the five vaccine-preventable diseases). 45.7% 46.2%17 60% % of children aged 12-23 months who received a measles vaccine. 69.5% 69.7% 80% Malaria % of children aged 0–23 months that slept under an ITN the previous night. 0.57%* 18.8%* 15% % of children aged 0-23 months with a febrile episode that ended during the last two weeks who were treated with an effective anti￾malarial drug within 48 hours after the fever began. 27.4% 0% (change protocol – ACT ) 40% Knowledge % of mothers of children aged 0–23 months who know at least two signs of childhood illness that indicate the need for treatment 79% 81.1% 95% % of sick children aged 0–23 months who received increased fluids and continued feeding during an illness in the past two weeks. 48.7% 10.7% % of sick children aged 0-23 months who received increased fluids - 10.7% 17 According to the CARE HQ backstop, this figure may have actually been lower because DPT3 was 35.2%. At the time of the MTE almost one year later, however, UNICEF and DHMT were engaged in a massive door-to-door EPI campaign, causing this number to have little relevance to the current situation. ** CSP staff realized they had been incorrectly assessing this indicator as the skip in the questionnaire captured only sick children as the denominator. This was corrected and a special LQAS study done post ITN distribution in November found 85.3% of children under five to have slept under an ITN the night previously. CARE Sierra Leone Child Survival XIX Project Midterm Evaluation Report June 4, 2006 47 during diarrhoea episode in the past two weeks. % of sick children aged 0-23 months who received increased food during diarrhoea episode in the past two weeks. 28.6% % of mothers with children aged 0-23 months who have ever heard about an illness called AIDS (the KPC questionnaire for HIV/AIDS has been revised so the original indicator is no longer comparable to subsequent assessments). 53.5% 62.1% The project assists in collecting and compiling DHMT data. Available monitoring data from the project M&E system, however, are somewhat difficult to interpret. Due to difficulties with the KPC “skip” pattern, the baseline ITN usage indicator only counted children who had fever, not all children. (But other surveys indicated ITN coverage nationwide was very low). Confidence intervals for the LQAS indicators are a bit wider than for the KPC survey, so results should be interpreted with caution. Although planned for every 6 months, the only LQAS took place in May 2005, prior to some of the major community-based project BCC activities, such as training the CBGPs, and almost one year prior to the MTE. The lower findings in some MNC intervention areas are not surprising, as the project only planned to begin many activities in this area after the MTE. On the other hand, referrals for danger signs and skilled delivery have risen, even before major MNC activities began. Many of CSP and MNC BCC messages are included in the initial CHC training topics. Another LQAS is scheduled for around June 2006. The results of the second LQAS will probably be more reflective of the impact of the CSP because most of the major CHC and CBGP training was started after the previous LQAS. In addition, the CBGP groups will not start working until after the MTE, although they have been helping the health workers at the PHUs since they received training. Data on process indicators are collected on a regular basis, submitted in regular reports to CARE and compiled in the Annual Reports to USAID. A community based HIS was done through disease monitoring boards, listing major diseases affecting community members are in all communities. The community monitors routinely complete the boards with periodic supervision from CARE SL as well as MOHS monitoring and evaluation officers. Monthly disease monitoring reports are sent to the MOHS through the PHU staff for further analysis. Given the low-literacy within communities, monitoring boards are currently being reviewed. It is anticipated that boards with pictures, depicting the most common diseases experienced in the community (their perception at least), will be drawn by the community members themselves. This might also encourage greater ownership and use of the boards on the part of the community members. Through a participatory planning process, project staff worked with the newly formed VDCs and community members to develop social maps of their respective villages, depicting the state of the village before the war, its present condition, and the common future that they anticipate to achieve. Figure 1 illustrates an example of a map depicting the future plans of Bendukura village (Wara Wara Yagala Chiefdom). The plan illustrates the vision for reconstruction of the village, CARE Sierra Leone Child Survival XIX Project Midterm Evaluation Report June 4, 2006 48 for the building of a new school, mosque and court barrie, football field, new toilets and water wells and outlines where new houses are going to be constructed. Fig 1: Linen Social Map depicting Plan for the Future of Bendukura village Village Based Census Through discussions with the District Health Management Team (DHMT), CARE SL recognized that demographic planning for district level health activities was based on numbers from the 1985 census. In recognition of the significant changes Sierra Leone has gone through in the past ten years as well as population growth, CARE SL in collaboration with target communities carried out a village level demographic data collection activity. The process provided useful information at two levels. It allowed for accurate planning as well as monitoring of health activities. Secondly, it helped CARE SL and the DHMT to identify more accurately how many beneficiaries are currently living in the five chiefdoms and the catchment populations for each PHU. A nationwide census is planned in Sierra Leone to update statistics beginning in late calendar year 2004. It is hoped that when the census is complete, it will pave the way for a comprehensive Demographic Health Survey (DHS) in the future. CSP data collection supports the DHMT/MOHS system. Additional info is collected for UNICEF-supported activities. UNICEF has developed a new ITN monitoring form, which will be used by the CSP when nets are distributed. Managers use monthly plans to assess individual workers. Extra mobilizer hired which increased female staff and she is doing well. The DMO has changed three times since the beginning of program and the new, experienced DMO is doing his own assessment of the situation in the District. He participated in the MTE stakeholders meeting where KPC and LQAS data, plus results of qualitative assessments were presented along with discussions of recommended next steps. Understaffing will make analysis and action a continuing challenge. Now that the District Council is involved, they also are using data. Based on their findings they are financing support for two new MCH aides, but not yet supporting employment for more doctors and nurses. CARE Sierra Leone Child Survival XIX Project Midterm Evaluation Report June 4, 2006 49 LQAS, COPE, PRCA (Participatory Rural Communication Appraisal survey) were used for multiple purposes, including formative research on cultural and household/community health practices, designing quality improvement strategies and monitoring program progress. Effectiveness of program strategies are monitored by direct observation, data on a variety of process monitoring indicators, health service utilization statistics and LQAS, community health club action plans and accomplishments, morbidity and mortality data; monthly reports; measuring increases in number of males participation in promoting the health of mother and child; number of health education sessions facilitated by CHC members at PHU. It would be useful for CSP to provide additional emphasis on building the capacity of DHMT in the LQAS methodology. 8. Technical and Administrative Support As part of the DIP development process, CARE and IRC received technical assistance to develop the CSTS CSSA sustainability framework. Although helpful in conceptualizing important elements of sustainability, the general consensus was that revisiting the framework at the time of the MTE would be too time-consuming to do well. The project internally reviewed the CSSA and finalized baseline indicator calculation immediately prior to the MTE; this was then reviewed and next steps considered by the CSP team and HQ backstop as they worked on an Action Plan post MTE. While some CSP interventions have been covered in CARE international meetings, additional technical support is required to address the interventions within the project. During the MTE fieldwork, the CARE HQ backstop assessed the nutrition technical support needs to the program and concluded that integrating the overall ENA approach was indicated and that whether or not Hearth would ultimately introduced should be reassessed based on the results of the CRS pilot program. At the annual international Child Survival meeting, CARE will invite ENA experts who have worked for BASICS and/or LINKAGES to do presentations. CARE will provide MNC technical assistance from their Reproductive Health office in the summer of 2006. When that assistance is provided, MNC M&E components should be addressed. CARE might consider adopting the social/verbal autopsy techniques developed by BASICS and Save the Children in Guinea. CARE Headquarters staff have visited the project at least once a year. The CARE Sierra Leone Health Coordinator has visited at least quarterly, until she left in July 2005. It will be necessary to develop a system so that technical materials produced by the project receive systematic review to assure information is accurate and updated over time. This will require CARE HQ and CARE SL to draw upon technical expertise that is not currently available within the CSP. Since it is probable that no single individual will have expertise in all CS interventions as well as CS management, CARE may consider developing an in-country TAG, or group of short-term consultants in various areas (malaria, nutrition, MNC, IMCI) to provide technical advice to the CARE health manager in Freetown as well as the CSP in Kabala. The CARE HQ backstop, a skilled and experienced public health and nutrition specialist is in close contact with CARE Sierra Leone and will be responsible for the follow-through of the MTE Action Plan. 9. Mission Collaboration CARE Sierra Leone Child Survival XIX Project Midterm Evaluation Report June 4, 2006 50 The USAID mission has been supportive of CARE in general and the CSP in particular. CARE has other USAID-funded programs in Koinadugu District, including Title II and the LINKS project that address other factors in food security and income generation not covered by the CSP. Mission staff met with members of the MTE team prior to the field work and was represented at the stakeholders meeting in Freetown. The new draft Mission strategy is rumored to have only one Strategic Objective and focus on Democracy and Civil Society. Recognizing that health is still a serious development concern in the country, CARE is concerned that the opportunity to get USAID/Sierra Leone support for future health program is in jeopardy. At the time of the MTE, the draft Strategic Plan had not yet been shared with partners. Therefore, there had not yet been the opportunity to discuss it with Mission representatives. The CARE Country Director indicated they felt it would be very important to advocate for the continuation of health in Mission objectives. D. Other Issues Identified by the Team No other issues were identified by the team. E. Conclusions and Recommendations (provided in italics) Assessing progress at midterm should be viewed in the context of the environment in northern Sierra Leone. After 15 years of civil war, health care development was frozen in time while many approaches to maternal and child care changed globally. Communities in Koinadugu district and the District government are undergoing rapid change on multiple levels, including decentralization, policy updates, and changes from relief to development environments where international organizations and NGOs have been shifting from direct service delivery to capacity building. CARE’s long involvement in Sierra Leone, which long predates the civil war, allows building on a positive history in the country. In view of the shift from relief to development and from highly centralized MOHS management to decentralized district level health care management, and the low overall baseline development in the area (less than 15% overall literacy), the CARE SL CSP has to be considered as extremely successful. This success significantly rests in the formation of community structures as the foundations for significant health development at the household and community level. Great strides have also been made at improving services the facility level, but given the limited human resources and upcoming shift from vertical programming to IMCI, much work is still left to be done. By its nature, the methodology used (CHCs, partnering with DHMT, UNICEF and other NGOs) builds civil society, directly linking government with the governed. Given the amount of time and trust-building required to organize community structures and develop functional working relationships with partners, especially DHMT, the CSP is on target in the sense of the usual process of program development. Recommendations listed in this report should be viewed in the context of technically strengthening a program that is already relatively strong and has accomplished visible results in reducing mortality and morbidity and improving health outcomes. Because of the integrated nature of the CSP design, attribution uniquely to CARE at the end of the program will be difficult, but this in no way diminishes the magnitude. CARE Sierra Leone Child Survival XIX Project Midterm Evaluation Report June 4, 2006 51 Given the accomplishments thus far, the CARE SL should now shift focus towards technically strengthening the intervention areas and accelerating an exit strategy away from directly assisting the DHMT in routine functions: routine immunizations, supervisory visits, antenatal care and emergency referrals within the MOHS system, and towards institutionalizing outreach and quality assurance as a health delivery strategies. During the final year of the project, CARE should engage the DHMT and partners, facilitating the meeting organization to assist the DMO if necessary, and develop a concrete exit strategy so that quality services are provided by the DHMT without relying on CARE by the end of the program. The conclusions of these meetings might require CARE SL to help the Koinadugu DHMT in advocating with the central MOHS for full staffing at the PHUs. Although the District Council has funded two students to increase staffing and the DMO is recruiting new graduates, the central MOHS should be strongly encouraged to prioritize Koinadugu District’s staffing needs. Sustainability of the CSP health facility strengthening efforts and the ability of PHUs to address the needs of seriously ill children and pregnant women depend upon it. There is evidence from project data and feedback from beneficiaries and partners that in communities where CARE is implementing the child survival program, mortality among mothers and children, as well as overall sickness has decreased. This impression was confirmed by health facility workers. There has been a coinciding increase in positive health behaviors, especially early initiation of breastfeeding, exclusive breastfeeding, increases in appropriate complementary feeding (by report), ITN use, skilled deliveries and hygiene and sanitation practices. Other organizations have already seen the value of the CARE CHC model and are adopting it into their programs. Obviously, this is an approach that should be scaled up. The question remains about how to sustain enthusiasm and commitment, as well as the quality of health and nutrition promotion activities over time. Scale-up recommendations CARE should determine the essential CHC components and estimate their costs. The current CHC training materials, including the manual should have technical content updated. Then the CHC methodology should be shared with other programs. A plan for disseminating new information over time should be developed with partners. A plan for retaining CHC membership over time should be made. This may involve some non-monetary incentives. The role of CARE in improving quality of care at the health facility level is unclear. On one hand, CARE states that the MOHS is responsible for clinical services and the direct service delivery in all aspects of the CSP. Yet CARE’s involvement in drug logistics, supervisory checklists, supervisory visits, and the COPE assessment with follow-up action plans would indicate that CARE’s involvement in the day to day function of health facilities is substantial. In addition, other organizations (especially UNICEF and World Bank) play substantial roles in health facility capacity building. Roles and responsibilities of each organization are not clear. All CSP interventions are being implemented in an environment where there are multiple organizations working in the same area. For example, UNICEF and the DHMT were completing an EPI and deworming tablets door to door mop-up in the area at the time of the MTE. In many cases, such as the initial free ITN distribution, CARE, DHMT and UNICEF coordinated well; in CARE Sierra Leone Child Survival XIX Project Midterm Evaluation Report June 4, 2006 52 other cases the project seems unaware of activities in the same communities where they are working. CARE should support the DMO to facilitate a meeting, or series of meetings, with all of the key players in the CSP intervention in Koinadugu District. The objective of these meetings would be the mapping of current CSP and related partner activities and develop plans to support the DHMT in coordination of the players in the CSP interventions. CARE should then decide which activities in the DIP require adjustment and submit these changes as part of the MTE Action Plan. Measuring program progress CARE has collected important population data via the KPC and LQAS, though some technical issues have missed opportunities to completely measure program impact. The COPE and Institutional Assessments are valuable in measuring the processes and inputs required to improve many aspects of quality of care. Program data collection is helping to understand coverage and access issues in communities and document behavioral changes resulting from increased awareness and changing social norms. There is evidence of increased health facility utilization and appropriate prevention messages at health facilities. Measurement of clinical health worker performance, when seriously ill children or pregnant women are referred to the facility for case management of malaria, malnutrition or delivery is an important missing element. In order to meet Objective 3 of the CSP, CARE and partners should develop a method for measuring changes in the quality of health worker clinical performance related to the CSP interventions. Partner roles and responsibilities relative to health worker clinical training should be clarified. Technical Strengthening Malaria remains the major cause of infant, child and mortality and morbidity. CARE has developed significant community-based strategies that other major RBM partners need to implement their programs. How CARE SL and CARE HQ can best build upon the lessons learned from the CSP over the long run remains an open question. The new DFID funding for MOSI will complement and extend these strategies and is recognition of the value of this contribution. Even if astoundingly successful, the CSP will not bring the malaria problem in project communities under control by 2007. CARE should decide whether or not to make a long-term programmatic commitment to malaria control and disseminate the lessons learned within other CARE programs in Sub￾Saharan Africa. CARE should actively engage the SL CCM and advocate for a major role in future Global Funds applications as well as engage the RBM partnership in SL for a greater role in the country. CARE has more experience in community-based programs and linking communities with first line health facilities than other members of the RBM partnership, and should strongly consider disseminating lessons learned from the CSP, and the complementary MOSI program with all of the partners. Community level surveillance for ITN utilization, MOUs with VDCs for drug accountability, and engaging communities for timely care-seeking for treatment are all innovative approaches where CARE’s CSP approach provides “value added”. CARE Sierra Leone Child Survival XIX Project Midterm Evaluation Report June 4, 2006 53 CARE planned for the MNC intervention to begin major activities in the 3rd year of the program. Given developments in Koinadugu District, the timing was probably fortuitous. Recent major District Hospital infrastructure improvements, funded by the World Bank and UNICEF, mean that comprehensive EmOC services will be available soon, when they were not at the beginning of the program. The ability of the GOSL to provide sufficient trained staff to support these services will be a major constraint. In spite of the COPE assessment and QI capacity-building at the PHU level, the ability of the personnel working there to deliver quality ANC, Labor and Delivery and postpartum clinical services is not known. Since this capacity was not measured at the beginning of the program, the baseline for measuring program impact should be determined in year 3 of the program, if at all. Specifics in the MNC intervention approach, with coinciding partner roles and responsibilities need to be clarified. CARE has already acknowledged the need for MNC technical assistance and CARE’s Senior Technical Advisor for Child Health, Joan Jennings, had already started the process of arranging the TA by the end of the MTE fieldwork. CARE has already developed ground-breaking Maternal and Newborn Care training materials and manuals which can be put to very good use in the program at this time. MNC technical assistance, as suggested by CARE HQ, should take place as soon as possible. TA should include clear follow-up action recommendations, adjustments to the DIP workplan and measurement methods. Partners’ roles and responsibilities should be well-defined. Recommendations from this TA should be submitted with the Action Plan, if possible. In the meantime, there are basic household prevention measures (such as clean birth and placing/keeping mother and child together immediately after delivery and addressing key factors hindering women from seeking post-partum care) that can be included in CHC training sessions prior to a more comprehensive plan. This plan should clearly state whether or not the CSP still intends to train HBLSS as described in the DIP, or substitute some other methodology. CARE HQ should plan for technically supporting this intervention. (See TAG recommendation below.) This may require budget adjustments. To provide leadership in supporting PHU and community capacity in IMCI and C-IMCI as planned in the DIP, CARE SL CSP staff will need IMCI capacity building themselves. This adjustment should be supported by CARE HQ staff and the new Health staff person in CARE Sierra Leone. Many CSPs in other countries have successfully improved quality case management during this change process. While quality of clinical management of malaria, malnutrition, and obstetric complications is primarily the responsibility of the MOHS, the CSP partnership, as stated in the DIP, undertook the responsibility to see that the quality of services at the health facility is appropriate. The CSP has already addressed many quality health facility elements. In situations where CARE does not provide direct services, then advocacy and capacity building support are appropriate. At this point, it is not clear the specific leadership roles that CARE CSP seeks to fulfill in moving MOHS policies and practices forward. Waiting for all DHMT systems to be in place will probably mean that there will not be enough time to implement the facility-level actions to meet all program objectives by the end of the program (primarily log-frame activities relative to Objective 3). At the community level, however, where CHC and VDC training supports the 16 key household behaviors18, the CSP should capture these 18Reaching Communities for Child Health and Nutrition: A Framework for Household and Community IMCI, Child Survival Technical Support Project (in collaboration with CORE and BASICS), 2001, pp. 45-46. CARE Sierra Leone Child Survival XIX Project Midterm Evaluation Report June 4, 2006 54 activities in their reporting as these are supportive of C-IMCI even if not explicitly stated as such. [See technical intervention recommendations] CARE can consult other PVO CORE, CSTS and other CARE programs for the lessons learned in the quality improvement process in the midst of changing from vertical to IMCI MOHS programs. The CHC model of community mobilization and venue for behavior change communication is effective and applicable to both health and integrated development programs. Training materials in current use by the CSP are useful, but some messages (notably in malaria and diarrhea treatment) need to be updated. Once this is accomplished, CARE should devise ways of revising these materials over time to keep them technically relevant. Copies of updates should be provided to other projects, both within CARE as well as other organizations after they have been done. Sustainability of program activities is divided between community-level efforts, primarily at the CHC and VDC levels and at the DHMT PHUs. Staff shortages may limit the extent that the DHMT can completely fulfill its partnership potential. CARE can not control the staffing problem, but working with the District Council and the DHMT can support them in advocating for additional staffing. Given the extremely high mortality and morbidity in Sierra Leone, it is reasonable to expect that these rates will remain among the highest in the world, no matter how successful the CSP is during the funding period. CARE HQ and CARE SL should begin strategizing now as to what the appropriate follow-on or scale up activities should be after the CSP ends. It would be a tragedy if the lessons learned during the initial program were not built upon. The Title II program and other donors may implement pieces of the methodology, but are unlikely to adopt the comprehensive CSP approach. The rumored dropping of health in the draft USAID Mission Strategic Plan is extremely worrisome, as the local USAID mission is the most logical place for CARE to turn for scale-up support. CARE HQ has to take many factors into consideration when selecting which country offices to support for CSP proposals. On the other hand, CARE HQ staff state that strong interest and involvement on the part of the country office is are strengths of the program. CARE SL and CARE HQ should begin discussions on follow-on funding options for after 2008 as soon as possible after the MTE. The ground work for mission support should be established by updating USAID mission health backstops on CSP progress. CARE SL should continue to advocate with USAID Sierra Leone to maintain health within Mission strategic objectives. F. Results Highlight (Note that CARE has been working to capture community members own views of their experience. This story was told to a CARE Sierra Leone Child Survival Project field staff, when asked to describe how they felt about the project after these first two years.) "There was great CARE Sierra Leone Child Survival XIX Project Midterm Evaluation Report June 4, 2006 55 excitement when CARE entered my village in Gbindi II. Our village is far from the nearest town and hard to reach. A meeting was held on Community Health Clubs (CHC) in which the entire village was invited to attend. CARE was to implement a water well and pit latrine construction project. In addition, we were informed that the Child Survival Project together with the Water and Sanitation project would be organizing weekly health education sessions in the village on such topics as hygiene, nutrition, malaria prevention and control, exclusive breastfeeding, complementary feeding, immunization and maternal and newborn care. Best of all, it was to be free of cost. I signed up to be a member and received a membership card. We were to be involved in all stages of the project - planning, designing the project activities and contributing local resources. We even signed memoranda of understanding with elected village development committees (VDCs) leadership for project transparency and accountability. I have since learnt a lot from the CHC sessions and has brought significant change in my life. I can now sit together even with the men in my community and be part of discussions and decision making about the village. I was always shy in making public statements because I thought my contributions were not meaningful, but with the arrival of CARE with the CHC clubs, together with the rest of the women in the community I have felt very useful in meetings and decision making in terms of developing the village. Secondly, my daughter was not going to school before the CHCs were introduced. We always thought that only the boys could manage the tough school, but with the knowledge gained from the CHC, today my daughter is now going to school and she is doing extremely well. I don’t suffer from such issues as diarrhea. I know how to prepare ORS and to source water from the CARE constructed wells for drinking. I know how to go about some first aid treatment like snake bite, drowning, fire burns etc. personal hygiene. I take my child for regular monthly weighing to know if she is growing well or not. I will continue to share all lessons with my children and one day hopefully, even with my grandchildren because my greatest wish is for long life and for ‘beteh fambul health’ (local expression meaning "for better family health"). I think upon these things as I go to wash my family’s clothes and share a laugh with my friends. It is funny how we used to dry our washed clothes on the ground but now that we have more knowledge, we know about hanging clothes on the wash line to keep them clean and free from insects and disease." III. Action Plan Introduction Please find Attachment H, a post-MTE Review of the DIP Work Plan, with comments on the status inserted in italics. Key issues pending from the DIP Work Plan with partner DHMT were the lack of national MOHS roll-out of IMCI in Koinadugu District and the depth of Quality of Care assessment and intervention completed by the project as compared to DIP descriptives (which is related to the lack of IMCI roll-out). Key issue from the DIP Work Plan with local NGO potential partner was a lack of follow through after institutional capacity assessment during the early stages of the project. CARE Sierra Leone Child Survival XIX Project Midterm Evaluation Report June 4, 2006 56 As the DIP Work Plan also described broad areas of action, a detailed post-MTE Action Plan Calendar was developed immediately after the MTE, with participation by CARE HQ and the full CARE Sierra Leone Child Survival Project (CSP) team. This was then shared with partners in Koinadugu District and further refined (Attachment I). The CSP team will also develop a more detailed Annual Work Plan in September 2006, for the next Fiscal Year of the project and taking into account advances during this fiscal year period. It is expected that filling the position of CARE Sierra Leone Health Sector Coordinator will provide a great deal of support for CSP activities planning, particularly in coordination with the new overlapping malaria project funded by DFID and in support of the project MNC strategy; therefore, some planning will wait for the participation of this person, who is expected to be hired as soon as possible. CARE Sierra Leone senior management, the SL Child Survival Project team, and CARE HQ agree with the general recommendation that the project shift the balance of action / responsibility / lead role for different intervention activities from CARE to partners, with the end of the project in mind. All of the project strategies have been and will continue to be developed with exit and longterm sustainability in mind. A concrete exit plan, in collaboration with partners, will be developed in the 4th quarter of the 4th fiscal year of the project, with handover occurring throughout the final year of the life of the project. Response to Recommendations 1. Scale up of Community Health Clubs: The project has received expressions of interest from multiple NGOs in Sierra Leone for training in the CHC methodology and use of the "Communicating Health, Communicating Rights" manual. A "technical update" sheet with comments to be included in any training and/or distribution of the manual can be found in Attachment J. CARE will facilitate training workshops at CARE sub-office sites throughout the country to share this methodology at scale with interested NGOs, beginning immediately. Facilitators have already been identified and organized; they have only been awaiting the technical update sheet. When the new full-time Health Sector Coordinator is hired, a strategy for any future technical update of the materials will be developed (if necessary; it is expected that the technical update comment sheet will be sufficient through the remaining life of this project). The project does include strategies for retaining CHC membership, including non-monetary incentives but will continue and strengthen focus and internal reflection on the effect of these strategies. These strategies include: linking CHC members to other CARE projects in the district (such as the LINKS micro-enterprise program), assisting CHC members and Village Development Committees to develop plans to access other opportunities, non-monetary incentives such as certificates or CARE t-shirts, etc. They will continue to keep CHC member retention in mind as new opportunities occur. 2. Strengthen Essential Nutrition Actions: Recommendations to strengthen the project Essential Nutrition Actions can build upon the good base of nutrition knowledge established with the Community Health Clubs and through the training of Community-Based Growth Promoters. Review of ENA messages was conducted during the post-MTE Action Planning period, with CARE Sierra Leone Child Survival XIX Project Midterm Evaluation Report June 4, 2006 57 copies of the USAID/CSHGP Technical Reference Materials for Nutrition provided, highlighting the ENA section. In addition, two staff from the CSP (the Acting PM and Health Educator) attended the CARE 11th Annual Child Survival workshop where the theme was "Infant and Young Child Feeding for Health" and the keynote speaker presented a one-day session on ENA. Additional ENA materials were provided during this workshop. The CARE Sierra Leone CSP team will be able to draw upon these materials as they develop additional ENA training and messages, with particular emphasis on complementary feeding for children 6-23 months of age and sick child feeding. 3. DHMT, partners and technical interventions: The continued delay of the roll-out of IMCI strategy in Koinadugu District, along with the changes in District Medical Officer (three times) has been one of the key obstacles to coming together to define complementary roles and contributions to technical interventions. As IMCI rolls out, CARE HQ will seek useful input from other CSHGP programs, CORE, CSTS, etc., for lessons learned in the improvement process when changing from vertical to the IMCI approach. To date, the various DMOs/DHMT for Koinadugu District have turned to the national MOHS for technical assistance for training, etc., in new areas such as the new Malaria protocols and/or IMCI (rather than to CARE or other NGOs). For initial technical trainings organized by UNICEF/MOHS, the DHMT more often seeks CARE assistance for logistic and other support (which is always provided). The CSP team has provided training to DHMT staff on the use of participatory adult education methods. CARE Sierra Leone will be more pro-active in two aspects: offering technical and logistic support for follow-up, supervision, on-the-j9ob training, etc. after the MOHS trainings; and the new Health Sector Coordinator will meet with MOHS and see if CARE can also be more proactive in getting nationally approved trainings to Koinadugu District more quickly. Malaria case management training for PHU staff that was postponed by the DHMT due to the EPI Rapid Result Initiative activity was completed in May 2006 and CARE provided logistic support, along with input on participatory training methods. Since the MTE fieldwork, the CSP team mapped NGO activities in MNC in Koinadugu District, discussed with DHMT and took the lead for convening a meeting of all actors (the meeting had to be postponed but will is rescheduled for July, prior to the planned visit from CARE HQ maternal health TA in August). As part of the detailed planning for the new CARE malaria project, a Malaria Working Group has been established involving the DHMT, CARE, Christian Children's Fund, Action for Development, the Red Cross and others. Two meetings have been held since the MTE fieldwork, in April and in May. As noted by the MTE consultant, the project does implement most of the actions at the community level which comprise the C-IMCI model but do not recognize it as C-IMCI. CARE HQ will provide documents on C-IMCI for project review and follow up on this understanding during future field visits. 4. Assessing clinical care performance: CARE does prefer for leadership of any DHMT staff assessment activities to come from the DMO. The CSP team has discussed with the DMO including some aspects of clinical care performance capacity into plans for joint supervision visits to the peripheral Health Units (DHMT/CSP). Prior to the MTE, the DMO was conducting CARE Sierra Leone Child Survival XIX Project Midterm Evaluation Report June 4, 2006 58 visits to the field to become oriented to the district. He has plans for potentially filling two sub￾district positions with additional staff, who he feels would then have responsibility for supervision of the quality of clinical care in their region. Since the MTE, a DHMT plan for joint DHMT-CARE supervision of PHU staff was established and monthly joint supportive supervision activities have commenced; however, these do activities do not yet fully assess the quality of clinical care for the various components involved in reducing child mortality. Note that the COPE assessment did reflect quality of care in terms of the availability of essential medicines, equipment, protocols, etc. and there has been follow up in some of these key areas. 5. CARE Sierra Leone role in Malaria: The part-time Health Coordinator for CARE Sierra Leone is an active member at the national level of the Malaria Working Group. CARE Sierra Leone is committed to being a key player in the Roll Back Malaria strategy. The recent submission and funding by DFID of the MOSI project (covering Koinadugu and 3 other districts in the region will provide multiple opportunities for the CSP team to share successful strategies. The process of sharing lessons learned regarding LLITN distribution is underway. CARE has been invited by UNICEF and Koinadugu DHMT to share lessons learned from the participatory distribution of ITNs with DHMTs from Kono, Bombali, Pujehun and Bo Districts. This will be scheduled as soon as possible, no later than the end of this fiscal year. 6. Technical assistance for MNC strategy, 2nd half of project: A CARE HQ advisor for maternal and newborn health will visit the project in August and assist the team and partners to develop a detailed MNC strategy. The MTE recommendations have already been shared with the advisor and will inform the strategy development. (See additional information No.2 above). An HBLSS consultant from the American College of Nurse Midwives gave a presentation on HBLSS to the participants of the recent CARE Annual Child Survival Workshop (June 24-30, 2006; in Atlanta). The CARE Sierra Leone Acting Project Manager and Health Educator attended and expressed interest in this strategy; however, it is not likely that there is sufficient time or funds available at this point in time (it could be considered for follow on maternal child health programming). It is noted that the CARE HQ technical advisor can draw upon the manuals "Promoting Quality Maternal and Newborn Care" (CARE) and "The Health Newborn" (CARE/CDC). Also, the regional Helen Keller International office has been in communication with several NGOs, including CARE Sierra Leone, with interest in activities in support of maternal nutrition. Recently the HKI regional representative and someone from the A to Z Project visited the CARE Sierra Leone CSP in Koinadugu and further planning is expected. Discussion of the assessment of quality of care in relation to the MNC strategy will need to be a part of strategic planning in August, with the DMO/DHMT. In regards to CSP efforts to establish emergency transportation plans with project communities and DHMT, since MTE the team developed a mileage matrix of distance/cost of fuel for all areas of the chiefdoms in which this CSP is active. CARE then facilitated a meeting between the DMO, the District Council and VDC representatives from each Chiefdom at which it was agreed that community members should provide fuel for the ambulance to and from communities for emergency referral. CARE has shared this mileage matrix at the community level through the CHCs and VDCs and encouraged community planning for savings and organization. CSP also hopes that the on-going CARE Sierra Leone Child Survival XIX Project Midterm Evaluation Report June 4, 2006 59 integration of CSP communities with the CARE Sierra Leone LINKs village savings and loan project will provide an opportunity for communities to generate funds that can be contributed to the emergency transport issue. In regards to critical behavior change communication that does not need to wait for MNC strategy development, the CSP team together with peripheral Health Unit staff has designed a sensitization strategy to encourage newborn/mother staying together post-partum both in PHUs and in the community post home delivery. This sensitization communication plan is being disseminated through the various CSP BCC channels (CHCs, radio, VDCs, etc.) and is targeted to all pregnant women, TBAs and elderly women in CSP communities. 7. Follow on funding options after 2008: Both CARE HQ and CARE Sierra Leone have this maternal and child health interventions on their "radar screen" for future funding opportunities. The US Ambassador to Sierra Leone and the USAID mission director visited CARE and other NGOs in Koinadugu District shortly after the MTE, and information on all CARE activities was shared. The USAID CSHGP team recently assisted CARE HQ in further ensuring the mission is aware of this project's achievements, coordinating a conference call between CARE HQ, IRC, and Christine Scheckler (USAID office in Freetown) during a visit to Washington in April 2006. Update on Activities To-Date, Post-MTE Action Plan Calendar (Attachment I) Activities scheduled for April and May have included: · CSP General: CARE has interviewed extensively for SL Health Sector Coordinator and are in the final decision-making processes. · Nutrition: Activities to continue to establish CB-GM are proceeding as planned. · MNC: District meeting of all actors involved in radio maintenance, sustainability was held; UNICEF has verbally provided some new information on radio source and states they have put in a procurement order. CSP will either track down more details from UNICEF and/or wait for shipment arrival to DHMT. · Malaria: As noted above, DHMT training activity for PHU staff which had been postponed was held in May, with CARE CSP support. A more active Malaria Working Group was (re)established. · HMIS: CSP was unable to fit into their programming an LQAS in June (turnover from International Project Manager to national Acting Project Manager; CARE Annual Child Survival workshop attendance, and more). CARE HQ has recommended they plan for June 2007 and put their efforts into involving DHMT more closely in the next LQAS, orienting and training them in this strategy. · BCC/Mass Media Strategy: CSP Health Educator met with Search for Common Ground who shared information and agreed to invite some CARE staff to all of their training workshops. Information on a Listener Survey was provided which CSP M&E Officer is reviewing; protocol for methodology will need to be defined. · Partnership with other NGOs: CSP PM met with key staff for CES in Koinadugu District -- they are unsure if they would like any specific support from CARE and suggested they meet again after a country level strategic planning workshop their organization was holding soon. CARE CSP and CES have agreed to better coordination and collaboration at the community level and put some immediate steps into place, sharing information on upcoming activities. 1 Attachment A. Baseline Information from the DIP CARE Sierra Leone (CARE-SL) and the Ministry of Health and Sanitation (MOHS) present the Detailed Implementation Plan for CARE’s Child Survival XIX Project under the standard category in Koinadugu district in the Northern Region of Sierra Leone. The project dates are October 1, 2003 through September 30, 2008. This project seeks to improve the health status of children under 5 and women of reproductive age in a remote district in a former rebel stronghold through innovative strategies that build partnerships between communities and government. Infant and child mortality rates in Sierra Leone are 170 and 286 deaths per 1,000 live births, respectively1 . The maternal mortality ratio is estimated to be 1,800 deaths per 100,000 live births2 . The Koinadugu District Knowledge, Practice and Coverage Survey (KPC) found vaccination rates in Koinadugu district as 45.7% of children 12-23 months fully vaccinated before their first birthday. Just over 69% of these children received a measles vaccine 47.2% of mothers receiving at least two tetanus toxoid injections (TT) before the birth of their youngest child aged 0-23 months. Underweight (< 2 SD) among children 0-23 months in the KPC survey was found to be 26.5%. Malaria is the most frequent cause of death in children under five at Koinadugu Hospital, accounting for 40 percent (53/132) of child deaths in 2001-023 . Less than 1% of children with a febrile episode in the last two weeks had slept under an insecticide treated bednet the preceding night. The population of Koinadugu is dispersed in small villages with limited access to haphazardly staffed peripheral care units (PHUs). CARE will work in all eleven chiefdoms of Koinadugu to strengthen the MOH health system, and conduct intensive project activities in five of them including Dembelia Sinkunia, Follosaba Dembelia, Neini, Sengbe, and Wara Wara Yagala, focusing on the most crucial partner in Koinadugu: the community. In these five chiefdoms, CARE in partnership with PHU will implement interventions in immunization (EPI), nutrition, malaria and maternal and newborn care (MNC) through a grassroots, civil-society building variation of the overarching Community Integrated Management of Childhood Illness (C-IMCI) approach. The CARE approach promotes voluntary participation in “community health clubs” (CHCs) for any community member who wishes to join. The CHCs will disseminate health information, promote healthy practices and spearhead community support for the formal health system. They will also focus on mobilizing community members for EPI outreach, community-based growth promotion and birth preparedness. The Project will collaborate with the Ministry of Health and Sanitation (MOHS) in district-wide activities to train PHU staff in IMCI, expand and improve services, and plan a Behavior Change Communication (BCC) campaign to improve family and community practices. The CARE capacity-building strategy will work through a partnership structure, with local organizations such as CHCs, Radio Bintumani (with the assistance of Talking Drum 1 Survey Report on the Status of Women and Children in Sierra Leone at the End of the Decade. GOSL. November 2000. 2 Source: Unpublished analysis. MICS2 database. UNICEF and Central Statistics Office, MODEP, GOSL. 3 Source: MSFB/DHO Database, Koinadugu District Hospital, Kabala, Koinadugu. 2 Studios) the MOHS and Christian Extension Services (CES). The CHCs will be responsible for implementing health promotion activities at community level, support government health services. Radio Bintumani will work with CARE and MOHS through support from Talking Drum Studios to design and broadcast key health messages, while CES will work through inter-sectoral collaboration at the community level for health promotion. The MOHS at all levels will be involved in setting policy, managing the health system, providing health services and promoting improved health practices. This approach builds local capacity and sustainability and allows the Project to maintain continuity of project activities if security declines. Direct beneficiaries of the Project include an estimated 48,630 children under five and 51,491 women aged 15-49 years. Secondary beneficiaries include direct beneficiaries of Health Units from the remaining six chiefdoms. The goal of the Project is to improve the health status of children under five and women of reproductive age in Koinadugu district through the achievement of three principal objectives: 1.Strengthened family and household knowledge and decision-making skills related to health of women and children resulting in the practice of positive behaviors to prevent, recognize and manage common diseases; 2.Enhanced community capacity to form groups and institutions that sustain health initiatives, demonstrate social cohesion, and promote good governance mechanisms; and, 3.Improved quality and accessibility of services provided by MOHS personnel and MOHS extension services. The Project will implement the following four interventions through the C-IMCI strategy: The EPI intervention (15%) will focus on raising vaccination coverage of children and pregnant women from current low levels. CARE will work with communities and MOHS to promote EPI outreach through the CHCs. The second prong of the CARE strategy is to support CHCs to use appropriate BCC strategies to increase demand for and utilization of EPI services. The nutrition intervention (20%) will work through CHCs and other community-based organizations (CBOs) to promote the early initiation of breastfeeding, exclusive breastfeeding (EBF), complementary feeding and improved Vitamin A (VA)/iron intake for women and children. CARE has complementary multi-sectoral activities that support improved nutrition such as water and sanitation in Koinadugu District. The malaria intervention (35%) will confront the high prevalence of malaria and self￾treatment by training PHU staff in recognition of malaria and standard case management, educating community members about malaria and its treatment, promoting intermittent treatment of malaria amongst pregnant women, and promoting and selling insecticide treated mosquito nets (ITNs) through social marketing. The MNC intervention (30%) will focus on improving access to information and basic maternal health care by providing educational sessions on danger signs recognition, birth preparedness at the community and household level, promoting TT vaccination and iron supplementation for pregnant women and VA supplementation for postpartum women, and training PHU staff in intermittent prophylaxis of malaria in pregnant women. 3 The proposed program is a collaborative effort with USAID Sierra Leone and incorporates the feedback of Ms. Kathy Jacquart, Reproductive Health Advisor, and USAID Sierra Leone. Other organizations consulted in the development of this program include WHO, UNICEF and IRC. The total budget is $2,013,054 with $1,488,582 from USAID and $524,472 from CARE. Principal authors of the DIP were Namita Kukreja from CARE HQ and Boiketho Matshalaga and Vandy Kamara from CARE Sierra Leone. The contact person for the proposed CS program is Sanjay Sinho at CARE HQ. 4 5 Attachment B. Evaluation Team Members and their Titles 1. CARE Sierra Leone Child Survival Project 2. MTE March 2006 – Data Collection Teams Team Alpha Jean Capps Consultant Momoh Koyanday CARE SL CSP4 CHM5 Hassan Kamara, TL6 (Team Leader) Koinadugu District Health NLTCP Supervisor Malloh Sama CARE SL DRP7 CCU PACA Health Sowo Tucker CARE SL CSP Health Ed. Officer Team Beta Bockarie Sesay, TL CARE SL CSP M&E Officer Andrew Papayo Sesay District Council Gibril S. Bangura CARE SL DRP M&E Officer, Makene Iysattu Kamara CARE SL CSP CH Field Supervisor Team Charlie Aminata Sesay8 CARE SL CSP CH Field Supervisor Sam Anthony IRC HIS Manager, Kenema Mariana Stephens, TL CARE USA CH Program Associate Sayoh Francis CARE SL CSP CH Field Supervisor Team Delta Juliette Tucker CARE SL SAY9 Field Associate Bernadette Udo IRC CSP Program Manager Vandy Kamara, TL CARE SL CSP Asst. Project Manager Foray Kamara 4 CSP: Child Survival Project 5 CHM: CSP Community Health Mobilizer 6 TL: Team Leader 7 DRP: Development Relief Project 8 IRC: International Rescue Committee 9 SAY: Sexual and Reproductive Health Project 6 7 Attachment C. Evaluation Assessment Methodology Background documents including project proposal, DIP, KPC Survey Report, two annual reports (including the LQAS and COPE reports) and other relevant technical reports and papers were reviewed prior to traveling to Sierra Leone. Additional information that CARE wanted to feed into the CARE Sierra Leone Strategic Plan was also included in the MTE plan. Draft evaluation tools were written and forwarded to CARE personnel for feedback. In Sierra Leone, CARE personnel at each level were interviewed about their knowledge of the project, and what they would like to learn from the evaluation. Managers of other relevant CARE programs in Sierra Leone, including the Title II program and authors of the new MOSI malaria project were interviewed. In Kabala, stakeholders and CARE staff participated in a two day planning process that included data review, project time line, review of program log frame objectives and indicators, and workplan. Participants broke into groups to identify key information that needed to be collected in community visits. Criteria for community selection was established prior to beginning the MTE, and refined during the planning workshops. Two participants from CARE’s colleague PVO, IRC, participated in on the MTE team and participated in the planning. Information was collected in community visits, key informant interviews, site visits to health facilities, and staff and management interviews. CARE International’s Senior Health Advisor for Child Health conducted additional key informant interviews and a follow-up to the recommendations from the COPE assessment conducted earlier in the project. Participants were divided into groups and asked to answer the questions on cross￾cutting approaches found in the MTE guidelines. During the subsequent group discussion, additional issues about equity, scale-up, and building civil society were noted for inclusion in the report. Stakeholder briefings about preliminary evaluation findings were conducted in Koinadugu District and in Freetown. Attendees included CARE and partner staff, international organizations, other PVOs, and representatives of the Ministry of Health and USAID Freetown. CARE’s Senior Health advisor remained in Freetown to develop the Action Plan and conduct additional discussions on the sustainability strategy. A draft report was sent to CARE and revised following comments. The Action Plan and Results Highlight were drafted by CARE for inclusion in the report. 8 9 Attachment D. List of Persons Interviewed CARE International in Sierra Leone Nicholas R. Webber Country Director Garth Van’t Hul Deputy Country Director Mohammad Ziauddin Finance and Administrative Coordinator Abu Bakarr Fufana Finance Controller Finbarr Sweeney Project Manager, Development Relief Program (DRP) George Tobaiwa Sector Coordinator USAID Freetown Edward Joseph Benya Team Leader, Strategic Objective One Reintegration Specialist Christian Extension Services Joseph K. Sesay Deputy National Director/Program Manager District Management Health Team, Koinadugu District CARE Kabala CSP Community Immobilizers Momodu Sesay Nieni Chiefdom Ramatu Rebecca Monsaray Wara Wara Yagala Chiefdom Momah Z. Koyanday Folosaba Dembellia Chiefdom Mohamed Kamara Sengbeh Chiefdom Edmond Joe Bodie Brandon Dembelia Sinkunia Chiefdom 10 11 Attachment E. List of attendees at Presentation of MTE Key Findings on 3/27/06 at CARE office in Freetown, Sierra Leone Name Position Organization Aminata Williams Executive Secretary/Administrator Africare Mohammed Kamara Community Health Mobilizer CARE Sayoh Francis Field Supervisor CARE Mustapha Sama PQCA/Health CORAD-CCU Momodu Sesay Community Health Mobilizer CARE Sowo Tucker Health Education Officer CARE Marian Bangura Programme Officer Helen Keller International Kedrick Kiawoin Health and Nutrition Officer UNICEF Frederica Wyse Asst. Project Officer, Nutrition UNICEF Ramatu Mansaray Community Health Mobilizer CARE Bockarie Sesay M&E Officer CARE Iysattu Kamara Field Supervisor CARE Edmond Brandon Community Health Mobilizer CARE Maureen Cunningham Health Consultant CARE Iyesha Josiah Coordinator Sierra Leone Association of NGOs Clifford Kamara Director, Planning and Information Ministry of Health and Sanitation (MOHS) Yilmar Robelle Health Coordinator International Rescue Committee (IRC) Edward Benya Team Leader USAID Momoh Koyanday Community Health Mobilizer CARE Dr. M.A.S. Jalloh IMCI Coordinator MOHS Garth Van’t Hul Deputy Country Director CARE John Perry Chief of Party CORAD-CCU Joseph Senesie Health Manager World Vision Musa Sesay Health Coordinator Christian Children’s Fund (CCF) Nick Webber Country Director CARE Sylvetta Scott Programme Manager, Nutrition MOHS Vandy Kamara Asst. Project Manager CARE Jean Capps External Consultant Joan Jennings Team Leader, Child Health CARE Boiketho Matshalaga Project Manager CARE 12 13 Attachment F. (Post MTE) Participatory Midterm Review of Action-to-date on COPE Assessment of Quality Action Plan (May 2005) and Selection of Priorities for Remaining Life-of-Project 1. Participants Name Position Organization Vandy Kamara Asst. Project Manager CARE SL Joan Jennings Senior Technical Advisor CARE USA Hassan Kamara NLTCP Supervisor Koinadugu DHMT Andrew Papayo Health Coordinator District Council 2. Priority Actions Selected for Remaining Life-of-Project 1. Facilitate the distribution of drugs to PHU, display price list and work with PHU, VDCs to create awareness on availability of drugs and prices. 2. Facilitate routine radio panel discussions (with balanced representation, with Bintumani Radio Health Public Relations Officer and one representative from DHMT, District Council, health NGO, and community member) on health related issues: e.g. Price list, Family planning, HIV screening, etc. 3. Cross visits of DHMT, PHU staff, District Council and community members to Kono to observe cost recovery and HMIS facilitated through IRC Child Survival Project. 4. Establish VHF radio communication system between PHUs and the District Hospital. CARE to provide. 5. COPE Committee Meeting should be held regularly with new DMO as part of DHMT; schedule quarterly. 3. Midterm Evaluation Review of Action-To-Date Follow-Up to Cope Assessment of Health Service Quality Problem/ Cause Recommendation May 2005 By Whom? When? MIDTERM REVIEW OF ACTIONS March 2006 Priority Action for CARE for 2006-07 Guide 1: Right of Information 1. Clients not aware of the cost of health services • Prepare & mount price lists at PHU’s • Advertise costs on through Radio announcements MCHA In Charge MOHS/ DSMC 3 months A few PHUs have posted drug price lists; many need the list at hand for book-keeping. Facilitate posted price lists and VDC awareness 14 announcements 2. IEC materials are needed on Nutrition, STI’s, HIV/AIDS and FP Supply ICE materials to all PHU facilities Health Ed Unit DSMC/DH S/DMO By Dec 2005 Most PHUs have received posters on nutrition from MOHS Nutrition Department Radio messages 3. Information on screening for HIV/AIDS Advise all PHU staff who the District HIV/AIDS Focal Person is and provide information on the screening process HIV/AIDS DHS DSMC Start Jun 05 Ongoing 2 nurses from District Hospital have been trained and mobile testing has started. N/A for Child Survival Project 4. Cultural taboos, beliefs and practices affecting health e.g. Children are not to eat eggs lest they become thieves • More health education • Target men for health talks PHU-In Charge Start Jun 2005 Ongoing With UNICEF support, some health education has been targeted for Imams to disseminate through mosques, to reach more men. Continued health ed. activities at community & PHU, radio 5. Reducing mosquito bites at PHU • Clear bush around PHU compound • Bury empty tins PHU-In Charge Cmty Health Committee Start May 05 Ongoing Many VDCs have become active in keeping PHU environment clean. CARE promotion VDC collaborate with PHUs 6. PHU support staff are not paid Work with Community Health Clubs and VDCs to provide incentives to support the PHU staff. PHU CHC/VDC Community Start May 05 Ongoing No action taken; discussion has continued and focus will be on incentives for non-MOHS payroll staff. Cross-visit to Kono Problem/Ca use Recommendation May 2005 By Whom When? MIDTERM REVIEW OF ACTIONS March 2006 Priority Action for CARE for 2006-07 Guide 2: Right of Access to Service 1. No referral system in place · District Level: Establish a radio communication CARE / DMO Dec 2005 District Level: Some assessment for VHF radio with UNICEF. CARE will call meeting with DHMT, UNICEF and 15 place system between PHU’s and the District Hospital · Community Level: Establish a Community Based system (hammocks) to carry emergency patients Community Level: Community referral systems (hammocks) in 35 of the 54 CARE CSP operational communities UNICEF and the Health Committee of the Koinadugu District Local Council: Plan for maintain and sustain radio system 2. Poor clinic attendance by clients due to poor attitude of some PHU staff Better mentoring and advice on behavior by supervisors DHS/CS June 2005 DHMT has started monthly meetings at district office: discussion of problems/solutions, and attitude. N/A CARE at this time, as new DMO-led system is tested. 3. No Standard Case Management Guidelines at PHU Provide Standard Case Management guidelines DMO/DHS July 2005 Malaria guidelines have been provided by the Malaria Control Programme of the MOHS and UNICEF CARE to facilitate the guidelines at PHUs and participate in refresher PHU trainings. 4. Inadequate trained and qualified staff both at District Hospital and PHU’s • Government to post more trained and qualified staff to the district. • Upgrade MCHA’s to higher qualification DMO DMO Dec 2005 Dec 2006 District Council has provided financial support for local youth (2) to be trained as MCH Aides DMO requests 5 of this year's graduates to be placed in Koinadugu 5. No delivery kits at most PHU’s Provide delivery kits to PHU’s DHS Dec. 2005 Unicef has provided. N/A 6. No refresher training for DH staff for more than 2 yrs · Conduct Training Needs Assessment for all Dist Staff · Conduct Training of Trainers · Conduct In Service Training DMO DOO Ongoing By Dec. 2006 In Sept.2005, DH staff received training on Safe Motherhood (UNICEF) CARE support for refresher and post-test of GM skills Problem/ Cause Recommendation May 2005 By Whom? When? MIDTERM REVIEW OF ACTIONS March 2006 Priority Action for CARE for 2006-07 16 Guide 5: Right of Access to Privacy 1.No HIV/AIDS testing or counseling available at PHU’s • Provide mobile HIV/AIDS testing to visit PHUs • Train PHU staff in confidential HIV counseling DMO/DHS Lab Tech/ Dist. Matron and HIV/AIDS Counselors August 2005 Mobile testing began in November 2005 Pending training in confidentiality and counseling N/A 2. Men do not accept family planning Sensitization on Family Planning for men through radio discussions and community meetings DSM Coordinator and Community Health Clubs On￾going Some of the sessions in the CARE Communicating Health Communicating Rights are directed to men; the CARE SRH project with youth (SAY) addresses both sexes Not a priority for Child Survival Project Problem/ Cause Recommendation May 2005 By Whom When? MIDTERM REVIEW OF ACTIONS March 2006 Priority Action for CARE for 2006-07 Guide 8: Staff need for good management and facilitative supervision 1. Little supportive supervision at PHU level Implement monthly supportive supervision DHMT Zonal Supervisor On￾going DMO is requesting CHO positions to be filled in Chiefdoms. DMO provided additional input to supervision checklist developed by DHMT and CARE, and DHMT approved CHOs and CARE will provide supportive supervision to PHUs 2. Birth & Death Register Booklets are not available in all Chiefdoms · All PHU’S must have access to the Birth & Death register booklets. · Obtain from the Chiefdom Birth & Death Registrar DHS Birth & Death Registrar On￾going Distributed to almost all PHUs by Birth and Death Officer N/A 17 Problem/ Cause Recommendation May 2005 By Whom When? MIDTERM REVIEW OF ACTIONS March 2006 Priority Action for CARE for 2006-07 Guide 10: Staff need for supplies, equipment and infrastructure 1. Inadequate Essential Drugs available at PHU’s (See COPE Drug Checklist) · Request UNICEF supply additional Essential Drugs · DHMT works with COPE Committee and CARE staff using the COPE Drug Check List and DHMT list to keep better inventory DMO / NGOs / UNICEF Very Soon Quarterly supplies are being received by PHU staff when they come to District Hospital CARE to assist in the transportation of the essential drug supplies to PHUs as per DHMT plan 2. Insufficient IEC materials available at PHU’s Provide more IEC materials, esp. HIV/AIDS to PHU’s DHS, HIV Counselor Imme￾diately See above; posters have been provided. CARE to assist in the distribution of posters and other IEC materials 3. Inadequate equipment at PHU’s (COPE Equipment Checklist) Supply needed PHU equipment DMO Very Soon DHMT to do inventory of equipment at PHUs to assess gaps and use the information (gap analysis) to advocate for supply of additional equipment CARE to procure some of the missing basic equipments based on available funds 4. Break down of Cold Chain equipment including Solar Refrigerators Repair or replace broken equipment Request UNICEF repair Cold Chain refrigerators or Train PHU staff in minor maintenance and repair DOO UNICEF DOO UNICEF UNICEF is reviewing and replacing for improved maintenance. Currently the District Operations Officer is undergoing 6 months of training on maintenance of solar refrigerators n/a 5. Unavailability of Heavy/V Duty gloves for disposing of contaminated medical waste (District has no supply) Supply necessary heavy duty gloves to PHUs CARE Decembe r 2005 Action pending. CARE to supply to DHMT for distribution. 18 Problem/ Cause Recommendation May 2005 By Whom When? MIDTERM REVIEW OF ACTIONS March 2006 Priority Action for CARE for 2006-07 Lack of medical waste disposal facilities at PHUs Construct burning pits at all PHUs PHU-In Charge CHC/VDC May 2005 Medical waste disposal pits dug by community members at PHUs CARE to continue to strengthen the relationship between communities and PHU staff Lack of sufficient rigid “sharps” containers at PHU Provide sufficient rigid boxes DOO / PHU-In Charge May 2005 Rigid “sharps” containers present at PHUs, supplied by the EPI unit of MOHS CARE to work with the DHMT to ensure the availability of sharp disposal facilities at PHU s Lack of disinfectant at PHU’s (Bulk district supply has not arrived) Supply PHU’s with disinfectant DHS & Matron Ongoing Supplied as part of UNICEF’s supplies for PHUs CARE to facilitate the movement of supplies to PHUs. 19 Attachment G. Project Data Sheet Child Survival and Health Grants Program Project Summary Jul-21-2006 CARE (Sierra Leone) General Project Information: Cooperative Agreement Number: GHS-A-00-03-00013-00 Project Grant Cycle: 19 Project Dates: (9/30/2003 - 9/29/2008) Project Type: Standard CARE Headquarters Technical Backstop: Khrist Roy Field Program Manager: Boiketho Matshalaga Midterm Evaluator: Jean Capps Final Evaluator: USAID Mission Contact: Seydou Doumbia Field Program Manager Information: Name: Boiketho Matshalaga Address: 35 & 35A Wilkinson Road Freetown Phone: 232-22-234-227 Fax: 232-22-234-280 E-mail: ketho@sl.care.org Funding Information: USAID Funding:(US $): $1,488,582 PVO match:(US $) $520,725 Project Information: Description: The goal of the Project is to improve the health status of children under five and women of reproductive age. CARE in partnership with PHU will implement interventions in expanded program of immunization (EPI), nutrition, malaria and maternal and newborn care (MNC) through a grassroots, civil-society building variation of the overarching Community Integrated Management of Childhood Illness (C-IMCI) approach. CARE will also work with community health providers – Blue Flag Volunteers (BFVs), traditional birth attendants (TBAs), traditional practitioners, and drug peddlers –or through the HCs. The Project will collaborate with the Ministry of Health and Sanitation (MOHS) in district-wide activities to train PHU staff in IMCI, expand and improve services, and plan a Behavior Change Communication (BCC) campaign to improve family and community practices. The CARE capacity-building strategy will work through a partnership structure, with local organizations such as HCs, Talking Drum (TD) and Norway-Sierra Leone Health Project (NSL). Location: Northern Province, Koinadugu District,Sierra Leone Project Partners: General Strategies Planned: Strengthen Decentralized Health System M&E Assessment Strategies: KPC Survey Health Facility Assessment Organizational Capacity Assessment with Local Partners Lot Quality Assurance Sampling Community-based Monitoring Techniques Participatory Evaluation Techniques (for mid-term or final evaluation) Behavior Change & Communication (BCC) Strategies: Social Marketing Mass Media Interpersonal Communication Support Groups Groups targeted for Capacity Building: PVO Non-Govt Partners Other Private Sector Govt Community US HQ (CS unit) Field Office HQ CS Project Team PVOs (Int'l./US) Local NGO Traditional Healers National MOH Dist. Health System Health Facility Staff Health CBOs CHWs Interventions/Program Components: Immunizations (15 %) (IMCI Integration) (CHW Training) (HF Training) - Classic 6 Vaccines - Mobilization Nutrition (10 %) (IMCI Integration) (CHW Training) (HF Training) - Comp. Feed. from 6 mos. - Maternal Nutrition (IMCI Integration) Micronutrients (10 %) - Iron Folate in Pregnancy Malaria (35 %) (IMCI Integration) (CHW Training) (HF Training) - ITN (Bednets) Maternal & Newborn Care (30 %) (IMCI Integration) (CHW Training) (HF Training) - Integr. with Iron & Folate - Birth Plans Target Beneficiaries: Infants < 12 months: 4,517 Children 12-23 months: 3,877 Children 0-23 months: 8,394 Children 24-59 months: 9,921 Children 0-59 Months 18,315 Women 15-49 years: 51,491 Population of Target Area: 112,921 Rapid Catch Indicators: Indicator Numerator Denominator Percentage Confidence Interval Percentage of children age 0-23 months who are underweight (-2 SD from the median weight-for-age, according to the WHO/NCHS reference population) 0 0 0.0% 0.0 Percentage of children age 0-23 months who were born at least 24 months after the previous surviving child 0 0 0.0% 0.0 Percentage of children age 0-23 months whose births were attended by skilled health personnel 22 95 23.2% 8.5 Percentage of mothers of children age 0-23 months who received at least two tetanus toxoid injections before the birth of their youngest child 28 95 29.5% 9.2 Percentage of infants age 0-5 months who were exclusively breastfed in the last 24 hours 8 25 32.0% 18.3 Percentage of infants age 6-9 months receiving breastmilk and complementary foods 0 0 0.0% 0.0 Percentage of children age 12-23 months who are fully vaccinated (against the five vaccine-preventable diseases) before the first birthday 44 95 46.3% 10.0 Percentage of children age 12-23 months who received a measles vaccine 66 95 69.5% 9.3 Percentage of children age 0-23 months who slept under an insecticide-treated bednet the previous night (in malaria-risk areas only) 0 0 0.0% 0.0 Percentage of mothers who know at least two signs of childhood illness that indicate the need for treatment 77 95 81.1% 7.9 Percentage of sick children age 0-23 months who received increased fluids and continued feeding during an illness in the past two weeks 10 95 10.5% 6.2 Percentage of mothers of children age 0-23 months who cite at least two known ways of reducing the risk of HIV infection 59 95 62.1% 9.8 Percentage of mothers of children age 0-23 months who wash their hands with soap/ash before food preparation, before feeding children, after defecation, and after attending to a child who has defecated 0 0 0.0% 0.0 Comments for Rapid Catch Indicators One key error was found during analysis related to the way way the question was asked regarding the bednet indicator -- this was only asked for those children who have a febrile episode in the past two weeks. The sick child indicator referred to children who had had diarrhea in the past two weeks. The possible answers to the HIV/AIDS question were modified slightly between Baseline and KPC. The most accurate methods will be used for the next LQAS and for Final Evaluation (a mini LQAS done only in an ITN distribution area had already corrected this question and skip pattern.) 20 Attachment H. Post-Midterm Evaluation Review of DIP Workplan, for incorporation into post-MTE Action Plan (comments in parenthesis) 4b. Project Workplan Objective 1: Strengthened family and household knowledge and decision-making skills related to health of women and children resulting in the practice of positive behaviors to improve maternal and child health and prevent, recognize and manage common diseases. Year 1 Year 2 Year 3 Year 4 Year 5 Personnel Major Activities Activity Focus* 1 2 3 4 1 2 3 4 1 2 3 4 1 2 3 4 1 2 3 4 Who at CARE Who (Other) Household 1. Formative research on feeding practices and food availability, health beliefs, social norms (including positive deviance behavior), perception of and management of disease, care-seeking practices, profile of community-based health providers. (Completed as scheduled) BC xx x M&E, APM, CHS, CHM, PM 2. KPC baseline study on key family health practices. (Completed as scheduled) BC x x All project staff DHMT Community 3. Project presentation to and initiation of dialogue within communities. (Completed as scheduled) BC x CHM, CHS PHU staff 4. Formation of health clubs (including pre-existing CBOs, as appropriate). (Completed as scheduled) BC x x x CHM, CHS PHU staff 5. Implementation of health promotion/education campaign through CHCs targeting HH knowledge, beliefs and practices. (Completed as scheduled) BC x x xx x xx x xx x xx x xx x x x CHM, CHS DHMT, Community Health Facility 6. Develop a productive interface between community surveillance systems developed through the CHCs and DNO/PHU that results in problem identification and response. Initiated and progress continuing BC xx x M&E, APM, HEO DHMT District 7. Development of BCC strategy and materials for communication. (Completed as scheduled, on-going additions) BC xx x PM, APM, HEO DHMT 8. Conduct training needs assessment (TNA) for community-based organizations, local partner organizations and MOHS and CARE staff. (Completed as scheduled for NGO -- will revisit as part of MTE Action Plan; new CARE SL Health Sector Coordinator and CSP PM will discuss with new DMO how CARE can add value to training plans for MOHS. CARE HQ and SL continue to seek out training opportunities for CSP staff.) Q and A x x HSC, PM, HEO DHMT 9. Training of PHU staff, and health club members to implement BCC strategy to HH members of community resulting in decrease in harmful practices; increase in practice of beneficial preventive practices; improved recognition of danger signs; and increase in appropriate care￾seeking behavior. (Completed as scheduled, PHU staff trained along with CHC members). Q and BC xx x xx x x CHS, CHM, APM, HSC, PM DHMT 21 Objective 2: Improved quality and accessibility of services provided by MOHS personnel and MOHS extension services Health Facility 10. Promote dialogue between communities and DHO/PHUs. (Completed as scheduled) Q x x xx x xx x xx x xx x x CHM, CHS, APM, PM PHU staff, DMO, WC 11. Conduct Quality of Care assessment with MOHS and UNICEF. (COPE completed with MOHS and UNICEF participation; pending incorporation of pre/post-test of clinical skills as part of training activities.) Q x x x HSC, PM PHU staff, DMO, WC 12. Facilitate training and implementation in supervision, training of trainers, and quality assurance for DHO/PHU staff. (TOT completed as scheduled; supportive supervision and QA are pending new DMO plans.) Q x x xx HSC, PM PHU staff, DHMT 13. Facilitate initial and refresher IMCI training. (Pending MOHS national plan; CARE Health Sector Coordinator and CSP PM will seek opportunities for CARE to facilitate.) Q x x x PM, APM DHMT 14. Conduct quarterly quality assurance (QA) workshops for PHU staff. (Strengthen COPE follow-up and review; CARE SL Health Sector Coordinator and CSP PM investigate upcoming training plans for DHMT and determine a detailed plan to add value.) Q x xx PM, APM DHMT 15. Build nutritional counseling and services for pregnant women into outreach services provided by PHUs and CHCs (supervised by PHUs). (To be included as part of MNC strategy, to be detailed before end of FY2006.) Q x x HEO DHMT District 16. Work with DHO to identify and prioritize problems in district health services (including HMIS) and design and implement solutions that are based on qualitative and quantitative data. (On-going) Q xx x xx x xx x xx x x PM, APM, M&E DMO, DHS, M&E, PHU staff 17. Assess organizational development and physical needs of MOHS in Koinadugu and support capacity-building activities. (Broadly worded; some activities done, many needs remain for org development -- action is incorporated into on-going governance activities with partners). Q xx HSC, PM, APM M&E, DMO, DHS 18. Conduct training needs assessment (TNA) for community-based organizations, local partner organizations and MOHS staff. (Completed as scheduled for NGO -- will revisit as part of MTE Action Plan; new CARE SL Health Sector Coordinator and CSP PM will discuss with new DMO how CARE can add value to training plans for MOHS. CARE HQ and SL continue to seek out training opportunities for CSP staff.) Q xx x x x HSC, PM, APM, HEO DHMT 19. Adapt/develop curricula for training of community health workers in collaboration with MOHS. (Completed as scheduled for broader group of CHCs and for CBGP) BC x x PM, APM, HEO DHMT 20. Development and implementation of Social Marketing Plan for the promotion of purchase and use of ITNs. (One of the strategies of the new MOSI project which is just in the process of initiating; coordination and collaboration between CSP and MOSI will be defined once new CARE SL Health Sector Coordinator begins). A x x All Staff DHMT 21. Work with partners (UNICEF, MOHS) to develop strategy/plan for ensuring supply of essential drugs in PHUs to support IMCI. (MOHS has its own plan at district level and is dependent upon, new, DMO decisions; CARE will continue to work on governance and BCC activities that strengthen transparency and accountability.) A xx x x HSC, PM, APM DHMT 22. Conduct ongoing monitoring of CS Project results in collaboration with DHO/PHU colleagues and feed results back into project including the use of LQAS. (On-going; CARE will seek to involve more DHO/PHU staff in LQAS activities to build additional skills). Q xx x xx x xx x xx x xx x xx x xx PM, M&E, APM, CHS, CHM, HEO PHU staff, M&E, DHMT 22 Objective 3: Enhanced community capacity to form groups and institutions that sustain health initiatives, demonstrate social cohesion, and promote good governance mechanisms Community 23. Formative research on attitudes and beliefs of community members regarding rights, civil society, community action to identify and act on communal problems, community support for GOSL health services, etc. (Completed as scheduled) BC xx x xx x CHS, CHM, APM, PM, M&E, HEO PHU staff 24. Present project to communities and begin a dialogue with all community members regarding community action, governance, health rights, etc. (Completed as scheduled) BC x x xx CHS, CHM, APM, HEO 25. Formation of health clubs (CHCs, including pre-existing CBOs, as appropriate). (Completed as scheduled) BC x x xx CHS, CHM, APM DHMT 26. Facilitate community dialogue regarding health volunteers and their roles and responsibilities both to community and to PHU. (On-going.) A x x CHS, CHM, APM 27. Assist well-run CHCs to identify needs of their organization and provide TA to build their capacity. (On-going). BC x x xx x xx x xx x xx CHS, CHM, APM, PM, HEO DHMT 28. Facilitate the formation and training of Community based growth promotion groups emerging from CHCs. (Training completed; CBGP to be established in Year 3 immediately post MTE.) A x xx x CHS, CHM, APM, PM, HEO DHMT 29. Collaborate with communities in a participative evaluation of their own efforts. (Part of CHC process but will be considered as a special separate activity for sustainability efforts in Year 5 and/or as part of Final Evaluation.) BC x xx x xx x xx x xx x xx x CHS, CHM, APM, M&E DHMT 30. Assist communities and CHCs to access multi-sectoral development opportunities to improve their communities. (Completed as scheduled -- some independent and sustainable progress.) A xx x xx x xx x xx x xx CHS, APM, PM, HSC PHU staff, DHMT District 31. Advocate with GOSL at national and district level to develop policies/processes that support responsibly managed community-level initiatives in remote areas where GOSL does not provide adequate services (e.g., sell GOSL-provided ITNs or essential drugs). (Will be a priority for new CARE SL Health Sector Coordinator, in coordination with CSP staff and partners.) A x xx x xx x xx x xx PM, APM, HSC DHMT 23 Sustainability of the CS project. Community 32. Facilitate cross visits for DHO/community leaders from Koinadugu, Kono and Kailahun districts. (Included in Action Plan post MTE.) BC x x xx x xx x xx x x PM, APM DHMT 33. Community, District and national level monthly, quarterly and annual collaboration on issues pertaining to CS, Malaria, Nutrition, Maternal and Newborn Care (MNC) and EPI. (On-going; community role to be continuously strengthened.) BC x xx x xx x xx x xx x xx x xx x x PM, APM, CHS, CHM, HEO PHU staff, VDC members, DHMT District 34. Provide TA for organizational development and capacity building. . (New CARE SL Health Sector Coordinator and CSP PM will discuss with new DMO how CARE can add value to training plans for MOHS; some training has been provided for VDCs, but additional trainings will be considered for all local government levels as part of CARE SL Koinadugu sub-office integrated programming for governance.) Q x xx x xx x xx Consultants, HSC, PM, APM DHMT, PHU staff 35. Create opportunities for partner organizations to develop inter￾organizational links, access to information and assistance, and accountability. (To be revisited for local NGO as part of post-MTE Action Plan; cross-visits for DHMT part of post-MTE Action Plan). Q xx x xx x xx x xx x xx x x PM, APM, HSC 36. Advocate for opportunities for partners to achieve or work towards financial viability. See Numbers 20, 21, 30, 31, 32 and 35. Q x xx x xx x xx x xx x xx x x HSC, PM DHMT 37. Support the planning process for the re-establishment of the District Medical Store. This was completed as scheduled but then torn down due to national MOHS construction plans that were not shared with DHO/CARE. A x x HSC, PM, APM DHMT * the activity addresses A=Access, BC=Behaviour Change, or Q=Quality 24 Attachment I. Post MTE -- Action Plan Priorities for FY2006 March 30, 2006 Intervention APR MAY JUNE JULY AUG SEP CSP (general) New CARE Sierra Leone Health Sector Coordinator and CSP PM investigate intervention-specific clinical care training plans for DHMT and negotiate a way for CSP to "add value" (pre/post-tests, facilitate national consultant trainer approved by MOHS, refresher training follow-up; logistic support) Nutrition Establish CB-GM in 54 villages Strengthen quality of CB-GM in 54 villages CB-GM in 54 villages ENA session at CARE Annual CS workshop Conduct joint assessment (DHMT/CA RE) of PHU skills in Growth Monitoring Design ENA training plans for Nov-Dec. MNC Meeting between DHMT, UNICEF, District Council and CARE re: VHF Radios maintenance & sustainability Discuss with NGOs active in MNC in Koinadugu complementary actions in MNC to improve maternal and newborn health Investigate and share key points of new MOHS MNC policy T.A. with CARE USA assistance: Develop MNC Strategy and Work Plan with timetable Malaria Consult IRC on tools for SCM assessment of PHU malaria treatment skill Facilitate refresher training in malaria treatment and dosage new drug protocol (ACT) with PHUs with post￾test assessment of skills HMIS Continue strengthening community surveillance continue continue continue continue continue BCC Mass Media Strategy Meet with Search for Common Ground: TA for design of radio-drama; Listener Survey Investigate other sources of Listener Survey tools (questionnaire, survey methodology) (Potential) random Listener Survey Update BCC Strategy & implementat ion plan 25 Intervention APR MAY JUNE JULY AUG SEP Partnership with other NGOs CSP PM meets with CES to hear their priorities from ISA -- consult with CARE SL on what other CARE projects can offer Develop matrix of active NGOs in health in District with details on MNC and malaria strategies for social marketing ITN Contact IRC after MTE and make plan for activity (before end of September) to share CSSA advances Strengthen monthly meetings at DHMT level Further orientation on CSP to DMO, 1-2 topics/visit Strengthen monthly meetings at DHMT level Further orientation on CSP to DMO, 1- 2 topics/visit Strengthen monthly meetings at DHMT level Further orientation on CSP to DMO, 1- 2 topics/visit Investigate other CARE project techniques to contribute to strengthened governance: ENCISS? Le Wi Lan? Cross-visit to IRC (DHMT, DC, PHU, community, 2-3 CARE): · Cost recover · HMIS Governance, sustainability and key partners: DHMT, District Council, community representatives 26 Attachment J. CHCR Addendum Sheet Please review when using: "Communicating Health: Communicating Rights A Participatory Toolkit for Field Agents" Sierra Leone Page 56: The discussion of prevention measures for malaria should strongly emphasize the benefits of sleeping under an insecticide-treated net (ITN). Please make sure the discussion activity on Page 57 (ASK: Who in your household has an ITN? What happens to other members of the household who do not sleep under an ITN? etc) is a strong element of your community sessions on malaria prevention. Also, please provide information on the importance of and procedures for re-treatment of standard ITNs and about the additional benefits of Long-Lasting Insecticide treated Nets (LLIN). Along with the important message that "pregnant women qualify for a free ITN which they can get at the local health centre", community activities should also communicate the important message that pregnant women receive appropriate ante-natal care for Intermittent Preventive Treatment for malaria during pregnancy. Although environmental clean-up measures (such as covering water containers or cleaning up garbage) are useful for multiple health reasons and may reduce the breeding ground for mosquitoes somewhat, the benefits from this strategy -- as compared to use of ITNs and appropriate health care seeking -- is much less and should not be presented as an equal strategy in importance. Page 61: Please reverse the order of demonstration for preparation of oral rehydration, emphasizing that mixing ORS packet and safe water is the most highly recommended approach, with alternatives to ORS such as coconut water, rice water, or other available liquid mash or gruel as the second choice. Please note that recent technical guidelines suggest that the use of homemade sugar-salt solution (SSS) is not recommended because the correct preparation of SSS at the household level has been found to be difficult, sometimes resulting in dangerously high concentrations of salt. Page 84: Please emphasize to community members that among the signs of pneumonia listed on this page, "fast, shallow breathing (more than 60 breaths a minute" and "lower part of chest goes in as child breathes" are the key signs for pneumonia (the other signs may be signs of various illnesses). It is also important that children with fast breathing and fever also be treated for malaria, which may also be causing fever. Page 160: In the list of Key Messages for Family Planning, please make sure that all participants are clear that prevention of STIs/HIV is an advantage of ONLY the condom or abstinence methods of family planning.