Grand Cape Mount Child Survival Project Improved Child Health in a Transitional State through IMCI Grand Cape Mount County, Liberia October 2006 – September 2010 MIDTERM EVALUATION October 2008 Child Survival and Health Grants Program (CSHGP) Cooperative Agreement No. GHS-A-00-06-00019 Authors Joan M. Jennings, MPH, consultant; assisted by Bill Massaquoi, Country Director, MTI Liberia; Debbie Doty, West Africa Program Manager, MTI HQ; Todd Nitkin, Monitoring and Evaluation Specialist, MTI HQ; Janis Lindsteadt, Director of Technical Services, MTI HQ; and Mary Helen Carruth, Child Survival Advisor, MTI HQ Contact Person at MTI HQ Mary Helen Carruth, Child Survival Advisor. Medical Teams International, 14150 SW Milton Ct. Tigard, OR 97224 Mailing address: P.O. Box 10, Portland OR 97207, mhcarruth@medicalteams.org, 503-624-1000 Table of Contents A. ACRONYM LIST.......................................................................................................................................1 B EXECUTIVE SUMMARY ........................................................................................................................2 C. ASSESSMENT OF RESULTS AND IMPACT OF THE PROJECT 1. RESULTS: TECHNICAL APPROACH A. Overview of the Project .............................................................................................................................5 B. Progress Report by Intervention Area ......................................................................................................18 C. New or Innovative Tools And Approaches..............................................................................................25 2. RESULTS: CROSS-CUTTING APPROACHES a. Community Mobilization......................................................................................................................26 b. Communication For Behavior Change...................................................................................................28 c. Capacity Building Approach.................................................................................................................29 d. Health Systems Strengthening ...............................................................................................................30 e. Policy and Advocacy ............................................................................................................................32 f. Contribution to Scale ............................................................................................................................32 g. Equity .................................................................................................................................................32 h. Sustainability Strategy ..........................................................................................................................33 D. CHANGES IN ORGANIZATION CAPACITY ..............................................................................34 E. MISSION COLLABORATION ............................................................................................................35 F. CONTEXTUAL FACTORS THAT HAVE INFLUENCED PROGRESS TO DATE .............35 G. CONCLUSIONS AND RECOMMENDATIONS............................................................................35 H. ACTION PLAN ........................................................................................................................................37 I. ANNEXES ANNEX 1: RESULTS HIGHLIGHT ANNEX 2: PROJECT MANAGEMENT EVALUATION ANNEX 3: PROJECT M AND E TABLE ANNEX 4: RAPID CATCH TABLE ANNEX 5: MIDTERM LQAS REPORT ANNEX 6: MIDTERM RAPID HEALTH FACILITY ASSESSMENT REPORT ANNEX 7: ORGANIZATIONAL CAPACITY SURVEYS FOR CHAL AND GRAND CAPE MOUNT COUNTY HEALTH TEAM ANNEX 8: MTI, CHAL AND GCM CHT TECHNICAL ASSISTANCE PLANS FOR INSTITUTIONAL CAPACITY BUILDING ANNEX 9: EVALUATION TEAM MEMBERS AND THEIR TITLES ANNEX 10: EVALUATION ASSESSMENT METHODOLOGY ANNEX 11: LIST OF PERSON INTERVIEWED AND CONTACTED ANNEX 12: PROJECT DATA SHEET Grand Cape Mount Child Survival Project Midterm Evaluation, Medical Teams International Liberia 1 A. ACRONYM LIST ACT Artemisinin-Combination Therapy AHA Africa Humanitarian Action CERF Central Emergency Response Fund CHAL Christian Health Association of Liberia CHC Community Health Committees CHDC Community Health and Development Committees CHP Community Health Promoters CHT County Health Team C-IMCI Community-based Integrated Management of Childhood Illness CM Certified Midwife CS Child Survival CSP Child Survival Project CSSA Child Survival Sustainability Assessment CSTS+ Child Survival Technical Support Plus Project DC District of Columbia DHS Demographic and Health Survey FY Fiscal Year GCMC Grand Cape Mount County GIK Gift In Kind HF Health Facility HHP Household Health Promoters IMC International Medical Corps IMCI Integrated Management of Childhood Illness IPTp Intermittent Presumptive Treatment (of malaria) during pregnancy IR Intermediate Result ITN Insecticide Treated Net IYCF Infant and Young Child Feeding KPC Knowledge Practice Coverage LLIN Long Lasting Insecticide-treated Net LQAS Lot Quality Assurance Sampling M&E Monitoring & Evaluation MTE Mid-Term Evaluation MTI Medical Teams International (formerly known as Northwest Medical Teams) PA Physician’s Assistant PMI President’s Malaria Initiative QIVC Quality Improvement Verification Checklist R-HFSA Rapid Health Facility Services Assessment SIA Supplemental Immunization Activity SO Strategic Objective SP Sulphadoxine-Pyrimethamine TBA Traditional Birth Attendant TT Tetanus Toxoid U5 Under Five WRA Women of Reproductive Age Grand Cape Mount Child Survival Project Midterm Evaluation, Medical Teams International Liberia 2 B. EXECUTIVE SUMMARY B1. DESCRIPTION: Medical Teams International has been successfully operating this Entry/New Partners four-year Child Survival Project in Grand Cape Mount County of Liberia since project start in October 2006. Project Goal is “to reduce morbidity and mortality of children under five and improve the health of women of reproductive age within Grand Cape Mount County in Liberia.” S.O. is “improved health outcomes through appropriate household practices & use of quality health services within a supportive sustainable environment by 2011.” The project directly benefits 18,022 children U5 and 15,421 WRA; capacity-building indirectly benefits all 129,055 in the county. o Intermediate Result 1: Communities assume responsibility for their own health with strengthened community structures, linkages with Health Facility staff, and enhanced human resource capacity. o Intermediate Result 2: Improved health behaviors and actions at the household level. o Intermediate Result 3: Improved quality of care in health facilities through the implementation of IMCI and capacity building in complementary areas. o Intermediate Result 4: Strengthened institutional capacity of MTI and partners to implement effective CS activities. Strategy and Approaches: In a post-conflict transitional environment, this CSP is using a two￾pronged approach to directly jump-start improvements to the health status of WRA and children U5: (a) In coordination with the Grand Cape Mount County Health Team, MTI staff contribute to the revitalization and roll-out of IMCI training for health facility staff in GCMC and provide follow￾up supervision and mentoring, and (b) MTI staff assist and supervise the local partner, Christian Health Association of Liberia, to cost-effectively provide intensive community mobilization and health education for behavior change using the Care Group model of Household Health Promoters (all female.) Community mobilization is based on the C-IMCI framework, with no community￾based treatment activities. HHPs are linked to local health facilities through a community referral system when health services are needed for U5s or WRA, to Community Health Committees, and (in the future) to Community Health and Development Committees at the district/health facility level. Institutional capacity building is a strong element in this grant, with actions that have strengthened MTI headquarters (HQ), MTI Liberia and partners (CHAL and GCM County Health Team) capacity in child survival. Since receiving this grant, MTI headquarters (HQ) has hired three new positions to support activities: a Capacity Building Advisor and a Child Survival Advisor, both based at HQ in Portland, Oregon and a Monitoring and Evaluation Specialist based in Washington, DC to facilitate collaboration with CSHGP and the CORE Group. B2. MAIN ACCOMPLISHMENTS AND PROGRESS: The project is essentially on track with the work plan and no major obstacles are foreseen in the future. The Project Manager recently resigned in August 2008 to take a position as Director of the MOHSW Family Health Department; however, project staff appear to have the skills to continue project implementation efficiently while a new manager is recruited. The project receives frequent and well designed backstop support from MTI HQ staff, and the project has a very strong monitoring and evaluation system in place, with a structured supportive supervision process, annual LQAS and use of the CSTS+ R-HFSA tool. Grand Cape Mount Child Survival Project Midterm Evaluation, Medical Teams International Liberia 3 The project has trained a network of 520 Household Health Promoters who are active in conducting home visits, referrals to health facilities for preventive or curative services for children U5 or WRA, and periodic group sessions with other persons of influence in 132 rural communities. The project has met or exceeded targets for IR1, with the exception of delayed training of communities in using HIS for decision-making, which will be done in Year 3. For IR2, midterm LQAS found the project to have met or exceeded midterm targets for 11 of 14 indicators (78%); this includes a Rapid Catch indicator, child weight-for-age, added to the M&E matrix. Indicators which met or exceeded midterm targets were found in each of the project intervention areas: immunization, recommended breastfeeding practices, diarrhea prevention and management, care-seeking for pneumonia, and malaria. Of special interest is a reduction in child undernutrition, from 27.1% to 20.8% as measured by weight-for-age. This likely represents a cumulative effect from multiple improvements in health and nutrition practices as targets for early initiation of breastfeeding, complete immunization, appropriate hand washing practices and care-seeking for pneumonia and malaria have been exceeded. As a result of collaboration with the President’s Malaria Initiative, more than two-thirds of children under age two are sleeping under long-lasting insecticide treated bednets (LLIN). The project has facilitated IMCI training for 42 health professionals (along with 6 MTI staff and partners, CHAL and County Health Team) and has conducted intensive follow-up and mentoring visits with those trained, of whom 36 are still present. Key findings from MTE R-HFSA include: improvement in supervision visits within 3 months of survey from a low of less than 5% at baseline to more than half (56%) at midterm; improvement in HF clinical assessment skills from 9% at baseline to 28% of health staff at midterm were completing ALL required assessment tasks for the sick child while on average, 66% of all tasks were completed; improvement in correct diagnosis and treatment: from 46% at baseline to 89% at midterm. MTI Liberia staff and partners, CHAL and County Health Team, have benefited from multiple capacity-building activities conducted by MTI HQ and regional staff, including several visits for financial and administrative management strengthening and workshops on the Care Group model, community mobilization, use of the BEHAVE model to develop a behavior change strategy, and project cycle management. In addition, CHAL and CHT partners have participated in KPC survey, LQAS and R-HFSA, along with this MTE. B3. AREAS IN NEED OF ATTENTION: The key need of the project at present is to find a new and skilled Project Manager. Project staff are able to carry out all project activities with the skills they have and with continued MTI HQ backstop support, but a Project Manager is needed for key areas such as advocacy and dissemination of the project’s strategies and accomplishments. MTI Liberia will shortly be receiving a new staff member as head of administration and finance, and this should provide additional support to the CSP. B4. CONCLUSIONS AND RECOMMENDATIONS: The project is on-track overall with activities and has had significant success to-date as seen on the Project Monitoring and Evaluation Matrix (Annex 3). The project strategy to jump-start improvements in health through sub-grant to a local NGO for community mobilization using the Care Group model, with MTI staff providing support to partners and intensive focus on IMCI mentoring of health facility staff, is considered particularly effective for a post-conflict environment. MTI has invested sizeable effort in providing HQ support and capacity-building activities for project staff and partners, and the benefits of this Grand Cape Mount Child Survival Project Midterm Evaluation, Medical Teams International Liberia 4 investment are visible. Project design has drawn upon effective tools available through the USAID￾supported child survival community, including use of the BEHAVE framework for developing a behavior change strategy and use of the Child Survival Sustainability Assessment tool to define and promote sustainability of project efforts. The project has a very strong monitoring and evaluation component, with tools for quality supportive supervision obtained from other CSP NGOs, and with the MTI HQ M&E Specialist providing support for capacity-building of staff and partners through implementation of the KPC survey, annual LQAS and R-HFSA. Key recommendations center on: 1. Refresher training for HHPs and stronger emphasis in community education for child caregivers on a few technical messages: Essential Nutrition Actions for complementary feeding, benefits of colostrum, importance of TT and IPTp for basic maternal care, and more refined recognition of when home care can be provided and when health facility services are needed. 2. Promote better understanding of the rational use of drugs, at all levels, including additional training for health facility on counseling caregivers in drug usage, training and increased follow￾up of referrals by HHPs and Year 3 work plan activities to engage “black baggers” (non￾accredited drug salespeople). 3. A new strategy to reach and improve the skills of health facility support staff not trained in IMCI but who provide service when IMCI trained staff are not present. 4. Building capacity of health facility-linked Community Health and Development Committees and links with Community Health Committees and Household Health Promoters. 5. Update project population figures when official census results are released as planned for April 2009. 6. Actively seeking ways for increased visibility of the project and accomplishments at national level and for promoting sharing and scale-up of project strategies. B5. Summary of Impact Model Elements Inputs Activities Outputs Outcome Goal o BCC materials o Training plans o Monitoring and supervision tools o Matching funds for essential drugs o Logistics support o IMCI training for health professionals. o C-IMCI / BCC training for Household Health Promoters. o Capacity-building workshops for partners, CHAL and CHT. o Strengthening community participation as stakeholders. o At least 60 IMCI trained health professionals in GCMC health facilities. o At least 520 HHPs actively improving health in 132 rural communities. o Partners with improved capacities for administrative functions and child health programming implementation. See Project M&E Matrix in Annex 3. To reduce morbidity and mortality of children under five and improve the health of women of reproductive age within Grand Cape Mount County in Liberia. Grand Cape Mount Child Survival Project Midterm Evaluation, Medical Teams International Liberia 5 C. ASSESSMENT OF RESULTS AND IMPACT OF THE PROJECT Medical Teams International has been successfully operating this Entry/New Partners four-year Child Survival Project in Grand Cape Mount County of Liberia since project start in October 2006. There have been no substantial change in activities from the Detailed Implementation Plan, other than choosing not to use the COPE (Client Oriented Provider Efficient) assessment tool due to the comprehensiveness of the CSTS Rapid-Health Facility Services Assessment tool which has been used for baseline and midterm, and will be used for final evaluation of project efforts to strengthen the quality of health service care. MTI CSP has also been using LQAS methodology annually for assessing improvements in mothers’ knowledge, along with an internal Quality Improvement Verification Checklist system1 for supportive supervision and continuous improvement of activities. Population estimates in post-conflict areas can be very unreliable. As reported in the Annual Report for Year 1, at start-up the project staff collected population data in the communities where Care Groups were planned and found a difference from population estimates of children under age five as reported in the DIP – 18,022 vs. 21,611. Estimates used in the DIP were projections based on a 1984 census. Population data gathered by staff showed a greater difference in the number of women of reproductive age, as compared to estimates included in the DIP – 15,421 vs. 29,239. A national census was conducted in 2008. Preliminary results have been released and show a gender disparity in Grand Cape Mount County, with a sex ratio of 107.7 men to 100 women. Preliminary results do not give age breakdowns, and total population found for Grand Cape Mount County exceeds that estimated in the DIP, at 129,055 vs. 127,124. This fits in with staff perceptions that a small number of people are continuing to return after years of displacement during the conflict. It is recommended that the project obtain the national 2008 census data with age breakdowns for GCMC as soon as available (planned for April 2009). In the meantime, field staff should continue to update household census figures periodically and seek to identify all households with local leaders. C1. RESULTS: TECHNICAL APPROACH C1a. OVERVIEW OF THE PROJECT Project Goal: To reduce morbidity and mortality of children under five and improve the health of women of reproductive age within Grand Cape Mount County in Liberia. The Demographic Health Survey (DHS) 2007 found infant mortality in the five years previous to average 71 deaths per 1,000 live births and child mortality to average 110 deaths per 1,000 live births in Liberia.2 As a cumulative indicator of health and nutrition, the CSP KPC baseline survey found 27% (C.I. 22.1%-32.6%) of children age 0 to 23 months to be underweight (<-2SD weight-for-age WHO reference population). At midterm, this indicator reflects the positive gains the project has made in all intervention areas, with a reduction to 20.8% (C.I. 12.7%-28.9%) of children age 0 to 23 months found to be underweight. 1 These supportive supervision tools were adapted from several toolkits developed by other NGOs, including “Quality Improvement Checklist: Monitoring Manager’s Tool” by Food for the Hungry, “Learning Session Observation Checklist” by Freedom from Hunger, and the WHO Supervisory Checklist for the Supervision of IMCI Activities. 2 About 35% of child mortality is due to neonatal and post-neonatal causes; the project does not have a specific maternal-newborn health intervention but seeks to indirectly affect post and neonatal mortality through improvements in maternal nutrition and promotion of use of available maternal care services. Grand Cape Mount Child Survival Project Midterm Evaluation, Medical Teams International Liberia 6 Strategic Objective: Improved health outcomes through appropriate household practices and use of quality health services within a supportive sustainable environment by 2011. Intermediate Result 1: Communities assume responsibility for their own health with strengthened community structures, linkages with Health Facility staff, and enhanced human resource capacity. Intermediate Result 2: Improved health behaviors and actions at the household level. Intermediate Result 3: Improved quality of care in health facilities through the implementation of IMCI and capacity building in complementary areas. Intermediate Result 4: Strengthened institutional capacity of MTI and partners to implement effective CS activities. Project Location and Beneficiaries: MTI CSP in Liberia assists at least 15,421 women of reproductive age (WRA) and 18,022 children under five (U5) in Grand Cape Mount County (GCMC) through improvements in health care through the County Health Team and promotion of behavior change directly in households in 132 communities located in all 4 districts of Grand Cape Mount County (Garwula, Tewor, Gola Konneh and Porkpa) and one Commonwealth, Tombey Chiefdom in Tollah Township. The project reaches most rural communities in GCMC, as planned in the DIP, with some smaller communities in the northeast of the county not included due to difficult access. Target population figures are based on population data collected by the project. A national census was done in 2008 and the project awaits official results for GCMC. MTI CSP field staff report increases in village population since the project’s inception, with displaced persons continuing to return as peace has been sustained since signing of the Accra Comprehensive Peace Agreement in August 2003. There are estimates that up to 80% of the rural population was displaced as some time during the conflict although almost all had left official IDP camps by the end of 2006. 3 The project builds the capacity of the County Health Team and health staff in 29 Health Clinics and 1 Health Center (in Sinje, more strategically located than the hospital at the county seat in Robertsport), improving the quality of care available for the county population of 129,055.4 Through relief funds from CERF/WHO and the Marie Lamfrom Foundation, MTI assisted in the rehabilitation of 5 Health Clinics, mobilizing community labor and providing materials for reconstruction, basic furniture and equipment. MTI is providing private donation funds to assist the CHT in upgrading Lofa Bridge Health Clinic to Health Center status at present, so that another Health Center will be a strategically located referral facility in the county. Interventions: Immunization 10%, Control of Diarrheal Disease 20%, Pneumonia Case Management 20%, Nutrition 30% & Malaria 20%. 3 “(draft) National Nutrition Policy” as quoted in the “(draft) Child Survival Strategy”, Government of the Republic of Liberia, Ministry of Health and Social Welfare; Monrovia, 2008. 4 “2008 National Population and Housing Census: Preliminary Results”, Government of the Republic of Liberia, Liberia Institute of Statistics and Geo-Information Services (LISGIS); Monrovia, June 2008. Grand Cape Mount Child Survival Project Midterm Evaluation, Medical Teams International Liberia 7 Strategy and Approaches: In a post-conflict transitional environment, this CSP is using a two￾pronged approach to directly jump-start improvements to the health of WRA and children U5: (a) MTI staff contribute to the revitalization and roll-out of IMCI training for health facility staff in GCMC and provide follow-up supervision and mentoring, and (b) MTI staff assist and supervise the local partner, Christian Health Association of Liberia, to cost-effectively provide intensive community mobilization and health education for behavior change using the Care Group model5 of Household Health Promoters (all HHPs selected are female; more detail provided in section C4a. Communication for Behavior Change). Community mobilization is based on the C-IMCI framework. Because the MOHSW does not presently have a policy regarding community-based treatment, project C-IMCI activities do not support provision of essential drugs at the community level. A network of volunteer Household Health Promoters (HHPs) are linked to local health facilities through a community referral system when health services are needed for U5s or WRA. The National Health Plan6 calls for stakeholder coordination through Community Health and Development Committees at the district level and Community Health Committees at the community level. The CSP has assisted community leaders and HHPs to form Community Health Committees (CHC) which support the HHPs in their work and have plans for in-kind activities to contribute to sustainability of HHPs in their outreach role. CHCs also coordinate to support emergency transport when a referral to a higher-level health facility is urgently needed. In line with MOHSW policy, the GCM County Health Team has established Community Health and Development Committees (CHDC) linked with the 16 health facilities that are able to provide the MOHSW Basic Package of Health Services. In the remaining two years of this project, the CHCs and HHPs will be linked with these CHDCs for collaboration in problem solving and to promote sustainability of community improvements. Institutional capacity building is a strong element in this grant, with actions that have strengthened the capacity of MTI headquarters (HQ), MTI Liberia and partners (CHAL and GCM County Health Team) to implement child survival activities. Since receiving this grant, MTI headquarters (HQ) has hired three new positions to support activities: a Capacity Building Advisor and a Child Survival Advisor, both based at HQ in Portland, Oregon and a Monitoring and Evaluation Specialist who is based in Washington, DC to facilitate close collaboration with CSHGP and the CORE Group. Several staff from MTI headquarters (HQ) have traveled to Liberia multiple times in the past two years, to provide capacity-building training and support to MTI Liberia CSP staff, the sub-grantee CHAL and MOHSW partners. Another workshop in facilitative supervision, by staff from Engender Health using the tool “Facilitative Supervision for Quality Improvement Trainer’s Manual” from the Acquire Project/Engender health, is planned for Year 3. 5 “The Care Group Difference: A Guide to Mobilizing Community-Based Volunteer Health Educators”, World Relief with the CORE Group, 2004. 6 National Health Plan 2007-2011”, Government of the Republic of Liberia, Ministry of Health and Social Welfare; Monrovia, 2007. Grand Cape Mount Child Survival Project Midterm Evaluation, Medical Teams International Liberia 8 C1b. SUMMARY M&E TABLE Intermediate Result 1: Communities assume responsibility for their own health with strengthened community structures, linkages with Health Facility staff, and enhanced human resource capacity Indicator Method BL MTE Result Status MTE Final Target % of HHPs referring patients to clinic. **7 HHP Supervision 0% 100% Target exceeded. The referral system was initiated in the 2nd Qtr of FY2 and is in use by all 520 HHPs. Project CHPs tabulate monthly referrals; the project will look at ways to record referrals received by health facilities. 75% % of CHCs (changed from CHDCs as formation done by CHT without MTI participation) with one or more women participating on the committee. CHT information 0% ~75% Target exceeded. Super￾vision areas 1 and 2 report an average of 1 female among the 5 members on each CHC, area 3 reports an average of 2 females, while area 4 reports no female CHC members. 65% % of communities with an economic plan for emergency health needs. ** Community Profile 0% 13% 60% % of communities with an emergency transport plan. ** Community Profile 0% 16% On track to meet final target. CHCs have recently been re-activated. Among 4 CHCs visited during MTE, all had future plans for small income generation to support the cost of emergency transport, and most could cite at least one recent instance of assistance provided for emergency transport. 65% Qualitative MTE found a very positive attitude towards the recently initiated (June 2008) formal referral system by all involved, including health facility staff, HHPs and mothers. More information on the referral system can be found in section C4d. Health Systems Strengthening. Community Health Committees have been reactivated or formed in Year 2. Often the traditional leader, the Town Chief, is a member of the CHC or communicates regularly with CHC members. Religious leaders, such as Imams and pastors are also involved. A lesson learned by project field staff was the importance of greater advocacy with community leaders to support the activities of the HHPs. At first, families did not understand the importance of the HHP role nor have confidence in their abilities. Community leaders were recruited to explain the importance of the HHPs work and the training and support the project provides. This has resulted in greater moral support for the HHPs and community acceptance of their efforts. Strengthening the capacity of CHCs to 7 Indicators marked ** are also used by the project for the Child Survival Sustainability Assessment (CSSA) tool. More information is provided in section C4h.Sustainability. Grand Cape Mount Child Survival Project Midterm Evaluation, Medical Teams International Liberia 9 implement change in their communities and linking them with HHPs and with MOHSW Community Health and Development Committees is part of the Action Plan for the remaining two years of this project. More information can be found in section C4a.Community Mobilization. Intermediate Result 2: Improved health behaviors and actions at the household level. Indicator Baseline MTE Target MTE Result Status at MTE Final Target % of newborns who were put to the breast within one hour of delivery and did not receive pre-lacteal feeds. 33.7% (28.3-39.3) 42% 51.0% (41.0-61.0) Exceeded 50% New target: 60% % of infants 6-9 months receiving breast milk and complementary foods. 37.5% (22.7-54.2) 51% 40.6% (30.8-50.5) Not met 65% % of children 0-23 months who are underweight (<-2 SD weight-for-age WHO/NCHS reference standards). 27.1% (22.1-32.6) 22% 20.8% (12.7-28.9) Exceeded 17% % of children aged 12-23 months who are fully vaccinated (BCG, DPT3, OPV3, measles vaccines) by 12 months of age. 18.9% (12.1-27.5) 30% 39.5% (29.8-49.4) Exceeded 40% New target: 55% % of mothers with children age 0-23 months who were protected against Tetanus (at least 2 TT) before the birth of the youngest child. 61.3% (55.6-66.9) 71% 66.7% (57.2-76.1) Not met 80% % of children 0-23 months with diarrhea in the last two weeks who received ORS and/or recommended home fluids. 74.2% (63.8-82.9) 80% 79.2% (71.2-88.7) Met 85% % of mothers of children 0-23 months who live in households with soap or ash at the place for hand washing and that washed their hands with soap or ash at least 2 of the appropriate times in a 24 hour recall period. 19.0% (14.7-23.9) 30% 43.7% (33.8-53.7) Exceeded 40% New target: 70% % of children age 0-23 months with chest-related cough and fast/difficult breathing in the last two weeks who were taken to an appropriate health provider. 43.2% (35.3-51.4) 54% 83.3% (75.9-90.8) Exceeded 65% New target: 85% % of children 0-23 months who slept under an ITN the previous night. 17.7% (13.5-22.5) 27% 67.7% (58.4-77.1) Exceeded 35% New target: 70% % of children 0-23 months with a febrile episode ended during the last 2 weeks who were treated with an effective anti-malarial drug within 24 h. after fever began. 12.5%8 (5.9-17.7) 32% 49.06% (39.0-59.0) Exceeded 50% New target: 65% 8 Baseline value and targets were recalculated at MTE upon review of KPC criteria. Grand Cape Mount Child Survival Project Midterm Evaluation, Medical Teams International Liberia 10 Among the complete list of 14 KPC indicators in the project M&E Matrix for IR 2 (the Rapid Catch indicator, weight-for-age, has been added to the matrix), midterm LQAS found the project to have met or exceeded midterm targets for 11 of the 14 indicators (78%). These include indicators in each of the project intervention areas: immunization, recommended breastfeeding practices, diarrhea prevention and management, care-seeking for pneumonia, and malaria. Of special interest is the reduction in child undernutrition, as measured by weight-for-age. This likely represents a cumulative effect from multiple improvements in health and nutrition practices as targets for early initiation of breastfeeding, complete immunization, appropriate hand washing practices and care-seeking for pneumonia and malaria have been exceeded. Thanks to the President’s Malaria Initiative, more than two-thirds of children under age two are sleeping under long-lasting insecticide treated bednets (LLIN). New targets are proposed by project staff for those indicators close to or having reached final targets. A very high degree of message consistency based on the USAID Technical Reference Materials was encountered during MTE qualitative focus groups with HHPs and mothers. Besides planned refresher training, topics which will receive additional attention during the second half of the project include: recommended complementary feeding practices for children age 6 to 23 months, and elements of maternal care, including promotion of tetanus toxoid and IPTp. Intermediate Result 3: Improved quality of care in health facilities through the implementation of IMCI and capacity building in complementary areas. Indicator Method BL MTE Result Status MTE Final Target % of HFs clinical encounters in which all assessment tasks are made by the HW for sick child (check ability to drink or breastfeed, vomits everything, convulsions, presence of cough or fast/difficult breathing, diarrhea, fever, assess nutritional status, feeding practices, check vaccination status) R-HSPA BL & FE 9% 28% (66% of all tasks completed on average) Improved. Additional efforts beyond that planned are recommended as needed if final target is to be reached. 85% % of HF clinical encounters in which treatment is appropriate to diagnosis for malaria, pneumonia and diarrhea. (Record review) R-HSPA BL & FE 46% 89% Final target met. Continued IMCI mentoring will be needed to maintain as staff turnover occurs. 85% % of HF clinical encounters in which the caretaker whose child was prescribed antibiotic, anti-malarial or ORS can correctly describe how to administer all prescribed drugs R-HSPA BL & FE 49% 34% No improvement. A weak area of IMCI at MTE. Project will add activities to promote counseling of care￾givers on drug use with training for HF staff and HHPs follow-up. 75% % of HF that received external supervision at least once in the last THREE months (2 or more: deliver supplies, check records/ reports, observe work, provide feedback) R-HSPA BL & FE 5% 56% Improved. Indicator is on track to reach final target. 75% Grand Cape Mount Child Survival Project Midterm Evaluation, Medical Teams International Liberia 11 This project has facilitated the first reactivation of IMCI national training post-conflict, with only staff in Grand Cape Mount County trained at present. Per MTE interview with the head of the MOHSW Family Health Department (the former manager of this CS project) there are plans for further roll-out in 2009 with Bomi, Gbarpolu and Margibi counties adjacent to GCMG a priority. Quality IMCI care requires correct and complete assessment, diagnosis, and treatment of common childhood illnesses. With follow-up mentoring by project staff, health facility staff have improved skills, with an average of 66% of all assessment tasks performed during R-HFSA (although only 28% of cases seen were completely assessed). The tasks most often missed include: asking if the child has been vomiting or convulsing and/or checking nutritional and vaccination status on the child health card. Assessment results varied to a high degree between health facilities, with a few facilities accounting for most poor results. The benefits to child health expected from the excellent results found for appropriate diagnosis and treatment according to protocols (89% at MTE, compared to 46% at baseline) can easily be undermined by caregivers who do not administer drugs as prescribed or complete the full course of treatment. R-HFSA found only about one-third of caregivers able to accurately describe how to use the medications prescribed. This indicates that IMCI trained health facility staff are not providing appropriate counseling on use of prescribed medicines and that health facility support staff, such as pharmacy technicians, have not been trained in IMCI protocol for counseling. It is recommended that the project include additional activities to strengthen health facility capacity for rational drug use and counseling of caregivers, along with training of HHPs to follow up with caregivers after referral and reinforce correct and complete treatment. The MOHSW has an accepted system of using dispensing bags with pictorial descriptions of dosage and frequency. It is suggested that the project should support this system as much as possible during the remaining life of the project and use it as the basis for HHP training. Intermediate Result 4: Strengthened institutional capacity of MTI and partners to implement effective CS activities. Indicator Method BL MTE Result Status at MTE Final Target % of indicators for capacity building are achieved IA BL & FE 0% Action has been taken to improve approximately 36% of indicators for all three partner agencies: MTI Liberia, CHAL, and Grand Cape Mount County Health Team. Improved. All key systems are in place with finalization of an emergency prepared￾ness plan as a priority for MTI. Review and strengthening of logistics in Qtrs 1 and 2 of Y3 is a priority for all 3 partners. 80% Use of CSSA is institutionalized (annual reviews are being conducted) Annual Report No Review at midterm conducted. Improved. Midterm review by MTI and partners (CHAL and CHT) refined the CSSA plan as prepared during the DIP and established retroactive baseline values for indicators. Yes The project has assisted the sub-grantee to put into place all administrative and financial systems needed to ensure appropriate management of the sub-grant, with staff from MTI HQ visiting and Grand Cape Mount Child Survival Project Midterm Evaluation, Medical Teams International Liberia 12 assisting this process in Year 1. Problems encountered by MTI Liberia in the implementation as planned by project partners have been resolved with the replacement of the CHAL Community Outreach Coordinator and 3 of the 4 Supervisors, and through intervention by County Health Team leadership regarding the health facility use of motorcycles which were donated by MTI during a previous CERF grant. MTI HQ staff (the Child Survival Advisor and the M&E Specialist) provided assistance to MTI Liberia CSP staff and partner leadership (CHAL and CHT) to review and refine the Child Survival Sustainability Assessment framework developed during DIP, and to establish baseline values for future comparison at final evaluation. C1c. WORK PLAN ACTIVITY STATUS TABLE FY2007-2008 (Oct. 2006 to Sept. 2008) For IR 1 and 2, the project is essentially on track with the work plan at the midterm point. Some activities were delayed in Year 1 due to a need for personnel changes among the sub-grantee partner agency. The present staff have been in place since the start of calendar year 2008 and have been meeting all benchmarks. IR 1: Communities assume responsibility for their own health with strengthened community structures, linkages with Health Facility staff, and enhanced human resource capacity Activities Year 1 and 2 Benchmarks Benchmark Achieved Activity Status Develop curriculum for training CHPs and HHPs, including 12 month rotational topics C-IMCI curriculum developed Yes All modules have been covered and interest expressed in adding information on basic prevention HIV/AIDS and stigma reduction. Train CHPs in C-IMCI 20 CHPs trained in behavior change and community mobilization Yes 20 CHPs have been trained in all modules of the C-IMCI. Eighteen are still presently with the project Develop a referral system from HHPs to health facilities 520 HHPs making referrals to health facilities Yes The HHPs had been verbally referring sick children to health facilities until June 2008, when they were trained in using laminated pictorial referral cards provided by the project. Develop a system for Supportive Supervision and conduct joint visits (adapt checklist, train) QIVCs developed for use during supervision support visits to supervisors, CHPs and HHPs. Yes QIVCs are being used during training and supervision visits to provide feedback to supervisors, CHPs and HHPs Develop community HMIS HMIS functioning and data provided to CS Project, CHT and communities. Yes Community level data (Care Group and Community Profile data) is being collected and shared with the CSP, CHT, and communities. HF data is being collected and shared with the CHT Develop economic plans with communities for emergency health care 60% of communities have an economic plan for emergency health needs (at final). N/A 17/132 (13%) of CHCs have an economic plan for emergency health care Grand Cape Mount Child Survival Project Midterm Evaluation, Medical Teams International Liberia 13 IR 2: Improved health behaviors and actions at the household level Activities Year 1 & 2 Benchmarks Benchmark Achieved Activity Status Develop IEC/BCC strategy using the BEHAVE Framework BCC strategy developed Yes Developed during DIP process and revised in October 2007. Procure appropriate training materials, equipment and IEC/BCC educational materials. C-IMCI Trainers Guide, Facilitators Manual and flip charts developed. Yes C-IMCI Trainers Guide, Facilitators Manual and flip charts developed Develop strategy for using traditional communication channels and radio. Traditional ways of learning (song, story telling, drama, case studies) integrated into C-IMCI curriculum Yes The CHPs use songs, role plays, stories, demonstrations and case studies during training sessions with HHPs. HHPs utilize role plays, demonstrations and songs during their home visits and community education sessions. MTE consultant advised against developing radio spots as these can be costly or ineffective in remote rural areas. The project has not done any survey of radio access and use. Final review of any health message spots by MTI HQ and MOHSW for message content would also be advised and would be a lengthy process at this point. HHPs conduct education for families with children U5 during home visits; and through group sessions for women in the community to involve older women who are influential for behavior change. Yes HHPs carry out monthly home visits and education sessions in their communities. Older women participate in home visits. Message dissemination during Friday prayers by Muslim religious leaders and during Christian church services on Sundays. Yes Imams and pastors have been provided with training in C-IMCI health messages and disseminate health messages during Friday prayers and Sunday workshop service Implementation of IEC/BCC activities Counseling and health education provided at Yes Health facility staff trained in IMCI have received training in counseling skills Develop with communities emergency transport plan 65% of communities have an emergency transport plan (at final) N/A 21/132 (16%) of CHCs have emergency transport plan Grand Cape Mount Child Survival Project Midterm Evaluation, Medical Teams International Liberia 14 IR 2: Improved health behaviors and actions at the household level Activities Year 1 & 2 Benchmarks Benchmark Achieved Activity Status HFs through the module Counsel the Mother and are using the skills they acquired to provide health education and counseling to mothers and caregivers. Political and traditional leaders involved through CHDC activities. Yes Formal and traditional leaders (town chiefs) are advisors on most of the CHCs that have been formed in the project communities. Formation of Care Groups 160 Care Groups established (2000 HHPs) Yes A network of 132 Care Groups with 520 Household Health Promoters (HHPs) has been recruited covering 132 communities. While working in communities to establish Care Groups, the CSP staff collected information that provides a more accurate estimate of the project target population. During recruitment of HHPs, project staff identified households in each community with WRA and children U5. They found these numbers to be considerably lower than official projections based on the 1984 census and which includes the urban area of Robertsport. Based on project info, the project has recruited 520 HHPs. Develop Care Group strategy CSP team trained in Care Group approach using World Relief materials. Yes At start-up, cross visits to World Relief and Food for the Hungry Mozambique were not possible due to difficulties acquiring visas and availability of WR team. The CSP CHAL Community Outreach Coordinator will visit WR in Mozambique in November 2008. A cross visit was made to Sierra Leon to IRC’s Child Survival Project and CARE’s CSP staff came to a group meeting. Both have similar social and behavioral change and community mobilization approaches. Monthly Care Group Meetings Care Group meetings held monthly to support and train HHPs Yes CHPs hold monthly meetings with HHPs to provide in-service training, solve problems and collect information on referrals and education provided. Community education for behavior change 2000 HHPs provide regular home visits and Yes 520 HHPs provide regular home visits and community education sessions. Grand Cape Mount Child Survival Project Midterm Evaluation, Medical Teams International Liberia 15 IR 2: Improved health behaviors and actions at the household level Activities Year 1 & 2 Benchmarks Benchmark Achieved Activity Status community ed. sessions. Develop curriculum for training community leaders Curriculum developed for training community leaders N/A It was decided there was no need to develop a separate curriculum. The team is utilizing selected C-IMCI modules developed for training HHPs. KPC survey at baseline and LQAS annually Yes See attached LQAS report for MTE. Community Profile established at baseline and updated midterm Yes Community Profiles have been updated as part of the MTE. Supervision checklist used to monitor behaviors at HHP, CHP and HF levels. Yes QIVC supervision checklists are being used to monitor social and behavioral change at each level. Monitoring of behavior change R-HFSA used to evaluate behaviors at HFs at baseline and MTE. Yes An R-HFSA has been conducted at baseline and midterm. For IR 3 almost all activities have met benchmarks at the midterm point. Joint supervision visits are not on target but individual supervision and monitoring is regularly provided by project staff and R￾HFSA shows roughly half of facilities to have had a supervision visit from MOHSW CHT within the previous 3 months. IR 3: Improved quality of care in health facilities through the implementation of IMCI and capacity building in complementary areas. Activities Year 1 and 2 Benchmarks Benchmark Achieved Activity Status Develop tools for monitoring implementation of clinical IMCI Supervisory checklist developed to monitor IMCI activities Yes The Project uses a checklist adapted from the WHO Supervisory Checklist for Monitoring /Supervision of IMCI Activities. Conduct training in IMCI for CSP and HF staff 48 Physician assistants, registered and licensed practical nurses and certified midwives training in IMCI and 100 health facility support staff provided with orientation to IMCI (targets for training years 1 – 3) Yes Forty five health facility staff have been trained in case management and 78 support staff have been provided with an orientation to IMCI. A third IMCI batch training is planned for Year 3. Implementation of IMCI at health facility Implement IMCI in 22 clinics and 1 health center. As HFs are rehabilitated IMCI will be implemented in at least 4 additional clinics, and possibly as many as 9. Yes IMCI protocol currently implemented at 27 HFs. Grand Cape Mount Child Survival Project Midterm Evaluation, Medical Teams International Liberia 16 IR 3: Improved quality of care in health facilities through the implementation of IMCI and capacity building in complementary areas. Activities Year 1 and 2 Benchmarks Benchmark Achieved Activity Status Monitoring and mentoring of health facility staff Monitoring and mentoring visits provided regularly to reinforce new skills Yes Monitoring and mentoring visits provided at 27 facilities every 2 -3 months. Supply GIK to HF as needed Provide funds and GIK to improve the availability of drugs and supplies for implementing IMCI. GIK will focus on drugs and supplies which will have the most impact on IMCI outcomes Drugs will be requested to fill gaps in the National Drug Service (NDS), not replace that system Yes From July 2007 to present, MTI has regularly supplied the 5 MTI￾supported clinics with essential IMCI drugs (except for TB or malaria, which are supplied by the MOHSW) and medical supplies. Since July 2008 GIK medical supplies have been distributed to three government-supported HFs and 5 MTI-supported clinics. Drugs and medical supplies were purchased from NDS. MTI provided one shipment of GIK medical supplies and purchased drugs or supplies not available in Liberia. GIK supplies have included gloves, syringes, needles, maternity pads, sutures, etc. Work with HF to set up ORT corners ORT corners established in each facility trained in IMCI Yes All 27 facilities with staff trained in IMCI have been provided with the equipment necessary for an ORT corner. During August IMCI supervision visits, it was reported that 20 of the 27 health facilities had ORT corners that were maintained and functioning according to IMCI protocol. Train Field Supervisors in principles of supportive supervision Facilitative supervision training provided to supervisors during monthly meetings with HF and CHT staff. No Training in supportive supervision has been delayed due to unavailability of consultant. This is planned for quarter 1 or quarter 2 of year 3. Grand Cape Mount Child Survival Project Midterm Evaluation, Medical Teams International Liberia 17 IR 3: Improved quality of care in health facilities through the implementation of IMCI and capacity building in complementary areas. Activities Year 1 and 2 Benchmarks Benchmark Achieved Activity Status Conduct supervisory visits to health facilities and communities Conduct joint supervision visits including CHT Supervisors, IMCI Mentor & Coordinator, M&E Coordinator, CHAL Supervisors, and MOHSW programs of malaria, EPI and IMCI. 75% % HF receive external supervision at least once in the last six months. No No Monitoring and mentoring visits are conducted at 27 facilities every 2 -3 months. A representative from the CHT accompanied MTI staff on one occasion. The CHT is regularly provided with the monitoring schedule and invited to participate. R-HFSA found 56% of 18 randomly selected facilities to have had a supervision visit by MOHSW in the last 3 months. Revitalize CHDCs to help with clinic management 32 health facilities have CHDCs (final) N/A 16/30 health facilities have CHDCs (remaining two facilities are hospital and facility undergoing renovation.) Some activities for IR 4 are pending due to changes in MTI Country Director about one year ago and the recent resignation of the CS Project Manager in June 2008 to take a post as head of the MOHSW Family Health Department. During year 2, coordination meetings among project partners have been irregular. Organizational capacity assessments have been updated annually but the capacity building plan was not updated at the end of year 1. Key systems were put in place for strengthened among partners in Years 1 and 2; additional capacity building activities are planned for Year 3. IR 4: Strengthened institutional capacity of MTI and their partners to implement effective and efficient child survival activities Activities Year 1 and 2 Benchmarks Benchmark Achieved Activity Status Based on Institutional Capacity Assessments, develop action plan with indicators for capacity building Action plan developed during DIP development process Yes A Technical Assistance Plan for Institutional Capacity Building was developed during the DIP development process and is included as Annex 15 in the Project DIP Carry out, monitor, and report on capacity building plans Capacity building plans are reviewed and revised annually. No Capacity assessments have been updated annually. Capacity building plans need to be revised. Grand Cape Mount Child Survival Project Midterm Evaluation, Medical Teams International Liberia 18 IR 4: Strengthened institutional capacity of MTI and their partners to implement effective and efficient child survival activities Activities Year 1 and 2 Benchmarks Benchmark Achieved Activity Status Develop a system for information sharing among MTI and stakeholders (quarterly meetings, annual evaluation/planning meeting) Develop system for information sharing among MTI and stakeholders No Meetings were held on a quarterly basis during the first year of the Project but have not been held on a regular basis during year 2. The County Health Team and CHAL representatives were involved during all stages of the MTE. Contribute to process of development of Standard Health Plan for the county Yes MTI participated in the development of the GCM county health plan in 2007. In 2008, the GCM County Health Team did not invite MTI or other NGOs working in GCM to their health planning meeting due to scheduling conflict. But the CHT has sent a draft of the plan & MTI provided input. Use CSSA focus with partners as part of the DIP development process CSSA was utilized during DIP process to develop sustainability plan and indicators Yes The Project used the CSSA framework to develop its sustainability strategy as part of the DIP process. C1d. PROGRESS BY INTERVENTION AREA Immunization (10%) i. Progress as measured Indicator Baseline MTE Target MTE Result Status at MTE Final Target % of children 6-23 months who received a dose of Vitamin A in the last six months (mother’s recall). 76% (67.8-83.3) 81% 85.4% (78.4-92.5) Met 85% % of children 12-23 months who received DPT3 before they reached 12 months by the time of the interview as card verified. 30.6% (22.2-40.1) 41% 60.4% (50.6-70.2) Exceeded 50% New target: 65% % of children aged 12-23 months who are fully vaccinated (BCG, DPT3, OPV3, measles vaccines) by 12 months of age (card verified). 18.9% (12.1-27.5) 30% 39.5% (29.8-49.4) Exceeded 40% New target: 55% % of mothers with children age 0-23 months who were protected against Tetanus (at least 2 TT) before the birth of the youngest child. 61.3% (55.6-66.9) 71% 66.7% (57.2-76.1) Not met 80% Grand Cape Mount Child Survival Project Midterm Evaluation, Medical Teams International Liberia 19 ii. Factors affecting achievement: Low baseline values for child immunization have been measurably improved as verified by Child Health Card, with 86.5% of mothers in possession of the Child Health Card. MTE results for DPT3 by 12 months of age exceeds national average of 47.2% found by the DHS in 2007. Several of the key child vaccines contributed in pulling down results for complete immunization by 12 months of age, although measles had the lowest coverage (Figure No.1). MOHSW introduced the pentavalent vaccine (DPT+HB+Hib) in 2008. Figure No.1 Vaccine Baseline Midterm BCG 48.6% (39.0%-58.3%) 83.50% (76.07%, 90.93%) Polio3 36.0% (27.1%-45.7%) 65.29% (56.83%, 75.75%) DPT3 30.6% (22.2%-40.1%) 60.42% (50.64%, 70.2%) Measles 25.2% (17.5%-34.4%) 49.98% (39.98%, 59.98%) Baseline values for Vitamin A supplementation and mothers TT immunization, which were in the 60-70% range at baseline, have increased to a much smaller degree. Only 31% of mothers of children age 0-23 months had a maternal health card. Promotion of prenatal care services, and in particular TT, will receive additional emphasis during the remaining two years of project life, in support of MOHSW goals. The project has provided 5 motorcycles (from a previously funded emergency response project) and monthly support with fuel for immunization outreach at 5 health facilities, along with 1 motorcycle assigned to the County Health Team for IMCI supervision. After a period of project implementation, the CSP management found it necessary to call upon the County Health Team leadership to ensure that the donated motorcycles are correctly used. At midterm, CSP staff and CHT management report the situation has been corrected. HHPs promote complete immunization by 1 year of age and provide a referral card to mothers with children under two years of age with incomplete immunization status. HHPs also promote the basic MOHSW maternal health care messages, including promotion of use of available prenatal care services and vaccination with up to 5 doses of tetanus toxoid (at least 2 during pregnancy until achieving the total of 5) and have a referral card for prenatal and/or post-natal care. Health facilities report good supply of vaccines and Vitamin A supplements through County Health Team provision. The R-HFSA, however, found that 28% (5 of 18) health facilities did not have Vitamin A available during the assessment of essential drug availability for child health. One Supplemental Immunization Activity (SIA) campaign was conducted in the previous year and MTI, along with other NGOs active in the area, provided vehicles for additional logistic support. R￾HFSA at midterm found 44% (8 of 18) facilities surveyed to have all child vaccines on hand (with an average of 69% of all vaccines; DPT was higher at 78% while BCG, OPV and MMR were each 67%). R-HFSA found 72% (13 of 18) facilities surveyed to have an observed supply of valid Vitamin A supplement, TT vaccine and/or refrigerator or cold box for storing vaccines. CHT reports receipt of new solar-powered cold storage materials which can be carried to the field and should contribute to improved immunization outreach. Grand Cape Mount Child Survival Project Midterm Evaluation, Medical Teams International Liberia 20 iii. Promising practices; potential for scale-up or expansion: The pictorial referral system in use by HHPs for referring both children and mothers for vaccines available at health facilities is a promising practice which could be scaled up anywhere in Liberia that community health volunteers are active. CSHGP has recently funded a second Child Survival Project (Cycle 24) for implementation by Curamericas in six sub-districts of Nimba County in Liberia and MTI’s CSP should share both the C-IMCI manual and the referral system. Control of Diarrheal Disease (20%) i. Progress as measured Indicator Baseline MTE Target MTE Result Status at MTE Final Target % of children 0-23 months with diarrhea in the last two weeks who received ORS and/or recommended home fluids. 74.2% (63.8-82.9) 80% 79.2% (71.2-88.7) Met 85% % of children 0-23 months with diarrhea in the last two weeks who were offered more fluids during the illness 51.7% (40.8-62.4) 61% 64.6% (55.0-74.1) Met 70% % of households of children 0-23 months that treat water effectively. 22% (17.1-26.8) 31% 7.3% (2.1-12.5) Not met 40% % of mothers of children 0-23 months who live in households with soap or ash at the place for hand washing and that washed their hands with soap or ash at least two of the appropriate times during 24 hour recall. 19.0% (14.7-23.9) 30% 43.7% (33.8-53.7) Exceeded 40% New target: 70% ii. Factors affecting achievement: During the post-conflict relief period the use of ORS was promoted and supplied by relief organizations. Mothers’ knowledge at baseline reflects these efforts and the project is on track to achieve the final target for use of ORS or home fluids, and for more fluids during diarrhea illness. Health facility staff report continued reliable supplies of ORS. R-HFSA at midterm found 89% of health facilities to have ORS on hand. The project has provided supplies to establish ORS corners at all health facilities. Qualitative MTE visit to health facilities found ORS corners to be prepared and hygienically maintained. Although the government policy for the Basic Package of Health Services recognizes the benefits of treatment with zinc and recommends its use during episodes of diarrhea, zinc is not yet available in health facilities in GCMC. The national IMCI training protocols include orientation to use of zinc. Hopefully, it will become available in the near future and MTI can contribute to capacity building of health staff in this improved protocol. HHPs have strongly promoted the prevention of diarrhea, with special emphasis on appropriate hand washing practices. MTE focus groups found mothers’ knowledge of hygiene and specific hand washing messages to be the most well-communicated set of messages to be spontaneously mentioned by mothers. Mothers do not perceive any barriers to recommended hand washing practices. There has been support from other NGOs and agencies to construct wells with appropriate technology hand pumps. All six communities visited during MTE had more than one hand pump available and functioning. LQAS at midterm included a question on the % of households with an improved source of drinking water, with MTE results increased to 82% from Grand Cape Mount Child Survival Project Midterm Evaluation, Medical Teams International Liberia 21 72% at baseline. Point of use treatment has not been promoted by the project; more effort has gone towards ensuring CHC involvement in maintenance of the new hand pumps. Communities also expect to receive support from the County Health Team to chlorinate well water. This is not reported as a practice in any community visited. HHPs promote use of coconut water and or the water which comes from boiling rice for home fluid management of childhood diarrhea. MTE focus groups found mothers’ knowledge to be quite strong in this area also. Mothers can correctly describe how to give home fluids (use of clean cup and spoon, give small amounts as often as possible). The draft Child Survival Strategy calls for distribution of ORS and zinc to Community Health Workers; if this occurs during the life of this project, training for HHPs on the use of zinc will need to be conducted. Up till now, the project has been promoting early care-seeking for any sick child. An MTE recommendation for the second half of the project is to better clarify with HHPs and mothers when diarrhea and/or cough and cold can be managed at home and when referral to health facility services is needed, along with additional efforts at all levels for rational use of drugs. iii. Promising practices; potential for scale-up or expansion: HHP promotion of the four recommended hand washing practices (before food preparation, before feeding children, after defecation and after cleaning children’s feces) is an appropriate component of this intervention which can easily be implemented by mothers. Research has proven this practice to be effective and should be scaled-up throughout Liberia. Pneumonia Case Management (20%) i. Progress as measured Indicator Baseline MTE Target MTE Result Status at MTE Final Target % children 0-23 months with chest-related cough and fast/difficult breathing in the last 2 weeks taken to appropriate health provider. 43.2% (35.3-51.4) 54% 83.3% (75.9-90.8) Exceeded 65% New target: 85% ii. Factors affecting achievement: Rates for care-seeking for pneumonia showed the greatest improvement among project indicators since baseline with a 40% increase. The MTE value of 83.3% exceeds the national average (62.2%) for children age 0 to 59 months reported by the DHS 2007. HHP promotion of early care-seeking and use of a referral system has contributed to this result. Health facility staff confirm HHP impressions that mothers are now seeking care early in child illness, whereas previously they would wait until the child’s condition had significantly deteriorated. HHPs also feel they are effective, through home visits, in convincing families to make use of available health services versus traditional healers. The three NGOs (MTI, AHA and IMC) which actively support all but six of the 30 Health Clinics functioning in GCMC provide almost 100% of essential drugs9 to these facilities through use of private donations to purchase through the Liberia National Drug Supply. R-HFSA random sample of 18 of 30 health facilities (including the 20% of facilities with no NGO support) found 94% to 9 This does not include essential drugs for malaria and tuberculosis, which are provided through the government MOHSW. National policy also calls for MOHSW to supply any HIV ARV/PMTCT treatment, but none is available in Grand Cape Mount County. Grand Cape Mount Child Survival Project Midterm Evaluation, Medical Teams International Liberia 22 have the first-line antibiotic for pneumonia on hand (trimethoprim-sulfa). MTI provides essential drugs and staff incentives to 5 health facilities. MTI has informed the CHT that they will be reducing support for incentives by approximately 50% starting in January 2009 and completing transition of support for incentives fully to the CHT by July 2009. The Country Director is currently discussing a phase-out plan with the CHT for transition of MTI support of medicines and supplies to the CHT in the final two years of the project. It is anticipated that other activities taking place in Liberia to strengthen the national health system will ensure there is no gap in clinic support. It is recommended that during Year 3 LQAS and final evaluation KPC, the project include additional questions in the survey to demonstrate the chain of effects between early care-seeking and access to effective treatment for pneumonia Up till now, the project has been promoting early care￾seeking for any sick child. A recommendation for the second half of the project is to better clarify with HHPs and mothers when cough and cold can be managed at home and when referral to health facility services is needed, along with additional efforts at all levels for rational use of drugs. In particular, it is recommended that the project train HHPs (in coordination with pharmacy staff at Health Clinics, if possible) to follow up and counsel mothers on correct use of the antibiotic provided for pneumonia. The MOHSW has limited supplies of drug dispensing bags that show dosage in a pictorial manner using the sun and moon. It is recommended the project provide material support for use of these pictorial labels. iii. Promising practices; potential for scale-up or expansion: The selection and training of all female HHPs as community volunteers is likely to have influenced the strong impact they have had on changing care-seeking behaviors. MTE qualitative interviews with HHPs, mothers and health facility staff all remarked upon HHPs’ abilities to convince caregivers to use available health services. Nutrition (30%) i. Progress as measured Indicator Baseline MTE Target MTE Result Status at MTE Final Target % of newborns who were put to the breast within one hour of delivery and did not receive pre-lacteal feeds. 33.7% (28.3-39.3) 42% 51.0% (41.0-61.0) Exceeded 50% New target: 60% % of infants 6-9 months receiving breast milk and complementary foods. 37.5% (22.7-54.2) 51% 40.6% (30.8-50.5) Not met 65% % of children 0-23 months who are underweight (<-2 SD wt-for-age WHO reference). 27.1% (22.1-32.6) 22% 20.8% (12.7-28.9) Exceeded 17% ii. Factors affecting achievement: Differing from national level DHS results, exclusive breastfeeding to 6 months was found to be relatively high at baseline (86.0%) in the project area. The project has focused on early initiation of exclusive breastfeeding, which was low at 33.7%. Through promotion of the benefits of colostrum, along with elimination of any pre-lacteal feeds and continued exclusive breastfeeding to 6 months of age, the project has increased early initiation of exclusive breastfeeding to 51.0%. Besides providing training to HHPs (who are exclusively female) in behavior change communication, MTI had also previously worked with traditional birth attendants (TBAs) in the area through a relief project. These TBAs are still vocal proponents of the messages HHPs promote and have had an important influence on the improvements found at midterm for early initiation of exclusive breastfeeding. Midterm LQAS found only 24% of births Grand Cape Mount Child Survival Project Midterm Evaluation, Medical Teams International Liberia 23 were attended by skilled health professionals. The project also promotes continued breastfeeding to at least 24 months and MTE focus groups found mothers to have good knowledge of this message. The project baseline survey was done before the new Infant and Young Child Feeding (IYCF) indicator was added to the KPC. The project, therefore, has been tracking the introduction of complementary feeding at 6 to 9 months of age. There has been no improvement in this indicator, with late introduction of semi-solids for more than half of children. Staff CHPs had done some food demonstrations with HHPs, preparing porridge (the traditional “weaning” food) and adding dried fish (bony) and crushed sesame seeds (bennis). MTE focus groups found a few mothers knew this recipe but few were clear on the importance of a variety of foods in the diet. Some mothers mentioned frequency of feeding, but had little concept of portion size. During the next two years, the project will put increased emphasis on all the recommended Essential Nutrition Actions for complementary feeding of children 6 to 24 months of age and assist mothers to develop active feeding skills. The MTI HQ CS Advisor obtained excellent training materials on complementary feeding from Freedom from Hunger and shared these materials with field staff during MTE. Project efforts to educate men/husbands on child health should also focus on IYCF recommendations, to ensure that husbands support their wives to purchase the food they need for their children. It is suggested the project consider repeating the Doer/Non-doer analysis they did at the start of the project, with a focus on complementary feeding practices. The project exceeded midterm target for reducing child undernutrition. This likely represents cumulative effects from improvement in each intervention area. This indicator is now closer in value to the DHS 2007 national average for children age 0 to 59 months in rural areas, 20.0%. Re￾emphasis of complementary feeding messages will enable the project to meet the final target. Government policy for the Basic Package of Health Services recognizes that growth monitoring is not being done at health facilities and has not prioritized this at present due to the level of effort needed to bring health staff up-to-date on IMCI protocols, along with recognition that training in correct growth monitoring with appropriate counseling of child caregivers also involves a significant level of effort. Project IMCI mentoring activities include follow up on health staff skills in weighing children and recording weight, as this information can also be necessary for prescribing the correct dosage of some medicines. At present, R-HFSA found some level of growth monitoring skills (weighing child and plotting on Child Health Card graph during sick child visit) to be done at 44% of facilities surveyed. The MTE consultant does not recommend the project initiate training in growth monitoring and counseling at this point. It is recommended that the project continue to strengthen health staff skills in weighing children and recording weight and in this way be prepared to support any national efforts to revitalize growth monitoring as they occur. iii. Promising practices; potential for scale-up or expansion: Involving traditional birth attendants as persons of influence for message dissemination and promotion of early initiation of exclusive breastfeeding is a promising practice. The project has been able to maintain the involvement of these TBAs without further inputs from any project. Grand Cape Mount Child Survival Project Midterm Evaluation, Medical Teams International Liberia 24 Malaria (20%) i. Progress as measured Indicator Baseline MTE Target MTE Result Status at MTE Final Target % of children 0-23 months who slept under an insecticide-treated bed net the previous night. 17.7% (13.5-22.5) 27% 67.7% (58.4-77.1) Exceeded 35% New target: 70% % of children 0-23 months with a febrile episode that ended during the last 2 weeks who were treated with an effective anti￾malarial within 24 hrs after fever began. 12.5%10 (5.9-17.7) 32% 49.06% (39.0-59.0) Exceeded 50% New target: 65% ii. Factors affecting achievement: Liberia is one of 8 countries to benefit from inclusion in the third round of countries selected for the President’s Malaria Initiative (PMI) in Africa. In line with the MOHSW National Malaria Control Program, the goal of the PMI is to cut malaria deaths by 50% by reaching 85% of the most vulnerable groups, including pregnant women and children U5. The PMI four key intervention strategies include distribution of long-lasting insecticide treated bednets (LLIN), indoor residual spraying (IRS), provision of artemisinin-combination therapy drugs (ACT) and support for intermittent presumptive treatment for malaria during pregnancy (IPTp). PMI was launched in Liberia in 2008, starting with distribution of more than 150,000 LLINs in Grand Cape Mount and Bomi counties. Further LLIN distribution continued in other areas of the country with a goal of reaching national household ownership of LLINs at 60%. Each household was entitled to 3 LLINs; mother and children are considered a household unit, which is important for the project area in which multiple wives are common among the Muslim population. The PMI activity has clearly contributed to midterm LQAS results which show almost 70% of children 0-23 months to be sleeping under a LLIN in the project area (the national Malaria Indicator Survey in 2005 found only 2.6% of children to be sleeping under ITNs). Through MTI’s CSP, Household Health Promoters check that households are using nets properly and promote the benefits of prevention through bednet use and care, avoiding tears or other damage. In February 2008, the PMI provided supplies for ACT treatment of almost ½ million malaria cases, with plans for additional drug supplies. Both the R-HFSA and MTE qualitative interviews at health facilities found good supplies of ACT (and only ACT for oral treatment, along with injectable quinine) available. The national Malaria Indicator Survey in 2005 found only 3.2% of children U5 to receive an effective anti-malarial within 24 hours of fever onset. The LQAS MTE result of 40.6% of children 0-23 months (as compared to 12.5% at baseline) can be partly attributed to improved drug supplies which are a key factor affecting household’s care-seeking behavior. The limiting factor for further improvement in this indicator requires additional encouragement of mothers and other child caregivers to seek treatment immediately when a child has fever. Through continued HHP counseling and use of a referral system during the next two years, further improvements in care￾seeking for children with signs of malaria can be expected. Such excellent supplies of appropriate but relatively expensive drugs underscores the need for rational drug use by all involved. The R-HFSA found health facility staff to correctly diagnose and prescribe treatment for 89% of sick child cases, but only 34% of caregivers of children could 10 Baseline value and targets were recalculated at MTE upon review of KPC criteria. Grand Cape Mount Child Survival Project Midterm Evaluation, Medical Teams International Liberia 25 correctly describe how to administer prescribed drugs. This indicates that caregivers are not receiving appropriate counseling by health staff on how to administer the medicines to their child. MTE recommendations include strengthening counseling skills, especially on drug use, through specific training activities with health facility staff (including pharmacy staff) and on-going IMCI mentoring. MTE qualitative interviews found a need for the project to include training of both literate and illiterate HHPs (13% and 87%, respectively) in correct drug use, so that they can provide follow-up counseling to caregivers. The MOHSW has a pictorial system (a dispensing bag with sun, moon and other pictures for both frequency and dosage) which MTI’s CSP should further seek to support, both in usage and supplies. Although MTI’s CSP does not have a maternal health intervention, HHPs have been trained to promote the key MOHSW recommendations for maternal care, including IPTp as part of the prenatal care package. Variable supplies of sulphadoxine-pyrimethamine (SP) were found during qualitative MTE interviews at health facilities, with some supplies past the expiration date. MTE focus groups also showed mother’s knowledge of IPTp to be weak. At present, HHPs only provide counseling on maternal care through home visits to pregnant women. A recommendation from MTE is that maternal care be promoted to all households, so that message dissemination will reach as broad an audience as possible, and that the importance of IPTp be re-emphasized in Year 3. iii. Promising practices; potential for scale-up or expansion: Linking community-based project efforts to the President’s Malaria Initiative which increases access to inputs (such as LLINs, ACT, IPTp) is likely to have a synergistic effect that increases the achievement of the national goal to reduce mortality from malaria in these combined project areas. It would be interesting in future to compare results in areas with community-based projects (such as this CSP in GCMC and the new Curamericas CSP) to results found in areas of Liberia without significant community interventions. C1e. NEW OR INNOVATIVE TOOLS OR APPROACHES: MTI has chosen to use a cost￾effective strategy to jump-start improvements in child health in a post-conflict environment. Partnering with a local NGO has been a cost-effective strategy for implementation of the somewhat staff-intensive Care Group model (Figure No.2), which has resulted in rapid improvements in family prevention and care seeking practices in rural areas with difficult access. Figure No.2: Structure of Care Groups 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 4 Supervisors 1 1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8 9 10 58 Care Groups with 6 to 8 HHPs each for a total of 520 HHPs in 132 communities - 20 Community Health Promoters 2 3 4 Each HHP visits 10 to 15 households Grand Cape Mount Child Survival Project Midterm Evaluation, Medical Teams International Liberia 26 World Relief, who has pioneered this model, has had success with the model in non-transitional areas, also. But it appears particularly suited for post-conflict settings which are transitioning from relief to development, resulting in rapid improvements in child health that are perceived by community participants. Midterm results show the model’s strategy for the selection and training of female Household Health Promoters to be an effective way to change mothers’ child health care practices. The project’s use of pictorial tools for health education and for creating a referral system between HHPs and health facilities is an effective strategy in an area with an extremely high level of illiteracy (Grand Cape Mount County has the lowest level of female literacy in the country, at 7.4%; 13% of HHPs are literate) and demonstrates MTI’s ability to adapt C-IMCI tools to a post-conflict environment. The health system in Liberia has been devastated by years of conflict. After providing support for relief and infrastructure rebuilding, MTI has been able to continue to support the transition to development through this CSP. Assistance provided for health facility outreach has resulted in a doubling of the percentage of children with DPT3 and/or complete immunization by 12 months of age. Linking communities to quality health services which have been improved through IMCI training and mentoring by MTI CSP staff who are Master Trainers and/or also trained in IMCI has resulted in an increase in early care seeking for common childhood illnesses. With a local NGO sub-grantee implementing community outreach for behavior change, MTI staff are also able to dedicate significant amounts of time to an intensive and high quality monitoring and evaluation system. The MTI IMCI Coordinator and IMCI Mentor use a Quality Improvement Verification Checklist for monitoring and feedback during IMCI follow-up visits. With technical support from MTI HQ M&E Specialist, MTI staff have conducted annual LQAS, along with a Rapid-Health Facility Services Assessment at baseline and midterm and which will be repeated in Year 3. This level of monitoring and evaluation is likely to also have kept the momentum for change on-going, with good results seen at midterm evaluation. Having a CSP in a country that benefits from the President’s Malaria Initiative and other USAID support also has contributed to the rapid improvements in health care in Grand Cape Mount County in the past two years, with indicators of care seeking for pneumonia and/or malaria doubled since baseline. As noted in the government’s Basic Package of Health Services11, it is recognized that improvements in people’s health and well-being constitute a “peace dividend” that can contribute not only to humanitarian, but also political and economic purposes as well. 2. RESULTS: FAMILY PLANNING (not applicable) 3. RESULTS: TUBERCULOSIS (not applicable) 4. RESULTS: CROSS-CUTTING APPROACHES 4a. COMMUNITY MOBILIZATION: The government recognizes the role of community health volunteers implicitly as noted in the policy for the Basic Package of Health and Social Welfare Services (BPHSWS) which calls for community support for such volunteers but does not support, at 11 “The Basic Package of Health and Social Welfare Services”, Republic of Liberia, Ministry of Health and Social Welfare; Monrovia. Grand Cape Mount Child Survival Project Midterm Evaluation, Medical Teams International Liberia 27 present, the use of any essential medicines by CHVs due to the national perception of existing CHV skill levels. The MOHSW is in the process of further developing policies related community health volunteers. The government mandates Community Health and Development Committees to be linked with health facilities, but policies for how these are to be linked to other elements, such as Community Development Committees (what this CSP terms “Community Health Committees”) are not yet defined. Within MTI’s CSP, communities are mobilized in several ways. First, the project provides training to female Household Health Promoters for C-IMCI behavior change communication through the Care Group model. These HHPs conduct home visits, occasional group sessions with women of influence in the community, and periodic community-level clean-up campaigns through linkage with local leaders, such as the Town Chief. Secondly, a referral system between HHPs and health facilities has been established, mobilizing mothers and other caregivers to seek available services for child and maternal health. Thirdly, Community Health Committees have been formed or revitalized and these are linked in support of HHP activities. At project start-up, a local consultant (Marion Subah) provided training to MTI staff and partners (CHAL and the County Health Team), along with some Health Clinic staff, on the Care Group model and community mobilization in a “Training for Transformation” workshop. A refresher training on this topic and orientation to participatory action and learning methods was provided by the MTI HQ CS Advisor in May 2008. (See also training provided by MTI HQ staff on social and behavioral change communication and C-IMCI in section 4b.Communication for Behavior Change.) At midterm evaluation, the first two elements of the community mobilization strategy are quite strong, while the third has just been initiated. CHCs have been encouraged to include women on their committee, with about 75% having 1 or 2 women among a group of 5 members. CHCs have been trained on problem analysis and action planning. Local leaders who are the key members of the CHCs endorse all HHP activities. This did not occur at first and was a lesson learned for project staff. After introducing the project to local leaders and community members, the focus moved purely to interaction with HHPs. Once the project included more direct interaction with local leaders, providing orientation to key health messages, their enthusiasm for HHP activities became apparent. Now that CHCs have been formed, they describe plans to provide financial or material support to HHPs. As most of these plans center around support for HHP farming activities or planting a group plot with sale of produce to benefit HHPs, the upcoming planting season will provide better evidence of CHC commitment. It is suggested at MTE that the project not only emphasize the need for CHC support for HHPs, but also that a role for CHC monitoring of HHP activities be promoted, with HHPs reporting periodically on their activities to the committee. It is also expected that CHCs will have a link to the Community Health and Development Committees (CHDCs) attached to health facilities. It is recommended that project assess the status of CHDCs at the end of Year 3 and look for ways to support and facilitate this linkage. Grand Cape Mount Child Survival Project Midterm Evaluation, Medical Teams International Liberia 28 4b. COMMUNICATION FOR BEHAVIOR CHANGE: During preparation of the DIP, CSP staff took the first steps in building a BEHAVE framework. During a two-day workshop participants helped identify Priority Behaviors, Motivators, Barriers and Channels for communication. Due to time limitations at that point, the Doer/Non-doer exercise was not completed. After project start-up activities were established, the MTI HQ CS Advisor visited the project to collaborate with the team in defining a Social and Behavioral Change Communication plan, refining the BEHAVE framework and completing the Doer/Non-doer exercise. This further refined plans and a revised BCC Plan was developed. A follow up workshop was held with representatives of the MOHSW, the GCM County Health Team, the national Breastfeeding Advisory Group (BAG) and CHAL to review the revised plan (along with C-IMCI materials, below). The MTI HQ Capacity Building Advisor visited Liberia to conduct a workshop to develop C-IMCI curricula that also would include adult training methodologies. The HQ CS Advisor assisted the team to develop flip charts to complement the C-IMCI curriculum. Examples of materials used for C-IMCI were gathered from the MOHSW, BAG, CHAL, Africare and the Ghana MOH and discussed and adapted. These were then field-tested by project staff with input, revision and review by MTI HQ Capacity Building Advisor and CS advisor. MTI’s CSP uses the following Channels of Communication within its BCC strategy: Interpersonal communication: Regular home visits are provided by each Household Health Promoter to 15 homes with children U5 in every community, prioritizing those with children 0-23 months of age. LQAS results at midterm show this strategy to be effective in changing behaviors. Refresher training will be provided in Year 3 to further strengthen HHP skills in one-to-one dialogue and negotiation for behavior change. Traditional ways of Learning: CHAL Community Health Promoters train HHPs during Care Group activities within communities, and mentor HHPs during health outreach activities within their communities, which (besides home visits) include occasional group sessions with mothers and/or community gatherings called by the Town Chief. In group activities, traditional adult learning techniques are used employing drama, stories and songs to disseminate messages. Print education materials: MTI CSP and CHAL staff, with support and review by MTI HQ CS Advisor and Capacity Building Advisor, have produced a laminated set of flipcharts with pictures that convey the key C-IMCI messages. Only 13% of HHPs are literate; government documents quote a national average for literacy among women over age 15 at 48%; the DHS in 2007 found similar results nationally, but only 25% in rural areas; the average for women in GCMC is 4.7% per County Health Team information. A draft version of the flipcharts was field tested early in 2008; slight revisions were made and the final product was distributed to HHPs in August of 2008. On￾going refresher training and mentoring of HHPs will strengthen use of these flipcharts for home visit counseling and is expected to contribute to even greater improvements in mothers’ knowledge and practices during the remaining two years of this project. Small laminated cards for referral were extracted from the pictures developed for this flipchart and provided to HHPs in June 2008, with training in their use. All materials appear in-line with recognized key messages for C-IMCI, and are appropriate for use in low literacy areas. Social support from family members present during home visits and/or through communication during normal activities with extended family and community members is another channel which Grand Cape Mount Child Survival Project Midterm Evaluation, Medical Teams International Liberia 29 contributes to message dissemination. Mothers need support from their husbands, other family members and friends to adopt improved prevention and care seeking practices. It is expected that social network support has contributed to some of the positive results found in the midterm LQAS, such as a doubling in the percentage of mothers now providing immediate and exclusive breastfeeding within one hour after childbirth. Previous support and training provided to Traditional Birth Attendants by MTI through a separate Primary Health Care project during the post-conflict relief period has also contributed to social network support for recommended behaviors. Midterm qualitative evaluation found a notably high degree of accuracy and consistency in the C-IMCI messages which HHPs and mothers spontaneously recall. Messages are fully in-line with CSHGP Technical Reference Materials. At present, the project reaches men through messages disseminated by imams and pastors during religious services. Men also participate in some home visits which are family oriented. Though there is representation by women on Community Health Committees (female participation ranges from a high of 3 men and 2 women on some CHCs, to a low of no women on the CHC in some project areas; average for CHC membership is 4 men and 1 woman), they are still somewhat of a traditional channel for message dissemination to men. At present the CHCs focus on problem￾solving; during the second half of this project it is planned that, through HHP links with CHCs, periodic activities for dissemination of key messages to men will be further strengthened. 4c. CAPACITY BUILDING APPROACH: Capacity building assistance has been provided to MTI Liberia as well as the project sub-grantee, CHAL. An institutional assessment, using a tool previously developed by MTI and included in their International Programs Manual, was conducted in Year 1 by the MTI HQ Regional Manager for Africa. This assessment was updated in Year 2. Capacity building activities center on the following areas: Leadership and agency planning; financial, administrative and human resource management; project design; infrastructure and relationships. The CHAL Community Outreach Coordinator has benefited from capacity building efforts in technical areas such as adult learning, behavior change and community mobilization. It is suggested that CHAL find ways to further contribute to her capacity as a manager, for example training in how to conduct performance appraisals and assisting staff to develop work plans. The updated organizational capacity assessments for CHAL and the County Health Team are included as Annex 7. An updated Capacity Building Plan for all three project partners is included as Annex 8. MTI HQ staff contributed to capacity building in financial, administrative and human resource management. The MTI HQ Vice President of Finance provided training to MTI and CHAL staff in December 2006; this was followed by a three month visit from the MTI West Africa Program Manager to further refine procedures. In November 2007, the MTI West Africa Program Manager advised and provided examples of models for inventory management systems to improve MTI Liberia’s health clinic supplies system, funded through private donations and gifts-in-kind. The MTI HQ Vice President of Finance returned in March 2008 to conduct a refresher training for MTI and CHAL on financial management and also led an internal audit. During the internal audit, the financial records of both MTI-Liberia and CHAL were reviewed. Both organizations received a list of written recommendations for improvements in processes and procedures, but that no reportable conditions were found. Grand Cape Mount Child Survival Project Midterm Evaluation, Medical Teams International Liberia 30 The Grand Cape Mount County Health Team, along with CHAL and MTI, has also participated in capacity building workshops provided by MTI HQ staff and consultants, including: Setting up Care Groups and community mobilization in November 2006 (Liberia consultant), use of the BEHAVE framework during DIP in December 2006 (international consultant), Project Cycle Management in May 2007 (Technical Services Specialist and Director of Regional Programs), training in adult education methodologies in May 2007 (Capacity Building Advisor), refining a BCC strategy in September 2007 (CS Advisor), and Participatory Action and Learning tools in May 2008 (CS Advisor). CHAL and CHT have also had their capacity for monitoring and evaluation strengthened through orientation to the QIVC system and through participation in the KPC survey at baseline, Year 1 and Year 2 LQAS, and midterm qualitative evaluation activities. 4d. HEALTH SYSTEMS STRENGTHENING: In Liberia, the health system consists of the following levels: the community, health clinics, health centers, the county hospital, and the key national referral hospital, JFK Memorial Hospital in Monrovia. The health system was devastated by the more than 14 years of conflict, and trained health workers had left rural areas and, in many cases, Liberia itself. MTI provided support during the relief period through mobile clinics which have been phased out as of October 2006. The Ministry of Health and Social Welfare has prioritized a Basic Package of Health Services. There are 32 health facilities in GCMC, one of which is the County Hospital and one which is privately run. In GCMC, 16 of the 30 Health Centers/Clinics have met BPHS criteria. The goal of the National Poverty Reduction Scorecard is for 70% of facilities to provide the BPHS. County Health Team staff have attended planning workshops and management courses. Health planning has been decentralized and each County Health Team has developed their own plan, in accordance with the National Health Plan and policy for the Basic Package of Health Services. Infrastructure and human resources: In the MOHSW system in Liberia, Health Clinics have at a minimum two assigned professional staff – a (registered) nurse and a certified midwife. In some cases, a licensed practical nurse and a trained traditional midwife fill these positions. Many physician’s assistants are placed as Officers-In-Charge of GCMC health facilities. The 30 Health Centers/Clinics in GCMC have a total of 14 Physician Assistants, 22 Registered Nurses, 7 BSc. degree Nurses and 19 Certified Midwives or a total of 62 health professionals (an average of 2.06 per facility). Health Clinics officially offer services from 8 a.m. to 4 p.m. Monday through Friday. Health Centers -- of which there is only one in GCMC at Sinje, with Lofa Bridge Health Clinic in the process of being upgraded to a Health Center, with support from MTI – offer services 24 hours/7 days a week. At project baseline in 2006, just a few years after the Accra Comprehensive Peace Agreement was signed, there were 23 health facilities functioning. Since then, the County Health Team, with NGO support, has refurbished facilities for a total of 31 functioning, including the County Hospital. At 24 Health Clinics (12 of the 16 that offer the BPHS), NGOs12 provide incentives to health professionals who are not official MOHSW staff but rather are contract workers employed by the County Health Team through relief and transition support. The government will need to make 12 Nine facilities supported by African Humanitarian Action, 10 by International Medical Corps, and 5 by MTI. Grand Cape Mount Child Survival Project Midterm Evaluation, Medical Teams International Liberia 31 official decisions on the status of these health workers in order to ensure the sustainability of health care improvements in this county (and others). Such issues are raised by MTI and other NGOs in Inter-Institutional Coordination Meetings at the national level, chaired by the MOHSW (and previously chaired by WHO during the relief period). Equipment and supplies: Among the 30 Health Clinics, NGOs (AHA, IMC and MTI) provide support for limited equipment and supplies in 24 facilities. In these 24, they have been providing all essential drugs (except for malaria and tuberculosis drugs, which are provided through the MOHSW) with approved purchase from the National Drug Supply. AHA provides support to 9 Health Clinics, IMC to 10, and MTI to 5. MTI will be reducing this support to 50% starting in 2009 but will continually assess the situation to determine if any critical essential medicines are lacking. IMCI Training and Mentoring: The original MTI CS Project Manager had been trained in IMCI in Nigeria in 2002. In August 2007, with support from Liberia’s MOHSW, he participated in a WHO￾supported training in Ghana to be certified as an IMCI Course Director. He then facilitated further cascade training of Master Trainers in Liberia, with support from IMCI consultants from Ghana. This project then supported IMCI training (by project staff and other IMCI master trainers) for 2 batches of health professionals. Batch one included the MTI CSP IMCI Mentor and partner CHAL staff, along with 2 members of the Grand Cape Mount County Health Team and 18 GCMC health professionals. Batch 2 consisted entirely of 24 GCMC health professionals (physician’s assistants and nurses) for a total of 42 trained, most (36 or 85%) of whom remain in positions at 27 health facilities in GCMC as of midterm evaluation. A third batch of IMCI training is planned for Year 3 with another 24 health professionals to be trained. The MTI CSP IMCI Coordinator and the MTI CSP IMCI Mentor conduct follow-up supportive supervision visits to IMCI trained health facility staff to mentor them in the application of their new skills, visiting each facility at least once a quarter. A Quality Improvement Verification Checklist form, derived from the CSTS+ R-HFSA tool, is used during IMCI mentoring visits. Findings are reviewed with the health facility staff themselves and CHT leadership. The MTI HQ CS Advisor and M&E Specialist also review random samples of these materials during backstop project visits. A weakness in the delivery of IMCI services identified during the MTE is that the health professionals trained in IMCI (which will be on average two per facility after Batch 3 training in Year 3) are not present in health facilities 100% of the time designated for service provision. A recommendation from MTE is to provide health facility support staff with a general orientation of the IMCI strategy and train them in the individual IMCI protocols that are relevant to their roles. R-HFSA Assessment Results: At midterm, the CSTS+ tool for Rapid Health Facility Services Assessment was used to assess IMCI skills. Key findings include: o Improvement in supervision visits within 3 months of survey: From a low of less than 5% at baseline to more than half (56%) at midterm. o Improvement in HF clinical assessment skills: From 9% at baseline to 28% of health staff at midterm were completing ALL required assessment tasks for the sick child. On average, 66% of all tasks were completed. o Improvement in correct diagnosis and treatment: From 46% at baseline to 89% at midterm. Grand Cape Mount Child Survival Project Midterm Evaluation, Medical Teams International Liberia 32 The R-HFSA shows about half of health facilities visited to be using HMIS information for decision-making; however, MTI staff report this area still requires significant improvement as there are consistent problems with thorough record-keeping. 4e. POLICY AND ADVOCACY: The MTI Country Director and/or Primary Health Care Coordinator regularly participate in Inter-institutional Coordination Meetings chaired by the MOHSW. The MTI Country Director does not have a health background but has eagerly learned the fundamentals of child survival and is becoming an effective advocate for the project. MTI’s PHC Coordinator is cited as a contributor to the MOHSW Child Survival Strategy paper in draft form at present. The former MTI CS Project Manager left the project in June 2008 to become head of the MOHSW Family Health Department. He continues to advocate for project strategies within his present position and met with the MTE team, but was unable to participate in the midterm evaluation. 4f. CONTRIBUTION TO SCALE: As noted previously, the MOHSW is in the process of defining their community health strategy. Community health volunteers, to be trained and monitored by the MOHSW but supported by communities, are already noted as key players in the national (draft) Child Survival Strategy. As seen by MTE results, MTI and CHAL have significant experience in successfully launching community mobilization for social and behavioral change, along with C-IMCI materials that would be useful for any effort within Liberia to mobilize community human resources. There has already been a request for MTI IMCI trained staff to participate as facilitator’s in the roll￾out of IMCI training in other counties. MTI management is aware that the amount of time staff can dedicate to activities outside the project will need to be carefully negotiated. 4g. EQUITY: The selection of Grand Cape Mount County for this Child Survival Project helps promote equity in Liberia. Liberia as a nation has a religious affiliation of 82% Christian and 16% Muslim. With 90% of the population Muslim and 9% Christian, GCMC has the highest Muslim population in the country. The Christian Health Association of Liberia as a sub-grantee for community implementation has encountered no religious difficulties, with both Imams and Christian pastors equally and jointly participating in community activities and providing support to the project. CHAL had previous and extensive experience working in community mobilization in Lofa, Nimba and Bong counties which have significant Muslim populations. CHAL field staff also report support from traditional leaders when traditional practices (such as initiation ceremonies) interfere with travel to communities, with traditional leaders ensuring the safe passage of staff. There are a variety of ethnic groups in Grand Cape Mount County, with Vai predominating at 69%, while Vai only compose 5.6% of the national population. Other ethnic groups in GCMC include Bassa, Gola, Fanti, Kru and Mende. These groups are readily inter-mixed throughout the villages. MTE qualitative interviews and focus groups found community members to perceive no specific inequity based on ethnicity but rather spoke more often of “neighbor disputes” that derive from problems in social interactions. When asked if these projects can contribute to maintaining peace or decreasing the potential for conflict, all local leaders and Community Health Committee members noted that these projects provide opportunities for community dialogue, which always is a positive Grand Cape Mount Child Survival Project Midterm Evaluation, Medical Teams International Liberia 33 factor for decreasing conflict. In GCMC, the problems during the previous conflict “came from outside”, so community members did not perceive a break in unity. A few instances of child soldiers having been abducted by different groups and now returning back to the village were encountered and said to create bad feelings among some, but mothers claimed that being brought together in group meetings for health had assisted in getting past these issues. Within GCM, 60% of the population speak Vai, 23% Gola and 6% Mende. Both staff Community Health Promoters and community volunteer HHPs have been recruited from GCM and therefore necessary language skills are available for good communication with household members. The fact that the project has selected only women to be trained as Household Health Promoters is a way to improve gender equity in communities by developing women’s leadership capacity. The project has encouraged Community Health Committees to include at least one woman in their committee (75% have one or more women out of five at midterm) and to provide moral and material support for these female HHPs in recognition of their contribution to the community. One interesting result of MTE qualitative interviews with health staff and project partners was to encounter a perception that “equity” had been improved in the area because health facilities now focus on children under five, whereas previously greater attention was paid to adult problems. 4h. SUSTAINABILITY: This project used the CSTS+ Child Survival Sustainability Assessment tool during the DIP process, holding a one-day Sustainability Workshop to complete Steps 1 through 3 of the CSSA process (1. Define the system, its actors and its vision; 2. Define goals and shared vision, challenges and strategies to overcome; and 3. Identify elements of each dimension, responsibilities and indicators). As planned, Step 4 (to measure and map indicators) was completed at the end of Year 2. The vision developed by the project and partners is: Viable, knowledgeable communities in control of their own health outcomes through positive behaviors which overcome barriers, and strong interconnected community structures in collaboration, and through linkages with high quality health (and other sector) structures within a decentralized, empowered and interconnected system. During the DIP, 11 CSSA indicators were selected. With advice from the MTE consultant, these were better defined and reduced to 8 indicators by MTI HQ staff (CS Advisor and M&E Specialist), MTI Liberia CSP management and project partners, CHAL and County Health Team leadership. Baseline values were established for these indicators and targets set, as presented below. Dimension I: Health outcomes and services Goal: To provide health & social services that will support the implementation of interventions to improve health indicators within communities in Grand Cape Mount County. Indicator Targets at final Progress as of September 2008 Ia. Improvement in indicators for health behaviors at community level and at health facility level See attached KPC indicator table See attached KPC indicator table. Grand Cape Mount Child Survival Project Midterm Evaluation, Medical Teams International Liberia 34 Ib. % of communities using a HMIS for decision making 40% of communities using a HMIS for decision making 0 community health committees using HMIS for decision making (Care Group data will be shared, beginning October 2008). Dimension II: Organizational capacity and viability Goal: To promote high quality care services in Grand Cape Mount County through collaboration /cooperation between the CHT and local NGOs with the involvement of local structures & sectors. Indicator Progress as of September 2008 Targets at final IIa. % of organizations using the HMIS to make decisions 44% health facilities using HMIS for decision making 0% CHDCs using HMIS for decision making 75% Health facilities and CHDCs using HMIS for decision making IIb. % of community structures and health facilities coordinating and implementing activities based on an approved Standard Health Plan A recent assessment by MOHSW indicated that 34% of health facilities in the County meet BPHS accreditation standards. 50% of health facilities meet accreditation standards for the BPHS by December 2008 and 70% by the December 2009. Dimension III: Community competence and political environment. Goal: Provide an improved, empowered community with the support of local, administrative & national leadership to develop appropriate policies that will sustain political, social & economic community-based programs for the people of GCM County. Indicator Progress as of September 2008 Targets at final IIIa. # of CHP and HHPs providing community services (homes visits and referrals) 20 CHPs, 520 HHPs providing home visits and referrals At least 266 HHPs (51%, ~2 per each of 132 communities) providing home visits and referrals. IIIb. # of communities with CHCs and health facilities with CHDC who have met in last 3 months. 126/132 (95.5%) of communities have CHCs who have met within the past 3 months 16 of 30 (53%) health facilities have CHDCs who have met in the last 3 months 132 communities have CHCs who have met within the past 3 months At least 24 (80% of 30) health facilities have CHDCs who have met in the last 3 months D. Changes in Grantee Organization Capacity Medical Teams International, as an Entry/New Partner to the USAID Child Survival Health Grants Program, has strengthened HQ capacity in child survival by hiring a Monitoring and Evaluation Specialist in January 2006, a Capacity Building Advisor in February 2007, and a CS Advisor in August 2007. All three have visited the project frequently within the first two years of project operation (see Annex 2). The M&E Specialist is based in Washington DC and presently chairs the CORE M&E Working Group. The Capacity Building Advisor specializes in training design and adult learning methodologies. The CS Advisor has lengthy experience in child health, having previously managed child survival and primary health care programs in Tajikistan, Azerbaijan, Burundi and Southern Sudan. Grand Cape Mount Child Survival Project Midterm Evaluation, Medical Teams International Liberia 35 E. Mission Collaboration The USAID Liberia Health Unit has recently grown and now is composed of a staff of five. The Health Officer, Chris McDermott, has readily been available for consultation by MTI and has met with MTI HQ and project staff frequently. The Health Officer reviewed the MTE plans and the entire Health Unit team (except for Dr. Duworko, who was in the USA at that time) participated in the MTE debriefing. The USAID Health Officer also regularly searches for opportunities to link appropriate staff to training opportunities and enabled the pharmacist from Grand Cape Mount County Health Team to attend a recent workshop on logistics strengthening in South Africa. During the MTE debriefing, MTI staff were also informed of a new opportunity for candidates to be proposed for further education, through a nominations committee within the Human Resource department of MOHSW. This project complements the large-scale efforts of the President’s Malaria Initiative in Liberia through promotion of behavior change and appropriate care practices at the community level, while also strengthening health facility staff use of IMCI protocols for malaria diagnosis and treatment. This project, through capacity-building activities with the Grand Cape Mount County Health Team, also supports the mission’s broad efforts to assist the Ministry of Health and Social Welfare and non-governmental providers to deliver a basic package of essential health services to under-served areas of the country. There has been no discussion as to whether there will be any local mission funds available to continue child survival activities when this project funding period ends. F. Contextual Factors that Have Influenced Progress to Date There have been no unexpected contextual factors to hinder progress of this project other than occasional short-term (no more than a few weeks) community access limitations due to deteriorating bridges. The very heavy rain period occurred as expected, and the dry harmattan winds occurred as expected and staff were able to reach communities and continue building the capacity of HHPs. The area has remained peaceful with a level of crime similar to that in other areas of the country. There are several contextual factors that positively influence the progress of this project, including government support for the role of community health volunteers and stakeholder involvement through Community Health and Development Committees, and interest by the Ministry of Health and Social Welfare for the revitalization of the IMCI framework for management of child illness. G. Conclusions and Recommendations Conclusions: The project is on-track overall with activities and has had significant success to-date as seen on the Project Monitoring and Evaluation Matrix (Annex 3). The project strategy to jump-start improvements in health through sub-grant to a local NGO for community mobilization using the Care Group model, with MTI staff providing support to partners and intensive focus on IMCI mentoring of health facility staff, is considered particularly effective for a post-conflict environment. MTI has invested sizeable effort in providing HQ support and capacity-building activities for project staff and partners, and the benefits of this investment are visible. Grand Cape Mount Child Survival Project Midterm Evaluation, Medical Teams International Liberia 36 Project design has drawn upon effective tools available through the USAID-supported child survival community, including use of the BEHAVE framework for developing a behavior change strategy and use of the Child Survival Sustainability Assessment tool to define and promote sustainability of project efforts. The project has a very strong monitoring and evaluation component, with tools for quality supportive supervision obtained from other CSP NGOs, and with the MTI HQ M&E Specialist providing support for capacity-building of staff and partners through implementation of the KPC survey, annual LQAS and the Rapid-Health Facility Services Assessment. General Recommendations 1. The project should continue work plan activities as planned for Year 3 and 4. 2. The project should obtain national census final results as soon as these become available. 3. Year 3 LQAS and final evaluation KPC survey should include additional questions to confirm that families practicing early care-seeking are receiving appropriate treatment for pneumonia. It should also include additional questions to assess progress in improving infant and young child feeding practices. 4. Increased visibility of the project at national level is encouraged, along with advocacy at national level for solution to critical issues at the county level. The Country Director and Project Manager should actively seeking ways to promote visibility of the project and its accomplishments at national level and for promoting sharing and scale-up of project strategies. 5. The project should continue capacity-building activities as planned based on institutional capacity assessment, for project partners, and adhere to a regular meeting schedule that results in planned action. Quarterly coordination meetings among project partners are an effective forum for ensuring good coordination, sharing information, facilitating problem solving and providing capacity building assistance. Priorities for capacity building for project staff are facilitative supervision, conflict negotiation. Recommendations for Social and Behavioral Change 1. Refresher training for HHPs should continue as planned and include additional emphasis on the following topic for child health: recommended infant and young child feeding practices. Refresher training specifically on the benefits of colostrum (besides the promotion of early initiation of exclusive breastfeeding) should be included. Increased focus should be on the complementary feeding of children age 6 to 23 months. It is suggested training be based on the Essential Nutrition Actions for complementary feeding of children age 6 to 23 months, and that training materials on this topic obtained by MTI HQ from Freedom From Hunger be used. 2. HHPs should be trained in rational drug use to provide follow-up for correct drug usage by community members referred to health facilities. HHPs and mothers should receive additional training in differentiating between common child illness signs and symptoms which indicate a need for referral to health facility. 3. HHPs and mothers should receive additional emphasis for the promotion of some elements of maternal care – specifically the promotion of tetanus toxoid and understanding of IPTp and its benefits. 4. To take some of the first steps in reducing the high level of stigma found in midterm LQAS regarding HIV, HHPs and CHCs should be oriented to basic prevention and reduction of stigma messages. The Facilitator Guide from Freedom from Hunger Facing AIDS Together is an excellent training resource on HIV and AIDS. Grand Cape Mount Child Survival Project Midterm Evaluation, Medical Teams International Liberia 37 Recommendations for Community Mobilization 1. The Community Health Committees have recently been established. To be effective, they need support in developing work plans and establishing links with the CHDCs. CHCs (including HHP members) should receive training as planned, with the addition of a module on basic conflict negotiation. It is suggested that CHAL might have useful materials for this topic. 2. The project should take a more active role in building the skills of the CHDCs formed by the County Health Team, with training in problem analysis and action planning, along with basic conflict negotiation. It is suggested training activities should be coordinated among NGOs providing support to the CHT, so that the same methods and materials are used as determined by the CHT. The status of CHDCs should be (informally) assessed at end of Year 3, at which time the project should look for ways to promote links between HHPs, CHCs and CHDCs. Recommendations for Health Services Strengthening 1. IMCI mentoring activities should include additional focus on strengthening health staff skills in completing all assessment tasks of the sick child and in counseling caregivers on the usage of drugs prescribed. Project support for pictorial drug dosage labels should be provided. 2. Health facility staff not trained in IMCI or scheduled for training in Year 3, including support staff, should be updated on individual protocols for common childhood illness and oriented to the basic IMCI framework. 3. It is not recommended that the project use the COPE tool (Client Oriented Provider Efficient) as noted in the DIP, as the R-HFSA tool is an effective replacement. 4. It is not recommended that the project take on full-scale capacity building of health staff skills in growth monitoring and counseling at this time, as it is not prioritized in the national plan for the Basic Package of Health Services. It is recommended that IMCI mentoring activities continue including a focus on strengthening staff skills to weigh and record child’s weight correctly. H. Action Plan The Action Plan was developed in a participatory manner as the final stage of participatory midterm evaluation, with a strong focus on and response to key recommendations. “Suggestions”, as noted in Section G, were discussed verbally for consideration during the implementation process. Intermediate Result 1: Communities assume responsibility for their own health with strengthened community structures, linkages with Health Facility staff, and enhanced human resource capacity. Recommendation Activity Link 1. Continue strengthening CHCs and improve links with HHPs. •Establish a system for HHP “reporting” to CHCs on their activities while continuing to promote CHC support for HHPs. CHCs – HHPs Grand Cape Mount Child Survival Project Midterm Evaluation, Medical Teams International Liberia 38 Intermediate Result 2: Improved health behaviors and actions at the household level. Recommendation Activity Link 1. Strengthen HHP technical capacity in recognizing the difference between cough and cold and pneumonia (danger signs) and when health facility services are needed. •Refresher training for HHPs with home visits to see children with danger signs. •Train on home management of ordinary cough. CHP – HHP – Mothers 2. Promote recommended Infant and Young Child Feeding Practices. •Refresher on exclusive breastfeeding and the benefits of colostrum. •Refresh and emphasize complementary feeding practices (Essential Nutrition Actions) using locally available food, stressing the benefits to child. CHP – HHP – Mothers and fathers; Training manual, Freedom from Hunger 3. Promotion of (basic messages) for maternal and newborn health •Refresher on key messages: Ante-natal care, post-natal care, nutrition for pregnant women, self care, danger signs in newborn. •Additional training to emphasize benefits: IPT, TT. CHP – HHP – Mothers – Fathers CHCs for ER Transport System 4. Promotion of HIV/AIDS awareness. •Educate mothers on prevention, community on stigma; promote use of PMTCT if services become available. • CHP – HHPs – Mothers and fathers 5. Conflict resolution •Train HHPs on how to resolve conflict at household level and/or community level for benefits of peace within the community, bringing development and healing. CHAL Trauma Healing program shared with CSP staff – HHPs – CHC/CHDCs 6. Rational drug use: Training of Trainers (TOT) •One to one counseling on drug use (frequency, quantity, etc. of drug use and benefits) CSP staff – HHPs – Mothers and fathers 7. Rational drug use: Black baggers •Orientation on CSP interventions for black baggers and dangers of providing uncontrolled drugs. •Teach mothers on dangers of going to black baggers. CHCs / CHDCs Grand Cape Mount Child Survival Project Midterm Evaluation, Medical Teams International Liberia 39 Intermediate Result 3: Improved quality of care in health facilities through the implementation of IMCI and capacity building in complementary areas. Recommendation Activities Link Conduct routine refresher for health workers at health facility Coordinate with CHT. Conduct official IMCI training Coordinate with central MOHSW for technical training 1. Strengthen the use of IMCI protocol at health facilities. Provide training for health facility support staff in IMCI protocol relevant to their role. Discuss idea at Monthly Health Workers Association meeting and make quarterly plan Conduct refresher training for health workers on one-to-one counseling on the use of drugs by mothers and fathers. Monthly IMCI mentoring activities. 2. Strengthen understanding of the proper use of drugs by care￾seekers. Conduct CSP staff TOT on one-to-one counseling techniques. Link to activities of other NGOs with one-on-one counseling program Ensure discussion and action planning for these issues at regular coordination meetings among health partners at the GCMC level. •Link to monthly GCMC Monthly Health Coordination Meeting • MTI in coordination with CHT and other NGOs 3. To foster joint problem solving of health care challenges at GCMC level (e.g. incentives, drug supply, logistics of higher level referral, and retention. Promote discussion of difficult issues at national level inter￾agency health coordination meeting Link to national level meetings Annex 1 Medical Teams International in Liberia, CSP22 Midterm Evaluation, Results Highlight Promising Practice: The health system in Liberia has been devastated by years of conflict. MTI has chosen to use a cost-effective strategy to jumpstart improvements in child health in this post￾conflict environment. Partnering with a local NGO has been a cost-effective strategy for implementation of the somewhat staff-intensive Care Group model, which has resulted in rapid improvements in key health practices in Grand Cape Mount County. Project community outreach Supervisors work with Community Health Promoters (CHPs) who are paid staff selected from the local area. CHPs, in turn, provide cascade training for more than 500 female Household Health Promoters who work together in Care Groups. The Care Groups meet monthly in their communities to receive support and in-service training. The Household Health Promoters have been trained in C-IMCI and conduct home visits to counsel mothers and other caretakers. These Household Health Promoters also use a referral system -- a pictorial system designed for an area with high illiteracy, more than 90% – to refer children and women for preventive and curative care. LQAS results of behavior change among child caretakers at midterm showed: 8 of 14 Project Monitoring and Evaluation Matrix indicators EXCEEDED midterm targets while an additional 3 have met targets. These include indicators in each of the project intervention areas: immunization, recommended breastfeeding practices, diarrhea prevention and management, care-seeking for pneumonia, and malaria. Of special interest is a reduction in child undernutrition, from 27.1% to 20.8% as measured by weight-for-age. This likely represents a cumulative effect from multiple improvements in health and nutrition practices as targets for early initiation of breastfeeding, complete immunization, appropriate hand washing practices and care-seeking for pneumonia and malaria have been exceeded. World Relief, who has pioneered the Care Group model, has had success with the model in non￾transitional areas, also. But it appears particularly suited for post-conflict settings which are transitioning from relief to development, resulting in quick improvements in child health that are perceived by community participants and project partners. Best Practice: Medical Teams International’s Child Survival Project in Liberia has an exceptionally strong monitoring and evaluation system. In addition to the KPC survey at baseline, the project has conducted Lot Quality Assurance Sampling annually. This has enabled the project to fine-tune activities to achieve impressive results by the two-year midterm stage. The project is also using the CSTS+ tool for Rapid-Health Facility Services Assessment to measure the results of project strategies for formal IMCI training of health professionals, which is followed by intensive IMCI mentoring visits for supportive supervision. Use of the R-HFSA at midterm evaluation has both shown the impressive improvements which have been achieved in a short period of time, while also assisting the project to identify specific areas for further strengthening. Medical Teams International has specifically hired a Monitoring and Evaluation Specialist to provide training and backstop support to strengthen its own, as well as partners’, capacities in monitoring and evaluation. These are skills that will not only serve this project and its beneficiaries but any future health activities of MTI and/or partners. As the Liberian health system is rebuilt and planning is decentralized to the county level, the Grand Cape Mount County Health Team’s monitoring and evaluation experience in partnership with MTI contributes to the strengthening of the national health system. Annex 2 Medical Teams International: CSP22 Midterm Evaluation, Project Management Evaluation a. Planning: The project’s proposal preparation process and development of the Detailed Implementation Plan were participatory processes that involved the Ministry of Health and Social Welfare, along with the Grand Cape Mount County Health Team, and the local NGO partner, the Christian Health Association of Liberia. CHAL project staff share an office with MTI project staff, and this office has been moved to Sinje town, in the target area during the last quarter of 2006. Partners do annual planning together; however, regular quarterly meetings at the executive level would be useful as the project moves towards considering exit strategies for sustainability at project end. The DIP process was intensive and appears to have worked well for the project overall. Due to limitations of time, the BEHAVE model was used to develop a first draft of a behavior change communication strategy. After project start-up, MTI HQ staff came to assist project staff in conducting Doer/Non-doer analysis to better refine and complete a behavior change strategy. The DIP also looked at using the COPE (Client Oriented Provide Efficient) method for self￾assessment of health facilities; however, MTI’s involvement in collaborating with the development of and use of the CSTS+ tool for Rapid-Health Facility Services Assessment superseded any plans for using COPE. b. Supervision: With technical assistance from MTI HQ CS Advisor, Capacity Building Advisor, and Monitoring and Evaluation Specialist, the project has established a strong system for supportive supervision and monitoring and evaluation. The project Training Coordinator, IMCI Coordinator, IMCI Mentor and CHAL Community Outreach Coordinator, along with Community Health Promoters, are now skilled in using supportive supervision tools (Quality Improvement Verification Checklist, adapted from tools from Freedom From Hunger, Food for the Hungry and CSTS+ R￾HFSA) to assess project staff and health workers’ skills and provide useful feedback in a mentoring fashion. HQ staff not only provides training and orientation to staff in the use of supervision tools, but also randomly samples supervision reports to use as examples for follow-up. c. Human Resources and Staff Management: Medical Teams International has well developed policies and procedures for human resource management and MTI Liberia staff have received training and backstop support visits to ensure these are in place. There was significant staff turnover by the partner agency, CHAL, in response to concerns expressed by MTI. This resulted in the selection of a new Community Outreach Coordinator and 3 of 4 new Community Outreach Supervisors. This new staffing appears to have contributed greatly to improved performance, good morale and strong cohesion among MTI and CHAL team members at midterm. During MTE interviews, all staff refer casually to providing additional support to other staff at any time needed as being a basic part of their overall duties, and state that they too receive support when needed. Staff are well oriented to the project’s goals and appear motivated to effect change in their communities. The CHAL Community Health Outreach Coordinator has benefited from capacity building efforts in technical areas such as adult learning, behavior change and community mobilization. It is suggested that CHAL find ways to build her capacity as a manager, for example, training in how to conduct performance appraisals and assisting staff to develop work plans. 2 The Project Manager had been an excellent resource for the project – with a strong resume, skills as an IMCI master trainer selected by WHO for training as course director, and with an MPH from the University of North Carolina; however, he accepted in June 2008 the position of head of the MOHSW Family Health Department. As such, he will continue to be a good advocate for the project’s child survival strategies. The MTI Liberia Country Director has also played an active role in attending coordinating and advocating with the MoHSW, other health INGOs and donors. For the past 3-1/2 months, the staff have been able to proceed as planned, as their skills enabled them to function independently in their positions. A new Administration and Finance Manager for the MTI country office will begin duties in November 2008 and it is believed that he will be able to provide sizeable support for logistics and administrative aspects of project management, as a new Project Manager is selected. Recruitment is underway to fill the position of Project Manager. d. Financial Management: In December 2006, shortly after project start-up, the MTI HQ Vice￾President of Finance visited and provided technical support to ensure adequate systems were in place for financial management of this project by MTI Liberia and the sub-grantee partner, CHAL. (Independently, an assessment of NGO capacity for performance-based contracting in Liberia done by John Snow, Inc., in May 2008 found the organization to have financial management systems in place and experience following USAID financial rules and reporting requirements.) This was followed by a visit from the MTI HQ Regional Manager for Africa who assisted the MTI Liberia Country Director to conduct an institutional capacity assessment of MTI and partners. From February to May 2007, the MTI, while serving in Liberia for three months as Acting Country Director the West Africa Program manager worked with staff to improve some procedures by MTI Liberia, including use of time cards, travel advances, and logistics procedures. The MTI HQ VP of Finance visited again in March 2008 to provide refresher training in financial management and to conduct an internal audit of MTI and CHAL. MTI HQ Finance is ultimately responsible for financial reports to the donor and tracks project expenditures vs. budget. At midterm, the Finance Department tracking system shows MTI HQ to have 45% of the total budget remaining, MTI Liberia to have 57%, and CHAL to have 60%. MTE review of this report with MTI HQ CS Advisor did not highlight any areas of concern. Project field staff and partners were encouraged during MTE to be closely attentive to expenses vs. budget in the final period of project implementation, as funds should be used so that the project reaches its end date appropriately as budgeted. The project uses matching funds to provide staff incentives and to procure and provide essential medicines (except those for malaria or tuberculosis) for 5 Health Clinics in Grand Cape Mount County, similar to that done for 19 other Health Clinics by the African Humanitarian Assistance NGO and the International Medical Corps. These have been MOHSW CHT approved procedures for transition during the relief period. MTI has informed the CHT that they will be reducing support for incentives by approximately 50% starting in January 2009 and completing transition of support for incentives fully to the CHT by July 2009. The Country Director is currently discussing a phase-out plan with the CHT for transition of MTI support of medicines and supplies to the CHT in the final two years of the project. It is anticipated that other activities taking place in Liberia to strengthen the national health system will ensure there is no gap in clinic support. e. Logistics: The project staff and partner encounter no major obstacles to logistics other than the anticipated frequent need for repair of aging vehicles and new motorcycles that are used on rough roads. MTI Liberia maintains mechanics on staff. A new Administration and Finance Manager will arrive during November 2008 and it is believe that this person will be able to train the two 3 administrative and logistics support staff (one in Monrovia and one in the project office in GCMC) to further facilitate efficient procedures. f. Information Management: With technical assistance from MTI HQ CS Advisor and M&E Specialist, the project has established a strong system for supportive supervision and monitoring and evaluation. Staff is now skilled in using supportive supervision tools (Quality Improvement Verification Checklist, adapted from tools from Freedom From Hunger, Food for the Hungry and CSTS+ R-HFSA) to assess project staff and health workers’ skills and provide useful feedback in a mentoring fashion. Project staff, headed by the project M&E Coordinator, and project partners -- CHAL and the County Health Team -- have been trained by the M&E Specialist and with his assistance are conducting annual LQAS. Results are then used to fine-tune training activities. With training and assistance from the M&E Specialist, the project and CHT partners conducted R-HFSA at baseline and again at midterm (while using part of this tool within the QIVC system for on-going visits to health facilities), and will repeat in Year 3 and at final evaluation. Midterm results have shown the soundness of project strategy with measurable achievements at midterm, while also highlighting specific skill areas (counseling) that need new strategies for improvement. MTE was participatory with CHAL and CHT partners and action planning was based on LQAS, R￾HFSA and qualitative results. Project staff will repeat an activity done at baseline and share some of the key midterm results for discussion at community level with HHPs, local leaders and Community Health Committees, along with local health facility staff. The health system in Liberia is undergoing rapid change and improvements. The project has not attempted to contribute to the CHT Health Information System other than to mentor and support health facility staff reporting skills. During the second half of the project, staff will discuss with CHT partners opportunities for the HIS to capture information on HHP referrals. It is recommended that the project develop a plan for greater dissemination of project achievements as Year 3 progresses, so that these activities can occur throughout Year 4. g. Technical and Administrative Support: Several MTI HQ and regional staff for West Africa have provided frequent, appropriate and timely technical assistance to this project. A summary is provided below in the table Technical Support Provided during Years 1 and 2. At midterm, the project is not lacking any specific assistance. The USAID health team for Liberia also provides information regarding opportunities for building partner capacity, such as a workshop on logistics attended by the Grand Cape Mount County Health Team pharmacist. Technical Support Provided during Years 1 and 2 Year One: Technical Support Provided Staff and Partners Trained Consultant/ Staff providing technical support Month Training and supervision of baseline KPC survey, providing community feedback on the KPC and training in health facility assessment 33 MTI, County Health Team and CHAL staff MTI Monitoring and Evaluation and Technical Services Specialist November and December 2006 4 Year One: Technical Support Provided Staff and Partners Trained Consultant/ Staff providing technical support Month Technical support for setting up Care Groups and training in community mobilization 31 CHT, CHAL, HF and MTI staff Marion Subah, Liberian consultant for community based maternal child health programs November – December 2006 Finance training for MTI Liberia and CHAL MTI and CHAL Accountants, MTI Country Director and Office Manager MTI Vice President of Finance December 2006 Assisted MTI Liberia Country Director to conduct organizational capacity assessment for CHAL and provided orientation on human resource needs and updating job descriptions MTI Liberia Country Director MTI HQ Regional manager for Africa December 2006 Analysis of baseline data, DIP design, training in BEHAVE framework and CSSA 28 CHT, CHAL, HF and MTI staff Renee Charleston, consultant January 2007 Management support as Acting Country Director to improve procedures for time cards, cash travel advances, vouchers, basic procedures and logistics. MTI Liberia finance and logistics staff MTI West Africa Program Manager February to May 2007 Training in LQAS and set up of HMIS 30 MTI and CHAL staff MTI HQ Monitoring and Evaluation Specialist and Technical Services Specialist April – May 2007 Training in Project Cycle Management 19 MTI, CHT and CHAL staff MTI HQ Technical Services Specialist and Director of Regional Programs May 2007 Training in adult education and development of C-IMCI manual 11 MTI and CHAL staff MTI Capacity Building Advisor May 2007 Training for IMCI course directors Project Manager WHO and MOH Ghana August 2007 Training in BEHAVE framework, refine BCC strategy and elicit feedback from MoHSW officials responsible for BCC activities, training on C-IMCI flip charts 30 MTI, CHAL and CHT staff MTI Director of Technical Services and Child Survival Advisor September to October 2007 5 Year Two: Technical Support Provided Staff and Partners Trained Consultant/ Staff providing technical support Month Update Capacity Assessment for CHAL CHAL Executive Director and MTI Liberia Acting Country Director MTI HQ Regional Manager for Africa October 2007 Assistance to improve MTI supply management system for clinic support MTI Liberia Country Director MTI West Africa Program Manager November/ December 2007 Technical assistance reviewing and revising C-IMCI flip charts Project Manager and CHAL Community Outreach Coordinator MTI HQ Capacity Building and Child Survival Advisors February 2008 Finance training provided for MTI Liberia and CHAL and internal audit performed. MTI and CHAL Accountants, MTI Country Director and Office Manager MTI Vice President of Finance March 2008 Training on Community Mobilization and Participatory Action and Learning tools 25 MTI, CHAL and CHT staff MTI HQ Child Survival Advisor May 2008 Technical Assistance conducting midterm LQAS survey, health facility assessment, and community profiles and mentoring for M&E coordinator MTI Liberia Monitoring and Evaluation Coordinator MTI HQ Monitoring and Evaluation Specialist September 2008 h. Strengthening the Grantee Organization: As one of the project’s Intermediate Results is “strengthened institutional capacity of MTI and partners to implement effective CS activities”, MTI has dedicated sizeable efforts in this area. More information can be found in section C1b Summary Monitoring and Evaluation Table, section 4c Capacity Building Approach, and section D Changes in Grantee Organization Capacity in the MTE report. Medical Teams International, as an Entry/New Partner to the USAID Child Survival Health Grants Program, has strengthened HQ capacity in child survival by hiring a Monitoring and Evaluation Specialist in January 2006, a Capacity Building Advisor in February 2007, and a CS Advisor in August 2007. All three have visited the project within the two years of project operation and provided timely and appropriate support. The M&E Specialist is based in Washington DC and is an active member of the CORE M&E Working Group. The Capacity Building Advisor has a Ph.D. in non-formal education and specializes in training design and adult learning methodologies. The CS Advisor has lengthy experience in child survival, having previously managed child survival and primary health care programs in Tajikistan, Azerbaijan, Burundi and Southern Sudan. Each of these human resources has drawn upon tools developed through the CSHGP community with CORE Group support, including the BEHAVE framework, C-IMCI tools, and LQAS and R-HFSA methodologies. These skills have significantly strengthened the support provided to community health programs implemented by MTI in Indonesia, Sri Lanka, Uganda, Cambodia, Mexico and Guatemala. i. Management Lessons Learned: In the course of the first two years of the CSP, management lessons learned were focused in three key areas: 1) the CSP program coordination meetings; 2) 6 annual review and revision of capacity building plans; and 3) logistics. In year one monthly coordination meetings were established and held regularly. However, near the end of year two meetings became sporadic which resulted in some gaps in communication among all partners to coordinate project efforts. MTI took the lead to close this gap and learned the critical importance of continuity of coordination meetings to keep everyone informed, coordinate planning, and strengthen accountability. Plans for Year 3 & 4 are to reinforce monthly meetings of the CSP committee and CSP partners. The MTI Liberia Country Manager will lead this initiative. MTI Liberia and partners, CHAL and CHT, completed annual capacity assessments but were delayed in developing capacity building plans. While there was progress in building capacity, a lesson learned is the need for greater effort on areas that need strengthening and more accountability, especially in the areas of administration and human resources. For example, the Community Outreach Coordinator receives minimal guidance and mentoring by CHAL leadership. MTI, CHT and CHAL have identified priority areas for capacity building based on the assessments and will develop capacity building plans by the end of 2008 for Year 3 and 4, to be reviewed and revised annually. Additionally, MTI Liberia will work more closely with partners to monitor their progress with the MTI Liberia Country Director taking the lead. Logistical support was established early in the project but the project experienced some gaps, for instance coordination of vehicles, support in procurement of goods or services, or constraints of roads, which at times created obstacles for staff in carrying out project activities. MTI, CHAL, and the CHT will plan steps to strengthen logistical support through staff training in procurement and transportation in the first quarter of year 3. MTI Liberia’s new Finance and Administration Manager will take the lead in this initiative. j. Other Issues Identified by the Team: No other issues were identified. 1 Annex 3 Medical Teams International in Liberia CSP22 Midterm Evaluation, Monitoring and Evaluation Matrix Intermediate Result 1: Communities assume responsibility for their own health with strengthened community structures, linkages with Health Facility staff, and enhanced human resource capacity Indicator Method BL MTE Result Status at MTE Final Target % of HHPs referring patients to clinic. HHP Supervision ** 0% 100% Target exceeded. The referral system was initiated in the 2nd Qtr of FY2 and is in use by all 520 HHPs. Project CHPs tabulate monthly referrals; the project will look at ways to record referrals received by health facilities. 75% % of HHPs who received a supervisory visit during the last three months HHP Supervision 0% 100% Target exceeded. All HHPs receive regular internal supervision visits by project staff, with use of a QIVC form for feedback. For final evaluation of this indicator should reflect HHP linkage and reporting to CHCs or to health facilities. 75% % of communities using information from community HIS for decision making Community Profile ** 0% 0% Not met. To begin in Year 3 providing feedback from Care Group Report. 40% % of CHCs (changed from CHDCs as formation done by CHT without MTI participation) with one or more women participating on the committee. CHT information 0% ~75% Target exceeded. Super-vision areas 1 and 2 report an average of 1 female among the 5 members on each CHC, area 3 reports an average of 2 females, while area 4 reports no female CHC members. 65% % of health facilities with active CHDCs who have met in the last three months. Community Profile ** 0% 53% On track to meet target. CHT reports 16 of 30 health facilities have formed CHDCs but few have prepared action plans. MTI CSP will assist with problem analysis and action planning in Years 3 and 4. 80% % of communities with an economic plan for emergency health needs. Community Profile ** 0% 13% 60% % of communities with an emergency transport plan. Community Profile ** 0% 16% On track to meet target. CHCs have recently been re-activated. Among 4 CHCs visited during qualitative MTE, all had future plans for small income generation to support the cost of emergency transport by public vehicles, and most could cite at least one recent instance of assistance provided for emergency transport. 65% * Indicators marked ** are also part of the project’s use of the Child Survival Sustainability Assessment (CSSA) tool. More information in section C4h.Sustainability 2 Intermediate Result 2: Improved health behaviors and actions at the household level. Indicator Method Baseline MTE Target MTE Result Status at MTE Final Target % of newborns who were put to the breast within one hour of delivery and did not receive pre-lacteal feeds KPC Survey: BL,FE LQAS: MTE ** 33.7% (28.3-39.3) 42% 51.0% (41.0-61.0) Exceeded 50% New target: 60% % of infants 6-9 months receiving breast milk and complementary foods KPC Survey: BL,FE LQAS: MTE ** 37.5% (22.7-54.2) 51% 40.6% (30.8-50.5) Not met 65% % of children 0-23 months who are underweight (<-2 SD weight-for-age WHO/NCHS reference standards). KPC Survey: BL,FE LQAS: MTE 27.1% (22.1-32.6) 22% 20.8% (12.7-28.9) Exceeded 17% % of children 6-23 months who received a dose of Vitamin A in the last six months (Mother’s recall). KPC Survey: BL,FE LQAS: MTE 76% (67.8-83.3) 81% 85.4% Met 85% % of children 12-23 months who received DPT3 before they reached 12 months by the time of the interview as recalled by the mother or card verified. KPC Survey: BL,FE LQAS: MTE 31% (22.2-40.1) 41% 60.4% Exceeded 50% New target: 65% % of children aged 12-23 months who are fully vaccinated by 12 months of age. KPC Survey: BL,FE LQAS: MTE 18.9% (12.1-27.5) 30% 39.5% (29.8-49.4) Exceeded 40% New target: 55% % of mothers with children age 0-23 months who were protected against Tetanus (at least 2 TT) before the birth of the youngest child. KPC Survey: BL,FE LQAS: MTE ** 61.3% (55.6-66.9) 71% 66.7% (57.2-76.1) Not met 80% % children 0-23 months with diarrhea in last 2 weeks who received ORS and/or recommended home fluids. KPC Survey: BL,FE LQAS: MTE ** 74.2% (63.8-82.9) 80% 79.2% (71.2-88.7) Met 85% % of children 0-23 months with diarrhea in the last two weeks who were offered more fluids during the illness KPC Survey: BL,FE LQAS: MTE ** 52% (40.8-62.4) 61% 64.5% Met 70% % of households of children 0-23 months that treat water effectively. KPC Survey: BL,FE LQAS: MTE ** 22% (17.1-26.8) 31% 7.3% Not met 40% % mothers of children 0-23 months who live in house-holds with soap or ash at the place for hand washing and washed their hands with soap or ash at least two of the appropriate times during a 24 hour recall period. KPC Survey: BL,FE LQAS: MTE ** 19.0% (14.7-23.9) 30% 43.7% (33.8-53.7) Exceeded 40% New target: 70% % of children age 0-23 months with chest-related cough and fast/difficult breathing in the last two weeks who were taken to an appropriate health provider. KPC Survey: BL,FE LQAS: MTE 43.2% (35.3-51.4) 54% 83.3% (75.9-90.8) Exceeded 65% New target: 85% % of children 0-23 months who slept under an insecticide￾treated bed net the previous night. KPC Survey: BL,FE LQAS: MTE ** 17.7% (13.5-22.5) 27% 67.7% (58.4-77.1) Exceeded 35% New target: 70% % of children 0-23 months with a febrile episode that ended during the last 2 weeks who were treated with an effective anti￾malarial within 24 hrs after fever began. KPC Survey: BL,FE LQAS: MTE 12.5%13 (5.9-17.7) 32% 49.0% (39.0-59.0) Exceeded 50% New target: 65% Baseline value recalculated at midterm using correct answer criteria. 3 Intermediate Result 3: Improved quality of care in health facilities through the implementation of IMCI and capacity building in complementary areas. Indicator Method BL MTE Result Status at MTE Final Target % of HF that offer growth monitoring (at least 30 days per month per R-HFSA). R-HSPA BL & FE 0% 44% Improved. Most HF in project area only offer services 5 days/week at present. 85% % of HFs clinical encounters in which all assessment tasks are made by the HW for sick child (check ability to drink or breastfeed, vomits everything, convulsions, presence of cough or fast/difficult breathing, diarrhea, fever, assess nutritional status, feeding practices, check vaccination status) R-HSPA BL & FE 9% 28% (66% on average) Improved. Additional efforts beyond that planned are recommended as needed if final target is to be reached. 85% % of HF clinical encounters in which treatment is appropriate to diagnosis for malaria, pneumonia and diarrhea. (Record review) R-HSPA BL & FE 46% 89% Final target met. Continued IMCI mentoring will be needed to maintain result as staff turnover occurs. 85% % of HF clinical encounters in which the caretaker whose child was prescribed antibiotic, anti-malarial or ORS can correctly describe how to administer all prescribed drugs R-HSPA BL & FE 49% 34% No improvement. This is the weakest area of IMCI at MTE. Project will add activities to promote knowledge of rational drug use with training for HF staff and HHPs for follow-up in the community. 75% % of HF that received external supervision at least once in the last THREE months (two or more of: deliver supplies, check records/reports, observe work, provide feedback) R-HSPA BL & FE 5% 56% Improved. Indicator is on track to reach final target. 75% % of HF utilizing information from the HMIS for decision making R-HSPA BL & FE ** NA 44% Improved. Indicator is on track to reach final target. 75% 4 Intermediate Result 4: Strengthened institutional capacity of MTI and partners to implement effective CS activities. Indicator Method BL MTE Result Status at MTE Final Target % of organizations with a functional financial management system IA BL & FE 33% n/a Indicator dropped as not relevant. Partners have functional systems. n/a % organizations (CDHCs) using information from HMIS for decision making IA BL & FE ** 0% 0% CHDCs recently formed in 16 of 30 health facilities. 100% % of organizations and health facilities meeting approved Standard Health Plan. CHT report 0% 34% Indicator on track to reach final target. 50% % of indicators for capacity building are achieved IA BL & FE 0% Action taken to improve 36% of indicators for all 3 partner agencies: MTI Liberia, CHAL, and GCM CHT. All key systems are in place with finali￾zation of an emergency preparedness plan as a priority for MTI. Review and strengthening of logistics in Qtrs 1 and 2 of Y3 is a priority for all three agencies (MTI, CHAL, CHT). 80% Use of CSSA is institutionalized (annual reviews are being conducted) Annual Report No Review at midterm conducted. Midterm review by MTI and partners refined the CSSA plan prepared during the DIP and established baseline values Yes Lessons learned and best practices are disseminated utilizing at least three different media (program manual, presentations, conferences, web site, articles, etc.) Final Evaluation No In process. MTI is preparing a discussion paper on best practices and lessons learned in health programs in post-conflict areas. Internal presentations of MTE results and discussions of lessons learned will be held at MTI headquarters and field offices. Midterm evaluation results will posted on the MTI website. Lessons learned from the Liberia Child Survival Project are disseminated and used in MTI community health programs in Africa and Asia. It is recommended that in Y3 that MTI and the CSP in Liberia prepare a plan for dissemination of best practices both in-country and CORE Group Y4. Yes % of annual national budget dedicated to health. Final Evaluation n/a n/a This indicator has been dropped as the project has limited ability to influence and measure influence of this indicator. n/a 1 Annex 4 Rapid CATCH Table, Midterm Evaluation Results, October 2008 CSHGP Intervention Area Rapid CATCH Indicator Baseline Midterm (1) Maternal TT Vaccination: Percentage of mothers with children age 0-23 months who were protected against Tetanus before the birth of the youngest child. (Protected refers to receiving at least 2 TT or Td injections before the birth of the youngest child sufficiently close to that birth to provide protection.) 61.3% (55.6%-66.9%) 66.67% (57.24%, 76.1%) (2) Skilled Delivery Assistance: Percentage of children age 0-23 months whose births were attended by skilled personnel 21.3% (16.8%-26.4%) 23.96% (15.42%, 32.5%) (3) Post Partum visit to check on mother within the first 3 days after birth: Percent of mothers of children 0-23 months who received a post-partum visit by an appropriate trained health worker within three days after the birth of the youngest child. 6.0% (3.6%-9.3%) 28.13% (19.14%, 37.12%) Maternal Newborn Care (4) Post Partum visit to check on newborn within the first 3 days after birth: Percent of children 0-23 months who received a post -partum visit by an appropriate trained health worker within three days after birth 7.0% (4.4%, 10.5%) 36.46% (26.83%, 46.09) (5) Immediate and exclusive breastfeeding of newborns: Percent of newborns who were put to the breast within one hour of delivery and did not receive prelactal feeds Breastfeeding 33.7% (28.3%-39.3%) 51.04% (41.0%, 61.0%) (6) Exclusive breastfeeding: Percentage of children 0-5 months who were exclusively breastfed during the last 24 hours 86.0% (78.5%-91.6%) NA Vitamin A Supplementation (7) Vitamin A Supplementation in the last 6 months: Percentage of children age 6-23 months who received a dose of Vitamin A in the last 6 months (Mother’s recall). 76.2% (67.8%-83.3%) 85.42% (78.36%, 92.48%) (8) Access to immunization services: Percentage of children 12-23 months who received DPT1 before they reached g12 months (card verified 46.8% (37.3%-56.6%) 86.46% (79.62%, 93.3%) (9) Health System Performance regarding Immunization services: Percentage of children12-23 months who received DPT3 before they reached 12 months (card verified). 30.6% (22.2%-40.1%) 60.42% (50.64, 70.2%) Immunization (10) Measles vaccination: Percentage of children 12-23 months who received a measles vaccination by the time of the survey (card verified) 42.3% (33.0%-52.1%) 50.00% (40.00%, 60.00%) (11) Child sleeps under an insecticide-treated bednet: Percentage of children 0-23 months who slept under an insecticide-treated bed net (in malaria risk areas, where bed net use is effective) the previous night. Malaria 17.7% (13.5%-22.5%) 67.71% (58.36%, 77.06%) (12) Child with fever receives appropriate antimalarial treatment: Percentage of children 0-23 months with a febrile episode that ended during the last two weeks who were treated with an effective anti-malarial drug within 24 hours after the fever began. 12.5% (5.9%, 17.7%) 48.96% (38.96%, 58.96%) 2 (13) ORT use: Percentage of children 0-23 months with diarrhea in the last two weeks who received Oral Rehydration Solution (ORS) and/or recommended home fluids. Control of Diarrhea 74.2% (63.8%-82.9%) 79.17% (71.22%, 88.72%) (14) Zinc: Percentage of children 0-23 months with diarrhea in the last two weeks who were treated with zinc supplements. 0% (0.0%-4.1%) NA ARI/Pneumonia (15) Appropriate Care Seeking for Pneumonia: Percentage of children age 0-23 months with chest-related cough and fast/difficult breathing in the last two weeks who were taken to an appropriate health provider. 43.2% (35.3%-51.4%) 83.33% (75.87%, 90.79%) (16) Point of Use (POU): Percentage of households of children 0-23 months that treat water effectively. Water and 21.7% (17.1%-26.8%) 7.29% (2.09%, 12.49% Sanitation (17) Appropriate Hand washing Practices: Percentage of mothers of children 0-23 months who live in households with soap at the place for hand washing that washed their hands with soap at least 2 of the appropriate times during a 24 hour recall period. 19.0% (14.7%-23.9%) 43.75% (33.83%, 53.67%) Child Spacing (18) Birth Spacing: Percentage of children age 0-23 months who were born at least 24 months after the previous surviving child. 76.0% (62.0%-90.0%) NA (19) Underweight: Percentage of children 0-23 months who are underweight (-2 SD for the median weight for age, according to WHO/HCHS reference population) Infant and Young 27.1% (22.1%-32.6%) 20.83% (12.71%, 28.95%) Child Feeding and Anthropometrics (20) Infant and Young Child Feeding: Percent of infants and young children aged 6- 23 months fed according to a minimum of appropriate feeding practices. 18.9% (10.2%-27.6%) NA 3 Additional Key Indicator collected Intervention Area Indicator Baseline Midterm (1) Possession of a child vaccination card or health booklet – Ever had: Percent of mothers of children 0-23 months who were ever given a vaccination card of health book for their youngest child 0-23 months. 77.3% (72.2%-81.9%) 94.79% (90.34%, 99.24%) (2) Possession of a child vaccination card or health booklet – Currently have: Percent of children 0-23 months who currently have a vaccination card of health book. 62.0% (56.2%-67.5%) 86.25% (79.45%, 93.23%) (3) Antigen and dose specific coverage: See below BCG 48.6% (39.0%-58.3%) 83.50% (76.07%, 90.93%) Polio1 47.7% (38.2%-57.4%) 84.36% (77.1%, 91.62%)) Polio2 39.6% (30.5%-49.4%) 79.17% (71.05%, 87.29%) Polio3 36.0% (27.1%-45.7%) 65.63% (56.13%, 75.13%)) DPT1 46.8% (37.3%-56.6%) 86.04% (79.11%, 92.97%) DPT2 38.7% (29.6%-48.5%) 61.46% (51.72%, 71.20%) DPT3 30.6% (22.2%-40.1%) 60.42% (50.64%, 70.2%) 3.1: Card verified by 12 months Measles 25.2% (17.5%-34.4%) 50.00% (40.00%, 60.00%) BCG 54.1% (44.3%-63.6%) 86.46% (79.62%, 93.3%) Polio1 60.3% (69.6%-85.6%) 86.46% (79.62%, 93.3%) Polio2 47.7% (38.2%-57.4%) 81.25% (73.44%, 89.06%) Polio3 46.8% (37.3%-56.6%) 77.08% (68.67%, 85.49%) DPT1 53.2% (43.4%-62.7%) 88.54% (82.17%, 94.91%) DPT2 45.9% (36.4%-55.7%) 87.50% (80.88%, 94.12%) DPT3 41.4% (32.2%-51.2%) 86.46% (79.62%, 93.3%) Immunization 3.2: Card verified by the time of the survey, 12-23 months Measles 42.3% (33.0%-52.1%) 69.79% (60.6%, 78.98%) (1) Increased fluid intake during diarrheal episode: Percentage of children 0-23 months with diarrhea in the last two weeks who were offered more fluids during the illness 51.7% (40.8%-62.4%) 64.58% (55.01%, 74.15%) (2) Continued feeding during a diarrheal episode: Percentage of children 0-23 months with diarrhea in the last two weeks who were offered the same amount or more food during the illness 18.2% (9.8%-29.6%) NA (3) Care Seeking for Diarrhea: Percent of children 0-23 months with diarrhea in the past two weeks whose mothers sought appropriate outside advice or treatment for the illness 28.18% (19.1%-38.6%) NA Control of Diarrhea (4) Use of Medicine during diarrhea: Percentage of children 0-23 months with diarrhea in the last two weeks who were not treated with anti-diarrheals or antibiotics. 91.0% (83.1%-96.0%) NA Annex 5 Grand Cape Mount Child Survival Project Improved Child Health in a Transitional State through IMCI Grand Cape Mount County, Liberia October 2006 – September 2010 In Partnership with Liberia Ministry of Health and Social Welfare Grand Cape Mount County Health Team Christian Health Association of Liberia LQAS Survey Grand Cape Mount County Liberia September 2008 1 Table of Contents Acronyms 2 Executive Summary 3 Chapter 1 Program Overview 7 Chapter 2 Purpose of the Monitoring Survey and Methodology 11 Chapter 3 Main Findings: Accomplishments, Priorities by Supervision area, and for the Program as a Whole 14 Chapter 4 Action Plan/Conclusions/Recommendations 38 Annexes: Annex 1: LQAS Summary Table of Project Matrix Indicators Annex 2: LQAS Summary Table of Client Satisfactions Indicators Annex 3: Additional Indicators Collected during Midterm LQAS 2 ACRONYMS ARI Acute Respiratory Infection ACT Artemisinin based combination therapy BAG Breastfeeding Advocacy Group BCC Behavior Change Communication BL Baseline Assessment CDD Control of Diarrheal Disease CHAL Christian Health Association of Liberia CHP Community Health Promoter CHT County Health Team CHW Community Health Workers C-IMCI Community IMCI CS Child Survival CORE Collaborations and Resources Group CSHGP Child Survival and Health Grant Program CSP Child Survival Project D Precision DPT3 Diphtheria, Pertussis and Tetanus vaccine 3 EBF Exclusive Breastfeeding EPI Expanded Program of Immunizations GCM Grand Cape Mount County GCMCSP Grand Cape Mount County Child Survival Project GIK Gifts-in-kind HF Health Facility HHP Household Health Promoter HQ Headquarters of MTI located in Portland, Oregon IMCI Integrated Management of Childhood Illnesses IPT Intermittent preventive treatment KPC Knowledge, Practice, and Coverage Survey M&E Monitoring and Evaluation MOHSW Liberia Ministry of Health and Social Welfare MTI Medical Teams International MTI/Liberia Medical Teams International/Liberia N Sample size NDS National Drug Service ORS Oral Rehydration Salts P Proportion PCM Pneumonia Case Management PHC Primary Health Care Rapid CATCH Core Assessment Tool on Child Health SA Supervision Area USAID United States Agency for International Development WFA Weight for Age 3 I. Executive Summary In October 2006, Medical Teams International was awarded funding by the United States Agency for International Development in Washington (USAID) for the Grand Cape Mount Child Survival Project: Improved Child Health in a Transitional State through IMCI aimed at reducing child and maternal mortality and morbidity in Grand Cape Mount, Liberia. In September 2008, the Child Survival team carried out a KPC survey using Lot Quality Assurance Sampling (LQAS) as part of a mid term assessment process to assess progress in meeting Project indicators during the first two years of the project, and to investigate other intervention areas that may be added to the project. The team interviewed mothers of children aged 0-23 months. Mothers were asked questions about breastfeeding, complementary feeding, immunization, and treatment and prevention of diarrhea, ARI, and malaria. In addition to questions regarding these areas of the project design, mothers were also asked about antenatal and HIV knowledge and practices, as well as questions pertaining to their satisfaction with the health facilities in the project area. Data entry and analysis were done in Excel. All 4 SAs were analyzed using LQAS decision rules to determine if performance meets the benchmarks chosen for the midterm of the project. New indicators were analyzed to determine their baseline, and to determine the performance of each SA relative to the cumulative combined frequency of all 4 SAs for this indicator, which indicates if any SA is having more difficulty with regard to this indicator than the average. The results obtained from the survey are being used to determine the current situation in the target districts and to revise strategies for the next two years of the project. All 4 SAs met the respective benchmarks for 10 of the 13 project matrix indicators, and the combined frequency for 8 of the 13 indicators has already met or exceeded the final target for that indicator. Some key survey findings include: Breastfeeding and Complementary Feeding: All 4 Supervision Areas were analyzed using LQAS decision rules to determine if performance meets their benchmarks with regard to all of the Project Design Logframe (PDL) indicators for breastfeeding and complimentary feeding. While all 4 SAs met the benchmarks set for these three indicators in year 1, as determined by the LQAS performed in December 2007, all 4 SAs met the benchmark for 1 of the indicators at midterm. The Results are as follows  % of newborns who were put to the breast within one hour of delivery and did not receive prelactal feeds: all 4 SAs met their benchmark, and the combined frequency for this indicator is 51.40%, which continues to surpass the expected target of 50% for the project. A new final target for this indicator was set at 60%  % of infants aged 6-9m receiving breastmilk and complementary foods: while all 4 SAs met their benchmark for this indicator in December 2007, only SA1 and SA3 met their midterm benchmarks, with SA2 and SA4 failing. The combined frequency, 40.63%, did not meet the midterm benchmark of 51% for this indicator. Substantial work will need to be done to meet the final target of 65%.  % of children 6-23m who received a dose of Vitamin A in the last 6 months: The cumulative frequency of 85.42% for this indicator continues to surpass the midterm benchmark (81%) and meet the project target (85%). However, SA1 failed to meet the benchmark for the midpoint of the project.  An additional indicator, the percentage of children age 0-23 months who are underweight, which is not in the Project Matrix but is a Rapid CATCH indicator, was added to the LQAS in order to assess the possible impact of the projects interventions in the area of malnutrition, particularly in the area of health behaviors surrounding immediate and 4 exclusive breastfeeding and continued feeding with the addition of complimentary foods. It was encouraging to find that the combined frequency is 20.83%, which is an improvement from the baseline finding of 27%. This includes the percentage of children age 0-23 months who are severely underweight, which is 10.41%. This compares to the baseline rate of severely underweight children of 13.10%. Immunization: All 4 Supervision Areas met their performance benchmarks for the midterm of the project for the 2 indicators regarding the immunization of children. However, while the indicator for the immunization of pregnant mothers against Tetanus met the chosen benchmark for year one in all 4 SAs, SA1 failed to meet its benchmark for the end of year 2. SAs 2-4 did meet their benchmarks for the midterm:  % of children 12-23 months who received DPT3 before they reached 12 months of age, card verified: the combined frequency of this indicator at midterm is 60.42%, which already surpasses the expected target of 50% for the project.  % of children aged 12-23 months who are fully vaccinated (received BCG, DPT3, OPV3, and measles vaccines) by 12 months of age, card verified: the combined frequency of this indicator is 39.58%, which exceeds the projected benchmark of 30% for the end of the second year of the project, and reaches the project target of 40%.  Percentage of mothers with children aged 0-23 months that were protected against Tetanus before the birth of their youngest child: the combined frequency of this indicator, 66.67%, did not reach the benchmark of 71% for the end of the second year. This is due to the fact that SA1 is well below its benchmark target. Prevention and Treatment of Diarrhea: All 4 Supervision Areas are performing adequately with regard to the 3 diarrhea and hygiene indicators in the project matrix:  % of children 0-23 months with diarrhea in the last two weeks who received Oral Rehydration solution (ORS) and/or recommended home fluids: the combined frequency of this indicator, 79.17%, meets the benchmark for this period (80).  % of children 0-23 months with diarrhea in the last two weeks who were offered more fluids during the illness: the cumulative frequency of 64.58% also exceeds the midterm benchmark of 61%.  % of mothers of children 0-23m who live in households with soap or ash at the place for hand washing and that washed their hands with soap or ash at least 2 of the appropriate times during a 24 hour recall period: the combined frequency, 43.75%, exceeded the benchmark of 30% and the project target of 40% for this period of the project. A new final target was set as 70%.  However, all 4 SAs did not reach the midterm benchmark for the indicator: % of households of children 0-23 months that treat water effectively. Because there has been an actual decline in this indicator at the midterm (7.29%) from baseline (22%), these results represent a decline from the last study at the end of year 1, where only SA2 failed to meet its benchmark The project will investigate the barriers that have caused the use of water treatment at the household level to decline from baseline, despite our programmatic efforts. Community feedback sessions are planned, and an Action plan to deal with this issue will be formulated in collaborative fashion with community members. It has been determined from the additional indicator added to this LQAS to further investigate this issue that the percentage of mothers who live in households with an improved source for drinking water has 5 improved from 72% at baseline to 82.29% at midterm. In addition, when using the combined frequency (83%) as the year 2 benchmark for this indicator, SA3 fails to reach this benchmark, indicating that this area has below average use of improved water sources. Following the community feedback it may be decided to target those communities who do not have improved water sources, as overall the diarrhea rate is low in GCM, but is most likely higher in these areas. ARI: All 4 SAs performed adequately with regard to the Project Matrix Indicator in the intervention area of ARI, receiving care from an appropriate provider:  % of children age 0-23 months with chest-related cough and fast/difficult breathing in the last two weeks who were taken to an appropriate health provider. The cumulative frequency of 83.33% exceeds the benchmark set for this time period of 54%, and the target of 65%. A new target was set at 85%. Malaria: All 4 SAs met their performance benchmark for the use of bednets, with a combined frequency of 67.71%, which far exceeds the project target of 35%. Also, each of the 4 SAs met their performance benchmark for treatment of children with fever with an appropriate antimalarial within 24 hours of the onset of symptoms  % of children 0-23 months who slept under an insecticide-treated bed net the previous night: the cumulative frequency of 67.71% far exceeded both the midterm benchmark of 27% and the target of 35% set for this indicator. This is due to wide distribution by the PMI of ITNs throughout all of the HFAs in GCM during the first quarter of 2008. A new final target was set at 70%.  Percentage of children 0-23 months with a febrile episode that ended during the last two weeks who were treated with an effective anti-malarial drug within 24 hours after the fever began: The cumulative midterm frequency of 48.96% surpasses the midterm benchmark of 32% and reaches the project target of 50%. Additional Indicators Collected: Due to the concerns of the County Health Team, CHAL and MTI staff regarding safe motherhood and HIV/AIDS in the county, the project collected additional information to assess needs in these areas. Prenatal Care  92.71% of mothers of children aged 0-23 m had at least one prenatal visit prior to the birth of her youngest child.  Only 31.25% of mothers have a maternal health card, and only 46.88% of mothers could name at least one prenatal danger sign. Delivery Care  The percentage of women giving birth in a facility is quite low at 25.00%, and the percentage of births that are attended by a skilled birth attendant, which is defined in this study as a doctor, nurse, physician assistant, or midwife, is only 23.96%. 6 Postnatal Care  Postnatal visits within 3 days of the birth of the child are now occurring at a rate of 28.13% for the mother and 36.46% for the child. These rates are markedly better than baseline (6% and 7%, respectively) but still quite low. HIV  While 90.63% of mothers have heard of AIDS, only 37.50% of mothers could name two or more correct methods of reducing the risk of contracting the virus that causes HIV.  Only 37.50% of mothers have heard of Voluntary Counseling and Testing (VCT), and only 26.04% of mothers have the knowledge of where such treatment is available in the community.  Only 13.54% of mothers know where one may receive treatment for HIV, and only 5.21% of mothers know that there is PMTCT treatment available and 8.33% of mothers know where PMTCT treatment may be received  Stigma against those living with HIV is a very real issue in GCM County; 43.75% of mothers believe that PLWHA should be ashamed and 53.13% believe that PLWHA should be forced to leave the community. 7 CHAPTER 1 Program Overview Project Area and Description: Medical Teams International is implementing the Grand Cape Mount County Child Survival Project (GCMCSP) in Grand Cape Mount County (GCM) located in southwestern Liberia. The goal of the GCMCSP is to reduce morbidity and mortality of children under five and women of reproductive age within Grand Cape Mount County, Liberia. The interventions and level of effort are: Nutrition 30%, PCM 20%, CDD 20%, Malaria 20%, and EPI 10%. All interventions are being implemented within the recently adopted IMCI framework for Liberia and in accordance with MOHSW policy. Implementing partners for this project include the Christian Health Association of Liberia (CHAL), a local NGO responsible for implementing the Care Group and community mobilization activities, and the County Health Team (CHT) which is charged with the responsibility of coordinating the delivery of health services throughout the county. Working in partnership with the MOHSW, the CHT, and CHAL, MTI/Liberia is implementing a project that aims to improve the health of village communities in GCM County through strengthening HFs and the MOHSW’s ability to address community health needs. This is accomplished through targeted behavior change at the household level, community mobilization, improving quality of care and access at the clinic level, and institutional capacity building for MTI and partners. Socioeconomic and Health Conditions in the Project Area: Liberia was immersed in a violent civil war for fourteen years, from 1989 to 2003. In 2003, 76% of the population lived on less than $1 per day and 52% lived on less than 50 cents a day. The Human Development Index (HDI) for Liberia in 1999 was .276, meaning that of the 175 countries where the HDI is calculated, Liberia ranked 174, with only Sierra Leone ranking lower. GCM is one of 15 counties in Liberia and is located west of Monrovia along its border with Sierra Leone. It is divided into five districts; Garwula, Tewor, Golah Konneh, Porka, and Commonwealth. The health and social conditions of GCM are relatively worse than the rest of the country. GCM suffered greatly during the civil war, losing not only health infrastructure but also the majority of its health personnel. GCM contains a high percentage of under-represented and vulnerable populations. Nationwide, Liberia has a religious affiliation of 82% Christians and 16% Muslim. GCM has the highest concentration of Muslims in the country-- 90% Muslim and 9% Christian. GCM is 69% Vai, a group which is only represented by 5.6% of the population nationally. 8 The project is being implemented in the five districts of GCM and benefits the entire population of GCM (127,124 people), including direct benefits to 21,612 children under five and 29,239 women of reproductive age. Immunization: Immunization rates are low in GCM County, with full EPI coverage by 12 months of age found to be only 18.9% at baseline in children aged 12-23 months. EPI Health System Performance, measured by card verified rates of DPT3 vaccination by 12 months of age, was only 30.6%. MTI and its partners are utilizing the IMCI and C-IMCI approach in order to improve vaccination rates and utilization of vaccination services by the community. Demand for EPI services is being created through CHPs, and HHPs, including messages to the community about bringing children in for immunization. The project provides logistical support for National Immunization days, advocates for the implementation of routine EPI services, and reduces missed opportunities for immunizations through IMCI. The project is presently addressing gaps in present immunization service delivery, including a lack of training of MOHSW staff and gaps in the cold chain and other equipment at the HF level. In addition, the MOHSW in conjunction with MTI/Liberia project staff is conducting immunization training with HF staff in logistics and drug forecasting, and injection technique and safety. The percentage of mothers at baseline survey receiving at least two immunizations in GCM County was only 61.3%, indicating some utilization of antenatal care services and tetanus immunizations being performed by antenatal health care staff when these services were utilized, but this rate needed improvement to ensure protection of mothers and children from tetanus. The project is addressing this problem by strengthening the MOHSW’s ability to address this need through enhancing the capacity of antenatal care staff and providing consistent preventative-based services that include immunization services. Respiratory Infections: The prevalence of ARI (children with a chest related cough and fast/difficult breathing) in children aged 0-23 months in GCM County was 51.7%. At baseline children with ARI were treated by seeking advice or treatment at a qualified health facility at a rate of only 43.2%. Also, children with ARI were treated by either seeking advice/treatment at a qualified health facility or treated with an antibiotic at a rate of only 49.7%, which revealed that most mothers were not routinely seeking help when their child was sick, and did not understand the importance of seeking timely medical advice or seeking proper antibiotic treatment for their children who are ill with cough and respiratory infection. To address these issues, the project utilizes the Case Management Training on IMCI to improve the knowledge and skills of health workers in the assessment and treatment of infants and young children with respiratory problems Through home visits, mothers and caregivers are being educated in recognizing the signs of ARI including cough and difficulty/fast breathing and the importance of seeking timely medical treatment for their child with ARI. Ensuring that antibiotics are readily available at the HF level is also a priority of the project through the GIK and working along with the CHT and other partners (NGOs) in the county who support other health facilities. 9 Diarrhea: While diarrhea is a major cause of morbidity in Liberia and GCM County, due to seasonal fluctuations the prevalence of diarrhea at the time of the baseline KPC survey was 30.1%, which is somewhat lower than found at other times of the year. Poor hand washing practices, poor water quality and inappropriate home treatment practices contribute to the high rates of diarrhea. While 74.2% of mothers treated their child’s diarrhea with Oral Rehydration Solution and/or a recommended home fluid, in this context usually coconut water or rice water, only 36.0% of mothers could correctly describe the correct preparation of ORS and there was no way to verify the quality of the home fluid being used. Only 51.7% of mothers offered their child more breastmilk or fluid and only 18.2% of mothers offered their child more or the same amount of food when their child had diarrhea. In the baseline KPC survey the percentage of mothers of children 0-23 months who lived in households with soap at the place for hand washing that washed their hands with soap at least 2 of the appropriate times during a 24 hour recall period was only 19.0%. In addition, while 72.0% of households in GCM County had access to an improved water source, only 21.7% of households were treating water effectively at baseline. Also, only 32.3% of households had access to a sanitary latrine. Key home practices that are emphasized by this project include continued breastfeeding and fluids as well as the frequent feeding of small amounts of food, and catch-up feeding. In the Care Groups and in home visits, CHPs, and HHPs are emphasizing these key home practices, proper hygiene including proper hand washing practices, and proper techniques to store and treat water effectively. The project is coordinating with the CHT to ensure a regular supply of ORT at the community level. Malaria: The prevalence of fever in children aged 0-23 months in GCM County was found to be quite high in the baseline survey at 70%. Part of the reason for this can be explained by the fact that at baseline only 17.7% of children less than 24 months were sleeping under an insecticide treated bed net. Of the children who had a fever that had ended in the 2 weeks prior to the survey, only 20.5% were brought to a qualified health facility within 48 hours of the start of the fever. Only 3.6% of mothers treated their child with an effective anti-malarial drug within 24 hours after the fever began. Project activities for malaria are focused on the prevention of malaria with the supporting the distribution of Insecticide Treated Nets (ITNs) provided by the PMI and community education on proper use of these nets through Care Groups and home visits. The project has begun to work with the MOHSW Malaria Control Division, NDS and GIK sources to ensure adequate ACT pharmaceutical availability at HFs in the county and training health care workers in their use. GIK will also supply Fansidar for use in IPT as well as drugs for malaria treatment, where gaps exist, according to MOHSWSW protocols. Nutritional Status: The nutritional status of children aged 0-23 months is of major concern in GCM County. The Baseline KPC Survey conducted in November 2006 found the overall rate of under-nutrition to be 27.1%, with 14.1% moderately underweight and 13.1% severely underweight. The survey also showed low rates of immediate and exclusive breastfeeding. Mothers were immediately and exclusively breastfeeding their newborns at a rate of only 10 33.7%, and providing prelactal feeds during the first 3 days of the newborns life at a rate of 26.8%. Only 62.8% of mothers continue to breastfeed their children until 23 months of age. Inappropriate complimentary feeding practices also contribute to the high rates of undernutrition. The baseline survey found that only 19.9% of breastfed and 16.7% of non￾breastfed children aged 6-23 months received the minimum suggested number of complimentary feedings in the 24 hours preceding the survey. Added to this is the fact that only 21.9% of breastfed and 12.5% of non-breastfed children eat the minimum number of essential food groups to meet the recommended dietary diversity standards. The percentage of children aged 0-23 months receiving supplemental Vitamin A in the last 6 months is 76.2%. The Child Survival Project through its case management training on IMCI of health workers and Community IMCI trainings of HHPs continue to promote immediate initiation of breastfeeding, exclusive breastfeeding, and proper complimentary feeding for children aged 6-23 months in regards to diversity, density and frequency of feeding. The introduction of IMCI at health facilities has also improved both growth monitoring and nutrition counseling services. 11 CHAPTER 2 Purpose of the Monitoring Survey and Methodology Purpose of the Survey: The purpose of this LQAS Survey was to monitor each supervision area to ensure that each area is reaching the benchmarks set for the midterm of the project for each of the projects main objectives, captured by the Project Matrix Indicators. This is quite useful in project management as it allows project managers to determine if a SA is falling behind in one or more intervention areas, or if all of the SAs are having difficulty in a certain intervention area, and which SAs are performing very well in the project’s intervention areas. This allows rapid action in order to determine the cause of any difficulties noted, examine the barriers related to overcoming these difficulties, and ascertain solutions to this problem. If one or more SAs are performing very well regarding the indicators collected, a solution may be found in evaluating the differences found between a successful SA and the SA that needs improvement. Staff from the SA that is performing well in a certain area may even be used to help train staff or overcome barriers found in the struggling SA. A LQAS survey was also performed at the end of the first year of the program, in December of 2007. Because the project had been so successful with regard to the project indicators at the end of the first year of the project, this LQAS also included indicators in intervention areas in which the project may be interested in moving. Survey Team: The survey team consisted of 5 Supervisors and 20 CHPs, all of whom participated in the baseline KPC survey, and the LQAS survey performed in 2007, and have been active participants in the project activities. A three day LQAS refresher training was provided for the survey teams before implementation of the survey began. Survey Questionnaire: The LQAS questionnaire was developed by adapting the previously successful baseline KPC questionnaire. The questions pertaining to the indicators in the project matrix were chosen and placed in the LQAS questionnaire, and to this additional questions were added to provide indicators in intervention areas that the project may have interest in adding to the project. A questionnaire was developed for each category (denominator) of child needed to be interviewed that contained the questions pertinent to that child’s category. There were a total of 7 questionnaires developed for the following categories: 1. Questionnaire for the mother of any child aged 0-23 months-to be asked at the first randomly chosen household (called the “main questionnaire”) 2. Questionnaire for the mother of any child aged 6-9 months 3. Questionnaire for the mother of any child aged 6-23 months 4. Questionnaire for the mother of any child aged 12-23 months 5. Questionnaire for the mother of any child who had diarrhea in the last 2 weeks 12 6. Questionnaire for the mother of any child who had a cough with difficult or fast breathing in the last 2 weeks 7. Questionnaire for the mother of any child who had a fever in the last 2 weeks and whose fever is now resolved The questionnaires were field tested to ensure accuracy and consistency in both language (Pigeon English) and numerical sequencing (skip patterns, etc.). Sampling The project area was divided into 4 Supervision Areas (SAs). These SAs were chosen on the basis of management structure and geography, with each SA being monitored by a Supervisor, and with CHPs and HHPs committed to villages in that SA. For each individual SA, each community was listed randomly, with its population beside it. In each SA, the cumulative population of each community was then determined by summing the total population of that community with the combined population of all the preceding communities on the list. The total cumulative population of the communities in that SA was then divided by 24 to obtain the sampling interval for that SA, because to optimize the results using the available Decision Rule table, while yielding an overall sample size of 96 in order to yield significant combined frequencies, 24 answers are desired for each question in the interview in each SA. A random number was then chosen, with the stipulation being that the number be less than or equal to the sampling interval (24). The cumulative population of each community was then consulted, and the community containing the random number was then chosen as cluster number 1. The remaining clusters were then identified by continuing to add the sampling interval to the number that identified the previous cluster. In this way, each cluster was randomly chosen, with proper weight assigned to each community based on its population size. Fieldwork: Under the supervision of the five Supervisors, interviews were all carried out by CHPs. The center of each cluster was determined by allowing the CHP, who is quite familiar with the community, to enlist the help of the head of the community leaders to determine the spot where they feel that an equal number of households are on each side. The CHP then chose a random starting direction by spinning a bottle in the physical center of the cluster. The CHP then walked in the direction the bottle pointed, and counted the number of households in that direction until the last household in that cluster was reached. The CHP returned to the center and then chose a random number from a random number table, with the requirement that it had to be less than the number of homes in that direction. The CHP then counted the doorways in the direction the bottle was pointing until the doorway that corresponded to the random number chosen. This was deemed the first house. If the chosen household contained a child aged 0-23 months that was present at the time of the interview and sleeps in the house at night, and a mother that was present and sleeps in the house at night, the survey was begun at this household. If there was no child under two present, or the mother was not available for the interview, the enumerator 13 moved to the household that had the closest door relative to the doorway of the household just eliminated. The first eligible mother was asked the main questionnaire, and then all of the questionnaires appropriate for that child’s age and status of illness. The enumerator would then continue to the house with the nearest doorway, as described above, and determine if there was a child living in the household and present that was 0- 23 months in age. If there was a child aged 0-23 months in the household, and the mother was present, the enumerator would determine the age and recent illnesses of that child to determine if the child met any of the age or illness requirements of one of the remaining questionnaires. This procedure was then repeated until the number of interviews that needed to be taken in that community, in order to have each question answered once, was completed. A parallel sampling guide was written for the CHPs to help them in this process. Data Analysis: Data analysis was performed in the MTI/Liberia office using the Epi-info 3.3.2 database. The results were then placed in a pre-designed Excel spreadsheet (see Appendix 1), based upon the Decision Rule Table (see Appendix 2)for LQAS analysis, which allows for easy identification of whether a SA is performing well in regards to that particular indicator based upon the decision rule for that indicator. LQAS is a form of stratified random sampling, but owes its accuracy to using binomial distributions versus frequencies. A binomial distribution is a count of the number of times an event occurs, in this case a positive response. The Decision Rule Table (DRT) created for LQAS samples was used to determine the number of positive responses required, based on the sample size in each SA and the benchmark chosen for each indicator, for each SA to be considered to be performing well in that particular indicator. If the number of desired responses equals or is greater than the decision rule chosen by using the DRT, the SA is considered to have met the decision rule and thus is performing well in regard to that indicator. Constraints/Difficulties: Because the LQAS survey needed to be performed during the rainy season, a few of the communities sampled were not accessible by vehicles and therefore some of the teams had to walk for hours in order to reach these communities. This had no effect on the data collected. CHAPTER 3 Main Findings: Accomplishments, Priorities by Supervision area, and for the Program as a Whole Overall, the project has had a significant impact on improving immediate and appropriate breastfeeding practices, immunization rates, and caretaking practices during episodes of diarrhea and respiratory infections. All SAs met the respective benchmarks for 10 of the 13 project matrix indicators, and the combined frequency for 8 of the 13 indicators has already met or exceeded the final target for that indicator. The LQAS performed at the end of the first year (LQAS1) revealed that all SAs performed adequately with regard to every indicator other than the water treatment indicator, in which only SA1 failed to meet the benchmark for year 1. There were, however, a few changes in the results noted in this midterm LQAS from LQAS1. The percentage of infants aged 6-9 months receiving breast milk and complementary foods was found to be lower than the result found in LQAS1, with SA2 and SA4 not meeting their benchmarks set for the midterm. While the indicator for the percentage of children 6-23m who received a dose of Vitamin A in the last 6 months is still very high and at the level of the final target chosen for the project, SA1 did not meet the midterm benchmark for this indicator in this LQAS. The percentage of mothers protected from Tetanus was reduced from the end of year one, and SA1 did not meet the midterm benchmark set for this indicator. There continues to be difficulty regarding the treatment of drinking water, with all 4 SAs not meeting the midterm benchmark. Addressing quality of drinking water will require targeting communities drinking from streams and rivers with water hygiene education messages and targeting communities with hand pumps with the means to successfully chlorinate hand pumps regularly. The results, listed by intervention area, are discussed below. Nutrition: The project intends to improve the nutritional status of children through the promotion of correct breastfeeding and complimentary feeding practices, including immediate breastfeeding following childbirth, exclusive breastfeeding of children under 6 months of age, and the introduction of digestible and nutritional complimentary foods in children 6 months and greater. The project is partnering with CHAL to use the Care Groups Methodology to promote the immediate initiation of breastfeeding, exclusive and prolonged breastfeeding, and proper complimentary feeding for children aged 6-23 months. Nutrition indicators that were measured were taken from the Project 15 matrix, with undernutrition also measured. As discussed with CSTS prior to the survey, as this is a midterm LQAS, IYCF would not be collected but will be collected during the Final Evaluation. Table 1: Immediate and exclusive breastfeeding of newborns: Percent of newborns who were put to the breast within one hour of delivery and did not receive prelactal feeds Supervision Area Decision Rule Indicator numerator/ Decision denominator 1 2 3 4 BE Target Bench -mark 1 2 3 4 1 2 3 4 Combined Frequency for the Project Area Sep 2008 CF from Dec 2007 Immediate yes 14 12 8 15 and exclusive BF (n) size 24 24 2 4 24 34% 50% 42% 7 7 7 7 Y Y Y Y 51.04% 52.50% At baseline only 33.7% of mothers were practicing immediate and exclusive breastfeeding of newborns, defined as newborns who were put to the breast within one hour of delivery and who did not receive prelactal feeds. Each SA has reached the decision rule for its respective midterm benchmark and sample size, which signifies that each supervision area is performing well in educating mothers in the benefits of immediate and exclusive breastfeeding. The performance is excellent as shown by the fact that the combined frequency of immediate and exclusive breastfeeding is 51.04%, which already has surpassed the final target for this indicator of 50%. Advances regarding this indicator have remained steady from its increase to 52.5% at the end of the first year. Table 2: Introduction of complimentary foods: % of infants aged 6-9m receiving breastmilk and complementary foods Supervision Area Decision Rule Indicator numerator/ Decision denominator 1 2 3 4 BE Targe t Bench -mark 1 2 3 4 1 2 3 4 Combined Frequency for the Project Area Sep 2008 CF from Dec 2007 Children 6- yes 10 9 13 7 9m with continued BF and compliment ary feeding (n) size 24 24 24 24 38% 65% 51% 1 0 1 0 1 0 1 0 Y N Y N 40.63% 54.43% Each supervision area had made quite an initial improvement with the interventions regarding the teaching of continued breastfeeding with the introduction of complimentary foods. The combined frequency for this indicator rose from 38% at baseline to 54.43% at the end of 16 the first year, due to the fact that Nutrition was the first intervention in which the CHPs and HHPs were trained. Since that time, communities have received nutrition education through home visits, which includes the provision of education in correct complimentary feeding practices for their children. However, in the second year of the project these results did not continue to increase. The combined frequency of 40.63% is lower than that found at the end of year 1, and is comparable to baseline. SAs 2 and 4 failed to reach their benchmark for this indicator, which explains the lower combined frequency. Health education programming focusing on continued breastfeeding with the introduction of complimentary foods and nutrition will be carried out again in all SAs, and will be repeated at regular intervals moving forward in order to reinforce the health education messages provided through home visits and the Care Groups in order to ensure these SAs increase to the levels of the other SAs, and that all of the SAs continue to show improvement over time. Table 3: Vitamin A Supplementation in the last 6 months: Percentage of children age 6-23 months who received a dose of Vitamin A in the last 6 months (Mother’s recall). Indicator numerator/ Supervision Area Decision Rule Decision denominator 1 2 3 4 BE Target Benc h￾mark 1 2 3 4 1 2 3 4 Combined Frequency Sep 2008 CF from Dec 2007 Children 6-23m yes 16 24 21 21 receiving Vitamin A in the last 6m (n) size 24 24 24 24 76% 85% 81% 18 18 18 18 N Y Y Y 85.42% 90.60% At the end of year 1 of the project each SA had met the year 1 benchmark and had a combined frequency that surpassed the project target. This is in large part due to the fact that the MOHSW , through the CHT, has collaborated closely with the CSP to conduct three integrated EPI campaigns which included the administration of Vitamin A. This has greatly increased the availability and use of Vitamin A in children under 5. The midterm LQAS reveals that the combined frequency continues to exceed the benchmark and meet the project target, with SA2, SA3, and SA4 well above the decision rule. SA2, however, did not meet the benchmark, and the project will investigate through use of the HHPs and Care Groups the barriers that have caused this SA to lag behind in Vitamin A use. 17 Additional Nutrition Indicators: Table 4*: Percentage of children age 0-23 months who are underweight (>-2SD for the median weight for age, according to WHO/NCHS reference population) Indicator numerator/ Supervision Area Decision Rule Decision denominator 1 2 3 4 BE Target Bench -mark 1 2 3 4 1 2 3 4 Combined Frequency Sep 2008 CF from Dec 2007 Children aged yes 20 19 18 19 0-23 months who are underweight (n) size 24 24 24 24 27% 17% 22% 16 16 16 16 Y Y Y Y 20.83% NA Table 5*: Percentage of children age 0-23 months who are severely underweight (>-3SD for the median weight for age, according to WHO/NCHS reference population) Indicator numerator/ Supervision Area Decision Rule Decision denominator 1 2 3 4 BE Target Bench -mark 1 2 3 4 1 2 3 4 Combined Frequency Sep 2008 CF from Dec 2007 Children aged yes 22 22 21 21 0-23 months who are severely underweight (n) size 24 24 24 24 13% NA CF 19 19 19 19 Y Y Y Y 10.41% NA * The numbers in the SA and decision rule boxes of these tables were reversed in order to get the required decision rule. For example, there were 4 underweight children in SA1. 24- 4=20, so 20 was used (instead f 4) so that that the more underweight children, the lower this number, which would make it fail bynot reaching the decision rule. The baseline, benchmark, target, and combined frequency values are not reversed. In order to assess the possible impact of the projects interventions in the areas of nutrition, particularly in the area of health behaviors surrounding immediate and exclusive breastfeeding and continued feeding with the addition of complimentary foods, WFA measurements were taken at midterm. Benchmarks and targets were set for the indicator of undernutrition (>-2SD for the median weight for age) to be in line with the other nutrition interventions, and it was encouraging to find that all SAs met the benchmark for the midterm and that the combined frequency is 20.83%, which is an improvement from the baseline finding of 27%. This includes the percentage of children age 0-23 months that are severely underweight, which is 10.41%. This compares to the baseline rate of severely underweight children of 13.10%. Health education programming focusing on continued breastfeeding with the introduction of complimentary foods and nutrition will be carried out again in all SAs, to continue to reduce malnutrition. 18 Immunization: At baseline, full EPI coverage by 12 months of age was found to be only 18.9% and Health System Performance (DPT3 vaccination by 12 months of age), was only 30.6%% in children aged 12-23 months. The project has begun utilizing the IMCI and C-IMCI approach in order to improve vaccination rates and utilization of vaccination services by the community. Demand for EPI services is being created through CHWs and Care Groups. The project is providing logistical support for National Immunization days, advocates for the implementation of routine EPI services, and has been reducing missed opportunities for immunizations through IMCI at the HF level. This includes immunization outreach which is presently being accomplished by 89% of the HF surveyed in the Health Facility Assessment. Table 6: Health System Performance regarding Immunization services: Percentage of children aged 12-23 months who received DPT3 before they reached 12 months by the time of the interview as verified by a vaccination card. Indicator numerator/ Supervision Area Decision Rule Decision denominator 1 2 3 4 BE Target Benc h￾mark 1 2 3 4 1 2 3 4 Combined Frequency Sep 2008 CF from Dec 2007 Health System yes 15 16 17 10 Performance (DPT3) (n) size 24 24 24 24 31% 50% 41% 7 7 7 7 Y Y Y Y 60.42% 57.89% Each SA has reached the midterm benchmark with regard to Health System Performance, and the combined frequency of 60.42% for all SAs has surpassed the project target of 50%. Demand has been created through immunization messages and a recent National Immunization Day has been supported by the project as well. MOHSW staff at 27 HFs in the county have received training in IMCI, and through this training are reducing missed opportunities to begin, continue, or complete immunizations. This training includes an emphasis on checking the Road to Health Card for all children under five attending their HF to ensure that they have been fully immunized, and sending them for immunization immediately if they have not, as well as stressing the importance of providing outreach EPI services to catchment communities in order to track defaulters. 19 Table 7: EPI Coverage: Percentage of children aged 12-23 months who are fully vaccinated (received BCG, DPT3, OPV3, and measles vaccines) by 12 months of age. Indicator numerator/ Supervision Area Decision Rule Decision denominator 1 2 3 4 BE Target Bench -mark 1 2 3 4 1 2 3 4 Combined Frequency Sep 2008 CF from Dec 2007 EPI Coverage yes 10 10 10 8 (n) size 24 24 24 24 19% 40% 30% 3 3 3 3 Y Y Y Y 39.58% 35.53% All 4 SA areas met the midterm benchmark with regards to full immunization coverage as well, and the combined frequency of 39.58% meets the final target of 40%. The recent immunization campaigns in the county, reducing missed opportunities at the HF through IMCI, and outreach services provided by vaccinators to catchment communities has improved this indicator significantly. In addition, through community education about these campaigns and through education about the importance of immunization during home visits by CHPs and HHPs, more mothers have brought their children to receive EPI services conducted in their communities. Table 8: Maternal TT Vaccination: Percentage of mothers with children age 0-23 months that were protected against Tetanus before the birth of the youngest child (protected refers to receiving at least 2 TT or Td injections before the birth of the youngest child sufficiently close to that birth to provide protection). Indicator numerator/ Supervision Area Decision Rule Decision denominator 1 2 3 4 BE Target Bench -mark 1 2 3 4 1 2 3 4 Combined Frequency Sep 2008 CF from Dec 2007 Maternal TT yes 12 16 18 18 Vaccination (n) size 24 24 24 24 61% 80% 71% 15 15 15 15 N Y Y Y 66.67% 87.18% At the end of the first year of the project, all four SAs had met the benchmark regarding maternal TT vaccinations, and the combined frequency of the four supervision areas was 87.18%, which surpassed the project target of 80%. This was due to the fact that the MOHSW had carried out two maternal neonatal tetanus (MNT) campaigns in the county during the first year of the project. In addition, the project has been working at the HF level to increase the capacity of the staff, and at the community level through Care Groups to increase the 20 knowledge of mothers about the importance and thus the demand for TT vaccination. The CSP and other supporting NGOs have also provided logistical support to ensure that vaccines are available at the various HFs in the county and at the community level during these campaigns. The midterm HFA showed that 72% of HF in the county had TT toxoid available. However, at midterm the frequency has dropped to 66.76%, with 3 SAs still meeting their benchmarks but SA1 falling below the benchmark and thus lowering the combined frequency. Control of Diarrhea: Key home practices to mitigate diarrheal disease are emphasized by this project through the Care Groups and home visits by the CHPs and HHPs. These practices include the appropriate use of ORT, continued breastfeeding and fluids, and the frequent feeding of small amounts of food (catch-up feeding). Proper hygiene, including proper hand washing practices and the building and use of dish racks in communities for household utensils, is also emphasized in the project along with proper techniques to store and treat water effectively. The project is coordinating with the CHT and other NGOs to ensure a proper supply of ORT is available at the various HFs and the midterm HFA revealed that 89% of HFs have ORS available. Table 9: ORT use: Percentage of children 0-23 months with diarrhea in the last two weeks who received Oral Rehydration solution (ORS) and/or recommended home fluids. Indicator numerator/ Supervision Area Decision Rule Decision denominator 1 2 3 4 BE Target Bench -mark 1 2 3 4 1 2 3 4 Combined Frequency Sep 2008 CF from Dec 2007 ORT Use yes 18 16 21 21 (n) size 24 24 24 24 74% 85% 80% 16 16 16 16 Y Y Y Y 79.17% 85.51% Each SA has met the midterm benchmark for ORT use, and while the baseline was already at 74%, the project, through use of the Care Groups and through home visits, has been able to raise the combined frequency of ORT use in all SAs to 79.17%, which meets the benchmark for the midterm. Through the GIK supplied by the CSP, ORS has been made available in communities of the project for use by HHPs for children under five during emergencies before they are referred to the clinic. In addition, ORS was shown to be available at 89% of the HFs in the county. 21 Table 10: Increased fluid intake during diarrheal episode: Percentage of children 0-23 months with diarrhea in the last two weeks who were offered more fluids during the illness Indicator numerator/ Supervision Area Decision Rule Decision denominator 1 2 3 4 BE Target Bench -mark 1 2 3 4 1 2 3 4 Combined Frequency Sep 2008 CF from Dec 2007 Increased fluid yes 16 13 15 18 during diarrheal episode (n) size 24 24 24 24 52% 70% 61% 13 13 13 13 Y Y Y Y 64.58% 71.62% The midterm benchmark for increased fluid intake was reached for each of the SAs, and the combined frequency for the project was 64.58% which exceeds the midterm benchmark. CHWs, CHPs, and HHPs have all had a significant impact through working with mothers at the household level and in the Care Group setting to raise awareness about the control and treatment of diarrhea. CHPs and HHPs have continuously stressed the importance of increased breastfeeding and using other homemade ORS substitutes such as coconut water and rice water in the absence of ORS for managing children with diarrhea quickly at home in order to prevent dehydration. Table 11: Point of Use (POU): Percentage of households of children 0-23 months that treat their water effectively Indicator numerator/ Supervision Area Decision Rule Decision denominator 1 2 3 4 BE Target Bench -mark 1 2 3 4 1 2 3 4 Combined Frequency Sep 2008 CF from Dec 2007 yes 2 2 0 3 Effective water treatment (n) size 24 24 24 24 22% 40% 31% 4 4 4 4 N N N N 7.29% 18.67% 22 Additional Water Indicator: Table 12: Additional Indicator: Improved Water Source: Percentage of mothers of children 0-23m who live in households with an improved source for drinking water Indicator numerator/ Supervision Area Decision Rule Decision denominator 1 2 3 4 BE Target Bench -mark 1 2 3 4 1 2 3 4 Combined Frequency Sep 2008 CF from Dec 2007 yes 20 20 17 22 Households with improved water source (n) size 24 24 24 24 72% NA CF 18 18 18 18 Y Y N Y 82.29% NA Only 22% of households treated their water effectively at baseline, and this frequency has decreased over time (18.67% at year 1 and 7.29% at year 2). Possible reasons include the fact that 82% of households in GCM County have access to improved water sources which creates the perception in most of the communities that water treatment is unnecessary. In addition, the areas with a lack of improved water sources (which are using stream and river water) are the most vulnerable communities and are the hardest to reach, which is a barrier the project will look to overcome by targeting services to these areas. Also, a midterm evaluation regarding the availability of clean water was added to the survey to survey to determine the change from baseline. There is access to clean water sources in GCM County as 82.0% of households have access to either piped or protected water sources, which has increased from 72% at baseline. The remaining 18% of the communities have limited or seasonal access and SA2 has more of these villages. This was determined by using the combined frequency (CF) or average as the “benchmark” for the Decision Rule Table. Using the CF as the average allows the project to determine if a SA is not meeting the average of all of the SAs combined. In this case, SA2 did not meet the average. Community feedback sessions are scheduled in 12 of the communities to determine the barriers around this indicator, and an Action Plan will be created during these sessions. Possible solutions may include targeting those communities (18%) that are using water from rivers and streams as no hand pumps are available, and working to ensure that hand pumps are chlorinated at regular intervals. 23 Table 13: Appropriate Hand Washing Practices: Percentage of mothers of children 0-23 months who live in households with soap at the place for hand washing that washed their hands with soap at least 2 of the appropriate times during a 24 hour recall period. Indicator numerator/ Supervision Area Decision Rule Decision denominator 1 2 3 4 BE Target Bench -mark 1 2 3 4 1 2 3 4 Combined Frequency Sep 2008 CF from Dec 2007 yes 9 9 14 10 Appropriate hand washing Practices (n) size 24 24 24 24 19% 40% 30% 3 3 3 3 Y Y Y Y 43.75% 28% At baseline only 19.0%of mothers reported washing their hands at least at 2 critical times (after defecation and at one of the following: before food preparation, before feeding children, or after attending to a child who has defecated). The project has trained CHPs and HHPs in sanitation and hygiene promotion messages, which they have disseminated in the Care Group setting and through on-going educational programs and interventions in the communities during home visits. This methodology is proving effective, and each SA has reached the benchmark for the midterm of the project. In addition, the combined frequency of the SA is 43.75%, which exceeds both the midterm benchmark and the project target. Pneumonia Case Management: Table 14: Appropriate Care Seeking for Pneumonia: Percentage of children age 0-23 months with chest-related cough and fast/difficult breathing in the last two weeks who were taken to an appropriate health provider. Indicator numerator/ Supervision Area Decision Rule Decision denominator 1 2 3 4 BE Target Bench -mark 1 2 3 4 1 2 3 4 Combined Frequency Sep 2008 CF from Dec 2007 yes 22 23 21 14 Appropriate care seeking for pneumonia (n) size 24 24 24 24 43% 65% 54% 10 10 10 10 Y Y Y Y 83.33% 60.56% 24 There has been a focus on the training of HF skilled and support staff, CHPs, and HHPs in health promotion/education in order to increase timely health seeking behaviors by mothers for their sick children, and in referring sick children to the HF during home visits. HF personnel are being trained in IMCI and are being trained to properly measure respiratory rate as this is the best measure of respiratory problems in infants and young children. Through Care Groups mothers are being educated in recognizing the signs of ARI, including cough, difficulty/fast breathing, and chest-in-drawing, and the importance of seeking timely medical treatment for their child with ARI. Children found to have signs of pneumonia, such as cough/difficulty breathing, are being referred to the health facility for treatment. Ensuring that antibiotics are readily available at the HF level is also a priority of this phase of the project. This concerted effort is providing results as all SA reached the midterm benchmark, with the combined frequency of 83.33% well above the midterm benchmark and project target of 65%. Malaria and the Management of Febrile Illness: Table 15: Child sleeps under an insecticide-treated bednet: Percentage of children 0-23 months who slept under an insecticide-treated bed net the previous night. Indicator numerator/ Supervision Area Decision Rule Decision denominator 1 2 3 4 BE Target Bench -mark 1 2 3 4 1 2 3 4 Combined Frequency Sep 2008 CF from Dec 2007 yes 12 15 18 20 Child sleeps under ITN (n) size 24 24 24 24 18% 35% 27% 3 3 3 3 Y Y Y Y 67.71% 22.08% Only 17.7% of children less than 24 months slept under an insecticide treated bed net at the time of the baseline KPC survey. During March of 2008 the CHT and Child Survival Project distributed nets provided by the President’s Malaria Imitative. HHPs, through home visits in their assigned communities, are making follow up visits to ensure that mothers and other caregivers are using the mosquito nets that have been provided to them in their communities for their children under five. We, therefore, expected midterm results for this indicator to be significantly higher. As hoped, all SA met the midterm benchmark for this indicator, and the combined frequency of 67.71% is well above the project target of 35%. 25 Table 16: Child with fever receives appropriate antimalarial treatment: Percentage of children 0-23 months with a febrile episode that ended during the last two weeks who were treated with an effective anti-malarial drug within 24 hours after the fever began. Indicator numerator/ Supervision Area Decision Rule Decision denominator 1 2 3 4 BE Target Bench -mark 1 2 3 4 1 2 3 4 Combined Frequency Sep 2008 CF from Dec 2007 yes 12 13 6 16 Child receives appropriate antimalarial (n) size 24 24 24 24 12.5% 50% 32% 4 4 4 4 Y Y Y Y 48.96% 40.58% *“Treated with an effective anti-malarial drug within 24 hours after the fever began” was defined as children who were taken to an appropriate health facility within 24 hours and given a medication at that facility. At baseline only 12.5% of children were treated with proper anti-malarial medication within 24 hours of the onset of fever. Project staff, HHPs, and Care Groups have now been trained in health promotion in order to increase timely health seeking behaviors by mothers for their sick children. IMCI is being instituted at all HF in the project area to ensure proper diagnosis and treatment of malaria with anti￾malarial medication. The midterm Health Facility Assessment has shown that ACT is available at 94% of the HFs in the County. All 4 SAs have met the mid term benchmark for this indicator, and the combined frequency of 48.96% is within range of the project target. The project will focus on continuing to improve this rate, and priority will be given to health promotion in order to increase timely health seeking behaviors by mothers for their sick children. It should be noted that for this survey this rate was calculated based on the child being seen in a qualified health facility and taking a medicine within 24 hours of the onset of fever. The new indicator developed by the CORE Group and CSTS, which requires the mother to identify that an antimalarial was taken, was not used in this survey as it was not yet included in the Rapid CATCH at the time of the baseline survey. It has been verified through the HFA that ACT is the first line DOC for malaria. 26 Additional Indicators Measured for Possible Project Intervention: ANTENATAL CARE Prenatal Care Table 17: Prenatal Care: Percentage of mothers of children 0-23m who had at least one prenatal visit prior to the birth of her youngest child less than 24 months of age Indicator numerator/ Supervision Area Decision Rule Decision denominator 1 2 3 4 BE Target Bench -mark 1 2 3 4 1 2 3 4 Combined Frequency Sep 2008 CF from Dec 2007 yes 20 24 23 22 Prenatal Care (n) size 24 24 24 24 56% NA CF 21 21 21 21 N Y Y Y 92.71% NA Table 18: Prenatal Care-3 or more visits: Percentage of mothers of children 0-23m who had at least three prenatal visit prior to the birth of her youngest child less than 24 months of age Indicator numerator/ Supervision Area Decision Rule Decision denominator 1 2 3 4 BE Target Bench -mark 1 2 3 4 1 2 3 4 Combined Frequency Sep 2008 CF from Dec 2007 yes 14 20 16 14 Maternal Health Card (n) size 24 24 24 24 NA NA CF 14 14 14 14 Y Y Y Y 66.67% NA 27 Table 19: Maternal Health Card: Percentage of mothers of children 0-23m who have a maternal health card Indicator numerator/ Supervision Area Decision Rule Decision denominator 1 2 3 4 BE Target Bench -mark 1 2 3 4 1 2 3 4 Combined Frequency Sep 2008 CF from Dec 2007 yes 7 6 9 8 Maternal Health Card (n) size 24 24 24 24 NA NA CF 4 4 4 4 Y Y Y Y 31.25% NA Table 20: Prenatal Danger Signs: Percentage of mothers of children 0-23m able to report at least one known prenatal danger signs indicating the need to seek health care Indicator numerator/ Supervision Area Decision Rule Decision denominator 1 2 3 4 BE Target Bench -mark 1 2 3 4 1 2 3 4 Combined Frequency Sep 2008 CF from Dec 2007 yes 11 11 8 15 Prenatal Danger Signs (n) size 24 24 24 24 NA NA CF 9 9 9 9 Y Y N Y 46.88% NA Table 21: Knowledge of prenatal Danger Signs: Percentage of mothers of children 0-23m who report that they would first go to an appropriate health provider if they experienced any prenatal danger symptoms Indicator numerator/ Supervision Area Decision Rule Decision denominator 1 2 3 4 BE Target Bench -mark 1 2 3 4 1 2 3 4 Combined Frequency Sep 2008 CF from Dec 2007 Prenatal yes 24 24 21 22 Appropriate Emergency Care (n) size 24 24 24 24 NA NA CF 21 21 21 21 Y Y Y Y 94.79% NA *It should be noted that all new indicators, that are not part of the project matrix, do not have a baseline, benchmark, or targets. In these cases the combined frequency is used as the benchmark upon which the Decision Rule is based. If a SA fails to meet this benchmark it is statistically significantly below the average for the entire project area. 28 It is quite encouraging that 92.71% of mothers had prenatal, with at least one prenatal visit prior to the birth of her youngest child, compared to 56% at baseline. SA1 is below this average, but is still quite high. The midterm evaluation also found that 67.71% of mothers had at least 3 prenatal visits, (the baseline did not include this indicator as it is not part of the Rapid CATCH or Prenatal Care module). However, only 31.25% of mothers have a maternal health card, and only 46.88% of mothers could name at least one prenatal danger sign. These are areas that could be improved through possible future programming. This would include working with the HF in order to ensure that mothers receive Maternal Health cards, and extending the health education messages given to the mothers to include the recognition of prenatal danger signs. It is also encouraging to note that 94.7% of mothers would first go to an appropriate health provider if they experienced prenatal danger signs. As far as transportation to the nearest health provider, 86.46% stated that they would walk, while only 16.67% stated they would take a car and 3.13% would take a motorcycle. An emergency transportation plan would be important for those mothers who do not have access to motorized transportation, and the project has been working to put this in place in each community, through the CHDCs. Delivery Care Table 22: Delivery in a HF: Percentage of children aged 0-23 months whose delivery was in an appropriate health facility Indicator numerator/ Supervision Area Decision Rule Decision denominator 1 2 3 4 BE Target Bench -mark 1 2 3 4 1 2 3 4 Combined Frequency Sep 2008 CF from Dec 2007 Delivery in an yes 6 7 6 5 appropriate health facility (n) size 24 24 24 24 NA NA CF 2 2 2 2 Y Y Y Y 25.00% NA 29 Table 23: Skilled Birth Attendant: Percentage of children aged 0-23 months whose delivery was attended by a skilled health personnel Indicator numerator/ Supervision Area Decision Rule Decision denominator 1 2 3 4 BE Target Bench -mark 1 2 3 4 1 2 3 4 Combined Frequency Sep 2008 CF from Dec 2007 Skilled Birth yes 7 6 7 3 Attendant (n) size 24 24 24 24 21.3% NA CF 2 2 2 2 Y Y Y Y 23.96% NA The percentage of women giving birth in a facility is quite low, at 25.00%, and the percentage of births that are attended by a skilled birth attendant, which is defined in this study as a doctor, nurse, physician assistant, or midwife, is only 23.96%. Additionally, at baseline traditional birth attendants (TBA) attended the delivery 73.0% of the time, and presently these birth attendants are not skilled in proper cord care or other clean birthing practices. This signifies that there is a lack of skilled personnel in these areas, and a need to train TBAs in proper birthing practices as they carry the majority of the burden regarding deliveries. At present, MTI does not have resources at this time to provide training for TBAs but plans to pursue funding possibilities. TBAs and HHPs should be trained in the importance of assuring that both mother and child receive skilled delivery and follow up care including post partum visit with a trained health professional, including referring the mother when necessary. Postnatal Care Table 24: Postnatal Visit-Mom: Percentage of mothers of children 0-23m who received a post-natal visit from an appropriately trained health worker within three days after birth Indicator numerator/ Supervision Area Decision Rule Decision denominator 1 2 3 4 BE Target Bench -mark 1 2 3 4 1 2 3 4 Combined Frequency Sep 2008 CF from Dec 2007 Postnatal Visit- yes 9 7 4 7 Mom (n) size 24 24 24 24 6.0% NA CF 3 3 3 3 Y Y Y Y 28.13% NA 30 Table 25: Postnatal Visit-Child : Percentage of mothers of children 0-23m who received a post-natal visit from an appropriately trained health worker within three days after birth Indicator numerator/ Supervision Area Decision Rule Decision denominator 1 2 3 4 BE Target Bench -mark 1 2 3 4 1 2 3 4 Combined Frequency Sep 2008 CF from Dec 2007 Postnatal Visit- yes 9 9 4 13 Child (n) size 24 24 24 24 7.0% NA CF 6 6 6 6 Y Y Y Y 36.46% NA Table 26: Knowledge of Neonatal Danger Signs: % of mothers able to report at least two known neonatal danger signs Indicator numerator/ Supervision Area Decision Rule Decision denominator 1 2 3 4 BE Target Bench -mark 1 2 3 4 1 2 3 4 Combined Frequency Sep 2008 CF from Dec 2007 Knowledge of yes 0 0 0 1 Neonatal Danger Signs (n) size 24 24 24 24 NA NA CF too low to calculate NA 1.04% NA At baseline, postpartum visits with a skilled professional occurred at an alarmingly low rate, which was 6.0% for mothers and 7.0% for newborns. By midterm these numbers have improved, due to health education messages provided in the Care Groups and home visits stressing the utilization of health services for child and antenatal care. Postnatal visits within 3 days of the birth of the child are now occurring at a rate of 28.13% for the mother and 36.46% for the child. These rates are markedly better but still quite low, and therefore the project is considering adding more specific health education messages about prenatal care, delivery care, and postnatal care. These messages will be given at the household level by the HHPs through the care Group structure. Knowledge about neonatal danger signs is extremely low, at 1.04%. Health education messages will be given to the mothers through the Care Group structure to teach them to recognize neonatal danger signs and to encourage them to seek prenatal, delivery, and postnatal care from appropriately trained health workers at the HF. 31 HIV Table 27: Knowledge of HIV : % of mothers of children 0-23m who have heard of AIDS Indicator numerator/ Supervision Area Decision Rule Decision denominator 1 2 3 4 BE Target Bench -mark 1 2 3 4 1 2 3 4 Combined Frequency Sep 2008 CF from Dec 2007 Heard of AIDS yes 20 23 21 23 (n) size 24 24 24 24 81.9% NA CF 21 21 21 21 N Y Y Y 90.63% NA Table 28: HIV Risk Reduction: % of mothers of children 0-23m who can correctly name at least 2 ways to reduce the risk of HIV/AIDS Indicator numerator/ Supervision Area Decision Rule Decision denominator 1 2 3 4 BE Target Bench -mark 1 2 3 4 1 2 3 4 Combined Frequency Sep 2008 CF from Dec 2007 Know 2 or more yes 10 6 11 9 ways to reduce risk of HIV (n) size 24 24 24 24 19.0% NA CF 6 6 6 6 Y Y Y Y 37.50% NA The prevalence of HIV/AIDS in the general population of Liberia is 8.2% as of 1986, but this is thought to be a low estimate. Liberia has incurred the same increase in exposure to the disease as the rest of Africa, and is presently ill prepared to deal with this epidemic as the knowledge about HIV/AIDS is extremely low. In GCM County, while 90.63% of mothers have heard of AIDS, only 37.50% of mothers could name two or more correct methods of reducing the risk of contracting the virus that causes HIV. Therefore, it is quite important to increase the knowledge of the community about HIV/AIDS and risk reduction. This intervention could be introduced through educational messages in the Care Groups and household visits. 32 Table 29: Heard of VCT: % of mothers of children 0-23m who have heard of VCT Indicator numerator/ Supervision Area Decision Rule Decision denominator 1 2 3 4 BE Target Bench -mark 1 2 3 4 1 2 3 4 Combined Frequency Sep 2008 CF from Dec 2007 Heard of VCT yes 7 10 11 8 (n) size 24 24 24 24 NA NA CF 6 6 6 6 Y Y Y Y 37.50% NA Table 30: Access to VCT : % of mothers of children 0-23m who know of a place where people can go to get VCT Indicator numerator/ Supervision Area Decision Rule Decision denominator 1 2 3 4 BE Target Bench -mark 1 2 3 4 1 2 3 4 Combined Frequency Sep 2008 CF from Dec 2007 Know where to yes 6 3 9 7 get VCT (n) size 24 24 24 24 NA NA CF 3 3 3 3 Y Y Y Y 26.04% NA Table 31: Access to HIV Treatment: % of mothers of children 0-23m who know of a place where people can go to get treatment for HIV Indicator numerator/ Supervision Area Decision Rule Decision denominator 1 2 3 4 BE Target Bench -mark 1 2 3 4 1 2 3 4 Combined Frequency Sep 2008 CF from Dec 2007 Know where to yes 4 1 4 4 get treatment for HIV (n) size 24 24 24 24 NA NA CF too low to calculate NA 13.54% NA 33 Table 32: Knowledge of PMTCT: % of mothers of children 0-23m who know that there are special medications that can be given to a pregnant woman infected with the AIDS virus to reduce the risk of mother-to-child transmission Indicator numerator/ Supervision Area Decision Rule Decision denominator 1 2 3 4 BE Target Bench -mark 1 2 3 4 1 2 3 4 Combined Frequency Sep 2008 CF from Dec 2007 Knowledge of yes 3 1 0 1 PMTCT (n) size 24 24 24 24 NA NA CF too low to calculate NA 5.21% NA Table 33: Access to PMTCT: % % of mothers of children 0-23m who know of a place in the community to go to get these special medications that can be given to a pregnant woman infected with the AIDS virus to reduce the risk of mother-to-child transmission Indicator numerator/ Supervision Area Decision Rule Decision denominator 1 2 3 4 BE Target Bench -mark 1 2 3 4 1 2 3 4 Combined Frequency Sep 2008 CF from Dec 2007 Know where to yes 4 1 2 1 get PMTCT (n) size 24 24 24 24 NA NA CF too low to calculate NA 8.33% NA While only 37.50% of mothers have heard of Voluntary Counseling and Testing (VCT), even fewer, 26.04%, of mothers have the knowledge of where such treatment is available in the community. Educational messages regarding the importance of knowing ones HIV status and receiving counseling on HIV may be introduced into the health education messages delivered through the Care Group structure. It is worrisome, but understandable that only 13.54% of mothers know where one may receive treatment for HIV. This is due to a combination of the stigma involved with disclosing information if you are HIV positive, and the fact that treatment at this time is only available in Monrovia. Even worse is the fact that only 5.21% of mothers know that there is PMTCT treatment available and only 8.33% of mothers know where PMTCT treatment may be received. More mothers answered that they knew where PMTCT treatment is available than answered that they knew what PMTCT actually is, most likely due to confusing PMTCT treatment with general HIV treatment. It is quite apparent that HIV is an area where much work may be done to increase awareness of risk reduction, VCT, and treatment options for HIV and treatment to reduce transmission to newborns. The project will consider adding these health education messages into the Care Group structure. 34 HIV Stigma Table 34: Know someone Stigmatized: % of mothers of children 0-23m who personally know someone who has been denied involvement in social events, religious services, or community events in the last 12 months because he or she is suspected to have the AIDS virus or has the AIDS virus? Indicator numerator/ Supervision Area Decision Rule Decision denominator 1 2 3 4 BE Target Bench -mark 1 2 3 4 1 2 3 4 Combined Frequency Sep 2008 CF from Dec 2007 Know someone yes 2 0 2 0 Stigmatized (n) size 24 24 24 24 NA NA CF too low to calculate NA 4.17% NA Table 35: Believe They Should Be Ashamed : % of mothers of children 0-23m who believe that people with the AIDS virus should be ashamed of themselves Indicator numerator/ Supervision Area Decision Rule Decision denominator 1 2 3 4 BE Target Bench -mark 1 2 3 4 1 2 3 4 Combined Frequency Sep 2008 CF from Dec 2007 Believe people yes 13 6 12 11 with HIV should be ashamed (n) size 24 24 24 24 NA NA CF 7 7 7 7 Y N Y Y 43.75% NA *In this instance SA2, by being significantly below the average, SA2 actually has less stigma against people with HIV than the other SAs Table 36: Believe They Should Be Forced Out of the Community: % of mothers of children 0-23m who belIeve that a person with the AIDS virus should be forced to leave the community. Indicator numerator/ Supervision Area Decision Rule Decision denominator 1 2 3 4 BE Target Bench -mark 1 2 3 4 1 2 3 4 Combined Frequency Sep 2008 CF from Dec 2007 Believe people yes 11 12 15 13 with HIV Should Be Forced Out of the Community (n) size 24 24 24 24 NA NA CF 10 10 10 10 Y Y Y Y 53.13% NA 35 While only 4.17% of mothers state that they actually know someone who has suffered outward stigmatization due to having HIV, it is quite apparent that high levels of stigma against people living with HIV/AIDS (PLWHA). Approximately half of the mothers interviewed believe that PLWHA should be ashamed of themselves (43.75%) and believe that PLWHA should be forced to leave the community (53.13%). This is an important revelation, because no knowledge of the methods of testing and treating HIV will be utilized by a society in which such fear of stigmatization exists. The project will add important messages against stigmatizing those with HIV to the Health education messages already being presented through the Care Group structure. Health Facility Client Satisfaction Survey Indicators In addition to the Health Facility Assessment (HFA) performed at baseline, a client satisfaction survey was added to the LQAS survey of mothers of children aged 0-23 months in place of performing it during the HFA. The reason for this is that bias is introduced into any client satisfaction survey given at the HFA, because clients dissatisfied with HF services are less likely to go to a HF, and therefore would not be included in the survey. A community based survey of mothers of children aged 0-23 months avoids this bias. Supervision Areas Target for this LQAS Combined Frequency for Project Area Table 1. 1 2 3 4 Base-line Final Target % of mothers with children age 0-23 months who have taken the child to the nearest HF in the last 12 months Numerator (yes answers) 91 92 75 90 NA TBD CF 97.75% Denominator (Total)- Sample size 93 93 77 93 Utilization of health services in GCM County is very high, with 97.75% of mothers of children under the age of two bringing their child to the HF within the last year. Due to the fact that only 7 of the 356 mothers interviewed did not utilize health services, reasons for not utilizing health services could only be asked of these 7 mothers, which do not yield statistically significant results. 36 Table 2. Supervision Areas Base-line Final Target Target for this LQAS Combined Frequency for Project Area % of mothers of children aged 0-23m who felt the time waited to be seen at the HF was reasonable (exc., good, Acceptable) Numerator (yes answers) 81 84 75 77 NA TBD CF 89.04% Denominator (Total)- Sample size 93 93 77 93 Table 3. Supervision Areas Base-line Final Target Target for this LQAS Combined Frequency for Project Area % of mothers of children aged 0-23m who felt the treatment received for their child's illness at the HF was reasonable (exc., good, fair) Numerator (yes answers) 88 82 71 85 NA TBD CF 91.57% Denominator (Total)- Sample size 93 93 77 93 Table 4. Supervision Areas Base-line Final Target Target for this LQAS Combined Frequency for Project Area % of mothers of children aged 0-23m who were likely or very likely to return to the same facility the next time the child is sick Numerator (yes answers) 90 89 74 90 NA TBD CF 96.30% Denominator (Total)- Sample size 93 93 77 93 37 Mothers were overwhelmingly satisfied with the amount of time they were required to wait to be seen at the HF (89.04%), and the treatment received at the HF (91.57%). It is not surprising, therefore, that 96.30% of mothers stated they were likely or very likely to return the same facility the next time the child is sick. CHAPTER 4 Action Plan/Conclusions/Recommendations The Grand Cape Mount Child Survival Project has improved the health of children and families in remarkable fashion since its inception in 2006. The midterm LQAS has verified that the project has had a significant impact on improving immediate and appropriate breastfeeding practices, immunization rates, and caretaking practices during episodes of diarrhea, respiratory infections, and malaria. All SAs met the respective benchmarks for 10 of the 13 project matrix indicators, and the combined frequency for 8 of the 13 indicators has already met or exceeded the final target for that indicator. There also are some areas in which the project needs to increase its effectiveness. The project has scheduled community feedback sessions in 12 of the communities surveyed in order to present the results of the midterm LQAS survey and involve the community in planning for next steps. The following is a summary of the Action Plan for the next two years that has been discussed with project partners:. Nutrition 1. The percentage of infants aged 6-9 months receiving breast milk and complementary foods was found to be 40.63%, which does not meet the midterm benchmark for this indicator and is lower than the result found in the LQAS at the end of year 1 (LQAS1). This is mainly due to the fact that SA2 and SA4 did not meet their benchmarks set for the midterm. Health education programming focusing on continued breastfeeding with the introduction of complimentary foods and nutrition will be carried out again in all SAs, and will be repeated at regular intervals moving forward in order to reinforce the health education messages provided through home visits and the Care Groups in order to ensure these SAs increase to the levels of the other SAs, and that all of the SAs continue to show improvement over time. 2. While the indicator for the percentage of children 6-23m who received a dose of Vitamin A in the last 6 months is still very high and at the level of the final target chosen for the project, SA1 did not meet the midterm benchmark for this indicator in this LQAS. An investigation into why this has occurred in SA1 will be conducted, with care being taken to ensure the availability of Vitamin A at the HF. Immunization 1. During the first year of the project the MOHSW had carried out two maternal neonatal tetanus (MNT) campaigns in the county, and the end of year 1 results were excellent, with all 4 SAs meeting their benchmark and the CF of 87% surpassing the project target. The midterm LQAS, however, shows that the rate of maternal Tetanus immunization has declined somewhat to 66.67%, which is above the baseline of 61%, but below the midterm benchmark of 71%. Also, SA1 did not meet the midterm benchmark. Health education messages regarding the importance of TT vaccination will be carried out in increased fashion, particularly in SA1, to increase utilization of this service at the HF. 39 This will be combined with efforts to ensure that TT toxoid is available at all of the HFs in the county and that all HWs in the facilities are checking the mothers TT immunization status during prenatal care visits Control of Diarrhea 1. The issue of not treating water effectively continues to be a concern in GCM County. Only 22% of households treated their water effectively at baseline, and this frequency has decreased over time (18.67% at year 1 and 7.29% at year 2). Possible reasons include the fact that 82% of households in GCM County have access to improved water sources which creates the perception in most of the communities that water treatment is unnecessary. In addition, the areas with a lack of improved water sources (which are using stream and river water) are the most vulnerable communities and are the hardest to reach, which is a barrier the project will look to overcome by targeting services to these areas. Community feedback sessions are scheduled in 12 of the communities to determine the barriers around this indicator, and an Action Plan will be created during these sessions. Possible solutions may include targeting those communities (18%) that are using water from rivers and streams as no hand pumps are available, particularly in SA3 which was significantly below the average, and working to ensure that hand pumps are chlorinated at regular intervals. The following activities may be planned, pending the results of the community feedback sessions: a. The GCM CSP, CHCs and the CHT will discuss with partners in water and sanitation the chlorination of handpumps and other sources of drinking water. b. Partners in WATSAN will also be asked to participate in our CG sessions so that they may teach HHPs how to treat water with Chlorox so that this information can be shared with households. c. CHPs and HHPs will increase the awareness in the communities about the importance of boiling water taken from untreated sources. d. The GCM CSP will Work with the CHT and the CHCs to advocate with NGOs involved with water and sanitation, such as the African Development Network and Norwegian Refugee Council, to provide hand pumps in these communities. In addition, the project has trained CHPs and HHPs in sanitation and hygiene promotion messages, which they have disseminated in the Care Group setting and through on-going educational programs and interventions in the communities during home visits. This methodology is proving effective, and each SA has reached the benchmark for the midterm of the project. In addition, the combined frequency of the SA is 43.75%, which exceeds the project target. While this result is excellent, it is still below 50%, and therefore the project will continue to improve this health behavior in the following manner: a. The CSP will conduct IEC/BCC at the community level through CHPs and HHPs b. The CSP will increase community awareness through the mosques and churches through the imams and pastors who have been trained in the five intervention areas of the CSP. c. The HHPs will work with their households to make sure water is always available for washing hands by members of the households 40 d. The HHPs will encourage families to have a hand washing station near their latrine e. The HHPs will encourage the washing hands with soap (locally made or commercial) or soap alternative such as ash. f. Parents will be instructed to set the example for their children by always washing their hands at appropriate times so that this behavior will continue to be practiced in the community. g. Parents will be instructed to ensure that their children wash their hands every time before eating. Malaria 1. The proper treatment of malaria with antimalarials within 24 hours has increased from 12.55% at baseline to 48.96% at midterm, which meets the project target. Project staff, HHPs, and Care Groups have now been trained in health promotion in order to increase timely health seeking behaviors by mothers for their sick children. IMCI is being instituted at all HF in the project area to ensure proper diagnosis and treatment of malaria with anti-malarial medication. The midterm Health Facility Assessment has shown that ACT is available at 94% of the HFs in the County. However, because malaria is such a burden in this population, the project will focus on continuing to improve this rate, and priority will be given to health promotion in order to increase timely health seeking behaviors by mothers for their sick children. Included in the Action Plan will be the following, which was devised by project staff: a. HHPs will continue to stress the importance of taking children with fever immediately to a HF in order to receive appropriate treatment. b. HHPs will make sure that children with fever are referred immediately, using their community referral cards, to a clinic. c. The CSP will advocate with the MOHSW to begin a pilot project in a few very remote communities in the county by the 3rd year of the project to do Operational Research on the availability of treatment to children by CHPs with antimalarial medications at the community level, because most of these communities are far away from the HFs. d. The Project will encourage communities to collect emergency funds to be used to ensure that children sick with fever or other severe conditions are quickly taken to a clinic for treatment. e. The Project will encourage communities to set up an emergency transport system (hammock/wheelbarrow) with young men placed in charge of the system so that sick children may be immediately taken to the clinic Possible New Intervention Areas Antenatal Care Only 31.25% of mothers have a maternal health card, and only 46.88% of mothers could name at least one prenatal danger sign. These are areas that could be improved through possible future programming. This would include: 41 a. Working with the HF in order to ensure that mothers receive Maternal Health cards b. Extending health education messages to the mothers to include the recognition of prenatal danger signs c. As far as transportation to the nearest health provider, 86.46% stated that they would walk, while only 16.67% stated they would take a car and 3.13% would take a motorcycle. An emergency transportation plan would be important for those mothers who do not have access to motorized transportation, and the project will increase efforts to put this in place in each community, through the CHDCs Delivery Care The percentage of women giving birth in a facility is quite low, at 25.00%, and the percentage of births that are attended by a skilled birth attendant, which is defined in this study as a doctor, nurse, physician assistant, or midwife, is only 23.96%. Additionally, at baseline traditional birth attendants (TBA) attended the delivery 73.0% of the time, and presently these birth attendants are not skilled in proper cord care or other clean birthing practices. Postnatal Care 1. Postnatal visits within 3 days of the birth of the child are now occurring at a rate of 28.13% for the mother and 36.46% for the child. These rates are markedly better than baseline (6% and 7%, respectively) but still quite low, and therefore the project is considering adding more specific health education messages about prenatal care, delivery care, and postnatal care. These messages will be given at the household level by the HHPs through the care Group structure. Also, TBAs and HHPs will be trained in the importance of assuring that both mother and child receive skilled delivery and follow up care including post partum visit with a trained health professional, including referring the mother when necessary. 2. Knowledge about neonatal danger signs is extremely low, at 1.04%. Health education messages will be given to the mothers through the Care Group structure to teach them to recognize neonatal danger signs and to encourage them to seek prenatal, delivery, and postnatal care from appropriately trained health workers at the HF. 42 HIV/AIDS 1. In GCM County, while 90.63% of mothers have heard of AIDS, only 37.50% of mothers could name two or more correct methods of reducing the risk of contracting the virus that causes HIV. Health Education messages regarding HIV could be introduced through educational messages in the Care Groups and household visits in order to increase the knowledge of the community about HIV/AIDS and risk reduction. 2. Only 37.50% of mothers have heard of Voluntary Counseling and Testing (VCT), and only 26.04% of mothers have the knowledge of where such treatment is available in the community. Educational messages regarding the importance of knowing ones HIV status and receiving counseling on HIV may be introduced into the health education messages delivered through the Care Group structure 3. Only 13.54% of mothers know where one may receive treatment for HIV. This is due to a combination of the stigma involved with disclosing information if you are HIV positive, and the fact that treatment at this time is only available in Monrovia. Even more worrisome is the fact that only 5.21% of mothers know that there is PMTCT treatment available and only 8.33% of mothers know where PMTCT treatment may be received. The project will consider adding these health education messages into the Care Group structure, along with ensuring that proper referral of patients in need of services to the proper venues. 4. Stigma against those living with HIV is a very real issue in GCM County. Approximately half of the mothers interviewed believe that PLWHA should be ashamed of themselves (43.75%) and believe that PLWHA should be forced to leave the community (53.13%). This is an important revelation, because no knowledge of the methods of testing and treating HIV will be utilized by a society in which such fear of stigmatization exists. The project will add important messages against stigmatizing those with HIV to the Health education messages already being presented through the Care Group structure. CONCLUSIONS/RECOMMENDATONS The GCMCSP continues to improve the lives of children and families in Grand Cape Mount County. The project remains very successful in attaining, and in many cases surpassing benchmark and target coverage for most of the indicators in its project matrix. The Action Plan outlined above will guide the third and fourth years of the project in order to continue to improve the indicators with which the project has had such success, and to overcome the barriers and improve the results of the interventions which need continued improvement. Because the project has been so successful, it will also strive to add the new activities discussed in the Action Plan in order to continue to broaden its scope and sustainable results in the communities it serves. Annex 1: LQAS Summary Table of Project Matrix Indicators Supervision Areas Base -line Final Target Bench￾mark for this LQAS Decision Rule Number X Axis is Target for this LQAS Y Axis is the Total sample Size Decision Rule (Yor N) for each Supervision Area CF for Project Area Sep 2008 CF from Dec 2007 1 2 3 4 1 2 3 4 1 2 3 4 Numerator (yes answers) 14 12 8 15 % of newborns who were put to the breast within one hour of delivery and did not receive prelactal feeds (bfimmAapp) Denominator (Total)-Sample size 24 24 24 24 34% 50% 42% 7 7 7 7 Y Y Y Y 51.04% 52.50% Numerator (yes answers) 10 9 13 7 % of infants 6-9m receiving breastmilk and complementary foods (BFandCF) Denominator (Total)-Sample size 24 24 24 24 38% 65% 51% 10 10 10 10 Y N Y N 40.63% 54.43% Numerator (yes answers) 16 24 21 21 % of children 6-23m who received a dose of Vitamin A in the last 6 months (Mother’s recall). (VitaminAChildSixMonths1) Denominator (Total)-Sample size 24 24 24 24 76% 85% 81% 18 18 18 18 N Y Y Y 85.42% 90.60% Numerator (NORMAL WEIGHT) 20 19 18 19 Additional Indicator:Percentage of children age 0-23 months who are underweight (>-2SD for the median weight for age, according to WHO/NCHS reference population) (WFA) Denominator (Total)-Sample size 24 24 24 24 27% 17% 22% 16 16 16 16 Y Y Y Y 20.83% NA Numerator (yes answers) 15 16 17 10 % of children 12-23 months who received DPT3 before they reached 12 months by the time of the interview card verified. (dpt3donebydate) Denominator (Total)-Sample size 24 24 24 24 31% 50% 41% 7 7 7 7 Y Y Y Y 60.42% 57.89% 2 Numerator (yes answers)yy35.5 3%52.50% 10 10 10 8 % of children aged 12-23 months who are fully vaccinated (received BCG, DPT3, OPV3, and measles vaccines) by 12 months of age (fullimmcvrgbyDATE) Denominator (Total)-Sample size 24 24 24 24 19% 40% 30% 3 3 3 3 Y Y Y Y 39.58% 35.53% Numerator (yes answers) 12 16 18 18 % of mothers with children age 0-23 months who were protected against Tetanus before the birth of the youngest child. (Protected refers to receiving at least 2 TT or Td injections before the birth of the youngest child sufficiently close to that birth to provide protection.) (Tetanusgood) Denominator (Total)-Sample size 24 24 24 24 61% 80% 71% 15 15 15 15 N Y Y Y 66.67% 87.18% Numerator (yes answers) 18 16 21 21 % of children 0-23 months with diarrhea in the last two weeks who received Oral Rehydration solution (ORS) and/or recommended home fluids. (ORTUse) Denominator (Total)-Sample size 24 24 24 24 74% 85% 80% 16 16 16 16 Y Y Y Y 79.17% 85.51 Numerator (yes answers) 16 13 15 18 % of children 0-23 months with diarrhea in the last two weeks who were offered more fluids during the illness. (diarrheaMoreDrink) Denominator (Total)-Sample size 24 24 24 24 52% 70% 61% 13 13 13 13 Y Y Y Y 64.58% 71.62% Numerator (yes answers) 2 2 0 3 % of households of children 0- 23 months that treat water effectively. (waterTx Denominator (Total)-Sample size 24 24 24 24 22% 40% 31% 4 4 4 4 N N N N 7.29% 18.67% Numerator (yes answers) 9 9 14 10 % of mothers of children 0-23m who live in households with soap or ash at the place for hand washing and that washed their hands with soap or ash at least 2 of the appropriate times during a 24 hour recall period. (approphandwashing) Denominator (Total)-Sample size 24 24 24 24 19% 40% 30% 3 3 3 3 Y Y Y Y 43.75% 28.00% 3 Numerator (NORMAL WEIGHT) 20 19 18 19 Additional Indicator:Percentage of children age 0-23 months who are underweight (>-2SD for the median weight for age, according to WHO/NCHS reference population) (WFA) Denominator (Total)-Sample size 24 24 24 24 27% 17% 22% 16 16 16 16 Y Y Y Y 20.83% NA Numerator (yes answers) 22 23 21 14 % of children age 0-23 months with chest-related cough and fast/difficult breathing in the last two weeks who were taken to an appropriate health provider. (HCPCCough) Denominator (Total)-Sample size 24 24 24 24 43% 65% 54% 10 10 10 10 Y Y Y Y 83.33% 60.56% Numerator (yes answers) 12 15 18 20 % of children 0-23 months who slept under an insecticide-treated bed net the previous night. (ChildNet) Denominator (Total)-Sample size 24 24 24 24 18% 35% 27% 3 3 3 3 Y Y Y Y 67.71% 22.08% Numerator (yes answers) 12 13 6 16 % of children 0-23 months with a febrile episode that ended during the last two weeks who were treated with an effective anti-malarial drug within 24 hours after the fever began. (FeverPropMed) Denominator (Total)-Sample size 24 24 24 24 13% 50% 32% 4 4 4 4 Y Y Y Y 48.96% 40.58% Numerator (yes answers) 20 24 23 22 % of mothers of children 0-23m who had at least one prenatal visit prior to the birth of her youngest child less than 24 months of age Denominator (Total)-Sample size 24 24 24 24 56% NA CF 21 21 21 21 N Y Y Y 92.71% NA Numerator (yes answers) 14 20 16 14 % of mothers of children 0-23m who had at least three prenatal visit prior to the birth of her youngest child less than 24 months of age Denominator (Total)-Sample size 24 24 24 24 NA NA CF 14 14 14 14 Y Y Y Y 66.67% NA 4 Numerator (yes answers) 7 6 9 8 % of mothers of children 0-23m who have a maternal health card Denominator (Total)-Sample size 24 24 24 24 NA NA CF 4 4 4 4 Y Y Y Y 31.25% NA Numerator (yes answers) 11 11 8 15 % of mothers of children 0-23m able to report at least one known prenatal danger signs indicating the need to seek health care? Denominator (Total)-Sample size 24 24 24 24 NA NA CF 9 9 9 9 Y Y N Y 46.88% NA Numerator (yes answers) 24 24 21 22 % of mothers of children 0-23m who report that they would first go to an appropriate health provider if they experienced ony prenatal danger symptoms Denominator (Total)-Sample size 24 24 24 24 NA NA CF 21 21 21 21 Y Y Y Y 94.79% NA Numerator (yes answers) 23 22 16 22 % of mothers of children 0-23m who report that they would walk to the nearest appropriate health provider Denominator (Total)-Sample size 24 24 24 24 NA NA CF NA NA NA NA NA 86.46% NA Numerator (yes answers) 2 3 9 2 NA % of mothers of children 0-23m who report that they would take a car to the nearest appropriate health provider Denominator (Total)-Sample size 24 24 24 24 NA CF NA NA NA NA NA 16.67% NA Numerator (yes answers) 1 2 0 0 % of mothers of children 0-23m who report that they would ride a motorcycle to the nearest appropriate health provider Denominator (Total)-Sample size 24 24 24 24 NA NA CF NA NA NA NA NA 3.13% NA Numerator (yes answers) 0 0 0 0 % of mothers of children 0-23m who report that they would take an oxcart to the nearest appropriate health provider Denominator (Total)-Sample size 24 24 24 24 NA NA CF NA NA NA NA NA 0.00% NA Numerator (yes answers) 0 1 0 0 % of mothers of children 0-23m who report that they would take a canoe to the nearest appropriate health provider Denominator (Total)-Sample size 24 24 24 24 NA NA CF NA NA NA NA NA 1.04% NA 5 Numerator (yes answers) 0 0 0 0 % of mothers of children 0-23m who report that they would take another form of transport to the nearest appropriate health provider Denominator (Total)-Sample size 24 24 24 24 NA NA CF NA NA NA NA NA 0.00% NA Numerator (yes answers) 6 7 6 5 % of children aged 0-23 months whose delivery was in an appropriate health facility Denominator (Total)-Sample size 24 24 24 24 NA NA CF 2 2 2 2 Y Y Y Y 25.00% NA Numerator (yes answers) 7 6 7 3 % of children aged 0-23 months whose delivery was attended by a skilled health personnel Denominator (Total)-Sample size 24 24 24 24 21% NA CF 2 2 2 2 Y Y Y Y 23.96% NA Numerator (yes answers) 9 7 4 % of mothers of children 0-23m who received a post-natal visit from an appropriately trained health worker within three days after birth Denominator (Total)-Sample size 24 24 24 7 24 6.0% NA CF 3 3 3 3 Y Y Y Y 28.13% NA Numerator (yes answers) 9 9 4 13 Percentage of children age 0-23 months who received a post￾natal visit from an appropriately trained health worker within three days after birth Denominator (Total)-Sample size 24 24 24 24 7.0% NA CF 6 6 6 6 Y Y Y Y 36.46% NA Numerator (yes answers) 0 0 0 1 % of mothers able to report at least two known neonatal danger signs Denominator (Total)-Sample size 24 24 24 24 NA NA CF too low to calculate 1.04% NA Numerator (yes answers) 20 23 21 23 % of mothers of children 0-23m who have heard of AIDS Denominator (Total)-Sample size 24 24 24 24 81.9% NA CF 21 21 21 21 N Y Y Y 90.63% NA Numerator (yes answers) 10 6 11 9 % of mothers of children 0-23m who can correctly name at least 2 ways to reduce the risk of HIV/AIDS Denominator (Total)-Sample size 24 24 24 24 19% NA CF 6 6 6 6 Y Y Y Y 37.50% NA % of mothers of children 0-23m Numerator 7 10 11 8 NA NA CF 6 6 6 6 Y Y Y Y 37.50% NA 6 who have heard of VCT (yes answers) Denominator (Sample size) 24 24 24 24 Numerator (yes answers) 6 3 9 7 % of mothers of children 0-23m who know of a place where people can go to get VCT Denominator (Sample size) 24 24 24 24 NA NA CF 3 3 3 3 Y Y Y Y 26.04% NA Numerator (yes answers) 4 1 4 4 % of mothers of children 0-23m who know of a place where people can go to get treatment for HIV Denominator (Sample size) 24 24 24 24 NA NA CF too low to calculate NA 13.54% NA Numerator (yes answers) 3 1 0 1 % of mothers of children 0-23m who know that there are special medications that can be given to a pregnant woman infected with the AIDS virus to reduce the risk of mother-to-child transmission Denominator (Total)-Sample size 24 24 24 24 NA NA CF too low to calculate NA 5.21% NA Numerator (yes answers) 4 1 2 1 % of mothers of children 0-23m who know of a place in the community to go to get these special medications that can be given to a pregnant woman infected with the AIDS virus to reduce the risk of mother-to￾child transmission Denominator (Total)-Sample size 24 24 24 24 NA NA CF too low to calculate NA 8.33% NA Numerator (yes answers) 2 0 2 0 % of mothers of children 0-23m who personally know someone who has been denied involvement in social events, religious services, or community events in the last 12 months because he or she is suspected to have the AIDS virus or has the AIDS virus? Denominator (Total)-Sample size 24 24 24 24 NA NA CF too low to calculate NA 4.17% NA Numerator (yes answers) 13 6 12 11 % of mothers of children 0-23m who believe that people with the AIDS virus should be ashamed of themselves Denominator (Sample size) 24 24 24 24 NA NA CF 7 7 7 7 Y N Y Y 43.75% NA Numerator (yes answers) 11 12 15 13 % of mothers of children 0-23m who believe that a person with the AIDS virus should be forced to leave the community. Denominator (Sample size) 24 24 24 24 NA NA CF 10 10 10 1 0 Y Y Y Y 53.13% NA Annex 2: LQAS Summary Table of Client Satisfactions Indicators LQAS SUMMARY TABLE OF CLIENT SATISFACTION INDICATORS Supervision Areas Target for this LQAS Combined Frequency for Project Area 1 2 3 4 Base￾line Final Target Numerator (yes answers) % of mothers with children age 0-23 91 92 75 90 months who have taken the child to the nearest HF in the last 12 months Denominator (Total)- Sample size 93 93 77 93 NA TBD CF 97.75% Numerator (yes answers) % of mothers who did not take their child 1 0 0 0 to the HF in the last 12 months due to distance Denominator (Total)- Sample size 93 93 77 93 NA TBD CF 0.28% Numerator (yes answers) % of mothers who did not take their child 0 0 1 0 to the HF in the last 12 months due to no HW available Denominator (Total)- Sample size 93 93 77 93 NA TBD CF 0.28% Numerator (yes answers) % of mothers who did not take their child 0 0 1 2 to the HF in the last 12 months due to lack of medicines Denominator (Total)- Sample size 93 93 77 93 NA TBD CF 0.84% 2 Numerator (yes answers) % of mothers who did not take their child 0 0 0 0 to the HF in the last 12 months due to poor treatment Denominator (Total)- Sample size 93 93 77 93 NA TBD CF 0.00% Numerator (yes answers) % of mothers who did not take their child 1 1 0 1 to the HF in the last 12 months due to other reasons Denominator (Total)- Sample size 93 93 77 93 NA TBD CF 0.84% Numerator (yes answers) % of mothers of children aged 0-23m who 81 84 75 77 felt the time waited to be seen at the HF was reasonable (exc., good, Acceptable) Denominator (Total)- Sample size 93 93 77 93 NA TBD CF 89.04% Numerator (yes answers) % of mothers of children aged 0-23m who 88 82 71 85 felt the treatment received for their child's illness at the HF was reasonable (exc., good, fair) Denominator (Total)- Sample size 93 93 77 93 NA TBD CF 91.57% Numerator (yes answers) % of mothers of children aged 0-23m who 90 89 74 90 were likely or very likely to return to the same facility the next time the child is sick Denominator (Total)- Sample size 93 93 77 93 NA TBD CF 96.30% Annex 3: Additional Indicators Collected during Midterm LQAS Supervision Areas Decision Rule Number X Axis is Target for this LQAS Y Axis is the Total sample Size Decision Rule (Yor N) for each Supervision Area CF for Project Area Sep 2008 1 2 3 4 Base￾line Final Target Benchmark for this LQAS 1 2 3 4 1 2 3 4 Numerator (yes answers) 15 16 17 10 % of children 12-23 months who received DPT3 before they reached 12 months by the time of the interview card verified. (dpt3donebydate) Denominator (Total)-Sample size 24 24 24 24 31.00% 50% 41% 7 7 7 7 Y Y Y Y 60.42% Numerator (yes answers) 20 24 20 17 % of children 12-23 months who received BCG before they reached 12 months by the time of the interview card verified. (BCGdonebydate) Denominator (Total)-Sample size 24 24 24 24 48.60% 83.50% Numerator (yes answers) 14 18 19 12 % of children 12-23 months who received OPV3 before they reached 12 months by the time of the interview card verified. (OPV3donebydate) Denominator (Total)-Sample size 24 24 24 24 36.00% 65.29% % of children 12-23 months who received immunization against meales before they reached 12 months by the time of the interview card verified. (MEASLESdonebydate) Numerator (yes answers) 12 13 11 12 25.20% 49.98% 2 Denominator (Total)-Sample size 24 24 24 24 Numerator (yes answers) 12 13 6 16 % of children 0-23 months with a febrile episode that ended during the last two weeks who were treated with an effective anti-malarial drug within 24 hours after the fever began. (FeverPropMed)****** Denominator (Total)-Sample size 24 24 24 24 12.5% 50% 32% 2 2 2 2 Y Y Y Y 48.960% Numerator (yes answers) 22 21 13 20 % of children 0-23 months with a febrile episode that ended during the last two weeks who who were brought to a health facility for treatment. (FeverHCPC) Denominator (Total)-Sample size 24 24 24 24 79.17% Numerator (yes answers) 20 18 18 18 % of children 0-23 months with a febrile episode that ended during the last two weeks who who were brought to a health facility for treatment within 24 hours. (FeverProp) Denominator (Total)-Sample size 24 24 24 24 77.10% Numerator (yes answers) 16 17 14 21 % of children 0-23 months with a febrile episode that ended during the last two weeks who were treated with an effective anti-malarial drug (FeverMed) Denominator (Total)-Sample size 24 24 24 24 70.83% Annex 6 Grand Cape Mount Child Survival Project Improved Child Health in a Transitional State through IMCI Grand Cape Mount County, Liberia October 2006 – September 2010 In Partnership with Liberia Ministry of Health and Social Welfare Grand Cape Mount County Health Team Christian Health Association of Liberia Midterm Rapid Health Facility Assessment Grand Cape Mount County Liberia September 2008 Table of Contents Acronyms 1 Executive Summary 2 Chapter 1 Program Overview 6 Chapter 2 Purpose of the Health Facility Assessment and Methodology 12 Chapter 3 Main Findings: Accomplishments and Priorities 17 Chapter 4 Action Plan/Conclusions/Recommendations 28 Annexes: Annex 1: List of Health Facilities Annex 2: Training of Survey Team 1 ACRONYMS ANC Antenatal Care ARI Acute Respiratory Infection CHAL Christian Health Association of Liberia CHP Community Health Promoter CHT County Health Team CHW Community Health Workers C-IMCI Community IMCI CS Child Survival CORE Collaborations and Resources Group CSHGP Child Survival and Health Grant Program CSP Child Survival Project EPI Expanded Program of Immunizations GCM Grand Cape Mount County GCMCSP Grand Cape Mount County Child Survival Project GIK Gifts-in-kind HF Health Facility HFA Health Facility Assessment HHP Household Health Promoter HW Health Worker IMCI Integrated Management of Childhood Illnesses M&E Monitoring and Evaluation MNC Maternal Newborn Care MOHSW Liberia Ministry of Health and Social Welfare MTI Medical Teams International MTI/Liberia Medical Teams International/Liberia N Sample size NDS National Drug Service ORS Oral Rehydration Salts PHC Primary Health Care RHFA Rapid Health Facility Assessment USAID United States Agency for International Development 2 I. Executive Summary In October 2006, Medical Teams International began the Grand Cape Mount Child Survival Project: Improved Child Health in a Transitional State through IMCI aimed at reducing child and maternal mortality and morbidity in Grand Cape Mount (GCM), Liberia. A large part of the project is to increase the capacity of 31 health facilities (HFs) and accompanying staff in GCM County. In September 2008, the Child Survival team carried out a Rapid Health Facility Assessment as part of a mid term assessment process to assess the projects progress in building the capacity of the 31 health facilities present in GCM County. The assessment measures capacity in the areas of access and inputs, processes, and performance. The team visited 18 of the 31 PHC facilities. Five teams of three individuals, consisting of one supervisor and two enumerators, were developed using the same staff that performed the baseline HFA. This staff was comprised of MTI Supervisors and CHT members. Each team contained at least two members with experience in health. All 18 HFs were scheduled to be surveyed in 4 days. It was necessary to return to 4 HFs to include the needed 6 children, as these 4 facilities were originally visited on slow days due to market days and other factors. Therefore, the entire HFA was completed in 5 consecutive days. The project has had a significant impact on improving access and care at the Health Facilities through IMCI and C-IMCI. The major areas of concern at the outset of the project, identified through the initial HFA performed at baseline, were in the areas of service availability, supervision, health worker performance and the referral of sick children in the community to the HF. At baseline, only 9.1% of HF clinical encounters occurred in which all assessment tasks were made by HWs for sick child. Through IMCI training of HWs by the project, coupled with alternating, bi￾monthly supervisory visits and mentoring visits by MTI Supervisors, this indicator has now been increased to 28%, with the percentage of assessment tasks being performed on average at each HF at 66%. No facilities were providing growth monitoring, which is the reason the indicator for the percentage of HF that offer child, immunization, and growth monitoring services was 0% at baseline. As part of the IMCI training, growth monitoring was to be implemented at each HF. 44% of HFs are now providing growth monitoring. At baseline only 4.5% of HF had received external supervision at least once in the previous 6 months. The project has implemented a regular system of supervision, and has increased this indicator greatly. The indicator has now changed to the percentage of HFs that have received external supervision in the last 3 months, versus 6 months, and still the project attained 56%. Household Health Promoters (HHPs) have been trained by the project and are now referring sick children to the facilities. It is encouraging that 100% of the HFs now offer ANC services at least once a week. In addition, 61% of health facilities have now received in-service or pre-service training in child health and in maternal neonatal care in the last 12 months. With regard to medications, 61% of HFs had all first line medications for child health in stock, with the percentage of HF attainment at 90% (meaning each HF had an average of 90% of the immunizations in stock). Also, 44% of HF were found to have all nationally mandated immunizations in stock, with the percentage of HF attainment at 69%. A summary of the 12 core indicators measured in the RHFA are as follows: 3 ACCESS (INPUTS) 1. Service Availability: At midterm, 44% of HFs were deemed to offer all 3 basic child health services. This is increased from baseline, in which 0% of HFs offered all 3 basic child health services. 2. Staffing: 56% of all staff who provide clinical services were present on the day of the midterm survey, and the average HF attainment was 77% (meaning the average HF had 77% of its staff present). This indicator has been changed since baseline, and therefore the midterm finding will be used to track future performance. 3. Infrastructure: All essential health infrastructure was present on the day of the survey in 33% of the HFs, with the average HF attainment at 45%. The limiting factors are emergency transport, which was available in only 6% of the HFs, electricity, which was available in only 11% of the HFs, and emergency communication, which was available in only 28% of the HFs on the day of the survey. 4. Supplies: a. Supplies – Child: While only 44% of HFs had all of the essential supplies to support child health, the average HF attainment of these essential supplies was 79% . This is a decrease from baseline, where 90.9% of HFs had the entire essential infrastructure required for child health. This is due to the fact that infant scales are available in only 67% of the facilities, and timers, pitchers for ORS and a cup or spoon for ORS are only available 78% of the time. Investigation into this revealed that at the start of IMCI all HFs were provided with the proper supplies for administering ORS and all facilities had functional infant scales, so this explains much of the decline. b. Supplies-MNC: None (0%) of the HFs have all of the essential supplies available to support maternal-newborn services. This is mainly due to the fact that only 6% of HFs have functioning vacuum extractors, and only 11% of facilities have partographs. c. Supplies-ANC: Similarly, 0% of the HFs have all of the essential supplies to support antenatal care. No facilities have hemoglobin or syphilis tests available and only 6% of facilities have urine albumin test strips available. 5. Drugs: a. Child: At midterm, 94% of facilities have first line drugs for pneumonia and malaria, and 89% had ORS available, showing an improvement in these areas. This compares to baseline where 64% of HFs had all first line medications available, which at that time was defined as ORS, a first line oral antibiotic for pneumonia, and a first line antimalarial. Using the new RHFA indicator, which also includes Vitamin A and a first line antibiotic for dysentary, 61% of HFs have all of the first line medications for child health available. This is comparable to baseline despite the fact that this new indicator is more rigid than the baseline indicator. The limiting factor in drug supply is now Vitamin A, as only 72% of the facilities have Vitamin A in stock 4 b. MNC: Only 33% of HFs had all of the essential delivery and neonatal drugs present on the day of the survey. The limiting factor in this indicator is Oxytocin, which is available in only 39% of the facilities. PROCESSES 6. Information Systems a. Child: The average facility has 82% of the elements required in having up to date records available for child care, with 39% of HFs have all elements. At baseline, 0% of the facilities had all of the elements in place b. ANC: The average facility has 81% of the elements required in having up to date records available for antenatal care, with 39% of HFs have all elements. At baseline, 0% of the facilities had all of the elements in place. 7. Training: 61% of the HF had HW who reported receiving in-service or pre-service training in both maternal neonatal care and child health in last 12 months. The baseline finding of 79% was for the indicator “receiving any in-service or pre-service education relevant to their work in last 12 months” versus these more rigorous indicators. However, these numbers are comparable or higher than the baseline numbers when you consider that MNC training alone was provided to 72% of HW. 8. Supervision: External supervision has increased since baseline, from 4.5% of HF receiving external supervision at least once in the last 6 months, to 56% of HFs receiving external supervision in the last 3 months PERFORMANCE 9. Utilization of Curative services: The percentage of HF with > 1 sick child encounter per child under the age of 5 in GCM County was 5.0%. This is an increase from the baseline of 1.89%. 10. HW Performance (Assessment): The midterm found that the average facility was performing 66% of the key assessment tasks. Therefore, only 28% of the facilities were found to routinely perform all key assessment tasks. This is an improvement over the baseline, however, where only 9% of facilities were found to routinely perform all key assessment tasks. 11. HW Performance (Treatment): It is encouraging that in 72% of the HFs, the treatment given by the HW is appropriate to diagnosis (meaning provided in at least 5 of the 6 observed cases). This is a marked improvement over the baseline frequency of 45.5%. 12. HW Performance (Counseling): The weakest area noted, in regards to HW performance, is in the area of counseling the caretaker on the proper method of administering the medicines prescribed. Only 11% of HFs were properly instructing caretakers in how to correctly administer drugs prescribed for their child. This is a decrease from baseline, where it was reported that 48.5% of caretakers could correctly describe how to administer all prescribed drugs. The major factor contributing to this low percentage is the fact that the most experienced HW, who has been trained by the project in IMCI, was often not at the 5 facility on the day of the survey. New regulations will be implemented in the HFs, by the CHT, to ensure that HWs are present during all normal HF hours. In addition, the project will follow up with the HWs trained in IMCI to ensure that they are committed to providing this training to the other pertinent staff in their HF, as agreed upon at the start of the IMCI training sessions. 6 CHAPTER 1 Program Overview Project Area and Description: Medical Teams International is implementing the Grand Cape Mount County Child Survival Project (GCMCSP) in Grand Cape Mount County (GCM) located in southwestern Liberia. The goal of the GCMCSP is to reduce morbidity and mortality of children under five and women of reproductive age within Grand Cape Mount County, Liberia. The interventions and level of effort are: Nutrition 30%, PCM 20%, CDD 20%, Malaria 20%, and EPI 10%. All interventions are being implemented within the recently adopted IMCI framework for Liberia and in accordance with MOHSW policy. Implementing partners for this project include the Christian Health Association of Liberia (CHAL), a local NGO responsible for implementing the Care Group and community mobilization activities, and the County Health Team (CHT) which is charged with the responsibility of coordinating the delivery of health services throughout the county. Working in partnership with the MOHSW, the CHT, and CHAL, MTI/Liberia is implementing a project that aims to improve the health of village communities in GCM County through strengthening HFs and the MOHSW’s ability to address community health needs. This is accomplished through targeted behavior change at the household level, community mobilization, improving quality of care and access at the clinic level, and institutional capacity building for MTI and partners. Quality of care and access at the clinic level At the county level, the MOH has created County Health Teams as a means of de￾centralizing management and encouraging good governance. The CHT is comprised of the County Health Officer, Community Director, Administrator, Nursing Director, Medical Director, Financial Officer, and a Logistics Officer. The decentralization process was on hold during recent war years, but is now resuming. At baseline there were a total of 17 functioning government health facilities out of a potential 31. This number has now increased to 31 at midterm. In the county, there one physician, 14 registered nurses, 20 certified midwives, 19 physician’s assistants, and 10 licensed practical nurses. The provision of essential drugs is still problematic. Public HF drugs are supplied through the National Drug Service (NDS) and mainly financed through international donors. In 2001 a cost sharing scheme was introduced; subsidized drugs were charged to patients at 50% of cost and drug revolving funds were established in 213 HFs. Subsequently, the system collapsed and currently all drugs and treatment are free of charge. The supply of drugs is becoming more stable at midterm through donor intervention. For example, 7 supplies of vaccine, ACT, Vitamin A, and ORS are currently on hand; and UNICEF and WHO are rehabilitating five drug supply depots and the distribution system. After fourteen years of instability, health staff knowledge regarding recent medical advances had not been updated at the start of the CSP. Most health staff had not received sufficient in-service training in the last decade. Supervision was inadequate, focusing on frequency of visits at the expense of structure and content, and collecting statistics rather than a process of sharing experiences, reinforcing skills, assessing needs, following up on achievements, and resolving conflicts. Access to health care was an over-riding issue. At baseline, access was constrained by the lack of adequate functional facilities, poor maintenance of facilities, an insufficient number of competent, well trained and motivated professionals, shortage of logistical support, and chronic shortages of drugs and medical supplies. Rural HFs are generally open from 8 a.m. to 4 p.m. On the demand side, utilization was constrained by negative cultural influences, long distances, and poor public transportation. The main means of transportation to a HF is by walking. It is estimated that less than 10% of the population have reasonable access to any kind of healthcare, a decrease from 30% before the war. The number of facilities in GCM County has risen from 14 to 30, which has greatly increased accessibility. The objective of the GCMCSP is to improve the quality of care and access at the health facility level by implementing IMCI through training, mentoring, supportive supervision, and systems development in referral and logistics. The project has implemented the three components of IMCI through the following activities: Component 1: Improving case management skills of the health facility staff: Implementing IMCI in 31 clinics and one health center by supporting skills application of health workers in assessment, classification, standard case management, referral, and counseling Facilitating the training of HF staff, MTI, CHAL, CHT in IMCI in the 11days IMCI training utilizing the national MOH training team Provide an orientation on IMCI for HF support staff, government, and traditional leaders Training HF staff in complementary topics to improve clinic management and quality of care such as counseling, supportive supervision, management, logistics, M&E, rational drug use, and BCC Facilitating an exchange visit for MTI, CHAL, and the MOH to a Child Survival project in Sierra Leone and to the Liberia Improved Community Health project Exploring coordination with a Liberian training institution for certification of staff after training Quality of care will be monitored and evaluated by HF staff, CHT, and MTI following IMCI training. Quality of care indicators will be developed with the IMCI task force, coordinated by WHO. 8 Component 2: Improving the overall health system: Funds and GIK donations have been made available to improve the availability of drugs and supplies for implementing IMCI. Coordinating with other agencies to improve drug supply (NDS, UNICEF, WHO, other NGOs) Providing training and support to HF staff to more effectively utilize logistics systems The referral system has been instituted utilizing the HHPs and CHPs throiugh the Care Group model. Implementing a mentoring and supportive supervision system for reinforcement of new skills Strengthening the existing HIS, with a focus on use of information for decision making and introduction of new tools such as LQAS and quality of care checklists Supporting monthly health sector County Coordinating Committee meetings chaired by the CHT to improve coordination between government and other NGOs working in the county Component 3: Improving family and community health care practices: C-IMCI is being phased into all districts of GCM and will include three elements: Element 1: Improving partnerships between health facilities and the communities they serve Clinic staff participate in training and supervising HHPs with assistance from MTI and CHAL. Developed a referral system to improve referral of sick children from HHPs to 1st level HFs Established monthly meetings at the HFs for CHWs and TBAs for information sharing, continuous education, and problem solving HHPs/CHCs will encourage families to use HF services and ensure equity of access. Element 2: Increasing appropriate and accessible care and information from community providers Training approximately 250 HHPs and 150 TBAs in 131 communities in C￾IMCI including management of illnesses, danger signs and prompt care￾seeking, maternal and child nutrition including micronutrients and breastfeeding, environmental health and hygiene, promotion of EPI (including TT), community HIS, communication/adult learning, home visits, and the 16 key family practice messages Providing assistance to HHPs/TBAs/CHCs in applying the above training through supportive supervision by HF/CHT/project staff Developing a plan for motivating and sustaining HHPs in coordination with the CHCs Encouraging the establishment of community-based emergency transport systems and the formation of Susu and/or Savings Clubs to fund health emergencies. Advocate with the MOH for the establishment of a supply of essential drugs at the community level (ORS, Vitamin A, and iron) 9 Element 3: Integrating promotion of key family practices critical for child health and nutrition Promote community education by CHW/TBA/CDC/clinic staff utilizing effective integrated IEC/BCC methodologies for the promotion and adoption of key household practices The increased capacity of the health facilities and health facility staff will improve the quality of treatment and the outcomes for all of the major childhood diseases causing morbidity and mortality in GCM County: Respiratory Infections: The prevalence of ARI (children with a chest related cough and fast/difficult breathing) in children aged 0-23 months in GCM County was 51.7%. At baseline children with ARI were treated by seeking advice or treatment at a qualified health facility at a rate of only 43.2%. Also, children with ARI were treated by either seeking advice/treatment at a qualified health facility or treated with an antibiotic at a rate of only 49.7%, which revealed that most mothers were not routinely seeking help when their child was sick, and did not understand the importance of seeking timely medical advice or seeking proper antibiotic treatment for their children who are ill with cough and respiratory infection. To address these issues, the project utilizes the Case Management Training on IMCI to improve the knowledge and skills of health workers in the assessment and treatment of infants and young children with respiratory problems. Ensuring that antibiotics are readily available at the HF level is also a priority of the project through the GIK and working along with the CHT and other partners (NGOs) in the county who support other health facilities. Diarrhea: While diarrhea is a major cause of morbidity in Liberia and GCM County, due to seasonal fluctuations the prevalence of diarrhea at the time of the baseline KPC survey was 30.1%, which is somewhat lower than found at other times of the year. Key home practices that are emphasized by this project include continued breastfeeding and fluids as well as the frequent feeding of small amounts of food, and catch-up feeding. The project is coordinating with the CHT to ensure a regular supply of ORT at both the level of the health facilities and at the community level. Malaria: The prevalence of fever in children aged 0-23 months in GCM County was found to be quite high in the baseline survey at 70%. Part of the reason for this can be explained by the fact that at baseline only 17.7% of children less than 24 months were sleeping under an insecticide treated bed net. Of the children who had a fever that had ended in the 2 weeks prior to the survey, only 20.5% were brought to a qualified health facility within 48 hours of the start of the fever. Only 3.6% of mothers treated their child with an effective anti-malarial drug within 24 hours after the fever began. Project activities for malaria are focused on the prevention of malaria with the distribution of Insecticide Treated Nets (ITNs) and community education on proper use of these nets through Care Groups and home visits. The project has begun to work with the MOHSW Malaria Control Division, NDS and GIK sources to ensure adequate ACT pharmaceutical availability at HFs in the county and training health care workers in their use. GIK will also 10 supply Fansidar for use in IPT as well as drugs for malaria treatment, where gaps exist, according to MOHSWSW protocols. Immunization: Immunization rates are low in GCM County, with full EPI coverage by 12 months of age found to be only 18.9% at baseline in children aged 12-23 months. EPI Health System Performance, measured by card verified rates of DPT3 vaccination by 12 months of age, was only 30.6%. MTI and its partners are utilizing the IMCI and C-IMCI approach at the 31 functioning health facilities in GCM County in order to improve vaccination rates and utilization of vaccination services by the community. Demand for EPI services is being created through CHWs, CHPs, and HHPs, including messages to the community about bringing children in for immunization. The project provides logistical support for National Immunization days, advocates for the implementation of routine EPI services, and reduces missed opportunities for immunizations through IMCI at the HF level. The project is presently addressing gaps in present immunization service delivery, including a lack of training of MOHSW staff and gaps in the cold chain and other equipment at the HF level. In addition, the MOHSW in conjunction with MTI/Liberia project staff is conducting immunization training with HF staff in logistics and drug forecasting, and injection technique and safety. The percentage of mothers at baseline survey receiving at least two immunizations in GCM County was only 61.3%, indicating some utilization of antenatal care services and tetanus immunizations being performed by antenatal health care staff when these services were utilized, but this rate needed improvement to ensure protection of mothers and children from tetanus. The project is addressing this problem by strengthening the MOHSW’s ability to address this need through enhancing the capacity of antenatal care staff and providing consistent preventative-based services that include immunization services. Nutritional Status: The nutritional status of children aged 0-23 months is of major concern in GCM County. The Baseline KPC Survey conducted in November 2006 found the overall rate of under-nutrition to be 27.1%, with 14.1% moderately underweight and 13.1% severely underweight. The survey also showed low rates of immediate and exclusive breastfeeding. Mothers were immediately and exclusively breastfeeding their newborns at a rate of only 33.7%. Inappropriate complimentary feeding practices also contribute to the high rates of undernutrition. The Child Survival Project through its case management training on IMCI of health workers and Community IMCI trainings of HHPs continue to promote immediate initiation of breastfeeding, exclusive breastfeeding, and proper complimentary feeding for children aged 6-23 months. The introduction of IMCI at health facilities has also improved both growth monitoring and nutrition counseling services. Antenatal Care: The Maternal Mortality rate is 578 per 100,000, one of the highest in the world. The principal causes of mortality are anemia, postpartum hemorrhage, and sepsis, as well as pregnancy related complications, toxemia and infections related to unsafe abortions. Thirty two percent of deaths occur in pregnancy, 34% during delivery, and 26% postpartum. Maternal morbidity in GCM is caused mainly by malaria and by complications during labor/delivery (GCM Surveillance 2005). Prenatal care levels were quite low, at 56.3% at baseline, but have now improved to 92.71% at midterm. In addition, delivery care was provided by a skilled health professional at only 21.3% at baseline. This rate continues to be 11 low at midterm, at 23.96%. At baseline, postpartum visits with a skilled professional occurred at an alarmingly low rate, which was 6.0% for mothers and 7.0% for newborns. By midterm these numbers have improved, due to health education messages provided in the Care Groups and home visits stressing the utilization of health services for child and antenatal care. Postnatal visits within 3 days of the birth of the child are now occurring at a rate of 28.13% for the mother and 36.46% for the child. The CSP, through its case management training on IMCI of health workers and Community IMCI trainings of HHPs continue to promote prenatal, delivery, and postnatal care at the HF. 12 CHAPTER 2 Purpose of the Rapid Health Facility Assessment and Methodology The objective of a Health Facility Assessment is to collect both quantitative and qualitative data regarding the health system in the project area, particularly the primary health care (PHC) facilities (level 1) providing maternal, neonatal, and child health (MNCH) services. The new Rapid Health Facility Assessment (RHFA) used to collect this data has been designed and recently upgraded by CSTS, and is now called RHFA Version 2.1. The RHFA is designed to be rapid and cost effective, and is designed to be used at the local level to devise strategies, with the MOH entity present in the project area, and to improve the delivery of integrated child health services. The baseline assessment was conducted before IMCI training had begun, and prior to any interventions aimed at health facilities so that an integrated strategy to improve the quality of heath care could be implemented. The midterm RHFA will measure progress in the 12 core areas of HF capacity, and will compare these results to baseline so that areas of needed improvement may be identified. The R￾HFA version 2.1 assessment includes the first six children, under the age of five, entering the health facility on the day of the survey with diarrhea, fever, or cough and examines three major areas of health care delivery. Case management: Does the healthcare worker (HW) assess, diagnose, and treat children with diarrhea, fever (malaria), and ARI properly? Does the HW explain follow up care to the caretaker well? Health facility infrastructure: Does the health facility have the necessary equipment, supplies, medications, and privacy to perform adequate MCH services? Management (Processes): Are the proper management processes being followed in the health facility (supervision, record keeping, and continuation of training)? The R-HFA focuses on the delivery of care for the most important causes of infant and child morbidity and mortality, which include: diarrhea, acute respiratory infections (ARI), malaria, measles, and malnutrition. The purpose of collecting this data is to allow the project, in conjunction with the MOH, DHO, and other health workers in the project area, to determine gaps in service and prioritize their response in order to provide essential, integrated health services. There are four main modules in the R-HFA, with a fifth optional module. All are formatted in Excel for ease of use. a. Observation Checklist for sick child care: To observe the HW in the assessment, diagnosis, and treatment of six consecutive cases of care of children under the age of five with fever, diarrhea, or breathing difficulty. The HW is assessed for adherence to the national (IMCI) protocol for assessment, classification, and treatment of childhood illness. b. Client (Caretaker) Exit Interview: To assess whether the caretaker has the correct knowledge of how to administer drugs given for diarrhea, malaria, and/or breathing difficulty (used a proxy for adequate counseling), and whether the caretaker knows under what circumstances the child is to return to the clinic. 13 c. Health Facility Checklist: To assess the presence of a minimal level of infrastructure, equipment, supplies, and medications. d. Health Worker Interview and Record Review: To assess the staffing, MNCH services offered, and also assess the frequency of training, supervision, and other key processes. e. CHW Survey and Checklist (optional): To collect data on CHWs regarding six of the twelve health facility core indicators (through examination of registers). Selecting the Sampling Frame The sampling methodology has been revised by CSTS to use a quality assurance type approach similar, but not identical, to that used in LQAS. The new methodology dictates that at least 80% of the health facilities perform adequately, according to the indicators chosen in the HFA, for the project area to pass. This is called the performance benchmark. Also, an unacceptable level of 50% is chosen as the level that should not go undetected in determining that the health facilities are not performing adequately in regard to a given indicator. The alpha and beta errors have been placed at 10%. The new R-HFA software automatically calculates the sample size needed, and calculates the results following the survey. Because the number of facilities in the project area is 31, using a performance standard of 80% and a lower threshold of 50%, the sample size needed is 18 health facilities Selecting the Sampling Unit The new R-HFA version 2.1 methodology made choosing the sampling frame a straightforward process. A simple random sampling methodology was used. In GCM County there are 31 PHC health facilities in the project area, and therefore the software determined that a sample size of 18 is needed. The listing of HFs is available in Annex 1. Each health facility was listed, and each was given a number between one and 31. Using a two-digit random number table, 18 random numbers were selected between 01 and 31. Selecting the Survey Teams Five teams of three individuals, one supervisor and two enumerators, were developed using the same staff that performed the baseline HFA. This staff was comprised of MTI Supervisors and CHT members. Each team contained at least two members with experience in health. Therefore, all 18 HFs were scheduled to be surveyed in 4 days. It was necessary to return to 4 HFs to include the needed 6 children, as these 4 facilities were originally visited on slow days due to market days and other factors. The entire HFA was completed in 5 consecutive days. Each element of the R-HFA is administered by the following members of the R-HFA survey team: a. Observation Checklist for sick child care: Enumerator with experience in health b. Client (Caretaker) Exit Interview: Enumerator, with guidance from Enumerator with experience in health c. Health Facility Checklist: Supervisor 14 d. Health Worker Survey: Supervisor of Enumerator with experience in health; should be performed by whichever team member has completed their other duties. e. CHW Survey and Checklist (optional): Enumerator with guidance from Supervisor or Enumerator with experience Training the Survey Team The training of the survey team required four days. The main objectives of the training were to discuss the purpose of the survey and the resulting information; discuss the logistics of the survey; review and practice each of the forms; and practice administering these forms in the facility setting. Sinje Clinic, a health facility that was near the training site and which was not randomly chosen for the HFA was used for the field test. The training schedule provided by CSTS, used to train the survey team is available in Annex 2. The Survey Process Observation of Clinical Care and Caretaker Exit Interview The first six children under the age of five presenting to the facility during the survey period whose caretakers describe them as having diarrhea/vomiting, fever/malaria, or cough/difficulty breathing/pneumonia were included in the sample. The caretakers were met as they entered the clinic, and if they agreed to take part in the survey they were followed throughout the facility. If the caretaker brought more than one sick child under the age of five, one child was randomly chosen to be the index child. The enumerator with experience in health observed the clinical encounter between the HW and the caretaker and child. The second enumerator conducted the Exit Interview with the caretakers of sick children outside of the facility as they exited, following receiving the child’s medications. Health Facility Checklist After ensuring that these interviews were proceeding well, the supervisor completed the Health Facility Checklist with an available HW at the facility. A HW was present because determining the conditions in the consultation room and of some of the equipment required some discussion with the HW. Health Worker Survey Following completion of the observation of six consultations between the HW and the caretaker/child, the enumerator with experience or the supervisor, whichever was available, performed the HW Survey. CHW Survey and Checklist The CHW Survey and Checklist were performed if CHWs are involved in the health facility. In total there were only 5 CHWs interviewed. Providing Feedback to the Staff 15 Surveyors were instructed to provide some feedback to staff on the day of the assessment. The feedback was positive wherever possible to alleviate any anxiety the staff may have felt due to the survey, but also included any comments necessary to improve clinical treatment and management techniques. It was recommended during the training that feedback regarding the following items be given. Strengths and problems in case management, particularly in the assessment and treatment of sick children Quality of home-care advice and communication between health workers and caretakers Inappropriate use of medications Problems in record keeping Ways to improve clinic organization Major barriers to effective practice Checking the Completed Questionnaires Completed questionnaires were checked by the supervisor or enumerator administering the questionnaire immediately at the conclusion of the interview so that any discrepancies or missed questions could be discussed with the person being interviewed. At the end of each facility session, the Supervisor reviewed all forms with the enumerators before leaving the facility. The completed forms were then brought to the central point chosen for data entry and given to the data entry staff. This was done nightly so that data entry could be performed during the data collection period. The data entry staff reviewed the completed questionnaires for accuracy while the survey team was there, in order to clarify or correct any unclear or incorrect items noticed in the forms. Data Entry During the data collection phase of the survey, data was entered into the R-HFA Excel program provided in the R-HFA zip file available on the CSTS website. Data entry was performed by the M&E Specialist and M&E Officer, who are trained in data entry. As much as possible, data was entered daily throughout the survey so that any discrepancies could be discussed with the supervisors as soon as possible. Cleaning of the survey data was accomplished by the data entry staff as the data was presented. Following data entry for the final assessment, all data was then combined in the single Excel file provided by CSTS. The R-HFA survey forms file has a tabulation plan for hand tabulating the disaggregated indicators (each indicator alone), and the aggregated indicators that comprise the 12 core indicators. This includes bar graphs and tables that will provide useful reporting tools to provide in the HFA Report. Constraints/Difficulties: Four of the HF did not have the required number of children present with illness, due to environmental factors such as market day in the village or an adjacent village, and the 16 preparations for Ramadan. Therefore, the same interview team returned to these facilities the following day to complete the survey. This did not effect the results. CHAPTER 3 Main Findings: Accomplishments and Priorities Overall, the project has had a significant impact on improving access and care at the Health Facilities through IMCI and C-IMCI. The major areas of concern, identified through the initial HFA performed at baseline were in the areas of service availability, Supervision, health worker performance and the referral of sick children in the community to the HF. At baseline, only 9.1% of HF clinical encounters occurred in which all assessment tasks were made by HWs for sick child. Through IMCI training of HWs by the project, coupled with alternating, bi-monthly supervisory visits and mentoring visits by MTI Supervisors, this indicator has now been increased to 28%, with the percentage of assessment tasks being performed on average at each HF at 66%. No facilities were providing growth monitoring, which is the reason the indicator for the percentage of HF that offer child, immunization, and growth monitoring services was 0% at baseline. As part of the IMCI training, growth monitoring was to be implemented at each HF. 44% of HFs are now providing growth monitoring. At baseline only 4.5% of HF had received external supervision at least once in the previous 6 months. The project has implemented a regular system of supervision, and has increased this indicator greatly. The indicator has now changed to the percentage of HFs that have received external supervision in the last 3 months, versus 6 months, and still the project attained 56%. Also, no CHWs were referring patients to the HFs at baseline. The project has instituted C-IMCI and has trained HHPs to visit every household in the county with children under 5, on a regular basis, and refer sick children to the HFs. This is monitored through monthly Care Group reports. It is encouraging that 100% of the HFs now offer ANC services at least once a week. In addition, 61% of health facilities have now received in-service or pre-service training in child health and in maternal neonatal care in the last 12 months. With regard to medications, 61% of HFs had all first line medications for child health in stock, with the percentage of HF attainment at 90% (meaning each HF had an average of 90% of the immunizations in stock). Also, 44% of HF were found to have all nationally mandated immunizations in stock, with the percentage of HF attainment at 69%. The results, listed by domain, are discussed below. ACCESS (INPUTS) 1. Service Availability Indic. # Domain Indicator % HF with all elements 1 CHILD Service Availability - Child % HF that offer all three basic child health services (growth monitoring, immunization, sick child care) 44% 18 1 ANC Service Availability - ANC % HF that offer ANC at least once a week 100% 1 NEO Service Availability - Delivery % HF that offer delivery services on all days 39% At midterm, 44% of HFs were deemed to offer all 3 basic child health services. This is increased from baseline, in which 0% of HFs offered all 3 basic child health services due to the fact that no facilities offered growth monitoring. As part of the IMCI protocol, growth monitoring has been introduced to the HFs, and at midterm 44% of HFs are providing growth monitoring services in the facility and/or through outreach. In addition, 100% of the facilities are offering Immunization services through the facility and/or outreach. Also, 100% of HF in GCM County offer sick child services through the facility and/or outreach for a total of 20 days or more per month. Most facilities provide sick child services Monday￾Friday, for a total of 20 hours per week, but only 11% of the HFs provide outreach. The project had been concentrating on providing outreach services through the HHPs, who refer sick children to the HFs. The project will continue to work to increase HFs capacity in providing growth monitoring, and will continue improving outreach services to the community through the HHPs and Care Groups. It is encouraging to note that 100% of HFs in GCM County offer ANC services, but only 39% of these facilities offer delivery services on all days. This is because most offer delivery services during their normal operational hours, which constitute 20 days per month, and do not offer outreach services. 2. Staffing Indic. # Domain Indicator % HF with all elements Index Value (% avg. HF attainment) 2 Staffing % HF with all staff who provide clinical services working on the day of survey 56% 77% This indicator has been changed since the baseline HFA was performed, at which time it was: “%HF with at least one provider meeting the country definition as qualified to provide curative care for children is present on day of survey”. All HFs met this requirement (100%) at baseline, and would have met it at midterm as well. However, the new indicator determines the number of each type of staff, and determines if they are all present on the day of the survey. In the midterm HFA, 56% were present, and the average HF attainment was 77% (meaning the average HF had 77% of its staff present). The project has already conducted meetings with the CHT to discuss this, because at this point in the project only the most experienced HW in each facility has been trained in IMCI, and has the responsibility of training the other HWs in their facility. This HW is therefore responsible to be at the HF to provide services, and also to train other staff in IMCI. The CHT has introduced new guidelines to all of the HW in the county to ensure their presence at the HF. 19 3. Infrastructure Indic. # Domain Indicator % HF with all elements Index Value (% avg. HF attainment) 3 Infrastructure % HF in which all essential infrastructure is present and functioning on day of the survey (improved water source; functional latrine for clients; setting allowing auditory and visual privacy; power; communication equipment; emergency transport; overnight beds) 33% 45% Has at least one bed 56% Has 24 hour staff coverage 56% Has functioning emergency communication 28% Has emergency transportation usable today 6% Has electricity from the grid or a generator with fuel 11% Has a usable client latrine 61% Has water from protected water source on or near grounds 56% Has auditory and visual privacy 89% All essential health infrastructure was present on the day of the survey in 33% of the HFs, with the average HF attainment at 45%. At baseline 50% of HFs had all essential infrastructure, but this is because this indicator is also more rigid than the last version of the HFA, in which communication equipment, overnight beds, and emergency transport was not required. As can be seen in the preceding table, the limiting factors are: emergency transport, which was available in only 6% of the HFs, electricity, which was available in only 11% of the HFs, and emergency communication, which was available in only 28% of the HFs. The project has been working with the communities through the CHCs to develop emergency transportation systems in each village in order to work around the fact that HFs do not have the means for transportation systems. The project will also continue to advocate for increased infrastructure through the CHT of the MOH, and will compare this midterm finding to final HFA performed at the final evaluation. 20 4. Supplies Supplies - Child Indic. # Domain Indicator % HF with all elements Index Value (% avg. HF attainment) 4 CHILD Supplies - Child % HF with all essential supplies to support child health on day of the survey (accessible and working scale for child, accessible and working scale for infant, timing device for diagnosis of pneumonia, spoon/cup/jug to administer ORS) 44% 79% Has functioning and accessible infant scale 67% Has functioning and accessible scale for children/adults 94% Has functioning timer or watch 78% Has pitcher for ORS 78% Has cup or spoon for ORS 78% While only 44% of HFs had all of the essential supplies to support child health, the average HF attainment was 79% of these essential supplies. This is a decrease from baseline, where 90.9% of HFs had the entire essential infrastructure required for child health. The table of disaggregated indicators reveals that infant scales are available in only 67% of the facilities, and timers, pitchers for ORS, and a cup or spoon for ORS are each only available 78% of the time. The project will coordinate with the MOH to ensure proper ORS supplies are available, and see if infant scales may be made available. Supplies - MNC Indic. # Domain Indicator % HF with all elements Index Value (% avg. HF attainment) 4 MNC Supplies - MNC % HF with all essential supplies to support maternal-newborn health present on day of the survey (partograph, vacuum extractor, resuscitation device, weighing scale, antibiotics and baby wraps) 0% 33% Has functioning neonatal resuscitation equipment 39% Has functioning and accessible infant scale 56% Has functioning vacuum extractor 6% Has neonatal wraps for warming 56% Has partographs 11% None (0%) of the HFs have all of the essential supplies to support maternal-newborn health available. This is mainly due to the fact that only 6% of HFs have functioning vacuum extractors, and only 11% of facilities have partographs. In addition, only 39% of facilities have neonatal resuscitation equipment, and only slightly more than half (56%) of facilities have neonatal wraps and infant scales. The project will work with the CHT to determine the feasibility of obtaining these badly needed items. 21 Supplies - ANC Indic. # Domain Indicator % HF with all elements Index Value (% avg. HF attainment) 4 ANC Supplies - ANC % HF with all essential supplies to support antenatal care present on day of survey (blood pressure machine, tetanus toxoid vaccine, hemoglobin reagents, syphilis testing kit, and albastix for protein) 0% 42% Has functioning refrigerator 72% Has functioning blood pressure equipment 94% Has hemoglobin testing reagents 0% Has syphilis testing kits 0% Has malaria test kits 78% Has urine albumin test strips 6% Has tetanus toxoid 72% Similarly, 0% of the HFs have all of the essential supplies to support antenatal care. No facilities have hemoglobin or syphilis tests available and only 6% of facilities have urine albumin test strips available. The other disaggregated indicators for this indicator are acceptable, so the project will work with the CHT to increase availability of the testing kits. 5. Drugs Drugs - Child Indic. # Domain Indicator % HF with all elements Index Value (% avg. HF attainment) 5 CHILD Drugs - Child % HF with all first line medications for child health present on day of the survey (ORS, oral antibiotic for pneumonia, first line oral antibiotic for dysentery, first line antiamalarial, vitamin A) 61% 90% Has ORS packets 89% Has first line child pneumonia drug 94% Has first line dysentery drug 100% Has first line antimalarial 94% Has vitamin A 72% At baseline, 64% of HFs had all first line medications available, which at that time was defined as ORS, a first line oral antibiotic for pneumonia, and a first line antimalarial. At midterm, 94% of facilities have first line drugs for pneumonia and malaria, and 89% had ORS available, showing an improvement in these areas. In total, 61% of HFs have all of the first line medications for child health available. This is comparable to baseline, but this new indicator is more rigid than the baseline indicator because it includes Vitamin A. The limiting factor in drug supply appears to be Vitamin A, as only 72% of the facilities have Vitamin A in stock. The project will work with the CHT to ensure the proper supply of 22 Vitamin A, ORS, first line antibiotics for pneumonia and dysentary, and first line antimalarials. Drugs - MNC Indic. # Domain Indicator % HF with all elements Index Value (% avg. HF attainment) 5 MNC Drugs - MNC % HF with all essential delivery & neonatal drugs present on day of survey (i.e., oxytocin, antibiotics for newborn sepsis and eye infections) 33% 74% Has antibiotics for newborn sepsis/pneumonia 89% Has neonatal eye ointment 94% Has oxytocin 39% Has niveripine (in high HIV prevalence areas only) 0% Only 33% of HFs had all of the essential delivery and neonatal drugs present on the day of the survey. The limiting factor in this indicator is Oxytocin, which is available in only 39% of the facilities. The project will work with the CHT to attempt to secure Oxytocin for the delivery unit. It should be noted that Niveripine is not available in GCM County. The project would need to secure supplies of Niveripine should the project expand into PMTCT of HIV. PROCESSES 6. Information Systems Information System - Child Indic. # Domain Indicator % HF with all elements Index Value (% avg. HF attainment) 6 CHILD Information System - Child % HF that maintain up-to-date records of sick U5 children (age, diagnosis, treatment) and for HF: have report in last 3 months and evidence of data use 61% 82% The average facility has 82% of the elements required for having up to date records available for child care, with 39% of HFs have all elements. At baseline, 0% of the facilities had all of the elements in place regarding records. They were not broken out into child or ANC, but this still reveals that there have been tremendous improvements made in record keeping. 23 Information System - ANC Indic. # Domain Indicator % HF with all elements Index Value (% avg. HF attainment) 6 MNC Information System - ANC % HF that maintain up-to-date records of antenatal care (TT, blood pressure, expected date of delivery) & deliveries (present & up to date) 39% 81% An ANC register was observed 100% ANC register with complete delivery information, last 3 months 83% ANC register with complete TT information, last 3 months 67% ANC register with complete BP information, last 3 months 89% ANC register with entry in last 7 days 83% Delivery register was observed 89% Delivery register was up to date (entry in last 30 days) 72% The average facility has 81% of the elements required for having up to date records available for antenatal care, with 39% of HFs have all elements. At baseline, 0% of the facilities had all of the elements in place regarding records, revealing that there have been improvements made in record keeping 7. Training Indic. # Domain Indicator % HF with all elements 7 CHILD Training - Child Health % HF in which interviewed HW reported receiving in-service or pre-service training in child health in last 12 months 61% 7 MNC Training - Maternal￾Neonatal Care % HF in which interviewed HW reported receiving in-service or pre-service training in maternal neonatal care in last 12 months 61% Interviewed HW received any MNC training in last 12 mo. 72% MNC Immunization training 28% Pneumonia case management training 44% Diarrhea case management training 44% Malaria case management training 56% ACT use training 56% ITN use training 50% Nutrition training 33% Breastfeeding promotion training 44% IMCI training 56% 24 CHILD HEALTH IPT use training 50% Newborn care training 44% Post-partum care training 6% ANC training 11% Infection control training 11% AMTSL training 11% Ob / neonatal emergencies referral training 22% 61% of the HF had HW who reported receiving in-service or pre-service training in both maternal neonatal care and child health in last 12 months. Training in MNC consists of training in MNC and at least one other of the trainings listed above in the MNC section. Training in child health consists of training in MNC and at least one other of the trainings listed above in the child health section In looking at the subjects in which training is given, MNC training was the highest, at 72%. ANC training was the lowest, at 11%. The baseline finding of 79% was for the indicator “receiving any in-service or pre-service education relevant to their work in last 12 months” versus these more rigorous indicators. However, these numbers are comparable or higher than the baseline numbers when you consider that MNC training alone was provided to 72% of HW. The project is commited to continuing training of HW staff in all aspects of IMCI. 8. Supervision Indic. # Domain Indicator % HF with all elements 8 Supervision % HF that received external supervision at least once in the last 3 months (supervision included one or more of the following: checked records or reports, observed work, provided feedback, gave praise, provided updates, discussed problems)) 56% External supervision has increased since baseline, from 4.5% of HF receiving external supervision at least once in the last 6 months, to 56% of HFs receiving external supervision in the last 3 months. Despite the fact that the indicator has changed from 6 months to 3 months, the midterm results are encouraging. The project is planning to conduct supervisory visits in conjunction with the CHT, and the project is confident that this percentage will continue to improve. 25 PERFORMANCE 9. Utilization of Curative services Indic. # Domain Indicator % HF with >1 encounter/child in GCM 9 CHILD Utilization of Curative Services Annualized number of clinical encounters for sick children per U5 population (% HF with > 1 sick child encounter per U5 in catchment area) 5.00% The percentage of HF with > 1 sick child encounter per child under the age of 5 in GCM County was 5.0%. This is an increase from the baseline of 1.89%. The number of sick children being seen in the facilities has increased over the 2 years of the project, and we will continue to strive to improve utilization of the services through the HHPs and Care Groups providing health education and providing referrals of sick children encountered in the community to the facilities. It should be noted that this indicator was hand tabulated using the number of clinical encounters with sick children fewer than 5 in last three complete calendar months divided by the most recent figure available from the CHT regarding the number of children under 5 in the catchment area. 10. HW Performance (Assessment) Indic. # Domain Indicator % HF with all elements Index Value (% avg. HF attainment) 10 CHILD HW Performance (Assessment) % HF where key assessment tasks are routinely performed (check presence of general danger signs, assess feeding practices, assess nutritional status, check vaccination status) 28% 66% In order for a HF to be considered as having key assessment tasks routinely performed, in greater than 80% (5 or 6 out of the 6 cases observed) of the encounters the HW must perform all of the key assessment tasks. The midterm found that the average facility was performing 66% of the key assessment tasks. Therefore, only 28% of the facilities were found to routinely perform all key assessment tasks. This is quite an improvement over the baseline, however, where only 9% of facilities were found to routinely perform all key assessment tasks. The survey team discussed this finding during and after the evaluation, and determined the major factor contributing to this low percentage is the fact that the most experienced HW, who has been trained by the project in IMCI, was often not at the facility on the day of the survey. A meeting was held with the CHT where this finding was discussed, and new regulations will be implemented in the HFs to ensure that HW are present during all normal HF hours. In addition, the project will follow up with the HW trained in IMCI to ensure that they are committed to providing this training to the other pertinent staff in their HF, as agreed upon at the start of the IMCI training sessions. The project will now begin to expand its IMCI training to include all HWs at each facility. This 26 will be accomplished by scheduling a planned training session at each HF in the county, in which all HWs that treat children will be given a one day training in the most needed aspects of IMCI-the performance of key assessment tasks and the counseling of caretakers in administering prescribed medications to their child. This will be in addition to the regular training that is to be provided to these HWs by the most experienced HW, who has been fully trained in IMCI. 11. HW Performance (Treatment) Indic. # Domain Indicator % HF with all elements 11 CHILD HW Performance (Treatment) % HF where treatment is routinely appropriate to diagnosis (for encounters in which at least one of the presenting problems was fever, breathing problem, or diarrhea) 72% It is encouraging that in 72% of the HFs, the treatment given by the HW is routinely appropriate to diagnosis. The HW must diagnose and treat the sick child correctly in greater than 80% (5 or 6 out of the 6 cases observed) of the encounters observed in order for a HF to be considered as providing correct treatment. This is a marked improvement over the baseline frequency of 45.5%. The project will continue to provide regular, scheduled training in IMCI, along with supervisory visits and mentoring, to continue to improve HW performance. 12. HW Performance (Counseling) Indic. # Domain Indicator % HF with all elements 12 CHILD HW Performance (Counseling) % HF where caretakers whose child was prescribed an antibiotic, antimalarial, or ORS, correctly describe how to administer all prescribed drugs 11% The weakest area noted, in regards to HW performance, is in the area of counseling the caretaker on the proper method of administering the medicines prescribed. Only 11% of HFs were properly instructing caretakers in how to correctly administer drugs prescribed for their child. This is a decrease from baseline, where it was reported that 48.5% of caretakers could correctly describe how to administer all prescribed drugs. Instruction was given to each of the HWs encountered during the survey that was having difficulty with this, during the feedback given to the HW following the data collection. The major factor contributing to this low percentage is once again the fact that the most experienced HW, who has been trained by the project in IMCI, was often not at the facility on the day of the survey. The HWs present that have not undergone IMCI training did not fully explain the medications prescribed for the child to the caretaker. New regulations will be implemented in the HFs, 27 by the CHT, to ensure that HWs are present during all normal HF hours. In addition, the project will follow up with the HWs trained in IMCI to ensure that they are committed to providing this training to the other pertinent staff in their HF, as agreed upon at the start of the IMCI training sessions. 28 CHAPTER 4 Action Plan/Conclusions/Recommendations HF Child Health Balanced Score Card 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Geographic Access Service Availability - Child Staffing Infrastructure Supplies - Child Drugs - Child Availability of Immuniza ions Availability of Guidelines Infection Control Information System - Child Training - Child Health Supervision HF-Community Coordination Community Referral Malaria Drug (ACT) Logis ics ITN/LLIN Logistics Laboratory Assessment Treatment Counseling ACCESS INPUTS PROCESSES PERFORMANCE HF MNC Balanced Score Card 0% 20% 40% 60% 80% 100% 120% Geographic Access Service Availability - ANC Service Availability - Deliveries Staffing Infrastructure Supplies - Neonatal Supplies - ANC Drugs - Neonatal Drugs - ANC Availability of Guidelines Infection Control Information System - ANC Training - Maternal-Neonatal Care Supervision HF-Community Coordination Malaria Drug (ACT) Logis ics ITN/LLIN Logistics Laboratory ACCESS INPUTS PROCESSES 29 The Grand Cape Mount Child Survival Project has improved the inputs, processes, and performance in the HFs in the first two years of the project. The midterm RHFA has verified that the project has had a significant impact on service availability, drug availability, information systems being instituted, supervision, and most areas of HW performance. Some areas require additional attention, and the CSP is committed to increasing the HFs capacity in these areas. The following is a summary of each of the 12 core indicators, along with the Action Plan items for each indicator in need of continued improvement. These items have been discussed with the CHT and will be implemented at the HFs during the course of the next two years. The graphs above provide a visual summary. ACCESS (INPUTS) 1. Service Availability: At midterm, 44% of HFs were deemed to offer all 3 basic child health services. This is increased from baseline, in which 0% of HFs offered all 3 basic child health services due to the fact that no facilities offered growth monitoring. As part of the IMCI protocol, growth monitoring has been introduced to the HFs, and at midterm 44% of HFs are providing growth monitoring services in the facility and/or through outreach. With regard to increasing referrals to the HFs, The project has been concentrating on providing outreach services through the HHPs, who refer sick children to the HFs. The project will continue to work to increase HFs capacity in providing growth monitoring, and will continue improving outreach services to the community through the HHPs and Care Groups. 2. Staffing: 56% of all staff who provide clinical services were present on the day of the midterm survey, and the average HF attainment was 77% (meaning the average HF had 77% of its staff present). This indicator has been changed since baseline, and therefore the midterm finding will be used to track future performance. The project has already conducted meetings with the CHT to discuss this, because at this point in the project only the most experienced HW in each facility has been trained in IMCI, and has the responsibility of training the other HWs in their facility. This HW is therefore responsible to be at the HF to provide services, and also to train other staff in IMCI. The CHT has introduced new guidelines to all of the HW in the county to ensure their presence at the HF. 3. Infrastructure: All essential health infrastructure was present on the day of the survey in 33% of the HFs, with the average HF attainment at 45%. The limiting factors are emergency transport, which was available in only 6% of the HFs, electricity, which was available in only 11% of the HFs, and emergency communication, which was available in only 28% of the HFs. The project has been working with the communities through the CHCs to develop emergency transportation systems in each village in order to work around the fact that HFs do not have funding for transportation systems. We will also continue to advocate for increased infrastructure through the CHT of the MOH, and will compare this midterm finding to final HFA performed at the final evaluation. 4. Supplies: a. Supplies – Child: While only 44% of HFs had all of the essential supplies to support child health, the average HF attainment was 79% of these essential 30 supplies. This is a decrease from baseline, where 90.9% of HFs had the entire essential infrastructure required for child health. The table of disaggregated indicators reveals that infant scales are available in only 67% of the facilities, and timers, pitchers for ORS and a cup or spoon for ORS are only available 78% of the time. Investigation into this revealed that at the start of IMCI all HFs were provided with the proper supplies for administering ORS and all facilities had functional incant scales, so this explains much of the decline. The project will coordinate with the MOH to ensure proper ORS supplies are available, and advocate to have infant scales made available . b. Supplies-MNC: None (0%) of the HFs have all of the essential supplies available to support maternal-newborn. This is mainly due to the fact that only 6% of HFs have functioning vacuum extractors, and only 11% of facilities have partographs. In addition, only 39% of facilities have neonatal resuscitation equipment, and only slightly more than half (56%) of facilities have neonatal wraps and infant scales. The project will work with the CHT to determine the feasibility of obtaining these badly needed items. c. Supplies-ANC: Similarly, 0% of the HFs have all of the essential supplies to support antenatal care. No facilities have hemoglobin or syphilis tests available and only 6% of facilities have urine albumin test strips available. The project will work with the CHT to increase availability of the testing kits. 5. Drugs: a. Child: At midterm, 94% of facilities have first line drugs for pneumonia and malaria, and 89% had ORS available, showing an improvement in these areas. This compares to baseline where 64% of HFs had all first line medications available, which at that time was defined as ORS, a first line oral antibiotic for pneumonia, and a first line antimalarial. In total, 61% of HFs have all of the first line medications for child health available. This is comparable to baseline, but this new indicator is more rigid than the baseline indicator because it includes Vitamin A. The limiting factor in drug supply appears to be Vitamin A, as only 72% of the facilities have Vitamin A in stock. The project will work with the CHT to ensure the proper supply of Vitamin A, ORS, first line antibiotics for pneumonia, and first line antimalarials. b. MNC: Only 33% of HFs had all of the essential delivery and neonatal drugs present on the day of the survey. The limiting factor in this indicator is Oxytocin, which is available in only 39% of the facilities. The project will work with the CHT to attempt to secure Oxytocin for the delivery unit. PROCESSES 6. Information Systems a. Child: The average facility has 82% of the elements required in having up to date records available for child care, with 39% of HFs have all elements. At baseline, 0% of the facilities had all of the elements in place regarding records. b. ANC: The average facility has 81% of the elements required in having up to date records available for antenatal care, with 39% of HFs have all elements. At baseline, 0% of the facilities had all of the elements in place regarding records, 31 revealing that there have been tremendous improvements made in record keeping. 7. Training: 61% of the HF had HW who reported receiving in-service or pre-service training in both maternal neonatal care and child health in last 12 months. 8. Supervision: External supervision has increased since baseline, from 4.5% of HF receiving external supervision at least once in the last 6 months, to 56% of HFs receiving external supervision in the last 3 months. Despite the fact that the indicator has changed from 6 months to 3 months, the midterm results are encouraging. The project is planning to conduct supervisory visits in conjunction with the CHT, and the project is confident that this percentage will continue to improve. 9. Utilization of Curative services: The percentage of HF with > 1 sick child encounter per child under the age of 5 in GCM County was 5.0%. This is an increase from the baseline of 1.89%. The number of sick children being seen in the facilities has increased over the 2 years of the project, and we will continue to strive to improve utilization of the services through the HHPs and Care Groups providing health education and providing referrals of sick children encountered in the community to the facilities. PERFORMANCE 10. HW Performance (Assessment): The midterm found that the average facility was performing 66% of the key assessment tasks. Therefore, only 28% of the facilities were found to routinely perform all key assessment tasks. This is quite an improvement over the baseline, however, where only 9% of facilities were found to routinely perform all key assessment tasks. The survey team discussed this finding during and after the evaluation, and determined the major factor contributing to this low percentage is the fact that the most experienced HW, who has been trained by the project in IMCI, was often not at the facility on the day of the survey. A meeting was held with the CHT where this finding was discussed, and new regulations will be implemented in the HFs to ensure that HW are present during all normal HF hours. In addition, the project will follow up with the HW trained in IMCI to ensure that they are committed to providing this training to the other pertinent staff in their HF, as agreed upon at the start of the IMCI training sessions. In addition, the project will now begin to expand its IMCI training to include all HWs at each facility. This will be accomplished by scheduling a planned training session at each HF in the county, in which all HWs that treat children will be given a 1 day training in the most needed aspects of IMCI-the performance of key assessment tasks and the counseling of caretakers in administering prescribed medications to their child. This will be in addition to the regular training that is to be provided to these HWs by the most experienced HW, who has been fully trained in IMCI. 11. HW Performance (Treatment): It is encouraging that in 72% of the HFs, the treatment given by the HW is appropriate to diagnosis (meaning provided in at least 5 of the 6 observed cases). This is a marked improvement over the baseline frequency of 45.5%. The project will continue to provide regular, scheduled training in IMCI, along with supervisory visits and mentoring, to continue to improve HW performance. 32 12. HW Performance (Counseling): The weakest area noted, in regards to HW performance, is in the area of counseling the caretaker on the proper method of administering the medicines prescribed. Only 11% of HFs were properly instructing caretakers in how to correctly administer drugs prescribed for their child. This is a decrease from baseline, where it was reported that 48.5% of caretakers could correctly describe how to administer all prescribed drugs. Instruction was given to each of the HWs encountered during the survey that was having difficulty with this, during the feedback given to the HW following the data collection. The major factor contributing to this low percentage is once again the fact that the most experienced HW, who has been trained by the project in IMCI, was often not at the facility on the day of the survey. New regulations will be implemented in the HFs, by the CHT, to ensure that HWs are present during all normal HF hours. In addition, the project will follow up with the HWs trained in IMCI to ensure that they are committed to providing this training to the other pertinent staff in their HF, as agreed upon at the start of the IMCI training sessions. CONCLUSIONS/RECOMMENDATONS The GCMCSP continues to improve the lives of children and families in Grand Cape Mount County through improving the access, capacity, processes, and performance of the HFs in GCM County. The project has been very successful in greatly increasing performance with regard to most of the core indicators of the RHFA. The Action Plan outlined above will guide the third and fourth years of the project in order to continue to improve the indicators with which the project has had such success, and to overcome the barriers and improve the results of the interventions which need continued improvement. Annex 1 LIST OF HEALTH FACILITIES Health Facility District Supporting Agency Status 1 Damballa Clinc Porkpa AHA Functional 2. Bandaja Clinic Porkpa World Vision Functional 3. Bamballa Clinic Porkpa NWMT To be revitalized 4. Mano River Clinc Porkpa - Non-functional 5. Kalwielahun Clinic Porkpa - Non-functional 6. Jenneh Wonde Clinic Tewor AHA Functional 7. Genolor Clinic Tewor World Vision Functional 8. Gordama Clinic Tewor World Vision Functional 9. Tieni Clinic Tewor World Vision Functional 10. Bo Waterside Clinic Tewor World Vision Functional 11. Diah Clinic Tewor World Vision Functional 12. Mambo Clinic Tewor CHT Functional 13. Bangormah Clinic Tewor - Non-functional 14. Tahn Maffa Clinic Tewor NWMT To be revitalized 15. Kulangor Clinic Tewor NWMT To be revitalized 16. Fanjah Clinic Tewor - Non-functional 17. Kpeneji Clinic Garwula Private/CAM Functional 18. Sinje Health Center Garwula AHA Functional 19. Kangar Clinc Garwula AHA Functional 20. Madina Clinic Garwula World Vision Functional 21. Jundu Clinic Garwula World Vision Functional 22. Bomboja Clinic Garwula NWMT Functional 23. Sembehun Clinic Garwula AHA Functional 24. Bendu Clinic Garwula World Vision Functional 25. Mbaloma Clinic Gola Konneh AHA Functional 26. Lofa Bridge Clinic Gola Konneh AHA Functional 27. Tahn Clinic Gola Konneh AHA Functional 28 Varguay Clinic Gola Konneh NWMT To be revitalized 29. Tallah Clinic Commonwealth World Vision Functional 30. Fanti Town Clinic Commonwealth AHA Functional 31. St. Timothy Hospital OPD Commonwealth CHT Functional Not Included D-8 Private Clinic Garwula Guthrie Rubber Plantation Functional Zaway Clinic Garwula Non-functional Annex 2 Training Of the Survey Team The training of the survey team should require four days. The main objectives of the training are to discuss the purpose of the survey and the resulting information; discuss the logistics of the survey; review and practice each of the forms; and practice administering these forms in the facility setting. A health facility that is near the training site and which was not randomly chosen for the HFA should be used for the field test. The schedule14 below combines a discussion of each form with actual facility based practice in collecting the information needed to complete the form. Day Activities 1 AM: Opening & General Information Opening • Introduction of the participants • Administrative information General information • Purpose of the survey • Training objectives • Survey protocol and techniques • Introduction of Participant Guidelines • Clarification of participant expectations or concerns PM: Introduction to first two forms: Clinical Observation & Sick Child Clinical Observation - Sick Child • Review the instrument • Role play Caretaker Exit Interview – Sick Child • Review the instrument • Role play 2 AM: Health facility visit for Clinical Observation and Caretaker Exit Interview • Visit to health facility for practice of Clinical Observations and Exit Interviews • Debriefing of the health facility visit PM: Intro to Health Worker Interview and Health Facility Checklist Health Worker Interview • Review the instrument • Role play Health Facility Checklist • Review the instrument • Role play 3 AM: Health facility visit for HW Interview and Health Facility Checklist • Visit to health facility to practice the HW interview and using the Health 14 Training schedule provided by CSTS; R-HFA short instruction 12-09-07; R-HFA Version 2.1; CSTS 2007; http://www.childsurvival.com/rhfa 1.cfm 2 Facility Checklist • Debriefing of health facility visit PM: Sampling Health Facility and Data Analysis Sampling health facilities in districts • Explanation of how health facilities were sampled in each district • Reviewing list of health facilities sampled and to be visited during the assessment Analysis of R-HFA data • Analysis of data at the health facility level. Identifying strengths and areas of needed improvement • Analysis of data at the district level. Identifying areas of needed improvement Annex 7 Medical Teams International in Liberia, CSP22 Organizational Capacity Surveys for CHAL and Grand Cape Mount County Health Team CHAL ORGANIZATIONAL CAPACITY SURVEY DEC 5, 2006 CHAL ORGANIZATIONAL CAPACITY SURVEY: SEP. 24, 2008 QUESTION CAPACITY CRITERIA QUESTION CAPACITY CRITERIA Limited Capacity Basic Capacity High Capacity Limited Capacity Basic Capacity High Capacity LEADERSHIP & AGENCY PLAN LEADERSHIP & AGENCY PLAN No written mission or governing principles (Not applicable) Written mission and governing principles No written mission or governing principles (Not applicable) Written mission and governing principles Does the organization have a written mission and/or governing principles? X Does the organization have a written mission and/or governing principles? X Staff are minimally aware of the organization’s mission Staff can generally describe the organization’s mission and/or values Staff are very familiar with the organization’s mission and/or values and can describe these in detail Staff are minimally aware of the organization’s mission Staff can generally describe the organization’s mission and/or values Staff are very familiar with the organization’s mission and/or values and can describe these in detail Is the organization’s mission/values known by all the members of the organization? X Is the organization’s mission/values known by all the members of the organization? X Staff can describe general goals of the organization, but they have never been agreed upon or written down Goals and plan/strategy written down but are out of date or not referred to Goals and plan/strategy are written down and are familiar to all staff. Goals and plan/strategy are regularly referred to and guide the organization’s work Staff can describe general goals of the organization, but they have never been agreed upon or written down Goals and plan/strategy written down but are out of date or not referred to Goals and plan/strategy are written down and are familiar to all staff. Goals and plan/strategy are regularly referred to and guide the organization’s work Does the organization have goals and a plan or strategy to meet the goals? X Does the organization have goals and a plan or strategy to meet the goals? CHAL needs to develop a strategic plan. X 2 Organization unsure of its strengths and niche; projects are chosen based on variable factors Organization seeks to focus on its strengths and niche but does not consistently adhere to its plans Organization consistently focuses on its strengths and niche and strategically chooses projects and interventions related to its long￾term plans Organization unsure of its strengths and niche; projects are chosen based on variable factors Organization seeks to focus on its strengths and niche but does not consistently adhere to its plans Organization consistently focuses on its strengths and niche and strategically chooses projects and interventions related to its long￾term plans Does the organization focus on its strengths and niche? X Does the organization focus on its strengths and niche? CHAL describes it strength as the caliber of its staff and its member-led organization structure X Committee established, but never meets Occasional meetings, but rarely agree on decisions Regular meetings, with useful guidance and decisions made for the organization Committee established, but never meets Occasional meetings, but rarely agree on decisions Regular meetings, with useful guidance and decisions made for the organization Does the organization have a committee or board that meets and makes decisions that guide the organization’s development and reviews fiduciary responsibility? X Does the organization have a committee or board that meets and makes decisions that guide the organization’s development and reviews fiduciary responsibility? CHAL board is not strong. At times the Board doesn't have quorum to hold discussions. It doesn't seem like Directors are passionate about the organization X Board is established but does not represent the organization’s target group(s) Board includes minimal number of target group representatives Board includes variety of members who represent the diverse needs of the organization and of the target group Board is established but does not represent the organization’s target group(s) Board includes minimal number of target group representatives Board includes variety of members who represent the diverse needs of the organization and of the target group Is the organization’s board composed of members who represent the needs of the organization and the varied interests of the people the organization seeks to serve? X Is the organization’s board composed of members who represent the needs of the organization and the varied interests of the people the organization seeks to serve? X 3 Leadership style is not consistent with governing values Leaders of the organization make an effort to ensure the leadership style is appropriate Leadership style is highly consistent with governing values and are a model for all staff Leadership style is not consistent with governing values Leaders of the organization make an effort to ensure the leadership style is appropriate Leadership style is highly consistent with governing values and are a model for all staff Do directors and senior staff use a leadership style that is appropriate to the organization and its purpose and values? X Do directors and senior staff use a leadership style that is appropriate to the organization and its purpose and values? CHAL Board of Directors need training or the organization needs to reconstitute a new board X FINANCIAL MANAGEMENT FINANCIAL MANAGEMENT No written financial policies and guidelines in place Some ideas on what financial policies and guidelines should be in place, but no system Financial policies and guidelines are written down and integrated into systems No written financial policies and guidelines in place Some ideas on what financial policies and guidelines should be in place, but no system Financial policies and guidelines are written down and integrated into systems Does the organization have written financial policies and guidelines? X Does the organization have written financial policies and guidelines? X Are budgets prepared, approved, and used for accounting control purposes? Budgets exist as general, broad line items and are not referred to for control purposes Detailed budgets are prepared and approved but are not used to guide and control financial processes Budgets are prepared with appropriate detail, approved for use, and consistently used for accounting control purposes Are budgets prepared, approved, and used for accounting control purposes? Budgets exist as general, broad line items and are not referred to for control purposes Detailed budgets are prepared and approved but are not used to guide and control financial processes Budgets are prepared with appropriate detail, approved for use, and consistently used for accounting control purposes X X Records kept of funds received and spent, but amount of money held at any one time is unknown Accounts kept up-to-date and balances and statements are prepared at year end Balances and statements are prepared quarterly and presented to external stakeholders at year end and can be retrieved as needed Records kept of funds received and spent, but amount of money held at any one time is unknown Accounts kept up-to-date and balances and statements are prepared at year end Balances and statements are prepared quarterly and presented to external stakeholders at year end and can be retrieved as needed Does the organization keep accounts of money that can be presented upon request? X Does the organization keep accounts of money that can be presented upon request? X 4 Bank reconciliations completed sporadically Bank reconciliations completed at least every other month Bank reconciliations completed monthly Bank reconciliations completed sporadically Bank reconciliations completed at least every other month Bank reconciliations completed monthly Does the organization have monthly bank reconciliations? X Does the organization have monthly bank reconciliations? X No cash-flow system in place Some ideas on how to deal with cash-flow difficulties, but no workable system Proven cash-flow system that works No cash-flow system in place Some ideas on how to deal with cash-flow difficulties, but no workable system Proven cash-flow system that works Does the organization have a system to deal with cash-flow difficulties? X Does the organization have a system to deal with cash-flow difficulties? X Controls are not written; unable to assess whether they are implemented Control policies are written down but are irregularly implemented Control policies are written down and effectively implemented by all staff Controls are not written; unable to assess whether they are implemented Control policies are written down but are irregularly implemented Control policies are written down and effectively implemented by all staff Are adequate financial and inventory controls in place and implemented? X Are adequate financial and inventory controls in place and implemented? X Receipts/invoices rarely kept on file Receipts/invoices kept on file but rarely “reviewed” or used for financial reporting of grants All receipts/invoices and other supporting documents filed for three years, used for financial reporting, and regularly reviewed by authorized personnel Receipts/invoices rarely kept on file Receipts/invoices kept on file but rarely “reviewed” or used for financial reporting of grants All receipts/invoices and other supporting documents filed for three years, used for financial reporting, and regularly reviewed by authorized personnel Are supporting receipts and invoices maintained for every expenditure from grants? X Are supporting receipts and invoices maintained for every expenditure from grants? X Reports are usually late and with incomplete information Some reports submitted on time but are often incomplete or questionable Reports always submitted on time and always meet all donor requirements Reports are usually late and with incomplete information Some reports submitted on time but are often incomplete or questionable Reports always submitted on time and always meet all donor requirements Are accurate financial reports provided to donors in a timely manner? X Are accurate financial reports provided to donors in a timely manner? X 5 Large grants are held in organization’s general bank account Not applicable Organization holds separate accounts for large grants Large grants are held in organization’s general bank account Not applicable Organization holds separate accounts for large grants Does the organization hold separate accounts for large grants? X Does the organization hold separate accounts for large grants? X Annual reporting rarely completed Annual reporting completed but not submitted to annual audits Annual reporting completed and submitted to annual audits Annual reporting rarely completed Annual reporting completed but not submitted to annual audits Annual reporting completed and submitted to annual audits Does the organization provide annual reporting and submit to annual audits? X Does the organization provide annual reporting and submit to annual audits? X Fundraising activities performed as needs arise Fundraising strategy is developed but not used consistently enough to generate needed funds Fundraising strategy is well developed, includes a variety of funding sources, and effectively raises sufficient funds Fundraising activities performed as needs arise Fundraising strategy is developed but not used consistently enough to generate needed funds Fundraising strategy is well developed, includes a variety of funding sources, and effectively raises sufficient funds Does the organization have an effective fundraising strategy? X Does the organization have an effective fundraising strategy? CHAL needs a fund raising strategy X ADMINISTRATION & HUMAN RESOURCES ADMINISTRATION & HUMAN RESOURCES Number of staff is not adequate for project needs Number of staff is adequate but are not fully trained for effective project implementation Organization has staff large enough to implement projects; staff possess requisite health program skills Number of staff is not adequate for project needs Number of staff is adequate but are not fully trained for effective project implementation Organization has staff large enough to implement projects; staff possess requisite health program skills Does the organization have adequate staff in terms of numbers and skills in health programs? X Does the organization have adequate staff in terms of numbers and skills in health programs? CHAL does not have the required capacity to directly implement health programs in the community X Recruitment and personnel practices are unclear Recruitment and personnel practices are defined in written policies but are not consistently followed Recruitment and personnel practices are defined in written policies and are regularly followed Recruitment and personnel practices are unclear Recruitment and personnel practices are defined in written policies but are not consistently followed Recruitment and personnel practices are defined in written policies and are regularly followed Are recruitment and personnel practices clearly defined and followed? X Are recruitment and personnel practices clearly defined and followed? It is not clear whether CHAL has a defined personnel practices that are consistently followed as it relates to the CSP project. X 6 Organization inconsistently follows local labor laws and does not regularly compare salary and benefits with local standards Local labor laws are usually adhered to, and salaries and benefits are sometimes in range with local standards Local labor laws are strictly followed and salary scales and benefits are reviewed and revised against local standards Organization inconsistently follows local labor laws and does not regularly compare salary and benefits with local standards Local labor laws are usually adhered to, and salaries and benefits are sometimes in range with local standards Local labor laws are strictly followed and salary scales and benefits are reviewed and revised against local standards Does the organization follow local labor laws and ensure that salary scales and benefits are commensurate with local standards? X Does the organization follow local labor laws and ensure that salary scales and benefits are commensurate with local standards? CHAL follows local labor laws in as far as payment of taxes and social security contribution are concern. However, CHAL does not pay benefits to project staff. Although CHAL maintains wage scale above the government, it is difficult to determine how CHAL wage scale compares to other NGOs. X Performance appraisals conducted every few years; compensation is reviewed at employee’s request Performance appraisals are conducted nearly every year along with review of performance￾based compensation packages Performance appraisals are routinely conducted annually and performance￾based increases are given when possible and appropriate Performance appraisals conducted every few years; compensation is reviewed at employee’s request Performance appraisals are conducted nearly every year along with review of performance￾based compensation packages Performance appraisals are routinely conducted annually and performance￾based increases are given when possible and appropriate Does the organization ensure that staff performance appraisals are conducted and compensation is reviewed annually? X Does the organization ensure that staff performance appraisals are conducted and compensation is reviewed annually? Staff appraisals and compensation reviews based on such appraisals are not institutionalized. X Staff training is rarely provided Staff training is occasionally provided at staff members’ request and initiative Staff training is planned and provided; topics are solicited via staff questionnaires and assessments Staff training is rarely provided Staff training is occasionally provided at staff members’ request and initiative Staff training is planned and provided; topics are solicited via staff questionnaires and assessments Does the organization provide regular staff training to improve staff members’ professional development? X Does the organization provide regular staff training to improve staff members’ professional development? Staff training is not regularly provided. X 7 Staff members rarely remain at organization more than a year Staff generally remain at organization less than five years Staff members tend to remain at organization at least five years Staff members rarely remain at organization more than a year Staff generally remain at organization less than five years Staff members tend to remain at organization at least five years Is there a high turnover of staff? X Is there a high turnover of staff? Staff turnover among the permanent employees of CHAL is low but staff turnover within the CS Project had been high. CHAL has lost seven supervisory staff and three junior-level staff X Gender equity is rarely considered when recruiting, hiring, or advancing staff Gender equity is sometimes considered when making personnel decisions; policies are informally understood Organization purposely seeks to promote gender equity in personnel decisions; gender equity policies are written down and followed Gender equity is rarely considered when recruiting, hiring, or advancing staff Gender equity is sometimes considered when making personnel decisions; policies are informally understood Organization purposely seeks to promote gender equity in personnel decisions; gender equity policies are written down and followed Is gender equity an objective in recruitment, hiring, and advancement of staff? X Is gender equity an objective in recruitment, hiring, and advancement of staff? X Organization management deals with conflict only in extreme cases Conflict resolution guidelines are written down but not always used Conflict resolution guidelines are adhered to and used when appropriate Organization management deals with conflict only in extreme cases Conflict resolution guidelines are written down but not always used Conflict resolution guidelines are adhered to and used when appropriate Does the organization have the ability to resolve conflict among the staff? X Does the organization have the ability to resolve conflict among the staff? Although CHAL runs a reconciliation department, it is not however clear if conflict resolution guidelines are in place for staff. X Political, family, or personal ties are often basis for recruiting or firing staff Political, family, or personal ties occasionally unduly influence recruiting and firing decisions Organization intentionally avoids undue influence of political, family, or personal factors when making personnel decisions Political, family, or personal ties are often basis for recruiting or firing staff Political, family, or personal ties occasionally unduly influence recruiting and firing decisions Organization intentionally avoids undue influence of political, family, or personal factors when making personnel decisions Do political, family or personal ties unduly influence the process of recruiting or firing staff? X Do political, family or personal ties unduly influence the process of recruiting or firing staff? X 8 Organization shares vehicles and computers with other organization Organization owns vehicles and computers, but these assets are inadequate for program needs Organization’s vehicles and computers fulfill program needs Organization shares vehicles and computers with other organization Organization owns vehicles and computers, but these assets are inadequate for program needs Organization’s vehicles and computers fulfill Are physical assets program needs (vehicles, computers etc.) adequate for program needs? X Are physical assets (vehicles, computers etc.) adequate for program needs? X PROJECT DESIGN AND MANAGEMENT PROJECT DESIGN AND MANAGEMENT Programming elicits minimal community involvement Programming involves community participants in assessment and design but does not foster community ownership Programming fully engages community participants throughout the project cycle; programming promotes ownership and self-sufficiency among participants Programming elicits minimal community involvement Programming involves community participants in assessment and design but does not foster community ownership Programming fully engages community participants throughout the project cycle; programming promotes ownership and self-sufficiency among participants Does the organization support community￾based programming that promotes participation, ownership, and self￾sufficiency? X Does the organization support community￾based programming that promotes participation, ownership, and self￾sufficiency? X Proposals rarely written by organization Organization develops/writes project proposals, but proposals are not well developed and rarely result in funding Organization consistently writes strong project proposals that result in project funding Proposals rarely written by organization Organization develops/writes project proposals, but proposals are not well developed and rarely result in funding Organization consistently writes strong project proposals that result in project funding Has the organization demonstrated its ability to develop/write project proposals? X Has the organization demonstrated its ability to develop/write project proposals? X Project participants are perceived as beneficiaries and do not actively participate in planning or monitoring & evaluation Project participants participate in some planning, management, and evaluation activities Project participants are fully represented in decision￾making roles and all stages of project planning, monitoring & evaluation Project participants are perceived as beneficiaries and do not actively participate in planning or monitoring & evaluation Project participants participate in some planning, management, and evaluation activities Project participants are fully represented in decision￾making roles and all stages of project planning, monitoring & evaluation Do target groups participate in planning, monitoring, and evaluation of projects? X Do target groups participate in planning, monitoring, and evaluation of projects? X 9 Projects rarely adapt to changing situations and needs Projects are fairly flexible but slow to change when needed Projects easily adapt to changing situations and needs Projects rarely adapt to changing situations and needs Projects are fairly flexible but slow to change when needed Projects easily adapt to changing situations and needs Does the organization’s projects adapt to changing situations and needs? X Does the organization’s projects adapt to changing situations and needs? Adapting to change is very slow in CHAL X Goal, objectives, activities, and indicators are unclear or missing Goal, objectives, activities, and indicators are present but not always clear Goals, objectives, and indicators convey the project’s logical framework of goal, objectives, activities, and indicators Goal, objectives, activities, and indicators are unclear or missing Goal, objectives, activities, and indicators are present but not always clear Goals, objectives, and indicators convey the project’s logical framework of goal, objectives, activities, and indicators Do projects include clear goal, objectives, activities, and indicators? X Do projects include clear goal, objectives, activities, and indicators? X Gender-disaggregated data rarely collected Gender￾disaggregated data collected for selected projects Gender￾disaggregated data consistently collected for all projects Gender-disaggregated data rarely collected Gender￾disaggregated data collected for selected projects Gender￾disaggregated data consistently collected for all projects Is gender-disaggregated data collected for projects? X Is gender-disaggregated data collected for projects? CHAL reports that most donors request gender￾disaggregated data collection. X Project designs do not include gender analysis Gender analysis is included but gender￾responsive objectives and activities are not included in project design All project designs include gender analysis and clear gender equity objectives and activities Project designs do not include gender analysis Gender analysis is included but gender￾responsive objectives and activities are not included in project design All project designs include gender analysis and clear gender equity objectives and activities Are the organization’s projects gender￾responsive X Are the organization’s projects gender￾responsive X Monitoring systems are occasionally used Monitoring systems are defined in project designs but are not consistently used Monitoring systems are well designed and consistently used to measure progress against objectives and indicators Monitoring systems are occasionally used Monitoring systems are defined in project designs but are not consistently used Monitoring systems are well designed and consistently used to measure progress against objectives and indicators Do monitoring systems exist to track progress against objectives and indicators? X Do monitoring systems exist to track progress against objectives and indicators? X 10 Organization is aware of best practices and related tools but seldom uses them Organization uses best practices and related tools on some projects Organization regularly refers to best practices/latest methodologies and technical resources throughout the project cycle Organization is aware of best practices and related tools but seldom uses them Organization uses best practices and related tools on some projects Organization regularly refers to best practices/latest methodologies and technical resources throughout the project cycle Does the organization use best practices/latest methodologies and technical resources? X Does the organization use best practices/latest methodologies and technical resources? X Lessons learned from projects are infrequently gathered Project managers document lessons learned from projects, but findings are rarely used to improve other projects Lessons learned are gathered and documented for all projects; results are applied to related projects Lessons learned from projects are infrequently gathered Project managers document lessons learned from projects, but findings are rarely used to improve other projects Lessons learned are gathered and documented for all projects; results are applied to related projects Does the organization document lessons learned from projects and use these lessons to improve other projects? X Does the organization document lessons learned from projects and use these lessons to improve other projects? There is no evidence that CHAL systematically documents lessons learned in order to improve projects X Projects are evaluated at manager’s discretion Most projects are evaluated, but quality of evaluation is inconsistent All projects are thoroughly evaluated Projects are evaluated at manager’s discretion Most projects are evaluated, but quality of evaluation is inconsistent All projects are thoroughly evaluated Are the organization’s projects evaluated regularly and thoroughly? X Are the organization’s projects evaluated regularly and thoroughly? X 11 INFRASTRUCTURE & RELATIONSHIPS INFRASTRUCTURE & RELATIONSHIPS Staff are aware of resources and information / documents but seldom are able to access tools regularly Staff periodically access useful tools, but not consistently enough to stay abreast of current health intervention trends Staff access useful tools as needed and are well informed of current developments in applicable health interventions Staff are aware of resources and information / documents but seldom are able to access tools regularly Staff periodically access useful tools, but not consistently enough to stay abreast of current health intervention trends Staff access useful tools as needed and are well informed of current developments in applicable health interventions Do organization staff access electronic state of the art resources and project information/documents for technical health intervention support? X Do organization staff access electronic state of the art resources and project information/documents for technical health intervention support? X Organization is not registered Organization is not registered but is in process of becoming so Organization is registered Organization is not registered Organization is not registered but is in process of becoming so Organization is registered Is the organization registered as a non￾profit in the country of operation? X Is the organization registered as a non￾profit in the country of operation? X Organization is unable to clear supplies n/a Organization can clear supplies Organization is unable to clear supplies n/a Organization can clear supplies For organizations receiving GIK, does the organization have the ability to clear supplies? X For organizations receiving GIK, does the organization have the ability to clear supplies? X Organization has limited infrastructure in place to provide logistical support for teams Organization can usually provide adequate logistical support Organization consistently provides exemplary logistical support to teams Organization has limited infrastructure in place to provide logistical support for teams Organization can usually provide adequate logistical support Organization consistently provides exemplary logistical support to teams For organizations receiving teams, does the organization have the ability to clear team supplies and facilitate visas, travel, and arrange accommodation for teams? X For organizations receiving teams, does the organization have the ability to clear team supplies and facilitate visas, travel, and arrange accommodation for teams? X 12 Security guidelines are informally understood and not consistently applied Security guidelines are written down and applied to some projects/locations Security guidelines are adequate for security needs and are adhered to in all projects/locations Security guidelines are informally understood and not consistently applied Security guidelines are written down and applied to some projects/locations Security guidelines are adequate for security needs and are adhered to in all projects/locations Does the organization have adequate security guidelines in place? X Does the organization have adequate security guidelines in place? X Projects are not integrated/collaborative with local MoH programs Projects are loosely integrated with local MoH programs Projects are fully informed by MoH programs and visa versa in all phases of the project cycle Projects are not integrated/collaborative with local MoH programs Projects are loosely integrated with local MoH programs Projects are fully informed by MoH programs and visa versa in all phases of the project cycle Are the organization’s health projects integrated or collaborative with local Ministry of Health programs? X Are the organization’s health projects integrated or collaborative with local Ministry of Health programs? X Organization builds relationships only with parties closely related to its own mission Organization builds relationships with selected parties Organization builds effective partnerships with parties that work in the project area or sector Organization builds relationships only with parties closely related to its own mission Organization builds relationships with selected parties Organization builds effective partnerships with parties that work in the project area or sector Does the organization effectively build relationships with a variety of parties in the project area and sector? X Does the organization effectively build relationships with a variety of parties in the project area and sector? X Organization is little known by the government and in the community Organization is familiar to the government and the community, but its work is not well known Organization and its projects are held in high regard by the government and in the community Organization is little known by the government and in the community Organization is familiar to the government and the community, but its work is not well known Organization and its projects are held in high regard by the government and in the community Is the organization well respected by the government and in the community? X Is the organization well respected by the government and in the community? X 13 GCM CHT ORGANIZATIONAL CAPACITY SURVEY DEC 19, 2006 GCM CHT ORGANIZATIONAL CAPACITY SURVEY SEPT 2008 QUESTION CAPACITY CRITERIA QUESTION CAPACITY CRITERIA Limited Capacity Basic Capacity High Capacity Limited Capacity Basic Capacity High Capacity LEADERSHIP & AGENCY PLAN LEADERSHIP & AGENCY PLAN No written mission or governing principles (Not applicable) Written mission and governing principles No written mission or governing principles (Not applicable) Written mission and governing principles Does the organization have a written mission and/or governing principles? X- NOT AVAILABLE Does the organization have a written mission and/or governing principles? X- NOT AVAILABLE Staff are minimally aware of the organization’s mission Staff can generally describe the organization’s mission and/or values Staff are very familiar with the organization’s mission and/or values and can describe these in detail Staff are minimally aware of the organization’s mission Staff can generally describe the organization’s mission and/or values Staff are very familiar with the organization’s mission and/or values and can describe these in detail Is the organization’s mission/values known by all the members of the organization? X Is the organization’s mission/values known by all the members of the organization? X 14 Staff can describe general goals of the organization, but they have never been agreed upon or written down Goals and plan/strategy written down but are out of date or not referred to Goals and plan/strategy are written down and are familiar to all staff. Goals and plan/strategy are regularly referred to and guide the organization’s work Staff can describe general goals of the organization, but they have never been agreed upon or written down Goals and plan/strategy written down but are out of date or not referred to Goals and plan/strategy are written down and are familiar to all staff. Goals and plan/strategy are regularly referred to and guide the organization’s work Does the organization have goals and a plan or strategy to meet the goals? X Does the organization have goals and a plan or strategy to meet the goals? X Organization unsure of its strengths and niche; projects are chosen based on variable factors Organization seeks to focus on its strengths and niche but does not consistently adhere to its plans Organization consistently focuses on its strengths and niche and strategically chooses projects and interventions related to its long-term plans Organization unsure of its strengths and niche; projects are chosen based on variable factors Organization seeks to focus on its strengths and niche but does not consistently adhere to its plans Organization consistently focuses on its strengths and niche and strategically chooses projects and interventions related to its long-term plans Does the organization focus on its strengths and niche? X Does the organization focus on its strengths and niche? X 15 Committee established, but never meets Occasional meetings, but rarely agree on decisions Regular meetings, with useful guidance and decisions made for the organization Committee established, but never meets Occasional meetings, but rarely agree on decisions Regular meetings, with useful guidance and decisions made for the organization Does the organization have a committee or board that meets and makes decisions that guide the organization’s development and reviews fiduciary responsibility? X Does the organization have a committee or board that meets and makes decisions that guide the organization’s development and reviews fiduciary responsibility? X Board is established but does not represent the organization’s target group(s) Board includes minimal number of target group representatives Board includes variety of members who represent the diverse needs of the organization and of the target group Board is established but does not represent the organization’s target group(s) Board includes minimal number of target group representatives Board includes variety of members who represent the diverse needs of the organization and of the target group Is the organization’s board composed of members who represent the needs of the organization and the varied interests of the people the organization seeks to serve? X Is the organization’s board composed of members who represent the needs of the organization and the varied interests of the people the organization seeks to serve? X 16 Leadership style is not consistent with governing values Leaders of the organization make an effort to ensure the leadership style is appropriate Leadership style is highly consistent with governing values and are a model for all staff Leadership style is not consistent with governing values Leaders of the organization make an effort to ensure the leadership style is appropriate Leadership style is highly consistent with governing values and are a model Do directors and senior for all staff staff use a leadership style that is appropriate to the organization and its purpose and values? X Do directors and senior staff use a leadership style that is appropriate to the organization and its purpose and values? X FINANCIAL MANAGEMENT FINANCIAL MANAGEMENT No written financial policies and guidelines in place Some ideas on what financial policies and guidelines should be in place, but no system Financial policies and guidelines are written down and integrated into systems No written financial policies and guidelines in place Some ideas on what financial policies and guidelines should be in place, but no system Financial policies and guidelines are written down and integrated into Does the organization systems have written financial policies and guidelines? X Does the organization have written financial policies and guidelines? X Are budgets prepared, approved, and used for accounting control purposes? Budgets exist as general, broad line items and are not referred to for control purposes Detailed budgets are prepared and approved but are not used to guide and control financial processes Budgets are prepared with appropriate detail, approved for use, and consistently used for accounting control purposes Are budgets prepared, approved, and used for accounting control purposes? Budgets exist as general, broad line items and are not referred to for control purposes Detailed budgets are prepared and approved but are not used to guide and control financial processes Budgets are prepared with appropriate detail, approved for use, and consistently used for accounting control purposes X X 17 Does the organization keep accounts of money that can be presented upon request? Records kept of funds received and spent, but amount of money held at any one time is unknown Accounts kept up-to-date and balances and statements are prepared at year end Balances and statements are prepared quarterly and presented to external stakeholders at year end and can be retrieved as needed Does the organization keep accounts of money that can be presented upon request? Records kept of funds received and spent, but amount of money held at any one time is unknown Accounts kept up-to-date and balances and statements are prepared at year end Balances and statements are prepared quarterly and presented to external stakeholders at year end and can be retrieved as needed X X Does the organization have monthly bank reconciliations? Bank reconciliations completed sporadically Bank reconciliations completed at least every other month Bank reconciliations completed monthly Does the organization have monthly bank reconciliations? Bank reconciliations completed sporadically Bank reconciliations completed at least every other month Bank reconciliations completed monthly X X No cash-flow system in place Some ideas on how to deal with cash-flow difficulties, but no workable system Proven cash-flow system that works No cash-flow system in place Some ideas on how to deal with cash-flow difficulties, but no workable system Proven cash-flow system that works Does the organization have a system to deal with cash-flow difficulties? X Does the organization have a system to deal with cash-flow difficulties? X 18 Are adequate financial and inventory controls in place and implemented? Controls are not written; unable to assess whether they are implemented Control policies are written down but are irregularly implemented Control policies are written down and effectively implemented by all staff Are adequate financial and inventory controls in place and implemented? Controls are not written; unable to assess whether they are implemented Control policies are written down but are irregularly implemented Control policies are written down and effectively implemented by all staff X X Receipts/invoices rarely kept on file Receipts/invoices kept on file but rarely “reviewed” or used for financial reporting of grants All receipts/invoices and other supporting documents filed for three years, used for financial reporting, and regularly reviewed by authorized personnel Receipts/invoices rarely kept on file Receipts/invoices kept on file but rarely “reviewed” or used for financial reporting of grants All receipts/invoices and other supporting documents filed for three years, used for financial reporting, and regularly reviewed by authorized personnel Are supporting receipts and invoices maintained for every expenditure from grants? X Are supporting receipts and invoices maintained for every expenditure from grants? X Reports are usually late and with incomplete information Some reports submitted on time but are often incomplete or questionable Reports always submitted on time and always meet all donor requirements Reports are usually late and with incomplete information Some reports submitted on time but are often incomplete or questionable Reports always submitted on time and always meet all donor requirements Are accurate financial reports provided to donors in a timely manner? X – to MoH Are accurate financial reports provided to donors in a timely manner? X – to MoH 19 Large grants are held in organization’s general bank account Not applicable Organization holds separate accounts for large grants Large grants are held in organization’s general bank account Not applicable Organization holds separate accounts for large Does the organization grants hold separate accounts for large grants? X Does the organization hold separate accounts for large grants? X Does the organization provide annual reporting and submit to annual audits? Annual reporting rarely completed Annual reporting completed but not submitted to annual audits Annual reporting completed and submitted to annual audits Does the organization provide annual reporting and submit to annual audits? Annual reporting rarely completed Annual reporting completed but not submitted to annual audits Annual reporting completed and submitted to annual audits X X Fundraising activities performed as needs arise Fundraising strategy is developed but not used consistently enough to generate needed funds Fundraising strategy is well developed, includes a variety of funding sources, and effectively raises sufficient funds Fundraising activities performed as needs arise Fundraising strategy is developed but not used consistently enough to generate needed funds Fundraising strategy is well developed, includes a variety of funding sources, and effectively raises Does the organization sufficient funds have an effective fundraising strategy? X Does the organization have an effective fundraising strategy? X 20 ADMINISTRATION & HUMAN RESOURCES ADMINISTRATION & HUMAN RESOURCES Number of staff is not adequate for project needs Number of staff is adequate but are not fully trained for effective project implementation Organization has staff large enough to implement projects; staff possess requisite health program skills Number of staff is not adequate for project needs Number of staff is adequate but are not fully trained for effective project implementation Organization has staff large enough to implement projects; staff possess requisite health program skills Does the organization have adequate staff in terms of numbers and skills in health programs? X Does the organization have adequate staff in terms of numbers and skills in health programs? X Recruitment and personnel practices are unclear Recruitment and personnel practices are defined in written policies but are not consistently followed Recruitment and personnel practices are defined in written policies and are regularly followed Recruitment and personnel practices are unclear Recruitment and personnel practices are defined in written policies but are not consistently followed Recruitment and personnel practices are defined in written policies and are regularly followed Are recruitment and personnel practices clearly defined and followed? X – not available Are recruitment and personnel practices clearly defined and followed? X – not available Organization inconsistently follows local labor laws and does not regularly compare salary and benefits with local standards Local labor laws are usually adhered to, and salaries and benefits are sometimes in range with local standards Local labor laws are strictly followed and salary scales and benefits are reviewed and revised against local standards Organization inconsistently follows local labor laws and does not regularly compare salary and benefits with local standards Local labor laws are usually adhered to, and salaries and benefits are sometimes in range with local standards Local labor laws are strictly followed and salary scales and benefits are reviewed and revised against local standards Does the organization follow local labor laws and ensure that salary scales and benefits are commensurate with local standards? X Does the organization follow local labor laws and ensure that salary scales and benefits are commensurate with local standards? X 21 Performance appraisals conducted every few years; compensation is reviewed at employee’s request Performance appraisals are conducted nearly every year along with review of performance￾based compensation packages Performance appraisals are routinely conducted annually and performance￾based increases are given when possible and appropriate Performance appraisals conducted every few years; compensation is reviewed at employee’s request Performance appraisals are conducted nearly every year along with review of performance￾based compensation packages Performance appraisals are routinely conducted annually and performance￾based increases are given when possible and appropriate Does the organization ensure that staff performance appraisals are conducted and compensation is reviewed annually? X Does the organization ensure that staff performance appraisals are conducted and compensation is reviewed annually? X Staff training is rarely provided Staff training is occasionally provided at staff members’ request and initiative Staff training is planned and provided; topics are solicited via staff questionnaires and assessments Staff training is rarely provided Staff training is occasionally provided at staff members’ request and initiative Staff training is planned and provided; topics are solicited via staff questionnaires and assessments Does the organization provide regular staff training to improve staff members’ professional development? X Does the organization provide regular staff training to improve staff members’ professional development? X Is there a high turnover of staff? Staff members rarely remain at organization more than a year Staff generally remain at organization less than five years Staff members tend to remain at organization at least five years Is there a high turnover of staff? Staff members rarely remain at organization more than a year Staff generally remain at organization less than five years Staff members tend to remain at organization at least five years X X 22 Gender equity is rarely considered when recruiting, hiring, or advancing staff Gender equity is sometimes considered when making personnel decisions; policies are informally understood Organization purposely seeks to promote gender equity in personnel decisions; gender equity policies are written down and followed Gender equity is rarely considered when recruiting, hiring, or advancing staff Gender equity is sometimes considered when making personnel decisions; policies are informally understood Organization purposely seeks to promote gender equity in personnel decisions; gender equity policies are written down and followed Is gender equity an objective in recruitment, hiring, and advancement of staff? X Is gender equity an objective in recruitment, hiring, and advancement of staff? X Organization management deals with conflict only in extreme cases Conflict resolution guidelines are written down but not always used Conflict resolution guidelines are adhered to and used when appropriate Organization management deals with conflict only in extreme cases Conflict resolution guidelines are written down but not always used Conflict resolution guidelines are adhered to and used when appropriate Does the organization have the ability to resolve conflict among the staff? X – not available Does the organization have the ability to resolve conflict among the staff? X – not available Political, family, or personal ties are often basis for recruiting or firing staff Political, family, or personal ties occasionally unduly influence recruiting and firing decisions Organization intentionally avoids undue influence of political, family, or personal factors when making personnel decisions Political, family, or personal ties are often basis for recruiting or firing staff Political, family, or personal ties occasionally unduly influence recruiting and firing decisions Organization intentionally avoids undue influence of political, family, or personal factors when making personnel decisions Do political, family or personal ties unduly influence the process of recruiting or firing staff? X Do political, family or personal ties unduly influence the process of recruiting or firing staff? X 23 Organization shares vehicles and computers with other organization Organization owns vehicles and computers, but these assets are inadequate for program needs Organization’s vehicles and computers fulfill program needs Organization shares vehicles and computers with other organization Organization owns vehicles and computers, but these assets are inadequate for program needs Organization’s vehicles and computers fulfill program needs Are physical assets (vehicles, computers etc.) adequate for program needs? X Are physical assets (vehicles, computers etc.) adequate for program needs? X PROJECT DESIGN AND MANAGEMENT PROJECT DESIGN AND MANAGEMENT Programming elicits minimal community involvement Programming involves community participants in assessment and design but does not foster community ownership Programming fully engages community participants throughout the project cycle; programming promotes ownership and self-sufficiency among participants Programming elicits minimal community involvement Programming involves community participants in assessment and design but does not foster community ownership Programming fully engages community participants throughout the project cycle; programming promotes ownership and self-sufficiency among participants Does the organization support community￾based programming that promotes participation, ownership, and self￾sufficiency? X Does the organization support community￾based programming that promotes participation, ownership, and self￾sufficiency? X Proposals rarely written by organization Organization develops/writes project proposals, but proposals are not well developed and rarely result in funding Organization consistently writes strong project proposals that result in project funding Proposals rarely written by organization Organization develops/writes project proposals, but proposals are not well developed and rarely result in funding Organization consistently writes strong project proposals that result in Has the organization project funding demonstrated its ability to develop/write project proposals? X Has the organization demonstrated its ability to develop/write project proposals? X 24 Project participants are perceived as beneficiaries and do not actively participate in planning or monitoring & evaluation Project participants participate in some planning, management, and evaluation activities Project participants are fully represented in decision￾making roles and all stages of project planning, monitoring & evaluation Project participants are perceived as beneficiaries and do not actively participate in planning or monitoring & evaluation Project participants participate in some planning, management, and evaluation activities Project participants are fully represented in decision￾making roles and all stages of project planning, monitoring & evaluation Do target groups participate in planning, monitoring, and evaluation of projects? X Do target groups participate in planning, monitoring, and evaluation of projects? X Projects rarely adapt to changing situations and needs Projects are fairly flexible but slow to change when needed Projects easily adapt to changing situations and needs Projects rarely adapt to changing situations and needs Projects are fairly flexible but slow to change when needed Projects easily adapt to changing situations and needs Does the organization’s projects adapt to changing situations and needs? X Does the organization’s projects adapt to changing situations and needs? X Goal, objectives, activities, and indicators are unclear or missing Goal, objectives, activities, and indicators are present but not always clear Goals, objectives, and indicators convey the project’s logical framework of goal, objectives, activities, and indicators Goal, objectives, activities, and indicators are unclear or missing Goal, objectives, activities, and indicators are present but not always clear Goals, objectives, and indicators convey the project’s logical framework of goal, objectives, activities, and indicators Do projects include clear goal, objectives, activities, and indicators? X Do projects include clear goal, objectives, activities, and indicators? X 25 Gender-disaggregated data rarely collected Gender￾disaggregated data collected for selected projects Gender￾disaggregated data consistently collected for all projects Gender-disaggregated data rarely collected Gender￾disaggregated data collected for selected projects Gender￾disaggregated data consistently collected for all projects Is gender-disaggregated data collected for projects? X Is gender-disaggregated data collected for projects? X Project designs do not include gender analysis Gender analysis is included but gender￾responsive objectives and activities are not included in project design All project designs include gender analysis and clear gender equity objectives and activities Project designs do not include gender analysis Gender analysis is included but gender￾responsive objectives and activities are not included in project design All project designs include gender analysis and clear gender equity objectives and activities Are the organization’s projects gender￾responsive X Are the organization’s projects gender￾responsive X Monitoring systems are occasionally used Monitoring systems are defined in project designs but are not consistently used Monitoring systems are well designed and consistently used to measure progress against objectives and indicators Monitoring systems are occasionally used Monitoring systems are defined in project designs but are not consistently used Monitoring systems are well designed and consistently used to measure progress against objectives and indicators Do monitoring systems exist to track progress against objectives and indicators? X Do monitoring systems exist to track progress against objectives and indicators? X 26 Organization is aware of best practices and related tools but seldom uses them Organization uses best practices and related tools on some projects Organization regularly refers to best practices/latest methodologies and technical resources throughout the project cycle Organization is aware of best practices and related tools but seldom uses them Organization uses best practices and related tools on some projects Organization regularly refers to best practices/latest methodologies and technical resources throughout the project cycle Does the organization use best practices/latest methodologies and technical resources? X Does the organization use best practices/latest methodologies and technical resources? X Lessons learned from projects are infrequently gathered Project managers document lessons learned from projects, but findings are rarely used to improve other projects Lessons learned are gathered and documented for all projects; results are applied to related projects Lessons learned from projects are infrequently gathered Project managers document lessons learned from projects, but findings are rarely used to improve other projects Lessons learned are gathered and documented for all projects; results are applied to related projects Does the organization document lessons learned from projects and use these lessons to improve other projects? X Does the organization document lessons learned from projects and use these lessons to improve other projects? X Projects are evaluated at manager’s discretion Most projects are evaluated, but quality of evaluation is inconsistent All projects are thoroughly evaluated Projects are evaluated at manager’s discretion Most projects are evaluated, but quality of evaluation is inconsistent All projects are thoroughly evaluated Are the organization’s projects evaluated regularly and thoroughly? X Are the organization’s projects evaluated regularly and thoroughly? X 27 INFRASTRUCTURE & RELATIONSHIPS INFRASTRUCTURE & RELATIONSHIPS Staff are aware of resources and information / documents but seldom are able to access tools regularly Staff periodically access useful tools, but not consistently enough to stay abreast of current health intervention trends Staff access useful tools as needed and are well informed of current developments in applicable health interventions Staff are aware of resources and information / documents but seldom are able to access tools regularly Staff periodically access useful tools, but not consistently enough to stay abreast of current health intervention trends Staff access useful tools as needed and are well informed of current developments in applicable health interventions Do organization staff access electronic state of the art resources and project information/documents for technical health intervention support? X- MoH Do organization staff access electronic state of the art resources and project information/documents for technical health intervention support? X- MoH Organization is unable to clear supplies n/a Organization can clear supplies Organization is unable to clear supplies n/a Organization can clear supplies For organizations receiving GIK, does the organization have the ability to clear supplies? X For organizations receiving GIK, does the organization have the ability to clear supplies? X Organization has limited infrastructure in place to provide logistical support Organization can usually provide adequate logistical support Organization consistently provides exemplary logistical support Organization has limited infrastructure in place to provide logistical support Organization can usually provide adequate logistical support Organization consistently provides exemplary logistical support For organizations receiving logistical supplies from partners and MoH, does the organization have the ability to clear team supplies? X For organizations receiving logistical supplies from partners and MoH, does the organization have the ability to clear team supplies? X 28 Security guidelines are informally understood and not consistently applied Security guidelines are written down and applied to some projects/locations Security guidelines are adequate for security needs and are adhered to in all projects/locations Security guidelines are informally understood and not consistently applied Security guidelines are written down and applied to some projects/locations Security guidelines are adequate for security needs and are adhered to in all Does the organization projects/locations have adequate security guidelines in place? X – not available Does the organization have adequate security guidelines in place? X – not available Projects are not integrated/collaborative with local MoH programs Projects are loosely integrated with local MoH programs Projects are fully informed by MoH programs and visa versa in all phases of the project cycle Projects are not integrated/collaborative with local MoH programs Projects are loosely integrated with local MoH programs Projects are fully informed by MoH programs and visa versa in all phases of the project cycle Are the organization’s health projects integrated or collaborative with local Ministry of Health programs? X Are the organization’s health projects integrated or collaborative with local Ministry of Health programs? X Organization builds relationships only with parties closely related to its own mission Organization builds relationships with selected parties Organization builds effective partnerships with parties that work in the project area or sector Organization builds relationships only with parties closely related to its own mission Organization builds relationships with selected parties Organization builds effective partnerships with parties that work in the project area or sector Does the organization effectively build relationships with a variety of parties in the project area and sector? X Does the organization effectively build relationships with a variety of parties in the project area and sector? X 29 Organization is little known by the government and in the community Organization is familiar to the government and the community, but its work is not well known Organization and its projects are held in high regard by the government and in the community Organization is little known by the government and in the community Organization is familiar to the government and the community, but its work is not well known Organization and its projects are held in high regard by the government and in the community Is the organization well respected by the government, central MoH and in the community? x Is the organization well respected by the government, central MoH and in the community? x Annex 8 Medical Teams International in Liberia, CSP22 Capacity Building Plan MTI, CHAL and GCM CHT Technical Assistance Plans for Institutional Capacity Building Date: September 30, 2008 Capacity Building needs for all three agencies Category Priority Activities Indicators Who is Responsible When will it be Completed Comment Leadership & Agency Plan Conduct monthly CSP program coordination meetings with partners and MTI # times CSP committee met CHO, CS Project Manager, Community Outreach Coordinator, MTI Country Director, September 2010 Meetings were established in Y1; at end of Y2 meetings were sporadic; Plan to reinforce regular meetings Yr 3 & 4 Financial Management Continue annual financial management reviews and provide training in areas of identified weakness # of reviews # of training MTI Finance & Administrative Manager CHAL Accountant CHT Administrator July 2007 Complete Administration & Human Resources Develop training/professional development plans for staff related to CSP # staff trained in CS related activities MTI Country Director CHAL Executive Director CHO September 2007 Performance appraisals were conducted but did not include establishing professional development plans for each staff person Review and/or develop procedures and policies on Logistics and medicines and supply management systems and train staff in the correct use of these systems # relevant staff familiar with policies and procedures related to logistics and medicines and supply management systems MTI F& A Manager MTI, CHAL & CHT Logistician, HQ Commodity support officer December 2007 Systems established; plan to strengthen logistics and provide additional training in supply mgmt in yr 3 2 Category Priority Activities Indicators Who is Responsible When will it be Completed Comment Project Design & Management Conduct training in Program Cycle Management # of CSP program and partner staff trained in PCM HQ Technical Services HQ Africa Region July 2007 Complete Develop Conflict Resolution guidelines and train staff in conflict resolution methods and local capacities for peace. # CSP program and partner staff who received training in conflict resolution strategies CHAL Peace Building Team June 2009 N/A Planned for year 3 Conduct training in BEHAVE Framework, IEC/BCC strategies and methods # of CSP program and partner staff trained in BEHAVE, IEC/BCC CS Advisor, BCC Specialist, CSP Training Coordinator December 2008 Complete Conduct training in IMCI # CSP program and partner staff trained in IMCI CSP Manager Training and IMCI Coordinators July 2008 Complete Yr 1&2, third batch training in Yr 3 Conduct training in adult learning teaching strategies and methods, and in development of curricula and teaching materials # CSP program and partner staff demonstrate competency in informal teaching methods HQ Capacity Building/Training Advisor May 2007 Complete Conduct training in Qualitative Care Methods # CSP program and partner staff who received training in COPE CS Advisor CSP Training Coordinator December 2009 Removed Conduct Training on Community Transformation and mobilization # CSP program and partner staff who received training in Community Transformation and mobilization Community Transformation Consultant November 2006 Complete Conduct Training on KPC, HFA and Qualitative Research Methods # CSP program and partner staff who received training KPC, HFA and Qualitative Research Methods M&E Specialist CS Consultant Community Consultant November 2006 Complete Conduct training on community participatory processes in planning and development # CSP program and partner staff trained in participatory processes and community development Development Education Network￾Liberia (DEN-L, local NGO) Nov 2007 Complete Training in use of monitoring and evaluation plans to report on progress against objectives and indicators # CSP program and partner staff competent in M&E and reporting procedures MTI M&E Specialist CSP M&E Coordinator July 2007 Complete Conduct Training in Health Management Information Management System # CSP program and partner staff who received training in Health Information Management Systems CS Specialist CS Project Manager MOH Disease Surveillance Division March 2008 Complete Develop a feasible health management information system Existence of an implemented HMIS that is used for decision making CHO CHAL Community Outreach Coordinator, May 2007 The HMIS is established. HHPs have been collecting health 3 CS Country Director information at the community level since July of 2008. CARE Group data will be shared with communities in October 2008. Develop CSP exit strategy Existence of a sustainable CSP exit strategy CS Project Manager CHAL Executive Director GCM CHO January 2010 N/A Planned for year 4 Infrastructure & Relationships Train staff use of electronic state of the art resources and project information/documents for technical health intervention support # staff who demonstrate competent use of electronic and technical resources MTI Country Director CSP Training Coordinator April 2010 N/A Planned for year 4 Annex 10 Medical Teams International in Liberia, CSP22 Midterm Evaluation Assessment Methodology A) Preparation o LQAS and R-HFSA conducted one month prior o Criteria for selection of health facilities and communities for qualitative MTE field visits: o 6 communities/health facilities to be visited by one of 2 teams during 3 days of fieldwork o At least 1 community selected from each of the 4 districts (one change was made as bridge was inaccessible due to previous heavy rains resulted in no visits to Porkpa District) o Variation in distance from project office, with one day distant (from 2 to 2-1/2 hours drive), one day average distance (from 1 to 1-1/2 hours drive) and one day close (from 30 to 45 minutes drive) o At least one CHDC to be visited by each team; CHCs in other sites o Invitation to partners for participation o Consultant/HQ CS Advisor and HQ Capacity-Building Advisor draft fieldwork guides o HQ CS Advisor finalizes fieldwork guides with project staff and partners B) Triangulation of results o Quantitative LQAS of mothers of children age 0-23 months o Quantitative Rapid-Health Facility Services Assessment o Qualitative field visits to health facilities and communities for interviews and focus groups C) Participatory o Participation in two teams by: o External consultant (1) o MTI HQ CS Advisor (1) o MTI Liberia executive and project staff (6) o CHAL executive and project staff (2) o Grand Cape Mount County Health Team (3) o One day orientation and training for team members D) Triangulation of perspectives from participatory fieldwork o Each of two teams includes external, MTI Liberia, CHAL and County Health Team participants o Within each team, participants rotate conducting interviews/focus groups and note taking; participants rotate each day between participation in: • Interviews with Household Health Promoters • Interviews with IMCI-trained Health Facility staff (3) • Focus groups with mothers (5) and fathers (1) • Focus group with Community Health Committees (4) • Focus group with Community Health Development Committee (2) 2 E) Participatory analysis of results and key findings o Participation in analysis by all members participating in fieldwork o Review of key LQAS and R-HFSA findings o External consultant leads participatory analysis of findings from Household Health Promoter focus group interviews; process repeated in small groups for analysis of fieldwork results from focus groups and interviews with mothers, fathers, and health facilities; large group analysis of findings from CHCs and Community Health and Development Committees o Findings structured according to the following headings: o Training process o Technical capacity o Activities o Coordination o Sustainability F) Finalization of key findings, recommendations and action planning o Debriefing to project partners on key findings and recommendations (one day) o Action planning by project and partner staff in response to recommendations (1-1/2 days) o Debriefing to USAID Liberia. 1 Annex 12: Child Survival and Health Grants Program Project Summary Medical Teams International (Liberia) General Project Information: Cooperative Agreement Number: GHS-A-00-06-00019 Project Grant Cycle: 22 Project Dates: (10/1/2006 - 9/30/2010) Project Type: Entry/New Partner MTI Headquarters Technical Backstop: Mary Helen Carruth Field Program Manager: George Kaine (Acting) Midterm Evaluator: Joan Jennings Final Evaluator: USAID Mission Contact: Christopher McDermott Field Program Manager Information: Name: George Kaine Address: Medical Teams International Liberia Airfield Short‐cut, Sinkor Monrovia, Liberia Phone: 011-2316-532075 Alternate Field Contact: Name: Bill Massaquoi Address: Medical Teams International Liberia Airfield Short‐cut, Sinkor Monrovia, Liberia Phone: 011-2316-532075 E-mail: bmassaquoi@mti-liberia.org Funding Information: USAID Funding:(US $): $1,249,881 PVO match:(US $) $1,441,509 Project Information: Description: Project Goal: To reduce morbidity and mortality of children under five and improve the health of women of reproductive age within the Grand Cape Mount County of Liberia Interventions:  Nutrition  Pneumonia case management  Control of diarrheal disease  Malaria  Immunizations 2 Strategies: 1. Targeted behavior change at the household level 2. Community mobilization through capacity building of community organizations and leaders including planning and evaluation, development of emergency transport systems and activities for sustainable health outcomes. 3. Quality of care and access at the clinic level will be improved by implementing IMCI through training, mentoring, supportive supervision, and the use of a quality assurance methodology (COPE), provision of drugs and supplies, and strengthening of referral and logistics systems. 4. Institutional capacity building for Medical Teams International and partners focusing on improved program quality and technical skills, strengthened project monitoring and evaluation, and institutionalization and dissemination of lessons learned. Location: Five districts in the Grand Cape Mount County in Liberia Project Partners Partner Name Partner Type USAID $ Allocated Christian Health Association of Liberia Sub-grantee $501,401.00 Grand Cape County Health Team Collaborating Partner Project Communities Collaborating Partner Sub-grant Total $501,401.00 General Strategies Planned: Advocacy on Health Policy Strengthen Decentralized Health System M&E Assessment Strategies: KPC Survey Health Facility Assessment Organizational Capacity Assessment with Local Partners Organizational Capacity Assessment for your own PVO Participatory Rapid Appraisal Lot Quality Assurance Sampling Participatory Evaluation Techniques (for mid-term or final evaluation) Behavior Change & Communication (BCC) Strategies: Mass Media Interpersonal Communication Peer Communication Support Groups 3 Groups targeted for Capacity Building: Capacity Building Targets Planned: PVO Non-Govt Partners Other Private Sector Govt Community US HQ (CS unit) Field Office HQ CS Project Team Local NGO (None Selected) National MOH Dist. Health System Health Facility Staff CHWs Interventions/Program Components: Immunizations (10 %) (IMCI Integration) (CHW Training) (HF Training)  Classic 6 Vaccines  Vitamin A  Injection Safety  Mobilization Nutrition (30 %) (IMCI Integration) (CHW Training) (HF Training)  ENA  Comp. Feed. from 6 mos.  Hearth  Cont. BF up to 24 mos.  Growth Monitoring  Maternal Nutrition Pneumonia Case Management (20 %) (IMCI Integration) (CHW Training) (HF Training)  Pneum. Case Mngmnt.  Case Mngmnt. Counseling  Recognition of Pneumonia Danger Signs  Zinc Control of Diarrheal Diseases (20 %) (IMCI Integration) (CHW Training) (HF Training)  Hand Washing  ORS/Home Fluids  Feeding/Breastfeeding  Care Seeking  Case Mngmnt./Counseling  Zinc 4 Malaria (20 %) (IMCI Integration) (CHW Training) (HF Training)  Training in Malaria CM  Adequate Supply of Malarial Drug  Access to providers and drugs  ITN (Bednets)  Care Seeking, Recog., Compliance  IPT  ACT Target Beneficiaries: Infants < 12 months: 5,085 Children 12-23 months: 4,576 Children 0-23 months: 9,661 Children 24-59 months: 11,950 Women 15-49 years: 21,611 Children 0-59 Months Women 15-49 years: 29,239 Population of Target Area: 127,124 Rapid Catch Indicators: CATCH Indicator DIP MTE Spacing, at least 24 months apart (Ch 0-23) 76.0 NA* Mothers receiving 2 TT Injections 61.3 66.7 Birth attended by skilled health person (Ch 0-23) 21.3 23.9 Postnatal visit for newborn within 3 days 7.0 36.5 Exclusive Breastfeeding, last 24 hours (Inf 0-5) 86.0 NA* IYCF 18.9 NA* Vitamin a Dose (Ch 6-23 months) 76.2 86.4 Measles Vaccination (Ch 12-23 months) 25.2** 50.00** DTP1 Vaccination (Ch 12-23 months) 46.8 86 DTP3 Vaccination (Ch 12-23 months) 30.6 50 Malaria Fever 3.3 9.4 ORT Use 74.2 79.1 Pneumonia Care Seeking 43.2 83.3 Point of Use 21.7 7.3 Approp Hand Washing 19.0 43.7 Slept under bednet (Ch 0-23) 17.7 67.7 Weight for Age (Ch 0-23 months) 27.1 20.8 *Birth spacing, IYCF, and Exclusive Breastfeeding indicators were not collected as this is a midterm LQAS for monitoring of project matrix indicators. These indicators will be collected during the final KPC Survey. **The measles indicator is for children aged 12-23 months who were vaccinated by 12 months of age, card verified. The baseline figure from the DIP has been changed to also be by 12 months of age, card verified.